Create a base container for a service running md model inference on requested exams file
The service should:

• Publish an API allowing new exams to be checked
• Has access to exam storage
• Accept new exams

Brainstorm about:

• Generator architecture
• Base generation algorithm
• Initial implementation for further incremental development

• #89
• #87
• #99

Inside the research module is the mark_generator, where the model is created. So get familiar with the 00_training.ipynb notebook and prepare the learning pipeline of the existing training path.

The README for research module can be created, and we can put the above diagram there.

Main idea: Introduction of WebDAV technology to be able to automatically upload scanned sheets from the printer to the cloud.

• #95
• Preparation of PoC application

This task involves improving the performance of our existing model, which may be struggling due to data quality issues. We need to update the mark_generator.py in the mark_generator module and implement a new data generation template. Additionally, we will integrate a new image into the data generation process to enhance the dataset and potentially boost model performance.

• Setting training pipeline
• Generated data augmentation
• Preparing the generator for creating training data with metadata
• Train model

As we are starting to develop main services which for sure will be placed in the end system we need to agree on the overall system architecture.
Propose the architecture containing:

• System services
  • Name
  • Short description
  • Planned software stack
• Visibility of services (if visible from the external network)
• Communication flow between them
• Use-Case diagrams for most important actions.

DoD:

• architecture in form of diagram and documentation proposed
• architecture approved by all team members

As in the title

Using black control rectangle on answer sheet detect page position and correct it if its rotated

Provide SVG graphic with exam template similar to one provided by supervisor.
Goals:

• Keep field structure, order and position
• Avoid "soft" edges for bounding boxes and exam checkboxes
• Be readable and clear for students
• Contain "removable" letters (constant grayscale)

Checked exam should has option to export it to csv.
One row should correspond to one student.
The format should be allow importing csv into eNauczanie platform

• Checking the required documents from the research project
• #106

@Krystian030 @S-cichocka

Latex Generator: https://github.com/cvlab-ai/splinter/tree/qr/qr
After consultation, this task is not required to write new exam generator. If you don't need it, you can close the issue as won't do.

Imported exams should be stored in a dedicated docker container which would be accessible only inside the internal docker network.

• Each user should be able to create an exam. An exam is a container that groups answer sheets from one exam term.
• Each owner of an exam should be able to edit or delete it.

• Prepare field extraction from the new template
• Classify the extracted fields
• Prepare fields for inference

• #97
• #90
• #91 (OPTIONAL)

@Krystian030 @PatrykGolembiewski @S-cichocka

Text rotation: https://developpaper.com/python-opencv-implementation-of-rotating-text-correction/
Text recognition: https://pyimagesearch.com/2018/08/20/opencv-text-detection-east-text-detector/

The task is complex. Should be done after #97
The aim is to further analyse of the project and prepare documentation for the API and existing methods.

PDF can contain pages which in fact are not student's answer sheets. We should recognize and delete such pages.

Prepare training dataset as randomly marked boxes with labels.

This issue is created from #32 (review)
The objective is to track all mocks and not implemented features in web application.
This issue should be split into separate features issues

Proposed exam_storage structure

└── /splinter/<exam_id>  # exam_path
       └── metadata.json - pdfs done
       └── pdfs
       └── students
       └── <index_nr>
 	      └── answers.json
 	      └── answers.jpg
 	      └── recognized_fields.jpg*
 └── results.csv
 └── answers_key_<group*>.jpg
 └── answers_key_<group*>.json

└── /uploads

Answers generated for students to fill exams must be parsed to a format useful for model training.

The issue is collecting data to detect numbers.

Hypotheses:

• How often do they fill in under the US system versus the European system?
• How often are mistakes made when transferring answers to the exam?

Future works:

• #98

• User registration
  • does not include sending verification mails
  • System should permit registration using eti.pg.edu.pl or pg.edu.pl domains
• User can log in

Create an Flask app responsible for handling model inference. The application API will include the following endpoints:

• check-exams (Create a files with detected answers, based on the provided path to the exams)
• check-exam (Create a file with detected answers, based on the provided path to the exams and file name)
• generate-exam-key (Create a file with detected answers as a key to exam, based on provided path to exams and answers template file)

The task includes generating the empty session template.

Create two exam directories:

• /splinter - for storing processed exams
• /uploads - for uploading pdfs from scanner using webdav - the access to it should be restricted by credentials

As for now, sample passwords are stored in configuration files which is not a secure approach. Before the system is prepared to release, the password handling must be reorganized in some more secure way.

To increase the accuracy of our model to the expected threshold (99.9%) we need to design a training pipeline:

• Configure data generation using our synthetic data generator
• Use real data as a validation resource

The task involves rewriting the exam generator from Latex to Markdown

This QR code generator is in Latex which causes it to take a long time to execute and the proposal is to make it in Markdown by adding your own code to the ready-made library which would automatically generate the fields for index, QR code and randomize the order of questions and answers.

• #84
• #85
• #86

• Generate images and required annotations (answers)
• Create google drive
• Write annotation instructions
• Publish and collect the data
• Deadline: 18.04.2022

Files uploaded from scanner using webdav protocol will be located in /uploads/ directory on exam_storage service.
When files from /uploads are assigned to exam it should be moved from /uploads to correct exam path.

• Read PDFs
• Detect UUID
• Recognize UUID
• Match partial UUID to full UUID
• Works with A5 format
• Save splitted exams to different files
• Prepare dataset
• Match missing page ID
• Sort exams if not in right order
• Fix issues with different detection due to different scale of image crop