Expected Behavior

Scan Categories should specify what could be labelled on the Scans by providing a list of predefined Labels Categories/Types.

Example: Kidneys category could specify that users can label two categories/types: left kidney, right kidney.

Actual Behavior

Labels aren't identified anyhow. They are just a list of selections.

Additional comment

It may be needed to work on this issue first: #65.

Expected Behavior

MedTagger should remember that user skipped given Scan. It tells us that there was nothing to label on the Scan, which is very important information!

Actual Behavior

User can skip Scans and MedTagger won't know whether given Scan wasn't shown to any user or users skipped them due to lack of potential issues.

Additional comment

It may be needed to work on this issue first: #65.

Expected Behavior

Labeling page should show loading screen or some progress bar for loading new scan as this may take a while.

Actual Behavior

Labeling page shows empty place for loading Scan. User is confused about what is hapenning on the page and may leave the site.

Steps to Reproduce the Problem

1. Go to the Labeling page.

Expected Behavior

Videos on tutorial page should be using English labels/texts.

Actual Behavior

MedTagger's frontend's first implementation was purely in Polish. It resulted in many labels/buttons/error messages that are still using Polish language. Videos on tutorial page were recorded on the old version of MedTagger which was using Polish.

Steps to Reproduce the Problem

1. Go to tutorial page.

Expected Behavior

User would like to add additional comment to Labels (eg. "Unusual", "Too big", "Left kidney"). Form for additional comment could be shown as a dialog box/pop-up after clicking on "Send label" button.

Actual Behavior

Labels are just selections without any additional knowledge.

Additional comment

It may be needed to work on this issue first: #65.

Expected Behavior

User can change its personal information at any time using Settings page.

User should only be able to change their first and last name. The E-mail field should be grayed out.

Actual Behavior

Form fields are filled with user data but there is no way to change them. User cannot modify these fields and there is no "Save" button available.

Steps to Reproduce the Problem

1. Go to Settings page.
2. Open "User data" section.
3. Done.

Additional comment

Consider changing the name of the section from "User data" to something else.

Expected Behavior

Labeling/validation page should show a message "No Scans available!" if there is no Scans available.

Actual Behavior

Labeling/validation page doesn't show any error or message.

Steps to Reproduce the Problem

1. Setup clean instance of MedTagger.
2. Go to the labeling/validation page.
3. Done.

Expected Behavior

Functional tests should use Alembic migrations and Python scripts to prepare Postgres.

Actual Behavior

Functional tests uses create_all() method which skips Alembic migrations. This may result in being inconsistent with production.

Additional comment

Functional/integration tests should be as much similar to production environment as possible!

Expected Behavior

There should be a page with information about the project.

Actual Behavior

The only documentation is stored in /docs directory.

Expected Behavior

MedTagger should be using better URLs for frontend pages. Example:

• /marker-tutorial -> /labeling/tutorial,
• /category-> /labeling/choose-category,
• /marker-> /labeling,
• etc.

Actual Behavior

MedTagger uses bad names. Examples above.

Expected Behavior

If user provides wrong password there should an error telling him why his/her account cannot be created.

At this moment the only constraint for password its it length: it has to be at least 8 characters. (https://github.com/jpowie01/MedTagger/blob/master/backend/medtagger/api/auth/serializers.py)

Actual Behavior

If user provides wrong password there is no error and nothing happends after clicking on 'Register' button. To know that something is wrong user would have to go to browser dev tools.

Steps to Reproduce the Problem

1. Fill in 'Register' card. Provide password that is too short ('abc')
2. Click on 'Register' (Nothing will happen).

Expected Behavior

Functional tests should clean the tables in Postgres and HBase.

Actual Behavior

Functional tests drops all the available tables and create them again. This may be (and probably is) time consuming.

Additional comment

Please compare both implementations before merge.

Expected Behavior

Using sending multiple scans mode, Slices should be sent one by one over WebSocket channel.

Actual Behavior

Right now, Slices are sent concurrently (each Scan individually in its own thread due to promises mechanism). It also results in broken progress bar which shows progress like below (uploading 10 scans at once):

Sent 10/500 slices.
Sent 20/500 slices.
Sent 30/500 slices.
...

Steps to Reproduce the Problem

1. Go to upload page.
2. Select multiple scans mode.
3. Select scans.
4. Click "Upload!".
5. Done. Look on the progress bar status.

Expected Behavior

Label should have new field that will tell us how much time given person spent on labeling.

Actual Behavior

We don't know anything about time taken for labeling.

Expected Behavior

User should be aware that registration process is in progress by the spinner indicator.

Actual Behavior

After clicking "Register" button, user has no feedback about the progress.

Steps to Reproduce the Problem

1. Go to the login page.
2. Switch to registration mode.
3. Fill the form and click "Register".
4. Done.

Expected Behavior

User should be able to upload new scans using 'Single 3D scan' option.

Actual Behavior

After choosing dicoms, the scans are not showing and the 'Upload button' is disabled. When checking logs we can see message: "$event.srcElement is undefined"

Steps to Reproduce the Problem

1. Go to upload page,
2. Choose 'Single 3D scan' option,
3. Choose at least one dicom file,
4. Nothing will happen...

Expected Behavior

All development environments should work (as much as possible) the same. That's why Vagrant should also be able to work on localhost.

Actual Behavior

Now, Vagrant virtual machine hosts itself on 10.0.0.99.

Steps to Reproduce the Problem

1. Run vagrant up.
2. Done.

Expected Behavior

MedTagger should load more Slices from Z orientation at once.

Actual Behavior

MedTagger fetches only 10 Slices at once. It could be increased to at least 25 or even more. Loading 10 Slices takes up to ~0.5s, so it's acceptable to load more that 10 Slices at once.

Expected Behavior

Python code in backend should be validated by "radon" tool in CI.

Actual Behavior

MedTagger backend uses a few linters already but we should add more validators to increate automation and code quality.

Expected Behavior

There should be a script that will allow administrators to back up entire system's data.

Actual Behavior

There is no scripts or any mechanisms that allows to back up stored data. In case of issues, system will lost its data.

Additional comment

Backup scripts should be able to store data on disk and restore them!

Expected Behavior

Backend Workers should log in the same way as API.

Actual Behavior

Worker logs are not saved on the disk.

Additional comment

Look for logging setup in API part.

Expected Behavior

MedTagger should support compressed DICOMs without any workaround.

Actual Behavior

GDCM does not support Python 3 properly. That's why we cannot pass binary data for parsing and cannot use its method for uncompressing data.

Additional comment

Let's create issues in both GDCM library and PyDICOM package to fix it.

Expected Behavior

MedTagger database should remember Scan's and Label's owner.

Actual Behavior

Scans and Labels are anonymous.

Expected Behavior

Docker containers with databases should save its state on disk (attached volume) in order not to lose any data.

Actual Behavior

HBase can crash and destroy its container. It's highly possible that it will corrupt collected data.

Steps to Reproduce the Problem

1. Setup Docker-Compose with docker-compose up.
2. Try to crash a DB container.

Expected Behavior

Roles should be represented as enum instead of string. This enum of roles should be also stored in database. The roles should be : admin, doctor, volunteer.

Actual Behavior

Roles are represented as string.

Expected Behavior

User can draw in whatever way he/she wants.

Actual Behavior

Drawing from right to left causes errors...

Call to API:

{
  "selections": [
    {
      "slice_index": 177,
      "x": 0.6191666666666666,
      "y": 0.71,
      "width": -0.12166666666666667,
      "height": -0.06166666666666667
    },
    {
      "slice_index": 178,
      "x": 0.4275,
      "y": 0.6116666666666667,
      "width": -0.10833333333333334,
      "height": -0.06333333333333334
    }
  ]
}

Response (status: 400):

{
    "errors": {
        "selections.0.width": "-0.12166666666666667 is less than the minimum of 0.0",
        "selections.0.height": "-0.06166666666666667 is less than the minimum of 0.0",
        "selections.1.width": "-0.10833333333333334 is less than the minimum of 0.0",
        "selections.1.height": "-0.06333333333333334 is less than the minimum of 0.0"
    },
    "message": "Input payload validation failed"
}

Steps to Reproduce the Problem

1. Go to the Labeling page.
2. Draw from right to left a few times.
3. Click "Send!" and check Dev Tools.
4. Done.

Expected Behavior

Users info should be displayed in that way to not go under the avatar. Additionally it should not go beyond the screen. If user first and/or last name is too long, only some of it should be displayed.

Actual Behavior

If users provied too long first and last name they are not shown properly. As shown on the picture, they are under the avatar and are going beyond the screen.

Steps to Reproduce the Problem

1. Create new user with long first and last name.
2. Log in.
3. Click on menu symbol in the right corner.

Additional comment

Expected Behavior

User would like to click on the 2D/3D images and label it by specifying points.

Actual Behavior

MedTagger does not support tagging by points.

Example

Expected Behavior

3D Scans should show views from X & Y axis.

Actual Behavior

MedTagger's backend is ready to convert and prepare Slices from other orientations than Z axis. These can be retireved by the WebSocket API in the same way as it is done for Z axis but to do so, you've got to add additional parameter called "orientation".

Steps to Reproduce the Problem

1. Go to the Labeling page.

Expected Behavior

User should be able to skip the tutorial and/or display tutorial only once.

Actual Behavior

User should walk through the tutorial each time he/she wants to label scans.

Steps to Reproduce the Problem

1. Go to the labeling page.
2. Go through the tutorial.
3. Label scan.
4. Go to the home page.
5. Go to the labeling page.
6. And... you still have to walk through the tutorial. Annoying.

Expected Behavior

Administrator should be able to manage Scan Categories from the UI.

Actual Behavior

Right now, Scan Categories are enabled and defined by logging into the DB and what's more, images are loading from the frontend repository.

Additional comment

It may be related to this issue #23.

Expected Behavior

SQLAlchemy session object should be handled properly.

Actual Behavior

My intuition is that we are not using ScopedSessions properly. Linters hints that sessions has no properties like query or commit(). We should check it out and make sure that current implementation is proper one.

Additional comment

Look for the SQLAlchemy docs.

Expected Behavior

When user wants to register all of the fields should be clear.

Actual Behavior

If user provides email in log-in card and then goes to register card the email field is already filled in.

Steps to Reproduce the Problem

1. Provide email in log-in card.
2. Click on 'Register' button.
3. Email in the register card will be already there.

Additional comment

Expected Behavior

Labels should have flexible data structure and be ready to handle new formats like:

• list of points,
• binary labels,
• labels on other axis.

Actual Behavior

Right now there is a fixed schema for the data structure.

Additional comment

It may be needed to move Labels from SQL to the HBase or at least introduce JSONB column to the model.

Expected Behavior

User should be able to cancel uploading scan(-s).

Actual Behavior

User cannot cancel uploading scans. It can even go to another page and all of the scans are still uploading!

Steps to Reproduce the Problem

1. Go to the upload page.
2. Select and upload wrong scan.
3. You cannot cancel it! Progress bar is updating and whole process cannot be stopped.

Additional comment

Consider adding warning while switching pages. Example taken from Facebook:

1. Write a post.

2. Try to close tab/window.
3. Boom! Warning appears.

Expected Behavior

Email should not be moved to another line if it's too long. (Or maybe it should be moved a little bit lower under the avatar)

Actual Behavior

When user has e-mail that is too long it is moved to another line under the avatar.

Steps to Reproduce the Problem

1. Create new user with long email ( mine had 32 characters)
2. Log in.
3. Click on menu symbol in right corner.

Additional comment

Expected Behavior

Nginx should be able to handle port during proxy forwarding to Swagger UI.

Actual Behavior

Going to http://host:1234/api/v1 makes call for http://host/api/v1/swagger.json which doesn't contain port.

Error on the Swagger UI:
Can't read from server. It may not have the appropriate access-control-origin settings.

Steps to Reproduce the Problem

1. Host Nginx on port other than 80 and go to Swagger UI.
2. Done

Expected Behavior

REST/WebSocket API calls should be pointed to the URL that includes port on which appliaction is hosted.

Actual Behavior

Currently, UI call REST/WebSocket API to the default port (80). It should check port on which UI has been hosted.

Steps to Reproduce the Problem

1. Host UI on a server.
2. Enable port forwardning on your server (eg. 50000 -> 80).
3. Go to the UI on port 50000.
4. Done. Now all API calls point to http://hostname/api/v1/....

Additional comment

This should as easy as changing prod environment configuration file.

Expected Behavior

MedTagger should be prepared for 2D Scans like X-Ray.

Actual Behavior

MedTagger only supports 3D Scans.

Additional comment

1. Upload page should be updated.
2. Marker page shouldn't show slider on the right and allow to send label with one selection.
3. Validation page should be ready to show 2D Scans.
4. Consider adding "ScanType" to the database.

Expected Behavior

Functional tests should not use development database for testing. It should create its own instances and perform tests on them. It should not break developer's environment.

Actual Behavior

Functional tests uses dockerized instances of Postgres and HBase. This is fine but... the same database instances are used in the development environment.

Steps to Reproduce the Problem

1. Prepare development setup for backend.
2. Run make functional_tests.
3. Your databases are corrupted.

Expected Behavior

Scan Categories should be assigned with a plugin which will process Scans after upload. Default implementation should stay as it is now.

Actual Behavior

Scans are processed always in the same way (after upload).

Additional comment

This may be a blocker for enabling new types of scans, eg. 2D or movies. Also, it's a blocker for enabling Scan Categories other than lungs (different HU values).

Expected Behavior

All of the form fields and all variables should be cleaned after clicking "Upload more scans" button after completed upload.

Actual Behavior

Not all form fields are cleared properly.

Steps to Reproduce the Problem

1. Go to the upload page.
2. Upload an example scan.
3. On success page click "Upload more scans".
4. Check form fields and variables.

Expected Behavior

Code comments and documentation for Selector interface. This will help contributors to easily create new kind of graphical selectors according to marker/viewer architecture.

Actual Behavior

Self-documenting code, not enough information to easily start creating new selectors.

Expected Behavior

Uploading scans should be as fast as ISP can handle.

Actual Behavior

Scans upload is slow and even setup on a local machine takes too much time.

Steps to Reproduce the Problem

1. Go to upload page.
2. Try to upload 20 scans.
3. Done. It takes ages...

Additional comment

Let's figure out how to speed it up. At the beginning I would start with profiling both frontend and backend app as it may be a problem on both sides. I believe that it may be connected with:

• reading files from the disk,
• WebSocket framework handling is slow.

Expected Behavior

User should move between login page and home page smoothly.

Actual Behavior

There can be a glitch that makes whole MedTagger unusable and unstable. In console logs there is an error:

TypeError: Cannot read property 'role' of null

Screenshot:

Steps to Reproduce the Problem

1. Log in.
2. Done (if you are lucky).

Addtional comments

I was using Google Chrome 63 (stable). Refreshing the page helped.

Expected Behavior

Enabling contributors to easily add support for different languages. Creation of translating service with support of .json formatted language files.

Actual Behavior

Hardcoded labels, english only.

Expected Behavior

User can use slider on the right to move between Slices.

Actual Behavior

User can only click on this slider which is not intuitive.

Additional comment

It would be nice to provide a hint for keyboard shortcuts.

Expected Behavior

User wants to develop frontend without setting up backend each time he/she wants to edit JS/HTML files.

Actual Behavior

Backend needs to be set up and working for developing frontend.

Additional comment

Consider simplifying backend environment. It should be as light as possible if the developer doesn't want to upload/label/verify scans.

Expected Behavior

MedTagger should handle movies.

Actual Behavior

Current implementation of MedTagger only works with 3D Scans.

Additional comment

This should be rather simple. Let's use current implementation of a backend, add new upload category and move slider from right to the bottom (play button would be nice but it can be done later).

Expected Behavior

Scan Categories' images should be defined/stored in the database (Postgres or Cassandra). Frontend should fetch them together with list of all available categories.

Actual Behavior

Scan Categories' images are now stored in the frontend repository in Git. Scan Categories endpoint returns path to them, so that the HTML knows which one to use.

Steps to Reproduce the Problem

1. Go to the labeling page.
2. Skip tutorial.
3. Once you are on the category selecting page, done.