vmware / data-annotator-for-machine-learning Goto Github PK

1. Modify the admin permits in annotate datagrid to show/hide the start/review btn
2. Set the 1st reviewee as the default in review page
3. Add some tool tip like how to add free text for the line
4. Review page, select the filename then update the related reviewee in the dropdown

Overview/README/Contributor's Guide needs spellcheck/grammar edit.

al service can handle the request but client will show 401 code error

[ ACTIVE-LEARNING ] [ ERROR ] QUERY-INSTANCE: Error: Request failed with status code 401
at createError (E:\vmware\Loop\GitHub\master\annotation-service\node_modules\axios\lib\core\createError.js:16:15)
at settle (E:\vmware\Loop\GitHub\master\annotation-service\node_modules\axios\lib\core\settle.js:17:12)
at IncomingMessage.handleStreamEnd (E:\vmware\Loop\GitHub\master\annotation-service\node_modules\axios\lib\adapters\http.js:260:11)
at IncomingMessage.emit (node:events:406:35)
at IncomingMessage.emit (node:domain:475:12)
at endReadableNT (node:internal/streams/readable:1348:12)
at processTicksAndRejections (node:internal/process/task_queues:83:21) {
config: {
url: '/al/model/query',
method: 'post',
data: '{"projectName":"non-ascii-code-testing","user":"[email protected]"}',
headers: {
Accept: 'application/json, text/plain, /',
Authorization: 'eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE2MzEwMTAwMzUsImVtYWlsIjoicG9jLW9zQHBvYy1vcy5jb20iLCJpYXQiOjE2MzEwMDgyMzV9.ovxbt-VGkmOJmuA9EO8A2nof-mduwVxW4-I0O6932fw',
'Content-Type': 'application/json;charset=utf-8',
'User-Agent': 'axios/0.21.1',
'Content-Length': 67
},

NER project creation page - add a section to allow select arbitrary number of columns to present to annotators but cannot be annotated.

If the user select create the NER project,
select the column with the existing label, but this column is not JSON format(or just the normal column, no existing label),
will save the format data to the database.

• Fix ner project words such as " do n't ", " is n't" split issue

Add a new field "Annotation Display Name" (Default: Passage) before "Annotation Question" field in new project page to allow project owners to config the annotation case display name in the annotation page.

Add a tooltip for the new field: This information is to inform annotators what data they are labeling.

1. change log exit link to game and call projects?src=annotate
2. make show more work in datagrid
3. sonar scan report an css issue in annotate.scss file

As a user, I should be able to choose what active learning query strategy to use instead of using the default uses the pool-based uncertainty sampling.

At the My Datasets Tab, the title of delete icon shows "Delete Project", but it should be show "Delete DataSet".

1. if the user make the file format is image, then the uploaded zip file must include at least one image type file in it, if there has no any image type file there then give a upload failed alert
2. if the user make the file format is txt, then the uploaded zip/tgz file must include at least one txt type file in it, if there has no any txt type file there then give a upload failed alert

It would be useful to add markdown support in the UI and use a flag to turn the formatting on and off.

glob-parent:
Remediation
Upgrade glob-parent to version 5.1.2 or later. For example:

"dependencies": {
  "glob-parent": ">=5.1.2"
}
or…
"devDependencies": {
  "glob-parent": ">=5.1.2"
}

normalize-url:
Remediation
Upgrade normalize-url to version 4.5.1 or later. For example:

"dependencies": {
  "normalize-url": ">=4.5.1"
}
or…
"devDependencies": {
  "normalize-url": ">=4.5.1"
}

There is no validate when the minimum value equals the maximum value in the Edit Project Modal

REFER:
annotation-app
active-learning-service
annotation-service

1. select an existing image dataset
2. the preview dataset box always loading can't see the top preview images
   

1. add checkbox for owner to decide to show the file name or not for the annotator in annotate page
2. also make this file name configuration is editable in edit page for owner and admin
3. add file name filter for reviewer in review page
4. debug APIs
5. regression test

1. when there is only one annotator, the front-end will give the total tickets number which caculated by the front-end to this annotator.
2. Then when the back-end caculate a different total tickets number after finish inserting all valide data into db, the front-end should use this new number and let owner can do edit the original assigned tickets number according to this new one

1. const a = [...text.matchAll(RegExp(eval(reg), 'g'))];
2. outNo += Math.floor(Math.random() * 10);

Project Creation Page

• during assign annotators, should show owner the default specifica ticket number that evenly distributed to every annotator

• need make sure the total distributed number should <= this project's valid total rows

Annotate Page

• for details displayed in annotation progress, this session's number should be the latest evenly distributed number, in case there is any newly added or deleted annotator of this project

• the progress bar should be based on his own session

The Annotate Tab

• in datagrid the column Reviewed/Complete/Total, the total need to be the current annotator's own distributed ticket number, the complete is his own annotated tickets number which maybe more than the total here

Projects/Admin tab

• the projects/admin tab page, in datagrid the column Annotator/Complete/Total need changed to Annotator/Complete/Assigned
• need remove the column Generate Updated Date and add new column to show the current project's total valid ticket number

Edit Modal

• to display the current # assigned tickets and # complete tickets of every annotator
• disable the complete tickets input and make sure the assigned tickets input is editable
• need make sure the total distributed number should <= this project's valid total rows

1. update existing label select function
2. add help text selection case
3. update since function for the datagrid delete some column the original sequence has changed
4. regression test

required: node.js version need 14+

caused: fs.rmdirSync(path[, options]) is not support recursive delete before node.js version 12 and will deprecated in the future version. so changed the method to fs.rmSync(path[, options]) required node version 14+

refer:
fs.rmdirSync(path[, options])
fs.rmSync(path[, options])

As a user, I should be able to use DAML with any cloud or local file system. DAML uses AWS S3 and SQS services now.

Our use case (ML/OSTC/OCTO) involves labeling GitHub reviews and comments. Currently the whitespace gets eliminated making human labeling of data more difficult.

1. add variable enableAWSS3 in env.ts to make this decouple is configurable
2. send all upload file in binary to backend

1. configure a default fileSize=1024 * 1024 * 1024 byte in the environment.ts file.
2. when choose file size is larger than default fileSize can not upload and show an error message.

1. when the backend is just inserting data into db, the edit btn shouldn't be clickable during this time. Because before the backend finish insertinng data, to edit project name will disturb the data insert correctly.
2. disable the edit btn inn project tab
3. disable the edit btn in admin tab

1. when select the label column that has more than 50 labels, there will pop out a modal to alert this info, then after click ok should clear the total rows number in page
2. in un-zip.service.ts, bug is, Introduce a new variable or use its initial value before reassigning "previewHeadDatas".

1. during append, upload new datasets, the non-ascii number calculate incorrect
2. during project creation, select totally numeric column, should display the min and max in labels area but should not the text label

1. Remove the “Text” and “Visual” buttons at the right panel.
2. Instead, add a project level display dropdown with Text (default) & Render Markdown values at the left panel below the “project” dropdown if the selected project supports markdown format.
3. If the annotator selects Render Markdown, save the preference in the session so he will see markdown rendered cases for the session.

Numerical annotation is completed by entering a number within the given range in the text-box and pressing enter. To make this interface more congruent and intuitive it would be appropriate to add a submit button (already present for categorical annotation) and a slider with a text box to indicate the numerical annotation.

Numerical Annotation:

Categorical Annotation:

Slider Example:

<div class="slidecontainer">
  <input type="range" min="1" max="100" value="50" class="slider" id="myRange">
</div>

w3 Schools Sliders in HTML

For certain use cases it might be desirable to annotate multiple metrics numerically. For example:

Query: Annotate "Good Job"
Annotation: Appreciation: 4/5, Encouraging: 3/5, Dismissive: 0/5.

Query: Annotate "Brilliant Work"
Annotation: Appreciation: 5/5, Encouraging: 4/5, Dismissive: 0/5.

So it would increase the utility to allow for adding multiple numeric labels, and consolidate functions with those used for class labels.

Issue:
In the annotation page, you have to use the horizontal scroll bar to see all the text when the passage text is long.

Please wrap long text onto the next line.

On uploading a dataset for text classification on annotation InstaML automatically filters out samples (rows) with values that do not fall within ASCII. A better resolution of such a conflict might be replacing Unicode with corresponding ASCII or replacement with blank values.

With the current annotate project web page, it is not immediately obvious how to get started because the start button is hidden and one must now to slide to the end. Perhaps display start at a fixed location, in a separate column?

1. for now always query with the first annotator as the default, but if the first one's complete tickets all have been reviewed, then the reviwer never can enter that review page
2. before query need to foreach the user list to find the one which has lefted non reviewed ticket

1. Not allow the annotator to see the flagged tickets
2. In review page, default the reviewee if there is only one annotator
3. Change the string "Modify" to "Return to annotator" on the Projects tab
4. Modify the review page head title from “Annotate Details” to “Review Details”
5. test