trentocrowdai / crowdhub-api Goto Github PK

Implement CRUD services for Jobs

GET /jobs
GET /jobs/

PUT /jobs/
DELETE /jobs/

• worker performance metrics
• visualizations related to specific tasks

This is a configuration per each "do" block. The configuration is "Allow only new workers?". TRUE means we block workers that previously perform a job in the current workflow, FALSE means we don't care and workers can participate in as many jobs as they want.

In F8 these are Levels 1, 2 or 3. The idea is to make this configuration available based on the platform that the user chooses.

The data of the workflow created by the client needs to be mapped to the expected object needed by the server code

/projects
/workflows
/workflows/:id/start
/cache
/items
/templates

A button for previewing the design of the job. For now, we just support things that do not involve highlighting.

only for those that cached the results

• support for downloading CSV or JSON
• retrieve the results from the results endpoint of each platform (full list of judgments without aggregation)

for instance, a script to automatically finish a job.

Implement image highlighting on Toloka and F8

Create tables in the DB for:

• project
• workflow
• cache
• item
• template

POST /jobs

• We should be able to define a default template for the workflow (accessing to it from the workflow level configurations).
• We should be able to define a specific template for each "do" operator. In the configuration context of each do operator, we should also be able to select a previously defined template (like from a dropdown)

Right now, we support visual blocks to create the design of the job. Also, we could add the possibility to view the "raw" code that builds up the design. This will imply mapping our current visual blocks to components at the code level.

This property is used as the key in the input object of the successors.

This one comes after revisiting the templates that we support out-of-the-box.

Create the db entity and implement the GET action for getting the block-types

• Use CSV files instead of fixed content (AMT)
• Implement it on Toloka.
• Interface alternative where each of the sentences has a checkbox in front. (F8)

Using test-short.csv and papers-short.csv.

The idea is that we can get the same workflow (or as close as possible) as in F8.
3 items per page.
The first page is for qualification. We don't have this functionality in AMT, but we have qualifications https://docs.aws.amazon.com/AWSMechTurk/latest/AWSMechanicalTurkRequester/Concepts_QualificationsArticle.html#qualification-tests

Using test-short.csv and papers-short.csv

After #3 we implement a simple service that allows us to publish the job to any of the supported crowdsourcing platforms.

Endpoint to implement: POST /jobs/<job id>/publish

For now, assume a fixed content (the design property that we discussed) for every platform. For this issue, we only care about being able to publish to all platforms by calling the same endpoint specified above.

Generate a public link for the workflow.

This means returning the results of the last operator. Also involves running the "do" operators.

The backend should check if the user exists if it does not then it just created it.

Also the overall progress of the workflow

This feature assumes the notion of authenticated users on the platform.

The idea of this features is that researchers can share their workflows for the purpose of "reproducible research". So others can clone and run them.

https://ai.google/research/pubs/pub40741

Publish csv items to Toloka

Define the attributes and methods of the class which will be the common interface for all the three platforms.

This estimated cost section will show the estimated cost in F8 and Toloka. A "Compute cost" button triggers the cost estimation feature, and after it renders the estimated costs one can recompute the estimations if there is any change in the parameters of the do operator.

Organize the folder of the API in a modular structure and modify the platforms rendering functions

• add owner_id to the project table
• another table where we can add users to a project
• add a share action in the project list. This will popup a form where the current user should enter the email of the collaborator and an autocomplete dropdown shows the user (retrieved from the DB).

Sometimes the F8 API returns error when trying to update the CSS of the job