Code Monkey home page Code Monkey logo

tomaat-backend-suggestion's Introduction

Suggested TOMAAT backend

Hi Fausto! :)

About

Architecture

see Architecture.txt No explicit locks are used. Due to the usage of a "ticketing" system, the progress of the requested operation must be polled.

Terminology

Unfortunately, I use the terms "Task" and "Service" as a synonym. Task is meant to be an explicit instantiation of a service operation.

Workflow

Data Transfer

The data is transfered via a single .tar.gz file during upload and download. The actual data is stored inside the .tar.gz file as -a file, probably with the following pattern: file-dataid.* -inside the main.json inside root/inputs/dataid The input data can be composed by defined input classes which perform a lot of simple error handling and extract the actual data. Currently implemented input classes are:

Multiple GPU Setup

The TaskScheduler takes care of assigning incoming tasks to different compute sources (GPUs). You can assign a set of possible tasks to each compute source individually.

Tasks/Services

Services can be defined easily. (e.g. see CustomTasks/FloatComputationTask.py) The given task_identifier has to match to the "service" parameter of the form of the initial POST request.

Settings

see all options in TOMAATSettings.py

Get It Running

Make sure the folder BASE_STORAGE_DIR defined in TOMAATSettings exists. Get a PostgreSQL database running with

  • username: tomaat-user
  • password: tomaat-password
  • database: tomaat then...
  • Run... python3 QueueManager.py python3 WebService.py python3 TaskScheduling.py

Example Usage

After Get It Running, using Chrome postman, you can perform a POST request to http://localhost:5000/request with body -> form-data and form entries:

  • service: FloatComputation
  • payload: payload_float_add.tar.gz

Get the result at http://host:5000/result/id&token

What about these messy (non-JSON) text responses

Yes, we should reorganize that.

tomaat-backend-suggestion's People

Contributors

j-frei avatar

Forkers

devhliu

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.