Code Monkey home page Code Monkey logo

iscc-web's Introduction

ISCC - Generator Microservice

Tests

About iscc-web

iscc-web is a microservice for generating International Standard Content Codes (ISCC) for media files. A public instance of this service is available at https://iscc.io

Overview

REST API

The microservice provides a REST API for generating ISCCs. The endpoints support file upload/download, metadata extraction/embedding and ISCC processing.

Files uploaded for processing are automatically deleted after a configurable timeout. An interactive API documentation is available at /docs

Demo Frontend

The service also hosts a Vue.js based demo frontend that shows how to upload media files, geneerate ISCCs, embed metadata, and compare ISCCs.


Configuration

Configuration is handled by environment variables:

Backend:

  • ISCC_WEB_ENVIRONMENT: development or production (default: development).
  • ISCC_WEB_SITE_ADDRESS: public site address (default: http://localhost:8000).
  • ISCC_WEB_PRIVATE_FILES: restrict file downloads to original uploader (default: true).
  • ISCC_WEB_MAX_UPLOAD_SIZE: max file size per file upload in bytes (default: 1073741824).
  • ISCC_WEB_STORAGE_EXPIRY: delete uploaded files after x seconds (default 3600).
  • ISCC_WEB_CLEANUP_INTERVAL: interval in seconds to run file cleanup task. Use 0 to deactivate (default: 600).
  • ISCC_WEB_LOG_LEVEL: set log level (default: DEBUG).
  • ISCC_WEB_IO_READ_SIZE: file read chunk size (default: 2097152).
  • ISCC_WEB_SENTRY_DSN: optional sentry dsn for error reporting (default: emtpy string).

The production Dockerfile also supports PORT to configure gunicorns default port. (see gunicorn docs for details)

Development

Both the backend and frontend servers need to run in parallel.

Backend

Having a Python 3.8+ environment with Poetry do:

git clone https://github.com/iscc/iscc-web.git
cd iscc-web
poetry install
iscc-web

Access the app at http://localhost:8000 Api documentation is at /docs

Before committing any changes run code formatting and tests with:

poe all

Frontend

Install Node.js with asdf or see .tool-versions for the correct version. Packages are managed by pnpm.

Run pnpm install to install the frontend dependencies.

Run pnpm run dev to run the development server.

Special thanks to the developers of

Deployment

There are many options to deploy a Python ASGI application. Here is a simple docker-compose based standalone deployment with automatic SSL/TLS configuration. Create these three files on your server:

Caddyfile

{
  email {$ISCC_WEB_SITE_EMAIL}
}

{$ISCC_WEB_SITE_ADDRESS} {
  reverse_proxy app:8000
}

.env

ISCC_WEB_ENVIRONMENT=production
ISCC_WEB_SITE_EMAIL=[email protected]
ISCC_WEB_SITE_ADDRESS=https://example.com
ISCC_WEB_PRIVATE_FILES=true
ISCC_WEB_MAX_UPLOAD_SIZE=1073741824
ISCC_WEB_STORAGE_EXPIRY=3600
ISCC_WEB_CLEANUP_INTERVAL=600
ISCC_WEB_LOG_LEVEL=INFO
ISCC_WEB_IO_READ_SIZE=2097152
FORWARDED_ALLOW_IPS=*

You can also configure iscc-core and iscc-sdk dependencies. For example to activate generation of granular fingerprints (currently only implemented for text) add the following to your .env:

ISCC_SDK_GRANULAR=true

For available environment variables see:

docker-compose.yaml

version: "3.8"

volumes:
  caddy-config:
  caddy-data:

services:
  app:
    image: ghcr.io/iscc/iscc-web:main
    init: true
    env_file: .env
  caddy:
    image: caddy:2.6.1-alpine
    restart: unless-stopped
    env_file: .env
    volumes:
      - ./Caddyfile:/etc/caddy/Caddyfile
      - caddy-config:/config
      - caddy-data:/data
    ports:
      - "80:80"
      - "443:443"
      - "443:443/udp"
    depends_on:
      - app

Make sure you have a DNS entry pointing to your servers IP and set the correct ISCC_WEB_SITE_ADDRESS in your .env file. You should also change ISCC_WEB_SITE_EMAIL.

Start the app

docker-compose up -d

Watch logs

docker-compose logs -f

Update to the latest docker image

docker-compose pull
docker-compose up -d

iscc-web's People

Contributors

strayer avatar titusz avatar

Stargazers

 avatar  avatar  avatar  avatar

Watchers

 avatar

Forkers

relves jmiziolek

iscc-web's Issues

Add endpoint that generates ISCCs from URLs

An endpoint that accepts a media asset URL and embeddable metadata as input.
The service should download the asset, embed metadata (if any), generate an ISCC asynchronously while blocking the client until the response is ready. The URL of the modified local media asset will be in the response so it is available for download within the cleanup timeout.

Open Questions:

Should the download be locked to the IP that created the request or should it be open for download.

If the processing result and/or modified asset needs to be stored somewhere else it might make sense to allow to provide a callback URL with the request, such that the server can push the result(s) to the provided callback URL.

If there is a callback URL the server wouldn´t need to block the client. Instead the server could return a job-id instantly. If the client needs to know the job status it could query the server.

Support deletion of uploaded files from frontend.

Uploaded files are deleted automatically after 60 minutes and can also be deleted manually via API. Add support for user triggered deletion via frontend. We could delete files when the user closes the result panel or add a dedicated button for deletion.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.