Code Monkey home page Code Monkey logo

limbo's Introduction

Limbo: the file sharing lightweight service

GitHub license GitHub latest tag GitHub code size in bytes Docker Image Size (tag) GitHub branch checks state

Branch CI Build Status using raw Python environments CI Build Status using Docker images
master CI status CI status
develop CI status CI status

This project implements lightweight web page with easy possibility to upload, download, list, remove files.
It requires no login.
All files are automatically deleted 24 hours after upload.

Requirements

Configuration

Config file

You can edit config.py to set custom options. All of them may be overloaded using environment variables.

Environment variables

  • LIMBO_WEB_SERVER
    Default value is wsgiref. Python web server name. wsgiref is available by default. Other values will need installing appropriate python component. Supported values: wsgiref, paste, cheroot, ...
  • LIMBO_LISTEN_HOST
    Default value is localhost. IP address to listen. Usually 127.0.0.1 or localhost should be used for local testing, 0.0.0.0 for production.
  • LIMBO_LISTEN_PORT
    Default value is 8080. IP port to listen (HTTP). Usually port 80 is used on production.
  • LIMBO_STORAGE_DIRECTORY
    Default value is ./storage. Directory to store uploaded files in. If not exists it will be created automatically with access rights 755. This may be absolute path of path relative to Limbo root directory.
  • LIMBO_STORAGE_WEB_URL_BASE
    Default value is empty string. Allows to specify alternative web url to read files stored in STORAGE_DIRECTORY through HTTP/HTTPS. It is expected this URL is served by standalone web server. Empty string disables this setting. Value requires ending / character.
  • LIMBO_MAX_STORAGE_SECONDS
    Default value is 86400. Time duration in seconds after which uploaded file will be automatically removed. 86400 seconds is equal to 24 hours. Automatic file purging happens approximately every 10 minutes.
  • LIMBO_IS_DEBUG
    Default value is 0. Enable debug mode in bottle web framework. It will disable web page template caching.

How to run the service

  1. Copy files from this repo

  2. Install service dependencies

    pip install -r requirements.txt
  3. Run the service:

    python server.py
  4. Open web page http://localhost:8080/

Docker

The following command will build docker image and will run container listening on localhost:8080.
After container is stopped it will be automatically deleted with all currently stored files.
Don't forget to elevate privileges before running docker! (sudo)

docker run --rm -it -p 127.0.0.1:8080:80 $(docker build --quiet .)

The following usage will do the same but will also mount persistent directory for stored files (~/limbo_storage).

docker run --rm -it -p 127.0.0.1:8080:80 -v ~/limbo_storage:/tmp/storage $(docker build --quiet .)

Run Limbo self-tests in docker:

docker run --rm -it $(docker build --file=tests/Dockerfile --quiet .)

Run in Windows

You can also run the service in Windows by the following command:

run-server-on-windows.cmd

Choosing underlying web server

Here is a number of bottle-compliant WSGI web servers tested with Limbo. Particular web server versions can be checked in requirements.dev.txt

  • wsgiref - logging to console, does not support async post body reading
  • cheroot (ex-cherrypy) - works, no logging to console
  • tornado - no logging to console, does not support big file upload (100+ MB)
  • twisted - slow works, no logging to console, very slow (ignores async nature of streaming_form_data)
  • waitress - slow works, no logging to console, very slow (ignores async nature of streaming_form_data)
  • paste - partly works, logging to console, but no support for utf-8 in URLs
  • flup - starts and do not handle any requests (Windows). All HTTP requests are hanging
  • gevent - can't resolve dependencies in Windows in runtime
  • gunicorn - can't resolve dependencies in Windows in runtime

Testing the service

  1. Copy files from this repo

  2. Install service dependencies

    pip install -r requirements.txt -r requirements.dev.txt -r requirements.func.txt
  3. Check syntax:

    make -- check-all-docker
    make -- check-all
  4. Run unit tests locally:

    make test

Running tests in Docker

docker build --pull --file=./tests/Dockerfile -- .
docker run --rm --tty --network=none "$(docker build --file=./tests/Dockerfile --quiet -- .)" make test

limbo's People

Contributors

kolomenkin avatar

Stargazers

 avatar

Watchers

James Cloos avatar

limbo's Issues

background thread crash

After some time Limbo server stopped cleaning outdated files.

Here is a corresponding message from docker container log:

DBG> 2019-02-25 15:12:55: FileStorage: Remove outdated file: /tmp/storage/xxxxx-update.zip"; size: 38511704
DBG> 2019-02-25 15:12:55: FileStorage: Remove outdated file: /tmp/storage/фуы5нпрц4унпцюфю.ыцпафуц"; size: 9
DBG> 2019-02-25 15:12:55: FileStorage: Remove outdated file: /tmp/storage/zzzzzzzz-x64.msi"; size: 111149056
Exception in thread Thread-1:
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/threading.py", line 916, in _bootstrap_inner
    self.run()
  File "/usr/local/lib/python3.6/threading.py", line 864, in run
    self._target(*self._args, **self._kwargs)
  File "/opt/limbo/lib_file_storage.py", line 177, in _retension_thread_procedure
    self._check_retention()
  File "/opt/limbo/lib_file_storage.py", line 198, in _check_retention
    for file in os_listdir(self._temp_directory):
PermissionError: [Errno 13] Permission denied: '/tmp/storage/incomplete'

destination file exists

DBG> 2019-03-27 10:35:01: Upload file begin
DBG> 2019-03-27 10:35:01: FileStorage: Upload file: фуы5нпрц4унпцюфю.ыцпафуц
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/bottle.py", line 862, in _handle
    return route.call(**args)
  File "/usr/local/lib/python3.6/site-packages/bottle.py", line 1740, in wrapper
    rv = callback(*a, **ka)
  File "/opt/limbo/server.py", line 163, in cgi_upload
    parser.data_received(chunk)
  File "/usr/local/lib/python3.6/site-packages/streaming_form_data/parser.py", line 57, in data_received
    retval = self._parser.data_received(data)
  File "streaming_form_data/_parser.pyx", line 178, in streaming_form_data._parser._Parser.data_received (streaming_form_data/_parser.c:4159)
  File "streaming_form_data/_parser.pyx", line 236, in streaming_form_data._parser._Parser._parse (streaming_form_data/_parser.c:5043)
  File "streaming_form_data/_parser.pyx", line 147, in streaming_form_data._parser._Parser.set_active_part (streaming_form_data/_parser.c:3512)
  File "streaming_form_data/_parser.pyx", line 92, in streaming_form_data._parser.Part.start (streaming_form_data/_parser.c:2568)
  File "/opt/limbo/server.py", line 138, in start
    self._writer = storage.open_file_writer(self.multipart_filename)
  File "/opt/limbo/lib_file_storage.py", line 123, in open_file_writer
    return AtomicFile(temp_fullname, fullname)
  File "/opt/limbo/lib_file_storage.py", line 48, in __init__
    raise Exception('Destination file already exists')
Exception: Destination file already exists

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.