Code Monkey home page Code Monkey logo

droppy's Introduction

Development on droppy has ceased because I don't have enough time or motivation to properly support it and because of its outdated technology stack, it became exceedingly boring to work.

droppy is a self-hosted file storage server with a web interface and capabilities to edit files and view media directly in the browser. It is particularly well-suited to be run on low-end hardware like the Raspberry Pi.

Features

  • Responsive, scalable HTML5 interface
  • Realtime updates of file system changes
  • Directory and Multi-File upload
  • Drag-and-Drop support
  • Clipboard support to create image/text files
  • Side-by-Side mode
  • Simple and fast Search
  • Shareable public download links
  • Zip download of directories
  • Powerful text editor with themes and broad language support
  • Image and video gallery with touch support
  • Audio player with seeking support
  • Fullscreen support for editor and gallery
  • Supports installing to the homescreen
  • Docker images available for x86-64, ARMv6, ARMv7 and ARMv8

General Information

Two directories will be used, one for configuration and one for the actual files:

  • config: defaults to ~/.droppy/config, override with -c /some/dir
  • files: default ~/.droppy/files override with -f /some/dir

droppy maintains an in-memory representation of the files directory. If you're on slow storage and/or serving 100k or more files, the initial indexing on startup will likely take some time.

Installation

Local Installation 📦

With Node.js >= 12.10.0 installed, run:

$ npm install -g droppy
$ droppy start -c /srv/droppy/config -f /srv/droppy/files

To make droppy run in the background, you can use the --daemon option, thought it is adviced that you install it as a persistent service in your system. For Linux, see these guides:

Docker installation 🐳

The silverwind/droppy multi-arch images supports amd64, arm64, arm/v7 and arm/v6 architectures. To pull and run, use:

$ docker run --name droppy -p 127.0.0.1:8989:8989 silverwind/droppy

This method uses automatic volumes for /config and /files which can be overridden through -v /srv/droppy/config:/config and -v /srv/droppy/files:/files. If you're using existing files, it's advisable to use -e UID=1000 -e GID=1000 to get new files written with correct ownership.

To update a docker installation, run

$ docker pull silverwind/droppy
$ docker stop droppy && docker rm droppy
$ docker run --name droppy -p 127.0.0.1:8989:8989 silverwind/droppy

docker-compose

Alternatively, you can use the example docker-compose.yml:

$ curl -O https://raw.githubusercontent.com/silverwind/droppy/master/examples/docker-compose.yml
$ docker-compose up

This example docker-compose.yml uses the subdirectories config and files of the current working directory for storing data.

Caddy

See the example Caddyfile.

Configuration

By default, the server listens on all IPv4 and IPv6 interfaces on port 8989. On first startup, a prompt to create login data for the first account will appear. Once it's created, login credentials are enforced. Additional accounts can be created in the options interface or the command line. Configuration is done in config/config.json, which is created with these defaults:

{
  "listeners" : [
    {
      "host": ["0.0.0.0", "::"],
      "port": 8989,
      "protocol": "http"
    }
  ],
  "public": false,
  "timestamps": true,
  "linkLength": 5,
  "linkExtensions": false,
  "logLevel": 2,
  "maxFileSize": 0,
  "updateInterval": 1000,
  "pollingInterval": 0,
  "keepAlive": 20000,
  "allowFrame": false,
  "readOnly": false,
  "ignorePatterns": [],
  "watch": true,
  "headers": {}
}

Options

  • listeners Array - Defines on which network interfaces, port and protocols the server will listen. See listener options below. listeners has no effect when droppy is used as a module. The default listens on HTTP port 8989 on all interfaces and protocols.
  • public boolean - When enabled, no user authentication is performed. Default: false.
  • timestamps boolean - When enabled, adds timestamps to log output. Default: true.
  • linkLength number - The amount of characters in a shared link. Default: 5.
  • linkExtensions boolean - Whether shared links should include the file extension. This can be used to allow other software to make a guess on the content of the file without actually retrieving it. Default: false.
  • logLevel number - Logging amount. 0 is no logging, 1 is errors, 2 is info (HTTP requests), 3 is debug (Websocket communication). Default: 2.
  • maxFileSize number - The maximum file size in bytes a user can upload in a single file. 0 means no limit. Default: 0.
  • updateInterval number - Interval in milliseconds in which a single client can receive update messages through changes in the file system. Default: 1000.
  • pollingInterval number - Interval in milliseconds in which the file system is polled for changes, which may necessary for files on external or network-mapped drives. Corresponds to chokidar's usePolling option. This is CPU-intensive. 0 disables polling. Default: 0.
  • keepAlive number - Interval in milliseconds in which the server sends websocket keepalive messages, which may be necessary when proxies are involved. 0 disables keepalive messages. Default: 20000.
  • uploadTimeout number - Request timeout for upload requests in milliseconds. Default: 604800000 which is 7 days.
  • allowFrame boolean - Allow the page to be loaded into a <frame> or <iframe>. Default: false.
  • readOnly boolean - Treat all files as being read-only. Default: false.
  • dev boolean - Enable developer mode, skipping resource minification and enabling live reload. Default: false.
  • ignorePatterns Array - Array of file path glob patterns to ignore when indexing files. See here for supported patterns. Default: [].
  • watch boolean - Whether to watch the local file system for changes. Disabling this may improve performance when dealing with a large number of files, but with the downside that changes not done via droppy won't be detected. Default: true.
  • headers Object: A object with key-value pairs of custom HTTP headers to set on all responses, for example {"Access-Control-Allow-Origin": "*"}. Default: {}.

Listener Options

listeners defines on which network interfaces, ports and protocol(s) the server will listen. For example:

"listeners": [
  {
    "host": "::",
    "port": 80,
    "socket": "/tmp/droppy",
    "protocol": "http"
  },
  {
    "host": "0.0.0.0",
    "port": 443,
    "protocol": "https",
    "key": "~/certs/example.com.key",
    "cert": "~/certs/example.com.crt"
  }
]

The above configuration will result in:

  • HTTP listening on all IPv4 and IPv6 interfaces, port 80 and on the unix domain socket /tmp/droppy.
  • HTTPS listening on all IPv4 interfaces, port 443 using the provided TLS files.

A listener object accepts these options:

  • host string/Array - Network interface(s) addresses to listen on. Required when port is given. Note that "::" will typically bind to both IPv4 and IPv6 on all addresses but a "0.0.0.0" address might be required if IPv6 is disabled.
  • port number/string/Array - Network port(s) to listen on. Required when host is given.
  • socket string/Array - Unix domain socket(s) to listen on.
  • protocol string - Protocol to use, http or https. Required.

For TLS the following additional options are available. Paths can be given relative to the configuration directory and ~ is resolved as expected.

  • cert string - Path to PEM-encoded TLS certificate file, which can include additional intermediate certificates concatenated after the main certificate. Required.
  • key string - Path to PEM-encoded TLS private key file not encrypted with a passphrase. Required.

Downloading from the command line

To download shared links with curl and wget to the correct filename:

$ curl -OJ url
$ wget --content-disposition url

Development

To start a live-reloading dev server:

$ git clone https://github.com/silverwind/droppy && cd droppy
$ npm i
$ node droppy start --dev

The Makefile has a few tasks for updating dependencies, pushing docker images, see the comment above for dependencies of those tasks.

© silverwind, distributed under BSD licence.

droppy's People

Contributors

aredridel avatar bl4de avatar cp1797 avatar gissehel avatar imronras avatar inmysocks avatar niieani avatar paulmillr avatar poorchop avatar pradyvr avatar r15ch13 avatar silverwind avatar tungel avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

droppy's Issues

IOS6 crashing on WebSocket connect

IOS6 browsers seem to crash reproducably on the first packet sent through the Websocket - I will have to investigate. Android Chrome seems to be fine.

Copy a file into itself in root dir

It thinks that it is a different path see:

2014-02-23 18:10:36 127.0.0.1:54222 copy: /coffeescript-mode.coffee -> //coffeescript-mode.coffee

Split content view

Pretty big feature here, but I'd like to be able to split the view and practically have two clients in one browser window so users can easily drag-and-drop files to copy/move them, in Midnight Commander style.

I think the easiest way to do this is essentially have two independent WebSockets open for each side of the split. It'll require a lot of refactoring/enhancing of the client side code.

After login/logout, content won't show

Steps:

  1. Log out of any folder view
  2. Log back in
  3. Notice content won't load, only fixed by a reload

I've tracked it down so far to these lines:

    view.find("#content").attr("class", "center");
    view.find("#content").html(html || emptyPage);

When using $("#content") instead of view.find("#content") it works, so I think the view reference gets lost somehow by logout/login.

A better light theme

While I really like base16-dark, the light theme seems to be a bit hard to read, especially the yellow text. If you have any preference for a better light theme, go ahead and switch it.

Mobile audio playback issues

Playing audio on mobile is really awkward and/or bugged right now. I'll see to improve them, once I get my IOS/Android emulators up and running.

[split-view] Weird view switching on paste

During paste, the page changes back and forth, I think this is only on the split-view branch.

I'll look into it within the next hour. It's likely one of the places I tried changing $("#paste") to view.find("#paste") which obviously won't work yet.

IE10/11: Zero-byte file uploads never finish

Steps to reproduce:

  1. Upload any 0-byte file
  2. Notice upload UI freeze on some arbitary percentage
  3. The XHR never seems to finish, so the server won't recognize the multipart fields

Setting the XHR to synchronous seems to fix this, but that would kill our progress bar, and also would require UA sniffing, which I'd like to avoid at all costs.

Open in new tab

If I upload a html file, it would be cool if I could view it as an html page somehow.

I'm not sure if this is a server-side and/or a client side change.

Improve Editor styling

Few things to do here:

  • Let the editor window behave just like a content window (slide in/out)
  • Close / Save buttons need better styling, background on hover
  • Darker color scheme option
  • Fix code editor placement to use the full height
  • Move the filename into the path above
  • Add options for tabwidth, wordwrap etc

More adaptive hierarchies

View Hierarchies

Okay, this is a really hard pill to swallow. I understand that this will take quite a bit of time and thought, but I feel that it is necessary for readability and will make the entire project easier to maintain and recyclable.

Directory Views

.view-container
    .view.directory
        ul.path
        ul.entry-list
            li.data-row

Document Views

.view-container
    .view.document
        .text-editor
            .editor-controls-container
                .file-name
                ul.editor-options
            .CodeMirror

Media Views

.view-container
    .view.file-selection
        .file-info
             .album-art
                 img
             .title
                 span
        .controls-container
            ul.file-operations
        .entry-list-container 
            // Selected files or playlist
            ul.entry-list
.view-container
    .view.media.media-video
        .media-container
            video
        .controls-container
            ul.video-buttons
.view-container
    .view.media.media-slideshow
        .media-container
            img
        .controls-container
            ul.slideshow-buttons

Hi-Res filetype icons

This topic came up in #13. To have a fully scaleable design, we need to replace the file type bitmaps with SVGs. The problem here is that it seems pretty hard to find suitable colored SVG iconsets for this.

For refernce, the current spritesheet:

spritesheet

Add folder size

If it won't impact performance too much, I'll add folder byte size in a bit.

Switching animation issues

There's a few animation artifacts when opening a new editor view, which we should resolve, probably by setting small timeouts.

Slow initial connect in debug mode

Maybe we can fix this, here's a log on my RPi server:

 ->> listening on 0.0.0.0:9000
2014-02-21 09:03:09 ip:port GET / [200] [56ms]
2014-02-21 09:03:13 ip:port GET /!/style.css [200] [4241ms] <--
2014-02-21 09:03:14 ip:port GET /!/client.js [304] [15ms]
2014-02-21 09:03:14 ip:port GET /!/OpenSans-Light.woff [304] [7ms]
2014-02-21 09:03:14 ip:port GET /!/content/5jhw [200] [19ms]
2014-02-21 09:03:14 ip:port GET /!/svg [200] [19ms]
2014-02-21 09:03:15 ip:port GET /!/OpenSans-Regular.woff [304] [5ms]

Rework folder size updates/caching

Folder sizes got broken through recent changes, and this gives me an opportunity to rework the model for it. I have the following in mind for it:

  • On startup, do a full du on the file tree, store every folder and its size in memory.
  • When a watcher updates, start another du on the affected folder, but make sure to not run more than one du at once, and not too often, too, as it can get pretty resource intensive, depending on the number of files.

Implement a element cache

For performance reasons, I'd like to implement a global cache off all jQuery elements which get select by id.

[CodeMirror] Syntax Highlighting

The server will need to be able to host all the filetype mode scripts so this will include backend code. I am thinking that all the scripts should be hosted separately on the server and then allowed caching client side.

Investigate uploading Blobs/ArrayBuffers

FormData has quite a few cross-browser issues and also adds the overhead of a multipart-parser on the server. It might be cleaner to just upload Blobs (or ArrayBuffers) over the Websocket instead.

Editor pages should have unique URLs

We should set a URL through pushState for the editor, so users can navigate directly to it. This feature also requires a small change on the backend to redirect to client into the directory containing the file.

Not sure yet about which URL to use, how about:

http://host/some/folder/~/filename.js

Slow (~2MB/s) upload speeds

Looks like node is for some reason hitting a cpu limit during uploads, spending most of the time in:

node.exe!v8::OutputStream::GetOutputEncoding+0x2e6fc

Copy into same directory rename

If someone copies a file into its own directory, it could append " - Copy" to the basename to help someone duplicate their files.

Server-side performance regression

The server seems to spend quite a lot time (1-2s on a RasPi) switching directories and creating/renaming stuff. I'm seeing some unusual CPU spikes on simple requests. Investigating..

Vertical flexbox for the main layout

I'm working on making the page layout cleaner and easier to modify. The basic layout is

[   navigation | flex 0 0    ]
[    content | flex: 1 1     ]
[      paste | flex 0 0      ]
[ upload progress | flex 0 0 ]
[  audio player | flex 0 0   ]

Showing and hiding will be done through the in|out classes altering flex-basis to auto|0.

Textfile editor

I think that we could benefit from having a basic textarea for editing text files, but I'm not sure if this is over-complicating the software.

In my head, I am envisioning droppy having the capability to be the next cpanel file manager.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.