Code Monkey home page Code Monkey logo

armadietto's Introduction

Armadietto npm Build Status

⚠️ WARNING

Please do not consider armadietto production ready, this project is still considered experimental. As with any alpha-stage storage technology, you MUST expect that it will eat your data and take precautions against this. You SHOULD expect that its APIs and storage schemas will change before it is labelled stable.

What is this?

Armadietto is a RemoteStorage server written for Node.js.

This is a complete rewrite of reStore.

Installation

  1. Ensure you have a maintained version of Node installed.
  2. If you will be using Apache as a reverse proxy, ensure it is version 2.4.49 or later.
  3. Run npm -g i armadietto

Usage

See the notes directory for configuring a reverse proxy and other recipes.

Modular (new) Server

  • Streaming storage (documents don't have to fit in server memory)
  • S3-compatible storage (requires separate S3 server; AWS S3 allows documents up to 5 TB)
  • Can run multiple application servers to increase capacity to enterprise-scale
  • Bug Fix: correctly handles If-None-Match with ETag
  • Bug Fix: returns empty listing for nonexistent folder
  • Implements current spec: draft-dejong-remotestorage-22

See the modular-server-specific documentation for usage.

Monolithic (old) Server

  • Stores user documents in server file system
  • More thoroughly tested
  • Implements older spec: draft-dejong-remotestorage-01

See the monolithic-server-specific documentation for usage.

Storage security

In production, we recommend that you restrict access to the files managed by your armadietto server as much as possible. This is particularly true if you host your storage on a machine with other web applications; you need to protect your files in the event that one of those apps is exploited.

You should take these steps to keep your storage safe:

  • Pick a unique Unix user to run your server process; no other process on the box should run as this user: sudo useradd armadietto --system --no-create-home

  • Do not run other applications as root, or as any user that could access files owned by your armadietto user

  • Make sure the directory path/to/storage cannot be read, written or executed by anyone but this user: sudo chmod 0700 /path/to/storage && sudo chown armadietto:armadietto /path/to/storage

  • Do not run armadietto as root; if you need to bind to port 80 or 443 use a reverse proxy like nginx, Apache2, caddy, lighttpd or enable bind capability: setcap 'cap_net_bind_service=+ep' `which armadietto`

  • Ideally, run your storage inside a container or on a dedicated machine

If you're using the Redis backend, apply similar access restrictions to the database and to any files containing the database access credentials.

Serving over HTTPS

Since RemoteStorage is a system for storing arbitrary user-specific data, and since it makes use of OAuth 2.0, we strongly recommend you serve it over a secure connection. You can boot the server to listen for HTTP or HTTPS requests or both.
If armadietto is behind a reverse proxy on the same machine, the proxy can handle TLS, so armadietto only needs to set enable and force in the https configuration. The reverse proxy must set the header x-forwarded-proto (or x-forwarded-ssl or x-forwarded-scheme) in the request passed to Armadietto. Armadietto does not yet support the Forwarded header.

This configuration boots the app on two ports, one secure and one plaintext:

const server = new Armadietto({
  store: store,
  http: {
    host: '127.0.0.1',
    port: 8000
  },
  https: {
    force: true,
    host:  '127.0.0.1',
    port:  4343,
    key:   'path/to/ssl.key',
    cert:  'path/to/ssl.crt',
    ca:    'path/to/ca.pem'    // optional
  },
  logging: {
    stdout: ["debug"],
    log_files: ["error"],
    log_dir: "./some-log-dir"
  }

});

server.boot();

For example, if you use certificates from Lets Encrypt, you will set

    cert: "/etc/letsencrypt/live/domainname/cert.pem",
    key: "/etc/letsencrypt/live/domainname/privkey.pem"

where domainname is (usually) the DNS name of your server.

The force: true line in the https section means the app will:

  • Return HTTPS URLs in WebFinger responses
  • Force sign-up and OAuth login pages onto an HTTPS connection
  • Refuse to process POST authentication requests over insecure connections
  • Block insecure storage requests and revoke the client's access

Armadietto considers a request to be secure if:

  • armadietto itself acts as an SSL terminator and the connection to it is encrypted
  • The X-Forwarded-SSL header has the value on
  • The X-Forwarded-Proto header has the value https
  • The X-Forwarded-Scheme header has the value https

So you can have an SSL-terminating proxy in front of armadietto as long as it sets one of those headers, and does not let external clients set them. In this setup, you can set https.force = true but omit https.port; this means armadietto itself will not accept encrypted connections but will apply the above behaviour to enforce secure connections.

Debugging an installation

Set the environment DEBUG to enable logging. For example DEBUG=true armadietto -c /etc/armadietto/conf.json

Development

See DEVELOPMENT.md

License

(The MIT License)

Copyright © 2012–2015 James Coglan Copyright © 2018–2024 remoteStorage contributors

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the 'Software'), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

armadietto's People

Contributors

agrueneberg avatar bencharp avatar dependabot[bot] avatar dougreeder avatar jakubner avatar jcoglan avatar lesion avatar michielbdejong avatar raucao avatar silverbucket avatar stokito avatar thornjad avatar untitaker avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

armadietto's Issues

Allow logging of technical details, without informing user

Most errors are communicated to the user using renderHTML (status, file, locals = {}) with the details of the error passed in the message or error field of locals. The message field is rendered in the page.

To make the server harder to hack, make logs more useful and make the server more user friendly, we should separate technical details of errors from the user message.

Thus, we should add a fourth argument to renderHTML for a technical error message, and clarify that locals is only for values to be interpolated into the page template. Also, renderHtml would call logRequest with the technical error message as the logNote field (or the locals.message field if no technical details are passed).

Of course, we also need to go through all the usages of renderHTML to ensure they don't leak info that would be useful to attackers. In particular, 404 responses from any part of the code should all look the same to the user. We can also check that the locals.message points the user toward resolving the problem. (Often that will just be "consult the admin")

Set up CI

The specs are currently not run automatically when pushing or creating PRs.

client cannot sync server data

the data on the server has been updated, but cannot update on the client side
storage/
└── laverna~
└── notes-db~
├── notebooks~
│   ├── 07807626-f90d-d395-dd10-99856417f709
│   ├── 1bd56c29-1bc1-0e84-2a29-7d85bd3846dd
│   ├── 23eaeaa6-6ce1-15f4-113d-65fa541b42bd
│   ├── 2bd12040-adb0-b2b1-6e2a-b1e93f72ae79
│   ├── 7855b6ec-c93c-ad31-d8e1-56d960f9ae73
│   ├── 87f87cda-1b30-d2ff-9a89-166271cce741
│   └── e92116e3-3b96-dc1b-8cc6-edc4e2722cd3
└── notes~
├── 059a5e58-efff-6751-6001-31f972c741a4
├── 0b9ff487-c34f-90db-17a3-e0f5c87d840c
├── 1970b843-e7f7-ab46-e2ef-531bfa499862
├── 1e2dd98e-4e53-f0c1-9fdb-2393741ac70b
├── 227534d3-17d8-1cd9-45c7-92bd9144484d
├── 2c258551-d746-4975-f647-80a5752ff8e2
├── 4145f07d-91ae-eed9-de8e-35021398ebdd
├── 6777abfd-b75d-b545-eaea-f7cbe261fd33
├── 7ee61fca-9c96-ff51-e111-efa98965e5db
├── 8e402ca0-1c91-3225-4f7a-86cec14637f2
├── 9bfe34c7-ef30-0a69-fbdf-014ce6b3b96a
├── a97c44cb-9411-f113-7292-bf85fb210651
├── b10fdf61-3217-d656-93e1-62250d08df5d
├── cc1f7afb-b38b-2bd7-9b13-e50ed38d6f04
└── e42e6ff7-347e-7004-75e8-75f2fda6ccc2

4 directories, 22 files

server side configuration
start a remotestorage server with below config
{
"allow_signup": true,
"storage_path": "/root/path",
"cache_views": true,
"http": {
"host": "xx.xx.xx.xx",
"port": 8000
}

when the client connect to server the generated auth.json information like below:
{
"email": "[email protected]",
"password": {
"salt": "Z7V4RAKKhQ+lcgvYbjC5Ag==",
"work": 10000,
"keylen": 64,
"key": "O4hVRKKEE1osCky42kTCuJ+GGnk+zQGMacHAY/STj9nTCjGRPXoIrYkesbWArqEtFKKyWeRiVU4GaqTatyjYYQ=="
},
"sessions": {
"Vq+U39F4rMTsQ2b4cRBnu7ZwQx8=": {
"clientId": "http://localhost:9000",
"permissions": {
"/laverna/": {
"r": true,
"w": true
}
}
}
}
}

I guess the problem is on the clientid and path, but I don't know how to modify them?

Switch from Travis CI to GitHub Actions

Travis CI's old Armadietto repo doesn't work anymore, the new opensource credit system is annoying and costs time, and the builds are still slower than GitHub actions. So I think we should just move this repo off of Travis and set up Actions for CI.

Error in `domain` usage

.add(res)

➜ bin git:(master) ✗ ./server.js -c example.conf
[INFO] Starting remoteStorage: http://0.0.0.0:8000
/data/dev/rs/armadietto/lib/armadietto.js:58
.add(res)
^

TypeError: Cannot read property 'add' of undefined
at Server.handler (/data/dev/rs/armadietto/lib/armadietto.js:58:7)
at emitTwo (events.js:126:13)
at Server.emit (events.js:214:7)
at parserOnIncoming (_http_server.js:602:12)
at HTTPParser.parserOnHeadersComplete (_http_common.js:116:23)

refactoring

I was doing an upgrade of the protocol version as specified in #3, but decided to doing the work @Raindeer44 has done in the main armadietto.js file to the other part of the project.
I'm working in feat/refactoring branch.

Audit logs for server + access

In order to actually use this in production we should implement a proper logging system that

  • Differentiates between info/debug/error/warnings
  • Has the ability to write log files to a user-defined location instead of just stdout
  • Log HTTP requests (as informational, perhaps access.log)

I'm happy to tackle this change.

Showing indeterminate progress indicator breaks signup submittions in Chrome

    This change seems to break signup for me. When trying to submit the form, the progress indicator shows up, but the POST request is not sent. Commenting out the click handler fixes the problem: https://github.com/remotestorage/armadietto/blob/ae4f16c861196d84afa1f9051cd4e48c58a9cb9c/lib/assets/armadietto-utilities.js#L36-L46

Anybody else seeing this? I'm a bit puzzled as to why it's not working. I don't see a preventDefault() call that would stop the actual submit event.

Originally posted by @galfert in #97 (comment)

Fix front-end for small screens

Just opening, because I noticed it's currently not rendering nicely at all on small screens, so it's kind of a bug until fixed. Should be solved with #62.

Start server without code based configuration

It would be useful if you could run something like the following:

npm install -g armadietto
armadietto --storage-type file --file-storage-path '/path/to/file' --host 0.0.0.0 --port 9999 --allow-signups

If you think this is a good idea, I'll flesh out this issue with a proposal and we can tackle it from there.

I think it goes a long way toward allow one-click installations of armadietto which, in turn, allows us to grow the number of people able to control their own storage.

When accepting connections as localhost, webfinger & storage requests will fail for public RS apps

This is due to Private Network Access, a new security mechanism, at present only on Chrome. A preflight request is sent before the webfinger request. At present, Armadietto does not support OPTIONS requests for webfinger paths, so the preflight fails with a CORS error, and the webfinger request is never made.

For the near term, RS apps on public servers can be used with Armadietto accepting connection on localhost, by using a browser that isn't Chrome. Longer term, no browser will allow this, which cramps testing of Armadietto on localhost.

RS apps also running on a HTTP server accepting connections on localhost will always be able to access Armadietto accepting connections as localhost. I think that's an acceptable workaround, as Private Network Access is a security mechanism.

Handle requests for a non-existent account gracefully

  1. Start Armadietto
  2. Delete any accounts named 'test' or 'test2'
  3. Run the API test suite against the server

Expected result: Armadietto logs a message indicated a client tried to access a non-existent account, at the 'warning' level

Actual result: Armadietto logs the message "Bad store.permissions implementation?" at the error level.

Document how to add user accounts

The default config is to not allow signups via the UI, but there seems to be no documentation on how to create accounts manually instead.

Re-add license!

The license blurb in the README is gone, and there's no LICENSE file either. This is actually a violation of the MIT license of reStore.

Document/allow theming

Would people be interested in a PR that documents how to change the theme/layout on the armadietto pages? Don't know how much it would be done, but just thought as it seems to be quite doable at the moment, it could encourage the setting up of more servers.
Could add it as a md file to the notes directory?

Add `revoke` functionality

Currently there's no way to revoke access for apps via. the API. Using the simple account dashboard, we should be able to list all apps that have access to our remote storage and optionally revoke their token with a simple click.
See #47

Feature: Generate OAuth read-only (and read-write) tokens for server apps

Other servers that access RS require setting the tokens manually, as they generally don't have a Web interface.

This is also required to run the RS api-test-suite.

This would probably be part of an admin UI.

Read-only tokens would also allow sharing with friends, but for most apps it's not clear how they would take advantage of that.

dependency updates

Tried to update all dependencies but got in to some issues with the node.js url deprecation of the parse method, seems plugging in url.URL was breaking things. Didn't have time to try to resolve as all tests were breaking but wanted to make a not in case someone else has time to look this over.

Support LDAP for authentication

Over at https://kosmos.org we are using LDAP for user accounts. We would like to eventually deploy Armadietto for our users, and I think it would also be useful for anyone else using LDAP already, if they could configure Armadietto to add remoteStorage support to their existing accounts.

(In a perfect world, this would even be done by email providers, so that your email address is the same as your RS user address. We also plan to add an optional email service to Kosmos accounts eventually.)

Approved app listing

Is it within the scope of the armadietto project to display all approved apps/permissions per user account? Or is this something better left to 3rd party tooling to implement?

Either way, it's probably something I'd want to take a stab at - just want to know where would be best.

In Production mode, responses should not leak information

When NODE_ENV=production is set in the environment

  • "Username not found" -> "The user name and password don't match"
  • "Incorrect password" -> "The user name and password don't match"
  • error.message should never be displayed, only an appropriate message or "An error occurred".

Other responses might need to be changed as well.

The log must still contain the details we hide from end users.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.