Code Monkey home page Code Monkey logo

drogulus's Introduction

The Drogulus

This is an unfinished work in progress!

A small and simple peer-to-peer data store and an exercise in practical philosophy.

It'll probably all come to nothing. ;-)

You can't do much with the drogulus at the moment although this should change very soon.

This is free and unencumbered software released into the public domain. Please read the UNLICENSE file for details.

Why?

The following three (somewhat out of date) blog posts from a couple of years ago explain what I'm up to. You should probably read them in the order they are listed:

Developer Setup

This project requires Python version 3.3 (or higher).

Make a new virtualenv (see: http://virtualenvwrapper.readthedocs.org/en/latest/) using Python > 3.3:

$ mkvirtualenv drogulus

Install the requirements:

$ pip install -r requirements.txt

The make command is a useful starting point. If you type make check and see a passing test suite followed by a coverage report then you should be set up all fine and dandy.

drogulus's People

Contributors

mhr avatar ntoll avatar terrycojones avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

drogulus's Issues

It should be possible for integration tests to bring up a drogulus network of N nodes running on localhost

For the integration test suite to work properly and fully exercise all the capabilities of target test-mode (see #22) nodes it should be trivial to bring up a "swarm" of nodes running on localhost.

There should be some convention for allowing the integration test suite (and users) to inspect and check the logs generated by the nodes in order to confirm the expected behaviour has taken place.

Proof-read documentation

One of my aims with Drogulus is to have great documentation from the start. Furthermore, writing coherent and useful documentation should be a core part of the development process and I want to ensure a strong and unambiguous precedent is set. I've been reading up on Don Knuth's "literate programming" (viz. http://en.wikipedia.org/wiki/Literate_programming), would like to borrow from these conventions and make use of the existing Sphinx document-creation suite (viz. http://sphinx-doc.org/).

To fulfil this ticket three requirements must be met:

  • It must be obvious how to build the documentation (something in the README.rst file and an updated requirements.txt file).
  • The documentation (in the doc directory) should build correctly and be complete (i.e. all appropriately formatted documentation in the source code should find its way in to the generated documentation).
  • The content of the documentation should be well organised, correct and make sense.

The first two items are technical and testable, the final item is more a matter of taste. However, I feel that the third item is the most important given that I am close to the code and it all makes sense to me. It would be wonderful to get feedback.

N.B. Most of the non-source derived documentation is in an untidy "brain-dump" state and will be improved soon. This will include a getting started guide for both users and developers.

Implement a test mode

A drogulus node may be started in "test mode" for the purposes of, er, testing. This means the following default behaviour will happen:

  • Very verbose logging to a specified test.log file.
  • Logging also emits to stdout.
  • Various non-standard settings may be specified in a configuration file.

The following constraints are enforced on a node running in test mode:

  • It will only accept request originating from localhost (on any port).
  • It will only be able to send outgoing requests to localhost (on any port).

Editing "How does Drogulous work?"

I intend to write the "Distributed Hash Table" and "Cryptographic Trust" sections of the documentation this weekend. I have raised this ticket to prevent double-effort or clashing.

You can follow my progress in my "documentation_iss8" branch in my fork of Drogulous.

Whitepaper

Could you provide a whitepaper? Not a bunch of different whitepapers, but the whitepaper about this project. And could you clarify, what is the difference (and pros and cons) between drogulus and its substitutes: freenet, Tahoe-LAFS, Datacoin, etc...

First quick docs proofread

I went through all of the docs and gave them a quick proofread. I have tried to make them more consistent (ID everywhere instead of half "id", half "ID") and less ambiguous in places.

I have kept your original wording where possible, and have only really added one or two words. Some sentences have been removed as they did not seem to be relevant to the function.

Create a tutorial for the website

To cover:

  • Downloading and installing
  • Simple use cases of:
    • Setting your own value
    • Getting your own value
    • Aliasing someone else and getting their values
    • A simple ITTT listener
  • Using the drogulus module in your own code
  • Getting involved with development

Integration tests should use built in command for starting local nodes

Currently the integration tests duplicate (and simplify) the code needed to start a local node. This is a bad thing since we should dogfood our own utilities. Ergo, the integration test suite should start local nodes using the drogulus start command (passing in a flag to denote test mode - see #22).

Create an about page for the website

There needs to be a more "in depth" about page that covers the following broad categories:

  • The aims of the project - what problem does the drogulus solve?
  • Historical background - why (and how) was the project started?
  • The technology - what is a DHT? what about "programmable"?
  • Example use cases - see #28 for where this should link to.
  • How to get involved - a call to action.

Bonus points if < 1000 words. Perhaps include video walk-throughs. ;-)

Move to configparser for user defined settings

The ConfigParser class implements a well known config DSL with support built into core Python. Given the requirements for the integration test suite to configure local nodes on-the-fly, and the desire that simple user configuration should be possible (it currently isn't) then the following minimal requirement should be implemented:

  • Specify and document the available settings that a user may specify.
  • Specify and document the settings that can be overridden when in "test mode".

See https://docs.python.org/3.4/library/configparser.html for details.

Create a root node on the open internet with a fixed IP address

Probably using #34

This will be the IP address / port made public for those wishing to join the network. For the purposes of debugging, information gathering we'll probably need:

  • Some mechanism for checking current availability (use a third party service)
  • A schedule for backing up the routing table and local dictionary
  • A simple mechanism for bringing up and restoring the node from snapshots in case of failure
  • A way to generate interesting stats about the interaction with the wider network:
    • Number of unique peers seen
    • Traffic volume
    • Number of user's encountered
    • Number of ITTT like triggers fired
  • General local performance of the node (for the purposes of performance review)

Remove "It'll probably all come to nothing..." at some appropriate time in the future.

In the README.rst there's a line, "It'll probably all come to nothing...". As @rossjones has pointed out (#12) this appears to be an un-optimistic outlook. However, I leave it in as a goad to make me work on the project (it makes me think "oh yes it bloody will"). I agree with Ross that it isn't very optimistic so it should be removed when all the following conditions are met (demonstrating that the project has come to something):

  • 5 or more regular core committers.
  • Continuous existence of the DHT for 6+ months.
  • Independently mentioned by a third party web/news source without prompting or suggestion by anyone connected with the project.
  • A comprehensive set of documentation including download instructions for Windows, Mac and Linux that non-techies could follow.

Create project landing page

A simple, clean and obvious landing page that:

  • provides the elevator pitch
  • sample uses
  • pathway to next steps

Implementing handle_value function in node.py

After reading CONTRIBUTE I suppose this is a better way to help out.

I have decided to try and implement the handle_value method of the Node class in node.py. This has meant the creation of some new methods in the Node class, but they haven't been fleshed out yet.

I am of course writing associated tests.

Currently being worked on in my implement_handle_value branch (soon to be renamed implement_handle_value_3)

Example applications needed for the documentation

The documentation needs example implementations of simple applications that make use of the drogulus module. Initial ideas include:

  • DrogBox - DropBox in X lines of Python (where X is some low number < 100) - hat tip to David Miller for the cool name. Using the fuse file-system libraries (e.g. https://github.com/terencehonles/fusepy) it should be relatively trivial to write a command line tool and associated fuse daemon that would sync/encrypt files in a specific directory with the DHT. If you copied over your .pem file (pub/priv key) to another machine - it should all just sync. As the wonderfully simple "create a blog" example was to Ruby on Rails, or the various "todo lists" are to I want to make this the "create something funky in no time at all" example to capture the attention of developers.
  • Webcache - A proxy-daemon to back up the web / host static web content. The web is the current platform du jour. People need to see the drogulus plays nicely with it. It also nicely demonstrates the fact that you don't need to host web related content on a single host - rather it's stored in the DHT, scales as required and contains no single point of ISP related control (so censorship is, in a sense, circumnavigated).
  • Browser plugin - e.g. drog://@davidmiller/kitteh.jpg where drog:// is the protocol specification, @davidmiller is a locally specified alias for a public key of an entity and kitteh.jpg is a meaningful name.
  • ITTT computation example. TODO: Logos updates.

Website -> Logos

Noticed on your website you still refer to Drogulisp rather than your new Logos. Checked your website HTML which seems to have included the updated name, so guess you just forgot to update the server?

UPnP investigation

Many nodes in the drogulus will be short-lived and exist on PCs / laptops behind routers / firewalls. These nodes should be contactable by peers on the network. For this to work some sort of uPNP (https://en.wikipedia.org/wiki/Universal_Plug_and_Play) jiggery-pokery needs to happen so that port foo on the router facing the external internet is NAT'd to the correct port/ip-address for the drogulus daemon running on the user's machine.

There are, of course, several other ways around this used by consoles, Skype and bittorrent (for example). uPNP is perhaps the most obvious / simplest..?

This ticket is for an investigation to deliver the following:

  • Feasibility of using uPNP to NAT drogulus daemons as described above using Python (see http://coherence-project.org/ for an existing Python/uPNP project).
  • Assuming success of the above, some example "spike" code.
  • Description of other potential techniques to achieve this end (bonus points for working code examples).

Create developer documentation for the website.

This should be Sphinx (http://sphinx-doc.org/) generated developer documentation for people of a technical disposition. Must include:

  1. Description of technical processes.
  2. Abstract definition of the drogulus.
  3. Description of the HTTP/JSON based API that implements 2
  4. Description of the Python implementation of 3
  5. A guide for developing with the drogulus module
  6. A guide for contributing to the drogulus
  7. A guide for implementing the drogulus in another language

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.