Code Monkey home page Code Monkey logo

vertex's Introduction

Build Status codecov.io

Divmod Vertex

Pull up a chair, and let me tell you a story of the once and future Internet.

The Age of Innocence

In the beginning of the Internet, all networking was “peer to peer”. If you were lucky enough to have an Internet-connected computer, you could talk to any other Internet-connected computer; in fact, being able to do so was practically the definition of “Internet connected”.

However, many of these computers were the size of a large truck and required a full time staff to operate. In the fullness of time, the Internet would grow to include much smaller devices, computers owned by individuals.

The Sundering

The growing number of connected devices created a number of problems. All these new devices needed IP addresses; at first, this was simply a logistical problem that you'd want to avoid, since you might want to connect multiple computers to your home network, and later, there weren't enough IP addresses to go around any more. So some clever folks came up with Network Address Translation. As far as the Internet was concerned, your whole network was a single device, with one address; internally, of course, each computer would have its own network address, but a device on the border of the network would hide all that. This meant that you could connect as many devices to your network as you liked, and you didn't have to ask permission, and nobody outside could even necessarily tell that you'd done so. NAT was therefore a great innovation - it allowed billions more devices to get connected to the internet.

But NAT also effectively means that as a “consumer” of Internet services, you can make outgoing requests but can not accept incoming ones. In other words, you can subscribe but you can't publish; you can listen but you can't speak. Slowly, this began to break things: Internet telephones that expected to be able to call each other. Games that expected to be able to make a direct connection between two players. Even things as simple as file transfer programs that allowed you to send information directly to another person. All of this can be worked around, of course, but there are few standards for how NAT devices behave, inconsistently implemented. Most require users to become experts at networking, or endure poor performance, security problems. Even basic tasks like updating a video game require memorization of long lists of numbers and familiarity with network administration.

Nevertheless, even as all of this is happening, as all of this functionality is disappearing, the Internet's popularity is exploding. What it does enable is so amazing that we all forget the even more amazing promise we've lost.

The Internet is Broken: Long Live the Internet.

The Desert Of The Now

Today, almost all basic Internet-connected functionailty takes the form of a server - almost always a web site - controlled by a third party, rather than a program you can control on your own computer. There's nothing wrong with web sites, of course; the web has been a fantastic innovation in its own right. But there is something wrong with needing to put your information under someone else's control just because that's the only way to get it from Point A to Point B, where Point A is your house and Point B is your friend's house.

Especially when there are other ways.

Teleconferencing software, video games, and file sharing networks have all had to solve this problem in order for their basic functionality to work. So it's possible to do. But they've all solved these problems in vastly different, application-specific ways, and none of them share any common infrastructure. for direct communication. If you want to create a new application that makes use of direct connectivity, you have to become an expert in about ten times as much technology as if you wanted to create a basic web site.

A False Hope

IPv6 is coming, of course, and in principle it could free us from all this.

But in practice, it won't.

After two decades of depending on NAT for security, home computers are not prepared for the onslaught of the public Internet. When IPv6 rolls out to the general public, it will need to be done in such a way that prevents incoming traffic by default. Without a secure way to allow incoming traffic, networked devices will stay shut off in the way.

Okay, What Is Vertex, Already?!

Vertex is a general purpose system for securely connecting to a program running on behalf of another person, with a trust model based on Trust On First Use (TOFU) and Persistence of Pseudonym (POP).

Currently, when a program wants to connect somewhere over the Internet, it gives the name of the machine, and a port number. Something like:

example.com 443
^ computer  ^ port

With Vertex, instead, a program identifies a person and a purpose. Like this:

[email protected]/messaging
^ person          ^ purpose
    ^ server

Let's say Alice has a chat program that she wants to use to talk to Bob. Alice puts in an identifier like the one above into her that program, and using Vertex, it can talk directly to the same program on Bob's computer; all communication is therefore secured.

What's the point?

If you want to have a program on your computer (or, potentially, your mobile device) communicate some information directly to another, you should be able to do it:

  1. easily,
  2. securely,
  3. quickly, and
  4. directly.

Vertex attempts to enable all of this, taking care of the details of networking so that applications can just communicate.

How's this supposed to work?

Alice runs a local Vertex agent, which she registers with a Vertex server on a.example.com as [email protected]; she gets a certificate signed by a.example.com, and then maintains a connection to that server. Bob registers with a Vertex server on b.example.com as [email protected]; he gets a certificate from b.example.com and maintains a connection to that server.

Alice then connects to b.example.com; since she's never talked to it before, she requests its certificate. (Alice can also ask a.example.com, or any of her existing connections to other Vertex clients or servers, to double-check on b.example.com's certificate, to make sure that they get the same result, potentially automating the usual call-somebody-up-to-ask-if-the-SSH-server's-key-really-changed workflow we all go through.)

Alice secures her connection to b.example.com with the certificate that a.example.com previously signed; b.example.com verifies it by talking to a.example.com. On that connection, she asks to speak to Bob.

At this point, b.example.com talks to Bob and sends along Alice's certificate. If Bob approves of Alice's connection, then (and only then!) b.example.com sends along instructions for how to connect to Bob.

These instructions are a list of potential connection techniques; TCPv4, TCPv6, multiple different UDP hole punching techniques, local (behind NAT) addresses, addresses discovered by talking to Vertex servers, and so on. All of these are attempted, and the best connection is used. Regardless of which connection is selected, the local Vertex agents on Alice and Bob's computers should use the same TLS certificates to communicate with each other, and the traffic should be encrypted.

Wow, this sounds great, what kind of shape is it in?

Sadly, Vertex's current status is that of "proof of concept". Many of the things in the story above say "should" instead of "does" because it doesn't actually do those things yet. It can make some connections over the Internet and transfer some bytes, but:

  • It doesn't yet implement a workable trust model, or any way to revoke certificates.
  • There's no mechanism to ask your peers to tell you about a certificate to guard aganst DNS cache poisoning on first use.
  • Despite all the fancy certificate memory stuff, fundamentally trust is established by plain passwords.
  • It stores user passwords in plaintext.
  • There's no UI for the local agent, and no real persistence of the "buddy list".
  • There's no support for UPnP, or any other kind of automatic router configuration.
  • Its UDP-over-TCP implementation doesn't implement window scaling, among other things; it is very slow.
  • When using UDP tunnelling, it doesn't currently use encryption at all. This is actually due to a design flaw, long since fixed, in Twisted's implementation of TLS; Vertex is one of the reasons that this flaw was fixed.
  • There's no defined protocol for an agent to talk to other applications; each agent currently contains all of the code for the applications that want to speak to other nodes.

But all these flaws and all this unfinished work are just a chance for you to be a hero and improve Vertex's functionality until it's actually useful!

What's "Divmod"?

Divmod is a now-defunct start-up company that open sourced many projects in the Twisted ecosystem, including this one. All the Divmod projects were therefore named “Divmod X”. As an acknowledgement of Divmod’s contributions, the current maintainers (some of whom worked for Divmod at the time) are preserving that nomenclature.

Why "Vertex"?

The Divmod projects are all named for various mathematical concepts.

The vertex of an angle is where two rays begin or meet, and Vertex is meant to be the meeting point for your network communications.

vertex's People

Contributors

chellygel avatar cyli avatar exarkun avatar glyph avatar insequent avatar lukasa avatar markrwilliams avatar mithrandi avatar tomprince avatar washort avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

vertex's Issues

Switch from epsilon.setuphelper to setuptools

Vertex's setup.py currently uses epsilon.setuphelper. This requires Epsilon to be installed separately before Vertex's setup can be executed, which makes indirect installations of Vertex via dependencies fail; in particular, this affects Mantissa, and thus all projects depending on Mantissa. We should just switch to setuptools like (almost) all of the other ex-Divmod projects have already done.

Automate coverage calculation

We can use https://coveralls.io/ and add a coverage percentage badge to the README alongside the travis badge.

This involves at least:

  • signing up for a coveralls account and adding the vertex repo
  • modifying travis.yml to install include a python coveralls library
  • running coverage from Travis-CI
  • updating the README to include a coverage badge
  • updating the CONTRIBUTING doc to include a requirement that coverage not drop (?)

See instructions at https://github.com/coagulant/coveralls-python

While coverage percentage doesn't necessarily say anything about the quality of the tests or whether the behavior is correct, it at least gives a baseline metric (i.e. coverage percentage should not drop)

vertex ought to be (primarily) an agent, not a library

First and foremost: writing applications that use Vertex should be easy from any language, not just Python. This means we need to make it reusable across different languages, which means requiring as little of it to be in-process as possible.

Also, the local instance of Vertex stores some sensitive information: the user's private keys, their buddy list, and their trust store. There's no reason that any application should be able to read this information.

"string-level" AMP garbage should be removed

There are a few places where we use callRemoteString and def amp_XXX APIs from AMP, which are nominally "for efficiency", since they're just relaying bytes, it gets to skip some overhead. This makes understanding the code harder though, and it's hard enough to understand Vertex already. These should be made into regular old Command objects, with paired @XXX.responder methods.

Fix adding a user via q2qclient to q2qstandalone

On current master, running vertex register cyli@localhost blech hangs when connecting to a running instance of twistd -noy q2q-standalone.tac from the doc directory.

This is due to outdated usage of AMP. This should be fixed before the documentation on how to use vertex is included in #12.

Remove `sendMessage` and `q2q-message` protocol

vertex currently includes support for sending arbitrary amp commands to a given address with the q2q-message protocol, but doesn't provide any helpers for listening to that protocol, or any description of what commands might be expected.

It seems like such a messaging system would be better off not tied to Q2QService, but rather implemented separately.

In any case, the code is currently untested.

Document coding standards and processes for how to contribute to Vertex

Various questions have arisen about how to translate the Twisted contribution process to Github, as well as whether the Twisted coding standard will be adhered to strictly (vs, for instance, pep8 whitespace).

Whatever the answer, it should be documented, possibly in the README since that is what is prominently displayed when looking at a repository.

Proposed content, from various conversations with everyone:

  • Standards:
    • Twisted coding standard (would be nice to use the twisted checker on travis, once issues have been fixed) and twisted test standard
    • suggestion from @tomprince to allow multiple _-separated camel-cased phrases in test case names (see #3 (comment))
    • Test coverage should be complete.
  • Github process:
    • Issues should have meaningful titles and descriptions
    • Issues should generally be opened first, and pull requests linked to issues for the following reasons:
      • If a pull request is abandoned, it may be closed without also closing the underlying issue.
      • If the work is taken over by someone else, both pull requests may be linked to the same issue.
      • Overall discussion of how to go about resolving the problem may happen in the issue, and code review can happen in a pull request.
      • There is also this if we want to just convert issues to pull requests: http://opensoul.org/blog/archives/2012/11/09/convert-a-github-issue-into-a-pull-request/ but there are issues with it detailed in the comments (relating to multiple pull requests per issue, etc.)
    • Pull requests must be reviewed prior to merging
    • Use the big green button to merge. Include the reviewers (and a fixes, if we do separate issues and pull requests), as in Twisted merge comments.
    • Delete the branch after, if using branches
    • I have no particular opinion on branches vs forking. I guess those who have push access to the repo can do branches, and everyone else use fork the project?

setup.py should not import vertex

This can cause problems executing setup.py. Mantissa et al parse the version module directly instead, so we should probably do that instead.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.