marcelklehr / smokesignal Goto Github PK

Build your own small (or larger) peer to peer network with node.js

JavaScript 100.00%

smokesignal's Introduction

Scale your real-time app with p2p message passing.

Simple: Broadcast stuff or just speak to a specific peer.
Autonomic: Have your nodes gather and reorder automatically.
Flexible: You define, how your nodes will behave.
Self-healing: Auto-detection of netsplits and disconnect events.

Smokesignal is not a gossip protocol. It does not implement p2p data replication for you. It is a plain, flexible peer-to-peer networking solution, onto which you can easily build your own replication model or use some event passing mechanism.

To faciliate this, direct connections from peer to peer as well as a network-wide broadcast (Node.broadcast) are simple duplex streams. Thus you can add all favours of stream-goodness, like remote-events, p2p-rpc-stream, and what not.

You can connect to new peers manually or allow your node to search for peers automatically. You can listen on peer list events to get notified when your node added or removed a peer. If you want, your node will automatically ping the network seed(s) once in a while, to detect netsplits and resolve them automatically. Nodes emit a connect event when the node adds the first peer, and a disconnect event, when the last peer disconnects. If you have specified some seeds, your node will automatically rejoin the network in this case.

This project is in development, so be prepared that things might break in some situations.

npm install smokesignal

Synopsis

var smoke = require('smokesignal')


var node = smoke.createNode({
  port: 8495
, address: smoke.localIp('192.168.2.1/255.255.255.0') // Tell it your subnet and it'll figure out the right IP for you
, seeds: [{port: 13, address:'192.168.2.100'}] // the address of a seed (a known node)
})

// listen on network events...

node.on('connect', function() {
  // Hey, now we have at least one peer!
  
  // ...and broadcast stuff -- this is an ordinary duplex stream!
  node.broadcast.write('HEYO! I\'m here')
})

node.on('disconnect', function() {
  // Bah, all peers gone.
})

// Broadcast is a stream
process.stdin.pipe(node.broadcast).pipe(process.stdout)

// Start the darn thing
node.start()

// mah, i'd rather stop it
node.stop()

Check out the complete chat app example!

API

smoke.createNode(opts:object)

Creates a new node. Options:

address: (compulsory) Your current ip address
port: (compulsory) Port to bind at
minPeerNo: (optional; default: 3) how many peers this node will actively try to bond with -- you can always connect to more manually!
maxPeerNo: (optional; default: 5) how many peers this node will accept at max. Infinity for no limit
seeds: (optional) an array of known nodes, that are part of the network, e.g. {port: 0, address: '127.0.0.1'}
pingTimeout: (optional; default: 3000) The time span in ms after which we consider the ping as failed
logger: (optional; default: empty object) An object that may provide the following methods: trace, debug, info, warn, error, fatal

Class: Node

Event: connect

Emitted when we have at least one peer.

Event: disconnect

Emitted when the last peer disconnects.

Node#broadcast

A duplex stream. Everyone will get what you write to it, and you'll get everything other people write to it also here.

Node#start()

Starts the node. The tcp server will be bound to the specified port and the node will try to enter the network.

Node#stop()

Stops the node. Will disconnect all peers and shut down the tcp server.

Node#addPeer(address:string, port:int)

Tries to connect to the node at the specified address and add it as a peer. This should allow people to pass a callback..

Node#peers

An instance of Peerlist

Class: Peerlist

Event: add

Emitted when a peer is added. This event is triggered with the corresponding peer object as the first parameter.

Event: remove

Emitted when a peer is removed. This event is triggered with the corresponding peer object as the first parameter.

Peerlist#inList(my_peer:Peer)

Returns a boolean indicating whether you're currently friends with that peer.

Peerlist#list

An array containing all nodes you're friends with. Please don't manipulate this directly. Instead, use Node#addPeer to try and connect to a node and Peer#close

Class: Peer

A duplex stream. Write something to it and the other end will get it out of their representational Peer object for your node. Vice versa, if the other side writes something to their object, you'll be able read it here.

Event: end

Emitted when all ties to this node have been cut.

Peer#remoteAddress

The remote address of this peer.

Peer#remotePort

The remote port of this peer.

Peer#id

The network-wide id of this peer.

Peer#close

Cut's everything that ties you to this node.

Todo

Use event-loop-friendly nextTick call(back)s
Maybe make options.port optional, so it just uses an available port

Legal

Changelog

0.2.1

Correctly inherit from EventEmitter

0.2.0

Replace socket.io-like interfaces with proper duplex streams

0.1.0

Don't depend on log4js

smokesignal's People

Contributors

Stargazers

Watchers

smokesignal's Issues

Event Emitter triggers all the objects of prototype "Node"

In the file node.js, the function Node extends Event Emitter. However, the emission of one event triggers the corresponding event of the other Nodes. ( I suspect that it is not the expected behaviour )

A link that describes the problem and how to solve it (requires very few changes):
http://opensourcejason.info/2013/10/08/properly-extending-eventemitter-in-node-js/

Btw, thanks for sharing smokesignal.

How to connect 2 computers?

I installed smokesignal on 2 computers and run node peer.js
I gave each ip address like 192.168.2.3 for 1st com, 192.168.2.12 for 2nd com.
Seed 192.168.2.100, port num is 192.168.2.100

After run node peer.js command, it shows connecting message... and then nothing happen..
Give me help

streams, streams, streams

Those socket.io-like thingies should really become simple streams!

Abstract storage

Cannot get the chat example to work

I am new to javascript and P2P so maybe I am doing something dumb.

On my first computer I run "node peer.js" from the command line. I get a printout of the node information and a "connecting..." message. I have tried this with and without a seed. On my second computer, I use a seed with the ip address and port of the first computer. Again I get a printout of the node information and a "connecting..." message. However, nothing will happen after this.

Is this the correct process? When creating the node, I do not use the "exports.localIp" function because it was giving me trouble. Instead I just hard code the ip address of the computer.

Avoiding netsplits in the first place

The underlying cause of netsplits are subnets that form because nodes automatically choose closest nodes as peers.

With network stability in mind they should actually do quite the opposite: choose nodes that are completely unrelated to you! That way we can keep the network tightly tied together.

So, we need a way of finding out how close two nodes are.
(Possibly by computing a comparable hash of a node's peers (that should also be represented by such a comparable hash) -- this is kinda going into the direction of the page rank algorithm.)

Modular transports

Consider allowing people to add other transports, e.g. maybe zmq.

Question regarding peer count

I am assuming smokesignal doesn't simply connect peers directly to each other, so a single peer will not be aware of every other peer on the network.

Given this assumption, how easy / difficult would it be to obtain a peer count for the whole network? Is this a tractable problem, or are my assumptions about the topology of a smokesignal network incorrect?

Thanks!

Not actually calling `isSufficientlyFilled`

It looks like this line is wrong:

https://github.com/marcelklehr/smokesignal/blob/develop/lib/node.js#L140

if(!this.peers.isSufficientlyFilled) {

It looks like that should be:

if(!this.peers.isSufficientlyFilled()) {

How is the peerlist used ?

I need an example to get myself going.
node.peerlist is showing as undefined...

Allow people to dictate ip address to use

Remove Server listen callback

node.connect(65415647, 541, function() {
  // this is called, when the node listens -- use on('connect') here! (make 'listening' event)
});

Security?

There's no authentication and encryption.

Since hash(message.content) is part of the message id, all messages should survive a change of content by an adversary.

Broadcasts are anonymous, so there's no inherent authentication.

Changing seeds

In a long running application with turnover and new deploys, the seed nodes could change.

I think a mechanism for updating the seeds in active modes would be useful.

Middleware?

Instead of wrapping your own stuff around smokesignal, why not be able to add it to smokesignal directly?

Exclusive networks

There is a small probability of a network becoming exclusive (and rejecting any new nodes) if they all hit the max peer number within the current network.

I hit that with a test run with a single seed today.

The root issue is that the response to a peer request is 50/50 once past the minimum peer number.

I don't have a solution yet, and I'm curious if others have ideas.

Performance

It's not yet optimized for performance:

nssocket adds a large overhead, I think, because we don't use any of it's features.
a custom/off-the-shelf binary protocol might be cleaner and come with less bl0at

More than one networks entry

Issue with LocalHost node ?

I'm setting up the smoke p2p on multiple ip addresses on my machine(Ubuntu shell on Windows 10). I'm testing the connectivity with a localhost node.
I'm getting this error concerning the netmask.js:
/mnt/c/Users/Seth/Documents/Syncs/Research_and_Dev/Current Projects/P2P-Nodes/P2P-Nodes/node_modules/netmask/lib/netmask.js:23
throw new Error("Invalid byte: " + byte);
^

Error: Invalid byte: undefined
at ip2long (/mnt/c/Users/Seth/Documents/Syncs/Research_and_Dev/Current Projects/P2P-Nodes/P2P-Nodes/node_modules/netmask/lib/netmask.js:23:15)
at Netmask.contains (/mnt/c/Users/Seth/Documents/Syncs/Research_and_Dev/Current Projects/P2P-Nodes/P2P-Nodes/node_modules/netmask/lib/netmask.js:106:17)
at Object.exports.localIp (/mnt/c/Users/Seth/Documents/Syncs/Research_and_Dev/Current Projects/P2P-Nodes/P2P-Nodes/node_modules/smokesignal/index.js:22:17)
at Object.startupLocalSmoke (/mnt/c/Users/Seth/Documents/Syncs/Research_and_Dev/Current Projects/P2P-Nodes/P2P-Nodes/custom_modules/functions.js:25:28)

Code:

startupLocalSmoke: function(defaultSeeds = ["127.0.0.1", "13"]) {
        console.log("startupLocalSmoke: " + defaultSeeds[0])

        var localNode = smoke.createNode({
            port: parseInt(defaultSeeds[1]),
            address: smoke.localIp(defaultSeeds[0] + "/255.0.0.0") // Tell it your subnet and it'll figure out the right IP for you
                //,seeds: [{port: defaultSeeds[1], address:defaultSeeds[0]}] // the address of a seed (a known node)
        })

seed list

piping from input file instead of process.stdin

Hello,

This is my last question. Thank you for your help.

I understand that this line is how the nodes communicate with each other.
process.stdin.pipe(node.broadcast).pipe(process.stdout)

I would like to send text from an input file instead of process.stdin. I have created a both a readstream and a duplex stream to send this input file text, but this does not work. No text is printed on either screen. If I pipe the readstream or duplexstream to the stdout of the node that has changed the text file, text is printed to that screen. What does not work is printing to the screen of the peer that is being sent the text.

Could you possibly direct me what type of stream I would use in place of "process.stdin." Thank you!