graftjs / graft Goto Github PK

I was just thinking about how it would be possible to have a channel that requires you to provide certain credentials before messages to pass. It also occured to me that it might be possible to adapt passport.js to do this.

Passport seems to be designed with stuff like this in mind : jaredhanson/passport#383

For instance, this bit of pseudocode would be to allow file uploads, but only for messages that authenticate against the passport local auth, or twitter, or json web token. whatever.

var graft = require('graft');
var passport = require('passport');
var LocalStrategy = require('passport-local').Strategy;
var passportStream = require('passport-stream')(passport);
var fs = require('fs');

passport.use(new LocalStrategy(/* snip */));

var graftServer = graft()
  .pipe(passportStream.authenticate('local'))
  .pipe(uploadService)

// client
var graftClient = graft();
graftClient.pipe(graftServer);

graft.write({ 
  username: 'adam',
  password: '***',
  file: fs.createReadStream('/file/to/upload'),
  location: '/path/on/server'
});

Are we supposed to create a new instance of SPDY and websocket client Session each time we have a graft message to send? My SPDY and Websocket graft client sessions to not seem to stay open for long periods of time, when attached to a graft object used by a long-running server.

I think we might want to provide a validation-as-a-microservice module, based around Joi.

What do you think?
I'm placing it here just because it's a place...

It should be kind-of compatible with GraftJS/graft-http#1.

Still a big picture question for us.

* Just to clarify, we are referring al already deployed modules.*

If a session get closed because of an Error, we need to auto-reconnect.

Right now graft is pretty great at taking multiple jschan sessions and orchestrating them to build new applications. But to make this be actually user-friendly and something that can be easily taught/demonstrated, it needs to have a good heap of sugar on top of it.

we need a library that will allow us to have the same kind of relationship to it, as backbone has to underscore. This issue serves as a way to collect all the various options.

Some of our requirements :

1. The ability to cleanly tie the library into the existing Graft object.
2. A series of additional higher order functions, such as .filter, .map, .where etc.
3. The ability to extend the functions through the use of mixins.
4. Relatedly, what we choose to implement needs to be non-kitchen sink.
5. It needs to handle flow control to handle our nested streams.
6. It will need to be based on streams2/streams3 to avoid compatibility mode forever and always.

related issues : #2 #5

Not much activity recently, is this repo still active? In what way does the death of SPDY (in favor of HTTP/2) effect this project? Has anyone built out a bigger example that can be open-sourced? Thanks.

A big part of our mission statement is to research and document how to use the various stream-oriented libraries that are being added to npm every day.

It's become very apparent to me over the last few months that this is probably one of the biggest challenges of building systems this way. There are hundreds, if not thousands, of stream modules, and just evaluating them is a full-time job.

I have some ideas on how we can make this easier for our users, but we really need to pose the question to a wider audience.

Hi there,

Just stumbled on this project and it looks interesting, but saw that there hasn't been any public activity for a year. Has the project been abandoned?

The current README is missing both the examples and the API. We have not really much to show, but still it's worth adding stuff.

Hi,
I'm currently reviewing approaches for building microservices with Node.js.
I would like to know whether this project's status.
No activity in several months seems to be dead for a Node.js project?!
BG

What API do we want for Graft?
I think something seneca-like is really nice, but built on jsChan (with no concept of request/response).

To some extent, I envision Graft as a router for jsChan, in a similar way to relationship between Express and Connect.

What do you think?

Graft is built atop a protocol called libchan, which used certain properties of SPDY to be able to provide features similar to Go's channels over the network.

Since SPDY has been killed, and all work is going towards HTTP2, it's kind of put the upstream protocol in flux. I mean, the stuff that's there still works, and works today... but there is some settling left to be done in the eco system.

To work past this on our side, we need to find / help build a mature http2 implementation for node.js.

Are we ready to release this one?

As we are just starting this project, we need to discuss some topics up front. This is not as glamorous as the jschan code, but it is just as important.

I've been building open source communities for a long time, and I was absolutely astounded at how completely the book "Producing OSS" mapped onto how I see the problem space.

@pelger @mcollina - all of us should read the getting started chapter before the next meeting, so we can discuss how we want to manage/grow our project.

so, we need to support HTTP and REST to some extent.

What I think is:

var graftHttp = require('graft/http');
http.createServer(graftHttp);

// or with connect

connect.use(graftHttp);

This will create messages with these properties:

• req, the actual readable stream
• res, the actual writable stream
• headers
• statusChan, Readable channel that can be used to change the status code.

What do you think?

The TestAnything protocol is a stream oriented way to run and consume tests.

It's more in-line with our stated goals, and I feel it's probably technically superior to what we are using now. (not that i have any issues with mocha).

Porting the tests to tape or similar is probably a very good task for somebody who is trying to figure out how Graft works, so if that sounds like you, please do help =)

currently the first argument to .branch returns a .truthy to decide whether or not to direct messages to the second argument.

it occured to me that we might need it to be asynchronous (only authenticated users past this point), and we might need to be in situations where it returns a different stream instead of the one specified (ie: pipe things onto it)

given a spdy server created with graft/spdy on port 3447, I have this client code, modeled after the example in README.md:

'use strict';

let graft = require('graft')();
let spdy = require('graft/spdy');

let time = graft.WriteChannel();

graft.pipe(spdy.client({ port: 3447 }));

graft.write({
  topic: 'register',
  id: 'time',
  data: time
});

setInterval(function() {
  time.write(new Date());
}, 1000);

(mind you, I have no idea if this code does what I think it does, because...)

events.js:141
      throw er; // Unhandled 'error' event
            ^
Error: Protocol "http:" not supported. Expected "https:".
    at new ClientRequest (_http_client.js:53:11)
    at Object.exports.request (http.js:31:10)
    at ClientSession.newStream [as _createNewStream] (/Volumes/alien/projects/digs/digs-graft/node_modules/graft/node_modules/jschan/lib/spdy/client.js:104:18)
    at createChannel (/Volumes/alien/projects/digs/digs-graft/node_modules/graft/node_modules/jschan/lib/spdy/client.js:142:11)
    at ClientSession.WriteChannel (/Volumes/alien/projects/digs/digs-graft/node_modules/graft/node_modules/jschan/lib/spdy/client.js:156:10)
    at SPDYClient.Client._write (/Volumes/alien/projects/digs/digs-graft/node_modules/graft/lib/client.js:36:30)
    at doWrite (/Volumes/alien/projects/digs/digs-graft/node_modules/graft/node_modules/readable-stream/lib/_stream_writable.js:279:12)
    at writeOrBuffer (/Volumes/alien/projects/digs/digs-graft/node_modules/graft/node_modules/readable-stream/lib/_stream_writable.js:266:5)
    at SPDYClient.Writable.write (/Volumes/alien/projects/digs/digs-graft/node_modules/graft/node_modules/readable-stream/lib/_stream_writable.js:211:11)
    at Graft.ondata (/Volumes/alien/projects/digs/digs-graft/node_modules/graft/node_modules/readable-stream/lib/_stream_readable.js:572:20)

UPDATE: I realize this may be the fault of the server code. Here's the server, in all its glory:

'use strict';

process.env.DEBUG = 'digs-graft*';

let graft = require('graft');
let spdy = require('graft/spdy');
let rc = require('rc-yaml');
let through = require('through2');
let pkg = require('./package.json');
let domain = require('domain');
let debug = require('debug')('digs-graft');

let cfg = rc(pkg.name, {
  port: 3447
});

let digs = graft();
let router = graft();

function data() {
  return through.obj(function(msg, enc, done) {
    let debug = require('debug')('digs-graft:data');
    debug(msg);
  });
}

function register() {
  return through.obj(function(msg, enc, done) {
    let debug = require('debug')('digs-graft:register');
    debug(`client ${msg.id} registered`);

    let d = domain.create();
    d.on('error', function(err) {
      debug(err);
    });
    d.run(function() {
      router.branch(function() {
        return msg.data; // branch if data property is truthy
      }, data());
    });
  });
}

digs.where({
  topic: 'register'
}, register());

spdy.server({
  port: cfg.port
})
    .pipe(digs);

I just discovered this project, and my interest is sparked.

Say I wanted to get real-time data from a sensor and pipe it "somewhere". Would Graft help me do that?

I do understand libchan is more of a request/response thing, but the README.md also mentions it's possible to use it for asynchronous messaging (though, unfortunately, this is not elaborated upon).

I also understand a goal of Graft is ease integration with the browser. This is something I may want to take advantage of, but for now let's just say my data is going to go "somewhere".

It seems like devices want to work over things like RabbitMQ, MQTT, etc. I get the impression these protocols are popular because they are simple, and can be deployed on devices with memory/storage constraints.

But it also seems (and I may be wrong here) that publishing the same message every n milliseconds (in the interest of providing "real-time" data) would be inefficient and potentially slow--especially since the published message would have to go through a broker. Naturally, a stream or socket--with a direct connection--would make more sense, right? And it'd make even more sense if the "endpoint" wasn't necessarily a browser, but another device?

Imagine some sort of setup wherein you were monitoring electricity usage. If the electricity usage gets too high, you wish to reduce it. Say you want to dim the lights by some function of the usage, and then brighten the lights again if the usage goes back down.

So perhaps you would have your meter open a socket to some server, which would then transform the data through your function, and pipe the data to a dimmer, which would reduce or increase the brightness as necessary.

But perhaps then you're using too much juice and the lights are all but off. At this point, you still need to reduce the power consumption. So then somebody (not sure which microservice) would then tell the climate control system to take it easy. And so on, and so forth.

Oh, and you want to see a dashboard in a browser of all this craziness.

Is libchan, and thus Graft, a solution?

It's kind of hard to make any decisions about what an API needs to look like, without at the very least trying to build something in the problem domain. (see: #2 )

So we've decided to build out a collaborative whiteboard app called AetherBoard.

The project itself is something I had been working on to be able to pull off in a node knockout situation, and is a new iteration of a project we did for the 2011 contest

We were going to aim for pseudocode at first, but after plotting out the spec, I think it's more feasible to build the pieces and then figure out how they want to be connected to each other.

We’re hoping to have a working demo by nodeconf, since @mcollina is doing a jschan presentation.

check out the issue queue and lend a hand if you are interested ?

https://github.com/AetherBoard/AetherBoard/issues