max-mapper / websocket-stream Goto Github PK

I do not have much background info on this one problem but I think the websocket-stream emits the end event after a certain timeout of inactivity.

Is this a well-known case? Can it reconnect when needed, i.E. with an option? Or better, on a .write() call?

If you look at mafintosh/duplexify#4 I suspect there are firewall issues, breaking stream functionality maybe. Not sure. Maybe the problem has other causes?

But do you have any tricks or commands I could use to double check this?

@maxogden how would you convert an ArrayBuffer to an Array best?

With the update to v1 I am seeing TypeError: Invalid non-string/buffer chunk errors now when calling stream.write(...)

Tested with the following:

var ws = require('websocket-stream')
var wss = ws.Server({ port : 5000 })
wss.on('connection', function(sock) {
  var stream = ws(sock)

  // echo to itself
  stream.pipe(stream)
})

var local = new ws('ws://localhost:5000')
local.on('open', function() {
  local.write('echo?')
})

local.on('data', function(data) {
  console.log('received ', data)
})

No open event is omitted by on('open', ....
How am I supposed to know when it is connected?

The following code on the client is sending broken images:

    function sendNewFrame() {
        var uri = canvas.toDataURL('image/jpg', clientSettings.image.quality)

        var bytes  = atob(uri.split(',')[1]),       // convert base64 to raw binary data held in a string
            buffer = new ArrayBuffer(bytes.length), // write the bytes of the string to an ArrayBuffer
            arr    = new Uint8Array(buffer),
            i

        for (i = 0; i < bytes.length; i++) {
            arr[i] = bytes.charCodeAt(i);
        }

        // Typed arrays like the Uint8Array are supported and enable binary sockets:
        // https://github.com/maxogden/websocket-stream#binary-sockets
        stream.write(arr)
    }

when saved on the server side, the images cannot be opened and are invalid. I followed exactly the instructions but this isn't working.

In the source code around this line https://github.com/maxogden/websocket-stream/blob/master/index.js#L81 I can see that no callback is hooked into the send() call.

According to https://github.com/einaros/ws#error-handling-best-practices a callback can be added to get a notification.

Why don't we add a new event here, called sent? If you agree, then I am happy to submit a PR.

I get the following error when testing the client:

Error: Cannot find module 'websocket-stream'

It comes about because of the first line in test-client:

var ws = require('websocket-stream')

I'm guessing you have your development environment set up so this works (perhaps by using npm link?). Should this line be changed to:

var ws = require('./')

or instructions be added to correctly set up the development environment?

Thanks for all the great work!

In 2.0.0 the websocket-stream is now running in object mode. I see why this is a good default, as it allows a consistent behavior on both ends when sending data (the framing is done by ws).

How about making this setting optional? As an example, I would want to disable this for mqtt.js.

Test case: Load single-page app, shut down server and try to connect. An error event occurs but it doesn't say anything.

Here the code on the client-side (single-page app):

var stream = websocket(...)

stream.on('error', function(err) {
    console.error(err.toString())
})

which will print an object blurb like that

Event {clipboardData: undefined, path: NodeList[0], cancelBubble: false, returnValue: true, srcElement: WebSocket…}

But I expect something like that

WebSocket connection to 'wss://localhost:8080/' failed: Error in connection establishment: net::ERR_CONNECTION_REFUSED

I'm using websocket-stream on the client and the server (thanks for the sharing). When I configure a verifyClient function the ws module will return a 401 response from it's abortConnection method when the verification doesn't succeed.

Right now that dumps an ugly error message on the client.

WebSocket connection to 'wss://localhost:8890/test' failed: Unexpected response code: 401 

That happens before the client-side error event actually fires. Is there a better way to catch and handle this error?

I'm trying to connect to a websocket server through an existing ssh stream. Client and server are both running node but one is on a remote server only accessible through ssh (via port forwarding).

Can the initial connection happen over an existing stream while still capturing a reference to it?

See websockets/ws#461 ... thanks

I am seeing this in Google Chrome:

_stream_readable.js:168 Uncaught Error: stream.push() after EOF
_stream_readable.js:168 readableAddChunk
_stream_readable.js:149 Readable.push
_stream_transform.js:145 Transform.push
stream.js:73 onmessage

No idea what it means. It's a special case where I have closed the stream, yet this is happening. Possible to add more context in the error?

this code works on the server but not on the browser:

stream = websocketStream(socket)
stream.ws.on('close', function(){
    console.log('closed')
})

ERROR: Object #<WebSocket> has no method 'on' 

The module is overwritting the write method of writable streams, but it does not accept encoding, callback or return true. Since this method is only buffering data if the connection is not open, ¿why doesn't it just buffer data in the _write method and leave the standard write method of the superclass?

My use case is that I'm trying to write directly to the stream some calculated data prior to piping the rest of my data, and I'm not being able to set encoding (I set it but it is ignored) so the receiving end gets a buffer instead of a string.

Hello, while going through the stream adventure, I received this error when trying test the websocket-stream module:
Error: Could not find a suitable bundler!
at onglobals (/Users/t/src/stream-adventure/node_modules/websocket-stream/node_modules/beefy/lib/setup-bundlers.js:50:18)
at /Users/t/src/stream-adventure/node_modules/websocket-stream/node_modules/beefy/node_modules/find-global-packages/index.js:40:25
at Object.cb as oncomplete
here is the full npm-debug.log

0 info it worked if it ends with ok
1 verbose cli [ 'node', '/usr/local/bin/npm', 'start' ]
2 info using [email protected]
3 info using [email protected]
4 verbose node symlink /usr/local/bin/node
5 verbose run-script [ 'prestart', 'start', 'poststart' ]
6 info prestart [email protected]
7 info start [email protected]
8 verbose unsafe-perm in lifecycle true
9 info [email protected] Failed to exec start script
10 verbose stack Error: [email protected] start: beefy test-client.js
10 verbose stack Exit status 8
10 verbose stack at EventEmitter. (/usr/local/lib/node_modules/npm/lib/utils/lifecycle.js:212:16)
10 verbose stack at EventEmitter.emit (events.js:98:17)
10 verbose stack at ChildProcess. (/usr/local/lib/node_modules/npm/lib/utils/spawn.js:14:12)
10 verbose stack at ChildProcess.emit (events.js:98:17)
10 verbose stack at maybeClose (child_process.js:766:16)
10 verbose stack at Process.ChildProcess._handle.onexit (child_process.js:833:5)
11 verbose pkgid [email protected]
12 verbose cwd /Users/t/src/stream-adventure/node_modules/websocket-stream
13 error Darwin 14.0.0
14 error argv "node" "/usr/local/bin/npm" "start"
15 error node v0.10.35
16 error npm v2.1.17
17 error code ELIFECYCLE
18 error [email protected] start: beefy test-client.js
18 error Exit status 8
19 error Failed at the [email protected] start script 'beefy test-client.js'.
19 error This is most likely a problem with the websocket-stream package,
19 error not with npm itself.
19 error Tell the author that this fails on your system:
19 error beefy test-client.js
19 error You can get their info via:
19 error npm owner ls websocket-stream
19 error There is likely additional logging output above.
20 verbose exit [ 1, true ]

Thanks in advance for the help!

https://github.com/maxogden/websocket-stream/blob/master/server.js#L18

FYI, we had to do the opposite trick: "connection" is a standard event we expect from a server, so
https://github.com/gritzko/stream-url-ws/blob/master/index.js#L21

Probably you should emit both?

Shouldn't every stream's .pipe() method close when the piped stream is closed?

If you clone https://github.com/fabiosantoscode/websocket-stream-issue, run node server.js, open a connection with wscat, close it, open another wscat, and type some data, server.js will crash because pipe is trying to write to a closed socket.

All it's doing is echoing incoming messages to other sockets in a simplistic way.

I might be wrong, but I expect pipe() to not crash my application when the piped-to stream closes.

Am I missing something?

Please excuse me if this issue contains anything offensive or plain wrong. I'm not as sober as I should be while trying to write networking code.

Why are you using _open , _close and _destroy instead of checking the ws.readyState?

Hi

I'm trying to use your module with JSONStream but every time i close the client connection i get Error: not opened

Example:

var WebSocketServer = require('ws').Server;
var websocketStream = require('websocket-stream');
var WebSocketClient = require('ws');
var JSONStream = require('JSONStream');

var wss = new WebSocketServer({port: 8081});

wss.on('connection', function(ws) {
  var stream = websocketStream(ws);

  stream
      .pipe(JSONStream.parse())
      .pipe(JSONStream.stringify(false))
      .pipe(stream);

});

var ws = new WebSocketClient('ws://localhost:8081/ws');

ws.on('open', function() {
  var serverCall = function () {
    ws.send('{"weee":"123"}');
  };

  var close = function () {
    ws.close();
  };

  setInterval(serverCall, 3000);

  setTimeout(closeConnection, 7000);

});
ws.on('message', function(data, flags) {
  console.log(data);
});

I have no clue why this is happening, maybe i'm doing something wrong...

PS: this code works with shoe streams

Happened few times with your latest version I checked out yesterday.

Points to this line

module.exports = WebSocket || MozWebSocket || window.WebSocket || window.MozWebSocket

User Agents were

Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)

and

Opera/9.80 (J2ME/MIDP; Opera Mini/4.2.23449/37.8474; U; en) Presto/2.12.423 Version/12.16

Is it possible to add the 'drain'-event?

I looked at the WebSocket specification and there is no such event and I'm not sure if there's a way to do it. But it would be great!

Maybe, you have any idea on how to implement it...

Duplex streams in Node.js inherit from stream.Readable:

Since JavaScript doesn't have multiple prototypal inheritance, this class prototypally inherits from Readable, and then parasitically from Writable.

websocket-stream doesn't do this and I can't use it with generic functions that dispatch on type.

My desire is for websocket-stream to inherit from stream.Readable like duplex streams do in Node.js.

It would be nice to have .unpipe() implemented or how else I can cleanup on disconnect?

It is possible to support Buffer in the browser, by converting between ArrayBuffers into Buffers using uint8.
It seems much better and node-ish.

I'll submit a pull-request and add it behind an option if you think it's worth having.

On the client side, in the websocket() constructor, how can I set a custom header there? I need this for an API ...

http://blog.nodejs.org/2013/02/07/peer-dependencies/

The new 1.0 version seems to only send binary WebSocket frames now, because a Buffer instance is always passed directly to socket.send().

I have a funny case here. The error event gives me a wrong parameter when the server is down, unreachable.

On the client side I have

        stream.on('error', function(err) {
            console.error(err)
        })

which outputs - when the server is down - following:

Event {clipboardData: undefined, path: NodeList[0], cancelBubble: false, returnValue: true, srcElement: WebSocket…}
bubbles: false
cancelBubble: false
cancelable: false
clipboardData: undefined
currentTarget: WebSocket
defaultPrevented: false
eventPhase: 0
path: NodeList[0]
returnValue: true
srcElement: WebSocket
target: WebSocket
timeStamp: 1403162196915
type: "error"
__proto__: Event

Why is the err parameter of type event??? It should be an instance of error, right?

I'm just curious about the reasons for implementing (and using) this module when there was something like binaryjs in development. For fun? Or do they differ in important ways?
Thanks.

Happens on v1.0.0 on the client side ... I think these parameters are not right

socket = new WS(target, options)

because the constructor is

function ws(uri, protocols, opts) {
  var instance;
  if (protocols) {
    instance = new WebSocket(uri, protocols);
  } else {
    instance = new WebSocket(uri);
  }
  return instance;
}

Looking at https://github.com/einaros/ws#echowebsocketorg-demo I can see that the message event also has flags in the parameters.

These are completely ignored in this package, see https://github.com/maxogden/websocket-stream/blob/master/index.js#L45.

I need these flags for performance measurements and internal adjustments for my webcam application.

Any suggestions?

index.js requires server.js
server.js creates a ws server instance. This is undefined when in a browser.
server.js then tries to inherit prototypes from this undefined WebSocketServer.
This inherit throws an error.

test-client isn't capturing this.

This error is thrown at this line

socket = new WS(target, protocols, options)

(https://github.com/maxogden/websocket-stream/blob/master/stream.js#L30)

parameters were

target: "wss://www.videomail.io?x-videomail-site-name=videomail.io”
protocols: undefined
options: Object
    __proto__: Object

User-Agent is Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/31.0.1650.63 Safari/537.36

Any ideas?

Thanks.

Just wondering if there are any plans for this module to be updated to use streams2? I'd be up for creating a PR for this but wanted to get your input first.

An error is thrown at line 50 in stream.js.

> WebSocket.prototype.addEventListener
undefined

Chrome version: 50.0.2661.94
According to MDN addEventListener is not a function on WebSocket, although it still works in Chrome Canary, Mozilla and Safari.

This is a tricky problem and I cannot locate the bug. Somehow, after upgrading to "websocket-stream": "^1.3.0", my client can write only once and then nothing happens.

After some debugging I found out that state.writing is always true after the first write operation. This in writeOrBuffer() of ./node_modules/websocket-stream/node_modules/duplexify/node_modules/readable-stream/lib/_stream_writable.js

  if (state.writing || state.corked)
    state.buffer.push(new WriteReq(chunk, encoding, cb));
  else
    doWrite(stream, state, false, len, chunk, encoding, cb);

In other words, after the first write operation, everything else is pushed in the buffer and nothing happens. Any clues?

I'm using another WS streaming library that only streams to one client at a time. Is it the same with this library, or can you emit a stream to all connected clients?

Thanks,
Andrew

Can we bump "ws": "^0.4.32" to v0.5?

So that npm won't complain

$ npm ls
npm WARN unmet dependency /var/www/videomail/staging/source/node_modules/websocket-stream requires ws@'^0.4.32' but will load
npm WARN unmet dependency /var/www/videomail/staging/source/node_modules/ws,
npm WARN unmet dependency which is version 0.5.0

I'm not sure how/why e has an assumed structure (https://github.com/maxogden/websocket-stream/blob/master/index.js#L35). I replaced this with e.data || e but perhaps there was some other intent here that I'm not aware of.

Also, I was trying to do something like levelup.readStream({ limit: 10 }).pipe(stream); on the server and it didn't work. I think pipe might be broken because the data event is returning undefined in its callback. 🐹

so, hunting down the issue with level-replication-test
got it working in with json-buffer,
but websocket-stream is accepting a Uint8Array, (which goes through correctly)
but outputting a ArrayBuffer

I think that is the problem... we want Uint8Array instead,
otherwise, bops probably needs to be updated to read from ArrayBuffer if that is not possible...

This means that the caller cannot adjust the buffering size (highWaterMark) nor the decodeStrings option.

I do not know and couldn't find any hard facts about this. Which one type results in less bandwidth? a) stream.write(blob) or b) stream.write(buffer)?

This for sending webcam images from the client to the server. It's not clear to me when looking at the source code and documentation of the ws and of this module.

As per mafintosh/csv-parser#7. I'm happy to make a PR for this, but would like to clarify the behaviour of the type option given this change.

Should a Buffer only be created if a type of Buffer is requested, or should a buffer always be created?

Hello there

We'd like to add websocket-stream to an existing web app based on RequireJS. Unfortunately we cannot change the whole client side code to run with browserify at this stage.

Agree, RequireJS is bad but do you think it is possible to add RequireJS support? Just the minimum for module loading.

Otherwise I'd have to copy your code, shrink it manually and add AMD support. That would be bad too.

What are your thoughts?

Hi,

Apologies if this is not the correct place to be asking this question. I'm trying to get a simple echo example to work and I don't understand why the below code doesn't work?

var WebSocketServer = require('ws').Server
var websocket = require('websocket-stream')
var wss = new WebSocketServer({port: 8000})
wss.on('connection', function(ws) {
  var stream = websocket(ws)
  stream.pipe(stream)
})

Thanks,
Warren.

I'm running into an issue with websocket-stream where when sending binary data through the stream from a file using pipe it closes early, and not all data is written. I have a simple example attached that reads from a file using createReadStream on the server, and uses createWriteStream on the client. When the input file is semi large (~1MB>) not all data is written to the file. The interesting thing is if I pass {end: false} to pipe all the data is written.

server:

var WebSocketServer = require('ws').Server;
var websocketStream = require('websocket-stream');

var wss = new WebSocketServer({port: 8098});
var fs = require('fs');

wss.on('connection', function connect(ws) {
  var stream = websocketStream(ws);
  var rs = fs.createReadStream('big.in');
  rs.pipe(stream, {end: true});
});

client:

var fs = require('fs');
var websocket = require('websocket-stream');

var wfs = fs.createWriteStream('big.out');

wfs.on('open', function () {
  var ws = websocket('http://localhost:8098');

  ws.on('open', function () {
    ws.pipe(wfs);
  }).on('error', function (err) {
    console.log(err);
  }).on('close', function (err) {
    console.log("Closing");
  });
});

I've looked into this, and it looks like all the data is being received by the client, but for some reason isn't getting piped correctly unless I force the server not to explicitly call end in the call to pipe.

The export in https://github.com/maxogden/websocket-stream/blob/master/ws-fallback.js fails in a Web Worker as it has no global window object.

Uncaught ReferenceError: window is not defined