Code Monkey home page Code Monkey logo

redis-node-client's Introduction

Redis client for Node.js Project Status

In a nutshell

  • Talk to Redis from Node.js
  • Fully asynchronous; your code is called back when an operation completes
  • Binary-safe; uses Node.js Buffer objects for request serialization and reply parsing
    • e.g. store a PNG in Redis if you'd like
  • Client API directly follows Redis' command specification
  • You have to understand how Redis works and the semantics of its command set to most effectively use this client
  • Supports Redis' new exciting PUBSUB commands
  • Automatically reconnects to Redis (doesn't drop commands sent while waiting to reconnect either) using exponential backoff

Synopsis

When working from a git clone:

var sys = require("sys");
var client = require("../lib/redis-client").createClient();
client.info(function (err, info) {
    if (err) throw new Error(err);
    sys.puts("Redis Version is: " + info.redis_version);
    client.close();
});

When working with a Kiwi-based installation:

// $ kiwi install redis-client

var sys = require("sys"), 
    kiwi = require("kiwi"),
    client = kiwi.require("redis-client").createClient();

client.info(function (err, info) {
    if (err) throw new Error(err);
    sys.puts("Redis Version is: " + info.redis_version);
    client.close();
});
  • Refer to the many tests in test/test.js for many usage examples.
  • Refer to the examples/ directory for focused examples.

Installation

This version requires at least Node.js v0.1.90 and Redis 1.3.8.

Tested with Node.js v0.1.95 and v0.1.96 and Redis 2.1.1 (the current unstable).

You have a number of choices:

  • git clone this repo or download a tarball and simply copy lib/redis-client.js into your project
  • use git submodule
  • use the Kiwi package manager for Node.js

Please let me know if the package manager "seeds" and/or metadata have issues. Installation via Kiwi or NPM at this point isn't really possible since this repo depends on a unreleased version of Node.js.

Running the tests

A good way to learn about this client is to read the test code.

To run the tests, install and run redis on the localhost on port 6379 (defaults). Then run node test/test.js [-v|-q] where -v is for "verbose" and -q is for "quiet".

$ node test/test.js
..................................................................
...........................++++++++++++++++++++++++++++++++++++

[INFO] All tests have passed.

If you see something like "PSUBSCRIBE: unknown command" then it is time to upgrade your Redis installation.

Documentation

There is a method per Redis command. E.g. SETNX becomes client.setnx.

For example, the Redis command INCRBY is specified as INCRBY key integer. Also, the INCRBY spec says that the reply will be "... the new value of key after the increment or decrement."

This translates to the following client code which increments key 'foo' by 42. If the value at key 'foo' was 0 or non-existent, 'newValue' will take value 42 when the callback function is called.

client.incrby('foo', 42, function (err, newValue) {
    // ...
});

This can get a little wacky. I'm open to suggestions for improvement here.

Note: for PUBSUB, you should use subscribeTo and unsubscribeFrom instead of the generated methods for Redis' SUBSCRIBE and UNSUBSCRIBE commands. See this and this.

Notes

All commands/requests use the Redis multi-bulk request format which will be the only accepted request protocol come Redis 2.0.

redis-node-client's People

Contributors

britg avatar elliottcable avatar fictorial avatar isaacs avatar nikhilm avatar tautologistics avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

redis-node-client's Issues

No callbacks are ever executed

Your test suite didn’t catch this, because it doesn’t actually ensure the callbacks in which the assertions are made are ever executed.

I didn’t know how to fix the problem itself, but I added failing assertions to your test suite:
elliottcable/redis-node-client@329a214474bb34aba7a49a207622d4f20ad201ec

TypeError: Cannot read property 'length' of null

TypeError: Cannot read property 'length' of null
at Client.handlePublishedMessage_ (/home/phazm/.node_libraries/redis-node-client/lib/redis-client.js:423:42)
at Client.onReply_ (/home/phazm/.node_libraries/redis-node-client/lib/redis-client.js:390:14)
at /home/phazm/.node_libraries/redis-node-client/lib/redis-client.js:136:34
at ReplyParser.feed (/home/phazm/.node_libraries/redis-node-client/lib/redis-client.js:198:25)
at Stream. (/home/phazm/.node_libraries/redis-node-client/lib/redis-client.js:337:28)
at Stream.emit (events:25:26)
at IOWatcher.callback (net:369:16)
at node.js:204:9

Using the latest from HEAD for redis, node, and redis-node-client

I'm not actually doing anything with pubsub, so I think there's an errant call somewhere, when it's running the code when it doesn't need to.

Problem with how Client.requestBuffer is used

OS: OSX 10.5.8
Node: 0.1.94
Redis: 1.3.11
redis-client: 5050251

Test case: http://gist.github.com/400261

The request sent to redis is consistently truncated in the same place. Either of two changes avoid the problem (but are not fixes):

A. Swap the order of "template" and "code" properties
B. Change Client.requestBuffer to be created with an initial size of 4096 instead of 512

I'm trying to work out what the problem is now and will submit a fix if I find it.

Multi bulk replies out of range case

In the case of LRANGE ( and perhaps others ), if the range is greater than the length, then its not an error. Rather *0<CR> is returned. But currently handle_multi_bulk_reply() only checks for -1. An empty list would be the appropriate behaviour. Patch is below. It isn't against the latest version, sorry.

diff --git a/redisclient.js b/redisclient.js
index 64dee0d..139f6f0 100644
--- a/redisclient.js
+++ b/redisclient.js
@@ -151,6 +151,8 @@ Client.prototype.handle_multi_bulk_reply = function (buf) {
   var next_reply_at = crlf_at + crlf_len;
   if (count === -1)                   // value doesn't exist
     return [ null, next_reply_at ];  
+  if (count === 0)
+    return [ [], next_reply_at ];
   if (next_reply_at >= buffer.length) 
     return null;
   var results = [];

"Error: disconnected" with latest client

The following code used to work with an older version of redis-node-client (before the latest refactoring):

var logInUser = flow.define(
  function(username, rawPassword) {  
    this.username = username
    this.rawPassword = rawPassword
    this.client = new redis.createClient();  // used to be redis.Client()
    this.client.get('user:username:'+this.username+':id', this)
  },function(err, userID){
    this.userID = userID
    if (this.userID == null) {
      self.flash('message', 'login failure')
      self.redirect('/login')
    } else {
      this.client.hgetall('user:id:' + this.userID, this)        
    }
  },function(err, user) {      
    if (sha1.hex(user.salt + this.rawPassword) != user.password) {
      self.flash('message', 'login failure')
      self.redirect('/login')
    } else {
      self.flash('message', 'login successful')
      self.session.user = user
      self.redirect('/chat')
    }
  }
);  

(This code uses flow-js [http://github.com/willconant/flow-js] to make the asynchronous logic easier to understand.)

The stacktrace culminates here:

Error: disconnected at Client.exists (/js hacking/redis-node-client/redisclient.js:582:19

(This is triggered right after the first call to client.get()).

redis-node-client itself works (for example, code like the following works fine):

setInterval(function () {
  client.get('getsetfoo', function (err, value) {
    sys.puts("value is: " + value)
  })
}, 5000); 

Is the failure that I noted expected behavior (i.e., do I need to write my code differently now), or is there something else wrong?

[patch] counter-example.js require fixture

diff --git a/counter-example.js b/counter-example.js
index cb31bb2..5328824 100644
--- a/counter-example.js
+++ b/counter-example.js
@@ -1,5 +1,5 @@
 var sys = require("sys");
-var redis = require("./redis");
+var redis = require("./redisclient");
 
 var client = new redis.Client();
 client.connect(learn_to_count);

test/test.js throwing has no method 'utf8Write' error

I am a active redis user and community member and wanted to try out redis on node.js

I installed the latest source from git for (node, redis, redis-node-client)
Both redis and node compiled w/o problem. I ran "sudo make install" node. Everything ok.
I fired up redis and ran "node test/test.js" which throws the error " has no method 'utf8Write'"
I am brand new to node.js so this could be a newbie bug, but I have no clue what is going on.

I am on Ubuntu 9.04.

LOGS:
sammy@laptop:~/Downloads/GIT/redis-node-client$ node test/test.js -v
DEBUG: [CONNECT]

[INFO] Testing AUTH

DEBUG: [ERROR] Connection to redis encountered an error: TypeError: Object # has no method 'utf8Write'
DEBUG: [ERROR] TypeError: Object # has no method 'utf8Write'
DEBUG: [RECONNECTING 1/10]
DEBUG: [WAIT 1000 ms]

node.js:50
throw e;
^
TypeError: Object # has no method 'utf8Write'
at Client.sendCommand (/mnt/newhome/Downloads/GIT/redis-node-client/lib/redis-client.js:694:37)
at Client.select (/mnt/newhome/Downloads/GIT/redis-node-client/lib/redis-client.js:768:26)
at clearTestDatabasesBeforeEachTest (/mnt/newhome/Downloads/GIT/redis-node-client/test/test.js:128:12)
at /mnt/newhome/Downloads/GIT/redis-node-client/test/test.js:1865:9
at Array.forEach (native)
at Client.runAllTests (/mnt/newhome/Downloads/GIT/redis-node-client/test/test.js:1863:22)
at Client.emit (events:27:15)
at Stream. (/mnt/newhome/Downloads/GIT/redis-node-client/lib/redis-client.js:325:16)
at Stream.emit (events:27:15)
at IOWatcher. (net:864:16)

The redis client does recieve a connection:
[7338] 19 Sep 12:07:31 - Accepted 127.0.0.1:43113
[7338] 19 Sep 12:07:31 - Client closed connection

Any help would be appreciated :)

nodejs 0.1.33 compatibility

require("tcp") changes to require("net")

this.conn = process.tcp.Connection()
becomes
this.conn = net.Stream()

Cannot read property 'length' of undefined

Getting this when I connect to Redis:

TypeError: Cannot read property 'length' of undefined

/usr/local/lib/node/.npm/redis-client/0.3.5/package/lib/redis-client.js:394

var callback = originalCommand[originalCommand.length - 1];

BULK

Hello,
Thanks for putting together this redis client. Did some testing today. Ran into a bug where there was no callback when "get"ing a key which had a large value. After some debugging, I noticed that the problem was related to this.valueBufferLen being reset.

See:

    // If the current value buffer is too big, create a new buffer, copy in
    // the old buffer, and replace the old buffer with the new buffer.

    if (this.valueBufferLen === this.valueBuffer.length) {
        sys.puts( ">>>> BUFFER OVERFLOW" );
        var newBuffer = new Buffer(this.valueBuffer.length * 2);
        this.valueBuffer.copy(newBuffer, 0, 0);
        ///D this.valueBufferLen = 0;
        this.valueBuffer = newBuffer;
    }

Are command arguments sanitized?

If I do the following:

input = getStringFromUser();
client = require('lib/redis-client').createClient();
client.get(input);

will the person giving me the input be able to do something nasty, i.e. execute commands other than GET? It would be nice if the documentation said something definitive on this point.

"testZINTER" "ERR unknown command 'zinter'"

~/.node_libraries/redis-node-client$ node test/test.js 
............................................................................................++
assert:80
  throw new assert.AssertionError({
        ^
AssertionError: "testZINTER"  "ERR unknown command 'zinter'"
    at /home/andi/.node_libraries/redis-node-client/test/test.js:121:25
    at Client.onReply_ (/home/andi/.node_libraries/redis-node-client/lib/redis-client.js:400:34)
    at /home/andi/.node_libraries/redis-node-client/lib/redis-client.js:143:30
    at ReplyParser.feed (/home/andi/.node_libraries/redis-node-client/lib/redis-client.js:160:55)
    at Stream.<anonymous> (/home/andi/.node_libraries/redis-node-client/lib/redis-client.js:337:28)
    at Stream.emit (events:26:26)
    at IOWatcher.callback (net:489:16)
    at node.js:769:9

Redis 2.0.2

[patch] Client.prototype.close() fixture - add `this` in `this.conn`

diff --git a/redisclient.js b/redisclient.js
index 6660a38..047a264 100644
--- a/redisclient.js
+++ b/redisclient.js
@@ -92,7 +92,7 @@ Client.prototype.connect = function (callback) {
 };
 
 Client.prototype.close = function () {
-  if (this.conn && conn.readyState === "open") {
+  if (this.conn && this.conn.readyState === "open") {
     this.conn.close();
     this.conn = null;
   }

Can't figure out how this is supposed to work without some ugly code

I want to get the value of item 0 in a list.

// No worky
a = redis.lindex('mylist', 0);
sys.puts(a);
=> undefined

// No worky with a callback
b = redis.lindex('mylist', 0, function (err, value) { return value; });
sys.puts(b);
=> undefined

// Works, but it's ugly
c = redis.lindex('mylist', 0, function (err, value) { somevar = value; });
sys.puts(somevar);
=> yayitworks

Is there a better way?

non-standard intepretation of trailing commas

if you try to compress redis_client.js with google closure you get this error:
redis-client.js:646: ERROR - Parse error.
Internet Explorer has a non-standard intepretation of trailing commas.
Arrays will have the wrong length and objects will not parse at all.

.. which of course does not affect the usage on nodejs but handicaps using compression in a project build process. simply removing the trailing comma on line 646 fixes this.

npm

I'd like to add a dependency on redis-node-client in npm, but it seems it's not in npm. The package installs cleanly locally, so it would just be a matter of "npm publish ."
Thanks!

Server never calls back on Node 0.2.4

I cannot make this version or the fork that claims to have fixed work with Node 0.2.4. 0.2.3 is absolutely fine however but on .4 I get no callbacks from server requests. I think it's related to originalCommand getting clobbered somehow.

Connection pooling?

Hello,

Looking at the code, it is my understanding that a Client can not be invoked "simultaneously". Is this correct? If so, the application needs to either create a new client per call (which has the overhead of creating and ending a connection for each invocation) or creating some kind of client pool and managing asking and releasing a client to the pool. Before going ahead and implementing such a pooling mechanism I was wondering 1) is my understanding correct (I am pretty new to both redis and node) and 2) if yes, do you have any plans to add connection pooling - should connection pooling be implemented in the redis client or is it something that the core node connection framework should offer. Any suggestion? [cross posted in the node google groups]

bomb when calling JSON.parse in redis-client callbacks

For example:

var kiwi = require('kiwi'),
    redisClient = kiwi.require('redis-client').createClient();
redisClient.subscribeTo("channel", function (channel, message) {
    var message2 = JSON.parse(message);
});

If a string which otherwise would be processed fine with JSON.parse is published to "channel", the above code will bomb (node.js v0.1.9x).

Maybe that's expected and I'm just clueless about how to properly buffer inbound chunks before referencing them later in the callback. The following seems to work .. most of the time:

var kiwi = require('kiwi'),
    redisClient = kiwi.require('redis-client').createClient();
redisClient.subscribeTo("channel", function (channel, message) {
    var message2 = '' + message,
         message3 = JSON.parse(message2);
});

What is the best way to go about this, if that's a good way to put it?

client.get -> TypeError: Cannot read property 'length' of undefined

Hi!
If i execute some code:
var client = require("./redis-node-client/lib/redis-client").createClient();
client.set('test', 'data');

when gets error like :
/home/node/redis-node-client/lib/redis-client.js:394
var callback = originalCommand[originalCommand.length - 1];
^
TypeError: Cannot read property 'length' of undefined

redis-node-client version 1.3.5
redis version 1.2.6
node.js version 0.1.97
Sorry for my bad english, i'm from russia)
Thanx.

"noconnection" event not emitted if Redis goes away after the first connection

To reproduce,

  1. create a script having a redis client object and a "noconnection" listener
  2. kill redis
  3. run the script - noconnection event fires as expected
  4. start redis
  5. run the script
  6. kill redis
  7. script bombs out with an exception thrown and no "noconnection" event

If you remove the "throw e" under the client.maybeReconnect() call in the stream "error" listener, that behaves as expected - a stream of "reconnecting" events followed by a "noconnection" after the maximum reconnect count is reached.

unknown command 'PSUBSCRIBE'

When I run:
node test.js

I get the following error:
DEBUG: read buffer: -ERR unknown command 'PSUBSCRIBE'-ERR unknown command 'PUNSUBS ...
DEBUG: error: ERR unknown command 'PSUBSCRIBE'
ReferenceError: error is not defined
at Client.handleReplies (/home/maritz/Programming/repos/e-Tourney/lib/redis-client/redisclient.js:219:28)
at Stream. (/home/maritz/Programming/repos/e-Tourney/lib/redis-client/redisclient.js:67:16)
at IOWatcher.callback (net:328:14)
at node.js:813:9

make test in my redis passes all tests.

GIT revisions:
redis-node-client: 44a3058
node: 53530e98 (was the same with master HEAD - had to change back for something else)
redis: cac154c5

If there is anything I can do to provide further help, let me know. :)

error passing a 3-digits number as a last value to mset

Script code:

   GLOBAL.DEBUG = 1;
   var redis = require("./lib/redis-node-client/redisclient");
   var rd_client = new redis.Client();
      rd_client.connect(function() {
         rd_client.mset("blah", 100).addCallback(function() {
            rd_client.close();
         }).addErrback(function(e) {
            rd_client.close();
         });
   });

Script output:

   ---
   call:   client.mset( 'blah','100' )
   command:*3$4mset$4blah$undefined100
   ---
   buffer: +OK-ERR unknown command '0'
   prefix: +
   result: [true,5]
   ---
   buffer: -ERR unknown command '0'
   prefix: -
   result: ["unknown command '0'",26]
   /Users/koles/tmp/node1/lib/redis-node-client/redisclient.js:246
       if (callback.promise) {
                   ^
   TypeError: Cannot read property 'promise' of undefined
       at [object Object].handle_replies (/Users/koles/tmp/node1/lib/redis-node-client/redisclient.js:246:17)
       at Connection. (/Users/koles/tmp/node1/lib/redis-node-client/redisclient.js:75:12)
       at node.js:950:9
       at node.js:954:1

The following alternatives of rd_client.mset("blah", 100) work correctly:
rd_client.set("blah", 100)
rd_client.mset("blah", 99)
rd_client.mset("blah", "100")
rd_client.mset("blah", 100, "bleh", 99)

OS: Darwin Kernel Version 9.8.0

Tested with all combinations of redis-1.1.95, redis-node-client and master branches of both.

Binary strings don't work between redis-node-client and redis-py

I'm having an issue between redis-node-client and redis-py (http://github.com/andymccurdy/redis-py), so I'm not sure which library it might be an issue with (or if it's my own issue), but this is the problem:

I have an app that stores compressed data in redis. When I store this data using redis-py, and then later try to read it with redis-node-client, I've noticed that the binary data gets mangled (gzip can't decompress the data anymore). I've attached some files that should make it pretty easy to reproduce the issue. The files to use are here: http://dl.dropbox.com/u/211886/redis_node_binary_issue.zip

  1. First, to see that things work, fire up your redis server (on the default port and host). Then run "node write_file_to_redis.js" followed by "node read_file_from_redis.js" and you should get the output of the testfile.txt.gz after it has been decompressed. This is using the node-compress module (http://github.com/waveto/node-compress) but I've included a binary in the attached files. But after running this, the output should be:
    This is some data that is going to be compressed

  2. Similarly, do the same as above but using the python scripts (so "python write_file_to_redis.py" and "python read_file_from_redis.py").

  3. Now, do the following:
    a) python write_file_to_redis.py
    b) node read_file_from_redis.js
    You'll see that no output appears

  4. Similarly:
    a) node write_file_to_redis.js
    b) python read_file_from_redis.py
    Python fails with an exception, claiming it's not a gzipped file.

I tried using redis-cli to inspect what's going on, and it seems like the binary data that gets written to redis is actually different when redis-node-client writes it vs when redis-py writes it.

Test suite error: ERR unknown command 'blpop'

Just an FYI, really, I get the following when I run the tests:

$ node ./test/test.js 
.............................................................................................AssertionError: "testBLPOP 2"  "ERR unknown command 'blpop'"
    at /Users/zach/.node_libraries/redis-node-client/test/test.js:1532:25
    at Client.onReply_ (/Users/zach/.node_libraries/redis-node-client/lib/redis-client.js:402:34)
    at /Users/zach/.node_libraries/redis-node-client/lib/redis-client.js:158:30
    at ReplyParser.feed (/Users/zach/.node_libraries/redis-node-client/lib/redis-client.js:175:55)
    at Stream.<anonymous> (/Users/zach/.node_libraries/redis-node-client/lib/redis-client.js:347:28)
    at IOWatcher.callback (net:303:16)
    at node.js:748:9

Given the environment:

mac os x 10.6.3
node 0.1.91
redis 1.2.6

Thanks,
Zach

subscriber.js erroring out

Using latest from git, after starting publisher, then starting subscriber, I get this:
$ node subscriber.js
waiting for messages...
TypeError: Cannot read property 'length' of undefined
at Client.onReply_ (/web/node/node/redis/redis-node-client/lib/redis-client.js:395:51)
at /web/node/node/redis/redis-node-client/lib/redis-client.js:150:34
at ReplyParser.feed (/web/node/node/redis/redis-node-client/lib/redis-client.js:212:25)
at Stream. (/web/node/node/redis/redis-node-client/lib/redis-client.js:346:28)
at Stream.emit (events:25:26)
at IOWatcher.callback (net:365:18)
at node.js:176:9

node test/test.js -q FAILED (nodejs 0.1.102 + redis 1.3.17 + redis-node 0.3.5)

The result is :

assert:80
throw new assert.AssertionError({
^
AssertionError: "testZINTER" "ERR unknown command 'zinter'"
at /home/ogoudron/fictorial-redis-node-client-30e9f41/test/test.js:121:25
at Client.onReply_ (/home/ogoudron/fictorial-redis-node-client-30e9f41/lib/redis-client.js:400:34)
at /home/ogoudron/fictorial-redis-node-client-30e9f41/lib/redis-client.js:143:30
at ReplyParser.feed (/home/ogoudron/fictorial-redis-node-client-30e9f41/lib/redis-client.js:160:55)
at Stream. (/home/ogoudron/fictorial-redis-node-client-30e9f41/lib/redis-client.js:337:28)
at Stream.emit (events:26:26)
at IOWatcher.callback (net:512:16)
at node.js:265:9

Tested on Ubuntu Desktop 10.04 (VIA C7) and Debian 5.0 (Celeron 220).

Bests regards, Olivier Goudron

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.