Hello there,

I am currently running into a problem with the keys command. During my research, I noticed that keys in config/commands.js are marked as keyless: true. IMO this is wrong, according to the documentation keys requires one argument (pattern). Since the config/commands.js is generated from the output of the command statement and the second value describes the command arity specification which determines the number of arguments.

The keys command has the arity 2, which means the command only accepts one argument and always has the format GET key.

I have adapted the tools/commands.js to include the arity when detecting keyless commands.

Command arity follows a simple pattern:

• positive if command has fixed number of required arguments.
• negative if command has minimum number of required arguments, but may have more.

I have forked the project and will open a pull request as soon as i am sure about my fix :)

Kind Regards
Lukas

Since we only care about getting a single random member of available and we throw away the rest of the array after we're done, these lines could just be replaced with:

return self.connections[available[Math.floor(Math.random()*available.length)]];

... for the same effect. Probably wouldn't make that much diff to perf since it's tiny and only used to get slots, but every little helps (and neater code = yay) :)

i'm unable to call the config command to configure the redis server because (new RedisClustr()).config is not a function

I ran into this exact problem today: gosquared/stash#7

Just wanted to keep a record of this in this repo in case anyone with this same problem can learn about it w/o googling.

Callback is called without any resulting arguments - we should be reconstructing a proper response and potentially respond with an array of errors if there are any.

Although license is in package.json, there's no LICENSE file.

I have an occasional issue which leads to an 'uncaughtException' in the redis-clustr lib; If my redis cluster experiences a state change, like this->

* FAIL message received from dac844f3307ed4d0e638d57edbb26a6c7d22c392 about dec1998f3930fbab0c735783e3f1505aedcfbfb7
# Cluster state changed: fail
# Cluster state changed: ok
* Clear FAIL state for node dec1998f3930fbab0c735783e3f1505aedcfbfb7: master without slots is reachable again.

I end up with an on error event from the redis library when a command is executed; something like this->
SET can't be processed. The connection is already closed.

In which case I'm currently ending the connection and re-creating it; though shortly after (1 or 2 secs) this I will get errors on my multi-exec commands from redis-clustr->

uncaughtException: TypeError: Cannot read property 'address' of undefined at /usr/local/lgbx/lib/node_modules/redis-clustr/src/RedisBatch.js:105:25

i'm using redis version 3.0.2

Should I be doing anything special after the reconnect?

Hi,
I would like to flushall redis in cluster.

let cluster = new RedisClustr({...});
cluster.flushall();

Will the above code call flushall to all nodes in cluster?

Hi I am trying to run a redis client with redis-clustr. However I am getting the following error,
MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 ready listeners added. Use emitter.setMaxListeners() to increase limit
I am using
redis: 2.8.0
redis-clustr: 1.7.0
node version: 10.15.3

trying to run on nodejs4.3 on aws lambda

2016-12-08T20:59:57.703Z	45fb8783-bd89-11e6-9e9a-493023932084	TypeError: e.substr is not a function
    at Command.callback (/var/task/node_modules/redis-cluster/index.js:181:18)
    at RedisClient.return_error (/var/task/node_modules/redis/index.js:559:25)
    at ReplyParser.<anonymous> (/var/task/node_modules/redis/index.js:308:18)
    at emitOne (events.js:77:13)
    at ReplyParser.emit (events.js:169:7)
    at ReplyParser.send_error (/var/task/node_modules/redis/lib/parser/javascript.js:296:10)
    at ReplyParser.execute (/var/task/node_modules/redis/lib/parser/javascript.js:181:22)
    at RedisClient.on_data (/var/task/node_modules/redis/index.js:535:27)
    at Socket.<anonymous> (/var/task/node_modules/redis/index.js:91:14)
    at emitOne (events.js:77:13)

Hello,

I'm using redis-clustr to initialize a clustr connection providing only a server list (host, port).

Packages

• socket.io-redis@^5.2.0
• redis-clustr@^1.7.0

Code

pub = sub = new RedisCluster({
    ...settings.redis
});

this.ws.adapter(adapter({
    key: 'quintessa.io',
    pubClient: pub,
    subClient: sub,
    requestsTimeout: 5000
}));

Redis

My setup is 3 masters with the with the following conf files:

port 6379
cluster-enabled yes
cluster-config-file nodes-6379.conf
cluster-node-timeout 5000
appendonly yes
dbfilename dump-6379.rdb

port 6380
cluster-enabled yes
cluster-config-file nodes-6380.conf
cluster-node-timeout 5000
appendonly yes
dbfilename dump-6380.rdb

port 6381
cluster-enabled yes
cluster-config-file nodes-6381.conf
cluster-node-timeout 5000
appendonly yes
dbfilename dump-6381.rdb

Error

The error does not occur when using a just 2 single connections to one of 3 masters or there is no redis setup at all, jsut a single server...

0|index  | TypeError: pub.send_command is not a function
0|index  |     at Redis.clients (D:\Development\GIT\Quintessa\Quintessa\node_modules\socket.io-redis\index.js:438:9)
0|index  |     at D:\Development\GIT\Quintessa\Quintessa\src\back-end\io\admin-sockets-namespace\events\signout\index.js:15:26
0|index  |     at Socket.<anonymous> (D:\Development\GIT\Quintessa\Quintessa\src\back-end\io\index.js:93:17)
0|index  |     at Socket.emit (events.js:311:20)
0|index  |     at Socket.EventEmitter.emit (domain.js:482:12)
0|index  |     at D:\Development\GIT\Quintessa\Quintessa\node_modules\socket.io\lib\socket.js:528:12
0|index  |     at processTicksAndRejections (internal/process/task_queues.js:79:11)
0|index  |  8256 Tue, 26 May 2020 17:39:46 GMT [MetalDust]: TypeError: pub.send_command is not a function
0|index  |     at Redis.clients (D:\Development\GIT\Quintessa\Quintessa\node_modules\socket.io-redis\index.js:438:9)
0|index  |     at D:\Development\GIT\Quintessa\Quintessa\src\back-end\io\admin-sockets-namespace\events\signout\index.js:15:26
0|index  |     at Socket.<anonymous> (D:\Development\GIT\Quintessa\Quintessa\src\back-end\io\index.js:93:17)
0|index  |     at Socket.emit (events.js:311:20)
0|index  |     at Socket.EventEmitter.emit (domain.js:482:12)
0|index  |     at D:\Development\GIT\Quintessa\Quintessa\node_modules\socket.io\lib\socket.js:528:12
0|index  |     at processTicksAndRejections (internal/process/task_queues.js:79:11)

I tried to investigate with the following code, I hope it helps. It seems like pub/sub between servers doesnt work ?

    pub.send_command = function (...args) {

        const connection = pub.getRandomConnection();
        console.log(...args);
        return connection.send_command.call(connection, ...args);

    };

Output:

0|index  | 17276 Tue, 26 May 2020 17:49:56 GMT [MetalDust]: quintessa.io-request#/admin# {"requestid":"leYPpU","type":0,"rooms":["ezeYMpR-gerpW2ekpWh-sCwKrZMS8tr6"]}

And new error after push.send_command:

0|index  | 17276 Tue, 26 May 2020 17:50:01 GMT [MetalDust]: Error: timeout reached while waiting for clients response
0|index  |     at Timeout._onTimeout (D:\Development\GIT\Quintessa\Quintessa\node_modules\socket.io-redis\index.js:457:48)
0|index  |     at listOnTimeout (internal/timers.js:549:17)
0|index  |     at processTimers (internal/timers.js:492:7)

Specifically, set up some automation on Travis for different versions. There's some existing projects around automatically setting up redis in cluster mode, so I'm sure we can take inspiration.

Would make it much easier to spot differences between redis versions / node versions too.

TIME doesn't have a key, and can run identically on any node.

The code uses the key to select the correct shard:
https://github.com/gosquared/redis-clustr/blob/master/src/RedisClustr.js#L362

But errors out with no key for command: time.

I think instead of erroring here it should just pick a shard at random and execute there. Im not sure if there are other scenarios where commands might be accidentally missing keys and require this error. If so then add a keyless config setting and pick at random (or shard 0) then.

node 6.9.3 on a Mac.

package.json

{
  "name": "Test",
  "version": "1.1.2",
  "dependencies": {
     "redis-clustr" : "1.5.2"
   },
  "devDependencies": {
    "mocha": "3.2.0",
     "chai": "3.5.0"
  }
}

simpleTest.js

var expect = require('chai').expect;
var should = require('chai').should()

var RedisClustr = require('redis-clustr');

var redisClient =  new RedisClustr({
  servers: [
    {
      host: 'localhost',
      port: 6379
    }
  ]
})


describe('Redis tests', function () {

  it('should hmset', function (done) {
    redisClient.set('key','value')
    redisClient.hmset("hashmap" , {"name1": "value1", "name2": "value2"})
    redisClient.setex(["hello",30000,'join ended'])
    done();
  })
})

Test execution

 simple% mocha simpleTest.js


  Redis tests
    ✓ should hmset


  1 passing (14ms)

 simple% redis-cli -h localhost
localhost:6379> get key
(nil)
localhost:6379> get hello
(nil)
localhost:6379> get hashmap
(nil)
localhost:6379>

  redisPub.monitor();
  redisPub.on("monitor", function (time, args, raw_reply) {
    log.debug("REDIS", args.join(" "));
  });

Unhandled exception has occurred: Error
Error: no key for command: monitor
at d:\Work\ninesix\webtriggers\session\node_modules\redis-clustr\src\RedisClustr.js:362:25
at module.exports.RedisClustr.parseArgs (d:\Work\ninesix\webtriggers\session\node_modules\redis-clustr\src\RedisClustr.js:345:11)
at d:\Work\ninesix\webtriggers\session\node_modules\redis-clustr\src\RedisClustr.js:333:12
at runCbs (d:\Work\ninesix\webtriggers\session\node_modules\redis-clustr\src\RedisClustr.js:170:7)
at Command.callback (d:\Work\ninesix\webtriggers\session\node_modules\redis-clustr\src\RedisClustr.js:254:7)
at normal_reply (d:\Work\ninesix\webtriggers\session\node_modules\redis-clustr\node_modules\redis\index.js:721:21)
at RedisClient.return_reply (d:\Work\ninesix\webtriggers\session\node_modules\redis-clustr\node_modules\redis\index.js:819:9)
at JavascriptRedisParser.returnReply (d:\Work\ninesix\webtriggers\session\node_modules\redis-clustr\node_modules\redis\index.js:192:18)
at JavascriptRedisParser.execute (d:\Work\ninesix\webtriggers\session\node_modules\redis-parser\lib\parser.js:538:12)
at Socket. (d:\Work\ninesix\webtriggers\session\node_modules\redis-clustr\node_modules\redis\index.js:274:27)
at emitOne (events.js:96:13)
at Socket.emit (events.js:191:7)
at readableAddChunk (_stream_readable.js:178:18)
at Socket.Readable.push (_stream_readable.js:136:10)
at TCP.onread (net.js:559:20)

Hi,
https://groups.google.com/forum/#!topic/redis-db/FDK59yE9pus
as mentioned in above thread, clusterdown would be sent to client for a transient time, during failover (maybe there are other situations), so, looks clustr better to retry on this error?

I have tested 3 master without any slave, while set 'cluster-require-full-coverage' to 'no', clustr do not catch this kind of error but just send error to the client immediately.

version 1.0.0

Hello,

I'm actually migrating my redis from a standalone server to a cluster made of 3 masters and 3 slaves. Previously I was using the node_redis client, and when migrating to redis-clustr some of my multi-key commands are not supported anymore. For information I've read the documentation about migrating to a redis cluster.

Here are my firsts questions :

• Why are there only 3 multi-key commands supported? By this question I mean, is there a reason why only these 3 multi-key commands are supported?
• Is it possible to add some new multi-key commands to the client?

The thing is I'd like to implement the HMSET method to the client, and I will probably try to do this. Of course the solution of rewriting my code without HMSET is not really conceivable in my case.

Here are my lasts questions :

• Are there some technical restrictions that could prevent me to implement the MHSET command into the client?
• Am I not using this client correctly by thinking it is possible to implement HMSET command, and why?
• Do you have any tips on how to do this?

Thanks !

I am using scan command to get the keys. since keys command is not recommended use in production. but instead of returning keys value it is returning the callback function l below .

function(err, resp) {
if (err && err.message && retries--) {
var msg = err.message;
var ask = msg.substr(0, 4) === 'ASK ';
var moved = !ask && msg.substr(0, 6) === 'MOVED ';

  if (moved || ask) {
    // key has been moved!
    // lets refetch slots from redis to get an up to date allocation
    if (moved) self.getSlots();

    // REQUERY THE NEW ONE (we've got the correct details)
    var addr = err.message.split(' ')[2];
    var saddr = addr.split(':');
    var c = self.getClient(saddr[1], saddr[0], true);
    if (ask) c.send_command('asking', []);
    c[cmd].apply(c, args);
    return;
  }

  if (err.code === 'CLUSTERDOWN' || msg.substr(0, 8) === 'TRYAGAIN') {
    // TRYAGAIN response or cluster down, retry with backoff up to 1280ms
    setTimeout(function() {
      cli[cmd].apply(cli, args);
    }, Math.pow(2, 16 - Math.max(retries, 9)) * 10);
    return;
  }
}

cb(err, resp);

}

When using multiple nodes, you should subscribe to fullReady instead of ready.
That crucial information is missing in documentation

RedisClustr.js:208:34

error: server crash!s.slice(...).map is not a function TypeError: s.slice(...).map is not a function
at Command.callback (...\node_modules\redis-clustr\src\RedisClustr.js:208:34)
at normal_reply (...\node_modules\redis\index.js:714:21)
at RedisClient.return_reply (...\node_modules\redis\index.js:816:9)
at JavascriptRedisParser.Parser.returnReply (...\node_modules\redis\index.js:188:18)
at JavascriptRedisParser.execute (...\node_modules\redis-parser\lib\parser.js:415:12)
at Socket. (...\node_modules\redis\index.js:267:27)
at emitOne (events.js:96:13)
at Socket.emit (events.js:188:7)
at readableAddChunk (_stream_readable.js:172:18)
at Socket.Readable.push (_stream_readable.js:130:10)
at TCP.onread (net.js:542:20)

Hey,

Maybe I'm asking a silly question but I couldn't find the answer in the repo.
Basically I have a 3-node cluster with 3 masters/slaves.

I can connect and perform get/set SOMETIMES.... When I get a "MOVED" error the client immediately crashes.

Is there any good way to handle these errors and simply "redirect" to correct slave/master with available/relevant slots?

Thanks!

Since node-redis update to 2.6 brought redis-parser 2.0.0, we've been seeing a lot of errors like this:

{ [AbortError: Fatal error encountert. Command aborted. It might have been processed.]
  code: 'NR_FATAL',
  command: 'QUIT',
  origin: [ReplyError: Protocol error, got "8" as reply type byte. Please report this.] }

where the "8" there varies but seems to be a whole load of different values.

For the time being I'm going to lock down the redis dependency here to ~2.5.0 until we have time to investigate further and figure out what exactly is going on with the newer redis parser.

Help me:

const client = new RedisClustr({
    servers: [
      {
        host: '127.0.0.1',
        port: 6379
      }
    ]
})
client.get(key, function (err, data) { console.log(err.message) }) // 'couldn't get slot allocation'

I know my local redis didn't support cluster mode. Is there a way to connect normal local redis?

RedisClustr.js:238
if (c.ready) {
^
TypeError: Cannot read property 'ready' of undefined

While digging into #25 I noticed that this line is being hit with interval === 0 for commands with movable keys like ZUNIONSTORE

As a result the command is getting sent to a random node, and the only reason it succeeds is because we're correctly handling the MOVED error response. But that means command duplication and a completely redundant re-fetch of the cluster config for every single one of these commands.

By my reckoning this only affects commands that do take keys but report an interval of 0, namely those that report movablekeys in response to COMMAND. These are: zinterstore zunionstore sort migrate eval and evalsha. We can discount the eval ones here since we handle them specially. Also I think we can ignore migrate since it's not a command you should be issuing against a cluster.

That leaves zinterstore zunionstore and sort - since these are all multi-key commands that only work if all the relevant keys exist on the same node, it should be safe to simply examine the first key given and use that to select a slot (and if you're using it in a situation where redis will complain, then it should error anyway)

I found the redis-clustr is very easy to use than other several node redis cluster libraries. I like it, but I cannot find the "@types/redis-clustr" for my typescript project.

Does not redis-clustr support typescript?

Currently if the slaves option is share then it looks like doing (for example) zrem immediately followed by zcard inside a batch/multi can see the zrem going to a master and the zcard going to a slave, so they don't necessarily occur in the right order.

Ideally a batch should preserve command order, so perhaps if there's any write commands it should override any slaves config and always go to the master.

Hi,

I am running clustr against an AWS ElastiCache cluster (5.0 version). I have one master node and two slaves. Iam using the cluster configuration endpoint to connect to the cluster.

var client = new RedisClustr({
  servers: [
    {
      host: 'xxxxx.jwkwkt.0001.euw1.cache.amazonaws.com'
    }
  ]
});

It is fine to run write task, but when it run read methods like llen, get, then it hangs there.

client.llen(vkey, function (err, sum) { // it is stopped at this stage
console.log("sum..."+sum);
client.set(key, JSON.stringify(i), 'EX', 60); // it can be run if it is not inside of llen method.
});

I also tried elasticache with cluster enabled to have 3 shards and 3 replica for each. It got same issue to run read method.

Same code works when it is not cluster.

Can you guide me please?

If RedisClustr is configured to never send requests to slaves and it connects to a slave first, queued requests will fail with a couldn't get client error. This is easily reproducible with this test:

    it('queues requests', function(done) {
      var r = new RedisClustr({
        servers: hosts,
        slaves: 'never'
      });

      r.ping(function(e) {
        if (e) throw e
        else r.quit(done)
      })
    });

One solution here is to only drain the request queue when fullReady is hit. I've done that here in my branch: segment-boneyard@246ad77

However, this has a few side-effects so I didn't just open this as a PR:

1. getSlots() no longer blocks while the command queue is drained. I can't think of any reason why this would be a problem, though.

2. Commands will be queued until all clients are ready, not just the first one. I believe this is desired behavior anyways because otherwise we're ignoring the maxQueueLength setting and just queueing commands in all the non-ready Redis clients.

If you're interested, I'd be happy to open a PR with my fix above if you want. Just let me know!

How to define a node is a master or slave when initializing ?