Code Monkey home page Code Monkey logo

miniredis's Introduction

Miniredis

Pure Go Redis test server, used in Go unittests.

Sometimes you want to test code which uses Redis, without making it a full-blown integration test. Miniredis implements (parts of) the Redis server, to be used in unittests. It enables a simple, cheap, in-memory, Redis replacement, with a real TCP interface. Think of it as the Redis version of net/http/httptest.

It saves you from using mock code, and since the redis server lives in the test process you can query for values directly, without going through the server stack.

There are no dependencies on external binaries, so you can easily integrate it in automated build processes.

Be sure to import v2:

import "github.com/alicebob/miniredis/v2"

Commands

Implemented commands:

  • Connection (complete)
    • AUTH -- see RequireAuth()
    • ECHO
    • HELLO -- see RequireUserAuth()
    • PING
    • SELECT
    • SWAPDB
    • QUIT
  • Key
    • COPY
    • DEL
    • EXISTS
    • EXPIRE
    • EXPIREAT
    • EXPIRETIME
    • KEYS
    • MOVE
    • PERSIST
    • PEXPIRE
    • PEXPIREAT
    • PEXPIRETIME
    • PTTL
    • RANDOMKEY -- see m.Seed(...)
    • RENAME
    • RENAMENX
    • SCAN
    • TOUCH
    • TTL
    • TYPE
    • UNLINK
  • Transactions (complete)
    • DISCARD
    • EXEC
    • MULTI
    • UNWATCH
    • WATCH
  • Server
    • DBSIZE
    • FLUSHALL
    • FLUSHDB
    • TIME -- returns time.Now() or value set by SetTime()
    • COMMAND -- partly
    • INFO -- partly, returns only "clients" section with one field "connected_clients"
  • String keys (complete)
    • APPEND
    • BITCOUNT
    • BITOP
    • BITPOS
    • DECR
    • DECRBY
    • GET
    • GETBIT
    • GETRANGE
    • GETSET
    • GETDEL
    • GETEX
    • INCR
    • INCRBY
    • INCRBYFLOAT
    • MGET
    • MSET
    • MSETNX
    • PSETEX
    • SET
    • SETBIT
    • SETEX
    • SETNX
    • SETRANGE
    • STRLEN
  • Hash keys (complete)
    • HDEL
    • HEXISTS
    • HGET
    • HGETALL
    • HINCRBY
    • HINCRBYFLOAT
    • HKEYS
    • HLEN
    • HMGET
    • HMSET
    • HRANDFIELD
    • HSET
    • HSETNX
    • HSTRLEN
    • HVALS
    • HSCAN
  • List keys (complete)
    • BLPOP
    • BRPOP
    • BRPOPLPUSH
    • LINDEX
    • LINSERT
    • LLEN
    • LPOP
    • LPUSH
    • LPUSHX
    • LRANGE
    • LREM
    • LSET
    • LTRIM
    • RPOP
    • RPOPLPUSH
    • RPUSH
    • RPUSHX
    • LMOVE
    • BLMOVE
  • Pub/Sub (complete)
    • PSUBSCRIBE
    • PUBLISH
    • PUBSUB
    • PUNSUBSCRIBE
    • SUBSCRIBE
    • UNSUBSCRIBE
  • Set keys (complete)
    • SADD
    • SCARD
    • SDIFF
    • SDIFFSTORE
    • SINTER
    • SINTERSTORE
    • SINTERCARD
    • SISMEMBER
    • SMEMBERS
    • SMISMEMBER
    • SMOVE
    • SPOP -- see m.Seed(...)
    • SRANDMEMBER -- see m.Seed(...)
    • SREM
    • SSCAN
    • SUNION
    • SUNIONSTORE
  • Sorted Set keys (complete)
    • ZADD
    • ZCARD
    • ZCOUNT
    • ZINCRBY
    • ZINTER
    • ZINTERSTORE
    • ZLEXCOUNT
    • ZPOPMIN
    • ZPOPMAX
    • ZRANDMEMBER
    • ZRANGE
    • ZRANGEBYLEX
    • ZRANGEBYSCORE
    • ZRANK
    • ZREM
    • ZREMRANGEBYLEX
    • ZREMRANGEBYRANK
    • ZREMRANGEBYSCORE
    • ZREVRANGE
    • ZREVRANGEBYLEX
    • ZREVRANGEBYSCORE
    • ZREVRANK
    • ZSCORE
    • ZUNION
    • ZUNIONSTORE
    • ZSCAN
  • Stream keys
    • XACK
    • XADD
    • XAUTOCLAIM
    • XCLAIM
    • XDEL
    • XGROUP CREATE
    • XGROUP CREATECONSUMER
    • XGROUP DESTROY
    • XGROUP DELCONSUMER
    • XINFO STREAM -- partly
    • XINFO GROUPS
    • XINFO CONSUMERS -- partly
    • XLEN
    • XRANGE
    • XREAD
    • XREADGROUP
    • XREVRANGE
    • XPENDING
    • XTRIM
  • Scripting
    • EVAL
    • EVALSHA
    • SCRIPT LOAD
    • SCRIPT EXISTS
    • SCRIPT FLUSH
  • GEO
    • GEOADD
    • GEODIST
    • GEOHASH
    • GEOPOS
    • GEORADIUS
    • GEORADIUS_RO
    • GEORADIUSBYMEMBER
    • GEORADIUSBYMEMBER_RO
  • Cluster
    • CLUSTER SLOTS
    • CLUSTER KEYSLOT
    • CLUSTER NODES
  • HyperLogLog (complete)
    • PFADD
    • PFCOUNT
    • PFMERGE

TTLs, key expiration, and time

Since miniredis is intended to be used in unittests TTLs don't decrease automatically. You can use TTL() to get the TTL (as a time.Duration) of a key. It will return 0 when no TTL is set.

m.FastForward(d) can be used to decrement all TTLs. All TTLs which become <= 0 will be removed.

EXPIREAT and PEXPIREAT values will be converted to a duration. For that you can either set m.SetTime(t) to use that time as the base for the (P)EXPIREAT conversion, or don't call SetTime(), in which case time.Now() will be used.

SetTime() also sets the value returned by TIME, which defaults to time.Now(). It is not updated by FastForward, only by SetTime.

Randomness and Seed()

Miniredis will use math/rand's global RNG for randomness unless a seed is provided by calling m.Seed(...). If a seed is provided, then miniredis will use its own RNG based on that seed.

Commands which use randomness are: RANDOMKEY, SPOP, and SRANDMEMBER.

Example

import (
    ...
    "github.com/alicebob/miniredis/v2"
    ...
)

func TestSomething(t *testing.T) {
	s := miniredis.RunT(t)

	// Optionally set some keys your code expects:
	s.Set("foo", "bar")
	s.HSet("some", "other", "key")

	// Run your code and see if it behaves.
	// An example using the redigo library from "github.com/gomodule/redigo/redis":
	c, err := redis.Dial("tcp", s.Addr())
	_, err = c.Do("SET", "foo", "bar")

	// Optionally check values in redis...
	if got, err := s.Get("foo"); err != nil || got != "bar" {
		t.Error("'foo' has the wrong value")
	}
	// ... or use a helper for that:
	s.CheckGet(t, "foo", "bar")

	// TTL and expiration:
	s.Set("foo", "bar")
	s.SetTTL("foo", 10*time.Second)
	s.FastForward(11 * time.Second)
	if s.Exists("foo") {
		t.Fatal("'foo' should not have existed anymore")
	}
}

Not supported

Commands which will probably not be implemented:

  • CLUSTER (all)
    • CLUSTER *
    • READONLY
    • READWRITE
  • Key
    • DUMP
    • MIGRATE
    • OBJECT
    • RESTORE
    • WAIT
  • Scripting
    • SCRIPT DEBUG
    • SCRIPT KILL
  • Server
    • BGSAVE
    • BGWRITEAOF
    • CLIENT *
    • CONFIG *
    • DEBUG *
    • LASTSAVE
    • MONITOR
    • ROLE
    • SAVE
    • SHUTDOWN
    • SLAVEOF
    • SLOWLOG
    • SYNC

&c.

Integration tests are run against Redis 7.2.4. The ./integration subdir compares miniredis against a real redis instance.

The Redis 6 RESP3 protocol is supported. If there are problems, please open an issue.

If you want to test Redis Sentinel have a look at minisentinel.

A changelog is kept at CHANGELOG.md.

Go Reference

miniredis's People

Contributors

al2klimov avatar alicebob avatar asalle avatar carlgreen avatar chowchow316 avatar cleroux avatar davars avatar dim avatar dntj avatar johejo avatar joseotoro avatar lsgndln avatar mangchiandjjoe avatar matiasinsaurralde avatar nathan-cormier avatar ntaylor-barnett avatar pje avatar pkierski avatar propan avatar readams avatar robstein avatar sejin-p avatar skateinmars avatar sklinkert avatar vk-outreach avatar wayneashleyberry avatar wszaranski avatar yfei1 avatar yves-tutti avatar zsh1995 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

miniredis's Issues

Does not build with Go 1.8?

I was trying to build it with Go 1.8.3, and always got the following errors:

../../_vendor/src/github.com/alicebob/miniredis/test.go:10: tb.Helper undefined (type testing.TB has no field or method Helper) ../../_vendor/src/github.com/alicebob/miniredis/test.go:19: tb.Helper undefined (type testing.TB has no field or method Helper) ../../_vendor/src/github.com/alicebob/miniredis/test.go:28: tb.Helper undefined (type testing.TB has no field or method Helper) ../../_vendor/src/github.com/alicebob/miniredis/test.go:37: tb.Helper undefined (type testing.TB has no field or method Helper)

I found this issue, which seems related.

SORT command not supported

Hi,
I am trying to write unit tests for our system using miniredis. Our system utilises SORT command of redis extensively. But miniredis doesn't support SORT command yet. I would like to request the addition of SORT command support in the library.
Regards

ZADD unable to produce error

Hello,
I can't seem to produce an error after using ZADD for my unit test. I tried to SET a key and then use the same key with ZADD, which works when i tried it in redis, giving WRONGTYPE Operation against a key holding the wrong kind of value error. However, it doesn't give an error when i did the same with miniredis.

I believe there is something wrong with this condition because i don't think there's a way to trigger both db.exists(k) and !db.exists(k) at the same time. Can you please check it?

Thanks

Incorrect behavior for ZRANGEBYLEX

Hi there, I ran into a scenario where ZRANGEBYLEX seems to handle the Min field incorrectly, at least that's what I suspect.

I have reproduced it using github.com/alicebob/miniredis/v2 v2.9.1-0.20190731113429-5da194369a10, with both redigo and go-redis.

The test cases are here: https://gist.github.com/shazow/2c9355d31d41cc0e654d374aed0e69bc

To summarize, this scenario returns "ccc" incorrectly for the second command:

redis> ZADD idx 0 "ccc"
(integer) 1
redis> ZRANGEBYLEX idx "[d" "[e"
(empty list or set)
redis> ZRANGEBYLEX idx "[c" "[d"
1) "ccc"

lua cjson.decode("null") returns nil instead of cjson.null

In production redis, it looks like cjson.decode("null") returns cjson.null whereas the lua cjson in miniredis returns nil.

Maybe gopher-lua should be updated to match redis? The change involved seems a bit deep though (requires changing gopher-lua to return a special sentinel value and making cjson.null also return that sentinel value).

max/min filtering in ZREVRANGEBYSCORE broken

Hello there. I checked this project and found it cool. So I wanted to use it in one of my pet projects. There I had some tests that now fail when running against the miniredis implementation. For reference, when playing around with the ZREVRANGEBYSCORE command on http://redis.io/commands/zrevrangebyscore you can do things like this. We add an element with an specific score and fetch the element with this specific score in the max and min setting. Note that max is the first, and min the second score identifier.

Here the interface specification.

ZREVRANGEBYSCORE key max min [WITHSCORES] [LIMIT offset count]

Here the test on the website.

redis> zadd key 3.3 element
(integer) 1
redis> zrevrangebyscore key 3.3 3.3
1) "element"
redis> zrevrangebyscore key 4.3 4.3
(empty list or set)
redis> 

Here the test using docker containers to make sure the redis website is not buggy. So run the redis server in one terminal.

docker run --rm --name redis -p 6379:6379 redis
1:C 09 Oct 21:43:55.441 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf
                _._
           _.-``__ ''-._
      _.-``    `.  `_.  ''-._           Redis 3.2.3 (00000000/0) 64 bit
...

And run the redis-cli in another terminal.

docker run -it --rm --link redis:redis redis bash -c 'redis-cli -h redis'
redis:6379> zadd key 3.3 element
(integer) 1
redis:6379> zrevrangebyscore key 3.3 3.3
1) "element"
redis:6379> zrevrangebyscore key 4.3 4.3
(empty list or set)
redis:6379>

Here is the go program that fails with the same commands using miniredis.

package main

import (
    "fmt"

    "github.com/alicebob/miniredis"
    "github.com/garyburd/redigo/redis"
)

func main() {
    s, err := miniredis.Run()
    if err != nil {
        panic(err)
    }
    defer s.Close()

    c, err := redis.Dial("tcp", s.Addr())
    if err != nil {
        panic(err)
    }
    _, err = c.Do("ZADD", "key", 3.3, "element")
    if err != nil {
        panic(err)
    }
    res, err := redis.Strings(c.Do("ZREVRANGEBYSCORE", "key", 4.3, 4.3))
    if err != nil {
        panic(err)
    }
    fmt.Printf("%#v\n", res)
}

The result of the go program is []string{"element"}, which is wrong, because there is no element in the set having score 4.3. I checked the code but it is a little bit tricky for me to comprehend what is going wrong and where to fix this.

SINTER results are incorrect when an input key does not exist

Scenario:
Intersection of sets where one key doesn't not exist.

From the SINTER docs:

Keys that do not exist are considered to be empty sets. With one of the keys being an empty set, the resulting set is also empty (since set intersection with an empty set always results in an empty set).

Expectation:
The result should be an empty set:

127.0.0.1:6379> SADD s1 aap noot mies
(integer) 3
127.0.0.1:6379> SINTER s2 s9
(empty list or set)

Actual:

Miniredis returns the contents of s1.

--- FAIL: TestSinter (0.00s)
    cmd_set_test.go:514: expected: []string{} got: []string{"aap", "noot", "mies"}

[feature] UNLINK command not supported

Hey there,

since redis 4.0.0 the project supports UNLINK (which is basically async DEL), detailed decription here.

Do you think it's going to be possible to add this command? How can I help?

Asa

Go mod cannot get release v2.10.0

I am using go 1.13 and go mod. Currently in go.mod the version stays in the version below.
github.com/alicebob/miniredis v2.5.0+incompatible

When I tried to force it to use v2.10.0 by go mod edit -require github.com/alicebob/[email protected]. It says require github.com/alicebob/miniredis: version "v2.10.0" invalid: module contains a go.mod file, so major version must be compatible: should be v0 or v1, not v2

[bug] LUA Eval is executing scripts in parallel. creates race condition absent in redis.

Redis executes LUA scripts in a transactional manor in a shard. (when the KEYS hash slots are in the same shard)
miniredis seems to be executing LUA scripts outside of a lock which caused some unreliable tests for me. I have a bit of LUA that depends on the transactional nature of it's execution.

This fixes the issue:
Adding scriptLock sync.Mutex to miniredis.go

// Miniredis is a Redis server implementation.
type Miniredis struct {
	sync.Mutex
	srv        *server.Server
	port       int
	password   string
	dbs        map[int]*RedisDB
	selectedDB int               // DB id used in the direct Get(), Set() &c.
	scripts    map[string]string // sha1 -> lua src
	signal     *sync.Cond
	now        time.Time // used to make a duration from EXPIREAT. time.Now() if not set.
	
	scriptLock sync.Mutex
}

And locking LUA execution in cmd_scripting.go:25 runLuaScript

// Execute lua. Needs to run m.Lock()ed, from within withTx().
func (m *Miniredis) runLuaScript(c *server.Peer, script string, args []string) {

	// ...
	l.Push(lua.LString("redis"))
	l.Call(1, 0)

	m.Unlock() // This runs in a transaction, but can access our db recursively
	defer m.Lock()
	
	// **********  NEW LUA LOCK  **************
	m.scriptLock.Lock()
	defer m.scriptLock.Unlock()

	if err := l.DoString(script); err != nil {
		c.WriteError(errLuaParseError(err))
		return
	}

	luaToRedis(l, c, l.Get(1))
}

I wont pretend that this solution is clean enough to be 'the' solution, but its working for me so far and showcases the issue for someone with more experience in the miniredis codebase to make a sufficiently elegant solution.

If you want, I could provide a test case for the issue (and a PR with my current solution, if you want it).

I'm finding miniredis vitally useful for testing my project.
My thanks to the creators and maintainers of this project!
:)

Let me know if I can assist in resolving this issue further.

Build constraints exclude all Go files in..

I get this error when importing it:

Build constraints exclude all Go files in '/home/user/go/pkg/mod/github.com/alicebob/miniredis/[email protected]'

I cannot use this package at all. go env:

GOARCH="amd64"
GOBIN=""
GOCACHE="/home/user/.cache/go-build"
GOEXE=""
GOFLAGS=" -mod="
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/user/go"
GOPROXY="direct"
GORACE=""
GOROOT="/home/user/go/go1.12.1.linux-amd64"
GOTMPDIR=""
GOTOOLDIR="/home/user/go/go1.12.1.linux-amd64/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/user/Projects/personal/dns/worker/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build691784442=/tmp/go-build -gno-record-gcc-switches"

Any ideas?

Possibly incorrect response when transactions are aborted

The code for aborting transactions reports a length of 0 during an abort as follows:

for t, version := range ctx.watch {
if m.db(t.db).keyVersion[t.key] > version {
// Abort! Abort!
stopTx(ctx)
c.WriteLen(0)
return
}
}

The redis-go/redis library, however, checks for a Null response to recognize that the transaction was aborted (https://github.com/go-redis/redis/blob/6bc7daa5b1e86745a6976ac1c4dfe6c76ea6af1f/redis.go#L425-L442)

	// Parse number of replies.
	line, err := rd.ReadLine()
	if err != nil {
		if err == Nil {
			err = TxFailedErr
		}
		return err
	}

	switch line[0] {
	case proto.ErrorReply:
		return proto.ParseErrorReply(line)
	case proto.ArrayReply:
		// ok
	default:
		err := fmt.Errorf("redis: expected '*', but got line %q", line)
		return err
	}

(Indeed, it will attempt to read a response for every command regardless of the returned length. This causes a read timeout which is how I wound up finding all this. Perhaps that's an issue for that library as well.)

The redis documentation for the EXEC command (https://redis.io/commands/exec) claims

When using WATCH, EXEC can return a Null reply if the execution was aborted.

which leads me to believe that Null is the appropriate response here, and in my testing, redis itself does report a Null. Should the server be changed to report a Null instead of 0?

I wanted to open an issue for discussion because from what I can tell, the current response has been the same from it's initial introduction ~4-5 years ago.

Simulate server unavailable?

I have a health check that pings redis to make sure it's alive. How do I configure or mock miniredis to pretend that is unavailable/unresponsive? Also specifying a timeout on the request?

Miniredis vs redis.io commands name mismatch

Hey there,

is there a reason why some set(datastructure)-related commands in miniredis are called something different from the official redis command list? (https://redis.io/commands)
For example:
redis.io minireids
SADD -> SetAdd
SMEMBERS -> Members
whereas SREM and SRem are in sync.

As a result, it takes some time to figure out which command is which, at least for me.
Also, thank you for this amazing project! ❤️

Stream XREAD support

Hi, thanks for maintainers.

is whether Stream support XREAD operations?

According the syntax, it seem not support yet.

// commandsStream handles all stream operations.
func commandsStream(m *Miniredis) {
	m.srv.Register("XADD", m.cmdXadd)
	m.srv.Register("XLEN", m.cmdXlen)
	m.srv.Register("XRANGE", m.makeCmdXrange(false))
	m.srv.Register("XREVRANGE", m.makeCmdXrange(true))
}

Lua scripts not executing atomically

Thanks for this library, it was very helpful during the development of CurlyQ.

I've run into an issue when running Lua scripts alongside blocking commands. I'm running against v2.11.1 of the library. My expectation according to the Redis spec is that Lua scripts should always execute atomically: if I push to a list that is the target of a blocking command inside of a Lua script, the Lua script should finish executing its remaining commands before the blocking operation proceeds. However, when using miniredis, it appears that the blocked operation will proceed immediately without waiting for the script to finish.

A quick repro:

package main

import (
	"log"
	"time"

	"github.com/alicebob/miniredis/v2"
	"github.com/go-redis/redis/v7"
)

func main() {
	server, _ := miniredis.Run()
	client := redis.NewClient(&redis.Options{
		Addr: server.Addr(),
	})

	go func() {
		client.BRPopLPush("signal", "done", 100 * time.Second).Err()
	}()

	script := redis.NewScript(`
		redis.call("LPUSH", "signal", "a")
		redis.call("LPUSH", "done", "b")
		return 1
	`)
	script.Run(client, nil).Err()

	completionOrder, _ := client.LRange("done", 0, -1).Result()
	log.Println(completionOrder)
}

This results in output [b a]. The expected output, and the result I get if I run the same test against a live Redis server, is [a b]. Converting the script to a multi/exec produces the correct output. Any thoughts on where I might start looking if I wanted to have a go at fixing this?

Missing package error - "github.com/alicebob/miniredis/v2/server"

The package fails to build with missing package error -

cmd_connection.go:8:2: cannot find package "github.com/alicebob/miniredis/v2/server" in any of:
/usr/local/go/src/vendor/github.com/alicebob/miniredis/v2/server (vendor tree)
/usr/local/go/src/github.com/alicebob/miniredis/v2/server (from $GOROOT)
/home/ibmadmin/git/veena-dev/go/src/github.com/alicebob/miniredis/v2/server (from $GOPATH)

Expire not working correctly

I'm working with miniredis inside my unit tests. And in my code I try to expire a key with a given TTL. (using github.com/mediocregopher/radix.v2) library for redis. But the expire won't work. When I do it using SetTTL it does work. But when calling it from the redis library nothing ever happens. Any suggestions on how to fix this ?

Error happens when SET big size bytes in miniredis.

When using github.com/gomodule/redigo/redis to store a byte array whose length exceeds 10K, the following error happen:
write tcp 127.0.0.1:60940->127.0.0.1:40545: write: connection reset by peer
Does miniredis has a limit on the size of the value?

FYI: Error message changed in redis between 5.0.3 and 5.0.7

stream_test.go:10: error error. expected: "ERR The ID specified in XADD must be greater than 0-0" got: "ERR The ID specified in XADD is equal or smaller than the target stream top item" case: main.command{cmd:"XADD", args:[]interface {}{"newplanets", "0", "foo", "bar"}, error:true, sort:false, loosely:false, errorSub:"", receiveOnly:false, roundFloats:0, closeChan:false, noResultCheck:false}

Executing Eval in transaction results in error

There seems to be a problem when multiple commands need to be run in transaction, and one of these is eval or evalsha. I receive an error in the following format: "redis: can't parse reply="*4" reading string".
Running the same test against a local redis works fine.

func TestMiniRedis(t *testing.T) {
	s, err := miniredis.Run()
	if err != nil {
		t.Fatal(err.Error())
	}
	client := redis.NewClient(&redis.Options{
		Addr: s.Addr(),
	})
	sha, e := client.ScriptLoad("return {KEYS[1],KEYS[2],ARGV[1],ARGV[2]}").Result()
	if e != nil {
		t.Fatal(e.Error())
	}
	r, e := client.EvalSha(sha, []string{"k1", "k2"}, "first", "second").Result()
	if e != nil {
		t.Fatal(e.Error())
	}
	fmt.Println(r) //result as expected
	_, e = client.TxPipelined(func(pipe redis.Pipeliner) error {
		pipe.Set("x", "123", 0) //not really needed, the test will fail and without it
		pipe.EvalSha(sha, []string{"k1", "k2"}, "first", "second")
		_, e := pipe.Exec()
		return e
	})
	if e != nil {
		t.Fatal(e.Error())
	}
}

undefined: redis.Int64s

With:
go 1.9.4 x86_64
github.com/alicebob/miniredis 2.3.2

Hello,
I try to build and test your project and it fails with :

FAIL github.com/alicebob/miniredis [build failed]

github.com/alicebob/miniredis

./cmd_scripting_test.go:107:13: undefined: redis.Int64s
./cmd_scripting_test.go:119:13: undefined: redis.Int64s
./cmd_scripting_test.go:125:13: undefined: redis.Int64s

Could you help me please ?

Thanks

Server fails processing bulk strings larger than 4k

Attempting to rpush a string larger than 4k fails in server/proto.go/readString() since the buffer used by bufio.Read() gets full. No additional attempts to read remaining data are made and the function returns an invalid request error.

The bufio.Reader is created in server/server.go/servePeer() and defaults to 4k in buffer size.

HSET doesn't support multiple field/value pairs

Hi,

As per https://redis.io/commands/hset HSET now allows multiple field/value pairs, which unfortunately miniredis does not.

cmdHset() in cmd_hash.go has a hard-coded check that args array contains three items (key, field, and value) and hashSet() in db.go would need to be a variadic function. I'm happy to have a go at implementing this if that would help.

prepend v to semver

Hello
The go module does not find this package because v is missing in front.

go get github.com/alicebob/miniredis
go: finding github.com/alicebob/miniredis latest
go get github.com/alicebob/miniredis: no matching versions for query "latest"

BRPOPLPUSH never returns if key doesn't exist when it was called

To reproduce:

import (
    "fmt"
    "github.com/alicebob/miniredis"
    "github.com/go-redis/redis"
    "time"
)

var addr string

func main() {
    fakeredis, err := miniredis.Run()
    if err != nil {
        panic(err)
    }

    addr = fakeredis.Addr()

    fakeredis.Lpush("queue1", "already existing value")

    go readVals()

    time.Sleep(1 * time.Second)

    fakeredis.Lpush("queue1", "new value")

    for {
    }
}

func readVals() {
    rdb := redis.NewClient(&redis.Options{Addr: addr})
    rdb.BRPopLPush("queue1", "queue2", 0)
    fmt.Println("This works OK")
    rdb.BRPopLPush("queue1", "queue2", 0)
    fmt.Println("Never gets here :-(")
}

readVals calls BRPopLPush which returns "already existing value", then calls BRPopLPush again, which blocks because the key is now empty. One second later in main another value is pushed. I expect BRPopLPush to return a second time but it does not.

Is there any support for the redis pipelining ?

Hi all,
I'm currently trying to write unit tests for my redis integration module which does something like

pipe := redisConn.Pipeline()
pipe.IncrBy(val1, val2)
pipe.Get(val3)
results, err := pipe.Exec()

Would it be possible to mock this pipelining with miniredis ? The Pipeline() I'm using is this one

ERR unknown command 'brpop'

This issue is rather a question. I see commands like brpop are not implemented. Can we have a discussion about this? Why are such commands not implemented? One of my pet projects makes use of brpop. Now I added miniredis to test my storage implementations. It would be supreme to have support for these commands. Cheers.

Error compiling script

  1. I start redis server:
redisServer, err = miniredis.Run()
redisServer.RequireAuth(redisPassword)
  1. Create a connection:
conn, err := redigo.Dial("tcp", addr, redigo.DialPassword(redisPassword),
					redigo.DialConnectTimeout(5*time.Second))
  1. Create a stored procedure:
const getScript = `
return redis.call("GET",KEYS[1])
`
storedProcGet = redis.NewScript(1, getScript)
  1. And when I execute it:
    _, err = storedProcGet.Do(conn, "key")

I get:

ERR Error compiling script (new function): <string>:2: NOAUTH Authentication required. stack traceback: [G]: in function 'call' <

However if I disable authentication or use real Redis, everything is fine.

Could you please look at that?

adding support for GEOADD

Hi,

I have a project that requires GEOADD operation as well as GEOPOS and GEORADIUS. Do you plan to implements those commands for miniredis ? I saw that it was in the "probably not going to be implemented" but I was asking just to be sure :-)

Use an internal random number source rather than math/rand directly

Commands like SPOP use rand.Intn directly, with the advised method of controlling the selection being to call rand.Seed. This means that any tests using miniredis which could normally be run in parallel cannot, as the rand package's seed is global and shared within the process.

Could a function be added to provide a seed, which could then be used internally to create a new rand.Rand? For backwards compatibility, it could default to the global functions in rand, but switch if a seed was provided. Something like:

func (m *Miniredis) Seed(seed int) {
    // ...
}

Note that rand.NewSource (needed to call rand.New) does not return a goroutine-safe Source; it needs to have locking to ensure Rand's accesses don't break things. (The stdlib has an unexported type for this.)

EXPIRE not working with lua script

I'm running the following lua script as a ratelimiter by IP address

local limit = tonumber(ARGV[1])
local ttl = tonumber(ARGV[2])
local k = "requests:" .. KEYS[1]

local current = redis.call("LLEN", k)
if current >= limit then
    return redis.error_reply("too many requests")
else
    if redis.call("EXISTS", k) == 0 then
	redis.call("RPUSH", k, 1)
	redis.call("EXPIRE", k, ttl)
    else
	redis.call("RPUSHX", k, 1)
    end
end

using redigo, I execute scpt.Do(con, "127.0.0.1", 1, 1) which should create a new key at "requests:127.0.0.1" with a TTL of 1. It appears to do this much, however the key never seems to expire. Whenever I make a second call with the exact same params I'm hitting too many requests which should not be the case since the key should have expired. I've tried to add time.Sleep(time.Second) in my test, however no matter how long it sleeps the key never expires.

If I switch to a live redis instance on my local machine my tests pass.

Here is a simple test to show the issue

package main

import (
	"fmt"
	"time"

	"github.com/alicebob/miniredis"
	"github.com/garyburd/redigo/redis"
)

var scpt = redis.NewScript(1, `
local limit = tonumber(ARGV[1])
local ttl = tonumber(ARGV[2])
local k = "requests:" .. KEYS[1]

local current = redis.call("LLEN", k)
if current >= limit then
    return redis.error_reply("too many requests")
else
    if redis.call("EXISTS", k) == 1 then
	redis.call("RPUSHX", k, 1)
    else
	redis.call("RPUSH", k, 1)
	redis.call("EXPIRE", k, ttl)
    end
end
`)

func main() {
	srv, _ := miniredis.Run()
	defer srv.Close()

	con, err := redis.DialURL(fmt.Sprintf("redis://%s", srv.Addr()))
	if err != nil {
		panic(err)
	}
	defer con.Close()

	_, err = scpt.Do(con, "127.0.0.1", 1, 1)
	if err != nil {
		panic(err)
	}
	// TTL is one second so sleep and wait for it to expire
	time.Sleep(time.Second)

	_, err = scpt.Do(con, "127.0.0.1", 1, 1)
	if err != nil {
		// this should not happen because the key should have expired
		panic(err)
	}
}

auto expire support

question

Why not add auto expire support? I add some code for this feature~

func (db *RedisDB) autoExpire() {
	checkDuration := time.Millisecond * 200
	tick := time.NewTicker(checkDuration)
	for {
		select {
		case <-tick.C:
			db.fastForward(checkDuration)
		case <-db.done:
			return
		}
	}
}
func (db *RedisDB) close() {
	db.done <- struct{}{}
}
func newRedisDB(id int, l *sync.Mutex) RedisDB {
	db := RedisDB{
		id:            id,
		master:        l,
		keys:          map[string]string{},
		stringKeys:    map[string]string{},
		hashKeys:      map[string]hashKey{},
		listKeys:      map[string]listKey{},
		setKeys:       map[string]setKey{},
		sortedsetKeys: map[string]sortedSet{},
		ttl:           map[string]time.Duration{},
		keyVersion:    map[string]uint{},
		done:          make(chan struct{}, 0),
	}
	// auto expire
	go db.autoExpire()
	return db
}
// Close shuts down a Miniredis.
func (m *Miniredis) Close() {
	m.Lock()

	if m.srv == nil {
		m.Unlock()
		return
	}
	srv := m.srv
	m.srv = nil
	for _, v := range m.dbs {
		v.close()
	}
	m.Unlock()

	// the OnDisconnect callbacks can lock m, so run Close() outside the lock.
	srv.Close()

}

Its works~

Can't store values with '\n' character

There is an issue where you can't store any values that contain a \n character. This is causing some problems for me as I'm trying to marshal proto into redis (which separates values with a \n char). It works on a live redis server but not in miniredis. Here's a simple test I just wrote up:

func TestNewLine(t *testing.T) {
	redisServer := setup()
	defer teardown(redisServer)

	// Query to check that it exists in DB
	redisClient := redis.NewClient(&redis.Options{
		Addr:     redisServer.Addr(),
		Password: "", // no password set
		DB:       0,  // use default DB
	})

	err := redisClient.ZAdd("effects", redis.Z{
		Score:  float64(0),
		Member: "value",
	}).Err()
	if err != nil {
		t.Errorf("got unexpected error: %v", err)
	}

	err = redisClient.ZAdd("effects", redis.Z{
		Score:  float64(0),
		Member: "value2\n",
	}).Err()
	if err != nil {
		t.Errorf("got unexpected error: %v", err)
	}

	keys, _ := redisClient.ZScan("effects", 0, "*", 0).Val()
	fmt.Println(keys)
	if len(keys) != 4 {
		t.Errorf("got %v, want %v", len(keys), 4)
	}
}

Output:

[value 0]
--- FAIL: TestNewLine (0.00s)
    /Users/zachary/Documents/code/github.com/terrariumai/simulation/pkg/datacom/redis_test.go:776: got 2, want 4
FAIL
FAIL	github.com/terrariumai/simulation/pkg/datacom	0.033s
Error: Tests failed.

Looking at the print out, only the first ZAdd is working. Whats more interesting is that the second doesn't error out. The value is just not getting added.

Contribute with GEORADIUSBYMEMBER

Hi!

First of all, thank you for a great package!

I would like GEORADIUSBYMEMBER for a hobby project. Can I try to implement it and create a PR for it?

Best regards,
Magnus

[feature] support streams

Redis streams have been available in 5.0 release candidates for some time (correction: 5.0.0 has officially been released). antirez recently greatly expanded documentation on streams. github.com/go-redis/redis has full stream command support.

TL;DR: streams are upon us

Can stream support be added to the roadmap?

TCP timeout when executing Select() in TxPipeline

I'm getting read TCP timeout error when executing Select(db) method on TxPipeline() instance. The issue occurs when I'm using miniredis with go-redis client. Example code:

	mr, err := miniredis.Run()
	if err != nil {
		panic(err)
	}
	defer mr.Close()

	redisClient := redis.NewClient(&redis.Options{
		Addr: mr.Addr(),
		DB:   1,
	})

	redisClient.Incr("someKey")

	pipe := redisClient.TxPipeline()
	pipe.Select(1)
	pipe.Get("someKey")

	if _, err := pipe.Exec(); err != nil {
		fmt.Println(err)
	}

Output:

read tcp 127.0.0.1:52258->127.0.0.1:52257: i/o timeout

The strange part is that when I'm executing Get() on non-existing key, I'm not getting this error, it simply returns nil in that case (as expected). Also, the code works as expected when I'm using Pipeline() instead of TxPipeline().

Lua Eval fails to process integers that are larger than 6 digits when used with redis.call

redis.call('expire', myKey, 1000000) will cause an error, "ERR Error compiling script (new function): :31: ERR value is not an integer or out of range"
redis.call('expire', myKey, 999999) will work without an issue
Reproduced in version 2.7.0
Edit:
The bug does not reproduce when working with a real redis server, so its only a mock server issue.
found a workaround, redis.call('expire', myKey, '1000000'), using a string instead of integer will work.

Update go.mod file

Current go.mod file is using redigo version v2.0.0, which makes projects using go modules unable to use the latest redigo version because of the version conflict. It would be great if miniredis can support the bleeding edge redigo version, especially the one that with DialContext.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.