me-io / memcached-util Goto Github PK

I'm on Ubuntu and getting this, any ideas? <

~/memcached-util$ make
building: bin/linux-amd64/memcached-util
Unable to find image 'golang:1.11-alpine' locally
1.11-alpine: Pulling from library/golang
9d48c3bd43c5: Pull complete
7f94eaf8af20: Pull complete
9fe9984849c1: Pull complete
ec448270508e: Pull complete
65ba82af53f7: Pull complete
Digest: sha256:09e47edb668c2cac8c0bbce113f2f72c97b1555d70546dff569c8b9b27fcebd3
Status: Downloaded newer image for golang:1.11-alpine
cmd/util/main.go:6:2: cannot find package "github.com/op/go-logging" in any of:
	/usr/local/go/src/github.com/op/go-logging (from $GOROOT)
	/go/src/github.com/op/go-logging (from $GOPATH)
Makefile:78: recipe for target 'bin/linux-amd64/memcached-util' failed
make: *** [bin/linux-amd64/memcached-util] Error 1

hi
i tried this tool in localhost it works fine. but I try to backup my Elasti-cache memcached cluster. it hanged with step 2

Hello,

I am trying to restore a dump of a memcached 1.5.22 server. The dump works perfectly but the restore just gets stuck and does nothing as you can see below:

$ ./memcached-util --addr 10.0.0.101:11211 --op "restore" --filename "mem_backup.json"
2021-07-15T15:18:28.516256 main ▶ INFO 001 address 10.0.0.101:11211
2021-07-15T15:18:29.206551 restoreCache ▶ WARNING 002 Key  already expired, skipping ..

On the server side with verbose mode in memcached enabled I see the following:

Jul 15 15:18:28 server systemd-memcached-wrapper[705468]: <27 new auto-negotiating client connection
Jul 15 15:18:29 server systemd-memcached-wrapper[705468]: 27: Client using the ascii protocol
Jul 15 15:18:29 server systemd-memcached-wrapper[705468]: <27 set  0 -553 0
Jul 15 15:18:29 server systemd-memcached-wrapper[705468]: >27 ERROR
Jul 15 15:18:29 server systemd-memcached-wrapper[705468]: <27
Jul 15 15:18:29 server systemd-memcached-wrapper[705468]: >27 ERROR

The backup itself seem to have worked correctly:

2021-07-15T15:03:33.822166 main ▶ INFO 001 address 127.0.0.1:11211
2021-07-15T15:03:34.351171 backupCache ▶ INFO 002 87676 values found in the storage
2021-07-15T15:03:38.374375 backupCache ▶ INFO 003 Output file successfully generated at: mem_backup.json

Any idea what's wrong?

================= How to reproduce ? ============

Prerequesite: A memcached up & running with 0 key

1. Save a TTL infinite data in memcached
   telnet localhost 11211
   set key1 0 0 9
   memcached

2. dump the content using memcached-util ( If you open the file generated, you will notice that this key is not present in JSON generated file)

3. Flush data in memcached
   telnet localhost 11211
   flush_all

4. Restore the content using memcahed-util

====================== Expected behaviour ==========
Data restored in memcached.

====================== What I got ================
Key key1 already expired, skipping

==================== Environment ==========

======= Operating system ====
Distributor ID: Ubuntu
Description: Ubuntu 18.04.1 LTS
Release: 18.04
Codename: bionic

======= Memcached version ====
1.5.12

==================== Possible workaround ===========
To restore with TTL infinite key do:

cat /tmp/memcached-backup | jq "map ( if .expiry then . + {expiry:$(date +%s -d "+1 days")} else . + {expiry:$(date +%s -d "+1 days")} end)" | jq -c "" >> /tmp/memcached-restore | memcached-util --op restore --filename /tmp/memcached-backup2

before restoring ( restore from memcached-backup2)

When I use this tool to dump memcache,it will lock my instance?

Backup operation does not backs up all values.

Test was performed using memcached version 1.6.9, memory limit was set to 4096 MB

Memcached server data was populated using the following python script:

from pymemcache.client.base import Client

def main():
    print("it works")
    client = Client("127.0.0.1")
    for i in range(10000000):
        new_key = 'z_' + str(i)
        client.set(new_key, i, 7200)


if __name__ == "__main__":
    main()

As a result 9999999 key values where set. With 7200 seconds TTL. Same test was also performed with no TTL it produced similar result.

Than I performed data dump using the flowing command:

./memcached-util --addr "127.0.0.1:11211" --op "backup" --filename "mem.json"

Command backed up only 58254 values so 9941745 values are missing.
Please see utility output:

2022-02-07T22:02:11.721038 main ▶ INFO 001 address 127.0.0.1:11211
2022-02-07T22:02:11.830346 backupCache ▶ INFO 002 58254 values found in the storage
2022-02-07T22:02:13.024945 backupCache ▶ INFO 003 Output file successfully generated at: mem.json

After the test I made sure that there are records present on Memcached server, but missing in mem.json file.