Code Monkey home page Code Monkey logo

redisql's People

Contributors

danielemegna avatar gitter-badger avatar olirice avatar sgc109 avatar siscia avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

redisql's Issues

Switch to redis allocator

With the new rust compiler is possible to select a global allocator.

Maybe it is a simple way to use the Redis allocator in all the module with ease.

execute complicate sql to query stream data lead the database not work

op_taxi_order have about 100k , and stream data write it continuely
op_taxi_order_all is small
when i run the sql , the result like this

select a.car_no,a.city_id,a.city_name,(a.num+CASE WHEN b.total_num is null THEN 0 ELSE b.total_num END) total_num from (select car_no,city_id,city_name,count(1) num,sum(CASE WHEN call=0 THEN meterKm ELSE 0 END) meterKm,sum(CASE WHEN call=0 THEN emptyDrivingKm ELSE 0 END) emptyDrivingKm,sum(CASE WHEN call=1 THEN meterKm ELSE 0 END) call_meterKm,sum(CASE WHEN call=1 THEN emptyDrivingKm ELSE 0 END) call_emptyDrivingKm from op_taxi_order where time>'2019-06-17 00:00:00' and time<'2019-06-17 23:59:59' group by car_no,city_id,city_name) a left join op_taxi_order_all b on a.car_no=b.car_no limit 10
(nil)
(10.06s)
redisql.exec athenaT "SELECT * from op_status limit 10"
(nil)
(10.07s)

and the result of other sql is wrong.
why? the athenaT of database is locked?
other database is good.

how can i check the redisql log?

Why distribute sqlite3.{c,h}

Why do you distribute sqlite3.{c,h}? Imho it's easier to take it from the packagemanager provided by the operating system.

apk add sqlite-dev sqlite-libs # alpine
pacman -S sqlite3 # arch
apt-get install libsqlite3-0 libsqite3-dev # debian
brew install sqlite # plastic os
zypper install libsqlite3-0 sqlite3-devel # some old rpm distros
dnf install sqlite-devel sqlite-libs # fedora

PayHip hates me

https://payhip.com/RediSQL

I am not able to buy the latest pro version.
Tried it yesterday and today and always get this error: "Apologies, could not complete transaction"

I can pay using my credit card at other payment providers, so my card is not the problem.

What can we do? :(

`GLIBC_2.18' not found in CentOS 7

This error occurs when I load the rediSQL v0.3.1 module in CentOS 7. I try to find the issue on Google, but not working. If you have any idea, please help me. Thanks !
3116:M 24 Feb 10:10:34.237 # Module modules/libredis_sql.so failed to load: /lib64/libc.so.6: version 'GLIBC_2.18' not found (required by modules/libredis_sql.so) 3116:M 24 Feb 10:10:34.237 # Can't load module from modules/libredis_sql.so: server aborting

Multiple statements first prepare and then execute

RediSQL support multiple statements in the same "EXEC", hence it is possible and correct to do something like:

REDISQL.EXEC DB "INSERT INTO foo VALUES(1,2,3); INSERT INTO bar VALUES(2,3,4);" 

Which is quite convenient when mixed with BEGIN, COMMIT and ABORT instruction.

One problem is that all the statements are first prepared all together, and then executed, all together.

This works fine in the general case, but has some limitation when to prepare a statement is necessary to have execute the previous one.

This is the case for creating a table and then create an index on that table, or create a second table with references to the first one.

The fix would be to return an iterator when generating the statements, so that each statement is generate only when the previous one is already execute.

Roughly here: https://github.com/RedBeardLab/rediSQL/blob/master/redisql_lib/src/community_statement.rs#L70

However there may be concurrency issues that I have overlooked.

Crash at redis server start when sqlite file is deleted

23189:C 25 Mar 2019 14:39:44.911 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
23189:C 25 Mar 2019 14:39:44.911 # Redis version=5.0.3, bits=64, commit=00000000, modified=0, pid=23189, just started
23189:C 25 Mar 2019 14:39:44.911 # Configuration loaded
23189:M 25 Mar 2019 14:39:44.913 # You requested maxclients of 10000 requiring at least 10032 max file descriptors.
23189:M 25 Mar 2019 14:39:44.913 # Server can't set maximum open files to 10032 because of OS error: Operation not permitted.
23189:M 25 Mar 2019 14:39:44.913 # Current maximum open files is 4096. maxclients has been reduced to 4064 to compensate for low ulimit. If you need higher maxclients increase 'ulimit -n'.
23189:M 25 Mar 2019 14:39:44.914 * Running mode=standalone, port=6381.
23189:M 25 Mar 2019 14:39:44.914 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
23189:M 25 Mar 2019 14:39:44.914 # Server initialized
23189:M 25 Mar 2019 14:39:44.914 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
23189:M 25 Mar 2019 14:39:44.915 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
23189:M 25 Mar 2019 14:39:44.916 * Configuration: concurrent mode: 1, ext load: (null), min prefix: 2, max expansions: 200, query timeout: 500ms, timeout policy: return
23189:M 25 Mar 2019 14:39:44.917 * Initialized thread pool!
23189:M 25 Mar 2019 14:39:44.918 * Module 'ft' loaded from /home/willi/.redis/redisearch.so
23189:M 25 Mar 2019 14:39:44.918 * Module 'rediSQL' loaded from /home/willi/.redis/redisql.so
23189:M 25 Mar 2019 14:39:44.922 * Thread pool created, using 8 threads.
23189:M 25 Mar 2019 14:39:44.922 * Module 'graph' loaded from /home/willi/.redis/redisgraph.so
23189:M 25 Mar 2019 14:39:44.922 * Reading RDB preamble from AOF file...
23189:M 25 Mar 2019 14:39:44.922 # Error loading data from RDB (short read or EOF). Read performed by module 'rediSQL' about type 'rediSQLDB' after reading '0' bytes of a value.

Error: Version `GLIBC_2.25' not found in Ubuntu 16.04.2 LTS when load module

Hi guys, thanks guys for awsome Redis module.

I have tried to install with an environment like:

  • RediSQL version 1.0.0 ( redisql_v1.0.0_x86_64.so )

  • Redis server v=5.0.4 sha=00000000:0 malloc=jemalloc-5.1.0 bits=64 build=81f9a02b211fe0e1

  • Ubuntu 16.04.2 LTS

And I have error:

Module /etc/redis/redisql_v1.0.0_x86_64.so failed to load: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.25' not found (required by /etc/redis/redisql_v1.0.0_x86_64.so)
Can't load module from /etc/redis/redisql_v1.0.0_x86_64.so: server aborting

I have checked and see the version of glibc in my server is:

apt-cache policy libc6
libc6:
Installed: 2.23-0ubuntu11
Candidate: 2.23-0ubuntu11
Version table:
*** 2.23-0ubuntu11 500
500 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 Packages
100 /var/lib/dpkg/status
2.23-0ubuntu10 500
500 http://security.ubuntu.com/ubuntu xenial-security/main amd64 Packages
2.23-0ubuntu3 500
500 http://archive.ubuntu.com/ubuntu xenial/main amd64 Packages

I have tried with version redisql_v0.9.0_x86_64.so and it works like a champ

Can you help take a look? , thanks you.

Empty select returns ["DONE", 0]

Returning an empty select return the couple ["DONE", 0]

It is not a completely unreasonable return value, is not even the best one.

Investingate CentOS

Necessary to investigate how RediSQL behave on Centos.

It seems quite hard to compile and run it on Centos

Few Suggestions

Hi it's a great project. I feel below things are missing in your doc. Can you please mention them for a better clarity?

  1. How to load data from a csv directly to the sqlite?
  2. Where is the original sqlite file getting stored?? How can we access it directly apart from the client?
  3. Because of the module's blocking nature it couldn't be executed in eval script of redis. So we need to run a microservice which directly hits the redis commands as we hit in the client terminal. Is there any other possible way of doing it in API??

We are planning to use this module as a queryable cache layer. For that we need to load data directly into the sqlite file. Please reply with your suggestions

"libredis_sql.so failed to load: dlopen(libredis_sql.so, 6): no suitable image found" on MAC OS

I am running redis "Redis 4.0.8 (00000000/0) 64 bit" on MAC OS 10.12.6. Loading libredis_sql.so (v0.3.1) gives the following error message:

32342:M 15 Mar 13:06:36.198 # Server initialized
32342:M 15 Mar 13:06:36.199 # Module libredis_sql.so failed to load: dlopen(libredis_sql.so, 6): no suitable image found. Did find:
libredis_sql.so: unknown file type, first eight bytes: 0x7F 0x45 0x4C 0x46 0x02 0x01 0x01 0x00
/Volumes/eswitch/tools/redis-stable/src/libredis_sql.so: unknown file type, first eight bytes: 0x7F 0x45 0x4C 0x46 0x02 0x01 0x01 0x00
32342:M 15 Mar 13:06:36.199 # Can't load module from libredis_sql.so: server aborting

Should libredis_sql.so (v0.3.1) be working on MAC OS or has it to be recompiled for Mac?

need client and demo for java

This is a great project. So far, I have seen several other language clients. When can the author produce the Java version of the client? There is no enough demonstration examples. Finally, I hope the project will get better and better!

Desiderata

I am in the process of restructuring the code base quite a bit.

This is the moment to ask for features or idea or any desiderata.

Please do not be afraid to speak up.

Thanks

Provide a Ruby example using rediSQL

Ruby has at least one Redis gem. Provide an example to demonstrate how to use rediSQL with Ruby. Bonus: Show how something like rake db:create db:migrate might work.

the lock issue, when streaming data write into a table, run the long-time sql

#66 (comment)

Does this timeout?

I need to fix the timeout as well.

What I believe happens, is that the query takes more than 10sec, RediSQL timeout, and it replies to you with an empty response. However, eventually the query terminates, and it tried to reply, but the internal state is "corrupted" by the previous timeouted reply, so the thread crash. Only the thread crash so RediSQL keep running and you just see something that stop working.

Can you confirm this with the debug build? Just post the logs if you can reproduce it.

i test this on two scenarios.
a table with 4000k data,execute the command "delete from a where id>2000000 and id<3000000", then my client get the error " read timeout". but the database can work, only "10s" timeout
redis log:

[2019-07-02T07:35:44Z DEBUG redisql_lib::redis] Exec | Query = "delete from a where id>2000000 and id<3000000"
[2019-07-02T07:35:44Z DEBUG redisql_lib::redis] do_execute | created statement
[2019-07-02T07:35:44Z DEBUG redisql_lib::community_statement] Execute | Acquired db lock
[2019-07-02T07:35:44Z DEBUG redisql_lib::community_statement] Execute | Read row modified before
[2019-07-02T07:35:54Z DEBUG redisql_lib::sqlite] FromIterator => DONECursor
[2019-07-02T07:35:54Z DEBUG redisql_lib::community_statement] Execute=> Executed trains of statements
[2019-07-02T07:35:54Z DEBUG redisql_lib::community_statement] Execute=>DONECursor
[2019-07-02T07:35:54Z DEBUG redisql_lib::redis] do_execute | statement executed
[2019-07-02T07:35:54Z DEBUG redisql_lib::redis_type] Free thread safe context
[2019-07-02T07:35:54Z DEBUG redisql_lib::redis] RedisContextSet | Drop
[2019-07-02T07:35:54Z DEBUG redisql_lib::redis] with_contex_set | exit
[2019-07-02T07:35:54Z DEBUG redisql_lib::redis] Exec | DONE, returning result
[2019-07-02T07:35:54Z DEBUG redisql_lib::redis] Loop iteration

another :
a table with 4000k data , when i execute the command "delete from b where id>1000000 and id<2000000", the streaming data always write into the table.
my client get the error " timeout". but the database can not work.

the redis log:
[2019-07-02T08:55:17Z DEBUG redisql_lib::redis] Exec | Query = "delete from b where id>=1000000 and id<2000000"
[2019-07-02T08:55:17Z DEBUG redisql_lib::redis] do_execute | created statement
[2019-07-02T08:55:17Z DEBUG redisql_lib::community_statement] Execute | Acquired db lock
[2019-07-02T08:55:17Z DEBUG redisql_lib::community_statement] Execute | Read row modified before
[2019-07-02T08:55:17Z DEBUG redis_sql::commands] Exec | GotDB
[2019-07-02T08:55:17Z DEBUG redis_sql::commands] Exec | BlockedClient
[2019-07-02T08:55:17Z DEBUG redis_sql::commands] Exec | Create Command
[2019-07-02T08:55:28Z DEBUG redis_sql::commands] Exec | GotDB
[2019-07-02T08:55:28Z DEBUG redis_sql::commands] Exec | BlockedClient
[2019-07-02T08:55:28Z DEBUG redis_sql::commands] Exec | Create Command
[2019-07-02T08:55:28Z DEBUG redisql_lib::sqlite] FromIterator => DONECursor
[2019-07-02T08:55:28Z DEBUG redisql_lib::community_statement] Execute=> Executed trains of statements
[2019-07-02T08:55:28Z DEBUG redisql_lib::community_statement] Execute=>DONECursor
[2019-07-02T08:55:28Z DEBUG redisql_lib::redis] do_execute | statement executed
[2019-07-02T08:55:28Z DEBUG redisql_lib::redis_type] Free thread safe context
[2019-07-02T08:55:28Z DEBUG redisql_lib::redis] RedisContextSet | Drop
[2019-07-02T08:55:28Z DEBUG redisql_lib::redis] with_contex_set | exit
[2019-07-02T08:55:28Z DEBUG redisql_lib::redis] Exec | DONE, returning result
[2019-07-02T08:55:28Z DEBUG redisql_lib::redis] Loop iteration

why? lock compete lead to?

large INT handling issues

Results seem to return -1 when a large bit should be returned

192.168.0.45:6379> flushall
OK
192.168.0.45:6379> REDISQL.CREATE_DB db
OK
192.168.0.45:6379> redisql.exec db "CREATE TABLE ip_to_asn(start INT8, end INT8, asn int, hosts int8)"
1) DONE
2) (integer) 0
192.168.0.45:6379> redisql.exec db "insert into ip_to_asn values (2883484276228096000, 2883484280523063295, 265030, 4294967295)"
1) DONE
2) (integer) 1
192.168.0.45:6379> redisql.exec db "select * from ip_to_asn where 2883484276228096000 between start and end order by hosts desc limit 1"

result

1) 1) (integer) 0
   2) (integer) -1
   3) (integer) 265030
   4) (integer) -1

expected

1) 1) (integer) 2883484276228096000
   2) (integer) 2883484280523063295
   3) (integer) 265030
   4) (integer) 4294967295

changing the INT8 to/from INT doesn't appear to make any difference

2883484276228096000 > 9223372036854776000 (2^63 - the signed max of a 64bit integer)

Running Redis 4.0.11 on Ubuntu 18.04 x64, with your static 0.70 binary

# Server
redis_version:999.999.999
redis_git_sha1:f54cde3a
redis_git_dirty:0
redis_build_id:269a808b2b47051f
redis_mode:standalone
os:Linux 4.4.0-134-generic x86_64
arch_bits:64
multiplexing_api:epoll
atomicvar_api:atomic-builtin
gcc_version:5.4.0
process_id:30150
run_id:001157c888ccf1ec2ddbc30e206322aa0c04b88a
tcp_port:6379
uptime_in_seconds:1770
uptime_in_days:0
hz:10
configured_hz:10
lru_clock:12764423
executable:/home/gateway/redis/src/./redis-server
config_file:/home/gateway/redis/redis.conf```

redisql.create_statement command has error

redisql.create_statement DB 'INSERT INTO table_aa (A,B) VALUES (?1,?2);'

there appears error
(error) Wrong number of arguments, it accepts 4

and I change the command like redisql.create_statement DB 'INSERT INTO table_aa (A,B) VALUES (?,?);'
the error also appears.

how to get the RediSQL .so file?

thanks a lot! this module is great! i want to learn it,but i don't know where to get the RediSQL .so file,need help !thanks !

cargo build returns error and perf improvement

➜ rediSQL git:(master) cargo build --release
error: failed to load source for a dependency on engine_pro

Caused by:
Unable to update file:///home/tetter/redis/rediSQL/engine_pro

Caused by:
failed to read /home/tetter/redis/rediSQL/engine_pro/Cargo.toml

Caused by:
No such file or directory (os error 2)

Looks like the Cargo.toml file is missing in engine_pro?

Thanks,
Xin

Semicolon and newline

There seems to be some weird issue when a query terminate with a semicolon ; and whitespaces or new lines.

So, queries like this works ok:

SELECT A, B from foo;

While this other seems to show the issue problem:

SELECT A, B from foo; -- with whitespaces

or also


--with newlines```

Where is appropriate java client for rediSQL ?

Hello,

I want to make query like Standard SQL using java. Please make a tutorial for us using java.

What dependencies are required to work properly like your python example using java please mention.

Thanks in advance.

'Use but do not abuse' unclear

In the README you state "use but do not abuse" rediSQL. However, I don't think it is entirely obvious what you mean by that. Could you provide some clarification of what would count as use vs abuse? Are the use cases that count as 'use' as broad as for SQLite in general or are there use cases for SQLite that you would consider abuse for rediSQL?

Not able to install redisql by all 3 ways

macOs Mojave - 10.14.3 - 64bit

Way 1 - docker run siscia/redisql:latest
Way 1.1 docker run -it --net host siscia/redisql
Error - Can't load module from /usr/local/lib/libredis_sql.so: server aborting

Way 2 - redis-server --loadmodule redisql_v0.9.2_x86_64.so
Error 1 Can't load module from 4GpEA2y1oztM2V4MmkXpQn.so: server aborting
Error 2 Can't load module from redisql_v0.9.2_ARMv7.so: server aborting
Error 3 Can't load module from redisql_v0.9.2_x86_64.so: server aborting

Way 3 Compiling module ON cargo build --release inside cloned repository
Error

Also Which one i should consider for 64bit ?
RediSQL 0.7.0 x86_64 SO
RediSQL 0.7.0 ARMv7 SO
RediSQL 0.7.0 ARMv7 TRIAL SO
RediSQL 0.7.0 x86_64 TRIAL

cargo build error

After i install rust use
curl https://sh.rustup.rs -sSf | sh
Then i Compiling the rediSQL following the instruction.
I don't know how to solve this problem,can you help me?

[root@localhost rediSQL]# cargo build --release
   Compiling void v1.0.2
   Compiling strsim v0.6.0
   Compiling scopeguard v0.3.3
   Compiling rayon-core v1.3.0
   Compiling regex-syntax v0.4.1
   Compiling ansi_term v0.9.0
   Compiling lazy_static v0.2.10
   Compiling term v0.4.6
   Compiling bitflags v0.9.1
   Compiling utf8-ranges v1.0.0
   Compiling fnv v1.0.6
   Compiling log v0.3.8
   Compiling libloading v0.4.2
   Compiling vec_map v0.8.0
   Compiling bitflags v0.8.2
   Compiling libc v0.2.33
   Compiling either v1.4.0
   Compiling glob v0.2.11
   Compiling cfg-if v0.1.2
   Compiling peeking_take_while v0.1.2
   Compiling unicode-width v0.1.4
   Compiling unicode-xid v0.0.4
   Compiling rustc-serialize v0.3.24
   Compiling unreachable v1.0.0
   Compiling which v1.0.3
   Compiling memchr v1.0.2
   Compiling atty v0.2.3
   Compiling rand v0.3.18
   Compiling num_cpus v1.7.0
   Compiling coco v0.1.1
   Compiling clang-sys v0.19.0
   Compiling textwrap v0.9.0
   Compiling syntex_pos v0.58.1
   Compiling thread_local v0.3.4
   Compiling aho-corasick v0.6.3
   Compiling nom v3.2.1
   Compiling uuid v0.4.0
   Compiling clap v2.27.1
   Compiling syntex_errors v0.58.1
   Compiling regex v0.2.2
   Compiling cexpr v0.2.2
   Compiling rayon v0.8.2
   Compiling syntex_syntax v0.58.1
   Compiling env_logger v0.4.3
   Compiling cc v1.0.3
   Compiling syntex v0.58.1
   Compiling quasi v0.32.0
   Compiling aster v0.41.0
   Compiling quasi_codegen v0.32.0
   Compiling bindgen v0.30.0
   Compiling rediSQL v0.3.1 (file:///root/Desktop/redisql/rediSQL)
error: failed to run custom build command for `rediSQL v0.3.1 (file:///root/Desktop/redisql/rediSQL)`
process didn't exit successfully: `/root/Desktop/redisql/rediSQL/target/release/build/rediSQL-024e4caa69221a00/build-script-build` (exit code: 101)
--- stdout
cargo:rerun-if-changed=src/CDeps
TARGET = Some("x86_64-unknown-linux-gnu")
OPT_LEVEL = Some("3")
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
TARGET = Some("x86_64-unknown-linux-gnu")
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
CC_x86_64-unknown-linux-gnu = None
CC_x86_64_unknown_linux_gnu = None
HOST_CC = None
CC = None
HOST = Some("x86_64-unknown-linux-gnu")
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
CFLAGS_x86_64-unknown-linux-gnu = None
CFLAGS_x86_64_unknown_linux_gnu = None
HOST_CFLAGS = None
CFLAGS = None
DEBUG = Some("false")
running: "cc" "-O3" "-ffunction-sections" "-fdata-sections" "-fPIC" "-m64" "-I" "src/CDeps/Redis/include" "-Wall" "-Wextra" "-o" "/root/Desktop/redisql/rediSQL/target/release/build/rediSQL-e9b4b83cec1bc012/out/src/CDeps/Redis/redismodule.o" "-c" "src/CDeps/Redis/redismodule.c"
cargo:warning=src/CDeps/Redis/redismodule.c:3:25: 警告:‘DBType’已初始化,却又被声明为‘extern’ [默认启用]
cargo:warning= extern RedisModuleType *DBType = NULL;
cargo:warning=                         ^
exit code: 0
TARGET = Some("x86_64-unknown-linux-gnu")
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
AR_x86_64-unknown-linux-gnu = None
AR_x86_64_unknown_linux_gnu = None
HOST_AR = None
AR = None
TARGET = Some("x86_64-unknown-linux-gnu")
TARGET = Some("x86_64-unknown-linux-gnu")
running: "ar" "crs" "/root/Desktop/redisql/rediSQL/target/release/build/rediSQL-e9b4b83cec1bc012/out/libredismodule.a" "/root/Desktop/redisql/rediSQL/target/release/build/rediSQL-e9b4b83cec1bc012/out/src/CDeps/Redis/redismodule.o"
exit code: 0
TARGET = Some("x86_64-unknown-linux-gnu")
cargo:rustc-link-lib=static=redismodule
cargo:rustc-link-search=native=/root/Desktop/redisql/rediSQL/target/release/build/rediSQL-e9b4b83cec1bc012/out
TARGET = Some("x86_64-unknown-linux-gnu")
OPT_LEVEL = Some("3")
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
TARGET = Some("x86_64-unknown-linux-gnu")
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
CC_x86_64-unknown-linux-gnu = None
CC_x86_64_unknown_linux_gnu = None
HOST_CC = None
CC = None
HOST = Some("x86_64-unknown-linux-gnu")
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
CFLAGS_x86_64-unknown-linux-gnu = None
CFLAGS_x86_64_unknown_linux_gnu = None
HOST_CFLAGS = None
CFLAGS = None
DEBUG = Some("false")
running: "cc" "-O3" "-ffunction-sections" "-fdata-sections" "-fPIC" "-m64" "-I" "src/CDeps/SQLite/include" "-Wall" "-Wextra" "-DHAVE_USLEEP=1" "-DNDEBUG=1" "-DHAVE_FDATASYNC=1" "-DSQLITE_THREADSAFE=2" "-DSQLITE_ENABLE_JSON1=1" "-DSQLITE_ENABLE_FTS3=1" "-DSQLITE_ENABLE_FTS4=1" "-DSQLITE_ENABLE_FTS5=1" "-DSQLITE_ENABLE_RTREE=1" "-o" "/root/Desktop/redisql/rediSQL/target/release/build/rediSQL-e9b4b83cec1bc012/out/src/CDeps/SQLite/sqlite3.o" "-c" "src/CDeps/SQLite/sqlite3.c"
cargo:warning=src/CDeps/SQLite/sqlite3.c: 在函数‘exprAnalyze’中:
cargo:warning=src/CDeps/SQLite/sqlite3.c:131527:39: 警告:此函数中的‘pLeft’在使用前可能未初始化 [-Wmaybe-uninitialized]
cargo:warning=         pNewTerm->u.leftColumn = pLeft->iColumn;
cargo:warning=                                       ^
cargo:warning=src/CDeps/SQLite/sqlite3.c:93128:39: 警告:此函数中的‘pRight’在使用前可能未初始化 [-Wmaybe-uninitialized]
cargo:warning=   return p ? exprDup(db, p, flags, 0) : 0;
cargo:warning=                                       ^
cargo:warning=src/CDeps/SQLite/sqlite3.c:131506:11: 附注:‘pRight’在此声明
cargo:warning=     Expr *pRight, *pLeft;
cargo:warning=           ^
cargo:warning=src/CDeps/SQLite/sqlite3.c:131529:28: 警告:此函数中的‘eOp2’在使用前可能未初始化 [-Wmaybe-uninitialized]
cargo:warning=         pNewTerm->eMatchOp = eOp2;
cargo:warning=                            ^
exit code: 0
TARGET = Some("x86_64-unknown-linux-gnu")
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
AR_x86_64-unknown-linux-gnu = None
AR_x86_64_unknown_linux_gnu = None
HOST_AR = None
AR = None
TARGET = Some("x86_64-unknown-linux-gnu")
TARGET = Some("x86_64-unknown-linux-gnu")
running: "ar" "crs" "/root/Desktop/redisql/rediSQL/target/release/build/rediSQL-e9b4b83cec1bc012/out/libsqlite3.a" "/root/Desktop/redisql/rediSQL/target/release/build/rediSQL-e9b4b83cec1bc012/out/src/CDeps/SQLite/sqlite3.o"
exit code: 0
TARGET = Some("x86_64-unknown-linux-gnu")
cargo:rustc-link-lib=static=sqlite3
cargo:rustc-link-search=native=/root/Desktop/redisql/rediSQL/target/release/build/rediSQL-e9b4b83cec1bc012/out

--- stderr
thread 'main' panicked at 'Unable to find libclang: "couldn\'t find any of [\'libclang.so\', \'libclang.so.*\'], set the LIBCLANG_PATH environment variable to a path where one of these files can be found (skipped: [])"', /checkout/src/libcore/result.rs:906:4
note: Run with `RUST_BACKTRACE=1` for a backtrace.

Remove default db

It is too complex to manage persistent for non key stuff.

We can create a command to set a default db but it sound like it will bring more complexity than benefits.

COPY command more than one time crashes redis server

Version-Info:
Redis 5.0.3
RediSQL: 0.9.0 Pro

Server crashes after second copy call.

On the first call, the copy is completely successful.
On the second call the crash prevents the successful copy.

There is also no crash report in server log and that's very strange.

PS: REDISQL.COPY.NOW works fine! The only seems to affect the default COPY command.

Example 1:
127.0.0.1:6381> REDISQL.CREATE_DB db
OK
127.0.0.1:6381> REDISQL.CREATE_DB db2
OK
127.0.0.1:6381> REDISQL.COPY db db2
OK
127.0.0.1:6381> REDISQL.COPY db db2
(nil)
(10.00s)
127.0.0.1:6381> REDISQL.COPY db db2
Could not connect to Redis at 127.0.0.1:6381: Connection refused
not connected>

Example 2:
127.0.0.1:6381> REDISQL.CREATE_DB db3
OK
127.0.0.1:6381> REDISQL.COPY db db2
OK
127.0.0.1:6381> REDISQL.COPY db db3
Could not connect to Redis at 127.0.0.1:6381: Connection refused
not connected>

Slow, design approach

Overall I find the whole module + sqlite quite slow, especially when it writes on disk.

On my machine I get roughly 8 inserts per second on disk using the most naive method, independently of the concurrency used (which makes sense since at least for now it is everything single threaded).

simo@simo:~/redis-4.0-rc1/src$ ./redis-benchmark -c 1 -r 10000 -n 100 REDISQL.EXEC DB 'INSERT INTO test VALUES(__rand_int__, __rand_int__, __rand_int__);'
====== REDISQL.EXEC DB INSERT INTO test VALUES(__rand_int__, __rand_int__, __rand_int__); ======
  100 requests completed in 12.30 seconds
  1 parallel clients
  3 bytes payload
  keep alive: 1

1.00% <= 110 milliseconds
2.00% <= 111 milliseconds
40.00% <= 112 milliseconds
43.00% <= 113 milliseconds
46.00% <= 116 milliseconds
47.00% <= 122 milliseconds
67.00% <= 123 milliseconds
72.00% <= 124 milliseconds
74.00% <= 133 milliseconds
84.00% <= 134 milliseconds
91.00% <= 135 milliseconds
92.00% <= 144 milliseconds
94.00% <= 145 milliseconds
97.00% <= 156 milliseconds
99.00% <= 167 milliseconds
100.00% <= 201 milliseconds
8.13 requests per second

clearly this is not enough.

However the slow performance are expected, if we hit the disk for each insert, we will be slow no matter what.

The obvious solution is to don't hit the disk for every and each insert, but batch multiple insert and execute all together inside a single transaction.

simo@simo:~/redis-4.0-rc1/src$ ./redis-benchmark -n 1 REDISQL.EXEC DB 'BEGIN TRANSACTION'; ./redis-benchmark -c 1 -r 10000 REDISQL.EXEC DB 'INSERT INTO test VALUES(__rand_int__, __rand_int__, __rand_int__);'; ./redis-benchmark -n 1 REDISQL.EXEC DB 'COMMIT TRANSACTION'
====== REDISQL.EXEC DB BEGIN TRANSACTION ======
  1 requests completed in 0.00 seconds
  50 parallel clients
  3 bytes payload
  keep alive: 1

100.00% <= 0 milliseconds
1000.00 requests per second

====== REDISQL.EXEC DB INSERT INTO test VALUES(__rand_int__, __rand_int__, __rand_int__); ======
  100000 requests completed in 10.62 seconds
  1 parallel clients
  3 bytes payload
  keep alive: 1

99.98% <= 1 milliseconds
100.00% <= 2 milliseconds
100.00% <= 3 milliseconds
9411.76 requests per second

====== REDISQL.EXEC DB COMMIT TRANSACTION ======
  1 requests completed in 0.19 seconds
  50 parallel clients
  3 bytes payload
  keep alive: 1

100.00% <= 188 milliseconds
5.32 requests per second

This other approach provide a very higher throughput (2 order of magnitude) with very small effort.

Now, before to go blindly ahead and create a pool of statements and provide a new command FLUSH, to force a transaction, I am going to explore more deeply what PRAGMA option SQLite can offer.

Usually there is a trade off between velocity and safeness, however, living inside redis I can use the guarantees of redis to make up some of the safeness lost and see if I can reach good enough performance.

[1]: Statement, something that modify the internal status of the database, it is in contraposition with query that simply read the database (INSERT, UPDATE, DELETE vs SELECT)

Is there a way to time a query?

The redis logs give the time taken for the query execution but this time includes both the query translation of rediSQL and the actual database access time. Is there any way to just get the access time alone?

redis.exceptions.ResponseError: Wrong number of arguments, it accepts 3

try use you soft from https://hub.docker.com/r/siscia/redisql/, but get error
python rediSQL_bench.py

Starting performance test
Creating tables
Traceback (most recent call last):
  File "rediSQL_bench.py", line 162, in <module>
    prepare_tables(redis_pool)
  File "rediSQL_bench.py", line 31, in prepare_tables
    r.execute_command("REDISQL.EXEC", t)
  File "/usr/local/lib/python2.7/dist-packages/redis/client.py", line 573, in execute_command
    return self.parse_response(connection, command_name, **options)
  File "/usr/local/lib/python2.7/dist-packages/redis/client.py", line 585, in parse_response
    response = connection.read_response()
  File "/usr/local/lib/python2.7/dist-packages/redis/connection.py", line 582, in read_response
    raise response
redis.exceptions.ResponseError: Wrong number of arguments, it accepts 3
  1. what i do wrong?
  2. i good understand what all data store in Redis, but use sqlite:memory how backend and possible use any extension for sqllite? and this do?

Refactor the code

Right now the code is barely readable, a refactor is badly needed.

Error when change version from 0.9 to 1.0.1

Hi guys,

Recently I have updated version from redisql_v0.9.0_x86_64.so to redisql_v1.0.1_x86_64.so and I got error:

11306:C 20 May 2019 15:24:33.488 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
11306:C 20 May 2019 15:24:33.488 # Redis version=5.0.4, bits=64, commit=00000000, modified=0, pid=11306, just started
11306:C 20 May 2019 15:24:33.488 # Configuration loaded
11306:C 20 May 2019 15:24:33.488 # upstart supervision requested, but UPSTART_JOB not found
11306:M 20 May 2019 15:24:33.488 * Increased maximum number of open files to 10032 (it was originally set to 1024).
                _._
           _.-``__ ''-._
      _.-``    `.  `_.  ''-._           Redis 5.0.4 (00000000/0) 64 bit
  .-`` .-```.  ```\/    _.,_ ''-._
 (    '      ,       .-`  | `,    )     Running in standalone mode
 |`-._`-...-` __...-.``-._|'` _.-'|     Port: 6379
 |    `-._   `._    /     _.-'    |     PID: 11306
  `-._    `-._  `-./  _.-'    _.-'
 |`-._`-._    `-.__.-'    _.-'_.-'|
 |    `-._`-._        _.-'_.-'    |           http://redis.io
  `-._    `-._`-.__.-'_.-'    _.-'
 |`-._`-._    `-.__.-'    _.-'_.-'|
 |    `-._`-._        _.-'_.-'    |
  `-._    `-._`-.__.-'_.-'    _.-'
      `-._    `-.__.-'    _.-'
          `-._        _.-'
              `-.__.-'

11306:M 20 May 2019 15:24:33.493 # Server initialized
11306:M 20 May 2019 15:24:33.494 * Module 'rediSQL' loaded from /etc/redis/redisql_v1.0.1_x86_64.so


Error: Statement already exists, impossible to overwrite it with this command, try with UPDATE_STATEMENT
Error: Statement already exists, impossible to overwrite it with this command, try with UPDATE_STATEMENT
Error: Statement already exists, impossible to overwrite it with this command, try with UPDATE_STATEMENT
Error: Statement already exists, impossible to overwrite it with this command, try with UPDATE_STATEMENT
Error: Statement already exists, impossible to overwrite it with this command, try with UPDATE_STATEMENT
Error: Statement already exists, impossible to overwrite it with this command, try with UPDATE_STATEMENT
Error: Statement already exists, impossible to overwrite it with this command, try with UPDATE_STATEMENT
Error: Statement already exists, impossible to overwrite it with this command, try with UPDATE_STATEMENT
11306:M 20 May 2019 15:24:39.669 # Error loading data from RDB (short read or EOF). Read performed by module 'rediSQL' about type 'rediSQLDB' after reading '0' bytes of a value.

Is still compatible when upgrading from version 0.9 to version 1.0.1?

C++ Adaptor/ C++ API for Redis SQL.

I can work on a C++ for Redis SQL inegration for exisiting C++ projects. I intend to use it in a blockchain implementation written in C++ primarily.

redisql.copy command delete the source db?

redisql version 0.7.1

redisql.copy test_a test_b
OK
redisql.exec test_a "select name from sqlite_master where type='table'"
(nil)
(10.05s)
redisql.exec test_b "select name from sqlite_master where type='table'"

    1. "RediSQLMetadata"
    1. "a"

Fast memory test PASSED, however your memory can still be broken. Please run a memory test for several hours if possible

.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O
Fast memory test PASSED, however your memory can still be broken. Please run a memory test for several hours if possible.

------ DUMPING CODE AROUND EIP ------
Symbol: (null) (base: (nil))
Module: /usr/local/src/redis-5.0.3/modules/redisql_v0.9.2_x86_64.so (base 0x7f449c010000)
$ xxd -r -p /tmp/dump.hex /tmp/dump.bin
$ objdump --adjust-vma=(nil) -D -b binary -m i386:x86-64 /tmp/dump.bin

=== REDIS BUG REPORT END. Make sure to include from START to END. ===

   Please report the crash by opening an issue on github:

       http://github.com/antirez/redis/issues

Suspect RAM error? Use redis-server --test-memory to verify it.

The above log information appears after I run a more complicated SQL statement. But the data in my database table is empty

Implement Iterator for returning results

Up to now, redisql accumulate all the results in memory, and return all of them together.

This design is all but ideal, but was implemented for simplicity reasons.

Our good friend from #69 discovered that in some real-world case this can lead to nasty problems. (Or at least I reproduce a similar issue.)

It should be possible to avoid the accumulation of the whole result in memory, and instead return an iterator.

This is a quite complex piece of code, and a very good test bench for Rust. Hopefully I will be able to pull also this one.

The code is complex because we need to pass back and forth an instance of a locked raw pointer, it should theorical be possible.

The piece of code concerned is:

https://github.com/RedBeardLab/rediSQL/blob/master/redisql_lib/src/sqlite.rs#L333

Before to move to this, I would like to move to the rust edition 2018.

Then, on a clean codebase, we can work on this.

Error building rediSQL on raspberry pi

I received the following error log when trying to build rediSQL on raspberry pi.
Please let me know if I can supply further information

Compiling rediSQL v0.3.0 (file:///home/pi/redis/modules/rediSQL-master)
libclang.solibclang.soerror: failed to run custom build command for rediSQL v0.3.0 (file:///home/pi/redis/modules/rediSQL-master)
process didn't exit successfully: /home/pi/redis/modules/rediSQL-master/target/release/build/rediSQL-2d6d521084489aa0/build-script-build (exit code: 101)
--- stdout
cargo:rerun-if-changed=src/CDeps
TARGET = Some("armv7-unknown-linux-gnueabihf")
OPT_LEVEL = Some("3")
TARGET = Some("armv7-unknown-linux-gnueabihf")
HOST = Some("armv7-unknown-linux-gnueabihf")
TARGET = Some("armv7-unknown-linux-gnueabihf")
TARGET = Some("armv7-unknown-linux-gnueabihf")
HOST = Some("armv7-unknown-linux-gnueabihf")
CC_armv7-unknown-linux-gnueabihf = None
CC_armv7_unknown_linux_gnueabihf = None
HOST_CC = None
CC = None
HOST = Some("armv7-unknown-linux-gnueabihf")
TARGET = Some("armv7-unknown-linux-gnueabihf")
HOST = Some("armv7-unknown-linux-gnueabihf")
CFLAGS_armv7-unknown-linux-gnueabihf = None
CFLAGS_armv7_unknown_linux_gnueabihf = None
HOST_CFLAGS = None
CFLAGS = None
DEBUG = Some("false")
running: "cc" "-O3" "-ffunction-sections" "-fdata-sections" "-fPIC" "-march=armv7-a" "-I" "src/CDeps/Redis/include" "-Wall" "-Wextra" "-o" "/home/pi/redis/modules/rediSQL-master/target/release/build/rediSQL-f6317b3371ca522c/out/src/CDeps/Redis/redismodule.o" "-c" "src/CDeps/Redis/redismodule.c"
cargo:warning=src/CDeps/Redis/redismodule.c:3:25: warning: ‘DBType’ initialized and declared ‘extern’
cargo:warning= extern RedisModuleType *DBType = NULL;
cargo:warning= ^~~~~~
exit code: 0
TARGET = Some("armv7-unknown-linux-gnueabihf")
TARGET = Some("armv7-unknown-linux-gnueabihf")
HOST = Some("armv7-unknown-linux-gnueabihf")
AR_armv7-unknown-linux-gnueabihf = None
AR_armv7_unknown_linux_gnueabihf = None
HOST_AR = None
AR = None
TARGET = Some("armv7-unknown-linux-gnueabihf")
TARGET = Some("armv7-unknown-linux-gnueabihf")
running: "ar" "crs" "/home/pi/redis/modules/rediSQL-master/target/release/build/rediSQL-f6317b3371ca522c/out/libredismodule.a" "/home/pi/redis/modules/rediSQL-master/target/release/build/rediSQL-f6317b3371ca522c/out/src/CDeps/Redis/redismodule.o"
exit code: 0
TARGET = Some("armv7-unknown-linux-gnueabihf")
cargo:rustc-link-lib=static=redismodule
cargo:rustc-link-search=native=/home/pi/redis/modules/rediSQL-master/target/release/build/rediSQL-f6317b3371ca522c/out
TARGET = Some("armv7-unknown-linux-gnueabihf")
OPT_LEVEL = Some("3")
TARGET = Some("armv7-unknown-linux-gnueabihf")
HOST = Some("armv7-unknown-linux-gnueabihf")
TARGET = Some("armv7-unknown-linux-gnueabihf")
TARGET = Some("armv7-unknown-linux-gnueabihf")
HOST = Some("armv7-unknown-linux-gnueabihf")
CC_armv7-unknown-linux-gnueabihf = None
CC_armv7_unknown_linux_gnueabihf = None
HOST_CC = None
CC = None
HOST = Some("armv7-unknown-linux-gnueabihf")
TARGET = Some("armv7-unknown-linux-gnueabihf")
HOST = Some("armv7-unknown-linux-gnueabihf")
CFLAGS_armv7-unknown-linux-gnueabihf = None
CFLAGS_armv7_unknown_linux_gnueabihf = None
HOST_CFLAGS = None
CFLAGS = None
DEBUG = Some("false")
running: "cc" "-O3" "-ffunction-sections" "-fdata-sections" "-fPIC" "-march=armv7-a" "-I" "src/CDeps/SQLite/include" "-Wall" "-Wextra" "-DHAVE_USLEEP=1" "-DNDEBUG=1" "-DHAVE_FDATASYNC=1" "-DSQLITE_THREADSAFE=1" "-DSQLITE_ENABLE_JSON1=1" "-DSQLITE_ENABLE_FTS3=1" "-DSQLITE_ENABLE_FTS4=1" "-DSQLITE_ENABLE_FTS5=1" "-DSQLITE_ENABLE_RTREE=1" "-o" "/home/pi/redis/modules/rediSQL-master/target/release/build/rediSQL-f6317b3371ca522c/out/src/CDeps/SQLite/sqlite3.o" "-c" "src/CDeps/SQLite/sqlite3.c"
exit code: 0
TARGET = Some("armv7-unknown-linux-gnueabihf")
TARGET = Some("armv7-unknown-linux-gnueabihf")
HOST = Some("armv7-unknown-linux-gnueabihf")
AR_armv7-unknown-linux-gnueabihf = None
AR_armv7_unknown_linux_gnueabihf = None
HOST_AR = None
AR = None
TARGET = Some("armv7-unknown-linux-gnueabihf")
TARGET = Some("armv7-unknown-linux-gnueabihf")
running: "ar" "crs" "/home/pi/redis/modules/rediSQL-master/target/release/build/rediSQL-f6317b3371ca522c/out/libsqlite3.a" "/home/pi/redis/modules/rediSQL-master/target/release/build/rediSQL-f6317b3371ca522c/out/src/CDeps/SQLite/sqlite3.o"
exit code: 0
TARGET = Some("armv7-unknown-linux-gnueabihf")
cargo:rustc-link-lib=static=sqlite3
cargo:rustc-link-search=native=/home/pi/redis/modules/rediSQL-master/target/release/build/rediSQL-f6317b3371ca522c/out

--- stderr
thread 'main' panicked at 'Unable to find libclang: "couldn't find any of ['libclang.so', 'libclang.so.*'], set the LIBCLANG_PATH environment variable to a path where one of these files can be found (skipped: [])"', /checkout/src/libcore/result.rs:906:4
note: Run with RUST_BACKTRACE=1 for a backtrace.

redisql statistics url access error

when i start the redisql , sometimes the error occurs.
The error caused the redis to crash.
Can i skip the request?

[2019-07-05T09:19:38Z WARN  telemetrics] Error in making the request to https://telemetrics.redisql.com/v0/statistics: https://telemetrics.redisql.com/v0/statistics: timed out
[2019-07-05T09:19:38Z DEBUG tokio_reactor] dropping I/O source: 0
[2019-07-05T09:19:38Z DEBUG hyper::client::connect::dns] resolving host="telemetrics.redisql.redbeardlab.com"
[2019-07-05T09:19:38Z DEBUG hyper::client::connect::http] connecting to 51.15.142.13:443
[2019-07-05T09:19:38Z DEBUG tokio_reactor] adding I/O source: 4194304
[2019-07-05T09:19:38Z DEBUG tokio_reactor::registration] scheduling Write for: 0
[2019-07-05T09:19:39Z DEBUG tokio_reactor] dropping I/O source: 0
[2019-07-05T09:19:39Z DEBUG hyper::client::connect::http] connecting to [2001:bc8:4400:2c00::2b:c17]:443
[2019-07-05T09:19:39Z WARN  telemetrics] Error in making the request to https://telemetrics.redisql.redbeardlab.com/v0/statistics: https://telemetrics.redisql.redbeardlab.com/v0/statistics: error trying to connect: Network is unreachable (os error 101)
[2019-07-05T09:19:39Z DEBUG hyper::client::connect::dns] resolving host="telemetrics.redisql.redbeardlab.com"
[2019-07-05T09:19:39Z DEBUG hyper::client::connect::http] connecting to 51.15.142.13:80
[2019-07-05T09:19:39Z DEBUG tokio_reactor] adding I/O source: 8388608
[2019-07-05T09:19:39Z DEBUG tokio_reactor::registration] scheduling Write for: 0
[2019-07-05T09:19:39Z DEBUG hyper::client::connect::http] connected to Some(V4(51.15.142.13:80))
[2019-07-05T09:19:39Z DEBUG tokio_reactor::registration] scheduling Read for: 0
[2019-07-05T09:19:39Z DEBUG tokio_reactor::registration] scheduling Read for: 0
[2019-07-05T09:19:39Z DEBUG hyper::proto::h1::io] flushed 160 bytes
[2019-07-05T09:19:39Z DEBUG hyper::proto::h1::io] flushed 717 bytes
[2019-07-05T09:19:40Z DEBUG hyper::proto::h1::io] read 329 bytes
[2019-07-05T09:19:40Z DEBUG hyper::proto::h1::io] parsed 6 headers
[2019-07-05T09:19:40Z DEBUG hyper::proto::h1::conn] incoming body is chunked encoding
[2019-07-05T09:19:40Z DEBUG hyper::proto::h1::decode] incoming chunked header: 0x84 (132 bytes)
[2019-07-05T09:19:40Z DEBUG reqwest::async_impl::response] Response: '404 Not Found' for http://telemetrics.redisql.redbeardlab.com/v0/statistics
[2019-07-05T09:19:40Z WARN  telemetrics] Return error code from http://telemetrics.redisql.redbeardlab.com/v0/statistics
[2019-07-05T09:19:40Z DEBUG tokio_reactor] dropping I/O source: 0
[2019-07-05T09:19:40Z WARN  telemetrics] Warning, impossible to send the telemetrics.
[2019-07-05T09:19:40Z ERROR telemetrics] Telemetrics not reachables, exit!

invoke from Java client

Is it possible to use this from java client? I noticed the sample python code redis_pool.execute_command("REDISQL.EXEC", command), wondering if there is similar thing in java

make error

rediSQL-master]# make
gcc -c -Wpedantic -Wall -fPIC -IRedisModulesSDK/ -Bstatic rediSQL.c -o rediSQL.o
cc1: error: unrecognized command line option "-Wpedantic"
make: *** [all] Error 1

Projection, please help

I believe there is a lot of interest into mixing futher Redis and SQLite.

I want to create an abstraction called Projection that will contains the best of both Redis AND SQL.

We project on a table.

Given the table

CREATE TABLE user VALUES(
    ID PRIMARY KEY, 
    username TEXT, 
    points INT)

We can use that table as a standard SQL table but also project its values on the redis key space.

We could access the values of username or points using the following syntax:
REDISQL.GET user:ID:username or REDISQL.GET user:ID:points

Then we could also set values using a similar syntax:
REDISQL.SET user:ID:username "siscia" or REDISQL.SET user:ID:points 12 this changes will be reported into the SQL table.

> SELECT points FROM user WHERE username = 'siscia';
12

And of course it will work the other way around.

REDISQL.EXEC DB "UPDATE user SET points = 24 WHERE username = 'siscia' and ID = 1234"
> OK
REDISQL.GET username:1234:points
> 24

The syntax that for now seems to make sense for me to create a projection is something like:

REDISQL.PROJECT DB table

after this command the SQL table and the redis key space should appear in sync.
Then it would be possible to remove the projection with

REDISQL.REMOVE_PROJECTION table

Clearly this means that tables with the same name in different databases cannot be projected at the same time.
This could be easily fixed adding as prefix, the name of the database, but I believe it will bring more problem than benefits.

Can't load module from libredis_sql.so: server aborting

I downloaded the so file from https://plasso.com/s/epp4GbsJdp-redisql and then reported the following error when starting the server. How do I handle this?
thanks

$ redis-server --loadmodule libredis_sql.so
665:C 09 Apr 18:25:16.900 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
665:C 09 Apr 18:25:16.901 # Redis version=4.0.9, bits=64, commit=00000000, modified=0, pid=665, just started
665:C 09 Apr 18:25:16.901 # Configuration loaded
665:M 09 Apr 18:25:16.902 * Increased maximum number of open files to 10032 (it was originally set to 1024).
                _._
           _.-``__ ''-._
      _.-``    `.  `_.  ''-._           Redis 4.0.9 (00000000/0) 64 bit
  .-`` .-```.  ```\/    _.,_ ''-._
 (    '      ,       .-`  | `,    )     Running in standalone mode
 |`-._`-...-` __...-.``-._|'` _.-'|     Port: 6379
 |    `-._   `._    /     _.-'    |     PID: 665
  `-._    `-._  `-./  _.-'    _.-'
 |`-._`-._    `-.__.-'    _.-'_.-'|
 |    `-._`-._        _.-'_.-'    |           http://redis.io
  `-._    `-._`-.__.-'_.-'    _.-'
 |`-._`-._    `-.__.-'    _.-'_.-'|
 |    `-._`-._        _.-'_.-'    |
  `-._    `-._`-.__.-'_.-'    _.-'
      `-._    `-.__.-'    _.-'
          `-._        _.-'
              `-.__.-'

665:M 09 Apr 18:25:16.904 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
665:M 09 Apr 18:25:16.904 # Server initialized
665:M 09 Apr 18:25:16.904 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
665:M 09 Apr 18:25:16.904 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
665:M 09 Apr 18:25:16.905 # Module libredis_sql.so failed to load: libredis_sql.so: cannot open shared object file: No such file or directory
665:M 09 Apr 18:25:16.905 # Can't load module from libredis_sql.so: server aborting


$ ll libredis_sql.so
-rwxrwxrwx 1 vagrant vagrant 4041448 Apr  9 17:15 libredis_sql.so*

redisql not support transaction?

redis shutdown because of the transaction.

my code:
Transaction transaction = jedis.multi();
transaction.executeSQL("test", "select * from op_driver");
transaction.executeSQL("test", "select count(*) from op_driver");
List exec = transaction.exec();
transaction.close();

my client error:
redis.clients.jedis.exceptions.JedisConnectionException: Unexpected end of stream.
at redis.clients.jedis.util.RedisInputStream.ensureFill(RedisInputStream.java:202)
at redis.clients.jedis.util.RedisInputStream.readByte(RedisInputStream.java:43)
at redis.clients.jedis.Protocol.process(Protocol.java:154)
at redis.clients.jedis.Protocol.read(Protocol.java:219)
at redis.clients.jedis.Connection.readProtocolWithCheckingBroken(Connection.java:313)
at redis.clients.jedis.Connection.getRawObjectMultiBulkReply(Connection.java:280)
at redis.clients.jedis.Connection.getObjectMultiBulkReply(Connection.java:284)
at redis.clients.jedis.Jedis.executeSQL(Jedis.java:3661)

redisql server log:
[2019-07-05T09:26:57Z DEBUG redisql_lib::redis] Loop iteration
[2019-07-05T09:26:57Z DEBUG redis_sql::commands] Exec | GotDB
[2019-07-05T09:26:57Z DEBUG redis_sql::commands] Exec | BlockedClient
[2019-07-05T09:26:57Z DEBUG redis_sql::commands] Exec | Create Command
[2019-07-05T09:26:57Z DEBUG redis_sql::commands] Exec | GotDB
[2019-07-05T09:26:57Z DEBUG redis_sql::commands] Exec | BlockedClient
[2019-07-05T09:26:57Z DEBUG redis_sql::commands] Exec | Create Command
[2019-07-05T09:26:57Z DEBUG redisql_lib::redis] Exec | Query = "select * from op_driver"
Segmentation fault (core dumped)

Failed to keep master and slave db in sync

The setup is pretty simple : one instance running as master and the other running as slave. Use redis-cli to create a table on master , insert a few entries and try to read them from slave. To read data from slave, "slave-read-only" was set to "no". However, it seems the latest entries didn't sync from master to slave, and the only way to keep them in sync again is to restart the slave, which cause full sync.

Note that AOF is disabled in both master and slave.

Why do you choose Rust?

In my opinion,the C language is the best choice to write a Reis Module,many other Redis modules are written by C language ,why do you choose Rust?
It's just my question, maybe you are good at Rust, it' wonderful too.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.