eddelbuettel / rcppredis Goto Github PK
View Code? Open in Web Editor NEWR interface to Redis using the hiredis library
License: GNU General Public License v2.0
R interface to Redis using the hiredis library
License: GNU General Public License v2.0
@bwlewis tells me that 0.1.4 can be subject to a seg.fault issue fixed in 1.0.0 upstream.
While that sounds promising, two caveats:
On the other hand, per #42 and thanks to @opencpu we have been using 1.0.0 for windows builds. So upgrade should be painless.
I have coded a shiny app that runs perfectly locally but when I try to deploy it to shinyapps.io I get the following output:
Preparing to deploy application...DONE
Uploading bundle for application: 115430...DONE
Deploying bundle: 549719 for application: 115430 ...
Waiting for task: 242805727
building: Parsing manifest
building: Building image: 540950
building: Fetching packages
building: Building package: RcppRedis
################################ Begin Task Log ################################
[2016-09-06T09:42:13.614224848+0000] Installing R package: RApiSerialize (0.1.0)
* installing to library ‘/usr/local/lib/R/site-library’
* installing *binary* package ‘RApiSerialize’ ...
* DONE (RApiSerialize)
(repeated 22 times for all other required packages)
[2016-09-06T09:42:37.199855491+0000] Building R package: RcppRedis (0.1.7)
/mnt/packages/build /mnt
* installing to library ‘/usr/local/lib/R/site-library’
* installing *source* package ‘RcppRedis’ ...
** package ‘RcppRedis’ successfully unpacked and MD5 sums checked
checking for g++... g++
checking whether the C++ compiler works... yes
checking for C++ compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C++ compiler... yes
checking whether g++ accepts -g... yes
checking how to run the C++ preprocessor... g++ -E
checking whether we are using the GNU C++ compiler... (cached) yes
checking whether g++ accepts -g... (cached) yes
checking for pkg-config... yes
checking for grep that handles long lines and -e... /bin/grep
checking for egrep... /bin/grep -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking hiredis/hiredis.h usability... no
checking hiredis/hiredis.h presence... no
checking for hiredis/hiredis.h... no
configure: WARNING: Hiredis headers not found with via default CXXFLAGS and CPPFLAGS, trying /usr/local/include
configure: WARNING: Unsetting ac_cv_header_hiredis_hiredis_h
checking hiredis/hiredis.h usability... no
checking hiredis/hiredis.h presence... no
checking for hiredis/hiredis.h... no
configure: error: ERROR: hiredis headers required; use '-Iincludedir' in CXXFLAGS for unusual locations.
ERROR: configuration failed for package ‘RcppRedis’
* removing ‘/usr/local/lib/R/site-library/RcppRedis’
################################# End Task Log #################################
Error: Unhandled Exception: Child Task 242805728 failed: Error building image: Error building RcppRedis (0.1.7). Build exited with non-zero status: 1
Any ideas on what could be causing this and how to fix it? Any help would be much appreciated...
That should allow us to retire use of package BH.
One problem I rediscovered with #11 is that when an auth provided password is provided, then is echoed to the console. In redisConnect I handled that by wrapping the constructor in base::capture.output(), but that isn't applicable now. Any ideas on how to suppress it?
Hi, I tried to use this package on MacOS. Installed hiredis
via brew, but cannot connect.
#R> redis = new (RcppRedis::Redis)
Error in new_CppObject_xp(fields$.module, fields$.pointer, ...) :
Redis connection error: Connection refused
Redis keeps adding new functions, e.g. PEXPIRE is only available since 2.6.0. We should probably check the version of the Redis server we connect to and disallow calls to functions that aren't available. Preferably, we would throw a warning. Ideally we'd simulate the desired effect. For example, a call to PEXPIRE could just round to the nearest full second and call EXPIRE with a warning.
The Status statement in README.md terminates abruptly mid-word.
redis$execv(c("LRANGE mylist 0 elephant")) returns the text error message from Redis but not an error in R. See also, ropensci-archive/RedisAPI#3
It might be possible to process the Redis documentation JSON to create a full set of API functions. We could avoid collision with existing functionality by making all calls all capitals. However, because Cpp has strict typing, it might be a lot easier to write in a way that works by having the code that provides those functions written in R; that might involve encapsulating the RcppRedis constructor in some R code so that we can add functions to the resulting object. All such functions would probably dispatch to execv in the Cpp code.
it would be better if redis connection could auto reconnect after fork when redis is used as shared memory for multicore style parallel computing.
A daughter issue to #17.
There is no way currently to cleanly close a redis connection. Some combination of CLIENT SETNAME/GETNAME and CLIENT KILL could avoid this. In practice, it probably isn't very important to solve because we tend to build a redis connection and to keep using it.
In other words make it .listen()
and also drop the help page? Is it useful for users or just used inside redisMonitorChannels()
?
Minimal, contrived, example:
system("redis-server --port 7777", wait=FALSE)
r <- new(RcppRedis::Redis, "localhost", 7777)
r$ping()
r$exec("SHUTDOWN")
will crash R with
> r$exec("SHUTDOWN")
[15435] 23 Nov 10:54:23.121 # User requested shutdown...
[15435] 23 Nov 10:54:23.122 # Redis is now ready to exit, bye bye...
*** caught segfault ***
address (nil), cause 'memory not mapped'
Traceback:
1: .External(list(name = "CppMethod__invoke_notvoid", address = <pointer: 0x2fab870>, dll = list(name = "Rcpp", path = "/home/rich/lib/R/library/Rcpp/libs/Rcpp.so", dynamicLookup = TRUE, handle = <pointer: 0x35d8390>, info = <pointer: 0x7fc132594c40>), numParameters = -1L), <pointer: 0x271b430>, <pointer: 0x362b4a0>, .pointer, ...)
2: r$exec("SHUTDOWN")
Possible actions:
1: abort (with core dump, if enabled)
2: normal R exit
3: exit R without saving workspace
4: exit R saving workspace
Seen on various flavours of OSX and Linux. The gdb backtrace is:
(gdb) bt
#0 Redis::extract_reply (this=0xef74a0, reply=0x0) at Redis.cpp:76
#1 0x00007ffff2d54884 in Redis::exec (this=0xef74a0, cmd=...) at Redis.cpp:157
#2 0x00007ffff2d531e3 in Rcpp::CppMethod1<Redis, SEXPREC*, std::string>::operator() (this=this@entry=0x1d6dcd0, object=0xef74a0, args=args@entry=0x7fffffffbde0)
at /home/rich/lib/R/library/Rcpp/include/Rcpp/module/Module_generated_CppMethod.h:111
#3 0x00007ffff2d5b25f in Rcpp::class_<Redis>::invoke_notvoid (this=<optimised out>, method_xp=<optimised out>, object=0x2007a60, args=0x7fffffffbde0,
nargs=<optimised out>) at /home/rich/lib/R/library/Rcpp/include/Rcpp/module/class.h:234
#4 0x00007ffff348b1ec in CppMethod__invoke_notvoid (args=<optimised out>) at Module.cpp:220
#5 0x00007ffff78f1161 in ?? () from /usr/lib/R/lib/libR.so
#6 0x00007ffff79369db in Rf_eval () from /usr/lib/R/lib/libR.so
#7 0x00007ffff7938b60 in ?? () from /usr/lib/R/lib/libR.so
#8 0x00007ffff79367e3 in Rf_eval () from /usr/lib/R/lib/libR.so
#9 0x00007ffff7937b6f in Rf_applyClosure () from /usr/lib/R/lib/libR.so
#10 0x00007ffff79365bf in Rf_eval () from /usr/lib/R/lib/libR.so
#11 0x00007ffff795da32 in Rf_ReplIteration () from /usr/lib/R/lib/libR.so
#12 0x00007ffff795dd81 in ?? () from /usr/lib/R/lib/libR.so
#13 0x00007ffff795de34 in run_Rmainloop () from /usr/lib/R/lib/libR.so
#14 0x00000000004007eb in main ()
#15 0x00007ffff7261ec5 in __libc_start_main (main=0x4007d0 <main>, argc=1, argv=0x7fffffffde78, init=<optimised out>, fini=<optimised out>,
rtld_fini=<optimised out>, stack_end=0x7fffffffde68) at libc-start.c:287
#16 0x000000000040081b in _start ()
A fix would be to add something like:
if (reply == NULL) {
error("Failure communicating with the Redis server");
}
after every redisCommand
/ redisCommandArgv
in src/Redis.cpp
While the above is through Redis server shutdown the same thing happens if the connection is lost (e.g. client and server on different machines on different hosts and a loss of the link between). I have confirmed this with toxiproxy.
... would be nice to have. Maybe even conditionally if a msgpack-c header is found.
The version packaged with Ubuntu seems woefully behind though. May need RcppMsgPack just to package the headers.
Hi,
I was trying to install it using devtools
library("devtools")
install()
but I get the following error message :
[...[
/usr/bin/ld: RcppExports.o: relocation R_X86_64_32 against `.rodata.str1.1' can not be used when making a shared object; recompile with -fPIC
RcppExports.o: error adding symbols: Bad value
collect2: error: ld returned 1 exit status
make: *** [RcppRedis.so] Error 1
ERROR: compilation failed for package ‘RcppRedis’
Thanks
Observed behavior: Sometimes RcppRedis does not obey the timeout and locks the R process.
Desired behavior: RcppRedis obeys timeout and produces error
Minimal example, something like:
library(RcppRedis)
options(redisRAMCache.host = "hostWithRunningRedisServer.com")
options(redisRAMCache.port=6379)
redisRAMCache <- new(Redis, getOption("redisRAMCache.host"), getOption("redisRAMCache.port"),"",2.0)
options(redisREcho.host = "hostWithoutRedisServer.com")
options(redisREcho.port=6379)
try(redisREcho <- new(Redis, getOption("redisREcho.host"),getOption("redisREcho.port"),"",0.5),silent=TRUE)
Except it seems like this edge case only gets hit sometimes and not in a way that is fully reproducable. This is a stub if anyone else encounters similar behavior perhaps they can narrow it down further.
It seems specific to the case where the 2nd server isn't there and may require that the first server have established a connection already.
When the timeout is above 1, I've never been able to replicate the issue.
Hi,
Attempting a devtools install on Ubuntu 15.10 amd64. Seems to build fine and then barfs during package load test...
devtools::install_github('eddelbuettel/RcppRedis')
Downloading GitHub repo eddelbuettel/RcppRedis@master
Installing RcppRedis
'/usr/lib/R/bin/R' --no-site-file --no-environ --no-save --no-restore CMD INSTALL
'/tmp/RtmpxdQHRv/devtools3f174bdccb2e/eddelbuettel-rcppredis-e625370'
--library='/home/br00t/R/x86_64-pc-linux-gnu-library/3.2' --install-tests
installing source package ‘RcppRedis’ ...
checking for g++... g++
checking whether the C++ compiler works... yes
checking for C++ compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C++ compiler... yes
checking whether g++ accepts -g... yes
checking how to run the C++ preprocessor... g++ -E
checking whether we are using the GNU C++ compiler... (cached) yes
checking whether g++ accepts -g... (cached) yes
checking for pkg-config... yes
checking for grep that handles long lines and -e... /bin/grep
checking for egrep... /bin/grep -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking hiredis/hiredis.h usability... yes
checking hiredis/hiredis.h presence... yes
checking for hiredis/hiredis.h... yes
configure: creating ./config.status
config.status: creating src/Makevars
Completed configuration and ready to build.
** libs
g++ -I/usr/share/R/include -DNDEBUG -I"/home/br00t/R/x86_64-pc-linux-gnu-library/3.2/Rcpp/include" -I"/home/br00t/R/x86_64-pc-linux-gnu-library/3.2/RApiSerialize/include" -D_FILE_OFFSET_BITS=64 -I/usr/include/hiredis -fpic -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -g -c Redis.cpp -o Redis.o
g++ -shared -L/usr/lib/R/lib -Wl,-Bsymbolic-functions -Wl,-z,relro -o RcppRedis.so Redis.o -lhiredis -L/usr/lib/R/lib -lR
installing to /home/br00t/R/x86_64-pc-linux-gnu-library/3.2/RcppRedis/libs
** R
** demo
** inst
** tests
** preparing package for lazy loading
** help
*** installing help indices
** building package indices
** testing if installed package can be loaded
*** caught segfault ***
address 0x20, cause 'memory not mapped'
Traceback:
1: .Call(Module__classes_info, xp)
2: Module(module, mustStart = TRUE, where = env)
3: doTryCatch(return(expr), name, parentenv, handler)
4: tryCatchOne(expr, names, parentenv, handlers[[1L]])
5: tryCatchList(expr, classes, parentenv, handlers)
6: tryCatch(Module(module, mustStart = TRUE, where = env), error = function(e) e)
7: loadModule(module = "Redis", what = TRUE, env = ns, loadNow = TRUE)
8: (function (ns) loadModule(module = "Redis", what = TRUE, env = ns, loadNow = TRUE))()
9: doTryCatch(return(expr), name, parentenv, handler)
10: tryCatchOne(expr, names, parentenv, handlers[[1L]])
11: tryCatchList(expr, classes, parentenv, handlers)
12: tryCatch((function (ns) loadModule(module = "Redis", what = TRUE, env = ns, loadNow = TRUE))(), error = function(e) e)
13: eval(expr, envir, enclos)
14: eval(substitute(tryCatch(FUN(WHERE), error = function(e) e), list(FUN = f, WHERE = where)), where)
15: .doLoadActions(where, attach)
16: methods:::cacheMetaData(ns, TRUE, ns)
17: loadNamespace(package, c(which.lib.loc, lib.loc))
18: doTryCatch(return(expr), name, parentenv, handler)
19: tryCatchOne(expr, names, parentenv, handlers[[1L]])
20: tryCatchList(expr, classes, parentenv, handlers)
21: tryCatch(expr, error = function(e) { call <- conditionCall(e) if (!is.null(call)) { if (identical(call[[1L]], quote(doTryCatch))) call <- sys.call(-4L) dcall <- deparse(call)[1L] prefix <- paste("Error in", dcall, ": ") LONG <- 75L msg <- conditionMessage(e) sm <- strsplit(msg, "\n")[[1L]] w <- 14L + nchar(dcall, type = "w") + nchar(sm[1L], type = "w") if (is.na(w)) w <- 14L + nchar(dcall, type = "b") + nchar(sm[1L], type = "b") if (w > LONG) prefix <- paste0(prefix, "\n ") } else prefix <- "Error : " msg <- paste0(prefix, conditionMessage(e), "\n") .Internal(seterrmessage(msg[1L])) if (!silent && identical(getOption("show.error.messages"), TRUE)) { cat(msg, file = stderr()) .Internal(printDeferredWarnings()) } invisible(structure(msg, class = "try-error", condition = e))})
22: try({ ns <- loadNamespace(package, c(which.lib.loc, lib.loc)) env <- attachNamespace(ns, pos = pos, deps)})
23: library(pkg_name, lib.loc = lib, character.only = TRUE, logical.return = TRUE)
24: withCallingHandlers(expr, packageStartupMessage = function(c) invokeRestart("muffleMessage"))
25: suppressPackageStartupMessages(library(pkg_name, lib.loc = lib, character.only = TRUE, logical.return = TRUE))
26: doTryCatch(return(expr), name, parentenv, handler)
27: tryCatchOne(expr, names, parentenv, handlers[[1L]])
28: tryCatchList(expr, classes, parentenv, handlers)
29: tryCatch(expr, error = function(e) { call <- conditionCall(e) if (!is.null(call)) { if (identical(call[[1L]], quote(doTryCatch))) call <- sys.call(-4L) dcall <- deparse(call)[1L] prefix <- paste("Error in", dcall, ": ") LONG <- 75L msg <- conditionMessage(e) sm <- strsplit(msg, "\n")[[1L]] w <- 14L + nchar(dcall, type = "w") + nchar(sm[1L], type = "w") if (is.na(w)) w <- 14L + nchar(dcall, type = "b") + nchar(sm[1L], type = "b") if (w > LONG) prefix <- paste0(prefix, "\n ") } else prefix <- "Error : " msg <- paste0(prefix, conditionMessage(e), "\n") .Internal(seterrmessage(msg[1L])) if (!silent && identical(getOption("show.error.messages"), TRUE)) { cat(msg, file = stderr()) .Internal(printDeferredWarnings()) } invisible(structure(msg, class = "try-error", condition = e))})
30: try(suppressPackageStartupMessages(library(pkg_name, lib.loc = lib, character.only = TRUE, logical.return = TRUE)))
31: tools:::.test_load_package("RcppRedis", "/home/br00t/R/x86_64-pc-linux-gnu-library/3.2")
aborting ...
Segmentation fault (core dumped)
ERROR: loading failed
sessionInfo()
R version 3.2.2 (2015-08-14)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 15.10
locale:
[1] LC_CTYPE=en_CA.UTF-8 LC_NUMERIC=C LC_TIME=en_CA.UTF-8
[4] LC_COLLATE=en_CA.UTF-8 LC_MONETARY=en_CA.UTF-8 LC_MESSAGES=en_CA.UTF-8
[7] LC_PAPER=en_CA.UTF-8 LC_NAME=C LC_ADDRESS=C
[10] LC_TELEPHONE=C LC_MEASUREMENT=en_CA.UTF-8 LC_IDENTIFICATION=C
attached base packages:
[1] stats graphics grDevices utils datasets methods base
loaded via a namespace (and not attached):
[1] tools_3.2.2
ticket for imminent PR.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.