nekipelov / redisclient Goto Github PK

I'm trying to write a program that subscribes to a keyspace change pattern. If it detects that a key has changed, I'd like to know which specific key it was. I began with redisclient-0.6.1/examples/async_pubsub2.cpp then simplified it to

#include <string>
#include <iostream>
#include <functional>
#include <boost/asio/ip/address.hpp>
#include <redisclient/redisasyncclient.h>

static const std::string channelName = "__keyspace@0__:x*";

class Client {
public:
    Client(boost::asio::io_service &ioService)
        : ioService(ioService) {}

    void onMessage(const std::vector<char> &buf) {
        std::string msg(buf.begin(), buf.end());
        std::cerr << "Message: " << msg << std::endl;
        if( msg == "stop" )
            ioService.stop();
    }

private:
    boost::asio::io_service &ioService;
};

int main(int, char **) {
    boost::asio::ip::address address = boost::asio::ip::address::from_string("127.0.0.1");
    const unsigned short port = 6379;
    boost::asio::ip::tcp::endpoint endpoint(address, port);

    boost::asio::io_service ioService;
    redisclient::RedisAsyncClient subscriber(ioService);
    Client client(ioService);

    subscriber.connect(endpoint, [&](boost::system::error_code ec) {
        if( ec ) {
            std::cerr << "Can't connect to redis: " << ec.message() << std::endl;
        } else {
            subscriber.psubscribe(channelName,
                    std::bind(&Client::onMessage, &client, std::placeholders::_1)
                    );
        }
    });
    ioService.run();
    return 0;
}

which subscribes to changes to any key beginning with x. It half-works: when I change the value of key xy (manually, using redis-cli from another terminal), the onMessage() method in the program above is called, but msg just contains "set". But which key was set--xy or xz or .... ?

How should the code be changed so that it also captures the name(s) of the changed key(s)?

Hi, thanks for the great work on this.
I am wondering, if it's possible to support pipelining of commands, similar to what the hiredis (https://github.com/redis/hiredis#pipelining) supports?
What I'm looking for is a way to perform a lot of hmget (or hgetall) commands. Currently if I understand correctly, the only way to do this is to chain callbacks for each command with the next command. Is there any other way that you can suggest to achieve this?

Thanks a lot,
Greg

Thanks for posting your code! I'm adding persistence to a Boost::Asio project, and this is exactly what I needed.

To better facilitate using your library from my code, I made a small change to your Makefile so that the OBJS appear in a shared object library. I'm using Gnu make's target-specific variables feature, but I believe this is in common widespread use. I've tested with GNU Make 3.82 and g++ (GCC) 4.8.2

--- a/Makefile
+++ b/Makefile
@@ -1,35 +1,39 @@
 SRCS = redisclient.cpp redisvalue.cpp redisparser.cpp
 HDRS = redisclient.h redisvalie.h redisparser.h
 OBJS = $(SRCS:.cpp=.o)
+MYLIBNAME = redisclient
+MYLIB = lib$(MYLIBNAME).so
 LIBS = -pthread -lboost_system
 LDFLAGS = -g
 CXXFLAGS = -g -O2 -std=c++0x -Wall -Wextra

 all: examples
 examples: example1 example2 example3 example4 parsertest

-example1: $(OBJS) example1.o
-       g++ $^ -o $@  $(LDFLAGS) $(LIBS)
+$(MYLIB): CXXFLAGS += -fPIC
+$(MYLIB): LDFLAGS += -shared
+$(MYLIB): $(OBJS)
+       $(CXX) $^ -o $@ $(LDFLAGS)

-example2: $(OBJS) example2.o
-       g++ $^ -o $@  $(LDFLAGS) $(LIBS)
+example1: example1.o $(MYLIB)
+       $(CXX) $^ -o $@  $(LDFLAGS) $(LIBS)

-example3: $(OBJS) example3.o
-       g++ $^ -o $@  $(LDFLAGS) $(LIBS)
+example2: example2.o $(MYLIB)
+       $(CXX) $^ -o $@  $(LDFLAGS) $(LIBS)

-example4: $(OBJS) example4.o
-       g++ $^ -o $@  $(LDFLAGS) $(LIBS)
+example3: example3.o $(MYLIB)
+       $(CXX) $^ -o $@  $(LDFLAGS) $(LIBS)

-parsertest: $(OBJS) parsertest.o
-       g++ $^ -o $@  $(LDFLAGS) $(LIBS) -lboost_unit_test_framework
+example4: example4.o $(MYLIB)
+       $(CXX) $^ -o $@  $(LDFLAGS) $(LIBS)
+
+parsertest: parsertest.o $(MYLIB)
+       $(CXX) $^ -o $@  $(LDFLAGS) $(LIBS) -lboost_unit_test_framework

 test: parsertest
-       ./parsertest
+       LD_LIBRARY_PATH=$(shell pwd) ./parsertest

-.cpp.o:
-       g++ $(CXXFLAGS) -c $< -o $@
-
 clean:
-       rm -f $(OBJS) example1.o example1 example2.o example2 example3.o example3 example4
+       rm -f $(MYLIB) $(OBJS) example1.o example1 example2.o example2 example3.o example3

 .PHONY: all examples clean

In the example,async_pubsub.cpp

the only callback function is onMessage , and the only parameter is the value ,So!!!
how can I know the channel when I'm using psubscribe("*")

How about to move all the staff to kind of "namespace Redis"? For now almost all classes starts with Redis anyway, and namespaces was actually designed to avoid such things.

Hello,

It would be very practical to provide a version of RedisBuffer that would store a copy of the data, not only pointer/reference to it.

Pointers/references are great to avoid copying, however in the asynchronous case they are an enormous pain to manage. The caller needs to maintain a std::deque of the arguments until completely sure that the request has been executed and the data are not needed anymore. That makes the asynchronous programming much more difficult and bug prone than it needs to be.

The argument lists is typically converted from some other data when preparing the Redis command anyway, so not copying it doesn't really save anything. If RedisBuffer allowed to copy the data (and thus acted as a true buffer instead of a wrapper), the application wouldn't need to store any temporary data and it would allow a true "fire & forget" Redis command execution.

Could the SUBSCRIBE command be changed to PSUBSCRIBE, please? Or provide a second/alternative function. Same for UNSUBSCRIBE vs PUNSUBSCRIBE.

PSUBSCRIBE allows for the use of wildcard patterns, this is important for monitoring large parts of the key space for changes, for example, without having to explicitly subscribe to every single key possible.

From the documentation it seems to me that it would be safe to simply change SUBSCRIBE to PSUBSCRIBE and UNUSBSCRIBE to PUNSUBSCRIBE - normal channel names without the globbing characters (?*) work as patterns too.

I have a question about the policy of invoking callback handler.

When I call RedisAsyncClient::command() during disconnected, callback function doesn't seems to be called. Is that intentional specification?

Here is the small example code that demonstrates the behavior.
I've tested it with f55ffdc.

#include <string>
#include <iostream>

#include <boost/asio/ip/address.hpp>

#include <redisclient/redisasyncclient.h>

int main() {
    boost::asio::ip::address address = boost::asio::ip::address::from_string("127.0.0.1");
    const unsigned short port = 6379;
    boost::asio::ip::tcp::endpoint endpoint(address, port);

    boost::asio::io_service ioService;
    redisclient::RedisAsyncClient redis(ioService);
    boost::system::error_code ec;

    redis.connect(
        endpoint,
        [&]
        (auto const& ec) {
            if(ec) {
                std::cerr << __LINE__ <<
                    ": Can't connect to redis: " << ec.message() << std::endl;
                return;
            }
            std::cout << __LINE__ <<
                ": Please restart redis service within 5 seconds for testing!" << std::endl;
            sleep(5);

            std::cout << __LINE__ <<
                ": do async command" << std::endl;
            redis.command(
                "SET",
                {"key", "value"},
                [&]
                (auto const& result) {
                    if( result.isError() )
                    {
                        std::cerr << __LINE__ <<
                            ": SET error: " << result.toString() << "\n";
                        return ;
                    }
                    std::cout << __LINE__ <<
                        ": result: " << result.toString() << std::endl;
                }
            );
            std::cout << __LINE__ <<
                ": finish async command" << std::endl;
        }
    );

    std::function<void(boost::system::error_code const&)> handler;

    handler =
        [&handler, &redis, &endpoint]
        (boost::system::error_code const& ec) {
        if (ec) {
            std::cout << __LINE__ <<
                ": ec: " << ec << std::endl;
            redis.disconnect();
            std::cout << __LINE__ <<
                ": trying reconnect inner..." << std::endl;
            sleep(1);
            redis.connect(
                endpoint,
                handler);
        }
        else {
            std::cout << __LINE__ <<
                ": reconnect success" << std::endl;
        }
    };

    redis.installErrorHandler(
        [&](std::string const& ec) {
            std::cout << __LINE__ <<
                ": ec: " << ec << std::endl;
            std::cout << __LINE__ <<
                ": trying reconnect..." << std::endl;
            redis.disconnect();
            redis.connect(
                endpoint,
                handler
            );
        }
    );

    ioService.run();
}

When I run the program, the following message is appeared:

26: Please restart redis service within 5 seconds for testing!

Then if I do nothing within 5 seconds, I got the following message:

30: do async command
47: finish async command
43: result: OK

The last line (43) indicates that RedisAsyncClient::command() callback is called.

If I restart redis service within 5 seconds, I got the following message:

26: Please restart redis service within 5 seconds for testing!

I do sudo systemctl restart redis.service.

30: do async command
47: finish async command
76: ec: End of file
78: trying reconnect...
69: reconnect success

That indicates that RedisAsyncClient::command() callback is NOT called. But the handler that is installed using Redis::installErrorHandler() is called.

Is that expected behavior?

I want to implement automatic re-execute RedisAsyncClient::command() mechanism if the redis connection is disconnected.

In order to do that, I think that the following mechanism is required:

1. Prepare a queue.
2. Before calling RedisAsyncClient::command(), I copy the contents of the command and push it to the queue.
3. If the RedisAsyncClient::command() callback is called, erase the pushed command. If installed error handler is called, reconnect to redis, and when reconnected, call RedisAsyncClient::command() for each commands in the queue.

Before I implement it in my client program, I'd like to clarify the library spec.

Hi.
First I want to thank you for this great library, we are using it for writing lots of data in our system and it does the job quite well.
Recently we changed a non-efficient loop of 1000 hmget's in our code (each hash contains 3 elements) to a Lua script which returns one big array (size of 3000), and we run this script using your evalsha implementation. Then we parse the result.
After profiling, it turned out that before the change, it took 87ms for the loop to run.
After the change, the evalsha command took ~230ms (results are 3 times worse than the ridiculous 1000 hmget loop solution). It turned out that 99% of the overhead comes from the parseArray method inside the redisparser.cpp file.

Do you plan anytime soon to improve this parser?
We did the same improvement in our C# program (with StackExchange evalsha) and the results there are almost 300 times faster (obviously that's because we do only one redis call instead of 1000 calls, and the StackExchange evalsha command is very efficient.)

Thanks.

error C2039: 'local': is not a member of 'boost::asio'

terminate called after throwing an instance of 'std::runtime_error'
what(): [RedisClient] unexpected message: OK
make: *** [run] Aborted (core dumped)

When I send commands async quickly, redisclient crash with the above error. The error is gone if I pause for a few millisec after each command. Any idea? The code is based on example1.cpp.

The "OK" within "unexpected message: OK" looks like a proper server reply that the driver didn't expect.

I am using the software below:
redis-3.0.0-rc1.tar.gz
centos 7 fully updated
boost 1.53 (comes w/ centos 7)
gcc 482 (comes w/ centos 7)
redisclient version
commit 96e2f60
Date: Thu Nov 13 11:57:16 2014 +0300

==> code.zip.base64 <==
UEsDBBQAAAAIABG8nUVDPMxUNAEAAMADAAAIABwATWFrZWZpbGVVVAkAAxJ0oVQSdKFUdXgLAAEE
6AMAAAQBAgAAlVNNT4NAED0zv2IONIE0u5WvC7GJVqPWkGLSg/WiQbrqAQph16T+e6EwhXQ1oRd4
zL5582aYjZaLNc6RZe9FIdWb/JFK5MdP9VWJZMtyBdHtXXR9f6B+Ios4R1a2p3Cz2QzPYhfZc5Jl
9VPsVZUgW3I+k1XaEB3nlCrVdp5Op47zXxZU37sQlZDqwgWDzzoE7TtE04oXj2u7Y/ACDNOqK9lo
viIr0LzCmtLZtxtYd2wDsUmZp2XZZ1rUk40sRfOyFQKoDYYo9kleZkICgWPIIeAS8Aj4BAKgPKf3
TpGx7nt+X3tcB2RQqz16ciThaRLeuRK+JuGfKxFoEsFoiUkzvok+t25JTyaXZiLZhWBU9fX4+OPP
6UvQx9zBiPTV6GP+oInBwvCnh3j1EmJzP2jp8OAHfgFQSwMEFAAAAAgAjT6eRSIGGPWpBAAAcQ4A
AAoAHAB0ZXN0MDIuY3BwVVQJAAM66aFUPemhVHV4CwABBOgDAAAEAQIAALVWbVPjNhD+XP+KPd80
tTmTF/pyUwf4klLKQHsdctN+9Ci2QjQ4kk9S6FGG/96VZCdK7OQ4ZmqGWF6tVtrnWe3uW8bzclVQ
OFVaMn53HrxdS5hAGSXLbdmScFb5opIp7X+r9qpcsyX1BTOBtgdEMTFg1YAUhaRK9RdV1VaaMV50
z+gF7tOak7RgKi8Z5XrgjfuL8yBQmmiWQy640qB0kabOa7CK1/QRziBccfZpRY+t6PiePh7Tz2RZ
lTQcf2H9X6Rc0ZaFByPFtUFeEqXgbyHvqQyegmo1K1meBoCPE0bWrTQ1sKQpE5mi8oHlFHpMTN0w
gVtjdGJdgp7cfMTWkHlSWKtH61GcgKcctRY+PQf2/SBYAYJPBOc01+ZEpXHXfNAi6fC8R6UU8nek
j9zReOwbmVIduQW3G3h6Fo9txcuXKSotqggFQSXZA9HUYfdl0JyJfcCNg2c0aTdwNKTp6/1HXsFN
52Kl4fQUsuzXq5uLLDPjMA2d6Obqj0a0AdvO2bWUF6U7NJtH8Ga9P9Rcral2O+EJrKkJ4d/p5rSg
hWM8BWvYP6az/Wx/aanojlUPnH4ulnjhiyicXnwME1hflMQL+WS9suup+THXOOptEMbgSEAvmEog
G8Vxc6TnFhWHouhVcKPBNtRfZ8SK7BH6WkxtIERxB32DgWWwpXmGSeLDdVjziVpPzWAv/pc7+PsL
vgL2yy7Y0cxzM1jHw+6ptmPtiqNTyJN1DeZSLLei7WXYdNF9KBe8iu7LLrr3U/PmzE/m3QT9L1A0
o072f7m4CZOXkb9F/J7kaLKXqJJGV9K5VylMVDS8FILrCZE06kh8jhNTOazq8nFaUlpFpcDJpaJ5
bEGrtzAhl7mCjRasYj1TCcU+Z6ZBSNMlK7GboLhZoSJrw57FZf9HnjtYfkNISiyXu2HSgwe351aI
hMVMsX/pmSNjPxHosttIU6Ujr1r0wHUQyQtKjdt/MMh3+JuXK7UgZRnaG8ewBHFsFEbD4fBoNDz5
oRHOSa6FNDNGYpHUQ/zMS5HfR3btXMjIqBIUD8f4OgU+hnfvSFzHqHXJ9GSnHlvnQOSdctHWWJg5
CzO04PY1ZmbxuhB4y11HB/cm71vax7UOSizG+M4swMR+uvGs0XI3w0zY0QFNc8p+hVhlM4Ieo9k+
7h01iaql4C6Xr+IS2S7+GEsaL5BZvcl5tiQCU3AnOMVsAAwqslIUj1YyrUvqJQry7WgY+1mgCXcU
byVRe5Edc6Nt5gqBHR+FQqPYjSM9OtZodgCTmw+T62n258VtNr2YwJENDW+RxHYHl3GcqWNkgIZa
hSvkLsz5VmwH35jz2aug3XyhuxXsNpZP0JXaTZvomombJWHcBBB2Qwsi4ejI5WWomyMnrFt60w2P
Tt73h/g3CseemrFUCWnA+On79z+Pg4PN3IFezlHt5a+x11LDP66z9hpop19rbe9XYZ7KK+tuJcwB
m0HU0qv9S1OT7zN3S6JaiL228SyufapjkdgEVveVjeFkX52uFRPoOQ9Mucb/ExPmW1C5nLoB6KP9
jl5WBeSKJ372qlFpYvvHYRPch/MZuJzZpEmfCjsrqV5JjskGQ+g/UEsBAh4DFAAAAAgAEbydRUM8
zFQ0AQAAwAMAAAgAGAAAAAAAAQAAAO2BAAAAAE1ha2VmaWxlVVQFAAMSdKFUdXgLAAEE6AMAAAQB
AgAAUEsBAh4DFAAAAAgAjT6eRSIGGPWpBAAAcQ4AAAoAGAAAAAAAAQAAAO2BdgEAAHRlc3QwMi5j
cHBVVAUAAzrpoVR1eAsAAQToAwAABAECAABQSwUGAAAAAAIAAgCeAAAAYwYAAAAA

my func is like this:

bool lindex(const std::string& get_key, std::int64_t pos , CallGet&& call_get )
{
redis_client_.command("LINDEX", get_key, pos, boost::bind(&RedisDao::on_lindex, shared_from_this(), call_get, _1));
return true;
}

complete error:

error: invalid conversion from ‘int64_t {aka long int}’ to ‘const char*’ [-fpermissive]

How can I set lindex index position?

Dear Sir,

This library is amazing, and I'm trying to use it in node.native project (node.js c++ port).
since they are both event-driven, I don't know where to put the ioService.run() code. (maybe due to my poor understanding of boost asio), please help:
ioService.run();
return native::run();
The above 2 lines are both blocking function calls.

    int main()
        {
            boost::asio::ip::address redisAddress = boost::asio::ip::address::from_string("127.0.0.1");
        const int redisPort = 6379;

        boost::asio::io_service ioService;
        RedisClient redis(ioService);

        if( !redis.connect(redisAddress, redisPort) )
        {
            std::cerr << "Can't connecto to redis" << std::endl;
            return EXIT_FAILURE;
        }

        http server;
        int port = 8080;
        if(!server.listen("0.0.0.0", port, [&redis, &ioService](request& req, response& res) {
            if ( req.url().path() == "/user/create" ) user::create(req, res, redis, ioService);//some function doing redis access and out http response
            else
            {
                    res.set_status(404);
                    res.end("404 Not Found");
            }

        })) return 1; // Failed to run server.

        std::cout << "Server running at http://0.0.0.0:" << port << "/" << std::endl;
            ioService.run();
        return native::run();
    }

Thanks & Regards,
Simon

I tried to execute the sync_set_get example in a machine (tried to connect to 127.0.0.1) where no Redis is installed and I got the following core dump:

Can't connect to redis:
*** Error in `./sync_set_get': free(): invalid pointer: 0x0000000001ccc448 ***
Aborted (core dumped)

I guess it should have returned the error message without core dump, unless I did something wrong.

Hello,

I'm interested in using this project in my work, but the build system is causing me some issues. Currently it is hardcoded to build static libraries and the CMake install only seems to install the source files. I can script generating new CMake easily enough, but if you are open to upgrade to the CMake 3 target-style system, I'd be happy to contribute back the CMake solution I come up with.

-Phil

I replaced the boost.asio with non-boost asio，

I found that redisclient cannot run under the windows platform because it calls the poll function.

So I replaced the poll function with the related function in asio

Here is my realization：

//redisclientimpl.cpp

    size_t socketReadSome(asio::generic::stream_protocol::socket &socket, asio::mutable_buffer buffer,
	const std::chrono::milliseconds &timeout,
	std::error_code &ec)
{
	asio::detail::buffer_sequence_adapter<asio::mutable_buffer, asio::mutable_buffer> buf(buffer);
	if (socket.native_handle() == asio::detail::invalid_socket)
	{
		ec = asio::error::bad_descriptor;
		return 0;
	}

	auto bytes = asio::detail::socket_ops::recv(socket.native_handle(), buf.buffers(), buf.count(), 0, ec);
	if (bytes > 0)
	{
		return bytes;
	}
	else if (bytes == 0)
	{
		ec = asio::error::eof;
		return 0;
	}

	if (ec != asio::error::would_block && ec != asio::error::try_again)
		return 0;

	if (asio::detail::socket_ops::poll_read(socket.native_handle(),
		asio::detail::socket_ops::internal_non_blocking, timeout.count(), ec) < 0)
	{
		return 0;
	}
	return socket.read_some(buffer, ec);	
}

size_t socketWrite(asio::generic::stream_protocol::socket &socket, asio::const_buffer buffer,
	const std::chrono::milliseconds &timeout, std::error_code &ec)
{
	asio::detail::buffer_sequence_adapter<asio::const_buffer, asio::const_buffer> buf(buffer);
	if (socket.native_handle() == asio::detail::invalid_socket)
	{
		ec = asio::error::bad_descriptor;
		return 0;
	}

	if (buf.all_empty())
	{
		ec = asio::error_code();
		return 0;
	}

	auto bytes = asio::detail::socket_ops::send(socket.native_handle(), 
		buf.buffers(), buf.count(), 0, ec);
	if (bytes==0 || bytes==buf.total_size())
	{
		return bytes;
	}
	else if (bytes < 0 && (ec != asio::error::would_block || ec != asio::error::try_again))
	{
		return 0;
	}
	else
	{
		socket.set_option(asio::ip::tcp::socket::send_low_watermark(buf.total_size() - bytes));

		const auto result = asio::detail::socket_ops::poll_write(socket.native_handle(),
			asio::detail::socket_ops::internal_non_blocking, timeout.count(), ec);
		if (result > 0)
		{
			socket.write_some(asio::buffer(asio::buffer_cast<const char *>(buffer) + bytes, buffer.size()), ec);
			bytes = buffer.size();
		}
		socket.set_option(asio::ip::tcp::socket::send_low_watermark(1));
		return bytes;
	}
}

//redissyncclient.cpp

void RedisSyncClient::connect(const asio::ip::tcp::endpoint &endpoint, std::error_code &ec)
{
pimpl->socket.open(endpoint.protocol(), ec);
if (!ec && tcpNoDelay)
pimpl->socket.set_option(asio::ip::tcp::no_delay(true), ec);

    // TODO keep alive option

pimpl->socket.non_blocking(true);

asio::detail::socket_ops::connect(pimpl->socket.native_handle(), endpoint.data(), endpoint.size(), ec);
if (ec != asio::error::in_progress
	&& ec != asio::error::would_block)
{
	return;
}

if (asio::detail::socket_ops::poll_connect(pimpl->socket.native_handle(), connectTimeout.count(), ec) < 0)
	return;

int connect_error = 0;
size_t connect_error_len = sizeof(connect_error);
if (asio::detail::socket_ops::getsockopt(pimpl->socket.native_handle(), 0, SOL_SOCKET, SO_ERROR,
	&connect_error, &connect_error_len, ec) == asio::detail::socket_error_retval)
	return;

ec = asio::error_code(connect_error,
	asio::error::get_system_category());

if (!ec)
    pimpl->state = State::Connected;

}

Winodw build redisclient failed。
Could someone help me？

D:\DevelopmentEnvironment\Workspaces\redisclient\redisclient\src\redisclient\impl\redisclientimpl.cpp(53): error C4430: missing type specifier - int assumed. Note: C++ does not support default-int
1>D:\DevelopmentEnvironment\Workspaces\redisclient\redisclient\src\redisclient\impl\redisclientimpl.cpp(53): error C2146: syntax error: missing ';' before identifier 'socketReadSomeImpl'
1>D:\DevelopmentEnvironment\Workspaces\redisclient\redisclient\src\redisclient\impl\redisclientimpl.cpp(55): error C2143: syntax error: missing ';' before '{'
1>D:\DevelopmentEnvironment\Workspaces\redisclient\redisclient\src\redisclient\impl\redisclientimpl.cpp(55): error C2447: '{': missing function header (old-style formal list?)
1>D:\DevelopmentEnvironment\Workspaces\redisclient\redisclient\src\redisclient\impl\redisclientimpl.cpp(86): warning C4244: 'initializing': conversion from '__int64' to 'std::size_t', possible loss of data
1>D:\DevelopmentEnvironment\Workspaces\redisclient\redisclient\src\redisclient\impl\redisclientimpl.cpp(90): error C2146: syntax error: missing ';' before identifier 'result'
1>D:\DevelopmentEnvironment\Workspaces\redisclient\redisclient\src\redisclient\impl\redisclientimpl.cpp(90): error C2065: 'result': undeclared identifier
1>D:\DevelopmentEnvironment\Workspaces\redisclient\redisclient\src\redisclient\impl\redisclientimpl.cpp(90): error C3861: 'socketReadSomeImpl': identifier not found
1>D:\DevelopmentEnvironment\Workspaces\redisclient\redisclient\src\redisclient\impl\redisclientimpl.cpp(94): error C2065: 'result': undeclared identifier
1>D:\DevelopmentEnvironment\Workspaces\redisclient\redisclient\src\redisclient\impl\redisclientimpl.cpp(107): error C2065: 'result': undeclared identifier
1>D:\DevelopmentEnvironment\Workspaces\redisclient\redisclient\src\redisclient\impl\redisclientimpl.cpp(116): error C2065: 'result': undeclared identifier
1>D:\DevelopmentEnvironment\Workspaces\redisclient\redisclient\src\redisclient\impl\redisclientimpl.cpp(125): error C4430: missing type specifier - int assumed. Note: C++ does not support default-int
1>D:\DevelopmentEnvironment\Workspaces\redisclient\redisclient\src\redisclient\impl\redisclientimpl.cpp(125): error C2086: 'int anonymous-namespace'::ssize_t': redefinition 1> D:\DevelopmentEnvironment\Workspaces\redisclient\redisclient\src\redisclient\impl\redisclientimpl.cpp(53): note: see declaration of 'anonymous-namespace'::ssize_t'
1>D:\DevelopmentEnvironment\Workspaces\redisclient\redisclient\src\redisclient\impl\redisclientimpl.cpp(125): error C2146: syntax error: missing ';' before identifier 'socketWriteImpl'
1>D:\DevelopmentEnvironment\Workspaces\redisclient\redisclient\src\redisclient\impl\redisclientimpl.cpp(127): error C2143: syntax error: missing ';' before '{'
1>D:\DevelopmentEnvironment\Workspaces\redisclient\redisclient\src\redisclient\impl\redisclientimpl.cpp(127): error C2447: '{': missing function header (old-style formal list?)
1>D:\DevelopmentEnvironment\Workspaces\redisclient\redisclient\src\redisclient\impl\redisclientimpl.cpp(158): warning C4244: 'initializing': conversion from '__int64' to 'std::size_t', possible loss of data
1>D:\DevelopmentEnvironment\Workspaces\redisclient\redisclient\src\redisclient\impl\redisclientimpl.cpp(162): error C2146: syntax error: missing ';' before identifier 'result'
1>D:\DevelopmentEnvironment\Workspaces\redisclient\redisclient\src\redisclient\impl\redisclientimpl.cpp(162): error C2065: 'result': undeclared identifier
1>D:\DevelopmentEnvironment\Workspaces\redisclient\redisclient\src\redisclient\impl\redisclientimpl.cpp(162): error C3861: 'socketWriteImpl': identifier not found
1>D:\DevelopmentEnvironment\Workspaces\redisclient\redisclient\src\redisclient\impl\redisclientimpl.cpp(166): error C2065: 'result': undeclared identifier
1>D:\DevelopmentEnvironment\Workspaces\redisclient\redisclient\src\redisclient\impl\redisclientimpl.cpp(179): error C2065: 'result': undeclared identifier
1>D:\DevelopmentEnvironment\Workspaces\redisclient\redisclient\src\redisclient\impl\redisclientimpl.cpp(188): error C2065: 'result': undeclared identifier

I am using version 0.5.0 of the library and I have a pretty similar code as in the PUBSUB example. My application subscribes to a channel and receives messages.
What I am facing is that every Monday, the application is not being able to receive messages from Redis.

Is there any timeout that I should handle in case the connection remains idle during the weekend? Shall I configure something extra in my application or in Redis to bypass this?

Thank you in advance,
George

Hi,
Does the RedisSyncClient support Redis transactions and execute the WATCH/MULTI/EXEC commands? Can the command() be used for that purpose?

Thank you in advance,
George

Is it possible to send multiple commands using pipelining?
Thank you

async_write in RedisClientImpl::asyncWrite causes a crash under the load. Should be wrapped by strand as well:

        boost::asio::async_write(socket, buffers,
                strand.wrap(std::bind(&RedisClientImpl::asyncWrite, shared_from_this(), std::placeholders::_1, std::placeholders::_2)));

This solves the issue.

Hey, thanks for the great work.
I saw that lately there were some fixes and improvements to the library, are you planning a release/tag?

I would rather work with a formal release instead of the "current latest"

Thanks
Greg

Hello, I'd tried to use connection to localhost with this code:

boost::asio::io_service io;
boost::asio::ip::tcp::resolver resolver(io);
boost::asio::ip::tcp::resolver::query query("localhost", "6379");
auto ep = resolver.resolve(query)->endpoint();
redisclient::RedisSyncClient redisInstance;
redisInstance.connect(ep);

but it fails with exception "Invalid argument". I made a research and found out the problem: boost 1.62 resolves localhost as ipv6 endpoint "::1", but redisclient supports only ipv4 (redissyncclient.cpp:65):

addr.sin_family = AF_INET;
addr.sin_port = htons(endpoint.port());
addr.sin_addr.s_addr = inet_addr(endpoint.address().to_string().c_str());

Hope it helps :)

P.S.: Latest version, commit f55ffdc

Sorry - never mind - had not actually looked at the code well enough. Interesting setup to allow lib or header only.

Hi,
In Redisvalue class you have methods to retrieve the underlying data, such as:

    // Return the value as a std::string if
    // type is a byte string; otherwise returns an empty std::string.
    REDIS_CLIENT_DECL std::string toString() const;

    // Return the value as a std::vector<char> if
    // type is a byte string; otherwise returns an empty std::vector<char>.
    REDIS_CLIENT_DECL std::vector<char> toByteArray() const;

    // Return the value as an array if type is an array;
    // otherwise returns an empty array.
    REDIS_CLIENT_DECL std::vector<RedisValue> toArray() const;

Any reason they don't return a const& to the underlying data (stored in the variant)? It would prevent a potentially unnecessary copy of the data. In addition there is a:

   REDIS_CLIENT_DECL std::vector<RedisValue> &getArray();

Any reason that there is no const overload?

Hi,

i Use this client to Get the integer from redis.

when i command GET, result is always a type of string. RedisValue.ToInt() return 0.

If there is any way to GET a integer from redis use RedisValue.

my code:

int main(int, char **)
{
    boost::asio::ip::address address = boost::asio::ip::address::from_string("172.30.46.160");
    const unsigned short port = 6379;
    boost::asio::ip::tcp::endpoint endpoint(address, port);

    boost::asio::io_service ioService;
    redisclient::RedisSyncClient redis(ioService);
    boost::system::error_code ec;

    redis.connect(endpoint, ec);

    if(ec)
    {
        std::cerr << "Can't connect to redis: " << ec.message() << std::endl;
        return EXIT_FAILURE;
    }

    redisclient::RedisValue result;

    result = redis.command("SET", {"key", "1"});

    if( result.isError() )
    {
        std::cerr << "SET error: " << result.toString() << "\n";
        return EXIT_FAILURE;
    }

    result = redis.command("INCRBY", {"key", "333"});
    if( result.isError() )
    {
        std::cerr << "SET error: " << result.toString() << "\n";
        return EXIT_FAILURE;
    }
    result = redis.command("GET", {"key"});

    if( result.isOk() )
    {
        std::cout << "GET: " << result.inspect() << "\n"; // always return a string
        return EXIT_SUCCESS;
    }
    else
    {
        std::cerr << "GET error: " << result_int.toString() << "\n";
        return EXIT_FAILURE;
    }
}

I've compiled examples provided with code, and valgrind shows memory leaks for all of them (both master and development versions).

valgrind_report.txt

In RedisSyncClient::connect, line 44, pimpl->processMessage(); causes exception throwing after any call of command(). Probably it should be removed.

to make this lib truely header only (link without boost libs), the following code is required in one of your cpp file

// use boost system error header only
#include <boost/system/detail/error_code.ipp>

Build fails when compiling with the latest boost (v1.66.0)
I was using boost v1.62 previously and compile worked fine on that version.

Error I'm seeing (compiled with gcc7.3 + boost1.66.0):

In file included from /redisclient/src/redisclient/redissyncclient.h:16:0,
from /redisclient/src/redisclient/impl/pipeline.cpp:11:
/redisclient/src/redisclient/impl/redisclientimpl.h:86:5: error: invalid use of template-name 'boost::asio::strand' without an argument list
boost::asio::strand strand;
^~~~~
/redisclient/src/redisclient/impl/redisclientimpl.h:86:5: note: class template argument deduction is only available with -std=c++1z or -std=gnu++1z

Hi,

I am trying to perform multiple command by using EVAL and a Lua script but with no luck. Could you please provide an example?

More specific, I am trying to retrieve multiple hashes like below:
redis-cli --ldb --eval /tmp/script.lua hash_key1 hash_key2

where script.lua:
local r = {}
for _, v in pairs(KEYS) do
r[#r+1] = redis.call('HGETALL', v)
end
return r

Can the above be achieved by using RedisClient?

I tried something below:
redisclient.command("EVAL", {"/tmp/script.lua", hash_key1, hash_key2}
but obviously is wrong.

Thank you in advance,
George

//clang++ -std=c++11 redis.cpp -lboost_system -o redis
#include "redisclient/redissyncclient.h"

int main(int, char **)
{
    boost::asio::ip::address address = boost::asio::ip::address::from_string("127.0.0.1");
    const unsigned short port = 6379;

    boost::asio::io_service ioService;
    RedisSyncClient redis(ioService);
    std::string errmsg;

    if( !redis.connect(address, port, errmsg) )
    {
        std::cerr << "Can't connect to redis: " << errmsg << std::endl;
        return EXIT_FAILURE;
    }

    RedisValue result;

    result = redis.command("HMSET", "mkey", "key1", "val1", "key2", "val2");

    if( result.isError() )
    {
        std::cerr << "SET error: " << result.toString() << "\n";
        return EXIT_FAILURE;
    }

    result = redis.command("HMGET", "mkey", "key2");

    if( result.isOk() )
    {
        std::cout << "GET key2 : " << result.toString() << "\n";
        return EXIT_SUCCESS;
    }
    else
    {
        std::cerr << "GET error: " << result.toString() << "\n";
        return EXIT_FAILURE;
    }
}

Prints:
GET key2: (empty)

On command line:

redis-cli HMGET mkey key2
Returns: "val2"

in /redisclient/src/redisclient/impl/redisclientimpl.cpp, LINE 208

    for(;;)
    {
        size_t size = socket.read_some(boost::asio::buffer(inbuff));

        for(size_t pos = 0; pos < size;)
        {
            std::pair<size_t, RedisParser::ParseResult> result = 
                redisParser.parse(inbuff.data() + pos, size - pos);

            if( result.second == RedisParser::Completed )
            {
                return redisParser.result();
            }
            else if( result.second == RedisParser::Incompleted )
            {
                continue;     // *** this makes 'pos' variable is not updated!!
            }
            else
            {
                errorHandler("[RedisClient] Parser error");
                return RedisValue();
            }

            pos += result.first;
        }
    }

Can you check on this?

(Parse Error does not appear after commenting out that line)

what version of boost are you using?

latest one is giving arg issues.

hi

i am facing a core dump when call RedisSyncClient::command() function.

My env :
boost: 1.7.0
gcc: 6.3.0
os: centos 7.6

gdb's bt:
(gdb) bt
#0 0x00007f46f5e1295f in __memmove_ssse3_back () from /lib64/libc.so.6
#1 0x000000000055151d in redisclient::RedisClientImpl::syncReadResponse(boost::posix_time::time_duration const&, boost::system::error_code&) ()
#2 0x000000000055137f in redisclient::RedisClientImpl::doSyncCommand(std::deque<redisclient::RedisBuffer, std::allocatorredisclient::RedisBuffer > const&, boost::posix_time::time_duration const&, boost::system::error_code&) ()
#3 0x0000000000552028 in redisclient::RedisSyncClient::command(std::string, std::deque<redisclient::RedisBuffer, std::allocatorredisclient::RedisBuffer >, boost::system::error_code&) ()
#4 0x0000000000551eef in redisclient::RedisSyncClient::command(std::string, std::deque<redisclient::RedisBuffer, std::allocatorredisclient::RedisBuffer >)()

Source code:
`
long get_update_remain(const void *sql_ctx, const void *redis_ctx, const char *user_id, unsigned int flow_size)
{
long remain_k = 0;
string key_remain_size = string(KEY_ORDER_REMAIN_SIZE) + user_id;

assert(redis_ctx);
auto redis = (redisclient::RedisSyncClient *) redis_ctx;

redisclient::RedisValue result;
result = redis->command("GET", {key_remain_size});
string res = result.toString();
if (result.isError() || result.toString().empty())
{
    printf("GET error, key[%s], result[%s]\n",
           key_remain_size.c_str(), result.toString().c_str());

    //select new order
    if (!select_order(sql_ctx, redis_ctx, user_id))
    {
        return 0;
    }
}
else
{
    try
    {
        printf("%s\n", result.toString().c_str());
        remain_k = std::stol(result.toString());
        printf("before DECRBY remain_k is %ld\n", remain_k);
    }
    catch (...)
    {
        printf("error");
    }
}

if(flow_size == 0)
    return remain_k;

//Redis Dec
result = redis->command("DECRBY", {key_remain_size, std::to_string(flow_size/1024)});
if (result.isError() || !result.isInt())
{
    printf("DECRBY key=%s error\n", key_remain_size.c_str());
    return 0;
}
remain_k = (long)result.toInt();
printf("after DECRBY remain_k is %ld\n", remain_k);

if (remain_k<=0)
    unset_order(sql_ctx, redis_ctx, user_id);

return remain_k;

}`

stateValid() seems be the only way to get the state of a RedisSyncClient connection.
And it has been marked protected with assert() method inside it.

Or am i missing some handler that gives one the state of a redis connection ?

Please, consider adding constructor like RedisBuffer(const void* ptr, size_t dataSize); not only RedisBuffer(const char *ptr, size_t dataSize);
this adds more convenient and more general way to pass buffer to RedisBuffer, because you can avoid casts in some cases

I'd like to use redisclient on multi thread program. In the program, io_service::run() is called from each threads.

I got some unexpected behavior.

Here is the list of behavior I ovserved:

1. Got stucked. Log output is stopped.

2. Core dumped.

*** Error in `./a.out': double free or corruption (fasttop): 0x00007f4dd0002720 ***
[1]    17510 abort (core dumped)  ./a.out

3. Segmentation falult.

SET: OK[1]    17549 segmentation fault (core dumped)  ./a.out

4. Redis returns error.

SET: unique-redis-value
Invalid value from redis: unique-redis-value

I wrote the code that reproduces the behavior based on
https://github.com/nekipelov/redisclient/blob/master/examples/async_set_get.cpp

The code continues SET, GET, DEL, SET, GET, DEL, ...
When I run the executable file again and again, the behavior above is observed.

#include <string>
#include <iostream>
#include <functional>
#include <boost/asio/ip/address.hpp>
#include <thread>

#include <redisclient/redisasyncclient.h>

static const std::string redisKey = "unique-redis-key-example";
static const std::string redisValue = "unique-redis-value";

class Worker
{
public:
    Worker(boost::asio::io_service &ioService, redisclient::RedisAsyncClient &redisClient)
        : ioService(ioService), redisClient(redisClient)
    {}

    void onConnect(bool connected, const std::string &errorMessage);
    void onSet(const redisclient::RedisValue &value);
    void onGet(const redisclient::RedisValue &value);
    void onDel(const redisclient::RedisValue &value);
    void stop();

private:
    boost::asio::io_service &ioService;
    redisclient::RedisAsyncClient &redisClient;
};

void Worker::onConnect(bool connected, const std::string &errorMessage)
{
    if( !connected )
    {
        std::cerr << "Can't connect to redis: " << errorMessage;
    }
    else
    {
        // two async requests
        redisClient.command("SET",  {redisKey, redisValue},
                            std::bind(&Worker::onSet, this, std::placeholders::_1));
        redisClient.command("SET",  {redisKey, redisValue},
                            std::bind(&Worker::onSet, this, std::placeholders::_1));
    }
}

void Worker::onSet(const redisclient::RedisValue &value)
{
    std::cerr << "SET: " << value.toString() << std::endl;
    if( value.toString() == "OK" )
    {
        redisClient.command("GET",  {redisKey},
                            std::bind(&Worker::onGet, this, std::placeholders::_1));
    }
    else
    {
        std::cerr << "Invalid value from redis: " << value.toString() << std::endl;
    }
}

void Worker::onGet(const redisclient::RedisValue &value)
{
    std::cerr << "GET " << value.toString() << std::endl;
    if( value.toString() != redisValue )
    {
        std::cerr << "Invalid value from redis: " << value.toString() << std::endl;
    }

    redisClient.command("DEL", {redisKey},
                        std::bind(&Worker::onDel, this, std::placeholders::_1));
}

void Worker::onDel(const redisclient::RedisValue &value)
{
    std::cerr << "DEL " << value.toString() << std::endl;
    redisClient.command("SET",  {redisKey, redisValue},
                        std::bind(&Worker::onSet, this, std::placeholders::_1));
}


int main(int, char **)
{
    const char *address = "127.0.0.1";
    const int port = 6379;

    boost::asio::io_service ioService;
    redisclient::RedisAsyncClient client(ioService);
    Worker worker(ioService, client);
    boost::asio::ip::tcp::endpoint endpoint(boost::asio::ip::address::from_string(address), port);

    client.asyncConnect(endpoint, std::bind(&Worker::onConnect, &worker,
                std::placeholders::_1, std::placeholders::_2));

    std::vector<std::thread> threads;
    const int num = 8;
    threads.reserve(num);
    for (int i = 0; i != num; ++i) {
        threads.emplace_back([&]{ ioService.run(); });
    }
    for (auto& e : threads) {
        e.join();
    }

    return 0;
}

Compile command:

clang++ -std=c++14 test.cpp -lboost_system -lpthread -O3

Environment:

clang version 5.0.0 (tags/RELEASE_500/final)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /usr/bin

boost 1.65.1

Linux 4.13.3-1-ARCH #1 SMP PREEMPT Thu Sep 21 20:33:16 CEST 2017 x86_64 GNU/Linux

Can I send async command like redisClient.command("SET", ...) via the same connection from multiple threads?

The command member function of the async client checks stateValid() before executing. Connected is currently the only state that qualifies as valid, so command throws an exception when the client is in the Subscribed state. Is this an intentional limitation and if not, could it be fixed to count both states as valid?

Hi,

I assume that the following piece of code should run fine:

#include <boost/asio.hpp>
#include <boost/asio/steady_timer.hpp>
#include <boost/lexical_cast.hpp>
#include <redisclient/redisasyncclient.h>
#include <chrono>
#include <ostream>
#include <iostream>
#include <memory>

namespace asio = boost::asio;
namespace sys = boost::system;


int main(){

    asio::io_service io_service;
    std::chrono::seconds sleep_delay(10);

    auto ip_address = boost::asio::ip::address::from_string("127.0.0.1");
    auto end_point = boost::asio::ip::tcp::endpoint(ip_address, 6379);

    {
        auto redis = std::make_shared<redisclient::RedisAsyncClient>(io_service);
        redis->asyncConnect(end_point, [redis_copy = redis](bool ok, const std::string &errmsg){
            if (ok) {
                std::cout << "connected\n";
            } else {
                std::cout << "not connected " << errmsg << "\n";
            }
        });
    }

    asio::steady_timer timer(io_service, sleep_delay);
    timer.async_wait([&](auto ec){
        std::cout << "timer triggers\n";
        io_service.stop();
    });
    std::cout << "starting\n";
    io_service.run();



    return 0;
}

Meanwhile I get:

 ./async_connect -s
starting
connected
terminate called after throwing an instance of 'std::runtime_error'
  what():  Operation canceled
zsh: abort      ./async_connect -s

It tries to cancel something, when object has been destroyed and rises appropriate exception, meanwhile it should not do that.

I am developing an application which has to get event notifications when some object expires.
To get notifications I have to set the 'notify-keyspace-events' filter or events won't be delivered.

I can set the filter from the redis CLI or redis.conf file and then my application can subscribe and start getting notifications.
But I really need to set this filter dynamically, yes it will be set from another process, which will set the filter, create a redis object and set expiration time for this object.

I use the redisclient command() function and the 'config set' command which works for me on the CLI::
redisClient.command("config set", {"notify-keyspace-events", "KEA"},
std::bind(&Worker::onConfig, this, std::placeholders::_1));
The above gives me an error: "ERR unknown command 'config set'".

When I look at the implementation of command() and doAsyncCommand() functions they are very generic and just post the request to the redis server.

Is there anything I am doing wrong? Why the same command that works from the CLI does not work from redisclient?

Thank you for your help.

In redisclientimpl

inline void bufferAppend(std::vector &vec, const redisclient::RedisBuffer &buf)
{
if (buf.data.type() == typeid(std::string))
bufferAppend(vec, boost::getstd::string(buf.data));
else
bufferAppend(vec, boost::get<std::vector>(buf.data));
}

Because there is no bufferAppend(std::vector &vec, const std::vector &v)

it will always recursion, resulting in stack overflow, you should add a one.

Hello.

Firstly, thanx for your work on redisclient.

While using it,
I've found there are some problems in multi-thread environment.

When you use RedisAsyncClient in multi thread environment - operating set / get / pub / sub to on single instance in concurrently - it crashes.

The details will be addressed in pull request.

Thanx.

All redis command working? Anything that doesn't work? can you provide hget hset, zrange examples?

I read the example, but can't find a way to get all fields and values of the hash. If I stored many fields and values, it's inefficient to get only one value, so can you give us an example to return all fields.

Thanks.

When a redis syncclient's connection is disconnected, the next redis command throws an exception. It's good.
Then I try to connect again using the same redis syncclient, I got Already open error.

I tried to find close() function but not found.
https://github.com/nekipelov/redisclient/search?utf8=%E2%9C%93&q=close&type=

I think that it's nice to have any of the following functionality.

1. close() : close the socket from the client side. It can execute after the socket is closed by server side.
2. Automatically close if the server closed the connection. I think that RedisSyncClient needs to catch the exception in command() and update connecting status then re-throw the exception.

Any ideas?

Here is the code example that reproduces the issue:

#include <string>
#include <iostream>
#include <functional>

#include <boost/asio/ip/address.hpp>

#include <redisclient/redissyncclient.h>

int main() {
    boost::asio::ip::address address = boost::asio::ip::address::from_string("127.0.0.1");
    const unsigned short port = 6379;

    boost::asio::io_service ioService;
    redisclient::RedisSyncClient redis(ioService);
    std::string errmsg;

    if( !redis.connect(address, port, errmsg) )
    {
        std::cerr << "Can't connect to redis: " << errmsg << std::endl;
        return EXIT_FAILURE;
    }

    redisclient::RedisValue result;

    std::cout << "restart redis service" << std::endl;
    sleep(5);

    try {
        result = redis.command("SET", {"key", "value"});

        if( result.isError() )
        {
            std::cerr << "SET error: " << result.toString() << "\n";
            return EXIT_FAILURE;
        }
    }
    catch (std::exception const& e) {
        std::cout << e.what() << std::endl;
        std::cout << "trying reconnect..." << std::endl;
        if( !redis.connect(address, port, errmsg) )
        {
            std::cerr << "Can't connect to redis: " << errmsg << std::endl;
            return EXIT_FAILURE;
        }
    }
}

You need to restart or redis service before 5 seconds.

Output

restart redis service
read_some: End of file
trying reconnect...
Can't connect to redis: Already open

when i use redis.command("rpush", "list_name", "value01 value02 value03"), it treats the values as one value. How to solve this?