basiliscos / cpp-bredis Goto Github PK
View Code? Open in Web Editor NEWBoost::ASIO low-level redis client (connector)
License: MIT License
Boost::ASIO low-level redis client (connector)
License: MIT License
tx_buff.commit(string.size());
seems redundant and leads to currupted buffer, catched by strace:
socket(AF_INET, SOCK_STREAM, IPPROTO_TCP) = 6
connect(6, {sa_family=AF_INET, sin_port=htons(6379), sin_addr=inet_addr("127.0.0.1")}, 16) = 0
sendmsg(6, {msg_name=NULL, msg_namelen=0, msg_iov=[{iov_base="*3\r\n$5\r\nbrpop\r\n$6\r\nmylist\r\n$1\r\n0\r\n\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0", iov_len=68}], msg_iovlen=1, msg_controllen=0, msg_flags=0}, MSG_NOSIGNAL) = 68
vs
socket(AF_INET, SOCK_STREAM, IPPROTO_TCP) = 6
connect(6, {sa_family=AF_INET, sin_port=htons(6379), sin_addr=inet_addr("127.0.0.1")}, 16) = 0
sendmsg(6, {msg_name=NULL, msg_namelen=0, msg_iov=[{iov_base="*3\r\n$5\r\nbrpop\r\n$6\r\nmylist\r\n$1\r\n0\r\n", iov_len=34}], msg_iovlen=1, msg_controllen=0, msg_flags=0}, MSG_NOSIGNAL) = 34
To prevent conflicts with includes, it might be better if the library header files were located in a directory called include/bredis
instead of bredis/
.
This way, anyone who uses the library can simply clone the repository, and add
-I{root}include/bredis
without worrying that they can pick up other includes (for example, trying to include t/catch.hpp
).
The package should be integrated to https://github.com/cpp-pm/hunter
what if i use boost 1.66 instead ?
prepared_command
, which already holds serialized form, and yous it as item in ConstBufferSequence
in other words, the idea is to avoid output (tx
) buffer linearization. Currently this technique is used with rx
buffer on (and it used completely on the experimental branch).
Avoiding output linearization will really make bredis zero cost, as the user's payload will not be copied by bredis, and will be passed into OS-kernel directly.
Hi,
I was initially using boost version 1.73. Wrapping the underlying type boost::asio::ip::tcp::socket
as a bredis connection works. However, doing the same for the unix sockets with type boost::asio::local::stream_protocol::socket
does not.
This is the code I am running at the moment, which fails during building.
boost::asio::io_context ioserv;
using socket_t = boost::asio::local::stream_protocol::socket;
using next_layer_t = socket_t;
boost::asio::local::stream_protocol::endpoint end_point("/shared/redis.sock");
socket_t sock(ioserv);
sock.connect(end_point);
bredis::Connection<next_layer_t> bredis_con(std::move(socket));
And here is the error I get.
In file included from /usr/local/include/boost/asio/executor.hpp:338,
from /usr/local/include/boost/asio/basic_socket.hpp:27,
from /usr/local/include/boost/asio/basic_datagram_socket.hpp:20,
from /usr/local/include/boost/asio.hpp:24,
from /atom/languages/cpp/third-party/cpp-bredis/include/bredis/Connection.hpp:20,
from ./clients/bredis_client.cc:7:
/usr/local/include/boost/asio/impl/executor.hpp: In instantiation of 'boost::asio::execution_context& boost::asio::executor::impl< <template-parameter-1-1>, <template-parameter-1-2> >::context() [with Executor = int (*)(int, int, int); Allocator = std::allocator<void>]':
/usr/local/include/boost/asio/impl/executor.hpp:177:22: required from here
/usr/local/include/boost/asio/impl/executor.hpp:179:22: error: request for member 'context' in '((boost::asio::executor::impl<int (*)(int, int, int), std::allocator<void> >*)this)->boost::asio::executor::impl<int (*)(int, int, int), std::allocator<void> >::executor_', which is of non-class type 'int (*)(int, int, int)'
return executor_.context();
~~~~~~~~~~^~~~~~~
/usr/local/include/boost/asio/impl/executor.hpp: In instantiation of 'void boost::asio::executor::impl< <template-parameter-1-1>, <template-parameter-1-2> >::on_work_started() [with Executor = int (*)(int, int, int); Allocator = std::allocator<void>]':
/usr/local/include/boost/asio/impl/executor.hpp:167:8: required from here
/usr/local/include/boost/asio/impl/executor.hpp:169:15: error: request for member 'on_work_started' in '((boost::asio::executor::impl<int (*)(int, int, int), std::allocator<void> >*)this)->boost::asio::executor::impl<int (*)(int, int, int), std::allocator<void> >::executor_', which is of non-class type 'int (*)(int, int, int)'
executor_.on_work_started();
~~~~~~~~~~^~~~~~~~~~~~~~~
/usr/local/include/boost/asio/impl/executor.hpp: In instantiation of 'void boost::asio::executor::impl< <template-parameter-1-1>, <template-parameter-1-2> >::on_work_finished() [with Executor = int (*)(int, int, int); Allocator = std::allocator<void>]':
/usr/local/include/boost/asio/impl/executor.hpp:172:8: required from here
/usr/local/include/boost/asio/impl/executor.hpp:174:15: error: request for member 'on_work_finished' in '((boost::asio::executor::impl<int (*)(int, int, int), std::allocator<void> >*)this)->boost::asio::executor::impl<int (*)(int, int, int), std::allocator<void> >::executor_', which is of non-class type 'int (*)(int, int, int)'
executor_.on_work_finished();
~~~~~~~~~~^~~~~~~~~~~~~~~~
/usr/local/include/boost/asio/impl/executor.hpp: In instantiation of 'void boost::asio::executor::impl< <template-parameter-1-1>, <template-parameter-1-2> >::dispatch(boost::asio::executor::function&&) [with Executor = int (*)(int, int, int); Allocator = std::allocator<void>]':
/usr/local/include/boost/asio/impl/executor.hpp:182:8: required from here
/usr/local/include/boost/asio/impl/executor.hpp:184:15: error: request for member 'dispatch' in '((boost::asio::executor::impl<int (*)(int, int, int), std::allocator<void> >*)this)->boost::asio::executor::impl<int (*)(int, int, int), std::allocator<void> >::executor_', which is of non-class type 'int (*)(int, int, int)'
executor_.dispatch(BOOST_ASIO_MOVE_CAST(function)(f), allocator_);
~~~~~~~~~~^~~~~~~~
/usr/local/include/boost/asio/impl/executor.hpp: In instantiation of 'void boost::asio::executor::impl< <template-parameter-1-1>, <template-parameter-1-2> >::post(boost::asio::executor::function&&) [with Executor = int (*)(int, int, int); Allocator = std::allocator<void>]':
/usr/local/include/boost/asio/impl/executor.hpp:187:8: required from here
/usr/local/include/boost/asio/impl/executor.hpp:189:15: error: request for member 'post' in '((boost::asio::executor::impl<int (*)(int, int, int), std::allocator<void> >*)this)->boost::asio::executor::impl<int (*)(int, int, int), std::allocator<void> >::executor_', which is of non-class type 'int (*)(int, int, int)'
executor_.post(BOOST_ASIO_MOVE_CAST(function)(f), allocator_);
~~~~~~~~~~^~~~
/usr/local/include/boost/asio/impl/executor.hpp: In instantiation of 'void boost::asio::executor::impl< <template-parameter-1-1>, <template-parameter-1-2> >::defer(boost::asio::executor::function&&) [with Executor = int (*)(int, int, int); Allocator = std::allocator<void>]':
/usr/local/include/boost/asio/impl/executor.hpp:192:8: required from here
/usr/local/include/boost/asio/impl/executor.hpp:194:15: error: request for member 'defer' in '((boost::asio::executor::impl<int (*)(int, int, int), std::allocator<void> >*)this)->boost::asio::executor::impl<int (*)(int, int, int), std::allocator<void> >::executor_', which is of non-class type 'int (*)(int, int, int)'
executor_.defer(BOOST_ASIO_MOVE_CAST(function)(f), allocator_);
~~~~~~~~~~^~~~~
I tried downgrading to boost 1.69, but got the following error:
/atom/languages/cpp/third-party/cpp-bredis/include/bredis/Connection.hpp: In instantiation of 'bredis::Connection<NextLayer>::Connection(Args&& ...) [with Args = {int (&)(int, int, int)}; NextLayer = boost::asio::basic_stream_socket<boost::asio::local::stream_protocol>]':
./clients/bredis_client.cc:61:66: required from here
/atom/languages/cpp/third-party/cpp-bredis/include/bredis/Connection.hpp:41:46: error: no matching function for call to 'boost::asio::basic_stream_socket<boost::asio::local::stream_protocol>::basic_stream_socket(int (&)(int, int, int))'
: stream_(std::forward<Args>(args)...) {}
^
In file included from /usr/local/include/boost/asio/basic_socket_streambuf.hpp:25,
from /usr/local/include/boost/asio/basic_socket_iostream.hpp:24,
from /usr/local/include/boost/asio.hpp:31,
from /atom/languages/cpp/third-party/cpp-bredis/include/bredis/Connection.hpp:20,
from ./clients/bredis_client.cc:7:
/usr/local/include/boost/asio/basic_stream_socket.hpp:185:3: note: candidate: 'template<class Protocol1> boost::asio::basic_stream_socket<Protocol>::basic_stream_socket(boost::asio::basic_stream_socket<Protocol1>&&, typename std::enable_if<std::is_convertible<Protocol1, Protocol>::value>::type*)'
basic_stream_socket(
^~~~~~~~~~~~~~~~~~~
/usr/local/include/boost/asio/basic_stream_socket.hpp:185:3: note: template argument deduction/substitution failed:
In file included from ./clients/bredis_client.cc:7:
/atom/languages/cpp/third-party/cpp-bredis/include/bredis/Connection.hpp:41:46: note: mismatched types 'boost::asio::basic_stream_socket<Protocol>' and 'int(int, int, int)'
: stream_(std::forward<Args>(args)...) {}
^
In file included from /usr/local/include/boost/asio/basic_socket_streambuf.hpp:25,
from /usr/local/include/boost/asio/basic_socket_iostream.hpp:24,
from /usr/local/include/boost/asio.hpp:31,
from /atom/languages/cpp/third-party/cpp-bredis/include/bredis/Connection.hpp:20,
from ./clients/bredis_client.cc:7:
/usr/local/include/boost/asio/basic_stream_socket.hpp:152:3: note: candidate: 'boost::asio::basic_stream_socket<Protocol>::basic_stream_socket(boost::asio::basic_stream_socket<Protocol>&&) [with Protocol = boost::asio::local::stream_protocol]'
basic_stream_socket(basic_stream_socket&& other)
^~~~~~~~~~~~~~~~~~~
/usr/local/include/boost/asio/basic_stream_socket.hpp:152:3: note: no known conversion for argument 1 from 'int(int, int, int)' to 'boost::asio::basic_stream_socket<boost::asio::local::stream_protocol>&&'
/usr/local/include/boost/asio/basic_stream_socket.hpp:134:3: note: candidate: 'boost::asio::basic_stream_socket<Protocol>::basic_stream_socket(boost::asio::io_context&, const protocol_type&, const native_handle_type&) [with Protocol = boost::asio::local::stream_protocol; boost::asio::basic_stream_socket<Protocol>::protocol_type = boost::asio::local::stream_protocol; boost::asio::basic_stream_socket<Protocol>::native_handle_type = int]'
basic_stream_socket(boost::asio::io_context& io_context,
^~~~~~~~~~~~~~~~~~~
/usr/local/include/boost/asio/basic_stream_socket.hpp:134:3: note: candidate expects 3 arguments, 1 provided
/usr/local/include/boost/asio/basic_stream_socket.hpp:114:3: note: candidate: 'boost::asio::basic_stream_socket<Protocol>::basic_stream_socket(boost::asio::io_context&, const endpoint_type&) [with Protocol = boost::asio::local::stream_protocol; boost::asio::basic_stream_socket<Protocol>::endpoint_type = boost::asio::local::basic_endpoint<boost::asio::local::stream_protocol>]'
basic_stream_socket(boost::asio::io_context& io_context,
^~~~~~~~~~~~~~~~~~~
/usr/local/include/boost/asio/basic_stream_socket.hpp:114:3: note: candidate expects 2 arguments, 1 provided
/usr/local/include/boost/asio/basic_stream_socket.hpp:93:3: note: candidate: 'boost::asio::basic_stream_socket<Protocol>::basic_stream_socket(boost::asio::io_context&, const protocol_type&) [with Protocol = boost::asio::local::stream_protocol; boost::asio::basic_stream_socket<Protocol>::protocol_type = boost::asio::local::stream_protocol]'
basic_stream_socket(boost::asio::io_context& io_context,
^~~~~~~~~~~~~~~~~~~
/usr/local/include/boost/asio/basic_stream_socket.hpp:93:3: note: candidate expects 2 arguments, 1 provided
/usr/local/include/boost/asio/basic_stream_socket.hpp:76:12: note: candidate: 'boost::asio::basic_stream_socket<Protocol>::basic_stream_socket(boost::asio::io_context&) [with Protocol = boost::asio::local::stream_protocol]'
explicit basic_stream_socket(boost::asio::io_context& io_context)
^~~~~~~~~~~~~~~~~~~
/usr/local/include/boost/asio/basic_stream_socket.hpp:76:12: note: no known conversion for argument 1 from 'int(int, int, int)' to 'boost::asio::io_context&'
make: *** [Makefile:102: clients/build/bredis_client] Error 1
I'm running inside of a Docker container with Debian GNU/Linux 10 (buster).
single_command_t("hset", "key", "value1", "", "value2", "")
this command produces an wrong command string that will send to server and can lead to errors.
reason:
Protocol::command_size() returns zero byte size if args size are zero, but to store zero as string 1 byte is needed instead 0.
Documentation states, that it's possible to use drop_result
as part of the parsing policy: https://github.com/basiliscos/cpp-bredis#parse_result_titerator-policy
However, it's pretty unclear how to use it with the Connection
object.
I read through the source code and Connection
seems to be have hard-coded keep_result
as policy type. How would I use drop_result
with Connection object?
Do I understand the intention of drop_result
properly, that it'd cause the parser to verify that the response isn't an error, but the payload itself is dropped.
It happens that connections can be dropped. Given that there is a basic demand to reconnect.
There are two options for this:
Connection::reconnect()
Connection
can get a new socket assigned. And the caller is responsible to establish the connection. asio::basic_stream_socket
(as well as unix domain sockets) perfectly fine supports the move assignment:At the moment we (@Icinga) have to use this workaround to compile this lib:
#if __cplusplus < 201402L
#include <boost/type_traits/decay.hpp>
#include <boost/core/enable_if.hpp>
namespace std
{
using boost::decay_t;
using boost::enable_if_t;
}
#endif /* < C++14 */
It would be nice if we wouldn't have to.
This looks like a good project! With some refactoring, I think it could be even better! I'd like to do a code review if possible, and leave individual comments on source code lines. I'm not entirely sure how to do that though. Maybe I will submit a pull request with C++ comments embedded.
Hello,
we are going to use cpp-bredis in our redis graph solution we have the following code:
executor_context_ptr_->conn.async_write(
executor_context_ptr_->tx_buff,
cmd,
asio::bind_executor(executor_context_ptr_->strand, [this, &result_promise](const sys::error_code& ec, std::size_t bytes_transferred)
{
if (!ec) {
auto self = this;
self->executor_context_ptr_->tx_buff.consume(bytes_transferred);
executor_context_ptr_->conn.async_read(
executor_context_ptr_->rx_buff,
asio::bind_executor(executor_context_ptr_->strand, [this, &result_promise](const sys::error_code& ec, result_t&& r) {
if (!ec) {
auto self = this;
auto result = r.result;
auto& replies = boost::get<r::markers::array_holder_t<Iterator>>(r.result);
// here how to parse the reply?
self->executor_context_ptr_->rx_buff.consume(r.consumed);
// parse and set result.
}
else
{
redisgraph::result_view read_error_view;
result_promise.set_value(read_error_view);
}
}));
}
We woud like to know how to parse and debug the list of arrays.
Best Regards,
Giorgio.
Hi,
There is some issue that I can't figure out from the documentation: when multiple messages are streamed in and the buffer contains more then one message, what's the correct way of using async_read to ensure that all messages are handled and none is lost. Is it handled within bredis or should I adapt on my side?
Thanks,
Silviu
Include cmake_minimum_required
check in examples/CMakeLists.txt
to fix CMP0000
policy.
CMake Warning (dev) in CMakeLists.txt:
No cmake_minimum_required command is present. A line of code such as
cmake_minimum_required(VERSION 3.14)
should be added at the top of the file. The version specified may be lower
if you wish to support older CMake versions for this project. For more
information run "cmake --help-policy CMP0000".
This warning is for project developers. Use -Wno-dev to suppress it.
bredis/Error.hpp
has only generic "protocol error", would be nice to have more details, which are helpful where rx_buff
was already parsed or corrupted.
After I build with Cmake and invoke the make command, these errors occur. What should I do to solve this?
/Users/Stephen/documents/cpp-bredis/include/bredis/impl/connection.ipp:61:40: error:
no matching constructor for initialization of
'asio::async_result<real_handler_t>' (aka
'async_result<coro_handler<boost::asio::executor_binder<void ()(),
boost::asio::executor>,
bredis::positive_parse_result_t<boost::asio::buffers_iterator<boost::asio::const_buffers_1,
char>, bredis::parsing_policy::keep_result> > >')
asio::async_result<real_handler_t> async_result(real_handler);
^ ~~~~~~~~~~~~
/Users/Stephen/documents/cpp-bredis/t/21-coroutine.cpp:53:35: note: in
instantiation of function template specialization
'bredis::Connection<boost::asio::basic_stream_socketboost::asio::ip::tcp
>::async_read<boost::asio::basic_streambuf<std::__1::allocator >,
boost::asio::basic_yield_context<boost::asio::executor_binder<void ()(),
boost::asio::executor> > >' requested here
auto parse_result = c.async_read(rx_buff, yield[error_code], 1);
^
/usr/local/include/boost/asio/async_result.hpp:50:7: note: candidate constructor
(the implicit copy constructor) not viable: no known conversion from
'real_handler_t' (aka 'coro_handler<boost::asio::executor_binder<void
()(), boost::asio::executor>,
bredis::positive_parse_result_t<boost::asio::buffers_iterator<boost::asio::const_buffers_1,
char>, bredis::parsing_policy::keep_result> >') to 'const
boost::asio::async_result<boost::asio::detail::coro_handler<boost::asio::executor_binder<void
()(), boost::asio::executor>,
bredis::positive_parse_result_t<boost::asio::buffers_iterator<boost::asio::const_buffers_1,
char>, bredis::parsing_policy::keep_result> >, void>' for 1st argument
class async_result
^
/usr/local/include/boost/asio/impl/spawn.hpp:316:12: note: candidate constructor
not viable: no known conversion from 'real_handler_t' (aka
'coro_handler<boost::asio::executor_binder<void ()(),
boost::asio::executor>,
bredis::positive_parse_result_t<boost::asio::buffers_iterator<boost::asio::const_buffers_1,
char>, bredis::parsing_policy::keep_result> >') to 'typename
detail::coro_async_result<executor_binder<void ()(), executor>,
void>::completion_handler_type &' (aka
'coro_handler<boost::asio::executor_binder<void (*)(),
boost::asio::executor>, void> &') for 1st argument
explicit async_result(
^
3 warnings and 2 errors generated.
make[2]: *** [CMakeFiles/t-21-coroutine.dir/t/21-coroutine.cpp.o] Error 1
make[1]: *** [CMakeFiles/t-21-coroutine.dir/all] Error 2
make: *** [all] Error 2
The package should be uploaded to https://bintray.com/conan-community/conan
PR: #13
bredis/Command.hpp
has rather generic interace, i.e. get
command should take only one string (key).
I'm trying to build the examples, are there any build instructions for linux?
I tried
ccmake .
make
but I get
$ make
CMake Warning (dev) in CMakeLists.txt:
No cmake_minimum_required command is present. A line of code such as
cmake_minimum_required(VERSION 3.5)
should be added at the top of the file. The version specified may be lower
if you wish to support older CMake versions for this project. For more
information run "cmake --help-policy CMP0000".
This warning is for project developers. Use -Wno-dev to suppress it.
-- Configuring done
-- Generating done
-- Build files have been written to: /home/derek/Downloads/cpp-bredis/examples
Scanning dependencies of target stream-parse
[ 16%] Building CXX object CMakeFiles/stream-parse.dir/stream-parse.o
/home/derek/Downloads/cpp-bredis/examples/stream-parse.cpp:30:33: fatal error: bredis/Connection.hpp: No such file or directory
compilation terminated.
CMakeFiles/stream-parse.dir/build.make:62: recipe for target 'CMakeFiles/stream-parse.dir/stream-parse.o' failed
make[2]: *** [CMakeFiles/stream-parse.dir/stream-parse.o] Error 1
CMakeFiles/Makefile2:67: recipe for target 'CMakeFiles/stream-parse.dir/all' failed
make[1]: *** [CMakeFiles/stream-parse.dir/all] Error 2
Makefile:85: recipe for target 'all' failed
make: *** [all] Error 2
I can find the include files, but I don't know how to config the make process to see them...
Need to check the possibility for using new redis stream example
Sorry for opening an issue for this, I did not find any better means of contacting you.
I would like to invite your community to write a review for an alternative Redis client library that is being proposed for inclusion in Boost. As the author of a Redis client yourself your opinion would be highly valuable. For more information see https://lists.boost.org/Archives/boost/2023/01/253871.php.
Thanks
asio_handler_invoke has been deprecated for a while, but in 1.74, they added compile-time detection for it if BOOST_ASIO_NO_DEPRECATED
is defined.
This makes it impossible to use bredis with that flag, unfortunately, which is a must-have for certain Windows builds (it prevents a mis-detection of a specific constructor overload that leads to a crash).
Hi,
Thanks for a great library, very useful.
I have an issue with multiple subscriptions. One way of subscribing is (following the documentation):
bredis::single_command_t subscribe_cmd{ "subscribe", "channel-1", "channel-2" };
Then one can validate the subscription using:
bredis::marker_helpers::check_subscription<Iterator> check_subscription{std::move(subscribe_cmd)}; ...; // get the 1st reply auto parse_result = ...; bool channel_1_ok = boost::apply_visitor(check_subscription, parse_result.result); ...; // get the 2nd reply parse_result = ...; bool channel_2_ok = boost::apply_visitor(check_subscription, parse_result.result);
However, if subscriptions are not simultaneous, that is we subscribe to channel-1 and later subscribe to channel-2, the validation fails on channel-2:
// get the 2nd reply parse_result = ...; bool channel_2_ok = boost::apply_visitor(check_subscription, parse_result.result);
The reason is that redis response has only 3 fields and the value in the 3rd field is 2, which doesn't comply with the validation tests in check_subscription.
Any work around?
Thanks,
Silviu
like including "bredis.hpp" would result in including every header under bredis/ ?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.