When load testing getting PG errors about crystal-pg HOT 5 CLOSED

Thanks for reporting this. All of the Postgres frames start with a single
ascii char, so that 2 there is suspicious. It'll be at least a week before
I can dig in, but wanted you to know I saw the issue.

On Friday, August 5, 2016, shail [email protected] wrote:

When I run my server and query it manually, everything works as expected.
But when I try to load test with wrk or ApacheBench, I immediately get
the error below, and even after I kill the load test, requests take a long
time and timeout. Here is the code:

DB = PG.connect("postgres://shailpatel@localhost:5432/supercontest_data")

get "/" do |env|
env.response.content_type = "application/json"
result = DB.exec({Int32, String}, "SELECT id, name FROM teams")
hash = { "data" => result.to_hash }
hash.to_json
end

Error:

Exception: Expected PQ::Frame::ParseComplete but got PQ::Frame::Unknown(@type='\u{2}', @bytes=Slice[]) (Exception)
[4449219811] *raise:NoReturn +163
[4449593329] *PG::Connection#exec<Tuple(Int32:Class, String:Class), String, Array(Array(JSON::Type) | Bool | Float32 | Float64 | Hash(String, JSON::Type) | Int32 | Int64 | PG::Numeric | String | Time | Nil)>:PG::Result(Tuple(Int32:Class, String:Class)) +11169
[4449576524] ~procProc(HTTP::Server::Context, String)@./src/supercontest_data_crystal.cr:8 +332
[4449754793] *Kemal::RouteHandler#callHTTP::Server::Context:HTTP::Server::Context +73
[4449809084] *Kemal::StaticFileHandler@HTTP::Handler#call_nextHTTP::Server::Context:(Bool | HTTP::Server::Context | HTTP::Server::Response | HTTP::Server::Response::Output | IO::FileDescriptor+ | Int32 | Nil) +556
[4449760866] *Kemal::CommonExceptionHandler#callHTTP::Server::Context:(Bool | HTTP::Server::Context | HTTP::Server::Response | HTTP::Server::Response::Output | IO::FileDescriptor+ | Int32 | Nil) +2370
[4449765265] *Kemal::CommonLogHandler#callHTTP::Server::Context:HTTP::Server::Context +1793
[4449757783] *Kemal::InitHandler#callHTTP::Server::Context:(Bool | HTTP::Server::Context | HTTP::Server::Response | HTTP::Server::Response::Output | IO::FileDescriptor+ | Int32 | Nil) +615
[4449703673] *HTTP::Server#handle_client<(TCPSocket+ | Nil)>:Nil +10489
[4449235224] *Fiber#run:(Int64 | Nil) +40

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#53, or mute the thread
https://github.com/notifications/unsubscribe-auth/AAAHtXhgIFdutqaN29EUAyunuPsoi2y_ks5qc9aegaJpZM4JeKJH
.

from crystal-pg.

@shail could you please give https://github.com/will/crystal-pg/tree/mutex a try, and see if putting a mutex around the query fixes it for you?

from crystal-pg.

@will I pulled that branch and now am not getting that error anymore. Thanks for the help!

from crystal-pg.

Thanks for verifying.

I want to do some more benchmarking to make sure that holding the mutex isn't too much of an overhead. I wasn't sure if this was something the connection should be doing, or if some higher level thing that takes care of pooling should.

I did a bit of benchmarking and it seems the overhead is very low, and if that turns out to be true, then might as well just do it.

from crystal-pg.

released as v0.9.1

from crystal-pg.