PGReplay query limit ? about pgreplay HOT 7 CLOSED

Maybe you want to use the "-s" option to speed up the replay on the destination host?

I often use pgreplay -j -s 10000 [...] to replay a sql load as fast as possible.

from pgreplay.

I'm not that interested in speeding up the replay, I'm more interested in knowing if the message Execution is 3 minutes behind schedule comes because postgres is loaded or because pgreplay is loaded.. I tend to thinks it's pgreplay but I am trying to confirm now.
We expect to test our master server also but this one has 10 times the amount of queries so if pgreplay is too loaded, it's gonna be an issue.

from pgreplay.

It means one of the following:

• pgreplay is indeed overloaded and cannot cope with running that many statements simultaneously.

• Your database is slower than the original database.

It is normal for pgreplay to keep one core busy, since it is constantly polling the database connections for messages from the database server; that is not necessarily a sign that it is not keeping up.

If execution does not fall more then 3 minutes behind schedule, I'd suspect that a couple of queries just took longer than expected. You might use a tool like pg_stat_statements or pgBadger to figure out which queries took long.

Usually, if the target system is consistently slower than the original database, you'll see pgreplay falling behind schedule more and more. If pgreplay does not fall more than 3 minutes behind schedule on a 20 hour run, I'd say there is nothing much to worry.

How many statements per second do you have? pgreplay tends to get overloaded if that goes into the thousands.

The option -j only speeds up execution if there are times without activity — it just skips these intervals instead of doing nothing.

Since pgreplay is single-threaded, it won't be faster if you run it on a machine with more cores.

from pgreplay.

Thanks for the explanations. I'll wait for the run to end in order to draw conclusions.
For the moment I'm at ~500 stmts/sec but when testing the next server (in a month probably ), I expect to reach ~5000stmts/secs. Will it be an issue ?

from pgreplay.

I don't know the limits of pgreplay (never used it on such busy databases), but I have heard reports that it cannot keep up with very many statements per second.

Try it and give me feedback :^)

from pgreplay.

All right, I'll you know at that point ;)
In the meantime I'll try to finish the test with my current setup and see how far it drifts

Thanks for the quick answers

from pgreplay.

The test finishes somehow properly (I open another question for that) and does not drift more than 3 minutes after 22hours. ;)

from pgreplay.