Comments (3)
I cannot prove that pgreplay is not the problem here, but it doesn't do anything different if you run it a second time with the same log file, so I would search for the problem on the database side.
My best guess is still caching. Try to reboot the database server machine between the runs (or drop the kernel page cache in a less invasive fashion) and see if you can still observe the increase in performance. If not, that will prove that the difference is on the server side. Use pg_stat_statements
to determine which statements are slow the first time and fast the second time. Or you can run with auto_explain enabled and auto_explain.log_analyze = on
and auto_explain.log_buffers = on
, then you can see the execution plans of the slow queries in the log. These plans will help to determine the cause of the problem. Note that auto_explain slows down processing!
from pgreplay.
Hi @laurenz
So, I've finally found the problem. I forgot to mention that I'm using an RDS restored from a snapshot.
Today I learned: "If you access data that hasn't been loaded yet, the DB instance immediately downloads the requested data from Amazon S3, and then continues loading the rest of the data in the background."
AWS calls this "lazy load", and it's documented here: https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_RestoreFromSnapshot.html
So basically, I was using the RDS from the moment it was "available", but all my queries were waiting a long time for the data to be fetched from S3 the first time they executed.
I've solved this, by importing the SQL dump the "good old way" with psql
from pgreplay.
That's interesting information.
Thanks for sharing this!
from pgreplay.
Related Issues (20)
- make install fails on Mac OS X HOT 5
- Unable to compile pgreplay on ubuntu 18.04 HOT 2
- pgreplay seems to have stopped suddenly when executing HOT 10
- Generating undo records by using pgreplay HOT 1
- pgreplay not working with RDS csvlog HOT 2
- Queries per second HOT 3
- Replay freezes, all queries stuck in ClientRead state. HOT 3
- replay and uuids HOT 1
- Will the program support pg11 onward? HOT 3
- Transaction Deadlocks on Replay HOT 7
- Invalid free on replaying server-side prepare "ROLLBACK; DEALLOCATE ALL". HOT 4
- Tests are failing after installation HOT 6
- connection to database failed for a specific session HOT 1
- Parsing line error HOT 12
- Upgrade from pg 12 to pg 14 HOT 2
- ./configure fails on pg15.0 HOT 2
- Opened connections HOT 2
- Will PQgetResult cause stat_stmt statistics errors? HOT 1
- docker build Fails with Error (from master, commit a2f7c113197ed5623b35f09bfc299aae971d56e8) HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pgreplay.