Comments (3)
Query to generate the report:
SELECT
relname AS TableName
,n_live_tup AS LiveTuples
,n_dead_tup AS DeadTuples,n_dead_tup::decimal/n_live_tup scale
,last_autovacuum AS Autovacuum
,last_autoanalyze AS Autoanalyze
FROM pg_stat_user_tables WHERE n_dead_tup > 100 AND schemaname = 'public' ORDER by deadtuples desc;
from h.
Results of running vacuum manually on the DB
annotation_metadata
vacuum (VERBOSE, ANALYZE) annotation_metadata;
INFO: vacuuming "public.annotation_metadata"
INFO: scanned index "pk__annotation_metadata" to remove 11371 row versions
DETAIL: CPU: user: 0.03 s, system: 0.02 s, elapsed: 0.18 s
INFO: "annotation_metadata": removed 11371 row versions in 6534 pages
DETAIL: CPU: user: 0.08 s, system: 0.10 s, elapsed: 0.69 s
INFO: index "pk__annotation_metadata" now contains 417776 row versions in 1179 pages
DETAIL: 11371 index row versions were removed.
0 index pages have been deleted, 0 are currently reusable.
CPU: user: 0.00 s, system: 0.00 s, elapsed: 0.00 s.
INFO: "annotation_metadata": found 12445 removable, 417776 nonremovable row versions in 12232 out of 12232 pages
DETAIL: 0 dead row versions cannot be removed yet, oldest xmin: 1252518253
There were 4571 unused item pointers.
Skipped 0 pages due to buffer pins, 0 frozen pages.
0 pages are entirely empty.
CPU: user: 0.24 s, system: 0.29 s, elapsed: 2.23 s.
INFO: vacuuming "pg_toast.pg_toast_13691950"
INFO: index "pg_toast_13691950_index" now contains 0 row versions in 1 pages
DETAIL: 0 index row versions were removed.
0 index pages have been deleted, 0 are currently reusable.
CPU: user: 0.00 s, system: 0.00 s, elapsed: 0.00 s.
INFO: "pg_toast_13691950": found 0 removable, 0 nonremovable row versions in 0 out of 0 pages
DETAIL: 0 dead row versions cannot be removed yet, oldest xmin: 1252518268
There were 0 unused item pointers.
Skipped 0 pages due to buffer pins, 0 frozen pages.
0 pages are entirely empty.
CPU: user: 0.00 s, system: 0.00 s, elapsed: 0.00 s.
INFO: analyzing "public.annotation_metadata"
INFO: "annotation_metadata": scanned 12232 of 12232 pages, containing 417776 live rows and 0 dead rows; 30000 rows in sample, 417776 estimated total rows
VACUUM
annotation_slim
vacuum (VERBOSE, ANALYZE) annotation_slim;
INFO: vacuuming "public.annotation_slim"
INFO: scanned index "pk__annotation_slim" to remove 149250 row versions
DETAIL: CPU: user: 0.31 s, system: 0.08 s, elapsed: 0.97 s
INFO: scanned index "uq__annotation_slim__pubid" to remove 149250 row versions
DETAIL: CPU: user: 0.72 s, system: 0.29 s, elapsed: 2.17 s
INFO: scanned index "ix__annotation_slim_created" to remove 149250 row versions
DETAIL: CPU: user: 0.34 s, system: 0.12 s, elapsed: 1.30 s
INFO: scanned index "ix__annotation_slim_updated" to remove 149250 row versions
DETAIL: CPU: user: 0.25 s, system: 0.17 s, elapsed: 1.18 s
INFO: "annotation_slim": removed 149250 row versions in 16438 pages
DETAIL: CPU: user: 0.31 s, system: 0.37 s, elapsed: 2.29 s
INFO: index "pk__annotation_slim" now contains 2017051 row versions in 6046 pages
DETAIL: 141763 index row versions were removed.
0 index pages have been deleted, 0 are currently reusable.
CPU: user: 0.00 s, system: 0.00 s, elapsed: 0.00 s.
INFO: index "uq__annotation_slim__pubid" now contains 2017051 row versions in 10557 pages
DETAIL: 118318 index row versions were removed.
0 index pages have been deleted, 0 are currently reusable.
CPU: user: 0.00 s, system: 0.00 s, elapsed: 0.00 s.
INFO: index "ix__annotation_slim_created" now contains 2017051 row versions in 6274 pages
DETAIL: 148895 index row versions were removed.
1 index pages have been deleted, 1 are currently reusable.
CPU: user: 0.00 s, system: 0.00 s, elapsed: 0.00 s.
INFO: index "ix__annotation_slim_updated" now contains 2017051 row versions in 6508 pages
DETAIL: 149184 index row versions were removed.
1 index pages have been deleted, 0 are currently reusable.
CPU: user: 0.00 s, system: 0.00 s, elapsed: 0.00 s.
INFO: "annotation_slim": found 13372 removable, 2017051 nonremovable row versions in 22717 out of 22717 pages
DETAIL: 0 dead row versions cannot be removed yet, oldest xmin: 1252518685
There were 61962 unused item pointers.
Skipped 0 pages due to buffer pins, 0 frozen pages.
0 pages are entirely empty.
CPU: user: 2.23 s, system: 1.26 s, elapsed: 10.06 s.
INFO: analyzing "public.annotation_slim"
INFO: "annotation_slim": scanned 22717 of 22717 pages, containing 2017055 live rows and 0 dead rows; 30000 rows in sample, 2017055 estimated total rows
user
vacuum (VERBOSE, ANALYZE) "user"
INFO: vacuuming "public.user"
INFO: scanned index "pk__user" to remove 106697 row versions
DETAIL: CPU: user: 0.37 s, system: 0.06 s, elapsed: 1.88 s
INFO: scanned index "uq__user__email" to remove 106697 row versions
DETAIL: CPU: user: 0.63 s, system: 0.40 s, elapsed: 3.40 s
INFO: scanned index "ix__user__userid" to remove 106697 row versions
DETAIL: CPU: user: 0.79 s, system: 0.34 s, elapsed: 3.03 s
INFO: scanned index "ix__user__nipsa" to remove 106697 row versions
DETAIL: CPU: user: 0.00 s, system: 0.00 s, elapsed: 0.00 s
INFO: scanned index "ix__user__email" to remove 106697 row versions
DETAIL: CPU: user: 0.14 s, system: 0.15 s, elapsed: 1.01 s
INFO: "user": removed 106697 row versions in 38838 pages
DETAIL: CPU: user: 0.33 s, system: 0.00 s, elapsed: 0.77 s
INFO: index "pk__user" now contains 2045951 row versions in 6573 pages
DETAIL: 44507 index row versions were removed.
0 index pages have been deleted, 0 are currently reusable.
CPU: user: 0.00 s, system: 0.00 s, elapsed: 0.00 s.
INFO: index "uq__user__email" now contains 2045956 row versions in 20778 pages
DETAIL: 105102 index row versions were removed.
0 index pages have been deleted, 0 are currently reusable.
CPU: user: 0.00 s, system: 0.00 s, elapsed: 0.00 s.
INFO: index "ix__user__userid" now contains 2045956 row versions in 17494 pages
DETAIL: 92136 index row versions were removed.
0 index pages have been deleted, 0 are currently reusable.
CPU: user: 0.00 s, system: 0.00 s, elapsed: 0.00 s.
INFO: index "ix__user__nipsa" now contains 327 row versions in 2 pages
DETAIL: 5 index row versions were removed.
0 index pages have been deleted, 0 are currently reusable.
CPU: user: 0.00 s, system: 0.00 s, elapsed: 0.00 s.
INFO: index "ix__user__email" now contains 2045951 row versions in 5641 pages
DETAIL: 6867 index row versions were removed.
0 index pages have been deleted, 0 are currently reusable.
CPU: user: 0.00 s, system: 0.00 s, elapsed: 0.00 s.
INFO: "user": found 24532 removable, 2045873 nonremovable row versions in 56607 out of 56609 pages
DETAIL: 0 dead row versions cannot be removed yet, oldest xmin: 1252519478
There were 441433 unused item pointers.
Skipped 2 pages due to buffer pins, 0 frozen pages.
0 pages are entirely empty.
CPU: user: 3.05 s, system: 1.43 s, elapsed: 17.18 s.
INFO: vacuuming "pg_toast.pg_toast_16818"
INFO: index "pg_toast_16818_index" now contains 0 row versions in 1 pages
DETAIL: 0 index row versions were removed.
0 index pages have been deleted, 0 are currently reusable.
CPU: user: 0.00 s, system: 0.00 s, elapsed: 0.00 s.
INFO: "pg_toast_16818": found 0 removable, 0 nonremovable row versions in 0 out of 0 pages
DETAIL: 0 dead row versions cannot be removed yet, oldest xmin: 1252519588
There were 0 unused item pointers.
Skipped 0 pages due to buffer pins, 0 frozen pages.
0 pages are entirely empty.
CPU: user: 0.00 s, system: 0.00 s, elapsed: 0.00 s.
INFO: analyzing "public.user"
INFO: "user": scanned 30000 of 56609 pages, containing 1085163 live rows and 0 dead rows; 30000 rows in sample, 2047666 estimated total rows
VACUUM
document
vacuum (VERBOSE, ANALYZE) "document";
INFO: vacuuming "public.document"
INFO: scanned index "pk__document" to remove 146555 row versions
DETAIL: CPU: user: 0.41 s, system: 0.21 s, elapsed: 1.27 s
INFO: "document": removed 146555 row versions in 50821 pages
DETAIL: CPU: user: 0.86 s, system: 1.16 s, elapsed: 26.01 s
INFO: index "pk__document" now contains 3197711 row versions in 11155 pages
DETAIL: 80802 index row versions were removed.
0 index pages have been deleted, 0 are currently reusable.
CPU: user: 0.00 s, system: 0.00 s, elapsed: 0.00 s.
INFO: "document": found 71494 removable, 3092826 nonremovable row versions in 93666 out of 96076 pages
DETAIL: 4 dead row versions cannot be removed yet, oldest xmin: 1252519827
There were 997946 unused item pointers.
Skipped 0 pages due to buffer pins, 2259 frozen pages.
0 pages are entirely empty.
CPU: user: 2.73 s, system: 3.38 s, elapsed: 43.28 s.
INFO: vacuuming "pg_toast.pg_toast_16986"
INFO: scanned index "pg_toast_16986_index" to remove 242 row versions
DETAIL: CPU: user: 0.00 s, system: 0.00 s, elapsed: 0.01 s
INFO: "pg_toast_16986": removed 242 row versions in 121 pages
DETAIL: CPU: user: 0.00 s, system: 0.00 s, elapsed: 0.01 s
INFO: index "pg_toast_16986_index" now contains 8309 row versions in 137 pages
DETAIL: 242 index row versions were removed.
0 index pages have been deleted, 0 are currently reusable.
CPU: user: 0.00 s, system: 0.00 s, elapsed: 0.00 s.
INFO: "pg_toast_16986": found 238 removable, 4038 nonremovable row versions in 730 out of 1624 pages
DETAIL: 0 dead row versions cannot be removed yet, oldest xmin: 1252520000
There were 349 unused item pointers.
Skipped 0 pages due to buffer pins, 569 frozen pages.
0 pages are entirely empty.
CPU: user: 0.01 s, system: 0.00 s, elapsed: 0.19 s.
INFO: analyzing "public.document"
INFO: "document": scanned 30000 of 96076 pages, containing 997799 live rows and 2 dead rows; 30000 rows in sample, 3195485 estimated total rows
VACUUM
annotation
vacuum (VERBOSE, ANALYZE) "annotation";
INFO: vacuuming "public.annotation"
INFO: scanned index "pk__annotation" to remove 5211584 row versions
DETAIL: CPU: user: 27.45 s, system: 4.85 s, elapsed: 95.88 s
INFO: scanned index "ix__annotation_groupid" to remove 5211584 row versions
DETAIL: CPU: user: 13.73 s, system: 3.38 s, elapsed: 48.87 s
INFO: scanned index "ix__annotation_tags" to remove 5211584 row versions
DETAIL: CPU: user: 5.32 s, system: 0.47 s, elapsed: 9.71 s
INFO: scanned index "ix__annotation_userid" to remove 5211584 row versions
DETAIL: CPU: user: 13.81 s, system: 6.12 s, elapsed: 86.94 s
INFO: scanned index "ix__annotation_updated" to remove 5211584 row versions
DETAIL: CPU: user: 5.92 s, system: 2.04 s, elapsed: 23.91 s
INFO: scanned index "ix__annotation_document_id" to remove 5211584 row versions
DETAIL: CPU: user: 9.03 s, system: 2.75 s, elapsed: 37.64 s
INFO: scanned index "ix__annotation_deleted" to remove 5211584 row versions
DETAIL: CPU: user: 16.25 s, system: 2.11 s, elapsed: 38.58 s
INFO: scanned index "ix__annotation_thread_root" to remove 5211584 row versions
DETAIL: CPU: user: 16.87 s, system: 2.65 s, elapsed: 39.59 s
INFO: scanned index "ix__annotation_created" to remove 5211584 row versions
DETAIL: CPU: user: 5.58 s, system: 1.49 s, elapsed: 21.56 s
INFO: "annotation": removed 5211584 row versions in 1262051 pages
DETAIL: CPU: user: 17.25 s, system: 26.47 s, elapsed: 439.31 s
INFO: index "pk__annotation" now contains 56448211 row versions in 298569 pages
DETAIL: 4867907 index row versions were removed.
0 index pages have been deleted, 0 are currently reusable.
CPU: user: 0.00 s, system: 0.00 s, elapsed: 0.00 s.
INFO: index "ix__annotation_groupid" now contains 56448234 row versions in 335018 pages
DETAIL: 5163092 index row versions were removed.
16619 index pages have been deleted, 376 are currently reusable.
CPU: user: 0.00 s, system: 0.00 s, elapsed: 0.00 s.
INFO: index "ix__annotation_tags" now contains 53965838 row versions in 23730 pages
DETAIL: 5641616 index row versions were removed.
434 index pages have been deleted, 486 are currently reusable.
CPU: user: 0.05 s, system: 0.16 s, elapsed: 1.18 s.
INFO: index "ix__annotation_userid" now contains 56448263 row versions in 582861 pages
DETAIL: 4694472 index row versions were removed.
13643 index pages have been deleted, 1631 are currently reusable.
CPU: user: 0.00 s, system: 0.00 s, elapsed: 0.00 s.
INFO: index "ix__annotation_updated" now contains 56448280 row versions in 201173 pages
DETAIL: 5208383 index row versions were removed.
44 index pages have been deleted, 37 are currently reusable.
CPU: user: 0.00 s, system: 0.00 s, elapsed: 0.00 s.
INFO: index "ix__annotation_document_id" now contains 56448289 row versions in 245514 pages
DETAIL: 5201344 index row versions were removed.
1990 index pages have been deleted, 597 are currently reusable.
CPU: user: 0.00 s, system: 0.00 s, elapsed: 0.00 s.
INFO: index "ix__annotation_deleted" now contains 56448289 row versions in 239822 pages
DETAIL: 4955033 index row versions were removed.
15105 index pages have been deleted, 293 are currently reusable.
CPU: user: 0.00 s, system: 0.00 s, elapsed: 0.00 s.
INFO: index "ix__annotation_thread_root" now contains 56448295 row versions in 255720 pages
DETAIL: 5202038 index row versions were removed.
13283 index pages have been deleted, 25 are currently reusable.
CPU: user: 0.00 s, system: 0.00 s, elapsed: 0.00 s.
INFO: index "ix__annotation_created" now contains 56448299 row versions in 175573 pages
DETAIL: 4648290 index row versions were removed.
0 index pages have been deleted, 0 are currently reusable.
CPU: user: 0.00 s, system: 0.00 s, elapsed: 0.00 s.
INFO: "annotation": found 1294481 removable, 27274824 nonremovable row versions in 4499257 out of 8502689 pages
DETAIL: 12 dead row versions cannot be removed yet, oldest xmin: 1252520396
There were 2038930 unused item pointers.
Skipped 1 page due to buffer pins, 3997430 frozen pages.
0 pages are entirely empty.
CPU: user: 177.92 s, system: 117.26 s, elapsed: 1760.33 s.
INFO: vacuuming "pg_toast.pg_toast_16966"
INFO: scanned index "pg_toast_16966_index" to remove 38012 row versions
DETAIL: CPU: user: 0.45 s, system: 0.21 s, elapsed: 12.48 s
INFO: "pg_toast_16966": removed 38012 row versions in 23440 pages
DETAIL: CPU: user: 0.31 s, system: 0.44 s, elapsed: 10.40 s
INFO: index "pg_toast_16966_index" now contains 7512272 row versions in 22906 pages
DETAIL: 38012 index row versions were removed.
1 index pages have been deleted, 1 are currently reusable.
CPU: user: 0.00 s, system: 0.00 s, elapsed: 0.00 s.
INFO: "pg_toast_16966": found 2514 removable, 2641514 nonremovable row versions in 286504 out of 852796 pages
DETAIL: 0 dead row versions cannot be removed yet, oldest xmin: 1252528637
There were 75776 unused item pointers.
Skipped 0 pages due to buffer pins, 562122 frozen pages.
0 pages are entirely empty.
CPU: user: 3.62 s, system: 4.38 s, elapsed: 65.13 s.
INFO: analyzing "public.annotation"
INFO: "annotation": scanned 30000 of 8502758 pages, containing 199513 live rows and 1 dead rows; 30000 rows in sample, 56547025 estimated total rows
from h.
Set some big tables scale factor to 5%. Closing this for now.
from h.
Related Issues (20)
- Identify snapshots on archiving websites with their source HOT 2
- ClinGen issue with tags HOT 9
- Import / Export - Role based permission HOT 4
- [passlib.handlers.bcrypt:WARNING] (trapped) error reading bcrypt version HOT 2
- Handled deleted users in annotation_slim sync
- Add a limit to purging deleted annotations
- Index limitations with the `annotation` and `document_uri` tables HOT 3
- Complete a UX design for the new self-service user deletion experience HOT 1
- Delete API tokens when deleting users
- Find a way to prevent coupling future migrations with business domain code by mistake HOT 2
- Migration `8e3417e3713b_back_fill_the_token_user_id_column.py` fails with `AttributeError: 'Token' object has no attribute 'userid'. Did you mean: 'user_id'`
- Replace Google Analytics integration with Google Tag Manager HOT 3
- Investigate integrating with Hubspot API for email opt outs.
- SPIKE: Billable users custom date range HOT 1
- Fix custom metrics reporting with New Relic's python agent 9.9.0
- K
- Consider parameterizing and/or using a different port (other than 5000) for dev install
- Finish filling `annotation_slim`
- `/search` crashes for members of groups whose creator was deleted: `AttributeError: 'NoneType' object has no attribute 'username'`
- Improve grouping of user creation errors in New Relic monitoring HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from h.