[Bug] `monero-blockchain-prune` copies the whole DB. Extremely slowly. about monero HOT 12 OPEN

What type of storage is F:\? HDD, SSD, ...?

from monero.

My fastest old HDD. Here's a Cross Platform Disk Test result:

This doesn't matter. If it took three days and gave me an actually deflated pruned DB — I wouldn't complain much.

Was it supposed to copy all of the txs_prunable? It all looked good until that started.

from monero.

We have a pull request to speed up monero-blockchain-prune: #8503

Together with an SSD it should take about 1-2h, we just haven't included the code yet in the latest release version, only master branch.

HDDs have slow random read / write speeds so they will always be significantly slower than SSDs in monero.

Regarding your reported issue, the easiest way to solve it would be to re-sync from scratch with setting --prune-blockchain from the beginning. This issue should stay open until someone reproduces it.

from monero.

So I had a node that was, as I recall, initially ran with pruning, then without, then pruned again using the Monero daemon.

Can you explain this in more detail? You can't unprune a daemon so it was always pruned, unless you deleted the blockchain in-between?

from monero.

No, I didn't delete it. Just ran the daemon without the pruning option after initial DB creation in GUI wallet, where I picked the pruned option, until it gradually reached 172 GB, which seems like the size of the full ledger.
It almost sounds like another issue to be opened for monerod separately, but I don't have detailed statistics to illustrate it.
Unless you wanna explain to me how a supposedly pruned DB of either/both monerod and monero-blockchain-prune is 172 GB.

monero-blockchain-prune reported the DB created by monerod as pruned.
And there was some option to check if it's pruned in monerod itself, IIRC, which also reported it as pruned.

from monero.

As for the HDD and speed… Look, the point I'm making is…
The pruned entries are already marked for rewriting/discarding.
All that I would expect of a program intended to deflate the DB, to discard those stained entries, is to simply sequentially copy the entries that aren't stained, skipping the stained ones.
How complicated could that really be?
I'd be able to copy this entire disk twice over in a matter of minutes, hours at most with the slow built-in OS copier. Not days. It's insult to injury when it copies the entries it promised to skip and then also takes an eternity to do what I never wanted it to do.

from monero.

No, I didn't delete it. Just ran the daemon without the pruning option

That doesn't matter. Once you use the option, that's recorded in the DB so it will always be pruning after that point. Only deleting the blockchain DB would reset that option.

from monero.

You'd think that, but 172 GB…
Refresh the page.
I edited the message you responded to with more detail.
Expected GitHub's dynamic UI to reflect my changes in real time.
Detailed it in the head post too now.

from monero.

"A" appears to be OPs case:

A) Internally Pruned

1. sync blockchain normally
   = ~170gb node (expected)
2. run with --prune-blockchain
   = The database is now pruned internally, but is still ~170gb on disk (expected)
3. Run monero-blockchain-prune --copy-pruned-database
   = copies the database from 2 (~170gb, expected?)

if OP did 3b (below):
3b. Run monero-blockchain-prune w/o the copy flag
= does this shrink the db? Or should this error out and say db is already pruned?
edit: op says this does indeed throw an error
Perhaps op expected this to shrink the db instead of throwing an error

This use case should work:
B) Unpruned BC

1. fully sync blockchain normally
   = ~170gb node (expected)
2. Run monero-blockchain-prune
   = prunes the database from 1 (expected)

from monero.

Unless you wanna explain to me how a supposedly pruned DB of either/both monerod and monero-blockchain-prune is 172 GB.

pruning doesn't free already allocated space on disk. It frees space within the db and wont allocate more until the "internal" db reaches the size on disk.

monero-blockchain-prune reported the DB created by monerod as pruned.
And there was some option to check if it's pruned in monerod itself, IIRC, which also reported it as pruned.

if you ever ran --prune-blockchain, it was pruned (pruned != shrunken) and monerod is reporting correctly.

This appears to be caused by usage of --copy-pruned-database.
I'm interested in if it shrinks a pruned but overallocated BC if the copy flag if dropped.

from monero.

I think this is an issue with a bugged / corrupted database, @Erquint says they initially started monerod with the --pruned-blockchain flag, which means it should have never gotten that big in the first place.

from monero.

Correction to a thing many of you seem to misinterpret: I didn't start the DB with a pruning flag. The official Monero GUI Wallet offered the two choices and I picked pruned in the GUI when initializing. I added the flag at some later point. But as we established, if the GUI did its job — the flag should do nothing.

Admittedly, I initialized the DB a very long time ago.

@nahuhh

Perhaps op expected this to shrink the db instead of throwing an error

It would be nice if it did what's on the tin. Can't argue there.

Run monero-blockchain-prune --copy-pruned-database
= copies the database from 2 (~170gb, expected?)

If you suppose this behavior expected, care to explain what the purpose is?
To basically replicate the function of monerod --prune-blockchain?
If that's the case — why didn't the monero-blockchain-prune complain that the DB is already pruned with the --copy-pruned-database flag as it did without it? This inconsistency created much confusion, without it even being clear if either both or none modes should raise that complaint.

pruning doesn't free already allocated space

Well aware. Everything I wrote in this here entire issue is under that very assumption.
I think you missed the monero-blockchain-prune part of my question. That's what was supposed to shrink it. Surely, running all of the options on the table must've produced a shrunk DB file — otherwise shrinking wouldn't even be a thing we're able to discuss as an applicable concept.

Alright, so if we go with the assumption that monero-blockchain-prune --copy-pruned-database worked as expected, then the issue boils down to monero-blockchain-prune (no such flag) not working as expected, printing Blockchain is already pruned, use --copy-pruned-database to copy it anyway and halting.

from monero.