Question - is there a way to export sacred results off-computer? about sacred HOT 7 CLOSED

I found this resource about zipping a database and tried that: https://dev.to/lucis/setting-up-a-mongodb-backup-on-google-drive-3bbe

mongodump -d DB_NAME --archive=DB_save_8-5-2022.gz --gzip

Then tried using mongorestore on another computer:

mongorestore --gzip --archive=DB_save_8-5-2022.gz

When I run mongosh I can see the restored database, I think:

test> show dbs
admin    40.00 KiB
config  108.00 KiB
local    72.00 KiB
DB_NAME  10.64 GiB
sacred   40.00 KiB

However, when I try to view the database in omniboard like I used to it is unfortunately entirely blank.

omniboard localhost:27017:DB_NAME
(node:28895) [MONGODB DRIVER] Warning: Current Server Discovery and Monitoring engine is deprecated, and will be removed in a future version. To use the new Server Discover and Monitoring engine, pass option { useUnifiedTopology: true } to the MongoClient constructor.
Omniboard is listening on port 9000!
Connection to sacred at mongodb://localhost:27017 established successfully!

Any tips welcome, I'm also still unclear how one would go merging experiments run on different computers.

from sacred.

I had a dumb typo,

omniboard localhost:27017:DB_NAME

Should instead have been

omniboard  -m   localhost:27017:DB_NAME

So the process above worked for zipping a database, restoring it on another computer, and reconnecting to it. If anyone has thoughts on combining experiment results from Computer A and B as mentioned in the original question that would still be valuable!

from sacred.

Hi @FranzEricSchneider, great that you found a solution! Other than that, you could use the FileStorageObserver, which stores the logs in JSON files that can easily be copied between machines. It doesn't support omniboard or other related tools, though.

from sacred.

Hi @thequilo , do you know if it is possible to convert the data stored from a Mongo observer into this file storage form? That might be one way to combine experiments from multiple computers, which I'm still struggling with.

from sacred.

There is currently no easy way to do this. You can loop through the data stored in your mongo db manually and pass them to a file storage observer, similar to #815 but the other way around.

from sacred.

Alright, I will close this issue. If anyone does come up with a way to combine sacred runs from multiple computers I would be very interested, even in my first usage of sacred it's been necessary to run tests on multiple computers because the tests take a long time to run. My only thought at the moment is to go through the existing database on one of the computers and alter all the ids to be something like (13 -> 1013, 24 -> 1024) to prevent ID clashes, then do the database zip/unzip mentioned above.

from sacred.

Okay, in order to zip sacred tests from multiple computers I needed to make sure none of the IDs overlapped. In order to deal with overlaps I went to computer B and changed the IDs in the mongo database like so:

$ mongosh
> # Use the relevant db name
> use mmseg
> # Repeat for all necessary IDs
> runX=db.runs.findOne({_id:X});
> runX._id=100X;
> db.runs.insertOne(runX);
> db.runs.deleteOne({_id:X});
> db.metrics.updateMany({run_id:X}, {$set:{"run_id":100X}});

Then I zipped the database on computer B, moved it to computer A, restored the database, and was left with all of the results on computer A.

from sacred.