Feature Request: Overwrite sbatch options via CLI about seml HOT 4 OPEN

In #79 Daniel mentioned that this should allow overriding Slurm parameters when starting an experiment, which is not addressed by this design.

However, I don't think it's ideal to have start do side-effects like changing the database. I'd rather propose a new command (update?) for updating config values. This would combine the filtering syntax of start with the overriding syntax of this and #79:

seml <collection> update -b 7 -sb mem=25GB -o dataset=imagenet

I'd suggest to add the overriding options to both add and update then.

from seml.

Having an update command raises the potential issue that an overwrite parameter could cause two identical configurations. We would then have to check for duplicates again and potentially remove these. Still, having an override option at add time seems beneficial to me.

from seml.

I'd suggest breaking this up into two issues, as these are more or less orthogonal:

1. Adding sbatch options to add (+ new syntax for jupyter)
2. Adding an update command.

from seml.

Yes, good point! Let's keep the PRs small and manageable.

update could also break our assumption that every batch has consistent Slurm parameters. Creating batch arrays and sharing GPUs both rely on this. As a first step we could only allow changing Slurm parameters of full batches (and document the reasoning behind these restrictions).

from seml.