multi-obj-baselines's Introduction

Bag of Baselines

Bag of Baselines implements several multi-objective opimisation methods to create a performance benchmark on two small datasets. To learn more about this work, check out the publication.

Methods

The following methods are proposed and implemented:

SH-EMOA: Speeding up Evolutionary Multi-Objective Algorithms
MO-BOHB: Generalization of BOHB to an Arbitrary Number of Objectives
MS-EHVI: Mixed Surrogate Expected Hypervolume Improvement
MO-BANANAS
BULK & CUT

Datasets

Performance of the methods was evaluated using the following datasets: Oxford-Flowers dataset and Fashion-MNIST.

Organization

The specific code for each of the methods (the main logic of each algorithm) is stored in the methods folder.
In the examples folder you will find a small Python script to run each of the available methods (for the "Fashion-MNIST" or the "flowers" dataset).
Code defining the search space and the evaluation function of the two different problems are defined in the problems folder.

multi-obj-baselines's People

Contributors

Stargazers

Watchers

multi-obj-baselines's Issues

Requirements of these codes

Thank you for sharing these codes.

To run examples, it seems that I need to install some packages, e.g., pytorch, gpytorch.

Could you please add more info about requirements of packages for running these examples?

Thanks a lot.
JM

Example for MSEHVI doesn't seem to run: "status_quo" issue

For context, I'm using the following versions of packages:

ax==0.52.0
ax-platform==0.2.3
botorch==0.6.0

When I try to run this code (by python3 -m examples.msehvi), I get the error

AttributeError: 'Metric' object has no attribute 'metric'

I solved this by changing for eg. line 20 of baselines/problems/simple_problems.py from

objective = MultiObjective([metric_a, metric_b])

objective = MultiObjective(objectives=[Objective(metric=metric_a), Objective(metric=metric_b)])

Now I get the error "ValueError: Optimization config has relative constraint, but model was not fit with status quo.", which I believe has to do with line 125-145 of baselines/methods/msehvi/msehvi.py. Could you please push a fix for this, or clarify the versions of libraries you used?

Recommend Projects