Pretrained Models / Training Dataset about oatomobile HOT 14 CLOSED

Hi.
I wonder if I could know when you will release the trained models. I have trouble with reproducing the results.

from oatomobile.

Hi @twsq,

You should be able to download the data by running:

python -c "OUTPUT_DIR='./data'; import oatomobile.datasets; oatomobile.datasets.CARLADataset(id='processed').download_and_prepare(OUTPUT_DIR)"

and train any of the methods, e.g., DIM, by running:

python -m oatomobile.baselines.torch.dim.train --dataset_dir=./data/processed --output_dir=./tmp --num_epochs=1024

We also plan to release the trained models (use this TensorBoard as a reference), as well as a few more examples of how to use oatomobile for other things, such as:

1. making the most out of the gym-compatible API;
2. adding new sensors;
3. creating new benchmarks;
4. adding new baselines;
5. running a sweep of experiments/benchmarks;
6. parsing the generated logs;
7. ...

Please let me know if that answers your questions and if so I will close the issue!

from oatomobile.

Thank you for providing the trained dataset and commands to download it and run training! I ran the training for DIM, and the training curves look similar to those in the provided TensorBoard.

from oatomobile.

Great @twsq!

I am closing this issue, feel free to re-open if you experience any relevant issues in the future.

from oatomobile.

Hi,
Is there a way to access other Carla visual modalities (like first-person view) related to the mentioned data?

from oatomobile.

The hosted dataset doesn't include these modalities, but you can collect and process your own data.

from oatomobile.

Thanks.
Is there anyway to collect the same scenes? like with some config files?
I want to be sure that carnovel benchmark will be OOD from collected data.

from oatomobile.

Unfortunately, we haven't stored the exact scenes :/
However, as long as you collect data from Town01 or/and Town02 then the carnovel benchmark will be OOD since no roundabouts, hills, 45-angle turns etc. are present there!

from oatomobile.

Thanks.

from oatomobile.

Yes, it will be great if you release the trained models. I can't get the same results either.

from oatomobile.

Due to ICLR & AAMAS deadlines, I would speculate that a major new release of the codebase along with the trained models will take place after first weeks of October!

In the meanwhile, I am happy to help with small things, if you have concrete points you would like them to be addressed!

from oatomobile.

Hi,

I needed other modalities like first-person and bird-view. So I recollected the dataset using your datasets.CARLADataset.collect() method and processed it using datasets.CARLADataset.process() with default arguments and randomly selected spawn and destination points from Town1.

The problem is when I train a DIM model (only using lidar) on the new dataset, I don't get the same results as I got on the original dataset (the latter is similar to what you have reported).

Can you share the settings you used for collecting the data? For example, there is a noise argument which is passed to the AutopilotAgent. The default value of this argument is 0.1 but in your paper, you mentioned that the dataset doesn't have any noise. I even recollected the dataset one more time but this time set the noise to 0. But still, the performance is significantly lower than yours.

So I was wondering, are there any other arguments that should not be set to their default values?
(like proximity_tlight_threshold and proximity_vehicle_threshold in AutopilotAgent, how to choose spawn and destination points, num_frame_skips in the process method)

from oatomobile.

Additionally, how did you separate the train and validation data? I mean, did you keep certain parts of the Town1 for validation, or did you just randomly used some scenarios as validation?

Thank you very much

from oatomobile.

Hello. I actually have the same problem. When I train DIM on your data, I get the same learning curves. But when I collect the data myself using the default values in your code, the final training and validation loss increase. Do you have any idea why this happens?

from oatomobile.