czkonverse / m5_uncertainty_3rd_place Goto Github PK

kaggle name: Ouranos

You can visit M5 Uncertainty competition for more info about the competition or/and raw data.
To learn more about my solution you can visit my write-up.

Our joint paper,with my colleague Konstantinos Vogklis, is published by IJF.

Preprint can be found here.

CPU: Intel(R) Xeon(R) CPU X5650 @ 2.67GHz with 24 cores
GPU: GeForce GTX 1080 (server has 2 gpus but I only used 1)
74 GB memory (used only a fraction)

Ubuntu 18.04.2 LTS

Python 3.6.7
CUDA 10.0.130
cuddn 7.5.0
nvidia drivers version 410.48

cd M5_Uncertainty_3rd_place
mkdir submissions
mkdir processed_data
mkdir lgbm_datasets
mkdir models

• python prepare_data.py
  Read training data from RAW_DATA_DIR (specified in SETTINGS.json)
  Run any preprocessing steps
  Save the processed data to PROCESSED_DATA_DIR (specified in SETTINGS.json)

• python train.py
  Read training data both from RAW_DATA_DIR and PROCESSED_DATA_DIR (specified in SETTINGS.json)
  Train and Save model weights to MODELS_DIR (specified in SETTINGS.json)

• python predict.py
  Predictions for M5 Accuracy competition, used as starting point to M5 uncertainty competition.
  Read test data both from RAW_DATA_DIR and PROCESSED_DATA_DIR (specified in SETTINGS.json)
  Load models from MODELS_DIR (specified in SETTINGS.json)
  Use models to make predictions on new samples
  Save predictions with filename 'lgbm3keras1.csv.gz' to SUBMISSION_DIR (specified in SETTINGS.json)

• python predict_uncertainty.py lgbm3keras1.csv.gz
  Predictions for M5 uncertainty competition.
  lgbm3keras1.csv.gz is the output of predict.py, located in SUBMISSION_DIR (specified in SETTINGS.json). Placing another file inside submissions folder and calling predict_uncertainty.py on that file will generate a different submission file (same name).
  Read test data from RAW_DATA_DIR (specified in SETTINGS.json)
  Load models M5 accuracy submission file placed in SUBMISSION_DIR (specified in SETTINGS.json)
  Make predictions on new samples
  Save predictions to SUBMISSION_DIR (specified in SETTINGS.json)

To make predictions on new data, replace raw files with new.
Raw files:

• sales_train_evaluation.csv
• calendar.csv
• sell_prices.csv
  sample_submission.csv is also needed.
  Second half (rowwise) of submission file correspond to new predictions.

• M5 Uncertainty final submission: submission_uncertainty.csv

• M5 Accuracy submission, used as starting point of M5 Uncertainty: lgbm3keras1.csv.gz

• M5 Accuracy keras1 submission: Keras_CatEmb_final3_et1941_ver-17last.csv.gz

• M5 Accuracy keras2 submission: Keras_CatEmb_final3_et1941_ver-EN1EN2Emb1.csv.gz

• M5 Accuracy keras3 submission: Keras_CatEmb_final3_et1941_ver-noEN1EN2.csv.gz

• M5 Accuracy average 3 keras submission: Keras_CatEmb_final3_et1941_ver-avg3.csv.gz

• M5 Accuracy lightgbm submission: lgbm_final_VER4.csv.gz

lgbm_datasets folder must be empty when starting a training run