Integer Sequence Learning is one of the Kaggle competitions. Basically, you are to guess right the last number of integer sequences. The competition already ended a few months ago, but we challenge it for fun. Luckily, it's possible to check the score after the deadline.
We apply recurrent neural networks to this task. (Why not?) This task particularly interests us as it's analogous to word prediction. That is, a number and its composing digits are equivalent to a word and characters. Based on this observation, we conduct two experiments. In the first one, we try to predict the last number based on the preceding numbers. In another one, we sequentially generate a digit based on the preceding digits.
numpy >= 1.11.1
keras >= 1.2.1
Check this for detailed description of the task.
- Training
- Inputs -> GRU Layer 1 of 1000 hidden units -> Dropout -> GRU Layer 2 of 1000 hidden units -> Dropout -> Time distributed dense -> Outputs
- Inference
- Inputs -> GRU Layer 1 of 1000 hidden units -> GRU Layer 2 of 1000 hidden units -> Dense -> Outputs
- STEP 1. Download raw training data and test data and extract them to
data/
folder. - STEP 2. Run
exp1/prepro.py
to make train/val/test data. - STEP 3. Run
exp1/train.py
. - STEP 4. Run
exp1/submit.py
to get the final prediction results. - STEP 5. Run
exp2/prepro.py
to make train/val/test data. - STEP 6. Run
exp2/train.py
. - STEP 7. Run
exp2/submit.py
to get the final prediction results. - STEP 8. Run
ensemble.py
to mix the results of exp1 and exp2.
- Download the output files of STEP 2, then extract them to
exp1/data/
folder. - Download the pre-trained model file for exp1, then extract it to
exp1/ckpt/
folder. - Run
exp1/submit.py
- Download the output files of STEP 5, then extract them to
exp2/data/
folder. - Download the pre-trained model file for exp2, then extract it to
exp2/ckpt/
folder. - Run
exp2/submit.py
- Run
ensemble.py
Model Type | Score |
---|---|
Exp1 | 0.13477 |
Exp2 | 0.14557 |
Ensemble | 0.15547 |