This github respository is for our NeurIPS 2020 paper "Neural Networks Fail to Learn Periodic Functions and How to Fix It", see the arxiv version below: https://arxiv.org/abs/2006.08195

We have now added a Jupyter notebook file for studying the extrapolation and interpolation properties of neural networks. See 'extrapolation_experiments.ipynb'. In the notebook, a very elementary implementation of the Snake activation function is given.

• Extrapolation experiments and simple implementation of Snake.
• Human body temperature regression task
• Atmospheric temperature regression task
• Regression task on market index

• more advanced implementations of Snake (such as learnable a and initialization correction)

We are on our way cleaning our code! It should be ready within a week or two. We will let you know once we make an update.

In the mean-while, let me comment that Edward Dixon had made an implementation of our method a while ago. It can be found here: https://github.com/EdwardDixon/snake

Ziyin, Tilman