Code Monkey home page Code Monkey logo

deepsphere-cosmo-tf2's Introduction

DeepSphere: a spherical convolutional neural network

Nathanaël Perraudin, Michaël Defferrard, Tomasz Kacprzak, Raphael Sgier

The code in this repository implements a generalization of Convolutional Neural Networks (CNNs) to the sphere. We here model the discretised sphere as a graph of connected pixels. The resulting convolution is more efficient (especially when data doesn't span the whole sphere) and mostly equivariant to rotation (small distortions are due to the non-existence of a regular sampling of the sphere). The pooling strategy exploits a hierarchical pixelisation of the sphere (HEALPix) to analyse the data at multiple scales. The graph neural network model is based on ChebNet and its TensorFlow implementation. The performance of DeepSphere is demonstrated on a discrimination problem: the classification of convergence maps into two cosmological model classes.

Resources

Code:

Papers:

  • DeepSphere: Efficient spherical CNN with HEALPix sampling for cosmological applications, 2018.
    [paper, blog, slides]
  • DeepSphere: towards an equivariant graph-based spherical CNN, 2019.
    [paper, poster]
  • DeepSphere: a graph-based spherical CNN, 2020.
    [paper, slides, video]

Installation

Binder   Click the binder badge to play with the notebooks from your browser without installing anything.

For a local installation, follow the below instructions.

  1. Clone this repository.

    git clone https://github.com/deepsphere/deepsphere-cosmo-tf1.git
    cd deepsphere-cosmo-tf1
  2. Install the dependencies.

    pip install -r requirements.txt

    Note: if you will be working with a GPU, comment the tensorflow==1.6.0 line in requirements.txt and uncomment the tensorflow-gpu==1.6.0 line.

    Note: the code has been developed and tested with Python 3.5 and 3.6. It should work on Python 2.7 with requirements_py27.txt. Please send a PR if you encounter an issue.

  3. Play with the Jupyter notebooks.

    jupyter notebook

Notebooks

The below notebooks contain examples and experiments to play with the model. Look at the first three if you want to use the model with your own data.

  1. Classification of data on the whole sphere. The easiest example to run if you want to play with the model.
  2. Regression from multiple spherical maps. Shows how to regress (multiple) parameters from (multiple) input channels.
  3. Classification of data from part of the sphere with noise. That is the main experiment carried on in the paper (with one configuration of resolution and noise). It requires private data, see below.
  4. Spherical convolution using a graph to represent the sphere. Learn what is the convolution on a graph, and how it works on the sphere. Useful to learn how the model works.
  5. Comparison of the spherical harmonics with the eigenvectors of the graph Laplacian. Get a taste of the difference between the graph representation and the analytic sphere.

Reproducing the results of the paper

Follow the below steps to reproduce the paper's results. While the instructions are simple, the experiments will take a while.

  1. Get the main dataset. You need to ask the ETHZ cosmology research group for a copy of the data. The simplest option is to request access on Zenodo. You will have to write a description of the project for which the dataset is intended to be used.

  2. Preprocess the dataset.

    python data_preprocess.py
    
  3. Run the experiments. The first corresponds to the fully convolutional architecture variant of DeepSphere. The second corresponds to the classic CNN architecture variant. The third corresponds to a standard 2D CNN (spherical maps are projected on the plane). The last two are the baselines: an SVM that classifies histograms and power spectral densities.

    python experiments_deepsphere.py FCN
    python experiments_deepsphere.py CNN
    python experiments_2dcnn.py
    python experiments_histogram.py
    python experiments_psd.py
    

The results will be saved in the results folder. Note that results may vary from one run to the other. You may want to check summaries with tensorboard to verify that training converges. For some experiments, the network needs a large number of epochs to stabilize.

The experiments_deepsphere.py, experiments_2dcnn.py, and experiments_psd.py scripts can be executed in parallel in a HPC setting. You can adapt the launch_cscs.py, launch_cscs_2dcnn.py, and launch_euler.py to your particular setting.

Once the results are computed (or using those stored in the repository), you can reproduce the paper's figures with the figure* notebooks. The results will be saved in the figures folder. You can also look at the original figures stored in the outputs branch.

Experimental

The experimental folder contains unfinished, untested, and buggy code. We leave it as is for our own future reference, and for the extra curious. 😉

License & citation

The content of this repository is released under the terms of the MIT license.
Please consider citing our papers if you find it useful.

@article{deepsphere_cosmo,
  title = {{DeepSphere}: Efficient spherical Convolutional Neural Network with {HEALPix} sampling for cosmological applications},
  author = {Perraudin, Nathana\"el and Defferrard, Micha\"el and Kacprzak, Tomasz and Sgier, Raphael},
  journal = {Astronomy and Computing},
  volume = {27},
  pages = {130-146},
  year = {2019},
  month = apr,
  publisher = {Elsevier BV},
  issn = {2213-1337},
  doi = {10.1016/j.ascom.2019.03.004},
  archiveprefix = {arXiv},
  eprint = {1810.12186},
  url = {https://arxiv.org/abs/1810.12186},
}
@inproceedings{deepsphere_rlgm,
  title = {{DeepSphere}: towards an equivariant graph-based spherical {CNN}},
  author = {Defferrard, Micha\"el and Perraudin, Nathana\"el and Kacprzak, Tomasz and Sgier, Raphael},
  booktitle = {ICLR Workshop on Representation Learning on Graphs and Manifolds},
  year = {2019},
  archiveprefix = {arXiv},
  eprint = {1904.05146},
  url = {https://arxiv.org/abs/1904.05146},
}
@inproceedings{deepsphere_iclr,
  title = {{DeepSphere}: a graph-based spherical {CNN}},
  author = {Defferrard, Michaël and Milani, Martino and Gusset, Frédérick and Perraudin, Nathanaël},
  booktitle = {International Conference on Learning Representations (ICLR)},
  year = {2020},
  url = {https://openreview.net/forum?id=B1e3OlStPB},
}

deepsphere-cosmo-tf2's People

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

deepsphere-cosmo-tf2's Issues

Error saving model

Hello guys,
After training a custom deep-sphere model (based on ResidualLayer+HealpyChebyshev), I decided to save it in keras as usual, via callbacks:
callback_model= tf.keras.callbacks.ModelCheckpoint(
filepath=filepath+"cp-{epoch:04d}.ckpt",
monitor="val_loss",
verbose=1,
save_best_only=True,
save_weights_only=False,
mode="min")
However, I've got this error:

~/anaconda3/lib/python3.8/site-packages/keras/saving/saved_model/save_impl.py in call_and_return_conditional_lossess(*args, **kwargs)
631 def call_and_return_conditional_losses(*args, **kwargs):
632 """Returns layer (call_output, conditional losses) tuple."""
--> 633 call_output = layer_call(*args, **kwargs)
634 if version_utils.is_v1_layer_or_model(layer):
635 conditional_losses = layer.get_losses_for(

TypeError: call() got multiple values for argument 'training'

Then, I tried to save the model located in the examples folder: "quick_start.ipynb", and also I obtained the same error!! Models cannot be saved with model.save_weights(checkpoint_path) either. I wonder if this is a problem from my environment, or this is due to the custom layers defined in the repository.
I really appreciate your comments.
Best,
Javier

Regarding the Autoencoder model

Hi,

This might not be the right place, but didn't find other contact options so posting it here.
It is regarding the autoencoder model in deepshphere tf2 tutorial. Using this tutorial, with a good amount of sample data, I am able to reconstruct a input healpy map with reasonably recovered features. However, instead of passing the training data in both x and y, if I try to pass a label dataset against the training dataset in y, in an attempt to denoising the map, I find that the model doesn't recover the desired features. I am wondering if there is any similar tutorial for denoising or if you have come across similar situation.

Many Thanks
Wasim

I have one question about pytest tests and notebook,

I try to run quick_start.ipynb in examples directory and I got the error

TypeError Traceback (most recent call last)
in
1 tf.keras.backend.clear_session()
----> 2 model = HealpyGCNN(nside=nside, indices=indices, layers=layers, n_neighbors=20)
3 batch_size = 16
4 model.build(input_shape=(None, len(indices), 1))
5 #model.summary(110)

~/deepsphere-cosmo-tf2/deepsphere/healpy_networks.py in init(self, nside, indices, layers, n_neighbors)
85 hp_nn.Healpy_Transformer,hp_nn.HealpyBernstein)):
86 # we need to calculate the current L and get the actual layer
---> 87 sphere = SphereHealpix(subdivisions=current_nside, indexes=current_indices, nest=True,
88 k=self.n_neighbors, lap_type='normalized')
89 current_L = sphere.L

~/.local/lib/python3.9/site-packages/pygsp/graphs/nngraphs/spherehealpix.py in init(self, subdivisions, indexes, nest, **kwargs)
114 ' {} subdivisions.'.format(k, subdivisions))
115
--> 116 super(SphereHealpix, self).init(coords, k=k,
117 kernel_width=kernel_width,
118 **kwargs)

~/.local/lib/python3.9/site-packages/pygsp/graphs/nngraphs/nngraph.py in init(self, features, standardize, metric, order, kind, k, radius, kernel, kernel_width, backend, **kwargs)
417
418
--> 419 neighbors, distances = function(features, metric, order,
420 kind, k, radius, kwargs)
421 # ------ MARTINO's MODIFICATION ------

~/.local/lib/python3.9/site-packages/pygsp/graphs/nngraphs/nngraph.py in _scipy_ckdtree(features, _, order, kind, k, radius, params)
64 params['k'] = features.shape[0] # number of vertices
65 params['distance_upper_bound'] = radius
---> 66 distances, neighbors = tree.query(features, **params)
67 if kind == 'knn':
68 return neighbors, distances

_ckdtree.pyx in scipy.spatial._ckdtree.cKDTree.query()

_ckdtree.pyx in scipy.spatial._ckdtree.get_num_workers()

TypeError: Unexpected keyword argument {'n_jobs': -1}

and also I tested installation via pytest test and got same error in
test_gnn_transformers.py
tests/test_healpy_networks.py .
Now I'm using python 3.9 and I'm program noob. If you have some advice to fix this, it's great to get that.

Conda install deeps-here

I created a conda environment, activated it and installed pip in it; I then followed steps 1 and 2 in https://github.com/deepsphere/deepsphere-cosmo-tf2, but it returns

"
note: This error originates from a subprocess, and is likely not a
problem with pip.
ERROR: Failed building wheel for scipy
Building wheel for PyGSP (setup.py) ... done
Created wheel for PyGSP: filename=PyGSP-0.5.1-py3-none-any.whl
size=1886744
sha256=622eda0ca58242241bf95de9e2aac471594f6db650332a7def884a07cd1d967c
Stored in directory:
/private/var/folders/5l/8sxy3dbs4_sbyqdhz5g0tv2w0000gn/T/pip-ephem-wheel-cac
he-1xrlcgi6/wheels/02/45/86/e3968e70e73a8a08f3f802ca85992fede12b1a01485f22f3
29
Successfully built PyGSP
Failed to build scipy
ERROR: Could not build wheels for scipy, which is required to install
pyproject.toml-based projects "

Installing scipy by hand doesn't solve the problem.
What can I do?
The python version is 3.11.5

TPU support

Hello, I was wondering if it's possible to use this library with a TPU, or if there are some limitations that prevent one to do it.

Thank you very much for your help!

Speedup by removing loops from Chebyshev polynomail computation

The polynomial values are computed in a loop:

for k in range(2, self.K):

For commonly used degree of 5, that requires 5 consecutive steps. It may be faster to execute this using a single sparse-dense-matmul.
That would require building a sparse laplacian for each polynomial component, which can be pre-computed.
The single sparse-dense-matmul can be launched once with input of
sparse-dense-matmul( block sparse-L, tile input )
followed by a reshape and a sum.
For N=polynomial degree and M=map length and ignoring channels and batch dimensions, the matrix multiplication could look like this

[ N*M x N*M ] is the block-sparse L
[ N*M x 1 ] is the input tiled N times
[ N*M x 1 ] = sparse-dense-matmul( [ N*M x N*M ], [ N*M x 1 ] )
[ N x M x 1 ] = reshape( [ N*M x 1 ], dims=[ N, M, 1 ] )

which gives N feature maps.

Any chance that would make sense?

Also tagging @Arne-Thomsen

Installation issue

Hi,

I am trying to install deepsphere tf2. Although, it shows that the installation is completed, however, when trying to import dependencies from deepsphere, the following error occurs:

cannot import name 'SphereHealpix' from 'pygsp.graphs'

How should I solve this?

Many Thanks

ValueError: All layers added to a Sequential model should have unique names.

Hello,
I am getting an error when I am running either 'pytest tests' or the quick_start notebook. This error is: "ValueError: All layers added to a Sequential model should have unique names. Name "" is already the name of a layer in this model. Update the name argument to pass a unique name.";
which appears during the creation of the model:
model = HealpyGCNN(nside=nside, indices=indices, layers=layers, n_neighbors=20).

I am running deepsphere-cosmotf2 in tf.2.7 and python3.8.10. Anyone has ideas to solve the problem?
Thank you very much.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.