lukethehecker / esinet Goto Github PK

EEG inverse solution with artificial neural networks. This package works with MNE-Python data structures for easy integration into your MNE-based M/EEG code

License: MIT License

Python 1.62% Jupyter Notebook 98.37% Batchfile 0.01%

eeg inverse solutions neural-network inverse-solutions anns convdip esinet mne-python meg

esinet's Introduction

esinet: Electric Source Imaging using Artificial Neural Networks (ANNs)

esinet let's you solve the EEG inverse problem using ANNs with the mne-python framework. It currently supports three main architectures:

ConvDip

A convolutional neural network as described in our first paper.

Fully-connected network

A fully-connected neural network which is trained on single time instances of M/EEG data. This model was described in our second paper alongside the LSTM.

Long-short term memory network

A Long-short-term memory (LSTM) model which is trained on sequences of EEG data. This model is described in detail and published in our second paper.

Neural network design was created here

Dependencies:

Python
mne
- Follow the installation guide
Tensorflow
- Follow the installation guide
Colorednoise
joblib
pyvista>=0.24
pyvistaqt>=0.2.0
tqdm
dill
sklearn

Installation from PyPi

Use pip to install esinet and all its dependencies from PyPi:

pip install esinet

Quick start

The following code demonstrates how to use this package:

from esinet import Simulation, Net
from esinet.forward import create_forward_model, get_info

# Create generic Forward Model
info = get_info()
fwd = create_forward_model(info=info, sampling='ico2')

# Simulate M/EEG data
settings = dict(duration_of_trial=0.1)
sim = Simulation(fwd, info, settings=settings)
sim.simulate(n_samples=200)

# Train neural network (LSTM) on the simulated data
net = Net(fwd)
net.fit(sim)

# Plot
stc = net.predict(sim)[0]
sim.source_data[0].plot()
stc.plot()

First steps

Check out one of the tutorials to learn how to use the package:

Tutorial 1: The fastest way to get started with esinet. This tutorial can be used as an entry point. If you want to dig deeper you should have a look at the next tutorials, too!
Tutorial 2: Use esinet with low-level functions that allow for more control over your parameters with respect to simulations and training of the neural network.
Tutorial 3: A demonstration of simulation parameters and how they affect the model performance.
opm_source: Source imaging of optically pumped magnetometer (OPM) data. The tutorial is based on the one provided by mne

Feedback

Please leave your feedback and bug reports at [email protected].

Literature

Please cite these publications if you are using this code:

Hecker, L., Rupprecht, R., van Elst, L. T., & Kornmeier, J. (2022). Long-Short Term Memory Networks for Electric Source Imaging with Distributed Dipole Models. bioRxiv.

Hecker, L., Rupprecht, R., Tebartz van Elst, L., & Kornmeier, J. (2021). ConvDip: A convolutional neural network for better EEG Source Imaging. Frontiers in Neuroscience, 15, 533.

Troubleshooting

Having problems with the installation? Check the package requirements

esinet's People

Contributors

Stargazers

Watchers

Forkers

huijun-sun sjbt007 neuro-platypus aysebetul nancy19990414 dengzuxuan moiraw valiantmind

esinet's Issues

The probelm about simulation and MEG mapping

Hi Luke,
Sourry to distrub you. Recently I face some warning about the dataset I created from T1.gmz by freesurfer when I change the signal to eeg/meg map.

The probelms I faced with are about the simulation and the function :

elec_pos = _find_topomap_coords(self.info, self.info.ch_names,ignore_overlap=True) in 625 of net.py of esinet.

The probelms are:

and

What's more I face another probelm about the channel type when I simulate data. The problem is

There are two type of my meg channel but I don't know how to drop them. So I comment these code out. But I worry about it will have bad effect on the simulated data.

The workflow of creating fwd.fif.
I firstly use the Freesurfer and
mne.bem.make_watershed_bem(subject=subject, subjects_dir=subjects_dir,
overwrite=True, verbose=True, show=False)
make_scalp_surfaces(subject,subjects_dir=subjects_dir,overwrite=True)
to creat bem layer from T1.mgz of subject and follow the tutorial of MNE to coregistate the bem forward model to get fwd.fif

The link is https://mne.tools/stable/auto_tutorials/forward/25_automated_coreg.html

To sum up, I face four problem. One is about the two channel types when I simulate data, others are about the elec_pos = _find_topomap_coords warnings

I dont know how to deal with them and if they have great negtivate effects on the result we get.
Looking for your relpy. Thanks.

pandas no longer supports .append() for DataFrames (replace with concat)

I ran into an error with pandas on line 392 of simulation.py within the simulate_source function. I see that at one time there was an intention to either use append or concat, but now that append is deprecated concat is necessary. The line should be changed from:

self.simulation_info = self.simulation_info.append(d, ignore_index=True)

self.simulation_info = pd.concat([self.simulation_info, pd.DataFrame([d])], ignore_index=True)

Best,
JKC

forward model issues

If I have multiple subjects, each subject has a fwd, how do I train the model? Train a model for each subject?

AttributeError: 'Info' object has no attribute 'pick_channels'

Hello, when I was running "qucik_start", I encountered the following error: `AttributeError Traceback (most recent call last)
Cell In[1], line 11
9 # Simulate M/EEG data
10 settings = dict(duration_of_trial=0.1)
---> 11 sim = Simulation(fwd, info, settings=settings)
12 sim.simulate(n_samples=1000)
14 # Train neural network (LSTM) on the simulated data

File ~.conda\envs\EsiNet\lib\site-packages\esinet\simulation.py:84, in Simulation.init(self, fwd, info, settings, n_jobs, parallel, verbose)
82 self.fwd = deepcopy(fwd)
83 self.fwd.pick_channels(info['ch_names'])
---> 84 self.check_info(deepcopy(info))
86 self.check_settings()
87 self.settings['sample_frequency'] = info['sfreq']

File ~.conda\envs\EsiNet\lib\site-packages\esinet\simulation.py:108, in Simulation.check_info(self, info)
107 def check_info(self, info):
--> 108 self.info = info.pick_channels(self.fwd.ch_names, ordered=True)

AttributeError: 'Info' object has no attribute 'pick_channels'`
I wonder if this is because of a problem with the version of a certain library or because some parameters are not set in the code. I would appreciate it if you could give me some guidance!

Incorrect File Name while creating forward model

There's an error in create_forward_model when both tutorials are executed.
---> 95 source = mne.read_source_spaces(savepath+"/"+sampling+"-src.fif", verbose=verbose)
should be ---> 95 source = mne.read_source_spaces(savepath+"\"+sampling+"-src.fif", verbose=verbose)

Overlapping position probelm

Hi, I faced wth a problem when I used ConvDip in tutorial 2 on sample data. Could you give me some advice or tips about how to fix it?
And, I am very interested in your research and paper. I think it meanful and creative. I like it and want to research it more deeply. But I didn't find the dataset in this repository. Could you release the code about how to simulate the source dataset and target dataset in your paper ConvDip and the test code about ConvDip, cMEM, eLoreta and LCMV? I would be very grateful if you could.

tutorial 2

ValueError: Sequential model 'FC-model' has already been configured to use input shape (None, None, 203). You cannot build it with input_shape (None, 203)

Interpolate step

How many epoches before the LSTM model overfit?

I have just copy the quick_start file to a python file, and run it with 1000 samples and 150 epoches, with the CosineSimilarity loss and 'mse'/'mae' evaluations. However, I got an increase of MSE and MAE both for training and evaluation set after only 7 epoches.
I wondor if it is normal, or I have set some hyperparameters wrong ?