Comments (15)
christianbrodbeck
commented on September 23, 2024
Hi @rudradeep4 , this looks like a problem on Windows. Note how all p-values are exactly 0, that might indicate that the permutation distribution has not actually been collected. I wonder if this is a problem with multiprocessing – problems with multiprocessing on Windows have come up in different contexts recently, something about the underlying architecture must have changed. Is this a test that is fast enough for you to repeat it with multiprocessing turned off? If so, could you disable multiprocessing with eelbrain.configure(n_workers=False) and then run the test again (in the same Python session) to check whether that is the source of the issue?
from eelbrain.
rudradeep4
commented on September 23, 2024
Thanks for the quick reply.
I tried eelbrain.configure(n_workers=False) , but now I get no signifiant clusters at the 0.05 level. However, it does seem to be a multiprocessing problem because with it disabled, I'm now getting the p-values instead of just getting 0s.
Clusters found in Mac (as before):
Clusters found in Windows (with multiprocessing disabled and pmin=0.1):
Interestingly, in Windows with n_workers=False, I'm getting the same cluster (id 3) as in the Mac but with double the p_value (0.06 instead of 0.03) and in the frontal instead of parietal sensors. (Not sure if that's relevant though)
I'm not sure which results to 'trust' now, to be honest.
from eelbrain.
christianbrodbeck
commented on September 23, 2024
Could you let me know the exact command you run? Just the command for the test, I'll try to replicate the issue with different data.
from eelbrain.
rudradeep4
commented on September 23, 2024
Oh I'm so sorry! I can replicate the Mac results on Windows with n_workers=False. Thank you so much. We can close this issue now, I think.
Any ideas if this is going to be a temporary issue or not?
from eelbrain.
christianbrodbeck
commented on September 23, 2024
Let's leave it open, this is a problem – there ought to be at least a warning when it happens. I don't know how widespread the issue is, need to investigate with different Python versions. Thanks for reporting!
from eelbrain.
rudradeep4
commented on September 23, 2024
Ah ok. I'm using Python 3.10.8, just for your information.
from eelbrain.
christianbrodbeck
commented on September 23, 2024
I don't seem to be able to replicate this with any test. Can you try with the sample code below? If that also works for you on Windows, can you send the exact test/command that failed?
ipython
Python 3.10.8 | packaged by conda-forge | (main, Nov 22 2022, 08:16:33) [MSC v.1929 64 bit (AMD64)]
Type 'copyright', 'credits' or 'license' for more information
IPython 8.7.0 -- An enhanced Interactive Python. Type '?' for help.
In [1]: from eelbrain import *
In [2]: ds = datasets.get_uts(utsnd=True)
In [4]: res = testnd.TTestRelated('utsnd', 'A', match='rm', ds=ds) Permutation test: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 10000/10000 [00:00<00:00, 27049.49 permutations/s]
In [6]: print(res.find_clusters())
id n_sensors tstart tstop duration p sig
---------------------------------------------------------
1 1 0.46 0.47 0.01 0.004 **
2 1 0.49 0.52 0.03 0.0015 **
3 1 0.53 0.54 0.01 0.0007 ***
4 1 0.55 0.56 0.01 0.0003 ***
5 1 0.58 0.61 0.03 0.006 **
6 1 0.62 0.64 0.02 0.0031 **
7 1 0.15 0.16 0.01 0.0386 *
8 3 0.19 0.37 0.18 0 ***
9 1 0.38 0.4 0.02 0.0256 *
10 1 0.41 0.42 0.01 0.0023 **
from eelbrain.
rudradeep4
commented on September 23, 2024
I tried the same code you used above, and I couldn't reproduce my error. So then I tried my code with a different dataset, and I still couldn't reproduce the error I had in the first place.
It seems to be a problem with that one dataset I'm working with. Do you have any ideas why this might happen in one dataset over another?
from eelbrain.
christianbrodbeck
commented on September 23, 2024
Can you pickle and share the dataset that causes the issue?
from eelbrain.
rudradeep4
commented on September 23, 2024
from eelbrain.
rudradeep4
commented on September 23, 2024
res = testnd.TTestRelated( 'eeg', 'condition', 'Rise', 'Fall', match='subject', ds=ds, pmin=0.05, tstart=0.100, tstop=0.600, mintime=0.1, # minsource=2 )
clusters = res.find_clusters(0.05)
print(clusters)
And this is the code I use.
from eelbrain.
christianbrodbeck
commented on September 23, 2024
Thanks! I've narrowed down the source of the issue, but it's so far mysterious to me:
With bigger datasets, RawArray, does not seem to be transferred to the child processes correctly. RawArray is used to send the data to the child processes that perform the tests on the permuted data. After converting to RawArray, the original numpy array can be recovered in the parent process using numpy.frombuffer, but calling the exact same function from the child process returns an all-zero array. So the dependent variable never gets transferred to the child processes.
from eelbrain.
christianbrodbeck
commented on September 23, 2024
This should now be fixed in Eelbrain 0.38.3. @rudradeep4 please let me know if you have a chance to test it.
from eelbrain.
rudradeep4
commented on September 23, 2024
I tested it, and can confirm that it works as expected. Thanks!
from eelbrain.
christianbrodbeck
commented on September 23, 2024
Thank you for verifying @rudradeep4 !
from eelbrain.
Related Issues (20)
- `boosting` with `multiprocessing` on Windows HOT 6
- Loading CND format dataset with load.cnd HOT 4
- Trouble adding NDVar to brain plot HOT 3
- calling plot functions in eelbrain cause python to abort (libc++abi: terminating with uncaught exception of type NSException) HOT 3
- Adding parcellation to vol source space HOT 2
- Boosting residual defined for individual eeg channels, but documentation states otherwise. HOT 1
- trouble getting same residuals with BoostingResult and Convolve HOT 15
- How should I import my meg raw file into eelbrain's NDVar format? HOT 11
- Eelbrain not rendering Brain objects HOT 6
- Cannot install eelbrain 0.39.5 HOT 10
- Having a dead kernel each time I run eelbrain on my data HOT 7
- Does eelbrain works with Mac M2 chip HOT 1
- Under-utilization of CPU on M1 Ultra during boosting HOT 3
- Eelbrain doesn't lauch after update HOT 5
- MNE Deprecations HOT 1
- The math under TTestRelated HOT 1
- plot.brain.brain(roi) crashes immediately HOT 2
- Missing dependencies keyring, wxPython, colormath, pymatreader HOT 2
- Improve documentation and/or GitHub readme HOT 5
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from eelbrain.