tknapen / nsd_access Goto Github PK
View Code? Open in Web Editor NEWpython package to access the data of the NSD (natural scenes dataset) fMRI project
License: MIT License
python package to access the data of the NSD (natural scenes dataset) fMRI project
License: MIT License
For the experiment, COCO images were cropped to be square. This means that the spatial parameters returned by the COCO interface are off, and need to be re-mapped to correct for the cropping.
Line 220 in ffa837a
@tknapen we have a bug with the mgz.ctab files not being read by pd.read_csv at the line referenced above. This works well for the nsdgeneral, visualsulc, or HCP_MMP1 atlases, but fails for Kastner2015 and thalmus.
Here is the full error trace (atlas=Kastner2015, sub=subj01, data_format='func1pt8mm'):
---------------------------------------------------------------------------
ParserError Traceback (most recent call last)
<ipython-input-30-9b391304df40> in <module>
2 mmp1, atlas_mapping = nsda.read_atlas_results(subject=sub,
3 atlas=atlas,
----> 4 data_format='func1pt8mm')
~/env/lib/python3.6/site-packages/nsd_access/nsda.py in read_atlas_results(self, subject, atlas, data_format)
219
220 mapp_df = pd.read_csv(os.path.join(self.nsddata_folder, 'freesurfer', 'fsaverage',
--> 221 'label', f'{atlas_name}.mgz.ctab'), delimiter=' ', header=None, index_col=0)
222 atlas_mapping = mapp_df.to_dict()[1]
223 # dict((y,x) for x,y in atlas_mapping.iteritems())
~/env/lib/python3.6/site-packages/pandas/io/parsers.py in parser_f(filepath_or_buffer, sep, delimiter, header, names, index_col, usecols, squeeze, prefix, mangle_dupe_cols, dtype, engine, converters, true_values, false_values, skipinitialspace, skiprows, skipfooter, nrows, na_values, keep_default_na, na_filter, verbose, skip_blank_lines, parse_dates, infer_datetime_format, keep_date_col, date_parser, dayfirst, cache_dates, iterator, chunksize, compression, thousands, decimal, lineterminator, quotechar, quoting, doublequote, escapechar, comment, encoding, dialect, error_bad_lines, warn_bad_lines, delim_whitespace, low_memory, memory_map, float_precision)
683 )
684
--> 685 return _read(filepath_or_buffer, kwds)
686
687 parser_f.__name__ = name
~/env/lib/python3.6/site-packages/pandas/io/parsers.py in _read(filepath_or_buffer, kwds)
461
462 try:
--> 463 data = parser.read(nrows)
464 finally:
465 parser.close()
~/env/lib/python3.6/site-packages/pandas/io/parsers.py in read(self, nrows)
1152 def read(self, nrows=None):
1153 nrows = _validate_integer("nrows", nrows)
-> 1154 ret = self._engine.read(nrows)
1155
1156 # May alter columns / col_dict
~/env/lib/python3.6/site-packages/pandas/io/parsers.py in read(self, nrows)
2057 def read(self, nrows=None):
2058 try:
-> 2059 data = self._reader.read(nrows)
2060 except StopIteration:
2061 if self._first_chunk:
pandas/_libs/parsers.pyx in pandas._libs.parsers.TextReader.read()
pandas/_libs/parsers.pyx in pandas._libs.parsers.TextReader._read_low_memory()
pandas/_libs/parsers.pyx in pandas._libs.parsers.TextReader._read_rows()
pandas/_libs/parsers.pyx in pandas._libs.parsers.TextReader._tokenize_rows()
pandas/_libs/parsers.pyx in pandas._libs.parsers.raise_parser_error()
ParserError: Error tokenizing data. C error: Expected 2 fields in line 2, saw 6
ImportError: cannot import name 'json_normalize' from 'pandas.io.json' (D:\anaconda3\envs\SD\Lib\site-packages\pandas\io\json_init_.py)
I use "pandas 2.1.1" in my project, and when I tried to from nsd_access import NSDAccess
, I found that in the latest version, pandas used "pandas.json_normalize" instead of "pandas.io.json.json_normalize"
To fix this issue, replace from pandas.io.json import json_normalize
with from pandas import json_normalize
in nsda.py
Line 333 in d0937b8
when fetching the captions for a long list of indices, you end up having to loop through image_indices before calling read_image_coco_info for that current image in the loop, ending up in many i/o to load the captions from memory at each loop iteration.
could one pass a list of indices (or a single image_index) and return a list of captions (or a single caption)?
Hi Tomas,
I just realised that there is a mismatch between the filenames in
os.path.join(nsd_dir, 'Stimuli', 'shared1000', '*.png')
and the image brick.
so when i want to read the stimulus : shared0953_nsd69215.png if I follow the logic explained in the nds_dataset.doc, 0953 is the index in shared1000 space, and 69215 is the index in 73KIDs space.
However, if I want to read and show that same image from the brick, using nsda.read_images() i need to query for 69214.
were you aware of this? I'm wondering then if the same thing happens with the captions? i.e. if I want to return the caption for shared0953_nsd69215.png, should I query for 69214?
ian
Need to add a valid setup.py installation route.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.