Check https://github.com/warner/python-versioneer/blob/master/INSTALL.md

e.g. by specifying the path to FS parcellation of the atlas!

I install graynet and hiwenet using pip install.
when I tried to import graynet, it gives the following error:

import graynet
/Users/zyc/Software/anaconda3/lib/python3.6/site-packages/h5py/init.py:34: FutureWarning: Conversion of the second argument of issubdtype from float to np.floating is deprecated. In future, it will be treated as np.float64 == np.dtype(float).type.
from ._conv import register_converters as _register_converters
Traceback (most recent call last):
File "", line 1, in
File "/Users/zyc/Software/anaconda3/lib/python3.6/site-packages/graynet/init.py", line 13, in
from graynet.vis_network import draw3Dnx
File "/Users/zyc/Software/anaconda3/lib/python3.6/site-packages/graynet/vis_network.py", line 4, in
import plotly
ModuleNotFoundError: No module named 'plotly'

Dear Pradeep and colleagues,

I kept running into errors when trying to use the Glasser2016 atlas and wondered what I might be doing wrong. I can get graynet to work with the fsaverage atlas, but not Glasser. For fsaverage, I (obviously) qcached everything from Freesurfer, but reading the graynet docs, it looked like I didn't have to do that for Glasser2016 (or am I incorrect about that?).

Here's the command I'm running--
graynet -s subject_ids_reduced.txt -f freesurfer_thickness -i /home/jamielh/Volumes/Hanson/Collaborative_MRI/CEDAR_MRI/Neuroimaging/BIDS/derivatives/FreeSurfer_6 -w manhattan -p 10 -o /home/jamielh/Volumes/Hanson/Collaborative_MRI/CEDAR_MRI/Neuroimaging/BIDS/derivatives/graynet_2020_11 -a glasser2016

And then here's my (error) output--

/usr/local/lib/python3.6/dist-packages/dicom/__init__.py:53: UserWarning: This code is using an older version of pydicom, which is no longer maintained as of Jan 2017. You can access the new pydicom features and API by installing pydicom` from PyPI.
See 'Transitioning to pydicom 1.x' section at pydicom.readthedocs.org
for more information.

warnings.warn(msg)

Data resampled to glasser2016 atlas, smoothed at 10 with node size None
Computing single edge ...

Processing freesurfer_thickness features
Traceback (most recent call last):
File "/home/jamielh/.local/lib/python3.6/site-packages/nibabel/loadsave.py", line 42, in load
stat_result = os.stat(filename)
FileNotFoundError: [Errno 2] No such file or directory: '/home/jamielh/Volumes/Hanson/Collaborative_MRI/CEDAR_MRI/Neuroimaging/BIDS/derivatives/FreeSurfer_6/sub-0202/surf/lh.thickness.fwhm10.glasser2016.mgh'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/graynet/freesurfer.py", line 42, in import_features
features[subject_id] = get_data(fs_dir, subject_id, base_feature, fwhm, atlas)
File "/usr/local/lib/python3.6/dist-packages/graynet/freesurfer.py", line 60, in get_data
feature=bare_name_feature, atlas=atlas, fwhm=fwhm))
File "/usr/local/lib/python3.6/dist-packages/graynet/freesurfer.py", line 101, in __read_morph_feature
vec = nibabel.load(thk_path).get_fdata() # typically of shape: (163842, 1, 1)
File "/home/jamielh/.local/lib/python3.6/site-packages/nibabel/loadsave.py", line 44, in load
raise FileNotFoundError(f"No such file or no access: '{filename}'")
FileNotFoundError: No such file or no access: '/home/jamielh/Volumes/Hanson/Collaborative_MRI/CEDAR_MRI/Neuroimaging/BIDS/derivatives/FreeSurfer_6/sub-0202/surf/lh.thickness.fwhm10.glasser2016.mgh'
Traceback (most recent call last):
File "/home/jamielh/.local/lib/python3.6/site-packages/nibabel/loadsave.py", line 42, in load
stat_result = os.stat(filename)
FileNotFoundError: [Errno 2] No such file or directory: '/home/jamielh/Volumes/Hanson/Collaborative_MRI/CEDAR_MRI/Neuroimaging/BIDS/derivatives/FreeSurfer_6/sub-0202/surf/lh.thickness.fwhm10.glasser2016.mgh'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/graynet/freesurfer.py", line 42, in import_features
features[subject_id] = get_data(fs_dir, subject_id, base_feature, fwhm, atlas)
File "/usr/local/lib/python3.6/dist-packages/graynet/freesurfer.py", line 60, in get_data
feature=bare_name_feature, atlas=atlas, fwhm=fwhm))
File "/usr/local/lib/python3.6/dist-packages/graynet/freesurfer.py", line 101, in __read_morph_feature
vec = nibabel.load(thk_path).get_fdata() # typically of shape: (163842, 1, 1)
File "/home/jamielh/.local/lib/python3.6/site-packages/nibabel/loadsave.py", line 44, in load
raise FileNotFoundError(f"No such file or no access: '{filename}'")
FileNotFoundError: No such file or no access: '/home/jamielh/Volumes/Hanson/Collaborative_MRI/CEDAR_MRI/Neuroimaging/BIDS/derivatives/FreeSurfer_6/sub-0202/surf/lh.thickness.fwhm10.glasser2016.mgh'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/graynet/run_workflow.py", line 289, in extract_per_subject_cortical
atlas=atlas_spec)
File "/usr/local/lib/python3.6/dist-packages/graynet/utils.py", line 560, in import_features
fwhm=fwhm, atlas=atlas)
File "/usr/local/lib/python3.6/dist-packages/graynet/freesurfer.py", line 47, in import_features
''.format(base_feature, subject_id))
ValueError: freesurfer_thickness data for sub-0202 could not be read!
/usr/local/lib/python3.6/dist-packages/graynet/run_workflow.py:293: UserWarning:

Unable to read freesurfer_thickness features for sub-0202
Skipping it.

Reading freesurfer_thickness for sub-0202 ...
graynet computation done.`

I'm only trying to do it for one subject, but figured the critical bit above was:
FileNotFoundError: [Errno 2] No such file or directory: '/home/jamielh/Volumes/Hanson/Collaborative_MRI/CEDAR_MRI/Neuroimaging/BIDS/derivatives/FreeSurfer_6/sub-0202/surf/lh.thickness.fwhm10.glasser2016.mgh'

Should I be qcache-ing my surfaces (with the files Kate Mills made) before running graynet? Any suggestions are much appreciated.

Many thanks (and apologies for my issues),
Jamie.

In readme and on Usage pages

Dear devs,

I tried to run the volumetric graynet module and obtained this error

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/usuario/environment_softwares/graynet/lib/python3.10/site-packages/hiwenet/pairwise_dist.py", line 373, in pairwise_extract
edge_value = weight_func(hist_one, hist_two)
File "/home/usuario/environment_softwares/graynet/lib/python3.10/site-packages/medpy/metric/histogram.py", line 125, in manhattan
h1, h2 = __prepare_histogram(h1, h2)
File "/home/usuario/environment_softwares/graynet/lib/python3.10/site-packages/medpy/metric/histogram.py", line 1248, in __prepare_histogram
h1 = h1 if scipy.ndarray == type(h1) else scipy.asarray(h1)
File "/home/usuario/environment_softwares/graynet/lib/python3.10/site-packages/scipy/init.py", line 139, in getattr
raise AttributeError(
AttributeError: Module 'scipy' has no attribute 'ndarray'
WARNING:root:Unable to compute edge weight between 105.0 and 114.0. Skipping it.
Traceback (most recent call last):
File "/home/usuario/environment_softwares/graynet/lib/python3.10/site-packages/scipy/init.py", line 137, in getattr
return globals()[name]
KeyError: 'ndarray'

Please, could you make a suggestion to solve this issue? Could be something associated with a conflict between scipy and numpy?

thank you very much for your assistance!

Hello,

I am interested in using your toolkit for gray matter volumetric connectivity using a custom ROI.

I have just downloaded the package and tried it on the example data provided, using the example code provided, however it returns an error. Is the current build functional for volumetric analyses? Output as follows below.

Thanks for your advice!
James

user:~/Desktop/graynet-master/example_data$ graynet -s volumetric_CAT12/sub_id_list.txt -f spm_cat_gmdensity -i volumetric_CAT12/ -w manhattan -a cat_aal -o volumetric_CAT12/graynet_volumetric/

Data resampled to cat_aal atlas, smoothed at 10 with node size None
Computing single edge ...

Processing spm_cat_gmdensity features
Reading spm_cat_gmdensity for CAM_0002_01 ... Done.

Processing id CAM_0002_01 -- weight manhattan (1/1) :Reading spm_cat_gmdensity for UBC_0018_01 ... Done.

Processing id UBC_0018_01 -- weight manhattan (1/1) :........................
Unable to extract manhattan features for UBC_0018_01
Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/graynet/volumetric.py", line 107, in extract_per_subject_volumetric
graph.node[roi]['x'] = float(centroids[roi][0])
AttributeError: 'Graph' object has no attribute 'node'
........................
Unable to extract manhattan features for CAM_0002_01
Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/graynet/volumetric.py", line 107, in extract_per_subject_volumetric
graph.node[roi]['x'] = float(centroids[roi][0])
AttributeError: 'Graph' object has no attribute 'node'
Done.Reading spm_cat_gmdensity for TGH_0070_01 ... Done.

Processing id TGH_0070_01 -- weight manhattan (1/1) :........................
Unable to extract manhattan features for TGH_0070_01
Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/graynet/volumetric.py", line 107, in extract_per_subject_volumetric
graph.node[roi]['x'] = float(centroids[roi][0])
AttributeError: 'Graph' object has no attribute 'node'
Done.
graynet computation done.

Hello,

Thank you for providing a wonderful toolbox.
I am now currently using volumentric graynet and I find the options to calculate distance is lacking the 'sorensen distance'
the distance is introduced in the article (https://onlinelibrary.wiley.com/doi/pdfdirect/10.1002/hbm.25820)
and calculated as

where p and q denote two probability distributions. dsor is in the range of [0 1].
I found a python code here : https://en.wikibooks.org/wiki/Algorithm_Implementation/Strings/Dice%27s_coefficient#Python

Could add this option in the command line?

Another thing I want to ask is could you provide references of calculating the distatnce?
For example any article that use the manhattan distance to calculate structural covariance matrix.
I guess you could have followed previous article when you developed the toolbox,,

Best wishes,
Minchul

Dear Pradeep,
I'm encountering an issue while processing a volumetric stream using graynet after CAT12.
The specific problem arises when trying to run the following command:

graynet -s subjects_ids.txt -f spm_cat_gmdensity -i ./CAT -w manhattan -a cat_lpba40 -o ./graynet -c 6

I have a list of patients correctly processed via CAT12 for cobra, lpba40, neuromorphometrics, suit, thalamic_nuclei, and thalamus atlases

I receive the following error message:

ValueError: Subject features and membership (group labels) differ in length or shape!
This may happen if volumes for atlas and subject differ in size/dimensions. Ensure all the subjects have voxel-wise correspondence with the atlas

Upon investigation, I noticed that my mwp1$subj.nii files have dimensions 113x137x113 and voxel size 1.500x1.500x1.500, which do not match the dimensions 121x145x121 of the cat_lpba40 atlas.

Version:
graynet 0.5.6
CAT12.8.2 (r2159) from 2022-12-07

Is there a way to correct this discrepancy in voxel-wise correspondence?

Thank you for your time.

Dear Pradeep and crew,

Apologies for the multiple notes in a day, but I wondered if you/we know how modulation might impact network construction/measures with volumetric graynet?

I know there was some work previously (citation, which I think PR sent me?) suggesting optimal settings for VBM (and it suggested to skip modulation). I wasn't sure what was known (about these effects) in regards to networks?

Any thoughts are deeply appreciated!

Cheers,
Jamie

Hello!

I am trying to run graynet on one of our freesurfer output (augumented by running recon-all -sd testdata -subjid output -qcache). Currently I am seeing the following error message.

+ singularity exec -e docker://brainlife/graynet:0.4.8 graynet -i testdata -o ./output -s subjects.txt -f freesurfer_thickness -w manhattan
Docker image path: index.docker.io/brainlife/graynet:0.4.8
Cache folder set to /mnt/scratch/hayashis/singularity/cache/docker
Creating container runtime...
Exploding layer: sha256:dc65f448a2e2f2ea557e69ed9ac65aa8ac0e16f1bce68f90de910b4d5a2f1ba1.tar.gz
Exploding layer: 
sha256:346ffb2b67d7b35729673ced818325ed0ea57284e69de67f8bbc48c2bf294716.tar.gz
Exploding layer: sha256:dea4ecac934f71d68d4f5edb171f6cff42588edfa3f70ba8709be56e321eeddc.tar.gz
Exploding layer: sha256:8ac92ddf84b35dac36ef6632f8d5a0c9c2d7038f6018f2d4fa1be056d90bd366.tar.gz
Exploding layer: sha256:a3ca60abc08a00e84d4fb6bd20998f38c2e171f6e085e5e7c26908be7223ea7c.tar.gz
Exploding layer: sha256:9253bd2ee3f6385e17fc1f73d5c39c7b5d7dcfeb37199920739468a633fdb2c0.tar.gz
Exploding layer: sha256:fad96c8dce4476346794f1159a3654ace3bd64fb18f15acdb7f55c48401cf530.tar.gz
Exploding layer: sha256:ec0f51d2752dda2cdf64ee7a29b560b850ca4d10d76f5490f71a262c949ff1ad.tar.gz
Exploding layer: sha256:1fa0065c6287419b4915768e1e34d270c1fe0dac95852a8e37a00463986c8177.tar.gz
Exploding layer: sha256:40139f995ea3f84c2091a623330a44d1831653201f842b86fac2cb3a47276866.tar.gz
Exploding layer: sha256:999f12ec77d12bea202dcae01de60cc8abfa47149dd618f3571ad833c557d79d.tar.gz

Data resampled to fsaverage atlas,  smoothed at 10 with node size None
Computing single edge ... 

Processing freesurfer_thickness features
Reading freesurfer_thickness for output ...  Done.

Processing id output -- weight manhattan (1/1) :Unable to extract manhattan features for output
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/numpy/core/function_base.py", line 117, in linspace
    num = operator.index(num)
TypeError: 'numpy.float64' object cannot be interpreted as an integer

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/graynet/run_workflow.py", line 319, in extract_per_subject_cortical
    graph = hiwenet.extract(data,
  File "/usr/local/lib/python3.8/site-packages/hiwenet/pairwise_dist.py", line 293, in extract
    edges = compute_bin_edges(features, num_bins, edge_range,
  File "/usr/local/lib/python3.8/site-packages/hiwenet/pairwise_dist.py", line 85, in compute_bin_edges
    edges = np.linspace(edges_of_edges[0], edges_of_edges[1], num=num_bins, endpoint=True)
  File "<__array_function__ internals>", line 5, in linspace
  File "/usr/local/lib/python3.8/site-packages/numpy/core/function_base.py", line 119, in linspace
    raise TypeError(
TypeError: object of type <class 'numpy.float64'> cannot be safely interpreted as an integer.
Done.
graynet computation done.

real	0m24.432s
user	0m15.266s
sys	0m6.415s

Could you help me understand what this error message means?

document draw3dnx, and improve the demo further

Consider supporting the following input formats
Ants
Cat
Fslvbm

I installed graynet using pip install -U graynet without any error shows.

But when I type graynet, it shows ModuleNotFoundError: No module named 'hiwenet'.

After pip install hiwenet, they another ModuleNotFoundError: No module named "plotly"

After pip install plotly, now finally graynet seems work :).

as no smoothing is involved as well as node sizes, we must remove the smoothing{}_size{} from the output file names

to ensure histogram computation is commensurable across all subjects (who may potentially different ranges even for the same feature), we need to use a fixed edge range for all subjects. Make a clear note, in docs everywhere, on computing this edge range from the full dataset

to avoid running into bad or empty ROIs, with questionable data, and resulting in NaN for edge weights etc

Prev title: As I mentioned in my email, I see no small world properties for cortical thickness network. I notice some nan in the csv file. I found one subject showing no nan in csv file. I still cannot see small-world properties.

It would be useful for users to offer a toolbox-independent format specify volumetric voxelwise probabilistic images. Two possible options are

1. folder of folders with a fixed file name (prob.nii?) for image such as
   --input_dir /project/vbm --id_list idlist.txt where /project/vbm would contain a bunch of folders each referring to one of the subject IDs in the id_list which must contain prob.nii
2. a single CSV file with two columns, the first being the subject ID and the second being full path to the corresponding image (no relation between ID and path enforced):

id1,/project/vbm/id1/somename.nii
random2,/project/vbm/notnecessarily_random2/someothername.nii
arbitrary3,/project/vbm/whateveruserchooses/filename.nii

Choose one and implement it.

Right now there are no checks to ensure the atlas matches in dimensions and resolution with input images, other than they both exist. This check is important to avoid cryptic fails for the user

Dear Pradeep and colleagues,

I was slowly getting back into graynet, but ran into some issues processing data. Right now, I'm running this command--

graynet -s subject_ids.txt -f freesurfer_thickness -i /home/jamielh/Volumes/Hanson/Collaborative_MRI/CEDAR_MRI/Neuroimaging/BIDS/derivatives/FreeSurfer_6 -w manhattan -p 10 -o /home/jamielh/Volumes/Hanson/Collaborative_MRI/CEDAR_MRI/Neuroimaging/BIDS/derivatives/graynet_2020_11

And get this output/error--

/usr/local/lib/python3.6/dist-packages/dicom/__init__.py:53: UserWarning: This code is using an older version of pydicom, which is no longer maintained as of Jan 2017. You can access the new pydicom features and API by installing pydicom` from PyPI.
See 'Transitioning to pydicom 1.x' section at pydicom.readthedocs.org
for more information.

warnings.warn(msg)
/usr/local/lib/python3.6/dist-packages/h5py/init.py:36: FutureWarning: Conversion of the second argument of issubdtype from float to np.floating is deprecated. In future, it will be treated as np.float64 == np.dtype(float).type.
from ._conv import register_converters as _register_converters

Data resampled to fsaverage atlas, smoothed at 10 with node size None
Computing single edge ...

Processing freesurfer_thickness features
Reading freesurfer_thickness for sub-0202 ... Done.

Processing id sub-0202 -- weight manhattan (1/1) :.............
Unable to extract manhattan features for sub-0202
Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/graynet/run_workflow.py", line 332, in extract_per_subject_cortical
graph.nodes[roi]['x'] = float(centroids[roi][0])
TypeError: 'method' object is not subscriptable
Done.
graynet computation done.`

I'm using Python 3.7.1 on Ubuntu 16.04.7. I checked the Freesurfer inputs seem reasonable. Thoughts on what I might be doing wrong? Previously, I had some numpy errors/issues, but I thought those were fixed, etc. Any suggestions are much appreciated!

Thanks much,
Jamie.

if qcache is not already run by the user for the atlas they chose, run it locally.

Dear Pradeep and crew,

I was trying to run volumetric graynet, but kept running into issues. Python 3.6. Ubuntu linux server, newest version of graynet, and I pasted my error output below.

graynet -s subjects.txt -f spm_cat_gmdensity -i /home/jamielh/Volumes/Hanson/Neuroimaging_Resources/Regions_And_Atlases/Brainnetome/graynet_test -w manhattan -a /home/jamielh/Volumes/Hanson/Neuroimaging_Resources/Regions_And_Atlases/Brainnetome/BN_Atlas_274_combined_1.5.nii -o /home/jamielh/Volumes/Hanson/Neuroimaging_Resources/Regions_And_Atlases/Brainnetome/graynet_test /usr/local/lib/python3.6/dist-packages/dicom/__init__.py:53: UserWarning: This code is using an older version of pydicom, which is no longer maintained as of Jan 2017. You can access the new pydicom features and API by installing pydicom` from PyPI.
See 'Transitioning to pydicom 1.x' section at pydicom.readthedocs.org
for more information.

warnings.warn(msg)
/usr/local/lib/python3.6/dist-packages/h5py/init.py:36: FutureWarning: Conversion of the second argument of issubdtype from float to np.floating is deprecated. In future, it will be treated as np.float64 == np.dtype(float).type.
from ._conv import register_converters as _register_converters
Traceback (most recent call last):
File "/usr/local/bin/graynet", line 6, in
from graynet.main import main
File "/usr/local/lib/python3.6/dist-packages/graynet/init.py", line 13, in
from graynet.vis_network import draw3Dnx
File "/usr/local/lib/python3.6/dist-packages/graynet/vis_network.py", line 4, in
import plotly
File "/usr/local/lib/python3.6/dist-packages/plotly/init.py", line 31, in
from plotly import (plotly, graph_objs, grid_objs, tools, utils, session,
File "/usr/local/lib/python3.6/dist-packages/plotly/plotly/init.py", line 10, in
from . plotly import (
File "/usr/local/lib/python3.6/dist-packages/plotly/plotly/plotly.py", line 31, in
from plotly import exceptions, tools, utils, version, files
File "/usr/local/lib/python3.6/dist-packages/plotly/tools.py", line 20, in
from plotly import graph_reference
File "/usr/local/lib/python3.6/dist-packages/plotly/graph_reference.py", line 518, in
GRAPH_REFERENCE = get_graph_reference()
File "/usr/local/lib/python3.6/dist-packages/plotly/graph_reference.py", line 86, in get_graph_reference
sha1 = hashlib.sha1(six.b(str(graph_reference))).hexdigest()
File "/usr/local/lib/python3.6/dist-packages/six.py", line 626, in b
return s.encode("latin-1")
UnicodeEncodeError: 'latin-1' codec can't encode character '\u25b2' in position 1365250: ordinal not in range(256)`

I posted relevant files here-- https://www.dropbox.com/sh/8vyn86ryayn9qnh/AAAtbrbk0igd8TFuHDSyVnaJa?dl=0
I wasn't sure what I might be doing wrong. Any thoughts? (Maybe I have some extra characters in my subjects input file? But I tried to rework in 3 different text editors to no avail).

Best,
Jamie.