Adding a pre-commit hook that is able to ensure that the Markdown files have a consistent format and follow the carpentries style would be a great addition and would allow developers to concentrate on the contents rather than the formatting.

Best would be to use a carpentries-wide adopted/recommended tool and the same configuration file.

Add a pre-commit hook to clean imports (sorting and detecting unused imports) and improving the style across notebooks (i.e. removing trailing whitespaces, line length, single vs. double quotes in strings, etc.).

Related to #147.

Reuse the out_dir and subj variables to create output directions through different cells using the command line in the preprocessing episode. This would reducing the risk of using the wrong paths (e.g. PRs #176, #177, #180).

The maintenance burden to keep all contents synchronized across episode Markdown files, Jupyter notebooks and the solution Jupyter notebooks is considerable. Whenever a change is made to an episode, inadvertently leaving behind one of the three parts is quite likely.

It may be worthwhile to explore how feasible is to use Jupytext to automatically (i.e. using the command line interface) convert the notebooks into Markdown, and whether the layout/formatting looks as nice as when writing natively in Markdown. But that might come with some other challenges and might require some work:

• The Jekyll commands (e.g. layout, includes, etc.) might need to be added manually to the generated files, or find some way to nicely (automatically) add what is needed.
• The Carpentries Markdown style recommend line breaks at 100 characters to keep a consistent style and improve readability. Not sure if that can be automated or enforced in case the Markdown files got generated automatically.

Similar to #30 , should add one or several exercises to both probabilistic and deterministic tractography episodes.

As @kaitj noticed, the current Binder link is out of date and points to my fork of the repository. I think this must have happened while we were transferring the repository from the CONP repo to mine and then to the Carpentries incubator.

I will update the link but I also noticed that we switched from using the gh-pages branch to build the website to the master branch. I have no preference on which branch to stick with. But if we're keeping the master branch, should we get rid of the gh-pages branch to avoid any confusion?

LaTeX maths should be displayed appropriately: according to carpentries/carpentries-theme@399bf2f they should be rendered appropriately. However, this lesson's LaTeX syntax are not being rendered as expected:

• $r(s)$ in https://carpentries-incubator.github.io/SDC-BIDS-dMRI/local_tractography/index.html
• `$\theta$ in https://carpentries-incubator.github.io/SDC-BIDS-dMRI/probabilistic_tractography/index.html
• $l$, $l_{max}$, $R = (l_{max}+1)(l_{max}+2)/2$, $R$, $l_{max} = {4, 6, 8}$, and $R = {15, 28, 45}$ in https://carpentries-incubator.github.io/SDC-BIDS-dMRI/constrained_spherical_deconvolution/index.html

Related issues and PRs
carpentries/styles#296
carpentries/styles#573
carpentries/styles#592

Especially the first step would benefit from a back and forth gif over a static side by side image.

First noticed in a different repository, but looking at the logs, it doesn't look like the cache is actually being used during the Github actions. Not a breaking issue, but probably something we want to look into.

Looking to also update Notebook 3 on Diffusion Tensor Imaging (DTI). This notebook should highlight what DTI is, why its important and the pros and cons.

Main things to update;

• Update with images / processing from ds000221
• Better explain different quantitative metrics (e.g. FA, MD, AD, RD)

A separate notebook to be included for higher order models / more current models used in research.

Add images showing different diffusion weighted images to show differences in directional sensitivity

At times, we manually edit the raw Jupyter notebooks (like #137 (comment)) and break the notebooks and they effectively become unusable.

Although the CI tests in place would potentially detect such cases, this is not ideal; we would like the CI tests to spend time in executing the cells, rather than exiting because the file cannot be parsed.

Adding a pre-commit hook that is able to ensure that the files can at least be parsed could be a solution for this.

In addition to the diffusion sensitive images two more files are collected as part of a diffusion dataset these are called b-val and b-vec files.....etc

Right now it sounds like those are your imaging data somehow.

The deterministic tracking episode has currently no figures. Generating the relevant figures would allow learners to have some insight on what can be achieved through the episode, especially when reading the Markdown version.

Content seems fine, but noticed it is using the unprocessed data. Will want to update this with any updates to figures.

I was playing around with the Exercise2 extension for Jupyter. It seems really easy to set up and I think it would help us combine the solution and non-solution notebooks into a single file.

Here's a demonstration below. The solution can be revealed by clicking on a button. You can also include a cell below the solution whether the learner is meant to enter their own code. Here, you could still include fill in the blank style content and have pytest skip those cells when running the tests.

WDYT? @kaitj @jhlegarreta

Verbatim/code and type face style is not consistent across the episodes. It should be made consistent by choosing one style:

• HMTL <code> vs. Markdown backticks vs. double backticks for verbatim/code mark up.
• Star (*) vs. underscore (_) and their double counterparts for italics/bold faces

Add one or several exercises to the CSD lesson, including their solutions.

Improve how the CSD reconstruction lesson screenshots are displayed:

• Make the screenshots be displayed inline instead of launching a separate FURY window
• Add additional screenshots corresponding to the anatomical orientations.

Improve CSD lesson theory: explain better the relationship between the SH representation and the SD methods.

To do:

• Add a more complex diffusion model (CSD?) as notebook 4
• Make tractography notebook 5

Related to the file storage, i just noticed 5 of the processed subjects are missing in the eddy step. May have been something related to the OSF push. Making a note here as a reminder to update the OSF.

I'm working on an update to the Intro to Diffusion episode (1). Here's an initial render of it: http://www.nipreps.org/nipreps-book/dwi/02-intro_dmri.html

Still needs more work but what I'd really like to include is:

• add an overview of different acquisition schemes (single-shell, multi-shell, DSI)
• introduce bvec orientation, elude to how it could affect tractography
• show metadata in the .json file that would be important for pre-processing (some of this is also discussed in the Preprocessing episode but maybe we could also introduce it here)

If we move to using NeuroLibre to host this content, we will be using Repo2Data to download the pre-processed data from OSF. Repo2Data cannot use the fetch command to download specific files and instead using the clone command to download all of the files from an OSF repository.

Can we migrate the existing ds000221_sub-010006.zip file to its own OSF repository?

@kaitj @jhlegarreta

Currently this lesson is rendering with a SWC logo. @fmichonneau - could you please pull in the remote styles and update to Incubator branding?

As of pybids v0.14.0, file extensions will need to include the leading dot ('.') when using the 'extension' entity. PR #169 dismisses a FutureWarning by including:
bids.config.set_option('extension_initial_dot', True)
but this won't be necessary once pybids is upgraded.

Add a figure to the camera setup discussion. The contribution in PR #77 would benefit from a figure. Since creating it might take some time, it is left out of the PR so that the PR can be merged and the figure created and added as time permits.

Cross-referencing #69 (comment) and #77 (comment).

The figure can be added just after the pitch, roll and yaw are introduced with a text like:

The following figure illustrates these concepts and how they are related to the
anatomical orientations.

![3D scene camera viewpoint concepts and anatomical views]({{ base }}/fig/discuss/camera_viewpoint_concepts_anatomy.png){:class="img-responsive"}

Good news is that the previously set up Github Actions now work. Bad news is it is failing when trying to install pkg-resources==0.0.0 from the requirements.txt.

Something to look into and fix to have valid tests!

Inline images are not being displayed inline, e.g.:
https://carpentries-incubator.github.io/SDC-BIDS-dMRI/diffusion_tensor_imaging/index.html

It looks like the the class is not yet available through the remote theme repository:
https://github.com/carpentries/carpentries-theme

Cross-referencing #157 (comment)

Can probably be done at same time or before #2.

Make note the image was converted to int from float.

Perform preprocessing for new dataset ds000221. Previous preprocessing for ds000030 made use of existing software and bash scripting prior to using dipy for processing. Would like to show some common preprocessing that is done (may exclude some steps that may be more specific to certain datasets / groups). Should also talk about existing software that helps with these steps.

To do:

• Brainmasking
• Distortion correction (e.g. TOPUP)
• Eddy correction
• Registration with T1w
• Convert to carpentries format

Things to consider:

• Denoising
• Unringing
• Gradient nonlinearity (would not be done, but could point to resources)

Happy to hear some thoughts about what should be included or if anything is missing with the preprocessing.

As noted here, the probabilistic tractography episode callout towards the end is not rendered nicely, so a fix would be welcome.

Use the scene.reset_camera() call in the fury camera setting methods when the issue no. 38 in fury gets fixed.

Cross-referencing #121 (comment).

Use the {{ relative_root_path }} placeholder to resolve figure paths in Jupyter notebooks.

Follow-up of #71, where the change was applied to the episode Markdown files.

Noted in #128, the module needs to be packaged or added to the sys.path for calls in both the actions workflow, as well as going through the notebooks locally.

A couple of things to update for episode 1:

• Need to update the subject in the episode from 010002 to 010006 (which is actually in the subset we are using)
• Also need to update exercises to use a different subject in the subset
• The jupyter notebook (located in code) should be updated to reflect the changes of the new dataset. These changes were only made to the episode

Tractogram image unable to be visualized through fury...tested with trackvis and able to save image with ~500000 streamlines.

Change string substitutions for f-strings to improve readability, compactness, and speed and to avoid unexpected errors.

See discussion in PR #65.

In order to try to avoid inadvertently committing and merging the output or result of the execution of the jupyter notebooks, we should incorporate a pre-commit hook that clears the output cells using for example the following command:

jupyter nbconvert --ClearOutputPreprocessor.enabled=True --clear-output *.ipynb

The above requires installing the nbconvert package. Other solutions that do not require any additional package might exist.

Cross-referencing for example #109.

The figure captions are now being rendered correctly, e.g. Streamline propagation differential equation in
https://carpentries-incubator.github.io/SDC-BIDS-dMRI/local_tractography/index.html or
Camera axis orientations in https://carpentries-incubator.github.io/SDC-BIDS-dMRI/discuss/index.html.

The figure index shows the Markdown image title for each image properly:
https://carpentries-incubator.github.io/SDC-BIDS-dMRI/figures/index.html

but those are not visible in the episodes, even when hovering over the image.

I get an error while running "from utils.visualization_utils import generate_anatomical_volume_figure" .
How can I fix it?
before that, I run "from dipy.tracking import utils" and it is ok.

Add a glossary of diffusion MRI terms as a markdown cheat sheet. There exists a specific reference.md page for this purpose.

Mentioned in PR #36.

The carpentries style checks done by the make lesson-check-all command in this repository has uncovered a number of warnings:
https://github.com/carpentries-incubator/SDC-BIDS-dMRI/runs/2395203393?check_suite_focus=true#step:16:9

They should be addressed to be consistent with the carpentries style.

With slice orientations fixed (#121), figures will need to be updated accordingly. Episodes that make use of the utils call include

• CSD
• Deterministic tractography
• Probabilistic tractography

I get this error while running "response, ratio = auto_response_ssst(gtab, data, roi_radii=10, fa_thr=0.7)" for CSD section in Introduction to dMRI.
how can I fix it?

After today's dry-run, I feel that the lesson would greatly benefit from a few improvements that could allow instructors to provide a clearer message concerning some key aspects with the help of some additional figures and some expanded explanations. Especially, if time does not allow to follow the episodes by typing all commands, this would also enable summarizing better the contents of an episode, and hopefully allow learners to grasp better the underlying ideas.

Specifically, it might be beneficial to:

• Add an isotropic vs. anisotropic/random walk picture in the introduction episode (without overlapping with/harm to the eigenvalues/eigenvectors picture in the DTI episode). Probably accompanied by/requiring a better explanation.
• Improve the explanations/add a short theory section about the dMRI physics and what it adds to a regular MRI (main magnetic field, RF pulses, gradients, briefly mentioning the sequences, hopefully with some pictures, briefly mentioning b-tensor encoding, QTI, etc. or adding a picture that shows the tensors and sequences/trajectories). It can be good to look at the sMRI lesson acquisition and modalities episode to avoid repeating things and to be able to assume that some aspects are already known/build bridges across the two lessons.
• Add the definition of the b-value, including its formula. Would maybe require briefly explaining where it comes from/showing the formula's around. Can also lead to linking these to the local models, EAP, FOD, etc.
• Improve the explanation of what the B₀ volume is.
• Improve the explanation about different sources of noise/distortion, hopefully adding pictures to show what such distortions look like/how to distinguish them (magnetic susceptibility, Eddy currents, magnetic field inhomogeneities, ghost artifacts, motion).
• Add a picture of how the FA changes with (an)isotropy.
• Mention the asymmetric ODFs, and why only symmetric ODFs are used customarily.
• Add pictures of the local model glyphs.
• Add an actual picture that shows better where a DTI model would fail (e.g. a crossing fiber).
• Add a picture that clearly shoes the difference of a det vs. prob tracking.
• Add further extra contents about downstream tasks/derivatives (tractography enhancement, bundling, tractometry, etc.).
• Add some extra contents about microstructure models.

Best would be to fix them in separate PRs so that contributions can be done more easily.

Related to #28 and #37.

The introduction and DTI episodes have each an exercise so that learners can demonstrate the acquired skills; however, the exercises are only written in the Markdown files, and they are missing from the Jupyter notebooks.

The exercise instructions should be added to the notebooks and the solutions should be included in the solution notebooks.

Visualizing the three orthogonal views of the peaks and the tractograms, it can be seen the sagittal right is indeed a sagittal left and that the coronal view is flipped upside-down flip (not sure if it is an anterior or posterior).

Note that we have used a sagittal right view in the matplotib scalar maps, so sticking to that view seems the most consistent choice.

Adjusting the roll, yaw and pitch parameters will probably make the trick.

A few {:class=”img-responsive”} tags exist throughout the episodes that seem not to be properly recognized (e.g. `https://carpentries-incubator.github.io/SDC-BIDS-dMRI/diffusion_tensor_imaging/index.html).

They should either be removed or fixed.

Episodes should be made consistent:

• Some Jupyter notebooks still contain the output cells (e.g introduction or DTI. This might be interesting for the solutions, to have a reference of what the output should look like (although they should be in the markdown files as well) but I'd say that it is not for the raw notebooks that will be used for teaching.
• We should maybe decide if Jupyter notebooks will contain the static images. I would maybe remove them. This saves maintenance work (until an automatic method that works flawlessly is found #85) and reduces the risk of discrepancies between the episode markdown and notebooks.
• Similarly, some episodes (e.g. DTI) do not contain the instructions to save the figures, whereas others do (e.g. CSD, tracking). Although this is strictly not relevant for learners, it is useful for developers; otherwise, they need to manually add the commands every time a change to the figures is needed.
• We should define as variables initialized at a single place and value values that are used repeatedly (e.g. the subject id, the FA threshold, etc.). Eases maintenance and reduces the possibility of incurring into bugs.