Unable to see Sherlock terminal's modules in Jupyter notebook about lessons HOT 9 OPEN

Ok! I figured one way to solve this. The only modules that will be usable in the jupyter notebook will be the ones that go in the py3-jupyter.sbatch file which is used when submitting the job to the compute node. py3-jupyter.sbatch resides in the forward folder and needs to be modified to include other modules.

from lessons.

I don't work at Stanford anymore, but I'd be happy to help as best as I can, and I suggest you reach out to sherlock support after that. These lessons were written ~5 years ago and I left Stanford in February so things are likely to change!

But I am running into issues as I am unable to use the modules that I have loaded on the interactive terminal of my Sherlock login outside of this jupyter notebook.

So I suspect that forward is not sourcing your bash profile, where likely you have these things set up - it's sort of a raw/default you'd get without doing anything. Have you tried sourcing your profile? And I'm assuming that's where your custom modules are loaded?

!python --version

I'm pretty sure Kilian used to install python3, so try python3 --version instead.

from lessons.

Also don't forget you are in a python session so you can just do:

import sys

sys.version_info
sys.version_info(major=3, minor=8, micro=8, releaselevel='final', serial=0)

from lessons.

Thanks for your prompt respose. Tried !python3 --version which returns Python 3.6.1 from the notebook. So that is good.

Tried

import sys
sys.version_info

and that returns sys.version_info(major=3, minor=6, micro=1, releaselevel='final', serial=0)

from lessons.

Perfect! So that means the Python on the path is the one you are interacting with in the notebook, which is probably what you want :)

from lessons.

Yes! But not sure why the modules are not syncing between what I see on my Sherlock terminal using module list and what I see in the notebook using module list (see screenshots from before). This is what is causing me all the problems as I am unable to use some python libraries

from lessons.

Did you see my comment above? You probably aren't sourcing your bash profile, as you would when you login to an interactive session.

from lessons.

When I do !source /home/users/sakshamg/.bashrc from Jupyter notebook, it returns /etc/profile.d/z99_srcc.sh: line 480: 'slurm-jobid_prpt': not a valid identifier

I checked that the .bashrc file is indeed located at /home/users/sakshamg/

from lessons.

Then you probably want to look in that file and see exactly what modules are added (e.g., module use) and make sure to add that to your notebook.

And for production / your final code, I highly recommend you build (and run) a container to not rely on cluster-specific modules!

I can't comment beyond that - I'd recommend research computing support.

from lessons.