HackMD Note: https://hackmd.io/@Tk4Zdl52TT6lvxneSz_IfA/rk-iwBXrr

Profiling:

• Snakeviz: https://jiffyclub.github.io/snakeviz/
• line-profiler: https://github.com/rkern/line_profiler
• Profiling and timing code: https://jakevdp.github.io/PythonDataScienceHandbook/01.07-timing-and-profiling.html

PyPI: Python Package Index

For packages installable from PyPI, PyPI's classifier allows finding a package from a set of defined categories:

https://pypi.org/search/

ncar-jobqueue: https://github.com/NCAR/ncar-jobqueue provides convenient utilities for using dask-jobqueue on some clusters used @ NCAR. Hobart is one of the supported clusters:

• Install ncar-jobqueue

pip install ncar-jobqueue

• Import and use NCARCluster from ncar-jobqueue

>>> from ncar_jobqueue import NCARCluster
>>> from dask.distributed import Client
>>> cluster = NCARCluster()
>>> cluster
NCARCluster(cores=0, memory=0 B, workers=0/0, jobs=0/0)
>>> cluster.scale(2)
>>> cluster
NCARCluster(cores=96, memory=192.00 GB, workers=2/2, jobs=2/2)
>>> client = Client(cluster)

cc @jkshuman

Originally posted by @matt-long in ncar-hackathons/faq#1

Updates to Cheyenne have, in the past, change the RSA keys for the analysis machines. This can lead to problems if the new keys don't match those in ~/.ssh/known_hosts. Remove the offending keys in ~/.ssh/known_hosts.

Originally posted by @matt-long in ncar-hackathons/faq#2

Python packages use the Python warnings module to issue warning messages. The details of using the warnings module are general to Python, and apply to any Python software that uses this system.

Since warnings are not fatal, a program may encounter the same warn-able situation many times in the course of running.

Warning messages are typically issued in situations where it is useful to alert the user of some condition in a program, where that condition (normally) doesn’t warrant raising an exception and terminating the program. For example, one might want to issue a warning when a program uses an obsolete module.

Categories

The warning categories are described in the documentation

How to hide warnings?

At the beginning of your script or jupyter notebook, include:

import warnings
warnings.filterwarnings('ignore')

Note: This might not suppress all the warnings, but it will suppress most of them.

Which warnings should I be attentive to?

DeprecationWarnings

It's a good practice to gradually deprecate one's library's API, so that users get advance warning of coming changes. Therefore, if you see deprecation warning from a package that you plan on updating in the future, you should be pay attention that features might go away in the future releases.

Which warnings should I be ignore?

TODO