This package implements a command line tool for creating Python project repositories for QTIM ML projects. The resulting repository has various capabilities to help orgnaize and standardize the lab's projects.
This is a Python package that supplies a command line tool when installed
inside your current Python environment or virtual environment. To install this
package in your current environment, either clone the source code and run pip install .
from the root of the reposiory, or run this to install directly
from git:
pip install git+https://github.com/cookiecutter-qtim/cookiecutter-qtim
The basic usage is simply to run the cookiecutter-qtim
command from the
command line. This will start the process to create a project repository in a
sub-directory of your current directory. To place the resulting output in
another directory, use the -o
option, e.g.:
cookiecutter-qtim -o ~/repos/
Note that -o
controls the directory within which the project sub-directory
will be created, not the path of the directory itself, which will be determined
by the name you give the project during the setup
(e.g. ~/repos/project-name
).
After running the command, you will be presented with instructions in your command line and will be asked to enter further information that will be used to construct your basic project files.
After completing the process, you will have a project codebase with the following capabilities and characteristics:
The resulting project repository is a Python package that can be installed
within a Python environment. This is achieved with the modern
pyproject.toml
format.
As a results, you can install the project Python package using pip either by specifying the path to the root of the cloned directory, e.g. from the root of the cloned directory:
pip install .
or directly from GitHub:
pip install git+https://github.com/group/project
After installation, you can import the code in the Python package from anywhere
without having to manipulate your PYTHONPATH
(which is generally confusing
and not repeatable).
When you are working on the project, it is recommended to install the package
in editable mode with the -e
flag. This means that the installed version of
the code is a symlink to the original source code (rather than a copy) so the
installed version is updated as you edit the files. Furthermore, to install the
list of development time dependencies (used for the automated tests described
below) by appending [dev]
to the installation target. Therefore to install a
local clone for development, run this from the repository root:
pip install -e ".[dev]"
The dependencies of the project are also controlled by the pyproject.toml
file. If you need to use a new library in your project, you must add it to
pyproject.toml
in the project dependencies section (or the
optional-dependencies section if it is needed for development only). When
created, your project dependencies will include some common packages that you
selected during the creation process.
It is recommended that when using the repository, you install it within a project-specific virtual environment, or within the docker image (see below). It is also strongly recommended that the version of python used for the local repository matches that used within your project docker image, or you will likely encounter many strange issues.
The following directories are created within the new repository:
- Python package. The python package itself, i.e. the directory containing
the python source code, will be given the same same as the project. Note
that following conventions, the project name should use
-
to separate words but the python package will change this to_
. For example a project calledproject-name
will have a package calledproject_name
. docs/
- This is where you should maintain instructions for yourself and other team members or future team members on how to use the code in this project repository. Markdown format is suggested to give easy to read documentation when viewed on github itself.notebooks/
- This is where you should store any jupyter notebooks that you want to version control.tests/
- This is used for storing regression tests to run withpytest
.
The project package is initialized with a suggested standardized structure,
with separate sub-packages (sub-directories) for preprocess
, train
,
inference
, analysis
, visualization
, and commands
. You may change this
structure if you feel that it is not appropriate for your project.
There are also three special files in the package:
locations.py
- This should be used to store all relevant file path locations for project files in one standardized place.enums.py
- This is a place to store enumerations related to the project. These are used for variables that may take one of a fixed list of values. Good use cases for enums include classification class names, and dataset partitions.constants.py
- This is a place to store constants for the project that are not file locations or enums.
Each "critical" task within a project should be implemented within the python codebase (not a notebook) with a command line interface (CLI) that allows you to control the behavior (such as input and output directories). Critical tasks are any tasks whose output is required to complete the project, or that is input to another critical task. This includes tasks such as training, preprocessing, post-processing, and inference to produce results, but may not include tasks such as analysis or visualization.
The initialized repository is configured with a command line interface that
is installed along with the project's Python package. If your project name
is proj-name
then you will be able to run:
proj-name <sub-command> <arguments>
from the command line from any directory to execute project tasks (e.g.
proj-name train
). Sub-commands are any module (i.e. Python file_ of the
commands
sub-package (i.e. in the commands
directory) that defines a
function whose name matches the module and is decorated with the
@click.command()
decorator. This makes it easy to add new sub-commands to the
project for each task. The project is initialized with an example train
command to demonstrate how this works.
This setup makes it very straightforward for other scientists to understand what tasks they need to run to be able to work with the project, and how they should run them. They can run
proj-name --help
for a list of sub-commands and
proj-name <sub-command> --help
to find options and arguments for a given sub-command.
For tutorials on writing command line interfaces using the recommended click
package, see its documentation.
The initialized repository contains an example train command that demonstrates a few recommended/required patterns that you should follow:
- How to set up a command for the CLI using
click
. - How to use the
pycrumbs
package and the@tracked()
decorator to automatically create job records for tasks. - How to read training parameters from a configuration file stored within the repository itself.
You can find the command in the commands/train.py
file of the Python
package. You can run it using
<project-name> train example <name_for_model>
pytest
is a package used for writing and running
regression tests for Python. Regression tests are pieces of code that are run
automatically when changes are made to the code that check that the code
still runs as expected. They are used to prevent you from introducing errors
when editing your code.
Tests can be time consuming to write in the short term but save considerable
time in the long term. Therefore it is up to you to determine what tests to
write for your project codebase. Any tests you write in the tests/
directory
in files beginning named test_*.py
will be run automatically whenever you use
the pytest
command from the project root, or run the pre-commit hooks (see
below).
Pre-commit is a tool for automating tests and fixes on your code before every commit is made. The new project repository comes pre-configured to run a number of different processes outlined below when commits are made:
- Black: This tool automatically fixes many style errors to bring the code in line with the PEP8 code guidelines to make it neat, readable and in-line with Python best practices.
- Flake8: The
flake8
tool performs various checks on Python code. It will spot obvious errors such as undefined variables or missing imports. It will also spot PEP8 style errors (that could not be fixed byblack
automatically). Furthermore, it will enforce correct use of docstrings using the numpy convention using theflake8-docstrings
plugin. It is rarely recommended, but in some situations you may need to ignore specific errors fromflake8
. You can configure its behavior from the project'ssetup.cfg
file. - Pytest: this runs all tests in the
tests/
directory usingpytest
as described above. - General code tidiness: there are several further small hooks that perform tasks such as removing trailing whitespace from files, normalizing line endings, removing cell contents from Jupyter notebooks, and checking JSON, TOML and YAML files have valid syntax.
When you first create the repository, pre-commit will have been installed and enabled for you. However, if you later move to a new machine you will have to manually enable it by running this from the project root:
# First install the project with development dependencies
# Make sure you do this in the correct virtualenv
pip install -e ".[dev]"
# Now enable pre-commit
pre-commit init
When pre-commit is enabled, it will run automatically on all changed files
before the commit is made. If any of the tests fail or result in changes to the
files, the commit you were trying to make will not complete. You will then need
to git add
the resulting changes again and attempt the commit again. This
should help ensure that files are kept organized throughout a project.
You can run the checks on all files without making a commit using:
pre-commit run --all-files
Refer to the pre-commit documentation for more information on using configuring and running pre-commit hooks including skipping tests if necessary.