Code Monkey home page Code Monkey logo

pythontemplate's Introduction

Python Template

Logo

GitHub Actions CI status

The SciKit-Surgery paper

Follow scikit_surgery on twitter

Python Template is a Cookiecutter template for creating a python project. When used with Cookiecutter Python Template will create a fully working python "Hello world" project with unit tests, GitHub Actions, linting, PyPi distribution and sphinx documentation.

Authors: Tom Doel, Stephen Thompson, Matt Clarkson, Thomas Dowrick, Mian Ahmad, Miguel Xochicale

Python Template is part of the SciKit-Surgery software project, developed at the Wellcome EPSRC Centre for Interventional and Surgical Sciences, part of University College London (UCL).

Purpose

Python Template was developed by the Wellcome EPSRC Centre for Interventional and Surgical Sciences, to assist researchers in the development of small but well engineered software components. It has been widely used in the development of SciKit-Surgery, but can be used as a starting point for any Python project. The template supports and encourages well engineered software by helping researchers with standard software development tasks (Software Process).

How to deliver standard software development tasks

The table below lists standard development tasks and how the Python Template helps to deliver them. Paying heed to the following tasks at the earliest stage of development should support the creation of well defined, maintainable, well documented, and well tested code.

Task What the Python Template Does
Requirements generation The template creates the file doc/requirements.rst and populates it with some minimum requirements for all SciKit-Surgery projects. Instructions and relevant links are provided to encourage the researcher to add more requirements to the file to describe what the project does, both before and during development.
Software design The Python Template provides a modular framework that integrates with the wider python ecosystem. Providing a highly modular environment enables the researcher to focus on algorithm development, testing and publication, with minimal consideration of the wider software design.
Managing source code versions and updates Source code versioning and updates is handled by git. The Python Template supports several git repository providers,
Configuring projects for specific platforms The Python language is cross platform. The Python Template takes advantage of the pip package manager to create modular projects that can be easily installed on multiple platforms.
Compilation and linking By using the python environment, the Python Template avoids the need for compilation and linking.
Testing the code at run time The Python Platform uses tox to automate tests (pytest), coverage analysis and linting (pylint) comes ready with appropriate unit tests to meet the minimum requirements (see first item). If using GitHub to host the code, cross platform continuous integration testing is set up by default. It is up to the researcher to write further tests as the code is developed.
Verifying the validity of output The validity of output is performed as part of the unit testing.
Documenting the code The Python Template provides a template for code documentation showing how to write in line documentation. Documentation can be automatically built using ReadTheDocs
Tracking and repairing bugs Issue tracking is handled by default by the chosen git repository host.

Whilst every effort has been made to automate tasks where possible, it is up to the researcher consider software requirements, write documentation, and create unit tests. Templates for each of these tasks are provided, so after creating your project, please take a little time before you start coding to think about what you want your software to achieve, and write it down in the file doc/requirements.rst.

How to use

  1. Install Cookiecutter package and other requirements using virtual environments (VE) in conda or mambaforce by replacing conda with mamba.
conda create -n ssptVE pip -c conda-forge
conda activate ssptVE
python -m pip install -e ".[dev]"
  1. Test template using tox and pytest
conda activate ssptVE
tox

3. Run Cookiecutter with the URL of this template. Alternatively, you can run it in the same path of this repository if you are developing a new feature for this template.

cookiecutter https://github.com/SciKit-Surgery/PythonTemplate.git
cookiecutter .

4. Answer the questions to configure your template (press enter to choose defaults), for further details on available answers. See "List of parameters for cookiecutter" below.

  1. Create a local git repository for your new project and commit the files.
cd MyNewProject
git init
python -m pip install -e ".[dev]"
git add .
git commit -m "Initial commit of My New Project"

Note: make sure you use "git add ." not "git add * " to pick up hidden files like .gitlab-ci.yml.

  1. Create a new project on GitHub, making sure the URL matches what you set in step 3.

7. Add a remote in order to link your local repository to your GitHub repository and push the files across. NOTE: use the SSH form of the repository URL for SSH key authentication (so you don't have to enter a username and password).

git remote add origin [email protected]:GITHUB_USERNAME/MyNewProject.git
git push origin master

8. If using GitHub, and assuming your URLs are all correct, GitHub Actions should automatically run tests for your project.

  1. You can verify your project has been set up correctly by installing and running tox.
cd MyNewProject
tox
tox -e lint
tox -e docs
cd ..

tox runs several different stages, including pytest for unit tests and pylint for check for PEP8 linting. These can also be run independently:

python -m pytest
pylint --rcfile=tests/pylintrc mynewproject
  1. Install your package in editable mode and build your package
python -m pip install -e .
python -m build

10. Take a moment to write some software requirements, and populate the README file with a basic description of what you want to do, then start coding.

List of parameters for cookiecutter

project_name
Name of the project as it will appear in the documentation (can have spaces)
project_slug
Project name as used in the URL; by default project_name with spaces removed
pkg_name
The main python package name for your project. By default this is the project_slug converted into lower case. pkg_name should not contain dashes (-)
repository_server
The URL of the git server that will host your code. If you don't want to use any of the predefined options, choose any and modify the project_url later
full_name
Your full name, for authorship information, and to suggest your profile name
repository_profile_name
Your personal profile name on GitHub. Or you can enter a group name that will be used to construct the repository URL
repository_path
Choose whether you want your project under your personal profile path or in a shared location
repository_url
The full URL to your project will be constructed from the previous options, and is of the form repository_server/repository_path
open_source_license
Recommend BSD-3 for internal projects, Apache for external collaborations
copyright_holder
If you are a member of UCL you should accept the default text

For developers

  1. Run Cookiecutter with the URL of this template.
cookiecutter .
cd MyNewProject
pip install -r requirements-dev.txt
tox
tox -e lint
tox -e docs
python -m pytest
pylint --rcfile=tests/pylintrc mynewproject
python -m pip install -e .
python -m build

Contributing

Please see the contributing guidelines.

Copyright 2017 University College London. Python Template is released under the BSD-3 license. Please see the license file for details.

Acknowledgements

Supported by Wellcome and EPSRC.

pythontemplate's People

Contributors

mattclarkson avatar mianasbat avatar mxochicale avatar tdowrick avatar thompson318 avatar tomdoel avatar

Stargazers

 avatar  avatar

Watchers

 avatar  avatar  avatar

pythontemplate's Issues

Add requirements file and improve documentation

In GitLab by @StephenThompson on Oct 18, 2018, 10:41

We should encourage people writing libraries to think about what the library is supposed to do. I think the best way to do that is to add a requirements file to the documentation with some example SNAPPY minimum requirements.
Whilst we're at it I'd like to make the documentation more readable to the less technical reader, I think the installation instructions could be structured better, and somewhere we need to make the case for SNAPPY. i.e. why should you use this template.

Use pytest in template

In GitLab by @StephenThompson on Nov 13, 2018, 15:40

I tried as part of #19 to get the template to use pytest rather than unittest, but failed. So at the moment the PythonTemplate used unitest and the generated projects used pytest. Would it be nicer if they used the same thing?

Formatting in CONTRIBUTING.rst

I think it will be nice to hide the links behind words in CONTRIBUTING.rst for a neater and cleaner look.

E.g. the following can be changed to


Look through the issues on https://github.com/UCL/scikit-surgerycore/issues and assign the relevant issue to yourself. If there is not an existing issue that covers your work, please create one: https://github.com/UCL/scikit-surgerycore/issues/new


Look through the issues on and assign the relevant issue to yourself. If there is not an existing issue that covers your work, please create one.


Move PythonTemplate to GitHub

I think that we decided that we want to move PythonTemplate to GitHub (correct?)

Are we going to move to GitHub Actions as the default CI?
Should we keep the GitLab CI template in, for any private projects that we want to keep on GitLab?
We should add github.com/UCL as an option for repository_server/repository_path
Anything else?

Can discuss in more detail at the next SNAPPY meeting.

@mianasbat @MattClarkson @thompson318

Remove tkinter from default generated project (or detect if not installed)

In GitLab by @ThomasDowrick on Jan 15, 2019, 10:11

The project generated by PythonTemplate used tkinter in the example code, but as we're not actually using tkinter for anything, it might be better to change it to someting else/remove it.

Also, as tkinter doesn't come with Python by default, if the user hasn't pip installed it, then the generated project won't run.

'Deploy docs to staging' step of CI reqires project folder to already exist on staging server

In GitLab by @ThomasDowrick on Oct 19, 2018, 15:14

The CI pipeline for a new project includes a 'deploy docs to staging' step. For a project in WEISS/SoftwareRepositores/ExampleProject, the WEISS/SoftwareRepositories folder needs to already exist on the staging server (gift-emperor, tvercauteren2.cs.ucl.ac.uk).

This folder now exists, but I had to ssh to the server, create it and use chown to change the owner to 'staging'. Any future project created under SoftwareRepositores should be fine, but it seems like there should be a better solution.

Deploy docs to production setup

In GitLab by @ThomasDowrick on Oct 31, 2018, 15:22

There are two 'deploy docs' steps in the CI pipeline.

Deploy docs to staging builds the documentation on the staging server, which is one of the runner machines.

Deploy docs to production isn't run by default, as there is no production server specified. I guess we need to set this to something, or setup a post commit hook?

Only run test suites/coverage for files which are generated by the user

In GitLab by @ThomasDowrick on Oct 19, 2018, 10:09

The files which are generated as part of the cookiecutter command shouldn't need testing/coverage analysis by the user once they have been generated.

If testing is needed, it should be as part of the PythonTemplate project build, rather than the generated project.

Coverage broken if specified in tox.ini

In GitLab by @MattClarkson on Nov 21, 2018, 21:02

Using git blame:

This commit:

eb9b7ad1 (Stephen Thompson 2018-11-13 13:54:49 +0000 16)            coverage run -a --source ./{{ cookiecutter.pkg_name }} --omit {{cookiecutter.pkg_name}}/ui/{{cookiecutter.pkg_name}}_command_line.py -m pytest

broke the coverage. Well, I say broke. By specifying a single file to omit, it appears to mean that the .coveragerc is not used. This means that anything else specified in the .coveragerc is now ignored.

@StephenThompson - is that what you intended?

I would prefer to put such a substitution in the .coveragerc file, if indeed that works.

Change default CI pipeline so that linting errors don't cause it to fail

In GitLab by @ThomasDowrick on Oct 22, 2018, 15:15

tox runs tests, coverage, linting stages etc.

At the moment, any linting errors cause the CI pipeline to fail at the testing stage. While it is important to have the code well presented, some of the linting errors can be OTT, and a bit fiddly to fix each time changes are made, and it makes it less apparent if the unit tests have actually suceeded.

We could:

  • Only have the linting tests on the master branch, but not for development on issue related ones. The latest version of pylint has a --exit-zero flag which could be used for this.

  • Not require a perfect linting score (10.0/10.0) to pass.

Contents of {{cookiecutter.project_slug}} folder not showing properly through web interface

In GitLab by @ThomasDowrick on Oct 19, 2018, 11:10

Going through the web interface, it is not actually possible to look inside the {{cookiecutter.project_slug}} folder, it seems to get into some wierd loop, showing the contents of the top level folder again.

If I git clone PythonTemplate, my local copy is fine. Also, looking at other cookiecutter projects, hosted on GitHub, the same problem doesn't appear.

Test created projects on Python 3.7

In GitLab by @ThomasDowrick on Jan 8, 2019, 17:38

We currently test using Python 2.7 and 3.6. However, the latest version of Python is 3.7, so this will be what most people will download by default.

So far I have found out that scikit-surgeryvtk doesn't work on 3.7, as there aren't the necessary wheels for PySide2 or vtk. I think opencv does have a 3.7 wheel, but haven't tested it. There isn't anything we can do about these external dependencies at the moment, but we should at least know about any issues that arise.

remove travis badge and second coverage badge

In GitLab by @ThomasDowrick on Nov 13, 2018, 16:42

Readme.rst currently has a badge for Travis-CI and two for coverage.

The Travis one isn't linking to anything, as we're not using Travis and the second coverage badge is redundant, so remove them both.

Table of Contents generation looks a bit wierd

In GitLab by @MattClarkson on Nov 7, 2018, 08:17

We now have requirements.rst and pycharm.rst and also the generated module list, and also a sphinx warning about modules.rst not being include.

So I tried fiddling with toctree, but only concluded that I didnt understand it.

Does anyone know what is meant to happen. The current example seems a bit odd, in that the index.rst has multiple toctree directives, and the index.rst includes README.rst which itself has a toctree directive.

So Im not even sure how many table of contents we should have.

Add unit tests

In GitLab by @StephenThompson on Oct 31, 2018, 14:35

Following on from issue #8 I have added three basic requirements. The initial unit tests need to be updated to test these requirements.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.