hpc-carpentry / hpc-python Goto Github PK

Hello hpc-carpentry team,

I am an Carpentry Instructor and I'd be quite interested to turn this lesson into an official Data Carpentry lesson ultimately as I see a considerable interest for Snakemake among Life Scientists.

As Snakemake is primarily used in genomics as far as I can tell, I'd like to change the example from words and books to some small sequencing dataset.
An example can be seen here in the official Snakemake read the docs.

What do you think?
Cheers
Marc

this and the dask documentation already has some really good examples that could serve as a jumping off point: https://github.com/sdsc/sdsc-summer-institute-2019/tree/master/hpc0_python_hpc

an overview of the "why" of dask: https://notamonadtutorial.com/interview-with-dasks-creator-scale-your-python-from-one-computer-to-a-thousand-b4483376f200

This should be done along with providing more detailed setup instructions.

To remove all files created by snakemake as output files to start from scratch, you can use

$ snakemake some_target --delete-all-output

https://snakemake.readthedocs.io/en/stable/project_info/faq.html#how-do-i-remove-all-files-created-by-snakemake-i-e-like-make-clean

Based on what I saw here, I think it's a nice treat to add (travis) CI for this repo. This could be integrated once #16 is merged.

I am not sure though if this interacts with Github Pages deployment somehow. Does anybody know?

The Carpentries stylesheet provides magic to format code. These should be applied throughout the hpc-python lessons.

Silent error unveiled by #40.

@zkamvar
We need to update style for this lesson so that jump lists (anchors for different sections) becomes automatically available for different levels of headings.

@tkphd I added you to the list of maintainers of this project for now. This should give you merge permission. We can revoke this permission at any time.

I am not aware of any code owners right now. But the content of this gem related to snakemake is simply too valuable IMHO. I'd like to make some more suggestions and PRs in order to edge out some inconsistencies in the plot of teaching (just taught this material for the first time). For this, I need an active maintainer to accept the PRs.

@jstaf Are you still involved in this project and have the capacities to review/merge PRs?

I am bound to teach snakemake at my place. People have expressed high interest, so I just went through the current state of the material. I think it contains a lot of gems and is well written. Thanks to all the authors.

However, I have the feeling that workflows are so important, that they potentially need a repo for their own. I could imagine refactoring the snakemake focussed modules into hpc-carpentry/hpc-workflow. I just wanted to put this idea out there and hope that people comment on this.

The GitHub diff interface wraps lines at 80 characters. Since we use 100 characters as the max width, this makes for ugly diffs with lots of continued lines. To improve the clarity of these diffs, as well as readability in terminal text editors, we should back away from this edge, and wrap our content at 79 characters.

This is easily done with emacs, but the change would touch almost every file in the repository, so this should be coordinated with other efforts to minimize merge conflicts and the number of branch rebases to be done.

Hi!

I forked this repo with the intent of contributing to it.

When I tried to render the lesson locally, it failed with the following error:

Could not find gem 'mdl' in any of the gem sources listed in your Gemfile.
Run `bundle install` to install missing gems.

After running bundle install, the dependency was installed and I could render the lesson.

However, I propose that the template Gemfile from carpentries/lesson-example be used. This will ensure consistency with other lessons and thus helps to advance the development stage of this lesson. I was able to successfully render the lesson locally using this Gemfile.

I will submit a PR addressing this, but please feel free to reject it if there's a reason to use a custom Gemfile.

Best,
V

As mentioned here, the call to

$ python3 plotcount.py isles.dat show

should open a window to produce a plot. On my system:

Python 3.7.5 (default, Oct 17 2019, 12:09:47) 
[GCC 9.2.1 20190827 (Red Hat 9.2.1-1)] on linux

Nothing happens.

There is a fork of this lesson in the Carpentries Incubator which focusses purely on Snakemake (and that is indeed the majority of the content in this lesson). Does it make sense to maintain this lesson if others are willing to maintain that independently? Should we put some effort into making the lesson in the incubator more transferable instead?

That link points to:
https://github.com/hpc-carpentry/hpc-python/blob/gh-pages/hpc-carpentry.github.io/hpc-novice

Which results in 404 error.

I propose to split the hpc-python in half:

• hpc-python: episode 1 to episode 6 remains where it is

• new hpc-carpentry repository 'hpc-workflows': episode 11 to episode 18

The introduction to snakemake (the second part of hpc-python) is very well done and would offer more effective real estate to focus contributors attention if in an isolated repository. I feel that it is a bit hidden currently, which I believe is a shame.

see also #13