ucl-arc / hpc-shell Goto Github PK

View Code? Open in Web Editor NEW

This project forked from hpc-carpentry/hpc-shell

0.0 2.0 1.0 14.12 MB

Materials to teach terminal fundamentals for HPC users

Home Page: http://rits.github-pages.ucl.ac.uk/hpc-shell/

License: Other

Makefile 3.03% HTML 40.35% CSS 8.92% JavaScript 1.52% R 3.64% Python 42.14% Shell 0.20% Ruby 0.21%

hpc-shell's Introduction

Introduction to using the shell in a High-Performance Computing context

This lesson provides an introduction to the bash shell aimed at researchers who will be using the command line to use remote, high-performance computing (HPC) systems. The material is also suitable for teaching the use of the shell for any remote, advanced computing resources.

Using this material

Follow the instructions found in the Software Carpentry example lesson source to create a repository for your lesson.
Edit _config.yml to modify the configuration options at the bottom for the remote host you will be using. These options set such things as the address of the host to login to and the ID that is used to incorporate host-specific code snippets (see below).
Create the required host-specific code snippets in subdirectories in _includes/snippets. These snippets provide inputs and outputs that are host-specific and that are included automatically based on the configuration in the _config.yml file.
1. Code snippets are in files named snippet_name.host_id and are included automatically when the lesson is built. For example, if the snippet_name was login_output and the host_id was ComputeCanada_Graham, then the snippet file would be called login_output.ComputeCanada_Graham.
2. Code snippets are placed in subdirectories that are named according to the episode they appear in. For example, if the snippet is for episode 01, then it will be in a subdirectory called 01.

Lesson writing instructions

This is a fast overview of the Software Carpentry lesson template. This won't cover lesson style or formatting (address that during review?).

For a full guide to the lesson template, see the Software Carpentry example lesson.

Lesson structure

Software Carpentry lessons are generally episodic, with one clear concept for each episode (example).

An episode is a markdown file that lives under the _episodes folder. Here is a link to a markdown cheatsheet with most markdown syntax. Additionally, the Software Carpentry lesson template uses several extra bits of formatting- see here for a full guide. The most significant change is the addition of a YAML header that adds metadata (key questions, lesson teaching times, etc.) and special syntax for code blocks, exercises, and the like.

Episode names should be prefixed with a number of their section plus the number of their episode within that section. This is important because the Software Carpentry lesson template will auto-post our lessons in the order that they would sort in. As long as your lesson sorts into the correct order, it will appear in the correct order on the website.

Publishing changes to Github + the Github pages website

The lesson website is viewable at https://hpc-carpentry.github.io/hpc-shell/

The lesson website itself is auto-generated from the gh-pages branch of this repository. Github pages will rebuild the website as soon as you push to the Github gh-pages branch. Because of this gh-pages is considered the "master" branch.

Previewing changes locally

Obviously having to push to Github every time you want to view your changes to the website isn't very convenient. To preview the lesson locally, run make serve. You can then view the website at localhost:4000 in your browser. Pages will be automatically regenerated every time you write to them.

Note that the autogenerated website lives under the _site directory (and doesn't get pushed to Github).

This process requires Ruby, Make, and Jekyll. You can find setup instructions here.

Example lessons

Some links to example SWC workshop lessons for reference:

Example Bash lesson
Example Python lesson
Example R lesson (uses R markdown files instead of markdown)

hpc-shell's People

Contributors

Watchers

Forkers

themkots

hpc-shell's Issues

"Scripts, variables, and loops" episode

consider using the term "section" instead of "column" for the text underneath "That’s a huge amount of output. Let’s see if we can understand what it is, working left to right." Also upstream
change line in script from "# This wont show anything" to "# This won't show anything." Yep, upstream too.
typo in Solution "truely" should be "truly". Yes, upstream

Errors on "Why use a cluster?" page

Extra 'a' in "Many HPC systems operate a pool of resources that are drawn on by a many users."
This needs to be resolved upstream too.
Change "Typing-based interfaces are often called a command-line interface" to "A typing-based interface is often called a command-line interface". And upstream.

In the section "f you didn’t get them in the last lesson, make sure to download the example files used in the next few sections", it should also mention the tar command before the ls. Also upstream
The Solution text "Output of ls is one line per item so counting lines gives the number of files." would be more precise as "Output of ls is one line per item, when chaining commands together like this, so counting lines gives the number of files." Also upstream
In Key Points, change "Redirect a commands output" to "Redirect a command's output". Also upstream.

Typos on "Connecting to the remote HPC system" page

Change "There are a variety of programs that can be used for this," to "There are a variety of programs that can be used for this;" Also upstream
Change "it is easy add plug-ins" to "it is easy to add plug-ins". And upstream
Change "It is strictly speaking not necessary" to "It is, strictly speaking, not necessary". And upstream
Change "as you would other software on you Windows " to "as you would other software on your Windows". And upstream
Potential security issue. "Note that you may want to paste in your password rather than typing it. Use Ctrl plus a right-click of the mouse to paste content from the clipboard to the PuTTY terminal." This implies you could have a file with the password stored on it - a bonus for hackers! This one already notified upstream by their issue 22.

Link to gh-pages

Under

About - Materials to teach terminal fundamentals for HPC users

on the top right of the repo home page, add the correct link to our gh-pages.

Typos on "Writing and reading files" episode

The following text does not correctly render the "less" command in "computer" format:
"Out of cat, head, tail, and less`, which method of reading files is your favourite? Why?"
Also upstream

Changes required to "Moving around and looking at things" page

Change "At the point in this lesson" to "At this point in the lesson". Also upstream
NB about exercise "Open a second terminal window on your local computer". Does Putty let you do this?. Also upstream
After sentence "Next, let’s find out where we are by running a command called pwd (which stands for “print working directory”)." Add "("Directory" is another word for "folder")." And upstream.
Move text "What is the ~ character? When using the shell, ~ is a shortcut that represents /home/yourUserName." to immediately before "Now that we know how to use cd". And upstream
Change text "/ is the “root” or base directory." to "/ by itself is the “root” or base directory." And upstream
After

To go back to your home directory, the following two commands will work:
$ cd /home/yourUserName
$ cd ~

add

$ cd

Also change 'two' to 'three' above. And on upstream

Change "(but check you local configuration to be sure!)" to "(but check your local configuration to be sure!)" Also upstream
Immediately after "Some systems may offer local scratch space while executing a job." add text "(A job is a program which you submit to run on an HPC system, and will be covered later.)". Also upstream
add closing bracket to text "(sometimes it’s ‘–’ followed by the name of the flag". And upstream
change "they syntax" to "the syntax". And upstream
It asks the user to open a second terminal window, with one of these on the local system. But if the user is using Putty, they will not be able to view the local system with it. Also upstream