Code Monkey home page Code Monkey logo

uabrc.github.io's Introduction

Researcher Facing Documentation

Our documentation is available at https://uabrc.github.io/.

Contributing

Please see https://uabrc.github.io/contributor_guide/.

Developer Notes

Generating Partition and QoS tables

The repo for generating these files is located at https://github.com/wwarriner/slurm_status_tools/.

To use, install the conda environment and run the following commands.

python -u sstatus.py -c partitions > partitions.csv
python -u sstatus.py -c qos > qos.csv

Generating and Maintaining Hardware tables

The repo for the main hardware table is located at https://gitlab.rc.uab.edu/rc-data-science/data-science-internal/cluster-fabric-docs.

To use follow the readme at the repo.

Useful Regex

Checking Internal Links are Relative

There is no way to fix this automatically, so we rely on checking and reporting. A useful regex is is \[.+\]\(/[a-zA-Z]+.*\). It searches for square brackets with text inside, followed by parentheses with text inside. The text inside the parentheses must start with a slash followed by letters. Another useful regex is similar \[.+\]\((?!https)[a-zA-Z]+.*\). It searches for the same as before, but instead of a slash followed by letters, it starts with any letters except the string https, since https links are external.

Checking Indentation

Currently Prettier bulleted list indenting is wonky for markdown. In addition to indenting list markers, it pads out spaces after the marker. Please see this issue for more details. As a result, we can't automatically format markdown documents, so we need to rely on spotting incorrect indents. Use the following regex ^[ ]{1,3}[^ ]. It will search for one to three spaces followed by a not-space character.

uabrc.github.io's People

Contributors

bdu-birhanu avatar bjoyce3 avatar flakrat avatar iam4tune avatar jgordini avatar jprorama avatar mdefende avatar premas avatar wwarriner avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

uabrc.github.io's Issues

Jupyter Notebooks caveat

Interactive jupyter jobs can fail to start and "disappear" without warning due to an issue with environments.

If the user supplies a conda activate line in the Environment Setup box, and DOES have jupyter installed but does NOT have nb_conda_kernels installed, the job will fail to start and the job card will disappear from My Interactive Sessions. Other errors crop up for other packages, but they aren't fatal. I will list these below.

I've talked with Louis about rearranging the order of operations, but at this time it isn't feasible. There are users who legitimately need to use custom versions of the jupyter server, such as in the Qiime2 module.

We need to update the docs to include this caveat. Something like

If you need to or want to use `conda activate <env>` in the Environment Setup box of the
Interactive Jupyter Notebook form, and your environment has the `jupyter` package installed,
you will need to install additional packages in your environment. In addition to the usual
`ipykernel`, you will need the following

nb_conda_kernels
nb_anacondacloud
nb_conda
nbpresent

Add Gitlab documentation

  • Github vs Gitlab and use cases for UAB researchers
  • Login using XIAS accounts
  • Login using BlazerIDs (only BlazerIDs! No uabmc as a warning)
  • Suggestions for using and setting up projects/groups and how that might be useful for research groups getting started

RStudio takes a very long time to start

The job starts quickly, but the studio server tab has the spinning wheel of death and times out, or gives a 502/503 error (indicating the RStudio Server is overloaded).

@mdefende Could you relate your understanding of this topic here, and we can craft it into a brief admonition/section for OOD Interactive?

RITM0511239

Add section on snapshots to cloud.rc docs section

INC0470298 and the following are useful starting points for modeling snapshots:

This is expected if your instance is booted from volume, (i.e a volume backed instance). During instance snapshot a of a volume backed instance, first a volume snapshot is created and then a zero byte image that is a pointer to the volume snapshot is created. It is zero bytes because it contains no real data just a metadata that points to the corresponding volume created during instance snapshot process.

You can still boot up an instance, there are two ways you can do it.

  1. Go to volumes->snapshots and click on the dropdown beside the create volume button. You will see an option to launch instance which will allow you to create an instance.

  2. Go to Compute->images and launch the instance snapshot. Just make sure the default option "Create new volume" is not changed.

This default option makes sure Cinder is doing the disk setup, and not Glance, and Cinder is able to get snapshot id from this image's metadata and create a new volume from the actual volume snapshot, which gets created when "Instance Snapshot" is used.
So "Instance Snapshot" can also be used, just have to always use "Create a new Volume" from such images, and use volume snapshot (which also gets created) to create a volume, not the 0 byte image.

Formatting yaml according to spec

The spec is here: https://yaml.org/spec/1.2.2/#73-flow-scalar-styles

Strings should use no quotes whenever possible, these are interpreted according to their apparent datatype. In mkdocs.yml they're pretty much always strings as far as I can see.

If we need to interpret something as a string that isn't a string, use single quotes '1024' is the string 1024, not the int 1024. These strings are literal, all characters are interpreted as they are written except '.

If we need to interpret a string with escaped characters then use double quotes so that Hello\nWorld becomes

Hello
World

The cost of the last option is we need to escape \ and ".

On a related note I use the prettier extension for formatting yaml (see the readme) and it keeps indenting things further in. It's driving me nuts, so I'm planning to reformat the yaml document that way. Besides which, it's automated so future changes are set to a standard.

Create FAQ

Add FAQ entries in comments.

  • Why is my job taking a while to schedule?
  • Can you install this software?
  • Can you create a Cheaha account for me?
  • Can you create a UAB Cloud account for me?

Stale internal links

There are some stale internal links (doc pointing to other doc), probably byproducts of conversion to markdown. We'll need to sweep through and fix them.

Look for constructs like

`text here<stuff here>`

MkDocs redirects

Not an issue so much as a solution for future problems. HTML redirecting is possible in MkDocs through the following plugin: https://github.com/datarobot/mkdocs-redirects

So if a page is renamed and the URL changes, we can create a redirect from the old page to the new page or to a section of a page that's replacing it

cloud.rc: Warning note about changing fingerprints for SSH

A result that is surprising to first-timers and novices is what happens when using SSH to attempt connection to a reused floating IP.

The fingerprint of the key on the remote will change, and the local ssh will balk at connecting to the remote host. Ideally they'll have strict checking and the ssh won't allow the connection.

The solution is:

  1. Identify the remote hostname (i.e. the floating IP or an associated domain name). The error message will explicitly state this.
  2. Use the command ssh-keygen -R HOSTNAME replacing HOSTNAME with the remote hostname found in step 1.

We should add a note about this, and a DANGER admonition cautioning that the solution needs to be carefully thought through each time it is used. It should ONLY be used in cases where there is absolute trust that it is the correct solution! Otherwise we could end up connecting to a bad actor's machine.

Add information about `sloss` to project spaces

The /data/project/sloss space is for small projects (5TB unreplicated). Small meaning in terms of data and/or time span. We need a section on this space in storage.md (short blurb there), and in support.md, and we need to advertise it in relevant places in the docs.

Add lightgallery functionality to all images

Two things on this issue:

  1. Use ![!alt text](image.png) to use the lightgallery. Notice the ! character INSIDE the brackets. That's needed to make the gallery work. ALL images now and forever should have that there.

  2. Add this info to the readme in #23 or related issue resolution.

Partition QoS table missing

We have no public-facing information on our partition QoS limits (cpus, memory, gpus). We need a table. I'll work on adding that to my slurm-status-tools code and get that up and running in the near future.

Update documentation links on other platforms

Update to include new documentation links:

  • UAB RC website "We're here to help you"
  • rc.uab.edu message of the day in OOD (code here: https://github.com/wwarriner/slurm_status_tools)
    -- Include spaces for the table to separate columns correctly (uabrc/CRI_XCBC#85)
    -- Include number of GPUs on partitions
    image
    -- Add CPUs per node and/or remove nodes (leave nodes per user)
    -- Usable memory per node
  • Terminal Message of the Day
    -- Get access to that
  • Update Cheaha and Cloud registration messages with new documentation and table for partitions

Need docs on folder/file permissions "gotchas".

There are a few "gotchas" for permissions in shared spaces.

Copying folders with typical flags will copy all of the permissions as they are. Need to research: https://man7.org/linux/man-pages/man1/cp.1.html

Users creating files/folders under a project can also create self-owned files that can't be modified by the parent folder owner. Not 100% sure what to do about this one, we may need Clyde's help understanding this one.

Add clarifying language to user registration/sign in pages

Add something like the following to any user reg or sign in pages about our services.

"Please note you are logging into a UAB campus service which only supports BlazerID and XIAS credentials. Please remember that you must use your BlazerID/XIAS credentials for authentication and not other identities you may have, for example a uabmc account."

Extra {:, center} element on some images

There are some pages whose images have a {:, center} markup on them. Is that leftover from sphinx? Or is that valid markdown of some sort that VSCode doesn't understand?

Tables have nans instead of empty cells

The table importer fills blanks with "nan". Doesn't appear to be an obvious solution, and the plugin has no config for it. We may need to investigate deeper.

Clarify GitLab login/registration procedures

Here is the order of operations for a blazerid user getting access:

Use blazerid credentials to login at https://gitlab.rc.uab.edu/ on the LDAP tab.

For a XIAS user:

Sponsor must add "https://gitlab.rc.uab.edu/" to the URIs needing access on the XIAS site
The XIAS user should send us an email at [[email protected]](mailto:[email protected]) requesting access to gitlab.
Once granted, login at the site with XIAS credentials (the part of their XIAS email before "@") on the "Standard" tab

UAB Cloud - Creating a Subnet

From @Premas in Slack

I had this issue while setting up my cloud account, "Failed to create subnet 192.168.0.0/24 for my network. Invalid input for operation: Gateway is not valid on a subnet."

Changing the Gateway IP as "192.168.0.1" fixed this issue and was able to successfully create the subnet.

We need to update the Cloud setup doc to add clarification to this issue in the section, "Creating the subnet".

152419481-0d00cc77-1ddf-41ed-827a-9da62edf83e1

Consistent naming scheme

Right now its frustrating to compare changes to files on my dev pages instance because the article titles, their name in the TOC, and their filename all can differ. These names should all be as consistent as possible. This is a priority issue because the longer we wait the harder it will be to fix.

Landing Page

Landing page for the docs should have links for office hours as well as the support email listed. Something like the section in the middle of the uab.edu research computing page where that information is listed would be nice

Branding

ReadTheDocs had a much larger space above the Navigation menu for branding, so the current logo there fit fine and was readable. This is not the case for MkDocs. When testing out the logo feature, the image was shrunk down to an unreadable size. Need to find a smaller, more minimalistic logo if we want branding at all.

Hide unfixable linting warnings

Related to pull request #25, some linting issues cannot be fixed. These are often related to the admonition syntax in conjunction with other surrounding entities that are not basic text as well as missing alt text in images. These linting warnings can be fixed on a per-line or per-file basis. Fixing on a whole-project basis I believe is possible, but I cannot find the documentation for it immediately.

To fix on a per-line basis:

<!-- markdownlint-disable <lint warning code> -->
deliberate space * in * emphasis
<!-- markdownlint-enable <lint warning code> -->

or

<!-- markdownlint-disable-next-line -->

To fix on a per-file basis, add the following to the top of the file:

<!-- markdownlint-disable-file <lint warning code> -->

Add Granting Resources docs

  • UAB CI Plan and Facilities Doc
  • Budgeting information for buy-in programs
  • Link to services catalog/digital marketplace/choose your own RC adventure

Modify Anaconda docs

I am a firm proponent in that the right way to do things is to make them version-controllable, even if you aren't using version control (and why aren't you??). Because then you get shareable and repeatable for free, and those are cornerstones of scientific research.

With that said, I think our page at https://uabrc.readthedocs.io/en/latest/resources/conda.html should be .yml-centric. It should make conda install feel ad-hoc, or like part of an experimental process, and adding packages to a .yml file feel natural and "right".

Maybe some of our users aren't there yet, but it's our job to get them there.

Automagic review of URLs

Automatic review of URLs within the docs to update if they are broken
Talk with Louis about continuous integration
Output to log or other file descriptor as necessary instead of just a bare print() call.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.