nsls-ii-cms / profile_collection Goto Github PK

Previsit

• Make sure that the current latest environments are pushed to all the bl machines
• Skim the IPython startup files in profile_collection
  • Check if it's safe to start bsui remotely (does it touch any hardware?) - use git grep .put or git grep caput
• Check for outstanding PRs and issues on beamline repos
• Check if beamline specific packages pushed to the BL machines
• (Optional) Enable CI

Housekeeping

• Add BL staff to the BL GitHub organization as owners
• Work with BL staff to commit any un-committed changes to their profiles
• Tag the profile as-found as 2019C3.x
• Discuss with BL staff which conda envs they want to keep/delete and perform the cleanup
• Check/update the beamline's inventory with the BL staff (in https://github.com/NSLS-II/playbooks/blob/master/production)

Test deployment

• In ~/.bashrc add the following and remember to source ~/.bashrc afterward.

  export BLUESKY_LOG_FILE=/var/log/bluesky/bluesky.log
  export BLUESKY_IPYTHON_LOG_FILE=/var/log/bluesky/bluesky_ipython.log
  

  and (if needed) create those directories, making them world-read/write-able.
• Append to 00-startup.py https://gist.github.com/jklynch/a4366b8900ec0c03883403455ae711b2
• Run profile against new environment
  • BS_ENV=collection-2021-1.0
• Update the profile as needed to run:
  • potential issue with images in databroker V2
• Run acceptance tests (if any) and run a representative set of scans for the beamline
• Tag the updated profile as 2021C1.0
• Update BSUI via ansible to use the new environment (http://nsls-ii.github.io/deployment/upgrading-beamlines.html#change-bsui-s-default-environment)

Previsit

• Make sure that the current latest environments are pushed to all the bl machines
• Skim the IPython startup files in profile_collection
• Check if it's safe to start bsui remotely (does it touch any hardware?) - use git grep .put or git grep caput
• Check for outstanding PRs and issues on beamline repos
• Check if beamline specific packages pushed to the BL machines
• (Optional) Enable CI

Housekeeping

• Add BL staff to the BL GitHub organization as owners
• Work with BL staff to commit any un-committed changes to their profiles
• Tag the profile as-found as 2019C3.x
• Discuss with BL staff which conda envs they want to keep/delete and perform the cleanup
• Check/update the beamline's inventory with the BL staff (in https://github.com/NSLS-II/playbooks/blob/master/production)

Test deployment

• In ~/.bashrc add the following and remember to source ~/.bashrc afterward.

  BLUESKY_LOG_FILE=/var/log/bluesky.log
  BLUESKY_IPYTHON_LOG_FILE=/var/log/bluesky_ipython.log
  

  and (if needed) create those directories, making them world-read/write-able.
• Append to 00-startup.py https://gist.github.com/jklynch/a4366b8900ec0c03883403455ae711b2
• Run profile against new environment
  • BS_ENV=collection-2020-2.0rc7 bsui
• Update the profile as needed to run:
  • Replace .value -> .get() if it's in a Device, or with yield from bps.read() if in a plan
  • history.sqlite issue - see https://blueskyproject.io/bluesky/metadata.html#persistence-between-sessions
  • potential issue with images in databroker V2
• Run acceptance tests (if any) and run a representative set of scans for the beamline
• Tag the updated profile as 2020C2.0
• Update BSUI via ansible to use the new environment (http://nsls-ii.github.io/deployment/upgrading-beamlines.html#change-bsui-s-default-environment)

Current collection workstations

Previsit

• Enable Azure-Pipelines CI
• Make sure that the current latest environments are pushed to all the bl machines
• Skim the IPython startup files in profile_collection
  • Check if it's safe to start bsui remotely (does it touch any hardware?) - use git grep .put or git grep caput
• Check for outstanding PRs and issues on beamline repos
• Branch https://github.com/NSLS-II/playbooks for the purpose of updating the CMS current_env_tag in the production file (create new pull request for CMS)

Housekeeping

• Update /opt/conda/.condarc and conda with ansible

  $ ansible-playbook -i production install-base-conda.yml --limit=CMS -kK -vvvv

• Add BL staff to the BL GitHub organization as owners
• Work with BL staff to commit any un-committed changes to their profiles
• Tag the profile_collection repository as-found as 2020C2.1 and push to GitHub

  $ git tag 2020C2.1 -m "Before 2021C1.0 deployment"

  Note: tagged https://github.com/NSLS-II-CMS/profile_collection/releases/tag/2020C2.1 with the changes from #8 (that code in the PR branch was used during the previous cycle and was in fact their 'master'). The follow-up updates will be in a separate PR.
• Discuss with BL staff which conda envs they want to keep/delete and perform the cleanup
• Check/update the beamline's workstation inventory with the BL staff (in https://github.com/NSLS-II/playbooks/blob/master/production)
• Ask beamline staff about the beamline unix group they want to use for bluesky logs. The members of the group can be checked with the following command:

  $ getent group <beamline-group>

  Add this group name to the collection vars in the inventory file (see this example).
• In ~/.bashrc, if necessary, update the logging environment variables to use directory /var/log/bluesky/... and add umask 0002.

  umask u=rwx,g=rwx,o=rx  # 0002
  export BLUESKY_LOG_FILE=/var/log/bluesky/bluesky.log
  export BLUESKY_IPYTHON_LOG_FILE=/var/log/bluesky/bluesky_ipython.log

  and (if needed) create that directory with the following permissions and ownership:

  # sudo mkdir /var/log/bluesky
  # sudo chown -Rv xf11bm:<beamline-group> /var/log/bluesky
  # sudo chmod -Rv g+rws /var/log/bluesky

  Remember to source ~/.bashrc.
• Remove explicit setting of ophyd logging level from the first startup script
• Double-check that if there is an open PR removing handlers, it is merged (and tested).
• Update vendored copy of PersistentDict to bug-fixed version (see the updated gist snippet: https://gist.github.com/jklynch/a4366b8900ec0c03883403455ae711b2).
• Make sure all repos have the BSD-3-Clause licenses (see NSLS-II-SRX/profile_collection#16 as an example)

Test deployment

• Run profile against new environment

  $ BS_ENV=collection-2021-1.0 bsui

  It may be necessary to disable jedi in ipython. Add this to the ipython command line arguments inbsui:

  --IPCompleter.use_jedi=False

• Update the profile as needed to run
• Test databroker v2

  from databroker import catalog
  cat = catalog['cms']
  cat[-1].primary.read()

• Run acceptance tests (if any) and run a representative set of scans for the beamline

Finish

• Tag the updated profile as 2021C1.0

  $ git tag 2021C1.0 -m "After 2021C1.0 deployment"

• Open and merge a PR against https://github.com/NSLS-II/playbooks setting current_env_tag="2021-1.0" for the CMS beamline
• Update bsui via ansible to use the new environment (http://nsls-ii.github.io/deployment/upgrading-beamlines.html#change-bsui-s-default-environment)