SLURM sacct constructor
Simple tool for constructing sacct command to run various accounting reports in the format expected by our business office. Its main goal is to calculate the last day of a given month, and automating the task of running reports for a previous month.
- Defaults are calculated to produce last month's accounting
- Defaults are shown in
--help - Business office output (uses specific queue & filename combinations)
- Automatically calculate last day of the month
- Execute constructed commands
- Python 2.7 & 3 compatible, using commonly available modules
-
Show how to run last month's aggregate report for all users and clusters:
./slurm_accts.pyoutput:
sacct -XTp -a -L -S 2018-08-01 -E 2018-08-31T23:59:59 -o JobID,User,Account,cluster,Partition,NodeList,NNodes,AllocCPUS,start,end,CPUTime,CPUTimeRAW,Elapsed,ElapsedRaw > ./results/2018-08-HPC-slurm-all.txt -
Run last month's aggregate report for all users and clusters, split into multiple files for business office:
./slurm_accts.py -b -x -
Run reports from beginning of February until June 15th, and save them in a filename with 'custom' suffix
./slurm_accts.py -sm 2 -em 6 -ed 15 -s custom -x
slurm_acct defaults to the following:
- time range is first day through last day of a previous month
- there is no limit to which clusters, partitions or users areused
usage: slurm_accts.py [-h] [-sd STARTDAY] [-sm STARTMONTH] [-sy STARTYEAR]
[-ed ENDDAY] [-em ENDMONTH] [-ey ENDYEAR] [-a ACCOUNT]
[-b] [-c CLUSTER] [-d] [-f FIELDS] [-o OUTPUT]
[-p PARTITION] [-r RESULTDIR] [-s SUFFIX] [-u USER] [-x]
Simple tool to construct sacct command to output accounting data in a
predefined format. Default date range is: 2018/8/1-31.
optional arguments:
-h, --help show this help message and exit
-sd STARTDAY, --startday STARTDAY
accounting start day (default: 1)
-sm STARTMONTH, --startmonth STARTMONTH
accounting start month (default: 8)
-sy STARTYEAR, --startyear STARTYEAR
accounting start year (default: 2018)
-ed ENDDAY, --endday ENDDAY
accounting end day (default: 31)
-em ENDMONTH, --endmonth ENDMONTH
accounting end month (default: 8)
-ey ENDYEAR, --endyear ENDYEAR
accounting end year (default: 2018)
-a ACCOUNT, --account ACCOUNT
limit query to specific account(s) (default: None)
-b, --business business office setup (default: False)
-c CLUSTER, --cluster CLUSTER
limit query to specific cluster(s) (default: None)
-d, --debug enable debug logging (default: False)
-f FIELDS, --fields FIELDS
accounting fields (default: JobID,User,Account,cluster
,Partition,NodeList,NNodes,AllocCPUS,start,end,CPUTime
,CPUTimeRAW,Elapsed,ElapsedRaw)
-o OUTPUT, --output OUTPUT
-p PARTITION, --partition PARTITION
limit query to specific partition(s) (default: None)
-r RESULTDIR, --resultdir RESULTDIR
destination directory for results (default: ./results)
-s SUFFIX, --suffix SUFFIX
custom filename suffix (default: all)
-u USER, --user USER limit query to specific user(s) (default: None)
-x, --execute execute constructed command (default: False)
For the purpose of our business office we have to filter sacct based on combinations of various partitions and save them to various files. This is stored as a key/value dictionary inside the script. In this example we have gpu_q partition being saved to a file with 'gpu' in its name, while all three pegasus_q, discovery_q and haswell_q get stored in a file with 'std' suffix.
# business output: partitions + file suffixes
BUSINESS_OUTPUT = {'pegasus_q,discovery_q,haswell_q': 'std',
'smp_q': 'smp',
'gpu_q': 'gpu',
'orion_q': 'orion',
None: 'all'
}
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent