d-hank / speccpu2017 Goto Github PK
View Code? Open in Web Editor NEWThis project forked from bowers/speccpu2017
CloudHarmony benchmark harness for SPEC CPU2017
License: Apache License 2.0
This project forked from bowers/speccpu2017
CloudHarmony benchmark harness for SPEC CPU2017
License: Apache License 2.0
SPEC CPU 2017 v1.0 SPEC CPU(R) 2017 is designed to provide a comparative measure of compute-intensive performance across the widest practical range of hardware using workloads developed from real user applications. Metrics for both integer and floating point compute intensive performance are provided, with both speed and rate metrics for each. Full documentation is available on the SPEC website: http://www.spec.org/cpu2017/. In order to use this benchmark, SPEC CPU must be installed and the [spec_dir]/config directory must be writable by the benchmark user. The runtime parameters defined below essentially determine the 'runcpu' arguments. SPEC CPU2017 consists of a total of 43 individual benchmarks divided into 4 suites: SPECspeed(R)2017 Integer, SPECspeed(R)2017 Floating Point, SPECrate(R)2017 Integer, and SPECRate(R)2017 Floating point. Aggregate scores are calculated for each suite by a geomean of the median results of the individual benchmarks in that suite. TESTING PARAMETERS * benchmark the benchmark(s) to run - any of the benchmark identifiers listed in config/spec-benchmarks.ini may be specified. This argument can be repeated to designate multiple benchmarks. You may specify 'all' for all benchmarks, 'intrate' for all Integer Rate benchmarks, 'intspeed' for all Integer Speed benchmarks, 'fprate' for all floating point rate benchmarks, 'fpspeed' for all floating point speed benchmarks, or the name of a specific benchmark. Benchmarks may be referenced either by their numeric or full identifier (e.g. --benchmark=500 or --benchmark=500.perlbench_r). Additionally, you may designate benchmarks that should be removed by prefixing them with a minus character (e.g. --benchmark=all --benchmark=-631). May also be specified using a single space or comma separated value (e.g. --benchmark "all -631") DEFAULT: all * collectd_rrd If set, collectd rrd stats will be captured from --collectd_rrd_dir. To do so, when testing starts, existing directories in --collectd_rrd_dir will be renamed to .bak, and upon test completion any directories not ending in .bak will be zipped and saved along with other test artifacts (as collectd-rrd.zip). User MUST have sudo privileges to use this option * collectd_rrd_dir Location where collectd rrd files are stored - default is /var/lib/collectd/rrd * comment optional comment to add to the log file DEFAULT: none * config name of a configuration file in [spec_dir]/config to use for the run. The following macros will be automatically set via the --define argument capability of runspec (optional parameters will only be present if specified by the user): cpu_cache: level 2 cpu cache (e.g. 4096 KB) cpu_count: the number of CPU cores present cpu_family: numeric CPU family identifier cpu_model: numeric CPU model identifier cpu_speed: the nominal CPU speed in MHz (e.g. 2933.436) cpu_speed_max: the maximum turbo CPU speed in MHz (e.g. 3500.0) cpu_vendor: the CPU vendor (e.g. GenuineIntel) compute_service_id: the compute service ID external_id: an external identifier for the compute resource instance_id: identifier for the compute resource under test (e.g. m1.xlarge) ip_or_hostname: IP or hostname of the compute resource is32bit: set if the OS is 32 bit is64bit: set if the OS is 64 bit iteration_num: the test iteration number (e.g. 2) meta_*: any of the meta parameters listed below label: user defined label for the compute resource location: location of the compute resource (e.g. CA, US) memory_free: free memory in KB memory_total: total memory in KB numa: set only if the system under test has numa support os: the operating system name (e.g. centos) os_version: the operating system version (e.g. 6.2) provider_id: the provider identifier (e.g. aws) region: compute resource region identifier (e.g. us-west) run_id: the benchmark run ID run_name: the name of the run (if assigned by the user) sse: the highest SSE flag supported storage_config: storage config identifier (e.g. ebs, ephemeral) subregion: compute resource subregion identifier (e.g. 1a) test_id: a user defined test identifier x64: set if the x64 parameter is also set if this parameter value identifies a remote file (either an absolute or relative path on the compute resource, or an external reference like http://...) that file will be automatically copied into the [spec_dir]/config directory - if not specified, a default.cfg file should be present in the config directory DEFAULT: none * copies the number of copies to run concurrently for a rate benchmark. A higher number of copies will generally produce a better score (subject to resource availability for those copies to run). This parameter value may be one of the following: cpu relative: a percentage relative to the number of CPU cores present. For example, if copies=50% and the compute instance has 4 cores, 2 copies will be run - standard rounding will be used fixed: a simple numeric value representing the number of copies to run (e.g. copies=2) memory relative: a memory to copies size ratio. For example, if copies=2GB and the compute instance has 16GB of memory, then 8 copies will be run standard rounding will be used. Either MB or GB suffix may be used mixed: a combination of the above 3 types may be used, each value separated by a forward slash /. For example, if copies=100%/2GB, then the number of copies will be the lesser of either the number of CPU cores or memory/2GB. Alternatively, if this value is prefixed by a +, the greater of the values will be used (e.g. copies=+100%/2GB) The general recommend ratio of copies to resources is 1.5 GB of memory for 64 bit binaries, 1 GB of memory for 32 bit binaries, 1 CPU core and 1 GB of free disk space. To specify a different number of copies for 32-bit binaries versus 64-bit binaries (based on the value of the x64 parameter defined below), separate the values with a pipe, and prefix the 64-bit specified value with x64: (e.g. copies="x64:100%/2GB|100%/1GB") DEFAULT: x64:100%/1.5GB|100%/1GB (NULL for speed runs) * define_* additional macros to define using the runspec --define capability (these will be accessible in the config file using the format %{macro_name}) - any number of defines may be specified. Conditional logic within the config file is supported using the format: %ifdef %{macro_name} # do something %else # do something else %endif More information is available about the use of macros on the SPEC website here: http://www.spec.org/cpu2006/Docs/config.html#sectionI.D.2 For flags - do not set a value for this parameter (e.g. -p define_smt translates to --define smt) DEFAULT: none * delay Add a delay of the specified number of seconds before and after each benchmark. The delay is not counted toward the benchmark runtime. DEFAULT: none * failover_no_sse When set to 1 in combination with an sse parameter benchmark execution will be re-attempted without sse if runspec execution with sse results in an error status code (runspec will be restarted without the sse macro set) DEFAULT: 0 * flagsurl Path to a flags file to use for the run - A flags file provides information about how to interpret and report on flags (e.g. -O5, -fast, etc.) that are used in a config file. The flagsurl may be an absolute or relative path in the file system, or refer to an http accessible file (e.g. $[top]/config/flags/Intel-ic17.0-official-linux64-revE.xml) Alternatively, flagsurl can be defined in the config file DEFAULT: none * huge_pages Whether or not to enable huge pages if supported by the OS. To do so, prior to runspec execution, if the file /usr/lib64/libhugetlbfs.so or /usr/lib/libhugetlbfs.so exists, it then checks that free huge pages are available in /proc/meminfo and if these conditions are met, sets the following environment variables: export HUGETLB_MORECORE=yes export LD_PRELOAD=/usr/lib/libhugetlbfs.so Note: In order to use huge pages, you must enable them first using something along the lines of: # first clear out existing huge pages echo 0 > /proc/sys/vm/nr_hugepages # create 500 2MB huge pages (1GB total) - 2MB is # the default huge page size on RHEL6 echo 500 > /proc/sys/vm/nr_hugepages # mount the huge pages mkdir -p /libhugetlbfs mount -t hugetlbfs hugetlbfs /libhugetlbfs Note: CentOS 6+ supports transparent huge pages (THP) by default. This parameter will likely have little effect on systems where THP is already enabled DEFAULT: 0 * ignore_errors whether or not to ignore errors - if 0, benchmark execution will stop if any errors occur DEFAULT: 0 * iterations How many times to run each benchmark. This parameter should only be changed if reportable=0 because reportable runs must use 2 or 3 iterations DEFAULT: 3 (not used if reportable=1) * max_copies May be used in conjunction with dynamic copies calculation (see copies parameter above) in order to set a hard limit on the number of copies DEFAULT: none (no limit) * nobuild If 1, don't build new binaries if they do not already exist DEFAULT: 1 * nocleanup Do not delete test files generated by SPEC (i.e. [spec]/benchspec/CPU2017/[benchmark]/run/*) DEFAULT: 0 * nonuma Do not set the 'numa' macro or invoke using 'numactl --interleave=all' even if numa is supported DEFAULT: 0 * nosse_macro Optional macro to define for --sse optimal if no SSE flag will be set * output The output directory to use for writing test artifacts. If not specified, the current working directory will be used * purge_output Whether or not to remote run files (created in the [spec_dir]/benchspec/CPU2017/*/run/ directories) following benchmarking completion DEFAULT: 1 * reportable whether or not to designate the run as reportable, only int, fp or all benchmarks can be designated as reportable. Per the official documentation: A reportable execution runs all the benchmarks in a suite with the test and train data sets as an additional verification that the benchmark binaries get correct results. The test and train workloads are not timed. Then, the reference workloads are run three times, so that median run time can be determined for each benchmark. DEFAULT: 0 * review Format results for review, meaning that additional detail will be printed that normally would not be present DEFAULT: 0 * run_timeout The amount of time to allow each test iteration to run DEFAULT: 72 hours * size Size of the input data to run: test, train or ref DEFAULT: ref * spec_dir Directory where SPEC CPU 2017 is installed. If not specified, the benchmark run script will look up the directory tree from both pwd and --output for presence of a 'cpu2017'. If this fails, it will check '/opt/cpu2017' * sse Run with a specific SSE optimization flag - if not specified, the most optimal SSE flag will be used for the processor in use. The options availabe for this parameter are: optimal: choose the most optimal flag none: do not use SSE optimizations CORE-AVX512: AVX-512 instructions CORE-AVX2: AVX2 instructions AVX: AVX instructions SSE4.2: SSE4.2 instructions SSE4.1: SSE4.1 instructions SSSE3: SSSE3 instructions SSE3: SSE3 instructions SSE2: SSE2 instructions SSE: SSE instructions DEFAULT: optimal * sse_max The max SSE flag to support in conjunction with sse=optimal - if a processor supports greater than this SSE level, sse_max will be used instead DEFAULT: CORE-AVX2 * sse_min The minimum SSE flag to support in conjunction with sse=optimal - if a processor does not at least support this SSE level sse optimization will not be used DEFAULT: SSE4.2 * sse_skip Use in conjunction with sse=optimal to skip a specific SSE flag * tune Tuning option: base, peak or all - reportable runs must be either base or all DEFAULT: base * validate_disk_space Whether or not to validate if there is sufficient diskspace available for a run - this calculation is based on a minimum requirement of 1.5GB per copy If this space is not available, the run will fail DEFAULT: 1 * verbose Show verbose output * x64 Optional parameter that will be passed into runspec using the macro --define x64 - this may be used to designate that a run utilize 32-bit versus 64-bit binaries - this parameter can also affect the dynamic calculation of the 'copies' parameter described above. Valid options are 0, 1 or 2 DEFAULT: 2 (64-bit binaries for 64-bit systems, 32-bit otherwise) * x64_failover This flag will cause testing to be re-attempted for the opposite x64 flag if current testing fails (e.g. if initial testing is x64=1 and it fails, then testing will be re-attempted with x64=0). When used in conjunction with failover_no_sse, sse failover will take precedence followed by x64 failover DEFAULT: 0 META PARAMETERS If set, these parameters will be included in the results generated using save.sh. Additionally, the parameters with a * suffix can be used to change the values in the SPEC CPU 2017 config file using macros. When specified, each of these parameters will be passed in to runcpu using --define [parameter_name]=[parameter_value] and will then be accessible in the config using macros %{parameter_name} * meta_burst If set to 1, designates testing performed in burst mode (e.g. Amazon EC2 t-series burst) * meta_compiler Details about the compiler (e.g. "Intel v16") * meta_compute_service The name of the compute service this test pertains to. May also be specified using the environment variable bm_compute_service * meta_compute_service_id The id of the compute service this test pertains to. Added to saved results. May also be specified using the environment variable bm_compute_service_id * meta_cpu CPU descriptor - if not specified, it will be set using the 'model name' attribute in /proc/cpuinfo * meta_fw_bios* Customer-orderable name and version of the system firmware, sometimes called BIOS. * meta_instance_id The compute service instance type this test pertains to (e.g. c3.xlarge). May also be specified using the environment variable bm_instance_id * meta_hw_avail* Date that this hardware or instance type was made available * meta_hw_nthreadspercore* Number of hardware threads per core - DEFAULT 1 * meta_hw_other* Any other relevant information about the instance type * meta_hw_ocache* Other hardware primary cache * meta_hw_pcache* Hardware primary cache * meta_hw_tcache* Hardware tertiary cache * meta_hw_ncpuorder* Valid number of processors orderable for this model, including a unit. (e.g. "2, 4, 6, or 8 chips" * meta_license_num* The SPEC CPU 2017 license number * meta_memory Memory descriptor - if not specified, the system memory size will be used * meta_notes_N* General notes - all of the meta_notes_* parameters support up to 5 entries (N=1-5) * meta_notes_base_N* Notes about base optimization options * meta_notes_comp_N* Notes about compiler invocation * meta_notes_os_N* Notes about operating system tuning and changes * meta_notes_part_N* Notes about component parts (for kit-built systems) * meta_notes_peak_N* Notes about peak optimization options * meta_notes_plat_N* Notes about platform tuning and changes * meta_notes_port_N* Notes about portability options * meta_notes_submit_N* Notes about use of the submit option * meta_os Operating system descriptor - if not specified, it will be taken from the first line of /etc/issue * meta_provider The name of the cloud provider this test pertains to. May also be specified using the environment variable bm_provider * meta_provider_id The id of the cloud provider this test pertains to. May also be specified using the environment variable bm_provider_id * meta_region The compute service region this test pertains to. May also be specified using the environment variable bm_region * meta_resource_id An optional benchmark resource identifiers. May also be specified using the environment variable bm_resource_id * meta_run_id An optional benchmark run identifiers. May also be specified using the environment variable bm_run_id * meta_storage_config Storage configuration descriptor. May also be specified using the environment variable bm_storage_config * meta_sw_avail* Date that the OS image was made available * meta_sw_other* Any other relevant information about the software * meta_test_id Identifier for the test. May also be specified using the environment variable bm_test_id DEPENDENCIES This benchmark has the following dependencies: SPEC CPU 2017 This benchmark is licensed by spec.org. To use this benchmark harness you must have it installed and available in the 'spec_dir' directory perl Used by SPEC CPU 2017 php-cli Test automation scripts (/usr/bin/php) zip Used to compress test artifacts TEST ARTIFACTS This benchmark generates the following artifacts: collectd-rrd.zip collectd RRD files (see --collectd_rrd) specrate2017_int.csv SPECint rate test results in CSV format specrate2017_int.html HTML formatted SPECint rate test report specrate2017_int.pdf PDF formatted SPECint rate test report specrate2017_int.txt Text formatted SPECint rate test report specrate2017_fp.csv SPECfp rate test results in CSV format specrate2017_fp.html HTML formatted SPECfp rate test report specrate2017_fp.pdf PDF formatted SPECfp rate test report specrate2017_fp.txt Text formatted SPECfp rate test report specspeed2017_int.csv SPECint speed test results in CSV format specspeed2017_int.html HTML formatted SPECint speed test report specspeed2017_int.pdf PDF formatted SPECint speed test report specspeed2017_int.txt Text formatted SPECint speed test report specspeed2017_fp.csv SPECfp speed test results in CSV format specspeed2017_fp.html HTML formatted SPECfp speed test report specspeed2017_fp.pdf PDF formatted SPECfp speed test report specspeed2017_fp.txt Text formatted SPECfp speed test report SAVE SCHEMA The following columns are included in CSV files/tables generated by save.sh. Indexed MySQL/PostgreSQL columns are identified by *. Columns without descriptions are documented as runtime parameters above. Data types and indexing used are documented in save/schema/speccpu2017.json. Columns can be removed using the save.sh --remove parameter # Individual benchmark metrics. These provide the selected, min, max and # (sample) standard deviation metrics for each benchmark as well as runtime and # reftime (reftime only included for speed runs: rate=0). For rate runs # (i.e. rate=1) the metrics represent the 'rate' metric - signifying that # multiple copies of the benchmark were run in parallel (i.e. --copies > 1). # Rate metrics essentially represent throughput. For speed runs these columns # contain a 'ratio' metric derived from ([base_run_time]/[ref_time]). For speed # runs, only 1 copy of the benchmark is run. These columns may be excluded using # --remove benchmark_4* 500_perlbench_r 500_perlbench_r_max 500_perlbench_r_min 500_perlbench_r_reftime 500_perlbench_r_runtime 500_perlbench_r_stdev 502_gcc_r 502_gcc_r_max 502_gcc_r_min 502_gcc_r_reftime 502_gcc_r_runtime 502_gcc_r_stdev 503_bwaves_r 503_bwaves_r_max 503_bwaves_r_min 503_bwaves_r_reftime 503_bwaves_r_runtime 503_bwaves_r_stdev 505_mcf_r 505_mcf_r_max 505_mcf_r_min 505_mcf_r_reftime 505_mcf_r_runtime 505_mcf_r_stdev 507_cactuBSSN_r 507_cactuBSSN_r_max 507_cactuBSSN_r_min 507_cactuBSSN_r_reftime 507_cactuBSSN_r_runtime 507_cactuBSSN_r_stdev 508_namd_r 508_namd_r_max 508_namd_r_min 508_namd_r_reftime 508_namd_r_runtime 508_namd_r_stdev 510_parest_r 510_parest_r_max 510_parest_r_min 510_parest_r_reftime 510_parest_r_runtime 510_parest_r_stdev 511_povray_r 511_povray_r_max 511_povray_r_min 511_povray_r_reftime 511_povray_r_runtime 511_povray_r_stdev 519_lbm_r 519_lbm_r_max 519_lbm_r_min 519_lbm_r_reftime 519_lbm_r_runtime 519_lbm_r_stdev 520_omnetpp_r 520_omnetpp_r_max 520_omnetpp_r_min 520_omnetpp_r_reftime 520_omnetpp_r_runtime 520_omnetpp_r_stdev 521_wrf_r 521_wrf_r_max 521_wrf_r_min 521_wrf_r_reftime 521_wrf_r_runtime 521_wrf_r_stdev 523_xalancbmk_r 523_xalancbmk_r_max 523_xalancbmk_r_min 523_xalancbmk_r_reftime 523_xalancbmk_r_runtime 523_xalancbmk_r_stdev 525_x264_r 525_x264_r_max 525_x264_r_min 525_x264_r_reftime 525_x264_r_runtime 525_x264_r_stdev 526_blender_r 526_blender_r_max 526_blender_r_min 526_blender_r_reftime 526_blender_r_runtime 526_blender_r_stdev 527_cam4_r 527_cam4_r_max 527_cam4_r_min 527_cam4_r_reftime 527_cam4_r_runtime 527_cam4_r_stdev 531_deepsjeng_r 531_deepsjeng_r_max 531_deepsjeng_r_min 531_deepsjeng_r_reftime 531_deepsjeng_r_runtime 531_deepsjeng_r_stdev 538_imagick_r 538_imagick_r_max 538_imagick_r_min 538_imagick_r_reftime 538_imagick_r_runtime 538_imagick_r_stdev 541_leela_r 541_leela_r _max 541_leela_r _min 541_leela_r _reftime 541_leela_r _runtime 541_leela_r _stdev 544_nab_r 544_nab_r_max 544_nab_r_min 544_nab_r_reftime 544_nab_r_runtime 544_nab_r_stdev 548_exchange2_r 548_exchange2_r_max 548_exchange2_r_min 548_exchange2_r_reftime 548_exchange2_r_runtime 548_exchange2_r_stdev 549_fotonik3d_r 549_fotonik3d_r_max 549_fotonik3d_r_min 549_fotonik3d_r_reftime 549_fotonik3d_r_runtime 549_fotonik3d_r_stdev 554_roms_r 554_roms_r_max 554_roms_r_min 554_roms_r_reftime 554_roms_r_runtime 554_roms_r_stdev 557_xz_r 557_xz_r_max 557_xz_r_min 557_xz_r_reftime 557_xz_r_runtime 557_xz_r_stdev 600_perlbench_s 600_perlbench_s_max 600_perlbench_s_min 600_perlbench_s_reftime 600_perlbench_s_runtime 600_perlbench_s_stdev 602_gcc_s 602_gcc_s_max 602_gcc_s_min 602_gcc_s_reftime 602_gcc_s_runtime 602_gcc_s_stdev 603_bwaves_s 603_bwaves_s_max 603_bwaves_s_min 603_bwaves_s_reftime 603_bwaves_s_runtime 603_bwaves_s_stdev 605_mcf_s 605_mcf_s_max 605_mcf_s_min 605_mcf_s_reftime 605_mcf_s_runtime 605_mcf_s_stdev 607_cactuBSSN_s 607_cactuBSSN_s_max 607_cactuBSSN_s_min 607_cactuBSSN_s_reftime 607_cactuBSSN_s_runtime 607_cactuBSSN_s_stdev 619_lbm_s 619_lbm_s_max 619_lbm_s_min 619_lbm_s_reftime 619_lbm_s_runtime 619_lbm_s_stdev 620_omnetpp_s 620_omnetpp_s_max 620_omnetpp_s_min 620_omnetpp_s_reftime 620_omnetpp_s_runtime 620_omnetpp_s_stdev 621_wrf_s 621_wrf_s_max 621_wrf_s_min 621_wrf_s_reftime 621_wrf_s_runtime 621_wrf_s_stdev 623_xalancbmk_s 623_xalancbmk_s_max 623_xalancbmk_s_min 623_xalancbmk_s_reftime 623_xalancbmk_s_runtime 623_xalancbmk_s_stdev 625_x264_s 625_x264_s_max 625_x264_s_min 625_x264_s_reftime 625_x264_s_runtime 625_x264_s_stdev 627_cam4_s 627_cam4_s_max 627_cam4_s_min 627_cam4_s_reftime 627_cam4_s_runtime 627_cam4_s_stdev 628_pop2_s 628_pop2_s_max 628_pop2_s_min 628_pop2_s_reftime 628_pop2_s_runtime 628_pop2_s_stdev 631_deepsjeng_s 631_deepsjeng_s_max 631_deepsjeng_s_min 631_deepsjeng_s_reftime 631_deepsjeng_s_runtime 631_deepsjeng_s_stdev 638_imagick_s 638_imagick_s_max 638_imagick_s_min 638_imagick_s_reftime 638_imagick_s_runtime 638_imagick_s_stdev 641_leela_s 641_leela_s_max 641_leela_s_min 641_leela_s_reftime 641_leela_s_runtime 641_leela_s_stdev 644_nab_s 644_nab_s_max 644_nab_s_min 644_nab_s_reftime 644_nab_s_runtime 644_nab_s_stdev 648_exchange2_s 648_exchange2_s_max 648_exchange2_s_min 648_exchange2_s_reftime 648_exchange2_s_runtime 648_exchange2_s_stdev 649_fotonik3d_s 649_fotonik3d_s_max 649_fotonik3d_s_min 649_fotonik3d_s_reftime 649_fotonik3d_s_runtime 649_fotonik3d_s_stdev 654_roms_s 654_roms_s_max 654_roms_s_min 654_roms_s_reftime 654_roms_s_runtime 654_roms_s_stdev 657_xz_s 657_xz_s_max 657_xz_s_min 657_xz_s_reftime 657_xz_s_runtime 657_xz_s_stdev benchmark_version: [benchmark version] benchmarks: pipe separated names of benchmarks run - or int, fp or all collectd_rrd: [URL to zip file containing collectd rrd files] comment config: config file name copies: number of copies (for rate runs only) delay failover_no_sse flagsurl: flags file url huge_pages ignore_errors iteration: [iteration number (used with incremental result directories)] iterations: number of iterations - 3 is default (2 or 3 required for compliant runs) max_copies meta_burst meta_compiler meta_compute_service meta_compute_service_id* meta_cpu: [CPU model info] meta_cpu_cache: [CPU cache] meta_cpu_cores: [# of CPU cores] meta_cpu_speed: [nominal CPU clock speed (MHz)] meta_cpu_speed_max: [max CPU clock speed (MHz)] meta_fw_bios [system firmware name] meta_instance_id* meta_hostname: [system under test (SUT) hostname] meta_hw_avail meta_hw_nthreadspercore meta_hw_other meta_hw_ocache meta_hw_pcache meta_hw_tcache meta_hw_ncpuorder meta_license_num meta_memory meta_memory_gb: [memory in gigabytes] meta_memory_mb: [memory in megabytes] meta_notes meta_notes_base meta_notes_comp meta_notes_os meta_notes_part meta_notes_peak meta_notes_plat meta_notes_submit meta_os_info: [operating system name and version] meta_provider meta_provider_id* meta_region* meta_resource_id meta_run_id meta_storage_config* meta_sw_avail meta_sw_other meta_test_id* nobuild nonuma numa: true if numa was supported and used (--define numa flag and numactl) num_benchmarks: total number of individual benchmarks run peak: true for a peak run, false for a base run purge_output rate: true for a rate run, false for a speed run reportable review size: input data size - test, train or ref spec_dir specspeed2017_fp_peak: peak, speed floating point score (only present if rate=0 and peak=1) specspeed2017_fp_base: base, speed floating point score (only present if rate=0 and peak=0) specfp_csv: [URL to the SPECfp rate CSV format report (if save.sh --store option used)] specfp_html: [URL to the SPECfp rate HTML format report (if save.sh --store option used)] specfp_pdf: [URL to the SPECfp rate PDF format report (if save.sh --store option used)] specfp_speed_csv: [URL to the SPECfp speed CSV format report (if save.sh --store option used)] specfp_speed_html: [URL to the SPECfp speed HTML format report (if save.sh --store option used)] specfp_speed_pdf: [URL to the SPECfp speed PDF format report (if save.sh --store option used)] specrate2017_fp_peak: peak, rate floating point score (only present if rate=1 and peak=1) specrate2017_fp_base: base, rate floating point score (only present if rate=1 and peak=0) specfp_text: [URL to the SPECfp rate text format report (if save.sh --store option used)] specfp_text_speed: [URL to the SPECfp speed text format report (if save.sh --store option used)] specspeed2017_int_peak: peak, speed integer score (only present if rate=0 and peak=1) specspeed2017_int_base: base, speed integer score (only present if rate=0 and peak=0) specint_csv: [URL to the SPECint rate CSV format report (if save.sh --store option used)] specint_html: [URL to the SPECint rate HTML format report (if save.sh --store option used)] specint_pdf: [URL to the SPECint rate PDF format report (if save.sh --store option used)] specint_speed_csv: [URL to the SPECint speed CSV format report (if save.sh --store option used)] specint_speed_html: [URL to the SPECint speed HTML format report (if save.sh --store option used)] specint_speed_pdf: [URL to the SPECint speed PDF format report (if save.sh --store option used)] specrate2017_int_peak: peak, rate integer score (only present if rate=1 and peak=1) specrate2017_int_base: base, rate integer score (only present if rate=1 and peak=0) specint_text: [URL to the SPECint rate text format report (if save.sh --store option used)] specint_text_speed: [URL to the SPECint speed text format report (if save.sh --store option used)] sse: sse optimization used (if applicable) sse_max sse_min test_started test_stopped tune: tune level - base, peak or all valid: 1 if this was a valid run (0 if invalid) validate_disk_space x64: true if 64-bit binaries used, false if 32-bit x64_failover USAGE # run 1 test iteration with some metadata ./run.sh --meta_compute_service_id aws:ec2 --meta_instance_id c3.xlarge --meta_region us-east-1 --meta_test_id aws-0914 # run with SPEC CPU 2017 installed in /usr/local/speccpu ./run.sh --spec_dir /usr/local/speccpu # run for floating point benchmarks only ./run.sh --benchmark fp # run for perlbench and bwaves only ./run.sh --benchmark 500 --benchmark 505 # save.sh saves results to CSV, MySQL, PostgreSQL, BigQuery or via HTTP # callback. It can also save artifacts (text report ) to S3, Azure Blob Storage # or Google Cloud Storage # save results to CSV files ./save.sh # save results from 5 iterations text example above ./save.sh ~/spec-testing # save results to a PostgreSQL database ./save --db postgresql --db_user dbuser --db_pswd dbpass --db_host db.mydomain.com --db_name benchmarks # save results to BigQuery and artifact (TRIAD gnuplot PNG image) to S3 ./save --db bigquery --db_name benchmark_dataset --store s3 --store_key THISIH5TPISAEZIJFAKE --store_secret thisNoat1VCITCGggisOaJl3pxKmGu2HMKxxfake --store_container benchmarks1234
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.