DataRaceBench is a benchmark suite designed to systematically and quantitatively evaluate the effectiveness of data race detection tools. It includes a set of microbenchmarks with and without data races. Parallelism is represented by OpenMP directives. OpenMP is a popular parallel programming model for multi-threaded applications.
Note that if you are using gcc for compiling the microbenchmarks, at least version 4.9 is required to have support for all used OpenMP directives.
DataRaceBench also comes with an evaluation script (check-data-races.sh). The script can be used to evaluate the tools Helgrind, Archer, Thread Sanitizer, and Intel Inspector. In addition a parameterized test harness (scripts/test-harness.sh) is available which allows to provide a number of different parameters for the evaluation. The test harness is used by the evaluation script with some pre-defined values.
| P-Label | Meaning (microbenchmarks with data races) | P-Label | Meaning (microbenchmarks without data races) |
|---|---|---|---|
| Y1 | Unresolvable dependences | N1 | Embarrassingly parallel |
| Y2 | Missing data sharing clauses | N2 | Use of data sharing clauses |
| Y3 | Missing synchronization | N3 | Use of synchronization |
| Y4 | SIMD data races | N4 | Use of SIMD directives |
| Y5 | Accelerator data races | N5 | Use of accelerator directives |
| Y6 | Undefined behavior | N6 | Use of special language features |
| Y7 | Numerical kernel data races | N7 | Numerical kernels |
| ID | Microbenchmark | P-Label | Description | Source |
|---|---|---|---|---|
| 1|2 | antidep1-(orig|var)-yes.c | Y1 | Anti-dependence within a single loop | AutoPar |
| 3|4 | antidep2-(orig|var)-yes.c | Y1 | Anti-dependence within a two-level loop nest | AutoPar |
| 5 | indirectaccess1-orig-yes.c | Y7 | Indirect access with overlapped index array elements | LLNL App |
| 6 | indirectaccess2-orig-yes.c | Y7 | Overlapping index array elements when 36 or more threads are used | LLNL App |
| 7 | indirectaccess3-orig-yes.c | Y7 | Overlapping index array elements when 60 or more threads are used | LLNL App |
| 8 | indirectaccess4-orig-yes.c | Y7 | Overlapping index array elements when 180 or more threads are used | LLNL App |
| 9|10 | lastprivatemissing-(orig|var)-yes.c | Y2 | Data race due to a missing lastprivate() clause |
AutoPar |
| 11|12 | minusminus-(orig|var)-yes.c | Y3 | Unprotected decrement operation -- |
AutoPar |
| 13 | nowait-orig-yes.c | Y3 | Missing barrier due to a wrongfully used nowait | AutoPar |
| 14|15 | outofbounds-(orig|var)-yes.c | Y6 | Out of bound access of the 2nd dimension of array | AutoPar |
| 16|17 | outputdep-(orig|var)-yes.c | Y1 | Output dependence and true dependence within a loop | AutoPar |
| 18|19 | plusplus-(orig|var)-yes.c | Y1 | increment operation ++ on array index variable |
AutoPar |
| 20|21 | privatemissing-(orig|var)-yes.c | Y2 | Missing private() for a temp variable |
AutoPar |
| 22|23 | reductionmissing-(orig|var)-yes.c | Y2 | Missing reduction() for a variable |
AutoPar |
| 24 | sections1-orig-yes.c | Y3 | Unprotected data writes in parallel sections | New |
| 25|26 | simdtruedep-(orig|var)-yes.c | Y1,Y4 | SIMD instruction level data races | New |
| 27 | targetparallelfor-orig-yes.c | Y1,Y5 | Data races in loops offloaded to accelerators | New |
| 28 | taskdependmissing-orig-yes.c | Y3 | Unprotected data writes in two tasks | New |
| 29|30 | truedep1-(orig|var)-yes.c | Y1 | True data dependence among multiple array elements within a single level loop | AutoPar |
| 31|32 | truedepfirstdimension-(orig|var)-yes.c | Y1 | True data dependence of first dimension for a 2-D array accesses | AutoPar |
| 33|34 | truedeplinear-(orig|var)-yes.c | Y1 | Linear equation as array subscript | AutoPar |
| 35|36 | truedepscalar-(orig|var)-yes.c | Y1 | True data dependence due to scalar | AutoPar |
| 37|38 | truedepseconddimension-(orig|var)-yes.c | Y1 | True data dependence on 2nd dimension of a 2-D array accesses | AutoPar |
| 39|40 | truedepsingleelement-(orig|var)-yes.c | Y1 | True data dependence due to a single array element | AutoPar |
| 73 | doall2-orig-yes.c | Y2 | Missing private() for inner loop nest's loop index variable |
New |
| 74 | flush-orig-yes.c | Y2 | Reduction using a shared variable, extracted from an official OpenMP example | New |
| 75 | getthreadnum-orig-yes.c | Y1 | Work sharing within one branch of a if statement |
New |
| ID | Microbenchmark | P-Label | Description | Source |
|---|---|---|---|---|
| 41 | 3mm-parallel-no.c | N2 | 3-step matrix-matrix multiplication, non-optimized version | Polyhedral |
| 42 | 3mm-tile-no.c | N2,N4 | 3-step matrix-matrix multiplication, with tiling and nested SIMD | Polyhedral |
| 43 | adi-parallel-no.c | N2 | Alternating Direction Implicit solver, non-optimized version | Polyhedral |
| 44 | adi-tile-no.c | N2,N4 | Alternating Direction Implicit solver, with tiling and nested SIMD | Polyhedral |
| 45 | doall1-orig-no.c | N1 | Classic DOAll loop operating on a one dimensional array | AutoPar |
| 46 | doall2-orig-no.c | N1 | Classic DOAll loop operating on a two dimensional array | AutoPar |
| 47 | doallchar-orig-no.c | N1 | Classic DOALL loop operating on a character array | New |
| 48 | firstprivate-orig-no.c | N2 | Example use of firstprivate | AutoPar |
| 49 | fprintf-orig-no.c | N6 | Use of fprintf() |
New |
| 50 | functionparameter-orig-no.c | N6 | Arrays passed as function parameters | LLNL App |
| 51 | getthreadnum-orig-no.c | N2 | single thread execution using if (omp_get_thread_num()==0) |
New |
| 52 | indirectaccesssharebase-orig-no.c | N7 | Indirect array accesses using index arrays without overlapping | LLNL App |
| 53 | inneronly1-orig-no.c | N1 | Two-level nested loops, inner level is parallelizable. True dependence on outer level | AutoPar |
| 54 | inneronly2-orig-no.c | N1 | Two-level nested loops, inner level is parallelizable. Anti dependence on outer level | AutoPar |
| 55 | jacobi2d-parallel-no.c | N7 | Jacobi with array copying, no reduction, non-optimized version | Polyhedral |
| 56 | jacobi2d-tile-no.c | N4,N7 | Jacobi with array copying, no reduction, with tiling and nested SIMD | Polyhedral |
| 57 | jacobiinitialize-orig-no.c | N7 | The array initialization parallel loop in Jacobi | AutoPar |
| 58 | jacobikernel-orig-no.c | N7 | Parallel Jacobi stencil computation kernel with array copying and reduction | AutoPar |
| 59 | lastprivate-orig-no.c | N2 | Example use of lastprivate | AutoPar |
| 60 | matrixmultiply-orig-no.c | N7 | Classic i-k-j order matrix multiplication using OpenMP | AutoPar |
| 61 | matrixvector1-orig-no.c | N7 | Matrix-vector multiplication parallelized at the outer level loop | AutoPar |
| 62 | matrixvector2-orig-no.c | N7 | Matrix-vector multiplication parallelized at the inner level loop with reduction | AutoPar |
| 63 | outeronly1-orig-no.c | N2 | Two-level nested loops, outer level is parallelizable. True dependence on inner level | AutoPar |
| 64 | outeronly2-orig-no.c | N2 | Two-level nested loops, outer level is parallelizable. Anti dependence on inner level | AutoPar |
| 65 | pireduction-orig-no.c | N7 | PI calculation using reduction | AutoPar |
| 66 | pointernoaliasing-orig-no.c | N6 | Pointers assigned by different malloc calls, without aliasing | LLNL App |
| 67 | restrictpointer1-orig-no.c | N6 | C99 restrict pointers used for array initialization, no aliasing | LLNL App |
| 68 | restrictpointer2-orig-no.c | N6 | C99 restrict pointers used for array computation, no aliasing | LLNL App |
| 69 | sectionslock1-orig-no.c | N3 | OpenMP parallel sections with a lock to protect shared data writes | New |
| 70 | simd1-orig-no.c | N1,N4 | OpenMP SIMD directive to indicate vectorization of a loop | New |
| 71 | targetparallelfor-orig-no.c | N1,N5 | data races in loops offloaded to accelerators | New |
| 72 | taskdep1-orig-no.c | N3 | OpenMP task with depend clauses to avoid data races | New |
| 76 | flush-orig-no.c | N2 | OpenMP private clause to avoid data races | New |
| 77 | single-orig-no.c | N2 | OpenMP single directive to avoid data races | New |
| 78 | taskdep2-orig-no.c | N3 | OpenMP task depend clause to avoid data races | New |
| 79 | taskdep3-orig-no.c | N3 | OpenMP task depend clause to avoid data races | New |
DataRaceBench was created by Chunhua Liao, Pei-Hung Lin, Joshua Asplund, Markus Schordan, and Ian Karlin.
DataRaceBench is released under a BSD license. For more details see the file LICENSE.txt. The microbenchmarks marked 'Polyhedral' in above table were generated as optimization variants of benchmarks from the PolyOpt benchmark suite. For those benchmarks see the license file LICENSE.OSU.txt.
LLNL-CODE-732144
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent