For the following benchmarks, -DNO_IMPLICIT_INCLUDE_CHECKED_HDRS had to be added in their Makefiles to avoid compilation errors at present.

Cause of compilation error: A function with a bounds-safe interface has another function call as an argument at one of its call-sites.
MultiSource/Benchmarks/BitBench/drop3
MultiSource/Benchmarks/FreeBench/pifft
MultiSource/Benchmarks/McCat/18-imp
MultiSource/Benchmarks/Prolangs-C/TimberWolfMC
MultiSource/Benchmarks/Prolangs-C/agrep
MultiSource/Benchmarks/Prolangs-C/assembler
MultiSource/Benchmarks/Prolangs-C/loader
MultiSource/Benchmarks/Prolangs-C/unix-smail
MultiSource/Benchmarks/Ptrdist/bc
MultiSource/Benchmarks/VersaBench/dbms

Cause of compilation error: Some of the variables/strings (like ON and n) used by Checked-C-specific declarations have been redefined as macros earlier + plus other issues.
MultiSource/Benchmarks/MallocBench/espresso
MultiSource/Benchmarks/McCat/05-eks
MultiSource/Benchmarks/MiBench/consumer-lame
MultiSource/Benchmarks/MiBench/consumer-typeset
MultiSource/Benchmarks/MiBench/network-patricia
MultiSource/Benchmarks/MiBench/office-ispell
MultiSource/Benchmarks/Prolangs-C/archie-client
MultiSource/Benchmarks/Prolangs-C/cdecl
MultiSource/Benchmarks/Trimaran/netbench-url

Stream is opened

Opened file never closed

power benchmark checkedc conversion issue

• checked array type array subscript requires dynamic_check
  • Almost reasoning facts of program can remove away redundant dynamic_check
  • Also static analysis such as constant folding can remove away dynamic_check
• checked pointer dereference
  • a->b == (*a).b, it means pointer dereference and requires non-null runtime check

As part of a paper submission, we counted the lines of code that were unchecked scopes. There are some converted benchmarks where we could reduce the amount of code in unchecked scopes:

• Olden/TreeAdd: node.c
• Olden/Power: main.c
  In addition, memset had a bounds-safe annotation that included an unchecked pointer type, which forced all memset calls to be unchecked code. This should be correct as well.
  There may be other converted benchmarks where the amount of unchecked code could be reduced.

One question is what is the criteria for putting code in a checked scope or unchecked scope. Here is what I propose initially:

• It is fine to put print calls that use varargs in unchecked scopes by themselves.
• Function bodies or blocks that are > 80% checked code (excluding print calls) should be checked.

• We should create a Wiki for this repo and track the PtrDist/Olden benchmarks that have been converted.
• We should make sure that the Checked C repo and the Checked C clang repo point to this repo.

On Linux, functions such as isalpha are turned into macros. The macros make function calls that return pointers to pointers to integers. We don't have enough functionality for Checked C implemented yet to annotate the functions called by those macros so that they can be used in checked code. As a work around, we undefined the macros in anagram so that the original library functions are called. When we do have enough functionality implemented, we should update the headers for Linux to annotate the functions called by the macros and undo the workaround in anagram.

There's a bug and ptrdist/ft is failing at execution time.

It's not in our list of our original 10 benchmarks for the preliminary evaluation, but we should get it checked and working. Not going to investigate it tonight, probably tomorrow afternoon.

• Use #pragma BOUNDS_CHECKED ON for checked scope
• If unchecked scope is required, use unchecked scope keyword unchecked/_Unchecked {...}
• For conversion, I have fixed bugs related to conversion in checkedc-clang/#302 issue

Many functions in the Ptrdist benchmark (e.g. in vcg.c) use global variables in the declared parameter bounds. The functions should instead accept parameters that are used in the declared bounds. For example:

int globalCount;
array_ptr<int> globalArr : count(globalCount);
void f(array_ptr<int> arr : count(globalCount));
f(globalArr);

should be

int globalCount;
array_ptr<int> globalArr : count(globalCount);
void f(array_ptr<int> arr : count(globalCount), int paramCount);
f(globalArr, globalCount);

The test scimark2 seems to run for a long time thereby being flagged as a FAIL.

Results

FAIL : 1
PASS : 1539
Results available at: db_None/v4/nts/1
++ grep FAIL /build/LNT-Results-Release-Linux/arm32/result.log
FAIL: MultiSource/Benchmarks/SciMark2-C/scimark2.execution_time (309 of 1540)
FAIL : 1
++ echo 'LNT testing failed.'
LNT testing failed.
++ exit 1

Originally posted by @mgrang in microsoft/checkedc-clang#610 (comment)

convert Ptrdist/ks benchmark into checked-c code

• KS (Kernighan-Schweikert graph partitioning) benchmark is converted to checked-c
• anagram/ft is slightly modified to follow check-clang modification
• checkedc-llvm-test-suite checked header files are modified to follow changes of checkedc header files

MultiSource/Benchmakr/Ptrdist all benchmarks are passed with lnt runtest

We find issues while converting existing benchmarks to be Checked C, mostly having to do with supporting specific platform's C standard library headers. We updated the headers in this repo. The changes to need to be propagated to the home repo of the headers, which is the Checked C repo.

The following tests in MultiSource/Benchmarks fail on LNT test runs after upgrading to Clang 11:

• ASC_Sequoia/IRSmk
• DOE-ProxyApps-C/miniAMR
• Olden/bh
• Prolangs-C/TimberWolfMC
• Prolangs-C/compiler
• Prolangs-C/loader
• Prolangs-C/simulator

These tests will be temporarily disabled while we debug in order to unblock the ADO runs.

Have to change Olden benchmark for checkedC.

Apparently the tolower and isalpha macros involve a function call that returns an unchecked pointer. Here is an example error:

mnt/d/autobahn1/llvm-test-suite/MultiSource/Benchmarks/Ptrdist/anagram/anagram.c:300:17: error: cannot use a return with an unchecked type in a checked scope or function
            if (isalpha(ch)) cLetters++;
                ^
/usr/include/ctype.h:198:21: note: expanded from macro 'isalpha'
# define isalpha(c)     __isctype((c), _ISalpha)
                        ^
/usr/include/ctype.h:89:6: note: expanded from macro '__isctype'
  ((*__ctype_b_loc ())[(int) (c)] & (unsigned short int) type)
     ^