This repository is part of the paper Better Safe Than Sorry! Automated Identification of Breaking Security-Configuration Rules accepted at the 4th ACM/IEEE International Conference on Automation of Software Test (AST).

Institutions like the Center for Internet Security publish security-configuration guides(also called benchmarks) that help us configure systems more securely. This configuration hardening can mitigate the risk of successful attacks, which may cause damage to our systems and data. A remaining problem with applying these guides are so-called "breaking rules." Applying breaking rules on a production system will break at least one functionality with the corresponding ramifications. We could safely apply the remaining rules if we identified all breaking rules and removed them from the guide.

Our new approach combines techniques from software testing, machine learning, and graph theory to automatically identify these breaking rules. This repository includes our Python scripts to

1. generate the covering arrays from a given security-configuration guide
2. Test the different covering arrays
3. Analyze the results to find the breaking rules

One can redo all our experiments presented in the article using the code in this repository.

The easiest way to use the scrips in this repository is to install the package from PyPi

pip install better-safe-than-sorry
better-safe-than-sorry --version

One can also use poetry to install the dependencies.

cd /path/to/better-safe-than-sorry/
poetry install
poetry run better-safe-than-sorry --version

See here.

See here.

See here

See here.

The folder rsc/sfera_automation_jsons contains variants of sfera_automation.json files based on the Windows 10 version 1909 guide by the Center for Internet Security. sfera_automation.json is a JSON-based file format used at Siemens to automatically implement Windows-based security-configuration guides. We generated the variants were generated using the IPOG and IPOG-D algorithms and include custom profiles for combinatorial testing of strength 2 to 5.

If you have any questions, please create an issue or contact Patrick Stöckle.