joakim-brannstrom / dextool Goto Github PK

I am unable to run the triangle mutate example out of tree.

john@workstation:~$ cd /tmp
john@workstation:/tmp$ make -f /home/john/devel/dextool/plugin/mutate/examples/triangle/Makefile 
/home/john/devel/dextool/plugin/mutate/examples/triangle//../../../../build/dextool mutate analyze --db test_triangle.mut.db \
		--in /home/john/devel/dextool/plugin/mutate/examples/triangle//src/triangle.c \
		-- -I/home/john/devel/dextool/plugin/mutate/examples/triangle//inc
info: Upgrading database from 0
info: Upgrading database from 1
info: Upgrading database from 2
info: Upgrading database from 3
info: Upgrading database from 4
info: Upgrading database from 5
info: Upgrading database from 6
info: Upgrading database from 7
info: Upgrading database from 8
info: Upgrading database from 9
info: Reading compilation database:

info: Analyze and mutation of files will only be done on those inside this directory root
info:   User input: .
info:   Real path: /tmp
info: Further restricted to those inside these paths
info: Analyzing '/home/john/devel/dextool/plugin/mutate/examples/triangle/src/triangle.c'
/home/john/devel/dextool/plugin/mutate/examples/triangle//../../../../build/dextool mutate test --db test_triangle.mut.db \
		--build-cmd /home/john/devel/dextool/plugin/mutate/examples/triangle//build.sh \
		--test-cmd /home/john/devel/dextool/plugin/mutate/examples/triangle//test.sh \
		--test-timeout 1000
info: Sanity check passed. Files on the filesystem are consistent
info: Preparing for mutation testing by checking that the program and tests compile without any errors (no mutants injected)
info: make[1]: Entering directory '/tmp'
make[1]: *** No rule to make target 'test_triangle'.  Stop.
make[1]: Leaving directory '/tmp'

error: Compiler command failed: 2
error: exiting...
/home/john/devel/dextool/plugin/mutate/examples/triangle/Makefile:47: recipe for target 'test_triangle.mut.db' failed
make: *** [test_triangle.mut.db] Error 1

The generation of the files need to be split from the above flags.

The intention of the flags --gen-pre-incl/post is to allow the user to inject "hooks" before and after the headers of what has been test doubled away.

Problem:

• The flags generate ready-to-use hooks with the content of the most common use case.
• In contrast to what dextool do in other cases the files are not overwritten in this case if they already exist.

Proposed solution:

• Introduce a new flag, --gen-cpp-hooks. It generates the default hooks that suppressed __cplusplus.
• Change --gen-pre-incl and post to not generate any files. To only inject the "#include"-hooks

CppMethod does not handle a const return type.

const std::string some_func(); has a returnType of std::string, it should be const std::string.

Implement mutation testing of stmt for C++.

Proposal
Do runtime mutation via control from for example an environment variable.
Runtime to shorten the cycle mutate->compile->run to run, run, run.
Prove MC/DC followed by mutation of candidates.

Currently it is annoying for the user to have to specify the same file as restrict/exclude and input.

Allow the user to tell dextool to be a little bit intelligent. The flags are only proposals.

• "--file-restrict-by-input", restrict the generated test double to the input
• "--file-exclude-by-input", exclude the input from the generated test double

Seems to be a forward reference to splitTypeId in cpptooling.analyzer.type.
Fixed it by moving splitTypeId(ref const TypeKind t) before splitTypeId(ref const TypeKind t, const uint indent = 0)

If libclang ends with .so.1, cmake fails to find it. Found on Linux mint.

One way to fix this would be to add this line before find_library() in FindLibClang.cmake.
list(APPEND CMAKE_FIND_LIBRARY_SUFFIXES .so.1)

I guess that better solutions to this exists (like symlinking libclang.so.1 to libclang.so).

When giving the in-flag a tilde it does not get expanded to /home/username (the tilde is never removed).
This ONLY occurs when having an equal sign after in. I guess that this has to do with bash not expanding the tilde (?).

For example:
Input: --in=~/something/something.hpp
Output: /home/username/~/something/something.hpp

Problem:

• The user has raised the design problem that tests become coupled when globals aren't initialized before the test start.

Questions:

1. How should an interface be designed to "notify" the user during compile
   time that a new global variable have been found that need to be
   initialized.
2. How should the design ensure "correct by construction"?
3. What information needs to be exposed in the interface?
   What can be hidden?

Design ideas to fulfil the questions:

1. Make a C++ interface that is used to initialize the global variables.
   Make each global a pure method.
   It forces the user to update the class implementing the interface when a
   new global has been added.

2. The adapter for the test double takes a class that implement the interface.
   It forces the user to implement "initializers" of the globals.
   Make a conscious decision.
   The adapter then calls the functions in its constructor.
   It sends "events" to the global initializer.

3. Make the interface just a plain, void methods. No arguments.
   It simplifies the implementation of the C++ interface.
   Less boiler plate.
   Makes it easier to implement because there are cases, especially C++, where it is "hard" to know how to pass the object.
   Leave as much as possible to the implementation.

   • A side effect of the design is that it is easy for the user to "ignore"
     initializing globals if so is desired.

4. (EXTRA) The global initializer from the user is NOT passed by reference.
   It is to make it possible for the user to call the adapter without first having
   to create an instance in the scope.

5. The most common desired initializer is one that emulates what the runtime do to the .bss-segment.
   Namely it zeroes everything in this segment.
   So a default initializer that would work in most cases is one that blast zeroes over all globals (memset).
   Generate such a default initializer inheriting from the InitGlobal interface.

Example

// software under test
extern int a;

// test double code
namespace TestDouble {
class I_InitGlobals {
public:
    virtual void a() = 0;
};

class ZeroGlobals : public I_InitGlobals {
public:
    virtual void a() {
        memset(&a, 0, sizeof(int));
    }
};

class Adapter {
public:
    Adapter(I_InitGlobals init_globals) {
        init_globals.a();
    }
};
} //NS: TestDouble

Using

TEST(.., ...) {
   TestDouble::Adapter(ZeroGlobals());
...
}

Allow the limits for the colors in the html report for files to be configurable. The user may have either looser or stricter requirements than those that are hard coded atm.

Input:

typedef unsigned int ll;
typedef ll some_array[1];

const some_array& some_func();

Generated output:

const unsigned int ( &)[1] some_func() const {
 	do_something();
}

Expected:

const some_array& some func() {
	do_something();
}

Let --in detect if it is a directory and then do for all files in it

Show what test case killed a mutant.

Generate operator== for comparing the fields in structs.

It is somewhat cumbersome to have to write them by hand.
Such a feature would make life easier for a developer.

A user must be able to tell dextool to use a bundle of files as input.

The usual flags like restrict and exclude are expected to work for the bundle.
The compiler flags are derived from the compilation database OR the usual CFLAGS that are passed after a "--".

• Implement for C
• Implement for C++

Since it is stated in README that dextool support both DMD and LDC, should the CI contain both compilers?

//NP

Generated gmocks are missing the destructor.

Enhance the generated UML diagrams by.

• Allow a component view by files instead of directories.
• Allow visual view of the directory structure by using graphviz clusters.
• List the files that are part of a component inside the component.
• Generate a xml file with the relations, sorted. Allows the data to be stored, imported, xslt transformed and diff'ed in git.

Uhm, generatetPreInclude shouldn't this be called generatePreInclude? ;)
Or is there any reason for its naming?

Can be found in cpptooling.generator.includes

-isysroot is a Mac specific argument to clang that says where the system files are (/usr/include etc.)
Unlike -I it has a space after -isysroot and then comes the path. Which makes compilation_db.d think it just is a flag without any parameters.

Without it, Clang can't compile since it does not know where the system headers etc. are.

Suggested fix for compilation_db.d in parseFlag:

        foreach (arg; r) {
            if (st == State.skip) {
                st = State.keep;
            } else if (st == State.skipIfNotFlag && isNotAFlag(arg)) {
                st = State.keep;
            } else if (st == State.isInclude) {
                st = State.keep;
                // if an include flag make it absolute
                rval.put("-I");
                rval.put(buildNormalizedPath(workdir, arg).absolutePath);
            } else if (isCombindedIncludeFlag(arg)) {
                rval.put("-I");
                rval.put(buildNormalizedPath(workdir, arg[2 .. $]).absolutePath);
            } else if (arg.among("-I")) {
                st = State.isInclude;
            }
            else if (arg.among("-isysroot")) { //This should be added
                st = State.isInclude;
            }
            else if (excludeStartWith(arg, flag_filter)) {
                st = State.skipIfNotFlag;
            } else if (isNotAFlag(arg)) {
                // skipping
            } else {
                rval.put(arg);
            }
        }

The example installation command is:

sudo apt install build-essential cmake llvm-4.0 llvm-4.0-dev clang-4.0 libclang-4.0-dev libsqlite3-dev

If this is the case should not both llvm-dev and libclang-dev be stated among the dependencies as well?

Best of regards,

John

A common scenario is a user that need to adapt the code generation.

Enhance the architecture to allow the user to adapt:

• CLI, custom parameters
• Main analyze -> generator runner (e.g. CppTestDouble)
• Code generation

Note: Keep the changes light. Probably enough to allow a D module to be added that then affect above parts.
KISS

Generate a whole program execution flow diagram for the static view.

Further improve by generating a diff between two flows.

Inline asm makes dextool plugin ctestdouble barf.

It shouldn't stop. It should log a warning and continue.

Wrong filename I cstub_tests.d
Skip compiling, apparently stdarg.h do not exist on all platforms

Multiple parameters for Et is not implemented (afaik).
At the moment one would need to create a string with all the parameters, would be better if Et takes an array (or something like an array) with strings.

For example:
Et("Some_template")("param1, param2") is what is required now.
What I am proposing: Et("Some_template")("param1", "param2")

Am I missing something or doesn't enums exist in dsrcgen?

It should :)

There are few laws regarding software but one that is certain is that entropy increases with time.

From the static analyze calculate the entropy.
No idea yet the algorithm(s).
Ideas are welcome.

Mutation is done on the AST level but the textual replacement is done on ranges in the source code. If a mutant is done that in the source code is a macro at that location the macro should be inserted.
Then the mutant should be injected in the expanded macro.

dsrcgen should support empty return statements.

return_() should become return; in the output code

Generate automatic integration wrappers for AFL.

Make it possible to load configuration data from either a system installed by default or a user supplied via CLI

Missing licenses for Dstep (boost) and Clang (Illinois...)

Use cases.
Internal design.
Intro.

Add cmake.

func_body args (dsrcgen) should be able to handle a CxParam[] since that is what CFunction is returning.

Hi,

I have been following the instructions in the readme.

When executing:

`CMake Error at cmake/Modules/FindDCompiler.cmake:72 (string):
string sub-command REGEX, mode MATCH regex "^[^

:]*" matched an empty string.
Call Stack (most recent call first):
cmake/CMakeDetermineDCompiler.cmake:2 (include)
CMakeLists.txt:5 (enable_language)

CMake Error at cmake/Modules/ExtractDMDSystemLinker.cmake:40 (message):
Failed to compile empty program using D compiler '/opt/dmd/2.088'
Call Stack (most recent call first):
cmake/Modules/FindDCompiler.cmake:76 (include)
cmake/CMakeDetermineDCompiler.cmake:2 (include)
CMakeLists.txt:5 (enable_language)

CMake Error: CMAKE_D_COMPILER not set, after EnableLanguage
-- Configuring incomplete, errors occurred!
See also "/home/myUser/TDDD04/DEXTOOL/dextool/build/CMakeFiles/CMakeOutput.log".
`
CMakeOutput.log

Best regards,

John

Noticed the failures when upgrading travis to xenial.

The failures seem to affect the unittests. This probably propagate to the plugins c/cpptestdouble.

A more feature complete build system is needed for the future.
Reggae seems to be suitable for the task.

Investigate and integrate reggae.

• Adjust the plugin system to be configurable via dub (NOT RELEVANT ANYMORE)
• Adjust the module to start with dextool intead of plugin (NOT RELEVANT ANYMORE)
• Adjust the directory structure to have a 1:1 mapping with the modules.

The build instructions are lacking.
Improve them to make it easier for a user to build and install dextool.

May require a migration from dub -> cmake.

Problem: It is cumbersome for a developer to create a test double for a c/c++ header because they have to find a implementation file in the compilation database that includes the specific header.

Implement a "search" function when using a compilation database.
It searchers through all the files for one that #include the desired header.
The flags for that implementation file is then used to parse the header and generate the test double.

It is cumbersome to create a test double for a file.
What is needed are the compiler parameters used to compile the file at hand.

Write tools to wrap the compiler command (gcc/g++ etc).
So that while the program is being built it is also calling deXtool, which in turn generates the test doubles.

How?
At the top of PATH have a gcc/g++ command wrapper.
The wrapper calls dextool and g++.

Produce a report that ranks the test cases by how many mutants they killed.

Summarize the most important and actionable information as one report page in the html output.

What is actionable information? That is a good questions....

It is cumbersome to specify the compiler flags.
There is a standard from Clang called Compilation Database which stores the three attributes:

• workdir
• filename
• compiler command

Clang Compilation Database

Add support to use such a database.

Proposal:

• A CLI to specify a DB and filename.
• Search the DB for the filename to find the compiler flags used.
• Filter the compiler flags.
• Pass on to libclang to parse the file.

Perform and generate a simplified data flow analysis.

Draft algorithm.
Do the data flow per function.
A and B is input to a function and C is the output.
Assume that both A and B affect C.