klassen-software-solutions / ksstest Goto Github PK

When compiling with clang++ on Linux, there are warnings regarding non-virtual destructors.

It dies on the to_string method. Likely we should change this to use a string stream.

See the go language books for some examples. Note that it will be important that although random, the failures must be repeatable. This will mean using and recording some type of seed that will be reported along with the test failure.

Specifically, it does not give up after the requested time, but always attempts to complete its lambda and only then checks the time. It needs to stop when the requested time has been passed and report the failure.

Specifically it is using "Source" so the all.h from any kss library will have the same ifdef guard.

Definitely default this to off, but some may want to stop immediately rather than continuing the tests. Of course this is a little more complex when they are running in separate threads. This will require a form of early termination to be added to the threads.

For example, if the user has

KSS_ASSERT(isEqualTo(23, []{ return 21; }));

instead of

KSS_ASSERT(23 == 21);

can we include some kind of "expected '23' but found '21'" in the error message?

Technical idea: One way to do this may be to have a thread local optional message that the "is..." methods fill if they are going to fail.

When running tests in parallel I am sometimes getting an E in the initial output, but no error is reported in the summary and all tests are reported as passed.

Need to ensure that we use the header comments in the code. They are written to support headerdoc which makes the docs available in Xcode. So we either need to use headerdoc or something that has compatible markup abilities.

• Find a suitable document generator
• Determine how we want to publish (e.g. GitHub pages)
• Update the build system to generate the docs.

When embedding ksstest.cpp into your own project, any files that create TestSuite objects that are compiled before ksstest.cpp have their tests ignore. This appears to be due to the use of the static testSuites vector.

The workaround is to ensure that ksstest.cpp is compiled before any other source files (perhaps by renaming it to _ksstest.cpp). Then it works as intended.

However we need to fix this. Perhaps by replacing the static object with a singleton pattern?

See the books on the go language for some examples. The key trick in C++ will be getting the test result to show the position of the failed test in the table rather than the position of the failed assertion.

Determine and implement a means of providing a coverage report.

This would be targeted at floating point values and would need to accept a target value and an epsilon with the epsilon defaulting based on the numeric_limits of the underlying type. The details of the result would output the values in a manner similar to isEqualTo.

There should also be a corresponding isNotCloseTo.

Ideally it would also give a compiler error if used on a non-floating point type.

I believe this is due to the currentTest state variable. Even though it is thread local, I get a bad access when it is accessed in another thread. Not yet sure why this would be the case.

The following example will fail (assuming it is in a test)...

thread th { []{
    KSS_ASSERT(true);
}};
th.join();

The following example will also fail...

auto fut = async([]{ KSS_ASSERT(true); });
fut.wait();

I've received an answer to my query on Stack Overflow (https://stackoverflow.com/questions/59439380/can-i-filter-a-github-action-step-based-on-the-commit-message) so I need to try it out.

We don't want to reproduce autoconf, but we do want to support the following:

./configure
make
sudo make install

In addition configure should support the --prefix=something option.

I've decided against having the build system as a git submodule, primarily due to how that adds a required step before you can build anything. So instead I'm going to add a build system upgrade script that when run will update to the latest master of the build system.

This may prove difficult, but it would be nice if we could check for memory leaks and show where they come from.

This would be useful to help find intermittent errors.

Something like "completesWithin(...)" that takes a duration.

In particular, could we add a "test driven development" syntax on top of the existing framework that would allow tests to be specified in a "given, when, then/should" format?

Specifically the following, together with more helpful messages:

• isLessThan
• isLessThanOrEqualTo
• isGreaterThan
• isGreaterThanOrEqualTo

• init statement in if and switch
• inline static variables
• [[nodiscard]] attribute
• [[maybe_unused]] attributre
• string conversions

Some things could be bold, or italicized. Perhaps use green/red as an ok/fail indicator.

The default for building should be the release version (since at KSS we normally do the debugging in Xcode). But "make check" should build the debug version and run the tests on it.

Other tests give the value when there is a failure. This one should too.

This should describe the possible options for building.

Look at how the GO test platform works. This would be specified on the command line and be accessible in the infrastructure to allow tests to do different things if you want to run a short (quick) version as opposed to a complete one.

Look at the Java Awaitility and see if we can do something similar in C++.