boost-ext / ut Goto Github PK
View Code? Open in Web Editor NEWC++20 μ(micro)/Unit Testing Framework
Home Page: https://boost-ext.github.io/ut
License: Boost Software License 1.0
C++20 μ(micro)/Unit Testing Framework
Home Page: https://boost-ext.github.io/ut
License: Boost Software License 1.0
Hi,
I'm working on the ApprovalTests.cpp integration and I've ran into a couple issues:
I need the filename from where the test is implemented (caller function). assertion_pass and assertion_fail have that information but not test_begin. As this information needs to be set before the test is actually run, test_begin needs to have that information. I've been looking at the UT code, but haven't quite figure out the best way to do it.
In ApprovalTests.cpp, when a test fails it throws an exception which import information inside the exception (e.what()). So when a test throws an exception I would like to check it's contents (and possibility it's type), not simply see if the test threw an exception or not, as it's implemented now. What do you think it would be the best approach here. Simply to add a std::string to the exception struct and store the exception data there? Again not sure the best approach to change the code here.
Any help would be appreciated. If you want to take a look at the integration between ApprovalTests.cpp and UT take a look at this pull request at the UTApprovals.h file.
Thanks!
Is it possible to have the predicate printed for constant expressions when there is a failure?
Take the following code as an example, which is taken from the tutorial but evaluated at compile-time.
int main() {
using namespace boost::ut;
expect(constant<1_i == 2>);
}
Is it possible to get output similar to the following at run-time where the predicate is shown on failure?
main.cpp:4:FAILED [1 == 2]
===============================================================================
tests: 0 | 0 failed
asserts: 1 | 0 passed | 1 failed
Did not find implementation for pattern "Given I have a victor"
Did not find implementation for pattern "When I resize begger"
Did not find implementation for pattern "Then The size suould increase"
All tests passed (3 fatal in 1 tests)
All tests passed (0 asserts in 1 tests)
int main() {
using namespace boost::ut;
bdd::gherkin::steps steps = [](auto& steps) {
steps.feature("Vector") = [&] {
steps.scenario("*") = [&] {
steps.given("I have a vector") = [&] {
std::vector v(5);
expect((5_ul == std::size(v)) >> fatal);
steps.when("I resize bigger") = [&] { v.resize(10); };
steps.then("The size should increase") = [&] { expect(10_ul == std::size(v)); };
};
};
};
};
"Vector"_test = steps |
R"(
Feature: Vector
Scenario: Resize
Given I have a victor
When I resize begger
Then The size suould increase
)";
}
In the following example:
"args"_test =
[](const auto& arg) {
expect(arg >= 1_i);
}
| std::vector{1, 2, 3};
You use a custom integer literal. Why? I noticed there is afair amount of code in the library to support this so I was just curious what its purpose is as it works fine without the literals.
That this code compiles:
catch (const std::exception& e)
{
expect(approvalMissingExceptionReceived == true_b)
<< "Unexpected exception received: " << e.what();
}
Compiler error:
.../ApprovalTests.cpp/tests/UT_Tests/UTApprovalTestTests.cpp:34:56: error: use of undeclared identifier 'true_b'
expect(approvalMissingExceptionReceived == true_b)
With v 1.1.7:
expect(false_b == true_b);
With v1.1.8:
/opt/local/bin/clang-mp-9.0 --version
clang version 9.0.1
Target: x86_64-apple-darwin19.6.0
Thread model: posix
InstalledDir: /opt/local/libexec/llvm-9.0/bin
Would it be possible to disable tests running as part of the build? It makes it difficult to incorporate this project as sub-build when failing tests block building the rest of the project. It would be nice to be able to then run all of the tests with a CTest invocation after the build step.
When configuring the [Boost].µT project with CMake, I receive the following warning:
CMake Warning (dev) at cmake-build-debug/_deps/ut-src/CMakeLists.txt:41 (elseif):
Policy CMP0054 is not set: Only interpret if() arguments as variables or
keywords when unquoted. Run "cmake --help-policy CMP0054" for policy
details. Use the cmake_policy command to set the policy and suppress this
warning.
Quoted variables like "MSVC" will no longer be dereferenced when the policy
is set to NEW. Since the policy is not set the OLD behavior will be used.
It looks like setting the minimum CMake version to 3.1 would silence this, according to the documentation of CMP0054.
If C++17* support (*with limitations) is advertised, things should mostly work when in C++17 mode.
Even
int main() {
boost::ut::expect(true);
}
fails to compile with MSVC saying:
[build] C:\Users\mnagy\Source\Repos\ut\.vscode\install\include\boost/ut.hpp(1402): error C7555: use of designated initializers requires at least '/std:c++latest'
[build] C:\Users\mnagy\Source\Repos\ut\.vscode\install\include\boost/ut.hpp(1402): note: This diagnostic occurred in the compiler generated function 'auto boost::ext::ut::v1_1_8::runner<TReporter,MaxPathSize>::on(boost::ext::ut::v1_1_8::events::test<Ts...>)'
This issue is not unique to MSVC, even Appveyor build with clang-cl have output such as:
[57/112] Building CXX object example\CMakeFiles\boost_ut_parameterized.dir\parameterized.cpp.obj
..\example\parameterized.cpp(30,20): warning: explicit template parameter list for lambdas is a C++20 extension [-Wc++20-extensions]
"types"_test = []<class T>() {
^
When the entry points to testing use C++20 features, I naturally don't expect Boost UT to work in C++17 mode, but most features should have C++17 workarounds (hence the asterisk "with limitations").
Either decide whether the library supports C++17 or not. If yes, actually specify/mark which features are known to not work. Run strict C++17 builds in CI (-Werror
) and disable such tests on purpose if there are no workarounds implemented.
BTW, this regression was introduced in v1.1.5
, as installing tagged version of v1.1.4
compiles the simplest example as intended.
cmake_minimum_required(VERSION 3.0)
project(Test-Boost.UT
LANGUAGES CXX
)
if (MSVC)
string(REGEX REPLACE "/W[0-9]" "" CMAKE_CXX_FLAGS ${CMAKE_CXX_FLAGS})
endif (MSVC)
find_package(ut REQUIRED)
add_executable(Test1 Test1.cpp)
target_link_libraries(Test1
PRIVATE
boost::ut
)
target_compile_options(Test1
PRIVATE
$<$<CXX_COMPILER_ID:MSVC>:
/W4 # Turn on all (sensible) warnings
/permissive- # Turn on strict language conformance
/EHsc # Specify exception handling model
>
)
C:\Kellekek\Microsoft\VisualStudio\2019\BuildTools\VC\Tools\MSVC\14.26.28801\bin\Hostx64\x64\cl.exe /nologo /TP -IC:\Users\mnagy\Source\Repos\ut\.vscode\install\include /DWIN32 /D_WINDOWS /GR /EHsc /MDd /Zi /Ob0 /Od /RTC1 /W4 /permissive- /EHsc /std:c++17 /showIncludes /FoCMakeFiles\Test1.dir\Test1.cpp.obj /FdCMakeFiles\Test1.dir\ /FS -c ..\..\Test1.cpp
This is more of a question than a bug report, as I haven't managed to narrow down my configuration enough to give you a minimal repro.
The question is: is the comma operator in this code intentional, and if so, could a static cast be added to silence the warning?
In some of my builds, where I treat warnings as errors, I am getting this:
/Users/clare/Documents/develop/ApprovalTests/ApprovalTests.cpp.CMakeSamples/ut/include/boost/ut.hpp:210:9: error: possible misuse of comma operator here [-Werror,-Wcomma]
++pi, ++si;
^
/Users/clare/Documents/develop/ApprovalTests/ApprovalTests.cpp.CMakeSamples/ut/include/boost/ut.hpp:210:5: note: cast expression to void to silence warning
++pi, ++si;
^~~~
static_cast<void>( )
1 error generated.
/opt/local/bin/clang-mp-9.0 --version
clang version 9.0.1
Target: x86_64-apple-darwin19.6.0
Thread model: posix
InstalledDir: /opt/local/libexec/llvm-9.0/bin
A function with a throws
/ nothrow
assertion should compile with the Visual C++ Compiler when given a lambda which uses a constexpr
std::array
variable... right? This is a very specific issue 😩
#include <array>
#include <boost/ut.hpp>
int main() {
using namespace boost::ut;
"example_test"_test = [] {
constexpr std::array<int, 1> i{{10}};
expect(nothrow([] { return i.front(); }));
};
}
It is also interesting to note this only occurs in a test case and not when used directly from main
, hence the following compiles:
#include <array>
#include <boost/ut.hpp>
int main() {
using namespace boost::ut;
expect(nothrow([] {
constexpr std::array<int, 1> i{{10}};
return i.front();
}));
}
Using a constexpr int
instead of a std::array
also compiles:
#include <boost/ut.hpp>
int main() {
using namespace boost::ut;
"example_test"_test = [] {
constexpr int i{10};
expect(nothrow([] { return i; }));
};
}
Perhaps this is a bug in the MSVC compiler?
Error C2326 is given stating that the lambda function cannot access the constexpr variable.
I can workaround this issue by putting the constexpr variable within the lambda itself, as follows:
#include <array>
#include <boost/ut.hpp>
int main() {
using namespace boost::ut;
"example_test"_test = [] {
expect(nothrow([] {
constexpr std::array<int, 1> i{{10}};
return i.front();
}));
};
}
Compiling the preceding snippets above using C++20 / MSVC 16.4.4 should showcase the error.
By the way, Clang compiles the problematic code just fine.
Boost.ut builds and tests pass with VS2019 MSVC with clang-cl compiler
I know this, as we have been building that with the Approval Tests integration, which works file with this compiler.
Builds fail, with the error:
Building Custom Rule D:/a/ApprovalTests.cpp.Builds/ApprovalTests.cpp.Builds/_deps/ut-src/example/CMakeLists.txt
clang-cl : error : unknown argument ignored in clang-cl: '-pedantic' [-Werror,-Wunknown-argument] [D:\a\ApprovalTests.cpp.Builds\ApprovalTests.cpp.Builds\_deps\ut-build\example\BDD.vcxproj]
clang-cl : error : unknown argument ignored in clang-cl: '-pedantic-errors' [-Werror,-Wunknown-argument] [D:\a\ApprovalTests.cpp.Builds\ApprovalTests.cpp.Builds\_deps\ut-build\example\BDD.vcxproj]
For an example failure, see https://github.com/claremacrae/ApprovalTests.cpp.Builds/runs/370748554
See https://github.com/approvals/ApprovalTests.cpp/blob/v.7.0.0/tests/UT_Tests/CMakeLists.txt#L17 for the type of check to use...
Someone previously reported this problem with ApprovalTests.cpp on this environment, and provided a fix which used windows-style compiler warnings if using clang-cl...
find_package(ut) # works
cmake -S test -B build/test -G Ninja -DCMAKE_PREFIX_PATH=/Users/clausklein/Workspace/cpp/ut/root -DTEST_INSTALLED_VERSION=1
-- The CXX compiler identification is GNU 10.2.0
-- Checking whether CXX compiler has -isysroot
-- Checking whether CXX compiler has -isysroot - yes
-- Checking whether CXX compiler supports OSX deployment target flag
-- Checking whether CXX compiler supports OSX deployment target flag - yes
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/local/bin/g++-10 - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
CMake Error at CMakeLists.txt:11 (find_package):
Could not find a configuration file for package "ut" that is compatible
with requested version "1.1".
The following configuration files were considered but not accepted:
/Users/clausklein/Workspace/cpp/ut/root/lib/cmake/ut/ut-config.cmake, version: unknown
-- Configuring incomplete, errors occurred!
diff --git a/test/CMakeLists.txt b/test/CMakeLists.txt
index b00892a..00cb6fb 100644
--- a/test/CMakeLists.txt
+++ b/test/CMakeLists.txt
@@ -4,5 +4,12 @@
# Distributed under the Boost Software License, Version 1.0.
# (See accompanying file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt)
#
+cmake_minimum_required(VERSION 3.14...3.20)
+project(boost.ut LANGUAGES CXX)
+
+if(TEST_INSTALLED_VERSION)
+ find_package(ut 1.1 REQUIRED)
+endif()
+
add_subdirectory(ut)
add_subdirectory(ft)
I am very interested in using your library. I would like to use FetchContent
and have access to your library through add_subdirectory
. However, your library does not create a convenience target to use with target_link_libraries
. While it is little effort for me to add the necessary include directory myself, I would much rather just "link" against your header only library like so:
target_link_libraries(MyTarget PRIVATE Boost::ut)
I have included an import prefix to mirror Boost's component linking behavior.
Defining an interface library through CMake with the appropriate include directories will make this possible.
The changes might look like the following.
add_library(ut INTERFACE)
add_library(Boost::ut ALIAS ut)
target_include_directories(ut INTERFACE $<BUILD_INTERFACE:${PROJECT_SOURCE_DIR}/include>)
This code also paves the way for the interface target to be exported and included with find_package
.
Hi,
I really like the [Boost].UT framework. It's clean and easy to use and does away with macros. Thanks for developing it!
Anyways, I want to use approval tests (https://github.com/approvals/ApprovalTests.cpp) in a project I'm working on, and I'm implementing the integration between UT <> ApprovalTests.cpp. I followed the guidelines (https://github.com/approvals/ApprovalTests.cpp/blob/master/doc/SupportingNewTestFramework.md#top) and the UT reporter example (https://godbolt.org/z/gsAPKg). I made this simple example:
#include <string>
#include "ut.hpp"
#include "ApprovalTests.hpp"
namespace ut = boost::ut;
namespace ApprovalTests
{
namespace cfg {
class reporter : public ut::reporter<ut::printer> {
private:
TestName currentTest;
public:
auto on(ut::events::test_begin test_begin) -> void {
std::string name = test_begin.name;
currentTest.sections.emplace_back(name);
ut::reporter<ut::printer>::on(test_begin);
}
auto on(ut::events::test_run test_run) -> void {
ut::reporter<ut::printer>::on(test_run);
}
auto on(ut::events::test_skip test_skip) -> void { ut::reporter<ut::printer>::on(test_skip); }
auto on(ut::events::test_end test_end) -> void {
while (!currentTest.sections.empty()) {
currentTest.sections.pop_back();
}
ut::reporter<ut::printer>::on(test_end);
}
template <class TMsg>
auto on(ut::events::log<TMsg>) -> void {}
template <class TLocation, class TExpr>
auto on(ut::events::assertion_pass<TLocation, TExpr> location) -> void {
currentTest.setFileName(location.location.file_name());
ApprovalTestNamer::currentTest(¤tTest);
ut::reporter<ut::printer>::on(location);
}
template <class TLocation, class TExpr>
auto on(ut::events::assertion_fail<TLocation, TExpr> fail) -> void { ut::reporter<ut::printer>::on(fail); }
auto on(ut::events::fatal_assertion fatal) -> void { ut::reporter<ut::printer>::on(fatal); }
auto on(ut::events::exception exception) -> void { ut::reporter<ut::printer>::on(exception); }
auto on(ut::events::summary summary) -> void { ut::reporter<ut::printer>::on(summary); }
};
} // namespace cfg
}
template <>
auto ut::cfg<ut::override> = ut::runner<ApprovalTests::cfg::reporter>{};
int main()
{
using namespace ut;
using namespace ApprovalTests;
"Approval"_test = []() {
//expect(true);
expect(nothrow([] { Approvals::verify("Approval Tests can verify text via the golder master method"); }));
};
}
The issue is unless the line containing expect(true); is uncommented it doesn't work. It complains that currentTest was not configured. This seems more like a UT issue than an ApprovalTests issue.
Can anyone help? I'm not familiar enough with the UT code logic to track this issue. Or am I wrong and this is an ApprovalTests.cpp issue?
I appreciate any help.
This is a report about some issues that occurred as I worked through the Tutorial.
OS: bento/ubuntu-20.04
Compiler: gcc version 9.3.0 (Ubuntu 9.3.0-10ubuntu2)
c++ -> x86_64-linux-gnu-g++-9
./ut-tutorial/
|
├── ut.hpp
|
└── main.cpp
import boost.ut;
int main() { }
$ c++ -std=c++2a main.cpp && ./a.out
main.cpp:1:1: error: ‘import’ does not name a type
1 | import boost.ut;
| ^~~~~~
Compilation with flag -std=c++2a
and statement import boost.ut;
failed.
#include "ut.hpp"
int main() { }
$ c++ -std=c++17 main.cpp
In file included from main.cpp:1:
ut.hpp: In member function ‘void boost::ut::v1_1_7::bdd::gherkin::steps::next(const TPattern&)’:
ut.hpp:2457:22: warning: range-based ‘for’ loops with initializer only available with ‘-std=c++2a’ or ‘-std=gnu++2a’
2457 | for (auto i = 0; const auto& step : gherkin_) {
| ^~~~~
Compilation with flag "-std=c++17" results in a warning flag.
$ c++ -std=c++2a main.cpp && ./a.out
-bash: ./a.out: Permission denied
$ ./a.out
$
The tutorial statement $CXX main.cpp && ./a.out
assumes something about permissions I needed to separate the commands. I found nothing on Google on how to address this. How did the combined command work for you?
⋮
⋮
#include "ut.hpp"
#include <vector>
int main() {
using namespace boost::ut;
"[vector]"_test = [] {
std::vector<int> v(5);
!expect(5_ul == size(v));
should("resize bigger") = [v] {
mut(v).resize(10);
expect(10_ul == size(v));
};
!expect(5_ul == size(v));
should("resize smaller") = [=]() mutable {
v.resize(0);
expect(0_ul == size(v));
};
};
}
$ c++ -std=c++2a main.cpp
$ ./a.out
All tests passed (4 asserts in 1 tests)
$
$ c++ -std=c++2a -Wall main.cpp
In file included from main.cpp:1:
ut.hpp: In instantiation of ‘void boost::ut::v1_1_7::bdd::gherkin::steps::next(const TPattern&) [with TPattern = std::__cxx11::basic_string<char>]’:
ut.hpp:2374:35: required from here
ut.hpp:2458:15: warning: comparison of integer expressions of different signedness: ‘int’ and ‘long unsigned int’ [-Wsign-compare]
2458 | if (i++ == step_) {
| ~~~~^~~~~~~~
$ ./a.out
All tests passed (4 asserts in 1 tests)
$
Compilation with warning flag "-Wall" resulted in a warning about comparing signed and unsigned integers.
Expected example/reporters to be built and run without a problem
It segfaults with the following backtrace
#112224 0x0000000000407b2d in boost::ut::v1_1_7::detail::neq_<std::basic_string_view<char, std::char_traits >, std::basic_string_view<char, std::char_traits > >::neq_ (this=0x7fffffffccb0, lhs=..., rhs=...) at ../include/boost/ut.hpp:766
#112225 0x0000000000406fa8 in boost::ut::v1_1_7::operators::operator!= (lhs=..., rhs=...) at ../include/boost/ut.hpp:1860
#112226 0x0000000000407abe in ZZN5boost2ut6v1_1_76detail4neq_ISt17basic_string_viewIcSt11char_traitsIcEES7_EC4ERKS7_SA_ENKUlvE_clEv (this=0x7fffffffcd90) at ../include/boost/ut.hpp:764
#112227 0x0000000000407b2d in boost::ut::v1_1_7::detail::neq<std::basic_string_view<char, std::char_traits >, std::basic_string_view<char, std::char_traits > >::neq_ (this=0x7fffffffcd90, lhs=..., rhs=...) at ../include/boost/ut.hpp:766
#112228 0x0000000000406fa8 in boost::ut::v1_1_7::operators::operator!= (lhs=..., rhs=...) at ../include/boost/ut.hpp:1860
#112229 0x00000000004066dd in operator() (__closure=0x0) at ../example/cfg/reporter.cpp:41
#112230 0x000000000040671d in _FUN () at ../example/cfg/reporter.cpp:42
#112231 0x00000000004072d7 in boost::ut::v1_1_7::events::test<void ()(), boost::ut::v1_1_7::none>::run_impl (test=0x40670f <_FUN()>) at ../include/boost/ut.hpp:504
#112232 0x0000000000407301 in boost::ut::v1_1_7::events::test<void ()(), boost::ut::v1_1_7::none>::operator() (this=0x7fffffffcf10) at ../include/boost/ut.hpp:500
#112233 0x0000000000407504 in boost::ut::v1_1_7::runner<cfg::reporter, 16>::on<void ()(), boost::ut::v1_1_7::none> (this=0x60c280 <boost::ut::v1_1_7::cfgboost::ut::v1_1_7::override>, test=...) at ../include/boost/ut.hpp:1364
#112234 0x0000000000407678 in boost::ut::v1_1_7::detail::on<, boost::ut::v1_1_7::events::test<void ()(), boost::ut::v1_1_7::none> > (event=<unknown type in /home/furkan/.local/cpp/ut/build/example/reporter, CU 0x0, DIE 0xaa46>) at ../include/boost/ut.hpp:1563
#112235 0x00000000004076d7 in boost::ut::v1_1_7::detail::test::operator=<>(boost::ut::v1_1_7::detail::test_location<void (*)()>) (this=0x7fffffffd030, test=...) at ../include/boost/ut.hpp:1586
#112236 0x0000000000406794 in main () at ../example/cfg/reporter.cpp:39
Edit: When I comment those operators, this time I get the same error in example/expect but this time string instead of string_view
Could you add an option to disable the compilation of the examples/tests so that a make install just installs the header into a desired prefix with no compilation steps?
The following should compile and work:
"test bool"_test = [] {
std::unique_ptr<int> ptr;
expect(ptr);
};
This does not compile:
/home/user/working/bsl/tests/ifarray.cpp: In lambda function:
/home/user/working/bsl/tests/ifarray.cpp:66:19: error: no matching function for call to ‘expect(std::unique_ptr<int>&)’
66 | expect(ptr);
| ^
In file included from /home/user/working/bsl/tests/ifarray.cpp:26:
/home/user/working/bsl/tests/ut.h:1606:9: note: candidate: ‘template<class TExpr, typename std::enable_if<(is_op_v<TExpr> || is_same_v<bool, TExpr>), int>::type <anonymous> > constexpr auto boost::ut::v1_0_0::expect(const TExpr&, const std::experimental::fundamentals_v2::source_location&)’
1606 | expect(
| ^~~~~~
/home/user/working/bsl/tests/ut.h:1606:9: note: template argument deduction/substitution failed:
/home/user/working/bsl/tests/ut.h:1604:79: error: no type named ‘type’ in ‘struct std::enable_if<false, int>’
1604 | type_traits::is_op_v<TExpr> or std::is_same_v<bool, TExpr>> = 0>
| ^
make[2]: *** [tests/CMakeFiles/test_ifarray.dir/build.make:63: tests/CMakeFiles/test_ifarray.dir/ifarray.cpp.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:246: tests/CMakeFiles/test_ifarray.dir/all] Error 2
make: *** [Makefile:130: all] Error 2
Printing values when a test fails doesn't seem to work with BOOST_UT_FORWARD
.
Given the two files:
a.cpp
:
#define BOOST_UT_IMPLEMENTATION
#include <boost/ut.hpp>
int main () {}
and b.cpp
#define BOOST_UT_FORWARD
#include <boost/ut.hpp>
using namespace boost::ut;
suite b = []
{
expect (that % 1 == 2);
};
and the command line
g++ -std=c++2a -o a a.cpp b.cpp && ./a
I get
b.cpp:8:FAILED [false]
===============================================================================
tests: 0 | 0 failed
asserts: 1 | 0 passed | 1 failed
The values of the test are not printed. Move the test into a.cpp
(or put some tests into this file) and the values are printed. E.g. replace a.cpp
above with
#define BOOST_UT_IMPLEMENTATION
#include <boost/ut.hpp>
using namespace boost::ut;
suite a = []
{
expect (that % 1 == 2);
};
int main () {}
and the result is
a.cpp:6:FAILED [1 == 2]
b.cpp:6:FAILED [1 == 2]
===============================================================================
tests: 0 | 0 failed
asserts: 2 | 0 passed | 2 failed
i.e. both tests print their values.
A fatal assertion with logged output for failure.
expect((true == false) >> fatal) << "Incorrect!";
When the assertion fails, "Incorrect!" should be printed.
FAILED [false] Incorrect!
When the assertion fails, "Incorrect!" is not printed.
FAILED [false]
On https://github.com/boost-experimental/ut/blob/dde2cba5123444faa82e5c17fcd99f68fe991d89/include/boost/ut.hpp#L176 is was checked for emptiness. So, it is redundant on line 181
All tests passed (1 asserts in 1 tests)
[32mAll tests passed[0m (1 asserts in 1 tests)
1.Download a modern Mingw (GCC-10/Clang-11) compiler from http://winlibs.com/
1.No installation required
1.Run test: "run"_test = [] {expect(42_i==42);};
Thanks!
The tests should allow the extraction of coverage data.
That's not the case.
Coverage data is extracted from the rest of the program (so it's not a problem of options) but not from anything that runs within the tests. It worked just fine with doctest too but nothing with UT.
The gcno / gcda files are found just fine by lcov, it captures everything that ran outside of UT, but nothing from inside (either from a suite or not).
Maybe an option I missed somewhere?
Not sure, it fails every time on my system and I haven't done anything particularly strange.
I'm using exclusively the BDD but tests made outside of BDD don't show up either.
ut::log
supporting at least std::string
.
ut::log
supports only compile-time known char[]
string literals.
utility::string_view
duplicates some std::string_view
functionality, but is incomplete and not convertible.
I am not a language lawyer, but I think it would be nice if mut was documented in a sense that if it is technically UB(as used in examples in tutorial), but it works in practice then that is is mentioned (in docs or code comments).
cppref says this:
const_cast makes it possible to form a reference or pointer to non-const type that is actually referring to a const object or a reference or pointer to non-volatile type that is actually referring to a volatile object. Modifying a const object through a non-const access path and referring to a volatile object through a non-volatile glvalue results in undefined behavior.
Apologies in advance if I misunderstood the cppref quote.
The runner example should build and run successfully.
The runner example program segfaults.
In case you're interested in upcoming versions of GCC, ut fails to compile with g++ (GCC) 10.0.0 20200108 (experimental)
$ g++ -std=c++2a -Wall -Wextra -I../../external/boost_experimental/ut/include -c -o tests.o tests.cpp
In file included from tests.cpp:5:
../../external/boost_experimental/ut/include/boost/ut.hpp: In instantiation of ‘constexpr boost::ut::v1_1_6::detail::neq_<TLhs, TRhs>::neq_(const TLhs&, const TRhs&) [with TLhs = std::basic_string_view<char>; TRhs = std::basic_string_view<char>]’:
../../external/boost_experimental/ut/include/boost/ut.hpp:1747:31: required from here
../../external/boost_experimental/ut/include/boost/ut.hpp:614:30: error: return type of ‘constexpr auto boost::ut::v1_1_6::operators::operator==(std::string_view, std::string_view)’ is not ‘bool’
614 | return get(lhs_) != get(rhs_);
| ~~~~~~~~~~^~~~~~~~~~~~
../../external/boost_experimental/ut/include/boost/ut.hpp:614:30: note: used as rewritten candidate for comparison of ‘std::basic_string_view<char>’ and ‘std::basic_string_view<char>’
../../external/boost_experimental/ut/include/boost/ut.hpp:616:12: error: cannot convert ‘<brace-enclosed initializer list>’ to ‘const bool’ in initialization
616 | }()} {}
| ^
make: *** [<builtin>: tests.o] Error 1
At first glance I'd say that the call to operator!=(std::string_view, std::string_view)
in neq_
uses the overload of operator==(std::string_view, std::string_view
defined just above, which of course doesn't return a bool
. Adding a using std::operator==
solves the issue.
Now as to why GCC 9 doesn't complain, I have no idea.
The ApprovalTests.cpp UT integration tests pass with the latest version of this project on macOS.
This combination passes:
On macOS, with both g++9 homebrew and clang-9 MacPorts, using 515c080, the UT tests all fail with:
*****************************************************************************
* *
* Welcome to Approval Tests.
*
* There seems to be a problem with your build configuration.
* We cannot find the test source file at:
* unknown
*
* For details on how to fix this, please visit:
* https://github.com/approvals/ApprovalTests.cpp/blob/master/doc/TroubleshootingMisconfiguredBuild.md
* *
*****************************************************************************
ApprovalTests.cpp/third_party/ut/include/boost/ut.hpp
UT_Tests
It's failing for me with:
CMAKE_VERSION = 3.16.2
CMAKE_GENERATOR = Unix Makefiles
CMAKE_SOURCE_DIR = /Users/clare/Documents/develop/ApprovalTests/ApprovalTests.cpp
CMAKE_CURRENT_SOURCE_DIR = /Users/clare/Documents/develop/ApprovalTests/ApprovalTests.cpp
CMAKE_CURRENT_BINARY_DIR = /Users/clare/Documents/develop/ApprovalTests/ApprovalTests.cpp/cmake-build-debug-g-9-brew
CMAKE_CXX_COMPILER = /usr/local/Cellar/gcc/9.2.0_2/bin/g++-9
CMAKE_CXX_COMPILER_ID = GNU
CMAKE_CXX_COMPILER_VERSION = 9.2.0
CMAKE_UNITY_BUILD =
and
CMAKE_VERSION = 3.16.2
CMAKE_GENERATOR = Unix Makefiles
CMAKE_SOURCE_DIR = /Users/clare/Documents/develop/ApprovalTests/ApprovalTests.cpp
CMAKE_CURRENT_SOURCE_DIR = /Users/clare/Documents/develop/ApprovalTests/ApprovalTests.cpp
CMAKE_CURRENT_BINARY_DIR = /Users/clare/Documents/develop/ApprovalTests/ApprovalTests.cpp/cmake-build-debug-clang-mp-90-brew
CMAKE_CXX_COMPILER = /opt/local/bin/clang++-mp-9.0
CMAKE_CXX_COMPILER_ID = Clang
CMAKE_CXX_COMPILER_VERSION = 9.0.1
CMAKE_UNITY_BUILD =
There are not many differences between 8004290 and 515c080
I suspect that the cause is this, as in both compiler cases, __APPLE__
is defined:
@@ -211,10 +211,10 @@ namespace reflection {
class source_location {
public:
[[nodiscard]] static constexpr auto current(
-#if (__GNUC__ >= 9 or __clang_major__ >= 9)
+#if ((__GNUC__ >= 9 or __clang_major__ >= 9) and not defined(__APPLE__))
const char* file = __builtin_FILE(), int line = __builtin_LINE()
#else
- const char* file = {}, int line = {}
+ const char* file = "unknown", int line = {}
#endif
) noexcept {
source_location sl{};
If this executes:
"unexcepted throw"_test = [] {
throw std::runtime_error("unexpected");
};
I would expect that the test would fail.
The test doesn't fail, and silently passed. All of the rest of the expect() in that test are ignored.
To receive no warnings when compiling.
I received the following warnings using GCC 9.1.1.
_deps/ut-src/include/boost/ut.hpp: In function ‘constexpr auto boost::ut::v1_1_7::operators::terse::operator==(const T&, const boost::ut::v1_1_7::operators::terse::detail::value_location<typename T::value_type>&)’:
_deps/ut-src/include/boost/ut.hpp:1974:11: warning: typedef ‘using type = using eq_t = class boost::ut::v1_1_7::detail::eq_<T, boost::ut::v1_1_7::operators::terse::detail::value_location<typename T::value_type> >’ locally defined but not used [-Wunused-local-typedefs]
1974 | using type = eq_t;
| ^~~~
_deps/ut-src/include/boost/ut.hpp: In function ‘constexpr auto boost::ut::v1_1_7::operators::terse::operator==(const boost::ut::v1_1_7::operators::terse::detail::value_location<typename T::value_type>&, const T&)’:
_deps/ut-src/include/boost/ut.hpp:1987:11: warning: typedef ‘using type = using eq_t = class boost::ut::v1_1_7::detail::eq_<boost::ut::v1_1_7::operators::terse::detail::value_location<typename T::value_type>, T>’ locally defined but not used [-Wunused-local-typedefs]
1987 | using type = eq_t;
| ^~~~
_deps/ut-src/include/boost/ut.hpp: In function ‘constexpr auto boost::ut::v1_1_7::operators::terse::operator!=(const T&, const boost::ut::v1_1_7::operators::terse::detail::value_location<typename T::value_type>&)’:
_deps/ut-src/include/boost/ut.hpp:2000:11: warning: typedef ‘using type = using neq_t = class boost::ut::v1_1_7::detail::neq_<T, boost::ut::v1_1_7::operators::terse::detail::value_location<typename T::value_type> >’ locally defined but not used [-Wunused-local-typedefs]
2000 | using type = neq_t;
| ^~~~
_deps/ut-src/include/boost/ut.hpp: In function ‘constexpr auto boost::ut::v1_1_7::operators::terse::operator!=(const boost::ut::v1_1_7::operators::terse::detail::value_location<typename T::value_type>&, const T&)’:
_deps/ut-src/include/boost/ut.hpp:2013:11: warning: typedef ‘using type = using neq_t = class boost::ut::v1_1_7::detail::neq_<boost::ut::v1_1_7::operators::terse::detail::value_location<typename T::value_type>, T>’ locally defined but not used [-Wunused-local-typedefs]
2013 | using type = neq_t;
| ^~~~
_deps/ut-src/include/boost/ut.hpp: In function ‘constexpr auto boost::ut::v1_1_7::operators::terse::operator>(const T&, const boost::ut::v1_1_7::operators::terse::detail::value_location<typename T::value_type>&)’:
_deps/ut-src/include/boost/ut.hpp:2026:11: warning: typedef ‘using type = using gt_t = class boost::ut::v1_1_7::detail::gt_<T, boost::ut::v1_1_7::operators::terse::detail::value_location<typename T::value_type> >’ locally defined but not used [-Wunused-local-typedefs]
2026 | using type = gt_t;
| ^~~~
_deps/ut-src/include/boost/ut.hpp: In function ‘constexpr auto boost::ut::v1_1_7::operators::terse::operator>(const boost::ut::v1_1_7::operators::terse::detail::value_location<typename T::value_type>&, const T&)’:
_deps/ut-src/include/boost/ut.hpp:2039:11: warning: typedef ‘using type = using gt_t = class boost::ut::v1_1_7::detail::gt_<boost::ut::v1_1_7::operators::terse::detail::value_location<typename T::value_type>, T>’ locally defined but not used [-Wunused-local-typedefs]
2039 | using type = gt_t;
| ^~~~
_deps/ut-src/include/boost/ut.hpp: In function ‘constexpr auto boost::ut::v1_1_7::operators::terse::operator>=(const T&, const boost::ut::v1_1_7::operators::terse::detail::value_location<typename T::value_type>&)’:
_deps/ut-src/include/boost/ut.hpp:2052:11: warning: typedef ‘using type = using ge_t = class boost::ut::v1_1_7::detail::ge_<T, boost::ut::v1_1_7::operators::terse::detail::value_location<typename T::value_type> >’ locally defined but not used [-Wunused-local-typedefs]
2052 | using type = ge_t;
| ^~~~
_deps/ut-src/include/boost/ut.hpp: In function ‘constexpr auto boost::ut::v1_1_7::operators::terse::operator>=(const boost::ut::v1_1_7::operators::terse::detail::value_location<typename T::value_type>&, const T&)’:
_deps/ut-src/include/boost/ut.hpp:2065:11: warning: typedef ‘using type = using ge_t = class boost::ut::v1_1_7::detail::ge_<boost::ut::v1_1_7::operators::terse::detail::value_location<typename T::value_type>, T>’ locally defined but not used [-Wunused-local-typedefs]
2065 | using type = ge_t;
| ^~~~
_deps/ut-src/include/boost/ut.hpp: In function ‘constexpr auto boost::ut::v1_1_7::operators::terse::operator<(const T&, const boost::ut::v1_1_7::operators::terse::detail::value_location<typename T::value_type>&)’:
_deps/ut-src/include/boost/ut.hpp:2078:11: warning: typedef ‘using type = using lt_t = class boost::ut::v1_1_7::detail::lt_<T, boost::ut::v1_1_7::operators::terse::detail::value_location<typename T::value_type> >’ locally defined but not used [-Wunused-local-typedefs]
2078 | using type = lt_t;
| ^~~~
_deps/ut-src/include/boost/ut.hpp: In function ‘constexpr auto boost::ut::v1_1_7::operators::terse::operator<(const boost::ut::v1_1_7::operators::terse::detail::value_location<typename T::value_type>&, const T&)’:
_deps/ut-src/include/boost/ut.hpp:2091:11: warning: typedef ‘using type = using lt_t = class boost::ut::v1_1_7::detail::lt_<boost::ut::v1_1_7::operators::terse::detail::value_location<typename T::value_type>, T>’ locally defined but not used [-Wunused-local-typedefs]
2091 | using type = lt_t;
| ^~~~
_deps/ut-src/include/boost/ut.hpp: In function ‘constexpr auto boost::ut::v1_1_7::operators::terse::operator<=(const T&, const boost::ut::v1_1_7::operators::terse::detail::value_location<typename T::value_type>&)’:
_deps/ut-src/include/boost/ut.hpp:2104:11: warning: typedef ‘using type = using le_t = class boost::ut::v1_1_7::detail::le_<T, boost::ut::v1_1_7::operators::terse::detail::value_location<typename T::value_type> >’ locally defined but not used [-Wunused-local-typedefs]
2104 | using type = le_t;
| ^~~~
_deps/ut-src/include/boost/ut.hpp: In function ‘constexpr auto boost::ut::v1_1_7::operators::terse::operator<=(const boost::ut::v1_1_7::operators::terse::detail::value_location<typename T::value_type>&, const T&)’:
_deps/ut-src/include/boost/ut.hpp:2117:11: warning: typedef ‘using type = using le_t = class boost::ut::v1_1_7::detail::le_<boost::ut::v1_1_7::operators::terse::detail::value_location<typename T::value_type>, T>’ locally defined but not used [-Wunused-local-typedefs]
2117 | using type = le_t;
| ^~~~
_deps/ut-src/include/boost/ut.hpp: In function ‘constexpr auto boost::ut::v1_1_7::operators::terse::operator&&(const TLhs&, const TRhs&)’:
_deps/ut-src/include/boost/ut.hpp:2130:11: warning: typedef ‘using type = using and_t = class boost::ut::v1_1_7::detail::and_<typename TLhs::type, typename TRhs::type>’ locally defined but not used [-Wunused-local-typedefs]
2130 | using type = and_t;
| ^~~~
_deps/ut-src/include/boost/ut.hpp: In function ‘constexpr auto boost::ut::v1_1_7::operators::terse::operator||(const TLhs&, const TRhs&)’:
_deps/ut-src/include/boost/ut.hpp:2143:11: warning: typedef ‘using type = using or_t = class boost::ut::v1_1_7::detail::or_<typename TLhs::type, typename TRhs::type>’ locally defined but not used [-Wunused-local-typedefs]
2143 | using type = or_t;
| ^~~~
_deps/ut-src/include/boost/ut.hpp: In function ‘constexpr auto boost::ut::v1_1_7::operators::terse::operator!(const T&)’:
_deps/ut-src/include/boost/ut.hpp:2154:11: warning: typedef ‘using type = using not_t = class boost::ut::v1_1_7::detail::not_<typename T::type>’ locally defined but not used [-Wunused-local-typedefs]
2154 | using type = not_t;
| ^~~~
It seems like these warnings can occur by just using a very basic test case.
For instance, expect(true);
.
This does not occur for all of my test executables, only simple ones.
no warnings
WarningErrorEntry(path='/Users/clausklein/Workspace/cpp/ut/include/boost/ut.hpp', lineno='154', severity='warning', message='multiple declarations in a single statement reduces readability [readability-isolate-declaration]', column='3')
WarningErrorEntry(path='/Users/clausklein/Workspace/cpp/ut/include/boost/ut.hpp', lineno='261', severity='warning', message='do not declare C-style arrays, use std::array<> instead [modernize-avoid-c-arrays]', column='19')
WarningErrorEntry(path='/Users/clausklein/Workspace/cpp/ut/include/boost/ut.hpp', lineno='263', severity='warning', message='multiple declarations in a single statement reduces readability [readability-isolate-declaration]', column='3')
WarningErrorEntry(path='/Users/clausklein/Workspace/cpp/ut/include/boost/ut.hpp', lineno='285', severity='warning', message='do not declare C-style arrays, use std::array<> instead [modernize-avoid-c-arrays]', column='19')
WarningErrorEntry(path='/Users/clausklein/Workspace/cpp/ut/include/boost/ut.hpp', lineno='288', severity='warning', message='statement should be inside braces [readability-braces-around-statements]', column='25')
WarningErrorEntry(path='/Users/clausklein/Workspace/cpp/ut/include/boost/ut.hpp', lineno='298', severity='warning', message='do not declare C-style arrays, use std::array<> instead [modernize-avoid-c-arrays]', column='19')
WarningErrorEntry(path='/Users/clausklein/Workspace/cpp/ut/include/boost/ut.hpp', lineno='300', severity='warning', message='statement should be inside braces [readability-braces-around-statements]', column='25')
WarningErrorEntry(path='/Users/clausklein/Workspace/cpp/ut/include/boost/ut.hpp', lineno='517', severity='warning', message="function 'what' should be marked [[nodiscard]] [modernize-use-nodiscard]", column='3')
WarningErrorEntry(path='/Users/clausklein/Workspace/cpp/ut/include/boost/ut.hpp', lineno='540', severity='warning', message='do not implicitly decay an array into a pointer; consider using gsl::array_view or an explicit cast instead [hicpp-no-array-decay]', column='10')
WarningErrorEntry(path='/Users/clausklein/Workspace/cpp/ut/include/boost/ut.hpp', lineno='546', severity='warning', message="return type 'const type_<TOther>' is 'const'-qualified at the top level, which may reduce code readability without improving const correctness [readability-const-return-type]", column='17')
WarningErrorEntry(path='/Users/clausklein/Workspace/cpp/ut/include/boost/ut.hpp', lineno='929', severity='warning', message='do not declare C-style arrays, use std::array<> instead [modernize-avoid-c-arrays]', column='11')
WarningErrorEntry(path='/Users/clausklein/Workspace/cpp/ut/include/boost/ut.hpp', lineno='1314', severity='warning', message="do not use 'else' after 'return' [readability-else-after-return]", column='7')
WarningErrorEntry(path='/Users/clausklein/Workspace/cpp/ut/include/boost/ut.hpp', lineno='1329', severity='warning', message="throwing an exception whose type 'events::fatal_assertion' is not derived from 'std::exception' [hicpp-exception-baseclass]", column='11')
WarningErrorEntry(path='/Users/clausklein/Workspace/cpp/ut/include/boost/ut.hpp', lineno='1428', severity='warning', message="method 'operator=' can be made const [readability-make-member-function-const]", column='18')
WarningErrorEntry(path='/Users/clausklein/Workspace/cpp/ut/include/boost/ut.hpp', lineno='1454', severity='warning', message='do not implicitly decay an array into a pointer; consider using gsl::array_view or an explicit cast instead [hicpp-no-array-decay]', column='26')
WarningErrorEntry(path='/Users/clausklein/Workspace/cpp/ut/test/ft/test_suite_2.cpp', lineno='14', severity='warning', message="throwing an exception whose type 'int' is not derived from 'std::exception' [hicpp-exception-baseclass]", column='62')
WarningErrorEntry(path='/Users/clausklein/Workspace/cpp/ut/benchmark/test.cpp', lineno='10', severity='warning', message="function 'main' exceeds recommended size/complexity thresholds [readability-function-size]", column='5')
WarningErrorEntry(path='/Users/clausklein/Workspace/cpp/ut/include/boost/ut.hpp', lineno='546', severity='warning', message="return type 'const type_<int>' is 'const'-qualified at the top level, which may reduce code readability without improving const correctness [readability-const-return-type]", column='17')
WarningErrorEntry(path='/Users/clausklein/Workspace/cpp/ut/include/boost/ut.hpp', lineno='1446', severity='warning', message='do not implicitly decay an array into a pointer; consider using gsl::array_view or an explicit cast instead [hicpp-no-array-decay]', column='28')
WarningErrorEntry(path='/Users/clausklein/Workspace/cpp/ut/include/boost/ut.hpp', lineno='1563', severity='warning', message='do not implicitly decay an array into a pointer; consider using gsl::array_view or an explicit cast instead [hicpp-no-array-decay]', column='25')
WarningErrorEntry(path='/Users/clausklein/Workspace/cpp/ut/include/boost/ut.hpp', lineno='2204', severity='warning', message='do not implicitly decay an array into a pointer; consider using gsl::array_view or an explicit cast instead [hicpp-no-array-decay]', column='44')
WarningErrorEntry(path='/Users/clausklein/Workspace/cpp/ut/test/ut/ut.cpp', lineno='71', severity='warning', message='use emplace_back instead of push_back [modernize-use-emplace]', column='25')
WarningErrorEntry(path='/Users/clausklein/Workspace/cpp/ut/test/ut/ut.cpp', lineno='73', severity='warning', message='use emplace_back instead of push_back [modernize-use-emplace]', column='25')
WarningErrorEntry(path='/Users/clausklein/Workspace/cpp/ut/test/ut/ut.cpp', lineno='75', severity='warning', message='use emplace_back instead of push_back [modernize-use-emplace]', column='25')
WarningErrorEntry(path='/Users/clausklein/Workspace/cpp/ut/test/ut/ut.cpp', lineno='96', severity='warning', message="throwing an exception whose type 'ut::events::fatal_assertion' is not derived from 'std::exception' [hicpp-exception-baseclass]", column='11')
WarningErrorEntry(path='/Users/clausklein/Workspace/cpp/ut/test/ut/ut.cpp', lineno='130', severity='warning', message="throwing an exception whose type 'int' is not derived from 'std::exception' [hicpp-exception-baseclass]", column='37')
WarningErrorEntry(path='/Users/clausklein/Workspace/cpp/ut/test/ut/ut.cpp', lineno='238', severity='warning', message="function 'main' exceeds recommended size/complexity thresholds [readability-function-size]", column='5')
WarningErrorEntry(path='/Users/clausklein/Workspace/cpp/ut/test/ut/ut.cpp', lineno='314', severity='warning', message="the 'empty' method should be used to check for emptiness instead of comparing to an empty object [readability-container-size-empty]", column='19')
WarningErrorEntry(path='/Users/clausklein/Workspace/cpp/ut/test/ut/ut.cpp', lineno='479', severity='warning', message="'auto old_cout' can be declared as 'auto *old_cout' [readability-qualified-auto]", column='7')
WarningErrorEntry(path='/Users/clausklein/Workspace/cpp/ut/test/ut/ut.cpp', lineno='480', severity='warning', message="'auto old_cerr' can be declared as 'auto *old_cerr' [readability-qualified-auto]", column='7')
WarningErrorEntry(path='/Users/clausklein/Workspace/cpp/ut/test/ut/ut.cpp', lineno='1006', severity='warning', message="the 'empty' method should be used to check for emptiness instead of comparing to an empty object [readability-container-size-empty]", column='14')
WarningErrorEntry(path='/Users/clausklein/Workspace/cpp/ut/test/ut/ut.cpp', lineno='1108', severity='warning', message="throwing an exception whose type 'int' is not derived from 'std::exception' [hicpp-exception-baseclass]", column='34')
WarningErrorEntry(path='/Users/clausklein/Workspace/cpp/ut/test/ut/ut.cpp', lineno='1109', severity='warning', message="throwing an exception whose type 'int' is not derived from 'std::exception' [hicpp-exception-baseclass]", column='54')
WarningErrorEntry(path='/Users/clausklein/Workspace/cpp/ut/test/ut/ut.cpp', lineno='1113', severity='warning', message="throwing an exception whose type 'int' is not derived from 'std::exception' [hicpp-exception-baseclass]", column='35')
WarningErrorEntry(path='/Users/clausklein/Workspace/cpp/ut/test/ut/ut.cpp', lineno='1150', severity='warning', message='statement should be inside braces [readability-braces-around-statements]', column='28')
WarningErrorEntry(path='/Users/clausklein/Workspace/cpp/ut/test/ut/ut.cpp', lineno='1150', severity='warning', message="throwing an exception whose type 'int' is not derived from 'std::exception' [hicpp-exception-baseclass]", column='35')
WarningErrorEntry(path='/Users/clausklein/Workspace/cpp/ut/test/ut/ut.cpp', lineno='1161', severity='warning', message="throwing an exception whose type 'events::exception' is not derived from 'std::exception' [hicpp-exception-baseclass]", column='15')
WarningErrorEntry(path='/Users/clausklein/Workspace/cpp/ut/test/ut/ut.cpp', lineno='1164', severity='warning', message="throwing an exception whose type 'int' is not derived from 'std::exception' [hicpp-exception-baseclass]", column='41')
WarningErrorEntry(path='/Users/clausklein/Workspace/cpp/ut/test/ut/ut.cpp', lineno='1605', severity='warning', message="function 'get' should be marked [[nodiscard]] [modernize-use-nodiscard]", column='7')
run-clang-tidy.py -p build -header-filter='.*' -checks='-cert-*' test
To execute the test and the assert inside.
I don't know how to co_await inside a test lambda, without being a coroutine.
All tests passed (0 asserts in 1 tests)
"hello coro"_test = []() -> ::cppcoro::task<> {
::boost::ut::log << "Not printed, because never reaches here.";
auto task_lambda = []() -> ::cppcoro::task<int> { co_return 43; }();
int coro_43 = co_await task_lambda;
expect(42_i == coro_43);
};
There are a number of cases when you want unit tests to run earlier than the program exits. E.g. an application may want to run and report failing unit tests on startup. While this is doable with a custom runner, it should be fairly straightforward to add a method to replicate the default runner's destructor behavior. Something that effectively allows:
int main(int argc, char **argv)
{
boost::ut::cfg<>.run(); // (1)
boost::ut::cfg<>.on(boost::ut::events::summary{}); // (2) currently not accessible
...
Possible APIs could be a bool run_and_report()
method to replicate the current destructor functionality or more flexible a void report_summary()
to just do (2).
A file test.cpp
contains
#include "ut.hpp"
int main() {
using namespace boost::ut;
test("set") = []{
test("one") = []{ expect(1==1); };
test("two") = []{ expect(2==2); };
};
}
Compilation and execution is
$ g++ -std=c++20 -Wall -Wextra -pedantic -c -o test.o test.cpp
$ g++ -std=c++20 -Wall -Wextra -pedantic -o test test.o
$ ./test
All tests passed (2 asserts in 2 tests)
All other things being equal,
$ ./test
All tests passed (2 asserts in 1 tests)
Project Directory Structure
./ut-test/
|
├── ut.hpp
|
└── test.cpp
See Expected Behavior.
OS: bento/ubuntu-20.04
Compiler: gcc version 10.0.1 (Ubuntu 10-20200411-0ubuntu1)
g++ -> x86_64-linux-gnu-gcc-10
In BDD syntax, the standard output is undesirable. For example, something like
#include "ut.hpp"
int main() {
using namespace boost::ut;
feature("sort") = []{
scenario("small arrays") = [] {
given("trivial arrays") = [] {
when("the array is empty") = [] { ... };
when("the array is of size 1") = [] { ... };
};
given("array of size 2") = { ... };
given("array of size 3") = { ... };
⋮
};
scenario("significant arrays") = [] {
given("array of size 20") = { ... }
⋮
};
⋮
scenario("mega arrays") = [] {
...
};
};
}
can result in the output
$ ./test
All tests passed (23192 asserts in 1 tests)
Add an optional parameter to the test function that will cause it to be counted as a test:
... auto test = [](const auto name, const bool isTest) { ...
Then, one could apply it as follows
#include "ut.hpp"
int main() {
using namespace boost::ut;
test("set") = []{
test("one", true) = []{ expect(1==1); };
test("two", true) = []{ expect(2==2); };
};
feature("sort") = []{
scenario("small arrays") = [] {
given("trivial arrays") = [] {
when("the array is empty", true) = [] { ... };
when("the array is of size 1", true) = [] { ... };
⋮
};
⋮
};
⋮
};
}
Employ a more elegant solution or update the tutorial if my understanding is off.
Thanks!
Something similar to
BOOST_REQUIRE_EXCEPTION(expression, exception_type, predicate);
In Boost.Test (at least) there is a useful opportunity to inspect exception object itself.
BOOST_REQUIRE_EXCEPTION
It seems now ut can not simulate this behaviour by overriding runner/reporter/... In the case of
expect(throws<std::runtime_error>([] { throw std::runtime_error{""}; }))
by the time runner::on(ut::events::assertion<TExpr>)
fires, a decision on the status of test validation has already been made, without the ability to somehow intervene either before or after for completion.
As a workaround I just embed a 'predicate function' to ut::throws
and ut::detail::throws_
as overriding variants.
template<class T>
void func_1(T const & e)
{
boost::ut::expect(e.code() == ex_code_1);
}
...
"test1"_test = [] {
/* some 'when' precondition */
expect(throws<some_exception>(
[] { raising_func(); },
func_1) ); // fails at func_1 body line if raising_func throws with ex_code_2
/* some other 'when' precondition */
expect(throws<some_exception>(
[] { raising_func(); },
func_2) ); // passes if raising_func throws with ex_code_2
expect((throws<some_exception>([] { throw some_exception(); })) >> fatal); // passes
expect((throws<some_exception>([] { raising_func(); })) >> fatal); // passes if raising any "some_exception" (or fail at this particular line)
expect((throws([] { throw 1; })) >> fatal); // as always!
}
And my question is: Will similar(somewhat) functionality be introduced to ut sometime?
Any plans on adding an expect_death test? One feature that I need badly is the ability to test for std::terminate() and continue on, and I am not sure how to best approach that as throwing from a std::terminate() handler is not an option when a function is marked as noexcept.
That it's easy to search text, and see the whole structure, of the docs on the front page:
https://github.com/boost-experimental/ut
Parallel Runner example should build without error.
The parallel_runner fails to build with GCC 9.1.1 (devtoolset-9) on RHEL 7 and with Mingw64 (GCC 9.2) on Windows 10.
The <execution>
header pulls in the exact same missing include on both platforms:
/opt/rh/devtoolset-9/root/usr/include/c++/9/pstl/parallel_backend_tbb.h:19:10: fatal error: tbb/blocked_range.h: No such file or directory
19 | #include <tbb/blocked_range.h>
| ^~~~~~~~~~~~~~~~~~~~~
Build the parallel_runner test on RHEL 7 using devtoolset-9 with C++20 enabled.
Or
Build the parallel_runner test on Mingw64 (GCC 9.2) with C++20 enabled.
Hi @krzysztof-jusiak - I have a patch for this, ready to submit as a PR... I thought it would be helpful to log the behaviour here to provide a more detailed description.
When there is an unexpected exception caught during the running of tests, the program should exit with a non-zero exit code.
When there is an unexpected exception caught during the running of tests, and no other assertion-type failures, there is a console message written out, but the program exits with exit code 0 - so ctest and CI systems see it as a passing test.
This is particularly important when using ctest --output-on-failure .
... ctest thinks the test passed, so it does not displaying any console output at all, and so the user sees only 100% success...
(This is the underlying cause of approvals/ApprovalTests.cpp#87.)
#include <boost/ut.hpp>
#include <stdexcept>
int result() {
throw std::runtime_error("no!");
}
int main() {
using namespace boost::ut;
"exceptions"_test = [] {
expect(result() == 1_i);
};
}
exception
programexit code 0
in the console output:.../ut-claremacrae/cmake-build-spaces/cmake-build-debug-gcc9-brew/example/exception
Running "exceptions"...
Unexpected exception with message:
no!
FAILED
===============================================================================
tests: 1 | 1 failed
asserts: 0 | 0 passed | 0 failed
Process finished with exit code 0
Observed on multiple different compilers and generators.
What do you think about making a [Boost-ext].UT package available in ConanCenter?
I would like to able to handle the dependency through Conan, and having it available in ConanCenter seems the easiest way to do so.
Their instructions for doing so are available here.
I found an example recipe for a header-only library on ConanCenter here.
This
#define BOOST_UT_FORWARD
#include <boost/ut.hpp>
using namespace boost::ut;
suite some_tests = []{
"x"_test = []{
log << "Hello";
};
};
does not compile. It does without #define BOOST_UT_FORWARD
.
ut should provide user-defined literals and type aliases for all fundamental types.
Currently there are user defined literals and type aliases for char
, unsigned char
, short
, unsigned short
, int
, unsigned
, long
, unsigned long
, long long
, float
, double
, long double
(and with slightly different behavior bool
).
But there's no such thing for signed char
or unsigned long long
.
I would even love to have user defined literals for the integer types from <cstdint>
, e.g. std::int64_t
or std::intmax_t
, and the two types std::size_t
and std::ptrdiff_t
from <cstddef>
. But I don't know ut well enough to know whether adding them makes sense.
#include <tuple>
#include "ut.hpp"
int main()
{
// === user-defined literals
using namespace boost::ut::literals;
std::ignore = 0_c; // char
std::ignore = 0_s; // signed short
std::ignore = 0_i; // signed int
std::ignore = 0_l; // signed long
std::ignore = 0_ll; // signed long long
std::ignore = 0_uc; // unsigned char
std::ignore = 0_us; // unsigned short
std::ignore = 0_u; // unsigned int
std::ignore = 0_ul; // unsigned long
std::ignore = 0._f; // float
std::ignore = 0._d; // double
std::ignore = 0._ld; // long double
std::ignore = "everything is true"_b;
// these two don't exist:
//std::ignore = 0_sc; // signed char
//std::ignore = 0_ull; // unsigned long long
// ==== type aliases
using namespace boost::ut;
std::ignore = _c(0); // char
std::ignore = _s(0); // signed short
std::ignore = _i(0); // signed int
std::ignore = _l(0); // signed long
std::ignore = _ll(0); // signed long long
std::ignore = _uc(0); // unsigned char
std::ignore = _us(0); // unsigned short
std::ignore = _u(0); // unsigned int
std::ignore = _ul(0); // unsigned long
std::ignore = _f(0); // float
std::ignore = _d(0); // double
std::ignore = _ld(0); // long double
std::ignore = _b(0); // bool
// these two don't exist:
//std::ignore = _sc(0); // signed char
//std::ignore = _ull(0); // unsigned long long
}
Several CMake options are likely to cause name collisions with similarly named variables, and variables are put into the CMake cache without consideration for parent projects.
Variables such as BUILD_TESTS
are usually prefixed with the project's name to avoid potential name collisions.
An example for [Boost].µT is UT_BUILD_TESTS
.
Projects incorporating [Boost].µT may wish to not have certain [Boost].µT-specific variables shown in the cache, like the BUILD_TESTS
option.
Best practice is to use a non-cache variable as this leaves the decision up to parent projects whether to make those variables available in the cache or not.
A common idiom used for testing is to automatically include the tests if the project is the top-level project or if the appropriate variable is enabled.
The example below demonstrates this behavior.
if(TEST_UT OR CMAKE_SOURCE_DIR STREQUAL PROJECT_SOURCE_DIR)
add_subdirectory(test)
endif()
There is no call to the option()
command, i.e.
option(TEST_UT "Enable building the tests" yes)
, as this populates the variable in the CMake cache.
Is there anything in place to allow me to do
expect( x.has_value() and that % x.value() == 10 );
in such a way that if the first part fails, the second isn't evaluated?
When I execute:
std::string msg = "The answer is: 42\n";
if (auto strm = std::ofstream("test.txt")) {
strm << msg;
}
I would expect the following:
The answer is: 42
I actually get:
{T, h, e, , a, n, s, w, e, r, , i, s, :, , 4, 2, }
If I comment out the unit test header, the above code works as expected.
I'd like to configure a parallel runner for a multi-translation unit program. Just took the example parallel_runner.cpp
, fit into my main.cpp
and expected the project to just build.
Tons of linker errors
parallel_runner.cpp
test.cpp
with tests[build] : && /usr/bin/c++ -g CMakeFiles/aoc.dir/main.cpp.o CMakeFiles/aoc.dir/test2.cpp.o -o aoc -lpthread -lssl -lcrypto -lgmp -ltbb && :
[build] /usr/bin/ld: /usr/bin/ld: DWARF error: could not find variable specification at offset befe
[build] CMakeFiles/aoc.dir/test2.cpp.o:(.bss._ZN5boost3ext2ut6v1_1_83cfgINS2_8overrideEJEEE[_ZN5boost3ext2ut6v1_1_83cfgINS2_8overrideEJEEE]+0x0): multiple definition of `boost::ext::ut::v1_1_8::cfg<boost::ext::ut::v1_1_8::override>'; /usr/bin/ld: DWARF error: could not find variable specification at offset c578
[build] CMakeFiles/aoc.dir/main.cpp.o:(.bss+0x0): first defined here
[build] collect2: error: ld returned 1 exit status
../example/tmp.cpp:19:12: error: 'boost::ut::v1_1_7::detail::eq_<boost::ut::v1_1_7::detail::integral_constant<42>, int>' is not a valid type for a template non-type parameter because it is not structural
19 | expect(constant<42_i == i> and type == type and
| ^~~~~~~~~~~~~~~~~~~
In file included from ../example/tmp.cpp:8:
../include/boost/ut.hpp:738:14: note: 'boost::ut::v1_1_7::detail::eq_<boost::ut::v1_1_7::detail::integral_constant<42>, int>::lhs_' is not public
738 | const TLhs lhs_{};
| ^~~~
en.cppreference.com states, for non-type template parameters
a literal class type with the following properties:
As a workaround, I've commented the ifdefs for Constant in line 2190
boost.ut
target seems to no longer be createdThat the target boost.ut
is still provided, to link against, when this project is obtained via add_subdirectory(), FetchContent() or similar...
#include <boost/ut.hpp>
Example code - this is how I obtain the boost.ut project - UtVersion is master:
FetchContent_Declare(boost.ut
GIT_REPOSITORY https://github.com/boost-ext/ut.git
GIT_TAG ${UtVersion})
FetchContent_MakeAvailable(boost.ut)
The repo goes on to use the target boost.ut
.
This directory and GitHub action exists to help the ApprovalTests.cpp project have early detection of any changes in test frameworks that might affect our users, when we do our next release.
This has been working for some months, and was still working on the 24th of Feb.
The next build, on 27th Feb at 01:10 GMT gave these errors
Fetching boost.ut...
CMake Error at CMakeLists.txt:137 (target_compile_options):
Cannot specify compile options for target "boost.ut" which is not built by
this project.
Building CXX object approvaltests.cpp_build/tests/UT_Tests/CMakeFiles/UT_Tests.dir/UTApprovalTestTests.cpp.o
In file included from /home/runner/work/ApprovalTests.cpp.CMakeSamples/ApprovalTests.cpp.CMakeSamples/ApprovalTests.cpp/ApprovalTests/ApprovalTests.hpp:68,
from /home/runner/work/ApprovalTests.cpp.CMakeSamples/ApprovalTests.cpp.CMakeSamples/ApprovalTests.cpp/tests/UT_Tests/UTApprovalTestTests.cpp:3:
/home/runner/work/ApprovalTests.cpp.CMakeSamples/ApprovalTests.cpp.CMakeSamples/ApprovalTests.cpp/ApprovalTests/../ApprovalTests/integrations/ut/UTApprovals.h:14:10: fatal error: boost/ut.hpp: No such file or directory
14 | #include <boost/ut.hpp>
| ^~~~~~~~~~~~~~
compilation terminated.
approvaltests.cpp_build/tests/UT_Tests/CMakeFiles/UT_Tests.dir/build.make:81: recipe for target 'approvaltests.cpp_build/tests/UT_Tests/CMakeFiles/UT_Tests.dir/UTApprovalTestTests.cpp.o' failed
Run this on a unix box with a recent version of either linux or gcc installed:
git clone https://github.com/claremacrae/ApprovalTests.cpp.CMakeSamples
cd ApprovalTests.cpp.CMakeSamples
git clone https://github.com/approvals/ApprovalTests.cpp.git
cd dev_approvals_fetch_content
./build.sh
Wait until all the dependencies have been cloned, and then one of the errors will occur - depending on whether you are using gcc or clang.
The failure was detected in a GitHub Actions build. The failing builds can be seen here:
https://github.com/claremacrae/ApprovalTests.cpp.CMakeSamples/actions/runs/604457152
It's likely that the ones that passed are skipping the building of boost.ut
Would it be possible to support creating microbenchmarks? It is not uncommon for test frameworks to incorporate microbenchmark support. Below are a couple of frameworks I'm aware of which closely tie microbenchmarking and unit testing.
I'm not aware of any libraries for benchmarking C++ code that are macro-free, and I figure I'm not the only one that would have a similar interest in macro-free microbenchmarking.
For this to work with ExternalProject_Add, it really should have an install target.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.