Optimizing compilation time for the test suite -- an analysis about corrade HOT 3 OPEN

Hi, thank you, useful analysis!

Sidetracked with compilation times, haha, I know that feeling very well :)

1. I have some work-in-progress changes for the TestSuite macros in #140. Got some results there but nothing significant enough to make that patch worth merging alone, still need to look into that more. OTOH the test code makes up maybe 90% of the codebase, the test files are heavy on their own, often being several thousands lines long with the CORRADE_COMPARE* macros used on almost every second line, so those showing up high in the profiler output isn't unexpected (I'd be surprised if those didn't show up high).

   Unfortunately I don't have a comparison to how gtest or other popular testing frameworks would behave at the same scale -- maybe I'm completely fine compared to those -- nevertheless any improvement is welcome, rebuild time of the 6700-line GltfImporterTest.cpp is almost 5 seconds and is enough to get me impatient :D

2. Question is whether those headers actually are heavy, or if it's just that they're used a lot -- they don't contain that much on their own (look at Pointer.h, there isn't much left to remove, apart from micro-optimizing some type traits), and all headers are parsed in about 50-80 ms which I think is fine. Which is also probably why PCHs didn't help you that much -- there just isn't any widely used header that would be heavy on its own for PCHs to make a difference.

   As I'm aware the most significant overhead comes from STL <sstream> usage, which is used in almost all tests for historical reasons, and which I didn't get to replacing yet because it involves reinventing/replacing many wheels, including float printing (tracking issue here: mosra/magnum#293).

   Finally, not sure if you had a debug or release build, but on a release build a lot of debug-only assertions from headers is compiled out, which has a considerable impact. Oh, and the on-by-default CORRADE_BUILD_DEPRECATED option also currently adds quite a few compatibility STL includes in many places, including Tester.h, to aid people with porting from APIs that used std::string to the new StringView. Disabiling it can make quite a difference -- if you had it enabled before, can you try again with it disabled?

3. Thanks! I'll cherry-pick those from your commit. I'm on Linux, so I have to admit I don't have an immediate feedback when windows.h accidentally escapes the cage somewhere.

4. Yup, I reached a similar conclusion recently. Can't really make C++14 a minimum yet (need GCC 4.8 compatibility for certain users), but maybe I could opt into variable templates if C++14 is available... or use compiler builtins instead. But there are downsides, I don't feel like maintaining my own variant of <type_traits> with all its compiler-specific magic :)

   Possibly related is that I use std::enable_if "wrong" in quite a few places, by having it in the function signature and thus participate in overload resolution. Need to clean that up to be on the return type instead.

from corrade.

@mosra: been playing around with some possible improvements here -- https://github.com/vittorioromeo/corrade/tree/wip_type_traits

I do get a minor compilation time speedup on my machine, I wonder if it's measurable on yours (or on CI).

from corrade.

Hey, that's great, thank you! :)

I have to think about the compilation time vs maintenance overhead (e.g., "what all do I need to change if a new compiler would implement a builtin of the same name but with different semantics", or the amount of additional test coverage that would be needed for each of these, or how hard will then be for a third party to contribute anything to this codebase given not even the standard type traits would be used anymore), but quite a lot of these seem quite simple and free of compiler magic.

Something similar is done in bgfx, and with success, so that's probably a path forward for me as well. Once I fix the bigger fish, like <sstream> 😅

from corrade.