Simple Benchmark of common Entity-Component-Systems: EnTT vs. entityx vs. anax vs. Artemis-Cpp

EnTT is a header-only, tiny and easy to use entity-component system in modern C++. ECS is an architectural pattern used mostly in game development. I started using another well known Entity Component System named entityx.

While I was playing with it, I found that I didn't like that much the way it manages its memory. Moreover, I was pretty sure that one > could achieve better performance with a slightly modified pool under the hood. That's also the reason for which the interface is quite similar to the one of entityx, so that EnTT can be used as a drop-in > replacement for it with a minimal effort.

Version: 1.1.0 (September 2017)

Entity Component Systems (ECS) are a form of decomposition that completely decouples entity logic and data from the entity "objects" themselves. The Evolve your Hierarchy article provides a solid overview of EC systems and why you should use them.

EntityX is an EC system that uses C++11 features to provide type-safe component management, event delivery, etc. It was built during the creation of a 2D space shooter.

Version: 1.1.2 (Juli 2017) and the compiler-time Version 2.0.0 (?) (August 2017)

anax is an open source C++ entity system designed to be portable, lightweight and easy to use. It is aimed toward Game Development, however it would be possible to use it for other projects.

Version: 2.1.0 (Januar 2017)

A C++ port of Artemis Entity System Framework.

The port was orignially written by Sidar Talei, in which he used several C++11 features such as deleted function, variadic templates, nullptr, etc… We wanted the framework to be portable, so we removed all C++11 feature usages.

Version: 1.x (October 2013)

Why benchpress?

The modern developer is faced with a rich variety of language alternatives to C++; however, C++ is still the choice of professionals facing demanding performance requirements. Despite this fact, there are few (if any) popular performance benchmarking frameworks.

Benchpress is inspired by Catch and by Golang's benchmark functionality.

It's simple, light and header-only.

Version: 1.x (Juni 2015)

I fork benchpress (add some utils and helper) to plot data for gnuplot.

I used CMake 3.9.3 for the build. Minimum is 3.2.

1. change directory to this Folder
2. mkdir ./build
3. cmake -H. -B./build -DCMAKE_BUILD_TYPE=Release -DCMAKE_EXPORT_COMPILE_COMMANDS=ON -DANAX_DEFAULT_ENTITY_POOL_SIZE:STRING="2097152" -DENTITYX_BUILD_SHARED:BOOL="0" -DBUILD_SHARED_LIBS:BOOL="0" -DBUILD_TESTING:BOOL="0" -DENTITYX_BUILD_TESTING:BOOL="0" -DBUILD_TESTING:BOOL="0" -DBUILD_SHARED_LIBS:BOOL="0" -DENTITYX_BUILD_SHARED:BOOL="0" -DINSTALL_HEADERS:BOOL="0" -DINSTALL_GMOCK:BOOL="0" -DBUILD_GMOCK:BOOL="0" -DENTITYX_BUILD_TESTING:BOOL="0" -DINSTALL_GTEST:BOOL="0"
4. cmake --build ./build --target ecs_benchmark

• modern C++14 Standard (-std=c++14)
• Default CMake Release Flags (-O3 -DNDEBUG)
• some Compiler Warnings (-Wall -Wextra -Wnon-virtual-dtor -Wcast-align -Woverloaded-virtual -pedantic)
• Threading (not used, yet?) (-pthread)

I used g++ 7.2.0, clang++ 3.8 should work, too.

• entityx (1.x)
• anax
• ArtemisCpp

benchpress, entityx (compile-time) and entt are header-only.

• Test and Examples are not build (-DENTITYX_BUILD_TESTING:BOOL="0" -DBUILD_TESTING:BOOL="0")
• Linked Libraries are static builds (-DENTITYX_BUILD_SHARED:BOOL="0" -DBUILD_SHARED_LIBS:BOOL="0")

EntityX2 has a Column Storage Capacity (ColumnStorage<Components, INITIAL_CAPACITY>) of 16777216, to avoid bad_alloc-Errors

• OS: Antergos Linux (4.13.3-1-ARCH) 64-Bit
• CPU: 4x Intel® Core™ i5 CPU 760 @ 2.80GHz
• RAM: 8 GB

I didn't benchmark Anax and Artemis, because it causes some bad_alloc-Errors.

(lower is better :)

Some bonus with EntityX (1.x) and eventpp.

Listen to EventA EventB and EventC

1. Build this Project, see Build
2. run ./scripts/run_benchmark.sh > ./doc/output.txt 2>&1 to print all kind of stuff - Note: artemis is disabled, it takes to long, but you can uncomment it 2.1. OR just run the direct benchmark with plotdata, ./build/ecs_benchmark --bench ".*entityx1.*update.*" --bench ".*entityx2.*update.*" --bench ".*entt.*update.*" --plotdata > ./doc/data.dat Now you got the data.dat 2.2. use gnuplot and the gnuplot-script to print the plot, or use this site http://gnuplot.respawned.com/

## 1:1 are Headers
plot  "data.dat" using 1:2 title 'EntityX1' with lines, \  # 1. Col
	"data.dat" using 1:3 title 'EntityX2' with lines, \    # 2. Col
	"data.dat" using 1:4 title 'EnTT' with lines           # 3. Col

You can edit the gnuplot-script to add new cols.

Benchmarks:

• creating, destroy and unpacking Entities
• Systems update

• PositionComponent
  • float x
  • float y
• DirectionComponent
  • float x
  • float y
• ComflabulationComponent
  • float thingy
  • int dingy
  • bool mingy
  • std::string stringy

• MovementSystem

void update() {
	position.x += direction.x * dt;
	position.y += direction.y * dt;
}

• ComflabSystem

void update() {
	comflab.thingy *= 1.000001f;
	comflab.mingy = !comflab.mingy;
	comflab.dingy++;
}

Benchmark Code (1 iteration):

auto entity = entitymanager.create();

entity.addComponent<PositionComponent>();
entity.addComponent<DirectionComponent>();
entity.addComponent<ComflabulationComponent>();

entity.destroy();

Run the "update system"-benchmark with different number of entities.

• 25, 50, 100, 200, 400, 800
• 1600, 3200, 5000
• 10000, 30000, 100000, 500000
• 1000000 (1M), 2000000 (2M), 5000000 (5M)
• 10000000 (10M) and 20000000 (20M) in some cases, if possible

Setup Benchmark:

// create application/world with systems

// create entities
for (size_t i = 0; i < nentities; i++) {
	auto entity = entitymanager.create();

	entity.addComponent<PositionComponent>();
	entity.addComponent<DirectionComponent>();

	if (i % 2) {
		entity.addComponent<ComflabulationComponent>();
	}
}

Benchmark Code (1 iteration):

world.update(fakeDeltaTime);

• https://github.com/blockchaindev/benchpress/blob/master/docs/getting_started.md
• https://github.com/miguelmartin75/anax/wiki/Using-the-Library

• http://tilemapkit.com/2015/10/entity-component-systems-compared-benchmarked-entityx-anax-artemis/
• https://github.com/LearnCocos2D/LearnCocos2D/tree/master/EntityComponentSystemsTest