nova is a header-only C++17 job system for Windows. It spins up a thread pool which you can push function invocations to syncronously, asynchronously, or semi-synchronously.

Tested on MSVC2017 and Clang 4.0.0.

• Getting started
• Synchronous usage
  • start_sync
  • call
  • bind
• Asynchronous usage
  • start_async
  • push
  • dependency_token
  • kill_all_workers
• Semi-synchronous usage
  • dependent
• Batching
  • bind_batch
  • parallel_for
• Main thread invocation
  • to_main
  • return_main
  • switch_to_main
• Footnotes
• API reference

Using the system is easy: download the headers, #include "nova.h", and use all the stuff in the nova namespace. Remember to enable C++17 features on your compiler.

Here's a sample program to get you started. It starts the job system, runs NextJob and JobWithParam in parallel, then exits.

#include <iostream>
#include <string>
#include "nova.h"

void NextJob() {
	std::cout << "Hello from NextJob" << std::endl;
}

void JobWithParam(int number) {
	std::cout << "Hello from JobWithParam, with param: " << std::to_string(number) << std::endl;
}

void InitialJob() {
	nova::call(&NextJob, nova::bind(&JobWithParam, 5));
}

int main() {
	nova::start_sync(&InitialJob);
}

We start with a call to nova::start_sync, which initializes the job system, enters the first job (represented here by InitialJob), and returns when that job finishes. The first job can be any callable object (e.g. function pointers, member function pointers, lambdas, functors, etc.), and if it needs any parameters you can add them like so:

void InitialJob(int number, Foo foo) { ... }

int main() {
	nova::start_sync(&InitialJob, 5, Foo());
}

After entering InitialJob we reach the call to nova::call, which takes one or more runnable objects (callable objects that can be called with no parameters), runs them in parallel, and returns¹ when they've all finished. You can use nova::bind (orstd::bind) to get a runnable wrapper for a callable object and its parameters².

Once NextJob and JobWithParam return nova::call will return, then InitialJob will return, the job system will shutdown, nova::start_sync will return, and the program will end.

Let's rewrite the sample program to work asynchronously:

#include <iostream>
#include <string>
#include "nova.h"

void NextJob(nova::dependency_token dt) {
	std::cout << "Hello from NextJob" << std::endl;
}

void JobWithParam(int number, nova::dependency_token dt) {
	std::cout << "Hello from JobWithParam, with param: " << std::to_string(number) << std::endl;
}

void InitialJob() {
	nova::dependency_token dt(nova::kill_all_workers);
	nova::push(nova::bind(&NextJob, dt), nova::bind(&JobWithParam, 5, dt));
}

int main() {
	nova::start_async(&InitialJob);
}

Here we start with nova::start_async, which doesn't return until nova::kill_all_workers is called elsewhere in the program. This allows us to use nova::push, which returns immediately, rather than nova::call.

If we had changed only those two calls, the program would run NextJob and JobWithParam in parallel then continue to run indefinitely in an idle state. To get nova::kill_all_workers to run once both NextJob and JobWithParam have finished we need to use a nova::dependency_token, which takes a runnable object and calls it once all copies of the token are destroyed.

Because InitialJob, NextJob, and JobWithParam all have a copy of dt, nova::kill_all_workers will only run once all three of those functions have returned, at which point nova::start_async will also return and the program will end.

Despite the syntax being heavier, asynchronous invocations are much more flexible than synchronous invocations; any dependency graph can be implemented with nova::push and nova::dependency_tokens.

We can rewrite the previous example in a way that uses both a synchronous start and an asynchronous invocation, and doesn't need any nova::dependency_tokens:

#include <iostream>
#include <string>
#include "nova.h"

void NextJob() {
	std::cout << "Hello from NextJob" << std::endl;
}

void JobWithParam(int number) {
	std::cout << "Hello from JobWithParam, with param: " << std::to_string(number) << std::endl;
}

void InitialJob() {
	nova::push<nova::dependent>(&NextJob, nova::bind(&JobWithParam, 5));
}

int main() {
	nova::start_sync(&InitialJob);
}

nova::dependent is a control, a type that can be passed to nova::push or nova::call as a template parameter to change their behavior. Although in nova::dependent's case it can only be passed to nova::push; it won't do anything to nova::call.

This particular control causes nova::push's invokees to extend the duration of an active synchronous invocation. That is to say, because InitialJob was invoked with nova::start_sync, which is synchronous, nova::start_sync won't return until InitialJob, NextJob, and JobWithParam have all returned. If we hadn't added the control it would only have been waiting on InitialJob, and our invocations of NextJob and JobWithParam may have been ignored.

nova::call is also synchronous, so the behavior would be the same if main was changed to the following:

int main() {
	nova::start_sync([]() {
		nova::call(&InitialJob);
	});
}

Semi-synchronous invocations are more expensive than asynchronous invocations when they actually extend a synchronous invocation (and negligibly more so when they don't), but they allow dependency graphs implemented with asynchronous invocations to be invoked synchronously.

nova::bind_batch allows you to take a callable object that takes a numerical range as two of its parameters³ and turn it into a batch runnable. Rather than being invoked as a single job, batch runnables are invoked as a set of jobs (one per thread), with each one receiving a contiguous portion of the original range.

For example, if this code was run on a machine with eight logical cores

void BatchJob(unsigned start, unsigned end){ ... }

nova::push(nova::bind_batch(&BatchJob, 0, 8000));

it would be functionally equivalent to the following:

void BatchJob(unsigned start, unsigned end){ ... }

nova::push(
	nova::bind(&BatchJob, 0, 1000),
	nova::bind(&BatchJob, 1000, 2000),
	nova::bind(&BatchJob, 2000, 3000),
	nova::bind(&BatchJob, 3000, 4000),
	nova::bind(&BatchJob, 4000, 5000),
	nova::bind(&BatchJob, 5000, 6000),
	nova::bind(&BatchJob, 6000, 7000),
	nova::bind(&BatchJob, 7000, 8000)
);

This will work with nova::call as well, and it will work with any combination of batch and non-batch runnables. The following are all valid:

nova::push(
	nova::bind_batch(&BatchJob, 0, 8000),
	nova::bind(&OtherJob)
);
nova::call(
	nova::bind(&OtherJob),
	nova::bind_batch(&BatchJob, 0, 8000),
	nova::bind_batch(&OtherBatchJob, 0, 100, Foo())
);
etc.

Batching is leveraged to create an efficient parallel_for implementation that is performant both with expensive operations on a small amount of data and with cheap operations on a large amount of data. The following call

nova::parallel_for([](unsigned index) {
	// do something
	...
}, 0, 1000);

will call the lambda 1000 times, but will only create as many jobs as the system can run concurrently.

However, if you can process multiple elements at once (e.g. SIMD) it may be more performant to use a batch function directly.

There are two other controls in addition to nova::dependent: nova::to_main and nova::return_main.

When passed to nova::push or nova::call, nova::to_main will cause the runnable objects to be invoked on the main thread. nova::return_main doesn't affect nova::push, but when passed to nova::call it will cause it to return to the main thread when its invokees return rather than returning on any available thread.

// These will run their invokees on the main thread.
nova::call<nova::to_main>(...);
nova::push<nova::to_main>(...);

// These may run their invokees on any thread.
nova::call(...);
nova::push(...);

// This will always return to the main thread.
nova::call<nova::return_main>(...);

// These may return to any thread (which may be different from the thread they were called on).
nova::call(...);
nova::call<nova::to_main>(...);

// Controls may be added in any combination and any order:
nova::call<nova::return_main, nova::to_main>(...);
nova::call<nova::to_main, nova::return_main>(...);
nova::push<nova::dependent, nova::to_main>(...);
nova::push<nova::to_main, nova::dependent>(...);
nova::push<nova::dependent>(...);
// etc.

nova::switch_to_main allows you to move to the main thread at any time:

... // We're on an arbitrary worker thread, main or otherwise.

nova::switch_to_main();

... // Now we're on the main thread.

¹ nova::call will not necessarily return to the same thread it was called from.

² By default, both nova::bind and std::bind will pass references to copies to a callable that expects references. If you want a true reference you need to use std::ref or std::cref:

void TestFunc(int& n){ n++; }

int test = 0;

nova::bind(&TestFunc, test)(); // TestFunc is passed a reference to a copy of test.
std::bind(&TestFunc, test)();

// test is still 0.

nova::bind(&TestFunc, std::ref(test))(); // TestFunc is passed a reference to test.
std::bind(&TestFunc, std::ref(test))();

// test is now 2.

³ nova::bind_batch assumes the parameters denoting the range are sequential (i.e. ..., start, end, ...), and it assumes that start is the first parameter to satisfy std::is_integral:

// Correct, uses start as the start and end as the end.
void CorrectBatchSignature(Foo foo, int start, unsigned end, char c);

// Doesn't compile (unless Foo can convert to std::size_t)
// Uses start as the start and foo as the end.
void IncorrectBatchSignature(int start, Foo foo, long end, char c);

// Compiles, but almost certainly incorrect.
// Uses c as the start and start as the end
void VeryIncorrectBatchSignature(Foo foo, char c, int start, unsigned end);