Is your feature request related to a problem? Please describe.
Ideally, we should try to leverage Benchmark.NET for the actual execution and running of these benchmarks. The IActorBenchmark
type is practically set up for this automatically:
public interface IActorBenchmark
{
IBenchmarkConfiguration Configuration { get; }
Task<(int totalActors, int messagesPerActor)> Setup(CancellationToken ct);
Task Run(CancellationToken ct);
Task Teardown(CancellationToken ct);
}
Describe the solution you'd like
I would like to do the following:
- 1
Benchmark
should be computed per IActorBenchmarkDefintion
;
- 1
BenchmarkCase
should be computed per unique IBenchmarkConfiguration
; and
- The
InvocationsPerOperation
number for each BenchmarkCase
should be set programmatically via the `IActorBenchmark.Setup~ computed value, if that's possible.
I believe we should be able to do this by tapping into the Benchmark.NET APIs directly and passing them into the BenchmarkSwitcher
- but it will take a bit of meta-programming magic to make this work.
Essentially, we need to make sure this function returns true
for whatever wrapping we do around Benchmark.NET:
https://github.com/dotnet/BenchmarkDotNet/blob/570eeefa20c8692662722cd4bf8a1b663bd361b7/src/BenchmarkDotNet/Running/TypeFilter.cs#L12-L45
Describe alternatives you've considered
It might actually be a lot easier to instrument this with NBench, as it allows for dynamic computation of operation invocations in a much more simplified manner than Benchmark.NET.