Examples:

BenchmarkDotNetVisualizer

Visualizes your BenchmarkDotNet benchmarks to Colorful images, Feature-rich HTML, and customizable markdown files (and maybe powerful charts in the future!)

In Simple Words: You can create something like this 👇

Table of Content

A Real World Demo (.NET Collections Benchmark 🚀)
Getting Started
Using BenchmarkAutoRunner to Run your benchmarks
Using JoinReports method to Join and Pivot your reports
- Pivot by .NET Runtime version column
- Pivot by Method column
Todo and Contribution

A Real World Demo (.NET Collections Benchmark 🚀)

A Comprehensive Performance Comparison Benchmark Between Different .NET Collections.

https://github.com/mjebrahimi/DotNet-Collections-Benchmark/

Getting Started

1. Install Package

PM> Install-Package BenchmarkDotNetVisualizer

2. Simple Using

Methods:

SaveAsImageAsync()
SaveAsHtmlAsync()
SaveAsHtmlAndImageAsync()

Example:

var summary = BenchmarkRunner.Run<JsonSerializersBenchmark>(); 

//[ProjectDirectory]\Reports\JsonSerializers\Benchmark.html
var htmlFileName = DirectoryHelper.GetPathRelativeToProjectDirectory(@"Reports\JsonSerializers\Benchmark.html");

//[ProjectDirectory]\Reports\JsonSerializers\Benchmark.png
var imageFileName = DirectoryHelper.GetPathRelativeToProjectDirectory(@"Reports\JsonSerializers\Benchmark.png");
await summary.SaveAsHtmlAndImageAsync(
    htmlPath: htmlFileName, 
    imagePath: imageFileName,
    options: new ReportHtmlOptions
    {
        Title = "Json Serializers Benchmark",
        GroupByColumns = ["Method"],                          // Groups by 'Method' column and highlights groups
        SpectrumColumns = ["Mean", "Allocated"],              // Colorizes 'Mean' and 'Allocated' columns as Spectrum
        DividerMode = RenderTableDividerMode.EmptyDividerRow, // Separates tables by Empty Divider Row
        HtmlWrapMode = HtmlDocumentWrapMode.Simple            // Uses simple HTML table
    });

Output HTML: Visit this HTML page at samples/Reports/JsonSerializers/Benchmark.html

Output Image:

3. Using Exporters

Exporters:

[RichImageExporter]
[RichHtmlExporter]

Example:

BenchmarkRunner.Run<JsonSerializersBenchmark>();

//Exports colorful image
[RichImageExporter(
    title: "Json Serializers Benchmark", 
    groupByColumns: ["Method"],             // Groups by 'Method' column and highlights groups
    spectrumColumns: ["Mean", "Allocated"], // Colorizes 'Mean' and 'Allocated' columns as Spectrum and Sorts the result by them 
    //format: ImageFormat.Webp or Jpeg      // You can set image format (Default is ImageFormat.Png)
)]  

//Exports feature-rich HTML
[RichHtmlExporter(
    title: "Json Serializers Benchmark", 
    groupByColumns: ["Method"],             // Groups by 'Method' column and highlights groups
    spectrumColumns: ["Mean", "Allocated"]  // Colorizes 'Mean' and 'Allocated' columns as Spectrum and Sorts the result by them 
    //sortByColumns: ["Mean", "Allocated"]  // You can also sort by other columns as you wish
)]

[MemoryDiagnoser(displayGenColumns: false)] // Displays Allocated column (without GC per Generation columns (Gen 0, Gen 1, Gen 2) due to false option)
public class JsonSerializersBenchmark { ... }

Output:

To see the results, navigate to the following path:

[ProjectDirectory]\bin\[Debug|Release]\[.NET-version]\BenchmarkDotNet.Artifacts\results\Benchmark-report-rich.html|png

For Example:

MyBenchmark\bin\Release\net8.0\BenchmarkDotNet.Artifacts\results\Benchmark-report-rich.png
MyBenchmark\bin\Release\net8.0\BenchmarkDotNet.Artifacts\results\Benchmark-report-rich.html

Using BenchmarkAutoRunner to Run your benchmarks

It's Recommend to use BenchmarkAutoRunner.Run() instead of BenchmarkRunner.Run() to run your benchmarks.

BenchmarkAutoRunner is similar to BenchmarkRunner, but uses Job.Dry with InProcessEmitToolchain is case of DEBUG Mode (due to ease of debugging), and your specified job in case of RELEASE Mode.

It also Warns you if you are running project incorrectly. (for example running with Attached Debugger while RELEASE Mode is enabled)

var summary = BenchmarkAutoRunner.Run<IteratorsBenchmark>();

Using JoinReports method to Join and Pivot your reports

Example:

Performance benchmark between for, foreach, and ForEach() in different versions of .NET

Pivot by .NET Runtime version column

//Recommend to use BenchmarkAutoRunner instead of BenchmarkRunner
var summary = BenchmarkAutoRunner.Run<IteratorsBenchmark>();

//[ProjectDirectory]\Reports\Iterators\JoinedBenchmark-PivotBy-Runtime.html
var htmlFileName = DirectoryHelper.GetPathRelativeToProjectDirectory(@"Reports\Iterators\JoinedBenchmark-PivotBy-Runtime.html");

//[ProjectDirectory]\Reports\Iterators\JoinedBenchmark-PivotBy-Runtime.png
var imageFileName = DirectoryHelper.GetPathRelativeToProjectDirectory(@"Reports\Iterators\JoinedBenchmark-PivotBy-Runtime.png");

await summary.JoinReportsAndSaveAsHtmlAndImageAsync(
    htmlPath: htmlFileName,
    imagePath: imageFileName,
    options: new JoinReportHtmlOptions
    {
        Title = "Performance Comparison between for, foreach, and ForEach() method",
        MainColumn = "Method",
        GroupByColumns = ["Categories", "Length"],           // Groups by column 'Categories' and 'Length'
        PivotProperty = "Runtime",
        StatisticColumns = ["Mean"],
        ColumnsOrder = [".NET Core 3.0", ".NET Core 3.1", ".NET 5.0", ".NET 6.0", ".NET 7.0", ".NET 8.0"], // Order of columns 
        DividerMode = RenderTableDividerMode.SeparateTables, //Separates tables by Grouping by 'GroupByColumns'
        HtmlWrapMode = HtmlDocumentWrapMode.RichDataTables,  //Uses feature-rich https://datatables.net plugin
    });

Output HTML: Visit this HTML page at samples/Reports/Iterators/JoinedBenchmark-PivotBy-Runtime.html

Output Image:

Pivot by Method column

//Recommend to use BenchmarkAutoRunner instead of BenchmarkRunner
var summary = BenchmarkAutoRunner.Run<IteratorsBenchmark>();

//[ProjectDirectory]\Reports\Iterators\JoinedBenchmark-PivotBy-Method.html
var htmlFileName = DirectoryHelper.GetPathRelativeToProjectDirectory(@"Reports\Iterators\JoinedBenchmark-PivotBy-Method.html");

//[ProjectDirectory]\Reports\Iterators\JoinedBenchmark-PivotBy-Method.png
var imageFileName = DirectoryHelper.GetPathRelativeToProjectDirectory(@"Reports\Iterators\JoinedBenchmark-PivotBy-Method.png");

await summary2.JoinReportsAndSaveAsHtmlAndImageAsync(
    htmlPath: htmlFileName,
    imagePath: imageFileName,
    options: new JoinReportHtmlOptions
    {
        Title = "Performance Comparison between for, foreach, and ForEach() method",
        MainColumn = "Runtime",
        GroupByColumns = ["Categories", "Length"],           // Groups by column 'Categories' and 'Length'
        PivotProperty = "Method",
        StatisticColumns = ["Mean"],
        ColumnsOrder = ["for", "foreach", "ForEach()"],      // Order of columns 
        DividerMode = RenderTableDividerMode.SeparateTables, // Separates tables by Grouping by 'GroupByColumns'
        HtmlWrapMode = HtmlDocumentWrapMode.RichDataTables,  // Uses feature-rich https://datatables.net plugin
    });

Output HTML: Visit this HTML page at samples/Reports/Iterators/JoinedBenchmark-PivotBy-Method.html

Output Image:

Todo

Dark Theme (Need some help for this, wanna help? Please design a beautiful style for dark theme and send a PR)
Chart Visualization

Contributing

Create an issue if you find a BUG or have a Suggestion or Question.

If you want to develop this project :

Fork it!
Create your feature branch: git checkout -b my-new-feature
Commit your changes: git commit -am 'Add some feature'
Push to the branch: git push origin my-new-feature
Submit a pull request

Give it a Star! ⭐️

If you find this repository useful and like it, why not give it a star? if not, never mind! :)

License

JoinReport with unexpected results

I do have the following Artifacts
BenchmarkDotNet.Artifacts.zip

I tried to create a joint Report

var options = new JoinReportHtmlOptions
              {
	              Title = "Benchmark of Collection Item Manipulation (Mean)",
	              MainColumn = "Method",
	              GroupByColumns = ["Categories", "Prefill", "PriceRange"],
	              PivotProperty = "Runtime",
	              StatisticColumns = ["Mean"],
	              ColumnsOrder = [".NET 8.0", "NativeAOT 8.0", ".NET 9.0", "NativeAOT 9.0"],
	              OtherColumnsToSelect = ["DataType"],
	              SpectrumStatisticColumn = true,
	              HighlightGroups = true,
	              DividerMode = RenderTableDividerMode.SeparateTables,
	              HtmlWrapMode = HtmlDocumentWrapMode.RichDataTables
              };

await benchmarkSummaries.JoinReportsAndSaveAsHtmlAsync(DirectoryHelper.GetPathRelativeToProjectDirectory("Reports\\Benchmark-Mean.html"), options).ConfigureAwait(false);

The result looks like

As you can see there are additional Lines (like line 2) that I did not expect. I did experiment a bit and found out that DataType maybe the culprit.

So I changed the settings to

var options = new JoinReportHtmlOptions
              {
	              Title = "Benchmark of Collection Item Manipulation (Mean)",
	              MainColumn = "Method",
	              GroupByColumns = ["DataType", "Categories", "Prefill", "PriceRange"],
	              PivotProperty = "Runtime",
	              StatisticColumns = ["Mean"],
	              ColumnsOrder = [".NET 8.0", "NativeAOT 8.0", ".NET 9.0", "NativeAOT 9.0"],
	              SpectrumStatisticColumn = true,
	              HighlightGroups = true,
	              DividerMode = RenderTableDividerMode.SeparateTables,
	              HtmlWrapMode = HtmlDocumentWrapMode.RichDataTables
              };

await benchmarkSummaries.JoinReportsAndSaveAsHtmlAsync(DirectoryHelper.GetPathRelativeToProjectDirectory("Reports\\Benchmark-Mean.html"), options).ConfigureAwait(false);

Now I get this

This is too fragmented, but all the doubled lines are gone and the lines are exactly what I want to show.
Basically, I want Table 1 + Table 2 joined in one table and Table 3 + Table 4 joined in one table and so on....

Did I miss some settings or maybe the Join Report do have a little flaw with merging the reports?

mjebrahimi / benchmarkdotnetvisualizer Goto Github PK