Comments (6)
Thanks for the script @Levi-Armstrong , although it would be nice if the github action supported this natively.
from github-action-benchmark.
I ended up writing a python script to join the files to avoid docker in docker.
#!/usr/bin/python3
import json
import os
build_dir = os.getcwd() + "/../../build"
result_files = []
search_path = build_dir + "/tesseract_collision/test/benchmarks"
for file in os.listdir(search_path):
if file.endswith(".json"):
result_files.append(os.path.join(search_path, file))
search_path = build_dir + "/tesseract_environment/test/benchmarks"
for file in os.listdir(search_path):
if file.endswith(".json"):
result_files.append(os.path.join(search_path, file))
cnt = 0
all_data = {}
for file in result_files:
# Opening JSON file
f = open(file, "r")
# returns JSON object as a dictionary
data = json.loads(f.read())
if cnt == 0:
all_data = data
else:
for benchmark in data["benchmarks"]:
all_data["benchmarks"].append(benchmark)
# endfor
# endif
# Closing file
f.close()
# Increment counter
cnt+=1
# endfor
# the base path to where the output must be stored
if not os.path.exists("/root/benchmarks"):
os.makedirs("/root/benchmarks")
# endif
# the json file where the output must be stored
out_file = open("/root/benchmarks/tesseract-benchmark_result.json", "w+")
json.dump(all_data, out_file, indent = 4)
out_file.close()
from github-action-benchmark.
Whilst it'd be nice to have this baked in, here's some C# code to combine benchmark results.
Not as pretty as @Levi-Armstrong's python solution, but seems to do the job if you work in dotnet 6+.
public static void CombineBenchmarkResults(
string resultsDir = "./BenchmarkDotNet.Artifacts/results",
string resultsFile = "Combined.Benchmarks",
string searchPattern = "Foo.*.json")
{
var resultsPath = Path.Combine(resultsDir, resultsFile + ".json");
if (!Directory.Exists(resultsDir))
{
throw new DirectoryNotFoundException($"Directory not found '{resultsDir}'");
}
if (File.Exists(resultsPath))
{
File.Delete(resultsPath);
}
var reports = Directory
.GetFiles(resultsDir, searchPattern, SearchOption.TopDirectoryOnly)
.ToArray();
if (!reports.Any())
{
throw new FileNotFoundException($"Reports not found '{searchPattern}'");
}
var combinedReport = JsonNode.Parse(File.ReadAllText(reports.First()))!;
var title = combinedReport["Title"]!;
var benchmarks = combinedReport["Benchmarks"]!.AsArray();
// Rename title whilst keeping original timestamp
combinedReport["Title"] = $"{resultsFile}{title.GetValue<string>()[^16..]}";
foreach (var report in reports.Skip(1))
{
var array = JsonNode.Parse(File.ReadAllText(report))!["Benchmarks"]!.AsArray();
foreach (var benchmark in array)
{
// Double parse avoids "The node already has a parent" exception
benchmarks.Add(JsonNode.Parse(benchmark!.ToJsonString())!);
}
}
File.WriteAllText(resultsPath, combinedReport.ToString());
}
from github-action-benchmark.
Here is an example:
https://github.com/open-telemetry/opentelemetry-cpp/blob/0b1a74c7abd3375ef459b28f6b3536a42cf9e71a/ci/do_ci.sh#L40-L49
from github-action-benchmark.
@esigo Does this create a single benchmark file which is to be uploaded?
from github-action-benchmark.
Is there a blocker that is preventing this action from taking a list of files versus just a single file?
from github-action-benchmark.
Related Issues (20)
- Flaky tests in CI HOT 3
- Bug: when summary-always and gh-repository options don't work together HOT 1
- Benchmark job never finishes HOT 18
- Create `goBackwardCompatibleMetrics` option that defaults to true HOT 4
- Incorrect commit id is added for pytest benchmarks HOT 3
- Add support for parsing loop times in Go benchmarks HOT 3
- Running go benchmarks: Use `-count` or not? HOT 3
- cargo & criterion: `Error: No benchmark result was found` HOT 6
- Rust benchmarks not comparing to baseline HOT 3
- Deno benchmarking tool PR HOT 1
- Open Issue on alert HOT 2
- Bug: Commit message does not comply with organisation policy. HOT 2
- For dotnet benchmarking the unit is currently hardcoded to nanosecond. Can it be derived from the json like for other benchmarks? HOT 1
- Catch2 v.3.5.0 parsing: reTestCaseStart regex HOT 1
- Summary table loses some of the extra information from go benchmarks when running with multiple CPU scenarios HOT 1
- Support sub-nanosecond precision on Cargo benchmarks HOT 1
- Add Support for [nanobench](https://github.com/martinus/nanobench) for C++
- command line tools for data manipulation
- Action is not creating any comments on PR HOT 1
- Action Will Not Create Summary Tables if Performance Alerts Occur
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from github-action-benchmark.