Comments (6)
mattwarren
commented on May 18, 2024
I fixed this in a commit a few days ago, would you be willing to download the latest source from GitHub and run your test again?
BTW as-far-as-I-know, you're the first person to use BenchmarkDotNet with F#, so it'll be interesting to see what happens!
from benchmarkdotnet.
wk-j
commented on May 18, 2024
Just try with latest source,
Because I don't find .RunCompetition() any more so I use .Run() instead.
[<Test>]
let ShouldExecuteBenchmark() =
let reports = BenchmarkRunner().Run(typeof<Db>)
()And I got another error messages.
// Program.cs(42,57): error CS0246: The type or namespace name 'File' could not be found (are you missing a using directive or an assembly reference?)
// Program.cs(44,62): error CS0246: The type or namespace name 'File' could not be found (are you missing a using directive or an assembly reference?)
// Program.cs(51,57): error CS0246: The type or namespace name 'File' could not be found (are you missing a using directive or an assembly reference?)
Full output
------ Run test started ------
NUnit VS Adapter 2.0.0.0 executing tests is started
Loading tests from Z:\Source\csharp\couchbase-lite-init\CouchbaseLite.Tests\bin\Debug\CouchbaseLite.Tests.dll
Run started: Z:\Source\csharp\couchbase-lite-init\CouchbaseLite.Tests\bin\Debug\CouchbaseLite.Tests.dll
// ***** BenchmarkRunner: Start *****
// Found benchmarks:
// Db_Insert (Throughput_X86_LegacyJit_NET-HostFramework)
// Db_Query (Throughput_X86_LegacyJit_NET-HostFramework)
// **************************
// Benchmark: Db_Insert (Throughput_X86_LegacyJit_NET-HostFramework)
// *** Generate ***
// Result = Success
// DirectoryPath = Z:\Source\csharp\couchbase-lite-init\CouchbaseLite.Tests\bin\Debug\Db_Insert_Throughput_X86_LegacyJit_NET-HostFramework
// *** Build ***
// Result = Success
// *** Exec ***
// Run, Process: 1 / 3
//
Method=Insert
// BenchmarkDotNet-Dev=v0.7.8.0+
// OS=Microsoft Windows NT 6.2.9200.0
// Processor=Intel(R) Core(TM) i5-3210M CPU @ 2.50GHz, ProcessorCount=2
// CLR=MS.NET 4.0.30319.42000, Arch=32-bit DEBUG
// Pre-Warmup: 1 op, 1250.7 ms, 1250685900 ns, 12506859 ticks, 1250685900 ns/op, 0.8 op/s
// Warmup (idle): 1 op, 0 ms, 3500 ns, 35 ticks, 3500 ns/op, 285714.3 op/s
// Warmup (idle): 1 op, 0 ms, 300 ns, 3 ticks, 300 ns/op, 3333333.3 op/s
// Warmup (idle): 1 op, 0 ms, 500 ns, 5 ticks, 500 ns/op, 2000000 op/s
// IterationCount = 1
// Target (idle): 1 op, 0 ms, 300 ns, 3 ticks, 300 ns/op, 3333333.3 op/s
// Target (idle): 1 op, 0 ms, 500 ns, 5 ticks, 500 ns/op, 2000000 op/s
// Target (idle): 1 op, 0 ms, 400 ns, 4 ticks, 400 ns/op, 2500000 op/s
// Target (idle): 1 op, 0 ms, 400 ns, 4 ticks, 400 ns/op, 2500000 op/s
// Target (idle): 1 op, 0 ms, 400 ns, 4 ticks, 400 ns/op, 2500000 op/s
// Warmup 1: 1 op, 2519.8 ms, 2519849000 ns, 25198490 ticks, 2519849000 ns/op, 0.4 op/s
// Warmup 2: 1 op, 911.1 ms, 911112500 ns, 9111125 ticks, 911112500 ns/op, 1.1 op/s
// Warmup 3: 1 op, 953.6 ms, 953578700 ns, 9535787 ticks, 953578700 ns/op, 1 op/s
// Warmup 4: 1 op, 1787.9 ms, 1787925900 ns, 17879259 ticks, 1787925900 ns/op, 0.6 op/s
// Warmup 5: 1 op, 888.6 ms, 888575900 ns, 8885759 ticks, 888575900 ns/op, 1.1 op/s
Target 1: 1 op, 954 ms, 954038500 ns, 9540385 ticks, 954038500 ns/op, 1 op/s
Target 2: 1 op, 838.9 ms, 838931100 ns, 8389311 ticks, 838931100 ns/op, 1.2 op/s
Target 3: 1 op, 870.2 ms, 870209000 ns, 8702090 ticks, 870209000 ns/op, 1.1 op/s
Target 4: 1 op, 975.7 ms, 975658800 ns, 9756588 ticks, 975658800 ns/op, 1 op/s
Target 5: 1 op, 857 ms, 856967700 ns, 8569677 ticks, 856967700 ns/op, 1.2 op/s
Target 6: 1 op, 886.9 ms, 886948400 ns, 8869484 ticks, 886948400 ns/op, 1.1 op/s
Target 7: 1 op, 841 ms, 840972800 ns, 8409728 ticks, 840972800 ns/op, 1.2 op/s
Target 8: 1 op, 892.9 ms, 892907000 ns, 8929070 ticks, 892907000 ns/op, 1.1 op/s
Target 9: 1 op, 1070.1 ms, 1070077500 ns, 10700775 ticks, 1070077500 ns/op, 0.9 op/s
Target 10: 1 op, 1210.8 ms, 1210794600 ns, 12107946 ticks, 1210794600 ns/op, 0.8 op/s
// Benchmark finished
// Run, Process: 2 / 3
//
Method=Insert
// BenchmarkDotNet-Dev=v0.7.8.0+
// OS=Microsoft Windows NT 6.2.9200.0
// Processor=Intel(R) Core(TM) i5-3210M CPU @ 2.50GHz, ProcessorCount=2
// CLR=MS.NET 4.0.30319.42000, Arch=32-bit DEBUG
// Pre-Warmup: 1 op, 1822.6 ms, 1822602600 ns, 18226026 ticks, 1822602600 ns/op, 0.5 op/s
// Warmup (idle): 1 op, 0 ms, 3800 ns, 38 ticks, 3800 ns/op, 263157.9 op/s
// Warmup (idle): 1 op, 0 ms, 400 ns, 4 ticks, 400 ns/op, 2500000 op/s
// Warmup (idle): 1 op, 0 ms, 300 ns, 3 ticks, 300 ns/op, 3333333.3 op/s
// IterationCount = 1
// Target (idle): 1 op, 0 ms, 400 ns, 4 ticks, 400 ns/op, 2500000 op/s
// Target (idle): 1 op, 0 ms, 400 ns, 4 ticks, 400 ns/op, 2500000 op/s
// Target (idle): 1 op, 0 ms, 300 ns, 3 ticks, 300 ns/op, 3333333.3 op/s
// Target (idle): 1 op, 0 ms, 500 ns, 5 ticks, 500 ns/op, 2000000 op/s
// Target (idle): 1 op, 0 ms, 400 ns, 4 ticks, 400 ns/op, 2500000 op/s
// Warmup 1: 1 op, 1391.1 ms, 1391081900 ns, 13910819 ticks, 1391081900 ns/op, 0.7 op/s
// Warmup 2: 1 op, 1698.5 ms, 1698489800 ns, 16984898 ticks, 1698489800 ns/op, 0.6 op/s
// Warmup 3: 1 op, 918.4 ms, 918402700 ns, 9184027 ticks, 918402700 ns/op, 1.1 op/s
// Warmup 4: 1 op, 853.8 ms, 853792200 ns, 8537922 ticks, 853792200 ns/op, 1.2 op/s
// Warmup 5: 1 op, 867.2 ms, 867163300 ns, 8671633 ticks, 867163300 ns/op, 1.2 op/s
Target 1: 1 op, 854.6 ms, 854572300 ns, 8545723 ticks, 854572300 ns/op, 1.2 op/s
Target 2: 1 op, 1695.7 ms, 1695748000 ns, 16957480 ticks, 1695748000 ns/op, 0.6 op/s
Target 3: 1 op, 889.6 ms, 889556000 ns, 8895560 ticks, 889556000 ns/op, 1.1 op/s
Target 4: 1 op, 889.5 ms, 889486200 ns, 8894862 ticks, 889486200 ns/op, 1.1 op/s
Target 5: 1 op, 1113.2 ms, 1113221800 ns, 11132218 ticks, 1113221800 ns/op, 0.9 op/s
Target 6: 1 op, 829.6 ms, 829617400 ns, 8296174 ticks, 829617400 ns/op, 1.2 op/s
Target 7: 1 op, 1874.8 ms, 1874836600 ns, 18748366 ticks, 1874836600 ns/op, 0.5 op/s
Target 8: 1 op, 1072.2 ms, 1072176400 ns, 10721764 ticks, 1072176400 ns/op, 0.9 op/s
Target 9: 1 op, 889.6 ms, 889587600 ns, 8895876 ticks, 889587600 ns/op, 1.1 op/s
Target 10: 1 op, 963.8 ms, 963817900 ns, 9638179 ticks, 963817900 ns/op, 1 op/s
// Benchmark finished
// Run, Process: 3 / 3
//
Method=Insert
// BenchmarkDotNet-Dev=v0.7.8.0+
// OS=Microsoft Windows NT 6.2.9200.0
// Processor=Intel(R) Core(TM) i5-3210M CPU @ 2.50GHz, ProcessorCount=2
// CLR=MS.NET 4.0.30319.42000, Arch=32-bit DEBUG
// Pre-Warmup: 1 op, 1779.8 ms, 1779812000 ns, 17798120 ticks, 1779812000 ns/op, 0.6 op/s
// Warmup (idle): 1 op, 0 ms, 11100 ns, 111 ticks, 11100 ns/op, 90090.1 op/s
// Warmup (idle): 1 op, 0 ms, 400 ns, 4 ticks, 400 ns/op, 2500000 op/s
// Warmup (idle): 1 op, 0 ms, 500 ns, 5 ticks, 500 ns/op, 2000000 op/s
// IterationCount = 1
// Target (idle): 1 op, 0 ms, 500 ns, 5 ticks, 500 ns/op, 2000000 op/s
// Target (idle): 1 op, 0 ms, 400 ns, 4 ticks, 400 ns/op, 2500000 op/s
// Target (idle): 1 op, 0 ms, 500 ns, 5 ticks, 500 ns/op, 2000000 op/s
// Target (idle): 1 op, 0 ms, 500 ns, 5 ticks, 500 ns/op, 2000000 op/s
// Target (idle): 1 op, 0 ms, 300 ns, 3 ticks, 300 ns/op, 3333333.3 op/s
// Warmup 1: 1 op, 1088 ms, 1088008700 ns, 10880087 ticks, 1088008700 ns/op, 0.9 op/s
// Warmup 2: 1 op, 1143.4 ms, 1143420300 ns, 11434203 ticks, 1143420300 ns/op, 0.9 op/s
// Warmup 3: 1 op, 815.3 ms, 815292800 ns, 8152928 ticks, 815292800 ns/op, 1.2 op/s
// Warmup 4: 1 op, 890.6 ms, 890554400 ns, 8905544 ticks, 890554400 ns/op, 1.1 op/s
// Warmup 5: 1 op, 842.1 ms, 842103500 ns, 8421035 ticks, 842103500 ns/op, 1.2 op/s
Target 1: 1 op, 841.2 ms, 841181400 ns, 8411814 ticks, 841181400 ns/op, 1.2 op/s
Target 2: 1 op, 909.2 ms, 909229400 ns, 9092294 ticks, 909229400 ns/op, 1.1 op/s
Target 3: 1 op, 879.4 ms, 879389300 ns, 8793893 ticks, 879389300 ns/op, 1.1 op/s
Target 4: 1 op, 884.8 ms, 884839300 ns, 8848393 ticks, 884839300 ns/op, 1.1 op/s
Target 5: 1 op, 901.8 ms, 901756300 ns, 9017563 ticks, 901756300 ns/op, 1.1 op/s
Target 6: 1 op, 1713 ms, 1712966100 ns, 17129661 ticks, 1712966100 ns/op, 0.6 op/s
Target 7: 1 op, 1109.1 ms, 1109138100 ns, 11091381 ticks, 1109138100 ns/op, 0.9 op/s
Target 8: 1 op, 942.7 ms, 942731100 ns, 9427311 ticks, 942731100 ns/op, 1.1 op/s
Target 9: 1 op, 960.6 ms, 960619900 ns, 9606199 ticks, 960619900 ns/op, 1 op/s
Target 10: 1 op, 898.3 ms, 898316000 ns, 8983160 ticks, 898316000 ns/op, 1.1 op/s
// Benchmark finished
AverageTime (ns/op): Avr=1017009750 +- 96707463.5046131
OperationsPerSecond: Avr=1.02943216490649 +- 0.0657486502698214
// **************************
// Benchmark: Db_Query (Throughput_X86_LegacyJit_NET-HostFramework)
// *** Generate ***
// Result = Success
// DirectoryPath = Z:\Source\csharp\couchbase-lite-init\CouchbaseLite.Tests\bin\Debug\Db_Query_Throughput_X86_LegacyJit_NET-HostFramework
// *** Build ***
// Program.cs(42,57): error CS0246: The type or namespace name 'File' could not be found (are you missing a using directive or an assembly reference?)
// Program.cs(44,62): error CS0246: The type or namespace name 'File' could not be found (are you missing a using directive or an assembly reference?)
// Program.cs(51,57): error CS0246: The type or namespace name 'File' could not be found (are you missing a using directive or an assembly reference?)
// Result = Failure
// ***** BenchmarkRunner: Finish *****
BenchmarkDotNet-Dev=v0.7.8.0+
OS=Microsoft Windows NT 6.2.9200.0
Processor=Intel(R) Core(TM) i5-3210M CPU @ 2.50GHz, ProcessorCount=2
HostCLR=MS.NET 4.0.30319.42000, Arch=32-bit DEBUG
Type |
Method |
Mode |
Platform |
Jit |
.NET |
toolchain |
Runtime |
Warmup |
Target |
AvrTime |
StdDev |
op/s |
----- |
------- |
----------- |
--------- |
---------- |
-------------- |
---------- |
-------- |
------- |
------- |
-------------- |
------------ |
----- |
Db |
Insert |
Throughput |
X86 |
LegacyJit |
HostFramework |
Classic |
Clr |
5 |
10 |
1,017.0098 ms |
270.2493 ms |
1.03 |
Benchmarks with troubles:
Db_Query_Throughput_X86_LegacyJit_NET-HostFramework
// *** Warnings ***
StdDev (270.2493 ms) is 27% of Mean (1,017.0098 ms).
// ***** BenchmarkRunner: End *****
NUnit VS Adapter 2.0.0.0 executing tests is finished
========== Run test finished: 1 run (0:01:22.3920015) ==========Generated code
Db_Query_Throughput_X86_LegacyJit_NET-HostFramework/Program.cs
using System;
using System.Diagnostics;
using System.Threading;
using System.Runtime.CompilerServices;
using BenchmarkDotNet;
using BenchmarkDotNet.Tasks;
using Microsoft.FSharp.Collections;
namespace BenchmarkDotNet.Autogenerated
{
public class Program : BenchmarkSpec.Db
{
private BenchmarkTask task = new BenchmarkTask(3, configuration: new BenchmarkConfiguration(mode: BenchmarkMode.Throughput, platform: BenchmarkPlatform.X86, jitVersion: BenchmarkJitVersion.LegacyJit, framework: BenchmarkFramework.HostFramework, toolchain: BenchmarkToolchain.Classic, runtime: BenchmarkRuntime.Clr, warmupIterationCount: 5, targetIterationCount: 10), parametersSets: new BenchmarkParametersSets("", false, intParams: new int[] { }));
public static void Main(string[] args)
{
try
{
System.Console.WriteLine(BenchmarkDotNet.EnvironmentHelper.GetFullEnvironmentInfo());
Program instance = new Program();
instance.RunBenchmark();
System.Console.WriteLine("// Benchmark finished");
}
catch (Exception ex)
{
System.Console.WriteLine(ex);
throw;
}
}
public Program()
{
setupAction = () => { };
idleAction = Idle;
targetAction = Query;
}
private Microsoft.FSharp.Collections.FSharpList<File> value;
private Action setupAction;
private Func<Microsoft.FSharp.Collections.FSharpList<File>> targetAction, idleAction;
public void RunBenchmark()
{
new BenchmarkMethodInvoker().Throughput(task, 1, setupAction, targetAction, idleAction);
}
private Microsoft.FSharp.Collections.FSharpList<File> Idle()
{
return default(Microsoft.FSharp.Collections.FSharpList<File>);
}
}
}from benchmarkdotnet.
mattwarren
commented on May 18, 2024
Yeah, I wondered if there would be more errors. Seems like we need to do some more work to support F# benchmarks. Thanks for the great repo though and it's cool you're also using RavenDB.
It seems to be tripping up on, we don't have the correct using statement for that type:
type File =
{ Name : string
Path : string
Extension : string
Length : int }When that's compiled down to IL, what namespace would it live in? I've not done much with F#, is it be related to the line of code module BenchmarkSpec?
from benchmarkdotnet.
wk-j
commented on May 18, 2024
Single module in F# will translate into static class (no namespace).
module BenchmarkSpec
type File { ... }
public static class BenchmarkSpec {
public class File { ... }
}
But when specify nested module It will extract as namespace.
module Tests.BenchmarkSpec
type File { ... }
namespace Tests {
public static class BenchmarkSpec {
public class File { ... }
}
}
In both case File will alway live under static class BenchmarkSpec and generated code could be
private Microsoft.FSharp.Collections.FSharpList<BenchmarkSpec.File> value;
Not just
private Microsoft.FSharp.Collections.FSharpList<File> value;
from benchmarkdotnet.
mattwarren
commented on May 18, 2024
@wk-j sorry for the long delay, but I've finally checked in a fix for this see 5d000fc
I'm now able to run this benchmark and I get the output shown below.
My benchmark is a cut-down version of yours (without the RavenDB dependencies), so when you get a chance can you run your full benchmark with the latest code?
from benchmarkdotnet.
AndreyAkinshin
commented on May 18, 2024
@mattwarren, @wk-j, you can install the latest version of the library via NuGet: https://www.nuget.org/packages/BenchmarkDotNet/0.8.1
from benchmarkdotnet.
Related Issues (20)
- Unable to get `-p ETW` working these days HOT 4
- Unable to use `wasm` or `monoaotllvm` as runtime HOT 2
- InvalidMeasurementUnitException when using `--statisticalTest` argument when comparing private builds HOT 2
- CS0119 thrown when benchmark named _.System()
- How to change the time display mode from microseconds to seconds or milliseconds? HOT 1
- HideColumns + custom corerun = incorrect table markdown HOT 2
- unity game engine support? HOT 1
- Cannot build NativeAOT 8.0 on Arch Linux HOT 6
- Add support for dotnet-script HOT 1
- What should I do if I need to see the output of each test item? HOT 2
- What should I do if I need to see the output of each test item? HOT 1
- Plots not generated for memory benchmarks (MemoryDiagnoser)
- Question about visibility of DotTraceDiagnoser's second constructor HOT 2
- BenchmarkDotNet does not copy needed assemblies to the run folder (again?) HOT 2
- Can't override `DefaultConfig.Instance` settings when using custom configs with `IConfigSource` attribute.
- Better support for asymmetric CPU cores HOT 3
- Total Number of Benchmarks
- `Job.WithGcForce` does not work
- `--generateBinlog` working incorrectly: not building due to msbuild error.
- BenchmarkDotNet test adapter not running tests with dotnet test
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from benchmarkdotnet.