https://gitter.im/real-logic/Aeron

ogreenz @ogreenz 16:02
Hi, there is some mixup with Aeron.Client which is really a show-stopper. Aeron.Clinet is not truly 1.0.0 as it claims to be. First of all looking at master branch of Aeron.Net at src/Adaptive.Aeron/Properties/AssemblyInfo.cs, it contains [assembly: AssemblyVersion("0.1.0.0")](changed on July 31st the day of the nuget release). Looking at the dll inside the nuget it says 0.9.9.0, so it's all rather confusing. But the main problem is that in the nuget file the Adaptive.Aeron.dll built withCncFileDescriptor.cs declaring CNC_VERSION = 4 which does not match the media-driver.jar driver version which was bumped as well on July 31st, causing the Aeron.Connect() to fail because of Cnc version incompatibility. Just to make it clear the GitHub version is all good (the driver and the cnc files versions match) it's the nuget that was published that is broken (old source code with new driver), right now I'm working from a side-loaded dll compiled by me but would appreciate the nuget file getting fixed or at least matching it to Aeron.Driver version, thanks!

What's a release cycle for Aeron.NET? I've seen that recently there's not that much activity in the project. Is is dead or maybe done?

When starting Aeron via a schedulded task instead of the console, the following exception is thrown.


Exception Info: Adaptive.Aeron.Exceptions.DriverTimeoutException
at Adaptive.Aeron.Aeron+Context.ConnectToDriver()
at Adaptive.Aeron.Aeron+Context.Conclude()
at Adaptive.Aeron.Aeron..ctor(Context)
at Adaptive.Aeron.Aeron.Connect(Context)
at xxx.AeronPublisher..ctor()
at xxx.NanexxServerImpl.Run()
at xxx.Main(System.String[])


What is missing to connect when using a schedulded task? Maybe I have missed something in the docu....
The aeron connection is build this way;

aeron = Aeron.Connect();
publisher = aeron.AddPublication(channel, streamId);

Unless there are other changes planned in the short term, could you publish NuGet packages for the current version (i.e. with #100 and #101)?

Thanks!

Hey Guys,

Just started playing with Aeron using your .NET implementation and I am noticing when I am trying to extend an existing recording I get this coming from the driver. io.aeron.archive.client.ArchiveException: cannot extend recording 0 image joinPosition 0 not equal to recording stopPosition 1536. I have attached a copy of the archiving code that I'm using.

Any advice would be appreciated, thank you so much. On a side note, is there any plans to update this client to work with the most recent build of Aeron?

program.txt

After running our tests on TravisCI it looks like there are some PlatformNotSupportedException running our tests. Surprisingly dotnet test detects it and skips those tests as the build is currently green.

Here's is the part of the log coming from TravisCI:

dotnet test -f netcoreapp2.0
Test run for /home/travis/build/AdaptiveConsulting/Aeron.NET/src/Adaptive.Agrona.Tests/bin/Debug/netcoreapp2.0/Adaptive.Agrona.Tests.dll(.NETCoreApp,Version=v2.0)
Microsoft (R) Test Execution Command Line Tool Version 15.3.0-preview-20170628-02
Copyright (c) Microsoft Corporation.  All rights reserved.
Starting test execution, please wait...
NUnit Adapter 3.8.0.0: Test execution started
Running all tests in /home/travis/build/AdaptiveConsulting/Aeron.NET/src/Adaptive.Agrona.Tests/bin/Debug/netcoreapp2.0/Adaptive.Agrona.Tests.dll
NUnit3TestExecutor converted 63 of 63 NUnit test cases
SetUp failed for test fixture Adaptive.Agrona.Tests.AtomicBufferTests
An exception was thrown while loading the test.
System.TypeInitializationException: The type initializer for 'Adaptive.Agrona.Tests.AtomicBufferTests' threw an exception. ---> System.PlatformNotSupportedException: Named maps are not supported.
   at System.IO.MemoryMappedFiles.MemoryMappedFile.CreateCore(FileStream fileStream, String mapName, HandleInheritability inheritability, MemoryMappedFileAccess access, MemoryMappedFileOptions options, Int64 capacity)
   at System.IO.MemoryMappedFiles.MemoryMappedFile.CreateNew(String mapName, Int64 capacity, MemoryMappedFileAccess access, MemoryMappedFileOptions options, HandleInheritability inheritability)
   at System.IO.MemoryMappedFiles.MemoryMappedFile.CreateNew(String mapName, Int64 capacity)
   at Adaptive.Agrona.Tests.AtomicBufferTests..cctor() in /home/travis/build/AdaptiveConsulting/Aeron.NET/src/Adaptive.Agrona.Tests/AtomicBufferTests.cs:line 46
   --- End of inner exception stack trace ---
NUnit Adapter 3.8.0.0: Test execution complete
Total tests: 63. Passed: 63. Failed: 0. Skipped: 0.

Some people having a simillar issue: https://github.com/dotnet/corefx/issues/20091

Hi,

Upon successfully installing the driver package in a vs2019 console app (.net core 3.1) (PM> Install-Package Aeron.Driver) no such folder 'driver' is created in the solution so i'm unable to run the driver via the .bat file since it doesn't exist.

Any thoughts would be much appreciated - thank you !

The constructor of AtomicCounter immediately writes 0 into the underlying counter, which makes it impossible to use this class for just reading a counter.

Notably, the java version doesn't write to the counter: https://github.com/real-logic/agrona/blob/master/agrona/src/main/java/org/agrona/concurrent/status/AtomicCounter.java

Would it be possible to change this or create a constructor overload (or parameter) that doesn't overwrite the value?

Thanks for great port of Aeron library.

Are there any plans to use the C Media Driver instead of the Java version of the Media Driver? I'm assuming it would no longer need installation of Java and be easier to control from within C#.

It is quite easy to pack a custom file as a resource into a dll and then extract it into a temp or fixed folder on application startup or some type initialization. Java is installed on most machines these days. Packed media driver will make .NET development experience seamless. It is not a big deal to start it manually from source, but NuGet experience and distribution of updates would be simpler.

Port changes from real-logic/aeron@a15baaa to real-logic/aeron@fbefcad

Are you going to support .NET Core soon? It should be quite trivial unless I missed something. It is mostly #ifs for different APIs, e.g. MemoryBarrier was moved from Thread to Interlocked, deprecated MemoryMappedFileSecurity, etc.

Hi, I generated some sbe codecs, and I wanted to use in clusterClient.Offer:

Java:

        MessageHeaderEncoder headerEncoder = new MessageHeaderEncoder();
        MutableDirectBuffer requestBuffer = new ExpandableArrayBuffer();
        RequestSaveEncoder requestSaveEncoder = new RequestSaveEncoder();

        requestSaveEncoder.wrapAndApplyHeader(requestBuffer, 0, headerEncoder);
        requestSaveEncoder.customerId(customerId);

        idleStrategy.reset();
        while (aeronCluster.offer(requestBuffer, 0,
                headerEncoder.encodedLength() + requestSaveEncoder.encodedLength()) < 0)
        {
            idleStrategy.idle(aeronCluster.pollEgress());
        }

.Net:

               RequestQuery request = new RequestQuery();
                var header = new MessageHeader();
                var bytesBuffer = new byte[200];
                var msgBuffer = new DirectBuffer(bytesBuffer);
                var msgBuffer2 = new UnsafeBuffer(bytesBuffer);

                request.WrapForEncodeAndApplyHeader(msgBuffer, 0, header);
                request.CustomerId = 12;

                while (c.Offer(msgBuffer2, 0, request.Size + MessageHeader.Size) < 0)
                {
                    idleStrategy.Idle();
                }

in sdk offer function accepts "Adaptive.agrona.IDirectBuffer" and msgBuffer type is "Org.SbeTool.sbe.Dll.DirectBuffer"

what is correct way of using sbe generated files in clusterClient offer?

Hello, I'm using aeron for my .net program and I find that when I build in release mode and offer buffers to a publisher, my program crashes after a little while.
The error in my log is

An unhandled exception of type 'System.AccessViolationException' occurred in Adaptive.Agrona.dll
Attempted to read or write protected memory. This is often an indication that other memory is corrupt.

My callstack looks like

	Adaptive.Agrona.dll!Adaptive.Agrona.Concurrent.UnsafeBuffer.GetLongVolatile(int index)	Unknown	No symbols loaded.
 	Adaptive.Agrona.dll!Adaptive.Agrona.Concurrent.Broadcast.BroadcastReceiver.ReceiveNext()	Unknown	No symbols loaded.
 	Adaptive.Agrona.dll!Adaptive.Agrona.Concurrent.Broadcast.CopyBroadcastReceiver.Receive(Adaptive.Agrona.Concurrent.MessageHandler handler)	Unknown	No symbols loaded.
 	Adaptive.Aeron.dll!Adaptive.Aeron.ClientConductor.Service(long correlationId, string expectedChannel)	Unknown	No symbols loaded.
 	Adaptive.Aeron.dll!Adaptive.Aeron.ClientConductor.DoWork()	Unknown	No symbols loaded.
 	Adaptive.Agrona.dll!Adaptive.Agrona.Concurrent.AgentRunner.DoDutyCycle(Adaptive.Agrona.Concurrent.IIdleStrategy idleStrategy, Adaptive.Agrona.Concurrent.IAgent agent)	Unknown	No symbols loaded.
 	Adaptive.Agrona.dll!Adaptive.Agrona.Concurrent.AgentRunner.Run()	Unknown	No symbols loaded.
 	System.Threading.Thread.dll!System.Threading.Thread.ThreadMain_ThreadStart()	Unknown	No symbols loaded.
 	System.Private.CoreLib.dll!System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state)	Unknown	No symbols loaded.

And the code that causes the issue is
let pub = aeron.AddExclusivePublication(config.Channel, config.PubId)

and in a loop that is called every 4 seconds I execute

    let arr = Array.zeroCreate 256
    use buffer = new UnsafeBuffer(arr)
    pub.Offer buffer

After 10-15 seconds I get the mentioned error. Am I doing anything obviously wrong?

Thanks,
Jeffrey

Target netstandard2.0 for libraries and netcoreapp3.1 (LTS) for samples and tests.

Hi!

Maybe I forgot something or never understood, but it's something probably really stupid.

If I put 2048 here

I'm having more than 1% errors. And the error is that this line:

returns zero and then Hdr throws.

This in on localhost.

Are you able to run this exact sample on your machines with 2048+ message sizes and without a single error?

PSs

(Otherwise it looks like sender's write is not visible after round-trip, but I hope I'm missing something simpler. Is it supposed to work with MTU+ sized messages?)

(I wrapped writing to Hdr in try/catch and see <99% message count, this is how I calculate >1% error rate)

(MTU in the title because I was testing latency below and above that. 1024-1350 works fine, 1500+ sizes give similar errors, so maybe it's related)

The AgentRunner on this line starts a foreground thread for the aeron client conductor.

So if I use this and the main thread of my application crashes in a way that is not able to Dispose the AeronClient, then this thread will keep the application alive indefinitely. I find this behavior very undesirable.

Would it be possible to add an option to start it as a background thread?

Copied from #31
There is call vs callvirt and interface method call minor issue. In JVM, everything is an object and probably method calls are optimized more than in CLR. However, inside tight loops I have noticed that interface method calls are noticeably slower than direct method calls. In the Direct/UnsafeBuffer case, all we do is writing to/reading from a pointer in unsafe code, e.g. PutLog, etc. Usually this is a very simple method with a couple of instructions, and then method call overhead becomes visible. That could matter in IPC case for Aeron, less so for networking. In your implementation, there is only one implementation of different buffer interfaces, so, at this stage, class hierarchy adds only overheads. Also, UnsafeBuffer currently works both with byte[] and IntPtr, and a single concrete implementation as a struct will be the fastest. (I separated them in my implementation initially, but will soon merge back to avoid using interfaces altogether).

Hi there! Thank you for the great work done on Aeron.NET!

I am playing around with the SimplePublisher and SimpleSubscriber samples using aeron:ipc channels. In my scenario, I have 1 publisher publishing "Hello + time" every second and 4 subscribers that print to screen what they receive and then sleep for 0.5s, 1s, 1.5s and 2.5s respectively. (The sleep is to simulate slow receivers.) The first 2 subscribers are in tandem with the publisher. The slower subscriber are however showing the older messages as they try to catch up to the publisher. This is expected behavior right?

What I want though is for those slow receivers to skip to the latest messages published. I am OK with dropping old messages. Is there a way to achieve this programatically or via media-driver configuration?

I just stumbled upon this line:

The default error handler calls Environment.Exit(-1) when the driver connection is lost. It took me a while to track this down as we are running in a Windows Service which doesn't have Console.Error by default. So the preceding messages are lost, all you get is a process that just exits intermittently without any indication as to why. While obviously one should anyway track Console.Error output and always provide a custom error handler, I think this may come as a surprise to others as well.

I would say it is rather unexpected that a library terminates the hosting process, especially as default behaviour. Shouldn't this be removed, i.e. the Console.Error output should be enough in a default error handler?

Useful as a scratch buffer when writing messages where size is not known up front.

Hi @JPWatson

I try to upgrade to netstandard and netcore3.1 to net6.0 then i run IpcThroughput

here's the result before upgrading

Duration 1,000ms - 10,062,930 messages - 322,013,760 bytes, GC0 0, GC1 0, GC2 0
Duration 1,000ms - 10,063,976 messages - 322,047,232 bytes, GC0 0, GC1 0, GC2 0
Duration 1,000ms - 10,065,513 messages - 322,096,416 bytes, GC0 0, GC1 0, GC2 0
Duration 1,000ms - 10,057,301 messages - 321,833,632 bytes, GC0 0, GC1 0, GC2 0
Duration 1,000ms - 10,057,799 messages - 321,849,568 bytes, GC0 0, GC1 0, GC2 0
Duration 1,000ms - 10,060,503 messages - 321,936,096 bytes, GC0 0, GC1 0, GC2 0
Duration 1,000ms - 10,062,570 messages - 322,002,240 bytes, GC0 0, GC1 0, GC2 0
Duration 1,000ms - 10,062,352 messages - 321,995,264 bytes, GC0 0, GC1 0, GC2 0
Duration 1,000ms - 10,059,734 messages - 321,911,488 bytes, GC0 0, GC1 0, GC2 0

then after upgrading

Duration 1,000ms - 9,474,969 messages - 303,199,008 bytes, GC0 0, GC1 0, GC2 0
Duration 1,000ms - 9,474,044 messages - 303,169,408 bytes, GC0 0, GC1 0, GC2 0
Duration 1,000ms - 9,471,702 messages - 303,094,464 bytes, GC0 0, GC1 0, GC2 0
Duration 1,000ms - 9,473,014 messages - 303,136,448 bytes, GC0 0, GC1 0, GC2 0
Duration 1,000ms - 9,475,162 messages - 303,205,184 bytes, GC0 0, GC1 0, GC2 0
Duration 1,000ms - 9,474,305 messages - 303,177,760 bytes, GC0 0, GC1 0, GC2 0
Duration 1,000ms - 9,432,768 messages - 301,848,576 bytes, GC0 0, GC1 0, GC2 0
Duration 1,000ms - 9,465,044 messages - 302,881,408 bytes, GC0 0, GC1 0, GC2 0
Duration 1,000ms - 9,467,925 messages - 302,973,600 bytes, GC0 0, GC1 0, GC2 0
Duration 1,000ms - 9,468,866 messages - 303,003,712 bytes, GC0 0, GC1 0, GC2 0

The environment i use is Amazon Linux 2022 on Graviton3 (ARM Neoverse V1) AWS c7g.

Looks weird. Will try to find out. It's also resulting the same stuff even after i try to disable/enable PGO and Tiered Compilation.

1. Is that why you don't upgrade the dotnet and still use it for old .NET version?
2. seems currently it's windows only? I might send pull request for Linux if that's interest you

Cheers

Are there any plans to continue updating this project to follow aeron as it is updated?

Cheers,
Jeff

• a3d92b6 [Java] Added toString() methods to each of the Aeron protocol messages flyweights to aid debugging.

• 35a9cf8 [Java] Add the ability to map log buffers read only. Useful for protecting images and using he LogInspector.

• 5d67a25 [Java] Migrate the responsibility for signalling when cleaning needs done from the publisher to the driver conductor.

• 2947adf [Java] Remove unused method.

  aacc406

Hello dear maintainer of Aeron.NET.

I was trying to use your library on Unix, and once I built the code, I ran into this issue:

Unhandled Exception:
Adaptive.Aeron.Exceptions.DriverTimeoutException: CnC file is created by not initialised.
  at Adaptive.Aeron.Aeron+Context.ConnectToDriver () <0x6c6478 + 0x001eb> in <filename unknown>:0 
  at Adaptive.Aeron.Aeron+Context.Conclude () <0x6c5c98 + 0x0014f> in <filename unknown>:0 
  at Adaptive.Aeron.Aeron..ctor (Adaptive.Aeron.Context ctx) <0x6c5ba8 + 0x00017> in <filename unknown>:0 
  at Adaptive.Aeron.Aeron.Connect (Adaptive.Aeron.Context ctx) <0x6c5b50 + 0x0002f> in <filename unknown>:0 
  at Adaptive.Aeron.Samples.Ping.Ping.Main () <0x6c3278 + 0x00337> in <filename unknown>:0 
[ERROR] FATAL UNHANDLED EXCEPTION: Adaptive.Aeron.Exceptions.DriverTimeoutException: CnC file is created by not initialised.
  at Adaptive.Aeron.Aeron+Context.ConnectToDriver () <0x6c6478 + 0x001eb> in <filename unknown>:0 
  at Adaptive.Aeron.Aeron+Context.Conclude () <0x6c5c98 + 0x0014f> in <filename unknown>:0 
  at Adaptive.Aeron.Aeron..ctor (Adaptive.Aeron.Context ctx) <0x6c5ba8 + 0x00017> in <filename unknown>:0 
  at Adaptive.Aeron.Aeron.Connect (Adaptive.Aeron.Context ctx) <0x6c5b50 + 0x0002f> in <filename unknown>:0 
  at Adaptive.Aeron.Samples.Ping.Ping.Main () <0x6c3278 + 0x00337> in <filename unknown>:0 

It happens with all the samples, the above is the Ping sample.

Probably near 100% of the people who will fork this repo will have name conflict with the original Java/C++ Aeron repo, and their fork will be renamed to Aeron-1 or similar. Unless this is going to be merged in the same way as C++/Java coexist together in the original repo, Aeron.NET is a less ambiguous name.

• How were the codecs (e.g. NewLeaderEventDecoder) in this project generated, please?
• Crucially, the generated code of the codecs in this project look different to what's generated by latest SbeTool
• The Adaptive.Agrona.IFragmentHandler.OnFragment() accepts a Adaptive.Agrona.IDirectBuffer parameter
• However, the SbeTool generated codec operate on Org.SbeTool.Sbe.Dll.DirectBuffer which does not implement Adaptive.Agrona.IDirectBuffer
• How can I translate between these two buffers (i.e. Org.SbeTool.Sbe.Dll.DirectBuffer <-> Adaptive.Agrona.IDirectBuffer) please ?

The Adaptive.Cluster project already builds a .nupkg, but I can't find Aeron.Cluster on nuget.org. Is there a reason this isn't published there?

Would it be possible to package Agrona as a separate nuget package. It would be nice to use some of the data structures without pulling in the whole Aeron.Client nuget package.

I wonder why you do not use structs vs classes where it makes code more idiomatic to .NET and helps to avoid allocations? I see that you have classes vs structs on some hot paths. E.g. BufferClaim is a class. UnsafeBuffer is also a class, however both are very close to ArraySegment<byte> by usage semantics. There is a mantra that short-lived objects are very cheap to allocate and collect in .NET, but the costs are not zero.

Or maybe your intent is to reuse objects when possible and use Wrap() methods, as commented here? E.g. you mutate BufferClaim that is passed to a method and then use the same instance in a loop. But what if we want to claim space for a result of some work, and then complete this work asynchronously using a task? We could pass a struct easily by value between threads in such a scenario, but cannot reuse a class and have to allocate. The same is true for UnsafeBuffers - if they are structs, we could create as many instances as convenient for free, and still reuse a single instance if we want to do so.

Additionally, there is call vs callvirt and interface method call minor issue. In JVM, everything is an object and probably method calls are optimized more than in CLR. However, inside tight loops I have noticed that interface method calls are noticeably slower than direct method calls. In the Direct/UnsafeBuffer case, all we do is writing to/reading from a pointer in unsafe code, e.g. PutLog, etc. Usually this is a very simple method with a couple of instructions, and then method call overhead becomes visible. That could matter in IPC case for Aeron, less so for networking. In your implementation, there is only one implementation of different buffer interfaces, so, at this stage, class hierarchy adds only overheads. Also, UnsafeBuffer currently works both with byte[] and IntPtr, and a single concrete implementation as a struct will be the fastest. (I separated them in my implementation initially, but will soon merge back to avoid using interfaces altogether).

I have just done a trivial test to show this. At first, I tested integer increment (i++), and struct method call was completely eliminated/inlined. In a more realistic scenario, structs do not differ much from sealed classes, but interface overheads are clearly visible.

Struct vs classes was the only change I did while porting LogBuffers. My implementation of DirectBuffer was originally based on a DirectBuffer C# port in SBE by Adaptive (which, in turn, was the same port from Argona), but I then reworked it for the reasons described above.

I recently updated from 1.4.0 to 1.9.3 and I have been seeing a bug every now and then where messages will not be sent over ipc. I haven't been able to reproduce this in debug or even reliably in release builds and the only time I've seen it happen is when I've had a connection open for a long period of time (multiple days) without sending any message over aeron. When I would then try to send a message the subscriber would not get any message and I haven't seen any error from the publisher. Then, when I restart the publisher application, the subscriber will receive all the messages I tried to send previously, and have no further problems sending messages.
While I continue to look into this to get more detail, my hunch is that there was some timeout added to connections if messages were not sent over a long period of time. Is this possibly the case? If not I'll try and keep digging.

Thank you,
Jeffrey

I am running media-driver.bat from the Java Aeron repo. When one of my Aeron.NET publishers tried to call Offer, it resulted in Publication.ADMIN_ACTION persistently.

What could have caused it? Are they any logs showing what is going on under the hood?

Aeron

• 88e974e [Java] Apply mapMode for largest size term buffers.
• e8051ac [Java] Link to .NET repo and update CMAKE requirements.
• 7483adc [Java] typo.
• 79b428d [Java] Add Javadoc to defaults in Aeron client.
• 4f236b0 [Java] Add the ability to set the driver error handler in the context.
• 6a84957 [Java] Group related properties and improve javadoc.
• 660c0a6 [Java] Whitespace.
• 0755f13 [Java] Further reorg of configuration properties.
• 3fae39e [Java] Further tidy up and javadoc for configuration.
• 8fabdbe [Java] Javadoc.
• 6a7c97b [Java] Javadoc.
• b7cd7ca [Java] Naming and increase size of event buffer to 2MB.
• afdb130 [Java] Javadoc.
• d7ee8be [Java] Update to ByteBuddy 1.4.6.
• 74b49f9 [Java] Limit scope of constants.
• db0aa9b [Java] Javadoc.
• f2182f5 [Java] On reflection the data frame header can be done with normal writes in any order given how the unblocker now works.
• 0516a57 [Java] Revert last change.
• 92b1acc [Java] Remove unused status field accessors.
• fcc5ff0 [cpp] Removed unused method.
• fae5d09 [Java] Depend on Agrona snapshot.
• 948d8b2 [Java] Correct typo and use CommonContext.SPY_PREFIX.
• 8d84f75 [Java] Busy spin the conductor for low latency.
• c2d7efd [Java] Only run client conductor duty cycle one every 10ms.
• 47c9b11 [Java] What for driver to initialise the CnC file. Issue #251.
• 1e1c1c3 [Java] Wait on CnC file to exist for allowing client to start before the driver. Issue #251.
• 96dcb0d [Java] Javadoc.
• afa4899 [Java] Javadoc.
• a93ca0e [Java] Writing the length field can be a normal put long provided it is followed by a store fence for the first write of the header fields.
• 14f2411 [C++]: Disable bounds checks for release builds of media driver.
• 4e014cf [C++]: add thread_fence() after volatile write of negative frame length.
• 714113a [Java] Use arrays rather than ArrayLists for positions in publications and images in the driver.
• 41a18e1 Merge remote-tracking branch 'origin/master'
• 0879627 [Java] Upgrade to ByteBuddy 1.4.7.
• 875f7bb [Java] Upgrade to ByteBuddy 1.4.8.
• 3733f6c [Java] Move tail counters into log meta data for the Publication which allows for simplification of the log structure and reduce data dependant loads throughout Aeron.
• 99ebefd [Java] Place the active partition index in the same cache line as the tail counters within the log.
• f704681 [C++]: reflect tail counters being moved into log meta data buffer.
• 18ec519 [C++]: add spins on CnC file to Aeron constructor.
• c54f276 [Java] Javadoc correction.
• dee27a0 Merge remote-tracking branch 'origin/master'
• b8d83be [Java] Give up the thread on an setup thread which is adding publications or subscriptions to that CPU resource is available to other threads to complete the action. Issue #244.
• 8815092 [Java] Naming.
• 3352811 [Java] Incremental cleaning of log buffers by the conductor to smooth out pauses in the driver conductor.
• 5800f5f [Java] Lag the cleaning by a term + window to allow for greater NAK range.
• b7a0be3 [Java] Naming of cleaning variables for clarity.
• b882e71 [Java] Check resend of a NAK is within one term length behind the sender position. Also clean incrementally from 2 term lengths behind publication limit for the NetworkPublication. Issue #257.
• 21c27b0 [Java] Naming.
• 3d060f1 [Java] Naming.
• d6eec29 [Java] README.
• 32504cb [Java] README.
• 6f2d1f1 [Java]: avoid issuing availableImage on addSubscription when existing image status isn't ACTIVE.
• bbaf75d [Java]: switch filtering of addImage for availableImage to be correlationId based instead of sessionId.
• bb6f682 [C++]: make hasImage be correlationId based.
• 6d3d702 [Java] naming.
• 34d53ea Merge branch 'master' of https://github.com/real-logic/Aeron
• 023d9e2 [Java] Use concrete typed position counters in status package.
• 80c5710 [C++] Added setting clientId on remove messages to the driver
• f07738b [Java] Simplify DriverProxy as it is only called on the conductor lock and benefit from the clientId field set in the buffer.
• 2808ab8 [Java] Simplify DriverProxy as it is only called under the client conductor lock and benefit from the clientId field set in the buffer.
• d1a76af Merge pull request #259 from lirm/master
• 27ade16 [Java] Styling.
• 677399a [Java] Upgrade to ByteBuddy 1.4.9.
• 957e552 [Java] Upgrade to Checkstyle 7.0.
• d31a838 [Java] Styling.
• 375b625 [Java] Exposed max payload length from a publication.
• c2e5378 [C++]: add maxPayloadLength to Publication
• 7b21578 1.0 released

Hello, it has been half a year since dotnet5 was released.
Is there any plan to support dotnet5?
Thanks.

Connecting with a CloseHandler passed via the Context causes a NullReferenceException:

var ctx = new Aeron.Context()
                .CloseHandler(() => { });
var aeron = Aeron.Connect(ctx);

causes a NullReferenceException here:

Adaptive.Aeron.dll!Adaptive.Aeron.Aeron.NextCorrelationId() Line 347	
Adaptive.Aeron.dll!Adaptive.Aeron.ClientConductor.ClientConductor(Adaptive.Aeron.Aeron.Context ctx, Adaptive.Aeron.Aeron aeron) Line 127	
Adaptive.Aeron.dll!Adaptive.Aeron.Aeron.Aeron(Adaptive.Aeron.Aeron.Context ctx) Line 68	
Adaptive.Aeron.dll!Adaptive.Aeron.Aeron.Connect(Adaptive.Aeron.Aeron.Context ctx) Line 117	

The reason seems to be that the ClientConductor ctor tries to get a NextCorrelationId(), but this in turn needs the Aeron._clientConductor that has not been constructed yet and is still null.

Note: A workaround for this is to not use the Context.CloseHandler() but instead use Aeron.AddCloseHandler() after connecting.
I noticed the issue when upgrading from 1.29.0 to 1.39.0, so it must have been introduced between those versions.

https://github.com/dotnet/corefxlab/blob/master/docs/specs/span.md

It seems the obvious place to start incorporating Span would be to add methods on the client facing APIs.
i.e. add a Publication.Offer(Span<byte>) and SpanFragmentHandler.OnHandle(Span<byte> buffer) + Image/Subscription overloads.

Open Questions

• How to replicate the current buffer operations efficiently with Span? GetLong(offset, Endianess)/PutInt/GetIntVolatile/....
• Where, if anywhere, should we expect to see performance improvements? (working with strings)
• What is the status of Memory?

Help & input is welcome.

Having removed a memory barrier here, there was quite a larger performance:
https://github.com/AdaptiveConsulting/Aeron.NET/pull/43/files

However @omariom pointed out that under the ECMA standard the writes can be re-ordered.

A strategy for protecting against re-ordering volatile writes while keeping most of the performance was to insert a dummy Volatile.Read:
https://github.com/dotnet/coreclr/issues/6121

But there is a bug in the RyuJIT that means that it can be dropped:
https://github.com/dotnet/coreclr/issues/6172

Given that C# compiler doesn't order writes, this isn't a problem for now but I'll leave this here for future reference and we can come back to it once the compiler has been fixed.

Would you consider a PR introducing a batched read for ring buffers, accessing the buffer directly and passing it to a handler? Something similar to https://github.com/Scooletz/RampUp/blob/feature/read-raw-chunk/src/RampUp/Ring/ManyToOneRingBuffer.cs#L129-L179

This would allow to use ring buffer as a "concurrent queue that can be read in batches" and clear the memory afterwards.

Moved from real-logic/simple-binary-encoding#492 (@juddgaddie)

Like Java, C# (unlike c/c++) is typically not compiled from sources for that reason I think the bounds checking would be better served with a runtime toggle we could use system environment variables (the same approach as c or c++ code uses for runtime configuration).

I also would think the default build that goes to Nuget should have bounds checking enabled by default.

Example of what I mean here: https://gist.github.com/juddgaddie/eeafbbb195a332bf72808748cdcd4336

Unless I am missing something inside MediaDriver implementation, it looks like both fences are not needed here because a frame is committed only after its lengths is written atomically. All usages of headerWriter.Write are followed by FrameDescriptor.FrameLengthOrdered, which is implemented as Volatile.Write that implicitly generates a memory barrier. TermReader then reads that length through Volatile.Read here and doesn't touch any other frame part until the length is positive. On my machine replacing the two lines with a simple PutLong increases IPC performance from c.14 Mops to c.16 Mops.

In .NET we could even create a DataHeader struct and write it directly like this. Also, HeaderWriter could be made sealed or a struct. These additional changes together add c.0.5 Mops to the IPCThroughput sample.