Having an indirect problem with this library. Does it work outside the Window?

[14:21:05 FTL] Unhandled exception
System.PlatformNotSupportedException: Socket.IOControl handles Windows-specific control codes and is not supported on this platform.
   at System.Net.Sockets.SocketPal.WindowsIoctl(SafeSocketHandle handle, Int32 ioControlCode, Byte[] optionInValue, Byte[] optionOutValue, Int32& optionLength)
   at System.Net.Sockets.Socket.IOControl(Int32 ioControlCode, Byte[] optionInValue, Byte[] optionOutValue)
   at System.Net.Sockets.Socket.IOControl(IOControlCode ioControlCode, Byte[] optionInValue, Byte[] optionOutValue)
   at AsyncIO.DotNet.NativeSocket.IOControl(IOControlCode ioControlCode, Byte[] optionInValue, Byte[] optionOutValue)

@somdoron is this expected?
Below code throws excetion.

static void Main(string[] args)
{
var s = AsyncSocket.Create(AddressFamily.InterNetwork, SocketType.Dgram, ProtocolType.Udp);
s.Bind(IPAddress.Parse("127.0.0.1"), 27000);

        var c = AsyncSocket.Create(AddressFamily.InterNetwork, SocketType.Dgram, ProtocolType.Udp);
        c.Connect(IPAddress.Parse("127.0.0.1"), 27000);
    }

Unhandled Exception: System.Net.Sockets.SocketException: An invalid argument was supplied
at AsyncIO.Windows.Socket.Connect(IPEndPoint endPoint) in D:\Repo\AsyncIO\Source\AsyncIO\Windows\Socket.cs:line 440
at AsyncIO.SocketExtensions.Connect(AsyncSocket socket, IPAddress ipAddress, Int32 port) in D:\Repo\AsyncIO\Source\AsyncIO\SocketExtensions.cs:line 19
at udptest.Program.Main(String[] args) in D:\Repo\dxdjglnetmqudp\src\udptest\Program.cs:line 22
Press any key to continue . . .

I've forked the current codebase with the intent to investigate a performance profiler hotspot but I can't get it to build. The project seems to be hardcoded into a .netstandard 1.3 build configuration and I've no ability to change it, to anything. I'm using VS 2017.2

Can you give me some guidance how I would build a full framework build to try out some options?

Microsoft allows only a subset of the namespaces System.Net.Sockets & System.Net.IPAdress for Windows Store apps. Instead they propose to use the more modern Windows.Networking namespace.
To make AsyncIO and thereby netMQ ready for Windows Store it would be great to have this changes included.

method that return multiple completion statuses for improved performance

Currently the APIs of AsyncSocket appear to support async TCP communications only.
Does it make sense to make an async UDP socket?

instead of call accept with socket the socket will be retrieved with get accepted socket

Hello,

I get this intermittently when I'm trying to load test my communication component based on NetMQ 3.3.3.4, sending messages from thousands of router sockets on a single machine, connected to a single router socket on the "server":
Unhandled Exception: System.AccessViolationException: Attempted to read or write protected memory. This is often an indication that other memory is corrupt.
at System.Runtime.InteropServices.Marshal.ReadInt32(IntPtr ptr, Int32 ofs)
at AsyncIO.Windows.Overlapped.get_Success()
at AsyncIO.Windows.CompletionPort.HandleCompletionStatus(CompletionStatus& completionStatus, IntPtr overlappedAddress, IntPtr completionKey, Int32 bytesTransferred)
at AsyncIO.Windows.CompletionPort.GetMultipleQueuedCompletionStatus(Int32 timeout, CompletionStatus[] completionStatuses, Int32& removed)
at NetMQ.Core.Utils.Proactor.Loop()
at System.Threading.ThreadHelper.ThreadStart_Context(Object state)
at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
at System.Threading.ThreadHelper.ThreadStart()

I can't really include my code, but it seems like there is a possible bug in AsyncIO library?

Martin

This is a bit confusing when implementing binding redirects.

Memory buffer is allocated when calling getter on properties RemoteEndPoint and LocalEndPoint and this memory is not freed. See the constructor of SocketAddress line 29:

      m_bufferHandle = GCHandle.Alloc(m_buffer, GCHandleType.Pinned);

The buffer is freed by Dispose() (SocketAddress.cs). Possible fix below:

        public override IPEndPoint RemoteEndPoint
        {
            get
            {
                using ( var socketAddress = new SocketAddress( AddressFamily, AddressFamily == AddressFamily.InterNetwork ? 16 : 28))
                {
                    int size = socketAddress.Size;

                    if (UnsafeMethods.getpeername(Handle, socketAddress.Buffer, ref size) != SocketError.Success)
                    {
                        throw new SocketException();
                    }

                    return socketAddress.GetEndPoint();
                }
            }
        }

        public override IPEndPoint LocalEndPoint
        {
            get
            {
                using (var  socketAddress = new SocketAddress(AddressFamily, AddressFamily == AddressFamily.InterNetwork ? 16 : 28))
                { 
                    int size = socketAddress.Size;

                    if (UnsafeMethods.getsockname(Handle, socketAddress.Buffer, ref size) != SocketError.Success)
                    {
                        throw new SocketException();
                    }
                    else
                    {
                        return socketAddress.GetEndPoint();
                    }
                }
            }
        }

Hi,
i got to this point using netmq library, i'm using a SubscriberSocket to connect to a remote publisher, on a netcore 3.01 console application.

I run the application with normal connectivity and it works as expected, i receive data.
Then i unplug the network (disable wifi for example) and correctly the api SubscriberSocket.TryReceiveFrameBytes returns false, if i turn on the network then on some computers (windows 10 1909) it start working again, on some other (same windows version and same .netcore version) it remain stuck.

I diagnosed the problem a bit thanks to netmq and asyncio source code, i got to this line of code

The even stranger thing is that if i unplug the network for less that 10seconds it recovers, otherway it does not 😄

I'm sorry if i'm not able to provide any further detail, and i could not understand what's differente on the computers to cause the problem (wifi driver ?). Do you have any adivce on how to diagnose/ fix the problem ?
Thanks

Hi,

I am using Dealer-Broker pattern in NetMQ (4.0.0.1), ASyncIO(0.1.26.0)
I can see some AccessViolationException in my UnHandledException logs, Not Sure where it is coming from. I am suspecting that the issue is happening during dealer creation.

CALL STACK OF EXCEPTION AccessViolationException: Attempted to read or write protected memory. This is often an indication that other memory is corrupt.
at System.Runtime.InteropServices.Marshal.ReadInt32(IntPtr ptr, Int32 ofs)
ExceptionServices(Kernal.Common): at AsyncIO.Windows.Overlapped.get_Success()
ExceptionServices(Kernal.Common): at AsyncIO.Windows.CompletionPort.HandleCompletionStatus(CompletionStatus& completionStatus, IntPtr overlappedAddress, IntPtr completionKey, Int32 bytesTransferred)
ExceptionServices(Kernal.Common): at AsyncIO.Windows.CompletionPort.GetMultipleQueuedCompletionStatus(Int32 timeout, CompletionStatus[] completionStatuses, Int32& removed)
ExceptionServices(Kernal.Common): at NetMQ.Core.Utils.Proactor.Loop()
ExceptionServices(Kernal.Common): at System.Threading.ThreadHelper.ThreadStart_Context(Object state)
ExceptionServices(Kernal.Common): at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
ExceptionServices(Kernal.Common): at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
ExceptionServices(Kernal.Common): at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
ExceptionServices(Kernal.Common): at System.Threading.ThreadHelper.ThreadStart()

Downloaded AsyncIO source for 0.1.69.0 , built code produces 0.1.42.0. I have a requirement to have the source code for the libraries I use. I am upgrading to NetMQ to 4.0.0.0 and it requires AsyncIO 0.1.69.0.

I have been using AsyncIO to write TCP applications for quite sometime. However, I encountered weird behaviors when the workload went a bit heavy, about several thousands of concurrent connections at a time.

I then studied the source code and deduct the possible production procedures of the issue:

1. Called the Send or Receive method of the AsyncSocket (via the Windows implementation, the one that P/Invoke winsocks).
2. Dispose the AsyncSocket before the overlapped operation is completed.
   1. The CancelIoEx is called, the cancellation is send on its way.
   2. closesocket is called.
   3. The Dispose (Free) of m_receivePinnedBuffer and m_sendPinnedBuffer will be called.
3. GC reclaims the memory assigned to AsyncSocket.

According to the documetation of CancelIoEx:

Most types of operations can be canceled immediately; other operations can continue toward completion before they are actually canceled and the caller is notified. The CancelIoEx function does not wait for all canceled operations to complete.
...

The operation being canceled is completed with one of three statuses; you must check the completion status to determine the completion state. The three statuses are:

1. The operation completed normally. This can occur even if the operation was canceled, because the cancel request might not have been submitted in time to cancel the operation.
2. The operation was canceled. The GetLastError function returns ERROR_OPERATION_ABORTED.
3. The operation failed with another error. The GetLastError function returns the relevant error code.

Thus, theoretically, the overlapped operation may still complete after the Dispose of AsyncSocket is called and write data back to the buffer that m_receivePinnedBuffer points to.

In most cases, it's OK since the aforementioned buffer is pointing to a managed byte array, which is just unpinned by the Free method and it usually may still be there.

However, if GC happens before the write-back, things can go wrong since the buffers have already been unpinned and can be moved by GC.

A rarer case is that the GC occurs when an overlapped sending operation and incorrect bytes will be read
by the outgoing sender, before the cancellation is executed.

How do you think about the above situation? Please correct me if I am wrong.

I've been using NetMQ in production for a while. NETMQ version number is 4.0.0.1,AsyncIO version number is 0.1.26.0
Recently, i found the program had crashed. Checked out the windows log to find out is an OOM exception。
Then I view the dump file through windbg.

!dumpheap -stat 
...
00007ff7ffbc0d50   536240     17159680 NetMQ.Core.Utils.Proactor+Item
00007ff7ffbca7f8   536242     17159744 NetMQ.Core.IOObject
00007ff7ffbcba70   536534     34338176 AsyncIO.Windows.AcceptExDelegate
00007ff7ffbcb7f0   536534     34338176 AsyncIO.Windows.ConnectExDelegate
00007ff7ffbcbdd8  1073068     60091808 AsyncIO.Windows.Overlapped
00007ff7ffbcb600   536534     90137712 AsyncIO.Windows.Socket
Total 3839215 objects

I checked the status of several objects. Confirm that the AsyncSocket object has called the Dispose method，and the value of InProgress equels true

!mdt 000000000390db20
000000000390db20 (AsyncIO.Windows.Overlapped)
    m_address:000000001ef5ffb0 (System.IntPtr)
    m_handle:(System.Runtime.InteropServices.GCHandle) VALTYPE (MT=00007ff85e5f3bc0, ADDR=000000000390db48)
    <OperationType>k__BackingField:0x1 (Receive) (AsyncIO.OperationType)
    <AsyncSocket>k__BackingField:000000000390da78 (AsyncIO.Windows.Socket)
    <InProgress>k__BackingField:true (System.Boolean)
    <Disposed>k__BackingField:true (System.Boolean)
    <State>k__BackingField:000000000390de08 (NetMQ.Core.Utils.Proactor+Item)

check gc table

!gchandles
GC Handle Statistics:
Strong Handles: 520519
Pinned Handles: 84
Async Pinned Handles: 0
Ref Count Handles: 0
Weak Long Handles: 43
Weak Short Handles: 116
Other Handles: 0
Statistics:
...
00007ff7ffbcbdd8   511752     28658112 AsyncIO.Windows.Overlapped
Total 520762 objects

check Finalize Queue

!finq -stat
Generation 0:
       Count      Total Size   Type
---------------------------------------------------------
           1             168   AsyncIO.Windows.Socket

1 object, 168 bytes

Generation 1:
       Count      Total Size   Type
---------------------------------------------------------
        1008          169344   AsyncIO.Windows.Socket
           2              48   System.Windows.Forms.VisualStyles.VisualStyleRenderer+ThemeHandle

1,010 objects, 169,392 bytes

Generation 2:
       Count      Total Size   Type
---------------------------------------------------------
           1             776   FC.Main.frmMain
           1             104   AsyncIO.Windows.CompletionPort
      535525        89968200   AsyncIO.Windows.Socket
...

It can be concluded that AsyncSocket called Dispose, but because the overlapped operation was being processed, it was not eventually notified of the io completion port

Then I found a lot of junk connection requests in the program log. I found through wireshark clutch bag due to multiple abnormal connections

i check the source code

public override void Receive(byte[] buffer, int offset, int count, SocketFlags flags)
{
m_inOverlapped.StartOperation(OperationType.Receive);

            m_receiveWSABuffer.Pointer = new IntPtr(m_receivePinnedBuffer.Address + offset);
            m_receiveWSABuffer.Length = count;

            int bytesTransferred;

            SocketError socketError = UnsafeMethods.WSARecv(Handle, ref m_receiveWSABuffer, 1,
              out bytesTransferred, ref flags, m_inOverlapped.Address, IntPtr.Zero);

            if (socketError != SocketError.Success)
            {
                socketError = (SocketError)Marshal.GetLastWin32Error();

                if (socketError != SocketError.IOPending)
                {
                    throw new SocketException((int)socketError);
                }
            }   
}

if socket was reset when need to read and write ,the asyncsocket will be dispose and the Io operation canceled. InProgress equels true so that the Overlapped hand not be free. it may cause io completion notification may never come.

Is this and 《Could it be a potential issue of AsyncSocket? #26》is same problem?

I met with a situation that the Send operation would not receive notification from the CompletionPort thread until several minutes later, a SocketErrorCode.ConnectionReset finally arrived.
How can we specify a timeout to terminate the Send operations which take too long?

This is confusing when performing runtime redirection...

Currently not implemented

The m_acceptSocket and its related members like m_acceptSocketBufferAddress are used only when the socket is listening. For ordinary connect, receive and send operations, those fields serves nothing.

Do you think it appropriated to extract m_acceptSocket and its siblings to a AsyncListenerSocket which derives from AsyncIO.Windows.Socket, so we can save some bytes and improve the performance by a little margin when many connections take place.
The class may look like the following:

class AsyncIO.Windows.Socket {
   internal virtual Accept() {}
   internal virtual Listen() {}

   class AsyncListenerSocket : Socket {
        AsyncSocket m_acceptSocket;
        IntPtr m_acceptSocketBufferAddress;
        int m_acceptSocketBufferSize;
        AcceptExDelegate m_acceptEx;
        internal override Accept() { implementation goes here }
        internal override Listen() { implementation goes here }
   }
}

Hi Doron,

It seems that the released version on nuget has no version set. This is then causing a compilation warning with NetMQ.

Hi,
would you mind to release a new version in nuget ?
Thanks !

In the attached code snippet, the Send method is called by various threads.

Some calls will print
996 Send 10 on the console, meaning that the IO operation is incomplete--please see https://msdn.microsoft.com/en-us/library/windows/desktop/ms740668(v=vs.85).aspx
It seems that the bytes are still received by the server.

Is it safe to treat 996 a successful event, or change the calling procedure to wait for the event, like the following SO answer suggested?
https://stackoverflow.com/questions/35208259/overlapped-socket-io-wsagetoverlappedresult-fails-with-996-errorcode

static void Main(string[] args) {
	CompletionPort completionPort = CompletionPort.Create();

	AutoResetEvent listenerEvent = new AutoResetEvent(false);
	AutoResetEvent clientEvent = new AutoResetEvent(false);
	AutoResetEvent serverEvent = new AutoResetEvent(false);

	AsyncSocket listener = AsyncSocket.Create(AddressFamily.InterNetwork,
		SocketType.Stream, ProtocolType.Tcp);
	completionPort.AssociateSocket(listener, listenerEvent);

	AsyncSocket server = AsyncSocket.Create(AddressFamily.InterNetwork,
		SocketType.Stream, ProtocolType.Tcp);
	completionPort.AssociateSocket(server, serverEvent);

	AsyncSocket client = AsyncSocket.Create(AddressFamily.InterNetwork,
		SocketType.Stream, ProtocolType.Tcp);
	completionPort.AssociateSocket(client, clientEvent);

	int received = 0, sent = 0;
	Task.Factory.StartNew(() =>
	{
		CompletionStatus[] completionStatuses = new CompletionStatus[1000];
		while (true) {
			int removed;
			var result = completionPort.GetMultipleQueuedCompletionStatus(-1, completionStatuses, out removed);

			if (result) {
				for (int i = 0; i < removed; i++) {
					var completionStatus = completionStatuses[i];
					Console.WriteLine("{0} {1} {2}", completionStatus.SocketError,
						completionStatus.OperationType, completionStatus.BytesTransferred);
					//if ((int)completionStatus.SocketError == 996) {
					//	System.Diagnostics.Debugger.Break();
					//}
					switch (completionStatus.OperationType) {
						case OperationType.Send:
							Interlocked.Add(ref sent, completionStatus.BytesTransferred);
							break;
						case OperationType.Receive:
							Interlocked.Add(ref received, completionStatus.BytesTransferred);
							break;
					}
					if (completionStatus.State != null) {
						AutoResetEvent resetEvent = (AutoResetEvent)completionStatus.State;
						resetEvent.Set();
					}
				}
			}
		}
	});

	listener.Bind(IPAddress.Any, 5555);
	listener.Listen(1);

	client.Connect("localhost", 5555);

	listener.Accept(server);


	listenerEvent.WaitOne();
	clientEvent.WaitOne();

	object o = new object();
	Parallel.For(0, 10, _ => {
		for (int i = 0; i < 100; i++) {
			lock (o) {
				client.Send(new byte[10] { 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 }); 
			}
		}
		Thread.Sleep(10);
	});

	server.Receive(new byte[100000]);
	clientEvent.WaitOne();
	serverEvent.WaitOne();

	server.Dispose();
	client.Dispose();

	Console.WriteLine($"sent {sent} bytes, received {received} bytes");
}

I encountered a possible memory leak after running AsyncIO on an environment where connection resets or disconnections happen frequently.

I tried with some code to reproduce the problem and observed that the memory usage kept rising after full GC. The code could be downloaded here:
ConsoleApp1.zip

Run the code and it will keep creating servers and clients connecting each other and then closing the connections. The memory usage and generation object counts will be printed on the console.

....................................................................................................Finished
Mem used after GC:140432
Gen 0: 982
Gen 1: 13
Gen 2: 3
....................................................................................................Finished
Mem used after GC:146136
Gen 0: 1964
Gen 1: 25
Gen 2: 6
....................................................................................................Finished
Mem used after GC:152952
Gen 0: 2946
Gen 1: 37
Gen 2: 9
....................................................................................................Finished
Mem used after GC:160336
Gen 0: 3928
Gen 1: 49
Gen 2: 12
....................................................................................................Finished
Mem used after GC:164880
Gen 0: 4910
Gen 1: 61
Gen 2: 15
....................................................................................................Finished
Mem used after GC:167152
Gen 0: 5892
Gen 1: 72
Gen 2: 18

If you have Process Explorer (https://docs.microsoft.com/en-us/sysinternals/downloads/process-explorer), you can also double click the ConsoleApp1.exe process listed on it and take a look at the Gen 2 Heap Size from the .NET Performance tab in the process properties dialog. You will see that the Gen 2 Heap Size keep rising while the program runs.

After some times of running... the Gen 2 Heap Size and the Number of GC Handles had increased.

Consider the following code:

var socket = new RequestSocket();
socket.Connect("tcp://localhost:32700")

On Linux this works fine with Mono
On Windows this works fine with .NET
But on Windows this crashes with Mono

at AsyncIO.Windows.Socket.SetSocketOption

The first response might be: Just use .NET on Windows but I am using NetMQ in the Godot game engine. When you export your "game" it bundles Mono so it always targets and not .NET.

In other words AsyncIO/NetMQ works great in Godot on Linux, not so much on Windows.

Unhandled Exception:
System.Net.Sockets.SocketException (0x80004005): An invalid argument was supplied.

  at AsyncIO.Windows.Socket.SetSocketOption (System.Net.Sockets.SocketOptionLevel optionLevel, System.Net.Sockets.SocketOptionName optionName, System.Int32 optionValue) [0x00018] in <3c72113274de4b3face0e2579126a901>:0
  at AsyncIO.AsyncSocket.set_NoDelay (System.Boolean value) [0x00000] in <3c72113274de4b3face0e2579126a901>:0
  at NetMQ.Core.Transports.Tcp.TcpConnector.OutCompleted (System.Net.Sockets.SocketError socketError, System.Int32 bytesTransferred) [0x00051] in <21696b85a92a4a0eb8332ff57aebfd69>:0
  at NetMQ.Core.Transports.Tcp.TcpConnector.StartConnecting () [0x00097] in <21696b85a92a4a0eb8332ff57aebfd69>:0
  at NetMQ.Core.Transports.Tcp.TcpConnector.ProcessPlug () [0x0001b] in <21696b85a92a4a0eb8332ff57aebfd69>:0
waiting
  at NetMQ.Core.ZObject.ProcessCommand (NetMQ.Core.Command cmd) [0x0007a] in <21696b85a92a4a0eb8332ff57aebfd69>:0
  at NetMQ.Core.IOThread.Ready () [0x00016] in <21696b85a92a4a0eb8332ff57aebfd69>:0
  at NetMQ.Core.IOThreadMailbox.RaiseEvent () [0x00008] in <21696b85a92a4a0eb8332ff57aebfd69>:0
  at NetMQ.Core.Utils.Proactor.Loop () [0x00050] in <21696b85a92a4a0eb8332ff57aebfd69>:0
  at System.Threading.ThreadHelper.ThreadStart_Context (System.Object state) [0x00014] in <e04d3ec971c54d368992eafd86c45930>:0
  at System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state, System.Boolean preserveSyncCtx) [0x00071] in <e04d3ec971c54d368992eafd86c45930>:0
  at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state, System.Boolean preserveSyncCtx) [0x00000] in <e04d3ec971c54d368992eafd86c45930>:0
  at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state) [0x0002b] in <e04d3ec971c54d368992eafd86c45930>:0
  at System.Threading.ThreadHelper.ThreadStart () [0x00008] in <e04d3ec971c54d368992eafd86c45930>:0
[ERROR] FATAL UNHANDLED EXCEPTION: System.Net.Sockets.SocketException (0x80004005): An invalid argument was supplied.

  at AsyncIO.Windows.Socket.SetSocketOption (System.Net.Sockets.SocketOptionLevel optionLevel, System.Net.Sockets.SocketOptionName optionName, System.Int32 optionValue) [0x00018] in <3c72113274de4b3face0e2579126a901>:0
  at AsyncIO.AsyncSocket.set_NoDelay (System.Boolean value) [0x00000] in <3c72113274de4b3face0e2579126a901>:0
  at NetMQ.Core.Transports.Tcp.TcpConnector.OutCompleted (System.Net.Sockets.SocketError socketError, System.Int32 bytesTransferred) [0x00051] in <21696b85a92a4a0eb8332ff57aebfd69>:0
  at NetMQ.Core.Transports.Tcp.TcpConnector.StartConnecting () [0x00097] in <21696b85a92a4a0eb8332ff57aebfd69>:0
  at NetMQ.Core.Transports.Tcp.TcpConnector.ProcessPlug () [0x0001b] in <21696b85a92a4a0eb8332ff57aebfd69>:0
  at NetMQ.Core.ZObject.ProcessCommand (NetMQ.Core.Command cmd) [0x0007a] in <21696b85a92a4a0eb8332ff57aebfd69>:0
  at NetMQ.Core.IOThread.Ready () [0x00016] in <21696b85a92a4a0eb8332ff57aebfd69>:0
  at NetMQ.Core.IOThreadMailbox.RaiseEvent () [0x00008] in <21696b85a92a4a0eb8332ff57aebfd69>:0
  at NetMQ.Core.Utils.Proactor.Loop () [0x00050] in <21696b85a92a4a0eb8332ff57aebfd69>:0
  at System.Threading.ThreadHelper.ThreadStart_Context (System.Object state) [0x00014] in <e04d3ec971c54d368992eafd86c45930>:0
  at System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state, System.Boolean preserveSyncCtx) [0x00071] in <e04d3ec971c54d368992eafd86c45930>:0
  at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state, System.Boolean preserveSyncCtx) [0x00000] in <e04d3ec971c54d368992eafd86c45930>:0
  at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state) [0x0002b] in <e04d3ec971c54d368992eafd86c45930>:0
  at System.Threading.ThreadHelper.ThreadStart () [0x00008] in <e04d3ec971c54d368992eafd86c45930>:0

I could not find a ShutDown or Close method to shutdown a connection.
How can we:

• properly kick a client socket away from the server side?
• disconnect a client from the server?
• release the allocated resource when the client is disconnected, either naturally or accidentally?