stormhub / netuv Goto Github PK

I'm trying to run multiple loops (Loop class) in separate threads.
Its works but I have problems on application shutdown.

So it is possible to run two (or more) loops in separate threads (or tasks) ?
Any tips on how to do this properly ?

To encounter framework issue
Hi Johnny, I try to apply NetUV in my code(.Net Framework4.5.2), but have some problem about .netframework
, may you thinking to support.Net Framework4.52, thank you.

In the Example EchoServer/Client ,when remote tcp server or client disconnected,there are some logs printed on console,but no callback. How can I do jobs when client disconnect?

Can I join you?

https://github.com/libuv/libuv/blob/v1.x/test/test-poll.c

I'm trying to apply NetUV in my project.
I am sending large amounts of data and I need entire data to process them.

Now I handle data in OnRead(StreamHandle streamHandle, ReadableBuffer data) event but in ReadableBuffer data is (of course) chunked.
What is the best and most effective "NetUV" way to collect this data and process them ?

For example: I want to get and process 10 000 bytes. Now I'm using ReadableBuffer.ReadBytes, read current portion of data to temporary array and copy them to my big array allocated before. But in this situation I have a lot of unnecessary operations.
How to do it better ?

Due to the way NetUV's ReadableBuffer is written, reading the data into a Pipeline requires a temporary copy step. It would be preferable if this could be avoided.

The optimal solution would be of course having NetUV supporting pipelines natively. :)

I am working on a Nntp server/client that uses a /r/n command delimiter but really it could be any delimiter people want instead of a fixed string length. since you are wrapping a socket lib the chances are people will want it for some sort of protocol so some way to check the location of something in the buffer and to extract to that point would be great. I don't really want to manage a string buffer on top of our ReadableBuffer as it seems like a waste to double buffer.

can you give a performance and stable test

Is there a recommended way for using the client when connecting to a server that uses ssl? Interested in being able to run ssl for the server as well but most important is the client at the minute.

run sample EchoServer, and telnet 127.0.0.1 9988, and then close telnet window

the sample will print

dbug: ScheduleHandle[0]
      UV_TCP 1859236454736 read completed.
trce: ScheduleHandle[0]
      UV_TCP 1859236454736 Read stopped

but OnError callback never be called.

Abstract

I converted the EchoServer/Client example to FSharp and and am running into a problem on linux that I don't fully understand. You can find the code here.

When I run the code on Linux using mono 4.6 I get a strange error when the client connects to the server:

Echo client write error NetUV.Core.Native.OperationException: EFAULT : bad address in system call argument

Now, when I run this project on Windows, using the Libuv dlls (1.9.2) the code works.

My Linux distribution is NixOS, and the system version of libuv is 1.11.0. I also built and tried with 1.9.1, but the same error occurs (there was no tag/release for 1.9.2). Any hints how I can get this to work?

Thanks!

Server Log

[nix-shell:~/src/projects/iris/netuv/build]$ mono NetUv.Server.exe
info: NativeHandle[0]
      Loop 28505264 allocated.
dbug: NativeHandle[0]
      UV_TCP 29243472 allocated.
dbug: PoolOptions[0]
      allocator.numHeapArenas: 16
dbug: PoolOptions[0]
      allocator.pageSize: 8192
dbug: PoolOptions[0]
      allocator.maxOrder: 11
dbug: PoolOptions[0]
      allocator.chunkSize: 16777216
dbug: PoolOptions[0]
      allocator.tinyCacheSize: 512
dbug: PoolOptions[0]
      allocator.smallCacheSize: 256
dbug: PoolOptions[0]
      allocator.normalCacheSize: 64
dbug: PoolOptions[0]
      allocator.maxCachedBufferCapacity: 32768
dbug: PoolOptions[0]
      allocator.cacheTrimInterval: 8192
dbug: ScheduleHandle[0]
      Stream UV_TCP 29243472 listening, backlog = 128
NetUv.Server+ServerType:Echo server started on 127.0.0.1:9988.
dbug: NativeHandle[0]
      UV_TCP 30757552 allocated.
trce: ScheduleHandle[0]
      UV_TCP 30757552 Read started.
dbug: ScheduleHandle[0]
      UV_TCP 29243472 client 30757552 accepted
Tcp:Echo server client connection accepted
dbug: ReceiveBufferSizeEstimate[0]
      ReceiveBufferSizeEstimate allocate, estimated size = 1024
dbug: ResourceLeakDetector[0]
      leakDetection.level: disabled
dbug: ResourceLeakDetector[0]
      leakDetection.maxRecords: 4
trce: Pipeline[0]
      Pipeline receive buffer allocated size = 1024
dbug: ScheduleHandle[0]
      UV_TCP 30757552 read completed.
dbug: NativeHandle[0]
      Disposing 30757552 (Finalizer False)
dbug: NativeHandle[0]
      UV_TCP 30757552 closed, releasing resources pending.
trce: ScheduleHandle[0]
      UV_TCP 30757552 Read stopped.
trce: NativeHandle[0]
      UV_TCP 30757552 GCHandle released.
dbug: NativeHandle[0]
      UV_TCP 30757552 memory and GCHandle released.

Client Log

[nix-shell:~/src/projects/iris/netuv/build]$ mono NetUv.Client.exe
info: NativeHandle[0]
      Loop 44839136 allocated.
dbug: NativeHandle[0]
      UV_TCP 45750368 allocated.
dbug: PoolOptions[0]
      allocator.numHeapArenas: 16
dbug: PoolOptions[0]
      allocator.pageSize: 8192
dbug: PoolOptions[0]
      allocator.maxOrder: 11
dbug: PoolOptions[0]
      allocator.chunkSize: 16777216
dbug: PoolOptions[0]
      allocator.tinyCacheSize: 512
dbug: PoolOptions[0]
      allocator.smallCacheSize: 256
dbug: PoolOptions[0]
      allocator.normalCacheSize: 64
dbug: PoolOptions[0]
      allocator.maxCachedBufferCapacity: 32768
dbug: PoolOptions[0]
      allocator.cacheTrimInterval: 8192
dbug: NativeHandle[0]
      UV_CONNECT 46521888 allocated.
Tcp:Echo client loop starting.
trce: ScheduleHandle[0]
      UV_TCP 45750368 Read started.
Tcp:Echo client connected, request write message.
dbug: ResourceLeakDetector[0]
      leakDetection.level: disabled
dbug: ResourceLeakDetector[0]
      leakDetection.maxRecords: 4
dbug: NativeHandle[0]
      UV_WRITE 48422672 allocated.
dbug: NativeHandle[0]
      Disposing 46521888 (Finalizer False)
dbug: NativeHandle[0]
      UV_CONNECT 46521888 GCHandle released.
dbug: NativeHandle[0]
      UV_CONNECT 46521888 memory released.
Echo client write error NetUV.Core.Native.OperationException: EFAULT : bad address in system call argument
dbug: NativeHandle[0]
      Disposing 45750368 (Finalizer False)
dbug: NativeHandle[0]
      UV_TCP 45750368 closed, releasing resources pending.
trce: NativeHandle[0]
      UV_TCP 45750368 GCHandle released.
dbug: NativeHandle[0]
      UV_TCP 45750368 memory and GCHandle released.
result: 0
Tcp:Echo client loop dropped out
dbug: NativeHandle[0]
      Disposing 44839136 (Finalizer False)
info: NativeHandle[0]
      Loop 44839136 walk all handles completed.
info: NativeHandle[0]
      Loop 44839136 closed.
info: NativeHandle[0]
      Loop 44839136 GCHandle released.
info: NativeHandle[0]
      Loop 44839136 memory released.

Hi Jonny

I've recently replaced my own implementation of a managed wrapper around LibUv with NetUV and got a question about the ReadableBuffer.ReadString overload that accepts a separator. I'm using this to split incoming character data into lines:

https://github.com/coinfoundry/mining-core/blob/master/src/MiningCore/JsonRpc/JsonRpcConnection.cs#L46

What I'm not sure about (without digging deeper into NetUv) is what happens if ReadString is called when a newline has not yet been received (meaning client is somehow sending data in chunks).

NetUV 0.1.125.0
Use NetUV as tcp server ,it happed when read and write lot of "packet".

System.NullReferenceException
HResult=0x80004003
Message=Object reference not set to an instance of an object.
Source=NetUV.Core
StackTrace:

NetUV.Core.dll!NetUV.Core.Common.ThreadLocalPool.Handle.Release<NetUV.Core.Requests.WriteRequest>(NetUV.Core.Requests.WriteRequest value = {NetUV.Core.Requests.WriteRequest}) line 34
NetUV.Core.dll!NetUV.Core.Requests.WriteRequest.Release() line 22
NetUV.Core.dll!NetUV.Core.Requests.WriteBufferRequest.OnWriteCallback(int status = 0) line 78
NetUV.Core.dll!NetUV.Core.Requests.WriteBufferRequest.OnWriteCallback(System.IntPtr handle = {System.IntPtr}, int status = 0) line 85
[native to manage]
[manage to native]
NetUV.Core.dll!NetUV.Core.Native.NativeMethods.RunLoop(System.IntPtr handle = {System.IntPtr}, NetUV.Core.Native.uv_run_mode mode = UV_RUN_DEFAULT) line 109
NetUV.Core.dll!NetUV.Core.Handles.LoopContext.Run(NetUV.Core.Native.uv_run_mode mode = UV_RUN_DEFAULT) line 78
NetUV.Core.dll!NetUV.Core.Handles.Loop.RunDefault() line 32
UvTcpPacketServer.dll!ServerEngine.UvTcpPacketServer<RoomServer.ToClient.ToClientSession>.Start.AnonymousMethod__0() line 26
System.Private.CoreLib.ni.dll!System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state)

My loop is running in a standalone thread,and write requests are other thread.

this project has tcp and http, can support websocket?
thanks

sir,you should update your prj to v2.1

我是个新手,我会使用C++版,但是项目需求需要移植到unity移动端,我想使用C#版,但是编译通过运行时报错：
“Echo client error System.TypeInitializationException: The type initializer for 'NetUV.Core.Native.NativeMethods' threw an exception. ---> System.DllNotFoundException: Unable to load DLL 'libuv' or one of its dependencies: The specified module could not be found.”; 是不是缺少了什么?

Port https://github.com/libuv/libuv/blob/v1.x/test/test-fs-poll.c

Dear ~

I got the exception IndexOutOfRangeException when received data, In my code only receive no send any data.

So, how to fix the problem?

System.IndexOutOfRangeException: index:0, length:256 expected range >= 0 and < 0.
於 NetUV.Core.Buffers.ByteBuffer.Validate(Int32 index, Int32 length)
於 NetUV.Core.Buffers.ReadableBuffer.ReadString(Int32 length, Encoding encoding)

how can i use multi threads to improve the throughput?

Same scene in #57 ,but sometimes the excetion is AccessViolationException.
System.AccessViolationException
HResult=0x80004003
Message=Attempted to read or write protected memory. This is often an indication that other memory is corrupt.
StackTrace:

[manage to native]
NetUV.Core.dll!NetUV.Core.Native.NativeMethods.WriteStream(System.IntPtr requestHandle = {System.IntPtr}, System.IntPtr streamHandle = {System.IntPtr}, ref NetUV.Core.Native.uv_buf_t[] bufs = {NetUV.Core.Native.uv_buf_t[1]}) line 122
NetUV.Core.dll!NetUV.Core.Handles.StreamHandle.WriteStream(NetUV.Core.Requests.WriteRequest request = {NetUV.Core.Requests.WriteRequest}) line 215
NetUV.Core.dll!NetUV.Core.Handles.Pipeline.QueueWrite(NetUV.Core.Native.BufferRef bufferRef = {System.Action<NetUV.Core.Handles.StreamHandle, System.Exception>}, System.Action<NetUV.Core.Handles.StreamHandle, System.Exception> completion = {NetUV.Core.Native.BufferRef}) line 135
NetUV.Core.dll!NetUV.Core.Handles.StreamHandle.QueueWriteStream(byte[] array = {byte[3518]}, int offset = 0, int count = 3518, System.Action<NetUV.Core.Handles.StreamHandle, System.Exception> completion = {System.Action<NetUV.Core.Handles.StreamHandle, System.Exception>}) line 182
UvTcpPacketServer.dll!ServerEngine.UvTcpPacketSession<RoomServer.ToClient.ToClientServer>.Send(byte[] bytes = {byte[3514]}, int offset = 0, int count = 3514) line 49
CommandServer.dll!ServerEngine.CommandSession<RoomServer.ToClient.ToClientServer>.Send(CommandModel.ICommand command = {RoomServer.Commands.ViewUpdate}) line 24
RoomServer.dll!RoomServer.Commands.ViewUpdate.OnExecute(RoomServer.ToClient.ToClientSession session = {RoomServer.ToClient.ToClientSession}) line 26
CommandServer.dll!ServerEngine.CommandBase<RoomServer.ToClient.ToClientSession>.OnExecute(object session = {RoomServer.ToClient.ToClientSession}) line 37
CommandServer.dll!ServerEngine.CommandServer<RoomServer.ToClient.ToClientSession>..ctor.AnonymousMethod__7_0() line 45
System.Private.CoreLib.ni.dll!System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state)

RunLoop is called later than Schedule(sometimes maybe), and then will cause a NullReferenceException

Is it valid to Dispose an Async Handle within its callback like this?:

var disposer = loop.CreateAsync((handle) =>
{
    tcp.Dispose();

    handle.Dispose();
});

The purpose of this example is to schedule the disposal of another loop-bound resource on the loop thread.

Very impress project,i am looking for a Libuv wapper for .netcore long time ago.
Thank you !
Do you plan to add more detail document?
please add more example, like chart room ...
i found when Tcp IStreamChannel WriteAsync message length great than 1024 will caurse exception ,but i don't known how to fix it ,please help!.

I'm getting repeated reports of Segfaults on Linux which all seem to be related to situations where the server actively terminates client connections.

Inspecting the core dump using gdb revealed this crash location: https://github.com/libuv/libuv/blob/v0.10/src/unix/stream.c#L867

Due to optimization being turned on it might be the assert three lines below . Another bug report points to uv__write_callbacks.

Flow of events

1. The client's subscription to the stream of incoming messages gets disposed. This may happen from any TaskPool-Thread
2. Therefore the actual disposal is marshalled to Event-Loop's thread using an AsyncHandle (same file, Line #172
3. Line 174 is executing on the Event-Loop thread and the Dispose on Line #174 invokes the actual connection shutdown
4. There's usually nothing to report beyond this point 😢

Exception detail

System.IndexOutOfRangeException occurred
  HResult=0x80131508
  Message=ReaderIndex: 826 (expected: 0 <= readerIndex <= writerIndex(413)
  Source=<Cannot evaluate the exception source>
  StackTrace:
   at NetUV.Core.Buffers.AbstractArrayBuffer`1.SetReaderIndex(Int32 readerIndex)
   at NetUV.Core.Buffers.ReadableBuffer.ReadBytes(Byte[] destination, Int32 length)
   at Shark.Internal.UvClient.OnAccept(Tcp tcp, ReadableBuffer readableBuffer) in C:\Users\XXX\Documents\Visual Studio 2017\Projects\Shark\Shark\Internal\UvClient.cs:line 103
   at NetUV.Core.Handles.Tcp.<>c__DisplayClass8_0.<OnRead>b__0(StreamHandle stream, ReadableBuffer buffer)
   at NetUV.Core.Channels.StreamConsumer`1.Consume(T stream, IStreamReadCompletion readCompletion)