Code Monkey home page Code Monkey logo

hp-socket's Introduction

HP-Socket

High Performance Network Framework

Description

  • Server Based on IOCP/EPOLL communication model, combined with technology of memory pool, private heap etc., efficient memory management is implemented to support large scale and high concurrent communication scenarios.
  • Agent The Agent component is essentially a Multi-Client component that uses the same technical architecture as the Server component. An Agent component object can create and efficiently handle large-scale Socket connections at the same time.
  • Client Based on Event-Select/POLL communication model, each component object creates a communication thread and manages a Socket connection. Client components are suitable for small-scale client scenarios.

Document

  • HP-Socket Development Guide [pdf]
  • HP-Socket Class Diagram [uml]
  • HP-Socket Class Diagram [jpg]
  • HP-Socket SSL Class Diagram [jpg]
  • HP-Socket HTTP Class Diagram [jpg]

Workflow

  1. Create listener object
  2. Create component object (and binding with listener object)
  3. Start component object
  4. Connect to dest host (for Agent Component only)
  5. process network events (OnConnect/OnReceive/OnClose etc.)
  6. Stop component object (optional: component object will be stopped before destroy in step 7)
  7. Destroy component object
  8. Destroy listener object

Agent Workflow

Example

  • C++ Example
#include <hpsocket/HPSocket.h>

/* Listener Class */
class CListenerImpl : public CTcpPullServerListener
{

public:
	// 5. process network events
	virtual EnHandleResult OnPrepareListen(ITcpServer* pSender, SOCKET soListen);
	virtual EnHandleResult OnAccept(ITcpServer* pSender, CONNID dwConnID, UINT_PTR soClient);
	virtual EnHandleResult OnHandShake(ITcpServer* pSender, CONNID dwConnID);
	virtual EnHandleResult OnReceive(ITcpServer* pSender, CONNID dwConnID, int iLength);
	virtual EnHandleResult OnSend(ITcpServer* pSender, CONNID dwConnID, const BYTE* pData, int iLength);
	virtual EnHandleResult OnClose(ITcpServer* pSender, CONNID dwConnID, EnSocketOperation enOperation, int iErrorCode);
	virtual EnHandleResult OnShutdown(ITcpServer* pSender);
};

int main(int argc, char* const argv[])
{
	// 1. Create listener object
	CListenerImpl s_listener;
	// 2. Create component object (and binding with listener object)
	CTcpPullServerPtr s_pserver(&s_listener);
	
	// 3. Start component object
	if(!s_pserver->Start("0.0.0.0", 5555))
		exit(1);
	
	/* wait for exit */
	// ... ... 
	
	// 6. (optional) Stop component object
	s_pserver->Stop();

	return 0;
	
	// 7. Destroy component object automatically
	// 8. Destroy listener object automatically
}
  • C Example
#include <hpsocket/HPSocket4C.h>

// 5. process network events
EnHandleResult __HP_CALL OnConnect(HP_Agent pSender, HP_CONNID dwConnID);
EnHandleResult __HP_CALL OnReceive(HP_Agent pSender, HP_CONNID dwConnID, int iLength);
EnHandleResult __HP_CALL OnSend(HP_Agent pSender, HP_CONNID dwConnID, const BYTE* pData, int iLength);
EnHandleResult __HP_CALL OnClose(HP_Agent pSender, HP_CONNID dwConnID, En_HP_SocketOperation enOperation, int iErrorCode);
EnHandleResult __HP_CALL OnShutdown(HP_Agent pSender);

int main(int argc, char* const argv[])
{
	HP_TcpPullAgentListener s_listener;
	HP_TcpPullAgent s_agent;

	// 1. Create listener object
	s_listener = ::Create_HP_TcpPullAgentListener();
	// 2. Create component object (and binding with listener object)
	s_agent    = ::Create_HP_TcpPullAgent(s_listener);
	
	/* Set listener callbacks */
	::HP_Set_FN_Agent_OnConnect(s_listener, OnConnect);
	::HP_Set_FN_Agent_OnSend(s_listener, OnSend);
	::HP_Set_FN_Agent_OnPullReceive(s_listener, OnReceive);
	::HP_Set_FN_Agent_OnClose(s_listener, OnClose);
	::HP_Set_FN_Agent_OnShutdown(s_listener, OnShutdown);
	
	// 3. Start component object
	if(!::HP_Agent_Start(s_agent, "0.0.0.0", TRUE))
		exit(1);
	
	// 4. Connect to dest host
	::HP_Agent_Connect(s_agent, REMOTE_HOST_1, REMOTE_PORT_1, nullptr);
	::HP_Agent_Connect(s_agent, REMOTE_HOST_2, REMOTE_PORT_2, nullptr);
	::HP_Agent_Connect(s_agent, REMOTE_HOST_3, REMOTE_PORT_3, nullptr);
	
	/* wait for exit */
	// ... ... 
	
	// 6. (optional) Stop component object
	::HP_Agent_Stop(s_agent);

	// 7. Destroy component object
	::Destroy_HP_TcpPullAgent(s_agent);
	// 8. Destroy listener object
	::Destroy_HP_TcpPullAgentListener(s_listener);
	
	return 0;
}

Component List

  • Basic Components

Basic Component

  • SSL Components

SSL Component

  • HTTP Components

HTTP COmponent

Reference Projects

Extension Projects

Technical Exchange Groups

hp-socket's People

Contributors

ldcsaa avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

hp-socket's Issues

关于心跳的问题

网络通讯,有时为了可靠,就像上次的openSSL出现漏洞的心跳包一样
我们往往会在程序中加入心跳 我不知到您是否考虑了盼回复

如何关闭TCP内置的心跳检测机制?

我自己实现了心跳功能,但是如何关闭默认的心跳?我在doc里看了UDP的关闭方法,但是tcp的没有? 是调用SetKeepAliveTime() / SetKeepAliveInterval() 吗?

关于心跳包的问题

使用版本:5.2.1
TCP心跳包检测机制:
超时间计算公式: KeepAliveTime + (KeepAliveInterval * N)
其中 N为固定值: WinXP以下系统 N=5 ;Win7 以上系统 N=10

使用的相关函数说明:
/* 设置正常心跳包间隔(毫秒,0 则不发送心跳包,默认:30 * 1000) /
virtual void SetKeepAliveTime (DWORD dwKeepAliveTime) = 0;
/
设置异常心跳包间隔(毫秒,0 不发送心跳包,,默认:10 * 1000,如果超过若干次 [默认:WinXP 5 次, Win7 10 次] 检测不到心跳确认包则认为已断线) */
virtual void SetKeepAliveInterval (DWORD dwKeepAliveInterval) = 0;

通过尝试 这两个函数的参数值只有设置成1000以上时才能Start TcpSocket Server成功, 这样的话至少10秒左右才能检测到断线, 这对于客户端连接数量不高,实时性要求高的场景检测断线的时间就有点长了,请问是否可以改为参数可灵活配置, 并且可以设置检测次数?

顺便再提一个 SendLocalFile 有文件大小的限制, 如果发送大文件的话,可以用什么方式,有没有好的建议

tcppack的maxpacksize问题

maxpacksize最大可以设置多大?
我发送几十K的小文件没问题,几百K的就连receive事件都不会触发了。(默认没有设置maxpacksize)
当我设置maxpacksize=0x1000之后依然无效,这个0x1000的单位是K吗?
求解答

issue about how to wrap class

to respect the culture of github and honor this project , i would like to express myself with english .

HP-socket is a wonderful lib , which could help to solve lots of problems in my work . like the heart-beat mode , efficiency of communication .
as a C++ programmer , i had been trapped in C++ for many years in the network communication . when i met issues , i tried to find a proper libs to solve my problems . when the solution seemed not work very well . failed fellow . . (as lots of my work is dealing with low computer communication which is base on the TCP/UDP byte string exchange) .
as there were no proper libs which could meet my requirement , so i became to learn how to wrap the class for myself , so i engaged the technique design pattern , refactory , FP ect . it seemed work for me , as it was promoted my work efficiency .
however when i tried to take this technique on the network , i was frustrated a lot , there were so many kind of condition that i had never take into consideration . the exceptions and issues were out of my control . i felt so depressed .
when i noticed this project , i found you were a great software developer . when you met your problems , you solved . that was no surprise . so did everyone
but what attracted me was that you could abstracted the common function and wrapped them gracefully and efficient . at the same time , you distributed your source code to the public , may you work could help more people . that was wonderful and respectful .
i would like to do that like you did . i think programming is to help people to solve problems . but i was confused .
could you share some tips or suggestions about: how to wrap class more efficient and gracefu ??
(wrap class efficient and gracefully , especially in the aspect of wrapping network class )
on the way wrapping API , i knew i was almost there , but i found i was trapped . could you do me a favor ?
chinese would be ok for me ,
appropriate
contact : [email protected]

use static lib on linux

I use libhpsocket.a , but error is " libhpsocket.a(background_thread.pic.o) : error : In function je_background_thread_ctl_init': 1>background_thread.c : error : (.text+0x3483): undefined reference to dlsym'"

(TBufferObj*)m_phBuffer.Alloc(sizeof(TBufferObj) + m_dwSocketBufferSize, HEAP_ZERO_MEMORY); 大量并发情况下,分配内存失败导致应用崩溃

在一个进程中开启了2个TcpServer,用于做代理转发。

一个端口为5678,设置为可接受10个连接
一个端口为6789,设置为可接受20000个连接

DEFAULT_FREE_BUFFEROBJ_POOL,和 DEFAULT_FREE_BUFFEROBJ_HOLD 使用的默认值

将6789端口接收到的所有包转发连接5678端口的所有服务器上。

客户端压力反复断开和连接,每次连接成功后,会发送大概500字节的包后,自动断开,然互重连。

6789端口的连接一直保持在3000以内,大概5~10分钟,就会发生 私有堆 无法分配的情况,即:
(TBufferObj*)m_phBuffer.Alloc(sizeof(TBufferObj) + m_dwSocketBufferSize, HEAP_ZERO_MEMORY) 返回为nullptr

如果6789端口只是接收,但并不通过 5678 转发,则不会出现此问题

TcpClient连上服务器后,收不到数据,发数据对方也收不到。

使用C#版本的封装,使用TcpClient连上服务器后,收到服务器链接成功事件,开始发送数据,显示发送成功,但对方未收到,对方给本地发数据,也收不到。隔一定时间后,收到一大包数据。 测试环境: 联想E570 win7 64位和华为V3 windows server 2008R2,物理网卡虚拟网卡都测试过,均存在这个问题。另外还有一台联想台式机Win7 64位测试没出现这个问题。

Agent场景中,connID赋值返回应该在HP_Set_FN_Agent_OnConnect事件之前

Agent场景中,connID赋值返回应该在Sockect_On_Agent_Accept事件之前

在HP_Agent_Connect连接方法中,传参数后,ConnID 连接ID 参数回调返回比 HP_Set_FN_Agent_OnConnect 连接回调函数慢,导致 agent发起连接的ConnId无效 无法再OnConnect方法中给与状态改变

建议:先在Hp_Agent_Connect 中先返回ConnId 然后在执行OnConnect事件回调

没找到java demo

在示例代码里面没有java的
我想用到 Android
请问是否可以呢?

c#封装SDK_OnSend SDK_OnReceive 崩溃问题

在TcpServer.cs中SDK_OnSend和SDK_OnReceive中,如果参数pData为null会导致 Marshal.Copy(pData, bytes, 0, length);崩溃,我添加if (length > 0)后能解决这个问题

我只用到了TcpServer.cs,其他封装代码中也应该会出现这个问题
protected HandleResult SDK_OnSend(IntPtr pSender, IntPtr connId, IntPtr pData, int length)
{
if (OnSend != null)
{
byte[] bytes = new byte[length];
if (length > 0)
{
Marshal.Copy(pData, bytes, 0, length);
}
return OnSend(connId, bytes);
}
return HandleResult.Ignore;
}

    protected HandleResult SDK_OnReceive(IntPtr pSender, IntPtr connId, IntPtr pData, int length)
    {
        if (OnPointerDataReceive != null)
        {
            return OnPointerDataReceive(connId, pData, length);
        }
        else if (OnReceive != null)
        {
            
            byte[] bytes = new byte[length];
            if (length > 0)
            {
                Marshal.Copy(pData, bytes, 0, length);
            }   
            return OnReceive(connId, bytes);
        }
        return HandleResult.Ignore;
    }

are there plan to develop static lib for hp-socket

HP-SOCKET is a awesome network communication library .
but it only provide two kind of methods to call the package :

  • *dll calling *
  • source calling

but in some of my project , especially some simple project , i prefer one file could solve all the problem instead of carrying a dll with the exe . there is only one way to do that , using source code to do that . which seems not convenient and unsafe as a lib ( prevent from modification by accident )

at the same time , some of former project used the static lib ,which could not compatible with the HP-SOCKET.

my question is : _are there any plan to develop a static lib for _hp-socket* * ?

c#demo中SSLHttpServer.cs第二次开始服务出错

SSLHttpServer创建时调用了一次Interlocked.Increment(ref ObjectReferer);
可是每次Uninitialize的时候都调用了 if (Interlocked.Decrement(ref ObjectReferer) == 0 && pServer != IntPtr.Zero)
而每次开始服务的时候都会调用Uninitialize,这样就导致Interlocked.Decrement(ref ObjectReferer) == 0返回false,从而不能调用SSLSdk.HP_SSLServer_CleanupSSLContext(pServer);
再次开始服务时候Initialize服务返回false

TCP Tunnel

hi..
is it possible create ssl tunnel with HP-Socket?
thanks

拜读了一下源码,提一点建议

简单做了个echo,用了5台PC跑了下,每个PC开了10000的连接,由10线程管理,总共连接数5W

客户机每隔100毫秒建立一个连接,连接全部建立成功后,开始收发包,发送1个Hello World给服务端,服务端回应,收到回应后继续发送。

被测机配置如下:
处理器: Intel(R) Core(TM) i7-3770K CPU @ 3.50GHz (4核心 + 超线程技术)
内存: 32768MB RAM

连接 5W,可以全部连上,但是,机器如果运行很久没有重启,有连不上的情况
开始发包后,会不定期持续掉连接,最后稳定在3万5000左右。

服务端用的收发规则是 SP_PACK 和 RP_SERIAL

建议:代码中用了临界区和RW锁或Spin锁,如果能使用无锁方式(lock-free)相信性还能有很大的提升。
仅仅是建议,作者不喜勿怪。

C#版SSL Demo需要更新了

5.0.1调整SSL相关函数后,C#相关Demo没有更新。
同时希望HPSocket类库SSL编译能支持VS2012以上版本,目前用VS2017编译提示
fatal error C1083: 无法打开包括文件: “SDKDDKVer.h”: No such file or directory

Delphi 7 測試 issue

開發環境:

Win8.0 + Delphi7:
TcpServer / Client : 可以成功編譯與執行

Win10 + Delphi7:
TcpServer: 可以成功編譯與執行
Client: 可以成功編譯,執行 connect 時 show error:
Access violation at address 61B0E876 in module 'HPSocket4C.dll'. Read of address 00000004
<-- 很奇怪!

另外: Http sample:
Win8.0 + Delphi7:
Http sample: 編譯不過 error 在
function OnRequestLine(pSender: HP_Http; ...
Request[0].name:= PChar('Content-Type');
Request[0].value:= PChar('text/html');
Request[1].name:= PChar('Connection');
Request[1].value:= PChar('Keep-Alive');

HPTypeDef.pas 原始碼:
{/*****名称:Name/Value 结构体 描述:字符串名值对结构体 **/}
TNVPair = array of array of PChar; //好像沒宣告 好像沒有宣告型態
implementation
end.

Win10 + Delphi7:
Http sample: 未測

TCP连接状态

建议增加TCP连接状态变化的事件。在C#使用TCP Client时,调用OnClose事件时,连接状态为ServiceState.Stoping,没有更新ServiceState状态的方法

断开网线后客户端一直在SS_STOPING

两台电脑通过无线局域网相连,拔掉网线之后,使用 GetState 获得的一直都是 STOPPING 的状态,无法进入STOPPED的状态。 测试环境 Linux 客户端 ubuntu Linux ,服务器端 arch Linux 。 PACK模式。原因大概是我再onclose 回调函数中去循环检查。

Client的GetConnectionID函数返回的不是实际id

调用client的GetConnectionID函数返回1,而不是实际在server端分配的id。我现在的做法是连接后还要从server端发送client实际的id给client。建议修复GetConnectionID返回错误id问题

RingBuffer.h 函数问题

LZ您好!
RingBuffer.h 中下面这个函数EmplaceIndex,主要作用干啥的呢?当客户端连接到服务器这边的时候,走到m_indexes.emplace(dwIndex);一直报错。
void EmplaceIndex(index_type dwIndex)
{
CWriteLock locallock(m_cs);
// m_indexes.emplace(dwIndex);
}

HP-SOCKET项目结构建议

问题

感觉目前的项目文件结构不甚方便, 建议对项目文件的结构进行一定的调整

建议点

  1. 作为库, include , libs , bin, demo等放置在根目录下

    这方方便快速调用. 下载更新库的时候, 对项目的改动也少. 如果需要自己生成静态库, 也可以只需要把生成的lib放置到目录下即可. 可以最大限度的减小对源项目的改动

  2. 建议提供可编译生成静态库的项目. 可减少软件编译时间, 同时利于工具型软件发布(无需携带dll)

C# OnAccept()事件参数pClient代表什么?

OnAccept事件委托定义了两个参数:
public delegate HandleResult OnAcceptEventHandler(IntPtr connId, IntPtr pClient);
想问问第二个参数pClient的含义?
看了C++版本代码,回调出来的就是一个原始SOCKET,C#没有源码,所以不清楚具体含义。

如何创建cer证书?

老哥,能否提供一下测试环境中SSL证书创建的步骤或者命令及工具?
谢谢~

不兼容.net core

win10 x64
.net core 1.1.2

测试表明,能添加引用,但报错需要引用 mscorlib

使用C#版WebSocketServer遇到一个问题

我在使用C#版HPSocket的WebSocketServer时遇到一个问题,如果WebSocket客户端连接服务端后,向服务端发送过消息,服务端就可以正常向客户端回复消息,但是当WebSocket客户端连接上服务端后,没有向服务端发送过消息之前,服务端向客户端发送消息就会出现异常,具体调试及复现过程已经上传至附件中,还请大神在百忙中解决一下,万分感谢!
HPSocketCS调试.zip

[Bug] v5.2.1中TCPPackClient连接成功SSLPackServer

HPSocket版本
v5.2.1
Bug描述
无论有无证书,TCPPackClient居然可以成功连接上SSLPackServer;理论上握手这一步都不能完成。当client给server发消息后,server才能发生错误并中断连接。
潜在影响
当用户从TCPPackClient迁移到SSLPackClient时,一些老的TCPPackClient仍然可以成功连接上SSLPackServer。
Bug截图
hpsocket_bug

编写httpserver遇到的奇怪问题

这个是我定义的监听类,工程使用的是静态库,加入了static宏,编译成功

class CHFServer :public CHttpServerListener
{
public:
	CHFServer(string strname){m_strName = strname;};
public:
	string m_strName;
private:
	virtual EnHandleResult OnClose(ITcpServer* pSender, CONNID dwConnID, EnSocketOperation enOperation, int iErrorCode);
	virtual EnHttpParseResult OnMessageBegin(IHttpServer* pSender, CONNID dwConnID);
	virtual EnHttpParseResult OnHeadersComplete(IHttpServer* pSender, CONNID dwConnID);
	virtual EnHttpParseResult OnBody(IHttpServer* pSender, CONNID dwConnID, const BYTE* pData, int iLength);
	virtual EnHttpParseResult OnMessageComplete(IHttpServer* pSender, CONNID dwConnID);
	virtual EnHttpParseResult OnParseError(IHttpServer* pSender, CONNID dwConnID, int iErrorCode, LPCSTR lpszErrorDesc);
public:
	bool ParseRangeByte(const char* strRange,range_byte* rb);
};
int _tmain(int argc, _TCHAR* argv[])
{
	CHFServer httpListener("http");
	CHttpServer server(&httpListener);
	
	if (!server.Start(DEF_ADDRESS,HTTP_PORT))
	{
		EnSocketError err = server.GetLastError();  //这里报错 SE_INVALID_PARAM
		return 0;
	}

	getchar();
	return 0;
}

请问哪里出错了,我是参考那个TestEcho-Http示例,没找到问题原因

建议:TcpPullServer增加一个"查找byte序列"的HP_TcpPullServer_Find()方法

目前只能每次用 HP_TcpPullServer_Peek 先取出来自己在临时的buffer中查找分隔符,多了很多无效的内存拷贝。

建议提供一个可以直接在内部缓冲区查找特定BYTE序列的函数,这样就不需要全部Peek出来了。

int HP_TcpPullServer_Find(HP_TcpPullServer pServer, HP_CONNID dwConnID, BYTE* pFind, int iLength);

pFind是需要匹配的序列, iLength 是pFind 的字节数(必须>=1)

返回值:如果 >0 表示匹配到了,可以Fetch的字节数(包含末尾的pFind),如果 <=0 则未找到

HP-Socket occur 10053 or crash

i use hp-socket for tcp server/client develop. at 1000~ clients, always will occur "server 10053 receive onclose event".
how can i debug or know why?
and sometime the exe will crash becasue of the module signature "HPSocket4C_U.dll", how to know the problem?

i download the source code, and build with debug version, got dll/pdb. hope next crash can do something.

我能做些什么来帮助定位10053问题,或者模块崩溃问题么?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.