Code Monkey home page Code Monkey logo

webserver's Introduction

WebServer

用C++实现的高性能WEB服务器,经过webbenchh压力测试可以实现上万的QPS

功能

  • 利用IO复用技术Epoll与线程池实现多线程的Reactor高并发模型;

  • 利用正则与状态机解析HTTP请求报文,实现处理静态资源的请求;

  • 利用标准库容器封装char,实现自动增长的缓冲区;

  • 基于小根堆实现的定时器,关闭超时的非活动连接;

  • 利用单例模式与阻塞队列实现异步的日志系统,记录服务器运行状态;

  • 利用RAII机制实现了数据库连接池,减少数据库连接建立与关闭的开销,同时实现了用户注册登录功能。

  • 增加logsys,threadpool测试单元(todo: timer, sqlconnpool, httprequest, httpresponse)

环境要求

  • Linux
  • C++14
  • MySql

目录树

.
├── code           源代码
│   ├── buffer
│   ├── config
│   ├── http
│   ├── log
│   ├── timer
│   ├── pool
│   ├── server
│   └── main.cpp
├── test           单元测试
│   ├── Makefile
│   └── test.cpp
├── resources      静态资源
│   ├── index.html
│   ├── image
│   ├── video
│   ├── js
│   └── css
├── bin            可执行文件
│   └── server
├── log            日志文件
├── webbench-1.5   压力测试
├── build          
│   └── Makefile
├── Makefile
├── LICENSE
└── readme.md

项目启动

需要先配置好对应的数据库

// 建立yourdb库
create database yourdb;

// 创建user表
USE yourdb;
CREATE TABLE user(
    username char(50) NULL,
    password char(50) NULL
)ENGINE=InnoDB;

// 添加数据
INSERT INTO user(username, password) VALUES('name', 'password');
make
./bin/server

单元测试

cd test
make
./test

压力测试

image-webbench

./webbench-1.5/webbench -c 100 -t 10 http://ip:port/
./webbench-1.5/webbench -c 1000 -t 10 http://ip:port/
./webbench-1.5/webbench -c 5000 -t 10 http://ip:port/
./webbench-1.5/webbench -c 10000 -t 10 http://ip:port/
  • 测试环境: Ubuntu:19.10 cpu:i5-8400 内存:8G
  • QPS 10000+

TODO

  • config配置
  • 完善单元测试
  • 实现循环缓冲区

致谢

Linux高性能服务器编程,游双著.

@qinguoyi

webserver's People

Contributors

libinyl avatar mamil avatar markparticle avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

webserver's Issues

构造ThreadPool时如果不设定参数会报错

报错信息:
include/c++/11/ext/new_allocator.h:162:11: error: call of overloaded ‘ThreadPool()’ is ambiguous
162 | { ::new((void *)__p) _Up(std::forward<_Args>(_args)...); }
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from main.cpp:2:
threadpool.h:37:5: note: candidate: ‘constexpr ThreadPool::ThreadPool()’
37 | ThreadPool() = default;
| ^~~~~~~~~~
threadpool.h:17:14: note: candidate: ‘ThreadPool::ThreadPool(size_t)’
17 | explicit ThreadPool(size_t threadCount = 8): pool
(std::make_shared()) {

原因:
无参构造函数与有默认参数的构造函数冲突了

heaptimer出现段错误或一些其他异常

程序运行一段时间后,heaptimer会出现段错误

// TimerNode里的这个比较函数会出现Segmentation fault
bool operator<(const TimerNode& t) {
        return expires < t.expires;
}

或者是其他错误,比如说

server: ../code/timer/heaptimer.cpp:21: void HeapTimer::SwapNode_(size_t, size_t): Assertion `j >= 0 && j < heap_.size()' failed.

是否具备处理非完整请求的功能?

源代码httpConn.cpp中,如果解析失败,就会释放连接。解析失败可能是请求不完整,需要继续读sockfd。

    if(readBuff_.ReadableBytes() <= 0) {// 没有请求数据
        return false;
    }
    else if(request_.parse(readBuff_)) {    // 解析请求数据
        LOG_DEBUG("%s", request_.path().c_str());
        // 解析玩请求数据以后,初始化响应对象
        response_.Init(srcDir, request_.path(), request_.IsKeepAlive(), 200);
    } else {
        // 解析失败
        response_.Init(srcDir, request_.path(), false, 400);
    }

解析失败可能是请求不完整,需要继续读sockfd。
image

Possible use-after-free on the method HttpConn:Close??

Dear developers:
We find a use-after-free bug on this function HttpConn:Close. It may a false positive. Thank you for your confirmation.

void HttpConn::Close() {
    response_.UnmapFile();
    if(isClose_ == false){
        isClose_ = true; 
        userCount--;
        close(fd_); //free here 
        LOG_INFO("Client[%d](%s:%d) quit, UserCount:%d", fd_, GetIP(), GetPort(), (int)userCount); //use here
    }
}

这里ToWriteBytes>10240有什么意义

ssize_t HttpConn::write(int* saveErrno) {
ssize_t len = -1;
do {
len = writev(fd_, iov_, iovCnt_);
if(len <= 0) {
saveErrno = errno;
break;
}
if(iov_[0].iov_len + iov_[1].iov_len == 0) { break; } /
传输结束 /
else if(static_cast<size_t>(len) > iov_[0].iov_len) {
iov_[1].iov_base = (uint8_t
) iov_[1].iov_base + (len - iov_[0].iov_len);
iov_[1].iov_len -= (len - iov_[0].iov_len);
if(iov_[0].iov_len) {
writeBuff_.RetrieveAll();
iov_[0].iov_len = 0;
}
}
else {
iov_[0].iov_base = (uint8_t*)iov_[0].iov_base + len;
iov_[0].iov_len -= len;
writeBuff_.Retrieve(len);
}
} while(isET || ToWriteBytes() > 10240);
return len;
`}```

客户端主动断开连接时,会调用两次CloseConn_

第一次是epoll发出的EPOLLHUP event,第二次是心跳计时器,超时时回调调用的CloseConn_

虽然 HttpConn::Close() 有isClose,不会两次close(fd),但是会调用两次Epoller::DelFd(int fd),每次都会执行epoll_ctl(epollFd_, EPOLL_CTL_DEL, fd, &ev);

请问这是可行的吗?不会导致异常吗?

make编译问题

使用make指令编译之后,显示如下信息:
mkdir -p bin
cd build && make
make[1]: Entering directory /home/zhangke/cplusplus/WebServer/build' make[1]: Leaving directory /home/zhangke/cplusplus/WebServer/build'
疑惑的是为什么在bin目录下不能创建server文件?

参考资料

大佬除了游双的那本书以外,还有其他网课或者书籍推荐吗?

用webbench测试效率很低

我把服务器搭在一个腾讯云轻量级应用服务器上
一台二核的机子跑webbench

测试命令是 ./webbench -c 10000 -t 5 http://106.52.96.198:9999/

对于TinyWebServer

Speed=185652 pages/min, 349260 bytes/sec.
Requests: 15471 susceed, 0 failed.

对于当前项目

Speed=9600 pages/min, 815102 bytes/sec.
Requests: 800 susceed, 0 failed.

足足20倍的差距,这差距也大得忒离谱了
这个项目不是TinyWebServer的c++版本吗,
虽然有些改动,可按理不该差距这么大啊,看起来更像是出了bug

有没有大佬能给小弟解答一下呢?????

threadpool.h中锁同步机制似乎错误

    explicit ThreadPool(size_t threadCount = 8): pool_(std::make_shared<Pool>()) {
            assert(threadCount > 0);
            for(size_t i = 0; i < threadCount; i++) {
                std::thread([pool = pool_] {
                    std::unique_lock<std::mutex> locker(pool->mtx);
                    while(true) {
                        if(!pool->tasks.empty()) {
                           //这里缺少lock,即使是第一个来到临界区的也应该加锁
                            auto task = std::move(pool->tasks.front());
                            pool->tasks.pop();
                            locker.unlock();
                            task();
                            locker.lock();
                        } 
                        else if(pool->isClosed) break;
                        else pool->cond.wait(locker);
                    }
                }).detach();
            }
    }

这是什么问题呢?

配好数据库,编译运行成功之后运行,网页不能访问。日志出现这个:
微信截图_20210831232412

threadpool中使用valgrind发现存在内存泄露

猜测应该是使用thread.detach导致的,这种情况会如何影响程序
==1869393== LEAK SUMMARY:
==1869393== definitely lost: 0 bytes in 0 blocks
==1869393== indirectly lost: 0 bytes in 0 blocks
==1869393== possibly lost: 1,152 bytes in 4 blocks
==1869393== still reachable: 864 bytes in 7 blocks
==1869393== suppressed: 0 bytes in 0 blocks
==1869393== Reachable blocks (those to which a pointer was found) are not shown.
==1869393== To see them, rerun with: --leak-check=full --show-leak-kinds=all

c++11 版本中 code——buffer.cpp

第105行,您写的是Append(buff, len - writePos_); 不应该是 Append(buff, len - writable); 吗?
我这里有点疑惑,可以帮忙解惑一下吗

单元测试异步Log时死锁

主线程调用最后一次LOG_BASE后循环程序运行完毕,开始调用~Log():

if(writeThread_ && writeThread_->joinable()) {
        while(!deque_->empty()) {
            deque_->flush();
        };
        deque_->Close();
        writeThread_->join();    //    <----------------------------------
    }

若此时子线程正在写fp,则会出现死锁。

void Log::AsyncWrite_() {
    string str = "";
    while(deque_->pop(str)) {
        lock_guard<mutex> locker(mtx_);   //  <----------------------------------正在写fp
        fputs(str.c_str(), fp_);       //  <----------------------------------正在写fp
    }
}

分析如下:
在~Log()中,首先主线程判断deque_是否为空然后deque_->flush()唤醒deque_中的consumer,直到deque_为空。
假设deque_.size() == 1,子线程执行deque_->pop(str)后deque_.size()==0,然后执行fputs(str.c_str(), fp_)。
主线程则析构Log执行deque_->Close(), 并在writeThread_->join()处等待。然后子线程fputs执行完毕,重新判断循环条件deque_->pop(str),进入deque_->pop后,由于deque_.size()==0,于是停在 deque_->pop 中的 condConsumer_.wait(locker)处。而此时主线程正在 join 等待子线程结束,从而出现死锁。

测试如下:
多次运行test,直到出现死锁

ubt@TvT:~/WebServer/test$ ./test

另起终端:

ubt@TvT:~/MyWebServer/unit_test$ ps -aux | grep test
ubt      12836  0.4  0.0  91048  5604 pts/5    Sl+  20:37   0:04 ./test

使用gdb查看函数调用栈:

ubt@TvT:~/MyWebServer/unit_test$ gdb attach 12836 -q
attach: No such file or directory.
Attaching to process 12836
[New LWP 12837]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
__pthread_clockjoin_ex (threadid=140035354838784, thread_return=0x0, clockid=<optimized out>, abstime=<optimized out>, block=<optimized out>) at pthread_join_common.c:145
145     pthread_join_common.c: No such file or directory.
(gdb) thread apply all bt

Thread 2 (Thread 0x7f5c8594c700 (LWP 12837)):
#0  futex_wait_cancelable (private=<optimized out>, expected=0, futex_word=0x5606be2d33f4) at ../sysdeps/nptl/futex-internal.h:183
#1  __pthread_cond_wait_common (abstime=0x0, clockid=0, mutex=0x5606be2d3398, cond=0x5606be2d33c8) at pthread_cond_wait.c:508
#2  __pthread_cond_wait (cond=0x5606be2d33c8, mutex=0x5606be2d3398) at pthread_cond_wait.c:647
#3  0x00007f5c8612ee30 in std::condition_variable::wait(std::unique_lock<std::mutex>&) () from /lib/x86_64-linux-gnu/libstdc++.so.6
#4  0x00005606bd583b2f in BlockDeque<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >::pop (item="2022-09-12 20:37:44.005142 [error]: Test 222222222 99999 ", '=' <repeats 13 times>, " \n", this=0x5606be2d3340) at /usr/include/c++/9/bits/std_mutex.h:103
#5  Log::AsyncWrite_ (this=0x5606bd5b3480 <Log::Instance()::inst>) at ../code/log/log.cpp:180
#6  0x00007f5c86134de4 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
#7  0x00007f5c86028609 in start_thread (arg=<optimized out>) at pthread_create.c:477
#8  0x00007f5c85f4d133 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Thread 1 (Thread 0x7f5c8594d740 (LWP 12836)):
#0  __pthread_clockjoin_ex (threadid=140035354838784, thread_return=0x0, clockid=<optimized out>, abstime=<optimized out>, block=<optimized out>) at pthread_join_common.c:145
#1  0x00007f5c86135047 in std::thread::join() () from /lib/x86_64-linux-gnu/libstdc++.so.6
#2  0x00005606bd584135 in Log::~Log (this=0x5606bd5b3480 <Log::Instance()::inst>, __in_chrg=<optimized out>) at /usr/include/c++/9/bits/unique_ptr.h:360
#3  0x00007f5c85e748a7 in __run_exit_handlers (status=0, listp=0x7f5c8601a718 <__exit_funcs>, run_list_atexit=run_list_atexit@entry=true, run_dtors=run_dtors@entry=true) at exit.c:108
#4  0x00007f5c85e74a60 in __GI_exit (status=<optimized out>) at exit.c:139
#5  0x00007f5c85e5208a in __libc_start_main (main=0x5606bd583780 <main()>, argc=1, argv=0x7fff0f1bd7a8, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7fff0f1bd798) at ../csu/libc-start.c:342
#6  0x00005606bd58380e in _start () at ../code/log/log.cpp:193

可以看到 Thread 1 正在join等待子线程结束,而 Thread 2 锁在pop中的condition_variable::wait处。

原因可能是Log中访问deque_时没有任何保护措施(同步操作)。由于本人技术有限,初步解决方案是pop的while中先判断 is_close_ 然后再cond_wait,可以解决此问题。

定时器中成员函数GetNextTick中res的类型不应该是size_t吧?

int HeapTimer::GetNextTick() {
    tick();
    size_t res = -1;
    if(!heap_.empty()) {
        res = std::chrono::duration_cast<MS>(heap_.front().expires - Clock::now()).count();
        if(res < 0) { res = 0; }
    }
    return res;
}

无符号整形值赋值-1会溢出吧?
如果std::chrono::duration_cast(heap_.front().expires - Clock::now()).count()也为负数,也会溢出

ThreadPool中锁同步机制似乎有问题

    explicit ThreadPool(size_t threadCount = 8): pool_(std::make_shared<Pool>()) {
            assert(threadCount > 0);
            for(size_t i = 0; i < threadCount; i++) {
                std::thread([pool = pool_] {
                    std::unique_lock<std::mutex> locker(pool->mtx);
                    while(true) {
                        if(!pool->tasks.empty()) {
                            auto task = std::move(pool->tasks.front());
                            pool->tasks.pop();
                            locker.unlock();
                            task();
                            locker.lock();
                        } 
                        else if(pool->isClosed) break;
                        else pool->cond.wait(locker);
                    }
                }).detach();
            }
    }

上述程序中,需要先抢到锁,才能进入下一次while循环。
如果抢到锁后,线程isClosed了,就会结束while循环,但是依然占据着锁。
最后造成只有一个线程抢到锁关闭了自己,其他线程再也抢不到锁,无法关闭。。

A question about HTTP request parsing. 一个关于HTTP请求解析的问题。

事先声明:鄙人才疏学浅,可能会有错误的地方,如果有错误敬请指出,理性讨论,请多多包涵!

鄙人花了几天时间学习了下本仓库的代码,总体感觉比原作者的代码优雅且易读很多,代码使用了C++11特性并且风格统一,个人感觉学到了不少东西。

不过我对项目里HTTP请求解析的部分有疑惑,在《Linux高性能服务器编程》中,作者采用的是有限状态自动机边读取边解析的方式来应对TCP协议面向字节流的特点(也就是应用层读取一次获得的并不一定就是一个完整的HTTP请求),通过判断HTTP请求中的\r\n来确定当前的状态,在没有获取完整的HTTP请求之前,是不会开始返回响应的。

我仔细学习了一下本仓库的代码,发现在HTTP请求解析的部分,貌似假定了应用层一次(这里的一次是指检测到了一次EPOLLIN事件)就能读取到一个完整的HTTP请求,读取一次之后就立即开始解析紧接着返回响应,这样的问题是,倘若客户端发送的是一个有效的请求,但是因为某种原因,该请求并不在一次事件中被应用层一次接收到,这时候由于服务端假定了一次就能接收到完整的HTTP请求,所以服务端就会把收到的请求的部分内容当成完整的请求进行解析,从而返回一个错误的响应。

带着这个疑惑,我决定尝试调试一下看看实际结果和我的想法是否一致。

因此我尝试使用cgdb(对gdb的包装,TUI界面更好)对服务端的代码进行调试,首先要确定HTTP请求解析的位置。
第一个是WebServer.cc中的Start()函数:

1. void WebServer::Start() {
2.     int timeMS = -1;  /* epoll wait timeout == -1 无事件将阻塞 */
3.     if(!isClose_) { LOG_INFO("========== Server start =========="); }
4.     while(!isClose_) {
5.         if(timeoutMS_ > 0) {
6.             timeMS = timer_->GetNextTick();
7.         }
8.         int eventCnt = epoller_->Wait(timeMS);
9.         for(int i = 0; i < eventCnt; i++) {
10.             /* 处理事件 */
11.             int fd = epoller_->GetEventFd(i);
12.             uint32_t events = epoller_->GetEvents(i);
13.             if(fd == listenFd_) {
14.                 DealListen_();
15.             }
16.             else if(events & (EPOLLRDHUP | EPOLLHUP | EPOLLERR)) {
17.                 assert(users_.count(fd) > 0);
18.                 CloseConn_(&users_[fd]);
19.             }
20.             else if(events & EPOLLIN) {
21.                 assert(users_.count(fd) > 0);
22.                 DealRead_(&users_[fd]);                    <--------------------------------------
23.             }
24.             else if(events & EPOLLOUT) {
25.                 assert(users_.count(fd) > 0);
26.                 DealWrite_(&users_[fd]);
27.             } else {
28.                 LOG_ERROR("Unexpected event");
29.             }
30.         }
31.     }
32. }

首先客户端请求连接客户端,发起一个三次握手,服务端接受请求并将其加入到epoll的监听中,监听其EPOLLIN读事件。
紧接着客户端发送HTTP请求,服务端监听到EPOLLIN事件,也就是:

20.             else if(events & EPOLLIN) {
21.                 assert(users_.count(fd) > 0);
22.                 DealRead_(&users_[fd]);
23.             }

下一步就是调用DealRead_(&users_[fd]);,该函数同样在WebServer.cc中,这里主要完成的任务就是将WebServer类中的OnRead()包装成一个可调用对象,并传递thisclient,将这个对象放入线程池的队列中,由线程竞争获得并消费。

1. void WebServer::DealRead_(HttpConn* client) {
2.     assert(client);
3.     ExtentTime_(client);
4.     threadpool_->AddTask(std::bind(&WebServer::OnRead_, this, client));    <--------------------------------------
5. }

各线程获得任务的关键代码在threadpool.h中:

1.     explicit ThreadPool(size_t threadCount = 8): pool_(std::make_shared<Pool>()) {
2.             assert(threadCount > 0);
3.             for(size_t i = 0; i < threadCount; i++) {
4.                 std::thread([pool = pool_] {
5.                     std::unique_lock<std::mutex> locker(pool->mtx);
6.                     while(true) {
7.                         if(!pool->tasks.empty()) {
8.                             auto task = std::move(pool->tasks.front());
9.                             pool->tasks.pop();
10.                             locker.unlock();
11.                             task();                   <--------------------------------------
12.                             locker.lock();
13.                         } 
14.                         else if(pool->isClosed) break;
15.                         else pool->cond.wait(locker);
16.                     }
17.                 }).detach();
18.             }
19.     }

可以看到第11行获得了对象并直接执行task(),这就是在执行刚才绑定的那个WebServer::OnRead()

1. void WebServer::OnRead_(HttpConn* client) {
2.     assert(client);
3.     int ret = -1;
4.     int readErrno = 0;
5.     ret = client->read(&readErrno);
6.     if(ret <= 0 && readErrno != EAGAIN) {
7.         CloseConn_(client);
8.         return;
9.     }
10.     OnProcess(client);                   <--------------------------------------
11. }

WebServer::OnRead_(HttpConn* client) 中,又调用了client->read(&readErrno),这里就是读取的关键,这一行读取了请求之后,如果没有发生错误(即readErrno == EAGAIN)或者客户端关闭,那么就会进一步执行第10行的OnProcess(client)
再来看它的代码:

1. void WebServer::OnProcess(HttpConn* client) {
2.     if(client->process()) {                   <--------------------------------------
3.         epoller_->ModFd(client->GetFd(), connEvent_ | EPOLLOUT);
4.     } else {
5.         epoller_->ModFd(client->GetFd(), connEvent_ | EPOLLIN);
6.     }

if()中调用了client->process(),这里进行了请求的解析,这个函数在httpconn.cpp中:

1. bool HttpConn::process() {
2.     request_.Init();
3.     if(readBuff_.ReadableBytes() <= 0) {
4.         return false;
5.     }
6.     else if(request_.parse(readBuff_)) {                   <--------------------------------------
7.         LOG_DEBUG("%s", request_.path().c_str());
8.         response_.Init(srcDir, request_.path(), request_.IsKeepAlive(), 200);
9.     } else {
10.         response_.Init(srcDir, request_.path(), false, 400);
11.     }
12. 
13.     response_.MakeResponse(writeBuff_);
14.     /* 响应头 */
15.     iov_[0].iov_base = const_cast<char*>(writeBuff_.Peek());
16.     iov_[0].iov_len = writeBuff_.ReadableBytes();
17.     iovCnt_ = 1;
18. 
19.     /* 文件 */
20.     if(response_.FileLen() > 0  && response_.File()) {
21.         iov_[1].iov_base = response_.File();
22.         iov_[1].iov_len = response_.FileLen();
23.         iovCnt_ = 2;
24.     }
25.     LOG_DEBUG("filesize:%d, %d  to %d", response_.FileLen() , iovCnt_, ToWriteBytes());
26.     return true;
27. }

重点在:

6.     else if(request_.parse(readBuff_)) {
7.         LOG_DEBUG("%s", request_.path().c_str());
8.         response_.Init(srcDir, request_.path(), request_.IsKeepAlive(), 200);
9.     } else {
10.         response_.Init(srcDir, request_.path(), false, 400);
11.     }

可以看到,如果请求解析成功,就会返回一个正确的响应,如果请求解析失败,那么就会返回400响应。

至此整个请求的解析逻辑梳理完毕,总结一下就是:

WebServer::Start()
-> WebServer::DealRead_(&users_[fd])
--> ThreadPool::threadpool_->AddTask(std::bind(&WebServer::OnRead_, this, client))
---> WebServer::OnRead_(HttpConn* client)
----> WebServer::OnProcess(HttpConn* client)
-----> HttpConn::process()
------> request_.parse(readBuff_)

由此可以看出,服务端只会读取一次(这里的一次是指检测到了一次EPOLLIN事件),无论这一次是否读取到了一个完整的HTTP请求,都会把它当成一个完整的HTTP请求进行解析。

接下来进行调试验证,开启一个终端,执行nc -v 192.168.1.101 1316,用netcat来模拟一个客户端程序:

dylan@dylan-VirtualBox:~/Workspaces/repo/WebServer$ nc -v 192.168.1.101 1316
Connection to 192.168.1.101 1316 port [tcp/*] succeeded!

假设我们有一个请求为:

GET /index.html HTTP/1.1\r\n\r\n

netcat输入这一行,然后按Ctrl+D(即EOF,如果直接按回车会在后面加一个\n),发现服务器成功返回了页面:

dylan@dylan-VirtualBox:~/Workspaces/repo/WebServer$ nc -v 192.168.1.101 1316
Connection to 192.168.1.101 1316 port [tcp/*] succeeded!
GET /index.html HTTP/1.1\r\n\r\nHTTP/1.1 200 OK
Connection: close
Content-type: text/html
Content-length: 3148

<!--
 * @Author       : mark
 * @Date         : 2020-06-30
 * @copyleft GPL 2.0
-->
<!DOCTYPE html>
<html lang="en">

<head>

     <meta charset="UTF-8">

     <title>MARK-首页</title>
     <link rel="icon" href="images/favicon.ico">
     <link rel="stylesheet" href="css/bootstrap.min.css">
     <link rel="stylesheet" href="css/animate.css">
     <link rel="stylesheet" href="css/magnific-popup.css">
     <link rel="stylesheet" href="css/font-awesome.min.css">

     <!-- Main css -->
     <link rel="stylesheet" href="css/style.css">

</head>

<body data-spy="scroll" data-target=".navbar-collapse" data-offset="50">

 ...省略...

</body>

</html>

接下来我们试一下,如果先只发送一部分会发生什么:

dylan@dylan-VirtualBox:~/Workspaces/repo/WebServer$ nc -v 192.168.1.101 1316
Connection to 192.168.1.101 1316 port [tcp/*] succeeded!
GET /index.htmlHTTP/1.1 404 Not Found
Connection: close
Content-type: text/html
Content-length: 3149

...省略...

返回了一个404,不过这里发送的请求是不包含请求首部的情况,我们假设一下,如果一个HTTP请求包含了很多首部字段,以至于分次才能收到一个完整的HTTP请求,这样的话服务端岂不是会出错?

再用cgdb调试一下,使用cgdb ./server载入文件信息,然后在webserver.cpp中给DealRead()打一个断点:

Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from ./server...done.
(gdb) break webserver.cpp: 98
Breakpoint 1 at 0x2399c: file ../code/server/webserver.cpp, line 98.

输入run开始执行程序,端口1316:

(gdb) run
Starting program: /home/dylan/Workspaces/repo/WebServer/bin/server 
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
[New Thread 0x7ffff5d52700 (LWP 12577)]
[New Thread 0x7ffff5551700 (LWP 12578)]
[New Thread 0x7ffff4d50700 (LWP 12579)]
[New Thread 0x7ffff454f700 (LWP 12580)]
[New Thread 0x7ffff3d4e700 (LWP 12581)]
[New Thread 0x7ffff354d700 (LWP 12582)]
[New Thread 0x7ffff2b3a700 (LWP 12583)]

程序执行成功,线程也初始化了,然后另一个终端启动netcat,发送一个不完整的请求:

dylan@dylan-VirtualBox:~/Workspaces/repo/WebServer$ nc -v 192.168.1.101 1316
Connection to 192.168.1.101 1316 port [tcp/*] succeeded!
GET /index.html

然后跟踪断点
image

image

image

image

image

image

可以看到,执行流直接走到了返回400的语句,根本没有等待客户端发送另一部分HTTP请求。

不知道这算不算一个bug,分析就到这里。

捉个虫= =貌似

HttpResponse.cpp第17行
{ "word", "application/nsword"}
是 msword 输错了吧

应当如何处理isKeepAlive的连接?

源代码中,对于isKeepAliveonWrite的请求,会在response的onWrite发回后,再次调用process处理新请求,这是一种怎样的考虑?

  • 是在readBuf里有一大串请求,需要一个个解开吗?
void WebServer::OnWrite_(HttpConn* client) {
    assert(client);
    int ret = -1;
    int writeErrno = 0;
    ret = client->write(&writeErrno);   // 写数据

    // 如果将要写的字节等于0,说明写完了,判断是否要保持连接,保持连接继续去处理
    if(client->ToWriteBytes() == 0) {
        /* 传输完成 */
        if(client->IsKeepAlive()) {
            OnProcess(client);
            return;
        }
    }

开启代理访问网页时失败

我用127.0.0.1访问web服务器均可以访问,但是换其他网卡的ip地址后,如果是开了代理那么打开网页有时正常有时失败,没有代理的话是正常的,这个bug应该是httpConn::read()那里出现问题,就是数据没有读取完,导致request header不完整

ssize_t HttpConn::read(int* saveErrno) {
    ssize_t len = -1;
    do {
        len = readBuff_.ReadFd(fd_, saveErrno);
        if (len <= 0) {
            break;
        }
    } while (isET);
    return len;
}

为什么每次登录或注册后日志会报一个error

日志的一部分:

2022-09-05 21:52:26.966223 [info] : Client[19] in!
2022-09-05 21:52:26.967184 [info] : Verify name:admin pwd:123
2022-09-05 21:52:26.969072 [error]: RequestLine Error
2022-09-05 21:52:26.969642 [info] : Client[18] quit!

关于线程池的类内结构体

将线程池信息存到结构体里面然后由子线程通过指针捕获的目的是什么呢?能否直接作为类的成员变量然后通过this指针传递给匿名函数。

缓冲区的线程问题

我最近想在现有缓冲区的基础上加一个上传文件的过程,写好代码后再用gdb调试,发现处理缓冲区的过程中会有别的线程进来读处理过的内容,导致读混乱,然后遇到buffer的断言,程序就被Absort了。
我的做法是解析body的时候,添加一个函数,解析我的multipart/form-data,也就改了httprequest文件。这个线程问题搞得我很迷茫阿,我也看了线程池的创建,解完锁再执行的task,再上锁。我感觉逻辑上也没什么问题。但现在就是几KB的文件可以完整的传上去,文件一大就会遇到问题

log按行数拆分文件存在问题

log文件,比如设置达到50行则拆分文件,实际结果,大多数情况下都不是50行一个log文件,有40多行的,也有50多行的。试过把锁的范围扩大,并没有解决这个问题。

关于RAII使用的疑惑

你好,我看了你的项目,其中说到,数据库连接池使用了RAII,但是在httprequest.cpp实现的233行你为什么还是手动释放sql。

设置文件为非阻塞

int WebServer::SetFdNonblock(int fd) {
    assert(fd > 0);
    return fcntl(fd, F_SETFL, fcntl(fd, F_GETFD, 0) | O_NONBLOCK);
}

其中F_GETFD是否应该改为F_GETFL?

build目录下的makefile写的有问题

错误的makefile如下:
CXX = g++
CFLAGS = -std=c++14 -O2 -Wall -g

TARGET = server
OBJS = ../code/log/.cpp ../code/pool/.cpp ../code/timer/.cpp
../code/http/
.cpp ../code/server/.cpp
../code/buffer/
.cpp ../code/main.cpp

all: $(OBJS)
$(CXX) $(CFLAGS) $(OBJS) -o ../bin/$(TARGET) -pthread -lmysqlclient

clean:
rm -rf ../bin/$(OBJS) $(TARGET)

错误如下
g++ -std=c++14 -O2 -Wall -g ../code/log/.cpp ../code/pool/.cpp ../code/timer/.cpp ../code/http/.cpp ../code/server/.cpp ../code/buffer/.cpp ../code/main.cpp -o ../bin/server -pthread -lmysqlclient
../code/http/httprequest.cpp: In static member function ‘static bool HttpRequest::UserVerify(const string&, const string&, bool)’:
../code/http/httprequest.cpp:185:18: warning: variable ‘j’ set but not used [-Wunused-but-set-variable]
185 | unsigned int j = 0;
| ^
../code/http/httprequest.cpp:187:18: warning: variable ‘fields’ set but not used [-Wunused-but-set-variable]
187 | MYSQL_FIELD *fields = nullptr;
| ^~~~~~
/usr/bin/ld: cannot open output file ../bin/server: 没有那个文件或目录
collect2: error: ld returned 1 exit status
make: *** [Makefile:10:all] 错误 1

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.