civetweb / civetweb Goto Github PK

Embedded C/C++ web server

License: Other

Perl 0.14% Makefile 1.13% C 79.33% HTML 2.93% Lua 5.64% C++ 2.40% Shell 1.07% PHP 0.03% QMake 0.06% Batchfile 3.12% CMake 3.41% JavaScript 0.63% Python 0.01% CSS 0.01% Dockerfile 0.09%

civetweb's Introduction

CivetWeb

The official home of CivetWeb is on GitHub https://github.com/civetweb/civetweb

Continuous integration for Linux and macOS (Travis CI):

Continuous integration for Windows (AppVeyor):

Test coverage check (coveralls, codecov) (using different tools/settings):

Static source code analysis (Coverity):

CodeQL semantic code analysis:

Project Mission

Project mission is to provide easy to use, powerful, C (C/C++) embeddable web server with optional CGI, SSL and Lua support. CivetWeb has a MIT license so you can innovate without restrictions.

CivetWeb can be used by developers as a library, to add web server functionality to an existing application.

It can also be used by end users as a stand-alone web server running on a Windows or Linux PC. It is available as single executable, no installation is required.

Where to find the official version?

End users can download CivetWeb binaries / releases from here on GitHub https://github.com/civetweb/civetweb/releases or SourceForge https://sourceforge.net/projects/civetweb/

Developers can contribute to CivetWeb via GitHub https://github.com/civetweb/civetweb

Due to a bug in Git for Windows V2.24 CivetWeb must be used with an earlier or later version (see also here).

Bugs and requests should be filed on GitHub https://github.com/civetweb/civetweb/issues

New releases are announced on Google Groups https://groups.google.com/d/forum/civetweb

Formerly some support question and discussion threads have been at Google groups. Recent questions and discussions use GitHub issues.

Source releases can be found on GitHub https://github.com/civetweb/civetweb/releases

A very brief overview can be found on GitHub Pages https://civetweb.github.io/civetweb/

Quick start documentation

docs/Installing.md - Install Guide (for end users using pre-built binaries)
docs/UserManual.md - End User Guide
docs/Building.md - Building the Server (quick start guide)
docs/Embedding.md - Embedding (how to add HTTP support to an existing application)
docs/OpenSSL.md - Adding HTTPS (SSL/TLS) support using OpenSSL.
docs/Docker.md - Use CivetWeb in a Docker container.
API documentation - Additional documentation on the civetweb application programming interface (civetweb.h).
RELEASE_NOTES.md - Release Notes
SECURITY.md - Security Policy
LICENSE.md - Copyright License

Overview

CivetWeb keeps the balance between functionality and simplicity by a carefully selected list of features:

Forked from Mongoose in 2013, before it changed the licence from MIT to commercial + GPL. A lot of enhancements have been added since then, see RELEASE_NOTES.md.
Maintains the liberal, permissive, commercial-friendly, MIT license
Project is free from copy-left licenses, like GPL, because you should innovate without restrictions.
Works on Windows, Mac, Linux, UNIX, IOS, Android, Buildroot, and many other platforms.
Scripting and database support (CGI, Lua Server Pages, Server side Lua scripts, Lua SQLite database, Server side JavaScript). This provides a ready to go, powerful web development platform in a one single-click executable with no dependencies. 0
Support for CGI, SSI, HTTP digest (MD5) authorization, WebSocket, WebDAV.
Experimental HTTP/2 support.
HTTPS (SSL/TLS) support using OpenSSL.
Optional support for authentication using client side X.509 certificates.
Resumed download, URL rewrite, file blacklist, IP-based ACL.
Can run as a Windows service or systemd service.
Download speed limit based on client subnet or URI pattern.
Simple and clean embedding API.
The source is in single file for drop in compilation.
Embedding examples included.
HTTP client capable of sending arbitrary HTTP/HTTPS requests.
Websocket client functionality available (WS/WSS).

Optionally included software

Optional dependencies

zlib

OpenSSL

Mbed TLS

Support

This project is very easy to install and use. Please read the documentation and have a look at the examples.

Recent questions and discussions usually use GitHub issues. Some old information may be found on the mailing list, but this information may be outdated.

Feel free to create a GitHub issue for bugs, feature requests, questions, suggestions or if you want to share tips and tricks. When creating an issues for a bug, add enough description to reproduce the issue - at least add CivetWeb version and operating system. Please see also the guidelines for Contributions and the Security Policy

Note: We do not take any liability or warranty for any linked contents. Visit these pages and try the community support suggestions at your own risk. Any link provided in this project (including source and documentation) is provided in the hope that this information will be helpful. However, we cannot accept any responsibility for any content on an external page.

Contributions

Contributions are welcome provided all contributions carry the MIT license.

DO NOT APPLY fixes copied from Mongoose to this project to prevent GPL tainting. Since 2013, CivetWeb and Mongoose have been developed independently. By now the code base differs, so patches cannot be safely transferred in either direction.

Some guidelines can be found in docs/Contribution.md.

Authors

CivetWeb was forked from the last MIT version of Mongoose in August 2013. Since then, CivetWeb has seen many improvements from various authors (Copyright (c) 2013-2021 the CivetWeb developers, MIT license).

A list of authors can be found in CREDITS.md.

CivetWeb is based on the Mongoose project. The original author of Mongoose was Sergey Lyubka(2004-2013) who released it under the MIT license. However, on August 16, 2013, Mongoose was relicensed to a dual GPL V2 + commercial license and CiwetWeb was created by Thomas Davis (sunsetbrew) as "the MIT fork of mongoose". The license change and CivetWeb fork was mentioned on the Mongoose Wikipedia page as well, but it's getting deleted (and added again) there every now and then.

Using the CivetWeb project ensures the MIT licenses terms are applied and GPL cannot be imposed on any of this code, as long as it is sourced from here. This code will remain free with the MIT license protection.

civetweb's People

Contributors

Stargazers

Watchers

Forkers

danieloaks harikamath gerhobbelt divyprakash bgreat pfalcon-mirrors grenclave anatolyko sameer76 dlethe bernhard- tzimmofsecure grredwings celeron55 bigjoe02 gherlein marc40000 animetrics ceph tonywilk jjlin snations ademko pawelmarc jmc- weidagang fremouw cholcombe973 fgaranda leonardosantos grahack zhao07 nathank cognitivenetworks jdblair foran xuxiandi catinred2 samcrow uplink03 dinesh-ramakrishnan doomhammer gdxn kuna-systems yehudasa jawi jshelley rjinswand jimevans snowwolf1 drumtechnologiesltd martinec tiborshub rpetrich iqwxiaopeng swc4848a nigels-com jjensen scott2014 seem-sky projectarchives wyrover p-brz kostyll jojo2nd yodamaster erguotou cdbishop tracedev marklakata joeoszlanczi nihildeb bloudraak sauoq torque59 dhh1128 tagami-keisuke wheelcomplex dialga keithel mattyclarkson vcatechnology da4089 masda70 baroborobotics evizbiz medialets david-loffredo c2go g-p-s nfnm kalphamon shawnye undersail bagobor phillab alecs2287 tempbottle zhouyuan enseed

civetweb's Issues

mg_start/set_ssl_option could not be called twice

If mg_start is called multiple times (e.g. multiple embedded webserver, restart of webserver), the function "set_ssl_options" is called multiple times. The result is a memory leak for "ssl_mutexes" and an undefined behaviour for the openssl crypto engine.
Maybe it makes sense to check whether a callback is already defined for the crypto engine (e.g. CRYPTO_get_id_callback()).

poll interface

Why is there no poll interface as with mongoose. I mean equivalent of mg_poll_server()?

I understand that current design might be more convenient to some but original one was much more flexible because threading model was up to the user.

In my case I am using civetweb in game engine and I had to really work-around current design to get back to original poll version.

First call to sslize() is very slow

Hi,
I'm using civetweb embedded in a c++ app. It works great except for the first https request it had to serve which is very slow (up to 10 seconds).
I located the problem in the sslize() function , and more precisely on this call : func(conn->ssl) == 1.
Unfortunately I don't really understand what this code do, and can't go further alone :(

This how i intiialize civetweb :

const char * options[] = {
    "document_root", DOCUMENT_ROOT, 
    "listening_ports", "80,443s", 
    "ssl_certificate", "D://server.pem", 
    "error_log_file", LOG_FILE,
    0
};
this->m_pServer = new CivetServer(options);

I'm using a self signed certificate generated with openssl. I followed the wiki procedure to generate the certificate.

This delay is only a problem if it occurs on a request. Is there something to initialize before being able to handle request ?

Thanks

Can Lua Server Page support `for`and `if` ? please see details

Can Lua Server Page support for and if like this, as JSTL do it :

<? for i=1,5 do ?>
  <li><?=i?></li>
<? end ?>

mg_read variable shadowing

mg_read has a function param of void* buf, but also declares a local variable char buf[64].

This should be fixed as it generates a warning with -Wshadow on clang. I can look at it later but figured I'd give you a heads up.

Wrong define in docs

Docs say that define for WebSocket support is: WITH_WEBSOCKET but it sould be USE_WEBSOCKET.

Generating HTML

I can implement an interface which would make the following code possible:

int helloWorld(struct mg_connection *conn, void *cbdata)
{
    struct mg_request_info * req_info = mg_get_request_info(conn);  
    char output[1000];


    el html=c(NULL,"html"); //create a new element called html
        el head=c(html,"head");
            el meta=c(head,"meta");
            sa(meta,"charset","utf-8"); //set attribute
        el body=c(html,"body");
            t(body,"Hello world!"); //text node
            c(body,"br");
        el table=c(body,"table");
        sa(table,"border","1");
        el td,tr;
            td=c(table,"td");
                    tr=c(td,"tr");
                        t(tr,"A third cell!");
                    tr=c(td,"tr");
                        t(tr,"A fourth cell!");
            td=c(table,"td");
                    tr=c(td,"tr");
                        t(tr,"A cell!");
                    tr=c(td,"tr");
                        t(tr,"A second cell!");

    int size=xmlToString(html,output,1000);

     mg_printf(conn,
    "HTTP/1.1 200 OK\r\n"
    "Content-Type: text/html\r\n"
    "Content-Length: %d\r\n"
    "\r\n"
    "%s",size, output);

    xmlDelete(html);
    return 0;
}

This would generate the following HTML:

<html><head><meta charset="utf-8" /></head>
<body>Hello world!<br /><table>...etc...</table></body></html>

Are you interested in this?

Sync tags

Currently this repo seems to be missing the v1.5 tag that is present in the https://github.com/sunsetbrew/civetweb repo.

Add SQLITE_DEFAULT_FILE_FORMAT=4 to SQLite compile options

Add -DSQLITE_DEFAULT_FILE_FORMAT=4 to SQLite compile options, this is the default format used by SQLite since version 3.7.10. Check https://www.sqlite.org/compile.html#default_file_format.

Method to get POST request parameters via C++ interface

Hi there.

First off, thanks for the great library.

I'm having troubles using the C++ interface to handle POST requests. Let's say we have a form and there's an input named "foo". I tried to use the CivetServer's getParam(conn, "foo", ...) method, but it segfaults at:
https://github.com/bel2125/civetweb/blob/4e62bd5b10b7873df7a9db959a3c7de30e0993b7/src/CivetServer.cpp#l154
at strlen() call, because query turns out to be NULL.

For now I use the mg_read() and mg_get_var() functions which work just fine. As I understand from the code, getParam() is targeted to data like "name1=value1&...", as in GET requests. So could you please add support of POST parameters in the C++ interface?

Platform: Arch Linux 3.13.5, x86_64

CGI POST is no longer working

Keep getting a 405 for method not supported. Looks like a bunch of REST code when in and broke it.

Incorrect poll() handling in getreq() for keep alive connections.

Recently I used civetweb as an embedded HTTP server in a project that requires the support of persistent (keep alive) connections, and I found that there seems to be some glitches in the polling code in the getreq() function.

In process_new_connection(), function getreq() is called in the beginning of the do ... while loop with timeout set to TIMEOUT_INFINITE, which is defined as an enum constant with value -1 in civetweb.h. However, in the beginning of getreq(), there is a code segment dealing with polling from the client socket that has unreachable poll() call in it:

if (timeout >= 0) {
    pfd.fd = conn->client.sock;
    switch (poll(&pfd, 1, timeout)) {
    case 0:
        snprintf(ebuf, ebuf_len, "%s", "Timed out");
        *err = 408;
        return 0;
    case -1:
        snprintf(ebuf, ebuf_len, "%s", "Interrupted");
        *err = 500;
        return 0;
    }
}

Since getreq() is called with timeout set to TIMEOUT_INFINITE that is equal to -1, the control will never reach the poll() call. Besides, even if the control could reach the poll() call, since there is no any event registered to the pfd.events field, the poll() call will block the execution until either timed out or being interrupted.

The improper poll() handling would probably not be a big problem for non-persistent connections since the client socket would most likely be ready to be read when getreq() is first called. However, it could cause problems for persistent connections when getreq() is called the next time in the do ... while loop if the client idles after the first request.

When getreq() is called after the loop starts again, since poll() call is skipped, the following read_request() then calls the pull() function, which in turn tries to read from underlying I/O channel and block the execution until something is read, the underlying I/O call (either recv() or SSL_read()) is interrupted, or the I/O call is timed out. If the client idles and the low level I/O call is timed out, the read_request() call returns error, and a 400 message is sent to the client before the connection is closed.

To demonstrate this problem, compile and execute the "civetweb" server in one console as follow:

$ echo 'Hello World' > hello.txt
$ ./civetweb -enable_keep_alive yes -document_root .
Civetweb V1.7 started on port(s) 8080 with web root [/tmp/civetweb-test]

and connect to localhost:8080 with telnet in another console window, send "GET /hello.txt" request with "Connection: Keep-Alive" header and then idle for more than 30 seconds:

$ telnet localhost 8080
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
GET /hello.txt HTTP/1.1
Connection: Keep-Alive

HTTP/1.1 200 OK
Date: Mon, 23 Mar 2015 15:43:20 GMT
Last-Modified: Mon, 23 Mar 2015 15:39:04 GMT
Etag: "55103398.12"
Content-Type: text/plain
Content-Length: 12
Connection: keep-alive
Accept-Ranges: bytes

Hello World
HTTP/1.1 400 Bad Request
Content-Type: text/plain
Content-Length: 52
Date: Mon, 23 Mar 2015 15:43:50 GMT
Connection: keep-alive

Error 400: Bad Request
Client sent malformed requestConnection closed by foreign host.

As shown above, an unexpected 400 response was sent by the server, and then the connection was terminated.

For now, I circumvent this problem by removing the if (timeout >= 0) check, and register POLLIN to the pfd.events field. In this way, the poll() call will poll the client socket with infinite timeout. I wonder if there is better solution to this, such as, instead of calling getreq() with TIMEOUT_INFINITE unconditionally, calling it with the timeout value of the "request_timeout_ms" config option when "enable_keep_alive" is set to "yes", and close connection without sending 400 response if the `poll()' call times out.

Please advise. Thanks!

Increase MAX_WORKER_THREADS so it can be used in high traffic websites

Hi,
MAX_WORKER_THREADS is set to 1024 in civetweb.c, [1] why not set it to something like 20K or 30K, this is useful and has no bad effect since the number is set in the config file, I needed to increase this to 20480 becuase I need a very large number of long connctions.

[1] https://github.com/bel2125/civetweb/blob/master/src/civetweb.c

Bad 'if' check in handle_websocket_request

In handle_websocket_request you put the result of match_prefix in lua_websock, which returns -1 if false. You then check if lua_websock is true, shouldn't this instead check if lua_wesock is greater than 0?

For me this was causing a crash elsewhere in your code but I see you've fixed that crash already. Do you have an idea on when you'll release 1.7?

mg_start_thread is not 64 bit safe

We are using CivetWeb in a 64 bit application. In reviewing some of our pointer casting, I noticed that mg_start_thread is casting the return of _beginthread as a long. I think the proper thing to do would be to cast it as a uintptr_t.

The value is not being returned so I don't think that this will cause any problems, it's just a question, is the cast needed? What happens if the value returned is larger then the long.

mg_upload file created with \ character

I'm running civetweb on an embedded Linux system and uploading a file to it using IE 11.0.9600.17358. The file on the file system that civetweb is creating starts with a \ character, which doesn't work so well on Linux.

Looking at the code in mg_upload(), you can see that an attempt was made to remove any / or \ characters from the filename in the request. However, if one of those characters is found, the code should increment it's string pointer past the character that it found but it's not doing that.

Possible bug on line 5237

On line 5237 of civetweb.c, the following appears:

        if ((conn->ctx->callbacks.websocket_data != NULL &&
#ifdef USE_LUA
            (conn->lua_websocket_state == NULL) &&
#endif
            !conn->ctx->callbacks.websocket_data(conn, mop, data, data_len)) ||
#ifdef USE_LUA
           (conn->lua_websocket_state &&
               !lua_websocket_data(conn, conn->lua_websocket_state, mop, data, data_len)) ||
#endif
           (buf[0] & 0xf) == WEBSOCKET_OPCODE_CONNECTION_CLOSE) { /* Opcode == 8, connection close */
           break;
        }

should the buf[0] in (buf[0] & 0xf) == WEBSOCKET_OPCODE_CONNECTION_CLOSE be mop?

Embeded in C++ with OpenSSL

Hi,
I successfully embedded the server in my C++ application. But now I need OpenSSL Support for that server. I couldn't find any documentation on how to do this? Could you add an example for this or provide some information about that?

Second question: I'm a bit confused about openssl and windows support. In docs for openssl you say there are windows packages, but in yaSSL you say this is needed for windows. I need my application to be executable on windows and linux. Is it possible to use OpenSSL on both platforms?

Thanks in advance!

Memory leak when only one handler defined

The mg_set_request_handler method does exit the initial loop too early when only one handler is defined and the caller tries to remove that handler. The memory allocated for the handler is thus never freed.

Just started with civetweb on Windows using Visual Studio 2012.
Deleting the CivetServer instance leaks the handler.

uri supprt for websockets

Will URI/path matching will also be supported for WebSockets?

I mean automatic invocation or registered callback (al'a mg_set_request_handler) for specified URI path when protocol is "ws:". Say mg_set_websocket_connection_handler?

Fix for Makefile for MinGW32/MSYS

I tried building civetweb with MinGW32/MSYS. The patch fixes missing libraries and adds a shared library target for windows. Apart from this, it works fine.

diff --git a/Makefile b/Makefile
index 133a1ef..e726444 100644
--- a/Makefile
+++ b/Makefile
@@ -24,6 +24,8 @@ DOCDIR = $(DATAROOTDIR)/doc/$(CPROG)
 SYSCONFDIR = $(PREFIX)/etc
 HTMLDIR = $(DOCDIR)

+UNAME := $(shell uname)
+
 # desired configuration of the document root
 # never assume that the document_root actually
 # exists on the build machine.  When building
@@ -100,6 +102,13 @@ ifeq ($(TARGET_OS),LINUX)
    CAN_INSTALL = 1
 endif

+ifneq (, $(findstring MINGW32, $(UNAME)))
+   LIBS += -lws2_32 -lcomdlg32
+   SHARED_LIB=dll
+else
+   SHARED_LIB=so
+endif
+
 all: build

 help:
@@ -174,7 +183,7 @@ endif

 lib: lib$(CPROG).a

-slib: lib$(CPROG).so
+slib: lib$(CPROG).$(SHARED_LIB)

 clean:
    rm -rf $(BUILD_DIR)
@@ -191,6 +200,10 @@ lib$(CPROG).a: $(LIB_OBJECTS)
 lib$(CPROG).so: CFLAGS += -fPIC
 lib$(CPROG).so: $(LIB_OBJECTS)
    $(LCC) -shared -o $@ $(CFLAGS) $(LDFLAGS) $(LIB_OBJECTS)
+   
+lib$(CPROG).dll: CFLAGS += -fPIC
+lib$(CPROG).dll: $(LIB_OBJECTS)
+   $(LCC) -shared -o $@ $(CFLAGS) $(LDFLAGS) $(LIB_OBJECTS) $(LIBS) -Wl,--out-implib,lib$(CPROG).dll.a

 $(CPROG): $(BUILD_OBJECTS)
    $(LCC) -o $@ $(CFLAGS) $(LDFLAGS) $(BUILD_OBJECTS) $(LIBS)

Crash when listening several ports

Hi,

When I listen to several ports, for example: "443s, 80r", but port 80 is already used, civetweb crash during ssl_locking_callback due to an invalid mutex.

Best Regards,
Eric

firewall warning on launching civetweb with command line arguments

I am trying to launch civetweb.exe from C++ code with some command line arguments defining new port and directory path. Everything works fine but I always get Windows firewall warning. How to suppress that? Can somebody help?

database written in same directory as civetweb was started from

Suppose the following shell session

$ ls ~/www
page.lp # this is the same file as test/page.lp
$ cd /tmp
$ civetweb -document_root ~/www/
Civetweb V1.6 started on port(s) 8080 with web root [/home/arch/www/]
# Now go to http://127.0.0.1:8080/page.lp and reload the page a few times
^CExiting on signal 2, waiting for all threads to finish... done.
$ ls
requests.db
$ pwd
/tmp

Expected results

requests.db to be written in ~/www, which was specified with -document_root

free() was not declared in this scope

Hi Guys,
in the latest version, i got an error message that while compiling " 'free' was not declared in this scope". CivetServer.h, Line 286.

This error will issue in linux 64 bit environment.

Question regarding multi file uploads

I have modified the upload example and simply changed the form into this:

<form method="POST" action="/handle_post_request" enctype="multipart/form-data">
   <input name="file" type="file"> <br>
   <input name="file2" type="file"> <br>
   <input value="Upload" type="submit">
</form>

Next, I uploaded 2 files. Only one of them was saved on the server.
How do I handle multiple file uploads?

Websocket + web

Hello,

i'm using civetweb embedded in a C++ app to serve web page (443 & 80).
I need to add websocket support to some pages.

Should i start a second instance of civetweb with differents ports and callbacks to handle the websocket ?

Something like that :

const char * options[] = {
    "document_root", DOCUMENT_ROOT, 
    "listening_ports", PORT, 
    "ssl_certificate", SSL_CERTIFICATE, 
    "error_log_file", LOG_FILE,
    "enable_directory_listing","no",
    "extra_mime_types",".crt=application/x-x509-ca-cert",
    0
};
this->m_pServer = new CivetServer(options);

const char * options2[] = {
    "document_root", DOCUMENT_ROOT, 
    "listening_ports", "40000", 
    0
};
this->m_pServer2 = new CivetServer(options2,callbacks);

Or is it possible to change the websocket port ?

Thanks

Question: how to get header, parse POSTed content in Lua?

I found mg_get_header() in civetweb.h, but no counterpart in lua mg(table)

and I can not find function that can easily parse POSTed content in lua, could you add it or give me aa direction about how to deal it?

Thank you.

Assertion failure if full websocket header is not directly available in conn->buf when read_websocket() is called (?)

This is somewhat hard to debug for me, but for what I have diagnosed so far, it seems that this assertion can fire in a completely valid situation:
https://github.com/bel2125/civetweb/blob/master/src/civetweb.c#L4963

I can only cause it by flooding a bunch of short websocket messages from my local system to a certain server.

I think the situation is that the initial TCP chunk that contained the request contains part of the websocket header. Not 0 bytes, not full header length.

Civetweb should be fixed to be able to wait for the full header to arrive and then proceed to what it is now trying to do immediately when it has >0 bytes of the header available.

Segfault when transferring files larger than 2GB

First off thanks for making this great library!

I can't explain what is happening here because I'm still a beginner at c/c++. When I try to send a file larger than 2.something GB I'm seeing a segfault with the following backtrace:

(gdb) bt
#2 0x000000000041fda5 in mg_read (conn=0x6082000e1200, buf=0x7fffc89d8670,

len=<optimized out>) at src/civetweb.c:2187

define BUFSIZE=512*1024;

I call mg_read(conn, buffer, BUFSIZE); in a while loop with a buffer of BUFSIZE. Any thoughts on what might be happening? When I transfer smaller files it works great.

Port the build system to cmake

Would there be interest for a pull request which replaces the makefiles with a CMake build implementation?

This change will help maintain & improve the cross platform support.

It would be beneficial if I could build Civetweb for multiple versions of visual studio for instance.

Thoughts?

embedding in cpp

This is not a bug or issue
Hi , I'm starting out with c++ into web development for some pet projects. While reading embedding guide for civetweb , I was a bit confused about this procedure :

1.Create CivetHandlers for each URI.
2.Register the handlers with CivertServer::addHandler()
3.CivetServer starts on contruction and stops on destruction.
4.Use contructor options to select the port and document root among other things.
5.Use constructor callbacks to add your own hooks.

I have programmed in c++ ( native , never web) and in LAMP stack too , but this approach is quiet new to me. What do these handlers stand for and what are URI?
From my requirements , I need CiverServer to listen to ports continously , taking up any incoming connection , parsing out http headers ( might remove them completely or read them and react accordingly ) and lets say execute a function according to request data.

Does URI stands for incoming connection? And I have no idea how it would handle multiple simultaneous connections ( 5 6 incoming connections ).
Thanks in advance 😄

Improvements

bel2125,

My name is Artem. I represent company from Mointain View, CA.

We are currently evaluating civetweb server for use in one of our products.
We'd like to make some improvements in its code before testing with our
users.
Are you interested in a work to help us with these improvements?

Please do not hesitate to contact me. My email is [email protected]

Implement a websocket client interface

I'll be working on implementing a websocket client interface. I just wanted to open this issue to track discussion of its development and API organization as I start issuing pull requests.

Compiler warning (error with -Werror) in send_file_data

gcc 4.9.2 from ubuntu repositories.

In function ‘__fread_alias’,
    inlined from ‘send_file_data’ at ../external/civetweb/civetweb.cc:4230:76:
/usr/include/x86_64-linux-gnu/bits/stdio2.h:290:66: error: call to ‘__fread_chk_warn’ declared with attribute warning: fread called with bigger size * nmemb than length of destination buffer [-Werror]
  return __fread_chk (__ptr, __bos0 (__ptr), __size, __n, __stream);

As far as I can see, this warning is spurious as the read length is 'always' less than sizeof(buf) in the code... scan-build doesn't suggest anything, and changing to_read to be size_t incase it was compiler thinking it might be negative integer didn't help either.

As it stands, I really don't want to have to turn off -Werror, so am adding in some pragmas to temporarigly disable that warning around the code.

Possible issue and possible fix in websocket handling

I'm stumbling upon some strange rare 100% CPU consumption on one of my servers and I'm suspecting the issue is what this patch fixes:

diff --git a/3rdparty/civetweb/civetweb.c b/3rdparty/civetweb/civetweb.c
index 37c080b..737aa01 100644
--- a/3rdparty/civetweb/civetweb.c
+++ b/3rdparty/civetweb/civetweb.c
@@ -4829,6 +4829,8 @@ static void read_websocket(struct mg_connection *conn)
                         error = 1;
                         break;
                     }
+                    if (n == 0) // EOF
+                        break;
                     len += n;
                 }
                 if (error) {

I need to continue testing though; I'll report back once I know.

Unused functions base64_decode, b64reverse.

I like to compile with warnings as errors and as many warnings turned on as possible :) It likes to tell me off for unused functions (like these two)

CGI environment variables are not correct

SCRIPT_NAME is not always correct, e.g.,
http://localhost:8081/test/env.cgi/a/b/c?x=1&y=2
==> SCRIPT_NAME=/test/env.cgi/a/b/env.cgi instead of /test/env.cgi

PATH_TRANSLATED is not as defined in the RFC (http://www.ietf.org/rfc/rfc3875) section 4.1.6
PATH_TRANSLATED must be DOCUMENT_ROOT .. PATH_INFO

C API is unable to provide client IPv6 address to host application

In struct mg_request_info there is long remote_ip, which contains the IPv4 address for such clients. However, if a client connects using IPv6 there is no way of getting the address.

I might implement this myself if nobody else does it in the near future.

But what should the interface be like?

DoS: Empty Post Request

I'm having a weird issue:
When sending a post request without any post-data (i used "Poster" Addon for firefox) the server hangs until the client times out and closes the connection. The point where it stucks is within mg_read function. I try to use it as follows:

std::string data;
int blockSize = 1024 * sizeof(char), readBytes;

char buffer[blockSize];

while ((readBytes = mg_read(this->conn, buffer, blockSize)) > 0) {
    data.append(buffer, 0, (readBytes / sizeof(char)));
}
return data;

At that time the server doesn't response to any request anymore - until the client connection times out. Seems that this is kind of DoS bug.. I can't exactly spot the position where it's stuck, because when running in debug mode it sometimes act's normal and somites act's weird. But i traced it the following way: mg_read -> pull_all -> pull
Within pull()-method i lost the trace, don't know where exactly it hangs.

Could anybody try to reproduce this bug? When I send something within the post data, the connections gets answered immediatly, as expected.

Issues when compiling with TDM-GCC

I'm getting a plethora of errors when attempting to compile with TDM-GCC on Windows.
Is building with MinGW/TDM-GCC still supported?

mg_stop() waits until websocket clients disconnect on their own

I am using websocket connections, and when I call mg_stop() on the server, it will not stop until the clients have disconnected by themselves or attempted to send data after mg_stop(). Some of the clients are using civetweb's mg_connect_websocket_client(), and some are web browsers.

This old issue seems to match the behavior: https://code.google.com/p/mongoose/issues/detail?id=268

I am able to fix the problem on GNU/Linux by doing something like this: (alternatively using select() or poll() would also work)

diff --git a/3rdparty/civetweb/civetweb.c b/3rdparty/civetweb/civetweb.c
index 9c50bb6..1b9853b 100644
--- a/3rdparty/civetweb/civetweb.c
+++ b/3rdparty/civetweb/civetweb.c
@@ -2454,7 +2454,13 @@ static int pull(FILE *fp, struct mg_connection *conn, char *buf, int len)
         nread = SSL_read(conn->ssl, buf, len);
 #endif
     } else {
-        nread = recv(conn->client.sock, buf, (size_t) len, 0);
+               nread = -1;
+        while(conn->ctx->stop_flag == 0){
+            nread = recv(conn->client.sock, buf, (size_t) len, 0);
+            if(nread == -1 && (errno == EAGAIN || errno == EWOULDBLOCK))
+                continue;
+            break;
+        }
     }

     return conn->ctx->stop_flag ? -1 : nread;
@@ -7307,6 +7313,17 @@ static void process_new_connection(struct mg_connection *conn)

     keep_alive_enabled = !strcmp(conn->ctx->config[ENABLE_KEEP_ALIVE], "yes");

+    // Enable read timeouts on the socket
+    {
+        struct timeval t;
+        t.tv_sec = 1;
+        t.tv_usec = 0;
+        if (setsockopt(conn->client.sock, SOL_SOCKET, SO_RCVTIMEO,
+                (char *)&t,  sizeof(t))) {
+            mg_cry(conn, "%s: setsockopt(SO_RCVTIMEO) failed", __func__);
+        }
+    }
+
     /* Important: on new connection, reset the receiving buffer. Credit goes
        to crule42. */
     conn->data_len = 0;

#undef before #define

Lines like:

#define malloc  DO_NOT_USE_THIS_FUNCTION__USE_mg_malloc

should have this before:

#undef malloc

I already redefined malloc/free/etc and this unguarded #define results in an annoying compiler warning.

Soap support

Hi,

I've just read your post from July (https://groups.google.com/forum/#!msg/civetweb/WT1oFs6eC6I/DfDq0t4VWJIJ) about adding SOAP support to CivetWeb. I'm trying to use your webserver but I need to support gSOAP requests.
So, I was wondering if you had added SOAP support to civetweb ?

Thanks,

Compile warnings using MSVC 2010

When compiling civetweb.c using MSVC 2010, you receive the following two warnings:

c:\projects\webdriver\third_party\civetweb\civetweb.c(2294): warning C4244: '-=' : conversion from 'int64_t' to 'size_t', possible loss of data
c:\projects\webdriver\third_party\civetweb\civetweb.c(2305): warning C4244: 'return' : conversion from 'int64_t' to 'int', possible loss of data

I believe casts to the appropriate data types would be the correct way to silence those warnings

Percent-encoding problems

I think return -1; of mg_url_encode() must be break;, or check
the returned value. It causes buffer overrun.
Then, the encoded char's of print_dir_entry() and print_dav_dir_entry() should be three times(worst-case) the size. This is important to non-English environments.

Compile error on Clang

The patch below fixes compiling for Clang. By default Clang builds with the C99 standard, while GNU GCC defaults to GNU89. There's a small difference in how inlines are handled, to quote the compatibility list of Clang,

In C99, inline means that a function's definition is provided only for inlining, and that there is another definition (without inline) somewhere else in the program. That means that this program is incomplete, because if add isn't inlined (for example, when compiling without optimization), then main will have an unresolved reference to that other definition.

See http://clang.llvm.org/compatibility.html#inline for more information.

The patch below fixes this error, and is also compatible with GNU89.

diff --git a/src/civetweb.c b/src/civetweb.c
index 90554d1..d31f855 100644
--- a/src/civetweb.c
+++ b/src/civetweb.c
@@ -448,10 +448,10 @@ static void * mg_realloc_ex(void * memory, size_t newsize, const char * file, un
 #define mg_free(a)        mg_free_ex(a, __FILE__, __LINE__)

 #else
-__inline void * mg_malloc(size_t a)             {return malloc(a);}
-__inline void * mg_calloc(size_t a, size_t b)   {return calloc(a, b);}
-__inline void * mg_realloc(void * a, size_t b)  {return realloc(a, b);}
-__inline void   mg_free(void * a)               {free(a);}
+static __inline void * mg_malloc(size_t a)             {return malloc(a);}
+static __inline void * mg_calloc(size_t a, size_t b)   {return calloc(a, b);}
+static __inline void * mg_realloc(void * a, size_t b)  {return realloc(a, b);}
+static __inline void   mg_free(void * a)               {free(a);}
 #endif

 #define malloc  DO_NOT_USE_THIS_FUNCTION__USE_mg_malloc

non file based PUT requests require authorization

Not sure if this is an issue or if I'm just using the library incorrectly.

I'm using civetweb to implement a RESTful api. I was initially developing against v1.6 and was able to get my project working. However when switching to the master (to take advantage of the changes since v1.6) I found my PUT requests returning 401 Unauthorized.

The cause I have found is the addition of a check for authorization, which is performed before the request is delegated to the handle_request callback. (I haven't defined NO_FILES as I do require supporting of GET requests to return html pages)

Ideally I do not want to implement any authorization for PUT requests as for my purposes I do not wish to force the user to be authorised to use this aspect of the api

I've found the simplest change to resolve this is for put requests to be allowed if no PASSWORD_FILE is found/specified via options, I've tested this change here: cdbishop@fe483bf

However I'm unsure if this change would cause other issue/break functionality for other users?

I guess the first question to ask: Is using civetweb to implement a RESTful api a valid use of the library or is this not the intention of it?

Also It seems based on code comments/function names that the uri is assumed to be pointing to a file (where I require uri end points that don't point to anything, e.g. I have an /api/objects uri which can accept PUT requests) Is this supported? Or am I likely to run into other issues where a file is assumed?

civetweb bug fix please

hi
First, I say that the English can not be well.
If you start the server by specifying a port that is already in use, it returns an error of AccessVioletion.
It will call the free_context while returns false in set_ports_option in mg_start function but occur at this point.
I occur in EnterCriticalSection part when calling pthread_cond_destroy in free_context function.
it seems it should be calling the function pthread_cond_init before free_context function call
I want you to fix that part.

next code

(void) pthread_mutex_init(&ctx->mutex, NULL);
(void) pthread_cond_init(&ctx->cond, NULL);
(void) pthread_cond_init(&ctx->sq_empty, NULL);
(void) pthread_cond_init(&ctx->sq_full, NULL);

Please call before move free_context(ctx)

Compilation issue with mingw-64 and USE_IPV6

Hi,

Changing the ifdef condition to enable the redefinition of "struct pollfd" makes the work.