#0  0x00000000008a8ee5 in event_active_nolock_ ()
#1  0x00000000008aeb49 in evmap_io_active_ ()
#2  0x00000000008b5388 in epoll_dispatch ()
#3  0x00000000008abd96 in event_base_loop ()
#4  0x00000000007eef9c in _evthr_loop ()
#5  0x0000003dfde079d1 in start_thread () from /lib64/libpthread.so.0
#6  0x0000003dfdae88fd in clone () from /lib64/libc.so.6

I suspect that it is the usleep trick to avoid thread concurrence problem.

I start 16 evhtp thread with evhtp_use_threads_wexit.

Details

Compile and start test_basic, start second instance of test_basic. Second instance will be ended with segfault in evhtp_unbind_socket. htp->server==NULL

Version

master

Details

memory leak

Steps or code to reproduce the problem.

code as flows:

 void
      reqHandler(evhtp_request_t * req, void * arg)
      {
          struct evbuffer * buffer    = evbuffer_new();
          char              buf[4096] = { 0 };
          int               i;
      
          evhtp_send_reply_chunk_start(req, EVHTP_RES_OK);
      
          for (i = 0; i < 10000; i++)
          {
              evbuffer_add(buffer, buf, sizeof(buf));
              evhtp_send_reply_chunk(req, buffer);
              evbuffer_drain(buffer, -1);
          }
      
          evhtp_send_reply_chunk_end(req);
      
          evbuffer_free(buffer);
      }
      
      int main(int argc,char **argv)
      {
              evbase_t          * evbase;
              evhtp_t           * evhtp;
      
              evbase            = event_base_new();
              assert(evbase);
              evhtp             = evhtp_new(evbase, NULL);
              assert(evhtp);
      
              evhtp_set_gencb(evhtp, reqHandler, NULL);
              int ret = evhtp_use_threads(evhtp, NULL, 64, NULL);
      
              ret = evhtp_bind_socket(evhtp, "0.0.0.0", 9010, 1024);
      
              ret = event_base_loop(evbase, 0);
      }

Version

1.2.12

Details

From looking at the code, HTTP keepalive should work automatically if supported by the client. It seems however that evhtp_connection_free is called from libevent before a response can be sent.
Running in gdb, c->request->flags seems to include EVHTP_REQ_FLAG_KEEPALIVE so I do not know what's wrong. I'm still searching.

Context: I'm working on fixing haiwen/seafile#1119

Steps or code to reproduce the problem.

Please note below that wget is trying to reuse the TCP connection and fails because of no data received.

$ ./examples/test_basic &
$ LANG=C wget http://localhost:8081/simple/ http://localhost:8081/simple/
--2017-09-25 12:45:42--  http://localhost:8081/simple/
Resolving localhost (localhost)... ::1, 127.0.0.1
Connecting to localhost (localhost)|::1|:8081... failed: Connection refused.
Connecting to localhost (localhost)|127.0.0.1|:8081... connected.
HTTP request sent, awaiting response... 200 OK
Length: 6 [text/plain]
Saving to: 'index.html'

index.html        100%[===================>]       6  --.-KB/s    in 0s

2017-09-25 12:45:42 (779 KB/s) - 'index.html' saved [6/6]

--2017-09-25 12:45:42--  http://localhost:8081/simple/
Reusing existing connection to localhost:8081.
HTTP request sent, awaiting response... No data received.
Retrying.

--2017-09-25 12:45:43--  (try: 2)  http://localhost:8081/simple/
Connecting to localhost (localhost)|127.0.0.1|:8081... connected.
HTTP request sent, awaiting response... 200 OK
Length: 6 [text/plain]
Saving to: 'index.html'

index.html.3        100%[===================>]       6  --.-KB/s    in 0s

2017-09-25 12:45:43 (548 KB/s) - 'index.html' saved [6/6]

FINISHED --2017-09-25 12:45:43--
Total wall clock time: 1.1s
Downloaded: 2 files, 12 in 0s (644 KB/s)

Version

1.2.12-1

Sorry if this has already been addressed. I noticed https://github.com/criticalstack/libevhtp/blob/70849e7a310718cdc2ece4793673435179d9d452/refcount.h#L81 unlocks the mutex and frees the data, but I do not see pthread_mutex_destroy anywhere. Is the mutex leaked?

Please, do not use - sign in version, it breaks rpm build.
Just use 1.12.1.1

Details

Hi,

After compile libevhtp with oniguruma package in ubuntu-14.04, call evhtp_set_regex_cb() function makes crash.

Steps or code to reproduce the problem.

• Install oniguruma package using apt-get
  -- $sudo apt-get install libonig-dev
• Install libevhtp
• compile sample code

Example code (if applicable)

#include <stdio.h>
#include <stdlib.h>
#include <evhtp.h>

void testcb(evhtp_request_t *req, void *a)
{
  const char* str;
  
  printf("Fired testcb.\n");
  str = a;
  
  evbuffer_add_printf(req->buffer_out, "%s", str);
  evhtp_send_reply(req, EVHTP_RES_OK);
}

int main(int argc, char** argv)
{
  evbase_t *evbase;
  evhtp_t *htp;
  
  evbase = event_base_new();
  htp = evhtp_new(evbase, NULL);
  
#ifndef EVHTP_DISABLE_REGEX
  printf("NOT defined EVHTP_DISABLE_REGEX.\n");
  
  evhtp_set_regex_cb(htp, "/simple1/*", testcb, "/simple/");
#endif
    
  evhtp_bind_socket(htp, "0.0.0.0", 8081, 1024);
  
  event_base_loop(evbase, 0);
  
  evhtp_unbind_socket(htp);
  evhtp_free(htp);
  event_base_free(evbase);
  
  return 0;
}

...

$ ./main &
[1] 23672
$ NOT defined EVHTP_DISABLE_REGEX.

$ curl -X GET localhost:8081/simple1/test
curl: (52) Empty reply from server
[1]+  Segmentation fault      (core dumped) ./main

Version

$ git branch -v

• develop d890899 added some new functions to examples/eutils

Details

libevent

/* Default values for max_single_read & max_single_write variables. */
#define MAX_SINGLE_READ_DEFAULT 16384
#define MAX_SINGLE_WRITE_DEFAULT 16384

http://www.wangafu.net/~nickm/libevent-book/Ref6a_advanced_bufferevents.html Limiting maximum single read/write size

Interface
int bufferevent_set_max_single_read(struct bufferevent *bev, size_t size);
int bufferevent_set_max_single_write(struct bufferevent *bev, size_t size);

ev_ssize_t bufferevent_get_max_single_read(struct bufferevent *bev);
ev_ssize_t bufferevent_get_max_single_write(struct bufferevent *bev);

Steps or code to reproduce the problem.

normal

evbuffer_add_reference(req->buffer_out, payload, 16000, NULL, NULL);

wrk -c64 -t4 -d5 --latency 'http://127.0.0.1:8080/test'
Running 5s test @ http://127.0.0.1:8080/test
4 threads and 64 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 664.27us 825.07us 19.51ms 93.01%
Req/Sec 20.70k 2.09k 29.85k 82.50%
Latency Distribution
50% 504.00us
75% 761.00us
90% 1.24ms
99% 4.27ms
412239 requests in 5.01s, 6.29GB read
Requests/sec: 82278.98
Transfer/sec: 1.26GB

throughput degradation

evbuffer_add_reference(req->buffer_out, payload, 16385, NULL, NULL);

wrk -c64 -t4 -d5 --latency 'http://127.0.0.1:8080/test'
Running 5s test @ http://127.0.0.1:8080/test
4 threads and 64 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 43.79ms 4.21ms 50.22ms 94.95%
Req/Sec 363.64 50.71 484.00 79.50%
Latency Distribution
50% 43.92ms
75% 44.19ms
90% 45.13ms
99% 48.62ms
7248 requests in 5.01s, 114.26MB read
Requests/sec: 1447.03
Transfer/sec: 22.81MB

Version

v1.2.11n

Details

I'm trying upgrade libevhtp on FreeBSD ports, but running into compilation errors that I can't quite figure out how to fix.

Sorry, forgot to provide full error:
upload-file.c:2373:23: error: incomplete definition of type 'struct evhtp_callback_s'
evhtp_set_hook(&cb->hooks, evhtp_hook_on_headers, upload_headers_cb, NULL);
~~^
/usr/local/include/evhtp/evhtp.h:38:8: note: forward declaration of 'struct evhtp_callback_s'
struct evhtp_callback_s;
^
[1] Is a full log of the build if interested.

All the evhtp_set_hook function calls return this error in the below Example code link.

[1] https://poudriere.ultimasbox.com/data/111amd64-test/2018-01-02_08h54m49s/logs/errors/seafile-server-6.2.3.log

Steps or code to reproduce the problem.

1. Install FreeBSD with the ports collection.
2. Patch libevhtp with [2].
3. cd /usr/ports/net-mgmt/seafile-server && make
4. Error mentioned x 11

[2] https://reviews.freebsd.org/D13742

Example code (if applicable)

https://github.com/haiwen/seafile-server/blob/20da7d028e9b80704f0d89c7e06b606d03200ef1/server/upload-file.c#L2342

Version

1.2.11 (works) --> 1.2.15 (breaks)

Special Thanks

Thanks for taking over this library. It definitely has many areas that need improvement but is promising. Using an out dated Oniguruma was driving me absolutely crazy! Thank you so much for moving this out of the source.

I had an issue with regexec() always returning mismatch on any regex I tried. Removing the oniguruma's regex lib from the build solved the issue.

Details

For example, ab (apache benchmarking tool) hangs in example_pause:

Steps to reproduce the problem.

[oleg@osboxes ~]$ build/examples/example_pause 
[INFO]  example_pause.c:108      response delayed for 10s: curl http://127.0.0.1:43129/

[oleg@osboxes ~]$ time curl http://127.0.0.1:43129/
time start 1522242044
time end 1522242054

real    0m10.016s
user    0m0.003s
sys     0m0.003s

[oleg@osboxes ~]$ time ab -n 1 http://127.0.0.1:43129/  
This is ApacheBench, Version 2.3 <$Revision: 1430300 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 127.0.0.1 (be patient)...apr_pollset_poll: The timeout specified has expired (70007)

real    0m40.047s
user    0m0.002s
sys     0m0.004s

Version

Latest version from develop branch.

1. coredump problem of short connection:

use libevhtp as http server, if request from client is a short connection(include "Connection":"close" header), i got a coredump at call htparser_get_error() in the _evhtp_connection_readcb() func.

2. reason of the coredump:

for the default callback registration in the _evhtp_connection_accept() func, when receive a short connection request, the action flow of server is:

• at first come into _evhtp_connection_readcb() func and get the request data.
• then call htparser_run() to verify and parse the request data. in the htparser_run(), after completed parse msg, it will callback the func(for example,the func name is: handle_cmd) that registered by evhtp_set_cb().
• in the handle_cmd func, after process the request body, in common we will write the response data(for example, call evhtp_send_reply() func) in the end.
• then _evhtp_connection_writecb that registered in the _evhtp_connection_accept() will be triggered by the write action.
• and in the _evhtp_connection_writecb func, if the request is a short connection, it will call evhtp_connection_free() to free the connection.
• after return from htparser_run func, the connection variable in the _evhtp_connection_readcb func is become invalid. so call htparser_get_error func get a coredump.

3. code modify:

static int _evhtp_connection_accept(...) {
	...
    bufferevent_enable(connection->bev, EV_READ);
    bufferevent_setcb(connection->bev,
                      _evhtp_connection_readcb,
                      //_evhtp_connection_writecb,       <-- after write response back to the accepted connection, don't call _evhtp_connection_writecb()
                      NULL,
                      _evhtp_connection_eventcb, connection);	
	
	...
}

static void _evhtp_connection_readcb(...) {
	...
    evbuffer_drain(bufferevent_get_input(bev), nread);

    if (c->request && c->request->status == EVHTP_RES_PAUSE) {
        evhtp_request_pause(c->request);
    } else if (htparser_get_error(c->parser) != htparse_error_none) {
        evhtp_connection_free(c);
    } else if (nread < avail) {
        // we still have more data to read (piped request probably)
        evhtp_connection_resume(c);
    } else {					         <-- add this else to deal with the connection.
        if(c->type == evhtp_type_server) {
            /*
            * if there is a set maximum number of keepalive requests configured, check
            * to make sure we are not over it. If we have gone over the max we set the
            * keepalive bit to 0, thus closing the connection.
            */
            if (c->htp->max_keepalive_requests) {
                if (++c->num_requests >= c->htp->max_keepalive_requests) {
                    c->request->keepalive = 0;
                }
            }

            if (c->request->keepalive == 1) {
                _evhtp_request_free(c->request);

                c->keepalive       = 1;
                c->request         = NULL;
                c->body_bytes_read = 0;

                if (c->htp->parent && c->vhost_via_sni == 0) {
                    /* this request was servied by a virtual host evhtp_t structure
                    * which was *NOT* found via SSL SNI lookup. In this case we want to
                    * reset our connections evhtp_t structure back to the original so
                    * that subsequent requests can have a different Host: header.
                    */
                    evhtp_t * orig_htp = c->htp->parent;

                    c->htp = orig_htp;
                }

                htparser_init(c->parser, htp_type_request);
                htparser_set_userdata(c->parser, c);
            } else {
                evhtp_connection_free(c);
            }
        }
    }
	...	
}

i have merged above changes in my project, it works correct now.
above modification is suitable or not, can you give some suggestions. Thanks in advance.

I'm sorry I can't find it.

htpparse_error
htparser_get_error(htparser * p) {
return p->error;
}

I have encountered this problem many times in this project,Not just this one.

Details

Steps or code to reproduce the problem.

1. The Array of string pointers errstr_map only have 13 item

2. But. The Enum type htpparse_error had defined 14 item

3. This code if (e > (htparse_error_generic + 1)) may cause value e large than that array errstr_map size.

Example code (if applicable)

const char * htparser_get_strerror(htparser * p)
{
    htpparse_error e = htparser_get_error(p);

    if (e > (htparse_error_generic + 1))
    {
        return "htparse_no_such_error";
    }

    return errstr_map[e];
}

Version

latest

Details

code as flows：

  void reqHandler(evhtp_request_t *req,void *arg)
{
        evhtp_send_reply_chunk_start(req, EVHTP_RES_OK);
        struct evbuf_t * buffer = evbuffer_new();
        assert(buffer);
        char buf[4096];
        int i;
       
        for(i=0; i < 10000; i++)
        {
                evbuffer_add(buffer, buf, 4096);
                evhtp_send_reply_chunk(req, buffer);
                evbuffer_drain(buffer, -1);
        }

        evhtp_send_reply(req, EVHTP_RES_OK);

        evbuffer_free(buffer);

}

int main(int argc,char **argv)
{
        evbase_t          * evbase;
        evhtp_t           * evhtp;

        evbase            = event_base_new();
        assert(evbase);
        evhtp             = evhtp_new(evbase, NULL);
        assert(evhtp);

        evhtp_set_gencb(evhtp, reqHandler, NULL);
        int ret = evhtp_use_threads(evhtp, NULL, 4, NULL);

        ret = evhtp_bind_socket(evhtp, "0.0.0.0", 9010, 1024);

        ret = event_base_loop(evbase, 0);
}

Steps or code to reproduce the problem.

 memory leak ，how to do ? who can help me,thanks

Version

libevhtp-1.2.12-1

Is it possible to get the http return code from an evhtp_hook_request_fini_cb? E.g., the second parameter I supply to evhtp_send_reply().

evhtp_bind_sockaddr() does not close created socket if bind() fails:
https://github.com/criticalstack/libevhtp/blob/develop/evhtp.c#L3668

if evhtp_accept_socket() fails for some reason, socked is leaked as well:
https://github.com/criticalstack/libevhtp/blob/develop/evhtp.c#L3673

There is no real regulation other than take_ownership that allows for a user to specify who owns what (e.g., who opens and closes a file descriptor).

See initial discussion: #6

Cascading the structures and adding a flag would be optimal:

struct socket {
   int flags; (EVHTP_SOCK_OWNER)
   int sock;
};

struct evhtp {
    int flags; (EVHTP_FLAG_OWNER)
    struct socket socket;
     ...
};

struct connection {
   int flags; (EVHTP_CONN_FLAG_OWNER)
   ...
};

struct request {
   int flags; (EVHTP_REQ_FLAG_OWNER)
   ...
};

Now that the parser war is over, let's figure out the best plan of action to replace htparser with httparser.

Details

Steps or code to reproduce the problem.

1. setup and run server using SSL
2. make multiple concurrent connections to the server
3. observe segfault in parser.c - htparser_get_error

In SSL mode (I haven't observed the segfault using plain http, at least) the call to htparser_run at https://github.com/criticalstack/libevhtp/blob/develop/evhtp.c#L2100 can potentially free the request and apparently also c->parser, so that the call to htparser_get_error in line 2139 leads to a segfault. The case that c->parser is NULL after line 2100 should be handled.

Version

very recent develop branch (commit 2307737 Nov 3)

Details

When libevhtp included in another project using add_subdirectory(libevhtp), CMAKE_SOURCE_DIR points to source directory of main project, not to source directory of libevhtp. To fix this problem replace with CMAKE_SOURCE_DIR with CMAKE_CURRENT_SOURCE_DIR

Steps or code to reproduce the problem.

Version

master and last tag

When I see the number exit_cb, I'm confused. Because the function type named evthr_init_cb
https://github.com/criticalstack/libevhtp/blob/77f20e606e01666a8b121c2ef4c09022456cdf50/thread.c#L48

And I find the type define of callback function.
https://github.com/criticalstack/libevhtp/blob/0da6bc9d3124e225d67f5d678f1a2961987757c1/include/evhtp/thread.h#L36

It helps in maintaining distro packages
Thanks!

Running make install after compiling places headers in /usr/local/evhtp/ dir (under ubuntu). Using those global headers causes compilation to fail because evhtp.h includes files assuming they are installed in /usr/local/ dir instead.

SECURITY UPDATE

@flokli emailed me today informing me that the pre-packaged version of onig is vulnerable to CVE-2017-9224

We will be removing this and be making it a third-party dependency from now until forever.

UNTIL THEN: libevhtp will use the global version of onig if it is found on the system (thus not compiling the pre-packaged version). So.. DO THIS!

Details

evhtp_hook_on_conn_error is never used. A quick search of the repo only shows this hook being set here evhtp.c:4325
evhtp_hook_on_error isn't used if only set in struct evhtp_connection_s. It must be explicitly set on every request struct. evhtp.c:758

Partial Workaround

Use evhtp_hook_on_path to set it on the evhtp_request_t with evhtp_request_set_hook(). However on_error will only be called if the connection get's to this callback. For Example

perl -e '$|=1; print "GET /file00 HTTP/1.1\r\n"; while(<>){}' | nc localhost 8080

$ ./server 8080
error_cb

Steps or code to reproduce the problem.

1. Set a short timeout, I used 1 second
2. Set hooks on_conn_error and on_error
3. connect to server with empty or partial request: ex nc localhost 8080
4. Wait for timeout to close connection and you won't see logging from either error callbacks.

Example code (if applicable)

evhtp_res 
conn_error_cb(evhtp_connection_t *conn, evhtp_error_flags type, void *arg)
{
    printf("%s\n", __FUNCTION__);
    return EVHTP_RES_OK;
}

evhtp_res 
error_cb(evhtp_request_t *req, evhtp_error_flags type, void *arg)
{
    printf("%s\n", __FUNCTION__);
    return EVHTP_RES_OK;
}

evhtp_res 
pre_accept_cb(evhtp_connection_t *conn, void *arg)
{
    evhtp_connection_set_hook(conn, evhtp_hook_on_conn_error, &conn_error_cb, NULL);
    evhtp_connection_set_hook(conn, evhtp_hook_on_error, &error_cb, NULL);
    return EVHTP_RES_OK;
}

int
main(int argc, const char ** argv)
{
    struct timeval = { .tv_sec = 1, .tv_usec = 0 };
    ...
    evhtp_set_timeouts(htp, &to, &to);
    evhtp_set_pre_accept_callback(htp, &pre_accept_cb, NULL);
    ...
}

Version

1.2.16

This happens when the callbacks object is NULL. Reason is evhtp_set_cb() not being called beforehand.

There should be a way to disable the use of scratch_buf within the evhtp_connection_s structure.

Details

The following program requires a large amount of memory during running。

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
2315 root 20 0 4176m 3.4g 1564 S 18.3 92.1 0:04.76 test-chunk

Steps or code to reproduce the problem.

Example code (if applicable)

#include <evhtp.h>

void
reqHandler(evhtp_request_t * req, void * arg)
{

    struct evbuffer * buffer    = evbuffer_new();
    char              buf[4096] = { 0 };
    int               i;

    evhtp_send_reply_chunk_start(req, EVHTP_RES_OK);

    for (i = 0; i < 500000; i++)
    {
          evbuffer_add(buffer, buf, sizeof(buf));
          evhtp_send_reply_chunk(req, buffer);
          evbuffer_drain(buffer, -1);
    }

    printf("begin.....\n");
    sleep(10);
    evhtp_send_reply_chunk_end(req);
    printf("end .....\n");
    evbuffer_free(buffer);
}

int main(int argc,char **argv)
{
        evbase_t          * evbase;
        evhtp_t           * evhtp;

        evbase            = event_base_new();
        assert(evbase);
        evhtp             = evhtp_new(evbase, NULL);
        assert(evhtp);

        evhtp_set_gencb(evhtp, reqHandler, NULL);
        int ret = evhtp_use_threads(evhtp, NULL, 4, NULL);

        ret = evhtp_bind_socket(evhtp, "0.0.0.0", 9010, 1024);

        ret = event_base_loop(evbase, 0);
}

Version

1.2.12-1

https://github.com/criticalstack/libevhtp/blob/12ed7e8c4a6d7cfec24c5c9f1036bff9049d7a4c/evhtp.c#L2165

if I comment out the assert, everything appears to work fine

Details

evhtp_hook_on_request_fini called twice on second and next requests. This

Steps or code to reproduce the problem.

add printf("fini\n"); to test_fini function in test.c file. Compile. Run.
make first request to localhost/bar using Chrome. One line with fini will be printed. Make another request. Two lines with "fini" will be printed.
I do some investigation and found that this issue possibly related to keepalive. Looks like evhtp closes connection after receiving request and browser send request again.

Version

master, Centos 7.3

Details

Starting program: /work/github/libevhtp-1.2.12/build/examples/test 

[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
*** Error in `/work/github/libevhtp-1.2.12/build/examples/test': double free or corruption (out): 0x00007fffffffd6a0 ***

Program received signal SIGABRT, Aborted.
0x00007ffff6da4cc9 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
56 ../nptl/sysdeps/unix/sysv/linux/raise.c: 没有那个文件或目录.
(gdb) bt
#0 0x00007ffff6da4cc9 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
#1 0x00007ffff6da80d8 in __GI_abort () at abort.c:89
#2 0x00007ffff6de1394 in libc_message (do_abort=do_abort@entry=1, fmt=fmt@entry=0x7ffff6eefb28 "*** Error in `%s': %s: 0x%s ***\n") at ../sysdeps/posix/libc_fatal.c:175
#3 0x00007ffff6ded66e in malloc_printerr (ptr=, str=0x7ffff6eefc58 "double free or corruption (out)", action=1) at malloc.c:4996
#4 int_free (av=, p=, have_lock=0) at malloc.c:3840
#5 0x0000000000406ce4 in htp__free (ptr=0x7fffffffd6a0) at /work/github/libevhtp-1.2.12/evhtp.c:173
#6 0x000000000040c1e3 in evhtp_kv_free (kv=0x6847b0) at /work/github/libevhtp-1.2.12/evhtp.c:2968
#7 0x000000000040c327 in evhtp_kvs_free (kvs=0x6835a0) at /work/github/libevhtp-1.2.12/evhtp.c:3007
#8 0x0000000000408611 in htp__request_free (request=0x6833c0) at /work/github/libevhtp-1.2.12/evhtp.c:1091
#9 0x000000000040aae6 in htp__connection_writecb (bev=0x682a90, arg=0x680890) at /work/github/libevhtp-1.2.12/evhtp.c:2201
#10 0x00007ffff7bad0df in bufferevent_writecb (fd=, event=, arg=0x682a90) at bufferevent_sock.c:297
#11 0x00007ffff7ba2a10 in event_persist_closure (ev=, base=0x67f040) at event.c:1319
#12 event_process_active_single_queue (activeq=0x67f480, base=0x67f040) at event.c:1363
#13 event_process_active (base=) at event.c:1438
#14 event_base_loop (base=0x67f040, flags=0) at event.c:1639
#15 0x0000000000406c3e in main (argc=1, argv=0x7fffffffe408) at /work/github/libevhtp-1.2.12/examples/test.c:663
(gdb)

Steps or code to reproduce the problem.

examples/test 

Version

libevhtp-1.2.12
libevent-2.0.2

Details

When I tried to run test_client example I got following assertion error

Assertion failed: c && c->htp && c->request && c->parser && bev (htp__connection_writecb_: libevhtp/evhtp.c:2165)

Seems the reason of this is that a client connection is every time created with evhtp_t equal to NULL, like on line 5173 in evhtp.c

  if (!(conn = htp__connection_new_(NULL, -1, evhtp_type_client)))
  {
      return NULL;
  }

And there is an assertion in evhtp.c on line 2165

evhtp_assert(c && c->htp && c->request && c->parser && bev);

So with client request c->htp is every time NULL and assertion fails.

Steps or code to reproduce the problem.

Simply run test_client.c

Version

Latest libevhtp version.

Currently, libevhtp is using SSL_CTX_use_certificate_file to load a certificate file. That function lacks the ability to load the pinned certificate chain (if any) which has a consequence of connecting clients not trusting the received certificate. By using SSL_CTX_use_certificate_chain_file we give the libssl the ability to read and send the entire certificate chain (if any), which clients can check against.

[removed]

Although there had judged upper limit
https://github.com/criticalstack/libevhtp/blob/bc52552641ef9d01713c6d8e9b41aa1c089091fe/parser.c#L728

But, if p->buf_idx arrival at (PARSER_STACK_MAX-1) value, The following will using two characters of space, then have a out of bounds memory write.
https://github.com/criticalstack/libevhtp/blob/bc52552641ef9d01713c6d8e9b41aa1c089091fe/parser.c#L776-L777

Proposed using if (p->buf_idx >= PARSER_STACK_MAX-1) replace.

Details

evhtp_request_pause will freeze current connections from sending data. I don't entirely understand how evhtp works, so this could be a problem in the seafile code and not evhtp, but I'm not positive.

In version 1.2.9, [1] changed to [2] and in every version after it. This caused the main issue that I am experiencing and have reverted the change as shown at [3]. My guess is that EV_WRITE isn't being re enabled when it should. It is very difficult to debug the seafile code though because there isn't any debugging code in it. If you have a suggestion of how to debug the library I am open to suggestions.

[4] [5] is the seafile code using this function, I'm fairly certain [4] is where the bug is occurring. Also want to mention that [6] is a patch I am using that the previous libevhtp developer suggested, which didn't fix the issue unfortunately.

[1] https://github.com/criticalstack/libevhtp/blob/1.2.9/evhtp.c#L1984
[2] https://github.com/criticalstack/libevhtp/blob/1.2.13/evhtp.c#L2865
[3] https://github.com/freebsd/freebsd-ports/blob/branches/2018Q1/www/libevhtp/files/patch-evhtp.c
[4] https://github.com/haiwen/seafile-server/blob/v6.2.3-server/server/access-file.c
[5] https://github.com/haiwen/seafile-server/blob/v6.2.3-server/server/upload-file.c
[6] https://github.com/freebsd/freebsd-ports/blob/branches/2018Q1/net-mgmt/seafile-server/files/patch-server_access-file.c

Steps or code to reproduce the problem.

1. Use [7] to patch libevhtp in FreeBSD ports
2. modify version run and make makesum
3. compile libevhtp and seafile-server, install seahub.
4. Attempt to upload files / download files. Will fail with, Error 404. [8] is curl -v output.

[7] https://reviews.freebsd.org/D13742
[8] curl -v "https://OBSCURED/seafhttp/files/c6629c8c-5324-OBSCURED-a9a46bf4aaa8/OBSCURED.rar"

• Trying 2601:646:OBSCURED:feed:c6a6...
• TCP_NODELAY set
• Connected to seafile.ultimasbox.com (2601:646:OBSCURED:feed:c6a6) port 443 (#0)
• ALPN, offering h2
• ALPN, offering http/1.1
• successfully set certificate verify locations:
• CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: none
• TLSv1.2 (OUT), TLS handshake, Client hello (1):
• TLSv1.2 (IN), TLS handshake, Server hello (2):
• TLSv1.2 (IN), TLS handshake, Certificate (11):
• TLSv1.2 (IN), TLS handshake, Server key exchange (12):
• TLSv1.2 (IN), TLS handshake, Server finished (14):
• TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
• TLSv1.2 (OUT), TLS change cipher, Client hello (1):
• TLSv1.2 (OUT), TLS handshake, Finished (20):
• TLSv1.2 (IN), TLS handshake, Finished (20):
• SSL connection using TLSv1.2 / ECDHE-RSA-CHACHA20-POLY1305
• ALPN, server accepted to use h2
• Server certificate:
• subject: CN=OBSCURED
• start date: Oct 30 07:28:47 2017 GMT
• expire date: Jan 28 07:28:47 2018 GMT
• subjectAltName: host "OBSCURED" matched cert's "OBSCURED"
• issuer: C=US; O=Let's Encrypt; CN=Let's Encrypt Authority X3
• SSL certificate verify ok.
• Using HTTP2, server supports multi-use
• Connection state changed (HTTP/2 confirmed)
• Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
• Using Stream ID: 1 (easy handle 0x5585f5bf7090)

GET /seafhttp/files/c6629c8c-5324-OBSCURED-a9a46bf4aaa8/OBSCURED.rar HTTP/2
Host: OBSCURED
User-Agent: curl/7.57.0
Accept: /

• Connection state changed (MAX_CONCURRENT_STREAMS updated)!
  < HTTP/2 404
  < server: nginx
  < date: Wed, 03 Jan 2018 22:09:47 GMT
  < content-type: text/plain
  < content-length: 0
  <
• Connection #0 to host OBSCURED left intact

Version

1.2.9 (works) --> 1.2.10 (Fails), 1.2.11 (Fails), 1.2.13 (Fails)
1.2.12 Also seems to be not work. Removing EV_WRITE on the line mentioned in [2] will cause everything to work again as it did in 1.2.9.

Details

I am profiling my application, which implements a REST API on top of libevhtp. And I'm finding htparser_run using most of the processing time. Here's an example gprof output:

Flat profile:

Each sample counts as 0.01 seconds.
  %   cumulative   self              self     total
 time   seconds   seconds    calls  ms/call  ms/call  name
 17.87      0.52     0.52                             htparser_run
  6.53      0.71     0.19                             fe_contact_api_read
  4.30      0.84     0.13                             api_v1_cmp_endpoint
  3.78      0.95     0.11  8600729     0.00     0.00  evbuffer_add
  2.75      1.03     0.08                             evhtp_kv_new
  2.41      1.10     0.07                             htp__request_parse_header_key_
  2.06      1.16     0.06   600124     0.00     0.00  evmap_io_add
  1.72      1.21     0.05 10001571     0.00     0.00  evbuffer_invoke_callbacks
  1.37      1.25     0.04   200013     0.00     0.00  rest_api_v1_cb
  1.37      1.29     0.04    30994     0.00     0.01  event_process_active_single_queue
  1.37      1.33     0.04                             htp__create_reply_
  1.37      1.37     0.04                             htp__malloc_
  1.03      1.40     0.03  2000130     0.00     0.00  evbuffer_expand_singlechain
  1.03      1.43     0.03   415868     0.00     0.00  evmap_io_active
  1.03      1.46     0.03   200349     0.00     0.00  _evbuffer_expand_fast
  1.03      1.49     0.03       35     0.86     0.86  evmap_io_init
  1.03      1.52     0.03                             evhtp_kvs_free
  1.03      1.55     0.03                             fe_get_1st_phoneline_by_contact
  1.03      1.58     0.03                             htp__create_headers_
  1.03      1.61     0.03                             htp__free_
  1.03      1.64     0.03                             htp__protocol_
  1.03      1.67     0.03                             htp__request_new_
  1.03      1.70     0.03                             htp__request_parse_header_val_
  1.03      1.73     0.03                             htp__request_parse_path_
  1.03      1.76     0.03                             htp__should_parse_query_body_
  0.69      1.78     0.02  5200338     0.00     0.00  client_matches
  0.69      1.80     0.02  2000130     0.00     0.00  evbuffer_expand
  0.69      1.82     0.02  1029358     0.00     0.00  event_queue_insert
  0.69      1.84     0.02  1029351     0.00     0.00  event_queue_remove
  0.69      1.86     0.02   606808     0.00     0.00  event_add_internal
  0.69      1.88     0.02   600119     0.00     0.00  evmap_io_del
  0.69      1.90     0.02   400441     0.00     0.00  evbuffer_unfreeze
  0.69      1.92     0.02   200014     0.00     0.00  bufferevent_init_common
  0.69      1.94     0.02   200013     0.00     0.00  COPY_CHAIN
  0.69      1.96     0.02                             contact_record_check
  0.69      1.98     0.02                             event_get_method

I haven't done a lot of profiling work (it's been years..). Now the request headers aren't too complicated, and it's not SSL (handled elsewhere).

My app basically does some lookups in hash tables, builds JSON and returns it. Granted, that's not much, but I'm surprised by htparser_run being at the top of the list.

Here's a sample request:

GET /v1/contact HTTP/1.0
Connection: Keep-Alive
accept: application/json
content-type: application/json
Authorization: Bearer ekCM83ry3hmevxGiahoXTVhS1hEXkK9pGnfKIT1iGb
Host: nvast2:5657
User-Agent: ApacheBench/2.3

BTW, I came here because I had the same problem with the original libevhtp, which I incorporated into my source tree 1.5 year ago.

Steps or code to reproduce the problem.

Don't know how this applies here

Version

Git clone as of today. Version says v1.2.12

Details

1. For unknown reason, bufferevent_socket_connect() calling in the evhtp_connection_ssl_new()@evhtp.c returns a unzero value, so evhtp_assert(rc == 0) failed.
2. in the evhtp_assert(), abort() func is called, so the program is exit.

My understanding of libevhtp:

libevhtp is common http(1.1) adaptor for the programs which use it. and for some unknown reason, the program exit from libevhtp is not gracefullly.

workaround of program abort

• libevhtp only output the error info, don't exit.
• and returns the error to the caller.

evhtp_connection_ssl_new() {
    ...
    rc = bufferevent_socket_connect(conn->bev,
                                    (struct sockaddr *)&sin, sizeof(sin));

   // the following code is workaround
   if(rc !=0 ) {
          // log error info
          // free conn
          // set conn to NULL
   }
    return conn;
}

Version

the newest libevhtp

@NathanFrench
how about the above case, any suggestions is welcome, thank you in advance.

Details

Steps or code to reproduce the problem.

evhtp_set_gencb don't work. Instead first evhtp_set_cb callback called.

• Compile code
• request http://localhost:8081/
• You will get "simple" instead of "root"
• Same result if you request http://localhost:8081/some_random_text_jfjfjfj

Example code (if applicable)

This is modified version of test_basic.c

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <stdint.h>
#include <errno.h>
#include <evhtp.h>

void
testcb(evhtp_request_t * req, void * a) {
    const char * str = a;

    evbuffer_add(req->buffer_out, str, strlen(str));
    evhtp_send_reply(req, EVHTP_RES_OK);
}

void
issue161cb(evhtp_request_t * req, void * a) {
    struct evbuffer * b = evbuffer_new();

    if (evhtp_request_get_proto(req) == EVHTP_PROTO_10) {
        evhtp_request_set_keepalive(req, 0);
    }

    evhtp_send_reply_start(req, EVHTP_RES_OK);

    evbuffer_add(b, "foo", 3);
    evhtp_send_reply_body(req, b);

    evbuffer_add(b, "bar\n\n", 5);
    evhtp_send_reply_body(req, b);

    evhtp_send_reply_end(req);

    evbuffer_free(b);
}

int
main(int argc, char ** argv) {
    evbase_t * evbase = event_base_new();
    evhtp_t  * htp    = evhtp_new(evbase, NULL);

    evhtp_set_cb(htp, "/simple/", testcb, "simple");
    evhtp_set_cb(htp, "/1/ping", testcb, "one");
    evhtp_set_cb(htp, "/1/ping.json", testcb, "two");
    evhtp_set_cb(htp, "/issue161", issue161cb, NULL);
// VVVVVVVVVVVVVVVVVVVVVVVVV
    evhtp_set_gencb(htp, testcb, "root");
// ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
#ifndef EVHTP_DISABLE_EVTHR
    evhtp_use_threads_wexit(htp, NULL, NULL, 8, NULL);
#endif
    evhtp_bind_socket(htp, "0.0.0.0", 8081, 2048);

    event_base_loop(evbase, 0);

    evhtp_unbind_socket(htp);
    evhtp_free(htp);
    event_base_free(evbase);

    return 0;
}

Version

Latest

• all static functions should return int, nothing else (makes error handling easier)
• structures will no longer be exposed. (**create a evhtp-compat header to include until fully deprecated)
• break up tiers base -> server -> connection -> request

Details

In case when evhtp_header_val_add() is called with val="static_string" and val_alloc is greater than 1 evhtp_header_t will contain non-heap value that evhtp_kv_free() will attempt to free.
This also applies to key and key_alloc parameters of evhtp_kv_new() function.

Steps or code to reproduce the problem.

Create htp_header_t instance with evhtp_kv_new with static string as key and key_alloc > 1 than attepmpt to call evhtp_kv_free.

Example code (if applicable)

#include <evhtp.h>

int
main(int argc, char ** argv)
{
    evhtp_header_t * header = evhtp_header_new("Connection", "Keep-Alive", 2, 2);
    evhtp_kv_free(header);
    return 0;
}

Version

1.2.16

I initially tried to implement chunked encoding using libevent but turned to this library when that didn't work out. Unfortunately, the same issue seems to persist here. I've been trying to send large amounts of data over a single HTTP request, but the amount is too large for one evhtp_send_reply. So, I turned to chunking, which seems like it should have been the perfect solution--it ostensibly sends the content in pieces so that a single buffer won't become overfilled (which causes the crashing on my machine). And yet, it doesn't work at all.

I stepped through the code using GDB (libevhtp's own test_chunking test) and found that evhtp_send_reply_chunk merely pushes the overflowing buffer back a step. The content is progressively added to req->conn->bev->output (where req is the evhtp_request_t) with each chunk call. This buffer grows until evhtp_send_reply_chunk_end flushes its content into the network. Nothing is sent over the network until all the chunks have been loaded into this intermediary buffer, a confounding discovery that seems to eliminate all utility of this encoding method. (The bufferevent_flush call at the end of chunk seems to do nothing at all--digging down, it ultimately calls libevent's be_socket_flush function, which just returns 0 because of course.) I furthermore did a practical test in which I inserted sleep calls in between the chunk calls and, indeed, the server simply hanged until all the sleep and chunk calls had gone through. If chunking were working properly, pieces of the request should have been sent incrementally.

Am I missing something? It just doesn't seem like chunking is working at all like it's supposed to. Any help would be greatly appreciated.

1. feature background

As a https server, for better security consideration, user may not keep the private key(for example: named as tls.key) in a clear text. it may use some encryption algo to encrypt the tls.key for protecting from server inbreak.
so when before use tls.key to create a https connection, user first decrypt the tls.key.

2. current implementation

now, in the evhtp_ssl_init(), cfg->privfile is used directly. this means that user should use a unencrypted privfile. user can not use a encrypted privfile.

    SSL_CTX_use_PrivateKey_file(htp->ssl_ctx,
                                cfg->privfile ? cfg->privfile : cfg->pemfile, SSL_FILETYPE_PEM);

3. solution

i'd like add a member callback (for example: decrypt_privfile_cb) in the evhtp_ssl_cfg_t struct. if this callback is set, it will be called to decrypt the privfile and used by SSL_CTX_use_PrivateKey().

@NathanFrench if this feature is welcome, i can make a commit. Thanks in Advance.

Build with SSL disabled
Execute test_basic - segfault.
With enables santizer next report generated

ASAN:DEADLYSIGNAL
==14636==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000008 (pc 0x7ff95654f191 bp 0x7ffcb1ccc600 sp 0x7ffcb1ccc5e0 T0)
#0 0x7ff95654f190 in evconnlistener_free (/lib64/libevent-2.0.so.5+0x1d190)
#1 0x4125a6 in evhtp_unbind_socket /home/vromanov/work/libevhtp/evhtp.c:3631
#2 0x404854 in main /home/vromanov/work/libevhtp/examples/test_basic.c:53
#3 0x7ff955967b34 in __libc_start_main (/lib64/libc.so.6+0x21b34)
#4 0x404568 (/home/vromanov/work/libevhtp/cmake-build-debug/examples/test_basic+0x404568)

AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: SEGV (/lib64/libevent-2.0.so.5+0x1d190) in evconnlistener_free
==14636==ABORTING

master branch, Centos 7.3

There was this exchange in the issues of elizy's repo that I would always keep going back to, and which never formally made it into the documentation, perhaps it is worth adding to the documentation, now that those old issues are now gone?

I have a question : why need pause and resume pairs.

This is actually a good question.

Pause and resume will essentially stop and start attempting to read from the client socket. But also has some extra logic around it. Since the parser is a state machine, you can get functions executed as data is passed to it. But what if you wanted to temporarily stop that input processing? Well you have to save the state on the current input, then turn off the reading.

You can do all the normal things you can do while the request is paused, add output data, headers, etc. But it isn't until you use the resume function will these actions actually happen on the socket. When resume is called, it does an event_active(), which in turn goes back to the original reader function.

For example, with my companies reverse proxy we wrote (https://github.com/mandiant/RProxy) we don't immediately send the request to a backend, they get queued up and only processed once a backend connection becomes available. If it was not for the pause functionality, if a connection started posting a crap-ton of data, it would buffer and eat up all the memory.

This source comment actually does a good job of describing: https://github.com/mandiant/RProxy/blob/master/src/rproxy.c#L1020