Code Monkey home page Code Monkey logo

goldy-client's Issues

Goldy segmentation faults and malloc assertion failures

I have been testing Goldy Client as an option to fronting Aeron, which uses UDP to transport and reassemble user messages. Configuration:

./goldy -l 127.0.0.1:60123 -b 127.0.0.1:40123 -c localhost.crt -k localhost.key --log=INFO
./goldy-client -l 127.0.0.1:50123 -b 127.0.0.1:60123 -c rootCA.crt --log=INFO

Simple (low volume) tests worked fine, but as I stressed the system Aeron started receiving corrupt UDP messages via the Goldy tunnel.

In an effort to verify correct operation of individual components, I used nc -ul 40123 > file to capture a file sent through Goldy with cat file | nc -u 127.0.0.1 50123. If this file was small, the shasum proved correct delivery. However larger files resulted in either corrupt delivery, Goldy Client segmentation faults, or Goldy Client assertion errors.

It is possible to produce the following segmentation fault using openssl rand 4027 |nc -u 127.0.0.1 50123. A random size of 4026 does not produce the fault.

$./goldy-client -l 127.0.0.1:50123 -b 127.0.0.1:60123 -c rootCA.crt --log=DEBUG
2017-03-15 14:17:17.266015 INFO  Goldy 0.2 starting up
2017-03-15 14:17:17.266154 DEBUG Binded UDP 127.0.0.1:50123
2017-03-15 14:17:17.266232 DEBUG Loaded server cacert file
2017-03-15 14:17:17.266271 DEBUG Seeded random number generator
2017-03-15 14:17:17.266300 INFO  Proxy is ready, listening for connections on UDP 127.0.0.1:50123
2017-03-15 14:17:17.266328 INFO  main_loop - start
2017-03-15 14:17:17.266335 DEBUG start_listen_io - 3
2017-03-15 14:17:19.612061 DEBUG global_cb fds: 3,3 revents: 0x01 count: 0
2017-03-15 14:17:19.612149 DEBUG connect_to_new_client: connected on fd 6
2017-03-15 14:17:19.612206 INFO  (127.0.0.1:41015) Client connected
*** Error in `./goldy-client': free(): invalid next size (fast): 0x0000000001fca6e0 ***
======= Backtrace: =========
/usr/lib/libc.so.6(+0x70c4b)[0x7f13430a7c4b]
/usr/lib/libc.so.6(+0x76fe6)[0x7f13430adfe6]
/usr/lib/libc.so.6(+0x777de)[0x7f13430ae7de]
/usr/lib/libc.so.6(freeaddrinfo+0x28)[0x7f134310c048]
./goldy-client[0x407288]
./goldy-client[0x404986]
./goldy-client[0x404a29]
./goldy-client[0x405da2]
./goldy-client[0x4488a3]
./goldy-client[0x44ca21]
./goldy-client[0x403886]
./goldy-client[0x405e72]
./goldy-client[0x405f34]
/usr/lib/libc.so.6(__libc_start_main+0xf1)[0x7f1343057291]
./goldy-client[0x40379a]
======= Memory map: ========
00400000-00477000 r-xp 00000000 09:00 530546                             /home/bpa/projects/goldy-client/goldy-client
00676000-00677000 r--p 00076000 09:00 530546                             /home/bpa/projects/goldy-client/goldy-client
00677000-00678000 rw-p 00077000 09:00 530546                             /home/bpa/projects/goldy-client/goldy-client
00678000-0067b000 rw-p 00000000 00:00 0 
01fbf000-01fe0000 rw-p 00000000 00:00 0                                  [heap]
7f133c000000-7f133c021000 rw-p 00000000 00:00 0 
7f133c021000-7f1340000000 ---p 00000000 00:00 0 
7f1342e20000-7f1342e36000 r-xp 00000000 09:00 1314335                    /usr/lib/libgcc_s.so.1
7f1342e36000-7f1343035000 ---p 00016000 09:00 1314335                    /usr/lib/libgcc_s.so.1
7f1343035000-7f1343036000 r--p 00015000 09:00 1314335                    /usr/lib/libgcc_s.so.1
7f1343036000-7f1343037000 rw-p 00016000 09:00 1314335                    /usr/lib/libgcc_s.so.1
7f1343037000-7f13431cc000 r-xp 00000000 09:00 1313965                    /usr/lib/libc-2.24.so
7f13431cc000-7f13433cb000 ---p 00195000 09:00 1313965                    /usr/lib/libc-2.24.so
7f13433cb000-7f13433cf000 r--p 00194000 09:00 1313965                    /usr/lib/libc-2.24.so
7f13433cf000-7f13433d1000 rw-p 00198000 09:00 1313965                    /usr/lib/libc-2.24.so
7f13433d1000-7f13433d5000 rw-p 00000000 00:00 0 
7f13433d5000-7f13434d8000 r-xp 00000000 09:00 1314023                    /usr/lib/libm-2.24.so
7f13434d8000-7f13436d7000 ---p 00103000 09:00 1314023                    /usr/lib/libm-2.24.so
7f13436d7000-7f13436d8000 r--p 00102000 09:00 1314023                    /usr/lib/libm-2.24.so
7f13436d8000-7f13436d9000 rw-p 00103000 09:00 1314023                    /usr/lib/libm-2.24.so
7f13436d9000-7f13436fc000 r-xp 00000000 09:00 1313964                    /usr/lib/ld-2.24.so
7f13438d5000-7f13438d9000 rw-p 00000000 00:00 0 
7f13438fa000-7f13438fb000 rw-p 00000000 00:00 0 
7f13438fb000-7f13438fc000 r--p 00022000 09:00 1313964                    /usr/lib/ld-2.24.so
7f13438fc000-7f13438fd000 rw-p 00023000 09:00 1313964                    /usr/lib/ld-2.24.so
7f13438fd000-7f13438fe000 rw-p 00000000 00:00 0 
7ffdc1798000-7ffdc17b9000 rw-p 00000000 00:00 0                          [stack]
7ffdc17c8000-7ffdc17ca000 r--p 00000000 00:00 0                          [vvar]
7ffdc17ca000-7ffdc17cc000 r-xp 00000000 00:00 0                          [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0                  [vsyscall]
Aborted (core dumped)

In addition, running openssl rand 4026 |nc -u 127.0.0.1 50123 with varying delays between each invocation will occasionally lead to:

2017-03-15 14:00:38.066473 INFO  Created socket to backend UDP 127.0.0.1:60123
2017-03-15 14:00:38.388375 INFO  (127.0.0.1:58484) Session closed
2017-03-15 14:00:38.388453 INFO  (127.0.0.1:52739) Client connected
2017-03-15 14:00:38.388483 INFO  Created socket to backend UDP 127.0.0.1:60123
2017-03-15 14:00:41.828988 INFO  (127.0.0.1:52739) Session closed
2017-03-15 14:00:41.829031 INFO  (127.0.0.1:53196) Client connected
2017-03-15 14:00:41.829047 INFO  Created socket to backend UDP 127.0.0.1:60123
goldy-client: malloc.c:2403: sysmalloc: Assertion `(old_top == initial_top (av) && old_size == 0) || ((unsigned long) (old_size) >= MINSIZE && prev_inuse (old_top) && ((unsigned long) old_end & (pagesize - 1)) == 0)' failed.

The above was compiled using Goldy Client 4f70c9f.

No data proxied by goldy when using goldy-client to send the DTLS

This is running on Ubuntu, compiling went fine and everything appears to run correctly until it gets to the data transfer part.

I generated a self signed certificate just for testing using: openssl req -new -newkey rsa:4096 -days 365 -nodes -x509 -subj "/C=US/ST=Denial/L=Springfield/O=Dis/CN=localhost" -keyout localhost.key -out localhost.cert

goldy is running with: goldy -l localhost:60000 -b 127.0.0.1:20000 -c localhost.cert -k localhost.key --log=DEBUG

goldy-client is running with: goldy-client -l 127.0.0.1:5000 -b localhost:60000 -c ~/localhost.cert --log=DEBUG

Test server to print UDP packets after goldy decrypts them: netcat -ul 20000
Send a UDP packet to goldy-client so it can encrypt it: echo "test" | nc -w1 -u localhost 5000 and send DTLS to goldy on port 60000

When goldy-client gets traffic the debug log looks like this:

2016-06-25 11:33:55.250506 INFO  Created socket to backend UDP localhost:60000
2016-06-25 11:33:55.250537 DEBUG global_cb - session_start - client_fd 6
2016-06-25 11:33:55.250563 DEBUG mbedtls_debug [2] ssl_tls.c:6302: => handshake
2016-06-25 11:33:55.250587 DEBUG mbedtls_debug [2] ssl_cli.c:3264: client state: 0
2016-06-25 11:33:55.250604 DEBUG mbedtls_debug [2] ssl_tls.c:2429: => flush output
2016-06-25 11:33:55.250619 DEBUG mbedtls_debug [2] ssl_tls.c:2441: <= flush output
2016-06-25 11:33:55.250634 DEBUG mbedtls_debug [2] ssl_cli.c:3264: client state: 1
2016-06-25 11:33:55.250648 DEBUG mbedtls_debug [2] ssl_tls.c:2429: => flush output
2016-06-25 11:33:55.250662 DEBUG mbedtls_debug [2] ssl_tls.c:2441: <= flush output
2016-06-25 11:33:55.250677 DEBUG mbedtls_debug [2] ssl_cli.c:0712: => write client hello
2016-06-25 11:33:55.250739 DEBUG mbedtls_debug [2] ssl_tls.c:2714: => write record
2016-06-25 11:33:55.250801 DEBUG mbedtls_debug [2] ssl_tls.c:2429: => flush output
2016-06-25 11:33:55.250819 DEBUG mbedtls_debug [2] ssl_tls.c:2448: message length: 401, out_left: 401
2016-06-25 11:33:55.250878 DEBUG mbedtls_debug [2] ssl_tls.c:2454: ssl->f_send() returned 401 (-0xfffffe6f)
2016-06-25 11:33:55.250913 DEBUG mbedtls_debug [2] ssl_tls.c:2473: <= flush output
2016-06-25 11:33:55.250933 DEBUG mbedtls_debug [2] ssl_tls.c:2861: <= write record
2016-06-25 11:33:55.250947 DEBUG mbedtls_debug [2] ssl_cli.c:1044: <= write client hello
2016-06-25 11:33:55.250968 DEBUG mbedtls_debug [2] ssl_cli.c:3264: client state: 2
2016-06-25 11:33:55.250986 DEBUG mbedtls_debug [2] ssl_tls.c:2429: => flush output
2016-06-25 11:33:55.251003 DEBUG mbedtls_debug [2] ssl_tls.c:2441: <= flush output
2016-06-25 11:33:55.251022 DEBUG mbedtls_debug [2] ssl_cli.c:1396: => parse server hello
2016-06-25 11:33:55.251036 DEBUG mbedtls_debug [2] ssl_tls.c:3739: => read record
2016-06-25 11:33:55.251052 DEBUG mbedtls_debug [2] ssl_tls.c:2221: => fetch input
2016-06-25 11:33:55.251070 DEBUG mbedtls_debug [2] ssl_tls.c:2282: in_left: 5, nb_want: 13
2016-06-25 11:33:55.251087 DEBUG mbedtls_debug [1] ssl_tls.c:2300: should never happen
2016-06-25 11:33:55.251113 DEBUG mbedtls_debug [1] ssl_tls.c:3768: mbedtls_ssl_fetch_input() returned -27648 (-0x6c00)
2016-06-25 11:33:55.251135 DEBUG mbedtls_debug [1] ssl_cli.c:1402: mbedtls_ssl_read_record() returned -27648 (-0x6c00)
2016-06-25 11:33:55.251155 DEBUG mbedtls_debug [2] ssl_tls.c:6312: <= handshake
2016-06-25 11:33:55.251179 DEBUG unknow mbedtls error SSL - Internal error (eg, unexpected failure in lower-level module)
2016-06-25 11:33:55.251201 ERROR (127.0.0.1:48145) session_cb - ssl handshake: SSL - Internal error (eg, unexpected failure in lower-level
 module) (-27648)
2016-06-25 11:33:55.251222 DEBUG session_deferred_free - session_cb - ssl handshake 18a7840 6
2016-06-25 11:33:55.251241 DEBUG session_free - sc=18a7840
2016-06-25 11:33:55.251290 DEBUG mbedtls_debug [2] ssl_tls.c:7021: => free
2016-06-25 11:33:55.251403 DEBUG mbedtls_debug [2] ssl_tls.c:7086: <= free
2016-06-25 11:33:55.251426 INFO  (127.0.0.1:48145) Session closed

while goldy looks like this:

2016-06-25 11:33:51.148966 DEBUG start_listen_io - 3
2016-06-25 11:33:55.250922 DEBUG global_cb fds: 3,3 revents: 0x01 count: 0
2016-06-25 11:33:55.251016 DEBUG connect_to_new_client: connected on fd 6
2016-06-25 11:33:55.251108 INFO  (127.0.0.1:33757) Client connected
2016-06-25 11:33:55.251137 DEBUG global_cb - session_start - client_fd 6
2016-06-25 11:33:55.251317 DEBUG (127.0.0.1:33757) DTLS handshake requested hello verification
2016-06-25 11:33:55.251341 DEBUG session_deferred_free - hello verification a5b860 6
2016-06-25 11:33:55.251358 DEBUG session_free - sc=a5b860
2016-06-25 11:33:55.251482 INFO  (127.0.0.1:33757) Session closed

Note the should never happen and unexpected failure in lower-level module in the goldy-client logs. Am I doing something wrong with my cert or do you see any other configuration errors?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.