@inetic says:

[..W]hen the origin is down, it takes ~40 seconds for the test to finish. I.e. when the origin is down it takes ~40 seconds between the injector_helper creating a request and having the request callback executed.

Not sure whether this is a bug in the injector helper on in the injector.

After either the injector or injector_helper exits, there is a huge list of memory leaks reported from ASan inside the libbtdht library. This makes it very hard to spot memory leaks inside the dcdn if we were to introduce some.

So far I've found two causes of this, first one being that blocks of memory allocated here are not freed in the destructor of the BlockAllocator struct (nor anywhere else).

The second problem is that the main object of type DhtImpl is never destroyed (Note: objects of type BlockAllocator are also members of this class). We do try to destroy it here but reference count of that smart_ptr has been incremented a few times inside libbtdht (e.g. here) and thus the destruction doesn't take place.

It is not clear to me whether this destruction can be forced cleanly and so we should investigate that further. If not, then we could try to suppress these errors as this problem only appears on app exit (or switch to another library).

Some tests that check for aggregate ability of the system to perform its intended function.

• check the coverage and add a badge to README with it
• devise a scheme for having a new static URL for the test (http://bbc.com/?base64-of-128bits-of-random, perhaps?)
• fetch a valid URL pointing to something that's very likely to be new and never previously injected
• fetch it again
• devise a mechanism for disabling origin connections (implement the policy flag, perhaps?)
• fetch again without origin connection
• make sure X-Sign is tested

We need a minimal app that shows how to integrate the SDK. This issue is just that.

In `examples/', create a very simple Android app that uses Android Studio Gradle-based build.

The example app could load and display the Bing image of the day of maybe of one of Flickr "interesting" images, whichever is more convenient. A button to load a new image of the same kind would be nice.

Even when everything is set up correctly, there is a good chance that the client will respond with failure to the browser with the "Bad Gateway (502)" HTTP status code.

This is due to the fact that each one of us (testers) uses a different set of public/private signing keys yet we all register at the same place in the DHT. Thus when the client contacts a random injector from the DHT it'll probably pick one with an incompatible key.

Given that anyone can fork this repository and join the injector swarm with a different set of signing keys, this is an easy way to deny service to clients.

To reproduce block the origin using a firewall rule such as

sudo iptables -t filter -I FORWARD -d wadiya.wikia.com -p tcp -j REJECT

Request the orign you'll get a
Bad Gateway

page.
I wonder how the dcdn decides if the origin is blocked.

• add the SDK to the example app
• show what was changed to add the SDK, step by step and line by line, in examples/app-name/README.md

After #3 and #10

I'm having problems which can be resolve we had a build automation instead of home brewed ./build.sh. For example I get this:

Building dht.o...
In file included from dht.cpp:1:
In file included from /usr/include/sys/types.h:25:

/usr/include/features.h:180:3: fatal error: "_BSD_SOURCE and _SVID_SOURCE are deprecated, use _DEFAULT_SOURCE" [-W#warnings]
# warning "_BSD_SOURCE and _SVID_SOURCE are deprecated, use _DEFAULT_SOURCE"
^
So if I drop -D_BSD_SOURCE in the source it will goes away and then I'll get

Building dht.o...
In file included from dht.cpp:14:
./dht.h:5:10: fatal error: 'Block.h' file not found
 #include <Block.h>
            ^~~~~~~~~
 1 error generated.

this is my clang version

clang version 4.0.0 (tags/RELEASE_400/final)

I know I can hack my way out but probably these are problems that others will face sooner and later, and build automation system has been invented to exactly sort out these kind of problems without human intervention.

• try clang-tidy
• check if it even works on C
• post the output in this issue
• figure out the relationship with #35
• decide if it's worth it to add it to the build
• add to Travis build

Customers use Android Studio Gradle-based build process.

To better understand what we'll need to do to get good test coverage, let's start with injector helper.

We already have -fsanitize=address.

Should we use the output of the leak sanitizer?

Are there any ASAN_OPTIONS or LSAN_OPTIONS we should use?

Should we use -fsanitize-address-use-after-scope?

https://travis-ci.org/clostra/dcdn/jobs/240955416

Any idea what's going on? Tests pass fine on the Mac, both locally and in Travis.

Rare but does occur: bugsnag breadcrumbs are accessed from both the Java thread and the NewNode thread.

Error in NewNode

Segmentation fault in libnewnode.so:298140
Fatal signal from native code: 11 (Segmentation fault)

View on Bugsnag

Stacktrace

libnewnode.so:298140 - (null)
libnewnode.so:297920 - json_value_free
libnewnode.so:322700 - bugsnag_event_add_breadcrumb
libnewnode.so:158128 - bugsnag_add_breadcrumb
libnewnode.so:147336 - bugsnag_leave_breadcrumb_log
libnewnode.so:278112 - bugsnag_log
libnewnode.so:210900 - http_request_cb
libnewnode.so:572644 - (null)

View full stacktrace

Created by Greg Hazel via Bugsnag

Instead of fetching net.dns, a bunch of Java code is required:

https://github.com/MiniDNS/minidns/blob/master/minidns-android21/src/main/java/org/minidns/dnsserverlookup/android21/AndroidUsingLinkProperties.java

Would you like to add more error handling for return values from functions like the following?

• fseek ⇒ dht_setup
• pthread_create ⇒ Java_com_clostra_newnode_NewNode_setCacheDir

Error in NewNode

Aborted in libnewnode.v1.5.1.so:244060
Fatal signal from native code: 6 (Aborted)

View on Bugsnag

Stacktrace

libnewnode.v1.5.1.so:244060 - bufferevent_free_checked
libnewnode.v1.5.1.so:305532 - ubev_bev_close
libnewnode.v1.5.1.so:307688 - ubev_write_cb
libnewnode.v1.5.1.so:400972 - bufferevent_writecb
libnewnode.v1.5.1.so:434864 - event_process_active_single_queue
libnewnode.v1.5.1.so:417536 - event_base_loop
libnewnode.v1.5.1.so:247968 - network_loop
libnewnode.v1.5.1.so:178732 - client_run
libnewnode.v1.5.1.so:178816 - client_thread

View full stacktrace

Created by Greg Hazel via Bugsnag

In particular, if coverage drops 0.01%, Codecov will fail the build. That's a bit too much.

As per the specification, a client (named peer in the spec) requesting and successfully fetching a URL from an injector or injector_helper, should result in that URL content being shared on the BitTorrent network.

However, testing this behavior results with failure using the current HEAD of the ghazel branch.

In particular, this is what's being tested:

1. Start an injector and wait for it to announce itself into the DHT, then
2. Start a client C1, wait for it to find the injector, then
3. Test that we can fetch (e.g.) bbc.com using C1 as a proxy, then
4. If successful, start another client C2 making sure it won't find any injector nor injector helper through the DHT. Then
5. Try to fetch bbc.com using C2 as a proxy.

It is expected that the step number 5 will succeed, because C2 should download the content from BitTorrent. Instead C2 return the Bad Gateway HTTP error code.

Can be reproduced when AddressSanitizer env options are set like this:

export ASAN_SYMBOLIZER_PATH=/usr/lib/llvm-4.0/bin/llvm-symbolizer
export MSAN_SYMBOLIZER_PATH=/usr/lib/llvm-4.0/bin/llvm-symbolizer
export ASAN_OPTIONS=strict_string_checks=1:detect_stack_use_after_return=1:check_initialization_order=1:strict_init_order=1

=================================================================
==24533==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x6250000027ee at pc 0x000000449d74 bp 0x7ffeaa98bfa0 sp 0x7ffeaa98b750
READ of size 5 at 0x6250000027ee thread T0
#0 0x449d73 in StrtolFixAndCheck(void*, char const*, char**, char*, int) (/home/peter/work/dcdn-ghazel/injector+0x449d73)
#1 0x44a2a1 in __interceptor_strtoll (/home/peter/work/dcdn-ghazel/injector+0x44a2a1)
#2 0x58054f in BencEntity::ParseNum(unsigned char const*) /home/peter/work/dcdn-ghazel/libbtdht/btutils/src/bencoding.cpp:544:11
#3 0x580624 in BencEntity::SetParsed(IBencParser::PARSE_T, unsigned char const*, unsigned long, BencEntity::AllocRegime*) /home/peter/work/dcdn-ghazel/libbtdht/btutils/src/bencoding.cpp:558:5
#4 0x580d7b in BencodedDict::ResumeDict(IBencParser*, BencEntity**, BencEntity::AllocRegime*) /home/peter/work/dcdn-ghazel/libbtdht/btutils/src/bencoding.cpp:703:11
#5 0x5810ca in BencEntity::DoParse(BencEntity&, IBencParser*, BencEntity::AllocRegime*) /home/peter/work/dcdn-ghazel/libbtdht/btutils/src/bencoding.cpp:811:39
#6 0x580472 in BencEntity::Parse(unsigned char const*, BencEntity&, unsigned char const*) /home/peter/work/dcdn-ghazel/libbtdht/btutils/src/bencoding.cpp:759:7
#7 0x514ff1 in load_dht_state(BencEntity*) /home/peter/work/dcdn-ghazel/dht.cpp:71:5
#8 0x5496a1 in DhtImpl::LoadState() /home/peter/work/dcdn-ghazel/libbtdht/src/DhtImpl.cpp:3827:2
#9 0x549501 in DhtImpl::Initialize(UDPSocketInterface*, UDPSocketInterface*) /home/peter/work/dcdn-ghazel/libbtdht/src/DhtImpl.cpp:359:2
#10 0x548ce9 in DhtImpl::DhtImpl(UDPSocketInterface*, UDPSocketInterface*, void ()(unsigned char const, int), void ()(BencEntity), ExternalIPCounter*) /home/peter/work/dcdn-ghazel/libbtdht/src/Dht
Impl.cpp:246:2
#11 0x545c45 in create_dht(UDPSocketInterface*, UDPSocketInterface*, void ()(unsigned char const, int), void ()(BencEntity), ExternalIPCounter*) /home/peter/work/dcdn-ghazel/libbtdht/src/dht.cpp:3
0:29
#12 0x515a61 in dht_setup /home/peter/work/dcdn-ghazel/dht.cpp:125:15
#13 0x52580a in network_setup /home/peter/work/dcdn-ghazel/network.c:163:14
#14 0x5231bc in main /home/peter/work/dcdn-ghazel/injector.c:388:18
#15 0x7f28255f182f in __libc_start_main /build/glibc-bfm8X4/glibc-2.23/csu/../csu/libc-start.c:291
#16 0x41cec8 in _start (/home/peter/work/dcdn-ghazel/injector+0x41cec8)

0x6250000027ee is located 0 bytes to the right of 9966-byte region [0x625000000100,0x6250000027ee)
allocated by thread T0 here:
#0 0x4d5548 in __interceptor_malloc (/home/peter/work/dcdn-ghazel/injector+0x4d5548)
#1 0x514faf in load_dht_state(BencEntity*) /home/peter/work/dcdn-ghazel/dht.cpp:67:24
#2 0x5496a1 in DhtImpl::LoadState() /home/peter/work/dcdn-ghazel/libbtdht/src/DhtImpl.cpp:3827:2
#3 0x549501 in DhtImpl::Initialize(UDPSocketInterface*, UDPSocketInterface*) /home/peter/work/dcdn-ghazel/libbtdht/src/DhtImpl.cpp:359:2
#4 0x548ce9 in DhtImpl::DhtImpl(UDPSocketInterface*, UDPSocketInterface*, void ()(unsigned char const, int), void ()(BencEntity), ExternalIPCounter*) /home/peter/work/dcdn-ghazel/libbtdht/src/DhtI
mpl.cpp:246:2
#5 0x545c45 in create_dht(UDPSocketInterface*, UDPSocketInterface*, void ()(unsigned char const, int), void ()(BencEntity), ExternalIPCounter*) /home/peter/work/dcdn-ghazel/libbtdht/src/dht.cpp:30
:29
#6 0x515a61 in dht_setup /home/peter/work/dcdn-ghazel/dht.cpp:125:15
#7 0x52580a in network_setup /home/peter/work/dcdn-ghazel/network.c:163:14
#8 0x5231bc in main /home/peter/work/dcdn-ghazel/injector.c:388:18
#9 0x7f28255f182f in __libc_start_main /build/glibc-bfm8X4/glibc-2.23/csu/../csu/libc-start.c:291

SUMMARY: AddressSanitizer: heap-buffer-overflow (/home/peter/work/dcdn-ghazel/injector+0x449d73) in StrtolFixAndCheck(void*, char const*, char**, char*, int)
Shadow bytes around the buggy address:
0x0c4a7fff84a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x0c4a7fff84b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x0c4a7fff84c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x0c4a7fff84d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x0c4a7fff84e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
=>0x0c4a7fff84f0: 00 00 00 00 00 00 00 00 00 00 00 00 00[06]fa fa
0x0c4a7fff8500: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c4a7fff8510: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c4a7fff8520: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x0c4a7fff8530: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x0c4a7fff8540: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable: 00
Partially addressable: 01 02 03 04 05 06 07
Heap left redzone: fa
Freed heap region: fd
Stack left redzone: f1
Stack mid redzone: f2
Stack right redzone: f3
Stack after return: f5
Stack use after scope: f8
Global redzone: f9
Global init order: f6
Poisoned by user: f7
Container overflow: fc
Array cookie: ac
Intra object redzone: bb
ASan internal: fe
Left alloca redzone: ca
Right alloca redzone: cb
==24533==ABORTING

There's a stub in .travis.yml, but I haven't looked at what is required to get that to work. You might need something from me besides merging the pull request.

@inetic Can you help with the Linux build?

I got it to dealing with libsodium; probably should at this point use the submodule, rather than an installed library.

Each build attempt takes me a few minutes because I don't have Ubuntu. Can you take a look and transfer whatever you're doing locally to make it build to .travis.yml?

You can view the Travis CI build status by clicking on the badge in README, which takes to https://travis-ci.org/clostra/dcdn

https://github.com/clostra/dcdn/blob/81dae45e4a1f53b945eafa406d8f898894e4eb9f/build.sh

The master branch has injector but doesn't have the client code and the ghazel branch builds the client.c file but doesn't build injector.c.

• design and specify
• expand this ticket when it's known what needs to be done

In file included from dht.cpp:7:
In file included from libbtdht/src/dht.h:26:
libbtdht/btutils/src/utypes.h:40:9: fatal error: 'INT64_MAX' macro redefined [-Wmacro-redefined]
#define INT64_MAX 0x7fffffffffffffffLL
^
~/dcdn/android-toolchain-armv7-a/bin/../sysroot/usr/include/stdint.h:188:9: note: previous definition is here
#define INT64_MAX (INT64_C(9223372036854775807))
^
1 error generated.

(Also I can't test with build anymore as there is no easy way of cleaning the android built libraries).

In Content-Location, which we add.

https://docs.travis-ci.com/user/osx-ci-environment/

Right now, the Linux build has made master very hard to work with from Mac OS X.

Having a Mac OS X build on Travis will make this much easier.

It looks to me like we could add it to the Travis build. Is there a better way in terms of GitHub integration—a separate existing service, maybe with a bot?

I would like to point out that identifiers like “__BEV_SPLICE_H__” and “__NETWORK_H__” do not fit to the expected naming convention of the C++ language standard.
Would you like to adjust your selection for unique names?

Separate environments so the keys can be different in dev and in prod. Prod keys obviously should not be in the repo.

This one is similar to the "40 seconds to fail" issue, but while before it took too long for injector_helper to realize the origin was down, now it takes it too long to realize the injector is down.

Here is a failing test showing the problem.

Injector proxy needs to support uTP on the client side.

• in the injector
• in the client

As mentioned in #45, I also got a build failure when running build.sh on a pristine clone of the repo (git clone --recursive URL) in a Debian Sid system:

In file included from dht.cpp:1:
In file included from /usr/include/x86_64-linux-gnu/sys/types.h:25:
/usr/include/features.h:148:3: fatal error: "_BSD_SOURCE and _SVID_SOURCE are deprecated, use _DEFAULT_SOURCE" [-W#warnings]
# warning "_BSD_SOURCE and _SVID_SOURCE are deprecated, use _DEFAULT_SOURCE"
  ^
1 error generated.

Simply replacing -D_BSD_SOURCE by -D_DEFAULT_SOURCE in build.sh makes the issue go away.

I guess that newer versions of GNU libc (mine is 2.24) are more picky with obsolete definitions, so upgrading to the new definition should be an easy and safe fix.

Currently, testing dCDN in machines connected to the Internet is hampered by the fact that there is just one dCDN network, since the base swarms are hardwired into the code (constants.h). The potential presence of other participants makes it difficult to e.g. ensure that just a given set of clients or injectors are connected to experiment the reactions of the system to injectors and clients going up or down etc.

One simple way to create different dCDN networks for testing would be to support having an optional salt string shared by all participants in the network which would be used in all swarm SHA1 operations (injector swarm, injector proxy swarm, URL swarm). The salt may be received as an optional command line argument to injector and client, empty if missing.

Of course this kind of "shared secret" feature may not be used for production networks, but for testing it should be very useful and not difficult to add.

Thanks!

@inetic says:

[..B]oth injector and injector_helper no longer enter the error handling code. Additionally, the test_server only sends back HTTP OK header and no body, so injector doesn't enter the body handling code. I'll add that while working on something else as that's an easy fix.

E/AndroidRuntime(10651): FATAL EXCEPTION: main
E/AndroidRuntime(10651): Process: com.example.webviewsample, PID: 10651
E/AndroidRuntime(10651): java.lang.UnsatisfiedLinkError: dlopen failed: cannot locate symbol "__cmsg_nxthdr" referenced by "libdcdn.so"...
E/AndroidRuntime(10651): 	at java.lang.Runtime.loadLibrary(Runtime.java:364)
E/AndroidRuntime(10651): 	at java.lang.System.loadLibrary(System.java:526)
E/AndroidRuntime(10651): 	at com.clostra.dcdn.Dcdn.<clinit>(Dcdn.java:5)
E/AndroidRuntime(10651): 	at com.example.webviewsample.MainActivity.onCreate(MainActivity.java:48)
E/AndroidRuntime(10651): 	at android.app.Activity.performCreate(Activity.java:5231)
E/AndroidRuntime(10651): 	at android.app.Instrumentation.callActivityOnCreate(Instrumentation.java:1087)
E/AndroidRuntime(10651): 	at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:2148)
E/AndroidRuntime(10651): 	at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:2233)
E/AndroidRuntime(10651): 	at android.app.ActivityThread.access$800(ActivityThread.java:135)
E/AndroidRuntime(10651): 	at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1196)
E/AndroidRuntime(10651): 	at android.os.Handler.dispatchMessage(Handler.java:102)
E/AndroidRuntime(10651): 	at android.os.Looper.loop(Looper.java:136)
E/AndroidRuntime(10651): 	at android.app.ActivityThread.main(ActivityThread.java:5001)
E/AndroidRuntime(10651): 	at java.lang.reflect.Method.invokeNative(Native Method)
E/AndroidRuntime(10651): 	at java.lang.reflect.Method.invoke(Method.java:515)
E/AndroidRuntime(10651): 	at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:785)
E/AndroidRuntime(10651): 	at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:601)
E/AndroidRuntime(10651): 	at dalvik.system.NativeStart.main(Native Method)
W/ActivityManager(  477):   Force finishing activity com.example.webviewsample/.MainActivity

I also saw rand missing.

Seems to me that KitKat is still quite common to be ignored, especially in Asia and Middle East.

• design the protocol and update the spec
• implement on the injector side
• implement on the client side
• (possibly) implement in the proxy

We have not customized it in any way and it's one of the things that slows down Travis the most.
Perhaps we can just install binaries?

Injector proxy does some work on startup/resume/disconnect and some on each request. Establishing a connection to an injector should be done on startup/resume/disconnect. Indeed, the success of that is what determines whether this even is a functional injector proxy and should advertise itself as such.

• make a test.sh script
• start injector and injector helper
• curl a http://bbc.com through injector helper, which goes to injector
• include the test in Travis CI
• get tests to pass