thetorproject / stegotorus Goto Github PK

This requires that the last part of the code in steg after calling get_payload to be moved to the call back function of curl receive.

vmon should read the proposal 203 by nickm for ideas about Stegotorus as an Apache module.

SRI-CSL/stegotorus@c799f30

SRI-CSL/stegotorus@2d4ac59

When creating upstream connction in chop_conn_t::recv_handshake, calling chop_circuit::close fails because it tries to delete a circuit with no id and not stored in the table yet.
Solutions:
Adding it properly even it's not a proper connection.
Write a different close function.

temp_pdf_debugging branch is latest functioning one - can probably hove this off onto f--stegize-js if necessary

find_content_type is returning zero, cant find end of header, let alone content type
On server, HTML works fine, but JS gets

67.2680 [debug] pick_appropriate_cover_payload: SERVER found the next HTTP response template with size 10010
67.2680 [debug] http_server_transmit: coping body of 9876 size
67.2680 [debug] http_server_transmit: SERVER embeding data1 with length 259 into type 1
67.2694 [warn] decode: ERROR: Invalid content type (0)
67.2704 [warn] http_server_transmit: decoding cannot recovers the encoded data consistantly for type 1
67.2705 [warn] send: <1.4> failed to transmit block
67.2705 [info] circuit_send: <1> error during transmit
67.2706 [warn] close: <1> destroying active circuit: fin-- eof- ds=1
67.2706 [debug] downstream_flush_cb: <0.4> 0 bytes still to transmit (reached EOF) (received EOF) (no circuit)
67.2707 [debug] close: <0.4> closing connection; 1 remaining
67.2708 [debug] close: <1> deleting circuit 723844364 from the table
67.2708 [debug] close: <1> closing circuit; 1 remaining
67.2709 [debug] close_cleanup_cb: cleaning up 1 circuits and 1 connections
67.3718 [info] server_listener_cb: 127.0.0.1:5000: new connection to server from 127.0.0.1:32909
67.3719 [debug] conn_create: <0.5> new connection
67.3719 [debug] downstream_flush_cb: <0.5> 0 bytes still to transmit (no circuit) (never received)
67.3725 [debug] downstream_read_cb: <0.5> 456 bytes available
67.3726 [debug] http_server_receive: <0.5> SERVER received request header of length 452
67.3726 [debug] http_server_receive: Cookie: DP0kK22I9=CEySOeDMgdCX-D585NhQF3vWTtvTDQRacgAAAAEAAAABgEAAAAAAAAAAAA;AAAAAA=AAAAAAAAAAA;AAAAA=A;AAAA=..

67.3726 [debug] transmit_soon: <0.5> must send within 100 milliseconds
67.3727 [debug] verify_and_extract: retrieved circ id 723844364
67.3727 [debug] recv_handshake: <0.5> stale circuit
67.3727 [debug] send: <0.5> must send (no upstream)
67.3727 [debug] transmit_room: <0.5> checking available capacity for type 1
67.3728 [debug] get_payload: contentType = 1, initTypePayload = 1, typePayloadCount = 280
67.3728 [debug] get_payload: first payload size=11789, best payload size=10010, num candidate=100

67.3728 [debug] pick_appropriate_cover_payload: SERVER found the next HTTP response template with size 10010
67.3729 [debug] http_server_transmit: coping body of 9876 size
67.3729 [debug] http_server_transmit: SERVER embeding data1 with length 41 into type 1

[error] Segmentation fault at 000000000923f538

Dec 15 19:27:39.393 [notice] Tor can't help you if you use it wrong! Learn how to be safe at https://www.torproject.org/download/download#warning
Dec 15 19:27:39.394 [notice] Read configuration file "/data/data/org.torproject.android/app_bin/torrc".
Dec 15 19:27:39.394 [notice] Read configuration file "/data/data/org.torproject.android/app_bin/torrc.custom".
Dec 15 19:27:39.404 [warn] Skipping obsolete configuration option 'WarnUnsafeSocks'
Dec 15 19:27:39.404 [warn] Skipping obsolete configuration option 'DNSListenAddress'
Dec 15 19:27:39.410 [notice] Opening Socks listener on 127.0.0.1:0
Dec 15 19:27:39.410 [notice] Socks listener listening on port 52182.
Dec 15 19:27:39.410 [notice] Opening DNS listener on 127.0.0.1:5400
Dec 15 19:27:39.411 [notice] Opening Transparent pf/netfilter listener on 127.0.0.1:9040
Dec 15 19:27:39.411 [warn] Could not bind to 127.0.0.1:9040: Address already in use. Is Tor already running?

src\protocol\chop.cc line 1846

should be
delete conn;

This thing where we stack everything on the command line has become just too unwieldy.

A config file would enable all sorts of goodness (mostly via being able to pass parameters to steg modules):

• getting rid of the "repeat the IP address N times" thing for controlling simultaneous connections
• merging nosteg and nosteg_rr
• moving the trace files around
• etc

manipulate the curl send in a way that the Get request look like the browser on the client

It is much better that instead of generating GET requests from scratch, use curl to send the url. If all the info are encoded in the url we even do not need to have access to the raw request

Figure out what to do with TracePayloadServer::capacityPDF in terms of inheritance from FileStegMod children, it seems like this is something we only do in the payload generation process anyway.

We should make the payload-generation and encoding API uniform. Something like this:

struct payload
{
  type_code?;
  string data;
  virtual get_capacity();
  virtual encode();
  virtual decode();
}

Also, "JS" and "JS in HTML" should be distinct operation modes (so we can, in future, just have an "HTML" mode).

Current master is not building (the one that was a result of merging tor-improve into) -- was going to use it to check baseline payload generation

src/protocol/chop.cc:417:2: error: #error "Key agreement not yet implemented"
#error "Key agreement not yet implemented"
^
depbase=echo src/protocol/null.o | sed 's|[^/]*$|.deps/&|;s|\.o$||';
g++ -I. -I./src -I./src/steg -I./src/steg/http_steg_mods -I./src/test/gtest -I./src/test/gtest/include -Werror -Wall -Wextra -Wno-missing-field-initializers -Wformat=2 -std=c++0x -g -O2 -std=gnu++11 -MT src/protocol/null.o -MD -MP -MF $depbase.Tpo -c -o src/protocol/null.o src/protocol/null.cc &&
mv -f $depbase.Tpo $depbase.Po
src/protocol/chop.cc: In member function ‘void {anonymous}::chop_config_t::init_handshake_encryption()’:
src/protocol/chop.cc:425:60: error: ‘passphrase’ was not declared in this scope
kgen = key_generator::from_passphrase((const uint8_t )passphrase,
^
src/protocol/chop.cc: In member function ‘virtual circuit_t {anonymous}::chop_config_t::circuit_create(size_t)’:
src/protocol/chop.cc:455:43: error: expected primary-expression before ‘...’ token
kgen = key_generator::from_passphrase(...);

Only have seen it happening on the server side.

T:162.6815: ckt 1 <ntp 48 outq 0>: recv 256 <d=0 p=12 f=DAT r=0>
162.6815 [debug] remove_next: next_to_process=48 data=(nil) op=XXX
162.6815 [debug] process_queue: <1> processed 0 blocks
162.6815 [debug] maybe_send_ack: <1> considering ACK
162.6815 [error] set_block_received: seq 0 too high (hsn 47)

We might want to check for "being closed" before the eof check. But more importantly we should not close a connection for too many open file error, because we can wait and more file probably will be available (while taking side measure like closing connections more aggressively i.e shorter axe timer).

2854.3218 [debug] send: <1941> 14 dead cycles
2854.3219 [info] create_one_outbound_connection: <1941> trying to connect to 127.0.0.1:5000
2854.3220 [info] create_one_outbound_connection: <1941> connection to 127.0.0.1:5000 failed: Too many open files
2854.3220 [warn] create_outbound_connections: <1941> no outbound connections were successful
2854.3221 [debug] downstream_flush_cb: <0.60159> 0 bytes still to transmit (reached EOF) (sent EOF) (no circuit) (never received)
2854.3221 [debug] downstream_flush_cb: <0.60403> 0 bytes still to transmit (reached EOF) (sent EOF) (no circuit) (never received)
2854.3221 [debug] downstream_flush_cb: <0.60226> 0 bytes still to transmit (reached EOF) (sent EOF) (no circuit) (never received)
2854.3222 [debug] downstream_flush_cb: <0.60380> 0 bytes still to transmit (reached EOF) (sent EOF) (no circuit) (never received)
2854.3222 [debug] downstream_flush_cb: <0.60457> 0 bytes still to transmit (reached EOF) (sent EOF) (no circuit) (never received)
2854.3223 [debug] downstream_flush_cb: <0.60284> 0 bytes still to transmit (reached EOF) (sent EOF) (received EOF) (no circuit)
2854.3223 [debug] close: <0.60284> closing connection; 0 remaining
2854.3224 [debug] close: <1941> deleting circuit 4070651211 from the table
2854.3224 [debug] close: <1941> closing circuit; 101 remaining
2854.3224 [debug] check_for_eof: <1941> sent and received FIN
2854.3225 [debug] close: <1941> deleting circuit 4070651211 from the table
2854.3225 [error] close: assertion failure at src/protocol/chop.cc:407: out->second == this

7a79326#commitcomment-7209659

Now that stegs aren't throwing invalid block headers all over the place....

The trace files are currently hardwired to be "traces/client.out" and "traces/server.out" relative to the current working directory of the stegotorus process. We need a way to configure their location.