thetorproject / lepidopter Goto Github PK

Lepidopter is quite confusing as a name and not having the OONI brand as part of it, distances it from OONI. Calling it OONI Pi would make more sense as it has brand consistency, is a short name and sounds phonetically great (ooniepie)

Edit (by Arturo): We should rename it to simply "OONI Probe"

What do you think 👍 👎 ?

Currently lepidopter uses for the period tasks (ooniprobe update, ooniprobe test runs, etc.) the @daily, @weekly cronjobs.

It's not clear to me exactly when these occur and it makes it harder to troubleshoot why a particular probe may not be submitting reports (i.e. is it due to the fact that the cronjob has not run yet or is it offline).

It would be better if we used UTC time for all the lepidopter probes for the daily measurements (so that they all run measurements at the same time, for example at 24:00 UTC) and that the weekly tasks always happen on the same day of the week.

This would also allow us to do better planning for when to issue a new release and ensure the raspberry pi images get updated.

So if I understand things correctly, if an upload fails for whatever reason, OONI will keep the data on the SD card until it finds room to upload them. We need to make sure this data sits encrypted on the card by default.

There should be some programmatic easy way of having an optional probe_id included as an annotation inside of the reports that are submitted by partners.

I would say we can also do the configuration of this as part of the testing and configuration of the probes specifically for partners and doesn't have to be shipped in the mainline distribution.

However there should be some file called something like /etc/ooniprobe/probe_id that when present and contains a name it will be added as an annotation {"probe_id": "something"}

I've just installed Lepidopter 1.0.0 on a Raspberry Pi B+ and when I open the web interface, I get this error twice as popup:

Failed to load decks
SyntaxError: JSON.parse: unexpected character at line 1 column 1 of the JSON data

The ooniprobe Dashboard shows that ooniprobe is not running and that it failed to load decks and recent results.

Using the refresh button on the top right results in the same error messages.

A quick look at the browser's network debugger shows that the requests to /status, /deck, /measurement and /nettest are failing with HTTP error code 500.

The response for /status shows following traceback as result:

web.Server Traceback (most recent call last):
exceptions.AttributeError: 'int' object has no attribute 'splitlines'
/usr/local/lib/python2.7/dist-packages/twisted/internet/defer.py:151 in maybeDeferred
150    try:
151        result = f(*args, **kw)
152    except:
/usr/local/lib/python2.7/dist-packages/klein/_app.py:134 in execute_endpoint
133        endpoint_f = self._endpoints[endpoint]
134        return endpoint_f(self._instance, *args, **kwargs)
135
/usr/local/lib/python2.7/dist-packages/klein/_app.py:250 in _f
249            def _f(instance, request, *a, **kw):
250                return _call(instance, f, request, *a, **kw)
251
/usr/local/lib/python2.7/dist-packages/klein/_app.py:55 in _call
54        args = (__klein_instance__,) + args
55    result = __klein_f__(*args, **kwargs)
56    if iscoroutine(result):
/usr/local/lib/python2.7/dist-packages/ooniprobe-2.3.0-py2.7.egg/ooni/ui/web/server.py:69 in wrapper
68                raise WebUIError(404, "Invalid XSRF token")
69            return f(instance, request, *a, **kw)
70
/usr/local/lib/python2.7/dist-packages/ooniprobe-2.3.0-py2.7.egg/ooni/ui/web/server.py:278 in api_status
277    def api_status(self, request):
278        return self.render_json(self.status, request)
279
/usr/local/lib/python2.7/dist-packages/ooniprobe-2.3.0-py2.7.egg/ooni/ui/web/server.py:260 in render_json
259        request.setHeader('Content-Type', 'application/json')
260        request.setHeader('Content-Length', len(json_string))
261        return json_string
/usr/local/lib/python2.7/dist-packages/twisted/web/http.py:1271 in setHeader
1270        """
1271        self.responseHeaders.setRawHeaders(name, [value])
1272
/usr/local/lib/python2.7/dist-packages/twisted/web/http_headers.py:220 in setRawHeaders
219        encodedValues = [_sanitizeLinearWhitespace(v)
220                         for v in self._encodeValues(values)]
221
/usr/local/lib/python2.7/dist-packages/twisted/web/http_headers.py:40 in _sanitizeLinearWhitespace
39    """
40    return b' '.join(headerComponent.splitlines())
41
exceptions.AttributeError: 'int' object has no attribute 'splitlines'

And the other requests (/deck, /measurement and /nettest) show this traceback as result:

web.Server Traceback (most recent call last):
exceptions.AttributeError: 'int' object has no attribute 'splitlines'
/usr/local/lib/python2.7/dist-packages/twisted/internet/defer.py:151 in maybeDeferred
150    try:
151        result = f(*args, **kw)
152    except:
/usr/local/lib/python2.7/dist-packages/klein/_app.py:141 in execute_error_handler
140        """
141        return handler(self._instance, request, failure)
142
/usr/local/lib/python2.7/dist-packages/klein/_app.py:368 in _f
367            def _f(instance, request, failure):
368                return _call(instance, f, request, failure)
369
/usr/local/lib/python2.7/dist-packages/klein/_app.py:55 in _call
54        args = (__klein_instance__,) + args
55    result = __klein_f__(*args, **kwargs)
56    if iscoroutine(result):
/usr/local/lib/python2.7/dist-packages/ooniprobe-2.3.0-py2.7.egg/ooni/ui/web/server.py:69 in wrapper
68                raise WebUIError(404, "Invalid XSRF token")
69            return f(instance, request, *a, **kw)
70
/usr/local/lib/python2.7/dist-packages/ooniprobe-2.3.0-py2.7.egg/ooni/ui/web/server.py:255 in web_ui_error
254            "error_message": error.message
255        }, request)
256
/usr/local/lib/python2.7/dist-packages/ooniprobe-2.3.0-py2.7.egg/ooni/ui/web/server.py:260 in render_json
259        request.setHeader('Content-Type', 'application/json')
260        request.setHeader('Content-Length', len(json_string))
261        return json_string
/usr/local/lib/python2.7/dist-packages/twisted/web/http.py:1271 in setHeader
1270        """
1271        self.responseHeaders.setRawHeaders(name, [value])
1272
/usr/local/lib/python2.7/dist-packages/twisted/web/http_headers.py:220 in setRawHeaders
219        encodedValues = [_sanitizeLinearWhitespace(v)
220                         for v in self._encodeValues(values)]
221
/usr/local/lib/python2.7/dist-packages/twisted/web/http_headers.py:40 in _sanitizeLinearWhitespace
39    """
40    return b' '.join(headerComponent.splitlines())
41
exceptions.AttributeError: 'int' object has no attribute 'splitlines'

currently, the only interface available on boot is eth0, even though the rpi 3 includes an on-board wifi chip

The current image doesn't actually fit on a 4GB SD card. If it's too big to fit entirely on a 4GB card and burning it to 4GB you end up with a corrupt image.

Currently there is no assurance that the deck is not being updated after ooniprobe update.
The ooniprobe deck generator should always run after ooniprobe update.

Currently un-submitted reports are only being uploaded every 24 hours. I would suggest we make that happen more often, since it doesn't add any extra cost to check if there are un-uploaded reports and it allows us to trigger uploads with a tighter interval.

This would avoid situations where the network is not working consistently at the daily schedule.

I would say we can trigger this cronjob every 15 minutes.

Evaluate options to reduce writes in order to minimize SD card wear out.
One idea is to move all logs files in RAM and save before shutdown/reboot.
In a case of an unexpected shutdown (power disruption) no logs files will be
saved, though SD card longevity is way more important!

This means making a tested build of the raspberry pi image somewhere on the web and pointing to it's download location on the lepidopter page.

I was trying to check the Raspberry Pi's temp during the test, but it seems that a couple of libraries are missing from the distribution.

lepidopter@lepidopter:$ /opt/vc/bin/vcgencmd measure_temp
/opt/vc/bin/vcgencmd: error while loading shared libraries: libvchiq_arm.so: cannot open shared object file: No such file or directory
lepidopter@lepidopter:$ ldd /opt/vc/bin/vcgencmd
linux-vdso.so.1 (0x7ecad000)
libvchiq_arm.so => not found
libvcos.so => not found
libpthread.so.0 => /lib/arm-linux-gnueabi/libpthread.so.0 (0x76f70000)
libdl.so.2 => /lib/arm-linux-gnueabi/libdl.so.2 (0x76f5d000)
librt.so.1 => /lib/arm-linux-gnueabi/librt.so.1 (0x76f46000)
libc.so.6 => /lib/arm-linux-gnueabi/libc.so.6 (0x76dfc000)
/lib/ld-linux.so.3 (0x54ae7000)

Make the necessary changes to include ooniprobe v2.0.0-alpha that will be later upgraded to the release v2.

• Include ooniprobe v2.0.0-alpha
• Upgrade to stable ooniprobe v2

The zip archives of the image should always be on the root folder and not inside
a directory as it is currently (images directory).

Currently we use vanilla tor in lepidopter to run ooni-probe and submit the
reports. We should instead use a fall-back to various pluggable transports
(such as obfs4 and meek) in case Tor connection is blocked. From what I know tor
client cannot be configured to use both bridges and non bridges.

A lepidopter image version is needed that will not perform the http-invalid-request-line test in the daily ooniprobe measurements.

This task should ensure that the http-invalid-request-line test is not being run with the rest of the default tests (web_connectivity and http_header_field_manipulation) of the deck.
The test deck and inputs (URL test lists) should be updated regularly (weekly) as per default in lepidopter.

Currently it's hard to learn what the address of a raspberry pi based probe and it would be nice to support some mDNS based announcement and discovery mechanism.

Bonjour or Avahi come to mind and it seems like there is a pretty good python implementation of this: https://github.com/jstasiak/python-zeroconf.

I think this feature should be something that goes into ooniprobe itself (the ooniprobe daemon actually) and enabled only on raspberry pi images of it.

This task should add the dependencies needed to perform OONI measurements for Lantern, Psiphon and OpeVPN tests.

This involves tagging this repository with a tag that is aligned to the ooniprobe release version it's pinned to.

The image should have enough free disk space for new ooniprobe measurements and system files in order for the OS to operate properly.

Currently we don't enforce any emergency deletion process, if for any reason the disk space fills up (cannot communicate with backend) the OS will not be able to recover and Raspberry Pi will turn in a boot cycle or became a useless brick that doesn't perform/submits any ooniprobe measurements.

On the other hand we may lose important measurement reports that occurred in a specific event i.e a country's network completely blocking outgoing/incoming connections for a long time. If we don't automatically delete these reports a person can manually recover the reports (from the SD card) and extract useful information about this incident from the OONI measurements.

cc: @hellais @willscott @bassosimone @agrabeli

Lepidopter image is headless so RAM usage from the GPU component is useless.
This ticket should evaluate if there are no issues introduced by minimizing the
allocated GPU RAM of Raspberry Pi.

Lepidopter torrent file points to an incorrect file and has no seeders.
Reported by @hellais

Currently the lepidopter image ships with a file called /etc/default/lepidopter that contains some version information, though it appears to only mention the label of the version (ex. LEPIDOPTER_BUILD="alpha") and not the actual full version string.

This should be a problem if we decide to make some future major releases that are also going to be in alpha.

I would also advise we change the format of the version to be something that is easier to be parsed (for example just including the raw version number) and perhaps place it inside of a more standard file (such as /etc/lepidopter_version similarly to how debian does it).

For backward compatibility reasons I would say we keep both files /etc/default/lepidopter being the variable and /etc/lepidopter_version being the raw version number.

It would greatly improve troubleshooting (as well as give us the ability to request people to power cycle the pi to recover from intermittend failures) if we were to run measurements as soon as the pi boots up.

Given the fact that we need to do some pre-configuration and testing on first boot, I would do this every other time that isn't the first boot (we can record the first boot inside of some file).

Currently we have no way of learning if a probe from a partner is working or not and we have to rely on waiting for them to perform the first measurement to know if they are even online.

I would say as a first easy thing to support we could have a service that we ping as soon as the probe comes online (and perhaps have it include the probe_id if present) so that at least we know which ones are able to connect to the internet.

For added stealth we could do this either via Tor or cloudfronting.

Currently there is a script that is not run when networking is up

/etc/network/if-up.d/run_oonideckgen

This script won't be run when network is up because it is not run by default without network manager. It can be fixed with adding the following at the eth0 section in /etc/network/interfaces.d/setup

post-up /etc/network/if-up.d/run_oonideckgen

Also script need to be executable

This will be displayed only the first time the probe is setup. It will guide the user through configuring the following setting from a web interface:

• The WPA password, default password for lepidopter system, and password for accessing the admin interface (perhaps they could both be the same?)
• If the probe should also expose a tor hidden service that grants OONI team remote access.
• If the Web GUI should also be exposed as a Tor Hidden Service (should this HS address be the same as the one above?).
• What types of measurements should be run with a description of each test and their associated inputs.
• When the wizard is complete it will provide a summary of the configured options.
• If the user has chosen to enable Tor Hidden Service remote administration it will also print out the tor hidden service address and the list of authorized keys. The user may add their own key or remove some keys from the list.
• When the users click save they will be redirected to the Web GUI: #38.

We should write documentation on how to access the lepidopter web interface via bonjour on Windows, OSX and Linux.

We should wipe data periodically right after a successful upload (assuming over .onion)
This help us to avoid running out of space on smaller SD cards and also avoid other security issues in the future.

A PATH should be set where changes will not affected by any potential lepidopter updates.
As requested by paja on the ooni-talk mailing list.

I am in a middle box situation with certainty. I actually caught the man doing an ARP attack in August of 2016. I have collected quite a pile of evidence in screen shots of ip addresses.

My telephone was configured for TOR and I am pretty sure it was set to Toriffy all traffic. Initially Ooniprobe did not detect a middle box. I am not sure of this step but I think that I changed the phone settings to ignore IPV6. Then Ooniprobe detected a middle box twice. Then my phone was hacked and now says that transparent proxy is not supported and TOR no longer works. Ooniprobe on my phone no longer detects a middle box.

I installed Lepidopter on a Raspberry pi and noticed that the screen reports an IPV4 address as it should but users=0 so I cant tell whether it is functioning. It just sits at the command prompt. The IPV4 address is 192.168.1.64. My computer shows an IPV6 address until I set it to ignore and reset. Then I see is 173.183.190.200 which is a change from the Ooniprobe from my phone. I just got a new service and router in the same building as is being monitored. It was on AS852.

The method being used here seems similar to a method listed on the forum to block individual web sites. IPV6/DNS misdirection. Here it is being used to redirect the entire internet.

Thank-you for your hard work on this magnificent tool! If I can help let me know.

Greg Rudy

Please make sure everything is plug and play. The operator shouldn't necessarily need to have a keyboard or monitor. Everything should be running smoothly as soon as it's powered up and plugged into ethernet.

When the Raspberry Pi first boots up it will setup a WPA encrypted WiFi network with a default password. The SSID shall contain the string "OONI" and it will be a captive portal that redirects any connection on the local network to a setup wizard page (or the web GUI if it's already configured).

There is no indication of ongoing update through web interface. It would be nice to have some simple page which could highlight that something is going on.

lepidopter may be exposed to Internet, it has ssh enabled with weak default password and authless ooniprobe web interface.

I can imagine several (unlikely, but imaginable) cases for the exposure:

• User's ISP & router are IPv6-capable providing routable IPv6 address to lepidopter
• User setting up port-forwarding carelessly to view nice ooni-probe wui from a 3G smartphone
• User is ISP and lepidopter is given routable IPv4 address
• Some unpredictable port-forwarding madness triggered by combination of systemd, Bonjour/avahi and uPNP

I can suggest couple of ways to restrict management interfaces:

1. on network-change event triggered by dhclient/systemd/whatever parse output of ip -o addr and allow source IPs from known subnets
2. on network-change event parse ip neight and deny source MACs of various routers

Raspberry Pi comes with a hardware based watchdog timer that causes a reset after detecting a hanged out process. A watchdog service should be added in order to use the hardware based watchdog.

The build script should also support configuring a tor hidden service that allows ssh access to some set of keys.

we only need the binaries compiled from go, we don't need all of the go source and runtime used for compilation in the produced image. this should save some space.

I was just looking at what resin.io are doing for managing big fleets of embedded devices and it seems like there are some things we can learn from them.

https://resin.io/how-it-works/

Basically they have a very lightweight base OS image https://github.com/resin-os that is based on yocto (http://www.yoctoproject.org/). Then everything is an app that runs inside of a docker container. When they need to provision updates, what they do is they push a new docker image to the docker repositories and then pull it in from the devices.

The devices are managed via a VPN that allows them to reach devices behind NAT.

In particular they have a supervisor that is written in go and based on docker for running "apps" on their devices:

• https://github.com/resin-io/resin-supervisor

A captive portal like thing for connecting devices to wifi networks:

• https://github.com/resin-io/resin-wifi-connect

This will be shown every time a user logs into the Lepidopter Web GUI.
It will expose the following functionality:

• List of measurements run so far with time-stamps and test type.
• The HS address of ssh and/or web GUI.
• The list of authorized ssh keys.

The size of the zip file for the 0.3.5 image seems to be very small (80 MB vs 182 MB for the xz version) and the signature file 404s.

See:
https://get.ooni.torproject.org/lepidopter/lepidopter-v0.3.5-beta-armel.img.zip
https://get.ooni.torproject.org/lepidopter/lepidopter-v0.3.4-beta-armel.img.zip.asc

There should be a way to automatically update the image of lepidopter.

Currently the only thing that is being updated is ooniprobe itself and this is done via pip.

The update system should support doing more advanced things as well such as:

• Updating configuration files
• Making changes to the period scripts that get run
• Installing additional software that is needed for newer version of lepidopter/ooniprobe
• Configuring ooniprobe to run in different ways

A Contribution file should be added with guidelines on how people could better
contribute to lepidopter.

Now that ooniprobe 1.4.2 is out we should ensure that the raspberry pi image is still working properly.