Comments (6)
Yes it makes sense to increase the trigger interval of reports uploading, we should though evaluate the following cases prior of introducing a short upload interval:
- Concurrent upload of reports.
On slow internet connections there may be a number of triggered upload report tasks that could potential re-upload the same un-submitted reports. This can be easily fixed with a file lock but still the cronjobs will queue up and waiting to be run. - Bandwidth consumption of invalid reports upload.
There are certain cases where reports can't be completely uploaded and are not tagged as incomplete triggering the upload process of oonireport. This occurrence will significantly increase the bandwidth consumption of the probe.
Does it makes more sense to have this interval being triggered every 12 or 7 hours instead?
from lepidopter.
Regarding 1. you can just make the lock fail when it is already acquired, the flock
option for this is --nonblock
.
Regarding 2. if this is a problem of oonireport it should be fixed in there rather than having workarounds for it in the raspberry pi image.
I think an interval of 12 hours or 7 hours is definitely too much as what we are seeing in the field is that often times people don't have the pi plugged in for that long and we should manage to get the measurements from it.
from lepidopter.
- I wouldn't recommend using
--nonblock
since it can result to reports not being run at all. There have been a number of ooniprobe measurements that have been stale, for instance due to network connectivity issues. - This needs further investigation but it's very hard to debug and catch these issues.
If we want to ensure that cronjobs are still running in non 24 hours operation we can instead use anacron that assumes a system is not running continuously and can execute cronjobs periodically in days instead of hours.
Note that Lepidopter is being designed for 24 hour operation, non maintained shutdowns such as power failures or disconnection from the mains will wear out SD cards way faster than normal and bring Lepidopter in an unusable state.
from lepidopter.
- I wouldn't recommend using --nonblock since it can result to reports not being run at all.
Why would this lead to reports not being run at all? To be clear I mean the lock should only be on the oonireport
command and not on ooniprobe
.
There have been a number of ooniprobe measurements that have been stale, for instance due to network connectivity issues.
What do you mean by "measurements that have been stale"?
I understand the concerns around "non maintained shutdowns", but given the reality of how it is being used we should try to take into account these.
from lepidopter.
Why would this lead to reports not being run at all? To be clear I mean the lock should only be on the oonireport command and not on ooniprobe.
You are right I was confused with the ooniprobe, yes this this can be included in oonireport but given the fact that we want to protect against improper shutdowns anacron may be a better fit.
What do you mean by "measurements that have been stale"?
Hanged ooniprobe/oonireport processes. Not sure under which circumstances this is triggered but I remember seeing on a Lepidopter platform on the field where I had no time at that point to investigate.
I understand the concerns around "non maintained shutdowns", but given the reality of how it is being used we should try to take into account these.
We should perhaps then prioritize #50.
Having an unusable Lepidopter installation due to SD card issues is not useful to anyone.
from lepidopter.
This is currently supported in ooniprobe version 2.0.0 and no need to be implemented in lepidopter.
from lepidopter.
Related Issues (20)
- Libraries missing for measuing temperature on Raspberry HOT 1
- Change name to OONI Probe HOT 42
- Missing Middle Box HOT 1
- Can i install lepidopter on LXC? HOT 1
- ooniprobe is not running and web interface fails to load decks and recent results HOT 11
- Run the default deck as soon as the raspberry pi boots up HOT 1
- Potential improvements for lepidopter HOT 4
- Build a lepidopter image that does not perform the http-invalid-request-line test HOT 2
- Add Lantern, Psiphon and OpenVPN test dependencies HOT 2
- Add ooniprobe v2.0.0 stable in lepidopter HOT 2
- Document how to access lepidopter web interface HOT 1
- run_oonideckgen is not run when network is up HOT 2
- Zip file for 0.3.5 is possibly corrupt and GPG sig is missing HOT 1
- Restrict management interfaces to LAN
- The latest image doesn't fit on a 4GB card HOT 3
- Lepidopter image should be on the root PATH of the zip archive HOT 1
- Set a PATH that will not be rewritten by any lepidopter updates HOT 3
- Show simple webpage during update HOT 3
- Add a watchdog daemon to auto reboot a hung Raspberry PI HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from lepidopter.