pagekite / upagekite Goto Github PK

I am pretty sure the ESP32's TLS code is not verifying relay certificates.

This seems pretty common in the IoT world, which suggests some users won't care. But we should do better.

People should be able to pip install upagekite

And upip as well, if that's feasible.

I should remove the old "exec()" based code execution path (webroot/path/index.py etc.).

The microframework pattern is easier to read and more in line with developer expectations, keeping (and documenting) both doesn't feel justified. The RAM savings don't really justify the complexity, and if history has taught us anything RAM availability is only going to increase.

The current license, LGPL, isn't suitable for some embedded projects.

The license probably needs to change.

When upagekite decides not to connect to anything, it's unclear from the debug output why. We need to be more verbose about what decisions are being made and various error states.

Although not strictly necessary, it would be silly not to leverage the existing httpd code for this.

We are currently ignoring all control frames, include the CLOSE and PING messages, which causes problems with at least Python's websockets library/client.

As soon as #1 gets fixed, this becomes critical (until then, it does not matter).

The current HTTPD cannot handle large requests, it only parses the HTTP headers, and only if those all fit in a single PageKite frame.

A state machine which handles requests spread over multiple frames and parses common POST request formats would be a really nice thing to have.

This needs further investigation.

From a user:

The ESP32 only can have 10 sockets opened at the same time. Sometimes, our code, was not able to open the UDP
socket, so this meant that someone was opening sockets without closing them correctly.

The problem is in the function http_get: When internet is not available(but Wi-Fi is available. Only local connection) or
when the connection fails, the socket remains opened.(you didn't close it). After 10 attempts, the board has 10 sockets opened,
and everything is not working.

The current code uses the frontend lookup intended for pagekite.py 1.0. We should switch to a upagekite specific name, once the relays support it.

Currently the httpd can only process requests that arrive over the pagekite tunnel.

We should make the httpd also listen on a normal socket, so upagekite.httpd can also be used for captive portals and initial setup (before we have WiFi credentials and internet access).

The demo code I wrote for the MCH workshop is much better than what I am shipping now.

The default source for DNS hints probably needs to change to match whatever scalable server-side solution I come up with. Making a note of that so I don't forget.

The ESP32's DNS resolver only ever returns one IP address for a given query. This means the connector cannot look up the entire pool of available relays, and cannot make performance-base choices.

The quickest way to solve this will probably be to add a helper to pagekite.py on the server (relay) side, to perform lookups on the device's behalf. This needs more work.

The micro-framework needs to support basic web security features by default.

A few ideas:

• A method which requires HTTP Basic auth (username/password)
• An option on the above method to allow unauthenticated access from the LAN
• Easy-to-use CSRF in forms and the form parsing code
• Review what kind of CORS headers should be included by default
• Use default Referrer-policy to not leak domains/URLs to outside
• Set cache-control defaults to "private" only

The documentation focus here isn't great - too much about how to bootstrap and set up a "my way" dev environment, not enough about how to use the lib itself.

I should remove most of the content from the README and move it to a separate tutorial in the upagekite-tutorial project. At some point, adding dry reference-style docs here might make sense, but the focus will be on the tutorial to begin with, since it is also informing the current development work (when writing tutorials feels clunky, that's a sign the lib needs work).

The current code never disconnects from a relay unless an error occurs.

This will cause unnecessary load on the relay network and needs fixing.

The current asyncio compatibility is incomplete; most socket operations happen on raw sockets and upagekite interleaves its own poll()-based event loop with that of asyncio.

This is useful because it facilitates the current, simple API for developing dynamic web apps.

But it's also inefficient and introduces some lag/jitter which it would be nice to eliminate or minimize.

The current code can only hand requests to built-in servers running in the same process (a basic HTTPD is included). Like PageKite on other platforms, we should also support proxying requests to an external server for people with other needs.

Once things have settled down a tiny bit, this needs to be better documented.