Code Monkey home page Code Monkey logo

lunet's Introduction

lunet.link

This is a litte experiment that explores idea of progressive peer-to-peer web applications (PPWA) in mainstream browsers. Primary goal is to deliver seamless experience that can be transparently enhanced to truly P2P (as in no server intermediaries) through the native application providing access to the corresponding network.

Status

This is proof of concept and an ongoing exploration. Things keep changing and you should not use it for anything other explorations / prototypes which will likely break here and there.

Current prototype provides access to the IPFS network, using in-browser JS IPFS node running in a shared worker. Prototype will also attempts to leverage ipfs-desktop application if one is installed and runing.

Local IPFS node with running daemon and gateway works just as well.

It works on Firefox and Chrome, probably on Edge (don't have access to test). In Safari only in-browser node is used due to deliberete choice by Apple to be incompatible with standards and block access to loopback address.

In Safari service worker is used instead of shared worker which are also difficult to debug there, which is to say technically it works on Safari but it's less tested and chances are it might be broken there from time to time.

Example PPWA

If you navigate to

https://lunet.link/peerdium.gozala.io/

You should a fork of https://peerdium.com/ application.

You could access same exact example by navigating to

https://lunet.link/ipfs/QmYjtd61SyXU4aVSKWBrtDiXjHtpJVFCbvR7RgJ57BPZro/

That is because peerdium.gozala.io is set up with IPFS DNSLink corresponding to that exact IPFS address:

dig +noall +answer TXT _dnslink.peerdium.gozala.io
_dnslink.peerdium.gozala.io. 4502 IN    TXT     "dnslink=/ipfs/QmYjtd61SyXU4aVSKWBrtDiXjHtpJVFCbvR7RgJ57BPZro/"

How does this work ?

Actual application resources are published to IPFS network and can be addressed as /ipfs/QmYjtd61SyXU4aVSKWBrtDiXjHtpJVFCbvR7RgJ57BPZro/. As you could see from dig command output there is also a corresponding DNS record for peerdium.gozala.io.

When https://lunet.link/peerdium.gozala.io/ is loaded it connects to the IPFS network through in-browser node in shared worker and / or ipfs-desktop.

It then resolve peerdium.gozala.io IPFS address recorded as DNSLink. And creates sandboxed iframe corresponding to it's origin

It takes CID QmYjtd61SyXU4aVSKWBrtDiXjHtpJVFCbvR7RgJ57BPZro and encodes it as base32 (to avoid case sesitivity) and uses that as subdomain to provide origin separation between apps. Specifically it loads following URL into iframe:

bafybeie2rd23t4aa6qebc4ivjs4tewkaapm6t2ibwe3d6mpwxzd47g76da.celestial.link

Note: celestal.link serves same static content regardless of the subdomain.

Sanboxed iframe is setup with a service worker such that all requestes will be served from shared IPFS node and are relative to IPFS application IPFS address - /ipfs/QmYjtd61SyXU4aVSKWBrtDiXjHtpJVFCbvR7RgJ57BPZro/

Application can also issue POST request to write posted data into user library managed by lunet.link.

This is not fully implemented yet, but idea is that application will not know where produced data ends up, what the address for it or whether it's saved encrypted. Application will only be able to access data provided to it without revealing any of the details about it. By navigating to

https://lunet.link/peerdium.gozala.io/ipfs/QmXg4doMsTcBXgZ3JQosowt12q9Z6xZgURZVkcwCDwRX42/

Lunet will setup application sandox such that it can access data provided /ipfs/QmXg4doMsTcBXgZ3JQosowt12q9Z6xZgURZVkcwCDwRX42/ through GET requests fetch('/data') and save updates into user library via PATCH request all without knowing address of the data or a key used to encrypt it before (if ever) it's published to IPFS.

Various methods provided by web platfrom are used to prevent application from every talking to any servers as we want an ecosystem that does not monetize on smuggled data, but rather keeps user in full control of it.

Wait, what ? How ?

When application is first loaded lunet client will install service worker that will act as proxy to the IPFS network. Then it will fetch page corresponding to it's location from IPFS (by resolving path to a mounted path) and update document accordingly. All the linked resources will also be server by service worker and there for be loaded from IPFS network from the mounted path.

Below diagram illustrates a flow through which in this setup browser fetches each resource

request flow diagram

Protocol Diversity

This prototype uses IPFS, however there is no reason why same approach could not be used to support Dat, SSB or other P2P protocol of your choice. In fact why not embrace diversity ? Everything is intentionally designed such that multiple protocols could be used in synergy, would not it be nice to e.g. load application from IPFS and data it reads / writes from Dat:

lunet.link/ipfs/QmYjtd61SyXU4aVSKWBrtDiXjHtpJVFCbvR7RgJ57BPZro/dat/e56c7ad7bb3d27f516970d14ee5cb9d2cfa7eb15184278cf5a2dd5bbccd02a6b/posts

How about application from Dat operating on data from SSB:

lunet.link/dat/6dd4a37c98ef31d2c6a13b27d27a25e2fc7fa9b7bc16b72617852b043367a0be/ssb/@B/Pg4xaGbgy2CFrza9g5kGZurAILCk+NapOcTXah98I=.ed25519

Can I trust you ?

How can I trust lunet.link / celestal.link not to compromise my privacy ? What if you tomorrow you deploy version that smuggles all my private data so you could sell it off to Cambridge Analytica ?

You can not, nor you should! You can embrace IndieWeb spirit in you and deploy lunet yourself! In fact in a future lunet will register protocol handlers so that instead of using https://lunet.link/peerdium.gozala.io/ we'll use ipfs://peerdium.gozala.io which will redirect to your own (maybe even altered) lunet deployment.

lunet's People

Contributors

gozala avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

lunet's Issues

Incorporate protocol name in the URLs.

At the moment implementation uses CIDs directly.

https://lunet.link/peerdium.gozala.io/
https://lunet.link/QmYjtd61SyXU4aVSKWBrtDiXjHtpJVFCbvR7RgJ57BPZro/

However maybe it would make more sense to follow follow a pattern used by IPFS already:

https://lunet.link/ipns/peerdium.gozala.io/
https://lunet.link/ipfs/QmYjtd61SyXU4aVSKWBrtDiXjHtpJVFCbvR7RgJ57BPZro/
https://lunet.link/dat/e56c7ad7bb3d27f516970d14ee5cb9d2cfa7eb15184278cf5a2dd5bbccd02a6b/

On one hand it makes URLs less usable, on the other hand it makes it more clear what underlying protocol is being used. Another alternative could be:

https://lunet.link/ipns://peerdium.gozala.io/
https://lunet.link/ipfs://QmYjtd61SyXU4aVSKWBrtDiXjHtpJVFCbvR7RgJ57BPZro/
https://lunet.link/dat://e56c7ad7bb3d27f516970d14ee5cb9d2cfa7eb15184278cf5a2dd5bbccd02a6b/

Support for paths

At the moment app url could be changing but that is not reflected in the acutall address-bar as app is loaded in <iframe sandbox>.

We need to intercept pushState / popState / hashchange / load events and reflect those in address-bar.

Dat support through `dat-js` would be nice.

I think there is some interesting opportunities to bridge protocols like ipfs, dat, ssb instead competing. The way things are designed to work it does not matter what protocol has being used to deliver app or publish to. In fact app can be coming from one and data from another. While things are already complicated & there are obvious risks in making it even more complicated I still thing there is a huge opportunity to find synergy across protocols that currently lacks.

SSB support

Similar to Dat support #10 there is nothing preventing SSB support. In fact I think there is some opportunity to find synergy across protocols and I would absolutely love to explore future where these things could co-exist and enhance each other.

Support streaming request / response body

At the moment SW proxy does forward ArrayBuffer for request / response bodies. Which seems to work in use cases currently being proved, however it won't be able to handle things like video streams etc...

Ideally ReadeableStream would have being made transferable, but sadly they are not (, at least yet). In order to support this we need to polyfill that via MessagePort API. We could either create one per request / responce and send it over with response or alternatively multiplex via same MessagePort. Former has benefit of not needing to keep track of request IDs, however later might have non-negligible impact on memory use.

Load apps / sites like `https://lunet.link/myapp.io/`

At the moment each app is hosted by an author on some domain which on first load imports lunet client which will setup a service worker to load app and generally make it available across sessions online or offline without having to interact with servers.

@jimpick and up implementing a simple server which generates bootstrapping doc / sw from subdomain. Specifically when loading https://cid.lunet.link/ it would serve html which mounts corresponding cid and lunet.js SW script.

@autonome in our conversation made a good point that user in this case needs to trust lunet.link because all the apps do more or less go through it.

This got me thinking about alternative way to architect lunet that would address both things in a really neat way.

https://lunet.link/ could be just a static site with a service worker setup such that when say https://lunet.link/peerdium.gozala.io/doc is loaded it will redirect to https://lunet.link/?redirect=peerdium.gozala.io/doc which will install SW and redirects back to https://lunet.link/peerdium.gozala.io/doc causing a SW to serve it.

Note: Redirect will only happen only if https://lunet.link/* never had being loaded in the past, in all other instances it SW will just serve it.

SW on request to https://lunet.link/peerdium.gozala.io/doc will do following:

  1. Will lookup dnslink TXT record to determine CID, let's say it's a $cid1.

    Note: navigating to https://lunet.link/$cid1/doc will be equivalent, except we won't need to do DNS TXT lookups.

  2. Will respond with a following markup:

    <html>
    <body>
       <iframe sandbox src="://${cid1}.lunet.link/doc` />
    </body>
    </html>
    
  3. lunet will serve same thing under all subdomains meaning that cid1.lunet.link will just register SW which will route requests (similar to how it's done today) to the corresponding client document in this case cid1.lunet.link that will route them back to the parent doc https://lunet.link/peerdium.gozala.io/doc which will take care of fetching cid1 from the IPFS and serving it.

Why / How is this better

  1. In this setup lunet.link becomes commodity, you don't have to trust it. You can deploy it anywhere (in fact fork github repo & enabling gh-pages probably will do fine). So in a way it addresses point made by @autonome you don't have to trust it just fork & deploy, best part being you'll need to load it once per device.

  2. We remove whole bootstrapping out of the equation. Knowing CID is just enough, better yet on first load we could allow user to choose to "install" it in local namespace meaning map it to say https://lunet.link/peerdium/. App could provide name suggestions, lunet could pick on that isn't taken and allow user to use that or override in place.

  3. Unlike current setup which requires app / site to cooperate (by serving bootstrap code which embeds lunet client from that domain) it always doc from own subdomain which will then serve corresponding app / site. In other words no cooperation is required.

  4. In a way we are removing dependency on DNS, well not really because we made it far more loosely coupled because

  5. User can just use CID

  6. User can map to whatever name desired.

  7. In a future we could offer alternative name resolution from DNS (web of trust anyone ?)

  8. We put browser in your browser! However maybe in a future we could remove top browser layer.

UI for lunet

As I wrote in response to @patrykadas mockups (shown below). I would like something along those lines in the top right corner to communicate what application is doing at a time & have persistent way to take user to the library. Would be nice to take first stab at it as currently long loading times often provide no feedback and appear as not working.

If someone would like to help with this that would be absolutely welcome.

Use `/data/` as namespace for initial data app works with.

Lunet applications are provided data from library they will work with, however app maybe lunched without a data (first run for instance). It would make sense for an app to provide a template data that is used in such case, any write to data would result in template + changes being written into user library.

Different way to pass site / app CID from meta tags

At the moment meta tag is used to set the IPFS path for the site being hosted, however it's problematic because on first load SW will cache and there for CID updates do not propagate. Either way HTML and meta tag isn't ideal so here are couple of alternatives to consider:

DNSLink

It might be a better idea to use DNSLink in which case SW could lookup CID via /api/v0/dns/ to figure out what the CID is and cache it locally. Occasionally it could check if it was updated and take that into account.

Pros

  • No need to deal to generate anything, meaning all the every app / site could just point to static site returning same html / sw js.
  • config free!

Cons

  • Updating DNS records isn't necessarily an improvement. Use of IPNS might be a way to avoid however IPNS update in practice propagates even slower than DNS records do.
  • DNS Lookup requires a server in case of in-browser implementation.
  • Everything will need an origin in this instance.

CID in importScripts

At the moment lunet looks at the meta tag and passes query argument to SW script. Instead of meta tag however we could pass CID to importScripts instead in which case update of CID will in fact cause SW update as well.

Pros

  • Fixes the issue

Cons

  • Awkward way to configure / update

Just fetch index.html occasionally

We could leave everything as is. Just add some code to SW to quire index.html every now and then to pick up updates.

API to add a pinning bot to lunet

Goals

  • Use may want to pin data in the remote IPFS node for availability.

Proposed solution

At the moment dealing with UI is probably not the best use of our time. Instead we can create special URL pattern such that navigating to it would add a pinning bot to the lunet. e.g.

https://lunet.link/@invite/pinbot/https://mypinningbot.sh?token=apiAccessToken

Above will add pinning service with a name pinbot to the registry. On pin requests it would issue request to the provided url https://mypinningbot.sh?token=apiAccessToken and pass API access token via query argument.

Lunet will not know when user wishes to pin data there. In long term app will issue request to lunet triggering share user interaction, where user will be able to choose who (bot or person) to share with. But in the meantime we could introduce yet another URL pattern to address that e.g.

https://lunet.link/peerdium.gozala.io/${CID}@pinbot/`

Could tell lunet to load CID document into peerdium.gozala.io and pin all PUT requests into service named @pinbot.

Prevent data smuggling through redirects.

I think there is an opportunity for app to smuggle data by navigating to some controlled URL like smuggler.io?data=....&redirect=... that would capture passed data and then redirect back to the app to stay unnoticed. We need a way to find a way to prevent such behavior.

Another maybe even more effective way to smuggle data might be to load <iframe src='//smuggler.io/' /> and keep passing data through postMessage.

Need to verify if combination of CSPs and SW can prevent this behavior of if there are some other mechanism to do so.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.