Code Monkey home page Code Monkey logo

go-fula's Introduction

box

Go Test

Client-server stack for Web3

Intro blog

You can see it in action in our Flagship App: Fx Fotos

Blox server demo

Motivation

There are currently two ways to interact with Web3 storage solutions:

  1. Through a pinning service and a gateway: the advantage is that files are served through URLs, an app can then access the files with conventional methods, e.g. simply putting a picture in <img src="gateway.example.com/Qm...">. The disadvantage is that there is a subscription payment associated with pinning services. Also this is not really decentralized!
  2. Turn the device to a full IPFS node: this model works beautifully in Brave desktop browser as an example, and makes sense for laptop and PC since they normally have large HDDs. It's much harder on mobile devices, however, biggest hurdle is to have Apple on board with the idea of relaxing file system access in iOS! Even if all goes well, a mobile device is NOT a good candidate for hosting the future Web! They get lost easily and are resource constrained (battery, memory).

blox aims to address these issues by creating a third alternative: Personal Server

A personal server is a commodity hardware device (such as a PC or Raspberry Pi) that is kept at home rather than carried with the user. It can help with actual decentralization and can save money in the long run, as there is a one-time cost for the hard drive and no monthly charges. From a privacy perspective, it also guarantees that data does not leave the premises unless the user specifically wants to share it.

To achieve this, we are developing protocols to accommodate client-server programming with minimal effort on developer's side.

Architecture

go-fula is an implementation of the Fula protocol in Go (Golang). It is designed to facilitate smooth communication between clients and a mesh of backend devices (servers) and provide APIs for developers to build mobile-native decentralized applications (dApps).

box architecture

A React Native app can communicate with servers using the @functionland/react-native-fula library, which abstracts the underlying protocols and libp2p connection and exposes APIs similar to those of MongoDB for data persistence and S3 for file storage.

Data is encrypted on the client side using WebNative Filesystem (WNFS) ( with bridges for Android and iOS ). The encrypted Merkle DAG is then transferred to the blox server using Graphsync.

The blox stack can provide backup guarantees by having the data pinned on multiple servers owned by the user. In cases where absolute assurance of data longevity is required (e.g. password records in a password manager app or scans of sensitive documents), the cids of encrypted data can be sent to the Fula blockchain and backed up by other blox owners, who are rewarded for their efforts.

By default, Libp2p connections are established through Functionland's libp2p relay over the internet or directly over LAN without the use of a relay.

Packages

Name Description
blox Blox provides the backend to receive the DAG created by fulamobile and store it
mobile Initiates a libp2p instance and interacts with WNFS (as its datastore) to encrypt the data and Send and receive files in a browser or an Android or iOS app. Available for React-Native here and for Android here
exchange Fula exchange protocol is responsible for the ctual transfer of data

Other related libraries

Name Description
WNFS for Android Android build for WNFS rust version
WNFS for iOS iOS build for WNFS rust version

Run

git clone https://github.com/functionland/go-fula.git

cd go-fula

go run ./cmd/blox --authorizer [PeerID of client allowed to write to the backend] --logLevel=[info/debug] --ipniPublisherDisabled=[true/false]

example on Windows is:

go run ./cmd/blox --authorizer=12D3KooWMMt4C3FKui14ai4r1VWwznRw6DoP5DcgTfzx2D5VZoWx --config=C:\Users\me\.fula\blox2\config.yaml --storeDir=C:\Users\me\.fula\blox2 --secretsPath=C:\Users\me\.fula\blox2\secret_seed.txt --logLevel=debug

The default generated config goes to a YAML file in home directory, under /.fula/blox/config.yaml

Build

goreleaser --rm-dist --snapshot

Build for the iOS platform (using gomobile)

This process includes some iOS specific preparation.

make fula-xcframework

License

MIT

Related Publications and News

go-fula's People

Contributors

ehsan6sha avatar farhoud avatar ghorbani-m avatar hhio618 avatar masih avatar mehdibalouchi avatar morzaka avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

go-fula's Issues

Error Handling

on mobile/fula.go
Error check should happens before any operation even defer.
Some error don't check and some errors won't handle and break the app. (call defer befor error check).

A golang Fula client SDK

As a developer I want to be able to integrate my application that may be running outside of Fula network, in Web 2 world, with other apps or services inside the Fula network. Example: a telegram bot that would notify me when a new picture is added to the photo app.

Add admin API

A local rest API to delegate access to the box.
Create root did for box and delegate user did access.

Need task queues for go-fula

Problem

go-fula/mobile is based on a P2P network and targets mobile devices. so we end up with a very unstable network:

  • users may want to upload and download only on WIFI.
  • box behind nat does not always connect on the first try. #32
  • for peer routing, we have to at least connect to 5 to 10 nodes and it takes time. #37
  • user wants the status of the download and upload progress. #prog
    With the above knowledge and test that I have done. the chance of the user calling our API and getting an error is very high.

Solution

Add task queues and task status API to go-fula/mobile

Question

ctx = network.WithForceDirectDial(ctx, "transiant wont do it")

I don't think direct dial an optimized approach specially when we use pools

On android when using WIFI can't resolve /dnsaddr

On android when using WIFI can't resolve the default bootstrap address.

default bootstrap nodes are :

"/dnsaddr/bootstrap.libp2p.io/p2p/QmNnooDu7bfjPFoTZYxMNLWUQJyrVwtbZg5gBMjTezGAJN",
"/dnsaddr/bootstrap.libp2p.io/p2p/QmQCU2EcMqAqQPR2i9bChDtGNJchTbq5TbXJJ16u19uLTa",
"/dnsaddr/bootstrap.libp2p.io/p2p/QmbLHAnMoJPWSCR5Zhtx6BHJX9KiKNN6tpvbUcqanj75Nb",
"/dnsaddr/bootstrap.libp2p.io/p2p/QmcZf59bWwK5XFi76CZX8cbJ4BhTzzA3gU1ZjYZcYW3dwt",

The root problem is /dnsaddr can't be resolved:

note on /dnsaddr:
bootstrap.libp2p.io does not have A or AAA record it has TXT record of another list of addresses.

hole punching not work on slow connection due to timeout.

Find out where I can find timeout.
Output log of go-fula/mobile test:

2022-07-30T17:36:59.467+0430    DEBUG   p2p-holepunch   holepunch/svc.go:108    Host now has a public address. Starting holepunch protocol.
2022-07-30T17:37:18.516+0430    DEBUG   p2p-holepunch   holepunch/svc.go:199    received hole punch request     {"peer": "12D3KooWMRfhfGvpBfV7qZZs3fsQZfpQpjWC8CqV1YdjXRX4uSwd", "addrs": ["/ip4/178.131.215.204/udp/50256/quic"]}
2022-07-30T17:37:19.265+0430    DEBUG   p2p-holepunch   holepunch/svc.go:257    starting hole punch     {"peer": "12D3KooWMRfhfGvpBfV7qZZs3fsQZfpQpjWC8CqV1YdjXRX4uSwd"}
2022-07-30T17:37:24.265+0430    DEBUG   p2p-holepunch   holepunch/util.go:64    hole punch attempt with peer failed     {"peer ID": "12D3KooWMRfhfGvpBfV7qZZs3fsQZfpQpjWC8CqV1YdjXRX4uSwd", "error": "context deadline exceeded"}
    /home/farhoud/workspace/functionland/go-fula/mobile/fula_test.go:147: send failed failed to dial 12D3KooWMRfhfGvpBfV7qZZs3fsQZfpQpjWC8CqV1YdjXRX4uSwd:
          * [/ip4/172.17.0.2/udp/4001/quic] hole punching attempted; no active dial
          * [/ip4/178.131.215.204/udp/50256/quic] hole punching attempted; no active dial
          * [/ip4/172.17.0.2/tcp/4001] dial tcp4 172.17.0.2:4001: i/o timeout
--- FAIL: TestEncryption (28.56s)
FAIL
FAIL    github.com/functionland/go-fula/mobile  28.584s

Error Handling

  • call send without running addBox end up in a fatal error.
  • in mobile if use set localhost (127.0.0.1) end up in a fatal error.

Expose controls for opportunistic push

fulamobile should enable the app to push things opportunistically and continue to work when no internet is available to push DAGs out to blox.

Changes:

  • Put does not throw an error if it cannot push.
  • instead it would capture all the CIDs it failed to push and expose them via an API that returns the list of such CIDs.
  • There is already a Push function that takes a CID and pushes it explicitly.
  • The mobile app can then, depending on whenever there is connectivity, could get that list, and for each CID call Push explicitly.

The reason the opportunistic logic should live in app level is because, app has access to Mobile OS APIs that should give it much richer information to take decisions, and even be able to run stuff as background processes in the app.

Question

testfile := filepath.Join(dir, "test")

What are the configs we are storing in the file? I believe we are not encrypting anything, right?

Question

core.PeerHost.SetStreamHandler(filePL.PROTOCOL, func(s network.Stream) {

is this the stream handler of IPFS instance or libp2p? I think it is for Kubo, but I couldn't find this method

Replace bootstap

The mobile/bootstrap function is copy of kuda/core/bootstrap.
Replace it with import from ipfs.

PeerStore

Use Libp2p PeerStore for keeping box peers instead of local array.
also should consider this #12

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.