mirage / irmin-server Goto Github PK
View Code? Open in Web Editor NEWA high-performance server for Irmin
License: ISC License
A high-performance server for Irmin
License: ISC License
It should be possible to use Blob
or Uint8Array
on the Javascript side to support binary websockets, which would make it possible to remove the base64 encoding that websocket payloads go through.
Currently there are two different APIs that require two different sets of commands to implement. It seems like a good idea to focus on the Store
client API with some additional functions (ping
, reconnect
, ...)
This would require:
command_backend.ml
Irmin_client.Make
to Irmin_client.Store.Make
irmin-server-internal
Batch
API to Store
clientWhen over 30,000 commits are replayed it looks like the server is being killed because it's running out of memory. A smarter way of managing trees on the server should fix this.
This could be a result of re-using the types from Store.Backend
It would be nice to release this after Irmin 3.0 is released.
There is need to support communication between the browser and irmin-server
.
Currently irmin-server
uses flows to communicate between the server and the client. Flows are byte-streams like a TCP connection and these are established using Conduit. In the browser, there is currently no mechanism we know of for having a bi-directional, stream-oriented communication channel
The closest thing is a websocket which solves bidirectional communication, but is inherently message-oriented and not stream-oriented. Websockets sound like a good solution but it could be implemented in a few different ways.
One solution is to impose a stream-like structure layered on top of the message-oriented websocket connection. This could look like the following for example:
let websocket_handler server client =
let flow = Obj.magic () in (* BAD! *)
let rec fill_ic channel client =
let* frame = Websocket_lwt_unix.Connected_client.recv client in
Lwt_io.write channel frame.content >>= fun () ->
fill_ic channel client
in
let rec send_oc channel client =
Lwt_io.read channel >>= fun content ->
Websocket_lwt_unix.Connected_client.send client (Websocket.Frame.create ~content ()) >>= fun () ->
send_oc channel client
in
let input_ic, input_oc = Lwt_io.pipe () in
let output_ic, output_oc = Lwt_io.pipe () in
Lwt.async (fun () -> fill_ic input_oc client);
Lwt.async (fun () -> send_oc output_ic client);
callback server flow input_ic output_oc
Although it is unclear if this will be sufficient but the basic idea is to have two pipes for reading and writing. When we receive a websocket message we add it to the reading channel and when we send a message we add it to the writing channel.
Going this route means that most features has to be reimplemented to support a message-oriented
communication
1. The IO module - The IO module currently uses Conduit_lwt_unix
, which communicates using flow.
2. The protocol - The protocol is currently structured around conn_intf which calls the IO module which uses flow.
I have this partially working and it reveals that websockets from the browser don't pass the tests, I'll try to fix the bugs and start upstreaming all of the various changes needed to put irmin-test
in the browser.
Will there be a version of irmin-client without the lwt?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.