mirage / capnp-rpc Goto Github PK

When closing a connection, send an abort message first explaining why.

When receiving an abort, disconnect and break all caps with the received message as the reason.

Currently, we can only send messages over Unix sockets. We should use Mirage flows instead, so the code can be used in unikernels or with mock networks for testing.

Add when_broken so that clients can recover from a lost connection. Document how to handle this.

Send this if we don't understand a request, and handle it if we receive it.

It should be possible to register a user-defined function to restore sturdy refs. It will need to cache each loaded capability until it is released to avoid duplicates.

I'm trying to install capnp-rpc-lwt (Ubuntu 14.04 with OCaml 4.04.0 and Opam 1.2.2) and seem to be running into an issue when using the command

opam depext -i capnp-rpc-lwt

the external dependencies seem to install fine but then there is an error when trying to compile capnp-rpc-lwt as shown below:

Any advice would be really appreciated.

If I understand this correctly, the client initiates a TLS connection to the server, and then sends over the capability. Is the server authenticated (i.e. to avoid a malicious person in the middle to intercept the communication and read the capability) in any way?

In the very nice tutorial, I could find some information about private keys on the server side, but no information about a server certificate (which in common TLS deployments has to be generated and signed and the signing authority needs to be present at the client to verify the chain of trust).

The C++ code has client and server implementations of the calculator example, and we do too, so we should be able to test different combinations.

Currently, the promise of an answer to a question can resolve to a struct, but any capabilities in the struct that were unresolved at the time of the return will always remain promises. This means that messages may be relayed unnecessarily through the remote host even after they turn out to be local.

some projects don't use lwt (lots of synchronous code). A blocking backend would be very useful.

Firstly, an excellent tutorial.
Currently working my way through and have an issue with Vat's. I would like to serve a Vat which needs to import a client capability for the initialisation of the server. Consider the server:

let serve config client_capability =
Lwt_main.run begin
    let service_id = Capnp_rpc_unix.Vat_config.derived_id config "main" in
    let restore = Restorer.single service_id (Http_server.local client_capability) in
    Capnp_rpc_unix.serve config ~restore >>= fun vat ->
    let uri = Capnp_rpc_unix.Vat.sturdy_uri vat service_id in
    Fmt.pr "echo service running at: %a@." Uri.pp_hum uri;
    fst @@ Lwt.wait ()  (* Wait forever *)
  end

The client_capability has already been created with:

let start_server server_config store_uri =
  ....
    Fmt.pr "Connecting to echo service at: %a@." Uri.pp_hum uri;
    let client_vat = Capnp_rpc_unix.client_only_vat () in
    let sr = Capnp_rpc_unix.Vat.import_exn client_vat store_uri in
    Sturdy_ref.connect_exn sr >>= fun client_capability ->
    serve server_config client_capability

The client capability (client_capability) has already been created by creating another client_only_vat as the server vat is only created after the client capability has been created. Reading through capnp_rpc_unix.ml I can see how to do a little hacking and ensure I just create one Vat.
This is all coming on the back of the https_unikernel example where I need to pass in the remote store capability to the http_server handler.
I guess what I'm asking is - would I be better served (yes,pun) by creating the store capability dynamically (already tried this and it gets hairy) or should I just stick to doing my own vat creation?

Add Noise protocol support alongside TLS, now that an ocaml-noise library is available.

Currently, we always send results back to the caller. For level 1, we should also support yourself (the service justs holds on to the results so that they can be used in another question).

Sharing these observed (race condition?) errors from the test suite.

┌──────────────────────────────────────────────────────────────────────────────┐
│ [FAIL]        lwt          9   Cancel.                                       │
└──────────────────────────────────────────────────────────────────────────────┘
application [INFO] -----       : Start test-case
  capnp-rpc [INFO] vat-S       : Waiting for UNENCRYPTED connections on unix:/var/folders/_l/v2016jrx2kndvkdf6p9phy_80000gn/T/build217130.dune/capnp-rpc-test-server
  capnp-rpc [INFO] -----       : Connecting to "/var/folders/_l/v2016jrx2kndvkdf6p9phy_80000gn/T/build217130.dune/capnp-rpc-test-server"...
  capnp-rpc [INFO] vat-S       : Accepting new connection
  capnp-rpc [WARNING] -----       : Error sending messages: Unix.Unix_error(Unix.ENOTCONN, "write", "") (will shutdown connection)
   endpoint [INFO] -----       : Connection closed
  capnp-rpc [INFO] vat-S       : Connection closed
  capnp-rpc [INFO] vat-C       : Connection closed
ASSERT Check log for warnings
FAIL Check log for warnings

   Expected: `0'
   Received: `1'

Raised at Alcotest_engine__Test.check in file "src/alcotest-engine/test.ml", line 196, characters 4-261
Called from Lwt.Sequential_composition.map.create_result_promise_and_callback_if_deferred.callback in file "src/core/lwt.ml", line 1981, characters 41-46

Logs saved to `~/code/ocaml/capnp-rpc/_build/default/test-lwt/_build/_tests/capnp-rpc/lwt.009.output'.
 ──────────────────────────────────────────────────────────────────────────────

Full test results in `~/code/ocaml/capnp-rpc/_build/default/test-lwt/_build/_tests/capnp-rpc'.
1 failure! in 0.718s. 27 tests run.
Fatal error: exception Alcotest_engine__Core.Make(P)(M).Test_error

┌──────────────────────────────────────────────────────────────────────────────┐
│ [FAIL]        lwt          3   Parallel.                                     │
└──────────────────────────────────────────────────────────────────────────────┘
application [INFO] -----       : Start test-case
  capnp-rpc [INFO] vat-S       : Waiting for UNENCRYPTED connections on unix:/var/folders/_l/v2016jrx2kndvkdf6p9phy_80000gn/T/build2fdbb9.dune/capnp-rpc-test-server
  capnp-rpc [INFO] -----       : Connecting to "/var/folders/_l/v2016jrx2kndvkdf6p9phy_80000gn/T/build2fdbb9.dune/capnp-rpc-test-server"...
  capnp-rpc [INFO] vat-S       : Accepting new connection
ASSERT Ping2 response
ASSERT Ping1 response
  capnp-rpc [WARNING] -----       : Error sending messages: Unix.Unix_error(Unix.ENOTCONN, "write", "") (will shutdown connection)
   endpoint [INFO] -----       : Connection closed
  capnp-rpc [INFO] vat-S       : Connection closed
  capnp-rpc [INFO] vat-C       : Connection closed
ASSERT Check log for warnings
FAIL Check log for warnings

   Expected: `0'
   Received: `1'

Raised at Alcotest_engine__Test.check in file "src/alcotest-engine/test.ml", line 196, characters 4-261
Called from Lwt.Sequential_composition.map.create_result_promise_and_callback_if_deferred.callback in file "src/core/lwt.ml", line 1981, characters 41-46

Logs saved to `~/code/ocaml/capnp-rpc/_build/default/test-lwt/_build/_tests/capnp-rpc/lwt.003.output'.
 ──────────────────────────────────────────────────────────────────────────────

Full test results in `~/code/ocaml/capnp-rpc/_build/default/test-lwt/_build/_tests/capnp-rpc'.
1 failure! in 0.564s. 27 tests run.
Fatal error: exception Alcotest_engine__Core.Make(P)(M).Test_error

When submitting to opam-repository, Travis failed with:

Unix.Unix_error(Unix.ENAMETOOLONG, "", "/home/travis/.opam/ocaml-base-compiler.4.04.2/.opam-switch/build/capnp-rpc-unix.0.3.2/_build/default/test-lwt/server")

We should probably create the test sockets in /tmp to avoid this.

See ocaml/opam-repository#14252 (comment)

Thank you for the nice examples!
I wonder if an alternative roadmap of presenting the examples might help the readers better apply this framework to their projects:

Current roadmap:

1. In-process echo
2. In-process echo + callback
3. Network echo + callback + network server in the same program
4. Network echo + callback + network server in different programs

Suppose I want "Basic echo + network server in different programs", I should first learn from (1) about how to write a .capnp file, and then go directly to (4), and strip away the callback and heartbeat parts.

I doubt if anyone would want to run a network server and a client in the same process, so (3) doesn't seem informative.

Does it make sense to roadmap as follows?

1. Syntax: How to write a basic .capnp file, and validate its functionality in an in-process echo example? (above 1)
2. Network: How to split the echo into different programs? (above 4 minus 2)
3. Semantics: How to make the protocol more interesting? (above 2)

Now the user may try any path: either (i) 1 -> 1+2 -> 1+2+3; or (ii) 1 -> 1+3 -> 1+3+2.

(discovered by AFL and simplified)

I think this relates to the two remaining XXX markers in capTP.ml. The client (on the left) answers q3 with a local promise (step 9), then resolves that to the server's export e0 (step 10).

The server is trying to disembargo x2 so that it can deliver m2. It sends a disembargo down q2, which returns the response back via q1, saying to use q3.

q3 has resolved to a promise at the client. Since this resolution was remote-to-remote the server doesn't do another embargo. This would be fine if i1 were the final answer, but in fact it has resolved to the local object e0. This causes the server to deliver m2 too soon (ahead of m1).

Instead, I think the server should realise this and send a disembargo request down i1.

Currently, you have to convert to a URI, then convert the URI to a string, then write the string to the message. This will also future-proof the API for when we write sturdy refs as structs.

If we receive a capability from a server and then send it back, we tell the remote to send messages via us. We should detect that the server already has direct access and use that.

Provide convenience functions to save sturdy refs to files and reload them. Allow file:foo.cap in command-line arguments as an alternative to capnp URIs. This is useful on shared systems (where arguments are visible in ps) and for use with e.g. Docker swarm secrets (which are provided as files).

We sometimes try to shorten a path when we shoudn't. This can lead to messages being delivered in the wrong order.

Ensure that returns and resolves always forward to the destination they gave in the message, even if a shorter part has been discovered since.

We normally use the public key as the vat ID. When not using TLS we create a new connection for each SR. We could do a slightly better job by using the vat address as its ID.

I'm trying to use capnp to connect an Ocaml process to a Python process. I have successfully followed the 'echo'-tutorial of capnp-rpc, getting an Ocaml client to talk to an Ocaml server. I created a similar setup in Python (having two python processes talk to each other). Now I wish to run the echo-server in a Python process and use an Ocaml client to connect to it. However, I am met with the following error:

atal error: exception (Failure
  "Error calling field(2) -> Failed: This vat only supports a bootstrap interface, not the old Cap'n-Proto-0.4-style named exports.(Echo.ping): Failed: This vat only supports a bootstrap interface, not the old Cap'n-Proto-0.4-style named exports.")

This error seems to originate from the C++ code that Python binds to:
https://github.com/capnproto/capnproto/blob/3f01eac812af3ca863d55981423fb90c08be0809/c%2B%2B/src/capnp/rpc.c%2B%2B#L2579-L2580

Am I doing something wrong while calling for the service, or is the implementation still using these 0.4-style named exports? Below is the code to reproduce:

@0xafda4797418def92;

interface Echo {
  ping @0 (msg :Text) -> (reply :Text);
}

open Lwt.Infix
open Capnp_rpc_lwt

let () =
  Logs.set_level (Some Logs.Warning);
  Logs.set_reporter (Logs_fmt.reporter ())

let run_client service =
  Echo_client.ping service "foo" >>= fun reply ->
  Fmt.pr "Got reply %S@." reply;
  Lwt.return_unit

let () =
  Lwt_main.run begin
    let client_vat = Capnp_rpc_unix.client_only_vat () in
    let sr = Capnp_rpc_unix.Vat.import_exn client_vat @@ Uri.of_string "capnp://insecure@localhost:5001" in
    Sturdy_ref.with_cap_exn sr run_client
  end

#!/usr/bin/env python3

import argparse
import capnp
import time

import echo_api_capnp

class EchoImpl(echo_api_capnp.Echo.Server):

    def ping(self, msg, _context):
        print(msg)
        print(_context)
        return "boe"


def parse_args():
    parser = argparse.ArgumentParser(usage='''Runs the server bound to the\
given address/port ADDRESS may be '*' to bind to all local addresses.\
:PORT may be omitted to choose a port automatically. ''')

    parser.add_argument("address", help="ADDRESS[:PORT]")

    return parser.parse_args()


def main():
    address = parse_args().address

    server = capnp.TwoPartyServer(address, bootstrap=EchoImpl())
    while True:
        server.poll_once()
        time.sleep(0.001)


if __name__ == '__main__':
    main()

Hi

Are there plans for an Async version, besides the Lwt one?

Thanks.

Currently there is no bound on the size of the queue in Endpoint. A client that keeps sending requests and never reads responses will keep using up more and more memory in the server.

We should add some stress tests and try to handle this. Dropping individual messages could break E-order, so we should probably drop the entire connection. We could also try not reading more input until the output has been consumed, although on its own this could deadlock.

Spec says:

However, in practice, the ability to restore SturdyRefs is itself a capability that may require going through an authentication process to obtain. Thus, it makes more sense to define a "restorer service" as a full Cap'n Proto interface. If this restorer interface is offered as the vat's bootstrap interface, then this is equivalent to the old arrangement.

Check https://groups.google.com/forum/#!topic/capnproto/d6uPbXf9e4E to see if we have one already.

We currently only support unencrypted communications.

If you try to connect to a sturdy ref hosted within the same vat, the vat should return the service directly, without making a CapTP connection back to itself.

I've asked about this at https://groups.google.com/forum/#!topic/capnproto/d6uPbXf9e4E

Also need to decide whether we have a single network containing Unix and TCP sockets, with TLS and without, or whether these are separate vat networks.

When a capability is broken due to a lost connection, the exception type should be "disconnected", not "failed".

Currently, invoking a sturdy ref creates a new client vat with a fresh connection. Instead, we should resolve it in the context of an existing vat. If the vat already has a connection to the target vat, use that.

Also, we need to cope with races if we resolve two refs at the same time, or make an out-bound connection at the same time as accepting in in-bound one.

This will not work without TLS, as we're using the public key as the vat's identity. Need to decide what to do for the non-crypto case.

The Return message doesn't provide a way to say that a capability reference is broken, but we could resolve it to broken immediately afterwards.

Payload_caps.of_content needs to extract all the pointer from the message. Currently, it assumes the pointer index is the same as the capability index, which is wrong.

Exercise: create a Callback submodule in echo.ml and implement the client-side Callback.log function (hint: it's very similar to ping, but use Capability.call_for_unit because we don't care about the value of the result and we want to handle errors manually)

have two problem in the excercise.

1. in ping,
   Capability.call_for_value_exn t method_id request >|= Results.reply_get but for log, there is no Results.reply_get

2. ping have a param t, but no t in the notify call of Callback.log.

Thanks!

Currently, we never free any question or import. We should:

• Add finalisers and free them on GC.
• Provide a way to release them explicitly.
• Free them automatically if the user never asked for them (e.g. with call_for_value).

tls pulls a lot of dependencies, it'd be very convenient to just use normal lwt networking functions.

Thank you for this library. It seems to be one of the few more fleshed out capnp-rpc implementations out there (apart from the reference C++ one). The examples are wonderfully written, the code looks exceedingly clean, ...

One thing that keeps confusing me is the inc_ref and dec_ref as far as capabilities are concerned. It confusing to me when we need to do that and there seem to be some subtleties here e.g. see the calc.ml example especially the release function.

Maybe I haven't understood the concept very well yet.

But the simple rule is: any time you create a local capability or extract a capability from a message, you must eventually call Capability.dec_ref on it.

What does "eventually" mean and when does it happen?

What exactly does it even mean to maintain a reference to a capability in the context of capnp-rpc ? Is it even possible to do this automatically and not have to have to involve the user? One of the strong points of OCaml is the garbage collector -- is it possible to just ride on it?

Some more clarity on this in the documentation would be very welcome !

See: https://github.com/capnproto/capnproto/blob/master/c%2B%2B/src/capnp/persistent.capnp

The service base class could provide a save method that any service can override to allow itself to be persisted. The dispatcher could detect calls to this interface as a special case.

Each time we send a message with a capability, we make a new export. We should reuse the existing one when possible.

Is there a release planned soon to support OCaml 4.06 for capnp-rpc-lwt and capnp-rpc-unix ?

I have an use case where Bob needs to connect to Carol, but can only see Alice. I guess the only real solution is Level 3 here - I'm quite happy to have a stab at doing this but would be grateful for any kind of input as to the best way forward.

I haven't had much opportunity to investigate further

#=== ERROR while compiling capnp-rpc-lwt.1.2.3 ================================#
# context     2.1.5 | linux/x86_32 | ocaml-base-compiler.4.14.1 | file:///home/opam/opam-repository
# path        ~/.opam/4.14/.opam-switch/build/capnp-rpc-lwt.1.2.3
# command     ~/.opam/4.14/bin/dune build -p capnp-rpc-lwt -j 71
# exit-code   1
# env-file    ~/.opam/log/capnp-rpc-lwt-7-e72851.env
# output-file ~/.opam/log/capnp-rpc-lwt-7-e72851.out
### output ###
# File "capnp-rpc-lwt/dune", line 8, characters 0-132:
#  8 | (rule
#  9 |  (targets rpc_schema.ml rpc_schema.mli)
# 10 |  (deps rpc_schema.capnp)
# 11 |  (action (run capnp compile -o %{bin:capnpc-ocaml} %{deps})))
# (cd _build/default/capnp-rpc-lwt && /usr/bin/capnp compile -o /home/opam/.opam/4.14/bin/capnpc-ocaml rpc_schema.capnp)
# kj/filesystem-disk-unix.c++:1697: warning: PWD environment variable seems invalid; pwd = /src; *e = kj/filesystem-disk-unix.c++:1693: failed: lstat(result.toString(true).cStr(), &pwdStat): Value too large for defined data type; result = src
# stack: f7d309ec
# *** Uncaught exception ***
# kj/filesystem-disk-unix.c++:305: failed: ::fstat(fd, &stats): Value too large for defined data type
# stack: f7d2cb7f 565daa7b 565db0cc 565e6419 565e67c8 f7d3e945 f7d3f912 f7d40217
# File "capnp-rpc-lwt/dune", line 13, characters 0-132:
# 13 | (rule
# 14 |  (targets persistent.ml persistent.mli)
# 15 |  (deps persistent.capnp)
# 16 |  (action (run capnp compile -o %{bin:capnpc-ocaml} %{deps})))
# (cd _build/default/capnp-rpc-lwt && /usr/bin/capnp compile -o /home/opam/.opam/4.14/bin/capnpc-ocaml persistent.capnp)
# kj/filesystem-disk-unix.c++:1697: warning: PWD environment variable seems invalid; pwd = /src; *e = kj/filesystem-disk-unix.c++:1693: failed: lstat(result.toString(true).cStr(), &pwdStat): Value too large for defined data type; result = src
# stack: f7d669ec
# *** Uncaught exception ***
# kj/filesystem-disk-unix.c++:305: failed: ::fstat(fd, &stats): Value too large for defined data type
# stack: f7d62b7f 565cea7b 565cf0cc 565da419 565da7c8 f7d74945 f7d75912 f7d76217

original logs at https://ocaml.ci.dev/github/ocurrent/ocaml-docs-ci/commit/0f917b96efc6af5be2c60ad95071033f35021927/variant/debian-12-4.14_x86_32_opam-2.1

Level 0 implementations don't support releasing caps and may implicitly ask us to free them by not clearing the releaseParamCaps flag. We never use this mode ourselves, but should support it for interoperability with others.

The code currently doesn't worry much about errors. It should be possible to respond with an error, and uncaught exceptions should be logged and reported as server errors.

I've followed the tutorial in the README, and am trying to adapt it to define a schema that has a Connection.create method which returns a Connection.Stream:

@0xdcaa43aa8c57d4a8;

interface Connection {
  create @0 () -> (stream :Stream);
  interface Stream {
    read @0 () -> (data :Data);
    write @1 (data :Data) -> ();
    close @2 () -> ();
  }
}

My create implementation instantiates a Stream and returns it as a result:

      method create_impl _params release_param_caps =
        let open Connection.Create in
        release_param_caps ();
        let stream = Stream.local in
        let response, results = Service.Response.create Results.init_pointer in
        Results.stream_set results (Some stream);
        Service.return response

The client calls this with:

  let create t =
    let open Api.Client.Connection.Create in
    let request, _params = Capability.Request.create Params.init_pointer in
    Capability.call_for_value_exn t method_id request |> Results.stream_get

However when I try to use the resulting Capability.t stream I get the exception: Invalid capability index!.

I've checked the reference counting, and it appears to be 2 after Results.stream_set results (Some stream);.

The full example can be found here, with the relevant files being:

• main.ml
• pipe.ml
• pipe_api.capnp

The output of which is:

Connection(rc=1)
Stream(rc=1)
Stream(rc=2)
Failed: Invalid capability index!
Failed: Invalid capability index!

Does the capability need to be a sturdy ref in order to be passed to the client?

I'm using the EIO port.