So I'm working on this project Crater for doing Rust regression testing. Looking into the feasibility of writing parts in Rust. I need an HTTP server that speaks JSON, along with a corresponding client. I don't see a lot of docs on how to do this so I'm recording my investigation.
I'm going to use Iron and Hyper, neither of which I have experience with.
Each commit in this repo corresponds with a chapter, so follow along if you want.
Edit: Some inaccuracies within! Thanks /r/rust!
I start by asking Cargo to give me a new executable project called 'httptest'. Passing --bin
says
to create a source file in src/main.rs
that will be compiled to an application.
$ cargo new httptest --bin
I'll use Iron for the server, so following the instructions in their docs add the following
to my Cargo.toml
file.
[dependencies]
iron = "*"
Then into main.rs
I just copy their example.
extern crate iron;
use iron::prelude::*;
use iron::status;
fn main() {
fn hello_world(_: &mut Request) -> IronResult<Response> {
Ok(Response::with((status::Ok, "Hello World!")))
}
Iron::new(hello_world).http("localhost:3000").unwrap();
println!("On 3000");
}
And type cargo build
.
$ cargo build
Compiling hyper v0.3.13
Compiling iron v0.1.16
Compiling httptest v0.1.0 (file:///opt/dev/httptest)
Pure success so far. Damn, Rust is smooth. Let's try running the server with cargo run
.
$ cargo run
Running `target/debug/httptest`
I sit here waiting a while expecting it to print "On 3000" but it never does. Cargo must be capturing output. Let's see if we're serving something.
$ curl http://localhost:3000
Hello World!
Oh, that's super cool. We know how to build a web server now. Good starting point.
Is rustc-serialize still the easiest way to convert to and from JSON? Maybe I should use serde, but then you really want serde_macros, but that only works on Rust nightlies. Should I just use nightlies? Nobody else is going to need to use this.
Fortune favors the bold. Let's go with serde and nightlies. Now my Cargo.toml 'dependencies' section looks like the following.
[dependencies]
iron = "*"
serde = "*"
serde_macros = "*"
Based on the serde README I update main.rs
to look like this:
#![feature(custom_derive, plugin)]
#![plugin(serde_macros)]
extern crate iron;
extern crate serde;
use iron::prelude::*;
use iron::status;
use serde::json;
#[derive(Serialize, Deserialize)]
struct Greeting {
msg: String
}
fn main() {
fn hello_world(_: &mut Request) -> IronResult<Response> {
let greeting = Greeting { msg: "Hello, World".to_string() };
let payload = json::to_string(&greeting).unwrap();
Ok(Response::with(status::Ok, payload))
}
Iron::new(hello_world).http("localhost:3000").unwrap();
println!("On 3000");
}
And then run cargo build
.
$ cargo build
Updating registry `https://github.com/rust-lang/crates.io-index`
Downloading quasi_macros v0.1.9
Downloading aster v0.2.0
Downloading serde_macros v0.3.1
Downloading num v0.1.22
Downloading quasi v0.1.9
Downloading rand v0.3.7
Downloading serde v0.3.1
Compiling quasi v0.1.9
Compiling aster v0.2.0
Compiling rand v0.3.7
Compiling num v0.1.22
Compiling quasi_macros v0.1.9
Compiling serde v0.3.1
Compiling serde_macros v0.3.1
Compiling httptest v0.1.0 (file:///opt/dev/httptest)
src/main.rs:20:12: 20:47 error: this function takes 1 parameter but 2 parameters were supplied [E0061]
src/main.rs:20 Ok(Response::with(status::Ok, payload))
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
error: aborting due to previous error
Could not compile `httptest`.
To learn more, run the command again with --verbose.
Lot's of new dependencies now. But an error. Response::with
has this definition:
fn with<M: Modifier<Response>>(m: M) -> Response
I don't know what a Modifier
is. Some
docs don't
help. I don't know what to do here but I notice that the original
example passed a tuple to Response::with
whereas my update treated
Response::with
as taking two parameters. It seems a tuple is a
Modifier
.
Add the tuple to Ok(Response::with((status::Ok, payload)))
, execute cargo run
, curl some JSON.
$ curl http://localhost:3000
{"msg":"Hello, World"}
Blam! We're sending JSON. Time for another break.
Next I want to POST some JSON, but before I do that I need a proper URL to post to, so I guess I need to learn how to set up routes.
I look at the Iron docs and don't see anything obvious in the main text, but there's a crate called router that might be interesting.
The module docs are "Router
provides fast and flexible routing for
Iron", but not much else. How do I use a Router
?! After I lot of
sleuthing I discover an
example.
OK, let's try to adapt that to our evolving experiment.
I add router = "*"
to my Cargo.toml
[dependencies]
section,
and begin writing. The following is what I come up with
before getting stuck reading the POST data.
#![feature(custom_derive, plugin)]
#![plugin(serde_macros)]
extern crate iron;
extern crate router;
extern crate serde;
use iron::prelude::*;
use iron::status;
use router::Router;
use serde::json;
#[derive(Serialize, Deserialize)]
struct Greeting {
msg: String
}
fn main() {
let mut router = Router::new();
router.get("/", hello_world);
router.post("/set", set_greeting);
fn hello_world(_: &mut Request) -> IronResult<Response> {
let greeting = Greeting { msg: "Hello, World".to_string() };
let payload = json::to_string(&greeting).unwrap();
Ok(Response::with((status::Ok, payload)))
}
// Receive a message by POST and play it back.
fn set_greeting(_: &mut Request) -> IronResult<Response> {
let payload = unimplemented!(); // How do I get the POST body as string?
let request: Greeting = json::from_str(payload).unwrap();
let greeting = Greeting { msg: request.msg };
let payload = json::to_string(&greeting).unwrap();
Ok(Response::with((status::Ok, payload)))
}
Iron::new(router).http("localhost:3000").unwrap();
}
This uses Router
to control handler dispatch. It builds and still
responds to curl http://localhost:3000
, but the handler for the
/set
route is yet unimplemented.
Now to read the POST body into a string. The docs for
Request
say the field body
is an iterator, so we just need to collect
that iterator into a string.
I first try let payload = request.body.read_to_string();
because I
know it used to work.
It does not work.
$ cargo build
Compiling httptest v0.1.0 (file:///opt/dev/httptest)
src/main.rs:30:36: 30:52 error: type `iron::request::Body<'_, '_>` does not implement any method in scope named `read_to_string`
src/main.rs:30 let payload = request.body.read_to_string();
^~~~~~~~~~~~~~~~
src/main.rs:30:36: 30:52 help: methods from traits can only be called if the trait is in scope; the following trait is implemented but not in scope, perhaps add a `use` for it:
src/main.rs:30:36: 30:52 help: candidate #1: use `std::io::Read`
error: aborting due to previous error
Could not compile `httptest`.
To learn more, run the command again with --verbose.
I throw my hands up in disgust. 'Why does this method no longer exist?
The Rust team is always playing tricks on us!' Then I notice the
compiler has - at some length - explained that the method does exist
and that I should import std::io::Read
.
I add the import and discover that read_to_string
behaves
differently than I thought.
101 $ cargo build
Compiling httptest v0.1.0 (file:///opt/dev/httptest)
src/main.rs:31:36: 31:52 error: this function takes 1 parameter but 0 parameters were supplied [E0061]
src/main.rs:31 let payload = request.body.read_to_string();
^~~~~~~~~~~~~~~~
Ok, yeah the signature of Read::read_to_string
is now fn read_to_string(&mut self, buf: &mut String) -> Result<usize, Error>
,
so that the buffer is supplied and errors handled. Rewrite the
set_greeting
method.
// Receive a message by POST and play it back.
fn set_greeting(request: &mut Request) -> IronResult<Response> {
let mut payload = String::new();
request.body.read_to_string(&mut payload).unwrap();
let request: Greeting = json::from_str(&payload).unwrap();
let greeting = Greeting { msg: request.msg };
let payload = json::to_string(&greeting).unwrap();
Ok(Response::with((status::Ok, payload)))
}
Let's run this and give it a curl.
$ curl -X POST -d '{"msg":"Just trust the Rust"}' http://localhost:3000/set
{"msg":"Just trust the Rust"}
Oh, Rust. You're just too bad.
Hey, I know all those .unwrap()
s are wrong. I don't care. We're prototyping.
Before we continue on to writing a client, I want to modify this toy example
to store some state on POST to /set
and report it later. I'll make greeting
a local, capture it in some closures, then see how the compiler complains.
Here's my new main
function, before attempting to compile:
fn main() {
let mut greeting = Greeting { msg: "Hello, World".to_string() };
let mut router = Router::new();
router.get("/", |r| hello_world(r, &greeting));
router.post("/set", |r| set_greeting(r, &mut greeting));
fn hello_world(_: &mut Request, greeting: &Greeting) -> IronResult<Response> {
let payload = json::to_string(greeting).unwrap();
Ok(Response::with((status::Ok, payload)))
}
// Receive a message by POST and play it back.
fn set_greeting(request: &mut Request, greeting: &mut Greeting) -> IronResult<Response> {
let mut payload = String::new();
request.body.read_to_string(&mut payload).unwrap();
*greeting = json::from_str(&payload).unwrap();
Ok(Response::with(status::Ok))
}
Iron::new(router).http("localhost:3000").unwrap();
}
src/main.rs:24:12: 24:51 error: type mismatch resolving `for<'r,'r,'r> <[closure src/main.rs:24:21: 24:50] as core::ops::FnOnce<(&'r mut iron::request::Request<'r, 'r>,)>>::Output == core::result::Result<iron::response::Response, iron::error::IronError>`:
expected bound lifetime parameter ,
found concrete lifetime [E0271]
src/main.rs:24 router.get("/", |r| hello_world(r, &greeting));
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
src/main.rs:25:12: 25:60 error: type mismatch resolving `for<'r,'r,'r> <[closure src/main.rs:25:25: 25:59] as core::ops::FnOnce<(&'r mut iron::request::Request<'r, 'r>,)>>::Output == core::result::Result<iron::response::Response, iron::error::IronError>`:
expected bound lifetime parameter ,
found concrete lifetime [E0271]
src/main.rs:25 router.post("/set", |r| set_greeting(r, &mut greeting));
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
src/main.rs:24:12: 24:51 error: type mismatch: the type `[closure src/main.rs:24:21: 24:50]` implements the trait `core::ops::Fn<(&mut iron::request::Request<'_, '_>,)>`, but the trait `for<'r,'r,'r> core::ops::Fn<(&'r mut iron::request::Request<'r, 'r>,)>` is required (expected concrete lifetime, found bound lifetime parameter ) [E0281]
src/main.rs:24 router.get("/", |r| hello_world(r, &greeting));
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
src/main.rs:25:12: 25:60 error: the trait `for<'r,'r,'r> core::ops::Fn<(&'r mut iron::request::Request<'r, 'r>,)>` is not implemented for the type `[closure src/main.rs:25:25: 25:59]` [E0277]
src/main.rs:25 router.post("/set", |r| set_greeting(r, &mut greeting));
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
error: aborting due to 4 previous errors
rustc is not happy. But I expected that. I'm throwing types at it just to get a response. Tell me what to do rustc.
Those error messages are confusing, but clearly the closure is the wrong type. The get
and post
methods of Router
take a Handler
, and from the doc page I see there's an impl defined as
impl<F> Handler for F where F: Send + Sync + Any + Fn(&mut Request) -> IronResult<Response>
That's a mouthful, but Handler
is defined for Fn
, not FnOnce
or
FnMut
, and it has to be Send + Sync
. Since it needs to be send,
we're not going to be capturing any references, and since the
environment isn't mutable we have to use interior mutability to mutate
the greeting. So I'm going to use a sendable smart pointer, Arc
, and
to make it mutable, put a Mutex
inside it. For that we need to import
both from the standard library like this: use std::sync::{Mutex, Arc};
.
I'm also going to have to move the captures with move |r| ...
to avoid
capturing by reference.
Updating my code like so yields the same error messages.
let greeting = Arc::new(Mutex::new(Greeting { msg: "Hello, World".to_string() }));
let greeting_clone = greeting.clone();
let mut router = Router::new();
router.get("/", move |r| hello_world(r, &greeting.lock().unwrap()));
router.post("/set", move |r| set_greeting(r, &mut greeting_clone.lock().unwrap()));
rustc doesn't like the lifetimes of my closure. Why? I don't know. I ask reem in #rust if he knows what to do.
Several hours later reem says
16:02 < reem> brson: Partially hint the type of the request art, rustc has trouble inferring HRTBs
HRTB means 'higher-ranked trait bounds', which means roughly 'complicated lifetimes'.
I change those same lines to hint the type of r: &mut Request
and everything works...
let greeting = Arc::new(Mutex::new(Greeting { msg: "Hello, World".to_string() }));
let greeting_clone = greeting.clone();
let mut router = Router::new();
router.get("/", move |r: &mut Request| hello_world(r, &greeting.lock().unwrap()));
router.post("/set", move |r: &mut Request| set_greeting(r, &mut greeting_clone.lock().unwrap()));
It was seemingly a bug in Rust's inferencer. That's lame.
Now it builds again, so we can test with curl.
$ url http://localhost:3000
{"msg":"Hello, World"}
$ curl -X POST -d '{"msg":"Just trust the Rust"}' http://localhost:3000/set
$ curl http://localhost:3000
{"msg":"Just trust the Rust"}
Now we're playing with power.
We've got a little JSON server going. Now let's write the client. This time we're going to use Hyper directly.
I add it to my Cargo.toml
: hyper = "*"
, then create src/bin/client.rs
:
extern crate hyper;
fn main() { }
Source files in src/bin/
are automatically built as executables by cargo. Run cargo build
and the target/debug/client
program appears. Good, universe is sane. Now figure out Hyper.
Cribbing off the Hyper client example I come up with this snippet that just makes a request for "/" and prints the body:
extern crate hyper;
use hyper::*;
use std::io::Read;
fn main() {
let mut client = Client::new();
let mut res = client.get("http://localhost:3000/").send().unwrap();
assert_eq!(res.status, hyper::Ok);
let mut s = String::new();
res.read_to_string(&mut s).unwrap();
println!("{}", s);
}
But now cargo run
no longer works.
$ cargo run
`cargo run` requires that a project only have one executable; use the `--bin` option to specify which one to run
I must type cargo run --bin httptest
to start the server. I do so, then cargo run --bin client
and see
$ cargo run --bin client
Running `target/debug/client`
{"msg":"Hello, World"}
Oh, man, I'm a Rust wizard. One last thing I want to do, make the POST
request to set the message. Obvious thing to do is change client.get
to
client.post
.
This returns a
RequestBuilder
,
so I'm looking for a builder method that sets the payload. How about body
?
My new creation:
extern crate hyper;
use hyper::*;
use std::io::Read;
fn main() {
let mut client = Client::new();
let res = client.post("http://localhost:3000/set").body("Just trust the Rust").send().unwrap();
assert_eq!(res.status, hyper::Ok);
let mut res = client.get("http://localhost:3000/").send().unwrap();
assert_eq!(res.status, hyper::Ok);
let mut s = String::new();
res.read_to_string(&mut s).unwrap();
println!("{}", s);
}
But running it is disappointing.
101 $ cargo run --bin client
Compiling httptest v0.1.0 (file:///opt/dev/httptest)
Running `target/debug/client`
thread '<main>' panicked at 'called `Result::unwrap()` on an `Err` value: HttpIoError(Error { repr: Custom(Custom { kind: ConnectionAborted, error: StringError("Connection closed") }) })', /home/rustbuild/src/rust-buildbot/slave/nightly-dist-rustc-linux/build/src/libcore/result.rs:750
An unknown error occurred
To learn more, run the command again with --verbose.
And simultaneously I see that the server has also errored:
$ cargo run --bin httptest
Running `target/debug/httptest`
thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: SyntaxError("expected value", 1, 1)', /home/rustbuild/src/rust-buildbot/slave/nightly-dist-rustc-linux/build/src/libcore/result.rs:750
It's because of my bad error handling! I didn't pass valid JSON to the /set
route. Fixing the body to be .body(r#"{ "msg": "Just trust the Rust" }"#)
lets the client succeed:
$ cargo run --bin client
Compiling httptest v0.1.0 (file:///opt/dev/httptest)
Running `target/debug/client`
{"msg":"Just trust the Rust"}
And just like that we've created a web service and client in Rust. Looks like the future is near. Go build something with Iron and Hyper.