Code Monkey home page Code Monkey logo

tiny_httpd's Introduction

Tiny_httpd build

Minimal HTTP server using good old threads, with stream abstractions, simple routing, URL encoding/decoding, static asset serving, and optional compression with camlzip. It also supports server-sent events (w3c)

Free from all forms of ppx, async monads, etc. ๐Ÿ™ƒ

Note: it can be useful to add the jemalloc opam package for long running server, as it does a good job at controlling memory usage.

The basic echo server from src/examples/echo.ml:

module S = Tiny_httpd

let () =
  let server = S.create () in
  (* say hello *)
  S.add_route_handler ~meth:`GET server
    S.Route.(exact "hello" @/ string @/ return)
    (fun name _req -> S.Response.make_string (Ok ("hello " ^name ^"!\n")));
  (* echo request *)
  S.add_route_handler server
    S.Route.(exact "echo" @/ return)
    (fun req -> S.Response.make_string (Ok (Format.asprintf "echo:@ %a@." S.Request.pp req)));
  Printf.printf "listening on http://%s:%d\n%!" (S.addr server) (S.port server);
  match S.run server with
  | Ok () -> ()
  | Error e -> raise e
$ dune exec src/examples/echo.exe &
listening on http://127.0.0.1:8080

# the path "hello/name" greets you.
$ curl -X GET http://localhost:8080/hello/quadrarotaphile
hello quadrarotaphile!

# the path "echo" just prints the request.
$ curl -X GET http://localhost:8080/echo --data "howdy y'all" 
echo:
{meth=GET;
 headers=Host: localhost:8080
         User-Agent: curl/7.66.0
         Accept: */*
         Content-Length: 10
         Content-Type: application/x-www-form-urlencoded;
 path="/echo"; body="howdy y'all"}

http_of_dir

Similar to python -m http.server, a simple program http_of_dir is provided. It serves files from the current directory.

$ http_of_dir . -p 8080 &
$ curl -X GET http://localhost:8080
...
<html list of current dir>
...

Static assets and files

The program http_of_dir relies on the module Tiny_httpd_dir, which can serve directories, as well as virtual file systems.

In 'examples/dune', we produce an OCaml module vfs.ml using the program tiny-httpd-vfs-pack. This module contains a VFS (virtual file system) which can be served as if it were an actual directory.

The dune rule:

(rule
  (targets vfs.ml)
  (deps (source_tree files) (:out test_output.txt.expected))
  (enabled_if (= %{system} "linux"))
  (action (run ../src/bin/vfs_pack.exe -o %{targets}
               --mirror=files/
               --file=test_out.txt,%{out}
               --url=example_dot_com,http://example.com)))

The code to serve the VFS from vfs.ml is as follows:

  โ€ฆ
  Tiny_httpd_dir.add_vfs server
    ~config:(Tiny_httpd_dir.config ~download:true
               ~dir_behavior:Tiny_httpd_dir.Index_or_lists ())
    ~vfs:Vfs.vfs ~prefix:"vfs";
  โ€ฆ

it allows downloading the files, and listing directories. If a directory contains index.html then this will be served instead of listing the content.

Steaming response body

Tiny_httpd provides multiple ways of returning a body in a response. The response body type is:

type body =
  [ `String of string
  | `Stream of byte_stream
  | `Writer of Tiny_httpd_io.Writer.t
  | `Void ]

The simplest way is to return, say, `String "hello". The response will have a set content-length header and its body is just the string. Some responses don't have a body at all, which is where `Void is useful.

The `Stream _ case is more advanced and really only intended for experts.

The `Writer w is new, and is intended as an easy way to write the body in a streaming fashion. See 'examples/writer.ml' to see a full example. Typically the idea is to create the body with Tiny_httpd_io.Writer.make ~write () where write will be called with an output channel (the connection to the client), and can write whatever it wants to this channel. Once the write function returns the body has been fully sent and the next request can be processed.

Socket activation

Since version 0.10, socket activation is supported indirectly, by allowing a socket to be explicitly passed in to the create function:

module S = Tiny_httpd

let not_found _ _ = S.Response.fail ~code:404 "Not Found\n"

let () =
  (* Module [Daemon] is from the [ocaml-systemd] package *)
  let server = match Daemon.listen_fds () with
    (* If no socket passed in, assume server was started explicitly i.e. without
       socket activation *)
    | [] -> S.create ()

    (* If a socket passed in e.g. by systemd, listen on that *)
    | sock :: _ -> S.create ~sock ()
  in
  S.add_route_handler server S.Route.rest_of_path not_found;
  Printf.printf "Listening on http://%s:%d\n%!" (S.addr server) (S.port server);
  match S.run server with
  | Ok () -> ()
  | Error e -> raise e

On Linux, this requires the ocaml-systemd package:

opam install ocaml-systemd

Tip: in the dune file, the package name should be systemd.

In case you're not familiar with socket activation, Lennart Poettering's blog post explains it well.

Why?

Why not? If you just want a super basic local server (perhaps for exposing data from a local demon, like Cups or Syncthing do), no need for a ton of dependencies or high scalability libraries.

Use cases might include:

  • serve content directly from a static blog generator;
  • provide a web UI to some tool (like CUPS and syncthing do);
  • implement a basic monitoring page for a service;
  • provide a simple json API for a service, on top of http;
  • use http_of_dir to serve odoc-generated docs or some assets directory.

Documentation

See https://c-cube.github.io/tiny_httpd

License

MIT.

tiny_httpd's People

Contributors

anuragsoni avatar c-cube avatar craff avatar slegrand45 avatar smorimoto avatar vphantom avatar yawaramin avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

tiny_httpd's Issues

Accept-encoding and chunked response

Tiny-httpd should not send chunked is

  • Accept-encoding is provided but chunked is not mentioned.
    Similarly if identity is not in Accept-encoding, this should enforce was is demanded or fail with
    the proper error code.

Provide some level of statistics ?

The PR #23 contains an access to the available connection slot. But it would be useful to
be able to compute some statistics like

  • average and max time to answer a request
  • number of request per time unit
  • ...
    I am ready to write the code ... But don't know exactly what I want.

May be the simplest is to write all request to a sqlite db (time of the arrival of request, time of answer and origin ?) then,
a status page could read that db to print the server status ?

Function `percent_decode` should not convert the `+` symbol

Function percent_decode should not convert the + symbol

See Tiny_httpd_util.ml#L55:

| '+' -> Buffer.add_char buf ' '; incr i (* for query strings *)

I'd completely remove that line (the + for SPC is another language AFAIU).

See also mirage/ocaml-uri/.../uri.ml#L318 which only decodes the percents:

ocaml# #require "uri";;
ocaml# Uri.pct_decode "hello%20world, hello+world";;
- : string = "hello world, hello+world"

PS: why not using the uri library directly BTW? it's pure ocaml and battle-tested.

Make client_addr of Request.t public

Would it be possible to expose the client_addr of Request.t?

AFAIU, it could boil down to

--- a/src/Tiny_httpd_server.ml	Thu Dec 19 14:06:40 2023 +0200
+++ b/src/Tiny_httpd_server.ml	Thu Dec 19 14:07:56 2023 +0200
@@ -176,6 +176,7 @@
 
   let headers self = self.headers
   let host self = self.host
+  let client_addr self = self.client_addr
   let meth self = self.meth
   let path self = self.path
   let body self = self.body
diff -r 24640471ca81 src/Tiny_httpd_server.mli
--- a/src/Tiny_httpd_server.mli	Thu Dec 19 14:06:40 2023 +0200
+++ b/src/Tiny_httpd_server.mli	Thu Dec 19 14:07:56 2023 +0200
@@ -127,6 +127,9 @@
   val host : _ t -> string
   (** Host field of the request. It also appears in the headers. *)
 
+  val client_addr : _ t -> Unix.sockaddr
+  (** Client address field of the request. *)
+
   val meth : _ t -> Meth.t
   (** Method for the request. *)

The primary use case would be access logging.

However, I may be missing something obvious... Is there another way to extract the remote address of the client?

test server sent events

maybe imitate ocaml-redis and use docker compose to start the toy server, and go to /count/10 (say) to get a reliable expect test.

more serious testing

maybe depend on conf-curl/ocurl/curly to do the tests?

  • test gzip decoding (upload with compression enabled?)
  • test gzip encoding (see echo.ml and /zcat/file endpoint)
  • test routing
  • have a stupid middleware to instrument stats, do 10k calls, see if the stats match expectations
  • have a stupid middleware that adds a header to response (X-tested: ok?)

for gzip: we can use a select expression in dune, to avoid a hard dependency on it.

session and handling of body in post method

Two features which are almost always needed are not provided:

  • handling of body with Post method (the normal case is ok with Tiny_httpd_util.parse_query, but things
    like multipart should be supported)
  • handling of a notion of user session
    I could contribute as I am implementing those in my server ... but maybe I am missing something?

Problem with mime type of css and js files

Hello,

I have some problems with the mime type for css and js files when http_of_dir serve them.

The mimetype given by the server is 'text/plain' which is not accepted by firefox, and the css file is even not loaded.

I saw in the code that it use the 'file' command which is not as good as the command 'mimetype', i believe.

Could you replace it, please?

Thanks by advance!

Unix.accept raising exception, not caught

Unix.accept may raise an exception resulting in the death of the server. Typically,
Unix.Unix_error(Unix.EINTR, "accept", "")
even with sigpipe blocked (by default)
I submit the issue and propose a pull request at the same time ...

Small file should not be chunked

When using Tiny_httpd_dir, all files are chunked and needs at least 2 write. It would be easy to get the
file size (there is already a call to Unix.stat to get mtime) and use String instead of Stream for small enough file.

Stream do not close the file descriptor

This piece of code in Tiny_httpd_server (function output_ does not close the stream str) Could be a problem if a client
keep a connection for a long time.

    begin match body with
      | `String "" | `Void -> ()
      | `String s -> output_string oc s;
      | `Stream str -> Byte_stream.output_chunked oc str;
    end;
    flush oc

basic "static file server" middleware

val serve_dir: ?upload:bool -> prefix:string -> path:string -> Tiny_httpd.t -> unit

registers a path "/prefix/%s" for GET-ing files and possibly PUT-ing files if upload=true.

The code can be adapted from http_of_dir.

Better data structure for headers

Headers may have much more that 10 fields and usually use common headers (see []https://en.wikipedia.org/wiki/List_of_HTTP_header_field])

We could encode header_field as
type _ header_field =
| Connection : 'a header_field
| Content_type : 'a header_field
| Cookie : Request.t header_field
| Set-cookie : Response.t header_field
| ...
| Other : string -> 'a header_field

and the headers itself as a hashtable or other data structure a bit more efficient that association list ?
Not clear to me what is the best data structure for medium size maps.

Middleware and accept overlap a bit ? Need more freedom for composition ?

  • Both optional argument do some extra treatment on handler.
  • Some middleware may want look at the request first or last and modify the response first or last, but it
    is not clear how to do that. Either go back to a single middleware + accept done before and then, let
    the user compose middleware functions himself (it is me who proposed lists ?). Provide a few composition operators ?

Invalid_argument("Unix.sigprocmask not implemented") on Windows

Trying this with the MSVC compiler:

C:\Program Files (x86)\Microsoft Visual Studio\2019\Community>Y:\source\dkml\build\pkg\bump\.ci\o\2.1.1\bin\http_of_dir.exe Y:\source\site-main\docs
Fatal error: exception Invalid_argument("Unix.sigprocmask not implemented")

If it is easy to fix I'll submit a PR.

handling of route

I have made several improvements (?) to route handling in simple_httpd you might want to use for tiny_httpd:

  • I have fixed an order on route: more precise first (currently, it is reverse of the order of add_route)
  • I dispatch using a tree for the method and the exact prefix of the route.
  • if no method is specified, it means GET|POST|HEAD but not PUT and DELETE
  • I removed the distinction between url_encode or not, I think it is simpler to always accept url_encoded path ?

Firefox hang on 301 redirect with no body

The spec for http 301 seems to require a body (not clear) and firefox (but not chrome) will hang in this case.
This affect redirection to index.html in directory managment.

Possible fix:

  • directly return the file
  • add a body

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.