[ANN] Shuttle 0.9.1 released

Community
1

Hi,

I’d like to announce a new point release (0.9.1) for shuttle and its companion libraries. They are available to install via opam:

opam install shuttle shuttle_ssl shuttle_http

The code is hosted on github, and the documentation can be seen online on ocaml’s package registry.

Shuttle is a small library that is a wrapper around file descriptors that provides an API for buffered channels using async.

Shuttle_ssl provides ssl support for shuttle and is built on top of async_ssl (async bindings for OpenSSL).

Shuttle_http is a library that implements to help write HTTP/1.1 servers and clients.

Shuttle_http is the library that has seen the most changes in this release:

  • Improved streaming response body handling. The library will reliably schedule a cleanup action that will run if the underlying socket is closed before the entire response body was consumed.
  • Support for one-shot HTTP/1.1 clients. These don’t support keep-alives and the underlying socket will be closed after one request → response cycle.
  • Support for HTTP/1.1 clients that support keep-alives. Client connection will be kept open if both client and server indicate that the connection supports keep-alive. This allows the same connection to be re-used for multiple request → response cycles. The connection object ensures that outgoing requests are serialized to respect that HTTP/1.1 doesn’t support multiplexing.
  • Support for persistent HTTP clients. Not to be confused with HTTP persistent connection. Persistent in this case refers to durable clients that will always hand the user an open connection, and attempts to reconnect if the underlying connection is lost. This leverages Async’s persistent module to provide an HTTP flavored persistent connection.

Uses and examples

A hello world http server

open! Core
open Async
open Shuttle_http

let hello_world (address : Socket.Address.Inet.t) (request : Request.t) =
  Log.Global.info !"%{sexp: Socket.Address.Inet.t} - %s" address (Request.path request);
  return (Response.create ~body:(Body.string "Hello World") `Ok)
;;

An example of a http handler that responds with a streaming body

open! Core
open Async
open Shuttle_http

let my_service (address : Socket.Address.Inet.t) (request : Request.t) =
  let%map reader =
    (* This an example to show a stream that works with an external resource. *)
    Reader.open_file "<some file path>"
  in
  (* Create a pipe from the reader that we will use as a streaming response body. *)
  let reader_pipe = Reader.pipe reader in
  (* Create a response from the reader's pipe. If the server is closed before the full
     response was served, the pipe will be closed which in-turn will close the reader and
     the underlying file descriptor. *)
  Response.create ~body:(Body.of_pipe `Chunked reader_pipe) `Ok
;;

A one-shot http client

let run () =
  let address =
    Client.Address.of_host_and_port (Host_and_port.create ~host:"httpbin.org" ~port:443)
  in
  Client.Oneshot.call ~ssl:(Client.Ssl.create ()) address (Request.create `GET "/get")
;;

Client supporting keep-alive

let run () =
  let stdout = Lazy.force Writer.stdout in
  let%bind httpbin =
    Deferred.Or_error.ok_exn
      (Client.create
         ~ssl:(Client.Ssl.create ())
         (Client.Address.of_host_and_port
            (Host_and_port.create ~host:"httpbin.org" ~port:443)))
  in
  Monitor.protect
    ~finally:(fun () -> Client.close httpbin)
    (fun () ->
      let%bind response = Client.call httpbin (Request.create `GET "/stream/20") in
      Log.Global.info !"Headers: %{sexp: Headers.t}" (Response.headers response);
      let log_body response =
        Body.Stream.iter
          (Body.to_stream (Response.body response))
          ~f:(fun chunk ->
            Writer.write stdout chunk;
            Writer.flushed stdout)
      in
      let%bind () = log_body response in
      let%bind response = Client.call httpbin (Request.create `GET "/get") in
      Log.Global.info !"Headers: %{sexp: Headers.t}" (Response.headers response);
      log_body response)
;;
4 Likes
2

Some new updates since the last release:

  • shuttle_http now supports HTTP/1.1 servers that use async_ssl based encrypted connections.
  • A server context is forwarded to http services. This context can be used to lookup peer client’s address, check if the http service is running on a SSL encrypted server, and if SSL is used lookup peer client’s SSL certificates.
  • HTTP servers support web socket upgrades now. The implementation is based on Janestreet’s web socket library (GitHub - janestreet/async_websocket: A library that implements the websocket protocol on top of Async), and shuttle has a companion library shuttle_websocket that provides the web socket upgrade negotiation for servers, and allows converting a web socket handler to a request -> response promise based http service that can be used by shuttle_http.

Example for a http service that serves web socket traffic on some requests:

open! Core
open! Async
open Shuttle_http

let websocket_handler =
  Shuttle_websocket.create (fun ws ->
    let rd, wr = Websocket.pipes ws in
    Pipe.transfer_id rd wr)
;;

let service context request =
  Log.Global.info "Peer address: %s" (Socket.Address.to_string (Server.peer_addr context));
  match Request.path request with
  | "/websocket"-> websocket_handler request
  | "/" -> return (Response.create ~body:(Body.string "Hello World") `Ok)
  | _ -> return (Response.create `Not_found)
;;

The latest release has been published to the opam package repository, and can be installed via opam:

opam install shuttle_http
opam install shuttle_websocket
2 Likes