# \[ANN\] shuttle v0.3.1 released

**URL:** <https://discuss.ocaml.org/t/ann-shuttle-v0-3-1-released/8684>\
**Category:** Community\
**Tags:** announce\
**Created:** [October 24, 2021, 9:40pm UTC](https://discuss.ocaml.org/t/ann-shuttle-v0-3-1-released/8684 "2021-10-24T21:40:53Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![anuragsoni](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ocaml.org/anuragsoni/32/1107_2.png) [@anuragsoni](https://discuss.ocaml.org/u/anuragsoni)\
**Post date:** [October 24, 2021, 9:40pm UTC](https://discuss.ocaml.org/t/ann-shuttle-v0-3-1-released/8684/1 "2021-10-24T21:40:53Z")

</div>

Hi!,

I’d like to announce the release of version 0.3.1 of [shuttle](https://github.com/anuragsoni/shuttle).

Installation: `opam install shuttle`

**Shuttle** provides an API for buffered I/O for applications using [async](https://opensource.janestreet.com/async/). It fills the same role as the Reader/Writer modules from async, but only supports file descriptors that support non blocking IO. Feature parity with the reader/writer modules is a non-goal.

The library grew out of experiments in replacing manually orchestrated buffer management in some of my older async based applications. The goal is to have a high level api that gives a similar api as reader/writer modules, while providing a little more control over how/when the writes are scheduled. Credits for the idea go to the Janestreet engineers and their implementation of a low latency transport that’s used in `async_rpc`.

The initial release consists of:

1. Shuttle → This is the core library that contains the channel implementation
2. Shuttle\_ssl → Encrypted channels using `async_ssl`
3. Shuttle\_http → Httpaf driver that uses `shuttle` instead of `httpaf-async`. Experimental module mostly used for testing, and some performance benchmarks.

**Additional Notes:**

- The httpaf driver has been contributed as a candidate for the http benchmarks maintained by the ocaml-multicore project → [Multicore OCaml: September 2021, effect handlers will be in OCaml 5.0!](https://discuss.ocaml.org/t/multicore-ocaml-september-2021-effect-handlers-will-be-in-ocaml-5-0/8554#benchmarks-22)
- For most use-cases people should still default to the Reader/Writer modules from async\_unix. They are battle tested and cover a lot more use-cases.
- Future work will involve a lot more testing, adding a driver for `ocaml-tls`, and adding minimal windows support mostly to allow at-least being able to build/develop shuttle based libraries on windows.

---

<div class="post-metadata">

**Author:** ![anuragsoni](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ocaml.org/anuragsoni/32/1107_2.png) [@anuragsoni](https://discuss.ocaml.org/u/anuragsoni)\
**Post date:** [August 29, 2022, 3:28pm UTC](https://discuss.ocaml.org/t/ann-shuttle-v0-3-1-released/8684/2 "2022-08-29T15:28:02Z")

</div>

There have been some significant changes since this last release (Versions 0.4.0 and 0.5.0 are available on opam):

- Buffered reader has a new utility method that allows reading lines
- Shuttle now supports file descriptors that don’t support nonblocking I/O. For blocking I/O Shuttle uses async’s support for running syscalls in its thread pool
- Buffered reader’s api has been simplified to remove `read_one_chunk_at_a_time` in favor of a more familiar `read`, `read_line` etc. `refill` operation is supported to perform a read syscall to fill a channel’s buffer, and `Input_channel.view` can be used to get a view into the channel’s underlying buffer.
- Supports v0.15 series of the janestreet libraries
- Buffered reader now uses an auto-growing buffer instead of a fixed size buffer than notified users that the internal buffer is full and no progress can be made unless some content is consumed. This should allow starting with a smaller buffer without needing to worry about implementing some client side buffering to hold unconsumed data. Channels allow configuring an upper bound on the internal buffer length, if a buffer grows beyond that an exception is raised.
- Buffered writer’s support a richer flush interface. Flush operations report errors encountered while attempting to write any pending bytes. This results in a flush operation that returns a deferred that will resolve at some point with either a success or an error, instead of the older flush operation that would return a deferred that never resolved if there was an error during a write syscall.

---

<div class="post-metadata">

**Author:** ![anuragsoni](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ocaml.org/anuragsoni/32/1107_2.png) [@anuragsoni](https://discuss.ocaml.org/u/anuragsoni)\
**Post date:** [January 4, 2023, 3:11am UTC](https://discuss.ocaml.org/t/ann-shuttle-v0-3-1-released/8684/3 "2023-01-04T03:11:37Z")

</div>

There’s a new version (0.8.1) of the library published to `opam`. Some highlights:

## Shuttle

- Buffered reader’s support timeouts for the `refill` operation. If a file descriptor is currently not ready for a read sys call, the reader will wait for the user-defined upper bound time span and raise an exception indicating a timeout was reached if the underlying file-descriptor isn’t ready for IO within that duration.
- The buffered reader and writer optionally accept an Async [Time\_source](https://github.com/janestreet/async_kernel/blob/master/src/time_source_intf.ml). They default to wall clock, but this input can be used to provide alternate time sources that match the Time\_source interface.

## Shuttle\_ssl

- Client and Server SSL connections forward the [SSL connection context](https://github.com/janestreet/async_ssl/blob/86882526834e0d58a0e35b4ae4ce3303cb369bf6/src/ssl.mli#L41) to the user provided callbacks. This can be useful to access the session certificates for ensuring they meet the user’s expectations.

## Shuttle\_http

This is a new companion library that implements the HTTP/1.1 server codec. There is more work needed to ensure we cover the HTTP 1.1 spec properly, but this initial release includes a usable implementation of the most common use-cases. The library ships with a hand-rolled parser with decent performance and good test coverage (Thanks to ocaml-afl and afl-fuzz I found some bugs I otherwise wouldn’t have!!), and simple service definitions that can support HTTP keep-alive, fixed length and chunked bodies, streaming support to help work with large bodies, request pipelining and centralized error handling for any unhandled exception in the user handler or the I/O runtime. Most of the implementation of `http_async` directly within the `shuttle` project. Future work for the HTTP runtime will happen within shuttle, and http\_async will focus on providing a more opinionated higher level interface for writing async http services.

Some additional features that aren’t necessarily related to the HTTP spec include timeout support for request header parsing (this helps shut down clients that are too slow), and a public api that provides enough control to allow scheduling resources cleanup for streaming response bodies (Resources backing a stream will be shutdown when the socket connection closes, either for EOF, user-action, or any unhandled error while the runtime performs i/o).

### Examples

```ocaml
open Shuttle_http

let hello_world (request : Request.t) =
  return (Response.create ~body:(Body.string "Hello World") `Ok)
;;

```

```ocaml
open Shuttle_http

let my_service (ctx : Server.t) (request : Request.t) =
  let%map reader =
    (* This an example to show a stream that works with an external resource. *)
    Reader.open_file "<some file path>"
  in
  (* Create a pipe from the reader that we will use as a streaming response body. *)
  let reader_pipe = Reader.pipe reader in
  (* Register a callback that's called when the server handler is closed that will also
     close the pipe, in-turn closing the Reader and the underlying file descriptor. This
     is useful in scenarios where the connection is interrupted before the response body
     is fully exhausted and helps avoid resource leaks. This example is demonstrating how
     to do this manually using the server handler. Creating a response via
     [Server.respond_stream] will automatically register the action to close a stream on
     Server connection closing. *)
  upon (Server.closed ctx) (fun () -> Pipe.close_read reader_pipe);
  let response_stream = Body.of_pipe `Chunked reader_pipe in
  Response.create ~body:response_stream `Ok
;;

(* The same service written using the utility method for creating streaming responses. *)
let my_service (ctx : Server.t) (request : Request.t) =
  let%map reader = Reader.open_file "<some file path>" in
  let reader_pipe = Reader.pipe reader in
  return (Server.respond_stream ctx (Body.Stream.of_pipe `Chunked reader_pipe))
;;

```

The library is available [via opam](https://ocaml.org/p/shuttle_http/0.8.1/). If you try the library and experience any issues, or have further questions, please report an issue on the [Github Issue tracker](https://github.com/anuragsoni/shuttle/issues).