It seems that we have 2 questions:
- a simple library to do HTTP requests
- the ability to choose dependencies to implement TLS layer (for instance)
It really hard to solve these questions at the same time. The first should provide the easiest API to be able to send a request. The second should abstract any details involved into the way to do an HTTP request. On the second problem, it requires that the end-user understand and want to change these details. Therefor, this is where such view will mecanicely complexfiy the API (at the expense of the first point). In this respect, a simple API necessarily makes arbitrary choices regarding dependencies and implementations in order to “hide” these technical details.
Now, and from the MirageOS perspective, as @yawaramin said, we want to abstract everything (up to the TCP/IP stack). This means above all that there are levels on which users can play to diverge from the choices we have made (like the choice of
ocaml-tls), for example with regard to
These levels are not necessarily highlighted but they exist and there is already one:
http/af. The design of the latter has the advantage of leaving the implementation of the I/O to the user (and that’s why its integration into MirageOS was not so difficult).
Leaving the choice of the I/O means leaving the choice of the TCP/IP stack but also, basically, of the communication medium used by HTTP. And this way can be replaced by a composition of TCP/IP with TLS for example.
http/af gives another possibility that is more subtle and makes it a bit more difficult to integrate - the scheduler. Indeed, the management of the I/O is very intrinsic to the scheduler. Thus, it is not only a question of offering an implementation of a protocol (TCP/IP or TCP/IP + TLS) but also of implementing a way of scheduling the read/write operations when it is about the HTTP protocol.
This is where a first choice, from Mirage’s point of view, is made: to implement the scheduling of
http/af. As far as Mirage is concerned, the choice was to use
Lwt. Thus, paf offers a scheduling implementation independent of the protocol (as long as the latter respects our
Then, from this abstraction, another choice can be made, the protocol. Again, from our experience with CoHTTP and Conduit, we decided to use mimic as a protocol implementation. The latter has the advantage, like the virtual methods in OCaml, to let the user inject his/her own protocol afterwards.
This method is more the order of the choices of abstractions we can/want to make than a real choice of protocol implementations. Indeed, we could choose an abstraction by the functor (again, mentioned by @yawaramin) but our experience showed us the limits of such a method.
Finally, the real choice that interests us regarding MirageOS is the injection of a particular TCP/IP stack. In this,
http-mirage-client has made the choice to use
ocaml-tls while waiting for the choice of a TCP/IP stack (which will be given afterwards by the
mirage tool). It should be noted that there is finally an equivalence between
http-mirage-client and we are thinking about factoring the first with the second as a specialization with the TCP/IP stack of the host system.
All this to say that there are several levers that users can play on. Mirage’s approach has always been to offer these levers at all levels and, of course, this requires, at each level, a thorough knowledge of what is going on.
http-lwt-client is just the result of all the choices one can make about HTTP. The real question now is not so much the API or the choice of dependencies, but at what level you are .
PS: I didn’t mention
h2 on purpose to stay on a basic explanation but of course, since
h2 shares the same design as
http/af, the clear advantage of all these levels is to offer requests with http/1.1 and h2!