A genuine thank you for all the comments everyone. Firstly, we are not proposing this for the OCaml stdlib: this thread is about the first ever tag in the git repository, and so we welcome all the discussion. I wanted to describe some of the goals of the eio project to clarify our aims more clearly.
We began eio in late 2020 to a backdrop of a flurry of design work on multicore OCaml itself, and needed a way to evaluate whether effects were practical for their promised use at all. The main goal (in the first para of the README) was specifically to build a direct-style alternative to Lwt, with structured concurrency for resource-safe cancellation and backtraces without exposing effects in the library interface.
The bulk of our work has been to test the hypothesis that direct-style IO could be significantly more performant and observably optimal with how they invoke modern OS async interfaces like io_uring, GCD or IOCP. The performance results here speak for themselves, and I hope others are also impressed with just how good it is. Some of the speedups are due to the use of modern IO interfaces, but Eio makes it ergonomic to use those async interfaces without much programmer knowledge. These IO backends are not monolithic: as they stabilise, new libraries (e.g. uring) have all been released separately. Their integration into eio is also modular as separate opam libraries, with the eio core being very portable.
A secondary aim (what with @talex5 and I both being long-time core MirageOS developers) is to investigate how to replace the need for functors in Mirage libraries, and indeed to form the basis for MirageOS 5. We’ve published hundreds of libraries to opam that use these device signatures, and the addition of a single argument to eio calls is sufficient to eliminate those functors while improving performance and retaining portability. We would like MirageOS code to be “just like normal code”, and this has been true since 2013 when the effects research started.
While we’re not set in stone on the use of objects and classes yet; it was not a design decision taken lightly. We want eio to be performant, portable and safe, and the interfaces it exposes so far achieve that balance pretty well. If there are other ways to achieve the subtyping that we need (for unidirection and two-way flows, for example) without lots of syntactic overhead, we can give that a try. But what is currently exposed to developers is very succinct at the moment.
I find the argument that foundational libraries should be conservative and unopinionated to be somewhat uncompelling at this stage in OCaml’s life. We’ve just designed, implemented, evaluated and have almost released two mega features that are transformative to the language: effects and multicore. If we don’t push the boundaries now and explore these features to reap the benefits, what was the point of all the effort? OCaml 5.0.0 will be evolving throughout 2022, and this is the perfect year for the community to go wild and try out other experiments along the same lines as eio and domainslib.
Not all the experiments will succeed, but that is the nature of scientific endeavour. What’s left behind to form a standard IO library for OCaml 5.0 will be stronger as a result. But such endeavour does also require engagement so that we learn as we go along: I don’t think we’ll succeed if it’s just “team eio” trying to build one true library. Some questions i have are:
- There’s been little discussion of Eio’s Fibre and cancellation semantics. Is this sufficient to unify both Lwt and Async’s current models? I do not believe that it is to anyone’s benefit to have two such models in the wild.
- What might migratory codecs look like for existing code, along the lines of @dbuenzli’s nbcodec or lwt_eio for example.
- How might live tracing and logging improve with eio-like backends? Can we create an even fancier alternative to the awesome tokio-console for OCaml code?
- Is the decision to make eio sequential-by-default the right one? It works well for IO-bound code, but does it work when there is more pure computation happening?
I wonder if a ‘IO working group’ along the same lines as we’ve done for the core OCaml upstreaming effort might be appropriate to form at this stage. We need to glue together multiple design requirements, and eventually end up with a single IO library that everyone’s happy to use in the longer term.