Eio 0.1 - effects-based direct-style IO for OCaml 5

With my compiler hat on, some things I’m mildly worried about is as follows:

  • Currently the Multicore-enabled compiler only provides low-level primitives for parallelism, and basically nothing for concurrency. (In fact, we do have the Threads module that works surprisingly well with Multicore. It was built for backwards-compatibility only but currently it’s, ironically, a fairly sensible option to write concurrent code in Multicore.)
  • The idea from the start was to be able to write concurrency support (an event loop, etc.) in user land on top of fibers / effect handlers, to experiment with what works best for various problem domains, etc.
  • But then, if everyone does something different, there is a risk of fragmentation of the ecosystem where we end up with 5 different concurrency abstractions that can’t talk to each other, and OCaml beginners have to choose the camp they belong to from day 1. (That wouldn’t happen, right? We totally didn’t do that for cooperative concurrency libraries, build systems, package managers, etc.) Exploring many different approaches is great, but ending up with insular ecosystems is not great.

I think that the EIO people are hoping that their honest effort to come up with a good concurrency story on top of the low-level compiler primitives will win everyone’s heart, people will adapt existing libraries (Lwt, Async, what have you) on top of it or move to their new code, no fragmentation anymore, end of story. And given that there are steady contributions to EIO, with excellent people like @talex5 working on this, coming from highly-respected OCaml shops that have a lot of influence in the ecosystem, for tooling decisions, etc., this sounds like a plausible scenario!

If people are worried that EIO “is not Foundational enough” (as in: we’d like to do something different that is not on top of EIO, but on top of the low-level primitives), do they have plausible proposals for how to avoid ecosystem fragmentation? For example, we could consider some of the following ideas:

  • Hope that the various concurrency libraries will be able to collaborate with each other (integrate their event loops together somehow?). That sounds very hard, and to my knowledge we already didn’t manage to do this in the sequential world – but then people like @antron know much better than I do about this.

  • Design a sort of mid-level concurrency layer that is more high-level than just domains, and expressive and non-opinionated enough that people want to build on top of that, and cooperate through that mid-layer. But what would this be? Isn’t the “concurrency primitives” part of EIO something like that?

  • Your suggestion here.

In theory, we are not in a real hurry to solve this question: OCaml 5.00 is not out yet, it will not be a production-ready release that all projects jump onto from day 1, we can expect a stream of increasingly robust 5.x releases with only some parts of the ecosystem getting their foot wet at first. And it’s not reasonable to expect people to come up with full answers to questions about how to use an implementation that is not released yet! (Note: the Multicore people have a head-start here thanks to their experiments on top of the Multicore runtime; everyone could have participated, but few people had the time (and funding) to do this, it’s great that they did many experiments.)

In practice, there are some basic questions about OCaml programming that are hard to answer right now about OCaml 5.0, because we don’t know what to recommend to users willing to write multicore code. For example, Lazy values cannot be used concurrently, and the reason why is that we don’t know how to block code that forces a thunk that is already being forced by someone else. (We could do this with a domain mutex, but this would block the whole domain, whereas people want something that will only block the “current fibre” and yield to another concurrent computation on the same domain.) This is one example of the Stdlib, but I’m sure there will be many other examples once people start porting their projects to Multicore: whenever the natural answer to a question is “well you should block until X happens”, nobody knows what’s the good way to do this, because we don’t know what concurrency layer the user wants to use – we don’t know what blocking means for our end-users.

So I think that we urgently need some sort of standard for these questions. I don’t know if EIO is the right answer, but I haven’t seen proposals in this thread that sound like better answers. (Of course people were discussing different things, like “what’s the best way to adapt full programs to Multicore”.) My intuition is that if we think hard enough about this, we may find reasonably simple answers that can please everyone, because “how should library block on an operation” sounds like an easier question than “what’s a good API design for all interactions with the OS?”. Well…

12 Likes

These programming practices are not imposed on us by a potentially foundational library. None of the examples you could easily think of involve potentially globally imposing a programming convention on all users. This type of argument again misses the point that is being made, which is a conjunction:

  • Object capability style is costly,
  • AND none of the benefits it has are as strong as claimed, in an OCaml context,
  • AND none of the benefits it has are strong compared to benefits of what else we are already do, and would have to keep doing,
  • AND a foundational library would try to specifically -impose- this style.

Addressing one multiplicand “branch” of this conjunction doesn’t address the point and misunderstands the nature of the objection.

Indeed, as I’ve already said,

So a reply of the form

  1. It is already sometimes used.
  2. We sometimes use other things in the same direction (most of which, by the way, are for the most part not this costly).

Can you find an example where we are considering widely imposing an artificial syntactic convention, forcing users to partially construct (still inaccurate) proof terms for some property as they write their program? Because one part of the point is that everything else that OCaml does is quite “light” and fits together nicely, and eio with ocap is not light, it is costly.

Current OCaml has awkward syntax mainly for rarely used features or in cases that are widely considered to be limitations of the language. An ocap-based I/O library that isn’t rare would effectively become a limitation of the language (ecosystem).

It seems reasonable to consider EIO experimental. We just need to avoid drifting into this becoming actually foundational.

1 Like

I’m essentially ready to design and implement some kind of “lower level” (than EIO) multicore and effect I/O library, that EIO with ocap could be interpreted over, if desired. Given I worked on Lwt and Luv already, I don’t find such a project personally intimidating. I’ll probably ping people in the coming days, to see what the objections are and get a better idea of what people would want.

8 Likes

Can you find an example where we are considering widely imposing an artificial syntactic convention, forcing users to partially construct (still inaccurate) proof terms for some property as they write their program? Because one part of the point is that everything else that OCaml does is quite “light” and fits together nicely, and eio with ocap is not light, it is costly.

Could we see a concrete example of how “costly” the passing-around-stuff explicitly idioms become when relying on EIO? Do people here have a code snippet from an existing project that shows maintenance problems when using this approach at scale?

3 Likes

This is pretty much how it works. For example, Eio_linux.Low_level wraps the Linux io_uring API (no objects here), and likewise for Eio_luv.Low_level, etc. The main Eio API is a cross-platform abstraction over these low-level interfaces.

What to use here is less clear. For example, the original versions of Unix (see unix-6th - 2) provided an open call, like this:

int open(const char *path, int oflag, ...);

However, POSIX later added openat:

int openat(int fd, const char *path, int oflag, ...);

The purpose of the openat () function is to enable opening files in directories other than the current working directory without exposure to race conditions.
[ source ]

Lwt wrapped the original API, e.g.

Lwt_io.(with_file ~mode:input) "./README.md" (fun x -> ...)

Whereas Eio wraps the new one:

Eio.Dir.with_open_in cwd "README.md" (fun flow -> ...)

There is no “unopinionated” choice here. The choice is the old API (which will work on all systems, but prevents using features of newer ones) or the new one (which requires some best-efforts hacks to run on old systems).

On Linux, io_uring does not provide open, but you can pass -100 as the FD argument to openat to get the old behaviour. On FreeBSD, access to the old open call is blocked once you’ve called cap_enter, and only the new API will work.

Incidentally, the Eio.Dir API is named after Rust’s cap_std::fs::Dir, although at the moment it doesn’t have many functions, and I haven’t looked much at that API. The corresponding Mirage API is a bit different, and treats a path as a list of strings rather than as a single string as POSIX and Eio do.

If I want to capture the current time, I don’t want anything more complicated than Mtime_clock.now() or Ptime_clock.now() (even Eio.Clock.now ~clock:Global.clock () seems quite annoying if you add the initialization bit). I certainly don’t want the rest of the code to know I’m calling that ( it might just be for tracing or debugging or logging edit: @talex5 addressed that, sorry).

I think I’ll just add Eio_unix.sleep to match Lwt_unix.sleep. Adding sleeps to help debug race conditions seems like a good use for this, and we can think about this more later.

We’re writing OCaml, not Haskell, and side effects and impure functions are normal .

Yes, the Haskell choice of completely-pure or can-do-anything doesn’t seem that useful. I think “capability-safe” adds a nice middle ground: you can do side-effects, but bounded by the inputs you’re given.

Another question I’ve had for quite a while, is: why do we need a new library, instead of working effects into Lwt in a retrocompatible way? Something to get val await: 'a Lwt.t -> 'a would be incredibly helpful, and then more and more APIs could provide effect-based versions that eschew promises.

Yes, that’s what Eio_lwt.Promise.await_lwt does.

At the same time, that would allow the whole ecosystem of Lwt users to gradually migrate to effects without having to go through the pains of show-stopping migrations (remember python3?).

Yes. The Lwt to Eio migration guide walks you through the process of converting an Lwt program to Eio, bit by bit, keeping the whole program working during the transition. You can call Eio from Lwt and Lwt from Eio.

Incidentally, if you want your ambient access to sleep now, you can do it like this:

Lwt_eio.Promise.await_lwt (Lwt_unix.sleep 1.0);

I talked about Globals module before, and that’s how it works. You just start your program with:

Eio_main.run @@ fun env ->
Lwt_eio.with_event_loop ~clock:env#clock @@ fun () ->
  ...

Lwt_eio creates a top-level switch and within it sets the Lwt engine (a global), which also holds the clock. Then any Lwt code can call Lwt_unix.sleep as normal without needing a clock argument. The global switch also allows Lwt.async to bypass the normal structured concurrency:

open Lwt.Infix
open Eio.Std

let some_fn () =
  Lwt.async (fun () ->
      Lwt_unix.sleep 1.0 >|= fun () ->
      traceln "Surprise - I'm still running!"
  )

let () =
  Eio_main.run @@ fun env ->
  Lwt_eio.with_event_loop ~clock:env#clock @@ fun () ->
  some_fn ();
  traceln "Structured concurrency means everything is finished now!";
  Eio.Time.sleep env#clock 2.0
$ dune exec -- ./test.exe
+Structured concurrency means everything is finished now!
+Surprise - I'm still running!

This is why I think Eio is “unopinionated” - we can bypass the capabilities and the structured concurrency and run existing Lwt code within it unmodified.

8 Likes

I’m kind of repeating myself, but this is what I think we should aim for: composable primitives.

At that point in time, I will be suspicious of anything that will want to take over my main loop or invert control in invasive ways. I want a certain degree of freedom on how my application is to be structured, if only to experiment new ways effects could give me.

Also one thing that I think is not stressed enough is that most code out there should actually not use concrete effects or effectfull primitives at all.

I have been waiting for effects for the past 7 years precisely so that I don’t have to use them. They eventually allow the separation of concerns I was seeking for: most library codec code should not be concerned about how you move data in and out: that’s for the application to decide, using any kind of effects it finds desirable.

In that respect developing and offering simple standard abstractions in the Stdlib like I mentioned in this message which are effect friendly without mentioning any of them are more important to me than deciding on concrete effects.

Basically I think there’s the need for infrastructure and patterns so that effects can be used in a compositional way and that using something like eio or not rather becomes a matter of application choice.

7 Likes

I totally agree with you, but until we get typed effects, using effects everywhere will turn OCaml into a very unsafe language. It’s only reasonable currently IMO to use it for concurrency, and even there, it’s borderline.

3 Likes

To get in the direction of concrete examples I know about: one design pattern that isn’t being served well today is on-demand initialization of library/program constants. (I think I’ve discussed this in the past with @dbuenzli and his work also uses this.) With Sequential OCaml, libraries or programs can privately define something like

let my_config =
  let config = lazy (Config.parse Config.config_file_path) in
  fun () -> Lazy.force config

(* several library functions do this *)
let some_function ... =
  if (my_config ()).foo then ... else ...

(Note: maybe people use a different style where Config.config_file_path is passed around as a context parameter, etc, in fact this line may be sitting within a functor parametrized on Config or what have you. I think the underlying idiom remains similar if this is not a toplevel declaration, but a configuration value explicitly passed around.)

I don’t know how to do this well in a way that works with Multicore OCaml today.

Using lazy in this style is now invalid (a programming mistake) as soon as the library code that reads the configuration may run in several domains. (And the library author doesn’t know if it will, the client may or may not use Parmap around this library, etc.)

Some ways to do this not well with the stdlib currently are to do:

(* repeats computation on races,
    what if configuration gets mutated between calls? *)
let my_config =
  let r = Atomic.create None in
  fun () -> match Atomic.get r with
  | Some config -> config
  | None ->
    let config = Config.parse Config.config_file_path in
    Atomic.set r config; config

(* blocks a complete domain, not what we want *)
let my_config () =
  let m = Mutex.create () in
  let thunk = lazy (Config.parse Config.config_file_path) in
  fun () ->
    Mutex.lock m;
    let@ () = Fun.protect ~finally:(fun () -> Mutex.unlock m) in
    Lazy.force thunk

We need a common abstraction to do something like the second version, but will work well with user-defined concurrency libraries.

2 Likes

That would be good. I am not sure how typed effects would work but, because of its generic interface, one nice thing I have found about Lwt is the ease with which you can write your own type-safe wrappers for things like Node.js’s asynchronous i/o: construct your promise and resolver with Lwt.task, resolve the promise in the callback running in Node’s event loop with Lwt.wakeup_later, and then plug this straight into your program.

I can show a concrete example from the Dream port to EIO. Take this diff line. For example, this code

fun _ ->
  Dream.stream
    ~headers:["Content-Type", Dream.text_html]
    render

became this code

fun request ->
  Dream.stream
    ~headers:["Content-Type", Dream.text_html]
    request
    (render ~clock:env#clock)

…and as far as I can tell, if the env from env#clock is only accidentally in scope here to begin with, because Dream’s examples are so concise and “syntactic.” In reality, env or its parts would have to be passed around somehow, either to a function defining this handler, or more likely through Dream’s own helpers, and would have to be retrieved from requests or from somewhere. Note how Dream.stream gained an extra request argument.

Note the ugly use of the class system in what I intended to be a fluently written Web framework that uses only a simple core of OCaml concepts.

Note that that whole diff is just of a simple and absolutely trivial example without deeply nested function calls, inversions of control, or anything else of the sort, and it is already made much uglier and much more tedious by EIO, and that’s under circumstances where some of the capabilities are already accidentally in scope due to how accidentally concise the example otherwise is (which is due to Dream having the opposite kind of API, a concise one).

I grant that some of these things can be partially mitigated by making some different choices in EIO, and still having object capabilities, but I don’t see how you could eliminate the tedium.

Note that parts of Dream internally do pass clocks around the way this function had to pass the clock to render. However, that is internal, and by my choice as the maintainer of Dream, to accept a PR from @dinosaure for Mirage support, at mine (and his) maintenance cost, and it does not impact my users in any way. That is the dual of what is being proposed by EIO: which is potentially forcing global Mirage support to be implemented by all users of EIO at their cost.

3 Likes

As for the “at scale” part, that’s something I would be interested in, but obviously can’t provide. I would hope that we would have a case study from the EIO team, given they are suggesting this.

But I suspect that forced ocap will be ugly and very annoying to refactor at scale, especially when using any non-trivial control flow.

To comment further on this, passing I/O objects through requests, besides the body streams that they inherently carry, would be very disappointing. It would be very strange to have to take ambient (“global”) I/O system calls that are already accessible, stuff capabilities for them into essentially data objects (requests) that have nothing to do with those system calls, and pass them down to request handlers that way.

Another option is passing capabilities to each handler when it is created, either through capture of capability variables in scope (which restricts composition, since it restricts where handlers would have to be defined) or by pervasively introducing extra arguments, which can then be applied in that one scope where those capabilities are visible and would have been captured from.

Another option is to change Dream’s handler type from

request -> response promise

to

capabilities -> request -> response promise

But I am not doing that, because that is even more ugly, and more fragile than just having extra arguments for any handlers that may want to do some non-trivial I/O. It’s better for each handler to ask for its capabilities, than to have Dream try to guess which capabilities should be included in its default capability set.

All of this is completely unnecessary.

I think I may have a solution to your problem. I noticed in dream/example/h-sql at master · aantron/dream · GitHub that there is a similar problem with database connections, where every handler might need to use a database:

let () =
  Dream.run
  @@ Dream.logger
  @@ Dream.sql_pool "sqlite3:db.sqlite"
  @@ Dream.sql_sessions
  @@ Dream.router [
    Dream.get "/" (fun request ->
      let%lwt comments = Dream.sql request list_comments in
      Dream.html (render comments request));

Perhaps the same technique can be used for other things, like access to directories? In the Eio branch, Dream.run already takes env an argument, so it can pass anything it likes on this way. The only downside would be that people looking at the type of Dream.run might wonder why it needed so much stuff.

Another option is passing capabilities to each handler when it is created, either through capture of capability variables in scope

That’s how I would do it if I were making something like Dream, because I’d like to be able to look at the pipeline and quickly see where the risky bits were. That would probably also correspond to the bits I’d want to configure (e.g. where the static assets were stored).

But if you want to have just Dream.load "/any/path" work on its own, then a global (or domain-local) variable would probably be best. As the author of Dream, that’s your decision to make.

1 Like

Thanks for the example! Here is what I understand.

First: the implementation of Dream makes already interacts with capability-style systems for Mirage compatibility this is @dinosaure’s PR #102. This does not show in the interface, instead the Dream initialization code sets up some global mutable state to pass this capability to the internal modules.

One could use a similar approach for other capabilities than clocks. The code could lie in Dream, but it’s not so pleasant, or it could lie in a separate library that packages this logic for users of EIO (or “Mirage stuff”), to pass it as global state. I’m not sure if @talex5 Globals module is a thought experiment or something that exists, and if it does exactly this, but let’s say this sounds a plausible approach. If a PR had been sent to Dream to use this approach, you would be less unhappy about this.

(This doesn’t suggest that you should implement Dream that way. But at least that there are options to build on top of EIO/Mirage building blocks without provoking your ire.)

Second: Dream is a framework that executes client code. There is a problem with the question of how to pass capabilities to this client code. Basically, Dream should not need to know anything about the requirements/effects of its client, except for those that it manages (web stuff, SQL connections, logging, etc.). If a client want to sleep for N seconds (as in the examples), or do some other weird effect independent of Dream, you don’t want this concern to show up in the Dream API itself.

This is the problem of effect-polymorphism, if I understand correctly: you like it that there is nothing at all to say about extra effects from the client, instead of having to explicitly “pass them through Dream” in a way that feels artifical. Effect handlers provide this for free: if Dream doesn’t handle an effect, it goes up the stack to whoever is willing to handle it, without requiring any change to Dream code. On the other hand, it’s much less clear how to do this with capability-passing code (or with monads, etc.).

If one wanted to build something like this on top of Eio, I guess one would use a construction similar to the following: for each notion of capability provided by Eio, define a specific effect to get the capability. Then implement handlers for these effects that take EIO capabilities in arguments. Invoke those handlers, then pass control to Dream, then in user code use those effects. The user code can itself be written in capability-passing style or not:

(* if the client wants capability-style *)
EIO_effects.handle_env env @@ fun () ->
Dream.run ... @@ fun request ->
let env = EIO_effects.get_env in
... EIO.sleep env delay ...

(* if the client wants global-style *)
EIO_effects.handle_env env @@ fun () ->
Dream.run ... @@ fun request ->
... EIO.effects.do sleep delay ...

I’m not suggesting that users should write code like this, but maybe these patterns could be hidden by intermediary libraries. The point is that they don’t require the Dream API to know anything about capability-style.

This is all proof-of-concept-y, and I don’t know how it would actually work to build something like this. (In particular: is it practical to pass various capabilities separately, or does the system become unergonomic and you have to pass a full env wholesale?)

Finally: people could then ask: but if we are going to re-export everything as effects anyway, why not provide an effectful API from the start? I don’t have a strong opinion either way, I guess it’s a matter of who designs/implements what and what their preferences are. But I’ll note that @lpw25’s encoding of capability-style (or: passing explicit values) on top of effects is more complex than the other way around. To implement effects on top of explicit values, you just design env/reader effects to read those globalized values. To implement explicit values on top of effects, the values that you pass around are effect-handlers reified as polymorphic CPS functions.

1 Like

None of this provokes my “ire” because this is all internal to Dream (and any such feeling over ordinary words is purely in the mind of the reader). It’s not forced in any way globally on users. And as an author of more than one project, I wouldn’t like to have to repeat this or a variant of it for all projects. I had the choice (with @dinosaure) of “paying” for this in Dream.

This code is quite convoluted, and one of the parts of Dream I have a TODO to more fully digest, simplify, and rewrite. While a mature ocap library would undoubtedly do this in a simpler way, it would be regrettable if “end” users had to face these kinds of decisions in all cases, and had to “pull” values from some global objects even if they made the choice not to use capability-passing and not to pay to be compatible with Mirage.

That is, this style of programming is not free even with the globals, because you have to constantly explicitly make reference to the globals. Unless, of course, you have a version of EIO that has already composed all the functions with the right globals — but that would be a normal non-ocap interface, exactly what we are asking for as an “unopinionated” API to begin with.

EDIT: Indeed, Dream itself presents two final libraries that are, for the user, already composed with the right globals, presenting “normal” interfaces: dream with no excessive capabilities for “normal” users, and dream-mirage with its extra configuration for Mirage users. It would be weird to turn Dream “inside out” and expose the Mirage-compatibility-argument-passing on the outside and force it on all users in both environments. At function granularity rather than module granularity.

I follow (parts roughly) and appreciate the rest.

The Scala community has done a lot of nice work on capability-passing designs, and over time they grew some language support to make it nicer.

My impression (from afar) is that the Scala community is deeply divided between:

  • people who use Scala like a JVM-based ML (impure, code that looks like a mix of java and ML, etc.). That includes Martin Odersky.
  • people who try to rebuild Haskell on top of Scala (ZIO, cats, etc.).

I imagine the people playing with capabilities belong in the second group, not the first one. Also, they have language support we don’t: implicits for Scala 2, typeclasses for Scala 3. OCaml is a lot more explicit in general.

But then, if everyone does something different, there is a risk of fragmentation of the ecosystem where we end up with 5 different concurrency abstractions that can’t talk to each other

In general, trying to solve fragmentation with 5 libraries doesn’t involve building a 6th one that has an even wider scope. As mentioned earlier in this thread, Eio seems very big and monolithic.

What I would hope for, personally, is a small, carefully chosen set of effects (not handlers) that spell the grammar for non blocking IO and lightweight concurrency. This way I can remove the functor (Io : IO) -> … in some of my code and use these instead. Then everyone is free to implement these handlers in their own event loop. This is like having Seq in the stdlib, and everybody building on that.

but I haven’t seen proposals in this thread that sound like better answers.

What about building on top of Lwt?

Could we see a concrete example of how “costly” the passing-around-stuff explicitly idioms become when relying on EIO

Imagine using directories or xdg to store some local data, and having to carry that from your main down to the deepest function that wants to use a cache. openat has its use, but don’t just deprecate open :angry:


Unrelatedly:

For example, Lazy values cannot be used concurrently, and the reason why is that we don’t know how to block code that forces a thunk that is already being forced by someone else.

I think Haskell sends the second thread into a “black hole”? On Lazy.force foo you could swap the thunk inside foo with one that suspends-and-yields. Maybe the problem is that domains don’t always come with a scheduler that provides the “yield” primitive, or maybe it just parks the current Thread.t.

3 Likes

My impression (from afar) is that the Scala community is deeply divided between:

  • people who use Scala like a JVM-based ML (impure, code that looks like a mix of java and ML, etc.). That includes Martin Odersky.
  • people who try to rebuild Haskell on top of Scala (ZIO, cats, etc.).
    I imagine the people playing with capabilities belong in the second group, not the first one.

actually it’s the other way around
Capture Checking (epfl.ch)

1 Like

TIL. But these “capabilities” look like an alternative to borrow checking, so I’m not sure it’s the same thing.

1 Like

It does exist; Lwt_eio does this. In fact, because the Dream PR is using Lwt_eio already, the ~clock argument isn’t actually needed. You can do a sleep like this right now:

Lwt_eio.Promise.await_lwt (Lwt_unix.sleep 1.0)

I thought it was interesting to highlight the fact that this HTTP resource includes delays. However, based on the feedback in this thread we’ll add Eio_unix.sleep so this works directly without having to depend on Lwt_eio.

…which opens another topic that I have not raised yet, how much more pervasive would the change be if Dream was fully ported to EIO with capability objects, instead of being in an intermediate porting stage where it is still largely using Lwt_eio? Thus, if a person would look at the PR right now, they would not get a complete picture of what a genuine port to EIO would entail.

I think it’s good that the PR actually materially used EIO itself for at least the sleep in the portion that I linked to, and we got to see the actual EIO usage syntax (at least as of EIO 0.1):

fun request ->
  Dream.stream
    ~headers:["Content-Type", Dream.text_html]
    request
    (render ~clock:env#clock)

Other parts of even that same example, AFAIK, are still using Lwt_eio, so we don’t see the full EIO impact.