Eio 0.1 - effects-based direct-style IO for OCaml 5

I totally agree with you, but until we get typed effects, using effects everywhere will turn OCaml into a very unsafe language. It’s only reasonable currently IMO to use it for concurrency, and even there, it’s borderline.

3 Likes

To get in the direction of concrete examples I know about: one design pattern that isn’t being served well today is on-demand initialization of library/program constants. (I think I’ve discussed this in the past with @dbuenzli and his work also uses this.) With Sequential OCaml, libraries or programs can privately define something like

let my_config =
  let config = lazy (Config.parse Config.config_file_path) in
  fun () -> Lazy.force config

(* several library functions do this *)
let some_function ... =
  if (my_config ()).foo then ... else ...

(Note: maybe people use a different style where Config.config_file_path is passed around as a context parameter, etc, in fact this line may be sitting within a functor parametrized on Config or what have you. I think the underlying idiom remains similar if this is not a toplevel declaration, but a configuration value explicitly passed around.)

I don’t know how to do this well in a way that works with Multicore OCaml today.

Using lazy in this style is now invalid (a programming mistake) as soon as the library code that reads the configuration may run in several domains. (And the library author doesn’t know if it will, the client may or may not use Parmap around this library, etc.)

Some ways to do this not well with the stdlib currently are to do:

(* repeats computation on races,
    what if configuration gets mutated between calls? *)
let my_config =
  let r = Atomic.create None in
  fun () -> match Atomic.get r with
  | Some config -> config
  | None ->
    let config = Config.parse Config.config_file_path in
    Atomic.set r config; config

(* blocks a complete domain, not what we want *)
let my_config () =
  let m = Mutex.create () in
  let thunk = lazy (Config.parse Config.config_file_path) in
  fun () ->
    Mutex.lock m;
    let@ () = Fun.protect ~finally:(fun () -> Mutex.unlock m) in
    Lazy.force thunk

We need a common abstraction to do something like the second version, but will work well with user-defined concurrency libraries.

2 Likes

That would be good. I am not sure how typed effects would work but, because of its generic interface, one nice thing I have found about Lwt is the ease with which you can write your own type-safe wrappers for things like Node.js’s asynchronous i/o: construct your promise and resolver with Lwt.task, resolve the promise in the callback running in Node’s event loop with Lwt.wakeup_later, and then plug this straight into your program.

I can show a concrete example from the Dream port to EIO. Take this diff line. For example, this code

fun _ ->
  Dream.stream
    ~headers:["Content-Type", Dream.text_html]
    render

became this code

fun request ->
  Dream.stream
    ~headers:["Content-Type", Dream.text_html]
    request
    (render ~clock:env#clock)

…and as far as I can tell, if the env from env#clock is only accidentally in scope here to begin with, because Dream’s examples are so concise and “syntactic.” In reality, env or its parts would have to be passed around somehow, either to a function defining this handler, or more likely through Dream’s own helpers, and would have to be retrieved from requests or from somewhere. Note how Dream.stream gained an extra request argument.

Note the ugly use of the class system in what I intended to be a fluently written Web framework that uses only a simple core of OCaml concepts.

Note that that whole diff is just of a simple and absolutely trivial example without deeply nested function calls, inversions of control, or anything else of the sort, and it is already made much uglier and much more tedious by EIO, and that’s under circumstances where some of the capabilities are already accidentally in scope due to how accidentally concise the example otherwise is (which is due to Dream having the opposite kind of API, a concise one).

I grant that some of these things can be partially mitigated by making some different choices in EIO, and still having object capabilities, but I don’t see how you could eliminate the tedium.

Note that parts of Dream internally do pass clocks around the way this function had to pass the clock to render. However, that is internal, and by my choice as the maintainer of Dream, to accept a PR from @dinosaure for Mirage support, at mine (and his) maintenance cost, and it does not impact my users in any way. That is the dual of what is being proposed by EIO: which is potentially forcing global Mirage support to be implemented by all users of EIO at their cost.

3 Likes

As for the “at scale” part, that’s something I would be interested in, but obviously can’t provide. I would hope that we would have a case study from the EIO team, given they are suggesting this.

But I suspect that forced ocap will be ugly and very annoying to refactor at scale, especially when using any non-trivial control flow.

To comment further on this, passing I/O objects through requests, besides the body streams that they inherently carry, would be very disappointing. It would be very strange to have to take ambient (“global”) I/O system calls that are already accessible, stuff capabilities for them into essentially data objects (requests) that have nothing to do with those system calls, and pass them down to request handlers that way.

Another option is passing capabilities to each handler when it is created, either through capture of capability variables in scope (which restricts composition, since it restricts where handlers would have to be defined) or by pervasively introducing extra arguments, which can then be applied in that one scope where those capabilities are visible and would have been captured from.

Another option is to change Dream’s handler type from

request -> response promise

to

capabilities -> request -> response promise

But I am not doing that, because that is even more ugly, and more fragile than just having extra arguments for any handlers that may want to do some non-trivial I/O. It’s better for each handler to ask for its capabilities, than to have Dream try to guess which capabilities should be included in its default capability set.

All of this is completely unnecessary.

I think I may have a solution to your problem. I noticed in dream/example/h-sql at master · aantron/dream · GitHub that there is a similar problem with database connections, where every handler might need to use a database:

let () =
  Dream.run
  @@ Dream.logger
  @@ Dream.sql_pool "sqlite3:db.sqlite"
  @@ Dream.sql_sessions
  @@ Dream.router [
    Dream.get "/" (fun request ->
      let%lwt comments = Dream.sql request list_comments in
      Dream.html (render comments request));

Perhaps the same technique can be used for other things, like access to directories? In the Eio branch, Dream.run already takes env an argument, so it can pass anything it likes on this way. The only downside would be that people looking at the type of Dream.run might wonder why it needed so much stuff.

Another option is passing capabilities to each handler when it is created, either through capture of capability variables in scope

That’s how I would do it if I were making something like Dream, because I’d like to be able to look at the pipeline and quickly see where the risky bits were. That would probably also correspond to the bits I’d want to configure (e.g. where the static assets were stored).

But if you want to have just Dream.load "/any/path" work on its own, then a global (or domain-local) variable would probably be best. As the author of Dream, that’s your decision to make.

1 Like

Thanks for the example! Here is what I understand.

First: the implementation of Dream makes already interacts with capability-style systems for Mirage compatibility this is @dinosaure’s PR #102. This does not show in the interface, instead the Dream initialization code sets up some global mutable state to pass this capability to the internal modules.

One could use a similar approach for other capabilities than clocks. The code could lie in Dream, but it’s not so pleasant, or it could lie in a separate library that packages this logic for users of EIO (or “Mirage stuff”), to pass it as global state. I’m not sure if @talex5 Globals module is a thought experiment or something that exists, and if it does exactly this, but let’s say this sounds a plausible approach. If a PR had been sent to Dream to use this approach, you would be less unhappy about this.

(This doesn’t suggest that you should implement Dream that way. But at least that there are options to build on top of EIO/Mirage building blocks without provoking your ire.)

Second: Dream is a framework that executes client code. There is a problem with the question of how to pass capabilities to this client code. Basically, Dream should not need to know anything about the requirements/effects of its client, except for those that it manages (web stuff, SQL connections, logging, etc.). If a client want to sleep for N seconds (as in the examples), or do some other weird effect independent of Dream, you don’t want this concern to show up in the Dream API itself.

This is the problem of effect-polymorphism, if I understand correctly: you like it that there is nothing at all to say about extra effects from the client, instead of having to explicitly “pass them through Dream” in a way that feels artifical. Effect handlers provide this for free: if Dream doesn’t handle an effect, it goes up the stack to whoever is willing to handle it, without requiring any change to Dream code. On the other hand, it’s much less clear how to do this with capability-passing code (or with monads, etc.).

If one wanted to build something like this on top of Eio, I guess one would use a construction similar to the following: for each notion of capability provided by Eio, define a specific effect to get the capability. Then implement handlers for these effects that take EIO capabilities in arguments. Invoke those handlers, then pass control to Dream, then in user code use those effects. The user code can itself be written in capability-passing style or not:

(* if the client wants capability-style *)
EIO_effects.handle_env env @@ fun () ->
Dream.run ... @@ fun request ->
let env = EIO_effects.get_env in
... EIO.sleep env delay ...

(* if the client wants global-style *)
EIO_effects.handle_env env @@ fun () ->
Dream.run ... @@ fun request ->
... EIO.effects.do sleep delay ...

I’m not suggesting that users should write code like this, but maybe these patterns could be hidden by intermediary libraries. The point is that they don’t require the Dream API to know anything about capability-style.

This is all proof-of-concept-y, and I don’t know how it would actually work to build something like this. (In particular: is it practical to pass various capabilities separately, or does the system become unergonomic and you have to pass a full env wholesale?)

Finally: people could then ask: but if we are going to re-export everything as effects anyway, why not provide an effectful API from the start? I don’t have a strong opinion either way, I guess it’s a matter of who designs/implements what and what their preferences are. But I’ll note that @lpw25’s encoding of capability-style (or: passing explicit values) on top of effects is more complex than the other way around. To implement effects on top of explicit values, you just design env/reader effects to read those globalized values. To implement explicit values on top of effects, the values that you pass around are effect-handlers reified as polymorphic CPS functions.

1 Like

None of this provokes my “ire” because this is all internal to Dream (and any such feeling over ordinary words is purely in the mind of the reader). It’s not forced in any way globally on users. And as an author of more than one project, I wouldn’t like to have to repeat this or a variant of it for all projects. I had the choice (with @dinosaure) of “paying” for this in Dream.

This code is quite convoluted, and one of the parts of Dream I have a TODO to more fully digest, simplify, and rewrite. While a mature ocap library would undoubtedly do this in a simpler way, it would be regrettable if “end” users had to face these kinds of decisions in all cases, and had to “pull” values from some global objects even if they made the choice not to use capability-passing and not to pay to be compatible with Mirage.

That is, this style of programming is not free even with the globals, because you have to constantly explicitly make reference to the globals. Unless, of course, you have a version of EIO that has already composed all the functions with the right globals — but that would be a normal non-ocap interface, exactly what we are asking for as an “unopinionated” API to begin with.

EDIT: Indeed, Dream itself presents two final libraries that are, for the user, already composed with the right globals, presenting “normal” interfaces: dream with no excessive capabilities for “normal” users, and dream-mirage with its extra configuration for Mirage users. It would be weird to turn Dream “inside out” and expose the Mirage-compatibility-argument-passing on the outside and force it on all users in both environments. At function granularity rather than module granularity.

I follow (parts roughly) and appreciate the rest.

The Scala community has done a lot of nice work on capability-passing designs, and over time they grew some language support to make it nicer.

My impression (from afar) is that the Scala community is deeply divided between:

  • people who use Scala like a JVM-based ML (impure, code that looks like a mix of java and ML, etc.). That includes Martin Odersky.
  • people who try to rebuild Haskell on top of Scala (ZIO, cats, etc.).

I imagine the people playing with capabilities belong in the second group, not the first one. Also, they have language support we don’t: implicits for Scala 2, typeclasses for Scala 3. OCaml is a lot more explicit in general.

But then, if everyone does something different, there is a risk of fragmentation of the ecosystem where we end up with 5 different concurrency abstractions that can’t talk to each other

In general, trying to solve fragmentation with 5 libraries doesn’t involve building a 6th one that has an even wider scope. As mentioned earlier in this thread, Eio seems very big and monolithic.

What I would hope for, personally, is a small, carefully chosen set of effects (not handlers) that spell the grammar for non blocking IO and lightweight concurrency. This way I can remove the functor (Io : IO) -> … in some of my code and use these instead. Then everyone is free to implement these handlers in their own event loop. This is like having Seq in the stdlib, and everybody building on that.

but I haven’t seen proposals in this thread that sound like better answers.

What about building on top of Lwt?

Could we see a concrete example of how “costly” the passing-around-stuff explicitly idioms become when relying on EIO

Imagine using directories or xdg to store some local data, and having to carry that from your main down to the deepest function that wants to use a cache. openat has its use, but don’t just deprecate open :angry:


Unrelatedly:

For example, Lazy values cannot be used concurrently, and the reason why is that we don’t know how to block code that forces a thunk that is already being forced by someone else.

I think Haskell sends the second thread into a “black hole”? On Lazy.force foo you could swap the thunk inside foo with one that suspends-and-yields. Maybe the problem is that domains don’t always come with a scheduler that provides the “yield” primitive, or maybe it just parks the current Thread.t.

3 Likes

My impression (from afar) is that the Scala community is deeply divided between:

  • people who use Scala like a JVM-based ML (impure, code that looks like a mix of java and ML, etc.). That includes Martin Odersky.
  • people who try to rebuild Haskell on top of Scala (ZIO, cats, etc.).
    I imagine the people playing with capabilities belong in the second group, not the first one.

actually it’s the other way around
Capture Checking (epfl.ch)

1 Like

TIL. But these “capabilities” look like an alternative to borrow checking, so I’m not sure it’s the same thing.

1 Like

It does exist; Lwt_eio does this. In fact, because the Dream PR is using Lwt_eio already, the ~clock argument isn’t actually needed. You can do a sleep like this right now:

Lwt_eio.Promise.await_lwt (Lwt_unix.sleep 1.0)

I thought it was interesting to highlight the fact that this HTTP resource includes delays. However, based on the feedback in this thread we’ll add Eio_unix.sleep so this works directly without having to depend on Lwt_eio.

…which opens another topic that I have not raised yet, how much more pervasive would the change be if Dream was fully ported to EIO with capability objects, instead of being in an intermediate porting stage where it is still largely using Lwt_eio? Thus, if a person would look at the PR right now, they would not get a complete picture of what a genuine port to EIO would entail.

I think it’s good that the PR actually materially used EIO itself for at least the sleep in the portion that I linked to, and we got to see the actual EIO usage syntax (at least as of EIO 0.1):

fun request ->
  Dream.stream
    ~headers:["Content-Type", Dream.text_html]
    request
    (render ~clock:env#clock)

Other parts of even that same example, AFAIK, are still using Lwt_eio, so we don’t see the full EIO impact.

I’d like to come back to a couple of points:

Why is this the case? Don’t fibres inherit handlers? If not, how is I/O being handled in forked fibres? Is a new I/O handler installed in each fibre?

  • How long it takes to handle an effect depends on how many effect handlers are on the fibre’s stack. Eio only installs one handler per fibre but if you add your own it will slow things down a bit.

Isn’t this a rather important limitation of effect handlers in OCaml? Handlers are supposed to make it easy to control effectful code in a fine-grained way. If their performance in real-world scenarios is not scalable, does this mean there is more work to be done here? Or is this specifically about Eio’s fibres?

@gasche > Are you suggesting adding a Yield effect to the stdlib that Lazy.force could perform? On the default runtime, this would block the domain until resolution but a library like eio could instead switch to another fibre?

Reading this thread, one gets the impression that the design is horrible… but reading the API documentation, it actually looks really good! I believe the explicit capabilities will make it a lot easier to test and mock side effects.

The use of objects is surprising, but in practice they require a lot less syntax than first class modules or records when it comes to subtyping (to remove capabilities).

val ftp : < net : Net.t ; cwd : Dir.t ; ..> -> unit

So for example, this ftp function would accept the default env : Eio.Stdenv.t without any annotation. The Dream clock example could just do (render ~env), since render will infer its argument as env : < clock ; ..> if it’s only used for sleeping. The two_way pipes is also a compelling use-case to reduce the API surface.

I was a bit confused by the semi-pervasive use of Switch, because I thought the name implied that they would be used for early cancellation, when in practice functions like open_in actually requires them to ensure deferred clean up? I guess switches will only show up when something fancy is going on with the resources… which is the case if you only use the structured with_stuff helpers, so they should be the recommended default in the documentation? (introduced before the low-level switch version? like Fibre does)

Should Promise.create accept an optional switch to guard against a never resolved bug due to a cancellation?

Regarding capabilities and effects, I expect a similar story to exceptions to play out where best practices evolve from the unchecked default to something more explicit:

val find : key -> t -> value (** @raise Not_found *)

val find_opt : key -> t -> value option

Yet even the later has disadvantages and doesn’t scale to effects system. We could have instead chosen to give permission to the function to fail:

val find_exn : key -> t -> exn -> value

There’s some precedent in dependently typed languages, where exn would be a proof of mem key t usable in the absurd case that the key is missing. This presumes that exceptions and effects are not globally available, but rather are obtained by setting up a match ... with exception handler (which is reminiscent of Donald Knuth’s event indicators in “Structured Programming with goto Statements” from 1974!)

Of course, this only makes sense if these capabilities can’t escape their scope. I would really like something like Scala’s capture checking linked by @0xa2c2a, it looks a lot more versatile than the typed effects arrows (and wouldn’t -[fs: Dir.t; ..]-> have a terrifying impact on libraries like Dream?)

(Anyway, perhaps eio is hiding effect handlers too much at a time where people really want to play with them. The global ref escape hatch “feels” bad even if it’s easy to setup.)

3 Likes

We often write code in a similar way if we need to pass in values in for testing — however by far not always, and not when we can get away without having to do that. The question is why should this convention be imposed by a foundational library?

One may like this kind of approach for testing their particular project (and I do, for some of my projects). How does it follow from that that this approach should pervade throughout the rest of the ecosystem?

On another axis, when I am testing my projects, I choose what I need to mock. Why should a foundational library pervasively force me to mock a larger set of entities than I need?

I can often get away with passing in mock entities directly to the functions of an inner module under test, for its unit tests. This allows me to keep the outer modules of the project cleaner, if I don’t need to mock for the kind of integration tests I am writing. Why should a foundational library force me to thread mock values all the way through the entire program, and pass them in at the main function? If I do need to do pass in values through the outer modules, that’s fine, but why should I and everyone be forced to?

Why should the extra cost of doing this be spread to all other users and projects, and for me, to more entities than I need treated in this way?

If I’ve adopted the convention that I’m going to pass some extra objects around in some of the functions of my project, why should some random other person working on something totally unrelated automatically have to do that as well?

For some context, the Eio README does state that it aims at becoming foundational (or at least standard) (at least at the ecosystem level):

It would be very beneficial to use this opportunity to standardise a single concurrency API for OCaml, and we hope that Eio will be that API.

With that in mind, I’m happy to see discussions of Eio interface as a candidate for the ecosystem standard IO library. I think a library with this ambition needs to be subject to a serious design and usability critique.


Now, I don’t have much to contribute on the ocap side of things, others seem to have that discussion rolling already. But I’d like to contribute a couple of cents on a different topic:

In the Eio README, is the following paragraph:

If you call a function without giving it access to a switch, then when the function returns you can be sure that any fibres it spawned have finished, and any files it opened have been closed. This works because Eio does not provide e.g. a way to open a file without attaching it to a switch. If a function doesn’t have a switch and wants to open a file, it must use Switch.run to create one. But then the function can’t return until Switch.run does, at which point the file is closed.

In essence, you apply region-type discipline on open file descriptors. A switch is a region, the file-opening primitives take that region as parameter, and all the files are closed when the region goes out of scope. From there a couple questions:

  • Are there other resources than fibres and file-descriptors which are handled this way?

  • Is it possible to attach other, user-defined resources to a switch? Can I have a stateful data-structure (say a cache, a task queue. or user dialog displayed on the screen) that I can set to automatically clear and close when the switch has finished running? If not, is this under consideration?

  • Is it possible to re-attach some resources to inner switches? Say I Switch.run (fun sw -> … let f = open sw "some-file" in … Switch.run ~transfer:[f] (fun innersw -> … (* f is closed here *)) …). This is somewhat equivalent to transferring the ownership in rust. If not, are there plans to support this kind of ownership transfer?

  • Conversely, is it possible to explicitly reattach a resource to an outer context? This is useful when you have some complex logic which you want to isolate in a switch (because you need some cancellation behaviour, because the library is made like this, etc.) which returns a resource. Is this only possible by passing the parent switch around as in `Switch.run (fun outersw → … Switch.run (fun sw → go_fetch_resource outersw sw) …)``?

  • And finally a more < waves hands > foundational </ waves hands > question: Considering there is no type-level support for maintaining any invariant around switches, resources, and scope, should this be so tightly integrated in the API? What happens if you leak the switch itself (let dead_switch = Switch.run (fun sw -> sw) in …) or if you leak a file-descriptor (you just get some exceptions when using it I guess).

    An alternative —which might be seen as more compatible with the standard/foundational aspirations of Eio— is to provide the tools to manage resources without designating any specific resources as such. Specifically, provide a notion of resource switch, provide a facility to attach a clean-up function to it (val at_switch_exit : switch -> (unit -> unit) -> unit), and that’s it. Then, on top of that foundation, you can have a wrapper (call it Eio_with_region_based_files or something snappier) you can build the current API of Eio with its runtime leak-proof file-descriptor management.

    I think that the current design (of the switch-resource system) is very system-programming oriented. It’s easy to explain in terms of the rust borrow system but with a runtime value rather than a compile-time type checker. And the rust borrow system has been designed based on best practices established in the systems programming community at large. But how well does it fits into other areas of programming? Will it require passing parent switches around the call graph just to be able to return resources?

1 Like

While toying a bit with a Fiber API I would like to use before going to sleep (don’t do that, you’ll get insomnia afterwards :–), I wasn’t really convinced by Eio's design on these matters that is the switch and cancellation aspects.

It looks like a recipe for convoluted control and cancellation flow (e.g. the ability to switch cancellation context) and, I suspect, more complicated than I would like it to be. Besides passing cancellation tokens (switch values) around explicitly will likely turn out to be error prone.

We already have a notion of scope in OCaml, it’s function scope and we already have at least one combinator to deal with resources in the Stdlib, that’s Fun.protect.

So I was rather converging on a simple design with enforced structured concurrency that would make good use of these pre-existing tool, rather than introduce new non-language aware notions of scope.

3 Likes