I’m curious about how the OCaml 5 effect system is being used in real world.
I checked the posts tagged with “effects”. It looks like there are a few libraries that implement threading with the effect system, replacing solutions like Lwt (which implemented it in C IIUC).
There is also one cool use to convert a push API (that calls a callback with generated things) to a lazy sequence. I can see how this can help when using a third-party push API without having to pass callbacks and pass the control to the library until the generation finishes.
However I’m not seeing any uses of them for dependency injection or to allow testing, which are how effects are commonly used in other langauges.
Things like: you define effects for database operations. In production you handle the effects by calling the actual database implementation. In testing you can use different handlers to emulate different failure modes, run tests in-memory without a database server etc.
Clearly this can be done with OCaml effect system as well as it’s quite general (with continuations being first class values, and without a static type system to limit what you can express). Is this done? Is anyone doing this kind of thing in production?
If not, why?
If you have any other use case I’d be interested in hearing them as well.
Thanks!
PS. Regarding the control inversion example, a while ago I compared push and iterator APIs for generating events (or tokens, in the context of parsing) here (see also follow-up posts at the home page). I wanted to experiment with inverting the control to convert a push API to an iterator one, but existing language with built-in static effect systems were are very frustrating to use, as they’re all new and experimental. I tried: Flix, Effekt, Koka. Most library-level solutions in languages without built-in effects don’t have first class continuations (e.g. effectful) or have extremely complicated types that make everything extremely painful (e.g. heftia). So I gave up. It would be interesting to see how these APIs perform in OCaml, with an additional benchmark variant where we convert the push API to iterator/lazy stream using effects.
Until OCaml has typed effects, it would be advisable to limit their usage to specific things like concurrency. Concurrency specifically benefits from no longer requiring monadic code throughout. Otherwise, we’re heading towards a very untyped and unsafe version of OCaml, which is not what most OCamler desire. I don’t know if this thought process is behind the lack of their usage in other domains, but it’s certainly something that would dissuade me from trying out libraries that make use of effects in other contexts.
I need to rewrite the whole thing with the new syntax. But the main idea works.
At some point I even had an intermediate version where the parser was doing the interpretation directly (rather than creating an AST that is interpreted afterwards). It’d be a pain to do by passing the PRNG state around, but it’s very easy to perform the effect directly from the .mly
I don’t know if you count that as real-world, but you can opam install odds and then roll 1d6
Similar to the above example, I’ve also experimented with effects as a way to manage state, and I’ve been mostly pleased with doing so. Instead of using a global mutable state or passing state around explicitly to functions, you can manage it within your effect handler. I especially like using shallow effect handlers for this, since you can manage it in a functional style sans mutation.
My understanding has been that typed effects would just make the failure to use effect handlers a compile-time error rather than raising an Effect.Unhandled exception at runtime. While that would be a good improvement, the current version doesn’t seem any less untyped or unsafe than regular OCaml code, at least in conventional sense of typing and safety. Is there more to typed effects that I’m not aware of?
Effects are very similar to exceptions. There’s a reason exceptions are discouraged in modern OCaml code: code that seems to behave just fine can suddenly throw an exception in specific circumstances. There’s no way to know that the exception is hiding there, and it can come from deep inside libraries that the user doesn’t even know are being used.
If we adopt untyped effects throughout, we lose any guarantee that OCaml won’t suddenly crash with an unhandled effect of some sort that is hiding deep inside our code. This is potentially even worse than exceptions, which are supposed to be in truly exceptional cases. I can tolerate this for concurrency, because the benefit of not writing monadic code throughout the codebase is huge. For other purposes, it’s adding a significant amount of risk to a mostly safe language.
My current thinking is that none of fully dynamic or fully static effect systems will be satisfactory from a practical point of view. They will both be painful to use.
(The issue says “exceptions” but that’s just a stepping stone, Fir will eventually have effects using the same type system features as checked exceptions.)
Static effects will always limit you in various ways in terms of flexilibity, which is fine in real world (e.g. you may want guarantees that a certain callback won’t have certain side effects), but not if I’m looking to experiment with e.g. new possibilities in testing. I can do this now in OCaml.
That’s why I was curious to see if anyone’s experimenting with this kind of thing already.
This mostly confirms my impressions. To be clear, I don’t disagree with your point that effects are probably best used judiciously. I’m only skeptical that they’re worse than exceptions. Given that the OCaml ecosystem recommends using exceptions for error handling and even control flow, the “unsafe” version of OCaml that also uses effects seems to me like the version that already exists.
Perhaps time will tell if Effect.Unhandled becomes like the infamous Not_found exception that tends to appear when least expected…
I’ve also had the impression that typed effects will probably not be on the horizon for a while anyway, although I’m happy to be proven wrong about that.