Effects with Lwt, a dead end for now?

Learning
, , ,
1

This thread is basically a discussion about this issue:

I am trying to integrate irmin into an application that makes heavy use of effects, in particular Algaeff, Asai and Eio.

Now, irmin uses lwt and while there is a solution for running lwt and eio code together, but it seems that in general, I can’t make use of code that uses effect handlers.

The question I have is: What is the best option for making progress?

  • Is my only option to wait for the issue at the lwt repo to resolve?
  • Should I study Lwt_eio? My thinking is that maybe that code can be generalised/adapted somehow, since it is an example of using effect handlers in conjunction with lwt. It might be a problem that the effect handlers that are causing issues in my code are library code, so I can’t modify it.
  • …?

There is an open PR for an Eio backend for irmin, but I assume that using irmin would still require using the monadic lwt API (can’t tell from the PR alone).

I’ll work on putting together a minimal working example of the code that causes my issues so that we have something more concrete to talk about. In the meantime I am eager for discussion. Thanks!

1 Like
2

Hey @kentookura,

The PRs you referenced from Irmin are to remove Lwt as much as possible. The API will be Lwt free – see the updated store interface. The implementation could use lwt_eio under the hood for some libraries until they are also ported away from Lwt.

Do share that minimal example once you have it to help explain exactly the issues you are hitting :))

3 Likes
3

I created a repo:

1 Like
4

Thanks for the reproduction code @kentookura! I think this might be a case of issues arising due to the nesting of the effect handlers (I wrote about this in a blog post). In the example you have:

  Eio_main.run @@ fun env ->
  Reporter.run ~emit:Term.display ~fatal:Term.display @@ fun () ->
  Lwt_eio.with_event_loop ~clock:env#clock @@ fun _ ->

The third line forks a new fiber and is handled by the first line escaping the scope of the handler installed by the second line – hence “unhanded effect”. In this case, I think it is fine to fix this by changing lines 1 and 2 so the reporter sits on top. Eio could also perhaps offer a better mechanism for users to add additional effect handlers (which it does have internally).

I forked the repo and converted it to the Irmin interface in the PR you mentioned here: irmin-effects-demo/bin/main.ml at escaping-effects · patricoferris/irmin-effects-demo · GitHub – it doesn’t fork anymore and so the old code works. However, I added a fork just to help show what was happening. Hope this helps clear things up a little ?

3 Likes
5

I’ve successfully pinned irmin at the eio branch. However, the code you added does not work for me, as Irmin_fs_unix.conf is reported as an unbound value. I made some modifications to the code so it builds, which you can look at on github. Unfortunately, a similar issue arises:
Fatal error: exception Stdlib.Effect.Unhandled(Irmin__Conf.Env.Fs)
This happens both with both permutations of Eio_main.run and Reporter.run

6

This still happens when removing all asai code, so it seems that irmin-fs itself fails:

module Store = Irmin_fs_unix.KV.Make (Irmin.Contents.String)

let test () =
  let config = Irmin_fs.config "store" in
  let repo = Store.Repo.v config in
  let main = Store.main repo in
  let info () = Store.Info.v 0L in
  let key = "Hello" in
  let _ = Store.set main [ key ] ~info "world!" in
  let v = Store.get main [ key ] in
  print_endline v

let () = Eio_main.run @@ fun env -> test ()
7

Looks like maybe you are pinning Irmin to a slightly older WIP commit – I was trying it with @art-w’s latest irmin-fs library changes: irmin-fs: eio backend · art-w/irmin@9d606d7 · GitHub

2 Likes
8

Yes, you’re right. Thanks a lot!

1 Like