Working with both Lwt and Async in the same project?

I want to use one library that works with Lwt (multipart_form_lwt) and another library that works with Async (mssql) in a project that is currently written using Lwt. I can envisage a variety of solutions:

  • Use the lower-level API of multipart_form to get it working with Async and port the whole program from Lwt to Async just so I can work with mssql in the hope that I do not then need an Lwt-only library in the future.
  • Split off mssql exposed as a web service on the same machine.
  • Find some kind of in-process interop between Lwt and Async. I’m not sure if this is as simple as just writing everything in CPS or if this is impossible.

I’m guessing I am not the first person to have faced this challenge so I’m wondering if there is a preferred solution?

1 Like

I’ve never used Async, but Lwt has Lwt_preemptive.run_in_main, which you could call from the Async thread presumably (with Lwt main running in a separate thread).

1 Like

Another option: use the lower-level API of mssql, which is ocaml-freetds. E.g.: https://github.com/kennknowles/ocaml-freetds/blob/master/examples/dbi_example.ml

2 Likes

I would also consider functorizing mssql to support Async/blocking IO. From a brief look at the code I think this would be non-trivial but doable.

1 Like

Functorizing is an interesting idea but I just the advice here (from Cohttp experience) was against doing so:
http://rgrinberg.com/posts/abandoning-async/
However, I found an old post here that mentions the idea of forking a child process that uses the other async library. That sounds like a great idea to me so I’ll give that a go first…

Functorizing works well sometimes, but it will not be very appealing in this case. The library in question (ocaml-mssql?) is split into many modules and relies on a few modules from async that might not have immediate Lwt analogues.

Why don’t you just the parent library ocaml-mssql is based on? Freetds should be quite simple to wrap with Lwt based connection handling.

3 Likes

Definitely a possibility. Only problem is that I’m new to these async libraries and have no idea what is going on yet. :slight_smile:

Hint: use the Lwt_pool module to set up a connection pool. It’s pretty nicely documented.

You might find some of the discussion here useful.

Yeah, you could probably make something similar to Mssql using Lwt_pool + however you spawn background threads in Lwt.

I think we’d be open to making mssql work on Lwt, but it’s probably non-trivial since the library uses Sequencers and threads.

(Disclaimer: I’m not offering any solution (nor advice), just sharing some folklore.)

A long time ago, @jeremiedimino and I wrote GitHub - janestreet/lwt-async: Lwt with async backend, which defines 'a Lwt.t as 'a Or_exn.t Async.Deferred.t (see here).

Btw, I found the raw DB interaction module that ocaml-freetds uses: [modcaml] Contents of /ocamldbi/dbi_freetds.mli … this seems to be the actual thing you’d use to connect, run queries, etc.