Working with both Lwt and Async in the same project?

[Jon_Harrop]

I want to use one library that works with Lwt (multipart_form_lwt) and another library that works with Async (mssql) in a project that is currently written using Lwt. I can envisage a variety of solutions:

• Use the lower-level API of multipart_form to get it working with Async and port the whole program from Lwt to Async just so I can work with mssql in the hope that I do not then need an Lwt-only library in the future.
• Split off mssql exposed as a web service on the same machine.
• Find some kind of in-process interop between Lwt and Async. I’m not sure if this is as simple as just writing everything in CPS or if this is impossible.

I’m guessing I am not the first person to have faced this challenge so I’m wondering if there is a preferred solution?

[thomas_ridge]

I’ve never used Async, but Lwt has Lwt_preemptive.run_in_main, which you could call from the Async thread presumably (with Lwt main running in a separate thread).

[yawaramin]

Another option: use the lower-level API of mssql, which is ocaml-freetds. E.g.: https://github.com/kennknowles/ocaml-freetds/blob/master/examples/dbi_example.ml

[roddy]

I would also consider functorizing mssql to support Async/blocking IO. From a brief look at the code I think this would be non-trivial but doable.

[Jon_Harrop]

Functorizing is an interesting idea but I just the advice here (from Cohttp experience) was against doing so:
http://rgrinberg.com/posts/abandoning-async/
However, I found an old post here that mentions the idea of forking a child process that uses the other async library. That sounds like a great idea to me so I’ll give that a go first…

[rgrinberg]

Functorizing works well sometimes, but it will not be very appealing in this case. The library in question (ocaml-mssql?) is split into many modules and relies on a few modules from async that might not have immediate Lwt analogues.

Why don’t you just the parent library ocaml-mssql is based on? Freetds should be quite simple to wrap with Lwt based connection handling.

[Jon_Harrop]

Definitely a possibility. Only problem is that I’m new to these async libraries and have no idea what is going on yet.

[yawaramin]

Hint: use the Lwt_pool module to set up a connection pool. It’s pretty nicely documented.

[mbacarella]

@Jon_Harrop

I’m guessing I am not the first person to have faced this challenge so I’m wondering if there is a preferred solution?

You might find some of the discussion here useful.

Best practice on mixing Async and Lwt?

Sorry if this is a FAQ or a point of contention, I couldn’t find an answer typing keywords into various search boxes. I’m starting a new app in Async but will need to use a library or two written in Lwt, so I’d like to minimize Lwt and wrap it in Async stuff if possible. I have a pretty good handle on Async’s conceptual model and making it co-exist in non-Async environments but not sure how it would collide with Lwt. Has anyone been down this road already? Any pointers besides “lol just use …

[BrendanLong]

Yeah, you could probably make something similar to Mssql using Lwt_pool + however you spawn background threads in Lwt.

I think we’d be open to making mssql work on Lwt, but it’s probably non-trivial since the library uses Sequencers and threads.

[trefis]

(Disclaimer: I’m not offering any solution (nor advice), just sharing some folklore.)

A long time ago, @jeremiedimino and I wrote GitHub - janestreet/lwt-async: Lwt with async backend, which defines 'a Lwt.t as 'a Or_exn.t Async.Deferred.t (see here).

[yawaramin]

Btw, I found the raw DB interaction module that ocaml-freetds uses: [modcaml] Contents of /ocamldbi/dbi_freetds.mli … this seems to be the actual thing you’d use to connect, run queries, etc.