Consider this a post where I think aloud about my experience migrating an Async project to Lwt. I’ve spent about a weekend doing such a thing, and if, in the process of talking about it here I can save a few people an hour or two (or perhaps inspire confidence to take on such a project in either direction) then it will have been worthwhile.
This wouldn’t be a complete post if I didn’t also mention @dkim’s translation of Real World OCaml’s Async examples to Lwt
This was born out of a previous effort where I tried to mix Lwt and Async in the same project. This didn’t go so well, so I tried converting the whole thing to Lwt, and it turns out adapting to Lwt if you’re an Async person is actually much easier than I thought it would be.
Basics
Both libraries involve promises/futures. Async calls its promises Deferred.t, whereas in Lwt they’re called Lwt.t.
In Async you start your program by saying never_returns (Scheduler.go ()) or Command.async_spec after you set up your initial Deferred.ts.
In Lwt you say Lwt_main.run on a top-level Lwt.t argument. Note you can re-run Lwt_main.run in a single program as many times as you want, but
perhaps you shouldn’t run multiple Lwt_main.runs in parallel.
There’s an easy correspondence between basic operators.
| Async | Lwt |
|---|---|
Deferred.bind |
Lwt.bind |
Deferred.return |
Lwt.return |
>>= |
>>= |
Deferred.map |
Lwt.map |
>>| |
>|= |
Deferred.don't_wait_for |
Lwt.async |
In_thread.run |
Lwt_preemptive.detach |
Deferred.List.iter |
Lwt_list.iter_s |
Deferred.List.iter ~how:Parallel |
Lwt_list.iter_p |
Starvation worries
The most important difference between Async and Lwt is that fulfilled promises are acted on immediately, whereas Async kinda punts them to the end of a work queue and runs their thunks later.
Thusly, a return loop like this starves the rest of Lwt:
open Lwt.Infix
let main () =
let rec loop () =
Lwt.return ()
>>= fun () ->
loop ()
in
Lwt.async (loop ());
Lwt_io.printlf "this line never prints!"
;;
let () = Lwt_main.run main ;;
whereas the corresponding Async loop does not starve:
open! Async
let main () =
let rec loop () =
Deferred.return ()
>>= fun () ->
loop ()
in
don't_wait_for (loop ());
printf "this line does print!\n";
return ()
;;
let () =
let cmd = Command.async_spec ~summary:"" Command.Spec.empty main in
Command.run cmd
;;
Fortunately, if you can predict this ahead of time, there’s an easy workaround. You can get something closer to the Async-style behavior in Lwt by using Lwt.yield () instead of Lwt.return ().
Spawning threads
From time to time you may need to run something in a system thread. In Async you say In_thread.run, whereas in Lwt you say Lwt_preemptive.detach. For simple things they’re pretty much interchangeable, but one stumbling point for me was that in Async you can create a named thread and always use that for the In_thread.run, with multiple simultaneous dispatches to that thread becoming sequenced.
This is really useful for interacting with libraries that aren’t so thread friendly.
Lwt’s detach doesn’t provide an easy way to do this out of the box, but I think you can still deal with thread unfriendly libraries by using the Lwt_preemptive.run_in_main call.
Basically, never exit the detached thread you started to interact with your library. Instead, when it’s finished, have it block on promise that gets filled with the next request via run_in_main. In this way you can sequence your detached Lwt thread similarly to Async.
Happy to explain further if this is unclear.
Other libraries
Async.Unix has a somewhat built-up conception of the UNIX API, whereas Lwt_unix is a more direct mapping of ocaml’s Unix module to promises.
Async Clock.every and Clock.after don’t have exact analogs, but you can make new versions pretty simply.
Example of a shallow imitation of Async Clock.every
let every span f =
Lwt.async (fun () ->
let span = Time.Span.to_sec span in
let rec loop () =
f ();
Lwt_unix.sleep span
>>= fun () ->
loop ()
in
loop ())
;;
Open questions
I haven’t sorted out a good Lwt substitute that’s as comfortable as Async Pipe yet. Though some combination of Lwt_stream, Lwt_sequence and lwt-pipe might fit the bill. If you just happen to know already feel free to cluephone.
Haven’t really examined network connections yet. Will circle back and update this part if it’s worth mentioning.
I’ve sort of been ignoring the fact that there’s no obvious mapping of Async Monitor to an Lwt thing. :x_this_is_fine_dog:
Closing remarks
This is basically everything? I’m almost suspicious that I’m not having more problems, but will happily accept grace where it arises.