Concurrent tasks in Lwt

Ecosystem
1

I have found that the easiest way to run multiple tasks concurrently on Lwt is:

Lwt_main.run @@ fun () ->
  ignore task1;
  ignore task2;
  ignore task3;
  Lwt.return ()

If task1, task2 and task3 are sequences of Lwt promises with long IO, sleep, pause, I can see the scheduler running them concurrently.

It can be simpler than:

  Lwt.both task1 @@ Lwt.both task2 task3

However, I wonder if it is an expected behaviour. I am contaminated with Haskell where an ignore doesn’t shortcut the monadic thread, but would really ignores its argument.

2

You’d want to at least know if any of the tasks fail, in which case you can use Module Lwt

3

Isn’t

Lwt_main.run (Lwt.all [task1; task2; task3])

what you want to do in this case?

4

Yes, this seems to be a better solution than nested both.

However, for a TCP server that would have to spawn promises, one for each connection, spawning them with both would most likely create a memory leak because of its join task. The Lwt.on_failure seems to fit the need, but we may have no usage of the exception handler.

5

I don’t think that this is exactly your code. For one thing it doesn’t type because Lwt_main.run expects a promise. But more importantly, this will not do any work in the tasks that’s beyond I/O or pauses/yield. E.g., consider the code below

let () =
  Lwt_main.run @@
    let open Lwt.Syntax in
    ignore (let* () = Lwt_unix.sleep 5. in print_endline "waited 5"; Lwt.return ());
    ignore (let* () = Lwt_unix.sleep 7. in print_endline "waited 7"; Lwt.return ());
    Lwt.return ()

The executable exits immediately, no printing happens.

That’s because Lwt_main.run simply “awaits” for the given promise to resolve at which point it returns the value into the OCaml world and the execution of the program continues as normal (i.e., as a non-Lwt program would).

If there are pending I/O jobs and unresolved promises in the scheduler then they are simply ignored.

For a server, you can pass your main-loop to run (in the case of a server, the loop that calls accepts, spawns a task and then calls accept again, indefinitely). Or sometimes you can just pass a never-resolving promise (fst (Lwt.task ()) or similar).

Regarding exception management, I’d recommend that you use Lwt.dont_wait rather than on_failure+ignore. It should be roughly equivalent but it will force you to have a unit promise and to wrap your promise inside a lambda which avoids exceptions leaking out.

E.g., the following will raise “outside of the promise” and the handler won’t be called

let p =
  if param < 0 then raise (Invalid_argument "");
  let* …
in
Lwt.on_failure p (fun exn -> …);
ignore p;

whereas the same code with dont_wait will pass the exception to the handler.