There is a function g : unit -> int in a library that may or may not use Lwt_main.run internally (perhaps it sometimes does, and sometimes doesn’t, depending on the phase of the moon).
I have written another function f: unit -> int that forks (using Unix.fork) and calls g (and maybe does other stuff).
f works fine if the main process is not executing Lwt_main.run. However, if the main process is running Lwt_main.run then I may get an error message when running f: “Nested calls to Lwt_main.run are not allowed” (depending on whether g actually uses Lwt_main.run internally when it runs).
f is supposed to be a library function that works regardless of whether the main process is using Lwt_main.run or not.
There are various questions that arise. My main question is: Is it possible, in the child process, to start with a “clean slate” as far as lwt is concerned? For example, after the Unix.fork I would like to run (the fictitious) Lwt.abandon_any_run_loops_reset_to_initial_state(), so that I don’t have to worry when calling g. Does anything like this exist?
Don’t know the answer to your precise question, but tangentially, it seems to me that a library should never call Lwt_main.run directly as it kills composability.
I sort-of agree. Lwt_main.run is clearly “special”, and a library should instead expose functions that return in the lwt monad.
But in my scenario it seems difficult to use this observation… So then g should presumably return in lwt, and likewise f. But then f is an lwt promise, and the rest of the codebase that uses f needs to be lwt-ified. But this is absolutely not acceptable to the owners of the code that want to use f.
I would also say that forking when a Lwt main loop is running is also going to be problematic in many cases, because the child process inherits the process image of the parent and you will end up with two versions of the loop running, including all the Lwt “tasks” running on the main loop at the time of the fork. Since one of the purposes of Lwt is to provide asynchronous i/o, and i/o is inherently side-effectful, that sounds like a recipe for trouble. It might work if you can secure that the first thing the child does is to cancel the promise on which the Lwt main loop is blocking, so cancelling all such tasks in its own process but that sounds quite difficult to achieve reliably in practice.
This seems rather like the debate about whether to call fork or fork-all in a program running more than one native thread of execution. Either approach is likely to lead you lnto trouble, but in different ways (leading to the advice that the only thing you should do after forking in a multi-threaded program is to call exec, and prior to the exec only call async-signal-safe functions).
@cvine Yes calling fork when an Lwt main loop is running feels like it could be problematic in a number of ways. But there is already a Lwt_unix.fork which does something similar to what you suggest in terms of cancelling promises (but I don’t think it cancels all promises… or terminates the Lwt_main.run loop if that was running… if only there was an Lwt_unix.fork ~and_terminate_lwt_main_loop_in_child that did this, the problem would be solved!).
Maybe I should add that basic fork+exec seems like an alternative, it’s just that I wanted to avoid the dependency on a “path to the exec”, when I already have the code I want to run present in the parent.
Ah right, I didn’t know there was a Lwt_unix.fork function available. In which case the child cancelling the promise on which the main loop is blocking seems like it should work (assuming Lwt_unix.fork is actually reliable).