It seems that my mental model of effect handlers is sligthly off. Perhaps a good soul here can help me solving that problem.
I have code that is morally equivalent to the “User level thread” example in the manual. The handler of this example is:
let rec spawn (f : unit -> unit) : unit =
match f () with
| () -> dequeue ()
| exception e ->
print_endline (Printexc.to_string e);
dequeue ()
| effect Yield, k -> enqueue k (); dequeue ()
| effect (Fork f), k -> enqueue k (); spawn f
| effect (Xchg n), k ->
begin match !exchanger with
| Some (n', k') -> exchanger := None; enqueue k' n; continue k n'
| None -> exchanger := Some (n, k); dequeue ()
end
in
spawn main
My problem revolve around the Fork case:
| effect (Fork f), k -> enqueue k (); spawn f
I’d like to change the semantics of Fork so that it continues the with the forker and enqueues the forkee for deferred execution. Naïvely I thought this was a matter of changing that line to:
| effect (Fork f), k -> enqueue_task (fun () -> spawn f); continue k ()
However when I do this the program starts leaking like hell. The full leaky example can be found in this gist.
So here are my questions:
What is the problematic leaky structure here? I can’t exactly wrap my head around it.
How can I actually implement the semantics I’d like without leaking ?
I’m not sure to understand when you say: “the program starts leaking like hell”. For my perspective, I find it a bit unusual to have effect handling in the spawn function (rather than in the run function). In fact, in my little tutorial on building a mini scheduler (not to be tried at home), it is indeed the run function that sets up the effect handler.
EDIT: On your gist, we must start with open Effect and open Effect.Deep to compile the code.
Yes, I just saw the result, thanks. I think it’s related to the way that stacks are allocated when we use effects. I’m not totally sure but I remember some discussions with @octachron about possible leaks with effects. It might be worth tracing how stacks are allocated and deallocated throughout your code, particularly by looking at runtime/fiber.c.
This may be off the mark, but isn’t there an issue with the fact that the code is enqueing 200_000_000 closures with enqueue_task (fun () -> spawn f)before any of them runs?
One take away of this discussion is that when you try to reduce a leaking bug due to a scheduler you should be careful not to produce a repro that is not one but just a DOS of your own scheduler (like I did here, it was less obvious in my case since the scheduler is parallel).
Meanwhile, I’m not sure I have nailed down my leak so I’d interested in having more details about what @dinosaure mentioned about possible leaks with effects.
I also have (re?)discovered that Domain.join leaks:
let main () =
for i = 1 to 1_000_000 do
Domain.join (Domain.spawn (fun () -> ()));
Gc.full_major ();
done;
0
let () = if !Sys.interactive then () else exit (main ())
I have a vague remembrance that this is somehow expected but I couldn’t dig up the comment upstream. While you are not supposed to create a million of domains, if you have a test suite exercising thousands of parallel scheduler runs including tear-up/down. It shows up. Perhaps it should at least be documented? Should I open an issue upstream about that (bug or documentation improvement) ?
Continuation stack fragments are not tracked by default by the GC, so you can have memory leaks if some continuations are not discontinued.
There is also a cache of pre-allocated continuation stacks in the runtime which is currently quite naive and can theoretically grow in an unbounded way.
With OCaml 5.5, you can check both issues with the live_stacks_words field of Gc.stat.
I think that the test case is tidy with respect to this. In fact the system only leaks once I spawn more than one domain. With a single domain the program memory usage stays at a nice and steady 4MB of memory.
Ah nice that could have saved me a couple of hours (I was rather doubting my continuation logic). So if I run the test with a single domain I get:
I tried to make a self-contained example that has the kind of control flow of the simple test case that exhibits the leak but annoyingly it didn’t succeed in reproducing these stack lacks.
The test case is small and basically equivalent to
for i = 1 to 50_000_000
let v0 = spawn (fun () -> yield ()) in
let v1 = spawn (fun () -> yield ()) in
join v0; join v1;
if i mod 10_000 = 0 then (Gc.compact (); …)
done
which is executed by the main domain and with the fibers potentially taken up by worker domains.
Is there a way to check if that could be the culprit ?
if your program can reach a state where all fibers have been handled, then x.live_stacks_words - x.stack_size will be the size of the cache.
you could check that the memory leak is worse in OCaml 5.5 compared to OCaml 5.4 (on non-macOs system): the stack cache was only half-working before OCaml 5.5 (except on macOS). If it is the culprit I expect that the leak would be significantly worse in OCaml 5.5 . If it is the case, I will be very interested if you could report it, because then I would move the update of the caching strategy to the upcoming 5.5.1 .
The cache issue should only appear if some parts of the program create a lot of small fibers that are live simultaneously. Then the memory usage of the cache will never go down from its peak usage.
Yeah, that’s why I don’t think my patch above is quite right, as it’s quite aggressive in tearing down the stack cache and so prioritises level memory usage a bit too much vs performant domain creation. It’s helpful to remove the stack cache as one of the factors in memory usage in order to see what other leaks lurk beneath though; perhaps we should make this an OCAMLRUNPARAM option for the runtime.
After many hours of trying to gradually reduce the test case of the full system, it turned out the problem of my simple example was that it was trying to use only two domains and at a certain point the problem no longer shows up for two domains.
So I now have a simple enough repro in this gist that leaks on both MacOS and Linux when used with a total number of domains > 2. See the top comment for details.
Thanks for the reproduction case. At least the reason behind the memory leak is obvious in hindsight: the stack cache increases in size whenever a domain perform an effect while its stack cache has been dried out by other domains. This is why the phenomenon only appears when there are more than one worker domains.
I’m glad it’s obvious :–) Tell me if you want me to open an issue upstream.
I have one tangentially related question. Suppose I have an Effect.Deep.continuation that has been (dis)continued but I still keep a reference on it in a closure that may not be gc’d for some time (or ever) what is the gc cost of keeping that resumed continuation value around, is just a dangling pointer ?
I am currently planning to have this bug fix be a part of the upcoming 5.5.1 release.
Concerning your question, in an already resumed continuation, the pointer to the stack inside the continuation has been set to the null pointer, so the GC costs should be just the cost of keeping around a pointer to a size 1 block.