How to prevent a closure that passed to a C function as a callback from being garbage collected prematurely?

Learning
, ,
1

I need to call an async C function that returns a future type(something like a promise type in Lwt or Eio). There is a future_set_callback function on the C side, and it receives a function pointer and will call this function when the future’s value is resolved by the underly event loop.

I want to wrap the C lib’s future with Eio’s promise type like below code:

    let from_future fut =
      let (promise, resolver) = Eio.Promise.create () in
      let callback = (fun r ->
        Eio.Promise.resolve resolver r 
      ) in
      let () = Future.set_callback fut callback in 
      promise

The problem is that the callback in the above code may be garbage collected by OCaml’s GC even before the C side’s event loop calls it.

Any method to solve this?

2

Rule 4 here might be what you need: OCaml - Interfacing C with OCaml.

1 Like
3

Thanks! I think using caml_register_global_root(&v) to register the callback value should work.

But how to use ctypes to do this? Currently I use Foreign.funptr in ctypes to pass the callback closure to the C side.

4

Storing the closure somewhere where the OCaml program can access it for the duration of the C event loop will prevent the closure being garbage collected.

One approach is to set up a mechanism similar to caml_register_global_root in OCaml, like this:

type value = R : _ -> value
type roots = { mutable next: int;
               mutable roots: (int * value) list }
let global_roots = {next = 0; roots = []}

let register_global_root r =
  let g = global_roots in
  let id = g.next in
  g.roots <- (id, R r) :: g.roots;
  g.next <- succ id;
  id

but whether this is the best approach will depend on the details of your program.

1 Like
5

Thanks.
But I don’t understand the type type value = R : _ -> value, could you explain it?

6

I have used Callback.register for the delayed execution of OCaml callbacks by C code upon the occurrence of some future event. By storing the callback in an OCaml container that way, it will be safe from garbage collection and this will also deal with the closure’s address being moved upon translation to the major heap or on heap compaction. See paragraph 7.2 of this section of the manual: Advanced topic: callbacks from C to OCaml , and see also Advanced example with callbacks.

It doesn’t suit all cases but it might suit yours.

1 Like
7

It means the same as

type value = R : 'a -> value

which says that

  • there’s a type value
  • there’s one constructor R with the type
    R : 'a -> value
    i.e. it takes a value of any type 'a and constructs a value: R 3 and R (fun x -> x + 1) and R [1;2;3] all build values of type value.
1 Like
8

That’s an interesting use of GADTs, which I have not seen before. Presumably the r argument of register_global_root is the closure. But since r is existential in R how could it be extracted for application? Storage as an existential in the GADT would protect a copy of the closure stored separately by C code from garbage collection but would it protect such a copy from, say, it being moved to the major heap?

Apologies if I have missed the point: you know a lot more about this than I do.

1 Like
9

In this case there’s no need to extract it: it’s the C code that will call the closure; the reference in global_roots only serves to prevent collection.

You’re quite right: it doesn’t protect the closure from being moved. For passing OCaml functions to C with ctypes it doesn’t matter if the closure is moved, because only an integer id is actually passed to C; that id is used to retrieve the closure from an OCaml table when the closure is called. If the closure is moved then the reference to it in the table will be updated by the runtime, too.

1 Like