Dealing with asynchronous callbacks in ctypes

Learning
1

I’m using ctypes to write C bindings to a library that allows the user to register a callback that is called when a specific event occurs. The C function that allows you to register this callback takes a function pointer and calls it when the event fires.

The problem is the library spawns a new pthread to check for new events and call the function pointer. This means that if I use Ctypes.Foreign.funptr to give it an OCaml function, the runtime lock is not acquired when the function is called, leading to a segfault.

I ran the program through lldb and received this backtrace, which leads me to believe that it is, in fact, a runtime issue:

(lldb) r
Process 3572 launched: '/Users/zbaylin/Development/ocaml-rtmidi/_build/default/example/ocaml_rtmidi_example.exe' (x86_64)
4302327232
ocaml_rtmidi_example.exe was compiled with optimization - stepping may behave oddly; variables may not be available.
Process 3572 stopped
* thread #2, stop reason = EXC_BAD_ACCESS (code=1, address=0x18)
    frame #0: 0x0000000100073c1b ocaml_rtmidi_example.exe`caml_thread_leave_blocking_section [inlined] caml_thread_restore_runtime_state at st_stubs.c:202:43 [opt]
Target 0: (ocaml_rtmidi_example.exe) stopped.
(lldb) bt
* thread #2, stop reason = EXC_BAD_ACCESS (code=1, address=0x18)
  * frame #0: 0x0000000100073c1b ocaml_rtmidi_example.exe`caml_thread_leave_blocking_section [inlined] caml_thread_restore_runtime_state at st_stubs.c:202:43 [opt]
    frame #1: 0x0000000100073c1b ocaml_rtmidi_example.exe`caml_thread_leave_blocking_section at st_stubs.c:248:3 [opt]
    frame #2: 0x00000001000844a3 ocaml_rtmidi_example.exe`caml_leave_blocking_section at signals.c:175:3 [opt]
    frame #3: 0x000000010009e6ef ocaml_rtmidi_example.exe`caml_write_fd(fd=2, flags=<unavailable>, buf=0x0000000108448048, n=11) at unix.c:94:3 [opt]
    frame #4: 0x0000000100092f3f ocaml_rtmidi_example.exe`caml_flush_partial(channel=0x0000000108448000) at io.c:194:15 [opt]
    frame #5: 0x0000000100093e28 ocaml_rtmidi_example.exe`caml_ml_flush [inlined] caml_flush at io.c:209:12 [opt]
    frame #6: 0x0000000100093e1e ocaml_rtmidi_example.exe`caml_ml_flush(vchannel=<unavailable>) at io.c:667:3 [opt]
    frame #7: 0x000000010002b207 ocaml_rtmidi_example.exe`camlStdlib__prerr_endline_386 + 87

I saw that funptr has an optional runtime_lock argument, but this allows the user to specify that they want to specifically release the runtime lock, not acquire it.

tl;dr: are there any examples of people using ctypes in an asynchronous manner such as this? I think I could do this writing C bindings on my own, but I’d prefer not to mix custom stubs with ctypes if I don’t have to.

Thanks,
- Zach

2

Are you passing ~thread_registration:true ? Threads other than the main one need to be registered with the runtime if you are calling OCaml code from them.

Also, you need to make sure your OCaml closure does not get garbage collected while it may be called from C. You can also use dynamic_funptr to better control the lifetime:

Cheers,
Nicolas

2 Likes
3

Ah, thanks @nojb! I missed that thread_registration flag.

I guess that leads me to a related question – is there any way to “preempt” the main thread with this? I.e. signal to the main thread that it should release the runtime lock and yield. Otherwise, it seems like the callback will spend all it’s time waiting for the main thread to yield, which doesn’t always happen in a timely manner.

Thanks again!

4

Not that I know of. Typically the main thread would enter some kind of event loop and would yield periodically or block waiting for the callback to fire. This is the situation in OCaml 4. In OCaml 5 you will be able to run OCaml code in different threads in parallel.

Cheers,
Nicolas

4 Likes