How to expose an abstract type to C code?

Ecosystem
1

I want to make C (and ultimately Python) bindings for an OCaml library I’ve made.
Suppose I have the following module:

module type Foo = sig
  type t
  val from_string : string -> t
  val frobnicate : t -> t
  to_string : t -> string
end

Now I try to use Cstubs_inverted to expose it to C:

module Stubs(I : Cstubs_inverted.INTERNAL) = struct
  let () = I.internal "from_string" (string @-> returning ...) from_string
end

And that’s where I’m completely lost. Returning what exactly? How do I make a Ctypes.typ from the abstract type?
If I’m doing something obviously wrong, please point me in the right direction.

2

One approach is to use the Root API to convert between Foo.t values and pointers that can be exposed through the C interface.

3

We actually have a solution for this, that should be probably eventually put into the Cstubs library as this is a common requirement, with a few caveats. But let’s discuss the general approach.

Since abstract types are represented as pointers to opaque structures in C, we need create them as follows:

type opaque_t = Opaque
type cstruct = opaque_t structure

let newtype name = 
   let t : cstruct typ = structure name in
   Internal.typedef t name

This will create a new opaque structure on the C side, along with the corresponding typedef (that will inject the name into the namespace of types), e.g.,

struct name;
typedef struct name name;

Now, you can transform between the representations, using the following functions:

let to_opaque (ovalue : ocaml_type) : cstruct ptr = 
   from_voidp t (Root.create ovalue)

let of_opaque (opaque : cstruct ptr) : ocaml_type =
   Root.get (to_voidp opaque)

So, that would be enough, but Root.get is not a typesafe function, as it has type unit ptr -> 'a. So, to use it, you need to trust that a void * pointer indeed points to an OCaml value of type ocaml_type. Honestly, you may never trust the C side, as it is very easy to pass a wrong type to the C function, and C typechecker won’t even consider this an error (will just issue a warning, that you’re passing incompatible structure types - yep, in C passing a wrong type to a function is just a warning, so make sure that you turn it into an error with -Werror). So since static typing doesn’t work in C, we need to add dynamic typing. This is done by making a fat pointer that besides a pointer to the OCaml value, will contain some type identifier, e.g.,

    type 'a fat = {
      typeid : int;
      ovalue : 'a;
    }

So now, we can ascribe a unique type for each exposed OCaml type, and in our of_opaque function check that the provided pointer indeed contains the right data, witnessed by the typeid. We can also provide nice error messages, and all the stuff.

So, we have a library for that (and much more than that, actually), called Opaque but unfortunately we do not distribute it - it just used as a part of our bindings. Feel free to use it, it is licensed under MIT. If @yallop is interested we can contribute it upstream or distribute as a separate library (after some generalizing, i.e., removing default "bap_" prefix).

P.S. we also have a solution for enumerations, OCaml strings, Containers, Iterators, etc.

4

@yallop, @ivg Thanks for your replies!
I’ve got a proof of concept to work today, with just type-unsafe Ctypes.Root.create/get for now.

5

I had a follow up question on the Root.get and create. I have the same issue, and I didn’t want to make another thread on it.
This is the type that I would like to expose

type t = D of int | E of int | F of int
val sub : int -> t

I think that I have an issue with the garbage collection as I bind this to C. I’m not sure if I have a functional understanding of this, but this is the allocation that I used:

let x = Root.create sub
let y = Root.get x
Root.release

My understanding of this is that the first line allocates storage for the function “testing”, and the second line retrieves the value at that spot. This is my Cstubs_internal module.

module Stubs(I : Cstubs_inverted.INTERNAL) = struct
let () = I.internal “sub” (int @-> returning y ) sub
end