[ANN] carray.0.0.1

Ecosystem
, , , ,
6

Note that you can, to a certain degree, build your own flat structures with the Bytes module. Compared to bigarrays, Bytes.t has less indirection, a lower constant memory overhead and can be allocated on the minor heap. The contents of Bytes.t are not scanned by the GC, just like bigarrays.

For example, a more efficient int32 Array.t:

module Int32_array : sig
  type t
  val equal : t -> t -> bool
  val create : int -> t
  val length : t -> int
  val get : t -> int -> int32
  val set : t -> int -> int32 -> unit
  val sub : t -> int -> int -> t
  val to_list : bytes -> int32 list
end = struct
  type t = Bytes.t
  let equal = Bytes.equal
  let create len = Bytes.create (4 * len)
  let length t = Bytes.length t / 4
  let get t i = Bytes.get_int32_le t (4 * i)
  let set t i x = Bytes.set_int32_le t (4 * i) x
  let sub t pos len = Bytes.sub t (4 * pos) (4 * len)
  let to_list t = List.init (length t) (get t)
end

A more efficient (int * int):

module Point : sig
  type t
  val create : int -> int -> t
  val x : t -> int
  val y : t -> int
end = struct
  external get_int64_unsafe : bytes -> int -> int64 = "%caml_bytes_get64u"
  external set_int64_unsafe : bytes -> int -> int64 -> unit = "%caml_bytes_set64u"
  type t = Bytes.t
  let create x y =
    let p = Bytes.create 16 in 
    set_int64_unsafe p 0 (Int64.of_int x);
    set_int64_unsafe p 8 (Int64.of_int y);
    p
  let x t = Int64.to_int (get_int64_unsafe t 0)
  let y t = Int64.to_int (get_int64_unsafe t 8)
end

(making a more efficient (int * int) Array.t is left as an exercise to the reader)

The downside compared to bigarrays is that it doesn’t support sub without copying. Also, bytes can be moved by the GC (during minor GCs or compaction), and therefore you cannot release the runtime lock when passing them to C. The latter point is less relevant with the multicore extensions, especially since there is no compactor yet. There is some related discussion on the eio repository: Decide on cstruct vs bytes · Issue #140 · ocaml-multicore/eio · GitHub

7 Likes