Do we have OCaml bindings to Apache Spark somewhere?

Community
1

https://spark.apache.org/

This framework looks yummy. :slight_smile:

2

It’s Java, so creating the bindings can be painful. Another approach would be to use the normal FFI as to any C library to the Rust alternative - Native Spark.
Because Rust allows exporting the functions with C ABI it should be fairly easy. You can read more in the blogpost “FastSpark: A New Fast Native Implementation of Spark from Scratch”.

And it is obviously faster than Java version:

Rust implementation CPU utilization during the execution (8 cores)

image
image864×576 55 KB

Spark CPU utilization during the execution (8 cores)

image
image864×576 68.7 KB

They have a Gitter channel if you are interested.