Compiling subset of OCaml to GoLang / Julia?

Learning
1

Is there a good resource on compiling a subset of OCaml to another language ?

GoLang: I like the libraries (litestream, pion), but not the lack of sum types.

Julia: I like the libraries, but not the lack of static type checking.

I’m wondering if there is a way to write code in a strict subset of OCaml (so I get the benefit of the ocaml type system + merlin + IDE features), then write a tiny

OCaml AST → { GoLang, Julia } AST function

and compile it to GoLang / Julia (with some properly defined FFI).

The goal here is to get the benefits of OCaml’s type checking system, but use the libraries of other languages.

I think bucklescript / ReScript / Reason do this at the OCaml AST level for JS, while jsoo does it at the bytecode level for JS.

I am wondering if there is a tutorial on a “minimal example” of doing this.

1 Like
2

I’m also interested in this (conceptually).

I intend to eventually buy this book to learn the fundamentals: https://craftinginterpreters.com

A tutorial sounds a little too optimistic to me. For the solution to be practical would probably require a lot of work, much more than just re-writing the libraries you need in OCaml I would think.

If you haven’t done so, you should probably take a good look at the PPX system mentioned here first: [ANN] Improvement on the PPX ecosystem documentation

I’d like to hear what knowledgeable people have to say on the matter though :slight_smile:

3

Sorry, I’m not seeing the relevance here. This is GoLang right?

I intend to skip the lexing, parsing, type checking phase – leveraging OCaml itself. I.e. something like a “embedded DSL within OCaml” → GoLang / Julia.

4

It doesn’t matter what the underlying technology is IMO.

If you already have the theoretical knowledge, then there are plenty of implementations in the wild to study, transforming source language A to the target language B.

5

It should be somewhat straightforward (but still a lot of work) if you manually write stubs for all the functions and resources of the target language. There are lots of approaches, here a naïve one comes to my mind. You would write almost a normal OCaml project, but where relying on target language you would leave the functions unfinished, you put e.g. let foo x y = ignore (x, y); raise (Julia_call "foo"). You would use Ppxlib metaquotation: dune stanza (preprocess (pps ppxlib.metaquot)) in your translator file. You’d have a recursive descent translator pattern matching on the expressions of the OCaml sublanguage, e.g.

  | [%expr ignore [%e? { pexp_desc = Pexp_tuple arguments; _ }];
           raise (Julia_call [%e? { pexp_desc = Pconst_string (fname, _, _); _}]) ] ->
    let arguments = List.map translate arguments in output_julia_call fname arguments
6

Compiling OCaml to Julia most likely is going to be a lot of work. As a short term solution though you can try to compile Julia code to a shared library with GitHub - JuliaLang/PackageCompiler.jl: Compile your Julia Package and then use OCaml C FFI to call it. I have never tried it, so I’m not sure how it will work in practice though.

7

To make an obvious point: usually the way we deal with interoperability between two source languages is not to compile one into another, but to link their programs together at a lower level (assembly, LLVM, the JVM or dotnet runtime, etc.). Both languages use a “foreign function interface” (FFI) to talk to each other. Most languages come with a good FFI for C, the common system language, and two arbitrary languages often end up talking to each other by going through C as an intermediate layer.

Using an FFI is a lot less work than trying to compile a language into another. Have you tried this first?

There are three common issues with FFI:

  • You can only exchange values directly at types that both languages understand. Often those are rather low-level types (arrays of numbers), and for everything else you have to write wrappers and this is boring work. This may or may not be an issue depending on the nature of the libraries from the other language you want to expose.
  • The wrapping/copying of values at the interface boundary can be a performance bottleneck in some cases. (Again, it depends on the data you exchange, and also on the expected number of FFI calls.)
  • They may break the usual safety guarantees of the language. Using them is risky in this sense. (Giving up on some reasoning guarantees is often unavoidable when you program hybrid systems using different languages.) But then, a compilation layer can also “change” the semantics of programs, potentially introduce bugs, for simplicity or performance reasons.
5 Likes
8

@lukstafi :slight_smile:

  1. Thanks! This looks like the type of blackmagic I’m looking for.

  2. I’m familiar with Lisp/Clojure style macros, as well as Rust macro_rules and Rust procedural macros, but new to ppx. What would you suggest I read to learn ppx for this precise purpose ?

I also have some mechanical questions:

  1. “ppx” sounds like a “preprocessor”, i.e. run before. In our case, we want the lexer/parser/typechecker to run on the MODIFIED code, i.e. have our transforms run AFTER lexing/parsing/type-checking. How does this work ?

  2. preprocessor sounds like (1) we read ocaml code (2) we output ocaml code, (3) we then compile the output ocaml code; however in our case, we want to do a transform where (1) we read ocaml code, (2) we output GoLang/JuliaLang, and (3) we stop execution; how does this fit in mechanically ?

9

I have not.

In the case of JS, I found writing JS/TS to be rather unpleasant compared to jsoo / ReScript.

Similarly, I find GoLang / Julia unpleasant in ways the OCaml type system can fix; I think it is perfectly reasonable to ask “What would jsoo/rescript for GoLang/JuliaLang look like?”

10

You may find this interesting: GitHub - jordwalke/rehp: ReHp

Under the hood, rehp is a fork of js_of_ocaml, refactored to make supporting other language backends easier. Most of this project can be merged upstream into js_of_ocaml , aside from the parts that are specific to only one language backend.

1 Like
11

  1. Thanks @yawaramin ! This looks really cool.

  2. If possible, I’d prefer a solution along @gasche 's line, just because, so far, in personal experiences, rescript has been beating jsoo in terms of compile time, [ see People with large codebases, what are the compile times like? - #2 by DZakh - dev - ReScript Forum ] and I’m hoping a ppx / AST based approach may give similar speed.

12

To just quickly address (3), I meant to use a preprocessor (the meta-preprocessor) to write the translator, not to write a preprocessor. To address (4), you would use the package ocaml-base-compiler, library ocamlcommon, module Compile_common, function parse_impl, to get the Parsetree.structure (AST). Then you would pass this to your translator function, and the translator function could write Julia code to a text file directly.

2 Likes
13

A project that has not been updated for a long time:

1 Like
14

Of note is also Caramel, which ran out of funding but went pretty far until that point for compiling a subset of OCaml to the BEAM VM (Erlang’s runtime):

1 Like
15

Well, I got it to a state when it produces correct code for a variety of test programs (including a tiny compiler in OCaml).

But lambda lacks typing information, and I had to resort to generating basically JavaScript, but in Golang with tons of reflection. It’s too slow to be practical.

I tried to apply naive HM on the lambda, but unfortunately I’m not knowledgeable enough to make it support polymorphism, and naive approach is also very slow to compute the unification.

I’m happy to explain what happens inside the codebase if someone’s interested.

1 Like