Js_of_ocaml, dune, and generate separate JS libraries

Ecosystem
1

Does anyone have experience with using dune and js_of_ocaml to generate from a big OCaml codebase multiple JS libraries that can be dynamically load and bundled separately?

The problem I have is that I have a command-line application that I would like to make available on the web (semgrep), but the generated code for it is currently huge because every language supported by semgrep and their parsers are included in the code. In most cases, the user interacts with only one language at a time, so I would like to generate a parsing_go.js library, a parsing_java.js library, etc. and not require a user to load all those javascript code when he reaches my website. I would like to load this JS code on-demand, separately for each language.

Does anyone have experience with that?
I’ve looked at the dune documentation on its integration with js_of_ocaml but it does not say much.

1 Like
2

I’m also open to other systems such as melange, as long as I don’t have to switch away from dune
for the build system of my application.

3

but the generated code for it is currently huge

What’s the current size ? Are you compiling with whole program compilation ?

Js_of_ocaml has some support for dynlink but it’s not free. In particular, dead code elimination will be limited

4

If you are comfortable with Dune … I had a variation of the problem you have and solved it with a generated Dune include file. My problem was how to statically combine N “components” into a small number of executables, where N has to be discovered at build time. It doesn’t involve JS, but perhaps the idea translates to your problem. Which … if I were to reframe your problem … is how to statically generate N libraries, where N has to be discovered at build time.

Recreating my steps:

  1. My generated Dune include file is at https://github.com/diskuv/dkml-installer-ocaml/blob/main/installer/bin/dune.inc; I purposely promote it to the source code directory, but you can let Dune keep it in the build directory if you’d like by adjusting (mode promote) in step 2.
  2. The Dune include file is both generated and included in https://github.com/diskuv/dkml-installer-ocaml/blob/main/installer/bin/dune. The core structure is (rule (target dune.inc) ...) (include dune.inc)
  3. The rule invokes executables I hand-crafted that discovers the N components (*) and prints out a dune.inc file. The central code is https://github.com/diskuv/dkml-install-api/blob/main/installer/dune-generator/main.ml. You would structure your dune.inc so that there is one (library ...) statement per semgrep language.

Not sure if that is what you were looking for.

(*) In my scenario … which is to generate an installer of components … I used ocamlfind to discover components at build time; in semgrep I think you have a JSON file you can parse.

5

Doesn’t semgrep use Tree-sitter for parsing the code? Are you using wasi for that part? I’m curious

6

I guess you mean https://wasi.dev/. We’re not using anything yet to generate JS.
tree-sitter indeed can generate WASM libraries for each language (I didn’t look what it’s using; maybe this wasi), but then OCaml side
we also have code that parse the output of tree-sitter (basically Concrete Syntax Trees of languages)
and this code is huge and I would like to generate different libraries for each OCaml wrappers of tree-sitter languages.