Ahrefs is now built with Melange

Ecosystem
, , , ,
3

Hey @zeroexcuses,

thanks for your interest.

I guess this is expected. It is a recent project, and up until now the focus has been on making sure the idea is feasible and it works. With the first iteration of Melange, @anmonteiro proved that it was possible to upgrade the project to be compatible with the most recent versions of the OCaml compiler, and also model it as a “compiler libs” library, rather than a full fork. It also integrated with Dune, but at a raw level: Melange would generate rules for Dune to execute, but there was no concept such as libraries and such yet. The second phase, which has been completed recently, has proven that a deeper integration with Dune was possible and Melange is usable in large projects. I guess we are entering now a third phase of the project which will consist on documenting the toolchain, as well as porting over existing libraries and bindings, so other people can start using it easily.

Besides the points you mentioned, I’d like to mention a couple more:

  • One of the biggest upsides imo is having access to the OCaml editor platform. Years of effort in Merlin, OCaml LSP and extensions like vscode-ocaml makes the Melange developer experience really ergonomic.
  • Another difference is how package management is handled: while with ReScript every dependency can be downloaded with just npm, Melange projects will have to use opam and npm. This is a trade-off: on one hand, most Melange projects will have to deal with both package.json and opam files. But on the other hand they can benefit from opam’s source-based package distribution model for things like PPXs, linters, or any other OCaml tooling. By comparison, consuming any OCaml ecosystem tool in ReScript is more challenging. As npm is a package manager designed for an interpreted language, distribution requires using prebuilt binaries. And the fact that it is based on a now quite old version of the compiler does not make things easier.

I think this is caused by a combination of factors affecting compilation times. Some of them can be solved, some of them are by design:

  • Dune’s design requires walking the whole project tree when building it to find dune files. ReScript, as it’s based on Ninja, can read all the project information in a centralized way, from a single file.
  • Dune doesn’t use file modification times, but rather calculates changes digesting a hash from the file contents.
  • Integration with Dune forced Melange to build things in a more “staged” way, so that object files (cmj) and resulting JavaScript files can be generated in 2 steps.
  • Melange compilation process has not been fully parallelized yet.
  • ReScript optimizes the way the module dependencies of a given module are calculated by caching the results, Melange uses ocamldep through its integration with Dune and this step is not optimized / cached.
  • ReScript has a “tighter” design, which provides more opportunities to optimize things like the ReactJS ppx, which can be “fused” with the compiler. On the other hand, Melange prioritizes extensibility and ecosystem integration, so the ReactJS pre-processing is done through a regular PPX.

I am sure there are many other things that I am not aware of, maybe someone with more knowledge can add or refine the points above :slight_smile:

Again, probably missing some things, but from the top of my head:

  • Melange won’t support anything running on OCaml 5 (yet).
  • Js_of_ocaml allows to compile the compiler itself and create “toplevels”, which is not possible with Melange.
  • I think that Marshal support works well in Js_of_ocaml, while Melange does not support it.
  • Libraries like Unix or Str are available in Js_of_ocaml but not in Melange.

On the upside, in Melange:

  • Writing bindings is a bit easier in Melange (imo) thanks to the inherited ReScript compilation model
  • There is great support for libraries like ReactJS or GraphQL clients
  • Smaller bundles
  • Straight-forward integration with JavaScript tooling like Webpack, NextJS,… thanks to the 1 module:1 JS file compilation model

These three projects are complex, and there is a lot of nuance, hopefully this helps clarifying some of the trade-offs.

18 Likes