[ANN] An experimental branch of Merlin based on Domains and Effects

Ecosystem
, ,
1

The Merlin Team :man_mage: is very happy to announce an experimental version of Merlin: Merlin Domains!

As a reminder, Merlin is an editor service that provides advanced IDE features for OCaml. It can be used as a standalone binary or as a library, and is notably used by OCaml LSP Server.In other words, if you are using the OCaml Platform extension with Visual Studio Code, or ocaml-eglot with Emacs, you are already using Merlin under the hood.

Even though over the years we’ve had relatively few complaints about Merlin’s performance, in some contexts (notably very large files), Merlin’s mechanism (parsing file (with recovery) → typing file (with recovery) → performing analysis) could sometimes cause slowdowns!

To address these slowdowns, we started an experimental branch that uses domains and effects, which allow us to interrupt the typing process at the right moment when an analysis request comes in.

In practice, let’s say you run an analysis command on a very large file: the type-checker will progress up to the location that makes the analysis possible, run the analysis phase, return the result, and then continue typing the file. This separation is possible thanks to control flow management enabled by effects, and by having two domains interact with each other.

In practice, this makes the analysis phases much more efficient! We think this is a great example of migrating a regular OCaml application to one that takes advantage of multicore. For a more technical explanation of how this experimental branch works internally, @pitag and @Lyrm will talk about it at Lambda World, in their talk When magic meets multicore - OCaml and its elegant era of parallelism.

Currently, the branch is still in its incubation phase, and we’d be delighted to get your feedback!

How to test the branch

To avoid installing this version of merlin unintentionally, we suggest pinning this branch in the switches where you want to experiment with this new version (compatible with Merlin’s main):

opam pin add https://github.com/ocaml/merlin#merlin-domains

Although this experimental branch passes the test-suite, your feedback is very important to help collect potential bugs we may have missed.
We’ve added a Bug/Merlin-domains label to organize tickets related to this experimental branch.

Next steps

In the future, the goal of this branch (which will be regularly rebased on main) is to become the main branch, so that all users can benefit from these improvements.
At the same time, the rest of the ecosystem depending on Merlin (such as the OCaml LSP Server) will be adapted to also take full advantage of these new features!

Happy Hacking!
The Merlin Team :man_mage:

30 Likes
2

I can only imagine the amount of magic required to make compiler-libs Domain friendly :wink: This is very exciting!

4 Likes
3

This is super fancy use of effects and domains. Great work getting this done and, more importantly, improving the performance!

1 Like
4

Seeing OCaml have such great editor tooling and there still being more effort spent in making it even better is great!

3 Likes
5

In practice, not much magic is required, making the whole thing even more interesting! The fun part is that it’s acceptable for Merlin to occasionally return a wrong result, because … this is not a critical system, and you can do the request again. Thanks to OCaml’s strong memory model, data races are not as problematic as in other languages and can therefore be (somehow and sometimes) considered acceptable. Part of this testing phase is determining whether the trade-offs we made (i.e. the data races we didn’t fix) fall within that acceptable range.

4 Likes
6

Yes, I insist on:

Part of this testing phase is determining whether the trade-offs we made (i.e. the data races we didn’t fix) fall within that acceptable range.

Even if, about data-races we have confidence (because of the test-suite) about having nice result (in a nice time frame!)

Please, test-it and give us some feedback! :innocent:

1 Like
7

I need to test this out with [smaws](GitHub - chris-armstrong/smaws: OCaml bindings for AWS APIs, built on eio) because it can generate some really large modules, but I’m not sure I’ll get round to it before this is released.

Can I test this with the current LSP version if I just pin this merlin branch as suggested above?

8

This experimental branch does not yet bring many changes in the context of LSP… this is mainly for Merlin users (baremetal). LSP is the next step. however, we would be delighted to have an experience. I think the ‘right way’ to proceed would be:

# In  the context of your project, in a local switch
opam pin https://github.com/ocaml/merlin#merlin-domains
opam pin add ocaml-lsp-server.dev https://github.com/ocaml/ocaml-lsp

Keep us posted and thank you for giving it a try!

9

Thanks. I’m trying to install it in a switch running 5.3.0 but I’m running into compile issues:

=== ERROR while compiling merlin.5.5-503 =====================================#
# context     2.3.0~alpha1 | linux/x86_64 | ocaml.5.3.0 | pinned(git+https://github.com/ocaml/merlin.git#merlin-domains#7287341c0fe4fda7d468f252a6bc3c48b5ab0f68)
# path        ~/dev/smaws/_opam/.opam-switch/build/merlin.5.5-503
# command     ~/dev/smaws/_opam/bin/dune build -p merlin -j 19
# exit-code   1
# env-file    ~/.opam/log/merlin-36634-5d6b3a.env
# output-file ~/.opam/log/merlin-36634-5d6b3a.out
### output ###
# (cd _build/default && /home/chris/dev/smaws/_opam/bin/ocamlc.opt -w -40 -open Ocaml_utils -open Ocaml_parsing -open Ocaml_typing -open Merlin_kernel -open Merlin_utils -open Merlin_analysis -open Merlin_kernel -open Merlin_commands -g -bin-annot -bin-annot-occurrences -I src/frontend/ocamlmerlin/.ocamlmerlin_server.eobjs/byte -I /home/chris/dev/smaws/_opam/lib/merlin-lib/analysis -I /home/chr[...]
# File "src/frontend/ocamlmerlin/new/new_merlin.ml", line 117, characters 14-26:
# 117 |               cancel_typer shared);
#                     ^^^^^^^^^^^^
# Error: Unbound value "cancel_typer"
# (cd _build/default && /home/chris/dev/smaws/_opam/bin/ocamlopt.opt -w -40 -open Ocaml_utils -open Ocaml_parsing -open Ocaml_typing -open Merlin_kernel -open Merlin_utils -open Merlin_analysis -open Merlin_kernel -open Merlin_commands -g -I src/frontend/ocamlmerlin/.ocamlmerlin_server.eobjs/byte -I src/frontend/ocamlmerlin/.ocamlmerlin_server.eobjs/native -I /home/chris/dev/smaws/_opam/lib/mer[...]
# File "src/frontend/ocamlmerlin/old/old_command.ml", line 130, characters 15-32:
# 130 |   let shared = Domain_msg.create () in
#                      ^^^^^^^^^^^^^^^^^
# Error: Unbound module "Domain_msg"

I thought you might be on 5.5, but the version constraints prevented that.

10

Hmm strange and if you try that:

# In  the context of your project, in a local switch
opam pin add merlin-lib.dev https://github.com/ocaml/merlin#merlin-domains
opam pin add dot-merlin-reader.dev https://github.com/ocaml/merlin#merlin-domains
opam pin add ocaml-lsp-server.dev https://github.com/ocaml/ocaml-lsp