Merlin: Goto definition inside installed libraries

I raised an issue on GitHub a little while ago about not being able to use MerlinLocate from inside installed libraries (I can jump to them from my project but then I hit a dead end from there). @trefis advised moving the discussion here.

From what I understand, to jump to definition Merlin needs .merlin files, which Dune auto-generates for your project but are not available for libraries installed through Opam. Excluding them seems to be a deliberate choice, which confuses me, since jumping through definitions and implementations of libraries is important to understand and work with them. My questions are:

  1. Am I correct that this is a design choice as opposed to a technical challenge, and if so, can this choice be revisited?
  2. If .merlin files will not be released, what is the best way to get jump to definition working locally? Should I create .merlin files in the opam directory tree (and if so what should they look like and where should they be placed)? Or should I symlink to the installed libraries from my project – or copy them to my project – so that Dune is able to generate Merlin files for them?
  3. Is this something that the Duniverse project will resolve?

In the general case, the source code of an installed library is not available. So there is no definition to jump to, except maybe the declaration in an MLI file but even that is not required for the compiler to link the library. Are you assuming a different setup where the library source code is available?

1 Like

Yes, I am assuming source is available, though that’s a fair point. The source does seem to be available for libraries like core and async, and I imagine for several others as well. So I suppose my questions should be qualified with that assumption.

Note that for topkg packages you can set TOPKG_CONF_DEBUGGER_SUPPORT=true in your environment before opam installing topkg packages. This should install sources for these packages.

Like I said, in the general case a package in Opam does not install its source code and therefore you can’t jump to a definition. However, in a project built with Dune or Jbuilder you can unpack the code of selected libraries in your project, where they will be built and visible for Merlin. See Dune Scopes.


Can you elaborate on how to get this to work? What my directory structure needs to be / what files I need (dune, dune-project, core.opam, etc. what needs to be created and placed where)? I’ve tried several iterations but none of them seem to be respected and the documentation on this is sparse.

I am working an a small webapp that uses Opium, Moustache and Ezjsonm as libraries. When I unpack them in the top-level of my project (opam source moustache, …) and build the project as before, now the unpacked sources will be used and I can jump to any code inside them using Merlin. My own code, dune files and so on are completely unchanged.

β”œβ”€β”€ Makefile
β”œβ”€β”€ checkout.opam
β”œβ”€β”€ db
β”‚   β”œβ”€β”€ Makefile
β”‚   β”œβ”€β”€ club.csv
β”‚   β”œβ”€β”€ crew.csv
β”‚   β”œβ”€β”€ db
β”‚   β”œβ”€β”€ db-export.sql
β”‚   β”œβ”€β”€ db.db
β”‚   β”œβ”€β”€ db.sql
β”‚   β”œβ”€β”€ equipment.csv
β”‚   └── purpose.csv
β”œβ”€β”€ dune-project
β”œβ”€β”€ ezjsonm.0.6.0
β”‚   β”œβ”€β”€
β”‚   β”œβ”€β”€ LICENSE
β”‚   β”œβ”€β”€
β”‚   β”œβ”€β”€ ezjsonm-lwt.opam
β”‚   β”œβ”€β”€ ezjsonm.opam
β”‚   β”œβ”€β”€ lib
β”‚   β”œβ”€β”€ lib_test
β”‚   └── pkg
β”œβ”€β”€ html
β”‚   β”œβ”€β”€ css
β”‚   β”œβ”€β”€ index.html
β”‚   └── index.json
β”œβ”€β”€ mustache.3.0.2
β”‚   β”œβ”€β”€
β”‚   β”œβ”€β”€
β”‚   β”œβ”€β”€ Makefile
β”‚   β”œβ”€β”€
β”‚   β”œβ”€β”€ examples
β”‚   β”œβ”€β”€ lib
β”‚   β”œβ”€β”€ lib_test
β”‚   β”œβ”€β”€ mustache.descr
β”‚   β”œβ”€β”€ mustache.opam
β”‚   β”œβ”€β”€ pkg
β”‚   β”œβ”€β”€ specs
β”‚   └── test
β”œβ”€β”€ opium.0.16.0
β”‚   β”œβ”€β”€ Makefile
β”‚   β”œβ”€β”€
β”‚   β”œβ”€β”€
β”‚   β”œβ”€β”€ VERSION
β”‚   β”œβ”€β”€ dune-project
β”‚   β”œβ”€β”€
β”‚   β”œβ”€β”€ examples
β”‚   β”œβ”€β”€
β”‚   β”œβ”€β”€ lib_test
β”‚   β”œβ”€β”€ opium
β”‚   β”œβ”€β”€ opium.opam
β”‚   β”œβ”€β”€ opium_kernel
β”‚   β”œβ”€β”€ opium_kernel.opam
β”‚   └── pkg
└── src
    β”œβ”€β”€ dune
    β”œβ”€β”€ main.mli

Does this work for libraries that were not themselves built with dune? When I try opam source-ing core, core_kernel, base, etc. and their dependencies into a vendor/ directory, dune builds them fine but MerlinLocate seems to ignore them. I suspect (though am not sure) this is because these libraries are missing dune files, as they have jbuild files instead. If this is the issue, is there a way to reliably translate the jbuild file into the needed dune file (they follow similar formats but I don’t think they are 1:1)?

This should work with jbuild files as well. I don’t know about the Janes Street projects in particular.

@lindig Do you mind showing a minimal working setup with core? I’m retrying this again (I never was able to get it working) and dune build starts complaining about conflicts between the libraries in my vendor/ directory and libraries in the opam directory. I started moving more and more of these conflicts into vendor/ but then ran into a problem with ppx_compare.runtime-lib, a base_bigstring dependency that still wasn’t being recognized even after I added ppx_compare to vendor/. Fwiw I also added (vendored_dirs vendor) to a dune file in the root of my dune workspace.

@diml Do you know if there’s a conflict arising here between vendoring and the usage of (preprocess (pps ppx_jane)) in the dune file for base_bigstring that’s causing it to look in the opam directory instead of the vendored directory?

I think

opam install -v <package>

is supposed to install the sources.

I know it’s from a while ago but did you have any joy with this @ddickstein?

I find it a little odd there appears to be no straightforward way to move around installed libraries in the cases where the source is available, and also that nobody seems to require this functionality, which makes me think I’m doing something wrong.

1 Like

If you use dune then jumping to definition works out of the box most of the time, without installing sources or using duniverse. For example:

 (name test)
 (libraries yojson mtime core containers))
(* *)
let _ = CCList.iter
let _ = Mtime.ns_to_s
let _ = Yojson.write_assoc
let _ = Core.List.iter

I can successfully jump to CCList, CCList.iter, Mtime, Mtime.ns_to_s, Yojson, Yojson.write_assoc, Core and Core.List.iter.

Jumping to Core.List fails due to

Jumping Mtime only brings you to the .mli file, since mtime doesn’t install its sources (which is quite useful in itself for browsing documentation).

In order to jump from libraries to other libraries, libraries would need to install .merlin files. I’m not sure if that’s a good idea, but I’ve opened an issue here:

Thank you for opening that issue! It is indeed the final scenario I’m after. Just as a concrete example I came across today, if I have the below in a file in my Dune project:

(* *)
let _ = Async.with_timeout

Then I can jump to Async.with_timeout ok. However it turns out this is defined as:

(* *)
let with_timeout = Clock.with_timeout

So it would be quite handy to be able to jump directly to Clock.with_timeout, though it’s not currently possible.

While searching I came cross this topic How to get merlin to do a good job on project dependencies? How to get dune to generate better .merlin files? and the suggestion of adding .../.opam-switch/build/** to my .merlin file worked, however it’s a manual fix and gets cleared the next time I run dune build.

If ppx_jane is available in the workspace, dune should prefer this version to the installed one. If you have a reproduction case, we can look into it.

Apologies for the delay here. I’ve opened an issue on GitHub with a repro case.

I think I’ve found a way to achieve this so thought I’d mention it here, it also required a bit of manual work so further thought it’d be good to post in case there’s a better way. Steps I took were:

  1. Initialise a dune project, e.g:
    dune init proj hello_dune --libs async

  2. Add package stanza + dependencies to dune-project (after running dune build), e.g:

(generate_opam_files true)

 (name hello_dune)
 (synopsis "Short description")
 (description "Looooong description")
   (dune (>= "2.0"))
   (async (>= "0.14"))))
  1. Run dune build followed by opam monorepo lock && opam monorepo pull to pull any dependencies into the local duniverse folder.

  2. After once again running dune build, it should be possible to jump around within dependencies, for example if I add the below to bin/, then I can jump to Async.with_timeout, and from there to Clock.with_timeout (and onwards if I wish).

open Async

let () =
  let _ = Async.with_timeout in
  print_endline "Hello, World!"

It’s not too painful but if there are any steps which could be streamlined I’d grateful to hear how! Particularly I’m wondering if there’s a better approach than manually adding the dependencies to the dune-project package stanza - is there a way to generate this section automatically somehow, maybe from the libraries section in the dune files?

1 Like