Merlin server difficulty on MSYS2

Hi everyone,

I thought I’d post a slightly arcane scenario on the off-chance that someone might be able to suggest an approach to resolving it. Please bail out when your “nonsense bingo” card is full!

merlin

I’m trying to get OCaml working in emacs under MSYS2 on a machine to which I don’t have admin access. I’m getting enormous lags (and failure) when typing in an OCaml buffer when merlin-mode from merlin-eldoc is enabled, even when dune build -w is running in the project directory.

merlin-call is getting end-of-file when trying to parse output from ocamlmerlin (probably because there is none).

An example merlin-debug-last-commands value is

((("ocamlmerlin" "server" "locate" "-protocol" "sexp" "-filename" "c:/msys64/home/username/Projects/Learning/OCaml/hello/bin/main.ml" "-position" "6:23" "-look-for" "ml")
  . ""))

An example (merlin-debug-last-commands) output is

"Last commands executed, most recent at the end:
command:  ocamlmerlin server locate -protocol sexp -filename c:/msys64/home/username/Projects/Learning/OCaml/hello/bin/main.ml -position 6:23 -look-for ml
- result: 
command:  ocamlmerlin server locate -protocol sexp -filename c:/msys64/home/username/Projects/Learning/OCaml/hello/bin/main.ml -position 6:26 -look-for ml
- result: "

ocamlmerlin does seem to work from the shell command line, at least as ocamlmerlin single -flags-help. I’m using fish, hopefully that isn’t adding any complication.

I tried manually calling ocamlmerlin with a similar command to the one implied by merlin-debug-last-commands above (and with reference to the protocol); it seems to exit with status 1 and no output. I wonder if it could be something to do with the way paths are expressed, but couldn’t find a working combination. The following did work though:

 ocamlmerlin single type-enclosing -position '1:11' -filename main.ml < bin/main.ml

returning

{"class":"return","value":[{"start":{"line":1,"col":10},"end":{"line":1,"col":11},"type":"int","tail
":"no"},{"start":{"line":1,"col":10},"end":{"line":2,"col":7},"type":"int -> int -> int","tail":"no"
}],"notifications":[],"timing":{"clock":228,"cpu":0,"query":0,"pp":0,"reader":0,"ppx":0,"typer":0,"e
rror":0},"heap_mbytes":0,"cache":{"reader_phase":"miss","ppx_phase":"miss","typer":"miss","cmt":{"hi
t":1,"miss":1},"cmi":{"hit":0,"miss":1}},"query_num":0}

Changing single to server caused there to be no output, and exit code 1. Changing server to single in merlin.el didn’t work (still end-of-file like original problem). It feels like either I’m not calling ocamlmerlin server correctly, or there’s some ?platform reason it’s not returning anything.

tuareg

I also got a build error with tuareg which I worked around; I’m not sure whether it’s relevant to the above problem.

I installed OCaml with winget, which was great. I installed emacs within MSYS2, which worked. opam install tuareg failed with Error: buffer-read-only when processing tuareg-site-file.el. There is no obvious permissions problem with that file. I looked at tuareg’s Makefile, and found this:

$(EMACS) --batch --eval '(if (>= emacs-major-version 28) (make-directory-autoloads "." "'`pwd`'/$@") (setq generated-autoload-file "'`pwd`'/$@") (batch-update-autoloads))' "."

(In elisp (if COND THEN ELSE...) apparently evaluates all ELSEs and returns the last, if COND yields nil.)

I ran the emacs line manually and got the same error, plus Package autoload is deprecated.

In any case, re-running the tuareg installation “worked”, presumably because a tuareg-site-file.el was generated, and therefore that Makefile stanza didn’t re-run.

Tuareg does seem to work, so again I’m not sure this is relevant to my merlin problem.

so…

Thanks for reading this far. I’d love to try any suggestions you may have.

I’ve never used Merlin, and never used MSYS2, but long ago, I was a past master of GNU emacs and elisp. Long ago == >20yr ago, so … haha, maybe this is all useless. Some suggestions:

(0) can you install MSYS2 in a virtual machine locally and try to do everything there, to test everything out? That would allow you full admin access, but also remove any lag/delay that wasn’t software-related.

(1) try to reproduce the steps that are failing on a machine you have full access to, and that is local, so you can go as deep as possible.

(2) can you SSH to this remote machine? Can you use SSH to this remote machine from within a LOCAL emacs (e.g. tramp-mode) ? If you can, then try to run the emacs end of tuareg on your local machine, with the Merlin running on the remote machine. If that works, then you have some evidence that the problem is emacs-related. If it doesn’t work … well, can’t know much.

(3) It seems that you’re seeing a low-level merlin invocation that is failing? If so, maybe try to get to where you can run that on a local install that is at the same stage as your remote install? To see what the difference is?

OK, this is probably all useless. Of all this, #0 is what I’d focus on: try to get a local MSYS2 install where I could have full control.

(4) I would look into whether there’s a way to run Merlin as a network server, with emacs connecting via a socket. If there is, then that would help with running emacs on my local machine with Merlin remote. And then I could use Wireshark to grab the TCP traffic and I could debug that way. Just an idea.

ETA: (5) If you haven’t already done so, it might be worth rehearsing everything you want to do, on a Cygwin install. That way, the differences you’ll encounter will be those that separate Cygwin from MSYS2.

Wow thank you. I neglected to mention that the “machine” is actually just a corporate laptop. I do have a personal Windows laptop and will investigate your suggestions.

aha, so the MSYS2 machine is also local. In that case, another suggestion: do these steps all on a local machine that I control, and if at all possible in VM instances:

(1) rehearse everything on a machine you know well, that is known to work. For me, that would be Linux; for you might be something else.

(2) rehearse on Cygwin (that should also be known to work – I’d look to see what’s known about that, just to be sure)

(3) then rehearse on MSYS2

#3 should fail, obvs, in the way you’re seeing. If it doesn’t, you know what the problem is grin – whatever the corporation did to the laptop. If it fails then I’d use the fact that I have complete control over the MSYS2 instance to start investigating.

(4) Are there unit-tests and integration-tests for Merlin? If so, then I’d run those for each of the stages above.

(5) You said that you’d got a Merlin invocation that didn’t produce the correct output ? If I have each of the environments above, I could run analogous versions of that command, and see what it should be producing – that might help me to figure out whether the problem was with Merlin, or emacs.

This is the part that I was alluding to: I think you’ll want to figure out (a) whether this invocation is correct (or emacs is screwing up) and (b) if so, what the output should be. That’s what the installs on Linux/Cygwin/MSYS2 are for – so you can run the various steps on the various instances and compare output. If you reach this point, you’ll have Merlin debugging ahead of you, but at least, it won’t be a massive system composed of emacs and Merlin talking over a pair of drink straws.

Do you have a particular reason to want emacs from MSYS2 rather than a Windows build of emacs (winget install GNU.Emacs, I think)? I’ve not tried it, but if the comparison with Vim-for-Windows is anything to go by, it at least removes worries about line endings and path translations from the equation.

Otherwise, is anything in Tuareg/Merlin on Cygwin of help? (similar setup, just with Cygwin)

Thank you. winget install GNU.Emacs actually asks for elevation to Administrator, and --scope user doesn’t work.

That cygwin post was very helpful! I fixed opam-share as it suggested, and also changed this line in merlin.el to
(filename (replace-regexp-in-string "\r$" "" (car (process-lines "cygpath" (buffer-file-name (buffer-base-buffer)))))).

Now it’s communicating correctly and giving feedback–yay!

(p.s. I should also mention that my emacs exec-path had some weird entries with C /path/elements rather than C:/path/elements, which I fixed manually. Will try to work out where they came from.)

1 Like

Excellent! Looks like part of that could go straight in a pull request to Merlin?

Given that you’re on MSYS2, I wonder if you might be seeing a variant of Incorrect message on windows for updating environment vars · Issue #6134 · ocaml/opam · GitHub (essentially a /c/ getting mistranslated somewhere)

Happy to sketch a PR once I identify the best way of detecting the platform accurately. (uname -a seems to end in Msys).

I haven’t got to the bottom of the erroneous exec-path entries yet but it looks like opam-init/variables.fish sets PATH incorrectly using set -gx PATH 'C \\Users\\username\\AppData\\Local\\opam\\default\\bin' $PATH; instead of set -gx PATH '/c/Users/...' $PATH.

Another heuristic may be to test that there is a cygpath binary in the same directory as the emacs binary which is running (assuming that’s readily available in elisp?). The snag with identifying via external tools is that even uname may have come from something else!

I have been using dkml-runtime-common/unix/crossplatform-functions.sh at caef94b147cd4acfb2622bacf2f20f897c13d0d0 · diskuv/dkml-runtime-common · GitHub to test for MSYS2.

1 Like