Comint version of utop in emacs

Ecosystem
1

Hello OCaml friends,

I have a weird Emacs-related puzzle that I was hoping you could help me with. The issue is that when I fire up a REPL using C-c C-s RET ocaml exec -- dune utop (C-c C-s being tuareg-run-ocaml) in one of my dune projects on my work machine, it loads a fully-functional OCaml REPL in comint mode that looks like this:

good-utop

Yay! This is what works best for me, because I like having a project-aware REPL with completion etc. while also being able to use all the features of comint mode.

However, when my colleague does exactly the same thing on his work machine, he gets a completely unusable REPL that looks like this:

bad-utop
bad-utop1040Ɨ526 123 KB

As you may be able to tell from the screenshot, that is running utop in the form in which it would appear in a normal terminal, which is completely unsuitable for Tuareg interactive mode because of all the ANSI escape sequences, etc.

Hereā€™s the puzzling part. I canā€™t tell what could be causing this. It canā€™t be anything in my Emacs init file, because if I fire up a fresh emacs -q /path/to/ml_file.ml, do M-x package-initialize, do M-x tuareg-mode, then do C-c C-s RET opam exec -- dune utop, I get the good comint REPL. When my colleague fires up a config-less Emacs on his machine using the same method, he gets the unusable REPL. My colleague and I have the same versions of utop, dune, opam, and emacs on our machines.

Furthermore, when I open Proced in Emacs, I see that although my REPL buffer visually resembles an old-school OCaml REPL (as opposed to an in-a-terminal utop with all of its colors, windows, and such), it is in fact running a built-by-dune-for-the-project utop. The command itā€™s running is:

/path/to/my/opam/switch/bin/ocamlrun /path/to/my/project/_build/default/lib/.utop/utop.bc

The PID of my buffer matches the PID of that process in Proced, which I think settles the question of whether this old school-looking thing is secretly a dune-built utop REPL.

As a matter of pure intellectual curiosity, I would love to learn what could be causing this discrepancy. But as a matter of practicality, I would like to know how to be able to reproduce the great dev environment I have on my work machine voluntarily, rather than as a happy accident.

2

I donā€™t use utop, but ā€¦ some suggestions (for both your and your friendā€™s environments ā€“ and then compare):

  1. check your homedir for ā€œ.ocamlinitā€ ?

  2. I have a special script that I use to set up my opam environment before starting OCaml; I donā€™t know how youā€™re doing it, but it might be worth checking that itā€™s done correctly.

  3. In this day and age, we have a ton of stuff in our homedirs and environments. It might be worth making a fresh user ID on each of your and your friendā€™s machines, and replicating the failure. This would eliminate all variation related to your actual homedirs and environments, leaving only what is given by the system and its installed packages.

If in #3 you find that either you donā€™t get the nice env you have in your home env, or your friend finds that they get the nice env in the fresh ID env, thatā€™ll reduce the distance to search for whatā€™s wrong. If you both find that your experience persists, then itā€™s time to look at the installed packages, e.g. whatā€™s installed for your emacs.

Just some ideas.

P.S. Needless to say, it would be very important to ensure that you and your friend are doing exactly the same thing. Iā€™d go as far as getting your ā€œprintenvā€ and comparing to make sure nothing odd is different.

3

Thanks!

I took your suggestion of creating a new UNIX user account with a minimal home directory, and some interesting discoveries ensued. I have determined that this behavior is due to the version of Tuareg mode that Iā€™m running. I accidentally had an old version installed on my main machine which I hadnā€™t noticed wasnā€™t upgrading.

So it looks like this awesome comint dune utop functionality was present in Tuareg version 2.2.3, and somewhere in between there and 3.0.1, it broke. Sad. I may try to get in touch with the Tuareg maintainers to see whether thereā€™s an easy way of bringing it back, but first Iā€™ll investigate the reasons for it via some Emacs debugging and post my findings here.

4

grin Iā€™m glad you were able to isolate which package was causing your problem! A further suggestion: before actually debugging, try installing different versions of Tuareg, so you can get it down to ā€œrelease N works; release N+1 failsā€. That should help you get to the actual culprit code change most easily. Itā€™s the old ā€œbisect the version historyā€ trick.

5

Great suggestion. I just did a deep dive of this kind, and I have confirmed that 2.2.0 > 2.3.0 is the Tuareg version delta that broke the nice, functional comint utop. I havenā€™t had the chance to look closely at the two sources yet, but I did notice that Tuareg 2.2.0 requires me to load one file, tuareg.el, whereas Tuareg 2.3.0 requires me to load these files in the following order:

  • tuareg-compat.el
  • tuareg-opam.el
  • tuareg.el

Not necessarily related to the issue, but one file vs. three files seems like a significant change to the code. Will take a closer look and post my findings here.

6

Ooh ooh ooh, I found the commit that breaks comint utop:

And thatā€™s not all. If I change that one line in the latest version of Tuareg (3.0.1) to say (process-connection-type nil), then the beautiful comint utop works once again! This line here:

1 Like
7

Ok, I made a PR!

8

Update: Stefan Monnier came up with this quick fix that can go straight in your init file:

    ;; Workaround problem where UTop assumes it's running in an
    ;; ANSI-compatible terminal (issue #314 of Tuareg).
    (advice-add 'make-comint :around #'my-utop-workaround)
    (defun my-utop-workaround (orig-fun name &rest args)
      (if (not (equal name "OCaml"))
          (apply orig-fun name args)
        (let ((process-connection-type nil))
          (apply orig-fun name args))))

He also made a pretty compelling case that utop shouldnā€™t run with all the fancy ANSI if $TERM=dumb. I guess Iā€™ll work on a PR to utop nowā€¦