Utop very slow?

Learning
1

I find that utop is very slow when I paste large programs. Is this normal, or maybe should I proceed differently ?
What I do is: I write programs in my text editor (visual studio code), and then I copy and paste the parts I want to test in utop (which I opened separately in a terminal), using ctrl+C and ctrl+shift+V, but when I paste more than, say, 200 lines, utop is very slow to even “ingest” it (tens of seconds before I can even type “Enter”). Once this is done, if I want to recall it using the up-arrow for history, there is no such delay. Since I have to do this very often to test almost every change, it gets a bit long.

Maybe I do not have a correct workflow ?

2

There may be workarounds but it’s likely normal.

Basically stdin is likely set to raw mode to be able to react to each keystroke, e.g. to perform completion. So it’s likely that when you paste your code, quite a bit of code gets executed on every input character which slow things down.

(There is the same defect in down see the issue here).

2 Likes
3

Have you tried using #use "file.ml" (see “Toplevel directives” in the OCaml manual)? Maybe the slowdown is caused by the large content in utop’s history, in which case this would help as you can #use the content of the file directly without loading it into the history.

2 Likes
4

Thanks for the suggestion. Actually, I usually test in utop my multifile project “up to a point”. Say this “point” is in the middle of my file c.ml. Then, I type in utop

#mod_use "lib/a.ml";;
#mod_use "lib/b.ml";;

(very fast indeed) and then I copy-paste the part of c.ml from its beginning up to said point. So what I would need is something like

#use_up_to_line "c.ml" 254;;

(a command which should not be too hard to implement ?).

But maybe I am facing these problems because I am not using an efficient workflow or good practices when developing multifile projects ? (It’s not too hard to self-learn algorithms and theory, but I find it harder to self-learn practices.)

I saw in the link provided by @dbuenzli that the standard REPL does not have this problem, and I may actually fall back to using

$ rlwrap ocaml

for these tests.

5

Well to react to your keyboard in order to provide line edition rlwrap will also have to put stdin in raw mode, so I doubt you won’t see the same problem.

6

Wouldn’t a #paste directive, as in ipython for instance, solve this in a simple way?

7

Actually, pasting in $ rlwrap ocaml is slightly slower than in $ ocaml, but nothing compared with $ utop.

1 Like
8

Perhaps you could use dune utop? Dune will automatically load any given library into the REPL. Then you can explore it interactively, redefine any definitions, and so on. No need for copy-paste.

1 Like
9

Unfortunately, it looks like dune utop <dir> can only load full directories, not individual files, and in my lib/ directory, I have files a.ml … g.ml which are correct “up to” a function somewhere in the middle of c.ml, which is why I use #mod_use on the files before c.ml, and then I manually copy-paste from the beginning of c.ml up to said point.

My projects are very elementary and standard, so I’m suspecting that I run into these problems, not because of tools which are lacking, but because I am not following good practices when developing a decent-sized multifile project (which I have never done, in any language, so far, and for which I am self-learning). I begin by writing my project linearly (first dune init proj myproj, then in lib/ I write a.ml, then b.ml, etc.) but sometimes, when I have already written a.ml…g.ml, I have to do a modification in the middle of c.ml which makes everything after that point not compile, and that’s where I’m facing the above problem when I want to test “everything up to a given point”). There are probably better ways to proceed (“better practices”)…

10

You may want to have a look at the tools mentioned by @Gopiandcode and @let-def in this thread.

2 Likes
11

In that situation I’d likely fix the compile error first, then load the library in dune utop. Another option would be to comment out all the breaking code in c.ml until the build succeeds, then load the library and go from there.

1 Like
12

If that’s what’s happening, one possible solution is to implement the bracketed paste protocol: the application writes a control sequence at start, and then instead of receiving pasted text as if it was typed, it also receives “start of paste” and “end of paste” markers. While paste is active, the application could disable completion.

7 Likes
13

Looks nice but for this to work you basically need support from the terminal app “system” paste function. Which ones do ? EDIT it’s written on the page.

14

There’s a list here. I think it’s a progressive enhancement - if the terminal emulator does not support the mechanism, it will ignore the query at startup and just omit the bracket commands, and it will work as before.

2 Likes
15

Thanks for the tip @emillon ! I did a quick and dirty implementation for down and here it works wonders.

In Terminal.app on macos pasting 200 locs would take around 13.7s, with bracketed paste the difference between ocaml and ocaml with down loaded seems barely noticable (and is less than 1s).

Note that I don’t even try to switch to buffered mode (there would be all sorts of complications to detect the paste end condition), it’s still reading char per char. One oddity is that, at least in that terminal, newlines were being reported as ‘\r’.

6 Likes
16

The built in Terminal on Macs is worthless, and not worth worrying about, IMO. Try with these:

I use alacritty (it is the fastest one IMO) so you may start with it :slight_smile:

17

you could also open emacs with tuareg-mode instead of your terminal.
Then you can paste any code and send the buffer (or any selected text) to the interpreter

2 Likes