OCaml REPL Driven Development?

Clojure is notorious for it’s REPL driven development: https://www.youtube.com/results?search_query=clojure+repl+driven+development

OCaml has “opam utop”, but I’m having trouble finding people describing their “OCaml Repl Workflow”

===

I want to define “Repl based development” not as “occasionally use the REPL”, but something like this:

  1. we start a REPL
  2. we type code in some editor, and continuously “send last expression” to the REPL, and we look at the new output
  3. eventually, we clean up the buffer a bit and store it as code

===

Question:

Does OCaml have a REPL based development workflow?

If so, where is it described? If not, is it due to culture of technical reasons (i.e. the type checker doesn’t work well with REPL based development).

3 Likes

I don’t use Clojure so can’t compare the experience. But I would say OCaml’s REPL experience is amazing.

Assuming you are using dune as your build tool, you can type dune utop to start a new REPL session where you can experience with your own code. But aside from Utop, which fits inside a terminal, there aren’t much editor integration.

Shamelessly plug: I’m building sketch.sh which is an online REPL.

1 Like

Do you use Emacs? People often write Emacs functions by:

  1. firing up blah.el
  2. writing short snippets of code, hitting C-x C-e to evaluate last expr
  3. eventually building the code they need

This is in contrast, to say C++ where one generally writes more code, compiles less frequently, runs even less frequently, and across different runs, there’s no persistent state (besides the file system).

Is it possible to get such a workflow in OCaml?

Well, this is exactly how I’m developing my code every day, and the style that I’m trying to promote. I’m using Emacs as my editor of choice, I believe there should be some sort of integration for other editors (I think I’ve seen something for vscode, but I hope that users of other editors will jump in and advertise their tools).

Initial Setup

Here is my workflow. I’m using Emacs’s Tuareg mode, coupled with ocp-indent for consistent indentation, and merlin for completion and incremental type checking. This is how you can set everything up. I’m not using utop, but an integration with it is also available and one may choose to use utop to be able to use Dune’s utop integration.

Basic workflow

My basic workflow follows your definition, except that if you need to use some external libraries you need to load them into the toplevel (remember that the toplevel is just a simple OCaml program, that is not by default linked with any libraries other than the standard library).

To make OCaml packages available to ocaml/utop toplevel we need to use the topfind utility, which is installed as a part of the ocamlfind package. Given that the latter package is the dependency of merely everything you will most likely have it installed. Therefore our first directive to the toplevel would be

#use "topfind";;

Make sure that you actually type # this is how directives to toplevel are issued. Also, this command is a good candidate to add to your ~/.ocamlinit file, which is evaluated every time you start any toplevel.
Hint: instead of typing ;; use S-Ret to send the phrase to the toplevel.

Now, we can load any library to the toplevel. This could be done with the require directive, e.g.,

#require "core_kernel";;

Hint: to get the list of available packages use the list directive, e.g.,

#list;;

Now, we can just hit C-c C-e to send our phrases to OCaml. Or C-c C-b to send the whole buffer, or C-c C-r to evaluate just a selected region.

Pro tips

Using printers.

While OCaml will try its best to print everything, sometimes it is necessary to install your own printer. You can have many different printers for your data, to give you different views on what is going on, so that you can switch between one and another. (For example, I’m even having a rendering overlay in Emacs, which renders graphs and trees directly in the toplevel. I used to have the same for matrices, when I was doing heavy math).

The install_printer directive enables custom printers for your data, which is very useful during development and debugging. It takes a function, which should have type Format.formatter -> t -> unit where t is the type of your data, e.g.,

type t = Student of int

let pp_name ppf (Student id) = 
   Format.fprintf ppf "%s" (Hashtbl.find_exn names id)

and now we can install it,

#install_printer pp_name

Tracing

OCaml toplevels also provide a nice feature called tracing, which shows how your functions are invoked and what they return. It is much easier to use then the good old debugging output. Especially, when you provide your custom printers (yes tracer will use the same printing facility, so everything will work seamlessly).

To trace a function use the trace directive, e.g.,

#trace find_best_student;;

To stop tracing use the #untrace

Tip: to get the list of available directives, use the help directive.

Developing Large Applications

The REPL driven development suits best the bottom-up style of development. When the application grows it becomes harder and harder to use toplevel. But with the right approach, it scales! Just keep in mind that REPL driven development actually facilitates modular design, and if you keep struggling with toplevel when your application grows it is an indicator that you have problems with your design. The idea of the bottom up development is that you develop small modules which are independent and do not need a lot of context to debug a module.

My approach for developing large applications with REPL is to debug each individual piece independently. Once they are debugged I can build and install them using normal building facitilites and then load them using the require directive like I was loading core_kernel and other dependencies. You can also use Dune utop integration, to run utop (from Emacs of course) that will make your libraries readily available. You can also build your custom toplevels, when necessary.

Another trick, that I find useful in real life. When for some reason your can’t load a dependency of your module in the toplevel, you can easily stub it, e.g.,

(* let's stub some complex external library which is developed 
    by some other guy, and is still not yet ready *)
module Database  : Database.S = struct 
   type t = string
   
   let connect str = printf "connect %s" str; 
   let select conn query = 
     printf "%s> %s" conn query;
     []
end

(* here comes our code that needs the database, which is not yet ready,
    but it doesn't stop us anymore *)

let start_driviving env = 
   let db = Database.connect env.main_host in
   let waypoints = Database.select db waypoints_query in
   drive_through waypoints

Further reading

P.S. I will also hope that users of other editors will jump in and share their own experience.

34 Likes

@ivg : Thank you for the detailed response. I don’t understand half the issues going on, but they sound precisely like the type of issues one runs into with reloading in a single persistent REPL.

If anyone has a similar workflow using either VIM or IntelliJ, I’d love to hear those too. Thanks!

This looks useful. Thanks.

@ivg : This is greedy on my part:

Some time ago, I saw http://www.parens-of-the-dead.com/e1.html which demonstrated the power of REPL driven development in Clojure.

Is there any chance you can make a short 10-15 minute video of you hacking on toy projects / demoing your setup? (Bonus points if some program show key presses on screen).

For someone new to OCaml, it’s much easier to mimic expert (even if not perfect) technique than to come up with new techniques.

3 Likes

There is always a chance :slight_smile: I will keep this request in my todo list, so that as soon as I will get some free time I would probably do this.

2 Likes

If 20 people watch the 15 min video, the amortized cost / promotion is only 45 seconds. :slight_smile:

2 Likes

I do something similar in vim using a plugin, briefly described here. I also use custom printers to render trees – but this is all ascii art.

1 Like

I use Vim and entr. ‘Save’ file and entr executes ‘ocaml’ in another terminal. Look forward to hearing about how to use Vim and ‘send last expression’

Happy to see that I am not the only one who likes bottom up development. :slight_smile:

1 Like

I got a very basic setup working via

  1. start a tmux session
  2. run “dune utop” in tmux
  3. in VIM, put cursor over ocaml expression, hit C-x C-e, enter tmux info (only first time)

I’m looking forward to hear more advanced techniques.

2 Likes

I use kakoune and my workflow is:

  • :repl ocaml to create a new window (managed by the window manager or tmux, depending on your setup) with the toplevel running in it. I sometimes do :repl only and do some setup in bash before starting the toplevel. Kakoune knows how to find the repl window (using xdotool and other programs) and send text to them (with the :send-text command)
  • I select the code (doing <alt-a>p to select a paragraph for example) and send it to repl with the key #.
  • Depending on how long I may be taking with that module or task but I sometimes put the setup statements (populating the scope with variables, etc) within my code to be sent to the repl

The downside is because the editor isn’t very popular (yet), sometimes I will need to write it myself when I need a plugin and nobody has written one for it yet (I built an ocp-indent plugin, for example)

2 Likes

It’s also worth noting that Down just landed, a really promising alternative to utop in my opinion.

5 Likes

What’s the problem with utop?
Utop is just amazing man! :star_struck:

1 Like

Curious to know where down scores over utop

I have been playing with Down and the Ctrl-t bound to ocp-index makes the difference imho. I know that there is ocp-index-top but it’s not the same.

2 Likes