What's your development workflow?

Learning
8

Having worked in Clojure and ClojureScript for about a decade prior to moving on (first to Haskell, briefly Rust, and then to OCaml), I absolutely understand and sympathize with your yearning for a quality REPL experience. There’s good and bad news:

  • the bad news is that non-lisps simply don’t have “good” REPLs as we would expect, for all sorts of reasons. However,
  • the good news is that OCaml provides IMO an almost uniformly superior development workflow to what I was accustomed to (in Clojure, but also in other lisps, even those with very integrated development environments like Racket)

(Parenthetically, utop is very good given the constraints that it must operate within given OCaml’s compilation and execution model, much better than the equivalents [when I last used them] in Rust and Haskell. You should use utop for quick spot experiments and such, but turn away once you’re working on code that should persist past a given toplevel session.)

Everything that makes it possible to work productively in OCaml comes from:

  1. the compiler toolchain being fast (no, seriously, really fast)
  2. dune & co. being fast (no, seriously, really fast)
  3. editor tooling (specifically, type hints, completions, go-to-definition, etc) being fast (enough), accurate, and “total” (i.e. it’s extraordinarily rare for type information not to be exactly where you’d expect, even in the face of e.g. nontrivial ppx transforms)

None of this is the case in e.g. Clojure (or any other lisp IME), so use those attributes to your advantage:

  1. Always have a terminal buffer on the side running e.g. dune runtest -w --no-buffer. Especially as you’re just starting out, unless you’re doing something pathological, you should get test feedback in a matter of milliseconds, every single time you hit save. This kind of rapid, total feedback should feel superior to typical Clojure workflows, whether you’re loading buffers into the REPL piecemeal (lots of mental accounting around which namespaces to load in which order), or having some tooling do full reloads on save (which tend to be quite slow even with smaller codebases).
  2. As your codebases get larger, use dune aliases to restrict the set of things that are run when you hit save; e.g. if I’ve defined a testarrangement executable in a test or tests stanza (which could define dozens or hundreds of separate test executables in total), then (rule (alias arrangements) (action (run ./testarrangement.exe))) will give me a @arrangements alias I can use to trigger just that one via dune build @arrangements --no-buffer -w
  3. In hindsight, so much of what we hype up as “exploratory programming” in the REPL is really just coping with the lack of useful type information. You don’t have to run to a REPL anymore to see if a given function takes a list or a seq, or see what the shape of a given bit of data is like. Since the vscode extension or merlin in emacs reveal the types in your program quite nicely, use them, you don’t have to rerun tests solely to verify that the type-checker has done its job properly. (Fully getting over this compulsion took me months!)
  4. Note that you absolutely don’t have to use any kind of particular test-driven approach; these executables are just modules that have top-level effectful expressions, so you can do anything you want in them, and produce any kind of feedback that’s helpful to you (terminal output, automatically dumping visualizations to disk that vscode or emacs can readily detect and show/update, trigger an audio ding to indicate a success condition, etc).
  5. Further on this point, you can always keep a “scratch” program off to the side, set off with its own alias, which you can abuse as you see fit.

At the end of the day, compared with Clojure/ClojureScript/Racket/CL/etc, I end up with the same kind of feeling of having a conversation with my program and the compiler, but with both much more granularity (because I always have type visibility at every level of the program), much broader scope (because I feel safe to make much larger changes in between e.g. test runs), yet with lower latency at every step.

Caveats to all of this include:

  • Yes, you’re throwing away aggregate program state every time the watcher restarts things. 99.8% of the time (my best guesstimate :wink: ) , that is completely okay (n.b. OCaml is fast, much faster for the same tasks than Clojure and even Java). In the extremely rare cases where your program / tests depend on some data that requires significant database interactions or a lot of compute to arrive at, it’s okay to have that work cache its results locally via a Marshal’d file so that subsequent runs are snappy again. I’ve only felt like I’ve had to do this once in ~three years.
  • very occasionally, I’ve witnessed the vscode extension somehow lose track of its type information (reporting everything to be weak types); restarting the language server or sometimes just making a single whitespace edit knocks things back into order (it thankfully happens so sporadically I’ve not bothered filing an issue)
  • You can’t open a REPL or toplevel into the running state of a remote system, something I used to use occasionally for debugging purposes. This is a genuine downside of non-lisp systems :person_shrugging:. I’ve compensated by being much, much more diligent about logging and error reporting otherwise, and by taking steps to be able to ship runtime state out of deployed systems when necessary for examination in a development environment.

I hope this helps :heart:

31 Likes