How do you stay productive in OCaml?

I wasn’t even aware that this can be an issue - when debugging I’ve always added the externally visible types 'a, 'b and etc.

for completeness adding:

and OCaml - Language extensions
which supports both function syntaxes

  let rec aux : type a b. a list -> f:(a -> b) -> acc:b list -> b list =
   fun xs ~f ~acc ->
    match xs with [] -> acc | x :: xs -> aux xs ~f ~acc:(f x :: acc)
  let rec aux2 (type a b) (xs : a list) ~(f : a -> b) ~(acc : b list) : b list =
    match xs with [] -> acc | x :: xs -> aux xs ~f ~acc:(f x :: acc)

Note that second case only works due to the typo aux xs ~f ~acc:(f x :: acc) in the body of aux2. Otherwise the locally abstract types are introduced too late and it is not possible to call aux2 on arguments whose types contain either a and b (because those types were defined after aux2).

If you want to use that notation you need to first introduce the locally abstract types and then define the recursive function:

let aux2 (type a b)=
   let rec aux2 (xs : a list) ~(f : a -> b) ~(acc : b list) : b list =
    match xs with
    | [] -> acc
    | x :: xs -> aux2 xs ~f ~acc:(f x :: acc)
   in
   aux2

This is starting to be quite subtle, so most of the time it is simpler to use the type a b. ... shorthand notation which works in all cases.

2 Likes

I’m going through similar frustrations, currently feeling stuck with my OCaml project.

I’m using a complicated 3rd party library (Irmin) which does actually have some docs. I’ve navigated the types and got code that compiles, but I’m not getting the result I was expecting. Surely the bug is on my side, but the relevant decision is happening somewhere inside Irmin.

In Python I would just drop into a debugger and step through to understand where I messed up.

But I have learned that ocamldebug can’t be used at all with a threaded program, ruling out my project because Irmin uses Lwt (which I think is a pretty common lib in the OCaml ecosystem wherever file or network IO is happening?)

AFAICT my options here are to temporarily vendor Irmin into my project and add some print debugging, or just try harder at reading the source code and running it in my head. Both of those can work, but it’s a spare-time “for fun” project and so far I’ve lacked the energy to embark of either of those tedious courses just yet.

Are there any alternative debuggers for OCaml that might help here?

I guess it’s one of those things that’s just harder and more cumbersome in a compiled language.

3 Likes

@anentropic maybe you could shoot your questions on Irmin Discussions (Discussions · mirage/irmin · GitHub)?

cc @CraigFe

I don’t mean to minimize the rest of the troubles, and I would not say no to a shiny threaded debugger, and this is more a curiosity question, but isn’t this relatively easy? For simple packages you can git clone the project into a subdirectory of your project and start hacking on it and dune will resolve it locally instead of from the system.

The more complicated packages, where the problem is in a thing that opam packages you use depend on, you have to opam pin add a copy of it and rebuild the world downstream of it. A bit more time consuming but definitely tractable.

This question is similar to other recent threads about getting started with OCaml (especially on Windows) and documentation practices. I think that all of these are related to the fact that the OCaml community is fairly small and, therefore, there’s just not a ton of current introductory and intermediate material out there. This is one of the reasons that we started OCaml Café which (shameless plug!) meets this coming Wednesday at 7pm Central.

I basically follow the same series of steps that @mudrz outlined because I want to understand what others have done, try to figure it out myself, etc. But the OCaml community is super friendly, so I suggest posting sooner rather than later to Discuss (or Reddit or SE).

7 Likes

I try to put some tests and examples source files when I publish a library…

1 Like

So do I. But what I learned when I worked at a company that actually knew how to do TDD, is that my tests were … wholly inadequate to the task of actually testing the software I wrote. And so, I learned to do a shit-ton more testing. And those tests … .well, they cover (or I hope they do) much more of the behaviour of components and inter-component linkages, than they used to.

Hence, there’s just a ton more code there to harvest as examples.

Also: I’m certainly not saying that I do a good-enough job. Rather, that until I worked at a place that actually did TDD for real (for reals, yo’) I didn’t understand that my level of testing diligence was inadequate. I do hope it’s adequate today, but probably not yet, ah well.

4 Likes

That sounds simple but I don’t know exactly how to do that right now. However I found this article which looks like it explains a bit more The joys of Dune vendoring | Notes from the Windows corner

The Irmin repo declares multiple opam packages so I guess I need to pin add each one, pointing to the local path?

I’m using opam switch and I can see there is a dir under _opam/lib/ for each installed package, with the .ml and .mli source files

Is it possible to just edit those and recompile, rather than actually vendoring the irmin repo into my project root? I only want to temporarily add some print debugging, not fork Irmin and vendor it permanently.

But it seems like the files from the switch libs are not recompiled when I do dune build even with --force

And it seems like I can’t use them as the local target for opam pin add …I guess because they don’t have any .opam package file any more.

Is there a way to do that? It’d be a convenient workflow for this kind of thing.

Wait, why are you “vendoring” lrmin into your project? When I’m working on a project, and need to debug (say) ocamlgraph I just “opam source ocamlgraph” in the directory -above- my project’s root; this unpacks it right there. Then I can build and install ocamlgraph. I guess I could also do opam install --working-dir . in the ocamlgraph directory. And i can of course hack away on the source before building, or hack/build-install/hack/uninstall/build-install/etc. viz.

~/Hack/Camlp5/src$ rm -rf ocamlgraph.2.0.0/
~/Hack/Camlp5/src$ opam source ocamlgraph
Successfully extracted to /home/chet/Hack/Camlp5/src/ocamlgraph.2.0.0
\~/Hack/Camlp5/src$ g ocamlgraph.2.0.0/
~/Hack/Camlp5/src/ocamlgraph.2.0.0 ~/Hack/Camlp5/src
~/Hack/Camlp5/src/ocamlgraph.2.0.0$ opam install --working-dir .
[ocamlgraph_gtk.2.0.0] synchronised (no changes)
ocamlgraph is now pinned to file:///home/chet/Hack/Camlp5/src/ocamlgraph.2.0.0 (version 2.0.0)
The following actions will be performed:
  ∗ install ocamlgraph     2.0.0*
  ∗ install ocamlgraph_gtk 2.0.0*
===== ∗ 2 =====

<><> Processing actions <><><><><><><><><><><><><><><><><><><><><><><><><><><><>
[ERROR] The compilation of ocamlgraph.2.0.0 failed at "dune build -p ocamlgraph
        -j 7 @install".

#=== ERROR while compiling ocamlgraph.2.0.0 ===================================#
# context     2.1.0~beta4 | linux/x86_64 | ocaml-base-compiler.4.13.0~alpha2 | pinned(file:///home/chet/Hack/Camlp5/src/ocamlgraph.2.0.0)
# path        ~/Hack/Opam-2.1.0-beta4/GENERIC/4.13.0~alpha2/.opam-switch/build/ocamlgraph.2.0.0
# command     ~/Hack/Opam-2.1.0-beta4/GENERIC/opam-init/hooks/sandbox.sh build dune build -p ocamlgraph -j 7 @install
# exit-code   1
# env-file    ~/Hack/Opam-2.1.0-beta4/GENERIC/log/ocamlgraph-136357-157930.env
# output-file ~/Hack/Opam-2.1.0-beta4/GENERIC/log/ocamlgraph-136357-157930.out
### output ###
# Error: I don't know about package ocamlgraph (passed through only-packages)



<><> Error report <><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>
┌─ The following actions failed
│ λ build ocamlgraph 2.0.0
└─ 
╶─ No changes have been performed
\~/Hack/Camlp5/src/ocamlgraph.2.0.0$ opam install --working-dir .
[ocamlgraph_gtk.2.0.0] synchronised (no changes)
[ocamlgraph.2.0.0] synchronised (file:///home/chet/Hack/Camlp5/src/ocamlgraph.2.0.0)
The following actions will be performed:
  ∗ install ocamlgraph     2.0.0*
  ∗ install ocamlgraph_gtk 2.0.0*
===== ∗ 2 =====

<><> Processing actions <><><><><><><><><><><><><><><><><><><><><><><><><><><><>
∗ installed ocamlgraph.2.0.0
∗ installed ocamlgraph_gtk.2.0.0
Done.

Notes:

  1. this is the second round-trip of this command-sequence (the first time, it installed a bunch of dependencies.
  2. the install cmd failed the first time, but I just reran it and it worked. No idea why it failed.
  3. you could also look at the contents of the “opam” file and run the commands therein, but typically that’ll install in a different place than when you build with opam. Still should work.

All of this is predicated on the assumption that you just want to hack on the package temporarily, and soon thereafter, you’ll be doing:

opam remove -y ocamlgraph
opam pin remove -y ocamlgraph

Note that the second command there is needed to erase opam’s memory that you were building the package from local sources.

2 Likes

thanks for the tips, I’m trying this now

do I need to re-run opam install --working-dir . each time after making debugging changes to the 3rd party lib?

yep it seems like I need to do that

but this method works well :+1:

also it turns out Irmin has lots of debug logging already that I can reveal just by adding

  Logs.set_reporter (Logs_fmt.reporter ());
  Logs.set_level (Some Logs.Debug);

to the entry point of my program

Or:
opam upgrade .

yes. typically I “remove” and then “install”. But honestly, I’d just copy out the build-commands from the “opam” file, and run them myself. It’ll be miles fsater.

To be honest, I was about to suggest OP should RTFM, but I never found opam documentation very clear about local dev workflow ; and it has changed significantly in the past.
Even today, I regularly stumble upon weird behavior, my favorite being that when you bump the version of the local opam file you have to opam upgrade twice (the first it complains that it cannot upgrade since the older version is pinned, but the second time it does it, which makes me suspicious about what could have changed in opam’s internal state after the error).
I suspect etsy has a more predictable behavior, at the expense of significant disk space wastage, but I’ve never tried it (the fewer prisoner-friendly(*) package managers the better, if you ask me)

(*): that make working in a closed environment easier at the expense of making it harder to interact with other tools and languages

This is actually the reason I moved my project from OCaml to Rust, which is arguably more difficult to comprehend than OCaml, but due to the fantastic documentation is overall easier to use.

It really breaks my heart that I won’t be using OCaml any further in any of my projects, work or otherwise, because I love the language – but I’ve realised that staying productive is more important than having fun writing code. At the end of the day the real value is in what I produce from my project, not what language I produce it in.

2 Likes

People used to complain that OCaml was lacking good libraries, and now that the libraries are missing good documentation. That’s progress!

13 Likes

Could you link an example of awesome Rust documentation?

1 Like

This may not be a popular opinion but I think at some stage one should learn the C FFI. For me that is one of the first things after the basics. The ctypes library really makes the basic interfacing easy to do. Once you have that confidence you really have access to a high quality pool of solutions to well-known problems and you can focus on doing the novel/creative part in OCaml.

As an example if you want a high quality http client, why not go with libcurl? I mean a basic client in any language isn’t hard but do you really want to deal with the nitty gritty of https, redirection, chunking, compression etc. that a thinly supported library may or may not work well with? Indeed I believe there is also a wrapper in OCaml for libcurl. But if you don’t need to access all features of libcurl you could write a thin layer of C to do what you want and just get the end result to OCaml, which could vastly reduce the interfacing surface to just a couple of functions.

The barrier to productivity is rarely the language or libraries. For personal projects, I might start something based on a hunch that I could do something novel/useful. Once I am over the initial humps it usually turns out that there are real fundamental difficulties to overcome. That is when the productivity hits a wall for me.

7 Likes

You can find docs for almost all rust packages at https://docs.rs

Here is documentation for the lwt equivalent in Rust called tokio
https://docs.rs/tokio/1.10.1/tokio/

In general rust documentation facilities are quite good. You can even search stuff.

You can check out rust std library reference here std - Rust — lots of example that can be run in the rust playground