I think the reality is that most people work off of code snippets they find, and there simply aren’t many of those available for OCaml libraries.
This is accurate. Others have mentioned the difficulty in getting OCaml library authors to write adequate documentation and examples. I have to confess, I’m as guilty as anybody else.
One thing I wonder about: perhaps if people wrote more tests, did more test-driven development, that might end up producing the plethora of examples/snippets you mention, and could be used by beginners.
thanks all, before reading source code I start with reading .mli
files as well, which works a lot of the time, but still find myself reading the source code to get started, when I want to find out more about specific arguments and etc.
I think the reality is that most people work off of code snippets they find, and there simply aren’t many of those available for OCaml libraries.
I can’t speak for others, but this is definitely how I work in other languages like C#, F#, JavaScript, TypeScript and Rust: when starting something new / new library: take with a basic example, change it to have something working for my use case, then if there is a need - dive deeper;
I don’t find it works quite as well in OCaml, but I also don’t find many threads with people encountering issues / errors, so I felt like maybe
- people have a different workflow
- or work in code bases written in OCaml that most of the time already have plenty of examples to look out for
- or maybe there’s a French forum (I don’t speak French)
(which is why I started this thread to learn what others do)
Which type do I need? Which types do I have?
Generally I start with the types, but I still find the errors often times rather confusing
For example in this simplified example:
module My_module : sig
val do_things : 'a list -> f:('a -> 'b) -> 'b list
end = struct
let rec aux (xs : 'a list) ~(f : 'a -> 'b) ~(acc : 'b list) =
match xs with [] -> acc | x :: xs -> aux xs ~f ~acc:(f x :: xs)
let do_things xs ~f = aux xs ~f ~acc:[]
end
I believe that I have been very explicit about what types I want, but the lsp puts lines in the whole module (making it difficult to find where the error really is)
The error in this case is also kind of in the wrong place - despite specifying the wanted type of aux
it is inferred as:
'b list -> f:('b -> 'b) -> acc:'b list -> 'b list
These make it difficult to track the simple bug:
- match xs with [] -> acc | x :: xs -> aux xs ~f ~acc:(f x :: xs)
+ match xs with [] -> acc | x :: xs -> aux xs ~f ~acc:(f x :: acc)
When interfacing with a more complex library such type errors are kind of difficult to troubleshoot and I find myself having to read the source code, create simple examples and etc.
Using unification variables is classical mistake when debugging or specifying types. As you have seen,
let id (x:'a) (y:'a) : 'a = x + 1
is well-typed, because the variable 'a
is an unification type variable that can be unified with int
.
Locally abstract types behave better for this purpose:
let rec aux: type a b. a list -> f:(a -> b) -> acc:b list -> b list =
fun xs ~f ~acc -> match xs with
| [] -> acc
| x :: xs -> aux xs ~f ~acc:(f x :: xs)
fails with the expected error
| | x :: xs -> aux xs ~f ~acc:(f x :: xs) ^^ Error: This expression has type a list but an expression was expected of type b list Type a is not compatible with type b
I wasn’t even aware that this can be an issue - when debugging I’ve always added the externally visible types 'a, 'b
and etc.
for completeness adding:
and OCaml - Language extensions
which supports both function syntaxes
let rec aux : type a b. a list -> f:(a -> b) -> acc:b list -> b list =
fun xs ~f ~acc ->
match xs with [] -> acc | x :: xs -> aux xs ~f ~acc:(f x :: acc)
let rec aux2 (type a b) (xs : a list) ~(f : a -> b) ~(acc : b list) : b list =
match xs with [] -> acc | x :: xs -> aux xs ~f ~acc:(f x :: acc)
Note that second case only works due to the typo aux xs ~f ~acc:(f x :: acc)
in the body of aux2
. Otherwise the locally abstract types are introduced too late and it is not possible to call aux2
on arguments whose types contain either a
and b
(because those types were defined after aux2
).
If you want to use that notation you need to first introduce the locally abstract types and then define the recursive function:
let aux2 (type a b)=
let rec aux2 (xs : a list) ~(f : a -> b) ~(acc : b list) : b list =
match xs with
| [] -> acc
| x :: xs -> aux2 xs ~f ~acc:(f x :: acc)
in
aux2
This is starting to be quite subtle, so most of the time it is simpler to use the type a b. ...
shorthand notation which works in all cases.
I’m going through similar frustrations, currently feeling stuck with my OCaml project.
I’m using a complicated 3rd party library (Irmin) which does actually have some docs. I’ve navigated the types and got code that compiles, but I’m not getting the result I was expecting. Surely the bug is on my side, but the relevant decision is happening somewhere inside Irmin.
In Python I would just drop into a debugger and step through to understand where I messed up.
But I have learned that ocamldebug
can’t be used at all with a threaded program, ruling out my project because Irmin uses Lwt (which I think is a pretty common lib in the OCaml ecosystem wherever file or network IO is happening?)
AFAICT my options here are to temporarily vendor Irmin into my project and add some print debugging, or just try harder at reading the source code and running it in my head. Both of those can work, but it’s a spare-time “for fun” project and so far I’ve lacked the energy to embark of either of those tedious courses just yet.
Are there any alternative debuggers for OCaml that might help here?
I guess it’s one of those things that’s just harder and more cumbersome in a compiled language.
@anentropic maybe you could shoot your questions on Irmin Discussions (Discussions · mirage/irmin · GitHub)?
cc @CraigFe
I don’t mean to minimize the rest of the troubles, and I would not say no to a shiny threaded debugger, and this is more a curiosity question, but isn’t this relatively easy? For simple packages you can git clone
the project into a subdirectory of your project and start hacking on it and dune will resolve it locally instead of from the system.
The more complicated packages, where the problem is in a thing that opam packages you use depend on, you have to opam pin add
a copy of it and rebuild the world downstream of it. A bit more time consuming but definitely tractable.
This question is similar to other recent threads about getting started with OCaml (especially on Windows) and documentation practices. I think that all of these are related to the fact that the OCaml community is fairly small and, therefore, there’s just not a ton of current introductory and intermediate material out there. This is one of the reasons that we started OCaml Café which (shameless plug!) meets this coming Wednesday at 7pm Central.
I basically follow the same series of steps that @mudrz outlined because I want to understand what others have done, try to figure it out myself, etc. But the OCaml community is super friendly, so I suggest posting sooner rather than later to Discuss (or Reddit or SE).
I try to put some tests and examples source files when I publish a library…
So do I. But what I learned when I worked at a company that actually knew how to do TDD, is that my tests were … wholly inadequate to the task of actually testing the software I wrote. And so, I learned to do a shit-ton more testing. And those tests … .well, they cover (or I hope they do) much more of the behaviour of components and inter-component linkages, than they used to.
Hence, there’s just a ton more code there to harvest as examples.
Also: I’m certainly not saying that I do a good-enough job. Rather, that until I worked at a place that actually did TDD for real (for reals, yo’) I didn’t understand that my level of testing diligence was inadequate. I do hope it’s adequate today, but probably not yet, ah well.
That sounds simple but I don’t know exactly how to do that right now. However I found this article which looks like it explains a bit more The joys of Dune vendoring | Notes from the Windows corner
The Irmin repo declares multiple opam packages so I guess I need to pin add
each one, pointing to the local path?
I’m using opam switch
and I can see there is a dir under _opam/lib/
for each installed package, with the .ml
and .mli
source files
Is it possible to just edit those and recompile, rather than actually vendoring the irmin repo into my project root? I only want to temporarily add some print debugging, not fork Irmin and vendor it permanently.
But it seems like the files from the switch libs are not recompiled when I do dune build
even with --force
And it seems like I can’t use them as the local target for opam pin add
…I guess because they don’t have any .opam
package file any more.
Is there a way to do that? It’d be a convenient workflow for this kind of thing.
Wait, why are you “vendoring” lrmin into your project? When I’m working on a project, and need to debug (say) ocamlgraph
I just “opam source ocamlgraph” in the directory -above- my project’s root; this unpacks it right there. Then I can build and install ocamlgraph. I guess I could also do opam install --working-dir .
in the ocamlgraph directory. And i can of course hack away on the source before building, or hack/build-install/hack/uninstall/build-install/etc. viz.
~/Hack/Camlp5/src$ rm -rf ocamlgraph.2.0.0/
~/Hack/Camlp5/src$ opam source ocamlgraph
Successfully extracted to /home/chet/Hack/Camlp5/src/ocamlgraph.2.0.0
\~/Hack/Camlp5/src$ g ocamlgraph.2.0.0/
~/Hack/Camlp5/src/ocamlgraph.2.0.0 ~/Hack/Camlp5/src
~/Hack/Camlp5/src/ocamlgraph.2.0.0$ opam install --working-dir .
[ocamlgraph_gtk.2.0.0] synchronised (no changes)
ocamlgraph is now pinned to file:///home/chet/Hack/Camlp5/src/ocamlgraph.2.0.0 (version 2.0.0)
The following actions will be performed:
∗ install ocamlgraph 2.0.0*
∗ install ocamlgraph_gtk 2.0.0*
===== ∗ 2 =====
<><> Processing actions <><><><><><><><><><><><><><><><><><><><><><><><><><><><>
[ERROR] The compilation of ocamlgraph.2.0.0 failed at "dune build -p ocamlgraph
-j 7 @install".
#=== ERROR while compiling ocamlgraph.2.0.0 ===================================#
# context 2.1.0~beta4 | linux/x86_64 | ocaml-base-compiler.4.13.0~alpha2 | pinned(file:///home/chet/Hack/Camlp5/src/ocamlgraph.2.0.0)
# path ~/Hack/Opam-2.1.0-beta4/GENERIC/4.13.0~alpha2/.opam-switch/build/ocamlgraph.2.0.0
# command ~/Hack/Opam-2.1.0-beta4/GENERIC/opam-init/hooks/sandbox.sh build dune build -p ocamlgraph -j 7 @install
# exit-code 1
# env-file ~/Hack/Opam-2.1.0-beta4/GENERIC/log/ocamlgraph-136357-157930.env
# output-file ~/Hack/Opam-2.1.0-beta4/GENERIC/log/ocamlgraph-136357-157930.out
### output ###
# Error: I don't know about package ocamlgraph (passed through only-packages)
<><> Error report <><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>
┌─ The following actions failed
│ λ build ocamlgraph 2.0.0
└─
╶─ No changes have been performed
\~/Hack/Camlp5/src/ocamlgraph.2.0.0$ opam install --working-dir .
[ocamlgraph_gtk.2.0.0] synchronised (no changes)
[ocamlgraph.2.0.0] synchronised (file:///home/chet/Hack/Camlp5/src/ocamlgraph.2.0.0)
The following actions will be performed:
∗ install ocamlgraph 2.0.0*
∗ install ocamlgraph_gtk 2.0.0*
===== ∗ 2 =====
<><> Processing actions <><><><><><><><><><><><><><><><><><><><><><><><><><><><>
∗ installed ocamlgraph.2.0.0
∗ installed ocamlgraph_gtk.2.0.0
Done.
Notes:
- this is the second round-trip of this command-sequence (the first time, it installed a bunch of dependencies.
- the install cmd failed the first time, but I just reran it and it worked. No idea why it failed.
- you could also look at the contents of the “opam” file and run the commands therein, but typically that’ll install in a different place than when you build with opam. Still should work.
All of this is predicated on the assumption that you just want to hack on the package temporarily, and soon thereafter, you’ll be doing:
opam remove -y ocamlgraph
opam pin remove -y ocamlgraph
Note that the second command there is needed to erase opam’s memory that you were building the package from local sources.
thanks for the tips, I’m trying this now
do I need to re-run opam install --working-dir .
each time after making debugging changes to the 3rd party lib?
yep it seems like I need to do that
but this method works well
also it turns out Irmin has lots of debug logging already that I can reveal just by adding
Logs.set_reporter (Logs_fmt.reporter ());
Logs.set_level (Some Logs.Debug);
to the entry point of my program
Or:
opam upgrade .
yes. typically I “remove” and then “install”. But honestly, I’d just copy out the build-commands from the “opam” file, and run them myself. It’ll be miles fsater.
To be honest, I was about to suggest OP should RTFM, but I never found opam documentation very clear about local dev workflow ; and it has changed significantly in the past.
Even today, I regularly stumble upon weird behavior, my favorite being that when you bump the version of the local opam file you have to opam upgrade
twice (the first it complains that it cannot upgrade since the older version is pinned, but the second time it does it, which makes me suspicious about what could have changed in opam’s internal state after the error).
I suspect etsy has a more predictable behavior, at the expense of significant disk space wastage, but I’ve never tried it (the fewer prisoner-friendly(*) package managers the better, if you ask me)
(*): that make working in a closed environment easier at the expense of making it harder to interact with other tools and languages