Why the OCaml ecosystem is poorly documented?

[mobileink]

@Chet_Murthy

Gotta say, this is just incredibly useful. I don’t need to keep the usage details of a gazillion C++ functions/classes/templates in my head: just vague ideas of the names, and I can lookup the rest.

Not sure what you mean. One does need a specific string to look up a manpage.

[Chet_Murthy]

What I mean is: I can search for (let’s say – I’m making this up right now) “c++ sort vector” and I get: https://cplusplus.com/reference/algorithm/sort/
This has both reference-style stuff, and cut-and-paste examples that I can start with for making something useful in my program.

Another example: “c++ iostream hex” gives std::dec, std::hex, std::oct - cppreference.com

I made these up right as I’m typing this reply, so no preparation. I’ve used examples like this as I go along writing my C++ program. Eventually I remember the idioms for the particular stuff I use a lot in my project and no longer need to search; but one cannot hold all the bits in one’s head for such large libraries (and multiple languages), so this sort of reference documentation is just great stuff.

Full disclosure: I am the last person who should complain about documentation, b/c I grew up with “use the source, Luke” and have never been good at writing docs.

[nojb]

@benjamin-thomas

It didn’t come to mind to bother anybody with this but since you seem to be open to such improvements, what would be the process to contribute those little snippets/examples?

Documentation improvements to the standard library should be proposed upstream at GitHub - ocaml/ocaml: The core OCaml system: compilers, runtime system, base libraries, typically in the form of a pull request.

For information, note that current efforts to improve the documentation of the standard library are focusing on adding an “Examples” section to each module, instead of adding examples to the documentation of each function…

Cheers,
Nicolas

[mimoo]

I’d also argue that when generated HTML docs look pretty (and these haskell docs look pretty to me!) then you’re more encouraged to produce them. We humans like pretty things. Golang also has an in-terminal documentation (in addition to their pretty html docs) that I find really dope personally:

[mimoo]

this is why things like GitHub - tldr-pages/tldr: 📚 Collaborative cheatsheets for console commands and http://bropages.org/ and GitHub - cheat/cheat: cheat allows you to create and view interactive cheatsheets on the command-line. It was designed to help remind *nix system administrators of options for commands that they use frequently, but not frequently enough to remember. and a dozens other exist, because people are bad at writing man pages

[edwin]

@mimoo

Golang also has an in-terminal documentation

OCaml has it too, try man Map or man Map.S. But TBH I never used it for library code (I do use it for tools), I usually look up documentation through the LSP server integrated in my editor (for single functions), or a browser (for whole modules).

[dbuenzli]

Note that I once tried to use text mode browsers (notably emacs’s eww) on odig’s generated docsets and I found it very pleasant to use. At least, in contrast to the man backend, it doesn’t destroy all the hard work documenters put into making relevant links in their docsets (man pages are an extremely poor documentation medium in our age).

I’ll might consider adding an odig man [ID] command that lookup the page for ID and invokes it on your text mode browser, the tech is there on the odoc side, it’s just a matter of integrating it, I’ve been lagging on this.

[edwin]

@cemerick

Leaving aside abandonware or obviously hobby projects, where is documentation lacking or of low quality? I can’t personally think of a clear example off the top of my head; and without getting concrete, generalized complaints are probably not very productive.

I don’t want to pick on a particular project, but one that I recently looked at was ocplib-simplex, which although had some documentation comments, it lacked comments on the most important place: how to get started to solve the inequality system and how to retrieve the results. I had to look at the unit test code to figure out how to invoke it (it doesn’t help that the type used is imperative style and the ‘solve’ function just gives you back the original value, but altered, and now it suddenly contains a result).

Compare this to lp and lp-glpk where it was immediately obvious how to solve the system (and the solved system has a different type from the original) and how to build up the inequality system (it is also not very far from how this would be done in other languages).

You’d think that it is clear then: use ‘lp’ instead of ‘ocplib-simplex’. However ocplib-simplex is still useful, because lp can only solve systems with some limited floating point precision, whereas ocplib-simplex seems to be able to work with arbitrary types (e.g. arbitrary precision rationals), and is also entirely self-contained and doesn’t depend on external solvers.

Instead of just complaining about the lack of documentation, here is one PR to try to improve it in a small way based on just what I just discovered: doc: add minimal documentation for solving system and retrieving results by edwintorok · Pull Request #16 · OCamlPro/ocplib-simplex · GitHub.
However it already shows one limitation of our documentation system when it comes to heavily functorized code: I was only able to cross reference the module (Result and SolveBounds) and not the function I wanted (at least dune build @doc was not able to solve it): Result.S.get or SolveBounds.S.get or Result.Make(Core).get.

This is one project that might benefit both from equational style documentation (it is about a precise mathematical system), and some examples, or more importantly cross references.
E.g. I solved the system, but now what, how do I retrieve the result, and do I need to pass the original objective or the one returned by maximize, does it matter (the type system would allow either…), etc.

[benjamin-thomas]

@nojb

For information, note that current efforts to improve the documentation of the standard library are focusing on adding an “Examples” section to each module, instead of adding examples to the documentation of each function…

Is this open for discussion?

I’m not so interested in examples at the top of a module because, from a practical perspective, I want to stay in my editor as much as possible.

Doing otherwise kinda breaks the programmer’s flow IMO.

[nojb]

@benjamin-thomas

Is this open for discussion?

As a general rule everything is open for discussion; but there is no guarantee on the outcome (to be clear, I don’t have a horse in this race.)

If you are interested in arguing this point, perhaps you (and anyone else who is interested in this subject) could start a Discussion over at ocaml/ocaml · Discussions · GitHub.

Cheers,
Nicolas

[dbuenzli]

@benjamin-thomas

I’m not so interested in examples at the top of a module because, from a practical perspective, I want to stay in my editor as much as possible.

I think the fix here rather belongs to the way your editor looks up documentation.

If we put the constraint that documentation is only consumed from the docstring of individual structure items then 1) in the current state of affairs you are going to miss a lot of things and broad explanations 2) There’s info you don’t want to repeat in each doc strings for many different reasons (dry, maintenance, usability of the mli when perused as pure text, explanations spanning more than one structure item, etc.)

[mars0i]

@cemerick

Just as a bit of a point of order: your Clojure link is not to the actual API reference, but to clojuredocs.org, a community-maintained repository of additional documentation layered on top of the official documentation. (I mention this only because I had some hand in helping clojuredocs get started many years ago, where it was a community reaction to the core dev team explicitly not wanting to have the sort of expanded sets of examples and cross-referencing that you and others are looking for in OCaml.)

Some details that might relevant to some of the subsequent discussion. (I made related points in a earlier discussion several years ago.)

• When you enter (doc <function-name>) in a Clojure REPL, you get a (usually) short docstring that lists the arguments with a brief description of what the function does. There are (typically) no examples. If you’ve configured your editor, you can get the same thing in the editor.
• With some editor configurations (e.g. vim-iced for vim/neovim), there is a separate command to pop up the ClojureDocs page for the function the cursor is on. This will include the official docstring, plus user-contributed examples and user-contributed cross-references to related functions.
• I personally don’t like overly long docstrings. When you just want to be reminded of the syntax and what the function does, examples are annoying.
• But I love ClojureDocs. Because of the examples, it’s indispensible. I can view it on the web, nicely formatted, or if I want to be able to pop up ClojureDocs pages in my editor, I can configure my editor to do that as an operation distinct from displaying the docstring alone. This is the best of both worlds.

That seems like one possible model for separate brief documentation and examples: example-free basic function documentation, with a separate, easily viewable example repository that can be viewed on the web or pulled up in an editor.

@cemerick

I do see the value in having such examples; having them available for OCaml libraries would I’m sure be helpful. The big questions are, who’s going to write and maintain those examples, and what does the infrastructure look like for maintaining them (i.e. having them in a testable form so their validity can be checked and expected output can be produced automatically).

When I mentioned ClojureDocs as a useful model in 2017, there was concern that user-contributed examples would require work to curate them. I asked around about how ClojureDocs examples are curated, because I couldn’t find any information about who was in charge. The answer is that ClojureDocs examples are not curated—at least not centrally or systematically. Anyone who creates an account on ClojureDocs can add an example and can edit others’ examples.

Yet I have never seen a horrible example on ClojureDocs. Sometimes there are examples that pretty much illustrate the same point as other examples, but that’s not necessarily bad. I have to assume there have been bad examples added to ClojureDocs at some point. I imagine that someone came along and noticed and improved or removed those examples. I don’t know. ClojureDocs works, and it’s great.

I think that if Clojure were as popular as Java or Python, ClojureDocs would be a mess, because more users means there’s a greater chance that someone will exercise poor judgement.

But Clojure is not that popular, and it’s largely a language that’s chosen rather than just a language that people use because they know it’s popular. There are fewer people to create random noise, and maybe people who use Clojure tend to be people who respect the language and the community. But that sounds like OCaml.

So maybe all that’s needed is a system that generates editable webpages headed by the official documentation, with a system for allowing users to add examples. This could start with the standard library, but maybe maintainers of other libraries would want to follow the same model. Then just wait to see what happens. If a lot of junk starts appearing, one can worry about how to deal with that then. I think that’s unlikely.

If “OcamlDocs” takes off, people might start adding functions to editor-IDE configurations that allow displaying the examples page for each function. Or maybe people would be happy viewing the examples only on the web. I sometimes work with a browser pointed at ClojureDocs so that it’s always available.

(Maybe one difference is that OCaml is a teaching language for CS at some universities. I’m not aware of Clojure being used in this way. If there’s a fear that students would add noise, maybe more policing would be needed?)

[K_N]

I did some quick and dirty experiments in VSCode with Python and JavaScript. In both cases, the docstring displayed seems to consists only of:

• The method signature (or signatures in case of overloading):
• The short specification

I don’t see any examples popping up e.g. for Python even when the standard library doc has examples (e.g. str.expandtabs(…)) and the docstring displayed is significantly shorter what is in the manual. This indicates that there are two sources of documentations (at least for Python, I don’t know where the doc comes from for JS). The docstring that is in the code is what is displayed by the VSCode extension. The standard library prose is longer, gives more details about edge cases and sometimes (not always) some examples for a given method. It also features examples in dedicated sections.

From what I understand, in OCaml, ocamldoc comments are the unique source of documentation and we are trying to use them in two very different ways (inline in the editor and as a reference text in a standalone manual).

Since having two distinct versions of docs to keep synchronized seems like a nightmare, would it be possible to have markers in ocamldoc syntax to delineate a short documentation (displayed in the editor) and some extra material to be shown in the full doc ?

As an aside, maybe the examples are needed in Clojure because there does not seem to be an standard way to describe the types of arguments?

[cemerick]

@K_N

Since having two distinct versions of docs to keep synchronized seems like a nightmare, would it be possible to have markers in ocamldoc syntax to delineate a short documentation (displayed in the editor) and some extra material to be shown in the full doc ?

Whatever happens in this vein going forward, I can’t think of a worse way to proceed than to have the sort of varied sets of examples flow from the mli-sourced ocamldoc:

• Libraries of examples can be quite large, and it would be an unreasonable burden on library maintainers to always have to swim through N examples while doing completely unrelated tasks.
• Wherever examples are maintained, they need to have some degree of automated testing applied, lest they rot in the face of drift in the APIs they exercise. If they’re defined in a library’s ocamldoc strings, then every project needs to have infrastructure and process in place to extract those examples and build and test them.
• Often, the best examples need to use multiple libraries; in these cases, it’s impossible to have them situated in a single function or module’s ocamldoc.

These are only some of the factors that drove clojuredocs to be built separately from the libraries and APIs it documents.

Stitching canonical reference documentation together with examples from a separate repository is not particularly hard, either for display in a tool like vs code, or as part of an uber-docs site that might splice relevant examples onto the end of official function or module docs. This is actually where a lot of magic can happen, as community efforts at examples development can surface helpful usage from across the ecosystem (e.g. Seq or Iter examples don’t all need be bland snippets using toy data; a well-tagged examples repository could surface an example Dream handler that uses Iter to lazily stream data over a websocket).

@K_N

As an aside, maybe the examples are needed in Clojure because there does not seem to be an standard way to describe the types of arguments?

No; the biggest driving impetus for clojuredocs was the fact that the stdlib documentation was considered less than helpful by many, and the core team did not want to expand it in the ways that clojuredocs contributors wanted (which went beyond examples, but that’s not germane here).

(As an aside, I sort of hate that Clojure/Clojuredocs has become a foil here, but alas…)

Everyone needs examples; what’s up in the air is where they go, who maintains them, etc etc. Even haskellers, no slouches when it comes to using types or applying equational reasoning, include basic examples in their docs.

[mimoo]

Sometimes you have to get to the root cause instead of trying to fix the symptoms

[sanette]

Sorry to interrupt with something less serious (or is it?)
Here are the “fortunes” about documentation:

Arnold’s Laws of Documentation:
(1) If it should exist, it doesn’t.
(2) If it does exist, it’s out of date.
(3) Only documentation for useless programs transcends the first two laws.

Documentation is like sex: when it is good, it is very, very good; and
when it is bad, it is better than nothing.
― Dick Brandon

Documentation is the castor oil of programming. Managers know it must
be good because the programmers hate it so much.

Manual, n.: A unit of documentation. There are always three or more on a
given item. One is on the shelf; someone has the others. The information
you need is in the others.
― Ray Simard

[benjamin-thomas]

@nojb

As a general rule everything is open for discussion; but there is no guarantee on the outcome (to be clear, I don’t have a horse in this race.)

If you are interested in arguing this point, perhaps you (and anyone else who is interested in this subject) could start a Discussion over at ocaml ocaml · Discussions · GitHub · GitHub .

Of course, I don’t expect any guarantee

I’d mostly like to avoid wasting anyone’s time if the subject has been mostly settled.

@dbuenzli

I think the fix here rather belongs to the way your editor looks up documentation.

This has nothing to do with my editor setup.

I’d like to see the standard library functions be documented in a similar fashion to dream, with copious explanations and example code.

Not too much, not too little.

This, to me, is documentation very nicely done.

[Johnny_Mnemonic]

Ok so I have a little case study of where I ran into a hole in Ocaml documentation.

The other day I was doing something or other with Core’s Out_channel module. There was a function that I had a decent idea of what it did (to be fair, “types are documentation” does a lot of genuine lifting), but I wanted to look it up on Ocaml’s documentation page to see if there was a little blurb or anything to confirm my expectations.

I’m using Core v0.17.1, so I go to Core’s doc page and select my version from the dropdown box: core v0.17.1 · OCaml Package

And yet… Out_channel is not there in the sidebar.

Goddamnit, I know the module exists. I can see it in utop. I even type it in the “Search names in this package…” and a module does pop up. (Of course it does, I know the module exists.)

But that link is broken: https://ocaml.org/p/core/v0.17.1/doc/Core/Out_channel/index.html

That really shouldn’t happen. Even just an undocumented html formatted mli would suffice. The undocumented html formatted mli page would tell me, the reader, “devs didn’t get around to documenting this”, which I’m OK with.

I think I pretty regularly run into situations where there’s packages I have installed but documentation on ocaml dot org is completely missing. Sometimes just certain versions are missing.

Maybe this is a bug in ocaml dot org I should report somewhere?

(Also I see this topic has been sorta quiet for a bit, if I violated forum etiquette by bumping it, feel free to let me know.)

[yawaramin]

It’s here: core v0.17.1 · OCaml Package

It doesn’t have a separate page within the Core docs because it’s actually a module alias for the Stdio.Out_channel module. So it has its own page there, which you can click through to from above.

[Johnny_Mnemonic]

Ok, that’s helpful, thanks. I’ll check the module pages directly in the future.

Though it should really still appear on the sidebar and that autocompleted link in the “Search names in this package” box shouldn’t 404. The former’s bad design and the latter’s a bug.