I recently learned of Diátaxis, described as “A systematic framework for technical documentation authoring”. It is a website with someone’s opinions about how to write good documentation (actually how to structure it), and I find it interesting. Basically it encourages authors to structure documentations in four categories (tutorials, how-to guides, references, explanations), and argues that they should be kept distinct. (See a slightly longer quote below.)
I wonder what people think, and if they have other pointers to recommend on how to write documentation. (Curiously there is much less content online about documentation writing than about code writing.)
Diátaxis divides documentation across two axes of knowledge: theory/practice , and acquisition/application .
Documentation therefore either contains theoretical knowledge or describes practical actions , and is concerned either with serving our acquisition or our application of knowledge. Hence the map, across which the four forms of documentation are laid out:
Note: I re-read parts of the 2018 OCaml Documentation Open Thread, which has a different general focus but also contains many useful comments!
I think it’s an interesting structure and I can certainly see variations of it in some good documentation corpuses I perused in the past (e.g. Apple’s ones at the beginning of the century, when they used to be good).
That being said I always worry a bit about such clear-cut distinctions which tend to lead to box ticking and to think in terms of siloed activities which may not match the convoluted ways of the brains. The (hyper)links between the identified distinctions may be more important than the distinctions themselves.
I don’t have pointers, but regarding your parenthesis I’d like to make this remark: I think people need to stop separating the documentation and code writing activities. They should be seen as concomitent, deeply interwinded and basically a by-product of design: both actually test it. Once for humans, once for computers.
Certainly a lost cause because most programmer generating curricula seem to almost entirely eschew design and especially interaction design (aka ux design, hci) which is now absurdingly a separate trade.
This leads to a trade almost entirely focused on techniques and running programs regardless on how they are built (the code is the specification) and the ethics of their execution. A trade devoid of design culture and knowledge of the human factors it is supposed to serve. But amusingly by far the best book about API and UI design (which are the same), I ever read is an old ergonomics book – you’ll even find justifications for functional programming in there :–)
So this little detour to say that while advice on how to write documentation are important – especially since we can certainly highlight patterns to speed it up – to me more important is to ask ourselves on how we can naturally generate it as a by-product of our work. For that my suggestion would be to try to foster a design and human factors culture in the trade, but then I like lost causes…
I don’t disagree, but I think that we also need precise, actionable advice for documentation writing.
The way I think of it: most of us programmers are bad at it (mostly by lack of training and practice), and we know we are bad, but we don’t really know where to start. General principles are interesting but they don’t have an immediate payoff, so it can be hard to prioritize them over the many other demands on our working hours. On the other hand if we had something like a “checklist of 6 simple steps to follow to write not-excellent-but-not-crappy documentation for your project”, I think it would not be too hard to convince people to give it a try. And once we’ve done this a couple times for different projects, it’s easier to tell us about the general principles, to start a more open-ended conversation.
(I know for example that I wasn’t terribly interested in UI before I got to learn some simple, actionable things like “if your buttons are too small it’s hard to click on them”. I needed to hear about the simple, “obvious” things that I hadn’t really thought about clearly before I could feel interested in learning more about the topic.)
Now this “Diátaxis” website does not really contain such a “simple checklist of step to follow” that I was talking about. It contains a small action-oriented section Just do something, but there is not much there – I wish there was more, in particular about how to start to document a new project. I still find the distinctions it introduces interesting, and I hope that I could use it in the future to discuss documentation choices.
Coming back @dbuenzli to your point that documentation and code should be written together. I find the idea interesting, and I have certainly noticed in the past that writing .mli documentation helps reveal issues with the existing API, etc. But then I have also felt held back by my lack of knowledge of how to deal with documentation. For example a big part of code (and documentation) development is code review and giving feedback, but giving feedback on documentation is often hard, I don’t have a toolbox of opinions, advices, and positive and negative patterns to recognize and mention to the authors. So I guess there is a bit of chicken-and-egg problem here.
My opinion, not that anyone necessarily wants it: the most important characteristics of documentation are that it must exist, must be accessible, must be complete, and must be understandable to the people reading it.
One can improve upon these basic qualities in simple ways (making the documentation well structured so that people can find what they are looking for quickly, making the documentation easy to understand) or in more complicated ways (systematizing the documentation so that it is consistently presented etc.)
However: existing, being accessible, being complete, and being understandable are the most important basic features documentation requires.
For the many of the most important parts of the OCaml ecosystem, these most basic constraints are violated. I think the first focus should be to fix this. The good news is that the last few years, some of this has improved. Much more, however, remains to be done.
Personally I don’t have evidence that programmers are bad at it. I would be more enclined to say that they simply do not try.
That is, being happy that the computer understood their ideas as witnessed by program execution, they do not really care to explain the program to other humans. Which only shows they do not take the Curry-Howard correspondence seriously :–) That’s the cultural aspect I was mentioning above.
Surely your idea of check lists could help and I do have a few written and unwritten rules and patterns for “reference” documentation (i.e. doc strings) that I mostly learned from (certain parts) of the stdlib itself. Simple stuff like do not use future tense, describe values not function calls etc. etc. But honestly I’m not sure that this is really what is going to help (nor is waiting on better documentation tools as some people sometime think, these have always been around and usable if you wanted to, even if not perfect).
It’s the mindset that is missing. For me the real important starting point is that you should be willing to try to explain with written words what your program does to another human, however badly. Explain your design and its mecanics starting from its smallest constituents.
The best way of quick starting that is to try it on yourself while designing and coding your APIs. You’ll often find out that if you fail to be able to describe and caracterize your design/module/type/function/value with written words in a simple and concise manner, you have a bad design.
This may not lead to docs in the full spectrum of diátaxis but should cater for a good chunk of reference, explanation and how-to.
I don’t have pointers, but regarding your parenthesis I’d like to make this remark: I think people need to stop separating the documentation and code writing activities. They should be seen as concomitent, deeply interwinded and basically a by-product of design: both actually test it. Once for humans, once for computers.
A big problem for me, when (trying to) document OCaml, is that I can
rely on the compiler to tell me when changes in place A entail changes
in place B (type errors, non exhaustive matches, etc.), and I can code
in a way that will leverage that better; but when I write docstrings
they can become stale very quickly and the only way to fight that would
be to regularly proof-read everything :-).
A possible solution is to do like yourself, @dbuenzli, and write single-module
libraries. I don’t know if it’s always applicable. I wish I had a
solution for documentation but maybe it’s just being less lazy.
Maybe implicit in what I wrote is that the more you take the habit and discipline of writing doc strings, the less you’ll need to change them. That is your design iterations reach the fixed point faster.
And like throwing away code, one should not be shy to throw away docs.
For me writing doc strings forces me to spell out and clarify what the values of my program represent, how I operate on them, how they interact together and reveal their hidden assumptions in a way that simply using them all over the code base cannot reveal with the same clarity.
There’s a lot of handwringing among developers for not writing more/better documentation. I think it’s important in these discussions to not allow the perfect to become the enemy of the good. As a rule more documentation is better than less. As long as the documentation isn’t completely misleading, it’s generally helpful or at least does no harm. But writing is hard and the payoff is often small or non-existent. There are two elements that I think could be made more explicit in their typology:
Motivation: Why are you writing documentation and what do you hope to accomplish?
Audience: Who are you writing this for?
I think it’s important that there are clear, pragmatic answers to these questions and practical benefits to putting forth one’s efforts. Otherwise it has little value (or, at least, can feel like it).
In my own work, I mostly write documentation for myself, while I’m thinking through a problem and/or so that I can figure out what the heck I was thinking when I return to the code later. (I document much less frequently with OCaml b/c my code is so much clearer now.)
And most of the documentation I produce for others are basically FAQs about how to use my programs. When I have to answer the same questions over and over again, that suggests that I need to write it up (and/or reconsider my design).
I was just pondering Diátaxis as @gemmag and @jonludlam also independently sent on the same link to me in the context of writing more longform tutorials for MirageOS. While I’m pretty happy with the state of writing individual docstrings for libraries, I do think there’s a missing gap in the story for how to document systems written in OCaml.
If you take a typical OCaml system, it consists of:
documenting types and values in a module. We have a pretty good story here with docstrings, and @dbuenzli’s library structure is a great example of how to weave multiple independent small bits of code together. Odoc’s cross-referencing really helps pull this together, as does using index.mld files.
documenting module signatures and functors. We have an “ok” story here, as you can use cross-referencing and includes to pull together documentation for a signature, but you can’t easily tie them together with the libraries that intended to satisfy that functor. I realise that the “set of interfaces that could satisfy a module signature” is very open-ended, but the author of a module signature often has a sense of some set of implementations that might satisfy it. For example, if you look at the Mirage_device or Cohttp_lwt signature, there’s no way to tie this together with the set of implementations that we also publish which satisfy those. It can be done manually, but is very hard to keep in sync – some automation would really help here.
documenting how to apply modules in combination. For larger codebases like Irmin, there are a truly impressive number of signatures exposed, that can be composed in many combinations (example). Here we really want to show how programming in the module language (to assemble implementations that satisfy interfaces) can be done to solve a particular programming problem (like building a fake test harness out of a real server, as in the xen-api). The OCaml module language is very rich, and we have no clear way of expressing how (for example) signature inclusion or destructive type substitution is intended to be used to satisfy some interface from a series of implementations. There is some nascent tooling to show dependency graphs (dot dumps from ocamldoc), but in the distant past we’ve had to write custom ocamldoc plugins (like the xen docs) to pull some of this information together.
I think I’m just missing an “odoc for the module system” to complement the existing support for types and values. If we had that, I could see how to satisfy the Diátaxis quadrants much more naturally for larger systems. Or perhaps someone already has a solution for documenting complex modular interactions that I’m simply unaware of!
Somewhat related, Emacs has a checkdoc functionality for writing elisp code that is quite effective at enforcing these kinds of soft rules - for example, it flags common style errors that developers make (not using imperative tense, not describing all the parameters, describing a function as returning true rather than non-nil values etc.). I find it quite invaluable for writing documentation that conforms to the standards of the elisp community without having to reverse engineer these constraints from existing code. Maybe something similar can be done for OCaml.
This approach to documentation is quite similar to the DITA-XML documentation architecture which is more general in field of technical activities and has more constraints, such as being topic oriented.
Basically, there are 3 types of topics by default:
Concepts: similar to diataxis’ explanation
Tasks: similar to diataxis’ how-to
References: similar to diataxis
One strong opinion in DITA is that the type of topic dictates its document structure (for instance, a task must have a list of steps) enforced by an xml schema. These basic document types can be extended with specializations: industry task (with safety warnings), mechanical task (with a list of required tools)…
More generally, I tend to think that Docbook was basically a good tool but a curse for program documentation because while it enforced a semantic writing rule, it also left completely open the architecture rules and did not help programmers writing useful documentation.
For specifics about Ocaml, I tend to refer to some good examples of use of a library in existing projects as poor man’s procedure. Maybe, adding some literate programming capability on some example applications of libs could help generate a better version of this kind of document.
Edit: changed URL and flipped “procedure” for “task”
I hate documenting but am doing it now. I’m always confused how to structure it. Glad someone is helping: I’ll try the method suggested. Thanks for the link!
Depending on the project, the effort to produce each of these 4 types of documents varies a lot. For example, if the library delivers OCaml bindings to a popular C++ library, the how-to guides will just be a pointer to how-to guides for the original library.
An advantage of having separate documents for different things is division of labor, and division of tools. The reference for a library is usually generated from mli files with ocamldoc or similar tools. Including tutorials in there feels like an unnecessary battle, they’re better published separately.
Personally I would highly suggest you do use the same tool for that. That is .mld files and odoc.
For document authors, it brings the cost for distributing and publishing your docs to zero and more importantly it allows you to build and easily maintain the cross references that are needed between these kind of documents (cross-references which, I insist, are very important).
For document readers it means they can peruse your docs in a single predictable place with a consistent layout, this being online in websites like https://docs.ocaml.pro/ or locally and offline via tools like odig.
I haven’t tried it out myself, I’m sure it’s great and makes things easier for the author.
My point is that from a reader’s perspective, it doesn’t matter. For most problems I face on a daily basis (how-to and explanation categories), I do a web search and I land on whatever page seems to answer my question. Whether it’s a page from the official documentation, a StackOverflow answer, or a an individual blog post, what matters is that I land on a solution to my problem. Ultimately the source doesn’t matter, although the StackOverflow/StackExchange format helps with spotting and discarding inadequate or outdated solutions.
Quality of documentation does matter to the reader’s perspective aswell especially if you don’t want to lose your time scouring the interwebs.
You can have as much howtos as you want but if they are outdated and full of broken links they are not of much use, if not unproductive. Personally I don’t see stackoverlfow as documention, but rather a bag of tricks of various (and sometimes dubious) quality.
To have checked crossrefs between your api docs and other documents is not a silver bullet but does help maintaining them to some extent. A further step would be code extraction but alas it has never been a priority in odoc’s development.
OCaml’s odoc (and predecessor ocamldoc) can provide both, with a little tweaking. In fact, I believe odoc can cover the full spectrum of Tutorials, How-to Guides, Explanation, and Reference. At its core, odoc’s system is a way of attaching docstrings to various elements of OCaml code. And then these get rendered alongside each other. This is similar to literate programming. E.g. look at what @antron is doing with Dream — Tidy, feature-complete web framework
That covers the Reference aspect, and that’s generally what we think of when we think of odoc. But it can do all other types of the Diataxis structure too, with some out-of-the-box project structuring. E.g., how about this structure:
Everything under doc/ can be full of just docstrings attached to each module item. They’re like chapters in a book. And it can be chock-full of cross-references. Of course, it can have code samples as well, depending on the lib project, so that they are guaranteed to compile. E.g.,
(* doc/tutorial/tutorial.ml *)
(** {1 Tutorial}
Bla bla bla*)
module Tutorial_01 = Tutorial_01
...
module Tutorial_NN = Tutorial_NN
And so on. Notice how by using dune’s built-in module nesting rules, we can include or exclude ‘chapters’ just like in serious typesetting systems like LaTeX with \include{...}.
To give more context and feedback, in addition to the reference@yawaramin linked, Dream also has a large set of examples (with more planned). The examples are divided into:
Several other categories of examples, including translations into Reason syntax, showcases of “adjacent” libraries and projects, and advanced or obscure techniques.
I don’t think these can be handled well by odoc, even with code extraction, because each example also serves as a complete, self-contained starter project, so that a user can begin with any of them. So, each example’s boilerplate files are also important (and themselves are examples). Also, different examples depend on different “third-party” libraries. I would like cross-referencing, but the Dream API is small enough that it doesn’t seem to be a big loss to do it manually.
Some of the examples serve as “how-to guides” or “explanations,” but I also plan to write non-example how-to Markdown files covering deployment, and the like.
In addition, Dream now has a playground, which is seems to be outside the Diataxis model, as it is for learning by experimentation. Maybe it’s somewhere in the far corner of “practical, task-oriented.” Most of the examples are loaded into the playground as starting points.
It depends up to where you push the idea. With a system where code extraction can extract multiple files and have hidden file parts and files in stop comments you can see a .mld as a “tarball” of sources which include all the boilerplate (either explicit and explained in the .mld rendering or hidden in the stop comments). Basically you meet literate programming.
