[ANN] Release of odoc 2.0.0

Hot on the heels of the OCaml 4.13 announcement(s!), the odoc team is pleased to announce the release of odoc 2.0.0!

tl;dr:
The new version produces much better output than the old version, it’s the engine at the core of the package docs in v3.ocaml.org, and it also has a new website.

This release has been a long time coming – years! – and contains several notable improvements over the odoc 1.5 series: a new language model, a new rendering layer allowing output in several formats, and improved control over the output structure.

New Features

New Language Model

The internal library used by odoc that models the OCaml module system has been completely rewritten over a multi-year effort by @jonludlam and @Julow, according to a design by @lpw25. The rewrite gives odoc a much better understanding of the module system compared to the original implementation. This library is used for two main processes:

  1. To perform expansions, which is the process where odoc takes complex module type expressions like this one from tyxml:
module Make
    (Xml : Xml_sigs.T with type ('a, 'b) W.ft = 'a -> 'b)
    (Svg : Svg_sigs.T with module Xml := Xml)
  : Html_sigs.Make(Xml)(Svg).T
    with type +'a elt = Xml.elt
     and type +'a attrib = Xml.attrib

Then turns it into an output page containing the correct types, values, modules, includes, and documentation.

  1. To perform resolutions, which is where odoc handles complex paths found in OCaml source in order to calculate the correct definition link. For example, in the following snippet:
module type A = sig
  module M : sig module type S end
  module N : M.S
end

module B : sig module type S = sig type t end end

module C : A with module M = B with type N.t = int

type t = C.N.t

resolution is the process by which odoc determines which documentation page to take you when you click on C.N.t.

The new model has logic to handle many features of the OCaml language, as can be explored here.

A particularly important improvement is in handling canonical modules (explained in the link above). The upshot of this is that there should never be any more odd double underscores leaking into your docs!

For some more info on this, as well as the new output renderers, see our talk at the OCaml workshop last year

New Output Renderers

@Drup put a considerable amount of work into replacing the odoc 1.5 custom HTML generator with a new rendering layer. This features a new intermediate format allowing new output formats to be added far more easily than before.

Included in odoc 2.0 are renderers for HTML and man pages (both contributed by @Drup) and LaTeX (contributed by @Octachron). The LaTeX renderer has already been integrated into the OCaml build process to generate docs (see documentation: configuration switch for an odoc documentation mode by Octachron · Pull Request #9997 · ocaml/ocaml · GitHub and related PRs). @jonludlam also made an alternative HTML renderer designed specifically for v3.ocaml.org. Finally, a new markdown renderer is being prepared by @lubegasimon and should land in the next release.

We look forward to many new renderers being created for the varied use cases present in the community!

Output Structure

odoc 2.0 introduces a new mechanism to specify the structure of the files produced. Although it’s a relatively simple new feature, it nevertheless has enabled odoc to be used in new ways. In particular, it has allowed odoc to construct the
package documentation for the new OCaml website, v3.ocaml.org. There is also an example driver, showing how odoc can be used to construct a stand-alone website for an OCaml package that contains fully-linked documentation for a package and all of its dependencies. This has been used to create odoc's new website.

New Drivers

Like the OCaml compiler itself, running odoc on your code requires careful sequencing of the invocations to produce the correct result. Fortunately both dune and odig understand how to do this, so most users don’t need to know the details. If you want more than these tools provide though, we’ve written a simple reference driver, documenting exactly what’s necessary to use odoc to produce rich documentation. A more complete (and more complex) example is the tool voodoo, which is being used to create the docs for v3.ocaml.org.

v3.ocaml.org

As previously posted, the new version of the OCaml website has been under development for some time now, and an important new feature is the integration of package listings, including documentation for every version of every package. More has been written about this elsewhere, but it’s important to note that the new OCaml.org website required a preview version of odoc 2.0 to work. We’ve made a few bug fixes since then, so we will update the pipeline to use the released version very soon. For more info on the pipeline to build the docs, see our recent talk at this year’s OCaml Workshop.

New Website

The website for odoc has been improved with guides for documentation authors, integrators, and contributors. This site is intended to grow over time with more content to help people write docs for their packages.

OCamldoc?

This release, particularly because of the new output renderers, puts odoc in a place where it supercedes OCamldoc in most respects. There are a few features we’re missing (see the comparison in the docs), including
most notably that we don’t render the source (OCamldoc’s --keep-code argument), and that there is no support for custom tags. If odoc is lacking features that you’re currently relying on in OCamldoc, we’d love to hear from you!

More Docs!

Finally, I’d like to use this opportunity to launch an invitation. With v3.ocaml.org now showing all the package docs in their current state, I’d like to invite all our package authors, maintainers, contributors, and users to take a look over their favourite packages and see what the documentation looks like. Good documentation is one of the most important requests from the previous OCaml developer surveys, and with v3.ocaml.org as a new documentation hub, now is a great time to be making improvements where they’re required. With this new release of odoc, previewing your docs should be as simple as dune build @doc.

Some packages already have great docs - a few examples are:

many others have more patchy docs. Let’s fix that!

We’re also looking for more contributors to odoc. It’s much improved now, but there’s still plenty more to do. Come and join the fun!

40 Likes

Congrats on the release, looks like a great improvement! Some of the excuses to write documentation has been the tooling, it might not be the case anymore :smiley:

btw, all the v3.ocaml.org/p/* links are broken:

2 Likes

Not only that, but also contributing to odoc and how to write good documentation :grinning_face_with_smiling_eyes:odoc (odoc)

1 Like

That’s odd! All the links seem to be working for me…? what do you get?

Weird, using curl I got a 404 and in the browser, I got 500.

curl -sI https://v3.ocaml.org/p/brr/0.0.1/doc/ffi_manual.html
HTTP/1.1 404 Not Found
Server: nginx/1.21.1
Date: Wed, 06 Oct 2021 18:23:33 GMT
Content-Type: text/html; charset=utf-8
Content-Length: 75619
Connection: keep-alive

Chrome:

This page isn’t working v3.ocaml.org is currently unable to handle this request.
HTTP ERROR 500

EDIT: Alright, It’s def not related with odoc 2.0, but more with the v3 website and my chrome. Works perfectly in Firefox and/or my phone. Sorry for the trouble!

Hum, working for me in Chrome, Safari and Firefox, but I do get the same 404 with curl. Well I’m just going to take this opportunity to point out that v3.ocaml.org is also looking for contributors and bugfixers :wink:

5 Likes

All of this is awesome, thanks for all the hard work!

Yay!

1 Like

Congrats and many thanks to all involved! This is wonderful :slight_smile:

FYI: in Firefox on my iPhone the pages do not legibly:

This is from the first link: Bigbuffer (odoc.core_kernel.Core_kernel.Bigbuffer)

2 Likes

Responsiveness is one of the things that v3.ocaml.org does rather a lot better than the stock odoc css - take a look at the equivalent page.

We’ve also got some work planned to improve the layout of long module types and types - see here - current roadmap here, and I think it’s likely we’ll be thinking about responsiveness when handling that, so I expect that odoc 2.1 will make the output much more iphone friendly.

5 Likes

Yes, the v3.ocaml.org looks beautiful on mobile and desktop!

4 Likes

I am noticing something on Core_kernel.Bigbuffer · core_kernel v0.14.2 · OCaml Packages – look at the part where it says “include Base.Buffer.S”. In the iphone version screenshot post (ignore, for now, the lack of responsiveness) you can see that around the arrow is a kind of “tab” that shows the extent of the items brought in by the signature inclusion. This is useful because an include can bring in many items.

This visual device is not present on the v3.ocaml page. Is this a theming issue – that they simply disabled the tab that shows which items are coming with a include?

It’s a CSS issue - tracked here. “Good first issue”, hint hint! :smiley:

4 Likes

Weird, using curl I got a 404 and in the browser, I got 500.

For the curl error, we’ve just found out that HEAD http requests are not properly handled. So thank you for spotting this !

An issue has been created for this: HEAD requests yield 404s · Issue #148 · ocaml/ocaml.org · GitHub

2 Likes

The documentation provided by Rust’s cargo doc gives you a search bar in your browser. This feature is implemented in the client frontend rather than backend by sending the search index as a json object. This avoid having to package a server backend with the docs.

There seems to be some discussion about providing a similar feature here but nothing much subsequently. I was curious if that had been more conversation subsequently.

Even a basic ability to fuzzy search a function name or type name would be useful.

If you decide to go the way of using a bloom filter for the search index, then, FYI, there’s a fully verified implementation of a bloom filter here: GitHub - verse-lab/ceramist: Verified hash-based AMQ structures in Coq :slightly_smiling_face:

Disclaimer: I’ve never actually tried running it.

1 Like

That particular issue is on the roadmap for 2.2. Search is definitely something we want to implement. The question of whether it’s client-side or server-side is something to consider carefully, and I think we’ll likely need some real-world experience before deciding. Certainly for v3.ocaml.org it seems unlikely that we’ll be able to implement search across all packages/versions client-side only!

2 Likes

Remember Jordan was fighting for that particular problem when they wanted to unify esy.sh, reason-native.com, reason-react docs.

He came up with GitHub - jordwalke/paradoc: One Click Docs with a mention to GitHub - jordwalke/paradoc: One Click Docs.

If I can suggest something I will try to push for a client-side and if that doesn’t workout can try a 3rd party with algolia. Their doc search is quite powerful. The basic reasoning behind this are a bunch of nice details: possibility for an offline search, translatable-friendly, way simpler to implement (IMHO).

The odoc documentation refers to the OCaml manual and lists the differences to OCaml. If we want odoc to become the new standard, I think a self-contained documentation would be helpful. I also think the format documentation should be at the center of the documentation and all other aspects should be secondary.

4 Likes