[ANN] Sherlodoc, a search engine for OCaml documentation

Ecosystem
, ,
1

We (@art-w, @EmileTrotignon) are pleased to announce the first release of Sherlodoc on opam!

Sherlodoc is a search engine for OCaml documentation, which supports search by name, documentation and fuzzy type search (similar to Hoogle in the Haskell world). You can try it at doc.sherlocode.com on the libraries published in the opam repository, thanks to all the documentation available on OCaml.org.

  • The main focus of this release is the integration with Odoc (and support for dune build @doc is in progress). The latest release of Odoc 2.4.0 introduces a search bar in its static html documentation, which enables a javascript search engine to run client-side (without a server). Since Sherlodoc is fully implemented in OCaml, it was straightforward to compile it to javascript using js_of_ocaml. See for example the Cmdliner package documentation with search hosted on github static pages.

  • Sherlodoc now uses the odoc.search library and is hence able to search for modules, types, constructors, record fields, etc as well as inside documentation comments. (Protip for library authors: Sherlodoc heavily favors documented functions :wink: )

  • There is also a new command-line interface that you could use to locally search packages installed in your switch, although the CLI was developed for debugging and could use some love!

Instructions for trying out this new version can be found in the readme, but it can be as simple as:

$ opam install sherlodoc odig
$ odig odoc
$ export SHERLODOC_DB=/tmp/sherlodoc.marshal
$ sherlodoc index $(find $OPAM_SWITCH_PREFIX/var/cache/odig/odoc -name '*.odocl'  | grep -v "__")
$ sherlodoc search "list map"

We hope you enjoy it to find your way in the ever-growing OCaml ecosystem! <3

This is an early release: if you believe that having a great search engine is important, please help us by reporting any issue you encounter, queries with unexpected results, or by contributing new PRs! (to e.g. improve the CLI, the website or even the algorithms if you ever wanted to learn how a search engine works!)

47 Likes
2

doc.sherlocode.com is amazing. It has been my number 1 tool to search for the functions that I need, for a while now.

7 Likes
3

Thanks a lot for this kind of Hoogle for OCaml programmers; very useful !

2 Likes
4

Amazing! Is there a plan to integrate it directly into ocaml.org OCaml Packages · Browse Community Packages?

4 Likes
5

Yaya!!! :heart: Plus one to this!!! :heart:

Just a point for the impact and utility of Sherlodoc and Sherlocode, Sherlocode ended up being invaluable for collecting benchmark programs for our publication Mostly Automated Proof Repair for Verified Libraries.

Thanks so much to the Sherlodoc and Sherlocode team for building this tool!!!

5 Likes
6

The one major thing that is needed to add Sherlodoc with ocaml.org is to implement indexing of new packages and new package versions, so that the search index stays up to date.

3 Likes
7

Wonderful! Have you considered compiling to Wasm with wasm_of_ocaml instead? We’ve seen some impressive performance gains there, ranging from 1.5x to 3x improvements in some cases.

2 Likes
8

Thanks everyone for your interest, it’s super motivating!
There was a lot more work done by @EmileTrotignon that led to this release than listed in the announce (one of the secret improvement was to keep the database as small as possible for client-side usage, but also a ton of quality testing). And of course it wouldn’t have been possible without @panglesd and @Juloo adding search support in Odoc… and let’s not forget the years of work on odoc and ocaml.org which enabled search to be a natural next step in Tarides commitment to improve the state of the OCaml documentation!

@Gopiandcode > Woah so glad to hear that! :heart: (the link to your beautiful paper wasn’t working: Mostly Automated Proof Repair for Verified Libraries (pdf)) Since you are also using Coq, you might enjoy Sherlocoq by @patricoferris!

@kayceesrk > In the short term we are hoping that the sherlodoc.js search engine will be easy to integrate with ocaml.org per-package search. The opam-wide search will require more work to stay uptodate, but we hope the community feels it’s worth the effort! :stuck_out_tongue: (as the current sherlodoc website is hard to discover)

@rikusilvola > Love this idea! We’ve added an issue to track the impact of wasm_of_ocaml on the search performances :slight_smile:

In other old news for French speakers, I gave a talk about sherlocode and sherlodoc usage and implementation at the OCaml Users in PariS (OUPS) in March 2023, which could provide some context if you want to take a stab at one of our issues :-°

6 Likes