How to use a custom generator for odoc in dune?

I am trying to write my own odoc generator, in a project that is using packages from opam, and dune. I’m kinda new to dune, so I’m not sure how to set it up.

Here is an explaination how to include a costum generator in ocamldoc OCaml - The documentation generator (ocamldoc)

According to this, the (documentation) stanza has only 2 fields package and mld_fiels. Nothing for custom generators. https://dune.readthedocs.io/en/stable/dune-files.html#documentation-stanza

Can I somehow tell dune to use this custom generator for dune build @doc?

Here is my example setup:

# tree
.
├── a.ml    # the executable
├── dune
├── dune-project
├── lib
│   ├── dune
│   ├── my_lib.ml
│   ├── something.ml
│   ├── something.mli   # here is the documentation
└── my_doc_generator.ml

a.ml

let () =
  Dream.run ~interface:"0.0.0.0"
  @@ Dream.logger
  @@ Dream.router [

      Dream.get "/echo/:word"
        (fun request ->
          Dream.html (My_lib.Something.append_bla (Dream.param request "word")));

  ]

dune

	(name a)
	(public_name a)
	(libraries my_lib))

(documentation
	(package docdoc))

dune-project

(lang dune 3.14)
(package (name docdoc))

my_doc_generator.ml

module type My_generator =
sig
  class html :
    object
      method generate : Odoc_info.Module.t_module list -> unit
    end
end

module Generator =
  struct
  class my_generator =
    object (self)

      method generate module_list =
        print_endline "Hallo Welt!";
        List.map (fun _ -> print_endline "bla") (Odoc_info.Search.values module_list);
  end
end

lib/dune

(library (name my_lib)
  (public_name docdoc)
  (libraries dream))

lib/my_lib.ml

module Something = Something.Something

lib/something.ml

(** The irgnored module comment *)

module Something = struct
  let bla = " something"
  (** Some ignored comment *)

  let echo s = s

  let append_bla s = s ^ bla
end

lib/something.mli

(** The module comment *)

module Something : sig
  val bla : string
  (** Some bla comment *)

  val echo : string -> string

  val append_bla : string -> string
end
1 Like

You can’t do that with dune (also, note that ocamldoc and odoc are different tools). You can try to use dune build @doc-json which will generate HTML fragments in JSON files; that could be enough to do what you’re trying to.

2 Likes

Thanks for pointing out that odoc and ocamldoc are different tools, that was indeed confusing.

I do need a custom writer, so I guess odoc is out. I tried now to focus on ocamldoc and ocamlc. I created a few cmi/cmo files, but the last compilation step isn’t working.

/lib# ocamlc -c something.mli
/lib# ocamlc -c something.ml
/lib# ocamlc -c my_lib.ml 
/lib# cd ..
# ocamlc -o bla a.ml -I /root/.opam/default/lib/dream/ -I lib/
File "a.ml", line 1:
Error: Module `Dream' is unavailable (required by `A')

Any idea what I’m doing wrong? (dream was installed via opam)

P.S., @doc-json is not really what I need.

1 Like

If you use ocamlc manually, for the final linking step you have to pass all the required libraries on the command line, not only their paths.
So maybe something like that:

# ocamlc -o bla -I /root/.opam/default/lib/dream/ -I lib/ dream.cma something.cmo my_lib.cmo a.ml 

Thanks, that almost worked. Except it wants also the dependencies of dream:

# ocamlc -o bla -I /root/.opam/default/lib/dream/ -I lib/ dream.cma something.cmo my_lib.cmo a.ml 
File "a.ml", line 1:
Error: Module `Bigstringaf' is unavailable (required by `Dream')

This is not gonna be fun, as dream is fairly big, and in the real project we have even more depndencies. Any idea how this could be done (without listing every single cma-file)?

I am open to other ways/projects, which allow me to inject my own generator for the documentation.

The standard way (outside dune) is to use ocamlfind:

# ocamlfind ocamlc -package dream -linkpkg -I lib/ something.cmo my_lib.cmo a.ml 
1 Like

I have simplified my example, but still not working.

Currently I have just one file main.ml:

(** Some module level comment *)

let text = "Hallo Welt!"
(** A greeting to the world *)

let () =
  Dream.run ~interface:"0.0.0.0"
  @@ Dream.logger
  @@ Dream.router [

      Dream.get "/echo/:word"
        (fun request ->
          Dream.html (text ^ " " ^ (Dream.param request "word")));

  ]

This is the first output:

# ocamlfind ocamlc -package dream -linkpkg main.ml 
File "main.ml", line 1:
Error: Cannot find file /root/.opam/default/lib/ocaml/threads.cma

Turns out, threads is a folder, which has the file. So I just made a symlink:

# ln -s /root/.opam/default/lib/ocaml//threads/threads.cma /root/.opam/default/lib/ocaml/threads.cma 
# ocamlfind ocamlc -package dream -linkpkg main.ml 
File "main.ml", line 1:
Error: Module `Digestif' is unavailable (required by `Dream__http__Http')

Trying to find it, revealed 2 locations:

# ls /root/.opam/default/lib/ocaml/digest.ml
/root/.opam/default/lib/ocaml/digest.ml
# ls /root/.opam/default/lib/digestif/
META           c/             digestif.cmi   digestif.cmti  digestif.mli   dune-package   ocaml/         opam    ```

It appears to be installed, but isn’t found:

# ocamlfind list | grep -i Digestif -
digestif            (version: 1.2.0)
digestif.c          (version: 1.2.0)
digestif.ocaml      (version: 1.2.0)
# ocamlfind ocamlc -package dream,digestif -linkpkg main.ml 
File "main.ml", line 1:
Error: Module `Digestif' is unavailable (required by `Dream__http__Http')

Works with dune

If I just add the dune and dune-project files, it works (even if I remove the symlink).

# cat dune
(executable
	(name main)
	(libraries dream))
# cat dune-project 
(lang dune 3.14)
# dune exec ./main.exe
27.03.24 17:18:45.929                       Running on 0.0.0.0:8080 (http://localhost:8080)
27.03.24 17:18:45.929                       Type Ctrl+C to stop

So I’m not sure if something is installed wrong? Am I not doing it right? Is dune taking dependencies from somewhere else?

Using the threads library with ocamlfind seems to be broken since at least 5.1.0 (likely 5.0.0). See here.

1 Like

Thanks. This is really pushing my understanding of ocaml and its ecosystem.
Can I do anything, or just wait for an update to fix it?

P.S. I’m with ocamlc 5.0.0

In case it helps someone in the future, downgrading to ocaml 4.14 solved it. The command needs to include the threads lib manually. If you add -html it will produce proper HTML output.

I can also give a custom generator with the -g option, but for now there is no output. Hopefully I’ll figure it out.

# ocamlfind ocamlc -thread -package dream -I +ocamldoc -c src/my_doc_generator.ml 
# ocamlfind ocamldoc -g src/my_doc_generator.cmi -d output/ -thread -package dream src/main.ml
# tree
.
├── dune
├── dune-project
├── output
└── src
    ├── main.cmi
    ├── main.cmo
    ├── main.ml
    ├── my_doc_generator.cmi
    ├── my_doc_generator.cmo
    └── my_doc_generator.ml

In case it helps anyone, the following actually worked:

ocamlfind ocamlopt -shared -o custom.cmxs -I +ocamldoc src/my_doc_generator.ml
ocamlfind ocamldoc -g custom.cmxs -d output/ -thread -package dream src/main.ml

For some reason, it needs to be a shared library. Maybe because I’m including dream? Anyway, see here: OCaml - Native-code compilation (ocamlopt) (-shared option).

The name of the file custom.cmxs has to have .cmxs ending, otherwise the ocamldoc -g isn’t really accepting it. It won’t give any warning, it will just silently do nothing.

Current structure is:

.
└── src
    ├── main.ml
    └── my_doc_generator.ml