How to use generated .ml files from `dune utop`?

Ecosystem
, ,
1

I sometimes insert #mod_use "foobar.ml" into .ocamlinit to debug my library on OCaml toplevel. When I use Dune, however, I noticed some auto-generated files (e.g. parser.ml generated by parser.mly by Menhir) are not recognized by dune utop. How to configure dune utop to be able to find these files? Should I build a custom toplevel using toplevel stanza?

How to reproduce

Files:

.
├── bin
│   ├── dune
│   └── main.ml
├── dune-project
├── foobar.opam
└── lib
    ├── .ocamlinit
    ├── dune
    ├── lexer.mll
    └── parser.mly

.ocamlinit:

#mod_use "parser.ml";;
#mod_use "lexer.ml";;

Commands:

$ cd lib
$ dune utop .
Entering directory '/path/to/foobar'
─┬─────────────────────────────────────────────────────────────┬─
 │ Welcome to utop version 2.4.1 (using OCaml version 4.08.1)! │
 └─────────────────────────────────────────────────────────────┘
Cannot find file parser.ml.
Cannot find file lexer.ml.

Type #utop_help for help about using utop.

─( 16:11:53 )─< command 0 >───────────────────────{ counter: 0 }─
utop # (* BTW Parser can be opened here *)
open Parser;;
─( 16:12:39 )─< command 1 >───────────────────────{ counter: 0 }─
utop #

Environment

  • Dune 1.11.3
  • utop 2.4.1
  • OCaml 4.08.1
2

You should not use #mod_init when using dune utop since it should load the library for you. What does your dune file for lib look like?

3

Thanks for your reply =) My lib/dune is:

(env
  (dev
    (flags (:standard -warn-error -39))))

(library
 (name foobar))

(ocamllex lexer)
(menhir
  (modules parser))

I confirmed I can open Parser on dune utop, but values which should be defined in parser.ml such as Parser.A (a type constructor of a token) are not bound.

$ dune utop .
Entering directory '/path/to/foobar'
─┬─────────────────────────────────────────────────────────────┬──
 │ Welcome to utop version 2.4.1 (using OCaml version 4.08.1)! │
 └─────────────────────────────────────────────────────────────┘
Cannot find file parser.ml.
Cannot find file lexer.ml.

Type #utop_help for help about using utop.

─( 17:39:57 )─< command 0 >────────────────────────{ counter: 0 }─
utop # open Parser;;
─( 17:39:57 )─< command 1 >────────────────────────{ counter: 0 }─
utop # Parser.A;;
Line 1, characters 0-8:
Error: Unbound value Parser.A
4

Since the library foobar is wrapped, all modules are available as Foobar.ModuleName. Your parser is thus Foobar.Parser. The Parser module that you see at the toplevel is the compiler one (this is unfortunate implementation detail of dune utop).

2 Likes
5

Thank you for the explanation! I can open Foobar.Parser. I can also snip Foobar. if I write open Foobar into .ocamlinit. #mod_use is unnecessary.

Can I ask another related question? If I locate all above codes in one directory, then how to open Parser? Foobar module is not found on dune utop. Is it necessary to separate lib and bin?

Files:

.
├── dune-project
├── foobar.opam
└── src
    ├── dune
    ├── lexer.mll
    ├── main.ml
    └── parser.mly

src/dune:

(env
  (dev
    (flags (:standard -warn-error -39))))

(executable
 (public_name foobar)
 (name main))

(ocamllex lexer)
(menhir
  (modules parser))

Commands:

$ cd src
$ dune utop .
Entering directory '/path/to/foobar'
───┬─────────────────────────────────────────────────────────────┬────
   │ Welcome to utop version 2.4.1 (using OCaml version 4.08.1)! │
   └─────────────────────────────────────────────────────────────┘

Type #utop_help for help about using utop.

─( 18:42:20 )─< command 0 >────────────────────────────{ counter: 0 }─
utop # open Foobar.Parser;;
Line 1, characters 5-18:
Error: Unbound module Foobar
─( 18:42:20 )─< command 1 >────────────────────────────{ counter: 0 }─
utop # open Main.Parser;;
Line 1, characters 5-16:
Error: Unbound module Main.Parser
6

Indeed, if you are not exporting a module as a part of a library (public or private), they would not be available in dune utop.

1 Like
7

Thank you for the information! It really helps me. I now fully understand why I can/cannot open modules in my real project.

But why does dune utop only handle libraries? I think it’s inconvenient, for example, when testing main.ml for executables in REPL. Is there any technical reasons? How about wrapping things defined in programs for executables by a module such as Main, like Dune does for libraries?

8

I think that the main idea is that the modules of an executable are private and possibly effectful.

9

That’s exactly the idea. That’s also why we can’t have inline tests in executables. I’ve been wondering about allowing to specify executables by function names rather than module names. That would make such things possible.

1 Like
10

I don’t understand why dune utop shouldn’t used for private codes; I think it’s used for testing internal/private codes by hands. “It is possibly effectful (more likely than libraries)” is a reason to understand a little more for me.

Anyway, thank you two for letting me know how to use generated files! It really helps me. I select octachron’s reply as a solution :slight_smile:

11

To workaround this problem, at ahrefs we write the whole code of the executable in a library and only one module for the executable itself. It’s not ideal, but we survive.

12

let’s consider the following OCaml program:

let () =
  print_endline "Hello, world!";
  exit 0

If you load it in the utop, utop would print Hello, world! and exit immediatly because of the exit 0, so you wouldn’t be able to do anything. The entry point of OCaml program is always a toplevel expression, that’s why we can’t load binaries in the toplevel or define inline tests in executables.

The workaround @Khady mentioned is indeed the only solution at the moment. In the future, we are thinking of allowing to declare executables by function names rather than module names. i.e. instead of the entry point being a toplevel expression, it would be a function like this:

let main () =
  print_endline "Hello, world!";
  exit 0

Then in your dune file you would write something like this:

(executbale
 (name foo)
 (main Foo.main)
 ...)
1 Like