Cannot find included file during preprocessing

Learning
1

Hello,
So I was recently tasked to update an old project, which uses dune, and I am not sure I fully grasp what is happening in one specific part.
The project was last used 2+ years ago with dune 1.8 and ocaml 4.09.0, which i updated to what I am now using; dune 3.8.0 and ocaml 5.0.0, as well as cppo 1.6.9.

When attempting to dune build, this happened:

File "dune", line 4, characters 22-54:
4 |   (preprocess (action (run %{bin:cppo}  %{input-file})))
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Error: File "ast.ml", line 613, characters 0-39
Error: Cannot find included file "visitorInternalDsl.ml"

Which dissappeared after i added (preprocessor_deps visitorInternalDsl.ml).
I then had the same error with ast.ml not finding some of the generated.ml templates:

File "dune", line 7, characters 22-54:
7 |   (preprocess (action (run %{bin:cppo}  %{input-file})))
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Error: File "ast.ml", line 616, characters 0-50
Error: Cannot find included file "templates/script.ml.generated.ml"

And I cannot seem to fix this one. So far I have tried:
Adding (source_tree _build/default/templates/) to the preprocessor_deps.
Adding (include_subdirs unqualified) to the dune file.
Modifying the path of the included file in ast.ml.
Using -I option in the cppo action.

My understanding is that I fail to specify to dune and/or cppo where it should look for the files, but even that may be wrong.

My dune file is:

(ocamllex lexer)
(menhir (flags ("--explain" "--dump")) (modules grammar))
(executable
  (preprocessor_deps visitorInternalDsl.ml)
  (preprocess (action (run %{bin:cppo}  %{input-file})))
  (name      parser)
  (libraries core))

An extract of ast.ml, where the problematic line is:

    method visit_script (my:script) = 
      #include "visitorInternalDsl.ml"

      (*INTERNAL DSL START*)
      #include "templates/script.ml.generated.ml"
      (*INTERNAL DSL STOP*)
      ht

My project tree is as follows:

.
├── ast.ml
├── _build
│   ├── default 
│   │   ├── ast.ml
│   │   ├── globals.ml
│   │   ├── globals.pp.ml
│   │   ├── grammar.mly
│   │   ├── grammar__mock.ml.mock
│   │   ├── grammar__mock.ml.pp.mock
│   │   ├── lexer.ml
│   │   ├── lexer.mll
│   │   ├── lexer.pp.ml
│   │   ├── parser.ml
│   │   ├── parser.pp.ml
│   │   ├── templates
│   │   │   ├── actions.ml
│   │   │   ├── actions.ml.generated.ml
│   │   │   └── every other template
│   │   └── visitorInternalDsl.ml
│   └── log
├── dune
├── dune-project
├── globals.ml
├── grammar.mly
├── io_externs.phaistos
├── lexer.mll
├── Makefile
├── merlin-init.sh
├── parser.ml
├── README.md
├── templates
│   ├── actions.ml
│   ├── arithmeticOp.ml
│   └── other templates
├── temp.txt
└── visitorInternalDsl.ml
2

I do not understand how your project structure works. Who generates templates/script.ml.generated.ml?

In general I would approach this by defining targets, e.g.

 (targets script.ml)
 (deps    script.cppo.ml)
 (action  (run %{bin:cppo} %{deps} -o %{targets})))

This way you can specify which modules need to be created and what depends on what and how it is created.

3

First, thank you for your reply.

all of the .generated.ml files are generated by a different part of the project, called in a previous make step.
I am a little confused as to what I should add in the targets/deps parts of the rule: To my understanding deps would be the files I need, so the generated ones, but the only ‘target’ I can identify in the current version of this is the target of the dune build command: parser.exe

but trying with this:

(rule
 (deps templates/scripts.ml.generated.ml)
 (targets parser.exe)
 (action  (run %{bin:cppo} %{deps} -o %{targets})))

give me No rule found for templates/scrips.ml.generated.ml
and so i think i misunderstood how to use your advice.

4

So generally I would avoid mixing build systems, because now you need to make both aware of each other. Maybe you can make dune build these too?

In any case, Dune resolves dependencies by 1. either checking whether the file exists (in your case, templates/scripts.ml.generated.ml relative to the dune file) or running the rule that has the dependency as a target (only one of them, if there is both a file and a rule it will complain because it is unclear which one to pick — unless you specify otherwise).

In your case, No rule found for templates/scrips.ml.generated.ml means that Dune did not find the templates/scripts.ml.generated.ml file nor is there any rule to produce this file. You’d need to make sure the paths fit.

Also, the rule you wrote means “to create parser.exe you need to procure templates/scrips.ml.generated.ml and run cppo templates/scrips.ml.generated.ml -o parser.exe” which will generate a parser.exe but it will not be an executable, it will be an text file with the output of cppo.

What you probably want is something more like

(executable
  (name parser)
  (modules parser))

(rule
 (deps templates/scripts.ml.generated.ml)
 (targets parser.ml)
 (action  (run %{bin:cppo} %{deps} -o %{targets})))

Which means “to create parser.exe you need to create parser.ml. parser.ml is generated by cppo, which needs templates/scrips.ml.generated.ml and will run cppo templates/scrips.ml.generated.ml -o parser.ml to create it”.