Is dune just for building targets, or can I also make it organize build output?

Learning
1

Let’s say my source code has the following structure:

src
|-- bin
|   |-- main.ml
|-- assets
    |-- icons
    |   |-- pin.svg
    |-- index.html
    |-- manifest.json

I want dune to spit out an output dir in the default context like this:

_build
|-- default
    |-- output
        |-- manifest.json
        |-- index.html
        |-- static
              |-- icons
              |    |-- pin.svg
              |-- main.bc.js

In short, I want dune to rearrange the artifacts in the output dir, along with building the targets.

It seems like dune is the right tool for this, but I spent a lot of hours trying to make dune do this, but failed.

Am I correct to assume that dune cannot do this? Is dune’s only job to produce the build targets (main.bc.js in this case)?

2

You should be able to do this using some copy_files stanzas or copy actions in rule stanzas.

3

Echoing what @sim642 mentioned, you can use copy_files or copy actions, or in the worst case, just shell out to bash (assuming you don’t care much about supporting non-unix environments (who uses them anyway? :wink: )). For example, I’m not sure if this was the most elegant way, but I was able to use dune to setup a website generator that performed some fairly complex re-arrangment of the source directory using the following build rules:

(rule
 (alias all)
 (deps (sandbox always) (glob_files_rec static/*) (glob_files_rec data/*) favicon.ico)
 (targets (dir web))
 (action
  (progn
   (bash "mkdir web")
   (bash "cp favicon.ico web/")
   (bash "cp -lR ./static/ ./web/static/")
   (run ./src/generator/generator.exe "./data" "./web")
)))
2 Likes
4

The copy_files and copy_files# stanzas specify that files from another directory could be copied to the current directory, if needed.

This would imply that I would need a output/dune file in source root, in which I would put the copy_files stanza. This would start shaping the source structure to accommodate the output structure.

I tried various iterations of copy stanza too, but as far as I remember, dune didn’t allow any of those for some or the other reason. One reason was that the source and dest files have the same name, other was related to not being able to copy into child dir, because the path of target and deps have to be the same. Sorry if none of my explanations make sense. dune’s didn’t either.

As an aside, what I did realize from all of this though, is that dune first copies the source files (and files marked as deps in the build alias) from source dir into the build context root, and then start building. This explains why it can’t overwrite files of the same name. I would appreciate if someone validates or corrects this assumption.

5

I wanted to stave off handing everything out to bash. But it looks like the only solution. None of the other actions seem to fit the purpose.

6

Have you tried GitHub - diskuv/diskuvbox: Basic, cross-platform set of commands to manipulate and query the file system, and OCaml library. (see also [ANN] diskuvbox: small set of cross-platform CLI tools)? Its purpose is to provide a small set of command-line tool helpers in a cross-platform fashion to avoid depending directly on an Unix shell.

Cheers,
Nicolas

7

I don’t mind using the Unix tools as long as reaching out for bash and run actions in dune is the only blessed way for my problem (i.e., as long as I am not missing any other approach).

If so, then I will mark Gopi’s answer as a solution.

1 Like
8

The subdir stanza can work around that actually.

1 Like
9

@sim642 This tip helped! In case it helps anyone else:

With the following stanzas

(executables
 (names bg long_click prefs)
 (modes js)
 (libraries js_of_ocaml js_of_ocaml-lwt promise_jsoo ocaml-vdom)
 (preprocess
  (pps lwt_ppx js_of_ocaml-ppx)))

(alias
 (name dev))

(subdir
 output
 (copy_files
  (alias dev)
  (files %{project_root}/{manifest.json,preferences.html}))
 (subdir
  js
  (copy_files
   (alias dev)
   (files %{project_root}/*.bc.js)))
 (subdir
  icons
  (copy_files
   (alias dev)
   (files %{project_root}/icons/*))))

dune build @dev maps the source dir with structure:

$ tree -a -I .direnv -I .git -I .vscode .
.
├── bg.ml
├── common.ml
├── dune
├── dune-project
├── icons
│   ├── pin-32.png
│   ├── pin-64.png
│   ├── pin-dark.svg
│   └── pin-light.svg
├── long_click.ml
├── manifest.json
├── preferences.html
├── prefs.ml

to the following output structure:

$ tree -a _build/default/output/
_build/default/output/
├── icons
│   ├── pin-32.png
│   ├── pin-64.png
│   ├── pin-dark.svg
│   └── pin-light.svg
├── js
│   ├── bg.bc.js
│   ├── long_click.bc.js
│   └── prefs.bc.js
├── manifest.json
└── preferences.html

A few notes:

2 Likes
10

You can avoid much of that by using diskuvbox. (I’m biased since I wrote it.)

Install it:

opam install diskuvbox js_of_ocaml-compiler

Then you can use an easy-to-understand solution:

(executables
 (names bg long_click prefs)
 (modes js)
 ;(libraries js_of_ocaml js_of_ocaml-lwt promise_jsoo ocaml-vdom)
 ;(preprocess
 ; (pps lwt_ppx js_of_ocaml-ppx))
 )

(rule
 (alias dev)
 (deps
  (glob_files icons/*)
  (:rootfiles manifest.json preferences.html)
  (:jsfiles bg.bc.js long_click.bc.js prefs.bc.js))
 (action
  (progn
   (run diskuvbox copy-file-into %{rootfiles} output/)
   (run diskuvbox copy-file-into %{jsfiles} output/js)
   (run diskuvbox copy-dir icons output/icons))))

Working test code is at GitHub - jonahbeckford/discuss-ocaml-org-10368 . Use diskuvbox on your command line to get help.

1 Like
11

I will try to summarise this thread.

  1. As mentioned here (Is dune just for building targets, or can I also make it organize build output? - #8 by sim642), a combination of subdir and copy_files stanzas is close to what I wanted as a solution. It also looks to be the most cross-platform with no extra dependencies, albeit a bit verbose (subjectively). I went with this solution.

  2. Another cross-platform solution diskuvbox (Is dune just for building targets, or can I also make it organize build output? - #10 by jbeckford) keeps things simple, but comes with a dependency.

  3. Dropping down to unix shell (Is dune just for building targets, or can I also make it organize build output? - #3 by Gopiandcode) is probably not cross-platform, but what I would have gone with, if I wasn’t informed of the combo mentioned in #1.

With so many good answers, I will mark this summary as the solution.

4 Likes