Dune build vs Makefile

Learning
1

I am trying to figure out this 10x difference.

time dune build @app
                                   
real	0m9.143s
user	0m8.279s
sys	0m0.883s

vs

 time make
ocamlfind ocamlc -g -linkpkg -package brr min.ml
js_of_ocaml a.out -o min.js

real	0m0.665s
user	0m0.606s
sys	0m0.059s

These are both incremental builds.

cat bin/dune ; echo "===="; cat Makefile ; 
(executables
  (names min)
  (libraries brr)
  (modes js))

(rule
  (targets min.js)
  (deps min.bc.js)
  (action (run cp %{deps} %{targets})))

(alias
  (name app)
  (deps min.js ))

====
build:
	ocamlfind ocamlc -g -linkpkg -package brr min.ml
	js_of_ocaml a.out -o min.js

fancy_console:
	ocamlfind ocamlc -g -linkall -linkpkg -package brr,brr.poked min.ml
	js_of_ocaml $(ocamlfind query -r -i-format brr.poked) -I . \
		--toplevel a.out -o min.js

.PHONY: build fancy_console

the min.ml consists of:

line 0: open Brr
line 1-1001: repeated 1000 times: let () = El.set_children (Document.body G.document) El.[txt' "Hello World!"]

===

Question

  1. what is the difference between “dune build @app” and “make” ?

  2. for tooling reasons, I’d prefer to make the dune build @app faster – how do I do that ?

2 Likes
2

It’s not clear what you are measuring but I assume that’s cold builds.

Someone more versed into dune’s js_of_ocaml compilation strategy may offer better answers but the explanations must be along this:

This invokes your build in --profile=dev. With this profile dune does separate JavaScript compilation.

This entails compiling bytecode library archives (the .cma files of the libraries you use) of each of the libraries you use in your program to separate JavaScript files. You’ll pay for that only once on cold builds or when you upgrade your library dependencies.

That’s long because you also end up compiling code you won’t use (there’s a lot of stuff in brr.cma). That’s not the case if you first link your bytecode with the .cma files perform dead code elimination and then compile to JavaScript.

If you prefer the latter you should compile with --profile=release (I don’t know if it’s possible to enable or disable separate compilation in dev mode, consult the dune manual).

Use --profile=release however when your app grows the numbers are going to turn upside down for incremental builds.

2 Likes
3

No, (assuming I’m doing everything right) these are incremental builds.

  1. build

  2. insert a “\n” in the middle of min.ml

  3. rebuild

4

Then I think dune also automatically generate source maps. Check out your build log you likely don’t have the same invocation as in your Makefile.

5

Indeed, and the manual claims this is faster: JavaScript Compilation With Js_of_ocaml — Dune documentation.
However in the case of javascript (at least for small examples like this) that is clearly not true, because in dev mode I get a 2.3MB .js file, whereas in release mode a 61KB one, and producing the latter is a lot less work:

Summary
  'dune build --profile=release @app' ran
    9.20 ± 0.38 times faster than 'dune build --profile=dev @app'

There is a flag to force whole program mode for js_of_ocaml, but that is still slow (probably due to lack of dead code elimination, still produces a 512KB js file):

(env
  (dev (js_of_ocaml (compilation_mode whole_program))))

The dev profile also has other things like source maps and pretty printed output enabled, the release build doesn’t (you can see exactly what it does with --verbose).

If you add the following to your dune file it’ll speed it up considerably, however you’ll lose the source map:

(env
  (dev (js_of_ocaml
         (flags (:standard --no-source-map))
         (compilation_mode whole_program)
         )))
5 Likes
6

Confirming this dropped my incremental dune build @app down to Makefile levels:

 time dune build @app
                                  
real	0m0.892s
user	0m0.785s
sys	0m0.104s
7

I have a dumb followup question that I don’t understand.

  1. with source map: 9+ seconds, w/o source map: < 1 second

  2. source map is just a giant hashmap mapping lines of js to line of ocaml right ?

  3. our program is only 1000 lines of ocaml; why is this tiny sourcemap (hash table) adding 8 seconds, taking build time from < 1 sec to > 9 secs ?

8
  • Sourcemap rely on ocaml debug info, processing the additional info take some extra time, but it’s not the culprit here.
  • When sourcemap is enabled, as a way to preserve more locations, jsoo put each function call in its own node in the control flow graph and I believe that your artificial example is hitting some quadratic algorithm. It depends on the number of function call inside a single function.
  • It’s not clear that it affect real program that much but free to open an issue Issues · ocsigen/js_of_ocaml · GitHub
3 Likes
9

I’ve a quick fix in Compiler: no longer split blocks at fun call to propagate location by hhugo · Pull Request #1407 · ocsigen/js_of_ocaml · GitHub. With your example, compilation time is around ~0.5s. I need to check that the generated sourcemap has the same amount of information. (Edit: need more work to preserve all the sourcemap info)

4 Likes