Replace jsoo/link with cat or esbuild?

Learning
1

Quoting Separate compilation

js_of_ocaml link my-runtime.js stdlib.js libname.js modname.js std_exit.js-o myexe.js

Is there a way to replace the js_of_ocaml link with cat or esbuild ? If not, what else does js_of_ocaml link do ?

2

The next version of js_of_ocaml will match the library archive link model of ocaml which is that unused compilation unit of archives are not linked in unless used by the program or -linkall is specified.

In that scenario I expect js_of_ocaml link to perform the compilation unit pruning from the library .js files.

3

I apologize for not being clear. My main focus is:

I.e. I am looking to minimize my dependency on js_of_ocaml. I am looking for a situation where I use it only to recompile updated *.cmo files, then using something blazing fast (like cat or esbuild) to combine everything together.

Phrase another way: the separate-compilation out put of jsoo – is it safe to join it using cat/esbuild, or is using jsoo link a hard requirement for the final step ?

4

It depend on what you mean by safe. As mentioned by @dbuenzli, with the next version of jsoo, jsoo link will perform some compilation unit pruning for libraries. So cat will not longer be equivalent.

jsoo link also does the merge of sourcemap when requested.

jsoo link is supposed to be fast (but maybe when sourcemap is requested). In particular, it doesn’t parse the javascript. I don’t know what you’re trying to achieve, but if jsoo link is too slow for you, contributing to jsoo could be a better use of your time (compare to working around the tool).

I’m curious, have you witnessed jsoo link being slower that esbuild ? do you have numbers ?

5

I can run the output js. With the exception of potential slowdowns, the code outputs the same answer.

I do not have numbers. I have a fairly slow incremental rebuild with try-eio. I am currently trying to dissect all the parts and build it manually w/o dune.

I have try-eio split into “server” (merlin-js + js_top_worker) and “client” (codemirror + merlin_js_rpc / js_top_worker_rpc) parts.

I can state that for the “client” part:

  • rescript individual files + combine with esbuild = blazingly fast
  • jsoo = very slow incremental builds
6

I can run the output js. With the exception of potential slowdowns, the code outputs the same answer.

Then it’s not 100% safe as linking compilation unit can perform side effects. Once jsoo link start omitting compilation units, running the code could output different answers.

jsoo = very slow incremental builds

How slow is very slow ? Your comments would be more actionable if it came with datapoints. Do you have a git repo to reproduce the issue ?

1 Like
7

Thank you for clarifying this.

I do not have a git repo yet. I apologize for this; I can see how my posts are quite frustrating from your perspective. (I was actually expecting a response of: it’s equiv to running esbuild w/ the following flags, and was not prepared for this line of questioning.)

8

If you’re really after milliseconds, I would suggest to use the development version of js_of_ocaml-compiler as there were many changes done recently in that area (some triggered by your posts). Also, as mentioned in other posts, you should probably collect data before trying to optimize things. I suspect jsoo link is the fastest step of all ,most of the time. The jsoo compiler flag --debug times can be very useful when chasing speed improvement.

2 Likes
10

Okay, I’m definitely wrong. jsoo link is near instantaneous.

11

Have you tried to run your dune build with --trace-file to see where the time is spent and what is actually slow?

12

Thanks, was not aware --trace-file existed. What tool do you recommend for viewing the output ?

13

One remark: I don’t think you should delete your posts, you’re just hidden context and it makes it harder to follow the discussion. One just see Okay, I’m definitely wrong but one no longer see what you’re referring to

Trying to compile js_of_ocaml itself to JS. The compiler is about 30k loc.

  1. senario: edit a file in compiler/bin_js-of_ocaml/
    • running ocamldep :10ms
    • recompiling the ml file to cmo : 35ms
    • recompiling the cmo to javascript : 41ms
    • comping jsoo link : 125ms
  2. senario: edit a file in compiler/lib
    In addition to the previous steps we now have theses two extra steps.
    • recompiling cma: 21ms
    • recompiling cma to js: 1700ms

Recompiling the cma.js file is quite slow, but it’s a big library. We could change the granularity of incrementality in dune and work at the level of cmo for libraries in the work tree.

14

I may be wrong – I don’t think I have deleted any posts that anyone has replied to. I have deleted posts where (1) I post a question, (2) I figure out the answer before anyone replies, and (3) I delete the post.

For this particular thread, the post I deleted to talks about unquantitative experiments with Rust, tsc, rescript, esbuild, jsoo. The post “Okay, I’m definitely wrong” refers to my earlier implied claim of jsoo link being the bottleneck.

15

about:tracing in chrome or https://magic-trace.org/ or https://ui.perfetto.dev/

16

Would you mind posting the make/model of your CPU ? I’m genuinely wondering if your hardware is 2-4x faster than mine. I’m using Intel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GHz

17

Sorry to slightly derail this thread, but is there some documentation about the various --debug flags for JSOO? The manual doesn’t seem to mention them, and running js_of_ocaml --help shows that --debug is an option but doesn’t list the arguments it can take.

18

Would you mind posting the make/model of your CPU ? I’m genuinely wondering if your hardware is 2-4x faster than mine. I’m using Intel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GHz

11th Gen Intel(R) Core™ i7-1185G7 @ 3.00GHz

I run my test with the dev version of js_of_ocaml and sourcemap disabled.

19

It’s not documented but you can get a list of possible values by passing an invalid one.

$ js_of_ocaml --debug random fake
js_of_ocaml: option '--debug': invalid element in list ('random'): invalid
             value 'random', expected one of 'link', 'mark-runtime-gen',
             'linker', 'main', 'lifting', 'shortvar', 'output', 'gen_tc',
             'gen', 'effects', 'deadcode', 'sourcemap', 'parser',
             'global-flow', 'flow', 'times' or 'invariant'
Usage: js_of_ocaml [COMMAND] …
Try 'js_of_ocaml --help' for more information.
1 Like
20

Would you be able to share the numbers you’re seing on your side ?

21

I never replicated your experiment. I added up your numbers from step 1 ~ 200ms; and that is insanely fast compared to my best config for time dune build ~ 600ms on a much smaller, from scratch project. [Sorry, not very scientific.]