# Benchmarking bucklescript vs. js\_of\_ocaml **URL:** \ **Category:** Learning\ **Tags:** js\_of\_ocaml, bucklescript, jsoo, performance\ **Created:** [January 16, 2019, 3:04am UTC](https://discuss.ocaml.org/t/benchmarking-bucklescript-vs-js-of-ocaml/3191 "2019-01-16T03:04:44Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![cemerick](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ocaml.org/cemerick/32/1383_2.png) [@cemerick](https://discuss.ocaml.org/u/cemerick)\ **Post date:** [January 16, 2019, 3:04am UTC](https://discuss.ocaml.org/t/benchmarking-bucklescript-vs-js-of-ocaml/3191/1 "2019-01-16T03:04:44Z")
I have been tinkering with OCaml, bucklescript, and js\_of\_ocaml. I have a microbenchmark that I’ve been implementing for various languages over the last month or so as I sample them; implementing it and running the results through BS and JSOO yielded surprising (at least for me) results: ``` === bucklescript (reason) === 9.23s === js_of_ocaml === 57.49s ``` I have some understanding of each compiler’s respective design constraints, so I was expecting that ordering, but not a margin that large. All of the source and compiler invocations, etc are available in [https://github.com/cemerick/silly-shootout](https://github.com/cemerick/silly-shootout), so if anyone sees any place where I’m making an egregious error, I’d appreciate that feedback. Thanks very much 😃 — Chas ---
**Author:** ![tiensonqin](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ocaml.org/tiensonqin/32/74_2.png) [@tiensonqin](https://discuss.ocaml.org/u/tiensonqin)\ **Post date:** [January 16, 2019, 11:37am UTC](https://discuss.ocaml.org/t/benchmarking-bucklescript-vs-js-of-ocaml/3191/2 "2019-01-16T11:37:08Z")
It might be polymorphic compare, In Bucklescript `min` and `max` will compile to type-specific functions even javascript primitive functions. ---
**Author:** ![cemerick](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ocaml.org/cemerick/32/1383_2.png) [@cemerick](https://discuss.ocaml.org/u/cemerick)\ **Post date:** [January 16, 2019, 4:52pm UTC](https://discuss.ocaml.org/t/benchmarking-bucklescript-vs-js-of-ocaml/3191/3 "2019-01-16T16:52:26Z")
Good suggestion! Using local implementations of `min` and `max` that call out to js directly: ```auto module Unsafe = Js_of_ocaml.Js.Unsafe;; let jsmin a b = Unsafe.fun_call (Unsafe.js_expr "Math.min") [|Unsafe.inject a; Unsafe.inject b|] let jsmax a b = Unsafe.fun_call (Unsafe.js_expr "Math.max") [|Unsafe.inject a; Unsafe.inject b|] ``` cuts the JSOO-emitted code’s runtime from 58s to 44s. A good improvement, but obviously there’s more going on. ---
**Author:** ![cemerick](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ocaml.org/cemerick/32/1383_2.png) [@cemerick](https://discuss.ocaml.org/u/cemerick)\ **Post date:** [January 16, 2019, 4:54pm UTC](https://discuss.ocaml.org/t/benchmarking-bucklescript-vs-js-of-ocaml/3191/4 "2019-01-16T16:54:30Z")
BTW, regarding BuckleScript: it does provide it’s own `min` and `max` impls, but they’re no slower than calling out to `Math.min`/`max`. It turns out that its runtime representation of records is responsible for the entirety of the margin between its generated code and the “native” JS implementation I have in that repo. Details here: [https://github.com/BuckleScript/bucklescript/issues/2922#issuecomment-454683001](https://github.com/BuckleScript/bucklescript/issues/2922#issuecomment-454683001) ---
**Author:** ![copy](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ocaml.org/copy/32/480_2.png) [@copy](https://discuss.ocaml.org/u/copy)\ **Post date:** [January 16, 2019, 5:01pm UTC](https://discuss.ocaml.org/t/benchmarking-bucklescript-vs-js-of-ocaml/3191/5 "2019-01-16T17:01:14Z")
I played around with the generated JS code a little bit and found that optimising the `ref` into a variable also makes a big difference. ```diff 1257,1258c1257 < A = 0, < B = J[1]; --- > A = 0; 1260a1260 > B = J[1], 1267c1267 < B = [au, V(aP, 20), aS, aR, aQ]; --- > J[1] = [au, V(aP, 20), aS, aR, aQ]; 1275d1274 < J[1] = B; ``` Timings on my machine: Initial: 45.408s With non-polymorphic min/max: 34.080s Optimised ref: 33.742s Both: 7.317s And, for reference, bucklescript: 6.885s And lastly, a functional version: ```auto let rec f i rect = let rect = union rect (r 20. 0. 100. (float_of_int i)) in if i = lim then rect else f (i+1) rect ``` Functional with non-polymorphic min/max: 7.208s ---
**Author:** ![cemerick](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ocaml.org/cemerick/32/1383_2.png) [@cemerick](https://discuss.ocaml.org/u/cemerick)\ **Post date:** [January 16, 2019, 5:22pm UTC](https://discuss.ocaml.org/t/benchmarking-bucklescript-vs-js-of-ocaml/3191/6 "2019-01-16T17:22:52Z")
Ech, of course this is just fine for a non-polymorphic compare, and yields the same results as the “unsafe” JS calls. ```auto let min (a: float) (b: float) : float = if a < b then a else b let max (a: float) (b: float) : float = if a > b then a else b ``` ---
**Author:** ![cemerick](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ocaml.org/cemerick/32/1383_2.png) [@cemerick](https://discuss.ocaml.org/u/cemerick)\ **Post date:** [January 16, 2019, 5:25pm UTC](https://discuss.ocaml.org/t/benchmarking-bucklescript-vs-js-of-ocaml/3191/7 "2019-01-16T17:25:16Z")
You’re right, cutting out the ref (via the recursive reduction) + monomorphic min/max gets very close to the bucklescript level, down to 12.5s on my machine. Good catch! ---
**Author:** ![cemerick](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ocaml.org/cemerick/32/1383_2.png) [@cemerick](https://discuss.ocaml.org/u/cemerick)\ **Post date:** [January 16, 2019, 5:46pm UTC](https://discuss.ocaml.org/t/benchmarking-bucklescript-vs-js-of-ocaml/3191/8 "2019-01-16T17:46:29Z")
Eliminating the `ref` is an obvious improvement, but having to dodge mutability to get solid performance isn’t great. (A great deal of the codebase I’m looking to port _is_ effectively pure, but it does sit on top of a core graph data structure that uses mutation heavily, especially when being initially loaded.) I thought maybe a mutable “box” record would get the same job done, but it actually had exactly the same performance as the `ref`: ```auto type 't box = {mutable v: 't} let _ = let rect = {v = r 25. 25. 200. 200.} in for i = 0 to lim do rect.v <- union rect.v @@ r 20. 0. 100. (float_of_int i) done; print_endline @@ string_of_float @@ rect.v.ty ``` Are mutable record fields actually `ref`s underneath? ---
**Author:** ![octachron](https://avatars.discourse-cdn.com/v4/letter/o/49beb7/32.png) [@octachron](https://discuss.ocaml.org/u/octachron)\ **Post date:** [January 16, 2019, 5:53pm UTC](https://discuss.ocaml.org/t/benchmarking-bucklescript-vs-js-of-ocaml/3191/9 "2019-01-16T17:53:19Z")
> [@cemerick](#): > > Are mutable record fields actually `ref` s underneath? The reverse is true: references are single mutable field records: [http://caml.inria.fr/pub/docs/manual-ocaml/libref/Pervasives.html#1\_References](http://caml.inria.fr/pub/docs/manual-ocaml/libref/Pervasives.html#1_References) ---
**Author:** ![cemerick](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ocaml.org/cemerick/32/1383_2.png) [@cemerick](https://discuss.ocaml.org/u/cemerick)\ **Post date:** [January 16, 2019, 6:45pm UTC](https://discuss.ocaml.org/t/benchmarking-bucklescript-vs-js-of-ocaml/3191/10 "2019-01-16T18:45:30Z")
Ah, appreciate the pointer. Somehow I was under the impression that refs and their operators were language primitives instead of part of the stdlib. 🤔 So with monomorphic min/max and the immutable recursive reduction, things are down to: ```auto === bucklescript (reason) === 8.87s === js_of_ocaml === 12.53s ``` That’s a big improvement for JSOO, but the tradeoff is unfortunate. Polymorphic compare can be worked around; the representation of mutable fields, not so much. I’ve thus pushed an update to the benchmark that includes the former, but sticks with the ref. Maybe `Unsafe` could be used to eliminate that overhead for the hottest spots… ---
**Author:** ![lpw25](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ocaml.org/lpw25/32/144_2.png) [@lpw25](https://discuss.ocaml.org/u/lpw25)\ **Post date:** [January 16, 2019, 7:11pm UTC](https://discuss.ocaml.org/t/benchmarking-bucklescript-vs-js-of-ocaml/3191/11 "2019-01-16T19:11:11Z")
Have you tried turning off `-g` for the js\_of\_ocaml case? IIRC `-g` turns off `ref` optimisation in bytecode and since js\_of\_ocaml works off the bytecode that’s going to have a negative affect. Since Bucklescript uses a modified compiler it might avoid that issue. ---
**Author:** ![copy](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ocaml.org/copy/32/480_2.png) [@copy](https://discuss.ocaml.org/u/copy)\ **Post date:** [January 16, 2019, 7:12pm UTC](https://discuss.ocaml.org/t/benchmarking-bucklescript-vs-js-of-ocaml/3191/12 "2019-01-16T19:12:31Z")
Note that this optimisation only applies to `ref`s (records with a single field). For the rather silly redefinition of `ref` below, jsoo again has a similar performance profile to bucklescript. ```ocaml type 'a ref = { mutable contents: 'a; bloat: unit } let ref x = { contents = x; bloat = () } let (!) { contents; _ } = contents let (:=) r x = r.contents <- x ``` ---
**Author:** ![cemerick](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ocaml.org/cemerick/32/1383_2.png) [@cemerick](https://discuss.ocaml.org/u/cemerick)\ **Post date:** [January 16, 2019, 8:23pm UTC](https://discuss.ocaml.org/t/benchmarking-bucklescript-vs-js-of-ocaml/3191/13 "2019-01-16T20:23:00Z")
I’m not using a build tool, so I can know/control exactly what’s being done. I’m not passing `-g`, so I presume debugging isn’t enabled (I don’t see any `-no-g` or similar, so I assume that’s the default). I’m compiling using this invocation: ```auto ocamlc bench.ml && js_of_ocaml --opt=3 a.out ``` Do let me know if I should be using something different. ---
**Author:** ![cemerick](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ocaml.org/cemerick/32/1383_2.png) [@cemerick](https://discuss.ocaml.org/u/cemerick)\ **Post date:** [January 16, 2019, 8:29pm UTC](https://discuss.ocaml.org/t/benchmarking-bucklescript-vs-js-of-ocaml/3191/14 "2019-01-16T20:29:48Z")
To clarify, you’re seeing this redefinition of `ref` et al. yield a significant improvement in JSOO output’s performance? I’ve not been able to replicate so far. ---
**Author:** ![lpw25](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ocaml.org/lpw25/32/144_2.png) [@lpw25](https://discuss.ocaml.org/u/lpw25)\ **Post date:** [January 16, 2019, 8:58pm UTC](https://discuss.ocaml.org/t/benchmarking-bucklescript-vs-js-of-ocaml/3191/15 "2019-01-16T20:58:19Z")
I found the issue: the uses of `@@` are preventing the optimisation. If you remove them then things should improve. ---
**Author:** ![copy](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ocaml.org/copy/32/480_2.png) [@copy](https://discuss.ocaml.org/u/copy)\ **Post date:** [January 16, 2019, 9:28pm UTC](https://discuss.ocaml.org/t/benchmarking-bucklescript-vs-js-of-ocaml/3191/16 "2019-01-16T21:28:43Z")
No, I was just trying to show that jsoo and bucklescript have a comparable performance when it comes to records with more than one field. ---
**Author:** ![Yaron\_Minsky](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ocaml.org/yaron_minsky/32/11_2.png) [@Yaron\_Minsky](https://discuss.ocaml.org/u/Yaron_Minsky)\ **Post date:** [January 16, 2019, 10:00pm UTC](https://discuss.ocaml.org/t/benchmarking-bucklescript-vs-js-of-ocaml/3191/17 "2019-01-16T22:00:25Z")
Ugh. Why does @@ lead to de-optimization? ---
**Author:** ![cemerick](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ocaml.org/cemerick/32/1383_2.png) [@cemerick](https://discuss.ocaml.org/u/cemerick)\ **Post date:** [January 16, 2019, 10:06pm UTC](https://discuss.ocaml.org/t/benchmarking-bucklescript-vs-js-of-ocaml/3191/18 "2019-01-16T22:06:11Z")
Thank you! I’ve pushed a change based on this suggestion, and the timings are now quite tight: ```auto === bucklescript (reason) === 9.98s === js_of_ocaml === 12.90s ``` The change also took ~20% off the runtime of the `ocamlopt.opt` output. A few interesting things determined experimentally: 1. Either composition operator (`@@` or `|>`) can trigger the deoptimization. 2. The only usage that was impactful was where I had used `@@` in declaring the binding of the `ref`. Using `@@` or `|>` anywhere else had no negative consequences. All of this is _very_ surprising, at least to me?: 1. In the languages I’ve used that have operators like this (e.g. `$` and co. in haskell, `->` and co. in Clojure et al., etc), they are fundamentally syntactic affordances. Clearly something more is going on here. 2. In cases where all of the arguments to a function are provided (like here, e.g. `let foo = ref @@ expressions...`), shouldn’t composition operators simply vanish? 3. To that point, [a post coinciding with the 4.0 release](https://www.ocamlpro.com/2012/08/20/ocamlpros-contributions-to-ocaml-4-00-0/) talked about implementing exactly that optimization, but apparently that is no longer the case? ---
**Author:** ![lpw25](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ocaml.org/lpw25/32/144_2.png) [@lpw25](https://discuss.ocaml.org/u/lpw25)\ **Post date:** [January 16, 2019, 10:07pm UTC](https://discuss.ocaml.org/t/benchmarking-bucklescript-vs-js-of-ocaml/3191/19 "2019-01-16T22:07:35Z")
The ref optimisation is very simple and syntactic. It would need to either detect `@@` as a special case or be done after inlining to catch this case. In flambda it is done after inlining so it should work fine in this case, but of course you can’t use flambda with bytecode/js\_of\_ocaml. ---
**Author:** ![cemerick](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ocaml.org/cemerick/32/1383_2.png) [@cemerick](https://discuss.ocaml.org/u/cemerick)\ **Post date:** [January 16, 2019, 10:10pm UTC](https://discuss.ocaml.org/t/benchmarking-bucklescript-vs-js-of-ocaml/3191/20 "2019-01-16T22:10:39Z")
FWIW, I am using `+fp+flambda`, and the `ocamlopt.opt` timings _were_ impacted by the composition operators, 5.15s vs. 6.32s. ¯\_(ツ)_/¯ [Next page](https://discuss.ocaml.org/t/benchmarking-bucklescript-vs-js-of-ocaml/3191.md?page=2)