Can compiled code automatically cache computation results?

Learning
1

I have a program in which I had originally computed the same sum of a block of 1000 floats 50,000 times, when I only needed to do sum those 1000 floats once. And I did that whole thing 2000 times, i.e. for 2000 blocks of 1000 floats. (I’m performing the sums on Owl matrices using either Owl’s sum or sum_cols functions.)

Now the code only computes the 2000 sums once, but it’s not any faster. On the other hand, if I only compute half of the sums, the program is 30% faster, so it’s not that the cost of the sums is so small that its effect on the run time is insignificant.

Is it possible that in the first case, with the redundant computations, ocamlopt recognizes that I’m doing the same thing repeatedly, and compiles to code that only performs the sum the first time, caching the result and using it in all of the subsequent cases?

2

ocamlopt will dump its intermediate representations if you ask with the -d command line options (eg, -dcmm), and this is probably the most straightforward way to tell whether the compiler is or is not performing some transformation. Of course, you are then faced with the problem of understanding the output.

3

It’s pretty much impossible. ocamlopt will have no idea what Owl operations are or if they have side effects, and therefore can’t eliminate them.

Going through some possibilities:

  1. Are you sure you’re measuring correctly, or the correct code is triggering when you expect it to?
  2. Are you producing the correct results with every method?
  3. Caching in the memory hierarchy could be the answer here: depending on the usage pattern, you could be accessing the same cached data repeatedly, making the actual computational cost negligible in comparison.
4

Thanks @bluddy, @gsg.

I see. Yes, that makes sense.

Yes. I’m comparing with output from the earlier, non-optimized commit.
I did find out that it will run faster when I completely leave out some computations. :slight_smile: Unfortunately, the output differs.

Yeah–good question. I believe so, but I will have check more closely. It’s always possible that I’m overlooking or misunderstanding something. Wouldn’t be the first time.

Hmm. Not something I know much about. I’ll think about that.

5

@gsg, tried that. Fascinating. Nasty stuff, but the line numbers are very helpful. Interesting.

I had thought that bringing sexps into the OCaml world was a weird Jane Street perversion due to Lisp influence. :slight_smile: Now I see that sexps have been buried deep inside OCaml all along.