Memory profiling in 5.x

Learning
1

Is there any way to profile the memory allocation of an OCaml 5.x program? I figured out how to use perf to profile the time spent by different function calls, but perf mem seems to only profile memory access rather than allocation. I want to know which of my functions are allocating the most memory. memtrace apparently doesn’t support multicore.

1 Like
2

We have a statistical memory profiler, known as statmemprof, to monitor allocation. Until recently it only worked on OCaml 4.14 and older, but it has recently been resurrected by @NickBarnes in trunk and support will resume from the next release (5.3) in ~6 months.

So you can either use it on 4.14, or use another mechanism known as runtime events which also allows you to monitor a non-small amount of allocation events and counters using handy tools such as olly.

3

Thanks! I can’t use 4.14 as my code uses effects. Can you explain how to use the other feature to tell me which of my functions are allocating the most memory?

4

You can visualize a profile of the allocations and GC over time but unfortunately I don’t think this easily integrates with a call stack visualizer. But I have hardly used it, so maybe I’m wrong here.

One make-do solution would be for you to manually emit custom events (see manual) at the beginning and end of some functions of interest, and try to close in on the big allocating functions this way.

5

You could also build on a 5.2.0+statmemprof switch (available with opam), but from what I’m being told, making it work also requires using a forked version of memtrace with Support 5.3 statmemprof by tmcgilchrist · Pull Request #22 · janestreet/memtrace · GitHub and GitHub - dra27/memtrace at 5_1_statmemprof which haven’t been merged yet.

6

Ok, thanks. That might be more than I’m up for at the moment, but I’ll see if I’m inspired to try it out.

7

Here’s a back ported version of statmemprof for OCaml 5.2 that you can setup in an opam switch.

$ opam switch create 5.2.0+statmemprof --repos=dra27=git+https://github.com/dra27/opam-repository#5.2.0+statmemprof ocaml-variants.5.2.0+statmemprof
$ opam pin add -n memtrace git+https://github.com/dra27/memtrace#5_1_statmemprof
3 Likes