How to measure the memory occupied by a given data structure?

Learning
1

Hello,

Supposed I have a function ‘Toto.create ()’ which creates my data structure of choice.

let x = Toto.create () in

How do I precisely measure the amount of memory occupied by x?

All my trials look like they return complete nonsense.

Thanks!

2 Likes
2

It seems that the delta in Gc.stat.minor_words would be the thing (though not in bytes), but I’m not sure this would always be true.

3

You could try using Obj.reachable_words:

Computes the total size (in words, including the headers) of all heap blocks accessible from the argument. Statically allocated blocks are excluded.
@Since 4.04

Obj.reachable_words takes input type Obj.t, so for example:

utop # Obj.reachable_words (Obj.repr (1, 2, 3));;
- : int = 4
1 Like
4

I am trying, it looks super slow …

5

spacetime should be an other solution - it provides a useful graph to have a global and a more specific view of your allocation. However, you probably need a powerful computer to compile switch.

6

Isn’t there a quick and dirty way?
I’m ok if the measurement is not super precise, I just want to have an order of idea.
I was thinking something using Gc stats should be possible.

7

At first sight, threads and signals would mess with the minor heap in hard to predict ways.
Then, it would work only if Toto.create does not allocate anything at all but the data that will eventually be reachable from the result (which seams unlikely), and that it does only allocate on ocaml heap, and that it does not trigger the Gc for any other reason.
Also, it would help to enlarge the default minor heap size (OCAMLRUNPARAM=s=100M ?) and to call Gc.minor first.
If all the above conditions are met then I would also expect this to work, including in native code compiled programs.

But to met all those conditions is hard. It took me a while to find a suitable exemple in the program I’m currently working on, and I eventually settled on a mere Hashtbl.create:

let some_empty_hash =
  Gc.minor ();
  let before = Gc.minor_words () in
  let x = ref (Hashtbl.create 11) in
  let after = Gc.minor_words () in
  Printf.printf "Size of Hashtbl(11) = %g - %g = %g\n%!" after before (after -. before) ;
  x

This convincingly printed:

Size of Hashtbl(11) = 205878 - 205852 = 26
1 Like
8

Thanks! I’ll try that.

9

Finally, I opted for:

/usr/bin/time -f '%M' un_marshal_from_file.exe my_file

where <my_file> only contains a dump (obtained via Marshal.to_channel) of my data structure.

10

Wasn’t the result then close to the size of the marshalled file itself?