# Debugging memory issues **URL:** \ **Category:** Learning\ **Created:** [January 24, 2019, 1:52am UTC](https://discuss.ocaml.org/t/debugging-memory-issues/3223 "2019-01-24T01:52:02Z")\ **Posts on this page:** 13\ **Page:** 1
**Author:** ![sean](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ocaml.org/sean/32/981_2.png) [@sean](https://discuss.ocaml.org/u/sean)\ **Post date:** [January 24, 2019, 1:52am UTC](https://discuss.ocaml.org/t/debugging-memory-issues/3223/1 "2019-01-24T01:52:03Z")
Hey all, I’m working on a [new compiler/transpiler](https://github.com/stan-dev/stanc3) written in OCaml and we’ve (unfortunately?) been using the Format library to emit the code that we’re generating. Today I’m just starting to run it on some huge test programs and finding that it’s taking up dozens of GB of memory pretty quickly and totally swamping the machine only a little ways into the program. I tried debugging with `ocamldebug` but I can’t find any specific code that’s repeating too much or anything like that - it just seems like something invisible is happening and my number one suspect is that it’s the building up of these long Format strings (with nested “%a” compositional formatting). But I’d like to figure out for sure so I don’t waste time switching away from the Format library or something if it’s an unrelated issue. Any help appreciated! Thanks 🙂 ---
**Author:** ![jfeser](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ocaml.org/jfeser/32/2093_2.png) [@jfeser](https://discuss.ocaml.org/u/jfeser)\ **Post date:** [January 24, 2019, 2:59am UTC](https://discuss.ocaml.org/t/debugging-memory-issues/3223/2 "2019-01-24T02:59:08Z")
You could give spacetime ([https://blog.janestreet.com/a-brief-trip-through-spacetime/](https://blog.janestreet.com/a-brief-trip-through-spacetime/)) a try. It’s a new ocaml memory profiler. You’ll need to install a spacetime-specific compiler switch though. ---
**Author:** ![c-cube](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ocaml.org/c-cube/32/1727_2.png) [@c-cube](https://discuss.ocaml.org/u/c-cube)\ **Post date:** [January 24, 2019, 4:33am UTC](https://discuss.ocaml.org/t/debugging-memory-issues/3223/3 "2019-01-24T04:33:58Z")
In my experience, `Format` doesn’t leak memory (if you use `fprintf` into some IO channel; if you print into a buffer then the buffer might grow, of course). It’s pretty fast and very convenient. I second the idea of using some form of profiling such as spacetime to find where the leak comes from. ---
**Author:** ![lindig](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ocaml.org/lindig/32/532_2.png) [@lindig](https://discuss.ocaml.org/u/lindig)\ **Post date:** [January 24, 2019, 8:07am UTC](https://discuss.ocaml.org/t/debugging-memory-issues/3223/4 "2019-01-24T08:07:05Z")
I have not used it myself, but `Obj.reachable_words` reports the words reachable from a given value. If you suspect some value already to be responsible for the leak, you could observe it with this function. ---
**Author:** ![chambart](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ocaml.org/chambart/32/769_2.png) [@chambart](https://discuss.ocaml.org/u/chambart)\ **Post date:** [January 24, 2019, 5:17pm UTC](https://discuss.ocaml.org/t/debugging-memory-issues/3223/5 "2019-01-24T17:17:55Z")
I have debugged quite a few things with [https://github.com/jhjourdan/ocaml/tree/memprof](https://github.com/jhjourdan/ocaml/tree/memprof) . It is available in the opam repository as ocaml-variants.4.07.1+statistical-memprof you should use it with [https://github.com/jhjourdan/statmemprof-emacs](https://github.com/jhjourdan/statmemprof-emacs) ---
**Author:** ![sean](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ocaml.org/sean/32/981_2.png) [@sean](https://discuss.ocaml.org/u/sean)\ **Post date:** [January 25, 2019, 9:58pm UTC](https://discuss.ocaml.org/t/debugging-memory-issues/3223/6 "2019-01-25T21:58:59Z")
Thank you all! I got spacetime to work (ish - I’m on a Mac) and was able to see that the memory leak is coming from the Format library, though I’m not sure exactly how I can avoid it: ```auto Time 1.314798 (1/1), Total 1429001264 live bytes [0x1085430f1;0x108542b47] 65.75% 939524144b buffer.ml:70,19--40 34.25% 489477120b bytes.ml:68,12--22 ``` Notably, when compiled with spacetime, it doesn’t allocate 20+GB of memory, it fails after a GB or two with an exception (to follow). I wonder if I’m just using the library wrong? Maybe there are rules about closing boxes in the same `fprintf` call or how to make multiple fprintf calls, one after another? ```auto Uncaught exception: (Invalid_argument Bytes.create) Raised by primitive operation at file "buffer.ml", line 70, characters 19-40 Called from file "buffer.ml", line 87, characters 34-46 Called from file "format.ml", line 343, characters 4-28 Called from file "format.ml", line 473, characters 6-72 Called from file "format.ml", line 480, characters 6-24 Called from file "format.ml", line 1197, characters 32-48 Called from file "format.ml", line 1187, characters 4-20 Called from file "format.ml", line 1199, characters 32-48 Called from file "format.ml", line 1248, characters 20-38 Called from file "lib/Stan_math_backend.ml", line 92, characters 2-129 ``` Here is the code on line 92 of my file (apologies in advance): ```auto let emit_for_loop ppf (loopvar, lower, upper, emit_body, body) = fprintf ppf "@[for (@[size_t %s = %a;@ %s < %a;@ %s++@])" loopvar emit_expr lower loopvar emit_expr upper loopvar ; fprintf ppf "@;<0 4>@[%a@]@]" emit_body body ``` ---
**Author:** ![c-cube](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ocaml.org/c-cube/32/1727_2.png) [@c-cube](https://discuss.ocaml.org/u/c-cube)\ **Post date:** [January 25, 2019, 10:51pm UTC](https://discuss.ocaml.org/t/debugging-memory-issues/3223/7 "2019-01-25T22:51:31Z")
Are you trying to allocate a huge (\> 1GB) text inside one buffer? Also check whether you actually flush the formatter sometimes (with `@.` for a newline, `@?` for a mere flush). It’s still weird that the buffer grows _that_ much. ---
**Author:** ![sean](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ocaml.org/sean/32/981_2.png) [@sean](https://discuss.ocaml.org/u/sean)\ **Post date:** [January 25, 2019, 11:28pm UTC](https://discuss.ocaml.org/t/debugging-memory-issues/3223/8 "2019-01-25T23:28:55Z")
Should be a couple hundred lines of ASCII, maybe a few KB max. How often is sometimes? We aren’t flushing at all now except to collect the resulting string from Format.str\_formatter. Doesn’t flushing close any open boxes? ---
**Author:** ![c-cube](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ocaml.org/c-cube/32/1727_2.png) [@c-cube](https://discuss.ocaml.org/u/c-cube)\ **Post date:** [January 25, 2019, 11:37pm UTC](https://discuss.ocaml.org/t/debugging-memory-issues/3223/9 "2019-01-25T23:37:44Z")
- I’d recommend not using `str_formatter`, global mutable state is generally a code smell. Try allocating a buffer and use `Format.formatter_of_buffer`, or even `Format.asprintf` directly - do you use `Buffer.contents` or `Format.flush_str_formatter`? The former would be incorrect since it wouldn’t clear the buffer. ---
**Author:** ![sean](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ocaml.org/sean/32/981_2.png) [@sean](https://discuss.ocaml.org/u/sean)\ **Post date:** [February 3, 2019, 9:07pm UTC](https://discuss.ocaml.org/t/debugging-memory-issues/3223/10 "2019-02-03T21:07:48Z")
Super weird. I think it was that we were using `Format.str_formatter` (just as an initial test that everything hooked up together correctly, I swear!). Using `Format.asprintf "%a" ` doesn’t show the same ridiculous memory consumption and crashing. Thank you so much! ---
**Author:** ![c-cube](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ocaml.org/c-cube/32/1727_2.png) [@c-cube](https://discuss.ocaml.org/u/c-cube)\ **Post date:** [February 3, 2019, 9:45pm UTC](https://discuss.ocaml.org/t/debugging-memory-issues/3223/11 "2019-02-03T21:45:49Z")
👌 perfect. Now for optimal ergonomics I’d suggest trying the `fmt` library, or the one in containers. ---
**Author:** ![sean](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ocaml.org/sean/32/981_2.png) [@sean](https://discuss.ocaml.org/u/sean)\ **Post date:** [February 4, 2019, 3:44pm UTC](https://discuss.ocaml.org/t/debugging-memory-issues/3223/12 "2019-02-04T15:44:51Z")
Thanks for the recommendation. What makes you like those APIs better? For example, we’re currently using the Format library’s sort of punctuation-based box API (`"@[%a@]"` etc). Is that frowned upon, or does `fmt` improve upon it? Is there additional functionality you would highlight for us to look into? We’re still getting a feel for ocaml and its ecosystem, so any advice appreciated! ---
**Author:** ![c-cube](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ocaml.org/c-cube/32/1727_2.png) [@c-cube](https://discuss.ocaml.org/u/c-cube)\ **Post date:** [February 4, 2019, 4:04pm UTC](https://discuss.ocaml.org/t/debugging-memory-issues/3223/13 "2019-02-04T16:04:05Z")
I personally use a lot of the string-based box API, because it’s compact and quite readable once you’re used to it. But `fmt` and `CCFormat` (in containers) provide APIs with more sensible names than `pp_print_string`, support for colors, and a lot of combinators for lists, options, arrays, etc. You can do fine without them but they’re simply quite convenient.