This is something I’ve been thinking of recently after starting to use Dune’s instrumentation a bit more, and I’m wondering if it exists or if anyone else would be interested.
I’d like a library which allows me to insert logging (by which I mean outputs strings of my choice + some information on the file/line it logged from, not necessarily different levels) which is completely removed at compile time unless I build with --instrument-with this-cool-logging-lib. An even nicer feature would be if it took a page out of landmarks and had an environment variable to select different “categories” of logging (so I could enable it for only some modules, for example, not necessarily severity levels).
By “completely removed” I am asking for there to not even be no-op function calls left over. In my current codebase I do this sort of debug logging using some runtime flags, and believe it or not the calls of debug_log which just return unit are starting to show up appreciably in my profiler.
Does this exist? If it doesn’t, would anyone else be interested in it? I’ve not really dabbled into writing PPXs myself but this seems like a reasonable way to get started.
There’s a slogan in some parts of the commercial software world: “we debug what we ship”. It means that the product release already contains all debug hooks, including logging, and they simply need to be turned-on by configuration. Similarly, it’s well-documented that Google’s systems all run with all debug code compiled-in.
You might consider using that PPX to expand to code that checks various debug variables in order to decide whether to even invoke logging functions. As an example of this, Google’s “glog” on various languages supports this. A different way of thinking about it is: you want to minimize the cost of not logging a line. The lower that cost is, the more inactive log-lines you can afford to strew throughout your program, waiting for the need to arise to turn them on.
Also, you might want to look at Google’s Dapper for inspiration for logging. There, logging can be enabled based on a binary condition of a log-line being conditionally-enabled, and then the current “transaction” being enabled for logging. This amounts to a flag in thread-local state being enabled. It allows pervasive and low-cost tracing to be used for performance-analysis, among other things. I’ve done this sort of thing in Java J2EE (spit) applications, and it can be quite effectiive for finding performanace and correctness problems.
While I agree with generally keeping no-ops around that can be enabled through configuration, I am still curious if there exist simple ways to include/exclude branches through compilation flags. When I worked in embedded systems, we couldn’t even fit debug builds on our production hardware and had to use specialized hardware (meaning other small differences) to enable them. So I think there’s still a place for compile-time exclusion of debug code, personally.
This is not supported at the OCaml language level. But there are a couple of preprocessors out there that behave like the C preprocessor #ifdef. So maybe possible. But again, if binary size is that much of an issue you would probably be using C in the first place
The motive isn’t binary size, it’s avoiding (possibly 10s of thousands) of calls to a function that looks like let debug_log x = if !debug then print_endline x else (), where x could be some not-cheap-to-evaluate string.
Using a PPX to move this if to the outside has some benefits, like avoiding the need to evaluate the argument, but still has a massive number of branches which will always either be all true or all false.
I’ve been dabbling in doing this today and it doesn’t seem like it’s too difficult to do in principle, but there are a lot of extra design considerations as @Chet_Murthy alludes to
Yeah, this is exactly the issue I ran into trying this myself. I have a mostly-working implementation, but in the mode where the calls are compiled out it never checks that you’re still passing strings and not who-knows-what.
Yeah, using closures is definitely a reasonable approach. I know there’s also a ppx floating around which lets you skip writing the fun () -> ....
The reason I am not already using Logs is I have my logging divided not by severity level (debug, info, warning, etc) but by portion of the program.
ppx_import runs from within the compiler process, using the -ppx flag directly. So it does not need particular arguments, it uses the compilation environment.
Wait … really? I thought PPX rewriters ran as an outboard process, invoked by the compiler, but still, a separate process ? Does the compiler pass along the -I arguments to the rewriter ?
Ppx rewriters invoked by the compiler get as input an AST that starts with an attribute describing some of the compilation context. The -I arguments in particular are added to the compiler’s load path, which is then transmitted to the ppx through this attribute.
The intended and historical way of using ppx rewriter is by passing the -ppx argument when compiling.
It is possible, instead, to create a separate process that will parse, rewrite (possibly several passes) and emit a marshalled AST.
This second approach is not possible for ppx_import since it requires access to the typing information of dependencies, and that is not available if you’re only parsing source code.
