Thread-global tags in Async.Log?

In some cases, we want to tag every log message with things like a process ID, a request ID, request path, user name, etc. I was hoping I could do this by setting thread-global tags with Scheduler.with_local, then looking them up inside my custom Output. Unfortunately, it seems like outputs run in a different Async context so they can’t find the tags.

Note: I can’t set normal globals because we handle requests concurrently.

Is what I’m trying to do possible?

@BrendanLong I solved this problem by also rebinding the Log.info|debug|error|sexp functions and adding tags corresponding to the values I stored in Scheduler.with_local. Eg.

(** Customer logger module using Async.Logger.Global as the internal implementation *)
module Log = struct
    let request_id =
      Univ_map.Key.create ~name:"request_id" sexp_of_string

    (** Invoke this function at the top level handler of your request *)
    let with_request_id (id:string) f =
      Scheduler.with_local request_id (Some id) ~f

    let with_request_id_tag tags =
      Option.(
        (map
           (Scheduler.find_local request_id) ~f:
           (fun request_id -> "request_id", request_id) |>
         to_list
        ) @ value ~default:[] tags
      )

    (** Add the request id tag always to logger *)
    let info ?tags =
      let tags = with_request_id_tag tags in Log.Global.info ~tags

   (* Do the same let binding override for debug, error, raw, sexp, etc... here *)
end
3 Likes

That’s basically what we do for our internal libraries, but I was hoping to find a way to do this where logging in other libraries would also have our tags (for example, if our database library logged an error, we’d like to be able to associate it with the request/user/etc. that caused the error).

@BrendanLong yeah I have similar concerns- although it’s a bit heavy, I was thinking that the database library (pgx) could provide a functor over the logger module that the dependent library must instantiate. It would be fairly similar to how such libraries functorize over lwt vs. async, for instance.

The other more universal way I think this could work is if the jane street logger itself is modified to copy the contents of the universal map (or the execution context itself) into the Log.Message.t when push_update is called.

Ah that’s a good idea. I’ll have to check if they’re interested in adding that sometime.

EDIT: I opened an issue in Async to see if they’re interested.

Yeah, I thinking of doing this, or even accepting a first-class module as an argument to connect. I’m a little worried about over-functorizing.

Another thing I’m considering is making an Async.Log backend for Logs so I can use the same log library no matter what the threading implementation is, although I’m still trying to get my head around how Logs works.

1 Like