Word diff in longer test_eq strings

Is there a way to make it easier to see the exact difference between two longer strings, when using inline tests?

[%test_eq: string] long_string second_long_string

Since the test run just spits out both strings, it’s hard to inspect what’s exactly different.

Maybe some way to run Unix diff on the two strings? Preferably without writing them to files first… :face_with_spiral_eyes:

The patdiff package includes a library called Expect_test_patdiff which we use at Jane Street to print diffs in tests. This is usually used in conjunction with let%expect_test-style tests, hence the name.

3 Likes

It should actually be pretty easy to implement a test assertion function that prints the result to a snapshot file and outputs the git diff if the result differs from the existing snapshot. It’s hard to beat the readability of a git diff especially if you take some care to preserve colours when working in the terminal and discard them when working in CI (and assuming the output is rendered with appropriate line breaks).

I’ve actually implemented this in a (Scala) project at work and it works really nicely. Of course the caveat is that you have to require git to be available on the machine. But that shouldn’t be too difficult nowadays.

EDIT: I just ported over my implementation from Scala to OCaml pretty much unchanged except generalizing it to use OCaml value formatting capabilities:

exception Snapshot_error of string

let gulp path =
  let inc = open_in path in
  let finally () = close_in inc in
  Fun.protect ~finally begin fun () ->
    let len = in_channel_length inc in
    really_input_string inc len
  end

let write_snapshot path contents =
  let outc = open_out path in
  let finally () = close_out outc in
  Fun.protect ~finally (fun () -> output_string outc contents)

let fail_snapshot path contents =
  write_snapshot path contents;
  print_endline "\nSnapshot test failed:";
  ignore (Unix.system ("git diff --color " ^ path));
  ignore (Unix.system ("git checkout -- " ^ path));
  print_endline "\nTo update, delete the snapshot file and rerun the test.";
  raise (Snapshot_error path)

let assert_snapshot path pp new_val =
  let snapshot_exists = Sys.file_exists path in
  let is_ci = Sys.getenv_opt "CI" in
  let new_str = Format.asprintf "%a\n" pp new_val in
  match snapshot_exists, is_ci with
  | false, Some "true" -> raise (Snapshot_error path)
  | false, _ -> write_snapshot path new_str
  | true, _ -> if gulp path <> new_str then fail_snapshot path new_str

(* Usage: assert_snapshot "path/to/file.snapshot" pp_int (1 + 1) *)

2 Likes

Wow, thanks guys! Will look at this later. :heart: