[image] Need feedback on my basic unit test tedious and brittle workarounds to deal with path, encoding, and line ending normalization. Can you elaborate on this? What is brittle here?

paths and line endings :slight_smile: of course, devil is in the detail of how you are producing the strings. But the core principle, in my view, is just that the meaning of a syntactically valid program in a “type-correct” language should never depend upon the particular representation used to im…

Can you expand what is brittle in practical terms? I am having a difficult time understanding what you mean specifically. Why are snapshot or expect tests brittle because of paths and line endings? But the core principle, in my view Does this override every other consideration, like eg the conve…

[image] yawaramin: Why are snapshot or expect tests brittle because of paths and line endings Because paths and line endings are represented differently between POSIX and Windows. So if you have a test that converts data that contains, e.g., an Fpath.t to a string (say as part of producing JS…

Agreed, path representations can be different on Windows. But isn’t this rather easy to work around? Windows understands the forward-slash as a path separator character (for a long time now), I don’t see any reason to use backslash on Windows. And printing an absolute path as part of a test would be…

I just had my lunch and was reflecting on the exchange here, and I’m afraid I fell into a bit of a dogmatic position. I don’t think there are absolutes here, or one way that is best under all circumstances. Snapshots certainly have their uses. E.g., if you have an essentially invariant bijection be…

For the cases where resorting to strings has its shortcoming, there are a few utils I have seen in this expect-test-helpers library allowing you to mix and match styles (they’re called require_*, and print a failure message if the condition does not hold). One workflow this permits is to output CR …

[image] shonfeder: If the approach to testing is that you generate particular string representations of your data just to serve the needs of reliable tests Isn’t that just a debug pretty-printer in OCaml land? Eg what you would get with the Fmt.Dump module. It’s useful not just for testing bu…

[image] mbarbin: For the cases where resorting to strings has its shortcoming, there are a few utils I have seen in this expect-test-helpers library These seem quite useful. IIUC, this approach takes the view that you can start with expect-style testing, but then reinforce this with tests tha…

FWIW personally I don’t equate snapshot testing with comparing strings, you can compare arbitrary OCaml values. Of course you are still comparing some form of representation because you need to store a reference value, but then that doesn’t have to be a string, you can use richer, but literal repres…

This sounds like a very nice approach! Snapshots without the stringy stuffy and decent data diffing sounds great. Glad to learn about your testing library!

Brittleness of snapshot tests

Learning

Topic		Replies	Views
Need feedback on my basic unit test Learning ppx , menhir , dune	11	318	May 27, 2025
Word diff in longer test_eq strings Community testing , ppx_inline_test	17	1314	January 22, 2023
How to set up unit testing in 2023 Learning unit-test	36	4290	October 9, 2024
What functions deserve tests? Learning language-agnostic	29	1015	October 21, 2023
Build systems and snapshot testing support Community build , testing	20	410	March 31, 2025

Brittleness of snapshot tests

Related topics