Closing Old Bugs With Tests: A Dune Contribution Story

Community
1

I recently started contributing to Dune with no prior familiarity with the code base. One thing I found rewarding was closing old issues — some filed as far back as 2018 — and I wanted to share the approach I used.

The Method: Close With a Test

Many open issues describe behaviour that has already been fixed, or behaviour that is working correctly but was never verified with a test. My approach: write a cram test that demonstrates the expected behaviour, link the PR to the issue with fixes #NNNN, and let the merge close the issue automatically.

This has several advantages over simply closing an issue with a comment like “this works now”:

  • It’s verifiable. Anyone can read the test and see exactly what behaviour is being asserted. There’s no argument about whether the issue is truly resolved.
  • It prevents regressions. If the behaviour breaks again in the future, the test fails. The issue doesn’t silently reopen — the CI catches it.
  • It grows the test suite. Every closed issue leaves behind a test that documents a real-world scenario someone cared about.
  • It respects the reporter. Someone took the time to file the issue. A test that captures their concern is a better acknowledgement than a drive-by close.

After a few weeks of contributing test PRs and bug fixes, I was given triager and then maintainer status on the project. This let me close issues directly when they were clearly resolved, and merge my own test PRs. Before that, the existing maintainers merged them on my behalf.

Honestly, I regret closing issues where I could have used this method but didn’t. The method is much superior to simply adding to a GitHub discussion about the bug and then closing the issue.

Issues I Helped Close

Here’s an accounting of what I helped close, grouped by what it took:

Confirmed resolved; closed directly

These issues described problems that no longer existed on main. In some cases I confirmed this myself, in others someone else had already verified it. As a triager/maintainer I was able to close them:

  • #1974 — @all target doesn’t interact well with (include_subdirs ...) (filed 2019)
  • #3173 — Can’t promote into a directory starting with underscore (filed 2020)
  • #3805 — No such file or directory when DUNE_BUILD_DIR is set during test (filed 2020)
  • #8242 — Very slow emacs compilation buffer updates (filed 2023)
  • #10360 — executables_implicit_empty_intf expects dune >= 2.9 but was only added in 3.0 (filed 2024)

Closed with a test PR

These issues were resolved by writing a test that demonstrates the correct behaviour. No code changes were needed — the test itself closed the issue:

  • #2370 — .h files copied to _build without foreign stubs (filed 2019, test: #14172)
  • #3362 — Cannot use %{lib:...} in the flags stanza (filed 2020, test: #14146)
  • #3484 — Promoting over a running binary (filed 2020, test: #14173)
  • #11110 — Libexec site install doesn’t set executable bit (filed 2024, test: #14171)

Closed with a test and a code fix

These required both a test and a code change to resolve:

  • #878 — dune subst adds duplicate version field in opam (filed 2018, fix: #14136)
  • #2445 — dune sends SIGKILL without prior SIGTERM (filed 2019, fixes: #14170, #14224)
  • #3916 — Duplicate bounds in autogenerated opam files (filed 2020, fix: #14175)
  • #6148 — root_module generates duplicate module definition (filed 2022, fix: #14135)
  • #9757 — dune runtest doesn’t execute byte-only inline tests (filed 2024, fix: #14174)
  • #10707 — Missing menhir lower bound in generated opam files (filed 2024, fix: #14168)
  • #11002 — opam file generation error-prone (filed 2024, fix: #14136)
  • #11106 — Redundant dune version constraint in opam files (filed 2024, fix: #14175)
  • #12007 — dune build @ocaml-index builds too many files (filed 2025, partially fixed with #14137)
  • #14089 — Incremental builds with (wrapped (transition ...)) broken (filed 2026, test: #14088, fix: #14090)

What I Learned

Writing tests is a good way to learn about a code base. Each test forced me to understand how a feature actually works — not the architecture in the abstract, but the concrete behaviour in specific scenarios. The understanding compounded.

Dune’s cram tests make this easy. Each test is a self-contained shell script with expected output, checked into the repo as a .t file. The barrier to writing a test is low, which is exactly what you want when your goal is to close issues by testing them. This occasionally backfired by being “too easy” - when I wrote tests to which maintainers objected because I had not done so in a Dune-idiomatic way; but this was still net positive for sure.

Consistent presence matters. I contributed nearly every day over several weeks. I think that this helped maintainers feel that I was “present” - that efforts that they made to help me were investments; so they were less prone to holding back reviews.

14 Likes