Hi @Gopiandcode,
I’ve been wanting a PPX to generate my Alcotest boilerplate for a while, and am very happy to see others thinking along the same lines! 
There’s an open issue about it, and I have an old prototype implementation of a similar PPX which I had hoped to finish and release “soon”. I think it’s definitely worthwhile releasing ppx_inline_alcotest as-is; I’m sure folks will find it useful!
Here are some general thoughts I had about the problem, which you may or may not find interesting 
When first working on the PPX, I became frustrated with the fixed 2-level structure that Alcotest imposes (i.e. each test suite must be a list of lists of tests, with no scope for more or less nesting). I’ve always found this restrictive, but trying to map the OCaml AST onto the Alcotest API really put it into sharp focus: OCaml syntax is much better at grouping than the Alcotest API will allow. Ideally, I’d like to be able to structure my tests into modules/files however I want and have it “just work”:
module%test Lorem = struct
let%test "ipsum" = ()
let%test "dolor" = ()
end
module%test Sit = struct
module%test Amet = Some_other_compilation_unit
let%test "adipiscing" = ()
end
We have some existing test suites written in this style (e.g. here), but the rigid Alcotest.test type makes things harder than they need to be.
Another limitation is that Alcotest doesn’t let the user attach source locations to test cases, so if a test fails then Alcotest isn’t able to point at it: I always have to search the code rather than having my editor jump straight to it. With a PPX, we can obviously attach the source location for free; the Alcotest runner just needs to support it.
This line of thinking led to this RFC for an improved Alcotest API, which is what my PPX prototype is now based on. The above example outputs this:
In addition to better grouping & location information, I decided on a few other requirements for such a PPX to be able to replace my own manual use of Alcotest:
-
playing well with multi-file test suites. In the Irmin project, we tend to split large test suites out over multiple files and pull them together in a single test binary per folder (e.g. here).
-
playing well with functors. If I define some tests inside a functor body & apply that functor to different arguments, the tests should run once for each set of arguments.
-
playing well with Lwt/Async test cases. I should be able to mix and match e.g. Alcotest_lwt and Alcotest test cases and have the generator do the right thing. Ideally, adding an Lwt test case to a previously non-Lwt suite wouldn’t require changing any of the existing tests.
As an aside, as mentioned in the ppx_alcotest issue, I think there are some other good candidates for other PPX features for the Alcotest API. At the very least, we should provide a deriver for “testables” and a set of test assertion functions that embed source locations automatically, but I think there’s also scope for things like writing test assertions as partial matches:
let%test "basic queue operations" =
let q = Queue.(empty |> add 1 |> add 2) in
let%check Some (1, q) = Queue.pop q in
let%check Some (2, q) = Queue.pop q in
let%check None = Queue.pop q in
()
(This would fix the limitation of not being able to bind new variables as part of a regular Alcotest.check.)
To conclude:
-
ppx_inline_alcotest is definitely worth releasing.
- The Alcotest test registration API is likely to become more conducive to PPX in the future.
- I’m very interested in adding a
ppx_alcotest package to the mirage/alcotest repo once (2) has happened, which I hope will handle test registration as well as some other things. You’re of course very welcome to contribute to this effort!
- If you have any thoughts / feedback on either the proposed Alcotest API, my prototype PPX implementation, or any other PPX ideas for Alcotest, then I’d love to hear them!
Thanks for the excellent library!
Cheers,
Craig
