[ANN] VS Code testing extension for Alcotest and inline Alcotest PPX 0.1.0 - initial release

Ecosystem
1

Hi,

I’ve just released a VS Code extension to integrate Alcotest and inline Alcotest tests with VS Code.
It supports multiple workspaces/folders and uses the ‘native’ Test Explorer of Code, not the Test Explorer UI extension.
OCaml Alcotest Test Explorer extension
GitHub repo
It only discovers tests when the extension is started or a test is run, no file watching (I don’t like extensions doing this on their own). I’ll add a ‘refresh’ button to the Test Explorer view, I guess, sometime.
The output of Alcotest’s list of test cases is used to populate the test tree, so this is always in sync with the tests the runners ‘know’ of. My compromise is the mapping of a test case to its source location, I just grep for the test case name (which can be truncated by Alcotest) in source files and pick the first match - which can be off from the ‘real’ location. I personally prefer having a correct Test Explorer tree to having a correct source location. In my experience with other test extensions sometimes both the tree and the source locations are incorrect.

Some Questions:

  • Currently it only works with dune, to run the test runners and because dune files are parsed for the name of the test runners. So my question is what it would take to make it work with other build systems. Making the commands to compile and/or run a test runner and the paths to search for test runner executables configurable is easy. Making the parsing of arbitrary build files for the right targets configurable is less easy, if it is feasible at all. Of course you could always fork of the extension to exactly fit your needs
  • the extension just calls dune in the root of each workspace and it does the right thing with my projects. My question is if the right environment (sandbox) is used automatically when using the OCaml Platform extension?
  • Would add disabling the grepping for the source locations make sense for you? So you just use the (correct) test tree to start and filter tests and not waste time searching through all source files, which could take too long and doesn’t give the correct result anyway.

The extension should also be quite easy to adapt to other testing frameworks (inline test runners), as long as they support some kind of list test command line argument.

And sorry, the link to Alcotest is broken in the README, I’m going to fix this is the next release.

4 Likes
2

I’ve just uploaded the version 0.2.0

Changes:

  • Drop support for VS Code versions < 1.65 (February 2022) because of Testing refresh action
  • Test discovery can be triggered by the ‘Refresh’ button in the Test Explorer view

Bugfixes

  • Remove deleted tests from the Test Explorer tree
  • Find multiline inline test case names like let%test\nTEST_CASE_NAME
  • README.md: fix link to Alcotest

Internal Changes

  • Refactor test name parsing in source files to new function in ./src/parsing.ts and add tests
1 Like
3

Thank you! Do you think the extension could evolve to also support %expect_test? It does seem complex to collect the required info though.

4

If the expect test runner works the same as the inline test runner
PPX Inline Test Command line arguments, which is implied by PPX Expect Build system integration, it is enough to just grep for the location of let%expect_test (or let%test or let%test_unit) and be able to run a single test (or all tests in a file). So no actual AST (and evaluation of the AST) ist needed and this (both expect an ‘normal’ inline tests) should be doable by me in a few days. With support for dune only, as I’m going to only parse files in directories with a ppx_inline_test stanza in in its dune file.

As this isn’t related to Alcotests the extension’s name would be a misnomer. So I’d rather make a new test extension called something like ‘Inline Tests’. I’ll just have to test if is a problem having the Alcotest and the inline test extension running in the same Test Explorer window.

2 Likes
5

See:

3 Likes
6

Hi, a new version of the Alcotest extension, 0.3.0, now retries running dune every 2.5s if another process holds the directory lock. There are also some additional, minor, changes.

I’ve also released a new version - 0.2.0 - of the Expect and Inline PPX extension. See:

Changelog:

Version 0.3.0 (2023-03-06)

  • Retry dune commands every 2.5s if another process holds the dune directory lock.
  • Change the name of the test profile to ‘Run Alcotest tests’.
  • Set the tests state to ‘run’ before processing the tests.
  • Add documentation of ‘Run Profiles’.

Internal Changes

  • Add GitHub test action.
  • Add GitHub Issue Template.
  • Add jsdoc plugin for Eslint.
  • Fix various jsdoc errors.
  • Use function workspaceLabel to set the workspace node label and id.