Unit testing with jbuilder

jbuilder
testing

#1

Hello,

what is a preferred approach for unit testing when using jbuilder?
So far I’m using separate folders for library/ and tests/ , but the problem is I need to export all internals in Library.Internal modules in order to test it.
What (ideally) I want to achieve is to have access to modules ignoring signatures.


#2

I’ve been trying to use ppx_inline_test with tests in a test/ subdirectory. Unfortunately I haven’t been able to figure out the right incantation to get jbuilder to correctly use the ppx_inline_test library. I get the following error on every instance of a tests.

Error: ppx_inline_test: extension is disabled because the tests would be ignored (the build system didn't pass -inline-test-lib)

#3

Daniel,

Do you have an example of a project for us to consider? One idea is that if you’re using the wrapped mode for library, you can use the private names for testing (e.g. Lib__MyModule), while not adding them as aliases in your lib.ml file.

Otherwise, I agree with the other poster that getting inline tests to work in jbuilder is the way to go in the future.


#4

I have no examples which I’m ready to show, but my use case is when I need to test some internal functions that are not exposed in *.mli file.

Yeah, but then you need to add your signatures to the “main” (wrapping) module to keep all internals exposed in private modules.

I’m not sure if inline tests are the ultimate solution to this problem. What if I want to use some external library for tests only? Also I’m prefer to keep tests in separate place and not clutter actual source.

What I ideally want is something like jbuilder “–ignore-interface-files” option. I can try to implement it by itself, but I’m not sure what other OCamlers think about it and whether this change would be accepted.


#5

I agree with this. However, I’m not sure testing private functions (what I assume you’re wanting to) is a good idea in general. Can’t you just test the public interfaces?


#6

By the way, a bit different topic but still about unit testing in jbuilder: does jbuilder runtest support multiple test files? Or do we have to use one entry point for the whole suite?

I’d like to separate my tests (using OUnit) into standalone files but I can’t find a way to run them with runtest. Actually, I’m also not sure if OUnit supports running tests in multiple files.


#7

I ported one of my projects to use Alcotest as a testing framework and have been really enjoying. I would recommend it for non-inline, public interface testing.

I have also been commenting on a bug report trying to get jbuilder to work with ppx_inline_test but I haven’t tested out their recommendations yet.

From what I understand, the runtest command in jbuilder simply looks up the runtest alias, which you can redefine any which way you choose. For example, in a current project, I have runtest defined as follows:

(alias
  ((name    runtest)
   (deps    (Test.exe))
(action (run ${<}))))

Test.exe is an executable defined separately in the same jbuild file that contains tests defined using Alcotest. You should be able to define multiple executables and put them all into the deps statement to get the effect you desire. I’m not sure how this would interact with OUnit.


#8

I see, so either way I would still need to enumerate the files (either in jbuild or in an entry point test.ml). OUnit allows specifying a list of suites, so current my approach is a single executable test.exe which wraps the standalone test suites together and run it. This cripples me from running a single test suite on its own, though (I can only run all), so I’ll explore the jbuild option later today. Thanks!


#9

OUnit.run_test_tt_main gives you options to list and run a single test or subhierarchy, cf --help.


#10

Oh, neat! I can run specific tests via the -only-test <path> flag. That’s nice. But I’m having trouble figuring out if the <path> option accepts some kind of shorthand/globbing. ./test.exe -list-test shows something like 0:Foo:0:bar and 0:Foo:1:baz. I can pass those (or their prefix, e.g. 0, 0:Foo, 0:Foo:1) as <path>, but it seems kinda awkward and they don’t support something like ./test -only-test 0:1.


#11

There is a feature request: https://forge.ocamlcore.org/tracker/index.php?func=detail&aid=771&group_id=162&atid=733

Passing -only-test multiple times also allows a hack:

ounit-run-grep() { "$1" `"$1" -list-test | grep "$2" | sed 's/^/-only-test /'`; }

#12

The workaround I’ve got for running multiple test suites is calling system action:

However, this makes the tests not portable, e.g. to Windows.


#13

Simpler is to simply create 2 runtest stanzas - 1 for every test. Yes, this is more boilerplate but at least it will be portable and also run in parallel.