Windows, Dune and portability: best approaches?

Ecosystem
,
1

Hello,

I’m trying to improve the portability of our Dune-based tool, which currently relies heavily on using Unix-like commands here and there, for instance:

(action (with-stdout-to config.ml (run sed -f config.sed config.ml.in)))

This action reads the content of a previously-produced file with sed commands (config.sed), then applies them to a config.ml.in file (which contains patterns in the form of @FOO@, to be replaced by the sed commands), producing a valid config.ml file in the end, that is then compiled by Dune along with the rest of the code.

Obviously, this does not work as-is from Windows, since sed does not exist (at least in my PowerShell-based setup).

In such cases, what is the recommended way to proceed in order to maximize portability among OSes? Here are a few possibilities I’ve considered:

  1. Write a mini sed-like tool in OCaml, use Dune to build it, then run it (maximally portable and flexible for our needs, but may require a lot of extra coding, and wheel-reinventing);
  2. Use the existing sed.exe from the opam-managed Cygwin installation (I’m not sure how to do it, that is, how to find its path and reference it from within the Dune file, and if there are cases where such an installation will not be available);
  3. Require that the user sets up some environment variables (e.g. $env:SED='path-to-sed.exe' on PowerShell, and SED=/usr/bin/sed on Unix) before running dune build (this seems equivalent to running a make or ./configure script, so I’m not convinced this is a good idea).

If necessary, we can assume that users will follow our installation instructions, so if we tell them e.g. “use a PowerShell-based setup with a Cygwin managed by opam”, they will comply. So we can limit the different setups that we need to handle. This likely diminishes portability, but if it can keep things simpler, it’s fine for us.

Do you have any comments/experience on these approaches, or suggestions of a better way to ensure that such code can be built with Dune on Linux, macOS and Windows?

2

Yes: 1. is by far the best approach: it is typically quite easy (and simpler) to implement the required functionality in OCaml, and you get a build system that is fully self-contained and cross-platform.

See also [ANN] diskuvbox: small set of cross-platform CLI tools for inspiration.

Cheers,
Nicolas

2 Likes
3

Not as much as one may think:

  • extra coding: yes, a little bit, but using the standard library (+ unix and str or equivalent) you can get most tasks done in a few lines, in an actual robust programming language instead of depending on arcane POSIX semantics.

  • wheel-reinventing: I think you will find that you are not so much reinventing the wheel, but simply using a different language to specify your helper scripts: str (or equivalent) instead of sed, unix instead of bash, etc.

Cheers,
Nicolas

4

While I also suggest option 1, there’s sometimes the option of rethinking the problem and e.g. reformulating the thing you’re doing with dune primitives, rewriting it to use existing tooling or changing the code that e.g. you don’t need sed.

For example, in some cases you can use CPPO (or, tooting my own horn, µCPPO). In other cases you can rewrite the code so instead of code generation it can get the value via some other ways, which is often nicer as that works better with tooling like Merlin/LSP.