Dune portable replacement for 'grep' / 'sed'

Ecosystem
,
1

I just realized that this dune action fails on Windows, where grep and sed are not available by default:

(rule
  (targets version.ml)
  (deps foo.opam)
  (action (system "grep ^version foo.opam | sed -e 's/version:/let version=/' > version.ml"))
)

Well, more generally this is the case for most Unix base tools. Just wondering if by chance there are some minor OCaml “coretools”-like packages out there that could serve as replacement for such simple tools, to improve cross-compatibility.

I remember now that diskuvbox exists, but its last release was 3 years ago, and I don’t see grep/sed commands in it. Is there something else like that?

(Note that in this case we are just getting a version string from a .opam file, and we probably could try writing the version directly in the dune-project file and then generate the .opam file from there, and have the version variable directly available to dune; but in a more general case, the grep/sed approach might still be useful.)

1 Like
2

Not a general purpose solution but Dune itself ran into the same issue, particularly for writing Cram tests. Our solution was to implement dune_cmd which is a portable implementation of the bits of sed and other tools we use.

However we don’t need compatibility with command line flags or particular behaviour. For us the important things are that these commands are portable and behave the same on the supported platforms.

3 Likes
3

Cool!

But currently it’s just a binary used inside dune itself, and not exported, right?

So if I wanted to use it, I’d have to copy the source files to my own project, and build it, right?

4

Yes, its not meant for anything more than just the usecase of dune development and is not guaranteed to work correctly past the usecases we have.

But the pattern might be still useful for you: if you have a few commands that you need, you can write them in OCaml, have dune build the binary, use the binary as part of the build and not install it.

This has the advantage that you don’t need to pull in any dependencies and the commands can be customized to exactly what you need, which is sometimes easier than santizing sed, grep or ls -l.

2 Likes
5

Perhaps using MSYS2 would fix this issue.

6

Except sed is not compatible across Linux and BSD/macOS AFAIR, so it’s better to use the general workaround anyway.

7

Yes, inasmuch as installing grep+sed will fix the issue!

In general, for portability in Dune situations like this, I’d just use OCaml - it’s always there! In this particular case, a better (portable) use of sed is:

(rule
  (with-stdout-to version.ml (run (sed -ne "s/version:/let version=/p" %{dep:foo.opam})))
)

but it’s also a tiny OCaml script. If slightly more power is needed on the string processing, there’s ocamllex - if even more is required, the re library is pretty dependency-free.

Given that I spend much of my time working on the compiler’s build system where we don’t have the luxury of being able to reach for OCaml, it alwaus surprises that there’s so much plumbing in dune projects which reaches for these very-difficult-to-use-portably shell tools so quickly :slightly_smiling_face:

1 Like
8

Out of curiosity, did you solve it in this case using a dune action? Eg

(rule
 (target version.ml)
 (action (write-file %{target} "let version = \"%{version:foo}\"")))
2 Likes
9

Yes, in this case such an action would work much better, thanks.