[ANN] diskuvbox: small set of cross-platform CLI tools

TLDR:

$ opam update
$ opam install diskuvbox

$ diskuvbox copy-dir --mode 755 src1/ src2/ dest/
$ diskuvbox copy-file --mode 400 src/a dest/b
$ diskuvbox copy-file-into src1/a src2/b dest/
$ diskuvbox touch-file x/y/z

$ diskuvbox find-up . _build
Z:/source/_build

$ diskuvbox tree --max-depth 2 --encoding=UTF-8 .
.
├── CHANGES.md
├── README.md
├── _build/
│   ├── default/
│   ├── install/
│   └── log

Problem: When writing cram tests, Dune rules and Opam build steps, often we default to using GNU binaries (/usr/bin/*) available on Linux (ex. /usr/bin/cp -R). Unfortunately these commands rarely work on Windows, and as a consequence Windows OCaml developers are forced to maintain Cygwin or MSYS2 installations to get GNU tooling.

Solution: Provide some of the same functionality for Windows and macOS that the GNU binaries in /usr/bin/* do in Linux.

diskuvbox is a single binary that today provides an analog for a very small number of binaries that I have needed in the Diskuv Windows OCaml distribution. It is liberally licensed under Apache v2.0. With your PRs it could emulate much more!

diskuvbox has CI testing for Windows, macOS and Linux. Usage and help are available in the diskuvbox README:

diskuvbox also has a OCaml library, but consider the API unstable until version 1.0.

Alternatives:

  • There are some shell scripting tools like shexp and feather that give you POSIX pipes in OCaml-friendly syntax. I feel these complement Diskuv Box.
  • Dune exposes (copy) to copy a file in Dune rules; theoretically more operations could be added.

Internally diskuvbox is a wrapper on the excellent bos - Basic OS interaction library.

Acknowledgements

The first implementations of Diskuv Box were implemented with the assistance of the OCaml Software Foundation (OCSF), a sub-foundation of the INRIA Foundation.

Two OCaml libraries (bos and cmdliner) are essential to Diskuv Box; these libraries were created by Daniel Bünzli (@dbuenzli) .

Examples

The following are examples that have been condensed from the diskuvbox README.md

Using in Dune cram tests

  $ install -d a/b/c/d/e/f
  $ install -d a/b2/c2/d2/e2/f2
  $ install -d a/b2/c3/d3/e3/f3
  $ install -d a/b2/c3/d4/e4/f4
  $ install -d a/b2/c3/d4/e5/f5
  $ install -d a/b2/c3/d4/e5/f6
  $ touch a/b/x
  $ touch a/b/c/y
  $ touch a/b/c/d/z

  $ diskuvbox tree a --max-depth 10 --encoding UTF-8
  a
  ├── b/
  │   ├── c/
  │   │   ├── d/
  │   │   │   ├── e/
  │   │   │   │   └── f/
  │   │   │   └── z
  │   │   └── y
  │   └── x
  └── b2/
      ├── c2/
      │   └── d2/
      │       └── e2/
      │           └── f2/
      └── c3/
          ├── d3/
          │   └── e3/
          │       └── f3/
          └── d4/
              ├── e4/
              │   └── f4/
              └── e5/
                  ├── f5/
                  └── f6/

Using in Opam build steps

build: [
  ["diskuvbox" "copy-file-into" "assets/icon.png" "assets/public.gpg" "%{_:share}%"]
]

Using in Dune rules

(rule
 (targets diskuvbox.corrected.ml diskuvbox.corrected.mli)
 (deps
  (:license %{project_root}/etc/license-header.txt)
  (:conf    %{project_root}/etc/headache.conf))
 (action
  (progn
   (run diskuvbox copy-file -m 644 diskuvbox.ml  diskuvbox.corrected.ml)
   (run diskuvbox copy-file -m 644 diskuvbox.mli diskuvbox.corrected.mli)
   (run headache -h %{license} -c %{conf} %{targets})
   (run ocamlformat --inplace --disable-conf-files --enable-outside-detected-project %{targets}))))
13 Likes

It is very useful indeed. The dune actions can be extended but before it is very interesting to see what kind of actions are really useful in ocaml packages.

There is no way to install without opam right?

Good timing. In preparation for the next bugfix release I started to auto-release binaries a few days ago. You can grab prerelease binaries now for Windows on Intel 32/64bit, Linux on Intel 32/64bit and macOS on Intel/Silicon binaries at https://github.com/diskuv/diskuvbox/releases/tag/0.1.1-prerel3

The “official” (release) binaries will be available this week or next. I’ll post here when that happens.

Tested downloading the macos M1 binary and it just works as expected.
Is it compiled and uploaded from GH actions? I took a look and only saw macos x86 in the matrix

The macos-latest host compiles natively to x86_64 but also cross-compiles to arm64. The cross-compiling OCaml compiler is at dkml-base-compiler 4.12.1~v1.0.0 · OCaml Package.

Yes, it is done from GitHub Actions. The GitHub Actions uses a similar build apparatus as https://github.com/diskuv/dkml-workflows-monorepo-example although it doesn’t directly use opam monorepo (because opam monorepo does not yet support dkml-base-compiler).

That is probably the 5th way of compiling Ocaml code that I read this month, LOL, very confusing. But I don’t want to offtopic your great announcement, so please ignore my comment.

Version 0.1.1 is now available from opam. Upgrade with:

opam update
opam upgrade diskuvbox

Changes include:

  • A bugfix for 32-bit OCaml systems; copying of large files now works.
  • This is the first version to officially include standalone binaries. They are available at Release 0.1.1 · diskuv/diskuvbox · GitHub for Linux/Intel 32- and 64-bit, macOS/Intel, macOS/Silicon, Windows/Intel 32- and 64-bit
1 Like