Opam-ed: quickly edit opam fields from the CLI

@AltGr released a fantastic tool into opam recently called opam-ed which I thought would be a nice “tip of the day” for OPAM contributors.

If you need to quickly edit a lot of opam files in order to add or remove fields, opam-ed lets you do it in a programmatic way.

For example, I needed to add a cstruct-lwt dependency to a lot of packages as part of the efforts to remove optional dependencies in the Cstruct package. This touches about 100 opam packages, but with opam-ed I could script it up pretty easily:

opam install opam-ed
opam-ed --preserve -i 'add-replace-item depends "cstruct-lwt" "cstruct-lwt"' -f mirage-block-unix.2.8.1/opam

The --preserve minimises whitespace changes in the file, and -i edits it inplace in the repository. Then there is a small DSL that specifies the changes. In this case, I wanted to add a cstruct-lwt dependency, but not duplicate it if one was already present.

I’ve also used it in other packages to add upper bounds via a script:

opam-ed --preserve -i 'add-replace-item depends "cstruct" "cstruct {<"3.0.0"}"' -f opam

Do opam-ed --help to see all the options. Thanks for writing this very useful tool, Louis!

5 Likes

Nice! I was previously using a personal fork of @avsm’s opam-tagger tool for similar purposes – opam-tagger only knows how to add tags, I added support for adding new dependencies, which I used to curate the opam-repository based on opam-builder’s output. Next time I get into massive opam-repository edit, I will try to use this new tool instead, that seems better designed to scale in that direction.

In my experience working programmatically with dependencies, I sometimes needed features that are not there. I would like to be able to do finer-grained transformations on dependency bounds/intervals: “you actually need (X.version > V), but keep current upper bound” or “you actually need (X.version < V), but keep current lower bound”, that are not yet supported by this tool that handles data generically. Hopefully someone (maybe me) will contribute this.

Also, this tool really needs more real-world usage examples that prospective users could adapt to their use-cases. @avsm has an example above, which is nice. If you have done your own transformations with the tool, please come post them here! We can them send pull requests to add them to the tool manpage.

1 Like

This seems a good improvement. The man page could however list a few example.

Higher-level edits like make all packages dependent on this package incompatible with this.1.0.0 would be nice.

Yes, this is fairly low-level (syntax tree level, to be exact), which is nice to limit dependencies, and work with all opam files (package definitions, but also .install, .config, ~/.opam/config, repo, etc.)

It does not allow, though, clever modifications, like adding an upper-bound to package dependencies without clutter. That needs knowledge of the specific file format, and mainly of the dependency/filter type. In short, you would want to use the opam-format part of the opam libs, which even includes functions for simplifying inequality-based formulas.

In short, you would want to use the opam-format part of the opam libs, which even includes functions for simplifying inequality-based formulas.

If someone wants to work on this for opam-ed, my patch against opam-fu might be useful inspiration:

let parse_filtered_constraints str =
  let lexbuf = Lexing.from_string str in
  let value = OpamBaseParser.value OpamLexer.token lexbuf in
  let filtered_constraints =
    let open OpamFormat.V in
    (package_formula `Conj (filtered_constraints ext_version)) in
  let pos = ("", 0, 0) in
  filtered_constraints.OpamPp.parse ~pos value

let add_dep ({OF.depends; _} as t) new_deps =
  let new_deps = parse_filtered_constraints new_deps in
  OF.with_depends (OpamFormula.And (depends, new_deps)) t

I hadn’t implemented constraint simplification, though.

1 Like