Welcome new maintainers of opam repository, and introducing Obi

A separate topic would be great – particularly with respect to releasing new versions of packages rather than simply constraining them.

While Obi is in experimental mode, the page will refresh every few days since it is semi-manually driven. The reason for this is important – the new CI runs on opam2-dev only since it takes advantage of new features (and @altgr is awesome at adding the interfaces we need for good CI and can’t do so on opam1).

Therefore, it is prone to running into bugs in opam2-trunk that cause transient blockage and so won’t be fully automated until opam 2.0 is released. We also have some instability coming up as we hook in Windows tests into it as well, which may require a rearrangement of the container layouts.

However, I hope it is useful in its fledgling form today to highlight packages that can be immediately fixed. I am too short on time due to other duties to do the actual opam-repository work myself at the moment, unfortunately.

1 Like

There is no such feedback mechanism yet, although I expect we will build that in the future. For now, I hope that having extensive build logs about maintainers’ own packages will make things easier for them to fix (as indeed, the datakit-ci reverse dependency checks have done).

The Obi triage page shows that a package is broken, including if one of its dependencies are down. @Altgr, is it worth distinguishing a dependency failure from an install target in the opam install error code, you think?

One tool that should come online to make this less manual soon is opam admin add-constraint (or a variation thereof for compiler revisions). @Altgr is working on a bidirectional branch for opam1/2, so in theory (if this works – he may well contradict me as being mad) we could use the opam2 administration tools to add constraints that reflect back into the opam1 git repository :slight_smile:

I’m reluctant to spend any time on improving the opam1 administration tools while we have our hands full with getting opam2 released, so this ability to use opam2 to improve opam1 would be very, very handy to have.

@altgr Does this include by any chance that opam tries to pick the smallest compatible version for a given package ? If that’s the case that’s very useful, I think many build failures (at least in my packages) can be attributed to lack of lower bounds.

I think the reason to distinguish them is, at least somewhat, is that one should focus on the packages that are themselves known broken. The packages depending on a broken package would often be fixed as soon as the dependency is fixed.

No, not by default, the best on the user side is still to try to get the latest possible version of everything.
See here for the changes, which were performance-oriented but shouldn’t change the end results.

You can, however, try to install lowest bounds by maximising, instead of minimising, the count[version-lag,changed] criterion. Note that mccs doesn’t perform well with maximisations, though. aspcud is still an option for such cases. Since the criteria format is annoyingly different, see opam config report --solver aspcud for the defaults. The criteria you want would be something along the lines of -count(removed),-sum(request,version-lag),-count(down),-count(changed),+sum(changed,version-lag). for installing mirage-clock on an empty 4.04.2 switch, for example, this gives me:

  ∗  install ocamlbuild    0.9.1     [required by fmt]
  ∗  install conf-m4       1         [required by ocamlfind]
  ∗  install ocamlfind     1.6.2     [required by jbuilder]
  ∗  install jbuilder      1.0+beta9 [required by mirage-clock]
  ∗  install fmt           0.7.0     [required by mirage-device]
  ∗  install mirage-device 1.1.0     [required by mirage-clock]
  ∗  install mirage-clock  1.3.0     

instead of

  ∗  install ocamlbuild    0.11.0     [required by fmt]
  ∗  install conf-m4       1          [required by ocamlfind]
  ∗  install result        1.2        [required by fmt]
  ∗  install uchar         0.0.2      [required by fmt]
  ∗  install ocamlfind     1.7.3      [required by jbuilder]
  ∗  install topkg         0.9.1      [required by fmt]
  ∗  install jbuilder      1.0+beta14 [required by mirage-clock]
  ∗  install fmt           0.8.4      [required by mirage-device]
  ∗  install mirage-device 1.1.0      [required by mirage-clock]
  ∗  install mirage-clock  1.3.0      
1 Like

The possibility has been proposed in the past to have “smallest version possible” by default in CI, but actually, since we now have a much better CI that can test many more things, why not just do both ?

1 Like

When going over the current list of open PRs against opam-repository, we can waste some time looking at issues that have already been taken care of by another repository maintainer.

I would propose to use “assigned” issues to avoid this. If as a maintainer you do the work of understanding an issue, you don’t merge right away but you feel confident that you will be able to take care of the interaction with the submitter from now on, feel free to just assign yourself the issue. Then other maintainers are free to skip this issue when looking for new issues requiring their attention – of course giving a second opinion on an assigned PR is always welcome.

6 Likes

@Chris00 over at https://github.com/ocaml/opam-repository/pull/10821 is asking for someone to test a configuration script under OSX. Among the new or old opam-repository maintainer, is someone running OSX and willing to do this?

(For the record, I’m running Fedora and could help debug Fedora-specific issues. Fedora is pretty bad for OPAM, though, given that there is no decent external solver packaged for recent releases.)

Thanks for publicizing my request. Travis says all is fine, including on Mac OSX so I guess I put the right compilation instructions… :slight_smile:

Now it is @UnixJunkie’s turn to need a MacOS user’s hand to try to build a conf-package in #10926. It would be mildly helpful to have a list somewhere of which kind maintainers to ping in that situation.

I’m not sure. osX maintainers have been thin on the ground, and the Travis infrastructure is unreliable. Help welcome here…

While not a maintainer, I am primarily a MacOS user and would like to find ways I can be more helpful/involved. I’d be happy to be on such a list, if simply having a person willing to go through the build steps is sufficient.

I’ll try to help on the linked PR when I get home tonight, if no one else beats me to it.

1 Like

Is there a place to have a discussion on package naming choices, or a procedure on who to ask when the question arises? @UnixJunkie is proposing a new package (GPR#10927) named “consent”, and I personally think the name is terrible. Now that the CI results are okay, should I ignore the bad-idea feeling and merge nonetheless? (Are there people interested in giving an opinion on such questions?)

2 Likes

It’s a normal thing to have. The need for it is more apparent when there are so many packages that categorization becomes necessary, as then package submission and categorization easily blurs together – that is, package maintainers are also maintaining the package taxonomy. A decade ago if you wanted to submit a package to CPAN you were advised to discuss your module and its name on a mailing list not used for much else. That was just an advisement: submitters wanted to use it and others wanted to advise you so that module names would be reasonable and useful. Of course, of a do-nothing module with a name like "The::Perl::Comunity::Sucks::Lol", or "Eat::This::<100 megabytes of characters remaining in this module name>", or "; DROP TABLE PACKAGES", corrections would be made. There is no need for a rule like “don’t deliberately pollute our service”, and trying to add rules for abuse will just invite various kinds of rule-fetishist.

But, if you’re interested in a forum that can actively reject names, not just offer advise about them, and not merely about abuse but about someone using names that others dislike for whatever reason, then I suggest you consider

  1. the hours that you will spend arguing that you were justified in saying “no” after having said “no”.

  2. the ease by which someone can fork the entire OPAM system and rerelease it without any kind of onerous gauntlet for package-submitters to go through.

  3. the likelihood of a politically neutral fork like this becoming the most popular module system after yours is clouded in drama.

If what bothers you is the feeling that you are endorsing a name when you merge a PR that adds it to OPAM, maybe a bot can do merges like that.

Alternately, you can exert more control over names and package quality, while encouraging people to want to submit to that control, by having a separate curated list of packages, or by curating some sign of quality that’s shown in normal opam listings of packages. … you know, like Twitter’s blue checkmarks. This would also be of genuine value to the OCaml ecosystem, as it helps package seekers find a probably-good package without having to do lots of research first. Meanwhile, since packages will still be available even if you haven’t given them your seal of approval, you’ll lose fewer hours to arguments about your decisions.

1 Like

Thanks for the perspective. I think that the specific situation I’m thinking about (although it’s good to have opinions and feedback on other case that may happen in the future) is, fortunately, quite far from this “should we actively reject or not?”. The question I’m asking, I guess, is whether other people thing that it makes some sens to push back a bit harder on a name, or whether it does sound reasonable to them.

3 Likes

Oh OK. Then I agree: this is some kind of extremely niche “chemoinformatics” software but it has a name that suggests that it handles “[Y/n]” queries in a terminal or double-opt-in requirements for email lists. The name fails to

  1. help non-users know it isn’t for them (people looking for graphics software might check it out) and,

  2. at least, it doesn’t have something obvious like chem_ in the name to help potential users find it.

Names are hard. What I at least look for are typosquatted names (since there are actual attackers http://incolumitas.com/2016/06/08/typosquatting-package-managers/ https://www.theregister.co.uk/2017/08/02/typosquatting_npm/ https://mail.python.org/pipermail/security-announce/2017-September/000000.html and others).

From https://github.com/ocaml/opam-repository/wiki/PR-checks the only mention is “no ocaml pre- or postfix” (which is unfortunately violated sometimes).

I gave up trying to police any names in there, since I saw several names which IMHO are way too generic (and/or where the description doesn’t explain anything).

I think the procedure followed in the PR is sensible. Request the author to change the name in the PR if there is a concern, and if we cannot reach consensus with the submitter of the PR, then the maintainer team will need to make a judgement call on that specific case.

2 Likes

Regrading typosquatting: Perhaps we could have the CI scripts check for names that are “similar” to other packages/PRs (levenshtein distance, phonetic matching) and give the author a heads-up if they’re proposing a name that might be confusing.

1 Like