Publishing to Opam

Community
1

This might be unpopular here but I think the common case of adding a package to the Opam repository is too complicated. I recently tried opam publish and dune-release and they failed with unexplained authentication problems or demanded conventions that go beyond what Opam requires like conventions where to publish documentation.

Obviously it’s my responsibility to have an Opam file that correctly specifies dependencies and some meta data. Beyond that, in the common case the Opam file is part of a GitHub repository that includes the code and GitHub hosts the archive based on the tag. Why is this more complicated than providing an URL to a GitHub repository and a version tag? I don’t want to exclude other methods and forms of hosting but I believe the ecosystem would benefit from a simpler process for this use case. I know that I publish fewer packages than I could because I simply don’t want to deal with the complexity.

4 Likes
2

Don’t let github generate archives for you. The checksums have been unreliable in the past and will (still ?) change if your repo is moved from an organisation to another.

That being said if you are looking for a simpler workflow why don’t you simply PR your opam file on the opam-repository manually ? That’s as simple as it gets.

3

While I have seen that, it’s rare. Roughly 70% or packages in Opam are hosted on GitHub like this - probably because it is easy.

As for raising a PR for opam-repository.git: I could do that but it forces me to obtain hashes and maintain an Opam file outside of the code repository. It forces the internal representation of Opam on me as a user for something that I feel could be simpler.

4

I’m curious what your source is for that 70% number. Personally I think it’s an issue. You know what happens when a checksum changes ? Systems that rely on the checksum most of the time just change to the new checksum.

Does actually someone take care to check what’s in the new tarball ? Of what I have seen in the past, no, precisely because github checksums are known to be unreliable.

Frankly it’s not hard to make your own tarball and upload it to github, there are tools that automate that for you topkg release does it, I hope dune-release kept that.

I don’t see why this entails maintaining an opam file out of your code repository. You can simply take the one that is in your repo.

That being said topkg opam’s release worfklow – I don’t know how it evolved in dune-release though. Does exactly this for you:

  1. Take the opam file of your repo.
  2. Download your release, checksums it and add the info
    to your opam file.
  3. Uses opam publish to submit the file.

Now it’s true that opam-publish’s experience has been totally subpar to me over years (difficult to build/too many dependencies for what a curl shell out would have solved in a few lines, unreliable/outdated github auth).

But for the rest these tools likely do what you are looking for. They are precisely made to make the act of releasing inane while maintaining a certain quality level. So if you are looking to scale your release train I suggest you have a look at them again (in particular have a read at dune-release help release).

5

Looking at the src: line in opam-repository.git using git grep you can see that most packages host archives on GitHub. Admittedly, this includes older versions, too.

6

And archives not generated by github itself.

7

The problem with publishing to opam is that it’s not frequent enough and I keep forgetting what to do. :frowning: I need to populate my projects with READMEs to supplement my bad memory.

Nowadays I let dune automatically generate the opam file with (generate_opam_files true), and it’s been working well enough. Then just pushing a tag to github and invoking opam publish does the work.

(except that I have received a deprecation notice from github about the Authorization API)

1 Like
8

Does opam publish work with GitHub’s 2FA for you?

9

same here. but publishing seems to continue to work, and I opened an opam issue, so … not sure there’s anything we can do about it.

10

I can’t say, I’m not using the two factor authentication

11

It’s a problem not of opam-publish or dune-release but of github package:

Essentially it boils down to rewriting github package to GraphQL.

12

Thanks for sharing these links. No, fixing this issue is not related to rewriting the github package to GraphQL (even if GraphQL is great). I’ve added comments to the last two issues to explain what I mean.

13

Yes, but it does not mean that these are GitHub-generated. The packages I host on GitHub have their archives on GitHub, but the tarballs are generated by dune-release and don’t face the problem of being unreliable.

2 Likes
14

forces me to obtain hashes

Usually I am manually pushing opam file PR, so the above was the annoying point indeed. In the end I solved it with a simple shell script (which also includes some workarounds for the non-standard versioning and naming of adopted projects which I didn’t want to change) : scraps/opam-url at master · ygrek/scraps · GitHub

1 Like
15

I’m using the hub command line tool to make pull requests from the command line:

hub pull-request --base ocaml:master -m "Add package PKG version XX"

(type this command from the right branch if you make several pull-requests)

This is quite easy, just forking/cloning opam-repository, editing/copying the opam package file, and then send the PR with hub. (it can be done in few minutes)

3 Likes