Best way to publish opam package

Hi community

I’m trying to find out how to publish an opam package.
I have found these two links, which give contradictory information (or, at least, not easy to reconcile)

http://opam.ocaml.org/doc/Packaging.html

I assume the last one is a bit old. But the information on the first one is quite scarse.
Is there somewhere an up-to-date tutorial for this?

Here is my situation:

  1. freshly installed ubuntu 18.04 (KDE neon, in fact) with system ocaml 4.05.0, installed opam 2, upgraded it, installed dune.
  2. my project uses dune, which autogenerates the .opam file. I have no problem installing a local opam package with opam install .
  3. my project is hosted on github (sanette/bogue)
  4. I tried opam publish from my source dir. First time, I got an authorization error. The next day, this authorization error disappeared. I assumed it is related to some ssh/token authorization from github. Not sure. But now, I cannot go past the second error which is
    $ opam publish

The following will be published:
- bogue version 1907 with opam file at /home/san/prog/ocaml/bogue/github/bogue/bogue.opam

[WARNING] These will be virtual packages: bogue.1907

Continue ? You will be shown the patch before submitting. [Y/n] y
Fetching the package repository, this may take a while...
[ERROR] Uncaught exception: "/usr/bin/git fetch --multiple origin user" exited with code 128

Any help welcome! Thanks

Since you are using dune you should maybe try to use dune-release.

(Regarding the particular error you mention I don’t have the time to look further into this but the user here looks suspicious:

1 Like

I remember I met some error too when tried opam-publish. And just published manually.

I think dune-release should be mentioned on both opam.ocaml.org and dune.build sites then.

P.S. Also I recommend to install OCaml 4.07.1 instead of the system one. 4.05 is too old.

Ouch, dune-release needs ocaml >= 4.06.0.

My idea to stay with system ocaml 4.05.0 was to make the program usable by the large number of ubuntuers that prefer to use their system out of the box.

Maybe this is a stupid question: can I use 4.06.0 in order publish to opam a package that can be installed by users having only 4.05.0?

Yes. dune-release just handles the incredibly annoying busiwork of packaging, calculating checksums, uploading, generating and publishing docs and generating OPAM files but it doesn’t modify the contents of your package. So if your package works with older versions of the OCaml compiler it will continue to work.

1 Like

Just a note on that, but these days, it’s de facto impossible to function in the OCaml ecosystem without running opam, and once you’re running opam, installing a newer compiler is just a question of typing opam switch. Generally I assume users are going to be on a fairly recent compiler unless they’re using bucklescript. 4.05 is pretty old at this point.

@sanette does mention that they want to make their program usable by Ubuntu users with the compiler that is shipped with the distribution, which I assume means creating a Debian package and PPA out of their program. In that case it is a bit difficult to use a newer compiler and system OPAM. Possible, for sure, since the resulting binary does not have a dependency on the compiler, but then the packaging step would need to pull in OPAM, build a newer compiler, build the binary and pull that binary into the Debian package.

Debian-based distributions are always too slow to update their packages. I know that as a radare2 developer, some people still come with the 0.9.6 version which is 5 years old. A better course of action would be to consider non-Debian-based distribution, like Fedora, Arch, Manjaro, Alpine, Void.
You can see that distinction clearly at https://repology.org/project/ocaml/versions

Telling a user they need to install a different distribution of their operating system seems to me that it would be a hard sell.

I doubt that’s what they’re asking for here, given that the request was with help opam packaging their software, and given that it’s a set of libraries mostly of interest to developers who will need opam anyway…

@perry
you’re right, the library is of course for ocaml programmers, and with opam it’s easy to install another kernel. So maybe I should not be so picky about alining with ubuntu’s versions.

On the other hand, I find the discussion interesting. One of the plans was to develop bogue to offer a lightweight GUI to ocaml beginners: so that they can easily see the result of small programs, and change variables on the fly. In this case, maybe they prefer to stick to the system ocaml version? [I know, for this purpose, a Web gui is probably easier, but well… it doesn’t hurt trying :wink: ]

@Leonidas
thanks, I guess I will try to do this: install a more recent ocaml in order to use dune-release, but keep (at least for the moment) 4.05.0 compatibility.

Just an observation - Debian 10 was released yesterday, so the testing and unstable shifted too, but the latest version is 4.05 still.

http://opam.ocaml.org/doc/Packaging.html

This document is mostly OK, but clearly need refreshing for usage with dune, which is the standard nowadays. Also, there are some obsolete tidbits at the end (references to ocaml-version)

If opam install . works, it means you have a correct opam file already anyway.

https://github.com/ocaml/opam-publish

This is recent and should work out-of-the box. You’ll want to look at the 2.0 branch, though: https://github.com/ocaml/opam-publish/tree/2.0 . The latest release is 2.0.1

Basically, if your project is hosted on Github, git tag -a TAG; git push origin TAG; opam publish should take care of everything for you. The error seems to be related to fetching the data from Github, you normally should be asked for authentication on the first run ; if that failed somehow, you could try to wipe the tokens in ~/.opam/plugins/opam-publish and look at your registered tokens at https://github.com/settings/tokens .

in fact I have the impression that this is not completely correct. When I try to use dune-release,
(after having installed ocaml 4.08.0)
it does try to compile my source. And this stops because there is some incompatibility with the Pervasive standard library used in 4.05.0.

Hi @AltGr
I’m still stuck :frowning:
The same errors appears, and as Daniel said, the user is suspicious. In fact I just saw the log file in my /tmp, and it says

fatal: ni ceci ni aucun de ses répertoires parents (jusqu'au point de montage /) n'est un dépôt git
Arrêt à la limite du système de fichiers (GIT_DISCOVERY_ACROSS_FILESYSTEM n'est pas 
défini).

(which means ‘I can’t find a git repository in your filesystem’)

However, I issued the command opam publish from the directory where my .git is.

I suspect that the issue comes from the fact that, instead of looking at the current path, opam starts from ~/.opam/plugins/opam-publish/repos/ocaml%opam-repository – and upstairs.

opam-publish needs to fetch two things:

  • the opam-repository (from github.com/ocaml/opam-repository), because submitting a package is actually submitting a pull-request there
  • the archive corresponding to your package release (a .tar.gz generated by GH when you push the tag), to check and determine the correct hashes.

It seems it’s the first part that fails here. What may have happened is that the initial clone (into ~/.opam/plugins/opam-publish/repos/ocaml%opam-repository) got corrupted or didn’t end properly, due to the original permission error. Opam-publish just checks for the existence of this directory to determine if it should clone or update (should probably be DIR/.git), so it’s now trying to update while there is no proper clone.

Maybe retry after a rm -rf ~/.opam/plugins/opam-publish/repos/ocaml%opam-repository ?

For some details about dune-release:

  • opam-publish is the low-level tool that will “just” take your package specification and the archive, and submit a pull-request to Github, without building anything.
  • dune-release is a much more comprehensive tool for managing your software releases (on Github only), including generating and publishing documentation, etc.

yes! that was it! thanks a lot for looking into this.
The pull request is done. :slight_smile: Let’s see how it goes now.

Filed https://github.com/ocaml/opam-publish/issues/86 to track this. Thanks!

Mmm; the journey is not over. Now the opam tests are failing

https://ci.ocaml.org/log/saved/docker-build-1cd472664ed86882cf3bcbf238d1ee43/7b21e977d7ac8eb704cc1cd93dab312790bcacaf

± The following actions failed
| - build tsdl 0.9.6

because they don’t have the SDL version required to compile tsdl. But that should not happen, because tsdl already exists in opam, right? I’m lost.

I hope to have some time to return to this soon. Not today unfortunately :frowning: