Replacing dune-release delegates

The plan is described in detail in this issue but I’ll try to summarize it here.

Natively, dune-release only knows how to publish documentation and distribution tarballs to github. It offers a delegate API so that you can implement your own publication command for documentation, tarballs or any extra item.

When publishing dune-release will parse the various URIs in your package’s opam file (home, doc or dev-repo). If the domain name for those URIs is not github.com, it will try to publish the corresponding item by invoking the delegate for that domain name (e.g. for readthedocs.org it will call the command readthedocs-dune-release-delegate) providing it a fixed set of command line arguments comprised of the path to the distribution tarball or the documentation directory and the publication message (the last section of the changelog by default).

This API is fragile and add quite a bit of logic in dune-release which is otherwise quite simple and straightforward.

What we suggest is to invert the relationship between external scripts and dune-release by providing a new command that can be invoked to obtain the information dune-release was passing on to the delegate commands.
The command would take a variable name as a command line argument and print its value on the standard output. If a script requires the path to the tarball it could use $(dune-release info tarball-path) to get it.

The idea is that now, external scripts would invoke dune-release and not the other way around.
This has the advantage to allow for extending this API without breaking existing scripts.

If you currently are using delegates or were planning on using them we’d like to know how, why and most importantly if this proposition prevents you from releasing with dune-release.

The current plan for delegates is:

  1. To provide that new dune-release info command in the next 1.4.0 minor release
  2. To depecrate any other delegate related features in the subsequent minor release
  3. To remove the delegate API in the 2.0.0 release

Alternatively, we’d be interested in hearing from people that do not use Github for their release process and learn what they use instead. We’d be happy to add support for different release processes given there is enough demand for it. For example, adding support for Gitlab or plain ftp/ssh/rsync seems reasonable.

1 Like

Is the long term plan to integrate the functionality of dune-release into dune? If that’s the case, then this work seems like an intermediate step that could be skipped.

one of my release steps is to locally copy tarball (and gpg signature) into git checkout directory, then git add and git push which gets it published in nginx directory on my server.

By the way, it would be awesome to merge the long standing PRs to opam-publish too: https://github.com/ocaml/opam-publish/pulls

I ended up on this forum page after trying to use dune release on a Gitlab project. Here is a log of what my problems are and the actions I took:

  1. The dune-release documentation does not explain how publish actually works, so I am left to guess that a certain number of forges are supported and some other are not. I don’t know if Gitlab is supported or not.

  2. dune-release distrib --dry-run tells me that “[FAIL] opam field doc cannot be parsed by dune-release”. Again, I have no idea what is the cause of the failure, how the documentation field is parsed and what are the supported formats. Adding -v -v as suggested in dune-release help troubleshoot does not help. I went ahead, cloned the sources of dune-release, grepped for “cannot be parsed” and found a testcase https://github.com/ocamllabs/dune-release/blob/master/tests/bin/non-github-doc-uri/run.t, whose name suggests that only github doc urls are supported (whatever that means).

  3. Then I ran dune-release publish doc --dry-run which fails with an actually-informative error message: gitlab-dune-release-delegate: package delegate cannot be found. Try dune-release help delegate for more information. This suggests that, indeed, gitlab is not supported. I read the documentation on delegates (which is reasonably clear) and tried searching the web for a gitlab-dune-release-delegate that someone would have written. This is how I ended up on the present topic.

  4. Reading the topic, I learn that delegates are deprecated. The last release of dune-release (v1.4.0) makes no mention of this in user-visible documentation (dune-release --help, etc.).

  5. On the topic I also find an issue report from Jérémie in January 2020 on a related problem (github project but non-standard documentation URL), which contains in particular a usability bug report (the dune-release documentation for how to set delegates is confusing and wrong), which as far as I can tell has not been fixed since.

My impression is that dune-release is not ready for usage for non-github projects. I will stop at the distrib stage for now.

4 Likes

Note: I realize now that the fact that dune-release is not expected to work outside Github is actually documented in the README of the source repository of the tool. It’s pure bad luck that I did not find this information as I was reading from the (generally pretty good) dune-release --help documentation instead.

1 Like

Indeed, at the moment dune-release only fully supports releasing on github. Adding support for gitlab seems sensible but that probably won’t happen overnight as we still want to improve the behaviour for the main use case and there’s still plenty of work there.

In the meantime you should be able to use dune-release sub commands in a script. I assume that you should be able to go as far as dune-release distrib indeed, then you can take care of the gitlab release process, tarball upload and get back to use dune-release opam pkg and dune-release opam submit, passing those the tarball download URL. If this is not possible please open an issue and we’ll make sure to add the right CLI options so it is.

In case it can help other people, I listed my own “dune-release log” (log of commands) for a package on opam-repository.

# I manually created a tag
git tag 0.2

# ask dune-release to create the release archive (giving the tag manually)
dune-release distrib --tag 0.2
  # Note: this creates an archive _build/path_glob-0.2.tbz

# upload the archive through Gitlab Pages, manually
git checkout pages
cp _build/path_glob-0.2.tbz docs/releases/
git add docs/releases/path_glob-0.2.tbz
git commit -m "dune-release archive for path_glob 0.2"
git push
git checkout main

# review the about-to-be-generated opam description for the release
#  (we need to pass --dist-uri because dune-release does not know
#   about the gitlab.com Pages URL convention)
dune-release opam pkg --tag 0.2 --dist-uri=https://gasche.gitlab.io/path_glob/releases/path_glob-0.2.tbz
less _build/path_glob.0.2/opam

# everything good, submit the PR through my local fork
dune-release opam submit --tag 0.2 -l ~/Prog/opam-repository --dist-uri=https://gasche.gitlab.io/path_glob/releases/path_glob-0.2.tbz
5 Likes