How can I install OPAM package from private Github repo?

[scjung]

I have a private local OPAM repository that contains OPAM packages served in private Github repo.

$ cat my-opam-repository/packages/my_package/my_package.1.0/OPAM
..snip...
url {
  src: "https://github.com/>>redacted<</archive/refs/tags/v1.0.tar.gz"
  checksum: "md5=e3e1b89c2bf1df417f79057ecf261e9e"
}

$ opam remote
...
 1 private git+file:///Users/me/my-opam-repository
 2 default https://opam.ocaml.org

When I try to install the package, OPAM fails to download the source archive in the private repository.

$ OPAMFETCH='/usr/local/bin/wget --header "Authorization: token >>redacted<<" -O %{out}% %{url}%' opam install -vvv my_package

...snip...

[ERROR] Failed to get sources of my_package.1.0: Download command failed

#=== ERROR while fetching sources for my_package.1.10 ===========================#
OpamSolution.Fetch_fail("https://github.com/>>redacted<</archive/refs/tags/v1.0.tar.gz (Download command failed: \"/usr/local/bin/wget --content-disposition -t 3 -O ... -U opam/2.1.4 -- https://github.com/>>redacted<</archive/refs/tags/v1.0.tar.gz\" exited with code 8 \"2023-03-22 14:40:43 ERROR 404: Not Found.\")")

...snip...

I’m aware of OPAMFETCH env variable, but as you may see the output above, OPAM does not honor this variable. I found a similar Github issues - `opam init` does not honour `OPAMCURL`, `OPAMFETCH`, ... · Issue #5108 · ocaml/opam · GitHub -, but it doesn’t seem to be related to my problem.

Am I doing something wrong here?

[kit-ty-kate]

@scjung

I found a similar Github issues - opam init does not honour OPAMCURL, OPAMFETCH, … · Issue #5108 · ocaml/opam · GitHub -, but it doesn’t seem to be related to my problem.

I disagree, I think this is related. Have you tried with opam master which solved this issue? I’m able to reproduce your problem with 2.1.4 but not with master.

You can try it out using:

git clone https://github.com/ocaml/opam
make -C opam cold
sudo install ./opam/opam /usr/local/bin/opam

[scjung]

OPAM on master branch indeed respects OPAMFETCH variable!

But sadly OPAM splits OPAMFETCH value by whitespaces, and it causes that --header argument is not parsed as a whole string like the below.

+ /usr/bin/wget "--header" "\"Authorization:" "token" ">>>redacted<<<\"" "-O" "/home/scjung/.opam/4.12.1/.opam-switch/sources/coreutil.1.10/v1.10.tar.gz.part" "%{url}"
- --2023-03-24 10:31:27--  http://token/
- Resolving token (token)... failed: Name or service not known.
- wget: unable to resolve host address ‘token’

I had to write a wrapper script passes --header argument to wget and set OPAMFETCH to call the wrapper script instead.

Thanks for the help

[dbuenzli]

@scjung

I had to write a wrapper script passes --header argument to wget and set OPAMFETCH to call the wrapper script instead.

@scjung please file an issue so that others won’t have to jump through the same hoops in the future.

[scjung]

I thought the Github issue I mentioned above (`opam init` does not honour `OPAMCURL`, `OPAMFETCH`, ... · Issue #5108 · ocaml/opam · GitHub) is the same issue here and the fix will be released in upcoming v2.2.

@kit-ty-kate, Should I file a new issue?

[dbuenzli]

@scjung

But sadly OPAM splits OPAMFETCH value by whitespaces,

That’s a new issue ^, when you allow to specify tool invocations as environment variables, simply splitting on spaces is user hostile – as you now have witnessed :–(

One should perform at least some basic tokenization, see e.g. here.

[scjung]

@dbuenzli

That’s a new issue ^, when you allow to specify tool invocations as environment variables, simply splitting on spaces is user hostile – as you now have witnessed :–(

I see. Filed a new issue here.

Tokenize OPAMFETCH value properly, not splitting by whitespaces