OPAM 2.0 and the "source" external dependencies

Community
1

Hi everyone,

Just a small announcement for OPAM packagers here.
According to the new external dependencies (depexts for short) implementation in OPAM 2.0 [1], the "source" depexts will not be supported anymore.

The "source" depext allowed you to execute any arbitrary script as root and is usually used for installing external dependencies from source when those are not available on some or any distributions.
This is not possible anymore with OPAM 2.0. The rational is that it is too much of a security risk, the user using opam depext has no control over what is executed and the installation process is quite arbitrary.

The last bits of "source" depexts will be removed soon enough from the opam-repository [2]. If you are maintaining any library that needed to use the "source" depext in order to comply with the CI system or anything of that matter, please do not use it and add a post-messages in case of failure telling how the users might want to install the missing dependencies in case they are not available in any or too few distributions.

[1] https://github.com/ocaml/opam/pull/3074
[2] https://github.com/ocaml/opam-repository/pull/11058

5 Likes
2

Thanks @jpdeplaix! We’d like to hear from anyone for whom the source tag is particularly important, but we’re not aware of anyone so far.

Note that opam2 also has nice support for sandboxing builds, so removing this particular escape hatch is even more important to ensure secure, reproducible builds.

3 Likes
3

IIRC, it’s no longer possible to install external dependencies from source, and there’s no alternative?

That’s a bit painful. I think for example about antiquated (but free) travis environments where the packages you need are just not available and you want a fully automated install for continuous integration (so a ‘post message’ just doesn’t cut it). Not to mention the bleeding edge where the dependency you need has not yet been packaged by the distro. I guess we’ll have to be creative then.

4

This particular opam depext mechanism is being deprecated, but there is always the option of doing a source-based installation via another opam package dependency. In the CI example, you could set an environment variable that would cause the source-dependency to be installed rather than just checked for.

Is there a particular example of the current use you’re putting it to that we could look at?

5

install external dependencies from source

Technically that’s what opam does, so you always have the option of packaging the external dependency as an opam package (nothing says they have to be in OCaml!). The only real limitation of this is that you’ll have to install into the opam switch.

6

Good thing. I feel safer if this feature is gone.

1 Like
7

I think there are 2 separate issues here:

  • source dependencies (probably there because the distro doesn’t offer it)
  • using/allowing sudo

In the case of orocksdb it does the following:
https://raw.githubusercontent.com/domsj/orocksdb/0.2.1/install_rocksdb.sh

#!/usr/bin/env bash
set -x
echo $(gcc --version)

VERSION=3.12
shared_lib_file="/usr/local/lib/librocksdb.so.${VERSION}"
if [ -e $shared_lib_file ]; then
    echo "$shared_lib_file exists"
else
    echo "cloning, building, installing rocksdb"
    git clone https://github.com/facebook/rocksdb/
    cd rocksdb
    git checkout tags/rocksdb-${VERSION}
    make shared_lib
    sudo make uninstall
    sudo make install-shared
    sudo ldconfig
fi

It’s easy to change the sudo parts into something like

PREFIX=${prefix} make uninstall install-shared

with ${prefix} pointing inside the current opam switch. But, what needs to be done with LD_LIBRARY_PATH & friends ?

8

Before the sudo problem, there is the problem of arbitrary code execution.
It doesn’t require to be root to harm my data/files.
And, arbitrary code execution can lead to privilege escalation, which gives you root access without sudo.

9

That’s right, but sudo is the main reason for not just bundling the source dependencies as a depopt in opam itself. If we redesign the feature to make it easier to bundle the ability to compile and install the source library dependency from within opam itself (including adjusting the LD_LIBRARY_PATH so the locally installed library is found), then this eliminates the need for privilege elevation entirely.

However, this will require a rethink of how we structure these source dependencies in a way that’s version controlled from within the opam-repository itself (something not true at the moment as they fetched from external sources). I’ll create a tracking issue to discuss it in the opam issue tracker before merging the removal of the sources tag PR.

10

Only if your OS is exploitable. If you really fear this, you cannot trust the ocaml packages opam installs neither, unless you personally verified their code. Demanding opam repository maintainers to do this for you is a bridge too far.

11

This is being mitigated in opam2, which has support for sandboxing individual package builds by using the local operating system support. For example, see the wrap-*.sh commands for using Linux namespaces. It should be possible to also use osx sandboxing in a similar fashion.

So once you wrap your external code in an opam package, the threat of arbitrary code execution is substantially lowered.

12

Ftr, we (openvstorage) can perfectly live with source being removed. I think it were the maintainers of opam themselves that pushed us a bit in adding an (optional) rocksdb shared lib installation side effect in order to get orocksdb accepted.

So please go ahead, and thanks for the heads up.

1 Like