we (the OCaml security) team is curious what the practice should be for opam packages that have known security vulnerabilities?
One way is to mark them with available: false in the opam-repository, which will make them not installable (apart from if you manually edit the opam file).
Especially for the OCaml compiler it is a tricky question: the recent vulnerability (in Marshal) affects everything < 4.14.3 | > 5 & < 5.4.1 – so marking these compilers as unavailable will result in lots of projects that require such older OCaml versions to not be easily installable anymore. Especially since the Marshal issue is not exposed in many (any?) applications, it feels wrong to disallow / make it hard for anyone to install these compilers.
Some ideas we have for the future:
annotate the exact function(s) that is/are vulnerabile in the security advisory (thus, you could automatically search for use of that function)
provide an “opam audit” which takes the advisory database and inspects your opam switch whether you have installed any vulnerable package
But for the short & medium term, we’d appreciate input on what to do with normal opam packages that have advisories; and also whether to treat the compiler packages differently or not.
An idea could be to have a dedicated security-message: field working like post-message:. On opam update you collate these messages for the installed packages and spit them out at the end of the update. And on install you spit them out at the end of the install if you installed a vulnerable package (or if there is any vulnerable package in the install).
As the days go along, you can take a peek at opam-audit GitHub - hannesm/opam-audit: Audit your opam switch for vulnerable packages · GitHub – a utility to figure out whether (and which) vulnerable packages you have installed in your opam switch. It uses some opam functionality, and the ocaml/security-advisories “database” underneath (together with the great jsont/cmdliner/bos/fmt/logs Buenzli libraries).
Sounds like a great idea. The only downside is that it requires changes to opam. But maybe the developers thereof are reading this carefully. Another downside is someone needs to add these security-message: fields to the opam files… But I guess that is manageable.
Somehow I don’t see myself spontaneously run opam audit, that’s the reason why I think it should rather be integrated in the commands you find yourself running routinely.
I like the security-messages field and an opam audit database! Some other ideas that don’t really work on their own:
opam supports flags: avoid-version which causes the solver to try really hard no to install that version (it’s a lighter way to do available: false, but AFAIK opam doesn’t warn when it can’t avoid a version). Users can do a cheap “audit” with opam list --installed --field-match=flags:avoid-version(the same query would also work for --field-match=security-messages:, but in any case it deserves better integration or a custom CLI as I wouldn’t expect users to guess these flags)
opam-repo could ship a patch for the affected functions to alert direct users (but not transitive ones, so it’s either insufficient or a huge pain to patch every potential endpoints)
If we had a database of vulnerable functions, Lexify’s ocamlgrep and/or Merlin’s ocaml-index can search for occurrences in a code-base, but similarly I don’t think they support querying transitive dependencies out of the box
For now we could adopt one of the ecosystem specific definitions from Go or Rust as the OCaml one? The symbol is ecosystem specific in some way anyway (e.g. we’ll probably specify the OCaml Module, Module.function, and not the mangled symbol name).
The idea is to list all exported bindings fully qualified, so that grep / sherlocode / … can be used to figure out whether something uses these bindings. Of course this is getting tricky with functors, and module aliases – but I’m sure there are tools coping with this properly.
I’m sorry, likely I’m missing something, but I don’t understand what you mean.
I’m by no means against some security-message: opam field. I’d be happy if someone picks that up and proposes it.
I’m still wondering what the best path between security-advisories and marking packages in opam-repository with such a security-message would be – but I’m sure a simple script could do that.
Of course, once this security-message: is in place, a potential opam audit could use that information instead.
When faced with a vulnerability, it’s common to evaluate whether it can be exploited in one’s context. Thus, I don’t think that preventing installation of packages is a good idea. With the Marshal vulnerabilities, it’s definitely possible to assess whether data can be loaded that way and whether that data is untrusted.
We almost don’t have memory-safety issues and have good guarantees thanks to strong typing that is more advanced than in most common languages. The number of security vulnerabilities for OCaml is lower than for other languages. Assessing exposure is doable.
I think opam audit is a good idea and while security vulnerabilities are annoying to deal with, it really doesn’t look like there is a tsunami of them. It seems a better balance to give users tools to do their own assessments and not put more load on the security team; maybe that in the future it will save time to extend tooling but it doesn’t feel like it’s currently the case.
I also think that having a warning each time opam install/upgrade/update runs is sensible. Worst case, it can be toned down later on.
An opam audit that told you if you were using a vulnerable package would bring us on par with industry standard for more mainstream language ecosystems.
An opam audit that told you if you were using vulnerable functions would be amazing and leap ahead of the state of the art.
Long version
The way security vulnerabilities are tracked in corporate environments is typically a compliance tool like Vanta inspecting a codebase and flagging package versions with known vulnerabilities. There are internal SLOs driven by things like SOC 2 in the US and CRA in Europe that specify how long after discovery vulnerable packages must be either updated to non-vulnerable versions (timeline depending on severity) or attested as unreachable or otherwise not a concern, which is often not trivial work.
OCaml is niche enough that there isn’t really a vulnerability database for the ecosystem. It would bring opam up to par if packages just tagged whether they had a known security vulnerability and the severity (ideally with a CVE link)[1] and opam audit let you know if you were using them.
It would leap ahead of the current state of the art if there were a compiler alert on known vulnerable functions. This would make reachability analysis easy, whereas it’s a real source of developer toil in most language ecosystems and has spawned many cottage security businesses that try to do impossible things like statically tell you if your Python code ever calls this function.
One caveat is that since we can’t go through already published packages and jam [@@alert] into vulnerable function definitions, the next best thing would be an opam metadata overlay.
Flagging all of OCaml <4.14.3 as having a vulnerability is not fun but it wouldn’t be unusual, and organizations that do have compliance obligations have workflows for attesting things like “we looked into it, we don’t use Marshal; we’re good”
[1] Free startup idea: vulnerability databases for the top 15-50 programming languages.
From your comments, it is not clear to me what your favourite workflow would be. I grasp you want an opam audit and suggest to have some security-message: in opam – and at the same time “Somehow I don’t see myself spontaneously run opam audit,”.
I want a security-message: opam field that allows packages to inform about the fact they are vunerable (like @adrien, I’m adamant that installation should not be prevented, there are many legitimate reasons you may still have to install a vulnerable package, if only temporarily, assuming we are not talking about malicious package compromise of course).
The security-messages: field of any installed package should be spit out at the end of opam update/install/upgrade runs.
The opam audit command should also spit out these messages (and perhaps do more) in case I want to consult them without running opam update/install/upgrade.
P.S. Regarding the function level warning I’m a bit doubtfull about that granularity. Surely you may not be affected because you are not using say Marshal but soon enough you might or a dependent can. So it seems better to me to simply move on to a non-vulnerable version. The function level granularity also ignores vunenrabilities at the system level (e.g. a conjunction of misused functions rather than functions being vulnerable themselves).
I think don’t understand your proposal. opam files are controlled by the owner of the library/package no? , or is your suggestion that the opam repository contains this information and not the package opam file?
I have concerns against using the opam file of a package to contain vulnerability information.
owner availability/willingness to update/report vulnerabilities
supply-chain attacks to the original forge where the package lives
I think vulnerabilities should be tracked and maintained elsewhere, not in the same repository or metadata as the packages themselves. If someone wants to trick users to use a vulnerable package they should compromise the package, and the vulnerability database.
The usage of opam-audit and osv proposed by @hannes looks good to me. I think this can ease 3rd party scanners to support for ocaml ecosystem.
Linking opam install and opam audit in a single run, or detail if an opam update fixes vulnerabilities is nice, and can also be implemented in opam-audit without requiring the opam file to include the information. Of course this will make the tool more complex and slow, requiring more queries ( may be to the osv free service :? )
