Hi Everyone,
I’d like to write a simple utility written in OCaml that interfaces with opam. Currently I am communicating with opam by issuing terminal commands. This seems horribly inefficient. I would like to directly interface with the opaml libraries, but I cannot find the functions that do what I want.
To make this more concrete, suppose that I want to execute the command opam install xyz. How would I issue this command directly to opam, bypassing the terminal entirely.
Thanks!
First of all welcome to the community 
!
I might not be the most qualified to answer such a question but hopefully this can help. The opam repo does expose useful libraries which you can reuse to get some of the functionality. The best places in my opinion to see this is action are the “Opam plugin” libraries for example opam-publish which depends on opam-state, opam-format and opam-core.
At the same time, unless you are doing quite complicated things that require low-level access, I don’t think calling opam install xyz as a terminal command would be that inefficient. You can use the very good Bos library for a nice combinator interface (and cross-platform support) to build these calls as for example the opam-compiler plugin does.
Hopefully this helps, but more well-versed opam hackers might be able to jump in and provide better information.
Happy Hacking :))
Sorry I also realised you might want to just know if it is possible, and I think it is. The problem is that something as seemingly simple as opam install irmin is actually very complicated involving global state and switch state. To understand how opam installs something for example, you can read the source code for the opam commands in the client – for example the install command.
Hopefully you’ll agree that it is very complicated but not impossible to reproduce. For example on my machine the following will install irmin:
let install () =
OpamClientConfig.opam_init ~current_switch:(OpamSwitch.of_string "4.11.0") ();
let pkg = OpamPackage.Name.of_string "irmin" in
OpamGlobalState.with_ `Lock_none @@ fun gt ->
OpamSwitchState.with_ `Lock_write gt @@ fun st ->
OpamClient.install st [ (pkg, None) ]
let () = ignore (install ())
This was fast and dirty (I happen to have a 4.11.0 switch) just to show that is ~possible~ but I’m not going to recommend it because I don’t understand enough of the internals of opam to make sure it doesn’t break lots of invariants etc. and leave someone’s opam in a broken state. For these complicated functions I still think using the command line is your safest bet.
For simpler functions, say to list the names of packages based on the available opam files in your current directory, then using these exposed modules makes sense as it doesn’t alter the internals of opam.
let opam_files () =
let lst = OpamPinned.files_in_source (OpamFilename.Dir.of_string ".") in
List.iter
(function
| Some f, _, _ -> print_endline (OpamPackage.Name.to_string f) | _ -> ())
lst
To get your head around the exposed functions I’d recommend installing the libraries from the opam repo and then using odig doc opam-<xyz> (odig) to have a look at the documentation (I don’t think it is currently online anywhere).
Hi @patricoferris, thanks for your explanations. This is already a very helpful starting point for me!
Well explained and understood by all, thanks! 
@patricoferris @LasseBlaauwbroek
The piece of code is a good use of the opam lib. OpamClient.install does the necessary checks, so nothing can be broken from this call. Just have in mind to use the optional arguments that usually are given via cli.
On the switch setting, OpamClientConfig.opam_init is able to select itself the current switch to use, but if you want to be sure, you can use the current_switch optional argument. Note also that all OpamClienConfig.opam_init optional arguments can be used to set some (other) cli arguments.
If you want to install more complex expression, you can also use OpamFormula.atom_of_string function. It is same format than cli PACKAGES.
For example:
let install () =
OpamClientConfig.opam_init ();
let pkg = OpamFormula.atom_of_string "irmin<2.0.0" in
OpamGlobalState.with_ `Lock_none @@ fun gt ->
OpamSwitchState.with_ `Lock_write gt @@ fun st ->
OpamClient.install st [ irmin ]
You can try opam lib in your favorite interpreter, there is a .ocaml_init on top of opam repository (from this initialisation, it won’t change your current state at it isn’t a right lock).
About API documentation, they are on opam website.
Hint: If you set the OPAMDEBUG environment variable, you’ll be able to follow what opam does when you launch your program (or set it with opam_init 
).
This is all very helpful. Is it true that as long as I run opam_init (), it is generally safe to execute any of Opam’s functions just based on their signature (within common sense of course)? Or are there other invariants that I need to keep in mind?
opam_init will configure the default environment (select current switch, download tool, repository, and all default values for all cli arguments, etc.).
For main functionnality, it is safe to OpamClient functions to launch them via opam lib, safe in the way that all checks are done, the new state is written, etc.
Concerning function types, state lock are embedded in state types, so you can see from type if the function could change your current state.
On states, if you want to do several actions, keep the state that you always get from a function, it is written on disk, and that return value permits you to not reload it from state and always have it up-to-date.
If you use some internal functions (e.g., OpamAction ones, or changing files via OpamFile), it could result on a non consistent state, you’ll need to understand more opam internal to know how to use them. But most of functions are safe to use, and there is always a more high-level function that performs an action.
If you don’t want to take the risk of breaking your current environment, you can initiate opam_init in your program with a different opam root.
I can’t really say you more without knowing more exactly what do you want to do.
Thanks, I’ll try to implement some functionality now, and report back if I run into trouble.
(Just in case you are curious, I’m implementing a utility to support Tactician, a machine learning tool for Coq).
So I’ve now written some code against ocaml-client, and I have to say that this is actually surprisingly smooth! Props on the API!
The only thing I wish for is a way to silence some of the messages that Opam is printing to the stdout. I would rather shield my users from some of the more confusing operations that I’m performing (modifying configuration files etc).
Glad to hear ! 
There is for the moment no way to silence opam output, the --quiet option is only for disabling verbose mode is set somewhere.