So what I gather from your link is that you prevent inlining from the libraries that are used by the library you use but that you do not use directly.
So if you develop an app you can always add the libraries whose usage is abstracted as direct dependencies and that should do it.
It’s rather unpleasant that flambda changes the already shady libary compilation model of OCaml (or maybe now that I think of it, it was maybe always that way). One more reason not to rely on flambda so far (and to have a good library standard upstream ;–).
On paper (implicit_transitive_deps false) is an amazing feature.
However, in the current state it has a big downside: type aliases defined in transitive dependencies become abstract types.
In practice, it means that if I have library a defining:
type t = int result
a library b defining a simple function
val f : unit -> A.t
now, in my project to be able to do:
match B.f () with
| Ok 0 -> ...
| _ -> ...
I am now forced to explicitly depend on a too but a is never used explicitly and if the b library stops using it at some point i have [*] no other choice but to keep this dependency, monitor closely (pain) the library and drop the dependency when b does it too.
A few months ago on Slack I asked around if adding a new -implicit-include parameter to ocamlc could be considered and if I remember correctly there was a bit of a push-back against it from people saying it could be a linter phase in dune instead, but also some other people who were saying “a PR would be welcome”, but I didn’t have the time nor motivation to do it (I could reconsider if someone else wants to collaborate)
[*]: or worse i want to support several versions of this package using several different, sometimes incompatible, libraries.
As far as I’m concerned a is used explicitely in your code. b did not make its use of a abstract and your code is using definitions of a, concretely you depend on a’s definition. It is a direct dependency of your code: a change in a will break your code.
Regarding monitoring the uses of the dependencies, I guess it could be solved by improving dead usage warnings. This is not a problem specific to this one: forgetting to remove dependencies when they are no longer used is quite common.
Another pain point with implicit_transitive_deps is the runtime
dependencies of ppx (e.g. ppx_deriving.runtime), which you need to add
explicitly again.
Note I’m not asking to do that in your apps. I’m asking you to do that on the libraries you publish on the opam repository.
Somehow the current state of affairs makes publishing on the opam repository quite an unpleasant experience. I’m fine with fixing rev-deps constraints if I introduce incompatible changes. I’m less happy about having to fix (possibily transitive) dependency under-specifications.
For this particular case, I think this can be fixed in dune. Could you open an issue for that if there isn’t any?
I’m failing to see where this is an explicit use.
To give a more concrete example:
currently if you use lwt and use any of the function returning a result type, it requires the user to use have a useless dependency on the result library even if you don’t use it.
the same thing is true for things like some part of conduit when using cohttp. Calling Cohttp_lwt_unix.Server.create with ~mode:(`TCP (`Port ...)) will fail to compile if you haven’t imported conduit-lwt-unix.
There are things like that everywhere and it makes everything more painful than it should be, especially when hitting those types of error messages:
File "test.ml", line 33, characters 4-9:
33 | | Ok () ->
^^^^^
Error: This pattern matches values of type ('a, 'b) result
but a pattern was expected which matches values of type
(unit, Capnp_rpc.Exception.t) Result.result
Result.result is abstract because no corresponding cmi file was found in path.
and even more painful when considering being nice and supporting a wide range of library versions.
Forgetting to remove an explicit dependency is one thing. But forgetting to remove a dependency that isn’t being referenced anywhere is another. In the former case you can at least detect that during patch review because such removal is explicit, where-as in the latter case, you have no idea why you even need this dependency and you have to remove them one by one blindly waiting for a compiler error, to see if you actually still need it, or even to remember why you need it (a comment would only cover one case at a time, but a library would have several such cases that you wouldn’t be able to discover).
I feel like I’m repeating myself but all I can say is that your code ends up using the result library definition of the result type. That’s the constructors you will match on, for me you are using the result library.
I don’t understand your point. Precisely, once all dependencies are explicit, it should be easy to see if they are no longer needed, this includes if in your first example if b no longer relies on a and all your usages of a were tied to it, a will no longer be used and this will be reported.
Actually Flambda tries to be the least surprising here. If you depend on values from another compilation unit, Flambda will look up these values in the corresponding cmx. If the cmx is not there, you will get a warning (that you can choose to disable) but compilation will still work.
The only difference with Closure is that since Flambda simplifies the code it has inlined, it often ends up trying to look up dependencies of your dependencies, so missing cmx files are more common.
But you can already have chained lookups with Closure:
(* a.ml *)
let foo x = x
(* b.ml *)
let bar = A.foo
(* c.ml *)
let baz = B.bar 0
In this example, compiling c.ml will lookup a.cmx even though C doesn’t directly depend on A.
So it seems if we want to be able to have precise and honest library dependency specifications that respect the language’s abstractions we really need two sets of includes, those that are used for typing the code and those that are used for compiling/optimizing. Currently everything is conflated under -I.
I believe that the library linking proposal (that you’re very familiar with) should help in this case. I haven’t heard about it in a while though; do you need help with moving the implementation forward ?
Not really, modulo a few details to pin down on the OCaml side the implementation was mostly there (though I guess it likely conflicts nowadays). There were however disagreements with other parties involved on the eco-system aspect which brought the whole endeavour to a halt.
Given all the hours I invested in it that were crushed for what I personally think are unwarranted reasons (which I won’t detail on this forum), I won’t work on this unless there is a strong upstream commitment to go for it. And with the latter now busy with multicore for the foreseable future I think we can effectively say: the whole thing is dead in the water.
But I still think the OCaml toolchain should mandate and understand a notion of library and library dependency. For examples if the archives know which library names they depend on, it could be used to good effect, e.g. to report better errors messages in the cases @kit-ty-kate mentioned above.