I’ve been looking for detailed documentation on Jane Street’s Core library, going beyond what’s auto-generated from the source (most functions don’t include (** ... *) comments, so names and signatures make up most of the documentation), and what I can learn from the source itself. There’s a lot that one can figure out just from names, signatures, and easy experimentation, but some things in Core are difficult to understand without further help.
Does any further documentation exist? I’ve looked repeatedly, but Google is not all-knowing, nor are links in websites, so I may have missed it.
If not, is there a plan to produce more documentation for Core?
Example: I’m trying to figure out how to use Core.Interval to create my own interval module with a custom compare for matrices, but I’m confused about how to do that based only on the generated documentation that’s online and the source. I’ve read the Functors chapter in RWO, which includes a series of implementations of simple interval modules, but what’s in Core is a fairly complex set of related modules, and it doesn’t map into any of the variants in that chapter in a way that’s obvious to me.
We’re definitely working on this, and are in fact looking to hire a technical writer to work on this kind of thing.
We’ve mostly fixed the documentation comment thing. We now have build-time checks that make sure that we’ve correctly formatted doc comments in our mlis. You can see that here:
But this module is indeed hard to understand, in part because it’s split between interval_intf.ml and interval.mli. We can hopefully start improving this now that our public release is all building with the new odoc doc generation tool.
Anyway, here’s the key bit you want:
module Make (Bound : sig
type t [@@deriving bin_io, sexp]
include Comparable.S with type t := t
end)
: S with type bound = Bound.t
Which you can find here:
And is sadly not documented. This is a functor for creating a new interval type, and it’s pretty easy to use as long as the thing in question already satisfies Comparable and supports bin-io and s-expression conversion. i.e., you can write:
Great–thanks very much. I have a lot of appreciation for how much JS has invested and continues to invest in Core as a public resource.
OK–that’s great. Thanks.
The other thing that I don’t understand what role is played by the various subsidiary modules–Gen, S, S1, etc. No need to explain now, but I hope that this sort of thing will eventually be documented.
That’s helpful.
It looks like it may be the “supports bin-io and s-expression conversion” part that will make Interval difficult for me.
(Not sure if this should be moved to the Learner category at this point.)
My guess was that the following would work using the owl’s double-precision dense matrix type and my own matrix compare function. (The standard polymorphic compare doesn’t do what I want.)
module Mat_interval = Core.Interval.Make(struct
type t = Owl.Mat.mat
let compare = Comparemat.compare
end);;
This produces a Signature mismatch error with many “required but not provided” items; this probably isn’t suprising–I doubt it’s worth reproducing the errors here. I think the problem is that I haven’t included anything related to bin-io and sexp conversion.
It sounds as if I would have to implement quite a bit of additional functionality in a wrapper around an owl matrix type to use Core.Interval with it. If so, that’s OK. I can always use chapter 9 in RWO as a model for a simpler kind of interval that would meet my needs. (There are always tradeoffs. I understand that Core.Interval is designed to integrate into larger systems in particular ways.)
You can get the functionality you need for this by using Comparable.Make, but I’m not sure it’s worth the trouble in this case. I wonder if we should adjust Core.Interval.Make to not require the full comparable interface, because just having the compare function should be enough, in principle.
Thanks much. I don’t know what the tradeoffs are, but having to use Comparable.Make doesn’t seem like much trouble. Just tried it, but I’m not sure I’m using it correctly. In utop with core v0.9.0:
# module MC = Core.Comparable.Make(struct
type t = Owl.Mat.mat
let compare = Mat.compare
end);;
Error: Signature mismatch:
...
The value `sexp_of_t' is required but not provided
The value `t_of_sexp' is required but not provided
It looks like your comparison function also needs to define sexp derivations so that they can be serialised to/from s-expressions despite being abstract types.
The easiest way to do this is for Owl to implement it, so you may want to raise an upstream issue requesting this feature in the Owl GitHub. Alternatively, you can also define your own sexp_of_t and t_of_sexp functions in your module signature.
OK, good, thanks, that confirms what I thought. I mentioned the error only because Yaron Minsky’s post seemed to suggest that all I needed to implement was compare. I wouldn’t want to implement the sexp functions at this stage, and don’t feel it would be important enough to ask Owl’s developer Liang Wang to do it. It’s really no problem, though. I can just define my own Interval module starting from the examples in RWO chapter 9 or other examples. This is a very exploratory project at this point, so I don’t want to implement a lot of extra functionality that might be unnecessary for me in the end. I would make use of well-tested library if possible, but it’s not a requirement.
Hi Marshall, how do you want to compare two matrices? Do you mean the common element-wise comparison (such Mat.equal, Mat.greater …) or do you refer to some specific comparison (such as determinant)?
Hi Liang. I want an interval between m1 and m2 iff each element of m1 is less than the corresponding element of x. Interval modules like Jane Street’s and RWO’s use compare to decide whether to create a proper interval or an empty interval. The default OCaml compare will return -1 as soon as it finds the first pair that satisfies numerical <, so I wrote a compare function for matrices that returns -1 or 0 if all elements are < or all corresponding elements are =, respectively, and returns 1 otherwise. (It doesn’t really make sense conceptually to do this–to return a 1 if two matrices have corresponding elements that are both less than and greater than–but it has the effect of producing an Empty interval, which does make sense. I’ve also thought about raising an exception in this case. The underlying relationship between matrices here is really a partially order.)
I just noticed that < in Owl seems to return true iff all elements are < than corresponding elements, and then found the documentation for less, etc. Right–Mat.less, etc. are conceptually what I’m building into compare. Is there a compare function in Owl I missed? I looked for a compare definition but didn’t find it. It may be that I just didn’t understand the source code or odig output.
(I should probably say that what I’m doing with Owl and with interval modules is using them to learn about OCaml, because I think I’m probably going to need these things later. I’m still studying the mathematical framework that I expect will be the basis of some computer models once I understand enough. I’ll definitely need matrices, and Owl seems very nice for my purposes, but I’m not sure about intervals between matrices, but by the time I need them, or something else, I think I’ll have a good enough understanding of OCaml to do what I want.)
OK, I see. You are right, there is no comparable function in Owl at the moment, but it is quite easy to make one according to what you described (not the most efficient solution though), e.g.,
let compare x y = Mat.(
if x < y then -1
else if x = y then 0
else if x > y then 1
else failwith ":("
);;
The Error in your previous code comes from missing sexp_of_t and t_of_sexp signature, as Anil mentioned. I can have a look at into that later this week. I think I can include compare function in Owl with a more efficient solution in the next release.
Thanks Liang. Adding the sexp functions would be very nice if that seems worthwhile otherwise.
I realized today that I could have written a simple compare that way, but I’d already written a more efficient version as an exercise. At present, it would just return 1 in the failwith case, but I know that’s non-standard.
(I’d be happy to contribute a version of my compare if you want, modified to raise the exception in the fourth case, but I expect that you would write a more elegant and/or more efficient version pretty easily.)
Of course, all the contribution are welcome and appreciated! You can open an issue in Owl repo for this feature and we can discuss the code there. I just released Owl.0.2.5 in opam minutes ago, so this feature will be included in the next release.
