Defining standard OCaml development lifecycle processes

Learning
21

@jjb this is a good callout. so, for “clone, install project deps” process, you perhaps are claiming that opam install ./foo.opam --deps-only is the BKM. Perhaps it is. Candidly, this feels obvious in retrospect :laughing: . I have not habitually created a foo.opam as part of bootstrapping a project, perhaps because I’ve been torn/unclear on how or when to produce it. I probably should start doing this first, 100% of the time.

This signals to me that a BKM on “bootstrapping a new ocaml project” ought be captured too.

  • drom has drom new <pkg> (generates .opam)
  • npm has npm init -y (generates package.json & package-lock)
  • cargo has cargo new <pkg> (generates Cargo.toml & src/*)

looking at only platform tools, i think it’s probably? agreeable that there is not a project initializer that preaches a blessed ocaml structure/config.

  • opam init is for getting opam ready, but not for starting an ocaml project
  • dune init is for adding stuff to an existing ocaml project, not creating an ocaml project

I don’t mean to suggest that a CLI cmd is required for bootstrap, but I would suggest that a well-known, minimal set of artifacts perhaps should define a blessed, MVP bootstrapped state.

  • ./foo.opam
  • ./dune
  • ./dune-project
  • ./src/lib/foo/dune
  • ./src/lib/foo/foo.ml
  • ./src/bin/foo/dune
  • ./src/bin/foo/foo.ml

An inconspicuously large amount of OCaml practices can be learned and derived just from seeing those files initialized, especially if produced by the platform. You infer that it is ocaml standard practice to have dune builds for each component. You infer that you partition libs/bins in separate folder hierarchies. You infer that dune can link projects in dissimilar folder hierarchies together. You :crossed_fingers: infer that foo.opam is genuinely critical to a package development workflow, vs something you maybe tack on later.

2 Likes
22

You can do dune init proj as well as dune init {lib,exe} which gives something like what you want.

Differences:

  • No ml files in lib
  • (slightly oddly) no dune-project
  • Adds a test directory

I use it relatively frequently. Personally I think it would be nice if it additionally generated (by default):

  • A dune-project with (generate_opam_files true) and enough other info to make that work
  • A .ocamlformat file
7 Likes
23

When init was implemented dune would create dune-project files when it was first run, and we didn’t want the dune-project being generated in two places. But, since then, the dune-project file has become more useful (and so prepopulating it with certain stanzas now makes sense) and recently the functionality to automatically create the file on first run was removed.

I opened an issue to rectify this: Make `dune init proj` create the `dune-project` file · Issue #4367 · ocaml/dune · GitHub

I agree that creating a .ocamlformat would also be nice!

5 Likes
24

I’m happy to see the trend toward a high bar for in-place metadata modifications. It is important to this bar to be high, such as installation and security issues as you note. In the past it has been problematic when things like a version constraint is added without a version number change to one of a package’s dependencies (e.g. due to a bug in some interaction in some cases being discovered post-release). This has broken builds before, even with lock files. So I think it is good to be clear that it is good to retroactively add constraints to keep packages installable, but it is IMO not good to add constraints that might prevent installation due to bugs in packages, as there can be clients that happen not to hit the bug and it is not necessary to break their build.

2 Likes
25

> mkfile
P9 represent!

Apologies for adding to the pile in the thread, but it would be truly joyful if we had dune automatically handle setting up a cross-compilation environment or at the very least walk us through the steps.
Perhaps opam was to simply be passed a C cross toolchain and to use that for compiling the target’s OCaml toolchain – streamlining the cross compiler building process instead of being limited to what ocaml-cross has to offer.
I may be a little daring with my hopes, but it’d be even better if OCaml eventually dropped the need for a C compiler entirely and went the Go way. I do realize though just how much resources go in the development of a language like Go to be able to enjoy such niceties. Evidently, Go’s compiler is already 1.5M lines of its language and twice the amount of commits in nearly half the age of OCaml. There’s just so much Google money flowing there.

Would be really nice when I can say dune release -x linux/arm7 and on the other side get a tarball ready to install on target though… One can dream!

2 Likes
26

Check out Nix :slight_smile: Antonio Monteiro has set up a static cross-compilation (musl) to release for ARM. https://twitter.com/_anmonteiro/status/1368960740623667205

2 Likes
27

Reason Mobile with esy can do it, you need patches for C deps, but OCaml deps using dune mostly “just works”. But for armv8, armv7 is problematic because you need cross word compilation, which is not supported, so you need a OCaml compiler running on x86_32, that’s okay for Linux and Windows but macOS doesn’t allow it anymore.

1 Like
28

On the third point

run default executable

If I put this in my Dune file:

(executable
 (public_name foo)
 (name foo))

I can then run my program with

dune exec foo

Maybe not everything you’re looking for, but at least it avoids having to remember the path to the .exe file. I can’t remember if dune init sets this up for you or not.

I guess it would be nice to be able to set up a default executable so you could just say dune exec or dune run without having to specify the name.

3 Likes
29

Absolutely. As a novice in OCaml and coming from mainly an Elixir and Rust career (at least in the last 5 years) I find myself wondering why we need all the extra typing of commands.

After silently spending some time on this forum I gathered that the core contributors are (a) more concerned about backwards compatibility and (b) leaving any future tooling innovation in the hands of the users (most likely because they have their hands full enough already).

I’ve been trying esy in the last few days and I am mostly pleased with it. It’s pretty far ahead (although f.ex. symlinking the latest target directory to _build so legacy scripts and IDEs don’t complain about _build not existing would be very nice!). I’d also love to have a formatting rules file generated when initializing a project as @shonfeder said (but not having it leading to fall back to global defaults is also good).

In general, having a cargo-like experience in OCaml is definitely going to make me a much more enthusiastic adopter (as opposed to carefully dipping my toes and often backing up slightly disappointed that things that are streamlined elsewhere require special care here).

I tried drom. First impression: way too many files. But that might not be so important. I’ll re-evaluate it at some point. It’s also doing Git commits which is a big “nope”.

Finally, just having some .gitignore file that encompasses a big chunk of the build tools ecosystem would be very nice, e.g. a file that includes _esy/ and node_modules/ and <your_project>.install from the get go will help clear confusion since I had to discover what’s ignoreable and what isn’t by myself.

Thinking of it, this might be something that even a complete novice like myself could contribute to esy.

2 Likes
30

i’m coming from elixir and rust too and opam + dune is a lot more flexible than cargo or mix. lack of namespaces is a bit painful but, you can get used to it.

1 Like
31

Preferences, I suppose. Not sure what you mean by more flexible but often times I find the tooling in my way and not enabling me (which is something that mix and cargo do).

But I am not here to argue preferences.

32

On you esy you don’t need to actually symlink it, you can add { "esy": { "buildsInSource": "_build" } } to put in _build

1 Like
33

That worked beautifully! Thank you.

I believe this should be put in the docs. For me it all started because Emacs’ tuareg-mode complained. :slight_smile:

So maybe a section named “Compatibility with LSP implementations and IDEs” or some such?

1 Like