Packaging an executable for homebrew install for non-OCaml users

Would anyone have an example I could look at of a homebrew “formula” for an OCaml program? I’m looking to distribute an executable to colleagues who are not OCaml users.

(There seem to be lots of moving parts and I’m not sure which one has stopped moving!).

Many thanks.

1 Like

All official Homebrew formulas are located in this GitHub repo: https://github.com/Homebrew/homebrew-core/tree/master/Formula

The interesting part is install.

Examples using make:

Dune:

Opam:

Examples using dune:

Coq:

Menhir:

Examples using Opam and something else:

Semgrep (Opam & Python):

Reference

Homebrew’s Formula reference: Formula Cookbook — Homebrew Documentation

3 Likes

Thank you! Very helpful.

These examples all depend on Homebrew actually building the OCaml code, but what I would like to do is distribute a binary in a way doesn’t require the user to have OCaml installed at all (and, most importantly, doesn’t depend on the state of the user’s opam switch). Does anyone have experience with distributing executables in this style?

An example from the Haskell ecosystem is Pandoc, which can be installed instantly via a package manager without needing to be built with ghc.

Sorry if my question is super naïve, I am totally new to packaging and distributing executables, etc.

https://discuss.ocaml.org/t/generating-static-and-portable-executables-with-ocaml may be relevant. I’ve followed the steps described in the ocamlpro blogpost with success in the past (though I’m running linux).

In the past, I used homebrew to install statically build executables.

If your CI already produces binaries of your tools in e.g. GitHub releases (see the post recommended above), you can do the following:

  1. Create <my_username>/homebrew-tap repository.
  2. Create the formulae inside the repository.

So, if you have have a tool called my_tool, you create the my_tool.rb file in the repository root with the following content:

class MyTool < Formula
  desc "Short description of my tool"
  homepage "https://github.com/my_username/my_tool"
  url "https://github.com/my_username/my_tool/releases/download/v1.0.0/my_tool_asset_name_for_osx"
  sha256 "<sha goes here>"

  bottle :unneeded

  def install
    bin.install "my_tool_asset_name_for_osx"
    mv "#{bin}/my_tool_asset_name_for_osx", "#{bin}/my_tool"
  end

  test do
    system "#{bin}/my_tool", "--version"
  end
end

Then, people who want to install your tool, can run the following commands:

$ brew tap my_username
$ brew install my_tool

There’re multiple blog post on the internet that describe how to do it. So I believe you can find them if you want to learn more details :slightly_smiling_face:

1 Like

I think I wasn’t clear about the part that is difficult. I agree it is not hard to use Homebrew to distribute binaries. What is highly non-trivial is to actually create these static binaries using the OCaml toolchain (I suggest trying yourself before asserting that it is easy for me to find a working blog post — you will see that the problem is not simple). For instance, there is a blog post on the OCamlPro website but the instructions for macOS do not actually work anymore. If you are aware of “multiple blog posts on the internet” that currently work, then I suggest you link them here. I have found several that exist, but none of them seem to solve the problem satisfactorily.

Other people have gone much more sophisticated routes, using tools like Esperanto and Cosmopolitan. This seems potentially promising, but I have been unable to actually execute any binary built using these tools, so this seems also non-trivial. Very open to constructive suggestions on how to proceed.

:slight_smile:

Most software is not built and distributed as static binaries (e.g. almost anything in a linux distro package manager is dynamically built), and it’s a bit painful to achieve no matter the programming language or build tool.

If we look at pandoc, since you picked it as an example:
The binary homebrew installed on my mac seems dynamically built to me (for what I understand of binaries on mac os…)


The package on Ubuntu depends on 7 libraries (Ubuntu – Details of package pandoc in kinetic)

So depending on how many platforms and versions of OS you are planning to support, you could write the homebrew formula that install the dynamically built executable you got out of dune + ensure the libraries you depend on are also installed.

In fact, pandoc is statically linked (see this documentation: Pandoc - Installing pandoc).

I’ve checked these links. Unlike the ubuntu version, their linux build is indeed static.

The Macos build however is not, it depende on dynamic libraries. (but depending on dynamic libraries if you are guarantee they will be available in a version you are compatible with is fine)
They may have some libraries statically linked in the mac version, but that would require checking the details of their build process.

It’s not that I want to pick a fight on what pandoc is doing, but static builds can be a pain, and it’s sometimes ok to take some shortcuts.
Which libraries does you binaries depends on? How these already on homebrew and could easily be added as a dependency in your formula?
This may lead you to a simpler answer than try to generate a static build, and could be as useful to your users.

1 Like

I am also new to packaging and distributing executables, but I have looked into this recently. I think the easiest way to accomplish this is to use a CI platform like Github Actions to build the executable in each target operating system. I am using the library raylib-ocaml, and the author has an example game that can be built for Windows, Mac, and Linux using this config. I was able to adapt that to my own project pretty easily, and I’ve tested the Linux and Mac executables and they worked fine.

3 Likes

Re: building executables for different targets in CI. The canonical example of this for OCaml is Unison. You can see their releases have multiple assets: Releases · bcpierce00/unison · GitHub

The GitHub Actions pipeline is quite involved (yours will probably be much simpler): https://github.com/bcpierce00/unison/blob/master/.github/workflows/CI.yml

You may have mixed results depending on how many libraries your application actually depends on. For example, I downloaded and ran the Unison binaries on my macOS 12.6 system. The unison CLI tool ran fine, but the unison-gui tool errored:

dyld[19384]: Library not loaded: '/usr/local/opt/freetype/lib/libfreetype.6.dylib'
  Referenced from: '/Users/yawarquadiramin/Downloads/unison-2.53.3-macos-x86_64/bin/unison-gui'
  Reason: tried: '/usr/local/opt/freetype/lib/libfreetype.6.dylib' (no such file), '/usr/local/lib/libfreetype.6.dylib' (no such file), '/usr/lib/libfreetype.6.dylib' (no such file)

I guess I don’t have a required library on my system.

EDIT: whoops, actually I downloaded the wrong architecture. The GUI in Unison-2.53.3-macos.app.tar.gz works fine.

2 Likes

I happily produce static binaries for MacOS for cpdf, though not available in homebrew.

The difficulty is that, whilst it’s easy to codesign a MacOS executable, it is not currently possible to notarize a pure executable (as opposed to a
package). So users have to do a dance:

2 Likes

Btw. I’m always amused how the CI providers managed to convince all the hard core nerds out there that builds are to be defined via hard to debug yaml files with an agonizing feedback cycle occurring through frigging webpages :joy:. The notion of a nice, restricted, prompt on a machine seems to have entirely vanished.

Having a macos install I can easily do builds locally for macos and linux via podman, but I’d be interested in a windows cli workflow too.

Ages ago I had some kind of workflow via vagrant but that must have bit rotten badly.

I’m wondering if anyone (or, most likely, the usual suspects) would have some kind of nice workflow to share that can be invoked to do that locally on a macos machine.

4 Likes

Well, whoever provides the compute power, is the king. We may be able to get back some of that strong deterministic build magic back with Nix though. Although I’m not crazy about its dynamically-typed config language.

1 Like

@JohnWhitington Can you share some tips about how you have achieved this for cpdf?

I use a makefile with OCamlMakefile, which spits out a static executable by default. I assume dune does the same.

To ensure backward compatiblity, you need to build using MACOSX_DEPLOYMENT_TARGET, both the OCaml switch, and your code i.e:

MACOSX_DEPLOYMENT_TARGET=10.12 opam switch create 4.14.1
MACOSX_DEPLOYMENT_TARGET=10.12 ./mybuildcommand

Codesigning is done with the codesign command line tool. I suspect you need a paid Apple Developer account for this.

You can also make universal (arm + intel) binaries, by combining the two using another command line tool before running codesign. I don’t think homebrew uses universal binaries though.

I’m not sure if homebrew provide codesigning and notarization at their end, or if you’re supposed to do it. You’d have to ask them.

1 Like

Thanks! To clarify, afaict dune does not emit static executables by default and there does not appear to be any working built in way to do this for macos targets with dune (there was a discussion on the ocamlpro blog about this, but their solution is (1) complicated and (2) no longer works). (I think this is not dune’s fault, btw, but here we are.) There is a simple flag you can pass to produce static executables with dune for linux targets, however.