Installing camlp5 on ocaml 5.0.0 / nixos

I was actually asking a much, much simpler question. Lots of GNU-based projects are built using build-systems that rely on GNU make, autoconf, automake, etc. Are those projects going to build correctly on Windows ?

Yes. #!/usr/bin/env is broken. Please donā€™t use it! Even if it works on nixos. (For any future readers of this thread).

The simplest answer is ā€œnoā€. The slightly more nuanced answer is ā€œoften yes, but for significant numbers of users it will be noā€. The biggest problem, completely unrelated to this thread, is that GNU build tools ā€¦ especially GNU Make and its non-GNU cousin pkg-config ā€¦ rely on paths not containing spaces. Spaces are quite common on Windows as home directory names are often FirstName LastName.

But why care about GNU build tools for GNU based projects?

GNU build tools are dead. Their use was dying 10 years ago, and it has almost died today. I realize that the OCaml compiler uses GNU build tools (and that has a viral effect on our small OCaml community), but broadly speaking the C world has moved on. Real numbers:

There is some nuance in there; for example, many many people are still hand crafting Makefiles. But Iā€™ve seen other sources (share of Google searches, etc.) show similar numbers for people using the full GNU autotools stack.

In contrast, the GNU shell (bash) is thriving.

Iā€™m assuming you are making some correlation between GNU bash and GNU autotools; the data above suggests there is an inverse correlation.

1 Like

When I build complex multi-language projects (e.g. C++, OCaml, Python), I use GNU build-tools. A project like GitHub - chetmurthy/thrift-nicejson: Hack on nice json wire format for thrift for example. But also, theyā€™re a stand-in for the general issue of ā€œUNIX compatibilityā€.

Itā€™s funny: you point at the POSIX specs, which encourage using sed to customize scripts. Others have suggested that #!/usr/bin/env yadda-yadda is the way to go. None of this is tenable. B/c when you build a significant project, youā€™ll have to update many different scripts that do different things. What scripting language will the script that invokes sed be written-in? And how will that script get customized? And how will the script that customizes that script get customized?

Itā€™s farcical. Part of UNIX standards is the location and meaning of standard system utilities. Weā€™re meant to be able to rely on the location and semantics of these suckers.

1 Like

I think you are misreading the specs. The guidance is:

ā€¦ update the ā€œ#!ā€ script appropriately as it is being installed (for example, with sed)

sed is a (parenthetical!) example of implementing the guidance, not the guidance itself. In this thread the ā€œas it is being installedā€ is being performed by opam install. No sed is necessary inside the OCaml source code of opam. And that chain that ends with ā€œhow will the script that customizes that script get customized?ā€ ā€¦ which I assume was written for humor ā€¦ doesnā€™t make sense. Nor does that chain help solve the real problem that end-users face.

The narrow nixos issue in this thread is actually simple to solve by installing a hard link in the sandbox. The flow is that source files (including configure.sh) get installed into a sandbox first by opam install. I think it is mostly rsync to get to that point. Make a hard link from the canonical location of the shell (ie. getconf PATH search with a command -v sh fallback; ignore the rest of the POSIX guidance) to /bin/sh inside the sandbox. Ditto for /bin/bash. And I suspect some modification of that may work on Windows as well if it could support a bwrap-like sandbox.

The thesis here is that in order to build a package, we need a non-script tool ? You donā€™t see a problem with that? UNIX has for nearly fifty years learned how to use scripting languages for many, many administrative tasks, and now, because we want to discard the normal guarantees for where critical system utilities will live (and their semantics), weā€™re going to discard that, and switch to complex compiled binaries ?

I mean, opam is incredibly opaque and difficult-to-debug when it goes wrong. The idea of opam being intimately involved in bashing on the contents of the files of my build-tree before it builds isnā€™t reassuring.

And for what? I mean, whatā€™s the gain? This isnā€™t necessary on any UNIX system except when using a distribution that Iā€™d characterize as fundamentalist. Thereā€™s nothing wrong with application software being versioned as Nix does. But the base operating system? Versioning that in this manner is foolishness.

I mean, weā€™re not talking about even Python (2.x, 3) or Perl (5.maybe many versions) here. Weā€™re talking about bash. The Bourne shell. Itā€™s the oldest shell in UNIX. Itā€™s incredibly stable. Versioning it with a hexadecimal string in the pathname ā€¦ thatā€™s ā€¦ thatā€™s ā€¦

What Iā€™ve been told is that e.g. bash is at /usr/local/bin/bash on *BSD, so /usr/bin/env bash is the only thing that works across all Unix-derived systems (NixOS included). I havenā€™t studied the matter further.

And the bash is optional like all things among the ports. I guess that some other non Linux systems may share this characteristic.

Then it is better to depends on plain old sh.

2 Likes

Ah, /bin/sh ā€“ sure thatā€™s fine. I had assumed that @otini was suggesting that the guidance was somehow that hard-coding any shell was verboten. Iā€™m 100% cool with switching to old-skool Bourne shell.

Nice (and sad, and probably very common) story.

Youā€™ll be happy to learn that for the OCaml core system, weā€™re working through the idiocies of sed/awk on various platforms just like D. was, and that my proposals to use better tools for scripting have been blocked or deflected so far. My proposal was Python, because weā€™re in the 21st century, but even Perl would be a huge improvement on what we currently have. The current deflection of the proposal is Lua, which doesnā€™t inspire much confidence in me.

Xavier,

Omigod. I am horrified. Yes, Python would be a perfectly cromulent stand-in for Perl. And Iā€™d go further and say ā€œPython 2.xā€. Pick something old, so that itā€™s going to be really, really stable. Lua ā€¦ my mind boggles, really it does. Itā€™s got nowhere near the adoption of Python/Perl, and in the end that adoption is what matters. Everything else is secondary.

Lua isnā€™t necessarily a bad choice, imho. Itā€™s not the most popular but
itā€™s far from dead (in particular itā€™s growing in the (n)vim ecosystem!)
and itā€™s a very small, self contained, C program. I guess perl has the
same properties but is far more, erm, idiosyncratic. Python is larger
and more of a moving target.

4 Likes

Re: Perl, I will only note that Debian uses Perl extensively internally: as long as that remains the case, Perl will remain very much alive. Sure, itā€™s not en vogue for sysadmins anymore. But itā€™s still the backbone of Debianā€™s management code.

Re: Python, thatā€™s why I suggested Python 2.x and not 3.x. B/c 2.x is going to be around forever, and there will be few new features added to it, so no churn.

At this point, python 2.x is less popular, less maintained, and less widely deployed than Lua :stuck_out_tongue: .

Anyway, I donā€™t have much stakes in this game, but having to install python 2 to compile OCaml would be very sad.

6 Likes

Python 2 is also increasingly not going to be packaged out of the box on most linux distros (and macOS) etc.

It isnā€™t a question of whether itā€™s packaged out-of-the-box or not; what matters, is that it can be installed by the system package-manager in a straightforward way. Now, Iā€™m not arguing that one must stick th Python 2.x; rather, Iā€™m noting that as long as itā€™s widely-available, the fact that it is a slow-moving target is a good thing, for critical build tooling.

Or just do what 57% (which is +15% in 3 years) of the rest of the C-based world is doing for build tooling and scripting: using CMake. One binary, available from almost all package managers, works on Windows, all the scripting frills (downloading, checksumming, file/dir operations, spawning, string manipulation, etc.) and first-class support from IDEs and C/ASM compiler vendors.

Yep, its syntax sucks and it has to maintain a legacy (non-ā€œmodernā€) semantics model that should never have been adopted. But for better or worse CMake won decisively, and in 2023 the lack of portable scripting across operating systems is now a solved problem. Edit: Solved for complex projects like ocaml/ocaml, not for one-off scripts.

1 Like

One more +1 for lua. It can be easily compiled with $(CC) *.c -o lua and hence easily included in any project. It would be awesome if OCaml could shed its build dependencies.

7 Likes

Just a word of caution: as Python 2.7 is end of life since 2020 that means no security updates which makes this a no-option in many environments.

Theoretically there is Active State Python (Linux, MacOS and Windows only) which still fixes CVEs:

1 Like

A CVE on Python2 can be problematic if a Python2 application is exposed on the Web, registered as a setuid program (or listed in allowed sudo programs) or feed with untrusted content. Such exposition could allow the use of CVE the gain non authorized provileges.

If Python2 is only accessible from a shell, the users which can launch it have already the possibility to execute whatever they what. The CVE wont help them to gain more privileges. As an opam author which would want to do something bad in the destination hosts, there would be far more obvious ways to proceed than to depend on a Python CVE.

1 Like