Error: Library "netcgi2" not found

I try to make a little CGI program. I have installed the system “libapache2-mod-netcgi-apache” on Debian 11.

utop does load easily netcgi2 (#require “netcgi2”), but using netcgi2 in the dune file produce the following error :

File "bin/dune", line 4, characters 17-24:
4 |  (libraries core netcgi2))
                     ^^^^^^^
Error: Library "netcgi2" not found.
-> required by _build/default/bin/main.exe
-> required by _build/install/default/bin/essai

But core seems to be found.

Curiously, the cma is in /usr/lib/ocaml/netcgi2/netcgi.cma. (not netcgi2/netcgi2.cma). There are no cmx… I guess I should ask dune to build a bytecode version. But even with (modes byte byte_complete), I have the same error.

One thing that may happening is that utop is configured to look into your system directory (/usr/lib/ocaml) but dune is using an opam switch (I suppose that this would happen if you’re using utop from the debian package).

Then what should be canonic way to use Debian provided packages AND opam provided packages in the same project ?

FInally, I have installed ocamlnet… and now :

Error: Required module `Condition' is unavailable

the Condition module is not something I have explictely called.

AFAIR, it is something related to threads…

I suspect the answer is: “don’t do that”. I never do. If I’m going to be using opam at all, I make sure that there’s no ocaml/ocaml packages installed via dpkg/apt on the machine.

I have removed Debian packages… same issue.

And now, I don’t have the Apache mod !

First, a clarification: I doubt that your bug is due to mixing packages between opam and Debian – if that were the problem, you’d see “interesting” linkage errors, not “can’t find file”.

Do you have a repo that reproduces your problem (or a cut-down version of your problem) that you can share? These sorts of problems are usually straightforward to debug.

Condition moved from systhreads into the stdlib recently, so I suspect there is a versioning issue.

Note, that with an environment with no Opam addition,

ocamlfind ocamlc -package netcgi2 main.ml

produce the error :

root@ks3001154:/tmp# ocamlfind ocamlc -package netcgi2 main.ml
ocamlfind: [WARNING] Cannot read directory /usr/local/lib/ocaml/4.11.1/stublibs which is mentioned in ld.conf
File "main.ml", line 1:
Error: Required module `Netcgi_cgi' is unavailable

The first line is :

let process (cgi: Netcgi.cgi) =

I have a switch with ocaml 4.14.0, ocamlnet installed.

$ which ocaml
/home/chet/Hack/Opam-2.1.2/GENERIC/4.14.0/bin/ocaml
$ ocamlfind ocamlc -package netcgi2 -c main.ml
$ cat main.ml
let process (cgi: Netcgi.cgi) = ()

How is your ocaml installed ? your ocamlnet ?

I think that one has to use either
ocamlfind ocamlc -package netcgi2 -c main.ml
which compiles the module but does not produce an executable by calling a linker, or
ocamlfind ocamlc -package netcgi2 main.ml -linkpkg
which produces an executable, and uses the -linkpkg option to ask ocamlfind to include its packages during linking. (I’m not sure I remember why this option is necessary, but this is the design of ocamlfind.)

I’m guessing here, but the fact that the ocamlfind ocamlc line failed early in the compilation process (during typechecking) I suspect means that we have an environmental problem.

Frederic, if you have a sample repo you can share, I’d be happy to figure out how to make it work, and then you could try that. That is to say, you have a “bad”, and you can either (1) figure out how to transmit that bad case to others (a “known bad”), who can debug, or (2) you can send the repo and we can figure out how to make it work (a “known good”).

The problem with #1 is that it can be a little involved, b/c you have to guess at what parts of your environment you need to transmit along with the code. With #2, that work is transferred to the person helping you, who can give you precise instructions on how to set up your environment.

Yes, this works. But the mod_netcgi seems to be broken (see Bug #1558649 “installation of libapache2-mod-netcgi-apache cause...” : Bugs : ocamlnet package : Ubuntu … same issue with Debian 11)

I am thinking in switching to Dream… however, I can’t find the dream_eml filter which interpret templates (dream is installed with opam).

I decided to look into this a little further. Some observations:

  1. the Debian apache module package you reference libapache2-mod-netcgi-apache is built from the ocamlnet sources. The same sources that are used to install the netcgi2 package.

  2. the Debian package is of an older vintage – and assumed that you were installing OCaml via Debian.

  3. So this means that everything needs to be installed via Debian, not some bits with Debian, and some bits with opam.

  4. Now, I think you mentioned that you were having trouble with starting apache and getting the netcgi2 module loaded ? This might mean there’s a bug with the way that that module is packaged – hence, a bug in the Debian packaging.

  5. Now, the opam package doesn’t build the Apache connector. Perhaps it should.

  6. So one could build the Apache connector directly – this probably requires installing some of the Apache dev packages

The root of this problem isn’t opam-related: if you were using Python, you could find the same problem if you installed some Python package with Debian, and then tried to install other python packages that were related to it, in a virtualenv or conda environment.

OK, I think I’ve figured out at least part of what is wrong.

Conclusion:

The Debian apache-netcgi module appears to be built wrong (doesn’t export symbols) and in any case, the config-file is out-of-date and doesn’t refer to the right symbol (even if it were exported)

I’ve never used netcgi2, but apparently you create a findlib package with your CGI in it – no idea how that’s supposed to be structured – and then by specifying it in the NetCgiRequire line, you get that module loaded and then I guess somehow it gets connected to the right URL-path prefix.

NOTE WELL: I built my own netcgi2 SO module, inside an opam switch. So that means that I didn’t need to use the Debian-installed OCaml.

===========================
Explanation of debugging

  1. As I wrote before, if you’re going to use the debian-created netcgi2 Apache module, that’s going to link in the Debian-created OCaml, and that means you’re committed to the Debian installation of OCaml. Probably there’s some way to mix-and-match with another OCaml installation, but it’s going to be dicey.

→ And note that this kind of thing isn’t only endemic to OCaml: it also happens with Rust (and is well-documented, btw). So when you use Rust, you need to make sure that everything was compiled by the same version of rustc.

  1. So I installed the debian-created libapache2-mod-netcgi-apache and observed the same error you did:
$ sudo /usr/sbin/apachectl configtest
apache2: Syntax error on line 146 of /etc/apache2/apache2.conf: Syntax error on line 1 of /etc/apache2/mods-enabled/netcgi_apache.load: Can't locate API module structure `netcgi_apache_module' in file /usr/lib/apache2/modules/mod_netcgi_apache.so: /usr/lib/apache2/modules/mod_netcgi_apache.so: undefined symbol: netcgi_apache_module
Action 'configtest' failed.
The Apache error log may have more information.

Notice I used configtest, which is enough to elicit the error.

  1. Next, let’s “nm” the SO file … and we find that it exports no symbols. None at all. Now, I’m no longer an expert on how shared-libraries work, but I think that this is a problem

→ THIS IS PROBABLY A BUG in the Debian module

  1. So then I downloaded ocamlnet and built it (this entailed installing apache2-dev, etc) so I could build:
./configure -enable-apache -apache usr/sbin/apache2

Note that this isn’t the default build-setup. Since Gerd wrote all this stuff prior to the arising of opam, he obviously wouldn’t set ocamlnet to automatically try to build the apache module … when apache might not even be installed on the machine.

  1. And then ran “nm” on the built SO file:
$ nm -B mod_netcgi_apache.so | grep module
000000000002a040 D netcgi_module
  1. And so we see the entrypoint of this SO. Let’s modify the module-load command above (in /etc/apache2/mods-enabled/netcgi_apache.load to use this SO):
$ cat /etc/apache2/mods-enabled/netcgi_apache.load 
#LoadModule netcgi_apache_module /usr/lib/apache2/modules/mod_netcgi_apache.so
LoadModule netcgi_module /home/chet/Hack/Camlp5/src/lib-ocamlnet3/code/src/netcgi2-apache/mod_netcgi_apache.so
NetcgiRequire netcgi2-apache

And now when we run that configtest:

$ sudo /usr/sbin/apachectl configtest
[Sat Jan 14 14:44:00 2023] [Netcgi_apache_mod] No such ocaml package: netcgi2-apache
AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 127.0.1.1. Set the 'ServerName' directive globally to suppress this message
Syntax OK
  1. I wonder what that error-message is?
~/Hack/Camlp5/src/lib-ocamlnet3/code/src/netcgi2-apache$ grep 'No such' *
grep: mod_netcgi_apache.so: binary file matches
grep: netcgi2-apache.cma: binary file matches
grep: netcgi_apache.cmo: binary file matches
netcgi_apache.ml:                                   "No such script.")) in
netcgi_apache.ml:                     "No such script.")) in
grep: netcgi_apache_mod.cmo: binary file matches
netcgi_apache_mod.ml:     log_error ("No such ocaml package: " ^ name)
netcgi_apache_mod.ml.in:          log_error ("No such ocaml package: " ^ name)

So it’s an error-message from OCaml code, saying that some OCaml findlib module isn’t present. This means that we’ve successfully booted-up the OCaml container under Apache, and it can’t find the application.

I’ll stop here.

1 Like

One more tidbit:

Whatever OCaml installation you use to build your netcgi SO module, you need to ensure that the environment variables (e.g. OPAMROOT, but all the environment variables set by the OPAM initialization process or whatever other process sets up your environment) are set correctly before you boot Apache. Since the standard Debian apache install is booted by systemd, that might make it tricky to set up the environment correctly for an opam-built installation, I don’t know. Obviously all of this is completely straightforward, if you run Apache yourself, and that isn’t hard to do, so if I were going down this path, this is what I’d do. After all, if I’m actually developing a website, I’m not going to be putting my static content in /var/www, nor my config files in /etc/apache2 – those are systemwide locations, and I’ll want to keep all my build-outputs in local directories, if nothing else so I know what inputs to a running Apache are being used.

Another: I was unable to find much information on how to use this “netcgi2”. Maybe Gerd has better documentation someplace that I didn’t find. But I suspect that improving the documentation would make most of these teething problems (OK, not the “debian package is broken” problem) go away.