How to add new tag (bootstrapping issues)

Learning
1

Hi everyone,

I have been moving tags around to make room for a new byte_tag in order to distinguish between bytes and strings in the Ocaml runtime and (eventually) try to add hash-consing for immutable strings. I understand that this requires a bootstrap so I followed the bootstrap instructions in BOOTSTRAP.adoc but failed on the last step (make bootstrap) with the error:

make[1]: Entering directory '/mnt/c/Users/sarag/ocaml_hc'
make -C stdlib \
   OCAMLRUN=../runtime/ocamlrun all
make[2]: Entering directory '/mnt/c/Users/sarag/ocaml_hc/stdlib'
../runtime/ocamlrun ../ocamlc -strict-sequence -absname -w +a-4-9-41-42-44-45-48-70 -g -warn-error +A -bin-annot -nostdlib -principal -safe-string -strict-formats  -nopervasives -c camlinternalFormatBasics.mli
../runtime/ocamlrun ../ocamlc -strict-sequence -absname -w +a-4-9-41-42-44-45-48-70 -g -warn-error +A -bin-annot -nostdlib -principal -safe-string -strict-formats  -nopervasives -c camlinternalAtomic.mli
echo '#!/usr/local/bin/ocamlrun' > camlheader
Fatal error: exception Invalid_argument("index out of bounds")
make[2]: *** [Makefile:213: camlinternalAtomic.cmi] Error 2
make[2]: *** Waiting for unfinished jobs....
Fatal error: exception Invalid_argument("index out of bounds")
make[2]: *** [Makefile:213: camlinternalFormatBasics.cmi] Error 2
make[2]: Leaving directory '/mnt/c/Users/sarag/ocaml_hc/stdlib'
make[1]: *** [Makefile:839: library-cross] Error 2
make[1]: Leaving directory '/mnt/c/Users/sarag/ocaml_hc'
make: *** [Makefile:273: coreboot] Error 2

However, just before running make bootstrap I tested the following program:

let closure = ((+)1)
let () = Printf.printf "closure tag: %d\n" (Obj.tag (Obj.repr closure))

and compiled it with:

./boot/ocamlrun ./ocamlc -I ./stdlib foo.ml

and ran ./runtime/ocamlrun a.out and ./boot/ocamlrun a.out which gave outputs of 245 and 247 respectively, so the tags were updated.

The changes I made were minimal (here is the code). I also tried bootstrapping with changes which actually used the new tag (code) and got the same error.

Any help would be greatly appreciated!

Sara

2

I did not look in detail but my suggestion would be to add a new tag at the end, after all existing assigned tags, instead of renumbering the whole bunch of existing tags; this will make things easier as it is backwards compatible change.

Cheers,
Nicolas

3

See also the following thread, and in particular @dra27’s helpful comment: Bootstrap problem when moving Cont_tag to be after No_scan_tag · Issue #11182 · ocaml/ocaml · GitHub

4

There’s also (referenced similarly in that issue), this set of commits which deal with some of the more terrifying details of moving tag numbers around. Certainly, if the tag number for Forward_tag is altered, you’ll definitely need to adopt the intermediate hack to disable the use of Forward_tag in the runtime[1]

I’m guessing that you’re using 250/252 so that 0xFD is a tag mask for “stringish” values?

I don’t think this is true - at marshalling, the special tags which can be marshalled are converted to different constants; the actual values for the tags I think are entirely opaque (there is a whole series of relationships between their values which must be maintained)

  1. Forward_tag is what a lazy value gets updated to after it has been forced - it simultaneously records that the result block has been forced and points to the computed result. The temporary hack causes lazy values still only to be computed once, but instead of using Forward_tag they’re updated to be fun () -> result. That temporarily means that Lazy.is_val doesn’t work, but this doesn’t matter as the compiler doesn’t use/rely on that. ↩︎