Disable unboxed float array by default on OCaml 5

Ecosystem
,
1

As OCaml will be making a considerable breaking change in the “near future”, I would like to propose that we should do --disable-flat-float-array by default.

It is a weird exception and makes reasoning about performance and the memory model more complex then it should, a good example of that is arr.(0), which should be just a read, so it shouldn’t allocate data, just read the pointer and keep it on a register, but is not possible, because you cannot have an unboxed float in a variable in OCaml, so it needs to allocate an array.

This implies in a performance hit, if you’re writing performance sensitive code as now you may be making an allocation without noticing and it also makes inlining less frequent.

Some assembly examples on why unboxed flat float array makes stuff harder to reason about. With --disable-flat-float-array pointer_field and obj_field, generates the same code

1 Like
2

I don’t quite follow how --disable-flat-float-array will improve the situation. Are you suggesting that this option, instead of always boxing floating-point numbers, should just cause the compiler to flat out reject any occurrence of float array? Otherwise, if float array is still allowed, the user will keep making allocations without noticing. Except that these allocations will always happen at array creation time, instead of seldom happen at array read time. (The allocation only happens if the value is not immediately consumed by an arithmetic operation, for example.)

The only point of --disable-flat-float-array is that it allows people who never use the float array type to not pay for people who do. In that case, the option does not help them avoid allocations, since they would have not performed any in the first place. But it does improve the instruction cache locality of the code for polymorphic functions, by virtue of it getting shorter. But, if the code is not fully polymorphic with respect to the array type, there is nothing to gain. For example, the generated code for the following function is exactly the same whether flat float arrays are allowed or not:

let get arr = fst arr.(0)
3

Unfortunately, making this change is not backwards-compatible since the runtime representation of arrays change (a value of type float array becomes boxed) which will make C bindings that depend on this representation crash brutally. This is on top of making float array suddenly take up 3x as much space in the heap, etc.

Cheers,
Nicolas

4

Unfortunately, making this change is not backwards-compatible since the runtime representation of arrays change (a value of type float array becomes boxed) which will make C bindings that depend on this representation crash brutally. This is on top of making float array suddenly take up 3x as much space in the heap, etc.

It is indeed not retro compatible, but for people who actually use float
arrays there is Float.Array since 4.08, which is as efficient
(possibly more since there’s no accidental polymorphism). Polymorphic
arrays in the general case could use boxed floats and accelerate
everyone else’s code.

3 Likes
5

Except that these allocations will always happen at array creation time, instead of seldom happen at array read time

Which makes sense, if you’re optimizing your code this is exactly what you would expect, and it’s how all the other values in 'a array behave.

My point is that having a simple runtime is a really nice feature from OCaml, but each exception makes everything harder not only to understand and code for, but also to teach.

edit: also your code only works with arrays, if you have unsafe code it still cannot fuse it, like this using Obj.field which means there is other block access that are not solved by simply that.

@nojb

Unfortunately, making this change is not backwards-compatible since the runtime representation of arrays change

I know about this, it’s why I’m talking about using the future breaking change as a way to say to people “we made some breaking changes, this probably will not affect you, but FFI code may be a problem”.

But for people using a lot of float array we can make userspace tools to warn about switching to Float.Array if at some point there is a chance of the code going through unsafe code like FFI or Obj.

6

We actually use float arrays heavily for numeric code at LexiFi. @nchataing is finishing an internship at LexiFi, the objective of which was switching our codebase from float array to floatarray. From memory, some of the issues that came up were:

  • C bindings that deal with floatarray need to be manually examined to rewritten to use the right C macros for accessing “flat” arrays.
  • need to find all functions which are only applied to float array and rewrite them so that they can be applied to floatarray. This is easier for local functions, but needs to be done across modules as well. Since the signature on .mli files is just an “upper bound” on the type of a value you need to get feedback from the typechecker in order to know which functions need rewriting;
  • code that is applied to some 'a array besides float array needs to be duplicated or otherwise refactored
  • no pattern matching or array literals exist for floatarray
  • need to make sure that you don’t miss any case and get hit by a hidden regression when disabling the flat float array hack. This can happen if you have code that uses purely polymorphic operations but is instantiated at float array at runtime.
  • what do you do with third-party libraries which you cannot easily rewrite?
  • unsafe code (Obj) needs to be carefully examined to avoid segmentation faults (see eg Fix segfault when flat-float-array mode disabled by nojb · Pull Request #1083 · ocsigen/js_of_ocaml · GitHub)
  • some code may need to stay with the “boxed” type (float array) in which case suitable conversions need to be added around its uses.

Solving any one of these points in a small codebase is easy, but in an industrial setting with a large codebase, it is anything but.

For the LexiFi project, @nchataing wrote a .cmt-based semi-automatic tool to help with the translation (GitHub - nchataing/caml-migrate-floatarray), but the translation still took several months of work (both on the tool and the human input necessary to put the codebase in a form that allowed the tool to work).

Cheers,
Nicolas

4 Likes
7

This is only part of the problem. Disabling the hack incurs in an automatic large time and space regression in any code that uses float array heavily.

Cheers,
Nicolas

8

If you have unsafe code, your code is already untyped anyway. So, just add type ascriptions to tell the compiler what you expect. Here are some variants of the unsafe code of your original post:

let fast_field x i = Obj.repr (Array.unsafe_get (Obj.obj x: int array) i)
let fast_field0 x = Obj.repr (!(Obj.obj x: int ref))
9

Thank you for the detailed writeup, very interesting. I didn’t even think of the matching/literal regression.

10

Here are some variants of the unsafe code of your original post:

Yes I know, if you look on the godbolt link there is such example, my point is that probably there will be other ways to access block which should be just a read, but can actually cause an allocation. Like Obj.field.

@nojb

Thanks for the points. My main argument, is “yes” for most of them, I’m willing to spend time and money helping to fix it, but it’s a proposed breaking change. Maybe @kit-ty-kate has an opinion on how we could do such transition?

what do you do with third-party libraries which you cannot easily rewrite?

If they’re open source, we can propose the change, if they’re abandoned, maybe we should fork it anyway? If they’re closed source, then, yes that’s a problem and there is no good general solution that I can think of.

C bindings that deal with floatarray need to be manually examined to rewritten to use the right C macros for accessing “flat” arrays.

unsafe code ( Obj ) needs to be carefully examined to avoid segmentation faults

Disabling the hack incurs in an automatic large time and space regression in any code that uses float array heavily.

This argument is really generic, and it only holds on because it’s already in place, if the float array hack was being proposed we would be asking “why stop at float array?”.

But, yes I know, this only a part of the problem, this is why I proposed the tools to identify such occurrences, I didn’t propose the solution to all problems, if you don’t care about the performance, you just fix your FFI, if you care about performance you move to Float.Array.

As you showed, there is the possibility of automated tools to do part of the job which is really cool. I doubt that it will takes “several months of work” for most of the projects, it may had been the case for LexiFi, but as soon as we start to do it in mass the cost for each projects goes down with each new tool, like caml-migrate-floatarray.