Not just hard, but plain impossible. Indeed, OCaml uses a garbage collector that requires every pointer to point to the start of a memory block and every memory block to be preceded by metadata. So, you cannot have a pointer to an array cell, as the previous cell would be interpreted as corrupted metadata by the garbage collector. (There is one exception: the very first cell of an array. You could convert it to a reference, as its metadata would then be sensible.)
More generally, the OCaml implementation does not support “raw interior pointers”, that is (raw) pointers that point to somewhere inside an OCaml value. Raw interior pointers are occasionally convenient but a major source of complexity in runtime systems. It is possible to use a more structured representation of interior pointers (not just a raw pointer), typically a pair of a raw non-interior pointer and an integer offset.
(If many people used such “fat interior pointers” representations in their programs, we could think of optimization to make them cheaper, but as far as I know this pattern is rare enough that there is no clear need.)
type 'a cell =
| Ref of 'a ref
| Arr of 'a array * int
let get cell =
match cell with
| Ref r -> !r
| Arr (arr, i) -> arr.(i)
let set cell v =
match cell with
| Ref r -> r := v
| Arr (arr, i) -> arr.(i) <- v
And as with any general rule, there is an exception, namely mutually-recursive closures (at the expense of littering metadata all over the memory block.)