Image processing with Owl

After some time I came back to using owl and was really excited how this library evolved. The framework covers now many aspects of scientific computing & data science, e.g., it has a dataframe data type for tabular data, features various machine learning algorithms and (of course :wink: ) neural networks. So, I am wondering whether we could utilize owl also for image processing.
By image processing, I mean not only “consumer” stuff like resizing, color conversion etc., but also “scientific” things like morpholigical operations, image segmentation, or blob detection.

Of course, there is bimage, which currently is my favorite image procesing lib for OCaml, but owl’s Ndarrays would be a very powerful core for image processing with all its operators, iterators, and slicing ops. That’s also the approach of Pythons scikit-image and the OpenCV bindings, which are backed up by numpy arrays.

Are there any plans to extend owl’s application range to image processing, e.g., having a library “owl-image”? Ideally, that would mean to have an integer Ndarray datatype, but I am not sure whether this breaks the architecture of Ndarray, as it seems to rely on BLAS, LAPACK, & Eigen as computational backend (which do not support integer data types, as far as I know).
Another approach might be to keep a separate library like bimage and provide some conversion function for easy switching between it and owl, e.g., do image processing with it and use owl to train ML-models on extracted features.

On the other hand integer data is not always crucial for image processing and even bimage often calculates with floats under the hood, so adding some image IO to owl (e.g. with the help of stb_image) already enables many image processing features with owl (maybe not always as efficient, as with integer arrays, but fast enough in most cases, I would guess).

So, I have thrown some thoughts here and would be curious to get the opinion of other users and maybe the developers of owl & bimage.

3 Likes

I am sorry for the very late reply. As far as I am aware there is no explicit facility in owl for image processing, but this is not out of lack of interest, instead it more about of lack of workforce.

Since Owl uses bigarrays under the curtains, it should not be hard to implement conversion functions and integrate other interesting libraries into it. I think it would make sense to keep it a separate library on top of owl, say owl-imageprocessing, including the useful higher level, IO and conversion functions. This could also help driving the addition of new interfaces and functions to owl itself to close any apparent gap.

Integer ndarrays are already supported in owl-base (which is basically full-fledged owl without linking BLAS/LAPACK/Eigen) so if some function is missing in the native interfaces it would be great to know and to fix this.

2 Likes

I agree that a (small) separate lib would make most sense. I have started some basic IO (with the help of stb_image) here.
Thanks for pointing me to owl-base!

1 Like