[ANN] Bytesrw 0.1.0 – Composable byte stream readers and writers

Thanks for your comment @c-cube. I’m glad you like the design.

Having a cursory look at them, I think they are quite different, there’s no notion of ressource in bytesrw and I don’t see any form of buffer ownership discipline that enables the buffer sharing between readers, writers and their clients that bytesrw allows.

Also note that in bytesrw you can’t choose how much you read and writers can’t choose how much they get written to, see this part of the design notes.

I think it’s good not to introduce the notion of buffering at this level. It can be quite economical. For example in jsont we work directly on byte stream reader provided slices, needing only a temporary 4 bytes buffer for those cases when an UTF-8 encoded character spans two slices.

I don’t think that’s written in the design notes. So here it goes.

In a corner of my head I remember dealing, a long time ago, with a stream system that would allow empty slices. It was quite brittle and messy to coordinate because not everyone agreed on what it meant. Some stream implementations would simply do not expect them and fail or run into infinite loops, other would interpret them as a signal for flushing and other would interpret it as the end of stream. It was unpleasant.

That doesn’t mean you should not allow them. But it indicates that if you allow them you should be very clear about their meaning.

Another thing I learned by implementing FRP systems for a language with side effects is that you should not invoke interconnected networks of functions gratuitously if nothing is happening. The semantics of bytesrw’s reader and writer is that you only get to observe finite parts of streams, slice by slice. Does it make sense to observe an empty slice or an arbitrary number of them between two bytefulls ? Not much in my opinion – except if you want to take an asynchronous stance but bytesrw definitively takes the direct style/blocking stance. It seems to me that for this semantics the natural interpretation of empty slices is: there are no more bytes in this stream; end of stream (bonus point: we don’t need an option or another signal to denote it).

Now deciding this doesn’t mean that it solves all of your problems. You no longer have the problem of each stream implementer interpreting empty slices in incompatible ways. However you have another problem which is: it’s very easy to produce empty slices if you are not careful. That’s the reason why the slice API is very explicit in distinguishing operations that may return empty slices (_or_eod suffixes, option values) from those that do not and loudly raise Invalid_argument if they do.

2 Likes