Are we rational? About exceptions and effects

Community
1

This post is a little bit more philosophical.

I was thinking about the fact that there’s no consensus about exceptions and whether to include them or not in a programming language. Think about Go. They decided to not add support for exceptions. Did they cite any study to support this decision, that supports the notion that exceptions in general lower the quality[1] of the ecosystem? Not that I know of. Now OCaml goes in the opposite direction - adding more ways to jump around in the code, with effects. Also no studies, no experiments. Correct me if I’m wrong.

So what are the possible interpretations of this situation?

  1. To include exceptions or not in a programming language has no effect whatsoever on the quality of the code (or very low effect). Hence there will never be consensus about it.

  2. To include exceptions in a programming language does affect quality, but there’s no scientific or logical way to find out which alternative is correct; it’s impossible to prove.

  3. To include exceptions does affect quality, but no one has yet figured out a way to prove it, either with empirical evidence or logic.

  4. To include exceptions does affect quality, but we’re not interested in finding out, for whatever reason. That is, we’re not rational, but rather tribal or inconsistent; to include exceptions in a programming language would have the same logical validity as having a favorite color.

The underlying assumption here is that if we are rational, and something is possible to prove, then there would be near consensus about a topic.

We all know there’s a great deal of “alchemy” rather than “chemistry” in software engineering still. People online appeal to authority more often than to logic or empiricism.

So what can we do about this? Do you agree with my assumptions? Why or why not? What does it mean to a field to accept that we’re not being rational?

[1] With “quality” we often mean something like “correct, mostly bug free code that’s easy to read, change and maintain”. I don’t think there’s an accepted way to measure this objectively still, though it’s often easy to agree on some basic “red flags” of low quality code.

3 Likes
2

Another possibility is that different individuals, teams, and organizations have different objective functions, i.e. they assign different weights to different things like readability, hack ability, maintainability, etc. So there isn’t a one size fits all or a Pareto optimum

Engineering is all about tradeoff. The right tradeoffs can be very context dependent, and there may notbe a single optimal point in the tradeoff space.

7 Likes
3

Maybe including the effects in OCaml is the experiment? I mean, how else can you experiment? You can do small scale stuff for sure, but at some point you need to test the effects in a large enough ecosystem with varied enough actors using varied enough styles… only then can you see if it was worth it all along?

7 Likes
4

Sure. Also considering there are no stakes, or low stakes, there’s no high burden-of-proof either.

5

I just put “empirical study of exceptions in software engineering” into a search engine, and there were a lot of results. I looked at a couple of abstracts. Relevant key terms seemed to be such as “exception handling”, “exception handling bugs”, “exception handling defects”.

Obviously, I did not investigate deeply, but it seems that there have been empirical studies. Like you, I don’t recall seeing any cited in discussions about the inclusion or exclusion of exceptions in a language. I think most of us who are users of programming languages are not empirical scientists of software engineering. However, an alternative to rigorous scientific experimentation might be gathering user reports, i.e., observational evidence from surveys or anecdotal evidence. That is less rigorous but not irrational. Also, we have evidence of adoption of languages that take one or the other approach to exception handling and/or tools designed to analyze exception flows. Software engineering is expensive and so are software bugs, so if we assume people/organizations are instrumentally rational about costs, then that might be informative unless there are too many other confounding variables.

How might a logical proof be constructed for or against the claim that exceptions affect code quality? There are some arguments available.

I believe that the original inclusion of exceptions was based on theory plus empirical observation of code/coding practice without exceptions (correct me if I’m wrong). The argument that I’ve heard was that it makes the code more readable by separating the normal flow of an algorithm from the error handling code that when mixed in would obstruct understanding. More contemporary arguments I’ve seen seem more disputable.

@smolkaj suggests that programmers or organizations directing programmers have different objectives and weights. Let zero be a possible value for a weight. Then we can just collect all possible objectives, and explore different utilities (weight * expected value). It seems we still need some data about how well exceptions do on each of the objectives/criteria. Is it possible that one of the options (i.e., include or exclude exceptions) is a net improvement over the other with regard to tradeoffs, whether across all applications or maybe just for some subset of applications that a programming language might target?

I think your idea that we might not be rational about this has plausibility. But it is a pretty complex question. It’s not clear what all of the alternatives are. For example, we have monadic error handling which seems like it can be okay for readability, but maybe it is too tedious? The code is written once but read and used many times and bugs can be very costly.

It seems plausible that in high-integrity applications, exceptions might be rationally less favored. Nvidia recently started using SPARK for some applications, which is a language that can statically check for the absence of all runtime errors and excludes exceptions. Elm claims to eliminate runtime errors and excludes exceptions. It seems in keeping with the functional philosophy since exceptions are side effects, and perhaps all/many arguments for/against side effects apply.

I’d say that the assumption that only scientific experiments and logical proofs provide rational warrants for including or excluding exceptions is the doubtful assumption.

3 Likes
6

We all know there’s a great deal of “alchemy” rather than “chemistry” in software engineering still. People online appeal to authority more often than to logic or empiricism.

So what can we do about this?

I’m also interested in that issue. I don’t know if it affects language design since many discussions about that seem to take place out of public view and I’ve sometime seen detailed rationales about language design decisions (e.g., blog posts, or for C++, a book by Bjarne Stroustrup). But there is evidently a difficulty resolving some controversies through public discussions.

I’m working on a deliberation dialogue system for my dissertation in the field of argumentation studies. Its primary intended use is for public policy deliberation, but I think it could be useful for deliberation among contributors to open source projects (e.g., about feature addition or removal and bug-fixing priorities). I hope to write prototype software early next year, and eventually, it could be nice to have some limited integration with Discourse.

1 Like
7

Good point! As well as your other text.

Medical science has a pretty strict hierarchy of what’s considered solid knowledge. Expert advice is the lowest on that scale. But, better than nothing, I guess, and, again, with different stakes in our case.

8

I’ve had similar thoughts, something like a wiki where each article instead is a (weighted) argument map (see pic), and people can give likes to arguments they like, flag fallacies, etc. Never got around to do it. Also not sure how to actually conclude anything from such a map.

image
image849×545 69.1 KB

3 Likes
9

What’s the difference between exception, effect, and continuation? My impression is that it is wrong to equate exceptions and effects.

10

From my point of view, gr-im.github.io - Basic dependency injection with objects provides a very good intuition for when to abstract using modules, functors, effects, or objects:

1 Like
11

I suspect effects are driven by academic curiosity and “because we can” - and I respect pushing the envelope. But what are realistic use cases where we need to change the semantics of existing code by using effect handlers rather than a function call or a callback? Just like call backs, yields need to be strategically placed ahead of time, too. Are we better off with several incompatible user-defined schedulers rather having one built into the runtime system? What are other use cases? Outside the novelty the effort seems rather disproportionate to the result and the examples are not particularly convincing for me.

2 Likes
12

Isn’t the ability to have direct style concurrency the most obvious, (and IMO, proven), realistic case in OCaml?

13

My perception is dominated by fights over how to implement concurrency using the new effect capability. The work that went into the runtime system is great but the access to it is almost too flexible. Is this not recreating Lwt vs. Async vs. Threads all over?

2 Likes
14

Absolutely, we’ve seen a lot of new fragmentation over the past few
years. That’s bound to happen when a feature is left for the community
to implement :slight_smile:

1 Like
15

@lindig I find your framing a bit uncharitable. While I also tend to be on the conservative side of technology, let’s not pretend we know how to build software at this point of history (amusingly something that seems to be taken for granted by the people who are coding with undulations). Because if we had done so twenty five years ago, we would now all be building perfectly inadequate class hierarchies.

3 Likes
16

I think the thing that’s missing in the current effect system is the types that go with them. IMO the one reason to go through all the effort of implementing algebraic effects, instead of an ad-hoc async functionality, or of the existing exception mechanism, would be to add functionality to the type system that keeps track of these effects, like in the Effekt language. When you do this then it becomes little different than what Haskellers do to keep track of effects, by modelling them via monads. But I would suspect that keeping track of effects, ie. In the way that Effekt does will be a net improvement to readability. Another example would be Java’s exception mechanism, which forces you to declare when exceptions are thrown.

It’s worth mentioning an almost identical question

Which @bluddy summed up with the answer “However, the plan is to integrate typed effects at some point, and this will solve every issue you mention.” So I sincerely hope that at some point Ocaml forces you to annotate your types with effects.