Why isn't all (most) programming declarative?

Community
1

It’s commonly agreed [citation needed] that declarative programming is better than imperative, since it hides irrelevant details from the programmer. This makes the code more compact, focused, and with less noise. Commonly, we care more about the “what” than the “how”. Still, most programming languages out there are not declarative in nature. Why?

You can argue that a good code-base would construct an embedded domain-specific language which is mostly declarative. In a sense, each function you write hides irrelevant details from the calling context.

You can also argue that being declarative is easier in a small domain, like SQL in database processing. Most programming languages are general-purpose, so it’s up to the programmer to construct his/her own declarative primitives, as noted above.

Is this what happens, though?

As a web developer I do of course REST APIs, and there’s no obvious reason why a REST API shouldn’t be 80-90% declarative, as a data mapping between an internal and external representation. Still I have to sit and worry about dependency injection (a clear “how” rather than “what”) and god knows what. At least on a personal level I’m failing at the “build declarative primitives” task.

I’d like to propose the possible solution that an external, rather than embedded, DSL could help. There are a couple of counter-arguments to this solution that one always hears:

  1. An in-house DSL can’t be debugged as easily, and won’t have support from IDE

  2. You’ll end up with a bad implementation of Lisp in the end.

  3. On-boarding cost will increase.

  4. Bus-factor will increase.

I have a growing feeling that accepting these arguments without further discussion can reduce programmers to cogs-in-the-machine. Also feels a bit like marketing jumbo from Java enterprise beans…

I also propose that S-expressions and Forth-like languages can be a basis of a family of external DSLs for different use-cases, languages easy enough to spin up in 100 LoC.

“Just use Lisp/Scheme/Racket” is not a valid counter-argument, because most of us are career-locked in other languages.

1 Like
2

Forth-like (FORTH itself, Postscript) is not more casual that Lisp. But Lisp parenthesis helps reading the code. (Compare 1 2 + 3 4 + * and (* (+ 1 2) (+ 3 4)). Then why privilegding FORTH ?

A difference between S-Expression and OCaml expressions is that the consistency of and S-Expression but be checked by your program, and with OCaml by the compiler. The best options depends if you can compile your DSL-module and linked with your program after each change or not.

3

Forth can read ahead in the string buffer, too, for prefix behavior, but the standard behavior is implicit stack, as you mention.

It was not my intention to privilege Forth-like over Lisp-like/S-expr. Both can be implemented with minimum amount of code and are greatly hackable from within the hosting language, hence why I mentioned them. :slight_smile: There might be other options, too, that I don’t know about.

The biggest difference between S-expressions and OCaml expressions is that S-expressions are available from all programming languages, with minimum effort. In the context of DSL, I mean.

4

There’s the famous paper by Landin “The Next 700 Programming Languages”. Lots of us -do- invent little DSLs to get our work done. But that doesn’t make things more “declarative”. It just removes easily-inferrable boilerplate.

BTW, if you’re working in Java, hail and well met (and please accept my condolences): I spent 1995-2010 in that hell, and am so glad I no longer have to live there.

I have a friend who worked at at a major video streaming company that was big into Java, and he told me that most programming there seemed to consist of figuring out how to wire together existing packages and write the requisite XML incantations to get it all working. “XML incantations” being a repeated sore point in his rant (which, honestly, I thought was fully deserved, I really felt for him).

What I mean is, it’s possible that you might not like a world full of DSLs of varying provenance, quality, and syntax. It might not be fun. And while, sure, you and I might want it all to be in s-expressions, we’re likely to get XML (or haha, maybe something worse) instead.

2 Likes
5

Isn’t removing boilerplate part of being declarative? You remove noise that’s not related to the “what” at hand.

6

Yea, we do have a never ending list of configuration languages, like ini-format, JSON, XML, YAML. S-expr is the only one that can do key-value lists easily, and at the same time contain pockets of logic, if need be. XML becomes horrendous mostly when it’s abused beyond being a data representation.

Maybe embedded DSL is the proper solution for more declarative code, but at the same time, many large languages limit the developers opportunity for this. OCaml and Haskell support custom and overloading (?) operators, but many does not, which means most [citation?] developers are stuck with a limited tool set. External DSL is a way to escape this trap, and give all developers the same tools. With the drawback being quite obvious, as you say. :slight_smile: I don’t know.

7

What I meant, by adducing Landin’s paper, is that we’ve been designing languages for a long time, and every designer of a new language comes along saying that it will make programming declarative. But it never does. That’s really all I meant. We remove (or automate the production/inference of) more and more boilerplate. But really, programs aren’t declarative: they are executing on a model that we keep in mind as we write.

AFAIK, There are only two (hence famous) examples of “non-procedural” programming languages: spreadsheets and SQL. Everything else, there’s an order of evaluation, a memory model, and while we might ignore it a lot of the time, the minute we care about performance or debugging, we have to take account of those things.

7 Likes
8

SQL is often advertised as an example of “declarative language” but this, in my opinion, only applies to the querying part. And in some sense, the query part of SQL manages to be declarative because the data specification part is very constrained (relational model). So data specification is very much not declarative here, you cannot say (famous example) “I have movies and actors and directors and movies have a title and maybe a subtitle etc…”. You have to bother with primary/foreign keys, unique constraints, indexes, the size of your scalar types, the collation of character data etc… Only when you have done everything by the book (and manually created indexes, fine-tuned the autovacuum because your rate of updates does not match the default policy, …) will you have your very declarative SELECT query return results in a reasonable time.

1 Like
9

I think you’re confusing “declarative” with “mind reading” here. If you want to actually model your sentence above in a consistent manner you rapidly find that you need to be precise in your specification of what data comprises an “actor” and exactly what relationships are expected or allowed between actors and other elements.

And if you’re bulk processing text without knowing its character encoding or collation you’ve just got it wrong.

Now - is SQL overly verbose and the result of design be several committees? Yep, you’ve definitely got a point there.

10

grin For sure, “declarative” is a word with different meanings to different people.

this is why I chose the word “non-procedural”: in SQL queries (and updates are classed as queries), you do not specify the -algorithm- (except via “hints” which are completely nonstandard and vary from implementation-to-implementation) but only the relational expression whose result you require. One must certainly specify access paths, indices, constraints, in order to get non-ridiculous performance. But when one submits a query, one is -not- specifying the order of joins (and which types, over which indices, etc) that will be performed.

It is only in this sense that I would describe SQL as non-procedural: you do not specify the -procedure- for your query.

But for sure, others might disagree.

11

OK, so maybe I have a new hypothesis why this is the case: You can’t be both declarative and general-purpose. So you need at least two languages when solving a problem domain - the general-purpose host language, and the external (or embedded) domain-specific language.

I’d like to do a comparison between external and embedded DSL and argue that external gives a range of tools that embedded never will, but I don’t have the time right now. Maybe next week. :stuck_out_tongue:

12

Let’s take SELECT * FROM table1 JOIN table2 ON table1.id = table2.id…

In OCaml, it is a short hand for cross_product table1 table2 |> filter (fun a b -> a.id = b.id)

However, that would be terribly inefficient. It is the interest of SQL: its very restricted use permits some optimisation (looking which table(s) have an index, which one is the smaller, etc). Then a better implementation is to have a new function join table1 "id" table2 "id".

The M language is interesting: it is a functionnal language like OCaml, but (like SQL) specialised for using tables. Then there are multiple short hands:

each [a]+[b] is a shorthand for (in OCaml) : fun row -> row.a + row.b. And there are multiple jointure/concatenation, etc. functions (Table.join t1 key1 t2 key2 for example). Note, that M has a Record.fieldfunction (Record.field r "a"returns r.a) that would be impossible with OCaml with its type erasure. This permits to give Table.join the field names as a string.

13

Perhaps the smaller the domain, the easier it is to write a descriptive language that solves or describes it? But how would one measure the size of a domain? Number of verbs, or nouns? And how would you know this size at the start of a new project?

15

I guess that with ADT, we could model any descriptives languages.

But it would not be easy to read or write. Think table1.a =table2.b would be written Eq (TableCol(“table1”,”a”), TableCol(“table2”,”b”))

SQL appears to be a quite complex language. After a SELECT, you can name the column with or without indicating the table, column can be also aliased… or not.

16

I am not sure there is any common agreement and I am also not sure if one could make such generalizing statements like this.