The dual-shotgun theorem of software engineering

There is a long-standing problem in software engineering: how does one recognize good code? It would be great if you could point a tool at a software base and get an objective measure of how good the code is. There are tools that pretend to do this, there are textbooks that pretend to guide you toward this objective, but nothing that I care to rely upon.

I have been arguing with folks on Twitter, including Brian Marick, about what makes good code. It all started from a quote by Alexis King retweeted by Eugene Wallingford :

“The best software is the software that is easy to change.”

Hear, hear.

People who never program always tend to assume that once you have code that does something “close” to what you need, then, surely, it is not much effort to bring the code to change a tiny bit its behavior.

Alas, a tiny change in functionality might require a full rewrite.

Not all code is equal in this manner, however. Some code is much easier to change.

You can pretty much recognize bad code without ever looking at it by asking the programmer to make a small change. How long does it take to add a new field to the user accounts… How long does it take to allow users to export their data in XML… How long does it take to add support for USB devices… and so forth.

We see companies collapsing under the weight of their software all the time. You have this company and it has not updated its software in the longest time… why is that? It might be that software updates are not a priority, but it could also very well be that they have 10 software engineers hard at work and achieving very little results because they are stuck in a pit of bad software.

As a practical matter, it is simply not convenient to ask for small changes constantly to measure the quality of your code. In any case, how do you recognize in the abstract a “small” change? Is getting your corporate website to work on mobile a small or large change?

The problem, also, is that really bad code grows out of sequences of small changes. Many small changes are bad ideas that should not get implemented.

It is like a Heisenberg effect: to measure the code quality through many small changes, you have to change it in a way that might lead to poor code. What you would need to do it is to ask for small changes, have experts verify that they are small changes, and then immediately retract the request and undo the small change. That would work beautifully, I think, to ensure that you always have great code but your programmers would quickly resign. Psychologically, it is untenable.

As an alternative, I have the following conjecture which I call the dual-shotgun theorem:

Code that is both correct and fast can be modified easily.

The proof? Code is not correct and fast on its own, you have to change it to make it correct and fast. If it is hard to change the code, it is hard to make it correct and fast.

Let me define my terms.

  • Code is correct if it does what we say it does. If you are not a programmer, or if you are a beginner, you might think that it is a trivial requirement… but it is actually very hard. Code is never perfectly correct. It is impossibly hard to do anything non-trivial without having some surprises from time to time. We call them bugs.
  • Code is fast if another engineer doing a rewrite can’t expect to produce equivalent software that is much faster without massive efforts. If you are programming alone or only starting out, you might think that your code is plenty fast… but if you work with great programmers, you might be surprised by how quickly they can delete whole pages of code to make it run 10x faster.

Both of these characteristics are actionable. Good programmers constantly measure how correct and how fast their code is. We have unit tests, benchmarks and so forth. These things can be automated.

They are not trivial pursuits. It is hard to know for certain how correct and how fast your code is, but you can productively work toward making your code more correct and faster.

It is a lot harder to work toward code that is easy to modify. One approach that people use deliberately for this goal is to use more abstract software. So you end up with layers upon layers of JavaScript frameworks. The result is slow code that fills up with weird bugs caused by leaky abstractions. Or they use higher-level programming languages like Visual Basic and end up in a mess of spaghetti code.

The irony is that code that is designed to be flexible is often neither correct nor fast, and usually hard to modify. Code designed to be flexible is nearly synonymous with “overengineered”. Overengineering is the source of all evils.

Yes, you have heard that early optimization was the source of all evils, but if you read Knuth in context, what he was objecting to was the production of code (with GOTOs) that is hard to read, hard to verify, and probably not much faster, if faster at all.

My theory is that if you focus on correctness and performance (in this order) then you will get code that is easy to modify when the time comes. It will work irrespective of the programming language, problem domain, and so forth.

Some might object that we have great tools to measure code complexity, and thus the difficulty of the code. Take cyclomatic complexity which essentially measure how many branches your code generates. But if you have many branches, it will be hard for your code to be correct. And frequent branches are also bad for performance. So cyclomatic complexity brings no new information. And, of course, overengineered code is often filled with what are effectively branches, although they may not come in the recognizable form of “if” statements.

What is nice about my conjecture is that it is falsifiable. That is, if you can come up with code that is fast and correct, but hard to modify, then you have falsified my conjecture. So it is scientific!

If you come up with a counterexample, that is code that is correct and fast, but hard to modify, you have to explain how the code came to be correct and fast if it is hard to modify.

17 thoughts on “The dual-shotgun theorem of software engineering”

  1. Have you ever tried to translate a pseudocode algorithm to actual code? I bet you tried and succeeded. I failed at first:
    https://stackoverflow.com/questions/6575058/tarjans-strongly-connected-components-algorithm-in-python-not-working

    Many algorithms in papers (formally proven to be correct), usually related with computationally complex problems [>O(n log n)] use clever tricks to be fast. As a result, of both circumstances, they are usually very hard to modify.

    My guess is that partitioning a problem in smaller problems as independent (decoupled) as possible is the best approach for software that is easy to modify. For example, ideally, in the functional programming paradigm, we would have a number of pure functions that are composed to create the program. It should be straightforward to modify one of these functions or the function composition (dropping, adding, permuting, or replacing functions).

    Finally, for evaluation, we could consider programming katas, where a problem needs to be solved and then the problem is modified. It may be untenable for teams of programmers, but it may be very instructive for groups of students. As students need to learn different paradigms (object oriented programming, functional programming, and what not) the testing and comparing of the paradigms and approaches should be feasible and could be interesting.

    1. Many algorithms in papers (formally proven to be correct), usually related with computationally complex problems [>O(n log n)] use clever tricks to be fast. As a result, of both circumstances, they are usually very hard to modify.

      I should make it precise that I meant “fast software”, not “fast algorithm on paper”.

      For example, ideally, in the functional programming paradigm, we would have a number of pure functions that are composed to create the program. It should be straightforward to modify one of these functions or the function composition (dropping, adding, permuting, or replacing functions).

      Our entire civilization runs on code written in C using no fancy paradigm… no object-orientation, no function programming… The Linux kernel is beautiful and powerful code that lots of people, me included, have modified for fun or profit.

      You can write slow and unparseable buggy code in Haskell, scala, and so forth. It is great that people enjoy functional programming with pure functions and no side-effects and all that… and sure, it can work, but there are many good ways to produce high-quality code. Fortran, Go, C++, Lisp, JavaScript… all of those can serve to write good code.

      Finally, for evaluation, we could consider programming katas, where a problem needs to be solved and then the problem is modified. It may be untenable for teams of programmers, but it may be very instructive for groups of students. As students need to learn different paradigms (object oriented programming, functional programming, and what not) the testing and comparing of the paradigms and approaches should be feasible and could be interesting.

      For training, that could be great.

      1. Crappy code (and analogously good code) can be written on every language and every paradigm, but in some it is easier than in others.

        “Computer languages differ not so much in what they make possible, but in what they make easy.” – Larry Wall

        Most paradigms (if not all) have been created to make better software more easily, taking different approaches. I am not saying that functional programming is “the right paradigm” (disclaimer: I certainly like it), but I think it is an easy context to explain the point of simple, composable, and modifiable parts of code to preserve ease of modification in large codebases.

        For example, we can consider microservices and software modules (e.g. Java packages). Both allow for the decoupling of the software, but microservices make harder to violate it. (There are many more and greater pros and cons to microservices). With pressing deadlines, and Alice on holidays, Bob may be tempted to create a subclass of a class from Alice in his package, “for a quick fix to be patched later” (which may be never).

        Wrt OOP, abstraction and encapsulation in classes and objects was a nice try, it has been a successful one, but it is troublesome as it hides state, and this is a problem for parallel and concurrent code execution. Changing a program to make it parallel is normally differently difficult in OOP and FP.

        Wrt the Linux kernel, after so many years either it is wonderful or Hell on Earth. To make it wonderful it takes expertise and discipline (probably in different amounts depending on languages and paradigms, if there was a choice). A different approach was Hurd, supposedly more modular, it has not been so successful (possibly for non-technical reasons). Would it be more easy to change? We may never know.

        Wrt fast software, it can be faster by removing nonsense or by using clever tricks. I have seen lots of both. While removing nonsense makes code cleaner, clever tricks usually make it more fragile, as they are often dependent on particular context conditions that may not be generalizable or not hold in the future. Therefore, I see the correlation between fast and maintainable as weak.

        1. It is hard to build really fast systems by adding up clever tricks on top of a spaghetti. First step in making code fast is to have a great design.

          Most paradigms (if not all) have been created to make better software more easily, taking different approaches.

          In my view, programming is now a lot better than it was 20 years ago, but very little has to do with programming paradigms. For every scala programmer who swears by functional programming, you have a Go programmer who swears by the minimalistic nature of the language… and then you have tons of C++ programmers who swear by its amazing powers… and then you have the Python folks and so forth.

          They are all correct.

          1. Nobody said it would be easy. Sometimes the spaghetti is created by adding the clever tricks, until the software is unmaintainable and needs to be re-started from scratch.

            Good software is robust. Bad software is fragile. Really bad software is unmaintainable. Good software processes result in anti-fragile software: modifying the code means refactorising it, producing better results. Software that is fast because it has been optimized for years may fall in the anti-fragile group, but what we would have in such case is a survivor bias. Software that has been optimized in ways that make it more fragile is simply not around for very long.

            In short, I agree when you say: “First step in making code fast is to have a great design”, but I would reword it as: “First step in making code fast _should be_ to have a great design”. Then, making it fast could serve as a validation method of the quality of the design, but focusing on making the code fast is not going to be sufficient (or necessary) to make a good design, or to make a design good.

            In contrast, I like this quote (found in the Falcon web page, a Python web framework):

            “Perfection is finally attained not when there is no longer anything to add, but when there is no longer anything to take away.” – Antoine de Saint-Exupéry

            The classic KISS, YAGNI, etc. is likely to produce code that is easy for humans to understand and maintain, and for computers to execute, to some extent, the problem being that computers and humans do not “think” in the same ways.

            Now, different languages and paradigms forget about KISS in different ways. There are talks, books, collections of examples (e.g. the DailyWTF), and jokes[1] on that. But in a “too late” attempt at KISS, I stop digressing about that now, because there is just too much to discuss about it.

            Finally, wrt everybody being correct, probably they are more than I will ever be, this is not my field of expertise (if I have any). Nevertheless, I think that using katas to evaluate paradigms and languages could be an interesting and acceptably scientific approach to some aspects of language design and software engineering that, still to date, are more akin to a craft or an art than to engineering as usually understood. Cognitive ergonomics is hard, IMHO.

            [1] https://i.imgur.com/mh3P7CD.png

  2. Correct and fast may be a good enough approximation for many cases. Alas, current systems require bespoke micro-optimizations to eke out the last 10s of percents of performance, as you are surely keenly aware.

    Those optimizations make the code more specialized, less readable and thus less malleable. This directly contradicts your theory.

    All is not lost however. Current plain Rust code will often perform within an order of magnitude of hand-optimized C code. The strong type system and borrow checker benefit writing correct code. Together this will likely lead to code that is easy to read and change, while being fast enough for many applications.

    We don’t have enough experience yet to tell whether this is also *good* code, but current results are encouraging.

  3. Neither correct nor fast code imply good code. Why the combination of both should imply good quality?

    Also you imply that optimizations can only be added to well designed code. That is false. Optimizations can be added easily to good code and can be added painfuly to bad code. The result is fast code, irrespective of quality. Similar statement can be said of correctness.

    Accorsing to theory of computation the same software (fast and correct) can be equivalently expressed in several languages and very different ways, that accounts for the existence of bad code and good code solving the same problem in equivalent forms.

    1. I think that there is some confusion between “applying optimizations” and having “fast code”.

      In my experience, it is very hard (in general) to make non-trivial software run faster. There is a reason why we have substantial performance differences between browsers, for example. It is not that the other guys, the team that is leading the slower browser, can just apply “optimizations” to make their browser faster. Optimizations don’t come in the form of a magical powder you can spray on existing code.

      It is very hard work to make things faster, just as it is very hard work to make it correct. Both are very demanding.

  4. > “The best software is the software that is easy to change.”
    Depends on the metric of the quality (whatever marketing guys tell us, not all the software needs to be changed) – but as a default, it will do.

    > Code that is both correct and fast can be modified easily.

    It is soooo wrong that I don’t even see where to start. Being “fast” has absolutely nothing to do with being maintainable. Moreover, the-absolutely-fastest code tends to be rather unmaintainable. As for the examples – they’re abundant; take, for example, OpenSSL – the fastest versions are still asm-based, and believe me, you really REALLY don’t want to read this kind of asm, leave alone modify it… The same thing follows from Knuth’s “premature optimization is a root of all evil”, and so on and so forth.

  5. This is demonstrably false. Correct code can be arbitrarily complicated while still being correct and this is true in a non-trivial sense. As for fast code being more maintainable, on the contrary, everyone knows to avoid premature optimization precisely because it’s understood that optimization of code typically leads to more complicated, harder-to-maintain code.

    For your argument that there’s some kind of pressure that causes streamlined code to become more maintainable: on the contrary, it’s fixing the corner cases and bugs that tends to cause large code bases to become even more complicated.

    As for counterexamples: take two implementations of the same functionality, but one is implemented in C (or assembly if you want an extreme example) and the other is implemented in, say Ruby or Python or Java. The C program will undoubtedly be faster. But it will also undoubtedly be many times longer in length, and harder to maintain. I think you can easily imagine this.

    As for the notion you assume that there is a trade-off between abstraction and performance, there’s a widely pursued long-held goal with both C++ and Rust to enable abstraction with zero (runtime) cost. As for the notion that abstraction complicates things, I’d agree – abstraction needs to be coupled with actual reuse to pay for itself. But I don’t think abstraction implies a performance cost.

    Am I misunderstanding your argument?

  6. Programming is about people, not code.

    The ease with which something can be modified depends upon the person doing the modification and the knowledge that they have: Knowledge of the system, it’s architecture, the requirements that it is designed to fulfil, the processes and tools used to develop; build; deploy and maintain it; how knowledge of different components is distributed around the organisation, and the social context within which it is developed and used. Even the notions of correctness and speed are a function of the expectations that have been set; and how those expectations have been communicated.

    Having said that, the OP’s instincts are right. Every time we add a new abstraction, we add new vocabulary, new concepts that need to be communicated and understood. Highly abstract code is only easier to modify if you already understand what it does.

    This is why we as a community are so keen on idioms, standards, and the principle of least surprise.

    As a final point, I would like to emphasise the fact that in order to perform well at our chosen discipline, we need to capture, organise and communicate a huge amount of information; only some of which is code. Architecture; vocabulary; the mental model which arises from our analysis of the problem, and the abstractions and concepts which arise in the design of it’s solution – All of these need to be captured and communicated.

    The teaching of other developers is the central problem that the programming discipline aims to solve. Recognise the truth in this, and we may finally make some progress.

  7. > What is nice about my conjecture is that it is falsifiable. That is, if you can come up with code that is fast and correct, but hard to modify, then you have falsified my conjecture.

    Judy arrays come to mind.

    1. I would say TeX is a counter example.

      It is correct because Don Knuth says so. Proof by authority. Also, in use by thousands of people all over the world.

      It is fast because it was usable on computer 30 years ago.

      It is not easy to change. Just look at XeTeX and LuaTeX. Adding a feature like Unicode support is not easy.

  8. For any compiled language, in order to even correctly measure its speed and correctness, you must first compile it into a less readable, less changeable instruction set. By definition, the compiled instruction set is equally fast and equally correct (as it’s the artifact actually being measured.) But it is unquestionably harder to modify

  9. Code that is both correct and _small_ can be modified easily.
    . smallness justifies merciless refactoring;
    . smallness justifies removal of extraneous information.
    . correctness, of course, justifies test-first development.

Leave a Reply

Your email address will not be published. Required fields are marked *