You cannot scale creativity

As a teenager, I was genuinely impressed by communism. The way I saw it, the West could never compete. The USSR offered a centralized and efficient system that could eliminate waste and ensure optimal efficiency. If a scientific problem appeared, the USSR could throw 10, 100 or 1000 scientists at it without having to cajole anyone.

I could not quite understand why the communist countries always appeared to be technologically so backward. Weren’t their coordinated engineers and scientists out-innovating our scientists and engineers?

I was making a reasoning error. I had misunderstood the concept of economy of scale best exemplified by Ford. To me, communism was more or less a massive application of the Fordian approach. It ought to make everything better and cheaper!

The industrial revolution was made possible by economies of scale: it costs far less per car to produce 10,000 cars than to make just one. Bill Gates became the richest man in the world because software offers an optimal economy of scale: it costs the same to produce one copy of Windows or 100 million copies.

Trade and employment can also scale: the transaction costs go down if you sell 10,000 objects a day, or hire 10,000 people a year. Accordingly, people living in cities are typically better off and more productive.

This has lead to the belief that if you regroup more people and you organize them, you get better productivity. I want to stress how different this statement is from the previous observations. We can scale products, services, trade and interaction. Scaling comes from the fact that we need reproduce many copies of the essentially the same object or service. But merely regrouping people only involves scaling in accounting and human ressources: if these are the costs holding you back, you are probably not doing anything important. To get ten people together to have much more than ten times the output is only possible if you are producing an uniform product or service.

Yet, somehow, people conclude that regroup people and getting them to work on a common goal, by itself, will improve productivity. Fred Brooks put a dent in this theory with his Brook’s law:

Adding manpower to a late software project makes it later.

While it is true that almost all my work is collaborative, I consistently found it counterproductive to work in large groups. Of course, as an introvert, this goes against all my instincts. But I also fail to see the productivity gains in practice whereas I do notice the more frequent meetings.

Abramo et al. (2012) looked seriously at this issue and found that you get no more than linear scaling. That is, a group of two researchers will produce twice as much as one researcher. Period. There is no economy of scale when coordinating human brains. Their finding contradicts decades of science policy where we have tried to organize people into larger and better coordinated groups (a concept eerily reminiscent of communism).

We can make an analogy with computers. Your quad-core processor will not run Microsoft Word four times as far. It probably won’t even run it twice as fast. In fact, poorly written software may even run slower when there are more than one core. Coordination is expensive.

The solution is to lessen the need for coordination: have different people work on different things, use smaller teams, and employ fewer managers.

14 thoughts on “You cannot scale creativity”

  1. just a reminder perhaps to be used some day ..
    – this being “introvert” vs. cost of coordination => scaling difficulties vs. the uses of the word “creativity”
    – introvert vs. type of knowledge used => extrovert = using types of knowledge which are easy to flatten into a 2dimensional string? = easy to scale, easy to replace => wins the meme-evolution

  2. Isn’t more detail needed regarding to Abramo study to comment on the nature of intellectual output w.r.t. team size?

    I agree with with the sentiment of your post, but if the study only measured # of researchers vs university output are we not erasing precisely the team-size network effects we seek to evaluate?

    In other words, if we detected linear productivity gains w.r.t. team size that would be absolutely wonderful scaling! We could solve any given research problem in half the time simply by doubling our scientists! However, if we don’t operationalize the notion of a “team” and “working on the same project” the Abramo result doesn’t say anything either way.

  3. @Nathan

    We know, if only from Computer Science, that throwing more people at a problem won’t speed up the solution in a proportional matter, and it may even slow things down.

    But people still believe that more central planning will improve the output of workers.

    Abramo et al. looked at the size of the organization. This size determines the number of researchers in any given domain that are under common management. If we could coordinate researchers to get a better output, then larger organizations would have more productive researchers. It is not the case. Basically, their results hint that whether you work at a tiny school or a very large one, your output will be the same.

    You can probably extrapolate their result by saying, for example, that engineers are Google are not more productive (everything else being equal) than researchers at Facebook even though Facebook has only a fraction of the number of engineers.

    This is not such a trivial result.

  4. @Hancock

    As a teenager, I thought that communism would bring about maximal productivity even if it wasn’t the original goal. It turns out that this belief was well spread: people did think that a coordinated economy could be more efficient.

    Of course, we know better: coordinated economies are slow to innovate and prone to collapse.

  5. You cannot scale creativity

    Agree. (Unless we perfect and legalise cloning 🙂

    One can only scale the outcome of creativity – Plato and the Republic / Marx and the USSR / Ford and the Assembly line / Edison and the Lightbulb / Alan Turing and Computing… Wall Street and World Economy Destroying Complex Derivatives.

    That said, just like propaganda for communism or socialism or capitalism has been used to produce corresponding social experiments at large scale, propaganda for creativity can be scaled too. Sort of like what Ken Robinson is keen to see happen.

    Daniel wrote: Accordingly, people living in cities are typically better off and more productive.

    For the same reason that a large group in a company – a committee, if you please – is an anti-pattern for creativity; a city quickly turns into anti-pattern for an efficient economy and/or a well-governed nation.

  6. You cannot scale creativity

    Agree. (Unless we perfect and legalise cloning 🙂

    One can only scale the outcome of creativity – Plato and the Republic / Marx and the USSR / Ford and the Assembly line / Edison and the Lightbulb / Alan Turing and Computing… Wall Street and World Economy Destroying Complex Derivatives.

    That said, just like propaganda for communism or socialism or capitalism has been used to produce corresponding social experiments at large scale, propaganda for creativity can be scaled too. Sort of like what Ken Robinson is keen to see happen.

    Daniel wrote: Accordingly, people living in cities are typically better off and more productive.

    For the same reason that a large group in a company – a committee, if you please – is an anti-pattern for creativity; a city quickly turns into anti-pattern for an efficient economy and/or a well-governed nation.

  7. DL: “We know, if only from Computer Science, that throwing more people at a problem won’t speed up the solution in a proportional matter, and it may even slow things down. But people still believe that more central planning will improve the output of workers.”

    By “workers”, do you mean “programmers”? Most workers, even many white-collar ones, do excruciatingly trivial work, the kind of non-intellectual stuff that you did not cover in this blog post.

    One could argue that the biggest problem of central planning is not scalability, but rather the fact that centralization is not robust to corruption. If humans were ideal soldiers, extremely obedient and willing to sacrifice, immune to jealousy and other “weaknesses”, there is no reason to believe that central planning is not the best system.

  8. @Rod

    If humans were ideal soldiers, extremely obedient and willing to sacrifice, immune to jealousy and other “weaknesses”, there is no reason to believe that central planning is not the best system.

    I think that there is. Our societies are not self-sustainable. Once abundant, resources become scarce. Once healthy, environments become polluted. Once vast and empty, countries become crowded.

    We are simply not in a stationary system. We cannot afford to do just do the same things again and again. We must constantly out-innovate our problems.

    Most workers, even many white-collar ones, do excruciatingly trivial work, the kind of non-intellectual stuff that you did not cover in this blog post.

    These jobs are also getting automated or outsourced faster and faster. When they are not, they represent weaknesses in the system. We simply cannot afford to have hordes of human being doing what machines can do.

  9. DL: “Our societies are not self-sustainable. Once abundant, resources become scarce. Once healthy, environments become polluted. Once vast and empty, countries become crowded.”

    I could not agree more. Which is why some central planning is needed. If a country gets too crowded, one can close the borders to stop the influx of immigrants, one can force a one-child policy until the population levels reach sustainable levels. This requires a functioning state, with functioning security forces. This also goes against the ideology of most people, for obvious reasons.

    The alternative is to hope that technology will solve all problems, which is itself a non-theistic religion.

    I certainly agree that some things cannot be scaled. But going from there to the conclusion that central planning does not work is a huge jump.

    To use an extreme example: the Red Army did not destroy some 80 divisions of the Wehrmacht and SS via decentralization.

    An interesting book on this topic is Red Plenty. A dangerous idea that has been emerging is that the Soviet central planning failed because the problem they were trying to solve was extremely expensive from a computational viewpoint. Now that numerical optimization problems with millions of variables can be solved, will central planning become feasible? If so, is it even desirable?

  10. @Rod

    To use an extreme example: the Red Army did not destroy some 80 divisions of the Wehrmacht and SS via decentralization.

    Though it might happen, you just don’t tend to see masses of young boys leaving their homes to go kill foreigners, without central planning. Left alone, they are more likely to drink beer and sleep with the girls than to kill people. So I would argue that the fact that the Wehrmacht and the Red Army used central planning to kill, destroy and rape is hardly a point in favor of central planning.

  11. I never heard about approach you described (taking large number of scientists).

    It was done other way: best people were taken and were directed what to do.

    Such approach was everywhere in USSR. For example, talented students were distributed by science-intencive jobs independently of their opinion or place of living.

    We were backward in technology but not in science. Science and technology are different things indeed. Somehow communism and technology does not match well.

  12. My opinion might be biased since I greatly dislike everything which remotely resembled socialism. I was born there and spent quite a long chunk of my life there.
    The reason socialism does not work is simple: it kills individual, it kills outliers, average is encouraged… Average is the death of creativity.

    I agree with the point you make if we are talking about software teams, however it’s hard to generalize the case for large groups of scientists. I believe that concentration of lots of smart people in one place leads to more ideas, more breakthroughs. A good example would be Bletchley Park during WWII.

  13. @Andrey

    I believe that concentration of lots of smart people in one place leads to more ideas, more breakthroughs.

    That’s a conjecture. Economists have been studying it and the evidence in its favor is not great as my reference to Abramo et al. (2012) illustrate. There is no shortage of other studies on this topic:

    We find a small, but meaningful relationship between the number of Nobel laureates in a person’s field that he or she is around and his or her probability of starting doing Nobel Prize-winning work, but no relationship between the probability of actually doing Nobel Prize-winning work and the number of Nobel Laureates present. Insofar as our estimates do not completely address causality, even our small spillovers might be overstated somewhat.

    See more at http://lemire.me/blog/archives/2011/10/25/it-is-not-where-you-work-but-who-you-work-with/

    Basically, your intuition tells you that if you put all of the great scientist and put them in the same building, you’ll get better science. But that intuition, that you can somehow do useful social engineering on creative people, does not pan out so well when you try to verify.

    Note that I am not saying that the social network is unimportant. It is obviously very important. As a scientist, I certainly owe some of my best work and my best skills to relationships with other brilliant people.

Leave a Reply

Your email address will not be published. Required fields are marked *