The myth of the unavoidable specialization

In a recent essay, Malone et al. claimed that we were entering the age of hyperspecialization. Their core assumption: human beings are more efficient when doing specialized tasks. Thus, they claim, we are moving toward a future where software will distribute hyperspecialized tasks to expert individuals. They believe that we will progressively work on narrower and narrower problems.

Among intellectuals, specialization is often seen as a good omen. It is the safe thing to do: stick with a narrow topic (e.g., how polar bears raise their offsprings, or the chemistry of sugar). The usual argument is that with the growth of knowledge, we have no choice but to become narrow specialists. Conversely, people with a wide range of interests are pursuing a high risk strategy. Whenever you attempt to contribute to a new problem, you risk ridicule: maybe everyone who has worked ten years on this topic knows that you are pursuing a dead-end.

So, yes, humanity knows more about every single subject than ever before. Conversely, our brains are are biologically identical to what they were 2000 years ago. Thus, we ought to be increasingly mentally challenged. But this logic is flawed because it equates the mind with our brains. We are expanding our minds exponentially! Indeed, our minds are increasingly externalized. First, we started telling stories, using other brains to support our own cognitive abilities. Then we invented writing. Then we invented the Web. At each step, human beings become smarter and smarter in every respect. One might object that it is not I who becomes smarter when I am connected to the Web. That somehow, saying so, is cheating. But this is pure semantics. The fact is, with access to the Web, I could run circles around Sir Isaac Newton, even if he were allowed to have an entire library at his disposal.

We could still conclude that as we expand knowledge, the specialists have a greater and greater edge: it becomes riskier and riskier to be anything but a specialist. But I believe the opposite is happening.  Far from moving toward hyperspecialization, we are in fact moving toward hypergeneralization. Millions of freelance workers worldwide fill out their taxes electronically, bypassing the specialists (accountants). Whereas researchers absolutely needed expert librarians to avoid wasting days in libraries, Google Scholar has made reference checking accessible to all, at no cost. I learned how to prepare pineapple like a chef in minutes using a simple YouTube query. Soon augmented reality glasses will allow you to walk in any park and know instantly the characteristics of any flower you encounter.

But wasn’t the XXth century about specialization? Of course not! The XXth century was about people like Einstein who invented  a new type of fridge and also a little something called relativity. The specialists are most often the poor people. You want to rise up in a company like Google or Facebook? Then be someone who can expand his mind as needed, not a silly Java specialist who can be replaced easily. Leaders like Henri Ford like specialization, for others, never for themselves.

Your future wealth is determined by how much you can expand your mind beyond the capacity of your biological brain, not by your current skills.

Take a chance and go work on a new problem, today.

Further reading: Lack of steady trajectories and failure and How information technology is really built. See also Serial Mastery.

: Alexandra Seremina translated this page in Romanian.

Daniel Lemire, "The myth of the unavoidable specialization," in Daniel Lemire's blog, July 4, 2011.

Published by

Daniel Lemire

A computer science professor at the University of Quebec (TELUQ).

18 thoughts on “The myth of the unavoidable specialization”

  1. I agree that we won’t get hyperspecialization, but I don’t think hypergeneralization is going to happen either. Deliberate practice/10,000 hours still matter to adding value.

    What I personally think will happen is de-canonization. Specialist/generalist are relative terms in some sort of collective social ontology of professions. Something that allows you to collect statistics and segment the labor market. What’s happening is that more people are becoming illegible and undefinable in traditional terms because their skills and knowledge, whether generalized or specialized, simply don’t fit any normal categories.

  2. You are right.The fragmentation of knowledge by high levels of specialization is a step in the wrong direction. Midgley highlighted: ‘Traditionally, the value of knowledge centred on understanding – on the power to see the connections of things, to wonder at them, and so to live wisely’ (Midgley, 1989, p. 3).This indicates the need not only to investigate deeper one particular issue, but also for interdisciplinarity and the making of connections to not only understand this particular phenomenon, but also its relation to the world.

  3. I think that everything is linked: thanks to specialization, we went further on specific topics, and thanks to books, then the web, we developed knowledge in a way than more or less anybody can access… Thus this ability to go toward what you call “hypergeneralization”.

    I rather believe that humanity is going forward on the two directions, one supporting the other. But this generalization phenomenon is indeed the most impressive and the most evolving one, turning everyone into a potential expert on almost any topic.

    I’m not sure of the consequences of all this, but I would say that if the essay’s idea is quite right when it comes to research (unless generalist geniuses like Einstein discover new specialties!), we indeed tend to be generalists in our everyday life.

  4. @Itman

    Engineers at Facebook or Google already have much autonomy. They often get to run many of their projects. If you are going to build GMail, you better have a broad expertise. The autonomy to build significant things requires that you to be a generalist at times. Hyperspecialists must be constantly managed. Those are not the employees Google and Facebook are looking for.

    This is broadly applicable. I’m with Dorian Taylor:

    those who per­son­al­ly gen­er­ate in­tel­lec­tu­al cap­i­tal [must be] treat­ed as man­agers in their own right, and pos­sess the ex­ec­u­tive au­thor­i­ty re­quired to de­cide how best to de­ploy their time.

    I think that the future belongs to those who will be able to quickly master new things. The so-called specialists will face mounting pressure as new non-specialists start to occupy their territory with greater and greater ease.

  5. Unless you manage, you do must have a certain specialization. Managing is a different issue, the technical knowledge does not matter much. What matters is the ability to bullshit confidently.

  6. Daniel,
    I think we define generalization too differently. Actually, I was working in a small team that was delivering a web-based e-mail to millions of people so that I could see what people were doing. Even those who were doing a lot of different tasks, in my opinion, specialized in a very narrow field. That is they had experience with C++, Linux, SQL, perl, mail protocols, and some search algorithms. If they were generalists they would also write phone aps, design IDE for Windows, MAC, etc…

  7. @Itman

    Take the best in this this team you describe. How long would it take them to master phone app programming and design?

    Now, let’s compare with how things were 20 years ago. How long did it take for someone to master a new platform, a new type of applications?

    My point is that it is getting easier, not harder, to catch up to the experts in any field.

  8. @Itman

    I stand by my statement:

    Your future wealth is determined by how much you can expand your mind beyond the capacity of your biological brain, not by your current skills.

    There is nothing risky about this proposition.

  9. It is definitely easier. It is just not feasible unless you manage. I am afraid of the perspective when everybody is becoming a super-generalist (without deep technical expertise) and who is doing the business? I want a plane to fly smoothly instead of crashing. Just because somebody was catching up with the experts too quickly.

  10. Daniel, I completely agree with you on the statement. Your or mine wealth does depend on it. However, this model is not scalable. If everybody starts to generalize, the civilization will collapse. Well, and anyways if I, e.g., don’t like overgeneralization, I would prefer to stick to my technical skills and be happier despite it would mean less money.

  11. I rather enjoyed reading your perspective, particularly when you’ve taken the time to reason about it so.

    I would enjoy reading more of your perspective on the topic in the future.

    – Bill

  12. @gregbo

    (…) questions asked required specific knowledge such as what TCP port a service ran on or what tcpdump command was used to print certain types of packets.

    Ironically, you can google these questions very easily.

    These are not questions I would ask a potential new Google engineer.

  13. Hmmm. I didn’t see much evidence of generalization in the Marcin
    Wichary article. The subject of the article is primarily a user
    experience (UX) engineer who leverages various web technologies
    (e.g. HTML5). I didn’t see much evidence of cross-disciplinary
    involvement, even within software engineering (e.g. operating systems
    or AI).

    My experience tends to match Itman’s, especially with regard to
    Google. The interviews I’ve had with them have been very specific and
    detailed; questions asked required specific knowledge such as what TCP
    port a service ran on or what tcpdump command was used to print
    certain types of packets. Not the higher-level conceptual engineering
    types of questions I was expecting. On a more general note, most
    people are not capable of being outstanding in several subfields of
    the computer industry, so they are forced to specialize in order to
    pass an interview such as the Google interviews I’ve had. Managers
    benefit from generalization more, because it enables them to employ
    multiple strategies to run their teams, but individual contributors seem
    to be judged on highly specific topics such as how to code a particular
    algorithm, etc.

    Perhaps Google has changed their interview strategies, but I haven’t
    seen much evidence of it from other sources of information on Google

  14. Daniel, it probably depends on a department. The last time I talked to a Google recruiter, ironically, they said they wanted a generalist. What they probably meant, however, was that the person would know C++, Unix, network protocols, databases, and a bit of the WEB (as opposed to, e.g., SQL coder), rather than being a computer scientist, a successful coder proficient in Linux/Windows/Mac technologies, who is running his own business in spare time.

  15. Even when technology enables easier access to knowledge doesn’t mean there wont be hyperspecialization. Even if we are expanding our minds exponentially, the knowledge base in the world is expanding even more exponentially! Because of this increasing distance between the general knowledge base and our brains ability to handle information, you have to focus on a narrower area in order to do any meaningful contribution to this knowledge base.

    Not saying that generalists doesn’t have their place too, but more in the role of creating applications with existing knowledge. So I wouldn’t say it’s more risky to work as a generalist than specialist.

Leave a Reply

Your email address will not be published.

You may subscribe to this blog by email.