In a provoking post, Matt Welsh—a successful tenured professor at Harvard—left his academic job for an industry position. It created a serious malaise: his department chair (Michael Mitzenmacher) wrote a counterpoint answering the improbable question: “why I’m staying at Harvard?” To my knowledge, it was the first time a departement chair from a prestigious university answered such a question publicly. Michael went even as far as arguing that, yes, indeed, he could get a job elsewhere. These questions are crazy if we consider that for every job advertised at Harvard, there are probably hundreds of highly qualified applicants.

picture of Matthew Crawford
But let me get back at Matt’s reason for leaving a confortable and prestigious job at Harvard:

(…) all of that extra work only takes away from time spent building systems, which is what I really want to be doing. (…) At Google, I have a much more direct route from idea to execution to impact. I can just sit down and write the code and deploy the system, on more machines than I will ever have access to at a university. I personally find this far more satisfying than the elaborate academic process.

In other words, Matt is happier when his work is more immediately useful. But where does the malaise about his decision comes from? After all, he will probably make as much or even more money at Google. Matt is not alone, by the way, Matthew Crawford—a Ph.D. in philosophy—left a high paying job in an American think tank for a job repairing motor bikes. His book Shop Class as Soulcraft tells his story.

I think that Matt’s decision might be hard to understand—at least, his departement chair feels the need to explain it to us—because he is putting into question the very core values of our society. These core values were explored by Veblen in his unconventional book The Theory of the Leisure Class. He argued that we are not driven by utility, but rather by social status. In fact, our society pushes us to seek high prestige jobs, rather than useful and productive jobs. In effect, a job doing research in Computer Science is more prestigious than an industry job building real systems, on the mere account that it is less immediately useful. Here are some other examples:

  • The electrician who comes and wires your house has a less prestigious job than the electrical engineer who manages vague projects within a large organization.
  • The programmer who outputs useful software has a less prestigious job than the software engineer who runs software projects producing software that nobody will ever use.
  • The scientist who tinkers in his laboratory has a less prestigious job than the scientist who spends most of his time applying for research grants.

Note how money is not always immediately relevant. While it is normally the case that manual labor has lower pay, it is almost irrelevant. And indeed, plumbers make more than software developers in some parts of the world (like Montreal)… Even though software jobs are usually considered more desirable.

There are at least three problems with this social-status system:

  • Nature is the best teacher. Working on real problems makes you smart. The German philosopher Heidegger famously made this point with a Hammer. To paraphrase him, it is not by staring at a hammer that we learn about hammers. Similarly, scientists who do nothing but abstract work in the context of funding applications are missing out. The best scientists work in the laboratory, in the field; they tinker.
  • By removing ourselves from the world, we risk becoming alienated. We become strangers to the world around us. Instead, we construct this incoherent virtual reality which has often much to do with soviet-era industrialism. We must constantly remain vague because truth has become subjective. Whereas the hammer hits, whereas the software crashes, whereas the experiment fails… projects are always successfully, marketing is always right and truth is arrived at by consensus. Yet, we know deep down that this virtual reality is unreal and we remain uneasy, trapped between reality and virtuality. The perfect example are the financial markets which are creating abstract products with agreed-upon values. As long as everyone plays along, the system works. Nobody must ever say that the emperor is naked. Everyone must accept the lies. Everything becomes gray.
  • Human beings like to make their own stuff. We value considerably more what we did ourselves. You may be able to buy computers for $200, but nothing will ever replace the computer you made yourself from scratch. It may be more economical to have some Indian programmers build your in-house software, but the satisfaction of building your own software is far more than what you get by merely funding it. Repairing your own house is a lot more satisfying than hiring handymen.

To summarize: trading practical work for high-level positions is prestigious, but it may make you dumber, alienated and unhappy. Back when I was a graduate student, we used to joke about the accident. The accident is what happens to successful professors: they suddenly become uninteresting, pompous, and… frankly… a tad stupid.

Thankfully, there is hope. The current financial crisis, mostly because it couldn’t happen according to most economists, was a waking call. The abstract thinkers may not be so reliable after all! The millions of college graduates who are underemployed in wealthy countries all around the globe have unanswered questions. Weren’t these high-level abstract college degrees supposed to pay for themselves?

How do we fix this broken caste system and bring back a healthier relationship with work? Alas, we cannot all become plumbers and electricians. But it seems to me that more and more people are realizing that the current system, with its neat white collar jobs and rising inequalities, could be improved upon drastically. The Do it yourself (or Do it with others) wave has been a revelation for me. Yes: Chinese factories can build digital thermometers much cheaper than I can. But making your own digital thermometer is far more satisfying. Saving money by abstracting out reality is not a good deal. And of course, building real systems is not the same as finding money for your students to do it for you.

Further reading: Working long hours is stupid and Formal definitions are less useful than you think.

23 Comments

  1. Here, here, Daniel! There’s definitely a problem with academic work w.r.t. concrete impact. Unless, of course, you are a public intellectual like Paul Krugman and your NY Times Op. Eds have a measurable effect on the leaders of the free world.

    I have trouble believing that financial reward wasn’t a significant incentive, though. They must be earning at least 2x more than they would be in the academy. You can add Thomas Hofmann and Peter Norvig to the list of C.S. “academic refugees” that Google has seduced.

    My question is – what about the “greater good” that results from the free flow of ideas in the academic world? I worry that Google has bought it’s way into owning the world’s C.S. intellectual property by (essentially) bribing (some of) the world’s smartest people.

    I wonder what they will say in their autobiographies 30 years from now.

    Comment by Andre Vellino — 22/11/2010 @ 12:16

  2. @Andre: no, it’s not only about the money. If these people were really driven by money, they wouldn’t have chosen an academic career in the first place!

    Also, I think you should have look at Dan Pink’s TED presentation:
    http://www.ted.com/talks/dan_pink_on_motivation.html

    He says that for creative jobs (such as software development or research), there are 3 factors for motivation: autonomy, mastery, and purpose. Money is definitely NOT one of those factors.

    I can tell you one thing I know for sure: in my 10-year career in software engineering, the only times I was truly happy were when I had autonomy, could fully use my mastery, and my job had a real purpose. Now I have the best pay I ever had, I enjoy more autonomy than I ever had, yet I work for morons on a doomed project. Guess how I feel?

    Comment by foo — 22/11/2010 @ 15:29

  3. @foo

    yet I work for morons on a doomed project

    Purpose is very important. The best job in the world, without purpose, can become hell:

    The gods had condemned Sisyphus to ceaselessly rolling a rock to the top of a mountain, whence the stone would fall back of its own weight. They had thought with some reason that there is no more dreadful punishment than futile and hopeless labor. (Albert Camus)

    Comment by Daniel Lemire — 22/11/2010 @ 15:36

  4. I am one of those systems-building people who is staying in academia. The reason I am still in academia is because I am interested in developing certain “blue-skies” ideas which have potential to become useful and commercial, but not for another decade or so. I stayed because I really care about what I do. But I had to make a lot of compromises

    1. I intentionally didn’t apply for a tenure-track job. I have talked to a great number of people in such positions, and could see that teaching, committees, and the “publish or perish” cycle were not going to let me do what I am interested in.

    2. Primarily as a consequence of (1), I get less respect in my department, because since I am research faculty, clearly I am not “good enough”. I raised more grant money in recent years than certain tenure-track faculty, and I have high-quality publications (including a recent ACL), but that seems to be not enough. Just sticking to research excludes me from department life and decision making in great many ways.

    (3) I settled on being less successful in academia than I could be if I were willing to work on things tenure-track people work. People routinely make comments like “that’s too much engineering, that’s not science”. Well, my ideas are not commercial as yet, but they are carefully evaluated with real users, and making contributions to my field. It could not be done without putting in the engineering effort, yet I hear again and again “engineering” said in a way that implies that it holds no value.

    (4) I have heard an academic say once “My student chose to go into industry. I feel betrayed – why did I waste my time on him?” I think this is sad that anyone would consider teaching people new ideas a waste of time. Yet in my school, which is one of the top computer science programs, this seems to be the attitude among tenured academics. I am happy with my choices, but I am also very sure that if I change jobs one day, it will be going to industry, and not to academia. I think engineering and making systems that work with real users is extremely important, and I don’t see much of it going on, at least where I am.

    Comment by MD — 22/11/2010 @ 16:28

  5. @MD

    I hear again and again “engineering” said in a way that implies that it holds no value.

    Your value is proportional to how little immediate application your work has. If you can afford to work on totally useless theory, then all the more prestige to you! If your work proves immediately useful, then that’s because you couldn’t afford to work on abstract nonsense.

    I have heard an academic say once “My student chose to go into industry. I feel betrayed – why did I waste my time on him?” I think this is sad that anyone would consider teaching people new ideas a waste of time. Yet in my school, which is one of the top computer science programs, this seems to be the attitude among tenured academics.

    The explanation is simple: funding agencies measure up professors by how many of their students become professors. We must urgently train as many Ph.D.s as possible, and they if they end up in industry, we have failed. None of this makes any kind of sense of practical sense, of course.

    Comment by Daniel Lemire — 22/11/2010 @ 18:06

  6. Higher-prestige jobs are often those with opportunity for broader impact. A software developer can build an application that brings value to thousands or millions of people. A professor can develop new methods that will impact an entire industry. A pop star can affect an entire culture.
    A political leader can affect the entire world.

    In contrast, there is a limit to a plumber’s productivity and overall impact, even if many plumbers are more productive than many academics. (Most doctors and lawyers affect a small number of individuals, but have a very large affect on those few, which seems to be another way to obtain higher prestige.)

    The catch is that, at higher levels of abstraction, productivity is harder to measure. If your job is filing papers, it’s easy to measure progress in number of papers filed. If your job is to invent something new that will change the world, it’s much harder to tell if you’re accomplishing anything from day to day. (Incidentally, this is one thing I like about teaching class: I can tell that I accomplished *something* each week, even if my research is uncertain.)

    Comment by Daniel Lowd — 22/11/2010 @ 20:23

  7. I think the analysis of Matt Welsh’s departure is missing somehting important: research in applied computer science (i.e., systems) is in a state of crisis (and it’s been a while).

    Matt’s main reasons seem to be that he wants to build large-scale systems that have impact and the only place he feels he can do this is in industry. And it’s really sad that he’s absolutely right!!

    I love systems work, but let’s be honest. Systems researchers like to boast about how practical their work is and how they have deployed actual systems. They also love to criticize their theory colleagues for doing useless things. But how many systems from academia have had any major impact recently? (and PageRank does not count because that was an algorithm!)

    How would you compare the “usefulness” of academic systems research to say the stuff Google is doing (e.g., MapReduce, Pregel, Big Table, etc…)? How about compared to the opensource community (e.g., Linux, the BSDs, Hadoop, Lucene, etc…).

    Of course this was not always the case. Places like CMU and Berkeley used to produce great systems (which inspired a lot of what’s going on in industry and in the opensource community), but that “golden era” of academic systems research seems to be long gone.

    Comment by Anonymous — 22/11/2010 @ 21:50

  8. Your post reminds me of the story I heard of an English professor who was denied tenure because he wrote novels — critically acclaimed novels — rather than spending all his time writing *about* novels. Unfortunately the reference escapes me.

    Comment by John — 23/11/2010 @ 4:42

  9. I’m going to more or less repost a comment I made elsewhere:

    You mention, “these core values were explored by Veblen in his unconventional book The Theory of the Leisure Class.”

    Veblen wrote another book after The Theory of the Leisure Class called The Theory of Business Enterprise. In it Veblen gives a strong critique of the advertising industry (at least the portion of it that is zero sum; in the same sense as the status games in Leisure are zero-sum).

    A large part of Google is advertising. A lot of that advertising is not zero sum–I believe it provides a lot of value–but over time they have moved more and more heavily into the zero-sum space (e.g. the purchase of DoubleClick, the general character of youtube ads). By that I mean things like brand-building ads that just associate an emotion or status/prestige with a product or company.

    Comment by cma — 23/11/2010 @ 23:14

  10. @cma

    Whether working at Google is the right choice (or the moral choice) is another story. I don’t think that the fact that Matt went to work at Google is extremely important to the story. The main point is that he left academia.

    But there are far worse companies than Google, I think.

    Comment by Daniel Lemire — 23/11/2010 @ 23:23

  11. No, I agree; I don’t mean to critique the move to Google (which is why I’m not really trying to quantify how much of Google’s business I think is wasteful).

    I was just trying to add a little balance. Veblen definitely had a lot of critique for academia too–there is a whole section in Leisure Class on 1800s academia’s insane over-focus on things like Latin; something anyone dealing with your captcha system can relate to =).

    Comment by cma — 24/11/2010 @ 0:00

  12. What I don’t understand is why academia cannot cooperate more with industry to produce more relevant research that has direct impact on industry? Everybody would benefit. Including scientists, who will get higher salaries without a necessity to do a lot of tedious and boring work (typical for an industry position)

    PS: Actually, plumbers make much less money than software developers.

    Comment by Itman — 24/11/2010 @ 12:14

  13. @Itman

    What I don’t understand is why academia cannot cooperate more with industry to produce more relevant research that has direct impact on industry? Everybody would benefit. Including scientists, who will get higher salaries without a necessity to do a lot of tedious and boring work (typical for an industry position)

    Professors have to write research papers and train graduate students. Sometimes these objectives fit well with deep industry collaboration, but not always. Professors who work too closely with industry, at the expense of their graduate students and research papers risk losing their research grants. They may also get in trouble with the university administrators.

    There are successful examples though, like Patrick O’Neil who has been working with IBM and then Microsoft. And he wrote textbooks. And he wrote influential papers. But pulling off what he pulled off is hard.

    Actually, plumbers make much less money than software developers.

    When was the last time you hired a plumber, assuming you could find a good one at all?

    The salaries are not very different when you make fair comparisons. Let’s compare a Plumber II with a Programmer II on monster.ca for the Montreal area:

    A typical Plumber II working in Montreal, QC earns a median base salary of $67,182, according to our analysis of data reported by corporate HR departments. Half of the people in this job earn between $63,042 and $75,341.

    A typical Programmer II working in Montreal, QC earns a median base salary of $61,101, according to our analysis of data reported by corporate HR departments. Half of the people in this job earn between $57,827 and $66,840.

    This means that most of the low- to medium level programmers earn less than plumbers. Maybe one can argue that programmers have a nicer workplace, with air conditioning, but they do earn less.

    And of course, with plumbers, there is a difference between the *reported* salary and their actual revenue. Many of them hardly pay taxes…

    Of course, a programmer III goes higher, at least in reported incomes:

    A typical Programmer III working in Montreal, QC earns a median base salary of $76,961, according to our analysis of data reported by corporate HR departments. Half of the people in this job earn between $69,050 and $81,369.

    However, a programmer III has typically a bachelor degree or more. Yet, the difference in median salary is only around 15%. And take into account that many of these people have master’s degrees (or more!) and often huge debts…

    In any case, many college graduates would kill for a $67,182 salary in Montreal.

    And frankly, it doesn’t matter. Plumbers could make 15% more and we would still look down on them compared to programmers… [Not that *I* look down on plumbers... I wouldn't want to get in trouble next time I need a plumber.]

    My point being that we often dismiss these “manual labor” jobs as being underpaid. But the truth is that they are “low prestige” because they are doing immediately useful work.

    Comment by Daniel Lemire — 24/11/2010 @ 13:57

  14. Professors who work too closely with industry, at the expense of their graduate students and research papers risk losing their research grants. They may also get in trouble with the university administrators.

    This is definitely a serious drawback of the education system.

    PS: Obviously, plumbers in Canada are highly valuable. :-) This is not the case of US. Many high salaried plumbers are actually business owners, i.e., there are several people in their command. Also note that hourly rate is not a good benchmark, because it does not directly translate into

    Comment by Itman — 24/11/2010 @ 14:37

  15. …does not directly translate into yearly salary.

    Comment by Itman — 24/11/2010 @ 14:38

  16. Really thoughtful post. It’s difficult in the software industry to avoid the allure of management positions (and many pursue them). I’ve always remained hands-on but sometimes had to fight for it or simply move on. I’ve had a growing disenchantment with the jobs and contract work available in my area. I’m now pursuing my passion as an independent software company, building things myself for my own company.

    I hadn’t noticed any do-it-yourself movement yet. However, I have noticed a simplicity/simple-living/frugal movement coming at least from the US. Also a minimalist movement that seems to be into nice toys (such as Macbooks) – but not many of them. Also there is the “Lean Thinking” which has become very popular in software circles. I recently came across the “Lean Startup” which seems to be a related movement/idea.

    Comment by Steven Shaw — 24/11/2010 @ 23:32

  17. @Anonymous Maybe it just takes a while with research systems. They have made a recent impact – ones that you mentioned! The *BSD from Berkeley and the Mach 3 microkernel from CMU have ended up in Mac OS X and iOS. In programming languages, Scala seems to be having a fairly immediate impact on the Java development community. The L4 microkernel seems to be having at least some impact – although this is likely on the back of govt (not industry) funding.

    Comment by Steven Shaw — 25/11/2010 @ 1:59

  18. @Shaw

    Great list!

    (1) I think we all agree regarding the importance of BSD and related work of this era. But it is not necessarily representative of what is going on right now.

    (2) Scala is, indeed, a great example of academic work having great impact. It is exceptional work.

    (3) It is unclear whether the L4 microkernels are typical academic work. From what I can tell, the bulk of the work was done at IBM’s Thomas J. Watson Research. While they continued for two years after the author moved to academia, it is unclear whether he would have initiated the same project as a professor.

    Comment by Daniel Lemire — 25/11/2010 @ 8:34

  19. Daniel, I hadn’t realised that L4 started out at IBM T.J.W Research labs. There is continuing work in Australia at Nicta/UNSW http://www.ertos.nicta.com.au/research/l4/

    Nicta is a Australian government owned and operated research agency. It’s affiliated with many of our top universities.

    Comment by Steven Shaw — 25/11/2010 @ 19:34

  20. @Shaw

    Nicta is a company supported by the Australian government. It is different from academia. It is probably somewhere between a government lab. and an industry lab.

    I’ve worked in a government lab. While on the surface, it looks like academia, there are important differences. The main one is time allocation. A researcher in a government lab. is not supposed to spend 40% of his time training students, and another 40% doing administration. The bulk of the researchers spend 80% of their time on a couple of research projects. Sure, they fill out forms, attend meetings, hunt for funding, meet with stakeholders… but the average researcher is not a manager…

    The net result is that you will find relatively few professors who hack difficult code. What they do, instead, is recruit students who do it for them. But that’s not the same thing at all. Meanwhile, you will find lots of researchers in government labs who write difficult code from scratch… often over periods of years. (Yes, professors do that too… but…)

    There is a huge difference between supervising someone who writes code, and actually writing the code yourself. It is hard to describe the difference in words… but I’d say it is like the difference between owning a garage, and being a mechanic. Not the same job.

    Comment by Daniel Lemire — 25/11/2010 @ 20:42

  21. Daniel, we do not disagree at all. I agree that a government funded research lab isn’t the same as academia. Nicta seems to be staffed predominantly by academics and PhD/Masters students so your assessment of what it’s like on the inside sounds right. In fact, I think I might like it as a research engineer / phd/masters student.

    Comment by Steven Shaw — 25/11/2010 @ 20:52

  22. Professors have to write research papers and train graduate students. Sometimes these objectives fit well with deep industry collaboration, but not always. Professors who work too closely with industry, at the expense of their graduate students and research papers risk losing their research grants. They may also get in trouble with the university administrators.

    perhaps we need to rethink this.

    apparent risks:
    1)losing research grants
    2)getting in trouble with admin

    if 80%ish people are not happy with their jobs, if Matt is saying:
    all of that extra work only takes away from time spent building systems, which is what I really want to be doing,
    we have to ask, what are the real risks here?

    maybe we all need to bring into question the very core values of our society.

    and yeah – the main reason i keep commenting – the latin i’m learning via your spam protection.
    grazie.

    Comment by monika hardy — 2/1/2011 @ 15:22

  23. There is a lot of truth in what you wrote here, Daniel. However, I think that finding a way to foster more balance might be the better solution instead of completely ignoring the human desire for recognition.

    Comment by Greg Linster — 20/1/2012 @ 12:34

Sorry, the comment form is closed at this time.

« Blog's main page

Powered by WordPress