Good ideas are overrated

As a college student, I was convinced that the most important part of science and engineering was to have good and original ideas. If you contemplate Einstein, it is not hard to come to this conclusion… The man had a never ending stream of great ideas. Notice however that we never read Einstein himself, or examine his workbooks… instead, we just hear about his ideas.

The same might be said about Google. Brin and Page came up with the idea of using hyperlinks to rank web pages. And they went on to become billionaires, just like that. We rarely talk about how the then dominating player (AltaVista) failed to keep its index updated for months and tried hard to milk every penny out of visitors by plastering its web site with ads… while Google just offered a working search engine kept up-to-date.

Matt Might, a computer science professor, recently wrote that “the limiting reagent for scientific progress is good ideas”.

Matt presents the scientific model in a way that is familiar to many, but contrary to my own experience. One imagines scientists idle, waiting for good ideas to come to them. Maybe we can imagine Einstein who sits by his window receiving all these great ideas, as if they were telegraphed from an Oracle directly to his brain. And then, once the idea hits, it is all over in an instant. Einstein grabs his pen and writes the paper in a minute. Brin and Page imagine PageRank and then build their search engine in a week-end, taking the world by storm.

When I supervise graduate students, I often see them struggling with this model. They have ideas, but they do not know which ones are good. Some of them choose to wait… maybe a great idea will come eventually? It never does.

My model is different. Instead of imagining Einstein as someone receiving these deep insights magically, I imagine an obsessive thinker who keeps asking… “yes, but what does it mean?”… or “yes, but how does it work?”. Whereas others are satisfied with recopying the accepted answers, or accepting the unknowns, he redoes the work, more carefully. I imagine Brin and Page not as people who surfed on a brilliant idea… but rather as people who took a problem that might have been considered solved, or too technical, that is “how do you build a search engine”… and they revisited it, putting a lot more care into it.

When I look around, I see a deeply flawed world with thousands of superior ideas waiting to be tried out. I am literally never running out of good ideas and I doubt others are. I am not exceptionally smart, but give me any interesting problem, in science or engineering, and I am sure I can come up with five approachable questions that have not been answered yet. In fact, I think that almost anybody can do this… Give me a few years to become really knowledgeable, and I can probably prototype a solution in a few weeks.

I have two young boys. Almost every day they come up with a science question that I cannot answer. Once a week, they come up with a question that I cannot answer with Wikipedia. Many of these questions are genuinely interesting and could make a research project of its own.

What is genuinely scarce is interest and motivation.

Let me put it another way. Imagine that I can go back in time to 1996. I know that the next critical piece of infrastructure is going to be a search engine. I know this, but there are already well entrenched leaders. I am in Montreal, and nobody is going to fund me, or join me to work on this problem… So I have to move to Silicon Valley where I will live in my car for a time. Then I have to find the ressources to build the new Google. It is really hard work. It is also very frustrating work because I know that Brin and Page are around… and maybe they will still come up on top.

Or maybe I could go back to 1900. I basically have a major in Physics from the 1990s. I know a lot of Physics that even Einstein cannot yet imagine. I could find a job to feed myself while I hack away at brilliant research papers. Or maybe I could try to rewrite one of Stephen King’s famous novels and get it published before he could… by going back in time.

Would I succeed? Maybe I would, maybe I would get discouraged. What is clear is that, even with perfect hindsight, success would be far from easy. Even with a perfect knowledge of all the great ideas… success would be really hard work.

Experienced software programmers know that, often, it is just as hard to understand someone’s code than to rewrite it from scratch. There are many ways to re-express this same idea. Some might say that execution is everything. But I think that the truth is even harsher: the concept of good idea is ill-defined. In hindsight, we try to explain success as “having a good idea”, but even if you had received this “good idea” on a silver platter, you might not have done anything with it in the end.

If you imagine the world a being two-dimensional, then there are many dead-ends, and few correct paths. The truth is that there are far more than two dimensions. You are not stuck in a dead-end. You are not out of good ideas. You are just bored.

Published by

Daniel Lemire

A computer science professor at the University of Quebec (TELUQ).

7 thoughts on “Good ideas are overrated”

  1. This post resonates with me, and I’d like to expand the idea: good ideas follow a pipeline from problem to solution, and not everyone is good to contribute at any part of this pipeline.

    I am now CTO (I love this smurf name) of a start-up making a computer security product and service. Four years ago, I was a struggling post-doc in medical imaging. How did I get from struggling to arguably successful? I’m *not* an infosec expert out there by a long shot. However, I have a few of these expertsas colleagues. My abilities are to write simple software, and to easily turn a prototype into good, simple production software. So, the current achievements of the company I work for start from the brilliant prototypes of my colleagues, which I then nip and tuck and spit-shine. These prototypes would make poor products, and I’ve had trouble coming up with really good ideas, but I can execute and refine very well.

    We hold up the example of people who seemed able to do push through at every part of this pipeline. Such great folk do exist, but many celebrated individual achievements actually stem from a (hidden) team’s work. People should learn to recognize the frontiers of their “efficiency zone” and have the humility to surround themselves with people that complete them. Within that zone, motivation very rarely lacks. In addition, when you get some success within that zone, it becomes easier to expand it.

  2. Very nice post. I would only change the last word, from “bored” to “frustrated”.

    It’s hard to find funding, without funding it’s hard to find time. Even with funding, nobody guarantees success, but in the eventual case of success, most likely the funders will be the ones reaping the fruit of your time and effort (and luck).

    All that considering individual work, as if one individual alone could do something (good post from #1 wrt this).

    I don’t think this is boredom, or even frustration. It feels like despair to me at this moment.

    The only hope I see are projects like kickstarter, but that only works for a few types of ideas.

  3. To differ from a prior comment, “bored” works for me.

    There is a word in English “fortune” that carries exactly the right freight, or at least an older definition.

    Most of what we hear is of those who got lucky. Their tale gets often retold. The guy with a good idea but lesser luck, not so much.

    Good ideas are not so hard. The chance to apply, less likely.

  4. I agree with Matt, and not with you.

    As you note, it’s really, really easy to find questions whose answer would be valuable and we don’t know the answer to. The trouble is that almost all of these problems are not important.

Leave a Reply

Your email address will not be published. The comment form expects plain text. If you need to format your text, you can use HTML elements such strong, blockquote, cite, code and em. For formatting code as HTML automatically, I recommend