Predicting your future performance

The Matthew effect says that “the rich get richer and the poor get poorer”. With this sole principle in mind, you would think that the future is easily predicted. Whoever is rich or famous today is going to be rich or famous tomorrow.

So which programming language should you learn if you are a programmer? The most popular language right now, or the fastest growing language? If you believe in the power of Matthew effect, you should always focus on the most popular language right now, since you believe that challengers are unlikely to succeed.

At a personal level, the Matthew effect can be depressing: your starting position in life determines the rest.

Mazloumian asked an interesting related question. Given a scientist, which is a better indicator of his future success (measured by citations):

  • the total number of citations received so far,
  • the average number of published papers per year,
  • the average annual citations,
  • the annual citations at the time of prediction,
  • the average citations per paper,
  • and so on.

Can you guess the best indicator of future success?

First, it is worth stating that Mazloumian found that the Matthew effect was weak:

Our results have shown that the existing citation indices do not predict citations of future work well, and hence should not be given significant weight in evaluating academic potential. Including various indicators and testing various prediction time horizons, our results are still in agreement with Hirsch’s study “past performance is not predictive of future performance.” Even combining multiple citation indicators did not significantly improve the prediction: apart from citation indicators, no better predictor of the impact of future work exists.

But, if you are going to use a single measure to predict the future success of a scientist, you should go with the annual citations at the time of prediction. This is consistent with saying that the past is a poor predictor of the future.

Of course, the Matthew effect is real. If you start out strong, you will tend to outdo your poorer peers. However, the Matthew effect is often much weaker than people believe. People at the top of their game are beaten by challengers coming from nowhere all the time.

In some sense, it is troubling because it says that we know less than we think we know. When recruiting a scientist, for example, it is very tempting to use his past performance over many years to predict his future performance. But this heuristic is weak.

It also means that it is hard to build lasting capital. Working hard today may not be sufficient to establish a long stream of successes. To keep on succeeding, you need to keep on working hard and be lucky.

On the plus side, it means that if you have not succeeded early, you can always make it big later. It does not mean that it is easy to rise up at the top from the bottom. By definition, only 1% of all players can be part of the top 1%. Even without any Matthew effect, you would still be unlikely to reach the top 1%. What is says however is that life is probably fairer than you think.

So how do you predict someone’s performance? With humility. And this includes yourself. You do not know how well or how poorly you will do in the future. Most times, you should avoid both arrogance and defeatism.

Daniel Lemire, "Predicting your future performance," in Daniel Lemire's blog, July 30, 2014.

Published by

Daniel Lemire

A computer science professor at the University of Quebec (TELUQ).

8 thoughts on “Predicting your future performance”

  1. Isn’t it interesting that we need to quantify something as subjective as “success” in the first place? Isn’t it interesting that we human beings are somehow hard wired to measure ourselves against each other, and do so by coming up with a myriad of metrics to attempt to quantify subjectivity?

    For some (literally God-known) reason we are hard wired to be on the top … of something — to be the best… at something. I suppose it’s because, as you stated, it’s easier to be on top when you’re on top. However, on the other side, it’s easier to be on the bottom — when you are on the bottom.

    Nature tends toward a normal distribution. It’s hard to get to the very, very top — and it’s hard to get to the very, very bottom. Our universe tends to pull us between the top and the bottom — and we’re always moving somewhere in the middle.

    What’s really interesting to me is that our very definition of success is dynamic and fleeting based on the environment and our needs. And, since, the unit of measure itself is changing, trying to live up to one single measure is a fools errand.

    The very best measure of success is whether you, personally, are hitting your goals (whether specified by you or someone else), which should be based on the need to achieve something. Therefore you are successful on a personal basis.

    You set a goal to land on Mars. When you’ve done that, you are successful. You set a goal to program the interface to fly the new Dragon2 spacecraft — when a person flies the new Dragon2 spacecraft using your interface — then you are successful.

    — my two cents.

  2. @David

    The number of citations is not really a sensible measure of success in practice… but for the purpose of my post, I needed to define “success” in an ambiguous, objective way.

    Also, success is not about belonging to the 1%. You do not have to beat others to be successful.

    This being said… while I agree that the pursuit of any one measure as defining success is wrong, I do not agree that success is strictly personal. Human beings are social animals.

  3. @Daniel, Hah… I’m sorry. I didn’t express myself correctly. We ultimately agree.

    Yes, success is not strictly personal in the sense that the only person that can define my success is me. I am defined by my company’s measure everyday — their goals for me must be my measure of success (at work) if I am to continued to be paid by them. I’m gauged as successful by my life-partner based on goals we’ve set for each other. These are goals set for me — therefore the success of them is “personal”.

    However, as you’re suggesting in your post, my success can not possibly be defined by society as a whole, since, in order for me to be successful against society, the measure needs to be so specific as to match my talents perfectly — or at least enough to exceed a given threshold of “success”.

  4. Since I don’t know the existence of any satisfactory, universal and measurable definition of ‘successful scientist’ I don’t see any rank, any top 1% or bottom 1%. There’s no linear scale there.

    Our job is already so hardly stressed by estimators, cuts, factors, indexes and ultimately marketing. We should just ignore this marketing stuff for a moment and do science.

  5. @Michele

    I agree but, for the purpose of my blog post, I needed to refer to a ranking of the researchers.

    When people talk about income inequalities (think: Occupy), they like to refer to the top 1%… so they rank human beings… from “richest” to “poorest”. I hear that there is a book by Piketty that makes a big deal of such things. Of course, such rankings are constructions that may have little relevance in the real world.

  6. But, if you are going to use a single measure to predict the future success of a scientist, you should go with the annual citations at the time of prediction. This is analogous to predicting the weather. If you don’t have advanced technology (e.g., meteorological satellites), one of the best heuristics is to say that tomorrow’s weather will be like today’s. What is analogous to meteorological satellites in a scientist’s career? Perhaps some instrument that measures passion and dedication?

  7. @Peter

    Though weather forecasts are handy, they are also short term (a few days). And they often provide probabilities instead of genuine predictions.

    Long term predictions, in the real world, are very hard.

Leave a Reply

Your email address will not be published.

You may subscribe to this blog by email.