Never reason from averages

StackOverflow published its list of “top paying technologies“. Worldwide, the best-paid programmers, on average, work in Clojure and Rust (these are programming languages in case you are wondering). In the US, the top paying programmers work in Go and Scala. In the UK, it is TypeScript and Ruby. Yes, these are all programming languages, but I will excuse you if you have never heard of them.

Here is the analysis we are being offered:

Globally, developers who use Clojure in their jobs have the highest average salary at $72,000. In the U.S., developers who use Go as well as developers who use Scala are highest paid with an average salary of $110,000. In the UK, it’s TypeScript at $53,763 (…)

The implication seems clear enough: go study little-known programming languages to improve your salary! Clojure, Go and TypeScript, here I come!

Sadly, for programmers looking to improve their financial outlook, this information may be worse than useless. It might be dangerously misleading. To be fair, at no point does StackOverflow recommends this list as career advice… but I think it might be perceived as such.

There is nothing wrong with learning Clojure, Go and TypeScript, of course… but will it improve your odds of earning the big bucks?

Suppose you have to choose one programming language. You can go for JavaScript or Go. Of course, you can actually learn both in a couple of week-ends, but humor me. You want an income of at least $110,000.

There are 100 opened Go jobs and half of them pay that much. So you get to apply to 50 jobs. Great!

There may be 10,000 opened JavaScript jobs… but a much smaller percentage offering at least $110,000, say 10%… yet, this leaves you with 1000 jobs. Much better than with Go.

And, of course, the abundance of jobs at a salary below what you value yourself is a bonus. If your current JavaScript job does not pan out, you can find another one quickly. With a niche language? Not so much. You may actually end up unemployed.

These are made up numbers, but my point is that the average salary is only likely meaningful if you are comparing sets having a similar size.

In practice, I strongly suspect that you maximize your odds at earning large salaries by focusing on “standard” programming languages.

I submit to you that it is no accident if the StackOverflow list of top-paying programming languages is made of obscure languages. They are comparing the average of a niche against the average of a large population.

We see a similar problem when doing international comparisons regarding academic achievements. It is not uncommon for the US to have a poor standing. Yet the US is a very large and diverse country. Just look at the last two presidents, Obama and Trump. Not very alike, are they?

If you divide the US in states, then you find that their rankings are often all over the place… but that residents of Massachusetts should not be worried in the least about these rankings…

The U.S. continues to trail its peers in global measures of academic excellence. Based on results from the latest Program for International Student Assessment (PISA) test, of the 65 countries ranked, the US ranks 31st in math, 24th in science, and 21st in reading. (…) if Massachusetts were allowed to report subject scores independently — much the way that, say, Shanghai is allowed to do so — the Bay State would rank 9th in the world in Math Proficiency, tied with Japan, and on the heels of 8th-ranked Switzerland. In reading, Massachusetts would rank fourth in the world, tied with Hong Kong, and not far behind third-ranked Finland.

Statisticians know all about this. They would never let you publish a paper that compares unqualified averages with vastly different populations. The rest of us need to be reminded periodically not to reason from averages.

Daniel Lemire, "Never reason from averages," in Daniel Lemire's blog, March 28, 2017.

Published by

Daniel Lemire

A computer science professor at the University of Quebec (TELUQ).

8 thoughts on “Never reason from averages”

  1. Great point! I remember reading an international comparison of broadband speeds and, surprise!, the leaders were tiny countries. I am sure one can find many other examples.

  2. Another point: JavaScript jobs may be related with frontend development, including Angular and what not. Scala jobs may be related with big data, including Spark and what not. Learning the languages (a couple of weekends) means nothing. Some other things to consider are technology stacks, career paths, additional knowledge required in role descriptions, etc.

    In particular, in frontend and big data there’s some new hotness every few months, knowledge becomes outdated fast. Best advice has always been “do something that you enjoy”, because in the end that joy may be the only thing you get.

  3. Another point is that with small populations or “skewed” distributions, the median is a more significant figure than average. The average of [10 11 12 80] is 28.25, but its median is 11.5

  4. I fully agree with the article. Statistics can be even more misleading as people tend to chase form over substance.

    Moreover, while the study might show a _correlation_ between Rust and higher salaries, in no way does the study point towards _causation_. Maybe people programming in these languages earn high salaries because some other attribute they posses (being early adopters, understanding new tech quickly) goes hand in hand with TypeScript or Rust. But in no way does that mean that repeating what they did will yield the same results.

    Even if we override the whole correlation-causation argument, we are left out with a typical portfolio theory trade-off: in order to increase your profitability (earn more per year) you also have to increase your risk profile (place your “bet” on a hyped tech that might as well be gone or regress to the market average in 3-5 years, for all we know).

    As a conclusion, I think having a practical, solution-seeking, non-dogmatic mentality to technology is better correlated with high income than a specific programming language. People who ask themselves “What’s the right tool for the job in this context?” tend to earn more than those who think that just because they have a Go/Rust/TypeScript hammer everything else has to be a nail.

Leave a Reply

Your email address will not be published.

You may subscribe to this blog by email.