Science and Technology links (April 28th, 2017)

It is estimated that our species, homo sapiens, appeared in Africa as far back as 200,000 years ago, and that we left Africa about 60,000 years ago. Confusingly, scientists found a 130,000-year-old archaeological site in southern California. So, maybe, there were human beings in America tens of thousands of years before our species left Africa. Homo sapiens were hardly the first human beings outside of Africa. It is believed that there were human beings of various types in Asia as far back as 600,000 years ago. It seems that some of them made it to America. What happened to them? How do they relate with us? Is any of this even true?

Our bodies produce myostatin, a protein that suppresses muscle growth. Some of us are naturally more muscular because we produce less myostatin. Obese people produce more myostatin. What happens if you take obese mice and prevent them from producing myostatin? Turns out that they appear to be healthier.

We all imagine that our ancestors were big and muscular whereas our descendants will be nerdy. In fact, human beings could get quite a bit more muscular in the coming decades. If you are paying attention, the process is probably already under way. When I was a kid, popular actors had unremarkable muscles, except maybe for a few like Arnold Schwarzenegger… I don’t imagine that most retirees will ever pump iron, but future therapies could keep older people quite a bit more muscular.

Kevin Kelly has a nice post on the myth of artificial intelligence as a threat to human beings. His core argument, which he made in his book the Inevitable and elsewhere, is that the concept of “general intelligence” is bogus. The Google search engine has intelligence that no human being can match, your dog has a kind of intelligence that you cannot match… and so forth. So we are unlikely to emulate human intelligence in our technology, and much more likely to produce forms of intelligence that best complement our own.

According to Yahoo! Finance, U.S. soda sales have been declining for the last 12 years. The article hints that public policies might be to blame:

The consumption of added sugar in foods and beverages has been linked to obesity and type 2 diabetes. The World Health Organization, the U.S. Food and Drug Administration and the American Heart Association have all recommended reducing consumption of soda as a way to cut down on added sugars.

So sugar is bad. What about salt? Low-sodium diet might not lower blood pressure: Findings from large, 16-year study contradict sodium limits in Dietary Guidelines for American. In other words, while you should cut down on sugar, salt is probably fine.

The Canadian prime minister Trudeau relies on cupping, the practice of creating a void within a cup that lies on the skin “to suck out pain, disease, and tension from the body”. Apparently, it can “suck out flu”. The prime minister appears in pictures with his shirt pulled up to clearly show the cupping marks, and his office confirmed the practice.

Economists seem to agree that the share of income that goes to labor is lower than it was. It is somewhat of a problem because most of us derive most of our income from our labor. Bloomberg has an article on the question. What is happening?

(…) companies themselves aren’t substituting machines for workers, as we might expect them to do if robots were getting really cheap. Instead, the economy is simply shifting resources toward a few large companies that are very capital-intensive, and away from the more numerous, smaller companies that use more human labor.

It is certainly true that companies like Google, Amazon, and Apple use fewer employees than comparable companies would have used in the past (like General Motors). They also rely a lot more on software and automation.

As you grow older, your bones and joints go to hell. I see many people in their forties and even thirties who suffer from painful joints, have chronic knee problems and so forth. It is not uncommon for people in their sixties and seventies to fall and break their bones. In a recent Nature paper, scientists found evidence that senescent cells, these cells that ought to die but slowly accumulate in our bodies over time, might be a significant part of the problem. It seems that removing senescent cells could help halt bone and joint aging (speculative).

Senescent chondrocytes are found in cartilage tissue isolated from patients undergoing joint replacement surgery, yet their role in disease pathogenesis is unknown. (…) Selective removal of the senescent cells from in vitro cultures of chondrocytes isolated from patients undergoing total knee replacement decreased expression of senescent and inflammatory markers while also increasing expression of cartilage tissue extracellular matrix proteins.

Billionaire Jack Ma predicts that in 30 years…

the Time Magazine cover for the best CEO of the year very likely will be a robot. It remembers better than you, it counts faster than you, and it won’t be angry with competitors

The puzzling part of his prediction is that he expects that the Time magazine will still be around in 30 years, and that it will still have a cover. Odd that.

Google has, at great cost, indexed 25 million books. A great fraction of these books are simply not available commercially. In many cases, it is not even possible to determine who “owns” the right to these books. Google could, at the flip of a button, make them available for free for the entire planet, to everyone rich and poor. Yet because of “copyright”, all these precious books will remain forever locked. You will have to go to Harvard or some other expensive school, if you want to consult the original that Google scanned. Though Google will not tell us about it, I bet that this large collection of books can prove invaluable for training machine learning software. In the novel Rainbows End, entrepreneurs are madly scanning books to train advanced artificial-intelligence software. Google had already done the scanning part: if the data is of any use for training artificial intelligence, they’d be silly not to use it. Of course, you won’t have access to the same data because the courts won’t allow it. (Further reading: Do we need copyright?)

IBM, after Google, is opening an artificial-intelligence research laboratory to collaborate with professor Bengio at the University of Montreal. So maybe it is a good time to do artificial intelligence in Canada, and in Montreal specifically. (Source: Claude Coulombe)

Crab blood is worth up to $14,000 per quart. Crabs like lobsters, spiders, and snails have blue blood because they rely on copper and not iron to carry oxygen. There are instances of all these animals that are very long lived, with lobsters exhibiting negligible senescence [meaning that their fitness does not diminish with age]. (Source: Gregor J. Rothfuss, P.D. Magnan)

Nice overview of the work accomplished by the late Hans Rosling:

Rosling’s first discovery was that many people are not aware of even the most basic facts about global health and global development. (…) He found that people’s worldviews often do not have much grounding in facts, even long before the “post-fact” era.

Rosling was criticized, rightly so, for offering a positive outlook on the world. However, what his critics often missed is that he offered all of us hard data… which is a lot better than whatever we go on usually.

Seth Godin asks What does “science” mean?

Science isn’t something to believe or not believe. It’s something to do.

This is a very important point. Science is a process. A scientist is someone who follows a rigorous process to arrive at the truth. There is faith in involved, but it is faith in the process, not in the results.

Believing something because it is in a science textbook as opposed to the bible is not what science is all about. The textbook is likely more reliable from a scientific point of view, but if you leave your doubts aside, you are missing the point of science.

I find that this point is regularly missed entirely, even by people who have a PhD in science. Science is not a body of knowledge. It is a process. The body of knowledge is useful, it is scholarship. But in that, a science like Physics is no different from Theology. Both of them benefit from the accumulated knowledge of the past. What makes Physics into a science is what physicists do, not what they know.

Byrne et al. in How Fast are Semiconductor Prices Falling? reassure us that the price of microprocessors is still falling fast if you adjust for quality:

The results from our preferred hedonic price index indicate that quality-adjusted microprocessor units prices have continued to fall rapidly, contrary to the picture from the Producer Price Index. Our results are consistent with other indicators of continued rapid technical progress in the semiconductor sector. Concerns that the semiconductor sector had faded as an engine of growth over the period covered by our analysis appear to be unwarranted.

Using stem cells, scientists have found a way to effectively rejuvenate the blood cells of old mice.

Immunotherapy is all the rage in oncology. The idea is to tweak your immune system into fighting cancer. When it works, it works better than just about anything. Sadly, it only works in 1 person out of 5, and we don’t know what sets these lucky people apart:

About 22 percent of melanoma patients that get a single round of treatment with Yervoy are alive 10 years later, (…)

What you should probably remember is that we are not winning the war against cancer. Not yet.

The Economist remind us that the reason we have so many cars is hat we effectively subsidize them through, among other things, free parking:

With such a surfeit of parking, most of it free, it is little wonder that most people get around Silicon Valley by car, or that the area has such appalling traffic jams.

Quickly pruning elements in SIMD vectors using the simdprune library

Modern processors have powerful vector instructions. However, some algorithms are tricky to implement using vector instructions.

I often need to prune selected values from a vector. On x64 processors, we can achieve this result using table lookups and an efficient shuffle instruction. Building up the table each time gets tiring, however.

Let us consider two of my recent blog posts Removing duplicates from lists quickly and How quickly can you remove spaces from a string? They follow the same pattern. We take a vector, identify the values that we want to remove, build a corresponding bit mask and then remove them. In one case, we want to remove repeated values, in another, we want to remove spaces.

Building the bit mask is efficient and takes only a few instructions:

  • We can identify the values to remove using vectorized comparisons (e.g., using the intrinsics _mm_cmpeq_epi8 or _mm256_cmpeq_epi32). This is typically very inexpensive.
  • We then build the bit mask from the comparison vector using what Intel calls a movemask (e.g., using the intrinsics _mm_movemask_epi8 or _mm256_movemask_ps). The movemask is relatively cheap, though it can have a high latency.

The pruning itself is ugly, and it requires a table lookup. I decided to publish a software library called simdprune to make it easier.

The library is quite simple. If you need to remove every other value in a 256-bit vector, you can get this result with the function call prune256_epi32(x,0b10101010).

Link: https://github.com/lemire/simdprune

The real lesson of the human genome project

When I was pursuing my PhD, the human genome project was often both regarded as overly ambitious (maybe even impossible) and full of possibilities. To many people’s surprise, the project was declared complete back in 2003, much earlier than expected. Today, we have a “roughly” complete map of the human genome.

Many announced, too soon it turns out, that it would quickly bring new cures, new therapies, that were previously unthinkable. Fifteen years later, the concrete results are somewhat thin on the ground.

But it made one very important difference. Before we could track down people’s genes, it was often believed that our destiny was written out in our DNA. You were what you genes said you were.

So people quickly moved to cataloging the genes of the super smart, the genes of the athletes, the genes of the centenarians. And what have they found? Not a whole lot.

There are a few genes that might help you live a bit longer. There are a few genes more frequent in athletes. And so forth. But the effects are often quite weak. There are exceptions here and there. Some genes give you a much higher chance of getting some cancers, or Alzheimer’s and so forth… but they affect relatively few of us in aggregate.

Chances are that if you have your genome sequenced, you will find that you are slightly lucky in some ways and slightly unlucky in others… but it is all quite boring.

If we could build a human body from the DNA up, selecting only the best genes, the result would be nothing like a superman.

If you have a toned body, some of your genes are activated, but it turns out that your nerdy friend who barely looks like a man has more or less the same genes, they are just silenced.

And so, I think that the main lesson is that human biology is far more malleable than many people thought. And this means that the badly dressed kid next to you, who looks like an unhealthy idiot? To a close approximation, he probably has a comparable genetic legacy to yours.

Science and Technology links (April 21st, 2017)

Can we trust software? Lance Fortnow, a famous computer scientist, answers

Sometimes I feel we put to much pressure on the machines. When we deal with humans, for example when we hire people, we have to trust them, assume they are fair, play by the rules without at all understanding their internal thinking mechanisms. And we’re a long way from figuring out cause and effect in people.

Luc Charlebois is a real-life Deus Ex character. He lost his leg ten years ago in a bike accident. The leg was just pulled off. He went to Australia where they remade his leg from scratch. He has an artificial leg that is directly connected to his skeleton. He can now walk and says that he when he does so, it “feels” like a true leg. That last part is critically important: we are a long way from wood pegs. But he had to spent a quarter of a million dollars to get it done. That’s not crazily expensive, but clearly out of reach to too many people. Given that this is now possible how long before we all ask for such high-quality artificial limbs?

Apple (the company) is allowed to test self-driving cars in California. “Siri, bring me back home.”

Daniel Lakens, an academic with a brilliant publication record, writes that blog posts are of higher scientific quality than journal articles. What he actually demonstrates is that it is quite easy to outdo scientific articles with something as silly as a blog post.

Using genetic engineering, we could one day selectively kill just one type of bacteria. This would turn modern-day antibiotics into blunt tools.

Want a growing industry? What about plastic surgery?

Somewhat mysteriously, obesity is a risk factor for bone health. Basically, beyond a certain point, the fatter you are, the more likely it is that your bones will break. That’s true even though heavier individuals tend to have higher bone mineral density. What is interesting is that we do not know why that is.

Currently, anesthesia is full of negative side-effects. Scientists are finding out that we could design better drugs that are free of these inconvenient side-effects.

The lens in our eyes tend to darken with age leading to a loss of vision, called cataracts. It is considered more or less unavoidable meaning that if you are old enough, you will have cataracts. We can simply replace the natural lens of your eyes, so it is not a crippling condition. There are several risk factors such as diabetes, age, sunlight… but wearing sunglasses, avoiding donuts, and staying inside won’t prevent cataracts. So what causes it? According to Beebe et al. it is the exposure of the lens to oxygen. Under normal conditions, in healthy young individuals, there is very little oxygen around the lens. But a liquefaction of the vitreous body of the eye or hyperbaric oxygen therapy can cause the lens to become exposed to oxygen, leading up to cataracts. Basically, your lens don’t interact well with oxygen. There is some evidence that if we can stop the exposure to oxygen, the lens could recover. Interestingly, we know how to repair the vitreous body with synthetic gels. So it is conceivable that we could one day develop preventive therapies against cataracts. At a high level, it might be as simple as keeping the vitreous body of the eye intact. The problem right now is that no doctor can tell you how much oxygen your lens are exposed to. So it is a difficult problem to study and catalog.

Netflix is reaching 100 million subscribers. This is far, far ahead of any cable TV company. In fact, cable TV companies are losing subscribers. Television is dying.

Following speculative claims that bees were being wiped out by the newest pesticides, neonicotinoids, the European Union banned these pesticides. Contrary to pesticides that you spray on the field, neonicotinoids are applied specifically on the seeds. These pesticides remain in wide use in Australia and North America, where both wild and honey bees are doing fine. Here is Matt Ridley on the consequences of the European ban in the Times:

In Britain, (…) farmers have more than quadrupled the number of insecticide applications on oil-seed rape (from 0.7 to 3.4 per growing season), but pest pressure has increased. Meanwhile, recent studies have demonstrated that declines among wild bees are driven mostly by land use changes and have not increased since neonics were introduced in the 1990s (…) This makes sense because neonics are mostly used as seed dressings, absorbed into the plant from germination, rather than sprayed on a growing crop. This makes them more lethal to pests such as flea beetles that eat the crop but less dangerous to innocent bystanders, including bees that collect pollen and encounter lower doses.

We ought to be critical of new technologies, but there is a difference between rejecting progress and being cautious. Some people do not want technological progress and they have much clout.

In diseases like Alzheimer’s, the environment of the brain deteriorates to the point that brain cells die. This lead to brain shrinkage and to cognitive decline. We are having a really like time fixing the environment of the brain. However, some clever scientists found out that we could stop the cells from dying. The brain might still be full of bad proteins, but the brain cells can still be coerced into surviving. In early work, the researchers achieved this effect but using a drug that would be toxic to human beings. However, Halliday et al. have now shown that we can get the same good effect with safe drugs that are already used in human beings. Speculatively, this means that if we can find in time that the environment of your brain is getting toxic for some cells, we could give you one of these drugs to prevent your cells from dying. This would not qualify as a cure for, say, Alzheimer’s, but it could turn it from a death sentence to something we can manage with medication and monitoring.

For a time, it looked like the blood (actually the plasma) of your people could rejuvenate older people. This lead to a good deal of unwarranted mockery. There is still much ongoing work in this direction, but I expect that it will end up being a dead end because, as suggested by work from the Conboy laboratory in Berkeley, it seems that we have “aging factors” in our blood… and not so much “youth factors”. Specifically, as we age, we might get too much of some factors in our blood. Thus, we could age younger people with the blood of older folks, but the other way around is unlikely to work. What we might need to do is to identify and normalize the aging signals in our blood.

The classic game Starcraft is now available for free, for both PCs and Macs.

We have this model of reality where we are this one person throughout time. So I remember the teenager I once was. I think of him as “me”. This may actually only be true in a very teneous manner. Psychologists have found out that there is very little correlation (none at all) between your personality as a kid and your personality as an elderly person. On the short term, you remain who you are, but your personality progressively changes and there does not seem to be any solid long-term anchor. The teenager I was? In a very real sense, he is dead. I no longer think like he did. Not in a meaningful way. It also means that even if I remain healthy for a very long time, who I am today will die over time and be replaced by someone else. Thus you cannot endure as an individual. I view this as a good thing.

Apple co-founder, Steve Wozniak, predicts that by 2075, we will have a colony on Mars.

Naked mole rates are long-lived tiny mammals. We still do not quite understand why they live so long. Interestingly, they can survive without any oxygen at all:

When the oxygen was completely removed and replaced with nitrogen, the mice died after 45 seconds. The naked mole rats passed out. But even after 18 minutes of no oxygen, they recovered when they were put back in normal air.

Why should you care about naked mole rats? Well. They are mammals not very different from us. They mostly have the same genes we do, plus/minus a few. If we can better understand how the cells of naked mole rates, we could use technology to mimick them. So, eventually, our cells could be taught to survive with very little oxygen. This would make us far more robust.

Science and Technology links (April 14th, 2017)

I announced last week that Tesla, the electric car company run by Elon Musk, had surpassed Ford in value. This week, we learned that Tesla has surpassed General Motors. Tesla is the most valuable American car maker.

Suppose I told you that you were about to die, right now. But I could offer you an extra 15 years of life if you just paid for it. How much would you pay? Probably a lot. Yet richer Americans live 15 years older than poorer Americans. If you care at all about inequality, you probably ought to be very concerned with health. Poor’s people health degrades significantly faster than that of richer people as they grow older. It is not just that rich people have access to better anti-aging creams… it is that they have fewer cancers, fewer strokes, and so forth. The solution, of course, is not to make Bill Gates sick and frail, but rather to find affordable technologies that can keep poorer people free from cancer and strokes.

On this note, the first major anti-aging trial on companion dogs has published its first results. There are too few dogs over too short a timescale, but results are promising. The drug being used, and believed to have anti-aging properties, is rapamycin:

Our results showed no clinical side effects in the rapamycin-treated group compared to dogs receiving the placebo. Echocardiography suggested improvement in both diastolic and systolic age-related measures of heart function (E/A ratio, fractional shortening, and ejection fraction) in the rapamycin-treated dogs (…)

Rapamycin is already used in human patients, so it is relatively safe. As a dog lover, I can’t want to have the means to slow down the aging of my 4-legged companions.

Octopuses are weird:

Octopuses have three hearts, parrot-like beaks, venomous bites, and eight semi-autonomous arms that can taste the world. They squirt ink, contort through the tiniest of spaces, and melt into the world by changing both color and texture. They are (…) capable of wielding tools, solving problems, and sabotaging equipment. (…) They (…) fine-tune the information encoded by their genes without altering the genes themselves.

Really weird:

Within months of mating with a female, the male will actually die. Consequently, males do not live as long in the wild as females do. Females will carry the fertilized eggs with her until they grow enough to be released, usually in strings hanging around her den. She can lay up to 100,000 eggs. For the next several months, the female protects her eggs from predators and ensures they receive sufficient oxygen. During these times, which can last 2 to 10 months depending on the octopus species, the female does not eat and slowly wastes away. She usually lives long enough to blow the eggs free from her den so the paralarvae can break free and join the plankton cloud.

Philip Greenspun asks:

If learning maritime history is mostly reading books that are available in libraries or are no longer within copyright, why couldn’t a degree in the subject be earned by downloading a reading list and writing some papers? If the mission of a public university is to educate the public at a reasonable cost, why didn’t they offer this kind of degree online starting in the 1980s when personal computers became popular or the 1990s with the rise of the Web?

The answer is obvious: public universities are not in the business of educating at a reasonable cost. Higher education is a luxury good, albeit a subsidized one.

We still don’t know what aging is, but we know that we can date the age of a cell using “epigenetic”, the set of conditions that determine which genes are expressed. As cells age, harmful genes that are normally inactive get activated while useful genes become silenced. We don’t know whether it is the cause or consequence of aging, or both, or neither. Some people suffer from a rare syndrome called Werner syndrome which is characterized by accelerated aging. So what happens to the cells of people suffering from Werner syndrome? It turns out that, unsurprisingly, the epigenetic of their cell is “old”. That’s interesting because if people suffered from accelerated aging without corresponding epigenetic changes, we might think that the relevance of epigenetic changes was quite secondary. However, this new observation tells us that epigenetic changes are most likely central to the paradigm of aging. In turn, this means that any therapy that reverses epigenetic changes is likely useful.

Fabian Giesen reminds us about how memory bandwidth has evolved over the years: “the available memory bandwidth per instruction has gone down substantially (…) It gets even worse if we look at GPUs.” This is the sort of non-trivial architectural issue that makes the difference between an algorithm that looks good on paper (and in a textbook) and code that runs fast. In my opinion, it also explains why stressing computational models, without accompanying empirical evidence, is a terrible way to teach programming. Or, to put it another way, mathematicians make really bad programmers. Some might argue that the evolution Giesen alludes to is an artifact of our current hardware designs. It might be, but there is also something fundamental at play: it gets more and more difficult to be efficient if the data is not close to where the computer occurs.

Gray hair seems to be a powerful predictor of bad health. Basically, the more gray hair you have, the more likely you are to have a stroke or a heart attack. This should come at no surprise: gray hair is a symptom that your body is not doing what it should. Dyeing your hair is a quick fix, but it is unlikely to affect your risk of death. Though there are a few anecdotal cases of gray-hair reversal (linked with cancer therapies), there is simply no known way right now (except for dye) to reverse gray hair. This is somewhat intriguing and suggests that we do not know enough about gray hair.

There is a widespread belief among people who do not regularly program software that if you coded something, then you somehow understand it. That’s often not even true to a first approximation. There is a growing realization that nobody understands the advanced artificial intelligence software we are building and deploying. Of course, we do not. And we won’t. So how do we make sure that the software does what we want it to do? Short answer: we run tests. This is in sharp contrast with much of computer science which is fascinated by the possibility of proving things mathematically. Sadly, the power of mathematics can be severely limited in its ability to handle complex real-world systems.

If you have a critical heart attack, doctors might install an artificial pump to keep you alive. Amazingly, many patients go on to fully recover and can live without the artificial pump or any further surgery:

For the first time, what we have shown is that heart function is restored in some patients – to the extent that they are just like someone healthy who has never had heart disease.

Price of virtual-reality headsets are declining. The Oculus Rift can be had for $500. The superior HTC Vive is now $700. Alas the software, though abundant, is still not sufficient to push everybody into wanting a headset.

Scientists believe that they have developed a more efficient kind of wheat that could improve yields by 20% thanks to more efficient photosynthesis. This could be either used to grow more wheat, or to grow the same about of wheat while leaving more land to wildlife.

Currently, developing new drugs is prohibitively expensive. In fact, the costs are rising exponentially, according to Eroom’s law. Any technology that can speed up tests has the potential to reduce costs. It seems that the American FDA is interested in supporting “organ on a chip” technologies. Basically, in this approach, we use synthetic miniature organs to test new drugs for safety.

Burger King ran an ad that triggered Google devices and got them to talk about Burger King’s products. So we live in a world where enough people have speech-enabled computers that such an advertisement campaign can be effective. That’s not amazing. What is amazing is that nobody seems to realize that it would have been hard science-fiction only five years ago.

Parkinson’s is a terrible disease, with no hint of a cure in sight. The main problem is that affected people lose neurons in charge of producing dopamine. Clearly, a cure would involve putting back these neurons. In recent work, scientists have managed to tweak other cells that are otherwise abundant, astrocytes, into producing dopamine. That’s an intriguing hack.

Which jobs can be automated? According to some list on the Internet, we can’t automate astronomers and models… but we can entirely automate aircraft cargo handling supervisors and logging equipment operators. That’s using the current technology. Well. I think that astronomy has been largely automated, and models… well, ever heard of Photoshop? It does automate a lot of the modeling work, doesn’t it?

Aleksey Shipilёv has a long article on the Java memory model. It is interesting whether you care about Java or not. The lesson is that the software code written by the programmer is not what the machine does. Rather, the machine has to emulate an abstract machine that would run the code. It has a few concrete consequences:

  • The machine does not have to execute the code in the order specified by the programmer.
  • The machine does not have to execute all of the code (or even any of the code) specified by the programmer.

These nuances will haunt you when you try to dive deep into software. It is a bit like Physics… for the most part, the world around us is intuitive… until you look too far and then things start to act strangely.
Here is a nice quote:

In the same way, the limit on the speed of communication in Special Relativity gives rise to the relativity of simultaneity, the propagation delays on real hardware deconstruct the intuitive notion of simultaneity and global time.

Toyota demonstrated a “leg brace” to help paralyzed people walk. Of course, it is nothing else but a partial exoskeleton. Such technology would be useful not only to paralyzed people but to many of us if it were cheap enough.

The US government wants to use drone equipped with facial recognition to patrol the borders. I am not sure we have the technology yet to pull this off. Keeping thousands of drones in the air at all time is hard. Battery lives are short. Recharging batteries is a slow process. Drones fail, get hacked… But in time, maybe in 5 years or so, I could imagine drones patrolling borders in a cost-efficient manner. Not just borders, of course… For example, I’d love to have a drone patrolling the surroundings of my house. It could be activated whenever something suspicious is happening. Last year, people set fire to garbages in front of my house, and it damaged the pavement. We never found out who did it… What if I had had a drone to protect my house by getting close enough to get clear images? And I’m not even the top market for such technology. What about single women living alone or with young children?

Removing duplicates from lists quickly

Suppose you have lists of numbers where some values are repeated (e.g., 1,1,2,3,3,3,4,0,0). You want these duplicates (or repeated values) to be removed (e.g., 1,2,3,4,0). To avoid potentially expensive memory allocations, we want to solve the problem in-place, writing back the answer is the current array. This is a surprisingly common problem that arises when merging lists, determining the distinct elements, and in several probabilistic algorithms.

To set a reference, suppose I generate 1024 random numbers in the range [0,1024) and I sort them. This will generate a few repeated values. I want to remove them.

I use integers for my test, but we could equally work with pointers to strings or arbitrary objects.

In C++, we have an STL function for this very purpose: std::unique. On a recent Intel processor, it takes over 11 cycles per value in the array. (Java has a distinct method that does the same work.)

You might assume that this result cannot be improved much. Let us see how fast we can go.

You can gain a little bit of efficiency over STL by writing your own function:

size_t unique(uint32_t *out, size_t len) {
    if(len ==  0) return 0; 
    size_t pos = 1;
    uint32_t oldv = out[0];
    for (size_t i = 1; i < len; ++i) {
        uint32_t newv = out[i];
        if (newv != oldv) { 
            out[pos++] = newv;
        }
        oldv = newv;
    }
    return pos;
}

Somehow, this saves about one cycle per array value (we are just under 11 cycles per value). I am not sure why it seems to be a tiny bit faster.

The main benefit of writing our own function, however, is that it gives us a chance to think about the algorithm.

What hurts us in this code are the mispredicted branches that occur when I compare the new value with the previous one. Because I have few repetitions, the processor predicts that there will be none at all. When a repetition does occur, the pipeline must unwind and fix the problems caused by the mispredicted branch.

We can multiply the speed by about a factor of 4 (to less than 3 cycles per array value) with a branchless approach:

static size_t hope_unique(uint32_t *out, size_t len) {
    if(len ==  0) return 0; // duh!
    size_t pos = 1;
    uint32_t oldv = out[0];
    for (size_t i = 1; i < len; ++i) {
        uint32_t newv = out[i];
        out[pos] = newv;
        pos += (newv != oldv);
        oldv = newv;
    }
    return pos;
}

Can we do better? It turns out that using SIMD instructions (with the AVX2 instruction set), we can get to about 1 cycle per array value. In that case, the code is not only branchless, but it also operates on vectors of several values at once…

int _avx_unique_store(__m256i ov, __m256i nv, __m256i *o) {
    // use the last value from ov and takes the rest from nv
    __m256i recon  = _mm256_blend_epi32(ov, nv, 0b01111111);
    // next two lines rotate the value, so that last is first
    const __m256i mbom = _mm256_set_epi32(6,5,4,3,2,1,0,7);
    __m256i vT = _mm256_permutevar8x32_epi32(recon,mbom);
    // we compare the newly generated vector with the original 
    // comparing values with their preceeding values
    int M = _mm256_movemask_ps(_mm256_cmpeq_epi32(vT, nv));
    // N records how many values need to be kept
    int N =  8 - _mm_popcnt_u64(M);
    // next two lines prune out the values based on
    // the bit mask M, see https://github.com/lemire/simdprune
    __m256i key = _mm256_loadu_si256(uniqshuf + M);
    __m256i val =_mm256_permutevar8x32_epi32(nv,key);
    _mm256_storeu_si256(o, val);
    return N;
}

I realize that the vectorized code looks like gibberish but my goal is to assess the benefits over vectorization.

With vectorization, we are fully one order of magnitude faster than STL’s std::unique function.

As usual, my code is freely available on GitHub.

Further reading: Quickly pruning elements in SIMD vectors using the simdprune library

Robots have not yet stolen our jobs

Though it is not yet widespread, I encounter more and more people who seem to believe that there is growing unemployment, or falling labor participation rate, due to technology.

It is true that technology has impacted the job market. Simply put, back in 1900, everyone was a farmer. Almost the opposite is true today. And that’s entirely due to technology.

However, some people seem to think that a large fraction of the population cannot find jobs that pay enough to get out of bed because of our ever-increasing reliance on computers.

It is important to set the frame of reference. If we think that computers are having a big effect on employment, it makes no sense to start our analysis 50 years go. Let us narrow it down to the last ten years. After all, it has been a technologically intense ten years. Recall that ten years ago, we did not have the iPhone, YouTube was an ugly site with much smaller traffic, cloud computing did not exist. And so forth.

Being Canadian, I am going to focus on Canada. What? You object that you do not care about Canada? Well. Consider this: the computer revolution is an international phenomenon. Thus, if computers are wiping out jobs in, say, the United States, they should also wipe out jobs in Canada, France, Germany and so forth. After all, we all use the same computers.

There is a nice analysis by Jansen, an economist of the Royal Bank of Canada. He does observe that there has been a decline in the labor participation rate of about 2% during the last ten years, in Canada.

“Ah! Maybe that’s because of the computers!”

I should point out that this decline occurred in Canada, but not in other technologically advanced nations like Germany (where they have more robots than Canada). But I said I’d focus on Canada…

Jansen produces the following chart, where he reports both the actual participation rate and the participation rate you would get if the demography remained constant.

In other words, the participation rate is declining slightly because of demography. If we correct for demography, then the labor participation rate is constant. This means that computers are not wiping out jobs.

I have a more detailed and international analysis if you do not believe this simple analysis.

The facts simply do not fit the theory that computers and robots are wiping out jobs.

Not yet.

Science and Technology links (April 7th, 2017)

Many people suffer from obesity. I am not sure we know how to combat this epidemic effectively. There is a never ending stream of wonder diets, but nothing seems to actually work. Or we just tell people to have more will power. New research suggests that inactivity is more a consequence than a cause of obesity. That is, while moving too little can contribute to making you obese, once you are obese, your body favors inactivity.

Some time ago, Amazon surpassed Walmart in market value: Amazon is worth (430B$) about twice as much as Walmart (225B$). Amazon has about 340,000 employees whereas Walmart has over 2 million employees. For some perspective, Apple is worth nearly twice as much as Amazon (800B$), but it has far fewer employees than Amazon (about 100,000). We have learned this week that Tesla, the electric car company, surpassed Ford in market value. Unlike Ford’s cars, all of Tesla’s cars have full self-driving hardware. Ford has 200,000 employees; Tesla has 30,000 employees. By the way, Amazon is worth about ten times as much as Ford.

Our cells produce their energy in its mitochondria. The mitochondrion is like a tiny cell (with its own DNA) that lives within our cells. (Red blood cells are an exception as they produce their energy by fermentation, like cancer cells.) Having lots of healthy mitochondria is critical. Cells that have a hard time producing energy don’t work as well. Our brain uses a lot of energy so it needs lots of good mitochondria. It is possible that neurological diseases (like Parkinson’s) could be tied to mitochondrial dysfunction. Anything from cancer all the way to skin wrinkles might be tied to mitochondria. So what harms mitochondria? We don’t know everything yet, but Toda et al. found that too much sugar may harm the mitochondria within our neurons.

What might you do if someone has too few good mitochondria? What about just injecting new ones? It seems that it might work:

Mitochondrial dysfunction is associated with a large number of human diseases, including neurological and muscular degeneration, cardiovascular disorders, obesity, diabetes, aging and rare mitochondrial diseases. Replacement of dysfunctional mitochondria with functional exogenous mitochondria is proposed as a general principle to treat these diseases. Here we found that mitochondria isolated from human hepatoma cell could naturally enter human neuroblastoma SH-SY5Y cell line, and when the mitochondria were intravenously injected into mice, all of the mice were survived and no obvious abnormality appeared.

If you care, however, about how many mitochondria you have, you should know that high-intensity training increases the mitochondria count of your muscles. This likely means that high-intensity training makes your muscles better at producing energy.

Quick: what operating system people use most to browse the Web? If you said Windows, you are wrong. Android is the Web’s most popular operating system. Android’s parent company (Google/Alphabet) is worth more than Microsoft.

Stem cells are these specialized cells in our bodies that are in charge of generating all other cells. There is currently a race to produce therapies based on stem cells. We are developing cheap ways to generate stem cells for therapeutic purposes. The idea is that you just inject stem cells at the right place in just the right way, you might spur regeneration. What might you want to regenerate? What about injured tendons for example? Sadly if you just inject stem cells in an injured tendon, you do not get the kind of regeneration we need. For example, the stem cells may just go elsewhere, where you do not need them. However, Peach et al. found that if you just implant the stem cells within a synthetic matrix, that acts as a niche for the stem cells, they stay in place and do the work we expect, regenerating tendons.

Greg Linden reports that though 6.3 million VR headsets were sold in 2016, “almost all so far just the cheap toys where you slot your smartphone in to use as the screen”. As an anecdote, my wife got a free VR headset with her latest smartphone, and she just told the clerk to keep it. Greg also reports that tablet sales a falling.

Apple quietly released a new iPad, its famous tablet. It is inexpensive (for an Apple product), it has great processor performance, and a nice screen (albeit without reflective coating). I own an iPad Air and I won’t be upgrading.

Apple has been designing its own mobile processors (the Ax series) with great success. Apple’s mobile products have fast processors. The single-core performance of Apple’s latest iPhone is on part with Intel’s laptop processors. It now looks like Apple wants to design and own its graphics processors. Up till this point, Apple was relying on designs from a company called Imagination Technologies. Apple told them that it was over. It seems very likely that Apple has been developing its own technology quietly for some time and that they are ready to produce it. This comes at a time when graphics processors are increasingly used for artificial intelligence, in addition to graphics.

Whereas mitochondria provide the cells with energy, the lysosome collects and recycle the garbage (what scientists call “autophagy”). When either of those goes bad, the cell is in trouble. A Korean team found that some kind of product (“ataxia telangiectasia mutated inhibitor KU-60019”) can do some good when it happens:

In turn, this reacidification induced the functional recovery of the lysosome/autophagy system and was coupled with mitochondrial functional recovery and metabolic reprogramming.

So, in yeast, these scientists were able to “reboot” the cells so that both energy production and garbage collection were back in full strength. Though you may not care about yeast, it is possible that the same trick could work in mammals like us. I presume that these researchers must be working on mice as we speak.

Ethiopia is building a space programme.

How fast is JavaScript, the programming language used by your browser? Fast enough to run high-performance databases:

This demonstration explores the novel and unconventional idea of implementing an analytical RDBMS in pure JavaScript so that it runs completely inside a browser with no external dependencies. Our prototype, called Afterburner, generates compiled query plans that exploit two JavaScript features: typed arrays and asm.js. On the TPC-H benchmark, we show that Afterburner achieves comparable performance to MonetDB running natively on the same machine.

Sabine Hossenfelder comments in Nature about the current state of science:

We’re judged by our publication count — or at least it’s what we think we’re being judged by — and stricter quality measures in theory development would cut back productivity. (…) To me, our inability — or maybe even unwillingness — to limit the influence of social and cognitive biases in scientific communities is a serious systemic failure. We don’t protect the values of our discipline. The only response I see are attempts to blame others: funding agencies, higher education administrators or policy makers. But none of these parties is interested in wasting money on useless research. They rely on us, the scientists, to tell them how science works.

Optimizing productivity is important, but you have to produce the right things. Producing research papers in great quantities is hardly worth anything. It is not like research papers are scarce as it is. We have a glut of research papers: more than you could ever read. Obviously, what we want, what we need, is scientific progress. I believe that producing research papers can be part of a productive process… telling others about your work is a great way to help push things forward… but it should never be the end product. Here is an analogy. The goal of a programmer is not to produce software. If we started remunerating programmers by how many lines of code they write, we would sure get many lines of code. But that’s obviously not what we, the users, want. Yet, of course, producing lines of code is part of a useful process to serve users. So how do we fix the problem that Hossenfelder outlines? She is right that it is useless to complain about the pressure to publish. We are all under pressure to be more productive and that’s not a bad thing. What is missing, and somehow Hossenfelder fails to mention it, is more “skin in the game”. What I mean is that scientists need to profit when their research help us progress, and they need to suffer when their research (irrespective of how many papers are published) fails to help us or, worse, holds us back. We have to realize that the current funding practices simply do not provide these necessary incentives. If you are a scientist doing useless work as part of a large cohort of equally useless people, you will be, everything else being equal, more successful than a lone scientist trying to find more useful alternatives… because of peer review. Peer review ensures mediocrity. Scientists should be reviewed by people who have strong incentives to fight against cargo-cult science.

The bulk of medical research occurs in the United States, and much of it is funded by the US government through the National Institutes of Health (NIH). It looks like White House is planning to cut by 20% NIH’s funding which could have adverse consequences on scientific progress. My understanding of US politics is that the president in not in charge of the budget, the congress is, so I am not sure what to make of it all. Moreover, the bulk of the medical research, in the US, is privately funded (75% of clinical trials are privately funded), and an overwhelming majority of new drugs and therapies come from the private sector. So while I would favor massive increases in funding for research, especially medical research, I do not think that, in the current system, there is a strong correlation between scientific progress and government funding. Sadly. We need to fix the incentives.

Moore’s law is the empirical observation that the number of transistors in processors doubles every two years. It was made in 1975 and it has held true for the last few decades. However, many people have proclaimed that Moore’s law is dead. Not so fast

If you divide the number of transistors in Intel’s current [processor] by the surface area (…), the rate of improvement still equals out to more than double every two years, keeping Moore’s law on track

What is certainly true, in my experience, is that successive Intel commodity processors do not generally make our software runs faster, much faster. That’s what leads people to call Moore’s law as being dead. But we should not conclude that the technology is stagnating just because one metric is plateauing (performance of Intel’s commodity processor).

Google is giving us more details regarding the hardware they use for machine learning. It seems to be between one and two orders more efficient than commodity processors. It is capable of 92 trillion 8-bit operations per second. For a company the scale of Google, it matters. Over time, it would be interesting to contrast these results with the evolution of Moore’s law. That is, can Google manage to keep on improving its custom processors at a rate that exceeds that of companies like Nvidia and Intel?

Another computer science professor warns us about technological unemployment:

“If you are a truck or taxi driver, you should be very concerned,” the Dalhousie University computer science professor [Stan Matwin] said in an interview Monday. “In five to 10 years, I think we will see a substantial amount of cars that will be driving automatically.”

We have been told for decades that salt is the silent killer, causing high blood pressure and leading to heart attack. Nah.

the consumption of a high salt diet is not the cause of hypertension and that there are other factors, such as added sugars, which are causative for inducing hypertension and cardiovascular disease

We have learned that it is useless to load up on antioxidants through supplements, and it might even be harmful. So taking vitamin C supplements is probably not good (but eating oranges is fine). However, that’s only true in general. Scientists have found that massive doses of vitamin C could be helpful for cancer patients. It seems that the effect is related to the fact that cancer cells tend to have dysfunctional mitochondria. These generate lots of active iron molecules that react with vitamin C to create nasty stuff damaging cancer cells specifically:

we verified convincingly that increased redox active metal ions in cancer cells were responsible for this differential sensitivity of cancer versus normal cells to very high doses of vitamin C

So maybe we shall treat cancer patients to megadoses of vitamin C in the future. Linus Pauling would be happy. To be clear, the point here is not that vitamin C can cure or prevent cancer, but rather that it can do a lot more harm to cancer cells than to rest of the body, thus complementing other treatments.

Maybe Earth is not a planet. But Pluto might be officially a planet again. It seems like astronomers have a lot of time on their hands.

We might be able to cure allergies using nanoparticles.

Last week, I reported how scientists they could, maybe, partially reverse aging by reversing the decline of nicotinamide adenine dinucleotide (NAD). Young mammals have lots of NAD, older mammals have little of it. NAD helps keep your cells healthier and having less of it means that your cells are more fragile. You can’t eat NAD, but you can convince your body to get more by consuming its precursors: nicotinamide riboside (NR) and nicotinamide mononucleotide (NMN). Both of these should be available at a health store near you, and you can order them from Amazon if you want. (Note: I do not recommend doing so, and I am not taking this stuff.) I thought it was early to go to the press and claim that a cure against aging was around the corner. As it turns out, according to this article in Nature, NAD-precursor clinical trials are already ongoing in Japan. So it is further along than I thought. Still, I think expectations should be modest. Given that NMN is freely available, if it turned back the clock in an obvious manner, we would know by now. It is certainly not going to make wrinkles and gray hair go away. However, we could hope for it make older people healthier. Maybe. Hopefully. So why Japan? I’d like to know. Here is something nice I’d like to stress… if these trials show that NMN is helpful… makes us healthier as we grow older… we can all go Amazon and order the stuff. So even if the impact is modest on each individual, the overall impact could be huge given that it is relatively cheap and widely available. But, again, please wait for the science to come in before you eat the stuff or put it on your face. We still know too little.

Need to get a surgery that you can’t get in your country? China wants you to come over as a tourist to get your nifty medical treatments. This could give the “made in China” statement a whole different meaning.

Looks like the next generation of PC memory chips (coming out in 2018) could have twice the speed. This could be a big deal. Getting access to the data is often half or more of the work. A slower processor with faster memory can sometimes be faster.

Though you might think that you can see all colors, in reality, we only see blue, red and green colors. Researchers have found a way to fool our eyes into seeing more colors with special filters.

The thymus in an organ responsible for producing new white blood cells. As you age, your thymus shrinks to nothing and stops doing its work. Younger people can also have a damaged thymus due, for example, to cancer. So your immune system becomes ineffective. In particular, it means that you lose much of your ability to fight cancer through your immune system. There are ongoing clinical trials to regenerate the thymus, with the hope that it will give people better health (fewer cancers, fewer diseases). In a Nature article, researchers explain how they were able to construct an “artificial thymus”. So if you have cancer, instead of relying either on donated white blood cells or on your own blood cells, we could create custom white blood cells using organoids that are engineers to fight particular diseases like cancer.

Senescent cells are cells that our bodies accumulate with time. They are supposed to die, but they just stay there, causing trouble. The more you get, the more trouble they cause, and since they won’t die, you get more and more over time. Clearing senescent cells is all the rage right now. There is a wonder drug called rapamycin that’s used as an immunosuppressant for transplant patients. It was first discovered on Easter Island, kept in a scientist’s personal fridge somewhere in Montreal after his employee shut down, and later commercialized. Rapamycin is not something you’d want to normally take, as it has several bad side-effects, like increasing your risk of diabetes and suppressing your immune system… However, it also seems to ramp up your body’s natural defenses through the Nrf2 pathway. It seems protective against cancer, for example. Incredibly, it looks like rapamycin is a senolytic, that is, a drug that helps clear out senescent cells. This makes rapamycin a very interesting subject of investigation. If only we can minimize its side-effects, maybe by altering it somehow, we could have a powerful drug that keeps us healthy.

It is now understood that the bacteria in your gut play a key role in keeping you healthy. They found that by injecting young poo into old fishes, they could make these old fishes live older. The equivalent process in human beings would be mind boggling. Close your eyes and imagine rejuvenation clinics where older people can receive, on a routine basis, transfusion of teenager’s poo. Hmmm…

Multiple sclerosis is a terrible degenerative disease. There is no therapy right now that can even slow the progression of the disease. However, there is renewed hope as a new drug, ocrelizumab, seems to do just that. It completed phase III clinical trials and it seems to work. Hopefully, it will be commercialized soon. It is not a cure yet for multiple sclerosis, but slowing down the disease is important.

How much are elite universities worth?

Will you do better if you attend a selective college?

If you are Caucasian (white) and from a middle class or better background… you will not: “Our estimates of the return to college selectivity (…) are generally indistinguishable from zero” (Dale and Krueger, 2014)

It is important to keep the facts straight. Kids who attend college, especially elite colleges, tend to have an excellent background and to be quite smart. So we know that they will do well. But they won’t do better if they have the name of an elite college on their resume. Having the ambition and the grades to get into an elite school is enough.

Further reading: What Is an Elite College Really Worth?