18 March 2026 · 1 min
Modern processors have the ability to execute many instructions per cycle, on a single core. To be able to execute many instructions per cycle in practice, processors predict branches. I have made the point over the years that modern CPUs have an incredible ability to predict branches.
It makes benchmarking difficult because if you test on small datasets, you can get surprising results that might not work on real data.
My go-to benchmark is a function like so:
while (howmany != 0) {
val = generate_random_value()
if(val is odd) write to buffer
decrement howmany
}
The processor tries to predict the branch (if clause). Because we use random values, the processor should mispredict one time out of two.
However, if we repeat multiple times the benchmark, always using the same random values, the processor learns the branches. How many can processors learn? I test using three recent processors.
- The AMD Zen 5 processor can predict perfectly 30,000 branches.
- The Apple M4 processor can predict perfectly 10,000 branches.
- Intel Emerald Rapids can predict perfectly 5,000 branches.
Once more I am disappointed by Intel. AMD is doing wonderfully well on this benchmark.
Daniel Lemire, "How many branches can your CPU predict?," in Daniel Lemire's blog, March 18, 2026, https://lemire.me/blog/2026/03/18/how-many-branches-can-your-cpu-predict/.
[BibTeX]
The Rust guy at work said that Rust’s borrow checker makes the Rust assembly more predictable which makes it easier for the CPU to do branch prediction and that’s why Rust is so much faster than less-predictable languages like C and C++. Maybe you could get even higher numbers if you rewrote your benchmark using Rust.
No. My tests have nothing to do with the programming language.
This is not the case, rust is LLVM backed same as C++. There is no magic here with rust being able to hint to the branch predictor. What your co worker probably conflated is a feature where certain rust code can be optimized away since llvm knows certain facts about the code at certain points. This is most often used to describe how rust often eliminates its bounds checking since llvm can infer the size of things making some checks redundant. If you wanted to hint the branch predictor you could use a keyword like likely or unlikely.
You’re right about the Rust but likely/unlikely has nothing to do with hinting the branch predictor. Long time ago (Pentium 4) there were instruction prefixes that you could use to give a static hint to the CPU but these are not used since then and are superseded by the runtime branch prediction engine encoded in the CPU itself. You can see in the codegen (on godbolt for example) that likely/unlikely doesn’t generate any specific instruction or prefix.
That said, likely/unlikely only affects the compiler codegen layout, optimizing for putting together the code that is more likely to be executed, and in that way decreasing the i-cache misses, along with some other effects too. GCC documentation says “You may use __builtin_expect to provide the compiler with branch prediction information.”.
Fair point! Didn’t know that’s how likely/unlikely ultimately achieves it’s goal, thanks!
That sounds less like “Rust will be faster” and more like “C++ allows you to shove sticks into the spokes of your own bicycle, if for some reason you want to do that”. I would expect well formed code to perform very similarly regardless of the language. Also, I’m sure whatever problem cases you can cook up for C++ can also be represented in (probably unsafe) Rust if you really try, but they would just be less idiomatic and more blatantly silly.
I like Rust plenty but let’s not pretend it’s magic.
A very fun and interesting post! You’ve mentioned a clever technique for confusing the branch predictor through random values. I’ll remember that one,!
I forgot to mention this in my last comment, but some compilers for functional languages use a technique called “definitions luxation”, which turns closures and function pointers into algebraic data types that are then pattern-matched upon.
It surprised me at first that this would speed up the code because of the extra branch tests from the pattern matching, but it seems to enable inlining optimisations. That could be worth it, unless there are too many branch tests introduced.
By the no-free-lunch theorem, and the fact this 30k random branch pattern is so atypical in the real world, it would imply the loser here (Intel) is more likely to be the best branch predictor.
At least that’s my prediction.
To be fair AMD did a complete revamp of the BPU for Zen 5 (see e.g., https://chipsandcheese.com/p/zen-5s-2-ahead-branch-predictor-unit-how-30-year-old-idea-allows-for-new-tricks). It was of similar size as Intel for quite some time and they doubled L2 BTB from Zen 4 (https://chipsandcheese.com/p/amds-zen-4-part-1-frontend-and-execution-engine). I think your methodology is sound but if you go deeper on this, see prior art on measuring in Matt Godbolt’s series from 10y ago (https://xania.org/201602/bpu-part-three). One of Agner Fog’s excellent Software Optimization Guides has the numbers on BTB sizes, but unfortunately not for recent CPUs.
IMO the consumer chips like M4 don’t matter in terms of BPU performance at this point. The hyperscaler ARM chips, Nvidia’s Grace CPU in the Grace Hopper system, and/or the Loongon chips may be more interesting to compare.
I imagine this depends on the concrete processors and not just on the family? Which ones did you benchmark exactly, are all three roughly comparable?
I am not sure what you mean by ‘concrete processor’. Do you have evidence that Intel or AMD change the branch predictor for different versions of the chip when using the same microarchitecture? That would be surprising. I’d love to have a reference on this.
I think it would be hard to get a reference to it since BPU is one of the most pressing designs in the CPU given how large the misprediction penalty is so I figure that vendors aren’t going to be publishing many of the details.
That said, I wouldn’t find it surprising that vendors fine-tune the BPU capabilities depending on the workload that particular chip is intended to be fine-tuned for. Server CPUs vs mobile vs desktop within the same uarch have different trade-offs and they definitely differ, and maybe not always in the most obvious ways like BTB size.
My Zen 3 CPU (5950X) predicts 20,000.
20000 : 0.702 ns 1.42 Gv/s 0.00 bm
22000 : 1.062 ns 0.94 Gv/s 0.08 bm
24200 : 1.957 ns 0.51 Gv/s 0.21 bm
Thanks.
I’m curious about how this scales with the number of if statements. If you have two ifs with their own list of random values, can the AMD Zen 5 predict 15000 of each?
My more practical question is, game code is typically very branch heavy. Is there a case to be made that removing branches can make other branches behave better? For example if a game has a total of 10000 ifs, and I remove 1000 of them, will the remaining 9000 behave better? (this is not as hypothetical as it might sound 🙂 )
It is my experience that removing branches improves branch prediction. Please see the following post.
Daniel Lemire, “Adding a (predictable) branch to existing code can increase branch mispredictions,” in Daniel Lemire’s blog, November 6, 2019, https://lemire.me/blog/2019/11/06/adding-a-predictable-branch-to-existing-code-can-increase-branch-mispredictions/.
I’m confused by the graph, how can CPUs mispredict less than 50% if the value is random? I assume the values are zero in the start because that’s the initial batch of samples, but then there’s a period where misprediction per value ramps up to 0.5; this I don’t understand. The ramp up suggests it’s predicting odds and evens better than random before either the prediction gets worse, or the number random generator becomes better.
The values are random, but repeated.
here is data from my Zen 4 (7950x3d) machine:
31623 : 0.791 ns 1.26 Gv/s 0.00 bm
56234 : 0.796 ns 1.26 Gv/s 0.00 bm
100000 : 2.462 ns 0.41 Gv/s 0.32 bm
177828 : 3.138 ns 0.32 Gv/s 0.46 bm
316228 : 3.313 ns 0.30 Gv/s 0.49 bm
here is data from my Zen 4 (7950x3d) machine:
31623 : 0.791 ns 1.26 Gv/s 0.00 bm
56234 : 0.796 ns 1.26 Gv/s 0.00 bm
100000 : 2.462 ns 0.41 Gv/s 0.32 bm
177828 : 3.138 ns 0.32 Gv/s 0.46 bm
316228 : 3.313 ns 0.30 Gv/s 0.49 bm
In case it’s interesting…
Zen 2 (AMD Ryzen 9 3900X 12-Core Processor)
3162 : 0.957 ns 1.05 Gv/s 0.00 bm
5623 : 1.295 ns 0.77 Gv/s 0.05 bm
10000 : 2.724 ns 0.37 Gv/s 0.26 bm