5 April 2026 · 3 min
Suppose you have several strings and you want to count the number of instances of the character ! in your strings. In C++, you might solve the problem as follows if you are an old-school programmer.
size_t c = 0;
for (const auto &str : strings) {
c += std::count(str.begin(), str.end(), '!');
}
You can also get fancier with ranges.
for (const auto &str : strings) {
c += std::ranges::count(str, '!');
}
And so forth.
But what if you want to go faster? Maybe you’d want to rewrite this function in assembly. I decided to do so, and to have fun using both Grok and Claude as my AIs, setting up a friendly competition.
I started with my function and then I asked AIs to optimize it in assembly. Importantly, they knew which machine I was on, so they started to write ARM assembly.
By repeated prompting, I got the following functions.
count_classic: Uses C++ standard librarystd::countfor reference.count_assembly: A basic ARM64 assembly loop (byte-by-byte comparison). Written by Grok.count_assembly_claude: Claude’s SIMD-optimized version using NEON instructions (16-byte chunks).count_assembly_grok: Grok’s optimized version (32-byte chunks).count_assembly_claude_2: Claude’s further optimized version (64-byte chunks with multiple accumulators).count_assembly_grok_2: Grok’s latest version (64-byte chunks with improved accumulator handling).count_assembly_claude_3: Claude’s most advanced version with additional optimizations.
You get the idea.
So, how is the performance? I use random strings of up to 1 kilobyte. In all cases, I test that the functions provide the correct count. I did not closely examine the code, so it is possible that mistakes could be hiding in the code.
I record the average number of instructions per string.
| name | instructions/string |
|---|---|
| classic C++ | 1200 |
| claude assembly | 250 |
| grok assembly | 204 |
| claude assembly 2 | 183 |
| grok assembly 2 | 176 |
| claude assembly 3 | 154 |
By repeated optimization, I reduced the number of instructions by a factor of eight. The running time decreases similarly.
Can we get the AIs to rewrite the best option in C? Yes, although you need SIMD intrinsics. So there is no benefit to leaving the code in assembly in this instance.
An open question is whether the AIs could find optimizations that are not possible if we use a higher-level language like C or C++. It is an intriguing question that I will seek to answer later. For the time being, the AIs can beat my C++ compiler!
Daniel Lemire, "Can your AI rewrite your code in assembly?," in Daniel Lemire's blog, April 5, 2026, https://lemire.me/blog/2026/04/05/can-your-ai-rewrite-your-code-in-assembly/.
[BibTeX]
My whole life everyone laughed knowingly about the promise of, but failure of, the “sufficiently smart compiler”. And while things like LLVM proved immensely smart, it’s clear there are whole new plateaus LLMs can reach
Super interesting idea!!! Thanks for sharing! It would be immensely interesting to see this idea being implemented in the compilers! What if, for each function/method, we extract the assembly instructions for that function/method, feed it to the LLM, get the optimized version from LLM, and replace those with the original instruction set! That would improve the performance a lot.
What optimization level did you use for the C++ baseline? Were the AI improvements faster than what the C++ compiler produced with -O3 ?
Yes, the AI did better than LLVM/clang when compiling in release mode. My source code and configuration is available (see blog post).
You didn’t post your configuration in the article or the GitHub.
I was using Apple M4 with the Apple compiler (llvm/apple).
It seems clang release does apply optimizations on release mode. My bad on the previous comment (in my defense: it was not obvious at all from quick googling).
Dunno if this situation a win though rather than the compiler optimizing for the cases more common string usages. Are you maybe thinking a process where we get LLM-custom-made assembly and run some fuzz tests vs the compiler to basically get improvements everywhere?
I don’t have a good model yet of where we are going. Thanks for your comments.
Did the LLM prompts include a limit on the length of the strings? I see none of them included a guard against lane counter overflow.
Compilers typically assume strings are about five characters long, so SIMD optimizations are sadly not popular.
I did not verify the code generated and, importantly, the AI were only instructed to go fast on this one benchmark.
Nobody should use this code in production. Ah!
It would only need to check after about 16k of input (on the 4-wide versions), so I’m wondering if it picked up the 1024 limit in the caller.
It’s a minor difference from the loop/horizontal sum in the current code (to do the horizontal sum every 255 loops), so I wonder if it tried to save those few instructions.
It had access to the code. So maybe.
What prompts did you use to get further optimizations beyond the initially generated versions? I mean, did it include any hints to use specific optimizations (like “use 64-byte chunks with multiple accumulators”) or did you simply repeat your initial request?
I did not give hints.
With all due respect, I think that Grok and Claude are not the best tools for AI assembly optimized rewrites, To my opinion, and based on my own experience, Ampcode is doing far better job.
I took your challenge to x86-64 using Ampcode.
On my Intel Core Ultra 9 275HX with GCC 15.2, std::count at -O3 -march=native already compiles to AVX2 vectorized code (I verified by inspecting the generated assembly). So unlike your ARM64 baseline which was scalar, I was trying to beat the compiler’s own SIMD output.
Ampcode generated 8 progressively optimized implementations in a single session.
The best one (AVX2 with 4 byte-accumulators and vpsadbw reduction) runs 2.17x faster than GCC’s vectorized std::count.
GCC wastes about 20 instructions per 32-byte chunk sign-extending bytes to 64-bit.
Our approach needs about 4.
There is also an arm64-neon branch ready for your M4 with 8 implementations, including two that go beyond your best (8 accumulators, 128 bytes per iteration).
Live results: https://zuwasi.github.io/Ampcode_assembly_benchmark/results.html
Source: https://github.com/zuwasi/Ampcode_assembly_benchmark
Impressive! Can you share the prompts that you used to achieve these results?
I did not use anything special for prompting. Just asking it to write fast inline assembly.
I have no secret.
I tried this locally on Gemma4 (26B), but I quickly hit a wall on the bitmask subtraction. It produced several bizarre alternatives including masking or shifting off the high bits to get a 1, or widening each lane to 16 bits. Adding hints that subtracting 0xFF increments an 8-bit integer, etc. sent it off into the weeds — at one point I had to stop it because it seemed to be in a loop.
Oddly when I fed it the “claude code assembly 3” solution and asked what that code does, it identified the “subtraction trick” and gave a reasonable outline of the whole, although its explanation of this step was vague.
I’m surprised because there should be plenty of training examples of this technique for counting matches.
Some kind of RAG might work: show it examples of code optimizations as part of your request.