Many people (Robert Clausecker, Sam Westrick, Tomasz Kowalczewski, Vinoth Deivasigamani, Sergey Slotin and many others<\/span><\/span>) proposed using ‘strided access’ benchmark.<\/p>\n I finally decided to test a strided copy<\/em>: from a large array, I copy every N<\/em> bytes to another large array. It is a problem that should be largely “memory access bound” as long as N<\/em> is not too small. I start N<\/em> at 16. Importantly, I never read my own writes, so I avoid concerns with 4K aliasing on Intel processors.<\/p>\n If N<\/em> is larger than twice the cache line, then I can effectively skip one cache line out of two. If N<\/em> is smaller than the cache line, then every cache line must be accessed. Having a stride value just above the cache line should not be sufficiently to see large gains: but you expect the speed to almost double once you reach twice the size of the cache line if the only thing that matters are cache lines.<\/p>\n Sadly, several other factors come into play on a modern system with such a benchmark. There is more than just the cache-line size as a variable! So we need to verify the model experimentally.<\/p>\n I wrote the benchmark in C<\/a>, but the actual C compiler is unlikely to be relevant. The original code was in Go and I got the same results, but I switched to C to make sure I avoided Go-specific issues. Interestingly, ChatGPT converted the Go code to C code automatically for \u00a0me, with just a couple of small mistakes. Mind you, it is deliberately simple code.<\/p>\n We are measuring the time it takes to copy one byte out of every N<\/em> bytes, using a total buffer size of 32 MB. We report the numbers in GB\/s, effectively dividing the total time by the buffer size. I stress that it is a strided<\/em> copy, not an actually copy, so as stride (N<\/em>) grows large, the speed<\/em> should increase. However, as long as the stride (N<\/em>) is lower than the cache line, we expect a flat speed.<\/p>\n I run each experiment, for each stride size, 10 times and I record the maximum, the minimum and the average. I use an Apple M2 processor running on my laptop and an Intel-based server. I do not require a particular memory alignment when allocating memory. I do not make any other attempt to control the results.<\/p>\n The numbers on the server are quite nice, with hardly any difference between the average, the maximum and the minimum. If your stride is 129, you are 66% faster than when your stride is 64. This suggests that the cache-line size is 64 bytes. The gain is not 2x as I would have expected but the processing might be loading cache lines speculatively. Observe how a stride that is a multiple of 64 (e.g., 128 or 256) is slightly bad: we see the performance dip visibly. It might be due to a caching issue e.g., only half the cache-line addresses are used which makes it more difficult for the processor to make full use of its cache (due to address aliasing).<\/p>\n The result on my laptop are much less clean even though I kept the laptop unused during the benchmarking. In this instance, if your stride is 257, you are more than 2 times faster than when your stride is 128. It suggests that the cache-line size is 128 bytes. Just like the Intel system, a stride of 128 is unfortunate: there is a visible performance dip.<\/p>\n Note that we do not actually copy data at 300 GB\/s, that would be impossible on its face on my hardware: but we can copy an array that fast if we just copy one byte per block of 512 bytes.<\/p>\n Thus you can empirically measure the size of the cache line with a strided copy of a large array… As soon you use a stride that is twice the cache line, you should be more than 50% faster.<\/p>\n","protected":false},"excerpt":{"rendered":" Our computers do not read or write memory in units of bits or even bytes. Rather memory is accessed in small blocks of memory called “cache lines”. For a given system, the cache line size is usually fixed and small (e.g., \u00a016 to 256 bytes). All Intel\/AMD x64 systems I have used relied on a … Continue reading Measuring the size of the cache line empirically<\/span><\/a><\/p>\n","protected":false},"author":56,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[84],"tags":[],"class_list":["post-21030","post","type-post","status-publish","format-standard","hentry","category-84"],"_links":{"self":[{"href":"https:\/\/lemire.me\/blog\/wp-json\/wp\/v2\/posts\/21030","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/lemire.me\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/lemire.me\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/lemire.me\/blog\/wp-json\/wp\/v2\/users\/56"}],"replies":[{"embeddable":true,"href":"https:\/\/lemire.me\/blog\/wp-json\/wp\/v2\/comments?post=21030"}],"version-history":[{"count":5,"href":"https:\/\/lemire.me\/blog\/wp-json\/wp\/v2\/posts\/21030\/revisions"}],"predecessor-version":[{"id":21304,"href":"https:\/\/lemire.me\/blog\/wp-json\/wp\/v2\/posts\/21030\/revisions\/21304"}],"wp:attachment":[{"href":"https:\/\/lemire.me\/blog\/wp-json\/wp\/v2\/media?parent=21030"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/lemire.me\/blog\/wp-json\/wp\/v2\/categories?post=21030"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/lemire.me\/blog\/wp-json\/wp\/v2\/tags?post=21030"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}![\"\"](\"http:\/\/lemire.me\/blog\/wp-content\/uploads\/2023\/12\/intelicelake.png\")
<\/a><\/p>\n![\"\"](\"http:\/\/lemire.me\/blog\/wp-content\/uploads\/2023\/12\/applem2.png\")
<\/a><\/p>\n