How fast can you open 1000 files?

1 March 2025 · 3 min

Jarred Sumner, the main author of the Bun JavaScript engine, commented a few days ago on X that opening many files on macOS could be slow due to thread contention: “your $5,000 computer is only capable of opening 1 file at a time”.

I was curious and I decided to test it out. I wrote a small C++ program which opens 10,000 files. The files are opened and closed immediately. However, there are 10,000 distinct (empty) files. It is a simple test: there is no writing and the files do not remain open long.

I use either 1 thread, 2 threads, 8 threads or 16 threads. The threads split the work equally: e.g., with two threads, you have two threads opening 5000 files each.

To make things interesting, let us add a “thread pool”: I create and launch all threads before the benchmark starts. Then I add all of the tasks (opening a file X) to the thread pool. This should be slower, but maybe more realistic of how engineers might solve such problems.

Indeed, with the thread pool, for each file, you need to grab a thread, assign the work, and wait for it to complete before more work can be assigned. You must grab thread for each file to be opened. When the unit of work is small (i.e., open a single file), thread pools are not very efficient. Assigning several files to be opened by a thread at once (in batches) is faster.

I run each test 10 times and I keep the median. There is some instability in the measures, I try to report representative numbers. My code is located on GitHub: I invite you to run it on your system.

Let us look first at the results on my Apple M2 laptop (it has 8 cores, 4 performance + 4 efficiency). Unfortunately, I do not have a beefier Apple computer at my disposal right now.

I report the total time.

Let us test it out under Linux on a big x64 server with 64 cores. Although it is a bigger machine, it has worse per-core performance than the Apple macBook in general (slower memory, slower clock, fewer instructions per cycle).

The macOS system has a faster disk, faster memory and faster cores. Yet opening files is clearly much slower under macOS according to this test.

I find it interesting that in both cases, using two threads minimizes the running time in the regular case: additional threads appear to slow things down.

With the big machine, a thread pool can go faster than the regular approach if I use four threads or more.

On my macBook, I cannot open much more than 120,000 files per second. My Linux server scales up to 400,000 files per second. In some cases, opening thousands of files could become a hard bottleneck. Throwing more threads at the problem might not work.