Multicore programming? Yawn!

It looks like Intel is trying to push parallel programming. No doubt many colleges are going to keep surfing on the parallel-programming hype — to predict a new surge of interest in Computer Science. Alas, there is no upcoming multicore revolution in computer programming.

  • For a large fraction of enterprise problems, the bottleneck is at the database level. The ubiquity of Web servers and distributed databases (see CouchDB) imply that many such problems are already parallelized. Database techniques like partitioning have been around for years to help you parallelize your databases. This blog runs on a server with several processors, and it has done so for years. Nothing new on the horizon.
  • MapReduce and Hadoop help you parallelize many of the remaining hard data processing problems without having to mess with threads, locking and synchronization.
  • Many hard problems are memory-bound: they are hard because all of the data does not fit in memory. If your problem is memory-bound or IO-bound, throwing more processing cores at it may not help at all.

I have stated for a couple of years that storage, not processing power, is changing Information Technology. What is most amazing is our ability to record almost every single bit of information, and never have to delete or forget anything. On this topic, see my posts One More Step Toward Infinite Storage, Solid-state drives: when external memory becomes as fast as internal memory and What is infinite storage?

The truth is that we are not very good at dealing with large quantities of data. Anyone knows what to do when handed 50 terabytes of raw data? Few of us have the required skills to manage and leverage extremely large databases. Those will be the valuable skills in the future.

Tweet about this on TwitterShare on Facebook0Share on Google+0Share on Reddit0Email this to someone

5 thoughts on “Multicore programming? Yawn!”

  1. I don’t know if I can agree that “there is no upcoming multicore revolution in computer programming”. I consider making parallel programming easier at the small scale a revolution in computer programming, just like the introduction of garbage collection into mainstream (by Java).

    By small, I mean code that runs on a local machine and is not about intensive data processing. I guess “client” is a good characterization. For instance, many interactive applications can benefit from multicore but not yet, usually because the ability to multithread is not exposed at a suitably high level and so it doesn’t get used until late in the game. However, I think all of us would enjoy some speedup here and there in our daily work on our laptops, no?

    Intel, in particular, has made a good step in releasing the Thread Building Blocks that efficiently supports nested and irregular parallelism. TBB is what I would think of when I hear “intel” and “multicore”, and it most certainly is not about replacing OpenMP and MPI which are geared for high performance computing. (And now one can start talking about MapReduce and friends… but we are getting “large”, and at the large scale, I am with you.)

    I should add that there is a common thread (pardon the pun!) among the parallel programming ideas large and small: expose the parallelism and let the scheduler worry about the actual scheduling. Now perhaps we can both agree that the only thing that have ever happened in computer programming is higher and yet higher level of programming. 😛

  2. I don’t know if I can agree that “there is no upcoming multicore revolution in computer programming”. I consider making parallel programming easier at the small scale a revolution in computer programming, just like the introduction of garbage collection into mainstream (by Java).

    By the Spolsky’s principle of leaking abstractions, I don’t think we can ever expect “small scale parallel programming” to be ever as easy as Java’s garbage collection.

    However, I think all of us would enjoy some speedup here and there in our daily work on our laptops, no?

    There is no question that some client software is CPU-bound. Video games and image processing software are examples. We might see a revolution in video editing tools, for example. I can imagine live 3D rendering with a photo-quality. The Playstation 3 has what? 7 cores? I would not think of designing a gaming engine without thinking about multicore CPUs.

  3. > By the Spolsky’s principle of leaking abstractions, I don’t think we can ever expect “small scale parallel programming” to be ever as easy as Java’s garbage collection.

    Ever tried Erlang?

  4. No multi-core revolution in enterprise, but there’s definitely one in multimedia processing. Just look at the speed-ups the Cell processor in the Playstation 3 can get in signal and graphics processing when used correctly.

Leave a Reply

Your email address will not be published. Required fields are marked *