It looks like Intel is trying to push parallel programming. No doubt many colleges are going to keep surfing on the parallel-programming hype — to predict a new surge of interest in Computer Science. Alas, there is no upcoming multicore revolution in computer programming.

  • For a large fraction of enterprise problems, the bottleneck is at the database level. The ubiquity of Web servers and distributed databases (see CouchDB) imply that many such problems are already parallelized. Database techniques like partitioning have been around for years to help you parallelize your databases. This blog runs on a server with several processors, and it has done so for years. Nothing new on the horizon.
  • MapReduce and Hadoop help you parallelize many of the remaining hard data processing problems without having to mess with threads, locking and synchronization.
  • Many hard problems are memory-bound: they are hard because all of the data does not fit in memory. If your problem is memory-bound or IO-bound, throwing more processing cores at it may not help at all.

I have stated for a couple of years that storage, not processing power, is changing Information Technology. What is most amazing is our ability to record almost every single bit of information, and never have to delete or forget anything. On this topic, see my posts One More Step Toward Infinite Storage, Solid-state drives: when external memory becomes as fast as internal memory and What is infinite storage?

The truth is that we are not very good at dealing with large quantities of data. Anyone knows what to do when handed 50 terabytes of raw data? Few of us have the required skills to manage and leverage extremely large databases. Those will be the valuable skills in the future.

5 Comments

  1. I don’t know if I can agree that “there is no upcoming multicore revolution in computer programming”. I consider making parallel programming easier at the small scale a revolution in computer programming, just like the introduction of garbage collection into mainstream (by Java).

    By small, I mean code that runs on a local machine and is not about intensive data processing. I guess “client” is a good characterization. For instance, many interactive applications can benefit from multicore but not yet, usually because the ability to multithread is not exposed at a suitably high level and so it doesn’t get used until late in the game. However, I think all of us would enjoy some speedup here and there in our daily work on our laptops, no?

    Intel, in particular, has made a good step in releasing the Thread Building Blocks that efficiently supports nested and irregular parallelism. TBB is what I would think of when I hear “intel” and “multicore”, and it most certainly is not about replacing OpenMP and MPI which are geared for high performance computing. (And now one can start talking about MapReduce and friends… but we are getting “large”, and at the large scale, I am with you.)

    I should add that there is a common thread (pardon the pun!) among the parallel programming ideas large and small: expose the parallelism and let the scheduler worry about the actual scheduling. Now perhaps we can both agree that the only thing that have ever happened in computer programming is higher and yet higher level of programming. :P

    Comment by Maverick — 25/3/2008 @ 10:55

  2. I don’t know if I can agree that “there is no upcoming multicore revolution in computer programming”. I consider making parallel programming easier at the small scale a revolution in computer programming, just like the introduction of garbage collection into mainstream (by Java).

    By the Spolsky’s principle of leaking abstractions, I don’t think we can ever expect “small scale parallel programming” to be ever as easy as Java’s garbage collection.

    However, I think all of us would enjoy some speedup here and there in our daily work on our laptops, no?

    There is no question that some client software is CPU-bound. Video games and image processing software are examples. We might see a revolution in video editing tools, for example. I can imagine live 3D rendering with a photo-quality. The Playstation 3 has what? 7 cores? I would not think of designing a gaming engine without thinking about multicore CPUs.

    Comment by Daniel Lemire — 25/3/2008 @ 12:21

  3. > By the Spolsky’s principle of leaking abstractions, I don’t think we can ever expect “small scale parallel programming” to be ever as easy as Java’s garbage collection.

    Ever tried Erlang?

    Comment by Jan — 25/3/2008 @ 19:22

  4. Well I agree with you that revolution is nonsense, even if it where only for the fact that it’s evolution.

    Comment by Michael B — 28/3/2008 @ 4:32

  5. No multi-core revolution in enterprise, but there’s definitely one in multimedia processing. Just look at the speed-ups the Cell processor in the Playstation 3 can get in signal and graphics processing when used correctly.

    Comment by Vaughn — 16/5/2008 @ 7:51

Sorry, the comment form is closed at this time.

« Blog's main page

Powered by WordPress