Though Herb Sutter has been a C++ guru for decades, his name is most often associated these days with two words: "free lunch." That's thanks to his oft-cited 2005 paper, "The Free Lunch Is Over: A Fundamental Turn Toward Concurrency in Software." The author, Microsoft software architect and chair of the ISO C++ Standards Committee, took time recently to speak with Go Parallel about the multi-core chip revolution and how it's affecting software developers.

Let's start with the "you say tomato, I say 'to-mah-to'" question: Are the words "concurrent" and "parallel" interchangeable when it comes to software?

I personally use those terms synonymously; the meanings are essentially the same. Now, historically, computer science has often formally distinguished between "parallel" and "concurrent" as tags for two different kinds of work: There, the idea is that "concurrency" means two or more threads or processes that can operate asynchronously; "parallelism," on the other hand, implies scalability, or how to use 100 cores to get something done faster. My first Effective Concurrency column, "The Pillars of Concurrency," distinguishes these as two of the three major pillars of concurrency requirements, and calls them "isolation" and "scalability," which I think better help convey what they're about than somewhat artificially distinguishing between "concurrency" and "parallelism." Your mileage may vary.

But most people don't draw that strong a distinction between the two words now. "Concurrency" is the general word for multithreading using multiple cores, whether it's to do work asynchronously or to get an answer faster or both. As a verb, it's more convenient to say "parallelize." I use them pretty interchangeably.

How do you compare Threading Building Blocks (TBB) with the .NET Task Parallel Library?

They're both about "How do I do work on collections of things to use many cores?" They're targeting same second "pillar" of concurrency, throughput and scalability and using more cores to get the answer faster. OpenMP also targets the same pillar—it has already broken a lot of ground.

Can't you drop these algorithms into the C++ standard?

Even with parallel for, there are lots of design options.

First, side effects matter: You can't just automatically parallelize every for loop in existing code without knowing about the loop body's side effects. That's why OpenMP lets you opt in by spelling it with a #pragma omp parallel. Similarly, you can't just automatically parallelize every existing call to for_each. So a parallel version of for_each has to be distinct somehow, whether using a different name or putting the parallel algorithm into a different namespace, so that users can opt into the parallel version.

Second, the semantics of the parallel version of an algorithm may be different from the sequential version, in a number of ways that lead to a lot of design choices. A simple example is that a parallel find may return "a" match rather than "the first" match closest to the front of the container. A more subtle one is that when the sequential version returns "the first" match, you can then find the next match by starting at that position and continuing, but there's no directly corresponding "okay, resume here and find the next one" concept for a parallel find. Because the semantics of the parallel version of the algorithm can be different in several ways, each of which is a design choice or design dimension, I think we need more experience to be sure that a given set of design choices is best before it's ready to bake into a standard.

Third, a major design choice area is that you can turn the knob on usability vs. safety. For example, I mentioned side effects a minute ago; you can try to prevent mistakes and make a parallel loop harder to use, or on the other end of the scale you can do what OpenMP does and trust the programmer completely and make it easy to use but also easy to cut himself. Generally, the safer you make something, the harder it is to use; in this case, the harder it is to express what you want to do easily and/or efficiently. Finding the right balance here will take experience.

Have you seen much response to Microsoft's Task Parallel Library?

TPL has been out in the field about two months, and I haven't seen the feedback yet.