f you're like many engineering-minded folks, chances are you've heard of the Discovery Channel show "Mythbusters." In a typical episode, the Mythbusters team takes a popular urban myth, such as the drop-a-penny-from-the-top-of-the-Empire-State-building-and-kill-somebody myth, and uses hard scientific experiments to prove it either true or false. It's a fascinating display of special effects wizardry and science. Plus, let's admit it, they love to blow things up and what hard-core geek doesn't like to see stuff get blown up?

Unfortunately, application developers have accepted several myths in our own industry. In particular, myths about running Java applications on multi-core systems are pretty pervasive amongst developers. While not as popular or ubiquitous as some of the ones debunked on Mythbusters, they're no less false or misleading.

We admittedly don't have the same kinds of experience in blowing stuff up, nor do we have a crash test dummy to subject to our experiments (poor Buster). However, we can still analyze some of the more common assumptions in the multi-core space today and show why they are, in fact, pure myths.

The "My App's Poor Performance Will Be Saved by Moore's Law" Myth
No other concept indirectly contributes more to poorly performing applications than Moore's Law (commonly cited as "processor speeds double every eighteen months"). To the developer who wants to focus on "getting it to work before getting it fast," Moore's Law is the magic that will make any application—no matter how slow on today's hardware—run twice as fast in just a year and a half. This is not only a common misrepresentation of Moore's Law, but as many developers are starting to notice, it doesn't hold true the way it once did.

Moore's Law actually states that the number of transistors that can be applied to a particular processor or chip is what doubles every eighteen months. For years, this indirectly led to the doubling of processor speeds, as any historical analysis of processor sales will display. But since 2001 or so, the steadily climbing speeds have suddenly flattened, and it doesn't take a lot of empirical evidence to see this. In August of that year, the average CPU speed in a high-end desktop or low-end server was around 2GHz, meaning that six years and four iterations of Moore's Law later we should be seeing processors at around 16GHz or so. A quick glance through online product descriptions from any major PC supplier will show that similar-ranged machines are hovering around the 3.0-3.6GHz range, more than a bit short of that 16GHz mark (click here for an example).

Interestingly enough, Moore's Law hasn't failed but the widespread perception is that it has. The number of transistors continues to double roughly every year and a half, but instead of trying to increase the raw speed of the processors themselves, the chip manufacturers are essentially "scaling out"—putting more CPU cores into the chip. In a sense, this means that developers are staring down the barrel of a multi-CPU machine every time they build an application. Even some of the lowest-end server hardware and many laptops are now multi-CPU systems.

So barring any major scientific breakthroughs in the next couple of years, developers can expect to see systems with multiple 2GHz processors rather than single CPU machines with 16, 32, or 64GHz processors. This means they will have to be deal with concurrency in order to deliver greater performance.

The "My App Server Will Scale My Java Code for Me" Myth
This popular myth emerged almost simultaneously with the release of the J2EE specification. Although it is true that Java app servers provide a degree of concurrency, many Java developers are under the mistaken impression that the app server will simply take care of all of their scalability needs. The attraction is undeniable: in this worldview, a developer need never think about hard problems like concurrency, transactional boundaries, or parallel processing. They can just think in terms of traditional Java applications and objects, and the app server will simply take whatever steps are necessary to ensure that everything just works and just scales. (For C++ developers, the lack of a standard app server means that there is even more work to do to ensure proper application concurrency.)

This is probably one of the most cherished viewpoints of the J2EE server space. As such, it is one of the most tenacious in the face of arguments to the contrary. Fortunately, it doesn't take a great deal of logical reasoning to see its inherent flaws. The belief begins with the basic assumption that when a particular J2EE application does not scale, it is due to the CPU not running fast enough. For most of today's applications, however, the CPU is not being kept busy. A large part of the application server execution time is spent transmitting data back and forth across components, taking out locks, or waiting for locks to be released. Spending some quality time with a profiler can show you just how much time your application spends on these tasks, which cannot benefit from a faster CPU.

This is an issue that an app server has little control over. In fact, an app server can sometimes even contribute to the problem if it tries to maintain shared state across multiple machines, as that state will need to be transferred back and forth across the various nodes in the cluster in order to maintain the illusion of zero hardware affinity.

Thus neither the CPU nor the app server can save the developer from the need to design concurrency into the application architecture.

The "My Cluster, Bus Operating System, or App Server Will Automatically Provide Ordered Control" Myth
At the heart of any clustering or web farm approach to scalability lies the belief that if you can't get better performance and scalability by running on faster hardware, you can at least get better scalability by running a bunch of machines in parallel and executing the code on all of them. This works only up to a point.

Amdahl's Law is less well known than Moore's Law but equally as important. It loosely states that the greater the percentage of sequential operations in a particular application or program, the less benefit can be derived from parallel execution. For example, even if only one percent of a particular program must be executed in sequence (which is an astoundingly small number that for most business applications is much greater), the maximum benefit that can be derived from parallelism is 100X. This number drops dramatically given even the smallest reduction in non-sequential code; the maximum speedup for code that is 90-percent parallel (meaning 10 percent of it must be executed sequentially) is 10X. Given even a number like 30-percent sequential code (which is still pretty good), then regardless of the number of processors thrown at the problem, the maximum speedup will be somewhere in the realm of 5X.

The implication of Amdahl's Law is that the diminishing returns of adding more processors accelerate as the percentage of sequential code grows (see Figure 1).

Figure 1. Speedup per Processor Is Limited by the Percent of Serialized Code

The top line shows that when there is only 0.1-percent serialization, speedup per processor is relatively consistent for large numbers of processors. However, when you look at the bottom line with 30-percent serialization, diminishing returns are quite obvious.

The thinking behind this myth is that the app server (or OS or some other piece of middleware) can automatically do some kind of analysis to discover the sequential operations, rearrange them in some fashion, and thus optimize the sequential operations in order to minimize the cost. But this implies that the application server could rewrite your application to reorder its execution to suit its own needs. Obviously this could have disastrous effects on the consistency and correctness of your code—something no application server vendor is going to risk. The app server must execute your code exactly as it sees it. And while it might be able to spin off certain threads in certain places to improve the parallelism, it will always be hamstrung by the basic requirement that it not break the order of execution of your code.

The "Concurrent Computing Has To Be Hard" Myth
Almost nothing scares a developer like the idea of explicitly dealing with the concurrency problems of multiple threads (or multiple machines), because concepts such as "deadlock," "livelock," "starvation," or "deadly embrace" are outside the core expertise of most application developers. And even if a skilled engineer knows how to handle these situations, it's not the ideal use of his or her time.

This has sparked resurgent interest in new ways to deal with concurrency, ranging from new language-based approaches (e.g., OpenMP or JR) to new approaches to languages altogether (e.g., functional languages like F# or Scala). While creating an entirely new language offers some unique opportunities to deal with concurrency in elegant ways, the thought of learning an entirely new programming language—much less a new platform—can be overwhelming to a developer.

Fortunately, building applications for concurrency doesn't have to be a task only rocket scientists, brain surgeons, and astrophysicists can perform. By understanding even some of the basic principles of how parallel processing best works (immutable objects rather than objects with mutable state and guarded code regions, making copies of shared state, and so on), developers can build applications to take advantage of multiple processors with relatively little work.

Of course, if one has a library or tool available to help ensure and enforce some of these design decisions, all the better. With the appropriate tools, a developer can not only mitigate the risks associated with rewriting multi-threaded code, but preserve the flexibility and agility of the application environment as well.

The Not-So-Mythical Java App on a Multi-Core System
Now that you can discern between myth and fact, taking advantage of the parallelism implicit in today's and tomorrow's multi-core CPUs doesn't have to mean subtle and irreproducible threading bugs, which consume thousands of man-hours only to disappear at debug time. Taking some straightforward steps to design more parallelism-friendly applications and using tools that help guide and enforce those design principles should make building a parallelism-friendly application as simple as building, say, an application that stores data into a relational database.

Oh, and if you're considering busting that Mentos-and-Diet-Coke-produce-spectacular-gushing-fountains myth, don't try it at home. Trust us; that one's a scientific fact. Hopefully this warning saves you from having to explain to your significant other why there's Diet Coke on the floor... and the walls... and the ceiling. Now, if you'll pardon us, we're curious to know if the metal-in-the-microwave-will-blow-it-up myth is true, and there happens to be an old microwave oven in the garage...