Is C++ usage really declining, as some biased analysts and journalists have been trying to convince us for years (often not without ulterior motives), or is this complete nonsense?
It's probably not complete nonsense. C++ use appears to be declining in some areas and appears to be on an upswing in other areas. If I had to guess, I'd suspect a net decrease sometime during 2002-2004 and a net increase in 2005-2007, but I doubt anyone really knows. Most of the popular measures basically measures noise and ought to report their findings in decibel rather than "popularity." Many of the major uses are in infrastructure (telecommunications, banking, embedded systems, etc.) where programmers don't go to conferences or describe their code in public. Many of the most interesting and important C++ applications are not noticed, they are not for sale to the public as programming products, and their implementation language is never mentioned. Examples are Google and "800" phone numbers. Had I thought of a "C++ inside" logo in 1985, the programming world might have been different today.
Among the positive signs for C++, I can mention an increase in my email and the number of speaking invitations. At the SDWest conference in March, the C++ track was by far the largest, even without counting the "Super tutorial" that I gave with Herb Sutter; more than 300 people attended that one—up by a factor of two compared to two years ago. These are, of course, personal and subjective measures, but they fit with other information that I get from innumerable sources in the industry (worldwide).
|"We don't know what the challenges will be and that's C++'s greatest strength."
It's a really big world "out there" and the increase in the number of users of one language does not imply the decrease in the numbers of another. I suspect we have reached the point where if you can count your users to the nearest million, you don't count as a major language. Similarly, I got into a discussion with some friends about how many billions of lines of C++ code there were "out there." We concluded that we did not know, but it didn't require much thought to demonstrate that the plural was appropriate.
One simple thing that confuses many discussions of language use/popularity is the distinction between relative and absolute measures. For example, I say that C++ use is growing when I see user population grow by 200,000 programmers from 3.1M to 3.3M. However, somebody else may claim that "C++ is dying" because it's "popularity" has dropped from 16 percent to 10 percent of the total number of users. Both claims could be simultaneously true as the number of programmers continues to grow and especially as what is considered to be programming continues to change. I think that C++ is more than holding its own in its traditional core domains, such as infrastructure, systems programming, embedded systems, and applications with serious time and/or space constraints. It never was dominant for scripting web applications.
Most of the popularity measures seem to measure buzz/noise, which is basically counting mentions on the web. That's potentially very misleading. Ten people learning a scripting language will make much more noise than a thousand full time programmers using C++, especially if the thousand C++ programmers are working on a project crucial to a company—such programmers typically don't post and are often not allowed to. My worry is that such measures may actually measure the number of novices and thus be an indication of a worsening shortage of C++ programmers. Worse, managers and academics may incautiously take such figures seriously (as a measure of quality) and become part of a vicious circle.
What are the biggest challenges that the software industry will face five, 10, or maybe 20 years from now? What makes C++ the right choice for software developers who are facing these challenges?
We don't know what the challenges will be and that's C++'s greatest strength. C++ is more general, flexible, and efficient (in run time and space) than alternatives and will therefore be a leading candidate to address these unknown challenges.
That relates to the technical challenges. The major problems for C++ are social. The C++ community is far less well organized than other communities; we don't have an owner, a dictator, a community web site, a journal, or a conference that defines the language. The closest we get to a focal point in the C++ community is the ISO standards committee and it's in itself a diverse body. However, as a focus for the C++ community, the committee is not ideal. For example, language and tools implementers are heavily overrepresented compared to the user community. Similarly, I don't think that the educational establishment really has a grip on C++. Too many don't understand it, partly because they don't understand the needs of the industries that are the primary users of C++. We need better, tutorials, courses, textbooks, web sites, etc. We also need better ways of evaluating and distributing libraries and tools. There are so many libraries out there of such varying quality that many users give up and say "C++ doesn't have X" even though there is a choice of a dozen reasonable Xs available.
It is often said that the past is not a good predictor of the future, but it's the only predictor we have, so I can recommend my HOPL paper "Evolving a language for and in the real world: C++ 1991-2006." It discusses the background for the current use of C++, the work on C++0x, and tries to describe C++ relative to the problems it faces.