Hadoop: A Solution in Search of a Problem
While enterprises and vendors alike seem to be sure that Hadoop is the answer, analysts wonder if they know what the question is. Woo likens it to the game show Jeopardy. "Often we find solutions and then we have to go back and find problems that they solve," he said.
In a webinar, Woo and a colleague noted, "Many initial Hadoop projects will fail to gain broad adoption. A key challenge for many is to find initial use cases that would deliver measurable value for the enterprise." They added, "Some of the development side are committed to the solution before they have found a suitable problem."
Connolly acknowledged the same problem and said that he expects vendors to begin developing more use cases that they can show to potential customers. "That's important because right now people are really trying to wrap their heads around exactly what kind of value can they get out of Hadoop," he said. "They understand it has potential, but they want to know where they can start."
Beware Hadoop Project Failure
Analysts say that enterprises should investigate Hadoop, but to expect some bumps on the road. In a study of early Hadoop adopters, Forrester Research concluded, "Although these early adopters have realized significant benefits, they acknowledge that Hadoop is an immature technology with many moving parts that are neither robust nor well integrated. Deploying, ramping up, and optimizing Hadoop clusters takes more time and custom coding than business process and application development and delivery (AD&D) professionals might expect."
IDC pointed out that most enterprises don't have highly skilled data scientists on staff who can help them design projects that will generate real business value. Because of this staff shortage, the immaturity of the tools, and the lack of well-developed business cases, Woo said that "Many of these [Hadoop] projects will fail, unfortunately."
However, that doesn't mean that developers shouldn't investigate the technology. Forrester recommended, "Application development and delivery (AD&D) professionals should consider Hadoop an immature but promising technology for addressing the most data-intensive analytics and application requirements."