Hadoop on the Rise as Enterprise Developers Tackle Big Data : Page 2

Hadoop and Managing Big Data in the Enterprise

Managing big data is one of the key challenges of the new decade, said David Menninger, vice-president and research director, Ventana Research.

"The solutions to this challenge vary, but interest in them seems to be universal," he said. "The largest database vendors and others that wish to compete with them are developing or acquiring various technologies, among them database appliances, massively parallel databases, and columnar databases."

Menninger noted that the rise of Hadoop has been dramatic. "It has been successfully deployed at some of the largest Internet-based organizations in the world, including eBay, Facebook, Google and Yahoo," he said. "Seeing this, other organizations whose business depends on managing large amounts of data have begun to explore Hadoop as well."

In the summer of 2011, Ventana Research released a report on Hadoop that found many organizations are using the platform to perform data mining and in-depth analytics. The report, based on a survey on enterprise adoption of Hadoop, was sponsored by Cloudera, Karmasphere and Pervasive Software.

Among many interesting trends and statistics, Ventana discovered:

• More than one-half (54%) of organizations surveyed are using or considering Hadoop for large-scale data-processing needs.
• More than twice as many Hadoop users report being able to create new products and services and enjoy costs savings beyond those using other platforms; over 82% benefit from faster analyses and better utilization of computing resources.
• 87% of Hadoop users are performing or planning new types of analyses with large-scale data.
• 94% of Hadoop users perform analytics on large volumes of data not possible before; 88% analyze data in greater detail; while 82% can now retain more of their data.
• Organizations use Hadoop in particular to work with unstructured data such as logs and event data (63%).
• More than two-thirds of Hadoop users perform advanced analysis -- data mining or algorithm development and testing.