RSS Feed
Download our iPhone app
Browse DevX
Sign up for e-mail newsletters from DevX


Learning from Yahoo's Hadoop Mistakes

Working with Hadoop is hard, but a new company called Continuity wants to make it easier.


At a conference in New York City, Todd Papaioannou, former chief cloud architect for Yahoo, talked about his experiences setting up 45,000 Hadoop servers in Yahoo’s 400,000 node private cloud. His group ended up creating “a bunch of redneck architecture” cobbled together from bits and pieces of various libraries, and the result wasn't very stable, explained Papaioannou.

“Hadoop is hard – let’s make no bones about it,” he said. “It’s damn hard to use. It’s low-level infrastructure software, and most people out there are not used to using low-level infrastructure software.”

To make Hadoop easier to use, Papaioannou has founded a new company called Continuity, which creates an abstraction layer on top of Hadoop. He says Continuity's AppFabric will help other companies to avoid the application architecture difficulties they experienced at Yahoo.

View article

Email AuthorEmail Author
Close Icon
Thanks for your registration, follow us on our social networks to keep up-to-date