RSS Feed
Download our iPhone app
Browse DevX
Sign up for e-mail newsletters from DevX


Alluxio Powers Baidu Search

The in-memory big data tool boasts much faster performance than Spark SQL alone.


Alluxio, an open source memory-centric distributed storage system that was formerly known as Tachyon, has just released its 1.0 version, and it is already getting attention from some of the biggest firms on the Internet. For example, Chinese search giant Baidu says it is using Alluxio to achieve blazing-fast performance.

Baidu Senior Architect Shaoshan Liu explained that the company had been using SparkSQL for queries but wasn't achieving the desired performance levels. "With Spark SQL alone, it took 100-150 seconds to finish a query; using Alluxio, where data may hit local or remote Alluxio nodes, it took 10-15 seconds. And if all of the data was stored in Alluxio local nodes, it took about five seconds, flat — a 30-fold increase in speed," he explained. "Based on these results, and the system's reliability, we built a full system around Alluxio and Spark SQL."

Other companies working with Alluxio include Barclays, Alibaba, Intel and IBM.

View article

Email AuthorEmail Author
Close Icon
Thanks for your registration, follow us on our social networks to keep up-to-date