Login | Register   
LinkedIn
Google+
Twitter
RSS Feed
Download our iPhone app
TODAY'S HEADLINES  |   ARTICLE ARCHIVE  |   FORUMS  |   TIP BANK
Browse DevX
Sign up for e-mail newsletters from DevX


advertisement
 

Alluxio Powers Baidu Search

The in-memory big data tool boasts much faster performance than Spark SQL alone.


advertisement

Alluxio, an open source memory-centric distributed storage system that was formerly known as Tachyon, has just released its 1.0 version, and it is already getting attention from some of the biggest firms on the Internet. For example, Chinese search giant Baidu says it is using Alluxio to achieve blazing-fast performance.

Baidu Senior Architect Shaoshan Liu explained that the company had been using SparkSQL for queries but wasn't achieving the desired performance levels. "With Spark SQL alone, it took 100-150 seconds to finish a query; using Alluxio, where data may hit local or remote Alluxio nodes, it took 10-15 seconds. And if all of the data was stored in Alluxio local nodes, it took about five seconds, flat — a 30-fold increase in speed," he explained. "Based on these results, and the system's reliability, we built a full system around Alluxio and Spark SQL."

Other companies working with Alluxio include Barclays, Alibaba, Intel and IBM.



View article



   
Comment and Contribute

 

 

 

 

 


(Maximum characters: 1200). You have 1200 characters left.

 

 

Sitemap
Thanks for your registration, follow us on our social networks to keep up-to-date