At a developer conference, Facebook unveiled its homegrown query engine called Presto. According to Facebook’s Martin Traverso, more than 850 Facebook employees use the tool every day to analyze 320 TB of data. It runs in memory and can perform simple queries in milliseconds, complex ones in just a few minutes.
“Historically, our data scientists and analysts have relied on Hive for data analysis,” explained Traverso. “The problem with Hive is it’s designed for batch processing. We have other tools that are faster than Hive, but they’re either too limited in functionality or too simple to operate against our huge data warehouse. Over the past few months, we’ve been working on Presto to basically fill this gap.”
Facebook plans to release Presto as an open source project later this year.
