RSS Feed
Download our iPhone app
Browse DevX
Sign up for e-mail newsletters from DevX


Using Python for Big Data Analysis

AppNexus says its reliance on Python helps it quickly move code from development to production.


At Friday's PyData conference in New York, engineers from AppNexus explained how their company is using Python for big data analysis. Python, which is known as one of the easier programming languages to learn, is an unusual choice for working with big data. But David Himrod from AppNexus said the language's simplicity has allowed its employees to work together more easily. "What's nice is we don't have to hire for a specific programming background. Python is easy to teach," Himrod said. "Python is a really clean, easy language to learn."

AppNexus processes about 15TB of data every day. That information is stored and processed through tools like MySQL, IBM's Netezza, Hewlett-Packard's Vertica, Apache Hadoop, and HBase. The company uses Python-based tools to extract and analyze that data. "We've been able to build a framework that makes it easy for us to grab data from all of these disparate data sources and model them. So instead of everyone spending their time writing database connector code, they are able to use a simple configuration and quickly get off the ground," Himrod said.

View article

Email AuthorEmail Author
Close Icon
Thanks for your registration, follow us on our social networks to keep up-to-date