devxlogo

Using Python for Big Data Analysis

Using Python for Big Data Analysis

At Friday’s PyData conference in New York, engineers from AppNexus explained how their company is using Python for big data analysis. Python, which is known as one of the easier programming languages to learn, is an unusual choice for working with big data. But David Himrod from AppNexus said the language’s simplicity has allowed its employees to work together more easily. “What’s nice is we don’t have to hire for a specific programming background. Python is easy to teach,” Himrod said. “Python is a really clean, easy language to learn.”

AppNexus processes about 15TB of data every day. That information is stored and processed through tools like MySQL, IBM’s Netezza, Hewlett-Packard’s Vertica, Apache Hadoop, and HBase. The company uses Python-based tools to extract and analyze that data. “We’ve been able to build a framework that makes it easy for us to grab data from all of these disparate data sources and model them. So instead of everyone spending their time writing database connector code, they are able to use a simple configuration and quickly get off the ground,” Himrod said.

View article

devxblackblue

About Our Editorial Process

At DevX, we’re dedicated to tech entrepreneurship. Our team closely follows industry shifts, new products, AI breakthroughs, technology trends, and funding announcements. Articles undergo thorough editing to ensure accuracy and clarity, reflecting DevX’s style and supporting entrepreneurs in the tech sphere.

See our full editorial policy.

About Our Journalist