Making good on a promise it made in August, Twitter has released the source code for Storm, a tool it acquired when it purchased social media analytics firm BackType. Described as the “Hadoop of real-time processing,” Storm offers highly scalable, reliable computational power for real-time data.
Twitter’s Nathan Marz explains that one of the ways Twitter uses Storm is to calculate the number of unique users who read a particular Tweet. “It can take minutes or worse to compute on a single machine,” says Marz. “With Storm, you can do every step of the reach computation in parallel and compute reach for any URL in seconds (and less than a second for most URLs).”
