he Getting Started with the Twitter API article described how Twitter’s founders see it as more than simply a web site where people can send short messages to those who choose to follow their message stream. Twitter is a new kind of publishing platform, and the simple, straightforward API for this platform is easy to use with a variety of tools and programming languages.

Tweeting Big Jumps in the Dow Jones Industrial Average

On a normal day last September, I would never check the Dow Jones Industrial Average. With the financial turmoil of recent months, though, I wondered more often whether the economy was getting us in deeper or if things had bounced back, so I checked too often, usually at CNBC’s home page. When I did this, I often wondered if there was a simple way to receive alerts when the Dow made newsworthy jumps.

Checking stock and financial index quotes is pretty simple with any scripting language, so I used the python-twitter API to write a script that checks the Dow and posts a Twitter message (or “tweet”) if the figure moves at least 50 points since the last check. The script is about 60 lines of code, which, in addition to basic file I/O, determines whether the DOW went up or down since the last check, and assembles a sentence describing what happened.

The script is not hard coded to only check the Dow Jones Industrial Average and tweet after a 50 point swing, which is why I called it tweetstockjump.cgi. By resetting variable values near the top of the script, you can change both the ticker symbol for the stock or index you want to check and the increment that triggers a tweet.

The web offers a wide choice of free stock quote services. Instead of going with a SOAP or other XML-based web service, I went with a simpler option: the one-line comma-separated lists of stock information available from Yahoo! Finance. For example, on the evening of November 19th, adding the ticker symbol IBM at the end of the URL http://download.finance.yahoo.com/d/quotes.csv?f=sl1d1t1c1ohgv&e=.csv&s= and pasting it into my browser (or using the wget or cURL utilities mentioned in Part 1 of this article) retrieved the following line of text:

"IBM",75.97,"11/19/2008","4:01pm",-4.11,79.93,81.00,75.73,12606807

The commas separate these nine pieces of information:

• Ticker symbol
• Last price (after a 20-minute delay)
• Date of that price
• Time of that price
• Change since the day’s opening
• Opening price
• Day’s high price
• Day’s low price
• Trade volume

Python, like Perl, has a split() function that separates a comma-delimited list of values into the individual slots of an array with one short line of code.

With all these tools in place, the tweetstockjjmp.cgi script checks the Dow Jones Industrial Average figure, evaluates the difference since the last check, and tweets if this difference is greater than 50 by performing the following steps:

1. Retrieve the latest information from Yahoo! Finance, split it up into an array, and assign the needed figures to variable names.
2. Grab the same information about the most recent polling of the same data from a one-line text file that was saved at the time. If there is no such file, calculate the figure’s jump from the day’s opening figure, which is the sixth field in the comma-separated list returned from Yahoo! Finance.
3. Compare the new figure with the old figure, and if the difference exceeds the value stored in the increment variable,

a. Construct a sentence in the variable tweetMsg following this model: “Dow Jones Industrial Average down 130.82 from 8128.1 at 3:51pm to 7997.28 at 4:06pm NY time, down 427.47 since today’s opening.” As with the increment that triggers a tweet and the symbol of the stock or index figure being checked, the full name of the stock or index is stored in a variable set near the top of the script so that it can be easily modified to check for other financial information.
b. Send the message (or, as Twitter calls it, an “update”) to the Twitter server.
c. Save the newly retrieved CSV file so that the next polling of the latest quote has something to compare it with.
d. Otherwise, if the figure moved less than increment points, the script is finished.

My first draft of the script was called checkDowJones.cgi (cgi because it’s a script being run under my account on my host provider’s system), but after storing the quote symbol, the referenced index’s full name, and the increment to check for in variables to make the script more flexible, I changed the name to tweetstockjump.cgi to reflect its potentially wider usefulness.After storing and testing the script on my host provider’s system, I set up a cron job to run the script at 52 minutes after the hour from 9 AM to 4 PM Eastern time. The market is open from 9:30 to 4:30, so with the 20-minute delay, the script checks every hour from the posting of the opening price to the posting of the closing price. Another cron job executed at 9:30 deletes any existing files that store the result of earlier tweetstockjump.cgi executions, so that the day’s first call to the script compares the latest price to the day’s opening price.

The complete script is shown in Listing 1.

The script starts with the import statements that identify any necessary specialized libraries:

• urllib to read the remote Yahoo! Finance data
• os.path to check for the existence of the data file with older data
• string to convert the retrieved comma-separated number strings to floating point numbers
• python-twitter to assemble and send a message to Twitter’s server

When I logged in to my host provider’s server and tried this interactively, it all worked fine, but the first time I had a cron job try to execute it, it didn’t know about the python-twitter library or the simplejson library that it depends on. Why? Because when I was logged in, it knew that these were in my PYTHONPATH, but the process started by the cron job didn’t know about my PYTHONPATH. That’s why my own copy of tweetstockjump.cgi has sys.path.append statements to point to the python, simplejson, and python-twitter libraries before the import twitter line: to ensure that the cron job process can find those. I omitted them above because your need for them depends on your own environment.

Along with the assignment of variables that let any Twitter ID check jumps of any increment in any stock or index, the beginning of the script also creates a dataFileStorage variable to identify the directory holding the files that store the results of recent checks of stock or index figures, so don’t forget to create this directory before running the script. The files themselves are named after the ticker symbol with a .csv extension, so that in the script as shown, the Dow Jones information is stored in a file named INDU.csv in a tweetstockjump subdirectory.

After setting all these variables, standard Python code reads the CSV file identified by the quoteService variable and the stock symbol into a newData string, which is then split into the newDataFields array and copied to variables that identify the information: newQuote, newTime, openingPrice, and dayDiff for the difference in the stock price since the day’s opening.

If no file exists with old quote figures for this stock, the openingPrice value is used for oldQuote and a time of 9:30AM is assigned as the oldTime. If the file does exist, values from there are read into an oldData variable that gets split into an oldDataFields array, and then assigned to variables for comparison with the new data. If the stock price has moved more than increment since oldTime, the next few lines assemble phrases into a complete sentence to post to Twitter’s server.

The commented-out print tweetMsg line is handy when debugging to make sure that all the preceding code does its job properly before you start sending messages off to the Twitter server. The real point of the script, though, is to tweet that message, which is what the next two lines do. Passing a Twitter username and password (the script above uses the fake username and password twitguy and tgpw; replace them with real values before trying the script) to the twitter.Api call creates an instance of the Api class. Among the variety of methods you can call for instances of that class, PostUpdate posts the string passed as a parameter just as if you had entered that string into the “What are you doing?” box on that Twitter account’s homepage.

This script essentially treats Twitter as a device to write to, but there’s plenty of data that you can read in with the API as well. The PostUpdate method returns a status message showing how well the method call went, and other Twitter API methods can return much more information, such as the results of a query or recent tweets by accounts that you follow. Put all this together and you have a new output medium, with more data to read every minute and millions of connected users. It all adds up to a powerful platform, just as its inventors intended.

As a bonus that its inventors didn’t intend, Twitter’s open-source competitor, identi.ca, offers a Twitter-compatible API so that a global replace of twitter.com with identi.ca/api in the URLs used for method calls (and appropriate changes to the username and password used for authentication) invoke those methods on identi.ca’s server. Identi.ca is based on the open source Laconica project, and you can install and run your own Twitter-like Laconica server, perhaps behind a firewall for use within your company. If so, slight changes to the same API URLs allow you to call these methods on your Laconica server, so the people and processes of your company can use it to communicate about the work that they’re doing.

This opens up even more application development possibilities. While some people use Twitter to announce to the world whether they’re having a second cup of coffee that morning, its API lets you create new kinds of communication between the people of a large or small community and the applications that they may wish to use—all with a few new calls in your favorite programming language or with a few well-chosen URLs.