Programmer and self-described internet entrepreneur Evan Prodromou has been involved in starting open source and open content projects. His best-known is Wikitravel, a Wiki site for collaboratively edited travel guides.

Indenti.ca, his current project, is his attempt to develop a free network service using shared, open data. But to the uninitiated, the site and service look—and function very much—like a clone of Twitter. The big difference is that the software running Indenti.ca, which Prodromou has also been developing, Laconica, is free and open source. People can copy the code and use it on their own servers.

Originally from San Francisco, Prodromou is now based in Montreal where he has established a company, Control Yourself, to turn Indenti.ca into a business.

Howard Wen: What term would you use to describe this communication architecture—the Twitter or Facebook-style “user status update” system?

Evan Prodromou: If it [were] up to me, I would probably choose something like “short message hub” or “universal messaging hub.” But the name that’s really sticking with this kind of service is “microblogging.” I think that comes out of the fact that a lot of [these] services have a web interface and it looks like a blog. I don’t think it’s a fully accurate term, but it’s good enough, and it gets the idea across.

HW: Can you sum up how the architecture for microblogging basically works in a service like Twitter or Indenti.ca?

EP: Typically, you have a web service that serves as a hub for messages, and messages can go into this hub through many different media. They can be posted from a web page, but they could also be sent from an IM client, SMS or e-mail. They can be sent in through a web API, too. Twitter, for example, [has] a number of desktop clients that support their API.

Then each user in these networks can subscribe to messages from friends, from people they’re interested in. So the messaging hub does this switching process of saying, “Of all these messages coming in, where do I send them out to?” The messaging hub sends them out to subscribers. These messages go out over RSS, web pages, IM, SMS or to API clients.

So, it’s a “multiple channels in; multiple channels out” messaging system with a social networking aspect.

HW: To clarify, what’s the difference between the archicture of a microblogging system and that of an instant messenging service?

EP: IM tends to not have a lot of persistence. You don’t expect to be able to find a particular message or something that someone said in IM kept around on the web forever, and that’s something that does happen with microblogging. We expect things to be persistent.

There’s also an expectation of real-time conversation with IM. Whereas in microblogging, the conversation can happen over a period of days. So it’s a more extended conversation.

Finally, most IM conversations are one-to-one. With microblogging, even a relatively antisocial person will usually end up with 50, 60, 100 people listening to them. It’s common to get up into the triple digits fairly quickly, even for someone who’s not looking to add a lot of friends.

One funny thing is that in IM if I’m in a conversation with you and another person, we can all three “hear” each other. But in microblogging, I might send out a notice, and you and the other person can both hear what I’m saying, but can’t hear what each other are saying, because you’re not subscribed to each other’s messages. So there’s a disconnect. The way notices propagate is a little bit more fractured across time and across a social network.

HW: Why did you create an open source version of the Twitter system?

EP: As someone who’s very active in open source software and open source web software, I’m very interested in how much of our online life we are putting into the hands of services that are very proprietary. Some people call it the “roach motel model”—you put your data in and it won’t come out. I put tons of information about myself and my life into services like Facebook. Twitter was another one that I was participating in quite a bit. But if anything happened with Twitter, if Twitter had reliability problems, or if Twitter wasn’t on the web for a while, I couldn’t take my data, put it into my instance of that software, and run it somewhere else. I find that frustrating. I think a lot of other people find that frustrating, too: giving up control of your social life and your sociality online to someone who doesn’t necessarily have your best interests at heart.

It’s not that the people who start Facebook or MySpace or Twitter are bad or anything. But losing one user’s data, or dropping one user’s set of messages is not a big deal to them, whereas it’s a really big deal to me. So I wanted to experiment with providing web services where the user is in control.

I’m interested in doing this in different fields in the web services area. Microblogging just happens to be very popular. I wanted to get the idea in front of people who are interested in Twitter, and say maybe there’s another way that we can do this.

HW: What language and libraries did you use to create Laconica, and why?

EP: Laconica is written with PHP and MySQL. We have off-line processing daemons that do a lot of the same kind of work that the Twitter off-line daemons do. They do routing and they do sending stuff out over different channels. We don’t have a dedicated queuing server that’s built into the system right now, but that will be in an upcoming version.

Right now, we just use our MySQL database as kind of an ad-hoc queuing server. It’s not a very good way to do things and it kind of hurts our performance.

I think it’s a little more unusual to do long-running processing, like queue handlers in PHP, than it is to do it with other scripting languages, like Ruby or Python or Pearl. However, once we made PHP work that way, it’s been actually a pretty decent choice for implementation. And the fact that the web interface and the off-line daemons use the same language and libraries makes contributing to both a lot easier.

HW: How similar is Laconica’s technology to Twitter’s?

EP: Let me say that I only know what is public information about Twitter. I don’t know anything about their internal source code or anything like that.

Superficially, I’ve tried to track very closely the same kind of functionality that Twitter has. So my intention is to have someone who’s very familiar with Twitter’s function set be able to set up their own Laconica instance and be like, “I’ve been using Twitter for a while. I know how to add a friend. I know how to send direct messages. I know that I can get RSS feeds, etc..”

Also, early on, we cloned the Twitter API function-for-function. Twitter has a published API that they use, and there have been a number of desktop, web, and mobile clients that use this API. Because we cloned an identical API, that means third-party developers can simply change a few constants in their code and their code will work just as well with Identi.ca as it does with Twitter. In fact, it sometimes works a little better.

HW: How different is Laconica from Twitter?

EP: Under the covers, [Laconica] looks very different:

As far as I know, and again this is mostly hearsay, Twitter is implemented as a Ruby on Rails application [on] the web side. It also uses a number of off-line queue processing demons. So if you post a message on Twitter through the web site, the message goes into a queue, and the off-line processing demons, which I think are also written in Ruby, also process it. And they say, “This message goes to this person… this message might go out over IM… this message needs to go out over the web… this one’s going to get stored here… this one gets sent there.”

I believe they use a queuing manager called Starling, which, again, is written in Ruby. That is about the extent of my knowledge of the Twitter architecture.

Laconica’s architecture is a little bit different because we have different requirements. The principal difference in our environment is that I wanted to make sure that our software can be installed practically anywhere. You can install [Laconica] on a PHP and MySQL server, and it should work fine.

However, at the higher end, we have to support a big system like Identi.ca. At that level, we need to have a very similar architecture to the one that Twitter has, so requirements are to scale. When people talk about scaling, they usually talk about getting bigger. For Laconica, we need to cover the scale from the very small implementation to very big implementations.

HW: What are the challenges when it comes to developing or setting up a microblogging system?

EP: People expect this kind of system to have a utility-like reliability. If I’m sending out a message to all my friends, I would like to have that message go through quickly, efficiently, reliably. So it raises the level of requirements for getting real-time (or close to real-time) communication, and having everything working on lots of different protocols.

Each time that your number of messages and your number of actual communication message-sending points go up by squared as you get end users, your scale goes up very quickly. Your scale goes up very quickly end squared against end. You need to cover a lot of different channels; there’s lots of different protocols that you need to be working with.

HW: As a developer, what do you find technically interesting about microblogging?

EP: One of the things that’s been interesting for me is developing software for multiple protocols and multiple ways of speaking over the internet, mobile devices and routing. The success of our projects is going to depend on routing over as many channels as we possibly can.

Another thing I’ve found very interesting is the necessity for reliability. We need to be getting fast, quick service to people on a fairly large volume of messages. I’m learning a lot about building messaging architectures, trying to keep up with how our system is growing.

Fortunately for me, the open source community and open source development community have been forthcoming. Lots of people have been participating in our project. Rasmus Lerdorf did a code review on the software and came up with a lot of suggestions and ideas.

HW: What do you foresee will be the future of microblogging? Basically, is it just a fad, or will it evolve into something truly useful?

EP: I don’t know of many communication systems that have grown to this kind of popularity without eventually becoming fairly important. These kinds of markets grow for a reason and they meet a need that people have. There’s somewhere around 110 microblogging services out there, not counting about 100 Laconica instances. So I don’t think those services are serving a need that’s illusory. I think there’s something real there.

The fact that it turns communication into a worldwide conversation very quickly is really powerful and really forceful. You see in the conversations around the U.S. elections, the way that people respond to emergencies like the earthquake in California a few weeks ago, that there’s something here that people are looking for.

Where will it go in the future? I think we’ll probably see even more sharing of structured data, things like images, video, [and] audio. It’ll become a bit more like micro-podcasting or micro-videocasting. You might take a quick video, send it out to all your friends, and it gets to them wherever they are.

There’s also a good case to be made for using microblogging in [the] enterprise. When people in different departments in a company need to know what each is doing, it’s a great way to keep people up-to-date. I think corporate microblogging is going to become a big part of what people do in their company.

So I don’t think it’s just a fad. It’s going to be ubiquitous, and, if we’re careful and smart, we can make it into a protocol that everyone can use, that lots of different vendors can support and implement, and that we can build on top of, instead of having competing services.