Right. So in other words, let's say if there's a hundred pieces in this file, and they are really like 512K chunks, right? So let's say if there's a hundred pieces and there's five other people trying to download at the same time. I download piece 1, 49 and 74 and somebody else downloads 5, 56 and 82 then the tracker knows that we have those pieces?
No. There's a common misconception about BitTorrent [around] how much information the tracker keeps track of. So the most important thing to remember about the tracker's role to play in this algorithm is that they only keep the IP address and that's it.
Carl Franklin: So you connect simultaneously to all the different peers on different threads and you say "What pieces do you [have]?" and "Here is what I have." and you do an exchange. Is that how it works?
David Smith: That's exactly it. The first message you send to your peer when you connect to them is, first a hello message, and then a piece message. And the piece message says, "These are the pieces that I have." So once you both receive those messages from each other, if you're not interested in what that peer has to offer you, you can disconnect the connection as soon as you know that. But if you are interested then you can make a request for the piece and start sending away.
Carl Franklin: And when you get a new piece from another peer do you notify all the other peers that you have that piece or do they have to poll for you?
David Smith: There's a message that gets sent out, it's called a piece message.
Carl Franklin: And it's not something they have to poll for? You stay connected on that port in other words?
David Smith: Yup.
Richard Campbell: Now how [does] this whole process get started? I mean, it presumes that the peers have pieces, there's got to be a starting point.
David Smith: Exactly, so the very first person that will ever start a torrent is called a seeder. This seeder will start the torrent. He will stay connected until at least one other person has the file or at least, until the file has been distributed once. So if three people connect, as soon as I start seeding, then I have to distribute that file to at least across those three peers before I log out, before I shut down my computer. And then those three peers can end up reconstructing the file between them.
Carl Franklin: And is it worth it having more seeders than one, or is one sufficient?
David Smith: It's a very common problem that having one person on a 56 K connection is not going to be able to start up a torrent very quickly. The ramp up time is very efficient but it's still going to be at least a while.
Richard Campbell: And the algorithm's smart enough to make sure that it's shipping each piece of the file once before it starts reshipping to other folks?
David Smith: So that actually depends on the client.
Richard Campbell: Oh yeah? Some clients are that bright?
David Smith: Right. So in my client yes, it is that smart. In Bram's client I would expect it to be that smart.
Carl Franklin: But you don't know?
David Smith: I don't know.
Richard Campbell: But either way someone behind a slow connection like that, still has to hold the whole file up at least once, even though it maybe has multiple sources, it's still got to get up there.
David Smith: Yes
Carl Franklin: But you can see what's happening here, is that as more clients get connected and they start downloading all these pieces from these other clients, obviously there is some efficiency stuff in there to break up the downloading of all these pieces over all these different clients. If one client has all the pieces and there are other clients out there that also have the same pieces, isn't it going to download simultaneously from all the clients rather than just wait for one to send everything?
David Smith: A better way of looking at it is: I am looking for pieces 1 to 100 and there are 100 peers out there. If I wanted to download all those 100 pieces from one person, it would take a really long time. If I want to download one piece from each of the 100 peers out there, that will take a much shorter time because everybody else's upload rate will be maybe 1 kilobyte a second rather than just one person upload rate being 200 kilobytes per second because that will be capped.
Carl Franklin: So the net effect is, if we could boil it down to this, the more people are downloading simultaneously the faster everybody gets the file.
David Smith: Absolutely.
Richard Campbell: Which is a total reverse of what we are used to. I mean, the normal model here is, I put a new file out on my server and everybody goes through it at once and I drop to my knees.
Carl Franklin: Yeah we have known about that for a long time.
Richard Campbell: Sure. But now we are going the other way. I put that file out there and the more people get involved the better it goes!
Carl Franklin: Yeah. And if that doesn't excite you, people, then
you are not excitable. (laughs) That's a fundamental shift in the way the bandwidth is used. More is better. I mean this is a way to go on demand, I mean this is the keyif you can combine the on-demandness of a download with the power of this multiple person downloading, then you have got something. Like, for example, broadcasts that are going to happen at a certain timeyou know what I mean? As soon as that file is ready, all the clients get notified and are interested in it and then they are all downloading at the same time. And it takes very little time for everybody to get it. So this is why that we think, and I am sure you do too David, that podcasting and BitTorrent go hand in hand. And the software isn't quite there yet, it's still a little geeky and tweaky, but it's getting there. Because we have got thousands of subscribers to our podcast when they all of a sudden see that there is a new file out there, they're all going to get notified within the same 20 minutes to half hour, and then they're all going to try to download it. Well, if they're downloading through BitTorrent that's great because we want them all to download at the same time. So, we can increase the speed at which they all get it.
David Smith: Yeah absolutely. Bram Cohen actually said on his Web site when he first made BitTorrent. He said, "Why did we make BitTorrent?" And his reason was, because he wanted to enable freedom of expression and I couldn't say it better myself; like, when I read that I was like, yeah that's exactly perfect. Because what BitTorrent does, is it allows more people to publish files to as many people as they would like to. However many people want to download the files, they can download it without paying for server costs, without having to know anything about the Internet or how to setup the server, and its just perfect for freedom and enabling podcasting.
Carl Franklin: Yeah it's freedom of information.
Richard Campbell: A key technological issue here is this recognition that in broadband at least, your download speed and your upload speed are totally independent of each other. So while you're downloading this file you are able to upload it without impacting your download.
This conversation continues online at http://shrinkster.com/ar8.