Context Navigation

← Previous Changeset
Next Changeset →

Changeset 4460

Timestamp:

04/10/07 17:52:31 (3 years ago)

Author:

mike

Message:

bringing dotorg up to date with the live site

Location:

dotorg/trunk/html

Files:

: 3 added
: 5 modified

Draft_DHT_protocol.html (modified) (1 diff)
css (added)
css/screen.css (added)
developer.html (modified) (1 diff)
fast_extensions.html (added)
index.html (modified) (1 diff)
introduction.html (modified) (1 diff)
protocol.html (modified) (1 diff)

Legend:

: Unmodified
: Added
: Removed

dotorg/trunk/html/Draft_DHT_protocol.html

r4448	r4460
1		<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
2		<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" dir="ltr" lang="en"><head>
3		<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
4		<meta name="KEYWORDS" content="DHT protocol,Template:Ref,Template:Note">
5		<meta name="robots" content="index,follow">
6		<link rel="shortcut icon" href="https://dev.bittorrent.com/favicon.ico"><title>DHT protocol - BitTorrentDev</title>
7
8		<style type="text/css" media="screen,projection">/<![CDATA[/ @import "/wiki/skins/monobook/main.css"; /]]>/</style>
9		<link rel="stylesheet" type="text/css" media="print" href="_files/commonPrint.css"><!--[if lt IE 5.5000]><style type="text/css">@import "/wiki/skins/monobook/IE50Fixes.css";</style><![endif]--><!--[if IE 5.5000]><style type="text/css">@import "/wiki/skins/monobook/IE55Fixes.css";</style><![endif]--><!--[if gte IE 6]><style type="text/css">@import "/wiki/skins/monobook/IE60Fixes.css";</style><![endif]--><!--[if IE]><script type="text/javascript" src="/wiki/skins/common/IEFixes.js"></script>
10		<meta http-equiv="imagetoolbar" content="no" /><![endif]-->
11
12
13
14
15		<script type="text/javascript" src="_files/index"></script> <script type="text/javascript" src="_files/wikibits"></script>
16		<style type="text/css">/<![CDATA[/
17		@import "/wiki/index.php?title=MediaWiki:Monobook.css&action=raw&ctype=text/css&smaxage=18000";
18		@import "/wiki/index.php?title=-&action=raw&gen=css&maxage=18000";
19		/]]>/</style></head>
	1	<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
	2	"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
	3	<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
	4	<head>
	5	<meta http-equiv="Content-type" content="text/html; charset=utf-8" />
	6	<title>BitTorrent.org For Developers</title>
	7	<link rel="stylesheet" type="text/css" href="./css/screen.css" media="screen" />
	8	</head>
	9	<body id="www-bittorrent-org">
	10	<div id="upper" class="clear">
	11	<div id="wrap">
	12	<div id="header">
	13	<h1><a href="./index.html">BitTorrent<span>.org</span></a></h1>
	14	</div>
	15	<div id="nav">
	16	<ul>
	17	<li><a href="./index.html">Home</a></li>
	18	<li><a href="./introduction.html">For Users</a></li>
	19	<li><span>For Developers</span></li>
	20	<!-- <li><a href="./blog">Blog</a></li> -->
	21	<!-- <li><a href="./donate.html">Donate!</a></li> -->
	22	</ul>
	23	</div>
	24	<!-- ### Begin Content ### -->
	25	<div id="second">
	26	<h1>Fast Extension</h1>
	27	<p>The Fast Extension packages several extensions: <i>Have None/Have All</i>,
	28	<i>Reject Requests</i>, <i>Suggestions</i> and <i>Allowed Fast.</i>
	29	These are enabled by setting the third least significant bit of the
	30	last reserved byte in the BitTorrent handshake:
	31	</p>
	32	<pre> reserved[7] \|= 0x04
	33	</pre>
	34	<p>The extension is enabled only if both ends of the connection set this bit.
	35	</p><p>The following proposed messages adhere to the syntax of messages found
	36	in v1.0 of the BitTorrent protocol. All integers are four bytes
	37	big-endian. All messages start with an integer message length. All messages
	38	but the Keep-Alive follow the message length with a single byte opcode
	39	and zero or more opcode-dependant arguments.
	40	</p><p>The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL
	41	NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and
	42	"OPTIONAL" in this document are to be interpreted as described in
	43	IETF <a href='http://www.ietf.org/rfc/rfc2119.txt' class='external' title="http://www.ietf.org/rfc/rfc2119.txt">RFC 2119</a>.
	44	</p>
	45	<table id='toc' class='toc'><tr><td><div id='toctitle'><h2>Contents</h2></div>
	46	<ul>
	47	<li class='toclevel-1'><a href="#Modifications_to_Semantics_of_Existing_Messages"><span class="tocnumber">1</span> <span class="toctext">Modifications to Semantics of Existing Messages</span></a></li>
	48	<li class='toclevel-1'><a href="#Have_All.2FHave_None"><span class="tocnumber">2</span> <span class="toctext">Have All/Have None</span></a></li>
	49	<li class='toclevel-1'><a href="#Suggest_Piece"><span class="tocnumber">3</span> <span class="toctext">Suggest Piece</span></a></li>
	50	<li class='toclevel-1'><a href="#Reject_Request"><span class="tocnumber">4</span> <span class="toctext">Reject Request</span></a></li>
	51	<li class='toclevel-1'><a href="#Allowed_Fast_Set_Generation"><span class="tocnumber">5</span> <span class="toctext">Allowed Fast Set Generation</span></a></li>
	52	</ul>
	53	</td></tr></table>
	54	<p><script type="text/javascript"> if (window.showTocToggle) { var tocShowText = "show"; var tocHideText = "hide"; showTocToggle(); } </script>
	55	</p>
	56	<h2 id="Modifications_to_Semantics_of_Existing_Messages"> Modifications to Semantics of Existing Messages </h2>
	57	<p>The Fast Extension modifies the semantics of the
	58	<i>Request</i>, <i>Choke</i>, <i>Unchoke</i>, and <i>Cancel</i>
	59	messages, and adds a <i>Reject Request.</i> Now, every request
	60	is guaranteed to result in EXACTLY ONE response
	61	which is either the corresponding reject or corresponding piece
	62	message. Even when a request is cancelled, the peer receiving
	63	the cancel should respond with either the corresponding reject or
	64	the corresponding piece: requests that are being processed are
	65	allowed to complete.
	66	</p><p>Choke no longer implicitly rejects all pending requests,
	67	thus eliminating some race conditions which could cause pieces
	68	to be needlessly requested multiple times.
	69	</p><p>Additionally, if a peer receives a piece that was never requested,
	70	the peer MUST close the connection.
	71	</p>
	72	<h2 id="Have_All.2FHave_None"> Have All/Have None </h2>
	73	<pre> <i>Have All</i>: <len=0x0001><op=0x0E>
	74	</pre>
	75	<pre> <i>Have None</i>: <len=0x0001><op=0x0F>
	76	</pre>
	77	<p><i>Have All</i> and <i>Have None</i> specify that the message sender
	78	has all or none of the pieces respectively. When present, <i>Have All</i>
	79	or <i>Have None</i> replace the <i>Have Bitfield.</i> Exactly one of <i>Have All</i>,
	80	<i>Have None</i>, or <i>Have Bitfield</i> MUST appear and only immediately after
	81	the handshake. The reason for these messages is to save bandwidth. Also
	82	slightly to remove the idiosyncrasy of sending no message when a peer
	83	has no pieces.
	84	</p><p>When the fast extension is disabled, if a peer receives <i>Have All</i> or
	85	<i>Have None</i> then the peer MUST close the connection.
	86	</p>
	87	<h2 id="Suggest_Piece"> Suggest Piece </h2>
	88	<pre> <i>Suggest Piece</i>: <len=0x0005><op=0x0D><index>
	89	</pre>
	90	<p><i>Suggest Piece</i> is an advisory message meaning "you might like to
	91	download this piece." The intended usage is for 'super-seeding'
	92	without throughput reduction, to avoid redundant downloads, and so that
	93	a seed which is disk I/O bound can upload continguous or identical
	94	pieces to avoid excessive disk seeks. In all cases, the seed SHOULD
	95	operate to maintain a roughly equal number of copies of each piece in
	96	the network. A peer MAY send more than one <i>suggest piece</i> message at
	97	any given time. A peer receiving multiple <i>suggest piece</i> messages
	98	MAY interpret this as meaning that all of the suggested pieces
	99	are equally appropriate.
	100	</p><p>When the fast extension is disabled, if a peer receives a
	101	<i>Suggest Piece</i> message, the peer MUST close the connection.
	102	</p>
20	103
21		<body class="ns-0">
22		<div id="globalWrapper">
23		<div id="column-content">
24		<div id="content">
25		<a name="top" id="top"></a>
26		<h1 class="firstHeading">DHT protocol</h1>
27		<div id="bodyContent">
28		<h3 id="siteSub">From BitTorrentDev</h3>
29		<div id="contentSub"></div>
30		<!-- start content -->
31		<p>BitTorrent
32		uses a "distributed sloppy hash table" (DHT) for storing peer contact
33		information for "trackerless" torrents. In effect, each peer becomes a
34		tracker. The protocol is based on Kademila and is implemented over UDP.
35		</p><p>Please note the terminology used in this document to avoid
36		confusion. A "peer" is a client/server listening on a TCP port that
37		implements the BitTorrent protocol. A "node" is a client/server
38		listening on a UDP port implementing the distributed hash table
39		protocol. The DHT is composed of nodes and stores the location of
40		peers. BitTorrent clients include a DHT node, which is used to contact
41		other nodes in the DHT to get the location of peers to download from
42		using the BitTorrent protocol.
43		</p><p><br>
44		</p>
45		<table id="toc" class="toc"><tbody><tr><td><div id="toctitle"><h2>Contents</h2> <span class="toctoggle">[<a href="javascript:toggleToc()" class="internal" id="togglelink">hide</a>]</span></div>
46		<ul>
47		<li class="toclevel-1"><a href="#Overview"><span class="tocnumber">1</span> <span class="toctext">Overview</span></a></li>
48		<li class="toclevel-1"><a href="#Routing_Table"><span class="tocnumber">2</span> <span class="toctext">Routing Table</span></a></li>
49		<li class="toclevel-1"><a href="#BitTorrent_Protocol_Extension"><span class="tocnumber">3</span> <span class="toctext">BitTorrent Protocol Extension</span></a></li>
50		<li class="toclevel-1"><a href="#Torrent_File_Extensions"><span class="tocnumber">4</span> <span class="toctext">Torrent File Extensions</span></a></li>
51		<li class="toclevel-1"><a href="#KRPC_Protocol"><span class="tocnumber">5</span> <span class="toctext">KRPC Protocol</span></a>
52		<ul>
53		<li class="toclevel-2"><a href="#Contact_Encoding"><span class="tocnumber">5.1</span> <span class="toctext">Contact Encoding</span></a></li>
54		<li class="toclevel-2"><a href="#Queries"><span class="tocnumber">5.2</span> <span class="toctext">Queries</span></a></li>
55		<li class="toclevel-2"><a href="#Responses"><span class="tocnumber">5.3</span> <span class="toctext">Responses</span></a></li>
56		<li class="toclevel-2"><a href="#Errors"><span class="tocnumber">5.4</span> <span class="toctext">Errors</span></a>
57		<ul>
58		<li class="toclevel-3"><a href="#Example_Error_Packets"><span class="tocnumber">5.4.1</span> <span class="toctext">Example Error Packets</span></a></li>
59		</ul>
	104	<h2 id="Reject_Request"> Reject Request </h2>
	105	<pre> <i>Reject Request</i>: <len=0x000D><op=0x10><index><begin><offset>
	106	</pre>
	107	<p><i>Reject Request</i> notifies a requesting peer that its request will not be satisfied.
	108	</p><p>If the fast extension is disabled and a peer receives a reject
	109	request then the peer MUST close the connection.
	110	</p><p>When the fast extension is enabled:
	111	</p>
	112	<ul><li> If a peer receives a reject for a request that was never sent then the peer SHOULD close the connection.
	113	</li><li> If a peer sends a choke, it MUST reject all requests from the peer to whom the choke was sent except it SHOULD NOT reject requests for pieces that are in the <i>allowed fast set.</i> A peer SHOULD choke first and then reject requests so that the peer receiving the choke does not re-request the pieces.
	114	</li><li> If a peer receives a request from a peer its choking, the peer receiving the request SHOULD send a reject unless the piece is in the <i>allowed fast set.</i>
60	115	</li>
61		</ul>
62		</li>
63		<li class="toclevel-1"><a href="#DHT_Queries"><span class="tocnumber">6</span> <span class="toctext">DHT Queries</span></a>
64		<ul>
65		<li class="toclevel-2"><a href="#ping"><span class="tocnumber">6.1</span> <span class="toctext">ping</span></a>
66		<ul>
67		<li class="toclevel-3"><a href="#example_packets"><span class="tocnumber">6.1.1</span> <span class="toctext">example packets</span></a></li>
68		</ul>
69		</li>
70		<li class="toclevel-2"><a href="#find_node"><span class="tocnumber">6.2</span> <span class="toctext">find_node</span></a>
71		<ul>
72		<li class="toclevel-3"><a href="#example_packets_2"><span class="tocnumber">6.2.1</span> <span class="toctext">example packets</span></a></li>
73		</ul>
74		</li>
75		<li class="toclevel-2"><a href="#get_peers"><span class="tocnumber">6.3</span> <span class="toctext">get_peers</span></a>
76		<ul>
77		<li class="toclevel-3"><a href="#example_packets_3"><span class="tocnumber">6.3.1</span> <span class="toctext">example packets</span></a></li>
78		</ul>
79		</li>
80		<li class="toclevel-2"><a href="#announce_peer"><span class="tocnumber">6.4</span> <span class="toctext">announce_peer</span></a>
81		<ul>
82		<li class="toclevel-3"><a href="#example_packets_4"><span class="tocnumber">6.4.1</span> <span class="toctext">example packets</span></a></li>
83		</ul>
84		</li>
85		</ul>
86		</li>
87		<li class="toclevel-1"><a href="#Footnotes"><span class="tocnumber">7</span> <span class="toctext">Footnotes</span></a></li>
88		</ul>
89		</td></tr></tbody></table>
90		<p><script type="text/javascript"> if (window.showTocToggle) { var tocShowText = "show"; var tocHideText = "hide"; showTocToggle(); } </script>
91		</p>
92		</div><a name="Overview"></a><h2>Overview</h2>
93		<p>Each node has a globally unique identifier known as the "node ID."
94		Node IDs are chosen at random from the same 160-bit space as BitTorrent
95		infohashes. A "distance metric" is used to compare two node IDs or a node
96		ID and an infohash for "closeness." Nodes must maintain a routing table
97		containing the contact information for a small number of other nodes.
98		The routing table becomes more detailed as IDs get closer to the node's
99		own ID. Nodes know about many other nodes in the DHT that have IDs that
100		are "close" to their own but have only a handful of contacts with IDs
101		that are very far away from their own.
102		</p><p>In Kademlia, the distance metric is XOR and the result is
103		interpreted as an unsigned integer. distance(A,B) = \|A ⊗ B\| Smaller
104		values are closer.
105		</p><p>When a node wants to find peers for a torrent, it uses the
106		distance metric to compare the infohash of the torrent with the IDs of
107		the nodes in its own routing table. It then contacts the nodes it knows
108		about with IDs closest to the infohash and asks them for the contact
109		information of peers currently downloading the torrent. If a contacted
110		node knows about peers for the torrent, the peer contact information is
111		returned with the response. Otherwise, the contacted node must respond
112		with the contact information of the nodes in its routing table that are
113		closest to the infohash of the torrent. The original node iteratively
114		queries nodes that are closer to the target infohash until it cannot
115		find any closer nodes. After the search is exhausted, the client then
116		inserts the peer contact information for itself onto the responding
117		nodes with IDs closest to the infohash of the torrent.
118		</p><p>The return value for a query for peers includes an opaque value
119		known as the "token." For a node to announce that its controlling peer
120		is downloading a torrent, it must present the token received from the
121		same queried node in a recent query for peers. When a node attempts to
122		"announce" a torrent, the queried node checks the token against the
123		querying node's IP address. This is to prevent malicious hosts from
124		signing up other hosts for torrents. Since the token is merely returned
125		by the querying node to the same node it received the token from, the
126		implementation is not defined. Tokens must be accepted for a reasonable
127		amount of time after they have been distributed. The BitTorrent
128		implementation uses the SHA1 hash of the IP address concatenated onto a
129		secret that changes every five minutes and tokens up to ten minutes old
130		are accepted.
131		</p>
132		<a name="Routing_Table"></a><h2>Routing Table</h2>
133		<p>Every node maintains a routing table of known good nodes. The nodes
134		in the routing table are used as starting points for queries in the
135		DHT. Nodes from the routing table are returned in response to queries
136		from other nodes. </p><p>Not all nodes that we learn about are equal. Some are "good"
137		and some are not. Many nodes using the DHT are able to send queries and
138		receive responses, but are not able to respond to queries from other
139		nodes. It is important that each node's routing table must contain only
140		known good nodes. A good node is a node has responded to one of our
141		queries within the last 15 minutes. A node is also good if it has ever
142		responded to one of our queries and has sent us a query within the last
143		15 minutes. After 15 minutes of inactivity, a node becomes
144		questionable. Nodes become bad when they fail to respond to multiple
145		queries in a row. Nodes that we know are good are given priority over
146		nodes with unknown status.
147		</p><p>The routing table covers the entire node ID space from 0 to 2<sup>160</sup>.
148		The routing table is subdivided into "buckets" that each cover a
149		portion of the space. An empty table has one bucket with an ID space
150		range of min=0, max=2<sup>160</sup>. When a node with ID "N" is
151		inserted into the table, it is placed within the bucket that has min
152		<= N < max. An empty table has only one bucket so any node must
153		fit within it. Each bucket can only hold K nodes, currently eight,
154		before becoming "full." When a bucket is full of known good nodes, no
155		more nodes may be added unless our own node ID falls within the range
156		of the bucket. In that case, the bucket is replaced by two new buckets
157		each with half the range of the old bucket and the nodes from the old
158		bucket are distributed among the two new ones. For a new table with
159		only one bucket, the full bucket is always split into two new buckets
160		covering the ranges 0..2<sup>159</sup> and 2<sup>159</sup>..2<sup>160</sup>.
161		</p><p>When the bucket is full of good nodes, the new node is simply
162		discarded. If any nodes in the bucket are known to have become bad,
163		then one is replaced by the new node. If there are any questionable
164		nodes in the bucket have not been seen in the last 15 minutes, the
165		least recently seen node is pinged. If the pinged node responds then
166		the next least recently seen questionable node is pinged until one
167		fails to respond or all of the nodes in the bucket are known to be
168		good. If a node in the bucket fails to respond to a ping, it is
169		suggested to try once more before discarding the node and replacing it
170		with a new good node. In this way, the table fills with stable long
171		running nodes.
172		</p><p>Each bucket should maintain a "last changed" property to
173		indicate how "fresh" the contents are. When a node in a bucket is
174		pinged and it responds, or a node is added to a bucket, or a node in a
175		bucket is replaced with another node, the bucket's last changed
176		property should be updated. Buckets that have not been changed in 15
177		minutes should be "refreshed." This is done by picking a random ID in
178		the range of the bucket and performing a find_nodes search on it. Nodes
179		that are able to receive queries from other nodes usually do not need
180		to refresh buckets often. Nodes that are not able to receive queries
181		from other nodes usually will need to refresh all buckets periodically
182		to ensure there are good nodes in their table when the DHT is needed.
183		</p><p>Upon inserting the first node into it's routing table and when
184		starting up thereafter, the node should attempt to find the closest
185		nodes in the DHT to itself. It does this by issuing find_node messages
186		to closer and closer nodes until it cannot find any closer. The routing
187		table should be saved between invocations of the client software.
188		</p><p><br>
189		</p>
190		<a name="BitTorrent_Protocol_Extension"></a>
191		<h2>BitTorrent Protocol Extension</h2>
192		<p>The BitTorrent protocol has been extended to exchange node UDP port
193		numbers between peers that are introduced by a tracker. In this way,
194		clients can get their routing tables seeded automatically through the
195		download of regular torrents. Newly installed clients who attempt to
196		download a trackerless torrent on the first try will not have any nodes
197		in their routing table and will need the contacts included in the
198		torrent file.
199		</p><p>Peers supporting the DHT set the last bit of the 8-byte
200		reserved flags exchanged in the BitTorrent protocol handshake. Peer
201		receiving a handshake indicating the remote peer supports the DHT
202		should send a PORT message. It begins with byte 0x09 and has a two byte
203		payload containing the UDP port of the DHT node in network byte order.
204		Peers that receive this message should attempt to ping the node on the
205		received port and IP address of the remote peer. If a response to the
206		ping is recieved, the node should attempt to insert the new contact
207		information into their routing table according to the usual rules.
208		</p><p><br>
209		</p>
210		<a name="Torrent_File_Extensions"></a><h2>Torrent File Extensions</h2>
211		<p>A trackerless torrent dictionary does not have an "announce" key.
212		Instead, a trackerless torrent has a "nodes" key. This key should be
213		set to the K closest nodes in the torrent generating client's routing
214		table. Alternatively, the key could be set to a known good node such as
215		one operated by the person generating the torrent. Please do not
216		automatically add "router.bittorrent.com" to torrent files or
217		automatically add this node to clients routing tables.
218		</p>
219		<pre>nodes = [["<host>", <port>], ["<host>", <port>], ...]
220		nodes = [["127.0.0.1", 6881], ["your.router.node", 4804]]
221		</pre>
222		<p><br>
223		</p>
224		<a name="KRPC_Protocol"></a><h2>KRPC Protocol</h2>
225		<p>The KRPC protocol is a simple RPC mechanism consisting of bencoded
226		dictionaries sent over UDP. A single query packet is sent out and a
227		single packet is sent in response. There is no retry. There are three
228		message types: query, response, and error. For the DHT protocol, there
229		are four queries: ping, find_node, get_peers, and announce_peer.
230		</p><p>A KRPC message is a single dictionary with two keys common to
231		every message and additional keys depending on the type of message.
232		Every message has a key "t" with a single character string value
233		representing a transaction ID. This transaction ID is generated by the
234		querying node and is echoed in the response, so responses may be
235		correlated with multiple queries to the same node. The other key
236		contained in every KRPC message is "y" with a single character value
237		describing the type of message. The value of the "y" key is one of "q"
238		for query, "r" for response, or "e" for error.
239		</p>
240		<a name="Contact_Encoding"></a><h3>Contact Encoding</h3>
241		<p>Contact information for peers is encoded as a 6-byte string. Also
242		known as "Compact IP-address/port info" the 4-byte IP address is in
243		network byte order with the 2 byte port in network byte order
244		concatenated onto the end. </p><p>Contact information for nodes is encoded as a 26-byte string.
245		Also known as "Compact node info" the 20-byte Node ID in network byte
246		order has the compact IP-address/port info concatenated to the end.
247		</p>
248		<a name="Queries"></a><h3>Queries</h3>
249		<p>Queries, or KRPC message dictionaries with a "y" value of "q",
250		contain two additional keys; "q" and "a". Key "q" has a string value
251		containing the method name of the query. Key "a" has a dictionary value
252		containing named arguments to the query.
253		</p>
254		<a name="Responses"></a><h3>Responses</h3>
255		<p>Responses, or KRPC message dictionaries with a "y" value of "r",
256		contain one additional key "r". The value of "r" is a dictionary
257		containing named return values. Response messages are sent upon
258		successful completion of a query.
259		</p>
260		<a name="Errors"></a><h3>Errors</h3>
261		<p>Errors, or KRPC message dictionaries with a "y" value of "e",
262		contain one additional key "e". The value of "e" is a list. The first
263		element is an integer representing the error code. The second element
264		is a string containing the error message. Errors are sent when a query
265		cannot be fulfilled. The following table describes the possible error
266		codes:
267		</p>
268		<table>
269		<tbody><tr>
270		<td> 201 </td><td> Generic Error
271		</td></tr>
272		<tr>
273		<td> 202 </td><td> Server Error
274		</td></tr>
275		<tr>
276		<td> 203 </td><td> Protocol Error, such as a malformed packet, invalid arguments, or bad token
277		</td></tr>
278		<tr>
279		<td> 204 </td><td> Method Unknown
280		</td></tr></tbody></table>
281		<a name="Example_Error_Packets"></a><h4>Example Error Packets</h4>
282		<pre>generic error = {'t':0, 'y':'e', 'e':[201, "A Generic Error Ocurred"]}
283		bencoded = d1:eli201e23:A Generic Error Ocurrede1:ti0e1:y1:ee
284		</pre>
285		<p><br>
286		</p>
287		<a name="DHT_Queries"></a><h2>DHT Queries</h2>
288		<p>All queries have an "id" key and value containing the node ID of the
289		querying node. All responses have an "id" key and value containing the
290		node ID of the responding node.
291		</p><p><br>
292		</p>
293		<a name="ping"></a><h3>ping</h3>
294		<p>The most basic query is a ping. "q" = "ping" A ping query has a
295		single argument, "id" the value is a 20-byte string containing the
296		senders node ID in network byte order. The appropriate response to a
297		ping has a single key "id" containing the node ID of the responding
298		node.
299		</p>
300		<pre> arguments: {"id" : "<querying nodes id>"}
301		response: {"id" : "<queried nodes id>"}
302		</pre>
303		<a name="example_packets"></a><h4>example packets</h4>
304		<pre> ping Query = {"t":"0", "y":"q", "q":"ping", "a":{"id":"abcdefghij0123456789"}}
305		bencoded = d1:ad2:id20:abcdefghij0123456789e1:q4:ping1:t1:01:y1:qe
306		</pre>
307		<pre> Response = {"t":"0", "y":"r", "r": {"id":"mnopqrstuvwxyz123456"}}
308		bencoded = d1:rd2:id20:mnopqrstuvwxyz123456e1:t1:01:y1:re
309		</pre>
310		<p><br>
311		</p>
312		<a name="find_node"></a><h3>find_node</h3>
313		<p>Find node is used to find the contact information for a node given
314		its ID. "q" == "find_node" A find_node query has two arguments, "id"
315		containing the node ID of the querying node, and "target" containing
316		the ID of the node sought by the queryer. When a node receives a
317		find_node query, it should respond with a key "nodes" and value of a
318		string containing the compact node info for the target node or the K
319		(8) closest good nodes in its own routing table.
320		</p>
321		<pre>arguments: {"id" : "<querying nodes id>", "target" : "<id of target node>"}
322		response: {"id" : "<queried nodes id>", "nodes" : "<compact node info>"}
323		</pre>
324		<a name="example_packets_2"></a><h4>example packets</h4>
325		<pre>find_node Query = {'t':0, 'y':'q', 'q':'find_node', 'a': {'id':'abcdefghij0123456789', 'target':'mnopqrstuvwxyz123456'}}
326		bencoded = d1:ad2:id20:abcdefghij01234567896:target20:mnopqrstuvwxyz123456e1:q9:find_node1:ti0e1:y1:qe
327		</pre>
328		<pre>Response = {'t':0, 'y':'r', 'r': {'id':'0123456789abcdefghij', 'nodes': 'def456...'}}
329		bencoded = d1:rd2:id20:0123456789abcdefghij5:nodes9:def456...e1:ti0e1:y1:re
330		</pre>
331		<a name="get_peers"></a><h3>get_peers</h3>
332		<p>Get peers associated with a torrent infohash. "q" = "get_peers" A
333		get_peers query has two arguments, "id" containing the node ID of the
334		querying node, and "info_hash" containing the infohash of the torrent.
335		If the queried node has peers for the infohash, they are returned in a
336		key "values" as a list with a single string containing "compact" format
337		peer information concatenated together. If the queried node has no
338		peers for the infohash, a key "nodes" is returned containing the K
339		nodes in the queried nodes routing table closest to the infohash
340		supplied in the query. In either case a "token" key is also included in
341		the return value. The token value is a required argument for a future
342		announce_peer query.
343		</p>
344		<pre> arguments: {"id" : "<querying nodes id>", "info_hash" : "<20-byte infohash of target torrent>"}
345		response: {"id" : "<queried nodes id>", "values" : ["<compact peer info string>"]}
346		or: {"id" : "<queried nodes id>", "nodes" : "<compact node info>"}
347		</pre>
348		<a name="example_packets_3"></a><h4>example packets</h4>
349		<pre>get_peers Query = {'t':0, 'y':'q', 'q':'get_peers', 'a': {'id':'abcdefghij0123456789', 'info_hash':'mnopqrstuvwxyz123456'}}
350		bencoded = d1:ad2:id20:abcdefghij01234567899:info_hash20:mnopqrstuvwxyz123456e1:q9:get_peers1:ti0e1:y1:qe
351		</pre>
352		<pre>Response with peers = {'t':0, 'y':'r', 'r': {'id':'abcdefghij0123456789', 'token':'aoeusnth', 'values': ['axje.uidhtnmbrl']}}
353		bencoded = d1:rd2:id20:abcdefghij01234567895:token8:aoeusnth6:valuesl15:axje.uidhtnmbrlee1:ti0e1:y1:re
354		</pre>
355		<pre>Response with closest nodes = {'t':0, 'y':'r', 'r': {'id':'abcdefghij0123456789', 'token':'aoeusnth', 'nodes': 'def456...'}}
356		bencoded = d1:rd2:id20:abcdefghij01234567895:nodes9:def456...5:token8:aoeusnthe1:ti0e1:y1:re
357		</pre>
358		<p><br>
359		</p>
360		<a name="announce_peer"></a><h3>announce_peer</h3>
361		<p>Announce that the peer controlling the querying node is downloading
362		the a torrent on a port.
363		announce_peer has four arguments: "id" containing the node ID of the
364		querying node, "info_hash" containing the infohash of the torrent,
365		"port" containing the port as an integer, and the "token" received in
366		response to a previous get_peers query. The queried node must verify
367		that the token was previously sent to the same IP address as the
368		querying node. Then the queried node should store the IP address of the
369		querying node and the supplied port number under the infohash in its
370		store of peer contact information.
371		</p>
372		<pre>arguments: {"id" : "<querying nodes id>", "info_hash" : "<20-byte infohash of target torrent>", "port" : <port number>, "token" : "<opaque token>"}
373		response: {"id" : "<queried nodes id>"}
374		</pre>
375		<a name="example_packets_4"></a><h4>example packets</h4>
376		<pre>announce_peers Query = {'t':0, 'y':'q', 'q':'announce_peers', 'a': {'id':'abcdefghij0123456789', 'info_hash':'mnopqrstuvwxyz123456', 'port' : 6881, 'token' : 'aoeusnth'}}
377		bencoded = d1:ad2:id20:abcdefghij01234567899:info_hash20:mnopqrstuvwxyz1234564:porti6881e5:token8:aoeusnthe1:q14:announce_peers1:ti0e1:y1:qe
378		</pre>
379		<pre>Response = {"t":"0", "y":"r", "r": {"id":"mnopqrstuvwxyz123456"}}
380		bencoded = d1:rd2:id20:mnopqrstuvwxyz123456e1:t1:01:y1:re
381		</pre>
382		<p><br>
383		</p>
384		<a name="Footnotes"></a><h2>Footnotes</h2>
385		<ol><li><a href="https://dev.bittorrent.com/wiki/index.php?title=Template:Note&action=edit" class="new" title="Template:Note">Template:Note</a><a href="http://www.cs.rice.edu/Conferences/IPTPS02/109.pdf" class="external text" title="http://www.cs.rice.edu/Conferences/IPTPS02/109.pdf" rel="nofollow">"Kademlia: A Peer-to-peer Information System Based on the XOR Metric"</a>,
386		</li></ol>
387		<p>Petar Maymounkov and David Mazieres,
388		</p>
389		<ol><li><a href="https://dev.bittorrent.com/wiki/index.php?title=Template:Note&action=edit" class="new" title="Template:Note">Template:Note</a> Use SHA1 and plenty of entropy to ensure a unique ID
390		</li></ol>
391		</body></html>
	116	<li>If a peer receives an excessive number of requests from a peer it is choking, the peer receiving the requests MAY close the connection rather than reject the request. However, consider that it can take several seconds for buffers to drain and messages to propagate once a peer is choked.</li></ul>
	117
	118	<h2 id="Allowed_Fast">Allowed Fast</h2>
	119	<pre><i> Allowed Fast: <len=0x0005><op=0x11><index></i></pre>
	120	<p>With the BitTorrent protocol specified in <a href="protocol.html">[1]</a>, new peers take several minutes to ramp up before they can effectively engage in BitTorrent's tit-for-tat. The reason is simple: starting peers have few pieces to trade.</p>
	121	<p><i>Allowed Fast</i> is an advisory message which means "if you ask for this piece, I'll give it to you even if you're choked." <i>Allowed Fast</i> thus shortens the awkward stage during which the peer obtains occasional optimistic unchokes but cannot sufficiently reciprocate to remain unchoked.</p>
	122	<p>The pieces that can be downloaded when choked constitute a peer's <i>allowed fast set.</i> The set is generated using a canonical algorithm that produces piece indices unique to the message receiver so that if two peers offer <i>k</i> pieces fast it will be the same <i>k</i>, and if one offers <i>k+1</i> it will be the same <i>k</i> plus one more. <i>k</i> should be small enough to avoid abuse, but large enough to ramp up tit-for-tat. We currently set <i>k</i> to 10, but peers are free to change this number, e.g., to suit load.</p>
	123	<p>The message sender MAY list pieces that the message sender does not have. The receiver MUST NOT interpret an Allowed Fast message as meaning that the message sender has the piece. This allows peers to generate and communicate allowed fast sets at the beginning of a connection. However, a peer MAY send Allowed Fast messages at any time.</p>
	124	<p>A peer SHOULD send Allowed Fast messages to any starting peer unless the local peer lacks sufficient resources. A peer MAY reject requests for already Allowed Fast pieces if the local peer lacks sufficient resources, if the requested piece has already been sent to the requesting peer, or if the requesting peer is not a starting peer. Our current implementation rejects requests for Allowed Fast messages whenever the requesting peer has more than <i> k </i> pieces.</p>
	125	<p> When the fast extension is disabled, if a peer recieves an Allowed Fast message then the peer MUST close the connection.</p>
	126
	127	<h2 id="Allowed_Fast_Set_Generation"> Allowed Fast Set Generation </h2>
	128	<p>The canonical algorithm for computing a peer <i>P'</i>s <i>allowed fast set</i>
	129	follows. All integers in this pseudocode are four bytes represented in network (big-endian) byte order. <i>[a:b]</i> denotes the sequence of consecutive integers from <i>a</i> to <i>b</i> excluding <i>b</i>, i.e., <i>(a, a+1, a+2,..., b-1)</i>. <i>x[a:b]</i> denotes a subsequence of elements in an array <i>x</i> starting from index <i>a</i> to but not including index <i>b</i>.
	130	</p><p>Let <i>ip</i> denote <i>P'</i>s IPv4 address. We currently have no
	131	provisions for IPv6. If a peer is behind a Network Address Translator
	132	(NAT) then <i>ip</i> should be the externally facing IP address of the
	133	NAT. Since the node sending the <i>Allowed Fast</i> messages computes
	134	the set, the correct <i>ip</i> is usually the <i>ip</i> address on the other
	135	end of the connection. The host computing the set MAY use the <i>ip</i>
	136	address on the other end of the connection regardless
	137	</p><p>Let <i>sz</i> denote the number of pieces in the torrent.
	138	</p><p>Let <i>a</i> denote the allowed fast set.
	139	</p><p>Let <i>k</i> denote the final number of pieces in the allowed fast set.
	140	</p>
	141	<pre> x = 0xFFFFFF00 & ip (1)
	142	x.append(infohash) (2)
	143	while \|a\| < k:
	144	x = SHA1(x) (3)
	145	for i in [0:5] and \|a\| < k: (4)
	146	j = i*4 (5)
	147	y = x[j:j+4] (6)
	148	index = y % sz (7)
	149	if index not in a: (8)
	150	add index to a (9)
	151	</pre>
	152	<p>Step (1) selects the most significant octets in peer <i>P'</i>s
	153	ip address. We do this to prevent a user that obtains more than one
	154	IP address on the same network from obtaining more than one
	155	<i>allowed fast set.</i> Use of three bytes is heuristic and
	156	historical.
	157	</p><p>Step (3) generates a 20-byte random number on each call. By
	158	performing a SHA-1 hash on the previous iteration's hash, we can
	159	generate an arbitrarily long pseudorandom sequence.
	160	</p><p>Steps (4) through (9) partition the 20-byte hash into piece indices
	161	and add them to the allowed fast set.
	162	</p>
	163	</div><a name="Example_Implementation"></a><h2> Example Implementation </h2>
	164	<p>The following C++ implementation was provided by CacheLogic:
	165	</p>
	166	<pre>void generate_fast_set(
	167	uint32 k, // number of pieces in set
	168	uint32 sz, // number of pieces in torrent
	169	const char infohash[20], // infohash of torrent
	170	uint32 ip, // in host byte order, ie localhost is 0x7f000001
	171	std::vector<uint32> &a) // generated set of piece indices
	172	{
	173	a.clear();
	174	std::string x;
	175	char buf[4];
	176	(uint32)buf = htonl(ip & 0xffffff00);
	177	x.assign(buf, 4);
	178	x.append(infohash, 20); // (3)
	179	while (a.size()<k) {
	180	x = SHA1(x); // (4)
	181	for ( int i=0 ; i<5 && a.size()<k ; i++ ) { // (5)
	182	int j = i*4; // (6)
	183	uint32 y = ntohl((uint32)(x.data()+j)); // (7)
	184	uint32 index = y % sz; // (8)
	185	if (std::find(a.begin(), a.end(), index)==a.end()) { // (9)
	186	a.push_back(index); // (10)
	187	}
	188	}
	189	}
	190	}
	191	</pre>
	192	<p>Example results generated by this function:
	193	</p>
	194	<pre>7 piece allowed fast set for torrent with 1313 pieces and hex infohash
	195	aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa for node with IP 80.4.4.200:
	196	1059,431,808,1217,287,376,1188
	197	9 piece allowed fast set for torrent with 1313 pieces and hex infohash
	198	aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa for node with IP 80.4.4.200:
	199	1059,431,808,1217,287,376,1188,353,508
	200	</pre>
	201	</div>
	202	<!-- ### End Content ### -->
	203	</div>
	204	</div>
	205	<div id="footer">
	206	<hr />
	207	<p>Copyright 2006 BitTorrent.org</p>
	208	</div>
	209	</body>
	210	</html>

dotorg/trunk/html/developer.html

r4448	r4460
1		<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
2		<html><head><title>BitTorrent Community Forum for Developers</title
	1	<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
	2	"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
	3	<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
	4	<head>
	5	<meta http-equiv="Content-type" content="text/html; charset=utf-8" />
	6	<title>BitTorrent.org » For Developers</title>
	7	<link rel="stylesheet" type="text/css" href="./css/screen.css" media="screen" />
	8	</head>
	9	<body id="www-bittorrent-org">
	10	<div id="upper" class="clear">
	11	<div id="wrap">
	12	<div id="header">
	13	<h1><a href="./index.html">BitTorrent<span>.org</span></a></h1>
	14	</div>
	15	<div id="nav">
	16	<ul>
	17	<li><a href="./index.html">Home</a></li>
	18	<li><a href="./introduction.html">For Users</a></li>
	19	<li><span>For Developers</span></li>
	20	<!-- <li><a href="./blog.html">Blog</a></li> -->
	21	<!-- <li><a href="./donate.html">Donate!</a></li> -->
	22	</ul>
	23	</div>
	24	<!-- ### Begin Content ### -->
	25	<div id="second">
	26	<p>The BitTorrent Community Forum coordinates the development of the BitTorrent protocol suite and its reference implementation. It is the wish of Bram Cohen that the BitTorrent remain open source and that the protocol development process be formalized and open to the BitTorrent developer community. At this point, the organizational structure and the procedures for proposing and specifying extensions to BitTorrent have not been defined, and we invite established client developers to join us in designing this organization.</p>
	27	<ul>
3	28
4		<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
5		<link href="community.css" rel="stylesheet" type="text/css"></head>
6
7		<body>
8		<!-- TITLE LINE STARTS -->
9		<div align="left">
10		<div style="margin-left: 20px;">
11		<a href="http://www.bittorrent.org/index.html">
12		<span class="org"> bittorrent.org </span>
13		<!--<img src="images/bittorrent_lg.gif" alt="BitTorrent" border="0" height="54" width="153">-->
14		</a>
15		<!--<span class="title"> Community Forum </span>-->
16		</div>
17
18
19
20		<!-- DOWNLINK STARTS -->
21		<div style="margin-left:12px;">
22		<div id="downlinks"><a href="index.html">Home</a></div>
23		<div id="downlinks"><a href="introduction.html">For Users</a></div>
24		<div id="downunlink"><a href="developer.html">For Developers</a></div>
25		<div id="downlinks"><a href="donate.html">Donate!</a></div>
26		<div style="clear:left; background-color:#f0f0f0;height:5px;"><PRE></PRE>
27		</div>
	29	<li><a href="./protocol.html">Protocol Specifications v1.0</a></li>
	30	<li><a href="./fast_extensions.html">Fast Protocol Extensions</a></li><li><a href="./Draft_DHT_protocol.html">Experimental Draft: BitTorrent Trackerless DHT Protocol Specifications v1.0</a></li>
	31	</ul>
	32	<p>The original version of the protocol specifications written by Bram Cohen are currently being expanded to improve specificity. They will soon be made available for community review.</p>
	33	<p>The revised specifications to the protocol suite will cover:</p>
	34	<ul>
	35	<li>Torrent file syntax and semantics</li>
	36	<li>BitTorrent client-tracker protocol</li>
	37	<li>BitTorrent peer-to-peer protocol</li>
	38	<li>BitTorrent trackerless DHT protocol</li>
	39	</ul>
28	40	</div>
29		<!-- DOWNLINK ENDS -->
30
31		<!-- CONTENT STARTS -->
32		<div id="content" align="justify">
33		<p>
34		The BitTorrent Community Forum coordinates the development
35		of the BitTorrent protocol suite and its reference implementation.
36		It is the wish of BitTorrent, Inc. that the BitTorrent
37		remain open source and that the protocol development process
38		be formalized and open to the BitTorrent developer community.
39		At this point, the organizational structure and the procedures
40		for proposing and specifying extensions to BitTorrent have not
41		been defined, and we invite established client developers to
42		join us in designing this organization.
43		</p>
44		<p>
45
46		<ul>
47		<li><a href="protocol.html"> Protocol Specifications v1.0 </a> </li>
48		<li><a href="Draft_DHT_protocol.html"> Experimental Draft: BitTorrent Trackerless DHT Protocol Specifications v1.0 </a> </li>
49		</ul>
50
51		<p>
52		The oringal version of the protocol specifications written by Bram
53		Cohen are currently being expanded to improve specificity. They will soon be
54		made available for community review.
55
56		<!--However, as it
57		stands, the final decision with regard to any changes to the protocol
58		or its specifications resides with the BitTorrent inventor, Bram
59		Cohen.-->
60		</p>
61
62		<p>
63		The revised specifications to the protocol suite will cover:
64		</p>
65
66		<ul>
67		<li> Torrent file syntax and semantics </li>
68		<li> BitTorrent client-tracker protocol </li>
69		<li> BitTorrent peer-to-peer protocol </li>
70		<li> BitTorrent trackerless DHT protocol </li>
71		</ul>
72
	41	<!-- ### End Content ### -->
73	42	</div>
74
75		~~<!-- FOOTER STARTS -->~~
76		~~<div class="dashlines">~~
77		~~<br>Copyright &copy 2006 bittorrent.org </A>~~
78	43	</div>
79		<!-- FOOTER ENDS -->
80
81
	44	<div id="footer">
	45	<hr />
	46	<p>Copyright © 2006 BitTorrent.org</p>
	47	</div>
82	48	</body>
	49	</html>

dotorg/trunk/html/index.html

r4448	r4460
1		<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
2		<html><head><title>BitTorrent Community Forum</title>
3
4
5		<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
6
7
8		<link href="community.css" rel="stylesheet" type="text/css"></head>
9
10		<body>
11		<!-- TITLE LINE STARTS -->
12		<div align="left">
13		<div style="margin-left: 20px;">
14		<a href="http://www.bittorrent.org/index.html">
15		<span class="org"> bittorrent.org </span>
16		<!--<img src="images/bittorrent_lg.gif" alt="BitTorrent" border="0" height="54" width="153">-->
17		</a>
18		<!--<span class="title"> Community Forum </span>-->
19		</div>
20
21		<!-- DOWNLINK STARTS -->
22		<div style="margin-left:12px;">
23		<div id="downunlink"><a href="index.html">Home</a></div>
24		<div id="downlinks"><a href="introduction.html">For Users</a></div>
25		<div id="downlinks"><a href="developer.html">For Developers</a></div>
26		<div id="downlinks"><a href="donate.html">Donate!</a></div>
27		<div style="clear:left; background-color:#f0f0f0;height:5px;"><PRE></PRE>
28		</div>
	1	<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
	2	"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
	3	<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
	4	<head>
	5	<meta http-equiv="Content-type" content="text/html; charset=utf-8" />
	6	<title>BitTorrent.org</title>
	7	<link rel="stylesheet" type="text/css" href="./css/screen.css" media="screen" />
	8	</head>
	9	<body id="www-bittorrent-org">
	10	<div id="upper" class="clear">
	11	<div id="wrap">
	12	<div id="header">
	13	<h1><a href="./index.html">BitTorrent<span>.org</span></a></h1>
29	14	</div>
30		<!-- DOWNLINK ENDS -->
31
32		<!-- CONTENT STARTS -->
33
34
35		<div id="content" align="justify">
36		<p>
37		The BitTorrent Community Forum is dedicated to nuturing the
38		BitTorrent community by serving as a place for users and developers
39		to share information about the BitTorrent protocol suite and the
40		open-source reference implementation.
41		</p>
42
43		<!--For a few words from the BitTorrent inventor, go to <a
44		href="blog">Bram Cohen's blog</a>.-->
45
46		<p><span class="header">For users:</span></p>
47		<ul>
48		<li><a href="introduction.html">Introduction </a></li>
49		</ul>
50		<p class="header">For developers: </p>
51		<ul>
52		<li><a href="protocol.html">BitTorrent protocol specification </a></li>
53		<li><a href="developer.html"> For Developers </a> </li>
54		</ul>
55		<p class="header">For academics: </p>
56		<ul>
57		<li><a href="http://www.bittorrent.com/bittorrentecon.pdf">BitTorrent Economics Paper </a> (PDF) </li>
58		</ul>
59		</div>
60
61		<!-- CONTENT ENDS -->
	15	<div id="nav">
	16	<ul>
	17	<li><span>Home</span></li>
	18	<li><a href="./introduction.html">For Users</a></li>
	19	<li><a href="./developer.html">For Developers</a></li>
	20	<!-- <li><a href="./blog.html">Blog</a></li> -->
	21	<!-- <li><a href="./donate.html">Donate!</a></li> -->
	22	</ul>
62	23	</div>
63		<!-- LEFT ENDS -->
64
65		<!-- FOOTER STARTS -->
66		<div class="dashlines">
67		<br>Copyright &copy 2006 bittorrent.org </A>
	24	<!-- ### Begin Content ### -->
	25	<div id="intro">
	26	<p><strong>BitTorrent.org</strong> is a forum for developers to exchange ideas about the direction of the BitTorrent protocol.</p>
68	27	</div>
69		<!-- FOOTER ENDS -->
70		</body></html>
	28	<div id="home-l">
	29	<h2>For Users</h2>
	30	<ul>
	31	<li><a href="./introduction.html">Introduction</a></li>
	32	</ul>
	33	</div>
	34	<div id="home-c">
	35	<h2>For Developers</h2>
	36	<ul>
	37	<li><a href="./fast_extensions.html">New - Fast Protocol Extensions</a></li>
	38	<li><a href="./protocol.html">BitTorrent Protocol Specification</a></li>
	39	<li><a href="./developer.html">For Developers</a></li>
	40	</ul>
	41	</div>
	42	<div id="home-r">
	43	<h2>For Researchers</h2>
	44	<ul>
	45	<li><a href="./bittorrentecon.pdf">BitTorrent Economics Paper</a>
	46	(PDF)</li>
	47	</ul>
	48	</div>
	49	<!-- ### End Content ### -->
	50	</div>
	51	</div>
	52	<div id="footer">
	53	<hr />
	54	<p>Copyright © 2006 BitTorrent.org</p>
	55	</div>
	56	</body>
	57	</html>

dotorg/trunk/html/introduction.html

r4448	r4460
1		<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
2		<html><head><title>BitTorrent Community Forum - Introduction</title>
3
4		<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
5
6		<link href="community.css" rel="stylesheet" type="text/css"></head>
7
8		<body>
9		<!-- LEFT STARTS -->
10		<div align="left">
11		<div style="margin-left: 20px;">
12		<a href="http://www.bittorrent.org/index.html">
13		<span class="org"> bittorrent.org </span>
14		<!--<img src="images/bittorrent_lg.gif" alt="BitTorrent" border="0" height="54" width="153">-->
15		</a>
16		<!--<span class="title"> Community Forum </span>-->
17		</div>
18
19
20		<!-- DOWNLINK STARTS -->
21		<div style="margin-left:12px;">
22		<div id="downlinks"><a href="index.html">Home</a></div>
23		<div id="downunlink"><a href="introduction.html">For Users</a></div>
24		<div id="downlinks"><a href="developer.html">For Developers</a></div>
25		<div id="downlinks"><a href="donate.html">Donate!</a></div>
26		<div style="clear:left; background-color:#f0f0f0;height:5px;"><PRE></PRE>
27		</div>
	1	<?xml version="1.0" encoding="utf-8"?>
	2	<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
	3	"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
	4	<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
	5	<head>
	6	<meta http-equiv="Content-type" content="text/html; charset=utf-8" />
	7	<title>BitTorrent.org » For Users</title>
	8	<link rel="stylesheet" type="text/css" href="./css/screen.css" media="screen" />
	9	</head>
	10	<body id="www-bittorrent-org">
	11	<div id="upper" class="clear">
	12	<div id="wrap">
	13	<div id="header">
	14	<h1><a href="./index.html">BitTorrent<span>.org</span></a></h1>
28	15	</div>
29		<!-- DOWNLINK ENDS -->
30
31		<!-- CONTENT STARTS -->
32		<div id="content" align="justify">
33		<div class="solution">
34		<center>
35		<span class="header">The Problem with Publishing:<br>More customers require more<br> bandwidth</span><br>
36		<img src="images/central" alt="problem" height="245" width="300"><br>
37		<span class="header">The BitTorrent Solution:<br>
38		Users cooperate in the distribution</span> <br>
39		<img src="images/torrent" alt="solution" height="245" width="300"><br>
40		</center>
41		</div>
42		<p><span class="welcome">What is BitTorrent?</span></p>
43
44		<p>BitTorrent is a free speech tool.</p>
45
46		<p>BitTorrent gives you the same freedom to publish previously
47		enjoyed by only a select few with special equipment and lots
48		of money. ("Freedom of the press is limited to those who own
49		one" -- journalist A.J. Liebling.)</p>
50
51		<p>You have something terrific to publish -- a large music or
52		video file, software, a game or anything else that many
53		people would like to have. But the more popular your file
54		becomes, the more you are punished by soaring bandwidth
55		costs. If your file becomes phenomenally successful and a
56		flash crowd of hundreds or thousands try to get it at once,
57		your server simply crashes and no one gets it.</p>
58
59		<p>There is a solution to this vicious cycle. BitTorrent, the
60		result of over two years of intensive development, is a
61		simple and free software product that addresses all of these
62		problems.</p>
63
64		<p>The key to scaleable and robust distribution is cooperation.
65		With BitTorrent, those who get your file tap into their
66		upload capacity to give the file to others at the same time.
67		Those that provide the most to others get the best treatment
68		in return. ("Give and ye shall receive!")</p>
69
70		<p>Cooperative distribution can grow almost without limit,
71		because each new participant brings not only demand, but
72		also supply. Instead of a vicious cycle, popularity creates
73		a virtuous circle. And because each new participant brings
74		new resources to the distribution, you get limitless
75		scalability for a nearly fixed cost.</p>
76
77		<p>BitTorrent is not just a concept, but has an easy-to-use
78		implementation capable of swarming downloads across
79		unreliable networks. BitTorrent has been embraced by
80		numerous publishers to distribute to millions of users.</p>
81
82		<p>With BitTorrent free speech no longer has a high price.</p>
83
	16	<div id="nav">
	17	<ul>
	18	<li><a href="./index.html">Home</a></li>
	19	<li><span>For Users</span></li>
	20	<li><a href="./developer.html">For Developers</a></li>
	21	<!-- <li><a href="./blog.html">Blog</a></li> -->
	22	<!-- <li><a href="./donate.html">Donate!</a></li> -->
	23	</ul>
84	24	</div>
85		<!-- CONTENT ENDS -->
86
	25	<!-- ### Begin Content ### -->
	26	<div id="second">
	27	<h2>What is <strong>BitTorrent</strong>?</h2>
	28	<div class="img-r">
	29	<img src="./img/central.gif" alt="central.gif" />
	30	<p><strong>The Problem with Publishing:<br />
	31	More customers require more bandwidth</strong></p>
87	32	</div>
88		<!-- LEFT ENDS -->
89
90		<!-- FOOTER STARTS -->
91		<div style="clear: both; padding-bottom: 15px;"></div>
92		<div class="dashlines">
93		<br>Copyright &copy 2006 bittorrent.org </A>
94		</div>
95		<!-- FOOTER ENDS -->
96		</body></html>
	33	<p>BitTorrent is a free speech tool.</p>
	34	<p>BitTorrent gives you the same freedom to publish previously enjoyed by only a select few with special equipment and lots of money. ("Freedom of the press is limited to those who own one" — journalist A.J. Liebling.)</p>
	35	<p>You have something terrific to publish -- a large music or video file, software, a game or anything else that many people would like to have. But the more popular your file becomes, the more you are punished by soaring bandwidth costs. If your file becomes phenomenally successful and a flash crowd of hundreds or thousands try to get it at once, your server simply crashes and no one gets it.</p>
	36	<p>There is a solution to this vicious cycle. BitTorrent, the result of over two years of intensive development, is a simple and free software product that addresses all of these problems.</p>
	37	<div class="img-l">
	38	<img src="./img/torrent.gif" alt="torrent.gif" class="left" />
	39	<p><strong>The BitTorrent Solution:<br />
	40	Users cooperate in the distribution</strong></p>
	41	</div>
	42	<p>The key to scaleable and robust distribution is cooperation. With BitTorrent, those who get your file tap into their upload capacity to give the file to others at the same time. Those that provide the most to others get the best treatment in return. ("Give and ye shall receive!")</p>
	43	<p>Cooperative distribution can grow almost without limit, because each new participant brings not only demand, but also supply. Instead of a vicious cycle, popularity creates a virtuous circle. And because each new participant brings new resources to the distribution, you get limitless scalability for a nearly fixed cost.</p>
	44	<p>BitTorrent is not just a concept, but has an easy-to-use implementation capable of swarming downloads across unreliable networks. BitTorrent has been embraced by numerous publishers to distribute to millions of users.</p>
	45	<p>With BitTorrent free speech no longer has a high price.</p>
	46	</div>
	47	<!-- ### End Content ### -->
	48	</div>
	49	</div>
	50	<div id="footer">
	51	<hr />
	52	<p>Copyright © 2006 BitTorrent.org</p>
	53	</div>
	54	</body>
	55	</html>

dotorg/trunk/html/protocol.html

r4448	r4460
1		<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
2		<html><head><title>BitTorrent Community Forum - Protocol</title>
3
4		<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
5
6		<link href="community.css" rel="stylesheet" type="text/css"></head>
7
8		<body>
9		<!-- LEFT STARTS -->
10		<div align="left">
11		<div style="margin-left: 20px;">
12		<a href="http://www.bittorrent.org/index.html">
13		<span class="org"> bittorrent.org </span>
14		<!--<img src="images/bittorrent_lg.gif" alt="BitTorrent" border="0" height="54" width="153">-->
15		</a>
16		<!--<span class="title"> Community Forum </span>-->
17		</div>
18
19
20		<!-- DOWNLINK STARTS -->
21		<div style="margin-left:12px;">
22		<div id="downlinks"><a href="index.html">Home</a></div>
23		<div id="downlinks"><a href="introduction.html">For Users</a></div>
24		<div id="downunlink"><a href="developer.html">For Developers</a></div>
25		<div id="downlinks"><a href="donate.html">Donate!</a></div>
26		<div style="clear:left; background-color:#f0f0f0;height:5px;"><PRE></PRE>
27		</div>
	1	<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
	2	"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
	3	<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
	4	<head>
	5	<meta http-equiv="Content-type" content="text/html; charset=utf-8" />
	6	<title>BitTorrent.org » For Developers » Protocol Specification</title>
	7	<link rel="stylesheet" type="text/css" href="./css/screen.css" media="screen" />
	8	</head>
	9	<body id="www-bittorrent-org">
	10	<div id="upper" class="clear">
	11	<div id="wrap">
	12	<div id="header">
	13	<h1><a href="./index.html">BitTorrent<span>.org</span></a></h1>
28	14	</div>
29
30		<!-- DOWNLINK ENDS -->
31
32		<!-- CONTENT STARTS -->
33		<div id="content" align="justify">
34		<!-- <span class="welcome">Documentation: Protocol</span><br>
35		<a href="http://www.bittorrent.com/guide.html">Guide</a> \| <a href="http://www.bittorrent.com/protocol.html">Protocol</a> \| <a href="http://www.bittorrent.com/bittorrentecon.pdf">Paper</a> <br>
36		-->
37		<p>BitTorrent
38		is a protocol for distributing files. It identifies content by URL and
39		is designed to integrate seamlessly with the web. Its advantage over
40		plain HTTP is that when multiple downloads of the same file happen
41		concurrently, the downloaders upload to each other, making it possible
42		for the file source to support very large numbers of downloaders with
43		only a modest increase in its load. </p>
44		<p class="header">A BitTorrent file distribution consists of these entities: </p>
45		<ul>
46		<li>An ordinary web server
47		</li><li>A static 'metainfo' file
48		</li><li>A BitTorrent tracker
49		</li><li>An 'original' downloader
50		</li><li>The end user web browsers
51		</li><li>The end user downloaders </li>
52		</ul>
53
54		<p>There are ideally many end users for a single file. </p>
55		<p class="header">To start serving, a host goes through the following steps: </p>
56		<ol>
57		<li>Start running a tracker (or, more likely, have one running already).
58		</li><li>Start running an ordinary web server, such as apache, or have one already.
59		</li><li>Associate the extension .torrent with mimetype application/x-bittorrent on their web server (or have done so already).
60		</li><li>Generate a metainfo (.torrent) file using the complete file to be served and the URL of the tracker.
61		</li><li>Put the metainfo file on the web server.
62		</li><li>Link to the metainfo (.torrent) file from some other web page.
63		</li><li>Start a downloader which already has the complete file (the 'origin'). </li>
64
65		</ol>
66		<p class="header">To start downloading, a user does the following: </p>
67		<ol>
68		<li>Install BitTorrent (or have done so already).
69		</li><li>Surf the web.
70		</li><li>Click on a link to a .torrent file.
71		</li><li>Select where to save the file locally, or select a partial download to resume.
72		</li><li>Wait for download to complete.
73		</li><li>Tell downloader to exit (it keeps uploading until this happens). </li>
74		</ol>
75
76		<p class="header">The connectivity is as follows: </p>
77		<ul>
78		<li>The web site is serving up static files as normal, but kicking off the BitTorrent helper app on the clients.
79		</li><li>The
80		tracker is receiving information from all downloaders and giving them
81		random lists of peers. This is done over HTTP or HTTPS. </li><li>Downloaders are periodically checking
82		in with the tracker to keep it informed of their progress, and are
83		uploading to and downloading from each other via direct connections.
84		These connections use the BitTorrent peer protocol, which operates over
85		TCP. </li><li>The origin is uploading but not
86		downloading at all, since it has the entire file. The origin is
87		necessary to get the entire file into the network. Often for popular
88		downloads the origin can be taken down after a while since several
89		downloads may have completed and been left running indefinitely. </li>
90		</ul>
91		<p>Metainfo
92		file and tracker responses are both sent in a simple, efficient, and
93		extensible format called bencoding (pronounced 'bee encoding').
94		Bencoded messages are nested dictionaries and lists (as in Python),
95		which can contain strings and integers. Extensibility is supported by
96		ignoring unexpected dictionary keys, so additional optional ones can be
97		added later. </p>
98		<p class="header">Bencoding is done as follows: </p>
99
100		<ul>
101		<li>Strings are length-prefixed base ten followed by a colon and the string. For example 4:spam corresponds to 'spam'.
102		</li><li>Integers
103		are represented by an 'i' followed by the number in base 10 followed by
104		an 'e'. For example i3e corresponds to 3 and i-3e corresponds to -3.
105		Integers have no size limitation. i-0e is invalid. All encodings with a
106		leading zero, such as i03e , are invalid, other than i0e , which of
107		course corresponds to 0. </li><li>Lists are encoded as an 'l' followed
108		by their elements (also bencoded) followed by an 'e'. For example
109		l4:spam4:eggse corresponds to ['spam', 'eggs']. </li><li>Dictionaries are encoded as a 'd'
110		followed by a list of alternating keys and their corresponding values
111		followed by an 'e'. For example, d3:cow3:moo4:spam4:eggse corresponds
112		to {'cow': 'moo', 'spam': 'eggs'} and d4:spaml1:a1:bee corresponds to
113		{'spam': ['a', 'b']} . Keys must be strings and appear in sorted order
114		(sorted as raw strings, not alphanumerics). </li>
115		</ul>
116		<p class="header">Metainfo files are bencoded dictionaries with the following keys: </p>
117		announce
118		<blockquote>
119		<p>The URL of the tracker. </p>
120
121		</blockquote>
122		info
123		<blockquote>
124		<p>This maps to a dictionary, with keys described below. </p>
125		<p>The name key maps to a string which is the suggested name to save the file (or directory) as. It is purely advisory. </p>
126		<p>piece
127		length maps to the number of bytes in each piece the file is split
128		into. For the purposes of transfer, files are split into fixed-size
129		pieces which are all the same length except for possibly the last one
130		which may be truncated. Piece length is almost always a power of two,
131		most commonly 2 18 = 256 K (BitTorrent prior to version 3.2 uses 2 20 =
132		1 M as default). </p>
133		<p>pieces maps to a string
134		whose length is a multiple of 20. It is to be subdivided into strings
135		of length 20, each of which is the SHA1 hash of the piece at the
136		corresponding index. </p>
137
138		<p>There is also a
139		key length or a key files , but not both or neither. If length is
140		present then the download represents a single file, otherwise it
141		represents a set of files which go in a directory structure. </p>
142		<p>In the single file case, length maps to the length of the file in bytes. </p>
143		<p>For
144		the purposes of the other keys, the multi-file case is treated as only
145		having a single file by concatenating the files in the order they
146		appear in the files list. The files list is the value files maps to,
147		and is a list of dictionaries containing the following keys: </p>
148		<p>length
149		The length of the file, in bytes. path A list of strings corresponding
150		to subdirectory names, the last of which is the actual file name (a
151		zero length list is an error case). </p>
152		<p>In the single file case, the name key is the name of a file, in the muliple file case, it's the name of a directory. </p>
153		</blockquote>
154
155		<p>Tracker
156		queries are two way. The tracker receives information via HTTP GET
157		parameters and returns a bencoded message. Note that although the
158		current tracker implementation has its own web server, the tracker
159		could run very nicely as, for example, an apache module. </p>
160		<p class="header">Tracker GET requests have the following keys:</p>
161		info_hash
162		<blockquote>
163		<p>The
164		20 byte sha1 hash of the bencoded form of the info value from the
165		metainfo file. Note that this is a substring of the metainfo file. This
166		value will almost certainly have to be escaped. </p>
167		</blockquote>
168		peer_id
169		<blockquote>
170
171		<p>A
172		string of length 20 which this downloader uses as its id. Each
173		downloader generates its own id at random at the start of a new
174		download. This value will also almost certainly have to be escaped. </p>
175		</blockquote>
176		ip
177		<blockquote>
178		<p>An
179		optional parameter giving the IP (or dns name) which this peer is at.
180		Generally used for the origin if it's on the same machine as the
181		tracker. </p>
182		</blockquote>
183		port
184		<blockquote>
185		<p>The
186		port number this peer is listening on. Common behavior is for a
187		downloader to try to listen on port 6881 and if that port is taken try
188		6882, then 6883, etc. and give up after 6889. </p>
189
190		</blockquote>
191		uploaded
192		<p>The total amount uploaded so far, encoded in base ten ascii. </p>
193		downloaded
194		<blockquote>
195		<p>The total amount downloaded so far, encoded in base ten ascii. </p>
196		</blockquote>
197		left
198		<blockquote>
199
200		<p>The
201		number of bytes this peer still has to download, encoded in base ten
202		ascii. Note that this can't be computed from downloaded and the file
203		length since it might be a resume, and there's a chance that some of
204		the downloaded data failed an integrity check and had to be
205		re-downloaded. </p>
206		</blockquote>
207		event
208		<blockquote>
209		<p>This
210		is an optional key which maps to started , completed , or stopped (or
211		empty, which is the same as not being present). If not present, this is
212		one of the announcements done at regular intervals. An announcement
213		using started is sent when a download first begins, and one using
214		completed is sent when the download is complete. No completed is sent
215		if the file was complete when started. Downloaders send an announcement
216		using 'stopped' when they cease downloading. </p>
217		</blockquote>
218		<p>Tracker
219		responses are bencoded dictionaries. If a tracker response has a key
220		failure reason , then that maps to a human readable string which
221		explains why the query failed, and no other keys are required.
222		Otherwise, it must have two keys: interval , which maps to the number
223		of seconds the downloader should wait between regular rerequests, and
224		peers . peers maps to a list of dictionaries corresponding to peers,
225		each of which contains the keys peer id , ip , and port , which map to
226		the peer's self-selected ID, IP address or dns name as a string, and
227		port number, respectively. Note that downloaders may rerequest on
228		nonscheduled times if an event happens or they need more peers. </p>
229		<p>If
230		you want to make any extensions to metainfo files or tracker queries,
231		please coordinate with Bram Cohen to make sure that all extensions are
232		done compatibly. </p>
233
234		<p>BitTorrent's peer protocol operates over TCP. It performs efficiently without setting any socket options. </p>
235		<p>Peer connections are symmetrical. Messages sent in both directions look the same, and data can flow in either direction. </p>
236		<p>The
237		peer protocol refers to pieces of the file by index as described in the
238		metainfo file, starting at zero. When a peer finishes downloading a
239		piece and checks that the hash matches, it announces that it has that
240		piece to all of its peers. </p>
241		<p>Connections
242		contain two bits of state on either end: choked or not, and interested
243		or not. Choking is a notification that no data will be sent until
244		unchoking happens. The reasoning and common techniques behind choking
245		are explained later in this document. </p>
246		<p>Data
247		transfer takes place whenever one side is interested and the other side
248		is not choking. Interest state must be kept up to date at all times -
249		whenever a downloader doesn't have something they currently would ask a
250		peer for in unchoked, they must express lack of interest, despite being
251		choked. Implementing this properly is tricky, but makes it possible for
252		downloaders to know which peers will start downloading immediately if
253		unchoked. </p>
254		<p>Connections start out choked and not interested. </p>
255
256		<p>When
257		data is being transferred, downloaders should keep several piece
258		requests queued up at once in order to get good TCP performance (this
259		is called 'pipelining'.) On the other side, requests which can't be
260		written out to the TCP buffer immediately should be queued up in memory
261		rather than kept in an application-level network buffer, so they can
262		all be thrown out when a choke happens. </p>
263		<p>The
264		peer wire protocol consists of a handshake followed by a never-ending
265		stream of length-prefixed messages. The handshake starts with character
266		ninteen (decimal) followed by the string 'BitTorrent protocol'. The
267		leading character is a length prefix, put there in the hope that other
268		new protocols may do the same and thus be trivially distinguishable
269		from each other. </p>
270		<p>All later integers sent in the protocol are encoded as four bytes big-endian. </p>
271		<p>After
272		the fixed headers come eight reserved bytes, which are all zero in all
273		current implementations. If you wish to extend the protocol using these
274		bytes, please coordinate with Bram Cohen to make sure all extensions
275		are done compatibly. </p>
276		<p>Next comes the 20
277		byte sha1 hash of the bencoded form of the info value from the metainfo
278		file. (This is the same value which is announced as info_hash to the
279		tracker, only here it's raw instead of quoted here). If both sides
280		don't send the same value, they sever the connection. The one possible
281		exception is if a downloader wants to do multiple downloads over a
282		single port, they may wait for incoming connections to give a download
283		hash first, and respond with the same one if it's in their list. </p>
284		<p>After
285		the download hash comes the 20-byte peer id which is reported in
286		tracker requests and contained in peer lists in tracker responses. If
287		the receiving side's peer id doesn't match the one the initiating side
288		expects, it severs the connection. </p>
289
290		<p>That's
291		it for handshaking, next comes an alternating stream of length prefixes
292		and messages. Messages of length zero are keepalives, and ignored.
293		Keepalives are generally sent once every two minutes, but note that
294		timeouts can be done much more quickly when data is expected. </p>
295		<p class="header">All non-keepalive messages start with a single byte which gives their type. The possible values are: </p>
296		<ul>
297		<li>0 - choke
298		</li><li>1 - unchoke
299		</li><li>2 - interested
300		</li><li>3 - not interested
301		</li><li>4 - have
302		</li><li>5 - bitfield
303		</li><li>6 - request
304		</li><li>7 - piece
305		</li><li>8 - cancel </li>
306
307		</ul>
308		<p>'choke', 'unchoke', 'interested', and 'not interested' have no payload. </p>
309		<p>'bitfield'
310		is only ever sent as the first message. Its payload is a bitfield with
311		each index that downloader has sent set to one and the rest set to
312		zero. Downloaders which don't have anything yet may skip the 'bitfield'
313		message. The first byte of the bitfield corresponds to indices 0 - 7
314		from high bit to low bit, respectively. The next one 8-15, etc. Spare
315		bits at the end are set to zero. </p>
316		<p>The 'have' message's payload is a single number, the index which that downloader just completed and checked the hash of. </p>
317		<p>'request'
318		messages contain an index, begin, and length. The last two are byte
319		offsets. Length is generally a power of two unless it gets truncated by
320		the end of the file. All current implementations use 2 15 , and close
321		connections which request an amount greater than 2 17 . </p>
322		<p>'cancel'
323		messages have the same payload as request messages. They are generally
324		only sent towards the end of a download, during what's called 'endgame
325		mode'. When a download is almost complete, there's a tendency for the
326		last few pieces to all be downloaded off a single hosed modem line,
327		taking a very long time. To make sure the last few pieces come in
328		quickly, once requests for all pieces a given downloader doesn't have
329		yet are currently pending, it sends requests for everything to everyone
330		it's downloading from. To keep this from becoming horribly inefficient,
331		it sends cancels to everyone else every time a piece arrives. </p>
332
333		<p>'piece'
334		messages contain an index, begin, and piece. Note that they are
335		correlated with request messages implicitly. It's possible for an
336		unexpected piece to arrive if choke and unchoke messages are sent in
337		quick succession and/or transfer is going very slowly. </p>
338		<p>Downloaders
339		generally download pieces in random order, which does a reasonably good
340		job of keeping them from having a strict subset or superset of the
341		pieces of any of their peers. </p>
342		<p>Choking is
343		done for several reasons. TCP congestion control behaves very poorly
344		when sending over many connections at once. Also, choking lets each
345		peer use a tit-for-tat-ish algorithm to ensure that they get a
346		consistent download rate. </p>
347		<p>The choking
348		algorithm described below is the currently deployed one. It is very
349		important that all new algorithms work well both in a network
350		consisting entirely of themselves and in a network consisting mostly of
351		this one. </p>
352		<p>There are several criteria a
353		good choking algorithm should meet. It should cap the number of
354		simultaneous uploads for good TCP performance. It should avoid choking
355		and unchoking quickly, known as 'fibrillation'. It should reciprocate
356		to peers who let it download. Finally, it should try out unused
357		connections once in a while to find out if they might be better than
358		the currently used ones, known as optimistic unchoking. </p>
359		<p>The
360		currently deployed choking algorithm avoids fibrillation by only
361		changing who's choked once every ten seconds. It does reciprocation and
362		number of uploads capping by unchoking the four peers which it has the
363		best download rates from and are interested. Peers which have a better
364		upload rate but aren't interested get unchoked and if they become
365		interested the worst uploader gets choked. If a downloader has a
366		complete file, it uses its upload rate rather than its download rate to
367		decide who to unchoke. </p>
368
369		<p>For optimistic
370		unchoking, at any one time there is a single peer which is unchoked
371		regardless of it's upload rate (if interested, it counts as one of the
372		four allowed downloaders.) Which peer is optimistically unchoked
373		rotates every 30 seconds. To give them a decent chance of getting a
374		complete piece to upload, new connections are three times as likely to
375		start as the current optimistic unchoke as anywhere else in the
376		rotation. </p>
	15	<div id="nav">
	16	<ul>
	17	<li><a href="./index.html">Home</a></li>
	18	<li><a href="./introduction.html">For Users</a></li>
	19	<li><span>For Developers</span></li>
	20	<!-- <li><a href="./blog.html">Blog</a></li> -->
	21	<!-- <li><a href="./donate.html">Donate!</a></li> -->
	22	</ul>
377	23	</div>
378		<!-- CONTENT ENDS -->
379
	24	<!-- ### Begin Content ### -->
	25	<div id="second">
	26	<p>BitTorrent is a protocol for distributing files. It identifies content by URL and is designed to integrate seamlessly with the web. Its advantage over plain HTTP is that when multiple downloads of the same file happen concurrently, the downloaders upload to each other, making it possible for the file source to support very large numbers of downloaders with only a modest increase in its load.</p>
	27	<h3>A BitTorrent file distribution consists of these entities:</h3>
	28	<ul>
	29	<li>An ordinary web server</li>
	30	<li>A static 'metainfo' file</li>
	31	<li>A BitTorrent tracker</li>
	32	<li>An 'original' downloader</li>
	33	<li>The end user web browsers</li>
	34	<li>The end user downloaders</li>
	35	</ul>
	36	<p>There are ideally many end users for a single file.</p>
	37	<h3>To start serving, a host goes through the following steps:</h3>
	38	<ol>
	39	<li>Start running a tracker (or, more likely, have one running already).</li>
	40	<li>Start running an ordinary web server, such as apache, or have one already.</li>
	41	<li>Associate the extension .torrent with mimetype <code>application/x-bittorrent</code> on their web server (or have done so already).</li>
	42	<li>Generate a metainfo (.torrent) file using the complete file to be served and the URL of the tracker.</li>
	43	<li>Put the metainfo file on the web server.</li>
	44	<li>Link to the metainfo (.torrent) file from some other web page.</li>
	45	<li>Start a downloader which already has the complete file (the 'origin').</li>
	46	</ol>
	47	<h3>To start downloading, a user does the following:</h3>
	48	<ol>
	49	<li>Install BitTorrent (or have done so already).</li>
	50	<li>Surf the web.</li>
	51	<li>Click on a link to a .torrent file.</li>
	52	<li>Select where to save the file locally, or select a partial download to resume.</li>
	53	<li>Wait for download to complete.</li>
	54	<li>Tell downloader to exit (it keeps uploading until this happens).</li>
	55	</ol>
	56	<h3>The connectivity is as follows:</h3>
	57	<ul>
	58	<li>Strings are length-prefixed base ten followed by a colon and the string. For example <code>4:spam</code> corresponds to 'spam'.</li>
	59	<li>Integers are represented by an 'i' followed by the number in base 10 followed by an 'e'. For example <code>i3e</code> corresponds to 3 and <code>i-3e </code>corresponds to -3. Integers have no size limitation. <code>i-0e</code> is invalid. All encodings with a leading zero, such as <code>i03e</code>, are invalid, other than <code>i0e</code>, which of course corresponds to 0.</li>
	60	<li>Lists are encoded as an 'l' followed by their elements (also bencoded) followed by an 'e'. For example <code>l4:spam4:eggse</code> corresponds to ['spam', 'eggs'].</li>
	61	<li>Dictionaries are encoded as a 'd' followed by a list of alternating keys and their corresponding values followed by an 'e'. For example, <code>d3:cow3:moo4:spam4:eggse</code> corresponds to {'cow': 'moo', 'spam': 'eggs'} and <code>d4:spaml1:a1:bee</code> corresponds to {'spam': ['a', 'b']}. Keys must be strings and appear in sorted order (sorted as raw strings, not alphanumerics).</li>
	62	</ul>
	63	<h3>Metainfo files are bencoded dictionaries with the following keys:</h3>
	64	<dl>
	65	<dt><code>announce</code></dt>
	66	<dd>The URL of the tracker.</dd>
	67	<dt><code>info</code></dt>
	68	<dd>This maps to a dictionary, with keys described below.</dd>
	69	<dd>The <code>name</code> key maps to a string which is the suggested name to save the file (or directory) as. It is purely advisory.</dd>
	70	<dd><code>piece length</code> maps to the number of bytes in each piece the file is split into. For the purposes of transfer, files are split into fixed-size pieces which are all the same length except for possibly the last one which may be truncated. <code>piece length</code> is almost always a power of two, most commonly 2 18 = 256 K (BitTorrent prior to version 3.2 uses 2 20 = 1 M as default).</dd>
	71	<dd><code>pieces</code> maps to a string whose length is a multiple of 20. It is to be subdivided into strings of length 20, each of which is the SHA1 hash of the piece at the corresponding index.</dd>
	72	<dd>There is also a key <code>length</code> or a key <code>files</code>, but not both or neither. If <code>length</code> is present then the download represents a single file, otherwise it represents a set of files which go in a directory structure.</dd>
	73	<dd>In the single file case, <code>length</code> maps to the length of the file in bytes.</dd>
	74	<dd>For the purposes of the other keys, the multi-file case is treated as only having a single file by concatenating the files in the order they appear in the files list. The files list is the value <code>files</code> maps to, and is a list of dictionaries containing the following keys:</dd>
	75	<dd><code>length</code> - The length of the file, in bytes.</dd>
	76	<dd><code>path</code> - A list of strings corresponding to subdirectory names, the last of which is the actual file name (a zero length list is an error case).</dd>
	77	<dd>In the single file case, the name key is the name of a file, in the muliple file case, it's the name of a directory.</dd>
	78	</dl>
	79	<h3>Tracker GET requests have the following keys:</h3>
	80	<dl>
	81	<dt><code>info_hash</code></dt>
	82	<dd>The 20 byte sha1 hash of the bencoded form of the info value from the metainfo file. Note that this is a substring of the metainfo file. This value will almost certainly have to be escaped.</dd>
	83	<dt><code>peer_id</code></dt>
	84	<dd>A string of length 20 which this downloader uses as its id. Each downloader generates its own id at random at the start of a new download. This value will also almost certainly have to be escaped.</dd>
	85	<dt><code>ip</code></dt>
	86	<dd>An optional parameter giving the IP (or dns name) which this peer is at. Generally used for the origin if it's on the same machine as the tracker.</dd>
	87	<dt><code>port</code></dt>
	88	<dd>The port number this peer is listening on. Common behavior is for a downloader to try to listen on port 6881 and if that port is taken try 6882, then 6883, etc. and give up after 6889.</dd>
	89	<dt><code>uploaded</code></dt>
	90	<dd>The total amount uploaded so far, encoded in base ten ascii.</dd>
	91	<dt><code>downloaded</code></dt>
	92	<dd>The total amount downloaded so far, encoded in base ten ascii.</dd>
	93	<dt><code>left</code></dt>
	94	<dd>The number of bytes this peer still has to download, encoded in base ten ascii. Note that this can't be computed from downloaded and the file length since it might be a resume, and there's a chance that some of the downloaded data failed an integrity check and had to be re-downloaded.</dd>
	95	<dt><code>event</code></dt>
	96	<dd>This is an optional key which maps to <code>started</code>, <code>completed</code>, or <code>stopped</code> (or <code>empty</code>, which is the same as not being present). If not present, this is one of the announcements done at regular intervals. An announcement using <code>started</code> is sent when a download first begins, and one using <code>completed</code> is sent when the download is complete. No <code>completed</code> is sent if the file was complete when started. Downloaders send an announcement using <code>stopped</code> when they cease downloading.</dd>
	97	</dl>
	98	<p>Tracker responses are bencoded dictionaries. If a tracker response has a key <code>failure reason</code>, then that maps to a human readable string which explains why the query failed, and no other keys are required. Otherwise, it must have two keys: <code>interval</code>, which maps to the number of seconds the downloader should wait between regular rerequests, and <code>peers</code>. <code>peers</code> maps to a list of dictionaries corresponding to <code>peers</code>, each of which contains the keys <code>peer id</code>, <code>ip</code>, and <code>port</code>, which map to the peer's self-selected ID, IP address or dns name as a string, and port number, respectively. Note that downloaders may rerequest on nonscheduled times if an event happens or they need more peers.</p>
	99	<p>If you want to make any extensions to metainfo files or tracker queries, please coordinate with Bram Cohen to make sure that all extensions are done compatibly.</p>
	100	<p>BitTorrent's peer protocol operates over TCP. It performs efficiently without setting any socket options.</p>
	101	<p>Peer connections are symmetrical. Messages sent in both directions look the same, and data can flow in either direction.</p>
	102	<p>The peer protocol refers to pieces of the file by index as described in the metainfo file, starting at zero. When a peer finishes downloading a piece and checks that the hash matches, it announces that it has that piece to all of its peers.</p>
	103	<p>Connections contain two bits of state on either end: choked or not, and interested or not. Choking is a notification that no data will be sent until unchoking happens. The reasoning and common techniques behind choking are explained later in this document.</p>
	104	<p>Data transfer takes place whenever one side is interested and the other side is not choking. Interest state must be kept up to date at all times - whenever a downloader doesn't have something they currently would ask a peer for in unchoked, they must express lack of interest, despite being choked. Implementing this properly is tricky, but makes it possible for downloaders to know which peers will start downloading immediately if unchoked.</p>
	105	<p>Connections start out choked and not interested.</p>
	106	<p>When data is being transferred, downloaders should keep several piece requests queued up at once in order to get good TCP performance (this is called 'pipelining'.) On the other side, requests which can't be written out to the TCP buffer immediately should be queued up in memory rather than kept in an application-level network buffer, so they can all be thrown out when a choke happens.</p>
	107	<p>The peer wire protocol consists of a handshake followed by a never-ending stream of length-prefixed messages. The handshake starts with character ninteen (decimal) followed by the string 'BitTorrent protocol'. The leading character is a length prefix, put there in the hope that other new protocols may do the same and thus be trivially distinguishable from each other.</p>
	108	<p>All later integers sent in the protocol are encoded as four bytes big-endian.</p>
	109	<p>After the fixed headers come eight reserved bytes, which are all zero in all current implementations. If you wish to extend the protocol using these bytes, please coordinate with Bram Cohen to make sure all extensions are done compatibly.</p>
	110	<p>Next comes the 20 byte sha1 hash of the bencoded form of the info value from the metainfo file. (This is the same value which is announced as <code>info_hash</code> to the tracker, only here it's raw instead of quoted here). If both sides don't send the same value, they sever the connection. The one possible exception is if a downloader wants to do multiple downloads over a single port, they may wait for incoming connections to give a download hash first, and respond with the same one if it's in their list.</p>
	111	<p>After the download hash comes the 20-byte peer id which is reported in tracker requests and contained in peer lists in tracker responses. If the receiving side's peer id doesn't match the one the initiating side expects, it severs the connection.</p>
	112	<p>That's it for handshaking, next comes an alternating stream of length prefixes and messages. Messages of length zero are keepalives, and ignored. Keepalives are generally sent once every two minutes, but note that timeouts can be done much more quickly when data is expected.</p>
	113	<h3>All non-keepalive messages start with a single byte which gives their type.<br />
	114	The possible values are:</h3>
	115	<ul>
	116	<li>0 - choke</li>
	117	<li>1 - unchoke</li>
	118	<li>2 - interested</li>
	119	<li>3 - not interested</li>
	120	<li>4 - have</li>
	121	<li>5 - bitfield</li>
	122	<li>6 - request</li>
	123	<li>7 - piece</li>
	124	<li>8 - cancel</li>
	125	</ul>
	126	<p>'choke', 'unchoke', 'interested', and 'not interested' have no payload.</p>
	127	<p>'bitfield' is only ever sent as the first message. Its payload is a bitfield with each index that downloader has sent set to one and the rest set to zero. Downloaders which don't have anything yet may skip the 'bitfield' message. The first byte of the bitfield corresponds to indices 0 - 7 from high bit to low bit, respectively. The next one 8-15, etc. Spare bits at the end are set to zero.</p>
	128	<p>The 'have' message's payload is a single number, the index which that downloader just completed and checked the hash of.</p>
	129	<p>'request' messages contain an index, begin, and length. The last two are byte offsets. Length is generally a power of two unless it gets truncated by the end of the file. All current implementations use 2 15 , and close connections which request an amount greater than 2 17.</p>
	130	<p>'cancel' messages have the same payload as request messages. They are generally only sent towards the end of a download, during what's called 'endgame mode'. When a download is almost complete, there's a tendency for the last few pieces to all be downloaded off a single hosed modem line, taking a very long time. To make sure the last few pieces come in quickly, once requests for all pieces a given downloader doesn't have yet are currently pending, it sends requests for everything to everyone it's downloading from. To keep this from becoming horribly inefficient, it sends cancels to everyone else every time a piece arrives.</p>
	131	<p>'piece' messages contain an index, begin, and piece. Note that they are correlated with request messages implicitly. It's possible for an unexpected piece to arrive if choke and unchoke messages are sent in quick succession and/or transfer is going very slowly.</p>
	132	<p>Downloaders generally download pieces in random order, which does a reasonably good job of keeping them from having a strict subset or superset of the pieces of any of their peers.</p>
	133	<p>Choking is done for several reasons. TCP congestion control behaves very poorly when sending over many connections at once. Also, choking lets each peer use a tit-for-tat-ish algorithm to ensure that they get a consistent download rate.</p>
	134	<p>The choking algorithm described below is the currently deployed one. It is very important that all new algorithms work well both in a network consisting entirely of themselves and in a network consisting mostly of this one.</p>
	135	<p>There are several criteria a good choking algorithm should meet. It should cap the number of simultaneous uploads for good TCP performance. It should avoid choking and unchoking quickly, known as 'fibrillation'. It should reciprocate to peers who let it download. Finally, it should try out unused connections once in a while to find out if they might be better than the currently used ones, known as optimistic unchoking.</p>
	136	<p>The currently deployed choking algorithm avoids fibrillation by only changing who's choked once every ten seconds. It does reciprocation and number of uploads capping by unchoking the four peers which it has the best download rates from and are interested. Peers which have a better upload rate but aren't interested get unchoked and if they become interested the worst uploader gets choked. If a downloader has a complete file, it uses its upload rate rather than its download rate to decide who to unchoke.</p>
	137	<p>For optimistic unchoking, at any one time there is a single peer which is unchoked regardless of it's upload rate (if interested, it counts as one of the four allowed downloaders.) Which peer is optimistically unchoked rotates every 30 seconds. To give them a decent chance of getting a complete piece to upload, new connections are three times as likely to start as the current optimistic unchoke as anywhere else in the rotation.</p>
380	138	</div>
381		<!-- LEFT ENDS -->
382
383		<!-- FOOTER STARTS -->
384		<div class="dashlines">
385		<br>Copyright &copy 2006 bittorrent.org </A>
386		</div>
387		<!-- FOOTER ENDS -->
388		</body></html>
	139	<!-- ### End Content ### -->
	140	</div>
	141	</div>
	142	<div id="footer">
	143	<hr />
	144	<p>Copyright © 2006 BitTorrent.org</p>
	145	</div>
	146	</body>
	147	</html>

BitTorrent.org

Context Navigation

Changeset 4460

Legend:

dotorg/trunk/html/Draft_DHT_protocol.html

dotorg/trunk/html/developer.html

dotorg/trunk/html/index.html

dotorg/trunk/html/introduction.html

dotorg/trunk/html/protocol.html

Download in other formats: