Show
Ignore:
Timestamp:
02/14/08 12:51:44 (2 years ago)
Author:
dave
Message:

rstify bep 0015 from olaf. Remove references to BitTorrent?, Inc. in favor of bittorrent.org.
Update bep 0000 to refer to bep 0015.

Files:
1 modified

Legend:

Unmodified
Added
Removed

  • dotorg/trunk_fixed/html/beps/bep_0015.rst

    r10757 r10759  
    1 BEP: 1 
     1BEP: 15 
    22Title: UDP Tracker Protocol for BitTorrent 
    33Version: $Revision$ 
    44Last-Modified: $Date$ 
    55Author:  Olaf van der Spek <olafvdspek@gmail.com> 
    6 Status:  Active Draft 
    7 Type:    Process 
     6Status:  Draft 
     7Type:    Standards Track 
    88Created: 13-Feb-2008 
    99Post-History: 
    … …  
    1212============ 
    1313 
    14 To discover other peers in a swarm a client announces it's existance to a tracker. 
    15 The HTTP protocol is used and a typical request contains the following parameters: 
    16 info_hash, key, peer_id, port, downloaded, left, uploaded and compact. 
    17 A response contains a list of peers (host and port) and some other information. 
    18 The request and response are both quite short. 
    19 Since TCP is used, a connection has to be opened and closed, introducing additional overhead. 
     14To discover other peers in a swarm a client announces it's existance 
     15to a tracker.  The HTTP protocol is used and a typical request 
     16contains the following parameters: info_hash, key, peer_id, port, 
     17downloaded, left, uploaded and compact.  A response contains a list of 
     18peers (host and port) and some other information.  The request and 
     19response are both quite short.  Since TCP is used, a connection has to 
     20be opened and closed, introducing additional overhead. 
    2021 
    2122Overhead 
    2223======== 
    2324 
    24 Using HTTP introduces significant overhead. There's overhead at the ethernet layer (14 bytes per packet), at the IP layer (20 bytes per packet), at the TCP layer (20 bytes per packet) and at the HTTP layer. 
    25 About 10 packets are used for a request plus response containing 50 peers and the total number of bytes used is about 1206 [1]. 
    26 This overhead can be reduced significantly by using a UDP based protocol. The protocol proposed here uses 4 packets and about 618 bytes, reducing traffic by 50%. 
    27 For a client, saving 1 kbyte every hour isn't significant, but for a tracker serving a million peers, reducing traffic by 50% matters a lot. 
    28 An additional advantage is that a UDP based binary protocol doesn't require a complex parser and no connection handling, reducing the complexity of tracker code and increasing it's performance. 
     25Using HTTP introduces significant overhead. There's overhead at the 
     26ethernet layer (14 bytes per packet), at the IP layer (20 bytes per 
     27packet), at the TCP layer (20 bytes per packet) and at the HTTP layer. 
     28About 10 packets are used for a request plus response containing 50 
     29peers and the total number of bytes used is about 1206 [1].  This 
     30overhead can be reduced significantly by using a UDP based 
     31protocol. The protocol proposed here uses 4 packets and about 618 
     32bytes, reducing traffic by 50%.  For a client, saving 1 kbyte every 
     33hour isn't significant, but for a tracker serving a million peers, 
     34reducing traffic by 50% matters a lot.  An additional advantage is 
     35that a UDP based binary protocol doesn't require a complex parser and 
     36no connection handling, reducing the complexity of tracker code and 
     37increasing it's performance. 
    2938 
    3039UDP connections / spoofing 
    3140========================== 
    3241 
    33 In the ideal case, only 2 packets would be necessary. However, it is possible to spoof the source address of a UDP packet. 
    34 The tracker has to ensure this doesn't occur, so it calculates a value (connection_id) and sends it to the client. 
    35 If the client spoofed it's source address, it won't receive this value (unless it's sniffing the network). 
    36 The connection_id will then be send to the tracker again in packet 3. The tracker verifies the connection_id and ignores the request if it doesn't match. 
    37 Connection IDs should not be guessable by the client. This is comparable to a TCP handshake and a syn cookie like approach can be used to storing the connection IDs on the tracker side. 
    38 A connection ID can be used for multiple requests. A client can use a connection ID until one minute after it has received it. Trackers should accept the connection ID until two minutes after it has been send. 
     42In the ideal case, only 2 packets would be necessary. However, it is 
     43possible to spoof the source address of a UDP packet.  The tracker has 
     44to ensure this doesn't occur, so it calculates a value (connection_id) 
     45and sends it to the client.  If the client spoofed it's source 
     46address, it won't receive this value (unless it's sniffing the 
     47network).  The connection_id will then be send to the tracker again in 
     48packet 3. The tracker verifies the connection_id and ignores the 
     49request if it doesn't match.  Connection IDs should not be guessable 
     50by the client. This is comparable to a TCP handshake and a syn cookie 
     51like approach can be used to storing the connection IDs on the tracker 
     52side.  A connection ID can be used for multiple requests. A client can 
     53use a connection ID until one minute after it has received 
     54it. Trackers should accept the connection ID until two minutes after 
     55it has been send. 
    3956 
    4057Time outs 
    4158========= 
    4259 
    43 UDP is an 'unreliable' protocol. This means it doesn't retransmit lost packets itself. The application is responsible for this. 
    44 If a response is not received after 15 * 2 ^ n seconds, the client should retransmit the request, where n starts at 0 and is increased up to 8 (3840 seconds) after every retransmission. 
    45 Note that it is necessary to rerequest a connection ID when it has expired. 
     60UDP is an 'unreliable' protocol. This means it doesn't retransmit lost 
     61packets itself. The application is responsible for this.  If a 
     62response is not received after 15 * 2 ^ n seconds, the client should 
     63retransmit the request, where n starts at 0 and is increased up to 8 
     64(3840 seconds) after every retransmission.  Note that it is necessary 
     65to rerequest a connection ID when it has expired. 
    4666 
    4767Examples 
    4868======== 
    4969 
    50 Normal announce: 
    51 t = 0: connect request 
    52 t = 1: connect response 
    53 t = 2: announce request 
    54 t = 3: annonce response 
    55  
    56 Connect times out: 
    57 t = 0: connect request 
    58 t = 15: connect request 
    59 t = 45: connect request 
    60 t = 105: connect request 
    61 etc 
    62  
    63 Announce times out: 
    64 t = 0: 
    65 t = 0: connect request 
    66 t = 1: connect response 
    67 t = 2: announce request 
    68 t = 17: announce request 
    69 t = 47: announce request 
    70 t = 107: connect request (because connection ID expired) 
    71 t = 227: connect request 
    72 etc 
    73  
    74 Multiple requests: 
    75 t = 0: connect request 
    76 t = 1: connect response 
    77 t = 2: announce request 
    78 t = 3: annonce response 
    79 t = 4: announce request 
    80 t = 5: annonce response 
    81 t = 60: announce request 
    82 t = 61: annonce response 
    83 t = 62: connect request 
    84 t = 63: connect response 
    85 t = 64: announce request 
    86 t = 64: scrape request 
    87 t = 64: scrape request 
    88 t = 64: announce request 
    89 t = 65: announce response 
    90 t = 66: announce response 
    91 t = 67: scrape response 
    92 t = 68: scrape response 
     70Normal announce:: 
     71 
     72  t = 0: connect request 
     73  t = 1: connect response 
     74  t = 2: announce request 
     75  t = 3: annonce response 
     76 
     77Connect times out:: 
     78 
     79  t = 0: connect request 
     80  t = 15: connect request 
     81  t = 45: connect request 
     82  t = 105: connect request 
     83  etc 
     84 
     85Announce times out:: 
     86 
     87  t = 0: 
     88  t = 0: connect request 
     89  t = 1: connect response 
     90  t = 2: announce request 
     91  t = 17: announce request 
     92  t = 47: announce request 
     93  t = 107: connect request (because connection ID expired) 
     94  t = 227: connect request 
     95  etc 
     96 
     97Multiple requests:: 
     98 
     99  t = 0: connect request 
     100  t = 1: connect response 
     101  t = 2: announce request 
     102  t = 3: annonce response 
     103  t = 4: announce request 
     104  t = 5: annonce response 
     105  t = 60: announce request 
     106  t = 61: annonce response 
     107  t = 62: connect request 
     108  t = 63: connect response 
     109  t = 64: announce request 
     110  t = 64: scrape request 
     111  t = 64: scrape request 
     112  t = 64: announce request 
     113  t = 65: announce response 
     114  t = 66: announce response 
     115  t = 67: scrape response 
     116  t = 68: scrape response 
    93117 
    94118UDP tracker protocol 
    95119==================== 
    96120 
    97 All values are send in network byte order (big endian). Do not expect packets to be exactly of a certain size. Future extensions could increase the size of packets. 
     121All values are send in network byte order (big endian). Do not expect 
     122packets to be exactly of a certain size. Future extensions could 
     123increase the size of packets. 
    98124 
    99125Before announcing or scraping, you have to obtain a connection ID. 
    … …  
    1031293. Send the packet. 
    104130 
    105 connect request 
    106 Offset  Size            Name            Value 
    107 0       64-bit integer  connection_id   0x41727101980 
    108 8       32-bit integer  action          0 // connect 
    109 12      32-bit integer  transaction_id 
    110 16 
     131connect request:: 
     132 
     133  Offset  Size            Name            Value 
     134  0       64-bit integer  connection_id   0x41727101980 
     135  8       32-bit integer  action          0 // connect 
     136  12      32-bit integer  transaction_id 
     137  16 
    111138 
    1121391. Receive the packet. 
    … …  
    1161435. Store the connection ID for future use. 
    117144 
    118 connect response 
    119 Offset  Size            Name            Value 
    120 0       32-bit integer  action          0 // connect 
    121 4       32-bit integer  transaction_id 
    122 8       64-bit integer  connection_id 
    123 16 
     145connect response:: 
     146 
     147  Offset  Size            Name            Value 
     148  0       32-bit integer  action          0 // connect 
     149  4       32-bit integer  transaction_id 
     150  8       64-bit integer  connection_id 
     151  16 
    124152 
    1251531. Choose a random transaction ID. 
    … …  
    1271553. Send the packet. 
    128156 
    129 announce request 
    130 Offset  Size    Name    Value 
    131 0       64-bit integer  connection_id 
    132 8       32-bit integer  action          1 // announce 
    133 12      32-bit integer  transaction_id 
    134 16      20-byte string  info_hash 
    135 36      20-byte string  peer_id 
    136 56      64-bit integer  downloaded 
    137 64      64-bit integer  left 
    138 72      64-bit integer  uploaded 
    139 80      32-bit integer  event           0 // 0: none; 1: completed; 2: started; 3: stopped 
    140 84      32-bit integer  IP address      0 // default 
    141 88      32-bit integer  key 
    142 92      32-bit integer  num_want        -1 // default 
    143 96      16-bit integer  port 
    144 98 
     157announce request:: 
     158 
     159  Offset  Size    Name    Value 
     160  0       64-bit integer  connection_id 
     161  8       32-bit integer  action          1 // announce 
     162  12      32-bit integer  transaction_id 
     163  16      20-byte string  info_hash 
     164  36      20-byte string  peer_id 
     165  56      64-bit integer  downloaded 
     166  64      64-bit integer  left 
     167  72      64-bit integer  uploaded 
     168  80      32-bit integer  event           0 // 0: none; 1: completed; 2: started; 3: stopped 
     169  84      32-bit integer  IP address      0 // default 
     170  88      32-bit integer  key 
     171  92      32-bit integer  num_want        -1 // default 
     172  96      16-bit integer  port 
     173  98 
    145174 
    1461751. Receive the packet. 
    … …  
    1501795. Do not announce again until interval seconds have passed or an event has occurred. 
    151180 
    152 announce response 
    153 Offset      Size            Name            Value 
    154 0           32-bit integer  action          1 // announce 
    155 4           32-bit integer  transaction_id 
    156 8           32-bit integer  interval 
    157 12          32-bit integer  leechers 
    158 16          32-bit integer  seeders 
    159 20 + 6 * n  32-bit integer  IP address 
    160 24 + 6 * n  16-bit integer  TCP port 
    161 20 + 6 * N 
     181announce response:: 
     182 
     183  Offset      Size            Name            Value 
     184  0           32-bit integer  action          1 // announce 
     185  4           32-bit integer  transaction_id 
     186  8           32-bit integer  interval 
     187  12          32-bit integer  leechers 
     188  16          32-bit integer  seeders 
     189  20 + 6 * n  32-bit integer  IP address 
     190  24 + 6 * n  16-bit integer  TCP port 
     191  20 + 6 * N 
    162192 
    163193Up to about 74 torrents can be scraped at once. A full scrape can't be done with this protocol. 
    … …  
    1671973. Send the packet. 
    168198 
    169 scrape request 
    170 Offset          Size            Name            Value 
    171 0               64-bit integer  connection_id 
    172 8               32-bit integer  action          2 // scrape 
    173 12              32-bit integer  transaction_id 
    174 16 + 20 * n     20-byte string  info_hash 
    175 16 + 20 * N 
     199scrape request:: 
     200 
     201  Offset          Size            Name            Value 
     202  0               64-bit integer  connection_id 
     203  8               32-bit integer  action          2 // scrape 
     204  12              32-bit integer  transaction_id 
     205  16 + 20 * n     20-byte string  info_hash 
     206  16 + 20 * N 
    176207 
    1772081. Receive the packet. 
    … …  
    1802114. Check whether the action is scrape. 
    181212 
    182 scrape response 
    183 Offset      Size            Name            Value 
    184 0           32-bit integer  action          2 // scrape 
    185 4           32-bit integer  transaction_id 
    186 8 + 12 * n  32-bit integer  seeders 
    187 12 + 12 * n 32-bit integer  completed 
    188 16 + 12 * n 32-bit integer  leechers 
    189 8 + 12 * N 
     213scrape response:: 
     214 
     215  Offset      Size            Name            Value 
     216  0           32-bit integer  action          2 // scrape 
     217  4           32-bit integer  transaction_id 
     218  8 + 12 * n  32-bit integer  seeders 
     219  12 + 12 * n 32-bit integer  completed 
     220  16 + 12 * n 32-bit integer  leechers 
     221  8 + 12 * N 
    190222 
    191223If the tracker encounters an error, it might send an error packet. 
    … …  
    1952273. Check whether the transaction ID is equal to the one you chose. 
    196228 
    197 error response 
    198 Offset  Size            Name            Value 
    199 0       32-bit integer  action          3 // error 
    200 4       32-bit integer  transaction_id 
    201 8       string  message 
     229error response:: 
     230 
     231  Offset  Size            Name            Value 
     232  0       32-bit integer  action          3 // error 
     233  4       32-bit integer  transaction_id 
     234  8       string  message 
    202235 
    203236Existing implementations 
    204237======================== 
    205 Azureus, libtorrent [2], opentracker [3], XBT Client and XBT Tracker support this protocol. 
     238 
     239Azureus, libtorrent [2], opentracker [3], XBT Client and XBT Tracker 
     240support this protocol. 
    206241 
    207242IPv6 
    208243==== 
    209244 
    210 IPv6 is not supported at the moment. A simple way to support IPv6 would be to increase the size of all IP addresses to 128 bits when the request is done over IPv6. 
    211 However, I think more experience with IPv6 and discussion is needed before including it. 
     245IPv6 is not supported at the moment. A simple way to support IPv6 
     246would be to increase the size of all IP addresses to 128 bits when the 
     247request is done over IPv6.  However, I think more experience with IPv6 
     248and discussion is needed before including it. 
    212249 
    213250Extensions 
    214251========== 
    215252 
    216 Extension bits or a version field are not included. Clients and trackers should not assume packets to be of a certain size. This way, additional fields can be added without breaking compatibility. 
     253Extension bits or a version field are not included. Clients and 
     254trackers should not assume packets to be of a certain size. This way, 
     255additional fields can be added without breaking compatibility. 
    217256 
    218257References and Footnotes