Show
Ignore:
Timestamp:
02/06/2008 02:55:28 AM (11 months ago)
Author:
dave
Message:

Clarify some issues in the "Tracker Peer Obfuscation" BEP.

In particular it is now clear that peers returned in response to a request with sha_ih
can be assumed to implement Message Stream Encryption.

Also the index and RC4 pseudorandom string length are now randomized and obscured.

Aslo some wordsmithing.

Files:
1 modified

Legend:

Unmodified
Added
Removed

  • dotorg/trunk_fixed/html/beps/bep_0008.html

    r10540 r10560  
    5858<p class="topic-title first">Contents</p> 
    5959<ul class="simple"> 
    60 <li><a class="reference internal" href="#announce-parameter" id="id7">announce parameter</a></li> 
    61 <li><a class="reference internal" href="#announce-response" id="id8">announce response</a></li> 
    62 <li><a class="reference internal" href="#peer-list-obfuscation" id="id9">peer list obfuscation</a></li> 
    63 <li><a class="reference internal" href="#optimizations" id="id10">optimizations</a></li> 
    64 <li><a class="reference internal" href="#backwards-compatibility" id="id11">backwards compatibility</a></li> 
    65 <li><a class="reference internal" href="#rationale" id="id12">rationale</a></li> 
    66 <li><a class="reference internal" href="#id2" id="id13">References</a></li> 
     60<li><a class="reference internal" href="#announce-parameter" id="id8">Announce Parameter</a></li> 
     61<li><a class="reference internal" href="#announce-response" id="id9">Announce Response</a></li> 
     62<li><a class="reference internal" href="#peer-list-obfuscation" id="id10">Peer List Obfuscation</a></li> 
     63<li><a class="reference internal" href="#optimizations" id="id11">Optimizations</a></li> 
     64<li><a class="reference internal" href="#backwards-compatibility" id="id12">Backwards Compatibility</a></li> 
     65<li><a class="reference internal" href="#rationale" id="id13">Rationale</a></li> 
     66<li><a class="reference internal" href="#references" id="id14">References</a></li> 
    6767</ul> 
    6868</div> 
    6969<p>This extends the tracker protocol to support simple obfuscation of the 
    70 peers it returns, using the info hash as a shared secret between the 
     70peers it returns, using the infohash as a shared secret between the 
    7171peer and the tracker. The obfuscation does not provide any security 
    7272against eavesdroppers that know the infohash of the torrent.  The goal 
    … …  
    7777<p>The key words &quot;MUST&quot;, &quot;MUST NOT&quot;, &quot;REQUIRED&quot;, &quot;SHALL&quot;, &quot;SHALL NOT&quot;, &quot;SHOULD&quot;, 
    7878&quot;SHOULD NOT&quot;, &quot;RECOMMENDED&quot;, &quot;MAY&quot;, and &quot;OPTIONAL&quot; in this document are 
    79 to be interpreted as described in IETF <a class="reference external" href="http://tools.ietf.org/html/rfc2119">RFC 2119</a> <a class="footnote-reference" href="#id3" id="id4">[1]</a>.</p> 
     79to be interpreted as described in IETF <a class="reference external" href="http://tools.ietf.org/html/rfc2119">RFC 2119</a> <a class="footnote-reference" href="#id6" id="id7">[5]</a>.</p> 
    8080<div class="section" id="announce-parameter"> 
    81 <h1>announce parameter</h1> 
     81<h1>Announce Parameter</h1> 
    8282<p>When using this extension, instead of passing the <tt class="docutils literal"><span class="pre">info_hash</span></tt> parameter 
    8383to the tracker, an <tt class="docutils literal"><span class="pre">sha_ih</span></tt> is passed.</p> 
    … …  
    9494</div> 
    9595<div class="section" id="announce-response"> 
    96 <h1>announce response</h1> 
     96<h1>Announce Response</h1> 
    9797<p>If the tracker supports this extension, the response should be exactly the 
    9898same as if the <tt class="docutils literal"><span class="pre">info_hash</span></tt> was passed, except that any field that contains 
    … …  
    103103</div> 
    104104<div class="section" id="peer-list-obfuscation"> 
    105 <h1>peer list obfuscation</h1> 
     105<h1>Peer List Obfuscation</h1> 
    106106<p>We distinguish between the <em>tracker peer list</em> and the <em>returned peer 
    107107list</em>.  The <em>tracker peer list</em> contains the ip-port pairs of all 
    … …  
    121121recovers the plaintext by generating the same pseudorandom string and 
    122122XOR'ing it with the ciphertext.  In generating the pseudorandom 
    123 string, the tracker and client MUST discard the first 768 bytes.  All 
    124 indices into the pseudorandom string ignore the first 768 bytes, i.e., 
    125 index 0 refers to the first undiscarded byte.</p> 
     123string, the tracker and client MUST discard the first 768 bytes.</p> 
    126124<p>To communicate an initialization vector, the tracker includes in the 
    127125bencoded response the key <tt class="docutils literal"><span class="pre">iv</span></tt> with value set to a byte string 
    … …  
    131129of the infohash are used as the RC4 key.  If the tracker provides an 
    132130initialization vector then the RC4 key is generated by appending the 
    133 vector to the infohash and then hashing with sha1.  The first 64 bits 
    134 of the resulting hash are then used as the RC4 key.</p> 
     131vector to the infohash and then hashing with SHA-1.  The first 64 bits 
     132of the resulting hash are then used as the RC4 key.  The string from 
     133which the RC4 key is derived whether it be the infohash or the SHA-1 of 
     134the initialization vector appended to the infohash is called the 
     135<em>intermediate string</em>.</p> 
    135136<p>For example, given infohash <tt class="docutils literal"><span class="pre">aaf4c61ddcc5e8a2dabedef3b482cd9aea9434d</span></tt> 
    136137and initialization vector <tt class="docutils literal"><span class="pre">abcd</span></tt> both represented in hex, the RC4 key 
    137138is derived as follows:</p> 
    138139<pre class="literal-block"> 
    139 hash = sha1( 'aaf4c61ddcc5e8a2dabedef3b482cd9aea9434dabcd' ) 
    140 key = hash[0:64] 
     140intermediate = sha1( 'aaf4c61ddcc5e8a2dabedef3b482cd9aea9434dabcd' ) 
     141key = intermediate[0:64] 
    141142</pre> 
    142 <p>where [i:j] denotes the ith through jth bit including the ith but 
    143 excluding the jth.  The resulting key in hex is <tt class="docutils literal"><span class="pre">f36e9cae87cf33e0</span></tt>.</p> 
     143<p>where [i:j] denotes the ith through <em>jth</em> bit including the <em>ith</em> but 
     144excluding the <em>jth</em>.  The resulting key in hex is <tt class="docutils literal"><span class="pre">f36e9cae87cf33e0</span></tt>.</p> 
    144145<p>A 64-bit key is used to avoid U.S. export restrictions.</p> 
    145146<p>It is RECOMMENDED that the tracker use the initialization vector, and 
    … …  
    148149</div> 
    149150<div class="section" id="optimizations"> 
    150 <h1>optimizations</h1> 
     151<h1>Optimizations</h1> 
    151152<p>The described optimizations are OPTIONAL for the tracker, but the 
    152 corresponding client-side MUST be implemented by those that support 
    153 this extension.  These extensions hobble the strength of the RC4 
     153corresponding client-side MUST be implemented by clients that support 
     154this extension.  These optimizations hobble the strength of the RC4 
    154155encryption in order to improve tracker performance.  In the <a class="reference internal" href="#rationale">rationale</a> 
    155156section we discuss why hobbling RC4 is reasonable and in many cases 
    … …  
    170171<p>The tracker MAY also cache the encrypted tracker peer list.  To 
    171172support this the tracker must pass two additional keys <em>i</em> and <em>n</em> 
    172 each with integer values.  Decryption starts by XORing from <em>6i</em> bytes 
    173 for ipv4 (or <em>18i</em> for ipv6) into the pseudorandom string.  Assuming 
     173each with 32-bit integer values.  If the tracker returns <em>i</em> and <em>n</em> 
     174then we reserve the first 8 bytes of the RC4 pseudorandom string to 
     175obscure <em>i</em> and <em>n</em>.  We come back to this momentarily.  Decryption 
     176starts by XORing from <em>6i</em> bytes for ipv4 (or <em>18i</em> for ipv6) into the 
     177pseudorandom string after the discarded and reserved bytes.  Assuming 
    174178that the tracker encrypted the tracker peer list starting from the 
    175 first byte in the pseudorandom string then <em>i</em> also corresponds to the 
    176 <em>ith</em> ip-port pair in the tracker peer list and the starting point of 
    177 the copy into the returned request.</p> 
     179first byte after the discarded and reserved bytes in the pseudorandom 
     180string then <em>i</em> also corresponds to the <em>ith</em> ip-port pair in the 
     181tracker peer list and the starting point of the copy into the returned 
     182request.</p> 
    178183<p>So that the client and the tracker do not have to generate an 
    179184arbitrarily long pseudorandom string to support large swarms, we 
    180185assume the tracker bounds the length of the pseudorandom string and 
    181 reports the length in ip-port pairs as the value to key <em>n</em>.  We 
    182 RECOMMEND that <em>n</em> be equal to the length of the tracker peer list or 
    183 random but within constant factor of the longest peerlist returned by 
    184 the tracker, whichever is smaller.  Thus the tracker encrypts the 
    185 <em>jth</em> byte of the <em>ith</em> ip-port pair in an ipv4 tracker peer list by 
    186 XORing with the byte <em>(6i+j)</em> <cite>mod</cite> <em>n</em> bytes into the pseudorandom 
    187 string.</p> 
    188 <p>Since providing <em>i</em> and <em>n</em> significantly reduces the cost for an 
    189 attacker to recover the pseudorandom string, the tracker MUST XOR the 
    190 value of <em>i</em> with the 32 bits beginning with the 8th least significant 
    191 byte of the infohash and <em>n</em> with the least significant 32 bits of the 
    192 infohash.  We do not use the first 64 bits of the infohash since the 
    193 first 64 bits are used as the RC4 key when not specifying an 
    194 initialization vector.</p> 
     186reports the length in ip-port pairs as the value to key <em>n</em>.  <em>n</em> 
     187excludes reserved and discarded bytes.  We RECOMMEND that <em>n</em> be equal 
     188to the length of the tracker peer list or random but within constant 
     189factor of the longest peerlist returned by the tracker, whichever is 
     190smaller.  Thus the tracker encrypts the <em>jth</em> byte of the <em>ith</em> 
     191ip-port pair in an ipv4 tracker peer list by XORing with the byte 
     192<em>(6i+j)</em> <cite>mod</cite> <em>n</em> bytes into the pseudorandom string.</p> 
     193<p>Transmitting <em>i</em> and <em>n</em> as plaintext would significantly reduce the 
     194cost for an attacker to recover the pseudorandom string.  The tracker 
     195MUST XOR the value of <em>i</em> with the first 32 bits of the pseudorandom 
     196string.  The tracker then XORs <em>n</em> with the next 32 bits from the 
     197pseudorandom string (see Figure 1).</p> 
     198<div class="figure"> 
     199<img alt="bep_0008_pseudo.png" src="bep_0008_pseudo.png" /> 
     200<p class="caption"><strong>Figure 1:</strong> The first 768 bytes of the RC4 pseudorandom 
     201string are discarded.  The key <em>i</em> in the tracker response has 
     202value <tt class="docutils literal"><span class="pre">x</span> <span class="pre">xor</span> <span class="pre">i</span></tt>.  The key <em>n</em> has value <tt class="docutils literal"><span class="pre">y</span> <span class="pre">xor</span> <span class="pre">n</span></tt>.</p> 
     203</div> 
    195204<p>We describe encryption in the following example for an ipv4 tracker peer 
    196205list consisting of 3 ip-port pairs, and using an RC4 pseudorandom string 
    197206of length <em>n=2</em>. <em>n</em> is small for purposes of illustration.  Also, for the 
    198207purpose of illustration, the tracker returns only 3 peers at a time.</p> 
    199 <dl class="docutils"> 
    200 <dt>::</dt> 
    201 <dd><p class="first">Given the following peer list 
    202 (208.72.193.86, 6881), (209.81.173.15,14321), (128.213.6.8, 6881)</p> 
    203 <p>As a packed array represented in hex it becomes</p> 
    204 <p>d048c1561ae1d151ad0f37f180d506081ae1</p> 
    205 <p>which we XOR with an RC4 pseudorandom string, e.g.,</p> 
    206 <p>a496e5f9b83e835013d42226</p> 
    207 <p>to generate</p> 
    208 <p class="last">74de24afa2df5201bedb15d72443e3f1a2df</p> 
    209 </dd> 
    210 </dl> 
     208<pre class="literal-block"> 
     209Given the following peer list 
     210(208.72.193.86, 6881), (209.81.173.15,14321), (128.213.6.8, 6881) 
     211 
     212As a packed array represented in hex it becomes 
     213 
     214d048c1561ae1d151ad0f37f180d506081ae1 
     215 
     216which we XOR with an RC4 pseudorandom string excluding discarded and 
     217reserved bytes, e.g., 
     218 
     219a496e5f9b83e835013d42226 
     220 
     221to generate 
     222 
     22374de24afa2df5201bedb15d72443e3f1a2df 
     224</pre> 
    211225<p>Because the RC4 pseudorandom string is shorter than the tracker 
    212226peer list, we wrap to the beginning of the pseudorandom string.</p> 
    213 <blockquote> 
    214 <p>In the first response, the tracker would return</p> 
    215 <p><em>peers=74de24afa2df5201bedb15d7</em>, <em>i=0</em>, <em>n=2</em></p> 
    216 <p>In the second response, the tracker would return</p> 
    217 <p><em>peers=5201bedb15d72443e3f1a2df</em>, <em>i=1</em>, <em>n=2</em></p> 
    218 </blockquote> 
     227<p>In the first response, the tracker would return:</p> 
     228<pre class="literal-block"> 
     229peers=74de24afa2df5201bedb15d7, i=0, n=2 
     230</pre> 
     231<p>In the second response, the tracker would return:</p> 
     232<pre class="literal-block"> 
     233peers=5201bedb15d72443e3f1a2df, i=1, n=2 
     234</pre> 
    219235<p>The tracker response MUST remain a valid bencoded message.</p> 
    220236</div> 
    221237<div class="section" id="backwards-compatibility"> 
    222 <h1>backwards compatibility</h1> 
     238<h1>Backwards Compatibility</h1> 
    223239<p>Trackers that support obfuscation are identified in the .torrent file 
    224240by the inclusion of an <tt class="docutils literal"><span class="pre">obfuscate-announce-list</span></tt> which otherwise has the 
    … …  
    240256in the same request, since that would defeat the purpose of the shared 
    241257secret.</p> 
    242 <ul class="simple"> 
    243 <li>any peer that reports a sha_ih is assumed to implement protocol encryption.</li> 
    244 <li>tracker OPTIONALLY includes a <tt class="docutils literal"><span class="pre">flags</span></tt> key that contains one byte per 
    245 peer in the reported peer list.  The first bit of each byte specifies 
    246 whether the peer support protocol encryption. (How do we protect 
    247 the flags field from modification?)</li> 
    248 <li>if no flags field is present then all peers should be assumed to support protocol encryption</li> 
    249 <li>if a flags field is present then those peers that support protocol 
    250 a</li> 
    251 </ul> 
     258<p>Any peer that requests with a <tt class="docutils literal"><span class="pre">sha_ih</span></tt> SHOULD implement Message 
     259Stream Encryption (MSE) <a class="footnote-reference" href="#mse" id="id1">[1]</a>.  Any peer returned from the tracker 
     260in response to a request with a <tt class="docutils literal"><span class="pre">sha_ih</span></tt> SHOULD be assumed to 
     261support Message Stream Encryption.  We include these provisions 
     262because if a peer communicates with another peer without using MSE 
     263then the BitTorrent protocol is trivially identified from the first 
     264twenty bytes of the BitTorrent header and the <tt class="docutils literal"><span class="pre">info_hash</span></tt> appears in 
     265plaintext as the next twenty bytes, hence also defeating the purpose 
     266of the shared secret.</p> 
     267<p>If the tracker does not know enough peers that support MSE to return 
     268the desired number of peers then it MAY include peers that are not 
     269assumed to support MSE.  If a peer closes a connection in response to 
     270an encrypted header then the initiating peer should try other peers in 
     271the peer list returning to the peer that closed the connection only 
     272when all other peers known or not yet known to support MSE have been 
     273tried and have failed to provide &quot;adequate performance.&quot;  We 
     274intentionally omit any definition of &quot;adequate performance.&quot;</p> 
    252275</div> 
    253276<div class="section" id="rationale"> 
    254 <h1>rationale</h1> 
     277<h1>Rationale</h1> 
    255278<p>This extension directly addresses a known attack on the BitTorrent 
    256279protocol performed by some deployed network hardware.  By obscuring 
    … …  
    262285performed such as severely rate limiting or blocking these 
    263286connections.</p> 
    264 <p>This hardware was presumably deployed to get around <a class="reference external" href="http://www.bittorrent.org/beps/bep_0013.html">BitTorrent 
    265 Protocol Encryption</a> <a class="footnote-reference" href="#id5" id="id6">[2]</a>.  Peers implementing BitTorrent Protocol 
     287<p>This hardware was presumably deployed to get around BitTorrent 
     288Message Stream Encryption <a class="footnote-reference" href="#mse" id="id2">[1]</a>.  Peers implementing BitTorrent Message Stream 
    266289Encryption obfuscate peer-to-peer connections by employing RC4 
    267290encryption on every byte from the first byte transferred. BitTorrent 
    268 Protocol Encryption thus increases the difficulty for a device 
     291Message Stream Encryption thus increases the difficulty for a device 
    269292observing passing packets to identify BitTorrent peer-to-peer 
    270293connections.</p> 
    … …  
    280303<p>The obfuscation method meets the following criteria:</p> 
    281304<ul class="simple"> 
    282 <li>The entire plaintext of the peer list is not easily obtained 
    283 even if an eavesdropper identifies ip-port pairs from 
    284 subsequent connections from a peer that has received a tracker response.</li> 
     305<li>The entire plaintext of the peer list is not easily obtained even if 
     306an eavesdropper identifies ip-port pairs from subsequent connections 
     307initiated by a peer that has received a tracker response.</li> 
    285308<li>Even when a subsequent connection from a peer that has received a 
    286309tracker response is observed by an eavesdropper, it is difficult to 
    … …  
    297320<p>The objective is NOT to create a cryptographically secure protocol 
    298321that can survive unlimited observation of passing packets and 
    299 substantial computational resources on network timescales.  The object 
     322substantial computational resources on network timescales.  The objective 
    300323is to raise the bar sufficiently to deter attacks based on observing 
    301324ip-port numbers in peer-to-tracker communications.</p> 
    … …  
    303326and subsequent connections, it is possible to attack the encryption. 
    304327RC4 is known to have a number of weaknesses especially in the way it 
    305 was used with WEP [and HERE].  However, with tracker peer obfuscation, the 
    306 number of bytes transferred between the tracker and a client is likely 
    307 significantly smaller than transfer between a wireless computer and a 
    308 basestation.  An attacker faces a much larger task in obtaining 
    309 sufficient probable plaintext to directly break the encryption.</p> 
     328was used with WEP <a class="footnote-reference" href="#borisov" id="id3">[2]</a> <a class="footnote-reference" href="#scott" id="id4">[3]</a> <a class="footnote-reference" href="#stubblefeld" id="id5">[4]</a>.  However, 
     329with tracker peer obfuscation, the number of bytes transferred between 
     330the tracker and a client is likely significantly smaller than transferred 
     331between a wireless computer and a basestation.  An attacker faces a 
     332much larger task in obtaining sufficient probable plaintext to 
     333directly break the encryption.</p> 
    310334<p>Hobbling the RC4 encryption by using a bounded-length RC4 pseudorandom 
    311335string for small swarms is likely to have negilgible impact on 
    … …  
    333357small constant factor more expensive than a shuffle on an input string 
    334358of equal length, 2) the generated pseudorandom string is only <em>n</em> 
    335 bytes long where recommended <em>n</em> is within a small constant factor 
    336 larger than the largest response size and thus much smaller than the 
    337 tracker peer list for large swarms, and 3) the cost of the XOR 
    338 operation is lighter weight than performing a random shuffle.</p> 
     359ip-port pairs long where recommended <em>n</em> is within a small constant 
     360factor larger than the largest <em>returned peer list</em> and thus much 
     361smaller than the <em>tracker peer list</em> for large swarms, and 3) the cost 
     362of the XOR operation is lighter weight than performing a random 
     363shuffle.</p> 
     364</div> 
     365<div class="section" id="references"> 
     366<h1>References</h1> 
     367<table class="docutils footnote" frame="void" id="mse" rules="none"> 
     368<colgroup><col class="label" /><col /></colgroup> 
     369<tbody valign="top"> 
     370<tr><td class="label">[1]</td><td><em>(<a class="fn-backref" href="#id1">1</a>, <a class="fn-backref" href="#id2">2</a>)</em> BitTorrent Message Stream Encryption 
     371(<a class="reference external" href="http://www.azureuswiki.com/index.php/Message_Stream_Encryption">http://www.azureuswiki.com/index.php/Message_Stream_Encryption</a>)</td></tr> 
     372</tbody> 
     373</table> 
     374<table class="docutils footnote" frame="void" id="borisov" rules="none"> 
     375<colgroup><col class="label" /><col /></colgroup> 
     376<tbody valign="top"> 
     377<tr><td class="label"><a class="fn-backref" href="#id3">[2]</a></td><td>Nikita Borisov, Ian Goldberg, and David Wagner. Intercepting 
     378mobile communications: the insecurity of 802.11. In ACM MobiCom 2001, 
     379pages 180-189. ACM Press, 2001.</td></tr> 
     380</tbody> 
     381</table> 
     382<table class="docutils footnote" frame="void" id="scott" rules="none"> 
     383<colgroup><col class="label" /><col /></colgroup> 
     384<tbody valign="top"> 
     385<tr><td class="label"><a class="fn-backref" href="#id4">[3]</a></td><td>Scott R. Fluhrer, Itsik Mantin, and Adi 
     386Shamir. Weaknesses in the key scheduling algorithm of RC4. In Serge 
     387Vaudenay and Amr M. Youssef, editors, Selected Areas in 
     388Cryptography 2001, volume 2259 of Lecture Notes in Computer 
     389Science, pages 1-24. Springer, 2001.</td></tr> 
     390</tbody> 
     391</table> 
     392<table class="docutils footnote" frame="void" id="stubblefeld" rules="none"> 
     393<colgroup><col class="label" /><col /></colgroup> 
     394<tbody valign="top"> 
     395<tr><td class="label"><a class="fn-backref" href="#id5">[4]</a></td><td>Adam Stubblefeld, John Ioannidis, and Aviel 
     396D. Rubin. A key recovery attack on the 802.11b wired equivalent 
     397privacy protocol (WEP). ACM Transactions on Information and System 
     398Security, 7(2):319-332, May 2004.</td></tr> 
     399</tbody> 
     400</table> 
    339401<!-- Local Variables: 
    340402mode: indented-text 
    … …  
    344406coding: utf-8 
    345407End: --> 
    346 </div> 
    347 <div class="section" id="id2"> 
    348 <h1>References</h1> 
    349 <table class="docutils footnote" frame="void" id="id3" rules="none"> 
     408<table class="docutils footnote" frame="void" id="id6" rules="none"> 
    350409<colgroup><col class="label" /><col /></colgroup> 
    351410<tbody valign="top"> 
    352 <tr><td class="label"><a class="fn-backref" href="#id4">[1]</a></td><td><a class="reference external" href="http://tools.ietf.org/html/rfc2119">http://tools.ietf.org/html/rfc2119</a></td></tr> 
    353 </tbody> 
    354 </table> 
    355 <table class="docutils footnote" frame="void" id="id5" rules="none"> 
    356 <colgroup><col class="label" /><col /></colgroup> 
    357 <tbody valign="top"> 
    358 <tr><td class="label"><a class="fn-backref" href="#id6">[2]</a></td><td><a class="reference external" href="http://www.bittorrent.org/beps/bep_0013.html">http://www.bittorrent.org/beps/bep_0013.html</a></td></tr> 
     411<tr><td class="label"><a class="fn-backref" href="#id7">[5]</a></td><td><a class="reference external" href="http://tools.ietf.org/html/rfc2119">http://tools.ietf.org/html/rfc2119</a></td></tr> 
    359412</tbody> 
    360413</table>