Show
Ignore:
Timestamp:
05/19/2008 04:16:31 PM (6 months ago)
Author:
dave
Message:

bep 0022 cache discovery

Files:
1 modified

Legend:

Unmodified
Added
Removed

  • dotorg/trunk/html/beps/bep_0022.html

    r11084 r11087  
    3737<tr class="field"><th class="field-name">BEP:</th><td class="field-body">22</td> 
    3838</tr> 
    39 <tr class="field"><th class="field-name">Title:</th><td class="field-body">Cache Discovery</td> 
     39<tr class="field"><th class="field-name">Title:</th><td class="field-body">BitTorrent Cache Discovery Protocol</td> 
    4040</tr> 
    4141<tr class="field"><th class="field-name">Version:</th><td class="field-body">11083</td> 
    … …  
    4343<tr class="field"><th class="field-name">Last-Modified:</th><td class="field-body"><a class="reference external" href="http://bittorrent.org/trac/browser/dotorg/trunk/html/beps/bep_0022.rst">2008-05-13 17:34:56 -0700 (Tue, 13 May 2008)</a></td> 
    4444</tr> 
    45 <tr class="field"><th class="field-name">Author:</th><td class="field-body">David Harrison &lt;dave&#32;&#97;t&#32;bittorrent.com&gt;, Stanislav Shalunov &lt;shalunov&#32;&#97;t&#32;bittorrent.com&gt;</td> 
     45<tr class="field"><th class="field-name">Author:</th><td class="field-body">David Harrison &lt;dave&#32;&#97;t&#32;bittorrent.com&gt;, Stanislav Shalunov &lt;shalunov&#32;&#97;t&#32;bittorrent.com&gt;, Greg Hazel &lt;greg&#32;&#97;t&#32;bittorrent.com&gt;</td> 
    4646</tr> 
    4747<tr class="field"><th class="field-name">Status:</th><td class="field-body">Draft</td> 
    … …  
    6161<p class="topic-title first">Contents</p> 
    6262<ul class="simple"> 
    63 <li><a class="reference internal" href="#the-discovery-mechanism" id="id5">The Discovery Mechanism</a></li> 
    64 <li><a class="reference internal" href="#network-address-translators" id="id6">Network Address Translators</a></li> 
    65 <li><a class="reference internal" href="#references" id="id7">References</a></li> 
    66 <li><a class="reference internal" href="#copyright" id="id8">Copyright</a></li> 
     63<li><a class="reference internal" href="#motivation" id="id9">Motivation</a></li> 
     64<li><a class="reference internal" href="#the-discovery-mechanism" id="id10">The Discovery Mechanism</a></li> 
     65<li><a class="reference internal" href="#iterative-queries" id="id11">Iterative Queries</a></li> 
     66<li><a class="reference internal" href="#network-address-translators" id="id12">Network Address Translators</a></li> 
     67<li><a class="reference internal" href="#example" id="id13">Example</a></li> 
     68<li><a class="reference internal" href="#references" id="id14">References</a></li> 
     69<li><a class="reference internal" href="#copyright" id="id15">Copyright</a></li> 
    6770</ul> 
    6871</div> 
     72<div class="section" id="motivation"> 
     73<h1>Motivation</h1> 
     74<p>Some Internet Service Providers (ISPs) may be interested in deploying 
     75BitTorrent caches to lower transit costs, reduce internal traffic, and 
     76improve user experience by speeding up downloads.</p> 
     77<p>A cache is simply a fast peer in the middle of the network. It might 
     78also have substantial disk space. The client communicates with a cache 
     79using the normal BitTorrent protocol.</p> 
    6980<p>With this extension, BitTorrent clients are able to discover caches 
    7081nearby on the network.  When a cache is present, the user benefits 
    7182from having a high capacity peer from which the user's client 
    7283downloads and to which it can delegate seeding.  When a cache inside 
    73 the user's Internet Service Provider (ISP) network seeds on behalf of 
    74 the client, it frees upstream capacity in the user's access network 
    75 benefiting the user and those that share the access network.  When 
    76 subsequent peers transfer from their ISP's cache, the ISP experiences 
    77 less transit traffic.</p> 
     84the user's ISP network seeds on behalf of the client, it frees 
     85upstream capacity in the user's access network benefiting the user and 
     86those that share the access network.  When subsequent peers transfer 
     87from their ISP's cache, the ISP experiences less transit traffic.</p> 
     88</div> 
    7889<div class="section" id="the-discovery-mechanism"> 
    7990<h1>The Discovery Mechanism</h1> 
    8091<p>To find the caches for its ISP, a BitTorrent client performs a reverse 
    81 DNS lookup on its host's IP address and then obtains the SRV resource 
    82 record associated with the host's domain name.  For example, a host with 
    83 IPv4 address w.x.y.z obtains the PTR record at</p> 
    84 <dl class="docutils"> 
    85 <dt>::</dt> 
    86 <dd>z.y.x.w.in-addr.arpa</dd> 
    87 </dl> 
    88 <p>Assume the returned PTR record contains example.com.  The client then 
    89 looks up the SRV record at</p> 
    90 <dl class="docutils"> 
    91 <dt>::</dt> 
    92 <dd>_bittorrent._tcp.example.com</dd> 
    93 </dl> 
    94 <p>The target field in the SRV resource record contains the domain name 
    95 of the cache and the port specifies the location on that cache where 
    96 the BitTorrent implementation listens.</p> 
    97 <p>The SRV resource record type is described in <a class="reference external" href="http://www.faqs.org/rfcs/rfc2782.html">RFC 2782</a> <a class="footnote-reference" href="#rfc-2782" id="id1">[2]</a>.  Reverse DNS 
    98 lookups are described in <a class="reference external" href="http://www.faqs.org/rfcs/rfc1034.html">RFC 1034</a> <a class="footnote-reference" href="#rfc-1034" id="id2">[1]</a>.</p> 
     92DNS lookup on its external IP address and then obtains the BitTorrent 
     93SRV resource record associated with the host's domain name.  For 
     94example, a host with address 69.107.0.14 obtains the PTR record at</p> 
     95<pre class="literal-block"> 
     9614.0.107.69.in-addr.arpa 
     97</pre> 
     98<p>The client's host IP address may not match the host's IP address as 
     99seen outside the client's private network.  We address this in Section 
     100<a class="reference internal" href="#network-address-translators">Network Address Translators</a>.</p> 
     101<p>The PTR resource record returned for this example contains domain name</p> 
     102<pre class="literal-block"> 
     103adsl-69-107-0-14.dsl.pltn13.pacbell.net 
     104</pre> 
     105<p>The client then looks up the SRV records at</p> 
     106<pre class="literal-block"> 
     107_bittorrent-tracker._tcp.adsl-69-107-0-14.dsl.pltn13.pacbell.net 
     108</pre> 
     109<p>If no SRV record is found, one or more subsequent queries take place as 
     110described in <a class="reference internal" href="#iterative-queries">Iterative Queries</a>.</p> 
     111<p>The target field in each returned SRV resource record contains the 
     112domain name of a tracker and the port on which the tracker runs.  This 
     113tracker is called a <em>cache tracker</em>, but the protocol to talk to this 
     114tracker is no different from the standard BitTorrent tracker protocol 
     115described in <a class="footnote-reference" href="#bep-3" id="id1">[1]</a>.</p> 
     116<p>When the BitTorrent client joins a swarm it announces to one or more 
     117of the trackers referenced in the .torrent file and announces to the 
     118cache tracker.  The cache tracker returns peers which may be caches or 
     119other peers that announced the same file to the cache tracker.</p> 
     120<p>A cache is a BitTorrent peer.  A client MAY treat it preferentially.</p> 
     121<p>Reverse DNS lookups are described in <a class="reference external" href="http://www.faqs.org/rfcs/rfc1034.html">RFC 1034</a> <a class="footnote-reference" href="#rfc-1034" id="id2">[4]</a>. 
     122The SRV resource record type is described in <a class="reference external" href="http://www.faqs.org/rfcs/rfc2782.html">RFC 2782</a> <a class="footnote-reference" href="#rfc-2782" id="id3">[5]</a>.</p> 
     123</div> 
     124<div class="section" id="iterative-queries"> 
     125<h1>Iterative Queries</h1> 
     126<p>The domain name returned from the reverse DNS lookup is specific to 
     127the querying host.  In the naive implementation in DNS, there would be 
     128one SRV resource record for every querying host.  The most obvious 
     129solution is to use a wildcard of the form:</p> 
     130<pre class="literal-block"> 
     131*.pacbell.net 
     132</pre> 
     133<p>However, if wildcards are implemented according to the algorithm in 
     134section 4.3.2 in <a class="footnote-reference" href="#rfc-1034" id="id4">[4]</a> then all subdomains of pacbell.net that 
     135do not have an exact label match will match the wildcard.  Thus,</p> 
     136<pre class="literal-block"> 
     137_jabber._tcp.pacbell.net 
     138</pre> 
     139<p>and</p> 
     140<pre class="literal-block"> 
     141_bittorrent-tracker._tcp.pacbell.net 
     142</pre> 
     143<p>both match *.pacbell.net and all SRV resource records with owner 
     144*.pacbell.net would be returned with the name set to the name in the 
     145query.  Thus it would be impossible to disambiguate Jabber from 
     146BitTorrent SRV records without further information.  This behavior is 
     147implemented with BIND 9.4.1.</p> 
     148<p>The natural solution would be to specify domain names of the type</p> 
     149<pre class="literal-block"> 
     150_bittorrent-tracker._tcp.*.pacbell.net 
     151</pre> 
     152<p>However, section 4.3.3 in <a class="footnote-reference" href="#rfc-1034" id="id5">[4]</a> specifies that wildcards only 
     153appear as the first label in a domain name.  This restriction was 
     154lifted in <a class="footnote-reference" href="#rfc-4592" id="id6">[6]</a>, but not with semantics applicable to our use 
     155case.  An asterisk not at the beginning of a domain name is not 
     156treated like a wildcard.  Only a lookup for the exact domain name</p> 
     157<pre class="literal-block"> 
     158_bittorrent-tracker._tcp.*.pacbell.net 
     159</pre> 
     160<p>matches.</p> 
     161<p>We propose an alternative that avoids wildcards and has the advantage 
     162that it allows suborganizations to override the SRV records provided 
     163by parent organizations: the peer starts by querying using its 
     164fully-qualified domain name returned from the reverse DNS lookup, and 
     165if this fails then it queries again after removing the most specific 
     166(leftmost) label in the domain name.  For example, if no SRV records 
     167are returned when querying for</p> 
     168<pre class="literal-block"> 
     169_bittorrent-tracker._tcp.adsl-69-107-0-14.dsl.pltn13.pacbell.net 
     170</pre> 
     171<p>then the client queries for</p> 
     172<pre class="literal-block"> 
     173_bittorrent-tracker.dsl.pltn13.pacbell.net 
     174</pre> 
     175<p>and then</p> 
     176<pre class="literal-block"> 
     177_bittorrent-tracker.pltn13.pacbell.net 
     178</pre> 
     179<p>The search removes one domain at a time terminating when one or more 
     180resource records are found or before querying the root domain or 
     181top-level domains that are not ccTLDs, e.g., .com, .org, .net. We 
     182avoid querying the root or global top-level domains given the low 
     183likelihood that caches would be defined globally, and thus clients 
     184would unnecessarily burden the root domain name servers with queries 
     185generating negative results. We considered stopping before querying 
     186country-level domains, but a country providing public infrastructure 
     187might choose to provide caches.</p> 
    99188</div> 
    100189<div class="section" id="network-address-translators"> 
    … …  
    102191<p>Many hosts on the Internet sit in private networks that connect to the 
    103192Internet via a Network Address Translator (NAT).  Such hosts may have 
    104 an IP address allocated from one of the three private IP address 
    105 ranges defined by IANA, i.e., 10.*.*.*, 172.*.*.*, or 192.*.*.*.  When 
    106 communicating with hosts outside the private network, the NAT 
    107 translates the private IP to an IP address that is globally routable. 
    108 This globally routable address is the host's <em>public IP address</em>.</p> 
    109 <p>When finding a cache, the BitTorrent client must use its host's public 
    110 IP address.  A BitTorrent client can obtain its host's public IP by 
    111 either querying the tracker and looking at the value to the returned 
    112 <em>external ip</em> key <a class="footnote-reference" href="#bep-24" id="id3">[4]</a> or from other peers using the <em>yourip</em> 
    113 extension defined for the <em>Extension Protocol</em> proposed in <a class="footnote-reference" href="#bep-10" id="id4">[3]</a>.</p> 
     193an IP address allocated from one of the private IP address ranges 
     194defined by IANA, e.g., ranges with prefixes 10/8, 172.16/12, and 
     195192.168/16.  When communicating with hosts outside the private 
     196network, the NAT translates the private IP to a globally-routable IP 
     197address.  This globally-routable address is the host's <em>external IP 
     198address</em>.</p> 
     199<p>When finding a cache, the BitTorrent client must use its host's 
     200external IP address.  A BitTorrent client can obtain its host's 
     201external IP either from the <em>external ip</em> key returned from a tracker 
     202implementing BEP 24 <a class="footnote-reference" href="#bep-24" id="id7">[3]</a> or from peers using the <em>yourip</em> 
     203extension defined for the <em>Extension Protocol</em> proposed in <a class="footnote-reference" href="#bep-10" id="id8">[2]</a>.</p> 
     204</div> 
     205<div class="section" id="example"> 
     206<h1>Example</h1> 
     207<p>In our example, we used AT&amp;T's PacBell network.  AT&amp;T could implement 
     208cache discovery by adding the following lines to the zone file for 
     209pacbell.net,</p> 
     210<pre class="literal-block"> 
     211; name                                ttl  cls rr  pri weight port target 
     212_bittorrent-tracker._tcp.pacbell.net. 600  IN  SRV 5   0      6969 tracker 
     213</pre> 
     214<p>Now when a client performs cache discovery, it performs three DNS 
     215queries removing labels before reaching the domain name pacbell.net, 
     216at which point the SRV record would be returned and the client would 
     217then query tracker.pacbell.net to obtain caches.</p> 
     218<p>In Python, the cache tracker's port and domain can be obtained using 
     219PyDNS using the following code:</p> 
     220<pre class="literal-block"> 
     221import DNS 
     222 
     223tlds = [&quot;com&quot;, &quot;net&quot;, &quot;org&quot;]  # add more TLDs here. 
     224 
     225name = DNS.revlookup( &quot;69.107.0.14&quot; ) 
     226names = name.split('.') 
     227while names and names[0] not in tlds: 
     228   name = &quot;_bittorrent._tcp.&quot; + &quot;.&quot;.join(names) 
     229   req = DNS.Request( name=name, qtype=&quot;SRV&quot;, protocol=&quot;udp&quot;) 
     230   response = req.req() 
     231   if response.answers: 
     232      break 
     233   del names[0] 
     234 
     235print &quot;response=&quot;, response.show() 
     236</pre> 
     237<p>which might generate output like</p> 
     238<pre class="literal-block"> 
     239response= ; &lt;&lt;&gt;&gt; PDG.py 1.0 &lt;&lt;&gt;&gt; _bittorrent._tcp.pacbell.net SRV 
     240;; options: recurs 
     241;; got answer: 
     242;; -&gt;&gt;HEADER&lt;&lt;- opcode 0, status NOERROR, id 0 
     243;; flags: qr aa rd ra; Ques: 1, Ans: 1, Auth: 2, Addit: 3 
     244;; QUESTIONS: 
     245;;      _bittorrent._tcp.pacbell.net, type = SRV, class = IN 
     246 
     247;; ANSWERS: 
     248_bittorrent._tcp.pacbell.net    600    SRV     (5, 0, 6969, 'cache.pacbell.net') 
     249 
     250;; AUTHORITY RECORDS: 
     251pacbell.net             86400   NS      ns1.pbi.net 
     252pacbell.net             86400   NS      ns2.pbi.net 
     253 
     254;; ADDITIONAL RECORDS: 
     255cache.pacbell.net       86400   A       69.107.0.1 
     256ns1.pacbell.net         86400   A       206.13.28.11 
     257ns2.pacbell.net         86400   A       206.13.29.11 
     258 
     259;; Total query time: 0 msec 
     260;; To SERVER: localhost 
     261;; WHEN: Mon May 19 16:00:12 2008 
     262</pre> 
     263<p>The answer above is fictional since AT&amp;T does not at this time 
     264implement SRV records for BitTorrent trackers.</p> 
     265<p>In Microsoft Windows, the port and domain name of the server can be 
     266obtained using WinDNS (Dnsapi.lib) using DnsQuery().  In Unix, the 
     267relevant call is res_query() from libresolv.</p> 
    114268</div> 
    115269<div class="section" id="references"> 
    116270<h1>References</h1> 
     271<table class="docutils footnote" frame="void" id="bep-3" rules="none"> 
     272<colgroup><col class="label" /><col /></colgroup> 
     273<tbody valign="top"> 
     274<tr><td class="label"><a class="fn-backref" href="#id1">[1]</a></td><td>BEP_0003. The BitTorrent Protocol Specification, Cohen 
     275(<a class="reference external" href="http://www.bittorrent.org/beps/bep_0003.html">http://www.bittorrent.org/beps/bep_0003.html</a>)</td></tr> 
     276</tbody> 
     277</table> 
     278<table class="docutils footnote" frame="void" id="bep-10" rules="none"> 
     279<colgroup><col class="label" /><col /></colgroup> 
     280<tbody valign="top"> 
     281<tr><td class="label"><a class="fn-backref" href="#id8">[2]</a></td><td>BEP_0010.  Extension Protocol. Norberg, Strigeus, Hazel 
     282(<a class="reference external" href="http://www.bittorrent.org/beps/bep_0010.html">http://www.bittorrent.org/beps/bep_0010.html</a>)</td></tr> 
     283</tbody> 
     284</table> 
     285<table class="docutils footnote" frame="void" id="bep-24" rules="none"> 
     286<colgroup><col class="label" /><col /></colgroup> 
     287<tbody valign="top"> 
     288<tr><td class="label"><a class="fn-backref" href="#id7">[3]</a></td><td>BEP_0024.  Tracker Returns External IP.  Harrison 
     289(<a class="reference external" href="http://www.bittorrent.org/beps/bep_0024.html">http://www.bittorrent.org/beps/bep_0024.html</a>)</td></tr> 
     290</tbody> 
     291</table> 
    117292<table class="docutils footnote" frame="void" id="rfc-1034" rules="none"> 
    118293<colgroup><col class="label" /><col /></colgroup> 
    119294<tbody valign="top"> 
    120 <tr><td class="label"><a class="fn-backref" href="#id2">[1]</a></td><td><a class="reference external" href="http://www.faqs.org/rfcs/rfc1034.html">RFC-1034</a>.  DOMAIN NAMES - CONCEPTS AND FACILITIES. Mockapetris, 
     295<tr><td class="label">[4]</td><td><em>(<a class="fn-backref" href="#id2">1</a>, <a class="fn-backref" href="#id4">2</a>, <a class="fn-backref" href="#id5">3</a>)</em> <a class="reference external" href="http://www.faqs.org/rfcs/rfc1034.html">RFC-1034</a>.  DOMAIN NAMES - CONCEPTS AND FACILITIES. Mockapetris, 
    121296November 1987. (<a class="reference external" href="http://tools.ietf.org/html/rfc1034">http://tools.ietf.org/html/rfc1034</a>)</td></tr> 
    122297</tbody> 
    … …  
    125300<colgroup><col class="label" /><col /></colgroup> 
    126301<tbody valign="top"> 
    127 <tr><td class="label"><a class="fn-backref" href="#id1">[2]</a></td><td><a class="reference external" href="http://www.faqs.org/rfcs/rfc2782.html">RFC-2782</a>.  A DNS RR for specifying the location of services (DNS 
     302<tr><td class="label"><a class="fn-backref" href="#id3">[5]</a></td><td><a class="reference external" href="http://www.faqs.org/rfcs/rfc2782.html">RFC-2782</a>.  A DNS RR for specifying the location of services (DNS 
    128303SRV). Gulbrandsen, Vixie, Esibov. February 2000. 
    129304(<a class="reference external" href="http://tools.ietf.org/html/rfc2782">http://tools.ietf.org/html/rfc2782</a>)</td></tr> 
    130305</tbody> 
    131306</table> 
    132 <table class="docutils footnote" frame="void" id="bep-10" rules="none"> 
    133 <colgroup><col class="label" /><col /></colgroup> 
    134 <tbody valign="top"> 
    135 <tr><td class="label"><a class="fn-backref" href="#id4">[3]</a></td><td>BEP_0010.  Extension Protocol. Norberg, Strigeus, Hazel 
    136 (<a class="reference external" href="http://www.bittorrent.org/beps/bep_0010.html">http://www.bittorrent.org/beps/bep_0010.html</a>)</td></tr> 
    137 </tbody> 
    138 </table> 
    139 <table class="docutils footnote" frame="void" id="bep-24" rules="none"> 
    140 <colgroup><col class="label" /><col /></colgroup> 
    141 <tbody valign="top"> 
    142 <tr><td class="label"><a class="fn-backref" href="#id3">[4]</a></td><td>BEP_0024.  Tracker Returns External IP.  Harrison 
    143 (<a class="reference external" href="http://www.bittorrent.org/beps/bep_0024.html">http://www.bittorrent.org/beps/bep_0024.html</a>)</td></tr> 
     307<table class="docutils footnote" frame="void" id="rfc-4592" rules="none"> 
     308<colgroup><col class="label" /><col /></colgroup> 
     309<tbody valign="top"> 
     310<tr><td class="label"><a class="fn-backref" href="#id6">[6]</a></td><td><a class="reference external" href="http://www.faqs.org/rfcs/rfc4592.html">RFC-4592</a>. The Role of Wildcards in the Domain Name System. Lewis 
     311(<a class="reference external" href="http://www.faqs.org/rfcs/rfc4592.html">http://www.faqs.org/rfcs/rfc4592.html</a>)</td></tr> 
    144312</tbody> 
    145313</table>