Obsolete:Routing problems

From Wikitech

French squids

The French squids at 212.85.150.{131,132,133} are experiencing strange routing problems. Some people have absolutely no problem accessing them, some experience huge packet losses and delays. All problems seem to happen between the last hop (212.85.144.6 ) and the squids. These problems seem to be happening on the return routes:

1. router-wikipedia.tlcy.ielo.n  0.0%    10    0.5   0.4   0.3   0.6   0.1
2. feth2-globix.th2.fr.core.iel  0.0%    10    1.5   1.9   1.2   4.0   0.9
3. ge-1-2-0.403.ar2.AMS2.gblx.n 20.0%    10  2820. 2983. 2820. 3161. 113.6
4. so1-0-0-2488M.ar1.AMS1.gblx. 80.0%    10  3129. 3000. 2870. 3129. 182.6
5. SURFnet-3.ge-3-2-0.nar1.AMS1 85.7%     7  3125. 3125. 3125. 3125.   0.0
6. PO11-0.CR1.Amsterdam1.surf.n 50.0%     6  2948. 3025. 2948. 3087.  70.8

Most problems seem to start occuring once packets travel across glbx.net (Global Crossing) links.

Since non-French users have been taken off the Paris squids, the situation has come back to normal, with no abnormal ping times or delays.

EXPLANATION

Lost-Oasis, the French company that offers us free rack space, pays quite a lot for transit to some foreign networks. Since we're a non-paying customer, they have implemented a rate limiting filter to these destinations.

This explains while it's perfectly possible to ping from these destinations when these countries are not using the squids, and why things go weird once the squids are in use from these destinations.

Lost Oasis has been asked which destinations are affected and for what rate.

Networks for which there are no problems

  • Club Internet, France
  • Proxad [Free Telecom], France
  • RENATER [French universities, research centers etc.], France
  • Tiscali, France
  • Wanadoo, France
  • Arcor, Germany
  • NTL, UK
  • Blueyonder, UK
  • Easynet, UK

Networks experiencing problems

  • Enternet (Hungary)
  • Surfnet (Netherlands)
  • Deutsche Telekom (Germany)
  • QSC (Germany)
  • REDIRIS (Spain)
  • Telefonica (Spain)
  • MicroLink (Lithuania, Latvia, Estonia)
  • Ownit (Sweden), routed through above.net


Solved ? No !

Looks solved:

Microlink (AS5546) to Ennael (AS29075):

traceroute to ennael.wikimedia.org (212.85.150.131): 2-20 hops, 38 byte packets
 2  194.106.111.14                      ipt-gw.online.ee    15.8 ms (ttl=254)
 3  62.237.177.169           ge5-2.ltt-pe7.hel.fi.sn.net    9.48 ms (ttl=253)
 4  195.66.224.76            ge3-0.pr1.lhr1.uk.above.net    42.8 ms (ttl=248!)
 5  208.184.231.69         pos3-0.mpr1.lhr1.uk.above.net    57.6 ms (ttl=248!)
 6  208.184.231.178       so-5-3-0.cr1.lhr3.uk.above.net    47.5 ms (ttl=248!)
 7  64.125.31.130          pos12-0.cr1.cdg2.fr.above.net    64.8 ms (ttl=249)
 8  64.125.27.110          pos5-0.er1a.cdg2.fr.above.net    57.6 ms (ttl=249!)
 9  62.4.77.156           lostoasis-gw.cdg2.fr.above.net    55.9 ms (ttl= 59!)
10  212.85.144.6     feth0-bestelle.tlcy.fr.core.ielo.net    57.8 ms (ttl= 58!)
11  212.85.150.131                  ennael.wikimedia.org    57.7 ms (ttl= 57!)

Back way (AS29075 to AS5546):

Host                                  Loss%   Snt   Last   Avg  Best  Wrst StDev
1. router-wikipedia.tlcy.ielo.net      0.0%    77    0.3   1.7   0.3  10.7   2.3
2. feth0-muffin.rdb.fr.core.ielo.net   0.0%    77    0.9   1.9   0.7  10.2   1.9
3. 73.ge2-0.er1b.cdg2.fr.above.net     0.0%    77   25.7   2.5   0.8  25.7   3.8
4. pos1-0.cr2.cdg2.fr.above.net        0.0%    77   11.1  19.1  10.6 165.1  28.7
5. pos15-0.mpr1.ams1.nl.above.net      0.0%    76   18.7  23.1  10.5 202.4  36.2
6. pos0-0.mpr2.ams1.nl.above.net       0.0%    76   10.5  11.2  10.3  17.6   1.2
7. ipt-gw.online.ee                    6.6%    76   62.0  54.4  45.0  67.9   4.9

From RedIRIS (AS766) to AS29075:

traceroute to 212.85.150.131 (212.85.150.131), 30 hops max, 40 byte packets
 1  CAT.SO2-1-0.EB-IRIS2.red.rediris.es (130.206.240.9)  16.312 ms  16.313 ms  16.331 ms
 2  RI.SO0-0-0.EB-IRIS5.red.rediris.es (130.206.240.126)  16.544 ms  74.323 ms  47.514 ms
 3  so-0-3-0.ar1.mad1.gblx.net (64.214.70.33)  16.779 ms  16.617 ms  16.621 ms
 4  so4-0-0-2488M.ar2.CDG2.gblx.net (67.17.65.90)  32.124 ms  33.831 ms  32.148 ms
 5  Lost-Oasis.ge-1-2-0.403.ar2.AMS2.gblx.net (67.17.159.222)  35.617 ms  35.527 ms  37.298 ms
 6  feth0-bestelle.tlcy.fr.core.ielo.net (212.85.144.6)  39.841 ms  36.473 ms  35.912 ms
 7  ennael.wikimedia.org (212.85.150.131)  38.410 ms  36.722 ms  37.706 ms

Back trace from AS29075 to AS766:

Host                                  Loss%   Snt   Last   Avg  Best  Wrst StDev
1. router-wikipedia.tlcy.ielo.net      0.0%    11    5.7   2.6   0.3   8.8   3.3
2. feth2-globix.th2.fr.core.ielo.net   0.0%    11    1.5   2.6   1.2  11.1   2.9
3. ge-1-2-0.403.ar2.AMS2.gblx.net      0.0%    11    6.1   8.7   5.0  37.9   9.7
4. so2-0-0-2488M.ar1.MAD1.gblx.net     0.0%    11   20.7  21.5  20.3  26.1   1.7
5. Dante-STM-4.so-0-3-0.ar1.mad1.gblx  0.0%    11   30.9  32.0  20.7 112.1  26.8
6. CAT.SO2-1-0.EB-IRIS2.red.rediris.e  0.0%    10   20.9  24.6  20.8  41.2   6.7