Reporting a connectivity issue

These instructions also available at https://wikitech-static.wikimedia.org/wiki/Reporting_a_connectivity_issue for folks who are not able to access Wikitech for some reason. When linking these instructions to those who are experiencing connectivity trouble, always link them the wikitech-static version.

Instructions on how to gather useful data and report a problem connecting to a Wikimedia website like en.wikipedia.org.

Your own Internet

First, if you're unsure if the issue is related to your Internet connection, you can for example use http://test-ipv6.com/ to verify it (even if you don't use IPv6). If the issue is related to slowness, tools like http://www.speedtest.net/ or https://fast.com are also useful.

Wikimedia's websites

If you're having difficulties reaching any of the Wikimedia websites, check www.wikimediastatus.net to see if there's a system-wide outage or other issue affecting all or many Wikimedia systems.

If you can connect to https://www.wikimedia.org but not to other Wikimedia sites, then your issue is probably not a connectivity issue.

If there's no evidence of a system-wide outage, gather the following information as part of your report to help the Site Reliability Engineering team to troubleshot the issue.

The more exhaustive the answers are, the easier it will be for us to help you.

Email the following details to nocwikimedia.org (or alternatively create a new security task (to ensure your details remain private) on Phabricator and add the "netops" and "Traffic" tags via the bottom of the page "Add Action > Change Project Tags" dropdown, or reach out on the #wikimedia-tech ^connect channel on the libera.chat IRC network):

What is the issue in your own words?
Is the issue intermittent or constant?
When did the issue start?
What device are you using? (laptop, mobile phone, etc)
What network are you connected to? Home/office/mobile? Name? Country/City?
What is the output of http://test-ipv6.com/helpdesk/ ?
What is your public IP address? (See link above)
Provide the output of a command line test request toward the problematic url using cURL:
- Getting curl:
  - Many linux computers will have curl installed already. Use your package manager to install it if necessary.
  - cURL for Windows can be downloaded from the open source project if you are running a build under Insider Build 17063, but is installed by default in cmd and Powershell on builds above (if you are on the most recent build, you most likely already have curl installed.
  - cURL is installed by default on Mac computers.
  - On a phone or tablet, search your app store for a curl app or other http debugging tool that will show the full request and response.
- From a command line shell, run curl -v <url>
  - For example: curl -v https://en.wikipedia.org/wiki/Main_Page
Run the same cURL command for the non-HTTPS version of the website:
- For example, curl -v http://en.wikipedia.org/wiki/Main_Page
Provide the output of a traceroute from your computer to the problematic hostname:
- Linux users:
  - Gather a TCP traceroute. In a terminal run traceroute --tcp --port=443 <hostname>. For example: traceroute --tcp --port=443 en.wikipedia.org
- Mac users:
  - In a terminal run traceroute <hostname>. For example: traceroute en.wikipedia.org
- Both Linux and Mac users:
  - The tool mtr can provide useful information as well (on Mac prepend sudo to the command): mtr --report-wide --show-ips --aslookup --tcp --port 443 en.wikipedia.org
- Windows users:
  - In a command line shell, run tracert <hostname>. For example: tracert en.wikipedia.org
- Android, iOS, other:
  - Search your app store for a "traceroute" app
Provide the output of a ping from your computer to the problematic hostname:
- Linux and Mac users:
  - In a command line shell, run ping -c 4 <hostname>. For example: ping -c 4 en.wikipedia.org
- Windows users:
  - In a command line shell, run ping <hostname>. For example: ping en.wikipedia.org
- Android, iOS, other:
  - Search your app store for a "ping" app

Additional troubleshooting

Test connectivity to multiple Wikimedia Point of Presence (PoP) locations

Re-run the tests above, using the following information:

Site	hostname (for traceroute, ping, etc)	curl example
ulsfo	text-lb.ulsfo.wikimedia.org	`curl -v https://text-lb.ulsfo.wikimedia.org --insecure -H 'Host: www.wikimedia.org'`
eqiad	text-lb.eqiad.wikimedia.org	`curl -v https://text-lb.eqiad.wikimedia.org --insecure -H 'Host: www.wikimedia.org'`
codfw	text-lb.codfw.wikimedia.org	`curl -v https://text-lb.codfw.wikimedia.org --insecure -H 'Host: www.wikimedia.org'`
esams	text-lb.esams.wikimedia.org	`curl -v https://text-lb.esams.wikimedia.org --insecure -H 'Host: www.wikimedia.org'`
eqsin	text-lb.eqsin.wikimedia.org	`curl -v https://text-lb.eqsin.wikimedia.org --insecure -H 'Host: www.wikimedia.org'`
drmrs	text-lb.drmrs.wikimedia.org	`curl -v https://text-lb.drmrs.wikimedia.org --insecure -H 'Host: www.wikimedia.org'`

Test IP fragmentation issues

IPv4 version: http://icmpcheck.popcount.org
IPv6 version: http://icmpcheckv6.popcount.org

Show latency for various parts of an exchange

Example with esams IPs:

curl -w @- -o /dev/null --resolve www.wikipedia.org:443:91.198.174.192 -s https://www.wikipedia.org <<'EOF'
    time_namelookup:  %{time_namelookup}\n
       time_connect:  %{time_connect}\n
    time_appconnect:  %{time_appconnect}\n
   time_pretransfer:  %{time_pretransfer}\n
      time_redirect:  %{time_redirect}\n
 time_starttransfer:  %{time_starttransfer}\n
                    ----------\n
         time_total:  %{time_total}\n
EOF

curl -w @- -o /dev/null --resolve www.wikipedia.org:443:2620:0:862:ed1a::1 -s https://www.wikipedia.org <<'EOF'
    time_namelookup:  %{time_namelookup}\n
       time_connect:  %{time_connect}\n
    time_appconnect:  %{time_appconnect}\n
   time_pretransfer:  %{time_pretransfer}\n
      time_redirect:  %{time_redirect}\n
 time_starttransfer:  %{time_starttransfer}\n
                    ----------\n
         time_total:  %{time_total}\n
EOF