tldr; Apple say IPv6 support vital, we offer £20pa off any VM that is IPv6 only.

Apple have announced that as of iOS 9, all apps require support for IPv6 and must run on an IPv6 only network. The motivation is fairly clear, T-Mobile USA runs an IPv6 only network, Comcast is deploying IPv6 and on IPv6 launch day in Finland 5% of users had IPv6 enabled simultaneously. Google sees around 7% of all traffic as IPv6 now.

Like it or not, IPv6 is here and the predictions of a lengthy period of being dual stack were wrong. Nobody bothered to turn on IPv6 until IPv4 ran out, then instead of IPv6 and Network Address Translation we’re skipping quickly to IPv6 only. If your application doesn’t work on an IPv6 only network an increasing fraction of users simply can’t use it.

At Mythic Beasts we’ve been using IPv6 for a long time. Two years ago we rebuilt the hosting infrastructure for Raspberry PI to be IPv6 only for all internal connections. A future article will explain our scale up to vastly more VMs, many IPv6 only. IPv6 at Mythic Beasts isn’t an add-on, if our IPv6 connectivity breaks, customers go offline. We’re steadily working on spreading IPv6 connectivity throughout other providers.

We’ve been offering developers IPv6 only Virtual Machines for experimentation for a while, and have one of the most comprehensive IPv6 connectivity checkers for hosted software which is very good at demonstrating that enabling a v6 address isn’t quite enough.

Every single connection to this website uses IPv6.

The best way to build the hosting infrastructure today, is to have an IPv6 only network for the whole thing and a single IPv4 address on the load balancer for ‘legacy’ IPv4 connections. To give everyone an incentive to do it right, today we’re extending our IPv6 only VM offer – all virtual machines that are IPv6 only will be discounted for the lifetime of the rental.

If you’re really interested, this presentation at the North American Operators Group about the largest US ISPs moving straight to IPv6 only deployments including the information that over 20% of US users have native IPv6.

Recently Mythic Beasts went to the first meeting of the UK IPv6 Council, a non profit group to assist in rolling out IPv6 across the UK. There was a rapid exchange of knowledge, ideas and progress between organisations.

We heard from network engineers within BT, BSkyB and Virgin Media covering well over half of all the end users in the UK. BT and Virgin have enough IPv4 addresses not to require rolling out IPv6, BSkyB don’t and therefore need to either implement IPv6 or Carrier Grade NAT (CGN), and they really don’t like CGN. Virgin are having portions of their address space taken by other parts of the parent company so may also need IPv6 or CGN. They too don’t like CGN, and already have IPv6 support in all their SuperHubs, even if the functionality is currently disabled. All three companies have IPv6 support in various levels of trial with internal staff members running dual stack. However, all three have plans to roll out customer trials in the first part of 2015.

We also heard from the Belgian IPv6 council about how roll-out in Belgium occurred to nearly 30% of all end users having native IPv6, they went from less than 1% in May 2013, to 16% in May 2014, and 27% now. Once a couple of their large providers started enabling IPv6 the roll-out was very fast. It’s likely the same thing could happen in the UK with three major providers having significant IPv6 plans within the next 12 months.

As an idea of how quickly things might happen, T-Mobile USA has gone from nowhere to the 10th largest IPv6 deployment with 44% of their network IPv6 enabled within 12 months.

So at a guess, Mythic Beasts think that IPv6 rollout in the UK by December 2015, will be either less than 1%, about 25% or roughly 50%. We aren’t sure which, but we think it’s wise to be prepared for every eventuality. To help you with that we have an IPv6 health checker.

We were asked on Twitter to add a page describing exactly how to get 10/10 on our IPv6 domain readiness checker, so here it is.

1. Add an IPv6 address to your web server

The first step is to get your web server listening on an IPv6 address, as well as an IPv4 address. How you achieve this will depend on how your web server is managed. If you’re on a shared hosting account, you’ll be dependent on your hosting provider. If you run your own server, you’ll need to obtain an IPv6 address from your hosting provider (assuming they support IPv6), configure your server to use it and then ensure that your web server (e.g. Apache is listening on this address).

If you’re using a standard Mythic Beasts hosting account, this step is already done for you. If you’re using a sphinx shell account, please contact support.

2. Add an AAAA record for your website

AAAA records are the IPv6 equivalent of A records, which resolve hostnames to IP addresses.  In order for users to find your website over IPv6, you will need to add an AAAA record for www.yourdomain.com pointing to the IPv6 address configured above.  You can check that this is in place using the dig command:

$ dig +short A www.mythic-beasts.com
93.93.131.39

$ dig +short AAAA www.mythic-beasts.com
2a00:1098:0:86:1000::15

It’s possible that your existing “www” record will be a CNAME for another hostname, in which case you should add the AAAA record to that hostname, rather than the “www” record.

If you’re using a Mythic Beasts hosting account, then provided that you are using the standard template for your account in the DNS configuration for your domain, then this step will already be done for you.

Our health checker will skip this test if your domain doesn’t have an A record for “www”.

3. Add an AAAA record for your bare domain

Most websites are configured to work if the user omits the “www” prefix from the name, for example http://mythic-beasts.com

In order for this to work, you will need an A record for your domain name itself, and to be IPv6-enabled, you’ll also need a corresponding AAAA record.

Once again, our checker will skip this test if the bare domain doesn’t have an A record.

4. Ensure your DNS servers have IPv6 addresses

The steps above make it possible to access your website over IPv6, but unless your DNS servers are accessible over IPv6, users (or more specifically, their DNS resolvers) will still need to use IPv4 in order to find your site in the first place. To avoid this, you need to ensure that your nameservers have IPv6 addresses.

You can find the nameservers for your domain using “whois”, and you can check whether the servers have IPv6 addresses using dig, as before:

$ whois mythic-beasts.com
[ ... ]
Name Server: NS0.BEASTS.ORG
Name Server: NS3.MYTHIC-BEASTS.COM
Name Server: NS2.MYTHIC-BEASTS.COM
Name Server: NS0.MYTHIC-BEASTS.COM
Name Server: NS1.MYTHIC-BEASTS.COM
[ ... ]
$ dig +short AAAA ns1.mythic-beasts.com
2600:3c00::f03c:91ff:fe96:beac

If your nameservers do not have IPv6 addresses, then unless you run your own nameservers, you’ll either need to persuade your hosting provider to enable IPv6, or switch your DNS provider to a different provider.

Mythic Beasts customers should be using ns1 and ns2.mythic-beasts.com, and both of these nameservers have IPv6 addresses, and IPv6 glue (see below).

For a full pass, our health checker requires that at least two of your servers have IPv6 addresses.

5. Add IPv6 glue for your nameservers, if necessary

In order to find the address for your website, a DNS resolver will first need to find the address of your nameservers. If your nameservers are in your own domain, this creates a bootstrapping problem. For example, in order to find the address for ns1.mythic-beasts.com, you need to ask the nameservers for mythic-beasts.com, which includes ns1.mythic-beasts.com. The solution to this is a glue record, a record containing the address of your nameserver which is held by the nameserver for the next zone up. In this case, the next zone up is .com, so the .com nameservers would contain glue records for the ns*.mythic-beasts.com nameservers.

If a nameserver has an IPv6 address, then any glue records for it should also contain that IPv6 address.

Checking for glue records is a little bit involved. The quickest way to do it is to use “dig +trace” to find a nameserver for the next zone up:

$ dig +trace ns1.mythic-beasts.com
[...]
com.      172800  IN  NS  a.gtld-servers.net.
com.      172800  IN  NS  b.gtld-servers.net.
com.      172800  IN  NS  c.gtld-servers.net.
[...]

We can now ask any of those servers for the NS records for our domain. Any glue records that exist will be returned in the “additional” section of the response:

$ $ dig NS mythic-beasts.com @a.gtld-servers.net.
[...]
;; ADDITIONAL SECTION:
ns1.mythic-beasts.com.  172800  IN  AAAA  2600:3c00::f03c:91ff:fe96:beac
ns1.mythic-beasts.com.  172800  IN  A 69.56.173.190
ns2.mythic-beasts.com.  172800  IN  AAAA  2a00:1098:0:80:1000::10
ns2.mythic-beasts.com.  172800  IN  A 93.93.128.67
[...]

If your servers are missing glue records, you will need to get your domain registrar to add them.

It’s worth noting that even if you don’t directly require glue because your nameservers are in a different zone, at some point along the chain there will be a nameserver that does require glue.

For a full pass, our glue checker requires at least two nameservers to be discoverable by a single-stack IPv6 resolver at every step of the chain of delegation.

6. Add IPv6 addresses for your incoming mail servers

In order to receive mail over IPv6, at least some of the mail servers listed in the MX records for your domain must have IPv6 addresses. You can find the mail servers for your domain using dig:

$ dig +short MX mythic-beasts.com
10 mx1.mythic-beasts.com.
10 mx2.mythic-beasts.com.

You can then check that these servers have IPv6 address by using dig to resolve an AAAA record, as before.

Once again, if you’re using a Mythic Beasts hosting account using the standard DNS template for your account type, this will be done for you.

In order to pass this test, at least one of the servers listed in your MX records must have an IPv6 address.

7. Add reverse DNS for your mail servers’ IPv6 address

It is generally advisable to have working reverse DNS for any addresses from which you send outgoing mail. In the case of IPv6, this becomes pretty much essential, as one of the biggest mail providers in the world, Google, will reject mail over IPv6 unless the sending server has working reverse DNS for its IPv6 address.

Unless you run your own mail servers, adding support for IPv6 will be down to your mail provider.

Unfortunately, there is no reliable way to obtain the outgoing mail servers that are used for a particular domain, so instead our health check makes a bold assumption that your outgoing servers are the same as the incoming servers listed in your MX records, and checks those. This assumption is certainly not true of all domains, which is why a failure of this test is only treated as a warning.

8. Check your SPF records

SPF (Sender Policy Framework) is a mechanism for publicly listing your outgoing mail servers, so that receivers can detect spoofed email sent from other servers. If you enable your outgoing mail servers to start sending mail over IPv6, and you have an existing SPF record, it is important that you make sure that it includes the IPv6 addresses for your mailservers.

There are various ways of doing this. If your incoming and outgoing mail servers are the same, then you can use the “mx” mechanism in your SPF record. This means that any hosts listed in the MX records for your domains will be regarded as a legitimate source of mail for your domain, and this will automatically include any IPv6 addresses (assuming you’ve done step 6).

If you list IPv4 addresses or address ranges in your SPF record explicitly, then you will need to add corresponding IPv6 addresses for those servers.

The rules applied by our health checker aren’t entirely trivial, as it’s not uncommon for legitimate third party servers to be included in a domain’s SPF record, and there’s no way of pairing up IPv6 addresses with their IPv4 counter parts. Our health checker applies some very broad rules: if you use the “mx” mechanism, then the checker requires at least one IPv6 address for a server listed in the relevant MX records. If there are any explicit “ip4” addresses or ranges specified in the record, then the health checker expects to find at least one explicit “ip6” mechanism.

If your domain does not list an SPF record then this test will pass automatically, as this effectively defaults to “accept from all”.

These rules aren’t watertight, but have proven to be quite effective in identifying mail sources that either haven’t been enabled by IPv6, or which have but haven’t been added to the SPF record.

Feedback

Hopefully this gives some insight into exactly what our domain tester is looking for, and how to comply with it. Customers who are having trouble getting 10/10 should email support. We’re also always very happy to hear any feedback on the health check, particularly suggestions for additional tests that we should be running.

We’ve just given our IPv6 Health Check a significant overhaul.

IPv6 is coming, and there’s a lot more to being ready for it than adding an AAAA record and enabling IPv6 on your web server. Google are often cited as an example of an IPv6-enabled website, but the truth is, in an IPv6-only world, nobody would ever find Google because none of their nameservers have IPv6 addresses:

Our IPv6 Health Check aims to reach the parts that other testers don’t reach to find out if your domain is really ready for IPv6.  Do your mail servers have IPv6 addresses?  Do they have reverse DNS for those addresses?  Do your SPF records include IPv6 addresses?

The latest version of our health check includes an experimental IPv6 nameserver delegation and glue test that checks that all nameservers the delegation chain between the root DNS servers and your zone have IPv6 addresses, and sufficient glue to allow you to find them in an IPv6-only world. This test has already uncovered some interesting anomalies which we’ll dig into further in a future post.

Nat Morris, UK Network Operators Forum director recently gave a presentation to DNS Operations, Analysis and Research centre, which included this remarkably nice slide:

What is Anycast?

Normally a server has a globally unique IP address, and the Internet knows how to send traffic from any other machine in the world to that IP address. With Anycast we share a single address across multiple machines, and your traffic is sent to the nearest machine with that address. This means that UK customers can be answered from a server in the UK and Australian customers from a server in Australia allowing you to have very fast responses to things like DNS queries because you’re always served by a server that’s close by, rather than your query having to travel half way around the world.

To set up an Anycast network, you need your own address space, your own network number (ASN), multiple BGP-aware routers that can announce your address space, and multiple servers that can answer the queries. Typically this would require a pretty hefty budget, but if you’re Nat Morris and you know what you’re doing with software routing on Linux, and you know all the right providers then you can bring up a global Anycast network with 10+ servers and sites on an annual budget of well under $1,000.

The key to doing this is finding ISPs, ideally well-connnected ISPs in key internet hubs, who will provide you with a BGP feed to your hosted server. That’s where a UK clueful hosting company comes into the picture having excellent connectivity, inexpensive virtual machines (VMs) and a willingness to support customers with more unusual configurations.

Quick introduction to BGP and routing

Normally when you have a VM you get a default route, which looks like this:

# ip route 
...
default via 93.93.128.1 dev eth1 

which says that to get to anywhere on the internet, send packets to our router at 93.93.128.1.

Over BGP, instead we send you the whole routing table:

# ip route 
...
1.0.7.0/24 via 5.57.80.128 dev eth3.4  proto zebra  metric 1 
1.0.20.0/23 via 93.93.133.46 dev eth6.220  proto zebra  metric 142080 
... 
500,000 more lines like this

For every block on the whole internet you have a different gateway depending on what you’ve decided is the preferred route. At today’s count this is about 490,000 entries in the routing table. Don’t type ‘route’ if you’re logged in over 3G!

So for this VM, instead of having a default route, Nat has four full BGP sessions, two to each of our two routers to the site. On each router, one session provides 490,000 IPv4 routes, the other provides 18,000 IPv6 routes, and the VM gets to decide which router to send data to.

The other side of the BGP relationship, and the important bit for Anycasting, is that we receive an advert from Nat’s VM for his /24 of IPv4 space and /48 of IPv6 space, which we then advertise out to the world. The 10+ other providers in this Anycast setup will do the same, and hosts will direct traffic to whichever is nearest.

Filtering

As Paul Vixie pointed out in the first question to Nat, the main customers of VMs with BGP are spammers who hijack address space for nefarious usage. At Mythic Beasts we filter our announcements and our customer routes, so if Nat messes up his configuration and accidentally announces that his VM is responsible for the whole of Youtube we’ll drop the announcement rather than expecting one very small VM to handle one fifth of the internet.

BGP on a virtual or dedicated server

If you’re a DNS provider or a content delivery network, you’ll probably want to have an Anycast setup at some point. At Mythic Beasts we remember what it was like to be the little guy which is why we offer full BGP routing (including IPv6 BGP) as an option to any virtual server, dedicated server, colocated server or router. Providing you own your own ASN and IP space we can transit it for you and we can keep the start-up costs very low and scale with you. You can locate your VM or server directly with us in Telecity, mere tens of metres from LINX and LoNAP for minimal latency and maximal available bandwidth.

If you’ve no idea what an ASN, BGP, LIR, RIPE are, we can help arrange your ASN, IP space and BGP config.

Some months ago we announced that we were planning an even cheaper version of our entry-level VPS Lite virtual servers.

It took a little longer than planned, but we’re now pleased to announce the launch of our IPv6-only VPS Lite, for the ludicrously low price of £5+VAT/month (or even less if you pay annually). This virtual server comes with 4 billion IPv6 addresses, but no IPv4 connectivity.

The world is running out of IPv4 addresses, and whilst we’ve got an allocation that isn’t going to run out any time soon, issuing an IPv4 address with every server we sell will ultimately be a barrier to our growth. If you’re happy to have a server without an IPv4 address, we’re happy to give you a discount, and that’s exactly what we’ve done with our IPv6-only VPS Lite.

IPv6 usage is currently running at about 3% and doubling roughly annually, so now is a great time to get familiar with IPv6.

Whilst these servers probably aren’t suitable for fronting a website just yet, we’ve already got a number of customers who run IPv6-only backend networks, with everything behind their load balancers on single-stack IPv6.

Although these Virtual Servers themselves don’t have IPv4 connectivity, the host server does, which means that you can get to the admin console, virtual serial console and VNC over an IPv4 connection.

If you’re a technology professional and have no idea what any of this means, we suggest you start your training by placing an order here.

You can now configure reverse DNS for IPv6 through our customer control panel. If you’ve previously been handling reverse DNS for your allocation through delegation and would prefer to use the control panel, then please get in touch.

If you’ve got a server with us and are interested in trying IPv6 and don’t already have an allocation then please email support and we’ll be happy to provide you with a block of addresses.

Our network has supported IPv6 for a while, but recently we’ve been making a concerted effort to enable IPv6 on more of our servers. What we’ve learned (mostly the hard way) is that the challenge in doing this is not so much in enabling specific services, such as making your webserver speak IPv6, but in the less obvious side effects of bringing up an IPv6 address on the server in question. Once you do this, the server will start making outgoing connections over IPv6 where possible, and that’s when you find out all the places that you’ve got IP-based access controls squirreled away.

One that caught us out recently when we brought up IPv6 addresses on our mail servers was an SPF record that listed our outgoing servers by their IP (v4) addresses. In hindsight, including IP addresses in an SPF record was never a great idea. It would be much better to use the “mx” or “a” SPF terms, referring to mail servers by name rather than address.

To help others avoid making the same mistake, we’ve added SPF record checking to our IPv6 Health Check. The rules on this are necessarily a bit arbitrary: if you have an explicit reference to an IPv4 address, it expects you to have at least one reference to an IPv6 address. In addition, any time that you use an MX term, it expects that MX to have both IPv4 and IPv6 addresses.

For an example of this, compare the results for twitter.com with the results for google.com. We fail twitter.com because of the “mx:one.textdrive.com” term. There are other parts of Twitter’s SPF that don’t appear to have IPv6 equivalents (e.g. “_netblocks.zdsys.com”) but there’s no easy way to determine which IPv6 address block corresponds to each IPv4 address block. Suggestions for better ways to categorise these test results gratefully received.

A few days ago we unveiled our IPv6 Health Check tool, and it very quickly proved its worth.

There are plenty of other IPv6 website checkers already out there that do a cursory check to make sure that you have some IPv6 addresses for your website, nameservers, and mail servers. Our checker attempts to dig a little deeper. Do your webservers actually respond over IPv6? On all addresses? Do all your MXs have working IPv6 reverse DNS? Are your DNS entries dependent on other zones that don’t have IPv6 nameservers?

One user pointed us towards the results for one of the Regional Internet Registries, initially because it broke the checker. A few bug-fixes (both in our code, and in CPAN modules) later, and we’d determined that there was a real problem behind it: www had two AAAA records, both servers were up, but only one was accepting connections on its IPv6 address. Connections to the other server eventually timed out.

Although this issue would cause real problems to an IPv6-only user, this is exactly the kind of problem that the Happy Eyeballs (aka Fast Fallback) algorithm does a perfect job of masking. If you pick the duff IPv6 server out of the DNS, it’ll almost immediately and silently fall back on an IPv4 server. Even if you use a tool like SixOrNot to show you what connection got used, it may not be obvious that something is amiss, as falling back to IPv4 becomes part of normal operation.

Even in a dual-stack world, such a problem isn’t without side effects, as it would likely lead to an imbalance in the load spread between the two web servers.

We’re continuing to broaden the range of tests performed by the tool in order to help catch the less obvious problems that can occur when IPv6-enabling your site.

Ever wondered whether users would get to your website in an IPv6-only world? Well, now you can find out. Our IPv6 Health Check tool checks not only that your web and mail servers are accessible via IPv6 addresses, but also that you can obtain those addresses from DNS using IPv6.

You can try it out by entering a domain name below:

Should you care? Well, typing domains for a few well-known websites into the checker reveals that the IPv6 Internet isn’t currently a very exciting place, so it’s going to be a little while before not having an IPv6 presence becomes a problem. On the other hand, there’s a growing number of users with both IPv4 and IPv6 connectivity, with the latter being preferred. Google recently announced that over 2% of their traffic was now over IPv6¹, so if you’re going to list IPv6 addresses for your servers then it’s important that they work.

1. Although you still need an IPv4 connection to find Google in the first place.