IPv6-only hosting in 2020

February 28th, 2020 by

It’s now nearly five years since we started offering IPv6-only hosting, and what started out as a source of interesting projects for enthusiastic early-adopters has become our default for most hosting requirements.

A few things have changed over the years that have made this possible:

  • The death of Windows XP, the last significant OS with a browser that didn’t support SNI (Server Name Indication). SNI makes it possible for us to proxy encrypted connections to IPv6-only hosts.
  • The widespread adoption of secure services. This means that protocols that don’t have their own proxying features (such as POP3 or IMAP) can be proxied in their encrypted form thanks to SNI.
  • Improvements to our hosting services, such as our SSH port forwarder.

This post gives a quick run-down of how we make IPv6-only hosting a reality.

Getting bytes in

There’s no getting away from the fact that an IPv6-only hosting server still needs to be able to talk to IPv4-only clients, but there’s now a good solution for doing so for pretty much all common scenarios.

Web traffic

This is the most common requirement, and also probably the easiest, as it can be handled by our v4 to v6 proxy.  The proxy is a set of servers with both IPv4 and IPv6 addresses that accept traffic for various protocols and forward it to an IPv6-only server.

The DNS for the hosted site points at our proxy servers, by means of either an ANAME or CNAME record to proxy.mythic-beasts.com.

Unencrypted HTTP traffic is easy to proxy as HTTP 1.1 is designed to support multiple websites on a single IP address.

HTTPS is also easy to proxy, thanks to the now-ubiquitous support for SNI (its successor, ESNI, may complicate this a bit in the future, but we’ll tackle that in a separate post).

Our proxy also supports PROXY protocol, which is a standard way of communicating the original client’s IP address on a proxied connection. Support for PROXY protocol is now a standard feature of NGINX and Apache.

IPv6 traffic can either follow the same route as IPv4 traffic through the proxy (as shown above) or can be routed directly to the hosting server by setting the AAAA records for the site to point at the server rather than the proxy:

This provides a slightly more direct route for IPv6 traffic, but can make the configuration on the server a little more complicated, particularly if you’re using PROXY protocol.

IMAP and POP3

These can both be proxied in their secure forms (IMAPS and POP3S) thanks to SNI, and thankfully these secure variants are now the default choice for all popular email clients.

SSH

Our customers typically want to administer their servers via SSH, and can’t guarantee that they’ll always be connecting from a v6-enabled network. The SSH protocol isn’t built on TLS/SSL so doesn’t have SNI support, and doesn’t have any equivalent features of its own.

We work around this by providing a port-forward to all virtual servers and Raspberry Pi servers from a host with a v4 IP address, so customers can make a connection to a different host on a non-standard port, and the connection will be forwarded to the IPv6 server on port 22. Details of the host and port can be found in our customer control panel.

SMTP

SMTP is a bit awkward. It’s used in two common scenarios:

  1. “Submission”, where an end-user client sends outgoing mail using authenticated SMTP
  2. Server-to-server delivery of email.

It has multiple ports in common use:

  • 25 – the standard port for server-to-server email
  • 465 – a port for SMTP over SSL
  • 587 – the standard SMTP submission port

Port 25 doesn’t use SSL/TLS at connection time, but can be upgraded to a secure connection via the STARTTLS command, which means it can’t be proxied using SNI.

Port 465 has a confused history, having been allocated by IANA for secure SMTP, then revoked in favour of STARTTLS and allocated to a different service, and then reinstated for secure SMTP submission by RFC 8314.  Port 465 is supported by our proxy, and is a good choice for SMTP submission.

Port 587 was historically plain SMTP (RFC 2476) with STARTTLS, but is being migrated to SSL by default (RFC 8314) which is proxyable thanks to SNI.  Our proxy assumes that port 587 traffic is encrypted (because it can’t do anything useful if it’s not) and as such can also be used for SMTP submission, provided you use SSL/TLS rather than STARTTLS.

For server-to-server delivery, it’s possible to use our dual-stack MX servers to handle incoming mail. This can be done by having the highest priority MX record point to the v6-only server, and then have a lower priority record
pointing to our MX servers. v4-only servers will deliver to our MX servers, and we’ll then pass it on to your v6-only server.

This isn’t a perfect solution, as it means you can’t do connection-time filtering of incoming mail.

Our MX servers need to be configured to accept mail for your domain. At present, this needs to be done by emailing support.

Getting bytes out

Your server may need to make outgoing connections to v4-only servers. Fortunately this is straightforward using our NAT64 resolvers. These are DNS resolvers that when asked for an address for a host that does not have any AAAA records will provide an IPv6 address that is mapped to the host’s v4 address. The v6 address is actually an address on one of our NAT servers that will then forward the traffic to the v4 address.

There’s a 1:1 mapping between v4 addresses and v6 addresses on the NAT server – with IPv6 we can easily allocate the equivalent of the full 32-bit IPv4 address space to a single server!

NAT64 works very well in almost all cases. We have come across a few bits of software which explicitly request an A record when doing a DNS lookup, which obviously doesn’t work.

As with any NAT configuration, you’re sharing a v4 address with other users, which can cause issues for sites that perform IP-based filtering or rate limiting.

Make the switch

Like most providers, we now charge for IPv4 addresses, but unlike most other providers it’s a tax you probably don’t need pay. We offer IPv6-only versions of all of our virtual and dedicated servers, and our Raspberry Pi servers area all IPv6-only.

Learn more

If you’d like to hear more, here are some videos of a presentation that Pete gave at the UK Network Operators Forum (UKNOF).

Working with talented people.

February 14th, 2020 by

You can buy another copy in a bookshop if your cat refuses to return the one you already own.


We like working with talented people be they staff, customers or suppliers. That’s why we give discounts to people who can navigate our jobs challenge even if they don’t want to work for us.

Occasionally we’ve drafted in Gytha Lodge to help us copy write various articles and turn a jumble of thoughts into a coherent and interesting article.

Formerly an aspiring author, her full title is now Richard and Judy book club pick and Sunday Times bestselling author, Gytha Lodge.

We’re also pleased to report that she took our advice on her first book seriously and the new book starts with a murder being watched over a webcam.

Security in DNS, TLSA records now available in our control panel to support DANE

February 11th, 2020 by

The Internet is better when it’s secure. Finally, thanks to Let’s Encrypt it’s possible to automatically get SSL certificates free of charge and as a result the Internet is dramatically more secure than it used to be. If you’ve used our DNS API you may have discovered that you can verify Let’s Encrypt SSL certificate requests using DNS records, including issuing wildcard certificates.

We support secure DNS (DNSSEC) which prevents DNS records from being forged, making the process of authenticating your SSL certificate through DNS records far more secure than the email-based authentication that was typically used for certificates issued by commercial certificate authorities. We have implemented support for CAA records which uses DNS to restrict the certificate authorities that can issue your certificates. This is most useful if the DNS is trustworthy which, again, requires DNSSEC.

However, there seems to be an opportunity here to improve things further. Rather than relying on a 3rd party certificate authority to confirm that you have control of your own DNS, why can’t you just publish your certificate in DNS directly? If you can trust DNS this would seem to be an obvious improvement, and with DNSSEC, DNS becomes trustworthy. We’ve now added support for the additional record type TLSA which allows exactly that, as part of DNS-Based Authentication of Named Entities (DANE).

Adding a TLSA record through our control panel.

DANE is a flexible mechanism that can be used to add an additional layer of security to certificates issued by a 3rd party authority, or to enable the use of self-signed certificates.

Unfortunately at the moment few clients support TLSA so for the majority of interactions you’re still going to rely on the certificate authority to verify the certificate. But implementations exist for both Exim and Postfix. Step by step, email is becoming more secure.

Two Factor Auth – TOTP now available

January 27th, 2020 by

Good security practice requires two different factors.

We’ve just rolled out a much requested feature to our control panel: Timed One Time Passwords or TOTP.

TOTP is a type of 2FA. If these acronyms are making sense to you, head over to the control panel and set up TOTP.

If not, read-on…

What is 2FA?

You’ll probably have noticed an increasing number of websites that you use encouraging or requiring you to enable “two factor authentication” or 2FA.

2FA refers to requiring two separate things to confirm your identity: something you know (your password) and something you have (e.g. your phone).

2FA protects against some of the most common ways in which accounts get compromised:

  • Username/password re-use. Despite advice not to do so, plenty of people re-use passwords across lots of different sites. Every now and again, sites get compromised, and databases of usernames and passwords become available on the shadier parts of the internet. These credentials will then be tried against other sites, looking for places that they’ve been re-used.
  • Email account compromise. If your email account is compromised, it’s very easy for an attacker to gain access to your other accounts, as it’s almost always possible to reset your password by sending an email.
  • Key-logging. If your computer is compromised, or you use an untrusted shared computer, key-logging malware may be installed that logs your password as you type it to log into your account.

2FA protects against all of these. It’s no longer sufficient to know the username and password to login, and you can’t reset your password just by having access to the email account. 2FA uses “one time passcodes” which means that whilst they can be captured by a key-logger, they’re of no value as they’ve already been used.

TOTP, SMS and Recovery Codes

We now support three different methods to provide the second factor: SMS, TOTP and recovery codes. With a Timed One Time Password your phone uses a secret key and the current time to generate a unique six digit code. The code is only valid for a short period, and can only be used once. The code proves that you have access to the secret key in the phone, but does not require you to send the secret key or any part of it to us.

With SMS we send you a time-limited, one-time code via a text message. Your phone collects this and you can type it in during login to prove that you’re holding your phone.

Recovery codes are intended to be a fall back should you lose access to your primary 2FA method. These are a set of one time codes that you can store securely (e.g. on paper, in a safe) and use each of them for a single login as required.

TOTP has a number of advantages over SMS. Firstly, it’s entirely offline on your phone so that if you’re somewhere with no phone signal you can still log in. Secondly, it doesn’t rely on trusting the mobile phone network; anyone with access to the phone network could intercept your SMS code or arrange for it to be delivered to another device. Similarly you may have things like message sharing enabled which means that your passcode is delivered to multiple devices.

Setting up TOTP

TOTP is very easy to setup. You’ll need an app on your phone. You can use Google Authenticator, but we prefer the open source FreeOTP. Once installed, go to the two-factor auth page in our control panel and hit the big green “Enable TOTP” button.

You’ll be shown a QR code which you can scan into the app on your phone, and you can then start generating codes. You need to enter a code to confirm that it’s set up correctly, and you can then choose to require 2FA whenever you log into your account.

Whilst you’re there, you should take the chance to print off some recovery codes.

IPv6 updates

December 16th, 2019 by

Last Thursday we went to the IPv6 Council to speak about IPv6-only hosting and to exchange information with other networks about the state of IPv6 in the UK.

IPv4 address exhaustion is becoming ever more real: the USA and Europe have now run out, and Asia, Africa and Latin America all have less than a year of highly-restricted supply left.

Perhaps unsurprisingly, we’re now seeing real progress in deploying IPv6 across the board.

The major connectivity providers gave an update on their progress. Sky already have an effectively complete deployment across their UK network, so instead they told us about the Sky Italia build-out that launches early next year. It will initially be 100% dual stack but they’re planning to migrate to single stack IPv6 with IPv4 access provided by MAP-T as soon as possible. BT/EE have IPv6 virtually everywhere and take-up is rising as HomeHubs are retired and replaced with SmartHubs. Three are actively enabling IPv6 over their network, as we noticed last month:

Smaller providers are also making progress; Hyperoptic and Community Fibre have both essentially completed their dual stack rollout this year, with both organisations having to consider Network Address Translation due to lack of IPv4 addresses.

We’ve been working hard for many years to make IPv6-only hosting a practical option for our customers, allowing us to considerably expand the lifespan of our IPv4 allocation (which, thanks to a few acquisitions and being a relatively old company, is a reasonable size).

We heard from Ungliech, who started more recently and don’t have a large historical allocation of IPv4 addresses. They gave an interesting talk about their IPv6-only hosting and how it’s an urgent requirement for a new entrant because a RIPE final allocation of 1024 addresses isn’t enough to start a traditional hosting company. Thanks to RIPE running out last month, any new competitor has it four times harder with only 256 addresses to get them started.

We also had interesting updates from Microsoft about their continuing journey to IPv6-only internally in their corporate network, and the pain of continuing to support IPv4 private addressing. When they acquire a company they already have overlapping internal networks, and making internal services available to the wider organisation is an ongoing difficult challenge.

There was also a fascinating talk from SITA about providing network and infrastructure to aviation. There is a huge amount of networking involved and the RFC1918 private IPv4 address space is no longer large enough to enable a large airport. They have a very strong push to use IPv6 even on networks not connected to the public internet.

Updates to sympl to continue to support Let’s Encrypt

October 25th, 2019 by

Before you 3D print the keys from the photo, you should know they are no longer in use.

We’ve now updated Sympl to support the new ACME v2 protocol for long term Let’s Encrypt support.

Let’s Encrypt is changing the protocol for obtaining and renewing certificates from ACME v1, to ACME v2 and the version 1 protocol is now end-of-life. In the next few days (1st November) this means that new accounts will no longer be able to be registered which will prevent new sites obtaining SSL certificates. Final end of life occurs in 2021 when certificate renewals will start to generate errors and then fail entirely.

Symbiosis is now end of life, as Sympl is an actively developed fork we’d recommend any Symbiosis users migrate to Sympl. We’d also recommend our managed hosting as a good place to run your Sympl server.

Multiple Mythic Beasts staff members contributed to this update.

Let’s Encrypt support for older Debian

October 9th, 2019 by
seure cat

This cat is secure, but not dehydrated. (Credit Lizzie Charlton, @LizzieCharlton

Debian Jessie and Debian Stretch include dehydrated, a useful command line tool for managing Let’s Encrypt certificates. We use it fairly extensively for managing certificates throughout our servers and with our managed customers. Unfortunately due to a change in capitalisation at Let’s Encrypt, the standard copy of dehydrated shipped with Debian Jessie and Debian Stretch is no longer compatible. As there’s no package in backports, we’ve spun our own packages of a newer version of dehydrated which is available on our mirror server.

If you use the older version you’ll see an error like


{
"type": "urn:acme:error:badNonce",
"detail": "JWS has no anti-replay nonce",
"status": 400
}

or


{
“type”: “urn:ietf:params:acme:error:malformed”,
“detail”: “Malformed account ID in KeyID header URL: “https://acme-v02.api.letsencrypt.org/acme/acct/””,
“status”: 400
}

The fix is very simple, you just need to install our dehydrated packages. This is very easy to do.

First add our signing keys


wget -O - -q https://mirror.mythic-beasts.com/mythic/support@mythic-beasts.com.gpg.key | apt-key add -

Then the correct repository based on your version of Debian

echo deb http://packages.mythic-beasts.com/mythic/ jessie main >/etc/apt/sources.list.d/packages.mythic-beasts.com.list

or

echo deb http://packages.mythic-beasts.com/mythic/ stretch main >/etc/apt/sources.list.d/packages.mythic-beasts.com.list

then

apt-get update
apt-get install --only-upgrade dehydrated
dehydrated -c

and your copy of dehydrated will be updated to 0.6 and your certificates can be created as normal.

Sympl fixes potential GDPR compliance issue in Symbiosis

July 29th, 2019 by

IP addresses may leak private information about the entity using your service.

A bug in the Symbiosis hosting management platform means that by default, the IP addresses of some website visitors are publicly accessible. This is potentially sensitive information, and critically, as IP addresses are considered ‘personal data’ under GDPR, this means that the default configuration of Symbiosis is not GDPR-compliant.

This bug is due to incorrect handling of the automatic web statistic generation flag in Symbiosis, which results in full statistics being enabled by default on all sites even if no access restrictions are in place. Existing statistics will persist even when statistics are disabled.

This issue has been addressed in Sympl, an actively-maintained fork of Symbiosis that focuses on security and usability. In Sympl, web statistics are disabled by default, and a password must be set to access them via a browser. While this is one of the most serious of the security issues from Symbiosis which have been fixed in Sympl, it is unfortunately not the only one.

For an immediate fix, we recommend users migrate to Sympl. This can be done by provisioning a new server running Debian Stretch or Debian Buster, and installing Sympl then migrating their content across to the new server.

GDPR compliance is a serious issue, with the potential for very substantial fines (up to 4% of annual global turnover or €20 million – whichever is greater), and recent cases have demonstrated that the ICO is prepared to impose such fines

For more information on what constitutes personal data under GDPR, please see the Information Commissioner’s Office website.

Introducing Sympl

July 9th, 2019 by

Unfortunately Sympl doesn’t include easy to manage graphic designers.

Hot on the heels of the Debian Buster release, we’re pleased to announce our first release of Sympl, an open-source hosting management platform for Debian.

What is Sympl?

Sympl is easiest to explain by example.

Want to create a secure website for https://example.com?

Simply create a directory:

mkdir -p /srv/example.com/public/htdocs

That’s it. Point the DNS at your server and start uploading your content. An SSL certificate will be obtained automatically from Let’s Encrypt.

Want to create a new mailbox for Brian? Simply create a directory:

mkdir /srv/example.com/mailboxes/brian

Your server now accepts mail for brian@example.com.

Mail is accessible using webmail, or using any device via secure IMAP/SSL.

Configuration is all done over SSH, so you gain all the security advantages of a highly locked down server, with much easier configuration management.

Works with you, not against you

Unlike other solutions, which take an all-or-nothing approach to managing your server, Sympl happily accepts you customising the configuration and will avoid overwriting any configuration files that you alter.

When it writes configurations for you, Sympl automatically picks best practice options. This includes things like limiting permissions for PHP, secure connections for web and email, and of course, IPv6 support throughout. It’s built on Debian Linux and runs on our dedicated servers, virtual servers and we also build the packages for the Raspberry Pi.

Sympl is 100% open source. It’s completely free to use, irrespective of the number of servers or domains you might want to use with it.

Installing Sympl

If you have a Mythic Beasts virtual server running Debian Buster you can install Sympl easily by using the install script:

wget https://gitlab.mythic-beasts.com/sympl/install/raw/master/install.sh
bash install.sh

If you want a managed Sympl server, we’ll do this for you as part of the setup.

Server management

Sympl pairs well with our managed hosting service. We monitor your server 24/7, apply security updates and take a daily backup leaving you to manage the sites running on it.

Future plans

Future plans for Sympl include automatic DNS configuration using OctoDNS, which supports a wide range of DNS providers, updated Let’s Encrypt support allowing automatic wildcard SSL certificates, and a fully functional command line parser for day to day administration tasks.

Find out more info on Sympl at sympl.host, which is (of course) hosted using Sympl.

VMHaus services now available in Amsterdam

July 3rd, 2019 by

Integration can be hard work

Last year we had a busy time acquiring Retrosnub, BHost and VMHaus. We’ve been steadily making progress in the background integrating the services the companies provide to reduce costs and complexity of management. We can now also announce our first significant feature upgrade for VMHaus. We’ve deployed a new virtual server cluster to our Amsterdam location and VMHaus services are now available in Amsterdam. VMHaus is using Mythic Beasts for colocation and network and in Amsterdam they will gain access to our extensive set of peers at AMSIX, LINX and LoNAP. Per hour billed virtual servers are available from VMHaus with payment through Paypal.

As you’d expect, every VM comes with a /64 of IPv6 space.

In the background we’ve also been migrating former-BHost KVM-based services to Mythic Beasts VM services in Amsterdam. Shortly we’ll be starting to migrate former-BHost and VMHaus KVM-based services in London to new VM clusters in the Meridian Gate data centre.