We recently announced our new DNS API which we’ve just moved out of beta and into production. 

One of the goals of the new API was better support for automating DNS-based challenges, such as those used by Let’s Encrypt to authenticate certificate requests. 

DNS-based challenges are needed to obtain wildcard certificates from Let’s Encrypt, and can be a convenient way to get certificates for hostnames that don’t a have publicly accessible web server, but can be tricky to implement due to delays in updating DNS records, and automatic requires having credentials capable of DNS records for your domain stored on your server.

The new API has a number of features to address these issues.

Restricted credentials

The DNS API allows you to create API credentials that are restricted to editing specific records within your domain.  Credentials can be restricted by hostname, record type, or both.

For example, you can create credentials that can only edit the _acme-challenge TXT record needed for Let’s Encrypt challenges. Access to the DNS API is potentially very sensitive, so it makes sense to limit access as much as possible.

Record verification

Updates made via the API do not become live immediately. There is a delay of up to a minute before they hit our master nameserver, and a potential further delay of a few seconds before the record propagates to our authoritative nameservers. When responding to a DNS-based challenge, you will typically want to ensure that the record is actually live before proceeding with verification.

Our DNS API provides a “verify” feature, that checks that records are live on all authoritative nameservers. For example, a GET request to the following URL would check that the nameservers have the latest update to the record:

https://api.mythic-beasts.com/dns/v2/zones/example.com/records/_acme-challenge/TXT?verify

This will return a 200 response if the nameservers are up-to-date, and 409 if they are not. This can be used to script a check after updating a record:

#!/bin/sh

ZONE=example.com
RECORD=_acme-challenge
TYPE=TXT

for i in $(seq 1 12); do
    RES=$(curl -n https://api.mythic-beasts.com/dns/v2/zones/$ZONE/records/$RECORD/$TYPE?verify -qs -w '%{http_code}' -o /dev/null)
    case $RES in
        200)    echo Records updated
                exit 0
                ;;
        409)    echo "Not yet updated ($i/12)"
                ;;
        *)      echo "Unexpected error: $RES"
                exit 1
                ;;
    esac
    sleep 10
done
echo Timed out
exit 2

Obtaining certificates the easy way

Our preferred Let’s Encrypt client is the excellent dehydrated, and we maintain a hook script for supporting DNS-based challenges in dehydrated. We haven’t yet updated the hook script to support our new API, but will be doing so soon and will post details here when it’s ready.

We’ve just launched our new DNS API, which provides a much cleaner and more powerful API for automatic management of DNS records.

Key features of the new API include:

• Configurable auth credentials – restrict access to individual records, or record types.  Ideal for Let’s Encrypt or other DNS-based challenges.
• Choice of JSON or zone file format for input and output.
• Atomic multi-record updates – update arbitrary sets of records in a single transaction.
• Form parameters for record creation – record creation can be trivially scripted using curl.
• Broad record type support – CAA, SSHFP, TLSA, SRV and many more.

For a walk through of the features of the new API, please see our DNS API tutorial , or for more details see the reference documentation.

To get started with the API, use the API keys section of the control panel to create some credentials.

Restricted API credentials for Let’s Encrypt challenges

The new API is currently in public beta, meaning that we reserve the right to make last minute breaking changes to the API, although we expect any such changes to be minor, and we would very much like to hear any feedback you may have.

Our forthcoming new DNS API seeks to make automating common DNS management tasks as simple as possible. It supports both JSON and “zone file format” for inputs and outputs, which opens up some interesting options for making bulk updates to a DNS zone.

Bulk TTL change

Suppose we want to drop the TTL on all of our zone’s MX records. We could get the records in zone file format and edit them using sed:

curl -n -H 'Accept: text/dns' https://api.mythic-beasts.com/dns/zones/example.com/records/@/MX \ 
| sed -E 's/^([^ ]+) +([0-9]+)/\1 60/' \
| curl -n -H 'Content-Type: text/dns' -X PUT --data-binary @- https://api.mythic-beasts.com/dns/zones/example.com/records/@/MX

Alternatively, we could get the zone in JSON format and modify it using the awesome command line JSON-wrangler, jq:

curl -n -H 'Accept: application/json' https://api.mythic-beasts.com/dns/zones/example.com/records/@/MX \
| jq '.records[].ttl = 60' \
| curl -n -H 'Content-Type: application/json' -X PUT --data-binary @- https://api.mythic-beasts.com/dns/zones/example.com/records/@/MX

How does this work?

The most common response to seeing the second example is “cool, I didn’t know you could do that with jq!”. So what’s going on here?

The first line gets us the the MX records for example.com (the “@” means fetch records for the bare domain). This gives us some JSON that looks like this:

{
    "records": [
        {
            "data": "mx1.mythic-beasts.com.",
            "host": "@",
            "mx_priority": 10,
            "ttl": 300,
            "type": "MX"
        },
        {
            "data": "mx2.mythic-beasts.com.",
            "host": "@",
            "mx_priority": 10,
            "ttl": 300,
            "type": "MX"
        }
    ]
}

The next line modifies this JSON. jq allows you to select values within the JSON. .records selects the “records” property of the response. [] gets us all of the members of the array, and .ttl selects the “ttl” property of each one. On its own, this would extract the values from the JSON, and give us “300 300”, but jq also allows us to modify the property by assigning to it. This gives us the following JSON:

{
    "records": [
        {
            "data": "mx1.mythic-beasts.com.",
            "host": "@",
            "mx_priority": 10,
            "ttl": 60,
            "type": "MX"
        },
        {
            "data": "mx2.mythic-beasts.com.",
            "host": "@",
            "mx_priority": 10,
            "ttl": 60,
            "type": "MX"
        }
    ]
}

We can now pipe this into a PUT request back to the same URL. This will replace all of the records that the first request selected with our new records.

The first approach works in a very similar way, except rather than working on a block of JSON with jq, we use sed to modify the following zone file formatted text:

@                      300 MX    10 mx1.mythic-beasts.com.
@                      300 MX    10 mx2.mythic-beasts.com.

Choose your weapon

We think that the JSON based approach is neater, but let us know what you think on our Twitter poll. We’d also be interested to hear of specific DNS management operations that you’d like to automate, so that we can see how they’d be tackled in our new API.

We’ve got a new version of our DNS API under development.  One of the neat new features is the ability to accept input in bind zone file format (aka RFC 1035). 

One of the things that this makes very easy is adding SSHFP records to the DNS. SSHFP is a mechanism for lodging your server’s public SSH keys in the DNS so that your SSH client can automatically verify a server the first time you connect to it, rather than prompting you to confirm the host key.

Using our new API, you can add or update SSHFP keys by piping the output of ssh-keygen straight into curl:

ssh-keygen -r myhost | curl -X PUT -n https://api.mythic-beasts.com/zones/example.com/myhost/SSHFP -H 'Content-Type: text/dns' --data-binary @-

What’s going on here?

ssh-keygen -r outputs your server’s SSH public keys in RFC 1035 format:

$ ssh-keygen -r myhost
myhost IN SSHFP 1 1 e579ff6aabc2f0acf714deca53108a0c1ea7d799
myhost IN SSHFP 1 2 7c47d5dfb748ff1fd244b7289d815e83dad8c2c1652b92ac8aed8ff166733d07
myhost IN SSHFP 2 1 c5caf4cc8870acc7fd113e5a7c866822ec0d94de
myhost IN SSHFP 2 2 9f11843fa1d9da318aa4bc09bbcaacaf4a9868c4d83dfc4bad6853d0c9597a31
myhost IN SSHFP 3 1 eb8644f5fcfd555341f2063bd92044075e20da89
myhost IN SSHFP 3 2 60f3e9780f9b87e5b4d6344f2ab46decbf705123e96ef07c3247f714ca220fc4
myhost IN SSHFP 4 1 139426de48381ea46ad75dde4e412bf1c9b11e61
myhost IN SSHFP 4 2 6f094181b510bbb573048835665773eb1a2a65fd4341d95207479ed71296491b

We then pipe that into curl to make a request to the DNS API.

A PUT request to the /myhost/SSHFP endpoint replaces all existing myhost SSHFP records.

-n tells curl to get auth credentials from a .netrc file, and the “Content-Type” header tells our API that we’re providing the new records in zone file format.

What’s the point of SSHFP?

Having lodged these SSHFP records in the DNS, and provided that DNSSEC is enabled for your domain, it’s possible to connect to a server without being prompted to verify the server’s host key.

$ ssh -o VerifyHostKeyDNS=yes root@myhost.example.com 

You can avoid having to specify the -o by putting VerifyHostKeyDNS=yes in your ~/.ssh/config file.

Feedback wanted

The new DNS API isn’t quite ready for deployment, so if there are any features that you’d really like to see in the new API, now’s the time to tell us, either on Twitter or by email.

It’s now nearly five years since we started offering IPv6-only hosting, and what started out as a source of interesting projects for enthusiastic early-adopters has become our default for most hosting requirements.

A few things have changed over the years that have made this possible:

• The death of Windows XP, the last significant OS with a browser that didn’t support SNI (Server Name Indication). SNI makes it possible for us to proxy encrypted connections to IPv6-only hosts.
• The widespread adoption of secure services. This means that protocols that don’t have their own proxying features (such as POP3 or IMAP) can be proxied in their encrypted form thanks to SNI.
• Improvements to our hosting services, such as our SSH port forwarder.

This post gives a quick run-down of how we make IPv6-only hosting a reality.

Getting bytes in

There’s no getting away from the fact that an IPv6-only hosting server still needs to be able to talk to IPv4-only clients, but there’s now a good solution for doing so for pretty much all common scenarios.

Web traffic

This is the most common requirement, and also probably the easiest, as it can be handled by our v4 to v6 proxy.  The proxy is a set of servers with both IPv4 and IPv6 addresses that accept traffic for various protocols and forward it to an IPv6-only server.

The DNS for the hosted site points at our proxy servers, by means of either an ANAME or CNAME record to proxy.mythic-beasts.com.

Unencrypted HTTP traffic is easy to proxy as HTTP 1.1 is designed to support multiple websites on a single IP address.

HTTPS is also easy to proxy, thanks to the now-ubiquitous support for SNI (its successor, ESNI, may complicate this a bit in the future, but we’ll tackle that in a separate post).

Our proxy also supports PROXY protocol, which is a standard way of communicating the original client’s IP address on a proxied connection. Support for PROXY protocol is now a standard feature of NGINX and Apache.

IPv6 traffic can either follow the same route as IPv4 traffic through the proxy (as shown above) or can be routed directly to the hosting server by setting the AAAA records for the site to point at the server rather than the proxy:

This provides a slightly more direct route for IPv6 traffic, but can make the configuration on the server a little more complicated, particularly if you’re using PROXY protocol.

IMAP and POP3

These can both be proxied in their secure forms (IMAPS and POP3S) thanks to SNI, and thankfully these secure variants are now the default choice for all popular email clients.

SSH

Our customers typically want to administer their servers via SSH, and can’t guarantee that they’ll always be connecting from a v6-enabled network. The SSH protocol isn’t built on TLS/SSL so doesn’t have SNI support, and doesn’t have any equivalent features of its own.

We work around this by providing a port-forward to all virtual servers and Raspberry Pi servers from a host with a v4 IP address, so customers can make a connection to a different host on a non-standard port, and the connection will be forwarded to the IPv6 server on port 22. Details of the host and port can be found in our customer control panel.

SMTP

SMTP is a bit awkward. It’s used in two common scenarios:

1. “Submission”, where an end-user client sends outgoing mail using authenticated SMTP
2. Server-to-server delivery of email.

It has multiple ports in common use:

• 25 – the standard port for server-to-server email
• 465 – a port for SMTP over SSL
• 587 – the standard SMTP submission port

Port 25 doesn’t use SSL/TLS at connection time, but can be upgraded to a secure connection via the STARTTLS command, which means it can’t be proxied using SNI.

Port 465 has a confused history, having been allocated by IANA for secure SMTP, then revoked in favour of STARTTLS and allocated to a different service, and then reinstated for secure SMTP submission by RFC 8314.  Port 465 is supported by our proxy, and is a good choice for SMTP submission.

Port 587 was historically plain SMTP (RFC 2476) with STARTTLS, but is being migrated to SSL by default (RFC 8314) which is proxyable thanks to SNI.  Our proxy assumes that port 587 traffic is encrypted (because it can’t do anything useful if it’s not) and as such can also be used for SMTP submission, provided you use SSL/TLS rather than STARTTLS.

For server-to-server delivery, it’s possible to use our dual-stack MX servers to handle incoming mail. This can be done by having the highest priority MX record point to the v6-only server, and then have a lower priority record
pointing to our MX servers. v4-only servers will deliver to our MX servers, and we’ll then pass it on to your v6-only server.

This isn’t a perfect solution, as it means you can’t do connection-time filtering of incoming mail.

Our MX servers need to be configured to accept mail for your domain. At present, this needs to be done by emailing support.

Getting bytes out

Your server may need to make outgoing connections to v4-only servers. Fortunately this is straightforward using our NAT64 resolvers. These are DNS resolvers that when asked for an address for a host that does not have any AAAA records will provide an IPv6 address that is mapped to the host’s v4 address. The v6 address is actually an address on one of our NAT servers that will then forward the traffic to the v4 address.

There’s a 1:1 mapping between v4 addresses and v6 addresses on the NAT server – with IPv6 we can easily allocate the equivalent of the full 32-bit IPv4 address space to a single server!

NAT64 works very well in almost all cases. We have come across a few bits of software which explicitly request an A record when doing a DNS lookup, which obviously doesn’t work.

As with any NAT configuration, you’re sharing a v4 address with other users, which can cause issues for sites that perform IP-based filtering or rate limiting.

Make the switch

Like most providers, we now charge for IPv4 addresses, but unlike most other providers it’s a tax you probably don’t need pay. We offer IPv6-only versions of all of our virtual and dedicated servers, and our Raspberry Pi servers area all IPv6-only.

Learn more

If you’d like to hear more, here are some videos of a presentation that Pete gave at the UK Network Operators Forum (UKNOF).

Good security practice requires two different factors.

We’ve just rolled out a much requested feature to our control panel: Timed One Time Passwords or TOTP.

TOTP is a type of 2FA. If these acronyms are making sense to you, head over to the control panel and set up TOTP.

If not, read-on…

What is 2FA?

You’ll probably have noticed an increasing number of websites that you use encouraging or requiring you to enable “two factor authentication” or 2FA.

2FA refers to requiring two separate things to confirm your identity: something you know (your password) and something you have (e.g. your phone).

2FA protects against some of the most common ways in which accounts get compromised:

• Username/password re-use. Despite advice not to do so, plenty of people re-use passwords across lots of different sites. Every now and again, sites get compromised, and databases of usernames and passwords become available on the shadier parts of the internet. These credentials will then be tried against other sites, looking for places that they’ve been re-used.
• Email account compromise. If your email account is compromised, it’s very easy for an attacker to gain access to your other accounts, as it’s almost always possible to reset your password by sending an email.
• Key-logging. If your computer is compromised, or you use an untrusted shared computer, key-logging malware may be installed that logs your password as you type it to log into your account.

2FA protects against all of these. It’s no longer sufficient to know the username and password to login, and you can’t reset your password just by having access to the email account. 2FA uses “one time passcodes” which means that whilst they can be captured by a key-logger, they’re of no value as they’ve already been used.

TOTP, SMS and Recovery Codes

We now support three different methods to provide the second factor: SMS, TOTP and recovery codes. With a Timed One Time Password your phone uses a secret key and the current time to generate a unique six digit code. The code is only valid for a short period, and can only be used once. The code proves that you have access to the secret key in the phone, but does not require you to send the secret key or any part of it to us.

With SMS we send you a time-limited, one-time code via a text message. Your phone collects this and you can type it in during login to prove that you’re holding your phone.

Recovery codes are intended to be a fall back should you lose access to your primary 2FA method. These are a set of one time codes that you can store securely (e.g. on paper, in a safe) and use each of them for a single login as required.

TOTP has a number of advantages over SMS. Firstly, it’s entirely offline on your phone so that if you’re somewhere with no phone signal you can still log in. Secondly, it doesn’t rely on trusting the mobile phone network; anyone with access to the phone network could intercept your SMS code or arrange for it to be delivered to another device. Similarly you may have things like message sharing enabled which means that your passcode is delivered to multiple devices.

Setting up TOTP

TOTP is very easy to setup. You’ll need an app on your phone. You can use Google Authenticator, but we prefer the open source FreeOTP. Once installed, go to the two-factor auth page in our control panel and hit the big green “Enable TOTP” button.

You’ll be shown a QR code which you can scan into the app on your phone, and you can then start generating codes. You need to enter a code to confirm that it’s set up correctly, and you can then choose to require 2FA whenever you log into your account.

Whilst you’re there, you should take the chance to print off some recovery codes.

Sympl is so simple it’s even usable by Cambridge graduates

We’re pleased to announce that we are now supporting the Sympl open source project.  Sympl is a fork of Symbiosis, a platform that makes hosting websites and email on a virtual or dedicated server simple.  Once installed, configuring a new website, or creating a new email address and mailbox, is as simple as creating a new directory.  Web server, mail server and DNS configuration is all taken care of for you.

We’ve already taken the first steps towards integrating Sympl into our infrastructure by implementing support for our DNS API in OctoDNS.  For our next step, we will be adding support for OctoDNS to Sympl.  This means that it becomes possible to use Sympl with our DNS infrastructure, but equally you can use any other provider supported by OctoDNS (we don’t believe in lock in!)

We’re now very pleased to welcome Paul Cammish, the newest member of the Mythic Beasts team.  Paul has considerable experience, having worked at a number of different ISPs since 2000, most recently at Bytemark.  Paul created the Sympl project earlier this year, in order to provide ongoing support and enhancements for the platform.

We’re very excited by the possibilities that Sympl provides, and have some interesting ideas for future developments once we’ve dealt with the immediate priorities of DNS integration, and support for the upcoming Debian Buster release.

The source code for Sympl is now available in our self-hosted GitLab instance.

We’ve just rolled out a major overhaul of our DNS management interface. We hope that you’ll find the new interface faster and easier to use. As well as improvements to the user interface, we’ve also added the ability to import zone files. This means that if you’ve got a domain that is currently hosted with another provider, you can now easily transfer all of your DNS configuration to our servers in bulk (provided that you can get them to give you a copy of your current zone file).

Our DNS management interface is included with all domain registrations.  It’s also available for domains registered elsewhere for customers of our other services, including hosting accounts, virtual servers, dedicated servers and Raspberry Pi servers.

The DNS interface includes DNS API access, allowing you to support dynamic DNS and to automate other DNS management tasks.

We believe in retaining customers through good service rather than lock-in, so naturally there’s a corresponding zone file export feature.

X now does Y 🙂

— Mythic Beasts (@Mythic_Beasts) 19 March 2019

Sensible people don’t like to hard code IP addresses in lots of different places in DNS. Better to assign it a name, and then reference that name, as it makes it clearer what’s what and if you ever need to change that IP, you’ve only got to do it one place.

CNAME records can be a good way to do this, by aliasing a DNS name to an IP. Unfortunately, the DNS specs prevent you using CNAMEs in various places that you might want to, most commonly at the root-level of your domain (the dreaded “CNAME and other data” problem).

This is where ANAME pseudo-records come in. They look just like a CNAME record, but rather than being added to the DNS, our server converts them into A and AAAA records. This allows you to get the benefits of a CNAME in places where a CNAME is not legal.

This week a customer suggested another use for ANAME records that we’d not previously thought of: round robin DNS. That is, a single DNS name that points to multiple servers. As you can’t have multiple CNAME records for the same hostname, implementing round-robin DNS means hard-coding A and AAAA records into your zone file. Like this:

proxy.mythic-beasts.com. 3600	IN	A	93.93.129.174
proxy.mythic-beasts.com. 3600	IN	A	46.235.225.189
proxy.mythic-beasts.com. 3600	IN	AAAA	2a00:1098:0:80:1000:3b:1:1
proxy.mythic-beasts.com. 3600	IN	AAAA	2a00:1098:0:82:1000:3b:1:1

Which is messy. Wouldn’t it be nicer to use the names of the servers involved? Like this:

proxy.mythic-beasts.com. 3600	IN	CNAME	 rproxy46-sov-a.mythic-beasts.com.
proxy.mythic-beasts.com. 3600	IN	CNAME    rproxy46-hex-a.mythic-beasts.com.

Sadly, the spec says you can’t do that, but thanks to a minor tweak to our DNS control panel code, you can now do it with ANAME records. Simply specify multiple ANAME records for your host name, and we’ll go and find all A and AAAA records for all of the hosts that are referenced.

Thanks to @grayvsearth for the suggestion on this one.

ANAME records are available in our DNS management interface, which is included with all domain registrations, and available for free on other domains for customers of other services. Other features include a DNS API, allowing you to obtain Wildcard Let’s Encrypt certificates.

Wildcard… sounds a bit like wildcat… cat pics!
Photo by Peter Trimming, CC BY 2.0

We’ve just made some changes to our plugin for dehydrated in order to better support Let’s Encrypt wildcard certificates.

Unlike normal certificates, which can be obtained using a web-based challenge, Let’s Encrypt’s wildcard certificates require a DNS-based challenge. In other words, you need to prove that you can control the DNS for the domain for which you are requesting a wildcard certificate.

Mythic Beasts provides a simple API for controlling DNS, which makes it possible to automate the process of responding to these challenges, and we provide a plugin for the popular dehydrated client that does just this.

We’ve just deployed a minor change which means that it’s now possible to obtain a single certificate for a domain, and a wildcard under that domain.

Access to our DNS API is included with all domain registrations. For more information, please see our instructions on using DNS-based challenges wih Let’s Encrypt. Please note that in order to obtain wildcard certificates you need to be using dehydrated version 0.6.0 or later.