Last month we attended Electromagnetic Field as a silver sponsor.  Despite being in a remote field in Herefordshire, the site had amazing connectivity, which we played a small part in providing.

We provided some optics to help get internet around the field and acted as an Internet Transit Provider to uplink the festival through our network.

We had a tour of the network operations centre. ElectromagneticField leased a single fibre to a telephone exchange in Gloucester and a donated private 40Gbps circuit hauls the traffic back to the London Network Access Point (LONAP). We used private VLANS over LONAP to link to the Mythic Beasts core network routers in Sovereign House and Telehouse and used this to provide our blend of transit providers and peers, including direct access over private fibre to some of the largest cloud providers.

The section from the field in Eastnor to Gloucester uses Dense Wavelength Division Multiplexing, a neat technology that uses multiple different frequencies to carry multiple signals on the same fibre at the same time. Each optical transceiver typically transmits at a specific wavelength on one fibre, and receives on the same wavelength on a second fibre. This is put into a multiplexer which combines the different frequencies from multiple optics into the same fibre and a second multiplexer splits them back out into the component frequencies at the other end, allowing multiple 10Gbps channels to operate over one fibre pair.

We use the same technique to multiply up the bandwidth in our core London network on our leased fibre that interconnects our core London points of presence.

To keep costs down at EMF there isn’t a fibre pair – just a single 60km fibre. The hack to get around this limitation is to use different frequencies in each direction and rely on the fact that the transceivers are frequency-specific for transmitting but not receiving – a transmitter that transmits at 1572.48nm will happily receive at 1572.89nm and vice versa. You can then use eight channels on one fibre as four bi-directional channels.

Around the campsite there were datenklo (a switch in a portaloo) which provided wifi and multiple 1Gbps wired uplinks. Each dataklo had a 10Gbps link back to the network operations centre to provide super-fast connectivity all around the site.

You can read more about some of the awesome things we saw at EMF 2024 in our previous blog post.

We went to Electromagnetic Field 2024 as a silver sponsor. Whilst there we found a lot of fantastic fun things and missed a vast number of others. The really amazing part of the festival was the massive variety of things the participants brought with them. Lock picking and blacksmithing courses were available. Geodesic domes were very popular and courses on how to build them were at the Maths Village. At least one dome integrated with the API from the bar, so the lighting changed colour based on what type of drinks were currently being ordered.

If you thought that ethernet and IP was a bit too modern there was a fully functioning DECT cordless phone network and you could access the live schedule information over ViewData (Prestel/Minitel).

We missed at least 98% of the talks. Fascinating ones we did see included a comprehensive explanation of the attempt to backdoor ssh with xzutils by Dr Matthew Garrett. Dr Matthew Bothwell gave a guide to Astrophysics for Supervillains covering things like ‘what happens if I crash the moon into the earth?’ (answer: you get a moon). Tim Hunkin of the Secret Life of Machines gave a short history of electric shocks and Ian B Dunne played the Theramin and musical saw. Much to our surprise, this was quite listenable.

Sadly the magic smoke came out of the Tesla Coil before the end so we had to make do with fire and lasers in additional to a traditional light show around the DJ area in the Null zone.

But this is a short summary, there was a fantastic kids creche, swap shop, night market for created things, crochet, hand built guitars, paper rockets, active satellite tracking, a 5km run, a fully stocked bar, a manual version of flappy bird to play and and and [approximately 100,000 further words cut to keep this post merely far beyond reasonable]

The now notorious swap shop gets a special mention. Not only did it have extremely dangerous materials like Linux install CDs from the late 1990s that may permanently corrupt young minds but some things that are rather harder to get hold of.

It has a direct entry on wikipedia and is going to cause a lot of festivals to have to update their terms and conditions to prohibit bringing radioactive materials to the site. Fortunately an attendee was familiar with safe disposal procedure and quickly removed the offending sources.

The last and arguably best thing at Electromagnic Field was an incredibly secretive project, the Great Camp Hexpansion Question (GCHQ.net). This was a series of locations (mostly, but not all static) where you could plug the quest markers into the official badge which would record you’d found them. A cross between a scavenger hunt and a technology preview it encouraged wandering and looking around the whole camp to find many cool things that were tucked away.

We asked the organisers if this was an official GCHQ sanctioned project. They said no. But that’s what you’d expect GCHQ would say.

We’re pleased to announce that we are silver sponsors of this year’s Electromagnetic Field festival.  As in previous years, we will also continue to support the event with free transit.  EMF is a long weekend camping in a field where people who are really very interested in things will tell you about the things that really interest them. There’s talks, demos, art installations and workshops on all kinds of creative things. In addition to camping, everyone gets power and high speed internet to their tent. Rumour has it there is also a bar.

Previous years have had an exceptionally wide variety of talks on a huge number of different subjects. The list of talks from the last festival in 2022 is long, but includes things as wild as:

• Ship vs Oil Rig
• The imitation game – using live data feeds from Network Rail to control a model railway
• Building a home-made enigma machine

We’re not giving a talk this year as we didn’t come up with a good idea in time. For 2026 we’ve already rejected the following presentation titles :

• I’ve got 99 problems and HEX ain’t one.
• D. E. P. R. E. C. I. 8. The importance of correct accounting policies delivered through the medium of Aretha Franklin covers.
• As a large language model I can’t assist with that. It’s illegal, unethical, and against my guidelines.

We’re looking forward to meeting up with lots of interesting people at EMF2024.

Recently Trend Micro, through their Zero Day Initiative, published a critical flaw for the Exim mail server. It’s described as allowing remote attackers to execute arbitrary code on the Exim server without authentication. On the face of it, any server running Exim and listening on the internet can immediately be taken over by an attacker. What makes this worse is that they claim they reported this in June 2022, and the Exim team have ignored fixing it.

ZDI say ‘The only salient mitigation strategy is to restrict interaction with the application.’ and have allocated a scarily high severity score of 9.8/10.

Mythic Beasts make pretty heavy use of Exim in our mail infrastructure, and mitigating the security risk by turning off email is a pretty severe step while we wait for a fix. On top of that amongst servers we manage for ourselves and clients there’s nearly a thousand installed copies of Exim that will need to be updated.

The Exim team have a different view on the severity, as do other reputable security specialists. Watchtowr have a nice write-up explaining that, by default, none of the six issues can be exploited. Cross checking to Mythic Beasts mail infrastructure we can quickly confirm we’re not affected, and we believe that none of the managed customers should be either.

As this is now not especially time critical, we can wait for the supported operating systems to release updated packages which we can install.

Patching

The security issue is definitely significant enough to meet our 0-day policy of patching immediately as it’s network listening software with a risk of compromise. Debian released packages with the most important fixes on Monday 2nd October. Because this issue covers a very large number of affected machines, some of which are absolutely critical we decided to stage the rollout. First we did our staging servers, then one of our core mailhubs. We then paused for a short while to check no functionality was affected. Then we completed the full roll-out to all managed servers both customer and internal. The final step is our audit – recheck the Exim package on every managed server to make sure the update had applied everywhere. The full rollout and audit completed in around three hours.

We’re expecting updated packages from Ubuntu shortly, which will then be rolled out to all supported managed Ubuntu customers when available.

 

Last year we enhanced our web hosting service with the ability to choose your own PHP verison. You can choose a different PHP version for each website hosted with us, so you can upgrade your staging site and test before you upgrade the production one. With PHP 8.0 about to go end-of-life, the addition of PHP 8.2 provides more options for migrating production applications.

Since the initial roll-out, we’ve added more PHP versions to help with moving and upgrading older applications. Not only is the newest version PHP 8.2 available, but you can also select the older 7.3 and 7.4 versions. We’re proud to sponsor Ondřej Surý who creates the debian packages we rely on.

Our hosting accounts still support unlimited websites, have free and automatic SSL through Let’s Encrypt to keep your sites secure, and include MariaDB databases.

We’re delighted to announce our sponsorship of Organic Maps. Organic Maps is a simple, user-friendly application that downloads complete Open Street Map data to your phone, allowing you to use their mapping application offline complete with route planning from the on-device database.

This is a wonderful application. It doesn’t track you, advertise at you or flood you with non-notifications, and it works without mobile data and conserves battery life. So if you’ve ever been lost without signal or somewhere where roaming data is prohibitively expensive, or to a very busy location where the mobile networks were overloaded this application is genuinely better than the alternatives.

While the app avoids the need for mobile data, this comes at the cost of a significant up-front download of all the mapping data that you may or may not use offline. This won’t trouble typical home broadband, but for the servers at the other end it adds up quickly. We’ve stepped in and offered two 4GB virtual servers with 400TB/month of free bandwidth to Organic Maps, split between our London and Amsterdam zones, reducing the reliance on a traditional and bankruptcy-inducing large cloud provider.

 

At our list prices this would be somewhat cheaper:

Qty	Item	Item price	Price
2	VPS4 virtual servers (4TB/mo bandwidth)	£32.14	£64.28
396	Additional bandwidth (per TB)	£5	£1,980.00
	Total		£2,044.28

Being 90% better value is achieved in part by not having to fund our own space programme.

Flatpak provides Linux desktop applications in a secure sandbox which can be installed and run independently of the underlying Linux distribution. Application developers can produce one Flatpak and select the versions of libraries that their application is built, tested and run with so it’s easy for users on any Linux OS to get whatever was intended by the application developer.

Flathub is a distribution service to make sure that Flatpaks are available for popular Linux desktop applications, and at its heart is a private cloud running BuiltBot which builds popular Linux and free/open source desktop apps in Flatpak format. This lives in Mythic Beasts’ Cambridge data centre.

At Mythic Beasts we like this idea so much we offered them lots of free bandwidth (100TB) to help get them started. We’ve now upgraded this with a pair of virtual machines in our core Docklands sites to provide redundancy and more grunt for traffic serving.

Are Flathub downloads going a bit faster for folks today? @Mythic_Beasts have been helping speed up the @FlatpakApps revolution this week. 🙂

— Robert McQueen (@ramcq) 23 February 2018

Some of their users noticed and were appreciative immediately:

2017-02-23 16:30:00irc wow! Flathub is *so* much faster i’m getting like 10 MB/s compared to less than 1 this morning … and the search is now instant

2017-02-26 11:35PersiFlathub is _really_ fast now, great job to whoever is responsible
🙂