Worldwide Internet Outage Caused By Single Configuration Error

The June 8th Internet outage which affected websites in dozens of countries across the Americas, Europe and Asia, as well as South Africa has been traced to a single configuration error at a little known but important IT infrastructure company, Fastly

Amazon, eBay and Boots were among retailers whose websites disappeared during the outage. Amazon and Boots alone report a combined £25bn in annual sales in the UK, meaning they would typically earn nearly £3m between them in the course of an average hour. Payment provider PayPal which processed $936bn of transactions in 2020, approximately $106m for every hour, was also temporarily unavailable.

Fastly said it had identified a service configuration that triggered disruptions across its servers and has now disabled that configuration.

The network run by Fastly had the outage for more than an hour and in most cases was occurring mid-morning London time. Investigation reveals the incident was caused by a cloud configuration fault affecting governments and businesses around the world in sectors ranging from media to online retail and telecoms. “We experienced a global outage due to an undiscovered software bug that surfaced on June 8 when it was triggered by a valid customer configuration change. We detected the disruption within one minute, then identified and isolated the cause, and disabled the configuration.... Within 49 minutes, 95% of our network was operating as normal”, said Fastly’s SVP  of Engineering & Infrastructure in a statement. 

The basic issue appears to be that Fastly had mitakenly taken down its own network with a bad software update, resulting in a blockage affecting millions of dollars in revenue for numerous corporations around the world.

All UK government websites using the gov.uk domain were swept up in the outage, which temporarily knocked out crucial services such as the online system by which taxpayers can file annual returns with HM Revenue and Customs. The British  Government said it was investigating reports that users were unable to book Covid-19 tests online as a “matter of urgency”.
Visitors to the official website of the White House were also greeted with a message likely to have been seen by hundreds of millions of internet users: “Error 503 service unavailable”.

According to analysis of data from Google’s AdSense platform, outlets lost approximately $300,000 in revenue during the period, as they missed out on clicks that would usually translate into payments from advertisers. The calculation, made by marketing agency connective3, covers lost revenue at affected advertising-reliant media sites including the The BBC, The Guardian New York Times, the Financial Times, CNN, Al Jazeera, Bloomberg, the Independent and the Evening Standard., were all among the websites that crashed. 

Fastly is a content delivery network (CDN), which maintains a network of servers that transfer content quickly from websites to users. It provides a layer of support between Internet companies and customers trying to access the various online platforms it services, when it goes down, access to those platforms can be blocked entirely. 

One of the reasons that the outage was so wide is that cloud computing service companies like Fastly are consolidating, leaving websites dependent on a shrinking number of providers, a critical vulnerability demonstrated recently by a similar large scale outage affecting Amazon Web Services in December 2020.

Fastly:      CNN:     Vox:     Computing:   Guardian:

You Might Also Read:

SANS Institute book: Practical Guide To Security In The AWS Cloud:

 

« Beware Of Credentials Phishing
AI Dominance On The Battlefield »

CyberSecurity Jobsite
Perimeter 81

Directory of Suppliers

Practice Labs

Practice Labs

Practice Labs is an IT competency hub, where live-lab environments give access to real equipment for hands-on practice of essential cybersecurity skills.

Jooble

Jooble

Jooble is a job search aggregator operating in 71 countries worldwide. We simplify the job search process by displaying active job ads from major job boards and career sites across the internet.

IT Governance

IT Governance

IT Governance is a leading global provider of information security solutions. Download our free guide and find out how ISO 27001 can help protect your organisation's information.

The PC Support Group

The PC Support Group

A partnership with The PC Support Group delivers improved productivity, reduced costs and protects your business through exceptional IT, telecoms and cybersecurity services.

CYRIN

CYRIN

CYRIN® Cyber Range. Real Tools, Real Attacks, Real Scenarios. See why leading educational institutions and companies in the U.S. have begun to adopt the CYRIN® system.

IGEL Technology

IGEL Technology

IGEL Technology is one of the world's leading thin client vendors. Thin clients increase data security and compliance.

Clearwater Security & Compliance

Clearwater Security & Compliance

Clearwater Compliance specialize in Privacy, Security, Compliance and Risk Management Solutions for Health Care, Law Firms and other businesses.

SysTools

SysTools

SysTools provides a range of services including data recovery, digital forensics, and cloud backup solutions.

Syskode Technologies

Syskode Technologies

Sykode Technologies is a next-generation global technology company offering an integrated portfolio of advisory services, products and solutions in areas including AI, IoT and Cyber Security.

Enigmatos

Enigmatos

Enigmatos is an Israeli based Automotive Cyber Security company. We provide solutions to the ever growing threat of vehicle hacking.

SYSGO

SYSGO

SYSGO is the leading European provider of real-time operating systems for critical embedded applications in the Internet of Things (IoT).

FifthDomain

FifthDomain

We are a specialist cyber security education and training company tackling the global cyber security skills shortage.

Document Security Systems (DSS)

Document Security Systems (DSS)

DSS anti-counterfeit, authentication, and brand protection solutions are deployed to prevent attacks which threaten products, digital presence, financial instruments, and identification.

Netenrich

Netenrich

The Netenrich operations intelligence platform is built from the ground up to help enterprises resolve everyday and futuristic problems for stable, secure environments and infrastructures.

StoneLock

StoneLock

StoneLock is a trusted leader in the design and manufacture of facial recognition software and technology.

Synamic Technologies

Synamic Technologies

Synamic Technologies was founded in 2018 as a start-up to automate cyber security processes. Our CISOSCOPE product automates vulnerability management, risk management and compliance.

Precursor Security

Precursor Security

Precursor Security are information security specialist, delivering all aspects of Security testing, Cyber Risk Management, and Continuous Security Testing.

Pathway Communications

Pathway Communications

Established in 1995, Pathway Communications – is part of the Pathway Group of Companies, a Canadian IT Managed Services organization.

Allstate Identity Protection

Allstate Identity Protection

Allstate make it easy to provide complete identity protection, so everyone can live more confidently online.

Panasonic Automotive Systems

Panasonic Automotive Systems

Panasonic Automotive Systems brings together security technologies and human resources cultivated across an extensive range of businesses into the automotive field.

DeepTempo

DeepTempo

At DeepTempo, we build AI models and related software that protect enterprises and service providers from sophisticated cyber threats.