You’re knee-deep in code, sipping your fourth cup of coffee, and suddenly, your app crashes! Your heart rate? Skyrocketing!
Before you realize what’s happening, the money clock starts ticking and the cost per minute soars.
That’s a software incident in action!
What is a Software Incident?
A software incident is any unexpected event that disrupts the normal functioning of a software application or system.
From downtime to data breaches, deployment failures to cybersecurity attacks, and application crashes to failures in payment gateways—numerous software incidents occur within a business.
To know more about incidents, check out this blog post.
Cost of Software Incidents
Last year, CISQ reported that poor software quality costs the US $2.41 trillion, with operational failures amounting to a staggering $1.81 trillion.
Let’s spotlight one incident—downtime.
- In 2014, the Gartner Report stated over 33% of enterprises lose $1-5 million for 1 hour of downtime.
- Axios reported the 4-hour AWS outage in 2017 resulted in losses of $150 million for S&P 500 companies and $160 million for U.S. financial services firms.
Now, let’s switch gears to another incident—data breaches.
- As of 2023, the global average cost per data breach amounted to $4.45 million, an increase from $4.35 million in the previous year.
- Remember Epsilon's epic incident in 2011? They took a $4 billion hit from a data breach that affected up to 75 clients, including Best Buy, JPMorgan Chase, and Target.
5 Real world software incidents
The following incidents will give a more detailed picture of how much software incidents can actually cost you. Get ready to be amazed!
1. Facebook Outage
In 2020, Facebook had its worst-ever outage, which lasted for at least 14 hours.
A configuration change triggered a domino effect in its data centers, causing a global network meltdown.
The downtime also affected its products—Instagram, Messenger, and WhatsApp.
Facebook lost ~$6.3 million for every hour of downtime. The final tally? About $90 million.
The blackout left millions of users hanging and caused Facebook's stock to drop 2% in after-hours trading.
Source: Google Finance
2. Equifax Data Breach
In 2017, Equifax, an American multinational consumer credit reporting agency, fell victim to a massive data breach.
This breach exposed the personal and financial information of ~147 million individuals, including sensitive data like social security numbers, birthdates, and driver's license numbers.
The breach even had its hands on credit card information from about 209,000 consumers.
The total loss? A jaw-dropping $1.7 billion.
Post-disclosure of the breach, Equifax's share price dropped by 31%.
Source: MarketWatch
3. Atlassian Outage
In April 2022, Atlassian had a major outage that lasted for 14 days. Jira, Confluence, Opsgenie, and other Atlassian products were unavailable for a subset of its customers.
The problem was triggered by a faulty application migration process. Engineers at Atlassian created a script to disable an outdated application, but due to communication problems between teams, the script accidentally disabled all Atlassian Cloud products.
To make matters worse, the script was apparently configured to permanently delete data rather than mark it for deletion. As a result, data in affected accounts was removed permanently from production environments.
This led to the immediate deletion of 883 sites representing 775 customers.
Prior to the outage, Atlassian's share price was $305.75, but it plummeted to $245.29 after the incident, resulting in a significant drop of ~19.56%.
Prior to the outage. (Source: Google Finance)
4. Cloudflare Outage
On June 21, 2022, Cloudflare suffered an outage that affected traffic in 19 data centers. These data centers handled a significant proportion of its global traffic.
Reason for the outage? Cloudflare was working on a project to increase resilience in its busiest locations. A change to the network configuration in those locations caused the outage.
Popular apps like Discord, Shopify, and Fitbit took a hit. Websites such as Udemy, Splunk, Quora, and Crunchyroll were affected. Downtime detectors that relied on Cloudflare were down too.
Despite only 5% of the Cloudflare network being directly affected, the consequences were disproportionately severe. A staggering 50% of total requests worldwide encountered delays or denials of access.
The Ponemon Institute's 2016 report estimated an average downtime cost of $9,000 per minute. Given this figure, the companies that relied on Cloudflare could face losses in millions per hour during the outage.
5. Google Server Outages
Google faced a series of outages in 2013, 2014, 2018, 2020, and 2022. In 2020 alone, it had three consecutive outages.
These outages affected key platforms such as Gmail, YouTube, Google+, Google Calendar, and various other Google services.
These interruptions had severe consequences, including disruptions in customer service and substantial financial losses for businesses that relied on Google's services.
For instance, during one of these outages, it is estimated that Google incurred a staggering loss of approximately $45.5 million in just 2.5 hours of downtime. This financial setback undoubtedly affected numerous companies that depended on Google's infrastructure and services.
Other Costs of Software Incidents
The costs of software incidents go beyond financial losses. They may include reputation damage, trust issues, regulatory fines, etc.
To understand these undercover costs, let’s again take the example of Equifax.
Equifax's data breach left customers feeling vulnerable and violated, eroding trust in its ability to safeguard their data.
The breach occurred in July 2017, but Equifax didn't publicly announce it until September 2017. This delay in notification led many customers to question the company's transparency and accountability.
The breach generated a significant public outcry, leading to numerous lawsuits and regulatory investigations.
Equifax shelled out $800 million to settle various federal and state investigations.
On top of that, the company spent an extra $337 million for technology and data security, legal and investigative fees, and product liability related to the breach.
Spike Can Help!
No doubt the costs of software incidents are staggering with both financial and non-financial burdens. But worry not, Spike has got your back!
With a user-friendly and budget-friendly incident management system, Spike helps you tackle all software incidents like a pro!
Sign up for a demo of Spike today: https://spike.sh/demo