SCOTT DETROW, HOST:
Americans awoke to their phones blowing up over the news of a global technological meltdown. The glitch left employees of airlines, banks, hospitals and emergency services staring at the dreaded blue screen of death today as computers around the world were hobbled by one of the biggest IT outages ever. And the U.S. had left thousands of travelers stranded at airports across the country.
MIKE WILSON: Oh, it's a mess. Yeah. Every one of us had to leave and hurry up and try to find something else, so - and other people still fighting to get it.
ARUNA CHAKRAVARTI: I guess the system is down, so they were unable to update us anyway. And I was told I'm on time, and I'm not.
LUNSTON DENNIS: There's nothing I can do. I can't fly the plane, so I have to be - sit here and be patient with everybody else.
DETROW: That was Mike Wilson in Austin, Texas, Aruna Chakravarti in Portland, Maine, and Lunston Dennis in Atlanta. All of this chaos was caused apparently by faulty software updates from the cybersecurity firm CrowdStrike. We have two NPR reporters here to help us make sense of what happened and why. Jenna McLaughlin covers cybersecurity, and Joel Rose covers transportation. Thanks to both of you.
JOEL ROSE, BYLINE: Hello.
DETROW: Joel, let's start with you.
JENNA MCLAUGHLIN, BYLINE: Hey, Scott.
DETROW: This went way beyond air travel, right?
ROSE: Correct. I mean, this was a much bigger problem. So many industries depend on cybersecurity software from this one company, CrowdStrike - as you mentioned, hospitals, retail businesses, broadcasters, ports, government agencies and emergency responders. But the most visible face of the problem could be the airlines. Thousands of flights canceled worldwide, including more than 2,600 flights that begin or end at U.S. airports. And that left lots of unhappy passengers, like Dawn Hodges from Atlanta. She was trying to fly to Colorado to see her daughter and her new granddaughter.
DAWN HODGES: I was really looking forward to the trip and wanted to see my girls. I'm number 999 in the wait, you know, for the help desk. So I won't get out today. So I'm mad, and I'm frustrated, and I'm sad.
ROSE: Delta, American and United Airlines were hit the hardest by this outage. All three say that they resumed some flights this morning. It's still a big mess, though, as the cancellations and delays rippled across the system throughout the day.
DETROW: So Jenna, that gets to the big question - how? How did this happen?
MCLAUGHLIN: Yeah, Scott. This was caused by a buggy security update. It was pushed to a large chunk of the millions of computers that are using CrowdStrike cybersecurity software. So for context, cybersecurity software is unique. It's installed deep on machines, has lots of special privileges - and that's so it can monitor everything that's going on and look out for unusual malicious activity. When that software is updated or changed, because it's so deeply embedded in how the computer functions, unexpected problems can crop up.
So companies push out software updates all the time for lots of different reasons, whether it's a security update, a new feature. In this case, CrowdStrike says it was a minor content update for Windows hosts. There was something wrong with that update. We don't know exactly what. But ultimately, that means everyone using CrowdStrike software who are running Microsoft Windows got that blue screen of death today. CrowdStrike says this was not a cyberattack but a mistaken code. Even so, the impact looks pretty similar.
DETROW: Yeah. And if the impact is similar, have we seen something like this before, but intentional?
MCLAUGHLIN: Absolutely. I mean, the scale here is remarkable, but we have seen similar incidents. If you remember SolarWinds...
DETROW: Yeah.
MCLAUGHLIN: ...Back in 2020, Russian hackers got deep inside the software company, and they purposefully pushed malicious software update like this. That allowed them to spy on SolarWinds customers, which included government agencies, for months and months. Of course, we've also seen criminal hackers, and they took down large segments of the health care industry recently using similar methods. Those hackers were trying to cause problems, but a simple mistake can actually be really damaging.
DETROW: So Joel, let's get back to all those people stuck at the airport. How much longer will these disruptions last?
ROSE: Yeah, it may take a while for the impacted airlines to sort all this out because even once their computer networks are back to normal, you can still have residual problems because their equipment, their planes and their personnel are just not where they are supposed to be. So the airlines, I think, could have problems tomorrow and even into the weekend. Railroads have fared a lot better than airlines. There were a few transit systems that had some problems with their displays telling passengers how long they have to wait for the next train. Some of those were down earlier this morning.
DETROW: And Jenna, what is being done about this massive outage?
MCLAUGHLIN: A lot. My inbox was flooded with emails this morning. IT teams across the country will be working around the clock to help fix this. CrowdStrike says that a fix has been deployed, and the cybersecurity software itself is functioning like it should. But the fix can be tricky, depending on the device. You often have to have physical access to the device. Meanwhile, President Biden has been briefed. DHS' cybersecurity and infrastructure security agency is working with CrowdStrike and the impacted victims.
DETROW: I mean, this just underscores how much the - things like this touch all aspects of the economy. So what does this mean going forward?
MCLAUGHLIN: Absolutely. I mean, as far as preventing things like it in the future, you know, the problem is that computers are really complicated, and they're only getting more complicated as they're more interwoven into every single aspect of our lives. I talked to Andy Ellis. He was the chief security officer at cloud company Akamai. He dealt with a really similar incident almost exactly 10 years ago. Here's what he said.
ANDY ELLIS: When you think about how a system works, you know, it's hard for normal humans to really grasp what's happening.
MCLAUGHLIN: So like he's saying, it's incredibly complicated. There's a lot that can go wrong. Even so, experts say testing software updates is really important. You need to catch those problems before they go live. And, you know, Scott, all this has me thinking that technology is so vital. It adds a lot of benefits to our lives, but it is really vulnerable. And going forward, you know, I think there's going to be a lot of questions about how best to make sure critical software is protected, works as planned, it minimizes outages. Otherwise, this could just be a part of our normal lives.
ROSE: And when you're talking about the aviation industry, there are a lot of redundant systems on planes so that if something critical breaks, there is a backup plan, and you can still land the plane. But as we are seeing today, there is not that same kind of redundancy, I think, when it comes to the IT systems that keep the airlines and the transportation system moving.
DETROW: That is NPR's Joel Rose, as well as Jenna McLaughlin. Thanks to both of you.
ROSE: You're welcome.
MCLAUGHLIN: Thank you. Transcript provided by NPR, Copyright NPR.
NPR transcripts are created on a rush deadline by an NPR contractor. This text may not be in its final form and may be updated or revised in the future. Accuracy and availability may vary. The authoritative record of NPR’s programming is the audio record.
