Security patch caused equipment shutdown and fire at NASA

Fire went unnoticed for three hours because the patch also took down the fire alarm system

A security patch applied by IT staff at NASA caused an equipment shutdown and subsequent fire that destroyed space-craft hardware, a report from the space agency's inspector general has revealed.

The "security patch caused monitoring equipment in a large engineering oven to stop running, resulting in a fire that destroyed spacecraft hardware inside the oven. The computer reboot caused by the software upgrade also impeded alarm activation, leaving the fire undetected for 3.5 hours before it was discovered", the inspector general found (PDF).

The potentially catastrophic series of events was contained in a critical report into the organisation's operational technology, which forms part of the space agency's critical infrastructure. However, the inspector general found NASA's operational technology to be inadequate in many respects.

The report continued: "A large scale engineering oven that uses operational technology to monitor and regulate its temperature lost this ability when a connected computer was rebooted after [the] application of a security patch update intended for standard IT systems.

"The reboot caused the control software to stop running, which resulted in the oven temperature rising and a fire that destroyed spacecraft hardware inside the oven. The reboot also impeded alarm activation, leaving the fire undetected for 3.5 hours before it was discovered by an employee."

The report found a series of items of critical infrastructure, such as control systems, that were not databased or looked after as operational technology. As a result, the agency exhibited security gaps in terms of the IT used by engineering, and the operational IT looked after by NASA's IT department.

The report found two further examples of equipment failures caused by IT, including "vulnerability scanning used to identify software flaws that can be exploited by an attacker [that] caused equipment to fail and loss of communication with an Earth science spacecraft during an orbital pass.

"As a result, the pass was rendered unusable and data could not be collected until the next orbital pass".

In another instance, a chilled water heating, ventilation and air-conditioning system was disabled - which caused IT equipment reliant on it in one of NASA's datacentres to be shutdown after temperatures rapidly rose to more than 50 degrees centigrade.