This article draws on real-world OT incident response experience. Client details are not disclosed.
A cyber attack on an industrial site is a different event from a cyber attack on a business network. The consequences are not measured in lost data or reputational damage alone. They are measured in stopped production, failed processes, and in serious cases, safety events. Recovery is not a matter of restoring files from backup. It is a matter of rebuilding an operational technology environment that may span dozens of servers, hundreds of workstations, and years of accumulated configuration.
This article describes what actually happens after a ransomware attack hits an industrial OT environment: what the first hours look like, what drives the length of the recovery, and what the experience reveals about the state of most OT security programmes.
The first hours
The first indication of an OT cyber event is rarely a clear alarm. It is more often something unexplained: a SCADA server that won’t start, a login screen that appears where it shouldn’t, an unusual network alert. The initial response is typically reactive: investigate the symptom, identify the scope.
Once the scope becomes clear, once it is apparent that multiple systems are affected and that this is a security event rather than a hardware failure, the immediate priorities are:
- Isolate affected systems: physically disconnect network cables from affected machines. Do not rely on software-level network isolation; ransomware can re-establish connections. Isolation limits further spread.
- Preserve evidence: do not wipe or rebuild machines before forensic investigation. The state of affected systems at the time of discovery is evidence. It determines what happened, when, how the attacker got in, and what data was accessed.
- Assess the blast radius: determine which systems are encrypted, which are clean, and which are in an unknown state. This assessment drives the recovery sequencing.
- Locate and verify backups: the availability of clean, verified backups is the single biggest determinant of recovery time. Sites with current, tested backups of all PLC programs, SCADA projects, and server images recover in days. Sites without them may take weeks.
What recovery actually looks like
In an OT environment, recovery from a significant ransomware attack is a methodical, sequential process. It cannot be rushed without creating new problems.
A typical recovery sequence for an industrial site:
- Build a clean environment first: install new operating system images on servers, either from a clean server image backup or from fresh installation media
- Establish domain infrastructure: if Active Directory domain controllers were affected, these must be rebuilt before other systems can rejoin the domain
- Restore SCADA servers: reinstall SCADA software from vendor media, restore project files from verified backups, reconfigure server roles and communications
- Restore engineering workstations: reinstall PLC programming software, restore PLC program backups, verify connectivity to PLCs
- Deploy endpoint security: install and configure endpoint detection and response software across the OT environment before returning systems to operation
- Verify PLC programs: before returning production to operation, verify that PLC programs on each controller match the verified backup versions. Ransomware that has been present in the OT network for an extended period may have modified PLC programs.
- Return to operation incrementally: restore production area by area, verifying normal operation before proceeding
The most time-consuming part of OT recovery is almost always the verification work: confirming that every system has been rebuilt cleanly, that every PLC program matches its verified backup, and that the environment is safe to return to operation. This cannot be shortcut.
What drives recovery time
Recovery time from an OT cyber incident is driven by three factors above all others:
- Backup quality: sites with current, tested backups of all SCADA projects, server images, and PLC programs recover in days. Sites that discover their backups are incomplete, corrupt, or out of date face reconstruction rather than restoration.
- Asset documentation: sites that have a current, accurate inventory of all OT hardware and software can scope the recovery, sequence the work, and track progress. Sites without one spend significant time in discovery, determining what needs to be rebuilt before beginning to rebuild it.
- Incident response plan: sites with a documented OT incident response plan that has been tested activate it. Sites without one build the plan as they go, under pressure, which is significantly slower and creates gaps.
What the experience reveals
Every OT incident response engagement reveals the same things, not because the sites are negligent, but because OT security has been a lower priority than production for most of industrial history.
The most common findings:
- SCADA backups exist but have never been tested, and some do not restore correctly
- PLC program backups are incomplete: programs that were modified during minor maintenance were never saved back to the backup location
- The asset inventory is out of date: servers and workstations discovered during the response were not in any documented register
- Remote access pathways are broader than anyone knew: connections that were installed for specific vendor support activities and never removed
- Endpoint security was absent from OT systems: the first indication that the attack had reached the OT network was the SCADA server going down
None of these are surprising findings. They are the normal state of most industrial OT environments. The purpose of an OT cybersecurity programme is to systematically address them before an incident makes them critical.
About the author
Dennis Murphy RPEQ is the principal engineer at Beetle Engineering & Automation. He has provided OT cybersecurity incident response services for industrial clients across Queensland’s resources and agricultural processing sectors. Contact: [email protected]