Failure is not an anomaly, it is an expectation. We know hard drives fail, so we build RAID systems to ensure data redundancy. We keep spare tires in cars, knowing that flat tires are an inevitability rather than a rare occurrence. Similarly, when developing systems that rely on human interaction, we must accept that humans will make mistakes. Designing with this principle in mind is essential to creating resilient systems.
The conversation in the team recently focused on people training to follow a procedure to prevent breaking the release package before the deployment date.
On the cybersecurity side, the same happens, users are drilled on recognizing phishing attempts, avoiding suspicious links, and rotating passwords. While these measures are important, they operate on the assumption that users will consistently behave perfectly. This assumption is not realistic.
Humans are fallible. Under stress, tight deadlines, or distraction, even the most well-trained individuals can and will make mistakes. They will click on malicious links, open suspicious attachments, and sometimes even enter their credentials into phishing sites. It is not a question of if but when it will occur. Therefore, systems must be designed to anticipate and prevent human error.
While user training remains important, it should not be the primary focus. The concept of designing for failure is not new:
In aviation, there is a pre-flight checklist, but also cockpit designs account for the possibility of pilot error, with automated systems and fail-safes to prevent catastrophic outcomes.
In manufacturing, the assembly lines often incorporate lean management with “Poka Yoke” mechanisms to catch errors before they propagate.
Elevators have emergency brakes that not many people know, for security Zero Trust architecture, and so on… systems are designed with the assumption that mistakes will happen. Technology is used to catch those mistakes before they result in harm. CI/CD, Cybersecurity and system design should be no different.
