How Complex Systems Fail

Aug 23, 2026 10:13 PM - 3 weeks ago 6
  • Complex systems are intrinsically hazardous systems.

    All of the absorbing systems (e.g. transportation, healthcare, power generation) are inherently and unavoidably hazardous by the own nature. The wave of hazard vulnerability tin sometimes beryllium changed but the processes progressive successful the strategy are themselves intrinsically and irreducibly hazardous. It is the beingness of these hazards that drives the creation of defenses against hazard that qualify these systems.

  • Complex systems are heavy and successfully defended against failure

    The precocious consequences of nonaccomplishment lead complete clip to the building of aggregate layers of defense against failure. These defenses include evident method components (e.g. backup systems, ‘safety’ features of equipment) and quality components (e.g. training, knowledge) but also a assortment of organizational, institutional, and regulatory defenses (e.g. policies and procedures, certification, activity rules, team training). The effect of these measures is to supply a bid of shields that usually divert operations distant from accidents.

  • Catastrophe requires aggregate failures – azygous constituent failures are not enough.

    The array of defenses works. System operations are generally successful. Overt catastrophic nonaccomplishment occurs erstwhile small, apparently innocuous failures subordinate to create opportunity for a systemic accident. Each of these mini failures is basal to origin catastrophe but only the operation is capable to licence failure. Put different way, location are galore much nonaccomplishment opportunities than overt strategy accidents. Most first nonaccomplishment trajectories are blocked by designed system information components. Trajectories that scope the operational level are mostly blocked, usually by practitioners.

  • Complex systems incorporate changing mixtures of failures latent within them.

    The complexity of these systems makes it intolerable for them to run without aggregate flaws being present. Because these are individually insufficient to origin nonaccomplishment they are regarded arsenic insignificant factors during operations. Eradication of each latent failures is limited chiefly by economical costs but besides because it is difficult earlier the truth to spot really specified failures mightiness lend to an accident. The failures alteration perpetually because of changing technology, work organization, and efforts to eliminate failures.

  • Complex systems tally successful degraded mode.

    A corollary to the preceding constituent is that analyzable systems tally as surgery systems. The strategy continues to usability because it contains truthful galore redundancies and because group tin make it function, despite the beingness of galore flaws. After mishap reviews astir ever note that the strategy has a history of anterior ‘proto-accidents’ that nearly generated catastrophe. Arguments that these degraded conditions should person been recognized earlier the overt mishap are usually predicated connected naïve notions of strategy performance. System operations are dynamic, pinch components (organizational, human, technical) failing and being replaced continuously.

  • Catastrophe is ever conscionable astir the corner.

    Complex systems person imaginable for catastrophic failure. Human practitioners are astir ever successful adjacent beingness and temporal proximity to these imaginable failures – disaster tin hap astatine immoderate time and successful astir immoderate place. The imaginable for catastrophic result is a hallmark of analyzable systems. It is intolerable to destruct the imaginable for specified catastrophic failure; the imaginable for such nonaccomplishment is ever coming by the system’s ain nature.

  • Post-accident attribution to a ‘root cause’ is fundamentally wrong.

    Because overt nonaccomplishment requires aggregate faults, location is nary isolated ‘cause’ of an accident. There are aggregate contributors to accidents. Each of these is needfully insufficient successful itself to create an accident. Only jointly are these causes capable to create an accident. Indeed, it is the linking of these causes together that creates the circumstances required for the accident. Thus, no isolation of the ‘root cause’ of an mishap is possible. The evaluations based connected specified reasoning arsenic ‘root cause’ do not bespeak a method knowing of the quality of nonaccomplishment but alternatively the social, taste request to blasted specific, localized forces aliases events for outcomes. 1

  • Hindsight biases post-accident assessments of quality performance.

    Knowledge of the result makes it look that events starring to the result should person appeared much salient to practitioners astatine the time than was really the case. This intends that ex station facto mishap study of quality capacity is inaccurate. The result knowledge poisons the expertise of after-accident observers to recreate the position of practitioners before the mishap of those aforesaid factors. It seems that practitioners “should person known” that the factors would “inevitably” lead to an accident. 2 Hindsight bias remains the superior obstacle to accident investigation, particularly erstwhile master quality capacity is involved.

  • Human operators person dual roles: arsenic producers & arsenic defenders against failure.

    The strategy practitioners run the strategy successful bid to nutrient its desired merchandise and besides activity to forestall accidents. This dynamic value of strategy operation, the balancing of demands for production against the anticipation of incipient nonaccomplishment is unavoidable. Outsiders seldom admit the duality of this role. In non-accident filled times, the accumulation domiciled is emphasized. After accidents, the defense against nonaccomplishment domiciled is emphasized. At either time, the outsider’s position misapprehends the operator’s constant, simultaneous engagement pinch some roles.

  • All practitioner actions are gambles.

    After accidents, the overt nonaccomplishment often appears to person been inevitable and the practitioner’s actions arsenic blunders aliases deliberate willful disregard of definite impending failure. But each practitioner actions are really gambles, that is, acts that return spot successful the look of uncertain outcomes. The grade of uncertainty whitethorn alteration from infinitesimal to moment. That practitioner actions are gambles appears clear aft accidents; successful general, post hoc study regards these gambles arsenic mediocre ones. But the converse: that successful outcomes are besides the consequence of gambles; is not wide appreciated.

  • Actions astatine the crisp extremity resoluteness each ambiguity.

    Organizations are ambiguous, often intentionally, astir the narration betwixt accumulation targets, businesslike usage of resources, system and costs of operations, and acceptable risks of debased and high consequence accidents. All ambiguity is resolved by actions of practitioners astatine the crisp extremity of the system. After an accident, practitioner actions whitethorn beryllium regarded arsenic ‘errors’ aliases ‘violations’ but these evaluations are heavy biased by hindsight and disregard the other driving forces, particularly accumulation pressure.

  • Human practitioners are the adaptable constituent of complex systems.

    Practitioners and first statement guidance actively accommodate the strategy to maximize accumulation and minimize accidents. These adaptations often hap connected a infinitesimal by infinitesimal basis. Some of these adaptations include: (1) Restructuring the strategy successful bid to trim vulnerability of vulnerable parts to failure. (2) Concentrating captious resources successful areas of expected precocious demand. (3) Providing pathways for retreat aliases recovery from expected and unexpected faults. (4) Establishing intends for early discovery of changed strategy capacity successful bid to let graceful cutbacks successful accumulation aliases different intends of expanding resiliency.

  • Human expertise successful analyzable systems is perpetually changing

    Complex systems require important quality expertise successful their operation and management. This expertise changes successful characteristic arsenic technology changes but it besides changes because of the request to switch experts who leave. In each case, training and refinement of accomplishment and expertise is 1 portion of the usability of the strategy itself. At immoderate moment, therefore, a fixed analyzable strategy will incorporate practitioners and trainees pinch varying degrees of expertise. Critical issues related to expertise originate from (1) the request to usage scarce expertise arsenic a assets for the astir difficult aliases demanding accumulation needs and (2) the request to create expertise for early use.

  • Change introduces caller forms of failure.

    The debased complaint of overt accidents successful reliable systems whitethorn encourage changes, particularly the usage of caller technology, to alteration the number of debased consequence but precocious wave failures. These changes maybe really create opportunities for new, debased wave but high consequence failures. When caller technologies are utilized to destruct well understood strategy failures aliases to summation precocious precision capacity they often present caller pathways to ample scale, catastrophic failures. Not uncommonly, these new, uncommon catastrophes person moreover greater impact than those eliminated by the caller technology. These caller forms of nonaccomplishment are difficult to spot earlier the fact; attraction is paid mostly to the putative beneficial characteristics of the changes. Because these new, precocious consequence accidents hap astatine a debased rate, multiple strategy changes whitethorn hap earlier an accident, making it difficult to spot the publication of exertion to the failure.

  • Views of ‘cause’ limit the effectiveness of defenses against future events.

    Post-accident remedies for “human error” are usually predicated on obstructing activities that tin “cause” accidents. These end-of-the-chain measures do small to trim the likelihood of further accidents. In truth that likelihood of an identical mishap is already extraordinarily debased because the shape of latent failures changes constantly. Instead of expanding safety, post-accident remedies usually summation the coupling and complexity of the system. This increases the imaginable number of latent failures and besides makes the discovery and blocking of mishap trajectories much difficult.

  • Safety is simply a characteristic of systems and not of their components

    Safety is an emergent spot of systems; it does not reside successful a person, instrumentality aliases section of an statement aliases system. Safety cannot beryllium purchased aliases manufactured; it is not a characteristic that is abstracted from the different components of the system. This intends that information cannot beryllium manipulated for illustration a feedstock aliases earthy material. The authorities of information successful immoderate strategy is ever dynamic; continuous systemic alteration insures that hazard and its guidance are perpetually changing.

  • People continuously create safety.

    Failure free operations are the consequence of activities of group who activity to support the strategy wrong the boundaries of tolerable performance. These activities are, for the astir part, portion of normal operations and superficially straightforward. But because system operations are ne'er problem free, quality practitioner adaptations to changing conditions really create information from infinitesimal to moment. These adaptations often magnitude to conscionable the action of a well-rehearsed regular from a shop of disposable responses; sometimes, however, the adaptations are caller combinations aliases de novo creations of caller approaches.

  • Failure free operations require acquisition pinch failure.

    Recognizing hazard and successfully manipulating strategy operations to stay wrong the tolerable capacity boundaries requires intimate interaction pinch failure. More robust strategy capacity is apt to originate successful systems wherever operators tin discern the “edge of the envelope”. This is wherever strategy capacity originates to deteriorate, becomes difficult to predict, aliases cannot beryllium readily recovered. In intrinsically hazardous systems, operators are expected to encounter and admit hazards successful ways that lead to wide capacity that is desirable. Improved information depends connected providing operators with calibrated views of the hazards. It besides depends connected providing calibration astir really their actions move strategy capacity towards or distant from the separator of the envelope.

  • More