Two Engines, One Batch

It was the last flight out of Helsinki on a Friday, departure near midnight. I did what I always do. I was asleep in two minutes.

I woke some time later, not to a noise but to the absence of one. The aircraft had a small, wrong tilt, and a quietness where there should not have been quiet. The seatbelt sign was on. I looked at the map on the seatback screen and saw we had turned. We were no longer pointed home. We were heading back.

After some long minutes, the captain came on. A problem with a fuel filter feeding the left engine. It had become blocked and was choking the fuel flow, so we would return to Helsinki for service. Calm voice, routine words. The cabin took it well.

I did not take it well, and not for the reason you would think. I was not afraid of a single stuck filter. That is precisely what the second engine is for. An airliner has two engines so one can fail, and you still fly, land, go home, and complain about the delay.

A single filter giving out over the dark forests south of Oulu is, on its own, an inconvenience, not a catastrophe. The aircraft's whole architecture assumes it. What kept me awake was the question underneath the captain's calm one. I lay there in the dark doing the only thing I know how to do with a problem. Take it apart. A filter clogged. Fine. The reason mattered more than the fact.

I wanted to know if the reason lived only in that one filter, or somewhere both filters could reach. Were the two filters, left and right, from the same manufacturing batch? If they were, whatever tolerance drifted or whatever flaw slipped through quality on the left was sitting, unbudged and identical, in the right.

There was a second question. Did both engines draw, somewhere upstream, from a common fuel supply? If there was something in the fuel itself- water, a contaminant, particulate from a single tank- it was not feeding one filter its problem. It was feeding both. In that case, the reassuring fact of two engines was not two at all. It was one failure mode, arriving at two filters at slightly different rates, wearing the costume of redundancy. That is the thought that will keep an engineer awake at thirty thousand feet.

A book names this fear precisely, and everyone who builds or runs anything complex should read it. Charles Perrow's Normal Accidents studied disasters in nuclear plants, chemical works, and aircraft. His central, uncomfortable finding was that in tightly coupled complex systems, failure is not an aberration to be engineered away. It is normal, built into the structure. One way it hides is the way it hid on my flight.

We add redundancy to feel safe, and the redundancy quietly shares a cause we never mapped. A single event takes down both copies at once. Reliability engineers have a plain name for it. Common mode failure.

The mathematics of redundancy is seductive, and it is conditional. Put two components in parallel, each ninety-nine percent reliable, and together they are 99.99 percent reliable, a hundredfold reduction in the chance of failure. It is a wonderful number.

It is also true only as long as the two components stay independent. Let them share a batch, a supplier, a power rail, a fuel tank, a firmware bug, and the multiplication collapses. The reliability drops back toward ninety-nine, or worse, because complexity has been added along with confidence that was never earned.

Earned confidence is the whole game. This should trouble you more than the filter troubled me. The pilot of a single-engine aircraft knows exactly what he has. He watches that one engine like it is the only thing between him and the sea, because it is.

The danger is not in the honest single point of failure you can see. The danger is in the redundancy you think you have. The second engine makes everyone stop looking, right up until the shared cause arrives and reveals there was only ever one.

I have spent the years since that flight finding this pattern on the ground, where it is quieter, and no captain announces it. Two sensors qualified for a critical measurement, from the same production lot. Two suppliers, dual-sourced for safety, both buying the same subcomponent from the same single factory in the same town.

Redundant controllers running identical firmware, so the identical bug sleeps in both. Two inspectors trained by the same person, carrying the same blind spot in perfect stereo. A backup, stored in the same building as the original.

Each one is two engines and one batch. Each one counts as redundancy on a slide and a single point of failure in the world. So I have learned not to count redundancies. Counting is what fools you. We have two; we are covered. The only question that means anything is the one I lay awake asking over the dark forests that night. Not do we have two. Do the two share a cause?

Trace the independence back, past the two things you can see, to the batches and tanks and suppliers and assumptions feeding them. Redundancy is not a number of copies. It is a promise about independence, and the promise is only as good as the point where the two paths secretly rejoin.

We landed back in Helsinki. The service took three hours. Then we lifted off again into an early morning sky and flew home without incident.

The second engine, it turned out, was independent enough that night, or the trouble had truly been one filter and nothing more. I never found out which, and it hardly matters. What I brought home was not the delay. It was a question I have asked of every safe-looking system since.

Two of them. Good. From how many batches?

Kauko Väinämö, CEO, Ginolis.