On March 21, 1986, a patient at the East Texas Cancer Center in Tyler, Texas, lay face down for his ninth radiation treatment, follow-up after doctors removed a tumor from his back. In the next room, an operator typed his prescription into the console of a Therac-25, a radiation accelerator. She entered the wrong treatment mode, caught it, and corrected it within seconds. The console locked up with a Malfunction 54 message and reported six monitor units delivered, against the two hundred and two she had ordered. She had seen that cryptic stop before, so she pressed the key marked P, for proceed. He later said he felt an electric shock, that his hand felt like it was leaving his body, and got up to pound on the treatment-room door. He had absorbed sixteen thousand five hundred to twenty-five thousand rads in under a second, over an area the width of a thumb. Paralysis spread through his arm, legs, and diaphragm over the next months; he died five months after the accident.
The reading on the console came from a real instrument: an ion chamber that measured the radiation the machine produced and reported six monitor units instead of two hundred and two. A timing flaw explains both numbers. An operator who corrected the treatment mode quickly enough, within the eight seconds the magnets took to reset, could change what the screen showed without one part of the program registering it, so the beam fired at full strength, unscanned, instead of the lower, spread-out exposure prescribed. The ion chamber, built to read that weaker beam, saturated under the stronger one and underreported it. The Therac-25's immediate predecessor, the Therac-20, had an independent hardware circuit watching the scanning mechanism itself, plus mechanical interlocks, apart from any software or sensor reading. Building the newer accelerator, the manufacturer relied on software for that check instead, so everything, the ion chamber included, ran through the program the flaw had already compromised.
The Therac-25 had misreported its own treatment before. Eight months earlier, at a hospital in Hamilton, Ontario, the same model had displayed no dose delivered while a patient absorbed somewhere between thirteen and seventeen thousand rads; the manufacturer blamed a faulty switch it could never reproduce. A 1983 safety analysis assigned the computer's selecting the wrong mode a probability of four in a billion, unjustified and, investigators found, modeled on no real account of how software fails. The whole control program had been written by one programmer, who left the company in 1986; AECL's own deposed employees could not describe his training, and the case settled before he could be questioned himself.
By January 1987, the Therac-25 had caused six known accidents at four clinics in the United States and Canada. Three patients died of their injuries. The machine was recalled that year, and the corrective design restored an independent hardware check on the beam, the safeguard software alone had been trusted to replace.
A console, a printout, a certificate: each one is a claim a system makes about itself, not proof, even with a sensor behind it. The Therac-25 had a sensor, and it failed the same way the software did, through the same beam. A number only means something once a check that could fail differently has looked at it too.