I remember many years ago, when I was a student, we had a discussion with some computer science and electronics students. Most of them believed that our professions weren’t that important and that a mistake you might make wouldn’t cost someone their life. I disagreed back then and even brought up the example of designing medical devices. I didn’t have a specific example in mind, but in the end, my theory turned out to be true… and in fact, 15 years before I had that discussion with my classmates
The Therac-25
The Therac-25 was developed in the 1980s by the company AECL (Atomic Energy of Canada Limited). It was the third model in a series of medical radiotherapy machines and provided treatment through two modes of radiation: electrons and X-rays. The system differed from previous models as it was the first to rely so heavily on its software for safety, whereas earlier models also used mechanical safety interlocks
How the software cost lives
The Therac-25 had serious software bugs that allowed excessively high doses of radiation to be administered to patients. This occurred mainly due to the following issues:
- Software design failure: The machine used a combination of software and hardware to regulate radiation doses. However, the software contained errors that could lead to overdoses without triggering the mechanical safety systems.
- Non-existent or insufficient safety checks: The design of the Therac-25 included software that had not been adequately tested, and there were not enough checks to ensure the machine was functioning properly. A critical omission was the lack of compensating mechanical safety mechanisms, as the system relied too heavily on the proper functioning of the software.
- Interaction with the operator: The users of the Therac-25, typically medical technicians, could unintentionally trigger the error by making rapid changes to the machine's settings. Due to a bug in the keyboard handling, the machine could start without the correct parameters for the radiation.
- Unclear and unhelpful error feedback: When errors occurred, the machine provided vague messages to the operators, which didn’t explain the risk. The technicians didn’t understand the severity of the errors and often ignored the messages.
These problems had disastrous results. Between 1985 and 1987, at least six incidents were recorded where patients received excessive radiation doses, resulting in severe burns, permanent injuries, and in some cases, death. The doses given to these patients were up to 100 times higher than the correct therapeutic dose.
How we improved software since then?
The failure of the Therac-25 forced manufacturers to tighten the software testing process, especially for software related to human life.
Error codes now provide clear information to operators so they know exactly what went wrong and why.
The Therac-25 remains a cautionary example of the dangerous consequences of inadequate design and insufficient testing in critical applications, and it is often studied in software engineering schools.