Case Study in Software Engineering

Case Study About Testing: George and the Jet

George Babbage is an experienced software developer working for Acme Software Company. Mr. Babbage is now working on a project for the U.S. Department of Defense, testing the software used in controlling an experimental jet fighter. George is the quality control manager for the software. Early simulation testing revealed that, under certain conditions, instabilities would arise that could cause the plane to crash. The software was patched to eliminate the specific problems uncovered by the tests. After these repairs, the software passed all the simulation tests.

George is not convinced that the software is safe. He is worried that the problems uncovered by the simulation testing were symptomatic of a design flaw that could only be eliminated by an extensive redesign of the software. He is convinced that the patch that was applied to remedy the specific tests in the simulation did not address the underlying problem. But, when George brings his concerns to his superiors, they assure him that the problem has been resolved. They further informed George that any major redesign effort would introduce unacceptable delays, resulting in costly penalties to the company.

There is a great deal of pressure on George to sign off on the system and allow it to be flight-tested. It has even been hinted that, if he persists in delaying the system, he will be fired. What should George do next?

Relevant Clauses

Principle 1: PUBLIC

Software engineers shall act consistently in the public interest. In particular, software engineers shall, as appropriate: Approve software only if they have a well-founded belief that it is safe, meets specifications, passes appropriate tests, and does not diminish the quality of life, privacy, or harm the environment. The ultimate effect of the work should be to the public. good. Disclose to appropriate persons or authorities any actual or potential danger to the user, the public, or the environment, that they reasonably believe to be associated with software or related documents.

Principle 2: PRODUCT

Software engineers shall ensure that their products and related modifications meet the highest professional standards possible. In particular, software engineers shall, as appropriate: Ensure adequate testing, debugging, and review of software and related documents on which they work.

Principle 3: MANAGEMENT

Software engineering managers and leaders shall subscribe to and promote an ethical approach to the management of software development and maintenance. In particular, those managing or leading software engineers shall, as appropriate: Ensure good management for any project on which they work, including effective procedures for the promotion of quality and reduction of risk. Do not ask a software engineer to do anything inconsistent with this Code.

Applying the Code

In this case, Mr. Babbage must contend with issues of physical safety that are dependent on software reliability. If we look at this case too narrowly, we might think that the safety of the test pilot is the exclusive safety concern. Although Mr. Babbage does have responsibilities towards the pilot, test pilots know about the risks inherent in their profession, and the test pilot may be quite willing to fly the plane despite Babbage’s misgivings.

However, the test pilot is not the only one endangered if the software is faulty; anyone under the plane is endangered if things go awry. Especially if the test flight might fly over populated areas (and remember that instability might lead the plane in unplanned directions before crashing), many people under the plane are unlikely to have given their consent to “testing” the software. Carl’s responsibilities to those people are a vital part of our analysis.

Clause 1.03 makes public safety a priority concern for a software engineer. It is exactly this concern that is central to George’s decision. George recognizes this obligation, and the obligation in clause 1.04 to disclose his professional opinion that the software has not been sufficiently certified as safe. Unfortunately, George’s superiors have not supported his decision about the software, and are trying to convince him to sign off on the software despite his reservations.

His superiors have put George in a difficult position. The Code sections above confirm Mr. Babbage’s ethical duty to refuse to sign off on the software before he is reasonably sure of its safety. We note that for almost all complex software, we can never be entirely sure software is reliable and safe. It is a professional judgment whether or not the software is “safe enough.” By pressuring George to sign off, his superiors are forcing George to choose between his loyalty to his employers (and his continued employment) and his obligation to public safety. It is hoped that the existence of, and support for, an effective ethics code can help someone in this position; but it is still difficult.

So far our analysis has concentrated on Mr. Babbage and his dilemma. However, the Joint Code also requires his managers to act ethically. The clauses in section 5 of the Code prohibit managers from forcing a software engineering employee to violate the code. The Code. also makes managers responsible for ensuring that there are processes to ensure the reduction of risks. The managers might object that they have adequate processes and that the process was followed. Simulation testing revealed problems, and those problems were addressed. The managers are not convinced that Mr. Babbage’s suspicions are well founded, and are not willing to jeopardize the project based on his misgivings.

The wording of clause 1.03 in the Code is an important part of our analysis of this case. That clause states that software engineers should approve software only if they have a “well-founded belief that it is safe” (our emphasis). The idea of a well-founded belief is key to the dispute between George and his superiors. Perhaps George is right about the software, but perhaps his managers are right.

Although the case does not offer many details about George’s misgivings, he did not present sufficient evidence to his superiors about the remaining problems in the software. (If the managers were convinced about the seriousness of the remaining problems, it seems unlikely that they would approve a test flight that would likely end in a costly disaster.) Perhaps this dilemma could be resolved to the satisfaction of all parties if the managers agreed to a short-term delay, not for a major redesign, but for further testing to either confirm George’s suspicions or convince George that the managers are correct and that the test flight should go on.

This resolution would be far better than George signing off on a system he thinks is deficient, and far better than George being fired for not doing so. The standard supported by the Code is to have the burden to demonstrate that the software is safe before deployment instead of having to prove it unsafe before deployment is halted.