by Robert Franklin
Harry Yoder is a well-known figure on the Silicon Valley University campus. The Samuel Southerland Professor of Computer Technology and Ethics, he has written numerous articles and texts on ethics and the social impact of computers. His courses are very popular, and most of his courses are closed long before the end of the registration period. Dr. Yoder received his Ph. D. in electrical engineering from the Georgia Institute of Technology in 1958. In 1976 he received a Master of Divinity degree from the Harvard Divinity School. In 1983 he received an MS in Computer Science from the University of Washington. He joined the faculty at Silicon Valley University in 1988.
I interviewed Dr. Yoder in his office on campus. My purpose was to get his reaction to the case of the killer robot and to "pick his brain" about the ethical issues involved in this case.
Sentinel-Observer: Going from electrical engineering to the study of religion seems like quite a jump.
Yoder: I was an electrical engineer by profession, but all human beings have an inner life. Don't you?
Sentinel-Observer: Yes.
Yoder: What is your inner life about?
Sentinel-Observer: It's about doing the right thing. Also, it's about achieving excellence in what I do. Is that what sent you to Harvard Divinity School? You wanted to clarify your inner life?
Yoder: There was a lot going on at the Divinity School, and much of it was very compelling. However, most of all I wanted to understand the difference between what was right and what was wrong.
Sentinel-Observer: What about God?
Yoder: Yes, I studied my own Christian religion and most of the major world religions, and they all had interesting things to say about God. However, when I discuss ethics in a forum such as this, which is secular, or when I discuss ethics in my computer ethics courses, I do not place that discussion in a religious context. I think religious faith can help a person to become ethical, but on the other hand, we all know that certain notorious people who have claimed to be religious have been highly unethical. Thus, when I discuss computer ethics, the starting point is not religion, but rather a common agreement between myself and my students that we want to be ethical people, that striving for ethical excellence is a worthwhile human endeavor. At the very least, we do not want to hurt other people, we do not want to lie, cheat, steal, maim, murder and so forth.
Sentinel-Observer: Who is responsible for the death of Bart Matthews?
Yoder: Please forgive me for taking us back to the Harvard Divinity School, but I think one of my professors there had the correct answer to your question. He was an elderly man, perhaps seventy, from Eastern Europe, a rabbi. This rabbi said that according to the Talmud, an ancient tradition of Jewish law, if innocent blood is shed in a town, then the leaders of that town must go to the edge of the town and perform an act of penance. This was in addition to any justice that would be meted out to the person or persons who committed the murder.
Sentinel-Observer: That's an interesting concept.
Yoder: And a truthful one! A town, a city, a corporation -- these are systems in which the part is related to the whole and the whole to the part.
Sentinel-Observer: You are implying that the leaders at Silicon Techtronics, such as Mike Waterson and Ray Johnson, should have assumed responsibility for this incident right from the start. In addition, perhaps other individuals, such as Randy Samuels and Cindy Yardley, bear special burdens of responsibility.
Yoder: Yes, responsibility, not guilt. Guilt is a legal concept and the guilt or innocence of the parties involved, whether criminal or civil, will be decided in the courts. I guess a person bears responsibility for the death of Bart Matthews if his or her actions helped to cause the incident -- it's a matter of causality, independent of ethical and legal judgments. Questions of responsibility might be of interest to software engineers and managers, who might want to analyze what went wrong, so as to avoid similar problems in the future.
A lot of what has emerged in the media concerning this case indicates that Silicon Techtronics was a sick organization. That sickness created the accident. Who created that sickness? Management created that sickness, but also, employees who did not make the right ethical decisions contributed to the sickness.
Randy Samuels and Cindy Yardley were both right out of school. They received degrees in computer science and their first experience in the working world was at Silicon Techtronics. One has to wonder whether they received any instruction in ethics. Related to this is the question as to whether either of them had much prior experience with group work. Did they, at the time that they were involved in the development of the killer robot, did they see the need to become ethical persons? Did they see that success as a professional requires ethical behavior? There is much more to being a computer scientist or a software engineer than technical knowledge and skills.
Sentinel-Observer: I know for a fact that neither Samuels nor Yardley ever took a course in ethics or computer ethics.
Yoder: I suspected as much. Let's look at Randy Samuels. Based upon what I've read in your newspaper and elsewhere, he was basically a hacker type. He loved computers and programming. He started programming in junior high school and continued right through college. The important point is that Samuels was still a hacker when he got to Silicon Techtronics and they allowed him to remain a hacker.
I am using the term "hacker" here in a somewhat pejorative sense and perhaps that is not fair. The point that I am trying to make is that Samuels never matured beyond his narrow focus on hacking. At Silicon Techtronics, Samuels still had the same attitude toward his programming as he had in junior high school. His perception of his life and of his responsibilities did not grow. He did not mature. There is no evidence that he was not trying to develop as a professional and as an ethical person.
Sentinel-Observer: One difficulty, insofar as teaching ethics is concerned, is that students generally do not like being told "this is right and that is wrong."
Yoder: Students need to understand that dealing with ethical issues is a part of being a professional computer scientist or software engineer.
One thing that has fascinated me about the Silicon Techtronics situation is that it is sometimes difficult to see the boundaries between legal, technical and ethical issues. Technical issues include computer science and the management issues. I have come to the conclusion that this blurring of boundaries results from the fact that the software industry is still in its infancy. The ethical issues loom large in part because of the absence of legal and technical guidelines.
In particular, there are no standard practices for the development and testing of software. There are standards, but these are not true standards. A common joke among computer scientists is that the good thing about standards is that there are so many to choose from.
In the absence of universally accepted standard practices for software engineering, there are many value judgments, probably more than in other forms of production. For example, in the case of the killer robot there was a controversy concerning the use of the waterfall model versus prototyping. Because there was no standard software development process, this became a controversy, and ethical issues are raised by the manner in which the controversy was resolved. You might recall that the waterfall model was chosen not because of its merits but because of the background of the project manager.
Sentinel-Observer: Did Cindy Yardley act ethically?
Yoder: At first, her argument seems compelling: she lied, in effect, to save the jobs of her coworkers and, of course, her own job. But, is it ever correct to lie, to create a falsehood, in a professional setting?
One book I have used in my computer ethics course is Ethical Decision Making and Information Technology by Kallman and Grillo.1 This book gives some of the principles and theories behind ethical decision making. I use this and other books to help develop the students' appreciation for the nature of ethical dilemmas and ethical decision making.
Kallman and Grillo present a method for ethical decision making and part of their method involves the use of five tests: the mom test, would you tell your mother what you did; the TV test, would you tell a national TV audience what you did; the smell test, does what you did have a bad smell to it; the other person's shoes test, would you like what you did to be done to you, and the market test, would your action be a good sales pitch?
What Yardley did fails all of these tests -- I think nearly everyone would agree. For example, can you imagine Silicon Techtronics using an ad campaign that runs something like this:
"At Silicon Techtronics, the software you get from us is bug free, because even if there is a bug, we will distort the test results to hide it, and you will never know about it. Ignorance is bliss!"
This shows that apparent altruism is not a sufficient indicator of ethical behavior. One might wonder what other unstated motives Ms. Yardley had. Could it be that personal ambition led her to accept Ray Johnson's explanation and his assurance that the robot was safe?
Sentinel-Observer: Are there any sources of ethical guidance for people who are confronted with an ethical dilemma?
Yoder: Some companies provide ethical guidelines, in the form of corporate policies, and there is such a document at Silicon Techtronics, or so I am told. I haven't seen it. An employee could also refer to ethical guidelines provided by professional societies, such as the ACM. Beyond that, he or she could read up on the subject to get a better feel for ethical decision making. Of course, one must always consult with one's conscience and innermost convictions.
Sentinel-Observer: Did Randy Samuels act ethically?
Yoder: Stealing software the way that he did was both unethical and illegal.
I think the most important issue with Randy Samuels has never been discussed in the press. I truly doubt that Samuels had the requisite knowledge that his job required. This kind of knowledge is called domain knowledge. Samuels had a knowledge of computers and programming, but not a very strong background in physics, especially classical mechanics. His lack of knowledge in the application domain was a direct cause of the horrible accident. If someone knowledgeable in mathematics, statistics and physics had been programming the robot instead of Samuels, Bart Matthews would probably be alive today. I have no doubt about that. Samuels misinterpreted the physics formula because he didn't understand its meaning and import in the robot application. It may be that management is partly responsible for the situation. Samuels might have told them his limitations and management might have said, "What the hell!"
Samuels had difficulty with group work, peer reviews and egoless programming. It is possible that he was trying to hide his lack of expertise in the application domain?
Sentinel-Observer: Did Ray Johnson act ethically?
Yoder: This 'Ivory Snow' business! The trouble with the Ivory Snow theory is that it was just a theory. If it were more than a theory and an actual methodology for keeping the likelihood of failure within statistically determined limits, like what is called "clean room software engineering," then there would be less culpability here.
Based upon the information that I have, the Ivory Snow theory was just a rationalization for getting flawed software out the door to customers on time. The Ivory Snow theory is only valid, ethically and professionally, if the customer is told of known bugs, or impurities, if we can use the soap jargon. In the case of Silicon Techtronics the Ivory Snow theory worked like this: we know it's not pure, but the customer thinks it is!
Of course, coercing Cindy Yardley the way Ray Johnson did was also not ethical. Did he believe what he told Ms. Yardley, namely that the robot was safe, or was that an out and out lie? If he believed that the robot was safe, why cover up with the false tests? If the user interface were so important as a last line of defense, why avoid more rigorous tests of the user interface?
Sentinel-Observer: What is your view of Mike Waterson in all this?
Yoder: If Johnson is the father of the Ivory Snow theory, Waterson is the grandfather. His demand that the robot be completed by a certain date or "heads would roll" might have caused Johnson to formulate the Ivory Snow theory. You see, it is apparent that Johnson thought that the delivery of Robbie to Cybernetics by the specified date was impossible unless the robot software had bugs.
In many regards I feel that Waterson acted unethically and irresponsibly. He placed Sam Reynolds in charge of the robot project, yet he, Reynolds, lacked experience with robots and modern user interfaces, Reynolds rejected the idea of developing a prototype, which might have allowed for the development of a better user interface.
Waterson created an oppressive atmosphere for his employees, which is unethical in itself. Not only did he threaten to fire everyone in the Robotics Division if the robot was not completed on time, he "eavesdropped" on private electronic mail communications throughout the corporation, a controversial right that some companies do claim.
My personal belief is that this kind of eavesdropping is unethical. The nature of e-mail is somewhat of a hybrid of normal mail and a telephone conversation. Monitoring or spying on someone else's mail is considered unethical, as is tapping a telephone. Indeed, these activities are also illegal under almost most circumstances. So, I believe it is an abuse of power to monitor employees the way that Waterson did.
Sentinel-Observer: Does the prosecutor have a case here?
Yoder: Against Randy Samuels?
Sentinel-Observer: Yes.
Yoder: I doubt it, unless she has information that has not been made public thus far. Manslaughter, to my understanding, implies a kind of reckless and irresponsible act, causing death of another. Does this description apply to Samuels? I think the prosecutor's best bet is to stress his lack of knowledge in the application domain if it can be shown that he did engage in a deliberate deception.
I read last week that 79% of the people favor acquittal. People are inclined to blame the corporation and its managers. Last night, one of the network news anchors said, "Samuels isn't a murderer, he's a product of his environment."
Sentinel-Observer: Could you restate your position on the matter of ultimate responsibility in the case of the killer robot?
Yoder: In my mind, the issue of individual versus corporate responsibility is very important. The corporation created an environment in which this kind of accident could occur. Yet, individuals, within that system, acted unethically and irresponsibly, and actually caused the accident. A company can create an environment which brings out the worst in its employees, but individual employees can also contribute to the worsening of the corporate environment. This is a feedback loop, a system in the classical sense. Thus, there is some corporate responsibility and some individual responsibility in the case of the killer robot.
Sentinel-Observer: Thank you, Professor Yoder.
----------------------------------------------------------
1 This is an actual textbook from McGraw-Hill.