Oct 11, 2011Dr. David Koepsell, TPM - Philosophy

Ethical aspectsof risks & hazards of technology

1. Design

2. Risk assessment

3. Risk communication

4. Perception

What is risk?

• One definition of “risk” is:

• “Exposure to the chance of injury or loss;

a hazardous or dangerous chance” *

• This definition involves both

• the probability of an event occurring

• the consequences of the event

* Webster’s Dictionary

An engineering definition of risk:

Risk = ×(probability of event) (consequences)

Risk is inherent in engineering

• All engineering involves risk.

• Innovation in design generally increases risk. More generally, any change (from proven practice) will often increase risk.

• Examples:• Tacoma Narrows Bridge--1940 collapse• Three Mile Island Power Plant--1979 radiation

release• Concorde airliner--2000 crash in Paris

Probability of failure

• A nuclear reactor will “meltdown” if the control rods fail and the cooling pump fails. What is the probability of this occurring?

Event tree analysis of failure probability

Ethical aspects of risks & hazards of technology

• Design:

- Making choices

- Professional codes

- Recognisability

- Ignorance

- Modeling

Ethical/regulatory questions: how ought we to design, and who is responsible for faults?

Ethical aspects ofrisks & hazards of technology

• Risk assessment:

- Assumptions

- Choices (of theory, model , impact…)

- Societal, cultural, legal context

Ethical aspects of risks & hazards of technology

• Communication:

- Professional code

- Specialist

- Credibility

- How much, when, who?

- Mode of presentation

Ethical aspects of risks & hazards of technology

• Acceptability:

- If risk(1)= chance(1)xeffect(1)= R

and risk(2)= chance(2)xeffect(2)= R,

then risk(1)=risk(2) ?

- Perception

- Voluntary versus involuntary (imposed)

Frameworks for decidingon acceptability

• Market regulation

- Invisible hand

- Right of free choice?

- Pareto improvement

- Negotiations; consumers not involved…

Frameworks for deciding onacceptability

• Norms

- Government interference

- Paternalism

- Choice of (natural) standards…

- Revealed preferences

Frameworks fordeciding acceptability

• Risk-cost-benefit analysis:

- Objective evaluation?

- Exploitation of some by others

- Disadvantages:

- Ignorance of relevant consequences

- Ditto plus uncertainty for new technology

Frameworks for deciding acceptability

• Precautionary approach (principle):

When indication that certain hazard(s) cannot be proven satisfactorily on scientific basis

- Reverses burden of proof

Implications:

- Obligation to monitor (possible) negative impacts

- Consequential financial liability

Frameworks for deciding acceptability

• Technology as experiment;

Informed consent

- Hazards emerge during practical use

- All people possibly affected must voluntarily consent to (the imposed) risks and hazards after having been informed

Acceptable risk...

• What is an acceptable risk?

Some acceptable risks...

• Note that the average American could, if he/she chose, reduce his/her annual risk of death by 173×10-6 by avoiding travel in automobiles or on highways. Since the average American chooses to accept this risk (because of the advantages of automobile transportation), the risk of death associated with automobile travel could be considered an “acceptable risk”, that is one assumed by a reasonable person.

• Similarly, the 8×10-6 risk of death in commercial aviation is accepted by most persons.

Cost-benefit risk assessment example

• The government is proposing legislation to limit formaldehyde emissions to 3 ppm. Industry estimates that to install and operate the necessary scrubbers will cost $300 million annually. Toxicologists estimate that this new standard will save 30 lives annually. Using cost/benefit analysis, should the new standard be implemented?

• Cost = $300 million/yr

• Benefit = (30 lives/yr)($ ??? / life)

• What is the dollar value of human life?

What is the value of human life ?

• Some methods to place a value on human life

• purchasing decisions involving safety (e.g. car purchase)

• future earnings

• extra pay needed for risky jobs

(e.g. house painter vs. smokestack painter)

Problems with using studies of purchasing decisions to determine the value of life...

• wealthy people are willing to pay more

• people will pay 7 times more to reduce risk of cancer than to reduce risk of death in an automobile

• decisions are based on perceptions (values)

• women value their lives more than men, i.e., men are more willing to engage in risky behavior

• A 1984 study by Shualmit Kahn indicates that people typically valued their lives at $8 million (Note: this figure is higher than is typically used in public policy analysis. Also note that Ford used $0.2 million in the 1970’s Pinto case study.)

Public Policy Expert’s Approach to Risk

• His/her first priority is to protect the public.

• Consider the consequences of an error in a study to determine whether a chemical is carcinogenic…• False Positive The chemical is banned as being

carcinogenic, when in reality it is not. The producer loses potential profits from the sale of this chemical.

• False Negative A dangerous chemical is approved as safe and sold to the general public. The death rate from cancer increases.

• A public policy expert will choose to err on the side of public safety, when the facts are not clear

Public policy expert approach (cont’d).

• In a democracy, the government policy makers respond to the public’s wishes. The public tends to react to different risks in different, and sometimes irrational ways. As a result, we tend to allocate differing amounts of money to save lives by different measures...

Allocation of Money

Layman’s approach to risk

• Respect for Persons Approach

• Key Issues:

• is the risk distributed equitably?

• are those assuming the risk compensated?

• is the risk voluntary?

• does the person assuming the risk understand it?

• does the person assuming the risk have control?

Layman’s approach to risk...

• Laymen often overestimate low probability risks

• Willing to accept higher voluntary risks than involuntary risks (by factor of 103)

• Laymen don’t compare a risk to already accepted risks

• Laymen overestimate risks of human origin compared to risks of natural origin

• Laymen’s approach more closely follows Respect-for-Persons approach than the Utilitarian approaches used by many experts

An Acceptable Risk is one that is...

• freely assumed with informed consent

• equitably distributed

• properly compensated

Informed Consent• RP says we should treat people as “moral

agents” (autonomous, self-governing individuals)…thus we should seek “informed consent” before assigning risk

• Criteria for informed consent• consent must not be coerced*

• person must be accurately informed*

• person must be competent* to assess information

*there are possible conceptual and applications

issues to be resolved

Problems with informed consent

• difficulty getting informed consent

• consent must be obtained before the risk is assumed

• consent requires negotiation

• holdouts or unreasonable preferences

• parties must be well informed and reasonable

• people are often hysterical regarding dramatic or

catastrophic risk

• people underestimate the consequences of risks that

have never happened before

When it isn’t possible to get informed consent...

• Only expose people to risks they would consent to, if they were informed of all known risks.

Or, ...

• As an alternative to gaining consent from everyone affected by the risk, the group leaders can decide to accept the risk for the group.

• Some people may give informed consent to things that are not in their interests, because of...

• misunderstanding information

• immaturity

• irrationality

• Such consent isn’t autonomous.

Problems with Informed Consent (cont’d.)

Problems with Informed Consent (cont’d.)

• If consent is not autonomous, then you should find a way to make consent autonomous.

Risk concepts--Example

• The electric power company proposes to build a nuclear power plant near your neighborhood. Given the newly deregulated electricity market, the power probably will be sold out of state because prices are higher there.

Risk concepts--Example (cont’d.)

• Is the risk voluntary?

• Does the person taking the risk understand it?

• Does the person taking the risk have control?

• Is the risk distributed equitably?

• Do those taking the risk get the rewards?

Risk concepts--Example (cont’d.)

• Is the risk voluntary?• Yes, within the limits of the democratic

process.

• Does the person taking the risk understand it?• No, the general public does not understand

nuclear energy.

• Does the person taking the risk have control?• No, the power company controls the plant.

Risk concepts--Example (cont’d.)

• Is the risk distributed equitably?

• No, those living close to the plant take a higher risk

• Do those taking the risk get the rewards?

• No, the power is shipped out of state.

Informed Consent by Group Leaders--Example

• The XYZ Chemical Company wants to build a new plant in Smallville. The chemical plant has a pollution effluent that may give one citizen cancer every five years. However, the plant will create 100 new jobs and a substantial tax base for Smallville, which will improve the local schools and hospital. The XYZ Chemical Company asks the town council for approval to build the plant in the industrial park.

Informed Consent by Group Leaders (cont’d.)

• Advantages:

• simplifies decision-making process

• Problems:

• How do we compensate those individuals who suffer the consequences of the risk?

• Approval of group leaders does not reflect the wishes of all individuals

• Works okay for small risks, but large risks may need individual consent

Paternalism

• Paternalism: the exercise of power by one person or institution over another in order to help or prevent harm to the latter, when...

• Weak paternalism--the latter is not exercising moral agency effectively.

• Strong paternalism--there is no reason to believe the latter is not effectively exercising moral agency.

• Commonly-accepted criterion for acceptable paternalism:

• A fully rational person informed of the relevant facts would consent to intervention in this case

• Paternalism often causes resentment.

• Paternalism (weak) is permissible if protected person is not autonomous

• but people will disagree over who is autonomous.

Paternalism (cont’d)

Summary

• Be aware that experts tend to use a utilitarian approach and the lay public tends to use a respect-for-persons (RP) approach

• Utilitarian and RP approaches each have their limitations

• It is difficult to quantify risk

• Peoples’ values differ regarding risk

• Promote informed consent within your limits as an engineer

For guidance...

• “People should be protected from the harmful effects of technology, especially when the harms are not consented to or when they are unjustly distributed, except that this protection must sometimes be balanced against (1) our need to preserve great and irreplaceable benefits and (2) the limitations on our ability to obtain informed consent.” Harris, et al.

Summary (cont’d.)

• Some technologies provide valuable and irreplaceable benefits, yet are inherently risky (e.g. automobiles)

• Engineers should be paternalistic and protect the public from harmful impacts of technology if:

• Consequences are severe

• Consequences are unjustly distributed

• Informed consent is not possible

Risk perception

• What is perception?

• It is not necessarily equal to “physical” observation

• A “filter” exists between the “strict” process of observing and the image one perceives

• The filter can be individual or collective

Risk perception

• Why is this concept relevant ?

• Ethical considerations are used to resolve dilemmas concerning societal, complex problems and issues whereby nature, applied science and technology play a role

• Various societal actors perceive these problems and roles differently and often change over the course of time

Perceptual filters

1. Hierarchists:

Ministries (introducing laws),

Regulatory permit granting bodies,

controlling authorities

2. Egalitarians:

Enviromentalists,

consumer interest groups

Motto: “If you cannot “prove” it is safe, assume

it is not!”

Perceptual filters

3. Individualists:The people who are prepared to take risks,entrepreneurs;Motto: “If you cannot prove that it is unsafe,

you must assume that it is safe”

4. Fatalists:The “man in the street”, who feels he has no influence or impact on the big societal problems and issues

Questions

• Which risks arise to ethical concerns, and which are not?

• How do we distinguish?

• Are physical harms always to be avoided? What percentage of which kinds of harms are acceptable?

• How much of a duty do we have to avoid which risks?

• Whose responsibilities extend to which risks?

• Personal vs. Paternalism?

Conclusions and implications

Conclusions

• Risks and hazards inherent in newly developed technology; ignorance and uncertainty

• Confrontation with ethical issues

• Decision on acceptability does not exclusively

and directly follow from risk assessment

• Various frameworks for deciding on acceptability

Conclusions and implications

Implications• Competence to carry out tasks within socially made agreements• Be conscious of experimental nature of technological

developments• Play part in informing those involved, affected, interested• Moral responsibility for organisation and execution of

technological projects• Have insight and foresight in required social changes required to

facilitate the implications