6 Stage 4 — Risk evaluation

When all single and common cause failures have been analysed, a list of the most serious risks can be made. The Raster tool assists the initial effort for this stage. Quick wins can be determined automatically, and simple “what if” analysis is available.

During this stage you make a judgement of which risks you consider to be too large. You write down arguments for your choice and propose risk treatments. You take into consideration both your assessment of vulnerabilities that affect the availability of telecom services and your expectation of reactions of other stakeholders to risk treatments. The result is a report to the sponsor, outlining, explaining and justifying your recommendations.

The Risk Evaluation stage consists of the following steps:

1. Determine longlist
2. Reduce longlist to shortlist
3. Make treatment recommendations, considering social risk factors
4. Prepare final report

6.1 Determine longlist

Based on the information presented by the Raster tool (see Analysis view), a longlist of the most serious risks must be compiled. These risks are:

• the combination of a single vulnerability and a single component, such as “power failure at the PABX”, or
• the combination of a single common cause failure vulnerability and a single cluster, such as “fire at equipment in the facilities room”.

It is up to the analysts to judge which risks are serious enough to be placed on the longlist. However, the list should include the “quick wins” reported by the Raster tool (see Failures and vulnerabilities). Quick wins are those vulnerabilities that by themselves determine the overall vulnerability level of a component. Reducing that vulnerability would immediately reduce the overall level.

Other good candidates for inclusion on the longlist are those risks that were computed as Extremely high or High, as well the risks that were computed as Ambiguous or Unknown.

6.2 Reduce longlist to shortlist

The longlist must be prioritised. Prioritisation requires more information than can be found in the diagrams and vulnerability assessments. For example, information on control relationships between components, or information about redundancy, cannot be found in the diagrams but is very important for risk prioritisation. Also, telecom services are not all equally important. Therefore, a risk that was assessed as “high” occurring in a service that is useful but non-essential may be listed below a risk that was assessed as “medium” to a vital service. The priority may further be affected by the service acting as backup to another service, or having fallback options itself.

All analysts must collectively examine each risk on the longlist. Based on consensus, risks may be raised or lowered on the list, or may be removed altogether. The result of this process is a prioritised shortlist of risks for which the analysts agree that risk treatment is warranted.

6.3 Make treatment recommendations

It is not the responsibility of the analysts the decide on how risks on the shortlist will be treated. The sponsor or decision maker will be responsible for these measures. However, the analysts do have the responsibility for providing them guidance, and to make reservations for the uncertainty in their assessments and limits to their knowledge.

For each risk on the shortlist, the analysts must give risk treatment recommendations. It is impossible to give a procedure for this, as the suitable treatment for a risk depends very much on the type of service, the nature of the risk, and the circumstances of the case organisation.

6.3.1 Select risk treatment option

In general, four general risk treatment options exist:

1. Avoid. Remove the risk completely (proaction), or discontinue the use of the component or service altogether. Proaction means eliminating structural causes of accidents to prevent them from happening in the first place (e.g. avoiding radio interference by replacing a wireless link by a wired link). When discontinuing an entire service, an alternative service will often be available. However, you should be careful to replace a service with known risks for a new one with unknown risks.

   Even when no alternative is available it may still be worth considering discontinuing use of the telecom service when the risk cannot be avoided. Rather than using a service that may fail unexpectedly, it may be preferable to not use the service at all to avoid unpleasant surprises at inopportune moments during crisis response.

2. Reduce. Make the risk more acceptable, by reducing either its likelihood (frequency) or impact. These activities encompass prevention and preparation. Prevention means taking measures beforehand that aim to make accidents less likely, and to limit the consequences in case incidents do occur (e.g. by imposing smoking restrictions and using fire-retardant materials). Preparation means ensuring the capacity to deal with accidents and disasters in case they do happen (e.g. by holding regular fire drills).

3. Transfer. Pass the risk to another party. Typical examples of risk transfer are insurance, or maintenance contracts whereby faulty equipment is replaced with spares on short notice. Risk transfer in effect buys certainty, by transferring the uncertainty to another party in return for payment.

4. Retain. Accepting the risk, in an informed decision. Reasons for accepting risks may be that other options would be too costly, that the likelihood is deemed to be very low, or simply the lack of suitable alternatives. In all cases it is much preferable to knowingly accept a risk rather than being confronted with it.

6.3.2 Assess social risk factors

The draft treatment of risks on the shortlist may lead to criticism by other stakeholders. The opinions of these stakeholders must be considered before final treatment recommendations are formulated. Otherwise, decision makers may unexpectedly have to deal with societal opposition, possibly forcing them to opt for a sub-optimal treatment that is nevertheless more acceptable to external stakeholders. Analysts must therefore assess additional risk factors that influence risk perception and risk acceptance by third parties. See the table of social risk factors.

Artificiality applies to situations where people oppose a technology because it is unnatural. For example, electromagnetic radiation from mobile telephony base stations is more often considered 'harmful', whereas natural sunlight is more often considered 'healthy'. However, there is scientific consensus that ill effects from electromagnetic radiation have not been demonstrated, whereas the incidence of skin cancer is cause for serious concern. Related to this issue is that of immorality. Immorality play a role when technological solutions go against people's ethical or moral principles.

Benefits can counterbalance the availability risks on the shortlist. Risky situations can be acceptable when the (perceived) benefits outweigh the (perceived) risks. For example, construction of a high broadcasting tower may meet with less opposition if it will be used for emergency communications instead of entertainment broadcasts.

Blame can sometime be apportioned to some actor (e.g. a telecoms operator), but natural risks cannot be blamed on anyone in particular. Risks without blame are often more acceptable than risk caused by some explicit actor.

Catastrophic potential makes risks less acceptable. The menace of wide-spread, large-scale destruction, regardless of likelihood, makes risks less tolerable. On the other hand, risks that have a small chronic effect over a period of time are often more easily accepted.

Children influence risk perception, sometimes in dramatic ways. People have strong feelings when children are affected by risks.

Familiarity with a risk may lead to complacency. The reverse is also true: novel risks may be less tolerable, simply because they are less well-known.

Fear is a general factor, related to catastrophic potential and familiarity. Strong feelings of fear decrease risk acceptance.

Institutional control can reassure people that risks are handled diligently. Sufficient trust in institutions is a prerequisite. When trust in institutions is low, risks will be perceived to be higher.

Media exposure can lead to increased perception of likelihood. Few people experience risks first-hand, and wide coverage by broadcasting or social networks can increased risk perception.

Mobilisation potential is relevant to decision makers. The 'nimby' phenomenon (“not in my backyard”) reflects mobilisation by nearby residents. Risks may provoke wide-spread and vocal opposition, making them less acceptable to decision makers.

Personal control refers to the amount of influence individuals can exert over the risky situation. For example, the risk of disturbances in communication are more acceptable when the user has the ability to control the device and participate in communication, instead of having only the passive ability to listen.

Violation of equity occurs when the benefits and the adverse effects are unevenly distributed. Opposition will be strong if the beneficiaries do not experience adverse effects at all.

Voluntariness is related to personal control. For example, people can choose whether or not to use a mobile phone, but construction of a mobile antenna mast in their neighbourhood is imposed upon them. Lack of voluntariness makes risks less acceptable.

6.3.3 Review the shortlist

The analysts must review each risk on the shortlist, to determine whether social risk factors may have a significant impact. This consists of the following steps:

1. Predict in what forms the risk factor would be expressed for various external stakeholders. For example, would it lead to a tarnished public image, reduced funding, or perhaps active opposition?
2. Assess the influence that this would have on the ability of decision makers to defend their choice of risk treatments. Can they easily deflect criticism, or will they be forced to select an alternative treatment?
3. Assess how the influence of the risk factor could be mitigated in advance, for example by informing stakeholders in advance, ask for their approval, or having them participate in a monitoring and oversight body.

If necessary, risk prioritisation should be adjusted and additional or different risk treatments should be recommended.

6.4 Prepare final report

The analysts have now collected all information for the final report. Not only can they present a prioritised shortlist of most serious risks with treatment recommendations, but they can also provide arguments for their proposals.

This final report must be reviewed by all analysts, and it must be approved by consensus before it is presented to the sponsor. The study is thereby concluded. A suggested outline of the final report is shown below.

1. Executive summary to the final report.

2. About the case organisation (internal scope):

   1. Position within wider system of stakeholders.
   2. Tasks.
   3. Responsibilities.
   4. Telecom services used, together with their role and purpose.
   5. Main actors.

3. About the environment of the case organisation (external scope):

   1. Disaster scenarios.
   2. External parties with whom the main actors may communicate.

4. Roles and stakeholders

5. Telecom services

   1. Diagram with explanation (once for each service)
   2. Important risks (single failures and common cause failures)

6. Risk shortlist, with for each risk:

   1. Description
   2. Relevant social risk factors
   3. Justification for risk priority, uncertainty, and limits to knowledge
   4. Recommended risk treatment

7. Conclusions and recommended actions

Appendices:

1. Glossary
2. Reports of single failures
3. Reports of common cause failures