Lecture Notes 9 - Department of Computing and Software

Previous Lecture
DDP: quantitative risk management for Req. Engineering
Defect Detection Prevention
CS/SE 3RA3
Ryszard Janicki
Department of Computing and Software, McMaster University, Hamilton,
Ontario, Canada
Ryszard Janicki
Defect Detection Prevention
1/8
Previous Lecture
DDP: quantitative risk management for Req. Engineering
Previous Lecture: Risk Analysis
Risk
identification
what system-specific
risks?
Risk
control
Risk
assessment
likely?
severe, likely consequences?
Ryszard Janicki
countermeasures
t
as new reqs
Defect Detection Prevention
2/8
Previous Lecture
DDP: quantitative risk management for Req. Engineering
Impact matrix
Effectiveness matrix
Optimal Balance
DDP = Defect Detection Prevention
Technique and tool developed at NASA in 2003
Quantitative support for Identify-Assess-Control cycles
Three steps:
Elaborate
I
t
risk
i k Impact
matrix
Elaborate
countermeasure
Effectiveness
matrix
Ryszard Janicki
Determine
optimal balance
risk reduction /
countermeasure cost
Defect Detection Prevention
3/8
Previous Lecture
DDP: quantitative risk management for Req. Engineering
Impact matrix
Effectiveness matrix
Optimal Balance
Three Steps
1
Build a risk-consequence table (impact matrix) with domain
experts for:
prioritizing risks by critical impact on all objectives
highlighting the most risk-driving objectives
2
Build a risk-countermeasure table with domain experts for:
estimating risk reduction by alternative countermeasures
highlighting most globally effective countermeasures
3
Determine optimal balance: risk reduction vs.
countermeasure cost
Ryszard Janicki
Defect Detection Prevention
4/8
Impact matrix: example for library system
For each objective obj and risk r , we provide:
Impact(r , obj) = estimated loss of satisfaction of obj by r
0 (no loss) → 1 (total loss)
Risk Criticality:
P
RC (r ) = Likelihood(r ) obj Impact(r , obj)Weight(obj)
Objective Loss:
P
Loss(obj) = Weight(obj) r Impact(r , obj)Likelihood(r )
Effectiveness matrix: example for library system
For each countermeasure cm, weighted risk r , we provide:
Reduction(cm, r ) = estimated reduction of r if cm applied
0 (no reduction) → 1 (risk eliminated)
Combines Reduction:
Q
CR(r ) = 1 − cm (1 − Reduction(cm, r ))
P
Overall Effect: OE (cm) = r Reduction(cm, r )Criticality (r )
Previous Lecture
DDP: quantitative risk management for Req. Engineering
Impact matrix
Effectiveness matrix
Optimal Balance
Determine optimal balance: risk reduction vs.
countermeasure cost
Cost of each countermeasure cm to be estimated with domain
experts
DDP can then visualize:
risk balance charts: residual impact of each risk on all
objectives if cm is selected
optimal combinations of countermeasures for risk balance
under cost constraints
simulated annealing search for near-optimal solutions
optimality criterion can be set by user
e.g. “maximize satisfaction of objectives under this cost
threshold”
“minimize cost above this satisfaction threshold”
Ryszard Janicki
Defect Detection Prevention
7/8
Previous Lecture
DDP: quantitative risk management for Req. Engineering
Impact matrix
Effectiveness matrix
Optimal Balance
IMPORTANT
NEVER TREAT LITERARY NUMBERS OBTAINED BY
ANY OF THE METHODS DISCUSSED!
NEVER TREAT THE EQUATIONS AS LAWS OF NATURE!
ALWAYS TREAT THE OUTCOME AS A DIRECTION, NOT
AN EXACT RESULT!
Ryszard Janicki
Defect Detection Prevention
8/8