Prediction of fault-proneness at early phase
in object-oriented development
Toshihiro Kamiya†, Shinji Kusumoto† and Katsuro Inoue†‡
† Osaka University
‡Nara Institute of Science and Technology
Background
Complexity metrics are used to estimate fault-proneness
of software component.
According to the value of the metrics, we can allocate the
effort of review/testing to the fault-prone component.
Chidamber and Kemerer’s metrics are the representative
complexity metrics for object-oriented software.
1
Chidamber and Kemerer’s metrics[1]
C&K metrics evaluate complexity of classes from the
following three viewpoints:
Inheritance complexity
・ DIT (Depth of inheritance tree)
・ NOC (Number of children)
Coupling complexity
・ RFC（Response for a class
・ CBO(Coupling between object-class)
Class internal complexity
・ WMC(Weighted methods par class
・ LCOM(Lack of cohesion in method)
[1] S.R. Chidamber and C.F. Kemerer, A Metrics Suite for Object Oriented
Design, IEEE Trans. on software eng., vol., 20, No. 6 (1994) 476-492.
2
Evaluation of C&K metrics
Several research studies evaluate the usefulness of C&K
metrics.
・ Chidamber and Kemerer confirm that C&K metrics
satisfy Weyuker’s properties [1].
・ Basili et. al. empirically evaluated that C&K metric suit
is better predictor of fault-proneness of class than
traditional code metrics [2].
・ Briand et. al. discussed several design metrics that
include C&K Metrics [3].
[2] Basili, V. R., Briand, L. C., and Mélo, W. L., A validation of object-oriented
design metrics as quality indicators, IEEE Trans. on Software Eng. Vol. 20,
No. 22, (1996) 751-761.
[3] Briand, L. C., Daly, J.W., and Wüst, J.K., A Unified Framework for Coupling
Measurement in Object-Oriented Systems, IEEE Trans. on software eng.,
vol.25, No.1, (1999) 91-121.
3
Difficulty in applying C&K metrics to
design
In previous researches, C&K metrics were applied to
source code. Because some of C&K metrics need
information such as algorithm or call-relationship, which
are determined later at design phase.
In order to allocate the review and testing effort efficiently,
early estimation of the fault-prone classes (components) is
preferable.
4
Proposed method
We propose a method to predict fault-proneness at early
phase in object-oriented development.
1. Introduce four checkpoints into design /
implementation phase.
2. Determine the available metric set at each checkpoint.
3. By multivariate logistic regression analysis, estimate
fault-proneness of the classes (components) at each
checkpoint.
We empirically evaluate how the metric sets predict faultprone classes at each checkpoint.
5
System
Analysis
Design
Introduced checkpoints
CP1: Association and attributes of
determined.
classes are
CP2: Derivation, interface(method), and reused
classes are determined.
t
Object ImplemenDesign
tation
CP3: Algorithm of each method is developed.
CP4: Source code is written.
6
Metrics
We use following metrics in this study.
C&K metrics
・ DIT, NOC, RFC, CBO, WMC, and LCOM
・ CBON (Coupling to newly developed classes)
・ CBOR (Coupling to reused classes)
CBO = CBON + CBOR
Other metrics
・ NIV (Number of instance variables)
・ SLOC (Source lines of code)
7
System
Analysis
Design
Checkpoints and metric sets
CP1: Association and attributes of
determined.
{ CBON, NIV }
classes are
CP2: Derivation, interface(method), and reused
classes are determined.
{ CBON, NIV, CBOR, CBO, WMC, DIT, NOC }
t
Object ImplemenDesign
tation
CP3: Algorithm of each method is developed.
{ CBON, NIV, CBOR, CBO, WMC, DIT, NOC,
RFC, LCOM }
CP4: Source code is written.
{CBON, NIV, CBOR, CBO, WMC, DIT, NOC,
RFC, LCOM, SLOC }
8
Estimation of fault-proneness of classes
“Multivariate logistic regression is a standard technique
based on maximum likelihood estimation, to analyze the
relationships between measures and fault-proneness of
classes.”
P1: fault-proneness (probability of fault detected)
CBO, NIV: metric values
C0, C1, C2: coefficients
1
P1 (CBO, NIV )
1 exp((C0 C1 CBO C2 NIV ))
If P1 of the target class > 0.5, then the class is
predicted as faulty.
9
Outline of the experiment
We empirically evaluate the proposed method using the
data collected from an experimental project.
・ The experimental project was performed at a computer
company for five days in August 1997.
・ Developers were new employees who finished on-thejob training of object-oriented design and C++
programming.
・ Developer teams developed an identical e-mail delivery
system using C++.
10
Experimental data
Fault tracking data
・ Location
・ Type
・ Effort to fix
Metric data
・ Metric values of developed classes
As the result, 80 faults of 141 classes were collected.
11
Statistics of empirical data
Metric
NIV
CBON
CBOR
CBO
DIT
NOC
RFC
LCOM
SLOC
Faults
Min. Max.
0
14
0
3
0
4
0
5
0
6
0
0
0
27
0 190
5 420
0
17
Ave. Std. dev. CP
4.00
2.67 1
0.53
0.99 1
0.86
0.99 2
1.39
1.59 2
3.44
1.41 2
0.00
0.00 2
8.23
6.81 3
22.42
36.84 3
96.43
81.01 4
0.57
1.93
12
Prediction by metrics (1/2)
With the collected data, we estimate fault-prone classes at
each checkpoint.
Prediction at CP1
Predicted
No fault
Faulty
No Fault
112
2
Actual
Faulty
18
9
13
Prediction by metrics (2/2)
Prediction at CP2
Predicted
No fault Faulty
No Fault
109
5
Actual
Faulty
11
16
Prediction at CP3
Predicted
No fault Faulty
No Fault
111
3
Actual
Faulty
9
18
Prediction at CP4
Predicted
No fault Faulty
No Fault
111
3
Actual
Faulty
8
19
14
Indicators for evaluation
To illustrate the precision of the estimation, two indicators
are used [2].
Completeness: percentage of classes correctly
predicted faulty in actual faulty.
Correctness: percentage of classes actual faulty in
predicted faulty.
Predicted
No fault Faulty
112
2
18
9
No Fault
Faulty
9
0.33
Completeness
18 9
Actual
Correctness
9
0.82
29
15
Precision (%)
Precision of estimation
100
90
80
70
60
50
40
30
20
10
0
82
76
59
85
63
86
70
Correctness
Completeness
33
CP1
CP2
CP3
CP4
On the whole, the precision of estimation improves as the
process progress.
– Correctness is relatively high at all checkpoints, so that
the estimation used to ‘seed’ the faulty classes.
– Completeness becomes better at later checkpoint. The
estimation at CP2 does well.
16
Conclusion
We have proposed a method to predict fault-proneness at
early phase in object-oriented development, and
evaluated the method empirically.
As further work, we are going to:
・ Use other metrics in the proposed method.
・ Develop the tool which support the proposed method.
17
Weyuker’s properties [4]
Let (c) denote a measurement of metric for class c,
and p + q denote the combined class of class p and q,
W1 p q, (p) (q).
W2 p q, (p) = (q), and p differs from q.
W3 p q, (p) (q), and p's functionality is equal to q's
(but p's design differs from q's).
W4 p q, (p) (p + q), and (q) (p + q).
W5 p q r, (p) = (q), and (p + r) (q + r).
W6 p q, (p) + (q) (p + q).
Chidamber and Kemerer proved that each metric WMC,
DIT, NOC, CBO, RFC, and LCOM satisfies W1, ..., W6,
except for NOC and LCOM which do not satisfy W4.
[4]Weyuker, E. J., Evaluating software complexity measures, IEEE Trans. on
Software Eng. Vol. 14, No. 9, (1998), 1357-1365.
18
Coefficients at each checkpoint
Metric
Coefficient
CP1
CP2
CP3
CP4
C0
-3.37
-1.23
-1.31
-2.69
NIV
0.420
EL
EL
EL
CBON
EL
EL
EL
EL
CBOR
0.934
0.890
EL
CBO
EL
EL
EL
NIM
0.336
EL
EL
DIT
-1.16
-1.28
-0.663
NOC
EL
EL
RFC
0.284
EL
LCOM
EL
SLOC
0.0302
The ‘EL’ means that the metric was eliminated
form the prediction equation by a backward
variable elimination process. The ‘-’s are not
applicable at the checkpoint.
19
© Copyright 2024 ExpyDoc
