Maturing in Agile: What Is It About?

Rafaela Mantovani Fontana
Sheila Reinehr
Andreia Malucelli
Maturing in Agile: What Is It About?






Introduction
Related Work
Research Design
Data Analysis
Discussion
Conclusion
Pontifical Catholic University of Paraná
Software Engineering Research Group
2014
Combination of agile methods with
CMMI-DEV and ISO/IEC 15504
• Higher maturity levels hamper agility
Agile maturity models
• Sustain agility
• Models are still initial and few are scientifically
tested
• There is evidence that practitioners dislike
“models”
Rafaela Mantovani Fontana
3
Pontifical Catholic University of Paraná
Software Engineering Research Group

2014
Models that focus on developing agile values
and practices by means of stages or levels:
Packlick (2007)
Sidky et al. (2007)
• Goal oriented approach
• 5 levels to reach maturity: awareness, transformation, breakthrough,
optimization, mentoring
• 5 maturity levels: collaborative, evolutionary, effective, adaptive,
encompassing
Nottonson and DeLong
(2008)
• 3 steps of agile adoption: craw, walk, run
Qumer and HendersonSellers (2008)
• 6 maturity levels: agile infancy, agile initial, agile realization, agile value, agile
smart, agile progress
Patel and
Ramachandran (2009)
• Similar to CMMI-DEV
• 5 levels: initial, explored, defined, improved, sustained
Rafaela Mantovani Fontana
4
Pontifical Catholic University of Paraná
Software Engineering Research Group

2014
Other approaches to measure agility
 Measure how agile a team is
▪ Layman et al. (2004), Williams et al. (2004)
 Evaluate agility comparatively
▪ Williams et al. (2010)
 Relate agile practices to project success
▪ Abbas et al. (2010)
 Define practices to assess maturity
▪ Buglione (2011)
 Assess adequacy of an agile method
▪ Soudarajan et al. (2012)
 Identify if agile practices match the organization
strategy
▪ Kettunen (2012)
Rafaela Mantovani Fontana
5
Pontifical Catholic University of Paraná
Software Engineering Research Group

2014
Practices were grouped according to SWEBOK areas and
used as foundation to the measurement instrument:













Focus on agile requirements
Focus on software architecture
Focus on agile coding
Focus on agile testing
Focus on agile planning
Focus on project monitoring
Focus on agile values in the team
Focus on agile quality assurance
Focus on defining an agile physical environment
Focus on involved customer
Focus on metrics
Focus on defining processes
Focus on controlling processes
Rafaela Mantovani Fontana
6
Pontifical Catholic University of Paraná
Software Engineering Research Group
Literature Research
2014
• List of agile
practices
grouped in 13
different
“focuses”, or
practices
Collecting data
• Respondents had
to number the 13
practices in the
sequence of
implementation
to get to maturity
• Respondents also
pointed out the
perceived
usefulness of an
agile maturity
model
• 87 valid
questionnaires
• Average
experience in
agile software
development of
respondents was
3.6 years
Rafaela Mantovani Fontana
Analyzing data
• 1) We analyzed
the usefulness of
an agile maturity
model
• 2) We verified for
each practice:
• Is it relevant in
the road map to
maturity?
• Is there a trend
to implement it
in the
beginning, in
the middle or in
the end of the
process?
• Is there a
difference in the
opinion of
respondents
based on their
experience?
7
Pontifical Catholic University of Paraná
Software Engineering Research Group
2014
The usefulness
of an agile
maturity model
The relevant
practices to
maturity
Rafaela Mantovani Fontana
The difference
in practitioners
opinions
8
Pontifical Catholic University of Paraná
Software Engineering Research Group
The usefulness
of an agile
maturity model
Is an agile maturity model
useful?

The difference
in practitioners
opinions
Yes, it is useful:
 To help organizations to recognize the
implementation of agile methods
 To help as a guide
 To define agility
 To help in the beginning of the adoption
3%
28%

No, it is not useful:
 Maturity is too dependent on the
69%
Yes
2014
The relevant
practices to
maturity
Not
Blank
context
 There is not a model to all organizations
 A model would limit agile values and
creativity
Rafaela Mantovani Fontana
9
Pontifical Catholic University of Paraná
Software Engineering Research Group
2014
The usefulness
of an agile
maturity model

The relevant
practices to
maturity
The difference
in practitioners
opinions
We analyzed data from three perspectives:
All respondents
2. Experienced practitioners (> 3 years)
3. Non-experienced practitioners (<= 3 years)
1.
Rafaela Mantovani Fontana
10
Pontifical Catholic University of Paraná
Software Engineering Research Group
The usefulness
of an agile
maturity model

The difference
in practitioners
opinions
All respondents
 All practices appeared to be relevant in the road map to maturity
 Indifference regarding the classification of the following practices:
Defining an agile physical environment, Agile quality assurance, Agile
project monitoring

The road map to maturity:
1.
2.
3.
4.
5.
2014
The relevant
practices to
maturity
Agile values in team
Involved customer
Agile planning
Agile requirements
Agile testing
Agile coding
7.
Software architecture
8. Metrics
9. Defining processes
10. Controlling processes
6.
Rafaela Mantovani Fontana
11
Pontifical Catholic University of Paraná
Software Engineering Research Group
The usefulness
of an agile
maturity model

The difference
in practitioners
opinions
Experienced practitioners (> 3 years)
 3 practices did not appear relevant to maturity: Defining processes,
Metrics and Controlling processes
 Indifference regarding the classification of the following practices:
Defining an agile physical environment, Software architecture, Agile
project monitoring, Agile quality assurance

The road map to maturity:
1.
2.
3.
4.
2014
The relevant
practices to
maturity
Agile values in team
Involved customer
Agile requirements
Agile coding
5.
6.
Agile planning
Agile testing
Rafaela Mantovani Fontana
12
Pontifical Catholic University of Paraná
Software Engineering Research Group
The usefulness
of an agile
maturity model

The difference
in practitioners
opinions
Non-experienced practitioners (<= 3 years)
 All practices appeared to be relevant in the road map to maturity
 Indifference regarding the classification of the following practices:
Agile testing, Defining an agile physical environment, Agile quality
assurance, Agile coding, Defining process, Agile process monitoring

The road map to maturity:
1.
2.
3.
4.
2014
The relevant
practices to
maturity
Agile values in team
Involved customer
Agile planning
Agile requirements
5.
6.
7.
Software architecture
Metrics
Controlling processes
Rafaela Mantovani Fontana
13
Pontifical Catholic University of Paraná
Software Engineering Research Group
2014
The usefulness
of an agile
maturity model

The relevant
practices to
maturity
The difference
in practitioners
opinions
Experienced and non-experienced practitioners
disagree on the implementation sequence of:
 Agile requirements
 Software architecture
 Agile coding
 Agile testing
Rafaela Mantovani Fontana
14
Pontifical Catholic University of Paraná
Software Engineering Research Group

2014
A maturity model is not well-suited for agility
because organizations are too different in context
 Traditional software process improvement endeavors
usually do not accomplish the benefits in small companies
(Staples et al., 2007)
 People focus on having the job done (Adolph et al., 2012)
 Codified processes tend to be abandoned (Coleman and
O’Connor, 2008)
Rafaela Mantovani Fontana
15
Pontifical Catholic University of Paraná
Software Engineering Research Group

When teams are recognized as complex adaptive systems
(Stacey, 1996) it is understood that the behavior is
emergent
 Trials to command and control overcome system’s potential
(McDaniel Jr., 2007)
 The trend to tailor agile methods is already established – agile
practitioners prefer to do the job as the context demands (Sheffield
and Lemétayer, 2012; Armbust and Rombach, 2011; Cesare et al.,
2010; Coleman and O’Connor, 2008; Kirk and Tempero, 2012; Bustard
et al, 2013)
 Although organizations have their particular way to improve
processes, it is possible to identify “best practices” (Eisenhardt and
Martin, 2000)
2014
Rafaela Mantovani Fontana
16
Pontifical Catholic University of Paraná
Software Engineering Research Group
2014

Process-related practices – metrics, defining processes and
controlling processes – were not related to agile maturity

Based on experienced practitioners responses we could
identify groups of practices and their implementation
sequence
Essential
Agile values
Intermediate
Involved customer
Agile coding
Agile planning
Agile testing
Optional
Metrics
Defining processes
Agile requirements
Controlling
processes
Software architecture
Agile physical environment
Agile quality assurance
Agile project monitoring
Rafaela Mantovani Fontana
17
Pontifical Catholic University of Paraná
Software Engineering Research Group

2014
Organizational maturity has been mainly expressed
as the adherence to a structured process
 Objective assessments

For agile maturity, we propose the evaluation or
description of software processes based on how
efficient it is to reach objectives (Packlick, 2007;
Kirk and Tempero, 2012; Kettunen, 2012)
 This goals-based approach would leave space for teams
to do the job as they please
Rafaela Mantovani Fontana
18
Pontifical Catholic University of Paraná
Software Engineering Research Group
Summary
2014
• We conducted a survey with agile practitioners
• Some disagree with the need for an agile maturity model
• We proposed, then, a guide to maturity
Limitations
• Study conducted exclusively with Brazilian practitioners
• Solely based on the perception of respondents, not on projects
observation
Take-home message
• We want to argue on the probable impossibility of predefining the path
to agile maturity
• More research needed on how to develop maturity leaving space to the
emergency of behaviors
• Maturity has to be rethought in some more subjective ways
Rafaela Mantovani Fontana
19
Pontifical Catholic University of Paraná
Software Engineering Research Group
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
2014
Abbas, Nouras; Gravell, Andrew M.; Wills, Gary B, 2010. Using Factor Analysis to Generate Clusters of Agile
Practices – A guide for agile process improvement. 2010 Agile Conference. DOI 10.1109/AGILE.2010.15. pp.
11-20. Available at http://goo.gl/5w95y4
Adolph, Steve; Krutchen, Philippe; Hall, Wendy. 2012. Reconciling perspectives: A grounded theory of how
people manage the process of software development. J Syst Softw. Vol. 85. Issue 6. June. pp. 1269-1286. DOI
10.1016/j.jss.2012.01.059. Available at http://goo.gl/vjUHS8
Armbrust, O., Rombach, D.: The right process for each context: objective evidence needed. ICSSP '11:
Proceedings of the 2011 International Conference on Software and Systems Process (ICSSP'11), 237--241
(2011)
Buglione, Luigi, 2011. Light Maturity Models (LMM): an Agile application. Profes '11: Proceedings of the 12th
International Conference on Product Focused Software Development and Process Improvement. pp. 57-61.
DOI 10.1145/2181101.2181115. Available at http://goo.gl/6GZhxL
Bustard, D., Wilkie, G., Greer, D.: The Maturation of Agile Software Development Principles and Practice:
Observations on Successive Industrial Studies in 2010 and 2012. 20th Annual IEEE International Conference
and Workshops on the Engineering of Computer Based Systems (EBCS). doi:10.1109/ECBS.2013.11 (2013)
Cesare, S. d., Lycett, M., Macredie, R. D., Patel, C., Paul, R.: Examining Perceptions of Agility in Software
Development Practice. Commun. ACM, 53 (2010)
Coleman, G., O’Connor, R.: Investigating software process in practice: A grounded theory perspective. J. syst.
softw., 81, 772--784 (2008)
Eisenhardt, K. M., Martin, J. A.: Dynamic Capabilities: What are they? Strat. Mgmt. J., 21, pp. 1105--1121
(2000)
Kettunen, Petri, 2012, Systematizing Software Development Agility: Towards an Enterprise Capability
Improvement Framework. J Enterp Transform, 2(2). pp. 81-104. DOI 10.1080/19488289.2012.664610.
Available at http://goo.gl/tbM5Fh
Kirk, D., Tempero, E. A lightweight framework for describing software practices. J. syst. softw., 85, 582--595
(2012)
Layman, Lucas; Williams, Laurie; Cunningham, Lynn, 2004. Motivations and Measurements in an Agile Case
Study. QUTE-SWAP '04: Proceedings of the 2004 workshop on Quantitative techniques for software agile
process. pp. 14-24. DOI 10.1145/1151433.1151436. Available at http://goo.gl/gcf9HD
McDaniel Jr., R. R.: Management Strategies for Complex Adaptive Systems. Performance Improvement
Quarterly, 20(2), 21--42 (2007)
Rafaela Mantovani Fontana
20
Pontifical Catholic University of Paraná
Software Engineering Research Group
13.
2014
14.
15.
16.
17.
18.
19.
20.
21.
22.
23.
Nottonson, K., DeLong, K. 2008. Crawl, Walk, Run: 4 Years of Agile Adoption at BabyCenter.com. pp.116-120,
Agile 2008. DOI http://doi.ieeecomputersociety.org/10.1109/Agile.2008.84. Available at
http://goo.gl/dEDxSN
Packlick, Jay, 2007. The Agility Maturity Map – a Goal Oriented Approach to Agile Improvement. Agile
Conference 2007. 13-17 Aug. pp. 266-271. DOI 10.1109/AGILE.2007.55. Available at http://goo.gl/VdaaTh
Patel, Chetankumar; Ramachandran, Muthu, 2009. Agile Maturity Model (AMM): A Software Process
Improvement framework for Agile Software Development Practices. Int J Softw Eng, vol. 2. no. 1. pp. 3-28.
Available at http://goo.gl/FGe0eE
Qumer, A.; Henderson-Sellers, B, 2008. A framework to support the evaluation, adoption and improvement
of agile methods in practice. J Syst Softw. Vol. 81. Issue 11. pp. 1899-1919. DOI 10.1016/j.jss.2007.12.806.
Available at http://goo.gl/LnrDMX
Sheffield, J., Lemétayer, J.: Factor associated with the software development agility of successful projects.
Int. J. Proj. Manag. doi:http://dx.doi.org/10.1016/j.ijproman.2012.09.011 (2012)
Sidky, Ahmed; Arthur, James; Bohner, Shawn. 2007. A disciplined approach to adopting agile practices: the
agile adoption framework. Innov Syst Softw Eng. Vol. 3. Issue 3. pp. 203-216. DOI 10.1007/s11334-007-0026-z.
Available at http://goo.gl/F4Vdxa
Soundararajan, Shvetha; Arthur, James D.; Balci, Osmar, 2012. A Methodology for Assessing Agile Software
Development Methods. Agile Conference 2012. 13-17 Aug. pp. 51-54. DOI 10.1109/Agile.2012.24. Available at
http://goo.gl/uGd9vO
Stacey, R.: Complexity and Creativity in Organizations. San Francisco: Berret-Koehler Publishers (1996)
Staples, M., Niazi, M., Jeffery, R., Abrahams, A., Byatt, P., Murphy, R.: An exploratory study of why
organizations do not adopt CMMI. J. syst. softw., 80, 883--895 (2007)
Williams, L.; Krebs, W.; Layman, L.; Antón, A.; Abrahamsson, P., 2004. Toward a Framework for Evaluating
Extreme Programming. 8th International Conference on Empirical Assessment in Software Engineering
(EASE 04), pp. 11-20. DOI 10.1049/ic:20040394. Available at http://goo.gl/r29m0P
Williams, Laurie; Rubin, Kenny; Cohn, Mike, 2010. Driving Process Improvement Via Comparative Agility
Assessment. Agile Conference 2010. 9-13 Aug. pp. 3-10. DOI 10.1109/AGILE.2010.12. Available at
http://goo.gl/DA3zP6
Rafaela Mantovani Fontana
21
Rafaela Mantovani Fontana ([email protected])

Download Report