How secure is your software really?

How secure is your software really?
A whitepaper by the Software Improvement Group
October 27, 2014
Abstract
This whitepaper describes the growing IT security problem and how the improvement of software
quality is essential in solving it.
Typically, software security is only addressed by testing it as a black box, hoping to find weaknesses
- for example by using a penetration test. However, to be able to find all weaknesses, one should
look where they are hidden: in the internal structure of a software system. Therefore, in order to
have a full view of security risks, it is essential to look at source code, design, specifications and configuration. This whitepaper presents an efficient, thorough, structured and repeatable approach to
find security weaknesses in software and measure security.
1
1
The software security problem
The chances of an IT security incident occurring are ever increasing. Just leaf through a newspaper
and security incidents fill the pages: stolen credit card information, system operations breaking
down, and unfortunately many more1. Growing complexity and coupling of systems make it hard to
structurally locate and address security concerns. IT is typically directly connected to the Internet
and often provides many points of access from the outside. The same principle of system coupling
holds true for the internal application landscape. Breaching a single weak system therefore often
allows for access to many others. In addition, attacks receive broad coverage by the media, which
motivates perpetrators and embarrasses organizations. Attacks are carried out increasingly professionally as cyber crime is getting better organized.
Furthermore, the impact of security issues increases. Organization’s ever-growing reliance on IT
means that IT becomes more valuable. An interruption in IT may mean serious loss of money, reputation and data. A recent study by Forrester [1] found that half of the 240 participating organizations in Europe and North America experienced at least one web application security incident in the
year before. 18% of respondents in that research reported that such breaches cost their organizations at least $500,000. In addition, the reported impact of software security breaches includes
damage to professional reputation, loss of revenue, loss of customer confidence, damage to brand
or image and loss of customer data.
Software applications currently create the biggest digital security risk. Gartner reports that 75% of
security incidents are caused by software mistakes as opposed to insufficient technical infrastructure [2]. While assessing systems from the outside with e.g. penetration tests is very useful, security
cannot be assessed fully without looking at the internal structure of software applications. Research
shows that performing penetration tests alone may cover only 43% of security vulnerabilities,
whereas code review finds 78% [3]. The internal structure contains design, specifications, source
code and software configuration. Organizational processes, too, should handle, prevent or mitigate
security risks, such as an incident response plan.
2 SIG assesses security of systems based on ISO 25010
The ISO/IEC 25010 standard for software product quality [4] offers a powerful framework for analysing software quality aspects including security, as depicted in the following Figure 1.
Usability
Reliability
Compatibility
Security
Performance
Efficiency
Functional
Suitability
Maintainability
Software
Quality
Portability
ISO 25010
Figure 1 - ISO/IEC 25010 quality characteristics for software product quality
1
For example: Breaches with over 30.000 stolen records are visualized at
http://www.informationisbeautiful.net/visualizations/worlds-biggest-data-breaches-hacks/
2
In ISO 25010, security is defined as:
“The degree to which a product or system protects information and data so that persons or
other products or systems have the degree of data access appropriate to their types and levels
of authorization.”
Thus, security is about how sure you can be that access to your assets is restricted to authorized
people or systems. Note that there is necessarily no such thing as being 100% secure!
However, ISO 25010 does not provide how to measure software security. The Software Improvement Group (SIG) has therefore developed a structured approach [5] based on ISO 25010 to measure software security. To do this, SIG analyses design, specifications, source code and software configuration. This complements existing approaches that address aspects like network infrastructure
and penetration tests. An overview of the SIG security model is discussed in detail in Appendix A.1.
The model provides a framework to analyse the relevant system properties. A SIG specialist performs this analysis with the assistance of dedicated tooling. The specialist looks at specifications,
design, source code and configuration to assess how well best practices have been implemented
and how well security vulnerabilities are prevented. This approach is technology-independent. These best practices and weaknesses are based on the ones documented by organizations such as
OWASP and CWE/SANS (see also Appendix A.2 for an overview of security technologies).
2.1
Importance of build quality and software development process
A good code review locates not only the inadequate protection against attacks but also situations
that encourage software defects. Defects can cause security vulnerabilities or data leaks, for example a user that sees the personal data of another user because of a flaw in the way sessions are handled. An important aspect of security therefore is the extent to which software is maintainable: how
easy is it to make alterations to software without introducing new issues? The proven SIG ISO 20510
software maintainability model [6] is a useful method to analyse this.
To be consistently applied, best practices should be integrated into the software development process. The available processes and tools for functional and non-functional testing of the system are
important factors to further mitigate maintainability risks.
3
Security Risk Assessment - a structured way to analyse system security “from the inside out”
The Software Improvement Group offers a Security Risk Assessment (SecRA) service that is targeted
at technology, rooted in science and based on international standards. The SecRA looks for vulnerabilities and security risks by investigating a system from the inside out: how systems are constructed from the perspective of technology, architecture and source code and how the involved organization works.
The objective of a SecRA is to identify the most urgent security risks and to give advice on effective
approaches to mitigate these. Typically, one out of eight SIG top risks could only have been found by
a penetration test. Likewise, one out of eight top risks originate from static code analysis tooling.
The remaining 6 risks are from manual code/design review of program logic, architecture, build
quality and configuration. This shows that expert review is essential in finding functional mistakes
and seeing whether essential functions are missing.
3
3.1
A broader scope than other approaches
SIG’s code/design review can be compared with testing the physical security of a bank by studying
the construction blueprints and assessing the building from the inside: Is the safe locked? How solid
is the construction? Are there special passages? Is every type of intrusion detected? The advantage
of this way of looking is that one can see how well or how poorly something has been built and pinpoint the causes of weaknesses. This makes it much easier to spot vulnerabilities, as opposed to trying many potential vulnerabilities, like a penetration test does. In that sense the outside approach
can be compared to attempting to open windows and doors to get inside. Both approaches are
complementary. See Appendix A.3 for a more detailed discussion on the differences between SIG’s
approach and penetration testing.
A code review aims to identify vulnerabilities that form a threat now and in the future. A penetration test assesses whether a system is able to withstand a set amount of known attacks but cannot
establish whether this is the case for unknown, future threats. A quick view inside a bank building
can reveal that the vault door is unlocked or that a passage exists to an adjacent building. Not being
able to enter a bank building today does not imply that the money is protected from a different attack tomorrow.
3.2
Security Risk Assessment process
The SecRA approach encompasses the SIG security model, the SIG maintainability model and a
method to assess the role of processes and organization, based on what is available in various international standards such as ISO27001 [7], BSIMM and MS-SDL. Figure 2 shows a diagram of the
SecRA process, involving several sessions in order to get a clear picture of strategy, organization,
process and technology.
Strategy
session
Technical
session
Technical
validation
session
Kick off
Risk
validation
session
Final
presentation
Final
report
Analysis in SIG laboratory
Review of processes and architecture
Figure 2 - Software Improvement Group “Security Risk Analysis” process outline
The Software Improvement Group has a proven track record in software quality and assessment of
code in which technical findings are linked to processes and put into organizational context. SIG’s
distinguishing capacity lies in the unique combination of technical analysis, understanding of the
software life cycle and advisory capabilities at the C-level. In addition, SIG offers a more thorough
assessment than the one provided by a simple scan based on the OWASP top-10, in the meanwhile
a more pragmatic and lightweight analysis than a heavy one like Common Criteria assessment. The
advisory services and the underlying rating model enable a client to invest in security in an efficient
and effective manner, in order to:
•
•
•
•
•
•
Avoid financial loss;
Prevent reputational damage;
Stop information leakages;
Minimize downtime;
Trace back, and minimize the impact of security issues when they do happen;
Limit the cost of security in system development.
In addition, the SecRA expresses the security quality level with a rating between one and five stars.
The more security best practices are implemented, the higher the rating. Five stars denotes that all
4
conceivable security best practices have been implemented. This allows comparing the level of security with the rest of the industry, based on systems that SIG has assessed.
The result of a SecRA is security control: a clear understanding of the security risks and a roadmap
with recommendations to mitigate them. The Software Improvement Group does not implement
these recommendations and therefore has an independent and objective position.
4 Afterword
Since the growing number of security incidents are for a large part caused by software problems it is
essential to assess security in a number of ways:
1.
2.
3.
4.
Analyse software quality to determine how well security aspects have been implemented in
the relevant parts of an application;
Determine the general build quality of the software, which indicates the risk of new vulnerabilities or data leaks to appear;
Assess how the relevant processes mitigate security risks;
Test the software’s security in production by penetration testing.
In this way the security is visible and controllable, from the early stages of development, taking into
account not only known attacks, but also new threats, and taking into account many security aspects that cannot be seen from the outside. The Software Improvement Group offers a security risk
assessment service that includes all the approaches above to provide insight in the security risks
and a roadmap to address root causes, first things first.
About the Software Improvement Group
The Software Improvement Group is a management consultancy that focuses on software-related
challenges. We provide management with fact-based insight into their current IT situation, along
with razor-sharp, pragmatic and highly actionable recommendations on how to improve on that
situation. We know how to govern software projects effectively, when to invest in quality improvement, how to rationalize, and how to control cost. By consolidating low-level technical analysis and
high-level financial analysis we make sure our recommendations are the best fit for your specific
business needs.
References
[1] Forrester. The software security risk report: The road to application security begins in development. Technical report, Forrester Consulting (Commissioned By Coverity), 2012.
[2] “Close to 75% of today's attacks are tunnelling through applications”, Lanowitz, T., “Now Is the
Time for Security at the Application Level”, Gartner, December 1st, 2005.
[3] Matthew Finifter and David Wagner. Exploring the Relationship Between Web Application Development Tools and Security. In USENIX Conference on Web Application Development (WebApps).
USENIX Association, June 2011.
[4] ISO. ISO/IEC25010:2011 Systems and software engineering – Systems and software Quality Requirements and Evaluation (SQuaRE) – System and software quality models. Technical report, International Organization for Standardization, 2011.
[5] H. Xu, J. Heijmans, and J. Visser. A practical model for ratings software security. In Proceedings of
the 7th International Conference on Software Security and Reliability, 2013.
5
[6] I. Heitlager, T. Kuipers, and J. Visser, A Practical Model for Measuring Maintainability, In proceedings of the 6th International Conference on the Quality of Information and Communications Technology (QUATIC 2007), pages 30-39, IEEE Computer Society Press, 2007.
[7] ISO. ISO/IEC27001:2013 Information technology – Security techniques – Information security
management systems – Requirements. International Organization for Standardization, 2013.
[8] Nine governmental organisations. Common methodology for information technology security
evaluation (evaluation methodology) version 3.1 revision 4 (CCMB-2012-09-004). Technical report,
Nine governmental organisations, September 2012.
[9] The OWASP Foundation. Open web application security project verification standard. Technical
report, The OWASP Foundation, 2009.
[10] B. Boehm and V. R. Basili. Software defect reduction top 10 list. IEEE Computer, 34:135–137,
2001.
6
A. Appendix
A.1
The SIG security model based on ISO 25010 in detail
The SIG security model (version 2.0) based on ISO 25010 is as follows:
Se
re
cu
Ev
e
us
ce
en
id
rm
n
io
t
en
at
ge
th
ng
a
or
re
st
c
ifi
ss
th
ng
h
gt
en
em
ag
an
a
er
ce
tr
e
tr
ts
en
ac
st
t
da
v
ut
tp
ou
re
cu
nd
ed
em
ag
an
th
ng
ts
en
re
t
or
sp
an
st
em
ag
an
m
m
tr
n
tio
riz
ho
ut
Se
A
s
n
io
es
ss
cc
a
ic
a
t
da
Authenticity
ta
pu
In
Se
A
re
cu
tif
en
Id
Se
Confidentiality &
Integrity
Non-repudiation &
Accountability
X
X
X
X
X
X
X
X
X
Figure 3 - Mapping of 9 system properties to ISO/IEC 25010 security sub-characteristics
The five security sub-characteristics according to ISO 25010 are:
•
•
•
•
•
Confidentiality: Ensures that data are accessible only to those authorized;
Integrity: Prevents unauthorized access or modifications;
Non-repudiation: Actions or events can be proven to have taken place;
Accountability: Actions of an entity can be traced uniquely to the entity;
Authenticity: The identity of a subject or resource can be proven to be the one claimed.
SIG uses the security model to review code, design, process and software configuration by measuring how well best practices have been applied in 9 assessable system properties:
• Secure data transport;
• Identification strength;
• Access management strength;
• Session management strength;
• Authorized access;
• Input and output verification;
• Secure data storage;
• Evidence strength;
• Secure user management;
By rating the quality of several aspects of the system properties, each property is scored on a scale
between one and five stars, which then is mapped to a score for each sub-characteristic and then
into a final system score.
A.2
A wide variety of IT security approaches and frameworks
Security control is hard to attain both in terms of processes and products. Figure 4 shows an overview of the variety of existing approaches to security.
7
BSIMM
process
NEN 7510
OpenSAMM
ISO 27001
MS-SDL
process
models
management
systems
development
Common
Criteria
operations
modeling &
measurement
testing &
monitoring
intrusion
detection
CWE/SANS
Top-25
ISO 25010
OWASP
ASVS
SAS 70
CVSS
product
ethical
hacking
penetration
testing
Figure 4 - Synthesis of cyber security assessment and risk mitigation approaches
The dimensions used in the circle in this image are the software life cycle (from development to operations) and a distinction between the software product and the processes governing its life cycle.
The quadrants are complementary in the sense that organizations should address IT security by using approaches covering all sections.
Security assessments typically encompass penetration tests, vulnerability scans, and organization
audits. Evaluation of security risks in source code and software architecture (code / design review), a
decidedly more technical approach, is less common. Well-known standards are the Common Criteria [8] and the Open Web Application Security Project (OWASP) Application Security Verification
Standard [9]. The former one typically involves a major effort over a protracted time period, while
the latter is limited in scope to web applications.
A.3
How does code/design review differ from penetration testing?
A common way of assessing security is black box penetration testing: an organized attack on an IT
system. The goal of such an attack is to discover vulnerabilities by using or abusing the system – not
by looking at how it is constructed. While this is very useful for finding vulnerabilities, it has three
key limitations that can be complemented by performing a code/design review:
1.
A penetration test does trial and error on a selection of known attacks on a selection of system elements. All known attacks cannot be tried because the test is bound by time. In contrast, by looking at the inside of the system through code/design review, vulnerabilities can
become logically apparent without trial and error.
2.
A penetration test is limited by what can be seen from the outside; the perimeter. By testing
a system one cannot see how well and how generic all internal parts of the system are built
to protect against (future) threats and data leaks. Is data stored securely in the database?
Are backups treated with care? Is sensitive data accidentally logged on the server? What has
been done to maximize resilience, to detect, reconstruct, and recover from attacks? In contrast, these aspects can be analysed through code/design review.
3.
A penetration test can only be applied to a system or component that is finished and therefore its benefit is limited during development. Adding or fixing security when software is already finished is expensive. Some studies report it to be 100 times more expensive to add
security after deployment compared to addressing it in the design stage [9]. In contrast, a
code/design review is possible from the design stage of the software development life cycle.
8

Download Report