Internal Audit Testing and Sampling Techniques Chartered Institute of Internal Auditors – May 2014 Controls Testing PwC Slide 1 Testing Priorities Risk B1 Risk B2 Risk C2 Risk A2 Risk C1 Risk A1 Controls testing Testing techniques Inquiry Observation Inspection/ Examination Re-performance PwC Slide 3 Controls testing Control testing Tests of controls are designed to obtain evidence to assess their operating effectiveness. Operating effectiveness means that the controls are functioning as designed on a consistent basis over the period under examination. PwC - Inquiry – consists of seeking information of knowledgeable people within the client - Observation – consists of looking at a process being performed by others - Examination ◦ inspection of information or data ◦ walkthrough – confirming our understanding of a process by tracing individual transactions from beginning to end - Re-performance – independent execution of procedures that were originally performed as part of management’s internal controls Slide 4 Controls testing Determining which Testing technique to use Re-performance Level of Comfort Inspection/ Examination Observation Inquiry PwC Slide 5 Controls testing Determining which testing technique to use Considerations: • The susceptibility of the control to change. • The frequency and extent of the control. • Our initial view of the likelihood of control weakness. • Significance of the control to the control environment and how much reliance is being placed on it. PwC Slide 6 Value Protection - execute Sampling PwC • Sampling is the application of auditing procedures to a representative group of less than 100% of the items within a homogenous population 3 Steps to follow: 1. Determine the control test objective, population and sampling unit • We use non-statistical sampling 2. Determining the sample size 3. Selecting the sample for testing Slide 7 Value protection – Execute Sampling Manual Controls Depends on: • Frequency of control or population size • Level of evidence that is judged to be necessary The table below, can be used as a general rule; however, we may use a smaller sampling size: Frequency of Control Assumed population size Sample Size Annual 1 1 Quarterly 4 2 Monthly 12 2 (minimum) to 5 (maximum), Select 3 if you require a mid-range. Weekly 52 5 to 15. Select 10 if you require a mid-range Daily 250 20 to 40. Select 30 if you require a mid-range Multiple times per day Over 250 25 to 60. Select 30 or 45 if you require a mid-range PwC Slide 8 Value protection – Execute Sampling Manual Controls Following factors may indicate that sample sizes should be selected at the higher end of the ranges: - The greater the potential financial loss or adverse event to the company if the control is not effective or fails: - The more complex the control - The greater the degree of judgment in control operation PwC Slide 9 Value protection – Execute Sampling Automated Controls If IT General Controls have been tested and found to be effective, it may be sufficient to only test one operation of the Automated Control PwC Slide 10 Documentation Audit documentation Audit documentation must contain sufficient information to enable an experienced auditor, having no previous connection with the engagement to: Remember: if what you did isn’t documented, it’s the equivalent of not performed! - Understand the nature, timing, extent and results of the procedures performed, evidence obtained, and conclusions reached - Determine who performed the work and the date such work was completed, as well as the person who reviewed the work and the date of such review. - Understand the linkage between conclusions and facts - Document what you have done and how you reached your conclusions PwC Slide 11 Confidential The changing shape of internal audit Increased use of technology Drivers for change (top 3): 1. Complexity increased use of technology within the business | higher volume of transactions | increased automation | businesses driven by data | devil is in the detail | how do you find a needle in the hay stack? 2. More for less pressure to deliver more with less | value | quality | efficiency | insight | pressure to deliver with less resource and using samples? 3. Resources skills sets | innovation | technologically minded team | reduced fear factor | development opportunities for your people? May 2014 PwC 12 CIIA - 14 May 2014 Confidential May 2014 PwC 13 CIIA - 14 May 2014 Confidential What are CAATs? Computer Assisted Audit Techniques A means of accessing large amounts of data in a format that can provide transparency not attainable through other auditing procedures. The results may be used to identify areas of key risk, fraud, errors or misuse; improve business efficiencies; verify process effectiveness; or influence business decisions. (ISACA August 2011) May 2014 PwC 14 CIIA - 14 May 2014 Confidential Data analytics - methodology Extract and upload raw data May 2014 PwC Map and organise data Analyse and visualise data Finalise audit evidence, identify anomalies and insight 15 CIIA - 14 May 2014 Computer Assisted Audit Techniques Advantages How can you ever pick a sample that is representative? Expandable model, allowing tests to be refined, tuned, added, removed Standing still or moving with the times? You can quickly identify and address emerging issues and risks In the future it will allow audit tests to be “pushed” into the organisation as monitoring controls May 2014 PwC 1 2 3 4 5 Increased coverage – 100% of transactions Efficiency – repeatable and automated Value and insight – improve the perception of IA Basis for prioritisation of where to look next in the organisation Climb the maturity curve – predictive business enabler 16 CIIA - 14 May 2014 Confidential Data analytics on vendor standing data Identify duplicate vendors based on the same or similar (fuzzy match) vendor name. Identifying and resolving duplicate vendor records is important as otherwise this could lead to loss, error or fraud. For example: loss of purchasing volume discounts available where spend with a specific supplier is recorded across two or more records for the same supplier, error if one vendor record is updated but the duplicate vendor record is not resulting in incorrect and inconsistent records, and fraud for example where duplicate vendor records are used to process payments below a review threshold. 12,253 vendors listed in standing data 1,031 perfect duplicates May 2014 PwC 46 96 fuzzy match with 1 character difference fuzzy match with 2 character difference 231 fuzzy match with 3 character difference 17 CIIA - 14 May 2014 Exercise You are the internal auditors to an NHS Trust. You have been asked to undertake a review to assess the accuracy of the information used to support the KPIs that are reported to the Board on a monthly basis and to external regulators quarterly. What would you consider in devising a testing approach? PwC 18 Exercise You are the internal auditors to an NHS Trust. You have been asked to undertake a review to assess the accuracy of the information used to support the KPIs that are reported to the Board on a monthly basis and to external regulators quarterly. You are driving to work and hear on the radio that a NHS Trust in another part of the country has got into serious trouble for mis-reporting cancer waiting times data. There seems to be an issue in distinguishing between cancellations and DNAs. Would you do anything differently. PwC May 2014 19 Exercise You are the internal auditors to an NHS Trust. You have been asked to undertake a review to assess the accuracy of the information used to support the KPIs that are reported to the Board on a monthly basis and to external regulators quarterly. You are driving to work and hear on the radio that a NHS Trust in another part of the country has got into serious trouble for mis-reporting cancer waiting times data. There seems to be an issue in distinguishing between cancellations and DNAs. In checking the above with the client you realise that they may have innocently mis-interpreted the above and that this might mean that they have been misreporting data to their external regulators. What would you do? PwC May 2014 20 This publication has been prepared for general guidance on matters of interest only, and does not constitute professional advice. You should not act upon the information contained in this publication without obtaining specific professional advice. No representation or warranty (express or implied) is given as to the accuracy or completeness of the information contained in this publication, and, to the extent permitted by law, PricewaterhouseCoopers LLP, its members, employees and agents do not accept or assume any liability, responsibility or duty of care for any consequences of you or anyone else acting, or refraining to act, in reliance on the information contained in this publication or for any decision based on it. © 2014 PricewaterhouseCoopers LLP. All rights reserved. In this document, “PwC” refers to PricewaterhouseCoopers LLP (a limited liability partnership in the United Kingdom) which is a member firm of PricewaterhouseCoopers International Limited, each member firm of which is a separate legal entity.