This was my first attempt at identifying risk to support the development of the annual risk-based audit plan (RBAP). I have been involved in the development of the RBAP – even responsible for it – over the years and always felt that it was more professional opinion than anything else. Some people built a spreadsheet with weighting factors 1-5 and fooled themselves into believing that there is a logic and quantitative underpinning to the RBAP, but in the end, the auditors are providing the weighted scores based on professional opinion.
My approach was to use data analytics to support the qualitative aspects of the plan (auditor judgement, interviews with managers, previous audit results, etc.). This was for two reasons: first, quantitative indicators are easier to update; and second they provide assurance that we were also considering emerging risks.
Below is part 1 of an article I submitted to the IIA magazine. It was not published because they did not consider it to be “relevant to internal auditors” (????????), despite the fact that the IIA standards call for a continuous risk assessment. I think that the reviewers didn’t understand the ease and utility of developing the data driven risk indicators. I hope you find the article useful.
Developing data-driven indicators of risk to support the ongoing assessment of risk – Internal auditors face a daunting task of identifying and assessing risk. The results of this activity are critical as they serve to ensure that scarce audit resources are being expended on activities that best address the risks identified by senior management. The initial assessment of risk typically includes reviews of the corporate risk profile, business plans, financial statements, previous audit reports, and interviews with senior managers with question such as “What keeps you awake at night?”. The process can take weeks even months to complete. Contrast this with the IIA standard #2010 which states that the chief audit executive must review and adjust the plan as necessary, in response to changes in risk, operations, programs, systems and controls and you can see where audit has a problem.
The current approach to risk-based planning process is time consuming and subjective – relying primarily on qualitative information and auditor professional judgment – and is not easily updated. As a result, a robust risk assessment process may only be performed once every two or three years, and only “updated” or “reviewed” on a yearly basis. This process is not only failing to meet IIA standards, but is also not addressing the Chief Audit Executive or senior management’s needs.
A PWC study of the internal audit profession in 2012 found that auditors need to improve risk assessment processes so they can analyze significant risks more frequently than current audit cycles typically allow. The study concluded that the ability to identify and analyze emerging risks and trends is essential; and internal auditors need to adopt a continuous, comprehensive approach to risk assessment. But this is not possible using current risk assessment methods given the rapidly changing competitive, legal, and technological environments.
As the risk landscape becomes more complex, stakeholders expect internal audit to be involved in the organization’s areas of greatest risk while simultaneously maintaining the traditional and critical focus on controls and compliance. This requires internal audit to become more agile and creative in order to meet these multiple priorities. The chief audit executive must determine not only how to address corporate risks by deciding ‘what risks should be audited’, but also conduct an ongoing assessment of risks to identify emerging risks.
Richard Chambers, President of the IIA, identified his top ten imperatives of the decade which highlight the challenges that auditors must face to provide value to management. These include enhancing proficiency with data mining and analytics; providing assurance on risk management effectiveness; and enhancing and leveraging a continuous focus on risk.
Accordingly, internal audit needs to maximize the use technology and analytics and develop an efficient and effective ongoing risk assessment process which uses key risk indicators (KRIs) that are data-driven. The introduction of data-driven risk indicators would supplement the qualitative information and auditor judgement and provide additional rigour to the risk assessment process. They would also allow auditors to identify changes in corporate risks on a more continuous basis, well in advance of breakdowns in internal controls, and would focus discussions with senior managers on the areas of highest risk.
The corporate risk profile often pinpoints the major risks facing the organization but does not identify where audit resources should be deployed to address the corporate risks. Thus, when examining corporate risks the chief audit executive should be able to drill down into each corporate risk and assess the risk at lower organizational levels. This can highlight specific entities or activities that are having the largest impact on the corporate risk (i.e. the areas with the highest risk) such as risks unique to a region, plant or key activity of the company. For example, if a key corporate risk is the discontinued generation and failure to protect intellectual property (IP), then R&D activities in specific geographic locations (those with a high percentage of workers nearing retirement age, etc.) would benefit more from an audit than areas that have less impact on IP risk. The establishment of data-driven risk indicators and the assessment of these at the organizational entity level helps to ensure the application of audit resources to those areas that will have the most impact on the overall corporate risks rather than trying to audit the entire corporate to assess a corporate risk.
Additionally, while the corporate risk profile may focus primarily on financial and cyber security risks, the chief audit executive should consider other risks that may be critical to the success of the organization. The IBM CFO Study 2008 findings suggest that businesses need to develop a more holistic view of risk. Facing a wide range of risks requires enterprises to broaden their risk apertures and focus on those risks with the greatest potential impact and occurrence; this includes strategic, regulatory and operational risks. To ensure that audit is assessing all risks, nonfinancial and macro trends such as health and wellness, workforce, product safety and other factors must be considered by internal auditors when assessing management’s risk management practices. This can be accomplished by defining key risk indicators for various categories of risk, such as financial, HR, technological, external environment, legal and regulatory, strategic, governance, and operational. This allows audit to examine risk from a functional perspectives on an ongoing basis and identify emerging types of risk. For example, tracking changes in federal legislation could highlight an increased legal risk before costly non-compliance with environmental regulations occurs.
Risk has long been measured by probability and impact. Developing data-driven indicators of risk requires audit to identify quantitative drivers that measure the probability of a risk occurring and the impact if it does. From a data-driven perspective, probability will increase when an entity, activity or process has more variability/change and complexity; and impact is a function of volume, materiality and size. An increase in variability/change and/or complexity of an area, and/or higher volume, materiality or size will be indicative of an increased level of risk for the entity. Each organizational entity or activity can be assessed on specific the data-driven risk indicators and an overall risk level determined. In addition, by examining the quantitative risk indicators on an ongoing basis, changing levels of risk or emerging risks can be highlighted.
To be effective, data-driven indicators must react to changes in risk levels and support the assessment of risk for each organizational entity/activity. To be efficient, they should be a by-product of operational systems (e.g. financial, HR and operational systems). Quantitative indicators obtained from operational information systems are less intrusive, easier to capture and measure, and do not rely on subjective assessments. This means that the risk assessment can be evaluated throughout the year (e.g. quarterly or semi-annually) with minimal effort. This allows audit to determine if the risk levels have changed for any of the activities or entities.