Build it and They (won’t) Come?

Reflecting on 35 years of performing analytics to support internal audit, finance, and fraud detection has led me to a – not so startling – revelation: you can build the best analytics in the world, and they still might not be used by internal audit.  Analytics I have developed, even with significant client input, have not been used or were limited in their use: either time or extent. The question is why?

Applying an audit approach to the problem:

• Risk: There is a risk that internal audit will not use analytics to support their audits resulting in audits that are more costly to perform, longer, and less effective than if  analytics were used.
• Criteria: The IIA standards, many professional articles, and CAEs state that analytics should be used.
• Condition: Limited use is being made of analytics beyond simple things like A/P duplicates.
• Cause: ?
• Recommendation: ?

It is impossible to make a valid recommendation regarding why analytics are not used without understanding the root cause for this failure. So, I offer my reflections on what I think are the primary reasons why analytics were not adopted.  For each cause, I will provide a specific recommendation to address the cause.

Note: This article focuses on the case where there is already some form of analytics capability used by internal audit, but it is not achieving the expected results.  For information on developing and maintaining an analytics function, refer to: https://caats.ca/2021/09/08/building-a-sustainable-analytics-function/

The primary causes of the failure to successfully employ analytics are:

Not understood: A recurring problem is that the analytics are not understood.  The auditors do not understand how the analytics worked and are not able to explain/defend them to the client.  The auditors do not see how the analytics results are related to the risk or control weakness. As a result, they cannot use the results of the analytics to identify control weaknesses and to make recommendations to address the cause of the weakness or to reduce the risk.

The users of the analytics must be intimately involved in their creation; and the analytics must be integrated with the audit program.  This means that the analytics, and results, are clearly linked to a specific risk and related control.  In addition, the auditors must be supported so they can explain the analytics (e.g., Yes, the duplicate test excludes reversed invoices) and articulate a recommendation that addresses the root cause. 

There is a mutual responsibility  for the auditors and the analytics developer to work together to identify useful analytics and to understand how they support the audit program.  By working together, the analytics will be integrated in the audit process and tied directly to audit steps and objectives.  It will also ensure that the auditors are able to explain the purpose, use, validity, and result of the analytics to the client.

Not accepted: There is a disconnect between the CAE and the Audit Director regarding the importance and reliance on analytics.  In addition, the use of analytics is not accepted by the client.

The adoption of analytics is very much a change initiative. All levels within the internal audit function must recognize the value of analytics and all levels of audit management must encourage their use. There is an ongoing need to educate all staff within the internal audit function, the audit committee, and senior management of analytics.  This includes informing the audit committee on how analytics will be used to support upcoming audits.  This will accomplish several things: the use of analytics will have the audit committee approval; senior management will be aware that audit will be asking for, and using, analytics as part of the audit process; the CAE, audit director, and team leader will understand that there is an expectation that analytics will be used to support the audit; and that there will be a requirement to report back on the use of analytics.

During the planning phase of the audit, the audit team leader develops the audit plan which includes the audit objective, steps to be performed by the auditors, and how the audit will be conducted (i.e., analytics will be used).  Client management should review and approve the audit plan. This includes understanding which analytics will be performed, data sources and access requirements, and how the analytics will be used to support the audit. Analytics should be linked, explicitly, to specific audit objectives (risks/controls) to ensure that all parties understand the role of the analytics in developing audit conclusions and recommendations.

Note: I produce a monthly report for the audit committee that highlights the use of analytics by audit conducted in the previous quarter and the planned use of analytics for the next quarter. This is done to encourage the use of analytics and to promote the value of analytics employed.

Difficult and Time Consuming: Accessing and understanding what data is required is not an easy task and takes time and effort.  Not enough planning and relationship building is spent on greasing the skids.

In general, access to the data must be supported by the audit charter, the audit committee, and senior management. Internal audit must have, or have access to, technical resources that are able to build relationships with data owners and IT system personnel.  These resources must develop an understanding of the IT systems containing the data, the authority and process required to extract, transform, and prepare the data for audit’s use.

Building a knowledge of potential data sources, ensuring access rights, and developing and maintaining a personal network of IT specialists and system owners is an ongoing job – not something that should wait for the launch of an audit.

Note: As thea resource responsible for supporting audit analytics, I review the Risk-based Audit Plan – covering the proposed audits for the next two years – getting an early start on identifying the sources of information, system owners, access requirements, etc. so that when the audit starts I already had a great deal of knowledge about the underlying systems and often have already developed data extraction, cleansing and loading functionalities.

Concluding Thoughts

Unless you address the lack of understanding and acceptance as well as the misconception regarding time and effort, your analytics usage will be sub-optimal. It is not simply a case of ‘build it and they will come’. The development, adoption and use are a significant change management task.  While internal auditors often expect (recommend) changes that require management to adopt new business processes, to address control weaknesses and manage risk, they are often less likely to do the same with their own processes and procedures. Practice what you preach.

Dave Coderre

www.caats.ca