As a promoter of data analysis for fraud, internal controls, and audit for more than 30 years, I am often asked, “Which software tools or application should I use?” While I have packages that I have used for 30+ years, and am a co-founder of an data analytics service provider (WWW.CTRLmatters.Com) , I hesitate to answer the question. The capabilities and robustness of data analytics are changing very quickly, and data analytics is an important element of identifying and assessing risk, including but not limited to fraud risk. Instead, I try to respond with general statements on how to manage risk with data analytics, focusing on the types of information that robust analytics can provide and how analytics should be an integral part of a successful risk management program.
I have been fortunate enough to be a contributor to the COSO Fraud Risk Management Guide. My focus has been on the use of data analytics to support fraud risk management. I, and the others working on this guide, believe that analytics should be considered essential in prevention and detection of fraud. With that in mind, it is important to mention that the tools themselves are not the controls, nor are they the fraud management program. The technology and analytics can only support a strong fraud risk management program. Controls must be designed effectively before being automated, and this includes having the right monitoring parameters, completeness and coverage of fraud red flags and scenarios, thresholds timing, etc. IT General Controls should be prominent and prevalent in the prevention or detection of fraud.
So, it is not a question of which tool or software, but how data analytics can support the identification and assessment of risk that matters. At the same time, if the only tool you have is a hammer, all your problems will look like (or be treated as) nails, is still a valid concern. There has been a continual ebb and flow between IT as responsive and IT as the driver. Should IT serve the business, or should IT dictate the requirements to which business must conform? This dilemma won’t be solved anytime soon, however, I contend that we must understand the requirements before deciding on the tool. In addition, requirements will continue to change and evolve, so any chosen tool must support this.
Which bring me to AI tools. This is all the rage – use AI to identify anomalies, fraud, etc. While AI can identify irregularities or transactions that should be reviewed, it often does not provide an understandable reason why the transaction was flagged. “We use a multi-dimensional model to compare….” Great, but what needs fixing – what controls are not working and how do I put in appropriate preventive controls, so I don’t have this problem again.
I believe that data analytics – in supporting a business process – must be closely linked to the business process rules and the IT and general controls. But it must also look not only for anomalies and errors, but also for the root cause. What are the control deficiencies that need to be addressed? I would rather my analytics tell me what to fix than tell which transaction were wrong (i.e., don’t simply identify the duplicate payments which I need to recoup, but tell how they occurred and how to stop them in the future.) Recovery companies that analyze your accounts payable to find duplicates provide you a value in the recovered duplicates, but if they don’t identify the control weaknesses that allowed the duplicate to occur, you will have to hire them again next year.
Internal audit should also consider this. If you only identify problems, you are not part of the solution. I was an internal auditor for most of my career and also spent 10 years in the professional practices areas of internal audit. My biggest complaint with internal audits was not that they didn’t identify problems, but that they didn’t add value. If management is supposed to do “A” and audit found that they were not doing “A”, a recommendation that states “Do ‘A’” adds zero value. Audit was lazy and did not identify the root cause – why they weren’t they doing “A”? The same is true of analytics. If it only identifies the transactions that are in error or needs to be examined, it is of little value. The analytics should address more than the recovery or correction actions, but also the preventive and detective controls that will improve the process.
Concluding thoughts
Whether you are developing analytics or considering what tools/software will support your efforts, keep the goal in mind. An A/P audit should not have an objective (goal) of recovering duplicate payments; but rather, improving controls to reduce the occurrence of duplicate payments. The same should be true of your choice of tools. Will they help you to identify the root cause rather than simply identify potential inappropriate transactions?
This relates closely to my article on Better Audit Recommendations (https://caats.ca/2022/01/20/better-audit-recommendations/) which emphasized the important of developing a good audit objective. The objective should be related to addressing the control objectives (preventive and detection controls) and not focussed on the corrective controls. Focus on solutions not problems.
Dave Coderre
Thanks for posting, Dave. You provided us with interesting dinner conversation. My 15 year old programs and trains AI models in exercises related to voice recognition and self driving car response. It’s good to consider the limitations of finding root cause for problems.
My 14 year old and I prove the example you mentioned about unhelpful recommendations. Alex should do his chores. Alex didn’t do his chores. Solution: Mom keeps telling him to do his chores. We will shift our focus to consider the root cause!
LOL. With my daughter I had to learn that I couldn’t make HER care. Which from an audit perspective points to impact.