By David Coderre, Author of ‘CAATTs and Other BEASTs for Auditors’; ‘Internal Audit: Increasing Efficiency through Automation’; and ‘The Fraud Toolkit’
The notion of continuous monitoring was first introduced to auditors in the 1980’s. Its basic premise was using ongoing automated data analysis to draw conclusions concerning risk in a subject area. The results would help to determine where an audit was required and to focus the audit on the areas of greatest risk. Unfortunately, auditors were not ready; they lacked the tools and necessary data access or were unwilling to embrace this idea at the time.
Now, however, there is a proliferation of information systems in the business environment, giving auditors and managers easier access to more relevant information.
Further, the rapid pace of business requires prompt identification of, and response to issues. Sarbanes-Oxley Section 409 requires timely disclosure to the public of material changes to financial conditions. This regulation, plus changes in auditing standards and the evolution of audit software, has combined to persuade auditors to adopt new approaches to assessing information.
Increasingly, the marketplace demands independent assurance that control procedures are effective and that the information produced for decision-making is both relevant and reliable. Often the need for high quality information for decision-making in the highly volatile business environment is greater than the need for reliable historical cost-based financial statements. If a company can’t adjust to changing market, technological and financial conditions, it won’t be in business for long. The environment, technology and audit standards are driving auditors to make more effective use of information and data analysis and encouraging auditors to adopt continuous monitoring. This has produced a shift in the focus of internal audit activities.
However, many auditors are still resistant to or confused about continuous monitoring, so its acceptance and implementation is far from widespread. One of the main reasons for the reluctance is the term ‘monitoring,’ which is seen as a management function. Another point of confusion is the application of the term “continuous monitoring” to both instantaneous auditing (a review of transactions in real time) and to the notion of ongoing or frequent, but not real time, audits.
Real-time analysis is still beyond the capabilities of many audit organizations. Therefore, proponents of continuous monitoring now define it as the identification of systems or processes that are experiencing higher-than-normal levels of risk, such as where the values of the performance attributes fall outside the acceptable range. In this context, continuous monitoring measures specific attributes that, if certain parameters are met, will trigger auditor-initiated actions. The nature of these actions will vary depending on the risk identified. They may range from sending an email to the manager to a rapid response audit of the area. For example, the financial system may notify the auditors of any journal vouchers over $250,000. The auditor’s response will depend on whether or not this is seen as a single item of concern or as a systemic problem.
Continuous auditing versus continuous monitoring
To help overcome some of the problems and confusion associated with the term “continuous monitoring,” auditors ought to consider the notion of “continuous auditing,” — a similar, but more powerful approach to identifying and assessing risk. I define continuous auditing as:
The identification and assessment of risk through the application of analysis and other audit techniques including, but not limited to,
- The identification of anomalies;
- The analysis of patterns within the digits of numeric fields (digital analysis);
- Comparisons against cut-off or threshold values;
- Comparisons across years; and
- Comparisons of one audit entity to another.
Both continuous monitoring and continuous auditing have their genesis in data analysis. But continuous auditing goes beyond simple data analysis and includes techniques from statistical analysis, trend analysis, digital analysis and neural networks. Typically, data analysis and, to a large extent continuous monitoring, is only used to identify transactions that fail a specified cut-off or threshold value, whereas continuous auditing helps auditors to identify and assess risk, as well as establish dynamic thresholds that respond to changes in the organization. Further, while data analysis contributes to an individual audit by identifying or supporting specific audit objectives, continuous auditing also supports risk identification and assessment for the entire audit universe – supporting the development of the annual audit plan – in addition to contributing to the objectives of a specific audit.
Continuous auditing is a unifying structure or framework that brings risk assessment, audit planning, digital analysis and the other audit tools and techniques together. It supports the macro-audit issues, such as using risk to prepare the annual audit plan; and micro-audit issues, such as developing the objectives and criteria for an individual audit. The main difference between the macro- and micro-audit levels is the amount of detail that is considered. The annual audit plan requires high-level information to establish the risk factors, prioritize risks and set the initial timing and objectives for the planned set of audits.
Individual audits start with the risks identified in the annual audit plan but use digital analysis and other techniques (interviews, control self-assessment, walk-throughs, questionnaires, etc.), to further define the main areas of risk and focus the risk assessment and subsequent audit activities.
There also are a number of differences between continuous auditing and continuous monitoring. The main differences are:
- Continuous auditing recognizes and acknowledges that monitoring is a management function – not an internal audit function.
- The frequency of continuous auditing is based on the assessed level of risk and is not “continuous” unless the level of risk justifies a real-time analysis of transactions.
- Continuous auditing uses not only the comparison of both individual and summarized transactions against cut-off or threshold values but also the comparison of an entity against other entities (e.g. one operational unit to all other operational units) and a time-wise comparison of the entity against itself (e.g. the entity’s performance over the last five years compared to its current performance).
- Continuous auditing also allows auditors to follow up on the implementation of audit recommendations.
Continuous auditing can be used by audit to determine if risk is at a level where audit intervention is required. However, it is not a form of monitoring that would determine if operations are functioning properly (which is a management issue). Continuous auditing allows auditors to quickly identify instances that are outside the allowable range (known thresholds), and those that can only be seen as anomalies when compared to other similar entities or when viewed across time (unknown thresholds). Simply knowing that an audit entity processed a journal voucher that is greater than a cut-off amount will not help auditors to gauge whether the entity has improved in its use of journal vouchers.
Continuous auditing seeks to measure not only transactions against a cut-off but the totality of the transactions. This allows one to test the consistency of a process by measuring the variability of each dimension. For example, measuring the variability in the number of defects is a method for testing the consistency of a production line. The more variability in the number of defects, the more concerns about the proper functioning of the production line. This premise can just as easily be applied to the measurement of the integrity of a financial system by measuring the variability (e.g. number and dollar value) of the adjusting entries over time and in comparison to other similar entities. The concept of variability, over time and against other audit entities, is the key differentiating factor in continuous auditing versus continuous monitoring or embedded audit modules.
Auditors need to be considering questions like: How many journal vouchers were processed this year? What percentage was above the threshold amount? How does this compare to last year and to other audit entities? Can we tighten the criteria and lower the cut-off value? Answering these questions will allow auditors to develop a dynamic set of thresholds that provide a better idea of the direction the organization is headed, rather than simply identifying a transaction that failed to meet a static cut-off value.
Supporting audit follow-up
Finally, continuous auditing supports automation of audit recommendation follow-up. With continuous auditing, auditors can track specific data-driven measures of performance to determine whether management has implemented the agreed-upon recommendations and whether they are having the desired effect. Tracking performance over time is critical to ensuring the organization is being successful in meeting established goals and in identifying additional actions to be taken. It is an integral element of performance measurement and continued improvement in operations.
Audit, through continuous auditing, can assess the quality of performance over time and ensure the prompt resolution of identified problems. Further, once the risks related to an activity are identified and activities to reduce such risks are undertaken, the review of subsequent performance (continuous auditing) can gauge how well the mitigation efforts are working. As the actions of an organization become more observable, continuous auditing facilitates the implementation of ongoing quality improvement and assurance.
The data-driven predictors of performance must be responsive to changes in performance, provide an early warning when performance is deteriorating, be easy to use and not be resource intensive. They should help an organization answer three basic questions if the indicator goes “Red:”
- What happened?
- What is the impact?
- What are we going to do about it?
Continuous auditing applied to accounts payable – an example
While continuous auditing can be used in any area of the organization, a simple example involving accounts payable illustrates the differences and strength of this approach. The example assumes that there are numerous separate accounts payable (A/P) processing centers, of different sizes, performing similar functions. The example will be used to discuss four main objectives:
- Identification and assessment of risk related to the accounts payable processes.
- Identification of trends related to performance and efficiency.
- The identification of specific anomalies and potential frauds.
- The tracking of the implementation of audit recommendations and their affect on accounts payable operations.
In each case, the analysis would consider trends over time and compare the accounts payable section under review to other accounts payable sections within the organization. Benchmarking against external A/P operations adds another dimension to the examination.
Risk Identification and Assessment. A wide variety of data-driven and non-data-driven risk factors should be included in the initial risk assessment. A comprehensive evaluation of business performance looks at cost, quality and time-based performance measures. Cost-based measures cover the financial side of performance, such as the labor cost for accounts payable. Quality-based measures assess how well an organization’s products or services meet customer needs, such as the average number of errors per invoice. Time-based measures focus on efficiency of the process, such as the average number of days to pay an invoice.
It is also possible to determine, for each A/P section, the types of transactions and dollar amounts for each. For example, look at number of correcting journal entries and manually produced checks. These are indicators of additional workload. The analysis also will tell you how many different types of transactions are being processed. Generally speaking, there is greater complexity in operations when more transaction types are processed. You can also examine organization structure: reporting relationships, number and classification/level of staff, length of time in job, retention rates and training received – this data should be available from the HR system. The combination of this type of information with the transaction types and volumes can help to identify areas of risk, such as understaffing or lack of trained staff to handle complex transaction types
Trends in performance and efficiency. When considering A/P, trending data will easily identify performance and efficiency concerns. For example, for each A/P operation, continuous auditing can determine:
- Number and classification/level of accounts payable staff.
- Number of invoices processed by each user at either end of the spectrum. (Too many or too few can increase risk.)
- Average dollar cost to process an invoice.
- Average number of days to process a payment.
- Percentage of invoices paid late; percentage paid early. (Particularly telling if early payment discounts are not taken.)
- Percentage of adjusting entries.
- Percentage of recurring payments or Electronic Funds Transfer (EFT) payments.
- Percentage of manual checks.
- Percentage of invoices that do not reference a purchase order.
- Percentage of invoices that are less than $500. (Purchase card could be used for more efficiency and less cost).
Efficiency measures allow you to compare one audit area to another:
Analyzing trends can help to identify not only problems but also areas where improvements have been made. The graph below shows that Division D still has the highest percentage of invoices without a purchase order reference, but they have made considerable improvements over the previous year, whereas Division G’s percentage has gone up.
Identification of anomalies or potential fraud. Within A/P, possible anomalies and measures of potential fraud include:
- The identification of duplicate payments (should include a comparison to previous years to see if operations are improving).
- Invoices processed against purchase orders that were created after the invoice date (back-dated purchase orders).
- Number of invoices going to suspense accounts.
- Identification of all functions performed by each user to identify incompatibility or lack of segregation of duties.
- Identification of vendors that were created by, and only used by, a single accounts payable clerk.
- Identification of instances where the entry user is the same as the user who approves payment.
- Identification of instances where the payee is the entry or approving user.
- Identification of duplicates in the vendor table or of vendors with names such as C.A.S.H., Mr., Mrs. or vendor with no contact information, phone numbers or other key information.
Tracking of recommendations. The final objective of continuous auditing is the tracking of recommendations. The aim is to determine whether management has implemented the recommendations and whether the recommendations are having the desired effect. Possible measures include:
- Evidence of increased used of purchase cards for low dollar transactions (reduction in percentage of invoices less than $500 and increase in percentage of purchase card payments less than $500).
- Reduction of duplicates in the supplier master table.
- Decrease in the number and dollar value of duplicate invoices.
- Improvements in the days-to-pay figures (reduction in late payment charges, and more opportunities to take early payment discounts).
- Improved operations – lower cost per invoice, more use of EFT payments.
The graph below shows how continuous auditing can be used to determine whether A/P operations in each division have successfully implemented the recommendation calling for purchase cards to be used for low dollar transactions.
Preparing for continuous auditing
Continuous auditing starts with the selection of audit projects, continues into the conduct and reporting phase and culminates with the ongoing monitoring and follow-up activities. All stages of the process should be risk-based and, to the maximum extent possible, data-driven. The basic implementation strategy must include a consideration of the risk, an assessment of the baseline assurance, the design of the predictive indicators, monitoring for changing conditions and follow-up as required. More detailed steps include:
Audit plan preparation and planning phase
- Identification of categories/areas of risk.
- Identification of sources of the data to support risk assessment.
- Understanding of the data and an assessment of its reliability.
- Assessment of the levels of risk.
- Prioritization of risk.
- Selection of audit projects.
Audit conduct phase
- Integration of audit procedures and technology.
- Definition of relevant variables (predictors) to be measured.
- Definition of the criteria for these variables to be used to predict outcomes.
- Definition of the desired traits for the variables (normal range, anomalies).
- Measurement of the variables (predictors).
- Assessment of the predicted level of risk.
- Follow-up audit activity as required.
- Revision to variables that will be measured, criteria and the traits.
The implementation of continuous auditing will place certain demands on internal auditors. In particular, the audit organization will have to develop and maintain the technical competencies necessary to access and manipulate the data in multiple information systems. If the auditors are not already using data analysis techniques to support audit projects, the audit group will have to purchase analysis tools and develop and maintain analysis techniques. The implementation of continuous auditing will also require the adoption of the concept by all persons within the audit organization.
Monitoring and review is the final component of an effective control framework (COSO’s five elements of a control). It is a key ingredient in an organization’s continuous improvement process. An effective monitoring and review environment uses both periodic reviews and those undertaken by internal and external audit, as well as built-in review mechanisms and internal review measures.
Continuous auditing will support and strengthen the monitoring and review environment in an organization. Finally, it will help focus the audit effort but will not obviate management’s responsibilities to perform a monitoring function.
Continuous Auditing Checklist
The following list of activities and tasks support the use of continuous auditing:
Data Access and Use
- Develop/Maintain access to key application systems.
- Understand the applications.
- Assess data integrity and reliability.
- Develop and maintain analytical skills within the audit organization.
Tools and Techniques
- Purchase analysis tools.
- Develop and maintain analysis techniques.
Use risk analysis to select an area.
- Select a suitable target for continuous auditing.
- Define entities to be evaluated.
- Run the analysis and calculate the indicators.
- Compare results to previous periods as well as to similar entities within the organization.
Anticipate all exceptions.
- Identify the most critical reports to execute.
- Review the processing flow and past audits.
- Obtain best practices and external insights.
- Bring the key players together.
- Enlist the support of operation management to discuss the following:
- The objective of the program or organization.
- An assessment of the effects of these risks, and what factors can increase risk.
- Tools currently used to monitor risks.
- The involvement of all pertinent personnel, in order to detect weaknesses.
- The process of creating a monitoring report.
- Make results known to appropriate management.
- Monitor and evaluate effectiveness of continuous auditing processes.
Prioritize and plan audit frequency.
- Determine which exceptions should be investigated and compare timeliness versus effectiveness.
- Schedule audits and continuous auditing frequency.