Documenting an analytics test properly
What belongs on file when an audit test is run with a data analytics tool, so that a reviewer can follow it, understand the judgements and re-perform it.
By Achekam · · 4 min read · For Accountants & audit firms in South Africa, Nigeria, Kenya
The working paper says "Journal entry analytics performed – no exceptions" and has a screenshot attached showing a chart and a table with a handful of rows. The reviewer, a year later, is preparing for an inspection. Which journals were tested? What counted as an exception? What happened to the rows in the table? Nobody who did the work is still on the team.
Analytics produces results quickly, and that speed tempts teams to document the result and not the test. An inspector, or a reviewer from another office, needs the test. This article sets out what the file should hold so that someone else can follow the work and, if they need to, run it again.
Why a screenshot is not enough
ACCA warns that audit evidence on file may be insufficient where data analytics procedures and inputs are not fully documented. It gives the example of a screenshot of a tool's results, which may not record the input conditions and the detail of the testing.
A screenshot shows what the screen displayed at one moment. It does not show what data went in, what filters were set, which version of the data was used, or what the team did with what came out. Without those, the result cannot be checked or repeated.
What the standard asks of any documentation
ACCA explains that, under ISA 230, audit documentation should be prepared on a timely basis so that an experienced auditor with no previous connection with the audit can understand the nature, timing and extent of the procedures performed, the results and audit evidence obtained, and the significant matters, conclusions and significant judgements made.
ACCA also explains that the auditor must record the identifying characteristics of the items tested, who performed the work and the date it was completed, and who reviewed the work and the date and extent of that review.
Applied to analytics, "identifying characteristics" becomes concrete. For a journal entry test, it is the population tested and the rule that selected each flagged entry. For a three-way match, it is the fields matched and the tolerance applied.
Inputs, parameters and the data version
The first half of an analytics working paper describes what went in:
- Objective. Which assertion or risk the test addresses, in one or two sentences.
- Source data. Which system it came from, who extracted it and when, and for what period.
- Completeness. How the population was agreed to the ledger or trial balance, with the result.
- Version. How the file tested can be identified later, such as its hash and row count.
- Parameters. Every filter, threshold and rule applied, written out in plain words.
A reviewer who reads only this half should be able to say whether the test was designed to answer the right question.
Results, exceptions and follow-up
The second half describes what came out and what was done about it:
- The number of items flagged, and how that was arrived at.
- How the team chose which flagged items to investigate, if not all.
- For each item investigated, what was examined and what was concluded.
- Any item that could not be resolved, and how it was dealt with.
- The overall conclusion, and how it links back to the objective.
The definition of analytics itself points this way. As reported by ACCA, the IAASB describes data analytics for audit as discovering and analysing patterns, deviations and inconsistencies for the purpose of planning and performing the audit. The deviations are the point; the file has to show what became of them.
Reproducible draws and recorded extracts in Achekam
Two features of Achekam make the "inputs" half easier to complete.
Journal entry testing. Staff load a whole-population journal extract as a CSV file and map its columns, and Achekam flags entries such as manual, weekend, after-hours and post-close postings for staff to select and test. The CSV is not stored; its hash and row count are. Those two values identify the file that was tested, so the working paper can state them, and a reviewer can confirm that a file produced later is the same one. The firm still has to keep the file itself, or a copy it can retrieve, if it wants the test to be re-performed.
Audit Intelligence. Achekam risk-scores a loaded population and proposes a sample. A reviewer who did not prepare the plan approves it before it is drawn, and every draw can be reproduced from its stored seed. The methods available are random, systematic, monetary unit, specific items and all items, and no sample size is proposed without approved materiality and a confidence level. The seed answers the question "would the same items be selected again?" with a yes that can be demonstrated; the approval records who agreed the plan before anything was drawn. More on reproducible sampling is in reproducible audit samples.
A layout for an analytics working paper
| Section | Contents |
|---|---|
| Header | Client, period, area, preparer and date, reviewer and date |
| Objective | Assertion or risk addressed |
| Data | Source, extraction, period, completeness check, file hash and row count |
| Parameters | Rules, filters and thresholds, in plain words |
| Results | Items flagged and how selected for follow-up |
| Follow-up | Work done on each item and its conclusion |
| Conclusion | What the test shows, linked to the objective |
The general principles of audit documentation are covered in audit documentation under ISA 230.
ACCA's summary of the documentation duty
ACCA's technical article on the effective audit service explains what ISA 230 expects audit documentation to show and record. Read it alongside the firm's analytics template, and fill in every row for the next test before the screenshot goes on file.
General guidance only, not advice on your own circumstances. Tax and payroll rules change: check the current position with the authority or your adviser.