Testing every transaction does not prove the data
Why analysing every transaction does not make the client's data correct, and how auditors should word results so clients do not over-read them.
By Achekam · · 5 min read · For Accountants & audit firms in South Africa, Nigeria, Kenya
"You looked at every single invoice, so the numbers must be right." A finance director says it with relief at the close-out meeting, and it is easy to let it pass. It sounds like a compliment to the audit. It is actually a misunderstanding that can come back to hurt the firm when a misstatement later turns up in a population the team "tested in full".
Data analytics has made full-population testing ordinary, even on smaller audits in South Africa, Nigeria and Kenya. That is a real gain. But a test that covers every transaction answers only the question it asks, about data whose own reliability is a separate matter. This article explains the 100 percent testing expectation gap, what full coverage does and does not achieve, and how to word results so nobody reads more into them than they say.
What full coverage really buys
ACCA explains that data analytics often makes it possible to test every item in a population, improving the coverage of audit procedures and reducing or removing sampling risk. Sampling risk is the chance that the items you happened to pick are not representative, so your conclusion on the sample differs from the one you would reach on everything. Test everything and that particular risk falls away.
What does not fall away:
- The risk that the test was the wrong test. A rule that flags journals posted on weekends finds weekend journals. It does not find a fictitious sale posted on a Tuesday.
- The risk that the data is incomplete or wrong. Every item in a file is not every item in the business.
- Judgement risk. Someone still decides which flagged items matter and what they mean.
- The evidence gap. An analytic tells you an entry has certain features. It does not tell you the entry is supported by a real transaction.
The gap ACCA describes
ACCA warns of an expectation gap among stakeholders who believe that because the auditor tests every transaction in an area, the client's data must be entirely correct. The belief is understandable. "All" sounds like certainty in a way "a sample" never did.
The audit was never meant to deliver certainty. ICAEW, explaining ISA 200, sets the auditor's overall objective as obtaining reasonable assurance that the financial statements as a whole are free from material misstatement, whether due to fraud or error. Testing more items can make that assurance better founded. It does not change what kind of assurance it is. Our article on reasonable assurance takes that distinction further for clients who want it explained.
The data under the test
Before any conclusion on a full population, there is a prior question: is it the full population? ACCA cautions that the completeness and integrity of data extracted from a client's system cannot be taken for granted, and that clients may be able to limit what is made available or manipulate it.
So a full-population result is conditional twice over. It holds if the extract is complete and unaltered, and only for the attributes the test examined. The working paper should show both conditions: how the extract was reconciled to the ledger and trial balance, and what exactly each routine looked for.
What journal flags in Achekam tell you, and what they do not
In Achekam, staff load a whole-population journal extract as a CSV and map its columns. Achekam then flags entries such as manual, weekend, after-hours and post-close postings for staff to select and test. The CSV is not kept; its hash and row count are, so the file tested can be identified later. A "Halo-Style" badge on the screen is a label, not a description of a method.
Read the flags for what they are. A flagged entry has a characteristic worth a closer look. An unflagged entry has not been shown to be correct; it simply lacked the features the rules look for. The testing happens after the flags, when staff pick entries and examine support. Our guide to journal entry testing walks through that step.
Elsewhere, in Audit Intelligence, Achekam risk-scores a loaded population and proposes a sample; "all items" is one of the available methods alongside random, systematic, monetary unit and specific items. A reviewer who did not prepare the plan approves it before it is drawn, every draw can be reproduced from its stored seed, and no sample size is proposed until materiality has been approved and a confidence level set. Choosing "all items" removes the sampling question. It leaves the data and evidence questions exactly where they were.
Wording findings so they are not over-read
Clients take their cue from your language. A few habits help:
- Name the attribute, not the population. "We examined all journals for postings outside working hours" rather than "we tested all journals".
- State the condition. "Based on the journal listing provided by management, reconciled to the trial balance at year end."
- Separate flags from findings. "The analysis identified entries for follow-up; of these, we examined the support for each and found..." makes clear the analytic did not reach the conclusion on its own.
- Avoid absolute words. "No exceptions in the attributes tested" is accurate. "The data is clean" is not something you can know.
The same care belongs in internal documentation. A reviewer reading "100 percent tested, no issues" six months later cannot tell what was tested for.
Putting analytics alongside other evidence
Full-population analytics is strongest as one strand among several:
- Agree the data to the ledger before analysing it.
- Use analytics to direct effort: flags and risk scores show where to look.
- Inspect support for the items selected, from documents outside the system where possible.
- Confirm externally where the assertion calls for it: bank balances, customer balances, loans.
- Stand back and ask whether the results, together, support the conclusion on the account.
ACCA sets out the advantages and the cautions side by side in its article on data analytics in audit, which is a good briefing for a team before its first analytics-led engagement.
When a client next says "you checked everything", take the moment to explain what was checked and what was not. It costs a few minutes at the meeting and saves a much harder conversation later.
General guidance only, not advice on your own circumstances. Tax and payroll rules change: check the current position with the authority or your adviser.
