How to vet a candidate: an AI answer with two errors

To vet a candidate in South Africa for whether they will simply believe the chatbot, hand them a one-page AI summary of your own anonymised data with two errors planted in it. They get 20 minutes to mark what is wrong and say how they checked. Two people score it on one rubric, and the result goes on the candidate file. It takes an afternoon to build and costs nothing per candidate beyond the time.

Why vetting a candidate now includes the AI answer

SME South Africa opens its piece on AI literacy in IT hiring with a scenario. A developer at a growing South African logistics company uses an AI tool to summarise customer data. The summary is "polished and confident, but wrong". Nobody questions it, and a pricing decision follows.

Neil Lund, managing director at Paracon by Adcorp, separates the two skills involved. Proficiency is the ability to build. The article's own summary of his point is that literacy is "knowing whether it can be trusted". Employers already rate this highly. In CompTIA's Workforce and Learning Trends 2025 research, 95% of South African businesses said digital fluency was becoming moderately or much more important, against 85% in the US and 83% in the UK.

Lund also names the SME's real problem: "defining what these capabilities mean within a particular role and then finding credible ways to assess them". An interview question about AI tests how someone talks about it. This test watches them use it.

Before you start

• One real report the role will read in its first month: a customer order summary, a job list, a stock count or a debtors' age analysis.
• The export the report was built from, as a spreadsheet.
• Two people who will score, one of whom will manage the new hire.

Step 1: pick the page the job will actually read

Choose a summary that a decision hangs on. For a dispatcher, the week's deliveries by route. For a developer, a summary of customer records like the one in the logistics scenario. You should end up with one source table of 30 to 50 rows and a clear question it answers.

Step 2: anonymise the data before anything else

Replace every customer name, contact person, phone number, email address, ID number and street address with invented ones. Keep the figures and dates, because those are what the candidate will check. When you are done, nothing on the sheet should let anyone identify a real customer or employee.

Step 3: have the AI write the summary, then check it yourself

Paste the anonymised table into the company's AI account and ask for a one-page summary with a recommendation. Then check every figure against the table by hand. You should now hold a summary you know to be correct.

Step 4: plant two errors that would change a decision

Make one a figure error: a customer's total that does not match the rows, or a count that is ten too high. Make the other a reasoning error: a trend stated backwards, or a recommendation the figures do not support. Neither should be a typo. Leave the confident tone alone, because the confidence is the test.

Step 5: try it on a colleague

Give the page to someone outside the hiring. If they find both errors in two minutes, the errors are too obvious. If nobody can find them in 20 minutes with the table open, they are too hidden. You want an error a careful person finds by going back to the source.

Step 6: write the rubric before any candidate sits it

Four lines, each scored 0 to 3. Two are simple: found the figure error, found the reasoning error. Line three is showed how they checked, by naming the rows or the sum. Line four is said what the errors would have done to the decision. Write one sentence per line describing a 3.

Step 7: run it the same way for everyone

Each candidate gets the summary, the source table and the same written instruction: "Mark anything wrong and write how you checked it. You have 20 minutes." Same files, same time, same room or the same email. You should see marked pages, not verbal answers.

Step 8: score separately, then file it

Each scorer fills in the rubric alone, then you compare. Where the two differ by more than a point, agree the line and note why. The candidate file keeps the version of the test, their marked page, both scoresheets and the agreed score, all dated.

What breaks here

Real customer data, not anonymised. Customer names and account figures are personal information under POPIA, and that includes business customers. Handing them to a job applicant is a disclosure you cannot justify. Anonymise first, every time, and keep the original out of the test folder.

Errors so obvious anyone finds them. A misspelt name or a total of R1 billion on a R40 000 account tests eyesight, not judgement. Every candidate scores full marks and you learn nothing.

The test reused until it leaks. Candidates talk, and a page that has been sat twenty times is known. Number each version and plant new errors every quarter, or after every hiring round, whichever comes first.

Scoring only the catch. A candidate who spots both errors by instinct and cannot say how is not the one Lund describes. The "how they checked" line matters as much as the errors found.

When this stops being enough

A folder and a Sheet cope with a few hires a year. Across several sites the test versions drift, marked pages sit in personal inboxes, and nobody knows which version a returning candidate sat last time.

At that point the test belongs on the candidate record, beside the work sample and the checks. That is how Zorah builds recruitment and onboarding: one record per candidate, carried into the employee record on day one.

On Monday, pull last month's customer order summary and its export. Anonymise the export and plant two errors in the summary. Then give it to a colleague and time them.