Executive summary

An organisation can finish every action from an AI incident and still leave the same weakness in its everyday work. This paper considers what boards and leaders should look for after the immediate repair: a clear account of what should now happen differently, evidence from ordinary work and a reason to return if the change stops helping. The aim is proportionate confidence, not a promise that nothing will go wrong again. A useful question runs throughout: what can we now see in practice that we could not see before?

Consider a fictional regional training provider. An AI tool has omitted an important detail from a learner’s application. The mistake has been corrected, the learner contacted and the staff reminded to check the original document. At the next board meeting, the action is marked complete.

A month later, the person who handled the incident is on leave. A colleague encounters a similar application. Does the new approach still work?

That is where institutional learning becomes visible. The answer depends on whether the lesson has reached the ordinary conditions of work, including the people who were not in the room when it was discussed.

Paper 16 considered putting things right for affected people and checking the repair. This paper follows the change into the weeks afterwards. It offers boards a way to question a reassuring completion report without asking them to manage every operational detail.

What should happen differently

“Staff have been reminded” tells us something happened. It leaves open what that reminder was meant to achieve.

For our fictional provider, the intended change might be that staff consult the original application whenever a summary leaves an important requirement unclear. They also need a way to hold the application while the uncertainty is resolved. Being more careful is difficult to observe; checking the source and recording why a case needs attention is something a manager can examine.

The board does not need to prescribe every step. It does need a clear connection between the failure, the proposed change and the evidence that would show the change is helping. If the original document remains hard to find, another reminder may do little. The organisation may need to change the screen, the workload or the task it asks the tool to perform.

This distinction reaches beyond AI. Australian National Audit Office guidance on implementing public-sector audit and parliamentary recommendations emphasises evidence that actions address the recommendation’s intent, rather than relying on reported completion. [1] For a regional board, the useful habit is to ask what the action was supposed to improve before accepting that it is finished.

Look at an ordinary day

A demonstration led by the person who designed the fix can be reassuring. A different colleague, working through a less familiar case, may reveal something the demonstration missed.

In our example, a manager could examine a small, authorised selection of recent applications and ask staff to explain how they handled uncertainty. Include cases where the summary looked convincing, as well as those already flagged for review. Protect personal information and give the board the findings it needs, rather than circulating learners’ files.

The selection should have a reason behind it. A handful of easy cases cannot establish that a rare but serious failure has disappeared. The consequences of being wrong should influence the breadth of checking and the expertise needed. Where there is too little evidence, the honest conclusion may be that the change is in place but its effectiveness remains uncertain.

NIST’s voluntary AI Risk Management Framework recommends regularly assessing control effectiveness, monitoring systems in production and bringing user feedback into evaluation. [2] It does not prescribe one sample size or review interval for every organisation. Those choices need to fit the use, its risks and the decisions the evidence must support.

Ask what the numbers leave out

Suppose the provider reports fewer mistakes. That sounds encouraging, but perhaps fewer applications were processed, staff stopped recording near misses or difficult cases were sent elsewhere.

A useful report explains what was checked, what changed and what remains outside view. It might show whether important details were preserved, whether uncertain cases reached a person and whether the extra checking delayed support. These are proposed measures for our fictional example, not a universal scorecard.

The people doing the work can explain what a count cannot. Is the new check easy to use when the queue is busy? Are colleagues quietly correcting summaries without recording the problem? Do learners know how to raise a concern?

The Commonwealth Ombudsman’s 2023 review of the Department of Veterans’ Affairs illustrates the limit. Its desktop review found all eight recommendations implemented, but could not give unqualified assurance that improved processes were followed in daily work. It called for periodic effectiveness reviews, informed by complaints. [3] This was an administrative review, not an AI incident. For our provider, complaints are one source of learning. Their absence alone cannot establish reliability.

Boards should make room for an inconvenient answer. If reporting a problem makes a team appear to have failed, the organisation may learn less precisely when it needs to learn more.

Return before the lesson fades

A repair can be effective today and become inadequate when the tool, documents or working arrangements change. The organisation needs to know who will notice and when the question comes back to leaders.

For the provider, a further review might be tied to the next intake or a supplier update. It should also be possible to reopen the issue sooner if the same omission appears or staff cannot carry out the agreed check. A calendar date is useful, but a meaningful reason to return is better than a routine reminder nobody owns.

In a small regional team, the person responsible for the change may also run the service. An independent department may be unrealistic. A colleague outside the immediate process, or a suitably qualified adviser for a higher-risk issue, can help challenge the evidence. The level of scrutiny should reflect what is at stake.

The board’s role is to decide whether the explanation and evidence justify confidence, what remains unresolved and when further assurance is needed. Management retains responsibility for making the process work. Once there is enough evidence to justify closure, an issue may move from incident reporting into normal monitoring, with a named owner and a clear route back if concerns reappear.

A practical test for the next meeting

Choose one improvement reported as complete after an AI-related problem. Ask the responsible person to show what now happens differently, using an ordinary example rather than only the original failure. Then ask what would make them reconsider their confidence and how that information would reach the board.

The answer may reveal a successful change. It may reveal that people need more time, clearer instructions or a different tool. Either result gives leaders something useful to act on.

For the learner whose application arrives while the experienced colleague is away, the value of the lesson is simple. The organisation still knows what to do. That is the kind of progress a board should be able to see.

About BRAIN

The Ballarat Region Artificial Intelligence Network (BRAIN) is a regional institution focused on understanding the impact of artificial intelligence across Ballarat and surrounding communities and strengthening the capability required to respond. Through research, publications and regional collaboration, BRAIN examines how AI can contribute to regional prosperity and wellbeing while helping institutions navigate its risks and wider consequences.

About the Author

Matt Bowd is Co-Founder and Chief Executive Officer of the Ballarat Region Artificial Intelligence Network (BRAIN). His work focuses on AI governance, institutional capability and the implications of artificial intelligence for regional organisations and communities.

Sources

[1] Australian National Audit Office, Implementation of Recommendations, 30 June 2021. Governance arrangements, implementation planning and providing assurance. Australian public-sector audit guidance.

[2] NIST, AI Risk Management Framework Core, AI RMF 1.0 (2023). MEASURE 1.2, 2.3–2.4, 3.3, 4.3 and MANAGE 4.1–4.3. Voluntary framework.

[3] Commonwealth Ombudsman, DVA Recommendation Implementation Report, November 2023, page 2 of 4, How did we assess implementation and Next Steps. Administrative review, not an AI incident investigation.

Sources checked 3 November 2026.