Executive summary
An organisation may have good reasons to pause an AI tool before it knows exactly what has gone wrong. For boards and leaders, the practical question is whether staff can act on a credible concern and keep people supported while it is examined. This paper explores how to make that possible through clear authority, a workable alternative and evidence that justifies restarting. A pause should give the organisation room to make a better decision. Its success depends on what happens to the people who still need the service.
Imagine a fictional regional training provider using AI to summarise course enquiries. The tool has helped a small team respond more quickly. Then a staff member notices that some summaries leave out requests for accessibility support. They check the original messages and find the requests are there.
Applications are still arriving. The supplier has been contacted, but there is no explanation yet. The manager who approved the tool is away. Do they keep using it, check everything themselves or stop the summaries?
For the person waiting to enrol, that uncertainty has a practical consequence. Their request may reach the right colleague, disappear into a queue or be missed altogether. The organisation's ability to pause safely matters before the cause is understood.
Paper 17 considered whether lessons from an AI incident change everyday practice. This paper turns to the decision that may come first: when confidence falls, can the organisation limit or stop the use that is causing concern without abandoning the people relying on it?
Give people authority they can use
A policy may say that humans remain in control. The staff member at the desk needs to know what that means when continuing feels unsafe.
In our example, they could have authority to stop using summaries for support enquiries identified by checking the original messages, route those messages to a person and alert the service manager. A wider or more serious concern might require suspending all summaries. The scope of the response should follow the possible harm and what is known, rather than the convenience of a single switch.
Australia's Guidance for AI Adoption describes human oversight as an organisational responsibility that includes meaningful intervention. [1] For a regional organisation, that means deciding in advance who can act, what they can change and whom they should tell. These arrangements need to work outside the hours when the most experienced person is available.
The board should expect management to make those responsibilities clear. It need not approve every operational pause. Requiring a meeting before staff can protect someone may leave the organisation unable to respond at the moment it matters. The board does need to know which concerns require prompt escalation and who can authorise continued use when a serious risk remains unresolved.
Decide what would change your confidence
Every imperfect summary does not call for the same response. A misplaced comma and an omitted support request have different consequences.
For the training provider, the concern is that information needed to support a learner is disappearing. Even if only a few cases have been found, the team may reasonably limit the tool while it checks how widely the problem extends. Waiting for a large failure count could leave more people exposed.
NIST's voluntary AI Risk Management Framework calls for organisations to determine whether a system should proceed, by assessing whether it achieves its intended purposes and stated objectives. It also calls for mechanisms to disengage or deactivate systems whose performance is inconsistent with their intended use. [2] These are useful prompts for a decision, rather than a universal numerical threshold.
Management should be able to explain what has changed, who could be affected and why the proposed response is proportionate. An unexplained change in outputs, an important check that staff can no longer perform or a supplier update that alters behaviour may justify looking again. Uncertainty belongs in that explanation. A pause can be warranted while the organisation gathers evidence, without claiming that a suspected cause has been proven.
Keep the service available
Turning off summaries does not turn off the need for help. The provider still has enquiries to answer, and a manual process may be slower.
A workable alternative begins with the original messages. Staff need access to them, time to read them and a way to direct urgent requests to the right person. If the existing team cannot handle the volume, a manager needs to decide what can wait and how people will be told. Otherwise, a technical pause can quietly become a service failure.
This is where a short rehearsal can help. Ask a colleague who did not design the process to handle an enquiry without the tool. Can they find the information and complete the task? What happens if the queue grows? The exercise may reveal a missing permission or a dependency on one person's knowledge while there is still time to address it.
A smaller organisation may use a supplier's product with no control over its underlying model. It can still examine whether it can disable the affected feature, stop relying on its outputs or use another process. If none of those options is workable, that limitation should be understood before the tool becomes essential.
Stopping use also does not settle what happened earlier. The provider should preserve relevant records securely and consider whether previous enquiries need review. People whose needs were missed may require a response even after the immediate risk has been contained.
Make restarting a fresh decision
A supplier's assurance that the problem is fixed is useful information. The organisation still needs to establish whether the tool works for its own task.
For the provider, that could involve checking the revised tool against examples containing support requests, including less obvious wording, and confirming that staff can spot uncertainty. Testing should use information the organisation is authorised to use and protect personal details. A limited return, with closer checking, may be appropriate if the evidence supports it. In some circumstances, the better decision will be to keep the feature off.
NIST also recommends post-deployment arrangements that include user input, override, incident response, recovery and change management. [2] Restarting therefore belongs within continuing oversight. Someone should own the decision, explain the remaining uncertainty and identify what would prompt another pause.
For the next board discussion, choose one AI-supported service and ask management to walk through a credible concern. Who could act that day, how would people continue to receive help and what would justify resuming normal use? Listen for an explanation that still works when the usual manager is away.
Our fictional staff member should not have to choose between ignoring a learner's needs and improvising an entire replacement service. Clear authority and a tested alternative give them a practical way to respond. They also give the board a firmer basis for confidence in the organisation's use of AI.
About BRAIN
The Ballarat Region Artificial Intelligence Network (BRAIN) is a regional institution focused on understanding the impact of artificial intelligence across Ballarat and surrounding communities and strengthening the capability required to respond. Through research, publications and regional collaboration, BRAIN examines how AI can contribute to regional prosperity and wellbeing while helping institutions navigate its risks and wider consequences.
About the Author
Matt Bowd is Co-Founder and Chief Executive Officer of the Ballarat Region Artificial Intelligence Network (BRAIN). His work focuses on AI governance, institutional capability and the implications of artificial intelligence for regional organisations and communities.
Sources
[1] National AI Centre, Guidance for AI adoption foundations. Practice 6 on human control. Voluntary Australian guidance; applicability of legal obligations depends on the organisation and use.
[2] NIST, AI Risk Management Framework 1.0, 2023. MANAGE 1.1, 2.4 and 4.1. Voluntary framework. The fictional example, rehearsal and board questions are BRAIN analysis, not prescribed universal requirements.
Matt Bowd