Executive summary

An AI tool can keep working while the reasons for approving it quietly stop being true. Staff may use it for a new purpose, a supplier may change the service or experience may reveal a problem that testing missed. This paper helps leaders recognise those changes and decide what should happen next. A resilient organisation can explain why continued use remains acceptable and can change course, including stopping, when it no longer is.

Think of a tool that began as a helpful way to summarise internal documents. Over time, people discover other uses. It drafts replies to customers, prepares advice for the website and suggests how difficult enquiries should be handled. The product has the same name and the subscription is still being paid. Yet its role in the organisation has changed considerably.

This is a hypothetical example, not an account of a particular institution. It illustrates a question that is easy to overlook: when did a useful assistant begin doing work nobody had assessed?

The original approval may have been perfectly sensible. Summarising internal material for an experienced employee is different from influencing advice that a member of the public will rely on. Leaders need a way to notice that difference before an incident makes it obvious.

Remember what you said yes to

An approval is most useful when it explains the judgement behind it. What work was the tool allowed to do? Who would be affected? Which limitations mattered, and what safeguards made the proposed use acceptable?

Without that account, “we approved this product” can become permission for almost anything the product can now do. An inventory may identify the software, but it cannot explain whether today’s use still fits the decision.

In our example, the initial approval might have required an employee to check summaries against the original documents. That safeguard may be reasonable for internal reading. It does not automatically make the same assistant suitable for advice about eligibility, deadlines or services. The source material, checking process and consequences all deserve another look.

BRAIN uses “governance drift” to describe this widening gap between what was approved and what happens in practice. The phrase is useful only if it helps people recognise an ordinary change. The aim is to make that change discussable, not to give staff another unfamiliar term to learn.

Notice changes outside the software

Some developments are easy to identify: a new model, a supplier announcement or a significant configuration change. Others happen without a technical update. Staff may enter different information, a new group of people may depend on the service or time pressures may weaken the review that originally justified its use.

This is why an uptime report cannot settle whether an AI use remains acceptable. A service can be available, secure and within budget while answering the wrong question or being used beyond its limits. Keeping it running and keeping it well governed are connected responsibilities, but they are not interchangeable.

NIST’s voluntary AI Risk Management Framework treats governance as work that continues throughout a system’s life. [1] Australia’s Guidance for AI adoption also asks organisations to consider each use and how it might change or be reused over time. [2] These resources support continuing attention; they do not create a general legal requirement that applies identically to every regional organisation.

For a small team, the most useful signals may come from people close to the work. A customer’s question, an employee’s workaround or a recurring correction can reveal something a scheduled report misses. People need to know where those observations can go and that raising them will not automatically be treated as admitting failure.

Give a changed situation a fresh decision

Regular reviews create a useful opportunity to pause and consider experience. Some changes should not wait for the next meeting, though. A material supplier update, a serious complaint or expansion into a consequential new task may need a decision when it occurs.

The appropriate triggers depend on the use. An assistant drafting low-stakes internal notes does not warrant the same attention as a tool influencing employment or access to essential services. The important point is that someone recognises the trigger and has authority to respond.

Returning to our example, a manager might discover that the summarising tool now drafts public advice. The response need not be an immediate permanent ban. The organisation could assess that use, add an effective review, restrict the subjects it handles or return it to its original purpose. What matters is that continuation becomes a considered choice rather than an unnoticed extension.

Record the reason for that choice and any conditions attached to it. A successor should be able to see what changed and why the organisation was comfortable proceeding, or why it was not. Paper 12 examines this handover in more detail: the next owner needs the reasoning as well as the permission.

Make changing course possible

A leader may formally have authority to stop a tool but find that stopping would leave a service unable to function. That is a practical constraint worth discovering early.

Ask what would happen if the supplier withdrew support, an important assurance gap emerged or the organisation could no longer obtain the evidence it needed. Could staff use an alternative process? Would essential records remain accessible? Would someone understand what happens to the organisation’s data when the relationship ends?

A fallback should be workable, not simply named in a plan. If the manual process depends on knowledge that staff no longer practise, it may need rehearsal. If critical records sit only in the supplier’s portal, access on exit needs attention before the contract ends.

The DTA’s technical standard offers a government-focused reference for lifecycle practices such as testing, changes and retirement. [3] Its scope and the distinction between required criteria and explanatory guidance matter. It should not be presented as a rule binding every Ballarat business, school or community organisation. The broader lesson here is practical: the ability to reconsider a decision is stronger when the organisation can actually act on it.

Carry the learning forward

Paper 9 explored what happens after an error or near miss. Resilience means that the resulting lesson survives the immediate response. A corrected source document may solve today’s problem; a clear owner and review process may help prevent the same weakness returning next year.

BRAIN’s Governance Pathway connects Responsibility, Stewardship, Accountability, Transparency, Assurance and Resilience. Resilience keeps the other capabilities alive as circumstances change. It does not imply that an organisation reaches a point where its judgement is finished.

Regional institutions may be able to help one another by sharing protected lessons about supplier changes, useful tests and warning signs. That is a potential contribution to shared capability, not evidence that Ballarat already has a measured map of common dependencies. Nor does it transfer decisions to BRAIN. Confidentiality and the possibility of recognising people from local details need consideration before any case-based lesson is shared.

For your next review, choose one significant AI use and put its original approval beside a description of what staff actually do today. Ask what has changed and whether the reasons for continuing still hold. If the two accounts no longer match, you have found a useful decision to make.

Resilience begins there: being willing and able to decide again, rather than expecting yesterday’s answer to do tomorrow’s work.

About BRAIN

The Ballarat Region Artificial Intelligence Network (BRAIN) is a regional institution focused on understanding the impact of artificial intelligence across Ballarat and surrounding communities and strengthening the capability required to respond. Through research, publications and regional collaboration, BRAIN examines how AI can contribute to regional prosperity and wellbeing while helping institutions navigate its risks and wider consequences.

About the Author

Matt Bowd is Co-Founder and Chief Executive Officer of the Ballarat Region Artificial Intelligence Network (BRAIN). His work focuses on AI governance, institutional capability and the implications of artificial intelligence for regional organisations and communities.

Sources

[1] NIST, AI Risk Management Framework 1.0, Core. 2023, GOVERN and MANAGE 4; voluntary framework.

[2] National AI Centre, Guidance for AI adoption: implementation guidance. Continuing monitoring, changed uses and human oversight; voluntary guidance.

[3] Digital Transformation Agency, Technical standard for government’s use of artificial intelligence. Government lifecycle reference. Read introduction and scope for the status of criteria and explanatory notes.

Sources checked 21 October 2026.