Governance Insights Series - Paper #12: Governing AI as Organisational Memory
Executive Summary
An institution can approve an AI system responsibly and still lose the knowledge required to govern it. The people who tested it leave. The vendor changes its model. A complaint exposes a limitation understood during the pilot but never recorded in a form the next manager can find.
AI governance therefore needs organisational memory: a dependable way to preserve what was decided, why, on whose evidence, under which conditions, and what subsequent experience revealed.
The aim is selective, trustworthy memory, not indefinite storage of every prompt and output. Australia’s National Archives advises government agencies to manage AI records according to their value and purpose, and to make risk-based decisions about retaining related prompts and inputs. The Australian Government’s AI technical standard likewise links auditability to documentation of decisions and changes across the lifecycle. [1][2]
If an organisation cannot reconstruct its reasons for using AI, it cannot reliably defend, revise or reverse that use. Memory is an operating capability of governance.
The Decision Outlives Its Makers
Paper 10 examined how governance responds to change. Paper 11 considered responsibility across organisational boundaries. Both depend on a further condition: someone must be able to recover the reasoning behind earlier decisions.
An AI system may begin as a small trial, pass through several leaders, acquire new features and become embedded in ordinary work. A human review step required during the pilot may later be removed because the reason for it was never recorded.
This is a failure of organisational memory. It is especially likely when governance is scattered across procurement files, risk registers, meeting minutes, vendor portals and the recollections of a few employees.
NIST’s AI Risk Management Framework treats documentation as support for transparency, human review and accountability, and calls for information about system limits and human oversight to be available to those making subsequent decisions. [3]
Preserve the Reason, Not Merely the Approval
An approval record saying “accepted” tells a successor surprisingly little. A useful decision record explains the intended purpose and affected people; the evidence considered; uncertainties and dissent; controls required; the person authorised to decide; and the circumstances that would invalidate the approval.
Consider an AI service used to help a regional employer screen applications. At the pilot stage, reviewers may find that its suggestions are less reliable for applicants with unconventional employment histories. A later team inherits only an approval and an accuracy figure. Without the limitation and its associated condition, it may treat human review as optional.
The relevant memory is not simply that testing occurred. It is the connection between the finding, the decision and the control. Good records preserve that chain without pretending the original evidence was conclusive.
The Australian Government’s AI technical standard calls for traceability from requirements to operational impacts, including records of modifications, authority and rationale. [2]
Memory Must Be Revised Without Being Rewritten
Institutional memory cannot be a static archive. It needs a current account of what is happening, linked to a historical account of how the organisation arrived there.
Suppose the screening vendor changes its model, evaluation method or data handling. The organisation should be able to identify the last approved configuration, the change, the evidence reviewed, the decision taken and any new condition imposed. It should not overwrite the original assessment as though the current arrangement had always existed.
This distinction matters when an incident is investigated or a decision is challenged. A record that shows only today’s understanding cannot reveal what was reasonably known when an earlier decision was made. Conversely, a preserved initial approval cannot justify current use after material changes.
The record therefore needs two connected views: a living description of the present use and a durable sequence of consequential decisions and observations. Each material change should answer whether the original basis still holds. Paper 10 set out the need for reassessment triggers; organisational memory makes those triggers intelligible and their outcomes traceable.
A successor must be able to understand why the change mattered, what evidence was missing and who had authority to permit continued use.
What Deserves to Be Remembered?
AI can generate more information than an organisation can sensibly retain. Capturing every prompt, draft, model response and telemetry event would create a large, poorly navigable archive while increasing privacy, security and storage risks.
The National Archives of Australia draws a useful distinction for Commonwealth agencies. AI-generated records are managed according to their business purpose and value. The final human-created business record will often matter more than transitory drafts. Related prompts and inputs may need retaining where they provide evidence for a significant output or decision; the decision is risk-based, rather than a universal instruction to keep all chat histories. The Archives also warns about over-retention. [1]
For any institution, the practical question is what a future reviewer would need to understand a consequential use, test an explanation, respond to a complaint or make a better decision. That may include an approval rationale, evaluation summary, supplier assurance evidence, a material prompt or instruction, a sample of outputs, an incident finding, or a record of human intervention.
Retention and access rules should be set with records, privacy and security specialists. A decision to keep less must itself be deliberate, particularly when a supplier holds the only usable logs. Memory is curated evidence, not accumulation.
Complaints and Near Misses Carry Knowledge
Formal assessments capture what an institution expected. Frontline staff and affected people often reveal what actually happened. A person challenging an AI-assisted outcome may identify an overlooked language pattern, an inaccessible process or a use that has moved beyond its approved purpose.
A functioning memory system records more than the fact that a complaint was closed. It connects the concern to the relevant AI use, tests whether it reveals a wider problem, records any change to the control and makes the lesson available to future owners.
Paper 9 addressed learning when things go wrong. The additional challenge here is transmission. A lesson confined to an incident report, or known only to a departing employee, does not reliably alter subsequent practice. The organisation should be able to show where the lesson changed a test, instruction, procurement question, approval condition or training practice.
NIST includes feedback from users and impacted communities in its approach to tracking AI risk over time. That feedback has governance value only if people can raise it safely and it reaches someone able to act. [3]
Memory Across Supplier and Staff Transitions
Paper 11 described AI governance across organisational boundaries. Memory must cross those boundaries too. A customer may hold its approval rationale while a vendor holds model documentation, change histories and operating logs. If the contract ends or a supplier withdraws access, the customer may discover that essential evidence was never within its control.
For consequential uses, procurement should establish which records each party keeps, how changes and incidents are communicated, what evidence the institution may retrieve, and what remains available on exit. These are questions of governance continuity as much as information technology.
Inside the organisation, transfer should be deliberate. A new system owner needs a compact account of purpose, approved conditions, unresolved concerns, vendor dependencies and the next decision due. Access to a folder is insufficient when its contents require the original project team to interpret them.
Staff may carry informal knowledge of workarounds, user behaviour and weak signals that formal monitoring missed. A structured handover should ask what worries the outgoing owner, what assumptions they would retest, and which affected groups have not yet been heard. Record the substance; do not make enduring governance depend on a person’s continued availability.
A Practical Memory for Each Significant Use
Institutions do not need a vast new repository before they can begin. For each significant AI use, one maintained record can point to five kinds of evidence: the approved purpose and boundaries; the decision and its rationale; the evidence and uncertainties; material changes, incidents and feedback; and the current owner, review date and conditions for intervention.
It should use language a new leader can understand, identify the source and status of each important claim, and retain prior decisions when updated. More consequential uses warrant stronger traceability and access controls. Ordinary low-impact assistance may need much less.
The test is practical. Give a successor who did not attend the pilot the record and ask: Can you explain why this use was allowed? Can you identify what has changed? Can you find the evidence behind the present position? Do you know what might require suspension? If the answers depend on calling a former employee, the institution has a memory gap.
What This Means for Ballarat
Regional institutions can face rapid staff turnover in specialised roles and rely on suppliers whose systems are developed elsewhere. A school, health provider, council, employer or community organisation may have only a small number of people who understand a particular AI use.
There is also a regional opportunity. BRAIN can help institutions develop shared questions, practical record formats and de-identified accounts of lessons learned. It can identify recurring gaps and translate changes in national guidance into questions for local decision-makers. Each organisation must retain control of its own records and responsibility for its own uses; confidential material should remain protected.
Over time, a region can remember more than any one institution. Recurring complaints, supplier changes and governance failures can inform stronger practice when lessons are shared lawfully and with care. Regional intelligence should preserve patterns and context without creating an unnecessary store of personal or commercially sensitive information.
Questions for Leaders
Leaders should ask whether a new owner could identify the evidence and conditions behind each significant AI approval. Where would they find the history of material changes and responses? Which complaints or near misses have changed practice? What information is held only by a supplier or a departing employee? Which records must be retained, who may access them, and when should they be disposed of?
Sound policies today do not guarantee explainable decisions tomorrow.
Conclusion
AI governance accumulates decisions. Some permit use; others restrict it, change a control or acknowledge an unresolved concern. If those decisions lose their reasons, later leaders inherit the appearance of authority without its foundation.
Organisational memory gives governance continuity without freezing it. It preserves enough evidence to explain the past, recognise when assumptions no longer hold and make new decisions in light of experience. The measure is not the volume of records retained. It is whether the institution can still exercise judgement when the people, systems and circumstances have changed.
About BRAIN
The Ballarat Region Artificial Intelligence Network (BRAIN) is a regional institution focused on understanding the impact of artificial intelligence across Ballarat and surrounding communities and strengthening the capability required to respond. Through research, publications and regional collaboration, BRAIN examines how AI can contribute to regional prosperity and wellbeing while helping institutions navigate its risks and wider consequences.
About the Author
Matt Bowd is Co-Founder and Chief Executive Officer of the Ballarat Region Artificial Intelligence Network (BRAIN). His work focuses on AI governance, institutional capability and the implications of artificial intelligence for regional organisations and communities.
Sources
Next in the Series
The Governance of AI-Generated Decisions
The next paper will examine how institutions distinguish an AI-generated recommendation from a human decision, preserve meaningful human judgement and make consequential outcomes understandable and contestable.
Written by Matt Bowd, Co-Founder and Chief Executive Officer of the Ballarat Region Artificial Intelligence Network (BRAIN).
Each study is a step toward a more intelligent and resilient region.
To participate in regional pilots or research partnerships, in our region or yours, connect via matt@brain.net.au.