Executive Summary

Artificial intelligence is rarely contained within a single organisation.

An organisation may select the system and determine how it will be used, while the model is developed elsewhere, the service is operated by a vendor, infrastructure is provided by another company, data comes from multiple sources and specialist partners support implementation. The people affected by the system may sit outside every organisation involved in creating it.

This creates a governance problem. Responsibility for operating an AI system can be distributed across many parties, but accountability for its use cannot simply disappear between them.

Australian Government guidance increasingly recognises this reality. The current Guidance for AI Adoption asks organisations to identify and document shared responsibilities across the AI supply chain, including responsibility for monitoring, human oversight, incidents and system updates. Its predecessor, the Voluntary AI Safety Standard, made the principle explicit: leaders cannot outsource accountability for the safe and responsible deployment and use of AI. (Australian Government Department of Industry, Science and Resources)

For organisations, the implication is significant. Vendor due diligence and contractual protections matter, but they are not enough. Effective governance requires institutions to understand the system of relationships upon which an AI use depends, establish clear decision rights, secure sufficient information to exercise oversight and preserve the ability to intervene when circumstances change.

The organisational boundary is becoming less useful as the boundary of AI governance.

The System Extends Beyond the Organisation

Traditional governance structures tend to begin with the organisation itself.

Boards oversee executives. Executives allocate responsibility. Policies define acceptable conduct. Risk processes identify threats. Procurement processes govern suppliers. Contracts establish obligations between organisations.

AI complicates this structure because a deployed system may represent the combined activity of many organisations. A regional employer using an AI-enabled recruitment platform, for example, may have no role in developing the underlying model. The software provider may itself rely upon another model developer and cloud provider. Training data may have originated elsewhere. External consultants may configure the system, while employees determine how recommendations are interpreted.

Yet applicants experience the consequences as a decision of the employer. This distinction matters. The organisation does not need to control every component of an AI system to remain responsible for deciding whether that system is appropriate for its purposes.

NIST's AI Risk Management Framework reflects this broader view of governance. Its Govern function includes third-party software, hardware and data, while its risk-management practices specifically address risks created by third-party entities and supply chains. (NIST AI Risk Management Framework)

Governance therefore has to follow the system, not stop at the organisational chart.

Distributed Responsibility Is Not Diluted Accountability

Different organisations can legitimately hold different responsibilities.

A model developer may be responsible for particular testing. A software vendor may monitor technical performance. A cloud provider may manage infrastructure security. A deploying organisation may determine acceptable use, train employees and oversee decisions involving customers or workers. Trying to make every participant responsible for everything would create confusion rather than accountability.

The governance challenge is to make the distribution explicit.

Australia's Guidance for AI Adoption recommends identifying accountability across model developers, system developers and deployers for monitoring and evaluation, human intervention, incident handling and system updates. (Guidance for AI Adoption: Implementation Practices)

But shared responsibility does not mean an organisation can transfer responsibility for its own judgement. If an organisation chooses to use an external AI system in recruitment, customer service, education, healthcare, financial decisions or another consequential activity, the existence of a supplier does not answer whether that use is appropriate.

The deploying institution still needs to understand what decision it is making, what evidence supports that decision, what it needs from its suppliers and what would cause it to intervene.

A useful principle follows:

Operational responsibilities may be distributed. Institutional accountability must remain visible.

Contracts Cannot Govern What Organisations Do Not Understand

Paper 5 of this series examined AI procurement and vendor relationships. Organisational boundaries expose a deeper problem that continues after procurement is complete.

Contracts can allocate obligations, establish reporting requirements, create audit rights and define responses to incidents. They cannot compensate indefinitely for an organisation that lacks the capability to understand what information it needs.

This becomes particularly important when suppliers possess substantially more information about a system than their customers.

A vendor may understand model architecture, testing results, system limitations, updates and dependencies that are difficult for the deploying organisation to observe independently. Some information may legitimately be protected for security, privacy or commercial reasons. The governance objective is therefore not complete technical visibility.

It is sufficient visibility to govern the intended use.

The former Voluntary AI Safety Standard described information that developers should provide downstream where possible, including system capabilities and limitations, relevant testing, known risks and mitigations, data-management practices and security arrangements. It also expected deployers to communicate intended use, incidents and unwanted outcomes upstream. (Voluntary AI Safety Standard: Guardrails)

That reciprocal information flow is important. An organisation cannot effectively govern a system when important knowledge stops at the supplier boundary. Equally, a supplier cannot understand emerging problems if customers conceal how systems are actually being used.

Transparency across organisational boundaries is therefore not simply disclosure. It is governance infrastructure.

Govern the Dependency, Not Just the Vendor

The language of vendor management can make an AI relationship appear simpler than it is.

An organisation may contract with one supplier while depending indirectly upon many others.

A software product may incorporate an external foundation model, rely on cloud infrastructure, use third-party datasets, connect with organisational systems and receive frequent updates outside the customer's direct control. Open-source components can add further dependencies without a conventional supplier relationship at all.

The governance unit should therefore be the dependency, not merely the contract.

For significant AI uses, organisations should understand which external parties or components could materially affect safety, performance, privacy, availability, transparency or continued operation. This does not require mapping every technical library. Proportionality remains essential.

A low-consequence productivity tool may require relatively modest oversight. A system affecting employment, health, education, finance, safety or access to essential services warrants considerably greater understanding of its dependencies.

NIST similarly recommends that organisations manage third-party AI according to their resources, risk profile and use case, while maintaining processes for third-party failures and incidents. (NIST AI RMF Playbook: Govern)

The practical question is not, “Do we know every supplier?”

It is, “Which dependencies could materially change our ability to govern this use?”

Assurance Must Cross the Same Boundaries as Risk

Paper 8 moved the BRAIN Governance Pathway from policy toward assurance. Paper 10 extended that argument into continuous change. Organisational boundaries bring those ideas together. An organisation cannot assure an AI system solely by examining what happens internally when material controls operate elsewhere.

If a supplier performs testing, the deploying organisation may need evidence that appropriate testing occurred. If the supplier monitors model performance, there needs to be clarity about what is monitored, what thresholds matter and when the customer will be informed. If an incident originates upstream, downstream organisations need a mechanism for learning about it quickly enough to respond.

The Australian Government's national AI assurance framework describes assurance as part of broader AI governance and establishes common practices intended to support safe and responsible government use. (National Framework for the Assurance of Artificial Intelligence in Government)

The principle has wider institutional relevance: assurance requires evidence.

A contractual promise that a supplier will manage a risk is not the same as evidence that the risk is being managed. For higher-consequence systems, organisations should therefore identify the evidence they need from external parties before deployment and throughout the lifecycle.

That turns supplier oversight from periodic relationship management into part of the organisation's assurance architecture.

Build Governance Interfaces Between Organisations

Most organisations do not need elaborate new cross-organisational governance structures. They need reliable interfaces. For each significant external AI dependency, four things should be clear: who is responsible for what; what information must move between parties; which events require notification or escalation; and who has authority to act.

These interfaces should connect to existing governance rather than sit beside it.

An incident reported by a vendor should reach the organisation's accountable owner. A material model update should trigger reassessment where appropriate. Monitoring evidence should feed assurance processes. Complaints from affected people should be capable of travelling upstream when they indicate a system problem. Changes in supplier arrangements should be reflected in the organisation's AI inventory and risk assessment. 

Current Australian guidance similarly treats supply-chain accountability as a lifecycle issue rather than a procurement event. It recommends documenting responsibility for performance monitoring, human oversight, incidents, issue resolution and updates across developers and deployers. (Guidance for AI Adoption: Implementation Practices)

The objective is not bureaucracy. It is to prevent important governance questions from falling into the spaces between organisations.

What This Means for Ballarat

Regional institutions have particular reasons to understand organisational dependency.

A council, health provider, university, school, financial institution, professional firm or community organisation in Ballarat may increasingly use sophisticated AI while possessing little ability to build or inspect the underlying technology itself. That is not inherently a weakness. External platforms allow smaller organisations to access capabilities that would otherwise be unavailable.

The vulnerability arises when dependence grows faster than governance capability.

Multiple regional institutions may also rely upon the same platforms, model providers, consultants and infrastructure. A weakness in one external system can therefore create correlated exposure across organisations that appear independent. There is an opportunity here for shared regional capability.

Institutions can exchange de-identified knowledge about supplier questions, material system changes, incidents, assurance approaches and recurring dependencies without transferring accountability for their decisions. BRAIN's role should be to help that knowledge accumulate.

It can identify recurring governance patterns, translate national and international developments into regional implications and help organisations understand the questions they should ask across organisational boundaries. 

Regional capability becomes stronger when each institution remains accountable while learning from the experience of others.

Questions for Leaders

  • Which external organisations and components are material to our most consequential AI uses?
  • Where does responsibility change hands across those relationships?
  • Who inside our organisation remains accountable for each use?
  • What information do we require from suppliers to exercise meaningful oversight?
  • Which changes, failures or incidents must external parties tell us about?
  • What evidence allows us to assure controls performed outside our organisation?
  • Can affected people raise concerns that reach the parties capable of addressing them?
  • What happens if a critical supplier changes its system, withdraws support or no longer provides sufficient information?
  • Could we suspend or replace the external capability if continued use became unacceptable?

If these questions cannot be answered, the organisation may have outsourced technology while inadvertently weakening its capacity to govern it.

Conclusion

AI challenges the assumption that organisational responsibility and technological control occupy the same boundary.

Increasingly, they do not.

Models, infrastructure, data, expertise and operational responsibilities can be distributed across networks of organisations. That distribution can make AI more accessible and capable. It can also create gaps through which responsibility, information and risk disappear.

Good governance does not require organisations to control the entire AI supply chain.

It requires them to understand where they depend upon others, establish clear responsibilities, obtain enough information to exercise judgement, assure material controls and preserve the capacity to intervene.

The BRAIN Governance Pathway remains applicable across these boundaries:

Responsibility → Stewardship → Accountability → Transparency → Assurance → Resilience

Responsibility establishes ownership. 

Stewardship considers the system in context. 

Accountability prevents obligations from disappearing between parties. 

Transparency enables information to cross boundaries. 

Assurance provides evidence that distributed controls are working. 

Resilience allows institutions to respond when dependencies change.

AI may be distributed. Governance must remain connected.

About BRAIN

The Ballarat Region Artificial Intelligence Network (BRAIN) is a regional institution focused on understanding the impact of artificial intelligence across Ballarat and surrounding communities and strengthening the capability required to respond.

Through research, publications and regional collaboration, BRAIN examines how AI can contribute to regional prosperity and wellbeing while helping institutions navigate its risks and wider consequences.

About the Author

Matt Bowd is Co-Founder and Chief Executive Officer of the Ballarat Region Artificial Intelligence Network (BRAIN). His work focuses on AI governance, institutional capability and the implications of artificial intelligence for regional organisations and communities.

Sources

Next in the Series

Governing AI as Organisational Memory

AI governance creates an expanding record of decisions: why systems were approved, what risks were accepted, which controls were required, what changed and what institutions learned.

The next paper will examine how organisations preserve this knowledge through leadership changes, staff turnover, vendor transitions and technological change—and why institutional memory is becoming an essential part of long-term AI governance.

Written by Matt Bowd, Co-Founder and Chief Executive Officer of the Ballarat Region Artificial Intelligence Network (BRAIN).

Each study is a step toward a more intelligent and resilient region.

To participate in regional pilots or research partnerships, in our region or yours, connect via matt@brain.net.au.

The link has been copied!