AIDecision-Making:WhoIsResponsible?

When AI systems make decisions, accountability lies with the organisations using them - not the AI itself or its vendors. Businesses must ensure oversight at every stage, from data sourcing to deployment, and senior leaders can face legal consequences if things go wrong. Key points to consider:

• Responsibility Types: Moral (individual care), legal (liability), and organisational (task distribution).
• Stakeholders: Developers build the tech, businesses oversee its use, and end users have the right to understand its decisions.
• Challenges: Assigning responsibility is complex due to the "many hands" problem and the opaque nature of some AI systems.
• Legal Risks: In the UK, under laws like GDPR and SM&CR, organisations - and sometimes individuals - are accountable for AI errors or harm.
• Oversight: Human involvement is critical, especially for high-stakes decisions. Clear escalation paths, monitoring, and governance frameworks are essential.

To avoid legal and operational risks, organisations need robust accountability measures, including audit trails, defined roles, and continuous system monitoring. AI should enhance efficiency but always remain traceable to human decisions.

What Responsibility Means in AI Systems

What does responsibility mean in AI?

Responsibility in AI can be broken down into three key categories.

Moral responsibility is about an individual's internal sense of duty - understanding their role and exercising care in their actions. Legal responsibility, on the other hand, deals with liability - determining who can be held accountable in legal settings when things go wrong. Finally, organisational responsibility is about how tasks and duties are distributed within a company. For example, a Chief Information Security Officer (CISO) might oversee technical risks, while a Data Protection Officer (DPO) ensures compliance with data protection laws.

A crucial distinction exists between responsibility and accountability. While responsibility for tasks such as model development, data sourcing, or output monitoring can be assigned to teams, accountability cannot be passed down. As John Airey succinctly states:

"Delegating operational responsibility is appropriate. Delegating accountability is not."

In practical terms, this means that while the work involved in managing AI systems can be shared, ultimate accountability lies with senior leadership. This division highlights how organisations must carefully manage the balance of responsibility and accountability.

Who are the key stakeholders in AI responsibility?

Responsibility for AI systems is shared among three main groups, each playing a distinct role.

• Developers and vendors: They handle the AI's infrastructure and security. Under a shared responsibility model, they are responsible for the "engine" of the system but not how it is applied.
• Business leaders and deployers: These individuals are in charge of the data fed into the system, the context of its use, and the decisions made based on its outputs. In the UK, legal accountability for these aspects rests firmly with this group.
• End users and affected individuals: They have the right to understand decisions that impact them. Under GDPR Articles 13, 14, and 15, organisations must provide "meaningful information about the logic involved" in automated decision-making.

Why is assigning responsibility in AI so difficult?

The complexity of assigning responsibility in AI stems from a few key challenges. One of the biggest is the "many hands" problem. AI systems often involve contributions from numerous developers, data providers, integrators, and business users. With so many players involved, identifying who is at fault when something goes wrong can be exceptionally tricky, leading to what is often referred to as an accountability gap.

Another major challenge is the "black box" problem. Many AI systems, particularly those using large language models or deep learning, produce outputs that are difficult to explain. Their internal workings are often opaque, which complicates efforts to determine how decisions are made. To address this, the industry is shifting its focus from full transparency to Explainable AI (XAI). Rather than attempting to audit every line of code, XAI aims to provide context-specific explanations that are tailored to the audience.

The legal risks tied to these challenges are significant. Rohit Parmar-Mistry from Pattrn Data highlights this point:

"The 'black box' defence, claiming you did not know how the AI reached its conclusion, is not a shield... it is effectively an admission of negligence."

The case of Moffatt v. Air Canada in February 2024 serves as a clear example. In this case, a Canadian tribunal held Air Canada liable for incorrect information provided by its AI chatbot, rejecting the airline's argument that the chatbot was an independent entity. The ruling reinforced a critical principle: if you deploy an AI tool, you are responsible for its outputs.

sbb-itb-1051aa0

How Responsibility Is Shared Across the AI Lifecycle

Who is responsible for data quality and fairness?

The organisation deploying an AI system, legally referred to as the Data Controller, carries the main responsibility for ensuring that data is handled lawfully and fairly. It's up to the AI team to source, clean, and validate datasets, ensuring they are both up-to-date and representative. However, ultimate accountability lies with senior leadership. The CEO is formally recognised as the accountable owner, with this responsibility documented in board minutes.

In industries like UK financial services, this accountability is not just a best practice - it's a legal obligation. Under the Senior Managers and Certification Regime (SM&CR), specific individuals can face personal liability if they fail to demonstrate that reasonable steps were taken to manage AI systems responsibly.

When AI errors occur, having clear accountability frameworks in place is essential.

Who is accountable when AI makes errors or shows bias?

If an AI system causes harm - whether through biased outputs, flawed recommendations, or misleading information - accountability doesn't rest with the technology itself. As Oleg Prosin, Managing Partner at WCR Legal, explains:

"The legal question is not 'What did the AI do?' Courts and regulators ask a different question: Who controlled the system? Who approved its deployment? Who benefited from its output?"

One practical way to assign accountability is by using the RACI matrix, a framework that maps specific responsibilities to designated roles. For example, the CISO handles technical risks, the DPO manages data-related risks, the General Counsel oversees legal and contractual issues, and the Compliance Lead ensures systems are audit-ready.

Role	Accountability Area
CEO	Ultimate accountable owner for AI risk
CISO	Model security, adversarial robustness, infrastructure integrity
DPO	Lawful basis, data minimisation, rights of data subjects
General Counsel	Vendor terms, indemnity, and intellectual property exposure
Compliance Lead	Audit readiness and mapping controls to standards

The consequences of failing to establish and manage accountability are steep. Under UK GDPR, organisations can face fines of up to £17.5 million or 4% of global annual turnover for serious violations. A well-maintained audit trail - including change logs, risk registers, and approval records - can be the difference between a manageable issue and a major legal problem.

While defining roles is critical, ongoing oversight is just as important.

Who monitors AI performance over time?

Accountability doesn't end once an AI system is deployed. Continuous oversight is essential because models can degrade over time, leading to unreliable or biased outputs. Monitoring must be an ongoing, structured process.

One effective strategy is to align monitoring with key decision points in the AI lifecycle, such as model approval, data sourcing, deployment, and incident response. For high-risk or newly introduced systems, monthly reviews of performance, drift, and bias indicators should be the baseline. For more established systems, quarterly board-level reporting is advised. Crucially, this monitoring requires active human oversight, not just superficial reviews.

"If you cannot understand, monitor or contain an agent's actions, it is not ready for deployment."

• National Cyber Security Centre (NCSC)

Human Oversight in AI Decision-Making

When discussing shared responsibility in the AI lifecycle, one thing becomes clear: human oversight is essential for maintaining accountability.

What is the difference between human-in-the-loop and fully automated systems?

The distinction between these two models is crucial. A human-in-the-loop (HITL) system involves a person reviewing and approving AI outputs before any action is taken. In contrast, a fully automated system allows the AI to operate independently, making decisions and adjusting its behaviour without human sign-off.

Neither approach is inherently better; the choice depends on the context and the stakes involved. For routine tasks with minimal consequences, such as sorting emails, full automation is often sufficient. However, decisions with moderate to high stakes - like handling insurance claims or detecting fraud - require closer human involvement. The spectrum of oversight ranges from fully automated to entirely human-controlled, with many hybrid models falling in between.

Regulators are increasingly focusing on the difference between formal oversight (having a process in place) and meaningful oversight (ensuring humans have the capacity to critically evaluate AI outputs). For example, under the UK's Data (Use and Access) Act 2025, individuals have the right to human intervention and to contest high-risk automated decisions. Simply having a human in the process is not enough if they cannot properly assess the system’s outputs.

This distinction highlights the need for organisations to address gaps in oversight.

How can organisations close responsibility gaps?

To close these gaps, organisations need to take several concrete steps:

• Define escalation paths: Every automated workflow should clearly outline when the system must defer to a human. For instance, this could happen when a fraud alert is triggered or when a financial threshold is exceeded. Without these predefined triggers, accountability becomes murky.
• Maintain delegation records: Each AI system or workflow should have a structured record detailing its scope, the accountable owner, confidence thresholds, and the individual with the authority to shut it down if necessary. These records provide a clear trail of intent, which regulators often examine during audits.
• Conduct Algorithmic Impact Assessments (AIAs): Similar to data protection impact assessments, AIAs require input and approval from IT, legal, and business teams before deploying a system. This ensures that responsibility is spread across departments rather than isolated in one area.

These measures help establish robust oversight frameworks, which are essential for addressing the broader impact of AI on human accountability.

How does AI use affect human accountability?

One significant behavioural risk of advanced AI systems is automation bias - the tendency to accept AI recommendations without question. This becomes more likely under time pressure or when the system seems highly accurate. Over time, this can lead individuals to defer their judgement, prioritising the AI’s output over their own critical thinking to avoid appearing "insufficiently data-driven".

The consequences are far-reaching. As Bess Obarotimi aptly stated in May 2026:

"Efficiency without accountability is not maturity. Speed without ownership is not transformation."

Another risk is the moral crumple zone, where blame for errors falls on the human operator, even if they had limited authority to influence the outcome. Addressing this requires structural changes. The people tasked with overseeing AI must have the authority, expertise, and time to challenge its decisions meaningfully.

Ultimately, accountability cannot be offloaded onto machines. It remains firmly with the people and organisations deploying these systems.

Building AI Workflows with Clear Accountability

How can workflows embed human accountability?

To ensure human accountability in AI workflows, it’s essential to embed oversight directly into the process. This involves establishing decision gates - specific checkpoints where a human must review, approve, or intervene before the system moves forward. These checkpoints often cover critical areas like model approval, data sourcing, deployment, monitoring, and incident response.

For AI systems that operate autonomously (often referred to as agentic AI), the level of human involvement should align with the associated risks. A tiered oversight approach works well:

Oversight Level	How It Works	When to Use It
Autonomous	The system acts independently, with actions logged for audit purposes.	Low-risk scenarios, such as decisions involving amounts under £1,000.
Approve	The system makes recommendations, but a human must approve before action is taken.	Medium-risk situations, like transactions over £5,000.
Manual	The system prepares the action, but a human executes the final step.	High-risk cases, such as transactions exceeding £50,000.

In addition to tiered oversight, organisations should designate someone with the authority to halt the system if needed. The National Cyber Security Centre (NCSC) underscores this point:

"If you cannot understand, monitor or contain an agent's actions, it is not ready for deployment."

By embedding these controls, organisations lay the groundwork for effective governance and ensure AI systems operate responsibly.

What governance mechanisms support responsible AI use?

Governance doesn’t have to be overly complicated, especially for smaller businesses. As LogiSam aptly puts it:

"AI governance for a small business is not a 200-page policy manual... It is a proportionate set of controls that match your size, your risk, and your regulatory environment."

A practical starting point for governance is an acceptable use policy that defines approved tools and prohibits certain data entries. Keep a simple record - such as a spreadsheet - detailing AI tools, their ownership, and review dates.

At a minimum, organisations should conduct quarterly reviews. These reviews should focus on:

• Identifying model drift (when AI performance deteriorates over time).
• Updating the risk register.
• Ensuring compliance with current regulations, such as the EU AI Act, which will be fully enforced from August 2026 and applies to UK businesses serving EU customers.

Additionally, from June 2026, UK organisations must comply with the Data (Use and Access) Act 2025, which requires a formal internal complaints procedure for individuals to challenge automated decisions.

The benefits of mature AI governance are clear. It can reduce incidents by 23% and accelerate market readiness by 31%. On the flip side, 42% of UK organisations investing in AI have abandoned those projects due to governance challenges and failed implementation efforts.

With these mechanisms in place, small and medium-sized enterprises (SMEs) can confidently work with development partners to create AI systems designed for accountability from the outset.

How can development partners help SMEs build accountable AI?

Many SMEs face hurdles not because they lack the desire to govern AI responsibly, but because they lack the technical expertise to integrate accountability into their systems. This is where specialist development partners come in - they can design workflows that have accountability built in from the start.

For example, Antler Digital collaborates with SMEs to create agentic workflows that include key accountability features like approval checkpoints, tiered autonomy controls, audit logging, and named override authority at every stage. By incorporating these elements during the development phase, compliance becomes a seamless part of the system rather than an afterthought. This approach is particularly valuable for SMEs in regulated industries such as FinTech or SaaS, where governance gaps can derail AI projects.

John Airey of QL Security captures this principle perfectly:

"Delegating operational responsibility is appropriate. Delegating accountability is not."

The aim isn’t to slow down AI innovation but to ensure that when issues arise, there’s always a clear process, a named individual, and a documented decision trail to rely on.

Conclusion: Placing Responsibility at the Centre of AI

AI is reshaping businesses in countless ways, but it’s crucial to remember that it doesn’t bear legal or moral responsibility. As Niresh Rajah, Chair of the UKAI Financial Services and AI Working Group, aptly states:

"AI is not accountable for anything. Fundamentally, the individual who signs off is accountable. That accountability can never be minimized by saying it's through AI."

Under English corporate law, accountability lies firmly with the organisation, not the algorithm - a principle reinforced by the Moffatt v Air Canada ruling.

The stakes for mismanagement are high. Breaching the EU AI Act could result in fines of up to €35 million or 7% of global turnover. Meanwhile, a striking 80% of organisations have already reported instances of their AI tools acting outside their intended scope. As Bess Obarotimi puts it:

"Efficiency without accountability is not maturity. Speed without ownership is not transformation."

This highlights the urgent need for accountability to be embedded throughout the AI lifecycle. Organisations must establish clear decision points with designated owners, enforce meaningful human oversight (not just token approvals), maintain secure and unalterable audit trails, and implement governance frameworks that align with evolving regulations like the EU AI Act.

For SMEs navigating this complex terrain, working with partners like Antler Digital ensures accountability is built into your AI systems from the outset. AI should be fast, capable, and - most importantly - always traceable to a human decision.