What changes when AI can act? 5 agentic AI risks leaders need to understand

For the last few years, organisations adopting AI have largely focused on the risks associated with what generative AI chatbots produce. This can include inaccurate information, biased outputs, inappropriate content and the possibility that employees might share sensitive information with AI tools. What happens when agentic AI enters the picture? Those concerns remain, but with the addition of new risks. Rather than just answering a question, an agent can go and do something. It can check a system, send an email, update a record, often several steps in a row, without a human approving each one. 

For leaders across businesses, charities and public sector organisations, this changes the risk question considerably. Instead of asking only "what has the AI agent told us?", organisations increasingly need to ask a different question. 

"What are we allowing the AI agent to do, what could go wrong and what would happen if it did?"

The term "AI agent" is used now in a wide range of systems without necessarily involving autonomy and action, so it’s worth checking what a specific tool does before assuming these risks apply. The UK's National Cyber Security Centre has warned that agentic systems introduce risks because of their autonomy, their access to data and tools. They can also act at a speed and scale that can make meaningful human oversight difficult.

This does not mean that organisations should avoid agents. Nor does it mean that every agent presents a significant risk. What matters is understanding the characteristics that determine how significant that risk is.

These 5 questions provide a useful starting point.

1. Autonomy: how much does the agent decide for itself?

The first source of risk is the degree of independence an organisation gives an agent. Does a person start the process? How much human involvement stays in place as the agent completes its work?

An employee asking an AI agent to summarise a document isn’t giving the agent much independence. In this example the person starts the task and receives an output to review. The situation changes when an agent monitors an inbox, identifies which messages require attention, determines what needs to happen next and continues working without waiting for a person to prompt each step.

As autonomy increases, organisations have fewer opportunities to identify mistakes before they progress through a process. An agent might misunderstand an instruction, fail to complete part of a task or repeatedly apply the same incorrect reasoning across many cases before somebody notices. These risks become more significant when agents operate continuously or initiate work themselves without proper human oversight. 

The UK's National Cyber Security Centre recommends that organisations consider explicitly how much autonomy agents should have. They should also agree where meaningful human oversight should remain, particularly as systems become more autonomous.

For leaders, the important question is how much judgement and control the organisation is comfortable delegating before a person needs to become involved.

2. Data access: what happens when an agent can see more of the organisation?

As organisations connect agents to email, shared drives, knowledge bases, customer systems and other sources of information, the potential benefits increase. As do privacy, security and information management risks.

An agent working only with information supplied directly by an employee has a relatively narrow information environment. An agent connected to a live CRM, shared mailbox or case management system may be able to access considerably more information. This might  include personal data, commercially sensitive information or records that different employees have different permissions to see.

The UK's Information Commissioner's Office has highlighted this issue in its work on agentic AI. They have warned that agents may process large amounts of personal information. Organisations should apply data minimisation, rather than giving agents access to information simply because it could prove useful.

Agents that consume information originating outside the organisation introduce an additional security concern. Emails, documents, websites and form submissions can contain instructions designed to manipulate an AI system. This is known as indirect prompt injection. It is a high security threat in which malicious instructions are introduced through data that an AI system processes, rather than directly through the user's prompt.

The risk is therefore not simply whether an organisation holds sensitive data. How much of that data an agent can access, where it comes from and what the agent is allowed to do with it.

3. System access: what can the agent change, send and who can it act as?

Giving an agent access to information creates one set of risks. Giving it permission to change organisational systems or communicate with other people can create significantly greater consequences when something goes wrong.

Compare an agent that reads several databases and produces an internal report, with one that can update those databases itself. The first might produce an incorrect conclusion that a person can challenge. The second could introduce incorrect information directly into an authoritative organisational record.

As agents receive wider permissions, the potential consequences can extend beyond inaccurate records. Depending on the system involved, an agent could send an inappropriate external communication, change someone's access permissions, alter a customer or service-user record, place an order or initiate a financial process.

It is also important to understand whose identity the agent is acting under when it takes these actions. If an agent operates using an employee's account or permissions, its actions may appear to have been taken by that individual. This creates additional risks around accountability, authorisation and auditability.

Organisations often focus heavily on the capabilities of the AI model while paying less attention to the permissions surrounding it. In practice, the damage an agent can cause depends on what the surrounding systems actually permit it to do.

Leaders in organisations therefore need to understand what authority has been given to the agent, whose identity and permissions it is acting under and whether actions are difficult to reverse, affect external parties or create financial, contractual or operational consequences.

4. Criticality: how serious would the consequences of a failure be?

An agent could make the same kind of mistake in two different places, and it could matter far more in one than the other. This is because of the decisions it’s helping with. 

An agent that incorrectly interprets information used in recruitment, benefit eligibility, financial controls, safeguarding or access to an essential service could affect an individual directly. It may also create legal, regulatory or reputational consequences for the organisation.

This matters most wherever the outcome impacts a person. Whether it’s public bodies and charities interacting with people who depend on their services, or businesses using agents in employment, credit, insurance or other financial decisions. 

UK data protection law also places particular requirements around certain forms of automated decision-making. The ICO's guidance on automated decision-making and profiling addresses decisions based solely on automated processing that produce legal or similarly significant effects, including requirements concerning safeguards and individuals' rights. The regulatory framework in this area is developing following the Data Act 2025. Organisations should ensure that they are working from current guidance and keep up to date as it changes.

The important leadership question is consequently not just "Could the agent make a mistake?" because every AI system can. It is "If this particular agent makes a mistake, what could the consequences be for the organisation and for the people affected by its actions?"

5. Access and ownership: what happens when agents spread across the organisation?

The accessibility of modern AI tools means that agentic systems do not necessarily arrive through a centrally managed technology programme. Employees and individual teams can increasingly configure assistants, automations and agents using tools that the organisation already provides.

This creates the possibility of agent sprawl. This is where different teams create similar agents, employees share them more widely than originally intended or automated processes continue operating after their original purpose or owner has changed.

Permissions can make this problem more significant if an agent is logged in as its creator. Those distinctions matter because they determine what the agent can access and whose identity appears to be responsible for the resulting actions.

An agent initially created by one person to solve a small operational problem can therefore become a different risk if it is subsequently shared across a department or organisation without reconsidering its permissions, purpose and oversight.

For leaders, this creates potential risks around accountability, security, consistency and organisational visibility. An organisation cannot effectively manage agents if it does not know which agents are operating, who owns them or what authority they have been given.

The greatest risk often comes from the combination

Looking at these 5 dimensions individually helps organisations understand where agentic AI risk comes from. However, the most significant exposure often appears when several of them combine.

An agent that monitors an external inbox, reads customer or service-user records, decides what action to take, updates an organisational system and sends an external response without review presents a fundamentally different risk profile.

This is why leaders should be cautious about trying to classify agentic AI as either inherently safe or inherently dangerous. The more useful question is whether the organisation understands the authority it has delegated to a particular agent and the potential consequences if that agent behaves incorrectly.

What should leaders do with this information?

Understanding these risks does not require organisations to create a complex approval process for every AI assistant., Many agents will support low-risk activities where existing controls and straightforward oversight are sufficient.

Organisations do, however, need enough visibility to spot the exceptions.  That means agents with greater autonomy, broader access to data or permission to change important systems. It also means agents involved in decisions that could  affect people or the organisation.

As agentic AI becomes easier to create and deploy, the leadership challenge will not simply be deciding whether an organisation should use AI agents. It will be understanding where the organisation has delegated authority to AI, what risks that delegation creates and whether the controls surrounding it are proportionate to the potential consequences.

If you want help working out where your own organisation sits against these 5 questions, get in touch, we're happy to talk it through.

In our next blog, we will move from understanding these risks to managing them in practice, with a starter governance tool that organisations can use to identify and register AI agents.

How we used AI in writing this blog

This article was written by Krizia Delgado and developed from AIConfident’s own framework on governing AI agents.

We used Claude to challenge the structure and purpose of the piece, simplify how the 5 areas of risk were explained and help distinguish the risks. We also used it to research and verify external sources supporting specific points, which we reviewed before including them.

Images in this piece are from Better Images of AI. We use them instead of generic AI stock photography because most AI imagery reinforces public mistrust of AI, leans on harmful stereotypes and spreads misleading cultural tropes about the technology.

Image credit Jamillah Knowles & Digit, Better Images, Creative Commons Licence