TrendAI’s global study of 3,700 business and IT decision-makers found that 67% had felt pressure to approve AI despite security concerns. One in seven described those concerns as extreme but said they had still been overridden to keep pace with competitors or internal demand. Only 38% of organisations surveyed had comprehensive AI policies in place, while 31% admitted they lacked sufficient observability or auditability over their AI systems.

Rachel Jin, Chief Platform and Business Officer and head of TrendAI, puts the gap down to something more basic than ignorance. Organisations, she argues, know there are risks but don’t necessarily have the conditions to manage them. TrendAI’s research found that 44% see AI agents accessing sensitive information as their biggest agentic AI risk, while concerns also extend to malicious prompts, expanding attack surfaces and autonomous code deployment. That has far-reaching implications as AI moves beyond answering questions and starts doing things.

An agent connected to company systems can be allowed to access information, call software tools or trigger actions without asking a person for approval at every step. The permissions differ enormously depending on how the system has been configured, which is precisely where TrendAI believes traditional security approaches begin to struggle.

TrendAI itself is a relatively new name. Trend Micro renamed its enterprise cybersecurity business TrendAI in March as it expanded its focus from securing infrastructure and applications towards governing how AI systems act and interact inside organisations. The change coincided with the launch of its Agentic Governance Gateway, a product designed to sit between autonomous agents and the systems with which they interact.

The marketing around any security product deserves scrutiny, particularly when its maker has an obvious commercial incentive to persuade companies that a new category of risk requires a new layer of software. There’s one phrase in TrendAI’s description of the Gateway gets to a much larger problem: organisations should be able to introduce human oversight at “critical decision points”. But who decides what makes a decision “critical” before the damage is already done? That question takes on far-reaching implications as AI shifts from merely answering questions to autonomously executing tasks.

Who decides what the AI is allowed to do alone?

Putting a human in the loop sounds straightforward until an organisation has to decide what the human is actually there to approve.

A customer-service agent reading an account balance may be considered low risk. Changing a customer’s contact details probably deserves more scrutiny. Moving money creates another level of exposure entirely. The same problem appears in cybersecurity, software development, HR systems and any environment where an agent can progress from retrieving information to altering something in the world.

TrendAI’s Agentic Governance Gateway reflects that distinction. It is designed to discover what agents can access, observe how they interact with other systems, enforce policies around their actions and stop behaviour that falls outside those rules. Human approval can then be introduced where an organisation decides the consequences justify it.

That model makes sense operationally because asking a person to approve everything would make autonomy considerably less useful. It also moves one of the most important decisions away from the moment an agent acts. Somebody has to decide which actions don’t require approval.

TrendAI’s research suggests organisations are making those calls while their broader governance remains incomplete. Around 40% of respondents supported an AI “kill switch” capable of shutting systems down after a failure or misuse, while nearly half remained uncertain about whether such a mechanism was necessary. TrendAI reads the disagreement as evidence that businesses are moving towards autonomous systems without having agreed on how they will retain control.

There is a useful tension in that finding. A kill switch deals with what happens once somebody realises a system should stop. Governance also has to deal with everything the agent was authorised to do before anybody reached for it.

Having a human there doesn’t guarantee much

The technology industry as a whole is arriving at a similar problem.

Cisco now distinguishes between human-in-the-loop and human-on-the-loop systems. The former asks a person to participate in individual decisions, while the latter allows an agent to operate independently under higher-level human supervision. AWS, meanwhile, explicitly warns that requiring human review for every agent action can produce reviewer fatigue and rubber-stamp approvals.

Cisco and AWS are useful examples, but they don’t resolve the underlying problem. An organisation can put a person into an approval workflow and still create a system where the machine effectively makes the decision.

That risk has already shown up in practice. Anthropic found that Claude Code users approved roughly 93% of permission requests when the coding agent asked before certain actions. The company found that repeated approval requests reduced people’s attention to them and responded by automating some lower-risk permissions while putting more emphasis on limiting what the agent could access in the first place.

The implication is uncomfortable. A person clicking Approve is evidence that a human participated in a workflow. It says considerably less about whether they exercised meaningful judgement.

Human-in-the-loop AI oversight was already beginning to show these limits before the current enterprise push towards more autonomous agents. As the number of automated decisions grows, meaningful supervision increasingly depends on designing the boundaries well rather than inserting a person into every transaction.

That places TrendAI’s “critical decision points” in a different light. The important control may be the policy determining when the agent has to stop, what information the reviewer sees when it does, and what happens when nobody responds.

South African law has already met the electronic agent

For South African organisations, this isn’t a theoretical debate waiting on future AI regulation; the legal foundation has been quietly sitting on the books for over two decades. Since 2002, the Electronic Communications and Transactions Act (ECTA) has explicitly recognised the concept of an “electronic agent,” defining it as an automated system capable of independently initiating actions or responding during a transaction without human intervention.

The technology lawmakers had in mind was considerably simpler than a generative agent capable of interpreting an objective and choosing how to accomplish it, but the Act still gives automated activity legal consequences. Under section 20, an agreement can be formed through an electronic agent and, subject to specified exceptions, a party using one can be presumed bound even if that person didn’t personally review the agent’s actions or the resulting terms. Section 25 also attributes certain automatically generated data messages to the organisation behind the system unless it can be shown that the information system failed to execute its programming properly.

Those provisions don’t settle liability for modern agentic AI. Attribution of a data message doesn’t automatically decide who is civilly, criminally or regulatorily liable for everything an agent might do, and a law drafted around automated transactions shouldn’t be stretched into an AI liability statute that Parliament never wrote.

It does make one assumption difficult to sustain: the absence of a human clicking the final button doesn’t automatically put the resulting action beyond the organisation that chose to automate it.

POPIA gives the human-oversight question a more contemporary form. Section 71 restricts certain decisions based solely on automated processing of personal information where the decision has legal consequences for a person or affects them substantially and the processing is intended to create a profile. The law specifically contemplates areas such as work performance, creditworthiness, reliability, health and conduct.

The section contains exceptions, so it doesn’t create a universal requirement for a human to approve every consequential automated decision. Where an applicable exception relies on measures protecting the person’s legitimate interests, however, POPIA can require an opportunity to make representations and enough information about the underlying logic for those representations to be meaningful.

There’s already a direct connection here with the unresolved questions around South Africa’s broader AI policy. South African companies are making decisions about AI deployment while the country’s dedicated policy framework remains unsettled, leaving existing legislation and sector rules to do much of the immediate work.

TrendAI’s governance argument therefore lands differently in South Africa. Companies don’t have the luxury of treating agent controls as something to design only once dedicated AI legislation appears. Depending on what the agent touches and what decision it makes, existing privacy, electronic-transactions and sector-specific rules may already apply.

Financial services makes the accountability question harder

TrendAI’s original concern becomes particularly relevant in banking, insurance and other regulated financial services because the consequences of automated access and action can escalate quickly.

The FSCA and Prudential Authority’s Joint Standard 2 of 2024 on cybersecurity and cyber resilience establishes governance requirements for covered financial institutions. Its focus is cybersecurity rather than AI specifically, so it shouldn’t be treated as a catch-all agentic AI law. But where autonomous systems interact with security controls, sensitive information or critical infrastructure, the governance obligations become relevant. The bigger issue is how responsibility is allocated before an autonomous system enters production.

An organisation can decide that an agent may read customer information but may not change it. It can permit routine account actions while escalating unusual cases. Security teams can restrict which tools an agent is allowed to invoke and what data it can see. TrendAI’s own approach is built around making those permissions visible and enforceable before and while agents operate.

That governance work is less dramatic than the idea of an AI suddenly going rogue, but it is probably where most real accountability questions will begin.

The important human may never see the agent act

TrendAI’s research describes organisations deploying AI despite concerns they already understand, often before the policies governing those systems are complete. Agentic AI adds another decision to that process because every useful degree of autonomy represents something a human has deliberately stopped checking case by case.

That doesn’t automatically make autonomy reckless. A system that requires a person to approve every routine action can become slow enough to defeat the purpose of deploying it, while overloaded reviewers may approve requests largely out of habit. The challenge is deciding where autonomy stops being an efficiency decision and becomes a risk the organisation is no longer prepared to delegate.

TrendAI’s idea of human oversight at critical decision points therefore raises a question that software alone can’t answer. Which decisions are critical, who gets to classify them that way, and how often should those boundaries be reconsidered as the agent gains new capabilities?

Those choices are made by people before an agent moves money, changes a permission or acts on someone’s information. The employee who eventually receives an approval request may be only one part of the accountability chain.

As companies hand more decisions to autonomous software, the human who matters most may increasingly be the one who decided the AI no longer needed to ask.

The post TrendAI says companies are deploying AI faster than they can govern it appeared first on Reframed.

TrendAI country MD Gareth Redelinghuys has published a position paper on the subject that goes further than the company’s recent infrastructure announcement in establishing what the regulatory basis for urgency actually is. The commercial interest is transparent, TrendAI sells locally governed data centre infrastructure, and the argument that organisations need South African soil under South African legal accountability is an argument for the product category they operate in. That doesn’t make the argument wrong. The facts it rests on hold up independently of who’s presenting them.

The enforcement gap that’s closing

POPIA has been fully enforceable since July 2021, but the Information Regulator spent the first years building institutional capacity. That phase appears to be over. The Regulator’s office is now processing an average of 198 breach notifications per month, a volume that signals functional enforcement infrastructure rather than a body still standing up its processes.

The more structurally significant development is the FSCA’s Joint Standard 2 of 2024. For financial institutions, this standard inserts board-level accountability into cybersecurity governance explicitly. The accountable party for cybersecurity failure isn’t the IT manager or the CISO. It’s the board. That’s a direct legal consequence, not a recommendation, and it applies now.

Together, these two frameworks have effectively closed a grace period that many South African organisations were still operating inside, a window in which POPIA existed on paper but enforcement remained light enough to treat as a future problem.

What physical residency doesn’t guarantee

The dimension that tends to get the least attention in local data sovereignty conversations is the US CLOUD Act. If a cloud provider is headquartered in the United States, US authorities can compel access to data held on that provider’s servers regardless of where those servers physically sit. A Johannesburg data centre operated by a US-headquartered company doesn’t make that data subject to South African jurisdiction alone. Physical residency and legal jurisdiction aren’t the same thing, and most local cloud migration decisions haven’t been made with that distinction clearly drawn.

For POPIA compliance purposes, this matters in ways most organisations haven’t formally assessed. The question of whether data is protected under South African law depends on which law can actually reach it.

What the paper doesn’t address

TrendAI’s position paper treats the regulatory “what” with precision. The “how” is where the gaps are. Cloud migration in South Africa has been deep and in many cases fast. For organisations already substantially dependent on hyperscalers headquartered outside the country, the path to meaningful data residency isn’t a procurement decision — it’s a multi-year infrastructure realignment that carries its own cost, complexity, and continuity risk. Naming the problem is easier than solving it for organisations mid-migration.

SABRIC’s figure of R2.2 billion in annual losses to cyberattacks on South African banks captures direct financial losses only. It excludes the reputational exposure of a publicly disclosed breach, the operational disruption of a ransomware event, and the regulatory consequence of a formal POPIA notification. The Postbank breach illustrated all three materialising simultaneously, and the lasting damage wasn’t the breach, it was the governance failure that made the breach possible.

What’s actually changed

South African organisations in financial services can no longer treat cybersecurity governance as an IT function with board oversight; the FSCA Joint Standard makes it a board function with IT execution. POPIA enforcement has moved past the point where delay functions as a strategy.

Local infrastructure is one credible answer to the jurisdiction question. Whether it’s the right answer for any specific organisation depends on what a proper data residency audit reveals, and most South African organisations haven’t conducted one with the specificity the Regulator would now require.

The post South Africa’s data sovereignty conversation now has regulatory teeth appeared first on Reframed.