AI is reshaping finance, but accountability remains human

Share this:
Image generated by Deeptech Times using ChatGPT

The Association of Chartered Certified Accountants (ACCA) centred its recent annual 2026 conference on the theme Leading in a Polycrisis. Given the current global situation, with geopolitical upheavals due to wars and disruption caused by AI, the reference to leadership in a polycrisis was apt. 

Polycrisis refers to concurrent shocks that combine to create an effect much worse than the sum of individual parts. The panel discussion on AI regulation and industry practice highlighted a critical issue: many financial practices and organisations deploy AI faster than they learn to govern it.

With accountability and risk management being the foundations of the financial sector, this has been a major concern. 

Going back to first principles, DW Consulting’s principal consultant Jazreel Luar, who moderated the session, noted that a common mistake was assuming that AI meant chatbots, LLMs, agentic AI agents and GenAI.

These tools represented just one branch of a broader ecosystem that included machine learning (ML), predictive analytics, fraud detection and recommendation engines, which businesses have been using for years, she noted.

While evaluating the business case, Luar asked the audience to keep in mind that AI was always probabilistic and not deterministic. “AI tells us what is most likely, not what is certain… and that is where the governance challenge begins,” she noted.

AI output is not production ready

The risk lay in treating AI-generated output as if it’s production ready. Responsible meant understanding where an AI model could go wrong, validating its outputs and putting controls around them so that human judgment had the final word.

In this context, Luar shared the recent incident in which the email addresses of nearly 100,000 Bee Cheng Hiang customers were accidentally disclosed in what has been described as Singapore’s first GenAI-related data breach.

The incident occurred in April when an employee used an AI tool to generate a Python script for distributing a marketing email. The resulting code exposed the recipients’ email addresses to everyone who received the email.

After an investigation, the Personal Data Protection Commission (PDPC) said the incident was not due to AI malfunction; it was caused by human error in developing the email distribution code with an AI tool.

Responding to an audience poll that showed that most in the room were still exploring or experimenting with AI, Amazon Web Services’ head of finance for ASEAN and Taiwan Chan Sijia, who was part of the panel, noted that AI’s near-term value came from both cheaper versions of existing processes and from teams already redesigning processes around technology. As a result, productivity was the immediate gain. 

She noted that in financial accounting reconciliations, agentic AI could do the first pass of matching and flag anomalies for analysts to investigate. 

IMAGE: ACCA

Redesigning workflows using AI

“But the true value [of agentic AI] was in redesigning workflows,” Chan added. Rather than asking an agent to automate step two, three or nine of a 10-step reporting process, “you give it the problem statement: the report you need and what it must explain”, she said. 

Chan added that the agent may get there in two or three steps. Her point was that people are so used to their current way of working that they cannot imagine the shorter route until an agent shows it to them.

Responding to a question about how the financial sector, which depended on exact accuracy, should manage AI risk, Chan said reconciliations still had to be balanced and “AI just gets you there sooner”. She argued that stakeholders must share responsibility across the AI value chain and that organisations must decide how much autonomy to give agents.

Talking about AWS finance, she noted that it sorts work into three tiers. One was agent-driven work comprising low-risk reports that only gave leaders visibility and needed no judgment. 

The next tier was agent-assisted work where the agent does the work and a human checks the result, and the final tier was purely human coaching, mentoring and business review without the involvement of AI.

Treat AI agents as your new intern

Chan had interesting advice for the use of AI agents.

Treat an agent like a new intern or junior analyst. “You start small and give it more to do as it proves itself. If it flags an anomaly, you question it instead of posting the entry – you trust, but you verify. 

“Agents get faster and more accurate as you train them and correct their mistakes and hallucinations; that is the way to make progress to use them,” she added.

Another panellist, WTFox AI’s chief operating officer and co-founder Geargina Tan, shared that she was someone who was not technology trained but could use tools. Her company sells AI-powered CRM to small businesses. 

Tan noted that she built her first software programme in 30 minutes just by describing it in plain English to an agentic AI tool. This “blew her mind” because, as an operator, she never had the technical resources to do something like this. AI decentralises power and gives it to people who have no coding or IT skills but have the business expertise, she said.

Tan noted that things have been moving at breakneck speed in AI: last year the conversation was about chat, and now it is about agents doing work for you. She added that within a year this will change how finance professionals work. 

Responding to a question from Luar about at what point the system acts on its own and where the human checkpoint is, Tan echoed AWS’ Chan’s point that agents should act alone only when a decision is a two-way door, meaning a mistake could be reversed without dire consequences. 

“If a mistake cannot be reversed, a human must check the work first,” she said, adding that in her company’s CRM product, this works out to checking data, invoices and figures before anything reaches a customer, because “the minute it goes out wrong, it erodes trust”.

What’s in it for entry-level workers?

When asked whether automating entry-level work with AI would create an experience gap for juniors, Tan said the role of junior team members would change. 

“They will do less repetitive work and instead mark the homework of AI agents, and this would help them to build their expertise in fields such as law and accounting.” 

As the policy expert in the discussion, Infocomm Media Development Authority of Singapore’s (IMDA) cluster director for AI governance and safety Lee Wan Sie, was asked by Luar about who should be held responsible for harm caused by an AI programme. 

Lee referred to a recent discussion paper by IMDA that convened Singapore’s legal community, private practice lawyers, in-house counsel, lawyers from AI labs such as Google, and government representatives. 

They were asked to respond to the question of who would be accountable when an agent causes harm, and who can be pursued for compensation. According to Lee, they concluded that stakeholders across the entire ecosystem should share accountability.

The IMDA official noted that current legal frameworks could deal with AI harms, but “suing someone means going to court and gathering evidence, which takes a very long time and only fixes things after the damage is done”. The better approach was to set out each party’s role in advance, across the value chain. 

Shared responsibility framework hard to frame but required

She admitted that these shared responsibility frameworks were hard to define. Even so, Lee argued that there was a need to start working on them now. She added that organisations deploying AI agents were responsible for deciding how they frame risk, which actions they let agents take and how much autonomy they grant.

All the speakers agreed that transparency had to be proportional to how autonomous an agent was. 

The key takeaway was that focus should be on building independent testing capabilities now, with certification being the next goal. The experts advised companies to start small with low-risk tasks, with humans in the loop and, in parallel, setting up a system that identified who would be accountable before something went wrong. 

Leave a Reply

Your email address will not be published. Required fields are marked *

Search this website