AI Monitor:
Enabling trust in an
AI-enhanced world
The accountancy and finance profession has always played a central role in establishing trust within capital markets. With the emergence of artificial intelligence (AI), new dynamics are being introduced to the traditional trust mechanisms that underpin the accountancy profession.
Why AI poses challenges to trust
AI has the potential to enhance processes and improve efficiency, but it also poses challenges around data use, reliability, accountability and more – which could impact the longstanding foundations of trust.
These concerns depend on the initial application of the technology, but are ultimately determined by the data and its derivatives. While there are some general risks surrounding many uses of AI, precise manifestation and mitigation techniques vary. For many uses, existing practices may provide sufficient oversight once new ways of working have been considered.
This means that there are two interlinked elements that need to be considered:
To accommodate these factors, accountancy and finance professionals should view trust in AI as a socio-technical challenge (see figure 1). This requires both sound governance with internal control frameworks and, where relevant, technical practices – such as machine learning operations (MLOps) – to support them in critical uses.
Trust is ultimately rooted in how people work together but we build mechanisms to help us sustain trust in complex and uncertain environments.
By understanding how these different areas operate together, accountancy and finance professionals will be better positioned for a more responsible and trustworthy adoption of AI at scale.
Figure 1: Balancing social and technical factors
What is trust?
Trust acts as a heuristic that simplifies complex decision-making by allowing us to rely on the expertise and ethics of professionals without needing to verify the details ourselves. Consequently, trust is a risk mitigator and an ethical shortcut that facilitates cooperation and empowerment.
Just as we trust doctors to take care of our health without understanding the intricacies of medicine – businesses, regulators and capital markets trust accountants and auditors to ensure the integrity of financial statements without scrutinising every figure themselves. This trust reduces transaction costs and enhances social capital by creating more seamless business interactions.
AI's impact on trust
‘All models are wrong, but some are useful’
When it comes to traditional accounting practices, AI can raise a variety of trust-related issues, for example:
- Activities related to business analysis or forecasting - If AI models are making consequential forecasts and recommendations that influence decision-making – the lack of clear explanations for their rationale could reduce trust and accountability.
- Auditing and assurance practices - If AI systems are flagging issues and shaping audit focus, there may be concerns about the over-reliance on AI dependent procedures and reduced application of human professional scepticism and judgment. The lack of transparency in AI decision-making could also hinder auditors in substantiating and defending AI-assisted assessments to clients or regulators.
- Areas such as fraud detection, risk assessment, and compliance monitoring - AI systems can be biased or make errors, potentially flagging false positives or missing relevant issues. Without careful governance and human supervision, over-reliance on AI could erode trust. There are also risks that bad actors could learn to exploit AI blind spots and circumvent detection.
- AI-powered virtual assistants are also beginning to be used for internal support in accountancy and finance contexts. These AI agents can manage routine inquiries, provide information and guidance, and even perform some basic tasks. While they can increase efficiency and availability of support, if AI chatbots give inaccurate or inappropriate responses it could reflect poorly on an organisation's competence and reliability
Understanding the
socio-technical challenge
Several common themes emerge. Whereas, the first two examples may relate to more technical means of mitigation, the latter are more socially orientated:
Prioritise outcomes
over outputs
Ultimately, the role of accountancy and finance professionals in the AI era is to focus on the outcomes driven by technology – instead of concentrating solely on the outputs. Outputs are viewed as raw materials that can be assessed on technical grounds. However, the true value lies in understanding how these outputs inform decisions and actions that drive business outcomes.
Technical solutions – like strict explainability requirements – are only one part of the equation in managing the challenges associated with AI in accountancy and finance. When addressing these challenges, it’s essential to adopt a realistic and pragmatic approach. The requirements for explainability, for instance, may vary significantly depending on the context in which AI is being used.
In situations where AI is employed for compliance, reporting, or the distribution of goods and services that could potentially breach established regulations or standards – explainability and interpretability should be given the highest priority. However, in other contexts, the need for explainability and even critical accuracy may be less pressing, allowing acceptable trade-offs to be made.
In some cases, the speed and impact of AI on decision-making under conditions of uncertainty may be more important than achieving the highest levels of accuracy. There may be diminishing returns or acceptable compromises when striving for increased accuracy. The focus should be on leveraging AI to make timely and effective decisions in real-world scenarios.
Using machine learning in financial planning and analysis (FP&A)
ACCA member James Best (Technology Global Forum member, Precision Finance Consulting) has explored the adoption of ML within the FP&A function based on operations across a range of organisations. While there has been extensive discussion around the need for explainability when using ML algorithms, he has found that, in practice, this requirement is less significant than other factors.
As with traditional ways of modelling, trust is fostered through collaborative efforts, transparent decision-making, and the ability of FP&A and business stakeholders to influence model inputs and adjust outputs based on additional context, including unforeseen events.
In practice, the ‘black-box’ approach presents a more efficient way to account for a far wider range of variables, leaving greater capacity to assess and augment outputs – and drive outcomes.
He has focused on a process of collaborative cross-functional creation for ML-generated budgeted costs, to find:
Figure 2: Complexities and concerns vary by use case
Indicative considerations:
How to develop
trust mechanisms
for using AI
As AI becomes more prevalent in accounting and finance, there are foundational steps that organisations can take to support employees and stakeholders:
Consider technical reinforcement
For many users, technical considerations will come down to effective vendor management. For those developing and deploying internal solutions, MLOps practices can embed governance standards into actual implementation and running of AI systems.
Essentially, MLOps can provide a technical backbone in critical uses where trust may give way to requirements for additional verification. For accountancy and finance, this can include:
- Maintaining comprehensive data and model version histories to enable traceability and reproducibility of AI model results – allowing financial forecasts to be traced back to the exact AI model version and training data that generated it.
- Implementing gated approval workflows to ensure that AI models used for key accountancy and finance functions have been adequately tested and validated – and that their deployment is formally approved by accounting leaders.
- Setting up monitoring dashboards to track AI model performance and fairness/bias metrics in production – proactively alerting accountancy and finance risk management teams to potential issues.
- Establishing rigorous access controls and authentication to ensure that only authorised individuals can approve, modify, or interact with AI models involved in financial reporting and other sensitive accounting domains.
MLOps and governance at Three UK
MLOps can improve transparency and trust in AI models in production, as explained by ACCA member Alec Manning, head of data science, Three UK.
‘We establish controls and processes that mean you can't interfere with what you're putting into production directly. You have to take it back through the whole pipeline. You're collecting loads of nuggets of information about the model, be that quality, performance metrics, model drift. You're storing that all so you can monitor it. At any point if an auditor came and said: “Oh, that model that you produced underpinning the finance gross margin forecast, has that model ever drifted?” You can instantly pull that stat out,’ he said.
Regular cross-functional ethical reviews of AI projects embed trust throughout development and involve legal, risk, and business stakeholders throughout the process.
Manning continues: ‘We maintain a register – this is all quite automated – of all the types of data that we're working with, the type of features that we're working with, the use of those features. And we review that every week, 30 minutes with the data protection officer (DPO).’
Laying the foundation for
trusted AI use
CFOs and senior accountancy and finance professionals may not need to understand all the technical details of MLOps – but they should be aware of the governance and trust implications as well as key metrics for review when required.
As AI starts to play a greater role in financial reporting, planning and other key accounting functions, CFOs and controllers will need to sign off on the adequacy of controls and oversight for those AI systems.
AI governance and risk management are already becoming more frequent boardroom and C-suite topics, so senior leaders should start taking some basic steps:
AI Monitor series
Upcoming issues of the AI Monitor will explore related issues pertaining to talent, risk and controls, data strategies, and sustainability.
The aim of the series is to identify and examine, from the accountancy profession’s perspective, some pressing AI challenges, setting out routes and next steps for finance professionals.
Acknowledgements
Members of The ACCA Global Forum for Technology
Dev Ramnarine (Chair)
Alastair Barlow
Alex Falcon Huerta
Brad Monterio
Heather Smith
James Best
Krishna Chaitanya Maddula
Rashika Fernando
Reshma Mahase
Robert van der Klauw
Bilal Surahyo
Scott McHone
Alec Manning
Andrew Chong
Sources
Lukyanenko, R., Maass, W. & Storey, V.C. ‘Trust in artificial intelligence: From a Foundational Trust Framework to emerging research opportunities.’ Electron Markets 32 (2022) https://doi.org/10.1007/s12525-022-00605-4
D. Kreuzberger, Kühl, N. & Hirschl, S. ‘Machine Learning Operations (MLOps): Overview, Definition, and Architecture.’ IEEE Access 11 (2023)
ACCA The Adelphi 1/11 John Adam Street London WC2N 6AU
United Kingdom / +44 (0)20 7059 5000 / www.accaglobal.com
