Three key takeaways
- AI adoption can move faster than governance. ASIC found that some financial services organisations were expanding their AI use before their risk controls were ready.
- AI creates risks beyond traditional technology issues. These include inaccurate outputs, bias, limited explainability and inappropriate reliance on third-party systems.
- Financial advice providers remain accountable. Using AI does not remove obligations to give suitable advice, protect client information and treat clients fairly.
Artificial intelligence is increasingly being used to analyse documents, draft communications, support research and improve client service.
ASIC’s report, Beware the gap: Governance arrangements in the face of AI innovation, warns that financial services organisations should not allow AI adoption to move ahead of their ability to govern it. Although the report is Australian, its findings are highly relevant to New Zealand financial advice providers.
AI adoption is accelerating
ASIC reviewed 624 AI use cases across 23 Australian financial services and credit licensees. It found that 57% of use cases were less than two years old or still being developed, while 61% of organisations planned to increase their use of AI over the following year. Most uses remained relatively cautious, with AI generally supporting rather than replacing human decisions. However, ASIC found that some organisations were adopting AI faster than they were updating their governance and risk-management arrangements. This creates what ASIC describes as an AI governance gap.
The risks extend beyond cybersecurity
AI can improve efficiency, accessibility and customer service. It can also produce or amplify harm.
ASIC identified risks including:
- inaccurate or misleading information
- biased or discriminatory outcomes
- inappropriate use of personal information
- decisions that cannot be properly explained
- customers not knowing AI influenced an outcome
- over-reliance on third-party platforms
Some organisations assessed AI mainly from the perspective of business risk. They gave less attention to how AI could affect customers, particularly through bias, unfairness or financial exclusion. For financial advisers, the level of governance should reflect the potential impact on the client. A tool used to improve an internal email presents less risk than a system that influences product selection, risk profiling or advice.
A working model may still be unsuitable
ASIC describes an organisation that used an AI model to help predict whether a customer might default on credit. The model affected decisions about whether credit would be offered and in what amount. However, the organisation had incomplete documentation, limited understanding of the third-party platform and no reliable way to explain how different factors affected a customer’s score. The model was eventually replaced with a simpler and more explainable system.
The lesson is clear: a system should not be trusted simply because its results appear consistent. A financial services organisation must also be able to test, monitor, challenge and explain the system.
Existing New Zealand obligations still apply
The FMA’s research found that New Zealand financial services providers were generally taking a cautious approach focused on customer outcomes, privacy, cybersecurity, staff training and human oversight.
Financial advice providers must also continue meeting their existing professional and legal obligations. AI does not reduce the need to:
- treat clients fairly
- provide suitable advice based on reasonable grounds
- help clients understand the advice
- protect confidential and personal information
- maintain appropriate competence and oversight
An adviser should not rely on an AI-generated output without checking its assumptions, accuracy and relevance to the client’s circumstances. Responsibility remains with the adviser and the financial advice provider, not the software developer.
What this means for your organisation
1. Identify where AI is being used
Maintain a register of approved tools, trials, embedded AI features and informal staff use.
Record what each tool does, what information it accesses and whether its output could affect a client.
2. Assess risks from the client’s perspective
Consider what could happen if the tool is wrong, biased, unavailable or misunderstood.
Apply stronger controls where AI influences recommendations, client communications, eligibility decisions or personalised advice.
3. Review third-party providers
Understand how the provider stores, retains and uses client information.
Check whether data is used to train models, whether the system can be tested and how the provider manages security incidents and system changes.
4. Define meaningful human oversight
Specify who reviews AI outputs, what they must check and when results must be rejected or escalated.
Human review should involve genuine professional judgement, not simply approving a generated response.
5. Monitor performance
Record errors, complaints, unusual results and occasions when staff override the system.
Review tools regularly to confirm they remain accurate, secure and suitable for their approved purpose.
Governance should lead adoption
ASIC’s report does not suggest that financial advice firms should avoid AI. It warns that governance must keep pace with adoption. For New Zealand providers, the best approach is to build on existing conduct, privacy and security obligations while introducing controls appropriate to each AI use case. AI can support advisers and improve client service. It should not weaken the reasonable grounds for advice, reduce transparency or make accountability harder to identify.
Sources
Financial Markets Authority, Understanding Artificial Intelligence in Financial Services, July 2024.
Qualification
ASIC’s report relates to Australian financial services and credit licensees. The FMA paper is research rather than formal regulatory guidance. This article provides general information and is not legal, privacy, cybersecurity or financial advice.
KiwiGen.AI 