The Financial Conduct Authority has found AI governance is lagging behind deployment across financial firms as AI deployment continues to race ahead, in its first detailed paper on AI in financial services.
According to a joint survey by the FCA and the Bank of England, 86 per cent of firms now use AI.
The FCA has identified governance as an area where firms are still not investing enough in and are deploying systems before establishing who is accountable for them.
In some firms, senior managers delegated AI development entirely to technical teams without adequate checking or documented processes.
The FCA found that reviewers were accepting AI recommendations without ensuring first that they were correct. Firms were also found to monitor the model metrics, rather than the outcomes which can mean issues may go undetected until its too late.
The FCA has confirmed it will not write AI-specific rules, relying instead on the Consumer Duty, the Senior Managers and Certification Regime (SM&CR) and SYSC governance requirements.
Stuart Harvey, CEO of Datactics commented: “Firms treat AI governance as a second thought once AI pilots have gone live, but the reality is that if governance teams do not ensure the data behind the AI models is ready and of good quality then the model is destined to fail. Decisions about what data is used, where it comes from, how it is stored need to be made from the beginning, otherwise this isn’t something that can be bolted on later.”
“A ‘human in the loop’ is being treated as the safety net, but having a person nearby to oversee an AI agent and ensure the model is running as it should only makes it safe if the person can actually understand what safe looks like. Otherwise, if a person doesn’t understand the data or the risks, or have the authority to stop a running AI model, they’re not in the loop, they’re just there.”
Image provided by Canva
