In this article we are going to touch on some of the legal, regulatory, and insurance implications of the use of AI by professionals, such as insurance brokers and lawyers. In doing so, we are going to ask you to draw a line not between humans on one side, and Generative AI on the other side, but between regulated professionals on one side, and other forms of intelligent life (including both unregulated humans, and silicon based Gen-AI) on the other side. We will start by explaining some of the key implications which follow from adopting that perspective, which we believe leads to a way of using Gen-AI which is simple to understand, and is safe. We will then end with a summary of the regulatory expectations for AI use in the insurance sector, across the world.

AI and us

As regulated professionals we are used to relying both on tools (e.g. textbooks, computers, emails, internet search engines), and other people, to help us produce our work. But because of the fundamentally different natures of tools and people, we apply very different standards to the reliance that we place on them. For the tools we work with, we tend to do two things: we accept the limitations about what those tools are capable of doing, but we expect the tools to be highly reliable (perhaps even nearly perfect) within the boundaries of those limitations. For the people we work with, on the other hand, we tend to be much more accepting of their imperfections (which may be many and various), and we focus instead on the value that those people can bring in particular areas.

The key to working effectively with Gen-AI is to recognise that whilst it has certain characteristics of a tool, it behaves in many ways like people do. That is unsurprising, given that it has been developed at least in part to mimic human behaviour, so that when we are interacting through a digital medium these days, it increasingly difficult to be sure about whether the “person” on the other end of a digital message is a human being or not. In our view we should be treating Gen-AI like a human who doesn’t have the same professional qualifications as we do (and so also doesn’t have the same regulatory obligations as we do), and who we don’t know very well. So, what does that mean, in practice?

When we meet other people for the first time we form an impression of their capabilities quite quickly. We might be unimpressed by them. Or we might form the view that they are highly capable. Perhaps even brilliant. But until we have got to know them, we will instinctively recognise two things that help to keep us safe in our interactions with them.

Firstly, we know that the new person who we have just met is capable of making mistakes. When we meet them we don’t know what type of mistakes this particular person is particularly susceptible to, but we do know that they are likely to be susceptible to certain types of error which will only reveal themselves over time.

For that reason, whilst it would make perfect sense for a lawyer to raise in front of the Court a particular case which she had read about in an authoritative legal text, it would make no sense for that lawyer to raise in front of the Court a case that she had been told about by someone she had just met in the street (even if the person in the street explained in an apparently convincing way that the case was useful and relevant).

Secondly, we know that there is a chance that the person might be untrustworthy. Whilst many people are consistently true to their word, some people are highly unreliable (despite often being well intentioned), and some other people are intentionally dishonest. Gen-AI is similar. We can usefully regard private Gen-AI systems which are produced by well-known companies, which have longstanding reputations for ethical behaviour, as being comparable to people who are consistently true to their word. On the other hand, we should regard public Gen-AI systems, which don’t retain information within a closed environment, and particularly systems which are produced by companies which are not familiar to us, or which do not have a longstanding reputation for ethical behaviour, as being comparable to people who we do not know. Whilst they might be trustworthy and reliable, they might also be unreliable and / or intentionally dishonest.

For that reason, whilst it would make perfect sense for a regulated professional to share confidential client information with a private Gen-AI system which is produced by a trusted provider (in the same way as most professionals have for years trusted their confidential client information with providers of cloud services), it would make no sense to share confidential client information with an unfamiliar public Gen-AI system.

A significant majority of the specific legal, regulatory, and ethical, implications of regulated professionals using Gen-AI follows from the two principles set out above. Because Gen-AI should be treated not as a tool, but more like a person: we should be open to interacting with Gen-AI in order to take advantage of its huge capabilities; but we must not rely on it for accuracy when it comes to producing information which falls within the scope of our own regulatory obligations as professionals; and we must be careful and particular about which forms of Gen-AI we share confidential information with. So:

It is ok for regulated professionals to use Gen-AI

Courts around the world have made clear that it is fine for lawyers to use Gen-AI. The US federal court in Mata v Avianca said that there is “nothing inherently improper about using a reliable artificial intelligence tool for assistance”. In Ayinde, the Divisional Court in the UK expressly noted that Gen-AI can be useful for lawyers even though it carries risks. Even Judges are permitted to use Gen-AI to write their judgments. According to the Judicial Guidance issues by Lord Justice Birss, the Lead Judge for AI in the UK, Judges are not prohibited from using AI, but bear personal responsibility for everything in their name, and they must use secure systems only.

Professionals cannot rely on the accuracy of the information produced

Whilst it is perfectly acceptable for professionals to use Gen-AI to efficiently capture and retrieve information, and to use it to gain creative insights (both in terms of analysis and the creation of ideas), the Courts and regulators have made clear that we cannot rely on the accuracy of the information produced by Gen-AI in performing our regulated functions.

In a recent 2026 decision involving the prestigious UK law firm Pinsent Masons, where a junior lawyer had relied on AI-hallucinated citations in a letter put before a Judge, the Court reaffirmed that Gen-AI cannot be relied upon to produce reliable legal research, and that lawyers have a professional duty to verify any AI output against authoritative sources before using it. The same principles apply just as much to other professionals as they do to lawyers. Similar comments were made by the Courts in Singapore in 2025 in Rasul v Mohideen, and in the same year by the Courts in Canada in Reddy v Saroya.

The use of Gen-AI without verifying the content of the work produced is also a problem from the point of view of insurance. A professional who incurs a liability for the work done by Gen-AI which they had checked, but negligently failed to spot errors in, could expect to be covered by their professional indemnity insurance. However, the same is not true for a professional who relies on Gen-AI context without checking its accuracy. Given that we know that Gen-AI has the propensity to hallucinate, and to create fictitious authorities to supports its arguments, a failure by a professional to verify Gen-AI output would be reckless rather than negligent (the test for recklessness being that a policyholder appreciates the risk of a particular type of harm, and then turns a blind eye to that risk, rather than trying to mitigate or avoid the risk). Whilst professionally indemnity insurance is designed to cover a professional’s liability for negligence, it does not cover reckless professionals. As a result, any professional who incurs a liability for the use of un-checked Gen-AI output can expect to have no insurance to protect them.

Table footnotes: 

1 Monetary Authority of Singapore 
2 The Monetary Authority of Singapore (MAS) MindForge AI Risk Management Toolkit, released on March 20, 2026, is a comprehensive framework designed to help financial institutions safely adopt traditional, generative, and agentic AI. 
3 APRA’s CPS 230 standard, effective 1 July 2025, mandates that by 1 July 2026, financial institutions must rigorously manage AI as a “material service provider,” ensuring operational resilience, accountability, and vendor oversight. 
4 https://www.iais.org/uploads/2025/07/Application-Paper-on-the-supervision-of-artificial-intelligence.pdf 
5 See para 3.20: https://www.fca.org.uk/publication/corporate/ai-update.pdf 

We must not share confidential information with open Gen-AI systems

Regulators around the world have made clear that professionals may not input confidential information into public AI systems. The Solicitors Regulation Authority in the UK requires there to be no raw client data [inputted] into public AI tools without informed consent.

Regulatory expectations for AI use in the insurance sector

Across major insurance markets, regulators have converged on a consistent supervisory approach: AI is regulated through existing insurance and financial-services frameworks, rather than through AI-specific insurance rulebooks. The focus is on outcomes, governance, accountability and evidence.