When AI Gets It Wrong: Lessons from Recent Legal Cases
Artificial intelligence (AI) seems increasing regarded as the solution to every business problem.
Need to reduce administration? Use AI.
Need to analyse large amounts of data? Use AI.
Need to draft a report, manage customer enquiries or improve productivity? AI can apparently do all of that as well.
Yet as organisations rush to embrace the technology, courts are beginning to confront a fundamental question: when AI gets it wrong, who is responsible?
The result is a growing body of cases that provide valuable lessons for anyone using AI in the workplace. Artificial intelligence (AI) seems increasing regarded as the solution to every business problem.
Moffatt v Air Canada – When AI gives the wrong Answer
One of the most widely reported examples involved Air Canada’s customer service chatbot.
A passenger relied on information provided by the chatbot regarding bereavement fares and purchased flights accordingly. When the airline later refused to honour the advice, it argued that the chatbot was effectively responsible for its own statements. The British Columbia tribunal rejected that argument and held Air Canada responsible for inaccurate information published through its own system.
The case serves as a stark reminder that organisations remain accountable for decisions and representations made through AI tools.
Cork v Smith – When AI gets the law wrong
In a recent case involving the law firm Pinsent Masons, a junior solicitor used AI to draft letters containing an incorrect legal proposition. The AI had actually warned the user to verify the authorities, but this was not done. Supervisors also failed to check the output before it was sent to court.
The judge described the failures as serious and emphasised that lawyers remain responsible for checking the accuracy of material generated using AI. The firm referred itself to the SRA.
The lesson extends far beyond the legal profession. Whether AI is used to produce reports, generate customer communications or assist with decision-making, responsibility remains with the individual or organisation using the tool.
AI can be a powerful assistant, but professional judgment cannot be delegated to software.
Amazon – When we train AI to adopt our bias
Although not a case before the court, another cautionary tale involves a recent Amazon recruitment scheme. An AI-powered recruitment tool was developed intended to identify the best candidates for technical roles. However, because the system was trained using historical recruitment data that predominantly reflected successful male applicants, it learned to favour male candidates. Reports indicated that the software downgraded CVs containing references to women’s organisations and all-women colleges.
Amazon ultimately abandoned the project, highlighting a central challenge of AI: if the data contains bias, the technology may replicate and amplify it.
Getty v Stability AI – When AI uses data for training itself
Getty Images alleges that Stability AI has used millions of Getty’s copyrighted images, without permission, to train its AI image generator, Stable Diffusion. The decision is currently under appeal and remains an important case in the development of AI and intellectual property law.
The case demonstrates that the biggest legal risk for many businesses may not be what AI creates, but what it learned from and whether it had the right to learn from.
Conclusion
While many of the headline cases have emerged overseas, UK courts, regulators and professional bodies are increasingly grappling with the same issues. Whether the issue is discrimination, inaccurate advice, misuse of personal data, copyright, the law is unlikely to accept “the AI did it” as a satisfactory defence.
The future of work may involve AI, but the future of accountability remains human.
For bespoke advice on this or any other area of law, get in touch with the team now.
Meet the wider team
