The source that never existed
How courts, consultancies and law firms now check references that AI helped write
IAPQB Report No. 2 · Practice report · Published 15 November 2026
International AI Practice Qualification Board (IAPQB)
In one minute
- AI writing tools can produce references that look correct and point to nothing: a court decision nobody made, a book nobody wrote, a quote no judge said.
- Courts in the United States and England have penalised lawyers who filed such references. A public database listed 2,293 court and tribunal decisions on AI-made false content by 14 November 2026 [1].
- In Australia, Deloitte agreed to repay part of a government fee after a report it wrote with AI help was found to cite works that do not exist [2, 3].
- Organisations that handle this well now do four things: they treat every AI-supplied reference as unchecked, check it at the original source, make a named person answer for the check, and say when AI was used.
1. A familiar situation
An analyst asks an AI tool for three studies that support a point in a report. The tool returns three neat references with authors, years, titles and page numbers. Two of the studies exist. One does not, and nothing on the screen shows which. When the analyst asks the tool whether the third study is real, the tool can say yes. In the best-known case below, a lawyer did exactly this, and the chatbot replied that the case "does indeed exist" [4].
2. Why AI invents sources
A reference has a fixed shape: author, year, title, publisher, pages. Language models produce text that fits patterns learned from very large amounts of writing, and a reference is one of the most regular patterns there is. The model can fill that shape without finding any document behind it. The US National Institute of Standards and Technology (NIST) lists this as one of twelve risks of generative AI and calls it confabulation: "confidently stated but erroneous or false content" by which users may be misled [5].
IAPQB view. In each case below, a false reference reached a court or a client because a professional signed the document without opening the source. The penalties and refunds that followed fell on the people and firms who signed.
3. Four cases
3.1 New York, 2023: six opinions no court wrote
In a personal-injury claim against the airline Avianca, two lawyers filed a brief that cited earlier court decisions. Six of them had been generated by ChatGPT and did not exist [6]. The airline's lawyers reported in March 2023 that they could not find the cases. The two lawyers did not admit the problem until 25 May 2023, after the judge ordered them to explain [4, 6].
On 22 June 2023, Judge P. Kevin Castel of the federal court in Manhattan imposed a $5,000 penalty on the two lawyers and their firm. He ordered them to send his decision to their client and to each real judge whose name had been attached to a fake opinion [4, 6]. He found bad faith based on "conscious avoidance and false and misleading statements to the Court" [6]. He also wrote that there is "nothing inherently improper about using a reliable artificial intelligence tool for assistance" [6]. The penalty was for filing fake opinions and continuing to defend them after the court had questioned them.
3.2 London, 2025: the court turns to the people in charge
On 6 June 2025, two senior judges of the High Court, Dame Victoria Sharp and Mr Justice Johnson, ruled on two cases together [7]. In Ayinde v London Borough of Haringey, five cases cited in legal arguments did not exist [7]. In Al-Haroun v Qatar National Bank, witness statements relied on 45 authorities, and 18 of them did not exist. The client said he had produced the research with AI tools and online sources, and his solicitor had used it without checking [8]. The court held that the duty to check belonged to the solicitor even though the client had supplied the material, and lawyers in both cases were referred to their regulators [8].
The judges said that lawyers who use AI for research have "a professional duty therefore to check the accuracy of such research" against authoritative sources: the official legislation website, the National Archives database of judgments, the law reports and reputable legal publishers [7, para 7]. The duty also covers lawyers who rely on another person's work [7, para 8]. They added that "practical and effective measures must now be taken" by heads of chambers, managing partners and regulators, and that future hearings will ask whether those leaders did so [7, para 9]. The judgment was sent to the Bar Council, the Law Society and the Council of the Inns of Court [8].
3.3 Canberra, 2025: a government report that cited a book nobody wrote
Australia's Department of Employment and Workplace Relations paid Deloitte about AU$440,000 for an independent review of the Targeted Compliance Framework, the IT system that automates penalties in the welfare system. The report was published in July 2025 [2, 3].
Chris Rudge, a University of Sydney researcher in health and welfare law, found up to 20 errors. They included a book attributed to a Sydney law professor that does not exist, references to reports that do not exist, and an invented quote attributed to a judge in a federal court case [2, 3]. A corrected version was published that disclosed the use of Azure OpenAI, a generative AI system, and Deloitte agreed to repay the final instalment of its fee [2, 3]. The department said the substance of the review and its recommendations had not changed [2, 3].
3.4 The count keeps rising
Damien Charlotin maintains a public database of court and tribunal decisions in which a judge found that a party relied on AI-made false content; some listed decisions concern alleged AI use that was not confirmed. On 14 November 2026 it listed 2,293 decisions [1]. It does not try to count every false reference filed, only decisions that deal with one [1].
4. How practitioners handle it now
The practices below come from the courts, professional bodies and organisations in these cases.
Every AI-supplied reference starts as unchecked. The English court treats checking as a professional duty for anyone who uses AI for research [7, para 7]. The New York court penalised lawyers who filed references they had not verified and then defended them [6].
The check happens at the original source, outside the AI tool. In New York, the lawyer's check was to ask the chatbot again, and it confirmed its own invention [4]. The London court named the sources to use instead: official legislation, the official database of judgments and the law reports [7, para 7]. For books and papers, the same rule points to the publisher's page or a library catalogue.
A named person answers for the check. The duty covers lawyers who rely on others' work, and it stayed with the solicitor when the client supplied the false citations [7, para 8; 8]. Heads of firms and chambers are now expected to put measures in place, and courts will ask about them [7, para 9].
AI use is disclosed. The federal court for the Northern District of Texas now requires, through its local rules, disclosure of generative AI use in documents that lawyers prepared with it [9]. Deloitte's corrected report added a statement of its AI use [3].
Checking time is part of the budget. The American Bar Association's first formal opinion on generative AI, issued on 29 July 2024, says that the duties of competence, confidentiality, communication and reasonable fees apply. A lawyer may bill for the time needed to review an AI draft for accuracy and completeness, and in most circumstances may not bill for learning how to use the tool [10].
5. What to take from this
For people who use AI at work. Before a reference goes out under your name, open it at its source. Asking the same AI tool whether its reference is real does not count as a check [4].
For managers. Build the check into the workflow and into the time estimate [10]. In the cases above, the cost of a missing check fell on the professionals who signed and on their firms [2, 6, 8].
For schools, training providers and employers. The London court now asks whether leaders took practical measures [7, para 9]. The same question works inside any organisation that lets staff use AI for research.
6. Limits of this report
All four cases come from courts and public reports in the United States, England and Australia, and rules elsewhere differ. Some facts come from news reports and commentary on the decisions; the references show which. The database count includes some decisions in which AI use was alleged but not confirmed [1]. This report does not test any AI tool.
References
[1] Charlotin, D. AI Hallucination Cases database. https://www.damiencharlotin.com/hallucinations/ (accessed 15 November 2026; page last updated 14 November 2026). CC BY 4.0.
[2] Sadler, D. (7 October 2025). Deloitte to refund government over AI errors. Information Age, Australian Computer Society. https://ia.acs.org.au/article/2025/deloitte-to-refund-government-over-ai-errors.html
[3] McGuirk, R. (7 October 2025). Deloitte to partially refund Australian government for report with apparent AI-generated errors. Associated Press, as published by News4JAX. https://www.news4jax.com/business/2025/10/07/deloitte-to-partially-refund-australian-government-for-report-with-apparent-ai-generated-errors/
[4] The Register (22 June 2023). Lawyers who cited fake cases hallucinated by ChatGPT must pay. https://www.theregister.com/2023/06/22/lawyers_fake_cases/
[5] National Institute of Standards and Technology (July 2024). Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile (NIST AI 600-1), section 2. https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf
[6] Mata v. Avianca, Inc., No. 1:22-cv-01461-PKC (S.D.N.Y.), opinion and order, Document 54, filed 22 June 2023. https://storage.courtlistener.com/recap/gov.uscourts.nysd.575368/gov.uscourts.nysd.575368.54.0_3.pdf
[7] R (Ayinde) v London Borough of Haringey; Al-Haroun v Qatar National Bank [2025] EWHC 1383 (Admin), Divisional Court, 6 June 2025. https://caselaw.nationalarchives.gov.uk/ewhc/admin/2025/1383
[8] Anastasiadou, I. (16 September 2025). AI fabricated citations and the legal profession: lessons from the High Court. British Institute of International and Comparative Law. https://www.biicl.org/blog/116/ai-fabricated-citations-and-the-legal-profession-lessons-from-the-high-court
[9] United States District Court for the Northern District of Texas. Judge Brantley Starr [judge's requirements page]. https://www.txnd.uscourts.gov/judge/judge-brantley-starr (accessed 15 November 2026)
[10] American Bar Association (29 July 2024). ABA issues first ethics guidance on a lawyer's use of AI tools [on Formal Opinion 512]. https://www.americanbar.org/news/abanews/aba-news-archives/2024/07/aba-issues-first-ethics-guidance-ai-tools/
How to cite
IAPQB (2026). The source that never existed: how courts, consultancies and law firms now check references that AI helped write. IAPQB Report No. 2. International AI Practice Qualification Board. https://iapqb.org/research/002-fabricated-citations.html
About this report
Prepared by IAPQB. Each fact comes from the sources listed, and each source was opened and checked on 15 November 2026; anything that could not be checked was left out. AI tools were used to search and to draft, and IAPQB checked every claim against its source and is responsible for the text. IAPQB runs an AI work qualification; this report does not promote it. Corrections: contact@iapqb.org or an issue at github.com/iapqb/iapqb.github.io. Licence: CC BY 4.0.
Archived with the series on Zenodo: doi.org/10.5281/zenodo.24248199 · Markdown sources: github.com/iapqb/research