Companies are selling AI to lawyers faster than courts can fine them for using it.
Andrew Harnik/Getty Images
So far this year, three of the four major AI labs have either launched or pitched products aimed at lawyers. Google put Gemini Enterprise for Legal into preview in late August with four of the world’s biggest firms, three months after Anthropic released what is now known as Claude Legal Solutions. Elon Musk is also interested, with his SpaceXAI lab having thrown up a dedicated legal solutions page on its website.
Together, it marks a push into legal services that has intensified even as lawyers continue to be fined and vilified for submitting AI filings packed with fictitious case law. In June, the Ontario Law Society Tribunal ordered lawyer Shahryar Mazaheri to pay tens of thousands of dollars for using filings drafted using Grok, which contained fake case citations. Researcher Damien Charlotin’s database has cataloged nearly 2,000 such cases worldwide, involving more than 800 lawyers and 1,100 self-represented litigants.
Lawyers keep getting caught submitting AI slop
The Ontario Law Society found last December that Mazaheri had used an earlier version of Grok to draft a factum (a written legal argument) for a client’s appeal. This, tribunal documents say, produced fabricated citations and invented legal principles. A costs decision issued back in June ordered Mazaheri to pay $31,150, with the adjudicators noting that an LLM “does not appreciate nuance or exercise judgment or use a moral compass,” and is “strongly predisposed to giving an answer — any answer — rather than admitting ignorance.” Adjudicators described his AI-generated materials as “gibberish.”
Sullivan & Cromwell, the white-shoe law firm that’s one of the highest-grossing in the United States and pays its brand-new, first-year attorneys a $225,000 base salary, is among the guilty. It filed an emergency letter with the Southern District of New York in April after a bankruptcy motion it submitted to Chief Judge Martin Glenn was found to contain 42 AI-generated fabrications. In Canadian courts alone, reported cases involving fabricated citations rose from just seven in 2024 to 86 in 2025, with 39 more in the first quarter of 2026.
Before any major AI company had released a specific legal tool, DoNotPay dropped plans in early 2023 to have an AI “robot lawyer” argue a case in court after state bar associations threatened criminal prosecution.
Built for verification
Bangla press/Shutterstock
Google and Anthropic both imply that their legal products are built for verification. Google’s product routes queries via connectors to external legal databases such as Everlaw and NetDocuments, rather than relying on the model’s training data in isolation. Google Cloud CEO Thomas Kurian said at launch that “ensuring every aspect of these agentic workflows is accurate, factual, and grounded in legal authority is of critical importance.” Gottlieb, Freshfields, Weil and Williams & Connolly, four of the world’s largest law firms, are involved in the preview program.
Anthropic has taken a similar approach with its Claude Legal Solutions product. It includes a reported 20 connectors to legal platforms alongside 12 pre-built plugins. The lab has claimed a 90.9 percent score on the BigLaw Bench legal reasoning benchmark for Opus 4.7 and counts Freshfields and Quinn Emanuel among its customers. The key difference between these products and the Mazaheri case is that rather than asking the model to generate citations from memory, it pulls them from external, verified sources.
SpaceXAI’s Grok-powered legal product, meanwhile, has thinner credentials. A Cursor blog post stated that Grok 4.5 was suited for “finance, legal work, or anything else you do on a computer,” but SpaceXAI’s later announcement for the model focused on “coding, agentic tasks and knowledge work” and made no mention of legal applications. The page also doesn’t disclose an architecture or mechanism for tracing citations to primary sources. SpaceX officially acquired Cursor for $60 billion on August 15, bringing the code editor’s legal marketing under the Grok umbrella.
Verification is still the lawyer’s job
In a 2024 study, Stanford’s RegLab found hallucination rates of 17 percent and 33 percent after testing purpose-built legal AI platforms from LexisNexis and Westlaw. There hasn’t yet been a comparable independent audit looking at Google or Anthropic’s legal tools.
All attorneys are subject to bar association rules that place a duty to verify citations on the attorney who files them, whether AI was used or not. Mazaheri’s $31,150 penalty wasn’t imposed because he used AI, but because its output wasn’t thoroughly checked. That obligation isn’t going to change, no matter how carefully designed a legal AI tool is, or how many vendor safeguards come packaged within.