In 2025, a study found that people trust legal advice from ChatGPT more than advice from actual lawyers — which is both remarkable and alarming, because in that same year multiple attorneys were sanctioned by federal courts for submitting AI-generated briefs containing citations to cases that don't exist, and at least one prominent case established that AI conversations with legal AI tools do not carry attorney-client privilege. The gap between how authoritative AI sounds when explaining legal questions and how reliable it actually is in a legal context is the thing everyone needs to understand before using it for anything that matters. AI can genuinely help with basic legal tasks. It can also confidently guide you into a legally catastrophic decision, and it won't flag the difference.
This is the second piece in our real-life AI series second volume — following the dating app algorithm piece. This one focuses on legal situations: what AI can realistically do for you without a lawyer, where the documented failure modes are, and how to use it without creating problems worse than the one you started with.
What AI Can and Can't Do in Legal Contexts
| Task | AI Capability | Risk Level | Verdict |
|---|---|---|---|
| Explaining legal concepts in plain language | High — AI is genuinely good at translating legal jargon into plain English | Low — general education, not advice | Use freely |
| Summarizing a contract you received | Good — AI can identify key clauses, unusual terms, and potential red flags | Low to Medium — misses jurisdiction-specific issues | Useful starting point; verify anything flagged as concerning |
| Generating a simple standard-form document (NDA, basic service agreement) | Moderate — AI produces plausible drafts, but misses jurisdiction-specific requirements | Medium — local laws and municipal rules may make AI drafts unenforceable | Use as a first draft; have a lawyer review before signing anything significant |
| Legal research (identifying relevant law or cases) | Unreliable — AI frequently cites cases that don't exist or misrepresents holdings | High — multiple attorneys have been sanctioned for relying on AI citations | Always verify every citation against Westlaw, LexisNexis, or official court databases before using |
| Advice on your specific legal situation | Low — AI lacks knowledge of your jurisdiction, local rules, and the specific facts that determine outcomes | High — wrong advice in specific legal situations has major consequences | Do not rely on AI for specific legal strategy without a lawyer |
| Court filings, motions, or pleadings | Dangerous — AI cannot verify citations, doesn't know local court rules, and produces unverified output | Very High — multiple sanctions issued in 2025–2026 for AI-generated court filings | Do not use AI to generate court filings without a licensed attorney reviewing every word and citation |
The Citation Hallucination Problem: Why It's a Bigger Deal in Law Than Anywhere Else
In most domains, an AI hallucination is embarrassing or inconvenient. In legal contexts, it can end a case, result in sanctions, and permanently damage a lawyer's career — or leave a non-lawyer in a significantly worse position than if they'd done nothing. The Mata v. Avianca case established this pattern in 2023, and the legal AI landscape in 2026 is still dealing with its ripple effects. As of 2026, federal judges are issuing standing orders requiring lawyers to verify AI-generated text before filing, a direct response to a pattern of briefs containing fabricated cases.
The case that matters most in 2026 is Heppner, referenced in multiple recent legal AI analyses as having changed the risk calculus for every attorney using general-purpose AI for client matters. Multiple lawyers have been sanctioned — including fines and referrals to grievance committees — in cases like Mata v. Avianca, Park v. Kim, and Wadsworth v. Walmart. In every case, the offense was submitting a filing with AI-generated citations that were never verified against Westlaw, LexisNexis, or Casetext.
For non-lawyers using AI to research their own legal situations, the hallucination risk is just as real but harder to catch — because unlike a lawyer who knows what a real citation looks like, most people won't recognize when an AI has invented a case name, a statutory provision that doesn't exist, or a legal standard that applies in a different jurisdiction. AI presenting fabricated legal authority in the same confident register as accurate information is a documented, recurring failure mode, not an edge case.
The Jurisdiction Problem Nobody Talks About
Law is local in a way that most AI explanations don't adequately reflect. Federal law applies uniformly, but the vast majority of legal situations that affect people's daily lives — landlord-tenant disputes, employment rights, small claims court, contract enforceability, estate planning requirements — are governed by state law, and frequently by county or municipal ordinances on top of that. A Wisconsin business will need a contract customized to Wisconsin law, and a document may need to be customized even further because local municipal or county rules may apply. Even small errors in legal language or structure can have major consequences, including unenforceable contracts, court disputes, or financial penalties.
When you ask AI to explain your rights as a tenant, it will typically give you a general answer based on common tenant protection principles. Whether those protections apply in your state, your city, or your specific lease arrangement is a different question — and AI frequently provides the general answer with the same confidence as if it were the jurisdiction-specific one. The consequence is that you might go into a dispute with your landlord thinking you have rights that your state law doesn't actually give you, or miss rights that your city's local ordinances do provide but that weren't covered in the general answer.
The Attorney-Client Privilege Gap
This is the issue that the Heppner decision specifically highlighted, and it matters for regular people as much as for lawyers. Attorney-client privilege — the legal protection that makes your communications with your attorney confidential and protected from disclosure in legal proceedings — does not automatically extend to your conversations with an AI tool. Lawyers should never input client-identifying data, privileged strategy memos, draft pleadings, or matter-specific facts into ChatGPT Free or Plus. Doing so constitutes disclosure to a party outside the attorney-client relationship.
For non-lawyers, the practical implication is this: if you're involved in a legal dispute and you describe the facts of your situation to ChatGPT, Claude, or Gemini in detail, that conversation is stored on the company's servers and is potentially discoverable in legal proceedings. The same detailed facts you share with an attorney are protected from disclosure; the same facts shared with an AI are not. If you're in a situation where legal proceedings are a realistic possibility, be careful about how much of the specific factual detail of your situation you share with a general-purpose AI chatbot.
What AI Is Actually Good For (and Worth Using)
The picture here isn't entirely negative — AI is genuinely useful for legal purposes when the task matches its actual capabilities rather than the capabilities people project onto it.
Plain-language explanation is the clearest genuine value. If you've received a contract, a legal notice, a court summons, or any document full of legal jargon you don't understand, AI is excellent at translating it into plain English and identifying the key things you need to understand. This isn't legal advice — it's legal literacy assistance — and it's something AI does well and that access to lawyers has historically been required to provide. According to Clio's 2025 Legal Trends Report, 79% of legal professionals now use AI tools precisely because this translation function has real value even for trained lawyers dealing with unfamiliar areas.
Simple standard-form documents for low-stakes situations are another reasonable use case — with the caveat that "low-stakes" needs honest evaluation. An NDA for a freelance project where the consequences of an unenforceable clause are limited is different from a lease agreement, a business partnership contract, or any document where the stakes of getting it wrong are significant. AI can produce a serviceable first draft; whether that draft is appropriate for your jurisdiction and situation requires human review.
Understanding your options before deciding whether you need a lawyer is probably the highest-value legal use case for AI. Understanding what a legal concept means, what the general process for a small claims court filing looks like, or what rights tenants generally have in your state helps you make an informed decision about whether your situation warrants professional legal advice — and if it does, helps you have a more productive initial conversation when you get there. This is fundamentally different from relying on AI as a substitute for professional advice.
When You Absolutely Need a Real Lawyer (Not AI)
- Any criminal matter. The consequences of criminal proceedings — fines, probation, incarceration, criminal record — are too severe and the procedural requirements too complex and jurisdiction-specific for AI-guided self-representation to be appropriate.
- Family law (divorce, custody, adoption). These proceedings directly affect children and involve highly jurisdiction-specific rules, discretionary judicial decisions, and emotional stakes that AI cannot adequately navigate.
- Immigration matters. Immigration law is federal but extraordinarily complex and consequential — mistakes can result in deportation or permanent bars from entry. Even experienced immigration attorneys specialize in specific areas because the field is so complex.
- Any court filing in an active case. The sanctions being issued to lawyers who submitted AI-generated court filings in 2025–2026 are a clear signal that AI-generated court filings without thorough attorney review create legal liability rather than resolving it.
- Estate planning (wills, trusts, powers of attorney). State-specific formality requirements for will execution vary significantly. An AI-drafted will that doesn't meet your state's witness and notarization requirements may be legally invalid — meaning your wishes aren't honored.
- Real estate transactions. Title issues, zoning restrictions, disclosure requirements, and local transaction customs vary enormously by location and can have major financial consequences if mishandled.
Frequently Asked Questions
Can AI give me real legal advice?
No — AI can explain legal concepts and provide general information, but it cannot provide legal advice in the professional sense. Legal advice requires a licensed attorney who understands your specific jurisdiction, your specific facts, and the specific applicable law, and who carries professional accountability for the guidance they give. AI carries no such accountability and frequently provides inaccurate jurisdiction-specific information with the same confident tone as accurate information.
Is it safe to use AI to draft contracts or legal documents?
For low-stakes situations, AI-generated first drafts can be a useful starting point — but local laws and municipal rules may make AI-generated documents unenforceable, and even small errors in legal language can have major consequences. Any document you intend to rely on for something significant should be reviewed by a licensed attorney before you sign it.
Why do AI tools give wrong legal information?
Two main reasons: citation hallucination (AI generates plausible-sounding case citations and legal references that may not exist), and jurisdiction blindness (AI typically provides general information rather than jurisdiction-specific advice, which may not reflect the law in your state, county, or municipality). Multiple attorneys have been sanctioned in federal court specifically for relying on AI-generated case citations without verifying them.
Are my conversations with AI about legal matters confidential?
No — attorney-client privilege does not apply to conversations with AI tools. If you share detailed facts about a legal situation with ChatGPT, Claude, or Gemini, those conversations are stored on company servers and are potentially discoverable in legal proceedings. The same information shared with a licensed attorney is protected; the same information shared with an AI is not.
What legal tasks is AI actually useful for?
AI is genuinely useful for understanding legal jargon in plain English, identifying key clauses in contracts you've received, getting a general overview of how a legal process works before deciding whether to hire a lawyer, and generating first drafts of simple standard-form documents for low-stakes situations. It is not appropriate for legal research you plan to act on, any matter involving court filings, or specific advice about how to handle a real legal dispute.