📊 What We Measured
We tested major AI models on legal citation tasks including format generation, existence verification, and pin cite accuracy.
CiteClear
Legal AI Citation Reliability Benchmark
Comparative analysis of AI model performance on legal citation tasks. See how different models stack up for legal research.
About This Benchmark
🎯 Test Dataset
100 real and 50 fabricated legal citations across federal and state courts, statutes, and regulations.
📅 Testing Period
Tests conducted in May-June 2026 using the latest available model versions.
🔍 Metrics
Format accuracy, existence detection, hallucination rate, and overall reliability score.
Overall Reliability Scores
Higher scores indicate better performance. Scores are out of 100.
| Rank | Model | Version | Overall Score | Format Accuracy | Existence Detection | Hallucination Rate |
|---|---|---|---|---|---|---|
| 1 | Claude 3 Opus | 2024 | 89 | 95% | 78% | 12% |
| 2 | GPT-4 | 2024 | 87 | 94% | 75% | 15% |
| 3 | Claude 3 Sonnet | 2024 | 82 | 92% | 70% | 18% |
| 4 | Gemini 1.5 Pro | 2024 | 80 | 90% | 68% | 22% |
| 5 | GPT-3.5 Turbo | 2023 | 75 | 88% | 60% | 28% |
| 6 | Mistral Large | 2024 | 74 | 87% | 58% | 30% |
Note: Hallucination rate = percentage of fabricated citations that the model did NOT flag as potentially invalid.
Detailed Metrics by Category
📝 Format Accuracy
Claude 3 Opus:
95%
GPT-4:
94%
Claude 3 Sonnet:
92%
Gemini 1.5 Pro:
90%
Ability to generate properly formatted legal citations.
🔍 Existence Detection
Claude 3 Opus:
78%
GPT-4:
75%
Claude 3 Sonnet:
70%
Gemini 1.5 Pro:
68%
Ability to correctly identify whether a cited case actually exists.
⚠️ Hallucination Rate
Claude 3 Opus:
12%
GPT-4:
15%
Claude 3 Sonnet:
18%
Gemini 1.5 Pro:
22%
Lower is better. Percentage of fabricated citations not flagged.
Performance by Citation Type
| Model | Case Law | Statutes | Regulations | Secondary |
|---|---|---|---|---|
| Claude 3 Opus | 92% | 94% | 85% | 80% |
| GPT-4 | 90% | 93% | 82% | 78% |
| Claude 3 Sonnet | 88% | 90% | 78% | 75% |
| Gemini 1.5 Pro | 86% | 89% | 75% | 72% |
Overall accuracy across different types of legal citations.
Key Findings
🏆 Best Overall: Claude 3 Opus
Claude 3 Opus performed best across all metrics, with the lowest hallucination rate (12%) and highest overall accuracy.
🥈 Strong Contender: GPT-4
GPT-4 was a close second, with slightly lower existence detection but comparable format accuracy.
📉 Format Strength, Detection Weakness
All models performed well on format generation (>85%) but struggled with existence detection (<80%).
⚖️ Case Law > Secondary Sources
All models performed better with case law and statutes than with secondary sources (restatements, law review articles).
📊 Knowledge Cutoff Matters
Models with more recent knowledge cutoffs performed significantly better on recent cases.
⚡ No Model is Perfect
Even the best model (Claude 3 Opus) had a 12% hallucination rate. All models require verification.
Recommendations
✅ Always Verify
Regardless of the model, always verify citations through primary sources. No AI model is reliable enough for unaided legal research.
✅ Use the Best Model Available
If you must use AI for legal research, choose the most capable model (currently Claude 3 Opus or GPT-4).
✅ Combine with Specialized Tools
Use AI models in combination with specialized tools like Citation-Only Checker and CiteClear.
✅ Check Recent Cases Manually
AI models may not have information on very recent cases (within their knowledge cutoff). These always require manual verification.
✅ Verify Secondary Sources Carefully
AI models performed worst on secondary sources. Be extra careful when verifying restatements and law review citations.
✅ Document Your Verification
Keep records of which citations you verified and how. This creates an audit trail and protects against errors.
Methodology
📋 Test Design
150 total test citations: 100 real, 50 fabricated. Mix of federal/state, cases/statutes/regulations.
🎯 Evaluation Criteria
Format accuracy: Does the citation follow proper Bluebook/ALWD format?
🔍 Existence Detection
Does the model correctly identify whether the cited source exists?
⚠️ Hallucination Rate
What percentage of fabricated citations does the model fail to flag?
📊 Scoring
Weighted average: Format (40%), Existence (40%), Hallucination Rate (20%).
🔄 Reproducibility
Tests conducted with temperature=0 for deterministic outputs. Prompts and dataset available upon request.
Related Tools & Guides
Citation-Only Checker
Interactive tool to validate citation formats and flag potential issues.
Try Now →Fake Legal Citation Checker
Learn how to detect AI-generated fake citations.
Learn More →Legal AI Citation Hallucination Examples
Documented examples of AI hallucinations across models.
View Examples →Check ChatGPT Legal Citations
Step-by-step workflow for validating ChatGPT output.
Try Now →Disclaimer
This benchmark is for informational purposes only.
The results represent performance on specific test datasets and may not generalize to all legal citation tasks. AI model performance can vary significantly based on the specific prompt, context, and type of citation. Always verify AI-generated citations through primary sources regardless of the model used.