Research

Publications

• “Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools.” 2025. Journal of Empirical Legal Studies 22(2): 216-242 (with Varun Magesh, Faiz Surani, Mirac Suzgun, Christopher D. Manning, and Daniel E. Ho). We demonstrate the distinctive challenges that retrieval-agumented generation (RAG) faces in the legal domain, stressing the need for transparent benchmarking of commercially available legal AI products.
  • Media: Fortune, Wired, VentureBeat, Law360, Law.com, Gizmodo, CalMatters
  • [Benchmarking dataset]
• “Large Legal Fictions: Profiling Legal Hallucinations in Large Language Models.” 2024. Journal of Legal Analysis 16(1): 64-93 (with Varun Magesh, Mirac Suzgun, and Daniel E. Ho). We show that popular AI tools like ChatGPT frequently “hallucinate,” or invent, false information about American case law, calling into question their ability to democratize access to justice.
  • Media: Bloomberg, NPR/Marketplace, The Hill, The Register, Fast Company, Scientific American, New York Times
  • Judicial citations: Supreme Court of British Columbia (Canada)
  • [Replication code]
  • [Benchmarking dataset]
• “Chain Novel, or Markov Chain? Estimating the Authority of U.S. Supreme Court Case Law.” 2024. Journal of Empirical Legal Studies 21(4): 861-898. I show how modeling the network of U.S. Supreme Court case law as a Markov chain unlocks an intuitive estimator of case authority that outperforms existing approaches in a variety of validation tasks.
  • [Replication code]
  • [HolmesRank scores]
  • [Depth of treatment data]
• “The Dogma Within? Examining Religious Bias in Private Title VII Claims.” 2021. Journal of Empirical Legal Studies 18(4): 742-764 (with Devan N. Patel and Matthew E.K. Hall). Examining an original dataset of private discrimination claims, we debunk the popular belief that judges’ religions cause them to decide cases in a distinctive way.
• “Eyecite: A Tool for Parsing Legal Citations.” 2021. Journal of Open Source Software 6(66): 3617 (with Jack Cushman and Michael Lissner). We open-source a highly performant engine for the analysis of legal citations, trained and deployed on the datasets of the Free Law Project and the Caselaw Access Project.
  • [Python package]

Working papers

• “Bye-bye, Bluebook? Automating Legal Drudgery With AI-Augmented Rule Following.” We show that off-the-shelf LLMs produce fully compliant Bluebook citations less than half of the time, cautioning against using such models to automate aspects of the law where fidelity to detail is paramount.
• “What Can We Learn About Discrimination From Random Judicial Assignment?” We precisely describe the empirical quantities that can (and those that cannot) be identified from random judicial assignment, providing new evidence for the sources of discrimination in the judiciary.

Other writing

• “Why did Sen. Graham grill Ketanji Brown Jackson about her religious faith?” The Washington Post. March 2022.
• “Which of Trump’s Supreme Court choices might be most reliably conservative?” The Conversation. September 2020.