The courtroom drama is no longer just human; Artificial Intelligence (“AI”) is writing, arguing, and even confusing the script. A trending story that caused outrage in the courtroom and memes on social media occurred in the New York State Supreme Court Appellate Division’s First Judicial Department earlier this year. Jerome Dewald, a plaintiff representing himself pro se for the appeal, presented a video of an AI avatar to the court that he intended to argue for him.[i] Justice Sallie Manzanet-Daniels questioned the authenticity of the video and questioned Mr. Dewald about the video, who admitted that it was not a real person on the screen. Justice Manzanet-Daniels expressed her displeasure for being misled and Mr. Dewald later sent the court a letter apologizing.[ii]

This is not an isolated instance where AI is entering the courtroom in ways that has displeased the court. In a Georgia federal court, United States District Judge William M. Ray II presided over a hearing on a motion for summary judgment and questioned an attorney about her client’s response brief in the Katt Williams case.[iii] The brief was riddled with “AI hallucinations” that had Judge Ray not noticed, she “could have denied a motion based on what [she] cited, which was not correct.”[iv] The court, having dealt with this situation numerous times already, has expressed that the ramifications for submitting fictitious legal authorities include “wasting the opposing party’s resources to expose inaccuracies, diverting the court’s attention from other matters, and damaging the reputation of courts, judges, and the parties involved.”[v]

AI, although a seemingly new thing that the masses have only recently began to use, has been around since early to mid 1900’s.[vi] But OpenAI was founded in 2015 with GPT-1, which is the foundational architecture for ChatGPT.[vii] OpenAI released an earlier version of ChatGPT in 2022, which went viral on social media with the public using it for planning trips, helping with work, and making school assignments easier. [viii] ChatGPT has impacted the education system in different ways with fifty-seven percent of teachers believing it makes their lives easier, meanwhile other educators fear that the program threatens academic integrity by encouraging cheating and plagiarism.[ix] Students’ dependency on ChatGPT and the fact that it is such a new tool have led several school districts to block access to the tool altogether.[x]

On the other hand, the same reservations can be applied to how AI is being used in the legal field. The Utah Court of Appeals sanctioned an attorney for a brief written by a law clerk that was AI generated.[xi] The brief had AI generated citations and quotations of a case that could only be found on ChatGPT and not any legal database. The Utah Court of Appeals felt that the petitioner’s counsel did not meet their obligations to the Utah State bar and commented, “[w]e agree that the use of AI in the preparation of pleadings is a legal research tool that will continue to evolve with advances in technology. However, we emphasize that every attorney has an ongoing duty to review and ensure the accuracy of their court filings.”[xii]

The Utah court is correct, AI is a powerful advancement in technology that can be a helpful tool. LexisNexis has taken advantage of this powerful tool and has implemented AI into its software.[xiii] LexisNexis’ implementation of AI serves to answer legal questions, generate documents, and summarize materials for its users’ research. LexisNexis has brought enhancement to their AI tools “to streamline legal research, provide comprehensive, current legal analysis, and save time so legal professionals can focus on higher-value work.”[xiv] Westlaw has taken a similar approach by providing AI-Assisted Research which uses AI to help streamline workflow by analyzing information, generating answers, and comparing statutes.[xv]  This shows that popular legal platforms have acknowledged the benefits of AI and are using it to advance their software in ways to help the legal profession. However, it is important to know how to effectively use AI as a tool that helps streamline information, instead of a substitute for one’s own work, to avoid reprimand by the court.

Moreover, with such a new tool comes a lack of clarity in situations that have never been addressed.  Another notable concern is the admissibility of AI-generated evidence in a courtroom. A new proposed federal evidence rule is supposed to help tackle this lack of clarity.[xvi] Proposed Federal Evidence Rule 707 requires AI-generated evidence that is not accompanied by the testimony of a human expert to the same standard of evidence offered with expert testimony to be admissible.[xvii] This provides for a higher bar that would clear up the inconsistency that has been created with the relatively recent introduction of AI into the courtroom. This new rule “is a necessary first step for dealing with the increasing use of AI-created evidence.”[xviii] However, there are still some unanswered concerns “such as access to proprietary source code and constitutional concerns like the right to confront witnesses in criminal cases.”[xix] Additional concerns include how this rule only applies to evidence that has been acknowledged by all parties as created by AI, but not when its origins are disputed.[xx] As AI becomes a bigger part of the courtroom, the legal field must find a balance between using it to make work efficient while ensuring the accuracy and fairness of the justice system stay intact.

[i] Julia Bonavita, AI-generated Attorney Outrages Judge Who Scolds Man Over Courtroom Fake: ‘Not a Real Person,’ Fox News (Apr. 9, 2025, at 10:57 ET), https://www.foxnews.com/us/ai-generated-attorney-outrages-judge-who-scolds-man-over-courtroom-fake-not-real-person [https://perma.cc/4Z6D-ZV3B].

[ii] Id.; Larry Neumeister, An AI Avatar Tried to Argue a Case Before a New York Court. The Judges Weren’t Having it, AP News, https://apnews.com/article/artificial-intelligence-ai-courts-nyc-5c97cba3f3757d9ab3c2e5840127f765 [https://perma.cc/LKX9-6K59] (Apr. 4, 2025, at 1:52 ET).

[iii]  Kelcey Caulder, Judge Scolds Atty in Katt Williams Case for ‘AI Hallucinations,‘ Law360 (Aug. 27, 2025, at 4:56 ET), https://plus.lexis.com/newsstand/law360/article/2381423/?crid=b470de12-cdc1-4260-aa47-a0e117ece420 [https://perma.cc/B777-GWYE]; see Tamara C. Jordan & Michael D. Simpson, Court Cautions Ai-Generated Legal Filings May Face Consequences, Wood Smith Henning & Berman L., https://www.wshblaw.com/experience-court-cautions-ai-generated-legal-filings-may-face-consequences [https://perma.cc/89WK-YRBN] (last visited Sep. 24, 2025) (addressing appellant’s use of generative AI, which fabricated “hallucinations,” could result in “wasting the opposing party’s resources to expose inaccuracies, diverting the court’s attention from other matters, and damaging the reputation of courts, judges, and the parties involved.”);  see also Alim Al-Hamim v. Star Hearthstone, LLC, 564 P.3d 1117, 1125–26  (Colo. Ct. App. 2024).

[iv] Jordan & Simpson, supra note iii; see also Elaine Hamill, What are AI Hallucinations?, SAS, https://www.sas.com/en_us/insights/articles/analytics/what-are-ai-hallucinations.html [https://perma.cc/SBF6-98G3] (last visited Sep. 24, 2025) (“AI hallucinations happen when the large language models (LLMs) that underpin AI chatbots generate nonsensical or false information in response to user prompts.”).

[v] Jordan & Simpson, supra note iii.

[vi] See What is the History of Artificial Intelligence (AI)?, Tableau, https://www.tableau.com/data-insights/ai/history  [https://perma.cc/U4L2-VR7S] (last visited Sep. 24, 2025).

[vii] See Bernard Marr, A Short History of ChatGPT: How We Got to Where We are Today, Forbes Tech (May 19, 2023, at 1:14 ET), https://www.forbes.com/sites/bernardmarr/2023/05/19/a-short-history-of-chatgpt-how-we-got-to-where-we-are-today/ [https://perma.cc/97YZ-Z6PK].

[viii] Id.; see Jack Densmore, ChatGPT: Artificial Intelligence’s Impact on Education, Ass’n of Texas Pro. Educators, https://www.atpe.org/News-Media/Magazine/ATPE-News-Summer-2023/ [https://perma.cc/7ZBD-Z4QM] (last visited Sep. 24, 2025).

[ix] See Densmore, supra note viii; see also Abreanna Blose, As ChatGPT Enters the Classroom, Teachers Weigh Pros and Cons, Nat’l Educ. Ass’ n Today (Apr. 12, 2023), https://www.nea.org/nea-today/all-news-articles/chatgpt-enters-classroom-teachers-weigh-pros-and-cons [https://perma.cc/VKL9-SCRR].

[x] Densmore, supra note viii.

[xi] Maya Young, US Lawyer Sanctioned After Being Caught Using ChatGPT for Court Brief, The Guardian (May 31, 2025, at 18:40 ET), https://www.theguardian.com/us-news/2025/may/31/utah-lawyer-chatgpt-ai-court-brief [https://perma.cc/EKF9-4Q9W].

[xii]  Id.

[xiii] See Generative AI for Legal Professions: 5 Ways Lexis+ AI™ Drives Better Outcomes, Lexis Nexis: Legal Insights (Jan. 30, 2024), https://www.lexisnexis.com/community/insights/legal/b/thought-leadership/posts/generative-ai-for-legal-professions-5-ways-lexis-aitm-drives-better-outcomes [https://perma.cc/8NT4-DJD6].

[xiv] Press Release, Lexis Nexis, LexisNexis Enhances Lexis+ AI with New Features, AI Models, and Graph Technology to Further Drive High Quality, Trusted Answers for Legal Professionals (July 22, 2024), https://www.lexisnexis.com/community/pressroom/b/news/posts/lexisnexis-enhances-lexis-ai-with-new-features-ai-models-and-graph-technology-to-further-drive-high-quality-trusted-answers-for-legal-professionals [https://perma.cc/P639-8SEN].

[xv] Westlaw Edge With Ai-Assisted Research, Thomson Reuters, https://legal.thomsonreuters.com/en/products/westlaw-edge/features [https://perma.cc/LK6M-KF8V] (last visited Sep. 24, 2025).

[xvi] See Jack Karp, New AI Evidence Rule is a Good Start, But More is Needed, Law360 (Aug. 27, 2025, at 4:15 ET), https://plus.lexis.com/newsstand/law360-pulse/article/2381199/?crid=c5900382-f665-4a0e-87e9-8a3c5a897e93 [https://perma.cc/FVT2-FYWH].

[xvii] See Fed. R. Evid. 702.

[xviii] Karp, supra note xvi.

[xix] Id.

[xx] Id.