Skip to content
Menu
Journal of Science and Technology Law
  • Blog
  • Publications
  • Membership
    • 2020-2021 JOST Executive Board
    • 2019-2020 JOST Executive Board
    • 2019 – 2020 Associate Editors
    • 2015 – 2016 JOST Staff
    • 2013 – 2014 JOST STAFF
    • 2012 – 2013 Staff
    • 2011-2012 Staff
    • Membership Requirements
    • Information for Prospective 1Ls
    • Bylaws
  • Partners
  • Contact
Journal of Science and Technology Law

ChatGPT: Will lawyers be replaced by Artificial Intelligence?

Posted on February 27, 2023February 27, 2023 by Miriam Mokhemar

By Nicole Cooper

Could artificial intelligence systems replace lawyers in the future? The short answer is no. However, the capabilities of these artificial intelligence (AI) systems are concerning. The most popular AI system in the world’s view right now is ChatGPT. ChatGPT is an AI chatbot system that OpenAI, a company in San Francisco, released in November 2022.[1] ChatGPT is a generative AI that allows users to enter written prompts and receive human-like text or images and videos generated by the AI.[2]  This system can produce human-like responses to just about any request or prompt from a user.[3]

One reason people are concerned ChatGPT could replace jobs is that it has passed a range of tests and exams, including law exams and portions of the bar exam.[4] Daniel Katz, a professor at Chicago Kent College of Law and Michael Bommarito, a professor at Michigan State University College of Law, tested OpenAI’s GPT3.5 model on the multistate multiple choice (MBE) section of the Bar exam.[5] Overall, it was correct 50.3% of the time as compared to regular test-takers, who get 68% of the questions right.

Recently, professors at the University of Minnesota Law School graded tests blindly. After completing 95 multiple-choice questions and 12 essay questions, the bot performed on average at the level of a C+ student, achieving a passing score.[6] Although this system is smart enough to pass complicated exams, its scores aren’t particularly high enough to show that it can perform better than a law student. It would be more concerning if the exam produced perfect scores.  

ChatGPT also has severe limitations. It cannot perform the complex tasks that lawyers do daily such as applying facts to the law, providing advice that is in the best interests of clients, interviewing witnesses, negotiating settlements, and much more. It also struggles to know what the user really wants from it as its answers to questions vary widely to how the prompts are phrased.[7] Its inability to perform interpersonal tasks will keep its potential advancements into the legal field limited to a research tool similar to Westlaw and LexisNexis.[8]

Lastly, I actually tested out ChatGPT myself and asked the AI a wide range of requests/questions. These prompts included a recipe for chocolate chip cookies, a poem about dogs, and lastly a question: can lawyers be replaced by ChatGPT? After typing out a great chocolate chip cookie recipe, and a heart-warming poem about dogs, the response to my last question was interesting as it came to the same conclusion that I did. ChatGPT states that “No, lawyers cannot be replaced by ChatGPT or any other AI language model. While AI technology has made significant strides in recent years, there are still many areas where human lawyers have critical advantage.”[9] I included the rest of the response in a screenshot below.

It is much appreciated that this AI does not think it will take over the legal field, however, lawyers should be careful if they choose to use this technology in practice. While it may be able to draft legal documents and conduct legal research faster than humanly possible, it can still make mistakes and provide misinformation. Even OpenAI stated on their website, “ChatGPT sometimes writes plausible-sounding but incorrect or nonsensical answers.”[10] If lawyers rely on AI to do the work for them, it will likely create a host of liabilities and malpractice concerns. 


[1] Ryan Browne, All you need to know about ChatGPT, the A.I. chatbot that’s got the world talking and tech giants clashing, cnbc (Feb. 98, 2023, 7:37 AM EST), https://www.cnbc.com/2023/02/08/what-is-chatgpt-viral-ai-chatbot-at-heart-of-microsoft-google-fight.html#:~:text=In%20its%20own%20description%2C%20ChatGPT,talk%20of%20the%20business%20world.

[2] Id.

[3] Id.

[4] Samantha Murphy Kelly, ChatGPT passes exams from law and business schools, cnn business (Jan. 26, 2023, 1:35 PM EST), https://www.cnn.com/2023/01/26/tech/chatgpt-passes-exams/index.html.

[5] Michael Bommarito & Daniel Katz, GPT Takes the Bar Exam, ssrn (Jan. 03, 2023), https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4314839.

[6] Kelly, supra note 4.

[7] Jason Hennessey, How Law Firms Can Leverage ChatGPT To Get More Cases, the national law review (Jan. 17, 2023), https://www.natlawreview.com/article/how-law-firms-can-leverage-chatgpt-to-get-more-cases#:~:text=Recent%20adopters%20have%20been%20impressed,would%20typically%20be%20manual%20tasks.

[8] Eran Kahana, ChatGPT Is a Cool Trick, But AI Won’t Replace Lawyers, lexis+ (Feb. 07, 2023, 3:39 PM EST), https://plus.lexis.com/newsstand#/law360-pulse/article/1573178.

[9] ChatGPT, openai, https://chat.openai.com/chat (last visited Feb 23, 2023).

[10] ChatGPT: Optimizing Language Models for Dialogue, openai, https://openai.com/blog/chatgpt/ (last visited Feb. 23, 2023).

Blog Posts

I THINK MY DAD’S GONE CRAZY: INTELLECTUAL PROPERTY RIGHTS ON YOUTUBE

Federal Court in Montana Blocks Proposed Mine in Cabinet Mountains for Failure to Consider Environmental Impact on Endangered Grizzly and Bull Trout Populations

We’re Calling About Your Car Warranty: A Glimpse Inside the Fraudulent Phone Calls Used to Obtain Sensitive Information and the Steps You Can Take to Stop These Scams.

RECREATIONAL MARIJUANA IN NEW YORK: HOW WE GOT HERE AND WHAT IT MEANS FOR OUR STATE

Technology in Legal Education and Practice: Then, Now, and Next

Volumes

Volume 35 - 2020-2021
Volume 34 - 2017-2018
Volume 33 - 2016-2017
Volume 32 - 2015-2016
Volume 31 - 2014-2015
Volume 30 - Spring 2014
Volume 29 - Fall 2013
Volume 28 - Spring 2013
Volume 27 - Fall 2012
Volume 26 - Spring 2012
Volume 22 - Spring 2010
Volume 21 - Fall 2009

Staff & Guidelines

2021-2021 JOST EDITORIAL STAFF

The Bluebook is the card catalog of the legal profession
©2023 Journal of Science and Technology Law | Powered by WordPress & Superb Themes