ChatGPT can almost pass bar exam, but AI won’t replace lawyers, says Illinois Tech law professor
AI tools like GPT are like a jetpack for attorneys, Professor Daniel Martin Katz says. It’s important for lawyers to adapt.
CHICAGO—March 7, 2023—OpenAI’s GPT-3.5 can nearly pass the Multistate Bar Exam—and it likely will succeed soon, according to Law Professor Daniel Martin Katz of Illinois Institute of Technology’s Chicago-Kent College of Law. Although his findings have garnered significant attention for their implications for the future of legal education, Katz wants to emphasize that artificial intelligence will not replace lawyers.
AI tools like GPT-3.5 can improve efficiency for lawyers, but there needs to be a “human in the loop” providing oversight, Katz says. But AI will change how attorneys do their jobs much sooner than many expected.
“Lawyers need to figure out how to use these tools. And those that do, it’ll be a very lucrative thing for them. We’re sitting on the dawn of a major increase in potential capacity,” says Katz. “These are tools that allow you to more effectively do your work, so you need to learn how to use them to maximum efficacy.”
Katz and co-author Michael J. Bommarito III published their paper “GPT Takes the Bar Exam” in November 2022, demonstrating that the popular AI software passed two portions of the bar exam—evidence and torts. ChatGPT is the popular free demo version of a fuller program, TEXT-DAVINCI-003, better known as GPT-3.5.
“To pass the test, you have to have a pretty good command of not just the English language, but the nuances of ‘legalese,’ or legal language, and that’s a difficult task, even for humans,” Katz says. “You have to go to seven years of postsecondary education, just to get to a test that one in five people don’t pass.”
But passing the bar exam wasn’t the point of the research. Katz says this software isn’t going anywhere, and he wants to show that lawyers have to get on board or risk falling behind.
“A person using tools like this is able to do a lot more, but they’re the ones providing the oversight,” Katz says. “If your lawyer uses a research tool like Westlaw or LexisNexis, they still have to stand behind the use of the tool. There needs to be a human in the loop. You shouldn’t use these tools with no human oversight, but a person using this has a jetpack now.”
So how soon will it be before GPT passes?
“We say in the paper that in 0–18 months it will pass the bar exam, I’ll stick by that,” Katz says.
Photo: Professor of Law Daniel Martin Katz
Illinois Institute of Technology
Based in the global metropolis of Chicago, Illinois Tech was born to liberate the collective power of difference to advance technology and progress for all. It is the only tech-focused university in the city, and it stands at the crossroads of exploration and invention, advancing the future of Chicago and the world. It offers undergraduate and graduate degrees in engineering, computing, architecture, business, design, science and human sciences, and law. Illinois Tech students are guaranteed hands-on experiences, personalized mentorship, and job readiness through the university's one-of-a-kind Elevate program. Its graduates lead the state and much of the nation in economic prosperity. Its faculty and alumni built the Chicago skyline. And every day in the living lab of the city, Illinois Tech fuels breakthroughs that change lives. Visit iit.edu.
Media contacts
Kevin Dollear
Communications Manager
Illinois Institute of Technology
Cell: 773-860-5712
kdollear@iit.edu
