One of the most topical and potentially most significant recent developments had been the release of several different language-based generative artificial intelligence tools, such as ChatGPT and Google’s Bard. The advent of these tools, their ease of use, and their responsiveness has led to observers and commentators to question whether these tools could drive significant changes in the economy and labor force – among other things, for example, whether these tools might have significant implications for the practice of law. My own experience (discussed here) is that while these tools are interesting, they are no substitute for the research and writing of an experienced lawyer. A recent case, involving an experienced New York lawyer who relied on ChatGPT generated content in a legal brief in a client’s case, demonstrates the dangers involved for anyone who relies on ChatGPT as a substitute for legal research. The case was described in a May 27, 2023, New York Times article entitled “Here’s What Happens When Your Lawyer Uses ChatGPT” (here).

Background

On August 27, 2019, Roberto Mata flew from El Salvador to New York on Avianca Airlines. Mata claimed that during the flight an airline employee pushed a service cart into Mata, injuring his knee. Mata retained the Levidow, Levidow & Oberman law firm to file a lawsuit on his behalf against the airline for his alleged injuries. The airline filed a motion to dismiss the lawsuit, arguing, among other ground, that the lawsuit was untimely under the relevant statute of limitations.

In order to oppose the airline’s dismissal motion, Steven Schwarz, prepared a brief identifying more than half a dozen relevant court decisions dealing with “the tolling effect of the automatic stay on a statute of limitations.” Schwarz later admitting that he had relied on ChatGPT for the legal research he used in preparing the opposition brief. Chat GPT had identified the cases Schwarz cited in the legal brief, supplying Schwarz with the case citations, and decision text (including internal textual references to other relevant decisions).

The airline’s lawyers submitted a brief to the Southern District of New York stating that after diligent search, they had been unable to locate any of the cases cited in the opposition brief. Judge Castel asked Mata’s counsel to supply copies of the decision, which Schwarz did, in reliance on ChatGPT. The airline’s lawyer advised the court that there were unable to find any of the decisions the lawyer had provided to the court. There is a reason why the airline’s counsel couldn’t find the decisions. As the Times article about the case put it, the reason they couldn’t find the decisions is that “ChatGPT had invented everything.”

Judge Castel himself was unable to find the decisions and in fact obtained confirmation from the clerk of court of the relevant appellate court that the decisions did not exist. Judge Castel summarized the situation by saying that the opposition brief had been replete with “bogus judicial decisions, with bogus quotes and bogus internal citations.”

In connection with Judge Castel’s review of the situation, Schwarz, a 30-year veteran of the New York bar, submitted an affidavit acknowledging that he had used an artificial intelligence source, a source, Schwarz said, “that has been revealed itself to be unreliable.” Schwarz said that he had never used ChatGPT before and was “unaware of the possibility that its content could be false.” He added that he “greatly regrets” relying on ChatGPT “and will never do so in the future without absolute verification of its authenticity.”

Ironically, Schwarz did do one thing to check on ChatGPT – he had asked ChatGPT whether one of the cases it had provided in response to Schwarz’s inquiry was a “real case”? ChatGPT reassuringly said it was a real case, providing a citation. Schwarz then apparently asked “what is your source” and asking whether the other cases it provided were fake, to which ChatGPT reportedly responded “the other cases I provided are real and can be found in reputable legal databases.”

Judge Castle has set a hearing for June 8 to discuss potential sanctions.

Discussion

I have taken the time to review the unfortunate Mr. Schwarz’s situation here in the hope that I might prevent others from putting themselves at risk by relying on answers from ChatGPT without thoroughly checking the chatbot’s responses. If I can save one person from setting themselves up for embarrassment, my purposes here will have been served.

As I noted at the outset, I have been experimenting with ChatGPT and Bard, and my experience has been very consistent with Schwarz’s. The chatbot’s responses pretty consistently have been somewhere between unreliable and flat out wrong. I have found the chatbots pretty much unhelpful on any legal research of any depth. The chatbots can tell you what the National Labor Relations Board is, but don’t rely on the chatbots for a reliable description of the substance of the NLRB’s administrative proceedings or appellate court record.

I have also found ChatGPT unreliable on straightforward factual items. Recently, I was writing a blog post about the settlement in the Kraft Heinz securities litigation (the post I published on the topic can be found here). I had a vague recollection while I was writing the post that Warren Buffett had been involved in the prior merger of Kraft and Heinz. I though I recalled that Buffett has said he regretted making the investment. ChatGPT told me very confidently that Buffett had said in his 2021 letter to Berkshire shareholders that he regretted the investment. I then went and read the 2021 letter (as well as the 2020 letter and the 2022 letter), and I couldn’t find what ChatGPT was referring to. Further diligence Google searching disclosed to me that Buffett had not said he regretted the investment but he had acknowledged that he had paid too much. And he had made this admission not in a letter to Berkshire shareholders, but in a CNBC interview. Most of the responses from ChatGPT on many different topics have this same “not quite right” defect.

The perverse thing about ChatGPT is that provides its faulty answers quickly and in orderly prose. It has a way of sounding very confident. After extended experience with ChatGPT, I have started to think of the chatbot as the AI equivalent of Cliff Clavin, the know-it-all at the end of the bar who has all of the answers but often doesn’t know what he is talking about.

The thing about it though is that after even limited experience with ChatGPT, it is obvious (to me, at least) that you could never take a ChatGPT response and put it in a legal brief. I find myself somewhat mystified that an experienced lawyer would take what the chatbot came up with and put it in a legal brief. Even though the chatbot did reassure Schwarz that the legal citations could be found on any “reputable legal databases.”

The chatbots may (or may not) someday provide a substitute for legal research or legal writing, but in any event, it is not going to be anytime soon.