by Richard W. Millar, Jr.

Artificial intelligence is a subject of continual discussion almost everywhere you turn. I am not sure why, as for years people have said that any intelligence I had was artificial. But, whatever it is, it is with us to stay, notwithstanding “the sky is falling” handwringing and other dire predictions that seem to accompany any new development that we do not really understand.

I lived through Y2K without a hitch despite the claims that the world was going to end because computers couldn’t cope with the change from 1999 to 2000. I remember that Nancy and I were staying at an expensive “boutique” hotel in San Francisco. I don’t remember the name nor why we were there, but it would have been an understatement to say that the room décor was overdone. The Arabian Nights were more modest.

In any event, we went to sleep in 1999 in what felt like several pounds worth of bedding, and awoke in 2000 none the worse for wear. The phones still worked, the clocks still worked, and the sun shone. Plus, having grown up in a world that never heard of computers, I was not terribly concerned about living without them if it came to that.

Nascent AI is permeating the legal profession. If you are not a good writer or simply not interested in writing a brief, AI will do it for you. And if it looks like the law may not be on your side, AI will make up cases to support you. So long as no judge or research attorney tries to look up your authority, you are home free. Well, “free” might not be the right word as you may end up paying sanctions, but I digress.

Those who tout artificial intelligence focus on its time-saving potential but, if it just makes stuff up, the fact that it can do so quickly is of little utility.

What do you do with expert witnesses who rely on AI in forming their opinions?

That, among other questions, was confronted in Matter of Weber in which a court in New York had to deal with a trust accounting involving property on Cat Island somewhere in the Bahamas. The general context of the dispute was whether the trustee treated the property correctly or hung on to it too long before selling, or at least I think that is what it was about as my boredom threshold was reached early on.

An expert identified as Mr. Charles Ransom testified for the objector to the effect that the property should have been sold earlier than it was. It was unclear to me what where Mr. Ransom’s expertise lay as he had no experience in real estate, but he testified as to damages for breach of fiduciary duty. The court ultimately rejected his testimony for a whole host of reasons, but also because he relied on artificial intelligence.

Apparently, Mr. Ransom had relied on Microsoft Copilot, “a large language model generative artificial intelligence chatbot.” He could not, however, remember what input he gave and, of course, had no idea which sources the chatbot relied upon. He, like me, had no idea how Copilot works or how it makes decisions. The court noted that “[The] [c]ourt has no objective understanding as to how Copilot works and none was elicited as part of the testimony.”

While, to me, that would have been a game-ending failure of proof, the court didn’t stop there but, essentially, called the chatbot as a witness. Using the court’s computer, the court asked Copilot to make the damage calculation. It came up with a different number than had Mr. Ransom. The court then used two other courthouse computers with the same question and came up with two more different numbers leaving it with four numbers which were close but not identical. Frustrated, the court asked Copilot questions not normally asked of an expert, at least on direct: (1) “Are you accurate?”; (2) “Are you reliable?”; and (3) “Are your calculations reliable enough for use in court?” The theme of the chatbot’s answers was that its information should be verified by experts, completing, I suppose, a useless circle.

But, have no fear.
No chatbots were used in the drafting of this article.

Richard W. Millar, Jr. is tired and retired. He can be reached at dickmillar9@gmail.com.