Artificial intelligence works better than human lawyers in accurately spotting risks in everyday business contracts.
That’s the conclusion of a landmark study pitting 20 experienced US-trained corporate lawyers against Tel Aviv based LawGeex, which makes artificial intelligence (AI) software for contract review and approval, in spotting issues in everyday contracts.
Both the lawyers and the LawGeex AI analyzed five previously unseen non-disclosure agreement contracts, containing 153 paragraphs of technical legal language under controlled conditions.
This is the first time that AI software has been tested with a typical task undertaken by lawyers on a daily basis.
The result: LawGeex software achieved a 94 percent accuracy rate at identifying risks in the NDAs. The average accuracy rate was 85% for the lawyers.
The highest performing lawyer in the study achieved 94% accuracy — matching the AI — while the lowest performing lawyer achieved an average 67% accuracy.
The challenge took the LawGeex AI 26 seconds to complete, compared to an average of 92 minutes for the lawyers. The longest time taken by a lawyer to complete the test was 156 minutes, and the shortest time was 51 minutes.
The experiment was done in conjunction with law school professors from Stanford, Duke and the University of Southern California and was overseen and administered by independent lawyer Christopher Ray. Prof. Yonatan Aumann from the department of computer science at Israel’s Bar-Ilan University served as a consultant.
“This experiment may actually understate the gain from AI in the legal profession,” said Gillian K. Hadfield, professor of law and economics at the University of Southern California.
“The lawyers who reviewed these documents were fully focused on the task: it didn’t sink to the bottom of a to-do list, it didn’t get rushed through while waiting for a plane or with one eye on the clock to get out the door to pick up the kids. The margin of efficiency is likely to be even greater than the results shown here.”
Hadfield said the research shows technology can help make contract management faster and more reliable, and free up resources in law firms.
The full findings can be download here.
More on News
