Part III - The Role of Domain Experts in the Success of Machine Learning-Based Contract Analysis

Written by: David Curle and Steve Obenski

August 18, 2020
Next Gen Lawyers

6 minute read

This article is the third of a series of four that trace the history of contract automation and technology, and its future. See the earlier installments here and here.

For a few years in the late 2010s, legal publications were awash in hype that, with the rise of AI in law, “robot lawyers” were going to threaten human lawyers’ jobs. Thankfully, that hype has mostly subsided. As professionals have educated themselves on the capabilities as well as the limitations of the available technology, they have gained a growing appreciation for the role of legal domain expertise in the design and application of any legal AI system.

Domain expertise, on the part of both business professionals and software providers, is central to the success of any application of AI to extract value from contract data. Done right, machine learning does not replace the expertise of the professionals that use contract analysis software. The software amplifies that expertise by embedding the professionals’ knowledge in analytical tools, and applying that expertise in large-scale workflows.

Machine learning does not operate in a vacuum. Systems using ML are designed by humans.

There are two main approaches to designing an AI system. A common approach is to have a basic system that requires fine tuning, customer-by-customer, by technical experts often referred to as “data scientists.” This fine tuning is typically performed by data scientists contracted to customers, whose customization work is either required as part of an implementation period or offered for a fee as consulting services. Many successful systems are built in this way; companies can buy them from many of the large systems integrators and consulting firms, and also now from some of the large alternative legal services providers. However, these arrangements are not truly scalable; if requirements change, services are required to implement the changes.

The other, more powerful approach, is when the ML is leveraged to magnify, amplify, and scale the knowledge of human domain or subject matter experts.

This is the approach that Kira takes. Kira’s Quick Study capability enables subject matter experts to create machine learning models just by highlighting examples on screen. Users have two ways to use the the technology:

  • We know that many users of an AI system will want to get started quickly, so we include pre-built models created by our own Legal Knowledge Engineering team. Despite the word “engineering” in their title, members of this team do not have technical degrees; these are the experienced lawyers, accountants and other professionals that are responsible for Kira’s Built-In Intelligence. Using Kira’s Quick Study interface, they have built over 1,000 machine learning models (called smart fields) that ship with Kira “out of the box” and allow Kira to extract the most commonly used contract terms. The LKE team listens closely to customers to help select built-in smart fields that will get the most usage across our user base to help power the existing use cases, while constantly identifying and evaluating new ones. Their expertise is the built-in intelligence that allows Kira to go to work, without additional training, on day one.
  • The second source of domain expertise in the Kira system is of course the Kira customer base. Their knowledge of their own business needs and interests, their understanding of clients’ businesses or their own, and their sensitivity to clients’ risk tolerances and objectives, will frame and drive the way they apply Kira’s power to a given matter or project. They do not need any special machine learning training, and they do not need implementation consultants. As of this writing, Kira users had well over 15,000 of their own unique smart fields deployed on contract projects and processes, all around the world.

Of course, these two broad approaches can sometimes be combined, with technical customization offered alongside human training to bridge gaps (or perceived gaps) to get to final results. Kira avoids this. Our team of research scientists never does customization for any specific customer. Rather, they constantly evaluate new external technologies and create new proprietary approaches. Unlike the less-scalable approach of using implementation consultants for each customer, all of Kira’s ML enhancements are built right into the core Kira product and become available to all Kira customers, often with minimal or no changes to the user experience. Kira’s scientists also follow, and contribute to, the state of the art in machine learning across the world, and bring that evolving expertise directly into Kira products.

Our vision as a company is that AI in this space is actually done correctly when the expertise of real human experts is amplified, empowering them to do more with it. We are giving them “superpowers” to apply that expertise to new tasks and challenges that were not previously addressable. Machine learning lifts the constraints that hindered even the strongest legal experts from solving their contract-related challenges.

Many lawyers remain concerned about the impact of AI on legal careers, but properly leveraging machine learning requires lawyers to engage in the process of contract analysis at the top end of their skill level. The technology is more than capable enough to learn to mimic the judgment of a professional expert (and at a level of accuracy that can surpass expert humans’ ability to agree on what constitutes a specific clause and what does not). Rather than using their time in large-scale manual reviews, they are leveraging higher-level skills by adapting technology to a given problem, augmenting their own professional judgment with better tools. In the meantime, new forms of legal work have emerged for legally-trained professionals to help build the models and delivery platforms that buyers of legal services need.

And even as AI has come a long way in releasing the power of contract data in combination with the domain expertise of these participants, there is still plenty of work for human professionals in analyzing the results. Products like Kira can quickly and accurately extract specific types of terms and data from large sets of contracts, but the results still require human analysis to arrive at business recommendations or legal judgments. For example, AI can automate the process of identifying all of the contract terms that include automatic renewal provisions, but it can’t yet tell a client whether they should or should not try to terminate those contracts. And it will likely be a very long time before the technology can do this (and when that day arrives, there may be many debates on whether it should).

In the meantime, the technology will continue to progress in meaningful ways, better supporting the recommendations of human experts. Currently, most of the domain-expert-trainable systems only identify and classify concepts. They do not yet provide structured answers about those concepts. Enabling subject matter experts to directly train automated systems to understand those substantive answers is the next great leap in this story. What if the system could provide structured answers about the concepts that the professionals were seeking to find? What if that could be done without extensive customization and without data scientist skills? That will be the subject of the final article in this series.

Learn how your team can leverage Kira’s machine learning capabilities. Join our sessions at ILTA>ON, or book a meeting at our virtual booth (Aug 24-28).

Share this article:

Get the latest legal tech insights sent straight to your inbox.