BIA is committed to ensuring ongoing quality service during the COVID-19 pandemic.View

AI Legal Document Review: Combining Human Effort and Machine Learning

AI legal document review

Read a more complete version of this article about AI and legal document review, in a great resource by Law.com, Cybersecurity Law & Strategy.

We hear a lot about the potential of artificial intelligence and machine learning to make the document review phase of the eDiscovery process faster and more accurate — and in many ways, the technology is living up to these promises.  AI Legal Document Review is here to stay.

But the human element remains a critical component, essentially creating a hybrid legal document review process that blends human and artificial intelligence (AI) to achieve the optimal results.

The legal industry has been revolutionized by AI – its ability to help humans quickly dig through large amounts of data, locate relevant information and make legal attorney review calls with a high degree of precision. Without the AI in legal document review, using the technological ability to organize files and data and present them in a way to identify similarities, trends, and outliers, attorneys would simply be unable to cost-effectively manage the sheer volume of data that is encountered today during the eDiscovery process.

Yet, AI is in many ways still in its infancy, and it’s important to realize that AI legal document review platforms utilizing the technology are heavily dependent on constant human interaction and training. As AI and machine learning techniques continue to develop, there will come a time when computers will be able to perform increasingly larger portions of projects with little or no human input, delivering a higher level of quality than humans can achieve on their own.

But for the foreseeable future, though, the human role in eDiscovery, and particularly in document review will be a vital part of the process, and a “hybrid” or combined technology-plus-people approach to eDiscovery will continue to be the most effective model, and one more and more being adopted by the Courts.

Let’s take a closer look at how human efforts and AI used in legal document review work together, and the ways in which each relies upon the other for performing legally defensible document review.

AI and Machine Learning

Machine learning, a subset of AI that has the ability to make predictions after undergoing initial human-driven training, is utilized in eDiscovery. Depending on how well it’s trained, a predictive machine-learning model can deliver faster, more accurate results than a human.

In eDiscovery, we use machine learning technology to wade through enormous data sets in search of relevant documents for legal matters. This process, called Technology Assisted Review (TAR), starts with a human reviewing a certain number of documents and coding them as either relevant or non-relevant.

Garbage-In, Garbage-out.

Typically, most competent document review services offered by eDiscovery providers today will include at least some TAR element as part of the managed review process.  When looking at those solutions, it is critical to ensure that the correct combination of elements – technology, people, and process – are aligned in any such service to ensure the success of the overall review effort.

Even with the newest technology, it is still the people involved on which it all hinges, especially AI legal document review. It is the latest example of one of the oldest adages in technology – computer systems tend to amplify the quality or defectiveness of the input (or “training” in TAR speak).  So, if the initial work is “garbage”, that’s exactly what the output will be too which is why humans are still a critical part of the process.

Power of the People

While a document reviewer’s subject matter expertise is an essential element of any TAR enabled review process, so are the reviewer’s technical experience and skill set, which matter just as much, if not more.  That’s why, at BIA, our legal document reviewers have mastered not just document review best practices and various subject matters, but also have become experts in a variety of different TAR and legal analytics technologies. Our legal document reviewers understand the impact and optimal use of those technologies, and that makes all the difference in the world.

That essential, hybrid blend of machine learning systems with the right technical and subject matter experts, has shown us time and time again to be the key to a truly successful TAR review project.  And the combination doesn’t just reduce costs, it also further bolsters accuracy, increases production speed, helps more quickly evaluate case issues and reduces the entire case timeline, making the overall legal dispute process much more efficient in all aspects.  Thus the use of AI in the legal document review process is necessary for modern litigation.

Of course, the human element is also critical to making the entire TAR process work to its best potential.  Indeed, Courts are more likely to embrace the use of TAR technologies in a case when shown that there is a highly skilled and experienced team backing up the AI technology.

The old ways aren’t dead.

To be clear, TAR doesn’t necessarily mean the end of the traditional search term approach.  Search terms still can be extremely valuable, especially in the early stages of culling and key document identification.  We often incorporate various types of keyword searches to locate relevant documents in a large data set. Among them are Boolean searches, which involve modifier words like AND, OR and NOT, and proximity searches, which examine how close certain search terms are to each other. 

In contrast with other forms of AI, there’s no training period required for that technology to work, as it is not dependent on the subject matter of a given case, so it can deliver results from the very beginning of the eDiscovery process. We expect this technology to become a common fixture in eDiscovery platforms, and additional experimentation with its capabilities will help it further show its value.

There are still shortcomings.

Even with the many advancements in AI-based technology used in eDiscovery software and legal technology, there are still areas that need attention.  For instance, tables and graphics are not easily scannable by automated systems. Excel spreadsheets, in particular, can be particularly problematic if they’re complex or contain a large amount of text. In cases like these, the human side of the equation – specifically, relying on experienced attorney reviewers becomes more important than ever.

Also, more advanced recognition algorithms such as identifying images or skin tone in graphics documents and objects in multi-media evidence are still lacking.  Though the technologies exist, there is still no integration into current legal document review platforms, and even when utilized, false positives are common.  Currently, people are still an essential element of ensuring that those graphical and multi-media file types are reviewed properly.

The human element is still key.

In the end, without the human element, TAR, machine learning, and text analytics are merely computer software programs, created by fallible humans, attempting to make the best decisions the algorithms allow during the document review process.

And because the results of those processes are used in legal proceedings to make decisions that have major impacts, not just on legal issues generally, but on the futures of people, governments and corporations, Courts are likely to continue to require the that the human element – experienced legal document reviewers and technology experts – continue to act as a quality control gateway for the technology the algorithms employed.

Humans also bring a number of special qualities to the document review process — among them, institutional knowledge, experience in specific legal areas and natural problem-solving skills — none of which can yet be replicated at the same level of quality by an algorithm.  And they bring backgrounds in areas such as intellectual property, antitrust, second requests, product liability, general commercial litigation, contract disputes, mass torts, governmental investigations or class actions — all of which is difficult or impossible for a computer algorithm to develop or replicate in a vacuum.

Thus, human intelligence still rules the ultimate result when it comes to ensuring an accurate and well-reviewed legal document review process.