Legal Software Analytics: Humans Still Matter

legal software analytics

Legal software analytics works great, but the human touch is still needed.

Unless you have been stranded on a desert island for the past several years, you’ve undoubtedly seen countless articles, whitepapers and even court opinions discussing how legal software analytics such as document context analysis, near-dupe identification, predictive coding and Technology Assisted Review (TAR) has changed – and continues to change – the eDiscovery landscape.

Indeed, it’s not just our industry but the entire world that is seeing incredible leaps and bounds in artificial intelligence, from self-driving cars to advanced analysis engines that can pour over thousands of data points to help speed and improve medical diagnosis.

Yet, more and more, we’re hearing about how humans still matter, and in fact, how humans often make legal software analytics better.

In January, the New Yorker reported on how Netflix uses pretty amazing analytics to help determine what new original programming to develop (it’s truly a fascinating read).  But even with the sea of data Netflix analyzes, the final decisions still factor in a heavy dose of human analysis, namely in the form of Netflix’s CEO Ted Sarandos.

In his book The Signal and the Noise: Why Most Predictions Fail – but Some Don’t, author Nate Silver discussed how the National Weather Service has tracked and compared computer-only weather predictions versus those where humans review and contribute to those computer models.  The results, Silver reports, show that humans actually improve the accuracy of weather forecasts by up to 25% over computer-only predictions.

Just today, Wired reported on how the automotive supplier Delphi just completed a cross-country, 15-state, 3,400 mile test of a self-driving Audi Q5.  The article points out how quickly the technology is improving by contrasting that feat with a 2004 DARPA test where even the best self-driving car was only able to complete less than 8 miles of a 132-mile course, and that car got stuck and caught fire.

But even with such an amazing pace of advance across all industries, it’s still unlikely that we’ll completely be able to eliminate the driver anytime soon.  Even if and when we could potentially do so, it’s highly likely that humans simply won’t give up that x-factor that is the human element.

These lessons are not so different than those being learned by the legal industry when it comes to legal software analytics such as Technology Assisted Review (TAR).  Back in 2008, when there weren’t great commercial TAR solutions available, we created our own technology assisted review solution with exactly that in mind – keep the humans, with the whole of their knowledge, expertise and experience, at the center of that solution. By doing so, we saw how combining knowledgeable, experienced experts with the latest and greatest analytics resulted in a better outcome than either could accomplish alone.

The overriding point across all of these experiences seems to be that, while analytics technologies (including legal software analytics) continues to advance by leaps and bounds, and improve (or at least impact) nearly every portion of our lives, we must not forget the crucial role that humans have played, and will likely continue to play for quite some time, in ensuring that the answers that come out of the black box du jour are the best and most accurate possible.  

Read the updated article about Artificial Intelligence and legal document review and how the human element is critical to passing the scrutiny of the courts.