Ask the Experts: Year-End Q&A with BIA

Ask the Experts: Year-End Q&A with BIA

What are those nagging eDiscovery questions you’ve had for a while but never got around to asking? What are those topics for which a quick online search doesn’t return satisfactory answers? Some questions—such as those about pricing—can even seem taboo and awkward to raise at any time. Additionally, 2020 has brought significant change and adjustment across all industries, and litigation, breaches, and tech issues have only increased across the board.

In this webinar, BIA tackles your eDiscovery questions and issues that arise across the entire EDRM. Our panel of experts handle these issues every day; hear them discuss legal holds, data identification and collection, digital forensics, data analytics, review hosting, managed document review, and a few curveballs, too.

In this Ask the Experts webinar, you will learn about:

  • how to approach data collection for remote workers and mobile devices, including smart watches
  • data preservation on international virtual cloud servers
  • how IT departments and in-house attorneys should collaborate when handling exiting employees
  • favorite analytics tools, and analytics tool usage mistakes
  • flat-rate document pricing versus hourly review pricing
  • …and more

Webinar Speakers

Watch the Webinar

Webinar Q&A

Select a question from below to read the answer given by our experts in the webinar. The full transcript is available below.

Legal Hold Questions

If we have Zoom meetings that we’ve recorded, how do I make sure they’re being preserved for a pending litigation?

So with respect to Zoom and other similar recording tools, let’s take a step back. If your company regularly uses Zoom or Teams or one of the other video conferencing tools, you’ll need to ask your groups if there’s a business reason to record the call. And if there isn’t a reason, then maybe perhaps you shouldn’t make that recording. And some of the tools have the ability for admins to turn the ability to record off. And if you’re faced with the triggering event that causes you to need to issue a legal hold, and you’re recording of a call is in the corporate system, you can more likely than not do your preservation there. However, if the recording is on a privately owned computer or system that isn’t controlled by your organization, we recommend that you do the preservation virtually immediately so that you don’t lose it along the way. And one way to determine whether or not you have any of these recordings is to ask your custodians through the use of a custodian questionnaire. And we find CQs to be invaluable across many many areas of the legal world. And there is one other consideration that I want to put on the table that I don’t think is asked often enough, and that is with the consent laws you need, and there’s dual consent requirements, and I think eleven jurisdictions in the US, to ask if it’s okay to do that recording, regardless of whether or not it has a valid business purpose.

Barry Schwartz, Esq., CEDS
Do I really need automated LH software? We have 3 small matters per year, sometimes none. Should I be managing my Legal Holds, or should my outside counsel do it for me?

We wrote a blog post on this question recently and no, automated software isn’t necessary as you can appropriately issue a hold to your custodians by first identifying them, informing them of what should be preserved, advise them why the hold was issued, let them know where the information should be preserved and let them know the duration of the hold, typically until released after the matter is completely final. In sending a manual hold notice, it is also advisable to send a Custodian Questionnaire along with it to aid in ferreting out the information you want. On the other hand, automated legal hold systems are not prohibitively expensive, and they have other uses in addition. For example, here at BIA, we have very little legal activity, but we use our automated system to track, as you can imagine, which of our employees are subject to client protective orders. We also use the system for HR purposes tracking trainings and other similar things. So, there are benefits beyond just legal hold in most systems.

Barry Schwartz, Esq., CEDS
I’m new to a company with a lot of back-up tapes and I’m concerned that historically, they’ve been used as sources of ESI for litigation purposes. How can I stop and is there a way I can claim them to be used for “disaster recovery only?”

Okay, so this question pertains to whether or not backup tapes are a burden, more or less. And with the most recent update to the federal rules, the burden question has more or less been addressed in those rules requiring a balancing act between the parties. And frankly, we find the need to go back to tapes is less and less than it has been in the more recent past. But for this question, I’ll pass it to Brian, who’s had significant experience dealing with BIA clients on the issue of accessing these backup tapes. Brian?

Barry Schwartz, Esq., CEDS

Well, thank you, Barry. Yeah, Barry is right. The occurrence of this issue is very, very rare these days. The federal rules really change the entire process, and you really have to make a showing that you’re likely to get some pertinent information from those backup tapes. Before I kind of go into that, though, one of the things I like to say is a precursor. Like Barry said, one of the best ways to avoid the Zoom meeting question is to tell your employees not to record Zoom or not to record meetings. Or if they record them to make sure they got their notes when they’re done with them, delete them if they don’t need them right away, and so you avoid the question altogether. The same kind of thing goes with tapes. In most organizations, or I should say most in many organizations today, backups are done differently. They’re done on the cloud; they’re done more automatically. The concept of backup tapes just doesn’t exist as much as it used to. And so one of the things that you should be thinking about is to the extent your organization has backup tapes at all. Is it time to get rid of them? Period. Is there any reason, business reason or regulatory reason, to keep those tapes around? Because you know, like, within—and this goes with any type of data—you’re hearing a lot of companies in the discovery space talk about this today; we’ve been talking about it for years, which is strategic data retention or strategic data deletion. However, you want to phrase it. The idea of keeping around the data that you need, or that the regulations require you to keep around. But don’t keep around anything else. And the same thing goes for tapes. For most companies, especially if you’re in a company where you don’t really use tapes anymore, you just have a bunch of these old tapes sitting on a shelf somewhere. You should make strategic decisions on whether or not you ever even need those. And if you don’t need those tapes, then get rid of them right away. And then you’ll never face this question.

But I do understand this particular issue because we had a client who at one point in time, their GC who wasn’t an eDiscovery person at all, didn’t really have any discovery knowledge, instructed their IT team instead of putting everybody on legal hold, that they were going to just do daily full backups of their email. And they did it for a period of about five or six years in a company with thousands, if not tens of thousands, of employees. Of which, at least a thousand of them were on some legal hold or involved in a case. And that created a disaster scenario because one of the things courts will look at is the purpose of the backup tapes. And in that case, the purpose was clearly for litigation, and it was a really, really bad idea and exposed them to potentially tens of millions of dollars of restoration. They had like over 30,000 backup tapes at one point, and it was just a complete and utter nightmare. And we helped them get through that process, stopped the bleeding, put a new process in place. And so it all comes down to the purpose. If you’re just using those backup tapes still for disaster recovery purposes, then there is a strong public policy that prevents the courts from making you go back to those. The only time you’re really going to have to go back to those tapes is if you don’t put a legal hold in place timely and properly in the first place. If you put a legal hold in place when a case first comes out, and you’re preserving data, it’s going to be very, very, very hard for somebody to make a strong case to go back to backup tapes and force you to restore. That said, even if you do, there are very good ways about doing selective testing that can help reduce those costs as well.

Brian Schrader, Esq.
How do I ensure a LH on ESI my custodians have on their personal computers/devices?

That’s a good question. First you need to find out which custodians have relevant data on personal devices and then determine which devices and on which systems that information resides. How do that: Custodian Questionnaires and Custodian Interviews. Note that CQs can be distributed simultaneously with your legal hold issuance, or as a standalone request at any time during the matter. The benefits of using an automated system to issue not only CQs but Legal Holds as well is that the responses to CQs comes back to you in a report format that allows comparison of the answers from custodian to custodian, and lets the custodian put in detailed information as to location and types of devices or systems they used. The responses will point you to mobile devices and the types. iOS or Android, cloud storage such as DropBox, iCloud; compilation tools such as Evernote. All the devices and locations need preservation and in this era of Working From Home has, as we know, caused the proliferation of device use and storage of data for convenience particularly for those organizations that do not have a robust IT infrastructure in place that directs and mandates the WFH employees methodology for data management.

Barry Schwartz, Esq., CEDS
With the surge of remove workers across the globe, what does this mean for legal holds and data preservation on international virtual cloud servers? And what about GDPR?

I’ll make it brief, so we can move on. With the surge of remote workers across the globe, what’s the requirement for legal holds? And how does cloud storage in GDPR fall under or into that realm? Well, legal holds can be issued to any custodian, at any time, in any location. And as far as cloud storage is concerned, one of the notions to keep in mind is that data is typically located in a geolocation, close to where the user resides. That is data in the US-based, for US-based individuals is typically located in the US. And for the US, that’s a potentially—well, it’s actually near certainty. And likewise, in the EU, custodian’s data is typically stored there. But remember, that’s not always the case because a US employee of an EU based company may have their data stored in the EU, and vice versa for EU based people. And with the GDPR in particular, the data access follows the individual, not the company. And that’s a tenant of the GDPR privacy rules. And with respect to accessing that data from an EU covered individual, there are two ways to get that information involved in a litigation matter. One is the direct consent of the custodian, and the other is showing a legitimate interest. And interestingly, it’s been a long time since GDPR has been in play now for two and a half years, almost three years. And the legitimate interest was just recently tested and reported on in a Dutch matter where it was said that the legitimate interest is a valid reason for producing data without the custodian’s consent.

Barry Schwartz, Esq., CEDS

If the custodian is overseas, but the data resides here on the server, does that implicate it?

That’s a tough question. And I don’t know that it has actually been tested in the courts either here or there. If US courts would have the tendency to say if it’s relevant to the case, admit it. However, with the EU rules, regulations, and the penalties associated with GDPR, if the data was introduced into US courts without the legitimate interests or the custodian’s permission, that could create a draconian fine for the company that produces that data. And so, as I said, I don’t think it’s been clearly tested yet.

Barry Schwartz, Esq., CEDS

Data Identification & Collection Questions

What should the IT department do when an employee is leaving the organization? As the in-house attorney, is it my job to guide them? How do I do that? 

We’ve written extensively on this in the past year, especially regarding how the new era of remote workers has changed this process. First and foremost, IT, Legal and HR should develop a process, either through a legal hold software platform or other internal centralized listing, where a current list of all employees subject to an existing litigation hold obligation is maintained so IT can make sure that no equipment or data is remediated, transferred or deleted without first checking that source to determine whether any legal hold obligations exist with respect to that employee. Second, many corporations have an onboarding process, but they don’t always have a well-defined off-boarding (or exiting employee) process. We recently updated our Exiting Employee Checklist that provides a recommended process for exiting employees that outlines this process from revoking system access to data remediation and much more. Finally, if the departing employee had routine access to sensitive data (i.e. senior management, product development, sales, accounting, etc.) or you suspect the employee may have acted improperly during their tenure or potentially took data with them as they left, you may want to have a forensic professional image their computer and other equipment for potential later investigation or other legal needs.

Adam Feinberg, CCFS, CEDS
When should we image computers as opposed to collecting live data?​

When it comes to most eDiscovery needs, imaging an entire computer is nearly always overkill. You wouldn’t copy every piece of paper in every filing cabinet in an office in reply to a document request; blindly imaging every custodian’s machine is the digital equivalent of just that. Ask yourself a simple question: do you simply need to preserve the custodian’s data, or do you need to conduct an investigation into what that custodian actually did with their computer? If it’s the latter, that you need to really dive into a deep investigation, then image the computer. Otherwise, standard data collection methods should be more than sufficient. If you have any doubt, consult an eDiscovery expert.

Adam Feinberg, CCFS, CEDS
Have you ever had to collect anything from a smart watch (FitBit, Apple Watch)? How about from construction wearables? Is this common – can we capture this from another place other than the device itself? ​

So the soft answer is yes. If you have a smartwatch, a Fitbit, an Apple watch, an Android type watch, generally that data is synced to your phone or computer. We’ve collected thousands of phones. So the answer is yes, we’ve collected that data. We can search that data that is collected. But oftentimes, that data is not the authoritative, complete set of information that exists about the use of that specific device. Oftentimes the applications on the device are stored in other places on cloud repositories. I’m a runner; I’m wearing my running watch, my Garmin. If you were to try and collect the information from my watch, you would only find probably 30 days’ worth of runs on there because it’s got limited memory, limited storage, and that data is synchronized up to my phone, to the Garmin Cloud, and also to Strava. So I would say that the other authoritative places would probably be the places to look for most, if not additional information.

The second part of the question is an interesting one. The construction wearable market is a newer market. There aren’t that many devices out there yet. There are a lot of companies that have some beta application of these things like smart hard hats, sensors that may go on people’s work clothes or in their boots, or even things that go on construction equipment like back loaders and dump trucks and what have you. Pretty much most of those devices synchronize again back to a system in the back end that would again be the most authoritative place to look for that data. Does anyone else want to jump in?

Adam Feinberg, CCFS, CEDS

Yeah, I was gonna say, I was just going to add there, Adam. Basically, kind of what you’re saying is that one of the best ways to do this is to make sure that the device itself is synchronized, right? That the most recent data is up into whatever application or cloud app or smartphone app, what have you, and then collect from that more definitive, more long-lasting resource as opposed to just collecting from the device itself. So the best practice is to make sure your most recent activity is synchronized and then collect from the most definitive, most comprehensive source. Would that be fair?

Brian Schrader, Esq.

Yes, I would agree. And again, any kind of collection effort you want to do what’s called data mapping. You want to speak to the custodian; you want to understand how they use a specific device. It doesn’t matter whether it’s a watch, a phone, a computer, or has access to any number of cloud resources. You want to work yourself through the use of various technology to understand how they use it and where they store the data and collect from the appropriate endpoint, I’ll call it. Again, the one that has the most information possible.

Adam Feinberg, CCFS, CEDS
Is it appropriate to rely on the BYOD policy of the company, even when they assure you that they do not have any more ESI on other personal devices?

I’ll take the first stab at this. This actually kind of goes hand in hand with the first slide, legal holds. It’s how do you deal with all this stuff? And this is much more important today because with everybody working remotely, or so many people working remotely, the use of personal devices has skyrocketed across the board. And so what we say is, it’s great to know that the company has that policy in place, but to use a phrase, I think it was Reagan who said, “Trust, but verify.” You know you can trust it, but the best way to handle that situation is to issue a custodian questionnaire. You can use SurveyMonkey if you want to do it. We don’t really recommend using free services like that because they’re not necessarily secure and private. They may be secure, but not necessarily private. Better to use some sort of a secure survey engine or custodian interviews. If it’s a handful, you can just give the custodians a call and walkthrough. And one of the things you want to address is this. The company had this policy; frankly, when we talk to companies about doing these things, we don’t tell them to remind the employee that they might be violating a policy. Don’t say—don’t start out your questionnaire by saying, Oh, the company forbids this, but let us know if you did it, because you’re getting people who are going to be like, “I didn’t violate the policy!” Just ask the question, you know: did you use these devices? Did you use any of your personal devices to access information? Whether that’s your cell phone, your iPad, your watch, you can even access your email with some smart home devices as well. And so the best way to do that is, even if the company has a policy, still ask the questions and issue the custodian questionnaire. There’s nothing better to really define the scope than a custodian questionnaire.

Brian Schrader, Esq.

I would add to that – bring your own device policies have their own issues. Number one, the end-user is bringing their own personal device. All of their lives are on those personal devices, and then when the time comes for litigation, the organization wants to dip into that personal device. That could be problematic. The person may not want to allow you to do it. Things may end up being more targeted than appropriate, depending on the specific situation. And if your policy doesn’t have some language in it that the end-user has to comply with those requests, you’re potentially looking for some trouble. I think we’ve all been involved just dealing with personal cell phones, and how much trouble that has become in the collection space, and in the trepidation of employees handing over those devices that again contain all their lives, their personal pictures, their personal communications, for litigation purposes. So you know it’s definitely problematic.

Adam Feinberg, CCFS, CEDS

Yeah, Adam. Let me interject here. This morning I was on a client call, and personal devices were in play. And we asked the question, “do you have a policy that allows for the company to collect those personal devices?”. Said, “no we don’t, but we need to have them.” So a very timely point.

Barry Schwartz, Esq., CEDS

Right. And then the offshoot of that is what happens when the user leaves the organization, and your ability to remediate the computer of any corporate material. Again, it is their personal device. You have no claim to their computer, but it may contain or continue to contain a treasure trove of your corporate secrets after that employee leaves. And if they leave, they can go right to a competitor, but you have no idea what’s on their computer because they brought their own device.

Adam Feinberg, CCFS, CEDS
Should my company image every computer and device for each and every exiting employee?​

Generally speaking, no. Our recently updated Exiting Employee Checklist and accompanying blog addresses this process, but essentially, it depends on the sensitivity of the individual’s role or whether you suspect wrongdoing. For example, you may want to image the computers and devices of senior managers, product developers, salespeople, senior accounting staff—essentially, those with access to especially sensitive data that, if they were to take it with them, could be especially risky or damaging to the future success of your company. Aside from those sensitive positions, any employee suspected of committing and wrongdoing (harassment, theft, improper disclosures, etc.) or that you fear may try to take data with them as they leave should also trigger an imaging process.

Adam Feinberg, CCFS, CEDS
My vendor says they de-duped; why am I still seeing duplicates in the review database?​

So, just like in the law, forensics have specific connotations. And a duplicate is defined as two electronic files that are the same by hash: digital hash or digital fingerprint. A calculation is made on both of those files using one or many fields to determine its exact digital fingerprint or hash. All of those criteria have to be met between two files in order for them to be a duplicate, an exact duplicate. Certain things won’t change hashes on a file, such as changing its name. That’s not in the calculation. But again, what’s inside the document is what matters. Two documents on the face of them, if you printed them out and held them both up and laid them on top of each other and you looked at them, they may look be exactly duplicate, but within them, there may be an extra space or a hidden character that you may not see visually that no longer makes those documents the exact duplicates. Lisa?

Adam Feinberg, CCFS, CEDS

And so where I usually see this is the documents have already gone through processing, they’ve been deduplicated. Deduplication is a process where it’s identifying the duplicate files, and it’s taking those files, and it’s suppressing them usually. But it’s also taking the metadata from them and putting it into other fields so that you can still track the fact that the document that’s sitting on the review tool has all the metadata that would have been attached to the ones you suppressed. That whole process is something that really doesn’t happen in the review tool. So once we’ve already deduplicated and removed the exact duplicates, the hash duplicates, we’ve got lots of tools that can help find what we call text duplicates. So two files that have the same information, the same words, but they don’t have the same hash. Usually, a good example of this is a Word document that you have printed to PDF. Chances are those are not going to have the same hash value, but clearly, they have the same text. You just printed it to PDF. Or an email that you’ve printed to PDF. So we’ve got lots of tools that can identify those, but we don’t really deduplicate them at that point because we can’t really just mix and match all of that metadata because they’re not exact duplicates anymore. They are different files. So we can identify duplicates that didn’t get deduplicated in processing, but we don’t really deduplicate on the review side.

Lisa Prowse, Esq., CEDS, RCE

And just to confirm, we is not BIA; we is the industry in general, right?

Barry Schwartz, Esq., CEDS

Yes, yes. There are a lot of tools. Just running searches for keyphrases a lot of times will help you pull out very similar documents. There are lots of documents where they will take all the words in the document and tell you all the other documents that have the exact same words. They aren’t duplicates at that point usually; they’re just what we call text duplicates. Or you can do near-duplicates, meaning that 80% o the words are the same, or 90% of the words are the same.

Lisa Prowse, Esq., CEDS, RCE

Digital Forensics Questions

What’s the advantage of imaging a HDD, if imaging and live data collection are both forensically sound?​

Live data collections target routine user data, like emails, Word and Excel documents, PowerPoint presentations, chat sessions and the like. A full device image will also preserve deleted data, system information, registry settings, user activity logs and more. So, if you just need to collect data to respond to a routine legal data request, a live data collection should be more than sufficient. However, if you need to really investigate a user’s activities, or if you suspect they improperly deleted data, then a hard drive image will help uncover those types of activities.

Adam Feinberg, CCFS, CEDS
How can I search 3 TBs of server data without collecting and processing it? ​

So, typically, in order to search any kind of data, you have to be able to index it first. That’s true of Windows search, or Spotlight on your computer, or if you’re trying to run a search against your organization or even run a search in SharePoint. Indexing has to be turned on somewhere in order to be able to run those searches; otherwise, it’s a very, very slow process because then where you’re searching from has to look at every document to see if it can find what it’s looking for. That said, for large data sets again, I keep on going back to data mapping. You can do a data assessment or triage on that data set. That assessment can often provide detailed knowledge about the type of data that is contained within the three terabytes on the server. Often we find that when dealing with large amounts of data on servers, not all the data is commonly processed. There could be a large amount of backup files, system files, program-related files, or multimedia files that just may not be relevant or potentially responsive to your case. So that assessment could be running file listings or file type listings on it just to understand what’s there, understand the scope of what’s there so that again you can start making an assessment of what might or might not be important. Looking at file listings shouldn’t be your end all be all necessarily, but it could be depending on the structure of data in that server share. So again, once the assessment is complete, it could provide you with additional information that you can your council can make to make an informed decision on what to possibly collect or what not to collect on the server. All of that information is just metrics to let you know how big your problem may be and what the cost might be of processing either the whole thing or being more targeted. That said, you should always speak to the custodians too. We know that data generally doesn’t end up there without any user interaction, and user interaction through the use of custodian questionnaires. You can find out who’s storing what where. How good, I use the term data hygiene or data structure, how they store their files, or groups of people store their data so that you can really drill into what you can capture or preserve.

Adam Feinberg, CCFS, CEDS
Does Zoom have embedded video transcripts like Microsoft Teams does? How is that data collected? What about Google Hangouts?

Most of the popular web-based systems, like Teams and Zoom, that have transcription features will store that data in the same place as the video recording itself is stored. So, any competent data collection process that includes the collection of those videos should also by default collect the supporting files, including the transcripts. The trick is to ensure that your provider can link the differing files together, as there is no inherent link within the data files themselves outside of those systems.

Adam Feinberg, CCFS, CEDS
Can mobile devices be collected remotely? Are we missing anything if we go that route?

Every engagement, every collection effort, we start off with data mapping. Not only speaking to the individual custodian, but also potentially speaking to their IT department to see if the device is under any kind of mobile device management, or MDM. It is possible that that is the case. If a device is managed under MDM, that poses challenges for collecting the device, being able to export data out of the device without specific permissions or authorization from that MDM system. That said, there is a possibility to remotely collect from iOS devices (that’s iPhones and iPads). But at the moment, again, from the phone directly, Android devices are much more difficult to collect remotely. iPhone collections typically require full device backups due to security measures that Apple has put in place both in the software and hardware of the devices to protect consumer data. However, once preserved, specific data such as text messages with specific individuals can be exported from the backups using forensic tools. Android devices, again, as I mentioned, are more hands-on just again because of the architecture. They require a lot more hands-on with the device to put it in special modes, apply specific settings to them, and the Android ecosystem. There are just hundreds of different kinds of phones there that all work a little bit differently, different features installed by the software, the hardware vendor specific to their devices, that makes it pretty difficult to do this stuff easily. Again, but that said, Android devices specific information can be extracted from those devices as opposed to a full; I’ll just use the Word backup or capture of an iOS device. iOS devices we have to capture the whole phone to get at text messages, whereas Android devices, when we can collect from them, we can target just text messages. So again, mileage will vary depending on the type of device, and specific operating version, and any secret sauce that the hardware vendor puts in software wise on that device.

Adam Feinberg, CCFS, CEDS

So you know, Adam, this kind of goes back to a proper scoping call at the beginning, right? Because you may not care about certain data points on the phone, but then you might also discover that yeah, the person that owns this phone does have the Words with Friends game, and potentially he’s using the chat feature inside that app to communicate with somebody about some insider training. So we keep drilling into the point of you ask the right questions at the beginning to facilitate a proper data preservation and collection and move into the review of that data. And cell phones are something that are not going to go away, or mobile devices, call them what you will. Showed my age there, sorry. But you have to ask the right questions, I think, is part of your answer, right?

Mark MacDonald, CEDS 

It’s always about asking the right question and listening to the answer because that answer could generate a few more questions for you to better understand what’s going on, and if the answer is past that initial question that’s being asked.

Adam Feinberg, CCFS, CEDS
Are there additional measures to take when a departing employee has been working remotely?

For those people who are interested in that, if you just Google “BIA exiting employee,” you’ll find the Exiting Employee Checklist that we just published that covers that topic, along with, like, literally step-by-step questions and steps you should take for an exiting employee, especially in the remote world. So search “BIA exiting employee” in Google, and you’ll get the full answer along with a checklist right there.

Brian Schrader, Esq.

Data Analytics & Review Hosting Questions

What are some of the biggest mistakes you see people making with analytics tools?​

I would start with: not having the right people running it. Simply because just because you have the tool does not mean you can just throw everything at it and it will work beautifully. And that kind of goes into also prepping the data properly, segregating the data properly, especially if we’re talking about like a TAR process. TAR works based on the text of the documents. To the extent that you’ve got documents where they don’t have any text, or they have a very small amount of text, emails where you’ve got just one line, or emails that literally just say ‘see attached’. When you run that through a TAR process, you’re really saying, let’s say you code that document, you tag that document as responsive, you’re telling the TAR engine find every document that basically just says ‘see attached’, and make it responsive, which is probably not what you’re expecting to do. So just being able to segregate out the data properly, and really anyone not coding. For a long time, we said you need to code on the four corners of the document, but even then, that’s even a little harder to understand. Because really what we need you to do is just look at the text. Anything outside of the text that you know about that document the analytics engine doesn’t know. So a lot of it is really understanding how the analytics works and working with it and not kind of working against it. And to that end, thinking that the analytics tool is going to do the review for you. You can throw half a million documents in the analytics tool, and its job is to help you identify documents and to help you segregate documents. Its job is not to do it for you. It still requires some work on both the vendor end and on the attorney review end. And that’s probably the biggest mistake is just thinking it’s going to do it for you and not knowing how to use it properly.

Lisa Prowse, Esq., CEDS, RCE

If a client gives you a list of search terms, do you rely just on that? Or do you still conduct your own search terms?

Basically: “Is keyword searching dead?” Or, “should we still use keyword searching?” Keyword searching is not dead yet. I don’t think it will ever be dead. Search terms are really good for finding something that you know is there. So, for example, if I want to know where my nearest Starbucks is, I’m going to Google Starbucks. That is a keyword, and that is what I’m looking for. But if I wanted to know all the coffee places around, searching just for the word Starbucks is only going to find me Starbucks. It’s not gonna find me the other coffee places. So to the extent that you’ve got a tool where you type in Starbucks, and it says okay, it seems like you’re looking for coffee places, do you want to look for other coffee places? That’s moving more into the conceptual searching. So whenever you’ve got keywords that you know will help you get to a place, help you get to documents you’re looking for, it’s great to start out with those. They’re probably just not going to be the best way to approach the entire document review overall. It’s just one piece that you use. One important piece of that would be looking for privileged documents. Privileged documents really, in my opinion, do not do well in a TAR process mainly because TAR is based on the concept of the document. It’s looking for similar documents, not necessarily similar words, but similar concepts. And privilege is more structural than that. If you’ve got an email from an attorney that says, you know, gives an instruction to an employee, that document itself may be privileged solely because it’s coming from that attorney. If you’ve got the exact same sentence, but it comes from just another employee in the company, it’s not going to be privileged. And a TAR engine is going to look at both of those emails and think they’re basically the same email, even though one would be privileged and one would not. So using keywords to help identify certain documents, especially if you know exactly the documents you’re looking for, or to help you identify privileged names and stuff, keywords will always be important for stuff like that.

Lisa Prowse, Esq., CEDS, RCE

There are so many analytics tools out there. How are they different and how do I go about choosing? Does BIA have any favorites?​

That’s quite the loaded question, and really, very hard to answer in an industry where these tools are changing rapidly and new tools being introduced all the time. With respect to Machine Learning (or TAR), our favorites are Brainspace and NextLP. We also utilize Matter Analytics, Relativity Analytics and other platforms as well. The best practice is to consult an experienced and knowledgeable expert in this area before making any decision. Every case is different, and while we probably use Brainspace more than anything, that doesn’t mean it’s the best solution for every case. For example, if your case has a lot of video or audio files, there are other solutions not even mentioned in the list above that would be worth considering as well.

Lisa Prowse, Esq., CEDS, RCE
Opposing counsel continues to send us productions as PDFs and it’s driving me crazy – and costing a fortune to convert to TIFF and OCR. It’s also costing us time. I’m also concerned that we are missing metadata. Any guidance would be appreciated! ​

So yeah, you’ve got an opposing counsel who keeps producing in PDF. Really, there’s two ways to look at it. If you’ve got an opposing counsel who’s frankly not very sophisticated, and that’s the reason they’re producing in PDF, then you know maybe it’s something that you just kind of work with them with. But if you’ve got an opposing counsel that’s doing it on purpose, and we definitely have seen those, then it’s time to go a little further. The federal rules say that you need to produce your data in a useable format. So it’s pretty accepted that you should be promoting metadata, searchable text, et cetera. Going back to that unsophisticated opposing counsel, sometimes they’ll think that they are producing text as long as they’re producing searchable PDFs. Unfortunately, most review platforms really won’t do anything with searchable PDFs. You can go ahead and load a searchable PDF, but it’s still going to just be the PDF. You’ll need to OCR it. So to the extent that you think they’re doing it on purpose, then there are definitely rules both federally, and most states have similar rules saying that they can’t withhold the metadata and the text. If they’re not doing it on purpose and they just don’t quite know, then maybe it’s a little easier to work with them. And maybe the answer is simply getting them to agree that they’ll help get that metadata for you because some of the metadata is definitely important, and you would be right to be concerned that you’re missing metadata if you’re just getting the PDFs and not the metadata. The metadata can tell an awful lot of that story.

Lisa Prowse, Esq., CEDS, RCE

One of the things that we talk about with our clients is early on in a matter; get an ESI production protocol in place. And a production protocol will address the issues Lisa was just talking about, including which metadata fields are produced, what the format of the production is to be in. And it almost never calls for a PDF production, unless that was the native version of the file. But also, TIF imaging is the way to go. Excel documents are typically produced natively. So spelling all of those issues out, those concerns out, in the ESI protocol goes a long way to assuring both sides are receiving the information that they’re entitled to receive. And the best time to get a production protocol in place is at the early meet and confers related to the matter. And that is about the only place in the federal rules where they talk about these types of things is get them involved early on in the meet and confers.

Barry Schwartz, Esq., CEDS

Hey Barry, just to add on to that. First off, one quick thing. This had come up a couple of times where we’ve dealt with individuals producing PDFs and even non-searchable PDFs. And they’ve warned them, saying, look, you’ve got to do at least TIFS with OCR and metadata, that’s industry standard. Like you said, natives pretty much for Excels and sometimes PowerPoints and other types of files audiovisual. The production standard in the industry is pretty standardized these days. I mean it used to be the day we would be called on by clients to fight that all the time. Every single time somebody has produced an unusable or unsearchable PDF or just the PDF without metadata, and we’ve gone to bat on that subject, we’ve won every single time. Every-single-time. So there’s no question that it’s not, that if you’ve got somebody producing stuff that way you could basically tell them look, you can either be cooperative about it if you don’t know what you’re doing, glad to help or have our vendor help you do it. But if we have to go back to the court and force you to do it, you’re going to lose, and you’re going to get dinged for it as well, and maybe even sanction.

Brian Schrader, Esq.

Is printing ESI on paper ever acceptable for production? If so, can you give an example of that?

And the answer is that really, I mean, technically printing the TIF is kind of the same thing as printing to paper. But nobody really produces paper anymore. You still see it in really small cases, so that’s one thing. And like I’ve always said, Barry talked about agreeing with your opposing counsel on a production format. If opposing counsel agreed to produce all their documents chiseled out on a stone tablet, if they both agreed to it, they agreed to it. So you can always agree to anything you want to. But in general, producing the paper, producing boxes of paper, not only is it wasteful and extraordinarily more expensive, but it’s rarely ever done, and wouldn’t really be appropriate unless for some reason again a small little case, a small arbitration, something the parties agree to produce a handful of documents that way. Other than that, I would say that it would never be appropriate.

Brian Schrader, Esq.
When and why do I need a clawback agreement? Are they used more often these days? 

You should have a written eDiscovery protocol in every case, and that should always include a standard clawback provision. Indeed, we highly recommend that you develop a standard eDiscovery production protocol/stipulation, and then use that in every single case. We have a sample on our site that includes standard clawback provisions that you can use to build your own. In the end, it’s about protecting your clients, and especially because clawback provisions are rather boilerplate and don’t often change, there’s no reason not to include it in every one of your eDiscovery production agreements.

Lisa Prowse, Esq., CEDS, RCE
Am I ethically obligated to disclose that I used TAR to review and produce ESI?

It depends on what your ESI order says. So if you’ve got an ESI order that says both sides have agreed, they’re going to use TAR, and they’re going to disclose what TAR engine they’re using and what their recall and precision—if you’ve got an ESI order that says all that, then yes, you are absolutely ethically obligated to disclose it because you signed it. But to the extent that you have an ESI order that says you are going to produce documents that are responsive to the request for production, then all you are obligated to disclose is that you have reviewed documents and produced them. Attorneys have to sign, and they have to swear that the documents they’ve produced meet the criteria that has been requested and that it’s not a document dump, and you’re not purposely withholding documents. How you get there, it really doesn’t matter. And 20 years ago, nobody asked, “Did you have document reviewers open and read every single one of these emails, and did you have a document reviewer go through every single file folder?” It was just accepted that if you produced it, you were okay with what was in the document, and you were giving it to the other side because it met that requirement. Nothing has really changed in that. Again, to the extent that it’s not specified in the ESI protocol, there really is no ethical obligation to say that you’re using any type of analytics to help you get to the production.

Lisa Prowse, Esq., CEDS, RCE

Managed Review Questions

Is key word searching dead yet? Does analytics make searching obsolete?​

Just like we said, I don’t think it’s dead yet. I don’t think it will ever be dead. There’s really no replacing keywords. In particular, when you open up any of the analytics tools, most of them have places to use keyword searching. So the keyword still gets used. One place that I didn’t mention earlier that we use keywords still is what we call anti-proximity searches. So let’s say you’re looking for the word “privileged,” but you don’t want it to hit in the footers of every single email now that has a footer that says “Privileged and Confidential,” or some combination of those words. We can use what we call anti-proximity searches. So, find me the world “privileged,” not within zero words or one word of privileged and confidential in the footer. It’s stuff like that where we still use keywords. It’s also even in the very beginning of the case: it’s a very quick and easy way to get some idea of how many documents are even talking about the topic that you’re looking at. So, I don’t think it will ever be dead; I would just say it requires more training than just knowing how to type a word into Google. It’s a little bit more complex than that if you want to use it appropriately and correctly.

Lisa Prowse, Esq., CEDS, RCE
I keep hearing conflicting information whether flat-rate per document pricing is better than hourly. Which is more cost effective and why? 

Generally speaking, flat-rate approaches to managed document reviews are always the better option. First, it establishes a clear budget with no surprises. Second, the risk is on the provider, not you, should it take more effort than expected. Finally, with a flat-rate approach, you are not on the hook for any delays, extended QC processes, system outages and the like. Plus, flat-fee solutions will often build in analytics and other features as well.

Lisa Prowse, Esq., CEDS, RCE
I don’t have any experience managing document review – but we have a case where there will be almost 500K documents. My outside counsel has limited resources but has a vendor they’ve used before – any suggestions? ​

In the end, it’s your money, so don’t just blindly trust any solution without vetting them yourself. If your outside counsel has a preferred vendor with which they’ve worked before, by all means, speak with them. But you should also seek to develop your own relationship with a managed review provider to ensure you are getting the best product overall. Make them all present their proposed budget and solutions to you. Any competent provider should be able to present a clear, concise picture of their services, and why they are the best choice, that you can easily understand and compare, even if you don’t have much experience yourself. Indeed, a good indicator of a quality provider is whether they can clearly communicate to you their proposed approach and overall budget. If they can’t do that, move onto the next.

Lisa Prowse, Esq., CEDS, RCE
Should we be using email threading? ​

Absolutely. Email threading not only will help make the document review process go much faster, but it also ensures that various parts of email threads are coded uniformly. It can also significantly help your attorneys as they prepare for depositions and trials —ensuring that they see an entire conversation, not just pieces scattered throughout. On a related note, if you can get your opposing counsel to agree to produce only the most inclusive emails in any given thread (including any varying branches of those conversations), then you can significantly reduce the amount of documents that you’ll need to review and produce—and of course, that can have a big effect on overall costs, too!

Lisa Prowse, Esq., CEDS, RCE
Will AI eventually eliminate the need for humans in document review?​

Will AI eliminate the need for document review? I mean, to some extent, it already has. It may not eliminate 100% of human review, but it certainly reduced it significantly. We’re able to tackle reviews of million-plus documents—and not pages, documents—in less than three weeks with a team of six expert analytical reviewers. In the past, that’s something that easily would have taken you 40 or 50 reviewers for a couple months. Beyond that, you’ll always need humans who understand how to use the tools and can accurately divide the data up and choose which tools to use. So, will it eliminate the need for humans in document review? I’m gonna say a lot of it, yeah. But not 100%, but some of it absolutely.

Lisa Prowse, Esq., CEDS, RCE

I just want to make a point. You’re saying less people, but I think this is one of those things where you could say it’s going to become more of a quality, a question of quality versus quantity. It’s not grabbing a bunch of people and throwing them at a project. If you’re going to grab those fewer people, those fewer people have to have higher level of skills, better analytics understanding, all of that stuff. So it’s a higher quality of training and experience, but a much lower quality – quantity, sorry – of people.

Brian Schrader, Esq.

Not quality, quantity. You need a handful of experts and not a big team of non-experts, basically.

Lisa Prowse, Esq., CEDS, RCE

And Lisa, let me add one point here on reviewers. With privilege review, typically, we see that we do need to have eyes on many of the documents.

Brian Schrader, Esq.

Miscellaneous Questions

If a law firm is hired to conduct collection, processing, review, and production but other law firms were hired to review that same information: Is it okay to suggest to the law firm that they should host the data (knowing that they have the capability to host it) or should one suggest that they inquire for a 3rd party because of risking exposing the law firm network to outside parties regardless of common interests?

In this scenario, both providers would be law firms, so each would have to be comfortable supporting outside users, which not all are prepared to or comfortable with doing. We always tell our clients that the more they can centralize their data, the easier it is to protect their data and control their costs. If you have one firm (or vendor) already holding your data, and you then bring in a second to essentially do the same, then you’ve just doubled your risk exposure and potentially your costs. And that goes even more so if you use different firms and vendors to collect, process and host your data across different cases too. For nearly two decades, we’ve been helping our clients centralize everything in a single, centralized solution. Instead of letting each new firm take your data and then hand it off to their own vendors (and those vendor’s subcontractors), you should develop a direct relationship with a full-service eDiscovery provider who can maintain a single, vetted and protected repository for all your legal data needs. By doing so, not only will it be much easier for you to ensure your data is protected (and you’ll always know where it’s all located), but you’ll be able to leverage your legal spend across all your legal needs, which can have a dramatic impact on lowering your overall costs too!

Brian Schrader, Esq.
How fast do you see AI deployment behind a client’s firewall?

Generally, this is not something that’s being done much at all today, at least with respect to eDiscovery. That’s because data generally needs to be extensively processed and prepared for the AI solutions, and that’s just work that most companies don’t want to take on themselves. To put it one way, a recent client said they don’t want to be involved in the sausage-making.

Brian Schrader, Esq.
If you are running out of time to prepare for the meet and confer and the amount of time is a day from now. What can you do to prepare when you have barely been brought up as a Project Manager. Would you still initiate a data map or would you talk to the client to better understand the locations of data and potential key custodians?

First, we’d recommend that you approach the opposing party and ask to extend the time. Most reasonable counsel, especially in today’s environment, will give you that time. In the end, it would be much more effective overall to ensure that all parties are prepared to have a production Meet-and-Confer session than to waste their time (and their client’s money). But, if you are truly stuck, then your best approach would to be talking to critical IT resources and walking through the various systems they use generally. Data mapping can take time, and if you’re truly in a crunch, you likely won’t have time to do that, so talking to everyone you can in the time you have available is the better solution.

Brian Schrader, Esq.
What information can you obtain from social media? I am talking about, if you go on a social media account what can you obtain (if any) information from just looking up the social media account without forensic/ exporting process?

We highly recommend that you don’t do social media investigations like this. Like most aspects of eDiscovery, such a do-it-yourself approach, especially if not done with proper knowledge and expertise, will do you more harm than good. Also, many platforms will tell the user who viewed their profiles, and that can cause some to start “cleaning up” their profiles before you can actually preserve that data. Sure, they might get caught and sanctioned for destroying evidence, but you still could lose on preserving critical information that could win your case, and it’d be very hard to prove what you saw if it’s gone before you can preserve it. A properly conducted social media investigation is one of the least expensive things you can do in eDiscovery, so it’s very much worth the cost to bring in an expert from the outset.

Brian Schrader, Esq.
What are the potential ramifications of inconsistent privilege determinations?

Different courts will come out differently on this, and every ruling tends to be very subjective to the facts of the individual case. The more the opposing party can show that the inconsistencies were the result of subpar efforts (or worse, recklessness), the more likely a court will find the privilege claims waived. Analytics and TAR processes can immensely help in ensuring that privilege calls are made consistently. You can bet that your opposing counsel will look for unredacted versions of documents you’ve redacted or produced versions of documents you’ve identified as privileged. And as the old saying goes, even if you can clawback those documents, once the toothpaste is out of the tube, the damage is often done regardless.

Brian Schrader, Esq.
What analytics work best for multi-language matters?

Brainspace, our preferred platform, supports a number of foreign languages, as do both our primary review platform offerings, Relativity and Reveal.

Brian Schrader, Esq.
What are some of the better options for auto-redaction? How would you QC such efforts (i.e. what are your best practices around that function?)

When it comes to auto-redactions, our favorite tool is Blackout, which is available as a plugin for Relativity, so it works within the platform (they are also developing the product for use outside Relativity). It provides for quick, comprehensive automatic redaction of images and files, including natively in Excels, as well as information not visible in the files themselves (i.e. metadata, notes, etc.). There are also built-in QC functions that enable a QC person to review, approve, reject and/or override the automated redactions.

Brian Schrader, Esq.
Where is the pricing moving for processing and hosting in 2021?

After years of commoditization in the data processing and hosting areas, prices have largely standardized and leveled off, and many industry pricing surveys have shown that to be true. The best way to ensure you have the best pricing is to centralize on a single, full-service eDiscovery provider. That way you can create a clear, repeatable and standardized process, which itself will lower both your internal and external costs, and allow you to leverage your legal spend across all your legal and regulatory matters – and even beyond. That said, what you will likely see in the place of price reductions in enhanced offerings with no additional costs.

Webinar Transcript

Mike: Hello, everyone, and welcome to the webinar channel of the Association of Certified eDiscovery Specialists. My name is Mike Quartararo, and I am the president of ACEDS. We are joined today by our fabulous partner BIA for an ask the expert year-end Q&A webinar. We’ll get to introductions in a minute. 

BIA, of course, is one of our longest-standing partners. Of course, we’re grateful for their continued support. BIA is a trusted leader in eDiscovery and digital forensics for two decades, managing projects across the eDiscovery life cycle. One of the cool things about BIA is that nearly all of their personnel and all of their project management team are CEDS certified. I know it’s a point of pride for them, and frankly, a great highlight of their commitment to professionalism and service to the eDiscovery community. You can learn more about BIA at biaprotect.com. 

Alright! Before we get started, we love questions. In fact, this particular webinar is dependent upon questions; it’s an ask the expert webinar. So please ask your questions using the webinar by using the Q&A widget on the bottom of your screen. 

Alright, so that we can get started, I will turn it over to Mark MacDonald at BIA. Mark? Please take it away. 

Mark: Thank you very much, Michael. And thanks to everybody for attending our webinar today. As we close out the year, we wanted to change things up a little bit here. So the idea with today’s webinar was that we wanted to engage the audience by asking questions to you the audience and take turns at answering them with our table of experts. We have lots of great questions, and so we’re going to get started really soon. We’ll do a quick round table and some intros. 

As we move through the slides, Peter Bruce from ACEDS will be polling the audience so that you can rank the questions on that slide in the order that we’ll answer them. So each slide we’ll devote about ten minutes to, and we’ll try to answer at least the first three questions for shorter questions. Of course, if we can, we’ll answer them all. Any questions we don’t get to will become part of our BIA blog, and you’ll all get an email when those are up. Let’s get started with some quick intros and then dive into questions. But we hope you’ll take away some knowledge and maybe a new perspective on how you handle your eDiscovery based on some of the information that we’ll share today. 

So today, we’ve got Brian Schrader. Brian’s a technologist and an attorney, having testified as an expert in many high profile cases since BIA’s founding back in 2002. Adan Feinberg is our EVP of services. He leads our digital forensics operation, and all of our professional services teams report up to Adam. Barry Schwartz leads our advisory team, helps our clients with all aspects of the EDRM and best practices from legal holds straight through production. And Lisa Prowse is a Relativity certified expert. She’s our CVP of litigation technology and managed document review. And my name is Mark MacDonald. I’m an SVP of business development here at BIA. 

As we approach the slides, we are going to give a one-minute time frame so that you guys can begin to vote. Click through our slides here – oh, thank you very much, it was magic. Our first set of questions is starting at the left-hand side of the EDRM around legal hold. And what we have are five questions. I’ll read them to you, and then in that window, you’ll be able to pick which question you want us to answer first. So question number one – if we have Zoom meetings that we’ve recorded, how do I make sure they’re being preserved for pending litigation? Question number two – do I really need an automated legal hold software? We have three small matters per year, sometimes none. Should I be managing my legal holds, or should I have my outside counsel do it for me? Number three – I’m new to a company with a lot of backup tapes, and I’m concerned that historically they’ve been used as sources for ESI for litigation purposes. How can I stop, and is there a way I can claim them to be used for disaster recovery only? Number four – how do I ensure legal hold on my ESI for custodians who have on their personal computers and devices? And question number five – with the surge of remote workers across the globe, what does this mean for legal holds and data preservation on international virtual cloud servers? And what about GDPR? So Peter can show our ranking window. And look at that! Already some quick first results. Did our minute expire? Are we ready? Or do we want to refresh that one more time? Let’s see. Looks like we’ve got a dead tie for question one, three, and five. How about that! Barry Schwartz, this is your territory, but other experts feel free to chime in. And let’s just start with question number one, Barry. 

Barry: Okay. Thank you and welcome everyone. So with respect to Zoom and other similar recording tools, let’s take a step back. If your company regularly uses Zoom or Teams or one of the other video conferencing tools, you’ll need to ask your groups if there’s a business reason to record the call. And if there isn’t a reason, then maybe perhaps you shouldn’t make that recording. And some of the tools have the ability for admins to turn the ability to record off. And if you’re faced with the triggering event that causes you to need to issue a legal hold, and you’re recording of a call is in the corporate system, you can more likely than not do your preservation there. However, if the recording is on a privately owned computer or system that isn’t controlled by your organization, we recommend that you do the preservation virtually immediately so that you don’t lose it along the way. And one way to determine whether or not you have any of these recordings is to ask your custodians through the use of a custodian questionnaire. And we find CQs to be invaluable across many many areas of the legal world. And there is one other consideration that I want to put on the table that I don’t think is asked often enough, and that is with the consent laws you need, and there’s dual consent requirements, and I think eleven jurisdictions in the US, to ask if it’s okay to do that recording, regardless of whether or not it has a valid business purpose. 

And so question three was the next one, Mark? 

Mark: Yeah.

Barry: Okay, so this question pertains to whether or not backup tapes are a burden, more or less. And with the most recent update to the federal rules, the burden question has more or less been addressed in those rules requiring a balancing act between the parties. And frankly, we find the need to go back to tapes is less and less than it has been in the more recent past. But for this question, I’ll pass it to Brian, who’s had significant experience dealing with BIA clients on the issue of accessing these backup tapes. Brian? 

Brian: Well, thank you, Barry. Yeah, Barry is right. The occurrence of this issue is very, very rare these days. The federal rules really change the entire process, and you really have to make a showing that you’re likely to get some pertinent information from those backup tapes. Before I kind of go into that, though, one of the things I like to say is a precursor. Like Barry said, one of the best ways to avoid the Zoom meeting question is to tell your employees not to record Zoom or not to record meetings. Or if they record them to make sure they got their notes when they’re done with them, delete them if they don’t need them right away, and so you avoid the question altogether. The same kind of thing goes with tapes. In most organizations, or I should say most in many organizations today, backups are done differently. They’re done on the cloud; they’re done more automatically. The concept of backup tapes just doesn’t exist as much as it used to. And so one of the things that you should be thinking about is to the extent your organization has backup tapes at all. Is it time to get rid of them? Period. Is there any reason, business reason or regulatory reason, to keep those tapes around? Because you know, like, within—and this goes with any type of data—you’re hearing a lot of companies in the discovery space talk about this today; we’ve been talking about it for years, which is strategic data retention or strategic data deletion. However, you want to phrase it. The idea of keeping around the data that you need, or that the regulations require you to keep around. But don’t keep around anything else. And the same thing goes for tapes. For most companies, especially if you’re in a company where you don’t really use tapes anymore, you just have a bunch of these old tapes sitting on a shelf somewhere. You should make strategic decisions on whether or not you ever even need those. And if you don’t need those tapes, then get rid of them right away. And then you’ll never face this question. 

But I do understand this particular issue because we had a client who at one point in time, their GC who wasn’t an eDiscovery person at all, didn’t really have any discovery knowledge, instructed their IT team instead of putting everybody on legal hold, that they were going to just do daily full backups of their email. And they did it for a period of about five or six years in a company with thousands, if not tens of thousands, of employees. Of which, at least a thousand of them were on some legal hold or involved in a case. And that created a disaster scenario because one of the things courts will look at is the purpose of the backup tapes. And in that case, the purpose was clearly for litigation, and it was a really, really bad idea and exposed them to potentially tens of millions of dollars of restoration. They had like over 30,000 backup tapes at one point, and it was just a complete and utter nightmare. And we helped them get through that process, stopped the bleeding, put a new process in place. And so it all comes down to the purpose. If you’re just using those backup tapes still for disaster recovery purposes, then there is a strong public policy that prevents the courts from making you go back to those. The only time you’re really going to have to go back to those tapes is if you don’t put a legal hold in place timely and properly in the first place. If you put a legal hold in place when a case first comes out, and you’re preserving data, it’s going to be very, very, very hard for somebody to make a strong case to go back to backup tapes and force you to restore. That said, even if you do, there are very good ways about doing selective testing that can help reduce those costs as well. 

Barry: Thanks, Brian. 

Brian: Last question on that list? 

Barry: I’ll make it brief, so we can move on. With the surge of remote workers across the globe, what’s the requirement for legal holds? And how does cloud storage in GDPR fall under or into that realm? Well, legal holds can be issued to any custodian, at any time, in any location. And as far as cloud storage is concerned, one of the notions to keep in mind is that data is typically located in a geolocation, close to where the user resides. That is data in the US-based, for US-based individuals is typically located in the US. And for the US, that’s a potentially—well, it’s actually near certainty. And likewise, in the EU, custodian’s data is typically stored there. But remember, that’s not always the case because a US employee of an EU based company may have their data stored in the EU, and vice versa for EU based people. And with the GDPR in particular, the data access follows the individual, not the company. And that’s a tenant of the GDPR privacy rules. And with respect to accessing that data from an EU covered individual, there are two ways to get that information involved in a litigation matter. One is the direct consent of the custodian, and the other is showing a legitimate interest. And interestingly, it’s been a long time since GDPR has been in play now for two and a half years, almost three years. And the legitimate interest was just recently tested and reported on in a Dutch matter where it was said that the legitimate interest is a valid reason for producing data without the custodian’s consent. And so back to you, Mark. 

Mark: Thank you, Barry. There’s a follow up to that question. If the custodian is overseas, but the data resides here on the server, does that implicate it? And you’ve got another minute and a half left. 

Barry: That’s a tough question. And I don’t know that it has actually been tested in the courts either here or there. If US courts would have the tendency to say if it’s relevant to the case, admit it. However, with the EU rules, regulations, and the penalties associated with GDPR, if the data was introduced into US courts without the legitimate interests or the custodian’s permission, that could create a draconian fine for the company that produces that data. And so, as I said, I don’t think it’s been clearly tested yet. 

Mark: Sorry for the curveball, but you took a great swing at it. Nice job. 

Barry: We tried. 

Mark: We’ve got a minute left on this slide, but I think for the interests of moving things along, let’s move to the next one. And so this will be on data identification and collection. And Adam Feinberg, you are the go-to for this one, but again, any other expert, feel free to chime in. So Peter if you’ll start the countdown clock at a minute? I’ll start reading our questions. Number one – what should the IT department do when an employee is leaving the organization? As the in-house attorney, is it my job to guide them? If so, how do I do that? Number two – when should we image computers as opposed to collecting live data? Number three – have you ever had to collect anything from a smartwatch (Fitbit, Apple Watch, or others)? How about from construction wearables? Is this common? Can we capture this from another place other than the device itself? Number four – should my company image every computer and device for each and every exiting employee? Number five – my vendor says they de-duped, but why am I still seeing duplicates in the review database? I might tap Lisa for that one as well. That was probably more than a minute, so Peter, we have some poll results? Whoa, look at that. I think we have a winner. Alright. Let’s start with number three, Adam. 

Adam: Let’s bring up that slide so I can see it, please. So, Mark, I’ll take have you ever tried to collect anything from a smartwatch for 100, please. 

So the soft answer is yes. If you have a smartwatch, a Fitbit, an Apple watch, an Android type watch, generally that data is synced to your phone or computer. We’ve collected thousands of phones. So the answer is yes, we’ve collected that data. We can search that data that is collected. But oftentimes, that data is not the authoritative, complete set of information that exists about the use of that specific device. Oftentimes the applications on the device are stored in other places on cloud repositories. I’m a runner; I’m wearing my running watch, my Garmin. If you were to try and collect the information from my watch, you would only find probably 30 days’ worth of runs on there because it’s got limited memory, limited storage, and that data is synchronized up to my phone, to the Garmin Cloud, and also to Strava. So I would say that the other authoritative places would probably be the places to look for most, if not additional information. 

The second part of the question is an interesting one. The construction wearable market is a newer market. There aren’t that many devices out there yet. There are a lot of companies that have some beta application of these things like smart hard hats, sensors that may go on people’s work clothes or in their boots, or even things that go on construction equipment like back loaders and dump trucks and what have you. Pretty much most of those devices synchronize again back to a system in the back end that would again be the most authoritative place to look for that data. Does anyone else want to jump in?

Brian: Yeah, I was gonna say, I was just going to add there, Adam. Basically, kind of what you’re saying is that one of the best ways to do this is to make sure that the device itself is synchronized, right? That the most recent data is up into whatever application or cloud app or smartphone app, what have you, and then collect from that more definitive, more long-lasting resource as opposed to just collecting from the device itself. So the best practice is to make sure your most recent activity is synchronized and then collect from the most definitive, most comprehensive source. Would that be fair? 

Adam: Yes, I would agree. And again, any kind of collection effort you want to do what’s called data mapping. You want to speak to the custodian; you want to understand how they use a specific device. It doesn’t matter whether it’s a watch, a phone, a computer, or has access to any number of cloud resources. You want to work yourself through the use of various technology to understand how they use it and where they store the data and collect from the appropriate endpoint, I’ll call it. Again, the one that has the most information possible. 

Brian: Sounds good. And there’s a question that came in as we were asking this, which I think really kind of goes hand in hand with this. The question is – is it appropriate to rely on the BYOD policy of the company, even when they assure you that they do not have any more ESI on other personal devices? I’ll take the first stab at this. This actually kind of goes hand in hand with the first slide, legal holds. It’s how do you deal with all this stuff? And this is much more important today because with everybody working remotely, or so many people working remotely, the use of personal devices has skyrocketed across the board. And so what we say is, it’s great to know that the company has that policy in place, but to use a phrase, I think it was Reagan who said: “trust but verify.” You know you can trust it, but the best way to handle that situation is to issue a custodian questionnaire. You can use SurveyMonkey if you want to do it. We don’t really recommend using free services like that because they’re not necessarily secure and private. They may be secure, but not necessarily private. Better to use some sort of a secure survey engine or custodian interviews. If it’s a handful, you can just give the custodians a call and walkthrough. And one of the things you want to address is this. The company had this policy; frankly, when we talk to companies about doing these things, we don’t tell them to remind the employee that they might be violating a policy. Don’t say—don’t start out your questionnaire by saying, Oh, the company forbids this, but let us know if you did it, because you’re getting people who are going to be like, “I didn’t violate the policy!” Just ask the question, you know: did you use these devices? Did you use any of your personal devices to access information? Whether that’s your cell phone, your iPad, your watch, you can even access your email with some smart home devices as well. And so the best way to do that is, even if the company has a policy, still ask the questions and issue the custodian questionnaire. There’s nothing better to really define the scope than a custodian questionnaire. 

Adam: I would add to that – bring your own device policies have their own issues. Number one, the end-user is bringing their own personal device. All of their lives are on those personal devices, and then when the time comes for litigation, the organization wants to dip into that personal device. That could be problematic. The person may not want to allow you to do it. Things may end up being more targeted than appropriate, depending on the specific situation. And if your policy doesn’t have some language in it that the end-user has to comply with those requests, you’re potentially looking for some trouble. I think we’ve all been involved just dealing with personal cell phones, and how much trouble that has become in the collection space, and in the trepidation of employees handing over those devices that again contain all their lives, their personal pictures, their personal communications, for litigation purposes. So you know it’s definitely problematic. 

Barry: Yeah, Adam. Let me interject here. This morning I was on a client call, and personal devices were in play. And we asked the question, “do you have a policy that allows for the company to collect those personal devices?”. Said, “no we don’t, but we need to have them.” So a very timely point. 

Adam: Right. And then the offshoot of that is what happens when the user leaves the organization, and your ability to remediate the computer of any corporate material. Again, it is their personal device. You have no claim to their computer, but it may contain or continue to contain a treasure trove of your corporate secrets after that employee leaves. And if they leave, they can go right to a competitor, but you have no idea what’s on their computer because they brought their own device. 

Brian: Or their phone, or their watch, or whatever. So Mark, do we have time to answer any more of these questions, or do we need to move on to the next slide? 

Mark: We have one minute and six seconds left on this slide. Maybe question five real quick for Lisa about deduplication? 

Lisa: Adam and I spoke about this yesterday. I’d like the forensic part first? 

Mark: You can totally do that!

Adam: So, just like in the law, forensics have specific connotations. And a duplicate is defined as two electronic files that are the same by hash: digital hash or digital fingerprint. A calculation is made on both of those files using one or many fields to determine its exact digital fingerprint or hash. All of those criteria have to be met between two files in order for them to be a duplicate, an exact duplicate. Certain things won’t change hashes on a file, such as changing its name. That’s not in the calculation. But again, what’s inside the document is what matters. Two documents on the face of them, if you printed them out and held them both up and laid them on top of each other and you looked at them, they may look be exactly duplicate, but within them, there may be an extra space or a hidden character that you may not see visually that no longer makes those documents the exact duplicates. Lisa? 

Lisa: And so where I usually see this is the documents have already gone through processing, they’ve been deduplicated. Deduplication is a process where it’s identifying the duplicate files, and it’s taking those files, and it’s suppressing them usually. But it’s also taking the metadata from them and putting it into other fields so that you can still track the fact that the document that’s sitting on the review tool has all the metadata that would have been attached to the ones you suppressed. That whole process is something that really doesn’t happen in the review tool. So once we’ve already deduplicated and removed the exact duplicates, the hash duplicates, we’ve got lots of tools that can help find what we call text duplicates. So two files that have the same information, the same words, but they don’t have the same hash. Usually, a good example of this is a Word document that you have printed to PDF. Chances are those are not going to have the same hash value, but clearly, they have the same text. You just printed it to PDF. Or an email that you’ve printed to PDF. So we’ve got lots of tools that can identify those, but we don’t really deduplicate them at that point because we can’t really just mix and match all of that metadata because they’re not exact duplicates anymore. They are different files. So we can identify duplicates that didn’t get deduplicated in processing, but we don’t really deduplicate on the review side. 

Barry: And just to confirm, we is not BIA; we is the industry in general, right? 

Lisa: Yes, yes. There are a lot of tools. Just running searches for keyphrases a lot of times will help you pull out very similar documents. There are lots of documents where they will take all the words in the document and tell you all the other documents that have the exact same words. They aren’t duplicates at that point usually; they’re just what we call text duplicates. Or you can do near-duplicates, meaning that 80% o the words are the same, or 90% of the words are the same. 

Mark: Right. Thank you. That was great answers to a question that I think haunts a lot of people both on the service provider side, as well as the client side. We went a little bit over on that slide, but I think that was valuable. Okay, next slide on digital forensics questions. Start our timer. Number one – what’s the advantage of imaging a hard drive – 

Brian: Sorry, I just got confused there for a second. Between the collection and the digital forensics. Go ahead, Mark. 

Mark: Alright. Question number one – what’s the advantage of imaging a hard drive if imaging and live data collection are both forensically sound? Two – how can I search three terabytes of serve data without collecting and processing it? Three – does Zoom have embedded video transcript like Microsoft Teams does? How is that data collected? What about Google hangouts? Four – can mobile devices be collected remotely? Are we missing anything if we go that route? Number five – are there additional measures to take when a departing employee has been working remotely? So similar themes to answer from a different perspective. Again, Adam, this is your world. But I’m sure that Brian is probably going to have some insight as well.

Peter: Alright, so we’re waiting on number four. Oh boy. 

Mark: Our audience is in sync. Alright, number four, mobile device questions. We see a lot of these. BIA was formed as a forensics company, and we have digital DNA, we like to say. So we do a lot of computer forensics work as well as our eDiscovery stuff. So Adam? The floor is yours again. 

Adam: Thank you. So again, just as I said on the previous slide, every engagement, every collection effort, we start off with data mapping. Not only speaking to the individual custodian, but also potentially speaking to their IT department to see if the device is under any kind of mobile device management, or MDM. It is possible that that is the case. If a device is managed under MDM, that poses challenges for collecting the device, being able to export data out of the device without specific permissions or authorization from that MDM system. That said, there is a possibility to remotely collect from iOS devices (that’s iPhones and iPads). But at the moment, again, from the phone directly, Android devices are much more difficult to collect remotely. iPhone collections typically require full device backups due to security measures that Apple has put in place both in the software and hardware of the devices to protect consumer data. However, once preserved, specific data such as text messages with specific individuals can be exported from the backups using forensic tools. Android devices, again, as I mentioned, are more hands-on just again because of the architecture. They require a lot more hands-on with the device to put it in special modes, apply specific settings to them, and the Android ecosystem. There are just hundreds of different kinds of phones there that all work a little bit differently, different features installed by the software, the hardware vendor specific to their devices, that makes it pretty difficult to do this stuff easily. Again, but that said, Android devices specific information can be extracted from those devices as opposed to a full; I’ll just use the Word backup or capture of an iOS device. iOS devices we have to capture the whole phone to get at text messages, whereas Android devices, when we can collect from them, we can target just text messages. So again, mileage will vary depending on the type of device, and specific operating version, and any secret sauce that the hardware vendor puts in software wise on that device. 

Mark: So you know, Adam, this kind of goes back to a proper scoping call at the beginning, right? Because you may not care about certain data points on the phone, but then you might also discover that yeah, the person that owns this phone does have the Words with Friends game, and potentially he’s using the chat feature inside that app to communicate with somebody about some insider training. So we keep drilling into the point of you ask the right questions at the beginning to facilitate a proper data preservation and collection and move into the review of that data. And cell phones are something that are not going to go away, or mobile devices, call them what you will. Showed my age there, sorry. But you have to ask the right questions, I think, is part of your answer, right? 

Adam: It’s always about asking the right question and listening to the answer because that answer could generate a few more questions for you to better understand what’s going on, and if the answer is past that initial question that’s being asked. 

Mark: So we had 100% on question four. But I think question two is also an interesting one because we see lots of corporate clients of ours that have very large data sets, structured and unstructured. So question number two is how can I search three terabytes of server data? I probably have to define what search really means. Without collecting and processing it, right? 

Adam: Yeah, I would agree with that. So, typically, in order to search any kind of data, you have to be able to index it first. That’s true of Windows search, or Spotlight on your computer, or if you’re trying to run a search against your organization or even run a search in SharePoint. Indexing has to be turned on somewhere in order to be able to run those searches; otherwise, it’s a very, very slow process because then where you’re searching from has to look at every document to see if it can find what it’s looking for. That said, for large data sets again, I keep on going back to data mapping. You can do a data assessment or triage on that data set. That assessment can often provide detailed knowledge about the type of data that is contained within the three terabytes on the server. Often we find that when dealing with large amounts of data on servers, not all the data is commonly processed. There could be a large amount of backup files, system files, program-related files, or multimedia files that just may not be relevant or potentially responsive to your case. So that assessment could be running file listings or file type listings on it just to understand what’s there, understand the scope of what’s there so that again you can start making an assessment of what might or might not be important. Looking at file listings shouldn’t be your end all be all necessarily, but it could be depending on the structure of data in that server share. So again, once the assessment is complete, it could provide you with additional information that you can your council can make to make an informed decision on what to possibly collect or what not to collect on the server. All of that information is just metrics to let you know how big your problem may be and what the cost might be of processing either the whole thing or being more targeted. That said, you should always speak to the custodians too. We know that data generally doesn’t end up there without any user interaction, and user interaction through the use of custodian questionnaires. You can find out who’s storing what where. How good, I use the term data hygiene or data structure, how they store their files, or groups of people store their data so that you can really drill into what you can capture or preserve. 

Mark: Thanks, Adam. And it looks like I might have made a presenter faux pas by not acknowledging there was question five, which the results tabulated. Sorry people who voted for question five, we’ll get to that in our blog. 

Brian: Well, Mark, actually, real quick. For those people who are interested in that, if you just Google “BIA exiting employee,” you’ll find the Exiting Employee Checklist that we just published that covers that topic, along with, like, literally step-by-step questions and steps you should take for an exiting employee, especially in the remote world. So search “BIA exiting employee” in Google, and you’ll get the full answer along with a checklist right there. 

Mark: Brian, I knew there was a reason we had you on the call today. That was applause-worthy right there. 

Adam: Just remember the exiting employee problem has gotten bigger because the remote employee issue has gotten bigger because these days, just about everybody is a remote employee. 

Mark: That’s right. Alright, let’s head to our next slide. And so this one is data analytics and review hosting questions. So question number one – what are some of the biggest mistakes you see people making with analytics tools? Question two – there are so many analytics tools out there. How are they different, and how do I go about choosing? Does BIA have any favorites? Three – opposing counsel continues to send up productions as PDFs, and it’s driving me crazy – and costing a fortune to convert to TIFF and OCR. It’s also costing us time. I’m concerned that we are missing metadata. Any guidance would be appreciated. Four – when and why do I need a clawback agreement? Are they used more often these days? And five – am I ethically obligated to disclose that I used TAR to review and produce ESI? 

These are a bunch of great questions. I’m going to set the timer, and let’s see if we have any results. 

Brian: I think we have to push the poll. 

Mark: That’s right. Looks like we’re going to be starting with question number three. Oh, wait a minute. I spoke too soon. We’ll still start with question three, and then we’ll do one and two. So Lisa, our data diva. 

Lisa: Three was our OCR, our PDFs, yeah. So yeah, you’ve got an opposing counsel who keeps producing in PDF. Really, there’s two ways to look at it. If you’ve got an opposing counsel who’s frankly not very sophisticated, and that’s the reason they’re producing in PDF, then you know maybe it’s something that you just kind of work with them with. But if you’ve got an opposing counsel that’s doing it on purpose, and we definitely have seen those, then it’s time to go a little further. The federal rules say that you need to produce your data in a useable format. So it’s pretty accepted that you should be promoting metadata, searchable text, et cetera. Going back to that unsophisticated opposing counsel, sometimes they’ll think that they are producing text as long as they’re producing searchable PDFs. Unfortunately, most review platforms really won’t do anything with searchable PDFs. You can go ahead and load a searchable PDF, but it’s still going to just be the PDF. You’ll need to OCR it. So to the extent that you think they’re doing it on purpose, then there are definitely rules both federally, and most states have similar rules saying that they can’t withhold the metadata and the text. If they’re not doing it on purpose and they just don’t quite know, then maybe it’s a little easier to work with them. And maybe the answer is simply getting them to agree that they’ll help get that metadata for you because some of the metadata is definitely important, and you would be right to be concerned that you’re missing metadata if you’re just getting the PDFs and not the metadata. The metadata can tell an awful lot of that story. 

Mark: And Barry, you’ve got some experience with this specifically. Is there anything you want to add? 

Brian: Oh, we can’t hear you, Barry. 

Lisa: No. 

Mark: Took a hard pass. 

Barry: Sorry about that. I muted so you wouldn’t hear me coughing. One of the things that we talk about with our clients is early on in a matter; get an ESI production protocol in place. And a production protocol will address the issues Lisa was just talking about, including which metadata fields are produced, what the format of the production is to be in. And it almost never calls for a PDF production, unless that was the native version of the file. But also, TIF imaging is the way to go. Excel documents are typically produced natively. So spelling all of those issues out, those concerns out, in the ESI protocol goes a long way to assuring both sides are receiving the information that they’re entitled to receive. And the best time to get a production protocol in place is at the early meet and confers related to the matter. And that is about the only place in the federal rules where they talk about these types of things is get them involved early on in the meet and confers.

Brian: Hey Barry, just to add on to that. First off, one quick thing. This had come up a couple of times where we’ve dealt with individuals producing PDFs and even non-searchable PDFs. And they’ve warned them, saying, look, you’ve got to do at least TIFS with OCR and metadata, that’s industry standard. Like you said, natives pretty much for Excels and sometimes PowerPoints and other types of files audiovisual. The production standard in the industry is pretty standardized these days. I mean it used to be the day we would be called on by clients to fight that all the time. Every single time somebody has produced an unusable or unsearchable PDF or just the PDF without metadata, and we’ve gone to bat on that subject, we’ve won every single time. Every-single-time. So there’s no question that it’s not, that if you’ve got somebody producing stuff that way you could basically tell them look, you can either be cooperative about it if you don’t know what you’re doing, glad to help or have our vendor help you do it. But if we have to go back to the court and force you to do it, you’re going to lose, and you’re going to get dinged for it as well, and maybe even sanction. 

One quick thing, there was a question – is printing ESI on paper ever acceptable for production? If so, can you give an example of that? I figured might as well answer that now because it goes right along with this. The answer is that really, I mean, technically printing the TIFF is kind of the same thing as printing to paper. But nobody really produces paper anymore. You still see it in really small cases, so that’s one thing. And like I’ve always said, Barry talked about agreeing with your opposing counsel on a production format. If opposing counsel agreed to produce all their documents chiseled out on a stone tablet, if they both agreed to it, they agreed to it. So you can always agree to anything you want to. But in general, producing the paper, producing boxes of paper, not only is it wasteful and extraordinarily more expensive, but it’s rarely ever done, and wouldn’t really be appropriate unless for some reason again a small little case, a small arbitration, something the parties agree to produce a handful of documents that way. Other than that, I would say that it would never be appropriate. 

Mark: Great answers, guys. Thank you very much. Question number one – what are some of the biggest mistakes you see people making with analytics tools? And we’ve got about five minutes. 

Lisa: Got it. I would start with: not having the right people running it. Simply because just because you have the tool does not mean you can just throw everything at it and it will work beautifully. And that kind of goes into also prepping the data properly, segregating the data properly, especially if we’re talking about like a TAR process. TAR works based on the text of the documents. To the extent that you’ve got documents where they don’t have any text, or they have a very small amount of text, emails where you’ve got just one line, or emails that literally just say ‘see attached’. When you run that through a TAR process, you’re really saying, let’s say you code that document, you tag that document as responsive, you’re telling the TAR engine find every document that basically just says ‘see attached’, and make it responsive, which is probably not what you’re expecting to do. So just being able to segregate out the data properly, and really anyone not coding. For a long time, we said you need to code on the four corners of the document, but even then, that’s even a little harder to understand. Because really what we need you to do is just look at the text. Anything outside of the text that you know about that document the analytics engine doesn’t know. So a lot of it is really understanding how the analytics works and working with it and not kind of working against it. And to that end, thinking that the analytics tool is going to do the review for you. You can throw half a million documents in the analytics tool, and its job is to help you identify documents and to help you segregate documents. Its job is not to do it for you. It still requires some work on both the vendor end and on the attorney review end. And that’s probably the biggest mistake is just thinking it’s going to do it for you and not knowing how to use it properly. 

Brian: Hey, Lisa? 

Lisa: Go ahead. 

Brian: Sorry, just really quick while you’re doing this. There’s a question that came in. If a client gives you a list of search terms, do you rely just on that? Or do you still conduct your own search terms? Just in terms of what you’re talking about, I think now would be a great time, when you’re done, to address that one too as part of it. 

Lisa: Yeah, no, yeah. So yeah, a lot of people will ask, and I know we have a question. I think it might be on the next slide. But basically, is keyword searching dead? Or should we still use keyword searching? Keyword searching is not dead yet. I don’t think it will ever be dead. Search terms are really good for finding something that you know is there. So, for example, if I want to know where my nearest Starbucks is, I’m going to Google Starbucks. That is a keyword, and that is what I’m looking for. But if I wanted to know all the coffee places around, searching just for the word Starbucks is only going to find me Starbucks. It’s not gonna find me the other coffee places. So to the extent that you’ve got a tool where you type in Starbucks, and it says okay, it seems like you’re looking for coffee places, do you want to look for other coffee places? That’s moving more into the conceptual searching. So whenever you’ve got keywords that you know will help you get to a place, help you get to documents you’re looking for, it’s great to start out with those. They’re probably just not going to be the best way to approach the entire document review overall. It’s just one piece that you use. One important piece of that would be looking for privileged documents. Privileged documents really, in my opinion, do not do well in a TAR process mainly because TAR is based on the concept of the document. It’s looking for similar documents, not necessarily similar words, but similar concepts. And privilege is more structural than that. If you’ve got an email from an attorney that says, you know, gives an instruction to an employee, that document itself may be privileged solely because it’s coming from that attorney. If you’ve got the exact same sentence, but it comes from just another employee in the company, it’s not going to be privileged. And a TAR engine is going to look at both of those emails and think they’re basically the same email, even though one would be privileged and one would not. So using keywords to help identify certain documents, especially if you know exactly the documents you’re looking for, or to help you identify privileged names and stuff, keywords will always be important for stuff like that. 

Brian: Yeah, I heard a timer go off there. But real quick, the one thing I also wanted to mention on this was like, like you were saying with the analytics tools a big part of that is using expertise. And isn’t there something similar with search terms? That was kind of one of the things I wanted to quickly touch on is search terms expertise comes into play there really important too. Can you do thirty seconds on that before Mark moves us on to the next? 

Lisa: Just one of the big things that we notice is that people will try and use search terms by going from the request for production and stuff and coming up with terms without really diving into the documents and finding out how people are actually talking. Knowing how people talk in a company, it will get you much, much further than just looking at the RFPs and looking to see what you’re supposed to be producing. 

Brian: And that bringing the expert to really help you do that will make your producing that much faster. 

Lisa: Absolutely. 

Brian: It’s like anything else. Somebody who spends all their time doing this kind of stuff is just going to naturally know how to make it more effective. And a little bit of money spent there will save you 10, 20, to 100 times the cost later on for really making those terms, if you’re using search terms, really making them effective. It’s that thing where a little bit of time and effort spent on really bringing in somebody who specializes in that can really make all the difference in the world and the rest of the process. 

Lisa: Absolutely, yeah. I think it’s really important to really nail those keywords down, but know that’s not the end of the process. Once you’ve gotten a really good set of keywords, then you can kind of go into your other analytic tools to help you find the documents that you’re not going to find with the keywords. 

Mark: As we move to the next slide, and we’re going to have about seven minutes to focus on this slide, Lisa. Just answer question number five if you would. Am I ethically obligated to disclose that I use TAR? It can be a short and sweet answer while I get ready for reading the next set of questions. 

Lisa: I have – so am I ethically – It depends on what your ESI order says. So if you’ve got an ESI order that says both sides have agreed, they’re going to use TAR, and they’re going to disclose what TAR engine they’re using and what their recall and precision—if you’ve got an ESI order that says all that, then yes, you are absolutely ethically obligated to disclose it because you signed it. But to the extent that you have an ESI order that says you are going to produce documents that are responsive to the request for production, then all you are obligated to disclose is that you have reviewed documents and produced them. Attorneys have to sign, and they have to swear that the documents they’ve produced meet the criteria that has been requested and that it’s not a document dump, and you’re not purposely withholding documents. How you get there, it really doesn’t matter. And 20 years ago, nobody asked, “Did you have document reviewers open and read every single one of these emails, and did you have a document reviewer go through every single file folder?” It was just accepted that if you produced it, you were okay with what was in the document, and you were giving it to the other side because it met that requirement. Nothing has really changed in that. Again, to the extent that it’s not specified in the ESI protocol, there really is no ethical obligation to say that you’re using any type of analytics to help you get to the production. 

Mark: Thanks. We get that question a lot, so thanks for her giving a detailed answer. Okay, our final slide! We’ve got about five minutes left till the top of the hour. We certainly want to end on time, so managed review questions. And this could be a whole webinar by itself, but again, we’ll answer any questions we don’t get to in our BIA blog. Question one – is keyword searching dead yet? I think we’ve covered that a little bit, actually in detail. Number two – I keep hearing conflicting information, whether flat-rate per document pricing is better than hourly. Which is more cost-effective, and why? Three – I don’t have any experience managing document review, but we have a case where there will be almost 500k documents. My outside counsel has limited resources but has a vendor they’ve used before, any suggestions? Four – should we be using email threading? Five – will AI eventually eliminate the need for humans in document review? And so people like question number one. I think it’s deserving maybe of a further discussion. And you know, how about we start with that, Lisa? And then maybe question number four, which I think will be an easy one. Should we use email threading? 

Lisa: Sure. Go back to my questions. Number one was – 

Mark: Is keyword searching dead yet? 

Lisa: Yeah. And you know, just like we said, I don’t think it’s dead yet. I don’t think it will ever be dead. There’s really no replacing keywords. In particular, when you open up any of the analytics tools, most of them have places to use keyword searching. So the keyword still gets used. One place that I didn’t mention earlier that we use keywords still is what we call anti-proximity searches. So let’s say you’re looking for the word “privileged,” but you don’t want it to hit in the footers of every single email now that has a footer that says “Privileged and Confidential,” or some combination of those words. We can use what we call anti-proximity searches. So, find me the world “privileged,” not within zero words or one word of privileged and confidential in the footer. It’s stuff like that where we still use keywords. It’s also even in the very beginning of the case: it’s a very quick and easy way to get some idea of how many documents are even talking about the topic that you’re looking at. So, I don’t think it will ever be dead; I would just say it requires more training than just knowing how to type a word into Google. It’s a little bit more complex than that if you want to use it appropriately and correctly. 

Mark: It’s an art form for sure. And Lisa, we’ve got two more minutes. 

Lisa: Which one? You said four? 

Mark: We’ve got some new poll results in. So question number five is, will AI eventually eliminate the need for a human review? I’ll give you a minute for that one, and then question number two is a flat pricing rate versus hourly doc review question. 

Lisa: So for – will AI eliminate the need for document review? I mean, to some extent, it already has. It may not eliminate 100% of human review, but it certainly reduced it significantly. We’re able to tackle reviews of million-plus documents—and not pages, documents—in less than three weeks with a team of six expert analytical reviewers. In the past, that’s something that easily would have taken you 40 or 50 reviewers for a couple months. Beyond that, you’ll always need humans who understand how to use the tools and can accurately divide the data up and choose which tools to use. So, will it eliminate the need for humans in document review? I’m gonna say a lot of it, yeah. But not 100%, but some of it absolutely. 

And then number two –

Brian: I just want to make a point. You’re saying less people, but I think this is one of those things where you could say it’s going to become more of a quality, a question of quality versus quantity. It’s not grabbing a bunch of people and throwing them at a project. If you’re going to grab those fewer people, those fewer people have to have higher level of skills, better analytics understanding, all of that stuff. So it’s a higher quality of training and experience, but a much lower quality – quantity, sorry – of people. 

Lisa: Not quality, quantity. You need a handful of experts and not a big team of non-experts, basically. 

Barry: And Lisa, let me add one point here on reviewers. With privilege review, typically, we see that we do need to have eyes on many of the documents. 

Lisa: Mark? 

Mark: Well, we are at the top of the hour, so I don’t want – I want to be respectful of everybody’s time. We’re really thankful that people joined today. All things considered, with webinar fatigue and virtual meeting fatigue, it really means a lot for us to have people show up and engage. And hopefully, this was an enjoyable walk down eDiscovery lane with our experts this beautiful Tuesday afternoon. Like I said, we will answer the rest of our questions on our blog. And happy holidays to everybody, and we look forward to hearing from you and talking with you more next year. And thanks to everybody at ACEDS. As you know, we appreciate you guys and look forward to talking again soon. Have a great day everybody, thanks for joining. 

Mike: Okay! Thanks so much, Mark, and thanks everyone at BIA. Thank you for a great presentation today and the expert answers. Please visit ACEDS.org for a complete list of upcoming webinars. Have a great day, everyone, and be kind to one another. 

Brian: Thank you. 

Barry: Thank you, Mark.