The issues associated with Electronically Discoverable Information (ESI) hang over the legal profession like the threat of Katrina II hangs over New Orleans. Lets face it: most judges and attorneys would do anything to avoid confronting the complexities of ESI. However, judges are good at forcing lawyers to face up to bad stuff, so it’s impossible to avoid the subject.
Of course, in a huge case involving large sums of money it’s no problem hiring a consulting firm that does all the work for the lawyers, and guides them every step of the way. However, that’s only 1 case in 100, if that. What about all the “little cases,” where expensive consultants are not an option?
The answer, not surprisingly, is the “keyword search.” After all, if we can search a trillion documents using Google, why not use key word search to find documents relevant to litigation.
Sadly, key word search is not very reliable. For example, if you have a million documents how would you formulate a key word search that would be certain to collect documents that relate to to people under age 12? In fact, recently the courts have noted the shortcomings of key word searching and criticized its use.
An excellent article in the April 2009 ABA Journal discusses these issues in some detail, and the good news is that some “very smart people” are working on a solution. The bad news, as I said, is that key word search is not very reliable, and this poses a problem until the problem is solved by these masterminds. To quote from the article:
It would be the ultimate discovery for e-discovery: a perfect method to turn terabytes of digital data into a collection of case-relevant documents.
Three years ago, a handful of lawyers and scientists started the quest, a project to save litigation from being buried in an avalanche of electronic documents. Since then, the Text Retrieval Conference Legal Track has been using different types of computer searches to wade through huge piles of digital information, hoping to get closer to a complete picture of what is issue-important in a computer’s data stores.
The good news: The TREC Legal Track team believes it is close to finding a protocol that can work. The bad: The project also found disturbing problems with the way lawyers work today.
And the harshest conclusion: Keyword searching—what most lawyers use to find litigation documents—misses the majority of relevant documents. Or as Jason Baron, one of the Legal Track study coordinators, puts it, “Lawyers need to understand that the way they have been searching for electronic documents has some serious flaws.”
So, just what is the “TREC Legal Track”? Well, the TREC Legal Track web page has a number of documents that discuss the efforts underway. The papers are, based on my perusal, not for the faint of heart – they involve technical assessments of the effectiveness of various search. Lets hope we can get by on old-fashioned key word searches until the happy day arrives when the TREC folks, or someone, finds that protocol.