Embed Size (px)
DESCRIPTION
Once a controversial tool in electronic discovery, technology assisted review (TAR), also known as predictive coding or computer assisted review, has gained judicial acceptance and is increasingly used for for document review in large-scale legal matters. Less recognized, however, is that TAR has a range of uses beyond simple review that can help in mastering large document sets, from information governance to early case assessment and preparing for depositions and trial. This presentation is by John Tredennick, Esq., CEO and founder of Catalyst Repository Systems. It covers how TAR works and the various ways lawyers are now using it.
Citation preview
Technology Assisted Review Moving Beyond the First Generation
John Tredennick CEO/Founder
Catalyst
§ 1,800 Exabytes
§ 1.8 million Petabytes
§ 1.8 billion Terabytes
§ 1.8 trillion Gigabytes
§ 1.8 quadrillion Megabytes
1.8 Zettabytes a year
Library of Congress—30 Terabytes
Exploding Content >> Big Data
Sixty Million Libraries of Congress each year!
60 million libraries a year...
... and growing
0"
50"
100"
150"
200"
250"
300"
2003" 2004" 2005" 2006" 2007" 2008" 2009" 2010" 2011" 2012"
Case Size (in Gigabytes)
Big Data >> Big Discovery
Telling Stories 1. Your job has not changed. 2. But it has gotten a bit harder. . .
þ Find the story
þ Tell the story
þ Prove the story
Trust
Is This New?
We Already Use It
Predictive Ranking
What is the Process? 1. Assemble your files
Shredding the Documents
1 2
3
What is the Process? 1. Assemble your files 2. Add seed documents to the mix 3. Analyze seeds and rank similar
documents
How Does it Work?
How Does it Work?
§ Support Vector Machines § Naïve Bayes § K-Nearest Neighbor § Geospatial Predictive Modeling § Latent Semantic
"I may be less interested in the science behind the "black box” than in whether it produced responsive documents with reasonably high recall and high precision.“ Peck, M.J. (SDNY)
What Goes on Under the Hood?
The computer builds a big, complex search!
What terms are most likely to be associated with good documents?
What terms are most likely to be associated with bad documents?
What is the Process? 1. Assemble your files 2. Add seed documents to the mix 3. Analyze seeds and rank similar
documents 4. Test results and provide more
samples—iterative process 5. Order review by ranking
Cut Point
Ranking a Document Set
Understanding the Savings
0%#
10%#
20%#
30%#
40%#
50%#
60%#
70%#
80%#
90%#
100%#
0%# 10%# 20%# 30%# 40%# 50%# 60%# 70%# 80%# 90%# 100%#
Percen
tage)of
)Rele
vant)Docum
ents)Foun
d)(Re
call))
Percentage)of)Documents)Reviewed)
Yield)Curve)
Percentage of relevant documents found
Number of documents in the review
Linear Review
0%#
10%#
20%#
30%#
40%#
50%#
60%#
70%#
80%#
90%#
100%#
0%# 10%# 20%# 30%# 40%# 50%# 60%# 70%# 80%# 90%# 100%#
Yield&Curve&
%&of&Documents&
%&Re
levan
t&
Review 12% and get 80% recall
Understanding the Savings
0%#
10%#
20%#
30%#
40%#
50%#
60%#
70%#
80%#
90%#
100%#
0%# 10%# 20%# 30%# 40%# 50%# 60%# 70%# 80%# 90%# 100%#
Yield&Curve&
%&of&Documents&
%&Re
levan
t&
Review 25% and get 95% recall
Understanding the Savings
12,000
10,000
8,000
6,000
4,000
2,000
Res
pons
ive
10,000 20,000 30,000 40,000 50,000 60,000 70,000 80,000 90,000
Reviewed
Wellington F Responsive Review
80% Recall Review 29,248
95% Recall Review 39,132
100% (Linear) Review 85,725
12,000
10,000
8,000
6,000
4,000
2,000
Res
pons
ive
10,000 20,000 30,000 40,000 50,000 60,000 70,000 80,000 90,000
Reviewed
Wellington F Responsive Review
80% Recall Review 29,248
95% Recall Review 39,132
100% (Linear) Review 85,725
Predict(Review 80%(Recall 95%(RecallResponsive 9,168 10,887Reviewed 29,248 39,112Reduction 56,477 46,613Saving<($4<Doc) $225,908< $186,452<
1. You only get one bite at the apple.
2. Subject matter experts are required for training.
3. You must train on randomly selected documents.
4. You can’t start TAR training until you have all of your documents.
5. TAR doesn’t work on foreign (Asian) language documents.
6. TAR doesn’t work with sparse collections.
The Five Myths of TAR
