M12S13 - RIM for the Next Generation: A Call to Action

Cohasset Associates, Inc.

NOTES

2012 Managing Electronic Records Conference 13.1

RIM FOR THE NEXT GENERATION: A CALL TO ACTION

©2012

All-Partner Teleconference | June 24, 2009MER Conference| May 8, 2012

Charles R. BoozInformation Governance, Inc.

Michael SalvarezzaLRN

Julia BrickellH5

©2012

©2012


NOTES


©2012

©2012

CHALLENGES

Electronic RMElectronic RMEmail ArchivingEmail Archiving

Morgan StanleyE‐Discovery irregularity fine $1.58b

HIPAA

Enron ScandalSarbanes‐Oxley

FinancialCrisis 2008

Zubulake‐UBS WarburgFRCP 2006

New Laws?

New Regulations?

New Laws?

New Regulations?

eDiscoveryeDiscovery Federated RMFederated RMEnterprise IG PlatformsEnterprise IG Platforms

Goldman Sachs

©2012

DoD 5015.2 9/11Patriot Act

MoReq


NOTES


CHALLENGESComplexity of Information Governance

Requirements

CurrentGenerationSolutions

Complexity of requirements grows

exponentially with size of organization

©2012

"Small" SMEs "Large" SME's Fortune 2000 Fortune 500 Fortune 100

Solutions

Complexity of rapidly changing compliance landscapeComplexity of corporate/legal structureComplexity of content/contextComplexity of infrastructureComplexity due to sheer volume of information

CHALLENGES

©2012

Patriot Act

SEC 17a-4Title 21 CFR 11

MiFID

DoD 5015.2

Businesses care about:•Agility•Complexity

Social Media Promotes:•Sharing•Collaboration

Records Management is about:

•Governance

Records Management vs. Social Media vs. Business

©2012

9

•Complexity•Access to information•Insights derived from information•Speed and results

•Open-ness•Rapid access to Information


NOTES


INFORMATION GOVERNACE

Enterprise Information Management

Governance, Risk Management, and Compliance (GRC)

Information Governance

ECM

IDARSLegal

Security

Regulation

©2012

Compliance (GRC) Governance

RM

FinancialReportingCompliance

Imaging Systems

Business Intelligence

Managing Risks

Records Information Management

SecurityOfficer

PrivacyOfficer

LegalCounsel Corporate

RM

ComplianceOfficer

BOD

CorporateIT

RiskOfficer

OtherOfficer

Retrieve Information?Capture Information?

LawsRegulations

Internal PoliciesBest Practices

Etc.

D t P i ?

Cost Governance?

POLICY: (e.g., Corporate Records

POLICY: (e.g., Corporate Records

Security?

Content Producers‐MS Office‐MS SharePoint‐ Alfresco‐ Business Applications‐ Social Media

Content Consumers‐MS SharePoint‐ Alfresco‐ Business Applications‐ Social Media

End Users End Users

l

©2012

Jurisdiction nJurisdiction COther JurisdictionsJurisdiction A

Data Privacy?

RecordsManagement?

Other Repositories

ECMSystem

RSD FoldersECM

System

Retention Schedule)Retention Schedule)

Federated RMFunctionality?

SocialMedia

Storage ILM?

Standard Metadata Definitions?

eDiscovery & Holds?

System Admin?

Lifecycle Event Sources‐ Business Applications‐ Processes

BusinessManagers

Events that impactinformation lifecycle?

Records Information Management

Physical documents

Electronic documents

Other typesof formats

‐ Paper‐ Film‐ Fiche

‐MS‐Office‐ PDF‐ Other

Web pagesSocial Media

Unstructured content(high volumes)

©2012

Types of Record Formats

Messages

Sections of large reports

Data in application databases

Data in data warehouses

‐MS Exchange‐ Lotus Notes‐ IM‐ Social Media

‐ AFP‐ PDF‐ Other

Entries in SQL Database

Entries in data warehouse

Structured content(very high volumes)


NOTES


IG Function RM IT

Lifecycle of unstructured content

Responsibility RM Program and Records Retention Schedule (retention policies).

Often views RM as low priority and limited to paper & electronic documents.

Lifecycle of structured content

Often unaware/unable to manage lifecycle of this content.

Maintains control of its lifecycle.

EDiscovery and holds Limited scope to unstructured documents declared as records.

Gaining role conducting discovery within corporate repositories and producing them.

Data Privacy of content

Often unaware/unable to manage data privacy aspects of records.

Expects RM to manage privacy aspects of unstructured content. Often views privacy f t t d t t d t it


©2012

of structured content as data security.

Reduce Cost Limited visibility and leverage over topic.

Focused in reducing cost of infrastructure but lacks ability to optimize infrastructure costs versus IG SLAs.

Philosophy Organize information – cannot rely on search

Why organize, simply search.

Differences in Lingo

Archive Move content offsite when no longer needed.

Backup and recovery term.

Information Lifecycle Management

Manage retention/disposition of content.

Move content to lower cost storage tiers(Storage ILM).


Retention & Disposition

Lifecycle of Paper Record (Storage)

Other Lifecycle facets

LegalCounsel

RM

ComplianceOfficer

RiskOfficer

OtherOfficer

©2012

Record Attributes

Lifecycle of Data Privacy settings

Lifecycle of Security

Classification

Lifecycle of Electronic Record (Storage ILM)

Lifecycle of Metadata Groups

Lifecycle of Content Index

SecurityOfficer

PrivacyOfficer

IT

IT

IT

PrivacyOfficer


SecurityOfficer

PrivacyOfficer


RM

ComplianceOfficer

BOD

CorporateIT

RiskOfficer

OtherOfficer

Information Governance Steering Committee

©2012

Corporate IT:Manage corporate information and IT infrastructure

Corporate RM:‐Manage process of creating IG policies‐ Ensure that policies are up to date‐ Ensure policies are available to field personnel

Legal Counsel: Responsible for legal department within organization ‐must be able to act decisively regarding legal challenges that face organization.

Risk Officer:Manage risk matters within organization

Privacy Officer: Oversee and manage compliance with Privacy laws and regulations

Compliance Officer: Oversee and manage compliance issues within organization

Security Officer: Responsible for security matters within organizations, including data security

Other Officer: Other corporate officer

BOD: Board of Directors with primary responsibility for approving corporate IG policy

Other: Depends on organization.


NOTES



Retrieve InformationCapture Information

Information Governance Policies‐ Retention and Disposition‐ Data Privacy‐ Discovery‐Migration across storage tiers‐ Standard Metadata DefinitionsOther

Information Governance Policies‐ Retention and Disposition‐ Data Privacy‐ Discovery‐Migration across storage tiers‐ Standard Metadata DefinitionsOther

Corporate IG Policies

Cost Governance

Security

Content Producers‐MS Office‐MS SharePoint‐ Alfresco‐ Business Applications‐ Social media

Content Consumers‐MS SharePoint‐ Alfresco‐ Business Applications‐ Social Media

End Users End UsersInformation GovernanceSteering Committee

SecurityOfficer

PrivacyOfficer


RM

ComplianceOfficer

BOD

CorporateIT

RiskOfficer

OtherOfficer

LawsRegulations

Internal PoliciesBest Practices

Etc.

©2012

Enterprise Information Governance Platform

‐ Other‐ Other

Jurisdiction nJurisdiction COther JurisdictionsJurisdiction A

DataPrivacy

RecordsManagement

Enforcement

IG Control & Admin

IG Policies

Enforcement

IG Control & Admin

IG PoliciesControl andAdministrationof lifecycle forALL information

Control andAdministrationof lifecycle forALL information

ECMSystem

Social media

RSD FoldersECM

SystemOther

Repositories

Storage ILM

Standard Metadata Definitions eDiscovery & Holds

System Admin

EDiscovery & Holds

BusinessManagers

Events that impactinformation lifecycle

Lifecycle Event Sources‐ Business Applications‐ Processes

Enforce lifecycle actions

Jurisdiction #1

Jurisdiction #2

INFORMATION GOVERNACERecords Information Management•Manual Retention Policy Methods•Policy remote Excel/Email/Paper/PDF•Unstructured documents•Retention policy ONLY•Little or no involvement of IT

File Plan

File Plan

File Plan

RecordsAdmin

RecordsAdmin

Manual RM

Manual RM

RMARMA

End User

File File File File

RetentionScheduleRetentionSchedule


©2012

Conventional Corporate RM Program

Jurisdiction #n

Corporate RM

Manual Administrationof RIM Program

RiskOfficer

RecordsAdmin


RM

RecordsAdmin

CorporateIT

RecordsAdmin

RecordsAdmin

RMARMA

RMARMA


Manual RM

Manual RM

Manual RM

Manual RM

Plan Plan Plan Plan

File Plan

File Plan

File Plan

File Plan

File Plan

File Plan






Jurisdiction #1

Jurisdiction #2

File Plan

File Plan

File Plan

File File File File

RecordsAdmin

Integrated Administration

Enforcement

IG Control & Admin

IG Policies

RecordsAdmin

End User

Strategic Information Governance•All facets of Information Lifecycle•Management of all forms of records•Policies in application integratable form•Direct involvement of IT & others

©2012

Information Governance Program

Jurisdiction #n

RRSRRS

RRSRRSRRSRRS

Plan Plan Plan Plan

File Plan

File Plan

File Plan

File Plan

File Plan

File Plan

Corporate RM

RecordsAdmin

Integrated Administrationof IG Program

Enforcement

IG Control & Admin

IG Policies

IG Policies

IG Platform technology deployed at CorporateRecordsAdmin

Enforcement

IG Control & Admin

IG Policies

RecordsAdmin

RecordsAdmin

Enforcement

IG Control & Admin

IG Policies

RiskOfficer


RM

CorporateIT


NOTES


Automated Policy EnforcementCurrent Situation


RM

PoliciesControl & Admin

Data PrivacyPoliciesControl & Admin

Structured Content Repositories

Rules(Policies)

Corporate Information Governance Policies

Tools

Information GovernanceCorporate/Regional/JurisdictionalControl and Administration Processes

©2012

Size of bubbles not to scale

Next Generation IntelligentContent Addressable Storage Repositories Policies

Control & AdminRepository

IDARS

PoliciesControl & AdminRepository

ECM

Control & AdminRepository Policies

Control & AdminRepositoryeDiscovery

PoliciesControl & Admin

Tools(Control &Admin)

RecordsMgmt

eDiscovery

DataPrivacy

Other

AuditMgmt

Information RepositoriesRegional/Jurisdictional/Local

Tools(Repositories)

Content inCAS Systems

Content inIDARS

Content inData Whse

Content inECM Systems

PoliciesControl & AdminRepository

Leveraging Automation

©2012

Leveraging Automation

© 2012 H5

Automated classification can help Records Managers to: Find the data

Review the data

Uses of Automation

©2012© 2012 H5

Sort the data

Classify the data you need

Reduce the data you don’t need


NOTES


Automation, chosen and customized with proper expertise: Can be used to decide what to keep and how to

categorize it– Better than people

Advantages of Automation

©2012© 2012 H5

p p– Faster than people

Can address: – Internal and external data– Data in motion

Increases likelihood of getting the job done

Consider the data stashes– Used for different purposes– Contain different content– Important for different reasons

Assess the information likely to be useful from each

Approach

©2012© 2012 H5

Assess the information likely to be useful from each data stash

– What types of information are important?– Whose data might contain that content?

What departments? What people?

Human Tagging– No desire to do it– Inaccurate– Inconsistent– Not exhaustive

Classification Quality with Automation

©2012© 2012 H5

Not exhaustive– Requires expensive training and documentation

Automated Tagging– Fast– Consistent, but…– Without expertise, results vary wildly


NOTES


TREC Legal Track

National Institute of Standards & Technology Text Retrieval Conference (“TREC”) has a Legal Track designed to evaluate effectiveness of search technologies and methods

M bilit t id tif d t i t h th ti l

©2012

Measures ability to identify data responsive to hypothetical “topics” or “requests”

Open to the public (vendors, law firms, academics)

Uses publicly available data sets

Quality is measured in terms of “precision” and “recall”

© 2012 H5

TREC 2008 Interactive Task

Finding all relevant documentsis 100% recall

Finding only relevant documents is 100% precision

n=

corr

ect

hits

tota

l hits

©2012

Sponsored by National Institute of Standards and Technology. See TREC 2008 Legal Track Overview at http://trec.nist.gov/pubs/trec17/papers/LEGAL.OVERVIEW08.pdf (p. 37)

Points represent results after an adjustment to account for state of OCR in the test collection

_correct hits__ total responsiveRecall =

Prec

isio

n

© 2012 H5

1.0

0.8

0.6

Finding all relevant documentsis 100% recall

Finding only relevant documents is 100% precision

TREC 2009 Interactive Task

n=

corr

ect

hits

to

tal h

its

©2012

Sponsored by National Institute of Standards and Technology. See TREC 2009 Overview http://trec.nist.gov/pubs/trec18/papers/LEGAL09.OVERVIEW.pdf

0.2 1.00.80.60.40.0

0.0

0.4

0.2

Recall

Pre

cisi

o

_correct hits__ total responsiveRecall =

© 2012 H5


NOTES


Richmond Journal of Law and Technology (2011)

___________________________

TECHNOLOGY-ASSISTED REVIEW IN

E-DISCOVERY CAN BE MORE

Search “superior to manual reviews”

“Of course, not all technology-assisted reviews (and not all manual reviews) are created equal. The particular processes found to be superior in this

“Of course, not all technology-assisted reviews (and not all manual reviews) are created equal. The particular processes found to be superior in this

©2012© 2012 H5

EFFECTIVE AND MORE EFFICIENT THAN

EXHAUSTIVE MANUAL REVIEW

Maura R. Grossman Gordon V. Cormack

processes found to be superior in this study are both interactive, employing a combination of computer and human input.” (p.48)

processes found to be superior in this study are both interactive, employing a combination of computer and human input.” (p.48)

XVII RICH. J.L. & TECH. 11 (2011), http://jolt.richmond.edu/v17i3/article11.pdf

A Better Search Results in Less Data

60 000

70,000

80,000

90,000

100,000

Non-Relevant

Relevant

Impact of calibrated filter over prior filter for every 100,000 records filtered

©2012

0

10,000

20,000

30,000

40,000

50,000

60,000

Starting Population:Contains

100,000 Documents

Prior Filter:Filters In

71,000 Documents

H5 Filter:Filters In

16,700 Documents

4,100 4,100 4,100

95,900

66,900

12,600

Calibrated Filter Filters in

16,700Documents

© 2012 H5

Search Myths

All search methods are created equal

All search tools are created equal

All search terms (queries) are created

©2012

(q )equal

No special expertise is required to search

© 2012 H5


NOTES


Tools have different capabilities

Tools have different default setups

Most tools index the data to search it

Test the Tools

©2012

Most tools index the data to search it

Most tools don’t index everything

© 2012 H5

Types of search: Keyword/Boolean Concept Search

•Ontology BasedA l d/ h l ti hi (B i )

Test the Terms

©2012

•Analyze word/phrase relationships (Bayesian) Clustering (Bayesian)

•Auto-classifier•Near Duplicates•Predictive Coding

Experts know the strengths and weaknesses

© 2012 H5

Effective Search Methodology

Design Design Measure Measure accuracyaccuracy

Interview Interview subject subject matter expertsmatter experts

Develop and test the Terms Technology

©2012

Define Define targettarget Apply Apply to data to data Design Design queriesqueries

accuracyaccuracy

Linguists, Statisticians, Researchers, Attorneys, Project Managers, Computer Scientists

Knowledge Transfer Measurement Consistency

Sample data

© 2012 H5


NOTES


H5 EDGE Classifier

Search Design Network

Define targetDefine target

Interview attorneys and Interview attorneys and subject matter expertssubject matter experts

Design queriesDesign queries

Measure accuracyMeasure accuracy2

3

Classifiers Classifiers

Effective Search Methodology

©2012

Client Network

1

5

Sample data4

4

Client/Archive

Client Infrastructure

Incumbent Engine

Hosted litigation Review platform

5

© 2012 H5

Search Realities

Search and automation is an expertise

Competency requires knowledge and process:– Tools: What’s the instrument? What is its

? ?

©2012

approach to processing? To indexing? To retrieval? What searches can it handle?

– Terms (search queries): What do you need to find? How do you craft queries? How do you test their effectiveness?

Quality requires expertise in the tools and the terms

© 2012 H5

Search Expertise Matters

“Whether search terms or ‘keywords’ will yield the information sought is a

li t d ti i l i th

“Whether search terms or ‘keywords’ will yield the information sought is a

li t d ti i l i th

©2012

complicated question involving the interplay, at least, of the sciences of computer technology, statistics and linguistics.”

complicated question involving the interplay, at least, of the sciences of computer technology, statistics and linguistics.”

United States v. O’Keefe,537 F. Supp. 2d 14 (D.D.C. 2008)

© 2012 H5


NOTES


Measurement Matters

[TREC] can be expected to identify … reliable search and information retrieval methodologies and best practice recommendations, which, if adhered to, certainly would support

[TREC] can be expected to identify … reliable search and information retrieval methodologies and best practice recommendations, which, if adhered to, certainly would support

©2012

adhered to, certainly would support an argument that the party employing them performed a reasonable ESI search, whether for privilege review or other purposes.

adhered to, certainly would support an argument that the party employing them performed a reasonable ESI search, whether for privilege review or other purposes.

TREC: National Institute of Standards and TechnologyText Retrieval Conference Legal Track

Victor Stanley, Inc., v. Creative Pipe, Inc.,250 F.R.D. 251 (D. Md. 2008)

© 2012 H5

STRATEGIC INFORMATION GOVERNANCE CONCEPT

• IG Steering Committee: ► Develop and maintain IG policies and procedures in Organization

• Dynamically Refresh Information Attributes and Lifecycle:► Automate collection, certification and disposition of all information

• Assure Information Integrity:► Auditability, Transparency and Authenticity

• Maintain Future Efficacy:► Anticipatory and Flexible

©2012

Information Governance Steering Committee

RepositoryRepository

IG Enforcement Activities

IG Control andAdministration Activities

Local IG Policies (Jurisdictions)

ORGANIZATION IG Policies

Organization

IG Policies:‐ Retention and disposition‐ Data Privacy‐ Electronic discovery‐ Lifecycle of content‐ Lifecycle of content indexes‐ Lifecycle of metadata‐ Other

Jurisdictions & Legal Entities

IG Policies in Jurisdictions and Legal Entities

Business Units

Information created and utilized in Business Units controlled by IG Policies

Inspirational Leadership for Records Management

• Ambiguity is actually OK

• Take risks – go on a TRIP

• Challenge the status quo – try something different

• Find the value

©2012

39

proposition

• Elevate the conversation


NOTES


RIM FOR THE NEXT GENERATION: A CALL TO ACTION

QUESTIONS

©2012

All-Partner Teleconference | June 24, 2009MER Conference| May 9, 2012

Charles R. BoozInformation Governance, Inc.

Michael SalvarezzaLRN

Julia BrickellH5

Education

M12S13 - RIM for the Next Generation: A Call to Action