Inhibitors to Information Sharing

Anticipating How Information Sharing Efforts May Fail

2010

Walter Kitchenman

wkitchenman@hotmail.com

1

• Identify challenges to Info Sharing among multiple depts. and agencies

• Show how Info Sharing adds value as users view data in new ways

• Direct the analysis to a general audience

• Focus on general functional requirements

Purpose

Agenda

1. Overview …..………………………………………………………….. 3

2. Conceptual ……………………………………………………………. 12

3. Economic ……..……………………………………………………..… 21

4. Technological …………………………………………………………. 27

5. Cultural …………………………………………………………………. 33

6. Personal ……………………………………………………………….... 38

7. Solutions ………………………………………………………………... 43

2

OVERVIEWInhibitors to Information Sharing

3

What is Info Sharing?

• Content and Knowledge Management

• Mining and deduping data from distributed databases

• Modeled data and a display of intellectual assets that facilitates expert

analyses

4

“Information sharing includes the cultural, managerial and technical

behaviors by which one participant leverages information held or created

by another participant.’’ - US Department of Defense

OVERVIEW> DEFINITIONS

5

Information is Created by Individuals

• The Lone Wolf Scenario

– Information product developed by individuals

– No central repository

– Individuals have proprietary methods and knowledge

• The “Old School” Scenario

– Key managers may be technology averse

– Main work product on paper and in personal files

Info Sharing isn‟t simply an IT solution for exchanging content on

distributed databases. People must contribute and use the system.

OVERVIEW> DEFINITIONS

6

• Information Overload

– Systems do not „connect the dots‟ opening the door to human fallibility

• The False Positive Scenario

– Search criteria or alerts are too broadly based

• The Turnover Scenario

– New systems projects are complex and long to complete, org. change

outpaces development times leading to a disruptive need to rebuild consensus

• The Ideological Scenario

– Alpha errors based on ideology or politics compromise data shared

– Pushing the envelope in Info Sharing challenges societal or corporate values

• Solution Degradation

– A failure to adequately test and upgrade software, hardware and data models

after implementation

Even Successful Info Sharing Solutions Fail Under Several Scenarios

OVERVIEW> SOLUTION DEGRADATION

7

• Dashboard/Portal

– Push of recent reports and content, flagged reports and alerts

• Auxiliary Profiles for Report Components

– Important sub-sets of info in reports (e.g., a Profile of George Washington to

which all books in which he is mentioned are linked)

• Modeling and Neural Networks

– Population divided into segments to predict statistically probable behavior

– May resemble profiling by race, gender or other factors prohibited by law

– Variables or inputs „fire‟ an algorithm to recognize patterns and recommend

certain outputs or actions

In addition to making conventional reports available across departments

and agencies; modeling techniques segment a population, neural

networks recognize patterns and suggest actions; and behavior, objects

or individuals are flagged for further review by domain experts.

Good Info Sharing Provides a Means to see Info in New and Improved Ways

OVERVIEW> FUNCTIONALITY

Modeling Data - Many Enhanced Info Products Use Non-Linear Modeling and Neural Nets

• SAS Institute (Cary, NC), Norkom (Dublin, Ireland) and Mantas (Fairfax,

VA) solutions identify money laundering through pattern tracking and

suspicious individual watch lists

• Fair Isaac‟s Falcon Credit Card Fraud Management System spots

suspicious card purchases in real-time

• IBM‟s Non-Obvious Relationship Awareness (NORA) determines the

relationships between people (initially developed for the gaming industry)

• Neural Nets are used to predict short-term increases in the NYSE

Composite Index

8

Non-Linear neural nets and other modeling techniques are used to mine

data in a wide variety of fields.

OVERVIEW> EXAMPLES

9

Modeling Data – Some Types of Segmentation

Demographic Segmentation

ATTITUDES

POPULATION

GRAND-PARENTS

POPULATION

REGIONSCOUNTRY

Geographic Segmentation

Attitudinal Segmentation

SINGLEWOMAN

YOUNG FAMILY

ELDERLY

SINGLEMAN

COUPLE

OVERVIEW> EXAMPLES

• Neural networks process non-

linear statistical data and model

complex relationships between

inputs and outputs

• Basic network consists of input

layer (variables), a hidden layer

of high dimension, and an output

layer

10

INPUT

HIDDEN

OUTPUT

Modeling Data - A Neural Network

OVERVIEW> EXAMPLES

11

1. Conceptual

2. Economic

3. Technological

4. Cultural

5. Personal

Inhibitors to adopting successful Information Sharing Solutions are

interrelated and involve people at almost every stage.

Inhibitors to Info Sharing

OVERVIEW> INHIBITORS

CONCEPTUALInhibitors to Information Sharing

12

Conceptual Challenges to Info Sharing

1. Scope

2. Architecture

3. Design

4. Functionality

13

Concepts at inception determine the accuracy, scalability and future

functionality of the Info Solution.

Failed Concepts Stem from Four Key IT-Related Issues

14

Scope

• Budget and Time-Frame

• Constituents

– Who are the participants and what confidentiality is mandated?

– Geography

• End Products

– Dashboard of available data? Real-time alerts?

• Legal and Cultural Restraints

• Roll-Out and Projected Growth

– How will solutions be rolled-out? By agency or dept.? By geography?

– Anticipated growth of content and participants (critical for IT decisions)

A failure to identify the best solution within the context of likely budget

constraints and other limitations can undermine ultimate success.

CONCEPTUAL>SCOPE

Factors Defining the Scope of an Info Sharing Solution

15

Architecture

• Core Organizing Principle

– One element every piece of content has in common regardless of dept. or

agency (e.g., publishers link books to an author)

• Taxonomy and Components

– Find a minimum number of categories or key words (e.g., books as fiction, non-

fiction, biographies, mysteries, etc.)

• Deduping and Parsing

– Names, places, objects have variable spellings, misspellings or name changes

• Database Management

– Which type of dbase management best meets the functionality desired?

– Understand the benefits and limitations of any choices proposed

A failure to identify core elements that information shared across

departments have in common can result in poor information architecture.

CONCEPTUAL>ARCHITECTURE

16

Architecture – Integrating Different Formats from Distributed Dbases

• Data Exchange Standards - A Common Shell

– Content in almost any format and from many different dbases (distributed) can

be linked to a shell with common tags

• Newly Shared Info May Still Require Restrictions

– Permissioning protects confidentiality and personalizes disclosures by user

– Should info disclosed be personalized by additional factors such as taxonomy,

content type, dept. or agency, geography or other?

• Accessibility and Usability

– How will users access the information? Web-based? Closed network?

– In general there is a conflict between strong Info Security and user friendliness

The product of different agencies and IT environments is made more

consistent and exchangeable by developing common standards.

CONCEPTUAL>ARCHITECTURE>DATA EXCHANGE

17

Design – Pages, Templates and Navigation that Display Information Best

• Info Architecture

– Site Map showing Pages of the Info Sharing System and the likely navigation

• Templates Map

– Number of distinct templates for pages and content displayed, e.g., Dashboard,

Profile Pages of auxiliary components (e.g., George Washington Bio) if

applicable, Alerts, search results

The effectiveness of a solution may hinge on how well it is organized into

discrete pages, such as a Dashboard and Search Results, with well

designed content which may include graphs, reports, profiles and alerts.

CONCEPTUAL>DESIGN

Templates Map - Example

18

CONCEPTUAL>DESIGN>TEMPLATES

Dashboard/Portal Map - Example

19

WebDev Folders/

Portal Page Groups

Items and

Sub-Pages

Portal Page

CONCEPTUAL>DESIGN>PORTAL

20

Functionality

• Automating Actions and Alerts

– Activity (or threat) profiled, e.g., stolen cards for electronic goods

– Variables or attributes that identify a profiled activity, e.g., buying flat screen TV

– Automating alerts and routing actions, e.g., requiring call to customer service

• Personalization

– Disclosure of info tailored to user with automated requests for more confidential

info routed to the originator when necessary

– Means of protecting data may be deemed insufficient by key contributors

• Admin Functions and Speed-to-Market

– Amount of control of contributor over content, tagging and permissions vs. IT

depts. will be controversial and impact on speed-to-market

– Audit trails and reporting are metrics to measure success and build confidence

Functionality varies greatly by enterprise, but in general a system should

highlight the latest content and incorporate Decision Tools.

CONCEPTUAL>FUNCTIONALITY

ECONOMICInhibitors to Information Sharing

21

22

1. Budget and Timeline

2. Documentation and Planning

3. Communication and Training

4. Maintenance and Enhancements

The internal budget process and policies on contracting agencies, vendors

and headcount often involve timeframes that defeat urgent projects.

Economic Inhibitors to Info Sharing

Accuracy of Financial Projections Hinges on Four Key Factors

23

Are the Budget and Timeframe Realistic?

• Introduce the Info sharing Solution in Logical Phases

– Hedge against budget overages by rolling out the solution in phases with

hooks for enhancements

• Timeframe

– Identify the internal processes for budgeting, submitting change-orders for a

project, that might need to be modified to meet deadline

• Using Contractors and Vendors

– The process for vendor selection, procurement, and the use of contractors

may need to be modified to meet deadlines

The lower the budget and more urgent the timeframe, the more important

the Inception Phase, workarounds and anticipation of obstacles.

ECONOMIC>BUDGET AND TIMEFRAME

24

A Proof of Concept and Detailed Description on Paper Will Save Time Later

• Documentation

– Temptation is to spend money on development alone

– Inception Phase and proof of concept are strongly recommended

– Documenting a project as it progresses requires resources from inception

• Project Planning

– The Project Plan is built around milestones and deliverables and the types of

inhibitors identified here should be considered in terms of timeline and budget

– Mid-development work will stop as unanticipated issues are addressed

– Avoid delays by identifying Decision-Makers at each constituent agency/dept,

and for the overall project, who are empowered to respond within 24 hours

ECONOMIC>DOCUMENTATION AND PLANNING

Requirements may change between initial requirements gathering and

implementation of new systems.

25

Constituents Using an Info Sharing Solution May Value Perceptions as Much as Reality

• Communication Plan

– Budgeting resources for a Communication Plan to promote the effort internally

is often overlooked but recommended in cases where Cultural and Personal

challenges loom large

– Constituents often value perceptions as much as substance

• Training

– New hires and existing personnel require education efforts about the use of

the Info Sharing Solution

Turnover within the enterprise may require a continual process of

consensus building and an on-going internal communications effort.

ECONOMIC>COMMUNICATION AND TRAINING

26

The Info Sharing Solution Will Require Significant Annual Support

• IT Enhancements

– Any solution will consist of hardware and software that will be upgraded over

the life-time of the system on an annual basis

• Solutions Degrade Overtime

– All underlying assumptions behind any advanced modeling techniques

require constant testing and refinement and this is particularly true in cases

where the macro-climate changes rapidly (e.g., a credit crisis)

Info Sharing Solutions introduced will require significant annual support,

equal to 15% of development cost on the IT side alone. A failure to

account for IT and model degradations can defeat the effort.

ECONOMIC>MAINTENANCE AND ENHANCEMENTS

TECHNOLOGICALInhibitors to Information Sharing

27

28

1. Design and Usability

2. Legacy Environments

3. Project Plan

Get the fundamental concepts or approach right at inception in order to live

within budgets and provide scalability and future functionality.

Technological Inhibitors to Info Sharing

Technology Decisions are Made Early Affecting Three Key Areas

29

Design and Usability Should be Considered From Inception

• Design

– Incorporate creative design from inception (often the „look‟ is slapped on last)

– Technical choices made at the beginning will limit Design choices later

• Info Security

– Will data be extracted from existing secure, token based environments?

– Will security protocols for sign-in and verification discourage use of the system?

• System Architecture

– Do you need a Database Management System (DBMS) that facilitates sharing

distributed dbases and conceptual modeling of data?

– Choose solutions that fit the projected scale of the production system identified

when considering the Scope of the project

– What are the performance requirements: Consider speed and timeouts?

– What are the benefits and limitations of various tech choices?

TECHNOLOGICAL>DESIGN AND USABILITY

A Semantic Data Model is Probably Best Suited for Advanced Info Sharing Solutions

• Traditional Database Management Systems (DBMS)

– Hierarchical, network or relational

• Semantic Data Model (aka Conceptual Data Model)

– Techniques to define the meaning of data within the context of its

interrelationships with other data

– First recognized by the U.S. Air Force in the mid-1970s as a result of the

Integrated Computer-Aided Manufacturing Program (ICAM)

– Conceptual schema control transaction processing in a distributed dbase

environment (e.g., U.S. Air Force Integrated Information Support System I2S2)

30

The logical data structure of a database management system (DBMS)

cannot totally satisfy the requirements for a conceptual definition of data.

TECHNOLOGICAL>DESIGN AND USABILITY>SEMANTIC DATA MODEL

31

Legacy Environments Create Migration Issues

• Diversity of Technology Environments

– What are the IT (or non-technology) environments that contribute data?

– How much data does each IT environment have to share and what is the

format, e.g., paper, .doc files, emails, other electronic, etc.?

– Are changes already scheduled that will inhibit or facilitate Info Sharing?

• IT Evangelism

– Is a particular contributing agency or individual tied to certain technologies and

vendors? Will this have an impact on the best Info Sharing solution?

• Window for Change

– It generally takes a lot of time to introduce big IT changes (Typically 18 mos)

– There is generally a small window to push IT changes without putting ongoing

operations at risk (e.g., avoid close of business each year)

IT departments at different agencies deploy and have resources

dedicated to support solutions provided by specific vendors (e.g., Oracle).

TECHNOLOGICAL>LEGACY ENVIRONMENTS

32

Data Exchange Formats Must Accommodate the Enhanced Functionality Required

• Migration of Legacy Data

– How will existing info in many formats and from different environments be

migrated to, or exchanged with, any joint Info Sharing System?

– Think through the best use of XML or other shells as first outlined in the section

on Conceptual inhibitors

• Blind Spots

– If information from a number of distributed databases is being shared

successfully what blind spots, if any, exist?

TECHNOLOGICAL>LEGACY SYSTEMS>DATA EXCHANGE

Ideally shells can be created and items tagged with a common taxonomy

automatically with human intervention limited to a relatively few exceptions.

CULTURALInhibitors to Information Sharing

33

34

1. Legally Dominated Environments

2. Info Security

3. Data Ownership

Cultural inhibitors are generally due to strong individuals within an

organization that set the tone. Many turf fights are well justified.

Cultural Inhibitors to Info Sharing

Three Basic Environments Impact Content Ownership

35

Legally Dominated Environments Tend to Discourage Info Sharing

• Fear of Discovery

– Information destroyed on a schedule to hedge against liability actions or other

judicial proceedings

• Sharing Proscribed

– Sharing of information may be prohibited, e.g., anti-trust regulations and

criminal proceedings

• Lengthy Review

– To provide for one institutional voice, designated parties review content

released to external parties adding days or weeks to a process

Collective experiences of companies, departments and individuals create

cultures that impede Info Sharing.

CULTURAL>LEGALLY DOMINATED ENVIRONMENTS

36

Info Security Dominated Environments are Built to Protect Data Not Share It

• Data Protection Paramount

– No tiered levels of data confidentiality since no breach is tolerated

– Not accessible by Internet and may require security tokens (e.g., SecurID or

biometrics)

• Hosting Environment Problematical

– Production environment not fully disclosed complicating development

– Applications of varying confidentiality levels housed on same servers (based on

resource management since all data is protected equally)

Any hacking of the environment or data destroys the culture and model so

all data is protected equally – complicating Info Sharing.

CULTURAL>INFO SECURITY

37

Data Ownership Regimes are Generally Meant to Restrict the Release of Information

• The Bureaucracy

– Several individuals sign off on data to be released internally or externally

• The Data Owner

– Within a dept., a Data Owner, controls info and its release

• The Chief Data Officer or Chief Information Officer

– An Executive-Level Manager develops and oversees compliance with policies

generally designed to restrict or protect data

Risk averse cultures make it unlikely that one individual can quickly sign off

on Info Sharing and the concurrence of several departments is required.

Identify the culture that you must engage and possibly change.

CULTURAL>DATA OWNERSHIP

PERSONALInhibitors to Information Sharing

38

39

1. Organizations Lose Interest

2. Reluctance to Share

3. Commissars and Apparatchiks

Personal Inhibitors to Info Sharing

Successfully implemented Info Sharing Solutions fail if people do not

contribute data or incorporate the system into the daily work-flow.

40

Organizations Lose Interest

• This Year‟s New Initiative

– New initiatives introduced by consultants every 12 – 18 months

– Info Sharing is one of many projects a jaded bureaucracy has seen before

• Turnover

– Managers move on to new roles or new organizations before completion

– Initial sense of urgency dissipates

Organizations can lose interest over time in projects that seem high

priority at any given moment.

PERSONAL>ORGANIZATIONS LOSE INTEREST

41

Reluctance to Share or Use a New Solution

• Undermining the Core Mission

– Sharing intellectual assets, especially outside the environments with which

people are familiar, is viewed as undermining the core mission

• Confidentiality of Sources at Risk

– Fear of losing control (and suffering the consequences)

– People who change jobs tomorrow need those promised confidentiality today

• Changing Personal Work Flow

– Individuals don‟t change their way of doing work to use, or contribute to, the

Info Sharing Solution

Info Sharing is neither the primary objective of the enterprise or the

individuals upon whom successful Info Sharing depends.

PERSONAL>RELUCTANCE TO SHARE

42

Commissars and Apparatchiks Survive Through Mastery of the Old Info Doctrine

• Commissars

– Greatest saboteurs negotiated their way through org. changes (purges) long

before the latest Info Sharing interlopers

– Info Sharing efforts require IT skills outside the comfort zone and decades-long

successful processes and survival techniques are challenged

• Apparatchiks

– Automatically enforce restrictive core business model or culture they know

– Value to organization is navigating arcane systems and procedures

An organization‟s trusted team may not facilitate Info Sharing projects.

Roadblocks may come from those who are very senior (commissars) or

very junior (apparatchiks) as survival techniques are challenged.

PERSONAL>COMMISSARS AND APPARATCHIKS

SOLUTIONSInhibitors to Information Sharing

43

44

Get Initial Concepts Right and Motivate the Right People

• Get Concepts and key taxonomy right upfront with a solid Proof of Concept

• Understand the benefits and limits of key tech decisions (e.g., DBMS)

• Identify how legacy info is migrated and data exchanged

• If sharing information from distributed databases what are the blind spots?

• Identify all business processes that must change

• Empower specific individuals to make project decisions within 24 hours

• Make the Info Sharing effort a critical part of job performance reviews

• Build consensus with a well executed Communications Plan

• Get the lead in an Info Sharing Project sufficient status (anticipate turnover)

• Use newly Shared Info to improve the product (e.g., Data Modeling)

SOLUTIONS>SUMMARY

For More Info –http://www.linkedin.com/in/wkitchenman

2010

Walter Kitchenman

wkitchenman@hotmail.com

Walter Kitchenman is an author and consultant on strategic issues in financial services. He

spent more than a decade as an international banker in Latin America and Europe and helped

launch the leading boutique advisory firm covering the strategic use of IT. Most recently he was

VP in charge of knowledge management at MasterCard Worldwide. He has a graduate degree

from Johns Hopkins School of Advanced International Studies (SAIS) and BA with special

honors from the Elliot School of George Washington University.