Data-Ed Slides: Best Practices in Data Stewardship (Technical)

Best Practices in Data Stewardship

Copyright 2016 by Data Blueprint Slide #1

In order to find value in your organization’s data assets, heroic data stewards are tasked with saving the day- every single day! These heroes adhere to a data governance framework and work to ensure that data is: captured right the first time, validated through automated means, and integrated into business processes. Whether its data profiling or in depth root cause analysis, data stewards can be counted on to ensure the organization’s mission critical data is reliable. In this webinar we will approach this framework, and punctuate important facets of a data steward’s role.

Date: June 14, 2016

Time: 2:00 PM ET/11:00 AM PT Presenter: Peter Aiken, Ph.D. & Mike Ogilvie

Executive Editor at DATAVERSITY.net


Shannon Kempe

Commonly Asked Questions


1) Will I get copies of the slides after the event?

2) Is this being recorded?

Get Social With Us!


Like Us on Facebook www.facebook.com/

datablueprint Post questions and

comments

Find industry news, insightful content

and event updates.

Join the Group Data Management &

Business Intelligence Ask questions, gain insights and collaborate with fellow

data management professionals

Live Twitter Feed Join the conversation!

Follow us: @datablueprint

@paiken Ask questions and

submit your comments: #dataed

• 30+ years in data management • Repeated international recognition • Founder, Data Blueprint (datablueprint.com) • Associate Professor of IS (vcu.edu) • DAMA International (dama.org) • 9 books and dozens of articles • Experienced w/ 500+ data

management practices • Multi-year immersions:

– US DoD (DISA/Army/Marines/DLA) – Nokia – Deutsche Bank – Wells Fargo – Walmart – …

Peter Aiken, Ph.D.

Copyright 2016 by Data Blueprint Slide #

• DAMA International President 2009-2013

• DAMA International Achievement Award 2001 (with Dr. E. F. "Ted" Codd

• DAMA International Community Award 2005

PETER AIKEN WITH JUANITA BILLINGSFOREWORD BY JOHN BOTTEGA

MONETIZINGDATA MANAGEMENT

Unlocking the Value in Your Organization’s

Most Important Asset.

The Case for theChief Data OfficerRecasting the C-Suite to LeverageYour Most Valuable Asset

Peter Aiken andMichael Gorman

5

Mike Ogilvie• 15+ years experience in data-centric solutions

• Architecture/Design experience in: • Data Warehouse

• Data Integration

• Data Quality

• Solutions/Data/Consulting experience for numerous government and commercial clients

• B.S. Physics - James Madison University

• Focus on Data Governance, Data Stewardship, Data Quality, and requirements consulting


Presented by Peter Aiken, PhD & Mike Ogilvie


With Great Data Comes Great Responsibility



1. Data Management Overview

2. Business needs for Data Stewardship

3. Data Stewardship Principles

4. Opportunities to foster a data-driven culture

5. Take Aways, References & Q&A

Tweeting now: #dataed

9Copyright 2016 by Data Blueprint Slide #

Best Practices in Data Stewardship1. Data Management Overview






UsesUsesReuses

What is data management?


Sources

Data Engineering

Data Delivery

DataStorage

Specialized Team Skills

Data Governance

Understanding the current and future data needs of an enterprise and making that data effective and efficient in supporting business activitiesAiken, P, Allen, M. D., Parker, B., Mattia, A., "Measuring Data Management's Maturity: A Community's Self-Assessment" IEEE Computer (research feature April 2007)

Data management practices connect data sources and uses in an organized and efficient manner • Engineering • Storage • Delivery • Governance

When executed, engineering, storage, and delivery implement governance

Note: does not well-depict data reuse

What is data management?


Sources

Data Engineering

Data Delivery

DataStorage


Resources

(optimized for reuse)

Data Governance

Ana

lytic

Insi

ght


You can accomplish Advanced Data Practices without becoming proficient in the Foundational Data Practices however this will: • Take longer • Cost more • Deliver less • Present

greaterrisk(with thanks to Tom DeMarco)

Data Management Practices Hierarchy

Advanced Data

Practices • MDM • Mining • Big Data • Analytics • Warehousing • SOA

Foundational Data Practices

Data Platform/Architecture

Data Governance Data Quality

Data Operations

Data Management Strategy

Technologies

Capabilities


Data$Management$Strategy

Data Management GoalsCorporate CultureData Management FundingData Requirements Lifecycle

DataGovernance

Governance ManagementBusiness GlossaryMetadata Management

DataQuality

Data Quality FrameworkData Quality Assurance

DataOperations

Standards and ProceduresData Sourcing

Platform$&$Architecture

Architectural FrameworkPlatforms & Integration

Supporting$Processes

Measurement & AnalysisProcess ManagementProcess Quality AssuranceRisk ManagementConfiguration Management

Component Process$Areas

DMM℠ Structure of 5 Integrated DM Practice Areas


Data architecture implementation

Data Governance

Data Management

Strategy

Data Operations

PlatformArchitecture

SupportingProcesses

Maintain fit-for-purpose data, efficiently and effectively

13

Manage data coherently

Manage data assets professionally

Data life cycle management

Organizational support

Data Quality

Data Management BoK The DAMA Guide to the Data Management Body of Knowledge Published by DAMA International • The professional

association for Data Managers

• 40 chapters worldwide DMBoK organized around • Primary data

management functions focused around data delivery to the organization

• Organized around several environmental elements


Data Governance from the DMBOK


from The DAMA Guide to the Data Management Body of Knowledge © 2009 by DAMA International















The Challenge of Managing DataCurrent state ! isolated databases and personal definitions • Stunted ability to

communicate • Lost documentation • Inconsistency across

departments

Ideal state ! unified organizational outlook • Regimented asset

ownership • Consistent names &

definitions • Meaningful data &

metadata


The REALITY of Managing Data:

*But it can’t fulfill its potential without careful maintenance of its source and storage…


DATA IS AN

ASSET*

The Solution: Data Governance• What is it?

• DAMA

– The exercise of authority, control, and shared decision-making (planning, monitoring, and enforcement) over the management of data assets

• Robert Seiner (TDAN and KII Consulting)

– The exercise and enforcement of decision-making authority over the management of data assets and the performance of data functions”

• Steven Adler (IBM)

– Coordinating communication to achieve collective goals through collaboration

• What is it, really?

– A custody battle over data instead of kids

– Critical to organizational success


What is Data Governance?


Managing Data with Guidance

In Practical Terms…

Managing Data with Guidance

Data governance decides who is responsible for the stewardship of data and metadata, and how they

collectively represent the organization


•Data & metadata capture & use

• Data quality -> measurement, optimization, and improvement

• Data policies & procedures

• Constant management and process evolution

Correctimplementations

Correctfunctionality

Correct designs

Correct specifications

Implementations based on

erroneous design

Implementations based on

erroneous specs

Incorrect implementations

Uncorrectableerrors

Hiddenerrors

Correctable functionality

(Adapted from [Mizuno 1983] as reproduced by Davis 1990.)

Design

Implementation

Requirements

Testing

imperfect program products

Data Stewardship ! Data Quality


Erroneous designs

Erroneous specifications

the "real" problem

Designs based on erroneous specs

23

Relative Cost/Effort to Repair System in Relation to Development Stage


(Adapted from [Davis 1990.)

$0.00

$2.00

$4.00

$6.00

$8.00

$10.00

$12.00

$14.00

$16.00

$18.00

$20.00

Coding Unit Test

Acceptance Test Maintenance

Nearly 50% of problems are detected only after

completion of acceptance tests

Requirements

Design

24















Operational Modes


LegendDecentralizationCentralizationInteraction

Organization Design

Organization & RolesProcesses & Procedures

Maintenance & Usage

Policy & Standards

“Totally Centralized”“Totally Decentralized” “Federated” “Centralized-Hybrid”

Degree of CentralizationNone Total

???Preliminary Recommendation of Target State???

Centralized Data Governance structure involves data stewardship councils organized into functional domains and managed centrally by the CDO organization

Federated Data Governance structure. Involves each functional data domain to be managed independently by the respective business lines. Aligned to a central data governance working group broken down by the core governance functions.

A hybrid centralized / federated model allows each LoB to manage the respective stewardship councils while being accountable to the central governance organization. The central CDO organization will manage enterprise-wide level data domains.

Metrics & Reporting

Decentralized Data Governance Structure. Involves each functional data domain to be managed independently by the respective business lines for all core governance functions. Limited alignment to a central governing body. 6

Current State

In Practical Terms (Again)…


All Ships Need Data Stewards

• Data stewardship

– the operational aspect of data governance

• Formalized accountability for data management

• Facilitates day-to-day actions & aspects of a data governance program

• Data stewards can extract knowledge from and make decisions regarding data assets and the people who develop & utilize them

• Formalization of unformalized tasks

– for which specific training is still required


A recent job posting Mid Career (2+ years of experience)

• The Data Governance Analyst IV will support the development and implementation of SCUSA’s data governance management and functional system administration across the organization.

• The person in this role will be a SME and the primary data steward for a particular domain within the Enterprise Data Governance organization and will support key initiatives especially the Risk Data Aggregation, Risk Data Reporting, FRY14 and Capital Plan in addition to firm-wide strategic projects with data impacts.

• The right candidate will have an opportunity to join a growing team and contribute towards shaping the operating policies and procedures.

• Master’s degree or MBA a plus.

• Eight (8) to ten (10) years of experience querying data, identifying anomalies, gaps and issues preferred.


Essential Functions:• Become a SME in the assigned area and assist Data Quality team with establishing data quality rules, thresholds and

quality reports and dashboards. • Serve as the primary data steward for a particular area / domain across the auto finance and unsecured lending business

lines. • Provide analytical support to Business partners to identify critical data elements for the projects in scope, perform initial

analysis and partner with MIS and Business teams to define and capture metadata, lineage, business rules and transformation logic.

• Own end-to-end governance process for all identified domains across the domain, plan and drive group / 1:1 discussions. Represent the domain in data governance working meetings and operating committee.

• Manage risk exposure by analyzing root cause for issues identified and engage the right parties across the teams with recommended strategies for timely resolution.

• Input and maintain the metadata into Informatica metadata tool and work with IT to establish processes for on-going maintenance and setting up exception reporting.

• Develop key relationships with SME’s and data owners and drive discussions to standardize / rationalize attributes & metrics and assist with establishing clear ownership of data elements.

• Collaborate with reporting analysis and data strategy functions to represent needs of the domain, assess and identify impacts across multiple complex projects.

• Contribute to the development of training materials and deliver / coach 1:1 or small groups on the new processes across the data governance lifecycle.

• Escalate project timeline and quality issues appropriately to ensure overall program success. • Supports the development and implementation of new data policies. • Supports the creation of program business definitions and data management goals and principles for execution. • Performs data analysis for various enterprise wide data quality initiatives. • Positions business areas for successful audits and regulatory exams by supporting the implementation of industry

guidelines and ‘best practice’ frameworks. • Coordinates the analysis of data gaps by collaborating with appropriate data owners and business partners. • May lead or direct the work of junior analysts.


Requirements:• Bachelor’s degree required in BA/BS Computer Science / Information System /

MIS/ Mathematics/ Statistics / Operations Research/ Economics; or equivalent combination of education and experience, required.

• Eight (8) to ten (10) years of related professional experience working with MIS, Databases, process management areas.

• Advanced understanding of a Business-Technology function, based on prior experience working across Business and IT areas.

• Innovative sense of business processes and demonstrated ability to link to the datasets and vice-versa.

• Advanced knowledge of Excel, Access, Visio, Powerpoint and SharePoint. • Demonstrated ability to turn ideas into visual representation for conveying complex

data issues to senior business partners. • Demonstrated ability to thrive in a demanding environment and excellent

collaborative skills to quickly establish partnerships across various stakeholders. • Advanced interpersonal, negotiation and collaboration skills. • Time management skills with strong attention to details. • Ability to work independently and manage multiple task assignments, with guidance

in only the most complex situations. • Ability to maintain confidentiality.


Working Conditions:• Extended working hours may be required by management and

business needs.

• Travel to multiple facilities may be required.

• May be required to lift, push, or pull materials weighing up to twenty (20) pounds.

• May be required to sit and review information on a computer screen for long periods of time.

• May require repetitive motions of the hands and wrist related to writing and typing at an electronic keyboard.

• Corporate / satellite office role.

• This job description does not list all the duties of the job. You may be asked by your supervisors or managers to perform other duties. You will be evaluated in part based upon your performance of the tasks listed in this job description.


1. Specific background in data is not required

2. Describes a relationship to data–not necessarily a position

3. A data steward is ideally dedicate to that role

4. Does not need to have the title

5. Public or Industry Data Steward Certification is maturing

6. Multiple steward types are possible for maturing organizations/operations

7. Training should be focus on formalizing accountability.

• adapted from http://tdan.com/seiners-rules-for-becoming-a-data-steward/16867

Becoming a Data Steward


Data Steward Types: Basic• Business data stewards

– Manage from the perspective of business elements (i.e. business definitions, data quality)

• Technical data stewards

– Focus is on use of data by systems and models (i.e. code operation)

• Project data stewards

– Gather definition, data quality rules, and project issues for referral to business and technical data stewards(Definitions adapted from Plotkin: Data Stewardship: An Actionable Guide to Effective Data Management and Data Governance)


Data Steward Types: Ancillary• Domain data stewards

– Manage steward data required across multiple business areas (i.e. customer data) and metadata documentation


• Operational data stewards – Responsible for directly

inputting data or instructing those who do; provide assistance to business data stewards in spotting data issues and identifying their root causes(Definitions adapted from Plotkin: Data Stewardship: An Actionable Guide to Effective Data Management and Data Governance)

Data Steward Responsibilities


Data Stewards Must Be…Accountable

• Focus on meaning and quality

• Identify needs for “official” process

• Operate as single point of contact for data owned by a business function


Data Stewards Must Be…Authoritative

• Answer questions about steward’s data

• Assign and oversee data-related work

• Enforce decisions


Data Stewards Must Be…Organized • Need to minimize

negative effects of: – Changes to business

processes – Transformations from

system to system

– Source data problems – Definition uncertainty – Changes to technical

platforms and processes • Lack of accountability +

failure to communicate = data surprises ! lack of trust in data & results
















Data Stewardship Implementation OptionsMinimally Intrusive • Identifying people into roles rather than assigning them • Leveraging existing responsibility Command and Control • Assigning people into roles • Giving people new responsibilities “2 x 4” • Data governance is “not optional” • People will have to make time


What does a data steward do?


OPERATIONALIZE

Proactive approach to unearthing good stewards• Build governance into what the data

stewards do

• Governance Activity Matrix

• Governance Processes RACI Chart

• System Development Life Cycle (SDLC)

• Project Planning

• Avoid referring to them as “data governance processes”(Adapted from http://tdan.com/seiners-rules-for-becoming-a-data-steward/16867)


Reactive Approach to Identifying Data StewardsFollow a data quality methodology

1. Qualify & prioritize data issue

2. Identify affected data domain & stewards

3. Conduct data systems & resource discovery, root cause analysis, and cost-benefit analysis

4. Analyze & recommend resolution

5. Gain approval, funding, and resources

6. Resolve data issue(Adapted from http://tdan.com/seiners-rules-for-becoming-a-data-steward/16867)


Data Stewardship Drives Data Culture


Data Stewardship Drives Data Culture


The Basis

Data$Management$Strategy

Data Management GoalsCorporate CultureData Management FundingData Requirements Lifecycle

DataGovernance

Governance ManagementBusiness GlossaryMetadata Management

DataQuality

Data Quality FrameworkData Quality Assurance

DataOperations

Standards and ProceduresData Sourcing

Platform$&$Architecture

Architectural FrameworkPlatforms & Integration

Supporting$Processes

Measurement & AnalysisProcess ManagementProcess Quality AssuranceRisk ManagementConfiguration Management

Component Process$Areas

DMM℠ Structure of 5 Integrated DM Practice Areas


Data architecture implementation

Data Governance

Data Management

Strategy

Data Operations

PlatformArchitecture

SupportingProcesses

Maintain fit-for-purpose data, efficiently and effectively

49

Manage data coherently

Manage data assets professionally

Data life cycle management

Organizational support

Data Quality

One concept for process improvement, others include:

• Norton Stage Theory

• TQM

• TQdM

• TDQM

• ISO 9000

and focus on understanding current processes and determining where to make improvements.

DMM Capability Maturity Model Levels

Our DM practices are informal and ad hoc, dependent upon "heroes" and heroic efforts

Performed (1)

Managed (2)

Our DM practices are defined and documented processes performed at the

business unit level

Our DM efforts remain aligned with business strategy using standardized and consistently implemented practices Defined

(3)

Measured (4)

We manage our data as a asset using advantageous data governance practices/structures

Optimized

(5)

DM is strategic organizational capability, most importantly we have a process for improving

our DM capabilities


Assessment Components

Data Management Practice Areas

Data Management Strategy

DM is practiced as a coherent and coordinated set of activities

Data Quality

Delivery of data is support of organizational objectives – the currency of DM

Data Governance

Designating specific individuals caretakers for certain data

Data Platform/Architecture

Efficient delivery of data via appropriate channels

Data Operations Ensuring reliable access to data

Capability Maturity Model Levels

Examples of practice maturity

1 – PerformedOur DM practices are ad hoc and dependent upon "heroes" and heroic efforts

2 – ManagedWe have DM experience and have the ability to implement disciplined processes

3 – Defined

We have standardized DM practices so that all in the organization can perform it with uniform quality

4 – Measured

We manage our DM processes so that the whole organization can follow our standard DM guidance

5 – Optimized We have a process for improving our DM capabilities


Data Program Coordination

Organizational Data Integration

Data Stewardship

Data Development

Data Support Operations

Data Management Maturity Measurement


Focus: Implementation and

Access

Focus: Guidance and

Facilitation

Optimizing (V)

Managed (IV)

Documented (III)

Repeatable (II)

Initial (I)

• CMU's Software Engineering Institute (SEI) Collaboration

• Results from hundreds organizations in various industries including: – Public Companies – State Government Agencies – Federal Government – International Organizations

• Defined industry standard • Steps toward defining data management

"state of the practice"

A Data Stewardship Maturity Model



How It Works

Top-down • Stemming from

executive vision & direction

Bottom-up • Initiated within

execution teams and then adopted by C-level


the Data Doctrine(.com)

We are uncovering better ways of developingIT systems by doing it and helping others do it.

Through this work we have come to value:

Data programmes preceding software development Stable data structures preceding stable code

Shared data preceding completed software Data reuse preceding reusable code

That is, while there is value in the items on

the right, we value the items on the left more.




1. Data Management Overview






Benefits of a Stewardship Program

• Programs are ongoing; projects end

• Programs are tied to the financial calendar

• Program management is governance-intensive

• Programs have greater scope of financial management

• Program change management is an executive leadership capability


Questions?

It’s your turn! Use the chat feature or Twitter (#dataed) to submit

your questions to Peter & Mike now!

+ =


Upcoming Events

Exorcizing the Seven Deadly Data Sins December 13, 2016 @ 2:00 PM ET/11:00 AM PT

Sign up here: www.datablueprint.com/webinar-schedule or www.dataversity.net


10124 W. Broad Street, Suite C Glen Allen, Virginia 23060 804.521.4056

Copyright 2016 by Data Blueprint Slide # 61