View
459
Download
3
Category
Preview:
Citation preview
1 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Security & Governance using
Apache Ranger & Apache Atlas
October 2016
Madhan NeethirajDirector - Engineering, Security & Governance
2 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Disclaimer
This document may contain product features and technology directions that are under development, may be under development in the future or may ultimately not be developed.
Project capabilities are based on information that is publicly available within the Apache Software Foundation project websites ("Apache"). Progress of the project capabilities can be tracked from inception to release through Apache, however, technical feasibility, market demand, user feedback and the overarching Apache Software Foundation community development process can all effect timing and final delivery.
This document’s description of these features and technology directions does not represent a contractual commitment, promise or obligation from Hortonworks to deliver these features in any generally available product.
Product features and technology directions are subject to change, and must not be included in contracts, purchase orders, or sales agreements of any kind.
Since this document contains an outline of general product development plans, customers should not rely upon it when making purchasing decisions.
3 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Agenda
• Introduction• Apache Ranger
• Overview• Authorization policies• Row-filter, Column-masking policies• Audit logs
• Apache Atlas• Overview• Lineage• Classification
• Demo• Q & A
4 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Apache Ranger: Overview
Centralized authorization and auditing across Hadoop components• HDFS, Hive, HBase, Knox, Strom, YARN, Kafka, Solr, ..• Audit logs to: Solr, HDFS, Log4j, ..
Access Authorization based on Resources, Resource Classification• Policies for specific set of resources – like a Hive database/table/column• Policies for resource classifications – like PII, PHI, PCI
Row-filter, Column-masking based on policies• Restrict the rows accessible in a table based on users/groups/runtime-context
• example: restrict users to access customer records for specific regions only• Mask or anonymize sensitive columns based on users/groups/runtime-context
• example: only last 4 digits of account number should be available to few user-groupsExtensible Architecture
• Custom policy conditions, context enrichers• Easy to enable Ranger authorization and auditing for new components
Encryption keys management to support Transparent Data Encryption
5 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Apache Ranger: Centralized Administration
Single pane of glass for security administration across multiple Hadoop components
6 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Apache Ranger: Authorization PoliciesConsistent authorization policy structure across Hadoop components
HDFS Resources
Users/Groups/Permissions
Hive Resources
Users/Groups/Permissions
7 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Apache Ranger: Row-filter, Column-masking Policies
Row Filter to apply
Mask to apply
8 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Apache Ranger: Tag-based Policies
Pick the tag
Deny access to data after expiry date with the exception of ‘admin’ user
9 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Apache Ranger: Access Audit Logs• Apache Ranger Plugins generate detailed audit logs of access to protected resources• Audit logs to multiple destinations: Solr, HDFS, Log4j appender• Interactive view of audit logs in Apache Ranger admin console
10 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Apache Ranger: Architecture
HBase
Ranger Administration Portal
HDFS
Hive Server2
Ranger Audit Server
Ranger Plugin
Had
oop
Com
pone
nts
Ent
erpr
ise
Use
rs
Legacy Tools and Data Governance
Knox
Ranger Policy Server
Storm
Solr
HDFS
Ranger Plugin
Ranger Plugin
Ranger Plugin
Ranger Plugin
Solr
YARN
Kafka
Ranger Plugin
NiFi
Atlas
Ranger Plugin
Ranger Plugin
Ranger Plugin
Ranger Plugin
Ranger UgSync
Ranger TagSync
LDAP/AD/OS
Atlas
11 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Apache Atlas: Introduction
Metadata Repository• Flexible type system to capture schema/metadata of multiple components• Out-of-box models for Hive, HDFS, Storm, Falcon, Sqoop
Data Lineage/Provenance• Captures data lineage across components
Classification• Use tags to classify the data – like PII, PHI, PCI, EXPIRES_ON• Support for attributes in tags – like expiry_date
Search• Search using classifications, attributes• Advanced search using DSL; convenient full-text search
Integrations• With Apache Hive, Apache Storm, Apache Falcon, Apache Sqoop for metadata and lineage• With Apache Ranger for classification based security
APIs to add support for more components
12 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Apache Atlas: Lineage
13 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Apache Atlas: Classification
14 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Apache Atlas: Architecture
15 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Demo
16 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Questions
17 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
References
18 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
References
• Apache Atlas• http://atlas.apache.org• http://hortonworks.com/apache/atlas
• Apache Ranger• http://ranger.apache.org• http://hortonworks.com/apache/ranger
• Apache Ranger wiki• https://cwiki.apache.org/confluence/display/RANGER
• Tag based policies• https://cwiki.apache.org/confluence/display/RANGER/Tag+Based+Policies
• Row-filtering and column-masking policies• https://
cwiki.apache.org/confluence/display/RANGER/Row-level+filtering+and+column-masking+using+Apache+Ranger+policies+in+Apache+Hive
Recommended