Design Summit - User stories from the field - Chris Jung

User Storiesfrom the field

3/17/13 Manage IQ Design Summit

Agenda

• Introduction

• Insight Europe

• Agile IT and DevOps

• Operators and Administrators

• Business impact

• Q&A


Who's Chris?

• Located in Germany

• Joined Red Hat in 2006 as Infrastructure Consultant

• Projects around System-Management, High availability, Linux Desktops, Identity Management

• Focus on Cloud and Virtualization in recent years

• Since April EMEA Technical Specialist CloudForms


What's his role?

• Support pre-Sales• Answer questions, support during scoping

• Run Demos or Proof of Concepts

• Support Consulting during delivery

Insight Europe


Insight Europe

• European customers are very sensitive about data protection and privacy• Where is my data stored? Who has access to it?

• Companies and public sector very sceptic about public cloud services

• Results in:High interest in private cloud or regional clouds


Privacy laws

• European law about personal data is very strict

• Some countries have even stricter laws

• Many differences between countries

• European law is setting minimum standards

• local law can be and often is more restrictive


Example: Personal data in Germany

• If a company allows private email usage, they automatically fall under the laws of postal secrecy

• They can no longer scan, read or analysis any mail (because they can not know if an email was private or work related)

• This creates special requirements for example for backups, SPAM and virus filters, out of office rules, data retention policies


Example: User Profiles

• Companies are not allowed to collect data which could potentially be used to create user profiles

• It is not even relevant if they actually use the data, storing the data already potentially violates the law!

• Log in times, logging of software usage, usage patterns, mail transfer logs, web proxies, ...


Personal data

• Microsoft has recently been forced by US authorities to provide personal data about end users even though the data is stored in the EU

• If they will lose this case and are forced to release the data, this will be another big blocker for public cloud adoption in Europe

• http://www.zdnet.com/blog/igeneration/microsoft-admits-patriot-act-can-access-eu-based-cloud-data/11225• http://www.washingtonpost.com/news/volokh-conspiracy/w

p/2014/07/07/what-legal-protections-apply-to-e-mail-stored-outside-the-u-s/

http://www.zdnet.com/blog/igeneration/microsoft-admits-patriot-act-can-access-eu-based-cloud-data/11225

http://www.zdnet.com/blog/igeneration/microsoft-admits-patriot-act-can-access-eu-based-cloud-data/11225

http://www.washingtonpost.com/news/volokh-conspiracy/wp/2014/07/07/what-legal-protections-apply-to-e-mail-stored-outside-the-u-s/




Impact on European Market

• This is actually a big opportunity for European or local cloud service providers

• They can argue that since they are not owned by an American company, they can not be forced by US authorities to release personal data

• The high bars set by law were often seen as a blocker

• Suddenly it's a competitive advantage

Impact on ManageIQ


Why is this a problem for MIQ?

• SmartState Analysis can fetch personal data from VMs• Desktop virtualization• Browser history, Registry settings, user generated content

• Mail Servers• Mail content, log files

• Log files can provide personal data• Log in times, user behavior, proxy logs

• Reports could be used to create user profiles• log in behavior (time, Geo location, ... )


But also an opportunity

• MIQ can help to respect European and local law

• Zones and Regions can be setup to adjust functionality to respect local law• e.g. SmartState Analysis could be disabled in specific zones or

regions

• Control policies can be used to cancel unauthorized activities• e.g. cancel SmartState Analysis based on tags by tagging all

virtual desktops, mail servers, proxies,....


Basic guidelines

• Data reduction• Always consider, do I really need this kind of user data or can I

complete the request without it?

• Anonymize data: statistical analysis is allowed

• Remove all references identifying a user, before storing the data record

• Use strong encryption• Clear text transfer is always forbidden if it contains user data

• Never store clear text credentials• use password attribute in MIQ instead

ÜbersetzungenConversioneThèmeTranslations


Localization

• A must have in some countries like France

• Nice to have for most German customers, but a challenge in the public sector

• Long term we need full UI localization

• Think beyond ASCII!


Localization today

• Localization works for • VMs

• Providers

• Users

• custom buttons

• service catalog items

• reports

• ....

... so we are almost there!

Special Industry requirements


Industry requirements

• Some industries have to meet additional requirements• financial sector

• Hosting or service providers

• regular audits to prove compliance with local and European law• Banks have to prove compliance by yearly audits

• BaFin (German Banking Supervision)


How MIQ can help

• control and compliance policies• verify compliance

• document compliance

• part of audit trail

• verify compliance• ShellShock

• Heartbleed

• SELinux Enforcement

• the next big thing


Summary

• Very sensitive about personal data• Where is it stored? Who has access?

• Complex law requirements

• Localization

• MIQ can help to address these!

Agile IT and DevOps


Agile IT and DevOps

• Internal IT is usually not fast enough to meet developer requirements

• Developers are under pressure to deliver results in short time frame

• Sooner or later Developers start using resources from AWS or Google• probably even with approval from their managers

• only way to meet schedule

• Cloud services put IT under specific pressure• “Why does IT need a week to build a VM while AWS only needs

minutes?”


MIQ can help

• reduce deployment time of VMs• Minutes instead of days

• Deep integration into internal IT tools and processes• No manual customization after deployment


Deep integration● integration to build

systems like Jason pre-configured

● backup, monitoring, etc. pre-configured

● developer tools like GCC, Java, IDE etc. are pre-installed

● VM with additional application (like DB, Application Server etc) can be ordered from a self service catalog

● IT will take care of security updates and bug fixes

● App will work in production without modifications


Summary

• Expectations on IT are growing

• IT tries to build compelling offers

• MIQ can help by automatically install standardized applications with deep integration into internal tools and processes!

• Potential integration with PaaS like OpenShift

Operators and Admins


Traditional IT

• Most customers are not Agile

• Some customers will never apply DevOps

• Often you just want to keep in full control of your IT infrastructure


Operators and Admins

• MIQ allows admins decide and influence how future software stacks will look like

• e.g. a software stack is split into frontend/middleware/backend• admins designs the stack and component is deployed into which

zone

• admins have control about deployed application versions, configurations, make sure proper firewall and/or load balancer rules are in place

• a full stack can be deployed by just ordering a single service catalog item or bundle


Stack Deployment


Operations and Admins

• Predictable and reliable platform for application development

• admin/ops still is in charge of what is deployed and where it is deployed• software release and lifecycle management, firewalls/load

balancers, monitoring etc.

• Integration with e.g. ticket systems can be used to track all requests created • will create automatic audit trail in external system about every

service request, lifecycle changes and retirement


Placement

• admins can define and enforce rules to meet their needs• placement polices: "not near" to avoid cluster nodes are running

on the same hypervisor

• placement to use "fast" storage only for I/O intensive workloads

• placement can also override manual migration of VMs


Placement “not near”

• Make sure certain workloads never run on the same hypervisor• Cluster pairs

• Applications with heavy I/O

• CPU intense applications

• Security requirements

• Use tagging to identify workload


Placement “not near”

• Based on existing code • Never re-invent the wheel!

• Fetch list of VMs for each hypervisor

• Check tags of each VM

• If match, skip hypervisor

• Result is a list of hypervisors which are good to go

Global Deployments


Global Deployments

• Customer is running multiple data centers around the globe

• Centralized management from “master” site

• MIQ for global deployments


Single entry point

• Admins and Ops are scheduling deployments from one entry point

• Deployment is orchestrated globally

• Heavy usage of SOAP API

• Automatic scaling• With geographical awareness

• Integrated with • License management (for proprietary software)

• Monitoring

• Configuration Management


Automatic Scaling

• Application has Management component which keeps tracks of Requests/s and Queue Length

• Management component calls MIQ to scale up based on actual load

• MIQ has multiple ways for scaling:• Dormant VM: Installed and configured, but powered off


Automatic Scale Up

• On scale up request:• Verify hard limit of maximum VMs has not been reached

• Check number of dormant VMs and bring up another one

• If number of dormant VMs is too low, create and install another VM and add it to the pool of dormant VMs

• Make sure minimum number of dormant VMs exists

• Active VM in Monitoring

• Keep track of license usage

• Add VM to CMDB


Automatic Scale Down

• On scale down request:• Make sure minimum number of running VMs exist

• Shut down not needed VMs and add to dormant pool

• If dormant pool is to large, retire VM (delete)

• Remove VM from monitoring

• Reduce License usage counter

• Deactivate in CMDB


New DC Deployments

• Automated process to bring up new DC's

• MIQ easy to deploy

• Configuration of MIQ via Puppet etc.

• Use MIQ to orchestrate VM deployment for new DC's• Minimum set of VMs for each application

• Infrastructure configuration (aka load balancers)

all orchestrated from “Master” Site


Summary

• MIQ allows admins/ops to define stack layouts• Instead of developers

• Admins have full control over used software versions, configs, infrastructure, ...

• MIQ helps to make sure they can stay in charge!

Business Impact


Business Impact

• Managers and budget owners want overview:

• How are my hypervisors utilized?

• When do I need more storage?

• Are all VMs compliant?

• Do I have enough resources for this new project?

• Do I have potential bottlenecks?

Reduces operational risks!


Business Impact

• Optimize and planning provides growth forecasts

• Automatic retirement can help to free up unused resources

• Right size recommendation can help to scale down over-sized VMs

Save

Money!!


Retirement logic

• Default retirement 6 months

• Notify user upfront

• Shutdown VM

• Move VM to...• Cheaper storage

• Backup

• And (optionally) delete VM!

• No more Excel sheets to track VM ownership and cleanup process


Business Impact

• reports can help to provide more transparency:

• report "non-VM" related disk usage to free up storage space

• color formatted reports highlight critical values• e.g. free disk space on a datastore is less than 10%

• reports on usage based on department, project, resource type • how much of my "expensive" storage is used and is it only used

for appropriate workloads?


Business Impact

• alerts can send out notifications on critical events

• compliance checks to avoid penalties

• Dashboards to provide role specific overviews


Summary

• Managers like• Reports

• Dashboards

• Trending

• PDF, Scheduler, Mail reports, ...

• MIQ can increase transparency

• Leads to more efficient infrastructure usage and saving money!

Wrapping things up


Wrap up

• Europe has specific challenges• Privacy, Regional differences, Localization

• Agile IT and DevOps give traditional IT a hard time• Rapid deployment, deep integration

• Admins can still be in control• Define stack deployments, enforce compliance and security

• Managers have better overview• Reports, Dashboards, Trending, Bottlenecks

Q&A

That's all folks!

Technology

Design Summit - User stories from the field - Chris Jung