Copyright 2010 Avenue Open, Inc.
Open Source Enterprise Service Management (OSESM) System
Open Source Test, Monitoring, and Management System for the Application Tier
Presentation to MIL-OSS 2010
MIL OSS 2010 2
Agenda
• Background – Problem Statement
• Current Research Project
• Full System Vision
• Invitation to Collaborate
MIL OSS 2010 3
Distributed Computing and ESM
• Distributed Computing - Opaque inNature; Requires Application Visibilityo Debug, Test, Monitor, Manage
• Enterprise Service Management (ESM)Systems Emerged in Responseo First Generation – Actional, AmberPoint, others
• Market Consolidation – IBM, Oracle, Progresso Second Generation – Focus on Pinpointing Specific Problems
• Transactional Analysis (BTM)• Infrastructure• Code Analysis
MIL OSS 2010 4
ESM Market Landscape• Ever-Increasing Number of Things to
Monitor End-to-End AND Top-to-Bottomo Services, Distributed Transactions, Application
Components (Java, .NET, others)o Through ESBs, Process Engines, Supporting Infrastructureo Extend to Legacy, Military Tactical, other corner cases
• Significant Need for Cross-Layer Analytics and Root-Cause Analysis
However….
• The Gap between Customer Needsand ESM Vendor Support isWidening, not Decreasing
MIL OSS 2010 5
Imperfect Markets• All Market Options today are COTS, Closed Source
o Limited architectural and functional extensibilityo ESM Products can Only be Extended by the Vendor
• ESM Vendors are the Bottlenecko Proprietary APIs and Instrumentation - Vendor Must Buildo Instruments Hardwired – No Multi-Purpose or Mix & Matcho Limited Support Outside of Proprietary “Vendor Stack”
• Analysis is too Narrow and Requires Duplicate Infrastructureo One Tool for Infrastructure, One for App, One for Codeo Install Different Proprietary Agents for Every System – Even Within
Proprietary Vendor Stacks!
Result – As the Need for Application Monitoring is becoming More Critical to the Success of Distributed Computing, ESM Vendors are Less Willing and Able to Meet Customer Needs
MIL OSS 2010 6
A NEW WAY OF THINKING….
MIL OSS 2010 7
Open Source Enterprise Service Management System (OSESM)
1. Commoditize Proven Capabilities
2. Advance the State-of-the-Art
3. Dramatically Drive Down Cost
MIL OSS 2010 8
What We Are Researching (“R&d”) Today
Initial OSESM Framework1. Mechanisms to Collect Application Tier Data – Various Types
and Forms
2. System to Disseminate Data to the Appropriate Processing points
3. Analysis - Correlation and Root Cause Analysis System
Foundation for Comprehensive Application Runtime Test and Monitoring System
MIL OSS 2010 9
Research Specifics
• New and varied forms of Application Tier Instrumentationo Collectors – Collect Raw Data from Application Infrastructure and
underlying Components – from Application Server, OS, and related− Project to result in the development of Agents or similar− Applicability:
– When the Mission has control of the Technology– When deep, detailed information is required
o Reflectors – Collect Raw Data from Network Devices and related− Non-invasive – Can be used when Mission does not own technology end-
to-end− Higher level data, useful for Visualization, Message Tracking, Inter-System
Analysis
• Data Dissemination Techniqueso From Instruments to appropriate Analytical Engine(s)o Foundation for Federation, needed for “Mission Thread” scale
MIL OSS 2010 10
Advancements in Data Analysis Techniques
• Specifically Targeted to Understand Mission Thread Errors and Breakages
• Researching Sophisticated Correlation Systemo Correlate Interesting and Related Data Across All Tierso What Happened at Each Level at Any Given Point in Time
−Application−Operating System−Runtime Infrastructure−Code
• Root Cause Analysis Applied to Correlated Datao What Broke?o Where, Why, How - Specificallyo When - Proactive Notification, not post-facto Analysis
MIL OSS 2010 11
Project Deliverables• Open, published APIs – Instrumentation and
Analytical Serverso Various forms - SOAP/WSDL/XML, REST, Binary (for Low
Resource, Low Bandwidth situations or Legacy)o Various Levels – Application Layer and Supporting
Resources
• Reference Implementations and Affiliated Documentationo Instruments – Collectors and Reflectorso Intermediary Message Processoro Analytical Servers – Root Cause and Correlation
MIL OSS 2010 12
Benefits• Dramatically Lower Cost - Estimated TCO is 10% of today’s
COTS Solutions• Flat Out - A Better ESM
o Faster Resolution for Problems that fall between OS, Hardware, Network, Application, and Mission Layers
• Extensible - by the Communityo Instrumentation from the Community – Inexpensive, Accessibleo Build New Instruments as Needed – Info Readily Available
• Adaptable – Legacy systems, Packaged Applications, Tactical Military, Hardware Appliances, Cloud Computing, others
• Decouple Instrumentation from Analysiso Mix-and-Match best-of-breed solutions
• New or Expanded Uses – Possibilities includeo Testing & Development - Debugging Distributed Applicationso Application Security Monitoring – New or non-standard Usage Alerting,
Intruder Tracing
MIL OSS 2010 13
Vision and Future Objective• Framework is a Foundation, a Starting Point• Full Vision – Complete System for Testing,
Monitoring, and Managing Applications at Runtime• Additional Requirements for Full System
o Metrics Processingo Discovery Mechanismso Visualization Engineo Logging Facilitieso User Portalo Administrative Console
• Ultimate Objective – Full Open Source System, with a Repository for New Contributionso Open Licensingo Open Repository for New Contributions
MIL OSS 2010 14
Notional End-State Functional Architecture
Service EService D
Service C
Service B
Users
Service A C
C
C
C C
Persistence Store /LogsIntermediary Message Processor
UI / Portal and Reporting Engine
Network Switch R
C
R
- Collector- Reflector
Root Cause
Analysis
Correlation EngineMetrics Visualization
MIL OSS 2010 15
Current Research and Development
Service EService D
Service C
Service B
Users
Service A C
C
C
C C
Persistence Store /LogsIntermediary Message Processor
UI / Portal and Reporting Engine
Network Switch R
C
R
- Collector- Reflector
Root Cause
Analysis
Correlation EngineMetrics Visualization
MIL OSS 2010 16
Invitation to Collaborate
• Net Centric Computing is growing Increasingly more Complex
• OSESM Sorely Needed• We Need:
o Transition Partners – Programs or Organizations Interested in using the Results of the efforts of Current Project
o Collaboration Partners – Willing to Fund and/or Contribute to development of Full Vision System
• Interested? Please Speak to me After the Presentation
MIL OSS 2010 17
Wrap Up
• OSESM will Dramatically Improve the ability to Field and Manage Net Centric Applications and Mission Threadso Gather and Analyze Application Tier Data at Runtime
o Proactive Notification when Mission Thread Breaks
−What Broke, Where, How, Why
−At Every Level – Application, Operating System, Runtime Infrastructure, Code
o At Far Lower Costs – Estimated TCO 10% of COTS
• Initial Framework Under Development Today• Transition and/or Collaboration Partners Strongly
Encouraged!
Copyright 2010 Avenue Open, Inc.
John EmersonPresident, Avenue Open, Inc.
Contact Information
Recommended
