Embed Size (px)
DESCRIPTION
Government Information Sharing and Interoperability via Data Services and SOA. Rob Cardwell CTO, MetaMatrix rcardwell @ metamatrix.com. June 21, 2006. Agenda. MetaMatrix Architectural Fit MetaMatrix Products & Capabilities Achieving Information Sharing - PowerPoint PPT Presentation
Citation preview
June 21, 2006
Rob Cardwell
CTO, MetaMatrix
rcardwell @ metamatrix.com
Government Information Sharing and Interoperability via Data Services and SOA
2
Agenda
• MetaMatrix Architectural Fit
• MetaMatrix Products & Capabilities
• Achieving Information Sharing– Service Enabling Data Assets (Data Services)– Resolving Semantics– Enabling Enterprise (Deep) Search
• Customers
• Conclusions
3
MetaMatrix Enterprise Data Services
• Project-level or Enterprise-wide data services layer– Integrated views of data from multiple sources– Metadata-driven – Optimized performance– Interoperable security
• Complements BI, ETL, ESB/EAI, DQ, CDI, Search
4
MetaMatrix Fit in FEA DRM Data Sharing
DRM Version 2 Data Access Services• Context Awareness Services• Structural Awareness Services• Transactional Services• Data Query Services• Content Search and Discovery Services• Retrieval Services• Subscription Services*• Notification Services*
5
ESB
Data Service Layer in SOAClient Process & Applications
Data Sources
Data Services Layer
Message Services (ESB)
Business Services
Business Process Services
App App App App App App
Data Service Data Service Data Service Data Service Data ServiceData Service
6
Agenda
• MetaMatrix Architectural Fit
• MetaMatrix Products & Capabilities
• Achieving Information Sharing– Service Enabling Data Assets (Data Services)– Resolving Semantics– Enabling Enterprise (Deep) Search
• Customers
• Conclusions
7
MetaMatrix Integration Server
Information Consumers
Web Svc XML RDBMSPackaged Connectors
Siebel,SAP
OracleApps
IMSIDMS
MetaMatrix Catalog
MetaMatrix Designer- Design and deploy data services
MetaMatrix Products
JMSODBC JDBC SOAP
QueryProcessor
ProcessorProcessorOptimizerOptimizer
Integration ServerVirtualDataBases
VDBVDBVDBVDB
IntegratedSecurity
UsersUsers
RolesRoles
EntitlementsEntitlements
AccessModels
ViewsViews XMLDocsXMLDocs
<a>
</a>
<b>
</b>…
ServicesServices
in outproc
MetaMatrix Connector Framework
8
Designing data services
MetaMatrix Approach to Data Services
xml
databases
warehouses
spreadsheets
services
<sale/> <value/></ sale >
geo-spatial
rich media
…Enterprise Enterprise Information Information
Sources (EIS)Sources (EIS)
Information Information ConsumersConsumers
Reusable,Reusable,Integrated Data Integrated Data
ObjectsObjects
ExposedExposedDataData
ServicesServices
<WSDL><WSDL>(contract)
<WSDL><WSDL>(contract)
<WSDL><WSDL>(contract)
Custom Apps
Web Services,Business Processes
Packaged Apps
Reporting, Analytics
EAI, Data warehouses
OD
BC
JDB
CS
OA
P
Logistics
Intelligence
9
MetaMatrixEnterprise
MetaMatrixDimension
MetaMatrix Product Lines
MetaMatrix Enterprise • Web services & SQL• Modeling enterprise data• Scalable deployment server• Metadata management• Application/legacy connectors
MetaMatrix Dimension • Web service-enablement of data sources• Expose business views as XML• Lightweight modeling – rapid integration• Standard WAR-based deployment
Ent
erpr
ise
Ent
erpr
ise
Pro
ject
, Nod
eP
roje
ct, N
ode
10
Search Engine Index / Metadata Catalog
Master Data Person / Facility / Vehicle
MetaMatrix Enterprise
StageSOA
App’s
Federal Agencies
Data Access Services• SQL, Web Service/XML• Staged Data (optional)
OntologyMgmt /Reasoning
Enterprise Service Bus
MetaMatrix Dimension
State/Local
State/Local• Security/Authentication• Operations Management • Error / Exception Management
• Orchestration• Encryption• High Availability
MediationXSLT, Multi-source
Information Exchange Architecture
11
MetaMatrix Dimension Differentiators
Dimension adds the following capabilities to an ESB…• Rich, advisor-based, model-driven design tool• Ability to leverage data models and manage metadata• Clear way to visualize and define mappings between non-
XML sources and XML views (even for complex industry schemas – NIEM, GJXDM, HL7, XBRL)
• Ability to do SQL-based transformations, not just XSLT (including multi-source, complex joins and unions)
• Query planner/optimizer that makes intelligent decisions about whether to execute transformations “at the source” vs. “on the bus”
• Automated semantic matching & generation of transformations
Data Services to connect ESB’s to Enterprise Data
12
Agenda
• MetaMatrix Architectural Fit
• MetaMatrix Products & Capabilities
• Achieving Information Sharing– Service Enabling Data Assets (Data Services)– Resolving Semantics– Enabling Enterprise (Deep) Search
• Customers
• Conclusions
13
Mediation: XML From Non-XML Sources
«XML»
<person> <addresses> … </addresses> <accounts> <accountID=…> … </accountID> </accounts></person>
«Text File»
«Relational»
«Application»MetaMatrix:Mapping from Data to XML
Source: Data Sources containingInformation to integrate
Target: Fixed (complex) XML SchemaNeed:Data complying to schema’s vocabulary
T
14
• Model XML Docs, Schemas
• Build XML Doc. models from XML Schemas
• Map XML Doc. models to other data models
• Enable data access via XML
Map Data Sources to XML & Deploy
MetaMatrix Designer – for XML-centric Data Services
15
Rapid Web Service-Enablement
MODELWebservice
1
PACKAGEWebservice
2
DEPLOYWebservice
3
• Dimension Designer• Model-based • Expose multiple sources, integrated• Expose business views of data• Output = integration container (vdb)
• Standard WAR file bundling:• Integration container• Query engine
• Web service fully defined
• Deploy WAR file to Web Server• Executable Web service• Access via SOAP• Data exposed using standard
vocabulary
Web Svc App (WAR)Web Svc App (WAR)
MetaMatrixQuery Engine
Web Server
16
MetaMatrix Dimension Modeling
• Rapid design & deployment of Web Services• Expose integrated data as XML-based business views• Deployment of Web Services as standard Web apps• Runtime execution optimized through use of MetaMatrix Query Engine
Dimension Dimension ModelsModels
Web Server
Data Sources
Business Views
<XML><XML><XML>
Web Service Operations
WSDLXSD
Source Models
DeployImport Map Model
WARastoto
17
Dimension – Choose your approach
• Rapid design & deployment of Web Services• Expose integrated data as XML-based business views• Deployment of Web Services as standard Web apps• Runtime execution optimized through use of MetaMatrix Query Engine
Dimension Models
Web Server
Data Sources
Business Views
<XML><XML><XML>
Web Service Operations
WSDLXSD
Source Models
DeployImport Map Model
WARastoto
Start Here?
Start Here?
18
Secure Access – Accredited
MetaMatrixClient AppClient Appusernamepassword
Membership Provider
Membership Provider
usernamepassword
authenticates
Connector
Connector
Connector
Connector
Data Source
Optionally accessessource-specific information
source-specific
trustedpayload
MetaMatrixClient AppClient App
Membership Provider
Membership Provider
Authentication Service
Authentication Service
logoninfo
authenticates,generates payload
trustedpayload payload
trustedpayload
authenticates,optionally modifies payload
payload
Username/Password Logon • Connector connects with same ID for all queries• Optional: Integrated with existing authentication system
Trusted Payload Logon:• Connector uses different credentials per connection, per query • Optional: Integrated with existing authentication system
Data Source
19
Agenda
• MetaMatrix Architectural Fit
• MetaMatrix Products & Capabilities
• Achieving Information Sharing– Service Enabling Data Assets (Data Services)– Resolving Semantics– Enabling Enterprise (Deep) Search
• Customers
• Conclusions
20
T
Authoritative Sources:• Mapped to logical
Multiple Internal/External Information Sources
Application views of information:
• Relational, XML
T T
XML Document<a>
</a>
<b>
</b>…
T
TT
ODBC/JDBC JDBC SOAP
WebServices
WebServices
Search Applications
Search Applications
BusinessIntelligence
Applications
BusinessIntelligence
Applications
Logical Data Model:• Agency or COI-specific• Rationalize, harmonize,
mediate
C2, Logistics, Intelligence, …
COI Data Dictionary
bldg_id SITENUM Facility_ID
Location_ID
bldg_type Depot_Number
Location_Type
21
FBI CBP NYC NY NJ
SemanticData Services
Matched (Confidence of 90%)
Gender ID
Person Sex Code
Ontology
“Sex” semantically related to “Gender”
Semantic Matching - example
Data Sources
Semantic Data Services– key component of information sharing
and interoperability programs – automated semantic mapping to aid
domain experts in quickly reconciling disparate schemas and vocabularies
– more rapid deployment of a mediation solution
MatchIt – an extensible ontology-driven tool– variety of algorithms for determining
semantic equivalence– discovers similarities between
elements of heterogeneous data, automatically exposing potential semantic matches.
– matches elements of data sources to target schemas of Data Services, such as TWPDES, GJXDM, NIEM, C2IEDM, HL7
22
Agenda
• MetaMatrix Architectural Fit
• MetaMatrix Products & Capabilities
• Achieving Information Sharing– Service Enabling Data Assets (Data Services)– Resolving Semantics– Enabling Enterprise (Deep) Search
• Customers
• Conclusions
23
Person Search - Conceptual Use Case
EnterpriseInformation:AddressesOrganizationsAffiliationsAccountsTransactionsCall HistoryAgreementsPolicies
Relationships inherent in the search results link to enterprise apps, databases, and other repositories
24
MetaMatrix and Google
MetaMatrixServer
RDBMS
ERP, CRM…
ContentRepository
LegacySystems
GoogleSearchIndex(GSA)
ContentRepository
...
CustomApplication
ContentRepository
Text Search w/ filtering
criteria (optional)
Structured Data crawling & index
build
Navigate to related data from Search
UI
HTML I/F
HT
ML
I/F
JDBC
Connect
or
Fram
ew
ork
Select & drill down to discover record details,
related data links, & metadata
Field name look-up in
Business Data Dictionary
1
2
3
4
25
Agenda
• MetaMatrix Architectural Fit
• MetaMatrix Products & Capabilities
• Achieving Information Sharing– Service Enabling Data Assets (Data Services)– Data Service Design Approaches
• Customers
• Conclusions
26
Sample Customers/Awards
Government– DISA Global Combat Support Systems (GCSS)– Fort Meade– USTRANSCOM– USJFCOM– CIA/In-Q-Tel– SPAWAR CIPC– Mitre (various DoD initiatives)
Commercial– Merrill Lynch– Credit Suisse First Boston– SAP– Motorola
27
MetaMatrix provides best-of-breed components for rapidly deploying data services for information sharing
MetaMatrix is a proven entity within the Intelligence agencies, DISA, other DoD agencies, and commercial organizations
Unique value-add for mapping existing data assets to vocabulary standards adopted by COI’s
Agencies can avoid building MetaMatrix-like functionality via custom-coding, instead deploy solutions more quickly
MetaMatrix provides an agile infrastructure for information sharing, built on SOA standards
Conclusions
Government Information Sharing and Interoperability via
Data Services and SOA
June 21, 2006
Rob Cardwell
CTO, MetaMatrix
rcardwell @ metamatrix.com
