11
Federal Government Federal Government IT StrategyIT Strategy
Michael LangMichael LangJanuary 8, 2007January 8, 2007
2
BackgroundBackground► I founded Metamatrix eight years agoI founded Metamatrix eight years ago
The federal government became our The federal government became our largest customer by accidentlargest customer by accident
I have worked with dozens of federal IT I have worked with dozens of federal IT programs and with dozens of integratorsprograms and with dozens of integrators
Mostly interested in information Mostly interested in information management and systems architecturemanagement and systems architecture
Now concentrating on semantic Now concentrating on semantic technologytechnology
3
AgendaAgenda►Federal IT OverviewFederal IT Overview►Federal Enterprise ArchitectureFederal Enterprise Architecture►Net Centric Enterprise ServicesNet Centric Enterprise Services►Communities of InterestCommunities of Interest►Domain VocabulariesDomain Vocabularies►Semantic TechnologySemantic Technology
4
Federal IT InvestmentFederal IT Investment► Your Federal Government is doing billions of Your Federal Government is doing billions of
dollars of R&D in the IT areadollars of R&D in the IT area There are hundreds of IT programsThere are hundreds of IT programs Orion – NASAOrion – NASA Trailblazer, Groundbreaker – Ft MeadeTrailblazer, Groundbreaker – Ft Meade TTIC, US Visit – DHSTTIC, US Visit – DHS Sentinel, NDEX, RDEX – DOJSentinel, NDEX, RDEX – DOJ DLA IDE, GCSS, GCCS - DODDLA IDE, GCSS, GCCS - DOD
5
Federal IT CommunitiesFederal IT Communities► There are three distinct communities in the There are three distinct communities in the
Federal IT spaceFederal IT space IntelligenceIntelligence
► Looks a little like financial service firmsLooks a little like financial service firms Department of DefenseDepartment of Defense
► Looks most like commercial enterprisesLooks most like commercial enterprises CivilianCivilian
► All three have very different use cases and All three have very different use cases and agendasagendas
6
IntelligenceIntelligence►Pre 9-11 systems were all secure silosPre 9-11 systems were all secure silos
Sharing was avoidedSharing was avoided Security was paramountSecurity was paramount A lot of custom codeA lot of custom code Fair mix of structured and unstructured Fair mix of structured and unstructured
informationinformation Use case is “analysis”Use case is “analysis”
7
IntelligenceIntelligence► An Executive Order mandating information An Executive Order mandating information
sharing across the intelligence community sharing across the intelligence community was issued right after 9-11.was issued right after 9-11. Information sharing is now paramountInformation sharing is now paramount Metadata management is keyMetadata management is key Logical data models for each domainLogical data models for each domain Data is being exposed as servicesData is being exposed as services Progress is very slow because of security Progress is very slow because of security
concernsconcerns
8
Department of DefenseDepartment of Defense►Mission changed with the collapse of the Mission changed with the collapse of the
Soviet Union and the arrival of Don Soviet Union and the arrival of Don RumsfeldRumsfeld Much nimbler warfighterMuch nimbler warfighter Smaller missions, faster responseSmaller missions, faster response Requires better co-ordination between military Requires better co-ordination between military
branches and commandsbranches and commands Largely client serverLargely client server Mostly structured informationMostly structured information
9
Department of DefenseDepartment of Defense►Move to SOA is well under wayMove to SOA is well under way
Data being exposed as servicesData being exposed as services Registries and repositories proliferateRegistries and repositories proliferate Many domain data modelsMany domain data models Many, many efforts under way to achieve Many, many efforts under way to achieve
greater degrees of interoperabilitygreater degrees of interoperability Throw spaghetti at the wall and see what Throw spaghetti at the wall and see what
stickssticks
10
CivilianCivilian►Mission changed with the arrival of the Mission changed with the arrival of the
InternetInternet Executive order creates eGov initiativeExecutive order creates eGov initiative Citizen centric servicesCitizen centric services No sense of urgency hereNo sense of urgency here Relatively small budgetsRelatively small budgets
1111
FEA and NCESFEA and NCESFederal Enterprise ArchitectureFederal Enterprise Architecture
AndAndNet Centric Enterprise ServicesNet Centric Enterprise Services
12
Overarching ProgramsOverarching Programs► There are two long running, overarching IT There are two long running, overarching IT
initiatives whose goal is to re-engineer the initiatives whose goal is to re-engineer the federal government IT infrastructurefederal government IT infrastructure FEA, Federal Enterprise ArchitectureFEA, Federal Enterprise Architecture
► Managed by OMBManaged by OMB► Top downTop down
NCES, Net Centric Enterprise ServicesNCES, Net Centric Enterprise Services► Managed by DOD, DISAManaged by DOD, DISA► Bottom upBottom up
13
FEAFEA► This program began in 2002 as a result of an This program began in 2002 as a result of an
executive order from the White House that executive order from the White House that created the eGov initiative created the eGov initiative http://www.whitehouse.gov/omb/egov/http://www.whitehouse.gov/omb/egov/ ““To transform the Federal government to one To transform the Federal government to one
that is citizen-centered, results-oriented, and that is citizen-centered, results-oriented, and market-based, the Office of Management and market-based, the Office of Management and Budget (OMB) is developing the Federal Budget (OMB) is developing the Federal Enterprise Architecture (FEA), a business-Enterprise Architecture (FEA), a business-based framework for government-wide based framework for government-wide improvement.”improvement.”
14
Architecture Principles FEAPMOArchitecture Principles FEAPMO
► Motherhood and Apple PieMotherhood and Apple Pie The federal government focuses on citizensThe federal government focuses on citizens
The federal government is a single, unified enterpriseThe federal government is a single, unified enterprise
Federal agencies collaborate with other governments and Federal agencies collaborate with other governments and peoplepeople
Information is a national asset Information is a national asset
The federal architecture is mission-drivenThe federal architecture is mission-driven
Security, privacy and protecting information are core Security, privacy and protecting information are core government needsgovernment needs
The federal architecture simplifies government operations The federal architecture simplifies government operations
15
Business Reference Model (BRM)• Lines of Business• Agencies, customers, partners
Service Component Reference Model (SRM)• Service domains, service types• Business and service components
Technical Reference Model (TRM)• Service component interfaces, interoperability• Technologies, recommendations
Data Reference Model (DRM)• Business-focused data standardization • Cross-agency information exchanges
Business-Driven ApproachBusiness-Driven Approach
Performance Reference Model (PRM)• Inputs, outputs, and outcomes• Uniquely tailored performance indicators
Component-Based Architecture
Component-Based Architecture
FEA Reference ModelsFEA Reference Models
16
FEA Current StateFEA Current State►Even though there are budgetary Even though there are budgetary
enforcement procedures mandating enforcement procedures mandating agencies to begin implementation of agencies to begin implementation of the FEA, they are largely ignoredthe FEA, they are largely ignored
►The root of the problem is that the The root of the problem is that the architecture does not hang together architecture does not hang together and the prospective users know itand the prospective users know it
►The DRM is not credibleThe DRM is not credible
17
Data Reference ModelData Reference Model► I spent two years working on the DRM, it is the most I spent two years working on the DRM, it is the most
troublesome layer of the stacktroublesome layer of the stack► The DRM provides a standard means by which data The DRM provides a standard means by which data
may be described, categorized, and shared. These may be described, categorized, and shared. These are reflected within each of the DRM’s three are reflected within each of the DRM’s three standardization areas:standardization areas: Data Description: Provides a means to uniformly describe Data Description: Provides a means to uniformly describe
data, thereby supporting its discovery and sharing data, thereby supporting its discovery and sharing Data Context: Facilitates discovery of data through an Data Context: Facilitates discovery of data through an
approach to the categorization of data according to approach to the categorization of data according to taxonomies; additionally, enables the definition of taxonomies; additionally, enables the definition of authoritative data assets within a community of interest authoritative data assets within a community of interest (COI) (COI)
Data Sharing: Supports the access and exchange of data Data Sharing: Supports the access and exchange of data where access consists of ad-hoc requests (such as a query where access consists of ad-hoc requests (such as a query of a data asset), and exchange consists of fixed, re-of a data asset), and exchange consists of fixed, re-occurring transactions between parties occurring transactions between parties
18
NCESNCES► Net Centric Enterprise ServicesNet Centric Enterprise Services
NCES started at about the same time as FEA, but NCES started at about the same time as FEA, but is an initiative out of DISA (Defense Information is an initiative out of DISA (Defense Information Systems Agency) the CTO office of DOD.Systems Agency) the CTO office of DOD.
NCES does not pay much attention to FEANCES does not pay much attention to FEA► Global Information Grid – GIGGlobal Information Grid – GIG
Includes the physical networks and other Includes the physical networks and other hardwarehardware
19
NCES MissionNCES Mission► NCES will enable the secure, agile, robust, dependable, NCES will enable the secure, agile, robust, dependable,
interoperable data-sharing environment for DOD where interoperable data-sharing environment for DOD where warfighter, business, and intelligence users share warfighter, business, and intelligence users share knowledge on a global network. This, in turn, facilitates knowledge on a global network. This, in turn, facilitates information superiority, accelerates decision-making, information superiority, accelerates decision-making, effective operations and net-centric transformation.effective operations and net-centric transformation. To enable successful conduct of warfare and other operations To enable successful conduct of warfare and other operations
in the Information Age. in the Information Age. Make information available on a network that people can Make information available on a network that people can
depend upon and trust. depend upon and trust. Populate the DOD networks with new, dynamic sources of Populate the DOD networks with new, dynamic sources of
information to defeat the enemy. information to defeat the enemy. ► Sounds a lot like any commercial enterprise mission Sounds a lot like any commercial enterprise mission
statementstatement
20
NCES MissionNCES Mission► NCES represents a different approach to NCES represents a different approach to
building and fielding DOD Information building and fielding DOD Information SystemsSystems Market-based approach, recognizing that a Market-based approach, recognizing that a
user's information technology (IT) needs are user's information technology (IT) needs are dynamic and are rarely satisfied by systems dynamic and are rarely satisfied by systems built with a set of pre-determined user needsbuilt with a set of pre-determined user needs
Users themselves are best able to define their Users themselves are best able to define their requirements requirements
The NCES approach is DOD-wideThe NCES approach is DOD-wide It offers unprecedented access to information It offers unprecedented access to information
from global sources while leveraging existing IT from global sources while leveraging existing IT investmentsinvestments
21
NCES Current StateNCES Current State► Service Oriented ArchitectureService Oriented Architecture► A lot of the infrastructure is in placeA lot of the infrastructure is in place
Metadata catalogs/repositoriesMetadata catalogs/repositories Services RegistryServices Registry Tools for converting relational to XMLTools for converting relational to XML Tools for creating and publishing servicesTools for creating and publishing services XML Schemas describing domainsXML Schemas describing domains Quality of service softwareQuality of service software Security software and hardwareSecurity software and hardware GovernanceGovernance
22
NCES Current BottleneckNCES Current Bottleneck► InteroperabilityInteroperability
As soon as the number of services proliferateAs soon as the number of services proliferate The number of silos proliferateThe number of silos proliferate
► They are more granular but still hard to use and manageThey are more granular but still hard to use and manage
► Pulled a lot of the funding from programs Pulled a lot of the funding from programs that are creating “services”that are creating “services”
► Funding a lot of pilot projects to solve Funding a lot of pilot projects to solve interoperabilityinteroperability
23
Domain VocabulariesDomain Vocabularies► Early efforts used XML Schema and ER Early efforts used XML Schema and ER
diagrams to define the domain “data model”diagrams to define the domain “data model” Global Justice XSDGlobal Justice XSD National Information Exchange Model – NIEMNational Information Exchange Model – NIEM Command and Control – C2IEDMCommand and Control – C2IEDM
► Not extensible, not semanticNot extensible, not semantic► No connection between the businessperson No connection between the businessperson
and the dataand the data
24
Communities of InterestCommunities of Interest► Communities of Interest form to create domain Communities of Interest form to create domain
vocabulariesvocabularies All of the terms in a domainAll of the terms in a domain
► Data dictionary, logical model, schemaData dictionary, logical model, schema What they meanWhat they mean How they are usedHow they are used How they are relatedHow they are related
► The Domain vocabulary is the interoperability The Domain vocabulary is the interoperability master keymaster key All data elements in all systems are mapped to terms in All data elements in all systems are mapped to terms in
the domain vocabulariesthe domain vocabularies
25
Use of VocabulariesUse of Vocabularies Permit humans express their concepts in a Permit humans express their concepts in a
machine readable languagemachine readable language Enable machines to perform the data Enable machines to perform the data
translation and transformation required by translation and transformation required by data integrationdata integration
Vocabularies are the essential underpins to Vocabularies are the essential underpins to sharing data or system interoperability that sharing data or system interoperability that requires “dynamic links” among unknown, requires “dynamic links” among unknown, unlimited numbers of data sourcesunlimited numbers of data sources
Essential to all semantic technologies, Essential to all semantic technologies, including semantic searchincluding semantic search
26
SemanticsSemantics► Most programs have moved to OWL for Most programs have moved to OWL for
defining domain vocabulariesdefining domain vocabularies http://www.opengroup.org/projects/soa-ontology/http://www.opengroup.org/projects/soa-ontology/ http://osera.gov/web/guest/projects/fea-rmohttp://osera.gov/web/guest/projects/fea-rmo Flexible and extensibleFlexible and extensible Naturally distributed, URI and URLsNaturally distributed, URI and URLs Best design-time metadata representation modelBest design-time metadata representation model Machine readable at runtimeMachine readable at runtime Functions at the scale of the WWWFunctions at the scale of the WWW
27
Semantic Technology Semantic Technology StandardsStandards
OWL Ontology
W3C Semantic Technology Standards
28
Why Ontologies are so important
“An ontology is an abstract representation of concepts and their relationships that enables deductive and inferential reasoning upon itself.”
They are uniquely capable of creating relationships, otherwise impossible to identify on a mass scale, that explicitly reason for all relationships.
Solving Data Relationships Solving Data Relationships (Related)(Related)
Graph-Based Approach
Term 1
2 23 3
4 4TermSemantic “cluster”
29
MBI’s SOA-Enabled DoDIIS Data Layer
1. Use Ontology to semantically match elements across disparate sources2. Build virtual layer3. Service enable data layer
30
Government Leads the WayGovernment Leads the Way► Semantic technologySemantic technology► The government last led the charge with The government last led the charge with
relational database technology and IP networksrelational database technology and IP networks DARPA funded the R&D for RDBMS for 10 yearsDARPA funded the R&D for RDBMS for 10 years And then became the early adopterAnd then became the early adopter
► DARPA created OWL (DAML+OIL) eight years DARPA created OWL (DAML+OIL) eight years agoago Numerous projects funded to employ semantic Numerous projects funded to employ semantic
technologytechnology Just making it into operational systemsJust making it into operational systems
31
ConclusionsConclusions►Bottom up architectural approach Bottom up architectural approach
works better than top downworks better than top down►Communities will form and participate Communities will form and participate
in the construction of the system in the construction of the system especially the domain vocabulariesespecially the domain vocabularies
►The effort should and can include The effort should and can include business people, technology people business people, technology people and data peopleand data people
32
ConclusionsConclusions►For transactional systems, data is being For transactional systems, data is being
represented by XML and exposed as represented by XML and exposed as services (WSDL) in an SOAservices (WSDL) in an SOA Domain vocabulary is being described in Domain vocabulary is being described in
OWLOWL InteroperabilityInteroperability
►For analysis, data is being represented as For analysis, data is being represented as RDF and queried using SPARQLRDF and queried using SPARQL The ontology is the integration layerThe ontology is the integration layer
34
Discovering and Binding Discovering and Binding ServicesServices
MappingVocabularies
A & B
VocabularyA
VocabularyB
“same as”or
“same class as”
Web Services
Web Services
Mapping Vocabulary
generate generatedescribethe RDF
describethe RDF
Describe the structure(elements & attributes)
Describe the structure(elements & attributes)
XSD XSDXML Messages(in RDF XML) XML Messages
(in RDF XML)
WSDL WSDL
reference reference
describe describe
You can haveone or more
of these
35
Using Service ResponsesUsing Service Responses
RDF Content
RDF Content
RDF Content
RDF Contentfrom all
Responses
XML Messages(in RDF XML)
Web Services
extract
extract
extract
combine
combin
e
combine
KNOWNFACTS
36
Vocabularies(OWL)
Composed at design-time
“Semantic Interpreter” or“Semantic Message Translator”
Small wrapper around Jena
submit produce
QUERY(SPARQL)
KNOWNFACTS
NEXT SERVICEREQUEST MESSAGE
Designed to obtaindesired message
for next service call
Composed from previousmessages in a SOAtransaction plus assertions(facts) obtained fromother sources
37
Single Vocabulary/DictionarySingle Vocabulary/DictionaryComposite(s)
Fields
Valid Entries
Unit Identification Code
Sequential Location Number
Nationality:string(2)- enumeration value="AF"- enumeration value="AL"- enumeration value="AG"- enumeration value="AQ"- enumeration value="AN" …
Armed Service:string(1)- enumeration value="F"/>- enumeration value="A"/>- enumeration value="C"/>- enumeration value="B"/>- enumeration value="J"/> …
Sequential Location Number:integer- min value="0000"- max value="999999"- pattern value="[0-9]{4,6}"
Nationality
Armed Service
+ Other Metadata
38
Enterprise Enterprise Vocabulary/DictionaryVocabulary/Dictionary
USMTF Vocabulary
FieldsCompositesValid Entries
Link 16 Vocabulary
VMF Vocabulary
FieldsCompositesValid Entries
FieldsCompositesValid Entries
Enterprise VocabularyFields
CompositesValid Entries
T T
B D AT
A
39
Logical (Relationship) ViewLogical (Relationship) View► Reference Model for naming conventions, data-typing conventions, and Reference Model for naming conventions, data-typing conventions, and
business component structurebusiness component structure► Purely Conceptual -- Represents abstract view of data relationships within Purely Conceptual -- Represents abstract view of data relationships within
a vocabulary (cannot be queried from data)a vocabulary (cannot be queried from data)► Improves ability to manage change and support new virtual models more Improves ability to manage change and support new virtual models more
quicklyquickly
40
Info Exchanges/Use CasesInfo Exchanges/Use Cases
USMTF Vocabulary
FieldsCompositesValid Entries
Link 16 Vocabulary
VMF Vocabulary
EnterpriseVocabulary
SetsMessages
Web Services
A
B D A
FieldsCompositesValid Entries
FieldsCompositesValid Entries
FieldsCompositesValid Entries
CommunitySpecific
HarmonizedStandard
Views
Specific Information Exchanges (Messages/Virtual Models)