Upload
austin-ward
View
217
Download
0
Embed Size (px)
Citation preview
IC Analysts &The AQUAINT Program
Dr. John D. PrangeAQUAINT Program Manager
http://www.ic-arda.org
2
• For ARDA and AQUAINT they are:– Intelligence Community (IC) Analysts
• For the Non-IC World there is (potentially) a similar type of Target Audience:– “Professional Information Analysts”– For Example:
• Investigative / “CNN-type” Reporters• Financial Industry Analysts / Investors• Historians / Biographers • Lawyers / Law Clerks• Law Enforcement Detectives• And Others
Target Audience for AQUAINT:Who are They?
3
Taking a Closer Look
What Do We See When We FocusDirectly In On OurIC Analysts?
IC Analysts
4
• First: There are different level of intelligence within the IC -- Strategic, Operational, Tactical.
– ARDA is focusing on Strategic Level IA’s
• Second: There is no stereotypical analyst even within our Strategic Level Intelligence Agencies.
– Clear, significant differences exist:: • Across the national IC agencies as well as within individual agencies• Across Single Discipline “INT” Analysts• Between Single “INT” Analysts and All Source Analysts
– These differences are greatly accentuated by thebreadth of all IC reporting requirements.
• Third: There are significant skill level differences among IA’s
– Yes, the most seniors IA’s are exceptional– But the junior IA’s aren’t bad either
On the One Hand:Major Differences Do Exist Among Our IC Analysts
IC Analysts
5
• We believe that these similarities are significant and strong enough that:
– Taken collectively they highlight key differences between Intelligence Analysts and the “Casual” Consumers of Information that is being fueled by the Information Revolution and targeted by the commercial world
– A common set of critically important Info-X problems for the IC can be identified and articulated
– Multi-agency R&D programs against these common Info-X problems can be developed to the benefit of all IC Agencies
IC Analysts
VS.
“Casual” Consumers Of Information
But If We Change Our Focus:Universal Similarities Can Be Identified
6
• They are far more than just casual users of information
• They work in an information rich environment where they have access to large quantities of heterogeneous data
• They are almost always subject matter experts within their assigned task areas
• They track and follow a given event, scenario, problem, or situation for an extended period of time
• They frequently have extensive collaboration with other analysts
• They are focused on their assigned task or mission and will do whatever it takes to accomplish it
• The end product that results from their analysis is often judged against the standards of:
Timeliness Accuracy UsabilityCompleteness Relevance
Intelligence Community Analysts:What do They have in Common?
IC Analysts
7
Information Exploitation (Info-X)
Presentation and Visualization
Information Discovery
AnalyticKnowledge
Information Retrieval
InformationUnderstanding
Assessmentand
Interpretation
Content Data Mark-up
Content Data Transformation
Synthesis and Fusion
IC Analysts
Data Filtering& Selection
Reporting and Dissemination
What Functions Does It Include?
Info-X is Focused on Content & Its Meaning!
8
Intelligence Community & The Intelligence Cycle
7Dissemination
2Planning &Direction
4Processing
3Collection
5Analysis
6Reporting
1Intelligence
Requirements
Focus of Info-X
9
“Information Spectrum”
Information Exploitation (Info-X)Its Role Within the Intelligence Cycle
Collection Processing Analysis Reporting
Part of the “INTELLIGENCE CYCLE
Raw Data
BITS
SIGNALS
SENSOR
PHYSI
CAL
DATA
OBJECTS
IM …
SIG …
HUM …
MAS …
OS ...
10
Collection:Five Basic Intelligence Sources or Disciplines
• Imagery Intelligence (IMINT)– Includes both overhead and ground imagery of all types– NIMA is IC functional manager for IMINT
• Signals Intelligence (SIGINT)– Information derived from intercepted communications, radar, and
telemetry– Includes subdisciplines of COMINT (COM=Communication), ELINT
(EL=Electronic), FISINT (FIS=Foreign Instrumentation Signals)– NSA is IC functional manager for SIGINT
• Measurement & Signature Intelligence (MASINT)– Technically derived intelligence data other than imagery and SIGINT– Employs nuclear, optical, radio frequency, acoustics, seismic and
material sciences– e.g. Distinctive radar signature of a specific aircraft system or
chemical composition of air and water samples– Central MASINT Organization (component of DIA) is focus for all
MASINT activities
11
• Human-source Intelligence (HUMINT)– Involves clandestine and overt collection techniques used mainly by
CIA, FBI, and the Defense & State Departments– Principal types of HUMINT collection:
• Clandestine source acquisition of information• Overt data collection by civilian and military personnel assigned to
US diplomatic and consular posts• Debriefing of foreign nations and of US citizens who travel abroad
and who access to foreign information• Official contacts/exchanges with foreign governments
• Open-source Intelligence (OSINT)– Publicly available information appearing in print or electronic form– Includes radio, television, newspapers, journals, internet, commercial
databases, videos, graphics, drawings– Major collectors are FBIS (Foreign Broadcast Information Service) &
NAIC (National Air Intelligence Center)
Collection: Five Basic Intelligence Sources or Disciplines (continued)
12
“Information Spectrum”
Information Exploitation (Info-X)Its Role Within the Intelligence Cycle
Collection Processing Analysis Reporting
Part of the “INTELLIGENCE CYCLE
Presentation and Visualization
Information Discovery
AnalyticKnowledge
Information Retrieval
InformationUnderstanding
Assessmentand
InterpretationContent Data
Mark-up
Content Data Transformation
Synthesis and Fusion
Data Filtering& Selection
Reporting and Dissemination
IC Analysts
InformationRaw Data
BITS
SIGNALS
SENSOR
PHYSI
CAL
DATA
OBJECTS
IM …
SIG …
HUM …
MAS …
OS ...
13
“Information Spectrum”
Information Exploitation (Info-X)Its Role Within the Intelligence Cycle
Collection Processing Analysis Reporting
Part of the “INTELLIGENCE CYCLE
Raw Data
BITS
SIGNALS
SENSOR
PHYSI
CAL
DATA
OBJECTS
IM …
SIG …
HUM …
MAS …
OS ...
Information
Single Discipline Analysis
All Source Analysis
… INT
IntelReport
IC Customers
Intelligence
Types of Intelligence• Current• Estimative• Warning• Scientific & Technical• Research
14
Types of Intelligence
• Current Intelligence– Addresses day-to-day events– Seeks to:
• Apprise consumers of new developments and related background• Assess significance• Warn of near-term consequences• Signal potentially dangerous situations in the future
– Presented in regular publications (daily, weekly etc.), in ad hoc memorandums/messages and oral briefings to senior officials
• Estimative Intelligence– Deals with what might be or what might happen; Starts with
analysis of available facts then migrates into the unknown or even unknowable
– Goal: Help policy makers / decision makers to navigate the gaps between available facts by suggesting alternative, plausible patterns that fit available facts and to provide informed assessments of the range and likelihood of possible outcomes
– Flagship reporting vehicle is National Intelligence Estimate
15
Types of Intelligence (continued)
• Warning Intelligence– Sounds an alarm or gives time critical notice; connotes urgency
– Often involves situations that might involve US military forces or might cause their deployment, threats to US facilities and personnel overseas, sudden and deleterious effects on US foreign policy (coups, ethnic violence, etc.)
– Warning analysis involves exploring alternative futures and low probability / high impact scenarios
• Scientific and Technical Intelligence– Information on Technical Developments and characteristics,
performance, and capabilities of foreign technologies including weapon systems or subsystems.
– Generally includes detailed technical measurements
– Derived from analysis of all-source data
– Generally produced in response to specific national requirements derived from weapons acquisition programs, arms control negotiations, or military operations
16
Types of Intelligence (continued)
• Research Intelligence
– Monographs & other in-depth studies
– Two specialized subcategories:
• Basic Intelligence
– Geographic, demographic, social, military and political data on foreign countries
• Intelligence of Operational Support
– Reports of all types that are tailored, focused, and rapidly produced for planners and operators
– Example: DIA support to military forces & its operation of JWICS (Joint Worldwide Intelligence Communication System)
17
“Information Spectrum”
Information Exploitation (Info-X)Its Role Within the Intelligence Cycle
Collection Processing Analysis Reporting
Part of the “INTELLIGENCE CYCLE
Raw Data
BITS
SIGNALS
SENSOR
PHYSI
CAL
DATA
OBJECTS
IM …
SIG …
HUM …
MAS …
OS ...
“Machine Readable”
“Interpretable& Actionable”
“Human Understandable”
Info-X is FOCUSED ON CONTENT & IT’S MEANING
“Artificial Languages”Dominate
““Natural Languages”Dominate
Information
Single Discipline Analysis
All Source Analysis
… INT
IntelReport
IC Customers
Intelligence
Types of Intelligence• Current• Estimative• Warning• Scientific & Technical• Research
18
Natural vs. Artificial Languages
• Media formats, encodings, protocols, and other externals are heavily based upon “Artificial Languages”; that is:– Closed or formally controlled symbol sets and lexicons
– Well-defined syntax; Precise semantics
– Strict requirements for data integrity; Extensive use of error correction
– Readability and Understanding often depends upon access to keys, proper synchronization, knowledge of protocols and encodings being used, setup variables, etc.
• The understanding and interpretation of the content and meaning of data is heavily based on “Natural Languages”:– Open symbol sets, lexicons and syntactic/semantic rules
– Unconstrained in its construction; Includes frequent exceptions and irregular constructions
– Inherently, even intentionally, ambiguous
– Heavy reliance on unbounded context and external world knowledge
– No “silver bullet” solution to understanding. Increasing depth, breadth and accuracy of understanding is an Incremental Process.
19
IC Info-X Requirements:Pushing the Envelop Along Every Dimension
Each Dimension Consists of Multiple ComponentsAll are important to IC Analyst Community • Data Characteristics
– Quantity– Quality– Types / Quantity / Source of Errors– Source of Creation Control
• Data Coverage– Topics/Domains– Media– Types– Languages– Time Frame and Span of Interest– Emphasis on “Recall” as well as “Precision”
• Depth of Info-X– Number of Details Tracked Simultaneously– Human cognitive skills that must be emulated– Depth of Understanding / Interpretation– Amount of Interpretation / Judgment– Importance of Predictions / I&W– Tenets of Intelligence always apply (Accuracy; Timeliness; Usability;
Completeness; Relevance)
DataCharacteristics
DataCoverage
Depth ofInfo-X
CommercialSystems
State-of-the-ArtResearch Systems
IC Info-XRequirements
Space of Info-X
Systems
20
The Heart & Soul of Info-X:
• Current data volumes pose significant difficulties; And vast increases in data volumes of all types are projected for the future
• Number of analysts will almost certainly continue to decrease.
• Growing requirement to analyze language data in foreign languages and in native character scripts; but fewer analysts per language
• For all of our R&D efforts, Image understanding and interpretation continues to remain well within the purview of our IC analysts
• Growing requirements to fuse information across multiple media types, across multiple data objects, across time and space; Current system architectures make this difficult, if not impossible
• Broader, more complex reporting requirements; Reporting timelines are shorter; expectation of consumers of Intelligence products are higher
Currently Our Highest Quality Info-X “Systems” are our IC Analysts BUT . . .
21
In a Word: Intelligence Production is “Analyst Bound”
Providing Info-X Systems that Significantly Assist, Augment, and Complement the Work of our IC Analysts is Clearly the Answer . . .
But that’s easier said than done!
22
AQUAINT: Attacking the Data Chasm
Future
Fully Intersected;Automatically
Generated;Variable Structure/Format;
Full Context Responses
Full Context-Based
QuestionScenario
Level III
Full Context-Based
QuestionScenario
Fully Intersected;Automatically
Generated;Variable Structure/Format;
Full Context Responses
Level II
Variable NarrativeSummary;
Multi-Media Presentations;
Simple InterpretedResults
Cross MediaCross Document
Simple Judgement
Level I
Fixed Templatesor
Tabular Lists
Mulit-ValuedFactual QuestionsQuestions
Answers
Today
50/250 BytePassage from
Single TextDocument
SingleFactualIsolated
Questions
Data Chasm
Missing Data
MANY Heterogeneous Data Sources;
All Types, Sizes, Locations
IncreasingVolumes
(Petabyte & up)
Synthesis Across“Documents”/Media
ContradictoryDataReliability
of DataMultiple
Perspectives
23
AQUAINT: Focusing on QA Scenarios
• Requires handling a Full Range of Complexity & Continuity of Questions
• Need to understand & track the analysts’ line of reasoning and flow of argument
• QA System requires significantly greater insight into knowledge, desires, past experiences, likes and dislikes of “Questioner”
• Place much higher value on recognizing and capturing “background” information
• Questioner/System dialogue is now more than just a means for clarification
FactoidQuestion?
WhyQuestions
?
InterpretiveQuestions?
Judgement Questions
?
OtherQuestions?
Information Analysts
Predictive Questions
?
Overarching Context / Operational Requirement
24
• A Different Paradigm may be useful when handling QA Scenarios:
• Current Analytic Paradigm:
AQUAINT: Fostering a Different Paradigm
– Sequentially “Filter Down” to the
final result
Processing & Analysis
Data
Results
– Works when QA’s are independent, isolated activities
– Cast a “wider net” while searching
for “golden nuggets” (Answers)
AnswersSpace of Data Objects and Sources
How Wide to Cast the “Net”?
What Info to Retain? In what form?
For how long?
– Automatically Extract, Represent,
and Preserve “closely related”
background information within
context of the QA Scenario
Background
Discarded
25
Continuing . . .
• First Your Questions and Comments
• Then More on IC Analyst Functions and Methods and their relationship to the AQUAINT R&D Program
– Kelcy Allwein, DIA
– Frank Hughes, JMIC DIA
• Followed by Breakout Sessions with Brief Backs tomorrow morning