International Technology Alliance inNetwork & Information Sciences

Dave Braines, John Ibbotson, Graham White (IBM UK)

Dave Braines, John Ibbotson, Graham White (IBM UK)

SPIE Defense Security & SensingSPIE Defense Security & SensingNext Generation AnalystNext Generation Analyst

MIPS:MIPS:A service-based aid for A service-based aid for

intelligence analysisintelligence analysis

SPIE Defense Security & SensingSPIE Defense Security & SensingNext Generation AnalystNext Generation Analyst

MIPS:MIPS:A service-based aid for A service-based aid for

intelligence analysisintelligence analysis

Research was sponsored by US Army Research Laboratory and the UK Ministry of Defence and was accomplished under Agreement Number W911NF-06-3-0001. The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of the US Army Research Laboratory, the U.S. Government, the UK Ministry of Defense, or the UK Government. The USand UK Governments are authorized toreproduce and distribute reprints forGovernment purposes notwithstandingany copyright notation hereon.

Research was sponsored by US Army Research Laboratory and the UK Ministry of Defence and was accomplished under Agreement Number W911NF-06-3-0001. The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of the US Army Research Laboratory, the U.S. Government, the UK Ministry of Defense, or the UK Government. The USand UK Governments are authorized toreproduce and distribute reprints forGovernment purposes notwithstandingany copyright notation hereon.

AgendaAgendaAgendaAgenda

What is MIPS?

IT Support for the Intelligence Analysis Process

MIPS and the International Technology Alliance (ITA)

The MIPS Architecture and an example of its use

Meeting the MIPS Objectives

Research transition project led by DSTL (Defence Science Technology Laboratory) UKApplying emerging technologies arising from the US/UK

ITA research program• Information Fabric, Gaian Database, Controlled English (CE) & CE Store

Provide a generic service-based information processing architecture

Support the information analyst in their analytic goals• Notify new information relevant to their current task

– In any form, from any source

• Automatic processing of future information to extract meaningful task-relevant domain information

Management of Information Processing Services

What is MIPS?What is MIPS?What is MIPS?What is MIPS?

Collection IT Processing can extract, filter and

transform data into information from large volumes of data

Cognitive Processing is restricted to IT process configuration due to large volumes of data

Processing Preparation of information for

interpretation by analysts

Interpretation Human activity through hypothesis

testing and other cognitive techniques

IT processing assists analyst through information management, rules based inferencing and visualisation

MIPS is a proof of concept to investigate how analysts can configure and benefit from an improved Collect/Process/Interpret processing pipeline

On the ground Increased volume and variety of

information sources Cannot manually inspect, process &

interpret IT is not appropriate for all phases

in intelligence analysis

Processing to support Intelligence Analysis

IT supports extraction, storage,

indexing

IT supports retrieval, visualisation,

presentation & communication

Human configuration, algorithms, modeling

Human intel analysis,

hypotheses

IT Support for the Intelligence Analysis IT Support for the Intelligence Analysis ProcessProcessIT Support for the Intelligence Analysis IT Support for the Intelligence Analysis ProcessProcess

MIPS Builds on work from the US/UK MIPS Builds on work from the US/UK International Technology AllianceInternational Technology AllianceMIPS Builds on work from the US/UK MIPS Builds on work from the US/UK International Technology AllianceInternational Technology Alliance

ITA research is a US/UK collaboration between industry, academia and governmentFocused on: Network science, Decision-making & Coalition operationsFrom May 2006 – May 2016

Research is fundamental (6.1), low Technical Readiness Level (TRL) Intended outputs are papers, proof of conceptsHigher TRL transition contracts take the core ideas to progress further

Emerging technologies arising from ITA research used by MIPS: ITA Controlled English

• Consumable by humans and machines

ITA Information Fabric• A lightweight service bus middleware

Gaian Database• A Dynamic, Distributed, Federated Database

What is it? Human-friendly language but machine readable Rich semantics, broad application Reasoning engine with rationale Supports agility for dynamic, evolving situations Enables hybrid human/machine collaboration Empowers non-technical users

Examples: the person Dave works for the company IBM. if (the person P works for the company C) then (the company C employs the person P). the person pn0123 is a suspect in the crime c713 because the person pn0123 was sighted in… conceptualise an ~ intelligence report ~ I that ~ mentions ~ the suspect S.

Why does it matter? Real fusion of machine precision and human cognition Can harness “collective intelligence” Facilitate human-human communication & socialisation

Download the CE Store: http://ibm.co/RDIa53

Using human language to enable smarter human/machine processing of information

ITA Controlled EnglishITA Controlled EnglishITA Controlled EnglishITA Controlled English

A two-way message bus and set of middleware services Connects all network assets to each other and to users Provides universal access to intelligence data, processing services and applications

Implemented using a multi-hop publish/subscribe architecture Messages are efficiently propagated without duplication thereby minimising bandwidth utilisation

Policies constrain how assets can be used and configured and information shared across coalition boundaries

Same policy mechanism as is used in the Gaian Database

Information fabric can integrate assets and services In MIPS, the Information Fabric is used to integrate information processing services

A lightweight service-bus middleware designed for the edge of the network

ITA Information FabricITA Information FabricITA Information FabricITA Information Fabric

Download the fabric:http://ibm.co/13FyW9X

8

1250 node Gaian Database

DDFDs provide distributed accessto distributed heterogeneous data

sources, are highly scalable & impose low management overhead

Obtaining information from multiple heterogeneous data sources does

not scale & requires significant management overhead

N11

N1

N0

N3

N4

N5

N2

N6

N9N10

N8

N7

Query

Query

DDFD is based on “store locally & query from anywhere” principle

A dynamic distributed federated database (DDFD)

The Gaian DatabaseThe Gaian DatabaseThe Gaian DatabaseThe Gaian Database

Download Gaian:http://ibm.co/15TMSBr

Objective 1: Notify analysts when new, relevant information has arrived Defining Relevant Information Maintaining and sharing analytical goals Indexing information & notifying analysts of matches with their analytical goals Operation in a mixed environment

Objective 2: Automatic processing of new information Identifying processing services and data sets Standardised description of processing services to enable reasoning Visualising current capabilities for the user Efficient operation Abstract services Automatic information processing in a mixed environment Facilitating collaboration between analysts

MIPS Objectives & ChallengesMIPS Objectives & ChallengesMIPS Objectives & ChallengesMIPS Objectives & Challenges

Composition Analyst describes a set of services and

links them to form a network (to achieve a goal)

Network can be deployed as a set of linked and running services

Information Fabric Provides nodes for services to run on

and message bus for service interconnection

Services perform specialised information processing

Sources & Sinks

Controlled English All data produced by MIPS services is

converted to CE and optionally stored in the CE Store

Data is stored and queried using CE Some services also implemented in CE

Analyst Tools A set of tools for users to operate

within the MIPS environment

MIPS ArchitectureMIPS ArchitectureMIPS ArchitectureMIPS Architecture

Integrates multiple SharePoint repositories

Document Extractor service extracts metadata facts in CE about the documents

Report Analytics text mining service extracts entities and relationships from document text

Implemented as a set of CE-based services

Watch List Notifier looks at extracted CE about people and queries watch lists to see if a person is present on the list

Multiple types of watch list are demonstrated

Notification Sink manages notifications that can be accessed by other services and to alert users

Information generated is stored in CE store

Example: A typical information flowExample: A typical information flowExample: A typical information flowExample: A typical information flow

Challenge 1: Defining relevant information Analysts indicate relevant information by defining service compositions, or by

querying the CE Store. Compositions and queries may be saved, shared and reused

Challenge 2: Maintaining and sharing analytical goals Analysts define goals using the Fabric Service Composition Tool. Goals so defined

may be reused and shared amongst teams of analysts Goals may also be encapsulated in queries (or rules) in the CE Store which may

be saved, shared and reused

Challenge 3: Indexing information and notifying analysts of matches with their analytical goals

Indexing of information may be via a MIPS service or an external application e.g. traditional indexing within a relational database

Example provided of local and external watch list inspection, notifying people matched, with a link to the original document text about the person of interest

Challenge 4: Operation in a mixed environment The MIPS demonstrator is capable of interacting with and integrating different data

sources and using a wide range of deployed processing services

Meeting the MIPS ObjectivesMeeting the MIPS ObjectivesMeeting the MIPS ObjectivesMeeting the MIPS ObjectivesObjective 1: Notify Analysts when new, relevant information has arrived

Challenge 1: Identifying processing services and data sets The MIPS Fabric Service Composition Tool (FSCT) allows users to identify services available to them

and the data sets required and produced by the services

Challenge 2: Standardised description of processing services to enable reasoning MIPS standardises the descriptions of the services available to a user together with the information

exchange formats used between services

Challenge 3: Visualising current capabilities for the user The MIPS processing pipeline is constructed and visualised using a graphical editor; the FSCT Complex queries to the CE Store can be constructed and visualised using a CE Query Builder; a

graphical tool for creating complex queries across linked information

Challenge 4: Efficient operation MIPS uses the ITA Information Fabric as its underlying messaging middleware. The fabric’s

publish/subscribe model provides efficient routing of messages between services

Challenge 5: Abstract services Abstract services are a generalisation of a group or type of service Abstract service descriptions may be defined in MIPS before the service has been created

Challenge 6: Automatic information processing in a mixed environment MIPS is capable of handling information and services from both within and external to its

environment.

Challenge 7: Facilitating collaboration between analysts Some rudimentary support is provided in MIPS for collaboration amongst groups of analysts. Further

support is expected in future work

Meeting the MIPS ObjectivesMeeting the MIPS ObjectivesMeeting the MIPS ObjectivesMeeting the MIPS ObjectivesObjective 2: Automatic processing of new information

Email: john_ibbotson@uk.ibm.com or dave_braines@uk.ibm.com

Questions?Questions?

Research was sponsored by US Army Research Laboratory and the UK Ministry of Defence and was accomplished under Agreement Number W911NF-06-3-0001. The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of the US Army Research Laboratory, the U.S. Government, the UK Ministry of Defense, or the UK Government. The US and UK Governments are authorized to reproduce and distribute reprints for Government purposes notwithstanding any copyright notation hereon.

Main links: IBM developerWorks downloads:

• Information fabric:http://ibm.co/13FyW9X

• Gaian database:http://ibm.co/15TMSBr

• CE Store:http://ibm.co/RDIa53

International Technology Alliancehttp://www.usukita.org