IT Acquisition Advisory Council

“A public/private partnership operating as an honest broker for IT Innovations, Standards of Practice, Agile Methods, and just-in-time expertise outside the reach of the Defense Industrial Base”

Robert Babiskin, Chief Engineer, IT-ACC

www.IT-AAC.org

703 768 0400

Decision Analytics for Managing RiskIn Enterprise Software Inventories and Complex Designs

The AAM Decision Analytics “CUBE”Acquisition Assurance Method

From

Interoperability ClearingHouse

bEFFECTS Proprietary

The Business Case

A Tradeoff Analysis of3 Decision Analytic “CUBEs”

1.0

Acce

ss

1.1

IdAM

1.2

Acce

ss C

ontro

l

2.0

Defe

nd

2.1

Ente

rpris

e Pr

otec

tion

2.2

Ente

rpris

e De

fens

e

2.3

Ente

rpris

e As

sess

men

t

2.4

Thre

ats a

nd V

ulne

rabi

litie

s Miti

gatio

n

2.5

Ente

rpris

e M

onito

ring

3.0

Oper

ate

3.1

Even

t/In

ciden

t Man

agem

ent

3.2

Out o

f Ban

d M

anag

emen

t

3.3

Conf

igur

atio

n M

anag

emen

t

4.0

Gove

rn

4.1

Polic

ies,

Proc

edur

es &

Sta

ndar

ds

4.2

Risk

Man

agem

ent

Over

all

Level 1 2 2 1 2 2 2 2 2 1 2 2 2 1 2 2

Weight 258201 57

553138 164 113 75 63

9419 38 38

9431 63 1000

Weight % 26%20% 6%

55%14% 16% 11% 8% 6%

9%2% 4% 4%

9%3% 6% 100%

As Is 3.3 3.3 3.3 2.4 2.1 2.8 3.2 3.0 3.0 2.2 3.0 1.0 3.0 3.3 3.3 3.3 2.7

CO A 1 4.8 4.8 4.7 3.1 3.3 3.3 3.7 4.0 4.0 4.0 4.0 4.0 4.0 4.8 4.8 4.7 3.8

CO A 24.8 4.8 4.7 3.1 3.3 3.3 3.7 4.0 4.0 4.0 4.0 4.0 4.0 4.8 4.8 4.7 3.8

CO A 3 4.8 4.8 4.7 3.1 3.3 3.3 3.7 4.0 4.0 4.0 4.0 4.0 4.0 4.8 4.8 4.7 3.8

CO A 4 4.8 4.8 4.7 3.0 3.3 3.1 3.7 4.0 4.0 4.0 4.0 4.0 4.0 4.8 4.8 4.7 3.7

Blue Meets a l l Req'ts

Green Meets most Req't

Yellow Risks in Meeting Rqt's

Red High Risk

60% Improvement

AAM - Sample AoA Results in Consumer Report Style

FOFU

IT Tools in VA Technology Category Framework

IT Tools in VA Technology Category Framework

13

Network and

Telecommunications

Productivity Software

Accounting and Finance Desktop Publishing File Manager & Viewer Graphics Design Software Health Care Multimedia Software Standard Office Suite Misc Productivity Tools &

Utilities Web Browser

Collaboration Software

Content Management Electronic (Instant)

Messaging Unified Messaging E-Mail and

Calendaring Real Time & Team

Collaboration Shared Whiteboard Process & Schedule

Synchronization Tools Computer Based

Training – CBT

BI & Data Warehousing Platforms

Business Intelligence Platforms

Data Warehousing Systems

Web Reporting Tools Dashboard/Scorecard

Tools Data Mining Tools Geospatial Tools Data Analytics

(Statistical Analysis, Prediction, and Modeling)

Point of Care (PoC) Analytical Applications

Unstructured Data/Natural Language Processing

Clinical Environment and Tools

Data Management

Database Connectivity Desktop DBMS Embedded DBMS Object-Oriented

DBMS Relational DBMS Columnar DBMS Non-Relational DBMS DB-Related

Management Tools Data Quality

Management Master Data

Management

Network Infrastructure

Collaboration and

Electronic

Workplace

Information Management Technology

Data Integration

Database Replication and Clustering

Extract, Transform, Load (ETL)

Data at Rest Data in Motion

(Common Message Terminology and Semantics)

Transport

Switching and Routing Load Balancing and

Failover Network Name & Address

Local/Campus Area Network (LAN/CAN)

Wide Area Network (WAN) Telecommunications

Wireless and Mobile

Wireless Networks Cellular Networks Short Range Wireless Radio and Satellite

14

Storage

Storage Long Term Backup Operational Recovery

Operating Systems

OS - Desktop/Laptop OS – Mainframe OS – Mobile Device OS – Server OS Cluster and

Availability Application and OS

Deployment OS Tools

Platforms and Storage

Cloud Services/Server Virtualization

Peripherals

Input Devices Output Devices Multifunction Devices

Miscellaneous

Telepresence VTC Systems Other

End User Computer Devices

Development Tools

Application Testing Software

Software Engines

Integration Software

Analysis, Design & Modeling Application Development Tools Build and Deployment Tools Defect Tracking Development Framework User Interface Design Tools Integrated Development

Environment (IDE) Legacy Modernization Process Management Tools Requirements Management Software Change and

Configuration Management Web Authoring Tools

Debugging Test Tools Functional Test Tools Load & Performance Testing Tools System Testing Tools Unit Testing Tools

Enterprise Service Bus (ESB)

Service Registry SOA Governance Messaging Oriented

Middleware Device Integration

Business Process Management Engine

Business Rules Engine Geographic Information

System Engine Search Engine Context Management

Application Delivery Platform Software

Application Server SW Web Server SW

Application Technology

User Interface

Web UI Framework Portlets Rich Internet Application

(RIA) Framework Mobile Framework

Physical Servers

Blade Servers, Chassis, and Racks

Terminal Servers Extreme Low Energy

Servers

Personal Computers (PCs) Small Form Factor Mobile

Devices

Virtualization SW Cloud Technologies

IT Tools in VA Technology Category Framework

Systems Management Tools

Alert Management Application Management Asset Management Data Center Automated Tools Disaster Recovery IT Service Desk* Knowledge Management Mobile Device Management Monitoring Network Performance

Optimization Project Management Remote Desktop Management System Change and

Configuration Management

Facilities and Infrastructure Management

Power Monitoring

Identity & Access Management

Identity Management Authentication Authorization

Systems Management Security

Operations Management

Data Security

Data Loss Prevention

Platform Security

Secure OS Boot Application Security

Network Security

Antivirus and Anti-malware Content Filtering Encryption Security Administration Security Event & Information

Management Vulnerability Management Network Auditing Network Intrusion Detection

and Prevention

Emergency Management Human Resources

138 Categories of Software in 51 areas

The Problem – What capabilities do need in each categories, how do you eliminate redundancy and its cost. Do we have too much software for our needs – YES.

+ 2500 VA Applications

1. Burp Suite2. HP Fortify Static Code Analyzer3. HP WebInspect 9.04. Rational AppScan5. Clang Static Analyzer6. Hashtab7. AppDetectivePro

SwA: Static Code Analysis Portfolio at VA

1.0 Platform Support

3Static code analysis technologies often represent a significant investment by software organizations looking to automate parts of their application security assurance programs. These technologies demand time and effort by staff members to setup, operate, and maintain them. In addition, staff members are required to check and act upon the results generated by the technology. Understanding the ideal deployment environment will maximize the derived value, help the organization uncover more potential security flaws and could avoid unplanned hardware purchase cost. The following factors are essential to understanding the technology's capabilities and hence ensuring its proper utilization.

2.0 Technology Support

2Most organizations use more than one programming language internally within their applications portfolio. In addition, more software frameworks are becoming mature enough for development teams to leverage and use across the board as well as other 3rd party libraries which are used both on the server and client side. Once these technologies, frameworks and libraries are integrated into an application, they become part of it and the application inherits any vulnerability within these components. The tool must be capably of supporting VA's current programming languages and framework as well as the business process of the department.

3.0 Scan, Command and Control

2

The scan, command and control of static code analysis tools has a significant influence on the user’s ability to configure, customize and integrate the tool into the organization's Software Development Lifecycle (SDLC). In addition, it affects both the speed and effectiveness of processing findings and remediating them. The tool should be capable of providing these functionalities

4.0 Product Signature Update

1

Product signatures (AKA rules or checkers) are what the static code analysis tools useto identify security weaknesses. When making a choice of a static analysis tool,one should take into consideration the following: Frequency of Signature Update, User Signature Feedback.

5.0 Triage and Remediation Support

2

A crucial factor in a static code analysis tool or service is the support provided in the triage process and the accuracy, effectiveness of the remediation advice. This is vital to the speed in which findings are assessed and remediated by the development team. The tool should have the capability to support this these functionality.

6.0 Reporting Capabilities

4The tool or service reporting capability is one of its most visible functionalities to stakeholders.The tool or service should provide different ways to represent the results based on the target audience. For example, developers will need as much details as possible in order to be able to remediate the weakness properly in a timely fashion. However, upper management might need to focus on the report's high level summary, or the risk involved more so than the details of every weakness.

7.0 Enterprise Level Support

5

When making a choice on a static analysis tool or service in the Enterprise, one should take intoconsideration the ability to integrate the tool or service into various enterprise systems, such as bug tracking, reporting, risk management and data mining. The tool should be able to support these capabilities.

Stat

ic C

od

e A

nal

ysis

1.0 Platform Support1.1 Deployment Model 1.2 Tool Installation Support1.3 Scalability Support1.4 Setup and Runtime Dependencies2.0 Technology Support2.1 Standard Languages Support2.2 Programming Environment Support2.3 Technology Configuration Support3.0 Scan, Command and Control3.1 Command Line Support3.2 IDE Integration Support3.3 Build Systems Support3.4 Customization3.5 Scan Configuration Support3.6 Industry-standards Based Testing

Capabilities/ Analysis4.0 Product Signature Update4.1 Frequency of Signature Update4.2 User Signature Feedback

5.0 Triage and Remediation Support5.1 Findings Meta-Data5.2 Meta-Data Management5.3 Remediation Support6.0 Reporting Capabilities6.1 Support for Role-based Reports6.2 Report Customization6.3 Report Formats7.0 Enterprise Level Support7.1 Integration Into Bug Tracking Systems7.2 Integration into Enterprise Level Risk

Management Systems7.3 Ability to Aggregate Projects7.4 Licensing Scheme

Capability Decomposition

Stat

ic C

od

e A

nal

ysis

Open Architecture Architecture designed to make adding, upgrading, or swapping components easy; allows implementers to see inside all or parts of the architecture without proprietary constraints. May include open business processes involved with open architecture (transparency).

Complexity Ease of Use, Degree of Operator changes,

Product Maturity Product maturity is based on its release level

Customer Training Product training is currently available from vendor or third party

Product Support Product is well-supported by a company or a robust community of user/developers (Open Source)

Software Assurance Elimination of the twenty five common weaknesses of software (CWA)

Management FactorsSt

atic

Co

de

An

alys

is

AAM - Sample AoA based of Function Point AnalysisAnalysis of Alternative – Option C1

AAM - Sample AoA based of Function Point AnalysisAnalysis of Alternative - Option A

AAM Sample Analysis Of Alternative Findings AAM Capability Score & Alternatives Complexity Indicator