Joseph HowertonIS 430, Section 210 David BixbySummer I -
2009-10Case Study 1: FBI Virtual Case File (VCF)
projectIntroduction to the FBI s VCF project failureThe FBIs
Trilogy IT modernization program was intended to upgrade/modernize
the IT Enterprise Architecture/Infrastructure of the FBI; by
providing a high-speed network linking the offices of the FBI,
modern workstations and software within each office for every FBI
employee, and a user application known as the Virtual Case File to
enhance the ability of agents toorganize, access, and analyze
information. From A Review of the FBIs Tril ogy Inf ormati on
Technology Modernization Program
Initial VCF project scopeIt was the User Applications Component,
which would ultimately become the VCF that staked out the most
ambitious goals. First, it was to make the five most heavily used
investigative applicationsthe Automated Case Support system,
IntelPlus, the Criminal Law Enforcement Application, the Integrated
Intelligence Information Application, and the Telephone
Applicationaccessible via a point-and-click Web interface. Next, it
would rebuild the FBI's intranet. Finally, it was supposed to
identify a way to replace the FBI's 40-odd investigative software
applications, including ACS. From A Review of the FBIs Trilogy I
nformation Technology Modernization Program
Executive SummaryThe back-story for the notorious FBI Virtual
Case File (VCF) project failure really began in the early 1990s,
when FBI Special Agent, Larry Depew, identified the need for a
database program that would help him (and other agents) to organize
the reams of evidence collected and compiled during their
investigation(s); and from wiretaps, interviews, and financial
transactions, over the course of two and a half years investigating
mafia involvement in skimming off millions of dollars in federal
and New Jersey state gasoline and diesel taxes. The problem that
emerged was one of an inability of FBI Special Agents to use a
software component to connect the dots, both on Agent Depews case
specifically, and any new dots that might come along.
There was no doubt that a new system was needed, which would
become the Virtual Case File (VCF) system project; to replace the
existing system, the archaic Automated Case Support (ACS) system,
as it was extremely cumbersome, inefficient, and limited in its
capabilities. As it turned out, Depew ended up writing a database
program on his own, that he used to trace relationships between
telephone calls, meetings, surveillance, and interviews; but he
still could not import information from other investigations that
would assist him in his own investigation(s). That inability
created the need for a new system which would result in the formal
creation of the FBIs VCF project.
Officially, the FBIs VCF project began in 2000 when the FBI
began to deal with its outdated IT systems infrastructure. On July
17, 2000, the FBI hired Bob E. Dies to create a plan for this IT
transformation. Dies, who was a former executive with IBM,
basically replaced Louis J. Freeh, who did not have the IT skills
or competencies required to complete such a task. We will find that
the FBIs lack of IT skills, a technical ignorance of technology,
combined with theirincompetence in delivering IT systems
infrastructure, is a common thread that binds this epic and
unnecessary IT project failure. And Dies was just the first of five
officials who, struggled over four years to lead the FBIs sprawling
and antiquated information systems, and get the VCF project under
way.
In the fall of 2002, the only early warning sign came from
Matthew Patton, a security expert working for SAIC on the VCF
development team. Patton unequivocally bashed the non-existent
technical expertise on the side of the FBI, as well as SAIC
management practices, declaring both organizations incompetent. In
the end of that scenario, Patton would not be granted security
clearance with the FBI, could not take a position with SAIC due to
their inability to base him in Chicago, and maybe most importantly,
Pattons declarations and objections were summarily dismissed, thus
ignored.
In 2005, Glenn A. Fine, the U.S Department of Justices inspector
general, submitted an 81-page audit of the FBIs VCF project. In it
he described eight factors that contributed to the VCFs failure.
Among the factors included were poorly defined and slowly evolving
design requirements; overly ambitious schedules; and the lack of a
strategic plan to guide hardware purchases, network deployments,
and software development for the bureau. Fines conclusion was
submitted in the light of the September 11th, 2001, terrorist
attacks on the World Trade Center; he surmised that four years
after 9/11, the FBI still did not have the software it needed
toconnect the dots with the data in their case files.
The project was killed (before completion) in May 2005. The FBI
spent $170 million on the VCF project, and in the end they are
going with a customizable, off the shelf system. The solution is
not to keep throwing money at the project, but to recognize the
serious issues surrounding the projects failure. It is this
researchers opinion that, with so much turnover at the highest
ranks of the FBI, and the inability to document lessons learned as
part of an IT Governance strategy, the FBI didnt, and couldnt have
learned much.
In the end, this IT project, like so many IT projects, suffered
from a total misunderstanding by the FBI on how to define and
implement Enterprise-wide architecture, based upon user
expectations. Every step of this project would have benefitted from
defining and designing around Usability Engineering tasks,
activities, deliverables, and communication vehicles. It is
imperative that these practices occur from the beginning to the end
of the project design and development lifecycle.
Research ApproachUsing IEEE Spectrums article on the VCF project
failure as an anchor, I created a sort of outline of the pertinent
details that works for me. Then I went online to find further
analysis on the topic. I found that (because) the IEEE Spectrum
article was (it seemed to me) the first to break the story in great
detail to the general public, I felt it was used as the base anchor
reference for (basically) all of the analysis briefs out in the
ether; so I figured that my analysis will be added to the many.
As I read through the IEEE analysis, Goldstein pointed to
another report of interest - A Review of the FBIs Tril ogy Inf
ormati on Technology Modernization Program report on the FBI
trilogy Project, which changed my viewpoint. I found that this
report is actually the most comprehensive and complete report that
I found on the failure of the VCF project. It details the issues
across the board, and puts forth detailed recommendations for the
project based on their exhaustiveanalysis.
From Anatomy of an IT disaster: How the FBI blew itFrom here, I
analyzed each of the other reference documents in deference to the
expertise and experience of the authors bringing it to the fore,
having the core ideas resonate in a new way, to capture their
thoughts on an idea, when golden, and (of course) to find any
differences or discrepancies from opinion to opinion. The general
consensus (I feel) in the research is one of acknowledging precepts
put forth in the IEEE Spectrum report that this enterprise project
fell into the most basic traps of software development, from poor
planning to bad communications throughout the lifecycle of the
project.
Key IssuesThe first key IT issue was the 800-page requirements
document put forth by the FBI. This document was totally unusable
from the start, and many analysts assert that with such a poorly
defined approach to a project, left most to wonder, how could it
possibly achieve success?
Further, the FBIs understanding of the requirements was not
strong at all. There was a seeming disconnect between the FBI and
SAIC from the beginning. Add to the mirth, a constant turnover of
FBI Directors, the FBIs lack of internal expertise, and a lack of
really any credible IT management and technical expertise internal
to the FBI. Of course the results would be (and were) catastrophic
for the VCF project.
The project demonstrated a systematic failure of software
engineering practices which include, from Wikipedia: Virtual Case
File
Lack of a strong blueprint from the outset led to poor
architectural decisions. Repeated changes in specification.
Repeated turnover of management, which contributed to the
specification problem. Micromanagement of software developers.The
inclusion of many FBI Personnel who had little or no formal
training in computer science as managers and even engineers on the
project.Scope creep as the requirements were continually added to
the system even as it was falling behind schedule.Code bloat due to
changing specifications and scope creep. At one point it was
estimated the software had over 700,000 lines of code.Violating
Brooks' law by adding people and resources to the project as it was
falling behind, slowing it further. Planned use of a flash cutover
deployment, which made it difficult to adopt the systemuntil it was
perfected.
Because of the bureaucracy (at least) inside the FBI (I think),
as well as inconsistent responses and communications from SAIC,
there was no cogent, cohesive blueprint laid out for achieving
success. There was no enterprise architecture (blueprint) to
describe how an organization operates currently, how it wants to
operate in the future, and includes a road map a transition plan
for getting there.leaving no detailed description of the FBIs
processes and IT infrastructure as a guideline.The National
Research Council pointed out that without this blueprint/road map
the bureau could not make coherent or consistent operational or
technical decisions about linking databases, creating policies and
methods for sharing data, and making trade-offs between
informationaccess and security.
Key IT ImpactsAccording to the National Research Councils (NRCs)
Report Review Committee in their report A Review of the FBIs Tril
ogy Inf ormati on Technology Modernization Program, the specific
factors that would end up in an implosion of the VCF project
include,
Enterprise architecture - the committee concluded that the FBIs
efforts and results in the area of enterprise architecture are late
and limited, and fall far short of what is required.
System design - The design process was well under way prior to
the expansion of the intelligence mission, and the requirements for
the processes supporting the intelligence mission were not included
in the VCF design. For this reason, and because of the significant
differences in IT requirements between systems supporting
investigation and those supporting intelligence, the committee
strongly recommends that the FBI refrain from using the VCF as the
foundation on which to build its analytical and data management
capabilities for the intelligence processes supporting the
counterterrorism mission.
Program and contract management - In practice, it is essentially
impossible for even the most operationally experienced IT
applications developers to be able to anticipate in detail and in
advance all of the requirements and specifications. Therefore,
internal development plans, and the development contracts with
supporting organizations, should call for an approach that isbased
on a process of extensive prototyping and usability testing with
real users. Doing so allows iterative development with strong user
feedback and involvement, thus increasing the chances that what is
ultimately delivered to the end users meets their needs.
Skills, resources, and external factors - the FBI lacks a human
resource and skill base adequate to deal with the bureaus IT
modernization program. Specifically, the FBI is extremely short on
experienced program managers and contract managers and senior IT
management team members with good communications skills.
Final AnalysisIts been a challenge just to figure out how to
approach the analysis of the FBIs failed VCF project. Its difficult
to truly understand, with so many resources, and supposed expertise
that one would expect at their disposal, how this project could
fail. I had to simplify how I was looking at it. Upon reflection,
my opinion is simply that the primary stakeholders on both sides of
the project turned out to be a bunch of people that didnt have the
guts to speak up and speak out about the gross incompetence at any
and every juncture of this project.
So, I approached my analysis of this colossal IT project failure
wearing two hats; usability engineer and project manager. My
opinion paper begins and ends with Usability Engineering (UE), as
it is my opinion that the VCF project was a classic case for the
need of insertion of UE and User Experience Design (UXD) addressed
across the project lifecycle to inform and speak forthe users. I
just dont see how they could execute this project so poorly. I know
from my own experiences that 2000-2005 were very tumultuous times
for IT Governance boards, specifically with the implosion (and
consequences) of the internet bubble. I can imagine that SAIC was
going through a lot of not-so positive or pleasant experiences
during those times as well.
In the final analysis, I assert that there is equal blame here
for both parties the FBI, and SAIC to accept responsibility for the
VCF project failure. I should say that I dont really like the term
blame, except to say that when something goes wrong, per David
Bixby, the IT Project Manager should step out in front to shield
the project team from said blame; Im pretty confident that this
didnt happen on either the side of the FBI or SAIC. Any IT project
is going to have changes,and requires constant communications,
which I dont feel was the case in this project. There were issues
from the very beginning that SAIC could have stopped the project in
its tracks to perform better, more comprehensive, up-front
strategy.
The VCF has become a primary cautionary tale for IT
professionals of all types. From the perspective of a Usability
Engineer, the lack of an appropriate created requirements document
left this project for dead from the beginning. In the end, I feel
we are left wondering about too many aspects of this IT project
failure. As a result, along the way, along with major scope creep,
the FBI went through five different CIOs, 10 Project Managers, and
36 contract changes. Aswell as reports from the Government
Accountability Office, the DOJs inspector general, and the National
Research Council, all submitted reports acknowledging the ongoing
problems with the project, as a result of not having an appropriate
blueprint/roadmap to success.
I dont feel that representatives from either side, the FBI or
SAIC, stepped up and took any responsibility for the comedy of
errors that the VCF became; and with the practice of Usability
Engineering taking root in IT development methodologies, a little
up-front inspection, and with checkpoint across the development
lifecycle, this project could have been saved.
Instead, what the FBI and SAIC engaged the taxpayer in, was a
project that was a roller coaster of inefficiencies, riddled with
change requests, constantly changing priorities in the guise of an
actual plan, schedule slippages, integration delays, the FBIs
sloppy inventories of existing networks, underestimates throughout
the development lifecycle, and significant management turbulence.
Throwing good money after bad to committee upon committee, and
never realizing the source of the problem, which was they employed
no constructive usability methodologies to appropriately engage the
design and development of such an advanced Enterprise
Architecture.
In the end, after spending at least $170 million for the VCF
piece of the FBIs Trilogy project,the FBI is going to go with an
off the shelf solution to address the requirements left unresolved
in the wake of the VCF project failure.
From a project management perspective, the lessons of the VCF
cautionary tool are many. In the light of day, the role of project
manager from both sides of the project, was not integrated intothe
project with any kind of authority.Writing this brief has been very
frustrating because the details of this project are enough to
frustrate any taxpayer alone, but also as an IT professional, with
a focus in User Experience Design and Project Management. In my
opinion, it speaks to the ongoing corruption of government; maybe
especially if its a corruption of confidence; because that
confidence has slipped dramatically.
At every step of the way, when the FBI asked, Congress approved
more and more money for the Trilogy project. Where did all the
money go for the VCF? The whole thing offends me quite frankly. And
Im not asserting that I could have managed this, such a complex
project. I am simply saying that I received my BS in HCI in
November 1999; so if this project started in 2000, why werent there
usability professionals assigned as resources to this project? Its
not clear to us if there were or not, but I assert here that they
could have saved this project at the beginning, by rejecting the
requirements documentation as put forth by the FBI.
The reason Im being so vehement here is simply because if things
can go this wrong when the FBI is working on an IT project, what
other projects are out there that are top secret, and we know
nothing of the waste and corruption, save what we hear from other
government agencies via committees and task forces after internal
audits. Its a bit nerve wracking when so much is just
unknown.References
Who Killed the Virtual Case
File?http://spectrum.ieee.org/computing/software/who-killed-the-virtual-case-file
A Review of the FBI s Tri logy Informat ion Technol ogy
Modernization Program
http://www.enterprise-architecture.info/Images/Documents/CSTB-
FBI%20Project%20Review%2006-2004.pdf
Congressional Testimony of US DOJ inspector general Glenn A.
Fine, February 2005
http://www.usdoj.gov/oig/testimony/0502/final.pdf
Matthew Patton's October 24, 2002 posting on InfoSec News about
VCFhttp://www.infosecnews.org/hypermail/0210/6688.html
The Fail ure Of The F BIs Virt ual Case Fi le Project
http://strategicppm.wordpress.com/2010/04/05/the-fbis-virtual-case-file-project-and-project-
failure/
The FBI's Upgrade That
Wasn'thttp://www.washingtonpost.com/wp-dyn/content/article/2006/08/17/AR2006081701485.html
Anatomy of an IT disaster: How the FBI blew
ithttp://www.infoworld.com/d/developer-world/anatomy-it-disaster-how-fbi-blew-it-243
FBI's Virtual Case File
Flopshttp://www.internetnews.com/ent-news/article.php/3459921/FBIs-Virtual-Case-File-Flops.htm
Wikipedia: Virtual Case File
http://en.wikipedia.org/wiki/Virtual_Case_File
GAO: Questionable fees paid on FBI Trilogy project
http://gcn.com/articles/2006/05/08/gao-questionable-fees-paid-on-fbi-trilogy-project.aspx
Report: FBI wasted millions on 'Virtual Case File'
http://www.cnn.com/2005/US/02/03/fbi.computers/
