Report on FBI IT Modernizationinfolab.stanford.edu/pub/gio/2004/Report-RJBandGioCom… · Web viewDRAFT!! version 01-22-04

DRAFT!!!! FOR COMMITTEE USE ONLY 11/25/0312/04/03 DO NOT DISTRIBUTE FURTHER!

Report on FBI IT ModernizationDRAFT!!

version 01-22-04<<further MS Word tracked Edits by Gio, with some <<meta

comments >> >>(needs revision based on new sections)

1. Charge....................................................................................................................................1

2. Background............................................................................................................................22.1 Missions...........................................................................................................................2

2.1.1 Criminal Investigations and Law Enforcement...........................................................22.1.2 Counterterrorism, Counterintelligence and Intelligence (CCI)...................................3

2.2 Key processes..................................................................................................................32.2.1 Investigation................................................................................................................32.2.2 Intelligence................................................................................................................342.2.3 Information Management..........................................................................................35

3. Issues for the FBI.................................................................................................................353.1 Enterprise Architecture Issues.......................................................................................36

3.1.1 What is an enterprise architecture?...........................................................................363.1.2 Problems faced by the FBI........................................................................................383.1.3 Items for Immediate Action..................................Error! Bookmark not defined.10

3.2 System Design Issues..................................................................................................3123.2.1 The Virtual Case File (VCF) Application...............................................................3123.2.2 Data management and the Integrated Data Warehouse (IDW)...............................3143.2.3 SCOPE....................................................................................................................3173.2.4 Mobile computing...................................................................................................3183.2.5 Security...................................................................................................................3183.2.6 Privacy.....................................................................................................................321

3.3 Management issues......................................................................................................3223.3.1 Overall development methodology.........................................................................3223.3.2 Contracting and Contract Management..................................................................3243.3.3 Program/Contractor Management...........................................................................324

3.4 Human Resources and External Constraints...............................................................3263.4.1 Human resources.....................................................................................................3263.4.2 External constraints.................................................................................................327

4. Recommendations..............................................................................................................328

1. CHARGE

As a result of an earlier real-time review in September 2002, the committee was asked to provide a more thorough review and set of recommendations on the FBI’s information technology modernization efforts, focusing primarily on the Trilogy upgrade but addressing other issues as necessary. Except as explicitly noted otherwise, the factual base regarding the FBI is derived from briefings to the committee on [October 27-28, 2003 and December 15-16, 2003dates of meeting 1 and meeting 2]. The committee’s conclusions and recommendations

1

123

123456789

10111213141516171819202122232425262728293031323334353637

38

39404142434445

4

DRAFT!!!! FOR COMMITTEE USE ONLY 11/25/0312/04/03 DO NOT DISTRIBUTE FURTHER!are based on its collective experience with large-scale IT system deployments.

2. BACKGROUND

Today, the FBI is undergoing a massive shift of mission priorities. Whereas the FBI was previously oriented almost exclusively towards law enforcement and the investigation of criminal activities, as a result of the changes in the threat environment its mission has expanded to include—and as its top priority—the detection of potential domestic terrorism and the prevention of the success of domestic terrorist activities. The leadership of the FBI recognizes the need for upgraded information technology to enhance its ability to collect, store, search, retrieve, analyze and share information in pursuit of these priorities. At the same time, the FBI has not traditionally been among Federal agencies that have been regarded as technology innovators, and for many years, the FBI has been criticized for inadequate attention and competence with respect to its use of information technology. Outdated, inadequate, hard-to-use, and stovepiped are terms that have often been used to characterize the Bureau's technology.

In any organization engaged in any business, the introduction of modern information technology and the concomitant reengineering of an organization’s key processes to exploit the technology constitute a major challenge. In the FBI’s case, this transformation is being managed under intense operational pressures: the FBI’s traditional work must continue even as new technology is introduced and a more IT-exploiting culture is evolved. Compounding the challenge is the added strain of a new focus—preventive counter-terrorism, where success demands a different mindset, operational skills, and novelthe exploitation of a radically expanded set of information sources.

To be effective, IT investments must be aimed at enhancing the business effectiveness of an organization. The return on an IT investment must be measured in business terms – more and better results, increased responsiveness and agility, and improved efficiency of operations. Maximizing the return on a major IT investment thus requires an intimate and dynamic interplay between technology supporting strategy, and the trajectory of technology enabling and stimulating change in strategy over time. Thus, this report begins its discussion from the business/strategy side.

2.1 Missions

The nature of an agency’s missions and the agency’s strategy and business objectives are the primary drivers of the kinds of information and communication it needs and the processes it must exploit, and thus of the architecture, design and functioning of its information technology systems. In the case of the FBI, the two main missions with associated business objectives are: first, the investigation of criminal activity and the prosecution of criminals; and second, the prevention of terrorism within the US and against US interests around the world. (The designations “first” and “second” do not refer to today’s priorities, but rather to those of the past.) Supporting these missions and the achievement of the related business objectives of these business segments are a set of key processes which are used in different degrees in achieving the objectives.

2.1.1 Criminal Investigation and Law Enforcement

2

567

46

47

48495051525354555657585960616263646566676869707172737475

76

77787980818283848586878889

90

8


The traditional role of the FBI is that of an investigative agency for the Department of Justice of the United States. This role in investigating and preparing much of the basis for the prosecution of crimes is one of the key missions of the FBI. The information developed by the FBI investigators is provided to prosecutors who in turn determine if an individual will be prosecuted. The investigations occur when either a violation of the federal criminal code occurs or when the U.S. Attorney’s office supports an investigation. ( Is this a complete set ? Does FBI ever open an investigation on their own suspicions ? JCM I believe we were told at an earlier meeting that the FBI could open a “case” at their own discretion, but that may not literally be the same as an “investigation.”)

This role of the FBI in this area is responsive in that is initiated in response to an action or event or by a request from a legal authority. Once the FBI investigative activity has been initiated, the FBI will use all the resources legally at its disposal to gather relevant information related to the situation, as discussed below.

2.1.2 Counterterrorism, Counterintelligence, and Intelligence (CCI)

Since September 11, 2001, the FBI has been obliged to placed much greater emphasis on identifying and protecting the U.S from acts of terrorism. The FBI’s objective is to prevent acts of terrorism in the U.S. and against U.S. persons and interests throughout the world. Accomplishing this daunting objective requires, among many other things, the accessing, analysis, and exchange of massive amounts of information, and close, daily coordination and cooperation among law enforcement, intelligence and many other involved organizations.

This role of the FBI in this area is proactive in that is necessarily ongoing. That is, the execution of the mission is not executed in response to any particular eternal event. (If a serious event has already occurred it is, then it is not unreasonable to suggest that the execution of the CCI mission has not been entirely successful.)

(I think we may be making too much of “responsive” vs “proactive.” The FBI would argue that it is proactive even in investigatory cases, and one could conceivably argue that CCI may be triggered by a leading event, warranting investigation (e.g., a bombing). For our committee’s work, is the more important distinction between investigatory data and CCI data, with the understanding that there could be some overlap of purpose?)

2.2 Key processes

It is important to distinguish between missions and the key processes that support the accomplishment of those missions. In some cases, a key process supports only one mission; in others, a key process may support multiple missions. In both, the use of modern data-processing technology is essential. Processes involve things like information acquisition and the workflow of its mangement—how information is acquired, who must act on it, how it must be processed and analyzed, what types of inferences must be drawn, and how information of all types flows within the organization.

This section describes some of the key processes within the FBI.

3

91011919293949596979899

100101102103104105106

107108109110111112113114115116117118119120121122123124125

126

127128129130131132133134135136137

12

DRAFT!!!! FOR COMMITTEE USE ONLY 11/25/0312/04/03 DO NOT DISTRIBUTE FURTHER!2.2.1 Investigation

The investigatory process is the primary process supporting the law enforcement mission. Investigation develops information from a variety of sources, including but not limited to information gathered directly by special agents or other law enforcement agencies, information obtained through informants, information obtained from other agencies such as the INS or local law enforcement agencies, laboratory developed information, and open source information (e.g., information on the Internet or in newspapers). The collection and analysis of information is usually under the control of the individual agent or special agent in charge that is directly responsible for the investigation. The information or a subset of the information such as pointers is placed in centralized FBI files for appropriate dissemination and use as part of the case file.

The agent or group of agents assigned to a case is and has been the focal point of FBI criminal and law enforcement activities. The agent is responsible for carrying out the investigative task and managing much of the information involved. In the case of a criminal investigation, the information developed is then conveyed to the prosecutor for decision and action. In the case of a background investigation, the information is delivered to the requesting agency. The agent is the focal point of the activity with support from administrative staff, analysts, and other FBI employees. The investigative information is organized around cases and agents as the fundamental units. Moreover, there are a variety of legal and procedural requirements in place to ensure that developed information can be used in court to support prosecutorial activities. If information is insufficiently supported or vetted it will create probems in the prosecution of the case.

2.2.2 Intelligence

Intelligence processes include information collection and analytical functions. Information gathered under intelligence auspices isare subject to fewer restrictions than is information gathered in the investigatory process, since itand thus is not usually not used to support prosecutorial activities. Analytical intelligence processes are used in both law enforcement and CCI missions.

Information gathering in the intelligence context requires that voluminous information resources from internal and external sources be logically brought together and analyzed with the goal of identifying potential threats of, or precursors to, terrorist crimes. The range of sources of information which must be selectively probed and analyzed is enormous, and much of the (mostly computerized) information will not be obtained from government-owned sources. Much of the information will in fact be open-source information, such as newspapers in foreign languages, or found on the Internet.

Analytical functions in the intelligence process must analyze hierarchies of information analysis of often uncertain relevance and quality, but the desired result is the distillation of conclusions that become increasingly certain as they are further aggregated and refined. Such analyses may, at different stages, result in warnings and may initiate deeper and more focused investigations.

The intelligence process generally requires that the FBI receive information from and disseminate information to local law enforcement agencies, the U.S. intelligence community, and often with International agencies. The ability to share information at multiple levels of security

4

131415

138

139140141142143144145146147148149150151152153154155156157158159160161162

163164165166167168169170171172173174175176177178179180181182183184185186

16

DRAFT!!!! FOR COMMITTEE USE ONLY 11/25/0312/04/03 DO NOT DISTRIBUTE FURTHER!classification with a wide variety of collaboratorsoperators is essential to the performance of the counter-terrorism mission. Sharing of informationSuch an ability plays a far more limited role is much more circumscribed in criminal investigation. Strong capabilities to access, manage, analyze and communicate information and intelligence across institutional boundaries are key to the analytic function at the core of the intelligence process.

Sharing information requires the support not only of technology, but also of cooperative relationships with the intelligence and law enforcement communities at levels, both from local andto international. Furthermore, its success demands a framework of policy and process to ensure appropriate balance among timely access, security and privacy rights. Trained analysts probe, tease apart and develop new information that can identify, confirm or deny an activity. When supporting the law enforcement mission, analysts must understand the investigative role of the Bureau and the agent’s operational processes. Similarly, agents must understand that the role of analysts is different than that of CID agents. Most analysts have specialized expertise and must be able to easily cooperate with colleagues on diverse topics. Analysts must be comfortable with the technology that is increasingly a source of information and a tool to support the sharing and analysis processes. And the individual analyst at the FBI must be highly skilled in the methods and processes that are used both for the criminal and CCI missions.

2.2.3 Information Management

The investigatory and intelligence processes used by the FBI are information-intensive, and the Bureau has recognized that state-of-the-artimproved information management that exploits available technology can significantly enhance the effectiveness of these processes. Furthermore, both counterterrorism and criminal activities are evolving in a way that spans traditional organizational boundaries in the FBI. Special-Agents-in-Charge (SACs) are organized around geography. Counterterrorism and crime no longer respect those boundaries, and thus a Bureau-wide technology deployment must be seen as a set of systems and data that span SACs.

;;;Example of something done well pre technology – don’t do IT that messes this up: [WHERE SHOULD THIS GO] <<move to the description of investigative processes.>>

The FBI culture supports a robust practice of remote tasking whereby an agent in one office executes an investigative task for an agent elsewhere.

Keep doing this, don’t screw it up. (But we should also note that that the right IT improvements can make this teamwork even more effective.)

The FBI relies extensively on a well-developed remote tasking practice whereby an agent in one location who needs information from another area can easily transmit a request, called a “lead,” to the appropriate field office where it will be followed up by a local agent. This practice is remarkable in that it allows the organization to function on a continental scale without a tremendous cost in time and money for travel, central management of every case, and yet without sacrificing the personal contact that is essential to the interviewing success of FBI agents. For this to work as effectively as it manifestly does is a testimony to the quality of training and the uniformity of culture within the FBI. A thoughtful application of technology can serve to support and enrich this practice, but the

5

171819

187188189190191192193194195196197198199200201202203204205206

207208209210211212213214215216217218219220221222223224225226227228229230231232233234235

20


FBI is urged to take care not to endanger the core culture that makes this practice work as effectively as it does.

Recognizing the need for improved support if invstigative processesse points, the FBI has embarked on a major IT modernization program, whose main focus today is the Trilogy program. Trilogy has two major objectives with corresponding project phases. The first is the installation of an end-user oriented infrastructure, consisting of a secure wide-area network and related local area networks, together with modern workstations, printers, scanners and base commercial software applications such as browsers. This infrastructure is intended to provide an enhanced platform for modern applications. The second element of Trilogy is an application which is oriented towards the traditional law enforcement mission of the FBI, a "Virtual Case File" (VCF) that provides user-friendly, Web browser capability to allow agents to electronically manage case-related information critical for law enforcement. Since the needs for, and capabilities of, IT continually increase, it must also be recognized that the delivery of Trilogy is but an initial phase of an operation that will continue as long as the Bureau is active.

(I thought there were more applications than just VCF in the second element of Trilogy? Weren’t some ‘business support’ systems also included? The key question from some readers’ point-of-view will be: Exactly what is Congress funding with the ever-increasing “Trilogy” budget? If the only application is VCF, so be it. RJB)

At this writing (February 2004), neither of the two initialse phases is fully complete, though significant progress has been made on both. In addition to these two major phases, a general requirement to support CCI activities has also been placed on Trilogy, although specifications for that novel task have not been developed. At the same time the FBI has embarked on the development and implementation of appropriate systems to support the intelligence functions, which are key to the counter-terrorism mission. Central to this thrust is the creation of a core <<?>>data repository, referred to as the IDW, the Information Data Warehouse (IDW).

3. ISSUES FOR THE FBI

The Committee’s review of the approach and methodology currently being used by the FBI to drive the introduction of these new systems has raised a number of major issues. These issues can be grouped into four major clusters. Issues in any one of these clusters would be serious in and of themselves, but when these issues arise together, the detrimental impact on the FBI’s IT modernization efforts is multiplied. These four clusters are:

Issues related to enterprise architecture: It is most important that the FBI’s business and operational needs drive its IT program. One common way of describing and ensuring this linkage is through the formulation of an enterprise architecture, which then drives the needed IT improvements. The first cluster is the most global, and concerns the match between the FBI’s current information technology program and its operational needs. The enterprise architecture becomesis the template on which the IT investment is rationalized and the enhancements to the FBI’s mission achievement, the return on the investment, are defined and quantified using appropriate metrics. When new needs, as engendered by the CCI requirements, arise, existing capabilities as well as lacunae can be clearly identified.

6

212223

236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265

266

267268269270271272273274275276277278279280281282283284

24


Issues related to system design: System design mustshould be driven by the architecture. Based on the committee members’ experiences with similar IT modernization efforts, we have noted some The issues in the second cluster derive in large part from the first, and relate to choices made and issues that are inadequately addressed in the areas of the system architecture, workflow, and the resulting designs of data repositories, system structure, security and access, and applications.

Issues related to implementation: Adequate and deeply involvedLarge scale pProgram mManagement isand other processes are critical to the success of IT modernization endeavors as massive as Trilogy. Issues here relateThe third cluster arise from the committee’s examination of the Trilogy program and to the approaches and processes used by the FBI across the implementation life cycle, including requirements determination, development methodology, contracting and project management, system roll-out, training, evaluation, and metrics of effectiveness. Dealing with these issues is key not only to the success of Trilogy, its ongoing development, and but also to the success of later IT modernization programs.

Issues related to skills and resources: The fourth and final set of issues cuts across the first three clusters, and generally relate to human resources and skills and to some of the other external constraints faced by the FBI in its IT modernization efforts.

In the next four sections we deal with these issues cluster by cluster.

3.1 Enterprise Architecture IssuesThe committee assigns a central role to an enterprise architecture, which will bridge the perceived business needs with their technical implementation.

3.1.1 What is an enterprise architecture?

An enterprise architecture is an overarching framework which describes the way in which an organization such as the FBI conducts its mission(s), how it organizes and uses technology to accomplish its goals and execute key business processes, and how the IT system is structured and designed in more detail to achieve these objectives. It bridges operational business needs with required IT support.<<good statement, moved up>>

It Enterprise Architectures are is generally represented as an operational schematic accompanied by appropriate narrative that shows the operating elements or nodes comprising the enterprise, the data/information flow requirements between the nodes, and other data reflective of the operational structure. It is the central document from which operational/functional, system, and technical views of the enterprise can be derived. It specifies the overall design and the set of building codes to which deployed systems must conform in order to make the IT investment effective, and describes how business needs can be supported and enhanced by appropriate application of technology. Only with a map between mission and technology is it possible to understand why certain IT acquisitions are or are not desirable andor how to create metrics that will help decision makers assess the effectiveness of any deployment.

An enterprise architecture provides a set of understandable, detailed, integrated diagrams and documents that describe an enterprise’s business framework, much as an architect’s drawings

7

252627

285286287288289290291292293294295296297298299300301302303304305306307

308309310311312

313314315316317318319320321322323324325326327328329330331332

28

DRAFT!!!! FOR COMMITTEE USE ONLY 11/25/0312/04/03 DO NOT DISTRIBUTE FURTHER!describe the basic structure of a building or a design engineer’s set of highway network diagrams describe a highway. Having an architectural specification that can be understood by management, the users, and by the implementors engenders cooperation and trust A well-documented and communicated enterprise architecture is a prerequisite for driving cultural and operational change and innovation at a pace which effectively capitalizes on the pace of technology improvement. Good models depict the as-is (today’s) environment as well as the to-be (future desired) environment, showing the transitions.

There are many ways to conceptualize an enterprise architecture. For example, the Federal Enterprise Architecture (FEA) is used by many government agencies, and contains 5 basic components (Figure 1). For purposes of this report, the essential lesson of the FEA approach is that missions drive the overall enterprise architecture, as specified in the Performance Reference and Business Reference models. Put differently, business processes are facilitated and enhanced by technology, not the other way around.

However, Tthe committee believes that the architectural triad of the Department of Defense provides a high degree of conceptual clarity in understanding enterprise architecture, and the next section will describe the DOD approach.

3.1.2 Creating an enterprise architecture

Beginning with the notion that business processes are facilitated and enhanced by technology, a good starting point is the triad of architectures used by the Department of Defense for achieving the required linkage between the strategy and business processes of an organization, and the design of the related supporting IT systems. This approach aims to create three consistent views of the enterprise and its IT systems.

The three views are:

the operational architecture, which describes the key business objectives, the processes used to achieve them, and a high level description of the information used and created in these processes;

the systems architecture, which describes at a high level the data repositories, systems, applications and communications infrastructure which supports the operational architecture;

the technical architecture, which captures the key technical standards, protocols and specifications required to implement the systems architecture.

As for content, Figure 2 has one description of primary and supporting activities in the FBI as they might contribute to an Enterprise Architecture, based on Michael Porter’s notion of Business Value Chain. This diagram would normally be a combined effort of management and technology folks and is intended to focus attention of the key structure of the business. With such a diagram in hand, the scope of various IT applications (i.e., the processes and activities to which key IT systems and data classes contribute) can be clearly seen (Figure 3).

These figures represent the committee’s perspective on the problem, and upon consideration, the FBI may not agree with all of it. Far more than accepting or rejecting the particulars of Figures 2 and 3, it is important for the FBI to develop their own versions of Figures 23 and 34 to help explain their strategies.

8

293031

333334335336337338339340341342343344345346347348349350351352

353354355356357358359360361362363364365366367368369370371372373374375376377378379380381382

32


The committee emphasizes that, though this task is critical, it need not be an enormously time-consuming one. With adequate time from senior management on the operational side working with key IT people, major progress could be made in a week of intense work, and a good high-level architecture should be possible in a 3 to 6 month time frame. Further, committee experience indicates that progress is best made with architecture teams composed of a small number of full-time professionals to the task. These individuals should understand the business and have some appreciation for technology, rather than being technologists with only a loose connection to the business side (a state that has characterized FBI architecture efforts to date).

The committee also notes that the FBI’s core missions rely on two processes, criminal investigation and intelligence, that are quitesufficiently different. It that the FBI would be wise for the FBI to view the development of an overall enterprise architecture as calling for two subarchitectures with well-defined interfaces between them. The investigation process has a long history, and the FBI has developed investigatory processes very well over the years. Thus, it would not be surprising if the subarchitecture corresponding to criminal investigations reflected these long-standing practices, though one test of its thoroughness might well be the extent to which it reflects the needs of fighting increasingly borderless criminal activities.

By comparison, the intelligence function within the FBI is relatively new, and thus the FBI has an opportunity to develop processes in this area without being encumbered by history. Thus, the intelligence subarchitecture can break new ground, and can be biased in favor of innovation. There willmay also be relevant lessons-learned from other agencies that have addressed similar intelligence challenges but in different domains. Development responsibility for the architecture should would rest with senior analysts, and management with operational responsibilities, and technologists that can inject realism in regard to available capabilities when needed.

The interface between the two subarchitectures is defined by the data that they must share. These interfaces will require policing and oversight so that only necessary data are passed between them, and it will take senior operational management attention to define what data are “necessary.”

Finally, the committee emphasizes that an enterprise architecture must be based on mission needs, and must be formulated, propagated, owned, and evolved by the operational leadership of an organization at the highest levels.1 An enterprise architecture has far more business content than technical content, and though a chief information officer should play a major role in driving the processes to create the enterprise architecture, it is the senior operational leadership who must have the deepest involvement in the process and the largest stake in its success or failure. The enterprise architecture for an organization in a dynamic setting will never be complete. It cannot be handed over to a contractor and then abandoned while waiting for delivery of a perfect solution.

The leadership of any large organization has many ongoing demands on the enterprise architecture. It., and tThe temptation is large to simply hire a CIO with a lot of technology

1 In principle, it is also possible to begin with a vision of how technology might enable missions to be accomplished—that is, to seek a technology-driven enterprise architecture. But such an approach demands an extraordinarily high level of sophistication about the capabilities and limitations of technology, and for reasons that will become obvious (when and where?), is not appropriate for the FBI.

9

333435

383384385386387388389390391392393394395396397398399400401402403404405406407408409410411412413414415416417418419420421422423424425426427

3637383940

41

DRAFT!!!! FOR COMMITTEE USE ONLY 11/25/0312/04/03 DO NOT DISTRIBUTE FURTHER!experience, delegate the task, and then forget about the problem. Given this fact, why must an organization’s operational leadership be deeply engaged in the development of an enterprise architecture for information technology? Why can’t this function be outsourced?

The committee’s experience is thatanswer without the drivesense of operational missions, technology deployments inevitably serve to automate existing functions at a relatively low level, and thus these processes remain static. Even worse, automation of existing functions can degrade the ability of an organization to carry out its missions. The reason is that the functions that a organization must serve are sometimes in conflict with each other—consider an organization that must keep sensitive information secure, from unauthorized access and yet make it available to everyone with appropriate access.

When people carry out these functions, they make judgments and ad hoc decisions on the basis of their experience and expertise. Such human flexibility has the The two results are that the businesssystem can function at an adequate level of performance, but also that these conflicts embodied in IT processes are hidden from view. Technology deployments bring out these potential conflicts, and force someone to decide, in advance, how the tradeoff should be resolved. When automation is implementednstalled without the benefit of operational knowledge, the tradeoffs involved in deciding what to deploy are made in a vacuum—and often wrongly due to paranoia. Only senior operational management is in a position to articulate explicitly how these tradeoffs should be resolved.

It is also for these reasons that the attempt to develop an enterprise architecture often results in a (highly desirable) reexamination of business processes. That is, to understand what it is that technology should do for an organization, attention is naturally drawn to the processes it is intended to support—and often the processes themselves are seen to be outmoded, unnecessary, or ripe for streamlining, since they were often based on the limitations of older technology.. Such discoveries are often valuable in themselves in that they help an organization function more efficiently even apart from its investments in technology.

3.1.3 Problems faced by the FBI

In the FBI’s case, no comprehensive enterprise architecture is in place, nor does there appear to be an adequate process in place to create one. Of most concern to the committee is that the senior operational management of the FBI does not appear to have been deeply engaged in shaping the Trilogy effort, and there appear to be no plans to obtain such engagement. (WOW! This is going to blow someone’s fuse. <<the terms have been mitigate the explosion; it would be worse to say: “senior operational management of the FBI is not deeply engaged” … What if we changed the second sentence to the following:)

The senior operational management of the FBI MUST become more deeply engaged in shaping the Trilogy effort, starting with frequent participation and review of an enterprise architecture. <<I’d rather not be wimpy.>>

Absent an enterprise architecture, it is essentially impossible for a large organization, including the FBI, to make coherent or consistent decisions about IT services in many areas and at many levels, both business and technical. These decisions include defining the appropriate data structures and linkages to other systems and data sources, policies and methods of information sharing, issues of security and the tradeoffs with information access, innovation and the exploitation of evolving technologies, and metrics of effectiveness of the IT system and its use.

10

424344

428429430431432433434435436437438439440441442443444445446447448449450451452453454455456457458459

460461462463464465466467468469470471472473474475476477

45


The absence of an enterprise architecture and the enforcement thereof has been a major contributor to the problems faced by the implementers of the Trilogy program. That is, there is no direction for the provision of an information and communication infrastructure, resulting in improvisation that has virtually no chance of resulting in an ordered, well mannered enabler for the enterprise to build upon. In fact, merely providing parts (e.g., computers and accessories, piece-part applications, etc.) is like buying brick, mortar, and lumber and expecting a builder to produce a functional building without benefit of the design drawings that are a reasonable analogy for an enterprise architecture and its attendant operational, technical, and system views. In the larger system sense, the technical view in the building construction analogy is the set of building codes for connecting subsystems and insuring the operational effectiveness of the building with the community electrical, water, sewage, and roadway systems, etc.

This is not to say that the FBI has undertaken no architectural efforts at all. For example, the Office of Intelligence has embarked on a laudable effort to develop the beginnings of its own architectural sub-framework, and spokespeople from that office were able to relate IT requirements to business process. Those responsible for the VCF project have also developed an architectural sub-framework that appears adequate to support the initial roll-out of the application in 2004, and VCF may well be a component that can be made, with some effort, to fit into the enterprise architecture when it is created. Nevertheless, these architectural efforts—important though they are—do not substitute for an overall architecture that addresses the missions of the FBI collectively.2

The FBI’s lack of a comprehensive enterprise architecture and the resulting disconnects between many information technology deployments and business processes has many manifestations. Two of the most significant are discussed below:

3.1.3.1 Confusion between the investigation and intelligence missions

Senior FBI personnel repeatedly said to the committee that the two mission areas of "investigation and intelligence are the same thing." This comment appears to the committee to reflect a failure to understand some fundamental differences between the businesses they are in, the goals of those businesses, and the underlying key business processes. The FBI has a half-dozen or so businesses, of which two of the most important are criminal investigation and counter-terrorism. The business process of intelligence analysis is used in both businesses, and the same is probably true to a lesser degree of the process of investigation. This fact does not make the mission of criminal investigation the same as the mission of counter-terrorism.

More importantly, the fact that the missions of criminal investigation and counter-terrorism both make use of the processes of investigation and intelligence does not make an IT

2 Indeed, the committee believes that the development of VCF would also have benefited from the availability of an overall enterprise architecture. But VCF development was greatly facilitated by the selection of a very experienced and computer-savvy FBI agent as project manager, whose operational insights have clearly been critical to the VCF design. Moreover, the VCF application is so close to deployment and so clearly represents a major step forward from ACS that it makes very little sense to delay its deployment. On the other hand, it may be wise to delay enhancements to VCF until an overall enterprise architecture is in place, and to ensure that the first enhancements to VCF are to make VCF consistent with the overall enterprise architecture (rather than to add additional functionality).

11

464748

478479480481482483484485486487488489490491492493494495496497498499500501502503504505

506507508509510511512513514515516517

495051525354555657

58

DRAFT!!!! FOR COMMITTEE USE ONLY 11/25/0312/04/03 DO NOT DISTRIBUTE FURTHER!infrastructure to support investigation the same as an IT infrastructureone thabt can intended to support the CIT intelligence process. For example, the measure of success of an infrastructure that supports investigation is whether it helps agents to do their investigation more effectively/efficiently. Agents are the primary actors in criminal investigations, and they task analysts; thus, the developers of an IT infrastructure for investigation should regard agents as their primary customers. By contrast, the mission of counter-terrorism is intelligence-heavy. The measure of success of an IT infrastructure for CIT intelligence is whether it helps the analyst perform analytical functions more effectively/efficiently. In the intelligence process, CIT analysts are the primary actors, in that they are the ones primarily responsible for analyzing a wide variety of information in order to identify impending terrorist actions that must be disrupted or pre-empted. Thus, agents will act on the information given them by analysts, and those actions are generally mapped into a criminal investigation. Presumably, agents also serve as collectors in the counter-terrorism mission, and therefore can be tasked by CIT analysts.

[WHAT ARE SOME OTHER CONCRETE CONSEQUENCES OF THIS CONFUSION? Ownership/stewardship of data gets murky because of this confusion, which may then result in too-lax, or too-tight, information protection. Also, we need to define a consistent set of terms to use for the names of the businesse they are in, vs the names of the key business processes]

(I think the terminology used in this section is fine)<<I just made some inserts to assure that the distinction is always clear, since the term “intelligence” has a broader connotation.>>

3.1.3.2 A “records management” approach to IT engineering.

The FBI’sThe IT engineering wasseems to have beenbe driven by a strong "records management" mindset. That is, field agents and collectors of information arehave been seen as sources of information that feed into centralized records management facilities, which then control further distribution. When the FBI’s mission was primarily law enforcement, this orientation made sense, because of the requirement to capture, store and protect evidentiary information in ways that would allow them to be used in court. That is, information had to be managed in accordance with the rules of evidence (e.g., maintained in achain of custody, and so on). Furthermore, because the law enforcement mission would be initiated by some event (e.g., the commission of a crime), it made a great deal of sense to organize, manage, utilize and control access to records based on cases triggered by those events.

However, in an environment in which intelligence and analysis have become far more important than they have been in the past, anthe enterprise architecture must reflect new business processes which are driven by different principles and reflect different requirements and constraints. In particular, analysis often will have a requirement to integrate across multiple cases, FBI developed information must be combined with information from other external sources, retention rules will be different, and the application of compartmentalization appropriate for case management may interfere with the timely access required for the analysis process.

In this environment, the records management mindset marginalizes the role of the agents who generate the data, who are understandably concerned about maintaining control over distribution and protection of sources and methods, and who may well lack of confidence in the security of large-scale and multi-pyrposethe IT systems of the records managers. The records management mindset increases the likelihood that important information, whose sources are sensitive, may not be entered at all. Agents or other information collectors may keep two sets of

12

596061

518519520521522523524525526527528529530531532533534535536537538539540541

542543544545546547548549550551552553554555556557558559560561562563564565566567

62

DRAFT!!!! FOR COMMITTEE USE ONLY 11/25/0312/04/03 DO NOT DISTRIBUTE FURTHER!“books,” one for official use and one for sensitive information, which allows themso as to maintain control over the disposition of sensitivethat information.

Note a that this problem based on realistic concerns is unlikely to be overcome by fiat. That is, a directive that agents and other collectors record officially all information, regardless of the sensitivity of its source, is likely to be quietly disregarded in many instances. Such disregard would not necessarily be gratuitous, but rather an entirely understandable reaction of collectors in the field who would be reluctant to simply trust the security of their sources to another party.

3.1.3.3 Cooperation with Other Agencies

The counter-terrorism mission requires access to and the exchange of comprehensive and timely information. This point raises the very important issue of the agencies with which the FBI will interact to exchange information. For example, under some set of circumstances, it is easy to imagine that the FBI will need to receive and/or send information from or to state and local law enforcement agencies (e.g., local police departments), its counterparts in foreign nations, or the members of the U.S. intelligence community. The identification of precisely which parties with whom the FBI will share information has profound implications for its enterprise architecture, and is indeed an issue that the operational leadership must address.

3.2 System Design Issues

This section comments specifically on a few important applications and issues, and in most cases describe the impact of the disconnect between enterprise architecture and technology planning and design.

3.2.1 The Virtual Case File (VCF) Application of Trilogy

The VCF is … (short description of VCF, and pointer to more information).

The Virtual Case File (VCF) is a new IT application designed to be the primary operations system for support of the FBI agents. The VCF was designed as a support tool for the investigative mandate of the FBI. The design process was well underway prior to the addition of the intelligence mission and the requirements for the intelligence mission were not included in the design.

The VCF is a significant step forward from the old Automated Case Support (ACS) system, and significant progress seems to have been made since September 2002. (The committee underscores the term “seems” because it was shown only a mock-up of the application, and not the application itself. That is, the committee viewed a canned presentation, because no working application was available. The committee notes that there is often a significant difference between the impression conveyed by a mock-up and what an actual application can do in practice.)

13

636465

568569570571572573574575576577

578579580581582583584585586587

588

589590591592593594

595596597598599600601602603604605606607608609610611

66


Perhaps the most important development in VCF is the appointment of a veteran FBI agent as project manager.3 To the committee, this individual appeared eminently capable of articulating user needs based on operational experience rather than uninformed speculation, and the committee believes that it is this manager’s expertise with the business side that has been a primary driver of significant progress. As such, VCF is likely to be able to serve agents well in a law enforcement capacity for the immediate future.

These comments notwithstanding, the committee is deeply concerned about a number of aspects of the VCF effort. Our concern is reinforced by the GSA’s November 2003 finding that the VCF prime contractor will fail to meet a critical delivery date for the Trilogy VCF component rollout on Dec 13, 2003. GSA officials said they are working with CSC and the FBI to determine a corrective course of action. “CSC and the FBI are taking positive steps to address the causes of program delays, and meetings are ongoing to schedule a new baseline for the program, which will now be managed by CSC's Enforcement, Security and Intelligence unit, which is led by Tim Sheahan,". The delay will at least amount to several months.

The committee is also concerned that in the absence of adequate technical specifications the delivered systems will lack functions important for the field agents and FBI management. In that case further rounds of fixes and frustrations are likely. [;;;blown the schedule on Trilogy – get exact finding].

According to FBI briefings on VCF, the FBI plans to implement the VCF in what it describes as a “flash cutover.” That is, it will be rolled out for agent usage all over the Bureau simultaneously (or nearly so). Moreover, under schedule pressure, the amount of testing has been reduced. This approach is nearly guaranteed to cause failures and further delays, as discussed below in Section 3.3.1.

Furthermore, there appear to be a number of significant limitations to VCF is to be delivered. These will require , mostly with respect to changes that may be needed in the future. These include:

VCF is driven by aAn embedded workflow model that is likely to be a significant limitation to implementing workflow changes later. For example, FBI practices seem to be built around the assumption that only supervisor-approved documents have any status as information worthy of access to parties other than the original creator. However, in a new world of greater information sharing, an unapproved draft may well have information that might prove useful to another agent or to another case, and procedures may need to be changed to accommodate a different sharing arrangement. Embedding the work flow in the application (that is, hard-wiring it) will make any such changes in the future much more difficult to implement. (Note that it is possible to develop such application with (commercial) “workflow engines” that can be configured to support any of a myriad of workflow arrangements without modify the application programs.

The lack of a VCF "offline use" mode. Without an offline mode, an agent working a case cannot take copies of case materials into the field for reference, nor create content offline which will then be synchronized when coming online. Today’s

3 In general, project managers must have two distinct sets of skills. One set of skills is process-oriented: they must be able to manage budgets and deadlines, motivate workers, and so on. A second set of skills has expertise in the subject area and makes sure the project team is doing the ‘right’ things.

14

676869

612613614615616617618619620621622623624625626627628629630631632633634635636637638639640641642643644645646647648649650651652653654655656657

70717273

74


administrative procedures may prohibit such practices, but in the event that mobile computing and remote access for agents becomes desirable and necessary, the inability to access VCF in an offline mode will become critical.

A manual indexing system, in which documents are indexed with terms that are manually specified by supervisory agents without any auxiliary capability for the automatic generation of index terms. Manual indexing reflects today’s practice, but modern indexing software can do a very good job of finding index terms in a corpus of text automatically. Manual indexing will supply terms that are outside of the context of the human indexer, as many CIT activities are likely to be. Today(Indeed, full-text index systems that index every word in a document are the foundation of powerful search engines, such as Google.) It is true that basic automatic indexing systems arewill generally be unable to provide index terms that are not explicitly represented in the text; for this reason, automatic indexing is not a replacement for manual indexing as much as it is a very powerful supplement for it.

The lack of automatic notification of changes that are made. (e.g., documents added to a case file). Case owners are made aware when documents are added to a case fileof such changes, but no one else is. In an environment in which many people may be users of a case file, such lack of notification places a large burden on those users to check the contents ofreturn to the case file periodically. They will, with the result that they waste time in checking a file that has not been altered and or they lose currencytime between the point of update and their access to the file. Lack of notification is thus an inhibitor of collaboration.

The apparent lack of bookmarking, favorites, or history features for an individual user. Without such features (commonly available on most Internet browsers), users are poorly equipped to remember where they are or where they saw interesting information. This is particularly significant when the volumes of information with which one is working are large.

The inability to sort columns listed in VCF. The ability to sort columns of names or dates or other such information is quite valuable when one has only a vague memory of the name or date associated with it. For example, a user may remember that a document was from the June 2001 timeframe. If the system does not allow a sort by date, the user is forced to search explicitly for all documents dated June 2001. This does not appear bad, until one realizes that his memory may be incorrect, and in fact the document appeared in the April 2001 timeframe. By arranging the entries according to date, the user can rapidly scan both sides of the putative date index, and the negative impact on information retrieval of errors in recollection can be reduced.

;;;;; [ED B: PLEASE ELABORATE: Consider drawing an analogy between cases and blogs. A case can be viewed as a highly stylized blog written by a team of agents and available to others. This metaphor might provide some insights in to the kinds of tools the FBI should be looking to exploit, e.g., RSS.]

The VCF interface was is needlessly inconvenient in some respects. For example, the committee saw no evidence that hot links to organizational and user information were provided, which such information could be provided easily. To increase VCF convenience, a case file should have listings of everyone who accesses the file, and provide online contact information about those users to facilitate contact.

15

757677

658659660661662663664665666667668669670671672673674675676677678679680681682683684685686687688689690691692693694695696697698699700701702703704705706707708

78


All of the above issues can be addressed in subsequent releases of VCF. But the committee’s greatest concern about VCF is not a failure of VCF per se but rather a mind-set abouttowards it. In particular, the tone of the discussion with FBI representatives was that they saw VCF as the “Solution for All Technology Shortcomings,” and that valuable information would be found only in VCF. Information “not present in VCF” meant that it was not useful information at all. [MARK, PLEASE ELABORATE]

3.2.2 Data management and the Integrated Data Warehouse (IDW)

The FBI has a tremendous quantity of data, most of which comes from its investigative processes. This data must be used to ensure effectiveness of the Bureau’s agents and intelligence analysts. Access capabilities for CIT intelligence analysis to determine possible events in the future often differs greatly from those required for reliable case records management, which has the mission of organizing and retaining information to meet legal requirements.

Thus, the information requirements of the Investigative and the CIT intelligence missions of the Bureau are very likely to require significantly different work processes and information access. It is very likely (nearly certain) that different data models will be required in order to provide efficient support for these separate missions. Furthermore, each mission of the Bureau has unique constraints for access to the information will require different security models. Implementation of these models is likely to show that distinct systems, with overlapping contents, may be needed to assure manageability in the presence of conflicting constraints.

The IDW is … (short description of IDW, and pointer to more information).

Briefings to the committee on a data model for the investigation mission suggested that efforts in this area were just getting started, and were in danger of being ignored by the vendor. Further, the data models of the integrated data warehouse (and the VCF for that matter) seem to be far too abstract to be very useful, though the presentation from the analysis side recognized that were differences in need between the investigative and intelligence analysis functions.

Three examples will suffice to illustrate a disconnect between data models and operational needs:

o Presentations to the committee suggested a mismatch between the expectation that intelligence analysts would have access to live databases containing the most current information and the design of the FBI’s data warehouse, which seems to be designed around copies of production databases. Copying (thus raisesing the issue of the rate at which data is copied from production databases to the warehouse).

o The data warehouse appeared to have been designed to overwrite old copies of databases with new copies, so data can be there one day, and not the next (that it, it was not equipped to handle time-series of versioned data). While having only the most recent copy of data may be appropriate for the purposes of an investigation (presuming the most recent copy is the most accurate and reliable), this assumption may not be valid for intelligence purposes.

16

798081

709710711712713714715716717

718719720721722723724725726727728729730731732733734735736737738739740741742743744745746747748749750751752753754755756

82


o Some of what was presented when the committee asked about data models were not data models at all, but rather hierarchical XML descriptions of various criminal justice documents (e.g., arrest forms, booking reports, sentence orders, etc.).

To illustrate what a high-level enterprise perspective on data management might entail, consider that IT systems for the FBI must serve multiple business functions. From the committee’s perspective, the major classes of data are:

The active case records used by the field agents - these are to be served by the VCF system under development.

A broad-based data warehouse to serve intelligence tasks now assigned to the FBI - these were served by the SCOPE prototype demonstration and are to be served by the IDW follow-up projects.

A reliable repository-of-record (RoR), that will serve the needs of legal prosecution of criminal cases.

A wide variety of management and administrative databases (ADMIN) for personnel, capital and registered item inventory, evidence tracking and inventory, etc., to allow FBI HQ to carry out their responsibilities.

Data in each of these classes are not entirely independent of each other, and thus applications to manage these types of data must have linkages among them. However, the fact that the applications must have linkages among them does not mean that the databases must be co-located or should be managed by the same staff under the same rules.

To understand fully the high-level requirements for these applications, their objectives must be made explicit. From the committee’s understanding, these objectives are as follows:

The VCF must:

1a. Serve the field agents and their supporting analysts, wherever they are, in tasks of information collection and analyzing the collected information on their cases.

1b. Be available 24/7.1c. Allow convenient upgrading of services as needed by the agents and enabled by

technology.1d. Provide linkages to data in IDM and other FBI databases, and allow insertion of

records from other sources that have been determined to be pertinent to a case by the responsible agent and his/her supervisor.

1e. Purge closed cases.1f. Keep all the information secure vis-à-visvis-à-vis outsiders.1g. Keep some elements secure vis-à-visvis-à-vis most FBI personnel.1h. Be flexible enough to accommodate changes in business processes that may be made

in the future.1i. Resist tampering with records. 1j. Support an appropriate degree of auditing before information is entered.

The Integrated Ddata Wwarehouse must:

2a. Serve broad intelligence functions of agents and analysts.

17

838485

757758759760761762763764765766767768769770771772773774775776777778779780781782783784785786787788789790791792793794795796797798799800801802803804805806807

86


2b. Acquire, integrate, and store data from other sources of potential relevance (OSoPR) for broader analyses, as sources from the DoState, Customs, INS, foreign sources, etc..

2c. Allow mining of information that is not (yet) related to a specific case, within 20 minutes of an incident.

2d. have the capability to store tentative conclusions, and to purge such information if determined to be misleading.

2e. Be flexible to support an increasing variety of data mining and analysis tools to be installed as they are found to be useful.

2f. Its content must be secure vis-à-visvis-à-vis outsiders and unauthorized insiders or turncoat insiders.

2g. It will not contain information that would inhibit access by any FBI intel qualified personnel.

2h. Support an appropriate degree of auditing before information is entered.

The Rrepository of Rrecord must:

3a. Serve broad investigator [STRUCK intelligence] functions of agents. ;;STRUCK this:primarily specialized, authorized personnel, since it will include sensitive data.

3b. Serve legally required documentary archival storage requirements, with complete audit trails, primarily to serve cases under prosecution or review.

3c. Maintain reliability and integrity, even at the expense of 24/7 availability and flexibility.

3d. Be fed primarily from the VCF, and exclude OSoPR material that is not the responsibility of the FBI.

3e. Support only a limited number of validated tools to prevent tampering, and will not be reorganized frequently. [;;;WHAT DOES LIMITED MEAN?]

3f. Not be routinely used as a working resource by FBI personnel.3g. B be maintained at a very high level of security.3h. Support appending of contents rather than replacement of record purges.3i. Resist tampering with records. 3j. Support an appropriate degree of auditing before information is entered.

Administrative databases must:

4a. S serve a variety of FBI management and administrative functions.4b. Be maintained and updated primarily by HQ personnel.4c. Be accessible to Field personnel for determining location of their colleagues,

resources, equipment, etc.4d. Be available at close-to-a-24/7 schedule, although updating of information

maybe constrained to business hours.4e. Be secure vis-à-vis outsiders.4f. N not contain information that would constrain their effective use by FBI

personnel.4g. Evolve to align with DOJ standards for administrative support systems (e.g.,

finance, e-mail, etc.). Support an appropriate degree of auditing before information is entered.

4h. Support an appropriate degree of auditing before information is entered.

18

878889

808809810811812813814815816817818819820821822823824825826827828829830831832833834835836837838839840841842843844845846847848849850851852853854855856857858

90


The FBI may disagree with this listing of essential objectives. But what is most important is not agreement in detail but rather that the FBI develops its own list of essential objectives, based on its own understanding of its essential business processes and how it expects to use these databases. In any event, the diversity of requirements makes it clear that these storage applications must be logically disjoint, and that different policies will be needed to assure the proper balance of accessibility, security, availability, and reliability.

It is likely that there will be a fair amount of duplication of contents in these applications. However, the cost of replicated storage is less than the cost of manpower needed to maintain a single complex system with multiple and potentially contradictory objectives. With multiple copies tThere is also a huge probability of inconsistent and /or out-of-synch information. This is especially true with information that can change over time. Cleaning and validating information will become a separate process unto itself.

One possible approach to help think through the differences in requirements for these four systems is to consider the disposition of the 6 million documents found in Afghanistan and currently being scanned. Once scanned, where do these documents go? A logical first step would be to translate these documents into English. Because of the volume, an automated translation process could be used to select key documents for human translation. In any event, translated documents, whether machine- or human-translated, would seem to belong in the information systems associated with the intelligence process. After analysis, some of the material becomes relevant to an investigation and finds its way into VCF, and then some of that material would be passed to the repository of record.

As in other examples in this report, this scenario represents a committee view (and is in any case overly simplified). The FBI may have its own views on how to handle these documents, and those views may well differ from those of the committee. But independent of however it decides to handle them, the FBI should be able to articulate the rationale for doing so in terms that make sense from an operational point of view.

3.2.3 SCOPE

The FBI demonstrated the Secure Counterterrorism Operational Prototype Environment (SCOPE) to the committee. This demonstration illustrated the analytic tool suite with synthetic data. The analytic tools were based on commercially available products that included support for data visualization, relationship analysis and automated language translation. Because this demonstration was not done using operational data or the Trilogy network, we cannot comment on the performance or scalability of this approach.

The Committee believes that the FBI has an important opportunity with SCOPE. The FBI is not burdened by a legacy of analytic tools and data models. It has a unique opportunity to assemble and integrate the most promising analytic ideas and tools. A well thought out Enterprise Architecture and data models are a critical to successfully exploiting this opportunity and creating an environment that can incorporate the best analytic ideas and products over time.

The FBI’s use of commercial products is laudable. This willingness to look beyond conventional sources for the best available ideas can provide the FBI with an analytic advantage. The commercial market for such tools is being driven by much broader trends that just counterterrorism. These trends include the private sector’s desire to analyze and support market

19

919293

859860861862863864865866867868869870871872873874875876877878879880881882883884885886887888889890

891892893894895896897898899900901902903904905906907908

94

DRAFT!!!! FOR COMMITTEE USE ONLY 11/25/0312/04/03 DO NOT DISTRIBUTE FURTHER!data, and the increasing globalization of markets. By harnessing the commercial market for analytic tools, the FBI can gain access to far more creativity than it could ever capture with internally developed tools. Due to security concerns commercial tools may not be appropriate for all FBI databases, but the IDW should not contain data that will inhibit the use of modern commercial tools. An approach rooted in commercial products need not rule out one-of-a-kind tools that are internally developed.

3.2.4 Mobile computing

The FBI’s interest in wireless data communications appears to be driven primarily by business continuity considerations. For the most part, the FBI’s experience with wireless data communications has been through Personal Digital Assistants (PDAs) used by a small number of senior leaders. Anecdotal reports suggest these individuals have found wireless data communications enhances the conduct of day-to-day business.

To make further progress in this area, the FBI is planning a trial of PDAs at a field office. Such a trial can be the basis for establishing a baseline characterization of agent and other activities in that office to understand if the deployment of PDAs has any impact. Thinking about how to create a baseline characterization of office activities can also contribute to quantifying an office’s implementation of the FBI’s business processes that are the critical input to the FBI’s Enterprise Architecture. The planning and results of this trial should be coordinated with the FBI’s Enterprise Architecture efforts.

3.2.5 Security

Loosely speaking, the security of an information system or network refers to its ability to continue to support authorized parties when it is under attack and its ability to resist the efforts of unauthorized parties to compromise its operation. For this report, it is helpful to conceptualize security issues along two dimensions—those that arise from the FBI’s disconnect between operational mission needs and enterprise architecture, and those that arise from inadequate thought about the implementation of whatever security policy is adopted.

The FBI’s disconnect between operational mission needs and enterprise architecture is reflected in the FBI’s expression to the committee of a philosophy of zero tolerance for security risk and hence sought not to manage risk but to avoid it. Given that the FBI’s information systems are prime targets for criminals, espionage organizations, saboteurs, and terrorists, this sentiment underlying this view are entirely understandable. Nevertheless, the only approach that truly provides zero risk is one that destroys all information as soon as it is collected or otherwise to make information entirely inaccessible to anyone.

A zero tolerance for security risk is even more unacceptable in an environment in which information must be shared. There is a deep and unavoidable tension between information sharing and information security, and while there are mechanisms that allow design architects to mitigate this tension is some tensions (a matter of implementation discussed below), at some point the senior leadership must decide what degree of security risk is acceptable in return for the benefits of broad-based information sharing.

For example, it is the committee’s understanding that intelligence analysts and agents are not routinely provided with Internet access for security reasons. It is certainly true that a lack of

20

959697

909910911912913914915916

917918919920921922923924925926927928929930931932

933934935936937938939940941942943944945946947948949950951952953954955956957

98

DRAFT!!!! FOR COMMITTEE USE ONLY 11/25/0312/04/03 DO NOT DISTRIBUTE FURTHER!Internet access will prevent Internet-based security compromises, but it also impedes the use of the Internet to search for potentially valuable open-source information. This is a reasonable policy only under the implicit assumption is then that open-source information is not useful in either intelligence or investigation—and this assumption is notwill become even less tenable foras the counterterrorism mission becomes more important. Furthermore, by isolating FBI staff from the Internet, they are not educated by casual exposure to the best new ideas and tools emerging in the global Internet community. While there are individuals with deep expertise, often gained by dedicated people spending their own time and money, and there are specific provisions for controlled access to the Internet, there is no substitute for the institutional and cultural impact of broad availability of Internet access to all FBI staff at all times and places.

Another important issue with profound consequences for security is raised by the attempt to make a single physical IT infrastructure serve the needs of both intelligence analysts and law enforcement investigators. Much intelligence information is classified, where there are rigid security requirements imposed by legal and national policy. However, much data related to investigations is unclassified (SBU or law-enforcement sensitive). Devising technologies and policies to manage both kinds of data under one rubric is thus potentially contradictory, unless one is willing to accept the severe operational penalties of treating unclassified information (SBU or law-enforcement sensitive) as classified.

Intelligence analysts also have a need for relatively unfettered “browsing” through information. (Investigators mightshould not have a need to browse if they are getting adequate analytic support. Browsing by an agent is probably a warning flag of weaknesses on the analytic side.) <<I think that is a narrow-minded statement, browsing is now a natural component of investigative process in early phases. It may lead to concepts that then can be delegated to analysts.>>On the other hand, sensitive information (e.g., names of informants) can be derived or deduced from aggregation and inference from large amounts of apparently unrelated data.

There are ways to manage these tensions, such as allowing intelligence analysts access to “pointers” to protected internal information rather than the information itself, so that the analyst may make a direct request for access to the party responsible for protecting that information. In the area of unclassified versus classified data, it would help to attempt to balance the threat of misuse or inappropriate access against the benefits of allowing more use, e.g., by remote agent access in the field; this process is known as a risk management assessment. Nevertheless, the tension cannot be eliminated, and only the senior operational management is in a position to judge how this tension should be resolved.

A third security tradeoff will manifest itself when mobile computing becomes an issue. The value of mobile computing is likely to be high, but for a variety of reasons, mobile computing is inherently more vulnerable than office-based computing, and the senior operational leadership will have to decide if the increased security risks are worth the added operational flexibility. Empirical data on mobile computing would be useful inputs to the leadership, and the FBI’s PDA experiment will help the leadership to think through how to differentiate and measure communications that involve classified information, sensitive but unclassified (SBU) information, and unclassified information, each of which have different security requirements.

In the absence of any specific information on the subject but based on previous experience, the committee suspects that a significant fraction, if not most FBI data communications traffic, need not be carried at the classified level. If the measurements associated with a PDA trial support this conjecture, the FBI will be in position to take advantage of wireless

21

99100101

958959960961962963964965966967968969970971972973974975976977978979980981982983984985986987988989990991992993994995996997998999

100010011002100310041005100610071008

102

DRAFT!!!! FOR COMMITTEE USE ONLY 11/25/0312/04/03 DO NOT DISTRIBUTE FURTHER!data communications in designing an infrastructure, systems, and processes to implement its Enterprise Architecture. Such systems and policies could differentiate between SBU and unclassified communication, handle them differently, and significantly improve FBI measures of efficiency and effectiveness. This would also require the FBI to establish a clear policy governing SBU communications.

The committee is also concerned about a number of security issues related to implementation. These include:

The use of passwords for authentication. The weaknesses of passwords as an authentication mechanism are well-known (e.g., they are easily compromised without the owner’s knowledge), and yet no thought seemed to have been given to alternatives.

The lack of consistent security models between the data warehouse and the VCF. In particular, this inconsistency suggested that searches of data contained in VCF from the data warehouse side would not be visible to case owners.

The operating system monoculture. The Trilogy IT modernization put into place a single operating system environment, and the security vulnerabilities of an OS monoculture are well-known.4 However, the deployment plan did not seem to take into account the possibility of worms and viruses being introduced into the system—and the fact that the system is not connected to the Internet provides no assurance at all that worms and viruses will not be introduced. While there is a good prevention program in place to minimize the risk of such an attack, it is not realistic to expect that even great diligence will be completely effective.

Contingency plans for operating under attack. A basic principle of managing a critical operational network is that plans for maintaining operation in the face of a compromised element must be made in advance.

- How are nodes in the network protected against other nodes that may have been compromised? (the principle of mutual suspicion)

- How will the FBI’s PKI be managed in the face of an insider threat5?- How will key revocation be handled? Emergency re-keying? Access overrides (i.e.,

how will authorized parties obtain access if access tokens and/or information is lost?)- How will worms and viruses be purged from the system if they are introduced?

Validation of approaches to security, which the committee found, on the basis of presentations, was missing or incomplete. Specifically, the FBI did not provide to the committee evidence that it had validated:

The putative design principle for security, which seemed to be characterized by an approach of "Build to DISA Gold Standard and back off until tools work."

The model of the threat that faces the FBI’s IT systems. (Note that a threat model must include threats emanating from both inside and outside the organization.)

The use of the NSA-developed electronic key management system in the context of the specific needs of the FBI.

4 Cite to PNPL5 Footnote on definition of PKI

22

103104105

10091010101110121013101410151016101710181019102010211022102310241025102610271028102910301031103210331034103510361037103810391040104110421043104410451046104710481049105010511052105310541055

106107

108


In each of these cases (the design principle, the threat model, the use of the NSA EKMS), the approach may be plausible and ultimately desirable. But a process of systematic validation is necessary before those conclusions can be drawn.

- The security model in ACS has been preserved in VCF, namely policy-based mandatory plus discretionary access controls. (Marc Seiden, pls explain SO WHAT?)

- Misclassified documents or copies of classified documents in accounting files accounted for a sizeable proportion of what Robert Hanssen sold to the Russians. How will such issues be addressed in a VCF environment. For example, having analysts review documents for classification, cluster analysis of documents to try to identify differently classified documents with closely similar content. (Marc Seiden)

- There is a need to log all outgoing information, and routinely and automatically process those logs. As long as security mechanisms only provide access control, and not monitor what is released, then misuse of systems by authorized users will not be visible until they turn up in the wrong places. <<gio>>

- Security aspects of RFID. The FBI has proposed tagging equipment so that it can easily be tracked by the FBI. However, if an RFID tag, even a passive one, is attached to equipment used by FBI personnel, then it is possible to illuminate the tags and get them to respond. An adversary could build a device to look for tags. (Do we want to mention this?)

3.2.6 Privacy

As part of the FBI’s briefings to the committee, the FBI’s privacy office addressed the committee, from which the committee concluded that the FBI did have some sensitivity to some of the relevant privacy issues. However, the committee has not undertaken a comprehensive study of privacy in the context of the FBI’s IT modernization program, and thus the comments below can only suggest some of the issues that may be involved.

The FBI must proceed with great sensitivity to privacy issues as the counterterrorism mission assumes greater importance, and both substance and perception are important in this domain.

There are many substantive issues associated with privacy. For example, - the FBI privacy office seemed to the committee to be geared primarily to

protecting personally identifiable information from being improperly shared outside the FBI (e.g., with contractors), and in the short presentation to the committee, their approaches to this problem seemed reasonable.

- Another privacy issue not addressed in briefings but of great concern to the privacy-sensitive segment of the public is the protection of the public from the abuse or improper use of such data by rogue FBI employees acting on their own or through some official though perhaps not publicly acknowledged FBI program. The committee has no comment on this issue, other than noting that the simple exhortation “trust us” is not likely to be reassuring to this segment even if in fact the FBI is doing everything possible to prevent such abuses from happening.

23

109110111

105610571058105910601061106210631064106510661067106810691070107110721073107410751076107710781079

1080108110821083108410851086108710881089109010911092109310941095109610971098109911001101110211031104

112


Perception is at least as important as substance in this domain. That is, a technically well-conceived privacy protection program could fail to reassure the public because privacy issues are handled improperly.

3.3 Management issues

This section addresses a variety of serious management issues in the FBI’s technology modernization program.

3.3.1 Overall development methodology

In the committee’s judgment, the management approach to the Trilogy modernization violates a number of basic principles that should govern the development of large systems.

Principle 1: Any organization undertaking a large-scale IT system development and deployment, but especially an organization without a strong track record in IT development and use, should develop a small-scale prototype before committing itself to an organization-wide program.

The development methodology for Trilogy seems to be based on a one-way process where rigid specifications are generated in advance. and that sSubsequent changes are can be argued to be “specification changes” that open the door for schedule delay and increased expense. In truth, it is essentially impossible even for the most operationally experienced applications developers to be able to anticipate in detail all of the requirements and specifications in advance. Therefore, development contracts should not make such an assumption, but rather should call for an approach to requirements specification that is based on a process of extensive prototyping and usability testing with real users.

To the best of the committee’s knowledge, no prototype has been developed for any of the major components of Trilogy. The advantage of a prototype is that it can be iteratively developed with strong user feedback and involvement, thus increasing the chances that what is ultimately delivered to the end users meets their needs. This point is relevant to many dimensions of system development, including the functionality desired in a new application, the convenience and intuitiveness of a user interface, and the nature and mix of the operating load that the system must support under real operational conditions.

In some instances, the intimate involvement of project managers with extensive operational experience can play an important role in some of these dimensions (e.g., defining the required functionality); indeed, the committee believes that it is the involvement of such an individual in the development of the VCF that has been responsible for what success it is likely to have when it is deployed. Nevertheless, in the end there is no substitute for realism in the evolution of a prototype.

Principle 2: Any organization that depends on the continued operation of its IT systems and is modernizing those systems should plan on the simultaneous operation of both old and new systems for some period of time, so that large-scale failures in the new system do not cripple the organization.

24

113114115

1105110611071108

1109

1110111111121113111411151116111711181119112011211122112311241125112611271128112911301131113211331134113511361137113811391140114111421143114411451146114711481149115011511152

116


Large-scale deployments involving new technology almost always come with problems, and tThere is some risk that the problems will be so severe as to prevent their effective use of the entire system for some period of time. Accordingly, there must be backup and contingency plans in place that anticipate a wide range of (temporary) failure conditions. However, the committee saw little evidence that such plans had been formalized, perhaps due in part to the cost of making such plans and undertaking the acquisitions or retaining equipment to prepare for the implementation of such plans. It seems that there is inadequate hardware available for concurrent use of ACS and VCF. (In fact, the committee would be highly surprised if the old ACS system were could be turned off and scrapped at the same instant that the new VCF application were turned on—but in the absence of contingency plans that anticipate a continuing if temporary need for ACS, ACS availability will be hampered by the need to proceed in an entirely ad hoc manner. It is unclear if users’ terminals, once their software is upgraded to the level needed for VCF, can still access ACS systems.

Principle 3: In large-scale system development and deployment, testing should account for as much as 50% of the project schedule if a successful deployment is to be achieved.

Testing is necessary to shake out bugs and flaws in a new system. Flaws include failures to retrieve needed information for a case, and will not be visible until after actual users deal with actual data. , and tThe 50% figure above is a rule of thumb that is widely accepted among those experienced in large-scale system design. Moreover, an organization without a strong track record in information technology could reasonably be expected to require even more time for system testing and integration.

However, the project schedule as it currently exists appears to leave inadequate time for testing. The committee was shown briefing charts indicating that the FBI allocated less than 10% of its schedule for testing, and under schedule pressure the contractor was trimming that amount even further. It was also surprising for the committee not to see, at a very late stage in VCF development, any kind of risk-based prioritization for the items remaining to be done. Instead, these items were lumped together in the category "unfixed bugs."

To provide a simple example of the need for testing, it is important to realize that the term “works” (as in the system “works”) has a wide spectrum of possible meaning. Consider a simple system, consisting of a network transport layer that moves packets and is responsible for encryption and decryption and an application running on top of it. Beginning with the least demanding definition, the expression “the system works” can plausibly mean that:

(1) The transport layer "lights up" but does not move application packets. Nodes can find each other, basic protocol functions such as “ping” work, and bits flow between a subset of the network and on the whole network.

(2) Definition (1), and in addition data packets move between a subset of the network and on the whole network.

(3) Definition (2), and in addition encryption and decryption is successful between nodes on a subset of the network and on the whole network.

(4) Definition (3), and in addition the transport layer can move application packets under simulated "realistic" load between nodes on a subset of the network and on the whole network.

25

117118119

115311541155115611571158115911601161116211631164116511661167116811691170117111721173117411751176117711781179118011811182118311841185118611871188118911901191119211931194119511961197119811991200120112021203

120


(5) Definition (4), and in addition users in different locations can use the application to pass the appropriate information between themselves.

(6) Definition (5), and in addition functions with actual users and actual data

(7) Definition (6) with realistic numbers of users and the full complement of files.

In the absence of a clearly specified test plan that is acceptable to all users, it is possible to make a straight-faced claim that the system works with any of these definitions, even though the point of failure of most projects is when they must continue to function under load. It is unclear which definition of `working’ is being used. Since currently the users systems have not been upgraded to the level needed for VCF it is clear that the higher levels have not yet been attained.

3.3.2 Contracting and Contract Management

[DOES THIS SECTION NEEDS TO BE ELABORATED MORE? IF SO, HOW??]

Both key contracts for Trilogy were awarded on a Cost Plus/Cost Reimbursable basis. Only optional “Tools” were awarded on a fixed-price basis. While the Task Orderscontract documents (#T0001AJM026 and #T001AJM028) for both phases of the Trilogy program detail the Firm Fixed Price to eight or nine significant figures, the Task Order Schedules are almost totally lacking in specifications and commitment to checkpoints, so that program and contract management becomes essentially impossible. The documents list a large number of plans to be created, with dates “TBD”;, we were unable to find any specifics. This weakness, aggravated by frequent turnover among key FBI players, makes it unsurprising that Trilogy is significantly behind schedule and over budget.

To illustrate some of the problems with contract management, the FBI told the committee that the Trilogy network had been made operational on [DATE]March 28, 2003, only because FBI personnel in the program office were pressed into overtime service to compensate when contractors have failed to meet commitments. While such efforts point to the FBI’s admirable dedication to duty, the need for the program office to stand in for a contractor is a sign of contractor failure. If these these “above and beyond” efforts of FBI staff could be converted into penalties for (or dismissals of) the contractors, it would send a very positive signal that the FBI program office is getting on top of contract management.

For a contract of this magnitude and importance to the FBI and this country, it is imperative that senior management of the FBI take an active role in monitoring contractor progress. A Senior-level contract manager, experienced and empowered, should be assigned for the duration of the project. In briefings, it appeared to the committee that contractors may be viewing this project as governmental “business as usual,” without due regard to the critical importance and congressional visibility of Trilogy’s success/failure.

Clear and detailed schedules with intermediate milestones, earned-value metrics, and severe penalties for missed dates and functionses are desperately needed.

26

121122123

12041205120612071208120912101211121212131214121512161217

121812191220122112221223122412251226122712281229123012311232123312341235123612371238123912401241124212431244124512461247124812491250

124

DRAFT!!!! FOR COMMITTEE USE ONLY 11/25/0312/04/03 DO NOT DISTRIBUTE FURTHER!3.3.3 Program/Contractor Management (I suggest putting all Contract

Management issues in 3.3.2)

In the committee’s judgment, program management of IT projects appears is weak, as it did appeared to many committee members in September 2002. The most serious issue is that essential tasks have been outsourced and delegated to vendors (e.g., data models and architectures). In essence, the FBI has left the task of defining and identifying its own essential business processes and its IT concept of operations to outsiders. This is not to say that outside contractors should have no role at all in these tasks, but rather that it should be the FBI and not the contractors that are defining and driving the process.

A small but telling example of the FBI’s dependence on contractors is that the FBI reported no contract provisions calling for the escrow of source code for the applications. Source code escrow is a common, even routine, practice in commercial firms in which the vendor of an application continues to own the source code associated with it. The practice calls for the deposition of source code files and appropriate documentation with a mutually trusted third party after the completion of the contract. In the event that the vendor becomes unable or unwilling to continue to provide service to the firm (e.g., the developer goes bankrupt), the source code is released to the firm so that it may seek another vendor to assume the first vendor’s responsibilities. Source code escrow is a common, even routine, practice in commercial firms in which the vendor of an application continues to own the source code associated with it.

A strong program management function must be established and supported by FBI management at all levels to provide appropriate oversight and management of FBI IT projects. An effective program management function will provide the FBI with a focal point for monitoring and collecting project data and allow for the reporting of the progress of active IT projects based on well-defined metrics.

For program managers to be effective, they must participate in the negotiation of contracts with outside vendors. The contracts must include performance measures for key deliverables, milestones, and service levels with penalties and escalation procedures. Contracts should ensure the vendor cannot easily exit the relationship if it is not meeting the performance measures, and vendor failure should result in a penalty that allows the FBI to transition to a new vendor with no financial impact to the FBI.

Program managers are responsible for validating, monitoring, supporting, and assisting in the area of project and life cycle costs. They may also supply information that can alter project priorities, based on resource availability or interdependent project conflicts. Most important, the program management function is responsible for the project management methodology, including at least the following:

The establishment of a framework for all project communications, reporting, procedural and contractual activity.

Development of task-specific project plans with timelines, critical paths, and specific deliverables.

Conducting project team meetings; set schedules and agendas and manage the meeting.

Tracking of milestone and deliverables to timelines. Providing project status updates to key stakeholders Escalation of deviations from plan and issues on timely basis for resolution.

27

125126127

12511252

125312541255125612571258125912601261126212631264126512661267126812691270127112721273127412751276127712781279128012811282128312841285128612871288128912901291129212931294129512961297129812991300

128


Management of project budgets. Creation of an environment where project team can succeed.

Weak program management leads to a reactive mode in dealing with issues and a lack of overall project control. Continued cost overruns and delays often result from a lack of effective program management. Alternatively, effective program management with effective management of cost and proactively managed IT projects significantly increases the likelihood of success.

A particularly useful management tool is a project charter. The charter includes detailed project requirements, resources and their roles on the project team along with identified milestones and deliverables. Risk, issue and scope management, as well as communication plans are included.

The project charter is designed to do several important things early in the project. The charter should:

Facilitate communication among stakeholders. Document approved purpose, scope, objectives, cost, and schedule baselines. Document the agreement between groups. Define roles and responsibilities inclusive of the overall project governance. Document planning assumptions and decisions. Provide the baseline for scope and expectation management.

In effect, the charter establishes a baseline of understanding among all stakeholders and especially between the project manager and the project sponsor(s), business owner(s), and vendor(s). The success of the project may be measured against this charter and importantly any changes must reference the charter.

3.4 Human Resources and External Constraints

The FBI recognizes, and the committee agrees, that the FBI is severely lacking in many of the technical skills to successfully plan, contract for, and implement a project of the breadth and scope of Trilogy. This situation must be remedied both to complete the initial phases of Trilogy, and more important, to deal with the issues described in the preceding sections. In addition to filling the top jobs such as the CIO position with people who are willing to and capable of effectively managing the larger issues, we cite a number of other gaps and constraints, some not of the FBI’s making.

3.4.1 Human resources

With a few exceptions, the presentations to the committee persuaded it that FBI lacks a human resource base adequate to deal with its IT modernization program.

The FBI lacks experienced program managers and contract managers, which has made it unable to deal aggressively with its contractors. For example, inexperienced managers generally lack the ability to assume proactive management roles, and are often held hostage to the perspectives of the contractor. Thus, their reactive tendencies partly

28

129130131

1301130213031304130513061307130813091310131113121313131413151316131713181319132013211322132313241325132613271328

1329

1330133113321333133413351336133713381339

13401341134213431344134513461347

132


underlie a contractor perception that the FBI can be bullied and jerked around with respect to IT matters.

Many of the present senior IT management team have poor communications skills. They did not exhibit ability se folks don't know how to judge an audience and prepare a brief that gives a coherent picture of their vision and objectives.

The FBI appears to have a personnel mind-set in which there are agents, and then there are all the others. Given the importance of IT personnel and analysts to the FBI’s core missions, such individuals must have career tracks that are regarded as worthy of respect and status within the Bureau.

On the positive side, the committee wishes to single out as especially noteworthy the head of security for the IT modernization program (Dean Hall) and the EAD for Intelligence (Maureen Baginski). Both of these individuals have given serious thought to their respective problems from the user perspective. For example, Maureen Baginski was prepared with a coherent and well-articulated concept of operations, as well as a clear vision of how to draw talent from the rest of the FBI. Dean’s presentations to the committee, though not long enough to provide very much detail, were also well-considered.

3.4.2 External constraints

[ALL OF THIS NEEDS TO BE AUDITED]

Management flexibility . Agile organizations are managerially flexible. Within a top-line total, they have great flexibility in moving people from one specialization to another, in allocating facilities and space, and in procuring equipment. Such flexibility means that prompt action can be taken. It is the committee’s understanding that the FBI is unable to take actions such as closing field offices, acquiring facilities of 10,000 square feet or more, or reprogramming amounts in excess of $500,000 without explicit Congressional approval. This constraint seems is inconsistent with the expectation that the FBI will move quickly and forcefully to reshape itself to deal effectively with the terrorism challenge.

Salary caps . Another serious obstacle is the fact that the FBI does not have the benefit of salary cap exemptions for IT positions enjoyed by other agencies in the intelligence community. This is a particularly serious impediment to obtaining top talent in the IT domain – and this is remains true despite the downturn in the IT sector.6

FBI Personnel commented on more than one occasion that their parent agency, the Justice Department, often sets contract directions or limitations by which the FBI must abide. While this may be proper, such guidance appears to have harmed, more than helped, the FBI’s contracts for Trilogy. Contract limitations/directions set by the Justice Department (Rich Baseil- PLEASE ELABORATE

6 Downturns in the IT sector are reflected in higher rates of unemployment for IT workers, but it is a reality of the IT industry that star performers generally do not have difficulty retaining employment. Indeed, an argument is often made that especially during a downturn, star performers are more necessary than ever if a company is to remain competitive.

29

133134135

134813491350135113521353135413551356135713581359136013611362136313641365136613671368

13691370137113721373137413751376137713781379138013811382138313841385138613871388138913901391

136137138139140

141


Audit pressure. The FBI reported that it had undergone over 100 investigations into the FBI’s performance this year. Responding to such investigations uses the time of senior management, both in preparation and presentation of material to those investigators. And, because personnel in the IT field are scarce within the FBI, the pace of work is slowed by such investigations.

Unrealistic personnel disqualification. The FBI reported that at least one candidate for a senior IT management position was disqualified because of a prior history of drug usage. In particular, the FBI reported that their requirements call for an individual to have abstained from illegal drug usage entirely for the past 3 years and to have fewer than 15 individual incidents of drug use in his or her lifetime. While the requirement for the recent past is entirely reasonable, few most creative IT people lack exhibit a deep and broad experimental streak, and this streak seems to correlate with youthful recreational drug usage. F – and for these people, the lack of an exemption for drug usage in early adulthood (say, before age 25), is highly problematic. Any concerns with the relaxation of a requirement here might be addressed by more frequent or deliberate monitoring post-employment.

;;;[I think there is also an issue with foreign nationals. I understand that the FBI does not hire first generation citizens. IMHO, there is a tradeoff between a perceived improved confidence in hires with an ability to find people who are effective in dealing with multi-national issues and foreign assignments. We might want to probe and comment on this point as well.]

4. RECOMMENDATIONS

What do you all think about making this statement: “The FBI’s current approach to IT modernization is not working well, and based on recent history, is not likely to be much improved without a major stimulus or event.” <<yes Gio>>

Culture change is absolutely imperative. While the enterprise must keep the forward-thinkers among its force of experts, it must also cull out the elements in its leadership structure that are resistant to the changes necessary to implement an information age organization. The current “force transformation” initiative in the DoD is an example of such sweeping change. While not all (or, perhaps, even a majority) of the DoD leadership is in tune with or understands what Force Transformation entails, many senior leaders have been replaced by those who do “get it.” New force programs (e.g., Future Combat Systems, software defined radio systems (Joint Tactical Radio Systems), while resisted by many in the Department, are now on contract—and many of the resistors are looking for new lines of work. The FBI must follow suit, or it could find its most prized objectives resident in other agencies and systems.

(The following sections should be resequenced to align with the major preceding sections, e.g., architecture, then system, then management, …) Shouldn’t we put Security recommendations with System Design, since that’s where we described the problems?)

30

142143144

13921393139413951396139713981399140014011402140314041405140614071408140914101411141214131414141514161417

1418

1419142014211422142314241425142614271428142914301431143214331434143514361437

145

DRAFT!!!! FOR COMMITTEE USE ONLY 11/25/0312/04/03 DO NOT DISTRIBUTE FURTHER!4.1 Regarding enterprise architecture

The committee believes that the development of a full enterprise architecture should have the highest priority in the FBI’s IT efforts. In the committee’s judgment, adoption of the following recommendations will improve the likelihood of success of such development.

The FBI should define a small team (perhaps a half-dozen people) to develop the broad outlines of the enterprise architecture, a task that should take no more than 4 months. This team should be composed primarily of senior operational managers from CID, the Office of Intelligence, and [NAME OF DIVISION FOR COUNTER-TERRORISM], with a senior IT architect to translate what these managers say into architectural terms. This team should have direct access to the most senior management of the FBI, including the Director and Deputy Director, to resolve issues that cannot be negotiated among themselves, as well as to staff support who understand the old and new business models.

The FBI’s top leadership, including the Director, must make the creation and communication of a full Enterprise Architecture a high priority. While the CIO can drive the process, ownership of the success of the process must reside with the top leadership team. They must be personally involved in the key decisions which the process will drive, such as the tradeoffs between security and access in the various data sources which can be used for both criminal investigation and counter-terrorism.

A reasonable first step in developing this architecture would be to define, by job, the information necessary for each FBI division to accomplish its task.

The FBI should start from scratch in defining a subarchitecture for the intelligence process, rather than beginning with the (implicit) architecture of the VCF. The reason is that the vision for, and basic architecture of, the Trilogy system – including the VCF, predates the post-"9/11" restructuring of the FBI's mission and relationships to intelligence. At that time, the FBI's (and Trilogy's) focus was on support for the law enforcement mission. The change in the mission since "9/11" underscores the need for agility and separation of mechanism and policy in the information system architecture; many functions that would have been prohibited by policy before 9/11 are now supposed to be routine parts of the FBI's operation.

Given the nature of the FBI’s expanded mission, and the challenges they and the other intelligence and law enforcement agencies face, it is essential that the other intelligence agencies provide input and comment to the enterprise architecture effort, and that they and their information resources are appropriately linked to the FBI systems.

Once the intelligence and investigatory subarchitectures are in hand, a search for commonality of information used will reveal natural points of coupling and interface between the two of them.

Reduction in the complexity of the FBI’s IT systems should be a high priority, even at the expense of increased costs for hardware that may appear duplicative or redundant. The reason is that the successful management of a complex IT system usually requires a large degree of technical sophistication, and one the FBI lacks such capabilities at present.

31

146147148

1438

143914401441144214431444144514461447144814491450145114521453145414551456145714581459146014611462146314641465146614671468146914701471147214731474147514761477147814791480148114821483148414851486

149

DRAFT!!!! FOR COMMITTEE USE ONLY 11/25/0312/04/03 DO NOT DISTRIBUTE FURTHER!4.2 Regarding system development and deployment

To deal with the most important of the system issues described in Section 3.2, the committee makes the following recommendations.

No applications beyond VCF should be initiated or developed until a full enterprise architecture is in place. The reason is that VCF is near the date of its initial operating capability, and will provide such a major step forward from ACS that little would be gained from halting its development and much lost at this time. But in the committee’s view, this is true for no other application on which the committee knows the FBI is working.

The FBI should plan to rework the next version of VCF to include a workflow engine as a high priority. The current implementation of the Virtual Case File (VCF) system, as described by the FBI and its contractors, has hard coded the workflows describing how information is to be entered, reviewed, and used. In the committee’s view, this is a significant flaw. The committee anticipates that the FBI will need to revise its workflows from time to time. Such revisions will be motivated by the changes the FBI makes to the processes it uses to run its operations. These changes are clearly driven by the need to refine how counter terrorism and established functions are integrated, and the increased emphasis on creating intelligence analysis as a core skill. By incorporating a workflow engine, the FBI will make the VCF more agile in its support of evolving FBI policies and practices, and reduce the risk of delaying implementation of new policies and practices that are critical to successfully countering terrorism. In addition, it will be more economical to evolve the VCF application, since workflow change will involve revising data and not code. This will enable the FBI to keep VCF relevant to the operational needs of the field and of headquarters more easily for a given maintenance budget. Finally, this will extend the lifetime of the VCF application by allowing it to be flexibly adapted to emerging needs of the organization.

4.3 Regarding human resources

To deal with human resource shortages in key areas (e.g., program management, data architecture, data modeling and data warehousing), the short-term fix is to borrow experienced personnel from other agencies. The expectation would be that these individuals would serve for a couple of years, during which time the FBI could begin to train permanent replacements in long-term career tracks with the FBI.

However, top management positions are not amenable to such an approach. In particular, the FBI needs a CIO and Chief Enterprise Architect, and the committee concurs with the Director’s judgment that filling these positions with appropriately qualified individuals ought to have the highest priority.

Seek relief from externally imposed constraints on reprogramming and facility/space allocation, or at least streamline the approval process.

Borrow contract managers from other agencies with more experience than possessed by current FBI contract managers.

32

150151152

1487

14881489149014911492149314941495149614971498149915001501150215031504150515061507150815091510151115121513151415151516

1517

151815191520152115221523152415251526152715281529153015311532153315341535

153


Distribute progress information to all outside stakeholders simultaneously to pre-empt the need for their independent investigations.

Use outside expertise to supplement in-house expertise in key areas such as architecture.

4.4 Regarding security

The committee believes that the FBI should adopt a risk management approach to security, for only in doing so will it understand the operational penalties it pays for risk avoidance. Acceptance of this premise results in several immediate items of high priority.

The FBI should establish a clear policy governing SBU communications (and data more generally). Such a policy can be risk-based.

The FBI should provide routine access to the Internet and its attendant information-providing capabilities to all staff.

The FBI should immediately develop plans that address recovery of data and functionality in the event that essential technology services come under denial of service attacks (e.g., from viruses and pervasively replicated software bugs).

If the FBI is not comfortable with moving to a risk management approach to security, at the very least, it should review security practices in a risk versus reward framework with an entire end-to-end consideration of the information gathering and information management requirements of the FBI’s staff.

4.5 Program management

The most important step to take in program management is to abandon the notion that a large-scale system can be deployed widely in one fell swoop. Moreover, the testing phase of system development must not be given short shrift, and it is highly desirable in the long run to allow adequate time for testing even if dates of initial operational capability must be delayed. Testing must include a full systems integration test and an adequate volume and stress test.

For applications other than VCF, the FBI should undertake a full contract review. Given the FBI’s history of contract missteps for information technology procurements, and their unfamiliarity with concepts such as “code escrow,” such a review would focus on continuity and availability. If necessary, contracts should be renegotiated to include best practices such as code escrow and support service-level agreements to protect the FBI against business failures and successes that can adversely impact the availability and support for products that the FBI will depend upon. The FBI should consult with both other government agencies and the private sector to develop a set of best contracting practices before undertaking the contract review.

33

154155156

1536153715381539154015411542

1543

15441545154615471548154915501551155215531554155515561557155815591560156115621563

1564

156515661567156815691570157115721573157415751576157715781579

157


Figure 1— The Federal Enterprise Architecture Framework

34

158159160

1580

158115821583

161


Figure 2— FBI Value Chain Diagram

35

162163164

15851586

1587

1588

15891590

165


Figure 3— Showing the scope of key applications on FBI Value Chain

36

166167168

15911592159315941595

1597159815991600

1601

1602160316041605

169


<<< not reviewed by Gio >>>APPENDIX 1: The DOJ IG report

1. Our report should reference the DoJ IG (final) audit report and focus on being complementary, as, for instance, referring to Recommendation 2:

"Ensure that the FBI-wide enterprise architecture currently under development is accompanied by a process map for information sharing that clearly defines the current state and an end for the information-sharing process so that the numerous information sharing initiatives can be coordinated and properly monitored and managed. "According to the FBI response [Appendix 7, pages 84 and 85]

"The FBI has already completed a detailed blueprint and process map on its intelligence and information sharing process."Have we seen it? The DoJ has not [Appendix 8, page 93]

2. Are we all in full agreement with the 8 guiding principles [p.47-48]? If we are, it would be good to state so. If we have concerns, it will be important to convey them.

1. All FBI data is to be shared within the FBI, with very few exceptions. 2. The ownership of FBI information is corporate; no individual division or employee

“owns” FBI information. 3. The FBI will have a single, integrated information space, in which the default will be to

share with agencies with due consideration for the protection of sources and methods, and the security and prosecutive objectives of investigations.

4. The FBI will not filter information internally but instead will create an overarching FBI-wide policy that balances the need for cross-correlation with the risks of misuse.

5. The view of what the FBI collects and what the FBI creates will look very similar. 6. All data collected must also be recorded, searchable, retrievable, and easily cross-

correlated. 7. FBI employees will have the ability to conduct federated queries across multiple systems

to identify relationships. 8. Technology will be in service of people instead of people in service of technology. The

FBI must have interconnectivity with Intelligence Community systems. Additionally, the FBI must leverage existing technologies instead of rebuilding them.

3. It is not surprising that:"when we requested a flow chart for the processing of intelligence or other

information received by FBI headquarters, none was available,"since such input was outside the mission of the FBI, and its IT systems, both ACS and

VCF. Within it's existing formal IT capabilities there is no place to collect general purpose (non-case specific) external intelligence information. We did not hear anything about:

Pending IT improvements that may resolve the problem, at the time of our audit ITOS I was setting up a system so that when communications arrive at the

37

170171172

16061607160816091610161116121613

16141615161616171618161916201621162216231624162516261627

16281629163016311632163316341635163616371638163916401641164216431644164516461647

164816491650165116521653165416551656

173

DRAFT!!!! FOR COMMITTEE USE ONLY 11/25/0312/04/03 DO NOT DISTRIBUTE FURTHER!FBI [CLASSIFIED INFORMATION REDACTED] would ensure that the information was forwarded to the proper FBI section.

4. If "the National Threat Center Section (NTCS) has neither an investigative nor an analytical role, [pages 37 and 38] how do we on deal with that in the architecture in our report.

5. The co-location of the TTIC, the CIA's CTC, and the FBI's CTD is essential at this time, but should not be an excuse to avoid broader electronically mediated access, dissemination, and interaction with other agency staff and field personnel. [pages 45 and 46]

6. Did we encounter the "Gateway" project? [page 37 and appendix 4.] It focuses on investigative information related to terrorism in the greater St.Louis area. It was announced in Oct. 2002 and was to be operational 60 days later. Ha anything been learned here?

7. I assumed that the Trilogy network working meant that theTS/SCI LAN is fully operational,but the report mentions when the TS/SCI LAN is fully operational, and considers that system a distinct,

"external system that interfaces with the secure Joint Worldwide Intelligence Communications System (JWICS) and other classified systems used by the intelligence community." (p.40) . Maybe the difference is due to the gap of the date of writing and releasing report; but our

report should state the situation correctly.

8. Another consideration (re page 39 of the report):

If we address the problem of variant transliterations of the names of foreign nationals, then we might state that a preferred or formal spelling should be the spelling used in the individuals passport. That's the name most suited for the watchlist. As far as I know, all foreign countries do provide a latinized name of their nationals in their passports. That does not solve the multiple ID problem, but seems better than us deciding what the `best' transliteration would be.

38

174175176

16571658165916601661

16621663166416651666166716681669167016711672167316741675167616771678167916801681168216831684168516861687

1688168916901691169216931694169516961697

177


Appendix 2: DOD view of enterprise architecture

• When you think about the Integrated Battlespace the customer has multiple “architecture” views• The customer, rightly so, focuses on the operational view. What is the concept of operations.

What are the Measures of Effectiveness.• Within the Strategic Architecture organization we focus on the technical view.• As the video clip suggested not all systems need the same level of information.• Collaboratively we work with the customer to define the systems view. Specifically – how must

systems interoperate to achieve the Measures of Effectiveness• This is the real challenge and we have made a significant breakthrough that I will describe shortly.

39

178179180

169816991700170117021703170417051706170717081709171017111712

17131714

181

Documents

Report on FBI IT Modernizationinfolab.stanford.edu/pub/gio/2004/Report-RJBandGioCom… · Web viewDRAFT!! version 01-22-04