Auditing Information Systems (AIS) Lecture – 7 ‘Systems and Infrastructure Life Cycle Management

Auditing Information Systems (AIS)

Lecture – 7

‘Systems and Infrastructure Life Cycle Management'

Project/Program Management

• A program is a group of projects and time-bound

tasks that are closely linked together through

common objectives, a common budget,

intertwined schedules and strategies

• Projects have a limited time frame (start and end

date) and organizational boundaries

The objectives of project management are:• Optimization of the results of the project

portfolio• Prioritizing and scheduling projects• Resource coordination (internal and external)• Knowledge transfer throughout the projects

A project needs clearly defined results that are specific, measurable, achievable, relevant and time-bound (SMART)

Project/Program Management (contd.)

Three major forms of organizational alignment for project management are:

Influence project organization

Pure project organization

Matrix project organization

Project Organizational Forms

Depending on the size and complexity of the project and the affected parties, communication when initiating the project management process may be achieved by:

• One-on-one meetings

• Kick-off meetings

• Project start workshops

• A combination of the three

Project Communication and Culture

The project manager needs to determine:

• The various tasks that need to be performed to produce

the expected business application system

• The sequence or the order in which these tasks need to

be performed / Priority of each task

• The duration or the time window for each task

• The IT resources available and required to perform

these tasks

• Budget / costing for each of these tasks

• Source and means of funding

Project Planning

Project Planning (continued)

What are the methods for the scoping of Business Application Development projects?

‘Software Size Estimation’Methods of determining the size of the application

software to be developed

Can be used as a guide for: allocation of resources, estimates of time and cost required comparison of the total effort required by the

resources available

Lines of Source Code The traditional and simplest method is by

counting the number of lines of source.

‘Software Size Estimation’


Function Point Analysis (FPA)

Widely used for estimating complexity in developing large business applications

The results of FPA are a measure of the size of an information system based on the number and complexity of the inputs, outputs, files, interfaces and queries with which a user sees and interacts

‘Software Size Estimation’ (contd.)


Gantt charts

• Constructed to aid in scheduling the activities (tasks)

needed to complete a project

• Charts show when an activity should begin and end

• Charts show which activities can be in progress

concurrently and which activities must be completed

sequentially


Includes management of:

▫ Scope

▫ Resource usage

▫ Risk Inventory Assess Mitigate Discover Review & evaluate

Project Controlling

• When closing a project, there may still be some issues

that need to be resolved, ownership of which needs to

be assigned

• The project sponsor should be satisfied that the system

produced is acceptable and ready for delivery

• Custody of contracts may need to be assigned, and

documentation archived or passed on to those who will

need it

Closing a Project

Business Application Development

The development / implementation process for business applications, commonly referred to as an SDLC, begins when an individual application is initiated as a result of one or more of the following situations:

• A new opportunity that relates to a new or existing

business process

• A problem that relates to an existing business

process

• A new opportunity that will enable the organization

to take advantage of technology

• A problem with the current technology

Business Application Development

• A business case provides the information required for an organization to decide whether a project should proceed

• The initial business case normally derives from a feasibility study as part of project planning

• The business case should be of sufficient detail to describe the justification for setting up and continuing a project

Business Case Development and Approval

Feasibility study• Concerned with analyzing the benefits and solutions

for the identified problem area

• Includes development of a business case, which determines the strategic benefits of implementing the system either in productivity gains or in future cost avoidance and to decide whether a project should proceed.

• Identifies and quantifies the cost savings of the new system and estimates a payback schedule for the cost incurred in implementing the system or shows the projected return on investment (ROI)

Note: The business case should be of sufficient detail to describe the justification for setting up and continuing a project

Description of Traditional SDLC Phases

Requirements definition

• Concerned with identifying and specifying the

business requirements of the system chosen for

development during the feasibility study

• Requirements include descriptions of what a system

should do, how users will interact with a system,

conditions under which the system will operate, and

the information criteria the system should meet

Description of Traditional SDLC Phases (Continued)

Entity relationship diagrams

• An ERD depicts a system’s data and how these data

interrelate

• An ERD can be used as a requirements analysis tool to

obtain an understanding of the data a system needs to

capture and manage

• An ERD can also be used later in the development

cycle as a design tool that helps document the actual

database schema that will be implemented


Software Acquisition

• Not a phase in the standard SDLC. However, if a decision was reached to acquire rather than develop software, software acquisition is the process that should occur after the requirements definition phase

• Decision based on various factors such as cost differential between development and acquisition, availability of generic software, and the time gap between development and acquisition


Software acquisition

• Request for Proposals (RFP) is send to the potential vendors.

• The project team needs to carefully examine and compare the vendors’ responses to the request for proposal (RFP)

• It is likely that more than one product/vendor fits the requirements with advantages and disadvantages with respect to each other

• Once vendors are short listed, it can be beneficial for the project team to talk to current users of each potential product


Ernst & Young

Show sample RFP

Design

Key design phase activities include:

Developing system flowcharts and entity relationship models

Determining the use of structured design techniques Describing inputs and outputs Determining processing steps and computation rules Determining data file or database system file design Preparing program specifications Developing data conversion plans


DevelopmentKey activities performed in a test/development

environment include: Coding and developing program and system-level

documents• Programming methods and techniques• Programming languages

Debugging and testing the programs developed Developing programs to convert data from the old

system for use on the new system Creating user procedures to handle transition to the

new system Ensure modifications are documented and applied

accurately


Testing Elements of a software testing process

• Test plan• Conduct and report test results• Address outstanding issues

• Testing classifications• Unit testing• Interface or integration testing• System testing• Final acceptance testing


Testing

Types of testing– Alpha and beta testing– Pilot testing– White box testing– Black box testing– Function/validation testing– Regression testing– Parallel testing

Automated application testing E.g. TestComplete 8, QACenter Performance Edition 5.0;

etc


Practice Question

3-1 To assist in testing a core banking system being acquired, an organization has provided the vendor with sensitive data from its existing production system. An IS auditor’s PRIMARY concern is that the data should be:

A. sanitized.B. complete.C. representative.D. current.

Implementation

• During the implementation phase, the actual operation of the new information system is established and tested

• Final user-acceptance testing is conducted in this environment (UAT)

• The system may also go through a certification and accreditation process to assess the effectiveness of the business application at mitigating risks to an appropriate level


Ernst & Young

Show / Add UAT document template / Filled document

Practice Question

3-2 An IS auditor is performing a project review to identify whether a new application has met business objectives. Which of the following test reports offers the MOST assurance that business objectives are met?

A. User acceptanceB. PerformanceC. SociabilityD. Penetration

Ernst & Young

Check Answer

Implementation planning– Phase 1—Develop to-be support structures– Phase 2—Establish support function

End-user training Data migration

– Refining migration scenario– Fallback (rollback) scenario

Changeover (go-live or cutover) techniques– Parallel changeover– Phased changeover– Abrupt changeover

Certification/accreditation


Post Implementation Review A post-implementation review should meet the

following objectives:• Assess the development project process (SDLC

Phases)• Assess the adequacy of the system functionality

(according to the initial requirements)• Evaluate the projected cost benefits or ROI• Develop recommendations that address the

system’s inadequacies and deficiencies• Develop a plan for implementing the

recommendations A post-implementation review is performed ideally an

independent third party jointly by the project development team and appropriate end users.


Ernst & Young

Show Post Implementation Review Report

An IS auditor’s tasks in system development, acquisition and maintenance include:

• Determine if the main components, objectives and user requirements of the system are properly identified

• Determine and rank the major risks to, and exposures of, the system

• Identify controls to mitigate the risks to, and exposures of the system

• Monitor the system development process• Participate in Post-Implementation reviews• Test system maintenance procedures• Evaluate the system maintenance process

Auditing Systems Development and Acquisition

Business risk relating to the likelihood that the new system may not meet the users’ business needs, requirements and expectations.

Risk Associated with Software Development Process

IS Auditor / IS Advisor Role in

‘Systems Development Life Cycle’

An IS auditor should perform the following functions:

• Review the documentation produced in this phase for reasonableness

• Determine whether all cost justifications/benefits are verifiable

• Identify and determine the criticality of the need

• Determine if a solution can be achieved with systems already in place

• Determine the reasonableness of the chosen solution

Feasibility Study


• Obtain the detailed requirements definition document

• Verify that project initiation and cost have received proper management approval

• Review the conceptual design specifications to ensure they address the needs of the user and the control specifications have been defined

• Determine whether a reasonable number of vendors received a proposal covering the project scope and user requirements (In case of Acquisition)

Requirements Definition


• Analyze the documentation from the feasibility study

• Review the RFP

• Determine whether the selected vendor is supported by RFP documentation

• Review the vendor contract prior to its signing

• Ensure the contract is reviewed by legal counsel before it is signed

Software Acquisition Process

Practice Question

3-7 An organization decides to purchase a package instead of developing it. In such a case, the design and development phases of a traditional software development life cycle (SDLC) would be replaced with:

A. selection and configuration phases.

B. feasibility and requirements phases.

C. implementation and testing phases.

D. nothing; replacement is not required.


• Review the system flowcharts for adherence to the general design

• Review the input, processing and out controls designed into the system for appropriateness

• Interview the key users of the system• Assess the adequacy of audit trails• Verify the integrity of key calculations and processes• Verify that the system can identify and process

erroneous data correctly• Review the quality assurance results of the programs

developed• Verify that all recommended corrections to

programming errors were made

Detail Design and Development

Practice Question

3-8 User specifications for a project using the traditional SDLC methodology have not

been met. An IS auditor looking for a cause should look in which of the following areas?

A. Quality assuranceB. RequirementsC. DevelopmentD. User training


• Review the test plan for completeness• Review error reports for their precision in

recognizing erroneous data and for resolution of errors

• Interview end users of the system for their understanding of new methods, procedures and operating instructions

• Review unit and system test plans to determine whether tests for internal controls are planned and performed

• Review parallel testing results for accuracy

Testing

An IS auditor should verify that appropriate sign-offs have been obtained prior to implementation, and perform the following:

• Review all system documentation to ensure its completeness and that all recent updates from the testing phase have been incorporated

• Verify all data conversion to ensure that they are correct and complete before implementing the system in production

Implementation Phase


• Determine if the system’s objectives and requirements were achieved

• Determine if the cost benefits identified in the feasibility study are being measured, analyzed and accurately reported to management

• Review program change requests performed to assess the type of changes required of the system

• Review controls built into the system• Review operators’ error logs• Review input and output control balances and reports

Post Implementation Review

System Software Change Control

“Change control procedures are designed to ensure that

changes are authorized and do not disrupt processing”

Standard Changes

• Formal initiation of change from end-user /

operational staff (Documentation of Change)

• Approvals for development of Change by concerned

business process owner

• Testing changed programs by QA / User Acceptance

Testing (UAT)

• Authorization for deployment of Change into

Production

Emergency changes

Change Management Process - Overview

An IS auditor should consider the following:• The use and existence of a policy for authorizing,

prioritizing and tracking system change requests• Whether emergency change procedures are

addressed in the IT Security policy manuals• Whether the change control log ensures all changes

shown were resolved• The user’s satisfaction with the turnaround of change

requests (UAT)• The adequacy of the security access restrictions over

production source and executable modules

System Change Procedures and the Program Migration Process

For a selection of changes on the change control log:• Determine whether changes to requirements

resulted in appropriate change-development documents

• Determine whether changes were made as documented

• Determine whether current documentation reflects the changed environment

• Evaluate the adequacy of the procedures in place for testing system changes

• Review evidence to ensure that procedures are carried out as prescribed by organizational standards

• Review the procedures established for ensuring executable and source code integrity

System Change Procedures and the Program Migration Process

(contd.)

Application Controls

Application controls are controls over input,

processing and output functions. They include

methods for ensuring that:

• Only complete, accurate and valid data are

entered and updated in a computer system

• Processing accomplishes the correct task

• Processing results meet expectations

• Data are maintained

Application Controls

Input authorization

• Input authorization verifies that all transactions have

been authorized and approved by management

• Types of authorization include:

– Signatures on physical forms or source documents

– Online access controls / System based Approval

– Example ??????

– Unique passwords

– Terminal or client workstation identification

Input / Origination Controls

Processing Controls

Data validation and editing procedures

• Data processing / validation identifies data errors, incomplete

or missing data and inconsistencies among related data items

• Procedures should be established to ensure that input

data are validated and edited as close to the time and

point of origination as possible

Ensure the completeness and accuracy of

accumulated data

The following can be examples of it:

• Run-to-run totals

• Reconciliation of file totals

• Automated verification of course pre-

requisite. (ZABDESK)

Processing Procedures and Controls – (Contd.)

Output Controls

Output controls provide assurance that the data delivered to users will be presented, formatted and delivered in a consistent and secure manner

Output controls include:

• Logging and storage of sensitive and critical data in a secure place

• Report distribution

• Verification of receipt of reports

• Output report retention

• Output error handling

Electronic Commerce

• With increasing emphasis on integrating the web channel with a business’ internal legacy systems and the systems of its business partners, company systems now typically will run on different platforms, running different software and with different databases

• The challenge of integrating diverse technologies within and beyond the business has increasingly lead companies to move to component-based systems that utilize a middleware infrastructure, based around an application server

Electronic Commerce (continued)

E-commerce models include the following:

• Business-to-consumer (B-to-C) relationships

• Business-to-business (B-to-B) relationships

• Business-to-employee (B-to-E) relationships

• Business-to-government (B-to-G) relationships

• Consumer-to-government (C-to-G) relationships

• Exchange-to-exchange (X-to-X)

relationships

• E-commerce risks:

Confidentiality

Integrity

Availability

Authentication and non-repudiation

Power shift to customers

• It is important to take into consideration the importance of security issues that extend beyond confidentiality objectives


An IS auditor should assess applicable use of:• Firewall mechanisms that are in place to mediate

between the public network and an organization’s private network

• A process whereby participants in an e-commerce transaction can be identified uniquely and positively ‘Digital signatures’

• An infrastructure to manage and control public key pairs and their corresponding certificates

• Logs of e-commerce applications• The methods and procedures to recognize security

breaches when they occur• The protections in place to ensure that data

collected about individuals are not disclosed without their consent


An IS auditor should assess applicable use of:

• The means to ensure confidentiality of data communicated between customers and vendors

• The mechanisms to protect e-commerce’s presence and their supporting private networks from computer viruses

• A plan and procedure to continue e-commerce activities in the event of an extended outage of required resources for normal processing

• A shared responsibility within an organization for e-commerce security?????


Practice Question

3-9 When introducing thin client architecture, which of the following risks regarding servers is significantly increased?

A. IntegrityB. ConfidentialityC. Availability

Electronic Data Interchange

• What is EDI?

• The benefits associated with the adoption of EDI include:• Less paperwork

• Improved information flow, database-to-database and company-to-company

• Fewer delays in communication

• Examples????

Controls in EDI Environment

To protect EDI transmissions, the EDI process should include:

• Standards set to indicate the message format and content are valid (Communication Protocol)

• Procedures to determine messages are only from authorized parties

Direct or dedicated transmission channels among the parties

Data should be encrypted using algorithms agreed to by the parties involved

• The EDI process needs the ability to detect and deal with transactions that do not conform to the standard format or are from/to unauthorized parties

• The critical nature of many EDI transactions requires that there be positive assurances that the transmissions were complete

The key controls for receipt of inbound transactions are:

• Use appropriate encryption techniques when using public Internet infrastructures for communication

• Log each inbound transaction upon receipt

• Ensure the exchange of control totals of transactions sent and received between trading partners at predefined intervals

Controls in EDI Environment

Electronic Mail

Security issues involved with e-mail include:• Denial-of-service (DoS) attacks may be directed to

the mail server

• Sensitive information transmitted unencrypted between mail server and e-mail client may be intercepted

Information within the e-mail may be altered

Viruses and other types of malicious code may be distributed throughout an organization

• Users may send inappropriate, proprietary or other sensitive information via e-mail

Digital signatures are a good method of securing e-mail transmissions in that:

• The signature cannot be forged

• The signature is authentic and encrypted

• The signed document cannot be altered

Electronic Mail – (Contd.)

E-Banking / Electronic Funds Transfer

• Electronic funds transfer (EFT) is the exchange of money via telecommunications without currency actually changing hands

• Usually function via an internal bank transfer from one party’s account to another or via a clearinghouse network

Security in an EFT environment ensures that:• All of the equipment and communication linkages

are tested for security issues• Upon receipt of data, the receiving party will

immediately transmit an acknowledgment or notification

• Data encryption standards are set

Conclusion

•Chapter 3 Quick Reference Review▫Page 137 of CISA Review Manual 2010

Documents

Auditing Information Systems (AIS) Lecture – 7 ‘Systems and Infrastructure Life Cycle Management