Mid Point PresentationShared Test Case Project

By Team Guardians

Agenda

Introduction The Problem What we have done so far Project status Risks Reflections Plan for the fall semester

The Guardians team

Dilip Narayanan

Gaurav Jalan

Nithya Janarthanan

Our mentors

The Mentors said….

Studio projects are like a box of chocolates, you never know what you’re gonna get….

Eduardo Miranda, Associate Teaching Professor, ISR

Vijay Sai, SEI

ClientJason Weighley, Rail Control Systems, Bombardier Transportation

External ConsultantsProf. Anthony TomasicFrederick Pfisterer

FAT Testing Division V&V Testing Division

FAT Test Document

V&V Test Case DocumentI see a lot of overlap. If only I could identify the common test cases !!!

The Problem

Are there any test cases that are similar to the one I am going to write now ?

Match Found

Match not Found

I don’t have to write a new test case.

I have to write a new test case.

This reduces costs.

FAT

V&V

Project Overview

Customer Goals: To help Bombardier testing division to identify at least 20% of the common test cases between their test groups

Academic Goals: To formalize the BT Test Case information in an efficiently searchable and retrievable manner

Project Plan

Approve Milestone plan with Eduardo and attach here, with milestones achieved in green, slipped in red and others in yellow.

Milestone plan

Milestone plan Contd...

M10 Complete examination and Approval of 274 test cases5th October, 2009

Why two tracks

CTCI track:For our domain learningTo reduce customer’s risk by ensuring critical common test cases are

delivered on schedule TCCS track:To provide a scalable solution that can be used for other test casesTo meet our learning objectives

CTCI Status

0

50

100

150

200

250

300

4 816

28 33 40

73

112 112 112 112 112 112 112 112 112 112 112

19

38

57

76

95

114

133

95

118 118 118

140

162

185

207

229

252

274

Cumulative Examined

Actual Planned

Week

# of

test

case

s

CTCI Status - 2

27; 49%27.8; 51%

% of success threshold achieved

Approved Common Need approval for success

112; 41%

162; 59%

Overall Status

Examined Remaining

Tester entersthe use case details

The following test cases are similar1. Test case 12. Test Case 2

Common Test Case Identification System

Hierarchical organization of domain data

Repository ofTest cases

Testing Domain

FAT

Wheel Brakes

V&V

Wheel

Test Case Comparison System - Updates

Technical Research – Identification of suitable approachesLiterature ReviewMentor MeetingsMeetings with Subject Matter Experts

– Prof. Anthony Tomasic– Frederick Pfisterer

Identified areas of Research

Areas of ResearchNatural Language Processing

– Text Processing SystemsVector space models (Lucene)Ontology(Protégé)Hybrid Approach

Identification of suitable approaches

Identify the Technology usage context and Evaluation goals

Develop Model Problem

Plan the Evaluation

Analyze model problems against technology usage context

Is technology a good fit ?

yesNo

Modify Technology or Context

Process for Approach Evaluation

Top 3 Risks

Serial No. Risk Mitigation Strategy

1Customer Unavailability Scheduled meeting with customer in one week's

advance. Negotiating the time period of the customer meeting.

2 Research and learning activity not resulting in a concrete results

Separate Learning Plan iscreated, exclusively for the learning activities.

3 Not Finding any approach to formalize the test cases

Meet with domain specialists from the Clients side to get more insight into the testing process and also scheduled meetings with technology experts.

Reflections - What went wrong

× Busy Customer× Lack of Clarity of project Scope× No Roles and Responsibilities× Lack of a proper Process for each activity× Ineffective Meetings× Activity based planning

Reflections (What went well)

Identified alternate points of contacts with the customer Fixing up the next customer meeting at the end of the current meeting Process for each of the activities

Meeting Common test case comparison Learning and Research activities

Meeting process Buffer time = 15% of total duration Following the time boxing of the agenda items Ideas list (in addition to the action items) Scribe uses a shared Google doc and projects it

Lessons Learnt

Deadlines for the tasksSending across a deliverable on its deadline irrespective of the status of completionWell- defined roles and responsibilities is importantImportance of having a Macro Plan

Plan for the Fall Semester

Completion of Common test case IdentificationVerification and validation of the selected approach to provide the solution

for identifying common test casesTechnical Evaluation Report

Questions for the mentors, etc.

Do we have anything for mentors?

Questions for us?

Backup slides

How we are tracking

G thinks that this slide can be removed. Or if not, at least put it up in bkup slides.

- for ctci i think v dont need to say how v track cuz its too trivial - v can say something on how we are tracking tccs

Processes (we shud delete this)

D and G agreed to remove this slide. Need N’s opinion. brief and generic overview of some of our processes – I don’t think

this is reqd as we don’t really have any other processes except for CTCI and for Meetings

CTCI process we shud put in the bkup slides.

CTCI Process

I’m not sure if this is reqd even for bkup Explain manual comparision diagrammatically, trying to de-

emphasize the manual aspect of this process

Backup slides

SLRC - System Level Requirements CatalogPast projects were Word Documents

Newer projects are moving to Doors/Slate

Design/Development Groups

a) Perform manual tests unique to Development group

Safety Group

a) Write automatic scripts unique to Safety Group

System/Factory Test Groups

a) Write test plan unique to Test Groupb) Test scripts describe manual tests

Design/Development Groups

a) Perform manual regression tests unique to Development group

Safety Group

a) Run scripts unique to Safety Groupb) Perform manual tests

System/Factory Test Groups

a) Perform manual tests unique to Test Group

Dev

elop

men

t Ph

ase

Test

Pha

se

Duplication of effort - Three different test methods are independently developedNo clear method to determine if software is ready for release to V&V and Factory TestsInconsistency across groupsManual testing yields slower turn-around timesLow confidence that all requirements were implemented and tested

Test Scripts / Plans are not continuously updated as the software is debugged.

Manual and incomplete mapping of requirements from SLRC to test scripts

Project Overview – Current Scenario

33

Slide Title to go here

Subheading (first level text)Bullet 1 (second level text)

– Bullet 2• Bullet 3

– Bullet 4

34

The Colours

35

Picture slide

Subheading (first level text)Bullet 1 (second level text)

– Bullet 2• Bullet 3

– Bullet 4

36

Example of suggested chart formatting

Delete categories / series as necessary

3.0 3.0 3.0 3.0 3.0 3.03.0 3.0 3.0 3.0 3.0 3.0

3.0 3.0 3.0 3.0 3.0 3.03.0 3.0 3.0 3.0 3.0 3.03.0 3.0 3.0 3.0 3.0 3.03.0 3.0 3.0 3.0 3.0 3.03.0 3.0 3.0 3.0 3.0 3.0

3.0 3.0 3.0 3.0 3.0 3.0

0

5

10

15

20

25

30

One Two Three Four Five Six

Axi

s tit

le h

ere

Series 8

Series 7Series 6

Series 5Series 4

Series 3Series 2

Series 1

37

Examples of suggested formatting styles

Column heading

Column heading

Column heading

Row heading Data Data Data

Row heading Data Data Data

Row heading Data Data Data

Text box

Text boxWith shadow

Text boxWith shadow