Ensuring a Quality Order Management Implementation

  • Published on
    19-Oct-2014

  • View
    202

  • Download
    2

Embed Size (px)

DESCRIPTION

Software testing is a process, or a series of processes, designed to make sure computer code does what it was designed to do and that it does not do anything unintended. - The program should be predictable and consistent, offering no surprises to users. - Test should start with the assumption that the program contains defects and the goal is to find as many defects as possible. Introduction to Quality and Testing - Why is Quality important to Sterling Order Management System - QA in OMS - Journey to QA Center of Excellence - Conclusion

Transcript

<p>PowerPoint Presentation</p> <p>Ensuring a Quality Order Management </p> <p>Implementation</p> <p>Subroto Majumdar, OMS Director</p> <p>subroto.majumdar@perficient.com</p> <p>Agenda</p> <p>2</p> <p> Introduction to Quality</p> <p>and Testing</p> <p> Why is Quality important </p> <p>to Sterling OMS</p> <p> QA in OMS</p> <p> Journey to QA Center of </p> <p>Excellence</p> <p> Conclusion</p> <p> Software testing is a process, or a series of processes, designed to make sure computer code does what it was designed to do and that it does not do anything unintended. </p> <p> The program should be predictable and consistent, offering no surprises to users. </p> <p> Test should start with the assumption that the program contains defects and the goal is to find as many defects as possible.</p> <p>Introduction</p> <p>3</p> <p>Why is quality important for Sterling OMS?</p> <p> OMS A Major Enterprise Initiative QA - Airplanes vs. Order Management Implementation</p> <p> Hopefully, no one gets hurt</p> <p> Impacts project timelines</p> <p> Companies lose money and delay their ROI</p> <p> It can mean the difference between business accepting or rejecting the solution</p> <p> It hurts the product and vendor reputation Bill Lear an engineer and pioneer in executive jets Grounded All </p> <p>customer planes till he could triage and resolve a mysterious issue with </p> <p>his jets.</p> <p> Quality cost money but is the difference between success and failure</p> <p> OMS Headless Application - Quality that cannot always be seen but felt</p> <p>4</p> <p>Importance of Testing and Validation</p> <p> Testing should be introduced in the early stage of the Software Development Life </p> <p>Cycle (SDLC) </p> <p> Cost of fixing the defect is higher if testing is not done in early stage &amp; </p> <p>defects found in later critical stages</p> <p> Retesting and Regression Testing is also important </p> <p> Ensure that no previously working functions have failed as a result of the </p> <p>fix </p> <p> New features have not created problems with previous software </p> <p>versions 5</p> <p>Typical Testing phases in OMS testing projects</p> <p>Testing Phases</p> <p>6</p> <p>UAT/Performance</p> <p>E2E</p> <p>FT</p> <p>CIT</p> <p>UT</p> <p> UT Unit Testing CIT Component </p> <p>Integration Testing</p> <p> FT Functional Testing</p> <p> E2E End To End or System Testing</p> <p> UAT User Acceptance Testing</p> <p> Performance Testing</p> <p>OMS - What can be Tested?</p> <p> Order flow across different status Holds resolution Credit card authorization Fraud check validation Sourcing and scheduling Release Ship confirmation, ship cancellation, </p> <p>status tracking</p> <p> Settlement and invoicing Data purge Order load (data conversion) Inventory load, ATP and sync Item load and sync Customer data management Alerts, emails and exceptions</p> <p>7</p> <p>OMS - What can be Tested?</p> <p>8</p> <p> UI and Database validations have their own importance</p> <p> UI validations recommended when the tester needs to validate the complete life cycle of a single order Eg: Validating the ship confirmation from the fulfillment center. For </p> <p>this scenario, the tester can validate the order creation in OMS, order processing till release and once the shipment confirmation arrives, the order status is updated to Shipped.</p> <p> Database validations beneficial when the tester has to validate from a range of data Eg: To validate if there is any order where tax is $0 or shipping </p> <p>discount of more than $5 can be done by creating a simple query to fetch the above results. Other DB validations include data purge, to check when the order reached a specific status, to confirm if the order was picked up by the agent or not, verify the bulk inventory and item feed coming from different systems to OMS.</p> <p>Performance Testing Scope - OMS</p> <p> Performance testing is very important since OMS always </p> <p>deals with a huge volume of </p> <p>order, item and inventory </p> <p>related data. </p> <p> At times, it also processes bulk orders in a single batch </p> <p>and hence performance </p> <p>testing plays a key role in any </p> <p>OMS implementation.</p> <p>9</p> <p>Automation Testing Scope - OMS</p> <p>Some of the many scenarios which can be </p> <p>automated: </p> <p> Order flow across various status Shipment charges Tax calculation Discounts and promotional charges Payment tenders and tokens Item validation Inventory reservation pre/post order creation Shipment and settlement process</p> <p>10</p> <p>Sample Waterfall OMS Testing Methodology</p> <p>CALENDAR</p> <p>Requirement Analysis</p> <p>Strategy </p> <p>Formulation</p> <p>Test Planning</p> <p>Test Case</p> <p>Creation</p> <p>Test </p> <p>Execution</p> <p>Scripting</p> <p>Post Deployment</p> <p>Evaluation</p> <p>Release</p> <p>Test Summary</p> <p>Report</p> <p>Benefits</p> <p>Onsite Onsite - Offshore</p> <p>1</p> <p>2</p> <p>3</p> <p>4</p> <p>5</p> <p>6</p> <p>7</p> <p>8</p> <p>9</p> <p>11</p> <p>Test Strategy/ </p> <p>Planning</p> <p>Test Case </p> <p>Development</p> <p> Vendor assumes full responsibility for all software testing activities for aspecific project, release, or enhancement, Manages delivery with guidanceand oversight from client testing/SQA leadership.</p> <p> Your team creates the test plan and manages overall testing delivery.</p> <p> Vendor executes full set of testing activities defined within the client test planunder clients direct management.</p> <p> Client manages overall testing delivery and creates all testing artifacts.</p> <p> Vendor is responsible only for executing test cases provided by the client,logging defects, and providing necessary reporting.</p> <p>Environment </p> <p>Preparation</p> <p>Test </p> <p>ExecutionAnalysis</p> <p>Mgmt </p> <p>Reporting </p> <p>Sign </p> <p>Off</p> <p>De</p> <p>liv</p> <p>ery</p> <p> Mo</p> <p>de</p> <p>l</p> <p>Full Ownership</p> <p>Combined Ownership</p> <p>Testing Execution Only</p> <p>Typical Delivery Models</p> <p>12</p> <p>Typical QA Team Responsibility</p> <p>13</p> <p> Testing Strategy</p> <p> Test Planning</p> <p> Test Case Creation</p> <p> Test Execution</p> <p> Defects Verification</p> <p> Reporting</p> <p> Automation</p> <p> Positive Sign Off</p> <p> Successful Go Live</p> <p>Sample QA Artifacts</p> <p>Detailed Scorecards - Integrated into Dashboard</p> <p>Executive Dashboard</p> <p>Partners Stakeholders</p> <p>ID RID Opened Description and Impact Mitigation Activities Severity Probability Due Date Owner Status Comments / Resolution</p> <p>001 D 2/10/04</p> <p>The project described in this </p> <p>Statement of Work is based on a </p> <p>projected start date of 2/9/2004. </p> <p>Project will start on 2/9 based on </p> <p>good faith.High Unlikely 2/9/04 F. Schutrumpf Closed</p> <p>CLOSED (2/9/04): Project started on schedule </p> <p>on good faith based on verbal ok from Bruce </p> <p>relayed from Warren. Issue 009 is tracking non-</p> <p>signed SOW.</p> <p>001 D 4/12/04</p> <p>Delivery of Pan and transition of code </p> <p>base to the Billing Implementation </p> <p>Team on 4/23/2004.</p> <p>Pan slips can jeopardize planned </p> <p>delivery of Billing functionality </p> <p>components that have Pan </p> <p>dependencies. </p> <p>In addition, given the finite number of </p> <p>iterations prior to year end, </p> <p>implementation of Pan functionality </p> <p>could limit scope of Billing, Payments, </p> <p>Fraud or Streamline functionality </p> <p>implemented by Year end</p> <p>First iteration designed to </p> <p>transfer, organize and cleanup </p> <p>existing Siebel code base.</p> <p>Schedule work for subsequent </p> <p>iterations that must have </p> <p>Requirements fully defined up </p> <p>front prior to iteration start to </p> <p>minimize developer churn</p> <p>Blending of Pan and Billing </p> <p>functionality on second and third </p> <p>iterations to continue working Pan </p> <p>functionality, but also include new </p> <p>billing functionality in each </p> <p>release.</p> <p>Scope impact evaluation at the </p> <p>end of the second iteration to </p> <p>assess the open issues </p> <p>surrounding Pan. </p> <p>Medium Certain On-going K. Sheen Open</p> <p>UPDATE 4/12/04: On-going dependency until </p> <p>all Pan dependencies for planned Billing, </p> <p>Payments and Fraud functionality have been </p> <p>resolved.</p> <p>UPDATE 4/26/04: Current first iteration </p> <p>contains multiple PAN user stories still. The </p> <p>first iteration will contain mostly Pan </p> <p>functionality (rather than billing functionality). </p> <p>This may require a change to scope (Payments </p> <p>or Streamline Integration) to maintain schedule. </p> <p>This will be evaluated at the completion of the </p> <p>first iteration.</p> <p>UPDATE (5/3/04): Second iteration will most </p> <p>likely still include mostly Pan functionality. In </p> <p>addition, specific BAs will need to be assigned </p> <p>to continue the Pan requirements wrap-up. This </p> <p>is expected to take 2 full time BAs through the </p> <p>end of the second iteration (assumes that Pan </p> <p>functionality can be wrapped up in the third </p> <p>iteration). This will definitely affect the amount </p> <p>of billing work that can be completed by </p> <p>10/22/04. A complete impact evaluation will be </p> <p>completed at the end of the second iteration </p> <p>(see also issue #010)</p> <p>002 I 4/12/04</p> <p>The project described in this </p> <p>Statement of Work is based on a </p> <p>Projected Start Date of 4/12/04. </p> <p>Team members can ramp up </p> <p>more slowly to effect transition. Medium Unlikely 2/10/04 K. Sheen Closed</p> <p>CLOSED (4/12/04): Project started on </p> <p>schedule.</p> <p>Page 1Technology Triage Center Severity 1 Defect Summary </p> <p>Sunday June 4th, 2006 (Data Pulled a 7:45pm)Overall Status</p> <p>Defect ID Status Partner Project Summary Due Date Action Item Detected on </p> <p>Date</p> <p>2299 New Visage Web Portal Production: Incorrect shipping cost shown in Siebel for Web Portal Order Per Darren, it appears that this is just a display item (totals &amp; amt charged is correct) 5/27/2006</p> <p>2382 New Visage CRM PROD - Adding the Shared Feature 'Guest Services Offer - Courtesy 100,200,50' throwing a </p> <p>System Failure Error in Siebel</p> <p>6/1/2006</p> <p>2435 New Visage Activation Activating a second suscriber under a demo account fails in Siebel 6/4/2006</p> <p>2437 New Patni Web Portal Mecury fails to send emails created within test director 6/4/2006</p> <p>2436 New Patni Web Portal Production: NSP: Entered LTDCASE1 into shopping cart - now when the page refreshes the </p> <p>sku is not there at all</p> <p>6/4/2006</p> <p>2151 Open Visage ROE ROE PROD: After ROE ESN activation, Welcome Email from Siebel is not sent to the ROE </p> <p>customer</p> <p>Rick confirmed all channels require this &amp; per Scott, Visage is working this. Needs to </p> <p>provide due date.</p> <p>5/22/2006</p> <p>2404 Open Convergys Web Portal Order Handset Process - Calculate Sales Tax error in Production with concurrent users New duedat 6/6 - need to confirm before placing it in the ticket - 6/4 6/1/2006</p> <p>2354 Reopen Visage Web Portal Production: Shared Features added or removed on Web Portal do not reflect despite giving an </p> <p>Order Confirmation</p> <p>Sent back to webportal dev team as it failed retest - 6/4 5/31/2006</p> <p>2355 Reopen Visage Web Portal Production: Non-Shared Features added or removed on Web Portal do not reflect despite giving </p> <p>an Order Confirmation</p> <p>Sent back to webportal dev team as it failed retest - 6/4 5/31/2006</p> <p>2416 Retest Telcordia Service </p> <p>Control</p> <p>Call setup latency - mobile-terminated calls are taking more than 10 seconds to ring through 11:30 fix tonight - retest tomorow - 6/3 5/31/2006</p> <p>2369 Retest Disney </p> <p>Mobile</p> <p>CRM Production: Activation failure for web sales order in Data Platform Sharique to do sales order and see if it passes - 6/4 6/1/2006</p> <p>1147 Reopen Patni Web Portal EID Verification needs to be integrated in the Checkout Process 5/31/2006 Equifax to push their side into prod tomorrow (6/1) at 6am - we can follow our prod </p> <p>push after this executes. Update tomorrow afternoon. 5/31</p> <p>4/27/2006</p> <p>2047 Retest mPortal Family Alerts Production: Config file incorrect for non-MDN accounts, preventing Family Alert! from working 6/3/2006 Into prod last night - ready for retest. - Ian to retest - 6/4 5/18/2006</p> <p>2427 New Patni Web Portal Production: Site Wide: Hitbox tracking not working properly - talk to Todd Sheive at Hitbox 6/4/2006 Going in tomorrow morning - 6/4 6/3/2006</p> <p>2429 New Patni Web Portal Production: SMS Messaging: need to set the originating address to 1002 in the XML in order to </p> <p>support the billing reference data agreed with Convergys</p> <p>6/4/2006 Working on, pushed by tuesday night - 6/4 6/3/2006</p> <p>2207 Open Patni ROE ROE PROD: Family Account with SA Portability failed at AO &amp; FM level ...Record unusually </p> <p>stored in Siebel</p> <p>6/4/2006 All ROE CR's will go in at end of next week (6/9) - 6/3 5/24/2006</p> <p>2274 Open Patni ROE ROE UAT: The Auto Bill Pay Page does not allow to proceed to Order Confirmation 6/4/2006 5/26/2006</p> <p>1793 Reopen Patni Web Portal BOGO Promotion SKUs are not uploaded in Visage Product Catalog 6/4/2006 Need to understand what actions Patni needs to take to resolve defect - Bobby, </p> <p>Dave, and Ashish to discuss real time - 6/3</p> <p>5/10/2006</p> <p>1925 Hold InPhonic Web Portal Production: Entertainment - Ringtones Preview hyperlink not working 6/6/2006 New due-date is Tuesday COB - 6/4 5/16/2006</p> <p>2380 Open Autodesk LBS Production: Family Locator not functioning on port-in-swapped handset 6/6/2006 Pushing for tomorrow evening - 6/4 6/1/2006</p> <p>Risk, Issue, Dependency Log</p> <p>Defect Tracking Log</p> <p>Connectivity</p> <p>Network Overall Status</p> <p>C-WAP</p> <p>Test Prod</p> <p>Billing Provider/CRM Provider: </p> <p>AVS Product</p> <p>Network and Operations</p> <p>Finance</p> <p>Legal</p> <p>Test Cases </p> <p>Defined</p> <p>Test Case </p> <p>Sign-OffStarted</p> <p>IT / Systems / Data Center</p> <p>Overall Status</p> <p>Page 1Integrated Application Testing Dashboard</p> <p>Tuesday, November 6th, 2007</p> <p>Overall Integration Test Status</p> <p>Milestones:</p> <p>Milestone one COMPLETEMilestone two COMPLETEMilestone three 10/17/07Milestone four 10/22/07Milestone five 10/24/07</p> <p> High level status messages go here</p> <p> Additional thoughts go here.</p> <p>Days until Launch*</p> <p>4 4Soft Launch 1/4/2008</p> <p>Key Issues:</p> <p>Issue one - ...Issue two - .Issue three - ..</p> <p>Green</p> <p>ATG Link</p> <p>Connectivity Overall Status</p> <p>Martin-Dawes / AAA</p> <p>Test Prod</p> <p> High level status messages go here</p> <p> Additional thoughts go here.</p> <p>Martin-Dawes / AirCell</p> <p>Agile (Louisville / Itasca)</p> <p>Passur / Itasca</p> <p>Yellow</p> <p>Yellow</p> <p>CompletedOpen Defects</p> <p>Sev 1 Sev 2 Sev 3</p> <p>Overall Status and Issue</p> <p> High level status messages go here</p> <p> Additional thoughts go here. Lkasjdlkjflkjsdflkajskldjflkjsdlkjfkjsdlfjlakjdflkajsdlkjflkadlskjflajslkdjfslkdjf</p> <p> Alkjdlkfjlaksdjfslkdjflkjskldjf</p> <p> Alkjdlkfjalksjdflkjalkjlkj</p> <p> ajlksdjflkjasldkjfljaslkdjflasjdlkfjlkajsdflajsdlfjljdflkjalkjdkljlkdjflakdlfkjalkdfjlkjfdljaslkdjflkajsdlkfjlksdjflkajsdlkfjkajsdfkljaldkjf</p> <p>Systems Overall Status</p> <p>Martin-Dawes</p> <p>Dev Test</p> <p> High level status messages go here</p> <p> Additional thoughts go here.</p> <p>AAA BridgePort</p> <p>Agile</p> <p>Passur</p> <p>Yellow</p> <p>Infor</p> <p>Martin-Dawes</p> <p>AAA BridgePort</p> <p>Agile</p> <p>Passur</p> <p>ESB (TIBCO)</p> <p>AAA BridgePort</p> <p>Agile</p> <p>Passur</p> <p>ESB (TIBCO)</p> <p>Prod</p> <p>Systems Integration Overall Status</p> <p>Martin-Dawes / AAA</p> <p>Dev Test</p> <p> High level status messages go here</p> <p> Additional thoughts go here.</p> <p>Infor / Agile</p> <p>etc / etc</p> <p>etc / etc</p> <p>Yellow</p> <p>etc / etc</p> <p>etc / etc</p> <p>etc / etc</p> <p>etc / etc</p> <p>etc / etc</p> <p>etc / etc</p> <p>etc / etc</p> <p>etc / etc</p> <p>etc / etc</p> <p>etc / etc</p> <p>Prod</p> <p>Critical AirCell Launch Milestones</p> <p> Integration Testing Environment Build-out</p> <p> Integration Testing Envir...</p>