Embed Size (px)
Citation preview
Mindfire Solutions Case Study
Copyright 2013, Mindfire Solutions. All rights reserved. [email protected]
Software Testing of a Content Management System A Manual and Automation Testing Case Study
Executive summary: The CMS being one of the most powerful content
management systems on the market today, is now available as
a cloud-based, SaaS CMS. Designed for maximum ease of use,
the application Cloud offers all the sophisticated features
prized by leading organizations and delivers them to you
through the cloud. That means there’s no hardware to install
or product updates to download. It’s seamlessly handled for
you in an infrastructure-free environment. This liberates your
organization from the burden of maintaining a Web
infrastructure, and allows IT to focus more time supporting
other strategic initiatives while enabling marketing to get on with theirs.
The initial engagement started the product development, laying the foundation for a long-term relationship including
development of next versions in the software product roadmap, along with the development or establishment of
supportive and extending tools, solutions, & related services with a motto in mind to simplify Web site development and
authoring since its introduction.
The product/system roadmap includes considerations for providing more features like Complete Web Publishing
Framework, XML Capabilities, Document and Image Versioning Management, Online Image Editing and versioning,
Custom Elements - Content Reuse, Cascading Style Sheets Support, Authoring and Publishing Content, Marketing
Features, Architecture, Open source components, Database Managed Content Architecture, Enterprise Level Features
like Static Site Generation. It also includes considerations for moving to a higher SaaS maturity models/levels beyond
level 2 (Configurable) i.e., Level 3 - Configurable and Multi-tenant Efficient, and then Level 4 - Scalable, Configurable and
Multi-tenant Efficient.
About our Client: Client: CMS Solution provider | Location: MA, USA | Industry: Publishing, Print industry
Business situation: The US firm set out to achieve its primary goal –testing of its internet/web based, workflow-driven content management
software through which enterprises could easily perform web content management operations. Although multiple
software service providers proposed solutions, Mindfire’s excellent testing solution(s) and strategy stood out, fulfilling
the core requirement of Manual Testing in a SaaS-based implementation, and providing a definitive roadmap to deliver
various Bug Free product versions.
Mindfire’s QA team took immediate charge of the project and immediately proposed a well thought out solution, in
order to cover every possible scenario for best of quality assurance processes to be executed with a focus on platform
compatibility. The application was tested across multiple operating systems ranging from Windows 7 and others such as
Windows Vista, Windows XP, Windows 2003, Mac 10.4, Mac 10.5, Mac 10.6 and Mac 10.7. For all these operating
system’s the team at Mindfire created the environment on different hardware and server resources.
Mindfire Solutions Case Study
Copyright 2013, Mindfire Solutions. All rights reserved. [email protected]
Initially the client used Waterfall model as the SDLC and Mindfire’s expert QA team immediately started validating the
wireframes provided by the client and immediately proposed a
Custom Automation Application framework ensuring highest quality
delivered to the client. Mindfire QA Team strength was immediately
increased from 4 to 8 resources by the client which showed the
level of quality commitment mindfire provided to the client.
Solution details:
With the client’s intent keeping in mind, Mindfire re-validated all the Wireframes and designed a set of business rules
and added the change request in the architecture of the product
which opened the gates to update the existing testcases to be more
functional than the testcases which the client initially had. Mindfire’s
QA team regularly monitored the test cases to track the changes in
the functionality of the application and then simultaneously changed
it in the test cases as well. Test cases were put on VSS server so that
any person involved in the project can easily check-out and check-in
the updated testcase at any point of time. QA team also added all fixed issues as part of the test cases which enhanced
the test script in uncovering more scenarios. Some important issues were also uncovered when the testers performed
functionality/ bug regression/installation and compatibility testing.
The client being technically strong designed an application to keep
track business rules changes for different components, methods
and types used in the application. Mindfire’s QA Team immediately
proposed for the design of an automation application which is a
web-based tool for creating and running unit tests on the parts of
the product code exposed by an XML-based interface. Starting
from the earlier versions, all new changes that have been created
were made supported by this interface.
The automation application was created with few things in mind: -
Mindfire QA Team writing new and modifying existing test cases
An unmanned scheduler that runs tests continually (referred to as ‘automation’ here) This is the ultimate intended audience of this program. Everything you do prepares tests that this process will run for every nightly build.
Mindfire QA Team with mutual understanding with the client agreed to follow certain standard set by market leaders.
They used to follow daily meetings to analyze progress on various milestones and resolve issues if any through
discussions and regular meeting for iterative development where requirements and solutions evolve through
collaborations between the different onsite/offsite dev teams in the project.
Tests were conducted as per their requirement using multiple operating systems ranging from the then recently
launched Windows 7 and MAC osX 10. 7 Lion and others such as Windows Vista, Windows XP, Mac 10.4, Mac 10.5, Mac
10.6 with browsers used such as IE7 - IE9(All Compatibility Modes), FFv3.5.6 – FFv7, Google Chrome and MAC Safari and
Mindfire Solutions Case Study
Copyright 2013, Mindfire Solutions. All rights reserved. [email protected]
MAC Firefox. Different environments were created to test the application on different hardware and server resources
such as Mac, Windows with multiple browsers. All the Bugs and CR’s were reported using a very well known software
configuration management (SCM) tool known as Accurev.
Achievements:
The client being technically strong came up with Waterfall model as the SDLC when the project begun. To make it more
process oriented and make it the best quality product in the market the testing team improvised the communication
process by adding specific agile processes to achieve all the milestones identified in the Software Project Schedule.
After the Go-Live, the end result delivered was very much satisfactory and the decision proved to be a fruitful one.
Monitoring the test cases was also very cumbersome, which our team handled well to overcome the issues and as a
result provided a suitable environment for the product’s functionality across different platforms.
Technologies:
Languages: Coldfusion, JAVA, Javascript, AJAX, XML.
Database: My SQL, MS SQL, Oracle.
Mechanisms: - Shared Database, Replication Database And Static Content Generation.
Automation: Test Library Automation Framework exposed by XML based interface.
Servers: Windows 2003, Linux, Solaris, Railo
Final Results:
Software System
Mindfire Solutions Case Study
Copyright 2013, Mindfire Solutions. All rights reserved. [email protected]
Figure showing Shared DB Concept in the Product.
Fig Showing Replication DB Concept in the Product
Mindfire Solutions Case Study
Copyright 2013, Mindfire Solutions. All rights reserved. [email protected]
Customer Benefits:
The specifications were more on the conceptual side. Mindfire provided product testing solutions to come to the best
possible Bug Free design & implementation. Client communication and co-ordination was the key to the decision-making
on feature details.
Mindfire has an excellent understanding of the business, architectural, operational, customer base specific
considerations and requirements for various steps/versions the complete product/system roadmap. Clear vision of the
scope and future requirements continues to be a big plus in reducing the number of iterations, redundancy, and
associated costs.
Mindfire sharply made it a bug free product in terms of finish, stability, and performance. This version allowed the client
to convince potential buyers/customers ahead of time, much before the system release – learning more and adapt to
customer requirements i.e., the targeted markets.
The client continues to benefit on overall cost for the software testing work with Mindfire. Mindfire’s skills, experience,
and aggressive pricing has greatly reduced the total costs.
Future relationship: Given our great commitment and high-quality delivery, we have continued to be the service provider for the next
versions in the product roadmap and also will be the future testing partner with the client. The client initially started
with testing of a single product with 4 QA testers. But, after Mindfire’s incredible performance and delivery, the client
increased the team size to 9, for testing multiple future releases. Relationship with the client is now into the 8th year and
we hope that it continues to grow.
Mindfire Solutions Case Study
Copyright 2013, Mindfire Solutions. All rights reserved. [email protected]
Need Dynamics CRM Help?
Phone: 1-248-686-1424
Web: www.mindfiresolutions.com
Email: [email protected]
Need Software Testing Help?
Phone: 1-248-686-1424
Web: www.mindfiresolutions.com
Email: [email protected]
Mindfire Solutions is a 13 years old leading Software Development and Testing services company with a strong track record of working with small and mid‐size clients in US, Europe, and Asia‐Pacific. With more than 750 spirited software engineers across two advanced development centers, Mindfire has successfully delivered over 1000 projects for its 300+ clients spanning SMBs, ISVs, SaaS, Global 2000, and Fortune 500 firms.
© 2013 Mindfire Solutions | www.mindfiresolutions.com
