Embed Size (px)
Citation preview
A Visual Comparison Approach to Automated
Regression Testing (PDF to PDF Compare)
Visu
al R
egre
ssio
n Te
sting
Regression Testing• Type of software testing that seeks to uncover new bugs in
existing functional areas of a system after changes have been made to them
• Ensure that a change has not introduced new faults and that the intended results are still being produced
2
Visu
al R
egre
ssio
n Te
sting
Visual Regression Testing• Compares visual output• New file = or reference file • Different underlying code can produce the same output
• Testing is necessary to see if anything changed• Produce wanted results• Produce unwanted results• Break something that was working fine before
• Involves testing either• A subset of documents• A large collection of documents (preferred)
3
Visu
al R
egre
ssio
n Te
sting
The Antenna House Problem• The need to regression test new releases of Antenna House
Formatter• At least 8 maintenance releases a year
• Regression Testing (Visual Method)• Has been done manually by visually comparing output of old and
new files• Time consuming• Prone to missing minor (but important) differences• Can lead to delays in product releases• Done only on candidate release versions• Increase in support calls due to missed errors
4
Visu
al R
egre
ssio
n Te
sting
Going Automated Challenges
•Compare outputs visually, not the underlying code•High speed performance• Support for large collection of
documents•Method to generate meaningful
reports5
Visu
al R
egre
ssio
n Te
sting
What is AHRTS?• Java application• Automated visual comparison tool• Precision pixel-by-pixel comparison of output documents• Scalable & fast regression testing• Regression testing of new releases of AH Formatter• An automated system for comparing versions of rendering
software or rendered output• Beyond regression testing• Pre-production system check• Stylesheet development• Installation validation• System(s) certification
6
Visu
al R
egre
ssio
n Te
sting
Homogenize Properties Compare
Homogenize Properties
Is the contentthe same?
PDF2PDF
Select PDF documents
to be compared
PDF 1 PDF 2
Extract PDF code as
character strings
Extract PDF code as
character strings
Return XML Report Page
No Difference
Yes
How does it work?
7
Visu
al R
egre
ssio
n Te
sting
Are pagesthe same?
PDF 1 PDF 2
ComparePixel by Pixel
Page by Page
Convert to Bitmap
Convert to Bitmap
Return XML Report Page Difference
NoCreate
Composite Image of
Differences
No
Return XML Report Page
No Difference
Yes
Format to PDF using
XSL-FO
How does it work?
8
Visu
al R
egre
ssio
n Te
sting
Benefits
• Automates regression testing• High speed performance • 90%+ reduction in human effort • Handles large document comparisons• Significantly more accurate and reliable• Able to catch more subtle differences• Able to test more often, fewer problems results
in overall better product• Fewer support issues 9
Visu
al R
egre
ssio
n Te
sting
Conclusion
Beyond regression testing, a PDF to PDF comparison tool like Antenna House
Regression Testing System, can have a significant impact on improving software as it may play many roles in the development
cycle for any system generating visual outputs.
10
