Embed Size (px)
DESCRIPTION
Due to the increasing popularity of web applications, and the number of browsers and platforms on which such applications can be executed, cross-browser incompatibilities (XBIs) are becoming a serious concern for organizations that develop web-based software. Most of the techniques for XBI detection developed to date are either manual, and thus costly and error-prone, or partial and imprecise, and thus prone to generating both false positives and false negatives. To address these limitations of existing techniques, we developed X-PERT, a new automated, precise, and comprehensive approach for XBI detection. X-PERT combines several new and existing differencing techniques and is based on our findings from an extensive study of XBIs in real-world web applications. The key strength of our approach is that it handles each aspects of a web application using the differencing technique that is best suited to accurately detect XBIs related to that aspect. Our empirical evaluation shows that X-PERT is effective in detecting real-world XBIs, improves on the state of the art, and can provide useful support to developers for the diagnosis and (eventually) elimination of XBIs.
Citation preview
X"PERT:(Accurate(Identification((of(Cross2Browser(Issues((in(Web(Applications(
)
Shauvik Roy Choudhary, Mukul Prasad, Alessandro Orso
Labs of America
Partially supported by
Web(Applications9
2
3
4
Mozilla(Firefox) Internet(Explorer)
5
Mozilla(Firefox) Internet(Explorer)
6
Google(Chrome)
7
Apple(Safari)
8
Apple(Safari)Mozilla(Firefox)
9
Mozilla(Firefox)
10
Mozilla(Firefox)
Internet(Explorer)
11
Mozilla(Firefox)
Internet(Explorer)
(9
Web(Developer)
(9
Web(Developer)
(9
Web(Developer)
(Browser(versions(last(year9
9
9Source:(StatCounter.com9
(Browser(versions(last(year9
with(at(least(1%(market(share99
150mn(visits/month((StatCounter.com)9
Challenges(9
Manual(inspection9is(expensive9
DOM(differs(9between(browsers9
Mimic(end(user’s9perception(9
Produce(readable,(9effective(&(actionable(9
reports9
Engineering9Issues…9
Modern(apps(have9many(dynamic(screens(9
Detection(of(Cross2Browser(Issues((XBI)9
Web(Application9
12
My Awesome Site
Model(Generation9 Model(Comparison9
Error(Report9
Model(Definition((and(Comparison9
WebDiff([ICSM’10])Roy(Choudhary(and(Orso9
CrossCheck([ICST’12])Roy(Choudhary,(Prasad(and(Orso9
CrossT([ICSE’11](Mesbah(and(Prasad9
Screen(Transition(Graph)
Model(Definition((and(Comparison9
WebDiff([ICSM’10])Roy(Choudhary(and(Orso9
CrossCheck([ICST’12])Roy(Choudhary,(Prasad(and(Orso9
CrossT([ICSE’11](Mesbah(and(Prasad9
Screen(Model)
Screen(image(9+(geometry9
DOM(Tree9
Model(Definition((and(Comparison9
WebDiff([ICSM’10])Roy(Choudhary(and(Orso9
CrossCheck([ICST’12])Roy(Choudhary,(Prasad(and(Orso9
CrossT([ICSE’11](Mesbah(and(Prasad9
Study(of(Real(World((Cross2Browser(Issues((XBIs)9• Examined 100 websites • Found 23 with XBIs
• XBI Types o Behavioral XBIs
o Content XBIs
o Structural XBIs
((9%)
22%)30%)
57%)
Text Visual
Random(URL(Generator9
Limitations(of((existing(techniques9
• Miss Structural XBIs
• Focus on symptoms rather than causes o Duplicate reports o Low Precision
Example9
Example(9
Example(9
Example(9
Behavior(XBI)9%)
Example(9
Example(9
Content(XBI)(Visual)(–(30%)
Example(9
Structural(XBI)57%)
Content(XBI)(Visual)(–(30%)
Example(9
Content(XBI)(Visual)(–(30%)
Content(XBI)(Text)(–(22%)
Structural(XBI)57%)
Comprehensive(XBI(Detection9
• Behavioral XBIs
• Content XBIs
• Structural XBIs
Text Visual
Comprehensive(XBI(Detection9
• Behavioral XBIs
• Content XBIs
• Structural XBIs
Text Visual
Behavior(check(using(9Graph(Isomorphism9
CrossCheck([ICST’12])Roy(Choudhary,(Prasad(and(Orso9
CrossT([ICSE’11](Mesbah(and(Prasad9
Comprehensive(XBI(Detection9
• Behavioral XBIs
• Content XBIs
• Structural XBIs
Text Visual
Text(Content)check(using(9String(equality9
WebDiff([ICSM’10])Roy(Choudhary(and(Orso9
CrossCheck([ICST’12])Roy(Choudhary,(Prasad(and(Orso9
CrossT([ICSE’11](Mesbah(and(Prasad9
Comprehensive(XBI(Detection9
• Behavioral XBIs
• Content XBIs
• Structural XBIs
Text Visual
Visual(Content)check(using(9Image(Matching9
Computer9Vision9
WebDiff([ICSM’10])Roy(Choudhary(and(Orso9
CrossCheck([ICST’12])Roy(Choudhary,(Prasad(and(Orso9
Detecting(Structural(XBIs(9• Intuition:
Disruption in the relative arrangement of web page elements leads to Structural XBIs
• Idea: Abstraction to represent relative arrangement and compare it to expose errors
Alignment(Graph((AG)(9
• A graph capturing structural relationships between elements (rectangles) on the webpage
Alignment(Graph((AG)(9
• A graph capturing structural relationships between elements (rectangles) on the webpage
• An AG has: o Containment relationships o Alignment relationships o Attributes
above9right2align9left2align9
Step(1:(AG(Construction(9
Step(1:(AG(Construction(9
Step(1:(AG(Construction(9
bodyheader
menu
hr1
main
hr2
footer
link1
link2
subHeading
countText
paperListpaper1
paper2
paper3
paper4
button1
button2
19
29
39
49
5969
79
89
99
109
119
129
139
149
159
169
179
189
Step(1:(AG(Construction(a.(Containment(Relationship)
bodyheader
menu
hr1
main
hr2
footer
link1
link2
subHeading
countText
paperListpaper1
paper2
paper3
paper4
button1
button2
e(contains(e’(iff9• ((x1(≤(x1’(��y1(≤(y1’(��x2(≥(x2’(��y2(≥(y2’()(,(and9• if((same(bounds(and((XPath(e)(≤(XPath(e’)(9
e9
e’9
Step(1:(AG(Construction(a.(Containment(Relationship)
bodyheader
menu
hr1
main
hr2
footer
link1
link2
subHeading
countText
paperListpaper1
paper2
paper3
paper4
button1
button2
bodyheader
menu
hr1
main
hr2
footer
link1
link2
subHeading
countText
paperListpaper1
paper2
paper3
paper4
button1
button2
bodyheader
menu
hr1
main
hr2
footer
link1
link2
subHeading
countText
paperListpaper1
paper2
paper3
paper4
button1
button2
Step(1:(AG(Construction(b.(Alignment(Relationship)
bodyheader
menu
hr1
main
hr2
footer
link1
link2
subHeading
countText
paperListpaper1
paper2
paper3
paper4
button1
button2
Step(1:(AG(Construction(b.(Alignment(Relationship)
bodyheader
menu
hr1
main
hr2
footer
link1
link2
subHeading
countText
paperListpaper1
paper2
paper3
paper4
button1
button2
bodyheader
menu
hr1
main
hr2
footer
link1
link2
subHeading
countText
paperListpaper1
paper2
paper3
paper4
button1
button2
Step(1:(AG(Construction(b.(Alignment(Relationship)
bodyheader
menu
hr1
main
hr2
footer
link1
link2
subHeading
countText
paperListpaper1
paper2
paper3
paper4
button1
button2
bodyheader
menu
hr1
main
hr2
footer
link1
link2
subHeading
countText
paperListpaper1
paper2
paper3
paper4
button1
button2
Step(1:(AG(Construction(c.(A[ributes)
Bugon(19 Bugon(29
bodyheader
menu
hr1
main
hr2
footer
link1
link2
subHeading
countText
paperListpaper1
paper2
paper3
paper4
button1
button2
bodyheader
menu
hr1
main
hr2
footer
link1
link2
subHeading
countText
paperListpaper1
paper2
paper3
paper4
button1
button2
Step(1:(AG(Construction(c.(A[ributes)
Bugon(19 Bugon(29
What(alignment(a[ributes(can(we(infer?()
bodyheader
menu
hr1
main
hr2
footer
link1
link2
subHeading
countText
paperListpaper1
paper2
paper3
paper4
button1
button2
bodyheader
menu
hr1
main
hr2
footer
link1
link2
subHeading
countText
paperListpaper1
paper2
paper3
paper4
button1
button2
Step(1:(AG(Construction(c.(A[ributes)
Bugon(19 Bugon(29
What(alignment(a[ributes(can(we(infer?()• Bugon(1(is(on(the(left(of(Bugon(2)
x2(<(x1’9
bodyheader
menu
hr1
main
hr2
footer
link1
link2
subHeading
countText
paperListpaper1
paper2
paper3
paper4
button1
button2
bodyheader
menu
hr1
main
hr2
footer
link1
link2
subHeading
countText
paperListpaper1
paper2
paper3
paper4
button1
button2
Step(1:(AG(Construction(c.(A[ributes)
Bugon(19 Bugon(29
What(alignment(a[ributes(can(we(infer?()• Bugon(1(is(on(the(left(of(Bugon(2)
• Bugon(1(and(Bugon(2(have(their(tops(aligned)))
y1(=(y1’9
bodyheader
menu
hr1
main
hr2
footer
link1
link2
subHeading
countText
paperListpaper1
paper2
paper3
paper4
button1
button2
bodyheader
menu
hr1
main
hr2
footer
link1
link2
subHeading
countText
paperListpaper1
paper2
paper3
paper4
button1
button2
Step(1:(AG(Construction(c.(A[ributes)
Bugon(19 Bugon(29
What(alignment(a[ributes(can(we(infer?()• Bugon(1(is(on(the(left(of(Bugon(2)
• Bugon(1(and(Bugon(2(have(their(tops(aligned))• Bugon(1(and(Bugon(2(have(their(bo[oms(aligned)
y2(=(y2’9
bodyheader
menu
hr1
main
hr2
footer
link1
link2
subHeading
countText
paperListpaper1
paper2
paper3
paper4
button1
button2
bodyheader
menu
hr1
main
hr2
footer
link1
link2
subHeading
countText
paperListpaper1
paper2
paper3
paper4
button1
button2
Step(1:(AG(Construction(c.(A[ributes)
What(alignment(a[ributes(can(we(infer?()• Bugon(1(is(on(the(left(of(Bugon(2(9
• Bugon(1(and(Bugon(2(have(their(tops(aligned()
• Bugon(1(and(Bugon(2(have(their(bo[oms(aligned)
leftOf)top"align)bo[om"align)
Step(2:(Matching(AGs(9
Step(2:(Matching(AGs(9
bodyheader
menu
hr1
main
hr2
footer
link1
link2
subHeading
countText
paperListpaper1
paper2
paper3
paper4
button1
button2
above
above
above
above
above
leftOf
above
above
aboveleft-alignright-align
aboveleft-alignright-align
aboveleft-alignright-align
leftOftop-align
bottom-align
Step(2:(Matching(AGs(9
bodyheader
menu
hr1
main
hr2
footer
link1
link2
subHeading
countText
paperListpaper1
paper2
paper3
paper4
button1
button2
above
above
above
above
above
leftOf
above
above
aboveleft-alignright-align
aboveleft-alignright-align
aboveleft-alignright-align
leftOftop-align
bottom-align
bodyheader
menu
hr1
main
hr2
footer
link1
link2
subHeading
countText
paperListpaper1
paper2
paper3
paper4
button1
button2
above
above
above
above
above
leftOf
above
above
aboveleft-alignright-align
aboveleft-alignright-align
aboveleft-alignright-align
aboveleft-alignright-align
bodyheader
menu
hr1
main
hr2
footer
link1
link2
subHeading
countText
paperListpaper1
paper2
paper3
paper4
button1
button2
above
above
above
above
above
leftOf
above
above
aboveleft-alignright-align
aboveleft-alignright-align
aboveleft-alignright-align
aboveleft-alignright-align
Step(2:(Matching(AGs(9
bodyheader
menu
hr1
main
hr2
footer
link1
link2
subHeading
countText
paperListpaper1
paper2
paper3
paper4
button1
button2
above
above
above
above
above
leftOf
above
above
aboveleft-alignright-align
aboveleft-alignright-align
aboveleft-alignright-align
leftOftop-align
bottom-align
Step(2:(Matching(AGs(9
bodyheader
menu
hr1
main
hr2
footer
link1
link2
subHeading
countText
paperListpaper1
paper2
paper3
paper4
button1
button2
above
above
above
above
above
leftOf
above
above
aboveleft-alignright-align
aboveleft-alignright-align
aboveleft-alignright-align
leftOftop-align
bottom-align
bodyheader
menu
hr1
main
hr2
footer
link1
link2
subHeading
countText
paperListpaper1
paper2
paper3
paper4
button1
button2
above
above
above
above
above
leftOf
above
above
aboveleft-alignright-align
aboveleft-alignright-align
aboveleft-alignright-align
aboveleft-alignright-align
Step(2:(Matching(AGs(9
bodyheader
menu
hr1
main
hr2
footer
link1
link2
subHeading
countText
paperListpaper1
paper2
paper3
paper4
button1
button2
above
above
above
above
above
leftOf
above
above
aboveleft-alignright-align
aboveleft-alignright-align
aboveleft-alignright-align
leftOftop-align
bottom-align
bodyheader
menu
hr1
main
hr2
footer
link1
link2
subHeading
countText
paperListpaper1
paper2
paper3
paper4
button1
button2
above
above
above
above
above
leftOf
above
above
aboveleft-alignright-align
aboveleft-alignright-align
aboveleft-alignright-align
aboveleft-alignright-align
Step(2:(Matching(AGs(9
bodyheader
menu
hr1
main
hr2
footer
link1
link2
subHeading
countText
paperListpaper1
paper2
paper3
paper4
button1
button2
above
above
above
above
above
leftOf
above
above
aboveleft-alignright-align
aboveleft-alignright-align
aboveleft-alignright-align
leftOftop-align
bottom-align
bodyheader
menu
hr1
main
hr2
footer
link1
link2
subHeading
countText
paperListpaper1
paper2
paper3
paper4
button1
button2
above
above
above
above
above
leftOf
above
above
aboveleft-alignright-align
aboveleft-alignright-align
aboveleft-alignright-align
aboveleft-alignright-align
Step(2:(Matching(AGs(9
bodyheader
menu
hr1
main
hr2
footer
link1
link2
subHeading
countText
paperListpaper1
paper2
paper3
paper4
button1
button2
above
above
above
above
above
leftOf
above
above
aboveleft-alignright-align
aboveleft-alignright-align
aboveleft-alignright-align
leftOftop-align
bottom-align
bodyheader
menu
hr1
main
hr2
footer
link1
link2
subHeading
countText
paperListpaper1
paper2
paper3
paper4
button1
button2
above
above
above
above
above
leftOf
above
above
aboveleft-alignright-align
aboveleft-alignright-align
aboveleft-alignright-align
aboveleft-alignright-align
Comprehensive(XBI(Detection9
• Behavioral XBIs
• Content XBIs
• Structural XBIs
Text Visual
Comprehensive(XBI(Detection9
Text Visual
Report9Generation9
Model(Generation9
...
Behavior9Checker9
Element9Matcher9
Structure9Checker9
Content9Checker9
Model(Comparison9
✔9 ✔9
✔9
Empirical(Evaluation(9
Tool: X-PERT (Cross-Platform Error ReporTer) Research Questions: • RQ1: Can X-PERT find XBIs in real web
applications?
• RQ2: How does X-PERT compare to the state-of-art?
Experimental(Protocol(9
v14.0.19 v9.0.99v/s9
Ran(X2PERT(on(Subject(Applications9
Subjects9
12
My Awesome Site
12
My Awesome Site
12
My Awesome Site
+9 =(XBIs9(Ground(Truth)9Manual(Check9
Error(Reports9 Manual(Check9
=9False(Positives(9&(Negatives9+9
Subjects(9
Name% Type% States% Transi/ons%DOM%Nodes%(per%screen)%max% min% average%
!Organizer! Produc/vity! 13! 99! 27482! 10001! 13051!!!GrantaBooks! Publisher! 9! 8! 37800! 15625! 25852!!!DesignTrust! Business! 10! 20! 26437! 7772! 18694!!!DivineLife! Spiritual! 10! 9! 140611! 9082! 49886!!!SaiBaba! Religious! 13! 20! 42606! 524! 12162!!!Breakaway! Sport! 19! 18! 45148! 8191! 13059!!!Conference! Informa/on! 3! 12! 878! 817! 853!!!Fisherman! Restaurant! 15! 17! 39146! 15720! 21336!!!Valleyforge! Lodge! 4! 12! 5416! 4733! 5046!!!UniMelb! University! 9! 8! 15142! 12131! 13792!!!Konqueror! SoUware! 5! 4! 17586! 15468! 16187!!!UBC! Club! 7! 7! 20610! 7834! 12094!!!BMVBS! Ministry! 5! 20! 19490! 12544! 15695!!!StarWars! Movie! 10! 9! 28452! 19719! 22626!!!
800(2(140K(elements9
Prior(Art((6)9
Example(and(Survey((4)9
Random((4)9
Subjects(9
Name% Type% States% Transi/ons%DOM%Nodes%(per%screen)%max% min% average%
!Organizer! Produc/vity! 13! 99! 27482! 10001! 13051!!!GrantaBooks! Publisher! 9! 8! 37800! 15625! 25852!!!DesignTrust! Business! 10! 20! 26437! 7772! 18694!!!DivineLife! Spiritual! 10! 9! 140611! 9082! 49886!!!SaiBaba! Religious! 13! 20! 42606! 524! 12162!!!Breakaway! Sport! 19! 18! 45148! 8191! 13059!!!Conference! Informa/on! 3! 12! 878! 817! 853!!!Fisherman! Restaurant! 15! 17! 39146! 15720! 21336!!!Valleyforge! Lodge! 4! 12! 5416! 4733! 5046!!!UniMelb! University! 9! 8! 15142! 12131! 13792!!!Konqueror! SoUware! 5! 4! 17586! 15468! 16187!!!UBC! Club! 7! 7! 20610! 7834! 12094!!!BMVBS! Ministry! 5! 20! 19490! 12544! 15695!!!StarWars! Movie! 10! 9! 28452! 19719! 22626!!!
800(2(140K(elements9
Effectiveness(9
NAME% XBI%BEHAVIOR% STRUCTURE%
CONTENT%TOTAL%TEXT% IMAGE%
TP% FP% TP% FP% TP% FP% TP% FP% TP% FP%Organizer! 10! 1! 0! 9! 0! 0! 0! 0! 0! 10! 0!
GrantaBooks! 27! 16! 0! 11! 0! 0! 0! 0! 0! 27! 0!
DesignTrust! 7! 2! 0! 5! 3! 0! 0! 0! 0! 7! 3!DivineLife! 11! 7! 0! 3! 6! 1! 0! 0! 0! 11! 6!SaiBaba! 5! 2! 0! 2! 9! 0! 0! 0! 0! 4! 9!Breakaway! 13! 0! 0! 10! 2! 0! 0! 0! 0! 10! 2!Conference! 7! 2! 0! 3! 0! 1! 0! 1! 0! 7! 0!Fisherman! 5! 1! 0! 3! 1! 0! 1! 1! 0! 5! 2!Valleyforge! 3! 0! 0! 2! 2! 0! 0! 1! 0! 3! 2!UniMelb! 2! 2! 0! 0! 0! 0! 0! 0! 1! 2! 1!Konqueror! 0! 0! 0! 0! 0! 0! 0! 0! 6! 0! 6!UBC! 0! 0! 0! 0! 0! 0! 0! 0! 0! 0! 0!BMVBS! 1! 0! 0! 0! 0! 0! 0! 0! 0! 0! 0!StarWars! 12! 0! 0! 12! 0! 0! 0! 0! 0! 12! 0!TOTAL% 103% 33% 0% 60% 23% 2% 1% 3% 7% 98% 31%
Effectiveness(9
NAME% XBI%BEHAVIOR% STRUCTURE%
CONTENT%TOTAL%TEXT% IMAGE%
TP% FP% TP% FP% TP% FP% TP% FP% TP% FP%Organizer! 10! 1! 0! 9! 0! 0! 0! 0! 0! 10! 0!
GrantaBooks! 27! 16! 0! 11! 0! 0! 0! 0! 0! 27! 0!
DesignTrust! 7! 2! 0! 5! 3! 0! 0! 0! 0! 7! 3!DivineLife! 11! 7! 0! 3! 6! 1! 0! 0! 0! 11! 6!SaiBaba! 5! 2! 0! 2! 9! 0! 0! 0! 0! 4! 9!Breakaway! 13! 0! 0! 10! 2! 0! 0! 0! 0! 10! 2!Conference! 7! 2! 0! 3! 0! 1! 0! 1! 0! 7! 0!Fisherman! 5! 1! 0! 3! 1! 0! 1! 1! 0! 5! 2!Valleyforge! 3! 0! 0! 2! 2! 0! 0! 1! 0! 3! 2!UniMelb! 2! 2! 0! 0! 0! 0! 0! 0! 1! 2! 1!Konqueror! 0! 0! 0! 0! 0! 0! 0! 0! 6! 0! 6!UBC! 0! 0! 0! 0! 0! 0! 0! 0! 0! 0! 0!BMVBS! 1! 0! 0! 0! 0! 0! 0! 0! 0! 0! 0!StarWars! 12! 0! 0! 12! 0! 0! 0! 0! 0! 12! 0!TOTAL% 103% 33% 0% 60% 23% 2% 1% 3% 7% 98% 31%
RQ1:()Can(X2PERT(find(XBIs(in(real(web(applications?99Answer:(Yes)
Improvement(9
NAME% XBI%XQPERT% CROSSCHECK%
Prec% FP% Recl% Dp% Prec% FP% Recl% Dp%Organizer! 10! 100%! 0! 100%! 0! 80%! 2! 80%! 13!GrantaBooks! 27! 100%! 0! 100%! 0! 96%! 1! 100%! 0!DesignTrust! 7! 70%! 3! 100%! 0! 5%! 122! 86%! 3!DivineLife! 11! 65%! 6! 100%! 0! 29%! 24! 91%! 3!SaiBaba! 5! 31%! 9! 80%! 0! 7%! 53! 80%! 10!Breakaway! 13! 83%! 2! 77%! 1! 13%! 49! 54%! 12!Conference! 7! 100%! 0! 100%! 0! 100%! 0! 100%! 0!Fisherman! 5! 71%! 2! 100%! 0! 44%! 5! 80%! 8!Valleyforge! 3! 60%! 2! 100%! 0! 50%! 1! 33%! 0!UniMelb! 2! 67%! 1! 100%! 0! 7%! 27! 100%! 0!Konqueror! 0! NA! 6! NA! 0! 0%! 11! NA! 0!UBC! 0! NA! 0! NA! 0! 0%! 1! NA! 0!BMVBS! 1! NA! 0! NA! 0! 0%! 2! 0%! 0!StarWars! 12! 100%! 0! 100%! 0! 10%! 91! 83%! 3!TOTAL% 103% 77%% 31% 95%% 1% 32%% 389% 74%% 52%
Improvement(9
NAME% XBI%XQPERT% CROSSCHECK%
Prec% FP% Recl% Dp% Prec% FP% Recl% Dp%Organizer! 10! 100%! 0! 100%! 0! 80%! 2! 80%! 13!GrantaBooks! 27! 100%! 0! 100%! 0! 96%! 1! 100%! 0!DesignTrust! 7! 70%! 3! 100%! 0! 5%! 122! 86%! 3!DivineLife! 11! 65%! 6! 100%! 0! 29%! 24! 91%! 3!SaiBaba! 5! 31%! 9! 80%! 0! 7%! 53! 80%! 10!Breakaway! 13! 83%! 2! 77%! 1! 13%! 49! 54%! 12!Conference! 7! 100%! 0! 100%! 0! 100%! 0! 100%! 0!Fisherman! 5! 71%! 2! 100%! 0! 44%! 5! 80%! 8!Valleyforge! 3! 60%! 2! 100%! 0! 50%! 1! 33%! 0!UniMelb! 2! 67%! 1! 100%! 0! 7%! 27! 100%! 0!Konqueror! 0! NA! 6! NA! 0! 0%! 11! NA! 0!UBC! 0! NA! 0! NA! 0! 0%! 1! NA! 0!BMVBS! 1! NA! 0! NA! 0! 0%! 2! 0%! 0!StarWars! 12! 100%! 0! 100%! 0! 10%! 91! 83%! 3!TOTAL% 103% 77%% 31% 95%% 1% 32%% 389% 74%% 52%
X"PERT)9Precision()((=(77%(((((((45%!)99Recall()((=(95%((((((14%!)9
Improvement(9
NAME% XBI%XQPERT% CROSSCHECK%
Prec% FP% Recl% Dp% Prec% FP% Recl% Dp%Organizer! 10! 100%! 0! 100%! 0! 80%! 2! 80%! 13!GrantaBooks! 27! 100%! 0! 100%! 0! 96%! 1! 100%! 0!DesignTrust! 7! 70%! 3! 100%! 0! 5%! 122! 86%! 3!DivineLife! 11! 65%! 6! 100%! 0! 29%! 24! 91%! 3!SaiBaba! 5! 31%! 9! 80%! 0! 7%! 53! 80%! 10!Breakaway! 13! 83%! 2! 77%! 1! 13%! 49! 54%! 12!Conference! 7! 100%! 0! 100%! 0! 100%! 0! 100%! 0!Fisherman! 5! 71%! 2! 100%! 0! 44%! 5! 80%! 8!Valleyforge! 3! 60%! 2! 100%! 0! 50%! 1! 33%! 0!UniMelb! 2! 67%! 1! 100%! 0! 7%! 27! 100%! 0!Konqueror! 0! NA! 6! NA! 0! 0%! 11! NA! 0!UBC! 0! NA! 0! NA! 0! 0%! 1! NA! 0!BMVBS! 1! NA! 0! NA! 0! 0%! 2! 0%! 0!StarWars! 12! 100%! 0! 100%! 0! 10%! 91! 83%! 3!TOTAL% 103% 77%% 31% 95%% 1% 32%% 389% 74%% 52%
X"PERT)9Precision()((=(77%(((((((45%!)99Recall()((=(95%((((((14%!)9
RQ2:()How(does(X2PERT(compare(to(the(state2of2art?99Answer:(X"PERT(has(be[er(precision(and(recall)
Experimental(Data(&(Tool9
• Release o Experiments (Crawl Data + Reports) o Layout testing algorithm implementation
http://gatech.github.io/xpert
Related(Work9o Industrial tools
• Test Suites for Web Browsers – Acid and test262
• MS Expression Web, BrowserShots, BrowserStack
• Browsera, MogoTest, BrowserBite
o Research techniques
• Eaton & Memon [IJWET’07]
• Dallmeier, Burger, Orth & Zeller [JSTools’12]
77
Contributions(9
Empirical(evaluation9shows(effectiveness9
New(Structural(XBI9Detection(Algorithm9
bodyheader
menu
hr1
main
hr2
footer
link1
link2
subHeading
countText
paperListpaper1
paper2
paper3
paper4
button1
button2
above
above
above
above
above
leftOf
above
above
aboveleft-alignright-align
aboveleft-alignright-align
aboveleft-alignright-align
leftOftop-align
bottom-align
Comprehensive(9Technique(to(detect(XBIs9
Web$Application.
12
My Awesome Site
Model$Generation. Model$Comparison.
Error$Report.
Study(of(Real(World(XBIs9
XBI$Distribu,on$Behavior$ 9%$Content$ Text$ 22%$
Visual$ 30%$Structure$ 57%$
