X-PERT: Accurate Identification of Cross-Browser Issues in Web Applications

X"PERT:(Accurate(Identification((of(Cross2Browser(Issues((in(Web(Applications(

Shauvik Roy Choudhary, Mukul Prasad, Alessandro Orso

Labs of America

Partially supported by

Web(Applications9

Mozilla(Firefox) Internet(Explorer)

Google(Chrome)

Apple(Safari)

Apple(Safari)Mozilla(Firefox)

Mozilla(Firefox)

Internet(Explorer)

Mozilla(Firefox)

Internet(Explorer)

Web(Developer)

(Browser(versions(last(year9

9Source:(StatCounter.com9

(Browser(versions(last(year9

with(at(least(1%(market(share99

150mn(visits/month((StatCounter.com)9

Challenges(9

Manual(inspection9is(expensive9

DOM(differs(9between(browsers9

Mimic(end(user’s9perception(9

Produce(readable,(9effective(&(actionable(9

reports9

Engineering9Issues…9

Modern(apps(have9many(dynamic(screens(9

Detection(of(Cross2Browser(Issues((XBI)9

Web(Application9

My Awesome Site

Model(Generation9 Model(Comparison9

Error(Report9

Model(Definition((and(Comparison9

WebDiff([ICSM’10])Roy(Choudhary(and(Orso9

CrossCheck([ICST’12])Roy(Choudhary,(Prasad(and(Orso9

CrossT([ICSE’11](Mesbah(and(Prasad9

Screen(Transition(Graph)

Screen(Model)

Screen(image(9+(geometry9

DOM(Tree9

Study(of(Real(World((Cross2Browser(Issues((XBIs)9•  Examined 100 websites •  Found 23 with XBIs

•  XBI Types o Behavioral XBIs

o Content XBIs

o Structural XBIs

22%)30%)

Text Visual

Random(URL(Generator9

Limitations(of((existing(techniques9

•  Miss Structural XBIs

•  Focus on symptoms rather than causes o  Duplicate reports o  Low Precision

Example9

Example(9

Behavior(XBI)9%)

Example(9

Content(XBI)(Visual)(–(30%)

Example(9

Structural(XBI)57%)

Example(9

Content(XBI)(Text)(–(22%)

Structural(XBI)57%)

Comprehensive(XBI(Detection9

•  Behavioral XBIs

•  Content XBIs

•  Structural XBIs

Text Visual

•  Content XBIs

Text Visual

Behavior(check(using(9Graph(Isomorphism9

•  Content XBIs

Text Visual

Text(Content)check(using(9String(equality9

•  Content XBIs

Text Visual

Visual(Content)check(using(9Image(Matching9

Computer9Vision9

Detecting(Structural(XBIs(9•  Intuition:

Disruption in the relative arrangement of web page elements leads to Structural XBIs

•  Idea: Abstraction to represent relative arrangement and compare it to expose errors

Alignment(Graph((AG)(9

•  A graph capturing structural relationships between elements (rectangles) on the webpage

Alignment(Graph((AG)(9

•  A graph capturing structural relationships between elements (rectangles) on the webpage

•  An AG has: o Containment relationships o Alignment relationships o Attributes

above9right2align9left2align9

Step(1:(AG(Construction(9

bodyheader

footer

subHeading

countText

paperListpaper1

paper2

paper3

paper4

button1

button2

Step(1:(AG(Construction(a.(Containment(Relationship)

bodyheader

footer

subHeading

countText

paperListpaper1

paper2

paper3

paper4

button1

button2

e(contains(e’(iff9•  ((x1(≤(x1’(��y1(≤(y1’(��x2(≥(x2’(��y2(≥(y2’()(,(and9•  if((same(bounds(and((XPath(e)(≤(XPath(e’)(9

Step(1:(AG(Construction(a.(Containment(Relationship)

bodyheader

footer

subHeading

countText

paperListpaper1

paper2

paper3

paper4

button1

button2

bodyheader

footer

subHeading

countText

paperListpaper1

paper2

paper3

paper4

button1

button2

bodyheader

footer

subHeading

countText

paperListpaper1

paper2

paper3

paper4

button1

button2

Step(1:(AG(Construction(b.(Alignment(Relationship)

bodyheader

footer

subHeading

countText

paperListpaper1

paper2

paper3

paper4

button1

button2

bodyheader

footer

subHeading

countText

paperListpaper1

paper2

paper3

paper4

button1

button2

bodyheader

footer

subHeading

countText

paperListpaper1

paper2

paper3

paper4

button1

button2

bodyheader

footer

subHeading

countText

paperListpaper1

paper2

paper3

paper4

button1

button2

bodyheader

footer

subHeading

countText

paperListpaper1

paper2

paper3

paper4

button1

button2

Step(1:(AG(Construction(c.(A[ributes)

Bugon(19 Bugon(29

bodyheader

footer

subHeading

countText

paperListpaper1

paper2

paper3

paper4

button1

button2

bodyheader

footer

subHeading

countText

paperListpaper1

paper2

paper3

paper4

button1

button2

Bugon(19 Bugon(29

What(alignment(a[ributes(can(we(infer?()

bodyheader

footer

subHeading

countText

paperListpaper1

paper2

paper3

paper4

button1

button2

bodyheader

footer

subHeading

countText

paperListpaper1

paper2

paper3

paper4

button1

button2

Bugon(19 Bugon(29

What(alignment(a[ributes(can(we(infer?()•  Bugon(1(is(on(the(left(of(Bugon(2)

x2(<(x1’9

bodyheader

footer

subHeading

countText

paperListpaper1

paper2

paper3

paper4

button1

button2

bodyheader

footer

subHeading

countText

paperListpaper1

paper2

paper3

paper4

button1

button2

Bugon(19 Bugon(29

•  Bugon(1(and(Bugon(2(have(their(tops(aligned)))

y1(=(y1’9

bodyheader

footer

subHeading

countText

paperListpaper1

paper2

paper3

paper4

button1

button2

bodyheader

footer

subHeading

countText

paperListpaper1

paper2

paper3

paper4

button1

button2

Bugon(19 Bugon(29

•  Bugon(1(and(Bugon(2(have(their(tops(aligned))•  Bugon(1(and(Bugon(2(have(their(bo[oms(aligned)

y2(=(y2’9

bodyheader

footer

subHeading

countText

paperListpaper1

paper2

paper3

paper4

button1

button2

bodyheader

footer

subHeading

countText

paperListpaper1

paper2

paper3

paper4

button1

button2

What(alignment(a[ributes(can(we(infer?()•  Bugon(1(is(on(the(left(of(Bugon(2(9

•  Bugon(1(and(Bugon(2(have(their(tops(aligned()

•  Bugon(1(and(Bugon(2(have(their(bo[oms(aligned)

leftOf)top"align)bo[om"align)

Step(2:(Matching(AGs(9

bodyheader

footer

subHeading

countText

paperListpaper1

paper2

paper3

paper4

button1

button2

leftOf

aboveleft-alignright-align

leftOftop-align

bottom-align

bodyheader

footer

subHeading

countText

paperListpaper1

paper2

paper3

paper4

button1

button2

leftOf

leftOftop-align

bottom-align

bodyheader

footer

subHeading

countText

paperListpaper1

paper2

paper3

paper4

button1

button2

leftOf

bodyheader

footer

subHeading

countText

paperListpaper1

paper2

paper3

paper4

button1

button2

leftOf

bodyheader

footer

subHeading

countText

paperListpaper1

paper2

paper3

paper4

button1

button2

leftOf

leftOftop-align

bottom-align

bodyheader

footer

subHeading

countText

paperListpaper1

paper2

paper3

paper4

button1

button2

leftOf

leftOftop-align

bottom-align

bodyheader

footer

subHeading

countText

paperListpaper1

paper2

paper3

paper4

button1

button2

leftOf

bodyheader

footer

subHeading

countText

paperListpaper1

paper2

paper3

paper4

button1

button2

leftOf

leftOftop-align

bottom-align

bodyheader

footer

subHeading

countText

paperListpaper1

paper2

paper3

paper4

button1

button2

leftOf

bodyheader

footer

subHeading

countText

paperListpaper1

paper2

paper3

paper4

button1

button2

leftOf

leftOftop-align

bottom-align

bodyheader

footer

subHeading

countText

paperListpaper1

paper2

paper3

paper4

button1

button2

leftOf

bodyheader

footer

subHeading

countText

paperListpaper1

paper2

paper3

paper4

button1

button2

leftOf

leftOftop-align

bottom-align

bodyheader

footer

subHeading

countText

paperListpaper1

paper2

paper3

paper4

button1

button2

leftOf

•  Content XBIs

Text Visual

Report9Generation9

Model(Generation9

Behavior9Checker9

Element9Matcher9

Structure9Checker9

Content9Checker9

Model(Comparison9

✔9 ✔9

Empirical(Evaluation(9

Tool: X-PERT (Cross-Platform Error ReporTer) Research Questions: •  RQ1: Can X-PERT find XBIs in real web

applications?

•  RQ2: How does X-PERT compare to the state-of-art?

Experimental(Protocol(9

v14.0.19 v9.0.99v/s9

Ran(X2PERT(on(Subject(Applications9

Subjects9

My Awesome Site

+9 =(XBIs9(Ground(Truth)9Manual(Check9

Error(Reports9 Manual(Check9

=9False(Positives(9&(Negatives9+9

Subjects(9

Name% Type% States% Transi/ons%DOM%Nodes%(per%screen)%max% min% average%

!Organizer! Produc/vity! 13! 99! 27482! 10001! 13051!!!GrantaBooks! Publisher! 9! 8! 37800! 15625! 25852!!!DesignTrust! Business! 10! 20! 26437! 7772! 18694!!!DivineLife! Spiritual! 10! 9! 140611! 9082! 49886!!!SaiBaba! Religious! 13! 20! 42606! 524! 12162!!!Breakaway! Sport! 19! 18! 45148! 8191! 13059!!!Conference! Informa/on! 3! 12! 878! 817! 853!!!Fisherman! Restaurant! 15! 17! 39146! 15720! 21336!!!Valleyforge! Lodge! 4! 12! 5416! 4733! 5046!!!UniMelb! University! 9! 8! 15142! 12131! 13792!!!Konqueror! SoUware! 5! 4! 17586! 15468! 16187!!!UBC! Club! 7! 7! 20610! 7834! 12094!!!BMVBS! Ministry! 5! 20! 19490! 12544! 15695!!!StarWars! Movie! 10! 9! 28452! 19719! 22626!!!

800(2(140K(elements9

Prior(Art((6)9

Example(and(Survey((4)9

Random((4)9

Subjects(9

Name% Type% States% Transi/ons%DOM%Nodes%(per%screen)%max% min% average%

!Organizer! Produc/vity! 13! 99! 27482! 10001! 13051!!!GrantaBooks! Publisher! 9! 8! 37800! 15625! 25852!!!DesignTrust! Business! 10! 20! 26437! 7772! 18694!!!DivineLife! Spiritual! 10! 9! 140611! 9082! 49886!!!SaiBaba! Religious! 13! 20! 42606! 524! 12162!!!Breakaway! Sport! 19! 18! 45148! 8191! 13059!!!Conference! Informa/on! 3! 12! 878! 817! 853!!!Fisherman! Restaurant! 15! 17! 39146! 15720! 21336!!!Valleyforge! Lodge! 4! 12! 5416! 4733! 5046!!!UniMelb! University! 9! 8! 15142! 12131! 13792!!!Konqueror! SoUware! 5! 4! 17586! 15468! 16187!!!UBC! Club! 7! 7! 20610! 7834! 12094!!!BMVBS! Ministry! 5! 20! 19490! 12544! 15695!!!StarWars! Movie! 10! 9! 28452! 19719! 22626!!!

800(2(140K(elements9

Effectiveness(9

NAME% XBI%BEHAVIOR% STRUCTURE%

CONTENT%TOTAL%TEXT% IMAGE%

TP% FP% TP% FP% TP% FP% TP% FP% TP% FP%Organizer! 10! 1! 0! 9! 0! 0! 0! 0! 0! 10! 0!

GrantaBooks! 27! 16! 0! 11! 0! 0! 0! 0! 0! 27! 0!

DesignTrust! 7! 2! 0! 5! 3! 0! 0! 0! 0! 7! 3!DivineLife! 11! 7! 0! 3! 6! 1! 0! 0! 0! 11! 6!SaiBaba! 5! 2! 0! 2! 9! 0! 0! 0! 0! 4! 9!Breakaway! 13! 0! 0! 10! 2! 0! 0! 0! 0! 10! 2!Conference! 7! 2! 0! 3! 0! 1! 0! 1! 0! 7! 0!Fisherman! 5! 1! 0! 3! 1! 0! 1! 1! 0! 5! 2!Valleyforge! 3! 0! 0! 2! 2! 0! 0! 1! 0! 3! 2!UniMelb! 2! 2! 0! 0! 0! 0! 0! 0! 1! 2! 1!Konqueror! 0! 0! 0! 0! 0! 0! 0! 0! 6! 0! 6!UBC! 0! 0! 0! 0! 0! 0! 0! 0! 0! 0! 0!BMVBS! 1! 0! 0! 0! 0! 0! 0! 0! 0! 0! 0!StarWars! 12! 0! 0! 12! 0! 0! 0! 0! 0! 12! 0!TOTAL% 103% 33% 0% 60% 23% 2% 1% 3% 7% 98% 31%

Effectiveness(9

NAME% XBI%BEHAVIOR% STRUCTURE%

CONTENT%TOTAL%TEXT% IMAGE%

TP% FP% TP% FP% TP% FP% TP% FP% TP% FP%Organizer! 10! 1! 0! 9! 0! 0! 0! 0! 0! 10! 0!

GrantaBooks! 27! 16! 0! 11! 0! 0! 0! 0! 0! 27! 0!

DesignTrust! 7! 2! 0! 5! 3! 0! 0! 0! 0! 7! 3!DivineLife! 11! 7! 0! 3! 6! 1! 0! 0! 0! 11! 6!SaiBaba! 5! 2! 0! 2! 9! 0! 0! 0! 0! 4! 9!Breakaway! 13! 0! 0! 10! 2! 0! 0! 0! 0! 10! 2!Conference! 7! 2! 0! 3! 0! 1! 0! 1! 0! 7! 0!Fisherman! 5! 1! 0! 3! 1! 0! 1! 1! 0! 5! 2!Valleyforge! 3! 0! 0! 2! 2! 0! 0! 1! 0! 3! 2!UniMelb! 2! 2! 0! 0! 0! 0! 0! 0! 1! 2! 1!Konqueror! 0! 0! 0! 0! 0! 0! 0! 0! 6! 0! 6!UBC! 0! 0! 0! 0! 0! 0! 0! 0! 0! 0! 0!BMVBS! 1! 0! 0! 0! 0! 0! 0! 0! 0! 0! 0!StarWars! 12! 0! 0! 12! 0! 0! 0! 0! 0! 12! 0!TOTAL% 103% 33% 0% 60% 23% 2% 1% 3% 7% 98% 31%

RQ1:()Can(X2PERT(find(XBIs(in(real(web(applications?99Answer:(Yes)

Improvement(9

NAME% XBI%XQPERT% CROSSCHECK%

Prec% FP% Recl% Dp% Prec% FP% Recl% Dp%Organizer! 10! 100%! 0! 100%! 0! 80%! 2! 80%! 13!GrantaBooks! 27! 100%! 0! 100%! 0! 96%! 1! 100%! 0!DesignTrust! 7! 70%! 3! 100%! 0! 5%! 122! 86%! 3!DivineLife! 11! 65%! 6! 100%! 0! 29%! 24! 91%! 3!SaiBaba! 5! 31%! 9! 80%! 0! 7%! 53! 80%! 10!Breakaway! 13! 83%! 2! 77%! 1! 13%! 49! 54%! 12!Conference! 7! 100%! 0! 100%! 0! 100%! 0! 100%! 0!Fisherman! 5! 71%! 2! 100%! 0! 44%! 5! 80%! 8!Valleyforge! 3! 60%! 2! 100%! 0! 50%! 1! 33%! 0!UniMelb! 2! 67%! 1! 100%! 0! 7%! 27! 100%! 0!Konqueror! 0! NA! 6! NA! 0! 0%! 11! NA! 0!UBC! 0! NA! 0! NA! 0! 0%! 1! NA! 0!BMVBS! 1! NA! 0! NA! 0! 0%! 2! 0%! 0!StarWars! 12! 100%! 0! 100%! 0! 10%! 91! 83%! 3!TOTAL% 103% 77%% 31% 95%% 1% 32%% 389% 74%% 52%

Improvement(9

X"PERT)9Precision()((=(77%(((((((45%!)99Recall()((=(95%((((((14%!)9

Improvement(9

X"PERT)9Precision()((=(77%(((((((45%!)99Recall()((=(95%((((((14%!)9

RQ2:()How(does(X2PERT(compare(to(the(state2of2art?99Answer:(X"PERT(has(be[er(precision(and(recall)

Experimental(Data(&(Tool9

•  Release o Experiments (Crawl Data + Reports) o Layout testing algorithm implementation

http://gatech.github.io/xpert

Related(Work9o  Industrial tools

•  Test Suites for Web Browsers – Acid and test262

•  MS Expression Web, BrowserShots, BrowserStack

•  Browsera, MogoTest, BrowserBite

o  Research techniques

•  Eaton & Memon [IJWET’07]

•  Dallmeier, Burger, Orth & Zeller [JSTools’12]

Contributions(9

Empirical(evaluation9shows(effectiveness9

New(Structural(XBI9Detection(Algorithm9

bodyheader

footer

subHeading

countText

paperListpaper1

paper2

paper3

paper4

button1

button2

leftOf

leftOftop-align

bottom-align

Comprehensive(9Technique(to(detect(XBIs9

Web$Application.

My Awesome Site

Model$Generation. Model$Comparison.

Error$Report.

Study(of(Real(World(XBIs9

XBI$Distribu,on$Behavior$ 9%$Content$ Text$ 22%$

Visual$ 30%$Structure$ 57%$

X-PERT: Accurate Identification of Cross-Browser Issues in Web Applications

Technology

Pert/Cpm. กับการวางแผนและควบค ุมงานPERT 3. ประโยชน ของ CPM และ PERT 4. ข อจ าก ดของ CPM

Pert Heritage

Pert Master

Browser oh browser browser

Ejercicios PERT

Project Scheduling and Gantt Chartskulkarni/teaching/EECS541/slides/gantt.pdf · •Proven less accurate than CPM and PERT ... •Special version of bar chart –Henry Gantt around

Postsecondary Education Readiness Test (PERT) PERT 2013 1

Browserbite: Accurate Cross-Browser Testing via Machine Learning Over Image Features

Pert Trabajo

PERT - CPM

Redes Pert

Publicacion Ejercicios CPM, PERT y PERT-COST

X-PERT: A Web Application Testing Tool for Cross-Browser ...shauvik.com/public/pubs/roychoudhary14issta_tool.pdf · X-PERT: A Web Application Testing Tool for Cross-Browser Inconsistency

Elaboración PERT

Metoda PERT

Pert cours

nanlyhsu.weebly.com · 2018. 9. 9. · 552 , r ( , 1986) o view Technique; PERT)ù Path Method; CPM) , PERT 7. : PERT ' PERT , 2000)

Pert Estimating

Math x Pert Demoqewqweqweqweqweqweqweqweqweq qweqweqweqweqweqweqweqweqAMath x Pert Demoqewqweqweqweqweqweqweqweqweq qweqweqweqweqweqweqweqweqAMath x Pert Demoqewqweqweqweqweqweqweqweqweq

X-PERT: Accurate Identification of Cross-Browser Issues in Web Applications

￼X-PERT: Accurate Identification of Cross-Browser Issues in Web Applications

X-PERT: Accurate Identification of Cross-Browser Issues in Web Applications