Live API Documentationrtholmes/baker/2014-ICSE_talk.pdf · 2014. 6. 5. · Live%APIDocumentaon%2%Siddharth%Subramanian% 2 • 21,696%Classes% • 179,408%Methods% • 3,463%Classes%

Live API Documentation

Siddharth Subramanian Laura Inozemtseva

Reid Holmes

Live API Documenta1on -‐ Siddharth Subramanian 2

•  21,696 Classes •  179,408 Methods

•  3,463 Classes •  30,420 Methods

•  307 Proper1es


•  1.5 million users and 11 million posts. •  Ques1ons are answered in a median 1me of 11 minutes.

[Mamykina et al., CHI ‘11]


Source Code

API DOCUMENTATION


Fully Qualified Names (FQN)

API DOCUMENTATION

Declara1on Ambiguity


Missing declara1on statement.

External Reference Ambiguity


Missing import statements.

895 •  org.jfree.chart.axis.LogAxis.setBase(double) •  net.sf.jsog.client.UrlBuilder.setBase(java.lang.String) •  android.widget.Chronometer.setBase(long) •  …

9

15,352 •  java.lang.Thread.start() •  com.mysql.jdbc.u1l.ServerController.start() •  android.widget.Chronometer.start() •  com.jpa]ern.shared.u1l.Chronometer.start() •  …

Overcoming Ambiguity

1 •  android.widget.Chronometer


Overcoming Ambiguity

Baker (deduc1ve linker -‐ Java and JavaScript)

Snippet FQN

Annotated Snippet

Oracle (API

Signatures)


Java : •  Type signatures •  Method signatures •  Inheritance informa1on

JavaScript : •  Func1on object

signatures

Java : •  1.5 Million types •  16 Million methods

JavaScript: •  1,600 Func1on objects

public class Quotes extends Activity {

@Override

public void onCreate( Bundle savedInstanceState) { super.onCreate(savedInstanceState); webview.getSettings( ).setJavaScriptEnabled(true); webview.setWebViewClient(new MyWebViewClient(this)); webview.loadUrl(“…");

... }

}


Poten1al API references in the snippet.


@Override


... }

}


Iden1fying local declara1ons


@Override


... }

}

44

0

37


0

44 37

Inferring types


@Override


... }

}


0

android.app.AcSvity.onCreate(android.os.Bundle)

1 1

44


1 1

Inferring overridden methods

37


@Override


... }

}


0

1

1


android.app.AcSvity.onCreate(android.os.Bundle) android.app.AcSvity.onCreate(android.os.Bundle)

Inferring method invoca1ons

44 37


@Override


... }

}


0

1

1



404 404 Inferring method invoca1ons

37

44

106 106 Return types Return types


@Override


... }

}


0

1

1



404 5 5 Inferring method invoca1ons

44 37

106 106 5

Return types


@Override


... }

}


0

1

1



404 5 4 4 Inferring method

invoca1ons

37

44

106 5

Return types


@Override


... }

}


0

1

1



5

2 2 Inferring method

invoca1ons

44 37

404 4

106 5

Return types

5 1


@Override


... }

}


0

1

1



5

2 2 1 1 1 Constraint consistency check

44 37

404

1

106 4 android.webkit.WebView

5

Return types


@Override


... }

}


android.app.AcSvity

android.os.Bundle

android.webkit.WebView

1

System Unique Relaxed

True Posi1ves

False Posi1ves

False Nega1ves

False Nega1ves

Android

GWT

Hibernate

Joda Time

Xstream

Total

Baker Accuracy : Java Precision: 0.98 Recall: 0.83


An API reference is a §  Unique •  True Posi1ve: Baker correctly iden1fied the target API. •  False Posi1ve: Baker incorrectly iden1fied a target API. •  False Nega1ve: Baker either could not correctly iden1fy the right target API or it produced an imprecise match. §  Relaxed •  False Nega1ve: Baker could not iden1fy the target API even imprecisely.


True Posi1ves

False Posi1ves

False Nega1ves

False Nega1ves

Android 40 1 9 1

GWT 43 0 7 0

Hibernate

Joda Time

Xstream

Total


True Posi1ves

False Posi1ves

False Nega1ves

False Nega1ves

Android 40 1 9 1

GWT 43 0 7 0

Hibernate 37 0 13 0

Joda Time 44 3 3 0

Xstream 40 0 10 0

Total 204 4 42 1

Baker Accuracy : JavaScript


True Posi1ves

False Posi1ves

False Nega1ves

False Nega1ves

JSCore/DOM 48 2 0 0

JQuery 47 2 1 0

Phonegap 46 2 2 0

Webworks 43 0 7 2

Total 184 6 10 2

Precision: 0.97 Recall: 0.96


Cardinality Distribu1on


Cardinality = 1 : 69% Cardinality < 5 : 89%

Num

ber o

f API usage instances

Cardinality

Imprecise Matches

Imprecise matches in the AbstractDocumentReader class belonging to the xstream Java library: •  com.thoughtworks.xstream.io.xml.AbstractDocumentReader •  com.cloudbees.shaded.thoughtworks.xstream.io.xml.AbstractDocumentReader •  com.ovea.je]y.session.internal.xstream.io.xml.AbstractDocumentReader •  cucumber.run1me.xstream.io.xml.AbstractDocumentReader •  org.pitest.xstream.io.xml.AbstractDocumentReader


Enabling Live API Documenta1on (Demo)


Querying Baker


$ cat JavaSnippet.txt final Chronometer chrono = (Chronometer)findViewById(R.id.chronometer); chrono.setBase(SystemClock.elapsedRealTime()); chrono.start();

$ curl --data "file = @JavaSnippet.txt" http://gadget.cs.uwaterloo.ca/Baker/getAPI.php

{

"api_elements" : [ ...

{ "precision" : "1", "name" : "setBase", "line_number" : "2", "type" : "api_method", "elements" : [ "android.widget.Chronometer.setBase(long)"

] } ...

]}

Summary

•  DeducSve Linking can iden1fy API references in source code snippets with high precision.

•  This helps enable “Live DocumentaSon” by bridging interac1ve Stack Overflow posts with authorita1ve API documenta1on.

•  Baker and related resources can be accessed at: h]ps://cs.uwaterloo.ca/~rtholmes/baker


References •  [1] Mamykina, Lena and Manoim, Bella and Mi]al, Manas and Hripcsak, George

and Hartmann, Bjorn. Design Lessons from the Fastest Q&A Site in the West.


Addi1onal Info



•  241 K posts •  121 K accepted answers •  89 K code snippets (in accepted answers)

•  277 K posts •  110 K accepted answers •  69 K code snippets (in accepted answers)

Challenges

In accepted answers tagged Java/JavaScript:

•  No. of snippets > 3 LOC : 159, 531

•  Median size : 9 LOC •  Mean size : 14.33 LOC


Method Name No. of Collisions

No. of Packages

onCreate 32 15

show 17 2

getResources 9 6

getContentResolver 4 2

getItemId 7 2

getAction 6 3

findViewById 5 3

getString 34 12

getData 12 9

close 137 36

Average 26.3 9.0

Naming Collisions among the 10 most-‐used Android methods.


Architecture


Oracle


Java : •  1.5 Million Classes •  16 Million Methods

JavaScript: •  1,600 Proper1es

Evalua1on

•  RQ1: Does Baker accurately iden1fy fully-‐qualified API references in code snippets?

•  RQ2: Is Baker robust and able to resolve references on a variety of libraries?


Example Diversity : Java System No. of Classes

(Total/Unique) No. of Methods (Total/Unique)

Android Apache Eclipse GWT Hibernate JDK Other Total


System No. of Classes (Total/Unique)

No. of Methods (Total/Unique)

Android 272 / 64 175 / 104 Apache 178 / 79 108 / 97 Eclipse 104 / 41 53 / 45 GWT 149 / 47 122 / 69 Hibernate 389 / 133 378 / 199 JDK 14,252 / 632 7,483 / 1,981 Other 5,956 / 487 1,339 / 747 Total 21,300 / 1,483 9,658 / 3,242

Example Diversity : JavaScript

System No. of ProperSes (Total/Unique)

JSCore/DOM 6,467/107

JQuery 1,793/96

Phonegap 126/27

Webworks 244/52

Other 1,297/300

Total 9,927/582


The Baker Framework


Quan1fying Naming Collisions in the Java Oracle

Types Methods Fields Total Average

FQN 1,646,650 14,206,944 3,149,206 19,002,800 -‐

PQN -‐ 9,455,644 2,571,384 12,027,028 -‐

UN 1,121,887 1,600,053 1,115,099 3,837,039 -‐

% ambiguous PQN -‐ 33% 37% -‐ 37%

% ambiguous UN 32% 89% 65% -‐ 80%

FQN : Fully Qualified Name (e.g., android.widget.Chronometer.start()) PQN : Par1ally Qualified Name (e.g., Chronometer.start()) UN : Unqualified Name (e.g., start())


Extensions

•  Heuris1cs to predict libraries imported.


public class Quotes extends Activity implements View.OnClickListener { @override


}

@override public void onClick( View v) { }

}

Extensions

•  U1lizing metadata wherever available.


Criteria crit = session.createCriteria(Person.class); crit.add( Restrictions.isNotNull("birthDate")); crit.add( Restrictions.eq("isStudent", true)); List students = crit.list(); Integer count = students.size();

Extensions

•  Pre-‐processing to re-‐order constraints and reduce running 1me.


public FirstPanel() { History.addHistoryListener(this); String token = History.getToken(); if (token.length() == 0) { History.newItem(INIT_STATE); } else { History.fireCurrentHistoryState(); }

}

34

62

3

Related Work

•  ACE – Does not need an oracle. – Cannot iden1fy FQNs (only PQNs).

•  RecoDoc – Specific target libraries. – Cannot resolve external references.


A mock-‐up of what could be achieved with an effec1ve traceability linking strategy across a

number of informa1on sources.

Traceability Link Recovery


Documents

Live API Documentationrtholmes/baker/2014-ICSE_talk.pdf · 2014. 6. 5. · Live%APIDocumentaon%2%Siddharth%Subramanian% 2 • 21,696%Classes% • 179,408%Methods% • 3,463%Classes%