Automated Historical Performance Analysis with kmemtracer

Automated Historical

Performance Analysis

with kmemtracer

LG전자 이경민

2 12th Kandroid Conference - www.kandroid.org

Why Automated Historical Performance Testing?

kmemtracer: What and How?

Enhancement I : Automating UI Interaction

Enhancement II: Adding More Performance Metrics

Contents

If you can’t measure it, you can’t manage it.

Peter F. Drucker (1909~2005)

If you can’t measure it, you can’t manage it.

Peter F. Drucker (1909~2005)

If you can’t automate the measurement of it,

it is difficult to improve it.

http://developer.android.com/about/dashboards/index.html

Platform Versions

Diversity or Fragmentation

http://developer.android.com/about/dashboards/index.html

Platform Versions

Screen Sizes and Densities

Diversity or Fragmentation

http://www.mactrast.com/2012/05/visualized-the-truth-about-android-fragmentation/

Why Historical Analysis?

Source: Memory and Performance at the 11th Korea Android Conference

Historical Performance Analysis

Activity Navigation & Lifecycle

How to Track Activity Navigation & Lifecycle?

Override onCreate, onStart, … of your Activity classes

Modify android.app.Activity

in the framework

Is there a way that does not require the modification of

an application or framework?

Is there a way that does not require the modification of

an application or framework?

Instrumentation

Android instrumentation is a set of control methods or "hooks" in the Android system. These hooks control an Android component independently of its normal lifecycle. They also control how Android loads applications. (http://developer.android.com/tools/testing/testing_android.html)

Contents

Historical Analysis w/ kmemtracer

Test Package

Application Package

android.test. Instrumentation

TestRunner

Control

Historical Analysis w/ kmemtracer

https://github.com/snailee/kmemtracer-libs

Test Package

Application Package

TestRunner

org.kandroid. memtracer.Memory

Instrumentation

Control Track

Trace Package

android.app.Instrumentation

void callActivityOnCreate(Activity, Bundle)

void callActivityOnDestroy(Activity)

void callActivityOnNewIntent(Activity, Intent)

void callActivityOnPause(Activity)

void callActivityOnPostCreate(Activity, Bundle)

void callActivityOnRestart(Activity)

void callActivityOnRestoreInstanceState(Activity, Bundle)

void callActivityOnResume(Activity)

void callActivityOnSaveInstanceState(Activity, Bundle)

void callActivityOnStart(Activity)

void callActivityOnStop(Activity)

void callActivityOnUserLeaving(Activity)

void callApplicationOnCreate(Application)

Call for Control

Override for Track

org.kandroid.memtracer.MemoryInstrumentation

public class MemoryInstrumentation extends Instrumentation { private static final String ARGUMENT_INSTRUMENT_CLASS = "class"; private MemoryTracer mMemoryTracer; private String mMainActivityName; ... @Override public void onCreate(Bundle arguments) { mMainActivityName = arguments.getString(ARGUMENT_INSTRUMENT_CLASS); mMemoryTracer = createMemoryTracer(); mMemoryTracer.startTracing(getTargetContext().getPackageName()); start(); } @Override public void callActivityOnCreate(Activity activity, Bundle icicle) { String tag = activity.getLocalClassName()+"-OnCreate"; Bundle snapshot = mMemoryTracer.addSnapshot(tag); super.callActivityOnCreate(activity, icicle); } }

Tracing Performance Snapshots in kmemtracer

public class MemoryInstrumentation extends Instrumentation { ... ... }

public class MemoryInstrumentation extends Instrumentation { ... private MemoryTracer mMemoryTracer; ... @Override public void onCreate(Bundle arguments) { ... mMemoryTracer = createMemoryTracer(); ... } ... ... }

Create an instance of MemoryTracer

public class MemoryInstrumentation extends Instrumentation { ... private MemoryTracer mMemoryTracer; ... @Override public void onCreate(Bundle arguments) { ... mMemoryTracer = createMemoryTracer(); ... } ... @Override public void callActivityOnCreate(Activity activity, Bundle icicle) { String tag = activity.getLocalClassName()+"-OnCreate"; Bundle snapshot = mMemoryTracer.addSnapshot(tag); ... } ... }

Create an instance of MemoryTracer

Add performance snapshot

MemoryTracer.addSnapshot

public Bundle addSnapshot(String label) { ...

public Bundle addSnapshot(String label) { ... long nativeMax = Debug.getNativeHeapSize() / 1024; long nativeAllocated = Debug.getNativeHeapAllocatedSize() / 1024; long nativeFree = Debug.getNativeHeapFreeSize() / 1024; Debug.MemoryInfo memInfo = new Debug.MemoryInfo(); Debug.getMemoryInfo(memInfo); ...

Get usage data

public Bundle addSnapshot(String label) { ... long nativeMax = Debug.getNativeHeapSize() / 1024; long nativeAllocated = Debug.getNativeHeapAllocatedSize() / 1024; long nativeFree = Debug.getNativeHeapFreeSize() / 1024; Debug.MemoryInfo memInfo = new Debug.MemoryInfo(); Debug.getMemoryInfo(memInfo); ... Bundle snapshot = new Bundle(); snapshot.putString(METRIC_KEY_LABEL, label); ... snapshot.putLong(METRIC_KEY_NATIVE_SIZE, nativeMax); snapshot.putLong(METRIC_KEY_NATIVE_ALLOCATED, nativeAllocated); snapshot.putLong(METRIC_KEY_NATIVE_FREE, nativeFree); ...

Get usage data

Save data to a bundle

public Bundle addSnapshot(String label) { ... long nativeMax = Debug.getNativeHeapSize() / 1024; long nativeAllocated = Debug.getNativeHeapAllocatedSize() / 1024; long nativeFree = Debug.getNativeHeapFreeSize() / 1024; Debug.MemoryInfo memInfo = new Debug.MemoryInfo(); Debug.getMemoryInfo(memInfo); ... Bundle snapshot = new Bundle(); snapshot.putString(METRIC_KEY_LABEL, label); ... snapshot.putLong(METRIC_KEY_NATIVE_SIZE, nativeMax); snapshot.putLong(METRIC_KEY_NATIVE_ALLOCATED, nativeAllocated); snapshot.putLong(METRIC_KEY_NATIVE_FREE, nativeFree); ... if (mResultsWriter != null) { mResultsWriter.writeTraceSnapshot(snapshot); } return snapshot; }

Get usage data

Write date using ResultsWriter

Save data to a bundle

MemoryTracerCsvWriter$writeTraceSnapshot

public class MemoryTraceCsvWriter implements MemoryTracer.ResultsWriter { ...

public class MemoryTraceCsvWriter implements MemoryTracer.ResultsWriter { ... public MemoryTraceCsvWriter(String[] metricKeys) { mMetricKeys = metricKeys; } ...

Create an instance with metrics to print

public class MemoryTraceCsvWriter implements MemoryTracer.ResultsWriter { ... public MemoryTraceCsvWriter(String[] metricKeys) { mMetricKeys = metricKeys; } ... private void openTraceFile(String filename) { ... mTraceOut = new PrintStream(new FileOutputStream(traceFile)); ... } ...

Open a trace file in the sdcard at init

public class MemoryTraceCsvWriter implements MemoryTracer.ResultsWriter { ... public MemoryTraceCsvWriter(String[] metricKeys) { mMetricKeys = metricKeys; } ... private void openTraceFile(String filename) { ... mTraceOut = new PrintStream(new FileOutputStream(traceFile)); ... } ... @Override public void writeTraceSnapshot(Bundle snapshot) { PrintStream out = mTraceOut; for (String key : mMetricKeys) { out.print(snapshot.get(key)); out.print(','); } out.println(); out.flush(); } ... }

Write snapshot to a line of the file

Open a trace file in the sdcard at init

How to Use kmemtracer?

1. Create an Android Test Project for the trace package.

2. Add ‘kmemtracer-libs.jar’ in the ‘libs’ directory.

3. Edit the <instrumentation> element in the Manifest file.

4. Install the trace package as well as the application package.

$ adb install –r ApiDemos.apk $ adb install –r ApiDemosTest.apk

5. Start the instrumentation with ‘am instrument’ in the shell.

$ adb shell am instrument -e class \ > com.example.android.apis.ApiDemos \ > com.example.android.apis.tests/\ > org.kandroid.memtracer.MemoryInstrumentation

6. Do interact with the application.

7. Pull the trace file ‘kmemtrace.csv’ from /sdcard/kmemtracer.

$ adb pull /sdcard/kmemtracer/kmemtrace.csv .

8. Open the trace file with Excel and create charts.

Contents

Automate Controlling UI

Test Package

Application Package

TestRunner

Instrumentation

Control Track

Trace Package

Test Package

Application Package

TestRunner

Instrumentation

Control Track

Trace Package

Test Package

Application Package

TestRunner

Instrumentation

Control Track

Trace Package

Three Alternatives

Instrumentation

API Level 1 ~ Single activity Framework APIs White-box

http://developer.android.com/tools/testing/testing_android.html

Three Alternatives

Instrumentation Robotium

2010/01 ~ Multiple activities High-level APIs White-box & black-box

http://developer.android.com/tools/testing/testing_android.html https://code.google.com/p/robotium/

Three Alternatives

Instrumentation Robotium uiautomator

2010/01 ~ Multiple activities High-level APIs White-box & black-box

API Level 17 ~ Multiple processes Simple APIs Black-box

http://developer.android.com/tools/testing/testing_android.html https://code.google.com/p/robotium/ http://developer.android.com/tools/testing/testing_ui.html

Case Study: ApiDemos

https://github.com/snailee/deview-2013-samples

Scroll down

Click Click

Scroll down

Click Click

Controlling UI Using Instrumentation

1. Create an Android Test Project for the test package.

$ android create test-project --main ../ApiDemos \ > –-name ApiDemosInstrumentTest --path ApiDemosInstrumentTest

2. Add a test class inheriting ActivityInstrumentationTestCase2.

public class ApiDemosInstrumentTest extends ActivityInstrumentationTestCase2<ApiDemos> { ... public ApiDemosInstrumentTest() { super(ApiDemos.class); }

3. Add a test method and launch the activity.

public class ApiDemosInstrumentTest extends ActivityInstrumentationTestCase2<ApiDemos> { ... public ApiDemosInstrumentTest() { super(ApiDemos.class); }

public void testNavigate() { final ApiDemos apiDemos = getActivity(); ... }

public void testNavigate() { ...

4. Find and act on a control.

public void testNavigate() { ... final ListView demoList = apiDemos.getListView(); final View graphicsDemo = getChildViewByText(demoList, "Graphics"); assertNotNull("Graphics demo should exist", graphicsDemo);

Find a view to control

public void testNavigate() { ... final ListView demoList = apiDemos.getListView(); final View graphicsDemo = getChildViewByText(demoList, "Graphics"); assertNotNull("Graphics demo should exist", graphicsDemo); apiDemos.runOnUiThread(new Runnable() { public void run() { int pos = demoList.getPositionForView(graphicsDemo); demoList.smoothScrollToPosition(pos); demoList.setSelection(pos); demoList.performItemClick(graphicsDemo, pos, graphicsDemo.getId()); } }); ...

Act on the view

public void testNavigate() { ... final ListView demoList = apiDemos.getListView(); final View graphicsDemo = getChildViewByText(demoList, "Graphics"); assertNotNull("Graphics demo should exist", graphicsDemo); apiDemos.runOnUiThread(new Runnable() { public void run() { int pos = demoList.getPositionForView(graphicsDemo); demoList.smoothScrollToPosition(pos); demoList.setSelection(pos); demoList.performItemClick(graphicsDemo, pos, graphicsDemo.getId()); } }); ...

Act on the view

private View getChildViewByText(ListView listView, String text) {

4. Find and act on a control: getChildViewByText.

private View getChildViewByText(ListView listView, String text) { View view = null; int count = listView.getCount(); Log.e(TAG, “no. children = " + count); for (int i = 0; i < count; i++) { TextView textView = (TextView) listView.getChildAt(i);

Iterate through child views

private View getChildViewByText(ListView listView, String text) { View view = null; int count = listView.getCount(); Log.e(TAG, “no. children = " + count); for (int i = 0; i < count; i++) { TextView textView = (TextView) listView.getChildAt(i); Log.e(TAG, i + "-th text view is " + textView); if (textView != null && textView.getText().equals(text)) { view = textView; break; } } return view; }

Compare texts

private View getChildViewByText(ListView listView, String text) { View view = null; int count = listView.getCount(); Log.e(TAG, “no. children = " + count); for (int i = 0; i < count; i++) { TextView textView = (TextView) listView.getChildAt(i); Log.e(TAG, i + "-th text view is " + textView); if (textView != null && textView.getText().equals(text)) { view = textView; break; } } return view; }

Compare texts

com.example.android.apis.instrument.test.ApiDemosInstrumentTest: INSTRUMENTATION_RESULT: shortMsg=junit.framework.AssertionFailedError INSTRUMENTATION_RESULT: longMsg= junit.framework.AssertionFailedError: OpenGL ES demo should exist INSTRUMENTATION_CODE: 0

private View getChildViewByText(final ListView listView, String text) { ... for (int i = 0; i < count; i++) { TextView textView = (TextView) listView.getChildAt(i); if (textView != null && textView.getText().equals(text)) { view = textView; break; } if (textView == null) { final int lastPos = listView.getLastVisiblePosition(); getInstrumentation().runOnMainSync(new Runnable() { public void run() { listView.smoothScrollToPosition(lastPos + 5, lastPos); } }); getInstrumentation().waitForIdleSync(); i = listView.getFirstVisiblePosition(); } } return view; }

4. Find and act on a control: Revised getChildViewByText.

private View getChildViewByText(final ListView listView, String text) { ... for (int i = 0; i < count; i++) { TextView textView = (TextView) listView.getChildAt(i); if (textView != null && textView.getText().equals(text)) { view = textView; break; } if (textView == null) { final int lastPos = listView.getLastVisiblePosition(); getInstrumentation().runOnMainSync(new Runnable() { public void run() { listView.smoothScrollToPosition(lastPos + 5, lastPos); } }); getInstrumentation().waitForIdleSync(); i = listView.getFirstVisiblePosition(); } } return view; }

Scroll down

4. Find and act on a control: Revised getChildViewByText.

private int getPositionForChildViewByText(ListView listView, String text) { ListAdapter listAdapter = listView.getAdapter(); int count = listAdapter.getCount(); Log.e(TAG, "children: " + count); for (int i = 0; i < count; i++) { String itemText = (String) ((Map)listAdapter.getItem(i)).get("title"); Log.e(TAG, i + "-th text item " + itemText); if (itemText != null && itemText.equals(text)) { return i; } } return -1; }

4. Find and act on a control: getPositionForChildViewByText.

... final ListView graphicsDemoList = graphicsDemoActivity.getListView(); int pos = getPositionForChildViewByText(graphicsDemoList, "OpenGL ES"); assertTrue("Cannot find OpenGL ES", pos != -1); ...

private int getPositionForChildViewByText(ListView listView, String text) { ListAdapter listAdapter = listView.getAdapter(); int count = listAdapter.getCount(); Log.e(TAG, "children: " + count); for (int i = 0; i < count; i++) { String itemText = (String) ((Map)listAdapter.getItem(i)).get("title"); Log.e(TAG, i + "-th text item " + itemText); if (itemText != null && itemText.equals(text)) { return i; } } return -1; }

4. Find and act on a control: getPositionForChildViewByText.

... final ListView graphicsDemoList = graphicsDemoActivity.getListView(); int pos = getPositionForChildViewByText(graphicsDemoList, "OpenGL ES"); assertTrue("Cannot find OpenGL ES", pos != -1); ...

5. Monitor a new Activity to launch.

public void testNavigate() { ... Instrumentation.ActivityMonitor graphicsDemoActivityMonitor = getInstrumentation().addMonitor( ApiDemos.class.getName(), null, false); final ApiDemos graphicsDemoActivity = (ApiDemos) graphicsDemoActivityMonitor.waitForActivity(); assertNotNull("Graphics Demo (ApiDemos) activity should not be null", graphicsDemoActivity); ...

5. Monitor a new Activity to launch.

6. Iterate step 4 & 5.

public void testNavigate() { ... Instrumentation.ActivityMonitor graphicsDemoActivityMonitor = getInstrumentation().addMonitor( ApiDemos.class.getName(), null, false); final ApiDemos graphicsDemoActivity = (ApiDemos) graphicsDemoActivityMonitor.waitForActivity(); assertNotNull("Graphics Demo (ApiDemos) activity should not be null", graphicsDemoActivity); ...

7. Build and Install the test package.

$ cd ApiDemosInstrumentTest $ ant debug $ adb install –r bin/ApiDemosInstrumentTest-debug.apk

8. Run the test and see the result.

$ adb shell am instrument -w \ > com.example.android.apis.instrument.test/\ > android.test.InstrumentationTestRunner

$ ant test

Controlling UI Using Robotium

$ android create test-project --main ../ApiDemos \ > –-name ApiDemosRobotiumTest --path ApiDemosRobotiumTest

2. Add ‘robotium-solo-<ver>.jar’ in the ‘libs’ directory.

public class ApiDemosRobotiumTest extends ActivityInstrumentationTestCase2<ApiDemos> { ... public ApiDemosRobotiumTest() { super(ApiDemos.class); } ...

4. Create an instance of Solo in the setUp method and

close open activities in the tearDown method.

... private Solo solo; ... @Override public void setUp() throws Exception { solo = new Solo(getInstrumentation(), getActivity()); } @Override public void tearDown() throws Exception { solo.finishOpenedActivities(); } ...

public void testNavigate() { solo.assertCurrentActivity("Should be on ApiDemos", ApiDemos.class); solo.clickOnText("^Graphics$", 1, true); solo.assertCurrentActivity("Should be on a new ApiDemos", ApiDemos.class, true); solo.clickOnText("^OpenGL ES$", 1, true); solo.assertCurrentActivity("Should be on a new ApiDemos", ApiDemos.class, true); solo.clickOnText("^Kube$", 1, true); solo.assertCurrentActivity("Should be on a Kube", Kube.class, true); solo.sleep(3000); }

$ cd ApiDemosRobotiumTest $ ant debug $ adb install –r bin/ApiDemosRobotiumTest-debug.apk

$ adb shell am instrument -w \ > com.example.android.apis.robotium.test/\ > android.test.InstrumentationTestRunner

$ ant test

Controlling UI Using uiautomator

1. Create an Java Project for the test jar.

2. Add ‘android.jar’ and ‘uiautomator.jar’ in the build path.

3. Add a test class inheriting UiAutomatorTestCase.

public class ApiDemosUiAutoTest extends UiAutomatorTestCase { ... }

4. Launch ApiDemos.

public void testNavigate() throws UiObjectNotFoundException {

4. Launch ApiDemos.

public void testNavigate() throws UiObjectNotFoundException { getUiDevice().pressHome();

Press home button

4. Launch ApiDemos.

public void testNavigate() throws UiObjectNotFoundException { getUiDevice().pressHome(); UiObject allAppsButton = new UiObject( new UiSelector().description("Apps")); allAppsButton.clickAndWaitForNewWindow();

Click ‘All Apps’ button

Press home button

4. Launch ApiDemos.

public void testNavigate() throws UiObjectNotFoundException { getUiDevice().pressHome(); UiObject allAppsButton = new UiObject( new UiSelector().description("Apps")); allAppsButton.clickAndWaitForNewWindow(); UiObject appsTab = new UiObject(new UiSelector().text("Apps")); appsTab.click();

Press home button

Click ‘Apps’ tab

4. Launch ApiDemos.

public void testNavigate() throws UiObjectNotFoundException { getUiDevice().pressHome(); UiObject allAppsButton = new UiObject( new UiSelector().description("Apps")); allAppsButton.clickAndWaitForNewWindow(); UiObject appsTab = new UiObject(new UiSelector().text("Apps")); appsTab.click(); UiScrollable appList = new UiScrollable( new UiSelector().scrollable(true)); appList.setAsHorizontalList(); UiObject apiDemos = appList.getChildByText( new UiSelector().className(TextView.class.getName()), "API Demos"); apiDemos.clickAndWaitForNewWindow();

Press home button

Click ‘ApiDemos’ icon

4. Launch ApiDemos.

public void testNavigate() throws UiObjectNotFoundException { getUiDevice().pressHome(); UiObject allAppsButton = new UiObject( new UiSelector().description("Apps")); allAppsButton.clickAndWaitForNewWindow(); UiObject appsTab = new UiObject(new UiSelector().text("Apps")); appsTab.click(); UiScrollable appList = new UiScrollable( new UiSelector().scrollable(true)); appList.setAsHorizontalList(); UiObject apiDemos = appList.getChildByText( new UiSelector().className(TextView.class.getName()), "API Demos"); apiDemos.clickAndWaitForNewWindow(); UiObject apiDemosPackage = new UiObject( new UiSelector().packageName("com.example.android.apis")); assertTrue("Should be on ApiDemos", apiDemosPackage.exists()); ...

Press home button

Click ‘ApiDemos’ icon

Validate the launch of ‘ApiDemos’

... UiScrollable demoList = new UiScrollable( new UiSelector().className("android.widget.ListView")); UiObject graphicsDemo = demoList.getChildByText( new UiSelector().className(TextView.class.getName()), “Graphics"); graphicsDemo.clickAndWaitForNewWindow(); UiObject alphaBitmapDemo = new UiObject( new UiSelector().text(“AlphaBitmap")); assertTrue(“AlphaBitmap should be visible", alpahBitmapDemo.exists()); ...

6. Iterate step 5.

... UiScrollable demoList = new UiScrollable( new UiSelector().className("android.widget.ListView")); UiObject graphicsDemo = demoList.getChildByText( new UiSelector().className(TextView.class.getName()), “Graphics"); graphicsDemo.clickAndWaitForNewWindow(); UiObject alphaBitmapDemo = new UiObject( new UiSelector().text(“AlphaBitmap")); assertTrue(“AlphaBitmap should be visible", alpahBitmapDemo.exists()); ...

7. Compile your test case into a JAR file.

$ android create uitest-project --name ApiDemosUiAutoTest \ > --target 1 --path ApiDemosUiAutoTest $ cd ApiDemosUiAutoTest $ ant build

8. Install the jar on your test device.

$ adb push bin/ApiDemosUiAutoTest.jar /data/local/tmp/

9. Run the test and view the test result.

$ adb shell uiautomator runtest ApiDemosUiAutoTest.jar \ > –c com.example.android.apis.uiauto.test.ApiDemosUiAutoTest

$ ant test

Doing both Controlling and Tracking

1. Modify org.kandroid.memtracer.MemoryInstrumentation.

public class MemoryInstrumentation extends InstrumentationTestRunner { ... @Override public void onCreate(Bundle arguments) { ... start(); super.onCreate(arguments); } @Override public void onStart() { String mainActivityName = getMainActivityName(); if (mainActivityName != null && mainActivityName.length() > 0) { launchMainActivity(getTargetContext().getPackageName(), getMainActivityName()); } else { super.onStart(); } } ...

3. Edit the <instrumentation> element in the manifest file.

$ adb shell am instrument -w \ > com.example.android.apis.instrument.test/\ > org.kandroid.memtracer.MemoryInstrumentation

6. Pull the trace file ‘kmemtrace.csv’ from /sdcard/kmemtracer.

$ adb shell am instrument -w \ > com.example.android.apis.instrument.test/\ > org.kandroid.memtracer.MemoryInstrumentation

$ adb pull /sdcard/kmemtracer/kmemtrace.csv .

Instrument Robotium uiautomator

Compatibility O O X

Multi-activities O O

Multi-processes X X O

Ease of APIs X O

Black box X O

Comparison

Contents

What to Measure?

Rendering (or GUI) Performance Frame Rate (FPS), Frame Time , Jankiness, Input Latency, …

Resource Consumption Memory, CPU, I/O (Disk, Network), Battery, …

Network Performance Response time, Throughput, … …

What to Measure?

Rendering (or GUI) Performance Frame Rate (FPS), Frame Time , Jankiness, Input Latency, …

Resource Consumption Memory, CPU, I/O (Disk, Network), Battery, …

Network Performance Response time, Throughput, … …

How to Measure Resource Consumption?

API cmds

Public Internal

API cmds

Public Internal Android Linux

API cmds

Easy Flexible

Measuring Memory Usage

Public

android.app.ActivityManager.MemoryInfo android.os.Debug android.os.Debug.MemoryInfo

Public Internal

dalvik.system.VMDebug

Public Internal Android

adb shell dumpsys meminfo [pid|pname] adb shell dumpsys gfxinfo [pid|pname] adb shell procrank

adb shell cat /proc/meminfo adb shell cat /proc/<pid>/[s]maps adb shell cat /proc/vmstat adb shell top adb shell ps

$ adb shell dumpsys meminfo com.example.android.apis Applications Memory Usage (kB): Uptime: 93065 Realtime: 93065 ** MEMINFO in pid 1259 [com.example.android.apis] ** native dalvik other total size: 3548 5831 N/A 9379 allocated: 3475 3335 N/A 6810 free: 48 2496 N/A 2544 (Pss): 699 382 4889 5970 (shared dirty): 1500 1876 8356 11732 (priv dirty): 628 40 2472 3140 ...

GB (API Level 10)

Measuring Memory Usage: dumpsys meminfo

$ adb shell dumpsys meminfo com.example.android.apis Applications Memory Usage (kB): Uptime: 93065 Realtime: 93065 ** MEMINFO in pid 1259 [com.example.android.apis] ** native dalvik other total size: 3548 5831 N/A 9379 allocated: 3475 3335 N/A 6810 free: 48 2496 N/A 2544 (Pss): 699 382 4889 5970 (shared dirty): 1500 1876 8356 11732 (priv dirty): 628 40 2472 3140 ...

GB (API Level 10)

$ adb shell dumpsys meminfo com.example.android.apis ... ** MEMINFO in pid 1875 [com.example.android.apis] ** Shared Private Heap Heap Heap Pss Dirty Dirty Size Alloc Free ------ ------ ------ ------ ------ ------ Native 1377 1256 1300 12084 7332 567 Dalvik 5274 5476 4952 6260 5712 548 Stack 76 12 76 Cursor 0 0 0 Ashmem 0 0 0 Other dev 4 20 0 .so mmap 1526 4836 592 .jar mmap 0 0 0 .apk mmap 267 0 0 .ttf mmap 5 0 0 .dex mmap 1085 0 0 Other mmap 18 8 4 Unknown 824 0 824 TOTAL 10456 11608 7748 18344 13044 1115 ...

JB (API Level 18)

$ adb shell dumpsys meminfo com.example.android.apis ... ** MEMINFO in pid 1875 [com.example.android.apis] ** Shared Private Heap Heap Heap Pss Dirty Dirty Size Alloc Free ------ ------ ------ ------ ------ ------ Native 1377 1256 1300 12084 7332 567 Dalvik 5274 5476 4952 6260 5712 548 Stack 76 12 76 Cursor 0 0 0 Ashmem 0 0 0 Other dev 4 20 0 .so mmap 1526 4836 592 .jar mmap 0 0 0 .apk mmap 267 0 0 .ttf mmap 5 0 0 .dex mmap 1085 0 0 Other mmap 18 8 4 Unknown 824 0 824 TOTAL 10456 11608 7748 18344 13044 1115 ...

JB (API Level 18)

https://android.googlesource.com/platform/frameworks/base.git/+/0e3328fbdd3845b0e2bec364e951498eaee6b079

How ‘dumpsys meminfo’ works?

adb shell dumpsys meminfo [-a] [-oom] [-process]

com.android.server.am. ActivityManagerService

android.os Debug

android.app.ActivityThread. ApplicationThread

android.os Debug

/proc/<pid>/smaps

mallinfo()

getNativeHeap*() getMemoryInfo()

android.app.ActivityThread.ApplicationThread.dumpMemInfo()

long nativeMax = Debug.getNativeHeapSize() / 1024; long nativeAllocated = Debug.getNativeHeapAllocatedSize() / 1024; long nativeFree = Debug.getNativeHeapFreeSize() / 1024; Debug.MemoryInfo memInfo = new Debug.MemoryInfo(); Debug.getMemoryInfo(memInfo); Runtime runtime = Runtime.getRuntime(); long dalvikMax = runtime.totalMemory() / 1024; long dalvikFree = runtime.freeMemory() / 1024; long dalvikAllocated = dalvikMax - dalvikFree;

android.app.ActivityThread.ApplicationThread.dumpMemInfo()

long nativeMax = Debug.getNativeHeapSize() / 1024; long nativeAllocated = Debug.getNativeHeapAllocatedSize() / 1024; long nativeFree = Debug.getNativeHeapFreeSize() / 1024; Debug.MemoryInfo memInfo = new Debug.MemoryInfo(); Debug.getMemoryInfo(memInfo); Runtime runtime = Runtime.getRuntime(); long dalvikMax = runtime.totalMemory() / 1024; long dalvikFree = runtime.freeMemory() / 1024; long dalvikAllocated = dalvikMax - dalvikFree;

struct mallinfo info = mallinfo(); (jlong) info.usmblks; // Debug.getNativeHeapSize() (jlong) info.uordblks; // Debug.getNativeHeapAllocatedSize() (jlong) info.fordblks; // Debug.getNativeHeapFreeSize()

android.os.Debug.getNativeHeap*()

struct mallinfo { MALLINFO_FIELD_TYPE arena; /* non-mmapped space allocated from system */ MALLINFO_FIELD_TYPE ordblks; /* number of free chunks */ MALLINFO_FIELD_TYPE smblks; /* always 0 */ MALLINFO_FIELD_TYPE hblks; /* always 0 */ MALLINFO_FIELD_TYPE hblkhd; /* space in mmapped regions */ MALLINFO_FIELD_TYPE usmblks; /* maximum total allocated space */ MALLINFO_FIELD_TYPE fsmblks; /* always 0 */ MALLINFO_FIELD_TYPE uordblks; /* total allocated space */ MALLINFO_FIELD_TYPE fordblks; /* total free space */ MALLINFO_FIELD_TYPE keepcost; /* releasable (via malloc_trim) space */ };

bionic\libc\upstream-dlmalloc\malloc.c

① Total allocated = uordblks ② Total free = fordblks ③ Footprint = uordblks + fordblks ④ Max footprint = usmblks

Measuring CPU Usage

Public

android.os.Process android.os.Debug.InstructionCount

Measuring CPU Usage

Public Internal

com.android.internal.os.ProcessStats

Measuring CPU Usage

adb shell dumpsys cpuinfo

Measuring CPU Usage

adb shell cat /proc/stat adb shell cat /proc/<pid>/stat adb shell top

Measuring CPU Usage: dumpsys cpuinfo

$ adb shell dumpsys cpuinfo Load: 0.0 / 0.01 / 0.05 CPU usage from 27006ms to 5405ms ago: 1.1% 1421/com.android.launcher: 0.3% user + 0.8% kernel / faults: 2187 minor 2.4% 1225/system_server: 0.4% user + 1.9% kernel / faults: 2030 minor 0.1% 949/surfaceflinger: 0% user + 0.1% kernel / faults: 3 minor 0% 952/mediaserver: 0% user + 0% kernel / faults: 8 minor 0% 959/adbd: 0% user + 0% kernel / faults: 11 minor 0% 1295/com.android.systemui: 0% user + 0% kernel / faults: 5 minor +0% 1901/com.example.android.apis: 0% user + 0% kernel 56% TOTAL: 10% user + 45% kernel + 0.7% softirq

How ‘dumpsys cpuinfo’ works?

com.android.internal.os ProcessStats

com.android.internal.os ProcessStats.Stats

android.os Process

android_util_Process.cpp

/proc/stat /proc/loadavg /proc/<pid>/stat /proc/<ppid>/task/<pid>/stat

procfs (or the proc filesystem) is a special filesystem in UNIX-

like operating systems that presents information about processes

and other system information in a hierarchical file-like structure,

providing a more convenient and standardized method for

dynamically accessing process data held in the kernel than

traditional tracing methods or direct access to kernel memory.

http://en.wikipedia.org/wiki/Procfs

http://manpages.courier-mta.org/htmlman5/proc.5.html

procfs (or the proc filesystem) is a special filesystem in UNIX-

like operating systems that presents information about processes

and other system information in a hierarchical file-like structure,

providing a more convenient and standardized method for

dynamically accessing process data held in the kernel than

traditional tracing methods or direct access to kernel memory.

http://en.wikipedia.org/wiki/Procfs

Measuring I/O Usage

Public

android.net.TrafficStats

Measuring I/O Usage

Public Internal

android.net.NetworkStats android.net.NetworkStatsHistory com.android.internal.net.NetworkStatsFactory

Measuring I/O Usage

adb shell dumpsys netstats adb shell dumpsys diskstats

Measuring I/O Usage

adb shell dumpsys netstats adb shell dumpsys diskstats

/proc/net/*

Measuring Network IO Usage: dumpsys netstats

$ adb shell dumpsys netstats Active interfaces: iface=eth0 ident=[[type=MOBILE, subType=COMBINED, subscriberId=310260000000000]] Dev stats: Pending bytes: 43783 History since boot: ident=[[type=MOBILE_HIPRI, subType=COMBINED, subscriberId=310260000000000], [type=MOBILE, subType=COMBINED, subscriberId=310260000000000]] uid=-1 set=ALL tag=0x0 NetworkStatsHistory: bucketDuration=3600000 bucketStart=1382443200000 activeTime=4813 rxBytes=17410 rxPackets=203 txBytes=25579 txPackets=173 operations=0 ... Xt stats: Pending bytes: 37557 History since boot: ident=[[type=MOBILE_HIPRI, subType=COMBINED, subscriberId=310260000000000], [type=MOBILE, subType=COMBINED, subscriberId=310260000000000]] uid=-1 set=ALL tag=0x0 ... UID stats: ... UID tag stats: ...

adb shell dumpsys netstats [--full] [--uid] [--tag]

How ‘dumpsys netstats’ works?

com.android.server.net. NetworkStatsService

com.android.server.net NetworkStatsRecorder

android.net NetworkStatsHistory

com.android.server.net NetworkStatsCollection

com.android.internal.net NetworkStatsFactory

android.net NetworkStats

com.android.server.net NetworkStatsCollection

/proc/net/xt_qtaguid/iface_stat_all /proc/net/xt_qtaguid/iface_stat_fmt /proc/net/xt_qtaguid/stats

com.android.server NetworkManagementService

@Override public Config getDevConfig() { return new Config(getSecureLong(NETSTATS_DEV_BUCKET_DURATION, HOUR_IN_MILLIS), getSecureLong(NETSTATS_DEV_ROTATE_AGE, 15 * DAY_IN_MILLIS), getSecureLong(NETSTATS_DEV_DELETE_AGE, 90 * DAY_IN_MILLIS)); } @Override public Config getXtConfig() { return getDevConfig(); } @Override public Config getUidConfig() { return new Config(getSecureLong(NETSTATS_UID_BUCKET_DURATION, 2 * HOUR_IN_MILLIS), getSecureLong(NETSTATS_UID_ROTATE_AGE, 15 * DAY_IN_MILLIS), getSecureLong(NETSTATS_UID_DELETE_AGE, 90 * DAY_IN_MILLIS)); } @Override public Config getUidTagConfig() { return new Config(getSecureLong(NETSTATS_UID_TAG_BUCKET_DURATION, 2 * HOUR_IN_MILLIS), getSecureLong(NETSTATS_UID_TAG_ROTATE_AGE, 5 * DAY_IN_MILLIS), getSecureLong(NETSTATS_UID_TAG_DELETE_AGE, 15 * DAY_IN_MILLIS)); }

com.android.server.net.NetworkStatsService.DefaultNetworkStatsSettings

$ adb shell cat /proc/net/xt_qtaguid/iface_stat_all eth0 1 0 0 0 0 220609 3228 2891157 2995

$ adb shell cat /proc/net/xt_qtaguid/iface_stat_fmt ifname total_skb_rx_bytes total_skb_rx_packets total_skb_tx_bytes total_skb_tx_packets eth0 163781 3231 2846738 2994

Dev: IP + Non IP layer traffic

Xt: Only IP layer traffic

$ adb shell cat /proc/net/xt_qtaguid/stats idx iface acct_tag_hex uid_tag_int cnt_set rx_bytes rx_packets tx_bytes tx_packets rx_tcp_bytes rx_tcp_packets rx_udp_bytes rx_udp_packets rx_other_bytes rx_other_packets tx_tcp_bytes tx_tcp_packets tx_udp_bytes tx_udp_packets tx_other_bytes tx_other_packets 2 eth0 0x0 0 0 163335 3226 2846228 2988 161473 3222 1862 4 0 0 2845850 2982 266 4 112 2 3 eth0 0x0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 4 eth0 0x0 1000 0 819 11 965 12 667 9 152 2 0 0 813 10 152 2 0 0 5 eth0 0x0 1000 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

Uid: UID level details

Measuring Battery Usage

Public

android.os.BatteryManager

Public Internal

android.os.BatteryStats com.android.internal.os.BatteryStatsImpl com.android.internal.os.PowerProfile com.android.server.os.BatteryStatsService

adb shell dumpsys battery adb shell dumpsys batteryinfo

None (?)

$ adb shell dumpsys batteryinfo Battery History: -1d00h43m02s057ms START -1d00h40m00s730ms 050 20080002 status=charging health=good plug=ac temp=0 volt=0 +plugged +sensor brightness=medium -1d00h39m57s332ms 050 60080002 +wake_lock -1d00h39m57s202ms 050 60080302 phone_state=off ... Per-PID Stats: PID 1227 wake time: +3s445ms PID 1227 wake time: +199ms ... Statistics since last charge: System starts: 29, currently on battery: false Time on battery: 0ms (0.0%) realtime, 0ms (0.0%) uptime Total run time: 1d 2h 1m 3s 606ms realtime, 1d 2h 1m 3s 603ms uptime, ... #1000: User activity: 26 other, 27 button, 93 touch Sensor 0: (not used) Apk com.android.keychain: (nothing executed) ... Statistics since last unplugged: Time on battery: 0ms (0.0%) realtime, 0ms (0.0%) uptime ...

Measuring Battery Usage: dumpsys batteryinfo

adb shell dumpsys batteryinfo

How ‘dumpsys batteryinfo’ works?

com.android.server.am. BatteryStatsService

com.android.internal.os BatteryStatsImpl

android.os BatteryStats

noteBluetoothOff() noteBluetoothOn() noteFullWifiLockAcquired(int) noteFullWifiLockAcquiredFromSource(WorkSource) noteFullWifiLockReleased(int) noteFullWifiLockReleasedFromSource(WorkSource) noteInputEvent() noteNetworkInterfaceType(String, int) notePhoneDataConnectionState(int, boolean) notePhoneOff() notePhoneOn() notePhoneSignalStrength(SignalStrength) notePhoneState(int) noteScreenBrightness(int) noteScreenOff() noteScreenOn() ... noteWifiScanStarted(int) noteWifiScanStartedFromSource(WorkSource) noteWifiScanStopped(int) noteWifiScanStoppedFromSource(WorkSource) noteWifiStopped(WorkSource)

If you can’t understand the measurement results,

it is impossible to improve it.

Automated Historical Performance Analysis with kmemtracer

Technology

Automated Performance Testing with HP LoadRunner

AUTOMATED DISCOVERY OF PERFORMANCE REGRESSIONS IN ... · automated approach to detect performance regressions from results of performance tests conducted on the implementation of

Semi-automated extraction of expressive performance ...charm.cch.kcl.ac.uk/redist/pdf/s4Earis.pdf · Semi-automated extraction of expressive performance information from acoustic

Historical Performance: Theory, Practice, and Interdisciplinaritymusic.indiana.edu/departments/academic/early-music/2017_HPI_Conf... · Historical Performance: Theory, Practice, and

Mining Performance Regression Testing Repositories for Automated Performance Analysis

Ciprian balea automated performance-testing

Automated Historical Performance Analysis with kmemtracer · ... android=" ... getInstrumentation().runOnMainSync ... > com.example.android.apis.robotium.test/\ > android.test

Automated aerodynamic performance estimates for flapping

Automated Linking of Historical Datakaeriksson.ucdavis.edu/uploads/6/0/6/7/60676749/w25825.pdf · 2019. 5. 13. · Automated Linking of Historical Data Ran Abramitzky, Leah Platt

Historical Performance Harmonic Patterns

Automated Customer-Centric Performance Analysis of

Automated Performance Testing with JMeter and Maven

DiPerF: automated DIstributed PERformance testing Framework

Automated Performance Measures for Snow & Ice Results

SPE IADC 119746 Automated Drilling Performance Meas

A HIGH PERFORMANCE SEMI-AUTOMATED CLINICAL …

Automated Performance Deviation Detection Across Software ... · Automated Performance Deviation Detection Across Software Versions Releases Abderrahmane Benbachir, Isnaldo Francisco

The Historical Performance of Music

Chapter 9 Analyzing Historical Performance

AUTOMATED LINKING OF HISTORICAL DATA