Debugging and Pro ling - ocw.cs.pub.roocw.cs.pub.ro/courses/_media/osp/lectures/lecture-debug.pdfDebug Session Setup I ndk-gdb script sets up the debug session I Launches the app using

Debugging and ProfilingLecture 10

Operating Systems Practical

11 January 2017

This work is licensed under the Creative Commons Attribution 4.0 International License. To view a copy of thislicense, visit http://creativecommons.org/licenses/by/4.0/.

OSP Debugging and Profiling, Lecture 10 1/56

Logging

Debugging

Troubleshooting

Profiling

Bibliography

Keywords


Outline

Logging

Debugging

Troubleshooting

Profiling

Bibliography

Keywords


Android Logging Framework

I Logd daemonI From Android 5.0I Centralized user-mode loggerI Uses 4 sockets

I Liblog library

I android.util.Log

I logcat


Android Logging Framework

I Logd sockets accessed only through liblogI Write log messages:

1. Log class2. Liblog library3. /dev/socket/logdw socket

I Read log messages:

1. logcat

2. Liblog library3. /dev/socket/logdr socket


Log Message Structure

I Priority - severityI Verbose, debug, info, warning, error, assert

I Tag identifies the component generating the messageI Logcat can filter log messages based on the tag

I Message: actual log text


Java Logging API

I android.util.LogI Log.v(String tag, String msg) -> verboseI Log.d(String tag, String msg) -> debugI Log.i(String tag, String msg) -> informationI Log.w(String tag, String msg) -> warningI Log.e(String tag, String msg) -> errorI Log.wtf(String tag, String msg) -> assertI Log.println(int priority, String tag, String msg)

I Example:I Log.i("MyActivity", "Get item number " + pos);I I/MyActivity(1557): Get item number 1


Native Logging API

I Exposed through android/log.h

I #include <android/log.h>

I Android.mk dynamically link native code to log libraryI LOCAL_LDLIBS += −llogI Before include $(BUILD_SHARED_LIBRARY)


Native Logging API

I __android_log_writeI Generate a simple string messageI Params: priority, tag, message

__android_log_write(ANDROID_LOG_WARN, "my_native_code",

"Warning message!");

I __android_log_printI Generate formatted string (like printf)I Params: priority, tag, string format, other params

__android_log_print(ANDROID_LOG_ERROR, "my_native_code",

"Errno =%d", errno);


Native Logging API

I __android_log_vprintI Additional parameters as va_list

void log_verbose(const char* format, ...){va_list args;

va_start(args, format);

__android_log_vprint(ANDROID_LOG_VERBOSE, "my_-

native_code", format, args);

va_end(args);

}I __android_log_assert

I Assertion failuresI Priority is not specified, always fatal

__android_log_assert("0 != errno", "my_native_code", "Big

error!");

I SIGTRAP to process - debugger inspection


Logcat

I Display log messages

I Command line through adb or Android Studio

I Set log level, search, apply filters

I Log format:date time PID-TID/package priority/tag: message

I Example:12-10 13:02:50.071 1901-4229/com.google.android.gms V/AuthZen:

Handling delegate intent.


Log control

I Cannot suppress log messages based on priority

I Preprocessor based solution

#define MY_LOG_NOOP (void) 0

#define MY_LOG_PRINT(level ,fmt ,...) \

__android_log_print(level , MY_LOG_TAG , "(%s:%u) %s: " fmt , \

__FILE__ , __LINE__ , __PRETTY_FUNCTION__ , ## __VA_ARGS__)

#if MY_LOG_LEVEL_WARNING >= MY_LOG_LEVEL

# define MY_LOG_WARNING(fmt ,...) \

MY_LOG_PRINT(ANDROID_LOG_WARN , fmt , ## __VA_ARGS__)

#else

# define MY_LOG_WARNING (...) MY_LOG_NOOP

#endif


Log Control

I In native code#include "my-log.h"

...

MY_LOG_WARNING("Message!");

I In Android.mkMY_LOG_TAG := \"my_native_code\"

ifeq ($(APP_OPTIM),release)

MY_LOG_LEVEL := MY_LOG_LEVEL_ERROR

else

MY_LOG_LEVEL := MY_LOG_LEVEL_VERBOSE

endif

LOCAL_CFLAGS += −DMY_LOG_TAG=$(MY_LOG_TAG)LOCAL_CFLAGS += −DMY_LOG_LEVEL=$(MY_LOG_LEVEL)


Log Console Messages

I STDOUT and STDERR not visible by defaultI Redirect STDOUT and STDERR to logging system

adb shell stop

adb shell setprop log.redirect-stdio true

adb shell start

I Display with logcat - tags stdout and stderrI Temporary config -> erased when booting device

I Permanent config -> modify /data/local.prop on device


Outline

Logging

Debugging

Troubleshooting

Profiling

Bibliography

Keywords


GDB

I NDK supports debugging using GNU Debugger (GDB)I ndk-gdb script

I Handles error conditionsI Outputs error messages

I RequirementsI Use ndk-build -> build system generates files needed for

debuggingI android:debuggable in AndroidManifest.xmlI Android version 2.2 or higher


Debug Session Setup

I ndk-gdb script sets up the debug sessionI Launches the app using Activity Manager through ADB

I Activity Manager sends the request to ZygoteI Zygote forks and creates new process

I ndk-gdb starts GDB server and attaches to the app

I Configures port forwarding to make GDB server accessiblefrom the host machine (debug port)

I Copies binaries for Zygote and shared libraries to the host

I Starts GDB clientI Debug session is active -> You can start debugging app

I Commands sent over the debug port


Debug from CLI

I Go to project directory

I rm -rf bin obj libs

I Compile native code using ndk-build

I We need build.xml -> android update project -p

I Compile and package the whole project in debug mode ant

debug

I Deploy app on device ant installd

I ndk-gdb --start to start app and the debugging session

I When GDB prompt appears run commands


GDB Commands

I break: Breakpoint in a location (function name, file name &line number)

I clear: deletes all breakpoints

I enable/disable/delete: operations on a certain breakpoint

I next: go to the next line in source code

I continue: continue execution

I backtrace: display call stack

I backtrace full: call stack with local variables on frames

I print: display variable, expression, memory address, register

I display: continue printing value after each step

I info threads: list running threads

I thread: select a certain thread


Android Studio Debugger

I Debug button

I Select the device running the app

I Set breakpoints in the Java or native code

I Examine variables or expressions at runtime

I Capture screenshots or videos of the app

I LLDB debugger for native code


Android Studio Debugger

Source: http://developer.android.com


Android Studio Debugger - Breakpoints

I Add line breakpoint:I Locate line, Ctrl+F8I Click Attach debugger to Android proccess

I When code execution reaches a breakpoint:I Examine object tree for a variableI Evaluate expressionI Advance to next line of code (Step Over)I Advance to first line inside a method call (Step In)I Advance to the next line outside the current method (Step

Out)I Continue running app


Dalvik Debug Monitor Server

I Integrated in Android Studio

I Launch Android Debug Monitor -> DDMS button

I Works with real devices and emulator

I Debugging Android applications

I Port-forwarding, screen capture, thread info, heap info,process state, radio state, incoming call, SMS spoofing,location spoofing, etc.


DDMS - Interaction with Debugger

I When started, DDMS connects to adb

I VM monitoring service is created between adb and DDMS

I The service notifies DDMS when a VM is started orterminated

I Obtains the pid, opens a connection to the VM’s debuggerthrough adbd

I Talks to the VM using a custom wire protocol


DDMS



DDMS - Heap Update

I View how much heap the process is usingI Select process in Devices tabI Update Heap to obtain heap infoI Cause GC to invoke Garbage Collection (refresh data)I Select object type to view number of allocated objects


DDMS - Heap Update



DDMS - Track Allocations

I Track memory allocationI Start Tracking in the Allocation Tracker tabI Get Allocations to obtain list of allocated objectsI Finally Stop TrackingI Detailed info about the method and line that allocated a

certain object

I Examine thread infoI Update Threads to obtain thread info for the selected process


DDMS - Track Allocations



Outline

Logging

Debugging

Troubleshooting

Profiling

Bibliography

Keywords


Stack Trace

I Use troubleshooting tools and techniques to identify the causeof a problem

I Observe the stack trace when an app crashes with logcatI Lines starting with # represent stack callsI Line #00 is the crash pointI After #00 the address is specified (pc)I Next lines - previous function calls


ndk-stack

I To add file names and line numbers to the stack trace

I adb logcat | ndk-stack -sym obj/local/armeabi

I Run command in the project directory


ndk-stack Example

********** Crash dump: **********

Build fingerprint: ’generic/google_sdk/generic /:2.2/ FRF91 /43546:

eng/test -keys ’

pid: 351, tid: 351 >>> /data/local/ndk -tests/crasher <<<

signal 11 (SIGSEGV), fault addr 0d9f00d8

Stack frame #00 pc 0000841e /data/local/ndk -tests/crasher :

Routine zoo in /tmp/foo/crasher/jni/zoo.c:13

Stack frame #01 pc 000083 fe /data/local/ndk -tests/crasher :

Routine bar in /tmp/foo/crasher/jni/bar.c:5

Stack frame #02 pc 000083 f6 /data/local/ndk -tests/crasher :

Routine my_comparison in /tmp/foo/crasher/jni/foo.c:9

Stack frame #03 pc 000191 ac /system/lib/libc.so

Stack frame #04 pc 000083 ea /data/local/ndk -tests/crasher :

Routine foo in /tmp/foo/crasher/jni/foo.c:14

Stack frame #05 pc 00008458 /data/local/ndk -tests/crasher :

Routine main in /tmp/foo/crasher/jni/main.c:19

Stack frame #06 pc 0000 d362 /system/lib/libc.so


CheckJNI

I Extended series of checks before calling JNI functionsI Enable CheckJNI on a device

I Rooted deviceadb shell stop

adb shell setprop dalvik.vm.checkjni true

adb shell start

I Logcat: D AndroidRuntime: CheckJNI is ON

I Regular deviceadb shell setprop debug.checkjni 1

I Logcat: D Late-enabling CheckJNI

I Error detected by CheckJNIW JNI WARNING: method declared to return

’Ljava/lang/String;’ returned ’[B’

W failed in LJniTest;.exampleJniBug


Libc Debug Mode

I Troubleshoot memory issues

I Enable libc debug modeadb shell setprop libc.debug.malloc 1

adb shell stop

adb shell start

I Libc debug mode valuesI 1 - detects memory leaksI 5 - detects overruns by filling allocated memoryI 10 - detects overruns by filling memory and adding sentinel

... testapp using MALLOC_DEBUG = 10 (sentinels, fill)

... *** FREE CHECK buffer 0xa5218, size=1024, corrupted 1

bytes after allocation


Valgrind

I Advanced memory analysis

I Open-source tool for memory debugging, memory leaksdetection and profiling

I Support for AndroidI Build from sources

I Binaries and components in Inst directoryI adb push Inst /data/local/I Give execution permissions

I Helper script#!/system/bin/sh

export TMPDIR=/sdcard

exec /data/local/Inst/bin/valgrind --error-limit=no $*

I Push in /data/local/Inst/bin and set execution permissions


Valgrind

I To run app under Valgrind, inject the script into the startupsequenceadb shell setprop wrap.com.example.testapp "logwrapper

/data/local/Inst/bin/valgrind_wrapper.sh"

I Property wrap.packagename

I Execute app

I Logcat displays Valgrind output


Strace

I Intercepts system calls and signals

I System call name, arguments and return value

I Useful for analyzing closed-source applications

I Included in Android emulator

I Run the application and obtain pidadb shell ps | grep com.example.testapp

I Attach strace to running appadb shell strace -v -p <PID>


Tombstones

I Tombstone - generated when a process crashes

I /data/tombstones/tombstone_*

I A file containing information about the crashed processI Build fingerprintI Crashed process, PID, TIDsI Signal and fault addressI CPU registersI Call stackI Stack content of each call

I Use with ndk-stack and addr2line to obtain the file andline where the process has crashed


Tombstones

Build fingerprint: ’Android/aosp_hammerhead/hammerhead :4.4.2/

KOT49H/eng.upb .20140407.130154: userdebug/test -keys ’

Revision: ’11’

pid: 27836, tid: 27836, name: test.nativeapp4 >>>

com.test.nativeapp4 <<<

signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 00000000

r0 00000000 r1 00000000 r2 6f8b70e4 r3 6f8b8328

[..]

backtrace:

#00 pc 00008 bbc /system/lib/libandroid.so (AAsset_close +3)

#01 pc 00000 d47 /data/app -lib/com.test.nativeapp4 -2/

libNativeApp4.so (displayAsset(ANativeActivity *)+18)

#02 pc 00000 db1 /data/app -lib/com.test.nativeapp4 -2/

libNativeApp4.so (ANativeActivity_onCreate +96)

[..]

stack:

bea91430 00000000

bea91434 401 a7315 /system/lib/libutils.so

(android :: SharedBuffer :: release(unsigned int) const +28)

bea91438 bea91450 [stack]

bea9143c 00000000

bea91440 00000000

bea91444 402 ad59b /system/lib/libandroidfw.so

bea91448 6f8b70e0 [anon:libc_malloc]


Outline

Logging

Debugging

Troubleshooting

Profiling

Bibliography

Keywords


GNU Profiler (Gprof)

I Unix-based profiling toolI Compute absolute execution time spent in each function

I Instrumentation with gcc when using -pg at compile timeI Sampling data stored at run-time in gmon.outI gprof uses gmon.out to produce profiling reports

I Android NDK includes gprof toolI Android NDK toolchain lacks the implementation of

__gnu_mcount_nc used for timing

I Open-source project Android NDK Profiler


Android NDK Profiler

I Install moduleI Download zip, extract in $NDK_HOME/sources, rename

directory to android-ndk-profiler

I Enable profilerI Update Android.mk to statically link profiling libraryI Include prof.h in the native code

#ifdef MY_ANDROID_NDK_PROFILER_ENABLED

#include <prof.h>

#endifI Start collecting profiling data


monstartup("libModule.so");

#endifI Stop collecting data


moncleanup();

#endif


Generate Reports

I The collected data is stored in /sdcard/gmon.out

I App needs permission to write on the SD card<uses-permission android:name="android.permission.WRITE_-

EXTERNAL_STORAGE" />

I Pull gmon.out from the SD cardI Run gprof

$NDK_HOME/toolchains/arm-linux-androideabi-4.4.3/prebuilt/

linux-x86/bin/arm-linux-androideabi-gprof

obj/local/armeabi-v7a/libModule.so gmon.out

I Gprof analyses data and generates a reportI Two sections: flat profile and call graphI Duration of each function


Systrace

I Gather code execution data -> Identify execution problemsand improve performance

I Show all processes on a common timeline

I Execution times, CPU frequency, CPU load, disk activity,threads


Systrace

I Android 4.1 or higher, root access, developer debuggingenabled

I GUI and CLI$ cd android-sdk/platform-tools/systrace

$ python systrace.py --time=10 -o mynewtrace.html sched gfx

view wm

I Open trace in a web browser


Systrace - Analyze the Trace

I Inspect frames, investigate alerts, identify performance issues



Systrace - Application Level Tracing

I From Android 4.3 use Trace class to add instrumentation tothe application code

I Trace calls can be nested

I Traces must begin and end in the same thread

Trace.beginSection("Start trace");

try {

// executing tasks

} finally {

Trace.endSection (); // end trace

}


Traceview

I Graphical viewer for execution logs

I Trace logs generated with Debug class

I Timeline panel - displays each thread and methodstarted/stopped

I Profile panel - summary of all time spent in a method


Traceview

I Two methods to generate trace logs:I Use methods of the Debug class to start and stop tracingI Use method profiling feature of DDMS (no precise log timing)

Debug.startMethodTracing("data"); // start tracing to

// "/ sdcard/data.trace"

// execute tasks

Debug.stopMethodTracing (); // stop tracing


Traceview - Timeline Panel

I Displays the execution of each thread in a separate row

I Each method - a different color

I Thin line below - extent of all calls to the selected method



Traceview - Profile Panel

I All time spent in a method (inclusive and exclusive times)

I Exclusive time = time spent in a method

I Inclusive time = time spent in a method + time spent in anycalled functions

I Last column - number of calls to this method + number ofrecursive calls



Outline

Logging

Debugging

Troubleshooting

Profiling

Bibliography

Keywords


Bibliography

I Onur Cinar, Pro Android C++ with the NDK, Chapter 5, 14

I Sylvain Ratabouil, Android NDK, Beginner’s Guide, Chapter11

I https://code.google.com/p/android-ndk-profiler/

wiki/Usage

I http:

//developer.android.com/tools/debugging/ddms.html

I http://bytesthink.com/blog/?p=133

I http://developer.android.com/tools/debugging/

systrace.html


https://code.google.com/p/android-ndk-profiler/wiki/Usage

https://code.google.com/p/android-ndk-profiler/wiki/Usage

http://developer.android.com/tools/debugging/ddms.html

http://developer.android.com/tools/debugging/ddms.html

http://bytesthink.com/blog/?p=133

http://developer.android.com/tools/debugging/systrace.html

http://developer.android.com/tools/debugging/systrace.html

Outline

Logging

Debugging

Troubleshooting

Profiling

Bibliography

Keywords


Keywords

I Logger

I Logging API

I Log control

I GDB

I DDMS

I Stack trace

I Tombstones

I CheckJNI

I Libc Debug Mode

I Valgrind

I Strace

I Gprof

I Android NDK Profiler

I Systrace

I Traceview


Documents

Debugging and Pro ling - ocw.cs.pub.roocw.cs.pub.ro/courses/_media/osp/lectures/lecture-debug.pdfDebug Session Setup I ndk-gdb script sets up the debug session I Launches the app using