Tips for better CI on Android

Tips for better CI on AndroidTomoaki Imai @Potatotips #22 2015/10/13

Hi I’m Tomo(今井智章)





Continuous Integration on Android

1. How do we CI on Android

2. 4 tips for better CI



How do we CI Mercari Android App?

CI for Mercari Android App

• Started April 2015

• Purpose

- Build Check each Flavour and BuildType

- Automate app tests

- Continuous deployment

Phase 1: Travis CI

Good!!

• Easy setup with yml • Build production app without

including secret keys into the repo(http://docs.travis-ci.com/user/encrypting-

files/)

• Most of Mercari project run CI on Travis CI

http://docs.travis-ci.com/user/encrypting-files/

So-so…• Need some tweaks to avoid Out of Memory

• Slow. Takes 12-15min/per build check

Phase 2: circleCI

Good!!• Has better features for testing and debugging

• Bit faster than Travis CI (1- 2min for each build)

• Mercari iOS project runs CI on CircleCI

So-so…• Need some tweaks to avoid Out of Memory

• Building production app looked a bit tricky (Haven’t tried yet)

Both CI service have good parts…

Why not use both?

Phase 3: Travis CI & circleCI

Check Production App Check Development App Run JUnit Tests

Deploy

Let each CI service do different tasks

Total CI Running time reduced to 8-7 min(about 2x shorter)



Tip 1: Avoid OOM(CircleCI)

• 4GB memory limit


• 4GB memory limit


https://circleci.com/docs/oombut was in vain…

Guide for 4GB limit

Check memory_usage.txt

PID RSS %CPU COMMAND21787 2250024 165 /usr/lib/jvm/java-7-openjdk-amd64/bin/java -Xmx4g -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant -cp /usr/local/android-sdk-linux/build-tools/21.1.1/lib/dx.jar com.android.dx.command.Main --dex --force-jumbo --output /home/ubuntu/mercari-android-as/app/build/intermediates/dex/us/dev /home/ubuntu/mercari-android-as/app/build/intermediates/classes-proguard/us/dev/classes.jar9477 1628300 104 /usr/lib/jvm/java-7-openjdk-amd64/bin/java -XX:+HeapDumpOnOutOfMemoryError -Xmx2048m -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant -cp /home/ubuntu/.gradle/wrapper/dists/gradle-2.2.1-all/6dibv5rcnnqlfbq9klf8imrndn/gradle-2.2.1/lib/gradle-launcher-2.2.1.jar org.gradle.launcher.daemon.bootstrap.GradleDaemon 2.2.1 /home/ubuntu/.gradle/daemon 10800000 b3531fa1-fad3-4599-a284-96a81cc9336b -XX:+HeapDumpOnOutOfMemoryError -Xmx2048m -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant9145 279164 4.7 /usr/lib/jvm/java-7-openjdk-amd64/bin/java -Xms256m -Xmx512m -Dorg.gradle.jvmargs=-Xmx2048m -XX:+HeapDumpOnOutOfMemoryError -Dorg.gradle.appname=gradlew -classpath /home/ubuntu/mercari-android-as/gradle/wrapper/gradle-wrapper.jar org.gradle.wrapper.GradleWrapperMain assembleDev -PdisablePreDex




JVM is using too much memory!2250024 + 1628300 + 279164 = 4157488KB ≒4GB



Progard + Gradle consumes memory a lot


Tuning tips: • Lower the total memory(JVM,Gradle) size to 4.0G • Adjust Xmx and MaxPermSize • Set timeout (>1000) for GC

Xmx -> Heap space (where object resources are allocated ) MaxPermSize -> Permgen(where class, methods are loaded)


machine: environment: ANDROID_HOME: /usr/local/android-sdk-linux _JAVA_OPTIONS: "-Xms256m -Xmx1280m -XX:MaxPermSize=350m" GRADLE_OPTS: '-Dorg.gradle.jvmargs="-Xmx1536m -XX:MaxPermSize=512m -XX:+HeapDumpOnOutOfMemoryError"' … override: - ./gradlew assembleDev -PdisablePreDex: timeout: 1500

Heap Perm Heap Perm other

JVM(dex) Gradle

4GB

• There is no best practice but I suggest that large memory be allocated to Xmx

Tip 2: Save the JUnit test result(CircleCI)

test: post: - mkdir -p $CIRCLE_TEST_REPORTS/junit/ - find . -type f -regex ".*/test-results/*/.*xml" -exec cp {} $CIRCLE_TEST_REPORTS/junit/ \;

• You can check Test result from ‘Test Failure’

http://qiita.com/tomoima525/items/b4a91d1624aaf0112b3fRef. CircleCIでJUnitの結果を見る

Tip 3: Continuous deployment for specific branches(CircleCI)

• Run shell script for specific branchesdeployment: jp: branch: [/deploy_jp_.*/, /release_.*/] commands: - ./deploy_jp.sh $CIRCLE_BRANCH us: branch: [/deploy_us_.*/, /release_.*/] commands: - ./deploy_us.sh $CIRCLE_BRANCH

RELEASE_MESSAGE=$1

#deploycurl -F "file=@app/apk/mercari_dev_jp.apk" -F "token=${DEPLOY_GATE_API_KEY}" -F "message=${RELEASE_MESSAGE}" -F "distribution_key=${JP_DEV_DIST_KEY}" -F "release_note=${RELEASE_MESSAGE}" https://deploygate.com/api/users/xxxx/apps

deploy_jp.sh

https://deploygate.com/api/users/xxxx/apps

Tip 3: Continuous deployment for specific branches(CircleCI)

• Set variables at `Environment variables`

Tip 4: Add dependencies manually

dependencies: override: - echo y | android update sdk --no-ui --all --filter extra-google-m2repository,extra-android-m2repository,build-tools-23.0.1, android-23

android: components: - build-tools-23.0.1 - android-23 - extra-android-m2repository - extra-google-m2repository

.travis.yml

circle.yml

• The latest packages(ex. Android M) are not installed on the VM

• Add manually with command line

• Tip 1: Check memory usage of JVM to avoid OOM

• Tip 2: Save the JUnit test result on the VM

• Tip 3: Continuous deployment for specific branches

• Tip 4: Add dependencies manually

Try it out for your project!

Software

Tips for better CI on Android