Click here to load reader
Upload
doug-sillars
View
2.551
Download
2
Embed Size (px)
Citation preview
© 2015 AT&T Intellectual Property. All rights reserved. AT&T, the AT&T logo and all other marks contained herein are trademarks of AT&T Intellectual Property and/or AT&T affiliated companies. All other marks contained herein are the property of their respective owners. Information contained herein is not an offer, commitment, representation or warranty by AT&T and is subject to change. Mention of a specific company or entity is not an endorsement by AT&T.
Doug Sillars
High Performance: What These App Did May Shock You
ARO Technical Lead - AT&T@Dougsillars
© 2015 AT&T Intellectual Property. All rights reserved. AT&T, the AT&T logo and all other marks contained herein are trademarks of AT&T Intellectual Property and/or AT&T affiliated companies. All other marks contained herein are the property of their respective owners. Information contained herein is not an offer, commitment, representation or warranty by AT&T and is subject to change. Mention of a specific company or entity is not an endorsement by AT&T.
The Next 40 Minutes Will Change Your App
Defects vs. PerformanceIs Performance REALLY that important? Yes!
https://t.co/7bMT4A1PIt
© 2015 AT&T Intellectual Property. All rights reserved. AT&T, the AT&T logo and all other marks contained herein are trademarks of AT&T Intellectual Property and/or AT&T affiliated companies. All other marks contained herein are the property of their respective owners. Information contained herein is not an offer, commitment, representation or warranty by AT&T and is subject to change. Mention of a specific company or entity is not an endorsement by AT&T.
Developers Make This Mistake Everyday,And it’s Killing Their Customer’s Battery
• Data Connections account for:• 40% of battery use (screen on)• 70% screen off
http://thenextweb.com/dd/2013/08/05/how-to-minimize-your-mobile-apps-power-consumption/http://commons.wikimedia.org/wiki/File:Free_EV_charge_station_sign_evinfra.svg
© 2015 AT&T Intellectual Property. All rights reserved. AT&T, the AT&T logo and all other marks contained herein are trademarks of AT&T Intellectual Property and/or AT&T affiliated companies. All other marks contained herein are the property of their respective owners. Information contained herein is not an offer, commitment, representation or warranty by AT&T and is subject to change. Mention of a specific company or entity is not an endorsement by AT&T.
• Cellular Networks are High Latency Environments• Not Optimizing Your Content can lead to delays:• 500ms:
• 26% frustration• 8% engagement
• Amazon.com & Wallmart: • 100ms delay• 1% revenue
• 4% of mobile users admit to throwing their phones
You Won’t Believe What Else Network Connections Do
http://www.flickr.com/photos/9009139@N08/1263954439http://blog.radware.com/applicationdelivery/applicationaccelerationoptimization/2013/12/mobile-web-stress-the-impact-of-network-speed-on-emotional-engagement-and-brand-perception-reporthttp://www.mobilejoomla.com/mobile-joomla-blog/172-responsive-design-vs-server-side-solutions-infographic.html
© 2015 AT&T Intellectual Property. All rights reserved. AT&T, the AT&T logo and all other marks contained herein are trademarks of AT&T Intellectual Property and/or AT&T affiliated companies. All other marks contained herein are the property of their respective owners. Information contained herein is not an offer, commitment, representation or warranty by AT&T and is subject to change. Mention of a specific company or entity is not an endorsement by AT&T.
Is Performance Holding Your App Back?
Retention Rate
Usage
Lifetime Value Lifetime Value
Active Users
Session Length
Average Revenue per user App Launch/Load Time
User Acquisition
User Experience/ Happiness
© 2015 AT&T Intellectual Property. All rights reserved. AT&T, the AT&T logo and all other marks contained herein are trademarks of AT&T Intellectual Property and/or AT&T affiliated companies. All other marks contained herein are the property of their respective owners. Information contained herein is not an offer, commitment, representation or warranty by AT&T and is subject to change. Mention of a specific company or entity is not an endorsement by AT&T.
One Developer Tried This One Simple Trick…
Abandon-ment
Discovery Adoption Trial
Long-Term
Usage
48% of customers abandon if dissatisfied with the performance of a mobile app
http://www.tatvic.com/blog/churn-analysis-for-mobile-app/http://offers2.compuware.com/APM_13_WP_Mobile_App_Survey_Report_Registration.html
© 2015 AT&T Intellectual Property. All rights reserved. AT&T, the AT&T logo and all other marks contained herein are trademarks of AT&T Intellectual Property and/or AT&T affiliated companies. All other marks contained herein are the property of their respective owners. Information contained herein is not an offer, commitment, representation or warranty by AT&T and is subject to change. Mention of a specific company or entity is not an endorsement by AT&T.
Understanding Application Performanceand Testing for
© 2015 AT&T Intellectual Property. All rights reserved. AT&T, the AT&T logo and all other marks contained herein are trademarks of AT&T Intellectual Property and/or AT&T affiliated companies. All other marks contained herein are the property of their respective owners. Information contained herein is not an offer, commitment, representation or warranty by AT&T and is subject to change. Mention of a specific company or entity is not an endorsement by AT&T.
Just Because Someone Says “Unsinkable”
98% of apps tested have potential Performance Optimizations
http://commons.wikimedia.org/wiki/File:RMS_Titanic_3.jpg https://www.flickr.com/photos/kb-monkeywrench/3639224089/
© 2015 AT&T Intellectual Property. All rights reserved. AT&T, the AT&T logo and all other marks contained herein are trademarks of AT&T Intellectual Property and/or AT&T affiliated companies. All other marks contained herein are the property of their respective owners. Information contained herein is not an offer, commitment, representation or warranty by AT&T and is subject to change. Mention of a specific company or entity is not an endorsement by AT&T.
Death by a Thousand CutsOr, It Isn’t Always the Iceberg
Minor issues found in testing
Big problems at scale
http://en.wikipedia.org/wiki/Hans_Brinker,_or_The_Silver_Skates#mediaviewer/File:Hans_Brinker_Madurodam.jpg
© 2015 AT&T Intellectual Property. All rights reserved. AT&T, the AT&T logo and all other marks contained herein are trademarks of AT&T Intellectual Property and/or AT&T affiliated companies. All other marks contained herein are the property of their respective owners. Information contained herein is not an offer, commitment, representation or warranty by AT&T and is subject to change. Mention of a specific company or entity is not an endorsement by AT&T.
• 44% of all issues are reported by users• 20% issues were only found because of
complaints in app reviews
Helps to Find Issues EarlySuccessful Applications Find That Testing
http://info.perfectomobile.com/rs/perfectomobile/images/why-apps-fail-infographic.pdf
© 2015 AT&T Intellectual Property. All rights reserved. AT&T, the AT&T logo and all other marks contained herein are trademarks of AT&T Intellectual Property and/or AT&T affiliated companies. All other marks contained herein are the property of their respective owners. Information contained herein is not an offer, commitment, representation or warranty by AT&T and is subject to change. Mention of a specific company or entity is not an endorsement by AT&T.
The Bigger They Are…
The Quicker Small Issues Magnify
© 2015 AT&T Intellectual Property. All rights reserved. AT&T, the AT&T logo and all other marks contained herein are trademarks of AT&T Intellectual Property and/or AT&T affiliated companies. All other marks contained herein are the property of their respective owners. Information contained herein is not an offer, commitment, representation or warranty by AT&T and is subject to change. Mention of a specific company or entity is not an endorsement by AT&T.
Find My Goat*
© 2015 AT&T Intellectual Property. All rights reserved. AT&T, the AT&T logo and all other marks contained herein are trademarks of AT&T Intellectual Property and/or AT&T affiliated companies. All other marks contained herein are the property of their respective owners. Information contained herein is not an offer, commitment, representation or warranty by AT&T and is subject to change. Mention of a specific company or entity is not an endorsement by AT&T.
Find My Goat
• Connecting to the network every 5 seconds• Creating a new connection every time• 18x more connections than this app’s architecture team expected• The triage team discovered that IP/port collisions = loss in sales
© 2015 AT&T Intellectual Property. All rights reserved. AT&T, the AT&T logo and all other marks contained herein are trademarks of AT&T Intellectual Property and/or AT&T affiliated companies. All other marks contained herein are the property of their respective owners. Information contained herein is not an offer, commitment, representation or warranty by AT&T and is subject to change. Mention of a specific company or entity is not an endorsement by AT&T.
Computers Are Great at Repetitive Stuff…
http://commons.wikimedia.org/wiki/File:Assen_Gert_Sennema_-_Sisyphus_1.jpghttp://commons.wikimedia.org/wiki/File:All_work_and_no_play_makes_Jack_a_dull_boy_(The_Shining)_(7957738500).jpg
But Don’t Punish Your Users
© 2015 AT&T Intellectual Property. All rights reserved. AT&T, the AT&T logo and all other marks contained herein are trademarks of AT&T Intellectual Property and/or AT&T affiliated companies. All other marks contained herein are the property of their respective owners. Information contained herein is not an offer, commitment, representation or warranty by AT&T and is subject to change. Mention of a specific company or entity is not an endorsement by AT&T.
Repetitive Download vs. Caching
100 seconds to load
© 2015 AT&T Intellectual Property. All rights reserved. AT&T, the AT&T logo and all other marks contained herein are trademarks of AT&T Intellectual Property and/or AT&T affiliated companies. All other marks contained herein are the property of their respective owners. Information contained herein is not an offer, commitment, representation or warranty by AT&T and is subject to change. Mention of a specific company or entity is not an endorsement by AT&T.
Caching Enabled!!
30 second startup 2nd startup
© 2015 AT&T Intellectual Property. All rights reserved. AT&T, the AT&T logo and all other marks contained herein are trademarks of AT&T Intellectual Property and/or AT&T affiliated companies. All other marks contained herein are the property of their respective owners. Information contained herein is not an offer, commitment, representation or warranty by AT&T and is subject to change. Mention of a specific company or entity is not an endorsement by AT&T.
Major Snap Sales App
http://pixabay.com/en/woman-smartphone-chatting-girl-410320/
© 2015 AT&T Intellectual Property. All rights reserved. AT&T, the AT&T logo and all other marks contained herein are trademarks of AT&T Intellectual Property and/or AT&T affiliated companies. All other marks contained herein are the property of their respective owners. Information contained herein is not an offer, commitment, representation or warranty by AT&T and is subject to change. Mention of a specific company or entity is not an endorsement by AT&T.
World Famous Goat RacingWas $79, Now $39
User has to wait for image to load
Thumbnails or Dumbnails…
Downloading the same image twice, in different sizes, doesn’t make sense.Download it once and use it in multiple ways.
54 kb
198 kb
World Famous Goat Racing Was $79, Now $39
© 2015 AT&T Intellectual Property. All rights reserved. AT&T, the AT&T logo and all other marks contained herein are trademarks of AT&T Intellectual Property and/or AT&T affiliated companies. All other marks contained herein are the property of their respective owners. Information contained herein is not an offer, commitment, representation or warranty by AT&T and is subject to change. Mention of a specific company or entity is not an endorsement by AT&T.
Major Brick and Mortar Retailer
Image: http://pixabay.com/en/woman-shopping-happy-bags-dresses-169286/
© 2015 AT&T Intellectual Property. All rights reserved. AT&T, the AT&T logo and all other marks contained herein are trademarks of AT&T Intellectual Property and/or AT&T affiliated companies. All other marks contained herein are the property of their respective owners. Information contained herein is not an offer, commitment, representation or warranty by AT&T and is subject to change. Mention of a specific company or entity is not an endorsement by AT&T.
A Rolling Stone Gathers No Moss, but Spinning Wheels Kill Sales
http://www.lukew.com/ff/entry.asp?1797
“With the introduction of these progress indicators, we had made people watch the clock. As a result, time went slower and so did our app.”
~ Luke Wroblewski
© 2015 AT&T Intellectual Property. All rights reserved. AT&T, the AT&T logo and all other marks contained herein are trademarks of AT&T Intellectual Property and/or AT&T affiliated companies. All other marks contained herein are the property of their respective owners. Information contained herein is not an offer, commitment, representation or warranty by AT&T and is subject to change. Mention of a specific company or entity is not an endorsement by AT&T.
AT&T DriveMode
© 2015 AT&T Intellectual Property. All rights reserved. AT&T, the AT&T logo and all other marks contained herein are trademarks of AT&T Intellectual Property and/or AT&T affiliated companies. All other marks contained herein are the property of their respective owners. Information contained herein is not an offer, commitment, representation or warranty by AT&T and is subject to change. Mention of a specific company or entity is not an endorsement by AT&T.
AT&T DriveMode for Android
3 min
480 GPS connections/24hr~20% Battery
© 2015 AT&T Intellectual Property. All rights reserved. AT&T, the AT&T logo and all other marks contained herein are trademarks of AT&T Intellectual Property and/or AT&T affiliated companies. All other marks contained herein are the property of their respective owners. Information contained herein is not an offer, commitment, representation or warranty by AT&T and is subject to change. Mention of a specific company or entity is not an endorsement by AT&T.
AT&T DriveMode for AndroidUse activity APIs to identify when your users are in motion
11 minute trace
DetectedActivity.IN_VEHICLEDetectedActivity.ON_BICYCLE:DetectedActivity.ON_FOOT:
http://commons.wikimedia.org/wiki/File:Baby_goats_jan_2007_crop.jpghttp://commons.wikimedia.org/wiki/File:Goat_in_a_car.jpghttp://commons.wikimedia.org/wiki/File:2009_goat_and_bicycle_in_India_6512003001.jpg
© 2015 AT&T Intellectual Property. All rights reserved. AT&T, the AT&T logo and all other marks contained herein are trademarks of AT&T Intellectual Property and/or AT&T affiliated companies. All other marks contained herein are the property of their respective owners. Information contained herein is not an offer, commitment, representation or warranty by AT&T and is subject to change. Mention of a specific company or entity is not an endorsement by AT&T.
Major Sports Reporting Application
• Regular NFL season Sept 4 – Feb 1• 50,000,000 installs on just Android• So what would you expect the behavior of the app
to be in June at 2am?
https://www.flickr.com/photos/perspective/33330283/
© 2015 AT&T Intellectual Property. All rights reserved. AT&T, the AT&T logo and all other marks contained herein are trademarks of AT&T Intellectual Property and/or AT&T affiliated companies. All other marks contained herein are the property of their respective owners. Information contained herein is not an offer, commitment, representation or warranty by AT&T and is subject to change. Mention of a specific company or entity is not an endorsement by AT&T.
Major Sports Reporting Application
Pinging the network 3 minutes 24 x 7 x365Consider 70,000 fans in a stadium with this app trying to hit the network
© 2015 AT&T Intellectual Property. All rights reserved. AT&T, the AT&T logo and all other marks contained herein are trademarks of AT&T Intellectual Property and/or AT&T affiliated companies. All other marks contained herein are the property of their respective owners. Information contained herein is not an offer, commitment, representation or warranty by AT&T and is subject to change. Mention of a specific company or entity is not an endorsement by AT&T.
Advertising Could Be Killing Your App
• Consider your advertising partner like you would a business partner• Do they compress their content?• Do they close their connections? Caching?
© 2015 AT&T Intellectual Property. All rights reserved. AT&T, the AT&T logo and all other marks contained herein are trademarks of AT&T Intellectual Property and/or AT&T affiliated companies. All other marks contained herein are the property of their respective owners. Information contained herein is not an offer, commitment, representation or warranty by AT&T and is subject to change. Mention of a specific company or entity is not an endorsement by AT&T.
Most of these issues have already been fixed by the app developers we’ve worked with, but if you’re wondering why the tops apps are staying on top, it’s because they are constantly checking and watching all of these Best Practices.
Constant Improvement
© 2015 AT&T Intellectual Property. All rights reserved. AT&T, the AT&T logo and all other marks contained herein are trademarks of AT&T Intellectual Property and/or AT&T affiliated companies. All other marks contained herein are the property of their respective owners. Information contained herein is not an offer, commitment, representation or warranty by AT&T and is subject to change. Mention of a specific company or entity is not an endorsement by AT&T.
Don’t Over Think Performance Testing
http://www.glendaleinquiry.ca/2014/09/what-were-we-thinking.html
© 2015 AT&T Intellectual Property. All rights reserved. AT&T, the AT&T logo and all other marks contained herein are trademarks of AT&T Intellectual Property and/or AT&T affiliated companies. All other marks contained herein are the property of their respective owners. Information contained herein is not an offer, commitment, representation or warranty by AT&T and is subject to change. Mention of a specific company or entity is not an endorsement by AT&T.
The Excuses
Time
Usage
Why fix something that isn’t broken
Skill sets
Return on Investment
Cost
Everyone else does the same thing
Too much effort
Users haven’t complained
© 2015 AT&T Intellectual Property. All rights reserved. AT&T, the AT&T logo and all other marks contained herein are trademarks of AT&T Intellectual Property and/or AT&T affiliated companies. All other marks contained herein are the property of their respective owners. Information contained herein is not an offer, commitment, representation or warranty by AT&T and is subject to change. Mention of a specific company or entity is not an endorsement by AT&T.
Performance Testing Guidelines
http://smarterappsguidelines.gsma.com/
http://knowmyapp.org/
http://www.appqualityalliance.org/aqua-performance-test-criteria
http://developer.att.com/application-resource-optimizer
© 2015 AT&T Intellectual Property. All rights reserved. AT&T, the AT&T logo and all other marks contained herein are trademarks of AT&T Intellectual Property and/or AT&T affiliated companies. All other marks contained herein are the property of their respective owners. Information contained herein is not an offer, commitment, representation or warranty by AT&T and is subject to change. Mention of a specific company or entity is not an endorsement by AT&T.
• Free Tools:• Application Resource Optimizer• Network Attenuator• Trepn• Android
• Systrace• Battery Historian• Etc.
• Two tests (minimum)• 40 minutes total testing time
Keep it Simple
© 2015 AT&T Intellectual Property. All rights reserved. AT&T, the AT&T logo and all other marks contained herein are trademarks of AT&T Intellectual Property and/or AT&T affiliated companies. All other marks contained herein are the property of their respective owners. Information contained herein is not an offer, commitment, representation or warranty by AT&T and is subject to change. Mention of a specific company or entity is not an endorsement by AT&T.
What You Find Might Surprise You
Active Use Idle Use
What the User Wants
What the App Needs
Periodic Transfers
GPS
Text Compression
Duplicate Content Analytics and AdvertisingClosing Connections
© 2015 AT&T Intellectual Property. All rights reserved. AT&T, the AT&T logo and all other marks contained herein are trademarks of AT&T Intellectual Property and/or AT&T affiliated companies. All other marks contained herein are the property of their respective owners. Information contained herein is not an offer, commitment, representation or warranty by AT&T and is subject to change. Mention of a specific company or entity is not an endorsement by AT&T.
AT&T DefectsDuplicate Content
<5% of KB
5-20% of KB
20+% of KB
Periodic Connections
0-20% of Power
20-50% of Power
50+% of Power
GPS/BT
0-25% Active (s)
25-65% Active (s)
65+% Active (s)
Text Compression
100-500KB
500-1000KB
1000+KB
Closing Connections
0-20% of Power
20-50% of Power
50+% of Power
Sev 3
Sev 2
Sev 1
© 2015 AT&T Intellectual Property. All rights reserved. AT&T, the AT&T logo and all other marks contained herein are trademarks of AT&T Intellectual Property and/or AT&T affiliated companies. All other marks contained herein are the property of their respective owners. Information contained herein is not an offer, commitment, representation or warranty by AT&T and is subject to change. Mention of a specific company or entity is not an endorsement by AT&T.
• Encourage testers to move at a speed that emulates real users • Create flexible (yet reproducible) test cases
• Test cases can even reflect more of a feature checklist• The Trace is not about the order of the features, but about the content
• Don’t limit testing to 2 traces if additional are needed • Some streaming apps can have a trace with just streamed content • Some apps are most inefficient at startup
• Be Curious• If something seems odd, explore it • Screen rotation best practice came out of tester noticing the network
lighting up each time they turned the device
Trace Collections Hints
© 2015 AT&T Intellectual Property. All rights reserved. AT&T, the AT&T logo and all other marks contained herein are trademarks of AT&T Intellectual Property and/or AT&T affiliated companies. All other marks contained herein are the property of their respective owners. Information contained herein is not an offer, commitment, representation or warranty by AT&T and is subject to change. Mention of a specific company or entity is not an endorsement by AT&T.
http://bit.ly/HighPerfAndroidApps
© 2015 AT&T Intellectual Property. All rights reserved. AT&T, the AT&T logo and all other marks contained herein are trademarks of AT&T Intellectual Property and/or AT&T affiliated companies. All other marks contained herein are the property of their respective owners. Information contained herein is not an offer, commitment, representation or warranty by AT&T and is subject to change. Mention of a specific company or entity is not an endorsement by AT&T.
Q&A
http://smarterappsguidelines.gsma.com/
http://knowmyapp.org/
http://www.appqualityalliance.org/aqua-performance-test-criteria
http://developer.att.com/application-resource-optimizer
http://bit.ly/HighPerfAndroidApps
© 2015 AT&T Intellectual Property. All rights reserved. AT&T, the AT&T logo and all other marks contained herein are trademarks of AT&T Intellectual Property and/or AT&T affiliated companies. All other marks contained herein are the property of their respective owners. Information contained herein is not an offer, commitment, representation or warranty by AT&T and is subject to change. Mention of a specific company or entity is not an endorsement by AT&T.
Thank You