How AppDynamics is shaking up the synthetic monitoring product space Ian Withrow – Group Product Manager End User Monitoring
History of synthetic
Synthetic is old. Very Old. • Keynote Systems
dates to the 90s. Think about that.
• You probably had dialup at the time
Copyright © 2015 AppDynamics. All rights reserved. 2
Traditionally synth has been built with capex
Copyright © 2015 AppDynamics. All rights reserved. 3
Source: Miran Rijavec (flickr) Source: gettheshot75 (flickr)
So here’s our checklist we’ll need for the synthetic product of the future
Copyright © 2015 AppDynamics. All rights reserved. 6
1 • We need compelling data that no one else has
• It has to run on browsers users actually use
3 • Scripting is painful, we need to make it less so
4 • Global footprint day one
• Combine together with other AppD products to create new value
Sources: *n3wjack's world in pixels (code), Caleb Roenigk (pencil), Hanna Sorensson (globe) (flickr)
Fortunately it’s not the 90s anymore
• Cloud computing, web standards, and open source technology have significantly lowered the barriers to entry.
• Going forward success in synthetic will be about innovation and execution. Not capital investment.
Copyright © 2015 AppDynamics. All rights reserved. 7
Open Source Tech: WebPageTest A real browser testing agent Built and maintained by Patrick Meenan at Google Tool of choice for front end developers when building websites Here are just a few reasons we picked it…
Copyright © 2015 AppDynamics. All rights reserved. 9
Can we do better than onload?
“The worst measure of user experience, except for all the other options”
Copyright © 2015 AppDynamics. All rights reserved. 10
WPT Benefit #1: Visual based timings
Copyright © 2015 AppDynamics. All rights reserved. 11
Visual complete time – How long it took to load everything above the fold
Speed Index – How much of above the fold content is complete overtime
Source: Randen Pederson (eye), Sean MacEntree (speedometer) (flickr)
WPT Benefit #2: High quality browser support
Copyright © 2015 AppDynamics. All rights reserved. 14
IE Support Version 6, 7, 8, 9, 10, and 11
Chrome/Firefox Latest versions
Others Safari (Windows)
Mobile Emulated mobile chrome, latest
One – 2 year wait for new versions
Not quite latest, e.g. Chrome 42
Emulated Old version of webkit
Webpagetest Legacy vendors
Webpagetest has the broadest browser coverage in the industry - handily
As a result We have Chrome Firefox, and IE11 at GA Time to market accelerated Quality is vetted by Google & thousands of website developers
Copyright © 2015 AppDynamics. All rights reserved. 15
WPT Benefit #3: Other features already built
• Film strip/video view • Screenshots • Connection Usage View • TCPDUMP • Export to HAR format
Copyright © 2015 AppDynamics. All rights reserved. 16
Webpagetest does have one feature gap
• No scripting… which is important
# of pages per visit
<4 4 to 6 7 to 10 > 10
Copyright © 2015 AppDynamics. All rights reserved. 17
Source: Dimensional Research 2015
We know how users feel about scripting
Copyright © 2015 AppDynamics. All rights reserved. 18
Source: Andy Blackledge (flickr)
Fortunately there is a standard: webdriver What is it? • W3C standard for remotely driving a browser via API calls
• Selenium 2.0 is a common example of webdriver
Copyright © 2015 AppDynamics. All rights reserved. 21
SE Benefit #1: Selenium scripts are easier
• Proprietary scripts, hard to edit.
• Selenium is: – Well established – Skill people put on resumes
Copyright © 2015 AppDynamics. All rights reserved. 22
SE Benefit #2: Continuing integration via API
The best script is the one you don’t have to write yourself! • QA probably does already
• Phase 1: Sync via our API • Phase 2: Push release • Phase 3: Profit
Copyright © 2015 AppDynamics. All rights reserved. 23
Source: Martin Fisch (flickr)
Having the best agent is pretty exciting but we need a network How should we do this? • Raise millions of dollars &
build out pops one business case at a time?
• Wait what about the cloud?
Copyright © 2015 AppDynamics. All rights reserved. 24
Source: Lenny DiFranza (flickr)
Cloud has been focused on few high scale locations The AWS model • A few high volume
locations • Microsoft, Google, etc
used the same cities
• Synthetic needs capillaries not arteries
Copyright © 2015 AppDynamics. All rights reserved. 26
Source: Matthew Hadley (flickr)
Cloud Benefit #1: Geo-distributed IaaS is here
Two Models: • Owned and operated - IBM
Softlayer • Federated –ComputeNext
• We are on board with both models
Copyright © 2015 AppDynamics. All rights reserved. 27
Source: Hanna Sorensson (flickr)
This is just the beginning
These vendors all have roadmap for more locations • Since our integration is based on API driven deployment
and their APIs are standardized that means fast adoption
Source: Sascha (flickr)
Cloud benefit #2: Flexibility
Copyright © 2015 AppDynamics. All rights reserved. 32
Source: SNIJLAB Rotterdam (flickr)
Now let’s look at what the future could hold
User Traffic
RUM Session
Path Analysis
Script Generated
Copyright © 2015 AppDynamics. All rights reserved. 35
Source: Uncalno Tekno (recorder), Dan Zen (paths) (flickr)
Idea #1: RUM simplifies synthetic setup
Copyright © 2015 AppDynamics. All rights reserved. 36
Source: wetwebwork (flickr)
Idea #2: Dynamically manage synthetic jobs
Copyright © 2015 AppDynamics. All rights reserved. 37
Source: Kevin Megens (flickr)
Idea #3: Continuous integration
Channel user behavior back to QA repo for release testing!
• Reduce surprise nasty regressions
• Removes a lot of manual work from the process
Copyright © 2015 AppDynamics. All rights reserved. 38
Putting it all together
Copyright © 2015 AppDynamics. All rights reserved. 39
• Visual based metrics - Measure and benchmark what users experience
• Browser support - Use the same tools your users have
• Standards based scripting - Spend as little timing scripting as possible
• Cloud infrastructure - Continually expanding network and flexibility
• Continual integration - Synthetic and RUM together can do amazing things
Source: Randen Pederson (eye), Lenny DiFranza (clouds), Kevin Megens (machine) (flickr)
Excited? Limited program for early adopters
• Until end of year • Limited to 10 customers in
NA, EU, and 5 ROW • Special price • 2 free days of professional
services to get you started • Virtual user groups with
other program members
Copyright © 2015 AppDynamics. All rights reserved. 40
Source: Eric Danley(flickr)
Text example
Lead-in sentence goes in this spot. Arial font, no period at the end • First level demotion has a round bullet
– Second level demotion has a standard en-dash • All demotion levels should use the same font size
Copyright © 2015 AppDynamics. All rights reserved. 42
Note: Slide titles are in sentence case - NOT IN UPPERCASE - Not In Title Caps
Note: Page numbers should appear on every page with limited exception