Another way to monitor performance of your site with Selenium and log analyzer
Citation preview
1. Synthetic performance monitoring with SeleniumAndriy
Samilyak OpsWay CEO
2. Who is talking? Working in IT since 2005 Since 2008 - Smile
Ukraine CTO Since 2013 OpsWay CEO Coaching with SmartMe.com.ua 3.
You know this... Metrics show that a 0.1 second delay in page
rendering can translate into a 1% drop in customer activity Amazon
research papers 4. Projects are changed daily Application
codeProduct databaseMarketing tagsServer configurationExternal
servicesetc and no QA engineers at all... 5. User checkout cycle
under control Add to cartSelect productSuccess page!Login~30-50
secBilling/Shipping optionsPayment 6. Is it still running smoothly?
Passive checks analyzing end-user activity Webserver log analysis
NewRelic Web analytics (real user monitoring)Active checks run test
and record results Gomez GPNUptrendsPingdom 7. NewRelicCool, sexy,
not so cheap and not really what we need 8. GoogleAnalytics with
UserTiming https://github.com/opsway/magento-mod-rum but we need
immediate alerts! 9. Server-side log analysis Complex models: 2 add
to cart requests timing1 login step timing5 checkout steps
timingsnot comparable side by sidenot representative when user
activity is low 10. Server-side logs: false-positive3 mins to
checkout? Are you serious? 11. Pingdom/Uptrends Transaction
monitoring via scriptingFancy scripting languageRobust, popular but
not so cheap for multiple projects Fine for public sites, but not
for intranet 12. We liked Compuware Gomez! 13. Compuware Gomez is
cool 14. But we have already Selenium! SeleniumSplunk1. Load
product pageSend timestamp via UDP2. Add to cartSend timestamp via
UDP3. Shopping cartSend timestamp via UDP4. Address 5. Payment 6.
Success page... 15. It looks like this in code... 16. Splunk
combines to transaction... 17. We can chart transactions 18. and
alert on abnormal behaviorSlow transaction Failed Transaction! 19.
Why did transaction fail? Selenium reported timeout not always
sufficientRarely reproducible (many servers)We needed screenshots!
20. Capturing a screenshot 21. Saving screenshot with Jenkins 22.
Jenkins build with errorsWe got it! 23. Why it took so long? We
don't know always what http request blocked transaction Inspired by
Gomez we integrated HAR recording of test sessions 24. Inspired by
Gomez ... 25. HAR is not perfect, but ... 26. HAR proxy Based on
BrowserMob proxy server http://bmp.lightbody.net/ HAR fileLaunching
WebDriver Selenium Grid Proxy on port 64001 Request passes through
proxywww.mysite.comSelenium node (Firefox) 27. Jenkins build with
errorsHere we have HAR 28. Issues we have Supporting test scripts
takes a lot of timeSupport test infrastructure is
tediousFalse-positivesStill not enough debug information 29. Andriy
Samilyak [email protected] skype: samilyaka http://opsway.com
