View
7
Download
0
Category
Preview:
Citation preview
Load Testing with Gatling
Alan Baird QA Lead at LivingSocial
@bluegrasscoder abaird@bairdsnet.net
!https://goo.gl/Ka0PPr
How do you QA for:
Black Friday
Mother’s Day deal
New Feature rollout
Load testing!
JMeter
Gatling
Tsung
The Grinder
LoadUI
LoadRunner
Neoload
Which should you use?
¯\_(ツ)_/¯
My Reasons
• Gatling lets you write code
• It’s not XML
• Open Source
• I don’t work with Java developers, and there are Scala developers where I work
…other reasons• Nice charts!
• no heavy IDE (works with Vim!)
!
!
!
!
• yeah...there’s a recorder too
Gatling is…
…a load test DSL written in Scala !
it leverages Scala’s concurrency so it’s fast, efficient, and takes advantage of other functional things I don’t understand...
…but this is easy enough to understand
https://blog.flood.io/stress-testing-jmeter-and-gatling/
A Gatling Simulation is made up of Virtual Users
who carry out repeated Scenarios over time
Simulation
./bin/gatling.sh -s LoadTest
1 class LoadTest extends Simulation {! 2 ...! 3 }!
Scenario 1 val edit = exec(http("Form")! 2 .get("/computers/new"))! 3 .pause(1)! 4 .exec(http("Post")! 5 .post("/computers")! 6 .headers(headers_10)! 7 .formParam("name", "Computer")! 8 .formParam("company", "37"))! 9 }!
Virtual Users 1 setUp( ! 2 scenario1.inject(atOnceusers(1)),! 3 scenario2.inject(rampUsers(10) over (2 ! 4 seconds)),! 5 scenario3.inject(constantUsersPerSec(2) ! 6 over (10 seconds))! 7 )!
And now for an extended example...
The Pizza API
Where is your favorite pizza? A project to track the the best pizza places in the world!
(unless you like deep dish*) !
https://github.com/stevekinney/pizza
🍕🍕🍕🍕🍕🍕🍕🍕🍕🍕🍕* this repository’s hating on deep dish pizza does not represent the views of the author of this presentation
INDEX: returns all locations api/v1/pizzerias !SHOW: returns specific location by id api/v1/pizzerias/:some_pizzeria_id !SEARCH: returns all properties for valid params api/v1/properties/search?property=value
live at: http://pizza-dev.elasticbeanstalk.com*
*Please don’t break my server until after the presentation
1 class LoadTest extends Simulation {! 2 val httpConf = http! 3 .baseURL("http://pizza-dev.elasticbeanstalk.com")! 4 ! 5 val scn = scenario("Pizzerias")! 6 .exec(http("get all pizzerias")! 7 .get("/api/v1/pizzerias"))! 8 ! 9 setUp( ! 10 scn.inject(atOnceUsers(100)) ! 11 ).protocols(httpConf) ! 12 } !
A simple index test
Great...so we got the list page test coded,
what about individual pizzerias?
1 class Pizzerias2 extends Simulation {! 2 ! 3 val httpProtocol = http! 4 .baseURL("http://pizza-dev.elasticbeanstalk.com")! 5 ! 6 val scnA = scenario("Pizza Place 1")! 7 .exec(http("get Pizza Place 1")! 8 .get("/api/v1/pizzerias/1"))! 9 ! 10 val scnB = scenario("Pizza Place 2")! 11 .exec(http("get Pizza Place 2")! 12 .get("/api/v1/pizzerias/2"))! 13 ! 14 setUp(! 15 scnA.inject(constantUsersPerSec(10) during(30 seconds)),! 16 scnB.inject(constantUsersPerSec(10) during(30 seconds))! 17 ).protocols(httpProtocol)! 18 }! 19 ! 20 !
• some applications cache the answers to certain requests
• if the same request is made over and over again, the cache is used to reply quickly
• if you want to characterize the actual service, caching will skew your results
• if you are going for real world load, this might not be bad
Aside: caching
• We need a feeder
• Feeders are gatling’s way of setting up an iterator that will make data available to a Scenario in a map
• You can make an iterator yourself
• You can make an iterator from a CSV
• You can make an iterator from a JSON source
• and others...
How do we fix this?
1 Iterator.continually(Map("id" -> (Random.nextInt(133))))
1 class PizzeriasRandom extends Simulation {! 2 ! 3 val httpProtocol = http! 4 .baseURL("http://pizza-dev.elasticbeanstalk.com")! 5 ! 6 val feeder = Iterator.continually(Map("id" ->! 7 (Random.nextInt(133)))) ! 8 ! 9 val scnR = scenario("Pizza Place Random")! 10 .feed(feeder)! 11 .exec(http("get Pizza Place 1")! 12 .get("/api/v1/pizzerias/${id}"))! 13 ! 14 setUp(scnR! 15 .inject(constantUsersPerSec(2) during(60 seconds)))! 16 .protocols(httpProtocol)! 17 }
remember what the API looks like: !
SEARCH: returns all properties for valid params !api/v1/properties/search?some_param=some query
And now for search
1 class PizzeriasQuery extends Simulation {! 2 ! 3 val httpProtocol = http! 4 .baseURL("http://pizza-dev.elasticbeanstalk.com")! 5 ! 6 val scn = scenario("Pizza Place Searches")! 7 .exec(http("search for pizza place")! 8 .get("/api/v1/properties/search")! 9 .queryParam("city", "Oklahoma City"))! 10 ! 11 setUp(scn.inject(constantUsersPerSec(2) ! 12 during(10 seconds)))! 13 .protocols(httpProtocol)! 14 }! 15 !
1 property,value! 2 city,Oklahoma City! 3 city,Boston! 4 city,Dallas! 5 city,Atlantis!
1 class PizzeriasQueryWithCSV extends Simulation {! 2 ! 3 val httpProtocol = http! 4 .baseURL("http://pizza-dev.elasticbeanstalk.com")! 5 ! 6 val searches = csv("pizza_search.csv").circular! 7 ! 8 val scn = scenario("Pizza Place Searches")! 9 .feed(searches)! 10 .exec(http("search for pizza place")! 11 .get("/api/v1/properties/search")! 12 .queryParam("${property}","${value}"))! 13 ! 14 setUp(scn.inject(constantUsersPerSec(2) ! 15 during(10 seconds)))! 16 .protocols(httpProtocol)! 17 } !
Debugging 1 <?xml version="1.0" encoding="UTF-8"?>! 2 <configuration>! 3 ! 4 <appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender">! 5 <encoder>! 6 <pattern>%d{HH:mm:ss.SSS} [%-5level] %logger{15} - %msg%n%rEx</pattern>! 7 <immediateFlush>false</immediateFlush>! 8 </encoder>! 9 </appender>! 10 ! 11 <!-- Uncomment for logging ALL HTTP request and responses -->! 12 <!-- <logger name="io.gatling.http.ahc" level="TRACE" /> -->! 13 <!-- <logger name="io.gatling.http.response" level="TRACE" /> -->! 14 <!-- Uncomment for logging ONLY FAILED HTTP request and responses -->! 15 <!-- <logger name="io.gatling.http.ahc" level="DEBUG" /> -->! 16 <!-- <logger name="io.gatling.http.response" level="DEBUG" /> -->! 17 ! 18 <root level="WARN">! 19 <appender-ref ref="CONSOLE" />! 20 </root>! 21 ! 22 </configuration>!
1 Request: ! 2 search for pizza place: OK ! 3 =========================! 4 Session:! 5 Session(Pizza Place Searches,1480189078212752348-! 6 19,Map(property -> city, value -> Atlantis),14421! 7 13956702,0,OK,List(),<function1>)! 8 =========================! 9 HTTP request:! 10 GET http://pizza-dev.elasticbeanstalk.com/api/v1! 11 /properties/search?city=Atlantis! 12 headers=! 13 Connection: keep-alive! 14 Host: pizza-dev.elasticbeanstalk.com! 15 Accept: */*! 16 =========================! 17 HTTP response:! 18 status=! 19 200 OK! 20 headers= ! 21 Content-Type: [application/json]! 22 Date: [Sun, 13 Sep 2015 03:14:12 GMT]! 23 Server: [nginx/1.6.2]! 24 X-Content-Type-Options: [nosniff]! 25 Content-Length: [2]! 26 Connection: [keep-alive]! 27 ! 28 body=! 29 []!
Validation 1 class PizzeriasQueryWithCheck extends Simulation {! 2 ! 3 val httpProtocol = http! 4 .baseURL("http://pizza-dev.elasticbeanstalk.com")! 5 ! 6 val searches = csv("pizza_search.csv").circular! 7 ! 8 val scn = scenario("Pizza Place Searches")! 9 .feed(searches)! 10 .exec(http("search for pizza place")! 11 .get("/api/v1/properties/search")! 12 .queryParam("${property}","${value}")! 13 .check(bodyString.not("[]")))! 14 ! 15 setUp(scn.inject(constantUsersPerSec(2) ! 16 during(10 seconds)))! 17 .protocols(httpProtocol)! 18 }! 19 !
Special Cases Or, how to know when your tests aren’t measuring what you think
they’re measuring
• Excessive Queuing
• Testing caching instead of your service
• Cannot assign requested address errors
Excessive Queuing
• Symptom
• You are sending more packets than your system says it’s receiving
• You don’t see any errors
• Fix
• Back off the rate you are sending
Testing caching instead of your service
• Symptom
• system responds faster than expected no matter what load you put on it
• Fix
• vary the types of requests so they are more random
Cannot assign requested address errors
Symptom
• You get “Cannot assign requested address” errors during a load test
• Other out of memory errors
Fix
• see note about how Gatling uses ports on the host OS: http://gatling.io/docs/2.1.7/general/simulation_setup.html?highlight=simulation#injection
• Follow tuning guidelines: http://gatling.io/docs/2.1.7/general/operations.html?highlight=tuning#os-tuning
• use a repeat statement in your scenarios to make connections last longer
How to get help with Gatling
• Documentation: http://gatling.io/docs/2.1.7/index.html
• Gatling User Group:
https://groups.google.com/forum/#!forum/gatling
The Pizza API and my Gatling repo
• pizza api: https://github.com/stevekinney/pizza
• gatling examples: https://github.com/abaird/gatling_examples
• running in Elastic Beanstalk: ??
• script to setup loadtest machine in AWS: ??
Happy Loadtesting!
Recommended