PRESENTED BY:

Bots Rule the Internet

30% Malicious

53% Automated Traffic

53%

reference

30%

ASMPR

OAC

TIVE BOT

DEFEN

SEBO

T SIG

NATU

RES

BEHAVIOR-BASEDWeb Scraping, Credential Stuffing, Human Detection

CAPABILITIES CHALLENGE

JS CHALLENGE

IP/DOMAIN VALIDATION

BOT SIGNATURESSIMPLE BOTScURL, ApacheBench, Nikto, NESSUS

IMPERSONATING BOTSGoogleBot???, Safari???, FireFox???

COOKIE/JS-ENABLED BOTSPhantonJS, SlimerJS, Selenium, HTMLUnit

AUTOMATED BROWSERS

••

••

••••

••

User-Agent: Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)

GoogleSource: 66.249.66.1 Bypass PBD

Bot Signature PBD Bypass

1. Signature has domain for validation2. Source IP/Domain is ACTUALLY validated3. Signature Category is set to “Report”

(This is the default for Search Engine category)

OR….1. Signature is in a “Benign” category2. Signature Category is set to “Report”3. DB variable dosl7.proactive_defense_exclude_benign_bots is set to

“enabled” (default is disabled).

All three conditions must be met!

security dos bot-signature "/Common/ADmantX Platform Semantic Analyzer" {category /Common/Crawlerrisk narule "headercontent:\"ADmantX Platform Semantic Analyzer\"; useragentonly; nocase;"user-defined false

}

security dos bot-signature "/Common/Google Keyword Suggestion" {category /Common/Crawlerdomains { .google.com }risk narule "headercontent:\"Google Keyword Suggestion\"; useragentonly; nocase;"user-defined false

}

•• •

•

• Block Suspicious Browsers

•

•

• Cross-Domain Requests

•

USER

BRO

WSER

SERVER

BIGIP

Initial Web Page AccessHTTP Request (no cookie)

Client-Side JS Challenge

Resend Request (with cookie)

Send Original HTTP Request

HTTP Response (main page)HTTP Response (main page)

Request objects (with cookie)

Requests for objects

Object ResponsesHTTP Response (main page)

Display page to user

••••

•••••

•••

•••••

••••

•••

••••••

Redirect challenge - Can client follow redirect AND maintain cookie stateJS-free test - HTTP/1.1, Keep-Alive, Language, other Header check

BIGIP

Site1

Site2

Browser

Set Cookie For Site1

Set Cookie For Site2

BIGIP

Site1

Site4

Browser

Set Cookie For Site1

Set Cookie For Site2

Site3

Site2

Set Cookie For Site3

Set Cookie For Site4

BIGIP

Site1

Site2

Browser

Set Cookie For Site1

Set Cookie For Site2

sys db variable Default Description

dosl7.proactive_defense_fictive_url /TSPD/ Prefix of the fictive URL that is used in the Proactive Bot Defense. The slashes at the beginning and end are required.

dosl7.proactive_defense_cookie_name TSPD_101 Name of signed cookie marking the request as validated. This cookie is global (not per VS), and the cookie name is as-is: not a prefix

dosl7.proactive_defense_prefix TSPD In some cases, some intermediate cookies and parameters are set. This is the prefix of their names.

dosl7.proactive_defense_validate_ip enable Allows disabling the validation of the client IP address in the cookie

dosl7.proactive_defense_validation_percent 100 Percentage of requests on which the signed cookie is validated (may be lowered to improve performance)

dosl7.proactive_defense_excluded_headers (empty) Comma-separated list of request headers that cause the Proactive Bot Defense to be bypassed

dosl7.proactive_defense_exclude_benign_bots disable Exclude bots from benign categories from PBD challenges

sys db variable Default Description

dosl7.browser_legit_min_score_captcha 60 Minimum score at which suspicious browser challenge will challenge with CAPTCHA (if enabled)

dosl7.browser_legit_min_score_drop 120 Minimum score at which suspicious browser challenge will block (if JS-challenge is possible)

dosl7.browser_legit_min_score_jsfree_drop 100 Minimum score at which suspicious browser challenge will block (if JS-challenge is NOT possible. i.e. cross-domain non-html resource)

dosl7.cors_ajax_urls None a comma-separated list of wildcard-supported URLs. The URLs in this list are HTML pages from which CORS AJAX requests could be sent.

dosl7.cors_font_urls None a comma-separated list of wildcard-supported URLs. The URLs in this list are CORS AJAX-requested fonts. (i.e. /t/cors/font/style.css,/t/cors/font/font.otf)

sys db variable Default Description

dosl7.max_normalization_cycles 2 The amount of normalization cycles done on uri before matching attack signatures.

dosl7.max_lookup_length 255 The maximum length in characters in which signature is searched for.Applies to both URLs and User-Agent strings.

dosl7.max_user_agent_occurrences 1 The maximum number of User-Agent header occurrences in whichthe signatures is searched.

dosl7.max_num_headers 50 Maximum number of headers in which the User-Agent string is looked for.

•

•

•

•

1.

2.

3.

sys db variable Default Description

dosl7d.shun_list enable Whether to use the shun list to block IP addresses

dosl7d.min_challenge_success_ratio 10 The minimum percentage of good transactions per IP address (or else add it to the shun list).

dosl7d.min_challenge_rps 10 The minimum requests per second before the system will apply shun mitigation

dosl7d.shun_prevention_time 120 The time in seconds to keep the IP address in the shun list.