Embed Size (px)
Citation preview
TMOS Deep Dive
the secrets to customization and automation
Aaron HOOLEY – WW Global Solution Architect
Jason RAHM – Mgr, Community Solutions
Philippe CLOUP – EMEA Solution Architect
F5 Agility 2014 2
TMOS– The Application Proxy PlatformState of the Art Building Blocks
Performance and Scalability
• High Performance micro-kernel
• Modular architecture allows rapid integration
• CMP for linear scalability • Chassis and Appliance
level products• Hardware offload with
programmable FPGAs
Proxy and Protocols
• High Performance optimized TCP Proxy
• High Performance HTTPForward/Reverse Proxy
• Full High performance Network stack
• Application optimizations• Protocol tunneling
support• Custom protocols with
iRules
Application Fabric
• Scale-N Fabric • VCMP for multi-tenancy • Virtual Editions• Configuration consistency
across platforms and VEs • BIG-IQ Management• Flexible licensing options
Programmabilityand Management
• iRules – Protocol and platform level programmability
• iControl – Automation for BIG-IP Services
• iStats – Programmable statistics with iRules
• iCall – Programmable orchestration rules
F5 Agility 2014 3
BUSINESSrequires Applications to
become more agilte
SECURITYmust be everywhere, from
the network to the applications and the
management
OPERATIONSNeed visibility of the
resources at different levels (App, DC, Network)
NETWORKis changing faster to support
new applications and customers
APPLICATIONSare dynamic and customer
aware (profiling)
Programmability is a key element of evolving word
Control, Data,Management
Planeprogrammability
F5 Agility 2014 5
Control Plane
Data Plane
Third-Party Management
DevCentral User Community
TMOS Programmability Extend the platform to meet your application delivery needs
DataPlane : iRules
F5 Agility 2014 7
• iRules allow you to perform deep packet inspection (entire header and payload)
• Programming language integrated into TMOS (based on industry standard TCL language)
• Coded around Events (HTTP_REQUEST, HTTP_RESPONSE, CLIENT_ACCEPTED etc.)
What makes iRules tick?
•User Requests Data•Client’s connection is established
•Request data passed to “server”•Session Begins
Request
•On Request:•Virtual Server Processes Request
•iRules Request Events Executed
•Request info Processed, modifications made, persistence applied, etc.
•On Response:•iRule Response Events Executed
•Response Data Processed•Response sent to client
Processing
•Request Arrives at server (possibly modified)
•Server Processes as needed•Response data is sent to client
Response
F5 Agility 2014 8
• Multiple events, linked to different F5 Modules are fired by the system• Based on connection state,• Based on application state• adapt the solution to a business specific requirement
iRules – a bunch of events
SSL
Com
pres
sion
Server
TCP
TMM
iRules
Client Rat
e Sh
apin
g
ASM
Web
Acc
el
PEM
/CLA
SSIF
WO
M
ICA
P
UD
P
APM
/AC
CES
S
GTM
AFM
RO
UTI
NG
SIP/
MSG
S
LIST
ENER
L7 P
OLI
CY
ENG
DN
S
CG
NAT
L7 D
OS
DIA
MET
ER
F5 Agility 2014 9
iRules – Life in a Web session
F5 Agility 2014 10
• Inspection based routing to a pool or node• HTTP request redirection• Inspection based persistence• Protocol agnostic request or response modifications• Header inspection and modification• Custom user experience (retries, custom error pages, etc.)
• Selective and/or advanced content encryption• Session or connection rate limiting• Network offloaded client authentication• Conditional configuration activation (Selective SNATing, etc.)
iRules Common Tasks
F5 Agility 2014 11
Solving problems with iRules
• Application changes are costly• Making changes to an enterprise application is often extremely costly. iRules allows you to make needed changes quickly and easily, without
compromising your application, or inducing extensive cycles of re-testing.
• Large scale code management is time consuming• Managing any changes, especially frequent ones, to data distributed across possibly hundreds of systems is extremely time consuming and often
error prone. By consolidating the management of data into a single point, your LTM, iRules allows you to maintain a single copy of the data and make changes once.
• Scalability via hardware can be costly and sometimes even inefficient• iRules allows the network layer to become a part of your application and work with you, allowing you to offload portions of your application logic that
function much better and more efficiently on the LTM than they ever could on your back end servers.
• iRules can see it all• Given our placement in the network and our ability to do in-depth packet inspection on anything on the wire, there really isn’t anywhere else in the
application flow where you’re likely to have as much information about what’s going on or what just happened as you can with the LTM and iRules. Even when the application doesn’t have all the necessary pieces of data, and the client can’t see it all, iRules likely can.
ControlPlane: iApps
F5 Agility 2014 13
An Easy ButtonUse F5-developed iApps to
rapidly deploy popular applications with verified
and supported configurations.
Standards EnforcementiApps with strict updates,
enforce standards, reducing training and operational risk.
App OrchestrationStandardize your unique application deployments using iApps, iControl and
BIG-IQ.
A Single View AppManage all application
components in one place.
An App Lifecycle ToolUnlike other
template/wizard strategies, iApps are fully re-entrant,
can manage the full lifecycle of the application.
iApps provide different values depending on Application and Organization.
F5 Agility 2014 14
Implementation: TMSH/TCL
Presentation: APL
Help: HTML
iApp Templates under the hood
A formatted text file/script (.tmpl file) with three sections:
Implementation • Builds the config.• Written in TMSH and TCL.
Presentation• Defines the iApp wizard.• Written in APL.
Help • Documents the iApp.• Good, ol’ HTML.
F5 Agility 2014 15
The Template StructureWhat do they look like? Here is the template boilerplate code.
sys application template f5.template_name {actions {
definition {html-help {
<HTML help definition goes here>} implementation {
<Back-end TCL and TMSH code goes here>}presentation {
<APL GUI code goes here>}role-acl { admin manager resource-admin } limit the iApp to certain rolesrun-as admin elevate role during execution
}
}}
F5 Agility 2014 16
iApp Service Object
• The collection of variables input by the user and generated by the template for each instantiation of the iApp.
Reentrancy
• The notion that a template can be re-executed to add/modify/delete/alter the application state due to changes in the application environment.
Strictness
• Prevents an administrator from changing objects within the Application Service manually. Enabled by default.
• Avoid reentrancy with ‘Strict Updates’ disabled.
• It all depends on what properties were changed.• If you change a property that templates don’t touch, reentering the template will not cause a
problem.
Key ConceptsReentrancy, Service Objects and Strictness
ControlPlane: iCall
F5 Agility 2014 18
iCall Overview
• Automation Framework to dynamically respond to events on the TMOS control plane
• Integrated with TMOS • Shares common scripting tool set TMSH/Tcl• Leverages dataplane Information• No External Systems Required
• Comprehensive control of the BIG-IP configuration
• Vibrant community support DevCentral
F5 Agility 2014 19
iCall Components
Event(System Generated)
Handler(Triggered)
Script(Action)
Event(System Generated)
Handler(Triggered)
Script(Action)
• Detect Failover • Identify traffic spikes (Statistic Thresholds)• Monitor health score• User defined: Triggered, Periodic, Perpetual
• Monitor System Conditions, trigger scripts.
• Modify LB Ratios• Populate Pool Member attributes• Send notification• Create Qkviews
F5 Agility 2014 20
Pool Update on DHCP ResponseiCall automates to simplify operations
Server 1
Server 2
Physical / Virtual / Cloud
Server 3Event Listen for new DCHP requestsHandler Checks for conditionScripts Add new servers to pool
Event Listen for new DCHP requestsHandler Checks for conditionScripts Add new servers to pool
iCall Script
DHCP
New server is spun-up
Load Balancing Pool
Server 2
Server 1
Server 3iCall
New server is Automatically added to pool
ControlPlane: iStats
F5 Agility 2014 22
• User defined statistics• A place to store counts, strings, or rates• Global to the device
• Accessible by iRules in the data-plane and Scripts in the control-plane
• Defined in iRules, TMSH Scripts• Set, Get, or Increment by a integer
What are iStats? (Released in 11.1)
CONFIDENTIAL22
F5 Agility 2014 23
iStats: Why make it easy?
• Why have a simple name when you can have a Key?
• istats::set “<key>” <value>• key = “<keyspace> <object_name> <type> <name>”• Eg. istats::incr “ltm.pool /Common/OnlineBallotingPool counter hits” 1• The above iStat is associated with a specific object• iStats associated with specific objects are listed with TMSH show and in
select places in the GUI
CONFIDENTIAL23
F5 Agility 2014 24
[root@myBigIP:Active:Standalone] config # istats set "ltm.rule/Common/iStats_Count_Connections counter Connections » 0
iStats example
when CLIENT_ACCEPTED {log local0. "new connection established ... increasing counter"ISTATS::incr "ltm.rule /Common/iStats_Count_Connections c Connections" 1
}
when CLIENT_CLOSED {log local0. "connection closed... decreasing counter"ISTATS::incr "ltm.rule /Common/iStats_Count_Connections c Connections" -1
}
[root@myBigIP:Active:Standalone] config # istats get "ltm.rule/Common/iStats_Count_Connections counter Connections »18
Variable initialisation
Get the Variable Content
iRul
eP
opul
atin
gth
e va
riabl
e
F5 Agility 2014 25
iStats displayed on iApp page
CONFIDENTIAL25
F5 Agility 2014 26
More about iStats?
• https://devcentral.f5.com/tutorials/tech-tips/introduction-to-istats-part-1-overview
CONFIDENTIAL26
ManagementPlane :iControl
F5 Agility 2014 28
iControl
All above are correct!
Web Service ?
F5 Agility 2014 29
.NET Based Languages
Toolkit: Microsoft Visual Studio
Typical User: Application architect
Great for: Windows Applications, Web based ASP Apps, and Command line apps.
Java
Toolkit: Apache Axis2
Typical User: Application Architect
Great for: Web Based JSP apps.
iControl – SDK opened to multiple languages
Perl
Toolkit: SOAP::Lite Typical User: Network ArchitectGreat for: command line/script automation.
Others
PowerShellpyControl…
Based on SOAP/XML
F5 Agility 2014 30
SDKConstruct iControl request XML
Check the description form iControl SDK referenceCheck the description form iControl SDK reference
F5 Agility 2014 31
• Requires: Python, suds, bigsuds, iControl Assembly
iControl example
python>>> import bigsuds>>> b = bigsuds.BIGIP(hostname = '192.168.255.133')>>> b.LocalLB.Pool.get_list()[]>>> b.LocalLB.Class.get_address_class_list()['/Common/aol', '/Common/private_net']>>> b.LocalLB.Class.get_external_class_file_format(class_names = ['/Common/aol'])['FILE_FORMAT_CSV']>>> b.LocalLB.VirtualServer.get_list()['/Common/anyUDP', '/Common/Wildcard’]>>> b.LocalLB.VirtualServer.get_destination_v2(['/Common/anyUDP'])[{'port': 0, 'address': '/Common/0.0.0.0'}]>>> b.LocalLB.VirtualServer.get_destination_v2(['/Common/Wildcard'])[{'port': 0, 'address': '/Common/0.0.0.0’}]>>> b.LocalLB.VirtualServer.get_object_status(['/Common/Wildcard'])[{'availability_status': 'AVAILABILITY_STATUS_BLUE', 'enabled_status': 'ENABLED_STATUS_ENABLED', 'status_description': "The children pool member(s) either don't have service checking enabled, or service check results are not available yet »}]>>> b.LocalLB.VirtualServer.get_profile(['/Common/Wildcard'])[[{'profile_name': '/Common/tcp-lan-optimized', 'profile_context': 'PROFILE_CONTEXT_TYPE_SERVER', 'profile_type': 'PROFILE_TYPE_TCP'}, {'profile_name': '/Common/tcp-mobile-optimized', 'profile_context': 'PROFILE_CONTEXT_TYPE_CLIENT', 'profile_type': 'PROFILE_TYPE_TCP'}]]
Management Plane:
REST API
F5 Agility 2014 33
• An application programming interface (API) simply specifies how some software components should interact with each other
• Traditional APIs were SOAP/CRUD based using XMLor JSON – REST APIs are more standards based
Why REST? Why Now?
API Server
F5 Agility 2014 34
RESTful APIs have driven growth
F5 Agility 2014 35
• REST is based on the following simple ideas:
• REST uses URIs to refer to and to access resources
• Uses HTTP methods to change the state of resources:
What’s this REST stuff?
GET – retrieve details or a list of something
POST – create something on the server side
PUT – update something on the server side
DELETE – delete something on the server side
F5 Agility 2014 36
XML JSON<person><first name>Johnny</firstname><last name>Userguy</lastname></person>
{ "person": {
"firstname": “Johnny","lastname": “Userguy"
} }
And Who is this JSON guy?
JSON (JavaScript Object Notation) is simply a way of passing data to a web page in a serialized way that is very easy to reconstitute into a javascript object.
{"name":"bigip-1-1","protocol":"HTTP","port": "80"
}
JSON classes are built into every major javascript engine, so every browser has JSON encode/decode support.
F5 Agility 2014 37
What does an F5 REST call look like?
F5 Agility 2014 38
iControl REST API – How to start?
• Starting Point at DevCentral :• https://devcentral.f5.com/wiki/iControlREST.HomePage.ashx
• Download Documentation:• https://devcentral.f5.com/d/icontrol-rest-user-guide-version-1150?download=true
• Some good examples are available here:• https://devcentral.f5.com/wiki/iControlREST.CodeShare.ashx
F5 Agility 2014 39
iControl REST API – Direct Access
# curl -k -u admin:admin https://172.29.86.62/mgmt/tm/
{"items":[{"link":"https://localhost/mgmt/tm/cloud/ltm/node-addresses"},{"link":"https://localhost/mgmt/tm/cloud/ltm/pool-members"},{"link":"https://localhost/mgmt/tm/cloud/ltm/pools"},{"link":"https://localhost/mgmt/tm/cloud/ltm/virtual-servers"},{"link":"https://localhost/mgmt/tm/cloud/services/iapp/http_Charlie_61/health"},{"link":"https://localhost/mgmt/tm"},{"link":"https://localhost/mgmt/tm/shared/licensing/activation"},{"link":"https://localhost/mgmt/tm/shared/licensing/registration"},{"link":"https://localhost/mgmt/tm/cloud/templates/iapp"},{"link":"https://localhost/mgmt/tm/shared/sys/backup"},{"link":"https://localhost/mgmt/tm/shared/iapp/blocks"},{"link":"https://localhost/mgmt/tm/shared/iapp/health-prefix-map
• cURL
• Web Browser
• Browser Plug-In
F5 Agility 2014 40
REST API example – list selfip# curl -k -u admin:admin https://172.29.86.62/mgmt/tm/net/self/internal_self2 | sed s/,/,\\n/g
{"kind":"tm:net:self:selfstate",
"name":"internal_self2",
"generation":0,
"lastUpdatedMicros":0,
"selfLink":"https://localhost/mgmt/tm/net/self/internal_self2",
"partition":"/Common/",
"address":"10.81.60.2/8",
"floating":"disabled",
"inheritedTrafficGroup":"false",
"trafficGroup":"traffic-group-local-only",
"unit":0,
"vlan":"internal"}
Summary
Different layers require programmability
- control plane
- data plane
- management plane
and it is part of F5 technology since years.
Any further questions or help:
