Upload
arnold-godley
View
215
Download
0
Embed Size (px)
Citation preview
Jasig Sakai Conference 1
High Availability in Hurricane Alley
Multi-site multi-node CASDeep in the Heart of Texas
Srinivas Varadaraj & Bill Thompson
Jasig Sakai Conference 2
Agenda
1. Strategy2. Technical requirements3. Constraints4. Stuff at hand5. Architectural decisions6. Cluster & production architecture7. Challenges and solutions8. Multi-site routing9. Production experiences10. Questions & Comments
Jasig Sakai Conference 3
Strategic requirements
Single Identity
Single Sign On/ Single Sign Off
Maximize self service tools
Improved user experience
Jasig Sakai Conference 4
Technical requirements
• Application Compatibility• High Availability• Rolling maintenance• Transparency • Scalability• AD integration• Customization(branding)
Jasig Sakai Conference 5
Constraints
• Limited budget , use existing resources.– Power in the datacenters– Single internet – High latency connectivity
• Limited in-house development & experience– Stay close to release code
• Aggressive timeframe
Jasig Sakai Conference 6
Stuff we had at hand
• SAN infrastructure with replication to DR• VM clusters• Site-to-site VPN based connectivity to DR• F5 loadbalancers • Dedicated firewalls • Opportunity
Jasig Sakai Conference 7
Decisions ! Decisions ! Decisions !
• Virtual Machines• SAN based storage• The great ticket registry debate• To replicate tickets or NOT !• Building by cloning• “Appliance” like• SSL Local vs Offloading • Cluster VS Standalone application servers• Timeout !
Jasig Sakai Conference 8
Cluster components
Jasig Sakai Conference 9
Final Architecture
Jasig Sakai Conference 10
“Holy troubles, Batman!”
• SSL offloading– Tomcat offloading workaround
• Authentication and Validation persistence– User and application can go to either site.– Enter site identifiers
• Multi-site ticket replication.– Latency in WAN
• Algorithm usage in phpCAS clients and Java CAS clients
• Slow performance of mod_auth_cas on VMs
Jasig Sakai Conference 11
Routing logic
• HTTP_REQUEST• HTTP_REQUST_DATA• HTTP_RESPONSE
Jasig Sakai Conference 12
HTTP_REQUEST(Request from the client)
HTTP_REQUEST{1) Grab header length to determine payload size2) If both sites are down, redirect to a branded
service unavailable page3) If URI has siteID of other site and other site is up, route to other site4) Otherwise default route to local site}
Jasig Sakai Conference 13
HTTP_REQUEST_DATA(Payload manipulation)
HTTP_REQUEST_DATA{1) Grab <samlp:AssertionAtrifact> from payload , this may contain siteID2) if we have a siteID of the other side { If the siteID is Loadbalancer introduced { blank the loadbalancer extension} Route to other side else { if we have a siteID of the local side { If the siteID is Loadbalancer introduced { blank the loadbalancer extension} Route to local side }}
Jasig Sakai Conference 14
HTTP_RESPONSE(Response from the server)
HTTP_RESPONSE{1) Grab server’s response headers2) If SiteID is not in the response header { Introduce a loadbalancer siteID to compensate for java CAS client} Release HTTP to client}
Jasig Sakai Conference 15
Jasig Sakai Conference 16
Experiences in Production
• Approx. 8 months in production• 7 Applications in production 10 in development• Survived two power outages at DR• Survived multiple internet outages• Successful rolling upgrades to MySQL & CAS• Flow based redesign.• LPPE • Re-visit ticket registry
Jasig Sakai Conference 17
Questions/Comments
• Credits:– CAS developers and community– F5 & F5 devcentral– Unicon– LU & Txstate
• Thank you for your time !!• Contacts:– Sri: [email protected]– Bill: [email protected]