Upload
ca-technologies
View
48
Download
0
Embed Size (px)
Citation preview
World®’16
CaseStudy:EmbratelDeliverstheGoldMedalforInfrastructureMonitoringatRio2016
WandersonCoelhodeSouza- OperationsManager- Embratel
DO4X121S
DEVOPS– AGILEOPERATIONS
2 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
©2016CA.Allrightsreserved.Alltrademarksreferencedhereinbelongtotheirrespectivecompanies.
Thecontentprovidedinthis CAWorld2016presentationisintendedforinformationalpurposesonlyanddoesnotformanytypeofwarranty. The informationprovidedbyaCApartnerand/orCAcustomerhasnotbeenreviewedforaccuracybyCA.
ForInformationalPurposesOnlyTermsofthisPresentation
3 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
Abstract
In2016,RiodeJaneirowillbecomethefirstSouthAmericancitytohosttheOlympicandParalympicGames.CapitalizingonitsexperiencehostingthePanAmericanGamesin2007,andtheWorldCupin2014,the‘MarvelousCity’surpassedChicago,TokyoandMadrid.TheprojectisbudgetedatBrazilianReal28.8bn($7.6bn).ThispresentationwillsharetheexperienceofhowtoplanandoperatetheOlympicInfrastructureusingCAUUnifiedInfrastructureManagement.Therewillbe10typesofservice,includingdatacenters,internetaccess,high-capacitydata,voice,mobility,CATVandwifi.Duringtheevent,RiodeJaneirowillhostrepresentativesfrom206countriesandmillionsofvisitors,anditisestimatedthatmorethan21,000journalistswillcovertheevent– allofwhichrequiredanunprecedentedtelecomsandITinfrastructure.TheRio2016OlympicGameswillusemorethan60,000networkpointsconnectingalltheOlympicarenas,andapproximately18,000fixedlines,16,000SIMcards,threemillionSMSmessagesand15,000pointsofCATV.TheOlympicbackbone(370km)isequivalentto971lapsofastandardfootballpitch,orthelengthof6,800Olympicswimmingpools.
WandersonCoelhodeSouzaOperationsManagerEmbratel
4 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
Agenda
OLYMPICSGAMESINFRASTRUCTUREREQUIREMENTOVERVIEW
ROLEOFCAUNIFIEDINFRASTRUCTUREMANAGEMENT(CAUIM)
KEYDASHBOARDS
MOBILEAPPLICATIONUSECASE
1
2
3
4
32
41
BARRA
DEODOROMARACANÃ
COPACABANA
OLYMPIC GAMES 42PARALIMPIC GAMES 23
65 CHAMPIONSHIPS44 Test events4 Ceremonies (openings and closures)100 DAYS Torch relay across the country
CHAMPIONSHIPS IN 12 DAYS
CHAMPIONSHIPS IN 17 DAYS
+
4.8Bworldwideviewersin220countries30Kjournalists70Kvolunteers15Kathletes206countries9Mtickets300Kaccreditations
Alltheadminnetworks,gamesandwebhostingwerehostedinEmbrateldatacentersandmonitoredwithCAUIMtechnology
1600 servers (physical/virtual)40 network DC devices1 Pb used data54 M on real-time games processing
Thewholeenvironmentmonitored24x7
Olympicvenuesinfrastructure
33,380hardwaremodules113,472LANports440UCSservers5,159wifi accesspoints177securitydevices(firewalls,IPS,IDS)
Whatwashostedatthedatacenter
ü Accesscontrolcoreequipmentofallvenues(ACS)ü ControllingaccesstotheAPsofallvenuesü Accesscontrolforallusersofthegames,forwiredand
wirelessnetworks(ISE)ü DHCPserver/DNS/VPNü Firewalls,IPS,IDS,balancerstrafficü AdminRio2016network(servers/storage/backup)ü Gamesnetwork- gameinformationdisclosuresystem
(servers/storage/backup)
Duringthegames,approximately2,200 ticketswereopenedforthedatacentersupportteam
SLA/GamesTime ResponseTime FixTime
Severidade1 5min(90%) 1hr(90%)
Severidade2 5min(90%) 2hr(90%)
Severidade3 30min(90%) 4hr(90%)
Severidade4 1hr(90%) 6hr(90%)
ServiceRequest 1hr(90%) 8hr(90%)
AggressiveSLAlevelswereagreedtocovernormaloperatingandcriticalperiods
• Events test• Technical simulations• Games time
CAUnifiedInfrastructureManagementUsedtomonitorthestatusofallgameserversandnetworkdevicesinvenuesforRio2016
CAUIMwasusedbecauseitisascalable,easy-to-use,cloudresilient,andpowerfulITmonitoringtoolandprovides360degreevisibilityintosystemsandinfrastructureperformance
CA Unified Infrastructure Management (CA UIM)Providespossibilityofcontrollingandmonitoringthefollowingparameters:
ü System Monitoringü %CPU usage, processor queue length, processor timeü Disk usage (% andMb), disk time, disk transfers/read/write per sec, pagingü Memory usage (% andMb)ü Windows services, Windows event logsü Scanning directories, files
ü Network Monitoringü SNMP queries are sent towards network equipment (switches and firewalls)ü Interface traffic from network equipmentü ICMP ping of all the devices, servers and VMs
ü Application Monitoringü URL availabilityü Jboss/JAVAü Application processes (CPU usage, memory usage, threads);ü Application servicesü SQL/Oracle
Applications Support Team
CreationofinteractivedashboardswithCAUIMwaskeytoproactiveissuedetection/prevention
The badges below represent some of the manyportals supportedNote that one of the badges has a major (orange) alarm; the team at TOC24x7 with the help of the right screens and alerts in front of them, was ableto verify with agility and act quickly to solve the problem
WithCAUIM,itwaspossibletoverifythemetricsinfothroughthemainmanagerinterface
MIBshavebeendefinedformonitoring(CPU,interfaces,uptime,etc.)AllappliedinSNMPcommunities,includingtheloadbalancerappliances
Probesandthresholdswereessentialingamestime,providingassertiveroleinidentifyingtherootcauseandproblemresolution
VMwareVMax LoadAverage
Database
It waspossibletomonitorthe
VmotionsandDRSsofVMs.
Identifydiskfailures,generatingalerts,even
beforetheidentificationinthestoragetoolitself
1-5minutes,identifyquicklythegrowthoftheapplicationqueues
Transactionslogandincreaseoftablespace
StatisticsfromAug.5toAug.17ü MobileApp
ü Totalvisitors:12,768,127ü Totalpageviews:781,877,340
ü Overallü TotaltrafficintheRio2016backbone:1,794PBsü Totalinternettraffic:632TBsü Uniquedevicesinthenetwork:155,226ü Peakofsimultaneouslyconnectedwifi devices:14,520ü Peakofinternettraffic:5.35Gbps
CAUIMhelpedtrackuserexperienceissueswithmobileapplication
ü The mobile application launch to follow the torch relayü Internet statistics showed that every year the internet consumption has undergone access via desktop
to mobile devices; in March 2012 the number of users was 2,336,000 in December 2015 around 3.366million and estimated growth of approximately 51% to games time
ü Mobile application launch for games timeü Withthenewapplication,itwaspossibletotracktheresultsofonlinegames,receivebreakingnews,
informationonVenues,athletesandmonitoringoftheentireevent,usingmobile
LoadBalanceTraffic InternalNetworkTraffic
OracleMemoryUtilization AllErrors@CDN
ContinousCAUIMdatasharingw/appdevfromAug.10-14resolvedseveralperformanceissues
23 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
SummarybenefitswithCAUIM
BETTERGAMESANDAPP
EXPERIENCE
SINGLEVIEWINTORIOOLYMPICSITINFRASTRUCTURE
IMPROVEDMOBILE
APPLICATIONPERFORMANCE
PROACTIVEISSUERESOLUTION
RAPIDROOTCAUSEANDTRIAGE
TicketsResolvedDuringRioOlympics
2200
24 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
Thankyou.
Stayconnectedatcommunities.ca.com
25 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
DevOps– AgileOps
FormoreinformationonDevOps– AgileOps,pleasevisit:http://cainc.to/wYXSg6