Upload
greg-hanchin
View
337
Download
1
Embed Size (px)
Citation preview
Copyright*©*2012*Splunk*Inc.*
Thinking*Beyond*Applica=on*Monitoring*Jus=n*Hardeman,*Unix*Administrator*Availity*Health*Informa=on*Networks*
Legal*No=ces*During*the*course*of*this*presenta=on,*we*may*make*forwardLlooking*statements*regarding*future*events*or*the*expected*performance*of*the*company.*We*cau=on*you*that*such*statements*reflect*our*current*expecta=ons*and*es=mates*based*on*factors*currently*known*to*us*and*that*actual*events*or*results*could*differ*materially.*For*important*factors*that*may*cause*actual*results*to*differ*from*those*contained*in*our*forwardLlooking*statements,*please*review*our*filings*with*the*SEC.**The*forwardLlooking*statements*made*in*this*presenta=on*are*being*made*as*of*the*=me*and*date*of*its*live*presenta=on.**If*reviewed*aTer*its*live*presenta=on,*this*presenta=on*may*not*contain*current*or*accurate*informa=on.***We*do*not*assume*any*obliga=on*to*update*any*forwardLlooking*statements*we*may*make.**In*addi=on,*any*informa=on*about*our*roadmap*outlines*our*general*product*direc=on*and*is*subject*to*change*at*any*=me*without*no=ce.**It*is*for*informa=onal*purposes*only*and*shall*not,*be*incorporated*into*any*contract*or*other*commitment.**Splunk*undertakes*no*obliga=on*either*to*develop*the*features*or*func=onality*described*or*to*include*any*such*feature*or*func=onality*in*a*future*release.**
Splunk,(the(engine(for(machine(data(is(a(registered(trademark(of(Splunk(Inc.(and/or(its(subsidiaries(and/or(affiliates(in(the(United(States(and/or(other(jurisdic=ons.(*All(other(brand(names,(product(names(or(trademarks(belong(to(their(respec=ve(holders.((
©2012(Splunk(Inc.(All(rights(reserved.*
2*
About*Availity*
3*
Pa=ents.*Not*Paperwork.®"• A*na=onal*leading*health*informa=on*network*(HIN)*• Op=mizing*informa=on*exchange*through*a*single,*secure*network*• 200,000+*physicians*and*1,000+*hospitals*
Mul=ple*payers.*One*website.*One*login.*• 1300+*health*plans*and*450+*industry*partners*in*one*place*� Aetna,*20+*Blue*Cross*and*Blue*Shield*plans,*CIGNA,*Humana*and*more*
• RealL=me*info*exchange*means*highest*accuracy*of*health*plan*info**
HIPAA*transac=ons*and*proprietary*products.*• File*claims,*check*eligibility,*review*clinical*history,*process*payments*• Products*include:**Availity*CareCollect®,CareProfile®,*CareRead®,*CareCost*Es=mator®,*
Remit*Reader®,"and*more*
About*Me*
4*
Jus=n*Hardeman*Unix*Administrator*Availity*Health*Informa=on*Networks**
Core*Func=ons*• Produc*on"Opera*ons"(4"years)"
L Monitoring*L Incident*response*L 24/7*Support*on*Unix*systems,*authen=ca=on*systems*and*webLportal*
• Resident"Splunk"user"(3"years)"L BeLall*and*endLall*of*Splunk*
Agenda*
• Going*beyond*system*monitoring*
• Challenges*we’ve*faced*at*Availity*
• Senior*leadership*views*–*how’s*the*business*doing*overall?*
• System*opera=ons*–providing*insights*into*the*nuts*and*bolts,*
and*maybe*even*the*broken*gears*
• Marke=ng*and*training*–*are*people*using*our*demos?*
• Wins!*
• Takeaways?*
*
Ask*Yourself*
I*already*have*monitoring,*so*that*should*be*good*enough,*right?*
6*
What*Do*I*Mean*by*Monitoring?*" The*easy*stuff*– Is*a*process*up?**Ac=ve*or*stale?*– What’s*my*CPU/disk/RAM*usage?*– I*need*to*monitor*a*log*file*for*a*par=cular*error*message*
" What*about*the*more*difficult*measurements*– How*fast*is*my*average*user*login*=mes?**What*was*it*last*week?*– What*is*the*top*error*code*for*a*par=cular*payer*and/or*transac=on*type?*
– How*many*hits*did*we*receive*to*a*training*demo*during*the*=meframe*of*last*week?**Last*month?*
** 7*
Before*Splunk*
8*
Reac=ve*approach*to*problems*
Lack*of*visibility*into*root*cause*of*failures*/*issues*
Long*resolu=on*=mes*–*always*needed*the*availability*of*the*right*person*at*the*right*=me*
• RealL=me*visibility*into*issues*with*proac=ve*aler=ng*
• Reports*and*dashboards*with*realL=me*transac=ons*and*B2B*workflows*
• Combining*mul=ple*sources*of*informa=on*in*varying*formats*into*a*single*view*for*capacity*planning*
• Secure,*roleLbased*views*for*IT*teams*
OPERATIONAL*CHALLENGES* MONITORING*REQUIREMENTS*
Transforming*Machine*Data*with*Splunk*
9*
Proac*ve"monitoring"
Real?*me"visibility"
Opera*onal"insight"
Enterprise"adop*on"
Availity’s*data* …*for*applica=on*intelligence*
Red"Hat"Linux"Servers" Oracle"Iden*ty"
Manager"
Custom"metrics"from"NFS"Servers"
PlaLorm"independent"SAN"systems"
Homegrown"Applica*ons""
Oracle"Databases"
Scheduled*Searches*Become*Alerts*
Recent*Request:*I(need(to(be(alerted(if(the(batch(aggrega=on(queue(backs(up*
! I*should*have*no*more*then*500*items*for*longer*than*30*minutes*
! I*need*to*receive*an*email*! I*need*this*yesterday*! You*should*be*able*to*find*what*you*need*in*aries.log*
*
10*
Scheduled*Search*Demo/Example*" Let’s*build*an*alert!**
11*
Use*Your*Alerts*to*Drive*Dashboard*Usage*
FollowLup*Request:*Can(you(add(some(visualiza=ons?(
! Can*you*provide*me*with*some*history?*
! Can*you*show*me*what*my*counts*are*in*rela=on*to*the*same*=me*last*week?*
! Can*you*build*it*so*that*I*can*easily*edit*and*add*to*it*later?*
*
12*
Dashboard*Demo/Example*" Let’s*build*a*dashboard!**
13*
Team*Dashboards*• Senior*Leadership**
– Overall*health*of*the*business*• Account*Management**
– Payer*stats*– Portal*stats*
• Opera=ons*– Transac=on*processing*measurements*– Job*tracking*
• Marke=ng*and*Training*– Page*hits*– File*downloads*
14*
Senior*Leadership*" What’s*really*going*on?*– I*don’t*need*to*see*the*dayLtoLday*issues,*I*need*the*big*picture.*
" Don’t*kill*me*with*data!*– I*should*receive*what*I*need*in*a*quick*glance.*
" “X”*number*of*events*isn’t*good*enough!*– There*are*5,000*events.**Is*that*good*or*bad?**Is*that*what*we*normally*run?*
15*
16*
Account*Management*" How*is*my*account*doing?*– Are*transac=ons*comple=ng*successfully?*– Have*response*=mes*increased/decreased?*
" What*do*I*need*to*report*back*to*my*account*contacts?*– Trouble*transac=on*types*– Regional*outages*
" What*might*cause*an*increase*in*call*volume*to*the*help*desk?*
17*
18*
Opera=ons*• How*do*the*transac=on*volumes*look?*• RealL=me,*B2B,*and*Batch*
• Are*automated*connec=on*jobs*occurring?*• Job*Scheduler*–*Tidal*• Example*log*entry*L*07/31 15:05:29:740[16:MD-5]: (mem=1609407968/2142568448)
ServerNode: Event = SystemActivityMessage: Job FTP MEDICAREB277IN[189] completed status [Completed Abnormally].*
• Any*issues*with*our*batch*processing*engine?*• Homegrown*Applica=on*–*Aries*• Example*log*entry*L*08/10/2012 13:41:47 [IR-0] INFO c.a.aries.gateway.GatewayScanner
- [splunk] tpt="66",qn="ARIES.INTERNAL.REQUESTS",ts="08-10-2012 13:41:47.723",di="9d531343-1d56-417c-abac-8c367e87ef99",psz="957",nn="10.xx.xx.xx",qt="76871",ct="16",si="Availity",evi="1636814868"!
19*
20*
Marke=ng*and*Training*" Who*uses*our*product?*– Page*hits*– Browser*usage*– Loca=on*– Market*penetra=on*
" ATer*hos=ng*a*webinar,*do*we*see*an*increase*in*demo*hits?*" Which*part*of*the*FAQ*receives*the*most*views?*" Who*is*reading*the*links*in*the*“Availity*News”*sec=on?*
**
21*
22*
23*
Availity’s*Top*Wins*with*Splunk*
24*
Ease*of*data*consolida=on*
From*IT*to*the*enterprise*
From*reac=ve*to*proac=ve*
" Find*clues*easily*with*simple*searches*
" Transform*the*data*to*eliminate*IT*problems*proac=vely*
" Trace*transac=ons*quickly*across*the*applica=on*and**infrastructure*stack*
" Incrementally*grow**based*on*need*
" Mine*exis=ng*structured*and*unstructured*data*
" Consolidate*and*integrate*mul=ple*sources*easily*without*any*prior*knowledge*on*the*data*
" Customized*roleLbased*views*in*less*than*10*minutes*
" RealL=me*views*for*our*execu=ve*management*(CTO)**
" Viral*adop=on*across*the*enterprise*–*over*126*ac=ve*users*
What*Should*I*Do*with*This?*" Don’t*stop*at*simple*shell*scripts.**Paint*a*picture*with*your*data.*" Encourage*your*users*to*build*their*own*dashboards.*" Try*to*replace*a*piece*of*monitoring*with*Splunk.**Improve*on*the**process,*and*offer*more*then*the*consumer*originally*asked*for.*
25*
Ques=ons?*
26*