Weekly Status Report

Database Support Team

Date: April 21st 2014 – April 27th 2014

1 Last Week Incident Summary

3 People Recognition

Agenda

4 Last Week Incident Summary

8 Mitigation

5 Weekly incident count6 Weekly SLA compliance

10 Problem Records9 Change Details

7 High priority incidents details

– strictly confidential, confidential, internal, public –

2 Week’s Highlights

11 Additional & adhoc activities12 Work Request / Activity list

Week’s Highlights

– strictly confidential, confidential, internal, public –

Operational Highlights• One High Priority incident resolved by DB Support last week IM0001932943/P2 (Oracle) .• DBA Supported two P2 Priority Incidents IM0001929189 (Informix), IM0001929611 (DB2).• 7 P3 and 145 P4-P6 incidents resolved this week within SLA.• Problem tickets resolved 5; Problem ticket RCA Submitted 3; Problem ticket update sent to PM 2.• 1 High risk in My SQL, 1 Medium Risk in Oracle, 2 low risk in DB2 and 1 low risk MS SQL changes

successfully implemented this week.• MS-SQL 10 Completed out of 39; Informix 1 completed out of 6; Oracle 17 completed out of 52;

DB2 100% completed for FMO with recommended patch.

Patching:• Revised Patching schedule updated.• New patching schedule published & raised respective changes for Informix DB servers.Oracle: • PTOPIC standby sync issue -- PTOPIC standby sync issue has been resolved manually.

permanent solution has been applied for it.• PCRD02 standby sync issue -- Root cause analysis is in progress.• WR 3255 P14 Nutmeg 3 - Excalibur Database Copy access – Completed • Space uplift activity for ABCS – Completed successfully.• SMC-24 new process testing is done.• SRP project – ORC started.DB2: • Need to move DB2 pre-prod digital backups from old backup server tsnv0039 to new one

tsukrv0139 – This has been completed with C000020460 change• There is some time amendment in terms of backup schedule for EDEN FMO databases under the

change(C000018417) has been completed successfullyInformix:• Table restore on discsoc database was implemented successfully.MySQL: • TOOLKIT release on 4/17/2014 was successfully implemented(under C000020129).

Week’s HighlightsChallenges/Ongoing Issues from this week

Informix:•Informix estate monitoring problem case needs to be presented to Ian/ Andy. Offshore DB team sent the ‘Position statement’ to Prasad & onsite team for review.•Changes created to resolve monitoring alerts are not getting approved in time even after providing all information. Where the situation is getting critical we are implementing the changes under P3 ticket with TSM approvals.•Patching changes delayed.•OVO Monitoring cannot be setup. We need to setup our own monitoring.•Addition of Fragments on two tables in JT was postponed twice last week since it was not fully approved. It is scheduled on 28/04/2014..•Another change on table reorg on cenmain was postponed since incorrect RFC was attached. It is scheduled for 30/04/2014 01:00:00.MS SQL:•Unknown MS SQL count is reduced from 200+ servers to 113 servers. End report need clarification –Discussed with Ian/Andrew. Not yet clarified. Still team is following up onsite folks. Next week call need to be setup with TSL DBAs for more information.Oracle:•BDA Tool Access issue with TICKS card.•Aston Request pending – got ASTON certificate for some of the team members and others are in progress.•WIW requests and ticks card requests are pending for new team members. This can be highlighted as a risk. Because Access has been revoked for released resources. Due to this cannot use current bandwidth efficiently. •ETU FMO performance issue.—It not yet clear whether it is in scope or not.•Need to create updated patching plan as per new PSU release by oracle.•PCRD02 standby sync issue—need confirmation whether to proceed with full restore.Patching:•PIVR(TSM did not approve) and PSOA database patching – Cancelled. (another SLT change was scheduled).

Next week – hot topics• PTOPIC switchover is in plan.• Excalibur post go-live change reschedule for 30th April.• Excalibur CMO database decommission request is planned.• Change C000019477 has been scheduled for TOOLKIT application to Increase MAX_CONNECTION

parameter for Toolkit Database.• ORC phase for SRP project from MS SQL area has been scheduled for next week.

People Recognition

– strictly confidential, confidential, internal, public –

Appreciations/Certifications

N/A

Last Week Incident SummaryDatabase– Executive Summary

Inflow P1 P2 P3 P4 P5 P6

Number of Incidents received 0 1 7 48 76 21Number of Incidents transferred 0 0 0 0 0 0

Number of Incidents closed 0 1 7 48 76 21Number of Incidents closed within SLA 0 1

5 48 76 21Number of Incidents closed within SLA (Mitigated)

0 0 2 0 0 0

Number of Incidents open 0 0 5 11 42 14Number of Incidents open -Older than 25 days 0 0 0 0 4 1

SLA Compliance (Mitigated) / CKPI Compliance

N/A 100% 71% 100% 100% 100%

Weekly incident countIncident Resolved Count - Weekly

Last 10 weeks dta– strictly confidentia, confidential, internal, public –

Week P1 P2 P3 P4 P5 P6 Count3rd Mar – 9th Mar 1 2 19 59 99 38 21810th Mar – 16th Mar 0 0 16 61 104 33 21417th Mar – 23rd Mar 0 0 13 50 82 29 17424th Mar – 30th Mar 0 0 6 39 108 32 18531st Mar – 6th Apr 0 2 4 29 80 6 1217th Apr – 13th Apr 0 0 16 43 80 25 16414th Apr – 20th Apr 0 0 6 64 104 70 24421st Apr – 27th Apr 0 1 7 48 76 21 153

Weekly SLA compliance

Week

P1 P2 P3 P4 P5 P6Actual

Mitigate

d

Actual

mitigated

Actual

mitigated

Actual

mitigated

Actual

mitigated

Actual

mitigated

3rd Mar – 9th Mar N/A N/A N/A N/A 100% 100% 100% 100% 100% 100% 100% 100%

10th Mar – 16th Mar N/A N/A N/A N/A 100% 100% 100% 100% 100% 100% 100% 100%

17th Mar – 23rd Mar N/A N/A N/A N/A 100% 100% 100% 100% 100% 100% 100% 100%

24th Mar – 30th Mar N/A N/A N/A N/A 100% 100% 97% 97% 97% 100% 100% 100%

31st Mar – 6th April N/A N/A 100% 100% 100% 100% 100% 100% 100% 100% 100% 100%

7th Apr – 13th April N/A N/A N/A N/A 94% 100% 93% 95% 96% 100% 92% 100%

14th Apr – 20th Apr N/A N/A N/A N/A 83% 100% 100% 100% 99% 99% 100% 100%

21st Apr – 27th Apr N/A N/A 100% 100% 71% 100% 100% 100% 100% 100% 100% 100%

% Mitigated SLA Compliance

Last 10 weeks data– strictly confidential, confidential, internal, public –

High priority incidents details

– strictly confidential, confidential, internal, public –

High Priority Incidents : IM0001932943 /P2(Oracle) Summary: Archival error on WFM database. Details: WF application purging was scheduled which generated lots of redo on database .Resolution : Archive mount point has been changed where enough space is available, also P3 is raised for backup team to ensure archive backup works with effective speed.Business Impact: business being unable to manage resource correctly causing advisors to be incorrectly assigned to queues and showing incorrect adherence states which will affect service levels and advisor bonus.Service Affected: WFMCurrent Status: ResolvedIncident raised at: 27/04/2014 08:22Incident reported at: 27/04/2014 08:35 Incident resolved at: 27/04/2014 08:57Actions taken:27/04/2014 08:35 IM accepted, Tech bridge opened27/04/2014 08:37 Technical bridge opened with Application team, DBA & IM .27/04/2014 08:39 Archival issue found on wfmprod database due to high volume of archive log generation.27/04/2014 08:45 As a work around, archive mount point was changed and confirm to application team to restart the work.27/04/2014 08:57 IM/application team confirm resolution of incident. 27/04/2014 10:15 ECR C000021334 which was raised for purging activity completed . Time Spent: 20 Min on call. 90 Min for monitoring as precautionary measure. RISK : No low priority incident was triggered by Tivoli/HP Ovo hence will raise incident for their investigation as well.

MitigationMitigation plan

– strictly confidential, confidential, internal, public –

Incident ID INM Close Group Name INM Resolution Code Priorit

yBreached by Cognizant Domains? (Yes/No)

IM0001910156 CSS-SK.SM.DBA-SK

SLA EE#13(SLA failure beyond our control - incidents breached when re-assigned - Internal TSL Issue)

P3No

IM0001909182 CSS-UK.SM.DBASLA EE#13(SLA failure beyond our control - incidents breached when

re-assigned - Internal TSL Issue) P3No

Change DetailsChange implemented

– strictly confidential, confidential, internal, public –

Domain Change ID Start Time End TimePre

change activity

EffortsPost

Change activity

Brief description Success Impact of Change

DB2 C000020950 25/04/14 10:00:00 25/04/14 15:00:00 30 min 180 min 0 min BAU | EE | CUST | Eden - EE Staging Success Low Risk

DB2 C000020954 25/04/14 10:00:00 25/04/14 15:00:00 30 min 180 min 0 min BAU | EE | CUST | Eden TMUK Staging Success Low Risk

MySQLC000020710 24/04/14 22:00:00 25/04/14 06:30:00 0 Min 30 Min 0 Min

UNIX - SECURITY PATCH - Using Bladelogic patch on EEUKRP0345

Success High Risk

Oracle C000021211 4/25/2014 23:00 4/27/2014 6:00 30 Min 120 Hrs 0 Min SLT - SOA DB migration Success Medium Risk

MS-SQL C000020114 24/4/2014 13:00:00 25/4/2014

13:00:00 30 Min 30 Min 20 Min

Execute the provided script on RAPROD01 in the FILE MATCH database

Success Low Risk

Top issues

Problem RecordsSummary

– strictly confidential, confidential, internal, public –

Problem Records_Details

Additional & adhoc activitiesSummary

Additional & Adhoc activitiesS.no Reference Number Date Description Effort Spent

Work Request / Activity listSummary

– strictly confidential, confidential, internal, public –

In Progress: 8 Early Engagement: 10 Waiting for formal request :5 Go live this week: N/A ORC: 4

Projects planned for Go Live next week: N/AProject names :N/ANew WR/Project request received: N/A

THANK YOU!