of 50 /50
THE 5 MINUTE DBA DBA Skills for Non-DBA’s Matt Yonkovit ,VP (Consulting) – Percona Percona University, Washington DC September 12, 2013

The 5 Minute DBA-DBA Skills for Non-DBA

Embed Size (px)

DESCRIPTION

The presentation provides you with the essential tips to ensure that your database runs smoothly if you are pressed for time. Percona provides an in-depth review of your database and recommends appropriate changes by performing a complete MySQL health check in which we identify inefficiencies, find problems before they occur, and ensure that your MySQL database is in the best condition.

Text of The 5 Minute DBA-DBA Skills for Non-DBA

  • 1. THE 5 MINUTE DBA DBA Skills for Non-DBAs Matt Yonkovit ,VP (Consulting) Percona Percona University, Washington DC September 12, 2013
  • 2. DBAs are hard to find! Company Confidential December 2010 -2-
  • 3. Non-DBAs in charge of DBs Company Confidential December 2010 -3- System Administrators Web Administrators Developers
  • 4. You get stretched Company Confidential December 2010 -4-
  • 5. WHAT CAN YOU DO IN 5 MINUTES? The top tips to ensure your database is running smoothly Company Confidential December 2010 -5-
  • 6. BIG DISCLAIMER! Company Confidential December 2010 -6- Its important to distinguish this up front: I am giving you the easy answers, which are not always the correct answers.. If you only have 5 minutes to spend on one of these database topics, you lose a lot of flexibility I could spend lots of time talking about everything here, instead I am focusing on maximizing your benefit. This presentation assumes you are running on Linux/Unix
  • 7. The Common Problems Kill You Company Confidential December 2010 -8-
  • 8. THOU SHALL HAVE BACKUPS! Company Confidential December 2010 -9- They can save your bacon!
  • 9. Probably the most common problem to get called on is some sort of slowness Keys to success Identify what is slow When its slow Can you repeat it Dig in The Slow Down Of Doom! Company Confidential December 2010 -10-
  • 10. No sense in targeting areas that do not matter Target your pain first Company Confidential December 2010 -11-
  • 11. Google is your friend to research problems But dont tune via Google Trust but verify Know where to look Company Confidential December 2010 -12-
  • 12. Know the tools out in the community Have them ready to jump in and fight fires at a moments notice Tools like: Percona Toolkit Cacti Home grown scripts etc Know thy tools Company Confidential December 2010 -13-
  • 13. Define what is slow? If its a single page, the process is drastically different then if its general slowness. Assume General Slowness #1 What has changed? A new deployment? More users? Etc? Sometimes a roll-back of a buggy code release is the best fix. DO NOT ASSUME ITS THE DATABASE! Start with the obvious and the quick and move to more complex Tuning walk through Company Confidential December 2010 -14-
  • 14. 5.1 -vs- 5.0 provides improvements 5.5 -vs- 5.1 provides improvements 5.6 vs- 5.5 provides improvements HUGE Differences in Versions! Company Confidential December 2010 -15-
  • 15. Often stock OS repos are out of date! Watch out for old versions installed as dependencies of off the shelf software Just because a hosting provider installed it does not mean its the latest or best Where did you get it? Company Confidential December 2010 -16-
  • 16. Innodb Innodb Plugin XtraDB TokuDB MyISAM Aria Many Many more.... Storage Engines Company Confidential December 2010 -17-
  • 17. Old Kernel? Multi-core servers, scalability getting better (~32 cores + ) Fit your hot data into memory! Typically more memory is better IO is a Major bottleneck RAID For your databases is a must Battery Backed Controller RAID 10 typically better then RAID 5 SSD's for optimal performance Network should not be forgotten 32bit binary or OS on 64 bit machine Hardware & OS Company Confidential December 2010 -18-
  • 18. Next I start by looking at the server (Sar, vmstat, top, iostat -x)... I typically start low and work my way into the database Too much io, maybe not enough memory, missing indexes, or bad config Cpu maxed out could be lots of users, lots of concurrent queries, or something else. Is the problem even MySQL? Often times we see issues with non-database software top process is something else ( i.e. apache, php, cron job, etc ) Network is often overlooked Review OS Metrics Company Confidential December 2010 -19-
  • 19. COMMON TOOLS For monitoring the OS/MySQL in a pinch Company Confidential December 2010 -20-
  • 20. Sar is your friend, every unix/linux server should setup sar to collect stats in the background Sar Can give you historical reports on : CPU, Disk, Memory, Network, and a lot more. Scheduled to collect data via cron Iostat is valuable in gauging disk performance avwait+svctim = response time (iostat -x) Systat Company Confidential December 2010 -21-
  • 21. Sar Network Stats Company Confidential December 2010 -22-
  • 22. Sar CPU Company Confidential December 2010 -23-
  • 23. Sar Memory Company Confidential December 2010 -24-
  • 24. Iostat Company Confidential December 2010 -25-
  • 25. VMSTAT Company Confidential December 2010 -26- Can give you a realtime view of changes on the system to CPU/Memory
  • 26. Top Company Confidential December 2010 -27- Get a current Snapshot of running processes
  • 27. Percona Toolkit Company Confidential December 2010 -28- Collection of scripts used for monitoring system and DB performance pt-stalk pt-diskstats pt-pmp Pt-Ioprofiler pt-mext
  • 28. Cacti Company Confidential December 2010 -29- Great Visual Presentation of Historical Stats MySQL, Apache, Sphinx, Memcached, (Entire Stack) Percona Maintains the MySQL Templates Can add a lot of value... Example: Mysterious Slowdown
  • 29. OMG The Server is Slow! Company Confidential December 2010 -30-
  • 30. Dig a little deeper Company Confidential December 2010 -31-
  • 31. Oh Wait it Always Happens Company Confidential December 2010 -32-
  • 32. Percona Monitoring Plugins- Cacti! Company Confidential December 2010 -33- Grab them here: http://www.percona.com/software/percona-monitoring-plugins/
  • 33. THE CONFIGURATION Reasonability Check Company Confidential December 2010 -34-
  • 34. MYSQL CHECKS Looking at the DB Company Confidential December 2010 -35-
  • 35. Configuration Options Top 10 Company Confidential December 2010 -36- General Configuration Options query_cache_size (Disable in many versions) log_slow_queries long_query_time max_connections expire_logs_days MyISAM Specific key_buffer_size
  • 36. Configuration Options Company Confidential December 2010 -37- Innodb Specific Options innodb_buffer_pool_size innodb_flush_log_at_trx_commit innodb_log_file_size Plugin/Xtradb/Mysql 5.5 Innodb Options innodb_io_capacity innodb_read_ahead innodb_write_io_threads innodb_read_io_threads
  • 37. Online Tools to help you set your my.cnf! Company Confidential December 2010 -38- https://tools.percona.com/wizard
  • 38. Memory and Swap Company Confidential December 2010 -39- Disk IO is bad, Avoiding swapping things that should be in memory to disk ( do not disable swap however ) SSDs/Fast Disks Hot Data in Memory! read_buffer, read_rnd_buffer, sort_buffer, join_buffer are all allocated per thread so be careful! You want temp tables to be built in memory not on disk
  • 39. Check Current Activity Mysql show full processlist show global status Show innodb status Mysqladmin Slow Query Log pt-query-digest Mysqldumpslow Innotop Mytop pt-mext Company Confidential December 2010 -40-
  • 40. Show Processlist Company Confidential December 2010 -41- mysql> show processlistG *************************** 1. row *************************** Id: 52 User: bench Host: domU-12-31-39-0C-41-81.compute-1.internal:42315 db: simple_benchmark Command: Sleep Time: 0 State: Info: NULL *************************** 14. row *************************** Id: 65 User: bench Host: domU-12-31-39-0C-41-81.compute-1.internal:42328 db: simple_benchmark Command: Query Time: 0 State: init Info: insert into simple_benchmark.simple_comments (lookup_id, comment, mydate) values ( '8519163', 'Comme *************************** 15. row *************************** Id: 66 User: bench Host: domU-12-31-39-0C-41-81.compute-1.internal:42329 db: simple_benchmark Command: Sleep Time: 0 State: Info: NULL
  • 41. Mysqladmin Company Confidential December 2010 -42- Mysqladmin extended -r -i1 will give you mysql stats at 1 second intervals, change the -i to increase decrease... A few stats of note:
  • 42. pt-mext Company Confidential December 2010 -43- Mext can take the output from mysqladmin and put the status variables side by side, this makes it very convenient to look for differences
  • 43. Slow Query Log Company Confidential December 2010 -44- The Slow Query log is your friend! As mentioned earlier queries that take over a certain amount of time are logged here. (Percona Patches for complete picture ) Summarize the slow query log with percona toolkit, pt- query-digest or mysqldumpslow if the percona toolkit is not available Look for not only the longest running query, but also the one with lots of executions and cumulated time
  • 44. Pt-query-digest output Company Confidential December 2010 -45-
  • 45. pt-query-digest Company Confidential December 2010 -46-
  • 46. Memcached/Redis/Other Caching Company Confidential December 2010 -47- Review: More memory the better Caching frequently used data is an excellent way to boost performance This typically requires changes to your app Performance improvement can be dramatic
  • 47. Query Tuning Company Confidential December 2010 -48- Review the slow query log, parse the log with pt- query-digest. If I see something here, I will analyze it If your still at a loss, try using tcpdump to dump SQL statements into pt-query-digest and look for repeated queries. ( Easier then the general query log if you need sub second slow queries).
  • 48. Query Tuning Company Confidential December 2010 -49- Use Explain Plan Add Indexes where needed, do not over index pt-duplicate-key-checker Lots of small queries can sometimes be more impactful than 1 big one.
  • 49. Remember Company Confidential December 2010 -50- The Disclaimer! Questions?