Upload
others
View
19
Download
0
Embed Size (px)
Citation preview
@MartinKlierDBA (Un)loved Child of Gen. Cloud - Oracle on VMware 2/50
The (Un)loved Child of Generation Cloud - Oracle on VMware
Martin Klier
Performing Databases GmbHMitterteich / Germany
Das (un)geliebte Kind der Generation Cloud - Oracle auf VMware
@MartinKlierDBA (Un)loved Child of Gen. Cloud - Oracle on VMware 3/50
Basics
@MartinKlierDBA (Un)loved Child of Gen. Cloud - Oracle on VMware 4/50
a Type-I-Hypervisor
VMware vSphere
CPUExtensions(VT-x,SIMD, etc.)
Para-VirtualizedController
@MartinKlierDBA (Un)loved Child of Gen. Cloud - Oracle on VMware 5/50
TEMP
Oracle Architecture (simplified)
Listener:1521
PMON
SGA
ServerServer
Buffer Cache
Shared Pool
Log Buffer
ARCnSMON
LGWR
DBWn
PGAs
CKPT
Server
Online
Redo
Logs
ArchivedRedoLogs
Tablespace Undo Tbs.
Blocks
@MartinKlierDBA (Un)loved Child of Gen. Cloud - Oracle on VMware 6/50
NUMA Architecture
@MartinKlierDBA (Un)loved Child of Gen. Cloud - Oracle on VMware 7/50
Core(s)+SharedCache, LLC
Core(s)+SharedCache, LLC
QPI-C
Core(s)+SharedCache, LLC
Core(s)+SharedCache, LLC
PCIePCIe
IMC IMCQPI-C
IMCQPI-CQPI-CIMC
NonUnifiedMemoryAccess
NUMA
@MartinKlierDBA (Un)loved Child of Gen. Cloud - Oracle on VMware 8/50
Oracle & NUMA
@MartinKlierDBA (Un)loved Child of Gen. Cloud - Oracle on VMware 9/50
_enable_NUMA_support = TRUE
MOS Doc ID 864633.1
• Multiple Buffer Caches
• Striped pools
=> cross context :((
=> pool access :(
Oracle & NUMA
@MartinKlierDBA (Un)loved Child of Gen. Cloud - Oracle on VMware 10/50
13GB+13GB=26 GB
One buffer cache for each node
Oracle & NUMA
@MartinKlierDBA (Un)loved Child of Gen. Cloud - Oracle on VMware 11/50
Oracle & NUMA
@MartinKlierDBA (Un)loved Child of Gen. Cloud - Oracle on VMware 12/50
Suggestions
• Useful in big environments only (think: DB consolidation)
• Test thoroughly and quantify use vs. effort (think: bugs)
• Use only if necessary to achieve your goals!
Oracle NUMA aware
@MartinKlierDBA (Un)loved Child of Gen. Cloud - Oracle on VMware 13/50
Hyperthreading
@MartinKlierDBA (Un)loved Child of Gen. Cloud - Oracle on VMware 14/50
HyperThreading (HTT)
CoreProcessing big workloads= large input queue
Performance =Throughput
125% :)Disclaimer:Very simplified example!
@MartinKlierDBA (Un)loved Child of Gen. Cloud - Oracle on VMware 15/50
OLTP & HTT
A B
B
A
Core
Tim
e
x
x
x
OLTP workloads= empty input queue
Performance =Response Time
50% :(
@MartinKlierDBA (Un)loved Child of Gen. Cloud - Oracle on VMware 16/50
OLTP & HTT
Core
Tim
e
A
A
DisableHyperThreading for OLTP systems!
- in hardware BIOS- on VMware level („Latency Sensitivity“ high)
@MartinKlierDBA (Un)loved Child of Gen. Cloud - Oracle on VMware 17/50
VMware vSphere (ESXi)CPU Usage
@MartinKlierDBA (Un)loved Child of Gen. Cloud - Oracle on VMware 18/50
My Test Setup
• HP ProLiant DL380 G6• 2x 64GB RAM• 2 CPUs, 2x 6 cores = 2 NUMA nodes
Intel(R) Xeon(R) CPU X5670 @ 2.93GHz boost 3.33GHz„Westmere EP“ 32nm technology• HyperThreading disabled (BIOS)
• VMware vSphere ESXi 6.0 U2• 2 VMs / Oracle Linux 7.4 w/ UEK Kernel
VMware CPU Usage
@MartinKlierDBA (Un)loved Child of Gen. Cloud - Oracle on VMware 19/50
VMware CPU Usage
1 Worker6 vCPU 22% each
OS Metric / Linux top20% CPU
2w / 44%
3w / 65%
4w / 85%4w / 85%
5w / 95%6w / 100%
OS: 40%
OS: 50%
OS: 65%
OS: 80%
OS: 95%
@MartinKlierDBA (Un)loved Child of Gen. Cloud - Oracle on VMware 20/50
VMware CPU Usage
6 cores active
6 cores idle~0% „CPU Ready“
100% CPU
@MartinKlierDBA (Un)loved Child of Gen. Cloud - Oracle on VMware 21/50
CPU Ready=
Wait for free CPU time
@MartinKlierDBA (Un)loved Child of Gen. Cloud - Oracle on VMware 22/50
VMware CPU Usage
6 cores 100%
Not in graph:0% „CPU Ready“
12 cores 100%
6 cores idle
@MartinKlierDBA (Un)loved Child of Gen. Cloud - Oracle on VMware 23/50
Green IT :)
6 cores 100% 12 cores 100%
@MartinKlierDBA (Un)loved Child of Gen. Cloud - Oracle on VMware 24/50
VMware CPU Usage8 vCores
13 workers to fill up to 100% in top
Each worker:
while true( bc < 5.5*7.7 etc.
)
@MartinKlierDBA (Un)loved Child of Gen. Cloud - Oracle on VMware 25/50
VMware CPU UsageVM w/ 8 vCores13 workers
> 1 NUMA nodestarts switchingcontexts :(
6 cores idle
VM w/ 6 vCores6 workers
runs on oneNUMA nodeclean & tidy
@MartinKlierDBA (Un)loved Child of Gen. Cloud - Oracle on VMware 26/50
Oracle Workload on ESXi
@MartinKlierDBA (Un)loved Child of Gen. Cloud - Oracle on VMware 27/50
Oracle Stress in VM
loop( select 1 row;
update this row;commit)
6 workerseach on a different set of 10k rows
runs ~20min until Free Buffer Waits
@MartinKlierDBA (Un)loved Child of Gen. Cloud - Oracle on VMware 28/50
Anti-Load in VM
14 workers (bc)55% avg. loadover 12 vCoreswe know:Will heavily switchCPU context!
Not nearly full power(CPU load 55%)
Owner will feel no guilt
@MartinKlierDBA (Un)loved Child of Gen. Cloud - Oracle on VMware 29/50
Oracle vs. Anti-LoadAnti-Load VM CPU Oracle VM CPU
Not nearly full power(CPU load 55%)
But „steals“ 40%CPU time
>30%„CPU Ready“
@MartinKlierDBA (Un)loved Child of Gen. Cloud - Oracle on VMware 30/50
So far, much wasmeasured w/ help of the
VMware Admin:(
@MartinKlierDBA (Un)loved Child of Gen. Cloud - Oracle on VMware 31/50
SMS=
Surrogate Measure
Sucks
@MartinKlierDBA (Un)loved Child of Gen. Cloud - Oracle on VMware 32/50
Let's do it ourselvesLet's do what we do best
@MartinKlierDBA (Un)loved Child of Gen. Cloud - Oracle on VMware 33/50
Use Oracle
@MartinKlierDBA (Un)loved Child of Gen. Cloud - Oracle on VMware 34/50
AWR Diff Report
Period 1 Period 2
@MartinKlierDBA (Un)loved Child of Gen. Cloud - Oracle on VMware 35/50
AWR Diff Report
409.888,3 BG / 5,9 s = 69.473 BG/s234.796.8 BG / 5,5 s = 42.690 BG/s = 61%
39% less buffer getswith SAME CPU TIMEand WORKLOAD
(Linux sees 100% all times)
@MartinKlierDBA (Un)loved Child of Gen. Cloud - Oracle on VMware 36/50
and use your brain!
@MartinKlierDBA (Un)loved Child of Gen. Cloud - Oracle on VMware 37/50
Simple Benchmark
Guess,what happenedhere? :(
First value: ID of test iteration### bar: VisualizationDecimal value: Seconds for
10.000 x SELECT+UPDATEof different single rows(Avg. per iteration over all workers,all workers on different rows)
Simulated User Experience of aDB on VMware (if built poorly)
@MartinKlierDBA (Un)loved Child of Gen. Cloud - Oracle on VMware 38/50
Knowledge is importantProof is better
@MartinKlierDBA (Un)loved Child of Gen. Cloud - Oracle on VMware 39/50
Prepare to Prove
What you need in your VM (Linux)
• VMware tools (VMware recommends package from your distributor)• Phython environment (usually present)• git (or copy yourself)• build environment like make (Oracle needs that anyway)• vmguestlib with vmguest-stats (made by Dag Wieers)
uses an API provided by vSphere (via VMW tools)
@MartinKlierDBA (Un)loved Child of Gen. Cloud - Oracle on VMware 40/50
vmguest-stats
@MartinKlierDBA (Un)loved Child of Gen. Cloud - Oracle on VMware 41/50
vmguest-stats
@MartinKlierDBA (Un)loved Child of Gen. Cloud - Oracle on VMware 42/50
Countermeasures
@MartinKlierDBA (Un)loved Child of Gen. Cloud - Oracle on VMware 43/50
@MartinKlierDBA (Un)loved Child of Gen. Cloud - Oracle on VMware 44/50
Ten Commandmentsfor OLTP on VMware
And the Lord said:
1) Thou shalt not overcommit your machine!2) Thou shalt not use Hyperthreading,
because HTT is CPU overcommitment3) Thou shalt make 100% CPU + RAM reservations4) Thou shalt disable CPU Hot Add (enable breaks vNUMA)5) Thou shalt disable RAM Hot Add (enable breaks vNUMA)6) Thou shalt not use more vCores in your VM than a NUMA node
of the host has7) Thou shalt not use more RAM in your VM than a NUMA node
of the host has8) Thou shalt configure VMs to Latency Sensitivity „HIGH“9) Thou shalt disable all BIOS based Energy Saving Options10) Thou shalt use Paravirtual Drivers at any cost
And peace be on earth.
@MartinKlierDBA (Un)loved Child of Gen. Cloud - Oracle on VMware 45/50
(More) RessourceConsumption
Makes you (more)vulnerable
@MartinKlierDBA (Un)loved Child of Gen. Cloud - Oracle on VMware 46/50
Speaker
● Martin Klier● Solution Architect and
Database Expert
● My focus– Performance Optimization– High Availability– Architecture DBMS
● Linux since 1997● Oracle Database since 2003
@MartinKlierDBA (Un)loved Child of Gen. Cloud - Oracle on VMware 47/50
Speaker
● Meet & Greet
● Contact: [email protected]
● Weblog: http://www.usn-it.de (English)
RegionalgruppenFachkonferenzen
Las Vegas, April 2018Tel Aviv, January 2018
Frankfurt, December 2018
@MartinKlierDBA (Un)loved Child of Gen. Cloud - Oracle on VMware 48/50
Performing Databases
● Experts for Database Technology
– Concept– Planning & Sizing– Licensing– Implementation and Troubleshooting
● Get in touch
– Performing Databases GmbHWiesauer Straße 2795666 Mitterteich, GERMANY
– Web: http://www.performing-databases.com– Twitter: @PerformingDB
@MartinKlierDBA (Un)loved Child of Gen. Cloud - Oracle on VMware 50/50
Q & A
Download my Presentations and Whitepapershttp://www.performing-databases.com