93
Virtualize with KVM Tools and Enterprise usage by Vincent Van der Kussen

KVM tools and enterprise usage

Embed Size (px)

DESCRIPTION

Open Source Virtualization based on KVM and available tools like Libvirt, Libguestfs, Qemu, ...

Citation preview

Page 1: KVM tools and enterprise usage

Virtualize with KVMTools and Enterprise usage

by Vincent Van der Kussen

Page 2: KVM tools and enterprise usage

#whoami

● Using Linux since 1996

● Sysadmin / Engineer / Opensource Enthousiast

● @vincentvdk

● http://blog.vanderkussen.org

Page 3: KVM tools and enterprise usage

So what is this KVM thing anyway?

Page 4: KVM tools and enterprise usage

It's NOT!

Page 5: KVM tools and enterprise usage
Page 6: KVM tools and enterprise usage
Page 7: KVM tools and enterprise usage

KVM : overview

Kernel-based Virtual Machine, a Kernel module that turns Linux into a hypervisor

Created by Quramnet (bought by Red Hat 2008)

Supports x86 (32 and 64 bit), s390, Powerpc

Included in Linux kernel since 2.6.20

Needs a CPU with virtualization extention !

Page 8: KVM tools and enterprise usage

KVM : overview

● Each vm is a process

● Each virtual cpu is a thread

● Open source

Page 9: KVM tools and enterprise usage

Features

CPU and memory overcommit

High performance paravirtual I/O

Hotplug (cpu, block, nic)

SMP guests

Live Migration

Power management

Page 10: KVM tools and enterprise usage

Features

PCI Device Assigenment and SR-IOV

KSM (Kernel Samepage Merging)

SPICE

NUMA

Page 11: KVM tools and enterprise usage

Not your typical VMWare workstation or VirtualBox.

VirtManager comes close but still a little rough on the edges

KVM is meant for virtualization on servers. Desktop usage is not always ideal (think vSphere, HyperV,..).

KVM : differences

Page 12: KVM tools and enterprise usage

Overview Tools

libvirt (api)

Virsh

Qemu

VirtManager

Libguestfs

virt-v2v

Page 13: KVM tools and enterprise usage

Overview tools

sVirt

virt-*

Boxgrinder

VirtIO (not really a tool)

Spice

Page 14: KVM tools and enterprise usage

Libvirt : The Virtualization API

Page 15: KVM tools and enterprise usage

Libvirt : overview

● “The Virtualization API”

● Toolkit to interact with the virtualization layer

● Written in C

● LGPL● Has bindings for common languages (Python,

Ruby, Perl, PHP, Java,..)● Supports AMQP via libvirt-qpid

Page 16: KVM tools and enterprise usage

Libvirt : overview

● Provides management of

– Virtual machines

– Virtual networks

– Storage

● Does NOT provide multi node management features like load balancing

● Suspend / resume support

Page 17: KVM tools and enterprise usage

Libvirt : terminology

Node

a physical machine

Page 18: KVM tools and enterprise usage

Libvirt : termonology

Hypervisor

A software layer that allows a node to run virtual

machines

Page 19: KVM tools and enterprise usage

Libvirt : terminologie

Domain

An instance (“virtual machine”) running on the

hypervisor

Page 20: KVM tools and enterprise usage
Page 21: KVM tools and enterprise usage

Libvirt : overview

Supports also :

XEN KVM

VMWare

LXC

OpenVZ

Microsoft HyperV

Virtualbox

Page 22: KVM tools and enterprise usage

virsh

Page 23: KVM tools and enterprise usage

Virsh overview

Command line for libvirt

Enables scripting

Libvirtd needs to be started

Most commands require root privileges

Page 24: KVM tools and enterprise usage

Virsh usage

virsh <command> <domain-id> [OPTIONS]virsh <command> <domain-id> [OPTIONS]

Page 25: KVM tools and enterprise usage

Virsh usage

Connect to hypervisor on the localhost

virsh connect qemu:///systemvirsh connect qemu:///system

Connect to hypervisor on remote system over SSH

virsh -c qemu+ssh://[email protected]/systemvirsh -c qemu+ssh://[email protected]/system

virshvirsh

or

Page 26: KVM tools and enterprise usage

Virsh usage

Page 27: KVM tools and enterprise usage

Virsh : storage

● Define storage pool● Local filesystem (ext3, ext4,...) ● Directory based● ISCSI● NFS● LVM (uses volume group)● Physical disk● multipath

Page 28: KVM tools and enterprise usage

Virsh : create storage pool

[root@localhost ~]# virsh pool-define dir_pool.xml Pool LOCAL-STORAGE defined from dir_pool.xml[root@localhost ~]# virsh pool-define dir_pool.xml Pool LOCAL-STORAGE defined from dir_pool.xml

[root@localhost ~]# virsh pool-start LOCAL-STORAGEPool LOCAL-STORAGE started[root@localhost ~]# virsh pool-start LOCAL-STORAGEPool LOCAL-STORAGE started

<pool type="dir"><name>LOCAL-STORAGE</name> <target> <path>/VMS</path> </target></pool>

Xml file ->

Page 29: KVM tools and enterprise usage

Virsh : create storage pool

[root@localhost ~]# virsh pool-listName State Autostart -----------------------------------------default active yes LOCAL-STORAGE active no

[root@localhost ~]# virsh pool-listName State Autostart -----------------------------------------default active yes LOCAL-STORAGE active no

[root@localhost ~]# virsh pool-autostart LOCAL-STORAGEPool LOCAL-STORAGE marked as autostarted

[root@localhost ~]# virsh pool-autostart LOCAL-STORAGEPool LOCAL-STORAGE marked as autostarted

Page 30: KVM tools and enterprise usage

Virsh : create VM

● VM is defined in xml file

● Manual creation of VM's disks

[root@localhost ~]# qemu-img create /VMS/vm02-fromxml.img 5GFormatting '/VMS/vm02-fromxml.img', fmt=raw size=5368709120 [root@localhost ~]# qemu-img create /VMS/vm02-fromxml.img 5GFormatting '/VMS/vm02-fromxml.img', fmt=raw size=5368709120

● Define VM in libvirt and start it

[root@localhost ~]# virsh define vm02_from_xml.xmlDomain vm02-fromxml defined from vm02_from_xml.xml

[root@localhost ~]# virsh start vm02-fromxmlDomain vm02-fromxml started

[root@localhost ~]# virsh define vm02_from_xml.xmlDomain vm02-fromxml defined from vm02_from_xml.xml

[root@localhost ~]# virsh start vm02-fromxmlDomain vm02-fromxml started

Page 31: KVM tools and enterprise usage

1 <domain type='kvm'> 2 <name>vm02-fromxml</name> 3 <uuid></uuid> 4 <memory>128288</memory> 5 <currentMemory></currentMemory> 6 <vcpu>2</vcpu> 7 <os> 8 <type arch='x86_64' machine='pc-0.14'>hvm</type> 9 <boot dev='hd'/>10 </os>11 <features>12 # features such as ACPI are defined here13 </features>14 <devices>15 <emulator>/usr/bin/qemu-kvm</emulator>16 <disk type='file' device='disk'>17 <driver name='qemu' type='raw' cache='none'/>18 <source file='/VMS/vm02-fromxml.img'/>19 <target dev='vda' bus='virtio'/>20 <alias name='virtio-disk0'/>21 <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>22 </disk>23 </devices>24 </domain>

Page 32: KVM tools and enterprise usage

Virsh : list vm's

[root@localhost ~]# virsh list --all Id Name State---------------------------------- 4 vm01 running - vm02-fromxml shut off

[root@localhost ~]#

[root@localhost ~]# virsh list --all Id Name State---------------------------------- 4 vm01 running - vm02-fromxml shut off

[root@localhost ~]#

Page 33: KVM tools and enterprise usage

Virsh : live migration

virsh migrate --live <guestname> qemu+ssh://<destination host>/systemvirsh migrate --live <guestname> qemu+ssh://<destination host>/system

Page 34: KVM tools and enterprise usage

Virsh : snapshots

● Only on qcow2 images

# virsh snapshot-create <domain># virsh snapshot-create <domain>

# virsh snapshot-list <domain># virsh snapshot-list <domain>

Name Creation Time State---------------------------------------------------

1295973577 2011-01-25 17:39:37 +0100 running 1295978837 2011-01-25 19:07:17 +0100 running

Name Creation Time State---------------------------------------------------

1295973577 2011-01-25 17:39:37 +0100 running 1295978837 2011-01-25 19:07:17 +0100 running

# virsh snapshot-restore <domain> <snapshotname># virsh snapshot-restore <domain> <snapshotname>

Page 35: KVM tools and enterprise usage

Qemu

Page 36: KVM tools and enterprise usage

qemu

● KVM istself cannot create VM's

● KVM is “just” a hypervisor

● Qemu as user space process

● Qemu communicates via /dev/kvm

Page 37: KVM tools and enterprise usage

qemu

● Qemu-img

– Check

– Convert

– Create

– Snapshot

– Info

Page 38: KVM tools and enterprise usage

Qemu

[root@HV01 ~]# qemu-img info /var/lib/libvirt/images/WIN7.img image: /var/lib/libvirt/images/WIN7.imgfile format: rawvirtual size: 12G (12884901888 bytes)disk size: 12G[root@HV01 ~]#

[root@HV01 ~]# qemu-img info /var/lib/libvirt/images/WIN7.img image: /var/lib/libvirt/images/WIN7.imgfile format: rawvirtual size: 12G (12884901888 bytes)disk size: 12G[root@HV01 ~]#

Inspect image

Page 39: KVM tools and enterprise usage

qemu

Example :

Convert raw to vmdk image

qemu-img convert -O vmdk <raw image> outimage.vmdkqemu-img convert -O vmdk <raw image> outimage.vmdk

Page 40: KVM tools and enterprise usage

Qemu

● file

● blkverify

● sheepdog

● blkdebug

● qcow2

● vvfat

● cow

● Raw

● vmdk

● vdi

● qcow

Supported disk formats

Page 41: KVM tools and enterprise usage

virtio

● "full virtualization" is a nice feature because It allows you to run any operating system virtualized.

● However, it's slow because the hypervisor has

to emulate actual physical devices.

Page 42: KVM tools and enterprise usage

virtio

● Virtio drivers solve this

● Virtio drivers know they run in a virtualized environment. (no emulation)

● Virtio drivers operate with hypervisor

Page 43: KVM tools and enterprise usage

virt-*

Page 44: KVM tools and enterprise usage

virt-*

● Virt-install

● Virt-viewer

● Virt-clone

● virt-top

Page 45: KVM tools and enterprise usage

Virt-install

[root@localhost ~]# virt-install --connect qemu:///system --graphics vnc,listen=0.0.0.0 --name=vm01 --ram 512 --disk pool=LOCAL-STORAGE,size=10,bus=virtio,cache=none /--network network=default,model=virtio /--location=http://ftp.debian.org/debian/dists/squeeze/main/installer-amd64 Cannot open display: Run 'virt-viewer --help' to see a full list of available command line optionsDomain installation still in progress. You can reconnect to the console to complete the installation process.

[root@localhost ~]# virt-install --connect qemu:///system --graphics vnc,listen=0.0.0.0 --name=vm01 --ram 512 --disk pool=LOCAL-STORAGE,size=10,bus=virtio,cache=none /--network network=default,model=virtio /--location=http://ftp.debian.org/debian/dists/squeeze/main/installer-amd64 Cannot open display: Run 'virt-viewer --help' to see a full list of available command line optionsDomain installation still in progress. You can reconnect to the console to complete the installation process.

● Can create VM and disks in one go

Page 46: KVM tools and enterprise usage

Virt-viewer

● Connect to the VM's console with – RDP – VNC – SPICE

● Default vnc port -> 5900

● virt-viewer

virt-viewer -c qemu:///system xpsp2virt-viewer -c qemu:///system xpsp2

Page 47: KVM tools and enterprise usage

virt-clone

● Clones an existing VM (Duh!!)

● Copies disk images

● Defines new machine based on source

– MAC address is auto generated

● Deploy many similar machines from one image

(template).

Page 48: KVM tools and enterprise usage

[root@HV01 ~]# virt-clone -o vm01 -n vm01-copy -f / /VMS/vm01-copy.imgAllocating 'vm01-copy.img' | 10 GB 03:37

Clone 'vm01-copy' created successfully.[root@HV01 ~]#

[root@HV01 ~]# virt-clone -o vm01 -n vm01-copy -f / /VMS/vm01-copy.imgAllocating 'vm01-copy.img' | 10 GB 03:37

Clone 'vm01-copy' created successfully.[root@HV01 ~]#

virt-clone

Page 49: KVM tools and enterprise usage

virt-top

virt-top 20:40:19 - x86_64 2/2CPU 2666MHz 3962MB4 domains, 4 active, 4 running, 0 sleeping, 0 paused, 0 inactive D:0 O:0 X:0CPU: 2.8% Mem: 2560 MB (2560 MB by guests)

ID S RDRQ WRRQ RXBY TXBY %CPU %MEM TIME NAME 14 R 0 0 1220 0 2.5 25.0 315:52.95 WIN7 15 R 0 4 146 42 0.2 12.0 0:54.64 vm01 11 R 0 0 0.2 12.0 15:38.34 vm02-fromxml 16 R 0 0 104 0 0.0 12.0 0:54.82 vm01-copy

virt-top 20:40:19 - x86_64 2/2CPU 2666MHz 3962MB4 domains, 4 active, 4 running, 0 sleeping, 0 paused, 0 inactive D:0 O:0 X:0CPU: 2.8% Mem: 2560 MB (2560 MB by guests)

ID S RDRQ WRRQ RXBY TXBY %CPU %MEM TIME NAME 14 R 0 0 1220 0 2.5 25.0 315:52.95 WIN7 15 R 0 4 146 42 0.2 12.0 0:54.64 vm01 11 R 0 0 0.2 12.0 15:38.34 vm02-fromxml 16 R 0 0 104 0 0.0 12.0 0:54.82 vm01-copy

Page 50: KVM tools and enterprise usage

Virtual machine manager

Page 51: KVM tools and enterprise usage

Virtual Machine Manager

● A GUI for virsh and virt-*

● Runs only on Linux/Unix

● Can discover machines running libvirtd (nodes)

● Local development / testing

Page 52: KVM tools and enterprise usage

Virtual Machine Manager

Page 53: KVM tools and enterprise usage

libguestfs

libguestfs :Converting, Inspecting and modifying VM's

Page 54: KVM tools and enterprise usage

libguestfs

● Manipulation of VM images

● Inspection of VM images

● Exists out of several separate tools

– List is available on http://virt-tools.org/docs/index.html

● No need to login on the VM itself

● Can be run on an active VM

Page 55: KVM tools and enterprise usage

What is this image?

Developers like to keep stuff

Is it still needed?

Page 56: KVM tools and enterprise usage

Libguestfs : virt-inspector

● What if you just have an image

● You cannot login to machine

● You would like to know what version of a

package is installed

● Generate list of all VM's

Page 57: KVM tools and enterprise usage

Libguestfs : virt-inspector

virt-inspector -d WIN7virt-inspector -d WIN7

[root@HV01 VMS]# virt-inspector -a /var/lib/libvirt/images/WIN7.img [root@HV01 VMS]# virt-inspector -a /var/lib/libvirt/images/WIN7.img

Inspect the domain

Inspect an image

Page 58: KVM tools and enterprise usage

1 <name>windows</name>

2 <arch>i386</arch>

3 <distro>windows</distro>

4 <product_name>Windows 7 Professional</product_name>

5 <product_variant>Client</product_variant>

6 <major_version>6</major_version>

7 <minor_version>1</minor_version>

8 <windows_systemroot>/Windows</windows_systemroot>

9 <windows_current_control_set>ControlSet001</windows_current_control_set>

10 <hostname>btr-PC</hostname>

11 <format>installed</format>

Page 59: KVM tools and enterprise usage

1 <applications>

2 <application>

3 <name>Mozilla Firefox 6.0.2 (x86 en-US)</name>

4 <display_name>Mozilla Firefox 6.0.2 (x86 en-US)</display_name>

5 <version>6.0.2</version>

6 <install_path>C:\Program Files\Mozilla Firefox</install_path>

7 <publisher>Mozilla</publisher>

8 <url>http://www.mozilla.com/en-US/</url>

9 <description>Mozilla Firefox 6.0.2 (x86 en-US)</description>

10 </application>

11 </applications>

Page 60: KVM tools and enterprise usage

Libguestfs : virt-df

[root@HV01 ~]# virt-df vm02-fromxmlFilesystem 1K-blocks Used Available Use%vm02-fromxml:/dev/sdb 3516418 3516418 0 100%vm02-fromxml:/dev/sda1 495844 30573 439671 7%vm02-fromxml:/dev/VolGroup/lv_root 3652680 623180 2843952 18%[root@HV01 ~]#

[root@HV01 ~]# virt-df vm02-fromxmlFilesystem 1K-blocks Used Available Use%vm02-fromxml:/dev/sdb 3516418 3516418 0 100%vm02-fromxml:/dev/sda1 495844 30573 439671 7%vm02-fromxml:/dev/VolGroup/lv_root 3652680 623180 2843952 18%[root@HV01 ~]#

● Allows you to view filesystem layout and space free/used

Virt-df <name of domain>

Page 61: KVM tools and enterprise usage

Libguestfs : virt-win-reg

● Modify registry of Windows machines

virt-win-reg --merge <vmname>.img viostor-regvirt-win-reg --merge <vmname>.img viostor-reg

Page 62: KVM tools and enterprise usage

Libguestfs : virt-win-reg

guestfish -i <vmname>.img

<fs>upload viostor.sys /WINDOWS/system32/drivers/viostor.sys

guestfish -i <vmname>.img

<fs>upload viostor.sys /WINDOWS/system32/drivers/viostor.sys

virt-win-reg --merge <vmname>.img viostor-regvirt-win-reg --merge <vmname>.img viostor-reg

Manually Prepare a Windows image with virtio drivers

Page 63: KVM tools and enterprise usage

Libguestfs : virt-tar-out

● Create backups from files on the VM

# virt-tar-out -d vm01 /home home-vm01.tar# virt-tar-out -d vm01 /home home-vm01.tar

● Create backups from files on the disk image

# virt-tar-out -a /VMS/vm01.img /home home-vm01-disk.tar# virt-tar-out -a /VMS/vm01.img /home home-vm01-disk.tar

Page 64: KVM tools and enterprise usage

virt-v2v : migration of virtual machines

Page 65: KVM tools and enterprise usage

Virt-v2v : short overview

● Tool to automate migrations to KVM/RHEV

● Migrate to KVM or RHEV from

– XEN

– VMWare

– KVM

● Installs virtio drivers and reconfigures machine

● Can be scripted for bulk operations

Page 66: KVM tools and enterprise usage

Virt-v2v : migrate from Vmware to RHEV

Page 67: KVM tools and enterprise usage

Virt-v2v : migrate from VMWare to RHEV

virt-v2v -ic esx://esx.example.com/?no_verify=1 -o rhev -os storage.example.com:/exportdomain --network rhevm vm-name

virt-v2v -ic esx://esx.example.com/?no_verify=1 -o rhev -os storage.example.com:/exportdomain --network rhevm vm-name

Page 68: KVM tools and enterprise usage

Virt-v2v : migrate VMWare to KVM

Page 69: KVM tools and enterprise usage

sVirt : secure your host and VM's

Page 70: KVM tools and enterprise usage

sVirt

Virtualization introduces new risks

Page 71: KVM tools and enterprise usage

sVirt

“Old” days

Machines where physically separated and “attackable” via the network

Page 72: KVM tools and enterprise usage
Page 73: KVM tools and enterprise usage

sVirt

With Virtualization :

A flawed hypervisor can give access to ALL VM's running on it

Bypassing network security

Page 74: KVM tools and enterprise usage
Page 75: KVM tools and enterprise usage

sVirt

sVirt :

Apply MAC for guest and resources

MAC policy is set by hypervisor host

Based on SELinux (don't reinvent the wheel)

Auto apply or manual

Page 76: KVM tools and enterprise usage
Page 77: KVM tools and enterprise usage

[root@HV01 VMS]# ls -Z |grep vm01qemu qemu system_u:object_r:svirt_image_t:s0:c656,c817 vm01-copy.imgqemu qemu system_u:object_r:svirt_image_t:s0:c690,c972 vm01.img

[root@HV01 VMS]# ls -Z |grep vm01qemu qemu system_u:object_r:svirt_image_t:s0:c656,c817 vm01-copy.imgqemu qemu system_u:object_r:svirt_image_t:s0:c690,c972 vm01.img

Disk images

Processes

[root@HV01 VMS]# ps auxZ | grep vm01 |awk '{print $1" " $22}' system_u:system_r:svirt_t:s0:c690,c972 vm01system_u:system_r:svirt_t:s0:c656,c817 vm01-copy

[root@HV01 VMS]# ps auxZ | grep vm01 |awk '{print $1" " $22}' system_u:system_r:svirt_t:s0:c690,c972 vm01system_u:system_r:svirt_t:s0:c656,c817 vm01-copy

Page 78: KVM tools and enterprise usage

A lot of tools but what to do with them?

Page 79: KVM tools and enterprise usage

RHEV is first example

Page 80: KVM tools and enterprise usage

Build your own stack?

Development environment?

Page 81: KVM tools and enterprise usage

Enterprise usage

Page 82: KVM tools and enterprise usage

RHEV 2.x

● RHEV (Red Hat enterprise Virtualization)

– Red Hat started adopting KVM (actually they bought it)

– Released 2.1

– Running on Windows platform ( the horror.. )

– RHEV 2.2 will be latest release in 2.x series

Page 83: KVM tools and enterprise usage

RHEV 3.0

● Port from .Net to Java

● Manager runs on Jboss

– no more windows server side

● Hypervisor based on RHEL 6

– Gives performance increase

● Local storage

● Reporing engine included

● WAN optimized SPICE

Page 84: KVM tools and enterprise usage

RHEV 3.0

● Power User portal

– Users can manage their own environment

● RESTful API

● Will be open sourced

Page 85: KVM tools and enterprise usage

RHEV adoption

Page 86: KVM tools and enterprise usage

OVA

● Open Virtualization Alliance● Foster the adoption of KVM as an enterprise-

ready open virtualization solution ● Accelerate the emergence of an ecosystem of

third-party solutions around KVM.● Encourage interoperability, promote best

practices, and highlight examples of customer successes.

Page 87: KVM tools and enterprise usage

Ova : members

Page 88: KVM tools and enterprise usage

specvirt

● Virtualization benchmark (score@ 'n' VM's)

● KVM

– 3894@240

● VmWare ESX 4.1

– 3723@228

Page 89: KVM tools and enterprise usage

conclusions

● KVM is till young

● Adoption is growing

● Builds on Linux features (feature velocity)

– Selinux, Cgroups, Scheduler, ...

● A lot of community activity

● scalability

● Growing ECO system

● No vendor lockin

● OVA (open virtualization alliance)

Page 90: KVM tools and enterprise usage

Questions?

Page 91: KVM tools and enterprise usage

http://www.flickr.com/photos/kalavinka/4617897952/

http://www.flickr.com/photos/zakh/337938459/

http://www.cyberciti.biz/faq/linux-kvm-vnc-for-guest-machine/

http://www.centos.org/docs/5/html/5.2/Virtualization/chap-Virtualization-Managing_guests_with_virsh.html

http://heprc.phys.uvic.ca/sites/heprc.phys.uvic.ca/files/reports/vliet-wtr.pdf

http://docs.redhat.com/docs/en-US/Red_Hat_Enterprise_Linux/6/html-single/Virtualization/index.html

Page 92: KVM tools and enterprise usage

Grind out appliances

Page 93: KVM tools and enterprise usage

BoxGrinder : overview

● Create a VM “offline”

● Runs on JBOSS

● Use puppet/chef/.. to configure it further

● Move from Fedora to RHEL by adjusting the file