Upload
vincentvdk
View
5.189
Download
1
Tags:
Embed Size (px)
DESCRIPTION
Open Source Virtualization based on KVM and available tools like Libvirt, Libguestfs, Qemu, ...
Citation preview
Virtualize with KVMTools and Enterprise usage
by Vincent Van der Kussen
#whoami
● Using Linux since 1996
● Sysadmin / Engineer / Opensource Enthousiast
● @vincentvdk
● http://blog.vanderkussen.org
So what is this KVM thing anyway?
It's NOT!
KVM : overview
Kernel-based Virtual Machine, a Kernel module that turns Linux into a hypervisor
Created by Quramnet (bought by Red Hat 2008)
Supports x86 (32 and 64 bit), s390, Powerpc
Included in Linux kernel since 2.6.20
Needs a CPU with virtualization extention !
KVM : overview
● Each vm is a process
● Each virtual cpu is a thread
● Open source
Features
CPU and memory overcommit
High performance paravirtual I/O
Hotplug (cpu, block, nic)
SMP guests
Live Migration
Power management
Features
PCI Device Assigenment and SR-IOV
KSM (Kernel Samepage Merging)
SPICE
NUMA
Not your typical VMWare workstation or VirtualBox.
VirtManager comes close but still a little rough on the edges
KVM is meant for virtualization on servers. Desktop usage is not always ideal (think vSphere, HyperV,..).
KVM : differences
Overview Tools
libvirt (api)
Virsh
Qemu
VirtManager
Libguestfs
virt-v2v
Overview tools
sVirt
virt-*
Boxgrinder
VirtIO (not really a tool)
Spice
Libvirt : The Virtualization API
Libvirt : overview
● “The Virtualization API”
● Toolkit to interact with the virtualization layer
● Written in C
● LGPL● Has bindings for common languages (Python,
Ruby, Perl, PHP, Java,..)● Supports AMQP via libvirt-qpid
Libvirt : overview
● Provides management of
– Virtual machines
– Virtual networks
– Storage
● Does NOT provide multi node management features like load balancing
● Suspend / resume support
Libvirt : terminology
Node
a physical machine
Libvirt : termonology
Hypervisor
A software layer that allows a node to run virtual
machines
Libvirt : terminologie
Domain
An instance (“virtual machine”) running on the
hypervisor
Libvirt : overview
Supports also :
XEN KVM
VMWare
LXC
OpenVZ
Microsoft HyperV
Virtualbox
virsh
Virsh overview
Command line for libvirt
Enables scripting
Libvirtd needs to be started
Most commands require root privileges
Virsh usage
virsh <command> <domain-id> [OPTIONS]virsh <command> <domain-id> [OPTIONS]
Virsh usage
Connect to hypervisor on the localhost
virsh connect qemu:///systemvirsh connect qemu:///system
Connect to hypervisor on remote system over SSH
virsh -c qemu+ssh://[email protected]/systemvirsh -c qemu+ssh://[email protected]/system
virshvirsh
or
Virsh usage
Virsh : storage
● Define storage pool● Local filesystem (ext3, ext4,...) ● Directory based● ISCSI● NFS● LVM (uses volume group)● Physical disk● multipath
Virsh : create storage pool
[root@localhost ~]# virsh pool-define dir_pool.xml Pool LOCAL-STORAGE defined from dir_pool.xml[root@localhost ~]# virsh pool-define dir_pool.xml Pool LOCAL-STORAGE defined from dir_pool.xml
[root@localhost ~]# virsh pool-start LOCAL-STORAGEPool LOCAL-STORAGE started[root@localhost ~]# virsh pool-start LOCAL-STORAGEPool LOCAL-STORAGE started
<pool type="dir"><name>LOCAL-STORAGE</name> <target> <path>/VMS</path> </target></pool>
Xml file ->
Virsh : create storage pool
[root@localhost ~]# virsh pool-listName State Autostart -----------------------------------------default active yes LOCAL-STORAGE active no
[root@localhost ~]# virsh pool-listName State Autostart -----------------------------------------default active yes LOCAL-STORAGE active no
[root@localhost ~]# virsh pool-autostart LOCAL-STORAGEPool LOCAL-STORAGE marked as autostarted
[root@localhost ~]# virsh pool-autostart LOCAL-STORAGEPool LOCAL-STORAGE marked as autostarted
Virsh : create VM
● VM is defined in xml file
● Manual creation of VM's disks
[root@localhost ~]# qemu-img create /VMS/vm02-fromxml.img 5GFormatting '/VMS/vm02-fromxml.img', fmt=raw size=5368709120 [root@localhost ~]# qemu-img create /VMS/vm02-fromxml.img 5GFormatting '/VMS/vm02-fromxml.img', fmt=raw size=5368709120
● Define VM in libvirt and start it
[root@localhost ~]# virsh define vm02_from_xml.xmlDomain vm02-fromxml defined from vm02_from_xml.xml
[root@localhost ~]# virsh start vm02-fromxmlDomain vm02-fromxml started
[root@localhost ~]# virsh define vm02_from_xml.xmlDomain vm02-fromxml defined from vm02_from_xml.xml
[root@localhost ~]# virsh start vm02-fromxmlDomain vm02-fromxml started
1 <domain type='kvm'> 2 <name>vm02-fromxml</name> 3 <uuid></uuid> 4 <memory>128288</memory> 5 <currentMemory></currentMemory> 6 <vcpu>2</vcpu> 7 <os> 8 <type arch='x86_64' machine='pc-0.14'>hvm</type> 9 <boot dev='hd'/>10 </os>11 <features>12 # features such as ACPI are defined here13 </features>14 <devices>15 <emulator>/usr/bin/qemu-kvm</emulator>16 <disk type='file' device='disk'>17 <driver name='qemu' type='raw' cache='none'/>18 <source file='/VMS/vm02-fromxml.img'/>19 <target dev='vda' bus='virtio'/>20 <alias name='virtio-disk0'/>21 <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>22 </disk>23 </devices>24 </domain>
Virsh : list vm's
[root@localhost ~]# virsh list --all Id Name State---------------------------------- 4 vm01 running - vm02-fromxml shut off
[root@localhost ~]#
[root@localhost ~]# virsh list --all Id Name State---------------------------------- 4 vm01 running - vm02-fromxml shut off
[root@localhost ~]#
Virsh : live migration
virsh migrate --live <guestname> qemu+ssh://<destination host>/systemvirsh migrate --live <guestname> qemu+ssh://<destination host>/system
Virsh : snapshots
● Only on qcow2 images
# virsh snapshot-create <domain># virsh snapshot-create <domain>
# virsh snapshot-list <domain># virsh snapshot-list <domain>
Name Creation Time State---------------------------------------------------
1295973577 2011-01-25 17:39:37 +0100 running 1295978837 2011-01-25 19:07:17 +0100 running
Name Creation Time State---------------------------------------------------
1295973577 2011-01-25 17:39:37 +0100 running 1295978837 2011-01-25 19:07:17 +0100 running
# virsh snapshot-restore <domain> <snapshotname># virsh snapshot-restore <domain> <snapshotname>
Qemu
qemu
● KVM istself cannot create VM's
● KVM is “just” a hypervisor
● Qemu as user space process
● Qemu communicates via /dev/kvm
qemu
● Qemu-img
– Check
– Convert
– Create
– Snapshot
– Info
Qemu
[root@HV01 ~]# qemu-img info /var/lib/libvirt/images/WIN7.img image: /var/lib/libvirt/images/WIN7.imgfile format: rawvirtual size: 12G (12884901888 bytes)disk size: 12G[root@HV01 ~]#
[root@HV01 ~]# qemu-img info /var/lib/libvirt/images/WIN7.img image: /var/lib/libvirt/images/WIN7.imgfile format: rawvirtual size: 12G (12884901888 bytes)disk size: 12G[root@HV01 ~]#
Inspect image
qemu
Example :
Convert raw to vmdk image
qemu-img convert -O vmdk <raw image> outimage.vmdkqemu-img convert -O vmdk <raw image> outimage.vmdk
Qemu
● file
● blkverify
● sheepdog
● blkdebug
● qcow2
● vvfat
● cow
● Raw
● vmdk
● vdi
● qcow
Supported disk formats
virtio
● "full virtualization" is a nice feature because It allows you to run any operating system virtualized.
● However, it's slow because the hypervisor has
to emulate actual physical devices.
virtio
● Virtio drivers solve this
● Virtio drivers know they run in a virtualized environment. (no emulation)
● Virtio drivers operate with hypervisor
virt-*
virt-*
● Virt-install
● Virt-viewer
● Virt-clone
● virt-top
Virt-install
[root@localhost ~]# virt-install --connect qemu:///system --graphics vnc,listen=0.0.0.0 --name=vm01 --ram 512 --disk pool=LOCAL-STORAGE,size=10,bus=virtio,cache=none /--network network=default,model=virtio /--location=http://ftp.debian.org/debian/dists/squeeze/main/installer-amd64 Cannot open display: Run 'virt-viewer --help' to see a full list of available command line optionsDomain installation still in progress. You can reconnect to the console to complete the installation process.
[root@localhost ~]# virt-install --connect qemu:///system --graphics vnc,listen=0.0.0.0 --name=vm01 --ram 512 --disk pool=LOCAL-STORAGE,size=10,bus=virtio,cache=none /--network network=default,model=virtio /--location=http://ftp.debian.org/debian/dists/squeeze/main/installer-amd64 Cannot open display: Run 'virt-viewer --help' to see a full list of available command line optionsDomain installation still in progress. You can reconnect to the console to complete the installation process.
● Can create VM and disks in one go
Virt-viewer
● Connect to the VM's console with – RDP – VNC – SPICE
● Default vnc port -> 5900
● virt-viewer
virt-viewer -c qemu:///system xpsp2virt-viewer -c qemu:///system xpsp2
virt-clone
● Clones an existing VM (Duh!!)
● Copies disk images
● Defines new machine based on source
– MAC address is auto generated
● Deploy many similar machines from one image
(template).
[root@HV01 ~]# virt-clone -o vm01 -n vm01-copy -f / /VMS/vm01-copy.imgAllocating 'vm01-copy.img' | 10 GB 03:37
Clone 'vm01-copy' created successfully.[root@HV01 ~]#
[root@HV01 ~]# virt-clone -o vm01 -n vm01-copy -f / /VMS/vm01-copy.imgAllocating 'vm01-copy.img' | 10 GB 03:37
Clone 'vm01-copy' created successfully.[root@HV01 ~]#
virt-clone
virt-top
virt-top 20:40:19 - x86_64 2/2CPU 2666MHz 3962MB4 domains, 4 active, 4 running, 0 sleeping, 0 paused, 0 inactive D:0 O:0 X:0CPU: 2.8% Mem: 2560 MB (2560 MB by guests)
ID S RDRQ WRRQ RXBY TXBY %CPU %MEM TIME NAME 14 R 0 0 1220 0 2.5 25.0 315:52.95 WIN7 15 R 0 4 146 42 0.2 12.0 0:54.64 vm01 11 R 0 0 0.2 12.0 15:38.34 vm02-fromxml 16 R 0 0 104 0 0.0 12.0 0:54.82 vm01-copy
virt-top 20:40:19 - x86_64 2/2CPU 2666MHz 3962MB4 domains, 4 active, 4 running, 0 sleeping, 0 paused, 0 inactive D:0 O:0 X:0CPU: 2.8% Mem: 2560 MB (2560 MB by guests)
ID S RDRQ WRRQ RXBY TXBY %CPU %MEM TIME NAME 14 R 0 0 1220 0 2.5 25.0 315:52.95 WIN7 15 R 0 4 146 42 0.2 12.0 0:54.64 vm01 11 R 0 0 0.2 12.0 15:38.34 vm02-fromxml 16 R 0 0 104 0 0.0 12.0 0:54.82 vm01-copy
Virtual machine manager
Virtual Machine Manager
● A GUI for virsh and virt-*
● Runs only on Linux/Unix
● Can discover machines running libvirtd (nodes)
● Local development / testing
Virtual Machine Manager
libguestfs
libguestfs :Converting, Inspecting and modifying VM's
libguestfs
● Manipulation of VM images
● Inspection of VM images
● Exists out of several separate tools
– List is available on http://virt-tools.org/docs/index.html
● No need to login on the VM itself
● Can be run on an active VM
What is this image?
Developers like to keep stuff
Is it still needed?
Libguestfs : virt-inspector
● What if you just have an image
● You cannot login to machine
● You would like to know what version of a
package is installed
● Generate list of all VM's
Libguestfs : virt-inspector
virt-inspector -d WIN7virt-inspector -d WIN7
[root@HV01 VMS]# virt-inspector -a /var/lib/libvirt/images/WIN7.img [root@HV01 VMS]# virt-inspector -a /var/lib/libvirt/images/WIN7.img
Inspect the domain
Inspect an image
1 <name>windows</name>
2 <arch>i386</arch>
3 <distro>windows</distro>
4 <product_name>Windows 7 Professional</product_name>
5 <product_variant>Client</product_variant>
6 <major_version>6</major_version>
7 <minor_version>1</minor_version>
8 <windows_systemroot>/Windows</windows_systemroot>
9 <windows_current_control_set>ControlSet001</windows_current_control_set>
10 <hostname>btr-PC</hostname>
11 <format>installed</format>
1 <applications>
2 <application>
3 <name>Mozilla Firefox 6.0.2 (x86 en-US)</name>
4 <display_name>Mozilla Firefox 6.0.2 (x86 en-US)</display_name>
5 <version>6.0.2</version>
6 <install_path>C:\Program Files\Mozilla Firefox</install_path>
7 <publisher>Mozilla</publisher>
8 <url>http://www.mozilla.com/en-US/</url>
9 <description>Mozilla Firefox 6.0.2 (x86 en-US)</description>
10 </application>
11 </applications>
Libguestfs : virt-df
[root@HV01 ~]# virt-df vm02-fromxmlFilesystem 1K-blocks Used Available Use%vm02-fromxml:/dev/sdb 3516418 3516418 0 100%vm02-fromxml:/dev/sda1 495844 30573 439671 7%vm02-fromxml:/dev/VolGroup/lv_root 3652680 623180 2843952 18%[root@HV01 ~]#
[root@HV01 ~]# virt-df vm02-fromxmlFilesystem 1K-blocks Used Available Use%vm02-fromxml:/dev/sdb 3516418 3516418 0 100%vm02-fromxml:/dev/sda1 495844 30573 439671 7%vm02-fromxml:/dev/VolGroup/lv_root 3652680 623180 2843952 18%[root@HV01 ~]#
● Allows you to view filesystem layout and space free/used
Virt-df <name of domain>
Libguestfs : virt-win-reg
● Modify registry of Windows machines
virt-win-reg --merge <vmname>.img viostor-regvirt-win-reg --merge <vmname>.img viostor-reg
Libguestfs : virt-win-reg
guestfish -i <vmname>.img
<fs>upload viostor.sys /WINDOWS/system32/drivers/viostor.sys
guestfish -i <vmname>.img
<fs>upload viostor.sys /WINDOWS/system32/drivers/viostor.sys
virt-win-reg --merge <vmname>.img viostor-regvirt-win-reg --merge <vmname>.img viostor-reg
Manually Prepare a Windows image with virtio drivers
Libguestfs : virt-tar-out
● Create backups from files on the VM
# virt-tar-out -d vm01 /home home-vm01.tar# virt-tar-out -d vm01 /home home-vm01.tar
● Create backups from files on the disk image
# virt-tar-out -a /VMS/vm01.img /home home-vm01-disk.tar# virt-tar-out -a /VMS/vm01.img /home home-vm01-disk.tar
virt-v2v : migration of virtual machines
Virt-v2v : short overview
● Tool to automate migrations to KVM/RHEV
● Migrate to KVM or RHEV from
– XEN
– VMWare
– KVM
● Installs virtio drivers and reconfigures machine
● Can be scripted for bulk operations
Virt-v2v : migrate from Vmware to RHEV
Virt-v2v : migrate from VMWare to RHEV
virt-v2v -ic esx://esx.example.com/?no_verify=1 -o rhev -os storage.example.com:/exportdomain --network rhevm vm-name
virt-v2v -ic esx://esx.example.com/?no_verify=1 -o rhev -os storage.example.com:/exportdomain --network rhevm vm-name
Virt-v2v : migrate VMWare to KVM
sVirt : secure your host and VM's
sVirt
Virtualization introduces new risks
sVirt
“Old” days
Machines where physically separated and “attackable” via the network
sVirt
With Virtualization :
A flawed hypervisor can give access to ALL VM's running on it
Bypassing network security
sVirt
sVirt :
Apply MAC for guest and resources
MAC policy is set by hypervisor host
Based on SELinux (don't reinvent the wheel)
Auto apply or manual
[root@HV01 VMS]# ls -Z |grep vm01qemu qemu system_u:object_r:svirt_image_t:s0:c656,c817 vm01-copy.imgqemu qemu system_u:object_r:svirt_image_t:s0:c690,c972 vm01.img
[root@HV01 VMS]# ls -Z |grep vm01qemu qemu system_u:object_r:svirt_image_t:s0:c656,c817 vm01-copy.imgqemu qemu system_u:object_r:svirt_image_t:s0:c690,c972 vm01.img
Disk images
Processes
[root@HV01 VMS]# ps auxZ | grep vm01 |awk '{print $1" " $22}' system_u:system_r:svirt_t:s0:c690,c972 vm01system_u:system_r:svirt_t:s0:c656,c817 vm01-copy
[root@HV01 VMS]# ps auxZ | grep vm01 |awk '{print $1" " $22}' system_u:system_r:svirt_t:s0:c690,c972 vm01system_u:system_r:svirt_t:s0:c656,c817 vm01-copy
A lot of tools but what to do with them?
RHEV is first example
Build your own stack?
Development environment?
Enterprise usage
RHEV 2.x
● RHEV (Red Hat enterprise Virtualization)
– Red Hat started adopting KVM (actually they bought it)
– Released 2.1
– Running on Windows platform ( the horror.. )
– RHEV 2.2 will be latest release in 2.x series
RHEV 3.0
● Port from .Net to Java
● Manager runs on Jboss
– no more windows server side
● Hypervisor based on RHEL 6
– Gives performance increase
● Local storage
● Reporing engine included
● WAN optimized SPICE
RHEV 3.0
● Power User portal
– Users can manage their own environment
● RESTful API
● Will be open sourced
RHEV adoption
OVA
● Open Virtualization Alliance● Foster the adoption of KVM as an enterprise-
ready open virtualization solution ● Accelerate the emergence of an ecosystem of
third-party solutions around KVM.● Encourage interoperability, promote best
practices, and highlight examples of customer successes.
Ova : members
specvirt
● Virtualization benchmark (score@ 'n' VM's)
● KVM
– 3894@240
● VmWare ESX 4.1
– 3723@228
conclusions
● KVM is till young
● Adoption is growing
● Builds on Linux features (feature velocity)
– Selinux, Cgroups, Scheduler, ...
● A lot of community activity
● scalability
● Growing ECO system
● No vendor lockin
● OVA (open virtualization alliance)
Questions?
http://www.flickr.com/photos/kalavinka/4617897952/
http://www.flickr.com/photos/zakh/337938459/
http://www.cyberciti.biz/faq/linux-kvm-vnc-for-guest-machine/
http://www.centos.org/docs/5/html/5.2/Virtualization/chap-Virtualization-Managing_guests_with_virsh.html
http://heprc.phys.uvic.ca/sites/heprc.phys.uvic.ca/files/reports/vliet-wtr.pdf
http://docs.redhat.com/docs/en-US/Red_Hat_Enterprise_Linux/6/html-single/Virtualization/index.html
Grind out appliances
BoxGrinder : overview
● Create a VM “offline”
● Runs on JBOSS
● Use puppet/chef/.. to configure it further
● Move from Fedora to RHEL by adjusting the file
●