Upload
others
View
29
Download
15
Embed Size (px)
Citation preview
TitlePageKVMVirtualizationCookbookLearnhowtoeffectivelyuseKVMinproductionKonstantinIvanov
BIRMINGHAM-MUMBAI
Copyright
KVMVirtualizationCookbook
Copyright©2017PacktPublishing
Allrightsreserved.Nopartofthisbookmaybereproduced,storedinaretrievalsystem,ortransmittedinanyformorbyanymeans,withoutthepriorwrittenpermissionofthepublisher,exceptinthecaseofbriefquotationsembeddedincriticalarticlesorreviews.
Everyefforthasbeenmadeinthepreparationofthisbooktoensuretheaccuracyoftheinformationpresented.However,theinformationcontainedinthisbookissoldwithoutwarranty,eitherexpressorimplied.Neithertheauthor,norPacktPublishing,anditsdealersanddistributorswillbeheldliableforanydamagescausedorallegedtobecauseddirectlyorindirectlybythisbook.
PacktPublishinghasendeavoredtoprovidetrademarkinformationaboutallofthecompaniesandproductsmentionedinthisbookbytheappropriateuseofcapitals.However,PacktPublishingcannotguaranteetheaccuracyofthisinformation.
Firstpublished:June2017
Productionreference:1140617
PublishedbyPacktPublishingLtd.LiveryPlace35LiveryStreetBirminghamB32PB,UK.
ISBN978-1-78829-467-6
www.packtpub.com
Credits
Author
KonstantinIvanov
CopyEditor
DiptiMankame
Reviewer
JayPayne
ProjectCoordinator
JudieJose
AcquisitionEditor
RahulNair
Proofreader
SafisEditing
ContentDevelopmentEditor
DevikaBattike
Indexer
RekhaNair
TechnicalEditor
PrachiSawant
Graphics
KirkD'Penha
ProductionCoordinator
AparnaBhagat
AbouttheAuthorKonstantinIvanovisaLinuxsystemsengineer,anopensourcedeveloper,andatechnologybloggerwhohasbeendesigning,configuring,deploying,andadministeringlarge-scale,highlyavailableLinuxenvironmentsformorethan15years.
Hisinterestsincludelargedistributedsystemsandtaskautomation,alongwithsolvingtechnicalchallengesinvolvingmultipletechnologystacks.
KonstantinreceivedtwomasterofscienceincomputersciencedegreesfromuniversitiesinBulgariaandtheUnitedStates,specializinginsystemandnetworksecurityandsoftwareengineering.
Inhissparetime,heloveswritingtechnologyblogsandspendingtimewithhistwoboys.
Tomyamazingwife,Deepa,mytrustycompanionthroughthemanylongdaysofwriting.
AbouttheReviewerJayPaynehasbeenadatabaseadministrator5atRackspaceforover10years,workingonthedesign,development,implementation,andoperationofstoragesystems.
Previously,Jayworkedonbillingandsupportsystemsforhostingcompanies.Forthelast20years,hehasprimarilyfocusedonthedatalifecycle,fromdatabasearchitecture,administration,operations,andreportingtodisasterrecoveryandcompliance.Hehasdomainexperienceinhosting,finance,billing,andcustomer-supportindustries.
www.PacktPub.comForsupportfilesanddownloadsrelatedtoyourbook,pleasevisitwww.PacktPub.com.
DidyouknowthatPacktofferseBookversionsofeverybookpublished,withPDFandePubfilesavailable?YoucanupgradetotheeBookversionatwww.PacktPub.comandasaprintbookcustomer,youareentitledtoadiscountontheeBookcopy.Getintouchwithusatservice@packtpub.comformoredetails.
Atwww.PacktPub.com,youcanalsoreadacollectionoffreetechnicalarticles,signupforarangeoffreenewslettersandreceiveexclusivediscountsandoffersonPacktbooksandeBooks.
https://www.packtpub.com/mapt
Getthemostin-demandsoftwareskillswithMapt.MaptgivesyoufullaccesstoallPacktbooksandvideocourses,aswellasindustry-leadingtoolstohelpyouplanyourpersonaldevelopmentandadvanceyourcareer.
Whysubscribe?
FullysearchableacrosseverybookpublishedbyPacktCopyandpaste,print,andbookmarkcontentOndemandandaccessibleviaawebbrowser
CustomerFeedbackThanksforpurchasingthisPacktbook.AtPackt,qualityisattheheartofoureditorialprocess.Tohelpusimprove,pleaseleaveusanhonestreviewonthisbook'sAmazonpageathttps://www.amazon.com/dp/178829467X.
Ifyou'dliketojoinourteamofregularreviewers,youcane-mailusatcustomerreviews@packtpub.com.WeawardourregularreviewerswithfreeeBooksandvideosinexchangefortheirvaluablefeedback.Helpusberelentlessinimprovingourproducts!
TableofContents
PrefaceWhatthisbookcoversWhatyouneedforthisbookWhothisbookisforSections
GettingreadyHowtodoit…Howitworks…There'smore…Seealso
ConventionsReaderfeedbackCustomersupport
DownloadingtheexamplecodeDownloadingthecolorimagesofthisbookErrataPiracyQuestions
1. GettingStartedwithQEMUandKVMIntroductionInstallingandconfiguringQEMU
GettingreadyHowtodoit...Howitworks...
Managingdiskimageswithqemu-imgGettingreadyHowtodoit...Howitworks...There'smore...
PreparingimagesforOSinstallationwithqemu-nbdGettingreadyHowtodoit...Howitworks...
InstallingacustomOSontheimagewithdebootstrapGettingready
Howtodoit...Howitworks...
ResizinganimageGettingreadyHowtodoit...Howitworks...
Usingpre-existingimagesGettingreadyHowtodoit...Howitworks...There'smore...Seealso
Runningvirtualmachineswithqemu-system-*GettingreadyHowtodoit...Howitworks...
StartingtheQEMUVMwithKVMsupportGettingreadyHowtodoit...Howitworks...There'smore...
ConnectingtoarunninginstancewithVNCGettingreadyHowtodoit...Howitworks...
2. UsinglibvirttoManageKVMIntroductionInstallingandconfiguringlibvirt
GettingreadyHowtodoit...Howitworks...
DefiningKVMinstancesGettingreadyHowtodoit...Howitworks...There'smore...
Starting,stopping,andremovingKVMinstancesGettingready
Howtodoit...Howitworks...
InspectingandeditingKVMconfigsGettingreadyHowtodoit...Howitworks...
BuildingnewKVMinstanceswithvirt-installandusingtheconsoleGettingreadyHowtodoit...Howitworks...
ManagingCPUandmemoryresourcesinKVMGettingreadyHowtodoit...Howitworks...There'smore...
AttachingblockdevicestovirtualmachinesGettingreadyHowtodoit...Howitworks...
SharingdirectoriesbetweenarunningVMandthehostOSGettingreadyHowtodoit...Howitworks...There'smore...
AutostartingKVMinstancesGettingreadyHowtodoit...Howitworks...
WorkingwithstoragepoolsGettingreadyHowtodoit...Howitworks...There'smore...
ManagingvolumesGettingreadyHowtodoit...Howitworks...
ManagingsecretsGettingready
Howtodoit...Howitworks...
3. KVMNetworkingwithlibvirtIntroductionTheLinuxbridge
GettingreadyHowtodoit...Howitworks...There'smore...
TheOpenvSwitchGettingreadyHowtodoit...Howitworks...There'smore...
ConfiguringNATforwardingnetworkGettingreadyHowtodoit...Howitworks...
ConfiguringbridgednetworkGettingreadyHowtodoit...Howitworks...
ConfiguringPCIpassthroughnetworkGettingreadyHowtodoit...Howitworks...
ManipulatingnetworkinterfacesGettingreadyHowtodoit...Howitworks...
4. MigratingKVMInstancesIntroductionManualofflinemigrationusinganiSCSIstoragepool
GettingreadyHowtodoit...Howitworks...There'smore...
ManualofflinemigrationusingGlusterFSsharedvolumes
GettingreadyHowtodoit...Howitworks...
OnlinemigrationusingthevirshcommandwithsharedstorageGettingreadyHowtodoit...Howitworks...There'smore...
OfflinemigrationusingthevirshcommandandlocalimageGettingreadyHowtodoit...Howitworks...
OnlinemigrationusingthevirshcommandandlocalimageGettingreadyHowtodoit...Howitworks...
5. MonitoringandBackupofKVMVirtualMachinesIntroductionResourceusagecollectionwithlibvirt
GettingreadyHowtodoit...Howitworks...There'smore...
MonitoringKVMinstanceswithSensuGettingreadyHowtodoit...Howitworks...There'smore...
SimpleKVMbackupswithtarandrsyncGettingreadyHowtodoit...Howitworks...
CreatingsnapshotsGettingreadyHowtodoit...Howitworks...
ListingsnapshotsGettingready
Howtodoit...Howitworks...
InspectingsnapshotsGettingreadyHowtodoit...Howitworks...
EditingsnapshotsGettingreadyHowtodoit...Howitworks...
RevertingsnapshotsGettingreadyHowtodoit...Howitworks...
DeletingsnapshotsGettingreadyHowtodoit...Howitworks...
6. DeployingKVMInstanceswithOpenStackIntroductionPreparingthehostfortheOpenStackdeployment
GettingreadyHowtodoit...Howitworks...
InstallingandconfiguringtheOpenStackKeystoneidentityserviceGettingreadyHowtodoit...Howitworks...
InstallingandconfiguringtheOpenStackGlanceimageserviceGettingreadyHowtodoit...Howitworks...
InstallingandconfiguringtheOpenStackNovacomputeserviceGettingreadyHowtodoit...Howitworks...
InstallingandconfiguringtheOpenStackNeutronnetworkingserviceGettingreadyHowtodoit...
Howitworks...BuildingandinspectingKVMinstanceswithOpenStack
GettingreadyHowtodoit...Howitworks...
StoppingKVMinstanceswithOpenStackGettingreadyHowtodoit...Howitworks...
TerminatingKVMinstanceswithOpenStackGettingreadyHowtodoit...Howitworks...
7. UsingPythontoBuildandManageKVMInstancesIntroductionInstallingandusingthePythonlibvirtlibrary
GettingreadyHowtodoit...Howitworks...
DefiningKVMinstanceswithPythonGettingreadyHowtodoit...Howitworks...There'smore...
Starting,stopping,anddeletingKVMinstanceswithPythonGettingreadyHowtodoit...Howitworks...There'smore...
InspectingKVMinstanceswithPythonGettingreadyHowtodoit...Howitworks...There'smore...
BuildingasimpleRESTAPIserverwithlibvirtandbottleGettingreadyHowtodoit...Howitworks...
There'smore...8. KernelTuningforKVMPerformance
IntroductionTuningthekernelforlowI/Olatency
GettingreadyHowtodoit...Howitworks...
MemorytuningforKVMguestsGettingreadyHowtodoit...Howitworks...
CPUperformanceoptionsGettingreadyHowtodoit...Howitworks...
NUMAtuningwithlibvirtGettingreadyHowtodoit...Howitworks...Thereismore...
TuningthekernelfornetworkperformanceGettingreadyHowtodoit...Howitworks...
PrefaceThefoundationofmostmodernclouddeploymentsissomesortofvirtualizationtechnology,suchasKernel-basedVirtualMachine(KVM).KVMhasbeenpartofthemainstreamLinuxkernelsinceversion2.6.20,releasedinFebruary2007,andsincethenhasenjoyedwidesystemadoption.
Virtualizationingeneralnotonlyprovidesawaytofullyutilizeserverresources,butalsoallowsgreatermultitenancy,alongwithrunningvariousworkloadsonthesamesystem.
TheOpenStackcloudoperatingsystemusesKVMasitsdefaultcomputedriver,providingacentralizedwayofmanagingthelifecycleofvirtualmachines:frombuilding,resizing,andmigratingtopausingandterminating.
ThisbookisaboutKVMandhowtobuildandmanagevirtualmachinesinthemostefficientway.UnlikecontainerizationsolutionssuchasDocker,KVMisdesignedtorunanentireoperatingsystemratherthanasingleprocess.Althoughcontainerizationhasitsadvantages,fullvirtualizationprovidesextrasecuritybyhavingthehypervisorlayerbetweentheguestOSandthehostandaddedstabilitybyrunningdifferentguestkernels(kernelpanicoftheguestinstancewillnotbringtheentirehostdown)orentirelydifferentoperatingsystems.
Thisbooktakesaratherdirectandpragmaticstep-by-stepapproach--youwilllearnhowtocreatecustomguestimages,installandconfigureQEMUandlibvirt,resizeandmigrateinstances,deploymonitoring,andprovisionguestsusingOpenStackandPython.
WhatthisbookcoversChapter1,GettingStartedwithQEMUandKVM,providesrecipesforinstallingandconfiguringQEMU,creatingandmanagingdiskimages,andrunningvirtualmachineswiththeqemu-systemutility.
Chapter2,UsinglibvirttoManageKVM,coverseverythingthatisneededtoinstall,configure,andrunKVMinstancesusinglibvirt.Youwilllearnwhatpackagesandtoolsarerequired,alongwithdifferentwaysofconfiguringvirtualmachinesusingXMLdefinitionfiles.Bytheendofthischapter,youwillhaveaLinuxsystemwithrunningKVMinstances.
Chapter3,KVMNetworkingwithlibvirt,willpresentrecipesforworkingwiththeLinuxBridgeandOpenvSwitchandwilldemonstratehowtoconnectKVMinstancesusingNAT,bridged,andPCIpass-throughnetworking.
Chapter4,MigratingKVMInstances,willshowexamplesonhowtoperformofflineandonlinemigrationofrunningKVMvirtualmachines.
Chapter5,MonitoringandBackupofKVMVirtualMachines,willpresentexamplesonhowtodeploycompletemonitoringsystemswithSensuandUchiwaanddemonstratehowtocreatesnapshotstouseasbackups.
Chapter6,DeployingKVMInstanceswithOpenStack,demonstrateshowtoprovisionKVMinstanceswithOpenStack.ItbeginsbyintroducingthevariouscomponentsthatmakeOpenStackandhowtousetheLXCNovadrivertoautomaticallyprovisionvirtualmachines.
Chapter7,UsingPythontoBuildandManageKVMInstances,willpresentrecipesforbuilding,starting,andmanagingthelifecycleofKVMinstancesusingthePythonlibvirtlibrary.WewillalsoseeexamplesonhowtobuildasimpleRESTfulAPItoworkwithKVM.
Chapter8,KernelTuningforKVMPerformance,showsrecipesfortuningthehostOSforbetterI/O,CPU,memory,andnetworkutilization.ThepresentedexamplescanalsobeusedinsidetheKVMinstances,dependingontheir
workload.
WhatyouneedforthisbookAbeginner-levelknowledgeofLinuxandthecommandlineisrequiredtofollowalongandruntherecipes.SomePythonexperienceisrequiredtofullyunderstandandbeabletoruntheexamplesinChapter7,UsingPythontoBuildandManageKVMInstances.
MostrecipesinthisbookhavebeentestedonbaremetalserverswithprocessorssupportingvirtualizationandthelatestversionofUbuntuLinux.
WhothisbookisforThisbookisforanyonewhoiscuriousaboutvirtualizationwithKVM--fromLinuxadministratorswhoarelookingforin-depthunderstandingofhowKVMcanbedeployedandmanagedinlargescaleproductionenvironmentstosoftwaredevelopersthatneedaquickandeasywaytoprototypecodeinisolatedguests.ADevOpsengineerismostlikelythebestjobtitleforthosewhowanttoreadthebookfromcovertocoverandtryallexamples.
SectionsInthisbook,youwillfindseveralheadingsthatappearfrequently(Gettingready,Howtodoit,Howitworks,There'smore,andSeealso).
Togiveclearinstructionsonhowtocompletearecipe,weusethesesectionsasfollows:
GettingreadyThissectiontellsyouwhattoexpectintherecipe,anddescribeshowtosetupanysoftwareoranypreliminarysettingsrequiredfortherecipe.
Howtodoit…Thissectioncontainsthestepsrequiredtofollowtherecipe.
Howitworks…Thissectionusuallyconsistsofadetailedexplanationofwhathappenedintheprevioussection.
There'smore…Thissectionconsistsofadditionalinformationabouttherecipeinordertomakethereadermoreknowledgeableabouttherecipe.
SeealsoThissectionprovideshelpfullinkstootherusefulinformationfortherecipe.
ConventionsInthisbook,youwillfindanumberofstylesoftextthatdistinguishbetweendifferentkindsofinformation.Herearesomeexamplesofthesestyles,andanexplanationoftheirmeaning.
Codewordsintext,databasetablenames,foldernames,filenames,fileextensions,pathnames,dummyURLs,userinput,andTwitterhandlesareshownasfollows:"Managingdiskimageswithqemu-img."
Ablockofcodeissetasfollows:
importlibvirt
frombottleimportrun,request,get,post,HTTPResponse
deflibvirtConnect():
try:
conn=libvirt.open('qemu:///system')
exceptlibvirt.libvirtError:
conn=None
returnconn
Anycommand-lineinputoroutputiswrittenasfollows:
root@kvm:~#apt-getupdate
Newtermsandimportantwordsareshowninbold.Wordsthatyouseeonthescreen,inmenusordialogboxesforexample,appearinthetextlikethis:"Thememory_checkfortheKVMinstanceisnowshowingintheUchiwadashboard."
Warningsorimportantnotesappearinaboxlikethis.
Tipsandtricksappearlikethis.
ReaderfeedbackFeedbackfromourreadersisalwayswelcome.Letusknowwhatyouthinkaboutthisbook-whatyoulikedordisliked.Readerfeedbackisimportantforusasithelpsusdeveloptitlesthatyouwillreallygetthemostoutof.
Tosendusgeneralfeedback,[email protected],andmentionthebook'stitleinthesubjectofyourmessage.
Ifthereisatopicthatyouhaveexpertiseinandyouareinterestedineitherwritingorcontributingtoabook,seeourauthorguideatwww.packtpub.com/authors.
CustomersupportNowthatyouaretheproudownerofaPacktbook,wehaveanumberofthingstohelpyoutogetthemostfromyourpurchase.
DownloadingtheexamplecodeYoucandownloadtheexamplecodefilesforthisbookfromyouraccountathttp://www.packtpub.com.Ifyoupurchasedthisbookelsewhere,youcanvisithttp://www.packtpub.com/supportandregistertohavethefilese-maileddirectlytoyou.
Youcandownloadthecodefilesbyfollowingthesesteps:
1. Loginorregistertoourwebsiteusingyoure-mailaddressandpassword.2. HoverthemousepointerontheSUPPORTtabatthetop.3. ClickonCodeDownloads&Errata.4. EnterthenameofthebookintheSearchbox.5. Selectthebookforwhichyou'relookingtodownloadthecodefiles.6. Choosefromthedrop-downmenuwhereyoupurchasedthisbookfrom.7. ClickonCodeDownload.
YoucanalsodownloadthecodefilesbyclickingontheCodeFilesbuttononthebook'swebpageatthePacktPublishingwebsite.Thispagecanbeaccessedbyenteringthebook'snameintheSearchbox.PleasenotethatyouneedtobeloggedintoyourPacktaccount.
Oncethefileisdownloaded,pleasemakesurethatyouunziporextractthefolderusingthelatestversionof:
WinRAR/7-ZipforWindowsZipeg/iZip/UnRarXforMac7-Zip/PeaZipforLinux
ThecodebundleforthebookisalsohostedonGitHubathttps://github.com/PacktPublishing/KVM-Virtualization-Cookbook.Wealsohaveothercodebundlesfromourrichcatalogofbooksandvideosavailableathttps://github.com/PacktPublishing/.Checkthemout!
DownloadingthecolorimagesofthisbookWealsoprovideyouwithaPDFfilethathascolorimagesofthescreenshots/diagramsusedinthisbook.Thecolorimageswillhelpyoubetterunderstandthechangesintheoutput.Youcandownloadthisfilefromhttps://www.packtpub.com/sites/default/files/downloads/KVMVirtualizationCookbook_ColorImages.pdf.
ErrataAlthoughwehavetakeneverycaretoensuretheaccuracyofourcontent,mistakesdohappen.Ifyoufindamistakeinoneofourbooks-maybeamistakeinthetextorthecode-wewouldbegratefulifyoucouldreportthistous.Bydoingso,youcansaveotherreadersfromfrustrationandhelpusimprovesubsequentversionsofthisbook.Ifyoufindanyerrata,pleasereportthembyvisitinghttp://www.packtpub.com/submit-errata,selectingyourbook,clickingontheErrataSubmissionFormlink,andenteringthedetailsofyourerrata.Onceyourerrataareverified,yoursubmissionwillbeacceptedandtheerratawillbeuploadedtoourwebsiteoraddedtoanylistofexistingerrataundertheErratasectionofthattitle.
Toviewthepreviouslysubmittederrata,gotohttps://www.packtpub.com/books/content/supportandenterthenameofthebookinthesearchfield.TherequiredinformationwillappearundertheErratasection.
PiracyPiracyofcopyrightedmaterialontheInternetisanongoingproblemacrossallmedia.AtPackt,wetaketheprotectionofourcopyrightandlicensesveryseriously.IfyoucomeacrossanyillegalcopiesofourworksinanyformontheInternet,pleaseprovideuswiththelocationaddressorwebsitenameimmediatelysothatwecanpursuearemedy.
Pleasecontactusatcopyright@packtpub.comwithalinktothesuspectedpiratedmaterial.
Weappreciateyourhelpinprotectingourauthorsandourabilitytobringyouvaluablecontent.
QuestionsIfyouhaveaproblemwithanyaspectofthisbook,[email protected],andwewilldoourbesttoaddresstheproblem.
GettingStartedwithQEMUandKVMInthischapter,wewillcoverthefollowingtopics:
InstallingandconfiguringQEMUManagingdiskimageswithqemu-imgPreparingimagesforOSinstallationwithqemu-nbdInstallingacustomOSontheimagewithdebootstrapResizinganimageUsingpre-existingimagesRunningvirtualmachineswithqemu-system-*StartingtheQEMUVMwithKVMsupportConnectingtoarunninginstancewithVNC
IntroductionQuickEmulator(QEMU)isthemaincomponentoftheQEMU/KVMvirtualizationtechnologysuit.Itprovideshardwarevirtualizationandprocessoremulation.QEMUrunsinuserspaceand,withouttheneedforkernel,driverscanstillprovidefastsystememulation.QEMUsupportstwooperatingmodes:
Fullsystememulation,whereQEMUemulatesanentirecomputersystem,includingtheCPUtypeandperipheralsUsermodeemulation,whereQEMUcanrunaprocessthathasbeencompiledonadifferentCPUarchitecturenatively
Inthisbook,wearegoingtofocusonfullsystememulationwiththehardwareaccelerationsupportprovidedbytheKernel-basedVirtualMachine(KVM)hypervisor.
Inthischapter,wewillstartbyinstallingQEMUonLinux,thenexplorevariousexamplesofbuilding,managing,andusingdiskimagesforthevirtualinstances.Wewillthenhaveanin-depthlookatrunningQEMUinfullsystememulationmode,usingtheprovidedbinaries.WewillseeexamplesofusingtheKVMkernelmoduletoacceleratetheQEMUprocesses.Finally,wearegoingtoendthechapterwithdetailsonhowtoconnecttothevirtualmachineswestartedearlier,usingVNCclients.
InstallingandconfiguringQEMUInthisrecipe,wewilllookatinstallingQEMUonasingleserverwiththeprovideddistributionpackages.Forproductionenvironments,werecommendusingprecompiled,packagedversionsofQEMUforeasierandmoreconsistentdeployments.However,wearegoingtoseeanexampleofhowtocompileQEMUfromsource,incaseyouneedacertainversionthatyoumightwanttopackagelater.
GettingreadyDependingonyourLinuxdistribution,thepackagenameandinstallationcommandswilldiffer.Youcanuseyoursystem'spackagemanager,suchasapt,dnf,oryumtosearchforanypackagescontainingtheQEMUstringandgetfamiliarwithwhatisavailableforyourparticularLinuxvariant.ThesourcecodecanbedownloadedfromtheofficialQEMUprojectwebsiteathttp://www.qemu-project.org/download/#source.
Howtodoit...PerformthefollowingstepstoinstallQEMUfrompackagesonUbuntu/DebianandRHEL/CentOSdistributions:
1. OnUbuntu/Debiandistributions,updateyourpackagesindex:
root@kvm:~#apt-getupdate
2. Installthepackage:
root@kvm:~#apt-getinstall-yqemu
3. OnCentOS/RHELdistributionsexecute:
root@kvm:~#yuminstallqemu-kvm
Toinstallfromsource,executethefollowing:
1. Downloadthearchivefirst:
root@kvm:~#cd/usr/src&&wget
http://download.qemu-project.org/qemu-2.8.0.tar.xz
2. Extractthefilesfromthearchive:
root@kvm:/usr/src#tarxvJfqemu-2.8.0.tar.xz&&cdqemu-2.8.0
3. Configureandcompilethesourcecode:
root@kvm:/usr/src/qemu-2.8.0#./configure
root@kvm:/usr/src/qemu-2.8.0#make&&makeinstall
Howitworks...InstallingQEMUisquitetrivial,aswejustsaw.Let'shavealookatwhattheQEMUmetapackageinstalledonUbuntulookslike:
root@kvm:~#dpkg--list|grepqemu
iiipxe-qemu1.0.0+git-20150424.a25a16d-1ubuntu1allPXEbootfirmware-ROM
imagesforqemu
iiqemu1:2.5+dfsg-5ubuntu10.8amd64fastprocessoremulator
iiqemu-block-extra:amd641:2.5+dfsg-5ubuntu10.8amd64extrablockbackend
modulesforqemu-systemandqemu-utils
iiqemu-slof20151103+dfsg-1ubuntu1allSlimlineOpenFirmware--QEMUPowerPC
version
iiqemu-system1:2.5+dfsg-5ubuntu10.8amd64QEMUfullsystememulationbinaries
iiqemu-system-arm1:2.5+dfsg-5ubuntu10.8amd64QEMUfullsystememulation
binaries(arm)
iiqemu-system-common1:2.5+dfsg-5ubuntu10.8amd64QEMUfullsystememulation
binaries(commonfiles)
iiqemu-system-mips1:2.5+dfsg-5ubuntu10.8amd64QEMUfullsystememulation
binaries(mips)
iiqemu-system-misc1:2.5+dfsg-5ubuntu10.8amd64QEMUfullsystememulation
binaries(miscelaneous)
iiqemu-system-ppc1:2.5+dfsg-5ubuntu10.8amd64QEMUfullsystememulation
binaries(ppc)
iiqemu-system-sparc1:2.5+dfsg-5ubuntu10.8amd64QEMUfullsystememulation
binaries(sparc)
iiqemu-system-x861:2.5+dfsg-5ubuntu10.8amd64QEMUfullsystememulation
binaries(x86)
iiqemu-user1:2.5+dfsg-5ubuntu10.8amd64QEMUusermodeemulationbinaries
iiqemu-user-binfmt1:2.5+dfsg-5ubuntu10.8amd64QEMUusermodebinfmt
registrationforqemu-user
iiqemu-utils1:2.5+dfsg-5ubuntu10.8amd64QEMUutilities
root@kvm:~#
Fromtheprecedingoutput,wecanseethattherearefewpackagesinvolved.Ifyouareinterested,youcanreadtheindividualdescriptiontogetmorefamiliarwithwhateachpackageprovides.
It'sworthmentioningthatallbinariesprovidedfromtheearlier-mentionedpackagesstartwiththeprefixQEMU.Youcanusetabcompletiontoseethelistofavailableexecutables:
root@kvm:~#qemu-
qemu-aarch64qemu-ioqemu-mips64elqemu-ppc64qemu-sparc32plusqemu-system-lm32
qemu-system-mipselqemu-system-sh4qemu-system-xtensa
qemu-alphaqemu-m68kqemu-mipselqemu-ppc64abi32qemu-sparc64qemu-system-m68k
qemu-system-moxieqemu-system-sh4ebqemu-system-xtensaeb
qemu-armqemu-make-debian-rootqemu-mipsn32qemu-ppc64leqemu-system-aarch64
qemu-system-microblazeqemu-system-or32qemu-system-sparcqemu-tilegx
qemu-armebqemu-microblazeqemu-mipsn32elqemu-s390xqemu-system-alphaqemu-
system-microblazeelqemu-system-ppcqemu-system-sparc64qemu-unicore32
qemu-crisqemu-microblazeelqemu-nbdqemu-sh4qemu-system-armqemu-system-mips
qemu-system-ppc64qemu-system-tricoreqemu-x86_64
qemu-i386qemu-mipsqemu-or32qemu-sh4ebqemu-system-crisqemu-system-mips64
qemu-system-ppc64leqemu-system-unicore32
qemu-imgqemu-mips64qemu-ppcqemu-sparcqemu-system-i386qemu-system-mips64el
qemu-system-ppcembqemu-system-x86_64
root@kvm:~#
Wecanseethatthere'sasingleexecutableforeachCPUarchitecturetypethatcanbeemulated.
Managingdiskimageswithqemu-imgTorunvirtualmachines,QEMUneedsimagestostorethefilesystemoftheguestOS.Theimageitselfisatypeoffile,anditrepresentstheguestfilesystemresidingonavirtualdisk.QEMUsupportsvariousimagesandprovidestoolstocreateandmanagethem.Inthisrecipe,wearegoingtobuildablankdiskimagewiththeqemu-imgutility.
GettingreadyTousethisrecipe,weneedtohavetheqemu-imgutilityinstalled.Ifyoufollowedthestepsinthefirstrecipe,youshouldhavethatcovered.TocheckwhatimagetypesaresupportedonyourLinuxdistribution,runthefollowingcommand:
root@kvm:~#qemu-img-h|grepSupported
Supportedformats:bochsvvfatrbdvpcparallelstftpftpftpsrawhttpsqcow
dmghttpqcow2quorumnull-aiocloopvdiiscsinull-covhdxblkverifyfilevmdk
host_cdromblkdebughost_devicesheepdogqednbd
root@kvm:~#
Fromtheprecedingoutput,wecanseethattherearemanysupportedimagesonthetestsystemthatweareusing.MakesurethatyourQEMUversionsupportstherawimagetype,asit'sthedefaultandthatiswhatwearegoingtouseinthisrecipe.Oneofthemostcommonlyusedimagetypeisqcow2,whichsupportscopyonwrite,compression,encryption,andsnapshotting.Wearegoingtoleveragethatinlaterrecipes.
PleasenotethateventhoughQEMUsupportsmultipleformats,thatdoesnotnecessarilymeanthatyoucanrunvirtualmachinesonthem.However,qemu-imgcanbeusedtoconvertdifferentimagestorawandqcow2formats.Forbestperformance,useraworqcow2imageformats.
Howtodoit...Performthefollowingstepstocreateablankrawimageofaspecifiedsizeandtoverifythatthefilewascreatedonthehost:
1. Createarawimagenameddebian.imgwithsizeof10GB:
root@kvm:~#qemu-imgcreate-frawdebian.img10G
Formatting'debian.img',fmt=rawsize=10737418240
root@kvm:~#
2. Checkthatthefilewascreated:
root@kvm:~#ls-lahdebian.img
-rw-r--r--1rootroot10GFeb1016:58debian.img
root@kvm:~#
3. Examinethefiletype:
root@kvm:~#file-sdebian.img
debian.img:data
root@kvm:~#
4. Obtainmoreinformationabouttheimage:
root@kvm:~#qemu-imginfodebian.img
image:debian.img
fileformat:raw
virtualsize:10G(10737418240bytes)
disksize:0
root@kvm:~#
Howitworks...Theqemu-imgutilityallowsustocreate,convert,andmodifyguestimages.
Instep1,weusedthe-fflagspecifyingtheimageformat;inthiscase,raw,thenameoftheimagetobecreatedandthesizeingigabytes.
Instep4,weusedtheinfosubcommandtogatheradditionalinformationabouttheexistingimage.Notehowthedisksizeisshowingascurrentlybeingzero.Thisisduetothefactthatthisisablankimage,notcontainingafilesystem.Wearegoingtocreateoneinthenextrecipe.
There'smore...Inthisrecipe,welistedthesupporteddiskimageformatsbyQEMU.Thefollowingisabriefdescriptionofthemostcommontypesthatyoumightencounter:
raw:Rawdiskimageformat.Thisisthedefaultformatandcanbeoneofthefastestfile-basedformats.Ifyouformatthisimagewithafilesystemthatsupportsholes,forexample,EXT3,thenonlysectorsthathavedatawillusespace.Themaindrawbackoftherawimagesisthelackoffeatures,makingthemidealfortestingandquickprototyping.qcow2:Aswementionedintheprevioussection,thisisoneofthemostfeature-richformats.ItsupportsVMsnapshots,compression,andencryptionforthepriceofslightlyreducedperformance.qcow:ThisisanolderQEMUimageformatthatsupportsbackingfiles,compactimagefiles,encryption,andcompression.dmg:ThisistheMacdiskimageformat.TheMacdiskimageprovidessecurepasswordprotectionandcompression,anditismostcommonlyusedtodistributesoftware,ratherthanrunningvirtualmachines.nbd:Thenetworkblockdevice,typicallyusedforaccessingremotestoragedevices.vdi:ThisdiskformatisusedbytheOracleVirtualBoxsoftwareandcanbeusedtorunvirtualmachinesonvariousCPUplatforms.vmdk:ThisistheVMwarediskimagetype,whereasinglevirtualharddiskcanspanmultiplefiles.vhdx:MicrosoftHyper-Vusesthisimageformat.Itprovideslargestoragecapacity,datacorruptionprotectionduringpowerfailuresandread/writeoptimizationforlargerdiskimages.
Inthisbook,wearegoingtousetherawandqcow2diskformats,astheyprovidethebestperformanceandtoolsetforrunningandmanipulatingthem.
PreparingimagesforOSinstallationwithqemu-nbdInthepreviousrecipe,wecreatedablankrawimage.Inthisrecipe,wearegoingtomakeapartitionandafilesystemonit,gettingtheimagereadyforfullguestOSinstallation.Whencreatingthepartitionandfilesystem,youshouldconsiderthetypeofloadthatthevirtualinstancewillcreate.IfyourapplicationsrunninginsideVMareIObound,youmightconsiderXFSfortheimagefilesystem.Forthisrecipe,wearegoingtouseEXT4,asmostLinuxdistributionssupportitoutofthebox.
GettingreadyForthisrecipe,wearegoingtousethefollowingtools:
qemu-nbd
sfdisk
Thenbdkernelmodulemkfs
MostLinuxdistributionsshouldalreadyhavethetoolsinstalled.Ifthat'snotthecase,consultyourdistribution'sdocumentationonhowtoinstallthem.
Howtodoit...Performthefollowingstepsoutlinedtopartitionandcreateafilesystemontheblankimage:
1. Loadthenbdkernelmodule:
root@kvm:~#modprobenbd
root@kvm:~#
2. Usingtheqemu-nbdtool,associatetheblankimagefiletothe/dev/nbd0blockdevice:
root@kvm:~#qemu-nbd--format=raw--connect=/dev/nbd0
debian.img
root@kvm:~#
3. Createtwopartitionsontheblockdevice.Onewillbeusedforswap,andtheotherastherootpartitionfortheguestOS:
root@kvm:~#sfdisk/dev/nbd0<<EOF
>,1024,82
>;
>EOF
Checkingthatno-oneisusingthisdiskrightnow...
OK
Disk/dev/nbd0:cannotgetgeometry
Disk/dev/nbd0:1305cylinders,255heads,63sectors/track
sfdisk:ERROR:sector0doesnothaveanmsdossignature
/dev/nbd0:unrecognizedpartitiontabletype
Oldsituation:
Nopartitionsfound
Newsituation:
Units=cylindersof8225280bytes,blocksof1024bytes,countingfrom
0
DeviceBootStartEnd#cyls#blocksIdSystem
/dev/nbd0p10+10231024-8225279+82Linuxswap/Solaris
/dev/nbd0p2102413042812257132+83Linux
/dev/nbd0p30-000Empty
/dev/nbd0p40-000Empty
Warning:noprimarypartitionismarkedbootable(active)
ThisdoesnotmatterforLILO,buttheDOSMBRwillnotbootthisdisk.
Successfullywrotethenewpartitiontable
Re-readingthepartitiontable...
IfyoucreatedorchangedaDOSpartition,/dev/foo7,say,thenuse
dd(1)
tozerothefirst512bytes:ddif=/dev/zeroof=/dev/foo7bs=512count=1
(Seefdisk(8).)
root@kvm:~#
4. Listtheavailableblockdevicesafterthepartitioning:
root@kvm:~#ls-la/dev/nbd0*
brw-rw----1rootdisk43,0Feb1018:24/dev/nbd0
brw-rw----1rootdisk43,1Feb1018:24/dev/nbd0p1
brw-rw----1rootdisk43,2Feb1018:24/dev/nbd0p2
root@kvm:~#
5. Createtheswappartition:
root@kvm:~#mkswap/dev/nbd0p1
Settingupswapspaceversion1,size=508KiB(520192bytes)
nolabel,UUID=c246fe39-1bc5-4978-967c-806264771d69
root@kvm:~#
6. MaketheEXT4filesystemontherootpartition:
root@kvm:~#mkfs.ext4/dev/nbd0p2
mke2fs1.42.13(17-May-2015)
Discardingdeviceblocks:failed-Input/outputerror
Creatingfilesystemwith26209284kblocksand655360inodes
FilesystemUUID:2ffa23de-579a-45ad-abbc-2a179de67f11
Superblockbackupsstoredonblocks:
32768,98304,163840,229376,294912,819200,884736,1605632
Allocatinggrouptables:done
Writinginodetables:done
Creatingjournal(32768blocks):done
Writingsuperblocksandfilesystemaccountinginformation:done
root@kvm:~#
Howitworks...Wetakeadvantageofthefunctionalitythatthenbdkernelmoduleprovidesbyallowingustoassociatearawimagefiletoablockdeviceusingtheqemu-nbdutility.Togetmoreinformationaboutthekernelmodulerunthefollowingcode:
root@kvm:~#modinfonbd
filename:/lib/modules/4.4.0-62-generic/kernel/drivers/block/nbd.ko
license:GPL
description:NetworkBlockDevice
srcversion:C67096AF2AE3C738DBE0B7E
depends:
intree:Y
vermagic:4.4.0-62-genericSMPmod_unloadmodversions
parm:nbds_max:numberofnetworkblockdevicestoinitialize(default:16)(int)
parm:max_part:numberofpartitionsperdevice(default:0)(int)
root@kvm:~#
Wecanexaminetheblockdevicemetadatacreatedinstep2byrunningthefollowingcommand:
root@kvm:~#file-s/dev/nbd0
/dev/nbd0:x86bootsector
root@kvm:~#
Aftercreatingthetwonewpartitionsinstep3,thetypeoftheimagefilehaschanged.Let'sexamineitagain:
root@kvm:~#file-sdebian.img
debian.img:x86bootsector
root@kvm:~#
Wechosetousethesfdiskutilitytocreatethepartitions,butyoucanusethefdiskutilityinteractivelyinsteadifyouprefer.Theendresultwillbethesame.
Nowthatwehaveanimagefilethatcontainstwopartitionsandafilesystem,wecanproceedwithinstallingtheguestOSinthenextrecipe.
InstallingacustomOSontheimagewithdebootstrapInthisrecipe,wearegoingtousethedebootstraputilitytoinstallaDebiandistributionontherawimagewepreparedintheprevioustworecipes.ThedebootstrapcommandisusedtobootstrapabasicDebiansystemusingaspecificpublicmirror.Bytheendofthisrecipe,weshouldhaveanimagecontaininganentireLinuxdistribution,readyforQEMUexecution.
GettingreadyWearegoingtoneedthefollowinginordertocompletethisrecipe:
TheblockdevicescreatedinthepreviousrecipeThedebootstraputilityThechrootutility
Toensurethattheswapandrootblockdevicesarestillpresentonthesystem,runthefollowing:
root@kvm:~#ls-la/dev/nbd0*
brw-rw----1rootdisk43,0Feb1018:24/dev/nbd0
brw-rw----1rootdisk43,1Feb1018:24/dev/nbd0p1
brw-rw----1rootdisk43,2Feb1018:24/dev/nbd0p2
root@kvm:~#
Ifthat'snotthecase,pleaserefertothePreparingimagesforOSinstallationwithqemu-nbdrecipeonhowtoassociatetherawimagewiththe/deb/nbd0blockdevice.
Toinstallthedebootstraputility,ifnotalreadypresentonyoursystem,executethefollowingcode:
root@kvm:~#aptinstall-ydebootstrap
...
Settingupdebootstrap(1.0.78+nmu1ubuntu1.2)...
root@kvm:~#
Howtodoit...FollowthesestepsoutlinedtoinstallanewDebianLinuxdistributionontherawimage:
1. MounttherootpartitionfromtheNetworkBlockDevice(NBD)deviceandensurethatitwasmountedsuccessfully:
root@kvm:~#mount/dev/nbd0p2/mnt/
root@kvm:~#mount|grepmnt
/dev/nbd0p2on/mnttypeext4(rw)
root@kvm:~#
2. InstallthelateststableDebiandistributionontherootpartitionmountedon/mntfromthespecifiedpublicrepository:
root@kvm:~#debootstrap--arch=amd64--include="openssh-servervim"
stable/mnt/http://httpredir.debian.org/debian/
...
I:Basesysteminstalledsuccessfully.
root@kvm:~#
3. Ensuretherootfilesystemwascreated,bylistingallthefilesatthemountedlocation:
root@kvm:~#ls-lah/mnt/
total100Kdrwxr-xr-x22rootroot4.0KFeb1017:19.
drwxr-xr-x23rootroot4.0KFeb1015:29..
drwxr-xr-x2rootroot4.0KFeb1017:19bin
drwxr-xr-x2rootroot4.0KDec2817:42boot
drwxr-xr-x4rootroot4.0KFeb1017:18dev
drwxr-xr-x55rootroot4.0KFeb1017:19etc
drwxr-xr-x2rootroot4.0KDec2817:42home
drwxr-xr-x12rootroot4.0KFeb1017:19lib
drwxr-xr-x2rootroot4.0KFeb1017:18lib64
drwx------2rootroot16KFeb1017:06lost+found
drwxr-xr-x2rootroot4.0KFeb1017:18media
drwxr-xr-x2rootroot4.0KFeb1017:18mnt
drwxr-xr-x2rootroot4.0KFeb1017:18opt
drwxr-xr-x2rootroot4.0KDec2817:42proc
drwx------2rootroot4.0KFeb1017:18root
drwxr-xr-x4rootroot4.0KFeb1017:19run
drwxr-xr-x2rootroot4.0KFeb1017:19sbin
drwxr-xr-x2rootroot4.0KFeb1017:18srv
drwxr-xr-x2rootroot4.0KApr62015sys
drwxrwxrwt2rootroot4.0KFeb1017:18tmp
drwxr-xr-x10rootroot4.0KFeb1017:18usr
drwxr-xr-x11rootroot4.0KFeb1017:18var
root@kvm:~#
4. Bindandmountthedevicesdirectoryfromthehosttotheimagefilesystem:
root@kvm:~#mount--bind/dev//mnt/dev
root@kvm:~#
5. Ensurethatthenbddevicesarenowpresentinsidethemountlocation:
root@kvm:~#ls-la/mnt/dev/|grepnbd0
brw-rw----1rootdisk43,0Feb1018:24nbd0
brw-rw----1rootdisk43,1Feb1018:26nbd0p1
brw-rw----1rootdisk43,2Feb1018:26nbd0p2
root@kvm:~#
6. Changethedirectorynamespacetobetherootfilesystemoftheimageandensuretheoperationsucceeded:
root@kvm:~#chroot/mnt/
root@kvm:/#pwd
/
root@kvm:/#
7. Checkthedistributionversioninsidethechrootenvironment:
root@kvm:/#cat/etc/debian_version
8.7
root@kvm:/#
8. Mounttheprocandsysfsvirtualfilesystemsinsidethechrootedenvironment:
root@kvm:/#mount-tprocnone/proc
root@kvm:/#mount-tsysfsnone/sys
root@kvm:/#
9. Whilestillinsidethechrootedlocation,installtheDebiankernelmetapackageandthegrub2utilities:
root@kvm:/#apt-getinstall-y--force-yeslinux-image-amd64grub2
IfaskedtoselecttargetdeviceforGRUBtoinstallon,donotselectanyandjustcontinue.
10. InstallGRUBontherootdevice:
root@kvm:/#grub-install/dev/nbd0--force
Installingfori386-pcplatform.
grub-install:warning:thismsdos-stylepartitionlabelhasnopost-MBR
gap;embeddingwon'tbepossible.
grub-install:warning:Embeddingisnotpossible.GRUBcanonlybe
installedinthissetupbyusingblocklists.However,blocklistsare
UNRELIABLEandtheiruseisdiscouraged..
Installationfinished.Noerrorreported.
root@kvm:/#
11. UpdatetheGRUBconfigsandtheinitrdimage:
root@kvm:/#update-grub2
Generatinggrubconfigurationfile...
Foundlinuximage:/boot/vmlinuz-3.16.0-4-amd64
Foundinitrdimage:/boot/initrd.img-3.16.0-4-amd64
done
root@kvm:/#
12. Changetherootpasswordoftheguest:
root@kvm:/#passwd
EnternewUNIXpassword:
RetypenewUNIXpassword:
passwd:passwordupdatedsuccessfully
root@kvm:/#
13. AllowaccesstothepseudoTerminalinsidethenewguestOS:
root@kvm:/#echo"pts/0">>/etc/securetty
root@kvm:/#
14. Changethesystemdrunleveltothemulti-userlevel:
root@kvm:/#systemctlset-defaultmulti-user.target
Createdsymlinkfrom/etc/systemd/system/default.targetto
/lib/systemd/system/multi-user.target.
root@kvm:/#
15. Addtherootmountpointtothefstabfile,soitcanpersistreboots:
root@kvm:/#echo"/dev/sda2/ext4defaults,discard00">/etc/fstab
16. Unmountthefollowingfilesystemsaswearedoneusingthemfornow:
root@kvm:/#umount/proc//sys//dev/
17. Exitthechrootedenvironment:
root@kvm:/#exit
exit
root@kvm:~#
18. InstallGRUBontherootpartitionoftheblockdeviceassociatedwiththerawimage:
root@kvm:~#grub-install/dev/nbd0--root-directory=/mnt--
modules="biosdiskpart_msdos"--force
Installingfori386-pcplatform.
grub-install:warning:thismsdos-stylepartitionlabelhasnopost-MBR
gap;embeddingwon'tbepossible.
grub-install:warning:Embeddingisnotpossible.GRUBcanonlybe
installedinthissetupbyusingblocklists.However,blocklistsare
UNRELIABLEandtheiruseisdiscouraged..
Installationfinished.Noerrorreported.
root@kvm:~#
19. UpdatetheGRUBconfigurationfiletoreflectthecorrectblockdevicefortheguestimage:
root@kvm:~#sed-i's/nbd0p2/sda2/g'/mnt/boot/grub/grub.cfg
root@kvm:~#
20. Unmountthenbd0device:
root@kvm:~#umount/mnt
root@kvm:~#
21. Disassociatethenbd0devicefromtherawimage:
root@kvm:~#qemu-nbd--disconnect/dev/nbd0
/dev/nbd0disconnected
root@kvm:~#
Howitworks...Alothashappenedintheprevioussection,solet'sstepthroughthecommandsandtalkalittlebitmoreaboutwhatexactlywasperformedandwhy.
Instep1,wemountedtherootpartitionwecreatedearlieronthe/dev/nbd0p2deviceto/mnt,sowecanuseit.Oncemounted,instep2,weinstalledanentireDebiandistributiononthatdeviceusingthemount-pointasthetarget.
InordertoinstalltheGRUBbootloaderontherootpartitionoftheimage,webindandmountedthe/devdirectoryfromthehostfilesystemtotheimagefilesystemin/mntinstep4.
Theninstep6,weusedthechroottooltochangeourdirectorynamespacetobe/mnt,sowecanperformoperations,aswearedirectlyonthenewOS.
Instep8,wemountedtheprocandsysfsvirtualfilesystemsinsidetheimagebecausetheGRUBbootloadertoolexpectthem.
Instep9,weproceededtoinstallthekernelsourceandGRUBtoolsinpreparationofinstallingthebootloaderonthebootpartitionandinstep10weinstalledthebootloader.
Instep11,theGRUBconfigurationfilesweregeneratedandthebootramdiskimagewasupdated.
Insteps12,13,and14,wechangedtherootpasswordandensuredwegetaccesstothepseudoTerminal,sowecanlogintotheVMlaterandchangetherun-levelfromthedefaultgraphicalinterfacetothemultiuser.
SincethefstabfileisemptyrightafterinstallingtheDebianOSontheimage,wehavetoaddtherootmountpoint,ortheVMwillnotbeabletostart.Thiswasaccomplishedinstep15.
Insteps16and17,weperformedsomecleaningupbyunmountingthefilesystemswemountedearlierandexitedthechrootenvironment.
Backonthehostfilesysteminstep18,weinstalledGRUBonthenbd0devicebyspecifyingthemountedlocationoftheimage.
Instep19,weupdatedtheGRUBconfigdevicenametobesda2becausethisisthenamethatwillappearinsidethevirtualmachineoncewestartit.Thenbd0p2nameisonlypresentwhilewehavetheassociationbetweentherawimageandthenetworkblockdeviceonthehostOS.FromtheVMperspective,thesecondpartitioninsidetheimagewecreatedbyisnamedsda2bydefault.
Andfinally,insteps20and21,weperformedsomecleaningbyremovingthemountpointanddisassociatingtherawimagefromthenetworkblockdevicenbd0.
ResizinganimageInthisrecipe,wearegoingtoexaminehowtoresizeanexistingrawimage,thepartitionshostedonitandthefilesystemontopofthepartitions.Wearegoingtobeusingtherawimagethatwebuildinthepreviousrecipes,whichcontainsaswapandarootpartitionwithanEXT4filesystemformattedonit.
GettingreadyForthisrecipe,wearegoingtousethefollowingtools:
qemu-img
losetup
tune2fs
e2fsck
kpartx
fdisk
resize2fs
MostoftheutilitiesshouldalreadybeinstalledonUbuntuwiththeexceptionofkpartx.Toinstallit,runthefollowing:
root@kvm:~#aptinstallkpartx
Howtodoit...Thenextstepsdemonstratehowtoaddadditionalspacetotherawimagewecreatedearlier,extendtherootpartition,andresizethefilesystem.Bytheendofthisrecipe,theoriginalrawimagefilesystemsizeshouldhavechangedfrom10Gto20G.
1. Obtainthecurrentsizeoftheimage:
root@kvm:~#qemu-imginfodebian.img
image:debian.img
fileformat:raw
virtualsize:10G(10737418240bytes)
disksize:848M
root@kvm:~#
2. Addadditional10GBtotheimage:
root@kvm:~#qemu-imgresize-frawdebian.img+10GB
Imageresized.
root@kvm:~#
Pleasenotethatnotallimagetypessupportresizing.Inordertoresizesuchanimage,youwillneedtoconvertittorawimagefirstusingtheqemu-imgconvertcommand.
3. Checkthenewsizeoftheimage:
root@kvm:~#qemu-imginfodebian.img
image:debian.img
fileformat:raw
virtualsize:20G(21474836480bytes)
disksize:848M
root@kvm:~#
4. Printthenameofthefirstunusedloopdevice:
root@kvm:~#losetup-f
/dev/loop0
root@kvm:~#
5. Associatethefirstunusedloopdevicewiththerawimagefile:
root@kvm:~#losetup/dev/loop1debian.img
root@kvm:~#
6. Readthepartitioninformationfromtheassociatedloopdeviceandcreatethedevicemappings:
root@kvm:~#kpartx-av/dev/loop1
addmaploop1p1(252:0):01024linear7:12048
addmaploop1p2(252:1):020967424linear7:14096
root@kvm:~#
7. Examinethenewdevicemaps,representingthepartitionsontherawimage:
root@kvm:~#ls-la/dev/mapper
total0
drwxr-xr-x2rootroot100Mar919:10.
drwxr-xr-x20rootroot4760Mar919:10..
crw-------1rootroot10,236Feb1023:25control
lrwxrwxrwx1rootroot7Mar919:10loop1p1
lrwxrwxrwx1rootroot7Mar919:10loop1p2
root@kvm:~#
8. Obtainsomeinformationfromtherootpartitionmapping:
root@kvm:~#tune2fs-l/dev/mapper/loop1p2
tune2fs1.42.13(17-May-2015)
Filesystemvolumename:<none>
Lastmountedon:/
FilesystemUUID:96a73752-489a-435c-8aa0-8c5d1aba3e5f
Filesystemmagicnumber:0xEF53
Filesystemrevision#:1(dynamic)
Filesystemfeatures:has_journalext_attrresize_inodedir_index
filetypeneeds_recoveryextentflex_bgsparse_super
large_filehuge_fileuninit_bgdir_nlinkextra_isizeFilesystem
flags:signed_directory_hash
Defaultmountoptions:user_xattracl
Filesystemstate:clean
Errorsbehavior:Continue
FilesystemOStype:Linux
Inodecount:655360
Blockcount:2620928
Reservedblockcount:131046
Freeblocks:2362078
Freeinodes:634148
Firstblock:0
Blocksize:4096
Fragmentsize:4096
ReservedGDTblocks:639
Blockspergroup:32768
Fragmentspergroup:32768
Inodespergroup:8192
Inodeblockspergroup:512
Flexblockgroupsize:16
Filesystemcreated:FriFeb1023:29:012017
Lastmounttime:ThuMar919:09:252017
Lastwritetime:ThuMar919:08:232017
Mountcount:12
Maximummountcount:-1
Lastchecked:FriFeb1023:29:012017
Checkinterval:0(<none>)
Lifetimewrites:1621MB
Reservedblocksuid:0(userroot)
Reservedblocksgid:0(grouproot)
Firstinode:11
Inodesize:256
Requiredextraisize:28
Desiredextraisize:28
Journalinode:8
Defaultdirectoryhash:half_md4
DirectoryHashSeed:f101cccc-944e-4773-8644-91ebf4bd4f2d
Journalbackup:inodeblocks
root@kvm:~#
9. Checkthefilesystemontherootpartitionofthemappeddevice:
root@kvm:~#e2fsck/dev/mapper/loop1p2
e2fsck1.42.13(17-May-2015)
/dev/mapper/loop1p2:recoveringjournalSettingfreeblockscountto
2362045(was2362078)/dev/mapper/loop1p2:clean,21212/655360files,
258883/2620928blocks
root@kvm:~#
10. Removethejournalfromtherootpartitiondevice:
root@kvm:~#tune2fs-O^has_journal/dev/mapper/loop1p2
tune2fs1.42.13(17-May-2015)
root@kvm:~#
11. Ensurethatthejournalinghasbeenremoved:
root@kvm:~#tune2fs-l/dev/mapper/loop1p2|grep"features"
Filesystemfeatures:ext_attrresize_inodedir_indexfiletypeextent
flex_bgsparse_superlarge_filehuge_fileuninit_bgdir_nlink
extra_isize
root@kvm:~#
12. Removethepartitionmappings:
root@kvm:~#kpartx-dv/dev/loop1
deldevmap:loop1p2
deldevmap:loop1p1
root@kvm:~#
13. Detachtheloopdevicefromtheimage:
root@kvm:~#losetup-d/dev/loop1
root@kvm:~#
14. Associatetherawimagewiththenetworkblockdevice:
root@kvm:~#qemu-nbd--format=raw--connect=/dev/nbd0debian.img
root@kvm:~#
15. Usingfdisk,listtheavailablepartitions,thendeletetherootpartition,recreateit,andwritethechanges:
root@kvm:~#fdisk/dev/nbd0
Command(mforhelp):p
Disk/dev/nbd0:21.5GB,21474836480bytes
255heads,63sectors/track,2610cylinders,total41943040sectors
Units=sectorsof1*512=512bytes
Sectorsize(logical/physical):512bytes/512bytes
I/Osize(minimum/optimal):512bytes/512bytes
Diskidentifier:0x00000000
DeviceBootStartEndBlocksIdSystem
/dev/nbd0p11164505598225279+82Linuxswap/Solaris
/dev/nbd0p216450560209648242257132+83Linux
Command(mforhelp):d
Partitionnumber(1-4):2
Command(mforhelp):n
Partitiontype:
pprimary(1primary,0extended,3free)
eextended
Select(defaultp):p
Partitionnumber(1-4,default2):2
Firstsector(16450560-41943039,default16450560):
Usingdefaultvalue16450560
Lastsector,+sectorsor+size{K,M,G}(16450560-41943039,default
41943039):
Usingdefaultvalue41943039
Command(mforhelp):w
Thepartitiontablehasbeenaltered!
Callingioctl()tore-readpartitiontable.
Syncingdisks.
root@kvm:~#
16. Associatethefirstunusedloopdevicewiththerawimagefile,likewedidinstep5:
root@kvm:~#losetup/dev/loop1debian.img
17. Readthepartitioninformationfromtheassociatedloopdeviceandcreatethedevicemappings:
root@kvm:~#kpartx-av/dev/loop1
addmaploop1p1(252:2):01024linear7:12048
addmaploop1p2(252:3):041938944linear7:14096
root@kvm:~#
18. Afterthepartitioningiscomplete,performafilesystemcheck:
root@kvm:~#e2fsck-f/dev/mapper/loop1p2
e2fsck1.42.13(17-May-2015)
Pass1:Checkinginodes,blocks,andsizes
Pass2:Checkingdirectorystructure
Pass3:Checkingdirectoryconnectivity
Pass4:Checkingreferencecounts
Pass5:Checkinggroupsummaryinformation
/dev/mapper/loop1p2:21212/655360files(0.2%non-contiguous),
226115/2620928blocks
root@kvm:~#
19. Resizethefilesystemontherootpartitionofthemappeddevice:
root@kvm:~#resize2fs/dev/nbd0p2
resize2fs1.42.13(17-May-2015)
Resizingthefilesystemon/dev/mapper/loop1p2to5242368(4k)blocks.
Thefilesystemon/dev/mapper/loop1p2isnow5242368(4k)blockslong.
root@kvm:~#
20. Createthefilesystemjournalbecauseweremoveditearlier:
root@kvm:~#tune2fs-j/dev/mapper/loop1p2
tune2fs1.42.13(17-May-2015)
Creatingjournalinode:done
root@kvm:~#
21. Removethedevicemappings:
root@kvm:~#kpartx-dv/dev/loop1
deldevmap:loop1p2
deldevmap:loop1p1
root@kvm:~#losetup-d/dev/loop1
root@kvm:~#
Howitworks...ResizinganimageforVMcanbesomewhatinvolving,aswesawfromallthestepsintheprevioussection.ThingscangetcomplicatedwhentherearemultipleLinuxpartitionsinsidethesameimage,evenmoresoifwearenotusingLogicalVolumeManagement(LVM).Let'sstepthroughallthecommandsweranearlierandexplaininmoredetailswhyweranthemandwhattheydo.
Instep1,weconfirmedthecurrentsizeoftheimagebeing10GB.
Instep2,weadded10GBattheendoftheimageandconfirmthenewimagesizeinstep3.
Recallthattheimagewebuiltfromearlierrecipescontainstwopartitions,swapandroot.Weneedawaytomanipulatethemindividually.Particularly,wewouldliketoallocatetheextraspaceweaddedinstep2totherootpartition.Todothatweneedtoexposeitasablockdevicethatwecaneasilymanipulatewithstandarddiskandfilesystemutilities.Weaccomplishedthatusingthelosetupcommandinstep5,resultinginamappingbetweentheimageandanewblockdevicenamed/dev/loop1.Instep6,weexposedtheindividualpartitionsastwonewdevicemappings.The/dev/mapper/loop1p2istherootpartitionthatwewouldliketoappendtheunuseddiskspaceto.
Beforewecanresizethepartitionedontheloopdevice,weneedtochecktheintegrityofthefilesystemonit,andthisiswhatwedidinstep9.Becauseweareusingajournalingfilesystem,weneedtoremovethejournalpriortoresizing.Wedothatinstep10andmadesurethatthehas_journalattributeisnotshowingafterrunningthetune2fscommandinstep11.
Now,weneedtoworkdirectlyonthemainblockdeviceandnottheindividualpartitions.Weremovethemappingsinsteps12and13andassociatedanewblockdevicewiththeimagefileusingtheqemu-nbdcommandinstep14.Thenew/dev/nbd0blockdevicenowrepresentstheentirediskoftheguestVMandit'sadirectmappingtowhat'sinsidetherawimage.Wecanusethisblockdevicejustlikeanyotherregulardisk,mostimportantlywecanusetoolssuch
asfdisktoexamineandmanipulatethepartitionsresidingonit.
Instep15,weusethefdiskutilitytodeletetherootpartitionandrecreateit.Thisdoesnotdestroyanyfilesystemdata,butchangesthemetadata,allocatingtheextraspaceweaddedearlieraspartoftherootpartition.
Nowthattheblockdevicehasallthediskspaceallocatedtotherootpartition,weneedtoextendthefilesystemthatisontopofit.Wedothatbyfirstrecreatingtheindividualpartitionmappingslikewedidearlier,toexposetherootpartitiondirectlysothatwecanyetagainmanipulateit.Wedothatinsteps16and17.
Insteps18and19,wechecktheintegrityoftherootfilesystem,thenweresizeittothemaximumavailablediskspaceontherootpartitionthatitresides.
Finally,instep20,weremovethemappingsagain.Nowtheimage,therootpartitioninsidetheimage,andtheEXT4filesystemontopoftheLinuxpartitionhavebeenresizedto20GB.
YoucancheckthenewrootpartitionsizebystartinganewQEMUinstanceusingtheimage.Wearegoingtodojustthatinaseparaterecipeinthischapter.
Usingpre-existingimagesIntheInstallingacustomOSontheimagewithdebootstraprecipe,wesawhowtousethedebootstrapcommandtoinstallDebianonanimagewebuilt.MostLinuxvendorsprovidealreadybuiltimagesoftheirdistributionsforvariousarchitectures.InstallableimagesarealsoavailableformanuallyinstallingtheguestOS.Inthisrecipe,wearegoingtodemonstratehowtoobtainandexamineCentOSandDebianimagesthathavealreadybeenbuilt.Inalaterrecipe,wearegoingtoshowhowtostartQEMU/KVMinstancesusingthosesameimages.
GettingreadyForthisrecipe,wearegoingtoneedQEMUinstalledonthehostOS.ForinstructionsonhowtoinstallQEMU,pleaserefertotheInstallingandconfiguringQEMUrecipefromthischapter.Wearealsogoingtoneedthewgetutilitytodownloadtheimagesfromtheupstreampublicrepositories.
Howtodoit...ToobtainDebianWheezyimagesforusewithQEMUandKVM,performthefollowing:
1. Downloadtheimageusingwget:
root@kvm:~tmp#wget
https://people.debian.org/~aurel32/qemu/amd64/debian_wheezy_amd64_standard.qcow2
--2017-03-0922:07:20--2Resolvingpeople.debian.org
(people.debian.org)...2001:41c8:1000:21::21:30,5.153.231.30
Connectingtopeople.debian.org
(people.debian.org)|2001:41c8:1000:21::21:30|:443...connected.HTTP
requestsent,awaitingresponse...200OKLength:267064832(255M)
Savingto:‘debian_wheezy_amd64_standard.qcow2’
debian_wheezy_amd64_standard.qcow2100%.
[===================================>]254.69M35.8MB/sin8.3s2017-
03-0922:07:29(30.9MB/s)-‘debian_wheezy_amd64_standard.qcow2’saved
[267064832/267064832]
root@kvm:~#
2. Inspectthetypeoftheimage:
root@kvm:~#qemu-imginfodebian_wheezy_amd64_standard.qcow2
image:debian_wheezy_amd64_standard.qcow2
fileformat:qcow2
virtualsize:25G(26843545600bytes)
disksize:261M
cluster_size:65536
Formatspecificinformation:
compat:1.1
lazyrefcounts:false
refcountbits:16
corrupt:false
root@kvm:~#
TodownloadCentOSimagesrunthefollowingcommands:
1. Downloadtheimageusingwget:
root@kvm:/tmp#wget
https://cloud.centos.org/centos/7/images/CentOS-7-x86_64-
GenericCloud.qcow2--2017-03-0922:11:34--
https://cloud.centos.org/centos/7/images/CentOS-7-x86_64-
GenericCloud.qcow2Resolvingcloud.centos.org(cloud.centos.org)...
2604:4500::2a8a,136.243.75.209Connectingtocloud.centos.org
(cloud.centos.org)|2604:4500::2a8a|:443...connected.HTTPrequest
sent,awaitingresponse...200OKLength:1361182720(1.3G)Saving
to:‘CentOS-7-x86_64-GenericCloud.qcow2’
CentOS-7-x86_64-GenericCloud.qcow2100%
[=========================================>]1.27G22.3MB/sin54s
2017-03-0922:12:29(24.0MB/s)-‘CentOS-7-x86_64-GenericCloud.qcow2’
saved[1361182720/1361182720]
FINISHED--2017-03-0922:12:29--Totalwallclocktime:54s
Downloaded:1files,1.3Gin54s(24.0MB/s)
root@kvm:/tmp#
2. Inspectthetypeoftheimage:
root@kvm:~#qemu-imginfoCentOS-7-x86_64-GenericCloud.qcow2
image:CentOS-7-x86_64-GenericCloud.qcow2
fileformat:qcow2
virtualsize:8.0G(8589934592bytes)
disksize:1.3G
cluster_size:65536
Formatspecificinformation:
compat:0.10
refcountbits:16
root@kvm:~#
Howitworks...TherearemanypublicrepositoriesontheInternetthatprovideimagesofvarioustypes,mostcommonlyqcow2forusewithQEMU/KVM.Intheprevioussection,weusedtheofficialCentOSrepositorytoobtaintheimageandananotheronecontainingprebuiltimagesforDebian.
Bothimagesareintheqcow2format,asweconfirmedinstep2.
There'smore...Sofar,we'veonlyseenhowtobuild,examine,manipulate,anddownloadimages.Inthenextrecipe,wearegoingtofocusonhowtoactuallyusetheimagestostartQEMU/KVMinstances.
SeealsoExaminethefollowinglinkstogetmoreinformationaboutwhatprebuiltimagesareavailableforthelisteddistributions:
OfficialUbuntuimages:https://uec-images.ubuntu.com/releases/OfficialCentOSimages:https://cloud.centos.org/centos/OfficialDebianimages:http://cdimage.debian.org/cdimage/openstack/OfficialFedoraimages:https://alt.fedoraproject.org/cloud/OfficialopenSUSEimages:http://download.opensuse.org/repositories/Cloud:/Images:/
Runningvirtualmachineswithqemu-system-*Inthisrecipe,wearegoingtodemonstratehowtostartvirtualmachineswithQEMU.QEMUprovidesbinariesthatcanemulatedifferentCPUarchitecturesusingeithercustomorprebuiltimagesfortheguestOS.
IfyoucompletedtheInstallingandconfiguringQEMUrecipe,youshouldhaveahostthatcontainsthefollowingbinaries:
root@kvm:~#ls-la/usr/bin/qemu-system-*
-rwxr-xr-x1rootroot8868848Jan2512:49/usr/bin/qemu-system-aarch64
-rwxr-xr-x1rootroot7020544Jan2512:49/usr/bin/qemu-system-alpha
-rwxr-xr-x1rootroot8700784Jan2512:49/usr/bin/qemu-system-arm
-rwxr-xr-x1rootroot3671488Jan2512:49/usr/bin/qemu-system-cris
-rwxr-xr-x1rootroot8363680Jan2512:49/usr/bin/qemu-system-i386
-rwxr-xr-x1rootroot3636640Jan2512:49/usr/bin/qemu-system-lm32
-rwxr-xr-x1rootroot6982528Jan2512:49/usr/bin/qemu-system-m68k
-rwxr-xr-x1rootroot3652224Jan2512:49/usr/bin/qemu-system-microblaze
-rwxr-xr-x1rootroot3652224Jan2512:49/usr/bin/qemu-system-microblazeel
-rwxr-xr-x1rootroot8132992Jan2512:49/usr/bin/qemu-system-mips
-rwxr-xr-x1rootroot8356672Jan2512:49/usr/bin/qemu-system-mips64
-rwxr-xr-x1rootroot8374336Jan2512:49/usr/bin/qemu-system-mips64el
-rwxr-xr-x1rootroot8128896Jan2512:49/usr/bin/qemu-system-mipsel
-rwxr-xr-x1rootroot3578592Jan2512:49/usr/bin/qemu-system-moxie
-rwxr-xr-x1rootroot3570848Jan2512:49/usr/bin/qemu-system-or32
-rwxr-xr-x1rootroot8701760Jan2512:49/usr/bin/qemu-system-ppc
-rwxr-xr-x1rootroot9048000Jan2512:49/usr/bin/qemu-system-ppc64
lrwxrwxrwx1rootroot17Jan2512:49/usr/bin/qemu-system-ppc64le->qemu-
system-ppc64
-rwxr-xr-x1rootroot8463680Jan2512:49/usr/bin/qemu-system-ppcemb
-rwxr-xr-x1rootroot6894528Jan2512:49/usr/bin/qemu-system-sh4
-rwxr-xr-x1rootroot6898624Jan2512:49/usr/bin/qemu-system-sh4eb
-rwxr-xr-x1rootroot4032000Jan2512:49/usr/bin/qemu-system-sparc
-rwxr-xr-x1rootroot7201696Jan2512:49/usr/bin/qemu-system-sparc64
-rwxr-xr-x1rootroot3704704Jan2512:49/usr/bin/qemu-system-tricore
-rwxr-xr-x1rootroot3554912Jan2512:49/usr/bin/qemu-system-unicore32
-rwxr-xr-x1rootroot8418656Jan2512:49/usr/bin/qemu-system-x86_64
-rwxr-xr-x1rootroot3653024Jan2512:49/usr/bin/qemu-system-xtensa
-rwxr-xr-x1rootroot3642752Jan2512:49/usr/bin/qemu-system-xtensaeb
root@kvm:~#
EachcommandcanstartaQEMU-emulatedinstanceforthespecificCPUarchitecture.Forthisrecipe,wearegoingtobeusingtheqemu-system-x86_64utility.
GettingreadyTocompletethisrecipe,youwillneedthefollowing:
TheQEMUbinaries,providedafterfollowingtheInstallingandconfiguringQEMUrecipeThecustomrawDebianimagewebuiltintheInstallingacustomOSontheimagewithdebootstraprecipeTheCentOSqcow2imagewedownloadedintheUsingpre-existingimagesrecipe
Let'shavealookatwhatCPUarchitecturesQEMUsupportsonthehostsystem:
root@kvm:~#qemu-system-x86_64--cpuhelp
x86qemu64QEMUVirtualCPUversion2.5+
x86phenomAMDPhenom(tm)9550Quad-CoreProcessor
x86core2duoIntel(R)Core(TM)[email protected]
x86kvm64CommonKVMprocessor
x86qemu32QEMUVirtualCPUversion2.5+
x86kvm32Common32-bitKVMprocessor
x86coreduoGenuineIntel(R)[email protected]
x86486
x86pentium
x86pentium2
x86pentium3
x86athlonQEMUVirtualCPUversion2.5+
x86n270Intel(R)Atom(TM)[email protected]
x86ConroeIntelCeleron_4x0(Conroe/MeromClassCore2)
x86PenrynIntelCore2DuoP9xxx(PenrynClassCore2)
x86NehalemIntelCorei79xx(NehalemClassCorei7)
x86WestmereWestmereE56xx/L56xx/X56xx(Nehalem-C)
x86SandyBridgeIntelXeonE312xx(SandyBridge)
x86IvyBridgeIntelXeonE3-12xxv2(IvyBridge)
x86Haswell-noTSXIntelCoreProcessor(Haswell,noTSX)
x86HaswellIntelCoreProcessor(Haswell)
x86Broadwell-noTSXIntelCoreProcessor(Broadwell,noTSX)
x86BroadwellIntelCoreProcessor(Broadwell)
x86Opteron_G1AMDOpteron240(Gen1ClassOpteron)
x86Opteron_G2AMDOpteron22xx(Gen2ClassOpteron)
x86Opteron_G3AMDOpteron23xx(Gen3ClassOpteron)
x86Opteron_G4AMDOpteron62xxclassCPU
x86Opteron_G5AMDOpteron63xxclassCPU
x86hostKVMprocessorwithallsupportedhostfeatures(onlyavailableinKVM
mode)
RecognizedCPUIDflags:
fpuvmedepsetscmsrpaemcecx8apicsepmtrrpgemcacmovpatpse36pn
clflushdsacpimmxfxsrssesse2sshttmia64pbe
pni|sse3pclmulqdq|pclmuldqdtes64monitords_cplvmxsmxesttm2ssse3cidfma
cx16xtprpdcmpciddcasse4.1|sse4_1sse4.2|sse4_2x2apicmovbepopcnttsc-
deadlineaesxsaveosxsaveavxf16crdrandhypervisor
fsgsbasetsc_adjustbmi1hleavx2smepbmi2ermsinvpcidrtmmpxavx512frdseed
adxsmappcommitclflushoptclwbavx512pfavx512eravx512cd
syscallnx|xdmmxextfxsr_opt|ffxsrpdpe1gbrdtscplm|i643dnowext3dnow
lahf_lmcmp_legacysvmextapiccr8legacyabmsse4amisalignsse3dnowprefetch
osvwibsxopskinitwdtlwpfma4tcenodeid_msrtbmtopoextperfctr_core
perfctr_nb
invtsc
xstorexstore-enxcryptxcrypt-enace2ace2-enphephe-enpmmpmm-en
kvmclockkvm_nopiodelaykvm_mmukvmclockkvm_asyncpfkvm_steal_timekvm_pv_eoi
kvm_pv_unhaltkvmclock-stable-bit
nptlbrvsvm_locknrip_savetsc_scalevmcb_cleanflushbyasiddecodeassists
pause_filterpfthreshold
xsaveoptxsavecxgetbv1xsaves
arat
root@kvm:~#
Fromtheprecedingoutput,wecanseethelistofCPUsthatwecanpassasparameterstothe-cpuflaginordertoemulatethatCPUtypeinsideourvirtualmachine.
Howtodoit...Tostartanewvirtualmachineusingtheqemu-systemutility,performthefollowingsteps:
1. StartanewQEMUvirtualmachineusingthex86_64CPUarchitecture:
root@kvm:~#qemu-system-x86_64-namedebian-vnc146.20.141.254:0-cpu
Nehalem-m1024-driveformat=raw,index=2,file=debian.img-daemonize
root@kvm:~#
2. Ensurethattheinstanceisrunning:
root@kvm:~#pgrep-lfaqemu
3527qemu-system-x86_64-namedebian-vnc146.20.141.254:0-m1024-
driveformat=raw,index=2,file=debian.img-daemonize
root@kvm:~#
3. TerminatetheDebianQEMUinstance:
root@kvm:~#pkillqemu
root@kvm:~#
4. StartanewQEMUinstanceusingtheprebuiltCentOSimage:
root@kvm:~#qemu-system-x86_64-vnc146.20.141.254:0-m1024-hda
CentOS-7-x86_64-GenericCloud.qcow2-daemonize
root@kvm:~#
5. Ensurethattheinstanceisrunning:
root@kvm:~#pgrep-lfaqemu
3546qemu-system-x86_64-vnc146.20.141.254:0-m1024-hdaCentOS-7-
x86_64-GenericCloud.qcow2-daemonize
root@kvm:~#
6. TerminatetheCentOSQEMUinstance:
root@kvm:~#pkillqemu
root@kvm:~#
MakesuretoreplacetheIPaddressofthe-vncparameterwiththeonefromyourhostmachine.
Howitworks...HowtostartavirtualmachinewithQEMU/KVMdependsgreatlyonthetypeofimageandhowthepartitionsarestructuredinsidethatimage.
Weusedtwodifferentimagetypeswithdifferentpartitioningschemestodemonstratethisconcept.
Instep1,weusedtheqemu-system-x86_64commandtoemulateax86_64CPUarchitecture,specificallywepassedthe-cpuNehalemflag,emulatingtheNehalemCPUmodel.WepassedtheIPaddressofourhostasaparametertothe-vncflag.ThisstartsaVNCserverintheVMsothatwecanlateruseaVNCclienttoconnecttotheQEMUinstance.Wespecifiedtheamountofmemorytobeallocatedtotheinstance,inthiscase,1GBwiththe-mflag.WeinstructedQEMUthatwearegoingtousearawimagewiththeformat=rawoptionandthenameandlocationoftheactualimagewiththefile=debian.imgparameter.
Recallthatthisrawimagecontainstwopartitionswiththesecondpartitioncontainingtherootfilesystemwherethebootloaderislocated.ThisisveryimportanttorememberbecauseweneedtospecifyfromwhatpartitionindextheguestOSshouldload.Wedothatwiththeindex=2flag.Finally,wepassthe-daemonizeparametertobackgroundtheQEMUprocess.
Instep4,westartedanotherQEMUinstance,thistimeusingtheqcow2CentOSimagewedownloadedearlier.Wedidnothavetospecifyfromwhatpartitionweneedtobootfromthisthistimebecausemostprebuiltimagesusethefirstpartition,oronlyhaveonepartition.Wealsousedthe-hdaflaginsteadofthe-driveparameter,justtodemonstratethatbothoptionscanbeusedwiththesameresult.The-hdaflagtellsQEMUthefirstdiskfortheinstanceshouldbeloadedfromthefilenamethatfollowsit.
StartingtheQEMUVMwithKVMsupportInthisrecipe,wearegoingtostartaQEMUvirtualmachinewithKVMacceleration.Kernel-basedVirtualMachine(KVM)isafullvirtualizationtechnologyforCPUarchitecturesthatsupportvirtualizationextensions.ForIntel-basedprocessors,thisistheIntelVT,andforAMDCPUS,itistheAMD-Vhardwareextension.ThemainpartsofKVMaretwoloadablekernelmodules,namedkvm.ko,whichprovidesthemainvirtualizationfunctionality,andasecondkernelmodulethatisprocessorspecific,kvm-intel.koandkvm-amd.koforbothmainCPUvendors.
QEMUistheuserspacecomponenttocreatevirtualmachines,whereKVMresidesinkernelspace.IfyoucompletedtheRunningvirtualmachineswithqemu-system-*recipe,youmightnotethatthedifferencebetweenrunningaKVMvirtualmachineandrunninganonacceleratedQEMUinstanceisjustasinglecommand-lineoption.
GettingreadyInordertostartaKVMinstance,youwillneedthefollowing:
TheQEMUbinaries,providedafterfollowingtheInstallingandconfiguringQEMUrecipeThecustomrawDebianimagewebuiltintheInstallingacustomOSontheimagewithdebootstraprecipeProcessorthatsupportsvirtualizationTheKVMkernelmodules
TocheckwhetheryourCPUsupportsvirtualization,runthefollowingcode:
root@kvm:~#cat/proc/cpuinfo|egrep"vmx|svm"|uniq
flags:fpuvmedepsetscmsrpaemcecx8apicsepmtrrpgemcacmovpatpse36
clflushdtsacpimmxfxsrssesse2sshttmpbesyscallnxpdpe1gbrdtscplm
constant_tscarch_perfmonpebsbtsrep_goodnoplxtopologynonstop_tsc
aperfmperfeagerfpupnipclmulqdqdtes64monitords_cplvmxsmxesttm2ssse3
sdbgfmacx16xtprpdcmpciddcasse4_1sse4_2x2apicmovbepopcnt
tsc_deadline_timeraesxsaveavxf16crdrandlahf_lmabmepbtpr_shadowvnmi
flexpriorityeptvpidfsgsbasetsc_adjustbmi1avx2smepbmi2ermsinvpcidcqm
xsaveoptcqm_llccqm_occup_llcdthermaratplnpts
root@kvm:~#
Thepresenceofthevmx(forIntel)orsvm(forAMD)flagsindicatethatyourCPUsupportsthevirtualizationextensions.
Theflagsfromthecpuinfocommandoutputsimplymeanthatyourprocessorsupportsvirtualization;however,makesurethatthisfeatureisenabledintheBIOSofyoursystem;otherwise,theKVMinstancewillfailtostart.
TomanuallyloadtheKVMkernelmoduleandensurethatit'sbeenloaded,runthefollowingcode:
root@kvm:~#modprobekvm
root@kvm:~#lsmod|grepkvm
kvm4558430
root@kvm:~#
Howtodoit...TostartaKVMinstance,ensurethatit'srunningandfinallyterminateit,executethefollowing:
1. StartaQEMUinstancewithKVMsupport:
root@kvm:~#qemu-system-x86_64-namedebian-vnc146.20.141.254:0-m
1024-driveformat=raw,index=2,file=debian.img-enable-kvm-daemonize
root@kvm:~#
2. Ensurethattheinstanceisrunning:
root@kvm:~#pgrep-lfaqemu
4895qemu-system-x86_64-namedebian-vnc146.20.141.254:0-m1024-
driveformat=raw,index=2,file=debian.img-enable-kvm-daemonize
root@kvm:~#
3. Terminatetheinstance:
root@kvm:~#pkillqemu
root@kvm:~#
Howitworks...TostartaQEMU/KVMvirtualmachine,allwehadtododifferentlyfromwhatweperformedintheInstallingandconfiguringQEMUrecipeispassthe-enable-kvmflagtotheqemu-system-x86_64command.
Instep1,wespecifiedanamefortheVMwiththe-nameflag,providedtheIPaddressofourphysicalhosttothe-vncflag,enablingVNCaccessforthevirtualinstance,allocated1GBofmemorywiththe-mflag,specifiedthepartitionwherethebootloaderislocatedwiththeindex=2parameter,theimageformat,andname,andfinallyweenabledKVMhardwareaccelerationwiththe-enable-kvmparameteranddeamonizedtheprocesswiththe-daemonizeflag.
Instep2,weensuredthattheinstanceisrunningandweterminateditinstep3.
There'smore...Asanalternativetodirectlyrunningtheqemu-system-*commands,onUbuntusystemsthere'stheqemu-kvmpackagethatprovidesthe/usr/bin/kvmbinary.Thisfileisawrappertotheqemu-system-x86_64command,anditpassesthe-enable-kvmparametertoitautomatically.
Toinstallthepackageandusethekvmcommandinstead,runthefollowing:
root@kvm:~#aptinstallqemu-kvm
...
root@kvm:~#kvm-namedebian-vnc146.20.141.254:0-cpuNehalem-m1024-drive
format=raw,index=2,file=debian.img-daemonize
root@kvm:~#pgrep-lfaqemu
25343qemu-system-x86_64-enable-kvm-namedebian-vnc146.20.141.254:0-cpu
Nehalem-m1024-driveformat=raw,index=2,file=debian.img-daemonize
root@kvm:~#
YoumighthavenotedthatstartingandstoppingQEMU/KVMinstancesissomewhatofamanualprocess,especiallyhavingtokilltheinstanceprocessinordertostopit.InChapter2,UsinglibvirttoManageKVM,wearegoingtowalkyouthroughasetofrecipesthatwillmakemanagingthelifecycleofKVMvirtualmachinesmucheasier,withtheuserspacetoolsthatthelibvirtpackageprovides.
ConnectingtoarunninginstancewithVNCInthisrecipe,wearegoingtoconnecttoarunningKVMinstanceusingaVNCclient.Onceconnected,wearegoingtologinandchecktheCPUtypeandavailablememoryoftheinstance.We'vealreadyseenhowtostartQEMU/KVMinstanceswithVNCsupportinthepreviousrecipes,butwearegoingtodoitagain,incaseyouarenotreadingthisbookfromcovertocover.
VirtualNetworkComputing(VNC)usestheRemoteFrameBuffer(RFB)protocoltoremotelycontrolanothersystem.Itrelaysthescreenfromtheremotecomputerbacktotheclient,allowingthefullkeyboardandmousecontrol.
TherearemanydifferentVNCclientandserverimplementations,butforthisrecipe,wearegoingtouseafreelyavailableversionnamedchickenoftheVNCformacOS.Youcandownloadtheclientfromhttps://sourceforge.net/projects/cotvnc/.
GettingreadyInordertocompletethisrecipe,youwillneedthefollowing:
TheQEMUbinaries,providedafterfollowingtheInstallingandconfiguringQEMUrecipeThecustomrawDebianimagewebuiltintheInstallingacustomOSontheimagewithdebootstraprecipeAprocessorthatsupportsvirtualizationTheloadedKVMkernelmodulesThechickenoftheVNCclient,installed,asdescribedintheprevioussection
Howtodoit...1. StartanewKVM-acceleratedqemuinstance:
root@kvm:~#qemu-system-x86_64-namedebian-vnc146.20.141.254:0-cpu
Nehalem-m1024-driveformat=raw,index=2,file=debian.img-daemonize
root@kvm:~#
2. Ensurethattheinstanceisrunning:
root@kvm:~#pgrep-lfaqemu
4987qemu-system-x86_64-namedebian-vnc146.20.141.254:0-cpuNehalem
-m1024-driveformat=raw,index=2,file=debian.img-daemonize
root@kvm:~#
3. StarttheVNCclientandconnecttotheVNCserverontheIPaddressanddisplayportyouspecifiedinstep1:
TheVNCloginscreen
4. Logintotheinstanceusingtherootuser,thenchecktheCPUtypeandavailablememoryasshownhere:
VNCsession
Howitworks...Instep1,westartedanewQEMUinstancewithKVMaccelerationandenabledaVNCserveronitwiththespecifiedIPaddressanddisplayport.WespecifiedtheamountofavailablememoryandtheCPUmodelname.
Instep4,weloggedintheinstanceusingtherootuserandthepasswordwecreatedwhenbuildingtheimage,thenobtainedtheCPUinformationbyrunningthelscpucommand.NotehowtheCPUmodelnamematcheswhatwespecifiedwiththe-cpuflagwhenwestartedthevirtualmachine.Next,wecheckedtheallocatedmemorywiththefreecommand,whichalsomatcheswhatwepreviouslyspecifiedwiththe-mparameter.
UsinglibvirttoManageKVMInthischapter,wewillcoverthefollowingtopics:
InstallingandconfiguringlibvirtDefiningKVMinstancesStarting,stopping,andremovingKVMinstancesInspectingandeditingKVMconfigsBuildingnewKVMinstanceswithvirt-installandusingtheconsoleManagingCPUandmemoryresourcesinKVMAttachingblockdevicestovirtualmachinesSharingdirectoriesbetweenarunningVMandthehostOSAutostartingKVMinstancesWorkingwithstoragepoolsManagingvolumesManagingsecrets
IntroductionInthepreviouschapter,wesawexamplesofprovisioningvirtualmachinesusingtheQEMUtoolsetandtheKVMkernelmodules.TheQEMUcommandsareconvenientforquicklystartingvirtualinstances;however,theydon'tprovideaneasywayofconfiguringandadministeringthelifecycleofthevirtualmachines.
Inthischapter,wearegoingtoworkwiththelibvirttoolset.Libivrtprovidesvarioususerspacecommandsandlanguagebindingsinordertobuild,configure,start,stop,migrate,terminate,anddootherfunctionstomanageyourvirtualmachines.Itprovidessupportfordifferentvirtualizationtechnologies,suchasQEMU/KVM,XEN,andcontainerswithLXC.
Wewillstartbyinstallingandconfiguringthelibvirttools,thenmoveontocreatingvirtualmachinesusingtheXMLconfigurationfilesthatlibvirtsupportsandexploremanyofthefunctionalitiesthatthetoolkitprovidesinordertomanagethelifecycleofKVMinstances.Alltherecipesinthischapteraregoingtobeinthecontextofbuildinghighlyavailable,multitenantenvironments.
InstallingandconfiguringlibvirtInthisrecipe,wearegoingtoinstalllibvirtfrompackagesprovidedbytheLinuxdistributionofchoiceandseewhatconfigurationfilesandoptionsareavailableforconfiguringit.Aswithanyotherproduction-readytools,werecommendusingpackagesforyourproductionenvironmentforeaseandconsistencyofdeployment;however,compilingthelatestversionfromthesourceisalsoanoptionifthepackagesfromyourLinuxvendorareolder.
GettingreadyDependingonyourLinuxdistribution,thepackagenameandinstallationcommandswilldiffer.Youcanuseyoursystem'spackagemanager,suchasapt,dnf,oryumtosearchforanypackagescontainingthelibvirtstringandgetfamiliarwithwhatisavailableforyourparticularLinuxvariant.Thesourcecodecanbedownloadedfromtheofficiallibvirtprojectwebsiteathttp://www.qemu-project.org/download/#source.
Howtodoit...Toinstalllibvirtfrompackagesandsourcefollowthefollowingsteps:
1. OnUbuntu,installthepackagebyrunning:
root@kvm:~#aptupdate&&aptinstalllibvirt-bin
root@kvm:~#
2. Ensurethatthelibvirtdaemonisrunningbyexecuting:
root@kvm:~#pgrep-lfalibvirtd
36667/usr/sbin/libvirtd
root@kvm:~#
3. Examinethedefaultconfiguration:
root@kvm:~#cat/etc/libvirt/libvirtd.conf|grep-vi"#"
|sed'/^$/d'
unix_sock_group="libvirtd"
unix_sock_ro_perms="0777"
unix_sock_rw_perms="0770"
auth_unix_ro="none"
auth_unix_rw="none"
root@kvm:~#
4. DisablethesecuritydriverinQEMUbyeditingtheqemuconfigurationfileasfollows:
root@kvm:~#vim/etc/libvirt/qemu.conf
...
security_driver="none"
...
root@kvm:~#
5. Restartthelibvirtdaemon:
root@kvm:~#/etc/init.d/libvirt-binrestart
libvirt-binstop/waiting
libvirt-binstart/running,process1158
root@kvm:~#
DependingonyourLinuxdistribution,thenameofthelibvirtservicemaybedifferent.OnRHEL/CentOS,thenameoftheserviceislibvirtd;torestartit,runservicelibvirtdrestart.
6. Examineallconfigurationfilesinthelibvirtdirectory:
root@kvm:~#ls-la/etc/libvirt/
total76
drwxr-xr-x5rootroot4096Mar2214:27.
drwxr-xr-x90rootroot4096Mar2123:17..
drwxr-xr-x2rootroot4096Feb52016hooks
-rw-r--r--1rootroot518Feb52016libvirt.conf
-rw-r--r--1rootroot13527Feb52016libvirtd.conf
-rw-r--r--1rootroot1176Feb52016lxc.conf
drwxr-xr-x2rootroot4096Mar2123:16nwfilter
drwxr-xr-x3rootroot4096Mar2123:57qemu
-rw-------1rootroot16953Mar2123:18qemu.conf
-rw-r--r--1rootroot2170Feb52016qemu-lockd.conf
-rw-r--r--1rootroot2213Feb52016virtlockd.conf
-rw-r--r--1rootroot1217Feb52016virt-login-shell.conf
root@kvm:~#
Howitworks...Instep1,weinstalledthepackageonUbuntu.Thepostinstallscriptstartedthelibvirtdaemonafterthepackagewassuccessfullyinstalled.Weverifiedthatinstep2.
Instep3,weexaminedthemainconfigurationfilefortheservice-sidedaemon-libvirtd.TheprocessrunsonthehostOSandmanagestasksforthevirtualmachines,suchasconfiguration,lifecyclemanagement,migration,storage,andnetworking,aswearegoingtoseelaterinthischapter.TheuserspacetoolsprovidedbythepackageweinstalledcommunicatewiththedaemonbysendingrequestsonalocalUnixdomainsocket.Thedefaultoptionswesawinstep3aresufficientfortherecipesinthischapter,buttheconfigurationfileisratherlarge.Weencourageyoutogothroughitandgetfamiliarwiththerestoftheavailableconfigurationoptions.Thefileisverywelldocumented.
Instep4,wedisabledthesecuritydriverforQEMU.BydefaultonRHEL/CentOSsystems,QEMUisconfiguredtouseSELinux.UbuntudistributionsuseAppArmor.Forsimplicity,wedisablethatfunctionalityinthisstep;however,inproduction,youshouldtakeadvantageoftheextrasecuritythatamandatoryaccesscontrolsystemsuchasSELinuxprovides.
Anychangetothelibvirtconfigurationfilerequiresarestart.Werestartthelibvirtserviceinstep5.
Therearefewimportantconfigurationfilesthatweneedtobefamiliar,whicharelistedinstep6:
libvirt.confistheclient-sideconfigurationfileforthevirshcommandthatwearegoingtouseinthisrecipe.WecanspecifyURIaliasesinit.Thedefaultsshouldbesufficient.libvirtd.confistheserver-sideconfigurationfile,aswesawinstep3.Itprovidesvarioussecurityoptions,requestlimits,andloggingcontrols.Forthepurposeofthisbook,thedefaultsaresufficient.qemu.confisthemainconfigurationfilefortheQEMUdriverthatlibvirtuses.WecanconfigureoptionssuchastheVNCserveraddress,the
securitydriverthatwesawinstep4andtheuserandgroupfortheQEMUprocess.OncewecreateaQEMU/KVMvirtualmachine,the/etc/libvirt/qemu/directorywillcontaintheXMLconfigurationdefinitionforthatinstance,aswearegoingtoseeinthefollowingrecipes.Finally,the/etc/libvirt/qemu/networks/directorycontainsconfigurationfilesforthenetworking.Wearegoingtoexplorethoseinmoredetaillaterinthischapter.
DefiningKVMinstancesInthisrecipe,wearegoingtodefineavirtualinstancebycreatingasimpleXMLconfigurationfilethatlibvirtcanusetobuildthevirtualmachine.WearegoingtodescribesomeoftheXMLschemablocksandlookatexamplesofhowtogeneratetheXMLdefinitionfileusingthevirt-installcommandratherthanwritingitmanually.
GettingreadyForthisrecipe,wearegoingtoneedthefollowing:
TheQEMUbinaries,providedafterfollowingtheInstallingandconfiguringQEMUrecipefromChapter1,GettingStartedwithQEMUandKVM.ThecustomrawDebianimagewebuiltintheInstallingacustomOSontheimagewithdebootstraprecipefromthepreviouschapter.
YoucanuseyourownvirtualmachineimageordownloadonefromtheInternet,asweshowedintheUsingpre-existingimagesrecipeinChapter1,GettingStartedwithQEMUandKVM.
Howtodoit...TodefineanewKVMvirtualmachine,runthecommandsoutlinedhere:
1. ListallvirtualmachinesonthehostOS:
root@kvm:~#virshlist--all
IdNameState
----------------------------------------------------
root@kvm:~#
2. CreatethefollowingXMLdefinitionfile:
root@kvm:~#catkvm1.xml
<domaintype='kvm'id='1'>
<name>kvm1</name>
<memoryunit='KiB'>1048576</memory>
<vcpuplacement='static'>1</vcpu>
<os>
<typearch='x86_64'machine='pc-i440fx-trusty'>hvm</type>
<bootdev='hd'/>
</os>
<on_poweroff>destroy</on_poweroff>
<on_reboot>restart</on_reboot>
<on_crash>restart</on_crash>
<devices>
<emulator>/usr/bin/qemu-system-x86_64</emulator>
<disktype='file'device='disk'>
<drivername='qemu'type='raw'/>
<sourcefile='/tmp/debian.img'/>
<targetdev='hda'bus='ide'/>
<aliasname='ide0-0-0'/>
<addresstype='drive'controller='0'bus='0'target='0'unit='0'/>
</disk>
<interfacetype='network'>
<sourcenetwork='default'/>
<targetdev='vnet0'/>
<modeltype='rtl8139'/>
<aliasname='net0'/>
<addresstype='pci'domain='0x0000'bus='0x00'slot='0x03'
function='0x0'/>
</interface>
<graphicstype='vnc'port='5900'autoport='yes'
listen='146.20.141.158'>
<listentype='address'address='146.20.141.158'/>
</graphics>
</devices>
<seclabeltype='none'/>
</domain>
root@kvm:~#
3. Definethevirtualmachine:
root@kvm:~#virshdefinekvm1.xml
Domainkvm1definedfromkvm1.xml
root@kvm:~#
4. Listallinstancesinallstates:
root@kvm:~#virshlist--all
IdNameState
----------------------------------------------------
-kvm1shutoff
root@kvm:~#
Howitworks...Instep1,weusedthevirshcommandandsuppliedallargumenttolistallactiveandinactiveinstances.Asexpected,westartedwithnoinstancesdefined.
Instep2,wecreatedadefinitionfileforanewKVMinstance.WeusedasmallsubsectionoftheavailableXMLschemaattributestosetthefollowingoptions:
TherootelementoftheXMLfileisrequiredforallvirtualmachinedefinitionsandisnameddomain.Ithastwoattributes--typeandid.Wespecifiedkvmasthetypeandanidas1becausethisisourfirstKVMvirtualmachine.Allotherattributesaredefinedunderthedomainrootelement.Wespecifiedanamefortheinstancewiththenameattribute.ThememoryattributedefinestheavailablememorytotheVM,inourcase,1GB.ThevcpuelementdefinesthemaximumnumberofvirtualCPUsallocatedfortheguestOS.Wespecified1,andweusedtheoptionalattributeplacementthatindicatestheCPUplacementmode;inthisexample,static.StaticplacementindicatesthatthevirtualinstancewillbepinnedtoalltheavailablephysicalCPUs.TheOSelementdefinesthearchitectureoftheVMwiththeuseofthetypeelement.Thehvmoptionindicatesthatwearegoingtousefullvirtualization,whichisgoingtobeKVM,asspecifiedinthedomaintypeattributewesawearlier.WespecifythebootdevicetheVMwillstartfromwiththe<bootdev>element.Thenextthreeelementsspecifytheactiontobetakenwhentheguestrequestsapoweroff,rebootoritcrashes.Inourexample,theVMwillbedestroyedwhentheguestOSispoweredoffandrestartedwhentheguestrebootsorcrashes.ThelargestsectionoftheXMLdefinitionisthedevicessection,whereweusevariousXMLelementstodescribedevicesprovidedtotheguestOS.Theemulatorelementspecifiesthepathtotheemulatorbinary.WearegoingtousethesameQEMUemulatorbinaryqemu-system-x86_64weusedinChapter1,GettingStartedwithQEMUandKVM.Inthelastfewsectionsofthedevicesattribute,wedefinethetypeofvirtualdiskweareusing,inthis
example,therawimagewebuiltinthepreviouschapter.Inasimilarfashion,wedescribetheVNCserverthattheguestshouldstartandthenetworkinterfaceinsidetheguestOS.
Withtheconfigfileinplace,wedefinedtheinstanceinstep3,usingtheimagewecreatedearlierin/tmp.
Onceanewinstancehasbeendefined,itdoesnotautomaticallystartbydefault.Wecanseethatthestatusofthenewinstanceisshutoffinstep4.
ForinformationonalloftheavailableXMLelementsandtheirattributes,pleaserefertotheofficialdocumentationathttp://libvirt.org/formatdomain.html.
There'smore...ConfiguringavirtualmachinebywritingtheXMLfile,canbequitetediousanderror-prone.AneasierwayofcreatingtheVMfromanexistingimage,orfromaninstallationmedia(whichcanbephysical,virtual,oranetworklocation),isusingthevirt-installtool.LetsseeanexampleofcreatingthesameKVMinstanceusingthattool.
1. Westartbyinstallingthepackage:
root@kvm:~#aptinstallvirtinst
...
root@kvm:~#
2. Next,wedefineandstartthenewinstancebyinvokingthevirt-installcommand(ifaninstancewiththesamenamealreadyexist,you'llneedtodestroyandundefineitfirst):
root@kvm:~#virt-install--namekvm1--ram1024--disk
path=/tmp/debian.img,format=raw--graphicsvnc,listen=146.20.141.158--
noautoconsole--hvm--import
Startinginstall...
Creatingdomain...|0B00:00
Domaincreationcompleted.Youcanrestartyourdomainbyrunning:
virsh--connectqemu:///systemstartkvm1
root@kvm:~#
3. ThenewVMhasnowbeendefinedandstarted.Toconfirm,execute:
root@kvm:~#virshlist--all
IdNameState
----------------------------------------------------
10kvm1running
root@kvm:~#
4. Wecanseethevirtualmachinedefinitionfilethatwasautomaticallygeneratedbyrunningthefollowingcode:
root@kvm:~#cat/etc/libvirt/qemu/kvm1.xml
<!--
WARNING:THISISANAUTO-GENERATEDFILE.CHANGESTOITARELIKELYTOBE
OVERWRITTENANDLOST.Changestothisxmlconfigurationshouldbemade
using:
virsheditkvm1
orotherapplicationusingthelibvirtAPI.
-->
<domaintype='kvm'>
<name>kvm1</name>
<uuid>c3892cbf-812a-2448-7ad2-098ea8381066</uuid>
<memoryunit='KiB'>1048576</memory>
<currentMemoryunit='KiB'>1048576</currentMemory>
<vcpuplacement='static'>1</vcpu>
<os>
<typearch='x86_64'machine='pc-i440fx-trusty'>hvm</type>
<bootdev='hd'/>
</os>
<features>
<acpi/>
<apic/>
<pae/>
</features>
<clockoffset='utc'/>
<on_poweroff>destroy</on_poweroff>
<on_reboot>restart</on_reboot>
<on_crash>restart</on_crash>
<devices>
<emulator>/usr/bin/qemu-system-x86_64</emulator>
<disktype='file'device='disk'>
<drivername='qemu'type='raw'/>
<sourcefile='/tmp/debian.img'/>
<targetdev='hda'bus='ide'/>
<addresstype='drive'controller='0'bus='0'target='0'unit='0'/>
</disk>
<controllertype='usb'index='0'>
<addresstype='pci'domain='0x0000'bus='0x00'slot='0x01'
function='0x2'/>
</controller>
<controllertype='pci'index='0'model='pci-root'/>
<controllertype='ide'index='0'>
<addresstype='pci'domain='0x0000'bus='0x00'slot='0x01'
function='0x1'/>
</controller>
<interfacetype='network'>
<macaddress='52:54:00:59:e3:4e'/>
<sourcenetwork='default'/>
<modeltype='rtl8139'/>
<addresstype='pci'domain='0x0000'bus='0x00'slot='0x03'
function='0x0'/>
</interface>
<serialtype='pty'>
<targetport='0'/>
</serial>
<consoletype='pty'>
<targettype='serial'port='0'/>
</console>
<inputtype='mouse'bus='ps2'/>
<inputtype='keyboard'bus='ps2'/>
<graphicstype='vnc'port='-1'autoport='yes'
listen='146.20.141.158'>
<listentype='address'address='146.20.141.158'/>
</graphics>
<video>
<modeltype='cirrus'vram='9216'heads='1'/>
<addresstype='pci'domain='0x0000'bus='0x00'slot='0x02'
function='0x0'/>
</video>
<memballoonmodel='virtio'>
<addresstype='pci'domain='0x0000'bus='0x00'slot='0x04'
function='0x0'/>
</memballoon>
</devices>
</domain>
root@kvm:~#
Starting,stopping,andremovingKVMinstancesInthepreviousrecipe,wesawhowtodefinenewKVMvirtualmachinebyeithermanuallywritingtheXMLdefinitionfileorusingthevirt-installtooltodefinetheinstanceforus.
IfyoudefineanewinstancefromanXMLfile,bydefaulttheinstancewillnotstartautomatically.Inthisrecipe,wewillseehowtostartaninstancethatwaspreviouslyconfigured.
GettingreadyForthisrecipe,wearegoingtoneedthefollowing:
TheQEMUbinaries,providedafterfollowingtheInstallingandconfiguringQEMUrecipefromChapter1,GettingStartedwithQEMUandKVM.ThecustomrawDebianimagewebuiltintheInstallingacustomOSontheimagewithdebootstraprecipefromthepreviouschapter.ThevirshtoolprovidedbycompletingtheInstallingandconfiguringlibvirtrecipe.ThedefinedinstancefromtheDefiningKVMinstancesrecipeinashutoffstate.
Howtodoit...Thefollowingstepsoutlinetheprocessoflisting,starting,andstoppingKVMinstancesusingthevirshcommand:
1. Listallinstancesinallstates:
root@kvm:~#virshlist--all
IdNameState
----------------------------------------------------
-kvm1shutoff
root@kvm:~#
2. Startthenewlydefinedinstanceandverifyitsstatus:
root@kvm:~#virshstartkvm1
Domainkvm1started
root@kvm:~#
root@kvm:~#virshlist--all
IdNameState
----------------------------------------------------
1kvm1running
root@kvm:~#
3. Examinetherunningprocessforthevirtualmachine:
root@kvm:~#pgrep-lfaqemu
1686/usr/bin/qemu-system-x86_64-namekvm1-S-machinepc-i440fx-
trusty,accel=kvm,usb=off-m1024-realtimemlock=off-smp
1,sockets=1,cores=1,threads=1-uuida9dfd1a1-7dd1-098e-a926-db9526785a9e
-no-user-config-nodefaults-chardev
socket,id=charmonitor,path=/var/lib/libvirt/qemu/kvm1.monitor,server,nowait
-monchardev=charmonitor,id=monitor,mode=control-rtcbase=utc-no-
shutdown-bootstrict=on-devicepiix3-usb-
uhci,id=usb,bus=pci.0,addr=0x1.0x2-drive
file=/tmp/debian.img,if=none,id=drive-ide0-0-0,format=raw-deviceide-
hd,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1-netdev
tap,fd=24,id=hostnet0-device
rtl8139,netdev=hostnet0,id=net0,mac=52:54:00:ce:dd:f2,bus=pci.0,addr=0x3
-chardevpty,id=charserial0-deviceisa-
serial,chardev=charserial0,id=serial0-vnc146.20.141.158:0-device
cirrus-vga,id=video0,bus=pci.0,addr=0x2-devicevirtio-balloon-
pci,id=balloon0,bus=pci.0,addr=0x4
root@kvm:~#
4. TerminatetheVMandensureitsstatuschangedfromrunningtoshutoff:
root@kvm:~#virshdestroykvm1
Domainkvm1destroyed
root@kvm:~#virshlist--all
IdNameState
----------------------------------------------------
-kvm1shutoff
root@kvm:~#
5. Removetheinstancedefinition:
root@kvm:~#virshundefinekvm1
Domainkvm1hasbeenundefined
root@kvm:~#virshlist--all
IdNameState
----------------------------------------------------
root@kvm:~#
Howitworks...Instep1,welistalldefinedinstances,regardlessoftheirstate.Fromtheoutput,wecanseethatwecurrentlyhaveoneinstancethatwedefinedintheearlierrecipe.
Instep2,westartedthevirtualmachineandensureditsstatushadchangedtorunning.
IfyoucompletedtheRunningVirtualMachineswithqemu-system-*recipefromChapter1,GettingStartedwithQEMUandKVM,youmightnotethattheXMLdefinitionforthisVMisverysimilartoallthecommand-lineoptionsweusedtostarttheQEMUinstance.Wecanseethesimilaritiesofhowthenewinstancewasstartedinstep3.ThemaindifferenceisthelargernumberofparametersthatlibvirtpassedtotheQEMUexecutable.
Finally,insteps4and5,westoppedtheVMandremoveditsdefinitionfile.TherawimageweusedfortheVMisstillavailablehoweverandcanbeusedagain.
InspectingandeditingKVMconfigsInthisrecipe,wearegoingtousethevirshtooltoinspectandedittheconfigurationforanexistingvirtualmachine.Aswesawearlier,oncewedefineandstartaKVMinstance,libvirtcreatestheXMLdefinitionfileinthe/etc/libvirt/qemu/directory.Wecandumptheguestconfigurationtodisk,forinspection,ortobackitup.Withthevirshcommandwecanalsoperformupdatestotheconfigurationinplace,aswewillseelaterinthisrecipe.
GettingreadyForthisrecipe,wearegoingtoneedthefollowing:
TheQEMUbinaries,providedafterfollowingtheInstallingandconfiguringQEMUrecipefromChapter1,GettingStartedwithQEMUandKVMThecustomrawDebianimagewebuiltintheInstallingcustomOSontheimagewithdebootstraprecipefromthepreviouschapter,oranyothervirtualmachineimage,ineitherraworqcow2formatThevirshtoolprovidedbycompletingtheInstallingandconfiguringlibvirtrecipeArunninglibvirtKVMinstance
Howtodoit...ThefollowingstepsoutlinetheprocessofinspectingandeditingtheXMLdefinitionofaKVMinstance:
1. EnsurethatyouhavearunningKVMinstancewithlibvirt,ifnot,followthestepsinthepreviousrecipe:
root@kvm:~#virshlist
IdNameState
----------------------------------------------------
11kvm1running
root@kvm:~#
2. Dumptheinstanceconfigurationfiletostandardoutput(stdout).Formoreinformationonstdoutrefertofolllowinglink:
https://en.wikipedia.org/wiki/Standard_streams
root@kvm:~#virshdumpxmlkvm1
<domaintype='kvm'id='11'>
<name>kvm1</name>
<uuid>9eb9a2e9-abb2-54c5-5cb3-dc86728e70fc</uuid>
<memoryunit='KiB'>1048576</memory>
<currentMemoryunit='KiB'>1048576</currentMemory>
<vcpuplacement='static'>1</vcpu>
<resource>
<partition>/machine</partition>
</resource>
<os>
<typearch='x86_64'machine='pc-i440fx-trusty'>hvm</type>
<bootdev='hd'/>
</os>
<features>
<acpi/>
<apic/>
<pae/>
</features>
<clockoffset='utc'/>
<on_poweroff>destroy</on_poweroff>
<on_reboot>restart</on_reboot>
<on_crash>restart</on_crash>
<devices>
<emulator>/usr/bin/qemu-system-x86_64</emulator>
<disktype='file'device='disk'>
<drivername='qemu'type='raw'/>
<sourcefile='/tmp/debian.img'/>
<targetdev='hda'bus='ide'/>
<aliasname='ide0-0-0'/>
<addresstype='drive'controller='0'bus='0'target='0'unit='0'/>
</disk>
<controllertype='usb'index='0'>
<aliasname='usb0'/>
<addresstype='pci'domain='0x0000'bus='0x00'slot='0x01'
function='0x2'/>
</controller>
<controllertype='pci'index='0'model='pci-root'>
<aliasname='pci.0'/>
</controller>
<controllertype='ide'index='0'>
<aliasname='ide0'/>
<addresstype='pci'domain='0x0000'bus='0x00'slot='0x01'
function='0x1'/>
</controller>
<interfacetype='network'>
<macaddress='52:54:00:d1:70:df'/>
<sourcenetwork='default'/>
<targetdev='vnet0'/>
<modeltype='rtl8139'/>
<aliasname='net0'/>
<addresstype='pci'domain='0x0000'bus='0x00'slot='0x03'
function='0x0'/>
</interface>
<serialtype='pty'>
<sourcepath='/dev/pts/0'/>
<targetport='0'/>
<aliasname='serial0'/>
</serial>
<consoletype='pty'tty='/dev/pts/0'>
<sourcepath='/dev/pts/0'/>
<targettype='serial'port='0'/>
<aliasname='serial0'/>
</console>
<inputtype='mouse'bus='ps2'/>
<inputtype='keyboard'bus='ps2'/>
<graphicstype='vnc'port='5900'autoport='yes'
listen='146.20.141.158'>
<listentype='address'address='146.20.141.158'/>
</graphics>
<video>
<modeltype='cirrus'vram='9216'heads='1'/>
<aliasname='video0'/>
<addresstype='pci'domain='0x0000'bus='0x00'slot='0x02'
function='0x0'/>
</video>
<memballoonmodel='virtio'>
<aliasname='balloon0'/>
<addresstype='pci'domain='0x0000'bus='0x00'slot='0x04'
function='0x0'/>
</memballoon>
</devices>
<seclabeltype='none'/>
</domain>
root@kvm:~#
3. Savetheconfigurationtoanewfile,asfollows:
root@kvm:~#virshdumpxmlkvm1>kvm1.xml
root@kvm:~#headkvm1.xml
<domaintype='kvm'id='11'>
<name>kvm1</name>
<uuid>9eb9a2e9-abb2-54c5-5cb3-dc86728e70fc</uuid>
<memoryunit='KiB'>1048576</memory>
<currentMemoryunit='KiB'>1048576</currentMemory>
<vcpuplacement='static'>1</vcpu>
<resource>
<partition>/machine</partition>
</resource>
<os>
root@kvm:~#
4. EdittheconfigurationinplaceandchangetheavailablememoryfortheVM:
root@kvm:~#virsheditkvm1
Domainkvm1XMLconfigurationedited.
root@kvm:~#
Howitworks...Libvirtprovidestwomainwaystomanipulatetheconfigurationdefinitionsofthevirtualinstances.Wecaneitherdumptheconfigfromanexistinginstance,aswedidinsteps2and3,oredittheXMLdefinitioninplace,aswedidinstep4.
SavingthecurrentconfigurationtoafileisaconvenientwaytobackuptheVMdefinition.ItalsoprovidesawayofdefininganewinstancebyeditingthesavedfileandjustchangingthenameandIDofthevirtualmachine.WecanthenusethatfiletostartanewVMonthesame,oradifferenthost,assumingthatthefilesystemorimageisalsoavailable.Wearegoingtoseeexamplesofmigratingandbackingupvirtualmachineswithlibvirtinlaterrecipes.
Whenmakingchangesinplace,asshowninstep4,thedefaultsystem$EDITORwillbeused.Onceintheeditingmode,notethattheXMLfilecontainsinformationaboutthecurrentstateofthevirtualinstance.The<uuid>and<currentMemory>attributesaresuchexamples.IfyouwouldliketochangetheavailablememoryfortheVM,afterupdatingthe<memory>attribute,youmightneedtodeletethe<currentMemory>stanza.Ifthereareanyissueswiththeedit,libvirtwillcomplainwithanerrormessageandpresentthefollowingoptions:
root@kvm:~#virsheditkvm1
error:XMLerror:currentmemory'1048576k'exceedsmaximum'524288k'
Failed.Tryagain?[y,n,f,?]:n
Domainkvm1XMLconfigurationnotchanged.
root@kvm:~#
Alsokeepinmindthat,ifyouwouldliketocreateanewinstancefromthedumpofanexistingone,youwillneedtochangethe<name>anddeletethe<uuid>attributes,asthelatterwillbeautogeneratedoncethenewinstancehasbeendefined.
BuildingnewKVMinstanceswithvirt-installandusingtheconsoleIntheConnectingtotherunninginstancewithVNCrecipefromChapter1,GettingStartedwithQEMUandKVM,youlearnedhowtoconnecttoaQEMU/KVMvirtualmachinethatwasrunningaVNCserver.Thisisagreatwaytoconnecttoaninstancethatisbeinginstalledorintheprocessofbootinginordertointeractwithit.
Sofar,we'veusedthecustomrawimagethatwecreatedearlier,whichcontainsaninstallationofDebian.RecallfromChapter1,GettingStartedwithQEMUandKVM,thatweusedthedebootstrapcommandtoinstalltheOSinsidetheimagefile.Inthisrecipe,wearegoingtousethevirt-installtooltoinstallanewLinuxdistribution,usingtheprovidedupstreamInternetrepository,asthesourceoftheinstallationandthenusethevirshcommandtoattachtotherunninginstance,usingtheconsole.
GettingreadyForthisrecipe,wearegoingtoneedthefollowing:
ThevirshcommandThevirt-installcommandInternetconnectivityinordertodownloadtheinstallationfiles
Howtodoit...TobuildanewKVMinstanceandconnecttoitusingtheconsole,performthefollowingsteps:
1. InstallanewKVMvirtualmachineusingtheofficialDebianrepository:
root@kvm:~#virt-install--namekvm1--ram1024--extra-args="text
console=tty0utf8console=ttyS0,115200"--graphics
vnc,listen=146.20.141.158--hvm--
location=http://ftp.us.debian.org/debian/dists/stable/main/installer-
amd64/--diskpath=/tmp/kvm1.img,size=8
RetrievingfileMANIFEST...|3.3kB00:00...
Retrievingfilelinux...|6.0MB00:00...
Retrievingfileinitrd.gz...|29MB00:00...
Creatingstoragefilekvm1.img|8.0GB00:00
WARNINGUnabletoconnecttographicalconsole:virt-viewernot
installed.Pleaseinstallthe'virt-viewer'package.
Domaininstallationstillinprogress.Youcanreconnectto
theconsoletocompletetheinstallationprocess.
root@kvm:~#
2. Attachtotheconsoletocompletetheinstallationbyrunningthefollowingcode:
root@kvm:~#virshconsolekvm1
Connectedtodomainkvm1
Escapecharacteris^]
3. Onceconnectedtotheconsole,youshouldbepresentedwithascreensimilartotheonehere:
Theconsoleoutputonceconnectedwiththevirshconsolecommand
4. Completetheinstallationbyfollowingthetextmenuprompts.5. StartthenewlyprovisionedVM:
root@kvm:~#virshstartkvm1
Domainkvm1started
root@kvm:~#
6. UsingyourfavoriteVNCclient,connecttotheinstance,loginwiththeusernameandpasswordyoucreatedduringtheinstallationprocessinstep3andenabletheserialconsoleaccessbyrunningthefollowingcommand:
root@debian:~#[email protected]
root@debian:~#[email protected]
root@debian:~#
7. ClosetheVNCsessionandconnecttothevirtualinstancefromthehostOS,usingvirsh:
root@kvm:~#virshconsolekvm1
Connectedtodomainkvm1
Escapecharacteris^]
DebianGNU/Linux8debianttyS0
debianlogin:root
Password:
Lastlogin:WedMar2216:38:10CDT2017ontty1
Linuxdebian3.16.0-4-amd64#1SMPDebian3.16.39-1+deb8u2(2017-03-07)
x86_64
TheprogramsincludedwiththeDebianGNU/Linuxsystemarefree
software;
theexactdistributiontermsforeachprogramaredescribedinthe
individualfilesin/usr/share/doc/*/copyright.
DebianGNU/LinuxcomeswithABSOLUTELYNOWARRANTY,totheextent
permittedbyapplicablelaw.
root@debian:~#free-m
totalusedfreesharedbufferscached
Mem:1000989024943
-/+buffers/cache:44956
Swap:3820382
root@debian:~#
8. DisconnectfromtheconsoleusingtheCtrl+]keycombination.9. Examinetheimagefilecreatedaftertheinstallation:
root@kvm:~#qemu-imginfo/tmp/kvm1.img
image:/tmp/kvm1.img
fileformat:raw
virtualsize:8.0G(8589934592bytes)
disksize:1.9G
root@kvm:~#
Ifyouarenotusingsystemd-basedinitsystemonthedistributionfortheKVMmachine,inordertoallowaccesstotheserialconsoleoftheinstance,youwillneedtoeditthe/etc/securettyorthe/etc/inittabfiles.
Howitworks...Alothappenedinthisrecipe,soletsgothroughallthestepsinmoredetail.
Instep1,westartedtheinstallationprocessforanewKVMinstanceusingthevirt-installutility.Wespecifiedtheserialconsoletobeenabledduringtheinstallationprocesswiththe--extra-argsparameter.Wealsousedthe--locationflagtotelllibvirtthelocationoftheinstallationfilesforthelatestDebiandistribution.WethenspecifiedthelocationandsizeoftheimagefilethatwillcontaintheguestOSfilesystem.Sincethisfiledidnotexist,virt-installcreateditasarawimage,asshowninstep9.
Withconsoleaccessenabledfortheinstallation,wewereabletoconnecttotheconsoleinstep2andcompletetheinstallationprocessinsteps3and4.
Aftertheinstallationcompleted,theconsolesessionwasterminatedandthenewKVMinstancereadytobestarted.Westartedtheinstanceinstep5.
Inordertoenableconsoleaccessontheserialport,wefirstconnectedtotherunningVMusingaVNCclientandinstructsystemdtostartedtheconsoleserviceinstep6.
Withconsoleaccessenabled,wewereabletoconnecttotheserialconsoleusingthevirshtoolinstep7.
Withallthiscompleted,wenowhavetwowaysofconnectingtoarunningKVMinstanceusingeitherVNCortheconsole.
Inthelaterrecipe,wewillenablenetworkingintheguestOSandprovideathirdwaytoconnectusingSSH.
ManagingCPUandmemoryresourcesinKVMChangingtheamountofallocatedmemoryorthenumberofCPUscanbedoneeitherbyeditingtheXMLdefinitionfortheVMorusingthelibvirttoolset.Inthisrecipe,wearegoingtolookatexamplesofchangingboththememoryandtheCPUcountforaKVMinstance.
GettingreadyForthisrecipe,wearegoingtoneedthefollowing:
ArunningKVMinstancewith1GBofmemory,1CPUallocated,andconsoleaccessThelibvirtpackageAguestOSwithatleast4GBofavailablememoryandminimumof4CPUs
Howtodoit...ToinspectandupdatethememoryandCPUresourcesassignedtoavirtualmachinefollowtheprocessoutlinedhere:
1. Getmemorystatisticsfortherunninginstance:
root@kvm:~#virshdommemstatkvm1
actual1048576
swap_in0
rss333644
root@kvm:~#
2. UpdatetheavailablememoryfortheVMto2GB:
root@kvm:~#virshsetmemkvm1--size1049000
root@kvm:~#
3. Stoptherunninginstance:
root@kvm:~#virshdestroykvm1
Domainkvm1destroyed
root@kvm:~#
4. Setthemaximumusablememoryto2GB:
root@kvm:~#virshsetmaxmemkvm1--size2097152
root@kvm:~#
5. Starttheinstance:
root@kvm:~#virshstartkvm1
Domainkvm1started
root@kvm:~#
6. Checkthecurrentallocatedmemory:
root@kvm:~#virshdommemstatkvm1
actual2097152
swap_in0
rss214408
root@kvm:~#
7. ConnecttotheKVMinstanceandcheckthememoryintheguestOS:
root@kvm:~#virshconsolekvm1
Connectedtodomainkvm1
Escapecharacteris^]
DebianGNU/Linux8debianttyS0
debianlogin:root
Password:
...
root@debian:~#free-m
totalusedfreesharedbufferscached
Mem:20109319175840
-/+buffers/cache:431966
Swap:3820382
root@debian:~#
root@kvm:~#
8. CheckthememorysettingsintheinstanceXMLdefinition:
root@kvm:~#virshdumpxmlkvm1|grepmemory
<memoryunit='KiB'>2097152</memory>
root@kvm:~#
9. GetinformationabouttheguestCPUs:
root@kvm:~#virshvcpuinfokvm1
VCPU:0
CPU:29
State:running
CPUtime:9.7s
CPUAffinity:yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy
root@kvm:~#
10. ListthenumberofvirtualCPUsusedbytheguestOS:
root@kvm:~#virshvcpucountkvm1
maximumconfig1
maximumlive1
currentconfig1
currentlive1
root@kvm:~#
11. ChangethenumberofallocatedCPUsto4fortheVM:
root@kvm:~#virsheditkvm1
...
<vcpuplacement='static'>4</vcpu>
...
Domainkvm1XMLconfigurationedited.
root@kvm:~#
12. EnsurethattheCPUcountupdatetookeffect:
root@kvm:~#virshvcpucountkvm1
maximumconfig4
maximumlive4
currentconfig4
currentlive4
root@kvm:~#virshdumpxmlkvm1|grep-icpu
<vcpuplacement='static'>4</vcpu>
root@kvm:~#
Howitworks...Instep1,wegatheredsomememorystatisticsfortherunningKVMinstance.Fromtheoutput,wecanseethattheVMisconfiguredwith1GBofmemoryindicatedbytheactualparameter,andit'scurrentlyusing333644KBofmemory.
Instep2,weupdatedtheavailablememoryto2GBandthenproceededtoupdatethemaximummemorythatcanbeallocatedtotheinstanceinstep4.Inordertoperformthatoperation,theinstancehadtobestoppedfirst,asshowninstep3.
Insteps6,7,and8,wemadesurethattheupdatestookplacebyfirstinvokingthedommemstatsubcommand,thenconnectedtotheVMsconsoleandfinallycheckedthecurrentconfigurationbydumpingtheinstancedefinition.
ThevirshcommandprovidesfewsubcommandstoinspecttheCPUstateforarunningVM.Insteps9and10,welistedtheallocatedvirtualCPUsforthekvm1instance,inthiscase,justoneandthecurrentstate,load,andaffinity.
Finally,insteps11and12,weupdatetheXMLdefinitionoftheinstance,allocatingfourCPUsandlistedthenewcount.
There'smore...Inthisrecipe,weusedthevirshcommandwithvarioussubcommandsinoneliners.Thisisparticularlyusefulifweneedtorunthecommandsfromascript.Thevirshcommandalsoprovidesaninteractiveterminal,whichsavessometyping,andprovidescontextualhelp.Tostartthevirtualization-interactiveterminal,runthefollowingcode:
root@kvm:~#virsh
Welcometovirsh,thevirtualizationinteractiveterminal.
Type:'help'forhelpwithcommands
'quit'toquit
virsh#
Typinghelpwilllistallavailablesubcommandswithashortdescription.Toobtainmoreinformationforaparticularsubcommandtype:
virsh#helpvcpucount
NAME
vcpucount-domainvcpucounts
SYNOPSIS
vcpucount<domain>[--maximum][--active][--live][--config][--current][--
guest]
DESCRIPTION
ReturnsthenumberofvirtualCPUsusedbythedomain.
OPTIONS
[--domain]<string>domainname,idoruuid
--maximumgetmaximumcountofvcpus
--activegetnumberofcurrentlyactivevcpus
--livegetvaluefromrunningdomain
--configgetvaluetobeusedonnextboot
--currentgetvalueaccordingtocurrentdomainstate
--guestretrievevcpucountfromtheguestinsteadofthehypervisor
virsh#
Allthestepsweperformedinthisrecipecanbedoneintheinteractiveterminal.
AttachingblockdevicestovirtualmachinesInthisrecipe,wearegoingtoexamineafewdifferentwaysofaddingnewblockdevicestoaKVMinstance.Thenewblockdevicecanthenbepartitioned,formatted,andusedasaregularblockdeviceinsidetheguestOS.Wecanadddiskstoliverunninginstances,orwecanattachthempersistentlybycreatingXMLdefinitionsfortheindividualblockdevicesoffline.FromthehostOS,wecanpresentanytypeofblockdevicefiletotheguest,includingiSCSItargets,LVMlogicalvolumes,orimagefiles.
GettingreadyForthisrecipe,wewillneed:
ArunningKVMinstancewithconsoleaccessTheddutility
Howtodoit...ToattachanewblockdevicetoaKVMguest,runthefollowing:
1. Createanew1GBimagefile:
root@kvm:~#ddif=/dev/zeroof=/tmp/new_disk.imgbs=1Mcount=1024
1024+0recordsin
1024+0recordsout
1073741824bytes(1.1GB)copied,0.670831s,1.6GB/s
root@kvm:~#
2. AttachthefileasanewdisktotheKVMinstance:
root@kvm:~#virshattach-diskkvm1/tmp/new_disk.imgvda--live
Diskattachedsuccessfully
root@kvm:~#
3. ConnecttotheKVMinstanceviatheconsole:
root@kvm:~#virshconsolekvm1
Connectedtodomainkvm1
Escapecharacteris^]
DebianGNU/Linux8debianttyS0
debianlogin:root
Password:
...
root@debian:~#
4. Printthekernelringbufferandcheckforthenewblockdevice:
root@debian:~#dmesg|grepvda
[3664.134978]sd2:0:2:0:[vda]2097152512-bytelogicalblocks:(1.07
GB/1.00GiB)
[3664.135248]sd2:0:2:0:[vda]WriteProtectisoff
[3664.135251]sd2:0:2:0:[vda]ModeSense:63000008
[3664.135340]sd2:0:2:0:[vda]Writecache:enabled,readcache:
enabled,doesn'tsupportDPOorFUA
[3664.138254]vda:unknownpartitiontable
[3664.139008]sd2:0:2:0:[vda]AttachedSCSIdisk
root@debian:~#
5. Examinethenewblockdevice:
root@debian:~#fdisk-l/dev/vda
Disk/dev/vda:1GiB,1073741824bytes,2097152sectors
Units:sectorsof1*512=512bytes
Sectorsize(logical/physical):512bytes/512bytes
I/Osize(minimum/optimal):512bytes/512bytes
root@debian:~#
6. DumptheinstanceconfigurationfromthehostOS:
root@kvm:~#virshdumpxmlkvm1
<domaintype='kvm'id='23'>
...
<devices>
<emulator>/usr/bin/qemu-system-x86_64</emulator>
<disktype='file'device='disk'>
<drivername='qemu'type='raw'/>
<sourcefile='/tmp/kvm1.img'/>
<targetdev='hda'bus='ide'/>
<aliasname='ide0-0-0'/>
<addresstype='drive'controller='0'bus='0'target='0'unit='0'/>
</disk>
<disktype='file'device='disk'>
<drivername='qemu'type='raw'/>
<sourcefile='/tmp/new_disk.img'/>
<targetdev='vda'bus='scsi'/>
<aliasname='scsi0-0-2'/>
<addresstype='drive'controller='0'bus='0'target='0'unit='2'/>
</disk>
</devices>
</domain>
root@kvm:~#
7. Getinformationaboutthenewdisk:
root@kvm:~#virshdomblkstatkvm1vda
vdard_req119
vdard_bytes487424
vdawr_req0
vdawr_bytes0
vdaflush_operations0
vdard_total_times29149092
vdawr_total_times0
vdaflush_total_times0
root@kvm:~#
8. Detachthedisk:
root@kvm:~#virshdetach-diskkvm1vda--live
Diskdetachedsuccessfully
root@kvm:~#
9. Copyorcreateanewrawimage:
root@kvm:~#cp/tmp/new_disk.img/tmp/other_disk.img
root@kvm:~#
10. Writethefollowingconfigfile:
root@kvm:~#catother_disk.xml
<disktype='file'device='disk'>
<drivername='qemu'type='raw'cache='none'/>
<sourcefile='/tmp/other_disk.img'/>
<targetdev='vdb'/>
</disk>
root@kvm:~#
11. Attachthenewdevice:
root@kvm:~#virshattach-devicekvm1--liveother_disk.xml
Deviceattachedsuccessfully
root@kvm:~#
12. Detachtheblockdevice:
root@kvm:~#virshdetach-devicekvm1other_disk.xml--live
Devicedetachedsuccessfully
root@kvm:~#
Howitworks...AttachingmorediskstorunningKVMinstancescanbequiteuseful,especiallywhenusingLVMinsidetheguestOS,asthisallowsforextendingthelogicalvolumes,thusaddingmorediskspaceonthego.Libvirtprovidestwodifferentmethodsforthisaswesawinthestepsoutlinedearlier.Wecanusethevirshattach-diskcommandbypassingthelocationoftheimagefileandthenameofthenewblockdevicefortheguestVMaswesawinstep2.
Instep1,wecreatedanewrawimageusingtheddcommand;however,wecouldhaveusedtheqemu-imgtoolaswesawintheManagingDiskimageswithqemu-imgandddrecipefromChapter1,GettingStartedwithQEMUandKVM.
Afterattachingthenewdiskinstep2,insteps3,4,and5,weconnectedtoVMandverifiedthatanewblockdeviceisindeedpresent.ThisisalsoreflectedintheXMLdefinitionoftheinstanceinstep6.
TomakethenewdeviceavailableafteraVMrestartandpersisttheXMLdefinitionchanges,passthe--persistoptiontothevirshattach-diskcommand.
Instep7,wedisplaysomeinformationaboutthenewdisk.Thisdataisquiteusefulinordertomonitortheread/writerequestsfortheblockdevice,withouthavingtoattachtothevirtualinstance.
Instep8,wedetachedthediskfromtherunningKVMinstance.Ifyoudumptheinstancedefinitionatthispoint,youwillnotetheabsenceoftheextradisk.
Analternativewayofattachingablockdeviceisshowninstep10.WefirstcreateanewXMLfilewiththedefinitionoftheblockdeviceweareattaching.Notehowsimilarthedefinitionistotheoutputinstep6.
Instep11,wedetachthenewdeviceyetagain.NotethatwehavetospecifythesamedeviceXMLdefinitionfileinordertodothat.
OncethediskisvisibleinsidetheguestOS,wecanuseitasaregularblockdevice,wecanpartitionit,createafilesystem,andmountit.
device,wecanpartitionit,createafilesystem,andmountit.
SharingdirectoriesbetweenarunningVMandthehostOSInthepreviousrecipe,wesawtwoexamplesonhowtoattachdiskstoarunningKVMinstance.Inthisrecipe,wearegoingtoshareadirectoryfromthehostOSandmakeitavailableinthevirtualmachine.Wecanonlyperformthisactiononastoppedinstancehowever.Ifyou'vebeenfollowingalong,youshouldalreadyhavealibvirtKVMinstancethatyoucanuse.
GettingreadyTheprerequisitesforthisrecipeareasfollows:
StoppedlibvirtKVMinstancewithconsoleaccessAguestOSwiththe9pandvirtiokernelmodules(availableonmostLinuxdistributionsbydefault)
Howtodoit...ToshareadirectoryfromthehostOStotheKVMguest,executethefollowing:
1. CreateanewdirectoryonthehostOSandaddafiletoit:
root@kvm:~#mkdir/tmp/shared
root@kvm:~#touch/tmp/shared/file
root@kvm:~#
2. AddthefollowingdefinitiontothestoppedKVMinstance:
root@kvm:~#virsheditkvm1
...
<devices>
...
<filesystemtype='mount'accessmode='passthrough'>
<sourcedir='/tmp/shared'/>
<targetdir='tmp_shared'/>
</filesystem>
...
</devices>
...
Domainkvm1XMLconfigurationedited.
root@kvm:~#
3. StarttheVM:
root@kvm:~#virshstartkvm1
Domainkvm1started
root@kvm:~#
4. Connecttotheconsoleasfollows:
root@kvm:~#virshconsolekvm1
Connectedtodomainkvm1
Escapecharacteris^]
DebianGNU/Linux8debianttyS0
debianlogin:root
Password:
...
root@debian:~#
5. Ensurethatthe9pandthevirtiokernelmodulesareloaded:
root@debian:~#lsmod|grep9p
9pnet_virtio170060
9pnet6163219pnet_virtio
virtio_ring175133virtio_pci,virtio_balloon,9pnet_virtio
virtio130583virtio_pci,virtio_balloon,9pnet_virtio
root@debian:~#
6. Mounttheshareddirectoryto/mnt:
root@debian:~#mount-t9p-otrans=virtiotmp_shared/mnt
root@debian:~#
7. Listthenewmount:
root@debian:~#mount|greptmp_shared
tmp_sharedon/mnttype9p(rw,relatime,sync,dirsync,trans=virtio)
root@debian:~#
8. EnsurethatthesharedfileisvisibleinthehostOS:
root@debian:~#ls-la/mnt/
total8
drwxr-xr-x2rootroot4096Mar2311:25.
drwxr-xr-x22rootroot4096Mar2216:28..
-rw-r--r--1rootroot0Mar2311:25file
root@debian:~#
Howitworks...Let'sgetthroughthestepsandseewhatwasaccomplishedinmoredetailsintheprevioussection.
Instep1,wecreateadirectoryandafilethatwewanttosharewiththeguestOS.Then,onthestoppedKVMinstance,weaddedthenew<filesystem>definitioninstep2.Weusedthemounttypebecausewearemountingadirectoryandspecifiedtheaccessmode,whichspecifiesthesecuritymodeforaccessingthesharedresource.Therearethreeaccessmodes:
passthrough:Thisisthedefaultmode,whichaccessestheshareddirectoryusingthepermissionsoftheuserinsidetheguestOSmapped:Inthismode,theshareddirectoryanditsfilesareaccessedusingthepermissionsoftheQEMUuser,inheritedfromthehostsquash:Thismodeissimilartothepassthroughmode;however,thefailuresofprivilegedoperationssuchaschmodareignored
Withthenewdefinitioninplace,westarttheVMinstep3andconnecttoitinstep4.
OntheDebianvirtualmachinewehavebeenusing,therequiredkernelmodulehasbeenloadedwhentheVMstarted.IfthisisnotthecaseforyourVM,loadthemodulesbyrunning:
root@debian:~#modprobe9pvirtio
root@debian:~#
Themainactionhappensinstep6,wherewemounttheshareddirectoryandensurethatithasbeensuccessfullymountedandthefilepresentinthesubsequentsteps.
There'smore...Inthischapter,wehavebeenstartingKVMvirtualmachinesusingthevirshcommand,providedbythelibvirttoolsetandlibraries.Ifyouchecktheprocesstreeafterstartingaguest,youcanseethatvirshcommandactuallycallsthe/usr/bin/qemu-system-x86_64binary.IfyourecallfromtheRunningvirtualmachineswithqemu-system-*recipeinChapter1,GettingStartedwithQEMUandKVM;thisisexactlywhatweusedtostartQEMU/KVMvirtualmachines.
NotetheprocessthatthelibvirtdaemonstartedwhenweranstartedtheKVMinstanceinthisrecipe:
root@kvm:~#pgrep-lfaqemu
6233/usr/bin/qemu-system-x86_64-namekvm1-S-machinepc-i440fx-
trusty,accel=kvm,usb=off-m2048-realtimemlock=off-smp
2,sockets=2,cores=1,threads=1-uuid6ad84d8a-229d-d1f6-ecfc-d29a25fcfa03-no-
user-config-nodefaults-chardev
socket,id=charmonitor,path=/var/lib/libvirt/qemu/kvm1.monitor,server,nowait-mon
chardev=charmonitor,id=monitor,mode=control-rtcbase=utc-no-shutdown-boot
strict=on-devicepiix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2-device
lsi,id=scsi0,bus=pci.0,addr=0x5-drivefile=/tmp/kvm1.img,if=none,id=drive-ide0-
0-0,format=raw-deviceide-hd,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-
0,bootindex=1-fsdevlocal,security_model=passthrough,id=fsdev-
fs0,path=/tmp/shared-devicevirtio-9p-pci,id=fs0,fsdev=fsdev-
fs0,mount_tag=tmp_shared,bus=pci.0,addr=0x6-netdevtap,fd=25,id=hostnet0-
devicertl8139,netdev=hostnet0,id=net0,mac=52:54:00:c5:c8:9d,bus=pci.0,addr=0x3
-chardevpty,id=charserial0-deviceisa-serial,chardev=charserial0,id=serial0-
vnc146.20.141.158:0-devicecirrus-vga,id=video0,bus=pci.0,addr=0x2-device
virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x4
root@kvm:~#
Insteadofusinglibvirt,wecanstartanewguestOSwiththesameshareddirectoryweuseinthisrecipe,byjustrunningthefollowing,justmakesuretostopthelibvirtinstancewestartedearlierfirst:
root@kvm:~#qemu-system-x86_64-namedebian-fsdev
local,id=tmp,path=/tmp/shared,security_model=passthrough-devicevirtio-9p-
pci,fsdev=tmp,mount_tag=tmp_shared-enable-kvm-usbdevicetablet-vnc
146.20.141.158:0-m1024-driveformat=raw,file=/tmp/kvm1.img-daemonize
root@kvm:~#
YoushouldbeabletouseyourVNCclienttoconnecttotheguestandperformthesamestepstomounttheshareddirectory,aswedidearlier.
AutostartingKVMinstancesOnceaKVMinstancehasbeendefinedandstarted,itwillrununtilthehostOSisup.OncethehostOSrestarts,instancesbuildwithlibvirtwillnotautomaticallystartoncethehostisupandthelibvirtdaemonisrunning.Inthisrecipe,wearegoingtochangethisbehaviorandensurevirtualinstancestartwhenthelibvirtdaemonstarts.
GettingreadyForthisrecipe,wearegoingtoneedasingleKVMinstancebuildwithlibvirt.
Howtodoit...ToconfigureaKVMguesttoautomaticallystartafteraserver,orlibvirtdrestart,runthefollowing:
1. EnabletheVMautostart:
root@kvm:~#virshautostartkvm1
Domainkvm1markedasautostarted
root@kvm:~#
2. Obtaininformationfortheinstance:
root@kvm:~#virshdominfokvm1
Id:31
Name:kvm1
UUID:6ad84d8a-229d-d1f6-ecfc-d29a25fcfa03
OSType:hvm
State:running
CPU(s):2
CPUtime:10.9s
Maxmemory:2097152KiB
Usedmemory:1048576KiB
Persistent:yes
Autostart:enable
Managedsave:no
Securitymodel:none
SecurityDOI:0
root@kvm:~#
3. Stoptherunninginstanceandensurethatitisintheshutoffstate:
root@kvm:~#virshdestroykvm1
Domainkvm1destroyed
root@kvm:~#virshlist--all
IdNameState
----------------------------------------------------
-kvm1shutoff
root@kvm:~#
4. Stopthelibvirtdaemonandensurethatitisnotrunning:
root@kvm:~#/etc/init.d/libvirt-binstop
libvirt-binstop/waiting
root@kvm:~#pgrep-lfalibvirtd
root@kvm:~#
5. Startbackthelibvirtdaemon:
root@kvm:~#/etc/init.d/libvirt-binstart
libvirt-binstart/running,process6639
root@kvm:~#
6. Listallrunninginstances:
root@kvm:~#virshlist--all
IdNameState
----------------------------------------------------
2kvm1running
root@kvm:~#
7. Disabletheautostartoption:
root@kvm:~#virshautostartkvm1--disable
Domainkvm1unmarkedasautostarted
root@kvm:~#
8. Verifythechange:
root@kvm:~#virshdominfokvm1|grep-iautostart
Autostart:disable
root@kvm:~#
Howitworks...Inthissimplerecipe,weenabledtheautostartfeatureofalibvirtcontrolledKVMinstance.
Instep1,weenabledautostartandverifiedthatithasbeenenabledinstep2.
Next,tosimulateaserverrestart,wefirststoptherunninginstanceinstep3andthelibvirtdaemoninstep4.
Instep5,westartedthelibvirtdaemonbackandobservethatitstartedthevirtualmachineaswell,asseeninstep6.
Finally,insteps7and8,wedisabletheautostartfeatureandensurethatitindeedhasbeendisabled.
WorkingwithstoragepoolsLibvirtprovidesacentralizedwayofmanaginginstancevolumes(beingimagefilesordirectories)bydefiningstoragepools.Astoragepoolisacollectionofvolumesthatthencanbeassignedtovirtualmachinesandusedtohosttheirfilesystemsoraddedasadditionalblockdevices.ThemainbenefitsofusingstoragepoolsistheabilityforlibvirttopresentandmanagethegivenstoragetypetoVMsinacentralizedway.
Asofthiswriting,thefollowingstoragepoolbackendsareavailable:
DirectorybackendLocalfilesystembackendNetworkfilesystembackendLogicalbackendDiskbackendiSCSIbackendSCSIbackendMultipathbackendRADOSblockdevicebackendSheepdogbackendGlusterbackendZFSbackendVirtuozzostoragebackend
Inthisrecipe,wearegoingtocreateadirectory-backedstoragepool,moveanexistingimagetoit,andthenprovisionanewKVMinstanceusingthestoragepoolandvolume.
GettingreadyForthisrecipe,wearegoingtoneedthefollowing:
TheDebianrawimagewecreatedintheBuildingnewKVMinstanceswithvirt-installandusingtheconsolerecipeThelibvirtpackage
Howtodoit...Thefollowingstepsdemonstratehowtocreateanewstoragepool,inspectit,andassignittoavirtualmachine:
1. CopytherawDebianimagefilewecreatedintheBuildingnewKVMinstanceswithvirt-installandusingtheconsolerecipeearlierinthischapter:
root@kvm:~#cp/tmp/kvm1.img/var/lib/libvirt/images/
root@kvm:~#
2. Createthefollowingstoragepooldefinition:
root@kvm:~#catfile_storage_pool.xml
<pooltype="dir">
<name>file_virtimages</name>
<target>
<path>/var/lib/libvirt/images</path>
</target>
</pool>
root@kvm:~#
3. Definethenewstoragepool:
root@kvm:~#virshpool-definefile_storage_pool.xml
Poolfile_virtimagesdefinedfromfile_storage_pool.xml
root@kvm:~#
4. Listallstoragepools:
root@kvm:~#virshpool-list--all
NameStateAutostart
-------------------------------------------
file_virtimagesinactiveno
root@kvm:~#
5. Startthenewstoragepoolandensurethatit'sactive:
root@kvm:~#virshpool-startfile_virtimages
Poolfile_virtimagesstarted
root@kvm:~#virshpool-list--all
NameStateAutostart
-------------------------------------------
file_virtimagesactiveno
root@kvm:~#
6. Enabletheautostartfeatureonthestoragepool:
root@kvm:~#virshpool-autostartfile_virtimages
Poolfile_virtimagesmarkedasautostarted
root@kvm:~#virshpool-list--all
NameStateAutostart
-------------------------------------------
file_virtimagesactiveyes
root@kvm:~#
7. Obtainmoreinformationaboutthestoragepool:
root@kvm:~#virshpool-infofile_virtimages
Name:file_virtimages
UUID:d51d500b-8885-4c26-8000-2ae46ffe9018
State:running
Persistent:yes
Autostart:yes
Capacity:219.87GiB
Allocation:7.99GiB
Available:211.88GiB
root@kvm:~#
8. Listallvolumesthatareapartofthestoragepool:
root@kvm:~#virshvol-listfile_virtimages
NamePath
----------------------------------------------------
kvm1.img/var/lib/libvirt/images/kvm1.img
root@kvm:~#
9. Obtaininformationonthevolume:
root@kvm:~#virshvol-info/var/lib/libvirt/images/kvm1.img
Name:kvm1.img
Type:file
Capacity:8.00GiB
Allocation:1.87GiB
root@kvm:~#
10. StartnewKVMinstanceusingthestoragepoolandvolume,thenensurethatit'srunning:
root@kvm:~#virt-install--namekvm1--ram1024--graphics
vnc,listen=146.20.141.158--hvm--diskvol=file_virtimages/kvm1.img--
import
Startinginstall...
Creatingdomain...|0B00:00
Domaincreationcompleted.Youcanrestartyourdomainbyrunning:
virsh--connectqemu:///systemstartkvm1
root@kvm:~#
root@kvm:~#virshlist--all
IdNameState
----------------------------------------------------
3kvm1running
root@kvm:~#
Howitworks...WestartthisrecipewithanimageofaDebianOSthatweinstalledearlierinthebook;however,youcanuseanempty,raw,orqcow2image,addittothestoragepool,andinstallthevirtualmachineOSonitwithalmostnochangestotherecipestepsifyoudon'thavethatimagealready.
Instep1,wecopytheVMimagetothedefaultlibvirtstoragepoollocationin/var/lib/libvirt/images/,butyoucancreateyourowndirectory,thelocationdoesnotmatteraslongasit'sdefinedinthestoragepoolconfigurationfile.Wedothatinstep2.
Instep3,wedefinethenewstoragepool,byspecifyinganame,targetdirectory,andthetypeofthepool,inthiscase,adirectorybackendpool.Wethenproceedtolistthenewpoolinstep4.Notethat,oncedefined,westillneedtostartit,justlikedefininganewKVMinstancefromanXMLfile.Bydefault,theautostartoptionisnotenabledonanewstoragepool.
Instep5,westartthestoragepoolandensurethatit'sactive.Wethenproceedtoenabletheautostartfeaturesothatthevolumescanbeusedincasethehostserverrestartsinstep6.
Althoughnotmandatory,wecheckthemetadataprovidedforthestoragepoolanditsvolumesinstep7.Notethattheallocationfieldshowshowmuchspaceisusedbythevolumesinthepool.Wecurrentlyhaveasinglerawimagewiththatexactsize.
Instep8,welistallvolumesthatareapartofthenewstoragepoolandobtainfurtherinformationaboutthesinglevolumeinstep9.
Finallyinstep10,westartanewKVMinstanceusingthestoragepoolandvolumebypassingthestoragepoolandvolumenamestothevoldisktype.
There'smore...Let'slookataslightlymorecomplicatedexampleofusingstoragepoolsbydefininganiSCSI-backedpool.
CreatinganiSCSItargetandloggingitontheinitiatorserverisbeyondthescopeofthisrecipe,soweassumethatyouhaveaniSCSItargetreadytobeusedfromaremoteserver.Thenewstoragepooldefinitionisasfollows:
<pooltype='iscsi'>
<name>iscsi_virtimages</name>
<source>
<hostname='iscsi-target.linux-admins.net'/>
<devicepath='iqn.2004-04.ubuntu:ubuntu16:iscsi.libvirtkvm'/>
</source>
<target>
<path>/dev/disk/by-path</path>
</target>
</pool>
Thefileisverysimilartothedirectory-backedstoragepool,themaindifferencearethefollowingattributes:
The<host>attributespecifiesthehostnameoftheiSCSItargetserverthatisexportingtheiSCSILUNThe<device>specifiesthenameoftheiSCSILUNwearegoingtologinOnceanewiSCSIblockdevicehasbeenloggedin,itwillappearinthelocationspecifiedin<path>,onmostLinuxdistributionsinthe/dev/disk/by-pathdirectory
Wedefineandstartthenewstoragepoolthesamewaywedidinsteps3and5earlierintherecipe.Oncethestoragepoolisactive,libvirtwilllogtheremoteiSCSItargetLUNs.WecanlisttheavailableiSCSIvolumesasusual:
root@kvm:~#virshvol-listiscsi_virtimages
NamePath
-----------------------------------------
10.0.0.1/dev/disk/by-path/ip-10.184.226.106:3260-iscsi-iqn.2004-
04.ubuntu:ubuntu16:iscsi.libvirtkvm-lun-1
root@kvm:~#
TostartanewinstallationprocessusingtheiSCSIvolumeasthetargetforthe
guestOSfilesystem,runthefollowingcode:
root@kvm:~#virt-install--namekvm1--ram1024--extra-args="textconsole=tty0
utf8console=ttyS0,115200"--graphicsvnc,listen=146.20.141.158--hvm--
location=http://ftp.us.debian.org/debian/dists/stable/main/installer-amd64/--
diskvol=iscsi_virtimages/10.0.0.1
Startinginstall...
...
root@kvm:~#
FormoreinformationabouttheXMLdefinitionoftheotherbackendtypes,pleaserefertohttps://libvirt.org/storage.html.
ManagingvolumesInthepreviousrecipe,wesawhowtocreatenewstoragepools,addavolumetoit,andcreateanewKVMinstanceusingthatvolume.Inthisrecipe,wearegoingtofocusonmanipulatingvolumesthatareapartofanexistingstoragepool.Strictlyspeaking,wearenotrequiredtousestoragepoolsandvolumesinordertobuildVMs.Wecanuseothertoolstomanageandmanipulatethevirtualinstanceimages,suchastheqemu-imgutility.Usingvolumesisjustaconvenienceforhavingacentralizedstoragerepositoryofvariousbackendtypes.
GettingreadyThemainrequirementofthisrecipeistohaveanexistingstoragepoolwiththedirectorybackend.Ifyouskippedthepreviousrecipe,nowisthattimetocreateanewone,aswe'llbeusingittomanipulatevolumes.
Howtodoit...Tocreate,inspectandassignvolumestoaninstance,runthefollowing:
1. Listtheavailablestoragepools:
root@kvm:~#virshpool-list--all
NameStateAutostart
-------------------------------------------
file_virtimagesactiveyes
root@kvm:~#
2. Listtheavailablevolumes,thatareapartofthestoragepool:
root@kvm:~#virshvol-listfile_virtimages
NamePath
--------------------------------------------------------------------
kvm1.img/var/lib/libvirt/images/kvm1.img
root@kvm:~#
3. Createanewvolumewiththespecifiedsize:
root@kvm:~#virshvol-create-asfile_virtimagesnew_volume.img9G
Volnew_volume.imgcreated
root@kvm:~#
4. Listthevolumesonthefilesystem:
root@kvm:~#ls-lah/var/lib/libvirt/images/
total11G
drwx--x--x2rootroot4.0KMar2320:38.
drwxr-xr-x8rootroot4.0KMar2123:16..
-rwxr-xr-x1libvirt-qemukvm8.0GMar2320:23kvm1.img
-rw-------1rootroot9.0GMar2320:38new_volume.img
root@kvm:~#
5. Obtaininformationaboutthenewvolume:
root@kvm:~#qemu-imginfo/var/lib/libvirt/images/new_volume.img
image:/var/lib/libvirt/images/new_volume.img
fileformat:raw
virtualsize:9.0G(9663676416bytes)
disksize:9.0G
root@kvm:~#
6. Usethevirshcommandtogetevenmoreinformation:
root@kvm:~#virshvol-infonew_volume.img--poolfile_virtimages
Name:new_volume.img
Type:file
Capacity:9.00GiB
Allocation:9.00GiB
root@kvm:~#
7. Dumpthevolumeconfiguration:
root@kvm:~#virshvol-dumpxmlnew_volume.img--poolfile_virtimages
<volumetype='file'>
<name>new_volume.img</name>
<key>/var/lib/libvirt/images/new_volume.img</key>
<source>
</source>
<capacityunit='bytes'>9663676416</capacity>
<allocationunit='bytes'>9663680512</allocation>
<target>
<path>/var/lib/libvirt/images/new_volume.img</path>
<formattype='raw'/>
<permissions>
<mode>0600</mode>
<owner>0</owner>
<group>0</group>
</permissions>
<timestamps>
<atime>1490301514.446004048</atime>
<mtime>1490301483.698003615</mtime>
<ctime>1490301483.702003615</ctime>
</timestamps>
</target>
</volume>
root@kvm:~#
8. Resizethevolumeanddisplaythenewsize:
root@kvm:~#virshvol-resizenew_volume.img10G--poolfile_virtimages
Sizeofvolume'new_volume.img'successfullychangedto10G
root@kvm:~#virshvol-infonew_volume.img--poolfile_virtimages
Name:new_volume.img
Type:file
Capacity:10.00GiB
Allocation:9.00GiB
root@kvm:~#
9. Deletethevolumeandlistallavailablevolumesinthestoragepool:
root@kvm:~#virshvol-deletenew_volume.img--poolfile_virtimages
Volnew_volume.imgdeleted
root@kvm:~#virshvol-listfile_virtimages
NamePath
------------------------------------------------------------------
kvm1.img/var/lib/libvirt/images/kvm1.img
root@kvm:~#
10. Clonetheexistingvolume:
root@kvm:~#virshvol-clonekvm1.imgkvm2.img--poolfile_virtimages
Volkvm2.imgclonedfromkvm1.img
root@kvm:~#virshvol-listfile_virtimages
NamePath
--------------------------------------------------------------------
kvm1.img/var/lib/libvirt/images/kvm1.img
kvm2.img/var/lib/libvirt/images/kvm2.img
root@kvm:~#
Howitworks...Westartthisrecipewiththefile_virtimagesstoragepoolwecreatedinthepreviousrecipe.Welistallstoragepoolsinstep1toconfirmthat.Instep2,weseethatourstoragepoolcontainsasinglevolume.Nosurpriseshereaswecreatedthatinthelastrecipeinthischapter.
Instep3,wecreateanewvolume,byspecifyingitsname,size,andthestoragepoolwewantittobeapartof.Sincethisisadirectory-backedstoragepool,wecanseethevolumeasarawimagefileinstep4.
Insteps5and6,wecollectmoreinformationaboutthenewvolume.Wecanseethatitisaraw,thereforebydefaultasparseimage.Sparseimagesdon'tallocateallofthediskspaceandgrowasmoredataisbeingwrittentoit.
Instep7,wedumpthedefinitionofthevolume.Wecanusethattodefineanewvolumelateronwiththevirshvol-createcommand.
Libvirtprovidesaconvenientwaytoresizeexistingimages.Thisiswhatwedoinstep8--weresizetheimageto10GB.Wecannowseethattheallocationsizeissmallerthanthecapacity;thisisbecausetheimageisraw.
Finally,instep9,wedeletetheimage,thoughwecouldhaveusedittoinstallanewvirtualmachine,asshownintheWorkingwithstoragepoolsrecipe.
Inthelaststep,weusetheexistingDebianimageandcreatedaclonevolumefromit.StartingavirtualmachineusingtheclonedvolumewillresultinanidenticalKVMinstance,astheoneweclonedthevolumefrom.ThiscombinedwithadumpoftheinstancedefinitionisagreatwaytobackupyourKVMinstances,aslongasyoustorethevolumeimagefileandtheXMLdefinitionfiletoaremotelocation.WearegoingtoexplorebackingupKVMinstancesinlaterrecipes.
ManagingsecretsLibvirtprovidesanAPItocreate,store,andusesecrets.Secretsareobjectsthatcontainsensitiveinformationsuchaspasswords,thatcanbeassociatedwithdifferentvolumebackendtypes.RecallfromtheWorkingwithstoragepoolsrecipe,whichwecreatedaniSCSIpoolandvolumefromaremoteiSCSItargetanduseditastheimageforaKVMguest.Inproductionenvironments,moreoftenthannotiSCSItargetsarepresentedwithCHAPauthentication.Inthisrecipe,wearegoingtocreateasecrettobeusedwithaniSCSIvolume.
GettingreadyForthisrecipe,wearegoingtoneedthefollowing:
AstoragepoolwithaniSCSI-backedvolumeThelibvirtpackage
Howtodoit...Todefineandlistsecretswithlibvirt,performthestepsoutlinedhere:
1. Listallavailablesecrets:
root@kvm:~#virshsecret-list
UUIDUsage
-------------------------------------------------------------------
root@kvm:~#
2. Createthefollowingsecretsdefinition:
root@kvm:~#catvolume_secret.xml
<secretephemeral='no'>
<description>PassphrasefortheiSCSIiscsi-target.linux-admins.net
targetserver</description>
<usagetype='iscsi'>
<target>iscsi_secret</target>
</usage>
</secret>
root@kvm:~#
3. Createthesecretandensurethatithasbeensuccessfullycreated:
root@kvm:~#virshsecret-definevolume_secret.xml
Secret7ad1c208-c2c5-4723-8dc5-e2f4f576101acreated
root@kvm:~#virshsecret-list
UUIDUsage
-----------------------------------------------------------------
7ad1c208-c2c5-4723-8dc5-e2f4f576101aiscsiiscsi_secret
root@kvm:~#
4. Setavalueforthesecret:
root@kvm:~#virshsecret-set-value7ad1c208-c2c5-4723-8dc5-e2f4f576101a
$(echo"some_password"|base64)
Secretvalueset
root@kvm:~#
5. CreateanewiSCSIpooldefinitionfile:
root@kvm:~#catiscsi.xml
<pooltype='iscsi'>
<name>iscsi_virtimages</name>
<source>
<hostname='iscsi-target.linux-admins.net'/>
<devicepath='iqn.2004-04.ubuntu:ubuntu16:iscsi.libvirtkvm'/>
<authtype='chap'username='iscsi_user'>
<secretusage='iscsi_secret'/>
</auth>
</source>
<target>
<path>/dev/disk/by-path</path>
</target>
</pool>
root@kvm:~#
Howitworks...Instep1,welistallavailablesecretsthatlibvirtknowsabout.Sincewehaven'tcreatedany,thelistisempty.
Instep2,wecreatetheXMLdefinitionofthesecret.TheXMLelementsthatweusetodefinethesecretare:
The<secret>rootelement,withanoptionalephemeralattribute,tellinglibvirtthatthepasswordshouldonlybestoredinmemory,ifsettoyes.The<description>attributecontaininganarbitrarydescription.The<usage>elementspecifieswhatthesecretsisgoingtobeusedforanditstype.Inthisexample,thetypeattributeissettoiSCSI.Theotheravailabletypesarevolume,ceph,andtls.Thetypeattributeismandatory.The<target>elementthatspecifiesanarbitrarynameistobeusedintheiSCSIpooldefinition.
Withtheconfigurationfileinplace,wecreatethesecretinstep3.Iftheoperationissuccessful,libvirtreturnsanUUIDthatidentifiesthesecret.
Instep4,wesetavalueforthesecret,bybase64encodingthesome_passwordstring,whichisthepasswordfortheiSCSItargetwewouldliketouse,asastoragepoolvolume.
Andfinallyinstep5,weaddthe<auth>attributeunderthe<source>sectionoftheiSCSIpooldefinition.NotethatthesecretwewouldliketheiSCSIvolumetouseisspecifiedinthe<secretusage='iscsi_secret'/>attribute.Libvirtcannowusetheiscsi_secretnametolocatetheactualpasswordthatithasstored.
KVMNetworkingwithlibvirtInthischapter,wearegoingtocoverthefollowingtopics:
TheLinuxbridgeTheOpenvSwitchConfiguringNATforwardingnetworkConfiguringbridgednetworkConfiguringPCIpassthroughnetworkManipulatingnetworkinterfaces
IntroductionWithlibvirt,wecandefinedifferentnetworktypesforourKVMguests,usingthealreadyfamiliarXMLdefinitionsyntaxandthevirshandvirt-installuserspacetools.Inthischapter,wearegoingtodeploythreedifferentnetworktypes,explorethenetworkXMLformat,andseeexamplesonhowtodefineandmanipulatevirtualinterfacesfortheKVMinstances.
TobeabletoconnectthevirtualmachinestothehostOSortoeachother,wearegoingtousetheLinuxbridgeandtheOpenvSwitch(OVS)daemons,userspacetools,andkernelmodules.BothsoftwarebridgingtechnologiesaregreatatcreatingSoftware-definedNetworking(SDN)ofvariouscomplexity,inaconsistentandeasy-to-manipulatemanner.TheLinuxbridgeandOVSbothactasabridge/switchthatthevirtualinterfacesoftheKVMguestscanconnectto.
Withallthisinmind,let'sstartbylearningmoreaboutthesoftwarebridgesinLinux.
TheLinuxbridgeTheLinuxbridgeisasoftwarelayer2devicethatprovidessomeofthefunctionalityofaphysicalbridgedevice.ItcanforwardframesbetweenKVMguests,thehostOS,andvirtualmachinesrunningonotherservers,ornetworks.TheLinuxbridgeconsistsoftwocomponents--auserspaceadministrationtoolthatwearegoingtouseinthisrecipeandakernelmodulethatperformsalltheworkofconnectingmultipleEthernetsegmentstogether.Eachsoftwarebridgewecreatecanhaveanumberofportsattachedtoit,wherenetworktrafficisforwardedtoandfrom.WhencreatingKVMinstances,wecanattachthevirtualinterfacesthatareassociatedwiththemtothebridge,whichissimilartoplugginganetworkcablefromaphysicalserver'sNICtoabridge/switchdevice.Beingalayer2device,theLinuxbridgeworkswithMACaddressesandmaintainsakernelstructuretokeeptrackofportsandassociatedMACaddressesintheformofaContentAddressableMemory(CAM)table.
Inthisrecipe,wearegoingtocreateanewLinuxbridgeandusethebrctlutilitytomanipulateit.
GettingreadyForthisrecipe,wearegoingtoneedthefollowing:
RecentLinuxkernelwithenabled802.1dEthernetbridgingoptions
Tocheckwhetheryourkerneliscompiledwiththosefeaturesorexposedaskernelmodules,runthefollowingcommand:
root@kvm:~#cat/boot/config-`uname-r`|grep-ibridg
#PC-cardbridges
CONFIG_BRIDGE_NETFILTER=y
CONFIG_NF_TABLES_BRIDGE=m
CONFIG_BRIDGE_NF_EBTABLES=m
CONFIG_BRIDGE_EBT_BROUTE=m
CONFIG_BRIDGE_EBT_T_FILTER=m
CONFIG_BRIDGE_EBT_T_NAT=m
CONFIG_BRIDGE_EBT_802_3=m
CONFIG_BRIDGE_EBT_AMONG=m
CONFIG_BRIDGE_EBT_ARP=m
CONFIG_BRIDGE_EBT_IP=m
CONFIG_BRIDGE_EBT_IP6=m
CONFIG_BRIDGE_EBT_LIMIT=m
CONFIG_BRIDGE_EBT_MARK=m
CONFIG_BRIDGE_EBT_PKTTYPE=m
CONFIG_BRIDGE_EBT_STP=m
CONFIG_BRIDGE_EBT_VLAN=m
CONFIG_BRIDGE_EBT_ARPREPLY=m
CONFIG_BRIDGE_EBT_DNAT=m
CONFIG_BRIDGE_EBT_MARK_T=m
CONFIG_BRIDGE_EBT_REDIRECT=m
CONFIG_BRIDGE_EBT_SNAT=m
CONFIG_BRIDGE_EBT_LOG=m
#CONFIG_BRIDGE_EBT_ULOGisnotset
CONFIG_BRIDGE_EBT_NFLOG=m
CONFIG_BRIDGE=m
CONFIG_BRIDGE_IGMP_SNOOPING=y
CONFIG_BRIDGE_VLAN_FILTERING=y
CONFIG_SSB_B43_PCI_BRIDGE=y
CONFIG_DVB_DDBRIDGE=m
CONFIG_EDAC_SBRIDGE=m
#VMEBridgeDrivers
root@kvm:~#
Thebridgekernelmodule
Toverifythatthemoduleisloadedandtoobtainmoreinformationaboutitsversionandfeatures,executethefollowingcommand:
root@kvm:~#lsmod|grepbridge
bridge1109250
stp129762garp,bridge
llc145523stp,garp,bridge
root@kvm:~#
root@kvm:~#modinfobridge
filename:/lib/modules/3.13.0-107-generic/kernel/net/bridge/bridge.ko
alias:rtnl-link-bridge
version:2.3
license:GPL
srcversion:49D4B615F0B11CA696D8623
depends:stp,llc
intree:Y
vermagic:3.13.0-107-genericSMPmod_unloadmodversions
signer:Magrathea:Glaciersigningkey
sig_key:E1:07:B2:8D:F0:77:39:2F:D6:2D:FD:D7:92:BF:3B:1D:BD:57:0C:D8
sig_hashalgo:sha512
root@kvm:~#
Thebridge-utilspackagethatprovidesthetooltocreateandmanipulatetheLinuxbridgeTheabilitytocreatenewKVMguestsusinglibvirtortheQEMUutilitiesoranexistingKVMinstancefromthepreviouschapters
Howtodoit...Tocreate,list,andmanipulateanewLinuxbridge,followthesesteps:
1. InstalltheLinuxbridgepackage,ifitisnotalreadypresent:
root@kvm:~#aptinstallbridge-utils
2. BuildanewKVMinstanceusingtherawimagefromtheInstallingacustomOSontheimagewithdebootstraprecipefromChapter1,GettingStartedwithQEMUandKVM,ifyouarenotreadingthisbookcovertocover:
root@kvm:~#virt-install--namekvm1--ram1024--disk
path=/tmp/debian.img,format=raw--graphicsvnc,listen=146.20.141.158--
noautoconsole--hvm--import
Startinginstall...
Creatingdomain...|0B00:00
Domaincreationcompleted.Youcanrestartyourdomainbyrunning:
virsh--connectqemu:///systemstartkvm1
root@kvm:~#
3. Listalltheavailablebridgedevices:
root@kvm:~#brctlshow
bridgenamebridgeidSTPenabledinterfaces
virbr08000.fe5400559bd6yesvnet0
root@kvm:~#
4. Bringthevirtualbridgedown,deleteit,andensurethatit'sbeendeleted:
root@kvm:~#ifconfigvirbr0down
root@kvm:~#brctldelbrvirbr0
root@kvm:~#brctlshow
bridgenamebridgeidSTPenabledinterfaces
root@kvm:~#
5. Createanewbridgeandbringitup:
root@kvm:~#brctladdbrvirbr0
root@kvm:~#brctlshow
bridgenamebridgeidSTPenabledinterfaces
virbr08000.000000000000no
root@kvm:~#ifconfigvirbr0up
root@kvm:~#
6. AssignanIPaddresstobridge:
root@kvm:~#ipaddradd192.168.122.1devvirbr0
root@kvm:~#ipaddrshowvirbr0
39:virbr0:<BROADCAST,MULTICAST,UP,LOWER_UP>mtu1500qdiscnoqueue
stateUNKNOWNgroupdefault
link/ether32:7d:3f:80:d7:c6brdff:ff:ff:ff:ff:ff
inet192.168.122.1/32scopeglobalvirbr0
valid_lftforeverpreferred_lftforever
inet6fe80::307d:3fff:fe80:d7c6/64scopelink
valid_lftforeverpreferred_lftforever
root@kvm:~#
7. ListthevirtualinterfacesonthehostOS:
root@kvm:~#ipas|grepvnet
38:vnet0:<BROADCAST,MULTICAST,UP,LOWER_UP>mtu1500qdiscpfifo_fast
stateUNKNOWNgroupdefaultqlen500
root@kvm:~#
8. Addthevirtualinterfacevnet0tothebridge:
root@kvm:~#brctladdifvirbr0vnet0
root@kvm:~#brctlshowvirbr0
bridgenamebridgeidSTPenabledinterfaces
virbr08000.fe5400559bd6novnet0
root@kvm:~#
9. EnabletheSpanningTreeProtocol(STP)onbridgeandobtainmoreinformation:
root@kvm:~#brctlstpvirbr0on
root@kvm:~#brctlshowstpvirbr0
virbr0
bridgeid8000.fe5400559bd6
designatedroot8000.fe5400559bd6
rootport0pathcost0
maxage20.00bridgemaxage20.00
hellotime2.00bridgehellotime2.00
forwarddelay15.00bridgeforwarddelay15.00
ageingtime300.00
hellotimer0.26tcntimer0.00
topologychangetimer0.00gctimer90.89
flags
vnet0(1)
portid8001stateforwarding
designatedroot8000.fe5400559bd6pathcost100
designatedbridge8000.fe5400559bd6messageagetimer0.00
designatedport8001forwarddelaytimer0.00
designatedcost0holdtimer0.00
flags
root@kvm:~#
10. FrominsidetheKVMinstance,bringtheinterfaceup,requestanIPaddress,andtestconnectivitytothehostOS:
root@debian:~#ifconfigeth0up
root@debian:~#dhclienteth0
root@debian:~#ipaseth0
2:eth0:<BROADCAST,MULTICAST,UP,LOWER_UP>mtu1500qdiscpfifo_fast
stateUPgroupdefaultqlen1000
link/ether52:54:00:55:9b:d6brdff:ff:ff:ff:ff:ff
inet192.168.122.92/24brd192.168.122.255scopeglobaleth0
valid_lftforeverpreferred_lftforever
inet6fe80::5054:ff:fe55:9bd6/64scopelink
valid_lftforeverpreferred_lftforever
root@debian:~#
root@debian:~#ping192.168.122.1-c3
PING192.168.122.1(192.168.122.1)56(84)bytesofdata.
64bytesfrom192.168.122.1:icmp_seq=1ttl=64time=0.276ms
64bytesfrom192.168.122.1:icmp_seq=2ttl=64time=0.226ms
64bytesfrom192.168.122.1:icmp_seq=3ttl=64time=0.259ms
---192.168.122.1pingstatistics---
3packetstransmitted,3received,0%packetloss,time1999ms
rttmin/avg/max/mdev=0.226/0.253/0.276/0.027ms
root@debian:~#
Howitworks...Whenwefirstinstalledandstartedthelibvirtdaemon,afewthingshappenedautomatically:
AnewLinuxbridgewascreatedwiththenameandIPaddressdefinedinthe/etc/libvirt/qemu/networks/default.xmlconfigurationfileThednsmasqservicewasstartedwithaconfigurationspecifiedinthe/var/lib/libvirt/dnsmasq/default.conffile
Let'sexaminethedefaultlibvirtbridgeconfiguration:
root@kvm:~#cat/etc/libvirt/qemu/networks/default.xml
<network>
<name>default</name>
<bridgename="virbr0"/>
<forward/>
<ipaddress="192.168.122.1"netmask="255.255.255.0">
<dhcp>
<rangestart="192.168.122.2"end="192.168.122.254"/>
</dhcp>
</ip>
</network>
root@kvm:~#
Thisisthedefaultnetworkthatlibvirtcreatedforus,specifyingthebridgename,IPaddress,andtheIPrangeusedbytheDHCPserverthatwasstarted.Wearegoingtotalkaboutlibvirtnetworkinginmuchmoredetaillaterinthischapter;however,we'veshownitheretohelpyouunderstandwherealltheIPaddressesandthebridgenamecamefrom.
WecanseethataDHCPserverisrunningonthehostOSanditsconfigurationfilebyrunningthefollowingcommand:
root@kvm:~#pgrep-lfadnsmasq
38983/usr/sbin/dnsmasq--conf-file=/var/lib/libvirt/dnsmasq/default.conf
root@kvm:~#cat/var/lib/libvirt/dnsmasq/default.conf
##WARNING:THISISANAUTO-GENERATEDFILE.CHANGESTOITARELIKELYTOBE
##OVERWRITTENANDLOST.Changestothisconfigurationshouldbemadeusing:
##virshnet-editdefault
##orotherapplicationusingthelibvirtAPI.
##
##dnsmasqconffilecreatedbylibvirt
strict-order
user=libvirt-dnsmasq
pid-file=/var/run/libvirt/network/default.pid
except-interface=lo
bind-dynamic
interface=virbr0
dhcp-range=192.168.122.2,192.168.122.254
dhcp-no-override
dhcp-leasefile=/var/lib/libvirt/dnsmasq/default.leases
dhcp-lease-max=253
dhcp-hostsfile=/var/lib/libvirt/dnsmasq/default.hostsfile
addn-hosts=/var/lib/libvirt/dnsmasq/default.addnhosts
root@kvm:~#
Fromtheconfigurationfileearlier,notehowtheIPaddressrangefortheDHCPserviceandthenameofthevirtualbridgematchwhatisconfiguredinthedefaultlibvirtnetworkfilethatwejustsaw.
Withallthisinmind,let'sstepthroughalltheactionsweperformedearlier:
Instep1,weinstalledtheuserspacetoolbrctlthatweusetocreate,configure,andinspecttheLinuxbridgeconfigurationintheLinuxkernel.
Instep2,weprovisionedanewKVMinstanceusingacustomrawimagecontainingtheguestOS.Thisstepisnotrequiredifyoucompletedtherecipesinthepreviouschapters.
Instep3,weinvokedthebridgeutilitytolistallavailablebridgedevices.Fromtheoutput,wecanobservethatcurrentlythere'sonebridge,namedvirbr0,whichlibvirtcreatedautomatically.Notethatundertheinterfacescolumn,wecanseethevnet0interface.ThisisthevirtualNICthatwasexposedtothehostOS,whenwestartedtheKVMinstance.Thismeansthatthevirtualmachineisconnectedtothehostbridge.
Instep4,wefirstbringthebridgedowninordertodeleteit,thenweusethebrctlcommandagaintoremovethebridgeandensurethatit'snotpresentonthehostOS.
Instep5,werecreatedthebridgeandbroughtitbackup.Wedothistodemonstratethestepsrequiredtocreateanewbridge.
Instep6,wereassignedthesameIPaddresstothebridgeandlistedit.
Insteps7and8,welistallvirtualinterfacesonthehostOS.Becauseweonly
haveoneKVMguestcurrentlyrunningontheserver,weonlyseeonevirtualinterface,thatis,vnet0.Wethenproceedtoadd/connectthevirtualNICtothebridge.
Instep9,weenabledtheSTPonthebridge.STPisalayer2protocolthathelpspreventnetworkloopsifwehaveredundantnetworkpaths.Thisisespeciallyusefulinlarger,morecomplexnetworktopologies,wheremultiplebridgesareconnectedtogether.
Finally,instep10,weconnecttotheKVMguestusingtheconsole,listitsinterfaceconfiguration,andensurethatwecanpingthebridgeonthehostOS.Inordertodothat,weneedtobringthenetworkinterfaceinsidetheguestupwithifconfigeth0up,thenobtainanIPaddresswiththedhclienteth0commandfromthednsmasqserverrunningonthehost.
There'smore...TherearefewmoreusefulcommandswecanuseontheLinuxbridge.
WealreadyknowthatabridgeforwardsframesbasedontheMACaddressescontainedtherein.ToexaminethetableofMACaddressesthebridgeknowsabout,runthefollowingcommand:
root@kvm:~#brctlshowmacsvirbr0
portnomacaddrislocal?ageingtimer
152:54:00:55:9b:d6no268.02
1fe:54:00:55:9b:d6yes0.00
root@kvm:~#
Fromtheprecedingoutput,wecanseethatthebridgehasrecordedtwoMACaddressesonitsonlyport.Thefirstrecordisanonlocaladdress,anditbelongstothenetworkinterfaceinsidetheKVMinstances.WecanconfirmthatbyconnectingtotheKVMguestasfollows:
root@kvm:~#virshconsolekvm1
Connectedtodomainkvm1
Escapecharacteris^]
root@debian:~#ipaseth0
2:eth0:<BROADCAST,MULTICAST,UP,LOWER_UP>mtu1500qdiscpfifo_faststateUP
groupdefaultqlen1000
link/ether52:54:00:55:9b:d6brdff:ff:ff:ff:ff:ff
inet6fe80::5054:ff:fe55:9bd6/64scopelink
valid_lftforeverpreferred_lftforever
root@debian:~#
ThesecondMACaddressistheaddressofthebridgeitselfandtheMACaddressofthevirtualinterface,belongingtotheKVMvirtualmachine,exposedtothehostOS.Toconfirmthis,runthefollowingcommand:
root@kvm:~#ifconfig|grep"fe:54:00:55:9b:d6"
virbr0Linkencap:EthernetHWaddrfe:54:00:55:9b:d6
vnet0Linkencap:EthernetHWaddrfe:54:00:55:9b:d6
root@kvm:~#
Whenthebridgeseesaframeononeofitsports,itrecordsthetimethenafterasetamountoftimenotseeingthesameMACaddressagain,itwillremovetherecordfromtheitsCAMtable.Wecansetthetimelimitinsecondsbeforethe
bridgewillexpiretheMACaddressentrybyexecutingthefollowingcommand:
root@kvm:~#brctlsetageingvirbr0600
root@kvm:~#
Thebrctlcommandiswelldocumented;tolistallavailablesubcommands,runitwithoutanyparameters:
root@kvm:~#brctl
Usage:brctl[commands]
commands:
addbr<bridge>addbridge
delbr<bridge>deletebridge
addif<bridge><device>addinterfacetobridge
delif<bridge><device>deleteinterfacefrombridge
hairpin<bridge><port>{on|off}turnhairpinon/off
setageing<bridge><time>setageingtime
setbridgeprio<bridge><prio>setbridgepriority
setfd<bridge><time>setbridgeforwarddelay
sethello<bridge><time>sethellotime
setmaxage<bridge><time>setmaxmessageage
setpathcost<bridge><port><cost>setpathcost
setportprio<bridge><port><prio>setportpriority
show[<bridge>]showalistofbridges
showmacs<bridge>showalistofmacaddrs
showstp<bridge>showbridgestpinfo
stp<bridge>{on|off}turnstpon/off
root@kvm:~#
MostLinuxdistributionspackagethebrctlutilityandthisiswhatweusedinthisrecipe.However,tousethelatestversion,orifapackageisnotavailableforyourdistribution,wecanbuildtheutilityfromsourcebycloningtheprojectwithgit,thenconfigureandcompile:
root@kvm:~#cd/usr/src/
root@kvm:/usr/src#apt-getupdate&&apt-getinstallbuild-essentialautomake
pkg-configgit
root@kvm:/usr/src#gitclone
git://git.kernel.org/pub/scm/linux/kernel/git/shemminger/bridge-utils.git
Cloninginto'bridge-utils'...
remote:Countingobjects:654,done.
remote:Total654(delta0),reused0(delta0)
Receivingobjects:100%(654/654),131.72KiB|198.00KiB/s,done.
Resolvingdeltas:100%(425/425),done.
Checkingconnectivity...done.
root@kvm:/usr/src#cdbridge-utils/
root@kvm:/usr/src/bridge-utils#autoconf
root@kvm:/usr/src/bridge-utils#./configure&&make&&makeinstall
root@kvm:/usr/src/bridge-utils#brctl--version
bridge-utils,1.5
root@kvm:/usr/src/bridge-utils#
Fromtheprecedingoutput,wecanseethatwefirstclonedthegitrepositoryfor
thebridge-utilsprojectandthencompiledthesourcecode.
OnaRedHat/CentOShost,theprocessissimilar:
[root@centos~]#cd/usr/src/
[root@centossrc]#
[root@centossrc]#yumgroupinstall"Developmenttools"
[root@centossrc]#gitclone
git://git.kernel.org/pub/scm/linux/kernel/git/shemminger/bridge-utils.git
Cloninginto'bridge-utils'...
remote:Countingobjects:654,done.
remote:Total654(delta0),reused0(delta0)
Receivingobjects:100%(654/654),131.72KiB|198.00KiB/s,done.
Resolvingdeltas:100%(425/425),done.
Checkingconnectivity...done.
[root@centossrc]#cdbridge-utils
[root@centosbridge-utils]#autoconf
[root@centosbridge-utils]#./configure&&make&&makeinstall
[root@centosbridge-utils]#brctl--version
bridge-utils,1.5
[root@centosbridge-utils]#
TheOpenvSwitchOVSisanothersoftwarebridging/switchingdevicethatcanbeusedtocreatevariousvirtualnetworktopologiesandconnectKVMinstancestoit.OVScanbeusedinsteadoftheLinuxbridge,anditprovidesanextensivefeatureset,includingpolicyrouting,AccessControlLists(ACLs),QualityofService(QoS)policing,trafficmonitoring,flowmanagement,VLANtagging,GREtunneling,andmuchmore.
Inthisrecipe,wearegoingtoinstall,configure,andusetheOVSbridgetoconnectaKVMinstancetothehostOS,inasimilarwaytowhatwedidinthepreviousrecipewiththeLinuxbridge.
GettingreadyInorderforthisrecipetowork,weneedtoensurethefollowing:
TheLinuxbridgeisdeleted,ifpresent,andOVSisinstalledWehaveatleastoneKVMinstancerunning
Howtodoit...TocreateanewOVSbridgeandattachthevirtualinterfaceofaKVMguest,followthesesteps:
1. RemovetheexistingLinuxbridge,ifany:
root@kvm:~#brctlshow
bridgenamebridgeidSTPenabledinterfaces
virbr08000.fe5400559bd6yesvnet0
root@kvm:~#ifconfigvirbr0down
root@kvm:~#brctldelbrvirbr0
root@kvm:~#brctlshow
bridgenamebridgeidSTPenabledinterfaces
root@kvm:~#
OnsomeLinuxdistributions,ithelpstounloadthekernelmodulefortheLinuxbridgebeforeusingOVS.Todothis,executeroot@kvm:/usr/src#modprobe-rbridge.
2. InstalltheOVSpackageonUbuntu:
root@kvm:~#apt-getinstallopenvswitch-switch
...
Settingupopenvswitch-common(2.0.2-0ubuntu0.14.04.3)...
Settingupopenvswitch-switch(2.0.2-0ubuntu0.14.04.3)...
openvswitch-switchstart/running
...
root@kvm:~#
3. EnsurethattheOVSprocessesarerunning:
root@kvm:~#pgrep-lfaswitch
22255ovsdb-server/etc/openvswitch/conf.db-vconsole:emer-vsyslog:err
-vfile:info--remote=punix:/var/run/openvswitch/db.sock--private-
key=db:Open_vSwitch,SSL,private_key--
certificate=db:Open_vSwitch,SSL,certificate--bootstrap-ca-
cert=db:Open_vSwitch,SSL,ca_cert--no-chdir--log-
file=/var/log/openvswitch/ovsdb-server.log--
pidfile=/var/run/openvswitch/ovsdb-server.pid--detach--monitor
22264ovs-vswitchd:monitoringpid22265(healthy)
22265ovs-vswitchdunix:/var/run/openvswitch/db.sock-vconsole:emer-
vsyslog:err-vfile:info--mlockall--no-chdir--log-
file=/var/log/openvswitch/ovs-vswitchd.log--
pidfile=/var/run/openvswitch/ovs-vswitchd.pid--detach--monitor
root@kvm:~#
4. EnsurethattheOVSkernelmodulehasbeenloaded:
root@kvm:~#lsmod|grepswitch
openvswitch709890
gre137961openvswitch
vxlan376111openvswitch
libcrc32c126441openvswitch
root@kvm:~#
5. ListtheavailableOVSswitches:
root@kvm:~#ovs-vsctlshow
e5164e3e-7897-4717-b766-eae1918077b0
ovs_version:"2.0.2"
root@kvm:~#
6. CreateanewOVSswitch:
root@kvm:~#ovs-vsctladd-brvirbr1
root@kvm:~#ovs-vsctlshow
e5164e3e-7897-4717-b766-eae1918077b0
Bridge"virbr1"
Port"virbr1"
Interface"virbr1"
type:internal
ovs_version:"2.0.2"
root@kvm:~#
7. AddtheinterfaceoftherunningKVMinstancetotheOVSswitch:
root@kvm:~#ovs-vsctladd-portvirbr1vnet0
root@kvm:~#ovs-vsctlshow
e5164e3e-7897-4717-b766-eae1918077b0
Bridge"virbr1"
Port"virbr1"
Interface"virbr1"
type:internal
Port"vnet0"
Interface"vnet0"
ovs_version:"2.0.2"
root@kvm:~#
8. ConfigureanIPaddressontheOVSswitch:
root@kvm:~#ipaddradd192.168.122.1/24devvirbr1
root@kvm:~#ipaddrshowvirbr1
41:virbr1:<BROADCAST,UP,LOWER_UP>mtu1500qdiscnoqueuestateUNKNOWN
groupdefault
link/etherb2:52:e0:73:89:4ebrdff:ff:ff:ff:ff:ff
inet192.168.122.1/24scopeglobalvirbr1
valid_lftforeverpreferred_lftforever
inet6fe80::b0a8:c2ff:fed4:bb3f/64scopelink
valid_lftforeverpreferred_lftforever
root@kvm:~#
9. ConfigureanIPaddressinsidetheKVMguestandensureconnectivitytothehostOS(iftheimagedoesnothaveconsoleaccessconfigure,connecttoitusingVNC):
root@kvm:~#virshconsolekvm1
Connectedtodomainkvm1
Escapecharacteris^]
root@debian:~#ifconfigeth0up&&ipaddradd192.168.122.210/24dev
eth0
root@debian:~#ipaddrshoweth0
2:eth0:<BROADCAST,MULTICAST,UP,LOWER_UP>mtu1500qdiscpfifo_fast
stateUPgroupdefaultqlen1000
link/ether52:54:00:55:9b:d6brdff:ff:ff:ff:ff:ff
inet192.168.122.210/24scopeglobaleth0
valid_lftforeverpreferred_lftforever
inet6fe80::5054:ff:fe55:9bd6/64scopelink
valid_lftforeverpreferred_lftforever
root@debian:~#ping192.168.122.1
PING192.168.122.1(192.168.122.1)56(84)bytesofdata.
64bytesfrom192.168.122.1:icmp_seq=1ttl=64time=0.711ms
64bytesfrom192.168.122.1:icmp_seq=2ttl=64time=0.394ms
64bytesfrom192.168.122.1:icmp_seq=3ttl=64time=0.243ms
^C
---192.168.122.1pingstatistics---
3packetstransmitted,3received,0%packetloss,time2001ms
rttmin/avg/max/mdev=0.243/0.449/0.711/0.195ms
root@debian:~#
Howitworks...Inordertosimplifyoursetupandavoidconflicts,it'sprudenttofirstremovetheLinuxbridgebeforecreatinganewOVSswitch.Wedeletethebridgeinstep1andoptionallyunloadedthekernelmodule.
Instep2,weinstalltheOVSpackagethatalsostartsthemainOVSdaemonovs-vswitchdresponsibleforcreatingandmodifyingthebridges/switchesonthehostOS.
Instep4,wemakesurethattheOVSkernelmodulehasbeenloaded,andwelistallavailableOVSswitchesonthehostinstep5.
Insteps6and7,wecreateanewOVSswitchandaddtheKVMvirtualinterfacetotheswitch.
Theovsdb-serverprocessthatwasalsostartedafterinstallingthepackage,asseenfromtheoutputinstep3,isadatabaseenginethatusesJSONRemoteProcedureCalls(RPC)tocommunicatewiththemainOVSdaemon.Theovsdbserverprocessstoresinformation,suchastheswitchnetworkflows,ports,andQoStonamejustfew.Youcanquerythedatabasebyrunningthefollowingcommand:
root@kvm:~#ovsdb-clientlist-dbs
Open_vSwitch
root@kvm:~#ovsdb-clientlist-tables
Table
-------------------------
Port
Manager
Bridge
Interface
SSL
IPFIX
Open_vSwitch
Queue
NetFlow
Mirror
QoS
Controller
Flow_Table
sFlow
Flow_Sample_Collector_Set
root@kvm:~#ovsdb-clientdumpOpen_vSwitch
...
Porttable
_uuidbond_downdelaybond_fake_ifacebond_modebond_updelayexternal_ids
fake_bridgeinterfaceslacpmacnameother_configqosstatisticsstatustag
trunksvlan_mode
----------------------------------------------------------------------------
---------------------------------------------------------------------------
------------------------------------------------------------
9b4b743d-66b2-4779-9dd8-404b3aa55e180false[]0{}false[e7ed4e2b-a73c-46c7-
adeb-a203be56587c][][]"virbr1"{}[]{}{}[][][]
f2a033aa-9072-4be3-808e-6e0fce67ce7b0false[]0{}false[86a10eed-698f-4ccc-
b3b7-dd20c13e3ee3][][]"vnet0"{}[]{}{}[][][]
...
root@kvm:~#
Notefromtheprecedingoutputthatthenewswitchvirbr1andportvnet0arenowdisplaying,whenqueryingtheOVSdatabase.
Insteps8and9,weassignIPaddressestotheOVSswitchandtheKVMguestandensurethatwecanreachthehostbridgefrominsidethevirtualmachine.
There'smore...OVSisarathercomplexsoftwareswitch;inthisrecipe,weonlyscratchedthesurface.Inthenextfewrecipes,wecanuseboththeLinuxbridgeandOVS,withminorconfigurationchangesinlibvirt,whichwearegoingtopointoutaswego.
ToremovetheKVMvirtualinterfacefromtheOVSswitch,executethefollowingcommand:
root@kvm:~#ovs-vsctldel-portvirbr1vnet0
root@kvm:~#
TocompletelydeletetheOVSswitch,runthefollowingcommand:
root@kvm:~#ovs-vsctldel-brvirbr1&&ovs-vsctlshow
e5164e3e-7897-4717-b766-eae1918077b0
ovs_version:"2.0.2"
root@kvm:~#
FormoreinformationabouttheOVS,pleasevisittheprojectswebsite,http://openvswitch.org/.
ConfiguringNATforwardingnetworkWhenthelibvirtdaemonstarts,itcreatesadefaultnetworkdefinedinthe/etc/libvirt/qemu/networks/default.xmlconfigurationfile.WhenanewKVMguestisbuildwithoutspecifyinganynetworkingoptions,itwillusethedefaultnetworktocommunicatewiththehostOSandotherguestsandnetworks.ThedefaultlibvirtnetworkisusingtheNetworkAddressTranslation(NAT)method.NATprovidesamappingfromoneIPaddressspacetoanother,bymodifyingtheIPaddressintheheaderoftheIPdatagrampacket.ThisisespeciallyusefulwhenthehostOSprovidesoneIPaddressallowingmultipleguestsonthesamehosttousethataddresstoestablishoutboundconnections.ThevirtualmachinesIPaddressesareessentiallytranslatedtoappearasthehostmachine'sIPaddress.
ThedefaultNATforwardingnetworkdefinesandsetsupaLinuxbridge,forthegueststoconnectto.Inthisrecipe,wearegoingtoexplorethedefaultNATnetworkandlearnabouttheXMLattributesusedtodefineit.Then,wearegoingtocreateanewNATnetworkandconnectourKVMguesttoit.
GettingreadyForthisrecipe,wearegoingtoneedthefollowing:
ALinuxhostwithlibvirtinstalledandthedaemonrunning.Theiptablesandiproute2packagesinstalledonthehostOS.Ifyouinstalledlibvirtfromapackage,chancesarethatiptablesandiproute2havebeeninstalled,asdependenciesofthelibvirtpackage.Ifyou'vebuiltlibvirtfromsource,youmightneedtoinstallthemmanually.ArunningKVMinstances.
Howtodoit...ToconfigureanewNATnetworkandconnectaKVMinstancetoit,runthefollowing:
1. Listallavailablenetworks:
root@kvm:~#virshnet-list--all
NameStateAutostartPersistent
----------------------------------------------------------
defaultactiveyesyes
root@kvm:~#
2. Dumptheconfigurationofthedefaultnetwork:
root@kvm:~#virshnet-dumpxmldefault
<networkconnections='1'>
<name>default</name>
<uuid>2ab5d22c-5928-4304-920e-bc43b8731bcf</uuid>
<forwardmode='nat'>
<nat>
<portstart='1024'end='65535'/>
</nat>
</forward>
<bridgename='virbr0'stp='on'delay='0'/>
<ipaddress='192.168.122.1'netmask='255.255.255.0'>
<dhcp>
<rangestart='192.168.122.2'end='192.168.122.254'/>
</dhcp>
</ip>
</network>
root@kvm:~#
3. ComparethatwiththeXMLdefinitionfileforthedefaultnetwork:
root@kvm:~#cat/etc/libvirt/qemu/networks/default.xml
<network>
<name>default</name>
<bridgename="virbr0"/>
<forward/>
<ipaddress="192.168.122.1"netmask="255.255.255.0">
<dhcp>
<rangestart="192.168.122.2"end="192.168.122.254"/>
</dhcp>
</ip>
</network>
root@kvm:~#
4. Listallrunninginstancesonthehost:
root@kvm:~#virshlist--all
IdNameState
----------------------------------------------------
3kvm1running
root@kvm:~#
5. EnsurethattheKVMinstancesareconnectedtothedefaultLinuxbridge:
root@kvm:~#brctlshow
bridgenamebridgeidSTPenabledinterfaces
virbr08000.fe5400559bd6yesvnet0
root@kvm:~#
6. CreateanewNATnetworkdefinition:
root@kvm:~#catnat_net.xml
<network>
<name>nat_net</name>
<bridgename="virbr1"/>
<forward/>
<ipaddress="10.10.10.1"netmask="255.255.255.0">
<dhcp>
<rangestart="10.10.10.2"end="10.10.10.254"/>
</dhcp>
</ip>
</network>
root@kvm:~#
7. Definethenewnetwork:
root@kvm:~#virshnet-definenat_net.xml
Networknat_netdefinedfromnat_net.xml
root@kvm:~#virshnet-list--all
NameStateAutostartPersistent
----------------------------------------------------------
defaultactiveyesyes
nat_netinactivenoyes
root@kvm:~#
8. Startthenewnetworkandenableautostarting:
root@kvm:~#virshnet-startnat_net
Networknat_netstarted
root@kvm:~#virshnet-autostartnat_net
Networknat_netmarkedasautostarted
root@kvm:~#virshnet-list
NameStateAutostartPersistent
----------------------------------------------------------
defaultactiveyesyes
nat_netactiveyesyes
root@kvm:~#
9. Obtainmoreinformationaboutthenewnetwork:
root@kvm:~#virshnet-infonat_net
Name:nat_net
UUID:fba2ca2b-8ca7-4dbb-beee-14799ee04bc3
Active:yes
Persistent:yes
Autostart:yes
Bridge:virbr1
root@kvm:~#
10. EdittheXMLdefinitionofthekvm1instanceandchangethenameofthesourcenetwork:
root@kvm:~#virsheditkvm1
...
<interfacetype='network'>
...
<sourcenetwork='nat_net'/>
...
</interface>
...
Domainkvm1XMLconfigurationedited.
root@kvm:~#
11. RestarttheKVMguest:
root@kvm:~#virshdestroykvm1
Domainkvm1destroyed
root@kvm:~#virshstartkvm1
Domainkvm1started
root@kvm:~#
12. Listallsoftwarebridgesonthehost:
root@kvm:~#brctlshow
bridgenamebridgeidSTPenabledinterfaces
virbr08000.000000000000yes
virbr18000.525400ba8e2cyesvirbr1-nic
vnet0
root@kvm:~#
13. ConnecttotheKVMinstancesandchecktheIPaddressoftheeth0interfaceandensureconnectivitytothehostbridge(iftheimageisnotconfiguredforconsoleaccess,useaVNCclientinstead):
root@kvm:~#virshconsolekvm1
Connectedtodomainkvm1
Escapecharacteris^]
DebianGNU/Linux8debianttyS0
debianlogin:root
Password:
...
root@debian:~#ipaseth0|grepinet
inet10.10.10.92/24brd10.10.10.255scopeglobaleth0
inet6fe80::5054:ff:fe55:9bd6/64scopelink
root@debian:~#ifconfigeth0up&&dhclienteth0
root@debian:~#ping10.10.10.1-c3
PING10.10.10.1(10.10.10.1)56(84)bytesofdata.
64bytesfrom10.10.10.1:icmp_seq=1ttl=64time=0.313ms
64bytesfrom10.10.10.1:icmp_seq=2ttl=64time=0.136ms
64bytesfrom10.10.10.1:icmp_seq=3ttl=64time=0.253ms
---10.10.10.1pingstatistics---
3packetstransmitted,3received,0%packetloss,time2000ms
rttmin/avg/max/mdev=0.136/0.234/0.313/0.073ms
root@debian:~#
14. OnthehostOS,examinewhichDHCPservicesarerunning:
root@kvm:~#pgrep-lfadnsmasq
38983/usr/sbin/dnsmasq--conf-
file=/var/lib/libvirt/dnsmasq/default.conf
40098/usr/sbin/dnsmasq--conf-
file=/var/lib/libvirt/dnsmasq/nat_net.conf
root@kvm:~#
15. ChecktheIPofthenewbridgeinterface:
root@kvm:~#ipasvirbr1
43:virbr1:<BROADCAST,MULTICAST,UP,LOWER_UP>mtu1500qdiscnoqueue
stateUPgroupdefault
link/ether52:54:00:ba:8e:2cbrdff:ff:ff:ff:ff:ff
inet10.10.10.1/24brd10.10.10.255scopeglobalvirbr1
valid_lftforeverpreferred_lftforever
root@kvm:~#
16. ListtheiptablesrulesfortheNATtable:
root@kvm:~#iptables-L-n-tnat
ChainPREROUTING(policyACCEPT)
targetprotoptsourcedestination
ChainINPUT(policyACCEPT)
targetprotoptsourcedestination
ChainOUTPUT(policyACCEPT)
targetprotoptsourcedestination
ChainPOSTROUTING(policyACCEPT)
targetprotoptsourcedestination
RETURNall--10.10.10.0/24224.0.0.0/24
RETURNall--10.10.10.0/24255.255.255.255
MASQUERADEtcp--10.10.10.0/24!10.10.10.0/24masqports:1024-65535
MASQUERADEudp--10.10.10.0/24!10.10.10.0/24masqports:1024-65535
MASQUERADEall--10.10.10.0/24!10.10.10.0/24
RETURNall--192.168.122.0/24224.0.0.0/24
RETURNall--192.168.122.0/24255.255.255.255
MASQUERADEtcp--192.168.122.0/24!192.168.122.0/24masqports:1024-
65535
MASQUERADEudp--192.168.122.0/24!192.168.122.0/24masqports:1024-
65535
MASQUERADEall--192.168.122.0/24!192.168.122.0/24
RETURNall--192.168.122.0/24224.0.0.0/24
RETURNall--192.168.122.0/24255.255.255.255
MASQUERADEtcp--192.168.122.0/24!192.168.122.0/24masqports:1024-
65535
MASQUERADEudp--192.168.122.0/24!192.168.122.0/24masqports:1024-
65535
MASQUERADEall--192.168.122.0/24!192.168.122.0/24
root@kvm:~#
Howitworks...WestartbylistingallavailablenetworksonthehostOSinstep1.Aswecanseefromtheoutputofthevirshcommand,there'sonlyonedefaultnetworkrunning.
Instep2,weexaminetheconfigurationofthedefaultnetwork.TheXMLdefinitionusesthefollowingattributes:
The<network>attributeistherootelement,instructinglibvirtthatwearedefininganetwork.The<name>elementspecifiesthenameofthenetworkandneedstobeunique.The<uuid>attributeprovidesagloballyuniqueidentifierforthevirtualnetworkandifomitted,itwillbeautogenerated.The<forward>elementanditsmodeattributedefinethenetworkasbeingconnectedtothehostnetworkstack,usingNAT.Ifthiselementismissing,libvirtwillcreateanisolatednetwork.The<nat>subelementfurtherdefinesthe<port>rangethatwillbeusedwhilethehostispreformingNAT.The<bridge>elementspecifiesthebridgetobecreated,itsname,andSTPoptions.The<ip>attributedefinestheIPrangefortheDHCPservertoassignaddressestotheguestVMs.
Instep3,welookattheconfigfileforthedefaultnetworkonthis.Notethatsomeoftheattributesaremissing.Libvirtautogeneratescertainattributesandassignsdefaultvalueswhereappropriate.
Instep4and5,wemakesurethatwehavearunninginstanceconnectedtothedefaultLinuxbridge.
Instep6,wecreateanewnetworkdefinitionusingthedefaultnetworkasatemplate.WechangethenameofthenetworkanddefinenewIPrange.
Withthenewnetworkdefinitionfileready,insteps7and8,wedefinethenewnetwork,startit,andmakesurethatitwillautomaticallystartwhenthelibvirt
daemonstarts,inthecaseofaserverreboot.
Afterobtainingmoreinformationaboutthenewlycreatednetworkinstep9,weproceedtoedittheXMLdefinitionoftheKVMguestinstep10.TomaketheVMpartofthenewnetwork,allweneedtodoisupdatethe<sourcenetwork>element.
AfterrestartingtheKVMguestinstep11,weproceedtolistallavailablesoftwarebridgesonthehostOSinstep12.Notethatwenowhavetwobridges,withthenewbridgehavingtheVMsvirtualinterfacevnet0connectedtoit.
WethenconnecttotherunningKVMguestandensurethatitseth0networkinterfacehasobtainedanIPaddressfromtheDHCPserverrunningonthehostandthattheIPispartoftheaddressrangeweconfiguredearlier.Wealsoensuredconnectivitytothehostbridgeusingthepingcommand.
BackonthehostOS,insteps14and15,wecheckwhatDHCPservicesarerunning.Note,fromtheoutputofthepgrepcommand,thatwenowhavetwodnsmasqprocessesrunning:oneforeachdefinednetwork.
TheNATforwardingisachievedbysettingiptablesrulesaswecanseeinstep18.EachtimewedefineandstartanewNATnetwork,libvirtcreatestherequiredrulesiniptables.Fromtheoutputinstep18,wecanobservethepresenceoftwosetsofNATrules,oneforeachrunningNATnetwork.
ConfiguringbridgednetworkWithfullbridging,wecanconnecttheKVMguestsdirectlytothehostnetwork,withoutusingNAT.However,thissetuprequiresanIPaddress,whichispartofthehostsubnet,foreachvirtualmachine.IfyoucannotallocatethatmanyIPaddresses,considerusingtheNATnetworksetup,asdescribedintheConfiguringNATforwardingnetworkrecipegivenbefore.Inthisnetworkingmode,thevirtualmachinesstillusethehostOSbridgeforconnectivity;however,thebridgeenslavesthephysicalinterfacethatisgoingtobeusedfortheguests.
GettingreadyForthisrecipe,wearegoingtoneedthefollowing:
AserverwithatleasttwophysicalinterfacesTheabilitytoprovisionandstartKVMinstanceswithlibvirtArunningKVMinstance
Howtodoit...Todefineanewbridgednetworkandattachaguesttoit,followthesteps:
1. Takedowntheinterfacewearegoingtobridge:
root@kvm:~#ifdowneth1
root@kvm:~#
2. Editthenetworkconfigurationfileonthehostandreplacetheeth1blockwiththefollowing,ifyourhostOSisDebian/Ubuntu:
root@kvm:~#vim/etc/network/interfaces
...
autovirbr2
ifacevirbr2inetstatic
address192.168.1.2
netmask255.255.255.0
network192.168.1.0
broadcast192.168.1.255
gateway192.168.1.1
bridge_portseth1
bridge_stpon
bridge_maxwait0
...
root@kvm:~#
3. IfusingRedHat/CentOSdistributions,editthefollowingtwofilesinstead:
root@kvm:~#cat/etc/sysconfig/ifcfg-eth1
DEVICE=eth1
NAME=eth1
NM_CONTROLLED=yes
ONBOOT=yes
TYPE=Ethernet
BRIDGE=virbr2
root@kvm:~#cat/etc/sysconfig/ifcfg-bridge_net
DEVICE=virbr2
NAME=virbr2
NM_CONTROLLED=yes
ONBOOT=yes
TYPE=Bridge
STP=on
IPADDR=192.168.1.2
NETMASK=255.255.255.0
GATEWAY=192.168.1.1
root@kvm:~#
4. Startthenewinterfaceup:
root@kvm:~#ifupvirbr2
root@kvm:~#
5. DisablesendingpacketstoiptablesthatoriginatefromtheguestVMs:
root@kvm:~#sysctl-wnet.bridge.bridge-nf-call-iptables=0
net.bridge.bridge-nf-call-iptables=0
root@kvm:~#sysctl-wnet.bridge.bridge-nf-call-iptables=0
net.bridge.bridge-nf-call-iptables=0
root@kvm:~#sysctl-wnet.bridge.bridge-nf-call-arptables=0
net.bridge.bridge-nf-call-arptables=0
root@kvm:~#
6. Listallbridgesonthehost:
root@kvm:~##brctlshow
bridgenamebridgeidSTPenabledinterfaces
virbr08000.000000000000yes
virbr28000.000a0ac60210yeseth1
root@kvm:~#
7. EdittheXMLdefinitionfortheKVMinstance:
root@kvm:~#virsheditkvm1
...
<interfacetype='bridge'>
<sourcebridge='virbr2'/>
</interface>
...
Domainkvm1XMLconfigurationedited.
root@kvm:~#
8. RestarttheKVMinstance:
root@kvm:~#virshdestroykvm1
Domainkvm1destroyed
root@kvm:~#virshstartkvm1
Domainkvm1started
root@kvm:~#
Howitworks...Tosetupbridgenetworking,insteps1and2,wefirstbringthephysicalinterface(eth1inthisexample)down,inordertoenslaveit(makeitapartofthenewbridgewearegoingtocreate).Wethencreatethenetworkconfiguration,specifyingthenewbridgeandthephysicalinterfacethatisgoingtobeapartofthatbridge.Thisineffectmapsthesubnetthatisconfiguredonthephysicalinterfacetothebridge.Ifyourserverhasonlyonenetworkinterface,youcanstillenslaveit.However,youwillneedanadditionalwayofconnectingtotheserverbecauseonceyoubringyourmaininterfacedown,youwilllooseconnectivityandtroubleshootingmightbeimpossibleoveranSSHconnection.
Oncethenewbridgehasbeenconfigured,westartitinstep3.
Instep4,weinstructthekernelnottoapplyiptablerulestoanytrafficoriginatingfromthevirtualguestsconnectedtotheLinuxbridgebecausewearenotusinganyNATrules.
Withthenewinterfaceup,wecannowseethebridgeandtheenslavedphysicalinterfaceattachedtoit,instep5.
Instep6,weedittheXMLdefinitionofthekvm1instance,wherewespecifythetypeofnetworkwewouldliketouse;forthisrecipe,it'sthebridgenetwork.IfyourecallfromtheConfiguringNATforwardingnetworkrecipe,weusedthenetworktypeinsteadofbridgeandwespecifiedalibvirtnetworkname,insteadofthebridgename.
Finally,afterrestartingtheKVMinstanceinstep7,theguestOSshouldnowbeabletoreachotherinstancesthatareapartofthesamesubnetwithoutusingNAT.
ConfiguringPCIpassthroughnetworkTheKVMhypervisorsupportsdirectlyattachingPCIdevicesfromthehostOStothevirtualmachines.WecanusethisfeaturetoattachanetworkinterfacedirectlytotheguestOS,withouttheneedforusingNATorsoftwarebridges.Inthisrecipe,wearegoingtoattachaNetworkInterfaceCard(NIC)thatsupportsSR-IOVSingleRootI/OVirtualization(SR-IOV)fromthehosttotheKVMguest.SR-IOVisaspecificationthatallowsaPeripheralComponentInterconnectExpress(PCIe)devicetoappearasmultipleseparatephysicaldevicesthatcanbesharedbetweenmanyvirtualmachinesonthesamehost,bypassingthehypervisorlayer,thusachievingnativenetworkspeeds.CloudproviderssuchasAmazonAWSexposethisfeatureforitsEC2computeinstancesthroughAPIcalls.
GettingreadyInordertocompletethisrecipe,wearegoingtoneedthefollowing:
AphysicalhostwithNICthatsupportsSR-IOVA802.1QbhcapableswitchwithconnectiontothephysicalserverCPUwitheithertheIntelVT-dorAMDIOMMUextensionsLinuxhostwithlibvirtinstalled,ready-to-provisionKVMinstances
Howtodoit...TosetupanewPCIpassthroughnetworkfollowthesteps:
1. EnumeratealldevicesonthehostOS:
root@kvm:~#virshnodedev-list--tree
computer
|
+-net_lo_00_00_00_00_00_00
+-net_ovs_system_0a_c6_62_34_19_b4
+-net_virbr1_nic_52_54_00_ba_8e_2c
+-net_vnet0_fe_54_00_55_9b_d6
...
|
+-pci_0000_00_03_0
||
|+-pci_0000_03_00_0
|||
||+-net_eth0_58_20_b1_00_b8_61
||
|+-pci_0000_03_00_1
||
|+-net_eth1_58_20_b1_00_b8_61
|
...
root@kvm:~#
2. ListallPCIEthernetadapters:
root@kvm:~#lspci|grepEthernet
03:00.0Ethernetcontroller:IntelCorporation82599ES10-Gigabit
SFI/SFP+NetworkConnection(rev01)
03:00.1Ethernetcontroller:IntelCorporation82599ES10-Gigabit
SFI/SFP+NetworkConnection(rev01)
root@kvm:~#
3. ObtainmoreinformationaboutNICthattheeth1deviceisusing:
root@kvm:~#virshnodedev-dumpxmlpci_0000_03_00_1
<device>
<name>pci_0000_03_00_1</name>
<path>/sys/devices/pci0000:00/0000:00:03.0/0000:03:00.1</path>
<parent>pci_0000_00_03_0</parent>
<driver>
<name>ixgbe</name>
</driver>
<capabilitytype='pci'>
<domain>0</domain>
<bus>3</bus>
<slot>0</slot>
<function>1</function>
<productid='0x10fb'>82599ES10-GigabitSFI/SFP+Network
Connection</product>
<vendorid='0x8086'>IntelCorporation</vendor>
</capability>
</device>
root@kvm:~#
4. Convertthedomain,bus,slot,andfunctionvaluestohexadecimal:
root@kvm:~#printf%x0
0
root@kvm:~#printf%x3
3
root@kvm:~#printf%x0
0
root@kvm:~#printf%x1
1
root@kvm:~#
5. Createanewlibvirtnetworkdefinitionfile:
root@kvm:~#catpassthrough_net.xml
<network>
<name>passthrough_net</name>
<forwardmode='hostdev'managed='yes'>
<pfdev='eth1'/>
</forward>
</network>
root@kvm:~#
6. Define,start,andenableautostartingonthenewlibvirtnetwork:
root@kvm:~#virshnet-definepassthrough_net.xml
Networkpassthrough_netdefinedfrompassthrough_net.xml
root@kvm:~#virshnet-startpassthrough_net
Networkpassthrough_nettstarted
root@kvm:~#virshnet-autostartpassthrough_net
Networkpassthrough_netmarkedasautostarted
root@kvm:~#virshnet-list
NameStateAutostartPersistent
----------------------------------------------------------
defaultactiveyesyes
passthrough_netactiveyesyes
root@kvm:~#
7. EdittheXMLdefinitionfortheKVMguest:
root@kvm:~#virsheditkvm1
...
<devices>
...
<interfacetype='hostdev'managed='yes'>
<source>
<addresstype='pci'domain='0x0'bus='0x00'slot='0x07'
function='0x0'/>
</source>
<virtualporttype='802.1Qbh'/>
</interface>
<interfacetype='network'>
<sourcenetwork='passthrough_net'>
</interface>
...
</devices>
...
Domainkvm1XMLconfigurationedited.
root@kvm:~#
8. RestarttheKVMinstance:
root@kvm:~#virshdestroykvm1
Domainkvm1destroyed
root@kvm:~#virshstartkvm1
Domainkvm1started
root@kvm:~#
9. ListtheVirtualFunctions(VFs)providedbySR-IOVNIC:
root@kvm:~#virshnet-dumpxmlpassthrough_net
<networkconnections='1'>
<name>passthrough_net</name>
<uuid>a4233231-d353-a112-3422-3451ac78623a</uuid>
<forwardmode='hostdev'managed='yes'>
<pfdev='eth1'/>
<addresstype='pci'domain='0x0000'bus='0x02'slot='0x10'
function='0x1'/>
<addresstype='pci'domain='0x0000'bus='0x02'slot='0x10'
function='0x3'/>
<addresstype='pci'domain='0x0000'bus='0x02'slot='0x10'
function='0x5'/>
<addresstype='pci'domain='0x0000'bus='0x02'slot='0x10'
function='0x7'/>
<addresstype='pci'domain='0x0000'bus='0x02'slot='0x11'
function='0x1'/>
<addresstype='pci'domain='0x0000'bus='0x02'slot='0x11'
function='0x3'/>
<addresstype='pci'domain='0x0000'bus='0x02'slot='0x11'
function='0x5'/>
</forward>
</network>
root@kvm:~#
Howitworks...InordertodirectlyattachPCINICfromthehostOStotheguestVM,wefirstneedtogathersomehardwareinformationaboutthedevice,suchasdomain,bus,slot,andfunctionIDs.Instep1,wecollectinformationaboutallavailabledevicesonthehostserver.Weareinterestedinusingtheeth1networkinterfaceforthisexample;therefore,wenotedowntheuniquePCIidentificationfromtheoutput--pci_0000_03_00_1inthiscase.
ToconfirmthisisindeedNICwewouldliketoexposetotheguest,welistallPCIdevicesinstep2.Fromtheoutput,wecanseethatthePCIIDisthesame03:00.1.
UsingthePCIIDfromstep1,weproceedtocollectmoreinformationaboutNICinstep3.Notethat0000_03_00_1IDisbrokendownintodomainID,busID,slotID,andfunctionID,asshownbytheXMLattributes.WearegoingtousethoseIDsinstep7;however,weneedtoconvertthemtohexadecimalsfirst,whichwedoinstep4.
Insteps5and6,wedefineanewlibvirtnetworkforourguest,startthenetwork,andenableautostartingincasethehostserverrestarts.Ifyoucompletedtheotherrecipesinthischapter,youshouldbealreadyfamiliarwithmostoftheattributesintheXMLdefinitionfileforthenetworkwejustcreated.Thehostdevmodedefinedinthe<forward>attributeiswhatinstructslibvirtthatthenewnetworkisgoingtousePCIpassthrough.Themanaged=yesparameter,asspecifiedinthe<forward>attribute,tellslibvirttofirstdetachthePCIdevicefromthehostbeforepassingitontotheguestandreattachingitbacktothehostaftertheguestterminates.Finally,the<pf>subelementspecifiesthephysicalinterfacethatwillbevirtualizedandpresentedtotheguest.
FormoreinformationontheavailableXMLattributes,pleaserefertohttp://libvirt.org/formatdomain.html.
Instep7,weedittheXMLdefinitionoftheKVMinstance,specifyingthePCIIDsweobtainedinstep3anddefinedaninterfacethatwillusethenewPCIpassthroughnetworkwecreatedinsteps5and6.
passthroughnetworkwecreatedinsteps5and6.
WerestarttheKVMinstanceinstep8andfinallyverifythatthephysicalPCINICdeviceisnowpartofthenewpassthroughnetworkwedefinedearlier.NotethepresenceofmultiplePCItypedevices.ThisisbecausethePCIpassthroughdeviceweareusingsupportsSR-IOV.AllKVMgueststhatwillusethisnetworkwillnowbeabletodirectlyusethehostNICbyassigningoneofthelistedvirtualPCIdevices.
ManipulatingnetworkinterfacesLibvirtprovidesahandywaytomanagenetworkinterfacesonthehostthroughthealreadyfamiliarXMLdefinitionsyntax.Wecanusethevirshcommandtodefine,provision,anddeleteLinuxbridgesandobtainmoreinformationaboutexistingnetworkinterfaces,asyou'vealreadyseeninthischapter.
Inthisrecipe,wearegoingtodefineanewLinuxbridge,createit,andfinallyremoveitusingvirsh.Ifyourecallfromearlierrecipes,wecanmanipulatetheLinuxbridgethroughutilitiessuchasbrctl.Withlibvirt,however,wehaveawaytocontrolthisprogrammaticallybywritingthedefinitionfileandusingtheAPIbindings,aswe'llseeinChapter7,UsingPythontoBuildandManageKVMInstances.
GettingreadyForthisrecipe,wearegoingtoneedthefollowing:
ThelibvirtpackageinstalledonthehostALinuxhostwiththebridgekernelmodule
Howtodoit...Tocreateanewbridgeinterfaceusinglibvirt,runthefollowingcommands:
1. Createanewbridgeinterfaceconfigurationfile:
root@kvm:~#cattest_bridge.xml
<interfacetype='bridge'name='test_bridge'>
<startmode="onboot"/>
<protocolfamily='ipv4'>
<ipaddress='192.168.1.100'prefix='24'/>
</protocol>
<bridge>
<interfacetype='ethernet'name='vnet0'>
<macaddress='fe:54:00:55:9b:d6'/>
</interface>
</bridge>
</interface>
root@kvm:~#
2. Definethenewinterface:
root@kvm:~#virshiface-definetest_bridge.xml
Interfacetest_bridgedefinedfromtest_bridge.xml
root@kvm:~#
3. Listallinterfaceslibvirtknowsabout:
root@kvm:~#virshiface-list--all
NameStateMACAddress
---------------------------------------------------
bond0active58:20:b1:00:b8:61
bond0.129activebc:76:4e:20:10:6b
bond0.229activebc:76:4e:20:17:7e
eth0active58:20:b1:00:b8:61
eth1active58:20:b1:00:b8:61
loactive00:00:00:00:00:00
test_bridgeinactive
root@kvm:~#
4. Startthenewbridgeinterface:
root@kvm:~#virshiface-starttest_bridge
Interfacetest_bridgestarted
root@kvm:~#virshiface-list--all|greptest_bridge
test_bridgeactive4a:1e:48:e1:e7:de
root@kvm:~#
5. Listallbridgedevicesonthehost:
root@kvm:~#brctlshow
bridgenamebridgeidSTPenabledinterfaces
test_bridge8000.000000000000no
virbr08000.000000000000yes
virbr18000.525400ba8e2cyesvirbr1-nic
vnet0
root@kvm:~#
6. Checktheactivenetworkconfigurationofthenewbridge:
root@kvm:~#ipastest_bridge
46:test_bridge:<BROADCAST,MULTICAST,UP,LOWER_UP>mtu1500qdisc
noqueuestateUNKNOWNgroupdefault
link/ether4a:1e:48:e1:e7:debrdff:ff:ff:ff:ff:ff
inet192.168.1.100/24brd192.168.1.255scopeglobaltest_bridge
valid_lftforeverpreferred_lftforever
inet6fe80::481e:48ff:fee1:e7de/64scopelink
valid_lftforeverpreferred_lftforever
root@kvm:~#
7. ObtaintheMACaddressofbridge:
root@kvm:~#virshiface-mactest_bridge
4a:1e:48:e1:e7:de
root@kvm:~#
8. ObtainthenameofthebridgebasedbyprovidingitsMACaddress:
root@kvm:~#virshiface-name4a:1e:48:e1:e7:de
test_bridge
root@kvm:~#
9. Destroytheinterface,asfollows:
root@kvm:~#virshiface-destroytest_bridge
Interfacetest_bridgedestroyed
root@kvm:~#virshiface-list--all|greptest_bridge
test_bridgeinactive
root@kvm:~#virshiface-undefinetest_bridge
Interfacetest_bridgeundefined
root@kvm:~#virshiface-list--all|greptest_bridge
root@kvm:~#
Howitworks...Instep1,wewritetheXMLdefinitionforthenewnetworkinterface.Wespecifybridgeasthetype,anIPaddressfortheinterface,andoptionallyaMACaddress.
Insteps2and3,wedefinethenewbridgeinterfaceandlistit.Defininganinterfacedoesnotautomaticallymakeitactive,soweactivateitinstep4.
Activatingthebridgecreatestheactualinterfaceonthehost,asshowninstep5.
Instep6,weconfirmthattheIPandMACaddressassignedtothebridgeareindeedwhatwespecifiedinstep1.
Insteps7and8,weobtainboththenameandMACaddressusingthevirshutilityandfinally,instep9,weremovethebridgeinterface.
MigratingKVMInstancesInthischapter,wearegoingtodemonstratethefollowinglibvirtKVMmigrationconcepts:
ManualofflinemigrationusinganiSCSIstoragepoolManualofflinemigrationusingGlusterFSsharedvolumesOnlinemigrationusingthevirshcommandwithsharedstorageOfflinemigrationusingthevirshcommandandlocalimageOnlinemigrationusingthevirshcommandandlocalimage
IntroductionMigratingKVMinstancesistheprocessofsendingthestateoftheguestvirtualmachine'smemory,CPU,andvirtualizeddevicesattachedtoit,toadifferentserver.MigratingKVMinstancesisasomewhatcomplicatedprocess,dependingonwhatbackendstoragetheVMisusing(thatis,directory,imagefile,iSCSIvolume,sharedstorage,orstoragepools),thenetworkinfrastructure,andthenumberofblockdevicesattachedtotheguest.Therearefollowingthetwotypesofmigrationsasfaraslibvirtisconcerned:
Offlinemigrationinvolvesdowntimefortheinstance.ItworksbyfirstsuspendingtheguestVM,thencopyinganimageoftheguestmemorytothedestinationhypervisor.TheKVMmachineisthenresumedonthetargethost.IfthefilesystemoftheVMisnotonasharedstorage,thenitneedstobemovedtothetargetserveraswell.Livemigrationworksbymovingtheinstanceinitscurrentstatewithnoperceiveddowntime,preservingthememoryandCPUregisterstates.
Broadlyspeaking,theofflinemigrationinvolvesthefollowing:
StoppingtheinstanceDumpingitsXMLdefinitiontoafileCopyingtheguestfilesystemimagetothedestinationserver(ifnotusingsharedstorage)Definingtheinstanceonthedestinationhostandstartingit
Incontrast,theonlinemigrationrequiressharedstorage,suchasNFSorGlusterFS,removingtheneedtotransfertheguestfilesystemtothetargetserver.Thespeedofthemigrationdependsonhowoftenthememoryofthesourceinstanceisbeingupdated/writtento,thesizeofthememory,andtheavailablenetworkbandwidthbetweenthesourceandtargethosts.
Livemigrationfollowsthisprocess:
TheoriginalVMcontinuestorunwhilethecontentofitsmemoryisbeingtransferredtothetargethost
Libvirtmonitorsforanychangesinthealreadytransferredmemorypages,andiftheyhavebeenupdated,itretransmitsthemOncethememorycontenthasbeentransferredtothedestinationhost,theoriginalinstanceissuspendedandthenewinstanceonthetargethostisresumed
Inthischapter,wearegoingtoperformofflineandlivemigrationsusingiSCSIandGlusterFSwiththehelpofstoragepools.
ManualofflinemigrationusinganiSCSIstoragepoolInthisrecipe,wearegoingtosetupaniSCSItarget,configureastoragepoolforit,andcreateanewKVMinstanceusingtheattachediSCSIblockdeviceasitsbackendvolume.Then,wearegoingtoperformamanualofflinemigrationoftheinstancetoanewhost.
GettingreadyForthisrecipe,wearegoingtoneedthefollowing:
Twoserverswithlibvirtandqemuinstalledandconfigured,namedkvm1andkvm2.ThetwohostsmustbeabletoconnecttoeachotherusingSSHkeysandshorthostname.AserverwithanavailableblockdevicethatwillbeexportedasaniSCSItargetandreachablefrombothlibvirtservers.Ifablockdeviceisnotavailable,pleaserefertotheThere'smore...sectioninthisrecipeforinstructionsonhowtocreateoneusingaregularfile.ThenameoftheiSCSItargetserverinthisrecipeisiscsi_target.ConnectivitytoaLinuxrepositorytoinstalltheguestOS.
Howtodoit...ToperformamanualofflinemigrationofaKVMguestusinganiSCSIstoragepool,followthesesteps:
1. OntheiSCSItargethost,installtheiscsitargetpackageandkernelmodulepackage:
root@iscsi_target:~#apt-getupdate&&apt-getinstalliscsitarget
iscsitarget-dkms
2. Enablethetargetfunctionality:
root@iscsi_target:~#sed-i
's/ISCSITARGET_ENABLE=false/ISCSITARGET_ENABLE=true/g'
/etc/default/iscsitarget
root@iscsi_target:~#cat/etc/default/iscsitarget
ISCSITARGET_ENABLE=true
ISCSITARGET_MAX_SLEEP=3
#ietdoptions
#Seeietd(8)fordetails
ISCSITARGET_OPTIONS=""
root@iscsi_target:~#
3. ConfiguretheblockdevicetoexportwithiSCSI:
root@iscsi_target:~#cat/etc/iet/ietd.conf
Targetiqn.2001-04.com.example:kvm
Lun0Path=/dev/loop1,Type=fileio
Aliaskvm_lun
root@iscsi_target:~#
Replacethe/dev/loop1devicewiththeblockdeviceyouareexportingwithiSCSI.
4. RestarttheiSCSItargetservice:
root@iscsi_target:~#/etc/init.d/iscsitargetrestart
*RemovingiSCSIenterprisetargetdevices:[OK]
*StoppingiSCSIenterprisetargetservice:[OK]
*RemovingiSCSIenterprisetargetmodules:[OK]
*StartingiSCSIenterprisetargetservice[OK]
root@iscsi_target:~#
5. Onbothlibvirthosts,installtheiSCSIinitiator:
root@kvm1/2:~#apt-getupdate&&apt-getinstallopen-iscsi
6. Onbothlibvirtservers,enabletheiSCSIinitiatorserviceandstartit:
root@kvm1/2:~#sed-i's/node.startup=manual/node.startup=
automatic/g'/etc/iscsi/iscsid.conf
root@kvm1/2:~#/etc/init.d/open-iscsirestart
7. Frombothlibvirtinitiatorhosts,listwhatiSCSIvolumesareavailablebyqueryingtheiSCSItargetserver:
root@kvm1/2:~#iscsiadm-mdiscovery-tsendtargets-piscsi_target
10.184.226.74:3260,1iqn.2001-04.com.example:kvm
172.99.88.246:3260,1iqn.2001-04.com.example:kvm
192.168.122.1:3260,1iqn.2001-04.com.example:kvm
root@kvm:~#
8. Ononeofthelibvirtservers,createanewiSCSIstoragepool:
root@kvm1:~#catiscsi_pool.xml
<pooltype="iscsi">
<name>iscsi_pool</name>
<source>
<hostname="iscsi_target.example.com"/>
<devicepath="iqn.2001-04.com.example:kvm"/>
</source>
<target>
<path>/dev/disk/by-path</path>
</target>
</pool>
root@kvm1:~#virshpool-defineiscsi_pool.xml
Pooliscsi_pooldefinedfromiscsi_pool.xml
root@kvm1:~#virshpool-list--all
NameStateAutostart
-------------------------------------------
iscsi_poolinactiveno
root@kvm1:~#
MakesuretoreplacethehostnameoftheiSCSItargetserverwithwhatisappropriateforyourenvironment.BothahostnameandanIPaddresscanbeusedwhenspecifyingtheiSCSItargethost.
9. StartthenewiSCISpool:
root@kvm1:~#virshpool-startiscsi_pool
Pooliscsi_poolstarted
root@kvm1:~#virshpool-list--all
NameStateAutostart
-------------------------------------------
iscsi_poolactiveno
root@kvm1:~#
10. ListtheavailableiSCSIvolumesfromthepoolandobtainmoreinformationonit:
root@kvm1:~#virshvol-list--pooliscsi_pool
NamePath
------------------------------------------------------------------------
------
unit:0:0:0/dev/disk/by-path/ip-10.184.22.74:3260-iscsi-iqn.2001-
04.com.example:kvm-lun-0
root@kvm1:~#virshvol-infounit:0:0:0--pooliscsi_pool
Name:unit:0:0:0
Type:block
Capacity:10.00GiB
Allocation:10.00GiB
root@kvm1:~#
11. ListtheiSCSIsessionandtheassociatedblockdevices:
root@kvm1:~#iscsiadm-msession
tcp:[5]10.184.226.74:3260,1iqn.2001-04.com.example:kvm
root@kvm1:~#ls-la/dev/disk/by-path/
total0
drwxr-xr-x2rootroot100Apr1216:24.
drwxr-xr-x6rootroot120Mar2122:14..
lrwxrwxrwx1rootroot9Apr1216:24ip-10.184.22.74:3260-iscsi-
iqn.2001-04.com.example:kvm-lun-0->../../sdf
root@kvm1:~#
12. ExaminethepartitionschemeoftheiSCSIblockdevice:
root@kvm1:~#fdisk-l/dev/disk/by-path/ip-10.184.22.74\:3260-iscsi-
iqn.2001-04.com.example\:kvm-lun-0
Disk/dev/disk/by-path/ip-10.184.22.74:3260-iscsi-iqn.2001-
04.com.example:kvm-lun-0:10.7GB,10737418240bytes
64heads,32sectors/track,10240cylinders,total20971520sectors
Units=sectorsof1*512=512bytes
Sectorsize(logical/physical):512bytes/512bytes
I/Osize(minimum/optimal):512bytes/512bytes
Diskidentifier:0x00000000
Disk/dev/disk/by-path/ip-10.184.22.74:3260-iscsi-iqn.2001-
04.com.example:kvm-lun-0doesn'tcontainavalidpartitiontable
root@kvm1:~#
13. InstallanewKVMguestusingtheiSCSIvolumeandpool:
root@kvm1:~#virt-install--nameiscsi_kvm--ram1024--extra-args="text
console=tty0utf8console=ttyS0,115200"--graphicsvnc,listen=0.0.0.0--
hvm--
location=http://ftp.us.debian.org/debian/dists/stable/main/installer-
amd64/--diskvol=iscsi_pool/unit:0:0:0
Startinginstall...
RetrievingfileMANIFEST...|3.3kB00:00...
Retrievingfilelinux...
...
root@kvm1:~#virshconsoleiscsi_kvm
...
Requestingsystemreboot
[305.315002]reboot:Restartingsystem
root@kvm1:~#
14. Refreshthepartitiontablelistandexaminethenewblockdevicesaftertheinstallation:
root@kvm1:~#partprobe
root@kvm1:~#ls-la/dev/disk/by-path/
total0
drwxr-xr-x2rootroot160Apr1216:36.
drwxr-xr-x6rootroot120Mar2122:14..
lrwxrwxrwx1rootroot9Apr1216:36ip-10.184.22.74:3260-iscsi-
iqn.2001-04.com.example:kvm-lun-0->../../sdf
lrwxrwxrwx1rootroot10Apr1216:36ip-10.184.22.74:3260-iscsi-
iqn.2001-04.com.example:kvm-lun-0-part1->../../sdf1
lrwxrwxrwx1rootroot10Apr1216:36ip-10.184.22.74:3260-iscsi-
iqn.2001-04.com.example:kvm-lun-0-part2->../../sdf2
lrwxrwxrwx1rootroot10Apr1216:36ip-10.184.22.74:3260-iscsi-
iqn.2001-04.com.example:kvm-lun-0-part5->../../sdf5
root@kvm1:~#fdisk-l/dev/sdf
Disk/dev/sdf:10.7GB,10737418240bytes
255heads,63sectors/track,1305cylinders,total20971520sectors
Units=sectorsof1*512=512bytes
Sectorsize(logical/physical):512bytes/512bytes
I/Osize(minimum/optimal):512bytes/512bytes
Diskidentifier:0x37eb1540
DeviceBootStartEndBlocksIdSystem
/dev/sdf1*2048200130551000550483Linux
/dev/sdf220015102209694714771855Extended
/dev/sdf5200151042096947147718482Linuxswap/Solaris
root@kvm1:~#
15. StartthenewKVMguestandensurethatit'srunning,andthatyoucanconnecttoitusingaVNCclient:
root@kvm1:~#virshstartiscsi_kvm
Domainiscsi_kvmstarted
root@kvm1:~#virshlist--all
IdNameState
----------------------------------------------------
19iscsi_kvmrunning
root@kvm1:~#
16. Tomanuallymigratetheinstancetoanewhost,firststoptheVMandtheiSCSIpool:
root@kvm1:~#virshdestroyiscsi_kvm
Domainiscsi_kvmdestroyed
root@kvm1:~#virshpool-destroyiscsi_pool
Pooliscsi_pooldestroyed
root@kvm1:~#iscsiadm-msession
iscsiadm:Noactivesessions.
root@kvm1:~#
17. DumptheXMLconfigurationoftheKVMinstancetoafileandexamineit:
root@kvm1:~#virshdumpxmliscsi_kvm>iscsi_kvm.xml
root@kvm1:~#catiscsi_kvm.xml
<domaintype='kvm'>
<name>iscsi_kvm</name>
<uuid>306e05ed-e398-ef33-d6e2-3708e90b89a6</uuid>
<memoryunit='KiB'>1048576</memory>
<currentMemoryunit='KiB'>1048576</currentMemory>
<vcpuplacement='static'>1</vcpu>
<os>
<typearch='x86_64'machine='pc-i440fx-trusty'>hvm</type>
<bootdev='hd'/>
</os>
<features>
<acpi/>
<apic/>
<pae/>
</features>
<clockoffset='utc'/>
<on_poweroff>destroy</on_poweroff>
<on_reboot>restart</on_reboot>
<on_crash>restart</on_crash>
<devices>
<emulator>/usr/bin/qemu-system-x86_64</emulator>
<disktype='block'device='disk'>
<drivername='qemu'type='raw'/>
<sourcedev='/dev/disk/by-path/ip-10.184.22.74:3260-iscsi-iqn.2001-
04.com.example:kvm-lun-0'/>
<targetdev='hda'bus='ide'/>
<addresstype='drive'controller='0'bus='0'target='0'unit='0'/>
</disk>
<controllertype='usb'index='0'>
<addresstype='pci'domain='0x0000'bus='0x00'slot='0x01'
function='0x2'/>
</controller>
<controllertype='pci'index='0'model='pci-root'/>
<controllertype='ide'index='0'>
<addresstype='pci'domain='0x0000'bus='0x00'slot='0x01'
function='0x1'/>
</controller>
<interfacetype='network'>
<macaddress='52:54:00:8b:b8:e3'/>
<sourcenetwork='default'/>
<modeltype='rtl8139'/>
<addresstype='pci'domain='0x0000'bus='0x00'slot='0x03'
function='0x0'/>
</interface>
<serialtype='pty'>
<targetport='0'/>
</serial>
<consoletype='pty'>
<targettype='serial'port='0'/>
</console>
<inputtype='mouse'bus='ps2'/>
<inputtype='keyboard'bus='ps2'/>
<graphicstype='vnc'port='-1'autoport='yes'listen='0.0.0.0'>
<listentype='address'address='0.0.0.0'/>
</graphics>
<video>
<modeltype='cirrus'vram='9216'heads='1'/>
<addresstype='pci'domain='0x0000'bus='0x00'slot='0x02'
function='0x0'/>
</video>
<memballoonmodel='virtio'>
<addresstype='pci'domain='0x0000'bus='0x00'slot='0x04'
function='0x0'/>
</memballoon>
</devices>
</domain>
root@kvm1:~#
18. RemotelycreatetheiSCSIstoragepoolfromthekvm1hosttothekvm2host:
root@kvm1:~#virsh--connectqemu+ssh://kvm2/systempool-define
iscsi_pool.xml
Pooliscsi_pooldefinedfromiscsi_pool.xml
root@kvm1:~#
IfyouarenotusingkeysfortheSSHconnectionbetweenboththeKVMhosts,youwillbeaskedtoprovideapasswordbeforethelibvirtcommandcanproceed.WerecommendthatyouuseSSHkeysonthelibvirthostsyouaremigratingbetween.
19. RemotelystarttheiSCSIpoolonthekvm2serverandensurethatit'srunning:
root@kvm1:~#virsh--connectqemu+ssh://kvm2/systempool-start
iscsi_pool
Pooliscsi_poolstarted
root@kvm1:~#virsh--connectqemu+ssh://kvm2/systempool-list--all
NameStateAutostart
-------------------------------------------
iscsi_poolactiveno
root@kvm1:~#
YoucanalsoSSHtothekvm2serverandperformallofthepoolandvolumeoperationslocally.Wedoitremotelytodemonstratetheconcept.
20. RemotelylisttheavailableiSCSIvolumesonthekvm2nodefromthesourcehost:
root@kvm1:~#virsh--connectqemu+ssh://kvm2/systemvol-list--pool
iscsi_pool
NamePath
--------------------------------------------------------------------
unit:0:0:0/dev/disk/by-path/ip-10.184.22.74:3260-iscsi-iqn.2001-
04.com.example:kvm-lun-0
root@kvm1:~#
21. SSHtothesecondKVMserverandensurethattheiSCSIblockdevicesarenowavailableonthehostOS:
root@kvm2:~#iscsiadm-msession
tcp:[3]10.184.226.74:3260,1iqn.2001-04.com.example:kvm
root@kvm2:~#ls-la/dev/disk/by-path/
total0
drwxr-xr-x2rootroot120Apr1217:44.
drwxr-xr-x6rootroot120Apr1217:44..
lrwxrwxrwx1rootroot9Apr1217:44ip-10.184.22.74:3260-iscsi-
iqn.2001-04.com.example:kvm-lun-0->../../sdc
lrwxrwxrwx1rootroot10Apr1217:44ip-10.184.22.74:3260-iscsi-
iqn.2001-04.com.example:kvm-lun-0-part1->../../sdc1
lrwxrwxrwx1rootroot10Apr1217:44ip-10.184.22.74:3260-iscsi-
iqn.2001-04.com.example:kvm-lun-0-part2->../../sdc2
lrwxrwxrwx1rootroot10Apr1217:44ip-10.184.22.74:3260-iscsi-
iqn.2001-04.com.example:kvm-lun-0-part5->../../sdc5
root@kvm2:~#
22. CompletethemigrationbyremotelydefiningtheKVMinstanceandstartingitonthetargethost:
root@kvm1:~#virsh--connectqemu+ssh://kvm2/systemdefineiscsi_kvm.xml
Domainiscsi_kvmdefinedfromiscsi_kvm.xml
root@kvm1:~#virsh--connectqemu+ssh://kvm2/systemlist--all
IdNameState
----------------------------------------------------
-iscsi_kvmshutoff
root@kvm:~#virsh--connectqemu+ssh://kvm2/systemstartiscsi_kvm
Domainiscsi_kvmstarted
root@kvm:~#virsh--connectqemu+ssh://kvm2/systemlist--all
IdNameState
----------------------------------------------------
3iscsi_kvmrunning
root@kvm1:~#
Howitworks...Inthisrecipe,wedemonstratedhowtomanuallyperformanofflinemigrationofaKVMinstancefromonehosttoanother,usinganiSCSIpool.IntheOnlinemigrationusingthevirshcommandrecipelaterinthischapter,wearegoingtoperformalivemigrationusingthesameiSCSIpoolandinstancewecreatedinthisrecipe,usingthevirshcommand,thusavoidingdowntimefortheinstance.
Let'sstepthroughtheprocessandexploreinmoredetailhowthemanualofflinemigrationwasaccomplished.
WestartwiththeserverthatisgoingtobepresentingtheiSCSItargetbyfirstinstallingtherequirediSCSItargetserverpackagesinstep1.Instep2,weenabletheiSCSItargetfunctionality,enablingtheservertoexportblockdevicesviatheiSCSIprotocol.Instep3,wespecifyanidentified(iSCSIqualifiedname)iqn.2001-04.com.example:kvmfortheiSCSItargetdevicethattheinitiatorsaregoingtouse.Weareusingthe/dev/loop1blockdeviceforthisexample.TheiSCSI-qualifiednamehastheformatiqn.yyyy-mm.naming-authority:uniquenamewhere:
iqn:ThisistheiSCSI-qualifiednameidentifieryyyy-mm:Thisistheyearandmonthwhenthenamingauthoritywasestablishednaming-authority:ThisisusuallyreversesyntaxoftheInternetdomainnameofthenamingauthorityorthedomainnameoftheserverUniquename:Thisisanynameyouwouldliketouse
FormoreinformationaboutiSCSIandthenamingschemeituses,pleaserefertohttps://en.wikipedia.org/wiki/ISCSI.
Withthetargetdefinitioninplace,instep4,werestarttheiSCSIserviceontheserver.
Insteps5and6,weinstallandconfiguretheiSCSIinitiatorserviceonbothKVMnodes,andinstep7,werequestallavailableiSCSItargets.Insteps8and
9,wedefineandstartanewiSCSI-basedstoragepool.Thesyntaxofthestoragepooldefinitionshouldlookfamiliarifyou'vecompletedtheWorkingwithstoragepoolsrecipefromChapter2,UsinglibvirttoManageKVM.
AftercreatingtheiSCSIstoragepool,weproceededtolistthevolumespartofthatpoolinstep10.Notethatwhenwestartedthepool,itloggedtheiSCSItargetin,resultinginanewblockdevicepresentinthe/dev/disk/by-path/directory,aswecanfurtherseeinstep11.WecannowusethisblockdevicelocallytoinstallanewLinuxOS.Instep12,wecanseethattheiSCSIblockdevicepresentedtothehostOSdoesnotyetcontainanypartitions.
Withthenewblockdevicepresent,weproceedtobuildanewKVMinstanceinstep13,specifyingthestoragepoolandvolumeasthetargetfortheinstallation.AftertheguestOSinstallationcompletes,wecannowseethattherearemultiplepartitionsontheiSCSIblockdeviceinstep14.Wethenproceedtostartthenewguestinstep15.
NowthatwehavearunningKVMinstanceusinganiSCSIblockdevice,wecanproceedwiththeofflinemanualmigrationfromthekvm1hoststothekvm2hosts.
WestartthemigrationprocessbyfirststoppingtherunningKVMinstanceandtheassociatedstoragepoolinstep16.WethendumptheXMLconfigurationoftheKVMguesttoafileinstep17.Wearegoingtouseittodefinetheguestonthetargetserver.Wehaveafewoptionsforthis:wecancopythefileovertothetargetserveranddefinetheinstancethereorwecandothatremotelyfromtheoriginalhost.
Insteps18and19,wecreatetheiSCSIstoragepoolremotelyfromtheoriginalhosttothetargethost.Wecouldhaveloggedintothetargethostandperformedthesameoperationslocallyaswellwiththesameresult.Thepointhereisthatwecanusetheqemu+sshconnectionstringtoremotelyconnecttootherqemuinstancesoverSSH.Insteps20and21,weensurethatthesameiSCSIvolumehasbeensuccessfullyloggedintothetargethost.
Finally,instep22,wedefinetheinstanceonthetargethostusingtheXMLconfigurationwedumpedinstep17andthenstartit.BecauseweareusingthesameXMLdefinitionfileandthesameiSCSIblockdevicecontainingtheguestOSfilesystem,wenowhaveexactlythesameinstancecreatedonthenewserver,thuscompletingtheofflinemigration.
server,thuscompletingtheofflinemigration.
There'smore...IftheiSCSItargetserverdoesnothaveanyavailableblockdevicestoexport,wecancreateanewblockdeviceusingaregularfilebyfollowingthestepsoutlinedhere:
1. Createanewimagefileofagivensize:
root@iscsi_target:~#truncate--size10Gxvdb.img
root@iscsi_target:~#file-sxvdb.img
xvdb.img:data
root@kvmiscsi_target:~#qemu-imginfoxvdb.img
image:xvdb.img
fileformat:raw
virtualsize:10G(10737418240bytes)
disksize:0
root@iscsi_target:~#
2. Ensurethattheloopkernelmoduleiscompiledin(orloaditwithmodprobeloop)andfindthefirstavailableloopdevicetouse:
root@iscsi_target:~#grep'loop'/lib/modules/`uname-r`/modules.builtin
kernel/drivers/block/loop.ko
root@iscsi_target:~#losetup--find
/dev/loop0
root@iscsi_target:~#
3. Associatetherawfileimagewiththefirstavailableloopdevice:
root@iscsi_target:~#losetup/dev/loop0xvdb.img
root@iscsi_target:~#losetup--all
/dev/loop0:[10300]:263347(/root/xvdb.img)
root@iscsi_target:~#
Instep1,wecreateanewimagefileusingthetruncatecommand.
Instep2,welistthefirstavailableblockdevicetouseandinstep3,weassociateitwiththerawimagefilewecreatedinstep1.Theresultisanewblockdeviceavailableas/dev/loop0thatwecanusetoexportiniSCSI.
ManualofflinemigrationusingGlusterFSsharedvolumesIntheManualofflinemigrationusinganiSCSIstoragepoolrecipe,wecreatedaniSCSIstoragepoolanduseditwhileperformingmanualofflinemigration.Withstoragepools,wecandelegatetheoperationofasharedstoragetolibvirtratherthanmanuallyhavingtologin/logoutiSCSItargets,forexample.Thisisespeciallyusefulwhenweperformlivemigrationswiththevirshcommand,aswearegoingtoseeinthenextrecipe.Eventhoughtheuseofstoragepoolsisnotrequired,itsimplifiesandcentralizesthemanagementofbackendvolumes.
Inthisrecipe,wearegoingtousetheGlusterFSnetworkfilesystemtodemonstrateanalternativewayofmanuallymigratingaKVMinstance,thistimenotusingstoragepools.
GlusterFShasthefollowingtwocomponents:
Servercomponent:ThisrunstheGlusterFSdaemonandexportslocalblockdevicesnamedbricksasvolumesthatcanbemountedbytheclientcomponentClientcomponent:ThisconnectstotheGlusterFSclusteroverTCP/IPandcanmounttheexportedvolumes
Therearethefollowingthreetypesofvolumes:
Distributed:ThesearevolumesthatdistributefilesthroughouttheclusterReplicated:ThesearevolumesthatreplicatedataacrosstwoormorenodesinthestorageclusterStriped:Thesearestripefilesacrossmultiplestoragenodes
Forhighavailability,wearegoingtousetwoGFSnodesusingthereplicatedvolumes(twobrickscontainingthesamedata).
GettingreadyTocompletethisrecipe,wearegoingtousethefollowing:
TwoserversthatwillhosttheGlusterFSsharedfilesystem.TwohostsrunninglibvirtandqemuthatwillbeusedtomigratetheKVMguest.Allserversshouldbeabletocommunicatewitheachotherusinghostnames.BothservershostingthesharedvolumesshouldhaveoneblockdeviceavailableforuseasGlusterFSbricks.Ifablockdeviceisnotavailable,pleaserefertotheThere'smore...sectionoftheManualofflinemigrationusinganiSCSIstoragepoolrecipeinthischapteronhowtocreateoneusingaregularfile.ConnectivitytoaLinuxrepositorytoinstalltheguestOS.
Howtodoit...TomigrateaKVMguestusingasharedGlusterFSbackendstore,runthefollowing:
1. Onbothserversthatwillhostthesharedvolumes,installGlusterFS:
root@glusterfs1/2:~#apt-getupdate&&apt-getinstallglusterfs-server
2. FromoneoftheGlusterFSnodes,probetheotherinordertoformacluster:
root@glusterfs1:~#glusterpeerstatus
peerstatus:Nopeerspresent
root@glusterfs1:~#glusterpeerprobeglusterfs2
peerprobe:success
root@glusterfs1:~#
3. VerifythattheGlusterFSnodesareawareofeachother:
root@glusterfs1:~#glusterpeerstatus
NumberofPeers:1
Hostname:glusterfs2
Port:24007
Uuid:923d152d-df3b-4dfd-9def-18dbebf2b76a
State:PeerinCluster(Connected)
root@glusterfs1:~#
4. OnbothGlusterFShosts,createafilesystemontheblockdevicesthatwillbeusedasGlusterFSbricksandmountthem:
root@glusterfs1/2:~#mkfs.ext4/dev/loop5
...
Allocatinggrouptables:done
Writinginodetables:done
Creatingjournal(32768blocks):done
Writingsuperblocksandfilesystemaccountinginformation:done
root@glusterfs1/2:~#mount/dev/loop5/mnt/
root@glusterfs1/2:~#mkdir/mnt/bricks
root@glusterfs1/2:~#
Makesuretoreplacetheblockdevicenamewithwhatisappropriateonyoursystem.
5. FromoneoftheGlusterFSnodes,createthereplicatedstoragevolume,usingthebricksfrombothserversandthenlistit:
root@glusterfs1:~#glustervolumecreatekvm_gfsreplica2transporttcp
glusterfs1:/mnt/bricks/gfs1glusterfs2:/mnt/bricks/gfs2
volumecreate:kvm_gfs:success:pleasestartthevolumetoaccessdata
root@glusterfs1:~#glustervolumelist
kvm_gfs
root@glusterfs1:~#
6. FromoneoftheGlusterFShosts,startthenewvolumeandobtainmoreinformationonit:
root@glusterfs1:~#glustervolumestartkvm_gfs
volumestart:kvm_gfs:success
root@glusterfs1:~#glustervolumeinfo
VolumeName:kvm_gfs
Type:Replicate
VolumeID:69823a48-8b1b-469f-b06a-14ef6f33a6f5
Status:Started
NumberofBricks:1x2=2
Transport-type:tcp
Bricks:
Brick1:glusterfs1:/mnt/bricks/gfs1
Brick2:glusterfs2:/mnt/bricks/gfs2
root@glusterfs1:~#
7. Onbothlibvirtnodes,installtheGlusterFSclientandmounttheGlusterFSvolumethatwillbeusedtohosttheKVMimage:
root@kvm1/2:~#apt-getupdate&&apt-getinstallglusterfs-client
root@kvm1/2:~#mkdir/tmp/kvm_gfs
root@kvm1/2:~#mount-tglusterfsglusterfs1:/kvm_gfs/tmp/kvm_gfs
root@kvm1/2:~#
WhenmountingtheGlusterFSvolume,youcanspecifyeitheroneoftheclusternodes.Intheprecedingexample,wearemountingfromtheglusterfs1node.
8. Ononeofthelibvirtnodes,buildanewKVMinstance,usingthemountedGlusterFSvolume:
root@kvm1:~#virt-install--namekvm_gfs--ram1024--extra-args="text
console=tty0utf8console=ttyS0,115200"--graphicsvnc,listen=0.0.0.0--
hvm--
location=http://ftp.us.debian.org/debian/dists/stable/main/installer-
amd64/--disk/tmp/kvm_gfs/gluster_kvm.img,size=5
...
root@kvm1:~#
9. Ensurethatbothlibvirtnodescanseetheguestimage:
root@kvm1/2:~#ls-al/tmp/kvm_gfs/
total1820300
drwxr-xr-x3rootroot4096Apr1314:48.
drwxrwxrwt6rootroot4096Apr1315:00..
-rwxr-xr-x1rootroot5368709120Apr1314:59gluster_kvm.img
root@kvm1/2:~#
10. TomanuallymigratetheKVMinstancefromonelibvirtnodetotheother,firststoptheinstanceanddumpitsXMLdefinition:
root@kvm1:~#virshdestroykvm_gfs
Domainkvm_gfsdestroyed
root@kvm1:~#virshdumpxmlkvm_gfs>kvm_gfs.xml
root@kvm1:~#
11. Fromthesourcelibvirtnode,definetheinstanceonthetargethost:
root@kvm1:~#virsh--connectqemu+ssh://kvm2/systemdefinekvm_gfs.xml
Domainkvm_gfsdefinedfromkvm_gfs.xml
root@kvm1:~#virsh--connectqemu+ssh://kvm2/systemlist--all
IdNameState
----------------------------------------------------
-kvm_gfsshutoff
root@kvm1:~#
12. StarttheKVMinstanceonthetargethosttocompletethemigration:
root@kvm2:~#virshstartkvm_gfs
Domainkvm_gfsstarted
root@kvm2:~#
WecanalsostarttheKVMinstanceonthedestinationhostfromthesourcehostusingtheqemu+sshconnectionasfollows:root@kvm1:~#virsh--connectqemu+ssh://kvm2/systemstartkvm_gfs
Howitworks...WebeginbyinstallingtheGlusterFSserver-sidepackageonbothserversinstep1.Then,instep2,weproceedtoformaclusterbysendingaprobefromthefirstGlusterFSnode.Iftheprobewassuccessful,wefurtherobtaininformationabouttheclusterinstep3.Instep4,wepreparetheblockdevicesonbothGlusterFSserversforusebycreatingafilesystemonthem,thenmountingthem.TheblockdeviceoncemountedwillcontainthebricksthatwillformavirtualreplicatedvolumeforGlusterFStoexport.
Instep5,wecreatethenewreplicatedvolumeononeofthenodes(thiswillaffecttheentireclusterandonlyneedstoberunfromoneGlusterFSnode).Wespecifythatthetypeisgoingtobereplicated,usingtheTCPprotocolandthelocationofthebrickswearegoingtouse.Oncethevolumeiscreated,westartitinstep6andgetmoreinformationaboutit.Notethatfromtheoutputofthevolumeinformation,wecanseethenumberofbricksinuseandtheirlocationinthecluster.
Instep7,weinstalltheGlusterFSclientcomponentonbothlibvirtserversandmounttheGFSvolume.BothKVMhostsnowsharethesamereplicatedstoragethatisphysicallyhostedontheGlusterFSnodes.WearegoingtousethatsharedstoragetohostthenewKVMimagefile.
Instep8,weproceedwiththeinstallationofanewKVMinstance,usingtheGlusterFSvolumethatwemountedinthepreviousstep.Oncetheinstallationiscomplete,weverifythatbothlibvirtserverscanseethenewKVMimage,instep9.
Westartthemanualmigrationinstep10byfirststoppingtherunningKVMinstance,thensavingitsconfigurationtothedisk.Instep11,weremotelydefinetheKVMguestusingtheXMLdumpandverifythatithasbeensuccessfullydefinedonthetargethost.Finally,westarttheKVMinstanceonthetargetserver,completingthemigration.
OnlinemigrationusingthevirshcommandwithsharedstorageThevirshcommandprovidesamigrateparameterthatwecanusetomigrateKVMinstancesbetweenhosts.Intheprevioustworecipes,wesawhowtomigrateinstancesmanuallywithdowntime.Inthisrecipe,wearegoingtoperformalivemigrationonaninstancethatuseseithertheiSCSIstoragepoolortheGlusterFSsharedvolumesthatweusedearlierinthischapter.
Ifyourecall,livemigrationonlyworkswhentheguestfilesystemresidesonsomesortofsharedmedia,suchasNFS,iSCSI,GlusterFS,orifwefirstcopytheimagefiletoallnodesandusethe--copy-storage-alloptionwithvirshmigrate,aswe'llseelaterinthischapter.
GettingreadyInordertocompletethisrecipe,wearegoingtoneedthefollowing:
Twolibvirthostswithasharedstoragebetweenthem.Ifyou'vecompletedtheearlierrecipesinthischapter,youcaneitherusetheiSCSIstoragepoolwecreatedandtheKVMinstancethatisusingitortheGFSsharedstoragewiththeKVMguest.Bothlibvirthostsshouldbeabletocommunicatewitheachotherusingshorthostnames.
Howtodoit...Toperformalivemigrationusingthesharedstorage,performtheoperationslistedhere:
1. EnsurethattheiSCSIKVMinstancewebuiltearlierisrunningonthesourcehost:
root@kvm1:~#virshlist--all
IdNameState
----------------------------------------------------
26iscsi_kvmrunning
root@kvm1:~#
2. Livemigratetheinstancetothesecondlibvirtserver(thetargetnodeshouldalreadyhavetheiSCSIpoolconfigured).Ifthisoperationerrorsout,pleaseconsulttheThere'smore...sectionofthisrecipefortroubleshootingtips:
root@kvm1:~#virshmigrate--liveiscsi_kvmqemu+ssh://kvm2/system
root@kvm1:~#
3. EnsurethattheKVMinstancehasbeenstoppedonthesourcehostandstartedonthetargetserver:
root@kvm1:~#virshlist--all
IdNameState
----------------------------------------------------
-iscsi_kvmshutoff
root@kvm1:~#virsh--connectqemu+ssh://kvm2/systemlist--all
IdNameState
----------------------------------------------------
10iscsi_kvmrunning
root@kvm1:~#
4. Tomigratetheinstanceback,fromthekvm2node,runthefollowing:
root@kvm2:~#virshmigrate--liveiscsi_kvmqemu+ssh://kvm1/system
root@kvm2:~#virshlist--all
IdNameState
----------------------------------------------------
root@kvm2:~#virsh--connectqemu+ssh://kvm1/systemlist--all
IdNameState
----------------------------------------------------
28iscsi_kvmrunning
root@kvm2:~#
Howitworks...WhenmigratingaKVMinstancethatisusingasharedstorage,suchastheiSCSIstoragepoolinthisexample,onceweinitiatethemigrationwiththemigrate--liveparameter,libvirttakescareofloggingouttheiSCSIsessionfromtheoriginalhostandloggingitintothetargetserver,thusmakingtheblockdevicecontainingtheguestfilesystempresentonthedestinationserverwithouttheneedtocopyallthedata.YoumighthavenotedthatthemigrationtookonlyafewsecondsbecausetheonlydatathatwasmigratedwasthememorypagesoftherunningVMonthesourcehost.
There'smore...DependingonyourLinuxdistributionandtheservertype(on-metaloracloudinstance)youarerunningthisrecipeon,youmightencounterafewcommonerrorswhentryingtomigratetheinstance.
Error:error:Unsafemigration:Migrationmayleadtodatacorruptionifdisksusecache!=none.Solution:EdittheXMLdefinitionoftheinstanceyouaretryingtomigrateandupdatethedriversectionoftheblockdevicetocontainthecache=noneattribute:
root@kvm1:~#virsheditiscsi_kvm
...
<devices>
...
<disktype='block'device='disk'>
<drivername='qemu'type='raw'cache='none'/>
<sourcedev='/dev/disk/by-path/ip-10.184.22.74:3260-iscsi-iqn.2001-
04.com.example:kvm-lun-0'/>
<targetdev='hda'bus='ide'/>
<addresstype='drive'controller='0'bus='0'target='0'unit='0'/>
</disk>
...
</devices>
Error:error:Internalerror:Attempttomigrateguesttothesamehost02000100-0300-0400-0005-000600070008.Solution:Someservers,usuallyvirtualized,mayreturnthesamesystemUUID,whichcausesthemigrationtofail.Toseeifthisisthecase,runthefollowingonboththesourceandtargetmachines:
root@kvm1/2:~#virshsysinfo|grep-B5-A3uuid
<system>
<entryname='manufacturer'>FOXCONN</entry>
<entryname='product'>CL7100</entry>
<entryname='version'>PVT1-X05</entry>
<entryname='serial'>2M2542Z069</entry>
<entryname='uuid'>02000100-0300-0400-0005-000600070008</entry>
<entryname='sku'>NULL</entry>
<entryname='family'>IntelGrantleyEP</entry>
</system>
root@kvm1/2:~#
IftheUUIDisthesameonbothservers,editthelibvirtconfigurationfileandassignauniqueUUID,thenrestartlibvirt:
root@kvm2:~#vim/etc/libvirt/libvirtd.conf
...
host_uuid="02000100-0300-0400-0006-000600070008"
...
root@kvm2:~#/etc/init.d/libvirt-binrestart
libvirt-binstop/waiting
libvirt-binstart/running,process12167
root@kvm2:~#
Error:error:Unabletoresolveaddresskvm2.localdomainservice49152:Nameorserviceisnotknown.Solution:Thisindicatesthatlibvirtisunabletoresolvethehostnameoftheinstances.Makesurethatthehostnamedoesnotresolvetolocalhostandthatyoucanping,orSSHbetweenthesourceandtargetserversusingthehostnameinsteadoftheIPaddressoftheserver.Anexampleofaworkinghostfileforbothlibvirtnodesisasfollows:
root@kvm1:~#cat/etc/hosts
127.0.0.1localhost
10.184.226.106kvm1.example.comkvm1
10.184.226.74kvm2.example.comkvm2
root@kvm1:~#
root@kvm2:~#cat/etc/hosts
127.0.0.1localhost
10.184.226.106kvm1.example.comkvm1
10.184.226.74kvm2.example.comkvm2
root@kvm2:~#
Youcanfindmoreinformationabouttheoperationofaninstancebyexaminingthefollowinglogs:
root@kvm1:~#cat/var/log/libvirt/libvirtd.log
...
2017-04-1219:26:02.297+0000:33149:error:virCommandWait:2399:internal
error:Childprocess(/usr/bin/iscsiadm--modesession)unexpectedexitstatus
21
...
root@kvm1:~#cat/var/log/libvirt/qemu/iscsi_kvm.log
...
2017-04-1317:59:48.040+0000:startingup
LC_ALL=CPATH=/usr/local/sbin:/usr/local/bin:/usr/bin:/usr/sbin:/sbin:/bin
QEMU_AUDIO_DRV=none/usr/bin/qemu-system-x86_64-nameiscsi_kvm-S-machinepc-
i440fx-trusty,accel=kvm,usb=off-m1024-realtimemlock=off-smp
1,sockets=1,cores=1,threads=1-uuid306e05ed-e398-ef33-d6e2-3708e90b89a6-no-
user-config-nodefaults-chardev
socket,id=charmonitor,path=/var/lib/libvirt/qemu/iscsi_kvm.monitor,server,nowait
-monchardev=charmonitor,id=monitor,mode=control-rtcbase=utc-no-shutdown-
bootstrict=on-devicepiix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2-drive
file=/dev/disk/by-path/ip-10.184.226.74:3260-iscsi-iqn.2001-04.com.example:kvm-
lun-0,if=none,id=drive-ide0-0-0,format=raw,cache=none-deviceide-
hd,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1-netdev
tap,fd=24,id=hostnet0-device
rtl8139,netdev=hostnet0,id=net0,mac=52:54:00:8b:b8:e3,bus=pci.0,addr=0x3-
chardevpty,id=charserial0-deviceisa-serial,chardev=charserial0,id=serial0-
vnc0.0.0.0:0-devicecirrus-vga,id=video0,bus=pci.0,addr=0x2-incomingtcp:
[::]:49153-devicevirtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x4
chardeviceredirectedto/dev/pts/0(labelcharserial0)
qemu:terminatingonsignal15frompid33148
2017-04-1318:34:49.684+0000:shuttingdown
...
root@kvm1:~#
OfflinemigrationusingthevirshcommandandlocalimagePerformingofflinemigrationwithvirshdoesnotrequireasharedstorage;however,weareresponsibleforprovidingtheguestfilesystemtothenewhost(bycopingtheimagefileandsoon).Theofflinemigrationtransferstheinstancedefinitionwithoutstartingtheguestonthedestinationhostandwithoutstoppingitonthesourcehost.Inthisrecipe,wearegoingtoperformanofflinemigrationusingthevirshcommandonarunningKVMguestusinganimagefileforitsfilesystem.
GettingreadyForthissimplerecipe,wearegoingtoneedthefollowing:
TwolibvirthostsandarunningKVMinstance.Ifoneisnotpresentonyourhost,youcaninstallandstartanewguestVMusingalocalimagefile:
root@kvm:~#virt-install--namekvm_no_sharedfs--ram1024--extra-
args="textconsole=tty0utf8console=ttyS0,115200"--graphics
vnc,listen=0.0.0.0--hvm--
location=http://ftp.us.debian.org/debian/dists/stable/main/installer-
amd64/--disk/tmp/kvm_no_sharedfs.img,size=5
Bothhostsshouldbeabletocommunicatewitheachotherusinghostnames.
Howtodoit...Toperformanofflinemigrationusingthevirshcommand,runthefollowing:
1. MakesurethatwehavearunningKVMinstance:
root@kvm1:~#virshlist--all
IdNameState
----------------------------------------------------
26kvm_no_sharedfsrunning
root@kvm1:~#
2. Migratetheinstanceusingtheofflinemode.Ifthisoperationerrorsout,pleaseconsulttheThere'smore...sectionoftheOnlinemigrationusingthevirshcommandrecipefortroubleshootingtips:
root@kvm1:~#virshmigrate--offline--persistentkvm_no_sharedfs
qemu+ssh://kvm2/system
root@kvm1:~#
3. Unlikethelivemigration,thesourceinstanceisstillrunning,andthedestinationinstanceisstopped:
root@kvm1:~#virshlist--all
IdNameState
----------------------------------------------------
29kvm_no_sharedfsrunning
root@kvm1:~#virsh--connectqemu+ssh://kvm2/systemlist--all
IdNameState
----------------------------------------------------
-kvm_no_sharedfsshutoff
root@kvm1:~#
Howitworks...Offlinemigrationsarequitesimple;thevirshcommandtransfersthedefinitionfilefromthetargethosttothedestinationanddefinestheinstance.TheoriginalKVMguestisleftrunning.Inordertostartthemigratedinstance,itsimagefileneedstobetransferredtothedestinationfirstandbepresentontheexactsamelocationastheoneonthesourceserver.ThemaindifferencewhenperforminganofflinemigrationascomparedwithjustdumpingtheXMLfileanddefiningitonthedestinationhostisthatlibvirtmakesupdatestothedestinationXMLfile,suchasassigningnewUUIDs.
Intheearlier-mentionedexample,theonlytwonewflagsweretheofflineandpersistentflags.Thepriorspecifiesanofflinetypemigration,andthelatterleavesthedomainpersistentonthedestinationhost.
OnlinemigrationusingthevirshcommandandlocalimageInthisrecipe,wearegoingtolivemigratearunninginstance,withoutsharedstorage.
GettingreadyForthisrecipe,wearegoingtoneedthefollowing:
TwolibvirtserverswitharunningKVMinstanceusingalocalimagefile.WearegoingtousetheKVMguestwebuiltinthepreviousrecipe,Offlinemigrationusingthevirshcommandandlocalimage.Bothserversmustbeabletocommunicatewitheachotherusingtheirhostnames.
Howtodoit...Tomigrateaninstancewithoutsharedstorage,usethefollowingsteps:
1. EnsurethattheKVMguestisrunning:
root@kvm1:~#virshlist--all
IdNameState
----------------------------------------------------
33kvm_no_sharedfsrunning
root@kvm1:~#
2. Findthelocationoftheimagefile:
root@kvm1:~#virshdumpxmlkvm_no_sharedfs|grep"sourcefile"
<sourcefile='/tmp/kvm_no_sharedfs.img'/>
root@kvm1:~#
3. Transfertheimagefiletothedestinationhost:
root@kvm1:~#scp/tmp/kvm_no_sharedfs.imgkvm2:/tmp/
kvm_no_sharedfs.img100%5120MB243.8MB/s00:21
root@kvm1:~#
4. Migratetheinstanceandensurethatit'srunningonthedestinationhost:
root@kvm1:~#virshmigrate--live--persistent--verbose--copy-storage-
allkvm_no_sharedfsqemu+ssh://kvm2/system
Migration:[100%]
root@kvm1:~#virshlist--all
IdNameState
----------------------------------------------------
-kvm_no_sharedfsshutoff
root@kvm1:~#virsh--connectqemu+ssh://kvm2/systemlist--all
IdNameState
----------------------------------------------------
17kvm_no_sharedfsrunning
root@kvm1:~#
5. Fromthedestinationhost,migratetheinstanceback,usingtheincrementalimagetransfer:
root@kvm2:~#virshmigrate--live--persistent--verbose--copy-storage-
inckvm_no_sharedfsqemu+ssh://kvm/system
Migration:[100%]
root@kvm2:~#
Howitworks...Afterweensurethatthesourceinstanceisinarunningstateinstep1,wetransfertheimagefiletothedestinationfile,intheexactsamelocationasthesourceinstep3.Withtheimagefileinplace,wecannowperformalivemigration,whichwedoinstep4andthenbackinstep5.
Thetwonewparameterswehaven'tusedsofarare--copy-storage-allandcopy-storage-inc.Thefirstoneinstructslibvirttotransfertheentireimagefiletothedestination,whereasthesecondperformsanincrementaltransfer,copyingonlythedatathathaschanged,reducingthetransfertime.
MonitoringandBackupofKVMVirtualMachinesInthischapter,wearegoingtocoverthefollowingtopics:
ResourceusagecollectionwithlibvirtMonitoringKVMinstanceswithSensuSimpleKVMbackupswithtarandrsyncCreatingsnapshotsListingsnapshotsInspectingsnapshotsEditingsnapshotsRevertingsnapshotsDeletingsnapshots
IntroductionItgoeswithoutsayingthatmonitoringandbackingupofproductionKVMinstancesisimportantinordertomeetuptimeService-LevelAgreements(SLAs)andtosatisfyhigh-availabilityandperformancerequirements.Monitoringandbackingupofvirtualmachinesisnotverydifferentfrommonitoringandbackingupofphysicalservers.Insomecases,it'sevenmoreconvenienttobackupasingleimagefileforVMorcreateasnapshot,ratherthanthefilesystemofanOSrunningonaphysicalserver.
Inthischapter,wearegoingtoseeexamplesonhowtogatherresourceusagemetricsforliveKVMinstancesandhowtomonitortheresourceusageandalertonpredefinedthresholdswithtoolssuchasSensu.Followingthis,wearegoingtofocusondifferentwaysofbackingupKVMguestsusingtoolssuchasrsync,andcreatingandmanagingsnapshotswiththehelpofthevirshcommand.
ResourceusagecollectionwithlibvirtThefirststepinmonitoringvirtualmachinesistogetfamiliarwiththetoolstocollectmetricsonthesubsystemswewouldliketolateralerton.Inthisrecipe,wearegoingtofocusonCPU,memory,andblockdeviceutilizationoftheKVMguests.WearealsogoingtolearnhowtousetheQEMUmonitoringsocketandtheQEMUguestagent.
Libvirtexposesasetofcallsthatthevirshcommandleveragestogatherresourceusageinformationonthespecifiedguest/domain.WearegoingtomonitorandalertontheinformationcollectedhereintheMonitoringKVMinstanceswithSensurecipelaterinthischapter.
GettingreadyForthisrecipe,wearegoingtoneedthefollowing:
AserverwithlibvirtinstalledandconfiguredArunningKVMinstance
Howtodoit...Tocollectvariousresourceusageinformationforarunninginstanceorthehypervisorhost,performthefollowing:
1. ObtaininformationonthehypervisorCPUutilization:
root@kvm:~#virshnodecpustats--percent
usage:0.0%
user:0.0%
system:0.0%
idle:100.0%
iowait:0.0%
root@kvm:~#
2. Collectinformationonthehypervisormemoryutilization:
root@kvm:~#virshnodememstats
total:131918328KiB
free:103633700KiB
buffers:195532KiB
cached:25874840KiB
root@kvm:~#
3. CheckthestateofaKVMinstance:
root@kvm:~#virshdomstatekvm1
running
root@kvm:~#
4. GetanumberofassignedvirtualCPUs(vCPU)foraKVMinstance:
root@kvm:~#virshvcpucount--currentkvm1--live
1
root@kvm:~#
5. CollectdetailedinformationaboutthevirtualCPUforaguest:
root@kvm:~#virshvcpuinfokvm1
VCPU:0
CPU:29
State:running
CPUtime:118.8s
CPUAffinity:yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy
root@kvm:~#
6. GatherinformationaboutthevCPUtimersfortheguestVM:
root@kvm:~#virshcpu-stats--totalkvm1
Total:
cpu_time175.003045493seconds
user_time2.610000000seconds
system_time7.510000000seconds
root@kvm:~#
7. CollectgeneralinformationabouttheVM:
root@kvm:~#virshdominfokvm1
Id:30
Name:kvm1
UUID:bd167199-c1c4-de7e-4996-43a7f197e565
OSType:hvm
State:running
CPU(s):1
CPUtime:175.6s
Maxmemory:1048576KiB
Usedmemory:1048576KiB
Persistent:yes
Autostart:disable
Managedsave:no
Securitymodel:none
SecurityDOI:0
root@kvm:~#
8. CollectthememoryutilizationfortheVM:
root@kvm:~#virshdommemstat--livekvm1
actual1048576
swap_in0
rss252684
root@kvm:~#
9. GetinformationabouttheblockdevicesassociatedwiththeKVMinstance:
root@kvm:~#virshdomblklistkvm1
TargetSource
------------------------------------------------
hda/var/lib/libvirt/images/kvm1.img
root@kvm:~#
10. ObtainsizeinformationontheblockdevicefortheVM:
root@kvm:~#virshdomblkinfo--devicehdakvm1
Capacity:8589934592
Allocation:2012381184
Physical:2012381184
root@kvm:~#
11. GetanyblockdeviceerrorsfortheKVMinstance:
root@kvm:~#virshdomblkerrorkvm1
Noerrorsfound
root@kvm:~#
12. PrintblockdevicestatisticforaKVMguest:
root@kvm:~#virshdomblkstat--devicehda--humankvm1
Device:hda
numberofreadoperations:42053
numberofbytesread:106145280
numberofwriteoperations:10648
numberofbyteswritten:96768000
numberofflushoperations:4044
totaldurationofreads(ns):833974071
totaldurationofwrites(ns):1180545967
totaldurationofflushes(ns):3458623200
root@kvm:~#
Howitworks...Inthisrecipe,wecollectvarioushypervisorandguestresourceutilizationinformationfromthehostOS.Inlaterrecipes,wewillusethatinformationinamonitoringsystemtoalertandtriggeractionsbasedonsetcriteriaandthresholds.Let'sgothroughthestepsweperformedearlierinmoredetail.
WestartbycollectinginformationabouttheCPUutilizationonthehypervisor/hostOSinstep1.Wethenproceedtogatherthememoryutilizationonthephysicalhostinstep2.NotethatwecanalsouseotherLinuxcommandstodothisinsteadofvirsh,butithelpsdemonstratetheconcept.
MonitoringthestateofaKVMinstanceisimportantincasetheVMterminatesunexpectedlyordoesnotstartautomaticallyafteraserverreboot.Instep3,weobtainthecurrentstateofthevirtualmachine.
Insteps4,5,and6,wecollectinformationaboutthevirtualCPUoftheguest.WecanseethenumberofassignedCPUsalongwithotherusefulinformation,suchasthetimetheCPUspendsrunningkernelanduserspacecode.
Instep7,wecollectmoregeneralinformationaboutthevirtualmachine;ofnotableinterestisthetotalandusedamountofmemorythatwecansetalertingthresholdson.
Instep8,wegetinformationaboutthememoryutilizationoftheKVMinstance.Wecanseethetotal,swap,andresidentmemoryusageintheoutput.
Instep9,welisttheblockdevicesattachedtothevirtualmachine,andweusethatinstep10togetinformationaboutitscapacity.Ifthereareanyerrorsassociatedwiththeblockdevice,theoutputofthecommandinstep11willshowthat,whichwecanusetotriggermonitoringalerts.
MonitoringtheperformanceofablockdeviceattachedtoaKVMinstancecanbedoneusingtheoutputofthecommandinstep12.
There'smore...Whenwecreateavirtualmachinewithlibvirt,theQEMUprocessthatisstartedexposesamonitoringsocketthatwecanconnecttoandcollectinformationabouttheguest.
Let'sseehowthatlooksfortheKVMinstancewe'vebeenusing:
1. Gettheprocessinformationfortheguestinstance:
root@kvm:~#pgrep-lfakvm1
32332/usr/bin/qemu-system-x86_64-namekvm1-S-machinepc-
i440fx-trusty,accel=kvm,usb=off-m1024-realtimemlock=off-smp
1,sockets=1,cores=1,threads=1-uuidbd167199-c1c4-de7e-4996-43a7f197e565
-no-user-config-nodefaults-chardev
socket,id=charmonitor,path=/var/lib/libvirt/qemu/kvm1.monitor,server,nowait
-monchardev=charmonitor,id=monitor,mode=control-rtcbase=utc-no-
shutdown-bootstrict=on-devicepiix3-usb-
uhci,id=usb,bus=pci.0,addr=0x1.0x2-drive
file=/var/lib/libvirt/images/kvm1.img,if=none,id=drive-ide0-0-
0,format=raw-deviceide-hd,bus=ide.0,unit=0,drive=drive-ide0-0-
0,id=ide0-0-0,bootindex=1-netdevtap,fd=24,id=hostnet0-device
rtl8139,netdev=hostnet0,id=net0,mac=52:54:00:55:9b:d6,bus=pci.0,addr=0x3
-chardevpty,id=charserial0-deviceisa-
serial,chardev=charserial0,id=serial0-vnc146.20.141.158:0-device
cirrus-vga,id=video0,bus=pci.0,addr=0x2-devicevirtio-balloon-
pci,id=balloon0,bus=pci.0,addr=0x4
root@kvm:~#
Noticefromtheprecedingoutputthe-chardevsocket,id=charmonitor,path=/var/lib/libvirt/qemu/kvm1.monitorandthe-monchardev=charmonitor,id=monitor,mode=controlparameterspassedtotheQEMUprocess.
2. Wecanaccessthissocketintwoways,eitherbyconnectingtoitusingtoolssuchasncandsocatorwiththevirshcommand,asfollows:
root@kvm:~#virshqemu-monitor-commandkvm1--hmp"info"
infoballoon--showballooninformation
infoblock[-v][device]--showinfoofoneblockdeviceorallblock
devices(anddetailsofimageswith-voption)
infoblock-jobs--showprogressofongoingblockdeviceoperations
infoblockstats--showblockdevicestatistics
infocapture--showcaptureinformation
infochardev--showthecharacterdevices
infocpus--showinfosforeachCPU
infocpustats--showCPUstatistics
infohistory--showthecommandlinehistory
infoirq--showtheinterruptsstatistics(ifavailable)
infojit--showdynamiccompilerinfo
infokvm--showKVMinformation
infomem--showtheactivevirtualmemorymappings
infomice--showwhichguestmouseisreceivingevents
infomigrate--showmigrationstatus
infomigrate_cache_size--showcurrentmigrationxbzrlecachesize
infomigrate_capabilities--showcurrentmigrationcapabilities
infomtree--showmemorytree
infoname--showthecurrentVMname
infonetwork--showthenetworkstate
infonuma--showNUMAinformation
infopci--showPCIinfo
infopcmcia--showguestPCMCIAstatus
infopic--showi8259(PIC)state
infoprofile--showprofilinginformation
infoqdm--showqdevdevicemodellist
infoqtree--showdevicetree
inforegisters--showthecpuregisters
inforoms--showroms
infosnapshots--showthecurrentlysavedVMsnapshots
infospice--showthespiceserverstatus
infostatus--showthecurrentVMstatus(running|paused)
infotlb--showvirtualtophysicalmemorymappings
infotpm--showtheTPMdevice
infotrace-events--showavailabletrace-events&theirstate
infousb--showguestUSBdevices
infousbhost--showhostUSBdevices
infousernet--showusernetworkstackconnectionstates
infouuid--showthecurrentVMUUID
infoversion--showtheversionofQEMU
infovnc--showthevncserverstatus
root@kvm:~#
3. ToobtaininformationaboutthenetworkinterfaceoftheKVMinstance,wecanrunthefollowingcode:
root@kvm:~#virshqemu-monitor-commandkvm1--hmp"infonetwork"
net0:index=0,type=nic,model=rtl8139,macaddr=52:54:00:55:9b:d6
\hostnet0:index=0,type=tap,fd=24
root@kvm:~#
QEMUprovidesaguestagentdaemonthatcanbestartedinsidetheKVMinstanceandthencanbeconnectedtofromthehostOS.Wecancollectadditionaldataorupdatecertainsettingsforthevirtualmachinedirectlyfromthehost.
Let'sseeanexampleofinstallingandusingtheQEMUguestagent:
1. CreatetherequireddirectorythatwillcontainthecommunicationsocketbetweenlibvirtonthehostandtheguestagentrunninginsidetheKVMguest:
root@kvm:~#mkdir-p/var/lib/libvirt/qemu/channel/target
root@kvm:~#chownlibvirt-qemu:kvm
/var/lib/libvirt/qemu/channel/-R
root@kvm:~#
2. EdittheconfigurationfortherunningVMandaddthefollowingdefinitionunderthe<devices>section:
root@kvm:~#virsheditkvm1
...
<devices>
...
<channeltype='unix'>
<sourcemode='bind'
path='/var/lib/libvirt/qemu/channel/target/kvm1.org.qemu.guest_agent.0'/>
<targettype='virtio'name='org.qemu.guest_agent.0'/>
<aliasname='channel0'/>
<addresstype='virtio-serial'controller='0'bus='0'
port='1'/>
</channel>
...
</devices>
...
Domainkvm1XMLconfigurationedited.
root@kvm:~#
3. RestarttheKVMinstance:
root@kvm:~#virshdestroykvm1
Domainkvm1destroyed
root@kvm:~#virshstartkvm1
Domainkvm1started
root@kvm:~#
4. ConnecttotheVMandinstallandstarttheQEMUguestagent,asfollows:
root@kvm:~#virshconsolekvm1
Connectedtodomainkvm1
Escapecharacteris^]
root@debian:~#aptupdate&&aptinstallqemu-guest-agent
...
root@debian:~#serviceqemu-guest-agentstart
root@debian:~#serviceqemu-guest-agentstatus
●qemu-guest-agent.service-LSB:QEMUGuestAgentstartupscript
Loaded:loaded(/etc/init.d/qemu-guest-agent)
Active:active(running)sinceWed2017-04-1914:44:08CDT;
33minago
Process:397ExecStart=/etc/init.d/qemu-guest-agentstart
(code=exited,status=0/SUCCESS)
CGroup:/system.slice/qemu-guest-agent.service
└─425/usr/sbin/qemu-ga--daemonize-mvirtio-serial-p
/dev/virti...
Apr1914:44:08debiansystemd[1]:StartingLSB:QEMUGuestAgent
startups.....
Apr1914:44:08debiansystemd[1]:StartedLSB:QEMUGuestAgent
startupscript.
Hint:Somelineswereellipsized,use-ltoshowinfull.
root@debian:~#
5. Backonthehost,wecanseethenewsocketfile:
root@kvm:~#ls-la/var/lib/libvirt/qemu/channel/target/
total8
drwxr-xr-x2libvirt-qemukvm4096Apr1919:43.
drwxr-xr-x3libvirt-qemukvm4096Apr1919:43..
srwxr-xr-x1libvirt-qemukvm0Apr1919:43
kvm1.org.qemu.guest_agent.0
root@kvm:~#
6. Let'sconnecttotheguestagentfromthehostandlisttheavailablecommandsthattheguestagentaccepts:
root@kvm:~#virshqemu-agent-commandkvm1--pretty--cmd
'{"execute":"guest-info"}'
{
"return":{
"version":"2.1.2",
"supported_commands":[
{
"enabled":true,
"name":"guest-set-vcpus",
"success-response":true
},
{
"enabled":true,
"name":"guest-get-vcpus",
"success-response":true
},
{
"enabled":true,
"name":"guest-network-get-interfaces",
"success-response":true
},
...
]
}
}
root@kvm:~#
7. GetinformationabouttheguestvCPUbyrunningthefollowingcode:
root@kvm:~#virshqemu-agent-commandkvm1--pretty--cmd
'{"execute":"guest-get-vcpus"}'
{
"return":[
{
"online":true,
"can-offline":false,
"logical-id":0
}
]
}
root@kvm:~#
Usingthemonitoringandguestagentsocketsprovidesanadditionalwayofcollectingmoreinformationabouttherunningvirtualmachinesonthelibvirthostthatwecanlateraddasmonitoringchecks.
MonitoringKVMinstanceswithSensuSensuisacompletemonitoringsolutionthatusestheclient-servermodel;theserverpublisheschecksinamessagequeueprovidedbytheRabbitmqservice.Theclients/agentssubscribetotopicsinthequeueandexecutethespecifiedchecksonthehosttheyrunon.StateandhistoricaldataisstoredinaRedisserver.
Inthisrecipe,wearegoingtoinstalltheSensuserver,Rabbitmqmessagequeue,andtheRedisserverononehost,writeasimplemonitoringcheckusingtheinformationweobtainedfromtheResourceusagecollectionwithlibvirtrecipeearlier,andinstalltheSensuagentinsidetheKVMguest.
GettingreadyForthisrecipe,wearegoingtoneedthefollowing:
ALinuxhostwithlibvirtinstalledandrunningAKVMinstancerunningonthelibvirthostNetworkconnectivitybetweentheKVMinstanceandthelibvirthost
Howtodoit...TosetupanewSensudeploymentanddefinevariousmonitoringchecks,performthefollowingsteps:
1. InstalltheRedisserverandensurethatitisrespondingtorequests:
root@kvm:~#apt-getinstall-yredis-server
...
root@kvm:~#redis-cliping
PONG
root@kvm:~#
2. InstalltheRabbitmqserver:
root@kvm:~#apt-getinstall-yrabbitmq-server
...
root@kvm:~#
3. CreatethevirtualhosttheSensuagentswillsubscribetoandthecredentialsandpermissionsfortheRabbitmqclients:
root@kvm:~#rabbitmqctladd_vhost/sensu
Creatingvhost"/sensu"...
...done.
root@kvm:~#rabbitmqctladd_usersensusecret
Creatinguser"sensu"...
...done.
root@kvm:~#rabbitmqctlset_permissions-p/sensusensu".*"".*"".*"
Settingpermissionsforuser"sensu"invhost"/sensu"...
...done.
root@kvm:~#
4. AddtheSensuupstreamrepositoryanditskeyandtheninstalltheSensupackage:
root@kvm:~#wget-qhttps://sensu.global.ssl.fastly.net/apt/pubkey.gpg-
O-|apt-keyadd-
OK
root@kvm:~#echo"debhttps://sensu.global.ssl.fastly.net/aptsensu
main"|tee/etc/apt/sources.list.d/sensu.list
debhttps://sensu.global.ssl.fastly.net/aptsensumain
root@kvm:~#apt-getupdate
...
root@kvm:~#apt-getinstall-ysensu
...
root@kvm:~#
5. SensuusesJSON-basedconfigurationfilesforitsconfiguration.CreatetheSensuAPIconfigurationfile:
root@kvm:/etc/sensu/conf.d#catapi.json
{
"api":{
"host":"localhost",
"bind":"0.0.0.0",
"port":4567
}
}
root@kvm:/etc/sensu/conf.d#
6. ConfigurethetransporttypeforSensu;weareusingRabbitmqforthisdeployment:
root@kvm:/etc/sensu/conf.d#cattransport.json
{
"transport":{
"name":"rabbitmq",
"reconnect_on_error":true
}
}
root@kvm:/etc/sensu/conf.d#
7. ConfigurewheretheRabbitmqserviceisacceptingconnections,thevirtualhost,andcredentials:
root@kvm:/etc/sensu/conf.d#catrabbitmq.json
{
"rabbitmq":{
"host":"0.0.0.0",
"port":5672,
"vhost":"/sensu",
"user":"sensu",
"password":"secret"
}
}
root@kvm:/etc/sensu/conf.d#
8. SpecifythehostandporttheRedisserviceislisteningon:
root@kvm:/etc/sensu/conf.d#catredis.json
{
"redis":{
"host":"localhost",
"port":6379
}
}
root@kvm:/etc/sensu/conf.d#
9. ConfiguretheSensuclient:
root@kvm:/etc/sensu/conf.d#catclient.json
{
"client":{
"name":"ubuntu",
"address":"127.0.0.1",
"subscriptions":[
"base"
],
"socket":{
"bind":"127.0.0.1",
"port":3030
}
}
}
root@kvm:/etc/sensu/conf.d#
FormoreinformationonSensu,pleaserefertohttps://sensuapp.org/docs/.
10. InstallthewebfrontendforSensu,namedUchiwa:
root@kvm:/etc/sensu/conf.d#apt-getinstall-yuchiwa
...
root@kvm:/etc/sensu/conf.d
11. ConfiguretheUchiwafrontend:
root@kvm:/etc/sensu/conf.d#cat/etc/sensu/uchiwa.json
{
"sensu":[
{
"name":"KVMguests",
"host":"localhost",
"ssl":false,
"port":4567,
"path":"",
"timeout":5000
}
],
"uchiwa":{
"port":3000,
"stats":10,
"refresh":10000
}
}
root@kvm:/etc/sensu/conf.d#
12. StarttheSensuserver,API,client,andfrontendcomponents:
root@kvm:/etc/sensu/conf.d#/etc/init.d/sensu-serverstart
*Startingsensu-server[OK]
root@kvm:/etc/sensu/conf.d#/etc/init.d/sensu-apistart
*Startingsensu-api[OK]
root@kvm:/etc/sensu/conf.d#/etc/init.d/sensu-clientstart
*Startingsensu-client[OK]
root@kvm:/etc/sensu/conf.d#/etc/init.d/uchiwarestart
uchiwastarted.
root@kvm:/etc/sensu/conf.d#
13. ConnecttotheKVMinstanceconsole;installandconfiguretheSensuclient:
root@kvm:/etc/sensu/conf.d#virshconsolekvm1
Connectedtodomainkvm1
Escapecharacteris^]
root@debian:~#wget-q
https://sensu.global.ssl.fastly.net/apt/pubkey.gpg-O-|apt-keyadd-
OK
root@debian:~#echo"debhttps://sensu.global.ssl.fastly.net/aptsensu
main"|tee/etc/apt/sources.list.d/sensu.list
debhttps://sensu.global.ssl.fastly.net/aptsensumain
root@debian:~#aptinstallapt-transport-https
root@debian:~#aptupdate&&aptinstallsensu
...
root@debian:~#cd/etc/sensu/conf.d/
root@debian:/etc/sensu/conf.d#catclient.json
{
"client":{
"name":"monitor_kvm",
"address":"10.10.10.92",
"subscriptions":["base"]
}
}
root@debian:/etc/sensu/conf.d#catrabbitmq.json
{
"rabbitmq":{
"host":"10.10.10.1",
"port":5672,
"vhost":"/sensu",
"user":"sensu",
"password":"secret"
}
}
root@debian:/etc/sensu/conf.d#cattransport.json
{
"transport":{
"name":"rabbitmq",
"reconnect_on_error":true
}
}
root@debian:/etc/sensu/conf.d#
ReplacetheIPaddressoftheclientwiththeIPaddressconfiguredinsidetheKVMinstance.UpdatetheIPaddressoftheRabbitmq
serverwiththeIPaddressconfiguredonthehostbridge.EnsurethattheKVMguestcanpingthebridgeIPonthehostOS.
14. StarttheSensuclient:
root@debian:/etc/sensu/conf.d#/etc/init.d/sensu-clientstart
Startingsensu-client:.
root@debian:/etc/sensu/conf.d#
15. ConnecttotheUchiwainterfaceandensurethatthehostSensuclientandtheKVMguestSensuclientarelistedintheCLIENTSsection:
TheUchiwafrontendshowingtheconnectedclients
16. WhilestillconnectedtotheKVMguest,installamemorycheckfromthegemrepositoryandtestit:
root@debian:/etc/sensu/conf.d#aptinstallrubygems
...
root@debian:/etc/sensu/conf.d#gemsearchsensu|grepplugins|grep
memory
sensu-plugins-memory(0.0.2)
sensu-plugins-memory-checks(2.1.0)
root@debian:/etc/sensu/conf.d#
root@debian:/etc/sensu/conf.d#geminstallsensu-plugins-memory-checks
...
root@debian:/etc/sensu/conf.d#/etc/init.d/sensu-clientrestart
configurationisvalid
Stoppingsensu-client:.
Startingsensu-client:.
oot@debian:/etc/sensu/conf.d#/usr/local/bin/check-memory-percent.rb-w
80-c9
MEMOK-systemmemoryusage:11%
root@debian:/etc/sensu/conf.d#
17. BackonthehostOS,definethenewmemorycheckfortheKVMguest:
root@kvm:/etc/sensu/conf.d#catcheck_memory.json
{
"checks":{
"memory_check":{
"command":"/usr/local/bin/check-memory-percent.rb-w80-c90",
"subscribers":["base"],
"handlers":["default"],
"interval":300
}
}
}
root@kvm:/etc/sensu/conf.d#
18. RestarttheSensucomponents:
root@kvm:/etc/sensu/conf.d#/etc/init.d/uchiwarestart
Killinguchiwa(pid15350)withSIGTERM
Waitinguchiwa(pid15350)todie...
Waitinguchiwa(pid15350)todie...
uchiwastopped.
uchiwastarted.
root@kvm:/etc/sensu/conf.d#/etc/init.d/sensu-serverrestart
configurationisvalid
*Stoppingsensu-server[OK]
*Startingsensu-server[OK]
root@kvm:/etc/sensu/conf.d#/etc/init.d/sensu-apirestart
configurationisvalid
*Stoppingsensu-api[OK]
*Startingsensu-api[OK]
root@kvm:/etc/sensu/conf.d#
19. Thememory_checkfortheKVMinstanceisnowshowingintheUchiwadashboard:
TheUchiwafrontendshowingthememorycheckfortheKVMguest
Howitworks...Intheprevioussection,weinstalledaSensuserverandalltherequiredinfrastructurecomponentsforittorunonthehypervisorhost.ThenweinstalledtheclientinsideaKVMinstance,installedthememoryrubycheck,anddefineditonthehost.Let'sexamineallthestepsinmoredetail.
Instep1,weinstalltheRedisserverandensurethatitisacceptingconnections.Redisisakey-valuestoreservicethatSensuusestostorethehistoricalinformationaboutthechecks,currentstate,andconnectedclients.
WiththeRedisserverinplace,weproceedtoinstallandconfigureRabbitmqinsteps2through9.RabbitmqisamessagebusconformingtotheAdvancedMessageQueuingProtocol(AMQP)standard.TheSensuserverandclientsproduceandconsumemessagesfromthequeuetotriggermonitoringactions.
Althoughnotrequired,insteps10and11,weinstallandconfigureawebfrontendfortheSensuservernamedUchiwa.WecanusethewebinterfacetocheckonthestatusofdifferentchecksfortheKVMguestswemonitor.
Instep13,weinstalltheSensuclientinsidetheKVMguestinstanceandproceedtoinstallamemorymonitoringscriptfromageminstep16.Amonitoringscriptcanbewritteninanylanguage(RUBYinthiscase)aslongasitreturnstheexpectederrorcodesthatSensuexpects.Inthenextsection,wearegoingtowriteanewcheckfromscratchusingBash.
There'smore...Intheprevioussection,wesawanexampleofhowtousearubycheckinsidetheKVMinstanceandmonitorthememoryutilization.SensuprovidesstandalonechecksthatcanbetriggeredfromtheSensuclient,independentlyfromtheSensuserverschedulingmechanism.Let'susethatfeatureandwriteasimplecheckinBashthatwillrunfromthehostOS,insteadoftheKVMguest,andusethevirshcommandtocheckthestatusofaKVMinstance:
1. Writeastandalonecheckdefinitionwiththecustomscriptthatsensu-clientwillexecutetoperformthecheck:
root@kvm:/etc/sensu/conf.d#catcheck_kvm_instance_status.json
{
"checks":{
"check_kvm_instance_status":{
"command":"check_kvm_instance_status.sh-nkvm1",
"standalone":true,
"subscribers":["base"],
"interval":60
}
}
}
root@kvm:/etc/sensu/conf.d#
2. IntheSensupluginsdirectory,writethissimpleBashscript:
root@kvm:/etc/sensu/conf.d#cd../plugins/
root@kvm:/etc/sensu/plugins#catcheck_kvm_instance_status.sh
#!/bin/bash
#ChecksifaKVMinstanceisrunning
usage()
{
echo"Usage:`basename$0`-n|--namekvm1"
exit2
}
sanity_check()
{
if["$INSTANCE_NAME"==""]
then
usage
fi
}
report_result()
{
if["$INSTANCE_STATE"=="shutoff"]
then
echo"CRITICAL-KVMinstance$INSTANCE_NAMEisnotrunning"
exit2
else
echo"OK-KVMinstance$INSTANCE_NAMEisrunning"
exit0
fi
}
check_instance_state()
{
declare-gINSTANCE_STATE="shutoff"
INSTANCE_STATE=$(sudo/usr/bin/virshdomstate$INSTANCE_NAME)
}
main()
{
sanity_check
check_instance_state
report_result
}
while[[$#>1]]
do
key=$1
case$keyin
-n|--name)
INSTANCE_NAME=$2
shift
;;
*)
usage
;;
esac
shift
done
main
root@kvm:/etc/sensu/plugins#
3. Makethescriptexecutable,addtheSensuuserinasudoersfile,andtestthecheckbyexecutingit:
root@kvm:/etc/sensu/plugins#chmodu+xcheck_kvm_instance_status.sh
root@kvm:/etc/sensu/plugins#chownsensu:sensu
check_kvm_instance_status.sh
root@kvm:/etc/sensu/plugins#echo"sensuALL=(ALL)NOPASSWD:ALL">
/etc/sudoers.d/sensu
root@kvm:/etc/sensu/plugins#sudo-usensu
./check_kvm_instance_status.sh--namekvm1
OK-KVMinstancekvm1isrunning
root@kvm:/etc/sensu/plugins#virshdestroykvm1
Domainkvm1destroyed
root@kvm:/etc/sensu/plugins#sudo-usensu
./check_kvm_instance_status.sh--namekvm1
CRITICAL-KVMinstancekvm1isnotrunning
root@kvm:/etc/sensu/plugins#virshstartkvm1
Domainkvm1started
root@kvm:/etc/sensu/plugins#
4. RestarttheSensuclientonthehost;checkthelogsandtheUchiwadashboardforthenewstandalonecheck:
root@kvm:/etc/sensu/conf.d#/etc/init.d/sensu-clientrestart
configurationisvalid
*Stoppingsensu-client[OK]
*Startingsensu-client[OK]
root@kvm:/etc/sensu/conf.d#cat/var/log/sensu/sensu-client.log|grep
check_kvm_instance_status
{"timestamp":"2017-04-
20T17:37:48.409805+0000","level":"warn","message":"loadingconfig
file","file":"/etc/sensu/conf.d/check_kvm_instance_status.json"}
{"timestamp":"2017-04-
20T17:38:16.746861+0000","level":"info","message":"publishingcheck
result","payload":{"client":"ubuntu","check":
{"command":"check_kvm_instance_status.sh-n
kvm1","standalone":true,"subscribers":
["base"],"interval":60,"name":"check_kvm_instance_status","issued":1492709896,"executed":1492709896,"duration":0.016,"output":"OK
-KVMinstancekvm1isrunning\n","status":0}}}
root@kvm:/etc/sensu/conf.d#
TheUchiwafrontendshowingthestandaloneinstancecheck
UsingtheexamplesfromtheResourceusagecollectionwithlibvirtrecipe,youshouldnowbeabletowriteavarietyofSensumonitoringchecksexecutedfromthehypervisororinsidetheKVMguests.
FormoreinformationonhowSensucanexecutescriptswhenanalertistriggered,pleaserefertothehandlerssectionoftheofficialdocumentationathttps://sensuapp.org/docs/latest/reference/handlers.html.
SimpleKVMbackupswithtarandrsyncInthisrecipe,wearegoingtocreateabackupofaKVMinstanceusingtarandrsyncandstoreitonaremoteserver.ThisistheeasiestwaytobackupaKVMinstance.Inthenextfewrecipes,wearegoingtocreatesnapshotsandusethemasacoldbackup.
GettingreadyForthisextremelysimplerecipe,wearegoingtoneed:
AlibvirthostwitharunningKVMinstance,usinganimagefileasitsbackingstoreThetarandrsyncLinuxutilitiesAremoteservertotransferthebackup
Howtodoit...Tobackupavirtualmachineusingtarandrsync,performthefollowingsteps:
1. Createthebackupdirectoryandchangetoit:
root@kvm:~#mkdirbackup_kvm1&&cdbackup_kvm1
root@kvm:~/backup_kvm1#
2. FindthelocationoftheimagefileoftheKVMguest:
root@kvm:~/backup_kvm1#virshdumpxmlkvm1|grep"sourcefile"
<sourcefile='/var/lib/libvirt/images/kvm1.img'/>
root@kvm:~/backup_kvm1#
3. Savethecurrentinstanceconfigurationtodisk:
root@kvm:~/backup_kvm1#virshdumpxmlkvm1>kvm1.xml
root@kvm:~/backup_kvm1#
4. StoptheKVMguestandcopytheimagefiletothebackupdirectory:
root@kvm:~/backup_kvm1#virshdestroykvm1
Domainkvm1destroyed
root@kvm:~/backup_kvm1#cp/var/lib/libvirt/images/kvm1.img.
root@kvm:~/backup_kvm1#ls-lah
total2.4G
drwxr-xr-x2rootroot4.0KApr2018:37.
drwx------7rootroot4.0KApr2018:36..
-rwxr-xr-x1rootroot8.0GApr2018:37kvm1.img
-rw-r--r--1rootroot3.0KApr2018:36kvm1.xml
root@kvm:~/backup_kvm1#
5. CreateasinglearchivefortheVM'sconfigurationandimagefiles:
root@kvm:~/backup_kvm1#tarjcvfkvm1_backup.tar.bz.
./
./kvm1.img
./kvm1.xml
root@kvm:~/backup_kvm1#rmkvm1.imgkvm1.xml
root@kvm:~/backup_kvm1#
6. Transferthebackuparchivetoaremoteserver:
root@kvm:~/backup_kvm1#rsync-vazkvm1_backup.tar.bzkvm2:/tmp
sendingincrementalfilelist
kvm1_backup.tar.bz
sent842,977,610bytesreceived35bytes26,761,195.08bytes/sec
totalsizeis845,671,214speedupis1.00
root@kvm:~/backup_kvm1#
7. Torestorefromthebackup,logintotheremoteserverandextractthearchive:
root@kvm2:~#cd/tmp/
root@kvm2:/tmp#tarjxfvkvm1_backup.tar.bz
./
./kvm1.img
./kvm1.xml
root@kvm2:/tmp#
8. Copytheimagefiletotheconfiguredlocationanddefinetheinstance:
root@kvm2:/tmp#cpkvm1.img/var/lib/libvirt/images/
root@kvm2:/tmp#virshdefinekvm1.xml
Domainkvm1definedfromkvm1.xml
root@kvm2:/tmp#virshlist--all|grepkvm1
-kvm1shutoff
root@kvm2:/tmp#
Howitworks...Aftercreatingthebackupdirectoryinstep1,wesavethecurrentguestdefinitiontodiskinstep3.Instep4,afterstoppingthevirtualmachine,wecopyitsimagefiletothebackupdirectory.Instep5,wecreateabzip2compresseddatafilewhichwetransfertoaremoteserverinstep6.
Ontheremoteserver,weextractthearchiveinstep7andcopytherawimagefiletowheretheXMLdefinitionoftheinstanceisexpectingit,thendefinetheinstanceinstep8.
Notethatinordertopreservetheconsistencyandintegrityofthedatawhilecopyingtheimagefiletothebackupdirectory,wehadtofirststoptheKVMguest.
CreatingsnapshotsAvirtualmachinesnapshotpreservesthecurrentstateofarunningorstoppedinstanceataspecificpointintime.Itcanlaterbeusedtorestoretheinstancefromthatpoint.Snapshotscanbeusedasbackupsorastemplatesforbuildingnewvirtualmachinesthatwillbecopiesoftheoriginalinstance.
Totakeadvantageofsnapshots,thebackingstoremustfirstsupportit.IfyourecallfromtheManagingDiskimageswithqemu-imgrecipeinChapter1,GettingStartedwithQEMUandKVM,wecreatedarawimagetypefortheKVMguest.Inthisrecipe,wearegoingtousetheQEMUCopy-On-Write(QCOW2)imageformatasthebackingstorefortheKVMinstance,becausetherawimageformatdoesnotsupportsnapshots.
UsingtheQCOW2imageformat,wecancreateabaseimagecontainingtheguestOSandeverythingelseweneedforthevirtualmachine,andthencreateseveralcopy-on-writeoverlaydiskimagesontopoftheoriginalbaseimage.Thesenewoverlayimagescanbeusedinnewvirtualmachinesrightaway,bycreatingnewXMLdefinitionfilespointingtothenewimage.
Let'sseeanexampleofusingQEMUtocreateimageoverlaysbeforeweproceedwithmakinglibvirtsnapshots:
1. TocollectinformationaboutaQCOW2image,wecanusetheqemu-imgutility:
root@kvm:~#qemu-imginfokvm1.qcow2
image:kvm1.qcow2
fileformat:qcow2
virtualsize:8.0G(8589934592bytes)
disksize:2.4G
cluster_size:65536
Formatspecificinformation:
compat:1.1
lazyrefcounts:false
root@kvm:~#
ToconvertanexistingrawimagetoQCOW2,run:root@kvm:~#qemu-imgconvert-fraw-Oqcow2
/var/lib/libvirt/images/kvm1.img/var/lib/libvirt/images/kvm1.qcow2
2. Let'screateanewoverlayimagebasedontheqcow2precedingimage:
root@kvm:~#qemu-imgcreate-fqcow2-b
/var/lib/libvirt/images/kvm1_copy.qcow2
/var/lib/libvirt/images/kvm1_copy_2.qcow2
Formatting'/var/lib/libvirt/images/kvm1_copy_2.qcow2',fmt=qcow2
size=8589934592backing_file='/var/lib/libvirt/images/kvm1_copy.qcow2'
encryption=offcluster_size=65536lazy_refcounts=off
root@kvm:~#
3. Gettinginformationaboutthenewoverlayimagenowshowsthebackingfileit'sbasedon:
root@kvm:~#qemu-imginfokvm1_copy.qcow2
image:kvm1_copy.qcow2
fileformat:qcow2
virtualsize:8.0G(8589934592bytes)
disksize:196K
cluster_size:65536
backingfile:/var/lib/libvirt/images/kvm1.qcow2
Formatspecificinformation:
compat:1.1
lazyrefcounts:false
root@kvm:~#
4. Wecancreateanewoverlayfilefromthepreviousoverlayfile:
root@kvm:~#qemu-imgcreate-fqcow2-b
/var/lib/libvirt/images/kvm1_copy.qcow2
/var/lib/libvirt/images/kvm1_copy_2.qcow2
Formatting'/var/lib/libvirt/images/kvm1_copy_2.qcow2',fmt=qcow2
size=8589934592backing_file='/var/lib/libvirt/images/kvm1_copy.qcow2'
encryption=offcluster_size=65536lazy_refcounts=off
root@kvm:~#qemu-imginfo/var/lib/libvirt/images/kvm1_copy_2.qcow2
image:/var/lib/libvirt/images/kvm1_copy_2.qcow2
fileformat:qcow2
virtualsize:8.0G(8589934592bytes)
disksize:196K
cluster_size:65536
backingfile:/var/lib/libvirt/images/kvm1_copy.qcow2
Formatspecificinformation:
compat:1.1
lazyrefcounts:false
root@kvm:~#
5. Let'slisttheentireimagechainforthelastoverlayfile:
root@kvm:~#qemu-imginfo--backing-chain
/var/lib/libvirt/images/kvm1_copy_2.qcow2
image:/var/lib/libvirt/images/kvm1_copy_2.qcow2
fileformat:qcow2
virtualsize:8.0G(8589934592bytes)
disksize:196K
cluster_size:65536
backingfile:/var/lib/libvirt/images/kvm1_copy.qcow2
Formatspecificinformation:
compat:1.1
lazyrefcounts:false
image:/var/lib/libvirt/images/kvm1_copy.qcow2
fileformat:qcow2
virtualsize:8.0G(8589934592bytes)
disksize:196K
cluster_size:65536
backingfile:/var/lib/libvirt/images/kvm1.qcow2
Formatspecificinformation:
compat:1.1
lazyrefcounts:false
image:/var/lib/libvirt/images/kvm1.qcow2
fileformat:qcow2
virtualsize:8.0G(8589934592bytes)
disksize:2.4G
cluster_size:65536
Formatspecificinformation:
compat:1.1
lazyrefcounts:false
root@kvm:~#
LibvirtusestheabilityoftheQCOW2imageformattocreateachainofoverlaysnapshotsthatcanbeusedasbackupsorastemplatesfornewvirtualmachines.Onceanoverlayiscreated,theoriginalbaseimageistreatedasread-only.Modificationstothebaseimages(inthisexample,kvm1.qcow2andkvm1_copy.qcow2becausebotharebaseimagesforthekvm1_copy_2.qcow2image)arenotrecommended.Here'sadiagrammaticrepresentationofthechainofoverlayimagefileswecreatedearlier:
ThechainofoverlayQCOW2images,eachoneservingasabaseimageforthenext
GettingreadyForthisrecipe,wearegoingtoneedthefollowing:
AlibvirthostwithanexistingQCOW2image,withnosnapshotsattachedArunningKVMinstanceTheQEMUtoolset
Howtodoit...TocreateanewKVMsnapshot,followthesesteps:
1. Createaninternalsnapshotoftherunninginstance:
root@kvm:~#virshsnapshot-createkvm1
Domainsnapshot1492797458created
root@kvm:~#
2. Examinethelocationofthenewsnapshotconfiguration:
root@kvm:~#ls-la/var/lib/libvirt/qemu/snapshot/
total12
drwxr-xr-x3libvirt-qemukvm4096Apr2114:05.
drwxr-x---6libvirt-qemukvm4096Apr2114:04..
drwxr-xr-x2rootroot4096Apr2117:57kvm1
root@kvm:~#ls-la/var/lib/libvirt/qemu/snapshot/kvm1/
total12
drwxr-xr-x2rootroot4096Apr2117:57.
drwxr-xr-x3libvirt-qemukvm4096Apr2114:05..
-rw-------1rootroot3089Apr2117:571492797458.xml
root@kvm:~#
3. ExaminethesnapshotXMLdefinition:
root@kvm:~#cat/var/lib/libvirt/qemu/snapshot/kvm1/1492797458.xml
<!--
WARNING:THISISANAUTO-GENERATEDFILE.CHANGESTOITARELIKELYTOBE
OVERWRITTENANDLOST.Changestothisxmlconfigurationshouldbemade
using:
virshsnapshot-edit
orotherapplicationusingthelibvirtAPI.
-->
<domainsnapshot>
<name>1492797458</name>
<state>running</state>
<creationTime>1492797458</creationTime>
<memorysnapshot='internal'/>
<disks>
<diskname='hda'snapshot='internal'/>
</disks>
<domaintype='kvm'>
<name>kvm1</name>
...
</domain>
<active>1</active>
</domainsnapshot>
root@kvm:~#
4. Collectinformationaboutthebaseimage:
root@kvm:~#qemu-imginfo/var/lib/libvirt/images/kvm1.qcow2
image:/var/lib/libvirt/images/kvm1.qcow2
fileformat:qcow2
virtualsize:8.0G(8589934592bytes)
disksize:2.4G
cluster_size:65536
Snapshotlist:
IDTAGVMSIZEDATEVMCLOCK
11492797458155M2017-04-2117:57:3803:41:16.790
Formatspecificinformation:
compat:1.1
lazyrefcounts:false
root@kvm:~#
5. Obtaininformationaboutthediskdeviceonthevirtualmachine:
root@kvm:~#virshdomblklistkvm1
TargetSource
------------------------------------------------
hda/var/lib/libvirt/images/kvm1.qcow2
root@kvm:~#
6. Createanexternal,disk-onlysnapshot:
root@kvm:~#virshsnapshot-create-askvm1kvm1_ext_snapshot"Diskonly
externalsnapshotforkvm1"--disk-only--diskspec
hda,snapshot=external,file=/var/lib/libvirt/images/kvm1_disk_external.qcow2
Domainsnapshotkvm1_ext_snapshotcreated
root@kvm:~#
7. Getinformationabouttheexternalsnapshot:
root@kvm:~#qemu-imginfo
/var/lib/libvirt/images/kvm1_disk_external.qcow2
image:/var/lib/libvirt/images/kvm1_disk_external.qcow2
fileformat:qcow2
virtualsize:8.0G(8589934592bytes)
disksize:196K
cluster_size:65536
backingfile:/var/lib/libvirt/images/kvm1.qcow2
backingfileformat:qcow2
Formatspecificinformation:
compat:1.1
lazyrefcounts:false
root@kvm:~#
Howitworks...Therearetwomaintypesofsnapshot:
Aninternalsnapshot:ThebaseimagefileitselfcontainsthesavedstateandallsubsequentchangestothevirtualmachineAnexternalsnapshot:Thebaseimagewillcontainthesavedstateofthevirtualmachinethusbecomingaread-onlybaseimage,andanewoverlayimageiscreatedtotrackanyfuturechanges
Bothtypesofsnapshotscanbeperformedonjustthediskorthememoryofthevirtualmachine,eitheronaliveorstoppedinstance.
Intheprecedingstep1,wecreateaninternalsnapshotofthevirtualmachine.Afterthesnapshot,there'sonlyoneimagefile:theoriginalimage,nowcontainingthesnapshot.Wecanseethattheimageisasnapshotinstep4,undertheSnapshotlistsectionoftheoutput.
Instep6,weperformanexternal,disk-onlysnapshot,byspecifyingthevirtualmachinedisk,name,andlocationforthesnapshot.Notethatafterthesnapshot,anewimagefilehasbeencreatedtotrackanyfurtherchanges.Weexaminethatfileinstep7.Notehowthebackingfileistheoriginalqcow2image.
Toperformthedisksnapshots,libvirtleveragestheQEMUfunctionality,liketheqemu-imgcommandwesawearlierwhencreatingtheoverlayimages.
Wecannowsavethesnapshotsasbackupsorusethemtostartnewvirtualmachines.Inthefollowingrecipes,wearegoingtoseeexamplesonhowtouseandmanipulatethesnapshots.
ListingsnapshotsInthepreviousrecipe,wecreatetwosnapshotsofthesameKVMinstance:oneinternalandonedisk-only,externalsnapshot.Inthisrecipe,wearegoingtolearnhowtolistexistingsnapshots.
GettingreadyForthisrecipe,wearegoingtoneed:
AlibvirthostwiththeQEMUtoolsetArunningKVMinstanceThesnapshotswecreatedintheCreatingsnapshotsrecipe
Howtodoit...Tolistallexistingsnapshots,followthenextsteps:
1. ListallsnapshotsforthespecifiedKVMinstance:
root@kvm:~#virshsnapshot-listkvm1
NameCreationTimeState
------------------------------------------------------------
14927974582017-04-2117:57:38+0000running
kvm1_ext_snapshot2017-04-2118:08:49+0000disk-snapshot
root@kvm:~#
2. Listonlythedisk-basedsnapshots:
root@kvm:~#virshsnapshot-list--disk-onlykvm1
NameCreationTimeState
------------------------------------------------------------
kvm1_ext_snapshot2017-04-2118:08:49+0000disk-snapshot
root@kvm:~#
3. Listonlytheinternalsnapshots:
root@kvm:~#virshsnapshot-list--internalkvm1
NameCreationTimeState
------------------------------------------------------------
14927974582017-04-2117:57:38+0000running
root@kvm:~#
4. Listtheexternalsnapshotsonly:
root@kvm:~#virshsnapshot-list--externalkvm1
NameCreationTimeState
------------------------------------------------------------
kvm1_ext_snapshot2017-04-2118:08:49+0000disk-snapshot
root@kvm:~#
5. Listallimagesinahierarchicaltreeformat:
root@kvm:~#virshsnapshot-list--treekvm1
1492797458
|
+-kvm1_ext_snapshot
root@kvm:~#
Howitworks...Weusedtheversatilevirshsnapshot-listcommandtolistallinternalandexternalsnapshotsforthespecifiedvirtualmachine.Notehowwecangetsimilarinformationusingtheqemu-imgcommanddirectlyontheimagefiles,aswesawearlierinthischapter.However,theAPIcallsthatlibvirtprovidesforlistingsnapshotsaremuchmoreconvenient.Inthenextchapter,wearegoingtoseeexamplesonhowtousethelibvirtPythonbindingstomanipulateKVMinstancesandtheirsnapshots.
InspectingsnapshotsInthisshortrecipe,wearegoingtoseeexamplesonhowtoobtainmoreinformationonexistingvirtualmachinesnapshots.
GettingreadyForthisrecipe,wearegoingtoneedthefollowing:
AlibvirthostwiththeQEMUtoolsetThesnapshotswecreatedintheCreatingsnapshotsrecipe
Howtodoit...Toinspectasnapshot,runthefollowingcommands:
1. ListallavailablesnapshotsforthespecifiedKVMinstance:
root@kvm:~#virshsnapshot-listkvm1
NameCreationTimeState
------------------------------------------------------------
14927974582017-04-2117:57:38+0000running
kvm1_ext_snapshot2017-04-2118:08:49+0000disk-snapshot
root@kvm:~#
2. Getinformationabouttherunningsnapshot:
root@kvm:~#virshsnapshot-infokvm1--snapshotname1492797458
Name:1492797458
Domain:kvm1
Current:no
State:running
Location:internal
Parent:-
Children:1
Descendants:1
Metadata:yes
root@kvm:~#
3. Getinformationaboutthedisksnapshot:
root@kvm:~#virshsnapshot-infokvm1--snapshotnamekvm1_ext_snapshot
Name:kvm1_ext_snapshot
Domain:kvm1
Current:yes
State:disk-snapshot
Location:external
Parent:1492797458
Children:0
Descendants:0
Metadata:yes
root@kvm:~#
4. DumptheXMLconfigurationforthedisksnapshot:
root@kvm:~#virshsnapshot-dumpxmlkvm1--snapshotnamekvm1_ext_snapshot
--security-info
<domainsnapshot>
<name>kvm1_ext_snapshot</name>
<description>Diskonlyexternalsnapshotforkvm1</description>
<state>disk-snapshot</state>
<parent>
<name>1492797458</name>
</parent>
<creationTime>1492798129</creationTime>
<memorysnapshot='no'/>
<disks>
<diskname='hda'snapshot='external'type='file'>
<drivertype='qcow2'/>
<sourcefile='/var/lib/libvirt/images/kvm1_disk_external.qcow2'/>
</disk>
</disks>
<domaintype='kvm'>
<name>kvm1</name>
...
<devices>
<emulator>/usr/bin/qemu-system-x86_64</emulator>
<disktype='file'device='disk'>
<drivername='qemu'type='qcow2'/>
<sourcefile='/var/lib/libvirt/images/kvm1.qcow2'/>
<targetdev='hda'bus='ide'/>
<addresstype='drive'controller='0'bus='0'target='0'
unit='0'/>
</disk>
...
</devices>
<seclabeltype='none'/>
</domain>
</domainsnapshot>
Howitworks...Instep1,welistalltheavailablesnapshotsforthekvm1virtualmachine.Insteps2and3,weobtaininformationaboutthesnapshots.OfparticularinterestistheParentandChildrenfields,showingusthehierarchyofthesnapshots.
Instep4,weexaminetheXMLdefinitionoftheKVMguestandthedisk-only,externalsnapshot.Wecanobservethesnapshot='external'typeandthebaseimagelocationspecifiedwiththe<sourcefile='/var/lib/libvirt/images/kvm1_disk_external.qcow2'/>stanza.
EditingsnapshotsInthisrecipe,wearegoingtoedittheXMLdefinitionofanexistingsnapshotandexaminethechanges.
GettingreadyForthisrecipe,wearegoingtoneedthefollowing:
AlibvirthostwiththeQEMUtoolsetThesnapshotswecreatedintheCreatingsnapshotsrecipe
Howtodoit...Toeditasnapshot,runthefollowing:
1. ListallavailablesnapshotsforthespecifiedKVMinstance:
root@kvm:~#virshsnapshot-listkvm1
NameCreationTimeState
------------------------------------------------------------
14927974582017-04-2117:57:38+0000running
kvm1_ext_snapshot2017-04-2118:08:49+0000disk-snapshot
root@kvm:~#
2. Editthedisksnapshotandchangeitsnameanddescription:
root@kvm:~#virshsnapshot-editkvm1--snapshotnamekvm1_ext_snapshot--
rename
<domainsnapshot>
<name>kvm1_ext_snapshot_renamed</name>
<description>Diskonlyexternalsnapshotforkvm1</description>
...
root@kvm:~#
3. Listthesnapshotsaftertheupdate:
root@kvm:~#virshsnapshot-listkvm1
NameCreationTimeState
------------------------------------------------------------
14927974582017-04-2117:57:38+0000running
kvm1_ext_snapshot_renamed2017-04-2118:08:49+0000disk-snapshot
root@kvm:~#
Howitworks...Libvirtprovidesawaytoeditthesnapshotdefinitionforavirtualmachine.WecanchangevariousXMLattributes,suchasthesnapshotname,description,orthelocationofthebackingimagefile.Instep1,welistallavailablesnapshotsforthespecifiedKVMinstance,thenproceedtoupdatethenameanddescriptionofthediskimage.Finally,instep3,wecanseethechangednamefortheexternalsnapshot.
RevertingsnapshotsInthisrecipe,wearegoingtocreateaninternalsnapshotofarunninginstance,introduceachange,thenrestorebacktotheoriginalinstancestateusingthesnapshot.
GettingreadyForthisrecipe,wearegoingtoneedthefollowing:
AlibvirthostwithanexistingQCOW2imageArunningKVMinstance,usingtheQCOW2imageTheQEMUtoolset
Howtodoit...TorevertthestateofaKVMinstancetoanolderstate,fromanexistingsnapshot,runthefollowing:
1. ConnecttotheKVMinstanceandcreateanewfile:
root@kvm:~#virshconsolekvm1
Connectedtodomainkvm1
Escapecharacteris^]
root@debian:~#touchSNAPSHOT
root@debian:~#
root@kvm:~#
2. Createaninternalsnapshotofthevirtualmachine:
root@kvm:~#virshsnapshot-createkvm1
Domainsnapshot1492802417created
root@kvm:~#
3. Getinformationaboutthesnapshot:
root@kvm:~#virshsnapshot-infokvm1--snapshotname1492802417
Name:1492802417
Domain:kvm1
Current:yes
State:running
Location:internal
Parent:1492797458
Children:0
Descendants:0
Metadata:yes
root@kvm:~#
4. Connectbacktothevirtualmachineanddeletethefilewecreatedinstep1:
root@kvm:~#virshconsolekvm1
Connectedtodomainkvm1
Escapecharacteris^]
root@debian:~#rm-fSNAPSHOT
root@debian:~#
root@kvm:~#
5. Restoretheinstancefromthelatestsnapshot:
root@kvm:~#virshsnapshot-revertkvm1--snapshotname1492802417
root@kvm:~#
6. Connecttothevirtualmachineandconfirmthatfilewedeletedinthepreviousstepexistsagain:
root@kvm:~#virshconsolekvm1
Connectedtodomainkvm1
Escapecharacteris^]
root@debian:~#ls-laSNAPSHOT
-rw-r--r--1rootroot0Apr2114:08SNAPSHOT
root@debian:~#
root@kvm:~#
Howitworks...Instep1,weconnecttotheKVMinstanceusingtheconsoleandcreateanemptyfile.Wearegoingtousethefiletotrackchangesonthevirtualmachine.Instep2,wecreateaninternalsnapshotandobtainmoreinformationaboutitinstep3.Instep4,weconnecttotheKVMguestagainanddeletethefile.Instep5,werestorefromthesnapshot,confirmingthatthestateoftheinstancehasbeenindeedrevertedtobeforethesnapshot,asshownbythepresenceoftheoriginalfilewecreatedearlier.
DeletingsnapshotsInthisquickrecipe,wearegoingtodeletethesnapshotswecreatedearlierintheCreatingsnapshotsrecipe,usinglibvirt.
GettingreadyForthisrecipe,weareonlygoingtoneedthefollowing:
AlibvirthostwiththeQEMUtoolsetThesnapshotswecreatedintheCreatingsnapshotsrecipe
Howtodoit...Todeleteasnapshot,followthesesteps:
1. Listallsnapshotspresentonthehost:
root@kvm:~#virshsnapshot-listkvm1
NameCreationTimeState
------------------------------------------------------------
14927974582017-04-2117:57:38+0000running
14928024172017-04-2119:20:17+0000running
kvm1_ext_snapshot_renamed2017-04-2118:08:49+0000disk-snapshot
root@kvm:~#
2. Deletethelatestsnapshotbasedonthecreationtime:
root@kvm:~#virshsnapshot-deletekvm1--snapshotname1492802417
Domainsnapshot1492802417deleted
root@kvm:~#
3. Listtheremainingsnapshots:
root@kvm:~#virshsnapshot-listkvm1
NameCreationTimeState
------------------------------------------------------------
14927974582017-04-2117:57:38+0000running
kvm1_ext_snapshot_renamed2017-04-2118:08:49+0000disk-snapshot
root@kvm:~#
4. Deletethelatestsnapshot:
root@kvm:~#virshsnapshot-deletekvm1--current
Domainsnapshot1492797458deleted
root@kvm:~#
Howitworks...Instep1,welistallsnapshotsonthehostOS.Wethendeletethelatestsnapshot,specifyingitsname,instep2.Instep3,weverifythatthesnapshothasbeenindeeddeleted.Finally,instep4,wedeletethelatestimagebyspecifyingthe--currentflag.
Pleasenotethatinordertodeleteorrestoreanexternalsnapshotalibvirtversionnewerthan1.2.2isrequired.IfyourLinuxdistributiondoesnotprovideanewerversioninitsrepositories,youwillhavetocompilelibvirtfromsource.
DeployingKVMInstanceswithOpenStack
Inthischapter,wearegoingtocoverthefollowingtopics:
PreparingthehostfortheOpenStackdeploymentInstallingandconfiguringtheOpenStackKeystoneidentityserviceInstallingandconfiguringtheOpenStackGlanceimageserviceInstallingandconfiguringtheOpenStackNovacomputeserviceInstallingandconfiguringtheOpenStackNeutronnetworkingserviceBuildingandinspectingKVMinstanceswithOpenStackStoppingKVMinstanceswithOpenStackTerminatingKVMinstanceswithOpenStack
IntroductionOpenStackisacloudoperatingsystemthatsimplifiesthedeploymentandmanagementofvirtualmachinesorcontainersinascalableandhighlyavailableway.Itoperatesonpoolsofcomputeresources(physicalorvirtualservers)andprovidesanintelligentschedulingmechanismtoselectappropriatehosts,andtobuildormigrateVMs.
OpenStackallowsaneasiermanagementofvirtualimagesandprovidesacentralizedwayofcreatingandmanagingsoftware-definednetworks.Itintegrateswellwithavarietyofexternalandinternalprojectsinordertodeliveruserandserviceauthentication.OpenStackmodulardesignallowsaddingandremovingservicesasneededwhereaminimalproductiondeploymentmayconsistofasfewastwoprojectsanimageandcomputeservice.
Thefollowingdiagramshowstheever-growinglistofOpenStackprojectsandtheinteractionbetweenthem:
TheOpenStackcomponentsandhowtheyinteractwitheachother
Inthischapter,wearegoingtocreateasimpleOpenStackdeploymentontwocomputenodesusingtheKeystone,Glance,Nova,andNeutronprojectsfromtheNewtonreleaseofOpenStack,onanUbuntuXenial16.04server.
NewtonreleaseofOpenStack,onanUbuntuXenial16.04server.
FormoreinformationontheOpenStackproject,pleasevisithttps://www.openstack.org/software/.
PreparingthehostfortheOpenStackdeploymentInthisrecipe,wearegoingtoinstalltheinfrastructurecomponentsthatOpenStackdependson,suchasthedatabaseserver,themessagequeue,andthecachingservice.Theprojectsthatwearegoingtousethroughoutthischapterdependontheseservicesforcommunicationandpersistentstorage.
GettingreadyForthisrecipe,wearegoingtoneedthefollowingcomponents:
AnUbuntuserverwithgreatvirtualizationcapabilitiesAccesstotheinternetforpackageinstallation
Howtodoit...InordertokeepthedeploymentsimpleandfocusontheprovisioningaspectofOpenStack,wearegoingtouseasinglephysicalservertohostallservices.Inproductionenvironments,itisacommonapproachtoseparateeachserviceontotheirownsetofservers,forscalabilityandhighavailability.Byfollowingthestepsoutlinedinthischapter,youshouldbeabletodeployallservicesonmultiplehosts,byreplacingtheIPaddressesandhostnamesintheconfigurationfiles,asneeded.
1. UpdatethehostandinstallthepackagerepositoryfortheNewtonrelease:
root@controller:~#aptinstallsoftware-properties-common
root@controller:~#add-apt-repositorycloud-archive:newton
root@controller:~#aptupdate&&aptdist-upgrade
root@controller:~#reboot
root@controller:~#aptinstallpython-openstackclient
2. InstalltheMariaDBdatabaseserver:
root@controller:~#aptinstallmariadb-serverpython-pymysql
root@controller:~#cat/etc/mysql/mariadb.conf.d/99-openstack.cnf
[mysqld]
bind-address=10.208.130.36
default-storage-engine=innodb
innodb_file_per_table
max_connections=4096
collation-server=utf8_general_ci
character-set-server=utf8
root@controller:~#
ReplacetheIPaddressofthenetworkinterfacethedatabaseserverbindsto,asperyourhost.
3. Restarttheserviceandsecuretheinstallation:
root@controller:~#servicemysqlrestart
root@controller:~#mysql_secure_installation
Forsimplicity,wearegoingtouselxcpasswordasapasswordforallservicesthroughoutthechapter.
4. InstalltheRabbitMQmessagingservice,createanewuser,password,andsetpermissions:
root@controller:~#aptinstallrabbitmq-server
root@controller:~#rabbitmqctladd_useropenstacklxcpassword
Creatinguser"openstack"...
root@controller:~#rabbitmqctlset_permissionsopenstack".*"".*"".*"
Settingpermissionsforuser"openstack"invhost"/"...
root@controller:~#
5. Installandconfigurethememcachedservice:
root@controller:~#aptinstallmemcachedpython-memcache
root@controller:~#sed-i's/127.0.0.1/10.208.130.36/g'
/etc/memcached.conf
root@controller:~#cat/etc/memcached.conf|grep-vi"#"|sed
'/^$/d'
-d
logfile/var/log/memcached.log
-m64
-p11211
-umemcache
-l10.208.130.36
root@controller:~#servicememcachedrestart
root@controller:~#
Howitworks...OpenStackusesaSQLdatabase,suchasMysql/MariaDB/Percona,tostoreinformationaboutitsservices.Inthefollowingrecipes,wearegoingtocreatedatabasesfortheKeystone,Glance,Nova,andNeutronprojects.WeinstallandconfigureMariaDBinsteps1through3.
Themessagingqueueweinstallandconfigureinstep4providesacentralizedwayfortheservicestocommunicatewitheachotherbyproducingandconsumingmessages.OpenStacksupportsafewdifferentmessagebusimplementations,suchasRabbitMQ,Qpid,andZeroMQ.
TheidentityserviceKeystonecachesauthenticationtokensusingthememcacheddaemon.Weinstallandconfigureitinstep5.
InstallingandconfiguringtheOpenStackKeystoneidentityserviceTheidentityserviceprovidedbytheKeystoneprojectisacentralizedpointinordertomanageauthenticationandauthorization,usedbyotherOpenStackcomponents,suchasNovacomputeandtheimageserviceGlance.Keystonealsokeepsacatalogofservicesandtheendpointstheyprovidethattheusercanlocatebysendingqueriestoit.
Inthisrecipe,wearegoingtoinstallandconfigureKeystone,createtwoprojects(aunitofownership)forourservicesandassignusersandrolestothoseprojects.
GettingreadyForthisrecipe,wearegoingtoneedthefollowing:
AnUbuntuServerwithGreatvirtualizationcapabilitiesAccesstotheInternetforpackageinstallationAdatabaseserver,amessagequeue,andmemcachedinstalledandconfigured,asdescribedinthePreparingthehostfortheOpenStackdeploymentrecipe
Howtodoit...Toinstall,configure,createnewprojects,userroles,andcredentials,performthefollowingstepsintheorderpresentedhere:
1. Createthekeystonedatabaseandgrantpermissionstothekeystoneuser:
root@controller:~#mysql-uroot-plxcpassword
MariaDB[(none)]>CREATEDATABASEkeystone;
QueryOK,1rowaffected(0.01sec)
MariaDB[(none)]>GRANTALLPRIVILEGESONkeystone.*TO
'keystone'@'localhost'IDENTIFIEDBY'lxcpassword';
QueryOK,0rowsaffected(0.00sec)
MariaDB[(none)]>GRANTALLPRIVILEGESONkeystone.*TO
'keystone'@'%'IDENTIFIEDBY'lxcpassword';
QueryOK,0rowsaffected(0.01sec)
MariaDB[(none)]>exit
Bye
root@controller:~#
2. InstalltheidentityserviceKeystonefromtherepositoryweconfiguredearlier:
root@controller:~#aptinstallkeystone
3. CreatethefollowingminimalKeystoneconfiguration:
root@controller:~#cat/etc/keystone/keystone.conf
[DEFAULT]
log_dir=/var/log/keystone
[assignment]
[auth]
[cache]
[catalog]
[cors]
[cors.subdomain]
[credential]
[database]
connection=mysql+pymysql://keystone:lxcpassword@controller/keystone
[domain_config]
[endpoint_filter]
[endpoint_policy]
[eventlet_server]
[federation]
[fernet_tokens]
[identity]
[identity_mapping]
[kvs]
[ldap]
[matchmaker_redis]
[memcache]
[oauth1]
[os_inherit]
[oslo_messaging_amqp]
[oslo_messaging_notifications]
[oslo_messaging_rabbit]
[oslo_messaging_zmq]
[oslo_middleware]
[oslo_policy]
[paste_deploy]
[policy]
[profiler]
[resource]
[revoke]
[role]
[saml]
[security_compliance]
[shadow_users]
[signing]
[token]
provider=fernet
[tokenless_auth]
[trust]
[extra_headers]
Distribution=Ubuntu
root@controller:~#
4. PopulatetheKeystonedatabase:
root@controller:~#su-s/bin/sh-c"keystone-managedb_sync"keystone
...
root@controller:~#
5. InitializetheFernetkeyrepositories:
root@controller:~#keystone-managefernet_setup--keystone-userkeystone
--keystone-groupkeystone
root@controller:~#keystone-managecredential_setup--keystone-user
keystone--keystone-groupkeystone
root@controller:~#
6. BootstraptheKeystoneservice:
root@controller:~#keystone-managebootstrap--bootstrap-password
lxcpassword--bootstrap-admin-urlhttp://controller:35357/v3/--
bootstrap-internal-urlhttp://controller:35357/v3/--bootstrap-public-
urlhttp://controller:5000/v3/--bootstrap-region-idRegionOne
root@controller:~#
7. AddthefollowingstanzainApacheandrestarttheservice:
root@controller:~#cat/etc/apache2/apache2.conf
...
ServerNamecontroller
...
root@controller:~#serviceapache2restart
8. DeletethedefaultSQLitedatabasethatKeystoneispackagedwith:
root@controller:~#rm-f/var/lib/keystone/keystone.db
9. Createanadministrativeaccountbydefiningthefollowingenvironmentvariables:
root@controller:~#exportOS_USERNAME=admin
root@controller:~#exportOS_PASSWORD=lxcpassword
root@controller:~#exportOS_PROJECT_NAME=admin
root@controller:~#exportOS_USER_DOMAIN_NAME=default
root@controller:~#exportOS_PROJECT_DOMAIN_NAME=default
root@controller:~#exportOS_AUTH_URL=http://controller:35357/v3
root@controller:~#exportOS_IDENTITY_API_VERSION=3
root@controller:~#
10. CreateaprojectinKeystonefortheservicestouseandlistit:
root@controller:~#openstackprojectcreate--domaindefault--
description"KVMProject"service
+-------------+-----------------------------------+
|Field|Value|
+-------------+-----------------------------------+
|description|KVMProject|
|domain_id|default|
|enabled|True|
|id|9a1a863fe41b42b2955b313f2cca0ef0|
|is_domain|False|
|name|service|
|parent_id|default|
+-------------+-----------------------------------+
root@controller:~#openstackprojectlist
+----------------------------------+---------+
|ID|Name|
+----------------------------------+---------+
|06f4e2d7e384474781803395b24b3af2|admin|
|9a1a863fe41b42b2955b313f2cca0ef0|service|
+----------------------------------+---------+
root@controller:~#
11. CreateanunprivilegedprojectandauserthatwillbeusedbyregularclientsinsteadoftheOpenStackservices:
root@controller:~#openstackprojectcreate--domaindefault--
description"KVMUserProject"kvm
+-------------+----------------------------------+
|Field|Value|
+-------------+----------------------------------+
|description|KVMUserProject|
|domain_id|default|
|enabled|True|
|id|eb9cdc2c2b4e4f098f2d104752970d52|
|is_domain|False|
|name|kvm|
|parent_id|default|
+-------------+----------------------------------+
root@controller:~#
root@controller:~#openstackusercreate--domaindefault--password-
promptkvm
UserPassword:
RepeatUserPassword:
+---------------------+----------------------------------+
|Field|Value|
+---------------------+----------------------------------+
|domain_id|default|
|enabled|True|
|id|1e83e0c8ca194f2e9d8161eb61d21030|
|name|kvm|
|password_expires_at|None|
+---------------------+----------------------------------+
root@controller:~#
12. CreateauserroleandassociateitwiththeKVMprojectanduser:
root@controller:~#openstackrolecreateuser
+-----------+----------------------------------+
|Field|Value|
+-----------+----------------------------------+
|domain_id|None|
|id|331c0b61e9784112874627264f03a058|
|name|user|
+-----------+----------------------------------+
root@controller:~#openstackroleadd--projectkvm--userkvmuser
root@controller:~#
13. ConfiguretheWebServiceGatewayInterface(WSGI)middlewarepipelineforKeystone:
root@controller:~#cat/etc/keystone/keystone-paste.ini
#KeystonePasteDeployconfigurationfile.
[filter:debug]
use=egg:oslo.middleware#debug
[filter:request_id]
use=egg:oslo.middleware#request_id
[filter:build_auth_context]
use=egg:keystone#build_auth_context
[filter:token_auth]
use=egg:keystone#token_auth
[filter:admin_token_auth]
use=egg:keystone#admin_token_auth
[filter:json_body]
use=egg:keystone#json_body
[filter:cors]
use=egg:oslo.middleware#cors
oslo_config_project=keystone
[filter:http_proxy_to_wsgi]
use=egg:oslo.middleware#http_proxy_to_wsgi
[filter:ec2_extension]
use=egg:keystone#ec2_extension
[filter:ec2_extension_v3]
use=egg:keystone#ec2_extension_v3
[filter:s3_extension]
use=egg:keystone#s3_extension
[filter:url_normalize]
use=egg:keystone#url_normalize
[filter:sizelimit]
use=egg:oslo.middleware#sizelimit
[filter:osprofiler]
use=egg:osprofiler#osprofiler
[app:public_service]
use=egg:keystone#public_service
[app:service_v3]
use=egg:keystone#service_v3
[app:admin_service]
use=egg:keystone#admin_service
[pipeline:public_api]
pipeline=corssizelimithttp_proxy_to_wsgiosprofilerurl_normalize
request_idbuild_auth_contexttoken_authjson_bodyec2_extension
public_service
[pipeline:admin_api]
pipeline=corssizelimithttp_proxy_to_wsgiosprofilerurl_normalize
request_idbuild_auth_contexttoken_authjson_bodyec2_extension
s3_extensionadmin_service
[pipeline:api_v3]
pipeline=corssizelimithttp_proxy_to_wsgiosprofilerurl_normalize
request_idbuild_auth_contexttoken_authjson_bodyec2_extension_v3
s3_extensionservice_v3
[app:public_version_service]
use=egg:keystone#public_version_service
[app:admin_version_service]
use=egg:keystone#admin_version_service
[pipeline:public_version_api]
pipeline=corssizelimitosprofilerurl_normalize
public_version_service
[pipeline:admin_version_api]
pipeline=corssizelimitosprofilerurl_normalizeadmin_version_service
[composite:main]
use=egg:Paste#urlmap
/v2.0=public_api
/v3=api_v3
/=public_version_api
[composite:admin]
use=egg:Paste#urlmap
/v2.0=admin_api
/v3=api_v3
/=admin_version_api
root@controller:~#
14. RequestatokenfortheadminandKVMusers:
root@controller:~#openstack--os-auth-urlhttp://controller:35357/v3--
os-project-domain-namedefault--os-user-domain-namedefault--os-
project-nameadmin--os-usernameadmintokenissue
+------------+----------------------------------+
|Field|Value|
+------------+----------------------------------+
|expires|2017-04-2618:29:03+00:00|
|id|gAAAAABZMIdwefsdfB8e4rFk5IALgM4U|
|project_id|123c1e6f33584dd1876c0a34249a6e11|
|user_id|cc14c5dbbd654c438e52d38efaf4f1a6|
+------------+----------------------------------+
root@controller:~#openstack--os-auth-urlhttp://controller:5000/v3--
os-project-domain-namedefault--os-user-domain-namedefault--os-
project-namekvm--os-usernamekvmtokenissue
+------------+----------------------------------+
|Field|Value|
+------------+----------------------------------+
|expires|2017-04-2618:29:52+00:00|
|id|gAAAAABZANkQmInUifl6Up_PzdH_9OHd|
|project_id|10a92eccbad9439d9e56c4edda6b211f|
|user_id|a186b226ed1e4717b25bb978f2bc9958|
+------------+----------------------------------+
root@controller:~#
15. Createthefilesthatwillcontaintheadminandusercredentials:
root@controller:~#catrc.admin
exportOS_PROJECT_DOMAIN_NAME=default
exportOS_USER_DOMAIN_NAME=default
exportOS_PROJECT_NAME=admin
exportOS_USERNAME=admin
exportOS_PASSWORD=lxcpassword
exportOS_AUTH_URL=http://controller:35357/v3
exportOS_IDENTITY_API_VERSION=3
exportOS_IMAGE_API_VERSION=2
root@controller:~#
root@controller:~#catrc.kvm
exportOS_PROJECT_DOMAIN_NAME=default
exportOS_USER_DOMAIN_NAME=default
exportOS_PROJECT_NAME=kvm
exportOS_USERNAME=kvm
exportOS_PASSWORD=lxcpassword
exportOS_AUTH_URL=http://controller:5000/v3
exportOS_IDENTITY_API_VERSION=3
exportOS_IMAGE_API_VERSION=2
root@controller:~#
16. Sourcetheadmincredentialsfile:
root@controller:~#.rc.admin
root@controller:~#
17. Requestanauthenticationtokenfortheadminuser:
root@controller:~#openstacktokenissue
+------------+----------------------------------+
|Field|Value|
+------------+----------------------------------+
|expires|2017-04-2618:30:41+00:00|
|id|gAAAAABZANlBdsu-DTmz6ME2Z8JFKjJM|
|project_id|123c1e6f33584dd1876c0a34249a6e11|
|user_id|cc14c5dbbd654c438e52d38efaf4f1a6|
+------------+----------------------------------+
root@controller:~#
Howitworks...WestartbycreatingtheKeystonedatabaseinMariaDBwiththenecessaryuserpermissionsinstep1.Instep2,weinstalltheKeystonepackage.
Instep3,wecreatetheconfigurationfilefortheservice.Asyoucanseefromtheoutput,mostoftheoptionshavebeenomitted,anddefaultonesareassumed.
Instep4,werunascriptthatpopulatestheKeystonedatabasebycreatingthedatabaseschema.
Keystoneusestokenstoauthenticateandauthorizeusersandservices.Therearedifferenttokenformatsavailableforuse,suchasUUID,PKI,andFernettokens.Forthisdeployment,wearegoingtousetheFernettokens.TheFernettokensdonotneedtobepersistedinabackendstore.Instep5,weinitializetheFernetkeyrepository.
Formoreinformationontheavailableidentitytokens,refertohttp://docs.openstack.org/admin-guide/identity-tokens.html.
Instep6,webootstrapKeystoneandupdatetheApacheconfigurationinstep7andperformsomecleanupinstep8.
Instep9,weexportalistofenvironmentvariablescontainingtheKeystoneuser,password,andendpoint.
Instep10,wecreateourfirstprojectinKeystonethatwillbeusedbytherestoftheservices.Projectsrepresentaunitofownership,whereallresourcesareownedbyaproject.Insteps11and12,wecreateanunprivilegedprojectandassociateduser.
Instep13,weconfiguretheWSGImiddlewarepipelineforKeystone.
Instep14,werequestandobtaintokensfortheadminandKVMusers,andinstep15,wecreatetwoenvironmentvariablefilesthatwecansourcewhenweneedtoswitchbetweenusers.
needtoswitchbetweenusers.
Insteps16and17,wesourcetheadmincredentialsandprojectendpointandobtainanauthorizationtoken.
InstallingandconfiguringtheOpenStackGlanceimageserviceTheGlanceimageserviceprovidesanAPIthatwecanusetodiscover,register,andobtainimagesforvirtualmachines.WhenwelateruseNovacomputetobuildanewKVMinstance,theNovaservicewillsendarequesttoGlancetoobtaintherequestedimagetype.
Inthisrecipe,wearegoingtoinstallGlanceandregisteranewUbuntuimage.
GettingreadyForthisrecipe,wearegoingtoneedthefollowingthings:
AnUbuntuserverwithgreatvirtualizationcapabilitiesAccesstotheinternetforpackageinstallationAdatabaseserver,amessagequeue,andmemcachedinstalledandconfigured,asdescribedinthePreparingthehostfortheOpenStackdeploymentrecipeTheKeystoneservicewedeployedintheInstallingandconfiguringtheOpenStackKeystoneidentityservicerecipe
Howtodoit...Toinstall,configure,andregisteranimagewithGlance,followthestepsoutlinedhere:
1. CreatetheGlancedatabaseanduser:
root@controller:~#mysql-uroot-plxcpassword
MariaDB[(none)]>CREATEDATABASEglance;
QueryOK,1rowaffected(0.00sec)
MariaDB[(none)]>GRANTALLPRIVILEGESONglance.*TO
'glance'@'localhost'IDENTIFIEDBY'lxcpassword';
QueryOK,0rowsaffected(0.00sec)
MariaDB[(none)]>GRANTALLPRIVILEGESONglance.*TO'glance'@'%'
IDENTIFIEDBY'lxcpassword';
QueryOK,0rowsaffected(0.00sec)
MariaDB[(none)]>exit
Bye
root@controller:~#
2. CreatetheGlanceuserandaddittotheadminrole:
root@controller:~#openstackusercreate--domaindefault--password-
promptglance
UserPassword:
RepeatUserPassword:
+---------------------+----------------------------------+
|Field|Value|
+---------------------+----------------------------------+
|domain_id|default|
|enabled|True|
|id|e566c6e2012148daa374cd68077b38df|
|name|glance|
|password_expires_at|None|
+---------------------+----------------------------------+
root@controller:~#openstackroleadd--projectservice--userglance
admin
root@controller:~#
3. CreatetheGlanceservicedefinition:
root@controller:~#openstackservicecreate--nameglance--description
"OpenStackImage"image
+-------------+----------------------------------+
|Field|Value|
+-------------+----------------------------------+
|description|OpenStackImage|
|enabled|True|
|id|d4d42a586551461c8b445b927f2144e1|
|name|glance|
|type|image|
+-------------+----------------------------------+
root@controller:~#
4. CreatetheGlanceAPIendpointsinKeystone:
root@controller:~#openstackendpointcreate--regionRegionOneimage
publichttp://controller:9292
+--------------+----------------------------------+
|Field|Value|
+--------------+----------------------------------+
|enabled|True|
|id|618af0c845194f508752f230364d6e0e|
|interface|public|
|region|RegionOne|
|region_id|RegionOne|
|service_id|d4d42a586551461c8b445b927f2144e1|
|service_name|glance|
|service_type|image|
|url|http://controller:9292|
+--------------+----------------------------------+
root@controller:~#openstackendpointcreate--regionRegionOneimage
internalhttp://controller:9292
+--------------+----------------------------------+
|Field|Value|
+--------------+----------------------------------+
|enabled|True|
|id|991a1b03f7194139b98bafe19acf3518|
|interface|internal|
|region|RegionOne|
|region_id|RegionOne|
|service_id|d4d42a586551461c8b445b927f2144e1|
|service_name|glance|
|service_type|image|
|url|http://controller:9292|
+--------------+----------------------------------+
root@controller:~#openstackendpointcreate--regionRegionOneimage
adminhttp://controller:9292
+--------------+----------------------------------+
|Field|Value|
+--------------+----------------------------------+
|enabled|True|
|id|991a1b03f7194139b98bafe19acf3322|
|interface|admin|
|region|RegionOne|
|region_id|RegionOne|
|service_id|d4d42a586551461c8b445b927f2144e1|
|service_name|glance|
|service_type|image|
|url|http://controller:9292|
+--------------+----------------------------------+
root@controller:~#
5. InstalltheGlanceservice:
root@controller:~#aptinstallglance
6. Configuretheservice:
root@controller:~#cat/etc/glance/glance-api.conf
[DEFAULT]
[cors]
[cors.subdomain]
[database]
connection=mysql+pymysql://glance:lxcpassword@controller/glance
[glance_store]
stores=file,http
default_store=file
filesystem_store_datadir=/var/lib/glance/images/
[image_format]
disk_formats=ami,ari,aki,vhd,vhdx,vmdk,raw,qcow2,vdi,iso,root-tar
[keystone_authtoken]
auth_uri=http://controller:5000
auth_url=http://controller:35357
memcached_servers=controller:11211
auth_type=password
project_domain_name=default
user_domain_name=default
project_name=service
username=glance
password=lxcpassword
[matchmaker_redis]
[oslo_concurrency]
[oslo_messaging_amqp]
[oslo_messaging_notifications]
[oslo_messaging_rabbit]
[oslo_messaging_zmq]
[oslo_middleware]
[oslo_policy]
[paste_deploy]
flavor=keystone
[profiler]
[store_type_location_strategy]
[task]
[taskflow_executor]
root@controller:~#
root@controller:~#cat/etc/glance/glance-registry.conf
[DEFAULT]
[database]
connection=mysql+pymysql://glance:lxcpassword@controller/glance
[keystone_authtoken]
auth_uri=http://controller:5000
auth_url=http://controller:35357
memcached_servers=controller:11211
auth_type=password
project_domain_name=default
user_domain_name=default
project_name=service
username=glance
password=lxcpassword
[matchmaker_redis]
[oslo_messaging_amqp]
[oslo_messaging_notifications]
[oslo_messaging_rabbit]
[oslo_messaging_zmq]
[oslo_policy]
[paste_deploy]
flavor=keystone
[profiler]
root@controller:~#
7. PopulatetheGlancedatabase:
root@controller:~#su-s/bin/sh-c"glance-managedb_sync"glance
...
root@controller:~#
8. StarttheGlanceservicedaemons:
root@controller:~#serviceglance-registryrestart
root@controller:~#serviceglance-apirestart
root@controller:~#
9. DownloadaQCOW2imagefortheUbuntudistribution:
root@controller:~#wgethttps://uec-
images.ubuntu.com/releases/16.04/release-20170330/ubuntu-16.04-server-
cloudimg-amd64-disk1.img
Savingto:‘ubuntu-16.04-server-cloudimg-amd64-disk1.img’
ubuntu-16.04-server-cloudimg-amd64-disk1.img100%
[===================================================>]309.75M31.1MB/s
in13s
2017-04-2617:40:21(24.5MB/s)-‘ubuntu-16.04-server-cloudimg-amd64-
disk1.img’saved[324796416/324796416]
root@controller:~#
10. AddtheimagetotheGlanceservice:
root@controller:~#openstackimagecreate"ubuntu_16.04"--fileubuntu-
16.04-server-cloudimg-amd64-disk1.img--disk-formatqcow2--container-
formatbare--public
+------------------+----------------------------------------------------
--+
|Field|Value
|
+------------------+----------------------------------------------------
--+
|checksum|87b0b7a4b03dd0bb2177d5cc02c80720
|
|container_format|bare
|
|created_at|2017-04-26T17:41:44Z
|
|disk_format|qcow2
|
|file|/v2/images/abce08d2-2f9f-4545-a414-
32019d41c0cd/file|
|id|abce08d2-2f9f-4545-a414-32019d41c0cd
|
|min_disk|0
|
|min_ram|0
|
|name|ubuntu_16.04
|
|owner|123c1e6f33584dd1876c0a34249a6e11
|
|protected|False
|
|schema|/v2/schemas/image
|
|size|324796416
|
|status|active
|
|tags|
|
|updated_at|2017-04-26T17:41:45Z
|
|virtual_size|None
|
|visibility|public
|
+------------------+----------------------------------------------------
--+
root@controller:~#
11. Listtheavailableimagesandtheirlocationonthefilesystem:
root@controller:~#openstackimagelist
+--------------------------------------+--------------+--------+
|ID|Name|Status|
+--------------------------------------+--------------+--------+
|abce08d2-2f9f-4545-a414-32019d41c0cd|ubuntu_16.04|active|
+--------------------------------------+--------------+--------+
root@controller:~#ls-lah/var/lib/glance/images/
drwxr-xr-x2glanceglance4.0KApr2617:51.
drwxr-xr-x4glanceglance4.0KApr2617:32..
-rw-r-----1glanceglance310MApr2617:41abce08d2-2f9f-4545-a414-
32019d41c0cd
root@controller:~#qemu-imginfo/var/lib/glance/images/abce08d2-2f9f-
4545-a414-32019d41c0cd
image:/var/lib/glance/images/abce08d2-2f9f-4545-a414-32019d41c0cd
fileformat:qcow2
virtualsize:2.2G(2361393152bytes)
disksize:310M
cluster_size:65536
Formatspecificinformation:
compat:0.10
refcountbits:16
root@controller:~#
Howitworks...WestartbycreatingtheGlancedatabaseinMariaDBinstep1.
Insteps2and3,wecreatetheuser,role,andservicefortheGlanceproject.Instep4,wedefinetheGlanceAPIserviceendpointsinKeystone.TheNovaserviceandtheOpenStacktoolcanusetheseendpointstoqueryGlanceforavailableimages.
Instep5,weinstalltheGlancepackageandcreateaminimalconfigurationfileinstep6.
Wethencreatethedatabaseschemasinstep7,byexecutingtheglance-managePythonscriptandrestarttheGlanceserviceinstep8.
Instep9,wedownloadaQCOW2Ubuntuimageandaddittotheglanceregistryinstep10.
Finally,instep11,welistthenewlyaddedimageandexamineitonthehostfilesystem.
InstallingandconfiguringtheOpenStackNovacomputeserviceTheOpenStackComputeservice,codenamedNova,managesapoolofcomputeresourcesandthevirtualmachinesrunningonthem.Novaisasuiteofservicestocreateandmanagethelifecycleofvirtualmachines.WewilluseNovatocreate,examine,stop,delete,andmigrateKVMinstances.
FormoreinformationonthevariousNovaservices,referto:http://docs.openstack.org/developer/nova/.
Inthisrecipe,wearegoingtoinstallandconfigurethefollowingNovacomponents:
nova-api:ThisistheservicethatacceptsandrespondstouserrequeststhroughaRESTfulAPI.Wewilluseitwhencreating,running,stopping,andmigratingKVMinstances.nova-scheduler:Thisistheservicethatmakesdecisionsonwheretoprovisioninstance,basedonfilters,suchasavailablememory,disk,andCPUresources.nova-compute:ThisistheservicethatrunsonthecomputehostsandisresponsibleformanagingthelifecycleoftheKVMinstance,fromprovisioningtodeletion.nova-conductor:ThisistheservicethatsitsbetweentheNovadatabasewecreatedearlierandthenova-computeservice.nova-consoleauth:Thisistheservicethatauthorizestokensforusersthatwanttousevariousconsolestoconnecttothevirtualmachines.nova-novncproxy:ThisistheservicethatgrantsaccesstoinstancesrunningVNC.
GettingreadyForthisrecipe,wearegoingtoneed:
AnUbuntuserverwithgreatvirtualizationcapabilitiesAccesstotheinternetforpackageinstallationAdatabaseserver,amessagequeue,andmemcachedinstalledandconfigured,asdescribedinthePreparingthehostfortheOpenStackdeploymentrecipeTheKeystoneservicewedeployedintheInstallingandconfiguringtheOpenStackKeystoneidentityservicerecipeTheGlanceservicewedeployedintheInstallingandconfiguringtheOpenStackGlanceimageservicerecipe
Howtodoit...ToinstallandconfiguretheNovaservicesoutlinedearlier,performthefollowingsteps:
1. CreatetheNovadatabaseanduserinMariaDB:
root@controller:~#mysql-uroot-plxcpassword
MariaDB[(none)]>CREATEDATABASEnova_api;
QueryOK,1rowaffected(0.00sec)
MariaDB[(none)]>CREATEDATABASEnova;
QueryOK,1rowaffected(0.00sec)
MariaDB[(none)]>GRANTALLPRIVILEGESONnova_api.*TO
'nova'@'localhost'IDENTIFIEDBY'lxcpassword';
QueryOK,0rowsaffected(0.03sec)
MariaDB[(none)]>GRANTALLPRIVILEGESONnova_api.*TO'nova'@'%'
IDENTIFIEDBY'lxcpassword';
QueryOK,0rowsaffected(0.00sec)
MariaDB[(none)]>GRANTALLPRIVILEGESONnova.*TO'nova'@'localhost'
IDENTIFIEDBY'lxcpassword';
QueryOK,0rowsaffected(0.00sec)
MariaDB[(none)]>GRANTALLPRIVILEGESONnova.*TO'nova'@'%'
IDENTIFIEDBY'lxcpassword';
QueryOK,0rowsaffected(0.00sec)
MariaDB[(none)]>exit
Bye
root@controller:~#
2. CreatetheNovauserandaddittotheadminroleintheIdentityservice:
root@controller:~#openstackusercreate--domaindefault--password-
promptnova
UserPassword:
RepeatUserPassword:
+---------------------+----------------------------------+
|Field|Value|
+---------------------+----------------------------------+
|domain_id|default|
|enabled|True|
|id|038aa8840aca449dbd3e653c5d2c5a08|
|name|nova|
|password_expires_at|None|
+---------------------+----------------------------------+
root@controller:~#openstackroleadd--projectservice--usernova
admin
root@controller:~#
3. CreatetheNovaserviceandendpoints:
root@controller:~#openstackservicecreate--namenova--description
"OpenStackCompute"compute
+-------------+----------------------------------+
|Field|Value|
+-------------+----------------------------------+
|description|OpenStackCompute|
|enabled|True|
|id|04132edd7f654f56ba0cc23ac182c9aa|
|name|nova|
|type|compute|
+-------------+----------------------------------+
root@controller:~#openstackendpointcreate--regionRegionOnecompute
publichttp://controller:8774/v2.1/%(tenant_id)s
+--------------+-------------------------------------------+
|Field|Value|
+--------------+-------------------------------------------+
|enabled|True|
|id|5fc54236c324412db135dff88807e820|
|interface|public|
|region|RegionOne|
|region_id|RegionOne|
|service_id|04132edd7f654f56ba0cc23ac182c9aa|
|service_name|nova|
|service_type|compute|
|url|http://controller:8774/v2.1/%(tenant_id)s|
+--------------+-------------------------------------------+
root@controller:~#openstackendpointcreate--regionRegionOnecompute
internalhttp://controller:8774/v2.1/%(tenant_id)s
+--------------+-------------------------------------------+
|Field|Value|
+--------------+-------------------------------------------+
|enabled|True|
|id|a0f623ed345e4bdb8fced929b7fe6b3f|
|interface|internal|
|region|RegionOne|
|region_id|RegionOne|
|service_id|04132edd7f654f56ba0cc23ac182c9aa|
|service_name|nova|
|service_type|compute|
|url|http://controller:8774/v2.1/%(tenant_id)s|
+--------------+-------------------------------------------+
root@controller:~#openstackendpointcreate--regionRegionOnecompute
adminhttp://controller:8774/v2.1/%(tenant_id)s
+--------------+-------------------------------------------+
|Field|Value|
+--------------+-------------------------------------------+
|enabled|True|
|id|3964db0d281545acbaa6c18abc44a216|
|interface|admin|
|region|RegionOne|
|region_id|RegionOne|
|service_id|04132edd7f654f56ba0cc23ac182c9aa|
|service_name|nova|
|service_type|compute|
|url|http://controller:8774/v2.1/%(tenant_id)s|
+--------------+-------------------------------------------+
root@controller:~#
4. InstalltheNovapackagesthatwillprovidetheAPI,theconductor,the
console,andtheschedulerservices:
root@controller:~#aptinstallnova-apinova-conductornova-consoleauth
nova-novncproxynova-scheduler
5. CreatetheNovaconfigurationfile:
root@controller:~#cat/etc/nova/nova.conf
[DEFAULT]
dhcpbridge_flagfile=/etc/nova/nova.conf
dhcpbridge=/usr/bin/nova-dhcpbridge
log-dir=/var/log/nova
state_path=/var/lib/nova
force_dhcp_release=True
verbose=True
ec2_private_dns_show_ip=True
enabled_apis=osapi_compute,metadata
transport_url=rabbit://openstack:lxcpassword@controller
auth_strategy=keystone
my_ip=10.208.132.45
use_neutron=True
firewall_driver=nova.virt.firewall.NoopFirewallDriver
[database]
connection=mysql+pymysql://nova:lxcpassword@controller/nova
[api_database]
connection=mysql+pymysql://nova:lxcpassword@controller/nova_api
[oslo_concurrency]
lock_path=/var/lib/nova/tmp
[libvirt]
use_virtio_for_bridges=True
[wsgi]
api_paste_config=/etc/nova/api-paste.ini
[keystone_authtoken]
auth_uri=http://controller:5000
auth_url=http://controller:35357
memcached_servers=controller:11211
auth_type=password
project_domain_name=default
user_domain_name=default
project_name=service
username=nova
password=lxcpassword
[vnc]
vncserver_listen=$my_ip
vncserver_proxyclient_address=$my_ip
[glance]
api_servers=http://controller:9292
root@controller:~#
6. Createthedatabasetables:
root@controller:~#su-s/bin/sh-c"nova-manageapi_dbsync"nova
...
root@controller:~#su-s/bin/sh-c"nova-managedbsync"nova
...
root@controller:~#
7. StarttheNovaservices:
root@controller:~#servicenova-apirestart
root@controller:~#servicenova-consoleauthrestart
root@controller:~#servicenova-schedulerrestart
root@controller:~#servicenova-conductorrestart
root@controller:~#servicenova-novncproxyrestart
root@controller:~#
8. Installthenova-computeservice,whichwillprovisionKVMinstances:
root@controller:~#aptinstallnova-compute
9. UpdatetheNovaconfigurationfile,asfollows:
root@controller:~#cat/etc/nova/nova.conf
[DEFAULT]
dhcpbridge_flagfile=/etc/nova/nova.conf
dhcpbridge=/usr/bin/nova-dhcpbridge
log-dir=/var/log/nova
state_path=/var/lib/nova
force_dhcp_release=True
verbose=True
ec2_private_dns_show_ip=True
enabled_apis=osapi_compute,metadata
transport_url=rabbit://openstack:lxcpassword@controller
auth_strategy=keystone
my_ip=10.208.132.45
use_neutron=True
firewall_driver=nova.virt.firewall.NoopFirewallDriver
compute_driver=libvirt.LibvirtDriver
[database]
connection=mysql+pymysql://nova:lxcpassword@controller/nova
[api_database]
connection=mysql+pymysql://nova:lxcpassword@controller/nova_api
[oslo_concurrency]
lock_path=/var/lib/nova/tmp
[libvirt]
use_virtio_for_bridges=True
[wsgi]
api_paste_config=/etc/nova/api-paste.ini
[keystone_authtoken]
auth_uri=http://controller:5000
auth_url=http://controller:35357
memcached_servers=controller:11211
auth_type=password
project_domain_name=default
user_domain_name=default
project_name=service
username=nova
password=lxcpassword
[vnc]
enabled=True
vncserver_listen=$my_ip
vncserver_proxyclient_address=$my_ip
novncproxy_base_url=http://controller:6080/vnc_auto.html
[glance]
api_servers=http://controller:9292
root@controller:~#
10. Specifythevirtualizationdrivertobeused:
root@controller:~#cat/etc/nova/nova-compute.conf
[DEFAULT]
compute_driver=libvirt.LibvirtDriver
[libvirt]
virt_type=kvm
root@controller:~#
11. Restartthenova-computeserviceandlisttheavailableservices:
root@controller:~#servicenova-computerestart
root@controller:~#openstackcomputeservicelist
+----+------------------+------------+----------+---------+-------+-----
-----------------+
|ID|Binary|Host|Zone|Status|State|
UpdatedAt|
+----+------------------+------------+----------+---------+-------+-----
-----------------+
|8|nova-consoleauth|controller|internal|enabled|up|
2017-04-26T17:58|
|9|nova-scheduler|controller|internal|enabled|up|
2017-04-26T17:58|
|10|nova-conductor|controller|internal|enabled|up|
2017-04-26T17:58|
|15|nova-compute|controller|nova|enabled|up|None
|
+----+------------------+------------+----------+---------+-------+-----
-----------------+
root@controller:~#pgrep-lfnova|uniq-f1
14110nova-consoleaut
14176nova-conductor
14239nova-novncproxy
20877nova-api
20994nova-scheduler
21065nova-compute
root@controller:~#
Howitworks...Insteps1and2,wecreatetheNovadatabaseanduserinMariaDB.Instep3,wecreatetheserviceandendpointsthatwecanusetosendAPIcallsto.
Insteps4and5,weinstallthepackagesfortheNovaservicesandproceedtocreateasimpleconfigurationfile.
Wecreatethedatabasetableschemasinstep6andstarttheNovaservicesinstep7.
Forthisexampledeployment,weareusingasinglenodetorunalloftheOpenStackservicesweareinterestedin.However,youcanuseasecondnodejustforthenova-computeservicethatwillprovisiontheKVMvirtualmachines.Weinstallthenova-computeserviceinstep8,updatetheconfigurationfile,andexaminethenova-computeserviceexternalconfigurationinsteps9and10.
WefinishtherecipebymakingsurethatallNovaserviceshavebeenconfiguredandrunninginstep11.
InstallingandconfiguringtheOpenStackNeutronnetworkingserviceTheOpenStackNeutronprojectprovidesnetworkingasaservicetomanagethenetworkingbetweenvirtualinstances.Itisresponsibleforsettingupvirtualinterfaces,configuringasoftwarebridge,creatingroutes,andmanagingIPaddressing.
FormoreinformationonthevariousNeutronservices,refertohttps://docs.openstack.org/security-guide/networking/architecture.html.
Inthisrecipe,wearegoingtoinstallandconfigurethefollowingNeutroncomponents:
neutron-server:ThisistheservicethatprovidesAPItodynamicallyrequestandconfigurevirtualnetworksneutron-plugin-ml2:Thisistheframeworkthatenablestheuseofvariousnetworktechnologies,suchastheLinuxBridge,OpenvSwitch,GRE,andVXLAN,thatwesawinearlierchaptersneutron-linuxbridge-agent:ThisistheservicethatprovidestheLinuxbridgepluginagentneutron-l3-agent:ThisisthedaemonthatperformsforwardingandNATfunctionalitybetweensoftware-definednetworks,bycreatingvirtualroutersneutron-dhcp-agent:ThisistheservicethatcontrolstheDHCPdaemon,whichassignsIPaddressestotheinstancesrunningonthecomputenodesneutron-metadata-agent:ThisistheservicethatpassesinstancemetadatatoNeutron
Inearlierrecipes,weconfiguredandusedtheLinuxbridgeandOpenvSwitchmanuallyandlaterdelegatedthemanagementofthenetworkingtolibvirt.OpenStackNeutronintegrateswithlibvirtandautomatesthisprocessevenfurtherbyexposingAPIcallsthatotherserviceslikeNovacanutilize.
furtherbyexposingAPIcallsthatotherserviceslikeNovacanutilize.
GettingreadyForthisrecipe,wearegoingtoneed:
AnUbuntuserverwithgreatvirtualizationcapabilitiesAccesstotheinternetforpackageinstallationAdatabaseserver,amessagequeue,andmemcachedinstalledandconfigured,asdescribedinthePreparingthehostfortheOpenStackdeploymentrecipeTheKeystoneservicewedeployedintheInstallingandconfiguringtheOpenStackKeystoneidentityservicerecipeTheNovaservicesweconfiguredintheInstallingandconfiguringtheOpenStackNovacomputeservicerecipe
Howtodoit...Toinstall,configure,andcreateanetworkmanagedbyNeutron,executethefollowingsteps:
1. CreatetheNeutrondatabase:
root@controller:~#mysql-uroot-plxcpassword
MariaDB[(none)]>CREATEDATABASEneutron;
QueryOK,1rowaffected(0.00sec)
MariaDB[(none)]>GRANTALLPRIVILEGESONneutron.*TO
'neutron'@'localhost'IDENTIFIEDBY'lxcpassword';
QueryOK,0rowsaffected(0.00sec)
MariaDB[(none)]>GRANTALLPRIVILEGESONneutron.*TO'neutron'@'%'
IDENTIFIEDBY'lxcpassword';
QueryOK,0rowsaffected(0.00sec)
MariaDB[(none)]>exit
Bye
root@controller:~#
2. CreatetheNeutronuserandaddittotheadminroleinKeystone:
root@controller:~#openstackusercreate--domaindefault--password-
promptneutron
UserPassword:
RepeatUserPassword:
+---------------------+----------------------------------+
|Field|Value|
+---------------------+----------------------------------+
|domain_id|default|
|enabled|True|
|id|02934ad74c94461482b95fff32d36894|
|name|neutron|
|password_expires_at|None|
+---------------------+----------------------------------+
root@controller:~#openstackroleadd--projectservice--userneutron
admin
root@controller:~#
3. CreatetheNeutronserviceandendpoints:
root@controller:~#openstackservicecreate--nameneutron--description
"OpenStackNetworking"network
+-------------+----------------------------------+
|Field|Value|
+-------------+----------------------------------+
|description|OpenStackNetworking|
|enabled|True|
|id|24b32d32d4b54e3ab2d785a1817b8e7e|
|name|neutron|
|type|network|
+-------------+----------------------------------+
root@controller:~#openstackendpointcreate--regionRegionOnenetwork
publichttp://controller:9696
+--------------+----------------------------------+
|Field|Value|
+--------------+----------------------------------+
|enabled|True|
|id|544821d511e04847869fc601f2ebf0f7|
|interface|public|
|region|RegionOne|
|region_id|RegionOne|
|service_id|24b32d32d4b54e3ab2d785a1817b8e7e|
|service_name|neutron|
|service_type|network|
|url|http://controller:9696|
+--------------+----------------------------------+
root@controller:~#openstackendpointcreate--regionRegionOnenetwork
internalhttp://controller:9696
+--------------+----------------------------------+
|Field|Value|
+--------------+----------------------------------+
|enabled|True|
|id|05e276ec603f424f85be8705ce7fe86a|
|interface|internal|
|region|RegionOne|
|region_id|RegionOne|
|service_id|24b32d32d4b54e3ab2d785a1817b8e7e|
|service_name|neutron|
|service_type|network|
|url|http://controller:9696|
+--------------+----------------------------------+
root@controller:~#openstackendpointcreate--regionRegionOnenetwork
adminhttp://controller:9696
+--------------+----------------------------------+
|Field|Value|
+--------------+----------------------------------+
|enabled|True|
|id|836b4309186146fb9143544490cd0bc1|
|interface|admin|
|region|RegionOne|
|region_id|RegionOne|
|service_id|24b32d32d4b54e3ab2d785a1817b8e7e|
|service_name|neutron|
|service_type|network|
|url|http://controller:9696|
+--------------+----------------------------------+
root@controller:~#
4. InstalltheNeutronpackages:
root@controller:~#aptinstallneutron-serverneutron-plugin-ml2
neutron-linuxbridge-agentneutron-l3-agentneutron-dhcp-agentneutron-
metadata-agent
...
root@controller:~#
5. CreatetheNeutronconfigurationfile:
root@controller:~#cat/etc/neutron/neutron.conf
[DEFAULT]
core_plugin=ml2
service_plugins=router
allow_overlapping_ips=True
transport_url=rabbit://openstack:lxcpassword@controller
auth_strategy=keystone
notify_nova_on_port_status_changes=True
notify_nova_on_port_data_changes=True
[agent]
root_helper=sudo/usr/bin/neutron-rootwrap/etc/neutron/rootwrap.conf
[cors]
[cors.subdomain]
[database]
connection=mysql+pymysql://neutron:lxcpassword@controller/neutron
[keystone_authtoken]
auth_uri=http://controller:5000
auth_url=http://controller:35357
memcached_servers=controller:11211
auth_type=password
project_domain_name=default
user_domain_name=default
project_name=service
username=neutron
password=lxcpassword
[matchmaker_redis]
[nova]
auth_url=http://controller:35357
auth_type=password
project_domain_name=default
user_domain_name=default
region_name=RegionOne
project_name=service
username=nova
password=lxcpassword
[oslo_concurrency]
[oslo_messaging_amqp]
[oslo_messaging_notifications]
[oslo_messaging_rabbit]
[oslo_messaging_zmq]
[oslo_policy]
[qos]
[quotas]
[ssl]
root@controller:~#
6. DefinethenetworktypeandextensionsthatwearegoingtousewithNeutron:
root@controller:~#cat/etc/neutron/plugins/ml2/ml2_conf.ini
[DEFAULT]
[ml2]
type_drivers=flat,vlan,vxlan
tenant_network_types=vxlan
mechanism_drivers=linuxbridge,l2population
extension_drivers=port_security
[ml2_type_flat]
flat_networks=provider
[ml2_type_geneve]
[ml2_type_gre]
[ml2_type_vlan]
[ml2_type_vxlan]
vni_ranges=1:1000
[securitygroup]
enable_ipset=True
root@controller:~#
7. DefinetheinterfacethatwillbeaddedtothesoftwarebridgeandtheIPthebridgewillbeboundto,replacingtheIPaddressandinterfacename(eth1inthisexample)asneeded:
root@controller:~#cat/etc/neutron/plugins/ml2/linuxbridge_agent.ini
[DEFAULT]
[agent]
[linux_bridge]
physical_interface_mappings=provider:eth1
[securitygroup]
enable_security_group=True
firewall_driver=
neutron.agent.linux.iptables_firewall.IptablesFirewallDriver
[vxlan]
enable_vxlan=True
local_ip=10.208.132.45
l2_population=True
root@controller:~#
8. ConfiguretheLayer3agentasfollows:
root@controller:~#cat/etc/neutron/l3_agent.ini
[DEFAULT]
interface_driver=neutron.agent.linux.interface.BridgeInterfaceDriver
[AGENT]
root@controller:~#
9. ConfiguretheDHCPagent:
root@controller:~#cat/etc/neutron/dhcp_agent.ini
[DEFAULT]
interface_driver=neutron.agent.linux.interface.BridgeInterfaceDriver
dhcp_driver=neutron.agent.linux.dhcp.Dnsmasq
enable_isolated_metadata=True
[AGENT]
root@controller:~#
10. Createaconfigurationforthemetadataagent:
root@controller:~#cat/etc/neutron/metadata_agent.ini
[DEFAULT]
nova_metadata_ip=controller
metadata_proxy_shared_secret=lxcpassword
[AGENT]
[cache]
root@controller:~#
11. UpdatetheconfigurationfilefortheNovaservicestoincludeNeutron.Acompletelyminimalworkingexamplefollowstolookasthefollowing:
root@controller:~#cat/etc/nova/nova.conf
[DEFAULT]
dhcpbridge_flagfile=/etc/nova/nova.conf
dhcpbridge=/usr/bin/nova-dhcpbridge
log-dir=/var/log/nova
state_path=/var/lib/nova
force_dhcp_release=True
verbose=True
ec2_private_dns_show_ip=True
enabled_apis=osapi_compute,metadata
transport_url=rabbit://openstack:lxcpassword@controller
auth_strategy=keystone
my_ip=10.208.132.45
use_neutron=True
firewall_driver=nova.virt.firewall.NoopFirewallDriver
compute_driver=libvirt.LibvirtDriver
scheduler_default_filters=RetryFilter,AvailabilityZoneFilter,
RamFilter,ComputeFilter,ComputeCapabilitiesFilter,
ImagePropertiesFilter,ServerGroupAntiAffinityFilter,
ServerGroupAffinityFilter
[database]
connection=mysql+pymysql://nova:lxcpassword@controller/nova
[api_database]
connection=mysql+pymysql://nova:lxcpassword@controller/nova_api
[oslo_concurrency]
lock_path=/var/lib/nova/tmp
[libvirt]
use_virtio_for_bridges=True
[wsgi]
api_paste_config=/etc/nova/api-paste.ini
[keystone_authtoken]
auth_uri=http://controller:5000
auth_url=http://controller:35357
memcached_servers=controller:11211
auth_type=password
project_domain_name=default
user_domain_name=default
project_name=service
username=nova
password=lxcpassword
[vnc]
enabled=True
vncserver_listen=$my_ip
vncserver_proxyclient_address=$my_ip
novncproxy_base_url=http://controller:6080/vnc_auto.html
[glance]
api_servers=http://controller:9292
[libvirt]
virt_type=kvm
[neutron]
url=http://controller:9696
auth_url=http://controller:35357
auth_type=password
project_domain_name=default
user_domain_name=default
region_name=RegionOne
project_name=service
username=neutron
password=lxcpassword
service_metadata_proxy=True
metadata_proxy_shared_secret=lxcpassword
root@controller:~#
12. PopulatetheNeutrondatabases:
root@controller:~#su-s/bin/sh-c"neutron-db-manage--config-file
/etc/neutron/neutron.conf--config-file
/etc/neutron/plugins/ml2/ml2_conf.iniupgradehead"neutron
INFO[alembic.runtime.migration]ContextimplMySQLImpl.
INFO[alembic.runtime.migration]Willassumenon-transactionalDDL.
Runningupgradeforneutron...
INFO[alembic.runtime.migration]ContextimplMySQLImpl.
INFO[alembic.runtime.migration]Willassumenon-transactionalDDL.
INFO[alembic.runtime.migration]Runningupgrade->kilo,kilo_initial
...
root@controller:~#
13. RestartallNeutronservicesandNova:
root@controller:~#serviceneutron-serverrestart
root@controller:~#serviceneutron-linuxbridge-agentrestart
root@controller:~#serviceneutron-dhcp-agentrestart
root@controller:~#serviceneutron-metadata-agentrestart
root@controller:~#serviceneutron-l3-agentrestart
root@controller:~#servicenova-apirestart
root@controller:~#servicenova-computerestart
root@controller:~#
14. VerifythattheNeutronserviceshavebeenregistered:
root@controller:~#openstacknetworkagentlist
+--------------------------------------+--------------------+------+----
---------------+-------+-------+---------------------------+
|ID|AgentType|Host|AvailabilityZone|Alive|State|Binary|
+--------------------------------------+--------------------+------+----
---------------+-------+-------+---------------------------+
|9242d71d-de25-4b3e-8aa8-62691ef72001|Linuxbridgeagent|kvm2|
None|True|UP|neutron-linuxbridge-agent|
|92b601de-06df-4b10-88c7-8f27bc48f6ab|L3agent|kvm2|nova|True|
UP|neutron-l3-agent|
|d249f986-9b26-4c5d-8ea5-311daf3b395d|DHCPagent|kvm2|nova|True
|UP|neutron-dhcp-agent|
|f3cac79b-a7c3-4672-b846-9268f2d58706|Metadataagent|kvm2|None|
True|UP|neutron-metadata-agent|
+--------------------------------------+--------------------+------+----
---------------+-------+-------+---------------------------+
root@controller:~#
15. Createanewnetwork:
root@controller:~#openstacknetworkcreatenat
+---------------------------+--------------------------------------+
|Field|Value|
+---------------------------+--------------------------------------+
|admin_state_up|UP|
|availability_zone_hints||
|availability_zones||
|created_at|2017-04-26T18:17:24Z|
|description||
|headers||
|id|b7ccb514-21fc-4ced-b74f-026e7e358bba|
|ipv4_address_scope|None|
|ipv6_address_scope|None|
|mtu|1450|
|name|nat|
|port_security_enabled|True|
|project_id|123c1e6f33584dd1876c0a34249a6e11|
|project_id|123c1e6f33584dd1876c0a34249a6e11|
|provider:network_type|vxlan|
|provider:physical_network|None|
|provider:segmentation_id|37|
|revision_number|3|
|router:external|Internal|
|shared|False|
|status|ACTIVE|
|subnets||
|tags|[]|
|updated_at|2017-04-26T18:17:24Z|
+---------------------------+--------------------------------------+
root@controller:~#
16. DefinetheDNSserver,thedefaultgateway,andthesubnetrangethatwillbeassignedtotheguests:
root@controller:~#openstacksubnetcreate--networknat--dns-
nameserver8.8.8.8--gateway192.168.0.1--subnet-range192.168.0.0/24
nat
+-------------------+--------------------------------------+
|Field|Value|
+-------------------+--------------------------------------+
|allocation_pools|192.168.0.2-192.168.0.254|
|cidr|192.168.0.0/24|
|created_at|2017-04-26T18:17:41Z|
|description||
|dns_nameservers|8.8.8.8|
|enable_dhcp|True|
|gateway_ip|192.168.0.1|
|headers||
|host_routes||
|id|296250a7-f241-4f84-adbb-64a45c943094|
|ip_version|4|
|ipv6_address_mode|None|
|ipv6_ra_mode|None|
|name|nat|
|network_id|b7ccb514-21fc-4ced-b74f-026e7e358bba|
|project_id|123c1e6f33584dd1876c0a34249a6e11|
|project_id|123c1e6f33584dd1876c0a34249a6e11|
|revision_number|2|
|service_types|[]|
|subnetpool_id|None|
|updated_at|2017-04-26T18:17:41Z|
+-------------------+--------------------------------------+
root@controller:~#
17. UpdatethesubnetinformationinNeutron:
root@controller:~#neutronnet-updatenat--router:external
Updatednetwork:nat
root@controller:~#
18. Createanewsoftwarerouter:
root@controller:~#openstackroutercreaterouter
+-------------------------+--------------------------------------+
|Field|Value|
+-------------------------+--------------------------------------+
|admin_state_up|UP|
|availability_zone_hints||
|availability_zones||
|created_at|2017-04-26T18:18:05Z|
|description||
|external_gateway_info|null|
|flavor_id|None|
|headers||
|id|f9cd8c96-a53c-4585-ad21-0e409f3b4d70|
|name|router|
|project_id|10a92eccbad9439d9e56c4edda6b211f|
|project_id|10a92eccbad9439d9e56c4edda6b211f|
|revision_number|3|
|routes||
|status|ACTIVE|
|updated_at|2017-04-26T18:18:05Z|
+-------------------------+--------------------------------------+
root@controller:~#
19. Astheadminuser,addthesubnetwecreatedearliertotherouterasaninterface:
root@controller:~#.rc.admin
root@controller:~#neutronrouter-interface-addrouternat
Addedinterface2e1e2fd3-1819-489b-a21f-7005862f9de7torouterrouter.
root@controller:~#
20. ListthenetworknamespacesthatNeutroncreated:
root@controller:~#ipnetns
qrouter-f9cd8c96-a53c-4585-ad21-0e409f3b4d70
qdhcp-b7ccb514-21fc-4ced-b74f-026e7e358bba
root@controller:~#
21. Listtheportsonthesoftwarerouter:
root@controller:~#neutronrouter-port-listrouter
+--------------------------------------+------+-------------------+-----
------------------------------------------------------------------------
-------+
|id|name|mac_address|fixed_ips|
+--------------------------------------+------+-------------------+-----
------------------------------------------------------------------------
-------+
|2e1e2fd3-1819-489b-a21f-7005862f9de7||fa:16:3e:0e:db:14|
{"subnet_id":"296250a7-f241-4f84-adbb-64a45c943094","ip_address":
"192.168.0.1"}|
+--------------------------------------+------+-------------------+-----
------------------------------------------------------------------------
-------+
root@controller:~#
22. ListtheNeutronnetworksandensurethattheonewecreatedearlierispresent:
root@controller:~#openstacknetworklist
+--------------------------------------+------+-------------------------
-------------+
|ID|Name|Subnets
|
+--------------------------------------+------+-------------------------
-------------+
|b7ccb514-21fc-4ced-b74f-026e7e358bba|nat|296250a7-f241-4f84-adbb-
64a45c943094|
+--------------------------------------+------+-------------------------
-------------+
root@controller:~#
Howitworks...WestartthisrecipebycreatinganewdatabaseforNeutroninstep1.WethenproceedtocreatetheuserfortheNeutronserviceandaddittotheadminrolefortheservice.Insteps2and3,wedefinetheserviceendpointsthatwillbeexposedforNovatouse.Insteps4and5,weinstalltheNeutronpackagesandcreateabasicconfigurationfile.Instep6,weselecttheVXLANtypeofnetworkingforthisexampledeployment.Insteps7,8,9,and10,weconfigurethebridgeagent,thelayer3agent,theDHCPagent,andthemetadataagent.
Instep11,weupdatetheNovaconfigurationfiletocontainasectionabouttheNeutronservice.Instep12,wecreatethedatabaseschemaandrestartallNeutronservicesinstep13,includingnova-apiandnova-compute.
Instep14,weverifythattheNeutronserviceshavebeenregisteredandproceedtocreateanewnetworkinstep15.
Instep18,wedefineanewsoftwarerouter.Weaddthesubnetwecreatedearliertoitinstep19,thenverifythenewrouteconfigurationinstep21.
Thelaststep22ensuresthatthenetworkwedefinedearlierisactive.
BuildingandinspectingKVMinstanceswithOpenStackInthisrecipe,wearegoingtobuildourfirstKVMinstanceusingtheOpenStackinfrastructureweputinplaceinthepreviousrecipes.BuildinganewKVMinstanceconsistsofthefollowingsteps:
1. WesendanAPIcalltothenova-apiservice.2. Thenova-apiservicerequestsatargetcomputehostfromthenova-scheduler
service.3. nova-schedulerpicksanavailablecomputehost,basedontheconfigured
filters,suchasavailablememory,disk,andCPUutilization.4. Oncethenova-schedulerselectsanappropriatehost,thenova-computeservice
ontheselectedhost,requeststheimagefromtheGlancerepository,ifnotalreadycachedlocally.Oncetheimageisonthenewserver,nova-computebuildsthenewKVMinstance.
GettingreadyForthisrecipe,wearegoingtoneedthefollowingthings:
Adatabaseserver,amessagequeue,andmemcachedinstalledandconfigured,asdescribedinthePreparingthehostfortheOpenStackdeploymentrecipe.TheGlanceservicewithanavailableimage.FormoreinformationonhowtodeployGlanceandaddanewimage,refertotheInstallingandconfiguringtheOpenStackGlanceimageservicerecipe.TheKeystoneservicewedeployedintheInstallingandconfiguringtheOpenStackKeystoneidentityservicerecipe.TheNovaservicesweconfiguredintheInstallingandconfiguringtheOpenStackNovacomputeservicerecipe.TheNeutronservicethatwasdeployedintheInstallingandconfiguringtheOpenStackNeutronnetworkingservicerecipe.
Howtodoit...TobuildanewKVMinstanceusingtheOpenStackcommand-lineinterface(CLI),performthefollowingsteps:
1. EnsurethatwehaveanavailableGlanceimagetouse:
root@controller:~#openstackimagelist
+--------------------------------------+--------------+--------+
|ID|Name|Status|
+--------------------------------------+--------------+--------+
|abce08d2-2f9f-4545-a414-32019d41c0cd|ubuntu_16.04|active|
+--------------------------------------+--------------+--------+
root@controller:~#
2. Createanewinstanceflavortype:
root@controller:~#openstackflavorcreate--id0--vcpus1--ram1024-
-disk5000kvm.medium
+----------------------------+------------+
|Field|Value|
+----------------------------+------------+
|OS-FLV-DISABLED:disabled|False|
|OS-FLV-EXT-DATA:ephemeral|0|
|disk|5000|
|id|0|
|name|kvm.medium|
|os-flavor-access:is_public|True|
|properties||
|ram|1024|
|rxtx_factor|1.0|
|swap||
|vcpus|1|
+----------------------------+------------+
root@controller:~#
root@controller:~#openstackflavorlist
+----+------------+------+------+-----------+-------+-----------+
|ID|Name|RAM|Disk|Ephemeral|VCPUs|IsPublic|
+----+------------+------+------+-----------+-------+-----------+
|0|kvm.medium|1024|5000|0|1|True|
+----+------------+------+------+-----------+-------+-----------+
root@controller:~#
3. CreateanewSSHkey-pair:
root@controller:~#openstackkeypaircreate--public-key
~/.ssh/kvm_rsa.pubkvmkey
+-------------+-------------------------------------------------+
|Field|Value|
+-------------+-------------------------------------------------+
|fingerprint|e9:7e:e6:05:8b:a4:31:c3:5e:41:65:0e:29:23:eb:2a|
|name|kvmkey|
|user_id|cc14c5dbbd654c438e52d38efaf4f1a6|
+-------------+-------------------------------------------------+
root@controller:~#openstackkeypairlist
+--------+-------------------------------------------------+
|Name|Fingerprint|
+--------+-------------------------------------------------+
|kvmkey|e9:7e:e6:05:8b:a4:31:c3:5e:41:65:0e:29:23:eb:2a|
+--------+-------------------------------------------------+
root@controller:~#
4. DefinethesecuritygrouprulesthatallowSSHandICMPaccess:
root@controller:~#openstacksecuritygrouprulecreate--protoicmp
default
+-------------------+--------------------------------------+
|Field|Value|
+-------------------+--------------------------------------+
|created_at|2017-04-26T18:17:13Z|
|description||
|direction|ingress|
|ethertype|IPv4|
|headers||
|id|ca28501a-1b3b-448f-8c1b-0fa6f9fa9263|
|port_range_max|None|
|port_range_min|None|
|project_id|123c1e6f33584dd1876c0a34249a6e11|
|project_id|123c1e6f33584dd1876c0a34249a6e11|
|protocol|icmp|
|remote_group_id|None|
|remote_ip_prefix|0.0.0.0/0|
|revision_number|1|
|security_group_id|050b8174-d961-4706-ab63-1cdd2a25fbdd|
|updated_at|2017-04-26T18:17:13Z|
+-------------------+--------------------------------------+
root@controller:~#
root@controller:~#openstacksecuritygrouprulecreate--prototcp--
dst-port22default
+-------------------+--------------------------------------+
|Field|Value|
+-------------------+--------------------------------------+
|created_at|2017-04-26T18:17:18Z|
|description||
|direction|ingress|
|ethertype|IPv4|
|headers||
|id|334130c3-42b2-4f1b-aba6-c46e91ad203e|
|port_range_max|22|
|port_range_min|22|
|project_id|123c1e6f33584dd1876c0a34249a6e11|
|project_id|123c1e6f33584dd1876c0a34249a6e11|
|protocol|tcp|
|remote_group_id|None|
|remote_ip_prefix|0.0.0.0/0|
|revision_number|1|
|security_group_id|050b8174-d961-4706-ab63-1cdd2a25fbdd|
|updated_at|2017-04-26T18:17:18Z|
+-------------------+--------------------------------------+
root@controller:~#
5. Listtheavailablenetworks,wedefinedearlier:
root@controller:~#openstacknetworklist
+--------------------------------------+------+-------------------------
-------------+
|ID|Name|Subnets
|
+--------------------------------------+------+-------------------------
-------------+
|b7ccb514-21fc-4ced-b74f-026e7e358bba|nat|296250a7-f241-4f84-adbb-
64a45c943094|
+--------------------------------------+------+-------------------------
-------------+
root@controller:~#
6. BuildanewKVMinstanceandlistitsstatus:
root@controller:~#openstackservercreate--flavorkvm.medium--image
ubuntu_16.04--nicnet-id=b7ccb514-21fc-4ced-b74f-026e7e358bba--
security-groupdefault--key-namekvmkeyubuntu_instance
+--------------------------------------+--------------------------------
-----------+
|Field|Value
|
+--------------------------------------+--------------------------------
-----------+
|OS-DCF:diskConfig|MANUAL
|
|OS-EXT-AZ:availability_zone|
|
|OS-EXT-SRV-ATTR:host|None
|
|OS-EXT-SRV-ATTR:hypervisor_hostname|None
|
|OS-EXT-SRV-ATTR:instance_name|
|
|OS-EXT-STS:power_state|NOSTATE
|
|OS-EXT-STS:task_state|scheduling
|
|OS-EXT-STS:vm_state|building
|
|OS-SRV-USG:launched_at|None
|
|OS-SRV-USG:terminated_at|None
|
|accessIPv4|
|
|accessIPv6|
|
|addresses|
|
|adminPass|Z23yEuDLBjLe
|
|config_drive|
|
|created|2017-04-26T19:11:23Z
|
|flavor|kvm.medium(0)
|
|hostId|
|
|id|0f4745b1-9d4b-4e8a-82f7-
9eaa1f9bb08f|
|image|ubuntu_16.04(abce08d2-a414-
32019d41c0cd)|
|key_name|kvmkey
|
|name|ubuntu_instance
|
|os-extended-volumes:volumes_attached|[]
|
|progress|0
|
|project_id|
123c1e6f33584dd1876c0a34249a6e11|
|properties|
|
|security_groups|[{u'name':u'default'}]
|
|status|BUILD
|
|updated|2017-04-26T19:11:23Z
|
|user_id|
cc14c5dbbd654c438e52d38efaf4f1a6|
+--------------------------------------+--------------------------------
-----------+
root@controller:~#openstackserverlist
+---------------------+-----------------+---------+------------------+--
------------+
|ID|Name|Status|Networks|
ImageName|
+---------------------+-----------------+---------+------------------+--
------------+
|0f4745b1-...-9bb08f|ubuntu_instance|BUILD|nat=192.168.0.11|
ubuntu_16.04|
+---------------------+-----------------+---------+------------------+--
------------+
root@controller:~#
7. Ensurethatthecontainerwasstartedsuccessfully:
root@controller:~#pgrep-lfaqemu
23388/usr/bin/qemu-system-x86_64-nameinstance-00000005-S-machine
pc-i440fx-xenial,accel=kvm,usb=off-cpuHaswell-
noTSX,+abm,+pdpe1gb,+rdrand,+f16c,+osxsave,+dca,+pdcm,+xtpr,+tm2,+est,+smx,+vmx,+ds_cpl,+monitor,+dtes64,+pbe,+tm,+ht,+ss,+acpi,+ds,+vme
-m1024-realtimemlock=off-smp1,sockets=1,cores=1,threads=1-uuid
0f4745b1-9d4b-4e8a-82f7-9eaa1f9bb08f-smbios
type=1,manufacturer=OpenStackFoundation,product=OpenStack
Nova,version=14.0.4,serial=6d6366d9-4569-6233-dad6-
4927587cc79f,uuid=0f4745b1-9d4b-4e8a-82f7-9eaa1f9bb08f,family=Virtual
Machine-no-user-config-nodefaults-chardev
socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-instance-
00000005/monitor.sock,server,nowait-mon
chardev=charmonitor,id=monitor,mode=control-rtcbase=utc,driftfix=slew
-globalkvm-pit.lost_tick_policy=discard-no-hpet-no-shutdown-boot
strict=on-devicepiix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2-drive
file=/var/lib/nova/instances/0f4745b1-9d4b-4e8a-82f7-
9eaa1f9bb08f/disk,format=qcow2,if=none,id=drive-virtio-disk0,cache=none
-devicevirtio-blk-pci,scsi=off,bus=pci.0,addr=0x4,drive=drive-virtio-
disk0,id=virtio-disk0,bootindex=1-netdev
tap,fd=26,id=hostnet0,vhost=on,vhostfd=28-devicevirtio-net-
pci,netdev=hostnet0,id=net0,mac=fa:16:3e:3c:c0:0f,bus=pci.0,addr=0x3-
chardevfile,id=charserial0,path=/var/lib/nova/instances/0f4745b1-9d4b-
4e8a-82f7-9eaa1f9bb08f/console.log-deviceisa-
serial,chardev=charserial0,id=serial0-chardevpty,id=charserial1-
deviceisa-serial,chardev=charserial1,id=serial1-deviceusb-
tablet,id=input0-vnc0.0.0.0:0-ken-us-devicecirrus-
vga,id=video0,bus=pci.0,addr=0x2-devicevirtio-balloon-
pci,id=balloon0,bus=pci.0,addr=0x5-msgtimestamp=on
root@controller:~#openstackserverlist
+---------------------------+-----------------+--------+----------------
--+--------------+
|ID|Name|Status|Networks
|ImageName|
+---------------------------+-----------------+--------+----------------
--+--------------+
|0f4745b1-...-9eaa1f9bb08f|ubuntu_instance|ACTIVE|
nat=192.168.0.11|ubuntu_16.04|
+---------------------------+-----------------+--------+----------------
--+--------------+
root@controller:~#
8. InspecttheKVMinstance:
root@controller:~#openstackservershowubuntu_instance
+--------------------------------------+--------------------------------
-----------+
|Field|Value
|
+--------------------------------------+--------------------------------
-----------+
|OS-DCF:diskConfig|MANUAL
|
|OS-EXT-AZ:availability_zone|nova
|
|OS-EXT-SRV-ATTR:host|controller
|
|OS-EXT-SRV-ATTR:hypervisor_hostname|controller
|
|OS-EXT-SRV-ATTR:instance_name|instance-00000001
|
|OS-EXT-STS:power_state|Running
|
|OS-EXT-STS:task_state|None
|
|OS-EXT-STS:vm_state|active
|
|OS-SRV-USG:launched_at|2017-04-26T19:11:37.000000
|
|OS-SRV-USG:terminated_at|None
|
|accessIPv4|
|
|accessIPv6|
|
|addresses|nat=192.168.0.11
|
|config_drive|
|
|created|2017-04-26T19:11:23Z
|
|flavor|kvm.medium(0)
|
|hostId|
c8d0t2jgdlkasdjg0iu4kjdg3o43045t|
|id|0f4745b1-9d4b-4e8a-82f7-
9eaa1f9bb08f|
|image|ubuntu_16.04(abce08d2-a414-
32019d41c0cd)|
|key_name|kvmkey
|
|name|ubuntu_instance
|
|os-extended-volumes:volumes_attached|[]
|
|progress|0
|
|project_id|
123c1e6f33584dd1876c0a34249a6e11|
|properties|
|
|security_groups|[{u'name':u'default'}]
|
|status|ACTIVE
|
|updated|2017-04-26T19:11:23Z
|
|user_id|
cc14c5dbbd654c438e52d38efaf4f1a6|
+--------------------------------------+--------------------------------
-----------+
root@controller:~#
Howitworks...WestarttherecipebyensuringthatwehaveanavailableGlanceimagetochoosefrom.WelistallavailableimagesinGlanceinstep1.Instep2,wecreateanewinstanceflavor;wespecifytheallocatedCPU,memory,anddiskresourcesforthenewinstancetype.Instep3,althoughitisnotmandatory,wecreateanewSSHkeypairthatcanlaterbeusedtoSSHtothenewinstance.Instep4,wecreatetwonewsecuritygrouprulesthatallowSSHandICMPtraffic.ThisishandyifwewouldliketopingandSSHtothenewinstance.Beforewebuildtheinstance,weneedtolisttheavailablenetworksinNeutron,whichtheguestwillbepartof;wedothisinstep5.
Withalloftheearlierprerequisitesinplace,webuildanewKVMinstanceinstep6,byspecifyingtheinstanceflavor,theGlanceimage,thenetwork,thesecuritygroup,andtheSSHkey.Wethenproceedtolistthestatusoftheinstance.Noticehowthetaskstateshowsasscheduling,meaningthatthenova-schedulerisselectingahosttoprovisiontheinstanceonandthestatusisBUILD.Sinceweareonlyusingasinglehostforthisexampledeployment,theinstanceisgoingtobeprovisionedonthesamecomputeserver.Fromtheoutputofthebuildcommand,wecanalsoseetheIPaddressthatwasassignedtothenewinstance.
Instep7,wecanseethatthenewinstancewassuccessfullyprovisioned,itsstatusnowshowsasACTIVEandanewQEMUprocesshasbeenstarted.
Finally,instep8,weexaminetherunninginstance;notethatthepowerstatefieldnowshowsRunningandthestatusfielddisplaysactive.
StoppingKVMinstanceswithOpenStackInthisshortrecipe,wearegoingtostoparunningKVMinstance,weprovisionedinthelastrecipe,usingthefamiliaropenstackcommandsyntax.
GettingreadyForthisrecipe,wearegoingtoneedthefollowing:
Adatabaseserver,amessagequeue,andmemcachedinstalledandconfigured,asdescribedinthePreparingthehostfortheOpenStackdeploymentrecipe.TheGlanceservicewithanavailableimage.FormoreinformationonhowtodeployGlanceandaddanewimage,refertotheInstallingandconfiguringtheOpenStackGlanceimageservicerecipe.TheKeystoneservicewedeployedintheInstallingandconfiguringtheOpenStackKeystoneidentityservicerecipe.TheNovaservicesweconfiguredintheInstallingandconfiguringtheOpenStackNovacomputeservicerecipe.TheNeutronservicethatwasdeployedintheInstallingandconfiguringtheOpenStackNeutronnetworkingservicerecipe.ArunningKVMinstance,provisionedwithOpenStack.
Howtodoit...TostoparunningKVMguestusingOpenStack,performthefollowingsimplesteps:
1. ListtheprovisionedOpenStackinstances:
root@controller:~#openstackserverlist
+---------------------------+-----------------+--------+----------------
--+--------------+
|ID|Name|Status|Networks
|ImageName|
+---------------------------+-----------------+--------+----------------
--+--------------+
|0f4745b1-...-9eaa1f9bb08f|ubuntu_instance|ACTIVE|
nat=192.168.0.11|ubuntu_16.04|
+---------------------------+-----------------+--------+----------------
--+--------------+
root@controller:~#
2. Stoptheinstance:
root@controller:~#openstackserverstopubuntu_instance
root@controller:~#
3. ListtheKVMguestsusinglibvirt:
root@controller:~#virshlist--all
IdNameState
----------------------------------------------------
-instance-00000001shutoff
root@controller:~#
4. EnsurethattheQEMUprocessfortheinstancehasterminated:
root@controller:~#pgrep-lfaqemu
root@controller:~#
5. CheckthestatusoftheKVMguest:
root@controller:~#openstackserverlist
+---------------------------+-----------------+--------+----------------
--+--------------+
|ID|Name|Status|Networks
|ImageName|
+---------------------------+-----------------+--------+----------------
--+--------------+
|0f4745b1-...-9eaa1f9bb08f|ubuntu_instance|SHUTOFF|
nat=192.168.0.11|ubuntu_16.04|
+---------------------------+-----------------+--------+----------------
--+--------------+
root@controller:~#
Howitworks...WestartbylistingtheavailableKVMinstances,provisionedwithOpenStackinstep1.Instep2,westoptheinstancebyspecifyingitsname.PleasenotethatwecanalsousetheinstanceIDtostopit.SinceOpenStackusesthelibvirttomanagethelifecycleoftheKVMinstances,instep3,weseethattheinstancehasbeenindeeddestroyed.Instep4,weensurethattheQEMUprocessfortheguesthasalsobeenterminated.Inthelaststep,wecanseethattheinstancestateisnowSHUTOFFinsteadofACTIVE.Instancesinthisstatecanbestartedagainbyexecutingthefollowingcommand:
root@controller:~#openstackserverstartubuntu_instance
root@controller:~#openstackserverlist
+---------------------------+-----------------+--------+------------------+-----
---------+
|ID|Name|Status|Networks|
ImageName|
+---------------------------+-----------------+--------+------------------+-----
---------+
|0f4745b1-...-9eaa1f9bb08f|ubuntu_instance|ACTIVE|nat=192.168.0.11|
ubuntu_16.04|
+---------------------------+-----------------+--------+------------------+-----
---------+
root@controller:~#
TerminatingKVMinstanceswithOpenStackInthisrecipe,wearegoingtoterminateaKVMinstanceprovisionedwithOpenStack.Terminatingtheinstancewillundefineitthroughlibvirt,releasetheallocatedCPUmemoryanddiskresourcesbacktothepoolofavailableresource,forthecomputehost,andmarkitsIPaddressasavailableintheNeutrondatabase.
GettingreadyForthisrecipe,wearegoingtoneedthefollowing:
Adatabaseserver,amessagequeue,andmemcachedinstalledandconfigured,asdescribedinthePreparingthehostfortheOpenStackdeploymentrecipe.TheGlanceservicewithanavailableimage.FormoreinformationonhowtodeployGlanceandaddanewimage,refertotheInstallingandconfiguringtheOpenStackGlanceimageservicerecipe.TheKeystoneservicewedeployedintheInstallingandconfiguringtheOpenStackKeystoneidentityservicerecipe.TheNovaservicesweconfiguredintheInstallingandconfiguringtheOpenStackNovacomputeservicerecipe.TheNeutronservicethatwasdeployedintheInstallingandconfiguringtheOpenStackNeutronnetworkingservicerecipe.ArunningKVMinstance,provisionedwithOpenStack.
Howtodoit...Toterminatearunninginstance,performthefollowingsteps:
1. ObtainthenameorIDoftheinstancetobeterminated:
root@controller:~#openstackserverstartubuntu_instance
root@controller:~#openstackserverlist
+---------------------------+-----------------+--------+----------------
--+--------------+
|ID|Name|Status|Networks
|ImageName|
+---------------------------+-----------------+--------+----------------
--+--------------+
|0f4745b1-...-9eaa1f9bb08f|ubuntu_instance|ACTIVE|
nat=192.168.0.11|ubuntu_16.04|
+---------------------------+-----------------+--------+----------------
--+--------------+
root@controller:~#
2. Deletetheinstancebyprovidingthename:
root@controller:~#openstackserverdeleteubuntu_instance
root@controller:~#
3. Ensurethattheinstancewasundefined:
root@controller:~#openstackserverlist
root@controller:~#virshlist--all
IdNameState
----------------------------------------------------
root@controller:~#
4. Examinethenova-api,neutron-server,andnova-computelogs:
root@controller:~#cat/var/log/nova/nova-api.log|grep-idelete
2017-05-0415:30:07.73320915INFOnova.osapi_compute.wsgi.server[req-
54dbe80f-9942-43d8-949a-d80daa2440a9cc14c5dbbd654c438e52d38efaf4f1a6
123c1e6f33584dd1876c0a34249a6e11-defaultdefault]10.184.226.74
"DELETE/v2.1/123c1e6f33584dd1876c0a34249a6e11/servers/0f4745b1-9d4b-
4e8a-82f7-9eaa1f9bb08fHTTP/1.1"status:204len:339time:0.1859989
root@controller:~#
root@controller:~#cat/var/log/neutron/neutron-server.log|grep-i
delete
2017-05-0415:30:08.40217910INFOneutron.wsgi[req-5c9674d6-c596-4b17-
b975-54625ac7adb2cc14c5dbbd654c438e52d38efaf4f1a6
123c1e6f33584dd1876c0a34249a6e11---]10.184.226.74--[04/May/2017
15:30:08]"DELETE/v2.0/ports/fdaf6ea1-b76a-4895-a028-db15831132fa.json
HTTP/1.1"2041730.320351
root@controller:~#
root@controller:~#cat/var/log/nova/nova-compute.log
...
2017-05-0415:30:07.74721065INFOnova.compute.manager[req-54dbe80f-
9942-43d8-949a-d80daa2440a9cc14c5dbbd654c438e52d38efaf4f1a6
123c1e6f33584dd1876c0a34249a6e11---][instance:0f4745b1-9d4b-4e8a-
82f7-9eaa1f9bb08f]Terminatinginstance
2017-05-0415:30:07.95221065INFOnova.virt.libvirt.driver[-]
[instance:0f4745b1-9d4b-4e8a-82f7-9eaa1f9bb08f]Instancedestroyed
successfully.
2017-05-0415:30:07.95321065INFOos_vif[req-54dbe80f-9942-43d8-949a-
d80daa2440a9cc14c5dbbd654c438e52d38efaf4f1a6
123c1e6f33584dd1876c0a34249a6e11---]Successfullyunpluggedvif
VIFBridge(active=True,address=fa:16:3e:3c:c0:0f,bridge_name='brqb7ccb514-
21',has_traffic_filtering=True,id=fdaf6ea1-b76a-4895-a028-
db15831132fa,network=Network(b7ccb514-21fc-4ced-b74f-
026e7e358bba),plugin='linux_bridge',port_profile=<?
>,preserve_on_delete=False,vif_name='tapfdaf6ea1-b7')
2017-05-0415:30:07.97021065INFOnova.virt.libvirt.driver[req-
54dbe80f-9942-43d8-949a-d80daa2440a9cc14c5dbbd654c438e52d38efaf4f1a6
123c1e6f33584dd1876c0a34249a6e11---][instance:0f4745b1-9d4b-4e8a-
82f7-9eaa1f9bb08f]Deletinginstancefiles
/var/lib/nova/instances/0f4745b1-9d4b-4e8a-82f7-9eaa1f9bb08f_del
2017-05-0415:30:07.97421065INFOnova.virt.libvirt.driver[req-
54dbe80f-9942-43d8-949a-d80daa2440a9cc14c5dbbd654c438e52d38efaf4f1a6
123c1e6f33584dd1876c0a34249a6e11---][instance:0f4745b1-9d4b-4e8a-
82f7-9eaa1f9bb08f]Deletionof/var/lib/nova/instances/0f4745b1-9d4b-
4e8a-82f7-9eaa1f9bb08f_delcomplete
...
root@controller:~#
Howitworks...WebeginbylistingallinstancesthatNovaknowsaboutinstep1,notingthenameoftheinstancewewouldliketodelete.
Instep2,wedeletetheinstancebyspecifyingitsname.NotethatwecanalsouseitsIDinstead.Instep3,weconfirmthattheinstancehasbeenundefinedbylibvirtandisnolongeravailableinOpenStack.
Instep4,wecanseetheAPIcallsthatweresenttothenova-api,neutron-server,andnova-computeservicesandtheactionthatthoseservicestook.
UsingPythontoBuildandManageKVMInstancesInthischapter,wearegoingtocoverthefollowingtopics:
InstallingandusingthePythonlibvirtlibraryDefiningKVMinstanceswithPythonStarting,stopping,anddeletingKVMinstanceswithPythonInspectingKVMinstanceswithPythonBuildingasimpleRESTAPIserverwithlibvirtandbottle
IntroductionThelibvirtlibraryexposesavirtualizationagnosticinterfaceforcontrollingthefulllifecycleofKVM(andothertechnologies,suchasXENandLXC)instances.UsingthePythonbindingswecandefine,start,destroy,anddeletevirtualguests,alongwithanythingelsethevirshuserspacetoolimplements.Infact,wecanseethatthevirshcommandusesvariouslibvirtsharedlibraries,byrunning:
root@kvm:~#ldd/usr/bin/virsh|greplibvirt
libvirt-lxc.so.0=>/usr/lib/x86_64-linux-gnu/libvirt-lxc.so.0
(0x00007fd050d88000)
libvirt-qemu.so.0=>/usr/lib/x86_64-linux-gnu/libvirt-qemu.so.0
(0x00007fd050b84000)
libvirt.so.0=>/usr/lib/x86_64-linux-gnu/libvirt.so.0(0x00007fd050394000)
root@kvm:~#
ThePythonlibvirtmodule,alsoprovidesmethodstomonitorandreporttheuseofCPU,memory,storage,andnetworkresourcesonthehypervisornodeandothercapabilitiesdependingonthetypeofhypervisordriverinuse.
Inthischapter,wearegoingtouseasmallsubsetofthePythonlibvirtAPItodefine,start,inspect,andstopaKVMinstance.
Foracompletelistoffunctions,classes,andmethodsthatthePythonlibvirtmoduleprovides,execute:root@kvm:~#pydoclibvirt
InstallingandusingthePythonlibvirtlibraryInthisrecipewearegoingtoinstallthePythonlibvirtmoduleanditsdependencies,createanewvirtualenvironment,andinstalltheiPythoncommandshellforinteractivecomputing.
GettingreadyForthisrecipewearegoingtoneedthefollowing:
AnUbuntuhost,withlibvirtandQEMUinstalledandconfiguredThedebian.imgrawimagefilewebuiltintheInstallingcustomOSontheimagewithdebootstraprecipefromChapter1,GettingStartedwithQEMUandKVMThePython2.7interpreter,usuallyprovidedbythepython2.7package
Howtodoit...ToinstallthePythonlibvirtmodule,theiPythonutility,andtocreateanewvirtualenvironmentforourtests,followthesesteps:
1. InstallthePythondevelopmentpackagespipandvirtualenv:
root@kvm:~#apt-getinstallpython-pippython-devpkg-configbuild-
essentialautoconflibvirt-dev
root@kvm:~#pipinstallvirtualenv
Downloading/unpackingvirtualenv
Downloadingvirtualenv-15.1.0-py2.py3-none-any.whl(1.8MB):1.8MB
downloaded
Installingcollectedpackages:virtualenv
Successfullyinstalledvirtualenv
Cleaningup...
root@kvm:~#
2. CreateanewPythonvirtualenvironmentandactivateit:
root@kvm:~#mkdirkvm_python
root@kvm:~#virtualenvkvm_python/
Newpythonexecutablein/root/kvm_python/bin/python
Installingsetuptools,pip,wheel...done.
root@kvm:~#sourcekvm_python/bin/activate
(kvm_python)root@kvm:~#cdkvm_python/
(kvm_python)root@kvm:~/kvm_python#ls-la
total28
drwxr-xr-x6rootroot4096May917:28.
drwx------8rootroot4096May917:28..
drwxr-xr-x2rootroot4096May917:28bin
drwxr-xr-x2rootroot4096May917:28include
drwxr-xr-x3rootroot4096May917:28lib
drwxr-xr-x2rootroot4096May917:28local
-rw-r--r--1rootroot60May917:28pip-selfcheck.json
(kvm_python)root@kvm:~/kvm_python#
3. Installthelibvirtmodule:
(kvm_python)root@kvm:~/kvm_python#pipinstalllibvirt-python
Collectinglibvirt-python
Usingcachedlibvirt-python-3.3.0.tar.gz
Buildingwheelsforcollectedpackages:libvirt-python
Runningsetup.pybdist_wheelforlibvirt-python...done
Storedindirectory:
/root/.cache/pip/wheels/67/f0/5c/c939bf8fcce5387a36efca53eab34ba8e94a28f244fd1757c1
Successfullybuiltlibvirt-python
Installingcollectedpackages:libvirt-python
Successfullyinstalledlibvirt-python-3.3.0
(kvm_python)root@kvm:~/kvm_python#pipfreeze
appdirs==1.4.3
libvirt-python==3.3.0
packaging==16.8
pyparsing==2.2.0
six==1.10.0
(kvm_python)root@kvm:~/kvm_python#python--version
Python2.7.6
(kvm_python)root@kvm:~/kvm_python#
4. InstalliPythonandstartit:
(kvm_python)root@kvm:~/kvm_python#apt-getinstallipython
...
(kvm_python)root@kvm:~/kvm_python#ipython
Python2.7.6(default,Oct262016,20:30:19)
Type"copyright","credits"or"license"formoreinformation.
IPython1.2.1--AnenhancedInteractivePython.
?->IntroductionandoverviewofIPython'sfeatures.
%quickref->Quickreference.
help->Python'sownhelpsystem.
object?->Detailsabout'object',use'object??'forextradetails.
In[1]:
Howitworks...Westartbyinstallingthedependencypackagesinstep1.SincewearegoingtouseaPythonvirtualenvironmentforourdevelopment,weinstallthevirtualenvpackageaswell.ThePythonlibvirtmoduleisgoingtobeinstalledinthevirtualenvironmentwiththepippackagemanager,sincewedon'twanttopollutethehostwithextrapackages.
Instep2,wecreateandactivateanewPythonvirtualenvironmentandinstallthePythonlibvirtmoduleinstep3.
Finallyinstep4,weinstallandstarttheiPythondevelopmenttool,whichwearegoingtousethroughoutthischapter.
DefiningKVMinstanceswithPythonInthisrecipewearegoingtodefineanewKVMinstanceusingthePythonlibvirtmoduleweinstalledinthepreviousrecipe.WearegoingtouseavirtualenvironmentandtheiPythondevelopmenttoolforthefollowingexamples.
GettingreadyForthisrecipewearegoingtoneedthefollowing:
AnUbuntuhost,withlibvirtandQEMUinstalledandconfiguredThedebian.imgrawimagefilewebuiltintheInstallingcustomOSontheimagewithdebootstraprecipefromChapter1,GettingStartedwithQEMUandKVMPython2.7,theiPythontool,andthevirtualenvironmentwecreatedintheInstallingandusingthePythonlibvirtlibraryrecipeinthischapter
Howtodoit...TodefineanewKVMinstance,usingthePythonlibvirtmodulefollowtheseinstructions:
1. IntheiPythoninterpreter,importthelibvirtmodule:
In[1]:importlibvirt
In[2]:
2. Createtheinstancedefinitionstring:
In[2]:xmlconfig="""
<domaintype='kvm'id='1'>
<name>kvm_python</name>
<memoryunit='KiB'>1048576</memory>
<currentMemoryunit='KiB'>1048576</currentMemory>
<vcpuplacement='static'>1</vcpu>
<resource>
<partition>/machine</partition>
</resource>
<os>
<typearch='x86_64'machine='pc-i440fx-trusty'>hvm</type>
<bootdev='hd'/>
</os>
<features>
<acpi/>
<apic/>
<pae/>
</features>
<clockoffset='utc'/>
<on_poweroff>destroy</on_poweroff>
<on_reboot>restart</on_reboot>
<on_crash>restart</on_crash>
<devices>
<emulator>/usr/bin/qemu-system-x86_64</emulator>
<disktype='file'device='disk'>
<drivername='qemu'type='raw'/>
<sourcefile='/tmp/debian.img'/>
<backingStore/>
<targetdev='hda'bus='ide'/>
<aliasname='ide0-0-0'/>
<addresstype='drive'controller='0'bus='0'target='0'unit='0'/>
</disk>
<controllertype='usb'index='0'>
<aliasname='usb'/>
<addresstype='pci'domain='0x0000'bus='0x00'slot='0x01'
function='0x2'/>
</controller>
<controllertype='pci'index='0'model='pci-root'>
<aliasname='pci.0'/>
<aliasname='pci.0'/>
</controller>
<controllertype='ide'index='0'>
<aliasname='ide'/>
<addresstype='pci'domain='0x0000'bus='0x00'slot='0x01'
function='0x1'/>
</controller>
<interfacetype='network'>
<macaddress='52:54:00:da:02:01'/>
<sourcenetwork='default'bridge='virbr0'/>
<targetdev='vnet0'/>
<modeltype='rtl8139'/>
<aliasname='net0'/>
<addresstype='pci'domain='0x0000'bus='0x00'slot='0x03'
function='0x0'/>
</interface>
<serialtype='pty'>
<sourcepath='/dev/pts/5'/>
<targetport='0'/>
<aliasname='serial0'/>
</serial>
<consoletype='pty'tty='/dev/pts/5'>
<sourcepath='/dev/pts/5'/>
<targettype='serial'port='0'/>
<aliasname='serial0'/>
</console>
<inputtype='mouse'bus='ps2'/>
<inputtype='keyboard'bus='ps2'/>
<graphicstype='vnc'port='5900'autoport='yes'listen='0.0.0.0'>
<listentype='address'address='0.0.0.0'/>
</graphics>
<video>
<modeltype='cirrus'vram='16384'heads='1'/>
<aliasname='video0'/>
<addresstype='pci'domain='0x0000'bus='0x00'slot='0x02'
function='0x0'/>
</video>
<memballoonmodel='virtio'>
<aliasname='balloon0'/>
<addresstype='pci'domain='0x0000'bus='0x00'slot='0x04'
function='0x0'/>
</memballoon>
</devices>
</domain>
"""
In[3]:
3. Obtainaconnectiontothehypervisor:
In[3]:conn=libvirt.open('qemu:///system')
In[4]:
4. Definethenewinstancewithoutstartingit:
In[4]:instance=conn.defineXML(xmlconfig)
In[5]:
5. Listthedefinedinstancesonthehost:
In[5]:instances=conn.listDefinedDomains()
In[6]:print'Definedinstances:{}'.format(instances)
Definedinstances:['kvm_python']
In[7]:
6. Ensuretheinstancehasbeendefined,usingthevirshcommand:
(kvm_python)root@kvm:~/kvm_python#virshlist--all
IdNameState
----------------------------------------------------
-kvm_pythonshutoff
(kvm_python)root@kvm:~/kvm_python#
Howitworks...Inthisrecipe,weareusingthepre-existingrawDebianimagewecreatedinChapter1,GettingStartedwithQEMUandKVM,todefinetheKVMinstance.
Instep1,weimportthelibvirtpackageandproceedtodefinethenewKVMinstance.WeassigntheXMLformattedstringtothexmlconfigvariableinstep2.Noticethatthedefinitioncontainsthenameofthenewinstanceandthelocationoftheimagefile.
Instep3,weobtainaconnectionobjectandassignittotheconnvariable.WecannowusetheavailablemethodstodefinetheKVMguest.
TolistallavailablemethodsforanobjectiniPython,typethevariablenamefollowedby.andpresstheTabkeytwice:In[7]:conn.Displayall117possibilities?(yorn)conn.allocPagesconn.getURIconn.nodeDeviceLookupByName
conn.baselineCPUconn.getVersionconn.nodeDeviceLookupSCSIHostByWWN
conn.c_pointerconn.interfaceDefineXMLconn.numOfDefinedDomainsconn.interfaceLookupByMACStringconn.numOfDefinedInterfaces
...
In[7]:conn.
Toobtainhelponamethod,appendthequestionmarkcharacterattheendofthemethod:In[7]:conn.defineXML?
Type:instancemethod
StringForm:<boundmethodvirConnect.defineXMLof<libvirt.virConnectobjectat
0x7fc5e57dc350>>
File:/root/kvm_python/lib/python2.7/site-packages/libvirt.py
Definition:conn.defineXML(self,xml)
Docstring:
Defineadomain,butdoesnotstartit.
Thisdefinitionispersistent,untilexplicitlyundefinedwith
virDomainUndefine().Apreviousdefinitionforthisdomainwouldbe
overriddenifitalreadyexists.
SomehypervisorsmaypreventthisoperationifthereisacurrentblockcopyoperationonatransientdomainwiththesameIDasthedomainbeingdefined;inthatcase,usevirDomainBlockJobAbort()tostoptheblockcopyfirst.
virDomainFreeshouldbeusedtofreetheresourcesafterthedomainobjectisnolongerneeded.
In[7]:
Instep4,weusethedefineXML()methodonthelibvirt.virConnectconnectionobject,passingtheXMLdefinitionstringandassignittotheinstancevariable.Wecanseethetypeofthenewobjectbyrunning:
In[7]:type(instance)
Out[7]:libvirt.virDomain
In[8]:
Instep5,welistthedefinedinstancesonthehost,byusingthelistDefinedDomains()methodandweconfirmtheresultbyusingthevirshcommandinstep6.
There'smore...Let'saddsomesimpleerrorcheckingtotheprecedingPythoncodeandwriteallofittoanewfile.Wearegoingtobeaddingtothisfileinthesubsequentrecipes:
(kvm_python)root@kvm:~/kvm_python#catkvm.py
importlibvirt
xmlconfig="""
<domaintype='kvm'id='1'>
<name>kvm_python</name>
<memoryunit='KiB'>1048576</memory>
<currentMemoryunit='KiB'>1048576</currentMemory>
<vcpuplacement='static'>1</vcpu>
<resource>
<partition>/machine</partition>
</resource>
<os>
<typearch='x86_64'machine='pc-i440fx-trusty'>hvm</type>
<bootdev='hd'/>
</os>
<features>
<acpi/>
<apic/>
<pae/>
</features>
<clockoffset='utc'/>
<on_poweroff>destroy</on_poweroff>
<on_reboot>restart</on_reboot>
<on_crash>restart</on_crash>
<devices>
<emulator>/usr/bin/qemu-system-x86_64</emulator>
<disktype='file'device='disk'>
<drivername='qemu'type='raw'/>
<sourcefile='/tmp/debian.img'/>
<backingStore/>
<targetdev='hda'bus='ide'/>
<aliasname='ide0-0-0'/>
<addresstype='drive'controller='0'bus='0'target='0'unit='0'/>
</disk>
<controllertype='usb'index='0'>
<aliasname='usb'/>
<addresstype='pci'domain='0x0000'bus='0x00'slot='0x01'function='0x2'/>
</controller>
<controllertype='pci'index='0'model='pci-root'>
<aliasname='pci.0'/>
</controller>
<controllertype='ide'index='0'>
<aliasname='ide'/>
<addresstype='pci'domain='0x0000'bus='0x00'slot='0x01'function='0x1'/>
</controller>
<interfacetype='network'>
<macaddress='52:54:00:da:02:01'/>
<macaddress='52:54:00:da:02:01'/>
<sourcenetwork='default'bridge='virbr0'/>
<targetdev='vnet0'/>
<modeltype='rtl8139'/>
<aliasname='net0'/>
<addresstype='pci'domain='0x0000'bus='0x00'slot='0x03'function='0x0'/>
</interface>
<serialtype='pty'>
<sourcepath='/dev/pts/5'/>
<targetport='0'/>
<aliasname='serial0'/>
</serial>
<consoletype='pty'tty='/dev/pts/5'>
<sourcepath='/dev/pts/5'/>
<targettype='serial'port='0'/>
<aliasname='serial0'/>
</console>
<inputtype='mouse'bus='ps2'/>
<inputtype='keyboard'bus='ps2'/>
<graphicstype='vnc'port='5900'autoport='yes'listen='0.0.0.0'>
<listentype='address'address='0.0.0.0'/>
</graphics>
<video>
<modeltype='cirrus'vram='16384'heads='1'/>
<aliasname='video0'/>
<addresstype='pci'domain='0x0000'bus='0x00'slot='0x02'function='0x0'/>
</video>
<memballoonmodel='virtio'>
<aliasname='balloon0'/>
<addresstype='pci'domain='0x0000'bus='0x00'slot='0x04'function='0x0'/>
</memballoon>
</devices>
</domain>
"""
conn=libvirt.open('qemu:///system')
ifconn==None:
print'Failedtoconnectotothehypervizor'
exit(1)
instance=conn.defineXML(xmlconfig)
ifinstance==None:
print'Failedtodefinetheinstance'
exit(1)
instances=conn.listDefinedDomains()
print'Definedinstances:{}'.format(instances)
conn.close()
(kvm_python)root@kvm:~/kvm_python#
Toexecutethescript,ensurethatthepython_kmvinstancehasbeenundefinedfirst,thenrun:
(kvm_python)root@kvm:~/kvm_python#pythonkvm.py
Definedinstances:['kvm_python']
(kvm_python)root@kvm:~/kvm_python#
Starting,stopping,anddeletingKVMinstanceswithPythonInthisrecipewearegoingtousethecreate()methodontheinstanceobjectwedefinedinthepreviousrecipetostartitandthedestroy()methodtostopit.
Toobtainmoreinformationonthecreate()method,run:
In[1]:instance.create?
Type:instancemethod
StringForm:<boundmethodvirDomain.createof<libvirt.virDomainobjectat
0x7fc5d9b97d90>>
File:/root/kvm_python/lib/python2.7/site-packages/libvirt.py
Definition:instance.create(self)
Docstring:
Launchadefineddomain.Ifthecallsucceedsthedomainmovesfromthe
definedtotherunningdomainspools.Thedomainwillbepausedonly
ifrestoringfrommanagedstatecreatedfromapauseddomain.Formore
control,seevirDomainCreateWithFlags().
In[2]:
GettingreadyForthisrecipewearegoingtoneedthefollowing:
AnUbuntuhost,withlibvirtandQEMUinstalledandconfiguredThedebian.imgrawimagefilewebuiltintheInstallingcustomOSontheimagewithdebootstraprecipefromChapter1,GettingStartedwithQEMUandKVMPython2.7,theiPythontool,andthevirtualenvironmentwecreatedintheInstallingandusingthePythonlibvirtlibraryrecipeinthischapterTheinstanceobjectwecreatedintheDefiningKVMinstanceswithPythonrecipeinthischapter
Howtodoit...TostarttheKVMinstancethatwasdefinedearlier,toobtainitsstatusandfinallytostopit,usethefollowingPythoncode:
1. Invokethecreate()methodontheinstanceobject:
In[1]:instance.create()
Out[1]:0
In[2]:
2. EnsuretheinstanceisinarunningstatebycallingtheisActive()methodontheinstanceobject:
In[2]:instance.isActive()
Out[2]:1
In[3]:
3. CheckthestatusoftheKVMinstancefromthehostOS:
(kvm_python)root@kvm:~/kvm_python#virshlist--all
IdNameState
----------------------------------------------------
5kvm_pythonrunning
(kvm_python)root@kvm:~/kvm_python#
4. Stoptheinstancewiththedestroy()method:
In[3]:instance.destroy()
Out[3]:0
In[4]:
5. Ensuretheinstancehasbeendestroyed:
In[4]:instance.isActive()
Out[4]:0
In[5]:
6. Deletetheinstanceandlistalldefinedguests:
In[5]:instance.undefine()
In[5]:instance.undefine()
Out[5]:0
In[6]:conn.listDefinedDomains()
Out[6]:[]
In[7]:
Howitworks...Instep1,wecallthecreate()methodtolaunchthedefinedinstance.Ifsuccessfultheguestwilltransitionfromshutoffstatetorunningaswecanseeintheoutputofthecommandinstep3.Instep2,weusetheisActive()method,tocheckthestatusoftheinstance.Anoutputof1indicatesthattheinstanceisrunning.
Instep4,westoptheinstanceusingthedestroy()methodandconfirminstep5.
Finallyinstep6,wedeletetheinstanceusingtheundefine()methodandlistalldefinedinstanceswiththelistDefinedDomains()call.
There'smore...Let'saddthenewcodetothePythonscriptwestartedintheDefiningKVMinstanceswithPythonrecipe.Theupdatedscriptshouldlooklikethefollowing:
(kvm_python)root@kvm:~/kvm_python#catkvm.py
importlibvirt
importtime
xmlconfig="""
<domaintype='kvm'id='1'>
<name>kvm_python</name>
<memoryunit='KiB'>1048576</memory>
<currentMemoryunit='KiB'>1048576</currentMemory>
<vcpuplacement='static'>1</vcpu>
<resource>
<partition>/machine</partition>
</resource>
<os>
<typearch='x86_64'machine='pc-i440fx-trusty'>hvm</type>
<bootdev='hd'/>
</os>
<features>
<acpi/>
<apic/>
<pae/>
</features>
<clockoffset='utc'/>
<on_poweroff>destroy</on_poweroff>
<on_reboot>restart</on_reboot>
<on_crash>restart</on_crash>
<devices>
<emulator>/usr/bin/qemu-system-x86_64</emulator>
<disktype='file'device='disk'>
<drivername='qemu'type='raw'/>
<sourcefile='/tmp/debian.img'/>
<backingStore/>
<targetdev='hda'bus='ide'/>
<aliasname='ide0-0-0'/>
<addresstype='drive'controller='0'bus='0'target='0'unit='0'/>
</disk>
<controllertype='usb'index='0'>
<aliasname='usb'/>
<addresstype='pci'domain='0x0000'bus='0x00'slot='0x01'function='0x2'/>
</controller>
<controllertype='pci'index='0'model='pci-root'>
<aliasname='pci.0'/>
</controller>
<controllertype='ide'index='0'>
<aliasname='ide'/>
<addresstype='pci'domain='0x0000'bus='0x00'slot='0x01'function='0x1'/>
</controller>
<interfacetype='network'>
<macaddress='52:54:00:da:02:01'/>
<sourcenetwork='default'bridge='virbr0'/>
<targetdev='vnet0'/>
<modeltype='rtl8139'/>
<aliasname='net0'/>
<addresstype='pci'domain='0x0000'bus='0x00'slot='0x03'function='0x0'/>
</interface>
<serialtype='pty'>
<sourcepath='/dev/pts/5'/>
<targetport='0'/>
<aliasname='serial0'/>
</serial>
<consoletype='pty'tty='/dev/pts/5'>
<sourcepath='/dev/pts/5'/>
<targettype='serial'port='0'/>
<aliasname='serial0'/>
</console>
<inputtype='mouse'bus='ps2'/>
<inputtype='keyboard'bus='ps2'/>
<graphicstype='vnc'port='5900'autoport='yes'listen='0.0.0.0'>
<listentype='address'address='0.0.0.0'/>
</graphics>
<video>
<modeltype='cirrus'vram='16384'heads='1'/>
<aliasname='video0'/>
<addresstype='pci'domain='0x0000'bus='0x00'slot='0x02'function='0x0'/>
</video>
<memballoonmodel='virtio'>
<aliasname='balloon0'/>
<addresstype='pci'domain='0x0000'bus='0x00'slot='0x04'function='0x0'/>
</memballoon>
</devices>
</domain>
"""
conn=libvirt.open('qemu:///system')
ifconn==None:
print'Failedtoconnectotothehypervizor'
exit(1)
instance=conn.defineXML(xmlconfig)
ifinstance==None:
print'Failedtodefinetheinstance'
exit(1)
instances=conn.listDefinedDomains()
print'Definedinstances:{}'.format(instances)
time.sleep(5)
ifinstance.create()<0:
print'Failedtostartthe{}instance'.format(instance.name())
exit(1)
ifinstance.isActive():
print'Theinstance{}isrunning'.format(instance.name())
else:
print'Theinstance{}isnotrunning'.format(instance.name())
time.sleep(5)
ifinstance.destroy()<0:
print'Failedtostopthe{}instance'.format(instance.name())
exit(1)
else:
print'Theinstance{}hasbeendestroyed'.format(instance.name())
ifinstance.undefine()<0:
print'Failedtoremovethe{}instance'.format(instance.name())
exit(1)
else:
print'Theinstance{}hasbeenundefined'.format(instance.name())
conn.close()
(kvm_python)root@kvm:~/kvm_python#
Startingthescriptshoulddefineanewinstance,startit,stopit,andfinallyremoveit:
(kvm_python)root@kvm:~/kvm_python#pythonkvm.py
Definedinstances:['kvm1','kvm_python']
Theinstancekvm_pythonisrunning
Theinstancekvm_pythonhasbeendestroyed
Theinstancekvm_pythonhasbeenundefined
(kvm_python)root@kvm:~/kvm_python#
Intheprecedingscript,weusedtheinstance.name()methodtogetthenameoftheKVMguestandprintit.Wealsocleanup,byclosingtheconnectiontothehypervisorwiththeconn.close()call.
InspectingKVMinstanceswithPythonInthisrecipewearegoingtocollectinstanceinformation,usingmethodsfromthelibvirt.virDomainclass.
FormoreinformationonthelibvirtPythonAPI,pleaserefertotheofficialdocumentationat:http://libvirt.org/docs/libvirt-appdev-guide-python/en-US/html/index.html.
GettingreadyForthisrecipewearegoingtoneedthefollowing:
AnUbuntuhost,withlibvirtandQEMUinstalledandconfiguredThedebian.imgrawimagefilewebuiltintheInstallingcustomOSontheimagewithdebootstraprecipefromChapter1,GettingStartedwithQEMUandKVMPython2.7,theiPythontool,andthevirtualenvironmentwecreatedintheInstallingandusingthePythonlibvirtlibraryrecipeinthischapterTheinstanceobjectwecreatedintheDefiningKVMinstanceswithPythonrecipeinthischapter,representingtheKVMguest
Howtodoit...TocollectCPU,memory,andstateinformationaboutarunninginstance,usethefollowingPythonmethods:
1. Getthenameoftheinstance:
In[1]:instance.name()
Out[1]:'kvm_python'
In[2]:
2. Ensuretheinstanceisrunning:
In[2]:instance.isActive()
Out[2]:1
In[3]:
3. CollectresourcestatisticsontheKVMinstance:
In[3]:instance.info()
Out[3]:[1,1048576L,1048576L,1,10910000000L]
In[4]:
4. Retrievethemaximumamountofphysicalmemoryallocatedtotheinstance:
In[4]:instance.maxMemory()
Out[4]:1048576L
In[5]:
5. ExtractCPUstatisticsfortheinstance:
In[5]:instance.getCPUStats(1)
Out[5]:
[{'cpu_time':10911545901L,
'system_time':1760000000L,
'user_time':1560000000L}]
In[6]:
6. Checkifthevirtualmachineisusinghardwareacceleration:
In[6]:instance.OSType()
Out[6]:'hvm'
In[7]:
7. Collecttheinstancestate:
In[82]:state,reason=instance.state()
In[83]:ifstate==libvirt.VIR_DOMAIN_NOSTATE:
....:print('Thestateisnostate')
....:elifstate==libvirt.VIR_DOMAIN_RUNNING:
....:print('Thestateisrunning')
....:elifstate==libvirt.VIR_DOMAIN_BLOCKED:
....:print('Thestateisblocked')
....:elifstate==libvirt.VIR_DOMAIN_PAUSED:
....:print('Thestateispaused')
....:elifstate==libvirt.VIR_DOMAIN_SHUTDOWN:
....:print('Thestateisshutdown')
....:elifstate==libvirt.VIR_DOMAIN_SHUTOFF:
....:print('Thestateisshutoff')
....:elifstate==libvirt.VIR_DOMAIN_CRASHED:
....:print('Thestateiscrashed')
....:elifstate==libvirt.VIR_DOMAIN_PMSUSPENDED:
....:print('Thestateissuspended')
....:else:
....:print('Thestateisunknown')
....:
Thestateisrunning
In[84]:
Howitworks...Inthisrecipe,weusedafewnewmethodsfromthelibvirt.virDomainclass.Let'sseewhattheydoinmoredetailandthenaddthemtothesimplekvm.pyPythonscriptwestartedintheDefiningKVMinstanceswithPythonrecipe.
Insteps1and2,wegetthenameoftheKVMinstanceandensureit'sinarunningstate.
Instep3,wecollectthefollowinginstanceinformation,returnedasaPythonlist:
state:Thestateoftheinstance,asdefinedinthevirDomainStateenumeratedtypeathttps://libvirt.org/html/libvirt-libvirt-domain.html#virDomainStatemaxMemory:Themaximummemoryusedbytheguestmemory:ThecurrentamountofmemoryusedbytheinstancenbVirtCPU:ThenumberofallocatedvirtualCPUscpuTime:Thetimeusedbytheinstance(innanoseconds)
Instep4,wecollectthememoryallocatedtotheinstance.Noticehowitmatchestheoutputofthefunctionfromstep3.
Instep5,wecollectinformationabouttheCPUoftheguestinstance.WecanseetheCPU,system,andusertimes.
ThehvmoutputoftheOSType()methodinstep6,indicatesthattheguestOSisdesignedtorunonbaremetal,requiringfullvirtualization,suchasKVM.
Inthelaststepofthisrecipe,wecallthestate()methodtoreturnthecurrentinstancestate.
There'smore...Let'sfinishupthischapterwithacompleteexamplescript,containingallofthemethodswehaveusedsofar:
(kvm_python)root@kvm:~/kvm_python#catkvm.py
importlibvirt
importtime
defmain():
xmlconfig="""
<domaintype='kvm'id='1'>
<name>kvm_python</name>
<memoryunit='KiB'>1048576</memory>
<currentMemoryunit='KiB'>1048576</currentMemory>
<vcpuplacement='static'>1</vcpu>
<resource>
<partition>/machine</partition>
</resource>
<os>
<typearch='x86_64'machine='pc-i440fx-trusty'>hvm</type>
<bootdev='hd'/>
</os>
<features>
<acpi/>
<apic/>
<pae/>
</features>
<clockoffset='utc'/>
<on_poweroff>destroy</on_poweroff>
<on_reboot>restart</on_reboot>
<on_crash>restart</on_crash>
<devices>
<emulator>/usr/bin/qemu-system-x86_64</emulator>
<disktype='file'device='disk'>
<drivername='qemu'type='raw'/>
<sourcefile='/tmp/debian.img'/>
<backingStore/>
<targetdev='hda'bus='ide'/>
<aliasname='ide0-0-0'/>
<addresstype='drive'controller='0'bus='0'target='0'unit='0'/>
</disk>
<controllertype='usb'index='0'>
<aliasname='usb'/>
<addresstype='pci'domain='0x0000'bus='0x00'slot='0x01'
function='0x2'/>
</controller>
<controllertype='pci'index='0'model='pci-root'>
<aliasname='pci.0'/>
</controller>
<controllertype='ide'index='0'>
<aliasname='ide'/>
<addresstype='pci'domain='0x0000'bus='0x00'slot='0x01'
function='0x1'/>
</controller>
<interfacetype='network'>
<macaddress='52:54:00:da:02:01'/>
<sourcenetwork='default'bridge='virbr0'/>
<targetdev='vnet0'/>
<modeltype='rtl8139'/>
<aliasname='net0'/>
<addresstype='pci'domain='0x0000'bus='0x00'slot='0x03'
function='0x0'/>
</interface>
<serialtype='pty'>
<sourcepath='/dev/pts/5'/>
<targetport='0'/>
<aliasname='serial0'/>
</serial>
<consoletype='pty'tty='/dev/pts/5'>
<sourcepath='/dev/pts/5'/>
<targettype='serial'port='0'/>
<aliasname='serial0'/>
</console>
<inputtype='mouse'bus='ps2'/>
<inputtype='keyboard'bus='ps2'/>
<graphicstype='vnc'port='5900'autoport='yes'listen='0.0.0.0'>
<listentype='address'address='0.0.0.0'/>
</graphics>
<video>
<modeltype='cirrus'vram='16384'heads='1'/>
<aliasname='video0'/>
<addresstype='pci'domain='0x0000'bus='0x00'slot='0x02'
function='0x0'/>
</video>
<memballoonmodel='virtio'>
<aliasname='balloon0'/>
<addresstype='pci'domain='0x0000'bus='0x00'slot='0x04'
function='0x0'/>
</memballoon>
</devices>
</domain>
"""
conn=libvirt.open('qemu:///system')
ifconn==None:
print'Failedtoconnectotothehypervizor'
exit(1)
instance=conn.defineXML(xmlconfig)
ifinstance==None:
print'Failedtodefinetheinstance'
exit(1)
instances=conn.listDefinedDomains()
print'Definedinstances:{}'.format(instances)
time.sleep(5)
ifinstance.create()<0:
print'Failedtostartthe{}instance'.format(instance.name())
exit(1)
ifinstance.isActive():
print'Theinstance{}isrunning'.format(instance.name())
else:
print'Theinstance{}isnotrunning'.format(instance.name())
print'Theinstancestate,maxmemory,currentmemory,CPUsandtimeis
{}'.format(instance.info())
print'TheCPU,systemandusertimesare{}'.format(instance.getCPUStats(1))
print'TheOStypeforthe{}instanceis{}'.format(instance.name(),
instance.OSType())
time.sleep(5)
ifinstance.destroy()<0:
print'Failedtostopthe{}instance'.format(instance.name())
exit(1)
else:
print'Theinstance{}hasbeendestroyed'.format(instance.name())
ifinstance.undefine()<0:
print'Failedtoremovethe{}instance'.format(instance.name())
exit(1)
else:
print'Theinstance{}hasbeenundefined'.format(instance.name())
conn.close()
if__name__=="__main__":
main()
(kvm_python)root@kvm:~/kvm_python#
Executingitprovidesthefollowingoutput,assumingthekvm_pythoninstancehasbeenundefinedfirst:
(kvm_python)root@kvm:~/kvm_python#pythonkvm.py
Definedinstances:['kvm_python']
Theinstancekvm_pythonisrunning
Theinstancestate,maxmemory,currentmemory,CPUsandtimeis[1,1048576L,
1048576L,1,40000000L]
TheCPU,systemandusertimesare[{'cpu_time':42349077L,'system_time':0L,
'user_time':30000000L}]
TheOStypeforthekvm_pythoninstanceishvm
Theinstancekvm_pythonhasbeendestroyed
Theinstancekvm_pythonhasbeenundefined
(kvm_python)root@kvm:~/kvm_python#
BuildingasimpleRESTAPIserverwithlibvirtandbottleInthisrecipe,wearegoingtouseallofthelibvirtmethodswesawintheearlierrecipestobuildasimpleRESTfullAPIserver,leveragingthebottlemicroframeworkforPython.
BottleisdescribedasafastandsimpleWebServerGatewayInterface(WSGI)microweb-frameworkforPython,whichisdistributedasasinglemodulefile.
Formoreinformationonthebottlemicroframeworkpleasevisittheofficialwebsiteat:https://bottlepy.org/docs/dev/.
ThesimpleAPIserverweareimplementing,willacceptthefollowingrequests:
list:getmethodthatlistsalldefinedlibvirtinstances.define:postmethodusedtodefineanewKVMinstance.WearegoingtoprovidetheXMLdefinitionasaheaderinthepostrequest.start:postmethodtostartaninstance.Thenameoftheinstancewillbeprovidedintheheaderoftherequest.stop:postmethodtospotaKVMinstance.undefine:postmethodtodeletetheinstance.
GettingreadyForthisrecipewearegoingtoneedthefollowing:
AnUbuntuhost,withlibvirtandQEMUinstalledandconfiguredThedebian.imgrawimagefilewebuiltintheInstallingcustomOSontheimagewithdebootstraprecipefromChapter1,GettingStartedwithQEMUandKVMPython2.7andthevirtualenvironmentwecreatedintheInstallingandusingthePythonlibvirtlibraryrecipeinthischapterThecurlcommandlinetoolfortransferringdatawithURLsyntax,usuallyprovidedbythecurlpackage
Howtodoit...ThefollowingstepsdescribehowtoinstallthebottlemoduleandthesimpleRESTfullAPIserverwritteninPython:
1. Installthebottlemodule:
(kvm_python)root@kvm:~/kvm_python#pipinstallbottle
Collectingbottle
...
Downloadingbottle-0.12.13.tar.gz(70kB)
100%|████████████████████████████████|71kB4.5MB/s
...
Successfullyinstalledbottle-0.12.13
(kvm_python)root@kvm:~/kvm_python#
2. Createanewfile,importthelibvirtandbottlemodulesandwritethelibvirtconnectionmethod:
(kvm_python)root@kvm:~/kvm_python#vimkvm_api.py
importlibvirt
frombottleimportrun,request,get,post,HTTPResponse
deflibvirtConnect():
try:
conn=libvirt.open('qemu:///system')
exceptlibvirt.libvirtError:
conn=None
returnconn
3. Implement/definetheAPIrouteandfunction:
defdefineKVMInstance(template):
conn=libvirtConnect()
ifconn==None:
returnHTTPResponse(status=500,body='Errordefininginstance\n')
else:
try:
conn.defineXML(template)
returnHTTPResponse(status=200,body='Instancedefined\n')
exceptlibvirt.libvirtError:
returnHTTPResponse(status=500,body='Errordefininginstance\n')
@post('/define')
defbuild():
template=str(request.headers.get('X-KVM-Definition'))
status=defineKVMInstance(template)
status=defineKVMInstance(template)
returnstatus
4. Implement/undefinetheAPIrouteandfunction:
defundefineKVMInstance(name):
conn=libvirtConnect()
ifconn==None:
returnHTTPResponse(status=500,body='Errorundefininginstance\n')
else:
try:
instance=conn.lookupByName(name)
instance.undefine()
returnHTTPResponse(status=200,body='Instanceundefined\n')
exceptlibvirt.libvirtError:
returnHTTPResponse(status=500,body='Errorundefining
instance\n')
@post('/undefine')
defbuild():
name=str(request.headers.get('X-KVM-Name'))
status=undefineKVMInstance(name)
returnstatus
5. Implement/starttheAPIrouteandfunction:
defstartKVMInstance(name):
conn=libvirtConnect()
ifconn==None:
returnHTTPResponse(status=500,body='Errorstartinginstance\n')
else:
try:
instance=conn.lookupByName(name)
instance.create()
returnHTTPResponse(status=200,body='Instancestarted\n')
exceptlibvirt.libvirtError:
returnHTTPResponse(status=500,body='Errorstartinginstance\n')
@post('/start')
defbuild():
name=str(request.headers.get('X-KVM-Name'))
status=startKVMInstance(name)
returnstatus
6. Implement/stoptheAPIrouteandfunction:
defstopKVMInstance(name):
conn=libvirtConnect()
ifconn==None:
ifconn==None:
returnHTTPResponse(status=500,body='Errorstoppinginstance\n')
else:
try:
instance=conn.lookupByName(name)
instance.destroy()
returnHTTPResponse(status=200,body='Instancestopped\n')
exceptlibvirt.libvirtError:
returnHTTPResponse(status=500,body='Errorstoppinginstance\n')
@post('/stop')
defbuild():
name=str(request.headers.get('X-KVM-Name'))
status=stopKVMInstance(name)
returnstatus
7. Implement/listtheAPIrouteandfunction:
defgetLibvirtInstances():
conn=libvirtConnect()
ifconn==None:
returnHTTPResponse(status=500,body='Errorlistinginstances\n')
else:
try:
instances=conn.listDefinedDomains()
returninstances
exceptlibvirt.libvirtError:
returnHTTPResponse(status=500,body='Errorlistinginstances\n')
@get('/list')
deflist():
kvm_list=getLibvirtInstances()
return"ListofKVMinstances:{}\n".format(kvm_list)
8. Invoketherun()methodtostarttheWSGIserverwhenthescriptisexecuted:
run(host='localhost',port=8080,debug=True)
Howitworks...Let'slookatthecodeinmoredetail.First,savetheprecedingchangesinafileandexecutethescript:
(kvm_python)root@kvm:~/kvm_python#pythonkvm_api.py
Bottlev0.12.13serverstartingup(usingWSGIRefServer())...
Listeningonhttp://localhost:8080/
HitCtrl-Ctoquit.
Inaseparateterminal,defineanewinstance,passingthefollowingXMLdefinition,asaheader:
(kvm_python)root@kvm:~/kvm_python#curl-s-i-XPOSTlocalhost:8080/define--
header"X-KVM-Definition:<domaintype='kvm'><name>kvm_api</name><memory
unit='KiB'>1048576</memory><vcpu>1</vcpu><os><typearch='x86_64'machine='pc-
i440fx-trusty'>hvm</type></os><devices><emulator>/usr/bin/qemu-system-
x86_64</emulator><disktype='file'device='disk'><drivername='qemu'
type='raw'/><sourcefile='/tmp/debian.img'/><targetdev='hda'bus='ide'/></disk>
<interfacetype='network'><macaddress='52:54:00:da:02:01'/><source
network='default'bridge='virbr0'/><targetdev='vnet0'/></interface><graphics
type='vnc'port='5900'autoport='yes'listen='0.0.0.0'><listentype='address'
address='0.0.0.0'/></graphics></devices></domain>"
HTTP/1.0200OK
Date:Fri,12May201720:29:14GMT
Server:WSGIServer/0.1Python/2.7.6
Content-Length:17
Content-Type:text/html;charset=UTF-8
Instancedefined
(kvm_python)root@kvm:~/kvm_python#
WeareusingtherawDebianimagewecreatedinChapter1,GettingStartedwithQEMUandKVM.TheXMLdefinitionshouldlookfamiliaraswell;we'vebeenusingitinmostoftherecipesinthischapter.
WeshouldnowhaveanewKVMinstancedefined.Let'susethe/listroutetolistallinstancesandconfirmwiththevirshcommand:
(kvm_python)root@kvm:~/kvm_python#curllocalhost:8080/list
ListofKVMinstances:['kvm_api']
(kvm_python)root@kvm:~/kvm_python#virshlist--all
IdNameState
----------------------------------------------------
-kvm_apishutoff
(kvm_python)root@kvm:~/kvm_python#
Nowthatwehaveadefinedinstance,let'sstartitusingthe/startrouteandensureit'srunning:
(kvm_python)root@kvm:~/kvm_python#curl-s-i-XPOSTlocalhost:8080/start--
header"X-KVM-Name:kvm_api"
HTTP/1.0200OK
Date:Fri,12May201720:29:38GMT
Server:WSGIServer/0.1Python/2.7.6
Content-Length:17
Content-Type:text/html;charset=UTF-8
Instancestarted
(kvm_python)root@kvm:~/kvm_python#virshlist--all
IdNameState
----------------------------------------------------
1kvm_apirunning
(kvm_python)root@kvm:~/kvm_python#
Tostoptheinstanceandremoveitcompletely,weusethe/stopand/undefineroutesfromthescript:
(kvm_python)root@kvm:~/kvm_python#curl-s-i-XPOSTlocalhost:8080/stop--
header"X-KVM-Name:kvm_api"
HTTP/1.0200OK
Date:Fri,12May201720:29:52GMT
Server:WSGIServer/0.1Python/2.7.6
Content-Length:17
Content-Type:text/html;charset=UTF-8
Instancestopped
(kvm_python)root@kvm:~/kvm_python#
(kvm_python)root@kvm:~/kvm_python#virshlist--all
IdNameState
----------------------------------------------------
-kvm_apishutoff
(kvm_python)root@kvm:~/kvm_python#
(kvm_python)root@kvm:~/kvm_python#curl-s-i-XPOSTlocalhost:8080/undefine--
header"X-KVM-Name:kvm_api"
HTTP/1.0200OK
Date:Fri,12May201720:30:09GMT
Server:WSGIServer/0.1Python/2.7.6
Content-Length:19
Content-Type:text/html;charset=UTF-8
Instanceundefined
(kvm_python)root@kvm:~/kvm_python#
(kvm_python)root@kvm:~/kvm_python#virshlist--all
IdNameState
----------------------------------------------------
(kvm_python)root@kvm:~/kvm_python#
Letusgothroughthecodeinmoredetail.
Instep1,weinstallthebottlemoduleinthePythonvirtualenvironment.
Afterimportingthelibvirtandbottlepackagesinstep2,wedefinethelibvirtConnect()method.Thefunctionsinourprogramwilluseittoconnecttothehypervisor.
Instep3,weimplementthe/definerouteanditsfunctionality.The@postdecoratorlinksthecodefromthefollowingfunctiontoaURLpath.Inourexamplethe/definerouteisboundtothebuild()function.Passingthe/defineroutetothecurlcommandwillexecutethefunction,whichinturnwillcallthedefineKVMInstance()methodtodefinetheinstance.
Weusethesamecodepatterninsteps4,5,and6tostart,stop,andundefinetheinstance.
Instep7,weusethe@getdecoratortoimplementafunctiontolistalldefinedinstancesonthehost.
Instep8,weusetherunclasswhichprovidestherun()methodweusetostartabuilt-inserver.Inourexampletheserverwillbelisteningonlocalhost,port8080.
Aswesawearlier,executingthescriptwillstartalisteningsocketonport8080,whichwecaninteractwith,usingthecurlcommand.
There'smore...Thefullcodeimplementationfollows:
importlibvirt
frombottleimportrun,request,get,post,HTTPResponse
deflibvirtConnect():
try:
conn=libvirt.open('qemu:///system')
exceptlibvirt.libvirtError:
conn=None
returnconn
defgetLibvirtInstances():
conn=libvirtConnect()
ifconn==None:
returnHTTPResponse(status=500,body='Errorlistinginstances\n')
else:
try:
instances=conn.listDefinedDomains()
returninstances
exceptlibvirt.libvirtError:
returnHTTPResponse(status=500,body='Errorlistinginstances\n')
defdefineKVMInstance(template):
conn=libvirtConnect()
ifconn==None:
returnHTTPResponse(status=500,body='Errordefininginstance\n')
else:
try:
conn.defineXML(template)
returnHTTPResponse(status=200,body='Instancedefined\n')
exceptlibvirt.libvirtError:
returnHTTPResponse(status=500,body='Errordefininginstance\n')
defundefineKVMInstance(name):
conn=libvirtConnect()
ifconn==None:
returnHTTPResponse(status=500,body='Errorundefininginstance\n')
else:
try:
instance=conn.lookupByName(name)
instance.undefine()
returnHTTPResponse(status=200,body='Instanceundefined\n')
exceptlibvirt.libvirtError:
returnHTTPResponse(status=500,body='Errorundefininginstance\n')
defstartKVMInstance(name):
conn=libvirtConnect()
ifconn==None:
returnHTTPResponse(status=500,body='Errorstartinginstance\n')
else:
try:
instance=conn.lookupByName(name)
instance.create()
returnHTTPResponse(status=200,body='Instancestarted\n')
exceptlibvirt.libvirtError:
returnHTTPResponse(status=500,body='Errorstartinginstance\n')
defstopKVMInstance(name):
conn=libvirtConnect()
ifconn==None:
returnHTTPResponse(status=500,body='Errorstoppinginstance\n')
else:
try:
instance=conn.lookupByName(name)
instance.destroy()
returnHTTPResponse(status=200,body='Instancestopped\n')
exceptlibvirt.libvirtError:
returnHTTPResponse(status=500,body='Errorstoppinginstance\n')
@post('/define')
defbuild():
template=str(request.headers.get('X-KVM-Definition'))
status=defineKVMInstance(template)
returnstatus
@post('/undefine')
defbuild():
name=str(request.headers.get('X-KVM-Name'))
status=undefineKVMInstance(name)
returnstatus
@get('/list')
deflist():
kvm_list=getLibvirtInstances()
return"ListofKVMinstances:{}\n".format(kvm_list)
@post('/start')
defbuild():
name=str(request.headers.get('X-KVM-Name'))
status=startKVMInstance(name)
returnstatus
@post('/stop')
defbuild():
name=str(request.headers.get('X-KVM-Name'))
status=stopKVMInstance(name)
returnstatus
run(host='localhost',port=8080,debug=True)
KernelTuningforKVMPerformanceInthischapter,wearegoingtocoverthefollowingperformancetuningrecipes:
TuningthekernelforlowI/OlatencyMemorytuningforKVMguestsCPUperformanceoptionsNUMAtuningwithlibvirtTuningthekernelfornetworkperformance
IntroductionInthischapter,wearegoingtoexplorevariousconfigurationoptionsandtoolsthatcanhelpimprovetheperformanceofthehostOSandtheKVMinstancesrunningonit.
WhenrunningKVMvirtualmachines,it'simportanttounderstandthatfromthehostperspective,theyareregularprocesses.WecanseethatKVMguestsareLinuxprocessesbyexaminingtheprocesstreeonthehypervisor:
root@kvm:~#virshlist
IdNameState
----------------------------------------------------
16kvmrunning
root@kvm:~#pgrep-lfaqemu
19913/usr/bin/qemu-system-x86_64-namekvm-S-machinepc-i440fx-
trusty,accel=kvm,usb=off-m1024-realtimemlock=off-smp
1,sockets=1,cores=1,threads=1-uuid283c6653-9981-9396-efb4-fb864d87f769-no-
user-config-nodefaults-chardev
socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-
kvm/monitor.sock,server,nowait-monchardev=charmonitor,id=monitor,mode=control
-rtcbase=utc-no-shutdown-bootstrict=on-devicepiix3-usb-
uhci,id=usb,bus=pci.0,addr=0x1.0x2-drive
file=/tmp/debian.img,format=raw,if=none,id=drive-ide0-0-0-deviceide-
hd,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1-netdev
tap,fd=26,id=hostnet0-device
rtl8139,netdev=hostnet0,id=net0,mac=52:54:00:2f:df:93,bus=pci.0,addr=0x3-
chardevpty,id=charserial0-deviceisa-serial,chardev=charserial0,id=serial0-
vnc0.0.0.0:0-devicecirrus-vga,id=video0,bus=pci.0,addr=0x2-devicevirtio-
balloon-pci,id=balloon0,bus=pci.0,addr=0x4-msgtimestamp=on
root@kvm:~#
ThevirtualCPUsallocatedtotheKVMguestsareLinuxthreads,managedbythehostscheduler:
root@kvm:~#ps-eLf
UIDPIDPPIDLWPCNLWPSTIMETTYTIMECMD
...
libvirt+199131199130314:02?00:00:00/usr/bin/qemu-system-x86_64-name
kvm-S-machinepc-i440fx-trusty,accel=kvm,usb=off-m1024-realtimemlock=off-
smp1,sockets=1,cores=1,threads=1-uuid283c6653-9981-9396-efb4-fb864d87f769-
no-user-config-nodefaul
libvirt+199131199140314:02?00:00:08/usr/bin/qemu-system-x86_64-name
kvm-S-machinepc-i440fx-trusty,accel=kvm,usb=off-m1024-realtimemlock=off-
smp1,sockets=1,cores=1,threads=1-uuid283c6653-9981-9396-efb4-fb864d87f769-
no-user-config-nodefaul
libvirt+199131199170314:02?00:00:00/usr/bin/qemu-system-x86_64-name
kvm-S-machinepc-i440fx-trusty,accel=kvm,usb=off-m1024-realtimemlock=off-
smp1,sockets=1,cores=1,threads=1-uuid283c6653-9981-9396-efb4-fb864d87f769-
no-user-config-nodefaul
...
root@kvm:~#
DependingonthetypeofI/Oscheduler,thelibvirtnetworkdriver,andmemoryconfiguration,theperformanceofthevirtualmachinecanvarygreatly.Beforemakinganychangestotheearlier-mentionedcomponents,itisimportanttounderstandthetypeofworktheguestOSwillbeperforming.TuningthehostandguestOSformemory-intensiveworkwillbedifferentfromI/OorCPUboundloads.
BecauseallKVMinstancesarejustregularLinuxprocesses,theQEMUdrivercanapplyanyofthefollowingControlGroup(cgroup)controllers:cpuset,cpu,memory,blkio,anddevicecontrollers.UsingthecgroupcontrollersprovidesmoregranularcontrolovertheallowedCPU,memory,andI/Oresources,aswearegoingtoseeinmoredetailinthefollowingrecipes.
Perhapsthemostimportantpointwhentuningandoptimizinganysystemistoestablishtheperformancebaselinepriortomakinganyadjustments.Startbymeasuringthebaselineperformanceofasubsystem,suchasmemoryorI/O,makesmallincrementaladjustments,thenmeasuretheimpactofthechangesagain.Repeatasnecessaryuntiladesiredeffectisreached.
Therecipesinthischapteraremeanttogivethereaderastartingpointforwhatcanbetunedonthehostandthevirtualmachinestoimproveperformance,oraccountforthesideeffectsofrunningdifferentworkloadsonthesamehost/VMandtheeffectsofmultitenancy.Allresourcesshouldbeadjustedbasedonthetypeofworkload,hardwaresetup,andothervariables.
TuningthekernelforlowI/OlatencyInthisrecipe,wearegoingtocoversomeofthediskperformanceoptimizationtechniquesbyselectinganI/OschedulerandtuningtheblockI/OusingLinuxcontrolgroups,forthevirtualguestandthehost.
TherearethreeI/OschedulerstochoosefromonthehostOSandintheKVMinstance:
noop:Thisisoneofthesimplestkernelschedulers;itworksbyinsertingallincomingI/OrequestsintoasimpleFIFO(FirstIn,FirstOut)queue.ThisschedulerisusefulwhenthehostOSshouldnotattempttoreorderI/Orequestswhenmultiplevirtualmachinesarerunning.deadline:ThisschedulerimposesadeadlineonallI/Ooperationstopreventstarvationofrequests,givingprioritytoreadrequests,duetoprocessesusuallyblockingonreadoperations.cfq:ThemaingoalofCompletelyFairQueuing(CFQ)istomaximizetheoverallCPUutilizationwhileallowingbetterinteractiveperformance.
SelectingtherightI/Oscheduleronthehostandguestsgreatlydependsontheworkloadsandtheunderlyinghardwarestorage.
Asageneralrule,selectingthenoopschedulerfortheguestOSallowsthehosthypervisortobetteroptimizetheI/Orequests,becauseitisawareofallrequestscomingfromthevirtualguests.However,iftheunderlyingstoragefortheKVMmachinesisiSCSIvolumesoranyotherremotestoragesuchasGlusterFS,usingthedeadlinescheduler,mightyieldbetterresults.
OnmostmodernLinuxkernels,thedeadlinescheduleristhedefault,anditmightbesufficientforhostsrunningmultipleKVMvirtualmachines.Aswithanysystemtuning,testingisrequiredwhenchangingtheschedulersonthehostandguestOS.
GettingreadyForthisrecipe,wearegoingtoneedthefollowing:
AnUbuntuhost,withlibvirtandQEMUinstalledandconfiguredArunningKVMvirtualmachine
Howtodoit...TochangetheI/OscheduleronthehostandtheKVMinstanceandsetanI/Oweight,performthefollowingsteps:
1. OnthehostOS,listtheI/Oschedulercurrentlyinuse,substitutingtheblockdevicewithwhateverisappropriateforyoursystem:
root@kvm:~#cat/sys/block/sda/queue/scheduler
noopdeadline[cfq]
root@kvm:~#
2. ChangetheI/Oschedulerondemandandensureitisinusebyrunning:
root@kvm:~#echodeadline>/sys/block/sda/queue/scheduler
root@kvm:~#cat/sys/block/sda/queue/scheduler
noop[deadline]cfq
root@kvm:~#
3. Tomakethechangepersistentacrossserverrestarts,addthefollowinglinetotheGRUBdefaultconfigurationandupdate:
root@kvm:~#echo'GRUB_CMDLINE_LINUX="elevator=deadline"'>>
/etc/default/grub
root@kvm:~#tail-1/etc/default/grub
GRUB_CMDLINE_LINUX="elevator=deadline"
root@kvm:~#update-grub2
Generatinggrubconfigurationfile...
Foundlinuximage:/boot/vmlinuz-3.13.0-107-generic
Foundinitrdimage:/boot/initrd.img-3.13.0-107-generic
done
root@kvm:~#cat/boot/grub/grub.cfg|grepelevator
linux/boot/vmlinuz-3.13.0-107-genericroot=/dev/md126p1ro
elevator=deadlinerd.fstab=noacpi=noirqnoapiccgroup_enable=memory
swapaccount=1quiet
linux/boot/vmlinuz-3.13.0-107-genericroot=/dev/md126p1ro
elevator=deadlinerd.fstab=noacpi=noirqnoapiccgroup_enable=memory
swapaccount=1quiet
linux/boot/vmlinuz-3.13.0-107-genericroot=/dev/md126p1rorecovery
nomodesetelevator=deadline
root@kvm:~#
4. FortheKVMinstance,setupthenoopI/Oschedulerpersistently:
root@kvm:~#virshconsolekvm1
Connectedtodomainkvm1
Escapecharacteris^]
root@kvm1:~#echo'GRUB_CMDLINE_LINUX="elevator=noop"'>>
/etc/default/grub
root@kvm1:~#tail-1/etc/default/grub
GRUB_CMDLINE_LINUX="elevator=noop"
root@kvm1:~#update-grub2
Generatinggrubconfigurationfile...
Foundlinuximage:/boot/vmlinuz-3.13.0-107-generic
Foundinitrdimage:/boot/initrd.img-3.13.0-107-generic
done
root@kvm1:~#cat/boot/grub/grub.cfg|grepelevator
linux/boot/vmlinuz-3.13.0-107-genericroot=/dev/md126p1ro
elevator=nooprd.fstab=noacpi=noirqnoapiccgroup_enable=memory
swapaccount=1quiet
linux/boot/vmlinuz-3.13.0-107-genericroot=/dev/md126p1ro
elevator=nooprd.fstab=noacpi=noirqnoapiccgroup_enable=memory
swapaccount=1quiet
linux/boot/vmlinuz-3.13.0-107-genericroot=/dev/md126p1rorecovery
nomodesetelevator=noop
root@kvm1:~#
5. Setaweightof100fortheKVMinstanceusingtheblkiocgroupcontroller:
root@kvm:~#virshblkiotune--weight100kvm
root@kvm:~#virshblkiotunekvm
weight:100
device_weight:
device_read_iops_sec:
device_write_iops_sec:
device_read_bytes_sec:
device_write_bytes_sec:
root@kvm:~#
6. Findthecgroupdirectoryhierarchyonthehost:
root@kvm:~#mount|grepcgroup
noneon/sys/fs/cgrouptypetmpfs(rw)
systemdon/sys/fs/cgroup/systemdtypecgroup
(rw,noexec,nosuid,nodev,none,name=systemd)
root@kvm:~#
7. EnsurethatthecgroupfortheKVMinstancecontainstheweightthatwesetupearlierontheblkiocontroller:
root@kvm:~#cat/sys/fs/cgroup/blkio/machine/kvm.libvirt-
qemu/blkio.weight
100
root@kvm:~#
ForadetailedexplanationonhowLinuxcgroupswork,refertotheContainerizationwithLXCbookfromPacktpublishingathttps://www.packtpub.com/virtualization-and-cloud/containerization-lxc.
Howitworks...WecanseewhatI/Oschedulerthekerneliscurrentlyusingbyexaminingtheschedulerfileinthe/sysvirtualfilesystem.Instep1,weseethatit'sthecfqscheduler.WethenproceedtochangetheI/Oschedulerontherunningsysteminstep2.Pleasekeepinmindthatchangingtheschedulerondemandlikethatwillnotpersistserverrestarts.Insteps3and4,wemodifytheGRUBconfigurationwhichwillappendthenewschedulerinformationtothekernelbootinstructions.RestartingtheserverorthevirtualmachinewillnowselectthenewI/Oscheduler.
Ifrunningmultiplevirtualmachinesonthesamehost,itmightbeusefultogivemoreI/Oprioritytosomeofthembasedoncertaincriteria,suchastimeofdayandVMworkload.Instep5,weusetheblkiocgroupcontrollertosetaweightfortheKVMguest.LowerweightwillgivebetterI/Opriority.Insteps6and7,wecanseethatthecorrectcgrouphierarchyhasbeencreatedandtheblkio.weightfilecontainsthenewweightwesetwiththevirshcommand.
MemorytuningforKVMguestsWhenitcomestomemorytuningofKVMgueststherearefewoptionsavailable,dependingontheworkloadofthevirtualmachine.OnesuchoptionisLinuxHugePages.
MostLinuxhostsbydefaultaddressmemoryin4KBsegments,namedpages.However,thekerneliscapableofusinglargerpagesizes.UsingHugePages(pagesbiggerthan4KB)mayimproveperformancebyincreasingtheCPUcachehitsagainstthetransactionLookasideBuffer(TLB).TheTLBisamemorycachethatstoresrecenttranslationsofvirtualmemorytophysicaladdressesforquickretrieval.
Inthisrecipe,wearegoingtoenableandsetHugePagesonthehypervisorandtheKVMguest,thenexaminethetuningoptionsthatthevirshcommandprovides.
GettingreadyForthisrecipe,wearegoingtoneedthefollowing:
AnUbuntuhost,withlibvirtandQEMUinstalledandconfiguredArunningKVMvirtualmachine
Howtodoit...ToenableandsetHugePagesonthehypervisorandtheKVMguestandusethevirshcommandtosetvariousmemoryoptions,followthesesteps:
1. CheckthecurrentHugePagessettingsonthehostOS:
root@kvm:~#cat/proc/meminfo|grep-ihuge
AnonHugePages:509952kB
HugePages_Total:0
HugePages_Free:0
HugePages_Rsvd:0
HugePages_Surp:0
Hugepagesize:2048kB
root@kvm:~#
2. ConnecttotheKVMguestandcheckthecurrentHugePagessettings:
root@kvm1:~#cat/proc/meminfo|grep-ihuge
HugePages_Total:0
HugePages_Free:0
HugePages_Rsvd:0
HugePages_Surp:0
Hugepagesize:2048kB
root@kvm1:~#
3. IncreasethesizeofthepoolofHugePagesfrom0to25000onthehypervisorandverifythefollowing:
root@kvm:~#sysctlvm.nr_hugepages=25000
vm.nr_hugepages=25000
root@kvm:~#cat/proc/meminfo|grep-ihuge
AnonHugePages:446464kB
HugePages_Total:25000
HugePages_Free:24484
HugePages_Rsvd:0
HugePages_Surp:0
Hugepagesize:2048kB
root@kvm:~#cat/proc/sys/vm/nr_hugepages
25000
root@kvm:~#
4. CheckwhetherthehypervisorCPUsupports2MBand1GBHugePagessizes:
root@kvm:~#cat/proc/cpuinfo|egrep-i"pse|pdpe1"|tail-1
flags:fpuvmedepsetscmsrpaemcecx8apicsepmtrrpgemcacmov
patpse36clflushdtsacpimmxfxsrssesse2sshttmpbesyscallnx
pdpe1gbrdtscplmconstant_tscarch_perfmonpebsbtsrep_goodnopl
xtopologynonstop_tscaperfmperfeagerfpupnipclmulqdqdtes64monitor
ds_cplvmxsmxesttm2ssse3fmacx16xtprpdcmpciddcasse4_1sse4_2
x2apicmovbepopcnttsc_deadline_timeraesxsaveavxf16crdrandlahf_lm
abmaratepbxsaveoptplnptsdthermtpr_shadowvnmiflexpriorityept
vpidfsgsbasetsc_adjustbmi1avx2smepbmi2ermsinvpcid
root@kvm:~#
5. Set1GBHugePagessizebyeditingthedefaultGRUBconfigurationandrebooting:
root@kvm:~#cat/etc/default/grub
...
GRUB_CMDLINE_LINUX_DEFAULT="rd.fstab=noacpi=noirqnoapic
cgroup_enable=memoryswapaccount=1quiethugepagesz=1GBhugepages=1"
...
root@kvm:~#update-grub
Generatinggrubconfigurationfile...
Foundlinuximage:/boot/vmlinuz-3.13.0-107-generic
Foundinitrdimage:/boot/initrd.img-3.13.0-107-generic
done
root@kvm:~#cat/boot/grub/grub.cfg|grep-ihuge
linux/boot/vmlinuz-3.13.0-107-genericroot=/dev/md126p1ro
elevator=deadlinerd.fstab=noacpi=noirqnoapiccgroup_enable=memory
swapaccount=1quiethugepagesz=1GBhugepages=1
linux/boot/vmlinuz-3.13.0-107-genericroot=/dev/md126p1ro
elevator=deadlinerd.fstab=noacpi=noirqnoapiccgroup_enable=memory
swapaccount=1quiethugepagesz=1GBhugepages=1
root@kvm:~#reboot
6. InstalltheHugePagespackage:
root@kvm:~#apt-getinstallhugepages
7. CheckthecurrentHugePagessize:
root@kvm:~#hugeadm--pool-list
SizeMinimumCurrentMaximumDefault
2097152250002500025000*
root@kvm:~#
8. EnableHugePagessupportforKVM:
root@kvm:~#sed-i's/KVM_HUGEPAGES=0/KVM_HUGEPAGES=1/g'
/etc/default/qemu-kvm
root@kvm:~#root@kvm:~#/etc/init.d/libvirt-binrestart
libvirt-binstop/waiting
libvirt-binstart/running,process16257
root@kvm:~#
9. MounttheHugeTablevirtualfilesystemonthehostOS:
root@kvm:~#mkdir/hugepages
root@kvm:~#echo"hugetlbfs/hugepageshugetlbfsmode=1770,gid=202100"
>>/etc/fstab
root@kvm:~#mount-a
root@kvm:~#mount|grephugepages
hugetlbfson/hugepagestypehugetlbfs(rw,mode=1770,gid=2021)
root@kvm:~#
10. EdittheconfigurationfortheKVMguestandenableHugePages:
root@kvm:~#virshdestroykvm1
Domainkvm1destroyed
root@kvm:~#virsheditkvm1
...
<memoryBacking>
<hugepages/>
</memoryBacking>
...
Domainkvm1XMLconfigurationedited.
root@kvm:~#virshstartkvm1
Domainkvm1started
root@kvm:~#
IfyouseethefollowingerrorwhenstartingtheKVMinstance:error:internalerror:hugetlbfsfilesystemisnotmountedor
disabledbyadministratorconfig,makesurethattheHugePagesvirtualfilesystemwasmountedsuccessfullyinstep9.
IfyouseethefollowingerrorwhenstartingtheKVMinstance:error:internalerror:processexitedwhileconnectingtomonitor:
file_ram_alloc:can'tmmapRAMpages:Cannotallocatememory,youneedtoincreasetheHugePagespoolinstep3.
11. UpdatethememoryhardlimitfortheKVMinstanceandverify,asfollows:
root@kvm:~#virshmemtunekvm1
hard_limit:unlimited
soft_limit:unlimited
swap_hard_limit:unlimited
root@kvm:~#virshmemtunekvm1--hard-limit2GB
root@kvm:~#virshmemtunekvm1
hard_limit:1953125
soft_limit:unlimited
swap_hard_limit:unlimited
root@kvm:~#
Howitworks...LibvirtandKVMsupportandtakeadvantageofHugePages.Pleasebeawarethatnoteveryworkloadwillbenefitofhavingpageslargerthanthedefault.InstancesrunningdatabasesandmemoryboundKVMinstancesaregoodusecases.Asalways,beforeenablingthisfeature,measuretheperformanceofyourapplicationinsidethevirtualmachinetoensurethatitwillbenefitfromHugePages.
Inthisrecipe,weenabledHugePagesonthehostandtheguestOSandsetahardlimitontheusablememoryfortheguest.Let'sgothroughthestepsinmoredetail.
Insteps1and2,wecheckthecurrentstateofHugePages.Fromtheoutput,wecanseethatthere'snoHugePagespoolcurrentlyallocated,indicatedbytheHugePages_TotalfieldandthecurrentsizeoftheHugePagesof2MB.
Instep3,weincreasetheHugePagespoolsizeto25000.Thechangeisondemandandwillnotpersistserverreboot.Tomakeitpersistent,youcanaddittothe/etc/sysctl.conffile.
InordertousetheHugePagesfeature,weneedtoensurethattheCPUofthehostserverhashardwaresupportforit,asindicatedbythepseandpdpe1flags,asshowninstep4.
Instep5,weconfiguretheGRUBbootloadertostartthekernelwithHugePagessupportandasetsizeof1GB.
Althoughwecanworkdirectlywiththefilesexposedbythe/procvirtualfilesystem,instep6,weinstalltheHugePagespackage,whichprovidesafewusefuluserspacetoolstolistandmanagevariousmemorysettings.Weusethehugeadmcommandinstep7tolistthesizeoftheHugePagespool.
ToenableHugePagessupportforKVM,weupdatethe/etc/default/qemu-kvmfileinstep8,mountthevirtualfilesystemforitinstep9,andfinallyreconfiguretheKVMvirtualmachinetouseHugePagesbyaddingthe<hugepages/>stanzafor
the<memoryBacking>object.
LibvirtprovidesaconvenientwaytomanagetheamountofallocatedmemoryfortheKVMguests.Instep11,wesetahardlimitof2GBforthekvm1virtualmachine.
CPUperformanceoptionsThereareafewmethodstocontrolCPUallocationandtheavailableCPUcyclesforKVMmachines-usingcgroupsandthelibvirt-providedCPUpinningandaffinityfunctions,wearegoingtoexploreinthisrecipe.CPUaffinityisaschedulerpropertythatconnectsaprocesstoagivensetofCPUsonthehostOS.
Whenprovisioningvirtualmachineswithlibvirt,thedefaultbehavioristoprovisiontheguestsonanyavailableCPUcores.Insomecases,Non-UniformMemoryAccess(NUMA)isagoodexampleofwhenweneedtodesignateacoreperKVMinstance(aswearegoingtoseeinthenextrecipe),thatit'sbettertoassignthevirtualmachinetoaspecifiedCPUcore.SinceeachKVMvirtualmachineisakernelprocess(qemu-system-x86_64morespecificallyinourexamples),wecandothisusingtoolssuchastasksetorthevirshcommand.WecanalsousethecgroupsCPUsubsystemtomanageCPUcycleallocation,whichprovidesmoregranularcontroloverCPUresourceutilizationpervirtualmachine.
GettingreadyForthisrecipe,wearegoingtoneedthefollowing:
AnUbuntuhost,withlibvirtandQEMUinstalledandconfiguredArunningKVMvirtualmachine
Howtodoit...TopinaKVMvirtualmachinetoaspecificCPUandtochangetheCPUshares,performthefollowing:
1. ObtaininformationabouttheavailableCPUcoresonthehypervisor:
root@kvm:~#virshnodeinfo
CPUmodel:x86_64
CPU(s):40
CPUfrequency:2593MHz
CPUsocket(s):1
Core(s)persocket:10
Thread(s)percore:2
NUMAcell(s):2
Memorysize:131918328KiB
root@kvm:~#
2. GetinformationabouttheCPUallocationfortheKVMguest:
root@kvm:~#virshvcpuinfokvm1
VCPU:0
CPU:2
State:running
CPUtime:9.1s
CPUAffinity:yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy
root@kvm:~#
3. PintheKVMinstanceCPU(VCPU:0)tothefirsthypervisorCPU(CPU:0)anddisplaytheresult:
root@kvm:~#virshvcpupinkvm100--live
root@kvm:~#virshvcpuinfokvm1
VCPU:0
CPU:0
State:running
CPUtime:9.3s
CPUAffinity:y---------------------------------------
root@kvm:~#
4. ListtheshareofruntimethatisassignedtoaKVMinstance:
root@kvm:~#virshschedinfokvm1
Scheduler:posix
cpu_shares:1024
vcpu_period:100000
vcpu_quota:-1
emulator_period:100000
emulator_quota:-1
root@kvm:~#
5. ModifythecurrentCPUweightofarunningvirtualmachine:
root@kvm:~#virshschedinfokvmcpu_shares=512
Scheduler:posix
cpu_shares:512
vcpu_period:100000
vcpu_quota:-1
emulator_period:100000
emulator_quota:-1
root@kvm:~#
6. ChecktheCPUsharesintheCPUcgroupssubsystem:
root@kvm:~#cat/sys/fs/cgroup/cpu/machine/kvm1.libvirt-qemu/cpu.shares
512
root@kvm:~#
7. ExaminetheupdatedXMLinstancedefinition:
root@kvm:~#virshdumpxmlkvm1
...
<vcpuplacement='static'>1</vcpu>
<cputune>
<shares>512</shares>
<vcpupinvcpu='0'cpuset='0'/>
</cputune>
...
root@kvm:~#
Howitworks...WebeginbygatheringinformationabouttheCPUresourcesavailableonthehypervisor.Fromtheoutputinstep1,wecanseethatthehostOShas40CPUsononesocket.
Instep2,wecollectinformationaboutthevirtualmachineCPUanditsaffinitywiththehostCPUs.Inthisexample,theKVMguesthasonevirtualCPU,denotedbytheVCPU:0recordandaffinitytoall40hypervisorprocessors,asindicatedbytheCPUAffinity:yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyfield.
Instep3,wepin/bindthevirtualCPUtothefirstphysicalprocessoronthehypervisor.Notethechangeintheaffinityoutput:CPUAffinity:y---------------------------------------.
Fromtheoutputofthevirshcommandinstep4,wecanobservethattheCPUsharesallocatedtotheKVMinstancearesetto1024.Thisvalueisaratio,meaningthatifanotherguesthas512shares,itwillhavetwicefewerCPUruntimethanthatofaninstancewith1024shares.Wereducethatvalueinstep5.
Insteps6and7,weconfirmthattheCPUshareswerecorrectlysetintheCPUcgroupsubsystemonthehostOS.Aswementionedearlier,CPUsharesareconfiguredusingcgroupsandcanbeadjusteddirectlyorbytheprovidedlibvirtfunctionality,bymeansofthevirshcommand.
NUMAtuningwithlibvirtNUMAisatechnologythatallowsthesystemmemorytobedividedintozones,alsonamednodes.TheNUMAnodesarethenallocatedtoparticularCPUsorsockets.Incontrasttothetraditionalmonolithicmemoryapproach,whereeachCPU/corecanaccessallthememoryregardlessofitslocality,usuallyresultinginlargerlatencies,NUMAboundprocessescanaccessmemorythatislocaltotheCPUtheyarebeingexecutedon.Inmostcases,thisismuchfasterthanthememoryconnectedtotheremoteCPUsonthesystem.
LibvirtusesthelibnumalibrarytoenableNUMAfunctionalityforvirtualmachines,aswecanseehere:
root@kvm:~#ldd/usr/sbin/libvirtd|grepnuma
libnuma.so.1=>/usr/lib/x86_64-linux-gnu/libnuma.so.1(0x00007fd12d49e000)
root@kvm:~#
LibvirtNUMAsupportsthefollowingmemoryallocationpoliciestoplacevirtualmachinestoNUMAnodes:
strict:Theplacementwillfailifthememorycannotbeallocatedonthetargetnodeinterleave:Memorypagesareallocatedinaround-robinfashionpreferred:Thispolicyallowsthehypervisortoprovidememoryfromothernodesincasethere'snotenoughmemoryavailablefromthespecifiednodes
Inthisrecipe,wearegoingtoenableNUMAaccessforaKVMinstanceandexploreitsimpactontheoverallsystemperformance.
GettingreadyForthisrecipe,wearegoingtoneedthefollowing:
AnUbuntuhost,withlibvirtandQEMUinstalledandconfiguredArunningKVMvirtualmachineThenumastatutility
Howtodoit...ToenableaKVMvirtualmachinetorunonagivenNUMAnodeandCPUusingthestrictNUMApolicy,performthefollowingsteps:
1. Installthenumactlpackageandcheckthehardwareconfigurationofthehypervisor:
root@kvm:~#apt-getinstallnumactl
...
root@kvm:~#numactl--hardware
available:2nodes(0-1)
node0cpus:012345678920212223242526272829
node0size:64317MB
node0free:3173MB
node1cpus:1011121314151617181930313233343536373839
node1size:64509MB
node1free:31401MB
nodedistances:
node01
0:1021
1:2110
root@kvm:~#
2. DisplaythecurrentNUMAplacementfortheKVMguest:
root@kvm:~#numastat-ckvm1
Per-nodeprocessmemoryusage(inMBs)forPID22395(qemu-system-x86)
Node0Node1Total
-----------------
Huge000
Heap112
Stack224
Private392159
------------------------
Total422365
root@kvm:~#
3. EdittheXMLinstancedefinition,setthememorymodetostrict,andselectthesecondNUMAnode(indexingstartsfrom0,sothesecondNUMAnodeislabeledas1),thenrestarttheguest:
root@kvm:~#virsheditkvm1
...
<vcpuplacement='static'cpuset='10-11'>2</vcpu>
<numatune>
<memorymode='strict'nodeset='1'/>
</numatune>
...
Domainkvm1XMLconfigurationedited.
root@kvm:~#virshdestroykvm1
Domainkvm1destroyed
root@kvm:~#virshstartkvm1
Domainkvm1started
root@kvm:~#
4. GettheNUMAparametersfortheKVMinstance:
root@kvm:~#virshnumatunekvm1
numa_mode:strict
numa_nodeset:1
root@kvm:~#
5. PrintthecurrentvirtualCPUaffinity:
root@kvm:~#virshvcpuinfokvm1
VCPU:0
CPU:11
State:running
CPUtime:8.4s
CPUAffinity:----------yy----------------------------
VCPU:1
CPU:10
State:running
CPUtime:0.3s
CPUAffinity:----------yy----------------------------
root@kvm:~#
6. PrinttheNUMAnodeplacementfortheKVMinstance:
root@kvm:~#numastat-ckvm1
Per-nodeprocessmemoryusage(inMBs)forPID22395(qemu-system-x86)
Node0Node1Total
-----------------
Huge000
Heap033
Stack022
Private0174174
------------------------
Total0179179
root@kvm:~#
Howitworks...WestartbyexaminingtheNUMAsetuponthehostOS.Fromtheoutputofthenumactlcommandinstep1,wecanobservethatthehypervisorhastwoNUMAnodes:node0andnode1.EachnodemanagesalistofCPUs.Inthiscase,NUMAnode1containsCPUsfrom10to19andfrom30to39andcontains64GBofmemory.Thismeansthat64GBofRAMisgoingtobelocaltothoseCPUsandaccesstothememoryfromthoseCPUsisgoingtobemuchfasterthanfromCPUsthatarepartofnode0.ToimprovememoryaccesslatenciesforaKVMguest,weneedtopinthevirtualCPUsassignedtothevirtualmachinetoCPUsthatareapartofthesameNUMAnode.
Instep2,wecanseethattheKVMinstanceusesmemoryfrombothNUMAnodes,whichisnotideal.
Instep3,weedittheguestXMLdefinitionandpintheguestonthe10thand11thCPUs,whichareapartoftheNUMAnode1,usingthecpuset='10-11'parameter.WealsospecifythestrictNUMAnodeandthesecondNUMAnodewiththe<memorymode='strict'nodeset='1'/>parameter.
Afterrestartingtheinstance,instep4,weconfirmthattheKVMguestisnowrunningusingthestrictNUMAmodeonnode1.WealsoconfirmthattheCPUpinningisindeedwhatwespecifiedinstep5.NotethattheCPUaffinityisflaggedonthe10thand11thelementsoftheCPUaffinityelement.
Fromtheoutputinstep6,wecanseethattheKVMguestisnowusingmemoryonlyfromtheNUMAnode1asdesired.
IfyourunamemoryintensiveapplicationbeforeandaftertheNUMAadjustmentandtest,youwillmostlikelyseesignificantperformancegainswhenaccessinglargeamountsofmemoryinsidetheKVMguest,thankstotheCPUandmemorylocalitythatNUMAprovides.
Thereismore...Inthisrecipe,wesawexamplesonhowtomanuallyassignaKVMprocesstoaNUMAnodebyeditingtheXMLdefinitionoftheguest.SomeLinuxdistributionssuchasRHEL/CentOS7andUbuntu16.04providethenumad(NUMAdaemon)service,whichaimsatautomaticallybalancingprocessesbetweenNUMAnodesbymonitoringthecurrentmemorytopology:
1. ToinstalltheserviceonUbuntu16.04,run:
root@kvm:~#lsb_release-a
NoLSBmodulesareavailable.
DistributorID:Ubuntu
Description:Ubuntu16.04.2LTS
Release:16.04
Codename:xenial
root@kvm2:~#aptinstallnumad
...
root@kvm:~#
2. Tostarttheservice,executethefollowingcode:
root@kvm:~#servicenumadstart
root@kvm2:~#pgrep-lfanumad
12601/usr/bin/numad-i15
root@kvm:~#
3. TomanageaspecificKVMguestwithnumad,passtheprocessIDoftheKVMinstance:
root@kvm:~#numad-S0-p$(pidofqemu-system-x86_64)
root@kvm:~#
4. TheservicewillloganyNUMArebalancingattempts:
root@kvm:~#tail/var/log/numad.log
ThuMay2521:06:422017:ChangingTHPscantimein
/sys/kernel/mm/transparent_hugepage/khugepaged/scan_sleep_millisecsfrom
10000to1000ms.
ThuMay2521:06:422017:Registeringnumadversion20150602PID12601
ThuMay2521:09:252017:AddingPID4601toinclusionPIDlist
ThuMay2521:09:252017:ScanningonlyexplicitPIDlistprocesses
root@kvm:~#
ThenumadservicecanbehelpfulonOpenStackcomputenodes,where
manualNUMAbalancingmaybetooinvolving.
TuningthekernelfornetworkperformanceMostmodernLinuxkernelsshipsufficientlytunedforvariousnetworkworkloads.Somedistributionsprovidepredefinedtuningservices(agoodexampleistunedforRedHat/CentOS),whichincludeasetofprofilesbasedontheserverrole.
Let'sgooverthestepstakenduringdatatransmissionandreception,onatypicalLinuxhost,beforewedelveintohowtotunethehypervisor:
1. Theapplicationfirstwritesthedatatoasocket,whichinturnisputinthetransmitbuffer.
2. ThekernelencapsulatesthedataintoaProtocolDataUnit(PDU).3. ThePDUisthenmovedontotheper-devicetransmitqueue.4. TheNetworkInterfaceCards(NIC)driverthenpopsthePDUfromthe
transmitqueueandcopiesittotheNIC.5. TheNICsendsthedataandraisesahardwareinterrupt.6. Ontheotherendofthecommunicationchannel,theNICreceivesthe
frame,copiesitonthereceivebuffer,andraiseshardinterrupt.7. Thekernelinturnhandlestheinterruptandraisesasoftinterrupttoprocess
thepacket.8. Finally,thekernelhandlesthesoftinterruptandmovesthepacketupthe
TCP/IPstackfordecapsulation,andputsitinareceivebufferforaprocesstoreadfrom.
Inthisrecipe,wearegoingtoexamineafewbestpracticesfortuningtheLinuxkernel,usuallyresultinginbetternetworkperformance,onmultitenantKVMhosts.
Pleasemakesurethatyouestablishabaselinebeforemakinganyconfigurationchanges,bymeasuringthehostperformancefirst.Makesmallincrementalchanges,thenmeasuretheimpactagain.Theexamplesinthisrecipearenotmeanttobecopied/pastedwithoutpriorunderstandingofthepossiblepositiveornegative
withoutpriorunderstandingofthepossiblepositiveornegativeimpacttheymightmake.Usetheexamplespresentedasaguideastowhatcanbetuned-theactualvaluesmustbecarefullyconsidered,basedontheservertypeandtheentireenvironment.
GettingreadyForthisrecipe,wearegoingtoneedthefollowing:
AnUbuntuhost,withlibvirtandQEMUinstalledandconfiguredArunningKVMvirtualmachine
Howtodoit...Totunethekernelforbetternetworkperformance,executethefollowingsteps(formoreinformationonwhatthekerneltunablesare,readtheHowitworks...section):
1. IncreasethemaxTCPsendandreceivesocketbuffersize:
root@kvm:~#sysctlnet.core.rmem_max
net.core.rmem_max=212992
root@kvm:~#sysctlnet.core.wmem_max
net.core.wmem_max=212992
root@kvm:~#sysctlnet.core.rmem_max=33554432
net.core.rmem_max=33554432
root@kvm:~#sysctlnet.core.wmem_max=33554432
net.core.wmem_max=33554432
root@kvm:~#
2. IncreasetheTCPbufferlimits:min,default,andmaxnumberofbytes.Setmaxto16MBfor1GENIC,and32Mor54Mfor10GENIC:
root@kvm:~#sysctlnet.ipv4.tcp_rmem
net.ipv4.tcp_rmem=4096873806291456
root@kvm:~#sysctlnet.ipv4.tcp_wmem
net.ipv4.tcp_wmem=4096163844194304
root@kvm:~#sysctlnet.ipv4.tcp_rmem="40968738033554432"
net.ipv4.tcp_rmem=40968738033554432
root@kvm:~#sysctlnet.ipv4.tcp_wmem="40966553633554432"
net.ipv4.tcp_wmem=40966553633554432
root@kvm:~#
3. EnsurethatTCPwindowscalingisenabled:
root@kvm:~#sysctlnet.ipv4.tcp_window_scaling
net.ipv4.tcp_window_scaling=1
root@kvm:~#
4. TohelpincreaseTCPthroughputwith1GBNICsorlarger,increasethelengthofthetransmitqueueofthenetworkinterface.Forpathswithmorethan50msRTT,avalueof5000-10000isrecommended:
root@kvm:~#ifconfigeth0txqueuelen5000
root@kvm:~#
5. Reducethetcp_fin_timeoutvalue:
root@kvm:~#sysctlnet.ipv4.tcp_fin_timeout
net.ipv4.tcp_fin_timeout=60
root@kvm:~#sysctlnet.ipv4.tcp_fin_timeout=30
net.ipv4.tcp_fin_timeout=30
root@kvm:~#
6. Reducethetcp_keepalive_intvlvalue:
root@kvm:~#sysctlnet.ipv4.tcp_keepalive_intvl
net.ipv4.tcp_keepalive_intvl=75
root@kvm:~#sysctlnet.ipv4.tcp_keepalive_intvl=30
net.ipv4.tcp_keepalive_intvl=30
root@kvm:~#
7. EnablefastrecyclingofTIME_WAITsockets.Thedefaultvalueis0(disabled):
root@kvm:~#sysctlnet.ipv4.tcp_tw_recycle
net.ipv4.tcp_tw_recycle=0
root@kvm:~#sysctlnet.ipv4.tcp_tw_recycle=1
net.ipv4.tcp_tw_recycle=1
root@kvm:~#
8. EnablethereusingofsocketsintheTIME_WAITstatefornewconnections.Thedefaultvalueis0(disabled):
root@kvm:~#sysctlnet.ipv4.tcp_tw_reuse
net.ipv4.tcp_tw_reuse=0
root@kvm:~#sysctlnet.ipv4.tcp_tw_reuse=1
net.ipv4.tcp_tw_reuse=1
root@kvm:~#
9. Startingwithkernelversion2.6.13,Linuxsupportspluggablecongestioncontrolalgorithms.Thecongestioncontrolalgorithmusedissetusingthesysctlvariablenet.ipv4.tcp_congestion_control,whichissettobic/cubicbydefaultonUbuntu.Togetalistofcongestioncontrolalgorithmsthatareavailableinyourkernel(ifyouarerunning2.6.20orhigher),runthefollowing:
root@kvm:~#sysctlnet.ipv4.tcp_available_congestion_control
net.ipv4.tcp_available_congestion_control=cubicreno
root@kvm:~#
10. Toenablemorepluggablecongestioncontrolalgorithms,loadthekernelmodules:
root@kvm:~#modprobetcp_htcp
root@kvm:~#modprobetcp_bic
root@kvm:~#modprobetcp_vegas
root@kvm:~#modprobetcp_westwood
root@kvm:~#sysctlnet.ipv4.tcp_available_congestion_control
net.ipv4.tcp_available_congestion_control=cubicrenohtcpbicvegas
westwood
root@kvm:~#
11. Forlong,fastpaths,itisusuallybettertousecubicorhtcpalgorithms.CubicisthedefaultforanumberofLinuxdistributions,butifitisnotthedefaultonyoursystem,youcandothefollowing:
root@kvm:~#sysctlnet.ipv4.tcp_congestion_control
net.ipv4.tcp_congestion_control=cubic
root@kvm:~#
12. IfthehypervisorisoverwhelmedwithSYNconnections,thefollowingoptionsmighthelpinreducingtheimpact:
root@kvm:~#sysctlnet.ipv4.tcp_max_syn_backlog
net.ipv4.tcp_max_syn_backlog=2048
root@kvm:~#sysctlnet.ipv4.tcp_max_syn_backlog=16384
net.ipv4.tcp_max_syn_backlog=16384
root@kvm:~#sysctlnet.ipv4.tcp_synack_retries
net.ipv4.tcp_synack_retries=5
root@kvm:~#sysctlnet.ipv4.tcp_synack_retries=1
net.ipv4.tcp_synack_retries=1
root@kvm:~#
13. Havingasufficientnumberofavailablefiledescriptorsisquiteimportant,sinceprettymucheverythingonLinuxisafile.Eachnetworkconnectionusesafiledescriptor/socket.Tocheckyourcurrentmaxandavailablefiledescriptors,runthefollowingcode:
root@kvm:~#sysctlfs.file-nr
fs.file-nr=1280013110746
root@kvm:~#
14. Toincreasethemaxfiledescriptors,executethefollowing:
root@kvm:~#sysctlfs.file-max=10000000
fs.file-max=10000000
root@kvm:~#sysctlfs.file-nr
fs.file-nr=1280010000000
root@kvm:~#
15. Ifyourhypervisorisusingstatefuliptablerules,thenf_conntrackkernelmodulemightrunoutofmemoryforconnectiontrackingandanerrorwillbelogged:nf_conntrack:tablefull,droppingpacket.Inordertoraisethatlimitandthereforeallocatemorememory,youneedtocalculatehowmuchRAMeachconnectionuses.Youcangetthatinformationfromtheprocfile
/proc/slabinfo.Thenf_conntrackentryshowstheactiveentries,howbigeachobjectis,andhowmanyfitinaslab(eachslabfitsinoneormorekernelpage,usually4KifnotusingHugePages).Accountingfortheoverheadofthekernelpagesize,youcanseefromtheslabinfothateachnf_conntrackobjecttakesabout316bytes(thiswilldifferondifferentsystems).Sototrack1Mconnections,you'llneedtoallocateroughly316MBofmemory:
root@kvm:~#sysctlnet.netfilter.nf_conntrack_count
net.netfilter.nf_conntrack_count=23
root@kvm:~#sysctlnet.netfilter.nf_conntrack_max
net.netfilter.nf_conntrack_max=65536
root@kvm:~#sysctl-wnet.netfilter.nf_conntrack_max=1000000
net.netfilter.nf_conntrack_max=1000000
root@kvm:~#echo250000>/sys/module/nf_conntrack/parameters/hashsize#
hashsize=nf_conntrack_max/4
root@kvm:~#
Howitworks...Instep1,weincreasethemaximumsendandreceivesocketbuffers.ThiswillallocatemorememorytotheTCPstack,butonserverswithalargeamountofmemoryandmanyTCPconnections,itwillensurethatthebuffersizeswillbesufficient.AgoodstartingpointforselectingthedefaultvaluesistheBandwidthDelayProduct(BDP)basedonameasureddelay,forexample,multiplythebandwidthofthelinktotheaverageroundtriptimetosomehost.
Instep2,weincreasethemin,default,andmaxnumberofbytesusedbyTCPtoregulatesendbuffersizes.TCPdynamicallyadjuststhesizeofthesendbufferfromthedefaultvalues.
Instep3,wemakesurethatwindowscalingisenabled.TCPwindowscalingautomaticallyincreasesthereceivewindowsize.
Formoreinformationonwindowscaling,pleaserefertohttps://en.wikipedia.org/wiki/TCP_window_scale_option.
Instep5,wereducethetcp_fin_timeoutvaluewhichspecifieshowmanysecondstowaitforafinalFINpacketbeforethesocketisforciblyclosed.Insteps6and7,wereducethenumberofsecondsbetweenTCPkeep-aliveprobesandfastrecyclingofsocketsintheTIME_WAITstate.
Asarefresher,thefollowingdiagramshowsthevariousTCPstatesaconnectioncanbein:
TCPstatediagram
Instep8,weenablethereuseofsocketsintheTIME_WAITstateonlyfornewconnections.OnhostswithlargenumbersofKVMinstances,thismighthaveasignificantimpactonhowfastnewconnectionscanbeestablished.
Insteps9and10,weenablevariouscongestioncontrolalgorithms.Thechoiceofcongestioncontrolalgorithmsisselectedwhenthekernelisbuilt.Instep11,weselectthecubicalgorithm,inwhichthewindowisacubicfunctionoftimesincethelastcongestionevent,withtheinflectionpointsettothewindowpriortothatevent.
Formoreinformationaboutnetworkcongestion-avoidancealgorithms,pleaserefertohttps://en.wikipedia.org/wiki/TCP_congestion_control.
OnsystemsexperiencinganoverwhelmingamountofSYNrequests,adjustingthemaximumnumberofqueuedconnectionrequeststhathavestillnotreceivedanacknowledgementfromtheconnectingclient,usingthetcp_max_syn_backlogandtcp_synack_retriesoptions,mighthelp.Wedothatinstep12.
Insteps13and14,weincreasethemaximumnumberoffiledescriptorsonthesystem.Thishelpswhenalargenumberofnetworkconnectionsarepresentbecauseeachconnectionrequiresafiledescriptor.
Inthelaststep,wehavethenf_conntrack_maxoption.Thisisusefulifwearetrackingconnectionsonthehypervisorusingthenf_conntrackkernelmodule.