Upload
dangque
View
226
Download
0
Embed Size (px)
Citation preview
OracleBigDataLite4.11QuickDeployment
Page1of15
QuickDeploymentStep-by-stepinstructionstodeployOracleBigDataLiteVirtualMachine
Version4.11
LastUpdated:1/10/2018
Pleasenote:Thisapplianceisfortestingandeducationalpurposesonly;itisunsupportedandnottobeusedinproduction.
CONTENTS
Introduction........................................................................................................................................................1
InstallationSteps.................................................................................................................................................3
TroubleshootingTips...........................................................................................................................................9
UsefulTips.........................................................................................................................................................11
INTRODUCTION
OracleBigDataLiteVirtualMachineprovidesanintegratedenvironmenttohelpyougetstartedwiththeOra-cleBigDataplatform.ManyOracleBigDataplatformcomponentshavebeeninstalledandconfigured-allow-ingyoutobeginusingthesystemrightaway.SeetheBigDataLitelandingpage(http://www.oracle.com/technetwork/database/bigdata-appliance/oracle-bigdatalite-2104726.html)onOTNfortutorials,videos,whitepapersandmore.
ThefollowingcomponentsareincludedonOracleBigDataLite:
• OracleEnterpriseLinux6.9
OracleBigDataLite4.11QuickDeployment
Page2of15
• OracleDatabase12cRelease1EnterpriseEdition(12.1.0.2)-includingOracleBigDataSQL-enabledex-ternaltables,OracleMultitenant,OracleAdvancedAnalytics,OracleOLAP,OraclePartitioning,OracleSpatialandGraph,andmore.
• ClouderaDistributionincludingApacheHadoop(CDH5.13.1)• ClouderaManager(5.13.1)• OracleBigDataSpatialandGraph2.4• OracleBigDataConnectors4.11
o OracleSQLConnectorforHDFS3.8.1o OracleLoaderforHadoop3.9.1o OracleDataIntegrator12c(12.2.1.3.0)o OracleRAdvancedAnalyticsforHadoop2.7.1o OracleXQueryforHadoop4.9.1o OracleDataSourceforApacheHadoop1.2.1o OracleShellforHadoopLoaders1.3.1
• OracleNoSQLDatabaseEnterpriseEdition12cR1(4.5.12)• OracleJDeveloper12c(12.2.1.2.0)• OracleSQLDeveloperandDataModeler17.3.1withOracleRESTDataServices3.0.7• OracleDataIntegrator12cR1(12.2.1.3.0)• OracleGoldenGate12c(12.3.0.1.2)• OracleRDistribution3.3.0• OraclePerfectBalance2.10.0
BigDataLiteincludessoftwareproductsthatareoptionalontheOracleBigDataAppliance(BDA),includingOracleNoSQLDatabaseEnterpriseEdition,OracleBigDataSQL,OracleBigDataSpatialandGraphandOracleBigDataConnectors.
Togetstarted,youshouldperformthefollowinghighleveltasks:
• Ensureyourhostcomputermeetstherequirementsforrunningthevirtualmachine• DownloadandinstallVirtualBoxandVirtualBoxExtensionPack–whichisacross-platformvirtualiza-
tionapplication• DownloadthezipfilesfromOTNanduse7-ziptoextracttheappliancefile• CreatetheVMbyimportingtheBigDataLite-xxx.ovafileintoVirtualBox• Startthemachine!• LogintoBigDataLiteusing:
OracleBigDataLite4.11QuickDeployment
Page3of15
userid:oraclepassword:welcome1
• OpentheStartHeredocumentonthedesktopfordetailsabouttheenvironment.
INSTALLATIONSTEPS
1. Prepareyourhostsystem.
• Minimum8GBofrealmemory;moreisbetter.5GmemorywillbededicatedtotheVM.Somedetails:o BigDataLitew/oClouderaManager(CM)requires5GBmemorydedicatedtotheVMo BigDataLitew/CMrunningrequires10GBmemory
• TurnonVirtualAssistfeaturesintheBIOS(usuallydonebydefault).RefertoTroubleshootingtipsinStep12formoredetails.
• ~73GBdiskspaceneededtodownloadandinstall:o 11zipfiles:~24GB(Canberemovedafterextraction)o .ovafile:~24GB(Resultofunzip.Itcanberemovedafterimporting–butnicetokeeparound)o importedimage:~49GB.
• Downloadandinstall7Zip.• Downloadandinstallmd5sum(optional)
2.DownloadandinstallOracleVirtualBoxandVirtualBoxExtensionPack(version4.3.26andaboveisrec-ommend).
ItishighlyrecommendedthatyoualsoinstalltheVirtualBoxGuestAdditions–whichprovidesen-hancedmousesupport,clipboardsupportandsharingoffilesbetweenthevirtualmachineanditshost.
3. DownloadBigDataLitefilesfromOracleTechnologyNetwork
• GototheBigDataLitelandingpageat:http://www.oracle.com/technetwork/database/bigdata-appliance/oracle-bigdatalite-2104726.html
• ClicktoaccepttheOTNlicense.• Downloadallofthefilestoasingledirectory.
4. Afterallzipfilesaredownloaded,extractthemwith7zip.
InWindows:• Right-clickonfileBigDataLite-xxx.7z.001fileandselect7-zip->ExtractHere
OracleBigDataLite4.11QuickDeployment
Page4of15
InLinux:
• Runthefollowingcommandtoextractthefile:
7za e BigDataLite-xxx.7z.001
Result:TheextractionwillcreatetheBigDataLite-xxx.ovaappliancefile.ThisfilewillbeusedtocreateanewmachineinVirtualBox.Thissinglefilecontainstheentiremachinedefinition,includingthephysicaldisksaswellasdefaultsforthemachineconfiguration(e.g.network,CPUs,memory,etc.).
5. StartOracleVirtualBoxManagerandImporttheAppliance
AfterstartingVirtualBox,createthenewvirtualmachineusingtheimportwizard:• ClickFile->ImportAppliancetolaunchtheimportwizard• IntheImportVirtualAppliancepage,clickOpenappliance…• LocatetheBigDataLite-xxx.ovafileandclickOpen.ClickNext.• Appliancesettingsgiveyouanoverviewoftheconfiguration.Inthisstep,youmayneedto
updatethelocationoftheVirtualDiskImages.Thesefileswillinitiallyuseapproximately50GBofdisk–andcanexpandasyouusethemachine.ChangethedefaultlocationoftheVir-tualDiskImagesifthecurrentlocationdoesnothaveenoughcapacity.Note,theVMuses4separatedisks.ThesedisksshouldnotbemovedafterimportingtheVM:
OracleBigDataLite4.11QuickDeployment
Page5of15
ClickImport
Result:AnewBigDataLite-xxxVirtualMachineisavailableintheVirtualBoxManager:
OracleBigDataLite4.11QuickDeployment
Page6of15
6.VirtualMachineSettings
BigDataLiteisconfiguredwiththefollowingparameters,whichisidealforan8GBhostsystem:• 5GBRAM• 2processors• NATNetworkAdapter
Asmentionedearlier,ifyouwouldliketorunClouderaManager,youwillneedtoincreasethememoryallocatedtotheVM.YoucanupdatethesedefaultsettingsbyselectingtheBigDataLiteVMinVirtualBoxManagerandclickingSettings:
YoucanupdatetheamountofmemoryandCPUsallocatedtothemachinefromtheSystemtab.PleasereviewtheVirtualBoxdocumentationtolearndetailsaboutupdatingtheconfiguration.
7.StartyourBigDataLite!
Double-clickonBigDataLite-xxxtostarttheVM.Logonastheoracleusertogetstarted:id: oraclepassword: welcome1
ThiswillbringyoutoaLinuxdesktop.TheStartHeredocumentonthedesktopwillprovideyoude-tailsabouttheinstalledsoftware,idsandpasswords,andmore.
OracleBigDataLite4.11QuickDeployment
Page7of15
BigDataLiteDesktop
8. Start/StopServices
UsetheStart/StopServicesscripttostart/stopyourservices(youcanalsotypeservicesatthelinuxprompt).Servicesonlyappeariftheminimummemoryrequirementismet.Ensurethatyoustartallservicesrequiredforanapplication(e.g.HiverequiresHDFS).
OracleBigDataLite4.11QuickDeployment
Page8of15
Starting/Stoppingservices
9. BigDataLiteSamples
BigDataLitesamplesarenowavailableongithub(https://github.com/oracle/BigDataLite).ThismakesiteasytoupdateyourVMwiththelatestcollateralavailable.ThereisabranchforeachBigDataLiterelease–startingwithBigDataLite4.3.0.Double-clicktheRefreshSamplesiconandfollowthein-structionstorefreshyourlocalcopyofthesampleswiththehostedrepository:
OracleBigDataLite4.11QuickDeployment
Page9of15
RefreshSamplesThesamplesaredownloadedtothe/home/oracle/srcdirectory.Note:thecontentsinthisdirectorywillbedeletedaspartoftherefreshprocess–soremembertosaveanycustomizationsyoumadeinthisdirectory.Also,directory/home/oracle/movieisabindmountthatreferstodirectory/home/oracle/src/movie.Thismeansthatthesetwofoldersareequivalent;anychangestothecontentsinonefolderisreflectedintheother.
TROUBLESHOOTINGTIPS
1.Failedtoimportappliance<path>/BigDataLite-xxx.ova.CouldnotopenthecurrentfileintheOVApack-age(VERR_TAR_CHKSUM_MISMATCH).
Thiserroroccurswhenattemptingtoimportanimage.Tosolvetheproblem,simplyupdateyourver-sionofVirtualBox.
2.ERROR:“Failedtoopenasessionforthevirtualmachine”whenattemptingtostartthemachine
TheVMrequiresthathardwarevirtualizationisenabledforthehostcomputer.Ifitisnotenabled,youwillreceivethefollowingerrorwhenattemptingtostartthemachine:FailedtoopenasessionforthevirtualmachineBigDataLite-xxx.VT-xisdisabledintheBIOS(VERR_VMX_MSR_VMXON_DISABLED)YoucanupdatetheBIOSatboottimeforthehost.Ensurethatboth"Intel(R)VirtualizationTechnolo-gy"and"Intel(R)VT-dFeature"areenabled.PleasereviewtheinstructionsprovidedbyyourhardwaremanufacturerthatdescribeupdatingBIOSsettings.
OracleBigDataLite4.11QuickDeployment
Page10of15
3.ERROR:"Thiskernelrequiresanx86-64CPU,butonlydetectedani686CPU.Unabletoboot-pleaseuseakernelappropriateforyourCPU"
EnsurethatyourmachineisrunningOracle(64-bit)Linux.IntheVMsettingsselect"General"andsettheVersiontoOracle(64-bit).
4.ERROR:“ImplementationoftheUSB2.0controllernotfound!”
InstalltheappropriateVirtualBoxExtensionPackforyourversionofVirtualBox.
5.Whiletheimageisstartingup,youinitiallyseeafewmessagesandthenablankscreen
Ifthishappens,thenstoptheimage.GobacktoVirtualBoxManagerandclickontheimage’sSet-tings.GotoDisplayandincreasetheVideomemorytoatleast6MB.Starttheimageandyoushouldnolongerseetheblankscreen.
6.Unabletocopy/pastebetweenthehostandBigDataLite
First,ensurethatyouhaveinstalledtheVirtualBoxGuestAdditions.IfyouhaveinstalledGuestAddi-tionsandcopy/pastestilldoesn’twork–gototheDevicesmenuandensurethatSharedClipboardisenabled.
7.IncorrectJDKusedbyHadoop.Note:thiswillnotoccurunlessyouaddJDK1.7totheVM.
TherearenumerouserrorsthatcanoccurbecauseCDHisusingJDK1.7insteadofJDK1.8(seebelow).TousethecorrectJDK:
• Openaterminalwindowbysingle-clickingtheTerminaliconinthetaskbar.• Enterthefollowingcommand:
o sudovi/etc/default/bigtop-utilso Addthefollowinglineattheendofthefile:
exportJAVA_HOME=/usr/java/latest• RestarttheVMtomakesurethattheupdatedsettingsareused.
ManyoftheseerrorsoccurwhenODIisattemptingtorunjobsontheVM.Forexample:
• com.sunopsis.core.SnpsForeignKeyReferencedObjectMissingException:ODI-10244:CannotimportDataTypeConversion:949-62-184(withglobalid:7663ea6c-f7c3-4d0b-bd80-
OracleBigDataLite4.11QuickDeployment
Page11of15
3117c1e06868andinternalid:949|62|184)becausereferencedTechnologywithglobalid7ffd3917-2b27-35a3-96a7-3bde26098531doesnotexist.
• Error:QueryResult:Unsupportedmajor.minorversion52.0
USEFULTIPS
1.SharingfoldersbetweenBigDataLiteanditshost
VirtualBoxGuestAdditionsprovidesausefulfeaturethatenablesyoutoshareahost’sfolderwithinthevirtualmachine.Toconfigureasharedfolder:
• SelectBigDataLite-xxxfromthelistofmachinesandclickSettings
OracleBigDataLite4.11QuickDeployment
Page12of15
• GotoSharedFoldersandclickAddFolder
• SpecifythefolderyouwouldliketoshareandselectAuto-mount.ThenclickOK.Below,the
H:\driveisbeingshared.
Youcanrepeatthisprocessformultiplesharedfolders.
OracleBigDataLite4.11QuickDeployment
Page13of15
• ToaccessthesharedfolderfromBigDataLite,navigateto/media/sf_<folder_name>
2.UsingtheVMasaHadoopClient
YoucanuseBigDataLiteasaHadoopclient–allowingyoutouseitasadevelopmentenvironmentfortheOracleBigDataAppliance.ToconnecttheBigDataLitetoBDA,logintotheClouderaManagerin-stancethatismanagingyourHadoopclusterandselectActions->ViewClientConfigurationURLs:
ThiswilltakeyoutotheClientConfigurationURLspage.SelecttheappropriateClientConfiguration(e.g.MapReduce):
OracleBigDataLite4.11QuickDeployment
Page14of15
Thiswilldownloadazipfilecontainingallthefilesrequiredtoconnecttothecluster.Afterdownload-ingthefile,executethefollowingtwosteps:
• Unzipthefileinto/home/oracle.• SettheappropriateenvironmentvariablethatwillpointBigDataLitetotheremotecluster.
Forexample,ifyoudownloadedbothhiveandmapreduceconfigurations:export HADOOP_CONF_DIR=/home/oracle/hadoop-conf export HIVE_CONF_DIR=/home/oracle/hive-conf
3.ConnectingexternalclientstoBigDataLite
YoumaywanttoconnecttoBigDataLitefromyourhostcomputer.VirtualBoxportforwardinghasbeenimplementedallowingyoutoconnecttheguestusingabrowserorSSH.Youwillconnecttoserver“localhost:<port#>”andVirtualBoxwillforwardtherequesttotheguest.ListedbelowaretheportmappingsforservicesrunninginBigDataLite:
Service PortNameNode 50070SecondaryNameNode 50090ResourceManager 8088NodeManager 8042JobHistoryServer 19888Hue 8888OozieServer 11000HistoryServer 18088
OracleBigDataLite4.11QuickDeployment
Page15of15
CloudeaManager 7180Moviedemo 7001SolrAdmin 8983Oracle 1521Apex 8080SSH 2222NoSQLDBAdmin 5001BigDataSpatial&Graph 8045