View
0
Download
0
Category
Preview:
Citation preview
DoingOracleAnalyticsTHISWay?
STOP IT
www.dimensionality.ch @Nephentur #obihackers |freenode
« Christian, you’re either boring or shocking...there is no middle way with you! »
--- Francesco TisiotLiterally 2 hours ago
Who am I?
www.dimensionality.ch @Nephentur #obihackers |freenode
• Owner of Dimensionality GmbH in Switzerland• Hacking OBI since 2001 (nQuire + Peregrin aquisitions by Siebel)• Oracle ACE Business Analytics• Speaker at OpenWorld, KScope, regional Oracle User Groups...• Part-time blogger on Analytics, BI, DWH (http://dimensionality.ch)• Full-time IRC (freenode | #obihackers) and OTN addict• Oracle Analytics trainer for Oracle University since 2006• Proud geek and gamer...but NOT a developer!• Responding to any and all questions 24 / 7 – especially on IRC
Who am I?
www.dimensionality.ch @Nephentur #obihackers |freenode
By the way...thief :-P
www.dimensionality.ch @Nephentur #obihackers |freenode
Whythispresentation?• OracleAnalyticshasmaturedandisquitewide-spread• Isseenas/consideredashavingbecomeacommodityknow-how• Asaresultit’softendoneworsethanever…• …especiallycomparedtowhenitwasanicheskill• Hence:Backtobasicspresentationseries• And:WorstPracticesmeanssomeoneelseranintothewall!
www.dimensionality.ch @Nephentur #obihackers |freenode
Disclaimer• Allrealusecasesfromclientsandintegrators• Allfromproductionsystems– noabandonedtry-outs• Justificationsandexplanationsmaysoundsarcasticbutthey’renot• IlikerantingaboutfailsbutIdon’tlikePowerPointorKeynote• Memes+text>>>fancygraphics• Don’tbotherreadingallthetext;mystoriesarealotfunnier
www.dimensionality.ch @Nephentur #obihackers |freenode
OracleAnalytics
www.dimensionality.ch @Nephentur #obihackers |freenode
OracleAnalytics
www.dimensionality.ch @Nephentur #obihackers |freenode
OracleAnalyticsCloud
www.dimensionality.ch @Nephentur #obihackers |freenode
…abitmoretomorrow,Tuesday16th – 1415 – Room10
#Failcategories• The3RPDlayers• System,DevOps andsecurity• Multidimensional• Front-end• Analyses+dashboard• Goingjusttoofar
www.dimensionality.ch @Nephentur #obihackers |freenode
#Failcategories• The3RPDlayers• System,DevOps andsecurity• Multidimensional• Front-end• Analyses+dashboard• Goingjusttoofar
www.dimensionality.ch @Nephentur #obihackers |freenode
www.dimensionality.ch @Nephentur #obihackers |freenode
PhysicallayerIncorrecthandlingofdatatypes
Example?INTvs.DOUBLEWhy?• “InoursourcewestoreitasINT.”• ProperdatatypehandlingunderestimatedortotallyignoredResult? • (−)2’147’483’648/-1• You’reingoodcompany.Hello,YouTube!
RandomlysettingdatabasefeaturesWhy?“Becauseyoureaditonablog/forum“Result?• GeneratedSQLchangessometimesdrastically• Painfultroubleshooting
Physicallayer#2Datatypeconversiononcolumnsusedforjoinspecificationsinthephysicallayermodel
Why? • Existingdatamodelsareeithernotproperlybuiltandcleanorwerenevermeanttobeusedtogetherforanalyticalpurposes• Databasedeveloperscan’tbebotheredtochangetheirtablecreationscripts• “Bestpractices”ofhowtocreatetablesareorganizationalrulesResult? Bye-byeoptimization,hellofulltablescans
www.dimensionality.ch @Nephentur #obihackers |freenode
Physicallayer#3UsingODBCinsteadofnativedrivers(likeOCI)
Why?• Toomuchhassletoinstallrequireddrivers• CorporateITsecuritydoesnotallowpost-setupchangestoinstalledOS/softwareor“their”serversResult? • Allnativecommunicationdisabled(optimization,functionshippingetc.)• Hugeperformanceimpactpotential
www.dimensionality.ch @Nephentur #obihackers |freenode
www.dimensionality.ch @Nephentur #obihackers |freenode
BusinessModelandMappingLayerUnwittinglyreturningwrongcalculationresultsduetoignoranceoftheimpactofpre-aggregation/post-aggregation
Background?• Derivedlogicalcolumnsexecutepostaggregation• PhysicalmappingcalculationsexecutepreaggregationWhy? OneofthemostoftenmisunderstoodfactsabouttheBMMlayerResult? “Wrong”resultsdependingoninterpretation
www.dimensionality.ch @Nephentur #obihackers |freenode
BMM#2Creatingmulti-factmodelswithnon-conformeddimensionalitywithoutproperlyconfiguringLTScontentandlevels
Why? • THEmostmisunderstoodfactsabouttheBMMlayer!• Non-conformeddimensionalmodelingspecificallyandLTScontentlevelsingeneralResult? • Missingresultsfromcross-starqueries• Analysessimplywon’trun• Wrongresultswhenqueriesimplicitlyhitwrongfacts
Are those cakes?No, Content
Levels!
www.dimensionality.ch @Nephentur #obihackers |freenode
BMM#2Non-conformeddimensionality/LTScontentlevels
www.dimensionality.ch @Nephentur #obihackers |freenode
BMM#3ENABLE_DIMENSIONALITYvariableisNOTyoursavior• Introducedin11.1.1.9duetosharedcodebasewithBICS• Forcesallnon-conformeddimensionsto“GrandTotal”• BICS,DV,DVDandVAalwaystreatENABLE_DIMENSIONALITY=true• FullOBIEEstillhasmoreoptionsandpossibilities
-------------------- SQLRequest,logicalrequesthash:c849d2caSETVARIABLEQUERY_SRC_CD='VisualAnalyzer',SAW_SRC_PATH='{"viewID":"view!3","currentCanvas":"canvas!1","path":"/shared/00.VAProjects/Datasources Examples/OBIEE-SalesAnalysis"}',PREFERRED_CURRENCY='USD',ENABLE_DIMENSIONALITY=1;SELECT0s_0,XSA('prodney'.'OBIEE- SampleSales')."Columns"."Brand"s_1,XSA('prodney'.'OBIEE- SampleSales')."Columns"."Year"s_2,CAST(NULLASDOUBLE)s_3,CAST(NULLASDOUBLE)s_4,REPORT_AGGREGATE(XSA('prodney'.'OBIEE- SampleSales')."Columns"."Revenue"BY)s_5,REPORT_SUM(CASEWHENXSA('prodney'.'OBIEE- SampleSales')."Columns"."Revenue"<0THEN
XSA('prodney'.'OBIEE- SampleSales')."Columns"."Revenue"ELSE0ENDBYXSA('prodney'.'OBIEE- SampleSales')."Columns"."Year")s_6,REPORT_SUM(CASEWHENXSA('prodney'.'OBIEE- SampleSales')."Columns"."Revenue">0THEN
XSA('prodney'.'OBIEE- SampleSales')."Columns"."Revenue"ELSE0ENDBYXSA('prodney'.'OBIEE- SampleSales')."Columns"."Year")s_7,XSA('prodney'.'OBIEE- SampleSales')."Columns"."Revenue"s_8
FROMXSA('prodney'.'OBIEE- SampleSales')FETCHFIRST5000001ROWSONLY
www.dimensionality.ch @Nephentur #obihackers |freenode
BMM#4Notcreatingdimensionalhierarchiesforanyofyourdimensions“becausetheRPDacceptsitanditeliminatesthehassleofmodelingcontentproperly“
Why? • Somedevelopersdon’tseethepointofcontentlevels• “Wedidn’tneedthatinOtherReportDrawingTool42.”• GettingthemrightcanbeabigconfigurationeffortforcomplexmodelsResult? • BasicallyitmeansturningoffoneofthetwocorefunctionalitiesoftheRPD:dimensionalhierarchies.(TheotheroneisLTSs)• Nolevel-basedmeasurecalculationslike“RegionalSalesTotal”either
www.dimensionality.ch @Nephentur #obihackers |freenode
BMM#5Notmodelingtimedimensionsaspropertimedimensions
Why? Toomucheffortandneedscorrectlyconfigured(orETL’d)chronologicalkeysResult? Bye-byetimeseriescalculations
Usingnon-unique(time)levelkeyswhichwillimplicitlyrollupdataExample?Using “January”ratherthan“January2015”Why? Expectationthat“timeworksautomatically”Result? Dataaggregatedacrossallmembersofdifferenttimehierarchybranches
www.dimensionality.ch @Nephentur #obihackers |freenode
BMM#6Creatingfact-to-factjoinsinsteadofconformeddimensionalityordimension-to-dimensionjoinstorepresentsnowflakesinthelogicalmodelWorstcase:Supportwith“1=1”joinspeconthephysicallayer“tomakethingswork”
Why? • Seeabove• PeoplemisinterpretJeffMcQuigg’s presentationafterfindingitonGoogleResult? • Cartesianproductsbuiltoverresultsets• Completelyinefficientquerysenttothedatasource• Allbenefitsofcross-factanalysisoverconformeddimensionalitiesareeliminated
PresentationLayerUNIONs,UNIONs,UNIONsallovertheplace…evenforsingleSubjectAreaanalyses
Why?• “Really?WecandothisintheRPD?”• Typical“Weusedreportingtool XYZbeforeandthereitwentlikethis.”
Result?WHYdidyoubuythisnewtoolagain?
www.dimensionality.ch @Nephentur #obihackers |freenode
PresentationLayerRenaminghappensonlyinthepresentationlayer.
KeepingalotoftechnicalUPPER_CASE_NAMESandSURROGATE_KEYSunlesssomeonecomplainsandthenonlychangeinthepresentationlayerratherthanthebusinesslayer
Why? Techiedevelopersdon’tnecessarilythinkaboutbusinesslanguageandenduserneedsResult? • AllnamesmustbechangedNtimesforNpresentationlayerrepresentationsofeachobjectineachSubjectAreaetc.• Life-cyclemanagementandmaintenanceofcatalogobjectsbecomesanightmare
www.dimensionality.ch @Nephentur #obihackers |freenode
www.dimensionality.ch @Nephentur #obihackers |freenode
PresentationLayer– Aliashandling• Changesneverproperlypropagatedtoandreflectedinthepresentationcatalogue
– CatalogManagerwithcommandlineinterfaceunderused• Tensofaliasesforeachpresentationlayerobject
withmostofthembeingreferencedNtimesinthepresentationcatalogue
• Worstcase:Presentationlayerobjectnamesgetreassignedandexistingpresentationcatalogueobjectseffectivelyreferencedifferentcolumnsthanoriginallyintended
Aliashandling#2Why? • Impactisunderestimated–“It’sjustanaliassowhat?”• DifferentdevelopersforRPDandpresentationcatalogueResult?• XMLpointstobothaliasesandactualnames• Nameconflicts&invalidateXMLreferences• Presentationcatalogueobjectskeepworkingimplicitlyuntilsomeonetouchesthealiases
• Lineagesolutionsnevertakealiasesintoaccount• Impactanalysisbecomesdifficult/meaningless
www.dimensionality.ch @Nephentur #obihackers |freenode
Aliashandling#312cversionsmitigatethissomewhat
www.dimensionality.ch @Nephentur #obihackers |freenode
Renamingcanstillbe(re-)enabledintheAdminToolthough
#Failcategories• The3RPDlayers• System,DevOps andsecurity• Multidimensional• Front-end• Analyses+dashboard• Goingjusttoofar
www.dimensionality.ch @Nephentur #obihackers |freenode
System,DevOps andSecurityNotturningonUsageTracking
Why?• TablesusednottothelikingofcorporateDBAsandtheirown“bestpractices”• “Whatfor?”• LegalconstraintsincertaincountriesResult? NoUTstatistics,noSummaryAdvisorandgenerallynowayofknowingwhat’shappeningwithoutlogfileanalysis:
• Monitoringperformance,systemusage,bottlenecksandpeaks• Usagepatternsandgeneralusage(ornon-usage!)ofthecatalogobjects• Impactanalysisforupgradeplanning• etc.
System,DevOpsandSecurity#2Notsimplyleveraging usingtheSampleApp UsageTrackingsolution
Why? • “Wecanbuilditbetter”.• Manypeoplestilldon’tknowSampleApp andthecontentitoffers.Result? • Re-inventingthewheelistheultimatewasteoftime.• Home-brewsolutionhardlyeverascompleteorintegrated.• Additionaly, SampleApp contains tons of additional lifecycle and administraton content pre-built and for free
TurningoncachingforperformanceWhy? It’saquickfixandaneasysmokescreentohideactualissuesintheDB.Result? • Withoutpropercachepurgingandseedingstrategystaledataisassured.• TheACTUALproblemforperformanceisneveraddressedandwillpersist!
www.dimensionality.ch @Nephentur #obihackers |freenode
www.dimensionality.ch @Nephentur #obihackers |freenode
System,DevOpsandSecurity#3Turningoffloggingforperformancereasons
Why?Manyofblogs,articlesand“performancechecklists”repeatthisblindlyResult?• D’uh;justneveraskmetodiagnoseanythingifyoudothat• AntonyHeljula completelydispelledthismythwithaverydetailedanalysisandpresentationsoyoudon’thavetodoit!
TryingtosolvesourceperformanceissuesinOracleBIinsteadofend-to-endWhy? • DBteam:“Webuiltthedatamarts andthey’refine.• ETLteam:“OracleBImustbeabletodothisasourOtherProduct doesit.”• Generally:ThinkingthatOBIphysicallytransformsandstoresdataResult?Againtheactualsourceofperformanceissuesisn’ttackled
System,DevOpsandSecurity#4UsingOBItosolvedataqualityissues
Example? CAST,CASEWHEN,FILTERassmokescreensintheRPDoranalysesWhy? Asaboveplus:OBImakesdataqualityissuesvisibleandisheldresponsibleResult?• Performancedegradation• Dataqualityissuesarehiddenratherthansolved• Additionalerrorswhensourcedatachanges• Disappearingdatawhennewsourcedataoutsidethehardcodedcases• ELSEstatementbecomestheonlypossiblesavior(“Invalidconversion”)
www.dimensionality.ch @Nephentur #obihackers |freenode
www.dimensionality.ch @Nephentur #obihackers |freenode
System,DevOpsandSecurity#5StoringallusersandgroupsintheWLS-embeddedLDAP
Why?Itexistsandthere’snoneedtointegrateResult?• Nointegrationwithcorporatesecurity• Deploymentbecomesevenmore“interesting”• Nopropermechanismforpasswordexpiration,checkandmanagement
Notusingsourcecontrolforallyourartifacts(RPD,catalog,customfiles)Why?Admineffortperceivedaswasteoftime/non-productiveResult?• Deploymentpainswhenchasingfilesandconfigurations• Evertriedtogobacktoacertainstateofanenvironment?• Hat-tiptoRobinMoffat:SourceControlandConcurrentDevelopmentforOBIEE
System,DevOpsandSecurity#6Manualsecuritymanagementonuserbasisratherthanapplicationroles
Why?GroupstorolesplusinheritancemorecomplicatedtodesignResult?Muchmoreadmineffort;tradingbrainpowerforlabour
CheatyourwayaroundimproperLDAPstructuresviaWLSsecurityrealmqueries
Why?InternalITsecuritydepartmentsandexistingsecuritystandards/setupsResult?Loginslowsdown
(((&(uid=%u)(objectclass=person))(&(|(uid=user0001)(uid=user0002)(uid=user0003)(uid=user0004)(uid=user0005)(uid=user0006)…(uid=user1000)))
www.dimensionality.ch @Nephentur #obihackers |freenode
#Failcategories• The3RPDlayers• System,DevOps andsecurity• Multidimensional• Front-end• Analyses+dashboard• Goingjusttoofar
www.dimensionality.ch @Nephentur #obihackers |freenode
Multidimensional/CuuuubesWhy?• Multidimensionalsourcesarethoughttobehave
exactlylikeallothers• “We’realreadydoingBIwithcubessowhyshould
therebeanyissue?”• Pullin“everythingHyperionbecauseit’sanother
analyticalOracleproductanditmustwork”– Guesswhat?ManyHyperionproductsaren’tanalyticalinnature…
www.dimensionality.ch @Nephentur #obihackers |freenode
Multidimensional#2Result?• HugeperformanceimpactsuptotakingdownHyperionservers• Irresponsiveandunusablefront-end• Cubesaren’tnecessarilysuitedfordynamicanalyticalaccess:
– Sparsecubes– ExtremelyspecificstoredmemberswhichrequireNobligatorydimensions+level+member references
ineachquery– Dynamiccalculations
• AnalysesmustbebuiltlikereportswithNfixedaxisplusfixedmemberselectiononlevels
• LossofhabitualdynamicnavigationpossibilitiesofOBI
www.dimensionality.ch @Nephentur #obihackers |freenode
Multidimensional#3• Measurehierarchyvsflattenedmeasures
www.dimensionality.ch @Nephentur #obihackers |freenode
Measurecolumns
NOmeasurehierarchy!
DimDim Dim Fact
Fact
Multidimensional#4• Directlyaccessingcubespurpose-builtfor
Hyperionapplicationsforanalyticalanalyses(pivot,drill,slice/dice,memberselection)…
• Accessinghugelysparsecubes(Essbase orMSAS)withmassiveamountsofextremelycomplexdynamiccalculationsforanalyticalanalyses
• ExpectingthatOBIEEisaperfectlynativeMDXgeneratortosmoothoutEPM/reportingoutlinebuilds
www.dimensionality.ch @Nephentur #obihackers |freenode
#Failcategories• The3RPDlayers• System,DevOps andsecurity• Multidimensional• Front-end• Analyses+dashboard• Goingjusttoofar
www.dimensionality.ch @Nephentur #obihackers |freenode
Front-endusageUsingOBIasanExcelexportingtool
Why?• EveryonehasExcel• DataontheirExcelsheetsmakesusersfeelsecure•“BIsystemsnevergiveyouenough.”•“Imustabletochangethedata.”Result?• DatapushedDB->OBIS->OBIPS->Browser->Excel• Datamultipliedinstaticformandopentochangeforeveryone
Multi-millionrowsettingsandhundredsofcolumnsininstanceconfig.xmlWhy?SeeaboveResult?TakingdownwholeOBIfarmswhengenerating5mrow,200columnPDFsbasedonananalysiswithalternatingrowhighlightingandconditionalicons.
www.dimensionality.ch @Nephentur #obihackers |freenode
www.dimensionality.ch @Nephentur #obihackers |freenode
Front-endusage#2UseOBIasadataentrytool(insteadofAPEXetal)
Why?• “OBIisawebapplicationandmustsupportthis”.• “ButOracleprovideswrite-backfunctionality!Nowmakeitworkwithfieldvalidationandpick-lists.”• “Wedon’twanttobuyanothertool.”• “Wedon’twanttouseanothertool.”Result?
www.dimensionality.ch @Nephentur #obihackers |freenode
Front-endusage#3Prettymuchrandomsettingofpresentationcatalogaccesspermissions
Why?• “Wewillonlyshowpeoplehowtousedashboards.”• “OurITguyshavesetthisup;andanywaysmostofourdataisvisibletoanybody.”Result?•AnybodywitheitheraccesstothecataloginterfaceorabasicknowledgeofURLscanaccessanythinghewants•Sameholdstrueforalluserpopulationswithdashboardcreationrights;publicorinpersonalfolder•“Funny”situationswhereBIConsumers caneliminateBIAdministrators fromaccesspermissions
#Failcategories• The3RPDlayers• System,DevOps andsecurity• Multidimensional• Front-end• Analyses+dashboard• Goingjusttoofar
www.dimensionality.ch @Nephentur #obihackers |freenode
AnalysesanddashboardsLeaving50columnsinthe"Excluded"sectionofanalyses
Why? Uncleandevelopment,“wemaystillneedthis”attitudeResult?• AllcolumnsretrievedfromtheDBeverytime• “Exclude” meanstheviewsandvisualizations– notthequery
www.dimensionality.ch @Nephentur #obihackers |freenode
Analysesanddashboards#2Settingallanalysesto"IncludeNullValues"=TRUEbydefault
Why? • “Wemightmisssomethingotherwise.”• Smoothoverdataqualityand/ormodellingissuesResult?Asusual:performance.
www.dimensionality.ch @Nephentur #obihackers |freenode
www.dimensionality.ch @Nephentur #obihackers |freenode
Analysesanddashboards#3Complexpost-calculationsandanalysisbasedaggregationsonhugedatastreamshandedtotheOBIPresentationServer
Example?MillionsofrowshandedtotheOBIPStocreatea15rowtableviewWhy?• NoaccesstoRPD• Separatedteams/rolesfor“back-end”and“front-end”• Silomentalitiesandpoorcommunication• HesitationtochangetheRPDforfearofdamagingexistingsolutions• Out-of-synchdevelopmentcyclesResult?• Again:performance,maintenance• AlltheclevernessoftheOBIserver’ssyntaxandquerytuningiseliminated
Analysesanddashboards#4• Clutterthedashboardswith30promptswithoutdefaults• Puttingalldashboardobjectsinseparatecondition-drivensections• Over-use(nested)viewselectors,columnselectorsoneverypage
– Pseudolibertyinsteadofthought-outcontenttellingaspecificstory– End-userconfusion“Whichofmyproblemsdoesthissolveagain?”
• ConstantlyreshufflingthepresentationcatalogfolderstructureandobjectswithoutproperlyadjustingtheXMLreferences
• Agentnotificationspam
www.dimensionality.ch @Nephentur #obihackers |freenode
#Failcategories• The3RPDlayers• System,DevOps andsecurity• Multidimensional• Front-end• Analyses+dashboard• Goingjusttoofar
www.dimensionality.ch @Nephentur #obihackers |freenode
www.dimensionality.ch @Nephentur #obihackers |freenode
Goingjusttoofar• GivingDirectDatabase
Requestcapabilitytousersandprovidingthemwiththeconnectionpoolnames
• Showingandgivinguserstheaccessto&IssueRawSQL• Usinghackstointeractwith
thedesktopPCfromabrowser• UsingJShackstomakeOBIEE
reactlikeanERP/dataentrysystem(masks,requiredfields,fielddependenciesandvalidations)
www.dimensionality.ch @Nephentur #obihackers |freenode
Recommended