Doing Oracle Analytics THIS Way? STOP IT · 2019. 10. 21. · BMM #2 Creating multi-fact models with non-conformed dimensionality without properly configuring LTS content and levels

DoingOracleAnalyticsTHISWay?

STOP IT

www.dimensionality.ch @Nephentur #obihackers |freenode

« Christian, you’re either boring or shocking...there is no middle way with you! »

--- Francesco TisiotLiterally 2 hours ago

Who am I?


• Owner of Dimensionality GmbH in Switzerland• Hacking OBI since 2001 (nQuire + Peregrin aquisitions by Siebel)• Oracle ACE Business Analytics• Speaker at OpenWorld, KScope, regional Oracle User Groups...• Part-time blogger on Analytics, BI, DWH (http://dimensionality.ch)• Full-time IRC (freenode | #obihackers) and OTN addict• Oracle Analytics trainer for Oracle University since 2006• Proud geek and gamer...but NOT a developer!• Responding to any and all questions 24 / 7 – especially on IRC

Who am I?


By the way...thief :-P


Whythispresentation?• OracleAnalyticshasmaturedandisquitewide-spread• Isseenas/consideredashavingbecomeacommodityknow-how• Asaresultit’softendoneworsethanever…• …especiallycomparedtowhenitwasanicheskill• Hence:Backtobasicspresentationseries• And:WorstPracticesmeanssomeoneelseranintothewall!


Disclaimer• Allrealusecasesfromclientsandintegrators• Allfromproductionsystems– noabandonedtry-outs• Justificationsandexplanationsmaysoundsarcasticbutthey’renot• IlikerantingaboutfailsbutIdon’tlikePowerPointorKeynote• Memes+text>>>fancygraphics• Don’tbotherreadingallthetext;mystoriesarealotfunnier


OracleAnalytics


OracleAnalytics


OracleAnalyticsCloud


…abitmoretomorrow,Tuesday16th – 1415 – Room10

#Failcategories• The3RPDlayers• System,DevOps andsecurity• Multidimensional• Front-end• Analyses+dashboard• Goingjusttoofar





PhysicallayerIncorrecthandlingofdatatypes

Example?INTvs.DOUBLEWhy?• “InoursourcewestoreitasINT.”• ProperdatatypehandlingunderestimatedortotallyignoredResult? • (−)2’147’483’648/-1• You’reingoodcompany.Hello,YouTube!

RandomlysettingdatabasefeaturesWhy?“Becauseyoureaditonablog/forum“Result?• GeneratedSQLchangessometimesdrastically• Painfultroubleshooting

Physicallayer#2Datatypeconversiononcolumnsusedforjoinspecificationsinthephysicallayermodel

Why? • Existingdatamodelsareeithernotproperlybuiltandcleanorwerenevermeanttobeusedtogetherforanalyticalpurposes• Databasedeveloperscan’tbebotheredtochangetheirtablecreationscripts• “Bestpractices”ofhowtocreatetablesareorganizationalrulesResult? Bye-byeoptimization,hellofulltablescans


Physicallayer#3UsingODBCinsteadofnativedrivers(likeOCI)

Why?• Toomuchhassletoinstallrequireddrivers• CorporateITsecuritydoesnotallowpost-setupchangestoinstalledOS/softwareor“their”serversResult? • Allnativecommunicationdisabled(optimization,functionshippingetc.)• Hugeperformanceimpactpotential



BusinessModelandMappingLayerUnwittinglyreturningwrongcalculationresultsduetoignoranceoftheimpactofpre-aggregation/post-aggregation

Background?• Derivedlogicalcolumnsexecutepostaggregation• PhysicalmappingcalculationsexecutepreaggregationWhy? OneofthemostoftenmisunderstoodfactsabouttheBMMlayerResult? “Wrong”resultsdependingoninterpretation


BMM#2Creatingmulti-factmodelswithnon-conformeddimensionalitywithoutproperlyconfiguringLTScontentandlevels

Why? • THEmostmisunderstoodfactsabouttheBMMlayer!• Non-conformeddimensionalmodelingspecificallyandLTScontentlevelsingeneralResult? • Missingresultsfromcross-starqueries• Analysessimplywon’trun• Wrongresultswhenqueriesimplicitlyhitwrongfacts

Are those cakes?No, Content

Levels!


BMM#2Non-conformeddimensionality/LTScontentlevels


BMM#3ENABLE_DIMENSIONALITYvariableisNOTyoursavior• Introducedin11.1.1.9duetosharedcodebasewithBICS• Forcesallnon-conformeddimensionsto“GrandTotal”• BICS,DV,DVDandVAalwaystreatENABLE_DIMENSIONALITY=true• FullOBIEEstillhasmoreoptionsandpossibilities

-------------------- SQLRequest,logicalrequesthash:c849d2caSETVARIABLEQUERY_SRC_CD='VisualAnalyzer',SAW_SRC_PATH='{"viewID":"view!3","currentCanvas":"canvas!1","path":"/shared/00.VAProjects/Datasources Examples/OBIEE-SalesAnalysis"}',PREFERRED_CURRENCY='USD',ENABLE_DIMENSIONALITY=1;SELECT0s_0,XSA('prodney'.'OBIEE- SampleSales')."Columns"."Brand"s_1,XSA('prodney'.'OBIEE- SampleSales')."Columns"."Year"s_2,CAST(NULLASDOUBLE)s_3,CAST(NULLASDOUBLE)s_4,REPORT_AGGREGATE(XSA('prodney'.'OBIEE- SampleSales')."Columns"."Revenue"BY)s_5,REPORT_SUM(CASEWHENXSA('prodney'.'OBIEE- SampleSales')."Columns"."Revenue"<0THEN

XSA('prodney'.'OBIEE- SampleSales')."Columns"."Revenue"ELSE0ENDBYXSA('prodney'.'OBIEE- SampleSales')."Columns"."Year")s_6,REPORT_SUM(CASEWHENXSA('prodney'.'OBIEE- SampleSales')."Columns"."Revenue">0THEN

XSA('prodney'.'OBIEE- SampleSales')."Columns"."Revenue"ELSE0ENDBYXSA('prodney'.'OBIEE- SampleSales')."Columns"."Year")s_7,XSA('prodney'.'OBIEE- SampleSales')."Columns"."Revenue"s_8

FROMXSA('prodney'.'OBIEE- SampleSales')FETCHFIRST5000001ROWSONLY


BMM#4Notcreatingdimensionalhierarchiesforanyofyourdimensions“becausetheRPDacceptsitanditeliminatesthehassleofmodelingcontentproperly“

Why? • Somedevelopersdon’tseethepointofcontentlevels• “Wedidn’tneedthatinOtherReportDrawingTool42.”• GettingthemrightcanbeabigconfigurationeffortforcomplexmodelsResult? • BasicallyitmeansturningoffoneofthetwocorefunctionalitiesoftheRPD:dimensionalhierarchies.(TheotheroneisLTSs)• Nolevel-basedmeasurecalculationslike“RegionalSalesTotal”either


BMM#5Notmodelingtimedimensionsaspropertimedimensions

Why? Toomucheffortandneedscorrectlyconfigured(orETL’d)chronologicalkeysResult? Bye-byetimeseriescalculations

Usingnon-unique(time)levelkeyswhichwillimplicitlyrollupdataExample?Using “January”ratherthan“January2015”Why? Expectationthat“timeworksautomatically”Result? Dataaggregatedacrossallmembersofdifferenttimehierarchybranches


BMM#6Creatingfact-to-factjoinsinsteadofconformeddimensionalityordimension-to-dimensionjoinstorepresentsnowflakesinthelogicalmodelWorstcase:Supportwith“1=1”joinspeconthephysicallayer“tomakethingswork”

Why? • Seeabove• PeoplemisinterpretJeffMcQuigg’s presentationafterfindingitonGoogleResult? • Cartesianproductsbuiltoverresultsets• Completelyinefficientquerysenttothedatasource• Allbenefitsofcross-factanalysisoverconformeddimensionalitiesareeliminated

PresentationLayerUNIONs,UNIONs,UNIONsallovertheplace…evenforsingleSubjectAreaanalyses

Why?• “Really?WecandothisintheRPD?”• Typical“Weusedreportingtool XYZbeforeandthereitwentlikethis.”

Result?WHYdidyoubuythisnewtoolagain?


PresentationLayerRenaminghappensonlyinthepresentationlayer.

KeepingalotoftechnicalUPPER_CASE_NAMESandSURROGATE_KEYSunlesssomeonecomplainsandthenonlychangeinthepresentationlayerratherthanthebusinesslayer

Why? Techiedevelopersdon’tnecessarilythinkaboutbusinesslanguageandenduserneedsResult? • AllnamesmustbechangedNtimesforNpresentationlayerrepresentationsofeachobjectineachSubjectAreaetc.• Life-cyclemanagementandmaintenanceofcatalogobjectsbecomesanightmare



PresentationLayer– Aliashandling• Changesneverproperlypropagatedtoandreflectedinthepresentationcatalogue

– CatalogManagerwithcommandlineinterfaceunderused• Tensofaliasesforeachpresentationlayerobject

withmostofthembeingreferencedNtimesinthepresentationcatalogue

• Worstcase:Presentationlayerobjectnamesgetreassignedandexistingpresentationcatalogueobjectseffectivelyreferencedifferentcolumnsthanoriginallyintended

Aliashandling#2Why? • Impactisunderestimated–“It’sjustanaliassowhat?”• DifferentdevelopersforRPDandpresentationcatalogueResult?• XMLpointstobothaliasesandactualnames• Nameconflicts&invalidateXMLreferences• Presentationcatalogueobjectskeepworkingimplicitlyuntilsomeonetouchesthealiases

• Lineagesolutionsnevertakealiasesintoaccount• Impactanalysisbecomesdifficult/meaningless


Aliashandling#312cversionsmitigatethissomewhat


Renamingcanstillbe(re-)enabledintheAdminToolthough



System,DevOps andSecurityNotturningonUsageTracking

Why?• TablesusednottothelikingofcorporateDBAsandtheirown“bestpractices”• “Whatfor?”• LegalconstraintsincertaincountriesResult? NoUTstatistics,noSummaryAdvisorandgenerallynowayofknowingwhat’shappeningwithoutlogfileanalysis:

• Monitoringperformance,systemusage,bottlenecksandpeaks• Usagepatternsandgeneralusage(ornon-usage!)ofthecatalogobjects• Impactanalysisforupgradeplanning• etc.

System,DevOpsandSecurity#2Notsimplyleveraging usingtheSampleApp UsageTrackingsolution

Why? • “Wecanbuilditbetter”.• Manypeoplestilldon’tknowSampleApp andthecontentitoffers.Result? • Re-inventingthewheelistheultimatewasteoftime.• Home-brewsolutionhardlyeverascompleteorintegrated.• Additionaly, SampleApp contains tons of additional lifecycle and administraton content pre-built and for free

TurningoncachingforperformanceWhy? It’saquickfixandaneasysmokescreentohideactualissuesintheDB.Result? • Withoutpropercachepurgingandseedingstrategystaledataisassured.• TheACTUALproblemforperformanceisneveraddressedandwillpersist!



System,DevOpsandSecurity#3Turningoffloggingforperformancereasons

Why?Manyofblogs,articlesand“performancechecklists”repeatthisblindlyResult?• D’uh;justneveraskmetodiagnoseanythingifyoudothat• AntonyHeljula completelydispelledthismythwithaverydetailedanalysisandpresentationsoyoudon’thavetodoit!

TryingtosolvesourceperformanceissuesinOracleBIinsteadofend-to-endWhy? • DBteam:“Webuiltthedatamarts andthey’refine.• ETLteam:“OracleBImustbeabletodothisasourOtherProduct doesit.”• Generally:ThinkingthatOBIphysicallytransformsandstoresdataResult?Againtheactualsourceofperformanceissuesisn’ttackled

System,DevOpsandSecurity#4UsingOBItosolvedataqualityissues

Example? CAST,CASEWHEN,FILTERassmokescreensintheRPDoranalysesWhy? Asaboveplus:OBImakesdataqualityissuesvisibleandisheldresponsibleResult?• Performancedegradation• Dataqualityissuesarehiddenratherthansolved• Additionalerrorswhensourcedatachanges• Disappearingdatawhennewsourcedataoutsidethehardcodedcases• ELSEstatementbecomestheonlypossiblesavior(“Invalidconversion”)



System,DevOpsandSecurity#5StoringallusersandgroupsintheWLS-embeddedLDAP

Why?Itexistsandthere’snoneedtointegrateResult?• Nointegrationwithcorporatesecurity• Deploymentbecomesevenmore“interesting”• Nopropermechanismforpasswordexpiration,checkandmanagement

Notusingsourcecontrolforallyourartifacts(RPD,catalog,customfiles)Why?Admineffortperceivedaswasteoftime/non-productiveResult?• Deploymentpainswhenchasingfilesandconfigurations• Evertriedtogobacktoacertainstateofanenvironment?• Hat-tiptoRobinMoffat:SourceControlandConcurrentDevelopmentforOBIEE

System,DevOpsandSecurity#6Manualsecuritymanagementonuserbasisratherthanapplicationroles

Why?GroupstorolesplusinheritancemorecomplicatedtodesignResult?Muchmoreadmineffort;tradingbrainpowerforlabour

CheatyourwayaroundimproperLDAPstructuresviaWLSsecurityrealmqueries

Why?InternalITsecuritydepartmentsandexistingsecuritystandards/setupsResult?Loginslowsdown

(((&(uid=%u)(objectclass=person))(&(|(uid=user0001)(uid=user0002)(uid=user0003)(uid=user0004)(uid=user0005)(uid=user0006)…(uid=user1000)))




Multidimensional/CuuuubesWhy?• Multidimensionalsourcesarethoughttobehave

exactlylikeallothers• “We’realreadydoingBIwithcubessowhyshould

therebeanyissue?”• Pullin“everythingHyperionbecauseit’sanother

analyticalOracleproductanditmustwork”– Guesswhat?ManyHyperionproductsaren’tanalyticalinnature…


Multidimensional#2Result?• HugeperformanceimpactsuptotakingdownHyperionservers• Irresponsiveandunusablefront-end• Cubesaren’tnecessarilysuitedfordynamicanalyticalaccess:

– Sparsecubes– ExtremelyspecificstoredmemberswhichrequireNobligatorydimensions+level+member references

ineachquery– Dynamiccalculations

• AnalysesmustbebuiltlikereportswithNfixedaxisplusfixedmemberselectiononlevels

• LossofhabitualdynamicnavigationpossibilitiesofOBI


Multidimensional#3• Measurehierarchyvsflattenedmeasures


Measurecolumns

NOmeasurehierarchy!

DimDim Dim Fact

Fact

Multidimensional#4• Directlyaccessingcubespurpose-builtfor

Hyperionapplicationsforanalyticalanalyses(pivot,drill,slice/dice,memberselection)…

• Accessinghugelysparsecubes(Essbase orMSAS)withmassiveamountsofextremelycomplexdynamiccalculationsforanalyticalanalyses

• ExpectingthatOBIEEisaperfectlynativeMDXgeneratortosmoothoutEPM/reportingoutlinebuilds




Front-endusageUsingOBIasanExcelexportingtool

Why?• EveryonehasExcel• DataontheirExcelsheetsmakesusersfeelsecure•“BIsystemsnevergiveyouenough.”•“Imustabletochangethedata.”Result?• DatapushedDB->OBIS->OBIPS->Browser->Excel• Datamultipliedinstaticformandopentochangeforeveryone

Multi-millionrowsettingsandhundredsofcolumnsininstanceconfig.xmlWhy?SeeaboveResult?TakingdownwholeOBIfarmswhengenerating5mrow,200columnPDFsbasedonananalysiswithalternatingrowhighlightingandconditionalicons.



Front-endusage#2UseOBIasadataentrytool(insteadofAPEXetal)

Why?• “OBIisawebapplicationandmustsupportthis”.• “ButOracleprovideswrite-backfunctionality!Nowmakeitworkwithfieldvalidationandpick-lists.”• “Wedon’twanttobuyanothertool.”• “Wedon’twanttouseanothertool.”Result?


Front-endusage#3Prettymuchrandomsettingofpresentationcatalogaccesspermissions

Why?• “Wewillonlyshowpeoplehowtousedashboards.”• “OurITguyshavesetthisup;andanywaysmostofourdataisvisibletoanybody.”Result?•AnybodywitheitheraccesstothecataloginterfaceorabasicknowledgeofURLscanaccessanythinghewants•Sameholdstrueforalluserpopulationswithdashboardcreationrights;publicorinpersonalfolder•“Funny”situationswhereBIConsumers caneliminateBIAdministrators fromaccesspermissions



AnalysesanddashboardsLeaving50columnsinthe"Excluded"sectionofanalyses

Why? Uncleandevelopment,“wemaystillneedthis”attitudeResult?• AllcolumnsretrievedfromtheDBeverytime• “Exclude” meanstheviewsandvisualizations– notthequery


Analysesanddashboards#2Settingallanalysesto"IncludeNullValues"=TRUEbydefault

Why? • “Wemightmisssomethingotherwise.”• Smoothoverdataqualityand/ormodellingissuesResult?Asusual:performance.



Analysesanddashboards#3Complexpost-calculationsandanalysisbasedaggregationsonhugedatastreamshandedtotheOBIPresentationServer

Example?MillionsofrowshandedtotheOBIPStocreatea15rowtableviewWhy?• NoaccesstoRPD• Separatedteams/rolesfor“back-end”and“front-end”• Silomentalitiesandpoorcommunication• HesitationtochangetheRPDforfearofdamagingexistingsolutions• Out-of-synchdevelopmentcyclesResult?• Again:performance,maintenance• AlltheclevernessoftheOBIserver’ssyntaxandquerytuningiseliminated

Analysesanddashboards#4• Clutterthedashboardswith30promptswithoutdefaults• Puttingalldashboardobjectsinseparatecondition-drivensections• Over-use(nested)viewselectors,columnselectorsoneverypage

– Pseudolibertyinsteadofthought-outcontenttellingaspecificstory– End-userconfusion“Whichofmyproblemsdoesthissolveagain?”

• ConstantlyreshufflingthepresentationcatalogfolderstructureandobjectswithoutproperlyadjustingtheXMLreferences

• Agentnotificationspam





Goingjusttoofar• GivingDirectDatabase

Requestcapabilitytousersandprovidingthemwiththeconnectionpoolnames

• Showingandgivinguserstheaccessto&IssueRawSQL• Usinghackstointeractwith

thedesktopPCfromabrowser• UsingJShackstomakeOBIEE

reactlikeanERP/dataentrysystem(masks,requiredfields,fielddependenciesandvalidations)


Documents

Doing Oracle Analytics THIS Way? STOP IT · 2019. 10. 21. · BMM #2 Creating multi-fact models with non-conformed dimensionality without properly configuring LTS content and levels