View
3
Download
0
Category
Preview:
Citation preview
RESEARCH PAPER
Differentiate Your Product With Unstructured Data Analytic Capabilities
January 2020
Sponsored by
Differentiate Your Product With Unstructured Data Analytic Capabilities
2 Computing | research paper | sponsored by OpenText
This document is property of Incisive Media. Reproduction and distribution of this publication in any form without prior written permission is forbidden.
ContEntS• Introduction p3
• Keyfindings p4
• Unstructureddata–howmuchisthereandwhydoes it matter? p4
• Obstaclestoinsight p8
• Unstructureddata’sstructuralchallenges p9
• Newambitions,oldtechnology p10
• Conclusion p13
• Aboutthesponsor,OpenText p14
Differentiate Your Product With Unstructured Data Analytic Capabilities
Computing | research paper | sponsored by OpenText 3
IntroductionThedigitaleconomyisgeneratingunprecedentedquantitiesofdata.Businesseshavemoredataat their disposal than ever before. Some of the most valuable data is unstructured and many businessesarestrugglingtounlockitsfullpotential.Withoutit,organisationsaren’tgeneratinga complete picture of their operations. Software providers and channel partners can help their customerstoextractinsight–andsubsequentlyvalue–fromthismountainofunstructureddata,and differentiate themselves from their competitors.
Computingsurveyed150individualsactivelyinvolvedinusing,testing,evaluatingorprocuringdataanalyticstoolsattheirorganisation.Theyrepresentcompaniesfromawidevarietyofindustries,includingbankingandfinance,logistics,manufacturing,retailandeducationtoexplorethestateofdataanalyticsinenduserorganisations,opinionsandusecasesaroundstructuredversusunstructureddata,theobstaclesmanyorganisationsarefacinginextractingvalueandinsightsfrom their unstructured data and how software providers and channel partners can respond to thesechallenges,addvalueandbecomekeyplayersintheshapingofdataanalyticsstrategyandsoftware.
Differentiate Your Product With Unstructured Data Analytic Capabilities
4 Computing | research paper | sponsored by OpenText
Key findings• Sixtypercentofrespondentsstatedthatamajority–between51and100percentoftheirdata–wasunstructured.
• Thetopmotivatorsforunstructureddataanalysisweregainingprocessefficiencies,includingautomation,andthedesireforgreatercustomerinsight.
• The top three use cases for unstructured data analysis were all very much customer focused –trendtracking,sentimentanalysisandbusinesscommunicationanalysis.
• Despitethecompetitivebenefitsthatunstructureddataanalysiscanconfer,only50percentofthosewesurveyedwereactuallydoingit.
• Sixty-ninepercentofrespondentssaidtheirorganisationsfounditquiteorextremelydifficulttogatherandanalyseunstructureddata.
• Fewerthanaquarterofrespondentssaidtheirorganisationhadbeensuccessfulingatheringandanalysingunstructureddata.
• Theprimaryobstaclestoderivinginsightfromunstructureddatawerecostofimplementation,ashortageoftechnicalexpertise,andthetechnologyitselffallingshortandbeingdifficulttointegrate.
• Only36percentofrespondentswereemployedinorganisationswhereaccesstodataanalysis was democratized with the empowerment of individual users.
• There is a mismatch between reported levels of data democratization (36 percent) and the proportionofrespondentsreportingtheuseofdatavisualisationdashboardswhicharesupposed to enable this democratization (53 percent).
• Only30percentofourrespondentswereusingArtificialIntelligence/MachineLearningaspart of their data analytics platforms.
Unstructured data – how much is there and why does it matter?Thefirststeptounderstandingwhyunstructureddataissoimportant,isunderstandingjusthowmuch of it exists in relation to more structured data.
Organisationshaveatorrentofdatapouringineachdayanditisbothincreasinginvolumeanddiversifyinginformat.Afewyearsago,mostunstructureddatausedtoconsistofmainlytext–emails,textfilesandsoon.Agreatdealofunstructureddataisnowintheformofvideoorphotosfromsocialmedia,audiofiles,andwebsitecontent.
There is also a considerable amount of data already in the public domain that can be accessed by businesscustomers–suchasgovernmentorcensusdataandmarketdatafromorganisationslikethe IMF. This public data is often in unstructured or semi-structured formats.
Differentiate Your Product With Unstructured Data Analytic Capabilities
Computing | research paper | sponsored by OpenText 5
Computingaskedrespondentstooursurveytosharewithusjusthowmuchoftheirorganisation’sdataisunstructured.Wecanseefromtheillustrationbelowthat60percentofthoseansweringthisquestionsaidthata majority of their data–between51and100percent–wasunstructured.Thisfindinganswersthefirstpartofthequestionofwhyunstructureddataisimportant.Unstructureddatamatterspartlybecauseitsimplycomprisesamajorityofthedataavailable for analysis.
Fig. 1 : What proportion of your organisation’s data is unstructured (such as emails and other communications, contracts, documents, and social media posts)?
Don’t know (1%)
0 - 25% (11%)
26 - 50% (27%)
51 - 75% (45%)
76 - 100% (15%)
Unstructureddataalsomattersbecauseoftheinsightthatcanbedrawnfromitsanalysis.Computingaskedtherespondentswhoseemployerswereanalysingunstructureddatawhattheyhopedtoachievefromdoingso(Fig.2,see next page).Thetopmotivator,citedby43percent,wasthescopeforreducedspendthroughgreaterefficiency/automation.Certainly,thepotentialforproductivitygainsbythebusinessprocessanalysisandoptimisationthatunstructureddataanalysiscaninformishuge.Businessprocessesofallkindsinvolveincreasingvolumesandvarietyof data and more applications are involved in business processes than in the past.
Manybusinessesarelookingtooptimisebusinessprocessesviagreaterautomationinordertobecomefasterandmoreflexible,andultimatelydeliverthespeedandpersonalisationofservicesthatconsumersareincreasinglydemanding.Replacingslow,manual,error-proneprocesseswithautomated,integratedworkflowscanbringabouthugeefficiencygains.However,inordertobesuccessful,themovetoautomationmustbedata-informedtoensurethatoptimisationisappliedwhereitcandeliverthebiggestgains.
Differentiate Your Product With Unstructured Data Analytic Capabilities
6 Computing | research paper | sponsored by OpenText
Fig. 2 : What are the main motivations for your organisation’s decision to gather and analyse unstructured data? [three maximum]
Reduced spend through efficiency/automation
Customer insights
Productivity gains
Competitive advantage
Improved working environment
New business models
Company reputation
Technology upgrades
New products
43%
42%
36%
26%
22%
21%
21%
15%
12%
Onlymarginallylessamotivatorwascustomerinsight,whichwascitedby42percent.Customerservicedataistypicallyunstructured–phonecalls,email,socialmediaposts,reviewsitesandevenchatbotinteractions.Gainingvisibilityofthisdataandanabilitytovisualise,sliceanddiceitenablesbusinessestogaininsightsintowhattheircustomerswantnow–andpredictwhattheyarelikelytowantinthefuture.Businessescangettoknowtheircustomersonalevelthatisimpossible when structured data is the only source of information.
Inarelatedquestion,wealsoaskedsurveyrespondentsexactlywhattheywereusingtheirunstructureddatafor.Thetopthreeusecaseswereallverymuchfocusedoncustomers,asthediagrambeneathillustrates.Themostpopularusecasewastrendtracking.Therearetwolevelsoftrendtracking.Manyofourrespondentswillbeanalysinghistoricalunstructureddatatolookfortrendsintheircustomerbehaviourwhichtheycanutiliseinthefuture,perhapsbysegmentingtheircustomersandtargetingthemaccordingly.
Differentiate Your Product With Unstructured Data Analytic Capabilities
Computing | research paper | sponsored by OpenText 7
Fig. 3 : What does your organisation currently use unstructured data analysis for? [select all that apply]
Trend tracking
Customer sentiment tracking
Business communications analysis
Intelligent recommendations
Paperwork processing
Predictive maintenance
AI-augmented document digitisation
Product recommendation
Other
44%
39%
39%
29%
28%
25%
19%
16%
4%
UsingAI-basedtextminingtoolsallowsbusinessesnottojustseewhathasoccurredbuttominehugedatasetsfrommanysources–blogs,socialmedia,reviewsitesarejustafewexamples–tomakepredictionsaboutmarkettrendsforkeytopics,productsorservices.Itisinterestingthatthesecond and third most popular use cases of unstructured data analysis were customer sentiment trackingandbusinesscommunicationsanalysis.Sentimenttrackingisamachinelearning-ledapplicationofAItopredictwhatcustomerswillwantanddeliveragainstexpectationsitbymeansof hyper personalised services and recommendations.
Lessloyalcustomerscanbeidentifiedandtargeted,whichmeansthatadvertisingcanbedeliveredwithlaser-likeaccuracy.BusinessCommunicationsAnalysisalsofocusesoncustomersto ensure that they are happy with the type of communications they have with companies rather thanbeingdissatisfiedwithinteractionstheydeemimpersonalorfaceless.
Differentiate Your Product With Unstructured Data Analytic Capabilities
8 Computing | research paper | sponsored by OpenText
obstacles to insightTheinsightthatunstructureddataanalysiscanprovideisunparalleled.Theusecasessetoutaboveillustratethispotential.Giventhesebenefitsitisreasonabletoexpectthatthemajority ofrespondentswouldbeanalysingthisdata.Theproblemis,asrevealedbythisresearch, that50percentofthemaren’t.
Thisposesanobviousquestion.Why?
Computingaskedaseriesofquestionsabouttheday-to-daytechnicalandorganisationalaspectsofunstructureddataanalysisanditistheanswerstothesequestionswhichprovideahintastowhybusinessesaren’tmakingbetteruseofthispreciousresource.Thefirstisshownbelowin Fig.4.Theanswerstothisquestionmakeitimmediatelyapparentthatmanymoreorganisationsthannotarefindingiteitherdifficultorverydifficulttogatherandanalysetherelevantdata.
Fig. 4 : on a scale of 1 to 5, ‘1’ being ‘extremely difficult’ and ‘5’ being ‘extremely easy’, how easy would it be for your organisation to gather and analyse its unstructured data at present?
Arelatedquestionaskingrespondentstoratethesuccessoftheirorganisationsatgatheringandanalysingunstructureddateyieldedaresultwhichcanprobablybebestdescribedasmediocre.Twenty-fourpercentawardedtheirbusinessesamoderatelyorhighlysuccessfulrating.Eighteenpercentdidtheopposite.Theremaining58percentchosefaintpraiseandwentwithamiddleratingofneitherparticularlysuccessfulnorunsuccessful.
Extremely difficult
23% 46% 22% 5% 4%
Extremely easy
1 2 3 4 5
Differentiate Your Product With Unstructured Data Analytic Capabilities
Computing | research paper | sponsored by OpenText 9
Unstructured data’s structural challengesWhat’sgoingwrong?Computingaskedrespondentstochoosethreemainobstaclestotheirorganisationsabilitytogatherandanalyseunstructureddata.Comingasasurprisetoabsolutelynoonewasthemostfrequentlycitedissue–costofimplementation(45percent.)Sofarthismakessense,butit’sworthsteppingbacktoviewthisfromapartnerperspectivetoaskwhy costs are so steep.
Fig. 5 : What are the main obstacles to your organisation’s ability to gather and analyse unstructured data? [three maximum]
Cost of implementation
Lack of expertise
Technology falling short
Technology poorly integrated
Security
Identifying possible applications
Privacy
Resistance on the ‘shop-floor’
Compliance
Governance
Building a business case
Don’t see the benefit
Board-level resistance
Ethical issues
Other
45%
41%
25%
25%
19%
17%
17%
15%
14%
13%
13%
10%
9%
8%
0%
Differentiate Your Product With Unstructured Data Analytic Capabilities
10 Computing | research paper | sponsored by OpenText
Publiccloudeconomicswereabeguilingprospecttobusinessesdesperatetofindcostsavingsintheharshbusinessclimateprevailinginthewakeofthefinancialcrisis.However,therealityhasn’tquitematchedexpectationsbecauseofthecomplexitythatmulti-cloudandhybridcloudinfrastructureshaveengendered.
It’simpossibletoanalysedataifyoucan’tseeit,andthisgrowingcomplexityhaslimitedvisibilityofdataandmadeitmuchhardertointegratedataandapplications.Itisnotacoincidencethatthesecondmostfrequentlycitedobstaclewaslackofexpertise,which41percentofrespondentsmentioned.Theexpertiseneededtointegrateandmanagethisfragilewebofapplicationsandunderlyingdataisinshortsupply–andthat’sbeforeyouevengettothedatascienceandanalysisskills.Seventy-eightpercentagreed,albeittovaryingdegrees,that,“itisdifficulttosuccessfullyintegratedataanalyticstoolsintoourtechnologystack.”
Thesefindingsrepresentarealopportunityforchannelpartnerslookingtoaddgreatervaluetofillskillsgapswheretheyexistonashort-termbasis.Employingstafffull-timeonpermanentcontractsisnotnecessarilyanapproachwellsuitedtobusinessestryingtomovetoamoreagilewayofworking–whichmanyaretryingtodo.WorkingAgileinvolvessmallteamstodeliverprojectswhichthenbreakandreassembleasrequired.Ideallyyoubringpeopletotheproject,ratherthantheotherwayaround.Byreducingthecostsandrisksassociatedwithpermanentemployment,theuseofcontracthiresfromchannelpartnersforprojectmanagement,consultancyandtechnicalskillsatalllevelscouldhelpbusinessestoshowsomequickwinsfordataanalysisprojects. It pays to lean on those that have been there and done it before.
new ambitions, old technologyAftercostandalackofexpertise,thenextmostcommonlyraisedobstaclestounstructureddataanalysisbothrelatedtotechnology.Twenty-sevenpercentofrespondentsraisedtechnologyfallingshortandpoorlyintegratedtechnologyasaproblem.Wehavediscussedthedifficultiesofintegrationalreadybutthesubjectoftechnologyfailingtokeeppacewithrequirementsisonethatrequiresfurtherexamination.
Therearelikelytobetwodimensionstothetechnologyproblem.Thefirstrelatestoorganisationalstructures–whoisresponsiblefordataanalysis?Thesecondismoreaboutthetoolsthemselves.Thesetwodimensionsarecloselyrelated–theeasierusersfindittovisualiseandcutdata,themoredemocratisedyourdataanalysiscanbe.Organisationalandprocessbarriersaresignificant.Eighty-fourpercentofrespondentsagreedtoatleastsomeextentthat,“itisdifficulttosuccessfullyintegratedataanalyticstoolsintoourbusinessprocesses.”
Ofcourse,mostorganisationshaveamixtureofanalysismethodsbeingapplied.Forty-twopercentofourrespondent’semployershadadedicatedcentralteamand44percenthadlocalspecialistteamsthroughouttheirorganisations.In36percentofcases,respondentsstatethatthey had a more democratised systems where individual employees were empowered.
Differentiate Your Product With Unstructured Data Analytic Capabilities
Computing | research paper | sponsored by OpenText 11
Fig. 6 : What organisational structures do you have in place to deliver data, insights and business value from data? [select all that apply]
26%
We have a dedicated central team
We have local specialist teams across the business
29%
19%
4%
4%
5%
7%
6%
Other
We practice data-democratisation (where individual employees use data analytics to inform their day-to-day decision making)
Fig.6indicatessomeoverlapwithorganisationsrunningmultiplescenarios.Forexample,it’squitepossibletohavealimitednumberofusersanalysingtheirowndataalongsideacentralresourcetohelpwithmoreintricateanalysis.Thelackofhybridapproacheshereshowsthelackofmaturityinmostorganisationsdataanalysisstrategy–particularlyatthoselargerorganisationswhostandtobenefitfromacombinationofmethods.
Theactualtoolsandtechnologiesinusevarysignificantly,asFig.7shows(see over).
Differentiate Your Product With Unstructured Data Analytic Capabilities
12 Computing | research paper | sponsored by OpenText
Fig. 7 : How does your organisation analyse its data? [select all that apply]
Thefindingstothisquestionprovideaseriesofcluesintowhyunstructureddataanalysisisprovidingsolittlecollectiveinsight.Thefactthatthereisamismatchbetweenreportedlevelsofdatademocratisationandtheproportionofrespondentsusingdatavisualisationdashboards,whicharesupposedtoenablethisdemocratisation,indicatesthatthevisualisationdashboardsaren’tasuserfriendly,orsuitableforbroaduse,astheyshouldbe.
The essence of self-service dashboards is that they enable the user to slice and dice data with ease butthecombinedfindingsheresuggestthattheystillneedconsiderableinputfromdataexpertstogettotheinsight.Thistimelagoninsightisprofoundlyunhelpfulforbusinessestryingtomakemoreinformeddecisionsfaster–afactnotlostonthe90percentofrespondentswhoagreedto atleastsomeextentwiththestatement,“datademocratisationisthebestapproachtobuilding adata-guidedorganisation.”
ThehighshowingofBusinessIntelligencetoolsalsohintsatpartofthereasonwhyunstructureddataanalysisisprovingsodifficultforsomanybusinesses.BItoolsjustweren’tdesignedwiththehugequantitiesofunstructureddatathatbusinessesneedtoanalyseinmind.ManyBIdashboardsonlyidentifyeventsaftertheyhaveoccurred.Thisisnotinsightful–iteffectivelyamountstomonitoring.Itcantellyouaboutaneventinthepastbutthereisoftenlimitedcontextarounddatapoints,sounderstandingandinsightisequallylimited.
ThirtypercentofourrespondentswereusingAIormachinelearning,andwhilstthesealgorithmscanderiveinsightsfromhugedatasets,theygenerallyrequiredatascienceskillstooperate,andwehavealreadyestablishedthattheseareinveryshortsupply.Itisalsolikelythatthelargeamountofbespokeandthird-partytoolswhichourrespondentsreportarecreatingfurthercomplexityandslowingdowntheprocessofderivinginsight.
Ourresearchsuggeststhataconsiderableproportionofbusinesseswhoaretryingtobringunstructureddataintotheiranalysisaredoingsobytryingtoadaptrelativelyoldanalysistoolstoaverynewchallenge.Thereportedsuccessratessuggestthisisunlikelytoproveawinningstrategy.
Business intelligence tools
Data visualisation dashboards
Third-party tools
Bespoke tools
AI/Machine learning
Third-party service provider(s)
Other
57%
53%
45%
30%
30%
16%
0%
Differentiate Your Product With Unstructured Data Analytic Capabilities
Computing | research paper | sponsored by OpenText 13
ConclusionIthasbecomeacommonclichéincommerceandtechnologytoassertthatdataisthenewoil.Thisisalimitedanalogybecause,unlikeoil,dataisincreasinginaggregatevolumeeverysecondofeveryday.Datascarcityisnotanissuethatbusinesscustomersarestrugglingwith.Whattheyarebattlingwithishowtorealisevaluefromthemountainofdataavailabletothem–particularlyunstructured data. Sixty percent of participants in our research told us that most of their data was unstructured.
Whatwererespondentshopingtogainfromunstructureddataanalysis?Reducedcostsviagreaterefficienciesandautomationandcustomerinsightwerethetopmotivators.Interestingly,themostpopularusecaseswereallverymuchfocusedoncustomers–trendtracking,sentimenttrackingandbusinesscommunicationsanalysis.Thedesiretounderstandthemotivationsandthoughtsofcustomersaboutvariousproducts,servicesorbrandsiswidespread,asisthedesiretopredict what customers will want.
Unstructureddataanalysissoundslikeanessentialcapabilityforbusinesseskeentodigitisetheirgoodsandservicepropositions.However,halfoftheorganisationswespoketowerenotdoingso.Almost70percentsaidthattheirorganisationsfounditdifficultorextremelydifficulttogatherandanalyseunstructureddataandfewerthanaquartersaidtheyhadbeensuccessfulindoingso. Themostcommonlyencounteredobstaclestogainingvaluefromunstructureddatawasthecostofimplementationandalackoftechnicalexpertise.Thesecostsandtechnicaldifficultiesarise,inpart,fromtheheadspinningcomplexityofhybridcloudinfrastructure.Thiscomplexitygivesrisetoseveralissuesbutrelevantherearethedifficultiesofdatavisibilityandintegration.Ahuge78percentofourrespondentsagreedthatitwasdifficulttosuccessfullyintegratedataanalyticstoolsintotheirtechnologystacks.
Analyticstechnologyisalsofallingshort.Thesefailuresareinpartcausedbyamismatchbetweenorganisationalstructuresandthetechnology,and,inpart,becausealotofthetechnologywasdesignedforabusinessenvironmentthatwasnotawashwithunstructureddata.
Perhapsthemosttellingfindingisthatonly30percentofrespondentswereusingAIormachinelearningtoanalysetheirunstructureddata.Businessesarelookingforhelpindefiningstrategiesandimplementingtherighttechnologytohelpthemextractinsight,andsubsequentvalue,fromtheir data mountains.
Softwarevendorsthatcanhelporganisationsovercomethesechallengesandtapintothebenefitsonoffer–leveragingtheclearhungerinthemarketforcapablesolutions–standtoflourishasunstructureddataanalysisbecomesubiquitous.
Often,thebestwaytodosoisbyembeddingalreadyavailablewhite-labelsolutions–bypassingthemassiveinvestment,timeandotherresourcesrequiredtodevelopAI-basedanalyticstools.Thisleavesvendorstofocusonwhattheydobest,andallowsthemtocreateproductsthatcanmakeunstructureddataanalysismoredemocratised.Whensimpletousebuthighlycapabletoolsareinthehandsofmoreendusers,informingtheirday-to-daydecisionmaking,unstructureddataanalysis reaches its potential.
83percentofthosewhosaiditisextremelyeasyfortheirorganisationtogatherandanalyseitsunstructureddata,revealedthattheanalysisofunstructureddatahadbeenextremelysuccessfulattheircompany.Softwarevendorsandotherpartnerscanhelpmakethisarealityformorebusinesses.
Differentiate Your Product With Unstructured Data Analytic Capabilities
14 Computing | research paper | sponsored by OpenText
About the sponsor, opentextOpenText,TheInformationCompany,enablesorganizationstogaininsightthroughmarketleadinginformationmanagementsolutions,on-premisesorinthecloud.
OpenTextprovidesorganisationswithkeycapabilitiesformanaginginformationatanystageoftheinformationlifecycle,andtheOpenTextOEMProgrammakesthissametechavailabletoothertechnologyvendorstobecustomized,extended,embeddedandwhite-labelledforuseintheirownproducts and services.
AkeyareaoffocusfortheOpenTextOEMProgramistheanalyticsspace.OpenText’sanalyse,predictandreportsolutionsprovideOEMpartnerswiththeabilitytoreadilyintegrateahostofanalyticcapabilities,includingunstructureddataanalytics.
FormoreontheOpenTextOEMProgram,pleasevisitourwebsiteorcontactustogetstartedtoday. You can also stay up to date on trends and opportunities that tech vendors care about by readingtheOpenTextOEMblog.
OEM Program: www.opentext.com/products-and-solutions
Contact: www.opentext.com/about/contact-us
Blog: blogs.opentext.com
Recommended