Text Analytics: The Industry At A Glance

Where We Are, Where We’re Going, and Your Text Mining Investment

Seth Grimes@sethgrimes#TAS11

Where we are

Ken Jennings, IBM Watson, and Brad Rutter play Jeopardy!https://secure.wikimedia.org/wikipedia/en/wiki/File:Watson_Jeopardy.jpg

Miles to go

http://www.businessweek.com/magazine/content/04_19/b3882029_mz072.htm

Milestones [and goal(s)?]

Language+ understanding.• Text, speech, images, and video.• Narrative, discourse, and argument.

Information extraction.

Knowledge structuring and integration.

Inference; synthesis.

Language generation.

Conversation; interaction; autonomy.

≈> Convergence, a.k.a. Singularity

Singularity?

Before we reach that point…

Text+ technologies today

Text analytics, by generating semantics, bridges search and BI to turn Information Retrieval into Information Access for online, social & enterprise content.

Search BI

Text Analytic

sSemantic search

Information access Integrated

analytics

Information management

Applications today

Broadly grouped --• Intelligence and counter-terrorism.• Life sciences.

• Content management, publishing & search.• Customer & market intelligence.• E-discovery.• Enterprise feedback.• Law enforcement.• Risk, fraud, compliance, and investigation.

Resegmenting the market

Information Acquisition

NLP (natural language processing) (including aaS)

Information management & semantics Databases, repositories, content management systems • Information integration • Semantic Web

Search-based/oriented applications E-discovery and compliance • Semantic search • Media & publishing • Advertising

Enterprise applications Customer experience/relationship management and marketing including social • Market research and competitive Intelligence • BI and research • Online commerce • Life sciences • Intelligence

Market size

I estimate a global, 2010 text-analytics market of –• $15 million ≈ Information acquisition (TA part)

E.g., 80legs, Informatica, ISYS Search, Kapow Software, Oracle.• $455 million ≈ NLP, semantics & text analytics

Installed & as a service, including vendor professional services.• $35 million ≈ Information management applications of TA

Companies such as EMC, IBM, MarkLogic, Open Text, and Oracle.• $30 million ≈ Enterprise applications of text analytics

Typically OEM TA licensees, e.g., Radian6, SatMetrix, Vovici.• $300 million ≈ Search-based applications (TA part)

Companies such as Autonomy, Cataphora, Dow Jones/Factiva, Elsevier, Endeca, FirstRain, Google, IBM, Lixto, Thomson Reuters.

= $835 million.

Last year’s estimate

I estimated a $425 million global TA market in 2009.• Up about 25% from $350 million in 2008, up in turn 40%

from $250 million in 2007.• Covers software licenses, vendor provided support and

professional services.

$(hundreds) million more value created by:• Universities and research centers, especially in the life

sciences.• Government, particularly for intelligence & counter-

terrorism.• OEM licensees, for listening platforms, e-discovery, etc.• Systems integrators and consultants.

Text technology initiatives

Now and near future.• Semantic search. • Sentiment analysis.• Listening platforms.• Question answering.• Text visualization.• Web 3.0 & the Semantic Web.

Text technology initiatives, revisited

But I used that list last year! Revising very slightly:• Semantic search. • Sentiment analysis.

Lots of market confusion, including from some folks at TAS11.• Listening platforms.• Question answering.• Text visualization.• Web 3.0 & the Semantic Web.

Ronen Feldman, Bar-Ilan University and Hebrew University: “Text analytics [is] driving the Semantic Web” (2006).Copious European government research funding, coupled with wishful thinking, is driving the Semantic Web.

Text technology initiatives++

Now and near future.• Beyond-polarity sentiment analysis.

Emotions, intent signals. etc.• Entity/identity resolution & profile extraction.

Online-social-enterprise data integration.• Semantic data integration, Complex Data. • Speech analytics.• Discourse analysis.

Because isolated messages are not conversations.

• Rich-media content analytics.• Augmented reality; new human-computer interfaces.

Where to?

And Your Investment?

Robust growth across applications.

Technical innovation.

New frontiers.

Consolidation and emergence.

Opportunity

You have two days to learn more!

Text Analytics: The Industry At A Glance

Where We Are, Where We’re Going, and Your Text Mining Investment

Seth Grimes@sethgrimes#TAS11