Everything you always wanted to know about search in typo3

http://www.dkd.de

Freitag, 15. Juni 12

d dkdevelopmentkommunikationdesign


WelcomeTYPO3 ConferenceQuebec Canada

Olivier Dobberkau, Founder and CIO dkd Member of the Expert Advisory Board TYPO3 Assoc.Twitter @[email protected]


mailto:[email protected]

mailto:[email protected]

Everything You Always Wanted to Know About Search in TYPO3. But Were Afraid to Ask


Woody Allen

Inspiration for this Talk:

Woody Allen Movie: „Everything You Always Wanted to Know About Sex * But Were Afraid to Ask“


Woody Allen

Inspiration for this Talk:

Woody Allen Movie: „Everything You Always Wanted to Know About Sex * But Were Afraid to Ask“

Internet Movie Database:http://www.imdb.com/title/tt0068555/


http://www.imdb.com/title/tt0068555/

http://www.imdb.com/title/tt0068555/

Agenda

A short history of Search

Slang

The need to Search

Who is searching and what is (s)he searching for?

Search in TYPO3 with Apache Solr

Questions & Answers


History

A short trip in the History of Searchsolutions in times of IT.

Really short, really lots of missing facts and not scienti!c at all.


Scratch your own itch, IBM.

At the beginning was the Mainframe

IBM develops in 1969 STAIRS (storage and information retrieval system)

Fulltext Search for Terminal Applications

Performance: „far below anyone‘s expectations“

First use in the DOJ Case againts IBM

Source: A history of online information services, 1963-1976 von Charles P. Bourne,Trudi Bellardo Hahn


Internet years are dog years

The Internet changes the needs in Fulltextsearch.

With Lycos, Alltheweb, Infoseek, Excite and Altavista Searchpages compete in solving the „How do i !nd something in the Internet?“

Its a race for the love of the seeking internet users in 1995.

Yahoo tries to be the Directory of Websites


And then came GOOGLE

Who does not know about Googles Secret?

The Anatomy of a Large-Scale Hypertextual Web Search Engine

http://infolab.stanford.edu/~backrub/google.html

Visionary Paper

The named technologies and principles are industry standard and are still changing our IT Industry. (Map reduce, Big data & Pagerank)

A must read!




Slang


Its all about words!

Irformation Retrieval (IR)

Term versus Query

Index

Recall & Precision

Relevancy

Index, Inverted Index & Posting List

Recency & Authority


The need to Search

What leads us when we search?How do we search? How does what we !nd change us?


People are like Bears (only less fur)

How do we search?

Marcia Bates, 1989

THE DESIGN OF BROWSING AND BERRYPICKING TECHNIQUES FOR THE ONLINE SEARCH INTERFACE

http://pages.gseis.ucla.edu/faculty/bates/berrypicking.html

Every search can be described with this






Marcia J. Bates Berrypicking techniques for the online search interface (1989)


Carrots & Sticks

Search Behavior Patterns, John Ferrara

http://www.boxesandarrows.com/view/search-behavior

Domain Expertise

Search Expertise

Cognitive Style

Goal Type

Mode of seeking

Situational idiosyncrasies






Neo: The Matrix

Matrix of Scope/Style of information needs

Scope & Type -Tyler Tate. Sohn et al. Church & Smythhttp://twigkit.com/blog/2011/12/06/mobile-information-needs.html


http://twigkit.com/blog/2011/12/06/mobile-information-needs.html

http://twigkit.com/blog/2011/12/06/mobile-information-needs.html

Search = Success for your Website

Bene!ts for your Visitors & Users

They will !nd it on your Website

Serendipity

Better and faster knowledge transfer

Business bene!ts

ROI

Agility

Awareness and Enablement


TYPO3 & Search

Shameless Plug: Apache Solr for TYPO3

I still have some „I love Indexed Search“ Buttons to giveaway.


Solr-Components

Indexing

Query

Analysis

Results

Additional Components


Indexing


What can be indexed?

TYPO3 Content

TYPO3 Databases (TCA Tables)

External Websites

RSS-Feeds

Files

...


Indexing Features

Synonyms

Stopwords

Protected words

External Content

RSS

Microsites

Application Data

...


Query


Query Options

Operators

“+” and “-” to add or exclude terms

Soon “and” und “or” to combine terms

Quotes to tie words togetherie. “This is a Search with many Terms”

Diacritical Characters

cuvée = cuvee

Søren = Sören = Soeren = Sœren = Soren


Query

Takes care of Access Control Rights

Autocomplete

Did you mean?


Results


Results

Searchresults linking to a result

Page Browser

Sorting

Relavancy (Score)

Author

Date (cr_date of TYPO3 Page)

your own criterias


Results

View-Helper to display additional Information like Custom Prices & Preview images.

Preset Filters so that Facets are activated with a Query


Results

Field Boosting (Terms in certain Field score higher. Can be freely set)

Boost-Functions (Functions on values of documents. I.e. newer documents are more ranked higher)

Query-Manipulation (Can be changed before they hit Solr)

Elevation (Paid content)


Results

Template Engine: "exible Template to customize your results listing fast and easily

Search word highlighting

Spell-Checking: "Did you mean?"

Common Searches

Recent Searches


Facets

Type-Facets

Author

Type of Document (Pages, News, Files & many more)

Range-Facets (Work in Progress)(ie. 1-10 $ or Slider)

Hierarchical Facets (Great if you have lots of categorized Data like in News or Filerepositories)

Facets can be combined with each other(ie. Show me all red & blue shoes)


Facets

Geo-Search (work in progress)(i.e If you want to search and display the location of your data of a certain type: Stores, Servicepoints, Bus-stops )

Geo-IP based on IP of your visitor(ie: Where is the next salespoint for your products)

Facets are TYPO3 content objects(can be manipulated with typoscript i.e Gifbuilder)

Filters can be preset(You can preset certain facets)

...


Analysis


Analysis

Query Logging

Stats on Queries (Work in Progress)

Userbased Ranking (Work in Progress)

Integration with analytics tools posible

Roll your own

There might be a Solr Server feature coming up ...





More like this Component on the Details page can show related additional documents

Its possible to access Indexed Data

Nutch Crawler to Index non TYPO3 Websites

Data Import Handler


d dkdevelopmentkommunikationdesign

Thank You! Merci.


Quellenangaben

Lucene Scoring for dummies: http://www.supermind.org/blog/378/lucene-scoring-for-dummies

Fotos: Søren Schaffstein


Technology

Everything you always wanted to know about search in typo3