Centralization vs. Decentralization Issues in Internet-based KMS: Experiences from Expertise Recommender Systems

July 13, 2000 TWIST 2000 Yimam & Kobsa

Centralization vs. Decentralization Issues in Internet-based KMS: Experiences from

Expertise Recommender Systems

Dawit Yimam, GMD-FIT.MMK &

Alfred Kobsa, UCI, ICS


Outline

• Background• First centralized approach• Alternatives - to centralize or decentralize ?• DEMOIR• Summary


Expert Recommenders/Finders

• Systems to help users in tracing human information and/or expertise sources in organizations

• part of knowledge management and knowledge sharing services.

• Traditionally done by manual construction and search of expertise descriptions of people, e.g.,

+ Expert Databases (“knowledge directories”)+ Personal web pages on the Web

• Automatically mining implicit sources of expertise evidence from electronic resources of an organization and its people.

Background

Alternatives

First appr.

DEMOIR

Summary


Characterizing Expert Finders

1. Expertise evidence/indicator source recognition and gathering

2. Expertise modeling- Expertise indicator extraction- expertise model representation

3. Expertise model deployment- query mechanisms- matching operation- output delivery/presentations- adaptation and learning operations

Background

Alternatives

DEMOIR

Summary

First appr.


Query-time expertise modeling

Web Site Indexing

Web DocumentsIndex

Background

Alternatives

DEMOIR

Summary

GlimpseFIT Peoples’and otherWeb Pages WebGlimpse

First appr.



Query(Boolean)

Background

Alternatives

DEMOIR

Summary

Web Site Indexing

Web DocumentsIndex


ExpertQuery

Interface

First appr.



ExpertDatabase

(Name, URL)

SearchRanked List of

ExpertsBackground

Alternatives

DEMOIR

Query(Boolean)

Web Site Indexing

Web DocumentsIndex


ExpertQuery

Interface

Search Result(passages containing

Keywords)

ExpertiseModeler& Tracer

First appr.

Summary



• Shortcomings:

+ high latency in query processing+ personal sources hard to include+ non-document sources (e.g. recommendation from people,

social relations, etc.) hard to include+ full reliance on availability of some search engine+ limited exploitation of info due to lack of persistent expertise

models

Background

Alternatives

DEMOIR

First appr.

Summary


Building apps on text Indexes

• Existing Web indexing systems use centralized indexes of distributed resources/collections.

• Distributed Indexing needed to cope with ever growing information on Internet.

• But, currently centralized global indexes (though may be distributed in a tightly coupled manner) consistently outperform decentralized indexing and query approaches.

• This favors centralizing the applications to be built on them.

Background

Alternatives

DEMOIR

First appr.

Summary


Pre-generation of Expertise Models

• Alternative 1: Personal expert finding agents + Decentralized multi-agent system.+ Expertise modeling as well as searching done by self-managing

personal agents residing in experts’ computers (e.g. Vivacqua, 1999; Foner, 1997).

• Alternative 2: Aggregated expertise modeling+ Based on centralized expertise models (that are either

dynamically aggregated or linked to a pre-constructed ontology) (e.g. simple versions in Kautz & Selman, 1998; Krulwich & Burkey, 1996).

+ Can be distributed among tightly coupled cluster of machines.

Background

Alternatives

DEMOIR

First appr.

Summary


Personal expert finding agents

Agent communicatio

n

Agent1

-----------------ModelExpertise

FindExpert

PersonalExpertise

Model

Agent2


FindExpert

PersonalExpertise

Model

Agentn


FindExpert

PersonalExpertise

Model

Agent3


FindExpert

PersonalExpertise

Model

Background

Alternatives

DEMOIR

First appr.

Summary


Aggregated Expertise Modeling

ExpertFindingServer

AggregatedExpertise

ModelBackground

Alternatives

DEMOIR

First appr.

Summary



ExpertFindingServer

AggregatedExpertise

Model

Gateway(broker)

localExpertise

Model

Server1

localExpertise

Model

Server1

localExpertise

Model

Servern

LAN

Background

Alternatives

DEMOIR

First appr.

Summary



ExpertFindingServer

AggregatedExpertise

Model

Gateway(broker)

localExpertise

Model

Server1

localExpertise

Model

Server1

localExpertise

Model

Servern

Gateway(broker)

Server1

Server1

Servern

CentralExpertiseModel

LAN

LAN

Background

Alternatives

DEMOIR

First appr.

Summary


Analysis

Factor Agents AggregatedLocality (processing/load distribution) Facilitated limited

Mining personal resources easier difficult

Personal Privacy preservation At the hand of the expert At the hand of maintainer

No single point of failure Single point of failure(backup mechanisms needed)

ScalabilityRobustness (in face offailure)

Extendability Easy by adding new agents Depends on design

Central administration No Yes

Experts feeling in control Yes Mostly not

Background

Alternatives

DEMOIR

First appr.

Summary


Analysis (contd.)

Factor Agents AggregatedOrganization-wide access to expertiseinfo.

Limited (e.g. to expertnetwork, etc.)

facilitated

Multi-purpose/optimal utilization ofexpertise info (analysis, visualization,browsing, etc.)

Limited facilitated

Sources of expertise evidence mined Mostly limited to personalresources

Organizational resources(repositories, databases,Web/Internet, etc.)

Query Performance (scalability) Low (due to the need toconsult many agents)

High due to single location ofinformation

Knowledge-based/statisticaltechniques support

poor Good

Coordination overhead High (e.g. getting agentsfind and interact with oneanother)

low

Background

Alternatives

DEMOIR

First appr.

Summary


Hybrid Approach

• Combine distributed agents with centralized expertise model server - “local-central” approach

• How ?1. Decentralized + centralized Expertise modeling

• Lightweight personal agents for personal sources• Configurable gatherers for organizational resources

2. Centralized (but “distributable”) expertise information server3. Decentralized Exploitation of expertise information (through

clients)

Background

Alternatives

DEMOIR

First appr.

Summary


DEMOIR - A Hybrid Architecture

Organizational InformationResources

Expertise-indicator

Source Gatherers

Source TypeIdentifier

SourceWrapper2

SourceWrapper1

SourceWrappern

...

EISM

Ontology,Organizationalstructure, etc.

AggregatedExpertise

Model

ExpertModels

RemoteExpert Details

API ClientsFusers

Expertise Information Space

Background

Alternatives

DEMOIR

First appr.

Summary


DEMOIR - A Hybrid Architecture

Gathering(decentralize

dCentralized)

Modeling(decentralized/centralized)

Exploitation(decentralized

)

Background

Alternatives

DEMOIR

First appr.

Organizational InformationResources

Expertise-indicator

Source Gatherers

Source TypeIdentifier

SourceWrapper2

SourceWrapper1

SourceWrappern

...

EISM

Ontology,Organizationalstructure, etc.

AggregatedExpertise

Model

ExpertModels

RemoteExpert Details

API ClientsFusers

Expertise Information Space

Summary


Summary/Observation

• Centralized and decentralized options have their advantages and disadvantages.

• Many problem domains involve both “centralizable” and “decentralizable” tasks

Challenges:+ isolating such tasks and identifying the tradeoffs b/n

centralizing and decentralizing their operations+ If both approaches are used, how to get them work together

Background

Alternatives

DEMOIR

Summary

First appr.


Summary/Observation

• Centralization/decentralization is only one dimension of a system’s architecture. Relate to:

+ size/complexity of system (e.g. number of different parts, dynamism of their interaction, etc.)

+ heterogeneity of data and their sources+ accessibility (e.g. permissions/privacy constraints, manner of

use)+ communication patterns among components

keep these in mind and analyze how they affect centralization/decentralization decision.

Background

Alternatives

DEMOIR

Summary

First appr.


Summary/ObservationWhat we did (in retrospect):

1. Identify system requirements/tasks

2. Identify and analyze centralized and decentralized alternatives of performing identified tasks

thereby identify and evaluate general centralization and decentralization factors in the problem domain.

3. Specify optimum system components as well as architecture (i.e. trying to achieve advantages and avoid disadvantages of alternatives)

aim at flexibility to allow varying degrees of centralization and/or decentralization to suit different deployment environments.

Background

Alternatives

DEMOIR

Summary

First appr.