Upload
meara
View
20
Download
0
Embed Size (px)
DESCRIPTION
Quality Modeling Digital Libraries Foundations Workshop at JCDL 2007 Vancouver – June 23. Edward A. Fox Virginia Tech, USA [email protected] http://fox.cs.vt.edu. Acknowledgements: Students. - PowerPoint PPT Presentation
Citation preview
Quality Modeling
Digital Libraries FoundationsWorkshop at JCDL 2007
Vancouver – June 23
Edward A. Fox
Virginia Tech, USA
[email protected] http://fox.cs.vt.edu
Acknowledgements: Students
• Ghaleb Abdulla, Robert France, Marcos Gonçalves, Doug Gorton, Rohit Kelapure, Neill Kipp, Aaron Krowne, Yi Ma, Paul Mather, Uma Murthy, Ananth Raghavan, Unni. Ravindranathan, Ryan Richardson, Rao Shen, Ricardo da Silva Torres, Srinivas Vemuri, Seungwon Yang, Qinwei Zhu, …
Acknowledgements: Faculty, Staff
• Lillian Cassel, Lois Delcambre, Debra Dudley, Joanne Eustis, Weiguo Fan, James Flanagan, Sandy Grant, Eric Hallerman, Douglas Knight, Alberto Laender, David Maier, Manuel Perez-Quinones, Layne Watson, …
Other Collaborators (Selected)
• Brazil: FUA, UFMG, UNICAMP
• Case Western Reserve University
• DELOS and its many members
• Emory
• Germany: Univ. Oldenburg
• Portland State University
Acknowledgements: Support
ACM, AOL, CAPES, CNI, DFG, IBM, IMLS, Microsoft, NDLTD, NLM, NSF (IIS-9986089, 0080748, 0086227, 0307867, 0325579, 0535057, 0535060; ITR-0325579; DUE-0121679, 0121741, 0136690, 0333531, 0333601, 0435059, 0532825, …), SUN, …
Quality
• Big picture
• Quality of union DLs
• 5SQual
• Prior Work on Measuring DL Success
• IT/ IS success/adoption
• DL success model
• DL success, info, seeking/utilization
Quality Big Picture
• Purpose, Goals, Objectives– Advancing scholarship/civilization, Communication,
Sharing/use, Open access/preservation, …• Perspectives, Observations
– User/patron/personal, organizational, community– System Performance
• Metrics, Measurements• Models• Analysis, Reporting• Continuous Improvement (logging -> enhancement)
• See also next talk for literature review
Asynchronous, Digital Library Mediated (Scholarly) Communication
Different time and/or place
Digital LibrariesShorten the Chain from
Editor
Publisher
A&I
Consolidator
Library
Reviewer
To a flatter, service-oriented, more general framework
Author
Reader
Digital
LibraryEditor
Reviewer
Teacher
Learner
Librarian: designing, acquiring, organizing, collaborating, evaluating
Machine
Context in which the DL is embedded
Quality dimensions and measurement for minimal union DLs
Union DL Concept within the 5S Framework
Quality dimension Measurement
Union Catalog
Consistency# of sets of metadata specifications per digital object
Completeness
# of digital objects without a set of metadata specifications; size of the described collection
Union Service(mapping, searching, browsing)
Mapping-completeness
# of local schema elements which have been mapped to global schema
Coverage
# of results retrieved from a union DL; # of total results retrieved from the DLs to be integrated
NavigationGain
# of navigation paths available in a union DL; # of navigation paths available in the DLs to be integrated
5S definitional structure
Digital Object
RepositoryRepositoryCollectionCollection Minimal DL
Metadata Catalog
Descriptive Metadata
Specification
Structural Metadata
Specification
StreamsStreams StructuresStructures SpacesSpaces ScenariosScenarios SocietiesSocieties
indexingindexing
browsingbrowsing searchingsearching
servicesservices
hypertexthypertext
Structured Stream
Digital Objects
Metadata
Services
• Completeness
• Conformance
• Accessibility
• Similarity
• Significance
• Timeliness
• Efficiency
• Reliability
Numeric
Indicators
5SQual - Dimensions
5SQual Archi-texture
Evaluations – Charts
has an exampleIS success model
Prior Work on Measuring DL Success
DeLone et al.
Seddon
has an exampleInformation seekingbehavior model
Ellis
Kuhlthau
has an exampleDL quality model Gonçalves
has an exampleUsability of DLsinspection of NCSTRL
evaluation of ACM, IEEE-CS,NCSTRL, and NDLTD
evaluation of ADL
evaluation of ADEPT
has an indicator
system usage
has an exampleTechnology
acceptance model Venkateshpredict Intention to re/use
Behavioral Attitude & Intension to Use — Venkatesh Model of IT Adoption
intention to use system
performance expectancy
system usage
Unified Theory of Acceptance and Use of Technology
effort expectancy
social influence
facilitating conditions
• Performance expectancy: – perceived usefulness, extrinsic motivation, job-fit, relative advantage,
and outcome expectations
• Effort expectancy– the degree of ease associated with the use of system
• Social influence– Subjective norms, social factors, and image
• Facilitating conditions– the degree to which an individual believes that an organizational and
technical infrastructure exist to support the system
Venkatesh Model of IT Adoption
DeLone and McLean Model of IS Success
System Quality
Information Quality
UserSatisfaction
IndividualImpact
OrganizationImpact
Use
Seddon Model of IS Success
System Quality
Information Quality
UserSatisfactions
Net Benefits to:
IndividualsOrganizations
Society
PerceivedUsefulness
DL Success Model
5S and minimal DLSynthesize
IS success and adoption models (see above) Information life cycle model (Borgman et al.) 5S-based DL quality model (Gonçalves et al.) Information-seeking behavior models (Ellis’ and
Kuhlthau’s)
From end user perspective
Information Life Cycle
AuthoringModifying
OrganizingIndexing
StoringRetrieving
DistributingNetworking
Retention/ Mining
AccessingFiltering
UsingCreating
crea
tion
distributionsee
kingutilization
search
ing,
browsing,
recommending
storing, archiving,
networkingau
thor
ing,
mod
ifyin
g,
desc
ribin
g org
anizi
ng, i
ndex
ing
pres
erva
bilit
y,
sim
ilari
ty,
tim
elin
ess,
accuracy, completeness,
conformance accessibility, preservability
pertinence,significance,
timelinessre
levan
ce
Activ
e Semi-active
Inactive
accessibility,accessing,
filteringGonçalves et al.
system quality (SQ)
performance expectancy (PE)
society, scenario,
space
service
accessibility
reliability
ease of use
joy of use
composabilityefficiency
effectiveness
extensibility
reusability
reliability
accessibility accuracy
completeness consistence conformance
pertinence preservability
relevance significance
similarity
timeliness
DL quality
dimension
information quality (IQ)
stream, structure
digital object
metadata
collection
catalog
repository
adequacy
relevance
reliability
scope
timeliness
understandability
social influence (SI)
DL success construct
society
5S and
DL concept
DL visibility
DL success
manifest variable
system quality (SQ)
performance expectancy (PE)
society, scenario,
space
service
accessibility
reliability
ease of use
joy of use
composabilityefficiency
effectiveness
extensibility
reusability
reliability
accessibility accuracy
completeness consistence conformance
pertinence preservability
relevance significance
similarity
timeliness
DL quality
dimension
information quality (IQ)
stream, structure
digital object
metadata
collection
catalog
repository
adequacy
relevance
reliability
scope
timeliness
understandability
social influence (SI)
DL success construct
society
5S and
DL concept
DL visibility
DL success
manifest variable
crea
tion
distributionsee
kingutilization
E1:starting
E2: chaining
E3: browsin
g
E4: diffe
rentia
ting
E5: monito
ring
E6: extra
cting
storing, archiving,
networking
K1:
in
itia
tion
K2:
se
lectio
n
K3:
explorationK4:
formulation
K5: collection
K6:
presentation
auth
orin
g, m
odify
ing,
desc
ribin
g org
anizi
ng, i
ndex
ing
pres
erva
bilit
y,
sim
ilari
ty,
tim
elin
ess,
accuracy, completeness,
conformance accessibility, preservability
DL Success Constructs
Activ
e Semi-active
Inactive
E: Ellis’ modelK: Kuhlthau’s model
satisfaction
performanceexpectancy
(PE)
behavioralIntention to
(re)use
relevance adequacy timeliness
reliability understandability scope
accessibilityease of use
joy of use reliability
userinterface
social influence (SI)
information quality(IQ)
system quality(SQ)
DL Success Model
information quality(IQ)
DL Concepts Regarding Information
collection metadata catalog
digital object metadata specification
repository
consist of
associated with
associated with
consist of consist of
accessibilitypreservability
pertinencerelevancesimilarity
significancetimeliness
digital object
accuracycompletenessconformance
metadata specification
impact factorcompletenesscollection
consistencycompletenesscatalog
consistencycompletenessrepository
understandabilityrelevancetimelinessreliabilityadequacy
digital objectmetadata specification
adequacy
catalogcollection
scope
repository
Information Quality (IQ)
Digital Library IQ
screen designnavigation
user interface
composabilityreusability
extensibilityefficiency
effectivenessreliability
serviceaccessibility
reliabilityease of usejoy of use
System Quality(SQ)
performanceexpectancy
Digital Library SQ
DL success
Construct
seeking phase utilization phase
starting (E1/K1)
selection exploration
(E2-E6)/(K2-K3)
formulation (K4)
collection (K5)
presentation (K6)
social influence DL visibility
information quality adequacy, scope accuracy
system quality ease of use
joy of use (interface)
accessibility accessibility accessibility
performance
expectancy
usefulness
(interface)
DL Success Constructs Associated with Seeking and Utilization Phases
DL Success Constructs Associated with Seeking Phase
E1: Starting’ activity in Ellis’ model (K1: ‘initiation’ stage in Kuhlthau’s model) Social Influence (SI) — DL visibility
Publicize existence of a DLProvide a DL alert service
DL Success Constructs Associated with Seeking Phase
E2-E6: ‘chaining’, ‘browsing’, ‘differentiating’, ‘monitoring’, and ‘extracting’ in Ellis’ model (K2-K3: ‘selection’ and ‘exploration’ stages in Kuhlthau’s model) Information Quality (IQ) System Quality (SQ) Performance Expectancy (PE)
DL Success Constructs Associated with Utilization Phase
K4-K6: ‘formulation’, ‘collection’, and ‘presentation’ stage in Kuhlthau’s model Information Quality
information accuracy information accessibility
Summary: Quality
• Big picture
• Quality of union DLs
• 5SQual
• Prior Work on Measuring DL Success
• IT/ IS success/adoption
• DL success model
• DL success, info, seeking/utilization
Summary: Quality Big Picture
• Purpose, Goals, Objectives– Advancing scholarship/civilization, Communication,
Sharing/use, Open access/preservation, …• Perspectives, Observations
– User/patron/personal, organizational, community– System Performance
• Metrics, Measurements• Models• Analysis, Reporting• Continuous Improvement (logging -> enhancement)
• See also next talk for literature review
Discussion