The User-Centered Design of a Non-Specialist Metadata Tool and Interface for the Internet Public Library

Authors

Michael Khoo

The iSchool at Drexel University

3141 Chestnut Street, Philadelphia, PA 19143, USA

Email: michael.khoo@ischool.drexel.edu

Jung-ran Park

The iSchool at Drexel University

3141 Chestnut Street, Philadelphia, PA 19143, USA

Email: jung-ran.park@ischool.drexel.edu

Xia Lin

The iSchool at Drexel University

3141 Chestnut Street, Philadelphia, PA 19143, USA

Email: xia.lin@ischool.drexel.edu

Good quality metadata are key in supporting user interactions with a repository. Poor quality

metadata hide resources, produce poor search results, and negatively affect user satisfaction

(Barton et al. 2004; Beall 2005; Geisler et al. 2002). Good quality metadata can be resource-

intensive to produce, and as a result there is a 'metadata generation bottleneck' between the

growing numbers of digital resources requiring description and the limited numbers of metadata

specialists who can create such descriptions (Liddy et al., 2001). This can restrict metadata

production and reduce the utility of repositories (Wilson, 2007). Approaches to addressing this

bottleneck include training more metadata specialists; automatic metadata generation (Liddy et

al., 2001; Ochoa & Duval n.d.); and also the solution discussed in this poster, the user-centered

design and development of a tool to support student volunteers and interns to engage in the

creation and quality control of Dublin Core metadata records for the Internet Public Library (IPL:

http://www.ipl.org/).

Non-Specialist Metadata Tools

Non-specialist metadata tools have had mixed success (Crystal & Greenberg, 2005). This mixed

success is partly related to the efficacy with which these tools convey complex metadata

concepts to non-specialists. Case studies have described how, despite good intentions, metadata

tools can be hard for users to understand, leading in turn to low levels of metadata production

(e.g. Greenberg et al. 2003, Kastens et al. 2005, Khoo 2005, Wilson 2007). Crystal and

Greenberg (2005) observe that "succinctly explaining the fields of a standardized schema such as

Dublin Core to metadata novices appears to be a key challenge for designers," and that

"developing a conceptual understanding of metadata records and their use in retrieval was found

to be challenging for users."

The Internet Public Library (IPL)

The IPL receives approximately 1 million visits a month from over 200 countries and territories

around the globe. It is currently crosswalking its metadata to Dublin Core. To support this work,

the IPL is developing a tool for the creation, editing, and quality control of Dublin Core metadata.

Wilson (2007) identifies three common metadata quality concerns that are also of concern to the

IPL: completeness (the number of fields utilized), accuracy (the structural correctness of the data

entered, e.g. for names, dates, etc.), and intelligibility (the semantic content of the metadata in

the field). For instance, with regard to completion, an initial analysis of the IPL shows that a few

fields (e.g. Title, URL, Abstract) are heavily used while other fields are lightly used (c.f. Moen et al.

2006, Ward 2003, Wilson 2007). With regard to the accuracy and intelligibility, while the original

IPL records were created by trained volunteers, there are still variations in format (George

Washington vs. Washington, George) or content (George Washnigton) (c.f. Barton et al. 2004;

Beall 2005). These issues will need to be addressed.

As advocated by Crystal and Greenberg (2005) the development of the IPL metadata tool is

following a user-centered 'interaction design' process (Norman 2002; Sharp et al. 2007) that

considers not just the functionality of the tool, but also users' needs. The work began with the

elicitation of initial design requirements from students in HCI classes at Drexel University. The

students reviewed IPL metadata records using existing prototype tools, critiqued these tools, and

then used a range of HCI techniques, including hierarchical task analyses, personas, and

scenarios, to develop basic functional paper prototypes. A common theme of these prototypes

was the integration of various task windows (e.g. for the resource, the catalog record, the catalog

rubric, the evaluation form, etc.) into one window. The prototypes were evaluated with heuristic

evaluations, cognitive walkthroughs, and think-alouds, and the evaluation data supported the

development of a functional prototype that will be subject to further user testing.

The accuracy and efficacy of the final tool will be assessed through a number of means, including

(a) a review of the accuracy of a random sample of completed cataloging tasks, (b) a comparison

of completed forms by different users to test for inter-user reliability, and (c) feedback from initial

users (i.e. student volunteers and interns).

Outcomes

The new IPL metadata tools will increase the quality of the library's metadata, positively impact

the search service, and directly benefit the IPL's millions of global users. The research findings

will be useful to other libraries involving users in their metadata work, and will have a direct

relevance to the digital library and HCI research communities. The activities are integrated into

courses taught at the Drexel iSchool, and so we will also report on the significant curriculum and

pedagogical outcomes and implications of this research.

Acknowledgements

This research is supported by a grant from the OCLC-ALISE Library and Information Science

Research Grant Program (OCLC LISRGP).

References

Barton, J., Currier, S., & Hey, J. (2003). Building Quality Assurance into Metadata Creation. Dublin

Core Conference 2003, 28th Sept - 2nd Oct, Seattle, Washington, USA.

Beall, J. (2004). Metadata and data quality problems in the digital library. Journal of Digital

Information 6(3).

Crystal, A., & Greenberg, J. (2005). Usability of a metadata creation application for resource

authors. Library & Information Science Research 27 (2005), 177-189.

Geisler, G., Giersch, S., McArthur, D., & McClelland, M. (2002). Creating virtual collections in

Digital Libraries: Benefits and implementation issues. JCDL '02, Portland OR, USA, pp. 210-218.

Greenberg, J., Crystal, A., Robertson, W., & Leadem, E. (2003) Iterative design of metadata

creation tools for resource authors. 2003 Dublin Core Conference, Seattle, WA, USA.

Kastens, K.A., DeFelice, B., Devaul, H., DiLeonardo, C., Ginger, K., Larsen, S., Mogk, D., &

Tahirkheli, A. (2005). Questions & challenges arising in building the collection of a digital library

for education: Lessons from five years of DLESE. D-Lib Magazine 11(11).

http://www.dlib.org/dlib/november05/kastens/11kastens.html.

Khoo, M. (2005). The Tacit Dimensions of User Behavior: The Case of the Digital Water Education

Library. JCDL 2005, Denver, CO, June 7-11, 2005, p. 213-222. ACM Press.

Liddy, E., Sutton, S., Paik, W., Allen, E., Harwell, S., Monsour, M., Turner, A., & Liddy, J. (2001).

Breaking the metadata generation bottleneck: preliminary findings. JCDL '01, p. 464. ACM Press.

Moen, W. E., Miksa, S. D., Eklund, A., Polyakov, S., & Snyder, G. (2006). Learning from artifacts:

metadata utilization analysis. JCDL '06, Chapel Hill, NC, USA, June 11 - 15, 2006. ACM, New York,

NY, 270-271.

Norman, D. (2002). The Design of Everyday Things. New York: Basic Books.

Ochoa, X., & Duval, E. (n.d.). Towards Automatic Evaluation of Metadata Quality in Digital

Repositories. Ms. http://ariadne.cti.espol.edu.ec/M4M/files/TowardsAutomaticQuality.pdf

Sharp, H., Rogers, Y., & Preece, J. (2007). Interaction design (2nd edition). New York, NY: Wiley.

Ward, J. (2003). A quantitative analysis of unqualified Dublin Core metadata element set usage

within data providers registered with the open archives initiative. JCDL 2003, May 27-31, 2003,

Houston, Texas pp. 315-317.

Wilson, A. (2007). Toward releasing the metadata bottleneck: a baseline evaluation of

contributor-supplied metadata. Library Resources and Technical Services, 51(1) (January 2007),

pp.16-28.