M. Agosti and C. Thanos (Eds.): ECDL 2002, LNCS 2458, pp. 218-229, 2002.© Springer-Verlag Berlin Heidelberg 2002

Renardus: Following the Fox from Project to Service

Lesly Huxley

Institute for Learning and Research Technology, University of Bristol,8-10 Berkeley Square, Bristol BS8 4JR, UK

lesly.Huxley@bristol.ac.uk

Abstract. The Renardus academic subject gateway service in Europe waslaunched in April 2002. The author first presented the challenges facing thispan-European collaborative project at ECDL 2000. This paper identifies theprogress made in information, technical and organisational developments anddeployment since Lisbon 2000, presents the results of evaluation activities andoutlines the challenges, setbacks and successes for Renardus transition�in June2002�from project to service.

1 Introduction: The Renardus Project

The Renardus academic subject gateway service in Europe[1] originated as a projectunder the European Union’s IST 5th framework programme[2]. Its name determinedthe fox logo and other branding that have helped to maintain a high project profile.The author outlined the challenges facing partners from seven European countries,and the immediate workplan for this collaborative pan-European project, at theEuropean Conference on Digital Libraries 2000 in Lisbon[3]. In April 2002,following several months of testing, pilot operation and evaluation, partners formallylaunched the Renardus broker service. As planned, it provides a single, multilingualuser interface for cross-searching and cross-browsing distributed metadata collectionsheld by twelve participating subject gateways across Europe, which currently serveRenardus with over 65,000 records. Based largely on existing technologies andstandards, the Renardus project has nevertheless demonstrated some forward-thinkingwork in the areas of classification cross-mapping and cross-browse navigation and,through evaluation, provided some useful insight into users’ perspectives on designand use of complex information systems.

As project funding draws to a close, this paper reviews the progress made bypartners against information, technical and organisational aims and highlights keyfindings from the evaluation programme. The new opportunities, challenges andpotential solutions for members of the Renardus Consortium during the transitionperiod from project to service operation are also indicated. Together theseexperiences provide a rich source of information that may be used to underpin futureresearch and development in key interoperability areas as well as to start to learn frompractice in developing similar projects into services.

Renardus: Following the Fox from Project to Service 219

1.1 Renardus Context and Aims

The Renardus project built on late 1990s trends towards greater collaboration,standardization and interoperability between existing quality-controlled subjectgateway services across Europe. Renardus’ rationale was very simple: none of thoseindividual services�whether at local, national or regional levels�could hope todiscover, evaluate, describe and organise the ever-growing numbers of Internetresources to support their target communities in higher education learning andresearch. A move was foreseen from individual and potentially isolationist projectapproaches towards collaborative service models. For Renardus, this was manifest indevelopment of a collaborative framework at European level that anticipated benefitsfor both service providers and end users alike. It recognised the importance ofensuring consideration of local and national cultural and linguistic requirements ininternational services and the need to strengthen, through collaboration, gateways’ability to maintain and sustain their services in future. Users would be able to benefitfrom a single interface to discover and connect to broader collections with greaterprovision for multi-lingual access and subject coverage than their local subjectgateways.

With the twin imperatives of potential benefits for both users and participatinggateway services, the Renardus project aimed to build the ‘Renardus broker service’.The service was to develop and/or deploy technical and information interoperabilitytools and standards to (1) Provide cross-search and cross-browse functionality in asingle Web interface; (2) Offer integrated views of metadata drawn from existingdistributed European subject gateways and collections of other Internet-accessibleresources; (3) Undertake experiments in metadata sharing and provision ofmultilingual access and (4) Develop a sustainable organisational and business modelfor an operational pan-European broker service.

The practical challenges of the collaborative framework became clear very early inthe project, when two key decisions were made that would have significant impact onsubsequent progress towards the aims listed above. As the author has describedelsewhere[4], partners agreed firstly that the original concept of collaboration onlybetween ‘national initiatives’ grounded in national libraries would not be viable in theevolving information landscape and secondly that a distributed architectural modelwas preferable to one of a centralised metadata repository. The first decision allowedfor participation of single cross-disciplinary gateways such as DutchESS[5] from theNational Library of the Netherlands; individual subject services based in universitiessuch as Germany’s Special Subject Guides[6] and national collaborative brokerservices from both university and library organisations, such as the Finnish VirtualLibrary[7], the UK’s Resource Discovery Network[8] and, most recently, the DanishSubject Portal gateways[9]. Contributions from research organisations such asNetLab[10] in Sweden and ILRT[11] and UKOLN[12] in the UK were as valued asthey were varied. The second�grounded in the need to allow gateways to maintainfull control over their content and intellectual property rights�provided impetus toresearch and develop new ways of sharing and integrating metadata from distributed,heterogeneous services and presenting them in a coherent way to users (challengesstill facing portal developers today).

220 L. Huxley

1.2 Evaluation: A Quantitative Overview

Project partners had to ensure that evaluation and feedback played a key part inservice development and enhancement to demonstrate both that it meets its plannedaims and is also appropriate for target audience’s needs. A discrete element of theproject’s workplan was dedicated to evaluation and testing, both from a user andservice provider perspective. Project partners at Jyväskylä University in Finland tookthe lead in developing and implementing the evaluation plan, although the majority ofother project partners were also involved. The evaluation plan covered three maindeliverables for the final months of the project: (1) An end user evaluationprogramme (2) A review of the project’s functional and technical specifications and(3) An evaluation of management, maintenance and availability procedures for theservice. The latter two reports and recommendations have yet to be finalised at thetime of writing, but their findings will inform the future running of the service.

The end user evaluation programme was undertaken in Autumn 2001 (September-November). Details of the programme methodology, and findings (includingstatistical tables and diagrams) have been published as a project deliverable report[13]and are briefly referred to at relevant points throughout this paper. The programmewas based on a survey of potential end users (researchers, librarians, informationscientists, academic staff) across Europe, delivered via the Web in five languages.Evaluators were invited, via announcements on the project Web site, the project’semail newsletter and relevant mailing lists across Europe, to apply for a username andpassword to access the beta version of the service, undertake some specific researchtasks and then complete the online survey. Project partners in five countries alsohosted 17 structured evaluation workshops where, in addition to completing thesurvey in the same way as individual evaluators, participants were invited to discusstheir experience of using Renardus in open forum.

A total of 296 questionnaires were completed, in 22 countries. Most responseswere obtained in Finland (n = 84), Germany (n = 65), France (n = 50), TheNetherlands (n = 31) and the UK (n = 27). Two thirds (68%) of the evaluatorsdescribed themselves as librarians or information specialists; 11 % as researchers,lecturers or other academic staff; 10% students; 3 % computing staff; and 8% others.Almost half, 45% (n = 133) completed the questionnaire in workshops. The lownumbers of academic and research users was disappointing and perhaps indicated afailing in the promotion of the evaluation programme. However, librarians andinformation specialists are also an important group of target users of the fullyoperational Renardus service, and their expert input was valuable. The majority ofevaluators described themselves as regular Internet users. Of the 296 people whoevaluated the Renardus beta system, about 80% said they would want to use Renardusagain.

All quantitative evaluation questions asked evaluators to rate their answers on afour-point-scale: ‘very clear’; ‘clear’ (or easy, or informative, depending on thequestion); ‘unclear’ and ‘very unclear’ (or difficult, or uninformative, againdepending on the question). The distributions of responses were very similar in mostof the evaluation questions, with a majority of responses at the positive end of thescale. For example, 10-40% of responses fell into the very clear/very easy/veryinformative category; 50-70% in the clear/easy/informative category. 10-20% of

Renardus: Following the Fox from Project to Service 221

responses were in the unclear/difficult/uninformative group and 1-10% in veryunclear/very difficult/very uninformative.

These distribution patterns were evident in questions relating to: the clarity of theRenardus service concept (presented in the About Renardus text); the correspondenceof search results with search terms and the informative value of search results; thevalue of online help; ease of use of the advanced search function; terminology andlanguage used throughout, visual clarity of the service and ease of navigation. Thedistribution was different for questions relating to the ease of use of the browsingfunctionality (very easy – 15%, easy – 44%, difficult – 35%, very difficult – 7%),which indicates that browsing was, for several respondents, a difficult feature to use.On the other hand, browsing was considered interesting and innovative and severalevaluators commented positively on the graphical features offered to supportbrowsing.

So to what extent has the Renardus project been able to meet the challenges set byits information, technical and organisational aims, and what did evaluators reallythink of the service? The remainder of this paper considers each of these challengesin turn with reference, where appropriate, to the quantitative and particularlyqualitative evaluation findings and project partners’ future plans.

2 The Project: Meeting the Information Challenges

Project work was scheduled to allow for parallel technical and informationdevelopments once an initial data gathering phase was complete. There were two keychallenges for achieving coherent integration and presentation of metadata fromdistributed and heterogeneous collections: (1) Development of�and ability to complywith�a common metadata set for participating gateways, and (2) Deployment of aconsistently-applied classification scheme specifically to meet the demands of cross-browsing. Subsidiary but no less challenging issues prevailed in notions of metadatasharing and multilingual information access.

2.1 Towards a Common Metadata Model

As other project partners have described in detail [14], a simple, eight-elementcommon metadata model was developed for the pilot service, based on seven DublinCore elements and one other. Mandatory elements were DC.Title, DC.Description,DC.Identifier, DC.Subject, whilst the remaining four elements DC.Creator,DC.Language, DC.Type and Country were strongly recommended. These contentelements were supplemented by two administrative elements providing informationabout participating gateways and the original metadata records that formed the basisfor the metadata served to Renardus: Subject Based Information Gateway (SBIG) IDand Full Record URL. The same model has been retained in the launched service.However, some participating gateways still struggle to provide content in all eightfields, leading to gaps in the information displayed to the user in anything other thanthe brief format of search or browse results. Provision of the full record URL and adynamically-linked icon for the originating gateway are offered to users in the brief

222 L. Huxley

display and go some way to filling the gaps when viewing the full Renardus record,allowing users to connect from Renardus’ results pages to the locally-held recordwhere other, richer data may be discovered.

The end user evaluation survey asked about the quality and adequacy ofinformation provided by the metadata presented in search and browse results: themajority of respondents reported that it was possible to assess the relevance of aresource from a reading of the metadata supplied, and that the metadata wereinformative or very informative, precluding the need to connect to the resource itselfto decide. Even in the beta system, with a minimum common metadata set where notall elements were supported by all gateways, respondents’ comments indicatedgeneral satisfaction with the results:

Remark 1: “The descriptions were particularly helpful as they give much moredetailed information than search engines such as Google.”

Remark 2: “I was quickly able to tell that some of my results … were irrelevant,thanks to the information about the record that was supplied”

There were, however, comments about the extent of information provided and thedifficulty, particularly for users outside the world of library and information science,in identifying the relevance of some of the metadata elements. Some reordering ofthe way in which the elements were presented (eg placing of those most likely to beunderstood by any user, such as title, description and URL) was suggested to enhanceusers’ understanding and assessment decisions.

2.2 Towards a Common Classification

Individual participating gateways have deployed and in some cases developed fromscratch classification schemes that suit their target audiences. Indeed, some servicesuse a number of subject-specific classification schemes within a single gateway wherethat service covers several subject areas. There are valid reasons for gateways toretain these, given that they can often provide richer and more meaningfulrepresentation and navigation of metadata records. However, in order to achieve oneof the key innovative elements of functionality planned for Renardus, project partnersneeded to identify a mechanism that would allow a common classification scheme tobe presented to end users for cross-browsing purposes, whilst minimising re-classification effort required of participating gateways.

The challenge of cross-browsing was met by a process of mapping participatinggateways’ classification schemes to a general scheme: the Dewey DecimalClassification (DDC)[15]. The DDC was selected for a number of reasons, includingits ready availability (through WebDewey), regular maintenance, widespread useworldwide and multilingual versions. OCLC supported Renardus’ use of the DDC inthe Renardus research context, and expressed interest in its development for similarservices.

Agreement on the classification scheme was an important but relatively small firststep towards meeting the demands of implementing the cross-browsing functionality.Participating gateways would need to contribute more or less effort in mapping theirown schemes to DDC, depending on the number, status and format of their existingrecords. To facilitate the process, Renardus partners adapted a mapping tooldeveloped initially by the Carmen[16] project. Called CarmenX, the Renardus cross-

Renardus: Following the Fox from Project to Service 223

mapping tool is Web-based, with a relational database ‘backend’�the open sourcesoftware mySQL�and a range of Web server applications and scripts. Staff fromparticipating gateways use two windows in the CarmenX interface to display andnavigate machine-readable versions of their local scheme and of the DDC, and a thirdwindow to enter mapping relationships and notes. Relationships can be specified atfive levels of detail (eg broader, narrower, fully equivalent, minor overlap, majoroverlap) and are used to create mapping links in the user interface and to generate aDDC mapping for each resource in the local gateways’ normalised Renardusdatabase. Project partners have described our research into and development of cross-mapping mechanisms (eg Neuroth, 2001)[17] so I will not repeat those findings here.The cross-mapping processes underpinned the cross-browsing functionality identifiedas a key innovative feature of the planned Renardus service – but how did end userevaluators react when using the resulting beta cross-browse pages to locate resources?

The cross-browse functionality received most comments and suggestions in theevaluations, positive and negative. Some found it difficult to use, largely because ofthe layout of results presented as the user clicks through the vertical subject browsingtree. On the other hand the innovative graphical browsing feature, although slow touse in the beta system, was seen to be interesting and useful:

Remark 3: “The "Graphical Navigation Overview" is brilliant! I've never seen thistype of presentation used in a www search engine before.”

Some users recognised that the graphical overview could have limitations whenviewing larger subject areas, but on the whole reported positively on this feature. Theoption to alternate between graphical and text-based views of the browsing structurewere also appreciated, although there was some criticism of the use of DDC and itslack of relevant captions and structures for some subjects.

2.3 Towards an Extended Collaborative Framework: Metadata Sharing

Although not part of the original project workplan, some experiments with metadatasharing were scheduled after the first year, to explore enhancements and extensions tothe collaborative framework beyond the project lifetime. If the Renardus service wereto become popular, and the current trends in the information landscape continue to‘squeeze’ the capacity and sustainability of individual gateways whilst movingtowards greater interoperability, then post-project there may be a case fordevelopment of metadata sharing agreements amongst some or all of the participatinggateways. The time available for metadata sharing experimentation was limited: thework therefore focused on sharing metadata focused on geographical subjectheadings. These were chosen as being of importance in a broad range of disciplinesand also because the geographical entity (lake, river, mountain, region, etc) is alwaysthe same, regardless of how it may be described in different metadata schemes.

As with the classification cross-mapping, the decision on what metadata to sharewas only a first step: the challenge was in the method deployed to achieve metadatasharing. Once again, Renardus partners were able to adapt an existing tool�firstdeveloped to support work in the German SSG-FI (Special Subject Guides) �for thepurpose. The full results of these experiments and the part they may play in anyfuture service organisation will not be known until the project ends in June 2002,when the subset of participating partners report.

224 L. Huxley

2.4 Towards a Multilingual Interface

Original project plans highlighted the potential for multilingual developments in asingle user interface designed to access metadata records from gateways acrossEurope. However, after an initial project review, plans for multilingual developmentswere scaled down to what was realistically achievable within the project timespan.The user interface is currently available in five languages: English (default), French,German, Dutch and Finnish. ‘Interface’ includes all the help text, the instructions andexplanations which appear on each page relating, for example, to search and browsefunctionality and the menu options.

A simple tagging mechanism has been developed to allow partners to translate textelements of the user interface, which is now available in five languages: The effortrequired to maintain these interfaces, including help and descriptive texts, graphicalicons and dynamically created user feedback messages is not inconsiderable, andprobably not sustainable in the longer term without specific support. To avoid userconfusion, the option to select user interface language is available only from the homepage: switching between languages in other sections of the service is not currently onoffer.

At the time of writing, the DDC subject headings displayed in the browse pages areonly available in English, regardless of user interface language selected. This willchange shortly following agreement from OCLC that Renardus can have access toversions of the DDC in several different languages although not, unfortunately, eachof those offered as options for the general user interface. Decisions are now beingmade on how best to implement the multilingual versions of the browsing structurebalancing, on the one hand, added value for users with, on the other, the potential foruser confusion and frustration because of the range of languages supported by DDC.

The evaluation questionnaire was offered in the same number of languages as theuser interface. Respondents were asked to indicate languages they generally prefer touse on the Internet: the majority reported a preferred use of English (85.1%), withFrench, German and Finnish also being popular choices. Not surprisingly there was astrong correlation between preferred languages and the main languages of evaluationrespondents, the majority of whom originated from the countries of project partnerorganisations. In the majority of these countries, English is also widely spoken. Butthere was also quite a lot of evidence in the evaluation responses for support for amultilingual service, given that other languages such as Spanish and Italian were alsochosen, and these may be supported by the DDC.

Unfortunately, project partners’ available effort for translation is limited. Whilstthe tagging mechanism was used to develop the initial pilot interface, some parts ofthe broker service formally launched in April 2002 are currently available in Englishonly. This includes a new About Us section which provides background details andaccess to the project document archive.

Renardus: Following the Fox from Project to Service 225

3 The Project: Meeting the Technical Challenge

In parallel with developments in Renardus’ information infrastructure, projectpartners brought their technical expertise and experience to bear on meeting thechallenge of developing an appropriate and sustainable system architecture, usableinterface and supporting tools. An early review of broker models[18] showed that themajority of other similar aggregating services were using either HTML or Z39.50protocols. In solving the problem of creating a distributed system architecture, bothwere deployed by the Renardus development team, led by NetLab in Sweden andDTV[19] in Denmark.

3.1 An Appropriate System Architecture

Participating gateways must set up and maintain a Renardus server with content andadministrative data fields as described above. Z39.50 is the search and retrievalprotocol used to perform queries against normalised, interoperable sets of highlystructured data provided by each participating gateway in accordance with thecommon metadata model. A Renardus server comprises a content database andadministrative data, and may be implemented in a number of ways: (1) Single server(services make their content available directly through a Z39.50 interface conformingto the Renardus profile); (2) Server with protocol conversion* (a Renardus-compliantZ39.50 front-end is directly interfaced to a native database server); (3) Joint server*(involving several services using a common joint server) or (4) Server to a brokerwith protocol conversion* (combining 2 and 3 above). Those marked * were notimplemented in the pilot.

Given the collaborative aspects of the project organisation, a number of tools havebeen developed to support participating gateways in normalisation and quality controlprocesses. For example, normalisation is achieved through use of a set of programsdesigned to import original gateway records and their subject mappings and exportrecords from these in the required format (profile) for Renardus.

Participating services must also assure the quality of their normalised records andresponse times. The former requires some human effort in checking and simulatingthe user experience of following links from Renardus to the local gateway and takingremedial action to improve record quality, local browsing structures or their own userinterfaces. An automated system also simulates the user experience and analyses it ina quantitative way, by submitting a range of simple and complex queries to eachgateway’s server and displaying the number of records returned and response times.A simple system and network monitoring package (Spong) is also available so thatsystem administrators at local and Renardus level can check availability and qualityof Renardus servers.

A full technical specification for the Renardus pilot system was published early inthe project (available from the Renardus Project Archive[20], alongside details oftechnical requirements for participating gateways). An internal document detailinginstallation and maintenance procedures is available for current participatinggateways and those that may join in future.

226 L. Huxley

3.2 An Appropriate User Interface

User interface design and usability are key to the user experience of any electronicservice and especially so in presenting a coherent and consistent view on distributedand heterogeneous resources. The pilot service interface design adapted some of thedesign elements of the original project Web site (eg the logo and stylesheet of fontsizes and colours). It is worth noting that the design has not entirely successfullytaken account of the different design needs of an information and news site (theoriginal project Web site) and of an information discovery and retrieval service basedon distributed components.

Nevertheless, evaluation programme feedback from users on the beta Renardusinterface was in general positive and constructive. Some suggestions forimprovement were made, some of which will be implemented before the project end.These included minor criticisms of spelling and translation mistakes, particularly ofthe help text, and the need for more clarity in descriptive text and instructions.Conversely, some found the texts and layout easy to understand and very useful:

Remark 4: “Presentation very clear due to the fact that the little part "AboutRenardus" presents simply and clearly the process of search used within this gateway.We know where we go!”

The distributed architecture does place some specific requirements for clarity onthe service. Some evaluation respondents were confused about what the serviceoffered and how they could best search and browse. Terminology (and particularlywhat was seen as library jargon) was also a problem reported by many evaluators,even though the majority of respondents were library or information specialists. Somealso found the advanced search options confusing, again often as the result of thescreen layout and descriptive text rather than the concepts themselves. The end userevaluation report details a list of suggested interface improvements which will beprioritised by project partners. Top priority changes are likely to be implemented inthe current service before the end of the project lifetime, although some morecomplex enhancements may be deferred until further developments can be fullysupported by a new organisational and business model.

4 A Service: Meeting Organisation and Business Challenges

In development of an academic subject gateway service in Europe, Renardus partnershave found that the major challenges lie not in the information and technical aspectsof the service (although it should not be understood by that that these were trivial).One of the biggest challenges has been development of an appropriate organisationalstructure and business model to sustain the service beyond the project lifetime.Exploitation plans are high on the European Commission’s agenda in reviewingprojects such as Renardus, and although very positive on most aspects of the project,reviewers have been at pains to point out the need to develop an adequate exit strategyfor Renardus.

A successful workshop for potential participant gateways was held in Copenhagenin November 2001[21] to present requirements and procedures for participation. TheDanish subject portal gateways subsequently joined Renardus and their records arecurrently accessible via the Renardus service. Several other gateways expressed an

Renardus: Following the Fox from Project to Service 227

interest in future participation and an interest in the models and standards developedand deployed by project partners. However, most gateways not already participatingin the project understandably raised questions about the service’s sustainability andlikely future organisational and business models.

Work on the organisational infrastructure focused initially on a description of tasksto be carried out by some kind of central Renardus organisation and by participatinggateways in a not-for-profit Consortium arrangement. Associated costs were alsoestimated, and risks and alternative strategies explored. An interim ManagementGroup has been working to develop proposals for further funding with a number ofinternational organisations, as well as securing commitment from existing partners tocontinue to participate and serve their data to the service. There is a considerableissue that pan-European initiatives in the public sector face: the lack of any coherentorganisation or network at European level that can take strategic decisions on serviceprovision, collaboration and development funding for information services. There arenetworks such as Terena that support collaborative projects in areas of infrastructure,but their members are not in the business of directly supporting content provision.Each of the countries involved in the Renardus project has a distinct model of publicfunding for educational content projects and services, either through national libraries,research or education funding councils, education ministries or even individualeducational institutions. It has therefore been difficult for Renardus partners toidentify any one organisation that can be seen as representative of the interests of thevaried ‘players’ across Europe.

The fragility of this particular part of the information landscape�one of the maindrivers for the development of the Renardus project�has been demonstrated by thereview of support for one or two participating gateways by their current fundingorganisations. National and other initiatives and organisations are being impelled toreconsider the decisions made in the 1990s, when subject gateways appeared to be theway forward. National libraries, in particular, are reviewing the way that they disposeof their limited resources and prioritise the services they provide. It is therefore asimportant now as at the beginning of the project that the Renardus Consortium is ableto develop and sustain a robust organisational structure and business model.

Commitment from current partners is vital, and has been largely obtained, both fora ‘lightweight’ service model including server maintenance at SUB Gottingen and aCVS repository at NetLab; the time and effort of Management Group members inlicence, partnership, coordination and dissemination activities, and data provision.Interest in the collaborative tools and cross-classification developments in the projecthas been demonstrated by commercial and not-for-profit organisations alike. Theproject will therefore undergo transition from project to service in June 2002 with acontinuing collaborative model that is expected to evolve over the next twelve monthsfrom lightweight maintenance activity to further enhancement and developmentthrough a more robust and internationally-supported Consortium of educational andcommercial organisations.

The project concludes with a new Management Group in place to take furtherfunding negotiations forward. A Consortium Agreement is being drawn up forsignature by all those interested in participation in this next phase of Renardus’maintenance and development. A Data Provider Agreement has also been drafted, forsignature by those Consortium members that also provide data from their subjectgateways or other services to Renardus. In the first instance, and for the foreseeable

228 L. Huxley

future, no membership fee will be levied for participation in the Consortium.Participating data providers will be expected to continue to maintain their Z39.50servers, normalization and quality routines and any additional cross-mapping effortwith their own resources (staff and funding). Support for maintenance of the centralRenardus server and for assisting new gateways to join Renardus in future is beingprovided by one of the original project partners, SUB in Göttingen, Germany. Otherpartners have undertaken to provide effort and limited travel funding to supportmanagement and dissemination tasks for a further year. In the meantime, theManagement Group and other Consortium members will be actively pursuingsponsorship or collaborative models to support continuing maintenance anddevelopment of Renardus.

The Project Archive, with links to all public deliverables, articles and reports, isavailable from the About Us[22] section of the new service. Service news will alsobe posted there so that interested parties in the research and user communities can‘follow the fox’ as progression from project to service is achieved.

References

1 Renardus academic subject gateway service in Europe. Renardus Consortium.<URL http://www.renardus.org/>

2 Information Society Technologies (IST) 5th framework programme. EuropeanCommission. <URL http://www.cordis.lu/ist/home.html >

3 Huxley, Lesly (2000). Follow the Fox to Renardus: An Academic Subject GatewayService for Europe. In J Borbinha and T Baker (Eds), ECDL 2000, LNCS 1923, pp395-298, 2000. Springer-Verlag Berlin Heidelberg

4 Huxley, L, Carpenter, L and Peereboom, M (forthcoming). The Renardus broker service:Collaborative Frameworks and Tools. The Electronic Library.

5 DutchESS. Dutch Electronic Subject Service. The National Library of the Netherlands.<URL http://www.kb.nl/dutchess/>

6 University of Göttingen. SSG-FI special subject guides Web site.<URL http://www.SUB.Uni-Goettingen.de/ssgfi/>

7 The Finnish Virtual Library. Web site (English version).<URL http://www.jyu.fi/library/virtuaalikirjasto/engvirli.htm>

8 Resource Discovery Network. RDN Web site. <URL http://www.rdn.ac.uk/>9 Deff.dk. Danmarks Elektroniske Forskningsbibliotek. Deff Subject Portals Web site

(English version). <URL http://deff.dk/?lang=eng>10 Lund University. NetLab Web site. <URL: http://www.lub.lu.se/netlab/>11 Institute for Learning and Research Technology (ILRT). University of Bristol.

<URL http://www.ilrt.bristol.ac.uk/12 University of Bath. UKOLN Web site. <URL http://www.ukoln.ac.uk/>13 Renardus Consortium. Project Deliverable D5.2 User Evaluation report (March

2002<URL: http://www.renardus.org/about_us/deliverables/d5_2/D5_2_final.pdf>14 Neuroth, Heike and Koch, Traugott (2001). Cross-browsing and cross-searching in a

distributed network of subject gateways: architecture, data model and classification.Proceedings of the ELAG2001 Conference, June 2001.<URL: http://www.stk.cz/elag2001/Papers/HeikeNeuroth/HeikeNeuroth.html>

15 OCLC Forest Press. Dewey Decimal Classification home page.<URL: http://www.oclc.org/dewey/>

16 University of Regensburg Library. Carmen project Web site.<URL http://www.bibliothek.uni-regensburg.de/projects/carmen12/>

Renardus: Following the Fox from Project to Service 229

17 Neuroth, Heike (2001). Metadata mapping and Application Profile. Approaches toProvide for Cross-searching of Heterogeneous Resources in the EU Project Renardus.Proceedings of the DC-2001 International Conference on Dublin Core and MetadataApplications. <URL: http://www.nii.ac.jp/dc2001/>

18 Renardus Consortium. Project Deliverable D1.1 Evaluation Report of Existing BrokerModels in Related Projects (April 2000).<URL: http://www.renardus.org/about_us/deliverables/d1_1/D1_1_final.pdf>

19 Technical and Knowledge Center and Library of Denmark (DTV). Web page (Englishversion). <URL: http://www.dtv.dk/index_e.htm>

20 Renardus Consortium Project Archive.<URL: http://www.renardus.org/about_us/project_archive.html>

21 Renardus Consortium. Project Deliverable D9.6 Workshop (November 2001).<URL http://www.renardus.org/about_us/deliverables/workshop/index.html>

22 Renardus Consortium About Us pages. <URL http://www.renardus.org/about_us/>