Upload
haley-davies
View
225
Download
0
Tags:
Embed Size (px)
Citation preview
QA Focus – Supporting JISC's Digital Library Programmes1
Quality Assurance For Digital Library Programmes:The QA Focus Approach
Brian Kelly Hamish James
UKOLN AHDS
http://www.ukoln.ac.uk/qa-focus/http://www.ukoln.ac.uk/qa-focus/
This talk describes the work of the JISC-funded QA Focus project and outlines some options for the future
This talk describes the work of the JISC-funded QA Focus project and outlines some options for the future
QA Focus – Supporting JISC's Digital Library Programmes2
An Introduction to QA
Why Is QA Needed Now? In the past JISC had standards-based philosophy but:
• Approach of "let a 1,000 flowers bloom"• Diversification as no killer application then (it could
have been Gopher!) Things are different now:
• The Web is the killer application • There is now a need for interoperability (not just
self-contained independent Web sites) • JISC Programmes increasingly intended as service • development and not 'speculative' research • There is a need for publicly-funded services to be
seen to provide value for money
Intr
od
uct
ion
QA Focus – Supporting JISC's Digital Library Programmes3
QA Focus
QA Focus:• Funded under the JISC 5/99 programme (CI as contact)
• Covers technical interoperability (digitisation, Web, metadata, software development & service deployment)
• Proposal for developmental approach agreed (QA Focus won't be responsible for checking project deliverables)
• Agreement to develop QA methodology which can be applied to other JISC programmes (QA Focus was funded after other 5/99 projects had started)
• Understanding that this is a new area of work (for both JISC & ourselves) with a steep learning curve
• Initially provided by ILRT and UKOLN (1.0 FTE)
• Provided by UKOLN and AHDS since 1 Jan 2003
QA Focus – Supporting JISC's Digital Library Programmes4
Approach TakenTwo possible approaches to ensuring compliance with standards and best practices: Enforce
• Inspect all project’s work • Strict auditing, with penalties for no-compliance
Encourage • Developmental:
Working with community Explaining reasons for compliance Documenting examples of best practices Providing advice on implementation & monitoring Developing a QA methodology Seeking to embed QA in working practices
This may have been initial assumption in ITT
The latter approach is felt to be preferable, especially in a HE context
The latter approach is felt to be preferable, especially in a HE context
QA Focus – Supporting JISC's Digital Library Programmes5
QA Focus Activities
QA Focus activities: • Focus groups to identify key concerns (in areas of
standards, implementation and service deployment) • Surveys to profile approaches, best practices and
common problems • Focussed advice primarily addressing:
Standards and best practices Implementation issues Compliance checking
• Case studies describing examples of best practices • Development of a QA methodology • Validation of approach • Embedding QA methodology within projects
QA Focus – Supporting JISC's Digital Library Programmes6
Survey FindingsSurveys of compliance of 5/99 project entry points helped to profile community and identify common problems and best practices
QA Focus – Supporting JISC's Digital Library Programmes7
Documentation: Advice
Advisory briefing documents are being producedThese are:
• Brief, focussed documents
• Informed by findings of the surveys
• Over 40 briefing documents have been published to date
Advisory briefing documents are being producedThese are:
• Brief, focussed documents
• Informed by findings of the surveys
• Over 40 briefing documents have been published to date
QA Focus – Supporting JISC's Digital Library Programmes8
Documentation: Case Studies
Case Studies are being commissionedThese are:
• Written by projects themselves (helping
• to achieve 'buy-in') • Describe the
solution adopted to a particular problem
• Include details of lessons learnt – not just a press release!
• Over 30 published
Case Studies are being commissionedThese are:
• Written by projects themselves (helping
• to achieve 'buy-in') • Describe the
solution adopted to a particular problem
• Include details of lessons learnt – not just a press release!
• Over 30 published
QA Focus – Supporting JISC's Digital Library Programmes9
Surveys For FAIR & X4L ProgrammesQA Focus now supports JISC's
FAIR and X4L programmes
We have developed a self-assessment interface for projectsThe aim is that projects will (a) check and, if necessary, fix their home page; (b) check/fix other key pages and (c) address underlying causes of problems
"I have found the webpage very useful for testing the technical compliance and accessibility of our website" "Many thanks for this - what a really useful tool!"
"I have found the webpage very useful for testing the technical compliance and accessibility of our website" "Many thanks for this - what a really useful tool!"
http://www.ukoln.ac.uk/qa-focus/surveys/fair/web-self-assessment/
QA
Fo
r W
eb
QA Focus – Supporting JISC's Digital Library Programmes10
Putting it Together
We have: • Explained why projects should implement standards • and best practices (they don't always know) • Provide case studies on approaches by projects • Described implementation architectures • Described various checking tools
But how do we ensure projects : • Check deliverables for themselves • Have systematic checking procedures
There is a need to describe: • Tools & architectures for projects for check
compliance with standards & best practices • Embed systematic QA procedures
To
ols
QA Focus – Supporting JISC's Digital Library Programmes11
Example - Link Checking ToolsWe should all be familiar with link checking tools. But:
• Do such tools check all types of links What about links
to external style sheet files, JavaScript files, etc.?
Are broken links really broken?
• Are such tools integrated with normal working practices?
To
ols
QA Focus – Supporting JISC's Digital Library Programmes12
Describing Tools and Approaches
As an example of approach which provides easier access to testing tools, see the ,tools approach deployed on UKOLN Web site. This approach:
• Applies to all resources on Web site
• Covers HTML and CSS validation, link checking and various other tests
• Some recursive apps (,rvalidate)
• Easily implemented with single line redirect
Approach is documented at <http://www.ukoln.ac.uk/qa-focus/documents/papers/iadis-2003/>
QA Focus – Supporting JISC's Digital Library Programmes13
Promoting Use Of Compliance- Checking Tools
As an example of approach which provides easier access to testing tools, see the ,tools approach deployed on UKOLN Web site.
This approach:• Applies to all resources on
Web site
• Covers HTML and CSS validation and various other tests
• Some recursive apps (,rvalidate)
• Easily implemented with single line redirect
As an example of approach which provides easier access to testing tools, see the ,tools approach deployed on UKOLN Web site.
This approach:• Applies to all resources on
Web site
• Covers HTML and CSS validation and various other tests
• Some recursive apps (,rvalidate)
• Easily implemented with single line redirect
QA Focus – Supporting JISC's Digital Library Programmes14
Self Assessment Toolkit
Further Deliverables
We are developing a self-assessment toolkit for projects to use, by individual projects or across project clusters
The toolkit consists of: • Self-assessment questionnaires • Examples of QA procedures • Documented examples of use of testing tools • Advice on standards and best practices • Case studies • FAQs • …
QA Focus – Supporting JISC's Digital Library Programmes15
Self Assessment Toolkit (1)
QA Focus – Supporting JISC's Digital Library Programmes16
Self Assessment Toolkit (2)
The toolkit is also available in paper format, suitable for use in workshops, meetings, etc.
This approach will be used in the forthcoming QA workshop
Toolkits on Metadata and Service Deployment will be used
QA Focus – Supporting JISC's Digital Library Programmes17
Next Steps – MetadataQA for metadata is at early stages (little documented work found). Thoughts on QA for metadata: Syntax Check syntax on embedded DC Cataloguing Rules Ensure projects have appropriate cataloguing rules Interoperability Ensure metadata can interoperate with third parties Fitness for purpose Ensure metadata is appropriate for
its purpose Change control Ensure architecture for managing
metadata can cope with change control (cf. Exploit Case Study)
See for example <http://www.ukoln.ac.uk/qa-focus/documents/briefings/#metadata>
See for example <http://www.ukoln.ac.uk/qa-focus/documents/briefings/#metadata>
QA Focus – Supporting JISC's Digital Library Programmes18
Next Steps – Software
QA for software development is mature, but there may
be a need to engage in culture change within the sector
Thoughts on QA for software:
Specification Process for developing systems spec, agreeing spec with stakeholders, etc.
Methodology Documentation of systems development methodology (cf UML case study)
Testing Documentation of testing methodology
Deployment Awareness of deployment challenges
See for example <http://www.ukoln.ac.uk/qa-focus/documents/briefings/#software>
See for example <http://www.ukoln.ac.uk/qa-focus/documents/briefings/#software>
QA Focus – Supporting JISC's Digital Library Programmes19
Next Steps – Service Deployment
QA for service deployment is important to protect
investment in project funding (but there is a lack of
awareness in some cases). Development Architecture Early public documentation of
development architecture
Service Architecture Awareness of likely service architecture and liaison with service
Security Awareness of security concerns
Licensing Licensing issues
ScenariosSoftware developed using cool open software environment – but JISC Service has no knowledge of environment. Resources digitised, but copyright clearance not documented. Service refuses to take delivery due to liability concerns.
ScenariosSoftware developed using cool open software environment – but JISC Service has no knowledge of environment. Resources digitised, but copyright clearance not documented. Service refuses to take delivery due to liability concerns.
QA Focus – Supporting JISC's Digital Library Programmes20
The QA Focus Methodology
The QA Focus methodology which we are advising
projects (and JISC) to deploy: • Based on well-established QA principles • Seeks to be lightweight and achievable • Seeks to provide benefits to the projects themselves, as
well as (downstream) interoperability benefits to JISC programmes
Consists of: • Documented policies • Procedures for monitoring compliance with policies • Audit trails • Sharing and collaboration
QA Focus – Supporting JISC's Digital Library Programmes21
Applying QA – Standards
Policy: Open standards used where possible. Proprietary formats with published spec and cross-platforms viewers and authoring tools may be acceptable
Monitoring: Project manager responsible Documentation: See QA Focus briefings 11, 31, … Exceptions: If open standards not available,
immature or costly to implement proprietary standards may be used. The project manager should document such decisions: why proprietary format was used, migration strategies to open formats in future, architecture for migration and indication of costs
QA Focus – Supporting JISC's Digital Library Programmes22
Applying QA – Web Standards
Policy: The Web site will use XHTML 1.0 and CSS 2.0 standards
Architecture: The Web site will be based on XHTML templates and SSI Monitoring: New and updated pages validated using ,validate and ,cssvalidate. Every month ,rvalidate will be used
Exceptions: HTML derived automatically (e.g. Save As HTML in PowerPoint) need not comply with standards. The files will be stored in a standard directory to enable such files to be excluded from checks.
QA Focus – Supporting JISC's Digital Library Programmes23
Applying QA – Accessibility (1)
Policy: The Web site will strive to attain WAI A guidelines. Consistent accessibility shortcuts will be used. An accessibility policy will be published.
Architecture: The Web site will be based on XHTML templates which comply with WAI A.
Monitoring: New and updated pages will be validated using ,bobby. A monthly batch checker will be used and audit reports published (to enable any trends to be spotted).
Exceptions: A list of permitted exceptions will be provided.
Note that University of Edinburgh Library have a useful Web accessibility policy
QA Focus – Supporting JISC's Digital Library Programmes24
Applying QA – Accessibility (2)
Policy: The organisation has no accessibility policy Authors are free to implement their own accessibility shortcuts (if at all)
Architecture: No centralised policy covering authoring tools or architecture will be provided
Monitoring: No monitoring will be carried out
Note that SENDA legislation is helping to encourage organisations to develop appropriate policies and procedures
Note that SENDA legislation is helping to encourage organisations to develop appropriate policies and procedures
QA Focus – Supporting JISC's Digital Library Programmes25
A Matrix Approach To Selection
Strict Compliance With Standards & Best Practices
• Projects must implement musts in standards and guidelines document
• Strict policing & penalties for non-compliance • cf. NOF-digitise programme
Deployment Of A Standards Culture • Standards document describes
recommended/preferred standards • Projects should complete selection matrix and seek
approval on choices • Selection will reflect issues such as (a) the standard
(maturity, complexity, etc.) (b) implementation issues (availability of tools) (c) organisational culture and (d) programme requirements
QA Focus – Supporting JISC's Digital Library Programmes26
Recommendations
QA Focus recommendations to JISC will include: • Project proposals should describe approaches to
selection of standards, technical architecture & QA • Once approved initial project report should document:
Standards chosen (e.g. see "ESDS Web Standards Policy" case study and "Matrix for Selection of Standards" briefing document)
Justification for any deviations from open standards (e.g. see NOF-TAS FAQ)
Technical architecture Intended service deployment environment
This process is not intended to be bureaucratic: case studies show that some projects are already engaged in such activities
QA Focus – Supporting JISC's Digital Library Programmes27
Validating Our Approach
Feedback on our approaches gained by publishing work in peer-reviewed papers / conferences:
• Developing A Quality Culture For Digital Library Programmes1, EUNIS 2003: gave an overview of QA Focus work (paper selected for publication in Informatica)
• Ideology Or Pragmatism? Open Standards And Cultural Heritage Web Sites1,2, ichim03: described application of QA Focus methodology for selection of standards
• Deployment Of Quality Assurance Procedures For Digital Library Programmes1,3, IADIS 2003: described deployment of QA Focus approaches in CDLR, Univ. of Strathclyde
• A Proposal For Consistent URIs For Checking Compliance With Web Standards1,3, IADIS 2003
1 Written by UKOLN and/or AHDS staff 2 Co-authored with TechDis staff 3 Co-authored with CDLR staff
QA Focus – Supporting JISC's Digital Library Programmes28
Thoughts On Timeliness of QA
Early adopters
Hype
Despondency
Realism & service deployment
QA implemented
Maturity Software
Digitisation
Web
Metadata
Web Services
Semantic Web
QA Focus – Supporting JISC's Digital Library Programmes29
What Of The Future? (1)
After The QA Focus Funding Ceases QA Focus Deliverables
• The QA Focus deliverables (documents, papers, tools, etc.) will continue to be hosted at a stable URL for an extended period (3-10 years)
QA Focus Recommendations • Recommendations to JISC may be implemented
in future JISC programmes and technical advice, etc. carried out by Programme Managers
• Recommendations may be ignored and(a) alternative approach deployed (b) QA issues not formally addressed
QA Focus – Supporting JISC's Digital Library Programmes30
What Of The Future? (2)
Option 2 Funding Continued
QA Focus Deliverables • The QA Focus deliverables (documents, papers, tools, etc.)
will be maintained, new areas addressed, existing areas addressed in more depth, etc.
• Remit extends beyond 5/99, FAIR & X4L QA Focus
Recommendations • Recommendations to JISC implemented in future JISC
programmes • Technical advice, involvement with programmes and
projects, etc. carried out by QA Focus team working closely with Programme Managers
QA Focus – Supporting JISC's Digital Library Programmes31
What Of The Future? (3)
Option 3 Alternative Approaches
Hardline QA • JISC wishes to implement a hardline approach to
QA • Remit of MU extended to include technical aspects
of project deliverables • No QA • JISC decides that implementing a formal QA
infrastructure is inappropriate • Projects left to 'do their own thing'
But what are the implications of these contrasting approaches?
But what are the implications of these contrasting approaches?
QA Focus – Supporting JISC's Digital Library Programmes32
The Big Picture
The QA Focus approach could potentially be applied to a variety of audiences:
• QA for current and new IE programmes • QA for all JISC programmes • QA for JISC • QA for JISC RSCs • QA for JISC Services • QA for UK HEIs and FEs • QA for CIE (and links with other sectors)
Clearly this raises many issues!
QA Focus – Supporting JISC's Digital Library Programmes33
Conclusions
We have described: • The approaches to QA which have been taken by
the QA Focus project • The resources which have been developed for • the community by QA Focus • A systematic methodology for QA • Options for future work in this area