A centre of expertise in digital information management
www.ukoln.ac.uk
Making Effective Use Of Benchmarking Tools
Brian Kelly
UKOLN
University of Bath
Bath
Email: [email protected]
UKOLN is supported by:
http://www.ukoln.ac.uk/web-focus/events/workshops/swmlac-2005-10/http://www.ukoln.ac.uk/web-focus/events/workshops/swmlac-2005-10/
This presentation describes the advantages and disadvantages of benchmarking tools and ways of maximising the effectiveness of such tools.
This presentation describes the advantages and disadvantages of benchmarking tools and ways of maximising the effectiveness of such tools.
A centre of expertise in digital information management
www.ukoln.ac.uk
2
Background
Problems You've identified some problem areas for users of Web sites:
• Functionality – it doesn't work• Usability – it's difficult for me to use• Accessibility – it's difficult for people with disabilities
to use
Solutions
Let's review the approaches you've taken to spot such problems and explore how to use the tools effectively:
• Tools that can help• Processes that can help • A Quality Assurance (QA) framework
A centre of expertise in digital information management
www.ukoln.ac.uk
3
Useful tools:• W3C's HTML validator:
can spot functionality & accessibility problems
Tools: Functionality (1)
HTML, CSS, … Validation:• Web page doesn't look right in my browser• First thing: validate page!
To
ols
• W3C's CSS validator• RSS validator (if you have
an RSS newsfeed)• …
A centre of expertise in digital information management
www.ukoln.ac.uk
4
Tools: Functionality (2)
Link Checking:• Clear need to ensure links work• Many tools available
Validated part of my Web areaFindings:
• 12,514 Web pages!• Only checked internal links• Large no. of errors – but
vast majority false errors• Some errors found in areas
provided by others• Others my fault – and
mostly fixed
Issues:• We can't always rely on tools• Why weren't errors spotted
previously?• What to do with large no. of errors?
Issues:• We can't always rely on tools• Why weren't errors spotted
previously?• What to do with large no. of errors?
A centre of expertise in digital information management
www.ukoln.ac.uk
5
http://webxact.watchfire.com/http://webxact.watchfire.com/
Tools: Accessibility
Many accessibility testing tools are available
To
ols
WebXact (formally known as Bobby) is probably best known
http://www.wave.webaim.org/http://www.wave.webaim.org/
The WAVE and Cynthia Says are other accessibility checkers
NOTES• Automated tools can't detect
all (many?) accessibility problems
• Findings from tools can be inconsistent
• Underlying WAI guidelines are open to interpretation
NOTES• Automated tools can't detect
all (many?) accessibility problems
• Findings from tools can be inconsistent
• Underlying WAI guidelines are open to interpretation
A centre of expertise in digital information management
www.ukoln.ac.uk
6
Tools: Usability Of The Tools (1)
There can be usability barriers to regular use of such testing tools:
• They require going to Web page, copying and pasting URL, etc
• Sometimes only single pages can be tested
Simple solution:
• On UKOLN Web site can append ,tools to any URL to run various tools on page
• Simple to implement – see QA Focus briefing no. 59
http://www.ukoln.ac.uk/qa-focus/documents/briefings/,rvalidate
http://www.ukoln.ac.uk/qa-focus/documents/briefings/,rvalidate
Tools:,validate ,rvalidate,checklink ,rchecklink,cssvalidate …
Tools:,validate ,rvalidate,checklink ,rchecklink,cssvalidate …
A centre of expertise in digital information management
www.ukoln.ac.uk
7
Tools: Usability Of The Tools (2)
'Bookmarklets' and Firefox extensions can make use of tools much easier and provide additional features
Web Developer allows: • Features disabled• Additional information
to be provided• Tools to be used
Checky allows: • Validation • Link checking• Access to misc tools
These tools are very useful and their use by all is strongly encouraged
These tools are very useful and their use by all is strongly encouraged
A centre of expertise in digital information management
www.ukoln.ac.uk
8
Quality Assurance
The tools aren't sufficient by themselves. Also need:• Documented policies: so we know what we're
expected to check for• Systematic procedures: for checking that we are
implementing our policies• Corrections made to workflow processes, and
not just fixing individual problemsIn addition it can be useful to have:
• Audit trails: to spot trends and identify possible problems in workflow processes (e.g. new tools deployed, new staff involved, …)
• Sharing experiences, so that we and others can learn
A centre of expertise in digital information management
www.ukoln.ac.uk
9
QA Examples (1)
Example of QA policies & procedures for file formats
Policy: The Web site will use XHTML 1.0 and CSS 2.0 standards
Architecture:The Web site will be based on XHTML templates and use of SSIs
Monitoring:New and updated pages validated using ,validate and ,cssvalidate. Every month ,rvalidate will be used & record kept
Exceptions:HTML derived automatically (e.g. Save As HTML in PowerPoint) need not comply with standards. The files will be stored in a standard directory to enable such files to be excluded from checks.
Policy for QA Focus Web site
A centre of expertise in digital information management
www.ukoln.ac.uk
10
QA Examples (2)
Example of QA policies & procedures for links
Policy for QA Focus Web sitePolicy: QA Focus will seek to ensure that links are functional. Monitoring:
New and updated pages checked using ,checklink and ,rchecklink. Every month ,rchecklink will be used & record kept and quarterly Xenu will be used.
Exceptions: Links in "publications" (e.g. papers which are formally published) which become broken may not be fixed.
If there are large numbers of broken links which would be time-consuming to fix we may not fix them.
We make no commitment to fix broken links once the QA Focus funding finishes.
A centre of expertise in digital information management
www.ukoln.ac.uk
11
Conclusions
To conclude:• Tools can help in identifying problems
areas• However tools may be flawed and
inconsistent• In order to make tools easier to use they
can be accessed: (a) using a URL interface or (b) within the browser
• Systematic use as part of a QA framework is desirable
Note, however, that is essential that you do not rely purely on automated tools – you will need to engage with your users
Note, however, that is essential that you do not rely purely on automated tools – you will need to engage with your users