A centre of expertise in digital information management Making Effective Use Of Benchmarking Tools Brian Kelly UKOLN University of Bath

A centre of expertise in digital information management

www.ukoln.ac.uk

Making Effective Use Of Benchmarking Tools

Brian Kelly

UKOLN

University of Bath

Bath

Email: [email protected]

UKOLN is supported by:

http://www.ukoln.ac.uk/web-focus/events/workshops/swmlac-2005-10/http://www.ukoln.ac.uk/web-focus/events/workshops/swmlac-2005-10/

This presentation describes the advantages and disadvantages of benchmarking tools and ways of maximising the effectiveness of such tools.

This presentation describes the advantages and disadvantages of benchmarking tools and ways of maximising the effectiveness of such tools.


www.ukoln.ac.uk

2

Background

Problems You've identified some problem areas for users of Web sites:

• Functionality – it doesn't work• Usability – it's difficult for me to use• Accessibility – it's difficult for people with disabilities

to use

Solutions

Let's review the approaches you've taken to spot such problems and explore how to use the tools effectively:

• Tools that can help• Processes that can help • A Quality Assurance (QA) framework


www.ukoln.ac.uk

3

Useful tools:• W3C's HTML validator:

can spot functionality & accessibility problems

Tools: Functionality (1)

HTML, CSS, … Validation:• Web page doesn't look right in my browser• First thing: validate page!

To

ols

• W3C's CSS validator• RSS validator (if you have

an RSS newsfeed)• …


www.ukoln.ac.uk

4

Tools: Functionality (2)

Link Checking:• Clear need to ensure links work• Many tools available

Validated part of my Web areaFindings:

• 12,514 Web pages!• Only checked internal links• Large no. of errors – but

vast majority false errors• Some errors found in areas

provided by others• Others my fault – and

mostly fixed

Issues:• We can't always rely on tools• Why weren't errors spotted

previously?• What to do with large no. of errors?

Issues:• We can't always rely on tools• Why weren't errors spotted

previously?• What to do with large no. of errors?


www.ukoln.ac.uk

5

http://webxact.watchfire.com/http://webxact.watchfire.com/

Tools: Accessibility

Many accessibility testing tools are available

To

ols

WebXact (formally known as Bobby) is probably best known

http://www.wave.webaim.org/http://www.wave.webaim.org/

The WAVE and Cynthia Says are other accessibility checkers

NOTES• Automated tools can't detect

all (many?) accessibility problems

• Findings from tools can be inconsistent

• Underlying WAI guidelines are open to interpretation

NOTES• Automated tools can't detect

all (many?) accessibility problems

• Findings from tools can be inconsistent

• Underlying WAI guidelines are open to interpretation


www.ukoln.ac.uk

6

Tools: Usability Of The Tools (1)

There can be usability barriers to regular use of such testing tools:

• They require going to Web page, copying and pasting URL, etc

• Sometimes only single pages can be tested

Simple solution:

• On UKOLN Web site can append ,tools to any URL to run various tools on page

• Simple to implement – see QA Focus briefing no. 59

http://www.ukoln.ac.uk/qa-focus/documents/briefings/,rvalidate

http://www.ukoln.ac.uk/qa-focus/documents/briefings/,rvalidate

Tools:,validate ,rvalidate,checklink ,rchecklink,cssvalidate …

Tools:,validate ,rvalidate,checklink ,rchecklink,cssvalidate …


www.ukoln.ac.uk

7

Tools: Usability Of The Tools (2)

'Bookmarklets' and Firefox extensions can make use of tools much easier and provide additional features

Web Developer allows: • Features disabled• Additional information

to be provided• Tools to be used

Checky allows: • Validation • Link checking• Access to misc tools

These tools are very useful and their use by all is strongly encouraged

These tools are very useful and their use by all is strongly encouraged


www.ukoln.ac.uk

8

Quality Assurance

The tools aren't sufficient by themselves. Also need:• Documented policies: so we know what we're

expected to check for• Systematic procedures: for checking that we are

implementing our policies• Corrections made to workflow processes, and

not just fixing individual problemsIn addition it can be useful to have:

• Audit trails: to spot trends and identify possible problems in workflow processes (e.g. new tools deployed, new staff involved, …)

• Sharing experiences, so that we and others can learn


www.ukoln.ac.uk

9

QA Examples (1)

Example of QA policies & procedures for file formats

Policy: The Web site will use XHTML 1.0 and CSS 2.0 standards

Architecture:The Web site will be based on XHTML templates and use of SSIs

Monitoring:New and updated pages validated using ,validate and ,cssvalidate. Every month ,rvalidate will be used & record kept

Exceptions:HTML derived automatically (e.g. Save As HTML in PowerPoint) need not comply with standards. The files will be stored in a standard directory to enable such files to be excluded from checks.

Policy for QA Focus Web site


www.ukoln.ac.uk

10

QA Examples (2)

Example of QA policies & procedures for links

Policy for QA Focus Web sitePolicy: QA Focus will seek to ensure that links are functional. Monitoring:

New and updated pages checked using ,checklink and ,rchecklink. Every month ,rchecklink will be used & record kept and quarterly Xenu will be used.

Exceptions: Links in "publications" (e.g. papers which are formally published) which become broken may not be fixed.

If there are large numbers of broken links which would be time-consuming to fix we may not fix them.

We make no commitment to fix broken links once the QA Focus funding finishes.


www.ukoln.ac.uk

11

Conclusions

To conclude:• Tools can help in identifying problems

areas• However tools may be flawed and

inconsistent• In order to make tools easier to use they

can be accessed: (a) using a URL interface or (b) within the browser

• Systematic use as part of a QA framework is desirable

Note, however, that is essential that you do not rely purely on automated tools – you will need to engage with your users

Note, however, that is essential that you do not rely purely on automated tools – you will need to engage with your users


www.ukoln.ac.uk

12

Questions

Any questions or comments?

Documents

A centre of expertise in digital information management Making Effective Use Of Benchmarking Tools Brian Kelly UKOLN University of Bath