Testing for Web Accessibility

John Slatin Access UMay, 2009

Jim Thatcherhttp://jimthatcher.com jim@jimthatcher.com

Accessibility ConsultingAustin, Texas

May 11-12, 2009

Testing for accessibility 2

Resources

These slides at http://jimthatcher.com/testing PowerPoint testing.ppt Resources in order of appearance there too

This class is very much a resource list!

May 11-12, 2009

Testing for accessibility 3

Questions

Please ask questions or … Make comments at any time … If it isn’t clear to you – ASK!

May 11-12, 2009

Testing for accessibility 4

Outline Testing for Web Accessibility – Means? Standards and Guidelines Testability

Contrast example Automatic Testing tools

Free tools Commercial Tools Testing tool report

Tools that assist with Human Review Lets test a page

May 11-12, 2009

Testing for accessibility 5

Testing for Web AccessibilityWeb Accessibility:

Web resources are accessible if people with disabilities can use them as effectively as non-disabled people

(UT Accessibility Institute)

So testing means user testing with disabled and non-disabled people?

Instead there are “standards and guidelines” against which we can test

May 11-12, 2009

Testing for accessibility 6

Standards and Guidelines WCAG 1.0 Web Content Accessibility Guidelines 1.0

http://w3.org/TR/WCAG10

Section 508 Standards http://www.access-board.gov/508.htm

WCAG 2.0 http://w3.org/TR/WCAG20/

Section 508 refresh See Section 6

http://www.access-board.gov/sec508/refresh/report/

IBM web guidelines http://ibm.com/able/guidelines/web/accessweb.html

May 11-12, 2009

Testing for accessibility 7

Priority Section 508

16 provisions all priority one!

WCAG 1.0 14 guidelines 65 Checkpoints – 16 Priority 1, 30 Priority 2, 19 Priority 3

checkpoints

WCAG 2.0 12 guidelines 56 Success Criteria – 18 Level A, 17 Level AA, 21 Level AAA

success criteria

508 Refresh – who knows – Public comment 1Q 2010 – at the earliest

May 11-12, 2009

Testing for accessibility 8

WCAG 2 – testability is key

All WCAG 2.0 success criteria are written to be testable. While some can be tested by computer programs, others require human testers for part or all of the test.

My favorite testability example … Contrast

May 11-12, 2009

Testing for accessibility 9

Testability - Example

WCAG 1.0 Priority 2: 2.2 Ensure that foreground and background color combinations provide sufficient contrast when viewed by someone having color deficits or when viewed on a black and white screen.

“Sufficient contrast”?

“Color deficits”?

May 11-12, 2009

Testing for accessibility 10

Flickr – black and white

May 11-12, 2009

Testing for accessibility 11

Testability - Contrast

WCAG 2.0 Level AA: 1.4.3 Text and images of text have a contrast ratio of at least 4.5:1 … 3:1 for larger text (18pt or 14pt bold) (Level AA)

Contrast Ratio? Tools will answer that question –

http://www.paciellogroup.com/resources/contrast-analyser.html (also pull down from toolbar)

Try that with http://Flickr.com Another to check out: http://Graybit.com

May 11-12, 2009

Testing for accessibility 12

Testable? Automatic tests? Testable by a computer vs. requiring human

judgment Only ~25% of accessibility errors can be

detected by computers (Mike Cooper) Many claim compliant/accessible because no

errors reported by testing tool Testing all for Section 508 Standards

discussed http://jimthatcher.com/testing1.htm

May 11-12, 2009

Testing for accessibility 13

A current example of testing …Recovery.gov complies with all of the automatic

checkpoints of the Section 508 Accessibility Guidelines, and has been manually verified for nearly all of the manual checkpoints. This compliance has been tested using Watchfire WebXACT program. (To be in America)

Accessibility of Recovery.gov http://jimthatcher.com/recovery.htm

May 11-12, 2009

Testing for accessibility 14

Automatic tests (about 12 tests)

(3) Missing alt on <img>, <area>, or <input> with type=“image”

(3) Empty alt on <img> in <a> without text, same for <button> or <input> with type=“image”

(1) Form control with no title or label (or empty) (2) No title on frame or iframe (1) Sever-side image map (image link with ismap) (1) Missing lang attribute for page (1) [No HTML headings (h1, h2, …) on page]

May 11-12, 2009

Testing for accessibility 15

Testing tools (outline) Automatic web accessibility testing Human Review – tools that assist User testing

Beyond the scope of this class But check out Shawn Lawton Henry’s

Just Ask: Integrating Accessibility Throughout Design The whole book is there

One of 11 co-authors of Web Accessibility: Web Standards and Regulatory Compliance

May 11-12, 2009

Testing for accessibility 16

Automatic Testing (outline) Automatic web accessibility testing

Free 1-page tools More general free tools Desktop tools Enterprise tools

Tools lists: http://www.w3.org/WAI/ER/tools/complete http://www.colostate.edu/Dept/ATRC/tools.htm http://www.webaim.org/articles/tools/

May 11-12, 2009

Testing for accessibility 17

Some Free 1-page tools on the web CynthiaSays (HiSoftware)

http://cynthiasays.com Bobby is gone! Truwex (Erigami)

http://checkwebsite.erigami.com/accessibility.html New kid on the block 2 years ago

Worldspace by Deque – New Kid this year http://worldspace.deque.com For Google Toolbar, http://tinyurl.com/9h7elc

Sample file http://jimthatcher.com/testing/testfile.htm

May 11-12, 2009

Testing for accessibility 18

Sample File for Testing http://jimthatcher.com/testing/testfile.htm

Issue Cynthia Truex Worldspace

1 Server-side map Yes Yes Yes2 Two input fields – label but same id No No No3 Empty alt on image link No Yes Yes4 No alt on image (duh) Yes Yes Yes5 title attributes on input fields (good) No Yes Yes6 No alt on image button Yes No Yes7 No alt in image map area Yes Yes Yes8 Iframe with no title No Yes Yes

May 11-12, 2009

Testing for accessibility 19

The Truex Map – very cool

May 11-12, 2009

Testing for accessibility 20

More General Free Tools The WAVE (Utah State) (version 4.0)

http://wave.webaim.org

AChecker http://www.atutor.ca/achecker/

(WCAG 2.0)

Functional Accessibility Evaluator (U of Ill) http://appserv.rehab.uiuc.edu/fae

May 11-12, 2009

Testing for accessibility 21

Desktop Commercial Tools Ramp ($69-$1499)

http://www.deque.com InFocus ($1795)

http://www.ssbtechnologies.com

May 11-12, 2009

Testing for accessibility 22

Enterprise Tools (expensive) WebKing by Parasoft

http://parasoft.com/jsp/products/home.jsp?product=WebKing

Rational Policy Tester by IBM (was WebXM by Watchfire) http://tinyurl.com/9hao4g

Worldspace by Deque http://deque.com/products/

May 11-12, 2009

Testing for accessibility 23

Six Commercial Tools,A Comparison Chapter 13 –

Web Accessibility: Web Standards and Regulatory Compliance … See http://jimthatcher.com/testing.htm.

Forty “issues” http://jimthatcher.com/step/new

Based on 70 files http://jimthatcher.com/step/new/stepfiles.htm

Example – how the tools deal with link text

May 11-12, 2009

Testing for accessibility 24

Link text - WCAG 1.0 WCAG 1.0 13.1: Clearly identify the target of each

link. (P2) (Not 508 requirement)

Interpreted to mean good link text or title attribute on anchor element.

It is testable – evaluators would usually agree on “good link text.”

May 11-12, 2009

Testing for accessibility 25

Link text – automatic tests Raise errors (or warnings) for Common phrases,

“more,” “click here,” “more …,” “details” etc.

List of such is editable with several tools.

Inadequate phrase like “click here” but with title attribute – should pass

Same link text different href – should fail without title.

Extreme case - Empty link text or spaces for link text is error without title.

May 11-12, 2009

Testing for accessibility 26

Link text – WCAG 2.0

2.4.4 The purpose of each link can be determined from the link text and its programmatically determined link context. (A)

2.4.9 The purpose of each link can be identified from the link text. (AAA)

May 11-12, 2009

Testing for accessibility 27

Testing tools and link textTest OK Bobby In-

Focus LIFT Ramp W-King W-XM

21. “Click here” 4 yes no no yes yes yes22. Image link with

empty alt-text4 yes no yes yes no yes

23. Image link with spaces for alt-text

5 yes no yes yes yes yes

24. Link with text and image with empty alt-text

3 no no yes yes no yes

25. “Click here” plus title

2 no no no yes yes no

Details at http://jimthatcher.com/testing4.htm

May 11-12, 2009

Testing for accessibility 28

Testing tools - ResultsTotal Bobby In-

Focus LIFT Ramp W-King W-XM

Correct Results 40 28 23 27 38 34 26

Details at http://jimthatcher.com/testing4.htm

May 11-12, 2009

Testing for accessibility 29

Human Review

Human Review Notepad (source code) Screen Readers and other AT Special issue tools (like contrast) Toolbars (the best news) Jim’s Favelets

(User testing – Beyond scope )

May 11-12, 2009

Testing for accessibility 30

Assistive TechnologyTesting with assistive technologies

HPR (IBM) No longer available

JAWS (Freedom Scientific) http://www.hj.com

Window-Eyes (GW Micro) http://www.gwmicro.com

Fangs http://www.standards-schmandards.com/projects/fangs/

May 11-12, 2009

Testing for accessibility 31

Toolbars - for anyone interested in accessibility

Web accessibility toolbar - The Paciello Group http://www.paciellogroup.com/resources/wat-ie-about.html

Best thing that has happened for accessibility – and a toolbar for Firefox

Web Developer Extension for Firefox http://chrispederick.com/work/webdeveloper

May 11-12, 2009

Testing for accessibility 32

IBM Accessibility Testing Matt King started monitoring all of IBM.com (13

million pages) in 1999. Reports to Mgrs and VPs Five Years later – Detectable errors way down. Accessible? Need Human Review – but too many pages Idea: use sampling – like a political poll –evaluate

sample pages and conclude results for whole site http://www.research.ibm.com/journal/sj/443/king.html

May 11-12, 2009

Testing for accessibility 33

Human Review for IBM Needed time per page around 3-5 minutes The toolbars don’t do what testing tools do.

Need value of software testing tools in the human review (e.g., no alt attribute or no form label) Don’t make me look for these.

Simplify human evaluation – scoring (AIR) My evaluation techniques (modified from Toolbar)

http://jimthatcher.com/favelets (includes description of the process)

May 11-12, 2009

Testing for accessibility 34

The Testing Process I useSample report

http://jimthatcher.com/testing/Sample-accessibility.doc

Lets look at some of the same pages – http://www.sec.gov!

It is a mix of Web Accessibility Toolbar and “Jim’s favelets”

The way to do accessibility testing

May 11-12, 2009

Testing for accessibility 35

Thanks! Thanks for being here I can be reached at

jim@jimthatcher.com You can contact Knowbility

knowbility@knowbility.org