User Studies Motivation

Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/1

User Studies MotivationUser Studies Motivation

January 30, 2007

How do we know whether How do we know whether security is usable?security is usable?

Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/3

Need to observe usersNeed to observe usersWe are not our users!

(you may be surprised by what users really do)

Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/4

Wireless privacy studyWireless privacy study Many users unaware that communications over

wireless computer networks are not private How can we raise awareness?

B. Kowitz and L. Cranor. Peripheral Privacy Notifications for Wireless Networks. In Proceedings of the 2005 Workshop on Privacy in the Electronic Society, 7 November 2005, Alexandria, VA.

Wall of sheepWall of sheep

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

Photo credit: Kyoorius @ techfreakz.org http://www.techfreakz.org/defcon10/?slide=38

Defcon 2001

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

Photo credit: http://www.timekiller.org/gallery/DefconXII/photo0003

Defcon 2004

Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/8

Peripheral displayPeripheral displayHelp users form more accurate

expectations of privacy

Without making the problem worse

Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/10

Experimental trialExperimental trialEleven subjects in student workspace

Data collected by survey and traffic analysis

Did they refine their expectations of privacy?

Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/11

ResultsResultsNo change in behavior

Peripheral display raised privacy awareness in student workspace

But they didn’t really get it

Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/12

Privacy awareness increasedPrivacy awareness increased “I feel like my information /activity / privacy

are not being protected …. seems like someone can monitor or get my information from my computer, or even publish them.”

Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/13

But only while the display But only while the display was onwas on

“Now that words [projected on the wall] are gone, I'll go back to the same.”

Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/14

Security and privacy Security and privacy indicatorsindicators

Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/15

Evaluating indicatorsEvaluating indicatorsCase study: Privacy Bird

Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/16

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

Platform for Privacy Preferences Platform for Privacy Preferences (P3P)(P3P)

2002 W3C Recommendation

XML format for Web privacy policies

Protocol enables clients to locate and fetch policies from servers

Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/17

Privacy BirdPrivacy Bird P3P user agent

Free download http://privacybird.org/

Compares user preferences with P3P policies

Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/20

Critique Privacy BirdCritique Privacy Bird Security people

• Can attackers spoof it?• What if P3P policy

contains lies?• Can P3P policies be

digitally signed?• What about main-in-

the-middle attacks?

Usability people• Green/red color blind

problem• Do people notice it in

corner of browser?• Do people understand

privacy implications?• Why a bird?

Typical Typical securitysecurity

evaluationevaluation

Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/22

Does it behave correctly when Does it behave correctly when notnot under attack? under attack?

No false positives or false negatives

Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/23

Anti-phishing toolsAnti-phishing tools

Y. Zhange, S. Egelman, L. Cranor, and J. Hong. Phinding Phish: Evaluating Anti-Phishing Tools. In Proceedings of NSSS 2006, forthcoming.

Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/24

Does it behave correctly when Does it behave correctly when under attack?under attack?

Can attackers cause wrong indicator to appear?

Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/25

Correct indicator

Wrong indicatorAttacker redirects through CDN

Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/26

Can it be spoofed or Can it be spoofed or obscured?obscured?

Can attacker provide indicator users will rely on instead of real indicator?

Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/27

Usability evaluationUsability evaluation

Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/29

C-HIP ModelC-HIP Model Communication-

Human Information Processing (C-HIP) Model• Wogalter, M. 2006.

Communication-Human Information Processing (C-HIP) Model. In Wogalter, M., ed., Handbook of Warnings. Lawrence Erlbaum Associates, Mahwah, NJ, 51-61.

Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/30

Do users notice it?Do users notice it?If users don’t notice indicator all bets are

off

“What lock icon?”• Few users notice lock icon in browser chrome,

https, etc.

C-HIP model: Attention switch, attention maintenance

Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/31

Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/32

Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/33

Do users know what it means?Do users know what it means?Web browser lock icon:

“I think that it means secured, it symbolizes some kind of security, somehow.”

Web browser security pop-up:“Yeah, like the certificate has expired. I don’t actually know what that means.”

C-HIP Model: Comprehension/Memory

J. Downs, M. Holbrook, and L. Cranor. Decision Strategies and Susceptibility to Phishing. In Proceedings of the 2006 Symposium On Usable Privacy and Security, 12-14 July 2006, Pittsburgh, PA.

Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/34

Netscape SSL icons

Cookie flag

IE6 cookie flagFirefox SSL icon

Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/35

Privacy Bird iconsPrivacy Bird icons

Privacy policymatches user’s

privacy preferences

Privacy policydoes not match user’s privacy

preferences

Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/36

Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/37

Do users know what to do when Do users know what to do when they see it?they see it?

C-HIP Model: Comprehension/Memory

Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/38

Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/39

Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/40

Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/41

Do users believe the Do users believe the indicator?indicator?

“Oh yeah, I have [seen warnings], but funny thing is I get them when I visit my [school] websites, so I get told that this may not be secure or something, but it’s my school website so I feel pretty good about it.”

C-HIP Model: Attitudes/Beliefs

Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/42

Are users motivated to take Are users motivated to take action?action?

May view risk as minimal

May find recommended action too inconvenient or difficult

C-HIP Model: Motivation

Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/43

Do they actually do it?Do they actually do it?“I would probably experience some brief, vague sense of unease and close the box and go about my business.”

C-HIP Model: Behavior

Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/44

Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/45

Do they keep doing it?Do they keep doing it?Difficult to measure in laboratory setting

Need to collect data on users in natural environment over extended period of time

C-HIP Model: Behavior

Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/46

How does it interact with How does it interact with other indicators?other indicators?

Indicator overload?

Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/47

Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/48

Summary: Security evaluationSummary: Security evaluationDoes indicator behave correctly when not

under attack?•No false positives or false negatives

Does indicator behave correctly when under attack?•Can attackers cause wrong indicator to

appear?

Can indicator be spoofed or obscured?•Can attacker provide indicator users will rely

on instead of real indicator?

Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/49

Summary: Usability evaluationSummary: Usability evaluation Do users notice it? Do they know what it

means? Do they know what they

are supposed to do when they see it?

Do they believe it? Are they motivated to do

it? Will they actually do it? Will they keep doing it? How does it interact with

other indicators?