Who are the users of a video search system?

www.axes-project.eu

Who are the users of a video search system?

Classifying a heterogeneous group with a Profile Matrix

Who are the users of a video search system? WIAMIS'12 Dublin 23-25 May 2012

Max Kemman, Martijn Kleppe, Henri BeundersErasmus University Rotterdam

www.axes-project.eu

User research

• Understanding the user [3, 4]– Acceptance– Performance– Capabilities– Weaknesses

• Goal– Creating systems that are intuitive and helpful to

the users


[3] Y. Liu, A. Osvalder, and M. Karlsson, “Considering the importance of user profiles in interface design,” no. May, 2010[4] J. Preece, Y. Rogers, and H. Sharp, “Interaction Design: Beyond Human-Computer Interaction,” Design, vol. 18, no. 1, pp. 68-68, 2002

www.axes-project.eu

Problem statement

• Investigating a heterogeneous user group• Extreme case: potential user group consists of everyone• Distinguishing groups, using classifications of:

[1]– Roles E.g. by occupations– Demographic characteristics E.g. by age, gender or socioeconomic status


[1] D. O. Case, Looking for Information - A Survey of Research on Information Seeking, Needs, and Behavior, vol. 29, no. 3. San Diego Academic Press, 2002

www.axes-project.eu

Research question

Can we classify (sub)groups of users in a heterogeneous group of people and describe their characteristics using the Profile Matrix?


www.axes-project.eu

Profile Matrix• Two axes [5]• Experience [3]– Amount of skills a

user may have or lack

• Goal-directedness [7]– Navigators– Explorers


Goal-directed Non-goal-directed

Experienced

Inexperienced

[3] Y. Liu, A. Osvalder, and M. Karlsson, “Considering the importance of user profiles in interface design,” no. May, 2010[5] K. G. Rasmussen, R. Iversen, and G. Petersen, “M3.2.3 Personas Catalogue,” EuropeanaConnect, 2010. [7] R. W. R. White and S. M. Drucker, “Investigating behavioral variability in web search,” Proceedings of the 16th international conference on World Wide Web - WWW ’07, pp. 21-30, 2007.

www.axes-project.eu

Experienced

Inexperienced

• Going directly for a specific goal, or wandering around• Navigators versus Explorers• Self-evaluated 7-point semantic

differential– Content: what– Location: where– Structure: how– Resource: adjust

• Cronbach’s α=0.753• Aggregated to linear scale from -2 to 2

Goal-directedness axis


-2 1 0 1 2


www.axes-project.eu

• Amount of skills a user may have or lack• Self-evaluated 5-point Likert scale

• Cronbach’s α=0.711• Aggregated to linear scale from -2 to 2


Experience axis

Experienced

Inexperienced

2

1

0

-1

-2

Features• Share• Subtitles• Reuse• Full-screen• Quality

Search• Success• Ease


www.axes-project.eu

How Profile Matrix worksSuppose we have a crowd of people...


http://www.flickr.com/photos/caitlinator/4006198703/






www.axes-project.eu

How Profile Matrix works...we can distinguish groups based on roles or demographics...


www.axes-project.eu

How Profile Matrix works...and then place these groups on a Profile Matrix

Experienced

Inexperienced



www.axes-project.eu

Survey

• Survey (N=970) for personal use of online video search systems– International

• The UK (318) • The Netherlands (311)• Germany (341)

– Representative samples• Age• Gender• Education

– 850 people watched video onlineWho are the users of a video search system?

WIAMIS'12 Dublin 23-25 May 2012

www.axes-project.eu

Survey

• A general overview of – Who the ‘home users’ are– What they use for online video

• Sections– Demographics– Video consumption– Profile Matrix– AXES interest

• Current data published Open Accesshttps://easy.dans.knaw.nl/ui/datasets/id/easy-dataset:49793


https://easy.dans.knaw.nl/ui/datasets/id/easy-dataset:49793

www.axes-project.eu

Majority appears to be inexperienced and goal-directed

Results: Overall Profile Matrix

Experienced

Inexperienced

Goal-directed Non-goal-directed-2 -1.5 -1 -0.5 0 0.5 1 1.5 2

-2

-1.5

-1

-0.5

0

0.5

1

1.5

2


www.axes-project.eu

-1 -0.9 -0.8 -0.7 -0.6 -0.5 -0.4 -0.3 -0.2 -0.1 0

-1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

Series1

• All averages within goal-directed half• Hours per week• Significant differences in experience between

all groups

Results: Consumption Profile Matrix

Goal-directed

Inexperienced


>8h 4-8h 1-4h <1h

www.axes-project.eu

-1 -0.8 -0.6 -0.4 -0.2 0

-1

-0.8

-0.6

-0.4

-0.2

0

Series1

• All averages within inexperienced goal-directed quadrant

• Significant differences in experience between– 18-24 and 45-54– 18-24 and 55+– 25-34 and 45-54– 25-34 and 55+– 35-44 and 45-54– 35-44 and 55+

Results: Age Profile Matrix

Goal-directed

18-24 25-34 35-44

45-54 55+

Inexperienced


<45

≥45

www.axes-project.eu

-1.0 -0.8 -0.6 -0.4 -0.2 0.0

-1

-0.8

-0.6

-0.4

-0.2

0

Series1

• All averages within inexperienced goal-directed quadrant

• No significant differences

Results: Education Profile Matrix

Goal-directed

PrimarySecundaryHigher


Inexperienced

www.axes-project.eu

Conclusion

• Profile Matrix helps to create (sub)groups of people

• Better understanding of the heterogeneous group

• Helps choosing groups for usability research


www.axes-project.eu

Future directions

• Investigating what these differences mean with regard to users’ performance or acceptance of video search system

• Groups based on roles instead of demographics– Broadcast Professionals– Academics– Journalists– Home Users

• Application for other search systemsWho are the users of a video search system?

WIAMIS'12 Dublin 23-25 May 2012

www.axes-project.eu

Questions or comments?

Do you think this is useful?

Contact:Max Kemman

[email protected]


Technology

Who are the users of a video search system?