User Centered Design in the Course Management Project

http://bugs.sakaiproject.org/confluence/display/CM/Home

Marc Brierley, Stanford University

Daphne Ogle, University of California, Berkeley

Agenda• Course management project overview• Framing the work in a user-centered design (UCD)

approach• User research

– interviews and observations

• Making sense of the research– user, goal and workflow modeling

• The rest of the UCD process– requirements, framework & design

• How does this process change things?

Overview of Course Management Project

• Why?– SIS integration– Confusing and overlapping functionality

• What?– New APIs– Framework enhancements– New and enhanced tools

• When?– Winter ‘07 for production

QuickTime™ and aTIFF (LZW) decompressor

are needed to see this picture.

QuickTime™ and aTIFF (LZW) decompressor

are needed to see this picture.

Stage 1 - User Research• Interviews and Observations

• Across several institutions and user groups

• Support panel: “Data” from the users’ perspective

User Research Goals

• Observing real users in real context• What we wanted to find out

– Problems / pain points– Domain knowledge– Broader context / not just product-focused– Basic understanding of current tasks– Clear understanding of goals

Interview & Observation Nuts & Bolts

• Interview Prep

• Running the interview

• Observing quietly and unobtrusively

• Processing the findings (right away!)

Interview Prep• Recruiting

– Who are users?– What if you don’t know m/any users?

• Bridge people

• Scheduling– People, time, location

• Coming up with questions– Categories & general questions

• Confirmation / Reminder

Interview QuestionsFlow1. Introductions2. Why we're here: we've been asked to design a next generation tool for the management of courses within a Learning Management System3. What we'll ask: your day, your frustrations

Overview Questions

1. Tell us about your responsibilities and typical day2. What would you say is the main goal in the task(s) you have?3. What’s your favorite aspect of your job?4. What don’t you like about your job? What’s tasks would you rather not have to do?

Activity questionsGeneral1. What kind of administrative tasks do you deal with around teaching your course?2. Can you talk generally about how students sign up for and switch classes? What’s the beginning of the term like? How far into the semester

can students switch courses?3. Is your class ever listed as a cross listing? What does that mean?4. What does the term "section" mean to you? * How do you handle sections? * What are your section types? * How often do students switch sections?

Interview QuestionsCMS-related (if there is an existing product)1. How do you handle the creation of a site for your course?2. How do you handle content, if any?3. With regards to course syllabus information, does your cms support uploading of a local file as well as entering text right

into the syllabus page? Which option do you use?4. How are students added and dropped from the site?

Other general question if they haven't come up:Opportunity* What things waste your time?

Priorities/Goals

* What makes a good work day? A bad one?

* What is the most important thing you do?* What does your boss think is important?

Support* What kind of training do you receive?

Interview QuestionsConnections* Who else do you work with to get things done?* Who else provides information to you and to whom do you provide information?

Final Questions/Wrap-up1. Are there any less frequent tasks you do that I might not have observed today?2. Follow-up on interesting points

Observation* Environment Observations* Observations/General

Running the interview• It takes 2

– Interview leader & primary note taker– Balance between natural conversation and moving things

forward

• Context is important!• Set expectations

– Introductions– Not questioning users skills, abilities, work– Ask permission to follow-up via email– Ask for suggestions of others to talk to

• Be respectful of time– You’ll likely want to talk to them again

Observing quietly and unobtrusively

• Observe REAL work

• Notice things outside of the system

• Environment is important

• Focus on detail - Analyze for goals

• Harder than it seems!

Document the findings (right away!)

• Collaborative analysis of what you learned while it’s fresh

• Organized note taking pays off here– Composition book– Indexes

• Don’t guess• Don’t have to wait until the last interview or

observation– Start creating models early and iterate as you learn more

Stage 2 - Model the Users, WHY?

• Helps key stakeholders understand

what was learned in research

• Guides the design and development of

the software

• Prioritizes requirements

Types of User Models

• Mental Models

• Personas

• Context Scenarios

Mental Models

• Across the various users, what are the mental constructs that shape how people think about the situation or problem?

• We found (no huge surprise) that perceptions of how courses are structured made a big impact.

Mental Models:Course Structures

Personas

• What are the users’ values and goals (not just tasks)?

• Looking for patterns of usage and behavior

• What are their human characteristics?

Sara Windsor

Faculty, MIT

Goals: Build on course materials from term to termNot to have to ask for helpTo spend as little time as possible doing administrative work; she’ll delegate to her TA’sTo get tenure (get credit for tenure for everything she does)Use technology to help create an engaging and interactive environment for her on-line students where she can track their progressTo be respected by students, colleagues and dean of school

Level of Expertise: Teaching for 10 years, uses software apps like Word, excel, ppt, email, on-line research

Sara taught at the University of Pennsylvania before joining MIT 2 years ago. She is married with 2 kids and stays extremely

busy teaching an undergrad psychology course for the 2nd year and a new on-line course in organizational management. She is fairly comfortable with computers. She’s excited to be using technology to help with her teaching but she sees as a means to an end in order to stay in touch with students and allow them access to course resources when they need it. In fact, the LMS will replace her face-to-face communication with students for her on-line course.

She knows that even students she sees in class twice a week would like to her to be more responsive and looks to the course site to help with that. She used Blackboard at PA and notices that she has expectations based on that experience…some good, some bad.

Carol Hudson

CMS part-time support/student

Carol is has been back in school for year working toward her masters degree Education. She’s doing support for

Yale’s LMS 12 hours a week to bring in some income while she’s in school. Her student loans barely cover tuition and rent.

Carol likes to help people and gets frustrated when she doesn’t feel like she has the answer for customers…or even worse is confused herself.

She uses the LMS for a couple of her own classes since she began working in the support group she’s gained empathy for the work faculty have to put into managing the course sites.

Goals: Make it through graduate school with good gradesEarn money to pay for living expense while in schoolGive the right answersTeach people

Level of Expertise: Experience as a student on LMS. Fairly comfortable with technology and the web.

Whiteboard Exercises

• Summarize notes

• Persona scales

Summarize Notes

Persona Scales

• Brainstorm types of characteristics to make scales

• Put each user on continuum or in a bucket

• Look for the groupings

Writing a persona

• State the user’s goals – Life: Retire by 45– Experience: Not feel stupid– End: Set up my course site

• Include behavior patterns, not job description

• Add some biographic info to make the persona seem real, but not too much

Sara Windsor

Faculty, MIT

Goals: Build on course materials from term to termNot to have to ask for helpTo spend as little time as possible doing administrative work; she’ll delegate to her TA’sTo get tenure (get credit for tenure for everything she does)Use technology to help create an engaging and interactive environment for her on-line students where she can track their progressTo be respected by students, colleagues and dean of school

Level of Expertise: Teaching for 10 years, uses software apps like Word, excel, ppt, email, on-line research

Sara taught at the University of Pennsylvania before joining MIT 2 years ago. She is married with 2 kids and stays extremely

busy teaching an undergrad psychology course for the 2nd year and a new on-line course in organizational management. She is fairly comfortable with computers. She’s excited to be using technology to help with her teaching but she sees as a means to an end in order to stay in touch with students and allow them access to course resources when they need it. In fact, the LMS will replace her face-to-face communication with students for her on-line course.

She knows that even students she sees in class twice a week would like to her to be more responsive and looks to the course site to help with that. She used Blackboard at PA and notices that she has expectations based on that experience…some good, some bad.

Carol Hudson

CMS part-time support/student

Carol is has been back in school for year working toward her masters degree Education. She’s doing support for

Yale’s LMS 12 hours a week to bring in some income while she’s in school. Her student loans barely cover tuition and rent.

Carol likes to help people and gets frustrated when she doesn’t feel like she has the answer for customers…or even worse is confused herself.

She uses the LMS for a couple of her own classes since she began working in the support group she’s gained empathy for the work faculty have to put into managing the course sites.

Goals: Make it through graduate school with good gradesEarn money to pay for living expense while in schoolGive the right answersTeach people

Level of Expertise: Experience as a student on LMS. Fairly comfortable with technology and the web.

The Rest of the UCD Process

• Stage 3 - Requirements

• Stage 4 - Conceptual Framework

• Stage 5 - Design

Stage 3 - Requirements• Constructing context scenarios, based on persona

and organizational goals• Creating an initial list of each persona's major

data/information needs• Creating an initial list of functional needs (these

are actions that personas' will need to take in relation to that data)

• Noting persona capabilities that will affect the design (for example, physical or training limitations)

Context Scenario:Create a New Course Site

• Goals:• Distribute course content

• Start communicating with students

• Organize materials for first time

• Two weeks before the term, Sara is teaching Abnormal Psychology for the first time and wants to get her “course site” going.

• Gathers a rough set of materials (books, articles, slides, etc.) and starts grouping things together

• Writes syllabus draft based on these materials

• Confirms with four previously recruited TAs that they will be working with her

• Three days later, she sits down at her computer in her home office and logs into her institution’s LMS to set up her course site

• She sees the “create site” function and chooses it• She sees a list of the courses she’s teaching this term including Psychology 101 and

Abnormal Psychology (including its discussion sections)• She sees Psychology 101 already has a course site and remembers she did that last

month• She creates the course site for Abnormal Psychology• She sees that the site was created• She goes to the site to post her syllabus and readings for the first week • She tells her students to read the materials before the first lecture• As Sara logs out of the LMS she hopes she will remember where she left off when

she puts

• What ifs:• She doesn’t see the courses she is teaching when she chooses “create site”?

Stage 4 - Conceptual Framework• A number of key-path scenario descriptions

• A coherent conceptual framework that describes the product, service, or system that satisfies your persona's goals and the organization's objectives

• Descriptions of key features and major interface elements and how they work together in the framework

• Primary navigation characteristics and information architecture.

Stage 5 - Design

• Full set of mockups and behavior specifications based on framework and scenarios

How do we expect the outcomes of this process to

impact the design?

Creating a Course Site - Current

Creating a Course Site - New

When do you do all this stuff?

Before development starts!

Other related activitiesDomain/market research

(CIO/Deans/marketing)

Development/Design Iteration

Usability Testing

Graphic Design/Branding

Resources• Course Management Confluence Site,

http://bugs.sakaiproject.org/confluence/display/CM/Home

• Cooper, http://www.cooper.com/– http://www.cooper.com/newsletters/2001_07/perfecting_your_personas.htm

• Menlo Innovations, http://www.menloinstitute.com/

Thanks!

• See you at the Course Management BOF, place and time?

• Questions?