Understanding Qualitative Assessment

Skills Session #3A-Team Training

November 13, 2007

Session II

Overview of Outcome Design

Creating Outcomes

Service, Learning, &

Development

Session III

Critiquing Outcomes

3 M’s

Intro: Assessment

Steps

Session I

Outcomes & Overview

What is Assessment

Assessment Language

Comp Model & Foundation

SALDOs

Session IV

Assessment Steps (cont)

Intro: Assessment

Plans

Session V

Review of Assessment

Plans

Session VIAssessment

Plan Presentation

Session Evaluation

Foundation Session OutcomesDemonstrate an understanding of “A”

language & literature

Define and plan an assessment project for your department

Increase Technical Skills—Perseus, Report Writing, Qual/Quan

Perform 1 + assessment projects

Assessment Resource for department and team

What is Qualitative Assessment?

Qualitative research seeks descriptively rich data from a small, purposeful sample with meaning and understanding as it end goals (Lincoln & Guba, 1985).

“The detailed descriptions of situations, events, people, interactions, and observed behaviors; use of direct quotes from people about their experiences, attitudes, beliefs, and thoughts; and analysis of excerpts or entire passages from documents, journals, or case histories…” (Patton, 1990).

Differences from Quantitative Assessment

Qualitative-Focus of research on Quality

(nature, essence)-Key concepts of meaning,

understanding, and description

Quantitative-Focus of research on Quantity (how much, how many)- Key concepts of statistical relationships, prediction control, description, hypothesis testing

Why is Qualitative Assessment important to

student affairs work?

Outcomes of Qualitative Assessment

Detailed descriptions Direct quotations Analysis of documents Deeply rich content

Key Assumptions in Qualitative Assessment

Meaning Making, understanding Not interested in applying broadly to situations

outside the case being studied The researcher is the primary means by which data

are collected and analyzed Inductive data leads to a finding/theory Social reality is constructed by the participants

What kinds of questions would one use as a guide for Qualitative

Assessment? Why How Process Perception Reflection of experiences, feelings

Common Types of Qual Research

Basic Interpretative Study Case Study Ethnography Phenomenology

Qualitative Techniques

Interviews Focus Groups Document Analysis Observations

Sampling

Key features: not random, smaller than quantitative, provide rich information

Several types: (Patton, 2003) Homogeneous-common characteristics Stratified-sample representative of different characteristics

of population (i.e. demographics, participants vs. non-participants)

Criterion-based on predetermined criteria Snowball-ask participants for other potential participants

Data Analysis

Constant Comparative Method (Glaser & Strauss, 1967) Constantly compare from the get-go Example: Researcher journal-hunches, interview,

transcribe, process is constant Often Data compared across multiple forms of data

(interviews, essays, observations)

Coding

Open Coding-General

Selective Coding-Categorizing

Thematic Coding-Themes

Coding Exercise

Candy Exercise

Interview Coding Practice

Coding Tips

After interviews, write hunches down in journal Pay attention to what you are hearing as you

transcribe, write down hunches Don’t overanalyze during general coding Use whiteboard or flip chart to start organizing

categories (from general codes) No formula, but set up in way that best suits your

skills (Remember: You are the research instrument )

Ensuring Rigor

Researcher Journal Hunches Decisions Thoughts Biases, Assumptions

Triangulation Data triangulation Researcher Triangulation

Peer Reviewers

Focus Group

Focus group-Interview a small group of participants (10-12 participants) Advantages/Disadvantages

Permits large amount of data gathered in small amount of time

Challenging to moderate multiple opinions and personalities

TIP: Employ another colleague to take notes! Read “Creating a Focus Group” p. 12-13 (Joel’s

article).

Any Questions?

For other resources, please visit our website.http://www.uga.edu/studentaffairs/assess/