Sakai in Language Courses:

1. Sakai in Language Courses: Present Uses and Future Possibilities
Ken Romeo, Ph.D.
Academic Technology Specialist
http://kenro.web.stanford.edu :: [email protected]

2. Outline
Background
The Present: Sakai in the Stanford Language Center
Objectives and Results
Summative Assessment
Formative Assessment
The future: What this could be
How students study
How teachers (would like to) teach
A framework for curriculum
Heads up! This is not just about language teaching.
July 9, 2009
Stanford Language Center / Academic Computing
2

3. Background
Me
ESL instructor (20 years)
Academic Technology Specialist (2006)
CourseWork (Sakai) team meeting observer / participant (2008)
Stanford Language Center
Language requirement
1995 new director: Prof. Elizabeth Bernhardt
Emphasis on assessment and professional development
This presentation
American Council on the Teaching of Foreign Languages (ACTFL 2007) Bernhardt, Molitoris, Miano, Gelmetti, Tsethlikai, Romeo
Sum of experience
3
July 9, 2009

4. Sakai and the Stanford Language Center
The Present:
July 9, 2009
4

5. Assessment Program
Objectives
Improve student performance
Enhance credibility (students and the public)
Programmatic consistency
Methods
ACTFL Oral Proficiency Standards (ACTFL, 1999)
Oral Placement and Exit exams
More face-time for instructors put diagnostic assessment online (CourseWork)
5
July 9, 2009

6. Overview: Results
More highly trained staff
95% go through ACTFL interview training
Over 1/3 certified
Professional conversation
20% first year and 24% second year increase
Highly positive student reaction
Use Sakai to:
Deliver formative assessments (SAMigo)
Connect to exit assessment (enrollment & archiving)
6
July 9, 2009

7. Foreign Language Placement
800 unregistered students: not in Sakai
Online tests during the summer
Oral assessment on campus: 1 hour, cassette tapes
7
July 9, 2009

8. Exit Assessment: SOPI Definition
Oral Proficiency Interview (OPI): Live
Nationally standard format used everywhere
Simulated Oral Proficiency Interview (SOPI): Recorded
1st year: ~10 items, 2nd year: ~20 items:
English instructions + line drawing
Thinking time
Native prompt
Response recording
Go to next item (NO USER CONTROL)
From 2008: Add a writing assessment
8
July 9, 2009

9. SOPI Application
July 9, 2009
9

10. SOPI Monitor Tool
July 9, 2009
10

11. SOPI Delivery
Scheduling, content creation human hours
Software requirements
High stakes: content security and reliability
Enrollment from existing courses (Sakai)
Non-standard roles: Coordinator, instructor
No student control = no web delivery
Playback or recording
Test progression
Securely archive audio recordings (Sakai)
11
July 9, 2009

12. SOPI Software (Project Manager: Makoto Tsuchitani)
Application (Developer: Casey Dunn)
Desktop Java application
Quicktime for Java play and record
WebServicesto communicate with Sakai
Sakai Monitor Tool (Developer: Zach Thomas)
Realtime progress of each student
UI depends on role
Packaging for further dissemination
12
July 9, 2009

13. Formative Assessments in Sakai
1st year: Oral diagnostic assessments
Collaboratively developed content (audio / video / text)
SAMigo: Audio Recording applet
Resource (course) sites
2nd year: Writing Diagnostic assessments
Collaboratively developed content (images / text)
SAMigo: Timed, short answer
Resource (course) sites
13
July 9, 2009

14. Technology Integration: Key Points
Each part has a pedagogically valid purpose and is not focused on technology.
Uses a standalone application with connection to Sakai to do what the web cant.
All instructors create material based on the same standards-based framework.
Implementation takes a huge number of human hours and coordination across many different groups.
14
July 9, 2009

15. What this could Be
The Future:
July 9, 2009
15

16. Prologue: Why use an LMS at all?
Privacy and authentication
Scalability:
class department university (?)
Modularity:
Centrally stored, clone-able units
One portal to existing technology (connection, organization, computation, audio, video, telephony, etc.)
16
July 9, 2009

17. How Students Study
July 9, 2009
17

18. Increased levels of connection
Cell phones: voice and text
Sharing small groups, whole class, future classes (Submission? Grading? Feedback?)
Online office hours
Social networking managing multiple identities, authentication to protect privacy
18
July 9, 2009

19. Familiarity with digital environment
New formats for work (video, etc.)
Dont always carry laptops often use clusters, do carry cell phones
Use multiple resources search / self-study / scaffolding
Note: Find the least common denominator
19
July 9, 2009

20. How Teachers (would like to) Teach
My Wishlist
July 9, 2009
20

21. Different tools for different tasks
In class teaching
Homework
Self-study
21
July 9, 2009

22. No-brainer: Video, everywhere
Upload ingest to streaming
Clip creation, indexing, delivery, annotation, collaboration
Control playback just once, twice
Why streaming? Too large to download.
Why not YouTube? No privacy / authentication. (which is what Sakai does very, very well!)
22
July 9, 2009

23. Organization (schedule, groups)
Course structure: Often linear, by definition
Tracking students
Integration with University registrars
Arbitrary groups of students
In/across courses/programs
Requirements, milestones, electives
23
July 9, 2009

24. Identities (roles)
Need a departmental or other arbitrary level
Social networking (or not)
Multiple identities
Retaining pedagogical control: Assessment
In/out groups: Fairness? Motivation?
24
July 9, 2009

25. Assessment (SAMigo)
Control response format:
Limit and time chances to view/listen
Limit and time chances to respond
Enable large scale assessment
Get rid of all those blue books
Work with infrastructure groups: machines, space
Telephone (voice) delivery of Sakai is a killer app:
Accessibility
Very controlled linear assessment
Anyone want to be partners in a grant?
25
July 9, 2009

26. Content creation (SAMigo)
Authoring: basic desktop tools, conversion, definitions, spell checking
Break up into modules: Re-usable, sharable, organize-able
Changing order of delivery
Changing details of content: Randomization of items / variables
Downstream control of shared content
26
July 9, 2009

27. Reports (gradebook, etc.)
Grading on an arbitrary curve
Item analysis: Stats, test theory
Assessments / items across departments / years (aggregate-able)
Log files: Reportable numbers to stakeholders
27
July 9, 2009

28. A Modular Curriculum
A Framework for the Future
July 9, 2009
28

29. How?
Homework: collaboration, or not pedagogical control
Self-study:
Students who need it
Students who want to do self-study
Publicly available open-source set of online activities
Classroom: more in a minute
29
July 9, 2009

30. Why?
Textbooks = information + activities + order
Textbooks unbound
Teacher must decide order
Information
Multiple sources
Multiple formats
Students can independently supplement
Basic unit = activity
Requires / allows creativity
30
July 9, 2009

31. Basic curricular unit: the Activity
Re-conceptualize interactive lecture / seminar
Control information in the classroom
Elicit search / curiosity outside of the classroom
Facilitate all departments products
Linguists tree diagrams
Engineering simulations
Video
Etc.
31
July 9, 2009

32. In the Classroom
Physical: personal interactions, hardware (or not)
Virtual: distance learning
Small group activities:
Quick, arbitrary, but airtight membership
Posting materials for each group separately
Posting product of each group separately
Quick, but controlled access to multimedia
Easy creation of multimedia product
An example: DIY fill in the blanks
32
July 9, 2009

33. Classroom Example: DIY Fill in the Blanks
2 groups
2 short (30 sec) audio clips
2 paper transcripts
2 pairs of scissors
Each group cuts out 15 words to make a fill in the blanks problem
Exchange transcripts
Play clips (x2-3)
Group with most correct answers wins
July 9, 2009
33

34. Digitize DIY Fill in the Blanks
Arbitrary groups
Secure content: audio and text
Modify text
Exchange text without revealing original
Play audio
Check answers
Group collaboration
July 9, 2009
34

35. Epilogue: Take home messages
Focus on the users: how can we change education? (ref: introduction of textbooks)
Understand what the important characteristics of pedagogy are (and are not).
Dont just facilitate pedagogy as it is, find out where pedagogy is going by talking to expert teachers.
Focus on pedagogically valid activities not on the tools.
Motivated students / creative teachers are NOT the problem: We need to reach everyone.
35
July 9, 2009

36. Thank you very much!
Ken Romeo :: http://kenro.web.stanford.edu :: [email protected]
July 9, 2009
36

Education

Sakai in Language Courses: