Evaluation of Web-Based Professional Development using

Action Research

E. Alana James, Ed.D.

http://www.ar4everything.com

2nd year of 3 year study

• Use of participatory action research (PAR) to study and improve educational practises for homeless or transient students in USA

• The transference of PAR assistance and facilitation from face-to-face to web-based communities of practise

Theoretical basis A

Adaptive Leadership(Heifetz, 2000)

• Educational disadvantage is an adaptive challenge indicative of distance between real and ideal

• An adaptive issue needs to be addressed through: a) creating a holding environment that contains the stress inherent in tackling BIG issues, b) keeping the focus on the distance between the ethical ideal and daily reality

• Strong evidence exists from the first cohort that PAR creates a holding environment with which educators can study issues of educational disadvantage

Theoretical Basis B

Communities of Practice (Wenger, 2002)

• Group of people joined in shared development of practise

• Engaged in joint activities where they help each other and share information

• 3 Variables emerge:– Joint enterprise (what the group is doing)– Mutual engagement (how it functions)– Shared repertoire (archives and

understandings)

Research Questions

• Whether and to what extent did the WBPD cohort successfully:– Create a holding environment with which to

study educational disadvantage?– Create a national community of practise to

advance the understanding of homeless and highly mobile students in the US?

1st Year: Who, what, where, how

• 18 educators: 9 administrators, 7 teachers from 1 rural, 3 small, 3 suburban, 2 urban schools and 1 homeless shelter for teens

• 16 completed a year of participatory action research projects

• Met for one full Saturday every two months to work together as a group for which they received $3,000

2nd year

• 41 educators and community members: 14 administrators, 15 teachers and 12 community members from primarily urban populations

• 29 Completed a year of participatory action research projects

• Met once for training, then locally in community teams They also had an online community of practice available for facilitation.

• They received $1,000 stipend at the end of the project.

CoP desired outcomesPrincipal

Community memberTeacher

Principals Teachers Community members

Face-to-Face Cohort

Community Teams

What really happenedPrincipal

Community memberTeacher

Principals Teachers Community members

Face-to-Face Cohort

Community Teams

Contextual similarities

• Recruited educators in schools known to have issues with homelessness and high mobility

• Teams included both educational roles of an administrator and a teacher

• Process geared towardsresearch

Contextual Changes

• Teams of 2 Teams of 3 w regular mtg• Face-to-face 1 training + Web-based

– Content in class Content on Web– Participatory elements in large group meeting

Participatory elements in local teams– CoPs equal to participatory teams CoPs

designed to occur through web-based practise

• One State Four states• Stipend decreases from $3,000 to $1,000

What we heard…positives

• As a professional development experience you live it you don’t forget it.

• More valuable - because it is on me and my kids.

• Our team is tight - we raised awareness in school, out of school, and overall interaction. We have so much fun together.

• Good to connect for community piece - kept us on our toes - and created more community awareness.

What we heard…..Challenges

• The computer (was a problem) it too too long to learn how to access it and use it - even getting on it was difficult - didn’t have the time to figure it out. Being older people, we are not meant to sit in front of the computer.

• Much more responsibility and therefore more difficulties keeping up and finding the time.

• Much more difficult because it is much more active - no chance to be passive here. We had to be busy - Focus us to become an active learner.

• 2 year not as good as 1st year didn’t have the accountability to the group meetings.

High attrition

• 2003-2004: Lost 2 people or 10%: 1 due to family illness and one to difficulty with team member

• 2005-2006: Lost 4 teams of people (11 people or 25%) due to change in jobs, personal issues and a basic inability to stay involved with the project

• Two of those teams “limped along” pretending to themselves and to us to be involved almost until the end

Results - CoP: Joint Enterprise• Community teams demonstrate Wenger’s

“renegotiation” by its members– She puts Jim in his place. A community

member “renegotiating” with the school principal.

• Little National joint enterprise + some discussion on the website– Interesting that the discussions which had the

most zest were on topics which came from their schools 1 on the hurricane homeless and another on the time it takes to enroll a student in special services

Mutual engagement

• When I presented my goal of helping highly mobile and homeless students become more academically competent to my colleagues, they found it to be too broad. They helped me narrow my focus challenging my assumptions about my students (1st cohort)

• We all wanted to know the information - our team worked together well. (2nd cohort).

• It was a thing we wanted to know - different approach not bottled we could do what we wanted - we liked structuring it our way - we could define our own project.

Shared repertoire• Language to some extent defined by literature used in

the project: – Access to educational services – Welcoming school culture– Flexible instructional strategies

• PAR process and the similarity in educational contexts contributed – the online discourse included discussions of research questions,

exchanging survey documents, and discussions of the challenges of enrolling these students in special programs)

• National project challenged by local dynamics:• Teams hit by hurricane homeless

– Teams who knew each other so well they didn’t interact with national group

– Teams whose members didn’t interact enough– Teams whose members changed jobs mid year

• More cross site familiarity in Colorado, where they met once in a large group.

Results: HeifetzCreating a holding environment

• I felt a connection to these issues because of the dynamics of the school, but I felt on the outside... As I went through the first cohort I started to pull myself from the outside to the edge, to the middle, and now I feel like I'm totally immersed. (1st cohort)

• I started this study blindly not really knowing what I was about. After reading the material, reflecting on my past experiences working with homeless and highly mobile students, interviewing parents and researching information, this one thing I know. I have just begun to unfold a project that is so important to the well being of our school. (2nd cohort)

Programmatic Outcomes: Advocacy

• Increase in attendance in case study (VA)

• Community awareness generated through the press (CO)

• Continuing work with out of school youth to “help them return to school.” (1st cohort)

• Continued work through district administration to “increase awareness and services” (1st cohort)

Knowledge outcomes

• Schools in 4 states addressed issues of welcoming school culture

• The importance of parent involvement to homeless parents and the barriers that prevent it

• How to improve homework for disadvantaged primary aged students

• Protocols for school office staff and for folders of information going home to parents

• Learned about the planning needed to cope with national disaster

Action Outcomes

• As a result of this research, an overhaul of the assessment program took place.

• As a result, a parent/child homework group was set up at a site near the housing program where families resided.

Personal Outcomes

I have grown in my capacity as an educator through my willingness to admit my

limitations and seek collaboration in the interest of all our students.

Similarities and Differences

In both cohorts:• People drifted in and out

of focus on the project and yet, at the end found that they had results they could be proud of

• Participants demonstrated both an increase in knowledge about and engagement with issues of H&HM students in schools

• Great variety in both the action outcomes and the degree of validity and reliability in results.

• The accountability to the project came from them and was thus much more difficult to maintain.

• Because of less hands on facilitation the results from the second cohort demonstrated fewer actions, falling into a tendency to see gathering data and knowledge as an end results rather than implementing new educational practise.

What we learned about the website

• It seemed to me the tasks were more linearly organized and this helped personally and to keep the group on task.

• Most found it to contain too much content (differentiation between PD and a course)

• Regular tasks were beneficial. One community member reported calling team meetings whenever she was afraid they were behind.

• More facilitation would have both kept the participants engaged with project guidance and created a stronger environment for research.

Conclusions

• PAR continues to provide a holding environment for adaptive leadership, although without the supporting CoP some members get lost, diminishing long term results

• Although evidence is conclusive that the new practises successfully build local CoPs, little evidence points to a National CoP

• We hope to overcome these challenges with the redesigned website for 2006-2007