23
Meeting the Challenges of Multi-Site, Multi-Researcher Interpretivist Research Dr. Dianne L. Ferguson, Professor Chapman University, One University Drive, Orange, CA 92866 [email protected] Dr. Susan Tetler, Associate Professor & Dr. Kirsten Baltzer, Associate Professor School of Education, University of Aarhus, Denmark, Tuborgvej 164, DK- 2400 Copenhagen NV [email protected] and [email protected] Abstract While the use of qualitative methods within an interpretivist research tradition is more common in educational research, the field is still exploring the challenges of large projects that employ multiple researchers across multiple sites. This paper describes a how a team of 7 researchers and 1 research consultant conducted a study in 23 school located in 19 Danish municipalities. It explores some of the challenges of this kind of research teaming with a particular focus on developing a semi-structured approach to observation. Offering a frank discussion and description of one team’s efforts can encourage others to take on the risks and challenges of cooperative team research and possibly avoid some of the struggles. Introduction: Team Qualitative Research as a Growing Need/Practice Despite some early examples, in the last 25 years more and more educational research in education has relied on interpretivist traditions and qualitative methods. This is true even in special education and disability study research (e.g., Davis, 1995; Edgerton, 1967, Ferguson, 1987; Ferguson, 1998; Minow, 1990; Naraian, 2008). Much of this work followed the tradition in interpretivist research of the single researcher (e.g., Ferguson, 1987; Goode, 1994; Groce, 1985;

Meeting the Challenges of Multi-Site, Multi …pure.au.dk/portal/files/41399219/Meeting_the_Challeng… · Web viewMeeting the Challenges of Multi-Site, Multi-Researcher Interpretivist

  • Upload
    others

  • View
    2

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Meeting the Challenges of Multi-Site, Multi …pure.au.dk/portal/files/41399219/Meeting_the_Challeng… · Web viewMeeting the Challenges of Multi-Site, Multi-Researcher Interpretivist

Meeting the Challenges of Multi-Site, Multi-Researcher Interpretivist Research Dr. Dianne L. Ferguson, Professor Chapman University, One University Drive, Orange, CA 92866 [email protected]

Dr. Susan Tetler, Associate Professor & Dr. Kirsten Baltzer, Associate Professor School of Education, University of Aarhus, Denmark, Tuborgvej 164, DK-2400 Copenhagen NV [email protected] and [email protected]

Abstract While the use of qualitative methods within an interpretivist research tradition is more common in educational research, the field is still exploring the challenges of large projects that employ multiple researchers across multiple sites. This paper describes a how a team of 7 researchers and 1 research consultant conducted a study in 23 school located in 19 Danish municipalities. It explores some of the challenges of this kind of research teaming with a particular focus on developing a semi-structured approach to observation. Offering a frank discussion and description of one team’s efforts can encourage others to take on the risks and challenges of cooperative team research and possibly avoid some of the struggles.

Introduction: Team Qualitative Research as a Growing Need/Practice Despite some early examples, in the last 25 years more and more educational research in education has relied on interpretivist traditions and qualitative methods. This is true even in special education and disability study research (e.g., Davis, 1995; Edgerton, 1967, Ferguson, 1987; Ferguson, 1998; Minow, 1990; Naraian, 2008). Much of this work followed the tradition in interpretivist research of the single researcher (e.g., Ferguson, 1987; Goode, 1994; Groce, 1985; Harry, 1992; Janko, 1993; Murray-Seegert, 1989). Even a cursory examination of popular textbooks rarely mention team research and when they do, only briefly (see, for example, Bogden & Biklen, 2007; Eisner, 1990; Lincoln & Guba, 1985;Maxwell, 2005; Patton 1990; Marshall & Rossman, 2006) . Most are written for the new researcher completing a dissertation which is, admittedly a lone pursuit (Wasser & Bresler, 1996). While some interpretivist educational research continues to rely on the single researcher, and likely, will for some time, the dominance of such research is eroding for several reasons.

First, the pragmatics of being a researcher and educator often demands cooperation with colleagues in the face of meeting the demands of teaching and service. Together, we can sometimes do more than on our own. More important, however, is that, again despite a long tradition, education as a field increasingly is recognizing the collective nature of knowing and the social processes of learning and development (e.g., Dewey,1938 ; Lave & Wegner, 1991; Vygotsky, 1986). This growing recognition of the collective nature of learning and knowing has also led to discussion of the social nature of research which has led to explorations of participatory research (e.g., McIntyre, A. 2007) and some forms of

Page 2: Meeting the Challenges of Multi-Site, Multi …pure.au.dk/portal/files/41399219/Meeting_the_Challeng… · Web viewMeeting the Challenges of Multi-Site, Multi-Researcher Interpretivist

collective action research (Anderson, Herr & Nihlen, 1994). The examples of more collective, or “team” research, are growing in use.

Second, the reality of educational research, whether solely qualitative, solely quantitative, or projects that draw upon multiple inquiry traditions, increasingly require collaborative group and team research. More and more funded research requires answering policy and practice questions that also require inquiry in multiple sites, by multiple researchers (sometimes using multiple research traditions), across contexts, drawing information from a range of participants. In our study, the ministry of education in Denmark was interested in knowing what outcomes expenditures on special education supports were achieving. The research faculty at the School of Education, University of Aarhus in Denmark won the research award to explore this policy question in several different ways. The project we are reporting on here focuses looking at the classroom practices of teachers who worked with 6 different groups of students with disabilities (those with ADHD, autism, blindness, cerebral palsy, dyslexia and learning difficulties). The study sought to understand if learning outcomes differed for these students depending on whether they were in more inclusive classrooms or more segregated educational settings.

There is certainly a longer tradition of team research in objectivitist/quantitative research, but there are only a few published discussions in the interpretivist literature on the advantages, and especially the challenges, or team qualitative research. For example, Whitt & Kuh (1991) used a team approach to explore the quality of out-of-class activities of students across 14 institutions of higher education. This report of their team efforts found that multisite qualitative research using multiple researchers is both time-consuming and expensive, but that it can work and yield descriptions of each site as well as across site descriptions of high-quality out-of-class experiences. The focus in this discussion is largely on the challenges of group analysis and composing and supporting the research team where different members have different roles in the overall project.

Wasser & Bresler (1996) push the discussion further push the discussion further to explore the role of collaborative interpretation. They propose the notion of an “interpretive zone” of collaborative analysis and learning in which a team of researchers engage in collaborative research, but also learning. The interpretive zone is the “mental location where interpretation takes place in this process (p. 13).” Researchers on the team “bring together their different kinds of knowledge, experience, and beliefs to forge new meanings through the process of the joint inquiry in which they are engaged (p. 13).” We too, found that this kind of collective interpretive process was part of what helped us move our efforts forward to completion.

More recently Kinzie, Magolda, Kezar, Kuh, Hinkle, and Whitt (2006) explore multi-site higher education research that relied primarily on short visits and interviews of key informants. Their focus is to explore the dynamics of large-scale, team research with an emphasis on methodological decision-points and the team processes needed to report results. The Kinzie, et al., also explore not only the complexities of logistical planning across institutions of higher education, the negotiation and procedures for working with institutional liaisons, but also issues of time on site and whether their efforts were able to avoid the critique of what Rist (1980) called “blitzkrieg ethnography”. Finally, they offer helpful examples of data reduction through a series of case accounts that became successive sources of data for eventual cross-case analysis and interpretation.

Guest and MacQueen, (2008) in a book devoted to team-based qualitative research, use the contexts of a series of large scale, international, multi-site, muti-researcher projects. These are often clinical trials for various HIV and other disease interventions in developing countries and treatments that also include an interview component. The authors offer very helpful discussions of such topics as 1) the

Page 3: Meeting the Challenges of Multi-Site, Multi …pure.au.dk/portal/files/41399219/Meeting_the_Challeng… · Web viewMeeting the Challenges of Multi-Site, Multi-Researcher Interpretivist

ethics and politics of gaining entry to various countries and the ability to maintain presence despite changing local contexts, 2) issues in selecting team members and developing team dynamics, 3) training researchers on various data collection and management protocols, 4) protocols for transcribing data and developing code-books for analysis. While this new book offers much useful detail for any qualitative team research effort, all of the studies relied primarily on interviews and discuss much less the management of observations across multiple researchers and sites. In fact, this is true of nearly all of the research we found on team qualitative research.

The focus on this paper then is to describe a team research effort that relied heavily on observation in addition to interviews, teacher diaries, document collection and analysis. We first describe, using the example of Van Maanen’s (1988) description of “realist tales” the research focus and process. Incorporated into this will be a deeper exploration of the development for this project of a semi-structured observation tool. Finally, we will describe the process of analysis and interpretation and reflect on the degree to which this team was able to engage in this work in a “zone of interpretation” (Wasser & Bresler, 1996).

The Team Project The study deals with the learning experiences of 26 students with disabilities – specifically groups of students diagnosed with ADHD, autism, blindness, cerebral palsy, dyslexia and learning difficulties – in their school settings. Fifteen students were included in general education classrooms, while the other 11 students were placed in more segregated settings such as special classes and special schools. All the students were in elementary schools. The research was funded by the Danish Ministry of Education and wished to explore the policy question of whether or not the resources being directed to special education services were “working.” Specific questions included: 1) Are there advantages and disadvantages that can be found in the specialized/segregated services as compared to the more included settings in terms of resources available and students’ benefits; and 2) What choices are offered to parents in choosing school placements for their children. In addition, the project sought to explore whether the pedagogical process used in more included as compared to separate settings were similar or different.

In order to explore ongoing pedagogical processes required the team to capture classroom complexity in a variety of ways. To this end, the research team used a range of methods including structured interviews with focus students (students with disabilities), and semi-structured interviews with their teachers and parents. The team also completed questionnaires about classroom climate with the focus students and their classmates (304 in all) and collected teachers’ diaries of “success stories”. Each of these sources of data provided a wealth of information that could be used to 1) understand the experiences of each focus students, 2) compare students within categories (e.g., blind, autistic, etc.), 3) compare students within categories across segregated and more inclusive settings, 4) compare categories of students with disabilities, and 5) compare categories of students with disabilities across segregated and inclusive settings.

A central data collection strategy involved observations and finding a ways to structure those observations across settings and researchers to that data could be compiled across students within settings, within categories of disability, and across settings and categories. In all the research was conducted with 26 focus students in 24 classrooms across 23 schools distributed among 18 Danish municipalities.

Page 4: Meeting the Challenges of Multi-Site, Multi …pure.au.dk/portal/files/41399219/Meeting_the_Challeng… · Web viewMeeting the Challenges of Multi-Site, Multi-Researcher Interpretivist

The Research Team The “inner circle” of the research team consisted of five members who were those most centrally involved and responsible for all aspects of the research process throughout the three years of the project. This core team was supported by four masters’ students completed their own research projects as part of the overall project and four research assistants transcribed all 80 semi-structured interviews. In addition, several different international research colleagues (including Ferguson) joined the research process at various points – primarily during team seminars – where they research was reviewed, adjustments were made in the methodology, and the analysis was advanced.

Four of the core research team members are former teachers, two with a master’s degree in special education, and two others with Ph.D. degrees. The fifth is currently completing her doctoral studies within the project. While the group has a range of experience with research – mostly earlier qualitative research – all have substantial experience in teaching and classroom research strategies. The core team also brings to the research effort specific expertise in the specific disabilities included, offering yet one more dimension of expertise to enhance the team effort.

Developing the Semi-Structured Observation and Other Tools The focus of this paper is the need for a semi-structured approach to observation in multi-site, multi-researcher qualitative projects. Developing a shared semi-structured interview guide has been explored (e.g., cites). But the process of participant observation maintain the trappings of the lone researcher: the researcher is the instrument who enters the setting and “records” as much as possible of what they see, hear, smell, and feel into fieldnotes. Fieldnotes are the core data for much interpretivist research. Yet, individuals see, hear, smell, and feel different things. While all information/data are valuable in understanding a phenomenon, the issue for team qualitative research is that in order to compile information across sites and researchers, it is necessary to have each researcher observe at least some of the same things. In earlier projects, for example, Ferguson has used simple observation guides that asked team researchers to notice certain aspects of classrooms and classroom practices. These prompts were included on a “semi-structured interview guide” and all researchers included this information in their field notes under the agreed upon headings.

Through previous collaborations on other projects, the principle investigator (Tetler) knew of tools previously developed by Ferguson and her collaborators that might be a useful starting point. One of these was the Student Membership Snapshot (SMS) (Rivers, Ferguson, Lester & Droge, 1996). The development of this observation tool was prompted by the increasing inclusion of students with disabilities in general education classrooms. It was designed to be used by a range of individuals who enter classrooms to provide support to classroom teachers. It is based on the premise that an observer might be able to see different aspects of the situation at hand (Schön, 1983) and in doing so generate information that the teacher could use to better understand the classroom problems faced and ways to solve them. It was designed essentially to be a tool that aids problem finding, problem solving, and capacity building of teachers to address classroom problems and improve their own practice. In the original field test of this tool (Rivers, 1993) and in subsequent field use, the observation tool has been used effectively by special educators, administrators, school psychologists, and a range of consultants who have as their focus responding to problems defined by teachers in their classrooms.

During year 1 of this project the core research team translated the SMS into Danish and adapted it somewhat for use in a Danish school context. Each member of the research team then used the SMS in each of their school settings. In addition they completed fieldnotes for each visit, and interviewed

Page 5: Meeting the Challenges of Multi-Site, Multi …pure.au.dk/portal/files/41399219/Meeting_the_Challeng… · Web viewMeeting the Challenges of Multi-Site, Multi-Researcher Interpretivist

teachers and family members. At the end of year 1, the core research team met with Ferguson to further practice and refine the SMS for research. This new version is included in Attachment 1.

As a research tool, the SMS allowed the observers to not only collect similar data that could be compiled across students, classrooms, researchers and sites, but it also allowed the team to collect information that focused on certain aspects of the teaching and learning processes in the classrooms. In discussions during regular meetings, team members reported that once out of the field they were often able to find things in the observation data that was not obvious to them during the actual observation. For example, recording what the whole class was doing as well as the focus student allowed researchers to later examine the similarity of difference of teaching and learning engagements between the focus student and others in the class across all parts of the lesson observed.

One of the issues that became clear for us during the first year of research was whether or not the students with disabilities were included on their own terms – that is, the teacher was able to incorporate them and meet their needs as part of the group – or was the student expected to do the work of fitting in? We also wanted explore how this looked across the settings. A central question, then, was, whether the position for a focus student could be defined as educationally included in a regular setting or just physically included? At its core, this is a question about whether a focus students included is taught parallel to the rest of his/her class mates – together with his/her teacher assistant (with an individualized curriculum), or whether a focus student has become an active participant in class room activities. Research has shown that these premises for educational inclusion very much depend on the teachers’ abilities to organize learning activities in a way that gives those students possibilities to participate on their own terms. It was this central question that the SMS helped us explore. Compiling all the observations across settings, this pattern showed up in Figure 1 and Figure 2.

<Insert Figures 1 and 2 about here>

The typical staffing in inclusive settings is the presence of two adults in the classroom (usually a teacher and a teacher assistant), which permits a more flexible organization of class room activities. Yes, it was surprising from our data how little the teaching activities actually were differentiated. Instead, the predominant pattern was the focus students sitting with the same materials as their class mates, working in the same ways and with the same goals and same expectations on ‘standard outcomes’. When it is possible to identify differences between student activities, it takes form of an individual parallel activity, often together with a teacher assistant.

Another issue we wished to explore was with whom an included focus student tends to interact. An important aspect of schooling and learning theory stresses the value of social as well as academic learning for all members of the learning community. It was important, therefore, for us to explore whether focus students’ interactions were predominantly with adults (e.g. teachers, teacher assistants) or with peers. Analysis of data from the SMS across all settings showed that even in the inclusive settings, focus students tended to interact more with adults than peers. And this was further confirmed by the structured student interviews with focus students and their classmates (Tetler & Baltzer, 2009). This pattern of student-adult interaction, which compromises social learning with peers, was even more extreme in the segregated settings as seen in Figures 3 and 4. As the figure shows, included focus students are given more possibilities to interact with their class mates -- one third of the teaching time. On the other hand, they interact very little with their teachers; it seems as the teaching responsibility for those students has been delegated to the teacher assistant.

<Insert Figures 3 and 4 about here>

Page 6: Meeting the Challenges of Multi-Site, Multi …pure.au.dk/portal/files/41399219/Meeting_the_Challeng… · Web viewMeeting the Challenges of Multi-Site, Multi-Researcher Interpretivist

The Research Process Throughout the project the research team met regularly. It was critical to the management of a multi-researcher, multi-site research effort that decisions about the progress of the research and the meaning of the emerging data were shared and reflected the consensus of the team. Typically the team met for several days approximately four times each year. These more intensive meetings were supplemented by regular phone and email conversations among the researchers.

Analyses proceeded through three stages. During the first stage all data from the Student Membership Snapshot (SMS) tool were compiled across all 26 participants and disaggregated by setting (integrated and segregated). These same data were then disaggregated by disability category and again compared across settings.

The second stage of analysis first engaged all the researchers in discussion of what they were learning from the SMS observations. These discussions resulted in identification of 10 “themes” or topics for each researcher to reflect upon as they reviewed all the data in the setting in order to prepare comprehensive student profiles for each participant. Some of the themes articulated, for example, ways in which such factors as teacher personality, classroom management, and theories of education influenced the students with special needs and their educational experiences. Other “themes” asked researchers to explore the use of differentiated instruction and the impact on students with special needs, or how local norms and values related to difference affected experiences of the focus students. By together discussing such themes, individual researchers together created a common understanding of what might be useful to explore through analysis of the data. In this way analysis relied on using the person closest to the data as the initial interpreter of the data. Each researcher completed individual case studies on each of the focus students that pulled together all of the data from the setting and included:

Teacher generated reports of student status and other planning and reporting documents specific to each participating student;

SMS observations Fieldnotes prepared for each SMS observation visit Structured interviews of each focus student Semi-structured interviews with each student’s teacher Semi-structured interviews with each students’ parents Teacher logs of experiences with the focus student

Table 1 includes the template the research team created together to guide the development of the individual student profiles. It was important for this intermediate stage of analysis that all of the profiles included similar information in order to make it possible to complete later analyses. At the same time, it was important to the team that the unique information each researcher brought to the task of both data collection and initial analysis was captured in these profiles.

Table 1: Student Profile Template

Student Overview – Facts & Figures

This section describes the student, their disability label, educational placement and grade level, family composition, and other pertinent “facts.”

Individual Student Planning & Achievement

A document analysis of available reports of student status, individual education plans, relationship of these individual plans to annual school plans, and any testing or other reports related to student progress/achievement.

Page 7: Meeting the Challenges of Multi-Site, Multi …pure.au.dk/portal/files/41399219/Meeting_the_Challeng… · Web viewMeeting the Challenges of Multi-Site, Multi-Researcher Interpretivist

Fieldnotes and SMS Comments Synthesis

A “fleshed out” synthesis of fieldnotes collected during each SMS observation with accounts that might serve as illustrative “vignettes” highlighted.

Analysis of Focus Student Structured Interviews

A summary and analysis of focus student structured interviews that identifies possible questions and confirmations across SMS observations and students’ personal perspectives.

Analysis Summary of Teacher Interviews and Diaries/Logs

Analysis and summary will highlight areas where teachers’ perspectives both confirm or disconfirm data from SMS observations and focus student perspectives. Any particularly revealing comments by teachers that might illustrate or elaborate results compiled from SMS observations should be highlighted.

Analysis Summary of Parent Interviews

This analysis/summary should, again, highlight any commonalities as well as differences of perspective when compared to the focus student and the teachers. In addition, any data that relates to family’s long term goals, expectations, concerns should be highlighted.

Analysis Summary of Peer Surveys

Compilation of peer surveys by student should also be reviewed for confirmations and differences with the perspectives of the focus student, families and teachers.

The third stage of analysis involved more in-depth analysis first within and then across categories. Again the team met and, through discussion of the individual profiles, developed an outline for the within-category case accounts. This time, the outline asked for each of the researchers assigned to a group of students with a specific category to first provide an overview of 1) all the classrooms and activities observed using the SMS; 2) family perspectives on the schooling experiences of their children, and 3) any information gained from students, teachers and family members about how the school situation was working for the student.

The cross case profile then described the overall dimensions of learning across all the settings providing examples from fieldnotes that included 1) the continuity and coherence of the students’ experiences, 2) support, resources, technology provided to the students, and 3) how the individual plans and reports gathered and analyzed were used by teachers and other adults. The profile then concluded with a summary description of 1) the experiences of the students in terms of such dimensions as membership, relationships, and participation; 2) experiences of families and teachers and how they feel about the students, and 3) any other information from all of the available data and the individual profiles that struck the researcher as interesting. This last category proved useful to the team because it allowed each researcher to identify aspects of their data that, while it might fall outside of the overall analysis, provided a challenge that served to deepen discussion and further refine that analysis.

The final report (Egelund & Tetler, 2009) provides a more comprehensive description of the cross category analysis, but for the purposes of this paper it is important to note that this cross category analysis occurred throughout the study. Data from the SMS observations, for example, were first compiled across all students and categories and disaggregated by settings (inclusive, segregated) and by category. Similarly, data from student interviews/surveys and teacher and family interviews were also compiled and disaggregated by these same dimensions. In this way, the team was able to focus on the “big picture” of what was emerging from the study as well as focus on each researcher’s individual students and schools. This tacking back and forth during the analysis between the overall study and the results emerging from individual schools and categories of disabilities allowed.

Page 8: Meeting the Challenges of Multi-Site, Multi …pure.au.dk/portal/files/41399219/Meeting_the_Challeng… · Web viewMeeting the Challenges of Multi-Site, Multi-Researcher Interpretivist

Lessons Learned in This Team Research Much was learned during this research effort about educational experiences of students with disabilities in Denmark from the perspectives and voices of the students themselves, their classmates, their families, and their teachers. These are reported elsewhere (Egelund & Tetler, 2009) and will likely be discussed in Denmark for some time to come. Here, however, we would like to reflect on the processes of team research and what we learned from the experience. There were challenges, to be sure; but also benefits.

Perhaps the largest benefit was team learning across a number of dimensions. First, the SMS semi-structured tool turned out to be a learning experience for everyone involved. It was certainly a different approach, and as emerged in team discussions, it was hard to learn to use at first. Yet most of the researchers found, in the end, that the tool did generate information that they had not expected. Not only did it meet the initially stated need to have some observation data that could be compiled across sites and researchers, but more importantly, using such a tool as a part of the typical qualitative observation also collected data that could be “mined” later to discover aspects of the teaching/learning situation that were not even noticed at first and were not recorded in fieldnotes.

An important focus of this project was to explore differences in pedagogy across settings and students. The SMS tool allowed the team to gather some fairly focused information that through our team seminars and meetings we were able to explore together. This collective analysis helped us to see aspects of the teaching/learning situation that might not have been surfaced through more traditional observation and fieldnotes alone – in part because each researcher takes a slightly different approach to observations: seeing different aspects of the situation and noting different things in their fieldnotes. We learned, for example, that in some settings students with disabilities are able to become members of the learning enterprise while in others they are more individual learners who rely less on a community of learners, and more on direct interaction with a teacher. The SMS was, we learned, not as helpful in generating useful information in more segregated settings. This occurred in part because these settings separated each student into a learning “island” that did not need to connect with other students’ “islands”.

Second, we all learned as researchers in this project. Our frequent meetings and multi-day seminars created an important space for each of us to learn from others on the team. Each of us brought different amounts of skill and experience doing research to the project. For those with less experience, the participation in group discussion grew their capacity as researchers. For those with more experience, the opportunity to discover the perspective and voice of colleagues more grounded in day-to-day practice expanded their capacity to be more inclusive and participatory in research. Each of us was able to learn from each others’ data and approach to, and experience with, gathering qualitative data. Each of us was also challenged by our colleagues in our interpretations, our analyses, and our ideas about what we were exploring. Such challenges led not only to broadening all of our understanding, but also to richly faceted discussions and analyses that could only occur as a collective effort, or “working in the interpretive zone” as a group (Wasser & Bresler, 1996).

Third, team research is certainly more time-consuming in some ways, and also more expensive to support. We found it critical to be able to have frequent meetings – in person, one the phone, and through email and internet – in order to ensure that everyone of the team was a fully participating member of the work. At the same time, having a team allowed a great deal of flexibility for each individual researcher and for the project director in organizing the research tasks and getting everything accomplished. While we needed to collectively negotiate some things such as deadlines for the different

Page 9: Meeting the Challenges of Multi-Site, Multi …pure.au.dk/portal/files/41399219/Meeting_the_Challeng… · Web viewMeeting the Challenges of Multi-Site, Multi-Researcher Interpretivist

stages of analysis (data collection, transcription, individual profiles, within category profiles, etc.) we were also able to have the freedom as team members to plan our own individual tasks, collaborate with one of the others in ad hoc ways, or in just improvising aspects of the work which could later be shared with the group for the benefit of everyone.

Fourth, because different members of our team had different levels of experience with qualitative research, there was some variation across our data. For example, some of the interviews reflected the interests of the individual interviewer more than that of the overall project. Some researchers were more skilled at interviews than others so that their transcripts might be richer in information. Nevertheless, the use of an interview guide provided enough continuity to make the cross case analyses possible. The differences across researchers were sometimes interesting to the whole project which provided yet another source of challenge that aided the overall process of learning and interpretation.

Similarly, the use of the SMS and the fieldnotes were somewhat varied across the researchers. Since the SMS was semi-structured, the variation was less than across the fieldnotes, but even in that case, some researchers gained comfort with the tool more quickly than others. Sometimes the data collected was hard to decipher and interpret even for the person who collected it. Nevertheless, we felt, in the end, that the use of the SMS profited the research by achieving the purpose of allowing us to gather similar, and compliable information across all settings.

In the end we concluded that the benefits of team research outweighed the challenges. We learned much and grew as researchers. While the use of semi-structured interview guides is increasingly common in team research, we have not found any references to the use of semi-structured observation guides. The “gold standard” of careful and comprehensive fieldnotes has long been the norm in qualitative research and that works admirably for the lone researcher. But the demands and needs of team research require the same kind of commonality across researchers for observations as the semi-structured observation has achieved for interviews.

Of course, observations are different and the SMS as an example of a semi-structured observation tool is fairly complex and detailed. In part, this detail is due to the focus of the project on aspects of learning and pedagogy for students with disabilities and not all such guides would necessarily be that detailed. One of us (Ferguson) has used much simpler semi-structured observation guides that are modeled more on the familiar interview counterpart. That is, sometimes an outline is provided to team members to make sure that they notice certain things and that their resulting fieldnotes address the information that the team has determined to be important to the overall project. What our experience with other semi-structured observation tools has not produced, however, that the SMS did, is information that was rich enough to very deeply analyze later that offered an expansion of what the researcher observed and experienced in the setting originally. We do not believe such data would have been included in the most complete fieldnotes, and certainly in a team situation when researcher skills and interests vary. W

In the end, we encourage team research because we believe our field needs it. We also support the development of research tools and processes that support that research. Semi-structured interview guides are part of that agenda and we offer our small experience to further it.

ReferencesAnderson, G.L., Herr, K.G. & Nihlen, A.S. (1994). Studying your own school: An educator’s guide to

qualitative practitioner research. CA: Corwin Press.

Page 10: Meeting the Challenges of Multi-Site, Multi …pure.au.dk/portal/files/41399219/Meeting_the_Challeng… · Web viewMeeting the Challenges of Multi-Site, Multi-Researcher Interpretivist

Bogdan, R.C. & Biklen, S.K. (2007). Qualitative research for education: An introduction to theories and methods. (5th ed.) Boston: Pearson Education, Inc.

Eisner, E. (1991). The enlightened eye: Qualitative inquiry and the enhancement of educational practice. New York: Macmillan.

Davis, 1995, L.J. (Ed.). (2006). The disability studies reader (2nd ed.). New York: Taylor and Francis.

Dewey, J. (1938). Experience & education. New York: Macmillan.

Edgerton, R.B. (1967). The cloak of competence. Berkeley: University of California Press.

Egelund, N. & Tetler, S (eds.) (2009). Effekter af specialundervisningen. Copenhagen, Danish School of Education Press.

Ferguson, D. L. (1987). Curriculum decision-making for students with severe handicaps: Policy and practice. New York: Teachers College Press.

Ferguson, D. L. (Ed.). (1998). Crossing borders: Learning from inclusion & restructuring research in Sweden, Denmark, Norway and the U. S. International Journal of Educational Research, 29 (2).

Goode, D. (1994). A world without words: The social construction of children born deaf and blind. Philadelphia, PA: Temple University Press.

Groce, N.E. (1985). Everyone here spoke sign language: Hereditary deafness on Martha’s Vineyard. Cambridge, MA: Harvard University Press.

Guest, G. & MacQueen, K.M. (Eds.) (2008). Handbook for team-based qualitative research. New York: Altamira Press.

Harry, B. (1992). Cultural diversity, families, and the special education system. New York: Teachers College Press.

Janko, S. (1993). Vulnerable children, vulnerable families: The social construction of child abuse. New York: Teachers College Press.

Kinzie, J., Magolda, P., Kezar, A., Kuh, G., Hinkle, S. & Whitt, E. (2007). Methodological challenges in multi-investigator multi-institutional research in higher education. Higher Education, 54, 469-482.

Lincoln, Y.S. & Guba, E.G. (1985). Naturalistic inquiry. Thousand Oaks, CA: Sage Publications.

Marshall, C., & Rossman, .B. (2006). Designing qualitative research (4th ed.). Thousand Oaks, CA: Sage Publications.

Page 11: Meeting the Challenges of Multi-Site, Multi …pure.au.dk/portal/files/41399219/Meeting_the_Challeng… · Web viewMeeting the Challenges of Multi-Site, Multi-Researcher Interpretivist

Maxwell, J. (2005). Qualitative research design: An interactive approach (2nd ed.). Thousand Oaks, CA: Sage Publications.

Minow, M. (1990). Making all the difference: Inclusion, exclusion, and American law. Ithaca, NY: Cornell University Press.

Naraian, S. (2008). Institutional stories and self-stories: Investigating peer interpretations of significant disability. International Journal of Inclusive Education, 12, 525-542.

Patton, M.Q. (1990). Qualitative evaluation and research methods (2nd ed.). Newbury Park, CA: Sage Publications.

Rist, R. C. (1980). Blitzkrief ethnography: On the transformation of a method into a movement. Educational Researcher, 9(2), 8-10.

Rivers, E.S. (1993). Moments of meaningfulness: A case study of student membership. New Orleans, LA: University of New Orleans doctoral dissertation.

Rivers, E. S., Ferguson, D. L., Lester, J., & Droege, C. A. (1996). Student membership snapshots: An ongoing problem-finding and problem-solving strategy. Module 4E. Eugene: University of Oregon, Specialized Training Program

Schön, D.A. (1983). The reflective practitioner. New York: Basic Books.

Tetler, S. & Baltzer, K. (2009). Educational participation: The issue of students’ voices. Presentation at the American Educational Research Association Annual Conference.

VanMaanen, J. (1988). Tales of the field: On writing ethnography. Chicago: University of Chicago Press.

Wasser, J.D. & Bresler, L. (1996). Working in the interpretive zone: Conceptualizing collaboration in qualitative research teams. Educational Researcher, 25(5), 5-15.

Whitt, E.J. & Kuh, G.D. (1991). Qualitatiave research in higher education: A team approach to multiple site investigation. Review of Higher Education, 14, 317-337.

Page 12: Meeting the Challenges of Multi-Site, Multi …pure.au.dk/portal/files/41399219/Meeting_the_Challeng… · Web viewMeeting the Challenges of Multi-Site, Multi-Researcher Interpretivist
Page 13: Meeting the Challenges of Multi-Site, Multi …pure.au.dk/portal/files/41399219/Meeting_the_Challeng… · Web viewMeeting the Challenges of Multi-Site, Multi-Researcher Interpretivist
Page 14: Meeting the Challenges of Multi-Site, Multi …pure.au.dk/portal/files/41399219/Meeting_the_Challeng… · Web viewMeeting the Challenges of Multi-Site, Multi-Researcher Interpretivist

Attachment 1

The Student Membership Snapshot Observation Tool – Research Version

Page 15: Meeting the Challenges of Multi-Site, Multi …pure.au.dk/portal/files/41399219/Meeting_the_Challeng… · Web viewMeeting the Challenges of Multi-Site, Multi-Researcher Interpretivist
Page 16: Meeting the Challenges of Multi-Site, Multi …pure.au.dk/portal/files/41399219/Meeting_the_Challeng… · Web viewMeeting the Challenges of Multi-Site, Multi-Researcher Interpretivist

Student:   Class:   Category:                        CP Dyslexia     Learn. Diff.     ADHD   Autisme   Blind  

Date:                                  Observer: Time: Otherwise::

5 minutes intervallSegment     1.

int.

2.

int.

3.

int.

4 in

t.

5.

int.

6. in

t

7.

int.

8.

int.

9.

int.

10.

int.

11.

int.

   What's Grouping Whole Class                         Comments

>5                           3-5                          pairs                          

going on 1.1/Independent                         CommentsTeaching guided teaching                        

lead/demonstrate                        ask/answer                        

generally? lecture/tell                        observe                          

Peer active/match                         CommentsComparison passive/match                        

active/not match                        passive/not match                        unengaged/disruptive                      engaged/disruptive                      

What's Fokus student active/match                         Commentsaction passive/match                        

active/not match                        s(he)'s passive/not match                        

unengaged/disruptive                      engaged/disruptive                      

doing                            Learning identical                           Comments

different content/materials                      different teaching                        different outcomes                      parallel activity                        

Grouping whole class                         Comments >5                          

Page 17: Meeting the Challenges of Multi-Site, Multi …pure.au.dk/portal/files/41399219/Meeting_the_Challeng… · Web viewMeeting the Challenges of Multi-Site, Multi-Researcher Interpretivist

3-5                          pairs                          1:1/Independent                        

Who Teacher cooperative groups                       Commentsinteractions independent                        

s(he) peer tutor                        teacher/assistent                        

doing 1:1/support specialist                      S-Adult teacher/natural/objective                       Comments

thins with Interactions support staff/natural                      teacher/artificial                        support staff/artificial                      none                          

S-Peer peer/natural                         Commentsinteractions peer/artificial                        

none                          Other things Setting mood discussion/ask/answer                       Comments

action/quiet                        going on mixed                          

seated/discussing/talking                      generally seated/quiet                        

on the move/asking/discuss