TOPIC 7ACTION RESEARCHData Collection Methods

SYNOPSISTopic 7 has been designed to explore the data collection methods that can typically be used during an action research project. Participants will be expected to evaluate and consider how to employ data collection methods effectively in their action research investigation.

LEARNING OUTCOMES By the end of this topic, you will be able to: Identify the types of data collection instruments that can be used in action research Understand strengths and limitations of data collection instruments in action research Become informed about how to design and create effective data collection instruments

fRAMEWORK OF TOPICS

Preview (30 min) Work with a partner fill in table one Discuss table one with another group Discuss table one as a whole class You may compare answers with the Data Collection Methods example

Table 1: Data collection methodsMethodAdvantageDisadvantageWhat I want to know/learn about this method

Survey

Interview

Document Analysis

Observation

Data Collection MethodsMethodAdvantagesDisadvantages

Interviews

-Flexibility as the researcher can observe the participant and total situation in which they are responding.

-More participants are willing to talk and react verbally than to write responses to questions.

-Face-to-face interviews establish rapport and motivation for participants.

-Useful method to gather extensive data on a small number of complex topics.

-The researcher can control the sequence of questions so the respondent cannot predict the next question.

-More expensive and time consuming than surveys.

-Only a limited number of participants may be interviewed due to time and financial considerations.

-Finding skilled and trained interviewers with appropriate interpersonal skills may be difficult.

-An interviewer effect may result from interaction between the interviewer and participant.

-Participants may feel that they must answer on-the- spot.

-Flexibility afforded by unstructured interviews may generate difficulties when attempts are made to categorise and evaluate responses.

Document Analysis

-More accurate and reliable data than a survey.

-Useful when the researcher is specifically interested in precise estimates of different kinds of behaviour.-Participants may become less proficient over time in their record keeping.

-Cognitive dissonance influences a participants recollection of events and can lead to framing of responses.

Observations

-Can record both simple and complex behaviour.

-Can begin to theorize and draw abstractions to validate observations.

-Researcher may use a tally sheet to categorize reoccurring behaviour according to criteria sheet.

-Cost effective and easy to implement.-Highly subjective.

-Not all observations are equal or expert.

-Difficult to evaluate the veracity of the observation.

-Variables are ever present and that means the observed behaviour may be impossible to replicate.

-Observations can easily become dominated by researcher bias.

-The researcher decides what phenomena to observe and what to ignore.

Survey -Cost effective and easy to implement

-Useful to collect data on specific populations.

-Tends to give respondent the privacy to answer truthfully.

-Standardized questions make evaluation of answers more precise by forcing uniform definitions on participants.

-Researcher may compare responses to search for emergent trends across participants.

-Researcher may use the survey to collect data remotely by mail, internet or email.

-Respondents can remain anonymous. -The researcher questions may contain bias and can potentially lead the participant to desired answers.

-Participants may not tell the truth over controversial question or question that cast the participant in an unfavourable light.

-Not all surveys are equal or expert.

-Very little control over the context the survey is taken. The lack of context can make it difficult to ascertain the reasoning behind participant responses.

-Open-ended questions generate large amounts of data that are time consuming to analyse.

-Participants may answer superficially to complete the task.

OBSERVATION Observation is the systematic description of events, behaviours, and artefacts in the social setting chosen for study. Observations enable the researcher to describe existing situations using the five senses, providing a "written photograph" of the situation under study. Fieldwork involves active looking, improving memory, informal interviewing, writing detailed field notes, and perhaps most importantly, patience. Observation methods are useful to researchers in a variety of ways. They provide researchers with ways to check for nonverbal expression of feelings, determine who interacts with whom, grasp how participants communicate with each other, and check for how much time is spent on various activities. Observation is the technique of obtaining data through direct contact with a persons or group of persons.[footnoteRef:1] Since, the main focus of qualitative research is naturalism, the researcher has to observe person or persons in their natural state as undisturbed as possible. The role of the researcher may be viewed as a continuum (se Figure 2.1). On one extreme, the researcher is a passive observer and on the other extreme the researcher is a participant observer. In between these two extremes, the researcher may be an active observer (Potter, 1996). [1: www.peoplelearn.homestead.com/Chapter2.Methods.QR.doc]

Passive Observer Active Observer Participant Observer Figure 2.1 Continuum of Observation Techniques

Passive observer: The best way to be not involved and keep distance from your subjects is to be a passive observer. As a passive observer, you simply gather documents and observe the individual or individuals without doing anything to disturb the situation. The researcher is unobtrusive and watches the group from the outside; i.e. the ethic or outsiders perspective. To do so, the researcher must gain access and be accepted by the individual or individuals being observed. For example, in collecting e-mails or essays written by subjects or learning journals of students, the researcher examines them without being involved. Similarly, when a researcher interested in studying children interacting in school canteens or the playground, merely observes them without being involved. A certain amount of distance is maintained between the researcher and the person or persons being observed.

Participant Observation: As the name participant suggests, the researcher participates in the activities of the persons being observed rather than being an observer. The researcher has two role as observer and as participant. The researcher participates as much as possible in the daily life of the subjects while also carefully observing everything he or she can about it. Through this, the researcher is seeking to gain what is called an emic perspective or the natives point of view or the insiders perspective. The researcher records detailed field notes, conduct interview based on open-ended questions and gather whatever site documents might be available in the setting as data. Participation can take many forms. For example, the researcher could show a film or video to stimulate discussion or question subjects and observe how they would react to the stimulus. The researcher takes an active position with the purpose of stimulating subjects to think about things they might never have thought about before. However, as pointed out by Hammersley and Atkinson (1983), there is the danger of the researcher going native which means being too involved or having too close a rapport with the person or persons being observed to the extent that you loose objectivity.Participant observation Observer participates actively, for an extended period of time May require observer to live or work in that area Assumes that observer will become accepted member of the group or community

Almost any setting in which people have complex interactions with each other, with objects, or with their physical environment can be usefully examined through participant observation. Since doing participant observation means being embedded in the action and context of a social setting (going native), we consider three key elements of a participant observation study:

1. Getting into the location of whatever aspect of the human experience you wish to study. This means going to where the action ispeoples communities, homes, workplaces, recreational sites, places of commercial interaction, sacred sites, and the like. Participant observation is almost always conducted in situ.

2. Building rapport with the participants. The point of participant observation is that you wish to observe and learn about the things people do in the normal course of their lives. That means they have to accept you, to some extent, as someone they can be themselves in front of. While you dont necessarily have to be viewed as a complete insider, a successful participant observer has to inspire enough trust and acceptance to enable her research participants to act much as they would if the researcher were not present.

Active Observer: Between being a passive observer and an active participant, the researcher could take a middle position of being an active observer. Here, participation is allowed but limited. The researcher may intrude into the lives of subjects such as entering their homes or their communities but remains passive once inside the environment so as not to influence the natural occurring behaviours and conversations. For example, a researcher interested in TV viewing habits may enter a household, eat with the family, play with the children and take part in family activities. Family members are told not to change their routines in order to accommodate the observer. However, the researcher tries as far as possible to be passive, saying as little as possible so as not to influence the behaviours and conversations of subjects.

Observer An observer is under the bed. A participant observer is in it. John Whiting Observer is an eavesdropper Someone who attempts to observe people without interacting with them With or without their knowledge that they are being observed[footnoteRef:2] [2: http://www.sagepub.com/upm-data/48454_ch_3.pdf SAGE chapter 3 December 2012.]

Direct observation is primarily a quantitative technique in which the observer is explicitly counting the frequency and/or intensity of specific behaviors or events or mapping the social composition and action of a particular scene. While most direct observation data collection is conducted by actual observers, many direct observation studies do not technically require a human data collector. The data captured in direct observation are, by definition, those that can be observed and do not inherently require any interaction (talk or discussion) between the observer and those being studied. Direct observation is about observable behavior and is typically associated with research objectives that require some sort of ordinal data or purely factual description: how often, how many, how intensely, who was there, and the like. As such, direct observation is normally a fairly structured form of data collection.

ParticipantsParticipants are a group that has been identified and isolated from the general population for research study. Since the focus in action research is usually on the teachers classroom, participants usually involve the teachers students in a classroom where the improvement and action research intervention will make the biggest difference to how teachers teach and how students learn.

DOCUMENT ANALYSIS[footnoteRef:3] [3: www.peoplelearn.homestead.com/Chapter2.Methods.QR.doc downloaded from the world wide web December 2012.]

If the focus of your study is the examination of documents, than you should have access to such material which may include letters, memos, notes, diaries, photographs, audiotapes, videotapes, films, articles, books, manuscripts, e-mails, online discussions and so forth. In general documents are any preserved recording of a persons thoughts, actions or creations (Potter, 1996). The examination of documents is especially important to historians who investigate patterns and trends from the past. Documents may be examined to investigate patterns and trends of the past as is commonly done by historians. If no humans remain alive to provide primary evidence, then documents are the in only source of data (Potter, 1996). Documents are also examined by researchers who are investigating subjects who are available. The examination of documents may also provide confirmatory evidence of the information obtained from interviews and observations.

Document for analysis student achievement data diagnostic assessment samples of student work attendance records anecdotal records files behavior records literature review web research journal lesson plans portfolio activity reports pre-test and post-test self-evaluation standardized test scores ______________________ ______________________ ______________________ ______________________ ______________________ ______________________

CHECKLIST

The checklist technique is commonly used in qualitative research to denote the frequency of a particular behavior that is related to the research question. Checklists are a popular data collection technique because it can be applied in a wide variety of research situation. In other words, we encourage participants to investigate and consult with a supervisor about how to create their own checklist as a research collection instrument to investigate action research.

Table 1: Checklist exampleIf my action research issue is gender question and answering in the English classroom, I can create a checklist to denote where and who answers the majority of questions in the classroom: boys or girls. This information may also shed light on where the teacher (me) is directing his or her questions towards most often (boys or girls). Second, the checklist as a data collection instrument can also identify the type of question asked OPEN ended (requiring a detailed response) or CLOSED (yes or no answer) to boys or girls in the classroom.

Lets practice[footnoteRef:4] [4: Instructor can hand this out at the start of lecture or tutorial and ask the students to fill it out as we go through class to monitor where questions are asked. Then at the end of the lecture, discuss findings of students checklist and identify where questions are asked in class and what this means.]

Read over Table 2 carefully Next, practice using this checklist in our class (right now) Use the checklist below to observe and see where questions are asked in our TSL 3113 class. Make sure you keep notes on what you have observed as that will be used for discussion at the end of class

TABLE 1: Teacher questioning checklistDate: Start TIME: Classroom: TSL 3113 Class duration: Number of participants: Number of girls: Number of boys: Observers Name:

Objective: This checklist is designed to collect information on:1. Where the teacher directed his/her questions in class 2. Responses according to gender 3. Locate the type of question (Open or Closed) that was asked according to gender.

In class, the peer observer will indicate in writing using the criteria below to denote the appropriate student responses as you observe the English lesson. For example F/O means girl open ended question and B/C = boy closed question.

Criteria

B/G; Boy/Girl Open Question (O) Closed question (C)

CLASSROOM Questioning Checklist

My anecdotal notes (what you see):

Observer reflection (should have insight and reflection)

Running Records[footnoteRef:5] [5: http://www.decd.sa.gov.au/northernadelaide/files/links/DECD_Running_Records_1_v8.pdf]

The most powerful teaching in classrooms takes place when teachers use the information gained from observations and assessments of childrens literacy development to plan their teaching. However teachers recognise that observations can sometimes be subjective and influenced by what they already know or believe about the students and their literacy development. Therefore it is important to add data from more formal assessments to their observations.

Tests of alphabet knowledge, phonics, phonemic awareness, and sight words form part of reading assessment but they dont provide the whole picture of how a student approaches the reading process. Gathering information from a Running Record, which gives a reliable and valid assessment of text reading, and adding this information to other assessments enables a teacher to gain a richer and more comprehensive assessment of a students reading ability.

How to take a Running Record1. Select a text at the students appropriate reading level and one that has been previously introduced and read by the student.2. Invite the student to read the text.3. As the student reads, use the conventions to note the students reading behaviours.4. Score and analyse the Running Record.5. Use this information to plan for future teaching.

INTERVIEWS

Structured interviews Interviewing is a technique of gathering data from humans by asking them questions and getting them to react verbally. There are many different ways of conducting interviews (see Figure 2.2). Structured interviews use an interview schedule that is similar to the survey questionnaire. You could phrase the question in such a way that so that you have a limited range of responses. For example, Do you think the image of teachers in society has gone down? Strongly agree, agree, somewhat agree, agree and strongly agree. Structured interviews are widely used in surveying opinions, beliefs and perceptions of people. Individual interviews are expensive and you should consider whether the same amount of data can be more efficiently collected using written questionnaires.

Semi-structured interviews Semi-structured interviews and unstructured interviews are widely used in qualitative research. Semi-structured interviews consist of a list of open-ended questions based on the topic areas the researcher intends to study. The open-ended nature of the questions provides opportunities for both the interviewer and interviewee to discuss certain topics in more detail. If the interviewee has difficulty answering a question or hesitates, the interviewer will probe. Three types of probes commonly used by the interviewer are:a) Detail-oriented probe When did it happen to you? Who was with you?b) Elaboration probe Tell me more about the incident. Can you give an example.c) Clarification probe Im not sure I understand what you mean by hanging out. Can you explain? You said that your principal is extremely autocratic.

What do you mean by autocratic?

Unstructured interviewsUnstructured interviews aim to obtain in depth interviews of persons interviewed. Only a limited number of topics are discussed, sometimes as few as one or two topics. Although only a few topics are discussed, they are covered in great detail. The interview may begin with a question such as Id like to hear your views of school discipline. Subsequent questions would follow from the interviewees responses. Unstructured interviews are used to find out about a specific topic but has no structure or preconceived plan or expectation as to how the interview will proceed.

Face-to-face or personal interviews are labour intensive but can be the best way of collecting high quality data, especially when the subject matter is very sensitive, if the questions are very complex or if the interview is likely to be lengthy (Mathers, Fox and Hunn, 2002).

ReferencesKawulich, Barbara. SAGE publication: Volume 6, No. 2, Art. 43 May 2005 Participant Observation as a Data Collection Method http://www.qualitativeresearch.net/index.php/fqs/article/view/466/996

Qualitative Data Collection Techniques Chapter 2: www.peoplelearn.homestead.com/Chapter2.Methods.QR.doc downloaded from the world wide web December 2012.

Southern Australia Department of Education, Adelaide. Running records downloaded from the world wide web, December 2012. http://www.decd.sa.gov.au/northernadelaide/files/links/DECD_Running_Records_1_v8.pdf.