Data Analysis

Types of Validity

Internal validitydegree to which results are true for the participants

External validitydegree to which results can be generalized beyond the participants in the study

Test validityDegree to which a test or assessment measures what it is supposed to measure

Validity ConsiderationsType of Validity Questions

Truth-value In what ways can I ensure my results are accurate and truthful?

Applicability/transferability How might my results be useful beyond my particular classroom and with other students?

Consistency/dependability Would I get the same results with other students similar to mine?

Neutrality/confirmability How do I show results are indicative of what actually occurred during the study and do not reflect my personal desires or biases?

Democratic validity Which stakeholders should collaborate or have a voice in the study? In what ways should I involve them?

Outcome validity How will I use results for continued planning, ongoing reflection, and deepening my personal understanding?

Process validity What do I need to do to ensure I have looked deeply at the problem so I understand the ways context and processes have impacted results?

Catalytic validity In what ways will processes and outcomes change my practice?

Dialogic validity Do I need a critical friend to discuss my study and results with so I can determine whether my planning and interpretations are on track?

Ways to Increase ValidityPeer debriefing

Discuss study with a colleague not invested in study to gain their insights

Persistent and prolonged observationsHelps to determine whether the intervention is effective after the newness wears off providing credibility

Record data accuratelyBe sure to include detailed notes to help answer questions

Member checksDiscuss your interpretations of data to determine whether your findings accurately represent the participants

Triangulation of dataThe use of multiple sources to corroborate findings

Researcher biasDefine any preconceived ideas you have and note them

Engage in ongoing reflective planningAllows a researcher to change data collection strategies based on experiences during the data collection phase.

Obtaining Permission

Will protect your participants, school and university

Parents will want to know if child is involved in action research

Always extend courtesy of obtaining permission from the participants in the study regardless of whether they are children or adults

Example of Consent Form

Dear Parents,

I will be conducting a study in our classroom to determine [include purpose here]. I am writing to ask permission to use the data I collect from your child during this process. Participation in this study involves only regular classroom activities. You may contact me at any time regarding your child’s participation. The principal of the school has approved this study.

The purpose of the study is to [briefly describe purpose here]. The study will take place at [name location] and will lastfor [give time frame]. During the study, I will collect various forms of data to determine whether [name of intervention] was successful. Possible types of data I will collect include [list data collection strategies such as samples of student work, survey/questionnaires, interviews, observations, test scores, etc.].

Benefits of participating in this study include [describe the benefits]. Only [name(s) of researcher and collaborators] will have access to the data collected in this study. Your child’s participation in this project is strictly confidential. Only [and names of collaborators and/or supervising professor if study is part of a university assignment] will have access to your child’s identity and to information that can be associated to your child’s identify. [If applicable, state when data or documentation will be destroyed].

Use of data from your child is voluntary. You may contact me at any time if you do not wish to have your child’s data included in this study. Please check the appropriate choice below, sign the form and return the form with your child.____ I give permission for my child’s data to be used in this study. I understand that I will receive a signed copy of this consent form. I have read this form and understand it.____ I do not give permission for my child’s data to be included in this project.

______________________ _____________________________ ________________ Student’s Name Signature of parent/guardian Date

Data Collection

Any form of information, observations, or facts that are collected or recorded

Should be collected systematically

Plan should be in place prior to collection

Describes what data will be collected

How, when and how often it will be collected

May want to use a calendar or checklist to record when and how data is collected

Types of Data Collection

Log or research journalNotebook used to record thoughts and observations related to all parts of research

Field notesWritten observations of what is taken place in the classroom

ChecklistsA list that specifies certain attributes such as behaviors, traits, assignments, or skillsChecked off when attribute is seen

InterviewsStudents respond to a specific set of questions that are prepared in advancedQuestions are asked in same manner and order each time to maintain consistency

Data retrieval chartsVisual organizer used to collect and organize data

Continued…

Student products or performancesA representative sample of each student’s work at different time periods to provide feel of student work

SurveysAllow you to get a variety of information quickly

Scores and other Quantifiable DataScores on tests, homework, grades

Attitude and rating scalesProvide respondents with a question or statement for which they are asked to select one of several answers that determines the strength of responseFive point rating scale

Example: 5= strongly agree to 1= strongly disagree

Archival dataPast scores or grades

Strategies for Data Analysis

Interim analysisOngoing analysis of the data throughout the study

Looking at and thinking about data as it is collected and then making changes or additions to strategies if necessary

Allows the researcher to make changes to data collection strategies during the study based on problems or questions that arise when analyzing the data

Analysis of Quantitative Data

Quantitative dataAny data that can be counted or averaged the number of responses for each item

Example: number of readers who indicated that they are strong, average, or weak readers

– Strong = 3, average=2, weak=1

– Provide each student with a score and then find the percentage of class that felt they were a 3,2, or 1

– Could average scores to find the mean score of the class to look at the average score of how students’ perceived their reading ability

Steps of Analysis for Quantitative Data

ReportingDisplaying the data and describing what it shows

ComparingLooking at various attributes and comparing to see if any differences appear

Example: Does gender make a difference in the scores? What does the data show?Example: Sample one as compared to sample two Is there a difference between the two samples? For example, did scores increase based upon what was implemented? (pre and post data)

Displaying dataUse charts and graphs to visual display data collectionMicrosoft – analyzing data using excel 2007 tutorial: www.microsoft.com/office/newday/default.mspx?nav+v1xcsubmenuxi6

Steps for Organizing and Analyzing Data

Gather together data sources and record informationRecord data in a grade book, journal or computer spreadsheet

Create graphical displays of dataDisplay data so that it is easy to understandCharts, tables, graphs

Examine displayed data to determine the best ways to explain results

Be sure to accurately describe data and data comparisonsShare your analyses and graphical displays with collaborators or peers

Colleagues can help you determine whether your analyses, charts, figures, graphs make sense

Make notes in your journal about how your analyses can be used to answer research questions

Refer often to your research questionMake sure to analyze data in ways to help answer the research question

Qualitative Data Analysis

The process of making meaning from data sources that can be interpreted in various ways

Requires a deeper analysis of data

Must first be converted to textLook for patterns or themes within the data

Steps for Analyzing Qualitative Data

Data must be written as textTextual data are studied for patterns to determine ways to code the dataTextual data and codes are analyzed so that patterns and themes can be developedAnswers to research questions are offered based on themes found in the data

Coding Examples (Bogdan and Bilken 2003)

Setting/contextProvides descriptive information on aspects of the research setting

Participants’ perspectivesDescribes what participants think about certain issues

Definition of situationIllustrates participants’ understanding of setting and context

Ways of thinkingDescribes participants’ understanding of self and/or others

ProcessesExplains patterns of behavior over time

ActivitiesIllustrates recurring, typical behaviors

EventsDescribes specific or particular nonrecurring, meaningful events

StrategiesDescribes methods used by participants to accomplish certain tasks

RelationshipsDefines social roles and typical behavioral patterns among or between people

Triangulating Data Sources and Drawing Conclusions from Data

TriangulationDetermining whether data sources corroborate with one another

Draw reasonable conclusions based on the interpretation of the data

Include the answers to the such questions as– Was the intervention effective? Why or why not?

– What would you do differently?

– You need to include future plans regarding how you intend to use the results and conclusions of the study to inform your practice as an educator.