From Evidence-based Practice to Practice-based Evidence: Behavior Analysis in Special Education...

From Evidence-based Practice to Practice-based Evidence: Behavior

Analysis in Special Education

Ronnie DetrichWing Institute

What is Evidence-based Practice?

• EBP is a decision-making approach that places emphasis on evidence to:guide decisions about which interventions to

use.evaluate the effects of an intervention.

Sackett et al (2000) defined evidence-based practice in medicine as: “the integration of best research evidence with clinical expertise, and patient values.”

What is Evidence-based Practice?

• Ultimately, EBP is a consumer protection issue.Assumes that evidence-based interventions

are more likely to be effective than interventions that are not evidence-based.

By validating interventions as evidence-based there is the implication that there are standards for reviewing interventions.Standards should be transparent.

The Scope of the Problem

• Kazdin (2000) identified 550 named interventions for children and adolescents.A very small number of these interventions

have been empirically evaluated.Of those that have been evaluated, the

large majority of them are behavioral or cognitive-behavioral.

The Scope of the Problem

• We have a great deal of evidence but much of that evidence is not being applied in practice settings.Evidence-based interventions are less likely to be

used than interventions for which there is no evidence or there is evidence about lack of impact (Kazdin, 2000).

In many instances practitioners are not aware of evidence-based interventions (Kratochwill, Albers & Shernoff, 2004).

There is very little guidance to parents and practitioners about how to decide among interventions.

Why the Increased Interest in Evidence-based Interventions?

• No Child Left Behind (NCLB) requires that interventions used to improve educational performance are based on scientific research. In NCLB there are over 100 references to

scientific research.

• Individuals with Disabilities Education Improvement Act [IDEIA] (2004) requires that interventions are scientifically based instructional practices.

Special Education and Evidence-based Interventions

• Pre-service and professional development for all who work with students with disabilities to ensure such personnel have the skills and knowledge necessary to improve the academic achievement and functional performance of children with disabilities, including the use of scientifically based instructional practices, to the maximum extent possible.

Special Education and Evidence-based Interventions

• Scientifically based early reading programs, positive behavioral interventions and supports, and early intervention services to reduce the need to label children as disabled in order to address the learning and behavioral needs of such children.

Special Education and Evidence-based Practice

• The Individualized Education Program (IEP) shall include a statement of the special education and related services and supplementary aids and services, based on peer-reviewed research to the extent practicable, to be provided to the child, or on behalf of the child, and a statement of the program modifications or supports for school personnel that will be provided for the child.

Ethical Conduct and Evidence-based Interventions

• Most national psychological and educational organizations have ethical standards requiring science-based practices to address problems.American Psychological Association Ethical

Standard 2.04: Psychologists’ work is based on the established

scientific and professional knowledge of the discipline.

Ethical Conduct and Evidence-based Practice

• National Association of School Psychologists Standard III F 4.

School psychology faculty members and clinical or field supervisors uphold recognized standards of the profession by providing training related to high quality, responsible, and research-based school psychology services.

Ethical Conduct and Evidence-based Practice

• National Association of School Psychologists Standard IV 4.

School psychologists use assessment techniques, counseling and therapy procedures, consultation techniques, and other direct and indirect service methods that the profession considers to be responsible, research-based practice.

Ethical Conduct and Evidence-based Practice

• Behavior Analysis Certification Board Standard 2.09a

The behavior analyst always has the responsibility to recommend scientifically supported, most effective treatment procedures. Effective treatment procedures have been validated as having both long-term and short-term benefits to clients and society.

Standard 2.09bClients have a right to effective treatment (i.e.,

based on the research literature and adapted to the individual client).

Controversies in Evidence-based Education

• There is no consensus about what constitutes evidence.The definition offered in NCLB permits both

quantitative and qualitative evidence without specifying the types of questions that each approach best answers.

In this context, we are most often concerned with evidence that establishes a causal relation between an intervention and a class of social or academic behaviors.

.

Controversies in Evidence-based Education

• There is no agreed upon standard for either the quantity or the quality of evidence necessary to validate an intervention as being evidence-based.Several organizations have established

standards but there is limited agreement among them.It is possible for an intervention to meet one

standard but not a second.

Applied Behavior Analysis and Evidence

• The research methods of single participant design are convincing demonstrations of causal relations.

• Behavior analysis research has often been criticized for limited generalizability because of the small number of participants in a research study.

Applied Behavior Analysis and Evidence

• Generally, the developed standards for validating an intervention as evidence-based have relegated single participant designs to a lower quality of evidence.This is problematic because in some areas

of educational research almost all of the evidence is based on these designs.AutismDevelopmental Disabilities

Applied Behavior Analysis and Evidence

• There are no established standards within applied behavior analysis for validating interventions.There is no single resource that decision-

makers can turn to for guidance about the best intervention to use for a particular problem.May lead decision makers to base decisions on

criteria other than evidence.

Applied Behavior Analysis and Evidence

Some related fields have published standards:Horner, et. al. (2005) provides guidelines for

evaluating evidence established with single participant research.

Flay, et. al. (2005) standards of Society for Prevention Research.

National Standards Project is developing standards of evidence for interventions for individuals with autism.

Task Force on Evidence-based Interventions in School Psychology.

What Works Clearinghouse.Only WWC has reviewed the literature in a

specific area to provide guidance about effective interventions.

Implementation: Research to Practice

Where good interventions go to die

• Identifying evidence-based interventions is necessary but not sufficient to assure they will be implemented in practice settings.

• It is also necessary to address complex issues associated with implementation (research to practice).

Four Concerns for Implementation

• Dissemination of evidence-based interventions in a manner that is accessible and useful to decision-makers.Publication in professional journals is not an

effective dissemination strategy for decision makers.Evidence about interventions are distributed across

multiple journals.Most practice level decision-makers are not

adequately trained to evaluate primary source data.Because of time constraints, reading professional

journals is a low priority activity for many decision-makers.

Four Concerns for Implementation

• Selection of evidence-based interventions.Often variables unrelated to effectiveness

influence selection of an intervention.CostExpert opinionPersonal experienceEffort associated with systems change

Four Concerns for Implementation• Initial Implementation of new intervention

What is necessary to gain practitioner support?Often inconsistent with previous training and experience.

What is required to train practitioners?What is necessary to assure integrity of

implementation?Implementation without integrity is implementation of an

unspecified intervention with unknown effects.How can new intervention be adapted to meet local

circumstances?If changed too much it becomes a different intervention for

which there are no data. If not modified to meet local circumstances it may not be

implemented at all.

Four Concerns for Implementation• Sustainability is the ultimate goal for

successful interventions.Definition of sustainable programs:

Maintains over time.Maintains across generations of practitioners.Supported with existing resources of system.

If effective interventions are not sustainable then the risk is great that alternative, ineffective programs will emerge.

The larger the scale of implementation the more complex all of the issues become and sustainability is more difficult to achieve.

Applied Behavior Analysis and Implementation

• Service delivery in applied behavior analysis is a mediated model.Behavior Analysts coach parents, teachers,

or other mediators to implement an intervention plan.

Mediated model requires that behavior analysts address many of the issues of implementation for each project.

Progress MonitoringPractice-based Evidence

• Even if an evidence-based intervention is implemented well, it is still necessary to systematically evaluate the impact.Progress monitoring is practice-based evidence

about the effects of an evidence-based practice.

It is difficult to predict which students will benefit from intervention prior to implementation.

No intervention will be effective for all students.Progress monitoring is consistent with ethical

guidelines.

Ethical Standards and Progress Monitoring

• National Association of School PsychologistsStandard IV C 1b.

Decision-making related to assessment and subsequent interventions is primarily data-based.

Standard IV 6.School psychologists develop interventions that

are appropriate to the presenting problems and are consistent with the data collected. They modify or terminate the treatment plan when the data indicate the plan is not achieving the desired goals.

Ethical Standards and Progress Monitoring

• Behavior Analysis Certification Board

Standard 4.04The behavior analyst collects data or asks the

client, client-surrogate, or designated other to collect data needed to assess progress within the program.

Standard 4.05The behavior analyst modifies the program on the

basis of data.

Progress Monitoring and Special Education

• Progress monitoring is fundamental to the IEP process.

• Progress monitoring is at the heart of Response to Intervention (RTI).All students are routinely and systematically

monitored to determine if they are making adequate progress against established standards.

Progress Monitoring and Behavior Analysis

• Data collection and review is the sine qua non of applied behavior analysis. It is not applied behavior analysis if

intervention data are not collected and reviewed to determine impact.

Behavior analysts rely on direct measures of behavior and its products to determine progress. Words read correctly as a measure of reading

ability.Frequency of specific problem behaviors or

adaptive behaviors.

Progress Monitoring and Behavior Analysis

• Behavior Analysis best represents the relationship between evidence-based practices and practice-based evidence.For many areas in special education there

are a wide range of behavior analytic evidence-based interventions.

The fundamental emphasis on progress monitoring in behavior analysis makes it well suited for leading the movement to a comprehensive approach to evidence-based practices.