Food for Thought - OSEP Ideas That Work · 2018. 8. 15. · This presentation and related EIT products were produced under the U.S. Department of Education (USDOE), Office of Special

1

Deciphering Sustainability:

Strategies from Five Years of Scaling-Up

Margo V. Izzo, PhD, Principal Investigator

[email protected]

Alexa Murray, Program Manager

[email protected]

Andrew Buck, Program Manager

[email protected]

The Ohio State University Nisonger Center

A University Center for Excellence in Developmental Disabilities (UCEDD)

Columbus, Ohio

This presentation and related EIT products were produced under the U.S. Department of Education

(USDOE), Office of Special Education Programs Grant H327S120022.

Presentation Objectives/Agenda

• Describe the Scaling-Up EnvisionIT grant

and intervention

• Share research findings

• Discuss strategies for achieving

intervention sustainability



Background and

Description of Intervention



Food for Thought

College and Career Readiness (CCR) initiatives and

standards expect our high school graduates to be

scholarly and employable with 21st century skills, but.…

• How do we prepare students with and without

disabilities for college and careers?

• How do we provide career development and guide

students to articulate postsecondary goals and plans

based upon their learning preferences, interests, and

skills?



mailto:[email protected]



2

Need for Evidenced-Based,

Standards-Aligned 21st Century Curricula

• “Special educators need assistance to develop […]

standards-based [Individualized Education Program] IEP

goals” (Caruana, 2015).

• Writing IEPs with goals that are meaningful and aligned

to CCR standards is difficult (Best & Cohen, 2013).

• “In 37 states, officials report facing challenges with

providing professional development to help teachers

align instruction for students with disabilities to the

Common Core” (Frizzell, 2013).



EIT Curriculum History

• Developed and evaluated since 2003

• Funded through successive U.S. Department of Education

(USDOE) grants

• Current Scaling-Up EnvisionIT (EIT) grant was awarded to

OSU Nisonger Center in 2012 from Office of Special

Education Programs (OSEP) at USDOE through

Stepping-Up Technology Implementation Program

• Purpose of current grant is to scale-up and sustain EIT in

Ohio and other states

– In 2014, we recruited Connecticut and New York through a Request

for Proposals (RFP) process

– EnvisionIT is being used in 31 schools in Ohio in inclusion, special

education, and general education classrooms



• FREE

• Evidenced-based and standards-aligned

• Blended and differentiated project-based learning

• Supports Individualized Education Programs (IEPs),

Individual Learning Plans (ILPs),Student Success Plans

(SSPs), and Individualized Plans for Employment (IPEs)

• 12 customizable units delivered online through learning

management systems (LMS) such as Google Classroom,

Schoology, and Canvas

Highlights of EIT Curriculum



• Course files also available on Google Drive

• Pre-unit and post-unit quizzes to gauge learning

• Learning supports (e.g., guided notes, review sheets,

glossary, videos)

• Supplemental content on reading, writing, and test-

taking strategies

• Enrichment activities and resource blog

Highlights of EIT Curriculum



3

21st Century Curriculum

The EnvisionIT (EIT) curriculum:

• Serves transition age youth with and without disabilities in grades 7-12

• Can be used by general educators, special educators/intervention specialists, and guidance counselors across a variety of educational settings

• Is customizable and supports blended learning through use of Internet and Learning Management Systems (LMS)




The EnvisionIT (EIT) curriculum aligns with state and national standards in:

• English Language Arts (ELA)

• Ohio ELA Extended Standards

• Technology Literacy

• College and Career Readiness (CCR)

• Career-Based Intervention (CBI),

• Financial Literacy

• Career Technical Education (CTE) Field Standards Strand 1






Common Core Standards

Transition Assessment

& Self-Directed Planning

Information Technology &

Computer Literacy

Alignment with Federal Mandate:

ESSA

Every Student Succeeds Act (ESSA) 21

U.S.C. 812(c) (3) Digital learning means any

instructional practice that effectively uses

technology to strengthen a student’s learning

experience and encompasses a wide

spectrum of tools and practices, including the

following:



4


ESSA EIT meets Every Student Succeeds Act (ESSA) criteria through content and

delivery:

a) Use of a Learning Management System (LMS) allows for interactive

learning

b) Students connect with databases such as O*NET

c) Students take online age-appropriate assessments such as the VARK

and O*NET Interest Profiler to help shape their career goals

d) Students can use blog features in selected LMS

e) EIT well-suited for blended learning where the teacher instructs and

leads students through the digital content

f) EIT can be accessed through the Internet and is compatible with

computers, tablets, and smart phones

g) Many activities can be printed off and require collaborative learning




IDEA Indicator 13

"Percent of youth with IEPs aged 16 and above with an IEP that includes

appropriate measurable postsecondary goals that are annually

updated and based upon an age appropriate transition assessment,

transition services, including courses of study, that will reasonably

enable the student to meet those postsecondary goals….” (20 U.S.C.

1416(a)(3)(B))

• In EnvisionIT, students complete age appropriate transition

assessments and develop postsecondary education and

employment goals

• EnvisionIT supports students to become active participants in their

IEP meetings




WIOA

EIT meets Workforce Innovation and Opportunity Act

(WIOA) required Pre-Employment Transition Services

(Pre-ETS) in the following areas:

a) Job exploration counseling

b) Work-based learning

c) Counseling on postsecondary education programs

d) Workplace readiness training

e) Self-advocacy skills

This presentation and related EIT products were produced under the U.S. Department of Education (USDOE), Office of

Special Education Programs Grants H327S120022 (awarded to OSU) and H027A160111A (awarded to ODE).

21st Century Curriculum The EnvisionIT (EIT) curriculum:

• Guides students in building a digital Transition Portfolio

• Includes age-appropriate transition assessments and activities to write measurable, postsecondary goals for employment and education/training

• Incorporates Universal Design for Learning (UDL) and Evidenced-Centered Design (ECD) features



5

EIT Transition Portfolio Students create a self-directed Transition Portfolio that includes any or all of the following:

1. Title Page

2. Age-Appropriate Transition Self-Assessment Results

Career Vision Survey

VARK

O*NET Interest Profiler

3. Compare and Contrast Careers Chart

4. Postsecondary Education/Training Research Chart




5. Postsecondary Goals for Career Plans A and B

6. High School Course Schedule

7. Short-Term Goals

8. Resume




9. Cover Letter

10. Job and/or College Applications

11. Job Interview Questions and Answers

12. Digital Presentation

13. Career Essay



Sample Courses (Ohio)

• Career Exploration

• Postsecondary Planning

• Transitions and Careers

• English Language Arts

• Financial Literacy

• Technology Literacy

• Career Advising

• Business Education

• Freshman Seminar

• Life Skills

• 21st Century Skills

• College and Career

Readiness



6

Why EnvisionIT?

By having students establish meaningful career

goals and plans through development of a

Transition Portfolio, EIT can help students:

• Gain important reading and 21st literacy skills

• Move toward postsecondary education and/or

employment



Quasi-Experimental

Research:

Project Years 3 (2014-2015)

and 4 (2015-2016)



• Research Hypotheses: Students who take

EnvisionIT will score significantly higher in

1) Reading Achievement

2) Information Technology (IT) Literacy

3) College and Career Readiness (CCR)

4) Self-Determination (SD)

• Setting: Intervention implemented in special and

general education classrooms at high schools in

Ohio and Connecticut for one semester or full

school year

Research Study Characteristics

(2014-2017)



• RQ1: What is the effect of EnvisionIT on reading?

• RQ2: Does this effect differ by grade and length of

class (semester or year)?

• Measure: AIMSweb 8th Grade Reading Maze

Research Question:

Does EIT Impact Reading?



7

Data Analytic Sample

• 18 teachers from 10 high schools in Ohio and

Connecticut who participated in Year 3 (2014-2015)

and first semester of Year 4 (2015-2016)

• 11 teachers implemented EIT (intervention group, n =

235) and 7 teachers did not implement (comparison

group, n = 120) (total n = 355)

• Intervention group: 49% of students were on IEPs, 3%

were on 504 plans, and 35% did not have a

documented disability (with 13% missing data)

• Comparison group: 51% of students were on IEPs, 3%

on 504 plans, and 45% did not have a documented

disability (with 5% missing data)



Method: Data Analysis

• Students (Level 1) were nested within teachers (Level 2)

• Dependent variable = difference score on AIMSweb

representing change in reading achievement from pre to

post

• Multilevel Linear Modeling (MLM; Snijders & Bosker, 1999)

was utilized

• To test this assumption a random effects analysis of

variance model was estimated so that the ICC –

representing the proportion of variance between teachers –

could be calculated ICC = 0.129

• Effect size was calculated with partial correlation

(Rosenthal & Rubin, 2003), where Large Effect = 0.52,

Medium Effect = 0.36, Small Effect = 0.14



Findings

• Significant and large effect of EnvisionIT on reading scores

• Intervention group students made more meaningful gains in reading (i.e., AIMSweb8 scores increased) – 9th graders +3.11

– 10th graders +4.56



• Effect differs by grade

• Length of class was not statistically significant



Quasi-Experimental

Research:

Project Year 5 (2016-2017)



8

RQ1. What is the effect of the EnvisionIT intervention

on gains in career readiness and is the effect

moderated by class setting?

RQ2. After controlling for class setting, does the effect

differ by instructional quality (i.e., fidelity) and dosage?

Research Question:

Do Setting, Fidelity, and Dosage Matter?



Measuring Fidelity (Years 3-5)

o EIT Fidelity Checklist included: o Instructional best practices adapted from:

o Ohio Department of Education: Teacher Performance

Evaluation Rubric

o Connecticut Department of Education: Common Core of

Teaching Rubric for Effective Teaching

o Qualitative classroom observations

o Curriculum specific practices

o Scoring protocol

o Designed to gauge relationship between effective

teaching and student achievement



Measuring Fidelity (Years 3-5) o EIT Fidelity Checklist consisted of 4 parts with 29 items

o Part I: General Teacher Behavioral Standards

o 9 items on a 3-point Likert scale with observer comments and

real-world examples

o Part II: Lesson Specifics: Computer Work

o 1 descriptive item and 8 dichotomous items with observer

comments and real-world examples

o Part III: Lesson Specifics: EIT Activity

o 7 dichotomous items with observer comments and real-world

examples

o Part IV: Global Rating of Fidelity

o 1 global rating item on a 3-point Likert scale and 3 items for

required observer comments

o Included companion planning, scoring, and instructional

coaching protocol



Measuring Fidelity (Year 5)

o Teacher Lesson Logs (i.e., dosage) o Checklist of core components

o Transition Portfolio activities

o Supporting content and activities

Qualitative teacher notes

o Additional activities (e.g., 21st Century Wednesdays)

o Classroom Coding (i.e., setting stratified by dosage) o Dedicated

o Integrated

o Resource Room



9

Training and Fidelity

o Initial Trainings o Customized based on teacher needs and schedules

o Hands-on with focus on use of LMS

o Classroom Observations o EIT Fidelity Checklist

o Initial site visit conducted to gather baseline data

o Site visits conducted 1x/month or until consecutive passing

scores (i.e., >75%)

o Teachers could also rate themselves

o Instructional Coaching and Technical Assistance o Built rapport with teachers, observations were not judgements

o Provided written, verbal, and visual feedback

o Highlighted strengths and identify areas for improvement

o On-going communication

o Follow-up trainings at school or via webinar



Method: Data Analysis

• Multilevel linear models were estimated in R

using the lme4 R package using full information

maximum likelihood

• RQ 1: Intercept-as-outcome model

• Intervention effect

• Setting, disability, grade level covariates

• RQ 2: Moderating effects

• Teacher fidelity and dosage scores (grand mean

centered) were tested as moderators of the

intervention effect



Results: VSSE Scores as Function of

Fidelity and Dosage



Results: SCCI Scores as Function of

Fidelity and Dosage



10

Findings

• Significant effect of intervention on career readiness

gains as measured by scores on the career measures

(Vocational Skills Self-Efficacy Scale (VSSE and Student

Career Construction Inventory (SCCI)

• Intervention effect significant while controlling for setting

• Students who received intervention from teacher with

above average fidelity showed amplified effects (i.e.,

career readiness gains)

• Students who received intervention from teacher who

taught above average percentage of the lessons showed

amplified effects (i.e., career readiness gains)



Findings

• Students with and without disabilities in intervention

group (i.e., those receiving EIT) showed

significantly higher gains in career readiness

scores than students in comparison group

• The curriculum is ideal for general education settings

• Students showed the largest gains in career

readiness when:

• They were taught more than 75% of the curriculum – on

average these students gained approximately 3 points on

the VSSE and 2 points on the SCCI

• Teacher fidelity scores were above average indicating

higher instructional quality



EIT Implications 2010-2017

• In past grants, EIT pre-post gains were

statistically significant in ICT literacy and

CCR/transition with trend gains in reading

• In current grant, EIT pre-post gains were

statistically significant in reading, ICT

literacy, and CCR/transition

• EIT is an evidenced-based tool



Establishing An

Evidence-Based Curriculum

1 Lombardi, A. R., Izzo, M. V., Gelbar, N., Murray, A., Buck, A., Johnson, V., Hsiao, J., Wei, Y., & Kowitt, J. (2017). Leveraging

information technology literacy to enhance college and career readiness for secondary students with disabilities. Journal of

Vocational Rehabilitation, 46(3), 389-397. Available at https://content.iospress.com/articles/journal-of-vocational-

rehabilitation/jvr875.

2 Lombardi, A. R., Izzo, M. V., Rifenbark, G. G., Murray, A., Buck, A., Monahan, J., & Gelbar, N. (2017). The impact of an online

transition curriculum on secondary student reading: A multilevel examination. Career Development and Transition for Exceptional

Individuals, 40(1), 15-24. Available at http://journals.sagepub.com/doi/full/10.1177/2165143416681287.

3 Izzo, M.V., Yurick, A., Nagaraja, H.N., & Novak, J.A. (2010). Effects of a 21st-Century curriculum on students’ information

technology and transition skills. Career Development for Exceptional Individuals, 33(2), 95-105. Available at

http://journals.sagepub.com/doi/abs/10.1177/0885728810369348.



https://content.iospress.com/articles/journal-of-vocational-rehabilitation/jvr875







http://journals.sagepub.com/doi/full/10.1177/2165143416681287


http://journals.sagepub.com/doi/abs/10.1177/0885728810369348

11

EIT Research Limitations 2014-2017

• Lack of randomization due to need to

recruit schools – many schools not

comfortable with randomization

• Selection bias with convenient sampling

• Self-report bias on CCR and SD measures

• Difficulties defining, measuring, and

enforcing fidelity



Sustainability



How does fidelity

Fidelity Versus Sustainability

o How do you achieve fidelity for an innovative

intervention that requires a flexible and

customizable approach in order to achieve

sustainability across diverse sites?

o The focus of the USDOE OSEP Stepping-Up

Technology Implementation Program is

more on scaling-up and sustaining an

intervention than pure research



Fidelity Sustainability



Balancing Project Priorities: Fidelity

12

Fidelity Sustainability



Balancing Project Priorities:

Sustainability Barriers to Sustainability

• Competing school or district priorities

• Lack of resources (e.g., technology, staff)

• School restructuring and schedule changes

• Lack of teacher/administrator support



Barriers to Sustainability

• Bureaucracy especially in large urban districts

• Inaccurate perceptions regarding curriculum

value and purpose

• Resistance to change including unwillingness

to integrate EIT into school goals or

infrastructure



Strategies that Encourage

Sustainability (School)

• Leveraging EIT as a resource to help comply

with federal mandates and/or enhance current

district or state initiatives

• Simultaneous teacher and administrator

champions

• Customizability of EIT to allow teachers to help

meet student learning needs



13



• Availability of EIT across venues

• Availability of sample course models as well as

full curriculum

• Availability of training and teacher resources

(e.g., curriculum pacing guides, site maps)





• Memorandums of Understanding (MOUs)

• Ongoing in-person or online training including

encouraging teacher leaders to train other

teachers

• Evidence of intervention effectiveness

(research and anecdotally)




Sustainability (State/National)

• Strong partnerships between State Education

Agencies (SEAs) and Local Education

Agencies (LEAs) in order to recruit and train

schools/districts on using, adapting, and

integrating EIT into school/district infrastructure

• Promotion of, and training on, EIT through

state-sponsored professional development (PD)

networks, teams, and venues




Sustainability (State/National)

• A Request for Applications process to recruit

states and build capacity to sustain the EIT

intervention in those states

• Partnerships with national organizations for

distribution through their networks



14

EIT and Implementation

Science Framework



Installation Initial Training Fidelity Visits Technical Assistance

• On-site computer lab

or classroom setting

• Schedule times with

teacher to observe

class

• Maintain regular

communication with

sites

• Structured, yet

customized, agenda

• Complete fidelity

checklist and assess

coaching needs

• Address specific

issues quickly on

case-by-case basis

• Interactive to promote

practice and group

work

• Ask teacher to

complete a fidelity

checklist as self-

assessment

• Post general

responses and

updates to group

online

• Professional

development

resources and

supports

• Respond to

questions, gather

feedback, provide

support and coaching

• Encourage group to

share resources,

ideas, questions and

feedback online

• End goal is ability to

use intervention

effectively

• End goal is improving

fidelity and rapport

• End goal is

enhancing support

and capacity



Initial Implementation Implementation

Driver Activity (Ohio Rural High School)

Competency

Enact communication

protocols

Schedule site visits to observe and assess

fidelity Maintain regular communication

Execute coaching plan

Assess fidelity and review results with

teachers Provide immediate feedback

Organization

Measure and report outcomes

Gauge student outcomes through pre-post

quizzes in EIT units

Collect pretest-posttest data and present findings to key personnel

Measure and

report fidelity

Gauge teacher performance with assessment

Update training, coaching, and fidelity checklist

Leadership Review recommendations for process improvement

Differentiate curriculum for students with and

without disabilities and across grades

Implement curriculum in an inclusive classroom, such as freshman seminar elective



Table adapted from Blase, K., van Dyke, M. & Fixsen, D. (2013). Stages of implementation

analysis: Where are we? National Implementation Research Network. Retrieved from

http://implementation.fpg.unc.edu/sites/implementation.fpg.unc.edu/files/NIRN-

StagesOfImplementationAnalysisWhereAreWe.pdf

Full Implementation



Ohio Suburban School District:

• District needed a high school curriculum that would address

transition planning needs of students with and without IEPs

• Transition planning course would be for a semester and

worth one-half (0.5) elective credit

• One special education teacher and one general education

teacher would be selected as instructors (separate

classrooms)

• The school district already used Schoology and needed an

online transition curriculum that would work with the LMS

• The curriculum would address college and career readiness

and support technology literacy to address new state

requirements

http://implementation.fpg.unc.edu/sites/implementation.fpg.unc.edu/files/NIRN-StagesOfImplementationAnalysisWhereAreWe.pdf



15

Other Projects: Strategies

That Work



Strategies That Work Survey

Survey was:

• Developed by committee in winter 2017 in order

to collect process information about lessons

learned regarding project implementation,

scale-up, and sustainability

• Created using Qualtrics online survey tool

• Distributed to OSEP Stepping-Up Technology

Implementation Program Grant Cohorts 8-12

representing initial grant awards from 2012 to

2016

Survey Questions

• What is it that you know now that would have been

really helpful or useful to know at the start of your

project?

• Knowing what you know now, how can a particular

challenge be avoided or successfully confronted?

• What resources did you/are you/will you acquire,

develop, enhance, or leverage in order to promote

implementation, scale-up, and sustainability of the

innovation? In this context, resources are broadly

defined as tools, knowledge, relationships, monies, etc.

Survey Questions

• In your opinion, what is the most significant factor or set

of factors that contribute to the successful

implementation of your innovation?

• Add your own lesson/strategy learned from project

• Categories of lessons/strategies

16

Categories of Lessons/Strategies

___ Technology development/refinement

___ Training/professional development/instructional coaching

___ Fidelity of technology implementation (adherence to how

technology should be used)

___ Research design/measurement

___ Field testing

___ Site recruitment

___ Product dissemination/distribution

___ Scale-up and sustainability

___ Quality assurance

___ Buy-in or support from stakeholders

___ Other: please indicate:

Response Rate

• A total of 21 individuals responded to the

survey

• 14 projects out of 20 were represented

(70%)

Results: Descriptive Categories

0.00%

10.00%

20.00%

30.00%22.59%

18.55% 21.05%

15.37% 17.86%

15.79%

Average Percentages Across MainCategories of Lessons/Strategies

Results: Qualitative Feedback • There was a lot of good feedback across categories – 40 pages of

single spaced comments

• Comments were distilled into 15 pages – while that may still be long,

decision was made to preserve meaning of direct quotes for handout

• Attempt was made to retain integrity of direct quotes while highlighting

key points

17

Discussion



Discussion Question 1

• In your projects, a) what challenges have

you experienced with implementing and

sustaining products or interventions, and

b) how did you work around these

challenges?




• In your projects, a) what strategies did

you employ in order to increase the use

and sustainability of your products or

interventions, and b) were these

strategies successful?




• In your projects, who were the

implementation drivers or key players that

enhanced success of your products or

interventions?



18


• Think about a current or recent project

you directed or helped implement.

Knowing what you know now, what

aspect of the project, if any, would you

have liked to have done differently in

order to augment project success?



• Izzo, M.V., Yurick, A., Nagaraja, H.N., & Novak, J.A. (2010). Effects of a

21st-Century curriculum on students’ information technology and transition

skills. Career Development for Exceptional Individuals, 33(2), 95-105.

Available at

http://journals.sagepub.com/doi/abs/10.1177/0885728810369348.

• Lombardi, A. R., Izzo, M. V., Gelbar, N., Murray, A., Buck, A., Johnson, V.,

Hsiao, J., Wei, Y., & Kowitt, J. (2017). Leveraging information technology

literacy to enhance college and career readiness for secondary students

with disabilities. Journal of Vocational Rehabilitation, 46(3), 389-397.

Available at https://content.iospress.com/articles/journal-of-vocational-

rehabilitation/jvr875.

EIT Published Journal Articles



• Lombardi, A. R., Izzo, M. V., Rifenbark, G. G., Murray, A., Buck, A., &

Johnson, V. (2016). A preliminary psychometric analysis of a measure of

information technology literacy skills. Career Development and Transition

for Exceptional Individuals, 39(4). Available at

http://journals.sagepub.com/doi/full/10.1177/2165143416682476.

• Lombardi, A. R., Izzo, M. V., Rifenbark, G. G., Murray, A., Buck, A.,

Monahan, J., & Gelbar, N. (2017). The impact of an online transition

curriculum on secondary student reading: A multilevel examination. Career

Development and Transition for Exceptional Individuals, 40(1), 15-24,

Available at

http://journals.sagepub.com/doi/full/10.1177/2165143416681287.

EIT Published Journal Articles



Margo Vreeburg Izzo, Ph.D.

Project Director

The Ohio State University

(614) 685-3190

[email protected]

Alexa Murray

Program Manager


(614) 685-3424

[email protected]

Andrew Buck

Learning and Development Consultant


(614) 688-7717

[email protected]

EIT Contact Information



For general inquiries

please contact OSU

Nisonger Center Transition

Services at (614) 685-3185

or [email protected]

http://journals.sagepub.com/doi/abs/10.1177/0885728810369348















19

Additional Resources EIT Curriculum in Google Drive:

http://go.osu.edu/eitlibrary

EIT Teacher Training Videos:

http://go.osu.edu/eitpd

EIT Student Videos:

http://go.osu.edu/eitvideos

U.S. Department of Education (USDOE) Office of Special Education Programs (OSEP):

https://www2.ed.gov/about/offices/list/osers/osep/index.html

The Ohio State University Nisonger Center EIT Website:

http://go.osu.edu/eit

The Ohio State University Nisonger Center Website:

http://nisonger.osu.edu

University of Connecticut Neag School of Education:

http://education.uconn.edu/



University of Rochester Institute for Innovative Transition:

http://www.nytransition.org

Schoology Learning Management System:

http://www.schoology.com

Council for Exceptional Children Division on Career Development and Transition:

http://www.dcdt.org

Association of University Centers on Disabilities:

http://www.aucd.org/

University of Oklahoma Zarrow Center for Learning Enrichment:

http://www.ou.edu/education/centers-and-partnerships/zarrow.html

The National Implementation Research Network’s Active Implementation Hub

http://implementation.fpg.unc.edu/

State Implementation and Scaling-Up of Evidence-Based Practices Center

http://sisep.fpg.unc.edu/

Additional Resources



• Best, J., & Cohen, C. (2013). Common Core State Standards and Implications for Special

Populations (Policy brief). Denver, CO: McREL. Available at

http://files.eric.ed.gov/fulltext/ED557609.pdf.

• Caruana, V. (2015). Accessing the Common Core Standards for Students with Learning

Disabilities: Strategies for Writing Standards-Based IEP Goals, Preventing School Failure:

Alternative Education for Children and Youth, 59:4, 237-243, DOI:

10.1080/1045988X.2014.924088: Available at http://dx.doi.org/10.1080/1045988X.2014.924088.

• Frizzell, M. (2013). Year 3 of Implementing the Common Core State Standards Transitioning to

CCSS-aligned Curriculum and Assessments for Students with Disabilities (Policy brief). Center

on Education Policy. Available at http://files.eric.ed.gov/fulltext/ED555351.pdf.

• Rosenthal, R. & Rubin, D. B (2003). R-equivalent: A simple effect size indicator. Psychological

Methods, 8 (4), 492-496.

• Snijders, T. & Bosker, R. (1999). Multilevel analysis: An introduction to basic and advanced

multilevel modeling. New York, NY: Sage.

References



Funding Support and Disclaimer

This presentation and related EIT products were produced

under the U.S. Department of Education (USDOE), Office of

Special Education Programs Grant H327S120022 (CFDA

84.327S). The views expressed herein do not necessarily

represent the positions or polices of the USDOE. No official

endorsement by the USDOE of any product, commodity,

service, or enterprise mentioned in this presentation is intended

or should be inferred.





http://go.osu.edu/eitpd

http://go.osu.edu/eitvideos

https://www2.ed.gov/about/offices/list/osers/osep/index.html

http://go.osu.edu/eit

http://nisonger.osu.edu/

http://nisonger.osu.edu/



http://www.nytransition.org/

http://www.schoology.com/

http://www.dcdt.org/

http://www.aucd.org/











http://files.eric.ed.gov/fulltext/ED557609.pdf



http://dx.doi.org/10.1080/1045988X.2014.924088

http://dx.doi.org/10.1080/1045988X.2014.924088

http://dx.doi.org/10.1080/1045988X.2014.924088



20

There are no copyright restrictions on this document. However,

please cite and credit the source when copying all or part of

this document. This document was supported in whole or in

part by the U.S. Department of Education, Office of Special

Education Programs (Award H027A160111A, CFDA 84.027A,

awarded to the Ohio Department of Education). The opinions

expressed herein do not necessarily reflect the policy or

position of the U.S. Department of Education, Office of Special

Education Programs, and no official endorsement by the

Department should be inferred.

Funding Support and Disclaimer



Acknowledgements

• We sincerely thank the following for their support of the

EnvisionIT (EIT) initiative:

o Ohio Department of Education (ODE)

o U.S. Department of Education (USDOE)

o Association of University Centers on Disabilities (AUCD)

o University of Connecticut

o University of Rochester



Training/Professional Development/Instructional CoachingAt the beginning of our project, we wanted to make sure that our participants had allthe training information that they needed to use our app. Therefore, we provided theparticipants with all the information needed on how to use the app at one time. As aresult, they did not retain all the information. This caused us to change our trainingstrategy.Many of our teachers have never used an iPad, refreshable braille display or both. Hadwe known this prior to the commencement of our project, we would have providedgroup or one-on-one instruction earlier in the process followed by training after eachteacher demonstrated competence. It is important to adequately train teachers more interactively, and monitor theirparticipation as part of product adoption. It would have been useful to know moreabout the need for flexible, individualized coaching to ensure quality data collectionduring the development stage of the project. In particular, we needed to adjust ourthinking from working with sites “equally” to “fairly” (in terms of each site getting whatis needed to enable a more similar baseline standard of implementation with somesites requiring significantly more support than others).

SURVEY QUESTION 3:

Our original project plan called for many iterative drafts, early in the project. Instead,we've found waiting a bit is more cost- and time-efficient and produces higher qualitymaterials more relevant to stakeholder needs. All of the bugs we have found in the braille translation tables have been our biggestchallenge and had there been some precedent we would have benefitted greatlyfrom learning from others. The technical aspects of making a braille game with the iOS has been the biggestchallenge. Apple products are amazingly accessible for people who are blind.However, the iOS is updated frequently. While our technical team has been creativeand diligent, I wish we had had someone who REALLY understands iOS and how itworks with braille from the very beginning.

Technology Development/Refinement

What is it that you know now that wouldhave been really helpful or useful toknow at the start of your project?

Stepping-Up Technology Implementation: Strategies That Work Survey Handout

N=19 responses

Site SelectionFor recruitment, we know now that there are times when it is best to approach theteachers directly, and there are times when it is best to recruit at the district level.When conducting smaller scale studies (i.e., development studies), it’s better torecruit teachers directly, but for larger, randomized controlled trials, it’s better to goto the district level. We learned we need to connect with district leaders more tofacilitate recruitment, and then also talk with teachers to gauge and support interest.It is critical to recruit teachers who are interested in the projects and want to usethem with their classes rather than seeing it as being a district-mandatedrequirement. Where our collective experience might be helpful for those newer to school-basedresearch is that without prior experience, it is easy to seriously underestimate thechallenges of collecting the descriptive and organizational information needed,conducting needed assessments, and working around school schedules that involvemany fall start up activities and later demands such as school testing dates andrequirements.

continued...

1 of 15

What is it that you know now that would have been really helpful or useful to know at thestart of your project?

SURVEY QUESTION3: CONTINUED...

The importance of taking the time to develop relationships with the school-basedsettings where one is going to be conducting intensive research. Be sure to develop partnerships with key stakeholders and identify champions(those most enthusiastic about and receptive to the intervention) at EACH level ofsystems change (such as a classroom teacher, school/district administrator, statedepartment of education representative) to facilitate adoption of your innovation.For instance, our most successful site has been a suburban district north ofColumbus where a champion teacher at one of the high schools and the districtdirector of pupil services worked simultaneously and together to scale-up our onlinecurriculum in the district.

Key Stakeholder Buy-In

Our original grant was to provide coaching support using technology for SPEDreading teachers in resource rooms where students are pulled out of their generaleducation classroom for reading and language arts instruction. At the beginning ofour first full year of the grant, we learned that all but one of our five school districtswere going to full inclusion instruction for elementary students. While this changemay benefit some students, it proposed a problem for us since we were observingSPED teachers and their students during instruction either in the classroom or viatechnology using our platform.

It would have been helpful to have considered combining some of the fidelitymeasures with the evaluation from the beginning of the grant. We went throughseveral iterations of collecting data over the past 4 years and recently created aGoogle form to include fidelity checklists and partner educator feedback in an easy-to-use familiar format.

Fidelity of Technology Implementation

The core vocabulary needs of students with significant cognitive disabilities who donot have any symbolic communication skills are not the same as the needs ofstudents who are already using single words, signs or symbols. Four things:

Other

1) How teachers think about their instructional lessons

2) How difficult it is to get teachers to report on the multi-task

nature of their work using a common descriptive language

3) How challenging our performance tasks on competence tests

were for teachers

4) The range of supports teachers need to use new technology

2 of 15

Training/professional development/instructional coaching(26.32%) (n=5)

SURVEY QUESTION 4:

Technology development/refinement (21.05%) (n=4)

By what theme or topic would youcategorize this lesson/strategy? (This question refers back toQuestion 3: Knowledge Sooner.)

Site recruitment (21.05%) (n=4)


N=19 responses

Total of 1), 2), and 3) = 68.42%

Remaining 31.58% of response choices were distributedfairly evenly across other categories

3 of 15

Training/Professional Development/Instructional CoachingAt the beginning of our project we presented our teachers with a lot of informationat one time in our trainings. As we began to see that teachers were becomingoverwhelmed with information, we discussed ways that we could change ourtrainings to better meet the needs of our teachers. We decided to break ourtrainings down to smaller chunks of information. Once teachers completed [initial]tasks we would provide them with the next task. This strategy change proved to bebeneficial to our teachers. We broke our teachers into mini-cohorts and trained them first on technology, thenon the app - and we each took ownership of a particular cohort to usher themthrough the process of onboarding, troubleshooting and training. It has beenchallenging to train and retain teachers remotely. In our second pilot study, wedecided to create cohorts of teachers, and assign them a support provider. Thesupport provider calls each teacher twice a month to check in with them and offertechnical support. This has been an effective method of keeping in contact withteachers, offering support, providing assistance, and moving teachers along in thestudy.Making sure teachers are able to show the students how to use the technology is animportant consideration we had not anticipated. We learned that teachers need (inaddition to training on the curriculum) training on how to use technology.

SURVEY QUESTION 5:

Treat iOS/Apple recommendations/best practices as requirements since theyeventually become requirements!If you are building an app that will work with the accessibility features of iOS andperipherals, make sure you have someone on the tech development team FROM THEVERY BEGINNING who understands BOTH the disability-specific accommodationsAND how the iOS works.


Knowing what you know now, how cana particular challenge be avoided orsuccessfully confronted?


N=19 responses

Site SelectionRecruitment seems to present an ongoing challenge for field-based researchers. Oneway we have successfully worked to meet this challenge in the current project is toserve on councils/committees that expose us to the target stakeholder groups on anongoing basis. Strategic recruitment of preschool sites can be challenging and time-consuming….[b]ecoming active in early childhood community initiatives has made strategicrecruitment easier because we have become familiar with local leaders who haveprovided recruitment guidance and greater understanding of potential landminesthat we might not have been privy to otherwise. Recruit other sites as-back up that can also implement your innovation during thedesired time period or school year and allow for recruitment of these sites at least 6months to 1 year prior to implementation of your innovation. Instead of subcontracts, we now offer to purchase communication-related suppliesfor classrooms as an incentive to participation. This has worked very well and seemsto be a strong incentive for participation.

continued...

4 of 15

Knowing what you know now, how can a particular challenge be avoided or successfullyconfronted?


The interest and leadership of school and district administration is key tosuccessful recruitment of teachers and students for participation. One of the great challenges in reaching sustainability within a school district(especially a large district) is bureaucracy. The administrative structure can slow orstop the implementation progress of a useful technology-based curriculum. Onelesson learned is that in large districts, it is extremely important to contactschool/district officials at least six months before the desired time ofimplementation. Summer is probably the best time to contact administrators.Working with administrators carefully to support how the program is brought in iscritical to get teacher buy-in up front so they feel happy to implement and so theydo not feel forced to do it.

Key Stakeholder Buy-In

Gaining a champion at the state level who is effective has been integral tosuccessful scale-up of our sites.

Scale-Up and Sustainability

[To simplify data management], we have created a Moodle course—an onlinecourse platform that walks the participants (and facilitators) through eachelement of participation. This provides one place for participants, facilitators,and project staff who are monitoring the data collection to manage and trackall aspects of the project related to participants….much of the data is now easilytracked and analyzed through the Moodle, making data management mucheasier for project staff.Professionals and parents need incentives to encourage them to provide thedata that we request to evaluate the success of the project over the course of afairly lengthy term of involvement. I think is important to continuously evaluate how things are going with yourproject and when you are faced with a challenge collaborate with your teamabout the challenge and come up with strategies with your team to confrontthe challenge. I think our team has been very good about doing that. We are struggling to get student background data from the schools….one ideato remedy this is to get the data up front prior to implementation. When theysign the contract, we would get the student data first from the school and ifthey cannot provide it, they are not able to continue in the study.

Research Design/Measurement

When a school or partner cannot resolve the challenges of implementing thetechnology, provide TA initially, then cut your losses before investing years offunding and staff time. Design a user competency test that uses more authentic evidence from actualpractice, [and] build in a virtual coaching component to enact more supportthroughout use of technology.

Field Testing

5 of 15

Training/professional development/instructional coaching (26.32%)(n=5)

SURVEY QUESTION 6:


By what theme or topic would youcategorize this lesson/strategy?(This question refers back toQuestion 5: Meeting Challenge.)

Site recruitment (21.05%) (n=4)


N=19 responses

Total of 1), 2), 3), 4), and 5) = 89.48%


Scale-up and sustainability (10.53%) (n=2)

Buy-in or support from key stakeholders and grant partners(10.53%) (n=2)

6 of 15

Training/Professional Development/Instructional CoachingProfessional development is far more meaningful when it encourages “togetherness”of teams, groups of teachers, and tons of tech hands-on time. The sharedgoals/context of teams experiencing PD together allows educators to reallyproactively explore the technology….[h]aving groups of teachers receiving PD togetherallow teachers to ask each other questions and talk amongst themselves (re:scheduling, device management, tips for behavior management). Hearing or seeinghow veteran users of the intervention [use or have used the technology] and justsharing out ideas for their own use was particularly fruitful, inspiring, and reassuring tonovices. We have a series of 14 professional development modules that address use of theintervention and core vocabulary in the classroom; tools to guide data-drivendecisions; coaching supports; self-assessments and fidelity checklists. Our goal is toprovide a comprehensive implementation program that is not dependent on experts.We are taking care to not be "content experts" as we enter into discussion withparticipating teachers but rather use the tools we have developed to encourageteachers to self-assess and reflect on practices. We created and distributed vignette videos and created other materials that reallyhelped our teachers learn on their own and allowed us to focus more on the specifictraining related to the app. We also instituted an in-app survey that each teacher wasrequired to take upon initial log-in to the app. This survey asked a battery of questionsrelated to their comfortability with basic technology skills. Teachers were asked tocheck off each skill that they were comfortable with. If they felt uncomfortable with askill, they could watch [an accompanying] video. Teachers had to respondaffirmatively to 90%+ of the skillsets before they were able to access the app.Ongoing professional development by our expert reading coaches was not only partof our grant’s procedures, but it was critical in improving SPED reading teachers’knowledge of effective, research-based instruction. We also have createdpowerpoints, handouts, and videos that teach/model/show our participating teachershow to use every function on the platform. Teaching resources and created lessonsand guidelines were constantly being updated on the platform as well.Dovetail application with existing lesson planning routines and recruit instructionalcoaches as "partners" in using the tool to generate actionable feedback to guideinstruction. We developed a website for each project as a way to deliver the curriculum andsupport implementation, scaling-up, and sustainability. We want to design websitesthat teachers can use on their own with minimal support from us. Such websiteswould include automated reports for teachers as well as automated assessments forstudents and teachers. We believe that online professional development has thepotential to support sustainability of technology-based resources.

SURVEY QUESTION 7:

We got feedback from a tech savvy person who understands app development,iOS, and braille a bit too late to make the fundamental decisions that should havebeen made early on. Again, the tech team has done great work to makeeverything work well, but I fear there will be a steep learning curve for someconsumers (students and teachers and parents) to make everything work welltogether. I think that high-quality tech support will need to be built into the appbefore its release and ongoing.


What resources did/are/will youacquire, develop, enhance, or leverageto promote implementation, scale-up,and sustainability of the innovation?


N=19 responses

7 of 15

What resources did/are/will you acquire, develop, enhance, or leverage to promoteimplementation, scale-up, and sustainability of the innovation?


Site RecruitmentWe have created a variety of resources for our project’s stakeholders. For example,for soliciting districts, we created a document that describes the platform in detail,including pictures and screen shots for further clarification. We also prepared aninformative Power Point presentation that highlighted not only the purposes,goals, and information about the grant, but also described the previous researchand findings from other related grants and work in schools.

The lesson that we have learned is that there must be an investment of time tobuild relationships at the incipient stage of project implementation. One exampleof this is the relationship that was built with a managing consultant in our StateDepartment of Education…[w]ithout this advocate’s enthusiasm for the quality ofthe curriculum and its potential to benefit students with disabilities, we would nothave the current number and variety of active sites. When working with large orsmall organizations, it is necessary to invest the time to find an advocate orchampion within that system. Our prior relationships with elementary school administrators, early-learningresearchers, and people who work in the State Department of Education havebeen an important resource to promote implementation, scale-up, andsustainability of the innovation. We have been careful to keep these peopleapprised of our progress from concept, to initial development, to field testing,refinement, and piloting. Many of them have expressed keen interest in helpingscale-up the tool we're refining once it is developed sufficiently to enable us torelease it on a larger scale. We are developing an “Exit Survey” for programs that discontinue use of ourtechnology. We often only consider the tools and strategies that ‘work’ inpromoting scale-up and sustainability, however it is also important to learn fromprograms that discontinue implementation. Gaining support and funding from our state department provided project staffwith additional resources that included funding, alignment with state priorities,leveraging the contacts of our state project officer, and partnering to accomplishmultiple outcomes to address state and project goals. [In addition to charging reasonable fees for use of our products] [w]e will attemptto generate revenue from selling ads to compatible companies that may findpotential consumers through our website or to sponsors who wish to support ourefforts….we [recently] launched a seasonal donations campaign aimed at takingadvantage of the end-of-year tax-deductible holiday season. After the New Yearwe will revert to an ongoing donations strategy. All donations are submittedthrough PayPal, and we ask contributors to optionally provide their name andtheir reasons for contributing, with their permission to publish their words on thewebsite....we may use this information to create a donators “wall”, with some of thebetter quotes from our supporters.


We are developing a more multi-faceted training approach to promote fidelity ofactivities implementation for scale-up. For example, we are developing briefreminder “as you go” videos and messages to accompany our developed resources.We have seen the need for greater teacher scaffolding and are leveraging ourtechnological system to build in additional supports for using our innovation asintended.


8 of 15


SURVEY QUESTION 8:


By what theme or topic would youcategorize this lesson/strategy?(This question refers back toQuestion 7: Developing Resources.)



N=18 responses

Total of 1), 2), and 3) = 55.56% (n=10)

Remaining 44.44% (n=8) of response choices weredistributed fairly evenly across other categories

9 of 15

Training/Professional Development/Instructional CoachingTraining and support were paramount. Once teachers felt comfortable using thedevice(s), they were able to administer the project with/to their students - and theirstudents, almost universally, caught on and excelled, and even loved the devices.Rigorous, careful training on the use of the technology. Financial incentives also playa role. [W]e have found that many of our teachers are not as tech-savvy as we hadoriginally thought. The most significant factor that contributes to teachers using thetechnology successfully is by conducting one-on-one instruction meetings with ourteachers, either in person or on the Hoot Education platform. We also have createdPowerPoints, handouts, and videos that teach/model/show our participatingteachers how to use every function on the platform.

SURVEY QUESTION 9:

I believe using attractive game-based activities that are universally designed for allstudents along with activities developed with students and teachers is whatcontributed to the success of Future Quest Island. The most significant factor to the success of implementation is the quality of theproduct. Hours of review and updates, based on suggestions from educators,students, and parents, went into EnvisionIT curriculum refinement. Educatorshave recognized the efforts to craft a curriculum that is effective for the studentand the teacher. The utility of the innovation, the degree to which it provides a really useful toolthat is both accessible and high-quality, is the most significant factor thatcontributes to its successful innovation. Technical support for teachers on how to use the app with peripheral devices.


In your opinion, what is the mostsignificant factor or set of factors thatcontribute to the successfulimplementation of your innovation?


N=19 responses

Key Stakeholder Buy-InBuy-in for full implementation from both top-level administrators as well asservice providers (home visitors, teachers, interventionists, etc). Successful implementation of our innovation will depend mostly on buy-in frompreschool site administrators (e.g., to purchase adequate tablets/wi-fi anddetermine whether our approach “fits” with their site philosophy and ways of theclassroom) and teacher use of the tablet-based system with fidelity (associatedwith the degree to which the approach fits with their teaching philosophy as well).

Scale-Up and SustainabilityOur website is attractive and easy to use. Use of it is free. It is open to all users,professional and nonprofessional alike. Teachers and schools have to juggle multiple priorities and demands each day. Forour intervention to be successful, it must align with and help teachers addressthese priorities and demands. For the first year of the project, we saw great thingshappening in Schools 1 and 2, but we did not see teachers integrating theUniversal Core into academic instruction. This led us to develop instructionalroutines that integrate the Universal Core in ELA and mathematics.

continued...

10 of 15

In your opinion, what is the most significant factor or set of factors that contribute to thesuccessful implementation of your innovation?


The validation process is critical to successful implementation so that claims canbe made. Flexibility is critical. The importance of flexibility is revealed in the classroom.Teacher comfort level with technology is an obvious predictor ofimplementation….[h]owever, we have been surprised to see, repeatedly, thatteachers’ general creativity and adaptability directly impact effectiveimplementation and integration of our technology into the classroom.Reminding teachers that the technology (e.g., KTEK) is a TOOL that they can useto their advantage or disadvantage is critical.

Field Testing

The successful implementation of our technology tool was due to the followingfactors: 1. The need and usefulness of the app for students who are visuallyimpaired. 2. The availability and access to technical support. 3. The enthusiasmand anticipation in knowing this is a tool that will be available for use after thepilot testing is completed. Developing and implementing an innovation that isbeneficial and useful for a population can contribute to the project success. Strong stakeholder buy-in/support is a prerequisite for project success – in otherwords, finding enthusiastic champions at all levels of project implementation –classroom, school/district, state, and even national – and having these championswork simultaneously and in concert with each other as change agents to createsynergy and momentum – will ultimately define the breadth and depth of projectreach, outcome, and sustainability.

We tell teachers up front that it is critical to implement with fidelity and weprovide a teacher log that allows them to track implementation. They are alsoprovided with a checklist up front so they can determine if they are ready toimplement the program. Data has indicated that teachers are happier and morelikely to use the program when they have implemented the program with fidelity.[T]eachers that don’t implement with fidelity seem to not get the same positiveoutcomes and are not as engaged.


Having a strong team and division of labor. The most significant factors related to implementation are readiness for tool use(administrative support, content expertise underlying the technology, buy-in andself-efficacy toward practice).Using an iterative design and development that utilizes feedback from teachers andstudents. Then having the best project staff that can take this feedback andcontinually update the products so it reflects the flexibility that teachers need toalign with multiple standards.

Other

11 of 15


SURVEYQUESTION 10:


By what theme or topic would youcategorize this lesson/strategy?(This question refers back to Question9: Significant Factor.)

Fidelity of technology implementation (15.79%) (n=3)


N=19 responses

Total of 1), 2), and 3) = 78.95% (n=15)


Buy-in or support from key stakeholders and grant partners(15.79%) (n=3)


12 of 15

Training/Professional Development/Instructional CoachingThe technology was relatively easy to use, but additional training was required touse it well.

SURVEY QUESTION 9:

Another key lesson/strategy learned is the importance of creating an innovationthat is flexible, adaptable, changeable, and customizable so that it can meet theneeds of the local implementation context in which the innovation is being used.For instance, in our project, educators in the sites that we worked with seemed toappreciate the fact that we were giving them a curriculum that they couldenhance with their own ideas, knowledge, and instructional strategies based ontheir own teaching experiences. Allowing for flexibility in the innovationconfiguration of our curriculum has resulted in more user ownership of theintervention. One of the aspects about the Stepping-Up program we find most helpful is thetime allotted for quality development and refinement. This focus on truly "gettingit right" before scaling-up is extremely valuable for developing a useful, high-quality technology product that will fulfill its potential in the field. Our agile development process/strategy for the iterative development of thescience notebook teacher training, supports, resources, etc. has been so veryuseful. We have held and continue to schedule teacher focus groups and workingsessions with teachers as to the content, organization, materials, etc. needed. Thisco-design process supports good development in finding and developing whatteachers need for their students and teacher buy-in for the tool overall.


Add your own lesson/strategylearned during this project.


N=13 responses

Key Stakeholder Buy-InWe feel that it is critical to maintain positive relationships with teachers byensuring we are available for technical support and problem solving. We foundthat having a point person that the teachers can call and discuss issues that ariseis critical to buy-in, successful implementation, and sustainability. In addition,establishing good relationships with teachers also facilitates data collection, as itmakes it easier for project staff to contact teachers and receive data. If teachersknow that they are able to contact project staff after the implementation toreceive login access, etc., then they are also more likely to do so. It also can help inconducting future projects at those school sites.

Site RecruitmentWe've learned that it's important to set implementation criteria for site selectionprior to working with a school or district.

continued...

Conduct research to demonstrate the impact on student outcomes that arecritical and valued by state and district leadership. Share personal stories of howteachers and students are impacted by completing the curriculum. Data plusstories assist with implementation and sustainability.


13 of 15

Add your own lesson/strategy learned during this project.SURVEY QUESTION11: CONTINUED...

Get your software developers IN THE FIELD! --No one reads pop-up alerts. --Makeyour default settings good. --UDL is king (i.e., features designed for a few tend to begreat for many (in unexpected ways) --Teacher/management characteristics don’tchange just because technology is involved so anticipate the range you'll encounterin the classroom. We have learned about the elusive, disruptive nature of inconsistent wi-fi signals….[t]his can make the technology appear “not to work” correctly to teachers whensynching with the server is disrupted (even though the local system continues tooperate). In terms of accessibility, in order to truly design a product that is universallydesigned, a lot of usability testing must be done with teachers and students whouse alternate means of access.

Field Testing

I believe that one lesson that we still have not fully learned is that we must work inthe given infrastructure of a school or district in order for sustainability to occur.We must realize that our project serves one certain need or purpose, but theschool or district has to address multiple needs. To address their needs, schoolsand districts have developed committees and departments for curriculum andprofessional development….[w]hen a project is not working jointly with thecurriculum and professional development structures in a school or district,disconnects occur that can cause a doubling of efforts and a loss of sustainability.If a technology innovation does not become part of school or districtinfrastructure, it will not be sustained.

From the beginning of the project, we had to adjust our research expectations tofit the practical realities of working with schools. I strongly encourage youngergrantee cohorts to tackle issues of innovation configuration and fidelity as soon asthey can in their projects. From the beginning, grantees need to define whatfidelity means in regards to their own respective technology innovations, askingquestions such as the following: a) how do we ideally want users to use thetechnology, b) what range of implementation variance is allowed if any, c) whatwill the project lose or gain if variance is allowed, d) how do we measure suchvariance, and e) how can we enhance our innovation based on the variance weobserve in real-world settings?


Having a strong team and division of labor. The most significant factors related to implementation are readiness for tool use(administrative support, content expertise underlying the technology, buy-in andself-efficacy toward practice). Using an iterative design and development that utilizes feedback from teachersand students. Then having the best project staff that can take this feedback andcontinually update the products so it reflects the flexibility that teachers need toalign with multiple standards.

Other

14 of 15

Buy-in or support from key stakeholders and grant partners (27.27%)(n=3)

SURVEY QUESTION 12:


By what theme or topic would youcategorize this lesson/strategy?(This question refers back toQuestion 11: Your Addition.)



N=11 responses

Total of 1), 2), and 3) = 63.63%

Remaining 36.37% (n=7) of response choices weredistributed fairly evenly across other categories

15 of 15

Documents

Food for Thought - OSEP Ideas That Work · 2018. 8. 15. · This presentation and related EIT products were produced under the U.S. Department of Education (USDOE), Office of Special