UNIVERSITY OF HAWAII lIBRAR?

FACTORS INFLUENCING USE OF A

WEB-BASED COMMUNITY SPACE

FOR K-12 PROFESSIONAL DEVELOPMENT

A THESIS SUBMITTED TO THE GRADUATE DIVISION OF THEUNIVERSITY OF HAWAI'I IN PARTIAL FULFILLMENT

OF THE REQUIREMENTS FOR THE DEGREE OF

MASTER OF SCIENCE

IN

INFORMATION AND COMPUTER SCIENCES

DECEMBER 2003

ByWilliam E. J. Doane

Thesis Committee:

Daniel D. Suthers, ChairpersonViolet H. HaradaLinda Johnsrud

© 2003, William E. J. Doane

iii

DEDICATION

I dedicate this effort to all the members of my extended family, on who's shoulders rest

the responsibility for making me everything that I am... much to their chagrin.

Thank you.

IV

v

ACKNOWLEDGEMENTS

This study would not have been possible without the support and hard work of the

graduate research assistants, Hawai'i Department of Education staff, educators, and of

course the professors associated with Hawai'i Networked Learning Communities.

The project reported on in this study was supported by the National Science

Foundation under Grant No. 0100393. Any opinions, findings, and conclusions or

recommendations expressed in this material are those of the author and do not necessarily

reflect the views of the National Science Foundation.

vi

ABSTRACT

This study examines participation in an online community space developed to

support K-12 professional development efforts conducted as part of a National Science

Foundation Rural Systemic lnitiative-Hawai'i Networked Learning Communities

(HNLC). HNLC was designed to contribute to science, mathematics, engineering, and

technology system reform efforts of the Hawai'i Department of Education. HNLC's

website, hnle.org, was designed to support the HNLC initiative and serve HNLC

participants as an online community space.

Results indicate that participants have not adopted hnle.org as a tool for their own

use, and are instead using hnle.org almost exclusively during highly structured

workshops where hnle.org use is required. Active participation in the online community

is rare.

Participant interviews and focus groups revealed that there exists a disconnect

between the stated needs of educators and the offerings of hnle.org. Participants were

primarily concerned with uses of technology by their students, while hnle.org is targeted

to educators' own professional development efforts.

vii

TABLE OF CONTENTS

Dedication ivAcknowledgements vAbstract. viTable of Contents viiList of Tables .ixList of Figures xList of Abbreviations xiChapter 1. Introduction 1

Purpose of this Study .4Research Questions 5Significance of this Study 6

Chapter 2. Literature Review 8Systemic Reform 8Education Reform 9

Necessary conditions for successful educational reform 10Barriers to successful educational reform 11

Professional Development 12Necessary conditions for successful professional development 14Barriers to successful professional development 14

Communities of Practice 15Technology in Education 16Information and Communications Technologies and Professional Development.. 19Summary 20

Chapter 3: Hawai'i Networked Learning Communities 22Background 22Purpose 22HNLC Summer Institute 24The HNLC Website: hnlc.org 24

Evolution of the homepage 26Unit plan template 30Discussion areas 32Story engine 34Resource database 37Technologies used 37

Summary 38Chapter 4. Methods 39

Participants 39Procedures 41

Vlll

Summer Institute Survey 41Server Logs 42Focus Group 44Interviews 45

Limitations of the Study 45Chapter 5. Results 47

Summer Institute Survey 48Profile of Survey Respondents 48

Server Logs 64Focus Group 69

The role of hnlc.org in supporting the HNLC initiative 74The role of hnlc.org in facilitating an active, online community of educators 74

Interviews 75Chapter 6. Discussion 77

Challenges 79What challenge is being addressed? 79Is technology the best solution to the challenge? 80How will success of the solution be evaluated? 80Is the solution consistent with teachers' goals? 81Does the solution bring with it such a positive change that teachers will adopt it? .. 81

Recommendations 81Conclusion 83

Appendix A. NSF's Drivers for Systemic Reform 85Driver 1 85Driver 2 85Driver 3 85Driver 4 86Driver 5 86Driver 6 86

Appendix B. Summer Institute Survey 87Appendix C. Sample Web Server Access Log 98

Typical Log Entries 98Log Entries Removed 98

Appendix D. Focus Group Discussion Guidelines 100Morning Session Focus Topics 100

Inquiry learning projects at their school 100Standards implementation 101Technology integration with teaching (curriculum) 101

Appendix E. Interview Questions 103Appendix F. HNLC COHORT Schools 105References 106

ix

LIST OF TABLES

Table I: Demographics: Primary Role of Respondents Within Their Schools .49

Table 2: Demographics: Number of years as an educator, time with HNLC, and nature of

involvement with HNLC. 50

Table 3: Respondent attitudes toward hnlc.org 52

Table 4: Responses to open ended questions about hnlc.org 53

Table 5: Frequency of use of computers for in-class instruction 54

Table 6: Respondent attitudes toward technology use in teaching 55

Table 7: Frequency of use of computers in-class by students 56

Table 8: Respondent attitudes toward technology use by students 57

Table 9: Frequency of use of severallCTs to support professional development.. 59

Table 10: Respondent attitudes toward technology use for professional development.. .. 61

Table 11: Responses to open ended questions about technology use for professional

development 62

Table 12: X2 analysis of attitudinal survey questions 63

Table 13: Monthly use of hnlc.org website 67

Table 14: Summary of issues raised by focus group attendees 71

x

LIST OF FIGURES

Figure 1: The first generation hnlc.org homepage 26

Figure 2: The second generation hnlc.org homepage, including a login panel. 27

Figure 3: The third generation hnlc.org homepage, including a dynamic story engine and

quick links to popular items (right-hand side, below the login panel) 29

Figure 4: The Unit Description form of the unit plan template 31

Figure 5: Opening section of the printable version of a unit plan 32

Figure 6: A typical Kiikakiika screen. The artifact of interest is displayed on the left;

discussion threads are shown on the top-right; and individual message are displayed

on the bottom-right. 33

Figure 7: Alu - the newer discussion area tool currently under development. 34

Figure 8: The story engine's story submission form 36

Figure 9: The number of unique visits to http://hnlc.org from 8/2002 - 8/2003. Also,

shown in black are the number of verified HNLC participant logins from 112003 -

8/2003 68

Figure 10: The number of verified HNLC participant logins (black) and verified HNLC

participant logins excluding web traffic from the HIDOE ATRB network. 69

xi

LIST OF ABBREVIATIONS

ATRB Advanced Technology Research Branch of the Hawai'i

Department of Education

AY American Academic Year; For K-12, usually taken to mean

September - May

CoP Community of Practice

HCPS/HCPS II Hawai'i Content Performance Standards

HIDOE. Hawai'i Department of Education

HNLC Hawai'i Networked Learning Communities, an NSF funded RSI

hnlc.org HNLC's website: http://hnlc.org

ICT Information and Communications Technologies

IP Internet Protocol

K-12 USA school grade levels, kindergarten through 12th grade level

NSF United States of America's National Science Foundation

RSI.. NSF's Rural Systemic Initiative

SMET Science, Mathematics, Engineering, and Technology

UHM University of Hawai'i - Manoa ,located in Honolulu on the island

ofO'ahu

URI NSF's Urban Systemic Reform Initiative

1

CHAPTER 1. INTRODUCTION

During the last decade, infonnation and communication technologies (lCT)' have

affected hundreds-of-millions of people around the world who have embedded the use of

lCT into both their personal and professional daily lives. In particular, computer networks

have facilitated new ways for computer users to interact with one another and exchange

infonnation by enabling such technologies as email, instant messaging, newsgroups,

electronic bulletin boards, and more recently the World Wide Web.

There is increased use of ICT to support or even supplant traditional educational

experiences. Universities and governmental organizations are investing in and depending

upon online delivery of information, courses, and other content targeted toward not only

K-12 students, but also toward training of educators (Means, 1998; Robinson & Latchem,

2003a). In addition to the development of online courses intended to meet initial teacher

training requirements and continuing certification requirements (Craig & Perraton, 2003),

there are also efforts to establish virtual communities of practice2 in an effort to

disseminate and promote best practices among educators (Schlager & Fusco, 2003). In

1 For the purposes of this study, Ief includes any and all computer software, hardware,and networking technologies, especially as applied to educational settings, as defined byMoursund, 2003.2 "Communities of practice are groups of people who share a concern, a set of problems,or a passion about a topic, and who deepen their knowledge and expertise in this area byinteracting on an ongoing basis ... These people don't necessarily work together everyday, but they meet because they find value in their interactions." (Wenger et aI., 2002:4­5)

2

part, this trend has been driven by the need to address the ever-increasing shortage of

certified teachers, the ever-decreasing budget and resource allocations, and the rapidly

changing requirements educators must meet (Robinson & Latchem, 2003a).

ICT allows users to interact across distances and time in ways not previously

possible. As such, educators who may not be able to gather in the same place or even

interact at the same time are able to interact using computer networks in synchronous and

asynchronous modes of communication. leT allows educators who are already burdened

by their daily job duties to engage with a like-minded community at a time and a place of

their own choosing.

This study examines Hawai'i Networked Learning Communities (HNLC)-an on

going, five year initiative intended to improve student learning in science, mathematics,

engineering, and technology (SMET) in the state of Hawai'i. In particular, this study

considers the first two years of development of the HNLC website, hnlc.org- an effort

to develop innovative leT tools to support the HNLC initiative and serve HNLC

participants as an online community space. HNLC is a National Science Foundation

(NSF) funded Rural Systemic Initiative (RSI) located in the state of Hawai'i. The NSF-

RSIs began in 1994 as an effort to support educational reform of science, mathematics,

and technology programs in economically disadvantaged, rural regions of the United

States of America. The goals of the RSIs are:

• "The improvement of science, mathematics and technology education inrural, economically disadvantaged regions of the nation, including, but notlimited to, access to high quality, standards-based instruction, innovativeuse of educational technologies for interactive delivery of instruction, and

3

the training of the teaching workforce to meet the demands of a newinstructional paradigm;

•

•

•

•

"Increased student achievement in the fields of science, mathematics, andtechnology, as measured by higher scores on standards-based assessments,increasing enrollment in higher level courses, and greater articulation toinstitutions of higher education;

''The preparation of a technologically competent workforce, bystrengthening the science, mathematics and technology instructionalcapacities in K-12 schools, through partnerships with two-year and four­year institutions of higher education, particularly as it relates to technicianeducation, lower division instruction of technical curricula, and scienceand mathematics instruction of the future teaching workforce;

"The enhancement of scientific understanding and appreciation amongstudents and the general community in rural, economically disadvantagedregions of the nation; and

''The development of community infrastructure to provide resources tosustain educational improvements, including education policy andeconomic development, governmental commitment, resource reallocation,and community support and involvement in rural schools and districts."

(National Science Foundation, 20(0)

HNLC is a collaborative effort of the Hawai'i Department of Education (HIDOE)

and researchers at the University of Hawai'i - Manoa (UHM). Begun in 2001, HNLC's

principal goal is to " ... prepare students in economically disadvantaged rural schools for

life and careers in today's complex and dynamic technological world by enabling them to

attain high standards in science, mathematics, engineering and technology (SMET)"

(Harada, et aI., 2001). In order to achieve this goal, HNLC was designed with the

intention of:

• "Creating a high quality teaching and learning environment in SMETeducation for over 60 K-12 rural schools in Hawai'i.

4

• "Supporting and strengthening state policies that impact theimplementation of a standards-based approach to content learning andstudent achievement.

• "Aligning and converging resources to build networked learningcommunities that support systemic reform efforts.

• "Increasing commitment and support from various stakeholder groups tobuild and maintain SMET programs.

• "Developing an assessment and accountability system that accurately andcomprehensively measures students' achievement in SMET.

• "Ensuring that [HNLC] efforts enrich the education of all members of[HNLC's1diverse population, particularly native Hawaiians."

(Harada, et aI., 2001, emphasis added)

Of particular import for this study is the notion of a networked learning

community. The major effort of the UHM HNLC development team to date has been the

creation of hnlc.org, which is intended to support the community of educators and other

stakeholders who are participating in the HNLC initiative. To this end, hnlc.org provides

online tools intended to support the professional development program developed by the

HIDOE team. The HIDOE team is charged with the task of:

"Developing and implementing a multi-strand, professional developmentprogram that targets various groups: master teachers serving as teacherpartners, building administrators, and school teams comprised of teachers,technology coordinators, and library media specialists. Major foci in thetraining will be alignment of curriculum with state-adopted content andperformance standards; infusion of a rigorous inquiry approach to scienceand mathematics learning; application of technology in instruction; anduse of alternative assessment strategies." (Harada et aI., 2001).

Purpose of this Study

This study considers one aspect of the HNLC initiative, the hnlc.org website,

intended to provide an online community space to support HNLC educators' professional

•

•

5

development and curricular planning efforts. The hnJc.org website provides users with

access to several tools including:

• a curricular unit planning template: an HTML forms-based template to aid in

the design and construction of unit plans;

• an artifact centered discnssion space: an enhanced threaded discussion

webpage where users' postings are linked to web content relevant to the posting

(Suthers & Xu, 2002);

a community-driven news story engine: a dynamic entry point to the website

that allows users to post short articles of interest to the HNLC community; and

a database of educational resources: a collection of links to websites,

organizations, materials, and individuals relevant to science, mathematics, and

technology curricula in Hawai' i.

The HNLC initiative and the hnlc.org website are described in detail in Chapter 3.

This study will describe the evolution of the hnlc.org homepage, evaluate the

frequency of use of the site, and assess participants' attitudes toward uses of online

technologies for teaching, learning, and professional development as those attitudes relate

to the nsage of hnlc.org.

Research Questions

This study is foremost a study of the hnlc.org website and the degree to which it

has been successful in satisfying its goals during the 2002-2003 academic year, namely

to:

•

•

facilitate an active', online community of educators; and

support the HIDOE team's efforts.

Secondarily, the present work should help guide future policy by gauging the

6

"success" of hnlc.org as defined by answers to the following questions:

•

•

•

•

What is the frequency of use of hnlc.org?

What is the frequency of use of hnlc.org by logged in HNLC participants?

Has frequency of use of hnlc.org increased over the life of the initiative?

What tools are used the most by HNLC participants, and under what

circumstances or for what purposes?

• What factors influence whether these tools are used in a manner supportive of

professional development and educators' communities of practice?

• How do participants' attitudes regarding technology use for (a) teaching, (b)

learning, or (c) professional development impact use of hnlc.org?

Significance of this Study

Due in part to the ongoing shortage of certified teachers, the demand for improved

performance of students with respect to both federal and state academic standards, and

the expectation that classroom teachers will develop standards-aligned curricula, leT is

being used increasingly in both initial teacher training and continuing professional

development efforts (Craig & Perraton, 2003; Robinson & Latchem, 2003a). However,

, Active is defined as project participants' use of the website and tools made available tothem.

7

the lack of well grounded research on educators' expectations of online environments,

educators' predisposition to use ICT, or educators' perceptions of ICT as a tool for

learning, both for themselves and for their students, undermines the success of such

efforts (Kent & McNergney, 1999; Robinson & Latchem, 2003a).

The hnle.org website is an example of an attempt to support educational reform

efforts by using ICT to directly support the work life of educators. This study is an effort

to deepen our understanding of hnlc.org. As such, this study will explore and identify

issues to be addressed in further research on hnle.org as well as serve as a model for

study of other efforts that have similar goals and methods. This study also has practical

implications for future hnle.org development in that it serves as an evaluation of the

effectiveness of hnle.org to date. Additionally, this study strives to contribute to the

broader literature on systemic reform efforts and ICT use in education by detailing one

such initiative.

8

CHAPTER 2. LITERATURE REVIEW

The present study considers hnlc.org - a website intended to support systemic

reform efforts undertaken under the auspices of the HNLC initiative. A wide range of

issues are relevant to the discussion of HNLC and hnlc.org, including such topics as

systemic reform efforts, education reform, professional development, educators'

communities of practice, and use of technology in educational settings.

Systemic Reform

The systemic reform movement is a result of systems theory originally proposed

by Ludwig von Bertalanffy in the 1940s (Stalinski, 2000; Webster, 2001).

Fundamentally, systems theory attempts to achieve better understanding of a

phenomenon by considering it as a whole. Rather than focus on rarified portions of the

phenomenon of interest and attempting to deduce cause-and-effect relationships between

minimal variable sets, systems theory encourages one to take a holistic view and to

consider not only the constituent parts of the phenomenon, but also the rich context in

which it takes place.

Peter Senge popularized systems theory as a corporate management model for the

so-called "learning organization" (Senge, 1990). He proposed "systems thinking,"

developed by Jay Forrester at the Massachusetts Institute of Technology, as a framework

for corporate behavior. Systems thinking is intended to bring all of the behaviors of a

complex organization into alignment in order to ensure that the organization-wide goals

9

are realized and reinforced at every level (Senge, 1990). By applying systems thinking

principles, Senge suggests, the possibility of positive organizational change is

maximized.

These notions of systems theory and systems thinking have been carried over into

the field of education in the form of system-wide or "systemic" reform efforts-attempts

to realize the benefits of systems thinking for America's schools, the foremost learning

organizations. Systemic reform efforts seek to ensure that all policy decisions, actions,

and initiatives intended to bring about change within the organization contribute to a

shared, focused vision. Systemic reform efforts apply the core tenets of systems thinking

to educational contexts, namely:

• building of shared visions of change;• making mental models of core processes explicit;• fostering team learning;• requiring personal mastery of new processes and practices.(Senge, 1990)

Education Reform

Formal education is a primary mechanism by which societies propagate their

collected knowledge and collective values. As such, all segments of a society have a

vested interest in having their particular views and experiences represented in the

education system.

In any pluralistic society, the inherent differences in perspectives and moral and

ethical frameworks will, over time, cause shifts in the accepted dominant views of that

society. With these shifts come changing priorities and expectations of the education

10

system- that is, as societal norms change, so too change society's expectations of the

education system to reinforce those goals. As long as the notions of educational practice

and curricula have existed, they have been the subjects of reform efforts.

Educational reform, however, is not a simple concept to be stated and adopted

with immediate effect. Rather, any desired change in educational practice or curricula

must gain acceptance among policy makers, be adopted by educational practitioners, and

bring with it actual changes in student learning in order to provide a lasting benefit to the

society.

Necessary conditions for successful educational reform

Given the complexity and import of educational reform, it is not only reasonable,

but also prudent to consider what factors influence the success or failure of reform and

school improvement efforts. Researchers have identified several critical factors necessary

to achieve lasting school improvement:

• "Participation: any change requires teachers to participate in planning anddecision-making.

• "Commitment: teachers need to be committed to the change as they will beinstrumental in implementing it.

• "Pressure and support: change requires the pressure for change to occurand the technical/emotional support to ensure that change happens.

• "External agency: assistance from external sources such as consultants.• "Staffdevelopment: staff development activities need to be put in place to

provide ongoing support for the new programme. The staff developmentactivities have to be task-specific and geared to teachers' concerns andskills."

(Harris, 2002:19, emphasis present in original)

11

Harris (2000, as cited in Harris 2002) also conducted an analysis of successful

school improvement programs to determine what common features they possessed. He

reports that successful programs:

•••

•

•

•••

"focus closely on classroom improvement;" ... are explicit in the models of teaching they prescribe;"apply pressure at the implementation stage to ensure adherence to theprogramme;"collect systematic evaluative evidence about the impact upon schools andclassrooms;"mobilise change at a number of levels within the organisation, e.g.,classroom, department, teacher level;"generate cultural as well as structural change;"engage teachers in professional dialogue and development;"provide external agency and support."

Fullan (\991, as cited in Harris 2002:42) has identified a similar set of

requirements for successful change to occur, including:

••••••

clarity of purpose;shared purpose;clear outcomes;forward planning;external support; andevaluative feedback.

Barriers to successful educational reform

However, the presence of some or all of the above factors does not assure success.

Harris (2002:19) further identifies factors that can hinder successful implementation of

improvement efforts:

•••

unclear purposes and goals;competing priorities;lack of support;

••

insufficient attention to implementation; andinadequate leadership.

Blumenfeld, et al. (2000) have identified from the literature three issues that

12

prevent lasting change. First, reforms that are inconsistent with existing local norms or

practices are easily rejected or subverted. Second, educators' understanding of the reform

effort and existing expertise needed to adopt the reform methods impact their reaction to

the reform effort. Third, failure to establish policies and management practices that

support the reform effort can undermine the effort.

Harris (2002) concludes that in order to ensure that positive factors for successful

school change will exist, and in order to mitigate or eliminate any negative factors

present, a successful professional development program must be a central component of

reform efforts.

Professional Development

Any reform effort wishing to succeed must consider the central role of

professional development. If the goal of reform is to impact student outcomes-whether

to deepen or broaden content knowledge, strive for equity, or improve results on norm-

referenced criterion- then critical to attaining that goal is the practice of teachers

(Cuban, 1993). Teachers are the conduits through which all educational reforms must

eventually flow. They guide student learning, enable classroom dynamics, and help

establish students' expectations of schooling.

13

Nevertheless, there is growing acknowledgement that traditional professional

development efforts have not had the intended positive impact on professional behavior

or student learning. There is little lasting impact from brief professional development

seminars, and professional development opportunities often reflect priorities driven by

rapidly shifting political forces from outside the professional's daily educational setting

(Schlager & Fusco, 2003).

Bredeson (2003) suggests that professional development programs need to shift

from being considered add-ons to an educator's job to being considered essential aspects.

Also, professional development should shift from activity-centered experiences to being

linked to real practice and student learning. These shifts connect professional

development with the daily practice of teaching as experienced by educators.

Professional development opportunities should allow teachers to share strategies

for teaching, to collaborate on new and interesting projects, and to collectively reflect on

their practice. Schlager & Fusco (2003:205) characterize the vision of professional

development put forth by the National Commission on Mathematics and Science

Teaching for the 21" Century (2000):

"Professional development is viewed as a career-long, context-specific,continuous endeavor that is guided by standards, grounded in the teacher'sown work, focused on student learning, and tailored to the teacher's stageof career development. Its objective is to develop, implement, and sharepractices, knowledge, and values that address the needs of all students."

Increasingly, professional development is being viewed as a life-long learning

process for educators, involving on-going updating not only of the educator's subject area

14

knowledge, but also broadened teaching methods, increased technological capabilities,

and improved understanding of current trends in educational reform.

Necessary conditions for successful professional development

What are the characteristics of good professional development programs? Borasi

& Fonzi (2002: 29-31) suggest that to be successful, programs should:

•••••

be sustained and intensive,be informed by research on how people learn best,focus on the critical activities of teaching and learning,promote collaboration, andoffer diverse professional development experiences.

Garet, et al. (2001) report similar findings stating that effective professional

development efforts should:

•••

•

be sustained and intensive (on the order of years);focus on real academic subject matter;give teachers the opportunity to do "hands-on" work as part of theprofessional development activity; andbe integrated into the daily life of the school.

While there is agreement in the literature on these key requirements (Darling-

Hammond & Ball, 1997; Schlager & Fusco, 2003), there is also concern that professional

development efforts are easily undermined (Schlager & Fusco, 2003; Hargreaves, 1994).

Barriers to successfulprofessional development

Simply having professional development opportunities available does not

necessarily mean that such opportunities will have positive impact on reform efforts.

Hargreaves (1994) warns that an atmosphere of contrived collegiality, present when

15

professional development activities are mandated and strictly controlled, undermines the

very purpose of professional development efforts. Nolan & Meister (2000) summarize

Hargreaves' argument:

"Hargreaves notes that collaborative cultures are working relationshipsthat are spontaneous, voluntary, development-oriented, fixed in time andspace, and predictable... [Hargreaves1submits that the following criteriacan create contrived collegiality. First, teachers are required to meet andwork together, which makes the collaboration administratively regulatedand compulsory. Second, teachers are required to implement the mandatesof others rather than programs decided on through discussion. Third,teachers only meet at the particular time and place that has been mandatedby the administration. Fourth, the collegiality is designed to have a highpredictability of outcomes since the administration has control over itspurpose and regulates its time and place."

Communities of Practice

In order to foster the necessary conditions for successful professional

development and reform efforts, some researchers believe that the notion of Communities

of Practice (CoP) should be applied to teacher education. Sarab et at. (2003:238) define a

CoP as: "a persistent, sustained social network of individuals who share and develop an

overlapping knowledge base, set of beliefs, values, history, and experiences focused on a

common practice and/or mutual enterprise." McLaughlin & Talbert describe CoPs as:

" ...collaborative and inclusive. Teachers take a collective stance on the issue of teaching

expertise, seeing one another as resources for their improved practice" (2001: 73).

However, Schlager & Fusco (2003) suggest that true CoPs cannot be formed by

external agency, but rather must evolve over time out of teachers' needs and intrinsic

16

motivations. This poses a challenge to any reform effort that relies upon or expects the

creation of new CoP.

Additionally, McLaughlin & Talbert (200l) note that some traditional attitudes

about the role and professional status of educators may undermine the formation of CoP.

For example, there is a long-standing tradition of teacher autonomy and personal

ownership over one's teaching practice. This tradition of individualism may discourage

honest criticism of others' teaching methods, the sharing of lesson plans, and perceived

best practices. Such prior self-restraint prevents a CoP from realizing its full potential,

since members cannot benefit from one another's expertise.

Despite the challenges for educational communities of practice, McLaughlin &

Mitra (2001, as cited in Schlager & Fusco, 2003) argue that sustaining reform efforts

"requires a community of practice to provide support, deflect challenges from the broader

environment, and furnish the feedback and encouragement essential to going deeper."

Schlager & Fusco (2003) are concerned with efforts to support educators' communities of

practice with the use of technology, such as Tappedln.org, an online community of

educators established in 1997.

Technology in Education

A critical component of current systemic reform efforts in science and

mathematics is the attempt to use technology both in order to promote change and to

foster scientific thinking (Ferneding, 2003). Use of technology in education has

frequently been touted as a panacea for many of the ills of the educational system. By

17

bringing technology into the classroom, it has been argued, students will learn better and

more quickly, teachers' burdens will be lightened, and learning experiences will be richer

and more meaningful. Such claims have been made concerning such technologies as

radio, television, film, and laser disc (Cuban, 1986).

However, Cuban (2001) notes that only a few of the technologies introduced into

the classroom during the past century have been widely adopted by educators - and those

adopted have been of the simplest form: overhead projectors and whiteboards, for

example. He states that simply making technology available in the classroom rarely

changes the way teachers teach.

Educators bring differing levels of technology-savvy and differing views of the

place of leT in education. Some view the World Wide Web as only a research tool for

themselves, to be browsed for pre-existing content to support their teaching practice. For

example, they may attempt to locate pre-existing curricular units that they can then adapt

and adopt for use in their own context. Others view the Web as a research tool for use by

their students, effectively treating use of the Web by students as a trip to the library, an

effort to collect images and information to be included in a student report or presentation.

R. E. Anderson & Ronnkvist (1999, as cited in Fishman, et aI., in press) report that,

despite years of aggressive funding efforts, the primary uses of technology in America's

public schools are still for drill and practice activities, word processing, and surfing the

web.

18

However, some view the Web as a medium for active communication, as a place

where both students and educators can seek advice and discuss common issues of interest

to their community of peers (Saint-Onge & Wallace, 2003; Robinson & Latchem, 2003a).

In this mode, leT addresses issues of isolation, promotes innovation, and increases the

speed at which new ideas can be assimilated and disseminated (Saint-Onge & Wallace,

2003: 68-73).

Some researchers have noted that teachers tend to teach using the same methods

they were taught with during their initial teacher training. This has been cited as a

contributing factor for the on-going lack of integration of technology into the classroom,

despite the high level of availability of technology for instructional use (Cuban, 2001;

Bell & Tai, 2003).

Attempting to identify successful avenues for technology adoption, Bell & Tai

(2003: 13) suggest that:

"As the new teacher workforce becomes more and more technologycapable, innovative uses of technology will enter the classroom not byway of a top-down model of professional development, but through amore grassroots model of hiring technology-capable teachers who canthen begin to influence their peers by modeling innovative technology useand acting as resources."

If there is to be a high reliance on leT as a tool for in-service professional

development efforts, this suggests that, in addition to modeling the use of technology for

instruction, teacher training programs should be modeling the use of leT to support

professional development communities from an early stage in pre-service training.

19

Fishman, et al. (in press) and Blumenfeld, et al. (2000) recently reported on

significant research regarding technology use and adoption of innovations as they relate

to systemic reform efforts. They suggest that a key factor to successful and sustained

adoption of technology in the classroom relies critically upon researchers' understanding

of how technology is perceived and supported at all levels of school organizations-as a

part of the school culture, in relation to existing educators' capabilities, and with respect

to school policy and management structures. Fishman, et al. propose a broad agenda for

research into the scalability of instructional innovations, the sustainability of such

innovations, and the usability of technology innovations in the context of systemic reform

efforts.

Information and Commnnications Technologies and Professional Development

With respect to ICT use in support of professional development efforts, Robinson

& Latchem (2003a) make a strong case that ICT enables teacher training to be successful

in circumstances where non-ICT methods are not possible, not cost effective, or not time

effective. For example, ICT allows geographically isolated teachers to communicate with

other teachers, content specialists, and mentors. leT may allow for faster propagation of

information about changing certification requirements, changing standards, and available

professional development opportunities. ICT can also be a tool for curriculum reform in

that ICT can allow educators to access best practices, curricular unit plans, and other

planning resources on a just-in-time basis. As such, leT can and does play an increasing

role in the support infrastructure for professional educators.

20

Craig & Perraton (2003), however, note that in addition to the significant

logistical difficulties of providing technology-based continuing professional

development, one must also address the generally lesser access to technology found in

rural schools, as compared to their urban counterparts. Also, they note the continuing

stigma that distance education is a lower quality alternative residential educational

options.

Necessary considerations for planning a successful Ier-based professional

development initiative as identified by Robinson & Latchem (2003b) include:

••

•

•••

Make sure that ... distance education is the right solution to the problem.Work within the relevant policy framework so that the [program] fits itsnational context and, where appropriate, its regional or international one.Establish base-line data on the extent and profile of needs and the natureof the target audience.Decide what learning outcomes are intended.Identify clearly the purpose and intended outcomes of the training.Plan an appropriate quality-assurance framework.

Summary

Efforts to reform and improve education have recently adopted systems theory as

a framework for conceptualizing successful education reform. While systemic reform

requires consideration of the reform effort as a whole, there are several elements that are

generally agreed to strongly influence the success of such reform efforts, namely:

••••

a successful professional development program;encouragement of successful Communities of Practice;overcoming barriers to uses of technology in education; andintegrating technology use with professional development efforts.

21

These are no small challenges and successfully meeting them requires a

commitment to high quality leadership, shared vision, and honest evaluation. While

systemic reform is elusive, the existing literature suggests that failure to attend to these

critical components will almost certainly undermine, if not entirely negate, reform efforts.

This study examines a specific use of technology (i.e., creating a web-based

community space), for supporting a systemic reform initiative. The research cited and

summarized in this chapter provides a critical framework for examining both the online

tools created and the factors influencing participants' use of these tools.

22

CHAPTER 3: HAWAI'I NETWORKED LEARNING

COMMUNITIES

Background

The National Science Foundation (NSF) began funding systemic reform efforts in

the early 1990s. The NSF established Statewide Systemic Initiatives (SSIs) to improve

mathematics and science education by effecting systemic change in areas such as

educational policy making and teacher training. NSF adopted the Drivers for Systemic

Reform (Appendix A) in order to identify issues critical for the success of systemic

reform efforts and to provide a consistent evaluation framework for NSF funded systemic

reform initiatives.

Realizing that the effects of SSIs were not reaching all students equally, in 1993

the NSF began funding Urban Systemic Reform Initiatives (URIs) in order to target 28 of

the poorest urban areas in the United States. In 1994, this effort was extended to create

the Rural System Initiatives (RSIs), intended to reach rural areas with extremely high

poverty levels, which had been largely unaddressed by SSI or URI efforts. Currently,

there are 30 funded RSls, including the Hawai'i Networked Learning Communities

initiative.

Purpose

Hawai'i Networked Learning Communities (HNLC) is an NSF funded RSI

serving the state of Hawai'i. HNLC was initially funded by an NSF development grant in

23

1999 and later received a five-year implementation grant in 2001. Headed by the Hawai'i

State Department of Education (HlDOE) and the University of Hawai'i - Manoa

(UHM), HNLC's goal is to improve students' learning of science, mathematics,

engineering, and technology (SMET) in rural, economically disadvantaged schools by

supporting standards-based, systemic reform efforts. In order to realize this goal, HNLC

was created with the intention of:

• creating high quality SMET teaching and learning environments;• supporting state policies implementing standards-based approach to

content learning and student achievement;• bringing together resources to build networked learning communities that

support systemic reform efforts; and• developing an assessment and accountability system that measures student

achievement in SMET.(Harada, et al., 2001)

In 1999, during the development phase of the initiative, a needs assessment was

conducted with the following findings:

•

•

•

•

•

•

•

The schools were unable to provide ethnic breakdowns on achievementdata and post graduation plans;Schools are aware of the Hawai'i Content and Performance Standards, butare not sure how to implement the standards;Professional development focuses on standards, but does not emphasizecontent knowledge;Instructional practices are largely traditional (lecture and textbook based)in science and math, but more hands-on in technology;Learning situations are largely whole group and individual; small grouplearning, which can be highly effective, is rarely used (Webb & Palincsar,1996);The use of local resources, experts as mentors, community volunteers inschools, and other community partnerships is low in SMET, particularly intechnology;Use of technology is weak or nonexistent in science and math instruction,and occasional at best for access to outside resources, professionaldevelopment, or mentoring;

24

• Parents and teachers view SMET careers for their children as desirable,but not likely.

(Harada, et aI., 2001)

Additional funding was received in 2001 in order to proceed with the

implementation phase of the HNLC initiative. In order to support the implementation

phase, the UHM team has provided software development support for the HIDOE

professional development program and has been the primary architect and maintainer of

the HNLC website, hnIc.org. The website serves as the focal point for HNLC online

activities providing a central news source and advanced tools intended to support

collaboration among educators and professional development efforts.

HNLC Summer Institute

In June, 2002 and again in June, 2003, HNLC participants were brought together

as a group for the HNLC Summer Institute. These institutes were designed to offer

training related to HNLC's goals to participants and to provide a forum for HNLC

participants to offer feedback to both the HIDOE and UHM teams. The main agenda was

conceived of and implemented by the HIDOE team with input from the UHM team.

UHM HNLC staff members were in attendance to provide technical support and to

conduct training sessions on newly developed or refined hnIc.org tools, described below.

The HNLC Website: hnlc.org

The hnIc.org website serves as the online presence of the HNLC initiative. Much

of the site is open to the general public and allows users to locate information about the

25

background and current status of HNLC. Other portions of the site offer content and

capabilities only available to HNLC participants and invited users. These users access

restricted pages by entering a unique username and password provided to them. Users

who wish to have an account on hnlc.org can complete an online form for review by

HNLC administrators.

Restricted webpages allow authenticated users to access interactive tools and

databases designed to support the HNLC mission. These tools include:

• Story engine: designed to allow HNLC participants to post stories believed to be

of interest to other members of the community - notices about HNLC activities,

upcoming funding opportunities, and ongoing research, for example;

• Resource database: a collection of websites, organizations, and materials

believed to be useful to support SMET education, with an emphasis on Hawai'i­

based resources;

• Unit plan template: designed to support the process of developing, sharing, and

discussing curricular unit plans;

• Discussion areas: designed as an artifact centered, threaded discussion space, the

hnlc.org discussion areas allow participants to hold online conversations while

viewing web content linked to the discussion.

The story engine and resource database have both a public and a participant-only

aspect. Each tool allows any visitor to the HNLC site to read existing content, but only

registered hnlc.org users are able to submit new content for inclusion on the website.

26

Evolution ofthe homepage

The hnlc.org homepage has been revised several times since 2001 both to support

upgrades to the services provided on the hnlc.org site and to provide more opportunities

for participants to exchange ideas and to provide substantive content for website. The

first generation homepage was a static HTML page stating the goals of HNLC. The first

generation homepage also included a short list of upcoming events that was maintained

by UHM HNLC staff and updated manually.

I'fo~ SlPff

COn'l('l'lun4\yl'iJrlNlr>

Aloha a Welcome

The Hawal'i Networked learning Communities (HNlC) Is a partnership of the HawlIl'lDeoartment of Education and the U"rversjtv at Hawal' [ to Improve science, mathematics andtechnology learning in K-12 rural schools.HNlC Is achieving this goal by:

• Fostering a standards approach to school improvement,

• Providing leadership development and online resources for school teams,

• Accessing Hawaii's rich ecological resources and cultural heritage to developmeaningful and relevant dassroom projects and studies.

HNlC Is a Rural SYStemic InitIative funded by the National Sdence Foundation.

Latest News

-~.~~~--~~..~~_...._.,.- ....._.._......~..._...- .._~

Cohort Schools meet with project: tellm

Teachers and administrators from nine schools designated as part of the first HNlCcohort met with project team members fora half-day meeting on Oahu. On theagenda: updates on the project, plans for the summer institute, and hands-onopportunities to examine the web site being created for the project. PartJdpantsused the Interactive discussion capabilities of the HNlC web site to post theircomments and their questions. SChools represented lnduded Hookena, Honokaa,Kaual High, Lanai, laupahoehoe, Maul Waena, waimea High, Waikoloa, and WaimeaClnyon.

Figure 1: The first generation hnlc.org homepage.

At the 2002 HNLC Summer Institute, the second generation of the hnlc.org

homepage was released. The primary purpose of the second generation homepage was to

Password

27

support user authentication for the purpose of accessing the unit plan template, which was

also introduced to participants at the 2002 institute and is discussed below. The second

generation page included links to static informational pages for non-participants who

wished to request an hnlc.org account or for existing users who forgot their account

password. Both of the static informational pages simply requested that the user contact

the HNLC staff by email or phone.

Aloha & Welcome IParticipant Login IThe Hawai'l Networked Leaming COmmunities (HNLC) Is a partnership of iUsemame Ithe Hawal I Department Qf Ed"Clltlon and the Unw!:Ciity of HjJWal' ! toimprove science, mathematics and technology leamlngln K-12 rural schools.HNLC is achieving this goal by:

• Fostering a standards approach to school improvement.

• Providing leadership development and Qnline resources for schoolteams.

~

~HM.CUam

• Accessing Hawaii's rich ecological resources and cultural heritageto develop meaningful and relevant dassroom projects andstudies.

HNLC Is a Rural Systemic Initiative funded by the National ScienceFoundation,

Latest News

06.2"26.2002 HNLC Summer lnstlt te on Ll!:llder5hIP DeveloPment

The summer Institute focused on developing leadership skills among HNLC schoolteams Indudlng processes and strategies to strengthen standards basedinstruction and inquiry Ieamlng approaches In science and mathematics. A fieldtrip and hands on experiences with technologies for learning was Incorporatedinto the institute. A special strand for school administrators centered on buildingInfrastructure suPlXlrt: for change.

More about this...

Figure 2: The second generation hnlc.org homepage, including a login panel.

28

Based on feedback from HNLC participants following the 2002 Summer institute,

additional links were added to the hnlc.org homepage to allow users quick access to

interactive tools described above, which were previously only available by clicking

through several pages. The "forgot your password" and "request an HNLC account" links

were also replaced with web-based forms pages allowing the user to enter their own

account information for processing by HNLC staff.

A significant change to the hnlc.org homepage occurred in the fall of 2002.

Reflecting on the goals of HNLC, the UHM HNLC development team agreed that the

hnlc.org site was intended to be community-focused with frequently changing content.

The UHM HNLC team realized that the homepage of hnlc.org should reflect the

community-focused nature of the initiative. In an effort to involve HNLC participants in

the creation of original content for the site and to promote active communication among

participants, the hnlc.org homepage was replaced with a dynamically generated, story­

based engine. Users who logged into the site could submit stories for inclusion on the

front page of hnlc.org and project staff could use the story-based engine to publish timely

information to the HNLC community.

Hawell'l Networked Leomfng CommunitiesAl'ir1ual_ity of eduGatorslllljl"'1rlIlg science, math, CI1d tcd_logy cdueatiDrl in rurm schools

About HNa..c

29

AbClUl. tJ"LC

DII.e~

"'0).<'<1 StarfCommllniLyP.rt~

.. IlINIQ! vi

~Rcillaa:b flOWOI'I

DWnmcnt Af~

B4ld

Hokule' a embarks on 5tah!wtde -educatlonal_1I1

submitted by Dan Sytbea on Frldlly, March 14, 2003

The Polynesian Voyaging SOCiety's vessel Hokuke' a begins astatewide sail March 15th to promote awareness of the ecologicalproblems we face. Take advantage of events for keikl and freeeducator's workshops! Read the story for more Information,

[ Read the full story.. I Vi"'" all stones in 'News !l Eveots' J

Online Dlsc:uplon Group

submltt<>d by Alan Nakagawa on Fridlly, March 14, 2003

An Invitation: be part of an Online Discussion Group!

The Harvard Smithsonian Center tor AstrophysicsEssential Science tor Teachers

.. PartJdpant Log n

I• Pa word

Forgotyour password?

Request anIHlliLC acrount

[ Read the full story•.. I Vi"'" all stones in 'Collaboration' J

Technology In Education Conferenu March i1-12th

submltt<>d by Dan SUthe... on Monclay, March 10, 2003

COme see innovative Ideas and projects that integrate technologyinto Instruction and build a global c:ommunity of learners!

[ Read the lull stOry... I View all stones In 'News &. E.ents' J

Legln tCol'nm_

Current Items

HNLC Discussions

ResQu rce Search

Unit Plans

Figure 3: The third generation hnlc.org homepage, including a dynamic storyengine and quick links to popular items (right-hand side, below the login panel).

The evolution of the hnlc.org homepage represents a progression in the

development team's conceptualization of the role of hnlc.org - namely, away from a

transmission of staff generated information and toward participant generated content.

Chapter 4 presents usage statistics that indicate the effectiveness of these changes.

In addition to the informational homepage, the hnlc.org site contains the

interactive tools described below.

30

Unit plan template

The unit plan template is designed to facilitate the process of developing

curricular units and identifying state content standards addressed by the unit. HNLC

participants were introdnced to the nnit plan template at the HNLC 2002 Summer

Institute. The template consists of eleven web-based forms, each of which allows the user

to enter information about one aspect of the unit plan. These eleven forms include: unit

description, unit designers (authors), grade level and curriculum areas addressed, Hawai'i

Content and Performance Standards (HCPS) addressed, assessments used, samples of

student work, high-level overview of the unit's goals, intended learning outcomes,

resources (organizations, websites, etc.) used, unit flow, and unit discussion (reflection on

the success of implementation of the unit). The sections of the unit plan can be revised by

any of the designated unit designers, and can be completed in any order. However, the

pages are sequenced to prompt authors to first identify the learning objectives of the unit

(in terms of the HCPS addressed) and how these objectives will be assessed before going

on to other details of unit plan implementation.

31

Jump to: ( UnitOeKrlption

Unit Descri tion

l1Ue of unit:

VISibility: Any registered HNLC partldpant can view this plan

• Only authorized editors of this plan can view It

If you have aeated a website relatingto this unit Plan, please provide the

URL:

Purpose of unit:

Unit dura on: 0

and

(Discard unsaved changes to this page'

",onlhs o : weeks and 0 • days

Q Check SpellInll

~save anctExlt~ ~Continue'» )

Figure 4: The Unit Description form of the unit plan template.

At any stage of completion, the user can choose to view a printable version of the

unit plan. Other hnlc.org users who are not designated designers of a given unit plan can

also view this printable version. From the printable version of the unit plan, users can link

to a discussion area to discuss the development of the unit with other users. A dedicated

discussion area exists for each unit plan, and is created dynamically the first time any

user attempts to enter the discussion for a given unit. (See discussion areas, below).

HNLC Unit Plan:f{E Discuss this unit plan

~ Printer friendIy fa rmat

32

UNIT DESCRIPTION

Title Computer Aided DraftingLink

Summary In the Computer-Aided Drafting mocluIe, the studentswill spend 8 days learning about the history,advantages, uses, and operations of computer-aideddrafting. Students will Interact with a CAD system,complete practice drUls/tutorials and drawingexercises, and produce hard ~oples of variousdrawings white learning techniques and benefits ofthis modern-day technology. For Activities # 1through #9, students will use reading material, TechTerms, Research Questions, a videotape, computersoftware, and hands-on activities to learn aboutcomputer-aided drafting. For Activity #10, stUdentswm complete a module post-test to help evaluatewhat they have teamed.

Duration 8 days

DESIGNERS _____________~_~~~_~~~___l

Figure 5: Opening section of the printable version of a unit plan.

Discussion areas

Since hnle.org was first released, online discussion areas have been part of the site

design. Initially, the Kiikakiika tool was used to support hnle.org discussion areas.

Kiikakiika was developed in the Laboratory for Interactive Learning Technologies (LILT)

at the University of Hawai'i - Manoa (Suthers & Xu, 2002). Kiikakiika is designed to

33

support artifact centered discourse, allowing discussants to view a related webpage while

reading and posting comments.

REPLYP1Cll9C click REPLY 10 post your comments IIboUl thi. unit plan.

HNLC DISCUSSIONS NEW THREAD RELOAD~

Subject From Date

j) Welcome 10 the Unit Plan DanOiacuuionst

~ Watarshed Woe. Automatic

ill Walar QualilylWalar Shed Automatic

;II Computer Aided Drafting Automatic

Ol Water Qualily Monitoring Automatic

HNLC Unit Plan:

UllJT OESCRlPTtON

ntl. Water Quality Monitoring (10/23/02)Link brtp'lIl6S 2.8.169.12Iwal!!!'l!lJilllly

Summary This project compliments our basic higb scbool biology cou"'ion ecosystems, Before (or while) partidpatlng In this project'an understanding of tbe following:

• 5cientif\c Method• Photosynthesis and Respiration• Ecology: ecological Interactions, food chains and webs, ten~~~s,~c. ,- Cycles or Maller: water, carbon, nitrogen, etc. L

• Evolution: spedatlon. adaptatlon, survival, etc. !• Computer applications: "",eadsheets, graphing, graphics, (,

~Through the hands-on activities that the students partldpa(experience with the major environmental factors in terrest.j'Students collect, organize and analyze data from the natu,,!:relate laboratory studies to natural systems. As students Iunit they are actively engaged In the process sdentists useenvironment.

Duration 4 weeks, 4 days

DESrGNERS

Alan Naka9Jllw8t <elln nAl<ogawa«pnRtC5,kl2 hl,uS>; Honelntermed.. e SchOOl

Steve HanKS, <Steye Hanks@notes.k12,hius>; Honokaa t~

Sd100l •

SUbjectWater Quality Monitoring

(10123/02)

From Dale

Automatic 6 Dec 2002 05:39:02GMT

Figure 6: A typical Kfikfikfikfi screen. The artifact of interest is displayed on theleft; discussion threads are shown on the top-right; and individual message aredisplayed on the bottom-right.

There is currently a new artifact centered discourse tool, tentatively named Alu,

under development by members of LILT that builds on the lessons of Kiikakiika . Early

versions of Alu are already in use on the hnlc.org website.

34

!!l httP'ltwwyr hn1c.gm1rejOyn;:talunltSJbmI5SiQnJY]!w ptw?pl,nhl

"* 72AtormOt:e dGood job you guys! This unit looks intenesting now that I knowhow to find it.

* • I'

HNlC Urnt PIt\J1;

UNIT DESCRJPTlON

Title A Population Study of Cpae Uia (Halocaridina rubra)

Link W;Ul65,248 169 12Jwaletll\lality/opacpapuI4t100S

Summary The purpose of this unit Is to determine thepopu~t1on levels 01 opae ula in various and'lralineponds along the westem coast or the Big Island,Using population sampling techniques students willmathematidally determine the opae u(a population inanchialine ponds at Waikoloa and Weliweli Point.These population counts will be used as a baselinemea:;urement Df the opae ula and provide a meansto assess future health and stability of the anchialineponds.

Duration 3 weeks,

DESIGNERSAlan Nakagawa, <alan. nakaoawa@nQleS.l<12.hLus>

Steve Hanks, <Steve Hanks@notes klZ.hl.u$>;Honokaa Hi9h and Intermediate School

UNIT OVERVIEW

Gracie Levels 9,10,11,12

Primary Mathematks,Sdence,TechnoiogyCurriculum

AreasSecondary Language Arts,Sodal Studies

CurriculumAreas

Hawal'l/eullural The anchiallne ponds and the Cpae ula are a yital

~ Student Sample. .alan._f\JkaGaW3 1001-(]9-1012;34~38

Figure 7: Alu - the newer discussion area tool currently under development.

Story engine

The story engine was introduced in the fall of 2002 as part of an effort to more

directly involve HNLC participants in shaping the hnle.org site. The engine allows

authenticated users to post stories to the website for viewing by any visitor to hnle.org.

Submitted stories are placed on the hnlc.org homepage.

By completing a minimal form, shown below, users provide the text of their story

and associate the story with one of several categories provided for organization,

including: collaboration, development, general, grants, math, news and events, questions

and answers, resources, schools, science, or technology. Additionally, HNLC staff can

35

select a category titled "HNLC announcements." Once a story has been entered, users can

choose to spell check their text or preview their story before submitting it for use on the

homepage. As a convenience to users, any text in the story that matches regular

expressions for email addresses or URLs are automatically converted to active links,

without the user needing to enter any HTML or proprietary markup.

..•

n Ie

Department Select...'-------------'

Introductionparagraph

Wltl be visibleon ltl!! fronLpage of the

HNlC site andemaUed to

H LCmembers.

fuR text ofyour storyWill only be

avall blev n a visiter

dld<s "Rendl c full

story••• •

An.,. RLs (I' cbsl'te 0 (IrQ cs) th t beg n vll'" ' http://,,or'',,y:'vllluto cllcally be con ertc Lo cL ve links. 1'le c tru for ny v I m II

eddress.

(!J) Check Spellill9

Preview Story

(you must. pN~ ... ll! 1I' c story fore submllUng Il)

Figure 8: The story engine's story submission form.

36

37

Resource database

The resource database was introduced to participants at the HNLC 2003 Summer

Institute. The intent of the resource database is to assist educators in more readily

locating high quality resources and to spotlight the unique resources available in Hawai'i.

These resources focus on Hawai'i-based organizations, materials, and individuals of

interest to SMET educators. The intent of the resource database is to ease the effort

required of educators in order to locate high quality resources they might use and to

spotlight the unique resources available in Hawai'i.

Technologies used

All of the components of the website have been developed using open source

software packages and programming languages with the explicit intent that running the

hnlc.org community should be inexpensive and easily reproducible on a range of

computing equipment. Core technologies currently in use include:

• Linux: the operating system on which the website is hosted is a standard

installation of Red Hat Unux server;

•

•

•

•

Apache web server: request for web pages on the hnlc.org site are served by a

standard Apache web server installation;

MySQL: provides database support and backend data persistence;

PHP: provides server-side scripting for dynamically generated content;

PHPMyAdmin: a web-based interface written in PHP, used to manage the

MySQL databases;

•

•

Qmail: processes both incoming and outgoing email for the hnlc.org server;

EZMLM: provides dynamic mailing list support via Qmail.

38

Summary

HNLC's website uses open source development tools in order to provide HNLC

participants with engaging tools for use in their work. These tools have changed over

time as the needs of the participants became clearer. These changes have brought with

them improvements to the structure, presentation, and scope of hnlc.org tools. The degree

to which HNLC participants have used these tools is a central question for this study.

•

•

39

CHAPTER 4. METHODS

This study draws upon several data sources including server logs, interviews,

focus groups, and responses to a summer institute survey in order to evaluate the ability

of hnlc.org to:

facilitate an active, online community of educators; and

support the HIDOE HNLC team's efforts.

The data sources aid in understanding how, why, and under what circumstances

hnlc.org is used as well as how well hnlc.org is meeting the needs of project participants.

The use of several independent data sources reflects the qnalitative research practice of

triangulation. The intention is to compare and contrast different sources of information

about the same phenomena in order to help offset bias that might exist in anyone source

and thereby gain a more balanced understanding.

Participants

As of fall 2003, HNLC's cohort is composed of 20 rural, public schools located

on six of the seven inhabited islands of the state of Hawai'i. These schools have been

selected by the HIDOE and lead UHM researchers based on their location (schools were

drawn from United States Census location codes 6, 7, and 8) and schools where at least

30% of students were eligible for the Free and Reduced Lunch program. Schools with a

high percentage of native Hawaiian/part Hawaiian students were preferred. Administrator

willingness to participate in the HNLC initiative was also a factor in school selection.

40

The schools have been invited to participate in the HNLC initiative on a voluntary

basis. Schools that opt to participate provide a team of teachers and administrators from

the school to serve as that school's HNLC partner team. Usually, these teams are

composed of one administrator (the principal or vice-principal), a math teacher, a science

teacher, and a specialist such as a librarian, technology support staff, or curricular support

staff member. The team members are chosen at the school level, and may be volunteers

or selected by their administrators. Each year after the initial year, 10 new schools are

being added to the HNLC cohort. Based on 2002 HIDOE data4, HNLC cohort 1 schools

serve 6,776 students while cohort 2 schools serve 5,942 students (see Appendix F for

details of each cohort).

Respondents to the summer institute survey, participants in the focus group, and

volunteers in the interviews are drawn from these cohort schools. Users of the

unrestricted portions of the hnlc.org site include not only members of the cohort schools,

but also UHM and HIDOE staff, as well as members of the general Internet-using public.

An application for approval of this study was filed with, and approval received

from, the Committee on Human Studies (CHS). The application covered all aspects of

this study including the form and content of the survey instrument, the format of the

focus group sessions and follow-up interviews. Participants received a letter detailing

4 Enrollment figures are available online fromhttp://lilinote.k12.hi.us/STATEiCOMM/DOEPRESS.NSF/a1d7af052e94dd120a2561f7000a037c/53dccf9837744c220a256c4200038d4e?OpenDocument

41

their rights as participants in the study as well as the steps being taken to protect their

identity.

Procedures

Summer Institute Survey

One of the primary research instruments used in this study is a survey (Appendix

B) designed to capture information from HNLC participants about their attitudes toward

uses of rCf in education. Respondents were asked to provide information concerning

availability of various technologies and technology support in their schools. Additionally,

they were asked to estimate their frequency of use of various technologies. The summer

institute survey was distributed at the 2003 HNLC Summer Institute held at the HIDOE

Advanced Technology Research Branch (ATRB) offices on O'ahu, Hawai'i. Survey

completion was voluntary and anonymous. The survey responses are analyzed in order to

better understand the attitudes of individuals who are associated with HNLC toward uses

of ICf in education for teaching, learning, and professional development activities.

Surveys are a well established research technique in many disciplines and are well

suited to this study because of the need to ascertain participant attitudes and gather data

about perceptions of individual practice and technology availability in the work place

(Suskie, 1992; Converse & Presser, 1986)

The survey was pilot tested by several members of HIDOE's Advanced

Technology Research Branch (ATRB). Questions for which the response was

42

inconsistent with the survey's intention or that were identified as confusing to the testers

were discussed with the testers and with the UHM project staff. Survey questions were

revised to address any difficulties found.

Survey respondents were asked to provide basic biographical information

(number of years as an educator, etc.), as well as information concerning the technology

capacity of their school. They were asked to report the frequency with which they used

hnlc.org, as well as any benefits they perceive hnlc.org as providing. Respondents were

asked to assess their attitudes toward technology used for teaching, for learning by

students, and for professional development. A five point Likert scale (strongly agree,

agree, neutral, disagree, strongly disagree) was used for questions concerning

respondents' attitudes. Finally, respondents were asked to report their frequency of use of

several common networking technologies (email, discussion boards, instant messaging,

etc.).

Server Logs

In order to address the success of the hnlc.org online community, web server

access logs were analyzed to determine actual usage patterns for the site'. Log files were

used to determine server usage by all individuals. Access logs were pre-processed to

eliminate any usage of the hnlc.org site that was identifiable as HNLC staff use,

demonstrations, or development/testing of the site. Any obvious attempts to break in to

'Web server logs are available from 11.August.2002.

43

the servers were removed from the logs. Also, access of the site from any computer

within the Information and Computer Sciences department (the home department of the

UHM HNLC development team) was removed from the logs. Lastly, any log entries

identifiable as search engine "spidering'" were removed. The remaining log entries

provide an upper-bound estimate of overall usage of the hnlc.org site, as well as typical

monthly upper-bound usage estimates. (See Appendix C for examples of entries

removed).

Portions of the hnlc.org website are restricted to authenticated users (see Chapter

3 for a discussion of user accounts). These restricted areas contain HNLC internal

documents as well as the specialized tools developed for HNLC participants including the

unit plan template, artifact centered discussion areas, the story posting facility, and the

resource database. Independent log files are kept detailing participant login to the

restricted portions of the hnlc.org site7. These login logs were used to assess use of the

specialized tools by HNLC participants and, as such, serve as an indicator of the

contributions made by members of the HNLC community (i.e., it is not possible to add

content to the hnlc.org site without having logged in).

6 "Spidering" refers to the process by which web search engines such as Google, AltaVista, Inktomi, Lycos, etc. explore and index the content of a website. Such use isentirely automated and does not indicate an interested individual browsing the website.7 User login to the hnlc.org site has been recorded only since 22.December.2002.

44

Focus Group

A research focus group was convened on 19.ApriI.2003 including five volunteer

HNLC participants, UHM researchers and graduate research assistants, and HIDOE

representatives. The purpose of this focus group was to discuss with the participants their

perceptions of the HNLC project, of the hnIc.org website, and of uses of leT in their

teaching in general. Participants were asked to discuss past innovative uses of leT as

well as to imagine how leT could be better integrated into their teaching. They were also

asked to discuss what, if any, barriers they perceived to better fCT integration with their

curriculum. Lastly, participants were introduced to the specialized tools available on the

hnlc.org website and were asked to discuss how such tools could be used by them to

improve their teaching practice, what relevance the tools had to their existing practice,

and what improvements, if any, they could imagine for the tools.

The focus group format was chosen as a means to bring together the hnlc.org

development team and the HNLC participants in an attempt to better understand the

participants needs for and perceptions of the hnlc.org website and the HNLC initiative as

a whole. The format allowed free form discussion about a broad range of issues raised by

the participants in an atmosphere where the participants felt safe to discuss their

perceptions of the initiative (Morgan & Krueger, 1993).

Notes were taken by HNLC staff during small-group discussion sessions and

audio tapes were made of full-group conversations. Questions used by HNLC staff to

guide the discussions are included in Appendix. Discussion audio tapes and staff notes

45

were reviewed for information that deepens understanding of the issues raised in this

study as well as to provide a context for the study.

Interviews

Lastly, participants at the HNLC 2003 Summer Institute were asked to volunteer

for telephone and/or email one-on-one interviews. All respondents who volunteered to be

contacted by email were sent a set of questions intended to delve deeper into the issues

raised by this study and to verify and clarify interpretations of the above data sources.

The questions presented to participants are included in Appendix E.

Limitations of the Study

Several limitations arise in this study. First, because of the small number of

individuals in the HNLC cohort teams, there are only 47 respondents to the survey - this

constitutes 100% of the people attending the 2003 summer institute where the survey was

administered. As such, no correlation analyses can reasonably be performed on the

survey data. Additionally, since the survey was completely anonymous, no correlation

between self-reported hnlc.org usage and login log recorded usage can be made.

Nevertheless, this study serves as a descriptive snapshot of the state of HNLC and

the success of hnlc.org. As such, it is hoped that this can serve as a basis for an active

research program within the HNLC initiative.

46

Second, the survey relies upon the willingness of respondents to give truthful

answers and their ability to give accurate answers. This limitation is addressed by

triangulating several additional data sources, including focus group, and interviews.

Third, this study is the first publication related to the HNLC initiative.

Additionally, this is one of only a few studies addressing the use of ICT to support

professional development programs in the context of systemic reform efforts. It is hoped

that this study will be used in future HNLC and hnlc.org evaluations and that the

interpretations presented here will be refined and reinforced by those future studies.

47

CHAPTER 5. RESULTS

This chapter describes findings of the summer institute survey that was conducted

during a portion of the HNLC 2003 Summer Institute held at the Bishop Museum on

O'ahu, Hawai'i. A total of 47 participants completed and returned the survey. Basic

demographic information of the respondents is summarized. Participant attitudes toward

and understanding of the hnlc.org website are then presented. This is followed by

summary statistics of participant attitudes toward use of technology for teaching,

learning, and professional development. Finally, frequency of use of hnlc.org and

frequency of login to hnlc.org as reported by respondents are presented.

Empirical usage statistics for the hnlc.org website are then presented. Much of the

hnlc.org website is accessible by any user visiting the hnlc.org site. Open access is

allowed in order to promote the HNLC initiative and invite community participation. All

of this traffic is tracked by the web server software and is recorded in web server access

logs.

In addition to the open access areas of the website, certain portions of the site are

restricted to use by HNLC participants and can only be accessed once users have logged

in to the hnlc.org site using validated user accounts. The restricted areas of the site

include the story engine, unit plan template, discussion areas, and submission of

resources to the resources database, as described in Chapter 3. Each successful login by

48

an hnlc.org user is logged independently of general website traffic. These login statistics

are presented along with the raw usage statistics mentioned above.

Analysis of an HNLC focus group held in April 2003 and HNLC participant

interviews are presented as the final section of this chapter. Highlights that both support

and contradict the summer institute findings are presented using the following themes

drawn from this study's research questions: (a) the ability of hnlc.org to support the

HNLC initiative; (b) the role of hnlc.org in facilitating an active, online community of

educators; and (c) factors influencing uses oflCT by HNLC participants.

Summer Institute Survey

The survey (Appendix B) consisted of 68 questions including such issues as

participant attitudes toward and frequency of use of various technologies for teaching,

learning, and professional development.

Profile ofSurvey Respondents

Of the 47 respondents, 40.4% identified their primary role as classroom teachers,

6.4% identified as HNLC teacher partners (half-time positions funded by HNLC who are

tasked with identifying and coordinating in-service, school level needs in science,

mathematics, and technology education), 10.6% identified as technology coordinators for

their schools, 6.4% as library media specialists, 19.1% identified as administrators, and

17% identified their primary role as "other," including: Standards Implementation

Design (SID) coordinator, resource teacher, special education resource teacher, retired,

49

lab technician, curriculum coordinator, and staff development coordinator (see Table 1;

note that in this and subsequent tables, gray shading indicated the modal response).

Table 1: Demographics: Primary Role ofRespondents Within Their Schools

# %Classroom teacher 19 40.4HNLC teacher partner 3 6.4Technology coordinator 5 10.6Library media specialist 4 6.4Administrator 9 L9.1Other 8 L7.0

----,_._._---~_._-------------

. Total respondents__,_, ~2 ._199.0

89.1% of respondents had 5 or more years experience as professional educators

and 82.6% reported 5 or more years as educators in the state of Hawai'i. Additionally,

76.1 % reported 10 or more years of service as professional educators and 65.2% reported

10 or more years of service in Hawai'i (see Table 2).

Only 4.3% of respondents have been with HNLC for the full 2 years of the

initiative while 29.8% have been HNLC participants for approximately 1 year. 65.9% of

respondents have been HNLC participants for less than 6 months (see Table 2).

A majority of respondents volunteered to participate in the HNLC initiative

(78.7%), while 21.3% were assigned to their school's HNLC team. Additionally, 47.8%

of respondents were involved with their school's decision to join HNLC (see Table 2).

Taken as a whole, this suggests that respondents to this survey are motivated,

seasoned professionals who are largely new to the HNLC initiative.

50

Table 2: Demographics: Number of years as an educator, time withHNLC, and nature of involvement with HNLC

# %Years as an educator

0-4 years5-10 years10+

.-!.?tal respondents

5 10.96 13.0

46 100.0

88

Years as an educator in Hawai'i0-4 years5-10 years

____}_Q±J'!~rs. _Total!~~pondents. 46

17.417.4

100.0

31 65.97 14.97 14.92__ 4.3

47 100.0-----,-"" ..,_..,..,------_..,..,--,-,-,_..,-_..,---,_..,------

Months involved with HNLC initiative0-5 months6-11 months12-23 months24 months

Total res ondents

Reason for HNLC participationRespondent volunteered 37 78.7Responden.!-~,~..s.. assig2-n_e_d_to_H_N_L_C l O ....,..,""_,.~,!,:.~,,_

~o~~l respondents _,_",_, 47 100.0

Participated in school's decision to join HNLCNoYes---- ","'-' ,-, ,-

Total responde!.1!~_,_, _

242246

52.247.8

100.0

Asked to characterize their understanding of hnlc.org (Table 3), a majority of

respondents "agreed" that the purpose of hnle.org was clear to them and that hnle.org was

relevant to the success of their students. However, when asked about the effectiveness

and appropriateness of hnle.org, at least half expressed a "neutral" feeling as to whether:

•

•

51

hnlc.org had a positive impact on their professional activities;

hnlc.org had a positive impact on science, math, and technology programs in their

schools;

• hnle.org provided services unavailable to them elsewhere.

Respondents were also asked indicate which services offered by the hnlc.org site

were most and least valuable to them and what, jf any, additional services they would like

hnle.org to offer. Their responses are summarized in Table 4.

Table 3: Respondent attitudes toward hnlc.org

The purpose of hn1c.org is clear to me #%

hn1c.org is relevant to the success of students #%

Strongly Agree Neutral Disagree Strongly Totalagree _ disagree

1 29 10 4 1 452.2 64.4 22.2 8.9 2.2 100.0

2 23 18 2 0 454.4 51.J 40.0 4.4 0.0 100.0

52

2 18 22 1 1 444.5 40.9 50.0 2.3 2.3 100.0

1 14 25 2 1 432.3 32.6 58.1 4.7 2.3 100.0

hn1c.org has a positive impact on my #professional activities %hnlc.org has had a posi6ve impact on the #science, math, and technology programs in my %schoolhn1c.org provides services to me that are #unavailable elsewhere %

12.3

1534.9

2455.8

37.0

o 430.0 100.0

Table 4: Responses to open ended questions about hnlc.org

The service that hnlc.org has provided that • Current info on technologyI find most valuable is... • Networking with other schools on problem based

learning• Resource listings• Ideas for units• The unit plan template

53

The service that hnlc.org has provided thatI find least valuable is ...

Additional services I would like to seehnlc.org provide include ...

•

••••

The discussion area

Grant writing supportOnline mentoringAdditional unit plan guidanceAdditional resource links to assessment topics

54

A great many of the respondents reported frequent use of computers in the classroom for

instructional purposes (Table 5). Of the 3 respondents reporting never using computers

for in-class instruction, 2 identified themselves as classroom teachers, while 1 was

identified as a non-teaching technology coordinator. Consistent with high usage of

computers by teachers, respondents felt strongly that computers were relevant to their

teaching and important for their efforts to bring innovative methods to their teaching

(Table 6). They also felt that computers enabled them to focus on individual students'

needs better, and that computers served as a valuable tool for instructional planning. Over

80% of respondents felt that computers did not interfere with their teaching. However,

only about 60% felt that they had sufficient training to incorporate technology

successfully into their teaching (Table 6).

Table 5: Frequency of use of computers for in-class instruction

# %Several times per weekAt least once per weekAt least once per monthAt least once per quarterAt least once per academic yearNeverTotal res ondents

17323

36

2.819.48.35.68.3

100.0

Table 6: Respondent attitudes toward technology use in teaching

Strongly Agree Neutral Disagree Strongly Totalagree disagree

Computers allow me to incorporate innovative # 12 18 8 1 0 39instructional methods in my classroom % 30.8 46.2 20.5 2.6 0.0 100.0Computers allow me to better address the # 9 23 7 0 0 39individual needs of my students % 23.1 59.0 17.9 0.0 0.0 100.0Computers are relevant to my teaching

~I16 16 7 0 0 39

41.0 41.0 17.9 0.0 0.0 100.0Online tools can be used to support my teaching # 7 26 5 1 0 39

% 17.9 66.7 12.8 2.6 0.0 100.0The Internet is a valuable resource for instructional # 18 20 1 0 0 39planning ideas % 46.2 51.3 2.6 0.0 0.0 100.0I actively seek ways to incorporate computers into # 9 17 9 4 0 39my lessons % 23.1 43.6 23.1 10.3 0.0 100.0I have sufficient technology training to incorporate # 7 15 11 4 1 38computers into my teaching successfully % 18.4 39.5 28.9 10.5 2.6 100.0Computers interfere with my teaching # 0 1 3 18 17 39

% 0.0 2.6 7.7 46.2 43.6 100.0Seeing how my students use computers has given # 9 15 14 1 0 39me new ideas about how to use computers more % 23.1 38.5 35.9 2.6 0.0 100.0effectively in my teaching

55

56

Respondents reported a high frequency of use of computers in-class by students

(Table 7), although not as high a rate of use as for "in-class instruction" (Table 5). In

sharp contrast to the 77.8% of respondents reporting computer use for in-class instruction

at least once a month or more frequently, more than 25% of respondents reported that

their students used computers in-class only once each quarter or less. Of the 4

respondents reporting that they never use computers in the classroom for student use, 2

identified themselves as classroom teachers and the remaining 2 identified themselves as

non-teaching technology coordinators. Although over 90% of respondents felt that

computers were relevant to students' success and were a valuable resource, over 65%

cited limited access to computers as a factor limiting students' success (Table 8). A

significant portion (45-47%) felt that the glut and poor quality of information online were

also factors limiting students' success.

While over 60% of respondents reported that they found inspiration for new uses

of computers from their students, only 39% felt that students were inspired by teachers'

use of computers.

Table 7: Frequency of use of computers in-class by students

# %Several times per weekAt least once per weekAt least once per monthAt least once per quarterAt least once per academic yearNever

~~t_~~!~~p_o_n_d_e_n_ts _

- 4 10.511 28.95 13.21 2.64 10.5

38 100.0

Table 8: Respondent attitudes toward technology use by students

Strongly Agree Neutral Disagree Strongly Totalagree disagree

Computers are relevant to students' success in # 13 26 3 1 0 43school % 30.2 60.5 7.0 2.3 0.0 100.0Computers are relevant to students' success in # 13 26 3 1 0 43life % 30.2 60.5 7.0 2.3 0.0 100.0The Internet is a valuable resource for students' # 15 24 3 1 0 43academic work % 34.9 55.8 7.0 2.3 0.0 100.0Limited access to computers hinders students' # 11 17 11 4 0 43success % 25.6 39.5 25.6 9.3 0.0 100.0Limited availability of online content hinders # 6 13 16 8 0 43students' success % 14.0 30.2 37.2 18.6 0.0 100.0Too much information online hinders students' # 2 17 15 9 0 43success % 4.7 39.5 34.9 20.9 0.0 100.0Lack of quality materials online hinders # 3 18 16 6 0 43students' success % 7.0 41.9 37.2 14.0 0.0 100.0Students I work with have been inspired to find # 3 14

1

20 6 0 43new ways to use computers themselves because % 7.0 32.6 46.5 14.0 0.0 100.0of my use~computers

57

58

In contrast to the import given computer technologies above, most respondents

reported that their use of Ief for professional development was limited to the use of

online resources (50% reported use at least once a week or more) and email (87%

reported use at least once a week or more), as summarized in Table 9. Over 50% reported

never having used online discussion boards for professional development while over 85%

reported never having used online chat rooms or instant messaging for professional

development. This suggests that HNLC participants may have only limited exposure to

and understanding different modes of online communication and collaboration as tools

for professional development.

Respondents were also asked to indicate what they perceive as the most valuable

and least valuable aspects of using computers for professional development. Their

responses are summarized in Table 1I.

59

Table 9: Frequency of use of several ICTs to support professional development

Online Email Discussion Chat rooms Instantprofessional boards messagingdevelopment

resources# % # % # % # % #

Several times per week 10 21.3 34 72.3 0 0.0 0 0.0 I 2.1-At least once per week C J4 29.8 7 14.9 4 8.5 0 0.0 1 2.1

At least once per month 11 23.4 4 8.5 11 23.4 3 6.4 2 4.3

At least once per quarter 4 8.5 I 2.1 2 4.3 2 4.3 2 4.3

At least once per academic 5 10.6 1 2.1 6 12.8 2 4.3 0 0.0yearNever 3 6.4 0 0.0 24 51.1 40 85.1 41 87.2

!,",

Total respondents 47 100.0 47 100.0 47 100.0 47 100.0 47 100.0

60

However, despite the limited set of online tools employed by the respondents for

professional development, over 75% felt that online communication and active

collaboration were important to their professional development (Table 10). This may be

related to the respondents' sense that they do not have the necessary resources within

their schools to support their professional development (55%). Consistent with this

finding, 65% of respondents reported that they were not averse to the use of online tools

to support their professional networking efforts.

Surprisingly, given that 65% of respondents cited lack of access to computers as a

limiting factor for their students' success (see Table 8), over 90% of respondents reported

that access to computers was not a limiting factor for their own professional development

(Table 10).

Similar to the finding, above, citing student use of computers as an inspiration for

teachers to find new ways to incorporate technology into their teaching (Table 6), 70% of

respondents also felt that student use of computers inspired them to find new ways to use

computers to support their own professional development goals (Table 10).

Table 10: Respondent attitudes toward technology use for professional development

Strongly Agree Neutral Disagree Strongly Totalagree disagree

I believe communicating with other teachers via #. 20 18 7 2 0 47technology is important to my professional growth % 42.6 38.3 14.9 4.3 0.0 100.0Active collaboration with other teachers via #

3/~ I 20 10 2 0 47technology is important to improve my teaching % 42.6 21.3 4.3 0.0 100.0I have sufficient access to computers to support my # 18 25 2 2 0 47professional growth % 38.3 53.2 4.3 4.3 0.0 100.0Limited availability of online content focused on # 2 11 17 17 0 47educators' professional development limits my % 4.3 23.4 36.2 36.2 0.0 100.0professional growthI have been inspired by my students' uses of # 7 26 9 4 0 46computers to find new ways to use computers to % 15.2 56.5 19.6 8.7 0.0 100.0support my professional developmentI am overwhelmed by the amount of professional # 2 10 28 7 0 47development material available online % 4.3 21.3 59.6 14.9 0.0 100.0I consider myself to be an active member of an # 3 20 12 12 0 47online educational community % 6.4 42.6 25.5 25.5 0.0 100.0I prefer not to use online communications tools for # 1 7 8 24 7 47professional networking % 2.1 14.9 17.0 51. J 14.9 100.0I believe that I can get the professional development # 2 6 8 23 8 47support I need completely from within my school % 4.3 12.8 17.0 48.9 17.0 100.0

61

Table 11: Responses to open ended questions about technology use for professional development

What I ffWst value about using computers • It's accessible at any time, from any placeto aid my professional growth is... • I can work at my own pace

• Sharing of information and ideas• Availability of up to date information

62

What I least value about using computersto aid my professional growth is...

••••••

Lack of personal contactDifficulty finding informationAsynchronous nature of interactionLack of time to use the computerCostTechnical difficulties (slow connection, cross­platform issues, etc.)

0.0000.000

63

A one-way X2 analysis was conducted on measures of attitude in order to assess

the null hypothesis that respondents answered the questions randomly. The analysis

indicates that for all attitudinal survey questions, the distribution of responses was

significantly different from what would be expected if respondents had chosen their

answers at random, and the null hypothesis is rejected (Table 12).

Table 12: i analysis of attitudinal survey questions

Survey Question pThe l,urpose of hn~~.org is clear to f!le .. ._____ ,._ .... 0.000_

..~!.1!£.org. is relevant to t:!Ie sll_~~~.s of students _. ..... __ ___.Q:QQQ..__~nlc.org_ has a positiye iJ!lEact on my professional activi!!~~ . .Q:Q.QO .

hnlc.org has had a positive impact on the science, math, and technology 0.000pr~gr_~lllsi~~Y-_~_chool . .._.__.._..._ _ .__.. , .... ...

._hnl~~~_provides ~~rvices to me that are unav-'!!ll!.ble else"Yh~.!e ._._... 0.000Computers allow me to incorporate innovative instructional methods in my 0.000classroom

---- ------------"."_.. _.....•..._-"--"----_Com...P.':l!t:E.~l!~.~~ me to better address tht:.,.i.i!:~.!~!~ual needs of my- stud~!.1!~_.______ 0.000_.~0!!!.P.II_!~_~~.~!.erelevant to my teachiJ!lL...._._. ... 0.000_Q.!.1!.!!.1~ tools can be ~~~<! t()_sUPP~E1:. my teaching __._0.000_._.I~~}nternet is a valuable res~urce_t.<:>.r instructional planning idea_~_________ 0.000..!..l!ct!~!.y_se~k ways to incorporate £.omfl.II!t?Fs into m)' lessons o.,.QOO_1 have sufficient technology training to incorporate computers into my 0.003

...1:t?~_~hing_ successfull)'. . ....__.._. _

...~omputers interfere with my teac~.~__ __._ ..._._Seeing how my students use computers has given me new ideas about how to

_.'!:~~ computers more effectivelyi~!UY- teachin_g _. . .SSJ_mpu!t?rs are relevant t'?_~.t.ll.dents' success in school 0.000._.....c~.!!!.Puters are relevant to stu.<!~nts' success in life 0.000

The Internet is a valuable resource for students' academic work 0.000._-" - ---.•.-- -..~imited access to comp_ute!!! hinders students' succes~ . ._._.9..:.900J:imit~~~,~!!ila~ili.t.Y- of online conte,!!. hinders students' succes_~_..... ._0.001....Too much information online hinders students' success 0.000-------_.__. ~.._-- '._-_._--"..---Lack of qllali!)' materials online hinders ~tudents' success . 0.000Students I work with have been inspired to find new ways to use computers 0.000

_.themse.!yes_because of m)' use of c~.!!.!p_'!:!~~__. . . . .__

0.000

0.0000.000

64

Table 12 (cont.): "i analysis of attitudinal survey questions

Survey Question pI believe communicating with other teachers via technology is important to 0.000

_1!~YJ~rofession~_g"r.owth _.""''',,.,,__._' _Active collaboration with other teachers via technology is important to 0.000improv~ my teaching ''_'''''''' ...''.

_I have su!f!£!en!_access to computers,,!~~upport my profe~.!'ionalg!.?~th

Limited availability of online content focused on educators' professionaldevelopment limits,,!!1l. professional g!owt~.........._ """_.,, .. _::-::~.I have been inspired by my students' uses of computers to find new ways to 0.000

.. use computers to supp~rt m~es~ional developme!1t ...... .__.__.I am overwhelmed by the amount of professional development material 0.000available online._--"-------_ _"- __._--_.-..--",,..,, - .. ---'I consider myself to be an active member of an online educational

.. communi!y_..__"""""""__I prefer not to use online communications !?Ols for pr?fessional .'!:~t~'?rki.'!:L_.. 0.000I believe that I can get the professional development support I need 0.000

..~omp!etely from within !.!!y~~ho?!.. ."..__ ,," . .."' _

Server Logs

In order to track usage of the hnlc.org website, web server access logs are kept

that detail the date and time of all requests reaching the web server. Additionally, the

source of the request" as well as specific information about what resource was requested,

the result of the request (success, failure, etc.), and the amount of information transmitted

8 The sources of requests are tracked using either the Internet Protocol (lP) address or thedomain name of the requestor. An IP address might be of the form "192.33.12.137",while a domain name might appear in the access logs as"a24b94n72client9.hawaii.rr.com". Using these Internet addresses, it's possible to trackthe behavior of a single visitor to the website, to see what resources were requested, etc.

65

in response to the request are all logged. Since the access logs record all visits to the

hnlc.org site, visits to the site for development and testing purposes are also logged.

Additionally, any attempts by Web search engines to index the content of the site, as well

as attempts to "hack" the web server are all logged. The hnlc.org server access logs were

pre-processed in order to omit from the analysis all such traffic in an attempt to give a

truer picture of "outside" usage of the website. See Appendix C for an explanation of the

log entries removed.

The pre-processed access logs were then analyzed using the popular Webalizer"

server log analysis tool in order to obtain monthly usage statistics (Table 13). Based on

these statistics, the monthly number of unique visits to the website is presented in Figure

9. A unique visit occurs when two different Internet addresses visit the website, or when

the same Internet address visits the site after making no requests for 30 minutes or longer.

A portion of the traffic captured by the web server access logs includes users who

visit the hnlc.org website and then use a unique user account (consisting of a

username/password pair) to "login" to the website. The user account has either been

created for the user or the user has, on a previous visit to the site, requested that an

account be created for them. Once a user account is activated, the user can login to the

hnlc.org site as a verified user. Successful login attempts are tracked in order to provide a

record of HNLC participant use of the hnlc.org website.

9 Webalizer can be found online at http://www.webalizer.comJ

66

By logging in, users gain access to restricted portions of the hnle.org site

including the interactive tools described in Chapter 3. These tools are intended to support

HNLC's systemic reform efforts by allowing HNLC participants to share resources and

collaborate on instructional planning and professional development activities. As such,

active use of these online tools by HNLC participants is a critical measure of the success

of hnle.org.

Login logs were pre-processed to eliminate logins by HNLC project staff. The

remaining nnmber of successful hnle.org logins per month was then caleulated and is

shown as "HNLC participant logins" in Table 13 and as black bars in Figure 9 and Figure

10, below. Additionally, participant login data was considered when web requests from

the HIDOE ATRB network were excluded, since the majority of logins from the ATRB

network were generated at the request of HNLC staff during HNLC sponsored

workshops, or originated from HNLC staff doing their work. The data shown as "HNLC

participant logins excluding HIDOE ATRB network" in Table 13 and as white bars in

Figure 10, below, provides a better estimate of use of hnle.org by teachers on their own

initiative. Note that in neither case does "participant logins" indicate the number of

unique participants who logged in to hnlc.org, but rather the total number of logins by

hnlc.org users, including repeat visits. Login logs have only been maintained since

January 2003, so only a partial picture of hnlc.org usage by verified HNLC participants is

available.

67

Table 13: Monthly use of hulc.org website

HNLC HNLC Total HNLCparticipant participant unique participant

logins logins visits logins as aexcluding percentage ofHIDOE total uniqueATRB visits

network8/2002 6079/2002 805

10/2002 6311112002 75812/2002 825112003 53 9 1144 4.6%2/2003 75 25 863 8.7%3/2003 48 26 926 5.2%4/2003 88 7 1152 7.6%5/2003 * 158 16 1620 9.8%6/2003 * 375 56 1235 30.4%7/2003 24 14 1232 1.9%8/2003 63 14 1792 3.5%

*The HNLC 2003 summer institute was held in 6/2003. Thisincreased usage most likely reflects participants responding toHlDOE instructions for HNLC team preparation for the instituteand guided group activities conducted at the institute, bothrequiring logging in to hnlc.org.

While the overall usage of hnlc.org, whether logged in or not, seems low, the

critical factor for success of hnlc.org is the use of the restricted-use tools: the curricular

unit plan template and resource database, for example. Since these can only be accessed

when logged in to hnlc.org, the "HNLC participant logins" column of the table above

gives the most direct indication of overall use of the hnlc.org tool set by participants.

68

The column labeled "HNLC participant logins excluding HIDOE ATRB

network" offers the best picture of participant adoption of hnlc.org tools for their own,

spontaneous use. This number reflects use of hnlc.org tools outside ofany HIDOE

scheduled HNLC event. The total participant logins and participant logins excluding use

from the HIDOE ATRB network are shown for comparison in Figure 10.

Unique visits and participant logins tohnlc.org

August 2002 - 20032000 ,.--------------------------..,1800 +-----------------------.......-11600 4--------------------rn;:r-----1

....~ ~~~~ +-------------------li;.!~.'II-r.~.~":'l;.~"--·"";:""i:------I'iii 1000 +---------------JTITIf------I:; 800 +----...,..,,------~-£

600 :.:. ~ml-·'"'""-' ::::400200

0+""'.:.i&.-..-'-"""--.=.i1.-,.......,.:.L....,r-'""'.L......,~__,.....-.......-...."'--L..r-L'

~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~co' 0)' ~' v "', 'v "', ",' ~' ~. (0' '\. CO'

~ "'I- "'I-

ISTotal visits to hnlc.org • HNLC participant logins I

Figure 9: The number of unique visits to http://hnlc.org from 8/2002 - 8/2003. Also,shown in black are the number of verified HNLC participant logins from 1/2003 ­8/2003.

69

Participant logins and participant loginsexcluding HIDOE network traffic

January-August 2003400 -.-----------------------,

350 +--------------­

300 +--------------­

250 +---------------.l!l.; 200 +--------------­;;

150 +------------

100 +-----------­

50

o1.2003 2.2003 3.2003 4.2003 5.2003 6.2003 7.2003 8.2003

I_ HNLC participant logins 0 Participant logins excluding HIDOE I

Figure 10: The number of verified HNLC participant logins (black) and verifiedHNLC participant logins excluding web traffic from the mDOE ATRB network.

Focus Gronp

To guide the development of the summer institute survey and to better understand

the trends of hnlc.org use highlighted by the server logs, HNLC participants were invited

to participate in a focus group discussion. The focus group was intended to inform the

UHM development staff, to guide existing development efforts, and to suggest new

possibilities for hnle.org projects. Five HNLC participants volunteered to attend the focus

group. The attendees were joined by seven HNLC staff members.

70

During a portion of the full day session, attendees were asked to discuss their

current practices, perceived challenges, and possible improvements with regard to three

key topics within the HNLC initiative: alignment of curriculum to the Hawai'i Content

Performance Standards (HCPS); use of technology to support and improve science, math,

and technology education; and integration of inquiry-based learning into the curriculum.

Small group discussions were observed by one HNLC staff member per group with the

conclusions of each group presented to the entire focus group.

Table 14 summarizes many of the issues raised by participants. Issues raised

included their broad concern that the goals of the HNLC initiative might be critically

different from their own goals and that HIDOE expectations where unreasonable, given

the stage of development of their reform efforts. Also, participants recognized that HNLC

was an evolving initiative and that mistakes that had been made with cohort 1 schools

could be averted by allowing cohort 2 schools to learn from the mistakes made with

cohort 1.

Table 14: Summary of issues raised by focus group attendees

Standards Alignment Technology Integration Inquiry-based Learning

Participants' current • Teachers have projects • Using GPS, and digital • Inquiry projects arepractices aligned to some standards, probes being developed.

but not entire unit plans • Using the Internet • Some teachers have• Standards alignment is most!y for research made use of inqui ry

done as an add-on, after projects alreadythe fact

Challenges to increased • Do HNLC's goals match • Do HNLC's goals • Do HNLC's goalsor improved practices teacher objectives? match teacher match teacher

• Documentation objectives? objectives?requirements from HIDOE • Lack of available • Too little timeare often far ahead of computers for student available to gather thecurrent practice use resources needed to

• No time available to • NSF assumes that implement inquiry-formalize unit plans schools already have based units

• The HCPS sometimes computers in place, • Difficult to locatepress teachers to move but this isn't a given in outside people to helpthrough the curriculum rural schools • Unresolved issuesfaster than teachers feel over liability forthey should student welfare

71

Table 14 (cont.): Summary of issues raised by focus group attendees

72

Suggestions forimprovements that wouldimpact current practices

Standards Alignment

• HNLC cohort 2 schoolswill be able to learn fromcohort I mistakes

Technology Integration

• HNLC cohort 2schools will be able tolearn from cohort 1mistakes

• More money needs tobe made available

Inquiry-based Learning

• HNLC cohort 2schools will be able tolearn from cohort 1mistakes

• UHM could improvethe public perceptionof the work beingdone by Hawai'i'spublic school children

• UHM could buildbridges betweenpublic school teachersand the Hawai'icommunity

73

During the remainder of the day, attendees were re-introduced to the existing

hnlc.org tool set. After a brief description and demonstration of each tool, attendees were

asked to discuss how each tool met or did not meet their perceived needs; what uses they

could imagine for the tool; and what improvements the thought were necessary to make

the tool more useful to them in their teaching.

While attendees responded well to the tools presented, they reported that they had

little or no experience using such tools online. Due in part to this lack of familiarity with

types of online tools presented, attendees offered few suggestions for how they might use

the tools outside of HNLC meetings. Also, the suggestions for improvement were largely

related to the visual appearance of the tools (color,layout, spacing, etc.) or suggestions

for changes in on-screen labels to better support novice users (e.g., one participant stated

that they were confused by questions on the unit plan template asking for "links" - they

didn't associate the idea of a link with the developers' intention that a web address be

provided).

Finally, attendees were asked to discuss any ideas they might have for new tools

that could be developed as part of the hnlc.org website. Without exception, the

suggestions made were for. tools that would be of use to their students, and not for tools

that would directly support their own professional development or professional activities.

For example, one teacher suggested that HNLC could provide a "polling website," where

students from one school could create a survey (poll) that could then be answered by

many students around the state. Also, during discussions of the hnlc.org resource

74

database, attendees primarily suggested the addition of websites that students would use

as part of lessons. One teacher stated, "when I'm using the web, I focus on sites that my

students can use, " while another commented, "there's too much information out there ­

I'm on overload. I gravitate toward information my students can use right away."

The role ofhnlc.org in supporting the HNLC initiative

Attendees did not perceive the HNLC initiative and the hn1c.org website as

distinct entities. They did not distinguish between the HNLC initiative as a whole, on the

one hand, and the role of hn1c.org as supporting the larger HNLC initiative, on the other.

As such, some participants reported that they perceived "HNLC meetings" as "web tool

training."

Several participants reported that they used tools such as the unit plan template

not because the tools helped in the construction of unit plans, but rather because HIDOE

ATRB required them to use the tools. For example, one participant commented that they,

"fill in the [unit plan] template after [they've] finished writing it in [Microsoft] Word."

However, hn1c.org is beginning to have some impact. At a recent school visit, one

teacher did report that they had followed up on a grant opportunity posted on the hnlc.org

homepage, and had received funding for their project as a result.

The role ofhnlc.org in facilitating an active, online community ofeducators

The focus group discussions made it clear that HNLC participants dominantly

thought of "technology in education" as "technology for use by students," and not as

"technology for use by educators." Not only were suggestions for hnlc.org development

75

efforts keyed toward student use of computers, but also discussions of how existing

hnlc.org tools could be used focused on how students could use hnlc.org in the

classroom.

Factors influencing uses ofICT by HNLC participants

Attendees identified three critical factors that prevented them from increasing or

improving technology integration. Among these factors were:

• funding for equipment (note: the NSF-RSI implementation grant does not provide

equipment funding for participating schools).

•

•

training in use of equipment

time to work one-on-one with students

While each of these factors focuses on the practical and very real implementation

issues that need to be addressed for successful instruction, the HNLC initiative is a

systemic reform effort, attempting to address not only implementation issues, but also the

larger reform agenda.

Interviews

27 summer survey respondents volunteered to be contacted by email. All 27 were

emailed a set of questions intended to verify and clarify the interpretation of the data. 5

emails were received in response to the emailed questions.

When asked what factors influence use of hnlc.org, respondents cited lack of time

and difficulty logging in. Many users have had difficulty logging on to hnlc.org at

HIDOE hosted events. In the first months following the introduction of user logins, there

76

were browser compatibility issues that affected users' ability to login from some machine

configurations. However, in my experience, inability to login has often been because the

user doesn't remember his or her username or password and simply resetting these has

resolved the issue. Several UHM HNLC staff reported that during recent school visits,

non-technical sources of users' inability to login reliably remained an issue and that as

many as one-quarter of those present needed to have a new password assigned during the

school visit.

Respondents were also asked what hnle.org tool they felt was most useful. The

consensus was that the unit plan template, one of the oldest tools current on the site, stood

out. One person said that, "the unit plan template helped to knowhow things fit together

and how to write the plan."

When asked whether "since becoming involved with HNLC you have grown

more likely to use computer technologies to support your professional activities to

support student learning," one respondent indicated that they were now more likely to use

Microsoft Powerpoint to create presentations for class. Another respondent indicated that

they had always used technology in the classroom, and that HNLC hadn't changed that.

Over all, respondents seem to have positive attitudes about the use of technology

for a range of purposes, but have not significantly altered their behavior because of their

use of hnle.org.

•

•

77

CHAPTER 6. DISCUSSION

This study sought to deepen our understanding of use of the hnlc.org website and

to assess the degree to which hnlc.org has:

facilitated an active, online community of educators; and

supported the HIDGE HNLC team's efforts.

To gain insight into these questions, I have presented data concerning frequency

of hnlc.org use by the general public and by hnlc.org user account holders. This data

shows that while a modest number of people visit the hnlc.org site, exceedingly few

hnlc.org account holders login of their own accord. Rather, account holders login and use

the interactive tools provided for them almost exclusively when attending HNLC

workshops where hnlc.org use is required.

Website usage data indicate that usage has been consistently low from January

2003 through August 2003, suggesting that on going HNLC efforts to improve hnlc.org

offerings have had little, if any, impact on user adoption of the hnlc.org tools by HNLC

participants. This lack of use critically undermines the developers' intent that hnlc.org

should support professional development activities and facilitate the formation of

communities of practice-without users, there can be no community.

Survey results indicate that HNLC participants are by and large seasoned

education professionals with strong ties to Hawai 'j, However, despite HNLC being in its

second year of implementation, most participants are relatively new to HNLC. This may

78

be related to an observed high rate of replacement of teachers and administrators. HNLC

staff have noted in passing that at HNLC meetings, there is often a need to create new

user accounts for new members of school teams. This turnover in membership may

undermine HNLC assumptions about the ability to build technology savvy and expertise

over time.

Survey respondents were largely positive about the import and impact of

technology on education. Most respondents use computers in the classroom weekly and

feel that computer technology provides a significant benefit for both themselves and their

students. However, despite positive attitudes toward technology in education and

technology use for professional development, respondents used only a very limited set of

online tools to support their own professional development. This poses serious policy

implications for hnlc.org's development team, since much of the development focus to

date has been on multi-mode, innovative online tools in support of professional

development activities.

It seems clear that hnlc.org has not been successful in creating an active, online

community of educators. Participants have not used the hnlc.org website to a significant

degree to engage with other educators. This may be due in part to the ways in which

hnlc.org has been integrated into HNLC events. For example, during the 2002 summer

institute, the use of hnlc.org was not well integrated with other institute activities. This

lack of adoption of hnlc.org by participants underscores the need to either focus on

79

modeling innovative uses of hnle.org during HNLC events or to reevaluate the purpose of

hnle.org.

Challenges

There can be little doubt that educators face challenges in their daily efforts to

provide high quality learning experiences for students. Many reformers believe that

technology can be used to aid educators in their efforts and that student learning can be

deepened in so doing. But in order for technology to assist educators, several issues must

first be considered:

•

•

•

•

What challenge is being addressed?

Is technology the best solution to the challenge?

How will success of the solution be evaluated?

Is the solution consistent with teachers' goals, or alternatively, does thesolution bring with it such a positive change that teachers will adopt it?

Once these issues are addressed, we can then consider how past successful

education reform, successful implementation of a professional development program, and

successful formation of communities of practice efforts can guide the development of

hnlc.org.

What challenge is being addressed?

The hnlc.org website was designed to support the HNLC initiative and to serve

HNLC participants as an online community space. In order to succeed, hnlc.org

development efforts must be closely aligned to HNLC's goals and must incorporate the

80

best practices of online communities, including support for user-to-user interaction and

collaboration. Inspiration should be drawn from other efforts to support online educators'

communities of practice, such as TappedIn.org (Schlager & Fusco, 2003).

Is technology the best solution to the challenge?

This remains an open question. Technology has the potential to facilitate

collaboration, foster innovation, and promote communities of practice. Particularly in

situations where geography and time are barriers to collaboration, technology promises

significant advantage. However, if educators view technology as a source of additional

work, rather than as a valuable effort-saving tool, then technological offerings are

unlikely to gain favor. Participants have repeatedly cited lack of time as a reason for not

integrating technology into their practice. Additionally, participants have indicated that

they regularly engage in the practice of "filling in the form" after the fact, where hnlc.org

is concerned. One HNLC participant recently noted, "We're putting it into HNLC

afterward, but it's just so that it will meet that [HIDOE] requirement." In order for

technology to be adopted, it cannot itself serve to slow educators' practice, and it cannot

be used only as an afterthought.

How will success ofthe solution be evaluated?

This study is the first step to assessing the success of hnlc.org. However,

significantly more research is needed on an ongoing basis to ensure that development

efforts are meeting participants' needs and that they are consistent with the reform goals

of HNLC. Formative evaluations need to be well defined and frequently executed in

81

order to maintain focus on reform goals and to serve as a constant check on development

activities.

Is the solution consistent with teachers' goals?

Focus group participants were clear: they are not convinced that HNLC's goals

are consistent with their own. As such, great care must be taken to evaluate this

relationship and ensure that HNLC is grounded in the work life of educators. It is

possible that the current hnlc.org offerings are so disconnected from educators' goals that

adoption of hnle.org's tools cannot occur.

Does the solution bring with it such a positive change that teachers will adopt it?

The effort to evaluate the impact of HNLC and hnlc.org on student learning has

not yet been completed. Therefore, there is no basis at this time to prefer the tools

provided by hnlc.org over other methods of attaining HNLC's reform goals.

Recommendations

The issues raised in the literature review on successful education reform, systemic

reform, professional development, and communities of practice indicate many common

features of success that need to be addressed within the context of HNLC and hnle.org,

namely:

• Efforts should focus on classroom improvement. While it is true that systemic

reform efforts may require years of effort before being able to assess impact on

•

•

•

•

82

student learning, efforts should be made in the short-term to ensure that teachers

and students are benefiting from the reform.

Models of teaching and learning should be explicitly stated. Only recently has

there been an attempt to make explicit the HNLC model of learning. These efforts

must be refined and expanded to include models of professional development and

instructional practice as a whole so that these can guide not only HNLC, but the

development of hnlc.org as well.

Systematic evaluation must be undertaken. This study is the first evaluation of

the impact of hnle.org. The development efforts of the UHM HNLC team must be

grounded in research on the appropriateness of the tools developed for the HNLC

community. Additionally, the degree to which the tools address real challenges

faced by participants must be assessed.

There must exist a shared purpose, and that purpose must be clear. It is the

nature of systemic reform that the vision of the reform effort is built

collaboratively by participants over time. Even now, the UHM and RlDOE teams

continue to engage in a process intended to clarify and crystallize the goals of

HNLC and hnle.org. These efforts must continue and must eventually be checked

against the realities of student achievement and of participant adoption of hnle.org

as a tool to support their work.

Intended outcomes must be clear. Much of the hnlc.org development has been

driven by intuition about participants' needs. For each proposed development

83

effort, the questions should be asked, "What will be achieved by this, and how

will we know when we have achieved it?"

• Competing priorities must be resolved, or at least understood. The

complexities of state-wide education reform inherently bring with them

competing priorities. HIDGE and UHM must work closely to ensure that hnlc.org

development efforts represent the best understanding of HNLC's goals.

In short, care must be taken to ensure that future HNLC and hnlc.org development

are grounded in existing research and that all members of the HNLC teams share a

common understanding of the rationale, goals, and intended outcomes of the initiative.

Conclusion

This study provides an analysis of the impact of the hnlc.org website during its

first two years of operation. Results indicate that hnlc.org has not achieved its intended

goals, but suggest that participants say they are open to uses of technology to support

student learning as well as their own teaching and professional development. In order to

make use of participant enthusiasm, hnlc.org must be carefully reconceived based on the

recommendations, above.

Additionally, there are open questions that should be addressed in order to fully

understand the successes and failures of HNLC and hnlc.org. For example:

• What is the model of professional development being used by HNLC, and how

can hnlc.org reinforce and support that model?

•

•

•

84

Are there uses of technology by students that would be fundamentally more

compelling for educators, and therefore more likely to draw educators to the use

of online tools?

Given the potential for Communities of Practices suggested in the literature, can

hnlc.org be reconceived to support already existing CoPs?

How does hnlc.org impact student learning?

The promise of technology to support educators' practice and improve student

learning is an enticing prize. However, in pursuit of that prize we must constantly check

and recheck our good intentions against past experience and ensure that we ask the

fundamental question of all education reform: "how does what we're doing improve

student learning?"

85

APPENDIX A. NSF'S DRIVERS FOR SYSTEMIC REFORM

Developed in response to the need for provable progress in NSF funded education

reform efforts, NSF identified six factors that are believed to improve chances of success

of systemic reform efforts. These drivers serve as a measure of accountability across all

NSF systemic reform efforts.

Driver 1

Implementation of a comprehensive, standards-based curriculum and/or

instructional materials that are aligned with instruction and assessment available to every

student served by the system and its partners.

Driver 2

Development of a coherent, consistent set of policies that supports provisions of

broad-based reform of mathematics and science at the K-12 level.

Driver 3

Convergence of all resources that are designed for or that reasonably could be

used to support science and mathematics education-fiscal, intellectual, materials-both in

formal and informal education settings-into a focused program that upgrades and

continually improves the educational program in science and mathematics for all

students.

86

Driver 4

Broad-based support from parents, policymakers, institutions of higher education,

business and industry, foundations, and other segments of the community for the goals

and collective value of the program that is based on an understanding ofthe ideas behind

the program and knowledge of its strengths and weaknesses.

The last two are "outcome drivers" that underscore the student-centeredness of the

entire systemic reform effort.

Driver 5

Accumulation of broad and deep array of evidence that the program is enhancing

student achievement through a set of indices. In the specific instance of student

achievements test scores, awardees on an annual basis, are expected to report the results

of student mathematics and science achievements in a multi-grade level context for the 51

impacted schools/districts/state(s) relative to appropriate cohort entities (non-51 districts,

the state), all of which are defined by the performance baseline. And,

Driver 6

Improvement in the achievement of all students, including those historically

underserved, as evidenced by progressive increments in student performance

characterized by the requisite specificity of the 51 as a catalytic resource and the

appropriateness of attendant attributions.

APPENDIX B. SUMMER INSTITUTE SURVEY

87

ABOUT YOU

What is your primary role in your school: - Classroom teacher

- HNLC Teacher Partner

- Tech coordinator_ Library media specialist

- Administrator

- Other:

How many years have you been an educator?

How many years have you been an educatorin Hawaii?

How did you become a member of your - I volunteered

school's HNLC team? _ I was assigned

Did you playa part in originally deciding to - No

involve your school with the HNLC - Yes

initiative?

How many months have you personally beenaffiliated with HNLC?

ABOUT YOUR SCHOOLConsider the technology planning that may have occurred in your school.

Does your school have a technology - No

coordinator (possibly you)? _ Yes, part time_ Yes, full time

88

If 'yes': _ Several times per week

How often do you discuss your technology _ At least once per week

needs with your school's technology_ At least once per month_ At least once per qnarter

coordinator? _ At least once per yearor - Never

If you are the technology coordinator for - Other:

your school, how often do teachers in yourschool discuss their technology needs withyou?

Does your school have a plan to maintain, - No

replace, and update computer hardware? _ We have a plan in place_ We are in the planning stages_ Maybe - I'm not snre

Does your school have a plan to maintain, - No

replace, and update computer software? _ We have a plan in place_ We are in the planning stages_ Maybe - I'm not snre

AVAILABILITY OF COMPUTERS & RELATED TECHNOLOGIESConsider the computer resources available to you and your students in your school.

Teachers: How many computers are locatedin your classroom for use by your students?

Librarians: How many computers arelocated in your library facility for use bystudents?

Administrators: On average, how manycomputers are located in each science andmath classroom for use by students?

Do students have access to a computer lab or - No

other collection of computers within the - Yes

school other than those above?

89

if 'yes': approximately how many computersare available for student use in the computerlab?

Do students have Internet/Web access from - No

school? _ Yes, with software content filters_ Yes, when supervised hy a

staff memher

._ Yes, without restriction

Do students have email accounts that they _ No, they are not allowed to

are allowed to access during school hours? use email from school_ Yes, accounts provided hy

the school_ Yes, through an outside service_If they do, it's on their own

Do students have electronic storage space for - No

hosting student web pages or other electronic _ Yes, provided by the school

work (iMovies, presentations, etc.)?_ Yes, through an outside service_If they do, it's on their own

Which of the following does your school have available for your use or for studentuse (please indicate how many of each item are available in the appropriatecolumn):

For your use For student useIma2e scannerStill-shot di2itaJ cameraAnalo2 video camera (8MM, etc)Di2ital video camera (DV, etc)Black & white printerColor printerOther:Other:Other:

90

HNLC.ORGConsider the HNLC website, the tools and information found there, and your experiencewith them to date.

How often do you visit the _ Several times per week

HNLCORG website _ At least once per week

(regardless of whether you_ At least once per month_ At least once per quarter

login to your account or not)? _ At least once per academic year

- Never_Other:

When you visit the HNLCORG _ Always (> 90% of the time)

site, how often do you login _ Almost always (75 - 90% of the time)

using your HNLC user account?_ Sometimes (about 50% of the time)_ Almost never (10 - 25% of the time)

- Never_Other:

The purpose of HNLC.ORG is Strongly I Agree INeutral IDisagree I Strongly

clear to me agree disagree

HNLCORG is relevant to the Strongly IAgree INeutral IDisagree IStrongly

success of students agree disagree

HNLCORG has a positive Strongly IAgree INeutral IDisagree IS~onglyimpact on my professional agree disagree

activities (e.g. teaching,administrative duties, etc.)

HNLC.ORG has had a positive Strongly IAgree INeutral IDisagree IS~ronglyimpact on the science, math, agree dIsagree

and technology programs in myschool

HNLCORG provides services Strongly IAgree INeutral IDisagree IStrongly

to me that are unavailable agree disagree

elsewhere (for example: unitplan support, listings of relevantresources, discussion spaces)

The service that HNLC.ORGhas provided that I find mostvaluable is ...

The service that HNLC.ORGhas provided that I find leastvaluable is ...

Additional services I would liketo see HNLC.ORG provideinclude...

Other, non-HNLC.ORG, online,education-oriented services Iuse include ...

91

IMPACT OF COMPUTER TECHNOLOGY ON TEACHINGConsider how computer technology in general, not just HNLC.ORG, influences yourteaching.Ifyou do not currently teach, please skip this section.

How often do you use _ Several times per week

computers for in-class _ At least once per week

instruction?_ At least once per month_ At least once per quarter_ At least once per academic year_Never

Other:

Computers allow me to Strongly IAgree INeutral IDisagree IStrongly

incorporate innovative agree dIsagree

instructional methods in myclassroom

92

Computers allow me to better Strongly IAgree INeutral IDisagree IS~ronglyaddress the individual needs of agree dISagree

my students

Computers are relevant to my Strongly IAgree INeutral IDisagree IStrongly

teaching agree disagree

Online tools (discussion spaces, Strongly IAgree INeutral IDisagree IStrongly

chat, email, guided professional agree dISagree

activities, etc) can be used tosupport my teaching

The Internet is a valuable Strongly IAgree INeutral IDisagree IStrongly

resource for instructional agree dISagree

planning ideas

I actively seek ways to Strongly IAgree INeutral IDisagree IStrongly

incorporate computers into my agree disagree

lessons

I have sufficient technology Strongly IAgree INeutral IDisagree IStrongly

training to incorporate agree disagree

computers into my teachingsuccessfully

Computers interfere with my Strongly IAgree INeutral IDisagree IStrongly

teaching agree disagree

Seeing how my students use Strongly IAgree INeutral IDisagree IS~onglycomputers has given me new agree dISagree

ideas about how to usecomputers more effectively inmy teaching

What I most value aboutincorporating computers intomy teaching is...

What I least value aboutincorporating computers intomy teaching is ...

93

IMPACT OF COMPUTER TECHNOLOGY ON STUDENT SUCCESSConsider how students' use of computers influences their academic success and what thechallenges and opportunities of such technologies are.

How often do students you _ Several times per week

work with use computers for _ At least once per week

in-class instructional purposes?_ At least once per month_ At least once per quarter_ At least once per academic year

- NeverOther:

Computers are relevant to Strongly I Agree I Neutral IDisagree I Strongly

students' success in school agree disagree

Computers are relevant to Strongly IAgree INeutral IDisagree IStrongly

students' success in life agree disagree

The Internet is a valuable Strongly IAgree INeutral IDisagree I S~onglyresource for students' academic agree dIsagree

work

Limited access to computers Strongly IAgree INeutral IDisagree IStrongly

hinders students' success agree disagree

Limited availability of online Strongly IAgree INeutral I Disagree I S~ronglycontent hinders students' agree dIsagree

success

Too much information online Strongly IAgree INeutral IDisagree IS~ronglyhinders students' success agree disagree

94

Lack of quality materials online Strongly I Agree INeutral I Disagree I Strongly

hinders students' success agree disagree

Students I work with have been Strongly IAgree INeutral IDisagree IStrongly

inspired to find new ways to agree disagree

use computers themselvesbecause of my use ofcomputers

What I most value aboutstudents' use of computers forlearning is...

What I least value aboutstudents' use of computers forlearning is ...

IMPACT OF COMPUTER TECHNOLOGY ON PROFESSIONALDEVELOPMENTConsider how computers and online content influence your continuing professionaldevelopment efforts.

I use online professional _ Several times per week

development resources and _ At least once per week

materials to support my_ At least once per month_ At least once per quarter

professional growth ... _ At least once per year

- NeverOther:

I believe communicating with Strongly IAgree INeutral IDisagree IStrongly

other teachers via technology is agree dlsagree

important to my professionalgrowth

95

Active collaboration with other Strongly IAgree INeutral I Disagree I Strongly

teachers via technology is agree dIsagree

important to improve myteaching

I have sufficient access to Strongly IAgree INeutral I Disagree I Strongly

computers to support my agree dIsagree

professional growth

Limited availability of online Strongly IAgree INeutral IDisagree IStrongly

content focused on educators' agree dIsagree

professional developmentlimits my professional growth

I have been inspired by my Strongly IAgree INeutral , Disagree I S~onglystudents' uses of computers to agree dIsagree

find new ways to usecomputers to support myprofessional development

I am overwhelmed by the Strongly IAgree INeutral IDisagree IS,trongly

amount of professional agree dIsagree

development material availableonline

What I most value about usingcomputers to aid myprofessional growth is...

What I least value about usingcomputers to aid myprofessional growth is ...

I consider myself to be an Strongly I Agree INeutral I Disagree I S.trongly

active member of an online agree dIsagree

educational community

%

I prefer not to use online Strongly IAgree INeutral IDisagree IS~nglycommunications tools for agree dIsagree

professional networking

I use email for professional _ Several times per week

networking... _ At least ouce per week_ At least once per month_ At least once per quarter_ At least once per year

- NeverOther:

I use discussion boards for _ Several times per week

professional networking... _ At least once per week_ At least once per month_ At least once per quarter_ At least once per year

- Never

- Other:

I use real-time, multi-person _ Several times per week

chat rooms for professional _ At least once per week

networking. "_ At least once per month_ At least once per quarter_ At least once per year

- NeverOther:

I use instant messaging (AIM, _ Several times per week

MSN Messenger, etc) for _ At least once per week

professional networking..._ At least once per month_ At least once per quarter_ At least once per year

- Never- Other:

I believe that I can get the Strongly IAgree INeutral IDisagree IS~ronglyprofessional development agree dIsagree

support I need completely fromwithin my school

97

For each of the following, have you had access to or made use of these types ofcomputer training (check all that apply - note that indicating that you "have used" atype of training necessarily implies that you also "have access to" it):

Have access to Have usedSelf-training tutorialsSchool sponsored in-serviceDOE sponsored in-serviceSessions at conferencesOutside courses (e.g. at auniversity or college)Online coursesOther:Other:Other:

98

APPENDIX C. SAMPLE WEB SERVER ACCESS LOG

Typical Log Entries

Valid accessType of eventEvent entry in log file

cache-daOl.proxy.aoI.com - - [I I/Aug/2002:05:19:22 -1000]"GET /aboutlmission.html HTTP/l.O" 200 15479

.--..----.-.--- - --:::=c::-::---------------------c7pub-216-250-141-186.center7.com - - [1l/Aug/2002:05:26:03 - Valid access}QQ.QL:Q.§I/b<.>~~.h!ml IqTP/1.0"__.:w:±_~?9 .. .._

__~!.:_83..:..~!.~.:~9._.:: ..:J!lIA~g/2002:07:57:04 ..:.!OOO] "GET" _?90Q__ Valid acces~ ... _128.255.120.201 - - [I I/Aug/2002:08: 17:47 -1000] "GET Break in attempt/default.ida?NNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNN%u9090%u6858%ucbd3%u7801 %u909O%u6858%ucbd3%u7801%u9090%u6858%ucbd3%u7801 %u9090%u9090%u8190%uOOc3%u0003%u8bOO%u531b%u53ff%uO078%uOOOO%uOO=a HTTP/l.O" 400 317-- ---_..__.._._--------_._.....-.__._---_.•.•_.__.._---------".-._-_.._----_.._._--_."."."210.212.52.3 - - [12/Aug/2002:01:12:40 -1000] "GET Break in attempt/_vtCbin/.. %255c..J..%255c../.. %255c..Iwinntlsystem321cmd.exe?/c+dir HTTP/l.O" 404 314

..•,-'-_._---"."._--_._--------,.- ---66.237.60.38 - - [1l/Aug/2002: 13:06: 18 -1000] "GET /robots.txt Search engineHTTP/l.O" 404 269 _ ... ._.~ide_~ ... ._._..._..._. __ ..66.237.60.38 - - [1l/Aug/2002:13:08:26 -1000] "GET /robots.txt Search engineHTTP/l.O" '¥J4 ~69._ ... .... ..__...... .. _ .!!piderdhcp-grp2-07.ics.hawaiLedu - - [1l/Aug/2002:13:30:37 -1000] HNLC Team"GET / HTTP/1.1" 20023609--.-----..--..-..-.------..----- ----------- ---------.::-:::-:=---128.171.10.85 - - [ll/Aug/2002:13:35:19 -1000] "GET HNLC Team/discussions/servletlnews?config=hnlc HTTP/l.1" 200 5099 _ _ _

Log Entries Removed

Access log entries representing attempts to break in to the HNLC servers were

removed for purposes of analysis. Additionally, requests originating from the following

99

domains and IP addresses were removed for purposes of determining the total number of

unique visits to the hnle.org website. These domains represent website traffic from UHM

HNLC team members and web search engines (i.e., log entries representing search

engines attempting to index the content of the hnle.org site, and not valid traffic that

reached hnle.org by first using a web search engine). An asterisk indicated any numeric

or text. For example, "128.171. IO. *" indicates that all values from 128.171. IO.O through

128.171. IO.255, the valid range of Internet Protocol address, were removed.

128.171.10.* (UHM ICS faculty staff subnet)128.171.20.* (UHM ICS lab subnet)128.171.140.* (UHM ICS network support subnet)128.171.104.* (UHM ICS research subnet)128.171.58.* (UHM LIS subnet; one PI is located in LIS)*.ics.hawaii.edu (home department of UHM HNLC development team)*.google.com (search engine spider)*.yahoo.com (search engine spider)*.inktomisearch.com (search engine spider)*.fastsearch.net

A large percentage of traffic to the hnlc.org website originates from within the

HIDOE ATRB network. Much of this traffic represents usage for demonstrations at

HNLC staff meetings and training of HIDOE staff. This traffic also includes HNLC 2003

Summer Institute traffic, since the 2003 institute was held at the HIDOE ATRB offices.

Traffic from the ATRB network was removed in order to determine how many HNLC

participants logged into the hnle.org site outside of HIDOE structured meetings.

Therefore, for part of the server log analysis, the following were also removed:

kala.k12.hi.uskala2.k12.hi.us

100

APPENDIX D. FOCUS GROUP DISCUSSION GUIDELINES

Morning Session Focus Topics

These are intended to be used by the moderators to spark conversation among

participants and to aid in focusing the discussion. While not intended as a checklist of

topics to be covered, these should offer some possibilities for inviting the participants to

carry on a lively discussion.

Inquiry learning projects at their school

Current practice

Current challenges

Ideas for improvement

•

•

••

•

•

••

•

•

•

•

•

Are you involved with any inquiry-learning projects atyour school?How have you been integrating inquiry learning intoyour classroom?Are your students currently working on any projects?Are there other teachers at your school incorporatinginquiry learning in interesting ways?What benefits have you gotten from your students'engaging in inquiry learning?

What problems do you see with incorporating moreinquiry-learning techniques in your classroom?How has inquiry learning affected your teaching?What support is there at your school for teachersincorporating inquiry learning?What do you think / how do you feel about inquirylearning methods?How could students gain more from their inquiryexperiences?What would help you to feel more successful bringinginquiry learning into your classroom?What do you need to support inquiry learning in yourclassroom?Are there other examples of inquiry learning that youadmire/emulate?

Standards implementation

101

• Imagine your ideal conditions for inquiry learning ­what would these be and what would it take to getthere?

Current practice

Current challenges

Ideas for improvement

•

•

••

•

••

•

•

•

•

•

Can you think of examples of how you're incorporatingHCPS into your planning?How is your school supporting your efforts tounderstand and incorporate HCPS in your teaching?How do HCPS relate to how you plan your classes?How do HCPS influence how you plan yourinstruction?What problems do you see with bringing HCPS intoyour classroom?What have been the downsides of HCPS for you?Have there been any problems working with HCPS inyour school?How could you be more successful working with theHCPS?What can the DOElHNLC do to better support your useof HCPS?What information do you need to betterunderstand/incorporate HCPS?How could HCPS be better integrated with whatteachers do?Imagine your ideal conditions for standardsimplementation - what would these be and what wouldit take to get there?

Technology integration with teaching (curriculum)

Current practice ••

•

How often do you use computers for instruction?Do you have access to the internet from yourclassroom?Do your students submit any work or projects to youonline?

Current challenges

Ideas for improvement

•

•

•

•

•

•

•

•

•

•

102

Do your students use Powerpoint or anything like thatto give presentations in your class?What other technologies do you use in your classes?(digital microscopes, digital probes, etc)What do you see as the biggest barrier to incorporatingtechnology into your classroom?What have been the benefits of technology for you? Foryour students?What problems have you had working withtechnology? What problems have your students had?Have you had many problems working in class withcomputers? Can you describe some of them?What is the most critical problem you face whenworking with technology?What improvements would help you to integratetechnology in your teaching?Can you think of anything that "you wish you had?"Anything that would significantly increase the value oftechnology for you? For your students?What is your "top 5" wish list for technology? What doyou want/want to be able to do that you can't rightnow?

103

APPENDIX E. INTERVIEW QUESTIONS

In additional to completing the 2003 Summer Institute Survey, participants in the

summer institute were asked to indicate their willingness to be contacted with follow up

questions for the purposes of clarifying the results of the survey. Those who indicated

that they would be willing to be contacted were emailed the following questions.

1. One goal of the hnlc.org website is to facilitate an active, online community of

educators. Do you perceive hnlc.org as serving that role? Why or why not?

lao What role, if any, would you play in such an online community?

2. Have you had difficulties with not being able to login to hnlc.org? Specifically,

have you experienced problems (possibly as simple as not knowing your password or

remembering the web address of the website) logging in that would prevent you from

logging in outside of official HNLC meetings?

3. Which of the online tools we've provided do you actually use? (i.e., story

posting facility, unit plan template, resource database, discussion space)

3a. If so, please give examples of how the tools have helped you.

4. When do you use hnlc.org tools? (i.e., only as lead up to HNLC meetings, only

in HNLC meetings, or outside of HNLC meetings, too?)

5. What have you observed about HNLC or hnlc.org that has influenced your use

of HNLC.ORG tools, either to use the tools more or to use them less?

104

6. When you hear the phrase "working collaboratively with others online," what

scenarios do you envision?

7. What has allowed or prevented you from using the tools to support working

collaboratively with other HNLC participants?

8. Do you believe that since becoming involved with HNLC you have grown

more likely to use computer technologies to support your professional activities

(teaching, administration, research, etc.)?

9. Do you believe that since becoming involved with HNLC you have grown

more likely to use computer technologies to support your professional activities to

support student learning?

9a. Why or why not?

10. Do you believe that since becoming involved with HNLC you have grown

more likely to use computer technologies to support your professional activities to

support your own professional development?

lOa. Why or why not?

105

APPENDIX F. HNLC COHORT SCHOOLS

The student enrollment numbers presented below are based on the HIDOE's

Official Enrollment Count for the 2002-2003 School Year.

Island StudentsCOHORT 1

812281237580509672

10071249566863

6776

Hawai'iHawai'iHawai'iHawai'iHawai'i

MauiMaui

KauaiKauaiKauai------

Honokaa High SchoolHookenaLaupahoehoeWaikaloaWaimea MiddleLanaiMaui Waena IntermediateKauai HighWaimea CanyonWaimeaHi h_._-_.__._---------_._._------Total Cohort 1

COHOR 2Haleiwa O'ahu 279Waiahole O'ahu 120Honokaa Elementary Hawai'i 381Kahakai Hawai'i 591Kealakehe Hawai'i 965Kohala Elementary Hawai'i 426Mountain View Elementary Hawai'i 416Pahoa Elementary Hawai'i 442Lokelani Intermediate Maui 773Molokai Maui 748Kapaa ~i~_d_Ie--=-- K_a_u_a_i 8-=---0'----1_Total Cohort 2 5942

106

REFERENCES

Anderson, R. E., & Ronnkvist, A. (1999). The presence ofcomputers in Americanschools (No. Report #2). Irvine, CA: Center for Research on InformationTechnology and Organizations, University of California, Irvine, and theUniversity of Minnesota.

Barab, Sasha A., MaKinster, James G., & Scheckler, Rebecca. (2003). Designing systemdualities: characterizing a web-supported professional community. TheInformation Society, 19. (pp 237-256).

Bell, Randy L., & Tai, Robert H. (2003). Transforming science Instruction withTechnology. In Arthur D. Sheekey (Ed.), How to ensure ed/tech is not oversoldand underused (pp. 11-24). Lanham, MD: Scarecrow Press, inc.

Bertalanffy, Ludwig von. (1976). General Systems Theory: foundations, development,applications. New York, NY: George Braziller, Inc.

Blumenfeld, Phyllis, Fishman, Barry J., Krajcik, Joseph, Marx, Ronald W., Soloway,Elliot. (2000). Creating usable innovations in systemic reform: scaling uptechnology-embedded project-based science in urban schools. EducationalPsychologist, 35(3), 149-164. Lawrence Erlbaum Associates, Inc.

Borasi, Raffaella & Fonzi, Judith. (2002). Professional development that supports schoolmathematics reform. Foundations, Vol. 3. Washington, DC: National ScienceFoundation, Division of Elementary, Secondary, and Informal Education.Retrieved September 30, 2003 fromhttp://www.nsf.gov/pubs/2002/nsf02084/start.htm

Bredeson, Paul V. (2003). Designsfor learning. London: SAGE Publications.

Converse, Jean M. & Presser, Stanley. (1986). Survey questions: handcrafting thestandardized questionnaire. London: SAGE Publications.

Cradler, John & Bridgforth, Elizabeth. (1996a). Recent research on the effects oftechnology on teaching and learning. San Francisco, CA: WestEd RegionalEducational Laboratory. Retrieved September 4, 2003 fromhttp://www.wested.org/techpolicy/

Cradler, John & Bridgforth, Elizabeth. (1996b). Technology as a catalyst for educationreform. San Francisco, CA: Far West Laboratory for the California Department ofEducation. Retrieved September 4,2003 from http://www.wested.org/techpolicy/

107

Craig, Helen & Perraton, Hilary. (2003). Open and distance education for teachers'continuing professional development. In B. Robinson & C. Latchem (Eds.),Teacher education through open and distance learning (pp. 91-111). London:Routiedge/Falmer.

Cuban, Larry. (1986). Teachers and machines: the classroom use oftechnology since1920. New York, NY: Teachers College Press.

Cuban, Larry. (1993). How teachers taught: constancy and change in Americanclassrooms, 1890-1990. New York, NY: Teachers College Press.

Cuban, Larry. (2001). Oversold and underused: computers in the classroom. Cambridge,MA: Harvard University Press.

Darling-Hammond, Linda & Ball, Deborah. (1997). Teaching for high standards: whatpolicymakers need to know and be able to do. Retrieved October 4, 2003 fromhttp://negp.gov/reports/highstds.htm

Femeding, Karen A. (2003). Questioning technology: electronic technologies andeducational reform. New York, NY: Peter Lang Publishing

Fishman, Barry, Marx Ronald W., Blumenfeld, Phyllis, Krajcik, Joseph, & Soloway,Elliot. (in press). Creating a framework for research on systemic technologyinnovations. Journal ofthe Learning Sciences.

Fullan, Michael. (1991). The new meaning ofeducational change. London: Cassell.

Garet, Michael S., Porter, Andrew C., Desimone, Laura, Birman, Beatrice F., & Yoon,Kwang Suk. (2001). What makes professional development effective? Resultsfrom a national sample of teachers. American Educational Research Journal, 38no. 4. (pp. 915-945).

Harada, Violet H., Kajioka, Victoria, & Suthers, Daniel D. (2001). Hawai'i NetworkedLearning Communities; Proposal to the National Science Foundation, ESR­Rural Systemic 1nitiative Program. NSF 0100393. Honolulu, HI: Hawai'iDepartment of Education.

Hargreaves, Andy. (1994). Changing teachers, changing times: teachers' work andculture in the postmodern age. New York, NY: Teachers College Press.

Harris, Alma. (2000). Effective leadership and departmental improvement. WestminsterStudies in Education, 23. (pp. 81-90).

108

Harris, Alma. (2002). School improvement: what's in itfor schools? London:Routledge/Falmer.

Hawai'i Educational Policy Center (HEPC). (2003). Just the facts ... A citizen's primer onHawai'i K-12 public education. Retrieved October 3, 2003 fromhttp://www.hawaii.edu/hepc/pdf/Reports/Primer.pdf

Hon-Chan, Chai & Mukherjee, Hena. (2003). Policy, planning, and management ofdistance education for teacher education. In B. Robinson & C. Latchem (Eds.),Teacher education through open and distance learning (pp. 48-71). London;Routledge/Falmer.

Kent, Todd W. & McNergney, Robert F. (1999). Will technology really changeeducation? London: SAGE Publications.

McLaughlin, Milbrey W. & Mitra, Dana. (2001). Theory-based change and change-basedtheory: Going deeper, going broader. Journal ofEducational Change 1. (pp. 1­24).

McLaughlin, Milbrey W. & Talbert, Joan E. (2001). Professional communities and thework ofhigh school teaching. Chicago, IL: University of Chicago Press.

Means, Barbara. (1998). Models and prospects for bringing technology-supportededucation reform to scale. Paper presented at the annual meeting of the AmericanEducational Research Association, San Diego, CA.

Morgan, David L. & Krueger, Richard A. (1993). When to use focus groups and why. InDavid L. Morgan (Ed.), Successful focus groups (pp. 3-19). London: SAGEPUblications.

Moursund, David. (2003). Computers in preservice and inservice teacher education.Eugene, OR: University of Oregon. Retrieved September 6, 2003 fromhttp://www.uoregon.edu/~moursund/ICT-planning/

National Commission on Mathematics and Science Teaching for the 21" Century. (2000).Before it's too late. U.S. Department of Education. Retrieved October 4,2003from http://www.ed.gov/americacounts/glenn

National Science Foundation (NSF). (2000). Rural systemic initiatives in science,mathematics and technology education. (NSF 00-47). Washington, DC: NationalScience Foundation. Retrieved September 6, 2003 fromhttp://www.nsf.gov/pubsys/ods/getpub.cfm?nsfOO47

109

Robinson, Bernadette & Latchem, Colin. (2003a). Teacher education: challenge andchange. In B. Robinson & C. Latchem (Eds.), Teacher education through openand distance learning (pp. 1-27). London: Routledge/Falmer.

Robinson, Bernadette & Latchem, Colin. (2003b).Conclusions. In B. Robinson & C.Latchem (Eds.), Teacher education through open and distance learning (pp. 234­243). London: Routledge/Falmer

Saint-Onge, Hubert & Wallace, Debra. (2003). Leveraging communities ofpractice forstrategic advantage. Burlington, MA: Butterworth-Heinemann.

Schlager, Mark S. & Fusco, Judith. (2003). Teacher professional development,technology, and communities of practice: are we putting the cart before the horse?The Information Society, 19. (pp 203-220).

Senge, Peter. (1990). The fifth discipline: the art and practice ofthe learningorganization. New York, NY: Doubleday/Currency.

Stanliski, Sherryl. (2000). Is a systems r(e)volution on the horizonfor education? AuroraNow Foundation. Retrieved October 9, 2003 fromhttp://www.auroranow.org/resource/SysRevolution.rtf

Suskie, Linda A. (1992). Questionnaire survey research: what works. Association forInstitutional Research, Tallahassee, FL.

Suthers, D., & Xu, J. (2002). Kukakuka: an online environment for artifact-centereddiscourse. Education track of the Eleventh World Wide Web Conference (WWW2002), Honolulu, May 7-11, 2002, pp.472-480. Retrieved September 30, 2003from http://www2002.org/CDROMIaltemate/252/

Webb, N. & Palincsar, A. (1996). Group processes in the classroom. In D. Berimer & R.Calfee, (Eds.), Handbook ofEducational Psychology. New York, NY: Simon &Schuster Macmillian.

Webster, Jennifer M. (2001). Faculty developmentfor outcome-based curriculum reformin the community college. Unpublished doctoral dissertation, Oregon StateUniversity. Retrieved Octeber 9, 2003 fromhttp://www.ejswebster.com/chapO_acknow.htm

Wenger, Etienne, McDermott, R., & Snyder, W. (2002). Cultivating communities ofpractice. Cambridge, MA: Harvard Business School Press.