Internet and Higher Education 14 (2011) 54–63

Contents lists available at ScienceDirect

Internet and Higher Education

The correlates of Taiwan teachers' epistemological beliefs concerning Internetenvironments, online search strategies, and search outcomes

Pei-Shan Tsai a, Chin-Chung Tsai a,b, Gwo-Jen Hwang c,⁎a Graduate Institute of Engineering, National Taiwan University of Science and Technology, #43, Sec.4, Keelung Road, Taipei, 106, Taiwanb Graduate Institute of Digital Learning and Education, National Taiwan University of Science and Technology, #43, Sec.4, Keelung Road, Taipei, 106, Taiwanc Department of Information and Learning Technology, National University of Tainan, 33, Sec. 2, Shulin Street, Tainan City 70005, Taiwan

⁎ Corresponding author. Tel.: +886 915396558; fax:E-mail address: gjhwang.academic@gmail.com (G.-J

1096-7516/$ – see front matter © 2010 Elsevier Inc. Aldoi:10.1016/j.iheduc.2010.03.003

a b s t r a c t

a r t i c l e i n f o

Keywords:

Web-based learningInformation-searching strategiesEpistemological beliefsConstructivism

This study aimed to explore the correlates among teachers' epistemological beliefs concerning Internetenvironments, their web search strategies and search outcomes. The sample of this study included 105teachers from 63 grades 1 to 9 schools in Taiwan. The results show that the teachers with more advancedepistemological beliefs concerning Internet environments could utilize more sophisticated web searchstrategies (i.e. less irrelevant information-selecting) to filter and organize the information than those withless advanced beliefs. Also, the sophistication of epistemological beliefs was positively correlated to thesearch outcomes for open-ended questions. Hence, epistemological beliefs concerning Internet environ-ments play an important role in web-based learning.

+886 6 3017001.. Hwang).

l rights reserved.

© 2010 Elsevier Inc. All rights reserved.

1. Introduction

Personal epistemology refers to individuals' conceptions regardingthe nature of knowledge (e.g. what individuals believe knowledge is)and knowing (e.g. how they come to know it) (Hofer & Pintrich,1997). Researchers have indicated that epistemological beliefs mayhave an effect on learning performance and approaches to learning(Hofer & Pintrich, 1997; Schommer, Crouse, & Rhodes, 1992). Thestudy of Tsai (2000) further showed that students' scientific episte-mological beliefs are related to their approaches to learning science.

Recently, the efficiency and popularity of the Internet has lentitself to integrating Internet-based learning activities into thecurriculum (Georgina & Olson, 2008; Muse, 2003; Tsai & Tsai, 2003).Several studies have demonstrated that epistemological beliefs areassociated with learning in Internet-based environments (e.g. Mason,Boldrin, & Ariasi, 2010; Tsai, 2004a; Tsai & Chuang, 2005; Tu, Shih, &Tsai, 2008). For example, Tu et al. (2008) found that eighth graderswith advanced epistemological beliefs had better performance inopen-ended search tasks in Internet-based learning environments,but the use of the Epistemological Belief Scale (EBS) in their study wasto assess students' epistemological beliefs in general, not particularlytoward Internet-based learning environments. Some studies havefocused on epistemological beliefs concerning Internet environments.For example, Bråten, Strømsø, and Samuelstuen (2005) developed aninstrument based on Hofer and Pintrich's (1997) model to surveystudents' Internet-specific epistemological beliefs about Internet-

based knowledge (e.g. what they believe knowledge is on theInternet) and knowing (e.g. how they come to know on the Internet).They found that students' Internet-specific epistemological beliefscould well predict their Internet activities of searching andcommunication.

In themeantime, several studies that took learners' preferences forlearning environments into account in exploring the epistemologicalbeliefs of learners have been conducted. For example, Moore (1989)developed the Learning Environment Preferences (LEP) survey tomeasure students' epistemological development, which has beenadopted by several researchers for identifying students' epistemolog-ical beliefs (e.g., Yang, 2005). Similarly, this study utilizes learningpreferences to assess epistemological beliefs concerning Internetenvironments.

Moreover, recent studies concerning preferences for Internet-based learning environments have shown the importance of prefer-ences to students and teachers (Chuang, Hwang, & Tsai, 2008; Chuang& Tsai, 2005; Lee & Tsai, 2005; Tsai, 2005, 2007, 2008). For example,Tsai (2008) developed a questionnaire to probe learners' preferencesin terms of constructivist Internet-based learning environments withadequate reliability and validity. The results showed that learnerswith higher average scores on the questionnaire were more likely tohold epistemologically constructivist-oriented (considered as moresophisticated) views toward Internet-based learning environments.Mason et al. (2010) also showed that through the think-aloudmethod, university students' epistemological beliefs were activeduring web information-searching processes. A similar perspectivewas expressed by Tu et al. (2008) that students' epistemologicalbeliefs were found to be associated with their web search strategies.Consequently, researchers have indicated that more advanced

55P.-S. Tsai et al. / Internet and Higher Education 14 (2011) 54–63

epistemological beliefs may help learners to develop more sophisti-cated standards for evaluating information search strategies in anInternet-based environment (Tsai, 2004b; Wu & Tsai, 2005).

Many studies have pointed out the importance of teaching studentshow to use search engines effectively for collecting information forproblem solving (e.g. Bilal, 2000; Hölscher & Strube, 2000). Therefore, itis important for school curricula to help students develop their abilitiesto gather and evaluate information, and to promote students' learningoutcomes and life quality (Bilal, 2002; Brand-Gruwel, Wopereis, &Vermetten, 2005). Some studies have been conducted to investigate theinformation-seeking behaviors of teachers (e.g. Tabatabai & Shore,2005), whichmay play an essential role in training the problem solvingskills of students.

In addition, learners' web search behaviors also influence theirsearch outcomes. Bilal (2000, 2001, 2002) compared successful andunsuccessful seventh grade students' search behaviors in varioussearch tasks and found significant differences between the searchbehaviors and search processes of the two groups of students. Similarfindings were reported by several researchers who indicated that thedifferences between the search behaviors of novices and experts mayhave caused the differences between their search performances(Hölscher & Strube, 2000; Kim, 2001; Tabatabai & Shore, 2005). Forexample, Tabatabai and Shore (2005) found that novices were moredisorientated in the web search process than the experts, which alsoinfluenced their search performance. Hwang, Tsai, Tsai, and Tseng(2008) investigated 220 fourth to sixth grade students' web searchstrategies for problem solving, and found that they were related totheir search outcomes for knowledge-finding questions.

In summary, some studies (e.g., Mason et al, 2010) havedemonstrated the relationships between learners' epistemologicalbeliefs and their search strategies. The interplay between learners'web search strategies and search outcomes have also been addressedby some researchers (e.g., Hwang et al., 2008). However, only alimited number of studies have investigated the relationships amongthese factors. The samples under investigation for these factors haveoften included students, but have rarely included school teachers.Also, even fewer studies have been conducted to explore epistemo-logical beliefs particularly related to Internet-based learning environ-ments. Therefore, this study not only investigated teachers'perceptions of constructivist Internet-based learning environmentsto understand their epistemological beliefs concerning Internetenvironments, but also explored the correlates with web searchstrategies and search outcomes, as shown in Fig. 1.

2. Methodology

This study aimed to investigate the relationship among teachers'epistemological beliefs concerning Internet environments, web searchstrategies and search outcomes. In the following subsections, theresearch design is expressed in detail.

Fig. 1. The possible interplay among epistemological beliefs concerning Internetenvironments, search strategies and search outcomes.

2.1. Participants

The participants of this study included 55 male and 50 femaleteachers from 63 grades 1 to 9 schools across the northern, southernand central areas of Taiwan. The median age of the participants was36. Most of the participating teachers had integrated the Internet intotheir instruction (n=102, 97%), and had computers (n=103, 98%)and Internet connections (n=101, 96%) at home. All of themparticipated in a training course for advanced studies on integratingthe Internet into their instruction. However, this kind of advancedtraining course for teachers is very rare in Taiwan.

2.2. Data collection

Computer classrooms, which were located in several elementaryschools in central and southern Taiwan, were used for data collection.In order to assess their epistemological beliefs concerning Internetenvironments, the participants were firstly asked to fill out theconstructivist Internet-based learning environments survey (CILES;Tsai, 2008). Then, an Internet-based problem solving environment,called Meta-Analyzer (Hwang et al., 2008), was introduced. Meta-Analyzer provides an environment for learners to access informationby utilizing one or more existing search engines (e.g. Yahoo andGoogle), and automatically records users' web search behaviors andtheir problem solving procedures. After being introduced to Meta-Analyzer, the teachers were further directed to search web-basedinformation for problem solving on an individual basis. That is, withthe implementation of Meta-Analyzer, each individual was requiredto answer four questions concerning the issues of building “nuclearpower plants.” These questions are described in detail later. Theparticipants were constrained to find the answers within 30 min, ontheir own, without any guidelines (such as encouragement to visitmore web pages or revisit web pages) or discussion. All of theparticipants finished the task within the time given. None of theteachers in this study requested more time to finish the task.

2.3. Questionnaire about teachers' epistemological beliefs concerningInternet environments

The questionnaire, the Constructivist Internet-based LearningSurvey (CILES) developed by Chuang and Tsai (2005), Tsai (2008)and Wen, Tsai, Lin and Chuang (2004), was applied to explore theteachers' epistemological beliefs concerning Internet environments. Itconsisted of nine scales (each of which contains five items), as shownin the following:

(1) The Relevance scale measures learners' preference for Internet-based learning environments that show authentic contents andrepresent real life situations, e.g., “When navigating in Internet-based learning environments, I prefer that they presentrealistic contexts for learning.”

(2) The Multiple sources scale measures learners' preference forInternet-based learning environments to provide variousinformation sources and interpretations, e.g., “When navigat-ing in Internet-based learning environments, I prefer that theycan guide me to rich relevant web resources.”

(3) The Challenge scale measures learners' preference for Internet-based learning environments to be challenging but helpful inproblem solving, e.g., “When navigating in Internet-basedlearning environments, I prefer that they help me to generatenew ideas.”

(4) The Student negotiation scale measures learners' preference forhaving opportunities to communicate with other learners inInternet-based learning environments, e.g., “In the Internet-based learning environment, I prefer that I can get the chanceto talk to other students.”

56 P.-S. Tsai et al. / Internet and Higher Education 14 (2011) 54–63

(5) The Inquiry learning scale measures learners' preference forhaving opportunities to take part in inquiry learning in Internet-based learning environments, e.g., “In Internet-based learningenvironments, I prefer that I can find out answers to questionsby investigation.”

(6) The Cognitive apprenticeship scale measures learners' prefer-ence for having opportunities to acquire timely guidanceprovided by Internet-based learning environments, e.g., “Whennavigating Internet-based learning environments, I prefer thatthey can offer timely guidance.”

(7) The Reflective thinking scale measures learners' preference forhaving opportunities to enhance self-reflective thinking inInternet-based learning environments, e.g., “In an Internet-based learning environment, I prefer that I can think deeplyabout new ideas.”

(8) The Critical judgment scale measures learners' preference forhaving opportunities to evaluate information in Internet-basedlearning environments, e.g., “In Internet-based learning envir-onments, I prefer that I can evaluate the features of variousinformation sources.”

(9) The Epistemic awareness scale measures learners' preferencefor having opportunities to explore the nature of knowledge inInternet-based learning environments, e.g., “When navigatingInternet-based learning environments, I prefer that they candisplay the source of the knowledge.”

The reliability (Cronbach's alpha) coefficients for these scales were0.91, 0.92, 0.81, 0.95, 0.90, 0.93, 0.94, 0.89, and 0.90, respectively. Eachquestionnaire item used a five-point Likert scale ranging from 1(strongly disagree) to 5 (strongly agree). That is, teachers with higheraverage scores on the scales were more likely to hold advancedconstructivist-oriented epistemological beliefs concerning Internetenvironments.

2.4. Searching questions

In this study, the participants were asked to answer the followingfour questions, regarding the issues of nuclear power plants:

(1) How many nuclear power plants are there in Taiwan? Whereare they located?

(2) What are the scientific principles of using nuclear power?(3) What are the advantages and disadvantages of nuclear power?(4) Do you agree or disagree with utilizing nuclear power? Why?

These questions were modified from those used by Tsai and Tsai(2003). It should be noted that Questions 1, 2 and 3 are viewed asknowledge-finding (more close-ended) questions, while Question 4 isviewed as an argument (open-ended) question. As the argumentquestion is expected to be time-consuming, this study only includedone such question due to the 30-mintue time constraint. Therefore,within the time constraint, the participants spent a relatively longertime on Question 4.

2.5. The analysis of teachers' web search behaviors

This study applies Meta-Analyzer, a web search environmentrecording the teachers' entire portfolios, including the keywords, thebrowsed web pages and the user behaviors on the web. The Meta-Analyzer was developed by Hwang et al. (2008). As shown in Fig. 2,the search interface of Meta-Analyzer consists of three operationareas: the question and answer area is located on the left-hand side,the information-searching area is located on the upper-right andincludes a set of control buttons (i.e. bookmark insertion/deletion/browsing and system demonstration) for providing several usefulfunctions for information-searching. The result area (i.e. the webpages found by the search engines and the list of learner's bookmarks)

is located on the lower-right of the window. For example, to answer aquestion, the teacher can input keywords to search for information,and then browse the web pages that might be relevant to the topic.

Fig. 3 shows a web page browsing example. The teachers can pastethe answers, and add the web page to the bookmark list if they thinkthat it is highly relevant to the topic. Most of them would re-organizeand refine the information found on the web.

Fig. 4 shows the bookmark management interface of Meta-Analyzer for individual learners. The bookmark information reflectseach learner's judgments of the degree of relevance for the web pagesin relation to the questions, which is very helpful for the researchersin analyzing the behaviors of the learners, and hence a certain part ofthe problem solving ability of each learner can be evaluated by usingthe bookmark information.

Fig. 5 shows the interface for showing the information-searchingportfolio and the statistical results of individual learners. The presentedinformation includes the answer to each question, the keywords, theweb pages that have been visited and the browsing time for each webpage, etc. The “operation” column records the behaviors of each learner,where 1 indicates “input keywords”; 2, “browsing web page”; 3, “addweb page to bookmark list”; 4, “remove the web page from thebookmark list”; 5, “the time of web page selection”; and 6, “revise theanswer”. The researchers can click on the corresponding link to tracethe actual content of each web page that has been browsed by thelearner.

In addition, Fig. 5 also shows that Meta-Analyzer automaticallytransformed each question's portfolios into fourteen quantitativeindicators. Hence, each question has its own fourteen quantitativeindicators. The following are the quantitative indicators produced byMeta-Analyzer:

(1) Maximum number of keyword sets used in a search operation.This indicator represents the maximum number of keywordsets in a search operation for searching for information. Forexample, if the teacher input the keyword sets “nuclear power”AND “nuclear energy in Taiwan” in a search operation and thisis the maximum keyword sets in a question, the indicator valueis equal to 2. It should be noted that all of the keyword setswere in Chinese, and the identification of keyword sets weremade by using space or Boolean logic such as AND, OR and NOT.

(2) Number of search attempts for answering the question. Thisindicator represents the number of search attempts to enterkeywords for searching information. For example, if the teacherinput the keyword sets “nuclear power” and “nuclear energyin Taiwan” respectively for a question, the indicator value isequal to 2.

(3) Total time for web page selection. This indicator represents thetotal number of seconds spent deciding to browse the webpages from the results which were returned by the searchengine. For example, if the teacher browsed fiveweb pages, andif before browsing these web pages, the teacher spent 10 s, 5 s,12 s, 6 s and 8 s on deciding to browse the web pagesrespectively, the indicator value is 10+5+12+6+8=41 s.

(4) Number of different non-adopted pages. This indicator includesthe number of different web pages which were browsed by thelearner, but not for answering the question. For example, if theteacher browsed five different web pages and applied three ofthem for answering the question, the indicator is equal to 2.

(5) Total time for browsing the different non-adopted pages afterexcluding the time of revisiting. This indicator includes thetotal seconds spent browsing different web pages, but whichwere not used for answering the question. But the total timehere excluded the time of revisiting non-adopted pages. Hence,it does not involve the time, which will be described later inindicator (11), that the teachers spent on revisiting non-adopted pages.

Fig. 3. Illustrative example of browsing a web page for problem solving.

Fig. 2. Search interface of Meta-Analyzer.

57P.-S. Tsai et al. / Internet and Higher Education 14 (2011) 54–63

Fig. 4. Bookmark management interface of Meta-Analyzer for individual learners.

58 P.-S. Tsai et al. / Internet and Higher Education 14 (2011) 54–63

(6) Number of different adopted pages. This indicator representsthe number of different web pages browsed by the participantfor answering the question.

(7) Total time for browsing the different adopted pages afterexcluding the time of revisiting. This indicator includes thetotal seconds spent browsing different web pages for answer-ing the question. But, it does not involve the time, whichwill bedescribed later in indicator (9), that the teachers spent onrevisiting adopted pages.

(8) Number of times for revisiting the adopted pages. Thisindicator represents the number of times in which the userrevisited the web pages adopted for answering the question.

(9) Total time for revisiting the adopted pages. This indicatorrepresents the total number of seconds spent revisiting webpages which were adopted for answering the question.

(10) Number of times for revisiting the non-adopted pages. Thisindicator represents the number of times of revisiting webpages by the learner, but not for answering the question.

(11) Total time for revisiting the non-adopted pages. This indicatorrepresents the total number of seconds spent revisiting webpages, but not for answering the question.

(12) Number of marked and adopted pages. This indicator representsthe number of web pages which have been added as bookmarksby the learner for answering the question. For example, if theteachermarked fiveweb pages and two of themwere applied foranswering the question, the indicator value is equal to 2.

(13) Number of marked but not adopted pages. This indicatorrepresents the number of web pages which have been added asbookmarks by the learner, but not for answering the question. Forthe example given above, the indicator value is equal to 3.

(14) Number of revisions made on the answer. This indicatorrepresents the number of revisions made by the participant for

revising the answer. For example, if the teacher revised theanswer once after the first time of filling in the answer, theindicator value is equal to 1.

Tseng, Hwang, Tsai, and Tsai (2009) investigated the opinions of79 elementary and junior high school teachers and 1343 elementaryschool students about the efficacy of Meta-Analyzer. Their resultsshowed that the teachers had higher satisfaction with Meta-Analyzer, which provided an effective and efficient way to find outlearners' web searching behaviors, than the students.

2.6. The analysis of teachers' search outcomes

In order to evaluate the participants' search outcomes, twoindependent researchers scored the participants' answers withoutconsidering their searching behaviors. Based on the correctness of theiranswers, a score ranging from 1 to 10 was assigned to Questions 1, 2,and 3 (the knowledge-finding questions). In a similarway, based on theparticipant's soundness of providing evidence and justifying claims, ascore ranging from 1 to 10 was assigned to Question 4 (the argumentquestion). The final score was the average of the two researchers' givenscores. Using the Spearman's pair-wise correlation method, the inter-rater reliability was 0.79 for the knowledge-finding questions and 0.80for the argument question, which were statistically significant.

3. Results

3.1. The descriptive statistics of the epistemological beliefs concerningInternet environments

Table 1 shows the average scores and standard deviations of theteachers for their epistemological beliefs concerning Internet

Fig. 5. Interface showing the information-searching portfolio and the statistical results of individual learners.

59P.-S. Tsai et al. / Internet and Higher Education 14 (2011) 54–63

environments. The scores on the nine scales of CILES are rather high(from 4.19 to 4.54), implying that the teachers have relativelyadvanced epistemological beliefs concerning Internet environments,with emphases on the features of these scales. When Chuang and Tsai(2005) and Tsai (2008) intended to explore students' epistemologicalbeliefs concerning Internet environments, they also found that,compared with traditional classroom settings, students have moreadvanced epistemological beliefs concerning Internet environments.In their studies, the students attained high scores (above 4 on a 1–5Likert scale) in CILES. Besides, Tsai (2009) further indicates thatstudents' interpretations of web-based learning are more sophisti-cated than their interpretations of learning in general. Hence, it may

Table 1Descriptive statistics of the epistemological beliefs concerning Internet environments(n=105).

Scales Mean S.D. Range

Relevance 4.53 0.65 1.0–5.0Multiple sources 4.51 0.62 1.0–5.0Challenge 4.19 0.63 1.0–5.0Student negotiation 4.22 0.72 1.0–5.0Inquiry learning 4.54 0.63 1.0–5.0Cognitive apprenticeship 4.50 0.65 1.0–5.0Reflective thinking 4.48 0.61 1.0–5.0Critical judgment 4.21 0.57 2.8–5.0Epistemic awareness 4.38 0.55 1.8–5.0

be hypothesized that when individuals (either students or teachers)learn in an Internet-based environment, their epistemological beliefsthey adopt are likely to be more sophisticated. A similar hypothesis ismade by Tsai (2009).

3.2. The descriptive statistics of the quantitative indicators of web searchbehaviors

This study applied the approach of Hwang et al. (2008), which wasdeveloped based on the study of Ford, Miller and Moss (2001, 2002),to analyze the factor structure among these indicators. Exploratoryfactor analysis was applied to analyze the variables to identify clustersof indicators and ensure the identification of statistically significantrelationships between indicators. Hence, for ease of representation,researchers can combine the indicators into several main searchstrategies. Hwang et al. (2008) further indicated that the exploratoryfactor analysis method was able to investigate the commonalitiesand differences between the indicators and recognize these indicatorsas the students' search strategies. Table 2 shows the descriptiveresults of the average values of each question's fourteen quantitativeindicators that summarize the web-based information-searchingbehaviors of teachers. That is, Meta-Analyzer automatically trans-formed each question's portfolios into fourteen quantitative indica-tors, and then this study averaged these four questions' fourteenquantitative indicators, respectively. It is worth noting that the meanof the indicator I1 is equal to 1 (i.e. an average of one keyword set for

Table 2Descriptive statistics of the quantitative indicators with n=105.

Quantitative indicators Mean S.D.

I1: Maximum number of keyword setsused in a search operation.

1.00 0.38

I2: Number of search attempts foranswering the question.

1.52 0.74

I3: Total time for web page selection. 117.20 184.52I4: Number of different non-adopted pages. 1.54 1.39I5: Total time for browsing the differentnon-adopted pages after excluding thetime of revisiting.

118.94 88.80

I6: Number of different adopted pages. 0.99a 0.43I7: Total time for browsing the different adoptedpages after excluding the time of revisiting.

83.44 63.41

I8: Number of times for revisiting the adopted pages. 0.44 0.51I9: Total time for revisiting the adopted pages. 26.72 37.26I10: Number of times for revisiting the non-adopted pages. 0.54 0.75I11: Total time for revisiting the non-adopted pages. 13.92 26.99I12: Number of marked and adopted pages. 0.33 0.37I13: Number of marked but not adopted pages. 0.13 0.26I14: Number of revisions made on the answer. 0.02 0.06

Note: Each value is the average value of each question's quantitative indicators.a As some of the teachers answered the questions without adopting any web content,

the mean of the indicator I6 was less than 1.0.

60 P.-S. Tsai et al. / Internet and Higher Education 14 (2011) 54–63

one question), suggesting that the issue of nuclear power plants is anissue of particular interest in Taiwan. Consequently, with an averageof one keyword set, a large amount of information can be obtained.Besides, since the four questions all relate to the issue of nuclearpower plants, the teachers may have found the answers of the laterquestions when looking for those of the previous questions. In fact, inthis decade, nuclear energy is an issue of heated public debate inTaiwan (Wu & Tsai, 2007a).

These indicators, as proposed by Hwang et al. (2008), includeddifferent types of units; for example, the number of adopted pagesand the time spent on the adopted pages. Therefore, the values of thefourteen quantitative indicators were standardized according to theirZ scores in advance, and then the exploratory factor analysis methodwas used to derive reasonable results. Table 3 presents the results

Table 3Rotated factor loadings and Cronbach's alpha values for the three factors of quantitativeindicators with n=105.

Items Factor 1 Factor 2 Factor 3

Factor 1: irrelevant information-selectingtendency, α=0.77

I5: Total time for browsing the differentnon-adopted pages after excluding thetime of revisiting.

0.79

I10: Number of times for revisiting thenon-adopted pages.

0.78

I11: Total time for revisiting the non-adopted pages.

0.77

I4: Number of different non-adopted pages. 0.69

Factor 2: Information-seeking ability, α=0.61I6: Number of different adopted pages. 0.74I7: Total time for browsing the differentadopted pages after excluding the timeof revisiting.

0.73

I3: Total time for web page selection. 0.73

Factor 3: Keyword-adopting ability, α=0.61I2: Number of search attempts foranswering the question.

0.83

I1: Maximum number of keywordsets used in a search operation.

0.68

Eigenvalue 2.94 1.89 1.11% of variance 32.95 21.04 12.33

Overall α=0.71, total variance explained is 65.96%.

derived by applying the exploratory factor analysis method, revealingthree factors among the indicators, referred to as “irrelevantinformation-selecting tendency” (factor 1), “information-seekingability” (factor 2), and “keyword-adopting ability” (factor 3).Explanations of these three factors are given as follows:

(1) Irrelevant information-selecting tendency: This factor repre-sents the behaviors of teachers who spent their time browsingnon-adopted pages. Teachers with a higher score for this factortended to browse the web pages without judging the webinformation in advance. In other words, teachers with higherirrelevant information-selecting tendency had less effectivesearch behaviors than others. Teachers with a lower score forthis indicator tended to have better online informationselection ability.

(2) Information-seeking ability: This factor represents the beha-viors of teachers who spent their time browsing adopted pages.Teachers with a higher score for this indicator tended tocorrectly integrate and judge the web information; hence, theyhave effective search behaviors.

(3) Keyword-adopting ability: This factor represents the behaviorsof teachers in terms of how they used keywords to search forinformation. Teachers with a higher score for this indicatortended to use advanced search syntax as keywords or morefrequent search attempts to enter keywords to search forinformation.

The eigenvalues of the three factors are all greater than 1.00, with avariance of 65.96% explained. An indicator within a factor wasretained only when its loading was greater than 0.50 on the relevantfactor, and less than 0.50 on the non-relevant factor. Accordingly, theinitial 14 indicators were reduced to 9 indicator items. The internalreliability indexes (alpha coefficients) of the three factors are 0.77,0.61, and 0.61, respectively; moreover, for the complete item set, thealpha coefficient is 0.71. These coefficients suggest that these factorsare sufficiently reliable for representing teachers' search strategies.

3.3. The descriptive statistics of web search outcomes

Table 4 shows the teachers' average scores and standard devia-tions for the knowledge-finding questions and argument question.They scored higher on the knowledge-finding questions, implyingthat they tended to answer the knowledge-finding questions betterthan the argument question.

3.4. The relationship between teachers' epistemological beliefsconcerning Internet environments and web search strategies

Among the teachers' epistemological beliefs concerning Internetenvironments and web search strategies, only irrelevant information-selecting tendency was significantly negatively correlated with therelevance scale (r=−0.28, pb0.01), the challenge scale (r=−0.21,pb0.05), the cognitive apprenticeship scale (r=−0.20, pb0.05), thereflective thinking scale (r=−0.20, pb0.05), the critical judgmentscale (r=−0.26, pb0.01) and the epistemic awareness scale (r=−0.23, pb0.05), as shown in Table 5. It is plausible that the moreadvanced epistemological beliefs concerning Internet environments,the better the teachers' ability to judge theweb information (i.e. lowerscore for the irrelevant information-selecting tendency). This is

Table 4Teachers' scores on the scales of the knowledge-finding questions and argumentquestion.

Questions Mean S.D. Range

Knowledge-finding questions 7.47 1.78 0.0–10.0Argument question 6.85 2.27 0.0–9.5

Table 5Correlation among teachers' epistemological beliefs concerning Internet environmentsand web search strategies (n=105).

Scales Irrelevant information-selecting tendency

Information-seeking ability

Keyword-adopting ability

Relevance −0.28** −0.09 −0.12Multiple sources −0.18 −0.07 0.00Challenge −0.21* −0.02 −0.02Student negotiation −0.12 −0.01 0.07Inquiry learning −0.16 −0.02 0.02Cognitiveapprenticeship

−0.20* −0.04 −0.03

Reflective thinking −0.20* −0.04 −0.05Critical judgment −0.26** −0.18 −0.03Epistemic awareness −0.23* −0.10 0.01

*pb0.05; **pb0.01.

61P.-S. Tsai et al. / Internet and Higher Education 14 (2011) 54–63

possibly because more advanced epistemological beliefs concerningInternet environments are consistent with a constructivist view, andtherefore, the teachers had better ability to judge the webinformation.

3.5. The relationship between teachers' epistemological beliefsconcerning Internet environments and web search outcomes

As shown in Table 6, the scores for the argument question weresignificantly positively correlated with the relevance scale (r=0.26,pb0.01), the multiple sources scale (r=0.20, pb0.05), the challengescale (r=0.28, pb0.01), the student negotiation scale (r=0.23,pb0.05), the inquiry learning scale (r=0.26, pb0.01), the cognitiveapprenticeship scale (r=0.25, pb0.01) and the epistemic awarenessscale (r=0.19, pb0.05). This means that the teachers' epistemologicalbeliefs concerning Internet environments, which indicated a prefer-ence for Internet-based learning environments more closely relatedto real life, richer web sources, more challenging but helpful inproblem solving, providingmore opportunities to communicate, moreopportunities for inquiry learning, more appropriate guidance andmore opportunities to explore the nature of knowledge, seemed tolead to better performance in the argument question. This is possiblybecause the nature of the argument question is related to openperspectives, and the Internet-based learning environments providedvarious ways to suit individual needs. Therefore, the more advancedepistemological beliefs concerning Internet environments an individ-ual acquires, the more information supporting perspectives and thebetter performance in the argument question he/shemay exhibit. Thisresult is also consistent with the constructivist view in that the moreadvanced epistemological beliefs concerning Internet environments,the better the ability of answering open-ended questions. However,the scores for the close-ended questions were not significantlycorrelated with any scale of epistemological beliefs concerningInternet environments.

Table 6Correlation among teachers' epistemological beliefs concerning Internet environmentsand web search outcomes (n=105).

Scales Scores for knowledge-finding Scores for argument

Relevance 0.03 0.26**Multiple sources 0.03 0.20*Challenge 0.06 0.28**Student negotiation 0.03 0.23*Inquiry learning 0.02 0.26**Cognitive apprenticeship 0.11 0.25**Reflective thinking 0.00 0.15Critical judgment 0.08 0.18Epistemic awareness 0.08 0.19*

*pb0.05; **pb0.01.

3.6. The relationship between teachers' web search strategies and searchoutcomes

Table 7 also shows that the web search strategies were related tothe search outcomes. Information-seeking ability was significantlypositively correlated with the scores for the knowledge-findingquestions (r=0.25, pb0.01) and the scores for the argument question(r=0.23, pb0.05). This means that integrating more related webpages to answer questions seemed to lead to better performance inthe knowledge-finding questions and argument question. That is,information-seeking ability seems to be an important factor correlat-ed with teachers' web search outcomes, regardless of the type ofsearch task.

Moreover, keyword-adopting ability was significantly positivelycorrelated with the scores for the knowledge-finding questions(r=0.39, pb0.01). This means that using more keywords seems tolead to better performance in knowledge-finding questions.

Furthermore, irrelevant information-selecting tendency was sig-nificantly positively correlated with the scores for the argumentquestion (r=−0.20, pb0.05). This suggests that integrating orreferring to more non-related web pages for answers would lead toless satisfactory performance in the argument question. Irrelevantinformation-selecting tendency plays an important role in answeringargument questions, that is, teachers' ability to judge the usefulness ofweb information.

4. Discussion and conclusions

This study presents an investigation of grades 1 to 9 schoolteachers' epistemological beliefs concerning Internet environments inTaiwan, and analyzes the correlations with their web search strategiesand search outcomes. The main finding derived from this studyindicates that teachers with more advanced epistemological beliefsconcerning Internet environments, consistent with the constructivistview, have more favorable web search strategies (i.e. less selection ofirrelevant information). This is possibly because the maturation oftheir epistemological beliefs concerning Internet environments helpsthem to adequately judge the usefulness of information provided bythe search engine. This finding is similar to the perspective reportedby Tu et al. (2008), that is, web users with more advanced episte-mological beliefs have better ability to more purposefully filter infor-mation. It implies that learners with more advanced epistemologicalbeliefs concerning Internet environments may have better web searchstrategies (i.e. less selection of irrelevant information) and betterability to judge the usefulness of information with regards to thesearch tasks, and thus obtain better search outcomes. This is alsosimilar to the perspective proposed by Tsai (2004b) and Wu and Tsai(2005) that learners with more advanced epistemological beliefs mayhave more sophisticated evaluative standards regarding web infor-mation and web search strategies.

In particular, this study also found that teachers' epistemologicalbeliefs concerning Internet environments were significantly positive-ly correlated with the scores for the argument question. The moreadvanced epistemological beliefs concerning Internet environmentspeople hold, the better abilities they have to answer argumentation-based questions. That is, epistemological beliefs concerning Internet

Table 7Correlation among teachers' web search strategies and search outcomes (n=105).

Web search strategies Scores for knowledge-finding Scores for argument

Irrelevant information-selecting tendency

0.05 −0.20*

Information-seeking ability 0.25** 0.23*Keyword-adopting ability 0.39** 0.06

*pb0.05; **pb0.01.

62 P.-S. Tsai et al. / Internet and Higher Education 14 (2011) 54–63

environments play a more essential role in tackling more open-endedquestions in Internet environments, similar to the finding revealed byTu et al. (2008).

Furthermore, teachers' web search strategies for seeking relevantinformation were positively correlated with the scores for both theknowledge-finding questions and the argument question; their websearch strategies for adopting keywords were significantly positivelycorrelated with the scores for the knowledge-finding questions only,and their irrelevant information-selecting tendencies were signifi-cantly negatively correlated with the scores for the argumentquestion only. In short, regardless of the type of search task, theteachers' selection of relevant information ability was found to be animportant factor. The teachers' keyword-adopting ability was onlyrelated to the knowledge-finding questions which were lower-levelquestions (i.e., finding facts), and their irrelevant information-selecting tendencies were related to the argument question, whichwas a relatively higher-level question. That is, irrelevant information-selecting tendencies play a role in tackling open-ended questions inInternet environments. The teachers with more advanced onlineinformation selection ability or better ability of filtering irrelevantinformation tended to exhibit better performance when dealing withmore open-ended questions.

In sum, this study found that both epistemological beliefsconcerning Internet environments and irrelevant information-select-ing tendencies, which indicate the ability to judge information foundon the web, are important factors in answering argument questions inInternet environments. In addition, teachers' ability to judge theusefulness of web information is related to their epistemologicalbeliefs. That is, teachers with more advanced epistemological beliefsconcerning Internet environments may have better ability to judgethe usefulness of web information (i.e. less selection of irrelevantinformation) and better search outcomes.

5. Limitations and future research

There are several limitations to the present study. Firstly, thisstudy did not consider participants' prior knowledge regarding theissue of nuclear power plants. However, it should be noted thatelementary school teachers in Taiwan are trained to teach almost all ofthe school subjects; therefore, they are likely to have had similartraining and a similar academic background. Further investigationbased on the differentiation of individuals' domain knowledge issuggested, such as administering a pre-test for the teachers' domainknowledge. In addition, the current study evaluated the argumentquestion (Question 4) based on the teachers' evidence justifying theirclaims, scored on a scale of 1 to 10. Nevertheless, Ravenscroft (2007)andWu and Tsai (2007a) have demonstrated the strength of rebuttalsin argumentation. Hence, it may be meaningful for the participants toprovide in-depth insights into the relevant issues and to offer arebuttal to their opponents to support their arguments in futurestudies.

Many researchers (e.g. Hoffman, Wu, Krajcik, & Soloway, 2003;Hwang et al., 2008; Lin & Tsai, 2007; Tu et al., 2008) have asserted thevalue of analyzing learners' web search strategies. Several studies (e.g.Lin & Tsai, 2007; Tsai, 2004b; Wu & Tsai, 2005, 2007b) have furtheraddressed the important issues of exploring learners' evaluativestandards, and assessing the quality of web-based informationsources. In the future, studies may be conducted to explore howlearners (or teachers) generate keywords that they consider efficientfor searching, and how they judge the quality of information sourcesthat they believe relevant for answering the questions. In particular,studies of these topics with a specific focus on school teachers are stilllimited. To enhance teacher education and to improve schoolteachers' instructional practices in the classroom, more researchfocusing on teachers is necessary.

Finally, the results of this study indicate significant correlationsamong teachers' epistemological beliefs concerning Internet environ-ments, their web search strategies, and their search outcomes. Toprobe the predictive relationships among these factors, it is suggestedthat structural equation modeling (SEM) methods be used to confirmthe quality of the measurement and directional influences.

Acknowledgements

The funding of this study is supported by the National ScienceCouncil, Taiwan, under grant contract numbers 97-2511-S-011-003-MY3 and NSC 98-2631-S-024-001.

References

Bilal, D. (2000). Children's use of the Yahooligans! Web search engine: I. Cognitive,physical, and affective behaviors on fact-based search tasks. Journal of the AmericanSociety for Information Science, 51(7), 646−665.

Bilal, D. (2001). Children's use of the Yahooligans! Web search engine: II. Cognitive andphysical behaviors on research tasks. Journal of the American Society for InformationScience and Technology, 52(2), 118−136.

Bilal, D. (2002). Children's use of the Yahooligans!Web search engine: III. Cognitive andphysical behaviors on fully self-generated search tasks. Journal of the AmericanSociety for Information Science and Technology, 53(13), 1170−1183.

Brand-Gruwel, S., Wopereis, I., & Vermetten, Y. (2005). Information problem solving byexperts and novices — Analysis of a complex cognitive skill. Computers in HumanBehavior, 21(3), 487−508.

Bråten, I., Strømsø, H. I., & Samuelstuen, M. S. (2005). The relationship betweenInternet-specific epistemological beliefs and learning within Internet technologies.Journal of Educational Computing Research, 33(2), 141−171.

Chuang, S. C., & Tsai, C. C. (2005). Preferences toward the constructivist internet-basedlearning environments among high school students in Taiwan. Computers in HumanBehavior, 21(2), 255−272.

Chuang, S. C., Hwang, F. K., & Tsai, C. C. (2008). Students' perceptions of constructivistInternet learning environments by a physics virtual laboratory: The gap betweenideal and reality and gender differences. CyberPsychology & Behavior, 11(2),150−156.

Ford, N., Miller, D., & Moss, N. (2001). The role of individual differences in Internetsearch: An empirical study. Journal of the American Society for Information Scienceand Technology, 52(12), 1049−1066.

Ford, N., Miller, D., & Moss, N. (2002). Web search strategies and retrieval effectiveness:An empirical study. Journal of Documentation, 58(1), 30−48.

Georgina, D. A., & Olson, M. R. (2008). Integration of technology in higher education: Areview of faculty self-perceptions. The Internet and Higher Education, 11(1), 1−8.

Hofer, B. K., & Pintrich, P. R. (1997). The development of epistemological theories:Beliefs about knowledge and knowing and their relation to learning. Review ofEducational Research, 67(1), 88−140.

Hoffman, J. L., Wu, H. K., Krajcik, J. S., & Soloway, E. (2003). The nature of middle schoollearners' science content understandings with the use of on-line resources. Journalof Research in Science Teaching, 40(3), 323−346.

Hölscher, C., & Strube, G. (2000). Web search behavior of Internet experts and newbies.Computer Network, 33(1–6), 337−346.

Hwang, G. J., Tsai, P. S., Tsai, C. C., & Tseng, J. C. R. (2008). A novel approach for assistingteachers in analyzing student information searching behaviors. Computers &Education, 51(2), 926−938.

Kim, K. S. (2001). Information seeking on the Web: Effects of user and task variables.Library & Information Science Research, 23(3), 233−255.

Lee, M. H., & Tsai, C. C. (2005). Exploring high school students' and teachers' preferencestoward the constructivist Internet-based learning environments in Taiwan.Educational Studies, 31(2), 149−167.

Lin, C. C., & Tsai, C. C. (2007). A “navigation flowmap”method of representing students'searching behaviors and strategies on the web. CyberPsychology and Behavior, 10(5), 689−695.

Mason, L., Boldrin, A., & Ariasi, N. (2010). Searching the Web to learn about acontroversial topic: are students epistemically active? Instructional Science, 38(6),607−633.

Moore, W. S. (1989). The learning environment preferences: Exploring the constructvalidity of an objective measure of the Perry scheme of intellectual development.Journal of College Student Development, 30(6), 504−514.

Muse, H. E. (2003). The Web-based community college student: An examination offactors that lead to success and risk. The Internet and Higher Education, 6(3),241−261.

Ravenscroft, A. (2007). Promoting thinking and conceptual change with digitaldialogue games. Journal of Computer Assisted Learning, 23(6), 453−465.

Schommer, M., Crouse, A., & Rhodes, N. (1992). Epistemological beliefs andmathematical text comprehension: Believing it is simple does not make it so.Journal of Educational Psychology, 82(4), 435−443.

Tabatabai, D., & Shore, B. M. (2005). How experts and novices search theWeb. Library &Information Science, 27(2), 222−248.

Tsai, C. C. (2000). The effects of STS-oriented instruction on female tenth graders'cognitive structure outcomes and the role of student scientific epistemologicalbeliefs. International Journal of Science Education, 22(10), 1099−1115.

63P.-S. Tsai et al. / Internet and Higher Education 14 (2011) 54–63

Tsai, C. C. (2004a). Beyond cognitive and metacognitive tools: The use of the Internet asan “epistemological” tool for instruction. British Journal of Educational Technology,35(5), 525−536.

Tsai, C. C. (2004b). Information commitments in Web-based learning environments.Innovations in Education and Teaching International, 41(1), 105−112.

Tsai, C. C. (2005). Preferences toward Internet-based learning environments: Highschool students' perspectives for science learning. Educational Technology & Society,8(2), 203−213.

Tsai, C. C. (2007). The relationship between Internet perceptions and preferencestoward Internet-based learning environment. British Journal of EducationalTechnology, 38(1), 167−170.

Tsai, C. C. (2008). The preferences toward constructivist Internet-based learningenvironments among university students in Taiwan. Computers in Human Behavior,24(1), 16−31.

Tsai, C. C. (2009). Conceptions of learning versus conceptions of web-based learning: Thedifferences revealed by college students. Computers & Education, 53(4), 1092−1103.

Tsai, C. C., & Chuang, S. C. (2005). The correlation between epistemological beliefs andpreferences toward Internet-based learning environments. British Journal ofEducational Technology, 36(1), 97−100.

Tsai, M. J., & Tsai, C. C. (2003). Information searching strategies in web-based sciencelearning: the role of Internet self-efficacy. Innovations in Education and TeachingInternational, 40(1), 43−50.

Tseng, J. C. R., Hwang, G. J., Tsai, P. S., & Tsai, C. C. (2009). Meta-Analyzer: A web-basedlearning environment for analyzing student information searching behaviors.International Journal of Innovative Computing, Information and Control, 5(3),567−579.

Tu, Y. W., Shih, M., & Tsai, C. C. (2008). Eighth graders' web searching strategies andoutcomes: The role of task types, web experiences and epistemological beliefs.Computers & Education, 51(3), 1142−1153.

Wen, L. M. C., Tsai, C. C., Lin, H. M., & Chuang, S. C. (2004). Cognitive–metacognitive andcontent-technical aspects of constructivist Internet-based learning environments:A LISREL analysis. Computers & Education, 43(3), 237−248.

Wu, Y. T., & Tsai, C. C. (2005). Information commitments: Evaluative standards andinformation searching strategies in web-based learning environments. Journal ofComputer Assisted Learning, 21(5), 374−385.

Wu, Y. -T., & Tsai, C. -C. (2007a). High school students' informal reasoning on a socio-scientific issue: Qualitative and quantitative analyses. International Journal ofScience Education, 29(9), 1163−1187.

Wu, Y. T., & Tsai, C. C. (2007b). Developing an information commitment survey forassessing students' web information searching strategies and evaluative standardsfor web materials. Educational Technology and Society, 10(2), 120−132.

Yang, F. Y. (2005). Student views concerning evidence and the expert in reasoning asocio-scientific issue and personal epistemology. Educational Studies, 31(1),65−84.