Identifying Learning Object pedagogical features to decide instructional setting

Anura Kenkre, Gargi Banerjee, Madhuri Mavinkurve, Sahana Murthy

Inter-Disciplinary Programme in Educational Technology Indian Institute of Technology Bombay

Mumbai, India email: anura.kenkre@iitb.ac.in, gargi@it.iitb.ac.in, mavinkurvemk@gmail.com, sahanamurthy@iitb.ac.in

Abstract— Despite the abundant availability of Learning Objects as valuable teaching-learning tools, their use amongst teachers and college instructors is limited. An important reason for this is that instructors are not easily able to search and retrieve the LOs which map to their instructional goals. LO metadata only partially addresses this problem since existing metadata does not contain all the necessary pedagogy related information. In this paper we identify the pedagogical features of an LO, and use them to classify an LO for the appropriate instructional setting. We also test our classification scheme of LOs in electrical engineering domain available in the OSCAR LO repository. Lastly we suggest metadata tags based on these pedagogical features that LO repositories can use for search and retrieval of LOs for specific educational goals.

Keywords-pedagogical features, instructional setting, learning objects

I. INTRODUCTION Learning object repositories have become a valuable

resource in a variety of instructional settings. Learning objects (LOs) have a number of important features, such as the power of visualization, possibility of independent learning and opportunities for generating higher-order learning outcomes, that make them a beneficial teaching and learning tool. However LOs can only realize their full potential if they are accepted and frequently used among instructors and students. An important role instructors play is in selecting the appropriate LO for various instructional goals. One challenge to LO use is that instructors may not be aware of how to use an LO in a given instructional setting. The problem begins at the selection of the LO: on what basis does an instructor select an appropriate LO, given the instructional context and goal?

LO repositories depend on metadata to provide users with easy search and retrieval mechanisms. Metadata provide users with information about LO topic, technical features, accessibility and many other characteristics, which help them select suitable LOs for their goals. Existing metadata standards also guide instructor and student users on certain pedagogy related issues. Yet, existing metadata formats do not provide sufficient

guidance to instructors on how to choose an LO according to its intended educational goal.

Towards the goal of identifying metadata tags focusing on educational needs, we analyze the pedagogical features of an LO associated with instructional goals. We map a set of pedagogical features with an instructional setting. We generate this mapping by analyzing the goals and practice of the faculty members from engineering colleges who are users of the LOs, thereby offering a field-perspective to this problem. Inserting educational-setting related metadata tags will make it easier for teachers to integrate LOs into their teaching practice, and thus promote the use of LOs. After identifying the pedagogical features that can be used to classify an LO for an instructional setting, we validate the classification by testing it with other teaching faculty, who need to select LOs for use in various instructional settings. We suggest metadata tags based on these pedagogical features that LO repositories can use for search and retrieval of LOs for specific educational goals.

II. LITERATURE SURVEY Pedagogy is one of the key quality dimensions of LOs

along with technology and content [1, 2, 3, 4]. The benefits of using an LO as a learning tool vis-à-vis a completely traditional mode of content delivery can be established only by evaluating its pedagogy characteristics [1, 5, 6, 7, 8]. Since e-learning is more learner-driven than instructor-driven, it is all the more important to focus on these characteristics to promote learning [9].

Researchers have identified a comprehensive list of pedagogy characteristics of LOs, validated from learning theory, to ensure quality. Some of these characteristics are:

• Learning goal alignment[10] • Feedback & adaptation[10 , 3] • Motivation/Engagement [10,11] • Presentation design[10] • Learner control [3] • Potential effectiveness as a learning tool [11,12] • Cooperative/Collaborative learning [3] • Multiple representation[2] • Flexibility [2] • Learner activity[2]

2012 IEEE Fourth International Conference on Technology for Education

978-0-7695-4759-6/12 $26.00 © 2012 IEEE

DOI 10.1109/T4E.2012.8

46

These characteristics enhance the potential effectiveness of LOs as a teaching and learning tool. In fact, the teachers play a pivotal role in using these resources since they are the ones who introduce students to the LOs [13]. They are one of the major stakeholders among LO users [14, 15, 16]. They have to take crucial decisions on selecting the appropriate LO for their students [13]. While making those decisions, teachers have certain goals in mind such as;

“(1) introduce new topics and skills, (2) provide reinforcement to existing skills, (3) extend learning by providing new means for

presenting curricular material (4) illustrate concepts that are less easily explained

through traditional teaching methods (5) support new types of learning opportunities not

available in a classroom environment, (6) provide enrichment activities for gifted and highly

motivated students” [17]. This list of instructional goals is supported by

additional empirical data [14, 18] as well as by pedagogical guidelines on teaching with LOs. While LOs offer the possibility of being of use in different instructional settings, like blended classroom course, online self-study course, homework and laboratory, there are not too many experiments to test how LOs can be reused in different instructional settings [19]. Most experiments that have been conducted focus on the reusability of LOs in different instructional contexts [19,20]. The term instructional context has been taken to include differing educational levels [21], different types of courses like short courses, graduate course, undergraduate course [22] but rarely has instructional setting been considered.

It is also known that only a small number of teachers incorporate LOs in their teaching [23]. There are many possible strategies to encourage wider use of LOs by teachers. Mapping pedagogical characteristics of LOs to instructional settings can motivate teachers to integrate LOs in their teaching. Given the tight schedule of teachers, they need the specific teaching function of the LOs specified in metadata [24]. Having the LOs tagged with descriptive information like metadata, helps in easy search and retrieval of the relevant LOs [25]. Metadata tags and labels should be defined in a language that all teachers can relate to [26,27].

Many organizations like IEEE, CanCore, Dublin core, ADL have proposed their own standard metadata formats. The widely adopted IEEE LOM standard does accommodate pedagogy concerns in metadata with 11 ‘Educational’ category items [28]. But it has proved to be inadequate [29]. Many researchers have suggested

additions to pedagogical metadata fields like adaptivity [29, 30], Bloom’s taxonomy of learning objectives and assessment questions, knowledge type [31]. Researchers have built customized LO Metadata Application Profiles [31,32] to address specific educational needs.

However, even with the present set of pedagogical metadata tags, it has been observed that teachers have to spend considerable amount of time trying to determine which LO would suit their choice of instructional setting. The current study is a first attempt at identifying the requisite educational metadata for LOs from the engineering domain in Indian context, that would make it possible to suggest the different instructional settings for an LO.

III. RESEARCH QUESTION AND CONTEXT When faculty members or teachers are searching for a

relevant LO, they generally have an instructional setting in mind where they want to use the LO. Hence, it would help them if they get to know which LOs have features suited for their chosen setting. However, the currently existing pedagogical metadata tags like educational levels, interactivity types, learning styles, instructional time, and cognitive levels do not make that connection with the possible instructional settings. They are thus not sufficient to predict relevant settings for an LO. In this context, the current paper addresses the following research question: What are the pedagogical features of LOs that would help identify the best instructional settings?

To answer this question we have based our study on LOs from the OSCAR repository [33]. OSCAR is an open-source LO repository (LOR) hosting LOs in science and engineering, at the tertiary level of education, created by faculty from India’s premier educational institutes. The OSCAR LOR was created as part of Government of India’s National Mission on Education through ICT (NMEICT) with the objective of aiding the spread of higher education to different corners of India. There are currently 169 LOs in the repository from various domains like Electrical Engineering, Civil Engineering, Mechanical Engineering, Chemistry, Computer Science, Bioscience and so on. In the current study, we have restricted our scope to LOs from Electrical Engineering domain that encompass different instructional settings like lecture, laboratory, homework and self-study. In lecture and homework settings, the LOs are assumed to be used as a supplementary teaching and learning tool instead of stand-alone instructional material.

47

IV. METHODOLOGY

A. Contexual inquiry methodology & sampling

Faculty members use LOs in various instructional settings depending upon their instructional goals. Our goal was to study the procedure by which faculty members select an LO for a particular topic in their subject domain of expertise. We also wanted to get an insight into the features that they look for in an LO while they are making this choice. In order to gather the data, we have opted for Contextual Inquiry (CI), which is a qualitative data gathering and data analysis method. It is a technique for working with users to help them articulate their current practices, system practices and associated experiences [34]. In our case, it is important that we get data on LO usage from the practitioners, that is, the potential instructor-users of the LOs. Hence, CI is a appropriate technique for our study as it fosters participatory research.

To conduct Contextual Inquiry, four principles have to be taken into consideration [34]: context, partnership, interpretation, and focus. The context means that the user must be observed in his/her workplace. Generally, instructors make their choice of an appropriate LO in an academic institution, at a time suitable to them. Hence, in this study, we invited faculty members to our institution (which is similar to their work place) in order to choose the LOs. The principle of partnership ensures a difference from a traditional interview, in that traditional interviews have an interviewer who is in charge of the topics discussed and the flow of conversation. CI ensures that the inquirer and the participant are equals [35]. In this manner, both the interviewer and the interviewee can explore the problem together and try to arrive at possible solutions for identifying pedagogical features in an LO for a given instructional setting. The interpretation principle refers to the interpretation of the results which will be discussed in detail in the later sections. Lastly, the focus principle suggests that the interview should be based on a clearly defined set of concerns, rather than on a list of specific questions (as in a survey) [35]. The focus of this CI was identifying pedagogical features for an instructional setting. Our sample consisted of five faculty members from engineering colleges in Mumbai. We opted for purposive sampling since these instructors had an insight about how to use the LOs and hence could help us in identifying the pedagogical features present in the chosen LO. These faculty members were domain experts in the field of electrical engineering and had more than 10 years of experience each. They were also familiar with concepts of educational technology.

B. Conducting the interview Two researchers conducted the interview. Either

researcher would ask the questions while both kept interview notes for future reference. Apart from this, audio recording was done for all the interviews after obtaining permission from the faculty members. This too was used while analyzing the data.

We asked faculty members to select a topic pertaining to their syllabus and then assigned them a suitable LO. Once the faculty member had viewed the LO, we began the interview by asking them where they would use this LO. After they had specified this, we asked further questions to understand specific reasons behind this choice. We then asked why they would not use this LO for any other instructional setting. For example: if a faculty member said that she will use the LO in a lecture, we asked her to explain why she considered the lecture as a suitable setting for using the LO. Later, we asked her why she would not use this same LO for any other setting, such as for lab/self study/home-work. We carefully noted what features faculty members mention are not suitable for a particular setting. For example, one faculty member said that she would not the LO in a lecture setting as it has too much interactivity. This would distract her students and hence the LO would be more suitable for labs. In this manner, we recorded all their responses to the features which would be suitable for the instructional settings of lectures, labs, self study and home work.

C. Data Analysis:Creation of Classification Scheme.

Once the interviews were conducted, we transcribed the responses from faculty members. While doing so, we have coded the data in two phases. We have adopted the open coding methodology to assign initial codes to the responses. In open coding, the researcher codes the data, creating new codes and categories where necessary, and integrating codes where relevant [36].

Two codes were assigned, one related to instructional setting and the other related to the information the responses are trying to convey. The codes for the instructional setting could be easily assigned since the CI established a partnership between the interviewer and the interviewee. The faculty members would mention their preference of a certain feature for particular setting while responding to a question. For example, “for LOs to be used in labs, students should be able to manipulate data/variables and observe the effect”, “In self study, feedback is a must since students are learning on their own”. The codes related to the information were short phrases conveying the key idea in the faculty members’ responses. We have multiple codes for these responses;

48

each pertaining to the data contained the response. Given below are few examples of codes.

Faculty members’ response Code related

to setting Code related to data.

Students should be able to design the circuit or experiment on the LO interface and implement it in reality if it works on the LO.

Lab Circuit Designing

In class you cannot draw all the examples on the board

Lecture Inability to draw multiple examples.

In LOs assigned for self study, interactivity has to be there since the student will be making sense on his own

Self Study Some interactivity needed

Once the interview notes were transcribed, an affinity

diagram was created. An affinity diagram is a categorization or grouping of questions that are similar or that seem to go together [35]. We did the affinity following the methodology suggested by Beyer and Holtzblatt (1998) [37]. All the responses that appeared to be related to each other were grouped into one category. Examples are shown in Table I. In this manner all the responses from the users were coded into various categories based.

TABLE I. EXAMPLE OF AFFINITY DIAGRAM

Low Interactivity

Faculty Members’ Response

Code related to setting

Code related to Data

Interactivity is not an issue in class usage. Only play pause is enough.

Lecture Play/Pause

LO used in class is just part of the instruction not the instruction itself and so interactivity could distract the students.

Lecture Interactivity causes distraction

LOs used in lectures should have an option to adjust the speed and the faculty member should be able to pause it in between so that they can stop there and explain further.

Lecture Pause to explain

A movie or an animation with minimum (play/pause) interactivity will be better for lectures.

Lecture Play/Pause

The categories determined were used to identify a

pattern between the instructional setting and the categories. We found that all the responses in a given category belong to the same setting. In this manner, we

were able to map the pedagogical features to the appropriate instructional setting. This classification is presented in Table II.

After the classification scheme was established, we had a list of features needed for the instructional settings of lab/lecture/HW/self study respectively. We adopted the axial coding method in grounded theory in order to classify these features into variables with values pertaining to each setting. The essence of axial coding is to find the interconnectedness of categories [36], discussed in detail in the next section.

V. RESULTS

A. The Classification Scheme On creating the affinity diagram, we have arrived at a

list of pedagogical features needed for each instructional setting of labs/lectures/homework (HW) and self study respectively. This is presented below in Table II:

TABLE II. PEDAGOGICAL FEATURES MAPPED TO INSTRUCTIONAL

SETTING

LABS LECTURES Variable manipulation Low degree of variable

manipulation Measurement and testing Low interactivity High interactivity Fixed content sequencing Data manipulation Stepwise explanation Hands-on activity or component building

Detailing

Options for designing and implementation

Application of concept

More number of examples Speed adjustment – play pause Single video

Single example

SELFSTUDY HW Comparative study between topics

Medium degree of variable manipulation.

Flexibility Medium interactivity Medium interactivity Medium number of examples Medium degree of variable manipulation

Feedback from assessment questions

Feedback from assessment questions

Degree of complexity of questions.

Stepwise explanation More no of assessment questions - more practice

Self-paced learning Learning goal alignment with assessment

LO should be self-contained Variety in type of assessment Presence of glossary More number Of examples

We synthesized the pedagogical features into

variables which take different values across each setting. For this, we applied axial coding. We started with a category from one instructional setting (from Table II) and looked for a variation of that category

49

with respect to the other instructional settings. For example, we observed that faculty members prefer the number of examples given in an LO to be high for self-study and lab setting whereas it is found to be medium for home work setting and low in lecture settings. Hence, we generated a variable of ‘Number of examples’ having levels of ‘High’, ‘Medium’, ‘Low’ respectively for the four instructional settings. In this manner we arrived at a classification of the categories from Table II into variables and their values for each instructional setting (Table III).

TABLE III. SYNTHESIS OF PEDAGOGICAL FEATURES INTO

VARIABLES AND LEVELS

Variable Level Instructional setting

Number of variables

manipulated

Nil Lectures

Low Homework

High Self Study & Labs

Interactivity Level

Low Lectures

Medium Self Study & homework High Labs

Explanation Steps High lectures & Self Study Low Labs & homework

Control of Sequencing

LO Creator Lecture, Self Study, Homework

User Defined Labs

Treatment of Topic

comprehensive lectures, self study Glossary homework & labs

Real life applications

Necessary lectures, self study not necessary homework & labs

measurements & opportunity for

error

Yes Labs

No Lecture, Self Study, Homework

Navigation Linear Lecture

Branched labs, self study, homework

Number of examples

Low Lecture Medium Labs & homework

High self study

Number of Assessment questions

Nil Lecture

Medium labs, self study

High Homework

Variation in assessment type

(blooms, difficulty level)

Nil Lecture

medium labs, self study

High Homework

B. Classification of OSCAR LOs

The OSCAR repository contains 27 LOs in various topics in Electrical Engineering such as: signals and systems, error correcting codes and image processing. Using the

features in Table II and variable levels in Table III, we manually classified the OSCAR LOs according to the various instructional settings.The method followed for classifying the LOs was, if 80 % or above features of an LO matched with those identified for a specific setting then, we propose that setting as the most suitable. On an average, 30% of the features in such LOs were common to settings other than the one proposed. However, for a few LOs, there was 70% overlap between features for two settings. In this case we reported both the settings as suitable. Examples of these cases are shown in Table IV.

TABLE IV. CLASSIFICATION PROCESS OF LOS

LO Lecture Lab home

work self

study Preferred

Setting LO1 80% 20% 40% 40% Lecture

LO2 40% 60% 70% 40% home work/lab

Figure 1 given below shows the classification of LOs in Electrical Engineering (N=27) across various instructional settings.

Fig 1: Classification result of LOs

On applying the classification scheme we found that majority of the LOs currently contained in the repository can be used in the classroom. Most of the LOs were uniquely classified into a single instructional setting while few others could be used in any two settings.

B. Testing of classification scheme Our goal was to test whether a faculty member’s

choice of instructional setting for an LO is consistent with the setting suggested by our scheme. Our sample consisted of 8 faculty members from various Engineering Colleges across Mumbai. These faculty members were allotted LOs from their subject and according to the level that they teach at (under-graduate or post-graduate level). The faculty answered survey questions related to preferred choices for instructional settings for their allotted LOs.

50

Each faculty member evaluated 4 LOs and for each LO we gathered responses from 2 to 3 faculty members. We allotted weights to the choices by assigning highest weight for the first choice and descending weights for remaining choices. Table V shows the comparison of the setting suggested by our classification scheme and the faculty member’s choice of setting.

A total of 12 LOs from the Electrical Engineering domain were used for the testing purpose. For 11 out of the 12 LOs, there was an agreement between the setting proposed by our classification scheme and the setting preferred by the faculty members. For one LO out of the 11, we had suggested two settings while the faculty member’s choice reflected only one of the two. For 4 out of the 11 LOs, our classification scheme suggested a single setting while faculty chose two settings as most suitable. These two settings chosen by faculty members included the setting suggested by the classification scheme as one of them. For only one out of 12 LOs, there was a disagreement between the faculty member’s choice and the setting suggested by our classification scheme.

TABLE V. COMPARATIVE STUDY OF INSTRUCTIONAL SETTING

LO Faculty member’s choice of setting

Setting suggested by our classification

scheme LO1 Lecture Lecture LO2 Lecture Self study/ Lecture LO3. Lecture Lecture LO4 Lecture Lecture LO5 Lecture Lecture LO6 Lab Lab LO7 Lab Lab LO8 Lab Lecture LO9 Lecture /lab Lecture LO10 Lecture lab Lecture LO11 Lecture /lab Lecture

LO12 Lecture/lab Lab

VI. DISCUSSION AND SUMMARY

In this paper we have tried to identify and test the process for selecting a suitable LO for a given instructional setting. Initially, we identified a number of pedagogical features for each setting that can be used to classify an LO into a preferred instructional setting. We later synthesized these features (a total of 34 features - 7 for labs, 9 for lectures, 9 for self study and 9 for homework) into a set of 11 pedagogical variables which take different values for each setting.

Later, we combined some of the pedagogical variables from Table III, into a new category titled ‘Essential Features’ while retaining the other variables as they are. These pedagogical variables, if converted into metadata tags, would aid faculty users in selecting an LO best suited for their instructional goals. We therefore propose the following metadata tags:

1. Variable Manipulation 2. Interactivity Level 3. Sequencing 4. Treatment 5. Number of examples 6. Navigation 7. Essential features

a. Measurement & instrumentation b. Real Life examples c. Variation in assessment d. Explanation Steps

Out of the metadata tags suggested above, the tag

for ‘Interactivity Level’ already exists in the IEEE LO Metadata standards, with levels of Active, Mixed and Expositive [28]. Hence, excluding that, we suggest six new metadata tags for pedagogical features pertaining to an instructional setting.

In terms of validating the classifications scheme for the choice of LOs for a particular instructional setting, we found that, for half the LOs that were tested, there was 100% agreement between the setting proposed and the setting chosen by the faculty. For the other half, there was a partial match between the setting proposed by our classification scheme and the faculty’s choice, while there was a complete mismatch for only one LO. The probable reasons behind this result could be that the faculty members are not adequately trained to use the LOs. This fact repeatedly came out during the interviews and the survey. Hence, their choice of the setting where the LO could be best used is based on their heuristics and experience with other kinds of instructional material. One way to address the lack of familiarity with LOs and their instructional use could be to conduct faculty training programmes where faculty users would be given guidance about the possible uses of LOs across multiple instructional settings.

Future steps to be undertaken include testing the classification scheme across other domains in the OSCAR repository. Metadata tags need to be implemented into the repository. In order to promote LO use among faculty members, training programmes as described above need to be conducted.

51

ACKNOWLEDGEMENT The authors would to thank the following

organizations and people for their contribution, help and support:

a. Project OSCAR Team: Ms. Malati Baru, Prof. Sridhar Iyer, Mr. Sameer S., Ms. Vijayalakshmi C., ID writers, Designers and Animators b. NMEICT, Ministry of Human Resources and Development, Govt. of India c. IIT Bombay students and faculty d. Faculty of D.J.Sanghvi college of Engg. & Thakur college of Engg., Mumbai University

REFERENCES [1] S. Lau S. and P.C. Woods, “An Empirical Study on Students`

Acceptance of Learning Objects”, Journal of applied sciences, vol. 8 (22), pp. 4079-4087, 2008.

[2] S. Hadjerrouit, “A conceptual framework for using and evaluating web-based learning resources in school education”,Journal of Information Technology Education, vol.9, pp.53-79, 2010.

[3] P. Nokelainen, “An empirical assessment of pedagogical usability criteria for digital learning material with elementary school students”, Educational Technology & Society, vol. 9(2), pp.178-197, 2006.

[4] V. Kumar, J. Nesbit and K.Han, ‘Rating Learning Object Quality with Distributed Bayesian Belief Networks: The Why and the How’, Proc. Fifth IEEE International Conference on Advanced Learning Technologies, pp. 685-687, 2005.

[5] T.Govindasamy, “Successful implementation of e-learning: Pedagogical considerations”, The Internet and Higher Education,vol. 4, pp. 287-299,2002.

[6] A.A.Hamid, “E-Learning: Is it the “e” or the learning that matters?”, The Internet and Higher Education,vol.4,pp. 311-316,2002.

[7] R.G.Saade,”Web-based educational information system for enhanced learning”, EISEL: Student assessment, Journal of Information Technology Education, vol.2, pp.267-278, 2003.

[8] D.M. Watson, “Pedagogy before technology. Re-thinking the relationship between ICT and Teaching”, Education and Information Technologies, vol. 6(4), pp. 251-266, 2001.

[9] B. Bannan-Ritland, N., Dabbagh, and K. Murphy, ”Learning object systems as constructivist learning environments: Related assumptions, theories and applications”, in The instructional use of learning objects, Ed. D.A. Wiley,Association for Educational Communications and Technology,2002.

[10] T.L.Leacock andJ.C. Nesbit, “A framework for the quality of multimedia resources”,Educational Technology & Society,vol. 10(2), pp. 44-59,2007.

[11] R.Kay and L. Knaack, “Assessing Learning, Quality, and Engagement in Learning Objects: The Learning Object Evaluation Scale for Students (LOES-S),” Educational Technology Research and Development, vol. 57(2), pp. 147-168, 2008.

[12] V. Schoner, D. Buzza,K. Harrigan and K Strampel, “Learning objects in use: ‘Lite’ assessment for field studies”, Journal of Online Learning and Teaching, vol.1(1), pp.1–18,2005.

[13] T.A. Bratina, D.Hayes and S.L. Blumsack, “Preparing teachers to use learning objects”, Faculty and Staff Development, November/December, 2002.

[14] P.Freebody, S. Muspratt and D. McRae,” Evaluating the le@ring federation's online curriculum content initiative”, final report, 2007.

[15] E.Busetti, P.Forcheri, M.G.Ierardi, M.T. Molfino, “Repositories of learning objects as learning environments for teachers", IEEE International Conference on Advanced Learning Technologies, pp. 450- 454, 30 Aug.-1 Sept. 2004.

[16] E.Busetti, G.Dettori, P. Forcheri, M.G. Ierardi, “Preparing Teachers to the use of LOs: an analysis of conceptions”,

Proceedings of ICALT, IEEE Comp. Soc. Press, Los Alamitos, pp. 669–673, 2005.

[17] Y. Akpinar and H. Simshek, “Pre-service Teachers ’ Learning Object Development : A Case Study in K-12 Setting”, Interdisciplinary Journal of E-learning and Learning objects,vol.3,pp. 197- 217,2007.

[18] S.R. Cramer, “Update Your Classroom with Learning Objects and Twenty-First-Century Skills”, The Clearing House, vol.80 (3), pp.126,2007.

[19] S. Conceicao, C. Olgren, P. Ploetz,“Reusing learning objects in three settings: Implications for online instruction”, Instructional technology and distance learning, April 2006.

[20] F.Krauss and M.Ally, “A study of the design and evaluation of a learning object and implications for content development”, Interdisciplinary Journal of Knowledge and Learning Objects, vol.1, pp. 1-22, 2005.

[21] D. Wiley, S. Wayers, D. Dawson, B.Lambert,M.Barclay and D.Wade,”Overcoming the limitations of learning objects”, Journal of Educational Multimedia and Hypermedia, 13(4): 507–521, 2004.

[22] S. Grunwald and K.R. Reddy, “Concept Guide on Reusable Learning Objects with Application to Soil, Water and Environmental Sciences”, Dec. 2007 ,Available at: ecolearnit.ifas.ufl.edu/documentation/concept_guide.pdf, Date accessed: March1,2012.

[23] T. L. Naps , G. Rößling , V. Almstrum , W. Dann , R. Fleischer , C. Hundhausen , A. Korhonen , L. Malmi,M. McNally,S. Rodger,J. Á.Velázquez-Iturbide, "Exploring the role of visualization and engagement in computer science education",ACM SIGCSE Bulletin, vol.35(2), pp. 131-152 2003 [doi>10.1145/782941.782998]

[24] M.A.Fitzgerald and J.McClendon,” The Gateway to Educational Materials: An Evaluation Study,2002.

[25] E.D.Wagner, “The new frontier of learning object design”, The E-Learning Developers Journal, pp.1-7, 2002.

[26] M. Tate and D.Hoshe, “A model for the effective management of re-usable learning objects [RLOs]”, Interdisciplinary Journal of E-Learning and Learning Objects, vol. 5, pp.51-72.2009.

[27] J. Qin, and C. J. Godby, “Incorporating Educational Vocabulary in Learning Object Metadata Schemas”, in Research and Advanced Technology for Digital Libraries, LNCS 2769/2004, Berlin: Springer, pp. 52-57, 2004.

[28] IEEE Learning Technology Standards Committee,” Learning object Metadata”, IEEE 20021484.12.1, 2002.

[29] Y.Yahya, and M. Yusoff, “Towards A Comprehensive Learning Object Metadata:Incorporation of Context to Stipulate Meaningful Learning and Enhance Learning Object Reusability”, Interdisciplinary Journal of E-Learning and Learning Objects, vol. 4, pp. 13-48,2008.

[30] O.Conlan, C. Hockmeyer, P.Lefrere, V. Wade, D. Alber, “Extending educational metadata schemas to describe adaptive learning resources”, Proc.of the 12th ACM conference on hypertext and hypermedia, ACM Press, New York, USA,2001,pp. 161–162

[31] D.G.Sampson, P. Zervas,S. Sotiriou,"Learning Object Repositories for Science Education: The Case of the Open Science Resources Repository," Advanced Learning Technologies (ICALT), 11th IEEE International Conference,pp.358-359,6-8 July 2011 doi: 10.1109/ICALT.2011.111 URL: http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=5992343&isnumber=59922398], 2011 .

[32] N. Friesen,,”Building Educational Metadata Application Profiles”, Proceedings of International Conference on Dublin Core and Metadata for e-Communities, pp. 63-69, Firenze University Press, Italy,2002.

[33] OSCAR LO repository http://oscar.iitb.ac.in [34] K.Holtzblatt and S. Jones, “Contextual Inquiry: A Participatory

Technique for System Design”, In D. Schuler and A. Namioka (eds.): Participatory Design: Principles and Practices. N.J.: Lawrence Erlbaum, pp. 177–210, 1993.

52

[35] M.E.Raven and A. Flanders, “ Using contextual inquiry to learn about your audiences”, Journal of Computer Documentation, vol. 20(1), pp. 1-13, 1996.

[36] L.Cohen, L. Manion and K. Morrison,” Research merhods in education”,pp. 493,2010.

[37] H.Beyer and K.Holtzblatt, “ Contextual Design: Defining Customer Centered Systems”, San Francisco, Morgan Kaufmann Publishers,1998.

53