UM:UO.551 :Labium - ERIC · Correlations between Predictor Variables and Final Exam Score: Course Approach 26 Table 4.12 Comparison of Special Needs Students' Academic Progress for

UM:UO.551 :Labium

ED 158 699 HE 010 451

TITLE Promoting Student Learning in'College by Adapting toIndividual Differences in Educational CognitiveStyle. Final Evaluation Report.

INSTITUTION American Coll. Testing Program, Iowa City, Iowa.SPONS AGENCY Fund for the Improvement of Postsecondary Education

(DHEW) , Washington, D.C.PUB DATE Dec 77NOTE . 68p.AVAILABLE FROM The American College Testing Program, Iowa City,

Iowa

EDRS PRICE MF-$0.83 HC-$3.50 Plus Postage.DESCRIPTORS Cognitive Processes; *Cognitive Style; *College

Instruction; Evaluation Methods; Higher Education;*Individual Differences; *Individualized Instruction;Intermode Differences; Learning Characteristics;Learning Modalities; Measurement Instruments; Models;Program Evaluation; *Student Improvement; Tables(Data)

IDENTIFIERS Macomb County Community College MI; Michigan StateUniversity; Oakland Community College MI

ABSTRACTTwo postsecondary institutions in Michigan, Macomb

County Community Colleget'and Michigan State University, participatedin a twc-year program that attempted to implement and evaluate a setof procedures developed for measuring and mapping each student'seducational cognitive style and using the resulting maps toindividualize educational programs to accommodate students' styles.Based on a program developed by Joseph E. Hill of Oakland CommunityCollege in Michigan, the project specified four objectives: (1) todevelop and test procedures for the measurement of the 32 dimensionsof Hill's model of educational cognitive style; (2) to train facultyand staff of the participating institutions to.understand the model;(3) to assist these trained faculty members in developing and'carrying out plans for implementing cognitive style mappingprocedures in making educational decisions in their individualcourses; and (11) to plan and carry out systematic evaluations of theeffect of the mapping procedures and data on educational outcomes -ineach setting. The project results did not establish the existence ofsignificant relationships between the use of cognitive style measuresand data on educational outcomes. Study results are presented innumerous tables and an appendix contains the reports of theevaluation advisory panel. (JMD)

***********************************************************************Reproductions supplied by EDRS are the best that can be made

from the original document.***********************************************************************

PROMOTING STUDENT LEARNINGIN COLLEGE BY ADAPTING TO

INDIVIDUAL DIFFERENCES INEDUCATIONAL COGNITIVE STYLE

-PE{-1`,11`,.,..SII11

1111S

,1,1%.11_1:1AL

BEE1'.161,1'11-11

111?

11-

)1IIIC,1

silW1;.1 It111C1T.^ 11)

t, V10 E1111(

01 .1:,110` ll11

INF

11,J.11COE

EC 1CSI I T.

U S OEPARTAENT OF HEALTH,EDUCATION L WELFARENATIONAL INSTITUTE OF

EDUCATION

THIS DOCUMENT HAS BEEN REPRO-DUCED E %AC 11. Y AS RECEIVED FROM

THE PERSON ORORGANI1A LION ORIGIN

A TiNC, IT POINTS OF VIEW OR OPINIONS

STA TED DO NOT NECESSARILY REPRESENT DI' CIAt NATIONAL INSTITUTE OF

EDUCATION POSITION OR POI. ICY

O

IrA

Final Evaluation ReportDecember 1977

Cia.athe American College Testing Program awe

[The research reported herein was supported by a grantfrom the Fund for the Improvement of PostsecondaryEducation, Department of Health, Education, and Welfare.

-01978 by The American College Testing Program. All rights reserved. Printed in the United States of America

TABLE OF CONTENTS

Acknowledgements

Project Staff vi

Chapter 1: Introduction 1

Overview of the Problem 1

Revievi% of Project Objectives 1

Project Limitations 2

Review of Project ACtivities 4

Purpose of the Report 4

Chapter 2: Participant Training 7

Description of Trainees 7Training Objectives 7

Training Strategies 8

Evaluation of Training Procedures 9

Conclusions about the Training Procedures 9

Chapter 3: ' Development of Cognitive Style Measures :11

Objectives of the Instrument Development Phase 11

Instrument Development Procedures 11

Outcomes of Instrument Development 12Summary of Instrument Development 18

Chapter 4: The Relationship between the Use of Educational CognitiveStyle Mapping and Educational Outcomes 19

Implementation Goals and Strategies 19

Alternative Perspectives on the Interpretation of the Implementation Results 19Implementation in a Science Course 20

Implementation in a Humanities Course 23

Implementation in a Special Needs Setting 25Implementation in a Physical Education Setting 28

Research in an Educational Psychology Setting 35Implementation in .a Graduate Education Course 36Summary of Teacher Debriefing Interviews 41

Summary of Student Attitude Survey 44

Conclusions Based on Impact Research 51

Chapter 5: Summary and Recommendations 53

Training 53

Developing Measures of Style 53Assessing Impact 54

APPENDIX: Evaluation Advisory Panel Reports 57

Year One Report 57End of Project Report 59

INDEX OF TABLES AND FIGURES

Table 1.1 Dimensions of the Cognitive Style Map 2

Table 2.1 Methods Used for Achieving Training Objectives 8

Table 3.1 Summary of Cognitive Style MeasOrement Procedures 13

Table 3.2 Relationships between Style Elements and Inventory of Competencies 15

Table 3.3 Intercorrelations among Tests and Subscales Contained in the Project CognitiveStyle Test and Inventory Booklet 16

Table 3.4 Results of Coadministration of OCC and Project Cognitive Style Batteries 17

Table 3.5 Correlations between Deciles on Inventory of Competency Subscales and Decile Valuefor Selected Qualitative Symbols as Measured by the OCC Cognitive Style Battery 17

Table 4.1 Summary of Implementations 20

Table 4.2 Final Average for Students in the Science, MCCC Class 21

Tattle 4.3 Summary Table Analysis of Covariance 21*

Table 4.4 Comparisons of Number of Student Dropouts in Prescriptive and Traditional Classesand Who Did or Did Not Receive Map Interpretation 22

Table 4.5 Probable Modes of Understanding of Hurrianities Instructional Methods 24Table 4.6 Stepwise Regression Equations Obtained by Using Appropriate Map Deciles and

Respective Match Scores as Predictors 25

Table 4.7 Stepwise Regression Equations Obtained by Using Match Scores on Appropriate_Map Elements as Predictors 25

Table 4.8 Correlations between Predictor Variables and Final Exam Score: Lecture Approach 26

Table 4.9 Correlations between Predictor Variables and Final Exam Score: Group Approach 26

Table 4.10 COrrelations between Predictor Variables and Final Exam Score: Media Approach 26

Table 4.11 Correlations between Predictor Variables and Final Exam Score: Course Approach 26

Table 4.12 Comparison of Special Needs Students' Academic Progress for 1975 and 1976 Classes 27

Table 4.13 Comparison of Map Results for Original Group of Students and a Subsetof Those Students Who Were Available for Retesting in December 1976 28

Table 4.14 Growth in Theoretical Map Elements and Reading Level between September al dDecember 1976 Testing 28

Table 4.15 Identification of Criterion Scores of Subjects in Successful.and Unsuccessful Groups 29

Table 4.16 Unsuccessful and Successful Collective Maps Determined by the70 Percent-of-the-Group ;vtethod 30

Table 4.17 Unsuccessful and Successful Collective Maps Determined by the Averages MethodTable 4.18 Analysis of Variance Results-Successful and Unsuccessful Groups 31

Table 4.19 Bonferonni Follow-up-Qual,tative Codes 32

Table 4.20 Comparison of Results from 70 Percent -of- the - Group, Averages, andANOVA Techniques 32

Table 4.21 Subscales from Education Psychology Attitude Instrument 36

Table 4.22 Results of Stepwise Regression Procedures to Predict Behavioral Observation andAttitude Measures Using the 16 Raw Scores Concatenated with 16 MatchScores as Predictor Variables. 37

Table 4.23 Results of Stepwise Regression Procedures to Predict Behavioral Observation andAttitude Measures Using 16 Match Scores as Predictor Variables 37

Table 4.24 Correlations between Predictor Variables and the Behavioral Observation Criterion 38

Table 4.25 Correlations between Predictor Variables and the Evaluation of TA Criterion 38

Table 4.26 Correlations between Predictor Variables and the Learning Outcomes Criterion 39

Table 4.27 Correlations between Predictor Variables and the Group Participation Criterion 39

Table 4.28 Referent Maps for Individual /independent and Peer/Discussion Groups 40

Table 4.29 Outcome Measures for Individual/Independent and Peer/Discussion Groups 41

Table 4.30 Analysis of Variance Results for Map Compcinents of Individual/Independent andPeer/Discussion Groups 41

Table 4.31 Sources of Student Attitude Data 44'

Table 4.32 Results of Student Attitude Survey 46

Figure 4.1 Map Profiles, Physical Education-MCCC 33

Figure 4.2 Map Profiles, Physical Education-MCCC 34

Figure 4.3 Map Profiles, Graduate Curriculum-P.45U 42

ACKNOWLEDGMENTS

The American College Testing Program project staff wish toacknowledge the special contribution made to this projectby the late Dr. Joseph E, Hill, President of Oakland County.Community College and originator of the concept ofeducational cognitive style. As an educator, Dr. Hill wassincerely dedicated to the improvement of educationalopportunities of his students. As a scholar, he was deter-mined to advance the frontiers of educational psychology.Both of the qualities were reflected in his enthusiasticparticipation in and invaluable contribution to this project.

In addition, we wish to express our sincere thanks for thepersonal commitment of those who participated in a yearof training and a second year of experimental imple-mentation of instructional procedures so that this studycould be completed, The participants from MacombCounty Community College were:

Sylvia Bartolucci, Science DepartmentGlen Lahti, Humanities DepartmentJoyce Monte, Student ServicesJohn Orris, Physical Education Department

Those participating from Michigan State University were:

Richard DeSpelder, College of EducationRalph Kron, University Counseling CenterPeggy Reithmiller, College of EducationCecil Williams, University Counseling Center,Irvin Lehmann, Office of Learning and Evaluation Services

We also thank the students enrolled in the courses inwhich the research was conducted. Their patience andwillingness to contribute to the effort were greatlyappreciated.

Significant contributions to the project were also made bymembers of the Oakland Community College faculty andstaff. Special thanks are due to James Orr, Derek Nunny,and Betty Setz.

Finally, our sincere appreciation is extended to themembers of the project's National Advisory Panel: RobertC. Birney, John t. Roueche, Richard E. Snow, and WilliamF. Taylor.

U

0

PROJECT STAFF

Project Codirectors

Dr. Richard J. StigginsDirectorACT Test Development Unit

Dr. Richard L. FergusonVice PresidentACT Research and Development Division

Special Consultant

Dr. Joseph E. HillPresidentOakland County Community College

On-site Project Coordinator

Ms. Linda HendersonResearch AssociateACT Test Development Unit

Coordinator of Instrument Development

Ms. Cynthia B. SchmeiserAssistant DirectorACT Test Development Unit

Coordinator of Data Analysis

Dr. Barbara PlakeResearch AssociateACT lest Development Unit

Evaluation Advisory Panel

Dr. Robert BirneyVice President for Academic AffairsHampshire College

Dr. John E. RouecheProfessor and DirectorCommunity College Leadership ProgramThe University of Texas at Austin

Dr. Richard E. SnowProfessor of EducationStanford University

Dr. William F. TaylorDean of StudentPolk Junior College

L

CHAPTER 1

INTRODUCTION

Overview of the Problem

In the past decade, there has been a marked change in thecharacteristics of the postsecondary student population.An unprecedented proportion of the total population ofhigh school graduates has sought some form of post-secondary education. Individuals traditionally deniedaccess to higher education, especially those from lowsocioeconomic backgrounds and members of minoritygroups, have enrolled in ever-increasing numbers. Thisincreased diversity of the postsecondary'student popu-lation has been accelerated by the growing number ofadults who are returning to school for the additionaltraining or retraining required in a dynamically changingsociety. As a result of all oflheie past and present changes,there are substantial variations in the background experi-ences, personality characteristics, and entering academicand occupational competencies of students enrolling inpostsecondary institutions.

In contrast to this great change in student-body character-istics, there have been few significant changes in instruc-tional procedures at most postsecondary institutions.Education is often viewed as a process in which the instruc-tor is the dispenser of knowledge and the student thereceptor. Only modest attempts have been made toprovide remedial or enrichment experiences for studentswho are ill equipped to enter the mainstream of the institu-tions' programs. Little emphasis has been placed on identi-fying educationally relevant individual differences amongstudents and even less Kas been placed on adapting thelearning environment to a; commodate those differences.

Attrition among postsecondary studentsa problem that isespecially apparent in community collegesmay be aresult of the institutions' failure to respond to the increas-ingly diverse needs of a changing student population.Since a complex technological society demands advancedtraining for an increasing number of citizens, students'different iieeds should be analyzed and new teaching tech-niques designed to provide a reasonable probability of suc-cessful accomplishment of educational objectives.

Although some attempts have been made to alleviate theproblems arising from the diversity in the capabilities ofentering students, these attempts have typically beenstopgap in na,Jre. Special recruitment programs andspecial tutorial and support services, although helpful,

have only made superficial attacks on symptoms ratherthan attempts to address causal factors. Moreover,attempts to provide tutorial assistance in sheltered learn-ing environments for students with academic deficiencieshave sometimes been accompanied by a lowering ofexpectations of performance.

A potential solution to this problem of accommodatingindividual differences in postsecondary instruction hasbeen developed by Dr. Joseph E. Hill, President,'OaklandCommunity College, Bloomfield Hills, Michigan, and hisassociates. Hill has suggested an approach to the measure-ment of student characteristics which is a reflection of themanner in which individual learners derive meaning fromtheir environment. Using measures of achievement, atti-tude, and interests, Hill maps each student's educationalcognitive style. He and others working with him assert thatthey can match styles of each student with diverse modesof instruction, thus enhancing the educational process.Indeed, over the past several years, they have imple-mented such procedures at several institutions, includingOakland Community College.

Review of Project Objectives .

The overall purpose of this project was to implement Hill'seducational cognitive style mapping procedures in institu-tions other than Oakland Community College so that theeffectiveness of such procedureS could be systematicallyand objectively evaluated in terms of educational out-comes. The specific objectives of the project were:

1. to devise, develop, field test, and refine procedures forthe measurement of the various dimensions of Hill's

()model of educational cognitive style;

2. to train faculty and staff in postsecondary educationalsettings to understand the conceptual framework ofeducational cognitive style;

3. to assist these trained (Acuity members in developingand carrying out plans for implementing cognitive stylemapping procedures in making educational decisions intheir individual settings; and

4. to plan and carry out systematic and objective evalua-tions of the effect of the cognitive style mapping proce-dures and data on educational outcomes in each imple-mentation setting.

Project limitations

There are many diverse conceptualizations of cognitivestyle that have been developed over the years. This studyconcentrated total'y on educational cognitive style as con-ceptualized by Hill. This conceptualization includes a total.

of 32 style dimensions in three categories: (1) Symbols andTheir Meanings, (2) Cultural Determinants, and (3)ties of Inference. Each of these style dimensions orelements is listed in the brief overview of educationalcognitive style elements provided in Table 1.1.

Definitions of cognitive style, such as those espoused byWitkin; Kagan, Siegel; and Moss; and others, were notaddressed in this project. Therefore; the term cognitivestyle in this document refers only to the Hill model. Inaddition, all generalilations and conclusions made applyonly to the Hill model as implemented in this study and notto other conceptualizations of cognitive style.

TABLE 1.1

Dimensions of the Cognitive Style Map

I. Symbols and Their Meanings

Two types of symbols, theoretical and qualitative, are created and used by individuals to acquire knowledge and derivemeaning from their environments and personal experiences. Theoretical symbols present to the awareness of the individualsomething different from that which the symbols are. Words and numbers are examples of theoretical symbols. Qualitativesymbols present and then represent to the individual that which the symbol is. Feelings,commitments, and values are examplesof the meanings conveyed by qualitative symbols. Theoretical symbols include:

T(VL)Theoretical Visual Linguisticsability to find meaning in written words .

T(AL)Theoretical Auditory Linguisticsability to acquire meaning through hearing spoken words

T(VQ)Theoretical Visual Quantitativeability to acquire meaning in terms of numerical symbols, relationships, andmeasurements that are .vritten

T(AQ)Theoretical Auditory Quantitativeability to acquire meaning in terms of numerical symbols, relationships, andmeasurements that are spoken

The four,quilitative symbols associated with sensory stimuli are:

Q(A)Qualitative Auditoryability to perceive meaning through the sense of hearing

Q(0)Qualitative Olfactoryability to perceive meaning through the sense of smell

Q(T)--Qualitative Tactileability to perceive meaning through the sense of touch, temperature, and pain

Q(V)Qualitative Visualability to perceive meaning through sight

The qualitative symbols that are programmatic in nature are:

Q(PF)Qualitative Proprioceptive (Fine)ability to synthesize a number of symbolic mediations into a performancedemanding monitoring of a complex task involving small, or fine, musculature (e.g., playing a musical instrument, typewriting)

Q(PG)nualitative Proprioceptive (Gross)ability to synthesize a number of symbolic mediations into a performancedemanding monitoring of a complex task involving large, or gross, musculature (e.g., throwing a baseball, skiing)

Q(PKF)Qualitative Proprioceptive Kinematics (Fine)ability to synthesize a number of symbolic mediations into . perfor-mance demanding the use of fine musculature while mor itoring a complex physical activity involving motion

Q(PKG)-- Qualitative Proprioceptive Kinematics (Gross)ability to synthesize a number of symbolic mediations into a perfor-mance demanding the use of gross musculature while monitoring a complex physical activity involving motion

2

Q(PTF)Qualitative Proprioceptive Temporal (Fine)ability to synthesize a number of symbolic mediations into .mance demanding the use of fine musculature while monitoring a complex physical activity involving timing

Q(PTG)Qualitative Proprioceptive Temporal (Gross)ability to synthesize a number of symbolic mediations into a perfor-mance demanding the use of gross musculature while monitoring a complex physical activity involving timing

The remaining are defined as:

Q(CEM)Qualitativc Code Empathetic sensitivity to the feelings of others

Q(CES)Qualitative Code Estheticability to enjoy the beauty of an object or an idea

Q(at)Qualitative Code Ethic--commitment to a set of values, a group of principles, obligations and/or duties

Q(CH)Qualitative Code Histrionicability to exhibit a deliberate behavior, or play a role to produce some particular effecton other persons

Q(CK)Qualitative Code Kinesicsability to understand, and to communicate by, non-linguistic functions such as facialexpressions and motions of the body (e.g., smiles and gestures)

Q(CKH)Qualitative Code Kinestheticability to pertorm motor skills, or effect muscular coordination according to a recom-mended, or acceptable, form (e.g., bowling according to form, or golfing)

Q(CP)Qualitative Code Proxemicsability to judge the physical and social distance that the other person would permit,between oneself and that other person

Q(CS)Qualitativ Code Synnoeticspersonal knowledge of oneself

Q(CT)Qualitative Code Transactionalability to maintain a positive communicative interaction which significantly influencesthe goals c : the persons involved in that interaction(e.g., salesmanship)

Q(CTM)Qualitative Code Temporalability to respond to or behave according to time expectations imposed on an activityby members in the role-set associated with'th'at activity

II. Cultural Determinants

There. are three cultural determinants of the meaning of symbols:

IIndividualityuses one's own interpretation as an influence on meanings of symbols

AAssociates--symbolic meanings are influenced by one's peer group

FFamilyinfluence of members of the family, or a few ;Jose personal friends, on the meanings of symbols

III. Medalities of Inference

The. third se, of cognitive style data includes elements that indicate the individual's modality of inference, i.e., the form ofinference used:

MMagnitudea form of categorical reasoning that utilizes norms or categorical classifications as the basis for accepting orrejecting an advai.ced hypothesis

DDiffeiencea tendency to reason in terms of one-to-one contrasts or comparisons of selected characteristics ofmeasurements

RRelationshipthe ability to synthesize a number of dimensions or incidents into a unified meaning, or through analysis of asituation to discover its component parts

LAppraisalthe modality of inference employed by an individual who uses all three of the modalities noted above (M, D, andR), giving equal weight to each in the reasoning process

3

Review of the Project Activities

Project activities spanned two years. During the first year,measurement procedures were developed, faculty weretrained, implementation plans were developed, andevaluation procedures fo- the implementation weredesigned. During the second year, the implementationplans were carried out and evaluation data were gathered,analyzed, and interpreted.

in developing procedures for measuring educationalcognitive style for this project, the initial step was toassemble a battery of measures Ora Nei to those'used atOakland Community College. To do this, Hill and selectedmembers of the Oakland Community College staffreviewed existir,g assessment instruments at ACT andselected portions of those instruments that could providethe most efficient measures of the various educationalcognitive style elements. ACT instruments were usedwhenever possible because they are both nationallynormed and nationally used. In instances where existingACT instruments did not provide mearuies of styleelements, .alternative instruments were identified and/ordeveloped. The measurement instruments were selectedby Hill and several of his colleagues at Oakland Commu-nity College because of their expertise in the technicalknowledge required to match iome of the complex cogni-tive style dimensions with measurement instruments. Afterthe individual measures for the -various cognitive styleelements were assembled into a battery, they were admin-istered on a pilot test basis, analyzed for psychometricproperties, and revised to maximize the efficiency, relia-bility, validity, and objectivity of the assessment.

Year one activities also included the training of partici-pating faculty. The training process began with orientationmeetings with faculty at the two particinating post-secondary institutions: Macomb County Community Col-lege (MCCC) in Mt. Clemens, Michigan, and MichiganState University (MSU) in East Lansing, Michigan. Theseinstitutions participated in the project because both offer a

full range of postsecondary programs to students who havea wide variety of backgrounds, goals, and aspirations.Evaluation of cognitive style in these contexts would maxi-mize the generalizability of results. In addition, these twoinstitutions were located in. relatively close proximity toOakland Community College (OCC), thus allowing easyaccess of the project staff and participants to the best avail-able technical knowledge about cognitive style. Thetraining process began with basic instruction on theconceptual framework of educational cognitive style andproceeded through six months of regular monthly, weekly,and individual meetings during which project staffmembers, Hill, and OCC faculty instructed the partici-pants on the' cognitive style mapping procedures. Thetraining culminated with eight participants developing

plans for the implementation of the mapping proceduresin :heir own instructional settings. Seven plans weredeveloped, as two participants developed a joint plan.

4

The implementation plans included provisions for evalu-ating the effect on educationa! outcomes of the use ofeducational cognitive style data and procedures. For eachimplementation setting, a study was planned to assess therelationship between the mapping of educational cogni-tive styles and student learning and attitudes. Procedureswere also developed for the systematic assessment of theeffect of implementation on the procedures and attitudesof participating faculty members.

At the onset of year two (i.e., during the 1976 fall term atMSU and MCCC), participating faculty members put theirplans into action. Project staff made periodic observationsthroughout the term to verify that implementation wascarried out according to plan. Six of the seven plans werecarried out. For each of these, evaluation data weregathered. These data and other data on teacher reactionsand opinions were then analyzed and interpreted.

s.A

Evaluation Advisory Panel

Periodically during the planning and completion of projectactivities, project staff met and consulted with an Evalua-tion Advisory Panel. This panel of four educators providedfci.mative input throughout the project and, in addition,provided expert judgment on. the meaning of the imple-'mentation and evaluation data. Panel members were: Dr.Robert Birney, Vice President for Academic Affairs,Hampshire College, Amherst, Massachusetts; Dr. John E.Roueche, Professor and Director, Community CollegeLeadership Program, University of texas, Austin; Texas; Dr.RiChard F. Snow, Professor orEducation, Stanford Univer-sity, Stanford, California; and Dr. William F. Taylor, Dean ofStudents, Polk. Junior College, Winter Haven, Florida. Thepanel met in formal session with project staff four timesduring tl-e two -year project. In addition, panel memberswere consulted on an individual basis at various timesthroughout the project. Formal Advisory Panel reportswere prepared and submitted to FIPSE at the end of eachproject year. Those reports are included in Appendix A ofthis report. Finally, panel members made recommenda-tions and provided input for this final report of the project.

Purpose of the Report

The primary purpose of this document is to provide adetailed record of the outcomes of the activities discussedabove. A second purpose is to indicate the project staff'spreliminary conclusions ahout the outcomes of theprojectespecially the potential- use of cognitive stylemapping procedures for improving postsecondary educa-tibnal practicesand to summarize the staff's recom-Mendations for continued exploration of educationalcognitive style.

Ii

Overview of the Report

Each of the next) three chapters of the report addresses aspecific facet of the project from the perspectives of theproject objectives, implementations, results, and recom-mendations.

Chapter 2 deals with the training phase of the project. Itdescribes the faculty participants, the objectives oftraining, the instructional strategies used in training thefaculty participants, and the training outcomes.

Chapter 3 concerns the development and evaluation of theprocedures/instruments used to measure educationalcognitive style elements. The development sequence isreviewed, the instruments are described, and the resuLs of

5

a series of psychometric analyses of the instruments arepresented. Technical issues associated with the measure-ment of educational cognitiVe style are explored.

Chapter 4 addresses the effect of cognitive style mappingon educational outcomes in each of the implementationsettings. Each implementation setting is described, as is theevaluation design and the results of that evaluation. Gen-eral project-wide evaluation data are then presented anddiscussed.

The fifth and concluding chapter provides a summary andan interpretation of project results. All conclusions arelisted, and unresolved issues are reviewed. Finally, recom-mendations for future research and development aresummarized.

CHAPTER 2

PARTICIPANT TRAINING

A key element in the study was the effective use of cogni-tive mapping procedures by faculty at the two po'tici-pating institutions. Thus, a training prdgram was plannedand conducted to provide faculty with a thorough knowl-edge of the conceptual framework of cognitive style, theindividual style elements, and their uses in educationalsettings. The purpose of this chapter is to describe the keyaspects of this training activity.

Description of Trainees

At the outset of the project, invitations were sent to alimited number of faculty at both MSU and MCCC invitingthem to participate in the project. Those interested wereencouraged to attend an introductory meeting; about 20persons did so. During the course of the project, changingresponsibilities and conflicting work commitmentsresulted- in the withdrawal of some of these volunteerparticipants from the project. Of the original grqup of 20,8completed training, 4 each at MCCC and MSU.

The participants from MSU included the Director of the'University Counseling Center, the Director of Research forthat unit, a faculty member in undergraduate teachereducation, and a professor in graduate curriculum andinstruction. MCCC participants included a faculty memberfrom the department of natural sciences, a physical educa-tion instructor, a humanities instructor, and the director ofa program for students-with special academic and socialneeds.

Training Objectives

The overall purpose of the training program was to ensurethat the participants understood and, could apply thecognitive style elements and the process of cognitive stylemapping in the educational context. To accomplish thispurpose, seven training objectives were identified. Upon

'completion of the training, each participant was to:

1. understand the conceptual framework for educationalcognitive style mapping;

2. understand the definitions of the cognitive styleelements and identify corresponding behavioralpatterns in students;

3. understand and interpret the data yielded by adminis-tration of the Educational Cognitive Style Test andInventory;

4. empirically map a subject by using observation anddiscussion and by adapting the cognitive style data gen-erated by the inventory to accommodate the behaviors'the student exhibited in particular situat:ons;

5. diagnose probable mode of understanding (elements ofcognitive style crucial for successful completion of agiven instructional task);

6. understand his or her own cognitive style;

7. develop strategies for using cognitive style _mappingprocedures in his or her own educational setting.

Attainment of the first 'objective was essential to theaccomplishment of all of the other objectives. Once theparticipants were knowledgeable about the conceptualframework for educational -cognitive style mapping, theythen learned definitions for the individual style elementsand how to translate student behaviors into these styleelements. This satisfied the second objective. The thirdobjective required participants to acquire the ability toaccurately interpret the data, resulting from' the Educa-tional Cognitive Style Test and. Inventory. These data werereported in the form of -a computer-generated cognitivestyle map. The process of literally translating test data into a.written map is called mathematical mapping.

Cognitive style mapping involves much more than an inter-pretation of the mathematical map. It also includes the useof information about the student obtained by the teacherthrough observation and conversation. The combined useof test and observational data determining cognitivestyle is referred to as empirical mapping. The fourth objec-tive of the training program was to teach participants tomap style empirically. Empirical mapping was stressed inthe training program as one of the most crucial skills theparticipants needed td acquire. Although mathematicalmapping results in.a useful profile of a student, empiricalmapping allows the educator or counselor to alter thecognitive style map to reflect a student's preferences orattitudes in a particulat'context.., Consideration' of thecontext within which a map,is interpreted (e.g., the par-ticular educational task and setting) is crucial to valid use ofthe map.

Thn fifth objective involved training the participants todi? the critical elements of cognitive style necessaryfor a ,udent to successfully complete a given instructionaltask. Once these critical elements of cognitive style arediagnosed, educational materials/methods that match thestudent's cognitive style can be assigned. Alternatively, theinstructor may assign materials/methods which areintended to expand or augment the student's style. Theinstructor or counselor must continuously monitor the stu-dent's progress and coordinate instructional rnaterials/methods with the style the student exhibits in changingeducational contexts. Empirical mapping can play a crucialrole in this process.

The sixth objective was for the participants to identify andunderstand their own cognitive styles and the implicationsof their styles for teaching, counseling, and administrativefunctions. Each participant took the Educational CognitiveStyle Test and Inventory and was then involved in a step-by-step interpretation of the elements contained in themap. This self-knowledge facilitated the participants'understanding of the potential differences between them-selves and their students, and of the need for reconcilingthe differences in planning instruction.

Participants attained the seventh and final objective bydeveloping implementation plans for applying cognitive

style npping procedures in their own educationalsettings. They formulated plans by integrating formativeinput from project staff with the knowledge and skillsgained through the training process.

Training Strategies

Participant training was coordinated and conducted by afull-time project staff member who had extensive trainingit and experience with mapping and using educationalcognitive style. This experience included planning andparticipating in workshops on cognitive style throughoutthe country. Hill also played a major role in the trainingprocess by presenting lectures and seminars and byworking on a one-to-one basis with trainees.

A variety of instructional methodsincluding lectures,small group discussions, seminars, simulations, films, andindividual meetingswere used to help the participantsachieve the seven training objectives. In addition, theparticipants were given reading materials and take-homeassignments. Training sessions were held at each institu-tion to'accommodate local scheduling needs. The particu-lar methods used to achieve each of the seven objectivesare indicated by Xs in Table 2.1.

TABLE 2.1

Methods Used for Achieving Training Objectives

Training Objective Lectures

SmallGroup

Discussions/Seminars

Simula-tions Films

IndividualMeetings

.1. Framework of wgnitivestyles' X

2. Definitions of elements X

3. Mathematical mapping X

4. Empirical mapping X

5. Diagnosis of instructionaltaSks X X

6. Understanding of owncognitive style X

7. Development of implemen-tation plans X X

X

X.

8

Two of the five instructional methods, small -group discus-sions/seminars and individual meetings, were used inachieving each of the seven objectives. Lectures were usedfrequently to accomplish six of the seven objectives.Simulation techniques were used to accomplish five of theobjectives. Participants were paired so that each personcould empirically map his/her partner. The end result wasa partial profile of each participant derived throughobservation and discussion with that person. Finally, a filmwas used once during the orientation session. Staff fromOakland Community College who were skilled in the useof educational cognitiv'e style mapping also assisted in thetraining process.

One of the most difficult training objectives to accomplishwas the mastery of the language specific to educationalcognitive style. To ease this difficulty, participants weredeliberately placed in training situations where they had touse the language. This technique, combined with the useof flash cards and other devices, eventually provedsuccessful.

Evaluation of Training Procedures

Four major evaluation activities were conducted to helpdetermine whether the training procedures were success-ful. First, at the end of the initial six-month trainingsequence, the participants received a questionnaire askingthem whether they felt comfortable and competent inusing information contained in a cognitive style map.Those participants who responded negatively receivedindividualized training duririg the summer prior to imple-mentation.

A more formal evaluation activity _was'.conducted at theend of the first year. Each participant was given five cogni-tive style maps to review and was asked to provide written

intefpretations of the maps. Project staff rated these inter-pretations on a scale from 1 (low) to 10 (high); the ratingyielded a score reflecting the participant's ability to inter -.pret maps. Each participant demonstrated a high level ofperformance.

The third activity was an evaluation of the degree of com-petency and understanding demonstrated by participantsin the design of their implementation plans. Concep-tualizing and designing an implementation plan required afundamental understanding- of ,the cognitive style processand language. Participants developed seven plans, each ofwhich exhibited the required understanding.

The final evaluation activity occurred at the end of theproject. Participants were asked to judge whether they hadbeen adequately trained for effectively implementing themapping procedures. All participants responded affir-matively.

Conclusions about the Training Procedures

Project staff, consultants, an.d .participants agreed at thebeginning of the project that a meaningful and objectiveevaluation of the effect of cognitive style mapping bnstudents' learning would be possible only if knowledge-able and well-trained instructors implemented themapping -procedures. Consequently, participants com-pleted an- extended period of training carefully gearedto the accomplishment of specific training objectives. Theend result was .a group of .participants well founded incognitive style mapping. These educators successfully com-pleted objective assessments of their competence; expertsin educational cognitive style.attested to their level of com-petence; and the trainees themselves reported that theywere confident of their knowledge of the cognitive styleprocedures. Thus, the training phase of the project wasjudged highly successful.

ri

J

CHAPTER 3

DEVELOPMENT OF COGNITIVE STYLE MEASURES

A principal goal of the project was to develop and refinemeasures of the various dimensions of educational cogni-tive style. These measures were to be Jsed' in mappingstudents' cognitive style and in evaluating the impact of heuse of cognitive style maps on educational outcomes. Thischapter describes the specific objectives, procedures, andresults of that effort.

Objectives of the Instrument Development Phase

The primary objective of the instrument developmentphase of the project was to select existing instruments or todevelop new instruments, for assessing the -32 cognitivestyle elements (see Table 1.1, pp. 2 -3): A second objectivewas to administer the instruments to a group of students inorder to carry out a comprehensive .technical analysis ofthe efficiency, objectivity, reliability, and validity of themeasures.

The paper and pencil instruments themselves constitutebut one step in the mapping of cognitive stylethat is, themathematical mapping process. A second step in cogni-tive style mapping, empirical mapping, is the process of

,generatinggenerating a cognitive stylemap on the basis of subjectiveassessments of student characteristics resulting' fromconversations with and direct observations of the student.Due primarily to the complexity of the empirical mappingprocess, it was not studied in detail in this project. Rather,all the description and technical analyses reported in thischapter address, mathematical mapping. Research shouldbe conducted on the empirical mapping process, itself,since it is a key element of the mapping process.

Instrument Development Procedures

The two project objectives related to instrument devel-opment were achieved by carrying out the following steps:

Step 1. Copies of the measurement instruments availableat-ACT were provided to the educational cognitivestyle specialists on the OCC staff, who identifiedthose style elements thalcould be effectively mea-sured by the instruments. The OCC staff also identi-fied for adaptation OCC-originated 'instruments

ir)that measured style element's that could not be

assessed by the ACT instruments. OCC had previ-ously developed/assembled a battery of instru-

' ments for measuring the cognitive style-elements.The battery had been extensively used in numer-ous contexts.

Step 1 was carried out early ir..he first year of the.project.OCC staff were uniquely knowledgeable of what consti-tuted appropriate measures for the very complex andhighly specialized set of variables that comprise the cogni-tive style map. Existing ACT instruments included mea-sures consistent with the Hill conceptualization of-cogni-tive styleaptitude measures, preference measures, andattitude measuresand therefore served-as the core of thecognitive style measurement instruments.

Step 2. The selected or adapted instruments wereassembled into a battery aid administered togroup of students (n =135)" on a pilot basis at MSUand MCCC for the purpose of analyzing 'the relia-bility and efficiency of the measures.'"

This step, also carried out during the first:, year of theproject, identified any obvious inadequacies in the assess-ment procedures, so that refinements and adjustmentscould be made prior' to project implementation duringyear two. In addition, this pilot testing aided in training theparticipants by giving them experience in administeringand interpreting the assessment battery.

Step 3. The revised battery was administered at MSU andMCCC (n=258), and the data were subjected to acomprehensive technical analysis from various per-spectives, including efficiency, objectivity, relia-bility; and intercorrelations among measures.

Step 3, carried out during the second year of the project,documented the psychometric characteristics of the mea-sures related to the elements of the educational cognitivestyle map. The data were also used to evaluate the con-gruence between the measures and the various cognitivestyle elements.

Step 4. Several major components of the cognitive stylebattery assembled specifically for the project and

..the original OCC-developed. battery were admin-istered simultaneously to a group (n =93) of stu-dents at MCCC.

This step also took place during year two. It provided a

basis for a systematic evaluation of the concurrent validityof the measures selected or adapted by the OCC staff forthe project. The question addressed was whether theproject battery assembled by educational cognitive styleexperts yielded measures of the individual style elementscomparable to the OCC measures.

Outcomes of Instrument Development

Instrument Assembly

The first four columns of Table 3.1 and all of Table 3.2indicate the linkage between the project measures and theindividual cognitive style elements. The first line of Table3.1, for example, indicates that the Language Usage Test ofACT's Career Planning Program was judged appropriate tomeasure the Theoretical Visual Linguistic (TVL) styledimension.

Results of the Preliminary Technical Analysis

The reliabilities of instruments in the preliminary version ofthe Project Cogh:tive Style Test and Inventory are listed inTable 3.1. All instruniciscs (excpt the Inventory of Pref-erences used to measure Cultural Determinantsaddressed below), yielded appropriately high internal con-

: sistency reliability estimates.

The data also show that the initial battery required 156minutes to administer. In an effort to reduce the amount oftesting time, three changes were made in the researchbattery: (1) the two math tests in the original battery wereequated to eliminate redundancy; (2) a different, shorterreading test was adopted; and (3) the mechanical reason-ing test was shortened by one-half. The resulting batteryrequired only 109 minutes to administer.

Results of the. Final Technical Analysis

The data resulting from administration of the battery. inyear two of the project were used in a second technicalanalysis of the instruments. This second analysis includedthe determination both of internal consistency reliabilityestimates for the measures and of a matrix showing coeffi-cient correlations between the various measures. The relia-bility 'estimates are reported on the right side of Table 3.1.They are all slightly lower than previous estimates, but arewithin acceptable limits with the exception of the measureof Cultural Determinants.

One obvious reason for the low reliability estimates of theCultural Determinant scales is the length of the scales. A10-item instrument with a reliability around .55, if extendedto 20 to 40 items, would yield a reliability estimate of .70 to.85, a rangecomparable to that of other instruments in the

12

battery. Therefore, the instrument was retained and usedduring the project in spite of the relatively low reliabilityestimates.

The matrix of intercorrelations is reported in Table 3.3.Several conclusions can be drawn from these data. First,with one notable exception, the various dimensions ofstyle appear to be statistically independent. The exceptionis the relationship between Theoretical Visual Quanti-tative (TVQ) and Theoretical Auditory Quantitative (TAQ).The correlation between TVQ and TAQ is .69, which verynearly approximates the reliabilities of the two instru-ments. This correlation suggests that these supposedlyindependent style elements cannot be reliably differ-entiated on the basis of the particular measures provided inthe battery. TVL is defined on page 2 as the "ability to findmeaning in written words." A high score in this areaindicates someone who reads with better than averagecomprehension. Yet, the correlation between the TVLmeasure selected by the OCC staff team and the standard-ized reading test is only .51. Further discussion of this pointwith specialists in educational cognitive style revealed that,Conceptually, TVL includes verbal reasoning and grammarin addition to reading. These additional factors are coveredby the language usage test included in the cognitive stylebattery.

Finally, of the many Inventory of Competency scales inTable 3.2 identified with the various Qualitative Symbolelements, scales said to measure the same style element are....uncorrelated. Such an outcome is contrary to expectation,given the traditional definition of construct validity.

Results of the Coadministration of OCC and ProjectBatteries

To further investigate. the validity of the project cognitivestyle measures, a subset of the OCC-generated and theproject cognitive style measures was administered simul-taneously to a group of 93 students The relationshipsamong nine Qualitative Synibol scores, three CulturalDeterminant scores, and four Modalities of Inferencescores resulting from this coadministration were deter-mined, and are reported in Table 3.4. The median corre-lation between independent measures of the sameelement over the 16 elements is .06. The correlations rangefrom -.17 to +.19, revealing a high level of statisticalindependence.

As a final step'in this search for relationships between theOCC and project measures, more intensive analysis wasconducted of the relationship between Inventory of Com-petency scales and OCC measures of selected style ele-ments. The OCC staff members involved reviewed thescales associated with each style element and identified theone scale of those listed that was most similar to the OCCmeasure of the same style element. The results, of thatcorrelational analysis are reported in Table 3.5. Once again,




Instrument Assembly









12










Instrument Assembly









12







3

TABLE 3.2

Relationships between Style Elementsand Inventory of Competencies

Style Element Corresponding Competency Scalea

Qualitative SymbolsAuditoryOlfactoryTactileVisualProprioceptive

FineKinematic GrossTemporal Gross

ProprioceptiveGrossKinematiclempora I

EmpatheticEs.theticEthicHistrionicKinesksKir:estheticProxemi6SynnoeticsTransactiona!Temporal

Modalities of InferenceMagnitudeDifferenceRelationshipAppraisal

Trade, BDTradeTrade, Tech, Arts, BDTrade, Tech, SC, Arts, SS, BD.

Trade, Tech, SC, Arts, BC, BDBCTrade, Arts, BC, BD,

Arts, BCBCBCSS, BCTrade, Tech, SC, Arts, SS, BC, Br

Trade, Tech, SC, Arts, SS, BC, BDTrade, Arts, SS, BC, BDArts, SS, BCTrade; Tech, SC, Arts, SS, BC, BDSS,

Trade, Tech, SC, Arts, 55,.8C, BD55, BCArts, SS, BC, BD

Trade, Tech, BDArtsSS, BCSC, Mechanical Reasoning Test

Pi

'aBC Business Contact; BD =. Business Detail; SC = Science;SS = Social Service; Tech = Technical.

15

\,

TABLE 3,3'

Intercorrelations among Tests and Subscales Contained in the

Project Cognitive Style Test and Inventory Booklet

TVL TAQ TVQ MR TAL Rdg, Trades Tech, Sci, Arts SS BC BD

TVL

TAQ

TVQ

MR

TAL

Rdg,

Trades

Tech,

Science

Arts

Social Serv..

Bus, Contact

Bus. Detail

I

A

.81 .41

.80

,37

.69

.77

.08

.39

.44

.79

.35

.46

.50

.37

.66

.51.

.55

.49

.25

.55

.81

-.23

.01

.18

.53

.09

-15

.86

-.02

.22*

.26

.50

.11

-,05

.74

`.83

,11

.10

.10

.21

.09

.06

.27

.46

.83

'.13

-.15

-.18

-.17

.04

,02

-.17

.38

.77

.13

.01

.02

-.08

.23

.12

-.05

.06

.25

.58

.87

.06

.06

..08

.10

.16

-.04

.27

.28

20

.25

.36

.63

a

.20

.11

.08

-.10

.15

.16

-.07

.03

.06

.26

35

.56

.81

.00

-.03

-.11

-.09

-.09

-.07

-.07

-.04

-.08

.04

-.06

-.04

.01

.54

.02

.20

.18

.12.

.11

.19

-.01

-.01

.01

-.01,

.00

,06

.10

.03

.51

.1S

.19

.19

.28

.11

.12

.09

.05

-,08

-.03

.05

.09

.07

.46

,52

Note: KR or coef alpha reliabilities are on diagonal,

TABLE 3.4

Results of Coadministration ofOCC and Project Cognitive Style Batteries

Style EleMent

OCC Project

Correlation X Decile S.D. X Det..e S.D.

-

Qualitative Symbols ..

Empathetic -.06 6.25 1.90 6.77 2.40

Esthetic .02 6.09 2.01 8.33 1.26

Ethic -.05 6.04 2.05 8.32 1.26

Histrionic .17 5.36 1.82 7.98 1.62

Kinesics .07 5.17 1.79 7.25 1.89

KinesthetiCs .19 5.30 1.83 8.32 1.26

Proxemics .13 5.01 1.84 6.77 2.40

Synnoetics .13 6.44 1.43 8.33 1.26

Transactional -.05 5.53 1.65 6.77 2.40

Cultural Deteiminants..

Individual .06 5.77 1.26 5.23 1.52

Associate .,-.17 3.45 0.95 4.51 1.37

Family .06 3.87 1.16 4.23. 1.40

Modalities of Inference.,Magnitude .05 4.00 1.34 7.83 1.49

Differenc-e ..08 2.98 1,89 5.48 2.13

Relationship .06 3.41 1.08 6.78 2.40

Appraisal .03 4.93 1.27 5.84 2.76a

TABLE 3.5

Correlations between Deciles on Inventory of Competency Subscalesand Decile Value for Selected Qualitative Symbolsas Measured by the OCC Cognitive Style Battery

ProjectOCC Measure

Measure CES CS CET ,CT CH CK CKH CEM CP

TR -0.12 0.12 -0.14 0.00 -0.22 -0.00 0.01 -0.12 -0.13

TE -0.13 0.07 -0.05 0.11 0.00 -0.12 0.12 -0.03 _ ' -0.02

SC 0.05 0.14 -0.09a 0.14 -0.17 0.07 0.07, -0.03 0.02

AR 0.10a 0.01 -0.15 0.06 -0.01a 0.11a 0.20 -0.01 0.01

SS 0.08 0.10a -0.00 0.17a 0.02 -0.04 0.11 '' -0.01 0.09

BC- -0.02 0.08 -0.04 -0.05 0.05 0.01 0.07 -0.1)A 0.06a

BD -0.10 0.11 0.05 -0.00 0.13 -0.06 0.04 -0.04 0.17

'Indicates the best possible match between the OCC measure and competency scale according to educational cognitive style specialists.

17

a universal lack of relationship between project measuresand OCC.,measures was revealed. Even those measureswhich were a good match' conceptually proved statisticallyunrelated. Clearly the OCC measures and the projectmeasures selected by OCC staff were not tapping the samestudent characteristics.

Summary of Instrument Development

The project objectives regarding instrument developmentwere: (1) to develop measures of the style dimensions and(2) to field test those measures in ordta to conduct acomprehensive analysis of their psychometric character-istics. Both objectives were achieved. Participating OCCstaff assembled a battery of instruments intended tomeasure the 32 dimensions of style, and that battery wasadministered and twice analyzed. .

18

These analyses yielded the following conclusions:

1. The battery could be administered in a relatively shortperiod of time. (usually less than two hours) and couldyield a great deal of data. Therefore, it was efficient.

2. With the exception of the measure of Cultural Deter-minants, the measures were found to be sufficiently reli-able for purposes of the study. The deficiency in theCultural Determinants measure was consideredcorrectable.-,,_

3. High intercorrelations for some of the measuressuggested some redundancy in the instruments.

4. Intercorrelations among independent measures. ofkor'ne of the cognitive style dimensions revealed.anabsence of construct validity among the measures:

The problem of validity as it relates to the ability to inter-pret the research results is addressed in the introduction toChapter 4.

CHAr (ER 4

THE RELATIONSHIP BETWEEN THE USE OF EDUCATIONALCOGNITIVE. STYLE MAPPING AND EDUCATIONAL OUTCOMES

Im'plementation Goals and Strategies

Another primary goal during the first year of the projectwas for participants to-develop detailed plans for imple-menting.the cognitive style mapping process in their owninstructional settings. Each plan contained three parts. First,the overall goal of the implementation was described interms of a mission statement. Second, the major objectiveswere described in terms ofdesign criteria. Finally, theprocedures to be used in achieving the objectives weredescribed asperformance goals. Collectively, these threeparts formed a comprehensive description of the imple-mentation procedures to be carried out by each partici-pant during, the second year of the 'project.

P.articipants.formulated their implementation plans over aperiod of three months. A total of seven plans weredeveloped: four by staff at Macomb County CommunityCollege ana three by staff at Michigan State University. Theprimary instructional and evaluation objectives of eachsetting are outlined in Table 4.1. These objectives wereachieved in six of the seven implementation' settings.Implementation was not successfully carried out at theMSU Counseling Center, where local conditions made itimpossible to complete the plan.

Implementation of plans in the remaining six contexts per-mitted exploration of the impact of educational cognitivestyle mapping on approximately 250 'students of diversebackgrounds who were striving to achieve a wide range ofeducational goals. This chapter describes each imple-mentation setting and the evaluation methodology used ineach setting to explore impact; it also presents and inter-prets the results of the evaluation. In addition, following areview of the implementation settings, the results of struc-tured teacher debriefing interviews and student attitudesurveys are presented.

Alternative Perspictives on the Interpretation of theImplementation Results

For any study involving an assessment component, thevalidities of the instruments used are crucial to the inter-pretation of the results. If the instruments lack validity forthe context in which they are used, the meaning of the out-comes of the study as measured by those instruments will

be confounded. Therefore, a basic question concerningthe instruments used in this study is: What constitutes anappropriate indicator of validity of the cognitive stylemeasure?

The data reported in the previou; chapter suggest that theparticular instruments assembled for this study lack con-struct validity, which is a type of validity some researchersfeel, is essential in a study of this type. However, there areother researchers who view construct validity as second-ary. Among those researchers who hold this alternativeview, there is a large number who rely heavily on thecontent and face validity of measurement instruments.-

This validity distinction is important in this study becausethe measures .tre used only to generate a mathematicalmap of inuividLals' cognitive style attributes. Hill's modeldoes not slop with the mathematical map, but ratherrequires an empirical mapping of cognitive style attributesthat combines relevant data from the mathematical mapwith all other ,types of data (e.g., the instructor's knowl-edge of an individual's prior experiences and informationgenerated through direct contact between the instructorand the student). Such an approach relies on the data gen-erated from the cognitive style measures to provide a pointof departure, rather than an end point, for determinationof an individual's cognitive style.

The cognitive style instruments used in this study wereassembled by experts knoWledgeable about the meaningof each of the cognitive style elements; these experts wereconfident that the various instruments provided adequatemeasures of the cognitive style elements from a contentperspective. Therefore, there is strong reason to believethat the measures do have both face and content validity.

The preceding points should be considered by those whowould conclude that the results of this study providedefinitive evidence on the effect of cognitive stylemapping in educational contexts similar to those at MSUand MCCC: No such conclusion can, be made solely on thebasis of this study by thole who would require that themeasures must possess construct validity because theevidence collected does not warrant such confidence inthe .instruments used. However, for those who are satis-fied that the instruments possess sufficient content or facevalidity and that this type of validity is most crucial, the

19 ,

TABLE 4.1

Summary of Implementations

ImplementationContext

InstructionalObjective

EvaluationObjective

MCCC Science

MCCC Humanities.

MCCC Special Needs

Tailor instructional materials toindividual student styles

Offer a variety bf instructional treatmentsto students'to meet the demands ofvarious students' styles

Provide preenroliment remediation tostudents with special academic needsusing cognitive style oriented instruction

MCCC Physical Differentiate successful from unsuccessfulEducation golfers on the basis of style elements

MSU Educational NonePsyChology

MSU GraduateCurriculum Course

Instruct students assigned to one of twoinstructional treatments designed to matchtheir style

MSU Counseling Conduct career/vocational counseling usingCenter information on style as a basis for the

counseling process

Assess impact of tailored instruction inacademic-and attitude outcomes comparedto traditional instruction

Assess the importance of match betweenstudent style and instructional methodin predicting course outcomes

Evaluate the impact of special treatmentin postenrollment educational performancecompared to previous groups

Assess the reliability of the differentiation

Evaluate the correlation between student-teacher style match and instructionaloutcomes

Assess degree of match between studentstyle and treatment style and comparegroup performance

Compare counseling Outcomes achievedboth with and without information aboutcounselor's style

results offer more substantive evidence on the issuesaddresSed oy the study.

The results of this study reflect only, the relationshipsbetween educational cognitive style as determined in thisstudy (combining both mathematical and empiricalmapping as proposed by ,Hil;) and the outcomes resultingfrom the implementations carried out. Additional researchis needed to address issues and questions on the potentialin education for other conceptual models of cognitivestyle.

Implementation in a Science Course

Description

Two science classes (experimental and control) with equalmean course pretest scores were formed at MCCC. Each

20

class was then randomly divined in half. Although allstudents in each class took the Educational Cognitive StyleTest and Inventory, only half the members of each classreceived map interpretations of their style. The controlclass was taught using a traditional approach; the experi-mental class was given prescriptive instruction. Students inthe experimental class whose maps were interpretedreceived prescriptions based on their mathematical cogni-tive style maps. Students in the remaining half of the classwere randomly assigned individual prescriptions for,learning.

Evaluation Methodology

Two analyses were performed on the resulting science-related data. First, the average class test performances forthe students in the traditional and prescriptive instruction

classes were compared by means of an analysis of covari-ance procedure that took into account the. students'entering ability level. This procedure ,was also used toinvestigate the impact of map interpretation and instruc-tion on final average in science class and to study the inter-active affect of method of instruction and receipt of mapinterpretation. Second, the dropout rate for the students ineach of the two classes was com, ared to the averagedropout rate, and the dropout rate for the prescriptiveclass was also compared to the average dropout rate farthree traditional science classes taught the same semesterby the same instructor.

There were two important limitations in the scienceevaluation. The instructor may have biased the control

.treatment by empirically (subjectively) mapping studentswho should have been randomly assigned a prescriptionfor learning. For some of these students, then, the prescrip-tions may have been partially based on empirically mappedcognitive styles rather than randomly assigned. Further,those students who received prescriptions based on theircognitive styles were permitted to self-select any othermaterials they wished. This tainted the experimentalcondition.

Resuhs

1. Final Average. Table 4.2 summarizes the final gradeaverages of the- two instructional groups involved in thescience implementation. The mean grade average fui thestudents in the prescriptive (experimental) class was higherthan that for the students in the traditional (control) class.The mean for the students in the prescriptive class whoreceived map interpretation was slightly lower than themean for students in the same class whose prescriptionswere not Leased on their cognitive style maps. It is notpossible to determine at this point whether the improve-ment in final grade average for the prescriptive class was afunction of the experimental nature of the individualizedinstruction or of the impact of cognitive style map inter-pretation and instruction, although the former interpreta-tion seems more likely. As further supp'ort for this inter-pretation, the analysis of covariance results shown in Table4.3 identifies a significant instruction effect, but not asignificant interaction effect.

In order to further investigate the implementation andnature of the instruction, a special set of interview ques-

TABLE 4.2

Final Average for Students in the Science, MCCC ClasI

Interpretation

yes

no

Total

Prescripti,e

Instruction

Traditional

75.4 71.1

(n=12) (n=12)

76.5 68.5

(n=15) (n=10)

76.0 69.8

Total

73.2

72.5

TABLE 4.3

Summary Table

Analysis of Covariance

Source df SS F P

Pretest 1 2166.05 18.86 0.0001

Interpretation 1 0.37 0.00 0.9546

Instruction 1 1028.77 8.96 0.0045

Interaction 1 50.78 0.44 0.5096

Error 44 5054.67

21

tions was developed for the instructor to answer. Theanswers suggest that the students who received both mapinterpretations and prescriptions based on their cognitivestyles were free to vary from and ati, ont their prescrip-tions as they desired. The same was true for the studentswhose prescriptions were not based on their cognitive stylemaps. There is some reason to believe that the instructormay have assigned these students prescriptions on the basisof her assessment of their cognitive style, rather thanrandomly as was called for in the experimental design.Answers to some of the questions in the interview suggestthat the students in the prescriptive class were aware of thenature of the experiment and had been advised that theywere the group for which large gains in performance wereexpected.

2. Dropout Rate. Table 4.4 displays the number of studentswho withdrew from each class before the end of the.semester. The analysis of this difference in dropout patternbetween the two classes yielded a X' value of 5.11, which issignificant at the .05 level.

A total of nine students dropped out of the two classes;seven of these were in the traditional class. The twostudents in the prescriptive class who did not complete thesemester both received individualized instructionalprescriptions based on their cognitive style maps.

During the fall semester, the instructor taught two scienceclasses in addition to the two classes involved in this experi-ment. A traditional instructional approach was used in bothclasses. For the three classes taught using traditional tech-niques in the fall 1976 semester, the average dropout ratewas 30 percent. When the 7.4 percent dropout rate for theprescriptive class is compared to the mean dropout rate forthe traditional approach, the difference is statisticallysignificant at the .05 level. it would appear, therefore, thatthere was a relationship between the method of instruc-tion and the dropout rate of students in the science classes.Furthermore, that relationship,' which favors prescriptiveinstruction, does not appear to be influenced by the='waythe prescription was determined.

TABLE 4.4

Comparisons of Number of Student Dropouts in Prescriptive and

Tiaditional Classes and Who Did or Did Not Receive Map Interpretation

yes

Interpretation

no

Total

Instruction

Prescriptive Traditional Total

2

0 b

2 7

6

22

Interpretation of Results

The impact of the instructional method is apparent ineach of the sets of data analyzed. The prescriptive instruc-tional method, whether or not the map was used, pro-vided more positive results in the academic, retention, andattitude outcomes than nonprescription instruction.Students in the prescriptive class achieved an averagegrade significantly higher than that of the students in thetraditional class. However, compared to the randomassignme'ht of a prescription for learning, the effect of mapinterpretation on the academic outcomes, as measured byfinal course test a-,?:age, was not substantial. The reten-tion rate was significantly higher 'for students in theprescriptive class than for students in the three traditionalscience classes taught by the same teacher the samesemester. Within the traditional setting, there was atendency for those students who received map interpreta-tions to remain in the class throughout the semester.

'Implementation in a Humanities Course

Description

The course instructor identified the pertinent cognitivestyle dimensions for each of three methods of presentingcourse materiallecture, group discussion, and mediaand specified the optimal level of performance foreach selected element. The map elements and optimalperformance levels 'for each method' were assembled toform the "probable mode of understanding" optimal stylefor each approach. In addition, a probable mode of under-standing that included,-. the common elements of thelecture, group discussion, and, media approaches wasdeveloped for the entire course. The correlation between

-student style/mode match and course outcomes wasevaluated.

Evaluation-Methodology

Student cognitive style maps were generated andcompared with the probable mode of understanding(optimal map) for each method and the course as a whole.The match between the actual student map and theoptimal or ideal was then used to predict the student's final_exam score.

Results

Student -to- Instructional- Method Match. The four instruc-tional .methods used to communicate the course mate-riallecture, group, media, and courseand their opti-Mal decile level are shown in Table 4.5. This table lists the

cognitive style elements and the scores on those elementsthat the student should have achieved to maximize thech'ances of () profiting from the instruction presented and(2) succeeding in the class:Four sets'of match scoresoneset for each of the probable modes of -understand-ingwere calculated for each student enrolled in a

humanities class in fall 1976. Match scores were calculatedfor the elements in the method's probable mode Of under-standing map. Since there are 15 elements in the probablemode of understanding map for the lecture method, 15match scores were calculated for every student in eachhumanities class. The formula used for the generation ofmatch scores was the one recommended and used by Hill:

XM XR

1 - I XRi - XS I

10

where:XMij = match score for element for student

XR = instructional approach (referent) optimal

decile level for map, element i

XS it e=student . decile value on map element .

For each instructional method, two stepwise regressionprocedures were used to predict the same criterion scoreon the final exam. The first stepwise regression includeddecile scores on the mode of understanding and the matchvariables as predictors. In a second analysis, the match vari-ables were used alone as predictor Variables. A signifi-cance level of .10-was stipulated for each analysis. Tables4.6 and 4.7 show sur imaries of the results of the analyses.Included are the - regression equati,..ns obtained and thepercent of variance accounted for by use of the appro-priate map deciles concatenated with match scores, andmatch scores on the appropriate map elements aspredictor va riables;respectively.

For each of the stepwise regression equaticins, the percentof variance accounted for by the predictor variables is lessthan 3G. To pursue further the relationship between thepredictor variables and the criterion, correlations werecalculated and are presented in Tables 4.8 through 4.11 forthe lecture, group, media, and course approaches, respec-tively. These tables indicate that, except for TAL and TVL,the correlations do not appear to be diffee'ent from zero atthe .05 significance level.

23.

TABLE 4.5

Probable Mode of Understanding ofHumanities Instructional Methods

(1) Lecture/DiscussionStyle element Percentile

(3) Media (Audio-Visual)Style element Percentile

FR T(AL) 80-89 FR T(AL) 70-7912 T(VL) 70-79

Q(V) 80-89Q(A) 70-79 Q(A) 70-79Q(CKHy, 80-89 Q(041) ' 80-89Q(CES) 4 70-79 Q(CES) 70-79Q' (a) 60-69 r Q(CET) 70 -79.Q' (CET) 60-69 Q(CTM) 70-79Q.(CEM) 60-69Q.(CTM) 50-59 80-89Q(CK) 70-79 A' 60-69

I 70-79 M 70-79.A' 60-69 R 70-79

80-8960-69

R' 50-59

(2) + Group Activities: Style element Percentilt

(4) Course OverallStyle element Percentile

12 T(AL) 70-79 12 T (AL) 7Q-7910 T(VL) 70-79 . I 70-79

A E0-69Q(V) 80-89 M ., 70-79Q(CT) 70-79 Q(CES) 70-79Q(CES) 7t5 :79 . Q(CET) 60-69Q(CEM) 70-79Q(CS) 70-79Q'(CET) 60-69QICH) 50-59Q'(CK) 50-59Q(CP) 50-59

70-79' A 70-79

F' 40-49

M 80-8960-69

24

TABLE 4.6

Stepwise Regression Equations Obtained byUsing Appropriate. Map Deciles and Respective

Match Scores as Predictors

InstructionalMethod R2 Prediction Equation

Lecture

Group.

Media

Course

.28

.29

.07

.16

32.01 + 1.00 (TAL) + 1.48 (TVL) - ,14.19 (match,TVL) - 8.49 (A)

37.04 + 1.07,(TAL) + 1.59 (TVL) - 16.86 (match'TVL) - 26.26 (match F)

32.88 + 1.27 (TAL) - 13.34 (match A)

30.15 + 1.27 (TAL) - 17.46 (match I)

TABLE 4.7

Stepwise Regression Equations Obtained, byUsing Match Scores on Appropriate Map Elements as Predictors

Instructional ,

Method. R2 Prediction Equation

Lecture

GroupMedia

Course

Interpretation of the Results

.21

.18

.13

.03

12.66 + 17.93 (TAL) + 11.49 (TVL) + 10.14 (R)

10.74 + 22.80 (TAL) + 11.50 (TVL) + 9.2,3 (Q(CP))

18.5111- 20.41 (TAL) + 5.61 (TVL)

34.45 - 25.21 (I) + 6.41 -(M)

Attempts were made to use map deciles concatenated withappropriate match scores or match scores aline to gen-erate a stepwise regression equation for predicting thescore on the final examination for any of the instructionalmethods; these attempts failed. The small percent ofvariance accounted for by the regre.gsion equations is due,at least in part, to the nearly zero correlations between-themajority of predictor variables and the criterion final exam -ination score. With the match criteria selected for thisresearch, there was essentially no statistical relationshipbetween the majority of the elements in the maps of themethods of instruction and the final examination score.The language, area is the one instance where there doesseem to be some relationship between test data .and'courseoutcomes. TVL and TAL which could be interpreted as

25

c,.

measures of verbal scholastic aptitude, were found to havea nonzero correlation with humanities course grades`.

Implementation in a Special Needs Setting

Description

MCCC offers preenrollment orientation classes for stu-dents whose probability of success at collegesis low.One objective of the orientation program is to build up thestudents' basic skills in order to improve their chances ofsuccess once they enroll. This implementation called forthe use of cognitive style information in choosing instruc-tional materials intended to help students develop in theirbasic skill areas.

TABL.E 4.8

Correlations between Predictor Variablesand Final Exam Score:

Lecture Approach

Map ElementRaw Score

Correlation withCriterion

Match ScoreCorrelation with

c Criterion

TAL a394a

TVL .388a

.249 a.110 .177

A -.065 -.146M -.046 -.014b -.103 .012

.008 .071Q(A) . .008 -.087Q(CEM) .008 .070Q(LES) -.038 -.049Q(CET) -.038 -.063Q(CK) -.045 .021Q(LKH) -.038 -.042Q(CT) .008 .070Q(CTM) -.034 -.030

aSignificantly different from zero at a = .05 level.

TABLE 4.9

Correlations between Predictor Variablesand Final Exam Score:-

Group Approach


-Correlation withCriterion

Match Score'Correlation with

Criterion

.TAL .405 .352 aTVL .388 a .249 a

.110 .177A -.065 -.065F . -.041 .007M -.046 -.1)14b -.103 .012Q(V) -.026 -.050Q(CEM) .008 .053Q(CES) -.038 -.049Q(CET) -.038 -.063Q(CA) -.095 .012Q(CK) -.045 .005Q(CP). .008 .071Q(CT) -.039 -.049Q(CTM) .008 .053

aSignificantly different from zero at "Zx = .05 level.

26

TABLE 4.10

Correlations between Predictor Variablesand Final Exam Score:

Media Approach


Correlation withCriterion

Match ScoreCorrelation with

Criterion.

TAL .405 a.352 a

.110 .177A -.065 -.147M -.046 -.010R .008 .053Q(A) .008 -.087Q(V) -.050Q(CES) -.038 -.050Q(CET) -.038 -.050,Q(CKH) -.038 -.042Q(CTM) -.038 -.079

aSignificantly different from zero at Gt = .05 level.

TABLE 4.11

Correlations between Predictor Variablesand Final Exam Scare:

Course Approach .

Raw Score Match ScoreMap Element Correlation with Correlation with

Criterion Criterion

TA L

'AMQ(CES)"Q(CET)

.405 a

.110-.065-.046-.038-.038

.005-.158-.035

.095

.044.044

aSignificantly different from zero at CV = .05 level.


Two data sets were examined in order to ascertain theimpact of, incorporating cognitive style infOrmation in theorientation program,of special needs students at MCCC.First, the academic progress of the 31 students in the 1976orientation classstudents for whom cognitive style datawere collectedwas compared with the progress of the 66students in the 1975 orientation classstudents for whomcognitive style data were not collected. Second, since 10 ofthe 31 members of the orientation class were retested onthe Educational Cognitive Style Test and Inventory after theorientation program was c,ompleted, growth on the basicskill map elements was evaluated.

There were two limitations in this evaluation. First, the yearwhen instruction was received was totally confounded withthe treatment condition, since control and experimentaltreatments occurred in different years. Second, only one-third of the experimental group participated in the retestto assess growth in basic skills.. Therefore, the results maynot be representative of the entire group.

Results

1. Academic 'Progress. A comparison was made of 'theindices of academic progress for the 1975 (control) and1976 (exOrimental) orientation groups. The indicesincluded percent completing orientation, percent of thosecompleting orientation who enrolled for credit, averagenumber of credits attempted, average proportion of creditscompleted, and mean grade point average. The results ofthose comparisons are presented in Table 4.12.

The results suggest that, although the sizes of the twogroups differed greatly, the proportion of students in eachgroup completing the orientation program was constant.The 'principal difference between the groups was in theproportion of those completing orientation who actuallyenrolled for,college credit. A higher, proportion of 1976students (experimental) enrolled. However, although theyenrolled for a slightly larger number of credits on the

-average, they tended to complete a slightly smaller propor-tion of those credits and achieved slightly lower grades.

A probable outcome of the incorporation of cognitive styleinformation rin the orientation program of the specialneeds students, therefore, may have been an increase inthe numbers of special' needs students who enrolled inacademic courses at MCCC. Since these students, on theaverage, had a lower grade point average than the 1975students, the use of cognitive -style information duringorientation may have given more confidence to the lessable students and motivated them to proceed' into coursework. An interesting follow-up study would involve deter-mining whether these students who were less successfulthan their 1975 counterparts actually have pursued their

27

academic 1:). grams into subsequent terms. Such data arenot presently available,

2. Growth in Cognitive Style Elements from September toDecember 1976. Of the 32 students in the orientation class,10 retested in December 1976. The small sample size for theretesting was a fpnction of scheduling problems andstudent availability. As was noted above, this small volun-tary sample may not be representative of the entire group.

Tabte 4.13 shows the mean decile ranking for. the twogroups for each of the 32 map elements. Although, on thewhole, the mean for the group tested in December washigher than the mean for the group tested in September,the question of growth is confounded by the select.natureof the December cohort. To examine the issue of growthmore fully, the five basic skill' map elements of most impor-tance to this setting (TAL, TAQ, TVL, TVQ, and ReadingLevel) were compared for each of the 10 students retestedto assess the change in scores between September andDecember. The results of, this analysis are presented inTable 4.14. Grade level norms are reported for the Theo-retical Symbols and Reading Level. An increase fromSeptember to December,testing was defined as a change ingrade level score of one year or more (about two standarderrors of measurement). A decline was defined as adecrease in performance reflected by a one year or moredrop in grade level score. All other changes in scores werereported as no change. Of the 50 possible sc3re increases,10 actually occurred. Theoretical symbols TVL and 'TVQ,

TABLE 4.12

Comparison of Special Needs Students' AcademicProgress for 1975, and 1976 Classes

1975 1976

31

56 (85%) 25 (81%)

39 (70%) 23 (92%)

Number scheduled to beginorientation 66

Number completingorientationNumber of those completingorientation who enrolled forcredits

Mean number of credits11.5

63.0 60.9

2.09 1.94

attempted 11.2

Mean proportion of creditscompleted

Mean GM

4.

and Reading Level were the map elements indicatinggrowth for some students. Ten percent of the scoresshowied decline in performance from September toDecember; TAL and TVL were the map elements on whichsome decline was noted. Seventy percent of the scoresfrom the two testing experiences differed by less'than onegrade level. Since no data were available on the normalgrowth experienced by the special needs students after theorientation program, no determination could be made ofthe extent to which the cognitive style implementationimpacted growth.

TABLE 4.13

Comparison of Map Results for OriginalGroup of Students and a Subset of Those

Students Who Were Available forRetesting in December 1976

Mean DecileMap Element September December

(n = 32) (n = 10)

TAL 4.5 4.5TAQ 2.8 3.6TVL 4.6 3.1TVQ 3.9 5.0Q(A). 7.1 7.7Q(0) '6.5 7.1Q(T) 7.5 8.5Q(V) 7.6 8.5Q(CEM) 6.1 6.2Q(CES) 7.8 8.6Q(CET) 7.8 8.6Q(CH) 7.7 8.3Q(CK) 6.7 7.5Q(CKH) 7.8 8.6Q(CP) 6.1 6.2Q(CS) 7.8 8.6

Q(CT) 6.1 6.2Q(PF) 7.8 8.6Q(PKF) 5.2 5.2Q(PTF) 7.7 8.3Q(G) 6.5 7.5;Q(PKG) 5.2 5.2Q(PTG) 5.2 S.2

Q(CTM) 7.0 7.6\5.1 5.6

A 4.5 4.8F 4.5 3.8L 4.1 5.1M 7.3 8.3

5.0 5.8'6.1 6.2

28

c..


More students from the 1976 orientation class completedcourses than did students frOm the 1975 class, but the 1976group's grade point average was lower than that for the1975 cohort. The fact that more of the less able studentswere motivated to complete courses may be viewed asevidence of an impact of the use of cognitive style infor-mation in the orientation. The results of an investigationinto the amount of giOwth from September to Decemberin the 1976 class were.confounded by the lack of data onthe typical amount of growth experienced by the specialneeds students. For the Theoretical Symbols and ReadingLevel, 70 percent of the students retested in December didnot appear to change in their relative abilities, 10 percentshowed score declines, and 20 percent showed scoreincreases. Since only a small, voluntary sample was avail-able for the retest and attitude data collection, generaliza-tionsbeyond these subjects are not justified.

Implementation in a Physical Education Setting

Description

Using videotape recordings, golf students were' evaluatedon a set of criteria for their golf swings. On the basis ofthese data, two groups were formed: successful-and unsuc-cessful. Cognitive style maps for the students in thesegroups were used to generate maps characterizing success-ful and unsuccessful students. _

TABLE 4:14

Growth in Theoretical Map Elements and Reading Levelbetween September and December 1976 Testing

MapElement

Growth

Decline No Change Increase

TAL 2 8 0

TAQ 0 10 -

TVL 3 2. 5

TVQ. 0 7

Reading Level 0

'Total 5 35 10


The physical education implementation afforded anopportunity to investigate techniques for developing mapsthat could he used to collectively describe or portray thecognitive style characteristics of a group. Three methodsfor determining group maps were compared. Also, astepwise regression procedure was used to generate aprediction equation on half of the students and to cross-validate on the other half. The overall intent was to try toidentify procedures that would reliably differentiate cogni-tive style among groups of students.

Results

On the basis' of the frequency distribution of criterionscores from two sections of golf students, two groups atextreme ends of the golf swing ability scale were selectedto form the successful and unsuccessful groups. Table 4.15displays the distribution of criterion scores and the scoresof the students in the two groups.

Three methods of forming group maps were investigated.T'he first was the method used most frequently in the workof Hill, his associates, and his students. According to thismethod of analyzing group cognitive style, if 70 percent ofthe members of the group indicated a decile ranking on aparticular map element of 70 or higher, the elemententered the group map at a "major" level. If 70 percent ofthe group showed a decile ranking on a particular mapelement of 30 or higher, the element became a member ofthe group map at the "minor" level. Each of the 32elements of the map was considered one at a time, and Itsmembership in the collective map for the group was deter-mined. Table 4.16 shows the maps which describe success-ful and unsuccessful groups of student determined by thismethod.

The second method for determining group maps was theaverages method. According to this method, the groupaverage decile ranking for each of the 32 map com-ponents was used to determine the major or minor statusof the elements in the group's collective map. Table 4.17displays the maps formed for the successful and unsuc-cessful groups using the averages: method.

The third method for determining group maps employedan analysis of variance procedure. The decile rankings forthe elemen6" of each of the four map areasTheoreticalSymbols, Qualitative Codes, Cultural Determinants, andModalities of Inferencewere subjected to an ANOVAprocedure to determine which elements reliably differ-entiated groups. Only the Qualitative Codes area pro -vided an F-ratio that was significant at the .0001 level. ABonferonni follow7up was used to'identify the elementsthat contributed to the group differences for the 21 quali-tative code symbols.

29

Since,the 21 qualitative codes are generated from the sevencompetency scales, an ANOVA procedure comparing thecompetency scale deciles for the two groups was con-sidered. This procedure provided an F-ratio that was notsignificant, indicating that the difference for the twogroups lay in the translation of the competency scaleresults into the qualitative codes and not in their perfor-mance on the competency scales. A summary of theanalysis of variance results is shown in Table 4.18. Table 4.19contains the results from the Bonferonni follow-up proce-dures that were used to identify tbe qualitative codes con-tributing to the significant F-ratio;

TABLE 4.15

Identification of Criterion Scores of Subjectsin Successful and Unsuccessful Groups .

Section Score

51005100510051005100

8888

110114115

Unsuccessful

51005101

510051005101

51005100510051005101

5101

5100.1100

510051005101

510051005101

130130

134134135137

138140143150153

154157160165

-165165180190

51005101

51005101

5101

.195195195

196210

Successful

A comparison of the collective maps determined by thethree methods described above is shown in Table 4.20.Only the 70-percent-of-the-group method was successfulin identifying any map elements that would oe unique tothe maps of the successful or unsuccessful groups.

A stepwise regression was used to determine the predicta-bility of the criterion score by the cognitive style elements.This regression procedure used raw scores on the 16 testsand subscales of the Educational Cognitive Style Test andInventory as predictors of the criterion score. Seventeen ofthe 31 students were systematically chosen to represent therange of criterion scores. The regression equation shownbelow accounted for 90 percent of the variance.

= -6.28 + 5.74 (Listening Comprehension) + 2.94 (Science)= -2.13 (Arts) + 4.98 (Business Contact) - 5.91 (Associates)

Raw scores for the remaining 14 students were then usedinthe regression equation to predict criterion scores. TheSpearman's Rho corre;atfon index for, the predicted andactual ranking on the criterion score was 0.3.

Since the averages and ANOVA methods were unsuccess-ful in determining unique map elements, and the cross-validation of the regression equation yielded unsatis-factory predictability, an investigation was initiated toexamine the characteristics of the profiles on the cognitivestyle elements of the students in the successful and unsuc-cessful groups. Sample profiles are shown in Figures 4.1and 4.2 for the map areas Theoretical Symbols and CulturalDeterminants. By examining these profil*s, it is possible tovisualize much within; group variance on the mapelements. This large within-group variance contributes tothe inability to reliably differentiate among groups withrespect to style.


The commonly used and accepted statistical decision ruleswere unable to reliably differentiate between thsuccess-ful and unsuccessful groups on the basis of these educa-tional cognitive style, data. The 70-percent-of-the-grouprule was the only one that was able to identify uniqueelements and differentiate the groups. The extent to which

,A

these groups are in fact p-sjf ehologically or educationallydifferent from one anotheras educational cognitive stylespecialists suspect they areis a matter deserving furtherinvestigation.

TABLE 4.16

Unsuccessful andsSuccessful Collective MapsDetermined by the 70-Percent-of-the-Group Method

Unsuccessful Map

TAL'TAQ'TVL'TVQ'

Successful Map

TAL'TAQ'TVL'TVQ'

Q(V)Q(CES)Q(CET)Q(CH)

Q(A)Q(0).Q(T)Q(V)Q(CEM)Q(CES)Q(CET)Q(CH)Q(CK)

Q(PF)Q(PTF)

Q(PG)Q(CTM)

Q(CKH)Q(CP)Q(CS)Q(CT)Q(PF)Q(PTF)Q(PG)Q(CTM)

A'F'

Q(PKF)'Q(PKF)'Q(PTG)'

MD'

A'F'

ML'

R

TE

SC'ARTS'BD'MECH

TR'TE'ARTS'BC'BD'

:'MECH R'

/30 v

TABLE 4.17

Uniutcessful and Successful Collective MapsDetermined by the Averages Method

....'Successful

TAL' Q(A) Q(CT) I I' L' TR'TAQ' Q(0)' Q(PF) .. A' M TE

TVL' Q(T) Q(PTF) F' SC'TVQ' Q(V) Q(PG) R' ARTS'

Q(CEM) Q(CTM) 55'Q(CES) . Q(PFK)' Bt'

(CET) Q(PKG)' BD'L4(CH) Q(PTG)' MECH R'

: Q(CK)Q(CKH)Q(CP)Q(CS)

Unsuccessful

TAL' Q(T) Q(A)' I' L' TR'TAQ' Q(V) Q(0)' A' M' TE'TVL' Q(CES) Q(CEM)' F', D' SC'TVQ' Q(CET) Q(CP)' R' AR -')'

Q(CH) Q(CT)' YQ(CK) Q(PKF)' F r

Q(CKH) - Q(PKG)' .1

Q(PF) Q(PTG) ..,SECH R'Q(PTF)Q(PG)Q(CTM)

r

TABLE 4.18

Analysis of Virianc, AestiP,Successful and Unsucc essful C soups

Map Component F Valv PR F

Theoretical Symbols 0.' i .6137Qualilative Codes 17 ,i3 .0001Affiliation f.02 .8827Modalities of Inference A.62 .4355COmpetency Scales 0.40 .5313.

I nix( 4.17

Bonferonni Followup

Qualitative Codes

E ';1% c ?.WU iu U U U U" " " ' ; '6666aaaaa6a 6a6ei a 6 6 6a

Unsuccessful X 4.8 4.4 0 7.4' 7.4 5.2 7.8 7.8 7.2 7.2 7.8 p 7.8 5.2 7.8 5.0 7.2 7.2 5.0 5.0 7.2

1 sd 2,6 2.9 -- 1.1 1.1 3.9 '0.8 0.8 1.5 1.5 0.8 3.9 0.8 3.9 0,8 4.2 1.5. 1.5 4.2 4.2 1.5

Successful ' g 7.4 5.2, 0 8.2 8.4 7.8 8.6 8.6 8.4 7,8 8.6 7.8 8.6 7.8 8.4 6.2 8.4 7.6 6,2 6.2 8.4

sd 1.5 1.1 -- 0.4 0.5 1.3 0.5 0.5 0.9 1'3l

0.5 1.3 0.5 13 0.5 3.1 0.5 1.1 3.1 3.1 0.8

Xs - Xti 2.6 0.8 0 0.8 1.0 2.6 0.8 0.8 1.2 0.6 0.8 2.6. '0.8 2.6 0.6 1:2 1.2 0.4. 1.2 1.2 1.2

t valuea , 1.73 0.27 -- 0.68 1.71 1.26 1.69 1.69 1.37 , .60 1.69 1.26 1.69 1.26 1.27 .46 1.52 .43 .46 .46 .48

40

,

'Using a Bonferonni technique to control the experimentwise type I error rate at Iht,05 level, ,05Ewt8 F 4,50.

TABLE 4,20

Comparison of Results

from 70%ofthe.Group, Averages, and ANOVA Techniques

Collective Map Formation

Technique

Number of Elements in

Collective Map

Number of Unique Elements in

Collective Map

Successful Unsuccessful Successful Unsuccessful

1.70% of the grli?p

2. Averages,: ,

3. ANOVA u

31

31

32

19

31

32

12a

0

0

0

0

Note: Maximum number of map elements . 32,

aSix of the map elements present at the "major" level in the Successful collective map were identified as members of the Unsuccessful

collective map at the "minor" level,

Theoretical Symbols

Unsuccessful

10 10

1

TAL TAQ TVL

10

TVQ

Y, 5O

Group Averages

Successful

1STAL

Unsuccessful

.a)

5C.)

1

TAL

TAQ

Successful

TAQ TVL TVQ

Figure 4.1. Map Profiles, Physical EducationMCCC.

433 2

TVL TVQ

NUnsuccessful

Cultural Determinants

Successful

A

10

73 5

1

Group Averages

...

Unsuccessful

Successful

-aA

Figure 4.2. Map Profiles, Physical EdurcationMCCC.

34 YvA -

A

Research in an Educational Psychology Setting

Description

.,Six interpersonal skills laboratory classes participated in thisimplementation. For each of the classes, match scoresbetween each student and the lab instructor were cal-culated on the tests and subscales of the EducationalCognitiye Style) Test and Inventory. Information derivedfrom the cogniitive style battery did not influence instruc-tion in the daises; neither teachers nor students had theirmaps interpreted until the end of the semester.

Evaluation Methodologyes

The participants from the MSU School of Teacher Educa-tion provided six classes for investigating the relationshipbetween the degree of student/teacher match on cogni-tive style elements ,Ind.course outcomes: A behavioral out-come measure and a series of attitudinal measures wereconsidered.

Results

For each of the six classes, raw scores on the 16 tests andsubtests of the Educational Cognitive Style Test and Inven-tory were used to calculate 16 match scores betweenteacher and student according to the Hill procedure:

= XR

where:

1 - I XRi - X5ii

ti

XMij = match score on testi for student

XR = referent (teacher) score an test i

XS ij = student score on testi

ti

= maximum score on test

The students' match scores were then concatenated withtheir raw scores on the 16 tests and subscales of the Educa-tional Cognitive Style Test and Inventory. The scores werethen used as predictor variables for four separate end-of-course criteria:

1. Final score on the course behavioral observation check-list

2. Attitude Instrument Subscale 1: Teacher Evaluation3. Attitude Instrument Subscale 2: Learning Outcomes4. Attitude Instrument Subscale 3: Group Participation

The teacher determined a student's final score on thecourse behavioral observation checklist, by rating thestudent at :three levels of mastery on a series of inter-personal skills. The three subscale scores were determinedfrom the end-of-course administration of an attitudeinstrument designed specifically for the MSU educationalpsychology course. The students were instructed to rateeach of 23 statements on a continuum from one throughfive ranging from strongly agree to strongly disagree. Theitems identified as measures of each of the three subscalesare listed in Table 4.21; scale reiiabilities are also given.

Table 4.22 summarizes the results of the stepwise regres-sion procedures used to predict each of the four criteriausing th,.! 32 predictor variables (16 raw scores and 16 matchscores). The criterion for the inclusion of variables was setat a = .10 for each regression equation. The regressionequation for the group participation subscale score cri-terion accounted for 37 percent of the variance. The twocriteria most directly related to teacher and student inter-actionthe behavioral observation checklist determinedby the teacher and the students' evaluation of theTAyielded regression equations that accounted for 11and 29 percent of the variance, respectively.

Since the major purpose of this implementation was todetermine the relationship between student/instructorcognitive style match and course outcome measures, thefour course outcome measures were used as criterionvariables in stepwise regression procedures using only the16 match :cores as predictor variables. The results of theseregression analyses are shown in Table 4.23.. The criterionfor variable inclusion was, again set at a = .10.

The correlations between the criterion measures and the32 predictor variables were examined in order to investi-gate further the lack of success in obtaining regressionequations that account for a satisfactory amount ofvariance. Tables 4.24, 4.25, 4.26, and 4.27 present corre-lations of the predictor variables with the behavioralobservation, evaluation of TA, learning outcomes, and

%group participation criteria. The magnitudes of the corre-lations suggest that there is very little relationship betweenthe predictor variables and the criterion. This is especiallytrue for the correlations between match score andcriterion.


No relationship was found between the matchas definedin the st. .1yof student and teacher style and any of thefour criteria used in the study.

35

TABLE 4.21

Subscales from Educational Psychology Attitude Instrument

Subscale 1: Evaluation of TA (reliability = .81)

Items

7. I feel that my TA cares about me as a person.

9. My TA does not confront me in the IPL group.

13. My TA is usually on time for the IPL session..

17. I feel very threatened by my TA.

19. My TA usually gives me responsible feedback.

20. In general, I am very satisfied with my IPL group leader.

21. My TA adequately integrates the Ed 200 content (textbook material) with IPL.

Subscale 2: Learning Outcomes (reliability = .67)

Items

. 2. I am satisfied with my own progress in mastering the IPL skills.

11. My TA presents the Ed 200 "subject matter" in a way I understand.

18. I feel that my TA's confrontation of me is facilitating my growth.

23. I am satisfied with my own level of 'mastery of the IPL skills.

Subscale 3: Group Participation (reliability = .89)

Items

3. My group leader usually helps me feel comfortable in the group.

-4. My TA is helping me feel like sharing myself honestly with this group.

5.' I feel that I belong in this group.

8. My TA is helping me feel that 1 belong in this group.

10. I feel comfortable participating in my grdup.

15. I know the names of most of the people in my IPL.

16. I usually feel like talking in my group.

22. Most (or all) group members help me, to feel-good about what is happening in the group.

Implementation in a Graduate Education Course for two units in a graduate education course. Referentmaps identifying the key map components for the

-Description approach were formed, for each group. This wasaccomplished by project staff and the instructor, who

Two approaches to instructionIndividual/Independent evaluated the material and methods to be used in the twoand Peer/Discussionwere designed and implemented approaches and determined the key map elements and

TABLE 4.22

Results of Stepwise Regression Proceduresto Predict Behavioral Observation and Attitude Measures

Using the 16 Raw Scores Concatenated with 16 Match Scores as Predictor Variables

Criterion

Behavioral Observation

Evaluating TA

Learning Outcomes

Group Participation

R2

.11

.28

.07

.37

Predictor Equation

49.11 - 38 (Social Service) + 34.16 (Match Social Science)

19.22 + 0.29 (TVL) - 0.20 (TAL) - 0.49 (Science) + 15.64 (Matdh Science) - 9.75 (Match Arts)

10.84 - .167 (Business Contact)

30.40 + 0.40 (TVQ) - 0.31 (Mechanical Reasoning) - 0.56 (.1"/L) - 2.56 (Social Service)+ 0.13 (Reading Rate) + 14.42 (Match Mechanical Reasoning) - 10.88 (Match i3dsines-iDetail)

TABLE 4.23

Results of Stepwise Regression Proceduresto Predict Behavioral Observation and Attitude Measures

Using 16 Match Scores as Predictor Variables

Criterion

Behavioral Observation

Evaluation of TA

Learning Outcomes

Group Participation

R2

.08

.06

.13

.17

Regression Equation

32.80 + 24.12 (Social Service) + 23.79 (Trades)

24.266 - 6.488 (Listening Comprehension) - 5.744 (Arts)

14.404 + 3.654 (Mechanical Reasoning) - 8.688 (Trades) - 7.423 (Business Detail)

21.310 + 18.285 (Language Usage) + 9.015 (Mechanical Reasoning)' - 22.479 (ListeningComprehension) - 12.900 (Business Detail)

their minimum decile value for optimum learning. A set ofmap elements was determined for each approach; studentsin the class were divided into two groups by comparingeach student's cognitive style map with the referent map ofeach approach.


Class members were .sorted into two groups so as tomaximize students' chances of learning the course

material. Assessment of the reliability of this sortingprocess was part of the evaluation. In addition, -twooutcome measures-scores on a written theme and scoreson a multiple-choice test-were used to compare theperformance of the two groups.

Results

1. Learning Outcomes. The primary objective of theimplementation plan was to sort the graduate education

37 .4 0

'-1

TABLE 4.24

Carelations between Predictor Variables and the Behavioral Observation Criterion

Raw ScoreCorrelation

with Criterion

Match ScoreCorrelation

with Criterion

Language Usage .14A .051Numerical Computation .185 .128Mathematics Usage .119 .038Mechanical Reasoning .150 , .065Listening Comprehension .121 .043Trades .035 .240Technical .022 .198Science .053 .088Arts -.045 .050Social Service -.073 .287Business Contact .009 .206Business Detail .113 .150Individual .074 -.069Associates -.038 .140Family -.009 -.153Reading Rate/

Comprehension .095 .023

TAB_E 4.25

Correlations between Predictor Variables and the Evaluation of TA Criterion


with Criterion


with Criterion

Language Usage .193 .073Numerical Computation .055 .083Mathematics Usage .135 -.061Mechanical Reasoning -.089 .017Listening Comprehension -.174 - -.144

-.114Trades -.076Technical -.086 -.012Science -.209 -.007Arts 0 -.105 -.187Social Service -.045 -.053Business Contact -.220 .011Business Detail -.104. -.121Individual -.051 .007Associates .044 .114Family -.054 -.045Reading Rate/


TABLE 4.26.

Correlations between Predictor Variables and the Learning Outcomes Criterion


with Ciiterion


with Criterion

..._

Language Usage .046 -.021

Numerical Computation .000 .006

Mathematics Usage .050 -.093Mechanical Reasoning -.055 .117

Listening Comprehension -.121 -.102Trades -.213 -.194Technical -.126 -.133Science -.092 -.063Arts -.056 -.144Social Service ' -.149 -.132Business Contact -.274 -.163Business Detail -.159 -.244Individual -.148 -.045Associates .067 .123

Family . .044 -.045Reading Rate/


TABLE 4.27

'Correlations between Predictor Variables and the Group Participation Criterion


with Criterion


with Criterion

Language`Usage .205 .108

.Numerical Computation .178 .122

Mathematics Usage .239 -.041

Mechanical Reasoning . -.035. .147

Listening Comprehension -.237 -.209

Trades -.194 -.162Technical -.148 -.149

Science -.181 -.061

Arts -.072 -.137Social Service -.264 -.193

Business Contact -.238 -.174

Business Detail -.176 -.193

Individual -.189 -.098Associates .146 .029

Family ' .004 -.003

Readir.g Rate/Comprehension .134 .017 -

class students into two groups on the basis of thecomparison of students' cognitive style maps withtwo referent maps. The referent maps, labeled theIndividual/Independent approach and Peer/Discussionapproach maps, are shown in Table 4.28. The eight studentswhose cognitive style maps matched the Individual/Inde-pendent referent map received that individualized instruc-tion. The remaining nine students, whose cognitive stylemaps matched the Peer/Discussion referent map, werepresented material through group discussion.

Two outcome measuresa term paper and a testwereused in the course. Results for these outcome measures arepresented in Table 4.29. Students in the Peer/Discussiongroup performed significantly better on the test than didstudents in the Individual/Independent group.

2. Reliability of Sorting. Profiles of the students in the twogroups were plotted so that an evaluation could be made

of the degree of success achieved in using cognitive mapswith referent maps to sort students into the two groups.Figure 4.3 shows sample profiles for th TheoreticalSymbols map am as: Three separate graphs ar displayed:Individual /Inds pendent, Peer/Discussion, an GroupAverages. Relevant referent points are indicated n thegroup averages graph. Examination of these plots ma es itpossible to determine visually if the two groups diffetheir map structures and also to check the degree of matcbetween the referent group map and the average mapprofiles of the two groups. This was done for all of theelements, although only a sample of these plots ispresented.

An analysis of variance procedure was used to identify mapelements that' were different for the two groups. Asummary, of the results is presented in Table 4.30. None ofthe F-ratios reported was significant at the .05 level.

TABLE 4.28

Referent Maps for Individual/Independent and Peer/Discussion Groups

PercentileElement Required Element

PercentileRequired Element

PercentileRequired

Individual/Independent12-FRT(VL) 70-79' 70-79 R 70-79Q(CS) 70-79 A' 50-59 70-79Q(CET) 80-89 M' 60-69Q(CTM) 70-79 D' 50-59Q(CH) 60-69Q(CEM) 40-49

Peer/Discussion

FR

T(AL) 80-89 70-79 70-79T(VL)' 50-59 A 70-79 M 70-79Q(CET) 70-79 F 40-49 R 60-69Q(CEM) 60-69 D 50-59Q(CH) 70-79Q(CS) 50-59Q(CTM) 70-79Q(CT) 70-79

40A -1

TABLE 4.29

Outcome Measures forIndividual/Independent and Peer/Discussion Groups

Individual/Independent

Peer/Discussion t value

n ; -8 9

paper 7 3.56 3.44 0.02sd 0.30 0.39

testsd

3.31

0.463.810.26 2.50

a .05 tt5 = 1.75

.05t

14= 1.76

TABLE 4.30

Analysis of Variance Results forMap Components of Individual/Independent

and Peer/Discussion Groups

MapMap Component F Value PR F

Symbols 0.91 .3429

ualitative Codes 0.10 .7491

C ltural Determinants 0.02 .8868

Mo alities of Inference 0.37 .5468

Com etency Scales 0.80 .3729

Interpretation\of Results

The results suggest there was little success in separating thestudents into two\ distinct groups corresponding to thereferent maps. ThiS\ is evidenced by the differencesbetween group profiles and referent profiles, and also bythe large within-group variance in style. It is impossible todetermine at this time whether another set of referentmaps would allow for more success in group sorting. Thetask is quite complex, since there are many map elementsto consider simultaneously in\ sorting the students intogroups having similar profiles. \\

Inability to differentiate between groups on the basis ofstyle was also seen in the physical, education setting;inability to reliably match a desired Or, optimal style wasseen in the humanities setting. These seem to be consistentoutcomes.

Summary of Teacher Debriefing Interviews

Results of postimplementation debriefing interviews eon-ducted with each participating teacher, are summarizedbelow. The introductory statements and the questions arepresented verbatim and are followed by general responses.The interviews were structured; each participant re-sponded to the same questions.

Project Training. Now that you have participated in thisinitial implementation of educational cognitive style, wewould like you to look retrospectively at the training phaseof the project.

Q. Did the training Process provide you with the skills andknowledge regarding cognitive style that you neededto implement the procedures properly? Please explain.

A. Without exception, the 'participants felt they weretrained well enough to effectively implement thecognitive style mapping procedures.

Q.

A.

Q.

What were the most notable positive aspects. oftraining?

Most positive aspects of training varied from partic-ipant to participant; they included the empiricalmapping and simulation exercises, the development ofimplementation plans, and the commitment to train-ing of OCC staff and project staff.

What revisions or improvements in the training pro-cess would you suggest?

A. Participants agreed that training was toe fragmented--a more intensive, extended training 'workshopwould have been more helpful. In additiOn, more mapinterpretation training sessions were desired.

Implementation. Of course, one important aspect of theproject is the wide variety of settings in which cognitivestyle is being implemented. It is very important thereforethat we obtain rather complete descriptions of the actualimplementation procedures in each setting.

Q. Would you please describe in some detail the mannerin which you used cognitive style infofmation in yourinstruction? How did this differ from the plan yououtlined over the summer?

A. With the exception of the MSU counseling center, allwere able to carry out implementations generally asplanned, though there were changes within somesettings.

Q. How did the instructional procedures resulting fromthis implementation differ from ybur normal pro-cedures or procedures you usad in previous terms.

p

V

2 1,7

Individual/Independent

Theoretical Symbols

Pee0Discussion

10

TVL TVQ TAL

Group Averages

TAQ1

TAL

Individual

0Peer

TVL TVQ

Figure 4.3. Map Profiles, Graduate CurriculumMSU.

* individual referent pointspeer referent points

A. In most cases, the majo- '',:ipact of cogs tive stylemapping was greater pre( 'aion in the kind, of instruc-tional treatments assign' 4, to students.

a

A.

Q.

A.

Q.

Q.

If you were to plan pottier impleme; cation in thissame setting, what you changes Why?

Participants genery agreed that they would spendmore time with interpretation.

Have you cc Itinued to use 1 he cognitive stylemapping prc. :.,dut4s this term? ',My or why 'not?

Of the se( participants, twr had continued intoanother se nester and a third inned to continue. Cir-cumstarrrs prevented others from doing'so.

In :ospect, did you have. the same confidence inyou mstrisctional procedures this term as you haveha,2,in previous terms? V /hy-or why not?

A. r,,e participants generally felt greater confidencen,cause they believed ',hey knew student needs better

.and understood how i istructional treatment would berelated to those neer s.

Anecdotal Information It is important in debriefing anexperience such as thi, to make a record of special events,situations, or occurrr,nces that seem noteworthy becausethey lead to some g( neralization or conclusion about theexperiente. We wr.old like to note any such anecdotalinstances in your implementation of educational cognitivestyle.

Q. Do any surtr events stand out in your mind?

A. Three par acipants mentioned that students seemed tosense an agreement between the maps and their ownview of themselves.

Q. As y Ju think about the`relationship between yourstur" ants and the instruction they received, were therear.. instances where you saw a particularly effectivem itch between style and instruction or a very obviousnismatch? Why do you-think this occurred and whatr. suited?

A. Strong, Associates people seemed to work welltogethe)-.. In fact, obvious matches or. mismatchesreported irk all cases involved the Cultural Deter-minants style dimension.

Q.

A.

a

Were there any special problems that arose? Howwere they handled?

The principal problem that arose was a shortage oftime to devote to the project.

4/

43

React/on to Measurement Procedures. Another phase ofthe project that we would like your opinion about is theCognitive Style Test and Inventory Booklet.

Q. At any time during the project, did you review thevarious instruments in the battery to see if theyseemed to reflect cognitive style elements as youunderstand them? If so, what were your conclusions?

A. Four participating teachers labeled the auditory mathtest very anxiety producing; four' cited the lack ofapparent correspondence between the competencyscales and qualitative map elements.

,theWould you judge tne battery to be an effective or

ineffective measure of style? Why?

A. The Participants thought that although the instrument. seemed to lack some face validity, it was an effective

measure (11 cognitive style:

Impact of Style on Students. We would also like youropinion about the impact of the implementation on yourstudents, though we realize the impact data are yet to be

`analyzed and that you will be speculating.

Q. Briefly and in very general terms, what do you thinkwas the impact of ,the project on students whosecognitive styles were mapped and whose instructionwas influenced by that map?

A. Students tend to gain self-awarenessnot just in areasof weakness, but in positive as well as value neutralareas.

Q. Were there any outcomes that you achieved that youhad not been able to achieve previouslylearning orattitude outcomes?

A. There was a general increase in individualization.

Q. Was there an impact on communication or any otheraspect of the interpersonal atmosphere ?.

A. There was no generalized impact on interpersonalenvironment.

Q. How familiar did your students become with the actualconstruct of educational cognitive style and thevocabulary associated with its measurement? If theywere instructed, did they understand the nature of theconstruct? How do you know?

A. There were no in-depth attempts to train students onstyle. Students generally understood the style con-cepts whenever they came into contact with them.

Level of Commitment The next area we would like toexplore is your attitude toward or commitment 'n educa-tional cognitive style mapping as an instructional pro-cedure.

Q. In retrospect, when you began training, what was yourcommitment?

A. Three participants were totally committed from thestart of the project. The other participants were willingresearchers who reported being committed to indi-vidualization.

Q. After training and before implernontatiOn, how wouldyou characterize your commitmer:t?

A. After training, all but one participant reported beingcommitted to the concept.

Q. How would you characterize your current attitude orcommitment?

A. All but one reported being highly committed at thetime of the interview.

Q. Would you encourage your colleagues to learn aboutand use cognitive styie mapping procedures in theirinstruction? Why?

A. All but one reported they would encourage colleaguesto learn about and use cognitive style.

Impact on Teachirg practice. Finally, we would like toexplore with you in some detail the impact of the projecton some specific aspects of your instruction.

Q. Do you feel that you know your own cognitive style inmost contexts?

Al All participants reported that they knew their owncognitive styles.

Q. Please state briefly how you think your knowledge ofyour own and your students' cognitive styles impactedeach of the following: (If you do not understand any ofthese, I will explain them.)

1. your course preparation ,2. your instructional methosis3. your ability to diagnose student strengths and

weaknesses4. your instructional efficiency5. your feelings of success6. your enjoyment of teaching7. your acceptance of students8. your understanding of students9. your expectations of students

A. In all but one setting, participants reported that knowl-edge of their own and their students' styles impactedvarious aspects of instructional practice.

44

Summary of Studect Attitude Survey

Students in several of the implementation studies wereasked to complete the Student Attitude Survey so thatproject staff could assess the impact of cognitive style map-ping on the attitudes of the student participants. Only stu-dents who received interpretations of their maps took thesurvey. Table 4.31 indicates the contexts in which surveydata were obtained.

The student responses are thoserof individuals with rela-tively limited exposure to cognitive style mapping and itspotential for, improving educational programs, and pro-cesses. Although the students who received map interpre-tations no doubt grasped much of the meaning of the cog-nitive style map as it related to them, the short-term natureof their exposure precluded'a complete understanding ofthe full impact of the Concept. The following data might besubstantially different were the students involved on a con-tinuing basis in an educatidnal environment where cogni-tive style mapping was an integral part of the educationalprocess.

The results of the survey are presented in Table 4.32.Responses are reported as percents and are broken downfor each of the implementation settings in which the Stu-dent Attitude Survey was administered. The results: )r eachitem were analyzed to determine whether the responsepattern for the of students as a whole differed from arandom pattern. All response patterns except that for item4b proved to be other than random.

TABLE 4.31

Sources of Student Attitude Data

Institution Number of Students

Michigan State University 17

Graduate Curriculum Course 17

Macomb County Community College 108

Science 21Humanities 79Special Needs 8

TOTAL 125 125

Two sets of conditionals were analyzed for each of the per-sonal reaction questions. Tie first conditional concernedindividual map interpretation; the second concerned timeduring the semester when the map was interpreted. Theresponse patter, ,s were subjected to a X test to ascertain ifeither variable influenced students' responses to thepersonal reaction items.

Results

1. Verification of the Administration and Interpretation., Eighty-two percent of the 125 students indicated that they

had gained knowledge of their educational cognitive styleduring the project. Fifty-six percent of the students indi-cated that their cognitive style maps had been individuallyinterpreted. This latter figure is deflazed by the students inthe humanities classes; only 37 percent of these studentsreported individual interpretations. In each of the otherclasses; over.75 percent of the students indicated that their'maps had been individually interpreted. Ninety-four per-crmt of the students received information about their cog-native style within the first three weeks of the semester.

2. Validation of the Information Provided in the Cognitiveyle Map. In every setting except one, less than half the

students responded that they felt the map and its interpre-tation provided an accurate picture of themselves. Nearlyas many students answered they were undecided. Fifty-four )..2rcent of those students reporting individual inter-pretation felt the map and its interpretation provided anaccurate appraisal of themselves. Of the 55 studentsindicating they did not receive individual map inter-pretation, 34 percent viewed the map and its inter-pretatiop as valid. Acceptance of the interpretation of thecognitive style map as accurate does appear to be affectedby the mode of map interpretation. Forty-three percent ofthe students reporting early feedback indicated that theyfelt the map and its interpretation presented an accuratepicture of themselves. Of the eight students not reporting'early feedback, 57 percent accepted the interpretation ofthe map as valid. In case, early feedback did not appearto facilitate acceptance of the map interpretation.

Approximately half the students (51 percent) indicated thatthe map provided new information about themselves. Ofthose students reporting individual interpretations,.64 per-cent responded that the map information was new. Of theremaining students who did not report individual inter-pretations, 35 consi`dered the information provided by themap to be new. Th`lis, it appears that the degree of newinformation gained by the map is a function, in part, of themode of map interpretation. Comparison of the percent ofstudents reporting early feedback with the percent notreporting early feedback shows that receiving map inter-pretation within the first three weeks does not appear toincrease the numlier of students who. perceive the mapinterpretations to be new information about themselves.

A large majority of the students (88 percent) felt that theircognitive styles were understood by their instructors. Indi-vidual interpretations and early feedback both appear tohave a positive effect, which is demonstrated in the num-ber of students reporting this opinion who i,ad either,indi-vidual (96 percent vs 79 percent) or early (89 percent vs 67percent) interpretations.

3. Using Cognitive Style Information for Making Educa-tional and Vocational Decisions. Less than half the students(42 percent) felt that the knowledge of their educationalcognitive style would help them to make better decisionsabout their educational plans. This and the other resultsreported in this section must be interpreted cautiously,however, because the students were given littlo instruc-tion on style and no instruction on or information abouthow it might assist them in making educational or voca-tional decisions. When the students reporting individualinterpretations were considered, 46 percent replied thatknowledge of their cognitive style would aid them in mak-ing better decisions about their educational plans. Thirty-six percent of the students not reporting individual inter-pretations felt that knowledge of their cognitive stylewould facilitate their educational planning. So it appearsthat, although individual interpretation does result in anincreased number of students who view educational cog-)nitive style as an aid in making educational decisions, lessthan half the students expressed this opinion regardless ofthe mode of map interpretation.

Of the students receiving early map interpretation, 44 per-cent felt that knowledge of their educational cognitivestyle would help them make better decisions about edu-cational plans. Of the group not reporting early i. iterpre-tation, none felt that such knowledge would be helpful.This seems to indicate that the timeliness of map interpre-tation has a marked effect on attitude toward this topic.

There is a similar pattern in the respOnses to the questionconcerning the use of knowledge of. educational cognitivestyle for making better vocational decisions. One-third ofthe students responded that they felt knowledge of theircognitive style would be useful for this purpose. Very littledifference is seen between the percent of students reply-ing "yes" to this question whether students were or werenot given individual map interpretations (35 percent and 32,percent, respectively). However, when the temporal fac-tor is add-essed, a difference in respoi.',..e tendency isnoted. Of those students receiving feedback on their mapwithin the first three weeks of the semester, 34 percentreplied that they felt knowledge of their educational cog-nitive style would be useful in making better vocationaldecisions. Of those students not reporting early feedback,only 14 percent responded affirmatively to this question.

4. The Effect of Knowledge of Cognitive Style on LearningOutcomes. It is crucial to note here that there were no pre-

TABLE 4.32

Results of Student Attitude Survey

1. Did you gain knowledge of your educational cognitive style during this course?

All MSU MCCC Science Humanities Special Needs

'X, yes 82 94 80 86 76 100'X, no 1C 6 21 14 24 0

2. Did you have your cognitive style map individually interpreted by your instructor?


'X, yes 56 76 53 95 37 100% no 44 24 47 5 63 0

3. Did you receive information about your cognitive style within the first 3 weeks of the semester?

All MSU MCCC

'X, yes 94 100 93% no 6 0 7

Personal RLactions

Science

95

5

Humanities Special Needs

92 1008 0

1.. Do you believe that your educational cognitive style map and its interpretation presented an accurate picture of yourself?

All MSU MCCC Science .Humanities Special Needs

'`X, yes "45 44 45 48 38 100'X, undecided 41 37 41 33 47 0

,

'X, no 15 19 14 19 14 0

Conditionals

'X, yes

'X, undecided'X, no

1

individualinterpret.

Yes No

54 3432 53

13 13*X2 =6.7

2

1st 3weeks

Yes No

434215

57

14

29 X2 =3.4

2. Did the map provide you with information about yourself that you did not have before?


'X, yes 51 47 52 52 48 88'X, undecided 20 13 21 24 22 0

'X, no 30 40 27 24 30

*X2 value is significant at X ='.05 level

46

Conditionals

1


2

1st 3weeks.

% yes% undecided

% no

Yes No

*x2 = 11.1

Yes No

X2 = 2.464 35 54 71

12 29 21

24 36 25 29

3. Do you feel your instructor understood your educational cognitive style?

Al II MSU MCCC Science Humanities Special Needs

% yes 88 88 95

% no 12 12 5

Conditionals

5i, yesoe- no'

1

individual`.interpret.

Yes No

96 79

4 21*X.2 = 7.8

2

1st 3weeks

Yes No

89 67

11 33

*x2

84 100

16 0

4. Do you believe that knowledge of your educational cognitive style will help you to make better decisions about your:

a) educational plans?

AU MSU MCCC Science Humanities Special Needs

% yes 42 29 44 40 40 88

% undecided 23 29 28 35 27 12

% no 30 41 29 25 33 0

1 2

individual. 1st 3

Conditionals interpret. weeks

Yes No Yes No

% yes 46 36 44 \ 0% undecided 30 24 X2 =3.8% 26 67 *X2 = 8.5

% no 24 40 29 33

b) vocational plans?


% yes 33 18 36 30 \\ 32 65

% undecided 30 24 32 35 \32 35

'X, no 36 59 33 35 37\ 0

*X2 value is significant at et = .05 level

47

NNN

(continued)

,TA8114.32 (continued)

Conditionals

1


2

1st 3weeks

Yes No Yes No

% yes 35 32 34 14

% undecided 33 26 X2 = 1.3 31 43 X2 = 0.3% no 32 42 35 43

5. Is it your opinion that knowledge of your educational cognitive style in this course probably helped you to:

a) learn course material faster than you usually do? ,

All MSU MCrc

% yes 16 12 17% undecided 34 35 34

% no 50 53 49

Science Humanities Special Needs

0 18 -5740 33 2860 49 14

1 2

individual 1st 3Conditionals interpret. weeks

Yes No Yes No

'X yes 16 17 17 14% undecided 31 39 X2 = 0.64 35 29 X2 = 0.6

% no 52 44 48 57

b) learn course material more thoroughly than you usually do?


% yes 25 24 26% undecided 30 35 29

% no 45 41 46

Conditionals

1


Yes No

% yes 28 23% undecided 25 34 X2 = 1.6

% no 46 43

48

22 23 71

38 27 1440 50 14

2

1st 3weeks

Yes No

27 0

29 5043 50

X2 =

f.

c) enjoy the course more than you usually do?


% yes 36 4 35 37 32 71

% undecided 27 24 27 21 31 14

% no , 37 35 38 42 37 14

1

individualConditionals interpret.

2

1st 3weeks

Yes No Yes No

. % yes 41 28 39 0

% undecided 18 40 *X2 = 7.6 27 X2 =2.7% no 41 32 34 67

6. Did knowledge of your cognitive style and the styles of others increase your awareness of the divcsity of human beings?


'X, yes 55 77 51 33 54 75

% undecided'X, no

20 .,

25

0

23

23

26

4324

1828

13

12

Conditionals

1


2

1st 3weeks

Yes No Yes No

'X, yes 72 54 ,65 50

% undecided 22 19 *X2 = 8.7 29 33 X2 = 0.7% no 6 27 17

7. Would you recommend that your friends take courses in which educational cognitive styles are mapped?


'X, yes 63 88 59 57. 57 100

'X, undecided 29 12 32 29 34 0

% no 8 0 9 14 9 0

Conditionals

'X, yes

1


Yes No

72 51

% undecided 22 38 X2 = 5.1'X, no 6 11

*X2 value is significant at a = .05 level

49

2

1st 3weeks

Yes No

65 57

29 \29

6 14

5 .3

x2

(continued)

TABLE 4.32 (continued)

8. Do you believe that other instructors would benefit from using educational cognitive style maps in their courses?


% yes 69 69 69 67 66 100% undecided 23" 25 23 29 24 0

% no 8 7 8 5 10 0

' Conditionals 1


2

1st 3weeks

Yes No Yes, No

% yes 70 65 69 71% undecided X2 = 1.7 2

= 3.324 23 25 0% no 6 12 5 29

conceived expectations regarding the effect of style on stu-dent' perception of learning outcomes. The questionsasked were simply exploratOry.

Half the students responded that knowledge of their cog-nitive style maps did not help them learn the course mater-ial faster than usual. The results do not seem to be affectedby the mode of map interpretation. Of those studentsreporting individual interpretations, 16 percent said "yes"to the question concerning learning course material faster.Of those students not reporting individual interpretations,17 percent responded "yes" to this question. The timing ofmap interpretations did not appear to impact these resultseither. Of the 117 students reporting early feedback, 17percent responded "yes," whereas 14 percent of those notreporting early feedback replied "yes." Therefore, regard-less of the mode of interpretation or time of feedback, lessthan 20 percent of the students felt that knowledge of edu-cational cognitive style helped them to learn the coursematerial faster.

It should be noted, however, that over half (57 percent)the special needs students felt that knowledge of educa-tional cognitive style facilitated the speed of acquisition ofcourse material. The students in the science, humanities,and graduate curriculum classes did not, on the average,view knowledge of educational cognitive style as an aid inlearning the course material faster:

When the issue of the impact of knowledge of style onlearning course material is addressed, results are similar forthe overall group and for those students who did or did notreport receiving individual interpretations. In each case,the percent of students who responded "yes" was about 25percent. Of those students who reported receiving early

50

map interpretation, 27 percent replied "yes." Of the stu-dents without early feedback, none thought that the thor-oughness of their learning of the course material was influ-enced by knowledge of their cognitive style. Of the stu-dents in the special needs setting, 88 percent indicated thatknowledge of their cognitive style did influence their if

learning, at least to some degree. This was not the case inany of the science, humanities, or graduate curriculumsettings.

Thirty-six percent of the students, still less than half,responded that knowledge of their cognitive style prob-ably did enable them to enjoy the course more than theyusually do. Of those students reporting individual in er-pretations, 41 percent indicated they enjoyed the coyrsemore because of their knowledge of their educational Cog-nitive style. Of those students not reporting individualinterpretations, only 28 percent responded "yes" t8 thisquestion. Thus, whether or not the students received indi-vidualized map interpretations did influence how they per-cei fed the effect of the knowledge of their cognitiye styleon :he enjoyment of the course. A similar result is fount;for tie issue of the time of map interpretation feedback. Ofthe students receiving early map interpretation, 39 percent-responded 'yes," whereas zero percent of the studentswho received no eariy feedback replied "yes." Once again,a large majority (79 percent) of the students in the special ,-'

needs setting reported that knowledge of their cognitivestyle aided in their enjoyment of the class. This reactionwas not found in the other classroom setting's.

5. Increasing the Awareness of the Diversity of HumanBeings. Fifty-five percent of the students indicated thatknowledge of their own style increased their awareness ofthe diversity of human beings. This increase in awarenessappears to be a function of the mode of interpretation,since 72 percent of those students receiving individual

interpretation signified an increase in awareness while 54percent of those students not reporting individualizedinterpretations indicated an increase. A potentially impor-tant though not statistically significant difference in thepercent of those responding "yes" is also seen betweenthose students who did and did not report eariy feedback(75 and 50 percent, respectively). The students in the grad-uate curriculum course and the special needs settingappeared to be more sensitive to the diversity of humanbeings, as indicated by 77 percent and 75 percent, respec-tively, responding "yes" to the item.

6. Recommendations for Future Use of Cognitive StyleMaps. The students definitely recommended that otherstudents take courses in which cognitive styles are mapped(63 percent said "yes") and believed that cognitive stylemaps would be beneficial to other instructors (69 percentresponds..... favorably). Of those students reporting indi-vidualized map. interpretations, 72 percent recommendedthat their friends take courses in which educational cogni-tive styles are mapped. Of those students not reportingindividual interpretations, 51 percent recommended suchclasses to their friends. Although the percentage of stu-dents responding affirmatively varies somewhat depend-ing on the mode of interpretation, both interpretationgroups tend to favor recommending courses employingcognitive style maps to their friends. Little practical differ-ence in the responses to this question is seen between thestudents reporting and not reporting early feedback; bothgroups reported a majority of "yes" respondents (65 and 57percent, respectively). The mode of interpretation andtime of map feedback do not appear to be factors for see-ing cognitive style maps as beneficial to instructors of othercourses.

Summary of Survey Results

The above results can be summarized as follows:

1. Most students verified that they had gained knowledgeof their cognitive style; most received individualizedinterpretations within the first three weeks of thesemester.

2. Less than half the students felt certain that the map andits interpretation provided an accurate picture of them-selves. A large proportion were not certain about theirperceptions in this regard.. However, they were not edu-cated about style per se and may therefore have had dif-ficulty judging the accuracy of data. In a relatedarea, the students were evenly divided with regard tothe extent to which they thought the map provided newinformation about themselves.

3. Most students either felt that knowledge of their cogni-tive style did not influence their academic performance

or enjoyment of the class, or they were undecided as toits influence.

4. The majority of students reported that knowledge oftheir own cognitive style increased their awareness ofthe diversity of human beings.

5. The majority of students demonstrated enthusiasm forthe cognitive style mapping experience as indicated bytheir readiness to recommend the experience to otherstudents.

6. The impact of the mode of interpretation is evidencedprimarily in the questions pertaining to the perceivedvalidity of the cognitive style map. Individual interpre-tation of a student's map by the instructor seemed toinfluence the perceived accuracy of the map and theextent to which students believed their cognitive stylewas understood by their instructor.

Results of the student questionnaire imply that the stu-dents, on the whole, did find the cognitive style mapexperience to be positive. However, the reason for thepositive attitude cannot be easily isolated and identified.

Conclusions Based on Impact Research

The following conclusions are based on the results.of edu-cational efforts in the six implementation settings and theresults of evaluation activities across settings.

1. Little or no relationship was found between themeasure.: of style used in this project and educationaloutcomes. (See the humanities and., educational psy-chology settings.)

2. When the match between student style and mode ofinstruction or teacher style was used to predict courseoutcomes, prediction was found to be inaccurate. (Seescience, humanities, and educational psychology.)

3. The large amount of within-group variance in style mea-sures relative to between-group variance made the reli-able differentiation of group on the basis of style diffi-cult. (See physical education and graduate education.)

4. Student reactions to the cognitive style mapping proce-dures ranged from undecided to positive. Studentsreported that these procedures had little real impact orvalue to them but suggested that cognitive style shouldbe considered by other teachers.

5. Participating teachers enthusiastically supported theconcept of cognitive style mapping.

These results must be carefully interpreted in light of thediscussion of the validity of the measures at the beginningof the chapter, p. 19.

51 6_0

CHAPTER 5

SUMMARY AND RECOMMENDATIONS

The two-year project was an attempt to implement andevaluate a set of procedures designed ;o aid educators inindividua;/zing instruction at the postsecondary level. Theevaluation focused specifically on procedures developedby Dr. Joseph E. Hill of Oakland Community College inMichigan for measuring and mapping each student's edu-cational cognitive style and using the resulting maps toindividualize educational programs to accommodatestudents' styles.

The specific project objective were:

1. to devise, develop, field test, and refine procedures forthe measurement of the various dimensions of Hill'smodel of educational cognitive style;

2. to train faculty and staff fpstsecondary educationalsettings to understand the conceptual framework ofeducational cognitive style;

3. to assist these trained faculty members in developingand carrying out plans for implementing cognitive stylemapping procedures in making educational decisions intheir individual settings; and

4. to plan and carry out systematic and objective evalua-tions of the effect of the'cognitive style mapping proce-dures and data on educational outcomes in each imple-mentation setting.

Dsuring year one of the project, the groundwork was laidfor the implementation and evaluation by developing themeasures of style, training faculty, developing specific im-plementation plans, and developing evaluation plans. Dur-ing year two, the implementation plans were put intoaction, and the evaluation data were gathered. Althoughthere were only six implementation and evaluation set-tings in two Michigan institutions, the project results willprovide direction to future educational cognitive styleresearchers. Consequently, it can be concluded, as sum-marized below, that the project achieved its objectives.

Training

Training objectives established at the outset of the trainingphase of the project called for participating staff at MCCCand. MSU to acquire specific knowledge and skills thatwould allow them to implement cognitive style mapsappropriately in their own educational environments. A

53

variety of training procedures were used over a period ofsix months; seven faculty members achieved sufficient pro-ficiency in the mapping of knowledge and skills. Multifac-eted evaluations of the cognitive and affective outcomes oftraining suggested that this phase of the pioject was car-ried out very effectively.

Developing Measures of Style

Objectives were also estabfished at the outset of the pro-ject for the selection, field testing, and evaluation of pro-cedures for measuring 32 dimensions of educational cog-nitive style. Participating cognitive style experts used avail-able ACT measurement instruments and adapted OCCinstruments to assemble ''a battery of cognitive style mea-sures. These experts also established the face and contentvalidity of the measures and the dimensions of style. Thebattery was pilot tested during year one. ExcessiVe admin-istration time during the pilot testing resulted in revisionsthat shortened the instruments for use during 'the year twoimplementations.

Technical analysis of the data resulting from the secondadministration suggested that the revisions yielded greaterefficiency. The analysis also suggested that the variousmeasures of style dimensions were sufficiently reliable.However, the validity of the measures as indicators of thevarious style dimensions could not be demonstrated

Intercorrelations between independent mea-sures of the same style dimen'sions using the project bat-tery and the most current OCC cognitive style batteryyielded chance correlations throughout, suggesting thatcorresponding measures were not tapping the samelearner characteristics.

, In the research on educational cognitive style completedto date, little or no data have been presented to demon-strate the psychometric adequacy of the measuresemployed. In those instances where validity is aodressed, itis nearly always content or face validity based on expertjudgment. Future research and development efforts focus-ing on educational cognitive style should give seriousattention to the development or adaptation of measuresthat are demonstrated to be validand reliable using allre!evant scientific and psychometric standards as criteria.1 his work must be completed before implementation andevaluation of the impact of style can be generalized across

.

settings or measures.

61

Assessing Impact

The final major objective of the project was to conduct athorough and systematic evaluation of the impact of edu-cational cognitive style on educational outcomes. Thisevaluation was conducted through the implementation ofcognitive style mapping procedures in a variety of post-secondary educational settings. Participating educatorsplanned for and implemented the experimental proce-dures in their regular classes. In addition, they planned forand supervised the,collection of evaluation data in thoseclasses. As a result, some useful information on the rela-tionship between cognitive style mapping and a variety ofeducational outcomes was generated.

Summary of Results

Seven implementation studies were planned and six werecarried out in two postsecondary settings: Macomb Coun-ty. Community College and Michigan State University.These addressed several important issues pertaining to theimpact of using cognitive style mapping results in practicaleducational settings. Results from all of the sites thatcarried out their research plans are summarized below.

Foil,- implementation studies were carried out at MCCC. Ina science class, retention rate ar d final grades wereenhanced by the individualized prescriptive treatmentwith or without cognitive style mapping. The true impact ofCognitive style information is not clear due to several con-founding factors in the design and its execution. In ahumanities class, a series of stepwise regression proce-dures using student cognitive style data and match scoresto predict final exam scores (using four probable modes ofunderstandinglecture, group discussion, media, andoverall course) showed little relationship between thesecognitive style variables and course outcomes. Specialneeds students who participated in the fall 1976 orienta-tion program (experimental, cognitive style) tended tocomplete more course hours than did the previous year'sorientation group. However, a lower grade-point averagealso resulted for the group that had its maps interpreted,and few students showed hoped-for growth in the basicskills elements of the map. Attempts to generate consis-tent collective maps across a variety of methods failed inthe physical education site. The large amount of within-group variance relative to between-group variance wascited as the principal explanation for this outcome.

Three implementations were planned at MSU and twowere carried out. Participants at the Counseling Centerwere unable to implement their research plans. F.duca-tional psychology classes were the context for ar investi-gation of the relationship between student and teachermatch scores on academic and attitudinal outcomes. Littlerelationship was found. In the graduate education curric-ulum class, students were sorted into two instructionalgroups to match their Cognitive styles. Partly because of thelarge amount of within-group variance on the key map ele-

ments used in group identification, it was found that thegroups did not differ reliably on key map elements.

Overall, the six studies did i;ot establish the existence ofsignificant relationships between the use of cognitive stylemeasures and data on educational outcomes. These pro-ject results were supplemented with firsthand reports ofparticipating teachers and students; teachers were inter-viewed after implementation and students were adminis-tered an attitude survey. Both groups had generally posi-tive attitudes about the experience.

Review of Limitations of the Research

Because of the nature of the evaluation contexts, some im-plementation problems occurred at nearly every site.These problems sometimes impacted the internal andexternal validity of the results.

In the science implementation, the instructor may havebiased the control treatment by empirically (subjectively)mapping students. For some control group students, then,the prescriptions were based on empirically mapped cog-nitive styles rather than randomly assigned. Further, thosest,...lents who received prescriptions based on their cogni-tive styles were permitted to self-select any other materialsthey wished. Finally, the results are further confoundedbecause students were given information about theresearch intent and expectations.

The students in the humanities class were subjected tonearly two weeks of testing and interpretation of testresults. Although- the instructor feels that more efficientinstruction was possible because of the test results, many ofthe students expressed concern over the loss of class timeto testing.

Students in the special needs program were not reiuiredto take the cognitive style test a second time in December.Although special arrangements were made to encouragestudent participation, less than one-third were retested.The characteristics of the volunteer sample are unknown,and thr generalizability of the results is definitely limitedby the small retest sample size.

Originally, the physical education implementation was tocontinue into a second term. The plan was to select poten-tial unsuccessful students, give half of them prescriptiveinstruction, then compare the performance of successfuland unsuccesful students. However, this plan was'not fol-lowed through, because the instructor accepted an admin-istrative position in the college.

The educational psychology implementation proceeded asplanned. It appears that the data resulting from this studywere relatively free of confounding factors.

The graduate education curriculum implementation alsDproceeded as scheduled. However, the small' sample size in

54 1 4

this setting and the lack of control data impacted theresearch design and the strength of the conclusions.

'The limitations of the implementation at the MSU Coun-seling Center were the most serious of all. The directors of.the Counseling Center who developed the implementa-tion plans were unable to obtain the necessary coopera-tion trom the coinselors at the center to carry out thestudy.

Finally, an additional factor necessitates careful interpreta-tion of the impact research results: the validity concernsidentified in the previous chapter. If the instruments usedin the study are not valid measures of the learner variablesthey are said to measure, then use of the map data couldwell result in erroneous decisions.

Collectively, these problems have the effect of reducingthe gerieralizability of the findings and the breadth of theconclusions that can be drawn. However, they do notentirely negate the value of the research. Although theymake it difficult to study cause and effect links betweenCognitive style mapping and educational outcomes, theydo represent acceptable analyses of the relationshipbetween cognitive mapping and educational outcome.This is an important distinction because, in addition to

6J55

revealing the impact of methodological problems, it alsohighlights the true value of the project.

Recommendations for Future Research

The five studies conducted during this project do not pro-vide a basis for judging the extent to which educationalcognitive style mapping can impact the outcomes of post-secondary educational experience. To date there has beenlittle hard data to document that students with differentstyles learn better when provided with different instruc-tional treatments. This project did not shed a great deal ofadditional light on this matter. Until such a causal link isestablished there is little reason to believe that the learn-ing characteristics represented in the cognitive style mapare any more appropriate or useful than the variablesincluded in any other profile of student aptitudes, inter-ests, and attitudes.

Establishing such links will require, as a first step, measureswhich reliably and validly measure cognitive style attri-butes. To that end, it is recommended that substantiveeffort and resources be devoted to the development oftechnically sound instruments for assessing cognitive styleelements.

APPENDIX

EVALUATION ADVISORY PANEL 'REPORTS

Year One Report

This is the first report to be filed in writing by the AdvisoryPanel for the Cognitive Styles Project. The panel members,whose names appear below, have made two site visits toACT headquarters in Iowa City: one in the fall of the yearand one in the spring. The members have been suppliedwith the proposal as funded, a supplementary documentregarding evaluation of the proposal, a copy of the renewalproposal, and publications from OCC related to cognitivestyles. During, the visits, the panel has had an opportunityto hear reports from Dr. Joseph Hill and Ms.' LindaHenderson, who had the major responsibility for carrying,..ut the activities described in the proposal. Reports havealso been received from staff members assigned to the pro-posal by ACT; an additional report was received from Pro-fessor Glen Lahti, Department of Humanities, MacombCounty Community College.

The format of the panel members' consultancy has been toreceive from the project participants written and verbalreports of plans and progress made thus far. The panelmembers have also reviewed the test materials developedby ACT staff members, but it is important to emphasizethere have been no direct site visits to the colleges wherethe program is being carried out.

An additional resource for- the panel's deliberations hasbeen the participation of Dr. William -. Taylor, Dean ofStudents at Polk Community College, where the CognitiveStyles Program has, been actively utilized for nearly fouryears. Dr. Taylor has the most long-standing acquaintancewith the content of the project, and the other members ofthe panel depend on him for background comment. Noneof the remaining panel members can claim intimateacquaintance with the literature or projects which havegrown out of .)r. Hill's work. '

It is thehOpe and intention of the panelists that this reportwill be found constructive in its comments and encourag-ing to the participants and to the Fund. The panel agreesthat there is ample reason to hope that the cognitive stylesapproach represents an improvement in concept whichmay lead to possibilities of implementation of instruc-tional strategies which heretofore have not followed from,.other similar kinds of testing efforts. The panel alsounderstands that the Fund is devoted to development andimplementation supports, rather than the support of theo-retical research. Within these constraints, we hope to offer

S7

suggestions which will prove beneficial to all concernedwith the projed.

Formative Evaluation

Dr. Hill has suggested that evaluation of the extent towhich the project is achieving the goals it set for itself inthe first year be viewed as formative, by which he meansdevoted to an assessment of whether or not the proposedactions are, in fact, being undertaken. Following comple-tion of these actions, the assessment of the project out-comes will constitute a final evaluation which must takeinto account the project as finally conducted. The panelaccepts this distinction and wishes to offer the followingformative comments.

A. The project plan calls for the selection of participant`teachers who will learn the cognitive styles system andthe arts of cognitive mapping. They will then modify

.their instructional plans to accommodate student stylesidentified by the information so gained and carry outtheir teaching plans so that assessment of student gainscan be made. The original intention was that the com-pletion of the teaching plan be achieved by.the begin-ning of the summer term. However, the decision wasmade to rely on teacher volunteers rather than personspaid a small stipend to take part. In addition, it was dis-covered that there was an opportunity for participationby members of the School of Education at MichiganState University. These events have led to a slowing ofcompletion of the schedule as planned. On the otherhand, the addition of Michigan State University mayprovide a unique and beneficial data source for theevaluation work. Teacher training has begun, and theparticipants are currently involved in the creation oftheir implementation plans.

It is dear to the panelists that it will be important for theProject Director to state the criteria for accepting or, ifnecessary, rejecting these plans as submitted. We sug-gest that it might be wise to evaluate the plans accord-ing to criteria identified by those who have a soundsense of the cognitive styles approach. It might be pos-sible to grade or rate each teacher's plan so that, in theevent some are quite superior to others, subsequentanalysis can be performed with the estimate squarely in

mind. It is also important that whatever criteria are usedto'assign such ranks or-grades-be objectively stated sothat they, too, can be subsequently reviewed during thesummative stage.

A second issue we wish to emphasize is the necessity tourge the participating teachers to state as clearly as pos-sible which aspects of studeni-Performance within theircourses they hope to see improved or strengthened bytheir teaching plan. It may be wise to urge upon them aminimum number of within-course examinations,papers, and other assessmentse.g., not fewer thanfour separate measuresso that adequate comparisonsacross courses may be made.

A third recommendation is that occasional course visi-tation be made in an effort to provide independentassessment of the recognizable degree to which the im-plementation plan of each teacher is, in fact, being met.This might be best accomplished by a modified criticalincident technique whereby the teacher is urged toinvite an observer to come and see a course sessionwhich the teacher feels is an especially good example ofthe type of instructional effort being made.

B. The instrument development schedule established forACT is clearly being met. The materials are now in hand,ready for use as scheduled. The preliminary statisticalanalyses of these new instrumentsdealing in particu-lar with reliability coefficientswill benefit from con-tinued refinement with special attention given to gen-erating reliabilities for the item cutoff strategies beingadopted for use in the computer scoring procedures.This second level of reliability assessment should proveValuable when the time comes to review those itemsand batteries which may prove to have least predictivevalue. Otherwise, it is the panel's judgment that theinstrument development schedule is being met accord-ing to plan.

C. In order to use the instrument, participating teachermust be trained in the art of cognitive mapping. or,panel feels it important that records be kept of the j

, formance evaluation made of the degree to whichteacher displays an acquaintanceship with the s. stemand an ability to understand such mapping. In p.,rticu-lar, we feel it important to elicit from the parti ,p;,(ingteachers their personal statement of how the r ..remativemapping made of their class group has alter ,d or pro-vided the base for modifications that they a ..ke in theirteaching plan. We are suggesting here hat a,- theteachers proceed through' the teaching ' m, they beencouraged to keep a journal or diary of they'areusing their knowledge of the cognitive style scores.

Summary

The general assessment that the pan& wishes to offer of theprogress toward first year objectives is that, although the

timing has necessarily lagged because o( .-/tarri,';es made inthe plan so as to take advantage of new 1.7,,,portunities, theobjectives are achievable. However, a :aleguard againsthaving to proceed to implementati' before all of Ihetraining time thought necessary hat (-:en spent, we urgeadoption of these additional corn', ..ir,ous evaluation pro-cedures so that subsequent variP z.,1 in perfor:nance mayturn out tobbe predictable 'from :".rese new assessments ofachievement in the training p' Ye. We find, too, that theprocedures established in the 1,iJt proposal are being hon-ored by all parties, and that r .a capacity and willingness toprovide improvements in tl.le current evaluation plans arevery much in evidence. TI; e chief deficiency that we senseto date relates to the cot pletion of the summative assess-ment design prior to he seed--ri-dear implementationphase. It is the panel's !.J,1.1gment that at least the outlines of,the design should be n place and agreed to by the partici-pants prior to the be;inning of theactual implementationphase. Under the p.t.;ent schedule, there is a strong possi-bility that this will ritY be achieved, and we urge that ACTstaff make the . nie:ement of this design'a 'high priorityeffort. Overall, ivy find the project on time and on coursewith some ex, iiing possibilities in prospect.

Addendurr

We are ,veil aware that FIPSE does not support basicresearc' This project focuses on the application of a sys-tem v.' ,ich has been several years information; upon itscomrlcdon, we may have evidence that the systemdes( yes further development and dissemination. How-evr c, we think it important that the funders realize that ino for the system to achieve legitimacy in the eyes ofr .;;chologically trained persons, who will most certainly beasked in time to pass judgments on further proposals for itsimplementation and- Use; it will. be necessary to providepotential users with a brief document, perhaps no morethan five pages, which summarizes the theoreircal frame-work out of which the cognitive styles thinking has come.This piece should also provide a significant bibliographychosen to provide the reader with an opportunity toexplore the theses and publications which will provide asense of the research evidence. 'At some point it will.benecessary to conduct analytic studies of the instrument soas to satisfy the normal and responsible q.restions whichwill be posed regarding the cognitive styles instrument and,its construct validity as seen in the con.ext of other similarresearch. This is an effort which can be incorporated inpart into the present project if other test data iron: the par-ticipating students can be made available to ACT. If thereare difficulties securing such material, it will simply mean apostponement of the time when such comparisons willhave to be made.

A second,kind of observation may also be made regardingthe issue of s;isterns evaluation. The project essentiallyentails the beginn:n7 of what will, eventually have to be anambit.f.rus effort to evaluate tile us( fulness of an entire sys-tem. The procedures being followed in the project are pro-

toty.41 ai, and )5'.01,/ looked on preliminary explord-lory expeciericetc..syszem buildiq. This project will shedcon.sijerab!c lifify; on !rtgy.,c the systnn c.,!vItion must pro-ceed. IA 00.1, e.,Jaltotion of the svA.em for its ovr. sakemust b,e cutied Thus. the,presehtation cothi-tive inttrurric,;.6,1t, tht? training of persons in its use, thea !..)tori of knol.L,N.'i';E:e. generated by the tes'i ba,rter9 tO

2ractrv.-.) he &sign of impt....pria%e assess-Iteehri:4q.4es aie yi6.1c1 to vireful and

,,pie..neirts is k.,eing pc.creva%.1.tat{ttf .n he 2,.:.ii;Phr.p.T1:101;'1, rtht.....h

more formal evalurtiOn project of ,!./unin systez!, upptold,,'will have to be carried out.

This brings Us to our final observation. We are impressc Jwith the magnitude of difficulty the project members facein achieving the objectives they have set for themselves.The two-year effort for which funding is presently in placewill, in our judgment, require repetition before suffic'entexperience is gained to provide relatively clear andunequivocal statements of the values which can be real-ized. We thilk it important to emphasize theopen-endedcharacter of the current project and the extent to whichthis project is part of a continuing effort to determine theusefulness the cognitive styles approach. It is for this rea-fon that we wish to reemphasize the need for ACT staff tocapture unobtrusively as much supplementary informa-tion as possible about etch and every phase of the projectas it 'progresses. This t} pe of informazion, examples ofwhich are given above, can then provide in the summativeevaluation stage first estimates of the effectiveness of vari-ous elements in the system which, strictly speaking, are notthe focus and object of Study.

We hope these suggestions and observations prove usefulto members of the project and the Fund for the Improve-ment of Postsecondary Education.

End of Project Report

Implicit in the FlP5E Project on f.ducational Cognitive Stylewas a "field test" of the Hill model at Oak/land CommunityCollege which would embrace the refinement of mea-surement Atruments, the training Of prn'tessienal staff ineducational cognitive style mapping, and the develop-ment.and.implementation of pr4ects ir, vat;ious settings atMichigan State University and at Macomb County Com-munity College.

Project Design Requirement,

The requirements of the design used in the educationalcognitive style project can be viewed from a systematic per-spective. Essentially, the project consisted of 1) construc-tion of assessment devices; 2) training in profile interpre-

*.e.,:i0f; 31. pedagogical applications: and 4) summative eval-uation of the impact of the applications on teacher atti-tudes and on studer attitudes and performance. Thesecomponents untold in time sequence, which is itselfmat kod oft by deadlines and. standards of competency. Anade.iuote test of the hacic effecti."eness of the cognitive

ir.F.TrArrtent requires (het tech step in the system meeta pplopriate In cases where such standards arerot inet, etc.:7..ni4i.: that vrovison be made icr repeat-ing procsi, urwil !Ai) 5t:o1-3.1rds are achieved,.Only 41'. :it ancl;rds vit. be sure that 'Lhe

it*/ 72r is :iccualiy :Aingthe urL,icci, frorn ?hi perci.,42ctive, :trect;;(:.o, walk-nesses of the effor 474!

Initial Phases of Recruitment and Selection of Teachers,Training of Teachers, and Pedagogical Design

Teachers were recruited for the project by experts in theuse of the cognitive style measures from Oakland Com-munity College and The American College Testing Pro-gram. The incentives for participation were modest, con-sisting of the necessary release time and small stipends forWeekend involvement. Thus the primary incentive for par-ticipation was the teachers' interest in the subject itself andtheir desire to explore its possible usefulness. Most of theteachers recruited were apparently well suited to the pro-ject, though it does seem clear that a wider applicant poolwith more powerful incentives would have produced agroup of teachers more knowledgeable about the field andbetter able to participate in the project.

Teacher training was carried out in two- to three-hourstudy sessions, with occasional opportunities to practiceprofile interpretation. Some participation was lost at thispoint; the experts selected the final seven participants byevalu'ating their ability to provide adequate interpretationof student profiles. Testimony of the participants, as well asthe report of the project staff, indicates that the range"ofperformance vis=a-vis the. minimum standard was consid-erable. There is a 5eneral sense that, had time and dead-lines permitted, more effort could have been profitablydevc,ced to this phase of the project.

Given the selection of the teachers and the conclusion thatthey had at least a minimum mastery of the instrument, thenext phase was the design of course applications of peda-gogical technique to alert- the profiles of the students,and/or to carry out certain clearcut instructional modeswith an eye to testing the effectiveness with which stu-dents with different profiles resp6nded to them. 'Expertsreviewed these designs to check on their theoreticalplausibility and suitability. Thic was a fairly demandingphase of the program for the teachers, and the combina-tion of monetary and intrinsic incentives was used. it is evi-dent that for many teachers this was the most enjoyableportion of the project, and, once again, they would haveliked more consultation and more time to gain confidence

59t)t

that the pedagogical modes selected were well designed. Itis our impression that the justification of these designs wasinforma'ity carried out in conference between the teacherand an expee, and not subject to any systematic reviewagainst some common set of standards. Ideally, in a care-it:Ily-designed effort, this would be the moment whencareful and intense review would take place, and thosedesigt, oroposalc found to be weak would be r ?turned forfurther

With this phase of the projec cnncluded, it then remainedlo implement the instructional tic:.!;;;--.including testingthe stutletlts i.%rvi informing them, where called for, ., theirprofile. findings. 1%.?:!!-_- experts periodically monitoredthe implementation. Subseg.Jently, teacher 2nd studen;estimates were gathered of the extent to which theinstructional effort was influenced in the use of thecognitive style assessm,...nt.

Implementations of this kind are always -art ietl :;v1the constraints defined by the larger system w;thir. whichthe experiment is taking place. This particular design didnot call for any accommodation by the larger system;rather, the instructional practice was wholly conventionalfrom the perspective of the institutions. This means that nospecial types of instruction had to be attempted beyondthose normally tolerated in the system, and no radicaldemands were plated on the system. It means, too, that theteachers and the students were operating within the usualand conventional expectations and incentives which moti-vate teacher and student outcomes. This is laudablebecause it minimizes the possible erects of "specialness"that often produce their own outcomes. Monitoring of t: reimplementation could only be occasional. Ordinarily, itwould be important to have frequent and widespreadobservation of the ways in which the teachers attempted toimplement their knowledge of the students' cognitivestyles.

The final stage of the desigo was the collection, collation,analysis, and interpretation of the outcome measuresselected to test the effects of the instruction. In this case,the measures as collected were primarily attitudinal andintrospective, with only a few objective performance mea-sures available. Obviously, a wide variety of measures werepossible. We might have chosen that particular teacherwhose efforts seem to have set the highest standards ofperformance to the design and have treated that class bythe case method: learn as much as possible about it and doour best to infer what may have occurred there. Or, wemight have used a multi-factorial design in which certaintreatments were systematically varied across others so thatfactors having social importancesuch as norm groups,subject matter, teaching method, and perhaps economicefficiencieScould be assessed as contributory to cogni-tive style effects. In this project, neither the case methodnor the multi-factorial design could be used. The sevenclasses for which data could be obtained may best bedescribed as providing us with a mixed picture which shed

60

some light on specific possibilities but which, when takentogether, were not sufficiently systematic to provide' aclearcut interpretation. The evaluative assessments as col-lected are reasonably usable, and do provide a rather clear-cut interpretable outcome. However, the causal analysiswhich we might seek is more difficult to complete s., ice theproject provides limited performance outcome data, andattitudinal expressions are not supportive of the inferenceof cognitive style effects.

The ACT Measurement System and its Analysis

1. The ACT staff, in cooperation with Dr. Hill, built a mea-...;;;.-..rnent system that seemed to reflect the cognitivestyle con:.*ructs previously embodied in the OCC instru-ments. It wa, reported that Dr. Hill seemed satisfied thatthe resulting ,';CT system was an adequate, perhapseven improved, r,.:_..ourement s3.mem.

7. 46 part of data collection in the varioui implementa-tion rtes, ACT gathered a variety of data bring on theevaluation of The instrument. However, unforeseen anduncontrollable .E.4,:i.(115 in some sites forced the compro-mise of data collection Hence, a full evaluation ofthe ACT instrument and its Felai.i:.En to the OCC instru-ment is not yet in hand.

3. Within the limits of the data collected, ACT has faith-fully endeavored to examine from several viewpointsthe educational and psychometric qualities of the mea-surement system. The process is one of detective workin a complex data set, rather than merely the applica-tion of routine psychometric and statistical methods.The ACT staff are to be commended for their thoroughand thoughtful pursuit of the various empirical ques-tions that can be asked of the data. At-this writing, allrelevant data analyses are not yet completed. Data arestill coming in, and further ideas on data analysisare stillemerging.

4. With respect to the evaluation of the ACT measure-ment system as an assessment of Hill's educational cog-nitive styles concept, one can apply six separate cri-teria': a) internal consistency reliability of.the measures;b) stability of scores over time; c) interrelationships (orredundancy) among the different dimensions presum-ably measured within or between instruments; d) extentof correlation between like constructs based on differ-ent methods of measurement (e.g., between likeconstructs measured by the ACT and the OCCmeasures); e) predictive validity of style constructs ineducational settings; and f) prescriptive or differentialvalidity of style constructs in such settings, to demon-strate that the constructs are actually useful in prescrib-ing different instructional treatments that benefit indi-

, viduals, relative to, e.g., random assignment totreatments.

5. The complexity of the assessment procedure and con-cepts, however, makes each of these six criteria appli-cable at three different levels in the measurement sys-tem. At one level, one can ask about the adequacy ofeach of the independent base measures in the system.At another level of abstraction, one can ask the samequestions about ,the map element scores derivedthrough linkage rules frOm the base measures. This iscomplicated by the fact that some linkage rules arecomplex. Beyond this, one can inquire about adequacyand usefulness of the system at the level of the style pro-file descriptions and the educational prescriptionsderived from these.

6. We can now review each of the six criteria, keeping inmind the three-level complexity, to document ACT's

evaluation activities regarding the measurement sys-tem. We shall not try to describe in any detail theempirical results; this is the province of ACT's furtheranalyses and iinal report, yet to be drafted.

a) ACT obtained internal consistency estimates of reli-. ability at the leyels of individual base measures and

some individual 'map elements. Most of these esti-mates show adequate reliability in this sense, thoughsome scales show a level of reliability insufficient forindividual prescription. It should be noted that indi-vidua prescription typically requires a much higherlevel of score accuracy than institutional or researchuses of educa' ional-psychological measurement

h) stab': :; over time is the more significant aspect ofreliability or interest here because, for prescriptivepurposes, style constructs must be regarded as rela-tively enduring characteristics of individuals. The sys-tem, measures derived from previous ACT testresearch may be said to possess stability over time tothe extent that these measures were not significantlychanged in translating them for use in the style mea-surement battery. Unfortunately, the project site thatwas expected to produce some stability data failed torun asanticipated, so data on stability of other basemeasures, map elements, or profiles were notobtained.. Ten individuals who were tested twice inone project show some variation in profile over time,but these individuals experienced insiructional inter-ventions between testings. Thus, the question of,stability remains open, at least with respect to most

.map elements and profile characteristics.

c) ACT has correlated base measures and some mapelements with one another, within and between theAC-T. and OCC batteries. Partial results suggest sub-stantial overlap between some purportedly differentdimensions, but it is not dear if this overlap is of adegree high enough to justify reducing or combin-ing measures. ACT rightly considers such a stepsomewhat premature.

d) One site provided data allowing cross-correlation oflike constructs from ACT and OCC batteries. Thesefail to support the claim that any pairs of measures

from the two batteries actually do measure the sameconstructs. It is therefore doubtful that profile clus-terings produced independently by the two batter-ies would place individuals in the same categories,but these analyses are as yet incomplete. ACT stillneeds to complete various statistical checks on theseissues.

e) Several sites yielded predictive validity data on theACT battery. Analyses of these data are incomplete,but preliminary evidence suggests that educationaloutcomes are not substantially 'predir.ed by styleconstructs. Some relationships may l e promising,however, and are being pursued.

f) Prescriptive (differential) validity data were notobtained. One site 'altered the planned conduct of'itsstudy. While this may have had other virtues, it com-promised the value of the data on this point. Anothersite yielded data that may provide some evidence ofdifferential. validity, ;4-71-ling further analyses.

7. Within the limitations of the data, then, ACT hasattempted to answer questions relating to the aboye sixcriteria as applied to the different levels of assessmentprovided by the system ,f style constructs. The limita-tions on the analyses possible are a function of prob-lems typically faced in field implementation and evalu-ation projects. It is unfortunate that the resourcesneeded to support a stronger, more comprehensive testof the measurement system and its associated style con-structs were not available. ACT has thought ahead in thisconnection and has plans for more systematic researchon and development of this system. This is to be com-mended. What seems required now is a thorough exam-ination of the concept of the system as it might be im-plemented in an instructional situation where compre-hensive and controlled evaluation studies could be car-ried out. it is hoped that ACT will be able to pursue thisline of work more intensively.

This review of the strengths and weaknesses olthe projectneeds to be understood as illustrative of the effects of seri-ous time.and money constraints. The project was ambi-tious, and the participants are to be commended fortime, intelligence, and effort they devoted to it. Hov.ever,the requirements of the project clearly were greater thancouldbe met under its suppOrt terms. It remains unfortu-nately ti uethat we continue to believe that such educa-tional efforts are relatively inexpensive and unsophisti-cated in their dernands. Actually, great patience, care, andtime , must be extended to implement and assess a pro-gram in education. As yet, it does not appear that this fact issufficiently appreciated.

61

Respectfully submitted,

Robert Birney, Hampshire Collegelohn E. Roueche, The University of Texas at Austin

William F. Taylor, Polk Community CollegeRichard Snow, Stanford University

Documents

UM:UO.551 :Labium - ERIC · Correlations between Predictor Variables and Final Exam Score: Course Approach 26 Table 4.12 Comparison of Special Needs Students' Academic Progress for