Nonbiased Assessment: A Supplemental Approach

I

CHAPTER 28

Nonbiased Assessment: ASupplemental Approach

JAMES C. KAUFMA.N ,

STANDARDIZED TESTS are often criticized as being biase~, ~":~these critici~ms c~"lcome in two main forms. A common lay approach to cntiazmg tests as bIased 15

, to point to significant differences that occur between males and females andamong ethnic groups on various tests of aptitude or ability. Indeed, a wide variety ormeasures of intelligence and ability have shown lower scores for African Americans andLatinos (see Loehlin, 1999, for an overview). Standardized tests such as the SAT Reasoning Test (SAT),American College Test (ACT), Graduate Record Exam (GRE), andAdvance Placement (AP) exams show similar results (Camara & Schmidt, 1999;Morgan & Maneckshana, 1996).

The differences between males and females are not as large for intelligence tests as

the differences found among ethnic groups, although males tend to obtain higher scoreson tasks involving mental rotation (Masters & Sanders, 1993) and females tend to obtain higher scores on verbal tests (Hedges & Nowell, 1995). Males tend to outscore females on the SAT,GRE, and Graduate Management Administration Test (GMAT), particularly on the quantitative sections (Coley, 2001).

Psychometric approaches to bias in testing take a more sophisticated view of theproblem and do not accept the view that the fact that two groups may perform differently on a mental test means the test itself must be in error or biased. Current approaches statistically evaluate content that may be inappropriate because it unfairlyfavors one group over another. Methods are commonly applied as well to determinewhether different constructs may be measured across nominal groups by the same test(e.g., a test may measure verbal ability in Caucasians, but may be measuring something quite different in a Latino population; Reynolds, 2000; Reynolds, Lowe, & Saenz,1999). These are only a few of the recent methods of considering bias in assessment;

The author would like to thank Gwyneth M. Boodoo for her many discussions, suggestions, and input onthis topic and John Baer for his invaluable comments on an early draft of the chapter.

824

Nonbiased Assessment: A Supplemental Approach 825

see Reynolds as well as chapter 27 of this volume for more detailed presentations ofrurrent methods for studying bias in mental testing.

One approach to seeking out nonbiased assessment is to supplement traditionaliassessment with additional measures of constructs that may be influencing a scorei.im a traditional test of ability or achievement. An example may be seen in creativity.~Someresearchers have proposed that differences on some IQ or achievement subtests,isuch as those involved in remembering the details of a story, may show larger African:American-White differences in part because African Americans approach the task dififerently (Heath, 1983; see Manly et al., 1998). This theory argues that Caucasians ap'proach the task as the test-makers intended-by trying to memorize as many appropriate details as possible and sticking to the presented story; and that, in contrast,African Americans may put more emphasis on telling the story creatively.

This difference in style would be consistent with Shade's (1989) theory of an Africanlunerican cognitive style. Her research with cognitive-style tests found that African

. Americans were more likely to be spontaneous, flexible, and open-minded. In contrast, Whites were more regulated and structured. Open-mindedness has been found

.to be highly correlated with creativity, regardless of the measure used (e.g., Kin~ McKee,·Walker, & Broyles, 1996;Wolfradt & Pretz,2001).

Yet, if there is a tendency for African Americans to approach a straightforward taskin a creative manner, two negative outcomes can occur. One, as mentioned, is that theirperformance on tests may suffer because their answers are different than the intendedanswers. Another possible negative outcome is that African Americans will be furtherpenalized for creative behavior. Baldwin (1985,in press) argues that creativity in AfricanAmericans can often be mistaken by teachers and other authority figures as unrulinessor disruptiveness. In addition, there are no traditional evaluation tools that assesscreativity-related behaviors.

When a test measures something different than what it was intended to measure forspecific groups, then it may be considered a biased instrument. A measure is fair to theextent that the score includes only those variables associated with the construct beingmeasured as well as random, error variance-in other words, a measure is fair to theextent that it minimizes systematic error in true score estimation as function of groupmembership. One possible way to work toward nonbiased assessment would be toadminister additional measures that may tap into other aspects of ability that are notassessed by typical components of the standard examinations common to school psychological practice. If measures of creativity, for example, are used to create a fullerpicture of a student, then these measures can also help minimize errors in decisionmaking regarding such students. This possibility for reduction in error can be conceptualized in two very different and complementary ways:

1. If the examiner is trying to understand a child's overall cognitive abilities, butonly a narrow range of relevant abilities (e.g., those measured by traditional IQtests) is being assessed (however fairly), then the ability of the examiner to understand the child's cognitive functioning is limited (but in different ways for different groups). If some abilities are more relevant to some cultural settings thanto others, the abilities missed will have differential impact on understanding children from certain cultural settings, thus creating bias in the assessment process.

2. Even if one is trying to assess only a limited range of a child's cognitive abilities(e.g., IQ), but the available tests are believed to be biased, it may be possible to

826 TESTING, ASSESSMENT, AND INTERVENTION ISSUES

correct some of the systematic error present if other cognitive abilities are testedthat are believed to influence scores adversely on other tests in the examination.If, for example, scores on a valid and reliable test of creativity could be shown tocorrelate with some systematic ethnic bias of IQ tests, then perhaps the creativity test scores could be used to attenuate any systematic error on the IQ tests.

This chapter will focus on creativity as an example of a construct that can supplementtraditional assessments. Creativity was selected as an example because of the manystudies that look at creativity, ethnici~ and gender, and because there have been severalinvestigations that have examined creativity and learning disabilities. This researchwill be discussed, in addition to an analysis of current and possible future measures.

WHAT IS CREATIVITY?

Although there are some who might argue that "creativity defies precise definition"(Torrance, 1988, p. 43), there have been a number of attempts to define the constructof creativity. One commonly accepted idea is that creativity is the ability to be both innovative and appropriate to the task. Appropriatel1ess is often identified as usefulness,and a component of high quality is also sometimes mentioned in attempts to definecreativity (Baer, 1997; Sternberg, 1999; Sternberg, Kaufman, & Pretz, 2002).

The Torrance Tests of Creative Thinking (rrCT; Torrance, 1974) examine four typesof creativity: fluency, flexibility, originality, and elaboration. These four types are examined in the verbal and figural domains. Fluency is the ability to produce a greatnumber of ideas; flexibility is the ability to produce many different types of ideas; orig

inality is the ability to produce the most unusual ideas; and elaboration is the ability todevelop these ideas.

Finke, Ward, and Smith (1992) focus on the components behind creative cognition.They propose a Geneplore model with two phases: generative, in which a variety ofpossible ideas are produced, and exploratory, in which these ideas are then evaluatedand interpreted.

Further distinctions are drawn between the creative process, the creative product,and the creative person. Both Bruner (1962)and Jackson and Messick (1965) focused onproducts, arguing that those products that are creative will invoke specific responsesin observers. Amabile (1996)proposed an operational definition of creativity that statedthat a product or response was creative insofar as "appropriate observers independently agree it is creative" (p. 33).

Boden (1994) focused on the creative process, asking "what generative processesactually went on its production .... A merely novel idea is one that can be describedand/ or produced by the same set of generative rules as are other, familiar ideas. Agenuinely original or radically creative idea is one that cannot" (p. 78). Mednick (1962)also examined the creative process, concentrating on how creativity is based in the associations that people make between different ideas.

Person-centered theories of creativity often focus on the congruence of many variables needed in a person to aid creativity. Amabile (1982,1996), for example, includesthree key variables as being essential for creativity to occur: domain-relevant skills,creati\ity-relevant skills, and task motivation. Domain-relevant skills include knowledge, technical skills, and specialized talent. Creativity-relevant skills include an appropriate cognitive style, knowledge of heuristics for novel idea generation, andan apFropriate work style. Task motivation includes attitudes and one's perceptions


of motivation. In a similar vein, Sternberg and Lubart (1996) include six personal resources that are needed for producing creative work: intelligence, knowledge, personality, environment, motivation, and thinking styles.

What can be synthesized from these many different interpretations? The idea thatcreativity involves an aspect of innovation and appropriateness appears consistentlythroughout many different conceptions and approaches. One caveat is the importanceof being specific when measuring or studying creativity as to whether the focus is onthe creative person, process, or product.

MEASURING CREATIVITY

Perhaps the most common creativity assessments are psychometric tests. The popularTfCT (Torrance, 1974) includes both verbal items and figural items. Verbal items, forexample, can involve showing participants a picture or a product and asking a seriesof questions about it. Figural items, for example, can involve presenting an idea (thecaterpillar with too many legs) and then asking participants to draw afigure based onthe idea. The test is scored using a rubric that rewards unusual responses. Guilford's(1967) divergent thinking measure is similar, allowing respondents to generate elaborations of a given shape, or hypotheses about different outcomes of an event (e.g.,if people didn't need sleep). Other researchers have designed similar tests (Getzels &Jackson, 1962; Wallach & Kogan, 1965); a good review of psychometric tests of creativity can be found in Plucker and Renzulli (1999).

One significant problem with most psychometric creativity tests is that they makethe assumption that creativity is general. A single score is given as a measure of general divergent-thinking ability and it is assumed that these same divergent-thinkingskills can be employed in any area. Yet, there is a significant body of work that suggests that creativity is domain specific. In fact, the processes underlying creative periOTm'i:\l\c:e ID. a.om'i:\ID.s \:\\'i:\t seem c.\ose\~ !e\'i:\\:ea., S'\1c.\\ 'i:\SW!\.tID.'{, "poetry a.l\a. wci.fu\'{,

short stories (Baer, 1994, 1996; J. c.Kaufman, 2001b) or creating different kinds of artwork (Runco, 1989), still appear to be quite different.

A second problem is that most psychometric creativity tests measure only divergentthinking. These measures can be quite extensive and have evolved significantly frominitial attempts to simply count the number of responses. Certainly, for example, theTTCT incorporates both the novel and the appropriate dimensions needed for something to be creative. But divergent thinking is only one piece of the puzzle. Recent theories of creativity (e.g., Amabile, 1996; Sternberg & Lubart, 1995, 1996) have tended topresent creativity as an interaction of many different components, and any test that measures only the ability to think divergently is missing much of what constitutes creativity.

Another method is to ask experts to rate a product's creativity to obtain a creativityscore. Amabile (1996) developed a consensual assessment technique (CAT), in whichraters, solicited for their experience in a domain, provide independent judgments aboutthe creativity of a product. The judgments compare products against each other, ratherthan against an absolute ideal, and the products are judged in random order (Amabile,1985;'Amabile, Hennessey, & Grossman, 1986; Conti, Coon, & Amabile, 1996). Otherstudies have used ratings by teachers (Besemer & O'Quin, 1986) or parents (Runco &Vega, 1990). The products rated range from inventions or devices (Finke et al., 1992) tocollages (Amabile et al.), haiku (Hennessey, 1994), musical compositions (Hickey,2001), pictures written to captions (Sternberg & Lubart, 1995), storytelling (Baer, 1994),and longer poems, stories, and personal narratives (Baer,Kaufman, & Gentile, in press).

828 TESTING, ASSESSMENT, AND INTERVENTION ISSUES---.- ----------

One concern of this method is that raters from different cultures can differ on their

perceptions of creativity (Niu & Sternberg, 2001). Any attempt to measure creatiV·E'performance through the use of this technique would need to ensure that the raterswere balanced across gender and ethnicity. Another concern is practicality-it is fairi"j" .

straightforward to administer a creativity test; it is not as simple to find experts in fu.,""area to judge a creative product. One recent study, however, found that the poetry raf'ings of gifted high school students did not significantly differ from the ratings of ex...perts in creative writing (Gentile, Kaufman, & Baer, 2003). These findings may imp~that using experts may be desirable, but those with an aptitude in an area may sen-eas a substitution if needed. Indeed, many of the experts used by Amabile (1996) werenot necessarily professionals in a given field, but merely were those ·with a reasonableaptitude or interest.

Other methods that are popular in creativity research are less applicable to assessment. Some researchers, for example, study groups such as eminent artists or writers.who are already known to be creative (e.g., Hall & MacKinnon, 1969;Wills, 1984).The..<;;?groups might then be tested or examined to determine any unique traits or characteristics that might be unique to creative people. In a similar vein, historiometric researchinvolves using found data and biographical research to draw conclusions about suchtopics as awards (J. C. Kaufman,2001a), length of career (Crozier, 1999), age at different accomplishments (Simonton, 1988), influence, productivity, and international recognition (Ludwig, 1995).

CREATIVITY, INTELLIGENCE, AND ACHIEVEMENT

Creativity has been found to be significantly associated with measures of intelligence(especially verbal intelligence), yet this association is not a particularly strong one (sei2

Barron & Harrington, 1981). Creativity's correlation with IQ is maintained up to a certain level (often reported to be approximately an IQ of 120), yet the relationship is notstrong at higher levels of ability (Fuchs-Beauchamp, Karnes, & Johnson, 1993;Getzels& Jackson, 1962). In fact, some have hypothesized that in extreme cases, a very highIQ individual may not be able to communicate his or her ideas (creative or otherwise}in an effective manner to other people (Simonton, 1994).

Scores on the TTCT have been found to strongly correlate with grade-point average(Altman, 1999). However, while the TTCT has accounted for some variance in aca

demic success, other cognitive variables have accounted for a higher percentage of thevariance (Niaz, Saud de Nunez, & Ruiz de Pineda, 2000). Similarly, Ai (1999) foundthat teacher assessments of student creativity were related to academic achievement(as measured by student self-report), but that more-psychometric tests of creativityshowed little relation. Flexibility scores were the most related to measures of achievement for males, while elaboration and fluency were the most related for females.A trend in the opposite direction was found by Toth and Baker (1990), in which students with high scores on the TTCT were less likely to be successful in a traditional academic setting.

Creativity is an important if nonessential part of most major theories of intelligences. Guilford's (1967) structure-of-intellect model, for example, includes divergentproduction as one of four operations. Divergent production-defined by Guilford as theability to produce information from memory to meet specific objectives-was conceived "through investigations of certain hypotheses regarding the component abilities most relevant to creative performance" (p. 169).

Many subsequent theories of intelligence have incorporated creativity-related abil-


ities into their structures. Creativity and originality appear in Carroll's three-stratummodel of abilities as part of Gr, or retrieval ability (Carroll, 1993), and in the expandedHorn Gf-Gc model of as Glr, or fluency of retrieval from long-term storage (Horn &Hofer, 1992). These two theories have been combined into the Cattell-Horn-Carroll(CHC) theory (see McGrew & F1anagan, 1998), and CHC theory calls Glr "Long-TermRetrieval."

The PASS (planning, attention, simultaneous, and successive) theory is a cognitiveprocessing theory based on the works of Luria (see Das, Naglieri, & Kirby, 1994, for anoverview). Planning abilities have been found to be related to creativity (Naglieri &Kaufman, 2001). A popular theory that has not yet been translated into a published IQtest is the triarchic theory of intelligence (Sternberg, 1985, 1996). This theory placescreative ability as one of three essential components, along with analytical and practical abilities.

One ideal way to supplement current assessments with a creativity measurementwould be to examine current ability measures and see whether any subtests could beanalyzed or scored for creativity. Yet, although creativity has a dose relationship withintelligence in theory, this relationship does not stand up in practice; creativity is a difficult construct to find measured in ability tests. The closest is the Ideational F1uency(PI) subtest on the Woodcock-Johnson III Tests of Achievement <WI III; Woodcock, McGrew, & Mather, 2001). The WJ III is based on the CHC theory, and PI is designed tohelp measure Long-Term Retrieval (Glr). PI tests the "ability to rapidly produce [sic] aseries of ideas, words, or phrases related to a specific condition or object" (McGrew,Woodcock, & Ford, 2002, p. 568). This subtest is quite similar to Activity 5 of the ITCT,

"Unusual Uses of Tin Cans, in which students are asked to name as many different possible uses for a tin can (Torrance, 1974). Another subtest that is relevant is Verbal F1uency on the NEPSY, a developmental neuropsychological assessment used to assesschildren with developmental difficulties (Korkman, Kirk, & Kemp, 1998). The Test ofVerbal Conceptualization and F1uency (Reynolds & Horton, in press) contains severalmeasures of fluency in multiple domains (e.g., semantic, categorical). Children areasked to generate as many words as they can in a specific category (e.g., animals) andby beginning sounds (e.g., the letter P or 1). Each of these tests one aspect of creativity-in Torrance's framework, the fluency component; in Finke et al.'s (1992) model,the generative component.

Most other major IQ tests do not have subtests that overlap with creative abilities.The Wechsler Adult Intelligence Scale-Third Edition <WAIS-III;Wechsler, 1997), forexample, has no subtests that could be considered to measure creativity or divergentproduction (A. S. Kaufman & Lichtenberger, 2002), nor does the Wechsler IntelligenceScale for Children-Fourth Edition (WISC-IV; Wechsler, 2003), although some subtestsmay be influenced by creativity (A. S. Kaufman, 1994).

GENDER AND ETHNICITY DIFFERENCES IN CREATIVITY

If creativity or other related noncognitive constructs are highly correlated to traditionally conceived intellectual abilities, then the same gender and ethnicity differenceswould also likely be found on these measures as on the common IQ test. Studies of themagnitude or scope of the intelligence studies have yet to be created, but the last 25years have seen many smaller studies take place that form a compelling argument: thatthere are few significant differences in creative abilities across gender and ethnicity.

Most studies have shown no gender differences in creativity, regardless of cultureand background (see Baer, 1999; Barron & Harrington, 1981; Kogan, 1974; Saeki, Fan,

830 TESTING, ASSESSMENT, AND INTERVENTION IsSUES

& Van Dusen, 2001; Wang, Zhang, Lin, & Xu, 1998). The few studies that have foundgender differences in other cultures have been often poorly done and are contradictory (see Lubart, 1999).

There are several studies that show a lack of race and ethnic differences in measuresof creativity. African Americans and Whites did not show significant differences onthe TTCT at the elementary school level (Glover, 1976b; Knox & Glover, 1978) and thecollege level (Glover, 1976a), while one study found AfrlcanAmericans having higherFluency and Originality scores on the Figural subtest of the TICT CKaltsounis, 1974). Inaddition, the poems, stories, and personal narratives by African Americans and Whitesdid not show any significant difference in their creativity ratings by expert judges (J. C.Kaufman, Baer, & Gentile, in press).

Other studies have shown no racial! ethnic differences in the ability to be trained oncreative thinking abilities (Moreno & Hogan, 1976) and no differences in the development of divergent thinking abilities in adolescents from South Africa and the UnitedStates (Ripple & Jaquish, 1982). Of 13 measures of giftedness, measures of creativityshowed some of the smallest African American-White differences (Harty, Adkins, &Sherwood, 1984).Stricker, Rock, and Bennett (2001) found no racial/ ethnic differenceson measures of aesthetic expression (with questions that tapped into creative abilities,such as artistic accomplishments).

Comparisons of Whites and Latinos also provide evidence that some measures ofcreativity show few differences. While three of four TICT Verbal forms showed Whitesscoring significantly higher than Latinos, there were no significant differences on theFigural forms (Argulewicz & Kush, 1984). In addition, the Creativity scale of the Scalesfor Rating the Behavioral Characteristics of Superior Students (SRBCSS) showed nosignificant differences (Argulewicz, Elliott, & Hall, 1982; Argulewicz & Kush). Ho'wever, low-income Latino elementary students scored below the norms on the TICT(Mitchell, 1988), and teachers rated White students as being more creative than Latinostudents, with highly acculturated Latinos receiving higher marks than less acculturated Latinos (Masten, Plata, Wenglar, & Thedford, 1999).

Results are less clear for Asian cultures. Artwork produced by American college students was rated as more creative than art produced by Chinese students by both American and Chinese raters (Niu & Sternberg, 2001). Yet a similar study that comparedAmerican and Chinese drawings of geometric shapes found that the two groups wererated similarly for creativity by both American and Chinese raters (Chen et al., 2002).In both studies, American and Chinese judges tended to agree on which products werecreative and which products were not creative, although Niu and Sternberg found thatthe Chinese judges tended to give higher scores than their American counterparts.

Rostan, Pariser, and Gruber (2002)studied Chinese and American students' artwork,with two groups in each culture: students with additional art training and classes andstudents with no such classes. Ages were distributed into four groups: 7, 9, 11, and 13years old. Chinese and American judges assigned ratings to each group's artw'ork (onedrawing from life and one drawing from imagination). There were no significant differences between cultures from either set of judges, only between art students versusnon-art students.

Other investigations have used paper-and-pencil measures to find that creativitytests usually favor people from Western cultures-but not always. Jellen and Urban(1989) administered a measure of creative thinking and drawing to children from several different countries, and found that in general, Western countries (such as Germany,England, and the United States) scored higher than Eastern countries (such as China

I

I

I

Nonbiased Assessmmt: A Supplemental Approach 831

and India). American college students scored higher on the TTCT than Japanese college students in one study (Saeki et al., 2001), and Americans from five different agegroups scored higher than similar individuals from Hong Kong (Jaquish & Ripple,1984). School children in Hong Kong scored higher on the Figural form of the TTCTthan their counterparts in Taiwan, Singapore, and America, but lower than Germanchildren; on the Verbal form, the results were in the opposite order (Rudowicz, Lok, &Kitto, 1995).Malaysian students scored higher than American, Indian, and Hungarianstudents on one self-report measure of creativity, but American students scored higherthan Malaysian students on a different self-report measure (Palaniappan, 1996).

CREATIVITY AND LEARNING DISABILITIES

In addition to possibly reducing error in estimating the overall cognitive skills of children as a function of gender and ethnicity, creativity measures may also help identifyhidden strengths in students with learning disabilities. Many remedial treatments andprograms incorporate creative therapy for people with mental retardation or learningdisabilities (e.g., Dossick & Shea, 1995), but there has been much less structured andempirical work investigating creativity in these populations.

One study of elementary school students found that those with learning disabilitiesengaged in less task persistence than average children and, as a result, scored loweron the TTCT in elaboration. On the other three components of the TTCT, however, thestudents with learning disabilities scored as well as the group of average children (Argulewicz, Mealor, & Richmond, 1979).Another study of gifted children with and without learning disabilities found that there were no significant differences between thetwo groups on measures of verbal creativity (Woodrum & Savage, 1994). A study oflearning disabled children in self-contained classrooms used a naming task (in whichstudents were asked to name things to eat). In this task, the children with learning disabilities were found to produce more original responses than a matched group of children with average abilities (N. L. Kaufman & Kaufman, 1980).

Cox and Cotgreave (1996) examined human figure drawings by 10-year-old children with mild learning disabilities CMLD)and 6- and 10-year-old children withoutMLD. They found that the drawings by the MLD children were easily distinguishedfrom those by the other 10-year-olds, but not from those by the group of 6-year-oldchildren, implying that while the MLD children may be developing artistic and creative abilities at a slower rate, the development still approaches a normal pattern.

Measures of creativity could lend particular insight into individuals with dyslexia.LaFrance (1997), for example, points to creative thinking as being a partiCularly goodway of distinguishing gifted students who have dyslexia, while Burrows and Wolf(1983) suggest creativity as a way of reducing frustration and improving self-attitudesin dyslexic children. The importance of creativity is consistent with other findings thatshow that dyslexic children frequently excel at divergent thinking (Vail, 1990). It is alsointeresting to note that case studies have been noted of individuals with dyslexia whowere highly creative in the literary domain-the very area of their learning disability(Rack, 1981).

Another learning disability with a strong connection to creativity is Attention-Deficit!Hyperactivity Disorder (ADHD). Several scholars have proposed that the behaviorsand characteristics associated with ADHD are highly similar to creative behaviors(Cramond, 1994; Leroux & Levitt-Perlman, 2000). Such traits as sensation and stimulation seeking and high usage of imagery, for example, are associated with both chil-

832 TESTING,ASSESSMENT,ANDLNTERVEl'\"'TIONISSUES

dren with ADHD and highly creative children (Shaw, 1992). High-IQ children withADHD scored higher on tests of figural creativity than high-IQ children who did nothave ADHD (Shaw & Brown, 1990, 1991). Students 'with ADHD showed specific creativity strengths in fluency, originality, and elaboration on the TTCT (Gollmar, 2001).

Indeed, a study of undergraduates found that having a wider breadth of attentionwas correlated with writing poems that were judged to be more creative, and that distracting noises disrupted creative performance more in those students with a widebreadth of attention (Kasof, 1997). In other words, the same aspects of ADHD that maymake students more prone to be creative may also may them more prone to being distracted and, in some situations, producing lower quality work.

Creativity can even be analyzed '\vith students with much more severe disabilities.Children with autism and Asperger' s syndrome were able to generate changes to an object as part of the TTCT. These children made fewer changes than a sample of childrenwithout impairment, and their changes were more reality based than imagination based(Craig & Baron-Cohen, 1999).But the very fact that creativity assessment was able to addinformation about this population's abilities is encouraging. An additional study compared children with autism and children '\vithAsperger's syndrome (Craig & BaronCohen, 2000). They found that while both groups showed less-imaginative events in astorytelling exercise than children without impairment, children with Asperger's syndrome were better able to demonstrate imagination than children with autism. Thisfinding was also demonstrated with a drawing task (Craig, Baron-Cohen, & Scott, 2001).

In a related fashion, researchers studied human figure drawings in children withDown syndrome (Cox & Maynard, 1998). While Down syndrome children scoredlower than both agemates and younger children, it is interesting to note that their drawings did not differ when drawn from a model or drawn from imagination, whereasboth groups of non-Down syndrome children improved when drawing from a model.This finding may indicate that creative processes may be a comparative strength forchildren with Down syndrome.

One particular learning disability, Williams syndrome (WS), is caused by a lack ofgenetic material that produces the protein elastin. Children vl'ith WS are developmentally delayed and often have profound disabilities in spatial cognition (Bellugi, Lichtenberger, Jones, Lai, & St. George, 2000). Yet children with WS have exceptional narrative skills for their cognitive ability level. Although their syntax was simpler and theywere more likely to make errors in morphology than average children, they also usedmore evaluative devices and-of most interestfor creativity studies-used much moreelaboration in their narratives (Losh, Bellugi, Reilly, &Anderson, 2000). Combined with

the hypersociability associated with WS, these children often engage in storytelling(Jones .et aI., 2000). And while their stories use less complex syntax compared to average chIldren, they are much more complex (and more expressive) than children withsimilar cognitive abilities with Down syndrome (Reilly, Klima, & Bellugi, 1990).

CREATIVITY AS A SUPPLEMENT: FORWARD DIRECTIONS

Creativity is a natural candidate to supplement traditional measures of ability andachievement. A growing trend among admission committees and educators is a focus

on noncognitive constructs, such as emotional intelligence, motivation, and creativity,to supplement current measures (Kyllonen, Walters, & Kaufman, 2002). Creativity is aprime candidate to be such a supplement. One reason is that creativity is related to intelligence and academic ability, yet not so closely related as not to account for addi-

,


tional variance. Another promising reason is the reduction in gender and ethnicity differences. Finally, many different voices in education have raised a specific interest inthe measurement of creativity.

Reform efforts in school standards, for example, are showing a renewed interest inliterature and creative writing (Standards for the English/Language Arts, 1996). More than50 colleges have decided to offer creative writing majors in the last 6 years (bringingthe total to more than 300); this increase comes at a time when the number of Englishmajors as a whole is decreasing (Bartlett, 2002). A survey of distinguished graduatefaculty members found that creativity was considered to be one of the most importantcompetencies deemed essential for success in graduate school (Enright & Gitomer,1989). Creativity was one of six noncognitive areas that Mayer (2001) recommendedas being valuable candidates for new measures, and creativity was one of five qualities singled out in a study of potential additional measures to the GRE (Walpole, Burton, Kanyi, & Jackenthal, 2001).

The fact that creativity is not assessed on current measures of ability and achievement is often cited by testing opponents as one reason that these tests are not valid orsignificant. Paul Houston, the executive director of the American Association of SchoolAdministrators, has said, "Children today need critical thinking skills, creativity, perseverance, and integrity-qualities not measured on a standardized test." (Assessment Reform Network, 2002). In a similar vein, former U.S. Secretary of Labor RobertReich wrote, "Many new jobs depend on creativity-on out-of-the-box thinking, originality, and flair .... Standardized tests can't measure these sorts of things" (Reich,2001). Whether standardized tests can or cannot measure creativity, it is possible tomeasure creativity on an individual basis-and this measurement can supplementtraditional measures and increase fairness in assessment.

Current assessments are limited by either theoretical constraints (e.g., psychometric tests producing only a general creativity score) or practical constraints (e.g., theneed to find experts). One proposal for an ideal creativity test might be a test that explores a multitude of domains. By its very nature, such a test would be an individually based test instead of a group test.

The first step would be to identify which domains should be measured, and howthese domains cluster together. Some initial work has been conducted on examiningthe factor structure of self-reported creativity in different domains. Early results showthree factors emerging. Factor 1 is related to expressing emotions, and consists of suchvariables as writing and interpersonal communication. Factor 2 is more hands on, andincludes arts, crafts, and physical performance. The third factor consists of mathematics and science (J. C. Kaufman & Baer, in press) ..

Much more research is certainly needed on the factor structure of creative domains,but this early research gives a hint of how it might look. A complete creativity testwould include different items that tap into each factor in many different ways. Sucha test would be closer in time administration and examiner demands to an IQ testinstead of the current group creativity tests. But the additional time and effort, webelieve, would result in a much more complete description and analysis of a student's creativity.

INTELLIGENT TESTING WITH CREATIVITY

In early years, the point of being assessed with an IQ test was to get one score-anIQ-that was often treated as a magic number that could open or shut doors of op-

I

I

I

---------------------------------------------.-----.-------- .... ----.-------


portunity. Creativity tests have no such power, but most assessments are also gearedtoward a single number or score. A more recent philosophy of IQ testing is that of "intelligent testing" (A. S. Kaufman, 1994).

Using this system, the tester is elevated above the test. The single number or scoremeans little by itself. The key is the score in context. The persons administering the testare expected to use their qualifications and training and bring their 0\.\711 experience tothe testing session. In this manner, the tester can help the child or adult being testedby understanding and interpreting a \l\ide range of behaviors and making inferencesabout any observed problem-solving strategies. Every aspect of psychology is broughtinto play to interpret a profile of scores in the context of accumulated research. Thisprofile is used to help solve problems and create solutions for the person tested, notmerely as a label or classification system (A. S. Kaufman, 1994).

We believe that this approach can be applied to creativity research. A qualifiedtester would be well versed in the fields of social, cognitive, and educational psychology (among others). The pattern of scores in the different domains could be interpreted for comparative strengths and weaknesses. Rather than merely producing asingle number that is of little use to a student, this new domain-specific creativity measure could help students discover and validate areas of creative talent in themselves.In addition, an administrator using the "intelligent testing" approach could look forsigns of insufficient motivation, a thinking style that might conflict with the task, orother additional areas that could be improved for enhanced creative potential.

Such "intelligent testing" might add to our understanding of subjects' cognitiveabilities and potential by providing additional information beyond that given byscores on other tests (such as IQ tests). If it could be shown that some race, ethnicgroup, or gender differences in testing were systematically related to testable differences in creativity, such creativity scores could also shed light on how best to interpretIQ and other test scores of minority group members, and perhaps even provide ameans to help attenuate error variance due to bias in testing. These hoped-for outcomes depend, of course, on the development of more valid, reliable, and meaningfulcreativity tests. Domain-specific creativity tests, based on a deeper understanding ofthe factor structure of creativity, may soon make this possible.

CONCL US ION

There are many additional constructs that may be used as part of such a supplemental approach to nonbiased testing (e.g., emotional intelligence, motivation, thinkingstyles), but creativity serves as a good exemplar of this approach. There are multipleways of measuring creativity, extensive studies have examined creativity across manydifferent possible groups, the relationship between intelligence and creativity hasbeen well explored, and the field is still actively studied. Future work on creativity andability or achievement testing may concentrate on creativity across domains, individual rather than group creativity tests, and an "intelligent testing" approach. An examiner who adds a creativity measure-or a measure of other alternative cognitiveconstructs-may be able to minimize possible testing bias.

REFERENCES

Ai, X. (1999). Creativity and academic achievement: An investigation of gender differences. Creativittj Research Jou1'I1al, 12,329-337.


Altman, W. S. (1999). Creativity and academic success. Dissertation Abstracts InternationalSection A: Humanities and Social Sciences, 59, 1O-A,3731.

Amabile, T. M. (1982). Social psychology of creativity: A consensual assessment technique.Journal of Personality and Social Psychology, 43, 997-1013.

Amabile, T. M. (1985). Motivation and creativity: Effects of motivational orientation in creative writers. Journal of Personality and Social P51jchologlj,48, 393-397.

Amabile, T. M. (1996). Creativity in context: Update to "The social psychology of creativity."Boulder, CO: Westview Press.

Amabile, T. M., Hennessey, B. A., & Grossman, B. S. (1986). Social influences on creativity:The effects of contracted-for reward. Journal of Personalihj and Social Psychology, 50, 14-23.

Argulewicz, E. N., Elliott, S. N., & Hall, R. (1982). Comparison of behavioral ratings of AngloAmerican and Mexican-American gifted children. Psychology in the Schools, 19, 469-472.

Argulewicz, E. N., & Kush, J. C. (1984). Concurrent validity of the SRBCSS Creativity Scalefor Anglo-American and Mexican-American gifted students. Educational and Psychological Research, 4,81-89.

Argulewicz, E. N., Mealor, D. J., & Richmond, B. O. (1979). Creative abilities of learningdisabled children. Journal of Learning Disabilities, 12,21-24.

Assessment Reform Network. (2002). What do the experts say about high-stakes testing? Retrieved January 17, 2004 from http://www.fairtest.org/ ami experts.html

Averill, J. R., Chon, K. K., & Hahn, D. W. (2001). Emotions and creativity, East and West.Asian Joumal of Social Psycholaglj, 4,165-183.

Baer, J. (1994). Divergent thinking is not a general trait: A multi-domain training experiment. Creativihj Research Joumal, 7, 35-46.

Baer, J. (1996). The effects of task-specific divergent-thinking training. Journal of Creative BehaviOl;30,183-187.

Baer, J. (1997). Creative teache1's,creative students. Boston: Allyn & Bacon.Baer, J. (1999). Gender differences in creativity. In M. A. Runco & S. Pritzker (Eds.), Ency

clopedia of creativihj (pp. 753-758). San Diego, CA: Academic Press.Baer, J., Kaufman, J. c., & Gentile, C. A. (2004). Extension of the consensual assessment

technique to nonparallel creative products. Creativity Research Joumal, 16(1), 113-117.Baldwin, A. Y. (1985). Programs for the gifted and talented: Issues concerning minority

populations. In F. Horowitz & M. O'Brien (Eds.), The gifted and talented: Developmentalperspectives (pp. 223-249). Washington, DC: American Psychological Association.

Baldwin, A. Y. (2003). Understanding the challenge of creativity among African Americans. Inquiry, 22, 5-7.

Barron, F., & Harrington, D. (1981). Creativity, intelligence, and personality. Annual Reviewof Psychologlj, 32, 439-476.

Bartlett, T. (2002, March 15). Undergraduates heed the writer's muse. Chronicle of HigherEducation, pp. A39-A45.

Bellugi, U., Lichtenberger, E. 0., Jones, W., Lai, Z., & St. George, M. (2000). The neurocognitive profile of Williams syndrome: A complex pattern of strengths and weaknesses.Journal of Cognitive Neuroscience, 12, 7-29.

Besemer, S. P., & O'Quin, K. (1986). Analyzing creative products: Refinement and test of ajudging instrument. Joumal of Creative Behavia1;20, 115-126.

Boden, M. A. (1994). Dimensions of creativihj. Cambridge, MA: MIT Press.Bruner, J. (1962). The conditions of creativity. In H. Gruber, G. Terrell, & M. Wertheimer

(Eds.), Conte111poraryapproaches to creative thinking. New York: Atherton Press.Burrows, D., & Wolf, B. (1983). Creativity and the dyslexic child: A classroom view. Annals

of Dyslexia, 33, 269-274.

I

__________________________________________________ u -_

836 TESTING, ASSESSMENT, AND lNrERVENTION ISSUES

Camara, W. J., & Schmidt, A E. (1999). Group differences in standardized testing and sociai

stratification (College Board Rep. No. 99-5). New York: College Board.Carroll, J. B. (1993). Human cognitive abilities. Cambridge, England: Cambridge University

Press.

Chen, c., Kasof, J., Himsel, A J., Greenberger, E., Dong, Q., & Xue, G. (2002). Creativity indrawings of geometric shapes: A cross-cultural examination with the consensual assess-menttechnique. Journal of Cross Cultural PSyc1lOlogy,33, 171-187. -

Coley, R. J. (2001). Differences in the gender gap: Comparisons across racial/ethnic groups in ed:ication and work. Princeton, NJ: Educational Testing Service.

Conti, R., Coon, H., & Amabile, T. M. (1996). Evidence to support the componential modei

of creativity: Secondary analyses of three studies. Creativihj Research Journal, 9,385-389.Cox, M. v., & Cotgreave, S. (1996). The human figure drawings of normal children ane

those with mild learning difficulties. Educational Psychologtj, 16,433-438.Cox, M. v., & Maynard, S. (1998). The human figure drawings of children with Down syn

drome. British JOIl1'11alof Developmental Psychologtj, 16, 133-137.Craig, J., & Baron-Cohen, S. (1999). Creativity and imagination in autism and Asperger

syndrome. Journal of Autism and Developmental Disorders, 29, 319-326.Craig, J., & Baron-Cohen, S. (2000). Story-telling ability in children with autism or Asperger

syndrome: A window into the imagination. Israelloumal of Psychiatn}, 37,64-70.Craig, J., Baron-Cohen, S., & Scott, F. (2001). Drawing ability in autism: A window into the

imagination. Israellournal of Psychiatry, 38, 242-253.Cramond, B. (1994). Attention-Deficit Hyperactivity Disorder and creativity: What is the

connection? JOll111alof Creative Behavior, 28, 193-210.Crozier, W. R. (1999). Age and individual differences in artistic productivity: Trends within

a sample of British novelists. Creativih} Research lOllmal,12, 197-204.Das, J. P., Naglieri, J. A, & Kirby, J. R. (1994). Assessment of cognitive process: The PASS theO/?

of intelligence. Boston: Allyn & Bacon.Dossick, J., & Shea, E. (1995). Creative: therapy III: 52 more exercises for groups. Sarasota, FL:

Professional Resource Press.

Enright, M. K., & Gitomer, D. H. (1989). Toward a description of successful graduate studellt~

(GRE Board Professional Rep. No. 89-09, GRE Board Research Rep. 85-17R). Princeton,NJ: Educational Testing Service.

Finke, R. A, Ward, T. B., & Smith, S. M. (1992). Creative cognition. Cambridge, MA: MIT Press.Fuchs-Beauchamp, K. D., Karnes, M. B., & Johnson, 1.J. (1993). Creativity and intelligence

in preschoolers. Gifted Child Quarterly, 73, 113-117.Gentile, C. A, Kaufman, J. c., & Baer, J. (2003). Do gifted student writers and creative writing

experts rate creativihJ the same way? Manuscript submitted for publication.Getzels, J. W, & Jackson, P. W. (1962). Creativity and intelligence: Explorations with gifted stu

dents. New York: Wiley.Glover, J. A (1976a). Comparative levels of creative ability in Black and White college stu

dents. Journal of Genetic PSt}cllOlogy,128, 95-99.Glover, J. A (1976b). Comparative levels of creative ability among elementary school chil

dren. ]ollmal of Genetic Psychologt}, 129, 131-135.Gollmar, S. M. (2001). An investigation of Attention Deficit/Hyperactivity Disorder, cre

ativity, and cognitive style: Interaction and impact on school success. Dissertation Abstracts Interlwtional Section A: Humanities and Sodal Sciences, 61, 9-A, 3464.

Guilford, J. P. (1967). The nature of 1m man intelligence. New York: McGraw-Hill.Hall, W, & MacKinnon, D. W (1969). Personality inventory correlates of creativity among

architects. Joumal of Applied Psychologtj, 53,322-326.


Heath, S. B. (1983). Ways with 'Words.Cambridge, England: Cambridge University Press.Hedges, L. v., & Nowell, A (1995). Sex differences in mental test scores, variabilityi and

numbers of high-scoring individuals. Science, 269, 41-45.Hennessey, B. A (1994). The consensual assessment technique: An examination of the re

lationship between ratings of product and process creativity. Creativity Research Journal,7, 193-208.

Hickey, M. (2001). An application of Amabile's consensual assessment technique for ratingthe creativity of children's musical compositions. Journal of Research in Music Education,49, 234-244.

Horn, J. L., & Hofer, S. M. (1992). Major abilities and development in the adult period. InR. J. Sternberg & C. Berg (Eds.), Intellectual development (pp.44-99). Cambridge, England: Cambridge University Press.

Jackson, P. w., & Messick, S. (1965). The person, the product, and the response: Conceptualproblems in the assessment of creativity. Journal of Personality, 33,309-329.

Jaquish, G. A, & Ripple, R. E. (1984). Allie-span developmental cross-cultural study of divergent thinking abilities. Inte1'l1ationalJournal of Aging and Human Development, 20, 1-11.

Jellen, H. U., & Urban, K. (1989). Assessing creative potential worldwide: The first crossCultural application of the Test for Creative Thinking-Drawing Production (TCT-DO).Gifted Educational International, 6, 78-86.

Jones, w., Bellugi, U., Lai, Z., Chiles, M., Reilly, J., Lincoln, A, & Adolphs, R. (2000). Hypersociability in Williams syndrome. Journal of Cognitive Neuroscience, 12, 30-46.

Kaltsounis, B. (1974). Race, socioeconomic status and creativity. Psychological Reports, 35,164-166.

Kasof, J. (1997). Creativity and breadth of attention. Creativity Research Journal, 10,303-315.Kaufman, A S. (1994). Intelligent testing with the WISC-III. New York: Wiley.Kaufman, AS. (2000). Tests of intelligence. In R. J. Sternberg (Ed.), Handbook of intelligence

(pp. 445-476). Cambridge, England: Cambridge University Press.Kaufman,A S., & Lichtenberger, E. O. (2002). Assessing adult and adolescent intelligence. Bos

ton: Allyn & Bacon.

Kaufman, J. C. (2001a). Genius, lunatics, and poets: Mental illness in prize-winning authors. Imagination, Cognition, and Personality, 20,305-314.

Kaufman, J. C. (200lb). The Sylvia Plath effect: Mental illness in eminent creative writers.JOllrnalof Creative Behaviol~35,37-50.

Kaufman, J. c., & Baer, J. (2004). Sure, I'm creative-but not in math! Self-reported creativity in diverse domains. Empirical Studies of the Arts, 22(2),143-155.

Kaufman, J. c., Baer, J., & Gentile, C. A (2004). Racial and gender differences in creativity as measured by ratings of three writing tasks. Journal of Creative Behavior, 38(1),56-69.

Kaufman, J. c., Gentile, C. A, & Baer, J. (in press). Do gifted student writers and creativewriting experts rate creativity the same way? Gifted Child Quarterly.

Kaufman, N. L., & Kaufman, A S. (1980). Creativity in children with minimal brain dysfunction. Journal of Creative Behavior, 14, 73.

King, L. A, McKee-Walker, L., & Broyles, S. J. (1996). Creativity and the five factor model.[ollmal of Research in Personality, 3D, 189-203.

Knox, B. J., & Glover, J. A (1978). A note on preschool experience effects on achievement,readiness, and creativity. [ol/rnal of Genetic Psychology, 132, 151-152.

Kogan, N. (1974). Creativity and sex differences. Joumal of Cl'eative Behavior, 8, 1-14.Korkman, M., Kirk, U., & Kemp, S. L. (998). NEPSY: A developmentalneuropS1jchological as

sessment. San Antonio, TX: The Psychological Corporation.


Kyllonen, P. c., Walters, AM., & Kaufman, J. C. (2002). Non-cognitive constructs in graduateeducation: A review and some applications. Princeton, NJ: Educational Testing Service.

LaFrance, E. B. (1997). The gifted/dyslexic child: Characterizing and addressing strengthsand weaknesses. Annals of Dyslexia, 47,163-182.

Leroux, J. A., & Levitt-Perlman,M. (2000). The gifted child with Attention Deficit Disorder:An identification and intervention challenge. Roeper Review, 22,171-176.

Loehlin, J. C. (1999). Group differences in intelligence. In R. J. Sternberg (Ed.), Handbook ofintellige11ce(pp. 176-193). Cambridge, England: Cambridge University Press.

Losh, M., Bellugi, U., Reilly, J., & Anderson, D. (2000). Narrative as a social engagementtool: The excessive use of evaluation in narratives from children with Williams syn

drome. Narrative Inquinj, 10, 265-290.Lubart, T. I. (1999). Creativity across cultures. In R. J. Sternberg (Ed.), Handbook of creativity

(pp. 339-350). New York: Cambridge University Press.Ludwig, AM. (1995). The price of greatness. New York: Guilford Press.Manly,J. J., Miller, S. W.,Heaton, R. K., Byrd, D., Reilly,J., Velasquez, R. J., et al. (1998). The

effect of Black acculturation on neuropsychological test performance in normal and HIVpositive individuals. Joumal of the International Neuropsychological Society, 4, 291-302.

Masten, W. G., Plata, M., Wenglar, K., & Thedford, J. (1999). Acculturation and teacher ratings of Hispanic and Anglo-American students. Roeper Review, 22, 64-65.

Masters, M. S., & Sanders, B. (1993). Is the genetic difference in mental rotation disappearing? Behavioral Genetics, 23, 337-341.

Mayer, J. D. (2001). On new methods of personality assessme11t. Paper presented at the Symposium on Noncognitive Assessments for Graduate Admissions, Toronto, Canada.

McGrew, K. 5., & Flanagan, D. P. (1998). The intelligence test desk refere11ce(!TOR): Gf-Gccross-batten} assessment. Boston: Allyn & Bacon.

McGrew, K. 5., Woodcock, R., & Ford, L. (2002). The Woodcock-Johnson Battery-ThirdEdition (WJ TIn. InA S. Kaufman &E. O. Lichtenberger, Assessing adult and adolescent intelligence (pp. 561-628). Boston: Allyn & Bacon.

Mednick, S. (1962). The associative basis of the creative process. Psychological Review, 69,220-232.

Mitchell, B. M. (1988). Hemisphericity and creativity: A look at the relationships amongelementary-age low-income Hispanic children. Educational Research Quarterly, 12,2-5.

Moreno, J. M., & Hogan, J. D. (1976). The influence of race and sodal-class level on the

training of creative thinking and problem-solving abilities. Journal of Educational Research, 70,91-95.

Morgan, R., & Maneckshana, B. (1996, April). The psychometric perspective: Lessons leamed

from 40 years of constructed response testing in the Advanced Placement Program. Paper presented at the Annual Meeting of the National Council of Measurement in Education inNew York.

Naglieri, J. A, & Kaufman, J. C. (2001). PASS theory, intelligence, and giftedness. RoeperReview, 23,151-156.

Niaz, M., Saud de Nunez, G., & Ruiz de Pineda, I. (2000). Academic performance of highschool students as a function of mental capadty, cognitive style, mobility-fixity dimension, and creativity. Joumal of Creative Behavior, 34,18-29.

Niu, W., & Sternberg, R. J. (2001). Cultural influences on artistic creativity and its evalua

tion. Intemational Joumal of Psychologlj, 36(4), 225-241 ..Palaniappan,A. K. (1996). Across-cultural study of creative perceptions. Perceptual and Mo

tor Skills, 82, 96-98.

Plucker, J. A., & Renzulli, J. S. (1999). Psychometric approaches to the study of human cre-


ativity. In R J. Sternberg (Ed.), Handbook of creativity (pp. 35-61). New York:CambridgeUniversity Press.

Rack, 1. (1981). Developmental dyslexia and literary creativity: Creativity in the area ofdeficit. Journal of Learning Disabilities, 14,262-263.

Reich, R B. (2001, July 20). Standards for what? Education Week, 20, p. 64.Reilly,J., Klima, E. S., & Bellugi, u. (1990). Once more with feeling: Affect and language in

atypical populations. Develapment and Psychopathologtj, 2,367-391.Reynolds, C. R (2000). Methods for detecting and evaluating cultural bias in neuropsy!chological tests. In E. Fletcher-Janzen, T. 1. Strickland, & c. R Reynolds (Eds.), Hand

book of cross-cultural neuropsychologtj (pp. 249-285). New York:Plenum.Reynolds, C. R, & Horton, T. (in press). Test of verbal conceptualization and fluency. Austin,

TX:Pro-Ed.Reynolds, C. R, Lowe, P.A, & Saenz, A 1. (1999). The problem of bias in psychological as

sessment. In C. R. Reynolds & T. B. Gutkin (Eds.), The handbook of school psychology(pp. 549-596). New York:Wiley.

Ripple, R E., & Jaquish, G. A (1982). Developmental aspects of ideational fluency, flexibility, and originality: South Africa and the United States. South African Joumal of Psychology, 12,95-100.

Rostan, S. M., Pariser, D., & Gruber, H. E. (2002). Across-cultural study of the developmentof artistic talent, creativity, and giftedness. High Ability Studies, 13, 125-156.

Rudowicz, E., Lok, D., & Kitto, J. (1995). Use of the Torrance Tests of Creative Thinkingin an exploratory study of creativity in Hong Kong primary school children: A crosscultural comparison. Inte1'l1ationalJoumal of Psychologtj, 30, 417-430.

Runco, M. A (1989). The creativity of children's art. Child Study Joumal,19, 177-190.Runco, M. A, & Vega, 1. (1990). Evaluating the creativity of children's ideas. Jou1'l1alof So

cial Behavior and Personality, 5, 439-452.Saeki, N., Fan, X., & Van Dusen, L. (2001). A comparative study of creative thinking of

American and Japanese college students. Journal of Creative Behavior, 35, 24-36.Shade, B. J. (1989). The influence of perceptual development on cognitive style: Cross

ethnic comparisons. Early Child Development and Care, 51, 137-155.Shaw, G. A (1992). Hyperactivity and creativity: The tacit dimension. Bulletin of the Psy

chonomic Society,30, 157-160.Shaw, G. A, & Brown, G. (1990). Laterality and creativity concomitants of attention prob

lems. Developmental Neuropsychologtj, 6, 39-56.Shaw, G. A, & Brown, G. (1991). Laterality, implicit memory and attention disorder. Edu

cational Studies, 17, 15-23.

Simonton, D. K. (1988). Age and outstanding achievement: What do we know after a century of research? Psychological Bulletin, 104, 251-267.

Simonton, D. K. (1994). Greatness. New York: Guilford Press.Standards for the English/language arts: A project of the Intemational Reading Association and

the National Council of Teachers of English. (1996). Newark, DE: International ReadingAssociation.

Sternberg, R. J. (1985). Be/jolldIQ: A trillrchic theory ofllll111anintelligence. Cambridge, England:Cambridge University Press.

Sternberg, R J. (1996). Successful intelligence. New York:Simon & Schuster.Sternberg, R. J. (Ed.). (1999). Handbook of creativity. Cambridge, England: Cambridge Uni

versity Press.Sternberg, R J.,Kaufman, J. c., & Pretz, J. E. (2002). The creativittj cOlllll1drum.Philadelphia:

Psychology Press.

Documents

Nonbiased Assessment: A Supplemental Approach