Cross-Cultural Psychometrics: the Eysenck Paradigm ...scindeks-clanci.ceon.rs/data/pdf/0048-5705/1998/... · This paper outlines the background, rationale, and strategy that encompasses

PSIHOLOGIJA, 1998, 3, 171-192

UDC 159.9.072.59

171

Cross-Cultural Psychometrics: the Eysenck Paradigm, Measurement,

and Psychological Science

PAUL T. BARRETT

The State Hospital, Department of Psychology Carstairs1 and University of Liverpool, Department of Clinical Psychology, Liverpool

SYBIL B. G. EYSENCK

Institute of Psychiatry, Department of Psychology, London This paper outlines the background, rationale, and strategy that encompasses the 22 years of Eysenck investigations into the cross-cultural comparison of the four personality "superfactors" measured by the Eysenck Personality Questionnaire. This effort has not taken place without encountering criticism of both its methodology and psychological meaningfulness. However, a recent paper by Barrett, Petrides, Eysenck, and Eysenck (in press) has answered these criticisms by correcting the comparison methodology, and subsequently demonstrating that the original results reported by the Eysencks remain valid. Although the investigative methodology seems to be largely descriptive, it is noted that the personality model and experimental basis of the factors fits within a coherent scientific model of personality investigation proposed by H. J. Eysenck (1997). However, in order that the constraints of a quantitative science are fully met, it is essential that the quantitative structure of the personality variables is established empirically. In practice, this requirement may be met by ensuring that the measurement scales conform to the axiomatic measurement principles embodied within the Rasch measurement model. The practical implications of this requirement are examined using the UK EPQ reference sample dataset. From the results of this analysis, and in agreement with Michell's (1997) postulates concerning the constituent properties of a quantitative science, a new scientific model for cross-cultural psychometrics is recommended.

———————— 1 Carstairs, Lanark, Scotland, ML 11 8RP, UK

P. T. Barrett, S. B. G. Eysenck

172

The Eysenck Cross-Cultural Paradigm

The paradigm chosen by the Eysencks for this particular research programme was generated specifically for testing the hypothesis that the fundamental variables that constitute the Eysenckian model of personality were universal. That is, the four personality traits of Psychoticism (P), Extraversion (E), Neuroticism (N), and Social Desirability (L) would be demonstrated to be measurable constructs in differing cultures/countries. The key feature of this hypothesis was that the personality factor scales (whose explanatory power was demonstrated in many forms of experimental and correlational studies) were measures of fundamental psychological and physiological functions, shared by all human beings regardless of race or culture. Eysenck and Eysenck (1985) provide a succinct overview of the entire range of work that has typically been used as supporting empirical evidence for the Eysenck model of personality.

The Eysenck paradigm has been described in great detail in Eysenck (1983). Further the Eysencks have used this paradigm for the investigation of both adult and child samples. However, since the majority of their work has used adult samples, this paper will concentrate solely on this group. The essential features of the paradigm were thus:

1. A psychometric measure of the four constructs was required. The

Eysenck Personality Questionnaire (EPQ: Eysenck and Eysenck, 1975) was chosen as the most suitable instrument. This questionnaire contained 90 items, 25 measuring P, 21 measuring E, 23 measuring N, and 21 measuring L. The psychometric properties of this questionnaire were well-known, and of a high quality with regard to item analyses, reliability indices, and the factor pattern. This was confirmed later by Barrett and Kline (1980) using a high quality stratified sample of UK adults (acquired by the Eysencks), and a separate sample of UK students. In fact, for many of the cross-cultural studies, a 101-item questionnaire was used that contained all the items of the 90-item EPQ, but also some extra items that loaded on each scale. These extra items were included in most of the studies in order to retain sufficient numbers of items in each scale should some need to be deleted due to cultural specificity.

2. The next step was to contact a psychologist in a particular country (or

await enquiries from psychologists in foreign countries), who was interested in collaborating on this form of cross-cultural project. Then, a translated version of the EPQ questionnaire would be generated for administration in the "target" country. This was a very detailed and carefully implemented task that required great diligence. As Eysenck (1983) points out, for some countries this translation exercise was more successful than for others. Finally, a projected sample of 500 male and 500 female adults was acquired from the target country. The primary sample specifications were that the respondents were to be drawn from within an age range between about 18 and 70 years of age, and that the use

The Eysenck Paradigm, Measurement, and Psychological Science

173

of students was to be limited. One final preference was the acquisition, where possible, of respondents from a variety of occupations. This would permit the foreign psychologist to develop norms for the test once the cross-cultural comparison had been made and was successful.

3. The method chosen for cross-cultural comparison was implemented in

two phases. In order to satisfy the requirement of testing the hypothesis that the same constructs could be measured in the UK as well as any target country, an initial comparison of factor patterns was made between the four factor solution of the UK reference sample dataset, and a four factor solution computed from the target country's data. The factor analysis methodology chosen for all analyses was principal components analysis. The first four component factors were then rotated via oblique Promax or, latterly, direct oblimin. Factor comparisons were then made using the Kaiser, Hunka, and Bianchini (KHB: 1971) congruential fit procedure, with similarity coefficients between 0.95 and 0.98 taken as indicating similarity, and above this as indicating essential identity (the KHB congruence coefficient varies between 0, and 1.0, with 0 indicating no similarity, and 1.0 indicating absolute identity). The factor analyses and subsequent comparisons made were by gender (target male vs. female factor patterns, then UK males vs. target males, and UK females vs. target females). Given this proved successful, an optimal score key was computed from the target country's factor pattern. This optimal country score key might well have included some of the extra 11 items administered with the 90 item UK EPQ item set. However, for comparative purposes, a second score-key was generated that used only items retained from the 90 item UK dataset. This "in-common" scorekey was then used to form mean scale score comparisons between the measured scales in both the UK and target sample, by gender.

In this manner, beginning in 1977 with the first published comparison

between a Japanese sample and the UK reference sample (Iwawaki, Eysenck, and Eysenck, 1977), adult data from 34 countries has been analysed within this paradigm, and the results reported in many papers. A bibliography of 25 of these papers is provided in Barrett and Eysenck (1984). A short paper by Eysenck, Barrett, and Eysenck (1985) published the mean factor similarity matrices for 24 countries. For the factors of E, N, and L, the mean factor congruence coefficients were 0.99 respectively (1.0 is the maximum). For the P factor, this value was 0.96. As at 1985, the conclusion based upon the combined analyses of 24 countries was that the hypothesis of universality of the four personality constructs had received unanimous support from the studies. However, these results did not go unchallenged.


174

The Criticisms of the Eysenck work, and the Rejoinders

Poortinga has been a major critic of the Eysenck work, expressing doubt about the validity of the scale score comparisons, and the mathematical validity of the KHB comparison technique. Poortinga (1984) first expressed doubt about the KHB coefficients when he suggested that the high values usually obtained in the Eysenck studies in fact do not indicate the essential identity of factors, and further suggested that chance factors might generate almost equally high indices of factor comparison. It was to answer these criticisms that Eysenck, Barrett, and Eysenck (1985) reported the results of comparing 24 countries factor patterns. These results indicated that the occurrence of extremely high KHB coefficients (near 1.0) was confined solely to homologous factor pairs, that is, between Puk-Pc, Euk-Ec, Nuk-Nc, and Luk-Lc (where the subscripts uk and c denote the United Kingdom and "other" country respectively). Mean non-homologous factor comparisons were valued at about 0.16 overall. It was concluded that there was no evidence to support Poortinga's criticism.

However, Bijnen, Van der Net, and Poortinga (1986) subsequently demonstrated that, when using a 40-variable x 8 factor matrix of artificial data, then permuting item loadings within each factor vector to create 16 "randomised" factor structures, they were able to demonstrate KHB coefficients as large as 0.98 between the original target factors and one or more permuted variable factors within the randomised matrices. They concluded that such evidence seriously weakened the evidence put forward by the Eysencks, on the basis of cross-cultural factor comparison. Barrett (1986) attempted to demonstrate that the KHB coefficients were meaningful, using a procedure of analysis that relied upon Monte Carlo simulation methods and incremental degradation of real EPQ factor patterns. The main conclusion reached in this paper was that the KHB procedure was sound, although the use of Kaiser's "mean solution cosine" was seen as a mandatory constraint on any future use of the technique. That is, unless this coefficient was high (above about 0.90), it was considered wise to carefully assess the factor comparisons at the individual item level (in order to determine the items that may be causing excessive disparity between the two factor patterns).

However, further statistical work by Bijnen and Poortinga (1988) conclusively demonstrated that the KHB similarity coefficients were actually not similarity coefficients, but rather were cosines indexing the amount of angular transformation required to bring a pattern matrix into maximum agreement with a target matrix, irrespective of whether or not the resulting maximally congruent matrices were similar to one another. In other words, the coefficients put forward by Kaiser et al. were not measures of factor similarity at all, but rather, simply a measure of the angular transformations required to minimise the vector disparities between two orthogonal factor patterns. The KHB procedure failed to take into account that the two sets of factor vectors could be completely dissimilar, yet might only require a small transformation to bring them into their maximum possible congruence,


175

yielding very high transformation cosines (near 1.0). This explained the observations by Bijnen et al. (1986), and Barrett (1986) that KHB coefficients could achieve near unity, using either random or virtually random data.

Ten Berge (1996) elaborated further on the use of the KHB procedure, noting that only where the product of the transpose of the target matrix with a comparison matrix is symmetric (where the numbers of factors are equal in both matrices being compared) and positive semidefinite, can the KHB congruential fit procedure be considered valid. However, the use of the KHB "similarity" coefficients is still incorrect, as demonstrated in a simple computational example by ten Berge. Finally, ten Berge concluded that given his own mathematical arguments, Bijnen et al. (1986), Bijnen and Poortinga (1988), and Van de Vijver and Poortinga (1994), all of which demonstrate the same flaw in the coefficients, the KHB method should be considered invalid as a method of factor comparison. This was an extremely serious criticism that essentially cast doubt upon the entire Eysenck enterprise.

In response to these more recent criticisms, Barrett, Petrides, Eysenck, and Eysenck (in press) decided to attack the problem from a fundamental basis by modifying and extending the KHB matrix algebra in order to both satisfy the valid criticisms and principles outlined by ten Berge, but also to establish meaningful comparison coefficients for the factor comparisons. Essentially, by focussing on the KHB task as an orthogonal procrustes procedure, and restricting factor comparisons to orthogonal Varimax target matrices, Barrett et al. were able to make factor comparisons that were mathematically justified. In addition, in order to demonstrate that oblique (non-procrustes) comparisons were also possible, conventional congruence coefficients were computed between the direct oblimin rotated oblique factor patterns for both genders across 34 countries, using the UK reference sample oblique factor pattern matrices as targets. The analyses in this paper demonstrated unequivocally that the Eysenck factors, as found in the UK, are strongly replicable across all 34 countries. Figures 1 and 2 provide a graphic demonstration of the degree of similarity associated with same factor comparisons vs. the other comparisons. Also, by using a box and whisker plot of all the coefficients computed across all 34 countries, utilising the median, interquartile range, and range, the variability of the coefficients associated with the 16 possible comparisons is available for inspection.

Whilst the values are less than the original 0.99 values associated with what were effectively no more than angular discrepancies between two factor vectors, the substantial size of the congruence coefficients is still very significant. Perhaps of more significance though is the disparity between the homologous (same factor pair) and non-homologous (different factors) comparison coefficients. There is no overlap at all between these coefficient distributions. These results were essentially replicated using non-targeted oblique factor comparison coefficients.


176

Figure 1: Median and interquartile range box and whisker plot for MALE Adult EPQ datasets, displaying the distributional form of the modified KHB congruence coefficients computed across the 34 countries. The 16 possible factor comparison pairs are plotted on the x-axis, with the absolute valued coefficient size on the ordinate axis. The homologous factors pairs are EE, NN, LL, PP.

Figure 2: Median and interquartile range box and whisker plot for FEMALE Adult EPQ

datasets, displaying the distributional form of the modified KHB congruence coefficients computed across the 34 countries. The 16 possible factor comparison pairs are plotted on the x-axis, with the absolute valued coefficient size on the ordinate axis. The homologous factors pairs are EE, NN, LL, PP.


177

Recently, some authors have also begun to recommend the use of structural equation modelling (SEM) as a comparison methodology for factor structures. For example, Van de Vijver and Leung (1997) discuss the use of this technique at length, and make recommendations as to its utility in cross-cultural psychometrics. There are many advantages in this technique that make it very attractive for such use, not least is the capacity to make precise hypothesis tests concerning both the model structure of factors and loadings within a single culture, as well as then proceeding to test the fit of several groups to each other. Poon, Chan, Lee, and Leung (1993) provide extensive details of such techniques and an empirical demonstration. Unfortunately, as with many who make such recommendations and discuss these issues in an empirical vacuum, the depressing reality of structural equation modelling is far removed from what its protagonists proclaim. For example, there are almost no published examples of replicated SEM models that extend beyond either a single dataset or across laboratories. Secondly, virtually no existing personality questionnaire models can be "confirmed" using SEM methodology (McCrae, Zonderman, Bond, Costa, Paunonen, 1996; Church and Burke, 1994; Borkenau and Ostendorf, 1990), including the latest version of Cattell's 16PF, and Costa and McCrae's NEO 5-factor model. One notable success is Hofer, Horn, and Eber's (1997) SEM analysis of Cattell's original 16PF questionnaire, though only being able to model the second-order factor structure (the primary factor pattern remains non-replicable). An example of just how poorly the new version of the 16PF (16PF5) performs in a SEM analysis is demonstrated below. Here, the published US correlation matrix between the 15 personality scales (excluding B - reasoning ability) was modelled using SEM, with the paths between factors and scales specified as per the model defined in the factor-score weight matrix (for scoring individuals on the proposed second order factors). The correlations between the hypothesised second order factors were also specified. All other paths were set to 0.0. The specified paths were required to be estimated. Figure 3 presents the results of this analysis.

The first point of interest is the rather unclear structure of the model. However, this is what is in use at the current time, and supported by extensive exploratory factor analyses and subsequent criterion validities. Secondly, the model fails to fit the data - using both inferential statistics (corrected or otherwise), and a range of heuristic similarity indices. That the Eysenck questionnaires are not even able to be analysed via SEM methods because of the large number of items in the test (except using item parcels, which themselves raise several critical issues) is another cause for concern. It would appear that to use SEM methods, investigators in personality research might well have to reconsider the construction of tests and variables, moving away from large numbers of items as measures of fairly broad concepts to smaller item groups that can be modelled as hierarchical components of general latent variables. This is the position largely espoused by MacCallum and Browne (1993). The solution to this potential dilemma is provided below, as part of a unifying model of scientific vs. methodological psychological investigation.


178

Figure 3: The Structural Equation Model for the 16PF5 questionnaire, using the US

normative dataset of N=2500 cases, specifying the paths identified in Table 1.3 and factor weights in Table 1.4, of the US Technical Manual (Conn and Rieke, 1994). All model fit indices indicated lack of acceptable fit: Chi-square df=78, p < 0.00001, RMSEA=0.102, CFI=0.79, AGFI = 0.84, James-Mulaik-Brett Parsimony = 0.58, Bentlet-Bonett Normed Fit Index = 0.78.


179

Before leaving this section, it is also relevant to note that Poortinga (1995) has questioned the meaningfulness of the measurement made. That is, even if the structural comparisons are made as in the Eysenck paradigm, are the scale scores so computed equally meaningful across cultures (especially since different cultures invariably use a slightly different item set). This issue is one Van de Vijver and Leung (1997) refer to as the "scalar equivalence" problem. Poortinga has viewed this issue as one of "cultural bias". That is, although some items may appear to be valid as part of a structural comparison, the response rates of different cultures on these items suggests that the items may not be as important or "meaningful" in some cultures as others. This then leads to problems when assuming that all items can be given equal weight when composing a scale score. Van de Vijver and Leung (1997) provide some detailed discussion of this problem, and its potential solution using Item Response Theory (IRT) models to examine both differential item functioning (DIF) and test-score bias. Once again, as with SEM models, the same enthusiasm for IRT models is evident. But, moving to IRT test models has substantial repercussions for the use of existing personality measures (especially the Eysenck scales) in that the fit to the only IRT model that possesses axiomatic measurement properties (the Rasch model) is sometimes very difficult to achieve (Hambleton and Jones, 1993) provide a clear description of the Rasch model and its comparative features with respect to the classical true-score model). For example, Barrett and Kline (1981) attempted to Rasch scale the entire EPQ item-set as a way of determining whether the factorial, covariance-based, dimensionality of items could be reproduced by Rasch scaling. As expected, it could not. This was significant in that it demonstrated that the relationship between factorial covariance dimensionality and Rasch stochastic dimensionality was not simple. More to the point though is a series of Rasch analyses undertaken in this paper, as exemplars for the kind of problem to be addressed when moving from a "liberal" classical test theory factor-based model through to a more constrained and precise measurement model like the Rasch.

For the Rasch analyses, the complete UK reference sample combined gender dataset (N=4140 cases) was used. A one-parameter Rasch model was fitted to the item responses for each scale separately. That is, 4 Rasch models were fitted to the scored data (0,1 response range), one model per scale. For the P scale, with 25 items, the model failed to fit the data at p < 0.00005, with 9 items being rejected at p < 0.0005. For the E scale, these results were: overall fit of the model p < 0.00005, with 16 items being rejected at p < 0.0005. For the N scale, these results were: overall fit of the model p < 0.00005, with 14 items being rejected at p < 0.0005. For the L scale, these results were: overall fit of the model p < 0.00005, also with 14 items being rejected at p < 0.0005. However, bearing in mind that the chi-square test of model fit is biased according to sample size (the greater the sample size, the more difficult it is to fit items to the model), a sample size of 200 was chosen and the same analyses re-run. This time, the P scale fit the Rasch model according to the chi-square statistic (p = 0.160), with only 2 items rejected. For the E, N, and L scales, the residual error was still greater than that


180

expected if the Rasch model fitted the data. However, many fewer items were rejected in each case. Given that fit to the model is so obviously a function of sample size (as with SEM based chi-square statistics), other ways of assessing item fit have to be achieved. Generally, these involve assessing the invariance of item or person estimates over subsamples of items and persons. If the items fit the model, then it should be possible to compute parameters under these differing sample conditions, but nevertheless observe invariant parameter values. These issues are discussed more fully in Hambleton (1989) and Hambleton, Swaminathan, and Rogers (1991). However, we will not continue these analyses here, as the only purpose in undertaking the analyses above was to demonstrate that the recommendation to use IRT methods is not as straightforward as many authors might wish to propose. Again, one has to weigh up carefully the necessity for adopting a different test theory/analysis methodology in the light of the substantive criterion validities for pre-existing measures, using what may be a less mathematically rigorous test model. This issue is addressed below.

At this point in our discussion, it is worthwhile to briefly examine the current position of the Eysenck theory in the status of personality psychology. For this, we need to use the model of personality investigation proposed by Eysenck (1997) as an explanatory framework. Figure 4 presents this model.

Figure 4: The Eysenck (1997) model of the investigative processes within the personality

domain.


181

Essentially, this model shows the central position of the personality factors, with antecedent causal variables and the consequent variables aligned either side of the factor representation. However, note the flow of causation from the Distal Antecedent to Distal Consequent. This is an implicit reductionist view. What is important about this model is that it places the cross-cultural paradigm into a context in which it is seen as part of a much wider set of investigations into personality as a whole. As noted above, the universality hypothesis is predicated upon the proposition that the four trait factors are measures of fundamental biological processes that are themselves caused by the interaction of genes. Further, experimental evidence of how these processes interact with the perception of external stimuli and cognition in general is seen in the "Proximal and Distal Consequences" region of the model. Therefore, the cross-cultural paradigm is more than just an ad-hoc "compare several questionnaire scores between samples of individuals in different cultures". Showing that the comparisons are valid actually has many important consequences for the acceptance of the biological model of personality as outlined by Eysenck (Figure 4). These comparison results are precisely those that are required if we are to maintain the explanatory coherence of the Eysenck personality model and it is for this reason that much time and effort since 1984 has been spent on ensuring that the comparison methodology was sound. In addition, it is also clear from the model that the questions asked in the EPQ need to be of sufficient generality so as not to become culturally bound. Again, the cross-cultural work with the EPQ has demonstrated that this is the case, given a substantial number of items that are present in the UK versions are retained in every culture. There is, however, a fundamental problem that exists with the measurement of personality in many personality questionnaires, including the EPQ. In a more general sense, the statements noted above from Poortinga, Berry, Van de Vijver, and Leung concerning bias, differential item function, and the item composition of questionnaire scales, indirectly address this issue. Essentially, within the Eysenck cross-cultural paradigm (and in the EPQ), there is a strong assumption that the items all measure the underlying trait variable equally in every country, which implies they can be used almost interchangeably, and that any scale scores so composed are valid for comparative purposes. Further, all measurement made assumes equal-interval, additive measurement. Whilst we have noted that based upon the available evidence, these assumptions do seem to hold to some degree, it is reasonable to explore whether a more precise measurement model might be used that would permit more direct testing of the various assumptions. The investigative model in Figure 4 was taken from a recent chapter where Eysenck was discussing the necessity that personality investigation should become more scientific. Much of the chapter was devoted to how this might be achieved within the antecedent and consequent regions of the model. However, at the same time the book chapter was being published, the constituent properties of a quantitative science were being published simultaneously in the


182

British Journal of Psychology by Michell (1997). It is now possible to specify completely the requirements for future research in psychology (let alone in this particular domain), dependant solely upon whether an investigator wishes to undertake scientific research vs. mere methodological investigations. Subsequently, the choice of measurement model also becomes simple, in that there is only one measurement model in questionnaire item psychometrics that satisfies the axioms of scientific measurement.

Science and Methodology

Let us explore Michell's reasoning, and the nature of what constitutes the process of science. Michell's key premise (1997, p.402) is:

If science is taken realistically (i.e. as the attempt to understand the ways of working of natural systems), and its successes leave us no reasonable alternative, then a major task for the philosophy of science is to specify the kind of place the world mmuusstt bbee, in its most general features, for it to be possible that some scientific theories are true (where, by true is meant absolutely true i.e. things being just as stated in those theories). Applying this to quantitative science (exemplified paradigmatically in physics), the task is to specify the character which quantitative attributes mmuusstt hhaavvee if they are both measurable and interrelated continuously.

He then deduces that there are two tasks that are the constituent components of a quantitative science. First, a logically prior scientific task of experimentally investigating the hypothesis that the relevant attribute is quantitative, the scientific task. Secondly, the instrumental task of devising procedures to measure magnitudes of the attribute demonstrated to be quantitative. Michell then defines the axiomatic (i.e. an axiom is a self-evident statement, a proposition that is stipulated to be true for the purpose of a chain of reasoning), properties of quantitative scientific measurement, which essentially devolve into two major laws. That is, firstly, this kind of measurement must possess the property of ordinality, and secondly, the property of additivity. Table 1 provides the 9 axioms that formally define the property of quantitative measurement.


183

Table 1: The nine uniformities of co-existence (J. S. Mill, 1848) reprinted in Michell (1990, p. 52)

Let X, Y, and Z be any three values of a variable Q. Then Q is ordinal if and only if: 1. If X ≥ Y and Y ≥ Z then X ≥ Z (transitivity) 2. If X ≥ Y and Y ≥ X then X = Y (antisymmetry) 3. Either X ≥ Y or Y ≥ X (strong connexity) A relation possessing these three properties is called a simple order, so Q is ordinal if and only if ≥ is a simple order on all its values. All quantitative variables are simply ordered by ≥ , but not every ordinal variable is quantitative, for quantity involves more than order, it involves additivity. This is not a mathematical additivity (as in the operation of arithmetic addition) but instead represents an empirical concatenation operation. This concatenation operation can be arbitrary, but, if we assign numbers to measure quantitative variables, the rules of measurement will map this concatenation operation onto the mathematical operation of addition. Additivity involves a ternary relation, symbolized as “X+Y=Z”. Let Q be any ordinal variable such that for any of its values X, Y, and Z 4. X+(Y+Z) = (X+Y)+Z (associativity) 5. X+Y = Y+X (commutativity) 6. X ≥ Y if and only if X+Z ≥ Y+Z (monotonicity) 7. If X > Y then there exists a value of Z such that X=Y+Z (solvability) 8. X+Y > X (positivity) 9. There exists a natural number n such that nX ≥ Y (where 1X = X and

(n +1)X = nX + X) (Archimedean condition) In such a case, the ternary relation involved is additive, and Q is a quantitative variable. It is also of use here to distinguish between extensive or direct, indirect, and

implicit measurement. A very useful summary of these three kinds of measurement can be found in Van der Linden (1994), and a more formal presentation in Michell (1990, 1994). These three terms also serve to help define the kinds of quantitative measurement that can be made. Extensive measurement (Campbell, 1920, 1928) is concerned with the discovery of ordering and concatenation relations on the objects that directly reflect the quantitative structure of the variable involved. Examples of such extensively measured variables are: length, weight, duration (time), electrical resistance. An extensive measure can be loosely defined as one where there is a more or less direct isomorphism between the numerical quantities of a scale, and


184

the property of concatenation of an object being measured. Within the physical sciences, indirect measures of say velocity, acceleration, force, work, etc. are composed of extensive measures (weight, time, length). However, as Luce and Tukey (1964) demonstrated, there were a set of axioms that could be derived such that the properties of ordinality and additivity could be inferred for a variable, even though the measures of a variable used to make such an inference were only at an ordinal or even categorical level. Further, there is no direct isomorphism between the numerical properties of a scale and the property of concatenation. This is what defines implicit measurement. The axioms that define the property of conjoint measurement are presented in Table 2.

Table 2: The Luce and Tukey axioms of conjoint measurement Conjoint Measurement relates to situations of the kind P=A+X or P=A x X (which can be represented logarithmically as an additive concatenation function). Its application is specifically for those instances where none of P, A, or X is already quantified. It requires that: 1. Variable P possesses an infinite number of values 2. P=f(A,X) (where f is some mathematical function) 3. There is a simple order, ≥ , upon the values of P 4. Values of A and X can be identified (i.e. that objects may be classified

according to the value of A and X they possess) Such a system satisfying 1-4 is a conjoint system. Then if ≥ on P satisfies: a) Double Cancellation (if certain pairs of values of P are ordered by ≥ ,

then the other particular pairs of values will also be ordered) b) Solvability c) the Archimedean condition then: 5. P, A, and X are quantitative 6. f is a non-interactive function Examples of conjoint measurement within psychology are rare - one

implementation is the Stankov and Cregan (1993) research that examines the hypothesis that intelligence (as proposed to be measured by a Letter Series task) could be considered a quantitative variable, measured conjointly by working memory capacity and motivation, thereby satisfying the 9 axioms given in Table 1. Further, it is significant that as Wright (1997), Andrich (1988), and Van der Linden (1994) point out, the one psychometric test model that satisfies all the principles of axiomatic and conjoint measurement is the Rasch measurement model.


185

When typical psychometric measures of personality and intelligence that have been generated using the classical test theory model are examined against these axiomatic conditions of quantitative scientific measurement, ordinal relations within our measurement can be demonstrated with little difficulty. However, the property of additivity (here defined specifically as the mathematical addition of the units of measurement, such as the items on a test, and having a one-to-one correspondence to the equal-interval magnitudes of the attribute being measured) is completely missing or at best, untested. This leads Michell to conclude that continuing with such measurement and attributing it to scientific measurement, is in fact, a pretence at science. For Michell, there is a clear distinction between the quantitative methodologies that we as psychologists may use to generate numbers, and the manner in which scientists will use these same methodologies as part of a scientific process of investigation.

One way to better conceptualise the issue is to stand back from Michell's arguments and take an even more broad approach as to what constitutes scientific investigation (given acceptance of Michell's arguments concerning the necessity for axiomatic quantitative measurement). Here it is possible to use a very useful proposition from the philosopher Brian Haig (in preparation), who has isolated a fundamental task for science as "phenomena identification". Both qualitative and quantitative observations can be used here so as to initially identify phenomena. Barrett (1998) has referred to this activity as the first phase of the scientific process. Then, it is necessary to engage in what Barrett has denoted as the second phase, that is, Michell's task of testing both the scientific hypothesis that the variables hypothesised to define the phenomena actually do possess a quantitative structure, and then performing the instrumental task of defining procedures to measure magnitudes of these variables. Finally, Barrett has proposed that science is also engaged in a third and final process of causal attribution. That is, science concerns itself with not only phenomena identification and measurement, but with the understanding of the processes that cause the phenomena to occur. Within this paragraph lies the heart of the problem for psychology. It is not that the psychometric concepts and constructs used in individual differences psychology (personality, intelligence, motivation, needs, values, etc.) lack pragmatic value, for obviously they do possess this quality. Rather, it is questionable to what extent they possess a scientific value. This is important when wishing to consider exactly what it is that is being measured with a psychometric test. As Kline (1998) has asked in his new book, what precisely is the unit of measurement of any current psychometric test? Consider when we make measurement of length, duration, weight, or speed, the units we use to express magnitudes of these attributes are meaningful and possess known properties. We know that 8 seconds is precisely two seconds longer in duration than 6 seconds. The "second" not only has specific measurement properties, but also is meaningful in that it conveys duration implicitly (although the name "second" might just as well be "blurg", as long as we maintain its properties as a unit of measurement of duration). However, when we use a scale that measures say "Independence", what is the unit? If we do not know this, can we ever hope to "understand" the processes by which we will explain the observed


186

phenomena? Involved here is the very essence of the meaning of what it means to say that a person possesses a disposition toward say extraversion, and that this disposition is being measured in a unit of say "extravs". Further, it has recently been announced (June, 1998) that schools in the Miami-Dade county of Florida are to adopt the Lexile reading unit as the standard measurement of reading ability amongst its children. Based on more than 40 years of research on reading comprehension, the Rasch-based Lexile Framework was developed over a 10-year period under the auspices of the National Institute of Child Health and Human Development (NICHD) in the US. This is the first example where a new, explicit, truly equal-interval unit of measurement has been generated within psychology. Much consideration has been given to this issue of the relation of a unit of measurement to the meaning of the measured attribute by both Jackson and Maraun (1996a, b), Maraun (1996, in press), and Ter Hark (1990).

The Implications for the Eysenck Cross-Cultural Paradigm

Given an investigator chooses to undertake scientific research, and adopt the procedures for investigation and measurement that are required for a quantitative science, then several adjustments would seem to be required to the Eysenck cross-cultural paradigm.

Firstly, the EPQ or the EPQ-R (Eysenck, Eysenck, and Barrett, 1985) will need to be analysed using the Rasch model. This is necessary in order that the measures of magnitudes on each of the variables are demonstrated to possess both additive and ordinal properties (Michell's scientific hypothesis test). This is not a trivial task as is evident from the initial analyses reported in this paper. However, it is likely that such a task will be successful, given the substantive body of evidence that gives credence to the existing ordinal measures. With Rasch scaled items, the nature of the scale use changes dramatically. The probabilistic cumulative property of the scales and the known "difficulties" of the items permits a far more accurate assessment of cultural differences (and similarities) than before. Given "core" or "in-common" items are defined within the UK and a target culture, then precise estimates of Rasch test scores are available. Also, any differences in same-item difficulty/facility parameter values across cultures are a significant source of information that can be used to make further statements about the importance of certain items within a culture. In short, by adopting a strong measurement model, we are now better able to compare and contrast differing cultural response patterns with our measuring instruments.

Secondly, the meaning of the measurement being made is required to be elucidated. That is, as was done with the Eysenck questionnaires, the meaning of the variables was first defined and then the measures created accordingly. As Maraun (1996b) and Ter Hark (1990) have argued on the basis of Wittgenstein's


187

"grammar of meaning", the meaning of what is being measured is logically not able to be discovered empirically. Rather, it is to be found in the rules and relationships that define the existence of a construct. Further, the rules that define the meaning of a construct are considered autonomous in that they are not discoverable by inductive empiricism, but only by deductive confirmation. For example, if we specify the rules and relationships between other variables for a construct (and so demonstrate the meaningfulness of that construct), and then attempt to define a measure of that construct and make observations of how the magnitudes so measured conform to our deductions, we can assess directly the degree to which we might be said to "understand" the construct. Where we do not observe the proposed relationships, we can either query our measure (in terms of the accuracy of our measurement) or our postulates concerning the rules that define the instantiation and meaning of the construct. If we are using an axiomatic (rule-based) approach to both our measurement and our conceptualisation of meaning, then we are more capable of disentangling the confound that exists between the validity and meaning of the measurement made. As Maraun (1996b) has demonstrated, the familiar construct validity approach of Cronbach and Meehl (1955) is unable to disentangle this confound.

So, in consideration of this argument, it is clear that the rules specifying the properties, existence, and relationships between other variables for the EPQ variables must be specified. The Eysencks have done this, although perhaps more informally than is required by a quantitative science. However, Eysenck's model in figure 4 provides the framework for investigation. It is now incumbent upon future researchers to generate more precise models concerning the personality constructs, models that embody the rule-base that was initially proposed by the Eysencks. This is already happening in the researches undertaken by another of Eysenck's ex-students (Robinson, 1982, 1987, 1996), whose model of personality rests upon a precise set of defined rules and hypothesised relationships. Further, Gray's (1970, 1972, 1991) work on personality, and the neurobiological model proposed by Depue and Collins (1998) are also examples of a more precise approach to investigation, both leading to some adjustments in the Eysenck model based upon consideration of proposed neurophysiological process (the antecedent). Looking for relationships between variables, given an ill-conceived notion of what it is that is being sought, is frankly no longer acceptable. Given the requirements for axiomatic measurement, there must be an end to approximations based upon measurement that is of uncertain accuracy. This does not assume that measurement will be made without error, but that what measurement is made possesses known properties (not simply assumed). So, within the cross-cultural domain, not only must the psychometrics conform to an axiomatic model, but the process of scientific investigation would require that the rules defining the meaning of the construct are also confirmed in any culture. This requires consideration of empirical evidence in both the antecedent and consequent regions of figure 4, within a culture. This is very difficult to achieve, but perhaps not impossible.


188

In conclusion, it is hoped that the reader gains some insight into the magnitude of what the Eysencks originally undertook in the mid-1970s and that this was not some ad-hoc inductive exercise, but rather was based upon a series of deductions concerning the hypothesised universality of the personality constructs. That this hypothesis has been largely confirmed is a magnificent achievement. However, in order that the exercise become scientific in the manner suggested by Eysenck and Eysenck (1985) and Eysenck (1997), we have used arguments from Michell, Maraun, Haig, and Barrett to show how the paradigm must be modified and extended in order that it might be considered part of psychological science (instead of remaining a methodological exercise). Also, it is hoped that the reader is able to consider that the constituent features of this new paradigm are those that are required for individual difference research as a whole, if it is not to degenerate into an almost mindless exercise of largely atheoretical empiricism. These are strong statements, but worthy we hope of the memory of the man who has done more for individual difference psychology than any psychologist to date.

References

Andrich, D. (1988) Rasch Models for Measurement. London: Sage Publications.

Barrett, P. T. (1986) Factor comparison: An examination of three methods. Personality and Individual Differences, 7, 3, 327-340.

Barrett, P. T. (1998) Science, Fundamental Measurement, and Psychometric Testing. Selection and Development Review, 14, 4, 3-10.

Barrett, P., and Eysenck, S. (1984) The assessment of personality factors across 25 countries. Personality and Individual Differences, 5, 6, 615-632.

Barrett, P., and Kline, P. (1980) Personality factors in the Eysenck Personality Questionnaire. Personality and Individual Differences, 1, 317-333.

Barrett, P. T., and Kline, P. (1981) A comparison between Rasch analysis and factor analysis of items in the EPQ. Personality Study and Group Behaviour, 1, 2, 11-28.

Barrett, P. T., Petrides, K. V., Eysenck, S. B. G., and Eysenck, H. J. (in press) The Eysenck Personality Questionnaire: An examination of the factorial similarity of P, E, N, and L across 34 countries. Personality and Individual Differences.

Bijnen, E. J. and Poortinga, Y. H. (1988) The Questionable value of cross-cultural comparisons with the Eysenck Personality Questionnaire. Journal of Cross-Cultural Psychology, 19, 2, 192-202.

Bijnen, E. J., Van der Net, T. Z. J., and Poortinga, Y. H. (1986) On cross-cultural comparative studies with the Eysenck Personality Questionnaire. Journal of Cross-Cultural Psychology, 17, 1, 3-16.


189

Borkenau, P. and Ostendorf, F. (1990) Comparing exploratory and confirmatory factor analysis: A study on the 5-factor model of personality. Personality and Individual Differences, 11, 515-524.

Campbell, N. R. (1920) Physics, the Elements. Cambridge University Press.

Campbell, N. R. (1928) An account of the principles of measurement and calculation. Longmans, Green, & Co.

Church, A. T., and Burke, P. J. (1994) Exploratory and confirmatory tests of the big five and Tellegen's three- and four-dimensional models. Journal of Personality and Social Psychology, 66, 1, 93-114.

Conn, S. R. and Rieke, M. L. (1994) The 16PF Fifth Edition Technical Manual. Illinois, US: Institute of Personality and Ability Testing.

Cronbach, L. J. and Meehl, P. E. (1955) Construct validity in psychological tests. Psychological Bulletin, 52, 281-302.

Depue, R. A. and Collins, P. F. (1998) Preprint of: Neurobiology of the structure of personality: dopamine facilitation of incentive motivation, and extraversion. Behavioural and Brain Sciences - http://www.cogsci.soton.ac.uk/bbs/Archive/bbs.depue.htm.

Eysenck, H. J. (1997) Can personality study ever be objective? In C. Cooper and V. Varma (Eds.) Processes in Individual Differences. Routledge.

Eysenck, H. J., Barrett, P., and Eysenck, S. B. G. (1985) Notes and shorter communications: Indices of factor comparison for homologous and non-homologous personality scales in 24 different countries. Personality and Individual Differences, 6, 4, 503-504.

Eysenck, H. J. and Eysenck, M. W. (1985) Personality and Individual Differences: A Natural Science Approach. New York: Plenum Press.

Eysenck, H. J. and Eysenck, S. B. G. (1975) Manual of the Eysenck Personality Questionnaire. London: Hodder and Stoughton.

Eysenck, S. B. G. (1983) One approach to cross-cultural studies of personality. Australian Journal of Psychology, 35, 3, 381-391.

Eysenck, S. B. G., Eysenck, H. J., and Barrett, P. (1985) A revised version of the psychoticism scale. Personality and Individual Differences, 6, 1, 21-29.

Gray, J. A. (1970) The psychophysiological basis of introversion-extraversion. Behavior Research and Therapy, 8, 249-266.

Gray, J. A. (1972) The psychophysiological nature of introversion-extraversion: a modification of Eysenck's theory. In V.D. Neblitsyn and J.A. Gray (Eds.) Biological bases of individual behaviour. London: Academic Press.

Gray, J. A. (1991) Chapter 10: Neural systems, emotion and personality. In J. Madden IV (Ed.). Neurobiology of learning, emotion and affect. New York: Raven Press Ltd.

Hambleton, R. K. (1989) Principles and selected applications of Item Response Theory. In R. L. Linn (Ed.). Educational Measurement, 3rd Edition. MacMillan Publishing.


190

Hambleton, R. K., and Jones, R. (1993) Comparison of Classical Test Theory and Item Response Theory and their Applications to Test Development. Educational Measurement Issues and Practice, Fall, 38-47.

Hambleton, R. K., Swaminathan, H., and Rogers, H. J. (1991) Fundamentals of Item Response Theory. London: Sage Publications.

Hofer, S. M., Horn, J. L., and Eber, H. W. (1997) A Robust five factor structure of the 16PF: strong evidence from independent rotation and confirmatory factorial invariance procedures. Personality and Individual Differences, 23, 2, 247-269.

Iwawaki, S., Eysenck, S. B. G., and Eysenck, H. J. (1977) Differences in personality between Japanese and English. Journal of Social Psychology, 102, 27-33.

Jackson, J. S. H., and Maraun, M. (1996a) The conceptual validity of empirical scale construction: the case of the Sensation Seeking Scale. Personality and Individual Differences, 21, 1, 103-110.

Jackson, J. S. H. and Maraun, M. D. (1996b) Whereof one cannot speak, thereof one must remain silent. Personality and Individual Differences, 21, 1, 115-118.

Kaiser, H. F., Hunka, S., and Bianchini, J. C. (1971) Relating factors between studies based upon different individuals. Multivariate Behavioral Research, 6, 409-422.

Kline, P. (1998) The New Psychometrics. London: Routledge.

Luce, R. D. and Tukey, J. W. (1964) Simultaneous Conjoint Measurement: A new type of fundamental measurement. Journal of Mathematical Psychology, 1, 1-27.

MacCallum, R. C., and Browne, M. W. (1993) The Use of Causal Indicators in Covariance Structure Models: Some Practical Issues. Psychological Bulletin, 114, 3, 533-541.

Maraun, M. D. (1996) Meaning and Mythology in the factor analysis model. Multivariate Behavioral Research, 31, 4, 603-616.

Maraun, M. D. (in press) Measurement as a Normative Practice: Implications of Wittgenstein's Philosophy for Psychological Measurement. Theory and Psychology Journal.

McCrae, R. R., Zonderman, A. B., Bond, M. H., Costa, P. T. Jr., and Paunonen, S. V. (1996) Evaluating replicability of factors in the revised NEO Personality Inventory: Confirmatory factor analysis versus procrustes rotation. Journal of Personality and Social Psychology, 70, 3, 552-566.

Michell, J. (1990) An Introduction to the Logic of Psychological Measurement. Lawrence Erlbaum.

Michell, J. (1994) Numbers as quantitative relations and the traditional theory of measurement. British Journal for the Philosophy of Science, 45, 389-406.

Michell, J. (1997) Quantitative Science and the definition of measurement in psychology. British Journal of Psychology, 88, 355-383 (Commentaries by Kline, Laming, Lovie, Luce, and Morgan, with a reply to all by Michell extend from pages 385-406 in the same issue).


191

Poon, W. Y., Chan, W., Lee, S. Y., and Leung, K. (1993) Preliminary analysis of multiple group structural equation modelling via cluster analysis. Proceedings of the American Statistical Association 1993 Convention, Social Statistics Section, 368-373.

Poortinga, Y. (1984) Interpreting cross-cultural differences in psychophysiological variables. Paper presented at the 2nd Eur. Conf. on Personality Psychology, Bielefeld, F.R.G.

Poortinga, Y. (1995) Use of Tests Across Cultures. In Oakland, T. and Hambleton, R.K. (Eds.) International Perspectives on Academic Assessment. Dordrecht: Kluwer Academic Publishers.

Robinson, D. L. (1982) Properties of the diffuse thalamocortical system and human personality: A direct test of Pavlovian/Eysenckian theory. Personality and Individual Differences, 3, 1-16.

Robinson, D. L. (1987) Chapter 8: A neuropsychological model of personality and individual differences. In J. Strelau, and H. J. Eysenck (Eds.). Personality dimensions and arousal: Part III Neo-Pavlovian concepts of temperament. New York: Plenum Press.

Robinson, D. L. (1996) Brain, Mind, and Behaviour: A New Perspective on Human Nature. New York: Praeger Press.

Stankov, L., and Cregan, A. (1993) Quantitative and qualitative properties of an intelligence test: Series completion. Learning and Individual Differences, 5, 2, 137-169.

ten Berge, J. M. F. (1996) The Kaiser, Hunka, and Bianchini factor similarity coefficients: A cautionary note. Multivariate Behavioral Research, 31, 1, 1-6.

Ter Hark, M. (1990) Beyond the Inner and the Outer. Dordrecht: Kluwer Academic Publishers.

Van der Linden, W. (1994) Fundamental Measurement and the Fundamentals of Rasch Measurement. In M. Wilson (Ed.) Objective Measurement: Theory into Practice Vol. 2. Connecticut, US: Ablex Publishing Corp.

Van de Vijver, F. and Leung, K. (1997) Methods and data analysis of comparative research. In Berry, J. W., Poortinga, Y. H., and Pandey, J. (Eds.). Handbook of Cross-Cultural Psychology, Vol. 1: Theory and Method. Allyn and Bacon.

Van de Vijver, F., and Poortinga, Y. (1994) Methodological issues in cross-cultural studies on parental rearing behavior and psychopathology. In Perris, C., Arrindell, W. A., and Eisemann, M. (Eds.). Parental Rearing and Psychopathology. Chichester, UK: Wiley.

Wright, B. D. (1997) Fundamental Measurement for Psychology. Memo 64, Rasch Measurement Transactions. Online publication at: http://mesa.spc.uchicago.edu/memo64.htm

Zuckerman, M., Joireman, J., Kraft, M., Kuhlman, D. M. (in press) Where do Motivational and Emotional Traits Fit within Three Factor Models of Personality? Personality and Individual Differences.


192

Međukulturna psihometrija: Ajzenkova paradigma, merenje i psihološka nauka

POL T. BERET SIBILA B. Ð. AJZENK

U radu su prikazani polazište, osnovne postavke i strategija dvadesetdvogodišnjih Ajzenkovih istraživanja međukulturnog poređenja četiri "superfaktora" ličnosti merenih Ajzenkovim upitnikom ličnosti (EPQ). Ajzenkov rad je nailazio na kritike, kako metodologije tako i psihološkog smisla. Ipak, skoriji rad Bereta, Petridesa, Ajzenka i Ajzenkove (Barret, Petrides, Eysenck & Eysenck, in press) odgovorio je na ove kritike popravljajući metodologiju poređenja i, shodno tome, pokazao da originalni rezultati koje su saopštili Ajzenkovi ostaju valjani. Premda istraživačka metodologija izgleda u velikoj meri deskriptivna, uočeno je da su model ličnosti i eksperimentalna osnova faktora u skladu sa koherentnim naučnim modelom istraživanja ličnosti koji je Ajzenk predložio 1997 (Eysenck, 1997). Da bi se u potpunosti zadovoljili zahtevi kvantitativne nauke, od suštinske je važnosti da se kvantitativna struktura varijabli ličnosti utvrdi empirijski. U praksi, ovaj zahtev može se ispuniti ako se obezbedi da merne skale budu usklađene sa aksiomatskim principima sadržanim u Rašovom modelu merenja. Praktične implikacije ovog zahteva ispitane su korišćenjem referentnog uzorka podataka EPQ u Velikoj Britaniji. Na osnovu rezultata analize, a u skladu sa Mičelovim (Michell, 1997) postulatima koji se odnose na bitna svojstva kvantitativne nauke, predložen je nov naučni model za međukulturnu psihometriju.

Me`dukul√turna® psihometri®:

paradigma AŸzenka, izmerenie i psihologi~eska® nauka

POL T. BERET SIBILA B. G. AYZENK

V nasto®çeŸ rabote rassmatrivaÓts® ishodn∫e pozicii, osnovn∫e poloèni® i strategi®, na prot®ènii 22 let provodim∫h AŸzenkom issledovaniŸ me`dukul√turn∫h sravneniŸ ~et∫reh "sverhfaktorov" li~nosti, izmerenn∫h li~nostn∫m oprosnikom AŸzenka (EPQ). Rabotu AŸzenka kritikovali, kak v otno{enii metodologii, tak i v otno{enii psihologi~eskogo sm∫sla. Odnako, nedavna® sovmestna® rabota Bereta, Petridesa, AŸzenka i AŸzenkovoŸ (v pe~ati) otvetila na ƒti kritiki, ispravl®® metodiku sravneni®, i, v sv®zi s ƒtim, pokaz∫va®, ~to original√n∫e rezul√tat∫ privedenn∫e AŸzenkom, ostaÓts® dostovern∫mi. Hot® issledovatel√ska® metodika kaèts® v bol√{oŸ stepeni deskriptivnoŸ, zame~aets®, ~to modeli li~nosti i ƒksperimental√na® baza faktov nahod®ts® v sootvetstvii s posledovatel√noŸ nau~noŸ model√Ó issledovani® li~nosti, predloènnoŸ AŸzenkom v 1997 godu. Dl® togo ~tob∫ v polnoŸ mere udovletvorit√ trebovani®m koli~estvennoŸ nauki, suçestvenno va`no koli~estvennuÓ strukturu peremenn∫h li~nosti polu~it√ ƒksperimental√n∫m putem. V praktike, ƒto trebovanie mo`no v∫polnit√ obespe~eniem soglasovannosti {kal izmereni® s aksiomati~eskimi principami, soderàçimis® v modeli izmereni® Ra{ova. Prakti~eskie implikacii ƒtogo trebovani® b∫li issledovan∫ primeneniem referentnogo obrazca dann∫h EPQ v Veliko Britanii. Na baze rezul√tatov analiza, a v sootvetstvii s postulatami Mi~el® (1997) o suçestvenn∫h osobennost®h

Documents

Cross-Cultural Psychometrics: the Eysenck Paradigm ...scindeks-clanci.ceon.rs/data/pdf/0048-5705/1998/... · This paper outlines the background, rationale, and strategy that encompasses