Establishing the predictive validity of a new spatial visualization test

Cheryl A. Cohen, PhD, drcheryl316@gmail.com Diana Baikaratova, PhD, dibairak@vt.edu

Theoretical Motivation

Study Design

Required drawing element  

Available points  Correct hatches (areas 1 and 2 )   2  Hollow areas (areas 3, 4, and 5)   3  

Total score:   5  

Spatial abilities in STEM fields Spatial thinking skills contribute to performance in many STEM

disciplines. There is natural variation in spatial skill among normal populations, putting some students at risk for low performance in STEM courses.

Recent evidence supporting the malleability of spatial skills (Uttal et al., 2013) has convinced educators that spatial abilities are worthy of systematic identification and nurturance (National Research Council, 2006). There is a need for reliable and valid assessments of spatial skills that are relevant to STEM disciplines.

Spatial visualization Spatial visualization is defined as the ability to encode and maintain

spatial information in working memory while transforming it (Carroll, 1993). Identifying the 2D cross section of a 3D geometric solid is a spatial visualization task that predicts performance in STEM fields, including biology (Rochford, 1985), engineering (Sorby, 2009) and geology (Orion, Ben-Chaim, & Kali, 1997).

Measuring spatial visualization skill The Santa Barbara Solids Test (SBST) is a new multiple-choice

instrument that measures the ability to identify 2D cross-sections of 3D geometric figures. The 30 SBST figures vary in complexity. These variations in figural complexity are assumed to correlate with varying demands on spatial working memory (Miyake et al., 2001). A previous study (Cohen & Hegarty, 2007) established that the test has satisfactory internal reliability (α = .86), and that it shared significant positive correlations with measures of mental rotation (Vandenberg & Kuse, 1978) and change in view perspective (Eliot & Smith, 1983).

Research question Does the SBST predict the accuracy of sectional views of 3D objects drawn by students in an introductory design engineering course?

A correlational study investigated if the SBST predicted scores on sketches of sectional views of objects drawn by students in an introductory engineering design class. One hundred and twenty-one upper division students (M = 110; F = 11) at the University of Oklahoma took the SBST and the Purdue Spatial Visualizations Test: Revised (PSVT: R) at the beginning of the Spring 2015 semester. During the semester, students received instruction in sketching sectional views of 3D objects and completed three drawing activities.

SBST scales and subscales The SBST has two scales and five sub-scales. Orientation of cutting plane o  Orthogonal cutting planes o  Oblique cutting planes

Spatial Measures

Over the semester, students completed three sets of drawing activities (a total of 14 problems). In each drawing problem, students were asked to sketch a specified section of a mechanical object, as shown in Drawing 5, at left. A coding rubric was developed for each drawing problem.

Engineering Drawings

Results & Implications

Purdue Spatial Visualization Test: R

The PSVT is a 30-item measure of mental rotation skill. Test items require the participant to mentally rotate a line drawing of a 3D object. A sample problem is shown at right.

Santa Barbara Solids Test (SBST) The SBST comprises 30 geometric figures, built from five primitive geometric solids.

As shown in the example at left, each SBST problem shows a geometric figure intersected by a cutting plane. The participant chooses the 2D shape that would result when the geometric figure is s cut by the plane.

**correlation is significant at the p<.01 level *correlation is significant at the p<.05 level

Table 1. Descriptive statistics for drawing activities and spatial tests

Inter-rater reliability: In a sample of 465 drawings, inter-rater reliability based on judgements by two scorers ranged from .93-1.00 across 5 drawings.

Implications •  Total SBST score: There were moderate positive correlations

between SBST total score (30 problems) and 2 of 3 sets of engineering design drawing activities (4-5 problems each), suggesting that aggregate SBST score is a good predictor of the accuracy of student engineering design drawings. o  Total SBST score at the beginning of a semester could

identify students who may benefit from targeted remediation.

•  SBST sub-scales: There were 18 (out of a possible 25) significant positive correlations between the five sub-scales of the SBST (simple, joined, embedded, orthogonal and oblique) and individual drawing problems. o  SBST subscales could be used to diagnose specific types

of difficulties (e.g., difficulties changing view perspective for oblique cutting planes) encountered by students learning engineering drawing.

•  SBST measures a skill distinct from mental rotation.The partial correlations of the SBST subscales, controlling for PSVT:R suggest that mental rotation ability is not the sole determinant of performance on the SBST. These results suggest that the SBST measures a skill that is somewhat distinct by that measured by the PSVT:R.

Geometric complexity o  Simple figures: one primitive solid o  Joined figures: two primitive solids joined at their edges o  Embedded figures: one primitive solid enmeshed inside another

N Min Max M SD Drawing activity 1 (5 drawings) 36 1.70 5 3.41 .77 Drawing activity 2 (4 drawings) 78 1.75 5 4.21 .81 Drawing activity 3 (5 drawings) 73 1.10 5 4.19 .92 SBST (30 problems) 121 3.20 30 23.98 6.10 PSVT: R (30 problems) 72 10 30 22.11 5.75

Drawing Activity 3  PSVT: R

(30 problems) SBST

(30 problems) Simple Figures

Joined Figures

Embedded Figures

Orthogonal

Figures Oblique Figures

Drawing 1   .42*   .34**   .30*   .25*   .36**   .28*   .35**  Drawing 2   .44**   .48**   .44**   .37**   .49**   .53**   .38**  Drawing 3   .42*   .29*   .32**   .20   .28*   .32**   .24*  Drawing 4   .21   .16   .14   .07   .23   .17   .13  Drawing 5   .45**   .30*   .34**   .26*   .22   .23   .32**  

Table 3: Individual drawing scores x PSVT:R total score, SBST total score and SBST subscales

References Carroll, J.B. (1993). Human cognitive abilities: A survey of factor analytic studies. Cambridge University Press Cambridge; New York. Cohen, C. A., & Hegarty, M. (2007). Sources of difficulty in imagining cross sections of 3D objects. In D. S. McNamara, & J. G. Trafton (Eds.), Proceedings of the Twenty-Ninth Annual Conference of the Cognitive Science Society (pp. 179 -184). Austin TX: Cognitive Science Society. Eliot, J. & Smith, I. M. (1983). An international directory of spatial tests. Windsor Berks: Nfer-Nelson. Miyake, A., Rettinger, D. A., Friedman, N. P., Shah, P & Hegarty, M. (2001). Visuospatial working memory, executive functioning and spatial abilities. How are they related? Journal of Experimental Psychology: General, 130, 621-640. National Research Council (2006). Learning to think spatially: GIS as a support system in K-12 curriculum. Washington: National Research Council Press. Orion, N., Ben-Chaim, D. & Kali, Y. (1997). Relationship between earth science education and spatial visualization. Journal of Geoscience Education 45: 129-132. Rochford, K. (1985). Spatial learning disabilities and underachievement among university anatomy students. Medical Education, 19, 13-26. Sorby, S. (2009). Educational research in developing 3D spatial skills for engineering students. International Journal of Science Education, 31(3), 459–480. Uttal, D. H., Meadow, N. G., Tipton, E., Hand, L. L., Alden, A. R.,Warren, C., et al. (2013). The malleability of spatial skills: A meta-analysis of training studies. Psychological Bulletin,139, 352–402. Vandenberg, S. & Kuse, A. (1978). Mental rotations: Group test of three-dimensional spatial visualization. Perceptual and Motor Skills, 47, 599-604.

Subscales Simple Joined Embedded Orthogonal Oblique Bivariate correlations Joined .77** Embedded .68** .79** Orthogonal .84** .85** .81** Oblique .83** .90** .86** .73** PSVT: R .44** .50** .59** .50** .55**

Partial correlations controlling for PSVT: R Joined .73** Embedded .56** .59** Orthogonal .80** .79** .72** Oblique .82** .84** .75** .65** ** correlation is significant at the p<.01 level

On average, participants answered approximately 80 percent of the SBST problems and 80 percent of the PSVT: R problems correctly.

As shown in the top of Table 2, correlations among the five subscales of the SBST were medium to large, indicating that the subscales of the SBST measure a common ability. The bottom half of Table 2 shows partial correlations among the SBST subscales, holding constant the PSVT: R. If mental rotation ability, as measured by the PSVT: R is the only ability measured by the subscales of the SBST, the partial correlations should not be significantly different from zero. As shown above, the partial correlations were significantly greater than zero.

Coding rubric for Drawing 5

Table 2. Bivariate and partial correlations between SBST subscales and PSVT: R

Individual drawing problems in Drawing Activity 3