Development of a Scale Dimensions of Coaching Performance€¦ · classes to yield a model of six dimensions of coaching performance. The dimensions explored were (a) team products,

Journal of Sport Management, 1995.9, 194-207 O 1995 Human Kinetics Publishers, Inc.

Dimensions of Coaching Performance: Development of a Scale

Joanne C. MacLean Packianat han ChelZadurai University of Windsor The Ohio State University

The purpose of this study was to define the dimensions of coaching performance for coaches and to develop a scale to measure those dimensions. The literature-based model used in this study espoused the use of three broad categories-behavioral product factors, behavioral process factors related to the task, and behavioral process factors related to maintenance of the organization. Each of these broad categories was further subdivided into two classes to yield a model of six dimensions of coaching performance. The dimensions explored were (a) team products, (b) personal products, (c) direct

~ s k b e h r u r i ~ r s , ( d > i n d i r e c t t a s k b e h a ~ - i ~ r ~ e ) a d m i n i s t r a t k m a ~ ~ ~ ~ ~ r e l a t i s n ~ ~ e n t y ~ n - a d m i n k h a t ~ r - and 363 coaches from Canadian Intercollegiate Athletic Union institutions responded to the coaching performance scale for the purposes of this study. Item-to-total correlations, confirmatory factor analysis, and internal consistency estimates supported the conceptual model and yielded a psychometrically sound Scale of Coaching Performance (SCP).

The personnel evaluation procedures used in organizations have continued to evolve over the last decade due to a shifting business climate that has resulted from intense competition and deregulation (Murphy & Cleveland, 1991). The dynamic, highly mechanized, and increasingly competitive nature of today's g i~bal market has resulted in an organizational environment obsessed with productivity and quality. Effective performance evaluations providing accurate feedback are regarded as critical to the success of both organizations and their employees (Latham & Wexley, 1981; Nathan, Mohrman, & Milliman, 1991). An evaluation is a judgment about quality, value, effectiveness, or impact (Cangelosi, 1991). It might focus on a product, process, or entire organization; however, the assessment of personnel, often termed performance appraisal, is regularly viewed as the most critical step in achieving organizational effectiveness (e.g., Latham & Wexley, 1981; Murphy & Cleveland, 1991; Steers, 1977).

Joanne C. MacLean is with the Faculty of Human Kinetics, University of Windsor, Windsor, ON, Canada N9B 3P4. Packianathan Chelladurai is with the School of HPER, The Ohio State University, Columbus, OH 43210-1284.

Dimensions of Coaching Performance 195

Performance appraisals serve both the individual and the organization. At the individual level, performance appraisals (a) reinforce and sustain good performance andlor improve performance, (b) provide insights into career goals, (c) pinpoint areas of strength and weakness, and (d) suggest training needs. From the organization's perspective, performance appraisals also facilitate informed personnel decisions such as linking rewards (e.g., pay, promotions, etc.) to performance and determining training needs. If these benefits are to be derived, it is necessary to institute a performance appraisal system for all employees. Such a system should include a clear statement of what constitutes "performance," how it will be assessed, who will evaluate it, and methods of providing feedback to the employee (Fisher, Schoenfeldt, & Shaw, 1990; Swan, 1991).

The focus on evaluating performance to enhance productivity is not unique to the business world. A variety of other organizations, including nonprofit agencies, endeavor to produce or provide outstanding products through people. Athletic organizations are among the associations that are justifiably intent on performance achievements and the evaluation of personnel. While there are administrative, coaching, and support personnel contributing to organizational effectiveness of collegiate athletic departments, coaches are the most important contributors to overall effectiveness (Maetozo, 198 1). The importance of coaching is recognized internationally as evidenced by the development of coaching education and accreditation systems in various countries (Gowan & Thompson, 1986; Laberge, 1992; Lyle, 1986; Milne, 1990; Pyke & Woodman, 1986; Rushall & Wiznuk, 1985).

Despite the importance of coaching, it is surprising that much less significance has been placed on evaluating coaches than on evaluating other personnel (Leland, 1988). Coaches often receive little evaluative feedback other than the kind that tends to result solely from game day results. Further, most coaches are evaluated-praised or condemned-by the media and the public. Such evaluations are obviously based on very limited information and focus only on the win/loss percentage of the team under the coach's charge. However, coaching, as a job, has maoy facets, all of which need to be evaluated before a fair judgment can be made about a coach's performance (Norcross, 1986; Sabock, 1985). The few attempts at performance appraisal of coaches have resulted in fragmented sets of criteria (e.g., Knorr, 1989; Norcross, 1986; Sabock, 1985).

Bennice (1990) stated that evaluating coaches is a difficult task traditionally avoided because collecting accurate data and implementing the evaluative process is extremely onerous. Specifically, the inability to define individual appraisal criteria has delayed the development of valid procedures of evaluating collegiate athletic coaches (Leland, 1988).

From the foregoing, it is clear that the identification of appropriate criteria to serve as an operational definition of successful performance is a prerequisite to successful evaluation (Ilgen & Barnes-Farrell, 1984; Latham & Wexley, 1981; Levy, 1989). A comprehensive procedure for identifying criteria representative of a collegiate coaching job would help alleviate many concerns. Accordingly, the purposes of this study were to (a) define the dimensions of coaching performance and (b) develop a scale to measure those dimensions.

Dimensions of Coaching Per$ormance The derivation of the dimensions of performance for any job must begin with defining the domain of performance associated with that task (i.e., all behaviors

MacLean and Cheuadurai

and tasks considered to be important contibutors to the individual's job performance). The domain of performance is identified by investigating the behaviors associated with the job (James, 1973; Latham & Wexley, 1981; Smith, 1976). These behaviors (or criteria) must be (a) job specific, (b) representative of performance that actually differentiates between effective and ineffective work, (c) reflective of all relevant aspects of job performance, and (d) reliable (Ilgen & Barnes-Farrell, 1984).

Buford, Burkhalter, and Jacobs (1988) and many other authors (Deets & Tyler, 1986; Ilgen & Barnes-Farrell, 1984; Murphy & Cleveland, 1991; Torring- ton, Weightman, & Johns, 1989) have argued that job descriptions need to be linked to performance appraisals, a practice historically problematic to the coaching field. While many behaviors are derived directly from the job description, other appropriate yet tangential factors should also be added. For example, the coach's job description may outline the organization's procedures and the fre- quency of team practice sessions. However, it may be necessary to add behaviors reflecting the institution's philosophical orientation that should guide the coach's actions. In the university setting this might involve support for the student- athletes' academic priorities when conflicts arise between attending practice or going to class. Thus, any behavior or behavioral outcome valued by the employer must be specified and formalized as criteria for evaluation of performance. Furthermore, Murphy and Cleveland (1991) emphasized the need to focus on actual behaviors rather than beliefs, values, and/or attitudes and suggested that contextual factors of the environment that might impact the job should be taken into consideration.

Basedon the above arguments, a model of coaching performance (shown in F igu rF I )Ts pToposed. -domain of ~ o ~ a n i w ~ s ~ z e ~ m the two perspectives of the results and/or the activities leading to those results (Cangelosi, 1991; Keeley, 1978; Landy & Farr, 1983; Murphy & Cleveland, 1991). The results of the behaviors associated with the job are labelled behavioral products. For a coach, behavioral products are indicators such as the team's win/ loss record, the team's standings in the various leagues, and the recognition accorded to individual athletes. The actual behaviors of the employee as they are performed are termed behavioral processes. Instead of evaluating the coach based on the team's win/Ioss record, the coach could be assessed on criteria associated with the process of coaching, such as application of good learning/ teaching principles during practice sessions or a demonstrated understanding of fundamental skills and strategies required by the sport. The distinction between behavioral products and behavioral processes is analogous to Latham and Wex- ley's (1981) distinction between cost-related outcomes and behavioral criteria.

It is widely accepted that basing evaluations solely on job outcomes (behavioral products) is inappropriate. Landy and Farr (1983) indicated that an emphasis on results of behaviors exclusively is likely to lead to behaviors that are unethical and dysfunctional for the organization, as maximizing outcomes becomes the only organizational goal and leads to a cutthroat work environment. In the context of coaching, it has been suggested that the extreme emphasis on winning is the causal factor in many violations of recruiting and academic regulations and other abuses (e.g., Knight Foundation Report, 1993; Lapchick & Slaughter, 1989; Porto, 1985).


Team Products

Personal Products

Coachina Performance

Direct Task Behaviors

I I Task Related

Figure 1 - Theoretical model of the dimensions of coaching performance.

Maintenance Related

Further, as Murphy and Cleveland (1991) suggested, too much emphasis on results clouds the issue of whether the individual or the situation is actually under evaluation. It has been recognized that performance in some cases may be constrained by factors (e.g., equipment breakdown, bad luck, etc.) that are beyond the control of the employee (Latham & Wexley, 198 1; Peters & O'Connor, 1980). This particular concern is more pronounced in the case of coaching, because coaches (and their athletes) are engaged in zero-sum games. That is,

Indirect Task Behaviors

Administrative Maintenance Behaviors

-

1

every contest has to produce a winner and a loser no matter how close the contest may be. For example, a team in an NBA contest may be declared a winner on the final score of 116 to 115. That win (or loss) would go in the assessment of that coach's (and team's) performance. Yet, the measure of winfloss could be contaminated by external factors such as random chance (e.g., bad bounce of the ball) and/or an official's wrong call (Courneya & Chelladurai, 1991). If we were to base our assessment of the coach solely on the basis of winlloss, we would be overlooking the contaminating factors and, more importantly, all the other factors that have contributed to the scoring of 115 points (a losing score). Therefore, it is essential that the criteria for evaluation reflect a balance of behavioral products and processes.

I I

Behavioral Process Factors

Public Relations Behaviors

Behavioral Products As noted before, the most popular approaches to evaluating coaches rely on the teams' and/or individual athletes' winfloss record. Despite the deficiencies

198 M & ~ R and CirelIhdumi

associated with results as the performance measure, its importance in evaluating coaches cannot be minimized because the purpose of intercollegiate athletics is the pursuit of excellence (Keating, 1964). From a different perspective, Broyles and Hay (1979) argued that the purpose of athletics is to provide entertainment to relevant publics, and the entertainment value is greatly enhanced through winning teams. Thus, winning is critical to the athletic context. These winboss measures are labelled team products (Figure 1).

One of the unique features of intercollegiate athletic coaching is the extent to which coaches are under public scrutiny. As we noted previously, coaches are constantly judged by the media, the public, and other coaches. Often, organized groups of media personnel, alumni, and/or coaches formally recognize and reward one or more coaches in a particular year or season. Such accolades and awards reflect these external agents' assessment of the coach's expertise and performance. Therefore, they must also be considered by administrators in evaluating a coach, in addition to what the team achieves in athletic competitions. This dimension is labelled personal products (Figure 1).

Behavioral Processes

As we previously suggested, it is important that the processes aimed at achieving the desired results also be considered in evaluating the performance of the coach. These behavioral processes may be directly associated with the tasks of the specific job assigned to an individual, or they may be behaviors associated with the maintenance and viability of the total organization (Astin, 1964, Carroll & Schneier, - -- 1982; - - Fleishman - - & QuaintantanceI1984:Sa1venndyY&Se)rm~u~73)-

The distinqio~between t a s k r e l a t e d - b e h a v i behaviorsx consistent with the literature on organizational effectiveness. Several scholars studying organizational effectiveness have suggested that managers should be concerned about (a) the immediate productivity of their employees and the organization as a whole and (b) the long-term viability and maintenance of the organization (e.g., Cameron, 1986; Chelladurai, 1985; Mott, 1972; Steers, 1977).

Task-Related Behaviors. For the coach, task-related processes include the quality of interaction with athletes, management of practice sessions, teaching skills, the ability to apply appropriate tactics and strategies, and proper utilization of the available talent. These behaviors are called direct task behaviors (Figure 1). Another set of a coach's behaviors relevant to the present context are normally performed off the court. Specifically, they relate to such matters as the recruiting and scouting of opponents. These are labelled indirect task behaviors in' Figure 1.

Maintenance-Related Behaviors. Maintenance-related behavioral processes bring the coach into the larger context of the total organization, its maintenance, and its overall effectiveness. The coach can contribute to the viability of the organization in two significant ways. First, a coach's adherence to rules and regulations, prompt attention to organizational processes, and warm interpersonal relations with peer groups and superiors will contribute to organizational health. These processes are termed administrative maintenance behaviors. In addition, the coach can also contribute to organizational welfare through hisher liaison activities with relevant constituencies (e.g., alumni groups, media, etc.). Such activities are labelled public relations behaviors.


In summary, the proposed model of coaching performance consists of six dimensions operationally defined as follows:

1. Team products: Outcomes of coaching that accrue only to the team or individual athletes comprising it.

2. Personal products: Outcomes of coaching that accrue only to the coach. 3. Direct task behaviors: Application of interpersonal skills and appropriate

strategies and tactics used to enhance the performance of individual athletes and the team as a whole.

4. Indirect task behaviors: Activities such as recruiting, scouting, and applying statistics that contribute indirectly to the success of the program.

5. Administrative maintenance behaviors: Adherence to policies, procedures, and budget guidelines, and interpersonal relations with superiors and peers that strengthen the administration of the whole enterprise.

6. Public relations behaviors: Liaison activities between one's program and relevant community and peer groups.

Development of the Scale of Coaching Pe@onnance

The utility of any conceptual model is dependent on the extent to which the constructs of the model can be reliably and validly measured. Accordingly, the second major purpose of this study was to develop a psychometrically sound scale to measure the above six dimensions of coaching performance. Such an endeavor also served the purpose of empirically verifying the above six-dimen- sional conceptualization of coaching performance. That is, empirical support for the hypothesized subscale structure would be tantamount to confirmation of the proposed model for developing coaching evaluation criteria. The development of the scale, which was carried out in three stages, is reported in the following sections.

Stage One: Item Generation and Validation

After extensive review of the literature, we generated an initial list of 56 criteria to reflect the six dimensions of the conceptual model of coaching performance previoudy outlined. The fit between the criteria and the respective dimensions (content validity) was assessed by a panel of 10 experts. Five of these experts were requested to rate the appropriateness of the criteria assignment to one of the dimensions of coaching performance on a five-point scale ranging from 1 (no for not appropriate) to 5 (yes for appropriate). A minimum rating of 18 out of a possible consensus rating of 25 (5 experts x 5-point rating) or 75% agreement per item was set a priori for accepting the criteria-category fit.

Simultaneonsly, the other five experts were given the 56 items and the definitions of the six dimensions of coaching performance. They were requested to place the criteria within the six dimensions. Items not being placed in the appropriate category by at least four of the five experts were considered for placement in another category, for modification, or for deletion. The panel of experts was also asked to assess the content of the entire questionnaire in terms of clarity and completeness of the criteria items. Based on this feedback, the

2M) MacLean and Chelladumi

wording of the criteria and their placements were suitably modified and all 56 criteria were retained for further analysis.

Following these modifications, each dimension was represented by 6-12 criteria. The order of the criteria appearing in the questionnaire was chosen at random. The items were preceded by the statement, "How important are each of the following criteria in evaluating the job performance of a university coach?" The response format was a seven-point Likert-type scale ranging from not very important (1) to very important (7). A second section requested demographic information.

The instrument was then field tested to establish face validity by submitting it to 10 administrative support staff in university athletics who were outside the frame of the study. The field test was undertaken to identify confusing items, to seek suggestions for improvement, to monitor the time it took to complete the instrument, and to evaluate its overall appearance. Criticism and recommendations were used to revise the instrument.

Stage Two: Pilot Study

The second stage of the scale development was a pilot test to assess empirically the stability of the subscale structure endorsed by the experts. The initial version of the scale was mailed to 65 assistant coaches outside of the sample frame of the study. Of these, 57 coaches (88%) responded to the questionnaire. Because of the summative nature of the subscales, internal consistency (Cronbach's alpha) was initially assessed for each of the dimensions. Cronbach's alpha is a conserva- tive estimatesfJhe actual r e l i ~ l i ~ ~ o f ~ t h e ~ c a l e ~ ~ m n b a c h , l 9 5 ~ ~ ~ p h ~ values ranged-between -69 a n d , 8 5 f o r - a - m e a n o U 8 _ a n h w e ~ e p t e d - a ~ a d e q u ~ ( ~ u n n a l l y , 1978).

In order to assess the test-retest reliability of the instrument, 13 of the 15 subjects who were requested responded to the questionnaire once again after a lapse of two weeks. The test-retest correlations ranged from .79 to .94 for a mean of $6. Thus the instrument was deemed to possess sufficient time-related reliability (Davis, 1971).

Stage Three: Confirmatory Analyses

Subjects. The final stage of the development of the scale consisted of administer- ing the 56-item scale to the total population of coaches and administrators of athletic programs within the Canadian Interuniversity Athletic Union (CIAU). As a precaution against any response set threat to validity (Fraenkel & Wallen, 1990), five versions of the questionnaire establishing various orderings of the criteria items were designed. The questionnaire was then distributed to the entire population of coaches (532) and administrators (87) at the 45 member institutions of the CIAU. Of these, 357 usable responses were received from the coach group for a response rate of 67%, and 77 were received from the administrator group for a response rate of 89%.

Analyses. To verify the stability of the subscale structure of the coaching performance scale, the total sample (N = 434) was divided into six subsamples- two larger samples of coaches and administrators and four smaller samples of female coaches, male coaches, female administrators, and male administrators.


Item-to-total correlations and internal consistency estimates were computed for each of the six subsamples'. An item had to correlate higher with its own dimension than with other dimensions in a majority of the six data sets to be included in that dimension. Also, each of the selected items was checked to ensure the scale alpha did not increase if it was removed.

After three iterations of these processes, 35 of the initial 56 criteria items were selected for the final version of the scale. The selected items, their item- to-total correlations and internal consistency estimates for the coach and administrator samples are presented in Table 1.

The distribution of the items across the subscales was 4 items each for team products, personal products, and public relations behaviors; 5 items for indirect task behaviors; 8 items for administrative maintenance behaviors; and 10 items for direct task behaviors.

As a final check on the subscale structure and construct validity of the scale, the more rigorous process of confirmatory factor analysis was employed. Joreskog and Sorbom's (1986) Linear Structural Relations or LISREL was the program selected for this confirmatory analysis. LISREL, which uses the maximum likelihood procedure, constrains the factor analysis to a priori specifications. LISREL also provides factor loadings, their significance, and several indices of the fit between the data and the prespecified model.

The LISREL factor loadings for all items on their respective dimensions were significant at the .001 level (see Table 1). The coefficient of determination (indicating an overall measure of reliability for the whole model) was .987. A measure of the goodness of fit between the data and the hypothesized factor structure is the chi-square a2) statistic and its associated degrees of freedom (Joreskog & Sorbom, 1986). Large x2 values correspond to bad fit and small x2 values to good fit. Because the x2 values tend to be inflated with higher degrees of freedom (dB, they need to be corrected for those degrees of freedom (Joreskog & Sorbom, 1986). The x2/df was 2.33 (x2 = 1270.7, df = 543, a value much lower than 5 as suggested by Wheaton, Muthen, Alwin, and Summers (1977), thereby indicating a reasonable fit between the model and the data.

The internal consistency estimates ranged from .67 to .87 for a mean of .78 in the data of administrators and from -65 to -87 for a mean of .76 in the data of coaches. These were considered adequate (Nunnally, 1978). The correlations among these six dimensions, presented in Table 2, were all significant and ranged from .23 to .66 with a mean of .44.

Discussion

The results of the study lend strong support to the proposed conceptual model of coaching performance (Figure 1) and the SCP. The psychometric properties of the scale are considered more than adequate. First, the subscale structure was confirmed by both item-to-total correlations and LISREL's confirmatory factor analysis. Second, the internal consistency estimates (Cronbach's alpha) for the subscales were all high, and certainly were higher than the minimum suggested by Nunnally (1978). Furthermore, the subscale structure was maintained consistently across six different subgroups of the sample. Finally, the content of the subscales are meaningful and reflective of the six dimensions of coaching performance.

202 M m ad-Ckmad&

Table 1 I--to-Tow1 Correlations and LISREL ^haclings

Dimensions LISREL Items 0 1 D2 D3 D4 D5 D6 loading

Team products Team winlloss ~ecord .79" .32 .13 .42 .28 .78

.80b .23 .I7 .37 .13 2 3 Making the playoffs .76 .39 .I0 .36 .22 .15 .87

.84 -24 .25 .47 .18 2 9 Impromentofathlete/ .66 .22 .34 .19 .33 -28 .53

team performance .72 .22 4 .42 .35 -39 from previous year

Performance of'individ- .61 -29 .42 .40 .25 .33 .53 ual athletes .72 .25 .42 .34 .30 .29

Personal proffacts Receiving ooaching .38 .70 .17 .22 3 3 .38 .37

awards .24 .66 .05 . l l . l l .19 Writing for publication .28 .85 .10 .23 .34 .52 .56

.I5 .73 .12 .13 .17 .41 Speaking engagements .36 .76 .26 .51 .33 .54 .75

.28 .76 .28 .40 .37 .53 Presenting at player/ .36 .86 .29 .49 .29 .65 .69

--6m&-Cl;n;rz . 2 - 1 r)r) .5 1 -. uzrect task: behaviors

Communicating with .11 .02 .54 .69 .39 .18 .47 athletes .18 .04 .51 .20 .23 .15

Applying sport .30 .20 .70 .43 .50 .40 .50 knowledge .18 .20 .58 .18 .26 .26

Motivating athletes to- .15 .01 .39 .22 .23 .21 .41 ward higher levels of .34 .19 .47 .26 .20 .23 achievement

Utilizing game tactics .36 .27 .79 .42 .44 .26 .66 and strategies .33 .15 .72 .36 .30 .24

Applying conditioning .28 .20 .73 .38 .45 .28 .59 principles .27 .15 .66 .32 .38 .24

Teaching techniques dur- .15 .18 .60 .17 .37 .26 .65 ing practice .29 .23 .69 .31 .33 .26

Making coaching deci- .24 .26 .74 .42 .34 .23 .65 sions during compe- .27 .15 .70 .37 .36 .21 tition

Conducting practice .18 .15 .78 .44 .57 .32 .67 sessions .26 .09 .68 .29 .34 .18

Planning and preparing .23 .24 .78 .46 .44 .34 .59 for the preseason, sea- .21 .14 .67 .38 .36 .32 son, and postseason

Developing game plans .29 .16 .78 .45 .48 -24 .69 3

(continued)


Table 1 (continued)

Items

Indirect task behaviors Making recruiting

contacts Scouting opponents

Establishing a recruiting plan

Recruiting a specific number of athletes

Recruiting quality athletes

Dimensions Dl D2 D3 D4

Administrative maintenance behaviors Monitoring athlete .31 .27 .27 .20

eligibility .24 . l l .28 .33 Adhering to budget .20 .18 .51 .41

.08 .16 .23 .22 Adhering to rules and .05 .03 .40 .27

regulations .11 .05 .37 .17 Working relationships .41 -39 .56 .41

with department staff .18 .28 .35 .25 Purchase of equipment .34 .42 .24 .28

.33 .20 .24 .26 Being on time with .13 .14 .48 .25

paperwork .13 .26 .36 .22 Working relationships .35 .54 .56 .49

with peer coaches .29 .26 .38 .30 Complying with institu- .19 .07 .53 .44

tion's philosophy .09 .19 .26 .26

Public relations behaviors Meeting with high .19 .32 .54 .53

school coaches .20 .43 .33 .50 Conducting summer .27 .56 .05 .43

sports camps .33 .44 .19 .35 Establishing working re- .26 .51 .30 .47

lationships with .31 .30 .24 .42 parents

Involvement with profes- .29 .53 .42 .43 sional associations .26 .45 .31 .30

Internal consistency estimates (Cronbach's alpha) Administrators' data .67 .81 .87 .76 Coaches' data .77 .68 .84 .87

LISREL loading

Note. 'Administrators (n = 77); bCoaches (n = 357). For all LISREL loadings, p < .001.

204 MacLean and Chelladurai

Table 2 Intercorrelations Among the Coaching Performance Dimensions

Dimensions Dl D2 D3 D4 D5 D6

Dl: Team products

D2: Personal products

D3: Direct task behaviors

D4: Indirect task behaviors

D5: Administrative maintenance behaviors

D6: Public relations maintenance behaviors

Note. p < .O1 for all except * p < .05 "Administrators (n = 77); bCoaches (n = 357)

Because each of the dimensions was derived from one central concept- coaching performance-it is not surprising that they were all significantly cone- lated with each-other.This-part ~~ular_resultwould~appear~to~egat~hesix- ,

d ~ e n s i o n a l ~ c ~ c e p t u a l i z & o n ~ o f ~ c o a ~ - tiny, the same results are supportive of the model, for, although the correlations among the subscales were significant, the maximum shared-variance between any two subscales exceeds 40% in only one instance-personal products of coaching with public relations behaviors (12 = .44). In all other cases, the shared variance was much less than 40%. which s u ~ ~ o r t s the conclusion that these

& L

subscales measure sufficiently distinct dimensions of coaching performance. However, it must be noted that although the LISREL goodness-of-fit indices were adequate, they also indicate that the scale could be improved by including more unidimensional items in future revisions.

The present results suggest that the conceptual model and the scale may profitably be used in future coaching evaluation research. Top-level administrators may apply the scale to facilitate their decisions on salary, promotion, and other personnel issues. Insofar as the effectiveness of the entire intercollegiate athletic program is largely a function of the coaching staff's effectiveness, the scale can be used as a significant measure of organizational effectiveness. The scale can also contribute to much of the research in fields such as sport psychology and sport management wherein the dependent variable is coaching effectiveness. An added advantage of the scale is that specific subscales can be related to specific independent variables and/or interventions. Similarly, from a managerial perspective, some decision makers may emphasize one or more of the dimensions based on their value system.

However, it should be noted that the scale measures only the importance attached to the criteria by the administrators and coaches. In this regard, future


research may contribute in two ways. First, the criteria that are in fact used by administrators in evaluating their coaches and the relative weights they attach to these dimensions are to be determined. It must be noted that the statement directing the respondents should be suitably altered if the SCP were to be used for the above purposes.

Second, this study assessed the perceptions of only two constituents of an athletic program-the coaches and athletic administrators. There are other equally, if not more, significant constituencies such as the athletes themselves, the university community, and the administrators of the larger system. In addition, the community at large and the media may also be vitally interested in university athletics. Therefore, it would be necessary to assess their perceptions to improve the model of coaching performance and the measurement thereof, as specified in this paper.

It is also important that the applicability of the SCP to other levels of sports competitions (e.g., professional sport, high school sport) be verified in future studies. For instance, it is not clear if the dimensions of public relations behaviors or team products (winfloss records) would be equally relevant at all levels of competition.

A related concern relates to cross-national and/or cross-cultural imperatives. The subjects of the present study were coaches and administrators from the Canadian university athletic scene. It is not clear if the proposed model and measurement scheme would be applicable to other national settings. For instance, university athletics in the U.S. is on a much larger scale. The involvement of the public, the media, and alumni in the American scene far surpasses that in Canada. This contrast between nations (and possibly cultures) offers a useful avenue for future research.

In summary, the present study descril?ed the development of (a) a literature- based conceptual model comprised of six dimensions of coaching performance and (b) a scale to measure those six dimensions of coaching performance. The psychometric properties of the scale were found to be adequate, lending support to the verity and utility of the model of coaching performance.

References

Astin, A. (1964). Criterion-centered research. Educational and Psychological Measure- ment, 24, 807-822.

Bennice, D.A. (1990, October). Evaluating division III professional staffs. Athletic Admin- istration, pp. 16-18.

Broyles, J.F., & Hay, R.D. (1979). Administration of athletic programs: A managerial approach. Englewood Cliffs, NJ: Prentice Hall.

Buford, J.A., Burkhalter, B.B., & Jacobs, G.T. (1988). Link job descriptions to performance appraisals. Personnel Journal, 6, 132-140.

Cameron, K.S. (1986). A study of organizational effectiveness and its predictors. Manage- ment Science, 32(1), 87-1 12.

Cangelosi, J. (1991). Evaluating classroom instruction. New York: Longman. Carroll, S.J., & Schneier, C.E. (1982). Performance appraisal and review systems: The

identification, measurement, and development of performance in organizations. Glenview, L Scott, Foresman.

206 MacLean and Cheltadumi

Chelladurai, P. (1985). Sport management: Macro perspectives. London, ON: Sports Dynamics.

Coumeya, K.S., & Chelladurai, P. (1991). A model of performance measures in baseball. Journal of Sport & Exercise Psychology, 13, 16-25.

Cronbach, L.J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16, 297-334.

Davis, J.A. (1971). Elementary survey analysis. Englewood Cliffs, NJ: Prentice Hall. Deets, N.R., & Tyler, D.T. (1986). How Xerox improved its performance appraisals.

Personnel Journal, 4, 50-52. Fisher, C.D., Schoenfeldt, L.F., & Shaw, J.B. (1990). Human resource management.

Boston: Houghton Mifflin. Fleishrnan, E., & Quaintance. M. (1984). Taxonomies of human performance: The descrip-

tion of human tasks. New York: Academic Press. Fraenkel, J.R., & Wallen, N.E. (1990). How to design and evaluate research in education.

New York: McGraw-Hill. Gowan, G.R., & Thompson, W.G. (1986, July 18-23). The Canadian approach to training

of coaches: Matching the paradigm. In Bemard Wright (Chair), Coach education: Preparation for aprofession. Proceedings of the W I Commonwealth and Intema- tional Conference on Sport, Physical Education, Dance, Recreation and Health. Glasgow, Scotland.

Ilgen, D.R., & Barnes-Farrell, J.L. (1984). Performance planning and evaluation. In J.E. Rosenzweig & F.E. Kast (Eds.), Modules in Management. Chicago: Science Re- search Associates.

James, L. (1973). Criterion models and construct validity for criteria. Psychological Bulletin, 80, 75-83.

h m - - O L O V l V V l f l , . of i inear ' srrucrurai relarzon- . . - p e s , ana lean squares methods (4th ed.). Mooresville, IN: Scientific Software.

Keating, J.W. (1964). Sportsmanship as a moral category. Ethics, 75, 25-35. Keeley, M. (1978). A contingency framework for performance evaluation. Academy of

Management Review, 3, 428-438. Kerlinger, F.N. (1973). Foundations of behavioral research. New York: Holt, Rinehart, &

Winston. Knight Foundation Report (1993, March). A new beginning for a new century: Intercolle-

giate athletics in the United States. Commission on Intercollegiate Athletics. Knorr, J. (1989). Evaluating athletic personnel. Texas Coach, 34(3), 20-21. Laberge, S. (1992). Employment situation of high performance coaches in Canada. Sport

Canada Occasional Papers, 3(1), entire issue. Landy, F.J., & Fan; J.L. (1983). The measurement of workperformance: Methods, theory,

and applications. New York: Academic Press. Lapchick, R.E., & Slaughter, J.B. (1989). The rules of the game: Ethics in college sport.

New York: Academic Press. Latham, G.P., & Wexley, K.N. (1981). Increasing productivity through performance

appraisal. Reading, MA: Addison-Wesley. Leland, T. (1988, NovemberPecember). Evaluating coaches-Formalizing the process.

Journal of Physical Education, Recreation and Dance, pp. 21-23. Levy, M. (1989). ~lmost-perfect performance appraisals. Personnel Journal, 4, 76-83. Lyle, J. (1986, July 18-23). Coach education: Preparation for a profession. Theme discus-

sion paper (paradigm paper). In Bernard Wright (Chair), Coach education: Prepara- tion for a profession. Proceedings of the VIII Commonwealth and International


Conference on Sport, Physical Education, Dance, Recreation and Health. Glasgow, Scotland.

Maetozo, M.B. (1981). Athletic coaching: Its future in a changing society. Journal of Physical Education and Recreation, 52(3), 40-43.

Milne, C. (1990). Higher education for coaches: Preparation survey results. Journal of Physical Education, Recreation and Dance, 61(9), 44-46.

Mott, P.E. (1972). The characteristics of effective organizations. New York: Harper & Row.

Murphy, K.R., & Cleveland, J.N. (1991). Pelformance appraisal: An organizational perspective. Boston: Allyn & Bacon.

Nathan, B.R., Mohrman, A.M., & Milliman, J. (1991). Interpersonal relations as a context for the effects of appraisal interviews on performance and satisfaction: A longitudi- nal study. Academy of Management Journal, 34(2), 352-369.

Norcross, T.H. (1986, July 18-23). Performance appraisal for coaches. In Bernard Wright (chair), Coach education: Preparation for a profession. Proceedings of the VIII Commonwealth and International Conference on Sport, Physical Education, Dance, Recreation and Health. Glasgow, Scotland.

Nunnally, J.C. (1978). Psychometric theory. New York: McGraw-Hill. Peters, L.H., & O'Connor, E.J. (1980). Situational constraints and work outcomes: The

influence of a frequently overlooked construct. Academy of Management Review, 5, 391-397.

Porto, B.L. (1985). Athletic scholarships as contracts of employment: The rensing decisions and the future of college sports. Journal of Sport & Social Issues, 9(1), 20-37.

Pyke, F.S., & Woodman, L.R. (1986, July 18-23). The education of sports coaches and sports administrators in Australia. In Bernard Wright (chair), Coach education: Preparation for a profession. Proceedings of the VIII Commonwealth and Intema- tional Conference on Sport, Physical Education, Dance, Recreation and Health. Giasgow, Scotland.

Rushall, B.S., & Wiznuk, K. (1985). Athletes' assessment of the coach: The coach evaluation questionnaire. Canadian Journal of Applied Sport Science, 10(3), 157- 161.

Sabock, R.J. (1985). The coach. Champaign, E Human Kinetics. Salvendy, G., & Seymour, W. (1973). Prediction and development of industrial work

performance. New York: John Wiley & Sons. Shultz, F.D. (1979). Toward athletic reform. Journal of Physical Education & Recreation,

50(1), 18-19, 48. Smith, P.C. (1976). Behaviors, results, and organizational effectiveness. In M. Dunnette

(Ed.), Handbook of Industrial and Organizational Psychology (pp. 745-775). Chi- cago: Rand-McNally.

Steers, R.M. (1977). Organizational effectiveness: A behavioral view. Santa Monica, CA: Goodyear Publ.

Swan, W.S. (1991). How to do a superior perjormance appraisal. New York: John Wiley & Sons.

Torrington, D., Weightman, J., & Johns, K. (1989). Effective management: People and organization. New York: Prentice Hall.

Wheaton, B., Muthen, B., Alwin, D.F., & Summers, G.F. (1977). Assessing reliability and stability in panel models. In D.R. Heise (Ed.), Sociological Methodology. San Francisco: Jossey-Bass.

Documents

Development of a Scale Dimensions of Coaching Performance€¦ · classes to yield a model of six dimensions of coaching performance. The dimensions explored were (a) team products,