Trainees Versus Staff: Exploring Counseling Outcomes in a College Counseling Center

Journal of College Student Psychotherapy, 28:229–240, 2014Copyright © Taylor & Francis Group, LLCISSN: 8756-8225 print/1540-4730 onlineDOI: 10.1080/87568225.2014.915172

Trainees Versus Staff: Exploring CounselingOutcomes in a College Counseling Center

GUY ILAGAN and MIKE VINSONThe Citadel, Charleston, South Carolina, USA

JULIA L. SHARP and PAMELA HAVICEClemson University, Clemson, South Carolina, USA

JILL ILAGANPrivate Practice, Charleston, South Carolina, USA

Investigators compared counseling outcomes among nonpaidgraduate-level trainees and professional staff at a collegecounseling center. Counseling outcomes for 331 college studentparticipants were measured using the Outcome Questionnaire 45.2(OQ45.2), employing a pretest–posttest design. The two groups ofservice providers did not differ significantly (p = .179; α > .05).A unique aspect of this study was the statistical control for theclients’ readiness to change. The implications for practice in col-lege counseling centers suggest the use of graduate-level traineesas effective counseling providers and a method to cope with highdemand for services.

KEYWORDS college counseling center, high demand, outcomes,trainees, transtheoretical model

University mental health clinics face high demand for services and limitedresources. In light of the human resource limitations and increasing sever-ity of students’ concerns, effective, efficient, and prompt service delivery ismore important than ever. There is a need to explore affordable ways to bol-ster services for students in need of counseling interventions. One potentialsolution for managing high demands for services is to utilize graduate leveltrainees.

Address correspondence to Guy Ilagan, PhD, School of Education, The Citadel, 332-CCapers Hall, 171 Moultrie Street, Charleston, SC 29409, USA. E-mail: [email protected]

229

mailto:[email protected]

230 G. Ilagan et al.

This study evaluated differences in mean counseling outcomes fortrainees compared to professional staff working in the same collegecounseling center. The outcome of interest was symptom improvement.There were 331 service users, assessed by differences in pretest andposttest administrations of the Outcome Questionnaire 45.2 (OQ45.2).Lambert, Burlingame, and colleagues (1996) reported that significant symp-tom improvement is shown by an OQ45.2 posttest score that is at least14 points lower than the pretest score.

LITERATURE OVERVIEW

A 2012 survey conducted by The Association for University and CollegeCounseling Center Directors (AUCCCD) indicated that high demand for ser-vices, coupled with limited resources, continues to be an obstacle for collegecounseling centers. Of the 400 college counseling center directors who par-ticipated in the study, 32% indicated that their centers need to utilize waitliststo manage the demand (Barr, Krylowicz, Reetz, Mistler, & Rando, 2012).Watkins, Hunt, and Eisenberg (2011) found similarly that increased demandcreated obstacles in providing counseling services to students at collegecounseling centers.

Data from the National Survey of College Counseling Center Directors(Gallagher, 2012) suggest that 2.2 million students in the United States soughtmental health treatment during 2011, and the estimated counselor-to-studentratio was 1 to 1,600 on average (Gallagher, 2012). In addition to the highvolume of students seeking services, an increase in severity of problems wasnoted (Kitzrow, 2009). Most college students who seek campus counselingservices identify as concerns anxiety, depressed mood, and relationships, inthat order (Barr et al., 2012). College counseling center directors reportedan ongoing trend of increased volume and severity, noting that 39% of thestudents seen have “severe psychological problems” (Gallagher, 2012).

One way that many college counseling centers have coped with thisdemand has been to utilize graduate level trainees to provide counseling ser-vices to students. A study conducted at a college counseling center yieldedunexpected and encouraging counseling outcome results for a group oftrainee counselors. Using the OQ45.2 to measure outcomes, Minami and col-leagues (2009) found that clients of the trainees showed greater counselingoutcomes than those of the center’s professional staff. Those investigatorssuggested caution with their findings, however, because clients with moresevere concerns were reassigned from the trainees to professional staff.

Similarly, Nyman, Nafziger, and Smith (2010), presenting findings at auniversity counseling center on outcomes across clinicians’ training levels,employed the College Adjustment Scales (CAS; Anton & Reed, 1991) and theOQ-45 (Lambert, Hansen et al., 1996) to measure client outcomes. On the

Trainees Versus Staff 231

whole, clients experienced moderate symptom relief independent of theclinicians’ training level. The authors reported that their findings contra-dicted basic assumptions that more experienced counselors would be moreeffective in achieving positive counseling outcome scores.

Budge and colleagues (2012), using the Behavioral Health Measure-43(BHM-43; Kopta & Lowry, 2002) and Behavioral Health Measure-20 (BHM-20; Kopta & Lowry, 2002), reported findings from a study conducted at acollege counseling center that showed no statistically different outcomesamong clinicians of differing experience levels. Forand, Evans, Haglin, andFishman (2011) conducted a study on mental health counseling trainees’effectiveness in a cognitive therapy clinic, measuring outcome via the BeckAnxiety Inventory (BAI; Beck, Epstein, Brown, & Steer, 1988) and the BeckDepression Inventory II (BDI-II; Beck, Steer, & Brown, 1996). The resultsshowed that the trainees were as effective as professional clinical staff inthis setting. Based on their findings, those authors asserted that cognitivebehavioral therapy can be provided effectively by trainees.

A few earlier studies compared experience levels of therapists butfocused on outcomes other than symptom reduction or increased reportsof well-being. Those studies painted a weaker picture of trainees’ perfor-mance, showing that the experience of the professional clinical staff affordedthem greater case conceptualization skills, including perceiving significantpatterns, than trainees (Eells, Lombart, Kendjelic, Turner, & Lucas, 2005),as well as having fewer clients discontinue counseling prematurely (Swift &Greenberg, 2012). The earlier studies that focused on symptom reduction didnot find client outcomes for trainees to be less than those of clinical staff,but instead presumed that the trained clinical staff should have clients whoshowed greater outcomes (Lambert, 2005).

METHOD

Participants

The participants were 331 clients of a college counseling center at a pub-lic, medium-sized university in the Southeast during the academic years2007–2009. All counseling center clients who attended between three andseven sessions and were over age 18 were evaluated. The time frame of aca-demic years 2007–2009 was selected because it was then that the researchsite began using the instruments pertinent to this study. Nearly one thirdof the participants lived on campus and 70% were female, which matchedthe demographics for the counseling center clients and the university. Theuniversity mental health clinic conducted over 5,600 counseling sessions peryear in academic years 2007–2008 and 2008–2009. The total number of clientswas 1,910. The study received approval from the institution’s InstitutionalReview Board.


Each counseling case file showed the clinicians’ status as professionalstaff or trainee. The professional staff included 12 professional counselorsholding state licensure, counselors who have graduated and are workingtowards state licensure as professional counselors, social workers and psy-chologists. Approximately one half of the professional staff consisted ofpart-time contractual employees.

The trainees consisted of 16 supervised graduate students enrolled ina masters or doctoral level practicum or internship from counseling or psy-chology programs. Trainees were placed at the research site for either oneor two semesters. The research site did not pay the trainees. The traineeswere required to engage in weekly group and individual clinical supervisionsessions with professional staff serving as supervisors.

Measures

The outcome data were collected from the OQ45.2 (Lambert, Hansen, et al.,1996). The OQ45.2 is a 45-item self-report questionnaire which assesses ther-apeutic change throughout the course of treatment, as well as at termination(Mueller, Lambert, & Burlingame, 1998). The OQ45.2 provides a total scorebased on all 45 items, as well as scores for three subscales—individual, inter-personal, and social role functioning. All items are on a 5-point Likert scale,scored from 0–4, with higher scores indicating higher severity of mentalhealth symptoms.

Pretest and posttest administrations of the OQ45.2 provide an accu-rate measurement of mental health counseling outcome (Vermeersch et al.,2004). To create pretest and posttest OQ45.2 scores, clients either completethe questionnaire at each visit to the mental health clinic, or at the first andlast counseling session (Lambert, 2005). The OQ45.2 has high internal con-sistency, Chronbach’s alpha = 0.93, and test–retest reliability of 0.84 (Miller,Duncan, Brown, Sparks, & Claud, 2003; Whipple et al., 2003).

The participants’ stage of change readiness was indicated by a self-reportitem on the counseling centers’ intake forms. At the time of this study, therewere no documented studies on this item of clients’ self-report of changereadiness, and the authorship of this item was not documented. The itemwas adopted for use in 2007 by a former research site director, who noticedit on the AUCCCD Listserv; queries to the AUCCCD on the authorship or useof the item were not successful.

The client self-report of change readiness item was on the CounselingIntake Form, a self-report questionnaire that all clients completed priorto beginning mental health counseling, and was used to place studentsinto the five ordered Transtheoretical Model stages of change categories:precontemplation, contemplation, preparation, action, and maintenance(Prochaska & Norcross, 2006). The item contains five statements cor-responding to the Transtheoretical Model’s stages of change, with the


precontemplation stage indicating the lowest motivation to change and themaintenance stage indicating the highest motivation to change. This item wasused for all clients starting in August 2007 as a brief alternative (M. Vinson,director of the research site mental health clinic, personal communication,March 2009) to the traditional 32-item Stages of Change Questionnaire, whichwas deemed to be time prohibitive. The client self-report of change readinessitem reads as follows:

Which one of the following statements most accurately characterizesyou?

● As far as I’m concerned, I do not have any problems that I need to change.● I am aware of some problems and am considering beginning to work on

them.● I have worked on problems unsuccessfully but intend to continue trying.● I am currently taking steps to overcome the problems that have been

bothering me.● I have already overcome some problems and want help now to avoid

backsliding.

Procedure

At the first visit to the counseling center, the student participants, like allclients, completed the pretest administration of the OQ45.2. For all clients,the OQ45.2 was then administrated prior to each counseling session, allow-ing the investigators to have OQ45.2 results for the first and last sessions oftreatment.

Data were collected only for students attending at least 3 sessions.Selecting a lower cutoff of two sessions would have allowed only one ther-apy session between the pretest and posttest; the lower limit of three sessionsallowed the outcome data to reflect the initial brief triage session and twofull-length sessions. The mean number of sessions at the research site was6.1. We selected the upper limit cutoff of seven sessions in order to includethe research site’s mean number of sessions as well as mean number ofsessions (6.2) for U.S. university mental health clinics (Gallagher, 2009).

The investigators identified three possible confounds: clients’ readinessto change, percentage of counseling appointments attended, and compul-sory counseling. The data for each of these variables were contained in thecounseling case files.

Clients’ readiness to change is a potential confounding variable becausea client’s stage of change readiness offers insight into whether a universitystudent is likely to benefit from campus mental health counseling (Norcross,Krebs, & Prochaska, 2011). Compulsory counseling is a possible confoundbecause data show that, on average, university students attending com-pulsory counseling do not achieve significant symptom reduction (Mun,


White, & Morgan, 2009). Compulsory attendance was indicated as 1 andnoncompulsory status was assigned 0 in the statistical analyses.

The third variable controlled for in the statistical analyses, the percentageof counseling appointments attended by the study participants, was selectedbecause nonattendance at scheduled counseling appointments is associ-ated with poor clinical outcome (Barrett, Chua, Crits-Christoph, Gibbons, &Thompson, 2008). The percentage of appointments attended was calculatedas a ratio of appointments attended over appointments scheduled.

Statistical Analysis

The hypothesis of the current study was: Counseling outcomes (differencein pretest and posttest OQ45.2 scores) for graduate trainees and for profes-sional staff are similar. Clients’ readiness to change, percentage of counselingappointments attended, and compulsory attendance were controlled for inthe analyses.

Mean group difference in pretest and posttest OQ45.2 scores was theprimary dependent variable. The independent variable was treatment by agraduate trainee or a professional staff member. A multiple regression anal-ysis was conducted to compare counseling outcomes for graduate traineesand professional staff. A significance level of 0.05 was used for all tests ofsignificance. SPSS v 21 (IBM, Armonk, NY) was employed for all analyses.

RESULTS

Table 1 shows the mean difference of OQ45.2 pretest and posttest for allstudy participants was 13.27 (SE = 0.95), slightly below what is considereda significant symptom reduction, that is, an OQ-45.2 posttest score of atleast 14 points lower than the pretest score (Lambert, Burlingame, et al.,1996). Table 1 also shows unadjusted and adjusted group means and stan-dard errors for OQ45.2 means for clients of trainees and professional staff.Although the mean differences (symptom reductions) were slightly greaterfor trainees than professionals, the differences in mean counseling outcomesbetween them, after controlling for clients’ readiness to change, percentageof counseling appointments attended, and students compulsory counselingattendance, were not significant, F(1, 323) = 1.82, p = .179. In otherwords, there was no significant difference in overall symptom improvementbetween clients treated by professional staff and clients treated by graduatetrainees.

Mean changes in pre–post OQ.45.2 scores for the “stage of change” con-trol variable were significant F(4, 323) = 2.58, p = .037 (Table 1). The meanchange in symptom severity was highest for students who indicated they


TABLE 1 Unadjusted and Adjusted Mean Changes and Standard Errors for OQ45.2 UsingReadiness to Change, Percentage of Counseling Appointments Attended, and Students’Compulsory Counseling Attendance as Control Variables

Unadjusted Adjusted+

Staff N M SE M SE

Total 331 13.27 0.95Graduate trainees 184 14.64 1.27 10.36 1.74Professional staff 147 11.56 1.42 7.79 1.75Stage of change∗§

Precontemplationa 19 4.71 3.90 5.42 3.90Contemplationa 122 14.04 1.54 11.28 2.01Preparationb 82 17.24 1.88 14.78 2.20Actiona 83 12.33 1.87 9.74 2.18Maintenancea 25 6.10 3.40 4.16 3.55

Compulsory Status∗

Compulsory 43 6.19 1.63 6.05 1.37Not Compulsory 288 14.33 1.05 12.10 2.63

Note: Symptom improvement is shown by an OQ45.2 posttest score that is at least 14 points lower thanthe pretest score (Lambert, Burlingame et al., 1996).+All means adjusted using the average percentage of appointments attended (67%).§The same superscript letter indicates that the means in those stages do not differ significantly (Fisher’sLeast Significant Difference, p < .05).∗p < .05.

were in the preparation and contemplation stages of change readiness. Thelowest adjusted mean change was observed for students who self-reported asbeing in the maintenance stage. This result was expected, because studentsin the maintenance stage presumably have made many of their changes priorto beginning counseling and sought assistance to maintain their progress.

Mean changes in pretests and posttests on OQ45.2 scores for compul-sory attendance status are also depicted in Table 1. This confounding factorwas significant, F(1, 323) = 4.08, p = .044, meaning that the mean changes inpre–post OQ.45.2 scores were significantly lower for participants attendingcounseling on a compulsory basis. The covariate percentage of counselingappointments attended was not significant, F(1, 323) = 0.026, p = .873.On average, the 331 participants attended 67% of their scheduled appoint-ments. The trainees’ clients attended 72% of their scheduled appointments.Clients of the staff attended 62% of scheduled appointments.

The numbers and percentages of the students in each stage of changereadiness category, for trainees and staff, are shown in Table 2. The datarevealed that 184 (55.6%) of the 331 subjects received counseling fromtrainees versus staff members. The trainees provided counseling to moreclients in the precontemplation stages (53%) than those of the staff (47%).However, a Chi-square test indicated no significant difference in the distri-bution of responses of readiness to change stage for graduate trainees andprofessional staff, χ 2 (4, n = 331) = 1.42, p = .84.


TABLE 2 Frequencies for Each Self-Report Stage of Change among Participants ReceivingCounseling from Trainees and Staff

Number of Cases Percentage of Cases

Client Motivation N Trainees Staff Trainees Staff

Total 331 184 147 56% 44%Precontemplation 19 10 9 53% 47%Contemplation 122 67 55 55% 45%Preparation 82 45 37 55% 45%Action 83 50 33 60% 40%Maintenance 25 12 13 48% 52%

TABLE 3 Frequencies and Relative Frequencies of Participants’ Stage of Change Readinessand Students’ Compulsory Counseling Attendance Status

Motivation Groups nNumber of Compulsory

CasesPercentage of Compulsory

Cases

Precontemplation 19 12 63%Contemplation 122 8 7%Preparation 82 9 11%Action 83 10 12%Maintenance 25 4 16%Total 331 43 13%

Table 3 shows the number and percentage of participants for each ofthe stages of change who were attending campus counseling on a compul-sory basis. Participants in the precontemplation stage, showing the lowestreadiness to change, included the highest percentage of students attendingcompulsory counseling. The proportion of participants that attended compul-sory counseling within the stages of change categories differed significantly,χ 2 (4, n = 331) = 26.64, p < .001. Only 13% (n = 43) of the participantsattended counseling on a compulsory basis, and those participants wereseen by both trainees and staff. Trainees provided counseling to 19 of theparticipants attending compulsory counseling, and staff provided counselingto 24 compulsory clients. The proportion of clients attending sessions on acompulsory basis did not significantly differ between graduate trainees andstaff, Z = 1.61, p = .106.

DISCUSSION AND IMPLICATIONS

The purpose of this study was to examine differences in client outcomesfor supervised graduate-level trainees compared to professional staff. Thecounseling outcome of interest was symptom improvement, as measuredby the difference in pretest and posttest administrations of the OQ45.2. The


mean outcome scores did not differ for the two groups. Participants receivingcounseling from trainees did not have a different mean symptom improve-ment score than students receiving counseling from a professional staffmember. In order to isolate possible confounds, the researchers controlledfor clients’ readiness to change, percentage of counseling appointmentsattended, and students attending compulsory counseling.

What accounts for the similarities in improvement for clients of staff andtrainees? Although the present study did not assess the reasons, other studiescomparing client outcomes attributed similarities to easier client assignmentsfor the trainees (Minami et al., 2009) and to client expectancy factors (Nymanet al., 2010). Thus, Minami and colleagues noted that at some sites clientswith more severe concerns are diverted to professional staff rather than beingassigned to trainees. In the present study, however, there was no such assign-ment based on type of case; trainees engaged in triage with any students whopresented to the clinic. When faced with clients with more severe concerns,trainees were expected to utilize frequent supervision throughout the courseof treatment, rather than referring these clients to professional staff.

It should also be noted that clients’ change readiness, a possible con-found, showed stage of change readiness distributed equally among theclients of the professional counseling staff and trainees. Statistical controlfor change readiness further equalized the effects of client’s readiness tochange as a confounding variable. In other words, a client who was nothighly motivated to change was just as likely to receive treatment from atrainee as from professional staff. Therefore, the lack of difference in treat-ment outcomes between professional staff and trainees cannot be explainedby trainees having “easier” or more motivated clients. The effects of theremaining two confounds, compulsory status and percentage of counselingappointments attended, were also controlled for, ruling these out as possibleexplanatory factors.

Another group of researchers, Nyman and colleagues (2010), suggestedthat equal outcomes for trainees and staff might be attributed to clients oftrainees recognizing trainees’ limitations and therefore restricting discussionsto superficial concerns. While this is theoretically possible, it seems unlikely.Another possibility is that certain students feel more comfortable openingup to counselors closer to their age and life experience. This possibility mayaccount for why some students at the research site were reported to haverequested counseling from a trainee. An exploration of the reasons for theeffectiveness of the trainees compared to staff is an interesting topic forfuture research.

At our research site, certain features may have contributed to thetrainees’ success. Trainees were required to participate in weekly individ-ual and group supervision in addition to weekly group supervision fromtheir graduate programs. These supervisory arrangements, allowing traineesmultiple opportunities to ask questions and express concerns, presumably


contributed to trainees’ treatment success. We acknowledge, like Watkins(2012), that additional research is needed to clarify empirically the idealcomponents and structures of clinical supervision.

One obvious implication of these findings for counseling centers is toconsider the use of trainees to assist in providing effective, prompt and effi-cient mental health service delivery to college students. Utilizing trainees canalso provide an effective method for coping with high demand for services.At the same time, of course, using trainees does require time by qualifiedstaff to provide adequate clinical supervision to the trainees and, of course,available office space.

Trainees can also provide additional services beyond individualcounseling. Supervised trainees can help meet the university students’ needsby providing group counseling or psychoeducational groups pertainingto stress, anxiety, relationships, adjustment to college, and homesickness.Similarly, trainees can be invaluable in assisting counseling centers with out-reach, providing workshops in classrooms, residence halls, student groups,or new student orientation. Workshop topics can include stress-copingskills, how to access university support systems, and a variety of wellnesstopics.

Future research can identify clinical supervision approaches that pro-mote effectiveness among trainees in college counseling centers. One areawhere trainees may need assistance is with the dropout rates of clients, whichaccording to one study exceed those of more experienced staff (Powell,Hunter, Beasley, & Vernberg, 2010). Additionally, both college counselingcenters and graduate counseling programs may benefit from additionaldata on factors that influence trainees’ outcomes in a college counselingcenter.

REFERENCES

Anton, W. D., & Reed, J. R. (1991). College adjustment scales: Professional manual.Odessa, FL: Psychological Assessment Resources.

Barr, V., Krylowicz, B., Reetz, D. R., Mistler, B., & Rando, R. (2012). The 2011Association for University and College Counseling Center Directors annualsurvey. Retrieved from http://files.cmcglobal.com/Monograph_2012_AUCCCD_Public.pdf

Barrett, M. S., Chua, W.-J., Crits-Christoph, P., Gibbons, M. B., & Thompson, D.(2008). Early withdrawal from mental health treatment: Implications of psy-chotherapy practice. Psychotherapy: Theory, Research, Practice, Training, 45,247– 267.

Beck, A. T., Epstein, N., Brown, G., & Steer, R. A. (1988). An inventory for measuringclinical anxiety: Psychometric properties. Journal of Consulting and ClinicalPsychology, 56 , 893–897.

Beck, A. T., Steer, R. A., & Brown, G. K. (1996). BDI–II, Beck Depression Inventory:Manual (2nd ed.). Boston. MA: Harcourt Brace.


Budge, S. L., Owen, J. J., Kopta, S., Minami, T., Hanson, M. R., & Hirsch, G. (2012).Differences among trainees in client outcomes associated with the phase modelof change. Psychotherapy, 50(2), 150–157.

Eells,T. D., Lombart, K. G., Kendjelic, E. M.,Tumer, L. C, & Lucas, C. P. (2005). Thequality of psychotherapy case formulations: A comparison of expert, experi-enced, and novice cognitive-behavioral and psychodynamic therapists. Journalof Consulting and Clinical Psychology, 73, 579–589.

Forand, N. R., Evans, S., Haglin, D., & Fishman, B. (2011). Cognitive behavioraltherapy in practice: Treatment delivered by trainees at an outpatient clinic isclinically effective. Behavior Therapy, 42(4), 612–623.

Gallagher, R. P. (2009). National survey of counseling center directors. Alexandria,VA: International Association of Counseling Services.

Gallagher, R. P. (2012). National survey of counseling center directors. Alexandria,VA: International Association of Counseling Services.

Kitzrow, M. A. (2009). The mental health needs of today’s college stu-dents: Challenges and recommendations. The National Association of StudentPersonnel Administration Journal, 46 , 646–666.

Kopta, S. M., & Lowry, J. L. (2002). Psychometric evaluation of the Behavioral HealthQuestionnaire-20: A brief instrument for assessing global mental health and thethree phases of psychotherapy outcome. Psychotherapy Research, 12, 413–426.

Lambert, M. J. (2005). Early response in psychotherapy: Further evidence for theimportance of common factors rather than “placebo effects.” Journal of ClinicalPsychology, 61, 855–869.

Lambert, M. J., Burlingame, G. M., Umphress, V., Hansen, N. B., Yancher, S. C.,Vermeersch, D., & Clouse, G. C. (1996). The reliability and validity of a newpsychotherapy outcome questionnaire. Clinical Psychology and Psychotherapy,3(4), 249–258.

Lambert, M. J., Hansen, N.B., Umphress, V., Lunnen, K., Okiishi, J., Burlingame,G. M., . . . Reisinger, C. (1996). Administration and scoring manual for the OQ45.2. Stevenson, MD: American Professional Credentialing Services.

Miller, S. D., Duncan, B. L., Brown, J., Sparks, J., & Claud, D. (2003). The OutcomeRating Scale: A preliminary study of the reliability, validity, and feasibility of abrief visual analog measure. Journal of Brief Therapy, 2(2), 91–100.

Minami, T., Davies, D. R., Tierney, S. C., Bettmann, J. E., McAward, S., Averill, L. A.,. . . Wampold, B. E. (2009). Preliminary evidence on the effectiveness of psy-chological treatments delivered at a university counseling center. Journal ofCounseling Psychology, 56(2), 309–320.

Mueller, R., Lambert, M. J., & Burlingame, G. (1998). Construct validity of theoutcome questionnaire: A confirmatory factor analysis. Journal of PersonalityAssessment, 70(2), 248–262.

Mun, E., White, H. R., & Morgan, T. J. (2009). Individual and situational factorsthat influence the efficacy of personalized feedback substance use interventionsfor mandated college students. Journal of Consulting and Clinical Psychology,77(1), 88–102.

Norcross, J. C., Krebs, P. M., & Prochaska, J. O. (2011). Stages of change. Journal ofClinical Psychology: In session, 67(2), 143–154.


Nyman, S. J., Nafziger, M. A., & Smith, T. B. (2010). Client outcomes across counselortraining level within a multitiered supervision model. Journal of Counseling &Development, 88(2), 204–209.

Powell, J. L., Hunter, H. L., Beasley, L. O., & Vernberg, E. M. (2010). Using finegrained indices of therapists’ experience and training to predict treatment out-comes in a university-based training clinic for children and families. Trainingand Education in Professional Psychology, 4, 138–144.

Prochaska, J. O., & Norcross, J. C. (2006). Systems of psychotherapy: Atranstheoretical analysis (6th ed.). Pacific Grove, CA: Brooks-Cole.

Swift, J. K., & Greenberg, R. P. (2012). Premature discontinuation in adult psy-chotherapy: A meta-analysis. Journal of Consulting and Clinical Psychology,80(4), 547–559.

Vermeersch, D. A., Whipple, J. L., Lambert, M. J., Hawkins, E. J., Burchfield, C. M.,& Okiishi, J. C. (2004). Outcome Questionnaire: Is it sensitive to changes tocounseling center clients? Journal of Counseling Psychology, 51, 38–49.

Watkins, C., Jr. (2012). Educating psychotherapy supervisors. American Journal ofPsychotherapy, 66(3), 279–307.

Watkins, D. C., Hunt, J., & Eisenberg, D. (2011). Increased demand for mental healthservices on college campuses: Perspectives from administrators. QualitativeSocial Work, 11(3), 319–337.

Whipple, J. L., Lambert, M. J., Vermeersch, D. A., Smart, D. W., Nielsen, S. L., &Hawkins, E. J. (2003). Improving the effects of psychotherapy: The use ofearly identification of treatment failure and problem-solving strategies in routinepractice. Journal of Counseling Psychology, 50, 59–68.

Copyright of Journal of College Student Psychotherapy is the property of Taylor & FrancisLtd and its content may not be copied or emailed to multiple sites or posted to a listservwithout the copyright holder's express written permission. However, users may print,download, or email articles for individual use.

Documents

Trainees Versus Staff: Exploring Counseling Outcomes in a College Counseling Center