Feasibility and effectiveness of computer-based therapy in community treatment

Journal of Substance Abuse Treatment 39 (2010) 227–235

Regular article

Feasibility and effectiveness of computer-based therapy incommunity treatment

Adam C. Brooks, (Ph.D.)a,⁎, Deanna Ryder, (M.A.)a,Deni Carise, (Ph.D.)a,b,c, Kimberly C. Kirby, (Ph.D.)a,c

aTreatment Research Institute, Philadephia, PA 19106bPhoenix House, New York, NY 10023

cDepartment of Psychiatry, University of Pennsylvania School of Medicine, Treatment Research Center, Philadelphia, PA 19104

Received 21 January 2010; received in revised form 14 May 2010; accepted 4 June 2010

Abstract

Computerized therapy approaches may expand the reach of evidence-based treatment; however, it is unclear how to integrate thesetherapies into community-based treatment. We conducted a two-phase pilot study to explore (a) whether clients' use of the TherapeuticEducation System (TES), a Web-based community reinforcement approach (CRA) learning program, would benefit them in the absence ofcounselor support and (b) whether counselors and clients would use the TES in the absence of tangible research-based reinforcement. InPhase 1, clients in the TES condition (n = 14) demonstrated large improvements in knowledge, F(1, 20) = 8.90, p = .007, d = 1.05, and weresignificantly more likely to select CRA style coping responses, F (1, 20) = 11.95, p = .002, d = 1.16, relative to the treatment-as-usual group(n = 14). We also detected small, nonsignificant, between-group effects indicating TES decreased cocaine use during treatment. In Phase 2,counselors referred only around 10% of their caseload to the TES, and the modal number of completed modules in the absence of tangiblereinforcement was three. Computer-based therapy approaches are viable in community-based treatment but must be integrated with incentivesystems to ensure engagement. © 2010 Elsevier Inc. All rights reserved.

Keywords: Computer-assisted; Behavior therapy; Cocaine; Coping skills; Community reinforcement approach

1. Introduction

Many psychotherapeutic approaches (particularly cogni-tive and behavioral approaches) have been computerized oraugmented with computer-based didactics components toincrease the reach of evidence-basedmental health treatments(Spek et al., 2007; Taylor & Luce, 2003; Tumur, Kaltentha-ler, Ferriter, Beverley, & Parry, 2007), to improve the cost-effectiveness of treatment (McCrone et al., 2004), and todecrease therapist contact time (Wright et al., 2005). Anumber of computerized andWeb-based approaches recentlyhave been applied in substance abuse treatment settings and

⁎ Corresponding author. Treatment Research Institute, 600 PublicLedger Building, 150 S. Independence Mall West, Philadephia, PA 19106.Tel.: +1 215 399 0980; fax: +1 215 399 0987.

E-mail address: [email protected] (A.C. Brooks).

0740-5472/$ – see front matter © 2010 Elsevier Inc. All rights reserved.doi:10.1016/j.jsat.2010.06.003

hold tremendous potential for expanding the reach ofevidence-based treatment. Web-based approaches havebeen demonstrated to be efficacious in the delivery ofempirically supported nicotine cessation strategies (Glenn &Dallery, 2007; Shiffman, Paty, Rohay, Di Marino, &Gitchell, 2001; Strecher, Shiffman, & West, 2005), motiva-tional enhancement strategies (Ondersma, Chase, Svikis, &Schuster, 2005; Ondersma, Svikis, & Schuster, 2007),moderation management (Hester, Squires, & Delaney,2005; Squires&Hester, 2004), andHIV prevention strategies(Marsch & Bickel, 2004). In addition, three separate researchgroups have shown the benefit of using a computerizedapproach to behavioral therapies (Bickel, Marsch, Buchhal-ter, & Badger, 2008; Carroll et al., 2008; Kay-Lambkin,Baker, Lewin, & Carr, 2009).

Computerization of behavioral psychosocial approachesoffers many advantages to substance abuse treatment.

http://dx.doi.org/10.1016/j.jsat.2010.06.003



mailto:[email protected]


228 A.C. Brooks et al. / Journal of Substance Abuse Treatment 39 (2010) 227–235

Behavioral treatments with proven efficacy, such as thecommunity reinforcement approach (CRA; Roozen et al.,2004) and cognitive–behavioral therapy (CBT; Dutra et al.,2008), often are technically complicated for communitycounselors to learn (Morgenstern, Blanchard, Morgan,Labouvie, & Hayaki, 2001). Despite intensive training,Morgenstern, Morgan, McCrady, Keller, and Carroll (2001)found that client outcomes did not differ between CBT andtreatment-as-usual (TAU) when CBT was delivered bytrained community clinicians. Computerized treatmentapproaches offer a range of solutions to the trainingproblem, from stand-alone programs that clients can engagein on their own as a supplement to the treatment program(Bickel et al., 2008; Carroll et al., 2008) to programs thatfunction as a “clinician extender” by performing the time-consuming task of teaching the repetitive didactics of CBTwhile leaving the task of application of principles up totrained counselors (Bickel et al., 2008; Kay-Lambkin et al.,2009). Computerized treatment approaches also offer theadvantages of assured standardization of presentation andthe option to seamlessly integrate video examples andlearning acquisition checks (quizzes and application assign-ments), and are likely ultimately to make substance abusetreatment more cost-effective (Bickel et al., 2008).Furthermore, Web-based approaches could greatly expandthe reach of treatment outside of specialty care clinics intoother settings and across borders. The means and strategiesby which these CBT approaches have been tested suggestfuture utility but also raise questions about their effective-ness in community settings.

1.1. Review of major studies

Carroll et al. (2008) tested a six-session computer-basedtraining in cognitive–behavioral therapy (CBT4CBT) as anadjunct to usual care within one community-based treatmentclinic. Counselors received no training for interacting withthe clients around their CBT learning. The results of thisexperiment were impressive; clients receiving CBT4CBTexhibited superior within-treatment and posttreatment sub-stance use outcomes (Carroll et al., 2008). These superioroutcomes were demonstrated to be durable out to 6 months(Carroll et al., 2009). Furthermore, clients in the CBT4CBTcondition exhibited improved coping skill acquisition, andthese improved coping skills mediated the effects of thetreatment (Carroll, 2008).

Bickel et al. (2008) tested the application of theTherapeutic Education System (TES) during opiate-replace-ment treatment to deliver the bulk of the didactic portion of aCBT known as CRA. The TES is a fluency-based learningprogram in which patients are exposed to short (10–12minutes) learning modules and then tested on timedrecognition and recall tasks with feedback until theyoverlearn core concepts. Patients were randomly assignedto (a) TAU, (b) therapist-delivered (TD) CRA + abstinence-based vouchers, or (c) TES-delivered CRA + abstinence-

based vouchers with a brief biweekly therapist check-in.Clients in both groups receiving CRA + vouchers evidencedsignificantly more weeks of urinalysis-verified continuousduring-treatment opiate-abstinence compared to TAU;however, there were no differences between the two CRAconditions on abstinence or therapeutic alliance, indicatingthat the TES was able achieve equitable outcomes withmarkedly reduced counselor involvement.

Similarly, Kay-Lambkin et al. (2009) compared a briefintervention with nine sessions of computer- versus TDCBT for people (n = 67) with comorbid depression andalcohol/cannabis use problems. The computer-delivered(CD) treatment was supplemented by brief therapist supportaveraging 12 minutes per session in comparison to theaverage 60 minutes per session in the TD condition. Thetwo CBT interventions produced similar outcomes for bothdepression and substance abuse measures, although the CDtreatment saved approximately 79% of therapist time overthe course of the trial. The research in computerized CBTfor substance abuse treatment poses new questions regard-ing how these therapies can be optimally integrated intocommunity-based treatment and how effective they will beif they are only partially implemented. Partial implementa-tion can occur with newly introduced approaches (e.g.,Miller & Mount, 2001), and computer-based technologiesmay be especially prone to partial implementation ifcounselors are not trained in how to integrate it into theirclinical practice. Computer-based therapies could beimplemented providing that treatment clinics are willingand able to invest in modest computer laboratories or mediacenters. With the Web-based approaches, clients could alsoaccess didactic portions of their treatment away from theclinic. Most of these approaches may also involve sometraining for counselors who monitor clients and assist in theapplication of learned principles (CBT4CBT is a provenexception, and the TES was also conceptualized as aprogram that could benefit clients without the involvementof a counselor). Although these requirements pose chal-lenges, they are in reach of well-organized clinics and couldreturn benefits associated with expanded reach andimproved treatment outcomes.

However, as with any new innovation, clinics mayattempt to employ computer-based interventions with little tono counselor training in the empirically based approachesunderpinning them, and thus, they may be only partially orimperfectly implemented. Under these real-world condi-tions, would clients (including those with limited educationand exposure to computer technology) still benefit fromthese computerized therapeutic approaches? How and towhat extent would counselors employ computerized thera-pies if they were provided with minimal training in their use?Finally, would patients use the systems given the typicaltypes of reinforcement that community counselors couldprovide for their use?

We conducted a two-phase pilot study to explore thesequestions. We selected the TES (Bickel et al., 2008) as the

229A.C. Brooks et al. / Journal of Substance Abuse Treatment 39 (2010) 227–235

computerized learning system to test because it is Web-basedand thus easily updated and accessed from the client's homeand it covers a broad sampling of therapeutic material andthus might be attractive to program directors. In the firststudy phase, we examined the effectiveness of TES plusreinforcement for completing modules, a context similar tothat under which its efficacy has been demonstrated, andcompared it to a TAU control group. In the second phase, weexamined client and counselor utilization of the TES withoutreinforcement; a condition more similar to those incommunity-based clinics.

2. Phase 1 method

2.1. Participants

Participants were recruited from an urban dual-diagnosisdrug and alcohol treatment facility that offered a continuumof care including Intensive Outpatient Program (IOP) andOutpatient Program (OP) services, onsite psychiatric con-sultation and treatment, on-site vocational training, art andmusic therapy, limited medical services, subsidized lunches,fitness facilities, and a computer laboratory with high-speedInternet access. The program operated from a cognitive–behavioral (CB) orientation, although not all counseling staffhad been formally trained in CBT.

Patients who were new treatment intakes reporting cocaineas a primary drug of choicewere referred to the study after theyhad attended their outpatient program for 1 week. Theyprovided consent for screening and were eligible to participatein the trial if they reported using cocaine at least once in the last6 weeks and met criteria for cocaine abuse or dependence (asmeasured by the Structured Clinical Interview for Diagnosticand Statistical Manual of Mental Disorders, Fourth Edition;First, Spitzer, Gibbon, & Williams, 2002).

2.2. Procedures

Eligible candidates provided informed consent to partic-ipate in the study and completed the baseline assessment.Participants were paid $10 for completing baseline measuresand an additional $8 for the urine sample. All participantsentering the study continued to receive standard treatmentservices provided by the clinic and were randomized into oneof two groups: (a) TES with cash incentives paid contingentupon module completion (TES) or (b) cash paymentsdelivered on a schedule that was yoked to that of a TESparticipant (yoked control [YC]). These participants were notexposed to the TES; the yoked cash payments were providedto keep all conditions between groups equivalent with theexception of TES exposure. Randomization was stratifiedbased on status of baseline urine sample so that cocaine-positive and cocaine-negative participants were placedequally into the two groups and YC participants wereyoked to TES participants that had the same baseline urineresult. Because some participants needed to complete the

TES condition first (to determine the voucher deliveryschedule for YC participants), the first three participantswere automatically assigned to TES.

2.2.1. TES groupClients assigned to the TES group could complete up to

48 modules on the TES during an 8-week period. The TES(developed by HealthSim, LLC) is an interactive, self-directed version of CRA (Budney & Higgins, 1998) thatprovides multimedia modules, including relapse prevention,HIV/STD prevention, and psychosocial functioning (rela-tionship and communication skills) and has been describedin detail elsewhere (Bickel et al., 2008). The information ispresented in the form of text, illustrations, and video. Thecontent is provided using a “fluency-based” instructionalapproach grounded in the “precision teaching” model(Binder, 1996). This model continually assesses patientacquisition of the material and adjusts the pace and level ofrepetition of material to promote instant, accurate responding(i.e., fluency) in the relevant skills and information. Weordered the TES modules into “tiers” to present essentialconcepts first (e.g., functional analysis and coping skills),then relationship and communication skills, employment andleisure skills, and HIV knowledge and prevention skills. Atthe conclusion of each module, the participant's knowledgeacquisition was tested with fluency-based recognition andrecall tasks.

Each week, TES participants were scheduled to visit theTES laboratory three times in the following week.Participants were able to reschedule visits if they gave theresearch assistant (RA) prior notice. At each visit, TESparticipants could complete up to two TES modules in thepresence of an RA who provided help if they had troublelogging on or had difficulty spelling words on the recallportion of the fluency-based learning tasks. The RA kept alog of the type and frequency of assistance provided toparticipants. At each visit, the RA confirmed that TESparticipants completed the requisite two modules and paid $8cash for each completed module.

2.2.2. Yoked control groupThe YC participants received cash during an 8-week

period based upon a TES participant with whom they wereyoked. They were told that they would receive incentivesbased upon a predetermined pattern but were unaware ofyoking procedures. When they arrived at their scheduledsession, they received $0, $8, or $16, depending on theearnings of their yoked participant at that visit. Participantswho missed visits without rescheduling within that studyweek lost the chance to be compensated for the missed visit.

2.3. Outcomes

Participants completed an assessment at baseline andagain at a 10-week postbaseline (2-week posttreatment)follow-up. These assessments included a measure of client


demographics, treatment history, and experience withcomputers, as well as self-report and biological measuresof drug use, measures of CRA/HIV knowledge and CRAskills in response to high-risk cocaine-related situations. Inaddition, at baseline participants also provided demographicinformation and contact information to allow follow-up forthe intent-to-treat study design and at the 10-week follow-up,they additionally completed a satisfaction measure regardingthe treatment learning experience. During the 8-weekexperimental intervention, participants provided weeklyurine samples on a randomly determined visit as well asself-reported cocaine use during the past 7 days.

2.3.1. CRA/HIV KnowledgeA 30-item multiple-choice quiz (CRA/HIV Quiz) was

developed by selecting items from the TES modules usedin the study; the quiz reflected basic CB skills, commu-nication skills, and HIV prevention knowledge that couldbe expected to be taught in the CBT-oriented communitytreatment program.

2.3.2. Coping skillThe Cocaine Risk Response Test (CRRT: Carroll, Nich,

Frankforter, & Bisighini, 1999) was used to assess thedevelopment of CB skills to manage situations where theparticipant is at higher risk for using cocaine. It presentsparticipants with 10 hypothetical risk situations and asks themto respond verbally and to give as many responses as possible.Responses are audiotaped and later coded across six indicesaddressing latency (speed of response), number of plans,quality of best response, quality of overall responses,categorization of response, and specificity of coping plans. Ithas very good interrater reliability and internal consistency(Carroll et al., 1999).

An RA blind to both study condition and study hypotheseswas trained to code CRRT responses using Carroll et al.'s(1999) coding system. We also developed a simple five-category system based on Carroll's original categories:participant responses could be categorized as using a response(participant clearly states they would use cocaine), a poorresponse (participant states they would not use but gives nostrategy for avoiding or coping), a generalized response(participant indicates theywould leave the situation), a 12-Stepresponse (participant indicates they would call a sponsor,attend ameeting, or read 12-Step literature), or aCBT response(participant indicates they would employ CB skills, or seeksupport other than 12-Step). Interrater reliability was checkedby having the first author, blind to study condition but nothypotheses, code a randomly selected 25% of the CRRTs.Interrater reliability was very good with intraclass correlations(continuous responses) for overall quality of responses, qualityof best responses, and ranked categorizations of responseranging from 0.84 to 0.91 and Cohen's kappa (categoricalresponses) for specificity of response, number of coping plansoffered, and categorization of responses into five types ofresponses ranging from 0.62 to 0.81.

2.3.3. Drug useSelf-report of drug use was collected at baseline to

establish eligibility and severity of use using the Time-LineFollow-Back procedure (TLFB: Sobell et al., 1980), astructured calendar interview that helps patients recall theirsubstance use over a specified period. Self-reported druguse was assessed for the 42 days (6 weeks) prior to thebaseline assessment.

Self-reported cocaine use during the intervention wascollected weekly using the Cocaine Use Inventory (CUI)modeled after the TLFB procedure, which asks the patient toreport on number of days during the past 7 days usingcocaine, as well as dollar value of cocaine used on largestusing day and average dollar value of cocaine used duringthe week. The CUI has been used in pharmacotherapy trialsto assess during treatment drug use (Nunes, Rothenberg,Sullivan, Carpenter, & Kleber, 2006).

Finally, participants were scheduled to provide eightweekly urine samples that were temperature-tested toensure veracity, then analyzed onsite for the cocainemetabolite benzoylecgonine using Uritox Single PanelQuick-Tests. An RA collected the urine specimen on arandomly selected day each week. During treatment,cocaine use was tallied by coding weeks as using weeksand nonusing weeks based on intersection of urine resultsand patient self-report. Missing urine samples were codedas positive.

2.3.4. Satisfaction measureThis visual analog scale consisted of six questions asking

participants to rate how (a) interesting, (b) satisfying, and (c)useful their counseling educational experience was and theextent to which they thought their educational experience (d)provided them with new information and (e) clarifiedmisconceptions they had, (f) satisfied they were. For eachquestion, individuals indicated their response on a 100-mmline, where a score of 0 referred to “none/not at all” and ascore of 100 referred to “highly/a great deal.” This visualanalog scale has been used in previous studies of the TES(Marsch & Bickel, 2004).

2.4. Data analysis

We conducted chi-square and t-test comparisons onbaseline variables related to demographics, cocaine useseverity, and experience with computers to verify equiva-lence of groups. Groups were compared on CRA/HIVknowledge acquisition, CRA skill (CRRT), and drug useusing repeated measures analyses of variance (ANOVAs).We calculated size of effect for knowledge and skillsacquisition and urine results using Cohen's d. To determinewhether clients that mastered the skills taught by the TESwould have better treatment outcomes, we examined therelationship between accuracy rate (the average percentageof correct answers participants offered on TES quizquestions) and how many modules they completed, as well

Table 1Baseline descriptive characteristics

Characteristics TES (n = 14) YC (n = 12)

DemographicsMale, n (%) 6 (43) 7 (58)Race, n (%)African American 14 (100) 11 (92)White 0 (0) 1 (8)

Age, M (SD), years 42.29 (9.81) 44.08 (8.84)Unemployed (%) 86 (11) 100 (12)Highest level of education, M (SD), years 11.69 (1.03) 10.91 (1.51)Computer experience, n (%)Any use 5 (36) 5 (42)Owned a computer 1 (7) 1 (8)Access to Internet 3 (21) 4 (33)Baseline cocaine use and previous treatment experienceNegative urinalysis at baseline (%) 69 82TLFB self-report past 6 weeksAverage days using, M (SD), days 7.21 (8.13) 8.17 (7.83)Largest dose, M (SD), $ 216.43 (269.63) 202.50 (179.35)Average use, M (SD), $ 114.93 (142.07) 134.00 (107.25)

Previous treatment episodes (M/SD N) 2.54 (1.61) 2.58 (1.31)


as the proportion of cocaine-positive urine samples providedusing bivariate and partial correlation coefficients.

3. Phase 1 results

A total of 51 clients were screened for eligibility for thestudy: 23 were excluded because they did not self-reportusing cocaine within the last 6 weeks. The remaining 28clients were consented into the study. Two consented YCclients did not complete baseline measures or participate inthe study intervention, and were dropped from all analyses.Table 1 provides differences between the TES and YCgroups on demographics, previous computer experience, andbaseline cocaine use and treatment experience variables.There was only one trend toward differences between groupson these baseline variables. Participants in the TES group

Table 2Repeated measures ANOVA of between group differences of CRA knowledge an

Variables

TES (n = 12) TAU

Baseline Follow-up Base

CRA knowledge 13.91 (4.68) 18.58 (4.78) 14.6CRRTLatency 3.03 (0.90) 3.24 (1.84) 3.15Number plans 0.60 (0.26) 0.74 (0.30) 0.58Quality of best response 3.80 (1.08) 4.06 (0.86) 3.98Quality of overall response 3.71 (1.08) 4.01 (0.83) 3.92Highest category response 8.02 (2.31) 9.29 (1.70) 8.76Specific response 0.92 (0.16) 0.96 (0.07) 0.87Response classificationUsing response 1.42 (2.11) 0.33 (0.89) 1.25Poor response 2.33 (1.37) 1.33 (1.77) 0.75General coping strategies 4.42 (1.73) 5.40 (1.70) 4.90CBT response 0.83 (0.94) 2.08 (1.08) 2.4012-Step response 1.00 (1.76) 0.83 (1.11) 0.67

reported attaining more years of education than participantsin the YC group, t(23) = −1.51, p = .14.

Participants in the TES and YC groups attended a similarnumber of the 24 possible intervention visits, with the TESgroup averaging 20.0 (SD = 5.8) and the YC group averaging19.3 (SD = 6.5) visits, t(24) = −0.28, p = .784. The TES (M =$321.71, SD = $88.32) and YC (M = $310.67, SD = $70.89)groups also earned an equivalent amount of cash forcompleting modules or attending research visits, t(24) =−0.348, p = .731. Clients in the TES completed an averageof 40.0 (SD = 11.9) TES modules and had an average end-of-module quiz accuracy rate of 73.15% (SD = 9.59%).

3.1. CRA/HIV knowledge

There were no baseline differences between groups on theCRA knowledge test scores, t(24) = 0.678, p = .504;

d coping skill acquisition

(n = 12)

F Significance dline Follow-up

7 (6.00) 15.83 (5.50) 8.90 .007 1.05

(1.06) 2.57 (0.95) 2.51 .128 0.62(0.25) 0.64 (0.28) 0.471 .500 0.29(1.29) 3.90 (0.96) 0.926 .346 0.39(1.21) 3.85 (0.96) 1.07 .313 0.42(2.60) 9.05 (2.14) 1.35 .258 0.47(0.25) 0.90 (0.24) 0.14 .907 0.04

(2.40) 0.50 (1.73) 0.225 .640 0.06(0.62) 1.50 (1.50) 5.75 .025 1.04(2.15) 5.83 (2.59) 0.010 .92 0.04(1.67) 1.58 (0.99) 11.95 .002 1.16(1.15) 0.58 (1.44) 0.022 .885 0.06

Table 3During-treatment cocaine use results based on weekly observation and self-report

Variables

TES (n = 14) TAU (n = 12)

t df Significance dM (SD) M (SD)

Number of weeks using (urine confirmed) 3.6 (3.5) 3.4 (3.4) −0.113 24 .911 –Average using frequency per week 0.45 (0.65) 1.03 (1.95) 1.052 24 .303 0.41Average dollar amount used per week 23.36 (39.16) 29.06 (47.91) 0.334 24 0.741 –


however, there was a significant Group × Time interactionon follow-up CRA learning scores. Clients in TESdemonstrated significantly greater improvement in CRAlearning when compared to clients assigned to YC, F(1,20) = 8.90, p = .007 (see Table 2 for means and standarddeviations). This difference in patient learning amounts to avery large effect (d = 1.05).

3.2. Coping skills

There were no baseline differences between groups oncoping skills as measured by the six coding indices for theCRRT (latency, number of plans, quality of best and overallresponses, high category response, and specificity ofresponse; see means and standard deviations in Table 2).Participants in the TES group showed a trend towardincreased latency of response relative to the participants inthe YC group, F(1, 20) = 2.51, p = .128, d = 0.62. Meanincreases in quality of responses and category of responsealso tended to be greater for participants in TES whencompared to changes for participants in the YC group, butthese differences were not significant. Estimated effect sizesfor these indices produced d values ranging from 0.39 to0.47 that would be considered a low medium effect size, andthe small sample size of the pilot study had only a 57%chance of detecting effects as large as .60.

We found a statistically significant Group × Timeinteraction effect for the frequency of giving a poorresponse, with TES participants showing a decrease inthese responses and YC participants demonstrating anincrease, F(1, 20) = 5.75, p = .025, d = 1.04. We alsofound a statistically significant Group × Time interactioneffect for the frequency of providing a CBT style response,with TES participants showing an increase in CBTresponses and YC participants demonstrating a decreasein frequency, F(1, 20) = 11.95, p = .002, d = 1.16. SeeTable 2 for full means and standard deviations.

Table 4During-treatment cocaine use results based on weekly observation and self-report

Variables

TES (n = 9) TAU (n

M (SD) M (SD)

No. of weeks using (urine confirmed) 1.9 (3) 3.00 (3Average using frequency per week 0.22 (0.67) 1.11 (2Average dollar amount used per week 1.22 (3.67) 23.80 (4

3.3. Drug use

We compared during-treatment cocaine use betweengroups, with missing weeks or missing urine samplesimputed as cocaine-positive (see Table 3). There were nodifferences between groups in number of weeks of urine-confirmed cocaine use, in average using frequency, or inaverage amount of cocaine used: not a surprising findinggiven that the low number of participants would allowdetection only of large effect sizes. There was anonsignificant difference in frequency of cocaine usefavoring participants in the TES condition, which sug-gested a small to moderate effect size (d = 0.41). Becausemost of the participants tested negative for cocaine atbaseline (a variable shown to predict greater improvementin psychosocial treatment), we also analyzed thesevariables using only baseline-negative participants(see Table 4). Cocaine use consistently was lower in theTES condition compared to the YC condition, and TESparticipants demonstrated a slight trend toward reducedamount of cocaine used per episode, t(17) = 1.537, p =.158, d = 0.69. The estimated effect size for reduced usingfrequency was in the medium range (d = 0.54). Althoughthe differences favored the TES group, t(17) = 1.198, p =.248, they were not statistically significant.

3.4. Relationship between TES skill and outcome

The analysis examining the relationship betweenmastery of skills taught by the TES and treatment outcomesindicated that the participants with high TES accuracy ratescompleted significantly more modules (r = .63, p = .015)and produced significantly fewer cocaine-positive urines(r = −.68, p = .007). We also calculated the relationshipbetween accuracy rate and cocaine-positive urines control-ling for number of modules completed with similar results(partial r = −.64, p = .019).

for clients who were negative at baseline cocaine use

= 9)

t df Significance d

) 0.745 17 .466 0.34.14) 1.198 17 .248 0.546.29) 1.537 17 .158 0.69


3.5. Client satisfaction

There were few between-group differences in satisfactionwith the treatment learning experience. Participants in bothconditions generally responded similarly on how interesting,satisfying, and useful the information they learned intreatment was, as well as how much new information theyfelt they had learned. Participants in the TES conditionshowed trends indicating that their treatment learningexperience was more likely to “clarify misconceptions,” t(13.55) = −1.55, p = .144, d = 0.62, as well as slight trendsindicating that their treatment learning experience was “easyto use,” t(22) = 1.39, p = .180, d = 0.55.

Nine (64%) TES clients required help with spellingrecall responses on the fluency-based learning quizzes.Participants would typically request information on how tospell certain words (e.g., catastrophizing) and how tonavigate the mouse while completing these modules;however, they needed spelling help infrequently, requiringsuch help only on an average of 1.3 visits (out of an averageof 20 visits).

4. Phase 2 method

Phase 2 was not a controlled study; it was conducted onlyto gather preliminary information regarding counselor andclient use of TES in the absence of explicit incentives ormonitoring and encouragement of researchers.

Five counselors who were interested in continuing to usethe TES consented to complete a brief training, complete aweekly report on counseling activities, and participate in afocus group. The 2-hour training provided a background onthe CRA and presented the TES and key modules with adescription of the main teaching points of each module.Finally, counselors were trained how to provide encourage-ment (e.g., social praise) to clients regarding the use of theTES and asked to avoid employing negative or punitivestrategies to influence clients to use it. Counselors were thenallowed to refer clients to use the TES.

The Weekly Client Report asked counselors (a) whatpercentage of their caseload they referred to the TES, (b)which of these clients they had encouraged to complete TESmodules, and (c) what content they had discussed. Clientscould visit the laboratory as often as they wished andcomplete as many modules as desired. Clients did not haveto participate in the research to use the TES, but those whodid consent allowed electronic collection of data on TESmodule completion, accuracy rates, and number of TESlaboratory visits.

After the 12-week pilot trial ended, the first authorconducted a 90-minute focus group with the participatingcounselors that concentrated on how they had decided whichclients to refer to the TES laboratory, how they used the TESin individual sessions, and what benefits and problems theyhad in using the system. We also asked counselors forfeedback on how the TES could be improved.

5. Phase 2 findings

5.1. Counselor referral rate

Counselors generally referred only a small portion of theircaseloads to work with the TES (around 0%–10%). By thehalfway point (Week 6), two of the counselors had begun torefer 11%–20% of their caseloads. In the focus groupdiscussion, most counselors reported that they were fairlyselective in which clients they referred to use the TESlaboratory and that they tended to refer clients who theythought were more cognitively able and would be interested inusing the TES. One counselor reported that hewasmore liberalin whom he referred but tended to refer based on the content inclients' therapy that was related to specific TES modules.

5.2. Counselor use of the TES in individual sessions

On average, counselors reported discussing TEScontent with clients over a mean of 1.35 sessions (SD =1.36) and encouraging clients to use the TES a mean of2.06 times (SD = 1.60). Neither self-reported frequency ofcontent discussions nor verbal encouragement correlatedwith the number of modules clients completed (r = −0.009and r = 0.09, respectively).

5.3. Client participation

During the 3-month study, 18 clients provided informedconsent to use their TES data. Client use of the TES wasquite low; only 10 clients (55.5%) completed more than 3modules. The average number of modules completed byclients was 11.11 (SD = 19.57), and the modal number ofcompleted modules was 3. The higher mean number is due to3 participants who completed an average of 52 modules(engagement similar to that obtained in Phase 1).

5.4. Counselor focus group feedback

Overall, counselors reported that they could see realpotential in using the TES to engage clients, but they felt thatin its current form, it would only be appropriate for selectedclients some of the time. They pointed out that the clients atthis inner-city facility were particularly educationallychallenged. Counselors generally thought a similar comput-er-assisted training program would be useful but thatchanging the presentation strategy to include more culturallyspecific references and more “street” language would make itmore effective for their clientele. They reported being lessinclined to use the TES when it provided core modules in aset order because clients might be working on something elsein therapy and indicated that the ability to choose moduleswould be preferable. Counselors also suggested tailoring themodules and ordering them to correspond with specific tasksthat clients needed to complete at various stages of treatmentwould be useful.


6. Discussion

This two-phase pilot study offers preliminary findingsshowing that clients in an urban, inner-city clinic willcomplete a full course of computerized CBT modules whenincentives are provided and that although they require someminimal supervision and monitoring while using the system,they are capable of navigating the program on their own aftersome practice. The Phase 1 study showed large effects forCBT and HIV prevention knowledge acquisition for clientswho used the TES compared to those who were not exposed,moderate-to-large effect estimates in coping skill acquisition,and low medium effect estimates in frequency and severityof cocaine use. Furthermore, during the first study phase,these gains were made while the hosting treatment programand the clients' current counselors had no involvement in theTES training beyond client referral to the research protocol.Less encouragingly, when cash incentives were not provid-ed, clients showed a precipitous drop in TES utilization, andbrief training for counselors in the content and appropriateintegration of TES materials into individual sessions yieldedonly a small number of client referrals to the TES laboratoryand modest engagement between the counselors and clientsover TES content.

It is encouraging to find that clients negotiating a Web-based learning system without guidance or involvementfrom a counselor can show promising clinical gains, findingsthat parallel those of Carroll et al. (2008, 2009), whodemonstrated that computer-based CBT as an adjunct toTAU promoted significant and lasting effects in clients.Importantly, their CBT4CBT product seemed to be engagingenough to clients in that they engaged it without incentives,and fewer sessions (M = 4) were necessary to convey coreprinciples. It seems likely that requiring clients to completethese computer-based CBT programs, integrating CBTprinciples into the treatment milieu, and prompting andpraise from counseling staff regarding their use could resultin better client utilization, better clinical outcomes, and moreefficient use of counselors and client' treatment time.

The low rates of client engagement with the TES whenreinforcement was withdrawn in the second phase of the studydemonstrate that a text and quiz-based learning systemprobably is not sufficiently engaging for clients witheducational and motivational limitations and that somestrategy is likely required to get effective levels of clientparticipation. The higher rate of participation in Phase 1probably was due to the incentives provided, but this and otherstrategies (e.g., making TES a program requirement, counselorprompting and praise, integrating TES into therapy) need to betested in future studies to empirically determine whether theyare effective solutions for increasing client participation. Inaddition, although it may be tempting to reduce counselortraining time in using a computerized evidence-basedtreatment, the results from the second phase suggest thatmore intensive training and continued supervision may beneeded to promote counselor utilization of the system.

This small study is limited in that it was not fullypowered, and therefore, findings are mostly only estimatesof probable effect and their generality is yet to bedetermined. In addition, it is not clear whether the resultsof this study would generalize widely to other treatmentprograms. The study was conducted at one treatmentprogram with comprehensive high-quality, best-practiceservices and with clients that demonstrated relatively lowcocaine use at the beginning of the study. Another limitationof the study is that we did not expose the control participantsto a similar amount of computer interaction or a controlservice and as such cannot say for certain that the effectswere specific to the TES program and not generic attentionor expectancy effects. In addition, the Phase 2 trainingintervention for counselors was intended to be minimal andso should not be interpreted to mean that communitycounselors would not be interested in learning to usecomputer-based therapy technologies in their practice. Onthe contrary, in the focus group, counselors indicated thatthey saw potential uses for the computerized therapy.Additional fully powered studies are needed to determinethe generality of these findings, the conditions under whichcounselors will frequently use computerized treatmentprograms, and whether clients in community-based treat-ment settings ultimately benefit from them.

Acknowledgments

This research was supported by a Commonwealth ofPennsylvania Department of Health research grant (SAP4100042753).

We gratefully acknowledge the coding work of RAJonathan Kaplan, who coded the CRRT responses.

References

Bickel, W. K., Marsch, L. A., Buchhalter, A. R., & Badger, G. (2008).Computerized behavior therapy for opioid-dependent outpatients: Arandomized controlled trial.Experimental andClinical Psychopharmacology,16, 132−143.

Binder, C. (1996). Behavioral fluency: Evolution of a new paradigm.Behavior Analysis, 19, 163−197.

Budney, A. J., & Higgins, S. T. (1998). A community reinforcement plusvouchers approach: Treating cocaine addiction.U.S.Department ofHealthand Human Services. Rockville (Md): National Institute of Drug Abuse.

Carroll, K. M. (2008). Computer based training for cognitive behavioraltherapy: CBT4CBT. Presented at 70th Annual Meeting of the College onProblems of Drug Dependence, San Juan, Puerto Rico.

Carroll, K. M., Ball, S. A., Martino, S., Nich, C., Babuscio, T. A., Nuro, K.F., et al. (2008). Computer-assisted delivery of cognitive–behavioraltherapy for addiction: A randomized trial of CBT4CBT. AmericanJournal of Psychiatry, 165, 881−888.

Carroll, K. M., Ball, S. A., Martino, S., Nich, C., Babuscio, T. A., &Rounsaville, B. J. (2009). Enduring effects of a computer-assistedtraining program for cognitive–behavioral therapy: A 6-month follow-up of CBT4CBT. Drug and Alcohol Dependence, 100, 178−181.

Carroll, K. M., Nich, C., Frankforter, T. L., & Bisighini, R. M. (1999). Dopatients change in the ways we intend? Assessing acquisition of copingskills in cocaine-dependent patients. Psychological Assessment, 11,77−85.


Dutra, L., Stathopoulou, G., Basden, S. L., Leyro, T. M., Powers, M. B., &Otto, M. W. (2008). A meta-analytic review of psychosocial interven-tions for substance use disorders. American Journal of Psychiatry, 165,179−183.

First, M. B., Spitzer, R. L., Gibbon, M., & Williams, J. B. W. (2002).Structured clinical interview for DSM-IV-TR axis I disorders, researchversion, patient edition (SCID-I/P). New York: New York StatePsychiatric Institute, Biometrics Research.

Glenn, I. M., & Dallery, J. (2007). Effects of Internet-based voucherreinforcement and a transdermal nicotine patch on cigarette smoking.Journal of Applied Behavior Analysis, 40, 1−13.

Hester, R. K., Squires, D. D., & Delaney, H. D. (2005). Drinker's check-up:12-Month outcomes of a controlled clinical trial of a stand-alonesoftware program for problem drinkers. Journal of Substance AbuseTreatment, 28, 159−169.

Kay-Lambkin, F. J., Baker, A. L., Lewin, T. J., & Carr, V. J. (2009).Computer-based psychological treatment for co-morbid depression andproblematic alcohol and/or cannabis use: A randomized controlled trialof clinical efficacy. Society for the Study of Addiction, 104, 378−388.

Marsch, L. A., & Bickel, W. K. (2004). Efficacy of computer-based HIV/AIDS education for injection drug users. American Journal of HealthBehavior, 28, 316−327.

McCrone, P., Knapp, M., Proudfoot, J., Ryden, C., Cavanagh, K., Shapiro, D.A., et al. (2004). Cost- effectiveness of computerized cognitive–behaviouraltherapy for anxiety and depression in primary care: Randomized controlledtrial. The British Journal of Psychiatry, 185, 55−62.

Miller, W. R., & Mount, K. A. (2001). A small study of training inmotivational interviewing: Does one workshop change clinician and clientbehavior? Behavioral and Cognitive Psychotherapy, 29, 457−471.

Morgenstern, J., Blanchard, K. A., Morgan, T. J., Labouvie, E., & Hayaki, J.(2001). Testing the effectiveness of cognitive–behavioral treatment forsubstance abuse in a community setting: Within treatment andposttreatment findings. Journal of Consulting and Clinical Psychology,69, 1007−1017.

Morgenstern, J., Morgan, T. J., McCrady, B. S., Keller, D. S., & Carroll,K. M. (2001). Manual-guided cognitive–behavioral therapy training: Apromising method for disseminating empirically supported substanceabuse treatments to the practice community. Psychology of AddictiveBehaviors, 15, 83−88.

Nunes, E. V., Rothenberg, J. L., Sullivan, M. A., Carpenter, K. M., &Kleber, H. D. (2006). Behavioral therapy to augment oral naltrexone for

opioid dependence: A ceiling on effectiveness? The American Journalof Drug and Alcohol Abuse, 32, 503−517.

Ondersma, S. J., Chase, S. K., Svikis, D. S., & Schuster, C. R. (2005).Computer-based brief motivational intervention for perinatal drug use.Journal of Substance Abuse Treatment, 28, 305−312.

Ondersma, S. J., Svikis, D. S., & Schuster, C. R. (2007). Computer-basedbrief intervention: A randomized trial with postpartum women.American Journal of Preventative Medicine, 32, 231−238.

Roozen, H. G., Boulogne, J. J., Van Tulder, M. W., Van Den Brink, W.,DeJong, C. A. J., & Kerkhof, A. J. F. M. (2004). A systematic review ofthe effectiveness of the community reinforcement approach in alcohol,cocaine and opioid addiction. Drug and Alcohol Dependence, 74, 1−13.

Shiffman, S., Paty, J. A., Rohay, J. M., Di Marino, M. E., & Gitchell, J. G.(2001). The efficacy of computer-tailored smoking cessation material asa supplement to nicotine patch therapy. Drug and Alcohol Dependence,64, 35−46.

Sobell, M. B., Maisto, S. A., Sobell, L. C., Cooper, A. M., Cooper, T.C., & Sanders, B. (1980). Developing a prototype for evaluatingalcohol treatment effectiveness. In L. C. Sobell, M. B. Sobell, & E.Ward (Eds.), Evaluating alcohol and drug abuse treatmenteffectiveness: Recent advances (pp. 129−150). New York: PermagonPress.

Spek, V., Cuipers, P., Nyklicek, I., Riper, H., Keyzer, J., & Pop, V.(2007). Internet-based cognitive behaviour therapy for symptoms ofdepression and anxiety: A meta-analysis. Psychological Medicine,37, 319−328.

Squires, D. D., & Hester, R. K. (2004). Using technical innovations inclinical practice: The drinker's check-up software program. Journal ofClinical Psychology, 60, 159−169.

Strecher, V. J., Shiffman, S., &West, R. (2005). Randomized controlled trialof a Web-based computer-tailored smoking cessation program as asupplement to the nicotine patch therapy. Addiction, 100, 682−688.

Taylor, B. C., & Luce, K. H. (2003). Computer- and Internet basedpsychotherapy interventions. Current Directions in PsychologicalScience, 12, 18−22.

Tumur, I., Kaltenthaler, E., Ferriter, M., Beverley, C., & Parry, G. (2007).Computerized CBT for obsessive compulsive disorder: A systematicreview. Psychotherapy and Psychosomatics, 76, 196−202.

Wright, J. H., Wright, A. S., Albano, A. M., Bascol, M. R., Goldsmith, J.,Raffield, T., et al. (2005). Reducing therapist time. American Journal ofPsychiatry, 162, 1158−1164.

Documents

Feasibility and effectiveness of computer-based therapy in community treatment