Florida State University Libraries

Electronic Theses, Treatises and Dissertations The Graduate School

2014

The Psychometric Properties of the BarkleyDeficits in Executive Functioning Scale(BDEFS) in a College Student PopulationTheodora Passinos Coffman

Follow this and additional works at the FSU Digital Library. For more information, please contact lib-ir@fsu.edu

FLORIDA STATE UNIVERSITY

COLLEGE OF EDUCATION

THE PSYCHOMETRIC PROPERTIES OF THE

BARKLEY DEFICITS IN EXECUTIVE FUNCTIONING SCALE (BDEFS) IN A COLLEGE

STUDENT POPULATION

By

THEODORA PASSINOS COFFMAN

A Dissertation submitted to the

Department of Educational Psychology and Learning Systems in partial fulfillment of the

requirements for the degree of Doctor of Philosophy

Degree Awarded:

Summer Semester, 2014

ii

Theodora Coffman defended this dissertation on May 6, 2014

The members of the supervisory committee were:

Frances Prevatt

Professor Directing Dissertation

Lee Stepina

University Representative

Beth Phillips

Committee Member

Debra Osborn

Committee Member

The Graduate School has verified and approved the above-named committee members, and

certifies that the dissertation has been approved in accordance with university requirements.

iii

ACKNOWLEDGEMENTS

I would like to thank Dr. Frances Prevatt for serving as my major professor, providing me

with support and encouragement during my graduate school career. I would also like to

recognize how smooth she made the process of doctoral training for me with her continued

guidance through every step of the way. I would like to thank my doctoral committee; Dr. Debra

Osborn, Dr. Beth Phillips, and Dr. Lee Stepina for their support throughout the dissertation

process and for being so giving of their time.

I thank my husband, Michael, for his continuous support over the years, and for his

willingness to listen to me talk about the ins and outs of executive functioning to exhaustion. I

thank my in-laws for their keen eye in catching typos in this document, and my parents for their

support. I would also like to thank all of my friends in the College of Education at FSU for their

continued support and willingness to order takeout at ALEC at all hours of the day and night to

make sure the work continued.

iv

TABLE OF CONTENTS

LIST OF TABLES ....................................................................................................................... vi

ABSTRACT ................................................................................................................................. vii

CHAPTER 1 .................................................................................................................................. 1

INTRODUCTION ........................................................................................................................ 1

Social Significance ...................................................................................................................... 1

Biological Factors Explaining the Relationship Between EF and ADHD .................................. 3

Barkley’s Model of EF and ADHD............................................................................................. 3

Assessing EF: Psychometric Tests or Impairment Rating Scales .............................................. 4

Test Construction ........................................................................................................................ 5

Statement of the Problem ............................................................................................................ 6

Research Questions ..................................................................................................................... 7

CHAPTER 2 .................................................................................................................................. 9

LITERATURE REVIEW ............................................................................................................ 9

ADHD and How the College Student is Different ...................................................................... 9

The Relationship Between ADHD and EF: How Biology and Neuropsychology Inform Our Understanding ........................................................................................................................... 15

EF Deficits as Manifested in ADHD......................................................................................... 25

Assessing EF ............................................................................................................................. 29

Test Construction and Validation Principles ............................................................................. 48

Proposed Study and Research Questions .................................................................................. 53

CHAPTER 3 ................................................................................................................................ 56

METHODS .................................................................................................................................. 56

Introduction ............................................................................................................................... 56

Hypotheses and Planned Data Analyses ................................................................................... 61

CHAPTER 4 ................................................................................................................................ 72

RESULTS .................................................................................................................................... 72

Demographic Variables and Statistics....................................................................................... 72

Research Question 1 .................................................................................................................. 72

Research Question 2 .................................................................................................................. 74

Research Question 3 .................................................................................................................. 75

Research Question 4 .................................................................................................................. 84

CHAPTER 5 ................................................................................................................................ 88

DISCUSSION .............................................................................................................................. 88

v

Relationship between BDEFS Self-Report Form and Other-Informant Form .......................... 88

Relationship between the BIA and BDEFS Factors.................................................................. 90

Reevaluation of ADHD-EF Index ............................................................................................. 93

Evidence of Factorial Validity .................................................................................................. 98

Limitations .............................................................................................................................. 100

Implications for Future Research ............................................................................................ 102

Implications for Clinical Practice ............................................................................................ 103

APPENDIX A ............................................................................................................................ 105

INFORMED CONSENT FOR ADHD GROUP .................................................................... 105

APPENDIX B ............................................................................................................................ 107

BARKLEY DEFICITS IN EXECUTIVE FUNCTIONING SCALES ................................ 107

APPENDIX C ............................................................................................................................ 111

INFORMED CONSENT FOR CONTROL GROUP ............................................................ 111

APPENDIX D ............................................................................................................................ 113

INTERNAL REVIEW BOARD FOR HUMAN SUBJECTS APPROVAL ........................ 113

APPENDIX E ............................................................................................................................ 115

DEMOGRAPHIC QUESTIONNAIRE .................................................................................. 115

APPENDIX F ............................................................................................................................ 116

BDEFS SCORING TEMPLATE ............................................................................................ 116

APPENDIX G ............................................................................................................................ 117

OTHER INFORMANT BDEFS .............................................................................................. 117

REFERENCES .......................................................................................................................... 121

BIOGRAPHICAL SKETCH ................................................................................................... 133

vi

LIST OF TABLES

1 Demographics ....................................................................................................................57 2 Suggested Ranges for Fit-Indices ......................................................................................71 3 Inter-Rater Correlations for College Student Sample, Comparing Self-Reports to Other-Reports .....................................................................................................................73 4 Inter-Rater Correlations for Barkley’s Sample, Comparing Self-Reports to Other-

Reports ...............................................................................................................................73 5 Means, t-Tests, and p-Value for Self vs. Other-Informant Forms, in the Current Sample................................................................................................................................74 6 Summary of Canonical Coefficients and Structure Loadings ............................................76 7 New 15-Item ADHD-EF Index ..........................................................................................80 8 Original 11-Item ADHD-EF Index ....................................................................................82 9 Summary of Canonical Discriminant Functions ................................................................82 10 ADHD-EF Index Description of Models ...........................................................................83 11 Functions at Group Centroids ............................................................................................83 12 Classification Rates ............................................................................................................83 13 CFA Models .......................................................................................................................85 14 Standardized Factor Loadings and Standardized Residual Variances ...............................85

vii

ABSTRACT

Approximately 4.4% of the adult population suffers from Attention-Deficit/Hyperactivity

Disorder (ADHD) (Keesler et al., 2010). The identification of adults with ADHD can be

difficult because the criteria in the Diagnostic and Statistical Manual of Mental Disorders (DSM;

APA, 2013) were originally designed with children in mind. Identifying high achieving college

students with ADHD has proven even more challenging due to masked academic difficulties

until later in life.

The specific population of adults in the college setting (college students) with ADHD are

more likely to have protective factors such as higher cognitive abilities and previous academic

success (DuPaul, Weyandt, O’Dell, & Varejao, 2009; Glutting, Youngstrom, & Watkins, 2005)

than non-college ADHD adults. Nevertheless, they tend to fall significantly behind persons in

college who do not suffer from ADHD (Barkley, Murphy, & Fischer, 2008; DuPaul et al., 2009;

Heiligenstein, Guenther, Levy, Savino, & Fulwiler, 1999) and those with ADHD have a higher

dropout rate than those without ADHD.

ADHD has been linked to deficits in Executive Functioning (EF) in the literature

(Cortease et al., 2005; Kassubek, Juengling, Ecker, & Landwehrmeyer, 2005; Koechlin, Corrado,

Pietrini, & Grafman, 2000; Lewis, Dove, Robbins, Baker, & Owens, 2004; Monchi, Petrides,

Strafella, Worsley, & Doyon, 2006; Niendam et al., 2012; Stuss & Alexander, 2000; Stuss,

Alexander, Floden, Binns, Levin, & McIntosh, 2002). There is also evidence that EF abilities

are not fully developed until around the third decade. Both established theory and fMRI imaging

support the idea of delayed development (Barkley, 2012). Therefore, it is hypothesized that

there will be a different set of characteristics for the college student population (average age 18-

30) than an adult aged 30 or above.

viii

Given that there are some weaknesses in traditional EF testing (e.g., EF and general

intellectual level is somewhat correlated) (Salthouse; 1996, 2005), the Barkley Deficits in

Executive Functioning Scale (BDEFS) was specifically designed to evaluate EF deficits in adult

individuals with ADHD. This is a new self-report measure identifying functional impairment in

EF abilities with five factors. The goal of this study was to provide further empirical support

regarding the validity and reliability of the Barkley Deficits in Executive Functioning Scale

(BDEFS; Barkley, 2011b). In addition to the five factors, this scale contains an ADHD-EF Index,

which provides an estimate of the likelihood of a diagnosis of adult ADHD (Barkley, 2012b). To

date, this scale has not been investigated for evidence of validity and reliability from an

independent researcher. Additionally, college students have not yet been studied (Barkley,

2011b). Therefore, this study evaluated (a) differences in self- and other-reports on the BDEFS,

(b) the relationship between the BDEFS scales and cognitive functioning, (c) the ability of the

BDEFS to predict ADHD, and (d) the factor structure of the BDEFS with a college student

population.

In this study BDEFS self-reports were collected from 596 college students (with and

without a diagnosis of ADHD). The mean age of the participants was 20.5 years of age and most

demographic variables were consistent with the statistics published for the university where the

data were collected. To evaluate the differences in the self-report form and the other-informant

form of the BDEFS, a Pearson Correlation was conducted comparing self- and other-reports,

using only the sample of students with a diagnosis of ADHD. It was determined that there were

statically significant correlations between the BDEFS-self form and the BDEFS-other form.

These correlations were also statistically significantly different from the correlations that Barkley

found in his original study. In addition, there were statistically significant differences in the

ix

means between the self- and other-informant forms, indicating that college students in general

perceive their symptoms differently, and generally more impaired, than those around them.

Another Pearson Correlation was conducted to determine if the general intelligence of the

participant was related to any of the five factors of the BDEFS. Results indicated that there was

an inverse relationship between intellectual ability and time management skills. As time

management skills decreased, intelligence increased.

When investigating the ADHD-EF index, which is a scale used to predict adult ADHD, a

discriminate analysis was conducted. It was determined that a different set of items was needed

to distinguish college students with and without ADHD than was needed to distinguish these

groups in the adult population. Finally, a confirmatory factor analysis was conducted to see if

the same factor structure of the BDEFS held true for the population of college students. Results

indicated a moderate to good fit for the factor structure in the college student population.

While additional support of validity is needed, the current study did provide additional

evidence for the validity and reliability of the BDEFS. A replication of the newly identified

items of the ADHD-EF Index that are most predictive of adults with ADHD in the college

student population is needed to provide additional support.

1

CHAPTER 1

INTRODUCTION

The goal of this study was to provide further empirical evidence regarding the validity

and reliability of the Barkley Deficits in Executive Functioning Scale (BDEFS; Barkley, 2011b).

The BDEFS is a new self-report measure specifically designed to evaluate executive functioning

(EF) deficits in individuals with Attention Deficit Hyperactivity Disorder (ADHD. Among other

factors, this scale contains an ADHD-EF Index, which provides an estimate of the likelihood of a

diagnosis of adult ADHD (Barkley, 2011b). To date, this scale has not been investigated for

support of validity and reliability from an independent researcher. Additionally, specific

subpopulations (such as college students) have not yet been studied. As part of this validation

process, a college student population was assessed for possible differences in the ways in which

the BDEFS captures information about their executive functioning, along with their likelihood of

having a diagnosis of ADHD. The current study evaluated (a) differences in self- and other-

informant reports on the BDEFS, (b) the relationship between the BDEFS scales and cognitive

functioning, (c) the ability of the BDEFS to predict ADHD, and (d) the factor structure of the

BDEFS. This manuscript proposes that the college student population differs from the general

adult population in terms of ADHD symptoms and EF deficits. A review of the biological

underpinnings of EF and ADHD, current theory, current assessment tools available to measure

EF, and the limitations of these tools is also provided.

Social Significance

Attention-Deficit/Hyperactivity Disorder (ADHD) is generally thought of as a childhood

disorder. It was once believed that the impairing symptoms of ADHD fade in adulthood.

However, research has shown that ADHD often persists past childhood, with similar debilitating

2

symptoms (Antshel et al., 2009; Faraone et al. 2006; Ramirez et al., 1997; Turnock, Rosen, &

Kaminski, 1998). According to results from the United States Comorbidity Replication Study

conducted in 2010, approximately 4.4% of the adult population suffers from ADHD (Keesler et

al., 2010). It is also estimated that 50-65% of children with ADHD will continue to experience

symptoms throughout adulthood (Booksh, Pella, Singh, & Gouvier, 2010; Goldstein, 2002).

ADHD is also one of the fastest growing disability categories on college campuses

(Antshel et al., 2009). College students with ADHD tend to have different characteristics from

their non-college peers who also have ADHD, likely due to differing sets of demands and

stressors. For one thing, they tend to have more protective factors such as previous academic

success and higher cognitive abilities than their non-college peers (DuPaul et al., 2009; Glutting

et al., 2005). Even with the presence of these protective factors, they tend to fall significantly

behind persons in college who do not suffer from ADHD. They are likely to have lower GPAs

and are on academic probation at higher rates than their academic peers (Barkley et al., 2008;

DuPaul et al., 2009; Heiligenstein et al., 1999).

Deficits in EF and symptoms of ADHD are highly problematic in the academic

environment and are correlated with high college drop-out rates and poor academic performance

(Norwalk, Norvilitis, & MacLean, 2009). Given that self-regulation is a requisite skill for

successful navigation of the college environment, a better understanding of the needs of this

college student sub-population must be a top priority. It should be noted that there is a high rate

of stimulant (used to treat ADHD) medication misuse on college campuses, leading to

difficulties in diagnosis due to malingering (Booksh, et al., 2010). Therefore, a closer analysis of

college students as a distinct population from other adults is needed.

3

Biological Factors Explaining the Relationship Between EF and ADHD

The link between EF and ADHD has been discussed in the literature for some time.

There is a large body of evidence from direct neuro-imaging studies that indicates that persons

with ADHD demonstrate patterns of hypoactivation in the areas of the brain responsible for EF

(Cortease et al., 2005). Through a meta-analysis, Cortese and colleagues (2012) provide some of

the strongest support for this assertion. A pattern of hypoarousal was found in children and

adults with ADHD in the areas of the brain responsible for EF (Kassubek et al., 2005; Koechlin

et al., 2000; Lewis et al., 2004; Monchi et al., 2006; Niendam et al., 2012; Stuss & Alexander,

2000; Stuss et al., 2002). This pattern was, however, slightly different for adults than children.

This provides support for the idea that EF development is not complete until later in the lifespan.

The pattern of hypoarousal in the areas of the brain responsible for EF is also supported in the

medical field through pharmacological intervention of ADHD. As Biederman, Spencer and

Wilens (2004) put it, most pharmacological intervention rests upon “the idea that

catecholaminergic hypoactivity in frontal subcortical circuits underlie the disorder” (p. 15). The

stimulant medication reduces this hypoactivity, thus returning the functioning of the EF system

to a normal level, relatively speaking.

Barkley’s Model of EF and ADHD

The biological model of EF and its relationship to ADHD only covers the one narrow

aspect (biology) of this relationship. No consideration had previously been given to variables

such as emotional control, self-awareness, or the social aspect of EF (Barkley, 2012). Given the

lack of breadth in the current biological model, Barkley, one of the prominent researchers in the

field of ADHD and EF, dedicated years of his research to developing a new model of EF and its

relationship to ADHD.

4

Barkley’s new model of EF (Barkley’s Hierarchical Theory of EF) incorporates six levels

of EF and they are arranged in a hierarchical fashion. The six levels are based on the idea of

self-regulation. Impairments in these areas are common in individuals with ADHD.

Many researchers in the field have linked ADHD to deficits of EF (Barkley, 1997, 2011a,

2012; Biederman et al., 2004; Biederman, Spencer, Wilens, Prince, Faraone, 2006; Biederman et

al., 2007; Boonstra et al., 2005; Brown, 2008; Chelune, Ferguson, Koon, & Dickey, 1986;

Grondzinsky, & Diamond, 1992; Fuster, 1997; Hervey, Epstein, & Curry, 2004; Nigg & Casey,

2005; Oosterlaan, Scheres & Sergent, 2005; Seidman et al., 2006; Welsh & Pennington, 1988;

Weyandt, 2009; Willcut et al., 2005). Thus, the controversy is not whether ADHD is associated

with EF deficits. Rather, the controversy is over the way in which they are associated.

Difficulties in measuring EF are at the forefront of this controversy.

Assessing EF: Psychometric Tests or Impairment Rating Scales

At the present time, there are two primary ways of measuring EF: neuropsychological

tests of some type of cognitive ability (psychometric tests) and rating scales of impairment

(questionnaires). Hereafter, these two types will be referred to as EF tests and EF behavior

rating scales, respectively. The results of using EF tests to determine impairment in people with

ADHD across the literature have been inconsistent (Alderman, Burgess, Knight, & Henman,

2003; Anderson, Anderson, Northman, & Mikiewicz, 2002; Barkley & Fisher, 2010; Barkley &

Murphy, 2010; Burgess, Alderman, Evans, Emsile, & Wilson, 1998; Chaytor, Schmitter-

Edgecombe, & Burr, 2006; Vriezen & Pigott, 2002; Wood & Liossi, 2006). Across studies, only

0-10% shared variance has been found between any single EF test and an EF behavior rating

scale (Barkley & Murphy, 2010). This means that an EF test measuring a specific cognitive

5

ability in the lab (e.g., working memory) may not correspond to daily functional impairments

(which are measured by the behavior rating scales).

Given both theoretical and empirical problems with using EF tests, Barkley developed

the Barkley Deficits in Executive Functioning Scale (BDEFS; Barkley, 2011 a, b). The BDEFS

is a behavior rating scale designed to assess functional impairment, rather than specific cognitive

abilities of EF. It is an 89-item self-report scale, which yields five factors: self-management of

time, self-organization/problem solving, self-restraint or inhibition, self-motivation, and self-

regulation of emotion. Additionally, an ADHD-EF Index can also be calculated, predicting the

likelihood of an adult diagnosis of ADHD. Barkley conducted multiple analyses to provide

support for validity and reliability for this new measure. He conducted multiple factor analyses

to provide support for construct validity. Although the final confirmatory factor analysis was

conducted on a pool of 100 items including the final items of the BDEFS, the majority of the

statistical procedures that were conducted used a prototype version of the scale. This prototype

version lacked one of the factors, thus the sub-factor of self-regulation of emotion has less

empirical support (Barkley, 2011b). Also, the analyses were conducted on an adult population

without giving special consideration to the younger portion of this population (i.e., the college

student).

Test Construction

It is important to provide evidence for the validity and reliability of a new measure before

it is used clinically. There are several types of reliability to consider: inter-rater reliability, split-

half reliability, and internal consistency. Reliability refers to the degree of which a measure can

be replicated (Messick, 1989). In this study; however, cross-informant reliability was

investigated. Validity refers to the measurement tool’s ability to actually measure what it

6

purports to measure (Messick, 1989). The clinical consequences of using a measure to

categorize human behavior are so great that it is imperative for researchers to make a concerted

effort to provide support for the validity of the measurement tool to measure the construct at

hand. There are multiple ways to provide support of validity: face-validity, criterion validity,

content validity, and construct validity (Heiman, 2002). The focus for this psychometric study

was construct validity. A factor analysis is one way to provide evidence of construct validity

using theoretical and empirical knowledge (Brown & Cutik, 1993; Brown, 2006). Additionally,

for a measure to be used in discriminating a clinical population from a non-clinical population,

support for its utility is imperative. In the case of the BDEFS, a discriminant analysis is one way

to determine the utility of the ADHD-EF Index. An important feature of a discriminant analysis

is that it provides an accuracy rate for the user in determining group membership (in this case,

ADHD or non-ADHD).

Statement of the Problem

Both established theory and fMRI imaging support the idea that the anatomical areas of

the brain responsible for EFs are not fully developed until an individual’s late 20’s or early 30’s

(Barkley, 2012). Therefore, a college student (average age 18-30) was hypothesized to have a

different pattern of responses than an adult aged 30 or above. The BDEFS is currently normed

by age group, with the youngest age group covering the span 18-35 years old. Based on the

changes occurring in this time period, the current clinical cut-offs and normative data for this

questionnaire might not be appropriate for its use with college students. If this hypothesis is

correct, and if the current clinical cut-offs and normative data are utilized, misdiagnosis might

occur. This, in turn, could lead to inappropriate treatment recommendations and ineffective

treatment. Therefore, the primary goal of this study was to add to the body of literature dealing

7

with the validity, reliability, and normative data of the BDEFS as applied to the college student

population. Several research questions were proposed to this end.

Research Questions

1. What is the relationship between the BDEFS self-report and other-informant report in a

college student population of students with ADHD? What are these relationships on the

following factors: Self-Management to Time, Self-Organization and Problem Solving,

Self-Restraint, Self-Motivation, Self-Regulation, ADHD-EF Index, and Total Executive

Functioning Symptoms? How do these correlations compare to the correlations that

Barkley found in his original study? Are the means of the self-informant reports higher

or lower than the means of the other-informant reports within the college student

population of students with ADHD?

2. In the college student ADHD sample, is there a correlation between intellectual

functioning (as measured by the BIA) and the BDEFS similar to the correlation between

the intellectual functioning and BDEFS in the norming sample? This relationship was

analyzed for the following factors: Self-Management to Time, Self-Organization and

Problem Solving, Self-Restraint, Self-Motivation, Self-Regulation, and Total Executive

Functioning Symptoms.

3. Are the same ADHD-EF Index questions the most predictive of a diagnosis of ADHD in

a college student population as they are in the original normative sample? The current

BDEFS ADHD-EF Index is composed of 11 items. This index was created using a

discriminant analysis to select those items (out of 89) that best discriminate those with

ADHD from a normative sample. Do the same 11 items best discriminate those with

ADHD from a normative sample in a college student population?

8

4. Is the factor structure of the BDEFS the same for college students as it is for the

normative sample, based on a confirmatory factor analysis?

The results of this study may have an impact in the way in which this questionnaire is utilized in

determining the deficits in executive functioning that a college student presents and the

likelihood of a correct diagnosis of adult ADHD and its consequent treatment.

9

CHAPTER 2

LITERATURE REVIEW

This literature review covers multiple aspects of Attention–Deficit/Hyperactivity

Disorder (ADHD) and Executive Functioning (EF). College students with ADHD represent a

distinctive subset of adults with ADHD, and the differences between college students and non-

college student adults are discussed. Specifically, the review focuses on the differences in regard

to ADHD and EF. Next, the review covers the biological and neuropsychological underpinnings

of how EF and ADHD relate to one another. Current measures of EF tests are reviewed,

including their limitations. Although the biological view of ADHD and EF lays the foundation

for further investigation, there is agreement in the field that this model lacks the needed

integration of social influences. Barkley (2012) has provided a theory of EF and ADHD to

address this aspect of the field, and a review of his theory follows. Finally, a review of Barkley’s

(2011b) new measure, The Barkley Deficits in Executive Functioning (BDEFS), is discussed.

An overview of test construction is provided, leading to the current psychometric study of the

BDEFS.

ADHD and How the College Student is Different

ADHD in Adults

Attention-Deficit/Hyperactivity Disorder (ADHD) was classically regarded as a disorder

that affects children, fades throughout adolescence, and virtually disappears in adulthood. As

research has advanced, it has become apparent that ADHD often persists into adulthood, causing

similar impairments across many stages of life (Antshel et al., 2009; Faraone et al., 2006;

Ramirez et al., 1997; Turnock et al., 1998). Results published by the United Stated Comorbidity

Replication Study conducted in 2010 (Kessler et al., 2010) indicated that approximately 4.4% of

10

the adult population displays symptoms of ADHD. Across studies, inattention appears to persist

to a greater extent than the symptoms of impulsivity or hyperactivity (Antshel et al., 2009;

Biederman, Mick, & Faraone, 2000). It is estimated that 50-65% of children with ADHD have

symptoms that persist into adulthood (Booksh, Pella, Singh, & Gouvier, 2010; Goldstein, 2002).

Moreover, DuPaul, Schaughency, Weyandt, Tripp, & Kiesner (2001) reported that there has been

a significant increase in college students reporting ADHD in higher educational settings,

(Antshel et al., 2009).

Prevalence Rates in College Students with ADHD

There have been relatively few studies conducted on the college student population with

ADHD; however, ADHD is one of the fastest growing disability categories on college campuses.

Initial findings by Wolf (2001) suggest that 25% of college students who received disability

services do so under a diagnosis of ADHD. However, this statistic is over a decade old. DuPaul

et al. (2009) indicated that it is hard to get an accurate prevalence rate for college students

because they are not required to disclose their disability at the college level. Based on a

literature review of multiple studies, it appears that 2-8% of college students may have a

diagnosis of ADHD (DuPaul et al., 2009). High ability students (those with high intelligence)

are often not identified as ADHD until college due to their adaptive skills (Booksh, et al., 2010;

Weyandt, Linterman, & Rice, 1995), making the prevalence even more difficult to accurately

report. In addition to the difficulties inherent in identifying college students with ADHD, the

subtypes of ADHD appear to be different in the adult and college populations, compared to the

child population. Heiligenstein and colleagues (1999) indicated that while there is variability

across subtypes in childhood, there is less variability in the adult population. The adult and

11

college populations tend to consist of primarily of the inattentive type or combined type rather

than hyperactive/impulsive type often seen in younger populations.

Academic Functioning in the College Student with ADHD

College students with ADHD are a unique subset of the population of adults with ADHD,

usually ranging in age from 18-30 years old. Although they technically fit into the adult

population, their characteristics are demonstrably different. College students with ADHD are

different from non-college adults with ADHD in several ways. These differences include more

advanced coping skills, past experiences with school success, and higher cognitive abilities

(DuPaul et al., 2009; Glutting et al., 2005). College students may also be burdened with a

different and perhaps higher level of stress stemming from their academic demands (Frazier,

Youngstrom, Glutting, & Watkins, 2007). While college students with ADHD appear to have in

common the aforementioned protective factors, they fall behind their non-ADHD college peers

in many areas. College students with ADHD report more academic problems, have lower GPAs,

and have a higher rate of academic probation than non-ADHD college students (Barkley et al.,

2008; DuPaul et al., 2009; Heiligenstein et al., 1999).

As mentioned previously, there have been relatively few studies on college students with

ADHD. Of the studies published, several have identified barriers that students with ADHD face

that are different than their non-ADHD counterparts. They have difficulty completing tests in a

timely manner, require longer amounts of time to complete assignments, and they have the

perception that they work harder than their non-ADHD peers (DuPaul et al., 2009).

Additionally, Norwalk, Norvilitis, & MacLean (2008) suggested that college students with

ADHD have poorer study habits, a lack of study skills, more difficulties with academic

adjustment (Anstshel et al., 2009; DuPaul et al., 2009), and a lower quality of life (Grenwald-

12

Mayes 2002). Another study by Shae-Zirt, Popali-Lehane, Chaplin, and Bergman (2005)

indicated that college students with ADHD had poorer social skills and lower self-esteem.

College students with ADHD tend to have difficulties with time-management,

organization, concentration, motivation, focusing, test-tasking skills, and study strategies

(Meaux, Green, & Broussard, 2009; Proctor & Prevatt, 2009; Reaser, Prevatt, Petscher, &

Proctor, 2007). They also identify as having poor reading comprehension, sleeping difficulties,

they tend to fail to utilize resources and supports (Meaux et al., 2009), and report more anxiety

(Proctor & Prevatt, 2009; Reaser, et al., 2007). College students with ADHD must learn to

navigate a variable schedule with no structure and an overwhelming number of campus activities

to choose from. These complexities were likely managed by their parents prior to college

(Meaux et al., 2009) resulting in a lack of developed skills to cope with these challenges. Given

these difficulties, it is not unexpected to see a higher school failure rate in college students with

ADHD (Antshel et al., 2009; Barkley et al., 2006; Biederman et al., 1993, 2006).

The college environment brings about many additional struggles in a person with ADHD

than is seen in the non-college environment. Due to this, college students with ADHD are

thought to struggle even more due to deficits in their executive functioning (EF) abilities

(Boonstra, Oosterlaan, Sergent, & Buitelaar, 2005). College students with ADHD have

significantly poorer EF ability, and EF problems are thought to directly affect performance in the

academic world (Nadeau, 2005). This includes problems focusing, making deadlines, task

completion, and sustaining effort in presumed irrelevant tasks (Murphy, 2005; Proctor & Prevatt,

2009). Additional skills, including self-organization and goal-setting, fall under the realm of EF

and are generally impaired in the ADHD population (Wolf, 2001).

13

Diagnostic Issues

ADHD is currently defined by the Diagnostic and Statistical Manual of Mental Disorders,

Fifth Edition (DSM-5) as “a persistent pattern of inattention and/or hyperactivity-impulsivity that

is more frequently displayed and more severe than is typically observed in individuals at a

comparable level of development” (APA, 2013, p. 59). There are three subtypes of ADHD:

ADHD Predominantly Inattentive Type, ADHD Predominantly Hyperactive-Impulsive Type,

and Combined Type. An individual must have six symptoms of inattention or six symptoms

hyperactivity-impulsivity to be classified as predominately that particular type. If an individual

has a minimum of six of each type, then they are considered to be combined type. For adults, the

minimum threshold of symptoms is five, rather than six in the new version of the DSM. These

symptoms must persist for at least six months and must cause impairment across multiple

settings, which are not consistent with their developmental level. Even though these are the

official diagnostic criteria, there is considerable debate in the field whether this set of standards

captures the essence of this disorder. As will be discussed, many top researchers in the field

have attributed several of the impairments associated with ADHD to be deficits in EF. There is

some evidence that the severity of symptoms of ADHD is associated with deficits as measured

by EF tests (Barkley et al., 2008; Jonsdottier et al., 2006; Stavro, Ettenhofer, & Nigg, 2007) and

EF behavior rating scales (Barkley, 2011c; Barkley & Murphy, 2010).

Diagnosing ADHD in the college student and adult population is controversial (Booksh et

al., 2010). In general, the diagnostic criteria for ADHD in all but the most recent version of the

DSM in 2013, were designed with children in mind, which makes accurate diagnosis particularly

challenging. One reason that has been suggested for this challenge is that the threshold to meet

the criteria for ADHD may not have the sensitivity needed to accurately diagnose a college

14

student or adult (Barkley, 1996; Heiligenstein et al., 1999). The age-appropriateness of the

symptoms designated in the DSM may explain some of the reason for this. Therefore, the

estimates of ADHD in adults and college students may be an underestimate. For example the

symptom “leaves seat when remaining seated is expected” is far less likely in an adult who is

capable of remaining seated for social reasons. However, the same internal struggle of

restlessness may exist and cause significant impairment. An additional difficulty in the accurate

diagnosis of college students with ADHD is the fact that in some cases, impairment does not

arise academically until the student reaches the college setting. One stipulation in the diagnostic

criteria is that some symptoms must be present as young as age twelve (previously age seven in

prior versions of the DSM). Academically advanced students may not be able to accurately

identify problems prior to that age (Booksh et al., 2010). However, research has shown that

there is no effective difference in symptoms and impairment between students who show early

symptoms and students whose symptoms do not present as impairments until later in life

(Faraone et al., 2006). Therefore, it is conceivable that an accurate diagnosis of the college

student population may not be feasible.

An additional concern in the diagnosis of college students with ADHD is that of

secondary gains. Given the rampant abuse of stimulant medications, some students do attempt to

feign a diagnosis of ADHD to gain access to cognitively enhancing medication (Booksh, to al.

2010). The stimulant medication used to treat ADHD also increases the ability to maintain

attention in the non-ADHD population (Rapoport, Buchsbaum, Zahn, Ludlow, & Mikkelsen,

1978), which can encourage misuse. The literature suggests that there has been an increase in

the misuse of simulant medication in the university setting (McCabe, Knight, Teter, & Wechsler,

2005; Teter et al., 2005; Tuttle, Sheurich, & Ranseen, 2007; White, Becker-Blease, & Grace-

15

Bishop, 2006). Incentives other than medication for ADHD malingering exist as well. With the

Rehabilitation Act of 1983, students with disabilities are entitled to the provision of

accommodations in the academic setting (Gordon & Keiser, 1998). The greatest incentive is the

accommodation for extended test-taking time, which can include high-stakes testing

accommodations. Information about ADHD is readily available on the internet, which may

increase the likelihood that a student will feign their symptoms to receive a diagnosis of ADHD.

Given these issues, effective assessment tools designed for the college population are needed.

The Relationship Between ADHD and EF: How Biology and Neuropsychology Inform Our

Understanding

The following section briefly discusses the biological and neuropsychology literature

which links ADHD and EF. The biological underpinnings of ADHD and how this is linked to

EF are addressed. Additionally, a brief synopsis of pharmacological treatment for ADHD is

discussed, especially as that informs our knowledge of EF.

There is a robust body of evidence from direct neuro-imaging studies which indicates

that persons with ADHD demonstrate patterns of hypoactivation in the areas of the brain

responsible for EF. A meta-analysis of 55 fMRI studies provides some of the strongest support

for this assertion (Cortese et al., 2012). In adults and children, Cortese and colleagues found a

consistent pattern of hypoarousal in areas of the brain responsible for EF (Kassubek et al., 2005;

Koechlin et al., 2000; Lewis et al., 2004; Monchi et al., 2006; Niendam et.al. 2012; Stuss &

Alexander, 2000; Stuss et al., 2002). These areas were specifically observed during EF tasks in

persons with ADHD, and the results of this meta-analysis confirms that the current body of

literature describes direct evidence of a differential level of activation in these areas of EF for

persons with ADHD vs. normal adults and children. It is notable that for adults in the Cortese et

16

al., study, the areas of hypoactivation were overwhelmingly (97%) associated with the

frontoparietal network, which is the area directly associated with EF. Children, on the other

hand, demonstrated similar rates of hypoactivation in ventral attention (44%) and frontoparietal

(39%) networks. This suggests that the neurobiology of adults and children with ADHD may

differ in measurable ways. The difference in age is an important point given that Barkley’s

theory (described in the following sections) proposes that EF abilities are not fully developed

until around age thirty (Barkley, 2012).

In addition to evidence from imaging studies, there has been a parallel course for medical

treatment of ADHD and areas of EF, particularly the ability to sustain attention.

Pharmacological interventions intended to improve sustained attention have been associated with

ADHD. Most pharmacological intervention rests upon, as Biederman, Spencer and Wilens

(2004) put it, “the idea that catecholaminergic hypoactivity in frontal subcortical circuits underlie

the disorder (p. 14).” Essentially, the individual with ADHD experiences lower cortical arousal

in prefrontal areas. The regions with the greatest role in attention, impulse control, and planned

behavior remain in a sub-optimal range. Despite discussion regarding the functions of specific

transmitters implicated in the reward pathways (see Gonon, 2008 for a detailed analysis),

suboptimal cortical arousal remains the primary explanatory paradigm. In other words, in the

same areas that persons with EF deficits due to injury have lesions, persons with ADHD display

a natural tendency toward hypo-activation. Thus, ADHD can be seen as a cluster of long-term,

naturally derived set of deficits in executive functioning due to suboptimal activation (rather than

damage) in key portions of the brain.

The theory of suboptimal arousal in ADHD (as described above) was most eloquently

and memorably laid out by Zentall & Zentall (1983). Although this preceded the current

17

diagnostic understanding of ADHD, they asserted that hyperactive behavior “represents a

functional set of homeostatic responses to conditions of abnormal sensory input. Attempts to

correct chronic imbalances in arousal through antecedent manipulations of chemical and sensory

stimulation have been relatively successful…” (pp. 446). In essence, hyperactive behavior is not

a direct symptom of ADHD. Rather, it represents an attempt on the person’s part to manipulate

or interact with their environment in such a way as to increase the amount of stimulation

available. This is done because (as noted above) the systems responsible for maintaining

vigilance and sustaining goal-directed behavior are receiving a sub-optimal level of stimulation.

This has been well established by assessing working memory (Rottschy et. al., 2012), and long-

term EEG patterns in ADHD (Doehnert, Brandeis, Schneider, Drechsler, and Steinhausen, 2012;

Loo, et al., 2009). The electrophysiological responses indicative of preparation or attention are

markedly suppressed in groups of persons with ADHD vs. normal populations.

A proper understanding of the neurobiology of ADHD and associated deficits in

executive functioning forms an important foundation for the comprehensive diagnosis and

treatment of ADHD. A common misunderstanding about ADHD in clinical practice is due to

the variability in symptoms. People with ADHD can focus without difficulty on activities they

find salient or when they are avoiding a negative consequence (Baxter & Murray, 2002). There

are times that they may be relatively unimpaired by their ADHD. This apparent ability to

selectively attend to some tasks and failing to attend to others is likely what led the general

public to assume that people with ADHD just lack willpower. According to Brown (2008),

however, the cause of the situational variability is essentially chemical. When confronted with a

task that is personally appealing or frightening because of consequences, the brain instantly

provides chemical stimulation via neurotransmitters to activate relevant EF (Brown, 2008).

18

Baxter & Murray (2002) suggest that this is not subject to voluntary control. The results here tie

in with the aforementioned body of research from Zentall and Zentall (1983), which is a

theoretical framework and empirical support for the perspective of ADHD as a deficit in

physiological arousal, rather than attention.

The suboptimal arousal model indicates a suboptimal natural level of arousal in the areas

responsible for focusing attention in the individual with ADHD. When internal and external

stimulation are insufficient, the individual attempts to regulate their own levels by changing

activities, attentional focus, or experience (Zentall, 2005). This is an important distinction

between persons with ADHD and persons with lesions in areas of the brain responsible for poor

executive functioning: For persons with brain damage, performance on measures of ability in

impacted areas may be expected to depend primarily upon the nature of the task. For persons

with ADHD, the context and level of arousal may be just as important as the actual criteria of the

task. Thus, there is an implication that although on average persons with ADHD can be

expected to display dysfunctional EF, performance on a single discrete task may not be impaired.

This poses a number of psychometric issues with EF tests, which is addressed later in the

manuscript.

Limitations of the Biology Based Model

Although the preceding biological model of EF as it relates to ADHD incorporates the

building blocks for future models of this relationship, the current model falls short of

conceptualizing the larger picture. The model does not take into consideration ideas of

emotional control, self-awareness, or the social aspect of EF (Barkley, 2012). Impairments in

emotional control, occupational difficulties, and moral difficulties are absent from the biological

model. However, such deficits are readily apparent in patients with injuries to the areas of the

19

brain responsible for EF. Barkley postulated that social facets of EF dysfunction are overlooked

because they are not as readily observed in a laboratory setting (measured by EF tests).

Barkley’s interpretation is not without precedent. Eslinger (1996) viewed EF as being

comprised of “social executors,” which serve the social functions. This is one of the only models

of EF to include discrete social components. These components are: social regulations, social

self-awareness, social sensitivity, and social salience. He considered social problems “the most

distinctive feature” of EF. However, there was no truly comprehensive model that addressed

social functioning of EF, and how it relates to ADHD, until Barkley (2012) promulgated his

theory of EF.

Barkley’s Hierarchical Theory of EF

Despite the ways (described in the preceding section) in which his theory differs from the

classical neuropsychological perspective, Barkley’s view of EF is anchored in self-regulation:

“EF is the use of self-directed actions (self-regulation) to choose goals, and to select,

enact, and sustain actions across time towards those goals, usually in the context of others

and often relying on social and cultural means. This is done for the maximization of

one’s long-term welfare as the person defines that to be” (Barkley, 2012 p. 167).

From a functional perspective, EF is “a form of self-directed action aimed at modifying

one’s behavior so as to make a future goal, end, or outcome more or less likely to occur.”

(p. 167).

Each component of EF represents a particular type of self-regulation (Barkley, 1997a,

1997b). In a variation on this then, Barkley refers to EF behaviors as “those self-directed actions

needed to choose goals and to create, enact, and sustain toward those goals (p.168),” meaning

that EF equals self-regulation (Barkley, 2012). This is not a novel conclusion. Wagner and

20

Heatherington (2011) and Eslinger (1996) found this to be the most common feature of EF in

neuropsychology. The view of EF as self-regulation comes up frequently in the role of working

memory, which is one of the many cognitive abilities of EF (Hoffman, Friese, Schmeichel, &

Baddeley, 2011). Self-regulation is also described in the linkage of attention networks

specifically, and EF more generally (Rueda, Posner, & Rothbard, 2011).

If we accept Barkley’s proposition that EF is a means for goal-directed action, a

definition of “goals” and “means” must be provided. Goals are understood to be things in the

future that provide a relative reduction in dissatisfaction compared to the individual’s present

state (Barkley, 2012; Mises, 1990). The actions resulting in attainment of the goal are the means

to that end. Goals are completely subjective and personal. This means that methods for

reduction of dissatisfaction are personal, and cannot be judged as good or bad from an external

perspective. Thus, this type of goal cannot be assumed for the sake of empirical measurement.

Furthermore, if two people make a decision using the same information, they may well come up

with differing (yet viable) solutions unique to the goals and values they have (Barkley, 2011a;

2012).

Barkley’s theory of EF is built on the idea of the extended phenotype. This comprises

four levels of EF, each incorporating a set of pre-executive levels (Barkley, 2012). The extended

phenotype is a biological concept introduced by Dawkins (1982). Essentially, a phenotype

extends beyond the organism into its environment, incorporating all effects that the gene has on

its environment. Dawkins considered the extended phenotype a critical component of the

evolutionary theory (1982). Barkley’s hierarchical model, along with implications for the

manner in which EF impacts phenotypic expression, has been described in the following

sections.

21

The pre-executive level comprises the central-nervous system (CNS). This is an

integrated system with a high degree of automaticity. Included are attention, memory, spatial

and motor functions, primary emotions and motivations. Pre-executive functions include

behaviors such as automatic responses and operant conditioning. Operant conditioning occurs

when animals learn, which allows rapid adaptation to the environment rather than relying on the

laborious evolutionary process (Barkley, 2012). Operant conditioning is not EF, but is a

prerequisite ability.

The first EF level is the instrumental-self-directed level (Barkley, 2012). The basic

neuropsychological functions at the pre-executive level take on a self-directed component in the

initial level of EF. According to Barkley, these include: self-directed attention (self-awareness),

inhibition (self-restraint), self-directed sensory-motor actions (nonverbal working memory;

imagination), self-directed private speech (verbal working memory; verbal thought), self-

directed appraisal (emotion-motivation), and self-directed play (innovation, problem solving).

Early in the organism’s development, these processes move from observable behaviors to

opaque, internalized functions. These are the underlying components of goal-attainment. The

title of “instrumental” is derived from the observation that these components are used to achieve

goals but are not directly capable of attaining the goals (Barkley, 2012).

The methodical- self-reliant level includes observable goal-directed actions and

behaviors (Barkley, 2012). A brief sequence of actions leading to a goal is termed a method (a

recipe). In this level, a single step is insufficient to attain a goal, necessitating the chain of

action. This level is typified by short or near-term goals. There is a component of self-reliance

because the goals are general oriented towards independence from defense against others. This

level is associated with executive behavior rather than executive cognition (Barkley, 2012).

22

In the tactical –reciprocal level, the concept of social symbiosis is introduced into the

EF system. At the methodical level, independence and self-defense are privileged. In this level,

others are used to obtain goals of mutual benefit, in what Barkley describes as a socially

symbiotic relationship. The word tactical refers to linking longer-term actions together to

achieve a higher order goal (Barkley, 2012).

The strategic-cooperative level refers to the hierarchical organization of tactics leading

to multifaceted, multi-stage behavioral patterns over an extended timeframe. This allows for

larger-scale, longer-term goals to be planned for. The hierarchy now consists of methods, nested

under tactics, which are nested under strategies that eventually attain goals at a substantial

distance in time. The strategic level shifts “cooperative” activities to the fore by relying on

higher order social engagement in attaining goals exceeding the ability of an individual (Barkley,

2012).

The first four levels of EF suffice in describing most human behavior. However, Barkley

argues that an additional level at the pinnacle of the hierarchy can be added. He believes that

some civilizations have reached the principled-mutualistic stage. In theory, this represents the

broadest community or social ecology in which most persons live. This level synthesizes sets of

strategies with an eye towards the most expansive and long-term aspects of human goal-seeking.

It applies principled behavior to a community setting in order to simultaneously attain long-term

self-interests and mutualistic community goals (Barkley, 2012).

Barkley (2012) describes the sequential development of each level as being directly

related to the increase in the prefrontal cortex capacities, primarily involving future-oriented

thought. Barkley indicated that through theory and fMRI imaging, individual’s anatomical areas

of the brain responsible for EF functioning are not fully developed until an individual’s late 20’s

23

or early 30’s (Barkley, 2012). Since the maturation process is not fully developed in the EF

system until approximately 30 years, it has been suggested the essential point in Barkley’s model

versus most previous models is the social component. Barkley’s (2012) model is tied heavily to

the concepts of a behavioral impairment, rather than focusing on specific brain function. As

such, his theory would be difficult to measure with EF tests (psychometric) alone, and can best

be assessed by functional outcomes (e.g. impairments). As will be discussed in a later section,

Barkley has developed a behavior rating scale to assess impairment in EF function, reducing the

need for EF test, which have been the gold standard in the field.

In assessing damage to the prefrontal cortex, EF functions can be classified along eight

parameters whose development is tied to the maturation of the prefrontal cortex. In his

explanation, Barkley (2012) identifies the eight characteristics as separate entities; however, he

admits that when the EF system is fully mature, they are interrelated. These characteristics are

spatial capacity, temporal capacity, motivational capacity, inhibitory capacity, conceptual/

abstract capacity, behavioral-structure capacity, social capacity, and cultural capacity. In spatial

capacity, the “individuals come to purposefully rearrange or organize their surrounding physical

environment to assist in goal attainment” (Barkley, 2012 p. 74). This requires an ability to

mentally represent spatial distances. The ability to “reflect how far into the future an individual is

capable of contemplating a goal” (Barkley, 2012 p. 74) is temporal capacity. Sometimes this is

referred to as the “time horizon.” This refers to the time between an event and the preparation

for that event. The preceding capacities are required as a basis of the motivational capacity,

“which is distinguished by the fact that this capacity reflects an appraisal of that future, while the

spatial and temporal capacities above are comprised of cognitive parameters or purely

informational” (Barkley, 2012 p 75). The motivation comes from the personal value placed on

24

the delayed goal or outcome that is given by the individual. Oftentimes, attainment of the goal

relies on how strongly the individual feels about the goal. The “extent to which and the duration

over which an individual must inhibit their responses to prepotent events, restrain their actions,

and otherwise subordinate their immediate interests for the sale of the goal” (Barkley, 2012 p.

76) is considered inhibitory capacity. Individuals who are unable to do this are often classified

as irrational, impulsive, or selfish. Conceptual/abstract capacity is “the degree of abstractness of

any rules that are being considered or followed to attain a goal” (Barkley, 2012 p 76). This can

be simple rules such as “stop” and can be more complex such as “do onto others as you would

have done onto you.” Behavioral-structure capacity “is a capacity for structuring increasingly

complex, hierarchically organized and appropriately sequenced actions towards goals” (Barkley,

2012 .p. 76). This sequenced action is represented in each of the levels of EF as indicated by the

first term, from the instrumental capacity to the principled capacity. The last two capacities

represent the social environment discussed above. Social capacity refers “to the number of other

individuals (and eventually social networks) with which the individual must interact, reciprocate,

and cooperate so as to effectively attain the goal being contemplated” (Barkley, 2012 p. 76).

Cultural capacity “refers to the degree of cultural information and devices (methods, inventions,

products, etc.) or scaffolding that the individual is adopting to attain the goal under

contemplation (Barkley, 2012 p. 77).

In summary, Barkley’s hierarchical model focuses on self-directed behavior and the

ability to utilize self-regulation. The hierarchical levels begin at the pre-executive level and

progresses to the most advanced level of the principled-mutualistic stage, which represents the

most socially and culturally bound level. Barkley then looks at eight developmental parameters,

which are more behavioral in nature and reflect impairments found in individuals with prefrontal

25

cortex damage. This includes motivational capacity, temporal capacity, and inhibitory capacity,

among others. Barkley has also added a sociocultural aspect to EF, which was lacking from

previous models and definitions. Additionally, Barkley’s model operationalizes the definition of

EF, allowing the concept to be empirically measured. As will be discussed in a later section,

Barkley has also developed a rating scale to assess for deficits in EF based on this model

(Barkley, 2011b).

EF Deficits as Manifested in ADHD

Background

In preceding sections, EF has been described largely from a biological or theoretical

(rather than clinical) perspective. However, several other clinical diagnoses are often associated

with deficits in EF. As was previously noted, brain damage (such as TBI or stroke) (Levin &

Hanten, 2005; Weyandt, 2009) often results in acquired deficits in EF. Some mental illnesses

(especially mood disorders) (Murphy, Barkley, & Tracie, 2001; Thompson et al., 2009) are also

associated with fluctuations in EF during acute phases. However, few diagnoses share the same

degree of overlap with EF deficits as ADHD. Over the years, multiple researchers in the field

have linked ADHD to deficits of EF (Barkley, 1997, 2011a, 2012; Biederman et al., 2004;

Biederman et al., 2006; Biederman et al., 2007; Boonstra et al., 2005; Brown, 2008; Chelune et

al., 1986; Grondzinsky, & Diamond, 1992; Fuster, 1997; Hervey et al., 2004; Nigg & Casey,

2005; Oosterlaan, Scheres & Sergent, 2005; Seidman et al., 2006; Welsh & Pennington, 1988;

Weyandt, 2009; Willcut et al., 2005). Based on this sample of current literature, the controversy

is not about whether ADHD is associated with EF deficits. Rather, it hinges on the way in which

they are associated. As previously discussed, there is sufficient evidence from the biological

perspective that ADHD and EF are related. Some researchers outside the field of biological

26

neuroscience indicate that there is overlap between ADHD symptoms and EF deficits as

determined by EF tests. Others, such as Barkley (2011c), have gone as far as to say that ADHD

and EF deficits are synonymous, that ADHD can be called Executive Function Deficit Disorder.

Barkley’s Theory of EF and its Relationship to ADHD

Barkley (2012) has presented, to date, one of the most comprehensive theories on the EF

as it relates to ADHD described above. Based on his Hierarchical Theory of EF, Barkley has

mapped the symptoms and impairments of ADHD onto levels in his theory of EF. As a review,

Barkley proposed a hierarchical theory of EF including six levels; pre-executive,

instrumental/self-directed, methodical/self-restraint, tactical/reciprocal, strategic/cooperative,

extended/utilitarian. He then maps the components of ADHD impairments onto these levels.

Again, Barkley has defined EF as:

“the use of self-directed actions (self-regulation to choose goals and to select, enact, and

sustain actions across time toward those goals usually in the context of others often

relying on social and cultural means. This is done for maximization of one’s long-term

welfare as the person defines that to be” (Barkley, 2011a p.167).

Barkley (2012) asserts in his theory that there are three brain networks associated with

ADHD and EF deficit. 1) The frontal–striatal circuit, which is related to deficits in response

suppression, inhibition, working memory, organization and planning. Barkley has dubbed this

the “what” system of EF. 2) The frontal-cerebellar circuit which is connected to motor

coordination deficits and difficulties with timeliness of behavior. This is recognized as the

“when” of EF network. 3) The frontal-limbic circuit is related to the systems of emotional

control, motivation deficits, hyperactivity/impulsivity, and low frustration tolerance (aggression),

or the “why” in Barkley’s model. Given that all of these brain networks of EF are readily

27

identified as ADHD symptoms, Barkley (2011b, c) has asserted that ADHD essentially equals

Executive Functioning Deficit Disorder.

Barkley (2011b) then supports this view with empirical research. How does ADHD fit

with EF? A factor analysis of ADHD symptoms (using symptoms checklists) and EF symptoms

(using a behavior rating scale) was conducted (Barkley, 2011a), resulting in one overarching

factor, similar to g in intelligence theory, which supports his claim that ADHD is Executive

Functioning Deficit Disorder.

Barkley (2011b, c) has indicated that he does not agree with the very name of ADHD

(attention deficit) because it is essentially useless as a diagnostic feature. Similarly, Zentall

(1983; 2005) argued that attention deficit was a misnomer. Many disorders, (depression, bipolar,

anxiety, schizophrenia) (Murphy, Barkley, & Tracie, 2001; Thompson et al., 2009) have

inattention as a symptom. Barkley narrowed down the concept of inattention in ADHD

specifically and defined it as inattention towards a task or outcome, future goal, which must be

organized and monitored over time. He suggests that working memory deficits are the reason for

this inability to remain focused on a future goal, and the diagnostic criteria of “changing from

activity to activity or not completing a task.” Again, when symptoms of ADHD and EF

symptoms were subjected to a factor analysis, one large factor was evident. If you add the

results from EF tests, it does not change the factor structure. Essentially, Barkley asserts, ADHD

is Executive Functioning Deficit Disorder. Barkley, however, does not appear to address the

issue of other deficits or disorders that also have deficient or damaged prefrontal cortex. For

example, a patient who is post cardio vascular accident generally has deficits in EF. It is not

addressed whether this would also be classified under the same umbrella of Executive

Functioning Deficit Disorder, as ADHD is in Barkley’s model. Barkley (2011b, c) claims all

28

ADHD is Executive Functioning Deficit Disorder, but is all Executive Functioning Deficit

Disorder also ADHD?

Barkley (2011b, c) indicates that severe prefrontal cortex injuries often cause significant

impairment in all levels of the EF hierarchy, including at the lower levels. This may be related to

the fact that EF impairments in individuals with traumatic brain injuries tend to show up on EF

tests more consistently than in individuals with ADHD. However, prefrontal cortex disorders

tend to show greater impairment in the highest two levels of EF, as presented by this model. EF

disrupts all five levels of EF/Self-Regulation. It is particularly troublesome in the tactical level

and higher levels, which in turn generates difficulties of self-regulation across time. The ability

to hierarchically organize behavior, anticipate forthcoming events, and maintain a long-term goal

direction is affected. Barkley suggests that it is not an Attention deficit disorder, but an Intention

deficit disorder (attention to mental events and the future). ADHD is a deficit in performance,

not knowledge (Barkley, 2011b).

The diagnosis of ADHD, especially in adults, is somewhat complex. There is no

biological or neurological test that can confirm a diagnosis. This introduces error into clinical

practice and research. A problem in the field arises when researchers attempt to evaluate the

relationship between ADHD and EF deficits using EF tests, which have been the “gold standard”

for the past several decades when assessing EF (Barkley, 2012). These tests measure cognitive

abilities such as working memory, inhibition, and attention. One criticism of EF tests is that they

measure a single point in time and do not require the subject to self-regulate behavior over time

(Clark, Prior, & Kinsella, 2000). The following sections will describe some of the current EF

tests that evaluate cognitive abilities, explain why they are controversial, and then describe

measurement of EF via behavior rating scales.

29

Assessing EF

There are a number of EF tests; however, there are also a number of inherent problems

with assessing the construct. Following is a discussion about some of the current available tests

of EF functioning, and by extension, measures of ADHD. One problem with assessing EF is that

there is substantial evidence supporting the view that, at least as assessed by common measures,

EF and intelligence (IQ) are related (Jung, Yeo, Chiulli, Sibbitt, & Brooks, 2000).

Certain researchers have gone so far as to suggest that EF and IQ represent essentially the

same thing (Obonsawin et al., 2002). However, this is far from a majority view. Some lines of

research indicate that EF is mediated by IQ (Antshel et al., 2010). Others say they are only

somewhat related, but are not the same thing (Ardila, Ostrosky-Solis, Rosselli, and Gomez,

2000; Miyake, Friedman, Emerson, Witzki, & Howerter, 2000). At the present time, there are

two primary ways of measuring EF: neuropsychological tests (psychometric) and rating scales

(questionnaires). Psychometric neuropsychological tests (or EF test) are defined as an objective

measure of an ability to perform a task, e. g., speed at which a task is completed or number of

digits repeated backwards. An EF behavior rating scale is a measure of subjective belief in a

person’s ability to accomplish a task and tends to focus on perceived impairment, e.g., a

questionnaire asking about motivation or frustration tolerance. There is a long-standing history

of utilizing EF tests in assisting in the diagnosis of ADHD; however, the use of EF behavior

rating scales of impairment is relatively new.

Given the many cognitive abilities associated with EF, there are multiple ways in which

researchers approach measuring the construct. Some EF tests are designed to assess specific and

isolated abilities, such as working memory or inhibition. Others are grouped into test batteries

which purportedly measure “executive functioning.”

30

EF Tests

A large variety of assessment tools exist, and a complete review is outside the scope of

this literature review. Thus, only representative measures and batteries will be discussed, and

only as they relate to the EF abilities which they were intended (or are commonly used) to

measure.

Working memory. Working memory is arguably the best studied construct of EF.

Several common working memory tests are: The Simon Game, testing non-verbal working

memory (Lezak, Howieson, & Loring, 2004); Digit Span and Arithmetic, subtests of the

Wechsler Adult Intelligence Scale (WAIS –IV; Wechsler, 2009); and the Six Element Task

(SET; Burgess & Shallice, 1996). Generally these EF tests measure working memory by

incorporating components of straight memorization and a manipulation (or interference) task.

For example, in the Digit Span subtest of the WAIS-IV, a participant repeats a string of numbers,

and then repeats them backwards. Finally, the numbers are recited in sequential order.

Inhibition. To assess the construct of inhibition, common EF tests include: The Stroop

Color Word Test (Stroop, 1935; Trenerry, Crosson, Deboe, & Leber, 1989), the Hayling

Sentence Completion Test (HSCT; Burgess & Shallice, 1996, 1997), and the Wisconsin Card

Sort (WCST; Heaton, 1981). As is common on this type of test, they measure the capability of

inhibiting opposing responses with the existence of salient and contradictory stimuli. Although

the WCST requires the ability to inhibit previous response patterns (which will increase the

number of perseverative responses), it also requires several other EF abilities. These include

planning, attention, set shifting, and working memory. The Color Stroop Test and the WCST are

the two most common tests in this category, with the Color Stroop Test being the most common

measure which is primarily an inhibitory task.

31

Problem solving and set-shifting. Problem solving and set-shifting are additional

constructs often assessed with EF tests. Common tests used are WSCT; Tower of London

(Shallie & Burgess, 1996), also known as the Tower of Hanoi; Trail Making Tests (TMT-A,

TMT-B; Reitan & Wolfson, 1985); and Cognitive Estimations Test (CET; Shallice & Evans,

1978). Other than the WSCT, discussed above, the Trail Making Test is the most notable. This

test purports to measure visual search, scanning, processing speed, and mental flexibility (set-

shifting). There were relatively few norms for this set of tests in its earlier years, despite its

common use (Matarazzo et al., 1974; Reitan, 1959). However, this subtest has been incorporated

into the Halstead-Reitan Neuropsychological Battery, which will be discussed later, and now

includes the ability to norm the scores appropriately.

Vigilance/attention. Vigilance, also referred to as attention, is another common

construct with several EF tests available to measure it. These are also used heavily in the

evaluation of ADHD, which is discussed in more detail in a following section. There are many

tests of vigilance or attention, but some of the most notable are the: Digit Vigilance Test (Lewis,

Kelland, & Kupke, 1990), Test of Variables of Attention (TOVA; Greenberg, 1991), Conners’s

Continuous Performance Test (CPT; Conners, 1995), and Gordon Diagnostic System (Gordon,

1983). Many of these tests work on the same basic principle as the CPT, and often incorporate

computerized administration. A single character is shown on the screen at a rate of

approximately one per second. The participant is to click a button when a specific character

(known in advance) is displayed. The total time of the test varies from a few minutes to

approximately 12 minutes. The relatively brief administration times (compared to more

research-oriented measures of vigilance in fields such as human factors) are likely chosen for

practical rather than psychometric reasons. Generally speaking, a score for total errors,

32

commission errors, and omission errors is given. The task requires the participant to remain

vigilant for extended lengths of time. The research has shown an increase in false positives with

a CPT type test (Fischer, Newby, & Gordon, 1995). Most of the research in this area is on

children with ADHD. In children with ADHD, a study showed that there was little ability for the

Conner’s CPT to discriminate between ADHD children and a clinical control group (McGee,

Clark, & Symons, 2000). A different study conducted by Forbes (1998), found that the TOVA

had a greater ability to predict ADHD children from control group children than the CPT did.

However, these types of studies have been plagued by methodological problems.

EF test batteries. In addition to EF tests that measure single constructs (such as the

CPT), several full batteries of neuropsychological tests (EF test batteries) have been developed to

measure a range of EF abilities. The most notable of these batteries is the Halstead-Reitan

Neuropsychological Test Battery (Reitan & Wolfson, 1985); Delis-Kaplan Executive

Functioning System (D-KEFS; Delis, Kaplan, & Kramer, 2001); and the NESPY-2 (Korkman,

Kirk, & Kemp, 1998), a developmental neuropsychological assessment for children. In addition

to these batteries, a single test may capture several aspects of EF. For example, the WCST is

also a single test design which purports to measure several constructs of EF collaboratively.

Similarly, the Rey-Osterrieth Complex Figure (Rey, 1941) is a single measure requiring the

ability to utilize several abilities.

The lack of a set of neuropsychological measures that had cohesive norming data was

once a primary concern in the clinical field of neuropsychology. The above mentioned test

batteries were developed by combining many of the commonly used single construct tests into a

battery, which was then normed at one time (Homack, Lee, & Ricco, 2005). These batteries

include subtests covering all of the constructs individually and in collaboration with each other.

33

Despite this, controversy remains active in the research community regarding these full-battery

tests of EF, especially about their reliability and validity (Schmidt, 2003). Additionally, research

and psychometric information on these tests often does not include a discussion of clinical

utility. The tests purport to measure a specific deficit; however, there is little information on

how that deficit affects the individual with regards to specific areas of day to day impairment.

Limitations of EF Tests

Disagreement exists regarding the best way to measure EF. Proponents of EF tests

measuring a single construct would state that EF represents multiple, separate processes working

together. However, the fact that few (if any) of these tests can measure purely one process lends

credence to the idea that EF is NOT a single ability, but a group of abilities working together.

One could not use working memory without first maintaining attention to the stimulus data. One

could not set-shift or problem solve without the ability to inhibit interference, use working

memory to maintain the problem to be solved, or maintain attention to the problem as it is

expressed. Although these processes are laid out as separate abilities, it is impossible to use one

without employing another.

It is noteworthy that the construct of IQ has been largely absent from the equation. Yet

Salthouse (1996, 2005), hypothesized that the underlying cognitive abilities of perceptual speed

and reasoning are factors related to all EFs. There has been some research to support this idea

(Duncan Emslie, Williams, Johnson, & Freer, 1996). Salthouse (2005) noticed that performance

of the WCST was related to reasoning ability and perceptual speed. In fact, the original manual

for the Gordon Diagnostic System (a relatively straightforward continuous performance task)

notes a relationship between the processes that the Gordon measures and intelligence.

34

In decades of research, only a minority of adults with prefrontal cortex damage or ADHD

consistently perform in the impaired range across all these tests of specific abilities or combined

batteries (Alderman, Burgess, Knight, & Henman, 2003; Vriezen & Pigott, 2002). The results of

EF test batteries, according to Barkley and colleagues, had no or low relationship to the

impairments evident in their daily life as measured by behavior rating scales, suggesting low

ecological validity (Anderson et al., 2002; Barkley & Fisher, 2010; Barkley & Murphy, 2010;

Burgess et al., 1998; Chaytor et al., 2006; Wood & Liossi, 2006). These studies have found only

0-10% shared variance between any single EF test and an EF behavior rating scale (Barkley &

Murphy, 2010). This means that an EF test measuring a specific process in the lab (e.g., working

memory) may not correspond to daily functional impairments (which are measured by the

behavior rating scales). There is little consistency throughout the literature on the validity of the

tests of EF. Chaytor (2004) attempted to increase the ecological validity of EF tests. In this

study, and replicated in a later study (Chaytor et al., 2006), results indicated that when selecting

from the most reliable and valid tests of EF (WSCT, TMT, and Stroop), the most variance that

could be accounted for in impairments measured by behavior rating scales was 18-20%.

There are several areas of concern regarding the ability of EF tests to accurately identify

deficits in EF in daily life. As Barkley (2012) has described it, it is clinically useless to know

how a patient performs in the lab if that performance does not translate to the real-world and

their daily lives. Moreover, many of the definitions and theories of EF suggest an organization

of behavior, cross- temporally, towards a future goal (Fuster, 1997). It is unclear how EF tests

are able to determine progress towards a goal when testing sessions vary anywhere from five

minutes to a few hours, and most single subtests of a construct last less than an hour. This does

not mean that a test of EF does not measure problem solving over that course of time, but it may

35

do little to shed light on the daily difficulties of maintaining attention to a goal over days, weeks,

and years (Barkley & Murphy, 2010), such as would be needed in earning a college degree. An

additional problem with EF tests is the complexity in administering and interpreting the results

(Anderson, 2002; Castellanos, Sonug-Barke, Milham, & Tannock, 2006). Finally, as was

mentioned, the research is unclear as to whether intelligence significantly affects the results of

EF tests (Mahone, Pillion, Hoffman, Hiemenz, & Denckla, 2002). Although an association with

IQ would not preclude the use of EF tests, it would require that the norming of the EF measures

incorporate IQ as a factor, just as age is currently incorporated. This is not currently the practice

for test manufacturers.

EF Rating Scales of Impairment

Given the inherent problems with EF tests described above, other methods of measuring

EF have come to the forefront in the literature. EF behavior rating scales have grown in

popularity with clinicians, but not without continued controversy in the field. In general, a

behavior rating scale is a standardized set of items (questions) in which the informant makes

judgments about his or her behaviors or someone else’s behavior (Merrell, 1994). This relies on

direct observation of the person or self-perceptions of that specified behavior (Merrell, 1994). In

the case of an EF behavior rating scale, items focus on the different aspects of EF, including the

temporal and social components often lacking in EF tests. Behavior rating scales in general

assume that the informant has sufficient knowledge of the index person’s relevant behavior to

make the judgment (Fennerty, Lambert, & Majsterek, 2000) across considerably longer periods

of time (Barkley, 2011b). These items capture symptoms in a way that purports to improve the

ecologically validity as they happen in social contexts (Barkley, 2011b) rather than a snapshot in

time, such as an EF test. Barkley (2011b) also argued that EF behavior rating scales have more

36

face validity because the items are based directly from his theory of EF. As mentioned

previously, EF tests are likely contaminated by intelligence, and EF behavior rating scales are

not likely subject to this contamination. An additional motivation behind EF behavior rating

scales (and rating scales in general) is that they are generally more cost effective to administer

than EF tests (Barkley, 2011b). They do not require a specialized set of skills to administer.

Rating scales in general have the benefit of accessing an informant’s perspective with a vast

knowledge of the index person across time and when assessing infrequent behavior. Rating

scales can also tap a large variety of items and constructs within a relatively short time,

increasing their efficiency (Barkley, 2011b). With all the benefits, there are some disadvantages

to behavior rating scales in general. The informants themselves are also factors in the difficulties

experienced with behavior rating scales. The informant’s intelligence, insightfulness, education,

life experiences, and motivation all affect their ratings (Barkley & Murphy, 2010). Bias in the

informant’s intentions or malingering can affect rating scale outcomes, as can other psychiatric

symptoms. For EF behavior rating scales, symptoms of depression may be undistinguishable

from some ratings of EF deficit symptoms (Barkley, 2011b).

There are several EF behavior rating scales available and include the Behavior Rating

Inventory of Executive Functioning (BRIEF; Gioia, Isquith, Guy, Kenworthy, 2000), BRIEF-A

(adult version), the Comprehensive Executive Function Inventory (CEFI; Naglieri & Goldstein,

2013) and the Barkley Deficits in Executive Functioning Scale (BDEFS; Barkley, 2011b). The

BDEFS will be the focus of the remainder of this discussion. Because this study is a

psychometric evaluation of the BDEFS, extensive details are given regarding test development.

The BDEFS. The BDEFS is an 89 item behavior rating scale, utilizing a Likert scale (0-

rarely or not at all, 1=sometimes, 2=often, and 3=very often). The items are based largely on

37

Barkley’s theory of EF (Barkley 1997a, 2011c); however, understanding of these concepts from

other researchers (Denckla, 1996; Fuster, 1997; Welsh & Pennington, 1988) and an examination

of patients with injuries to their frontal lobes (Luria 1966; Burgess et al., 1998) were also a

foundation for items. There are five major constructs of EF measured by the BDEFS. These

constructs are noted in Barkley’s theory and in general, are listed in the literature as constructs of

EF (Castellanos et al., 2006; Willcutt, Doyle, Nigg, Faraone, & Pennington, 2005). The

constructs include the following: “inhibition; nonverbal working memory (self-directed sensing,

especially visual imagery, sense of time and time management); verbal working memory (self-

directed private speech, verbal contemplation of one’s behavior before acting, etc.); motivational

self-regulation (motivating one’s self during boring activities, etc.); and planning and problem-

solving (reconstitution, generativity, and goal-directed inventiveness)” (Barkley & Murphy,

2010, p. 41). There are two versions of the BDEFS: a self-report form and an other-informant

report form. The BDEFS has five factor scores: self-management to time, self-

organization/problem solving, self-restraint, self-motivation, and self-regulation of emotions.

Additionally, a total EF summary score and an ADHD-EF Index are given. The ADHD-EF

index is a separate score evaluating the likelihood that the individual may have adult ADHD

using 11 questions from the questionnaire (Barkley, 2011b).

Development of the BDEFS. The Barkley Deficits in Executive Functioning Scale

(BDEFS) was published in early 2011 after over a decade of development. The desire to have a

cost-effective means of conveniently capturing neuropsychological, behavioral, emotional, and

motivation symptoms often attributed to EF deficits was the motivation to develop this new

scale. Two federal grant-funded studies helped the development of the prototype of the BDEFS

(Barkley, 2011b). The first was the UMASS Study and examined clinic-referred adults with

38

ADHD, comparing these adults to a clinical control group and a community control group. The

clinical control group consisted of participants who were self-referred to the clinic to be

evaluated for ADHD, but who were not given a diagnosis based on subclinical symptomatology.

The second was the Milwaukee Study, which was a follow-up study of hyperactive children as

they entered young adulthood. Results of these two studies will be discussed in the following

sections. The original prototype consisted of 91 items developed to reflect inhibition, nonverbal

working memory; verbal working memory, emotional/motivational self-regulation, and

planning/problem solving. The focus of the BDEFS was to assess problem/deficit functioning

rather than normal functioning (Barkley, 2011b).

Barkley (2011b) published the final version of the BDEFS as an 89 item, paper-and-

pencil questionnaire, utilizing a Likert scale answer format (1-rarely or not at all, 2=sometimes,

3=often, and 4=very often). There are three separate instruments that were developed from the

original 91-item pool: self-rating scale (herein referred to as the BDEFS), other-informant rating

scale hereafter referred to as BDEFS-other, and a 20-item short-form. The self-rating scale and

other-informant scale are designed as described above. The short-form is a 20 item Short-Form

for screening purposes (Barkley, 2011b). The initial set of analyses discussed below were

conducted on the prototype BDEFS which has the original 91 items.

Initial factor structure validation. Several factor analyses were conducted (Barkley,

2011b) prior to the publication of the BDEFS. First, using the 351 clinic referred adults for an

ADHD evaluation from the UMASS study, a factor analysis revealed 10 factors with eigenvalues

over 1.00. However, only five of these factors had at least 10 items with high loading on a

factor. Three items did not have a loading ≥.400, so they were removed from the scale. The first

five factors accounted for more than 63% of the variance in the unrotated factor solution

39

(Barkley, 2011b). Also, one large factor accounted for the majority of EF deficits in daily life.

A varimax rotation was then conducted to see whether allowing the factors to correlate provided

a better fit. The factors from this initial data after rotation were:

Factor 1 (Self-management to Time) had 23 items and accounted for 15.7% of the

variance.

Factor 2 (Self-Organization/Problem Solving) had 21 items and accounted for 15.2%

of the variance.

Factor 3 (Self-Restraint or Inhibition) had 23 items an accounted for 14.1% of the

variance.

Factor 4 (Self-Motivation) had 11 items and accounted for 9.8% of the variance

Factor 5 (Self-Activation/Concentration) had 10 items and accounted for 8.6% of the

variance.

Even though these factors emerged, there were significant inter-correlations among

factors, ranging from .74 to .88 for the BDEFS and .75 to .88 for the BDEFS-other (Barkley,

2011b). Therefore, 56-77% of the variance was shared by the factors, which is in support of the

theory of an underlying metaconstruct of EF deficits (Barkley & Murphy, 2010). These above

two factor analyses are limited in that they do not represent the factor of Emotional Self-

Relegation, which was added later (Barkley, 2011b).

In analyzing the UMASS data, Barkley (2011b) determined that the adults with ADHD

rated themselves as having more severe EF deficits on all factors, compared to both the clinical

control group and the community control group. These results yielded statistically significant

differences on all of the factors. There appeared to be a progression of significant symptoms,

with the clinical control group endorsing more items, resulting in more significant deficits than

40

the community control group, in both the prototype of the BDEFS and the BDEFS-other. Using

the community control group, they were able to determine a cutoff for clinical norms for deficits

in EF, which was specified as 1.5 standard deviations above the mean of that group on the

prototype BDEFS (Barkley, 2011b). When using this cut-off, the group diagnosed with ADHD

had statistically significantly higher (more impaired) scores on the Self-Motivation sub-factor

than either of the other two groups. On all other factors, there was a statistical difference

between the community control group and the other two groups, but there was not a statistical

difference between the two clinic groups (clinical control and ADHD). This cut-off is useful to

differentiate between community controls and those with symptoms of ADHD (but not

necessarily diagnosed). Although this is not as useful as one might prefer, the prototype BDEFS

does differentiate a community sample, and the group with ADHD was higher than the clinical

control on all factors, just not statistically significant.

Up to this point, the prototype BDEFS had been used in all the analyses (Barkley,

2011b), which again lacked one of the five published factors (Emotional Self-Regulation). On

the BDEFS (published 89-item version), only items with a factor loading of at least .500 were

retained. Because of this trimming, the regulation of emotions scale was underrepresented, with

only a few items. The few items that were on the prototype BDEFS that were probing about

regulation of emotion loaded on the Self-Restraint/Inhibition factor. Therefore, additional items

were added using a model of self-regulation developed by Gross (1998). The version of the

scale on which the national norms are based had 100 items (with the above mentioned added in).

The final factor analysis, using the entire norming sample of 1249 adults (to be described

in a following section), was then conducted on the 100-items (Barkley, 2011b). The principal-

41

component factor analysis (PCFA), after rotation yielded the following factors with their percent

of the variance they accounted for:

Self-Organization to Time (Factor 1) - 13.9%

Self-Management to Time (Factor 2) - 12.0%

Self-Regulation of Emotion (Factor 3) - 10.2%

Self-Restraint (Factor 4) - 9.0%

Self-Motivation (Factor 5) - 8.1%

These five factors noted here are the final five factors in the published BDEFS that was

used in this study. However, of the 100 items used in the norming sample, only 89 were retained

on the final version of the BDEFS.

Norming procedures of the BDEFS. From the data collected from the normative

sample, the BDEFS factors were evaluated based on demographic characteristics of the sample.

Barkley (2011) enlisted a national survey company, Knowledge Networks®, to collect the survey

data used in the norming sample of the BDEFS. As noted above, the 100 item version was used

and then paired down based on the CFA. The sample was a Web-enabled Knowledge Panel®,

which is a method using a probability-based panel so that the sample is representative of the

population of the United States. A random sample of telephone numbers and residential

addressed was used. Then members of households were randomly sampled (Barkley, 2011b).

BDEFS-other surveys in the norming sample were not collected (Barkley, 2011b). An

exclusionary criterion for mental disorders was not employed. Therefore, results are based on a

true random sample of functioning in the adult population. A total of 1249 adults completed the

BDEFS. Participants spanned ages 19-81 and were equally distributed across the lifespan and

42

gender. Additionally, race/ethnicity, geographic location, education, income, marital status, and

employment status were all proportionate to the general population (Barkley, 2011b).

When looking at the relationship among the demographic factors and the BDEFS, gender

and age significantly correlated with some factors of the BDEFS. Age did not significantly

correlate with Self-Organization/Problem Solving; however, it did significantly correlate to a

small degree with the other factors and the total score. A possible reason for the differences

among age categories is that the prefrontal cortex is thought to not reach maximum potential

until the late twenties or early thirties (Barkley, 2011b). Additionally, there is thought to be a

decline in later adulthood, and again this is likely a reason the college student population may

show differences. When looking at the variable gender, women were significantly more likely to

show higher problems with Self-Regulation of Emotions. There was a marginal, and not a

statistically significant (p=.055) relationship between gender and Self-Restraint, with men

showing more problems than women. Gender was not significantly correlated with any other

factor or total score. Barkley (2011b) then looked to see if there was an interaction of gender by

age and there were no significant interactions.

Creation of the ADHD-EF Index. The ADHD-EF Index scale was developed given

previous research linking ADHD and EF deficits (Barkley, 2011b). The original data base of the

prototype BDEFS (UMASS study) was analyzed to see which items were most likely to identify

adult ADHD. A binary logistic regression of the group with ADHD and the community control

group yielded just five items needed to distinguish the groups with 98.1% accuracy (remember

from previously that the UMASS sample consisted of an ADHD group, a community group, and

a clinical control group). These five items correctly predicted that an individual in the ADHD

group was in fact ADHD 99.1% of the time and correctly predicted that an individual did not

43

have ADHD 96.9% of the time. The five items identified were: procrastinates or puts things off

until the last minute (item 1), trouble completing one activity before starting into another one

(item 16), trouble organizing my thoughts (item 24), difficulty changing behavior when I am

given feedback about my mistakes (item 50), and take short-cuts in my work and do not do all

that I am supposed to do (item 65). The same analysis was then used to distinguish between the

ADHD group and the clinical control group. Again, the clinical control group consisted of a

group of participants who were referred for an ADHD evaluation (believed they had ADHD), but

did not receive a diagnosis (based on the DSM-IV; American Psychiatric Association, 2000) of

ADHD. In this analysis, seven items were needed to best differentiate the groups, with an

accuracy rate of 72.3%. These seven items correctly identified an individual as ADHD 56% of

the time and correctly identified the individual in the clinical control group 82.9% of the time.

These seven items were: trouble planning ahead or preparing for upcoming events (item 6),

trouble motivating myself to work (item 13), difficulty stopping my activities or behaviors when

I am given feedback about my mistakes (item 49), difficulty changing my behaviors when I am

given feedback about my mistakes (item 50), not aware of things I say or do (item 55), more

likely to drive a motor vehicle much faster than others (item 60), and depends on other to get my

work done (item 69). This indicated that it was more difficult to distinguish the clinical control

group from the ADHD group with rating scales alone (accuracy of 72.3%) than it was to

distinguish the ADHD group from the community control group (accuracy of 98.1%). Of the

seven items needed to identify between the ADHD group and the clinical control group, only one

was a duplicate (difficulty changing my behavior once given feedback, item 50) of the five items

needed to distinguish between the ADHD group and the community control group. The five

items found to distinguish the ADHD group from the clinical community controls was added to

44

the seven items used to distinguish the ADHD group from the clinical controls (less one

overlapping item) for a total of 11 items. These items constituted the ADHD-EF Index (Barkley,

2011b). The 11-item ADHD-EF index has not been replicated on multiple age categories or

severity levels of ADHD, or in the college student population. However, the 11-item ADHD-EF

index was subsequently tested on the norming sample (see previous section for description of the

normative sample). The participants in the norming sample were given an ADHD symptom

checklist along with the BDEFS. The participants in the top fifth percentile of ADHD symptoms

were classified as the ADHD group. The ADHD-EF Index was accurate in predicting this group

94% of the time (Barkley, 2011b).

Reliability

Inter-observer agreement and disparity. In the UMASS study, self-report and other-

informant report data were collected (Barkley, 2011b). The correlations between self- and other-

informant reports had a reasonable level of agreement with r between .66 and .79 across the four

factors (this was the prototype BDEFS without the Self-Regulation of Emotions scale). Other-

informant reports were not collected in the normative data; therefore, the agreement between the

two remains to be evaluated with the final five-factor version of the BDEFS. In the UMASS

study, the community control group (the group most similar to the norming sample) had

relatively little discrepancy between the self-report and the other-report. However, there was a

larger than expected standard deviation around the mean. For the clinical control and ADHD

groups, the disparities between the BDEFS and BDEFS-other were significantly higher, but they

did not differ between the two groups (ADHD versus clinical control). The absolute values of

the disparities between the BDEFS-self and BDEFS-other were taken in this case, so the

45

direction of the disparity was not taken into account. Additionally, age and IQ did not affect the

disparity between the BDEFS-self and BDEFS-other (Barkley, 2011b).

Internal consistency and test-retest. A Cronbach’s alpha was conducted to analyze

internal consistency of the BDEFS and it was found to be .918 for the Total EF Symptoms. The

Cronbach’s alpha for the five factors ranged from .914 to .958. The ADHD-EF Index was .842.

A test/retest comparison was completed on 62 randomly selected participants and was adequate

at .62 to .90 for the five factors and .70 for the Total EF Symptoms Score. Both of these

analyses were conducted on the full BDEFS from the norming sample (Barkley, 2011b).

Validity

Discriminant Validity. Data from the UMASS (prototype BDEFS) study showed that

80-98% of adults with ADHD were in the clinical range (above the 93rd percentile, 1.5 SD above

the mean) across the various factors of the BDEFS, versus only 8-11% across the various factors

in the community control group (Barkley, 2011b), using the self-report data. At first glance, this

appears to have excellent discriminant validity in distinguishing between adults with ADHD and

the control group. However, the clinical control group (those who believed they had ADHD, but

were not diagnosed), were in the clinically significant range at a rate of 83-98%. This could be

the result of the self-report aspect of this measure. At least 45% of the adults in the clinical

control group endorsed enough symptoms of ADHD to meet diagnostic criteria, if based solely

on self-report information. Therefore, if they are likely to endorse a high number of symptoms

on the ADHD criteria scale, they are equally as likely to do so on the BDEFS. In addition,

people who are willing to seek help (e. g., those who self-refer to a clinic for evaluation) could

present differently than those who do not seek help for a variety of reasons. The issue of

secondary gain or malingering could be at play here in over-endorsing symptoms. This analysis

46

should be replicated on a population who is already diagnosed with ADHD and does not have the

potential of a secondary gain. Additionally, looking at a clinical population who does not

believe they have ADHD may also be beneficial. The evidence given above does support that

the BDEFS is adequate at discriminating the normal population from the clinical population

(Barkley, 2011b); however, further evaluation is needed to distinguish between the clinical

populations. Additionally, these were all conducted with the prototype BDEFS, rather than the

final five-factor BDEFS.

ADHD-EF Index Validity. Again, in looking at results from the UMASS study, the

ADHD-EF Index was a good predictor of adults with ADHD. In fact, 98.5% of the group with

ADHD had a score above the 93rd percentile cutoff on the ADHD-EF index. However, the

clinical control group was also high with 96.6% above the same cutoff. Only 7.6% of the

community control group was above the 93rd percentile Barkley, 2011b). Given that the ADHD-

EF index was created based on its ability to discriminate those with an ADHD diagnosis, it is

expected that it would show good predictability (Barkley, 2011b).

At this time, no studies have been conducted using the BDEFS to discriminate between

control groups and other clinical groups that may have EF deficits, such as neurological disorder

and traumatic brain injuries, and there is a need for further research to be conducted.

Additionally, there have been no studies on the college student population, and as reported, the

UMASS study used the prototype BDEFS, rather than the finalized BDEFS.

Criterion Validity. Severity of ADHD symptoms is one of the most researched areas

with the BDEFS (Barkley, 2011b). As was mentioned, the correlation was significant for the

Total Score on the BDEFS with ADHD symptom criteria. Total ADHD symptoms were

measured by self-report on the Barkley Adult Rating Scales for ADHD, which is based on DSM

47

criteria (Barkley, 2011b). The ADHD symptoms in the above mentioned analysis were the total

symptoms (combining inattentive, hyperactive, and impulsive symptoms); however, further

analysis was conducted on inattentive symptoms and hyperactive/impulsive symptoms

separately. When looking at the symptoms for each subtype separately, there were also high

correlations with the BDEFS items. For inattentive symptoms of ADHD, the correlations ranged

from .80 to .92 across the five factors and the ADHD-EF Index of the BDEFS. For

hyperactive/impulsive symptoms of ADHD, the correlations were slightly less at .68 to .71

(Barkley, 2011b) across the five factors and the ADHD-EF Index. These statistics were analyzed

using both the UMASS sample and the Milwaukee sample, but they have not been replicated by

someone other than the author of the rating scale.

Construct Validity. A factor analysis was conducted using the UMASS (prototype

BDEFS) sample to see if ADHD symptoms and BDEFS symptoms were measuring the same

construct (Barkley, 2011b). As was discussed in the literature review of EF, Barkley’s Extended

Phenotype Theory proposed that ADHD and EF deficiencies were the same thing (Barkley,

2012). If this is the case, the factor analysis should show one construct when using ADHD

symptoms and the BDEFS items. The analysis showed a high factor loading onto one construct,

adding support for Barkley’s theory that ADHD and EF were just different names for the same

construct. The author then replicated this factor analysis with the normative sample (the

participants also completed the Barkley Adult Rating Scales for ADHD) and the analysis was

identical (Barkley, 2011b). These analyses should be replicated by a researcher other than the

author to continue to show support for this theory.

Limitations of Rating Scales. To summarize, there are a plethora of tools, tests, and

rating scales that purport to measure EF. Many of these EF tests and behavior rating scales have

48

evidence supporting validity and reliability of their results. The multiple and opposing models of

EF have also led to significantly different approaches to measuring EF. This section reviewed

common EF tests and behavioral rating scales commonly used to assess EF, along with the

benefits and consequences associated with each type. Additional research into the current

measures that are available with tighter control of the methodologies is needed. Finally,

behavior rating scales, specifically the BDEFS, are relatively new in the field and require

substantial research to collect evidence of validity and reliability. In summary, it appears that the

administration of both EF tests and behavior rating scales may add incrementally to our

understanding of the EF of an individual. The EF tests are less influenced by the patient’s level

of insight or potential for secondary gain, yet they are not tied to a specific diagnosis. Behavior

ratings scales, on the other hand, have greater diagnostic sensitivity in higher functioning

individuals. However, they may be vulnerable due to their higher face validity (allowing for

intended or unintended manipulation) and the necessity for patient insight.

Of the available behavior rating scales to measure EF, the BDEFS has recently been

published. To date, there have been no independent research studies published to provide

additional evidence of validity and reliability. To add to the body of research on the BDEFS,

there is a need to evaluate the types of test constriction and validation principles which are

required to provide the psychometric properties of a new test.

Test Construction and Validation Principles

This section will describe important considerations in attempting a psychometric

evaluation of a test, specifically as it relates to the BDEFS.

49

Reliability

The reliability of a measure refers to the degree to which the score produced by the

measure is reproducible (Reis and Judd, 2000). There are four classical primary types of

reliability: inter-rater reliability, split-half reliability, internal consistency, (Carmines & Zeller,

1979; Heiman, 2002), and alternate format (Carmibes & Zeller, 1979). Reliability is important

in multiple ways when constructing and utilizing measurements. Given self-reporting of

executive functioning symptoms, inter-rater reliability is an important factor in determining the

reliability of the measure. As has been suggested, individuals with deficits in EF may lack self-

awareness about their symptoms. Since the BDEFS is a self-report measure of these symptoms,

inter-rater reliability gives the interpreter a way to compare self-reported symptoms to other-

informant reported symptoms. When a measurement tool is reliable, it generally means that the

error of the measurements is reduced (Goodwin, 2010). Additionally, reliability is correlated

with improved validity (Fink & Litwin, 1995). After determining support for reliability of a

measure, determining validity becomes a priority.

Validity

Once a measure shows evidence of reliability, it requires support for validity. Does the

measurement tool actually measure what it is intended or purported to measure? To quote

Messick (1989), the question of validity can be summarized as “to what degree-if at all- on the

basis of evidence and rationales, should the test scores be interpreted and used in the manner

proposed?” (p. 5). Given the importance placed on results of measurement tools in academia and

in clinical practice, using a measure that has support of validity is critical. Broadly speaking,

validity speaks to the empirical and theoretical basis for the interpretation of test scores

(Messick, 1989, p. 6). Beyond this, Messick suggests that all methods of validation are

50

variations on the scientific method. This extends to inferences (comparable to hypotheses) and

validation (comparable to hypothesis testing). These inferences regarding the scores are only

meaningful in the context of theory, which is again consistent with the way in which information

is interpreted and conclusions are drawn within the scientific method (Messick, 1989). Once

there is evidence to support the assertion that the measure appears to be assessing what it is

intended to, inferences can be made regarding its use (Fink & Litwin, 1995). There are many

different types of validity to add support to the idea of whether a measure is actually measuring

what it is purported to measure. The four primary types of basic validity are face validity,

criterion validity, content validity, and construct validity.

Face validity merely means that the test appears to be valid based on face-value (Fink &

Litwin, 1995; Heiman, 2002). Criterion validity is the degree to which a measure correlates with

a behavior that the individual is either currently presenting (concurrent) or presents in the future

(predictive) (Jackson, 2008; Heiman, 2002). This usually includes an empirical correlation

between an observable (or measureable) behavior and the measurement tool being validated

(Messick, 1989). Content validity refers to the extent to which the items produce responses that

measure or represent that of the construct or concept (Akien, 2003). This is usually

accomplished by an expert judgment based on opinion, rather than empirical evidence. None of

the above types of validity are concerned with differences in responding across setting or groups,

internal or external structure, or the social consequences of using such a measure (Messick,

1989).

Construct validity, however, is evidence to support that the measure tests the actual

construct that it intends to measure. This is important given that clinical impressions (social

consequence as Messick puts it) are often made from measures, such as the BDEFS. Within

51

construct validity, there are two types: convergent and discriminant validity. Convergent

validity refers to the correlation of the scores in question to another procedure which is already

accepted as valid measuring the same construct. Discriminant validity refers to extent to which

the scores obtained are not correlated with those of an accepted procedure that measures other

variables (Heiman, 2002). In construct validity, the score resulting from the measure is one

piece of the many different indicators that represent the construct (Messick, 1989). It is

important to remember when conceptualizing construct validity that the relationships between

criterion and content validity as well as the empirical evidence from construct validity should

have a theoretical commonality (Gulliksen, 1950). These things together, combining the

scientific underpinnings and the ethical nature of social consequences, form the basis for modern

test theory (AERA, APA, & NCME, 1999; Messick, 1989).

Construct validity has two main threats: construct underrepresentation and construct-

irrelevant variance. If the items on the measurement tool are too narrow and do not cover the

depth of the construct, underrepresentation occurs. On the other hand, if items are harder or

easier for one group than another on a variable that has nothing to do with the construct, then

irrelevant variance is introduced (Messick, 1989). When a new measure is published (such as the

BDEFS) it is important that before serious clinical conclusions are drawn, evidence of validity is

provided (Fink & Litwin, 1995). This includes evidence to support construct validity in multiple

populations (Messick, 1989; Heiman, 2002; Fink & Litwin, 1995), as well as from independent

researchers.

Discriminant Function Analysis

One way to determine whether a measure discriminates between two or more groups is a

discriminant function analysis. This is used when the goal is to analyze a relationship between a

52

dichotomous dependent variable and either a continuous or dichotomous independent variable.

The discriminant analysis endeavors to use the independent variable(s) to discriminate or

distinguish among the dependent variable groups (Myers, Gamst, & Guarino, 2006). In this

case, ADHD or non-ADHD. Another analysis that is similar to the discriminant function

analysis is logistic regression. Both of these techniques predict group membership; however, in

a logistic regression, the independent variable can be either continuous or dichotomous.

Additionally, a logistic regression is non-linear, and a discriminant function analysis is linear

(Myers et al., 2006).

An important feature of a discriminant function analysis is that it yields an accuracy rate,

which indicates how useful the tool is in determining group membership. In doing so, this

analysis helps to establish the boundary of the groups (Myers et al., 2006). To accomplish the

accuracy rate, eigenvalues are used, as they are in confirmatory factor analysis (Myers et al.,

2006).

Confirmatory Factor Analysis

In attempting to provide evidence of construct validity of a new measure, factor analysis

is a popular statistical procedure. It provides support for the factor structure of a measure when

attempting to provide valid information on latent factors. A latent factor is a construct that

describes a set of symptoms or behaviors measured by the measurement tool (e. g. self-

motivation to time) (Brown & Cutik, 1993; Brown, 2006). There are two basic types of factor

analyses: exploratory factor analysis (EFA) and confirmatory factor analysis (CFA).

Traditionally, an EFA is used in the preliminary analysis of a new measure. It is exploratory in

that it uses a data driven approach to determine the number of factors and the nature of those

factors (Brown, 2006). Once empirical and theoretical frameworks have been established, a

53

CFA is then the statistic of choice. A CFA uses a priori decisions based on the empirical

evidence (from an EFA) and theory to determine the number and nature of the parameters. The

CFA is an essential statistical procedure for providing evidence for construct validity. It

produces persuasive evidence for convergent and discriminant validity of the theoretical

concepts of the latent factor or construct (Brown, 2006).

There are other statistical procedures to examine the relationship among variables and

factors, such as multiple regression and correlations. However, the CFA has an advantage over

these procedures as it is the only one to take into account error variance. The CFA’s primary

intent is to determine the nature and number of latent variables, or factors (Floyd & Widaman,

1995). CFA is a structure equation model (SEM) which deals specifically with measurement

models and the relationships between observed measures or indicators and the latent variables

they propose to represent (Brown 2006). There are multiple ways to extract the factors when

conducting a CFA. These include “maximum likelihood, principal factors, , imaging analysis,

unweighted least squares, generalized least squares, minimum residual analysis, weighted least

squares, and alpha factor” (Brown, 2006. P. 21), to name a few. The maximum likelihood is the

most common because it has the ability to reproduce the factors and provides a goodness of fit

model (Brown, 2006).

Proposed Study and Research Questions

As was laid out in the literature review, college students with ADHD differ from their

adult counterparts in both the demands placed on their daily activities (more heavily EF based in

general), and their different characterological attributes. While determining whether a college

student has a diagnosis of ADHD is complex, it is also an important factor given the rate of

academic difficulties and drop-out in this population. In addition, the cost of misdiagnosis is

54

great. Deficits in EF have been related to the diagnosis of ADHD in the literature, both on the

biological level and theoretical level. fMRI studies have shown the relationship between ADHD

and EF deficits time and again. There are a range of EF tests that purport to measure EF;

however, this method of measurement is flawed at best. Given this lack of appropriate

measurement, behavior rating scales have come to the forefront in the quest in assessing EF. The

BDEFS is a new EF behavior rating scale which has been published recently. It has yet to have

evidence of validity and reliability through independent sources, and has not been researched in

the college student population.

The current research study proposed to add to the support of construct validity, reliability,

and diagnostic usefulness of the BDEFS. The BDEFS purports to measure the latent traits of

executive functioning: self-management to time, self-organization/problem solving, self-

restraint, self-motivation, and self-regulation. The BDEFS also purports to have diagnostic

validity in determining the likelihood that an adult individual has ADHD. Given that this is a

new measure and little has been done in the way of support of validation, this study set out to

contribute to the literature regarding the usefulness of this measure for the college student

population. The research questions that were proposed are as follows:

1. What is the relationship between the BDEFS self-report and other-informant report in a

college student population of students with ADHD? What are these relationships on the

following factors: Self-Management to Time, Self-Organization and Problem Solving,

Self-Restraint, Self-Motivation, Self-Regulation, ADHD-EF Index, and Total Executive

Functioning Symptoms? How do these correlations compare to the correlations that

Barkley found in his original study? Are the means of the self-informant reports higher or

55

lower than the means of the other-informant reports within the college student population

of students with ADHD?

2. In the college student ADHD sample, is there a correlation between intellectual functioning

(as measured by the BIA) and the BDEFS similar to the correlation between the intellectual

functioning and BDEFS in the UMASS study. This relationship was analyzed for the

following factors: Self-Management to Time, Self-Organization and Problem Solving, Self-

Restraint, Self-Motivation, Self-Regulation, and Total Executive Functioning Symptoms.

3. Are the same ADHD-EF Index items the most predictive of a diagnosis of ADHD in a

college student population as they are in the original normative sample? The current

BDEFS ADHD-EF Index is composed of 11 items. This index was created using a

discriminant analysis to select those items (out of 89) that best discriminate those with

ADHD from a normative sample. Do the same 11 items best discriminate those with

ADHD from a normative sample in a college student population?

4. Is the factor structure of the BDEFS the same in college students as it is for the normative

sample, based on a confirmatory factor analysis?

56

CHAPTER 3

METHODS

Introduction

In the literature review, the difficulties in diagnosing college students with ADHD, the

difficulties with deficits in EF, and the importance of test validation were laid out. This chapter

describes the Barkley Deficits in Executive Functioning Scale (BDEFS) and the development

process of this scale in more detail. Additionally, the methods used to recruit participants and

the analyses used are discussed below.

Participants

The 596 participants were all college students. These students were either recruited to

participate in the control group (n=459) or were college students ADHD (n=137). The

participants attended a large, highly selective southeastern university in the United States. The

demographic breakdown of the groups with ADHD and without ADHD is located in Table 1.

The gender breakdown in this sample was consistent with the published statistics of the

university where the data were collected. All students with ADHD in this sample were either

inattentive type or combined type ADHD. There were no participants in this sample with

hyperactive/impulsive type. The absence of students with the hyperactive/impulsive subtype is

consistent with the literature on adults and college students with ADHD (Heiligenstein et al.,

1999).

Participants in the ADHD group were part of an archival data set, who were recruited

through a campus clinic where they had been referred for an evaluation to determine whether they

had a diagnosis of ADHD. These participants were clients who, through the course of their

evaluation, completed the BDEFS survey while they were students. The students who were given a

57

diagnosis of ADHD were coded in the ADHD group. Participants who had co-occurring disorders

are included in the ADHD group as ADHD is highly comorbid with many other DSM disorders

(APA, 2013). The students who did not receive a diagnosis were excluded from the sample for

most analyses, with the exception of the confirmatory factor analysis. For the ADHD group, only

students who had previously given consent to participate in research were utilized (HSC approval

2012.7742). Please see appendix A for informed consent and for IRB approval.

Table 1 Demographics

Demographic Descriptors

Total Sample n=596

M (SD)

ADHD Group n=137

M (SD)

Non-ADHD Group n=459

M (SD) Age 20.71 (2.46) 21.5 (3.12) 20.53 (2.26)

Female 63.4 52.8 65.8

Male 36.6 47.2 34.2

Freshman 17.6 19.4 17.2

Sophomore 28 29.6 27.7

Junior 29 27.8 30.3

Senior 24.5 23.1 24.8

Caucasian 66.6 63.7 67.1

African American 16.5 14.7 16.8

Asian 3.6 2 3.9

Hispanic 10.4 15.7 9.2

Other 3.1 3.9 2.9

Note: the numbers represented are percentages for each sample (gender, year in college, and

ethnicity). The numbers for the variable age, are represented in years

Diagnosis of ADHD

As part of the diagnosis of ADHD in the preceding archival data set, the clients completed the

following forms created by Barkley (2011a): Employment History, Developmental History, Social

58

History, Work Performance Rating Scale, Health History, Current (ADHD) Symptoms Scale (self-

report form and other-informant form), Childhood Symptoms Scale (self-report form and other-

informant form), Driving Behavior Survey, and the Driving History Survey. Additionally, the clients

completed a pencil and paper copy of the BDEFS (Barkley, 2011b). The clients also completed the

Academic Success Inventory for College Student (ASICS; Prevatt, Wells, Festa-Dreher, Yelland, &

Lee, 2011) and a checklist of symptoms based on the DSM to rule-out other mental health diagnoses.

Additionally, other-informant reports were collected using the BDEFS-other. Clients were instructed

to select someone to be their other-informant who knew them well and interacted with them on a

regular basis. Additionally, the other-informant should have known the client for a minimum of six

months. Generally speaking, the other-informant was a significant other, best-friend, roommate,

sibling, or parent. In most cases, the other-informant report form was emailed directly to the

informant through an online survey. The results were then sent directly back to the clinic through the

survey management software. This was done to improve the likelihood that the informant would be

forthcoming and not concerned about the participant’s feelings when rating the impairment. A

clinical interview was conducted with the client to provide personal anecdotes corroborating their self-

reported symptoms on the rating scales. As far as psychometric testing was concerned, the client was

administered three subtests from the Woodcock-Johnson III Tests of Cognitive Abilities (WJ-III

COG; Woodcock, McGrew, & Mather, 2001b). This provided an estimate of the client’s cognitive

processing abilities. The subtests were: Verbal Comprehension, Concept Formation, and Visual

Matching. These three subtests taken together form the Brief Intellectual Ability (BIA) cluster.

According to Schrank, Mather and Woodcock (2011), the reliability coefficients for the BIA

ranged from .94 to .98. Concurrent validity with other measures of intelligence (full IQ

measurements) was reported to range from .60-.69 (McGrew & Woodcock, 2011), which falls in

59

the moderate to acceptable range (Heiman, 2002). In addition, three subtests from the Woodcock-

Johnson III Tests of Achievement (WJ-III ACH; Woodcock, McGrew, & Mather, 2011a) were

administered to evaluate academic achievement. These subtests included Understanding Directions,

Passage Comprehension, and Reading Fluency. These brief measures of cognitive abilities and

academic achievement has an age range from 2 to 90+ years old. The achievement testing data

was not utilized in this study.

A diagnosis of ADHD was made if the client met the criteria set out by Barkley (2011a) and

the DSM-IV-TR: (a) there was evidence that the client experienced symptoms of ADHD in early

childhood, (b), these symptoms appeared no later than middle school and impaired their

functioning across multiple settings, (c) there was evidence that the client was currently

experiencing symptoms of ADHD, which cause marked and chronic impairment across settings,

and (d) there were no explanations other than ADHD that better accounted for the client’s current

symptoms. The process of determining whether a student met the above ADHD criteria was a

clinical process which took into account not only checklists, but a lengthy diagnostic interview

gathering information from multiple aspects of the student’s life. No client was included into this

data set where the diagnosis was made with the newest addition of the DSM (DSM-5). However,

there were approximately 20 students who did not receive a diagnosis of ADHD that were collected

during this time frame of this data collection, and they are not included in this sample. Some of

these students may have been eligible for a diagnosis of ADHD under the new criteria in the DSM-

5.

The evaluators were graduate-level students working towards a master’s degree in School

Psychology or a doctoral degree in Combined Counseling Psychology and School Psychology. All

cases were supervised by both an EdS level School Psychologist and a doctoral level Clinical

60

Psychologist. Agreement between the evaluator and the two supervisors was required for a

diagnosis of ADHD.

The control group was a combination of archival data and newly collected data from the

same university as the group with ADHD. The Educational Psychology and Learning Systems

Department at the university maintains a subject pool to aid in recruiting participants. However,

this subject pool was mostly female; therefore, additional recruitment procedures were taken to

increase the size of the sample and to balance the gender disparity. This was done by recruiting

from other places on campus such as the library, student union, and club-sports. In addition, several

classes were selected based on instructor approval to be offered the opportunity to participate.

Classes for recruitment included: General Biological Sciences for non-majors (BSC 1005),

Principles of Macroeconomics (ECO 2012), Principles of Microeconomics (ECO 2013), Race and

Ethnicity in the US (AHM 2097), and Dynamic Earth (GLY 1000). Completion of the BDEFS

survey (Appendix B) was voluntary for all students and the informed consent can be found in

Appendix C. Human Subjects Approval (HSC No. 2013.10087) is in Appendix D. The survey

also requested demographic information from the participant such as gender, ethnicity, and year in

college (see appendix E).

For the participants in the control group, the data was collected via an online survey or by a

paper and pencil copy. The BDEFS and the demographic questions were converted from the

original paper-format to an online-format to make the completion easier for students; however, a

paper copy was also available for participant choice. Participants were either provided extra credit

in their classes, fulfilled a class research requirement, or entered into a lottery for a $15 gift

certificate to the store of choice. Additionally, participants had the option to choose a gift card for a

free coffee drink from the on-campus coffee shop for completing the survey. The type of incentive

61

depended on the instructor preference for their students and the setting in which the recruitment

occurred. The participants provided their names and contact information in a separate survey that

was hyperlinked to the original survey if they selected the lottery option. This ensured

confidentiality for participant names and contact information. The personal information was kept in

a separate data file and could not be matched up with their responses to the original survey after the

initial search of duplicate names.

The survey tool prompted students to answer each question; therefore, in the control

sample, there was no missing data. In the archival data set, the evaluator was to check all forms to

ensure they were complete, and there were very few pieces of missing data. The missing data

points were replaced with the mean score for that variable.

All participants were enrolled in classes, at least part-time, and students under 18 years of

age and older than 30 years of age were excluded. These cut-offs were selected given the purported

changes in the development of the EF system at around age 30 (Barkley, 2012). Graduate students

were also excluded from this study given that their age range is generally older, and their general

characteristics may be fundamentally different from that of an undergraduate student. A search of

participant names was conducted to make sure there were no duplicates participants (a student who

visited the clinic and was in one of the control group collection areas).

Hypotheses and Planned Data Analyses

Preliminary Analysis

Archival data set- (ADHD group). The archival data set was coded into the Statistical

Package for the Social Sciences (SPSS), version 18 for Windows. The appropriate variables

were extracted from that data set and merged with the control participant data. A search for data

points that fell outside the appropriate range for that particular variable was conducted, as well as

62

a search for missing data. All missing or inaccurate data was corrected by pulling the participant

file and recoding the data if available. A listwise deletion process was used to handle missing

variables from a participant. This procedure is appropriate given that there was small amount of

missing data (Graham, 2009) because the data was being collected by the evaluators during the

evaluation session.

Given that a percentage of the college student population has ADHD, approximately five

percent of students recruited for the non-ADHD group reported a prior diagnosis of ADHD

(based on results from the demographic section). Students who reported a prior diagnosis were

only used in the confirmatory factor analysis and were not used in any other analyses.

Newly collected data (non-ADHD group). The BDEFS and demographic questions had

previously been collected from a sample of college students. This data was collected through an

on-line survey management tool and then was converted into an SPSS file though the survey

management system. For the surveys collected with paper and pencil, this researcher coded the

data into Excel. These data sets were then merged with the archival data set and again, a search

for missing and inaccurate data was conducted. The data set did not have out-of range data or

missing data given that the survey tool was set to disallow students to skip questions and the

answers were in multiple choice format rather than open-answer.

Planned Analyses

Research Question 1. What is the relationship between the BDEFS self-report and

other-informant report in a college student population of students with ADHD? What are these

relationships on the following factors: Self-Management to Time, Self-Organization and

Problem Solving, Self-Restraint, Self-Motivation, Self-Regulation, ADHD-EF Index, and Total

Executive Functioning Symptoms? [Both the original-11-item ADHD-EF-index and the new 15-

63

item-ADHD-EF index (see research question three below) were included in this analysis.] How

do these correlations compare to the correlations that Barkley found in his original study? Are

the means of the self-informant reports higher or lower than the means of the other-informant

reports within the college student population of students with ADHD?

In research question one, only the ADHD group was utilized. That group had 137

participants who had given informed consent to be included in this research study. To answer

the first part of this question, a Pearson product-moment correlation was utilized to determine the

relationship between the self-informant report (BDEFS-self) and the other-informant form

(BDEFS-other) of the five factors, total score, and both ADHD-EF Indexes. The hypothesis for

this part of the analysis was that the college student sample would yield adequate inter-rater

agreement, based on the fact that Barkley obtained significant correlations when he looked at

inter-rater agreement in his original analysis. In addition to the simple correlation, a Fisher r-to-z

transformation was conducted using Barkley’s (2011b) reported correlations and the correlations

found in this study to see if there was a statistically significant difference between the two

(Lowry, R., 2013). Finally, t-tests were conducted to see if the means for each factor separately

were statistically significantly different between the BDEFS-self and the BDEFS-other. An a

priori power analysis was conducted to determine a suitable sample size for this statistical test.

The G-Power 3.1.7 program (Faul, Erdfekder, Buchner and Lang, 2013) was utilized to conduct

the power analysis using two-independent pearson correlations, with a projected large effect size

of 0.66 and an alpha error probability of 0.01, and a power value of 0.8, An alpha level of .01

(.05 divided by 5 tests = .01) was used due to the need for a Bonferonni correction to correct for

family-wise error rate. There are five factors (the ADHD-EF Index and Total Score are

combinations of the same items). Barkley showed a large effect size between the self-form and

64

other-informant form when he ran the correlation on his norming sample (.66-.79); therefore, it is

presumed that a large effect size will be found in this analysis as well. The power analysis

indicated that 120 participants were needed. For the t-tests that were conducted to see if there

was a statistically significant difference in the means for each factor between the BDEFS-self

and the BDEFS-other, the p-value will be .01 rather than .05 as well given the same need as

above to use a Bonferroni correction.

Research Question 2. In the college student ADHD sample, is there a correlation

between intellectual functioning (as measured by the BIA) and the BDEFS similar to the

correlation between the intellectual functioning and BDEFS in the norming sample? This

relationship was analyzed for the following factors: Self-Management to Time, Self-Organization

and Problem Solving, Self-Restraint, Self-Motivation, Self-Regulation, and Total Executive

Functioning Symptoms.

To answer this question, a Pearson product-moment correlation was utilized to determine

the relationship between the BIA and the five factors and total score of the BDEFS. As with the

previous research question, only the ADHD group was utilized in this analysis.

The same a priori power analysis was conducted to determine a suitable sample size for

this statistical test as the previous research question.

Research Question 3. Are the same ADHD-EF Index questions the most predictive of a

diagnosis of ADHD in a college student population as they are in the original normative

sample? The current BDEFS ADHD-EF Index is composed of 11 items. This index was created

using a discriminant analysis to select those items (out of 89) that best discriminate those with

ADHD from a normative sample. Do the same 11 items best discriminate those with ADHD

from a normative sample in a college student population?

65

As discussed previously, the original 11-item ADHD-EF Index was developed by using

two separate analyses comparing different samples. One analysis compared an adult sample of

participants with ADHD to a clinic referred sample who did not qualify for a diagnosis of

ADHD. The items that were yielded from that analysis are labeled below as (CL). The next

analysis determined items that best discriminated the group of adults with ADHD from a

community sample. The items yielded in this analysis are labeled below as (COM). The original

11-item ADHD-EF Index is composed of the following items:

Procrastinate or put off doing things until the last minute (item 1; COM)

Have trouble planning ahead or preparing for upcoming events (item 6; CL)

Have difficulty motivating myself to stick with my work and get it done (item 14; CL)

Have trouble completing one activity before starting into a new one (item 16; COM)

I have trouble organizing my thoughts (item 24; COM)

Have difficulty stopping my activities or behavior when I should do so (item 49; CL)

Have difficulty changing my behavior when I am given feedback about my mistakes (item

50; COM; CL)

Not aware of things I say or do (item 55; CL)

More likely to drive a motor vehicle much faster than others (excessive speeding) (item 60;

CL)

Likely to take short cuts in my work and not do all that I am supposed to do (item 65; COM)

Have to depend on others to help me get my work done (item 69; CL)

In Barkley’s normative sample, the 11 items above best discriminated those adults who

had symptoms of ADHD from clinic-referred adults without ADHD and from a community

66

sample without ADHD. The question at hand is whether, in a college sample, the same 11 items

best discriminate those with ADHD from those without ADHD.

The Barkley analysis comparing the ADHD group with the community control group

(COM) is the most closely matched to the sample in this study; therefore, these items (1, 16, 24,

50, and 65) were analyzed separately to see how well they match the analysis conducted by

Barkley and will be referred to as the 5-item community control ADHD-EF Index. In summary,

this analysis was conducted twice, once using all 11 items and once using only the 5 items from

the community control norming group.

The current analyses used discriminant function analysis. Logistic regression and

discriminant function analysis are very similar in that they predict the likelihood of a

dichotomous group membership (ADHD group or Control group). In a logistic regression, the

predictor variables (the items in this case) are either continuous or dichotomous; however, in a

discriminant function analysis, they are always continuous (Meyers, Gamst, & Guarino, 2006).

Additionally, if the model is non-linear, then a logistic regression should be used. However,

results indicated that the model was in fact linear. Therefore, the more appropriate statistical

analysis is the discriminant function analysis (Meyers, Gamst, & Guarino, 2006).

The discriminant function analysis is a type of general linear analysis; therefore, similar

assumptions must be met. These include normality, linearity, non-multicollinearity, independent

predictors, and homoscedasticity (Meyers, Gamst, & Guarino, 2006). While the assumptions are

relevant for a DFA, the DFA makes fewer statistical demands than does the MANOVA. In the

case of a MANVOA, inferences are being made. However, if one achieves high classification

rates in a DFA, the shape of the distributions is less important (Tabachnick and Fidell (2007).

67

In addition to group identification, sensitivity and specificity were analyzed through the

discriminant function analysis. Sensitivity denotes how well the identifiers accurately detect

whether a participant belongs in a group. In contrast, specificity indicates the accuracy of the

cutoff score in excluding participants who do not belong in the group (Hood & Johnson, 2006).

This is expressed as a percentage of accuracy for each.

There is discrepancy in the literature about the appropriate sample size for a discriminant

function analysis. According to Meyers, Gamst, and Guarino (2006) the minimum requirement

to run this statistical procedure is: the maximum number of independent variables = N-2, where

n represents the sample size. Therefore, since this analysis has 89 independent variables, the

minimum sample size is 91 cases. Meyers et al (2006) also states that although this is the

minimum number needed in the smallest group (ADHD group in this case), it is not

recommended. The most common recommendation for determining sample size comes from

Tabachnick and Fidell (2007). They also confirm that the smallest group should exceed the

number of predictor variables (in this case 89); however, they state that a power analysis should

be conducted similarly to a MANOVA, given that the discriminant function analysis is

essentially a reverse MANOVA. Given this, a G-Power analysis was conducted for this analysis

with a Test of MANOVA, Global Effects was used given the discussion earlier from Tabachnick

and Fidell (2007) stating that the sample size for a Discriminant Function Analysis should follow

that of a simple MANOVA. Using a projected effect size of 0.25 and an alpha error probability

of 0.05, a Power value of 0.8, G-Power indicates that 218 participants are needed. In this

analysis, 596 participants were collected, surpassing the total indicated by the power analysis.

While a significantly larger sample size was used in this analysis, the two groups (ADHD or

Control) did not have an equal n. Therefore, since there were a greater number of control

68

participants, the probability of the model predicting someone to be in the control group was

almost four times as likely. Given this, SPSS has a feature compute from group sizes. This uses

the observed group sizes in the sample to define the prior probabilities of group membership,

statistically controlling for this uneven sample.

The discriminant function (the linear equation in the discriminant analysis) attempts to

maximally differentiate the two groups (ADHD and non-ADHD) on the independent variables

(test items). The prediction score is calculated based on the prediction weight (similar to a beta

weight in multiple regression) (Meyers, Gamst, & Guarino, 2006). The function is represented

as:

Di = a + bi Xi + b2X2 +…+ bn-Xn

The X’s are the predictor variables (items) and the b’s are the beta weights. The discriminant

function analysis uses the best fit method of maximum likelihood, rather than a least squares

solution as in a regression. This is an iterative process that determines the best fit (Meyers,

Gamst, & Guarino, 2006).

To interpret the outcomes, the structure matrix must be analyzed. The discriminant

loading (correlations between the variables and the discriminant function) should be at least .40

(Meyers, Gamst, & Guarino, 2006) to be considered a variable that discriminates between the

two groups. This will also include sensitivity and specificity. The analysis looks at which items

in general were the best predictors of ADHD in the college student population. The study also

analyzed how the new model found above predicts group membership compared to the original

11-item ADHD-EF Index.

Research Question 4. Is the factor structure of the BDEFS the same for college students

as it is for the normative sample, based on a confirmatory factor analysis?

69

The final research question addresses whether the factor structure of the BDEFS is the

same in college students as it was for Barkley’s normative sample. A confirmatory factor

analysis is the most suitable statistical procedure to answer this research question. As discussed

in the literature review, there are many different types of factor analyses, and the two primary

categories are exploratory factor analysis (EFA) and confirmatory factor analysis (CFA) Brown,

2006; Meyers, Gamst, & Guarino, 2006). Because Barkley (2011b) has already conducted an

EFA and CFA on the multiple samples of adults, there is no need to conduct another EFA.

Moreover, there is theoretical and empirical support for the proposed factor structure. The CFA

is useful to determine whether a different group (college students) has the same factor structure

as the original group from the normalization sample.

To evaluate the proposed research question, the statistical software M-Plus 7.0 was

utilized. The data was converted to an MPlus file. There are five steps in evaluating a structure

model: “(a) model specification, (b) model identification, (c) model estimation, (d) model

evaluation, and (e) model respecification” (Meyers, Gamst, & Guarino, 2006, p. 549).

The sample size needed to conduct a CFA is difficult to determine. Experts in the field

do not necessarily agree on a standardized practice. According to Brown (2006), “Many rules of

thumb have been offered, including minimum sample size (e. g., N ≥ 100 to 200), minimum

number of cases per each freed parameter (e. g., at least 5 to 10 cases per parameter), and

minimum number of cases per indicator in the model (cf. Bentler & Chou, 1987; Boomsma,

1983; Ding, Velicer, & Harlow, 1995; Tanaka, 1987). Such guidelines are limited by their poor

generalizability to any given research data set. That is, the models and assumptions used in

Monte Carlo studies to provide sample sized guidelines are often dissimilar to the types of

models and data used by the applied researcher. Indeed, requisite sample size depends on a

70

variety of aspects such as the study design (e. g., categorical, continuous) and distribution of the

indicators, estimator type (e. g. ML, robust ML, WLSMV), the amount and patterns of missing

data, and the size of the model (model complexity).” P. 412-413. A more simplistic

recommendation was given by Tabachnick and Fidell (2007). They set their guide as “50 as very

poor, 100 as poor, 200 as fair, 300 as good, 500 as very good, and 1000 as excellent.” Therefore,

this researcher aimed for a sample of approximately 500-600 participants. While approximately

600 participants were collected, the sample size was again heavily skewed in favor of the control

group. The analysis was run twice, once with the entire sample and once with a sample in which

the control group was randomly reduced to have an equal number of participants in each group

(ADHD vs. Control). The random reduction was dine to more closely mimic Barkley’s first

CFA which was conducted on only the ADHD group, and Barkley’s second CFA conducted on

the norming sample. Given that there was a statistically significant difference between the

genders of the two groups, the female group was randomly reduced to equalize the two groups

on this variable. Results for both samples can be found in Table 13.

The most regularly used goodness-of-fit index is chi-squared (χ 2), which under the

classic maximum likelihood (ML) estimation model, is represented as: χ 2 = FML (N-1). In

MPlus, it is represented as χ 2 = FML (N). Even though χ 2 is the traditional model of ML, it is

rarely used on its own because it has some flaws (Brown, 2002). Primarily, it is susceptible to

distortions with high sample sizes and non-normally distributed samples. Given this, many

alternative fit indices have been developed to evaluate model fit and will be utilized in this study.

Table 2 lists various fit indices and their suggested values to indicate a good-fitting model.

Brown (2002) suggested that the research community is divided about the most appropriate

71

indices to utilize in CFA, and a bit of controversy surrounds it. He suggested three different

types of indices: absolute fit, parsimonious fit, and comparative fit.

Table 2 Suggested Ranges for Fit-Indices

Fit Indices Suggested ranges Reference X2 P≤ .05 Brown (2002)

X2/df < 2

Tabachnick & Fidell (2007)

CFI

< .90

Hu & Bentler (1999), Tabachnick & Fidell (2007)

RMSEA <.1

Not to exceed .06 <.05=good-fit, between .05-.08= reasonable-fit, <.08= poor-fit

Tabachnick & Fidell (2007) Taylor & Pastor (2007) Hu & Bentler (1999)

TLI < .95

< .90

Tabachnick & FIdell (2007) Hu & Bentler (1991)

SRMR ≤ .08 Tabachnick & Fidell (2007), Taylor &

Pastor (2007)

Absolute fit indices include χ 2, standardized root mean square residual (SRMR), root mean

square (RMR) (Brown, 2002), and goodness-of-fit (GFI) (Meyers, Gamst, & Guarino, 2006).

The analysis used in this study was the SRMR. Parsimonious indices are different from the

absolute fit indices because they have a consequence for poor model parsimony. The most

frequently used index of this type is root mean square error of approximation (REMSA) (Brown,

2002) and this is used in this analysis. In addition, the comparative fit index (CFI) and the

Tucker Lewis Index (TLI), also known as the non-normed fit index (NNFI) are used in this

study.

72

CHAPTER 4

RESULTS

Demographic Variables and Statistics

Demographic statistics were reviewed for all data collected and are represented

previously in Table 1. There were statistically significant differences between the ADHD

group and the Non-ADHD group for the variable Age (t = -3.73, p=.00). On average, participants

in the ADHD group were one year older than participants in the non-ADHD group. Chi-Square

tests were conducted to compare the two groups (ADHD verses non-ADHD) with regard to gender

(X2=6.44, p=.01), year in school (X2=.66, p=.88), and ethnicity (X2=4.96, p=.29), of which only

gender was statistically significant. The non-ADHD group had significantly more women than the

ADHD group.

Research Question 1

What is the relationship between the BDEFS self-report and other-informant report in a

college student population of students with ADHD? What are these relationships on the

following factors: Self-Management to Time, Self-Organization and Problem Solving, Self-

Restraint, Self-Motivation, Self-Regulation, ADHD-EF Index, and Total Executive Functioning

Symptoms? How do these correlations compare to the correlations that Barkley found in his

original study? Are the means of the self-informant reports higher or lower than the means of

the other-informant reports within the college student population of students with ADHD?

To evaluate the relationship between the self-report form (BDEFS) and the other-

informant report form (BDEFS-other), Pearson Correlations were conducted for the following

variables: the five factors, total score, and ADHD-EF Indexes. Results can be found in Table 3.

The correlations between BDEFS-self and BDEFS-other from Barkley’s analyses are listed in

73

Table 4 for comparison (Barkley, 2012b). A Fisher r to z transformation was conducted on four

of the factors to see if the results found in this study were significantly different from the results

found by Barkley. Recall that Barkley’s data did not have the Self-Regulation of Emotion, and

Barkley did not give inter-rater agreement correlation rates for the Total Score or the original 11-

item ADHD-EF Index, so these comparisons could not be made using the Fisher r to z

transformation. Results showed the following: Self-Management to Time z = -7.55, p =.00; Self-

Organization/Problem Solving z = -3.35, p=.00; Self-Restraint (z = -2.34, p =.019); and Self-

Motivation (z = -5.95, p = .00). These statistics indicate that all BDEFS-self/BDEFS-other

correlations in the current study are significantly different from the correlations reported by

Barkley. All of the comparable Barkley self/other correlations are higher than the current

self/other correlations.

Table 3 Inter-Rater Correlations for College Student Sample,

Comparing Self-Reports to Other-Reports

Factor r2 p

Self-Management to Time .22 .00 Self-Organization/Problem Solving .42 .00 Self-Restraint .39 .00 Self-Motivation .35 .00 Self-Regulation of Emotion .51 .00 Total Score .38 .00 Original 11-item ADHD-EF Index .80 .00 New 15-item ADHD-EF Index .24 .02

Table 4 Inter-Rater Correlations for Barkley’s Sample,

Comparing Self-Reports to Other-Reports

Factor r2 p

Self-Management to Time .79 < .00 Self-Organization/Problem Solving

.66 < .00

Self-Restraint .74 < .00 Self-Motivation .69 < .00

74

Following these analyses, a series of t-tests were conducted to compare the means of the

BDEFS-self and the BDEFS-other, in the current sample. These results were significant and can

be found in Table 5.

Table 5 Means, t-Tests, and p-Value for Self vs. Other-Informant Forms, in the

Current Sample

Means Self Other t p

Self-Management to Time 57.78 50.22 49.09 .00 Self-Organization/Problem Solving 60.59 50.03 46.06 .00 Self-Restraint 39.54 38.01 34.78 .00 Self-Motivation 26.46 22.44 35.53 .00 Self-Regulation of Emotion* 26.45 29.97 30.01 .00 Total Score 217.59 205.60 - - ADHD-EF Index 28.66 26.62 - -

Research Question 2

In the college student ADHD sample, is there a correlation between intellectual

functioning (as measured by the BIA) and the BDEFS similar to the correlation between the

intellectual functioning and BDEFS in the norming sample?

To evaluate the relationship between the Brief Intellectual Functioning (BIA) and the five

BDEFS factors, the original 11-item ADHD-EF Index, the new 15-item ADHD-EF index, and

Total Score; Pearson correlations were conducted. Please refer back to pages 75-78 for more

details regarding this analysis. The average Brief Intellectual Index (BIA) was 101.9 with a

standard deviation of 10.9. The correlations between the BIA and each of the factors were as

follows: Self-Management to Time (r=.26, p=.00); Self-Organization/Problem Solving (r=.04,

p=.64); Self-Restraint (r=.09, p=.28); Self-Motivation (r=.12, p=.16); Self-Regulation of

Emotion (r=.13, p=.15); Total Score (r=.15, p=.07); original 11-item ADHD-EF Index (r=.20,

p=.02), and new 15-item ADHD-EF index (r=.16, p=.06). In sum, the BIA was significantly

75

correlated with only two factors: Self-Management to Time and the original 11-item ADHD-EF

Index.

Research Question 3

Are the same ADHD-EF Index questions the most predictive of a diagnosis of ADHD in a

college student population as they are in the original normative sample?

To evaluate whether the same items from the original 11-item ADHD-EF Index were

needed to discriminate between students with and without ADHD in the college student

population, a control sample was obtained to compare to the clinic sample of students (a

description of this sample can be found in the Preliminary Analyses section, and is represented in

Table 1).

Models

In order to determine which items on the BDEFS best discriminate between college

students with and without ADHD, a discriminant function analysis (DFA) was conducted. First,

a DFA was conducted on all 89 items on the BDEFS. The DFA was then rerun with only the

items with a Structure Matrix loading of .4 or higher, which was the preset cutoff. There were

14 items at or over a Structure Matrix loading of .4. In addition, five items approached the .4

mark (.394-.372). Table 6 is a review of the canonical coefficients and structure coefficients of

each item with the items in bold to denote highest coefficients.

76

Table 6 Summary of Canonical Coefficients and Structure Loadings

Predictor (Scale Item) Canonical Coefficient

Structure Loadings

1 Procrastinate or put off doing things until the last minute -.145 .233

2 Poor sense of time -.009 .352

3 Waste or mismanage my time -.245 .315

4 Not prepared on time for work or assigned tasks -.223 .372

5 Fail to meet deadlines for assignments .094 .363

6 Have trouble planning ahead or preparing for upcoming

events. -.045 .375

7 Forget to do things I am supposed to do .149 .436

8 Can't seem to accomplish the goals I set for myself .083 .423

9 Late for work or scheduled appointments .233 .366 10 Can't seem to hold in mind things I need to remember to do .018 .426

11 Can't seem to get things done unless there is an immediate deadline

-.019 .357

12 Have difficulty judging how much time it will take to do something or get somewhere

.135 .417

13 Have trouble motivating myself to start work -.027 .298 14 Have difficulty motivating myself to stick with my work

and get it done .173 .402

15 Not motivated to prepare in advance for things I know I am supposed to do

-.122 .337

16 Have trouble completing one activity before starting into a new one

.020 .394

17 Have trouble doing what I tell myself to do -.042 18 Difficulties following through on promises or commitments

I may make to others .173 .364

19 Lack self-discipline .044 .299

20 Have difficulty arranging or doing my work by its priority or importance; can't "prioritize" well .318 .469

21 Find it hard to get started or get going on things I need to get done .223 .445

22 I do not seem to anticipate the future as much or as well as others

-.137 .291

23 Can't seem to remember what I previously heard or read about

-.124 .348

24 I have trouble organizing my thoughts .394 .527

77

Table 6 continued Predictor (Scale Item) Canonical

Coefficient Structure Loadings

25 When I am shown something complicated to do, I cannot keep the information in mind so as to imitate or do it correctly

-.083 .347

26 I have trouble considering various options for doing things and weighing their consequences

-.132 .325

27 Have difficulties saying what I want to say -.068 .244

28 Unable to come up with or invent as many solutions to problems as others seem to do

-.150 .210

29 Find myself at a loss for words when I want to explain something to others

-.121 .269

30 Have trouble putting my thoughts down in writing as well or as quickly as others

.114 .291

31 Feel I am not as creative or inventive as others of my level of intelligence -.216 .110

32 In trying to accomplish goals or assignments, I find I am not able to think of as many ways of doing things as others

-.100 .258

33 Have trouble learning new or complex activities as well as others

-.179 .296

34 Have difficulty explaining things in their proper order or sequence

-.094 .360

35 Can't seem to get to the point of my explanations as quickly as others .205 .345

36 Have trouble doing things in their proper order or sequence -.117 .376 37 Unable to "think on my feet" or respond as effectively as

others to unexpected events .007 .235

38 I am slower than others at solving problems I encounter in my daily life

.090 .293

39 Easily distracted by irrelevant events or thoughts when I must concentrate on something

.091 .470

40 Not able to comprehend what I read as well as I should be able to do; have to reread material to get its meaning .241 .413

41 Cannot focus my attention on tasks or work as well as others

.164 .536

42 Easily confused .227 .420

43 Can't seem to sustain my concentration on reading,

paperwork, lectures, or work .287 .505

44 Find it hard to focus on what is important from what is not important when I do things

-.151 .381

45 I don't seem to process information as quickly or as accurately as others

-.013 .334

78

Table 6 continued Predictor (Scale Item) Canonical

Coefficient Structure Loadings

46 Find it difficult to tolerate waiting; impatient .140 .303 47 Make decisions impulsively -.242 .247 48 Unable to inhibit my reactions or responses to events or

others -.194 .269

49 Have difficulty stopping my activities or behavior when I should do so.

.126 .301

50 Have difficulty changing my behavior when I am given feedback about my mistakes.

.051 .349

51 Make impulsive comments to others. .162 .270 52 Likely to do things without considering the consequences

for doing them. .051 .276

53 Change my plans at the last minute on a whim or last minute impulse.

-.060 .292

54 Fail to consider past relevant events or past personal experiences before responding to situations (I act without thinking).

.065 .288

55 Not aware of things I say or do. .028 .285 56 Have difficulty being objective about things that affect me. -.199 .166 57 Find it hard to take other people's perspectives about a

problem or situation. -.249 .098

58 Don't think or talk things over with myself before doing something.

.147 .311

59 Trouble following the rules in a situation. .164 .293 60 More likely to drive a motor vehicle much faster than others

(Excessive speeding). -.120 .102

61 Have a low tolerance for frustrating situations .156 .284 62 Cannot inhibit my emotions as well as others. .027 .202 63 I don't look ahead and think about what the future outcomes

will be before I do something (I don't use my foresight). .206 .275

64 I engage in risk taking activities more than others are likely to do.

.014 .207

65 Likely to take short cuts in my work and not do all that I am supposed to do.

.091 .352

66 Likely to skip out on work early if my work is boring to do. -.059 .301 67 Do not put as much effort into my work as I should or than

others are able to do. -.006 .321

68 Others tell me that I am lazy or unmotivated. -.082 .235 69 Have to depend on others to help me get my work done. .195 .314 70 Things must have an immediate payoff for me or I do not

seem to get them done. -.123 .313

79

Table 6 continued Predictor (Scale Item) Canonical

Coefficient Structure Loadings

71 Have difficulty resisting the urge to do something fun or more interesting when I am supposed to be working.

.053 .309

72 Inconsistent in the quality or quantity of my work performance.

.000 .370

73 Unable to work as well as others without supervision or frequent instruction.

.027 .323

74 I do not have the willpower or determination that others seem to have.

-.193 .327

75 I am not able to work toward longer term or delayed rewards as well as others. .254 .366

76 I cannot resist doing things that produce immediate rewards, even if those things are not good for me in the long run.

-.034 .283

77 Quick to get angry or become upset. -.149 .153 78 Overreact emotionally. -.045 .124 79 Easily excitable. -.313 .076 80 Unable to inhibit showing strong negative or positive

emotions. -.043 .154

81 Have trouble calming myself down once I am emotionally upset.

-.089 .142

82 Cannot seem to regain emotional control and become more reasonable once I am emotional.

-.139 .130

83 Cannot seem to distract myself away from whatever is upsetting me emotionally to help calm me down. I can't refocus my mind to a more positive framework.

.286 .257

84 Unable to manage my emotions in order to accomplish my goals successfully or get along well with others. .200 .279

85 I remain emotional or upset longer than others. -.101 .155 86 I find it difficult to walk away from emotionally upsetting

encounters with others or leave situations in which I have become very emotional.

.013 .126

87 I cannot re-channel or redirect my emotions into more positive ways or outlets when I get upset.

-.093 .196

88 I am not able to evaluate an emotionally upsetting event more objectively.

-.103 .194

89 I cannot redefine negative events into more positive viewpoints when I feel strong emotions.

-.128 .191

Note: Canonical Coefficient items in boldface account for the highest importance for describing differentiation among groups. Structure loadings in boldface account for the highest amount of contribution to the significant discriminant function.

80

The model was run with these five items and combinations thereof to see if they increased the

sensitivity and specificity of the model. Only one of these items (item number 16 with a matric

loading of .394) improved the model; therefore, item number 16 was included in the model.

These 15 items are listed in Table 7 with a description and the factor in which they belong.

Table 7 New 15-Item ADHD-EF Index

# Item description Factor r

7 Forget to do things I am supposed to do Time .149

8 Can’t seem to accomplish the goals I set out for myself Time .083

10 Can’t seem to hold in mind things I need to remember to do Time .018

12 Having difficulty judging how much time it will take to do something or get somewhere

Time .065

14 Having difficulty motivating myself to stick with my work

and get it done

Time -.084

16 Have trouble completing one activity before starting into a

new one

Time -.074

20 Having difficulty arranging or doing my work by its priority or importance; can’t “prioritize” well

Time .238

21 Find it hard to get started or get going on things I need to get done

Time .066

24 I have trouble organizing my thoughts Organization .292

39 Easily distracted by irrelevant events or thoughts when I must concentrate on something

Organization .120

40 Not able to comprehend what I read as well as I should be able to do; have to reread material to get its meaning

Organization .146

41 Cannot focus my attention on tasks or work as well as others Organization .189

42 Easily confused Organization .053

43 Can’t seem to sustain my concentration on reading, paperwork, lectures, or work

Organization .167

Note: Bolded items overlap with the items found on the original 11-item ADHD-EF Index

81

The original 11-item ADHD-EF Index items are listed in Table 8 for comparison. Three of these

items overlap between the models and are identified in bold. The new 15-item ADHD-EF Index

had the ability to significantly discriminate between the ADHD group and the control group, and

this model represented the highest degree of specificity and sensitivity than the other models

attempted. The model is presented in Table 9 along with several other models run for

comparison (described below). The new 15-item ADHD-EF Index accounted for 44.8% of the

total relationship between the items and diagnosis. In addition to the new 15-item model

represented previously, several separate DFA’s were run using different sets of items. The

different models are described in Table 10.

Group Centroids

The group centroids for the four models are represented in Table 11. The group centroid

for the new 15-item ADHD-EF Index discriminates the most between the ADHD group and the

non-ADHD group.

Table 8

Original 11-Item ADHD-EF Index

# Item Description Factor

1 Procrastinates or puts things off until the last minute Time

6 Have trouble planning ahead or preparing for upcoming events Time

14 Having difficulty motivating myself to stick with my work and get it

done

Time

16 Having trouble completing one activity before starting into a new

one

Time

24 I have trouble organizing my thoughts Organization

49 Having difficulty stopping my activities or behavior when I should do so Restraint

82

Table 8 continued

Item Description Factor

50 Having difficulty changing my behaviors when I am given feedback about my mistakes

Restraint

55 Not aware of things I say or do Restraint

60 More likely to drive a motor vehicle much faster than others Restraint

65 Likely to take short cuts in my work and not do all that I am supposed to do

Motivation

69 Have to depend of others to help me get my work done Motivation

Note: Bolded items overlap with the items found on the new 15-item ADHD-EF Index

Table 9 Summary of Canonical Discriminant Functions

ADHD-EF Index Eigenvalue

% Variance

Canonical Correlation

R*

Canonical R2

Lambda Chi-

Square df Sig

New 15-item

.809 100.00 .669 .448 .553 330.65

1 14 .00

Original 11-item .660 100.00 .630 .397 .603

283.425

11 .00

5-item .602 100.00 .613 .376 .624

264.997

5 .00

2-item .636 100.00 .623 .388 .611

277.562

2 .00

Classification Rates

As far as overall classification rate, the highest rate was found with the new 15-item

ADHD-EF Index. The new 15-item ADHD-EF Index also had a higher sensitivity rate.

However, the specificity rate was relatively equal across all models. The classification data for

each model are represented in Table 12.

83

Table 10 ADHD-EF Index Description of Models

Model Description

Original 11-item ADHD-EF Index Barkley’s 11-item model which is published in the BDEFS manual (Barkley, 2011b).

New 15-item ADHD-EF Index The 15 items with the best discrimination between the ADHD and Non ADHD groups determined in this study.

5-item community control ADHD-EF Index The five items that Barkley’s logistic regression yielded from comparing the community control sample with the ADHD sample in his study. These five items are included in the 11-item model.

2-item ADHD-EF Index (screening tool) The two-item screening model was derived by using the two items with the highest canonical correlations from this study.

Table 11 Functions at Group Centroids

Function

Group New 15-

item Original 11-item

Community Control 5-

item

2-item (screener)

Control Group -.436 -.393 -.376 -.386 ADHD Group 1.851 1.671 1.596 1.641

Table 12 Classification Rates

Model Overall Sensitivity Specificity

New 15-item ADHD-EF-Index

91 81.5 93.2

Original 11-item ADHD-EF Index

89.1 69.4 93.7

Community Control 5-item model

87.7 63 93.5

2-item (screener) 88.4 68.5 93

84

Research Question 4

Is the factor structure of the BDEFS the same for college students as it is for the

normative sample, based on a confirmatory factor analysis?

The CFA was conducted using the modeling software M-Plus 7 on the 89-items of the

BDEFS. The hypothesized model is the model presented by Barkley (2011b) in that items 1-21

belong to Factor 1 (Self-Management to Time), items 22-45 belong to Factor 2 (Self-

Organization/Problem Solving), items 45-64 belong to Factor 3 (Self-Restraint), items 65-76

belong to Factor 4 (Self-Motivation), and items 77-89 belong to Factor 5 (Self-Regulation of

Emotions). This five-factor model is hypothesized to be the same in the college student

population as it was in Barkley samples. The five factors are hypothesized to covary with each-

other. The assumptions of multivariate linearity and normality were reviewed through SPSS.

Maximum Likelihood estimation was used to estimate the models. The model was analyzed on

the full sample of 596 and on the reduced model of 310 (to match Barkley’s clinic sample). For

both of the models, the chi square, comparative fit index (CFI), root mean square error of

approximation (REMSA), Tucker Lewis index, and standardized root mean square residual were

calculated.

Barkley first used an EFA model on his clinic referred sample of adults. He then tested

the CFA on the same clinic referred sample and then on the normative sample. The full sample

in this study most closely matches the normative sample. The reduced sample in this study most

closely matched his clinic referred sample; therefore, results from both samples were compared

and are represented in Table 1. The hypothesized model was tested and there was reasonable

support for Barkley’s five factor model in both samples.

85

Table 13 CFA Models

Model X2 df X2/df CFI RMSEA TLI SRMR p

N=595 10467.201 3817 2.742 .840 .054 .836 .055 .000 N=310 8412.733 3817 2.204 .801 .062 .796 .062 .000

In comparing the models, the suggested fit indices listed in Table 2 are utilized. After

comparing the two models, it was determined that both models were similar in characteristics.

Tabachnick and Fidell (2007) indicate that a standardized factor loading to indicate good fit

should be a value of .6 or above. The standardized factor loadings are represented in Table 14.

However, Comrey and Lee (1992) suggest that any factor loading above .55 is “good.” If using

the Comrey and Lee suggestions, only two of the eighty-nine factor loadings had a fit less than

“good.” If using Tabachnick and Fidell’s (2007) slightly more conservative fit cut-off, only

seven of the eighty-nine factor loadings were not in the acceptable range. The correlations for all

factors were significant (p=.000) and are represented in Table 14 as well.

Table 14 Standardized Factor Loadings and Standardized Residual Variances

Factor/Item Number

STDYX p -value

Factor 1 EF1 .667 0.00 FF2 .698 0.00 EF3 .794 0.00 EF4 .782 0.00 EF5 .700 0.00 EF6 .779 0.00 EF7 .737 0.00 EF8 .792 0.00 EF9 .668 0.00 EF10 .761 0.00 EF11 .793 0.00 EF12 .745 0.00 EF13 .776 0.00 EF14 .852 0.00 EF15 .822 0.00 EF16 .782 0.00

86

Table 14 continued Factor/Item Number

STDYX p -value

Factor 1 EF17 .799 0.00 EF18 .696 0.00 EF19 .743 0.00 EF20 .771 0.00 EF21 .814 0.00

Factor 2

EF22 .611 0.00 EF23 .740 0.00 EF24 .817 0.00 EF25 .690 0.00 EF26 .702 0.00 EF27 .673 0.00 EF28 .687 0.00 EF29 .734 0.00 EF30 .641 0.00 EF31 .478 0.00 EF32 .730 0.00 EF33 .726 0.00 EF34 .783 0.00 EF35 .774 0.00 EF36 .783 0.00 EF37 .683 0.00 EF38 .712 0.00 EF39 .753 0.00 EF40 .703 0.00 EF41 .808 0.00 EF42 .783 0.00 EF43 .780 0.00 EF44 .778 0.00 EF45 .782 0.00

Factor 3 EF46 .544 0.00 EF47 .741 0.00 EF48 .778 0.00 EF49 .710 0.00 EF50 .716 0.00 EF51 .767 0.00 EF52 .793 0.00 EF53 .635 0.00 EF54 .797 0.00 EF55 .712 0.00

87

Table 14 continued Factor/Item Number

STDYX p -value

Factor 3 EF56 .724 0.00 EF57 .557 0.00 EF58 .734 0.00 EF59 .663 0.00 EF60 .459 0.00 EF61 .655 0.00 EF62 .676 0.00 EF63 .704 0.00 EF64 .585 0.00

Factor 4

EF65 .818 0.00 EF66 .760 0.00 EF67 .755 0.00 EF68 .640 0.00 EF69 .691 0.00 EF70 .811 0.00 EF71 .693 0.00 EF72 .785 0.00 EF73 .695 0.00 EF74 .744 0.00 EF75 .806 0.00 EF76 .745 0.00

Factor 5 EF77 .705 0.00 EF78 .783 0.00 EF79 .555 0.00 EF80 .642 0.00 EF81 .820 0.00 EF82 .827 0.00 EF83 .805 0.00 EF84 .812 0.00 EF85 .787 0.00 EF86 .700 0.00 EF87 .815 0.00 EF88 .836 0.00 EF89 .834 0.00 Note: Structure loading in bold indicate items under the structure loading cut-off of .6.

88

CHAPTER 5

DISCUSSION

The aim of this study was to investigate the psychometric properties of the Barkley

Deficits in Executive Functioning Scale (BDEFS) in a college student population. The BDEFS

was published in 2011, and there has yet to be independent support for the psychometric

properties of this measure. The BDEFS is an 89-item self-report scale used to measure

impairment in executive functioning. Barkley’s research suggests that the BDEFS is a five-

factor model. This study analyzed the self-informant vs. other-informant correlations for the five

factors of the BDEFS. Additionally, this study analyzed the relationship between the five

BDEFS factors and a measure of brief intellectual ability. In addition, a discriminant function

analysis was conducted to see which BDEFS items best discriminated between participants with

and without a diagnosis of ADHD in a college student population. Finally, a confirmatory factor

analysis was conducted to see if the factor structure of the BDEFS in a college population was

similar to the original factor structure.

This chapter discusses the results of the study and conclusions of the analyses conducted.

The findings are organized by research question and the practical implications for these findings

are discussed. Finally, limitations of the study are analyzed.

Relationship between BDEFS Self-Report Form and Other-Informant Form

This research question had multiple goals: (a) what is the relationship of the self-

informant ratings and the other-informant ratings on the five factors of the BDEFS, (b), are these

correlations statistically significantly different from the correlations that Barkley found, and (c)

are the self-informant ratings higher or lower than the other-informant ratings. The current study

identified significant self-informant rating and other-informant rating correlations for all BDEFS

89

factors. It is interesting that the factors which include more outward behaviors (Self-Restraint

and Self-Regulation) were correlated highest, and the factors which include more inward

behaviors were more weakly correlated (Self-Management to Time, Self-Motivation, and Self-

Organization/Problem Solving). Specific demands of the college student which are often

different from the general adult population rely heavily on the inward behaviors; therefore, the

college student may feel more pressure in these areas and rate themselves more impaired than

others would rate them. The college student with ADHD often is able to reach deadlines, but

procrastinates until the final moments. They often become exceedingly frustrated with their time

management and motivation issues (rating themselves more impaired on those BDEFS question);

however, others may not see this struggle (rating them less impaired).

In looking at how the correlations in this study compared to the correlations in the study

Barkley conducted, it was found that all BDEFS self-ratings and BDEFS other-ratings

correlations in the current study were significantly lower than the correlations identified by

Barkley. We can speculate as to why the correlations between self- and other-ratings were lower

in this sample than in Barkley’s sample. One, many of the questions on the BDEFS are about

symptoms that cannot necessarily be seen by another individual. For example, “wastes or

mismanages time” (item 3), “has trouble motivating self to work” (item 13), “has difficulty

saying what he/she wants to say” (item 27), “or cannot focus on the task at hand as well as

others” (item 41) may be particularly difficult for someone else to rate. On the original study

published by Barkley (211b), correlations between self-informants and others-informants were

strong at .66-.79. The correlations found in this study were weak to moderate (.295-.534), and

statistically significantly smaller than those found by Barkley. The only information about the

other-informant given from Barkley’s analysis was that this should be a person who knows the

90

person well. In the present study, this was often a significant other, parent, or roommate. As was

just discussed, a reason that the college students’ self-ratings may have varied more from the

other-informant ratings versus the general adult population is perhaps due to the difference in

demands placed on the two different groups. The responsibilities in the college setting tend to be

a greater reliance on executive functions than in the adult in general. Given that the majority of

these are unseen by others, it is not improbable that these results were found.

In addition, it was found in the current study that the BDEFS self-ratings were higher, in

all but one case, than the BDEFS other-ratings. The BDEFS self-ratings were not higher than the

BDEFS other-ratings for the factor Self-Regulation of Emotion. In evaluating these results, the

researcher must consider several possibilities. Are differences in self-other ratings scores a

function of the scale itself or a function of the rater? Several possibilities can be speculated as to

why the self-ratings tended to be higher than the other-ratings. First, in the case of college

students the risk of malingering must be considered. As was discussed in the literature review,

stimulant medications such as used in the treatment of ADHD have been misused and abused at

alarming rates on college campuses. Students may try to look worse than they are in order to

obtain medication (Booksh, et al., 2010). Second, related to this issue, is that of “crisis”.

Students who are referred to this clinic for evaluation of their symptoms of ADHD are generally

doing so because they are in some kind of crisis situation, such as failing a class or academic

probation. Therefore, they are likely to rate themselves as more impaired than would their other-

informant.

Relationship between the BIA and BDEFS Factors

The second area of research was aimed at examining the relationship between the

intellectual ability of college students with ADHD and their responses on the BDEFS. This is an

91

important analysis, in that it addresses a major limitation of traditional EF tests. The traditional

EF tests were often contaminated by IQ, specifically indices of intelligence including measures

of motor and naming speed, which is not related to EF (Salthouse (1996, 2005). In addition,

there is a significant overlap between IQ and EF tests in relation to working memory (Antshel et

al., 2010). The current study found that intelligence was significantly correlated with one BDEFS

factor: Self-Management. This relationship was in the positive direction (meaning as IQ

increased, so did the level of impairment in Self-Management). Additionally, the correlation was

relatively weak (r = .26). There are several plausible explanations for the correlation between

intelligence and the factor Self-Management to Time. The first issue is that of selection bias. All

participants in this analysis were non-randomly selected, in that they were either self-referred or

referred by a doctor, counselor, or academic advisor for evaluation of ADHD. Furthermore, all

participants received a diagnosis of ADHD following their evaluation. As was discussed in the

literature review, college students with ADHD tend to have some protective factors to help them

succeed in their academic programs. Once such protective factor may be intelligence (DuPaul et

al., 2009; Glutting et al., 2005). High school students who have ADHD (whether known or

unknown) and continue on to college after graduation likely do so based on their intellectual

abilities, rather than organizational skills. More specifically, the skill of time management may

have never been developed. Put succinctly, students with intellectual capacity sufficient to

complete high-quality work at the last minute with minimal organization may never have been

forced to organize their workload in high school. Students with ADHD (i.e. poor time

management skills) who did not have the requisite intellectual functioning likely did not self-

select to attend college. Therefore, they are not represented in this sample. In essence, the

higher the intellectual ability, the less the need for developing time management skills to succeed

92

in high school. This may partially explain why, as intelligence increased, so did the impairment

of time management. In a student with both ADHD and high IQ, the degree of intellectual

ability might actually function to progressively reduce the consequences for failing to learn basic

time management strategies, resulting in an inverse correlation between these areas.

An alternative (and complementary) way to view this correlation is that a college student

who has high intelligence and also has good time management skills (whether diagnosed with

ADHD or not) would be unlikely to find themselves being evaluated for ADHD. Mostly

students in “crisis” situations or struggling in some way were included in this sample. On an

item level, the types of information gathered in the factor of Self-Management to Time relate

heavily to procrastination and self-discipline relative to school work, which is consistent with

research on salience and ADHD. It has been noted anecdotally and supported by research

(Zentall, 2005), that students with ADHD tend to become most productive when a deadline is

approaching. Again, students who procrastinate until the last minute and do not have the ability

to produce good work at the last minute (e.g. students with lower relative intellectual ability) are

less likely to find themselves in college. For higher ability individuals, college may represent the

first time that their intellectual abilities are overmatched by the need for time management,

leading to increased “crisis” situations and subsequent referrals for evaluation.

Barkley’s analysis utilizing an early version of the BDEFS found only the Self-

Organization/Problem Solving factor to be associated with IQ. The results of the current study

supported the researcher’s original hypothesis that Self-Organization/Problem Solving would not

be significantly correlated with IQ. While the prediction was borne out, it was predicated on the

assumption that the intellectual abilities in the college sample would be higher than the general

population. However, the Brief Intellectual Ability (BIA) in the current study was average

93

(X=101.9, SD=10.6). There are a few reasons why this may have been the case. First, Barkley

used a full scale IQ in his study, and the current study used the BIA. The BIA has only .60-.69

correlation with a full measure of IQ (McGrew & Woodcock, 2011), and clinical experience has

demonstrated that it is generally an underestimate of full scale IQ. Furthermore, the BIA is only

made up of three sub-tests. Two of these are heavily influenced by timed testing (Concept

Formation and Decision Speed) and one (Concept Formation) is consistent in design with tasks

recruiting working memory and attention to detail. Students with ADHD tend to struggle with

processing speed tasks (Weyandt, 2005), potentially demonstrating a lowered BIA than if a non-

timed or mixed measure were used. Given this, it remains unclear to what degree intelligence is

related to the Self-Management to Time factor. However, this would be consistent with the

hypothesis that utilizing the BIA yields suppressed IQ scores overall, while allowing for

reasonable comparison within the sample.

Reevaluation of ADHD-EF Index

The current 11-item ADHD-EF Index developed by Barkley was created to provide

clinicians with a brief tool to predict a diagnosis of ADHD in the adult population. Of the 89

items on the BDEFS, these 11 items were selected using two logistic regressions which is

discussed in detail in the literature review of this manuscript. While having such a brief index is

valuable in the clinical setting to identify individuals who are likely to have ADHD for the

purposes of referring them for a more extensive assessment, this index has not been validated on

a college student population. Therefore, the same items that predict ADHD in the adult

population may not be the best predictors of ADHD in a college student population. When

examining the ability of some items on the BDEFS to successfully discriminate between the

ADHD group and the non-ADHD group, results show that the best model was the new 15-item

94

ADHD-EF-Index which successfully categorizes 91% of the participants and accounted for

44.8% (canonical R2 = .448) of the variance in a college student population. When the original

11-item ADHD-EF Index was used on the current college student sample, the 11 items were able

to discriminate between those students with and without ADHD, but to a slightly lesser extent

(89%, canonical R2= .397). Interestingly, only three of the items from the original 11-item index

overlap with the new 15-item ADHD-EF Index, thus indicating that, in a college population,

different items are needed to accurately discriminate between those with ADHD and those

without.

The three items that overlap are all from the factors Self-Management to Time and Self-

Organization/Problem Solving, and these items are: “Having difficulty motivating myself to stick

with my work and get it done” (item 14, Self-Management to Time), “Having trouble completing

one activity before starting into a new one” (item 16, Self-Management to Time), and “I have

trouble organizing my thoughts” (item 24, Self-Organization/Problem Solving).

The original ADHD-EF Index scale uses 11 items to discriminate individuals with

ADHD from those without. Of the items that overlapped between the two scales, one was from

Barkley’s comparison of the ADHD group to his clinical control group, and two were derived

from his comparison between his ADHD and the community control group. When using the five

items from the original 11-item ADHD-EF Index that were obtained from the sample most

closely related to the sample in this study, these five items (1, 16, 24, 50, and 65) discriminate

almost as well as the original 11-item ADHD-EF Index (at 87.7% vs. 89%). Therefore, when

looking at the college student population, the original 11-item ADHD-EF Index may not be the

most efficient model when attempting to discriminate students with and without ADHD. In fast-

95

paced clinical settings, gaining approximately one percentage point of discriminant ability may

not justify a greater than 100% increase in the length of the screening instrument.

In looking at the difference between the items on the original 11-item ADHD-EF Index

and the new 15-item ADHD-EF Index, there are some general themes. On the original 11-item

ADHD-EF Index, there are four items from the factor Self-Management to Time, one from Self-

Organization/Problem Solving, four from Self-Restraint, and two from Self-Motivation. The

new scale pulled only from Self-Management to Time (eight items) and Self-

Organization/Problem Solving (six items) all of which are listed below.

Self-Management to Time

Forget to do things I am supposed to do (item 7)

Can’t seem to accomplish the goals I set out for myself (item 8)

Can’t seem to hold in mind things I need to remember to do (item 10)

Having difficulty judging how much time it will take to do something or get somewhere

(item 12)

Having difficulty motivating myself to stick with my work and get it done (item 14)

Have trouble completing one activity before starting into a new one (item 16)

Having difficulty arranging or doing my work by its priority or importance; can’t

“prioritize” well (item 20)

Find it hard to get started or get going on things I need to get done (item 21)

Self-Organization/Problem Solving

I have trouble organizing my thoughts (item 24)

96

Easily distracted by irrelevant events or thoughts when I must concentrate on something

(item 39)

Not able to comprehend what I read as well as I should be able to do; have to reread

material to get its meaning (item 40)

Cannot focus my attention on tasks or work as well as others (item 41)

Easily confused (item 42)

Can’t seem to sustain my concentration on reading, paperwork, lectures, or work (item 43)

As discussed in the literature review, the population of college students with ADHD

tends to be either Combined Type ADHD or Inattentive Type ADHD (Barkley and Murphy,

2011). Far fewer of the Hyperactivity Type ADHD only is seen in the college population.

(DuPaul et al., 2009). In fact, none of the participants in the current sample were of the

Hyperactive subtype. There are several hypotheses of why this might be; however, it is widely

accepted that students either “grow-out” of these behaviors or the acts of being accepted into and

attending college self-selects for students with more self-control, successful academic histories,

advanced coping skills, and higher cognitive abilities (DuPaul et al., 2009; Glutting et al., 2005).

Thus, it is not surprising that a major difference between the original 11-item ADHD-EF Index

derived from an adult population has several items from the factor, Self-Restraint, and the model

derived from the college student population has no such items. Along the same lines, no items

from the factor Self-Motivation were noted to be derived from the college student population.

Again, college students are self-selected to be more motivated than the non-college student

(Reaser et al., 2007); therefore, items tapping these types of symptoms would not differentiate

between students with and without ADHD.

97

The time demands facing a college student may be more than that of non-college student

adults. Given the increased pressure in college to have good time management and

organizational skills, it is not unlikely that the items needed to differentiate the ADHD group

from the non ADHD group would be different in the college student population verses the adult

population. For example, when surveying most college students about procrastination and skills

related to planning-ahead, one might expect to find most students struggling with the perception

of difficulties in these areas. Therefore, items tapping these symptoms may not necessarily

distinguish between college students with and without ADHD as it does in the general adult

population. The results here are similar to what was seen by Murphy (2005) and Proctor and

Prevatt (2009) when they asserted that college students with ADHD have more problems

focusing, making deadlines, task completion, and sustaining effort in presumed irrelevant tasks.

Overall, when comparing the specificity rate (percentage of students without ADHD

correctly identified in the non-ADHD group) there were no clinically relevant differences,

indicating all of the models were equally effective in accurately classifying participants as non-

ADHD. However, there were clinically relevant differences with regards to sensitivity (the

ability to correctly identify someone with ADHD). The new 15-item ADHD-EF Index had a

sensitivity rate of 81.5% versus the original 11-item ADHD-EF Index (69.54%), the 5-item

community control ADHD-EF Index (63%), or the 2-item ADHD-EF Index screener (68.5%)

which will be discussed below. This provided evidence in support of using a different model to

predict or screen for ADHD in the college student population. If given as a screening device, the

new 15-item ADHD-EF Index has a much better rate of correctly identifying college students

who likely have ADHD. The clinical utility of a quick screening tool is substantial in a

university or college campus setting. Given that college students with ADHD experience

98

significantly more academic consequences than college students without ADHD (Reaser et al.,

2007), it is important to efficiently identify these students in order to offer treatment and

assistance. It is worth emphasizing that the ADHD-EF Index is meant to be a screening tool.

With this in mind, if error is present, it is desirable to err in the direction of over-identifying

students as belonging to the potential-ADHD group so that they can be referred for a full

diagnostic evaluation of their symptoms.

In addition to the new 15-item ADHD-EF Index and the 5-item Community Control

ADHD-EF Index, another interesting result came of these analyses. A very brief two-item model

identified items number 20 (having difficulty arranging or doing my work by its priority or

important; can’t prioritize well) and 24 (I have trouble organizing my thoughts) as generally

equal in specificity and sensitivity to the original 11-item ADHD-EF Index. Thus, these two

items alone may be quite beneficial as a quick screening tool. If a clinical or academic advisor

asks a student if they are having difficulty prioritizing their tasks and difficulty organizing their

thoughts, the likelihood of missing a student who has ADHD is only about 7%. Of course, this is

not diagnostic given that the sensitivity is in the high 60% range. However, this is a useful,

quick, and low cost screening method for determining which students to refer for more thorough

assessment.

Evidence of Factorial Validity

Only one researcher, the author of the BDEFS, has evaluated the factor structure of the

BDEFS to date. No study has yet been published looking at this factor structure for the BDEFS

on a college student population. The current study found moderate support for the published

factor structure of the BDEFS in the college student population. This support was found in both

samples analyzed in this study. While there was support for the factor structure, some of the fit

99

indices were not as strong as had been hoped for. As mentioned in the planned analysis section,

there are multiple ways to determine goodness of fit, or evaluating the model, for a CFA. One of

the most common is the Chi Squared (X2). A goodness of fit is represented by a non-significant

X2 and in this analysis, the X2 was significant. However, one of the short-falls with relying on X2

is the influence of a large n on the statistic making most any differences statistically significant

(Tabachnick & Fidell, 2007). Given the problems with sample size and the underlying

assumptions that influence the results, other statistics have been proposed. Related to the X2, a

cursory measure to look at fit is to use the ratio of X2 to the degrees of freedom. Referring back

to Table 2, if this is less than two, this provides support for the model (Tabachnick & Fidell,

2007). In this analysis, the X2 is 2.2 indicating a level just slightly over the generally accepted

range. The CFI and TLI fit indices were at the moderate level showing near their respective

specified cut off points. However, the larger sample (more closely aligned with the normative

sample) had fit indices values that were slightly stronger. The REMSA and the SRMR are the

two fit indices which show the best support for Barkley’s factor model of the BDEFS. These

indexes are best suited for larger samples (like the current sample) and may explain the reason

these indices were more supportive than the preceding indices that are heavily influenced by the

larger sample size. Another potential explanation for the moderate results relates to the

assumption that a sample must be normally distributed. This is an impairment rating scale.

Therefore, the participants in the control sample and the participants in the ADHD sample likely

rated the items quite differently, and their responses were not normally distributed. There is also

the possibility of a floor effect for the control sample.

When looking at the items on an individual level, all 89 items has a statistically

significant factor loading, meaning each item contributed to its respective factor indicating

100

evidence of validity to the factor structure of the BDEFS in a college student population. While

all items were significant, there were a few items that fell just under the .55 cut off score for

“good” fit. The two items were “I feel I am not as creative or inventive as others of my

intelligence level” (item 31) and “More likely to drive a motor vehicle faster than other” (item

60). While still statistically significant, these two items account for the least amount of variance

in the model. When looking at the item pertaining to creativity, the college student population

may differ on their perceptions of creativity as it relates to intelligence in a different way than the

general adult population does. Specifically, college students are generally higher in cognitive

abilities than the adult population and creativity may be more revered on college campuses as

well. As published, this item belongs to the factor Self-Organization/Problem Solving. This

question about creativity does not necessarily fit with the other type of items when referring to a

college student population. For instance, other items in this area focus on tasks of concentration

and organizing, with only one other questions discussing something similar to creatively (coming

up with a new way to solve a problem). The question pertaining to creativity may be viewed as a

difficult area in the college student population regardless of any impairment on that factor.

The item related to driving falls within the factor of self-restraint. Other items in this factor

pertain mostly to impatience and impulsivity. While driving fast is generally related to ADHD

and EF deficits, it is also common for this age group in general. Thus, endorsing this item may

not have much to do with the endorsement of other items included in this factor making it a less

than desirable fit.

Limitations

There are several issues that could affect the findings of the study. All participants in this

study were acquired from a large public university in the southeastern United States. The results

101

of this study may not generalize to other college student populations such as private colleges or

universities, community colleges, or universities in northern or western areas. In addition, these

students were mostly traditional students under the age of 30. The results found may not

generalize to non-traditional (or second career) students or graduate students. Students in this

study were recruited in several different ways, which could have affected the results. The

control group was primarily from one college of the university and was heavily female in

composition. To remedy this, additional control participants were collected from the same

university in several undergraduate classes and in general areas on campus. Participants in the

one college received either extra credit for participating or participation fulfilled a course

requirement. The participants recruited in other parts of the university were offered monetary

incentives for participating. These different methods could have influenced the way the

participants responded to the survey.

In regards to the participants in the ADHD group, there are several issues which could

have affected the validity of the results as well. As reported, the participants in the ADHD group

were evaluated at an on-campus clinic to determine whether or not they qualified for a diagnosis

of ADHD. Students often presented for an evaluation when they were in a crisis situation such

as academic probation or losing a scholarship due to poor academic performance. In addition,

students may have been looking for a diagnosis of ADHD to acquire stimulant medications

(secondary gain). All of these factors could have influenced the way in which they completed

the survey which measured impairment.

As far as the evaluation and determination of ADHD is concerned, all students who were

given a diagnosis met criteria for ADHD using the DSM-IV-TR. Since that time, the DSM-5 has

been published, with a slightly less stringent criteria for diagnosing ADHD in adults and with

102

more relevant criteria. Therefore, there were likely many students who did not receive a

diagnosis of ADHD and were excluded from this sample that would receive such a diagnosis if

currently evaluated It is possible that a sample with more moderate levels of ADHD could

influence the results. Finally, the students in the ADHD group were given the Brief Intellectual

Abilities Index (BIA) from the Woodcock Johnson Tests of Cognitive Abilities as a proxy for

IQ. Unfortunately, the BIA has only a .60-.69 correlation with IQ from other full measures of

intelligence (McGrew & Woodcock, 2011). This may explain why results from that research

question were moderate in nature.

Implications for Future Research

Given that the BDEFS is in its infancy, there are multiple avenues for continued research.

The Discriminant Function Analysis (DFA) in this study was conducted with sufficient sample

size; however, there were not enough participants in each group to run a split-half analysis. This

would entail conducting the DFA on half of the sample, then using the results from that analysis

to check the specificity and sensitivity on the other half of the data. This would increase the

validity of the results. In addition, given that the new 15-Item ADHD-EF Index has been

produced on this single sample, it is recommended that replication studies be conducted before

this is used in clinical practice as a standalone measure.

The control sample in the current study had a higher representation of females; therefore

a reduced sample was used for the CFA. It is advised that a sample of at least 500 participants,

with equal gender distribution, be collected to run the CFA again. In addition, a multi-sample

CFA should be conducted given the anticipated difference in these groups regarding impairment.

In continuing to investigate the psychometric properties of the BDEFS on a college student

population, several other analyses should be considered. This current study did not exclude

103

participants in the ADHD group who had co-occurring disorders. During the course of the

ADHD evaluation, participants complete a DSM checklist of clinical symptoms. These

symptoms should be correlated with the five factors of the BDEFS to see if there is a

relationship. As well, Barkley (2012b) noted that when there were higher internalizing

symptoms, the participant was more likely to rate themselves more impaired on the BDEFS than

their other informant. This should be replicated in the college population. Finally, concurrent

validity should be evaluated by comparing responses on the BDEFS to the Behavior Rating

Inventory of Executive Functioning (BRIEF-A), which has evidence of validity.

Implications for Clinical Practice

One of the most notable implications resulting from this study is the identification of the

new 15-item ADHD-EF Index that better discriminates college students with ADHD from

students without ADHD. In addition to this new scale, the two brief screening questions

identified as highly identifying students that may have ADHD could be used by campus

professionals to quickly identify students who may need a more extensive evaluation. These

professionals may include mental health counselors, academic counselors, and medical

professionals. Another finding indicating that the 5-item ADHD-EF Index (from the original

ADHD-EF Index) was also quite useful in discriminating the college student with and without

ADHD and can also be used as a screening tool.

There are several other clinical implications that could be considered when using the

BDEFS with the college student population. First, there were statistically significant differences

between the self-report and the other-informant report form of the BDEFS in this sample. This

result highlights the need for clinicians to gain not only the students’ perception of their

impairments, but the perceptions of those who interacted with the student the most. In addition,

104

results suggest that clinicians view the self-report rating with caution, taking into consideration

the student’s circumstances such as a crisis situation or the possibility of secondary gain.

Second, high cognitive ability students tend to have more impairment in the area of self-

management of time. Students in the category of high cognitive ability tend to be overlooked

because of their abilities; however, the results of this study highlight the need to provide a skills-

based intervention to improve skills for time-management. This may include setting alarms or

alerts for tasks and activities and encouraging the student with ADHD to have visual reminders

of time (countdown clocks on their desks).

105

APPENDIX A

INFORMED CONSENT FOR ADHD GROUP

Consent Form

1. I consent to receiving a psycho-educational assessment from the Adult Learning Evaluation Center at Florida State University.

2. I understand that no information concerning my evaluation will be released from the Adult Learning Evaluation Center within the limits of confidentiality that have been specified (see Client Information). 3. I understand the information provided to me regarding supervision and observation of services. 4. I understand that the fee for a psycho-educational assessment is $500.00 and is payable on the first day of the evaluation unless other arrangements have been finalized through financial aid. 5. I understand that it is in my best interest to put forth my best effort during the psycho-educational evaluation. 6. The following section specifically applies to a research project that you are being asked to consider.

I freely and voluntarily and without element of force or coercion, consent to be a participant in the research project, Exploration of the Factors Underlying Academic Difficulty in College

Students.

I understand that this research is being conducted by Dr. Frances Prevatt at Florida State University. I understand the purpose of the research project is to create an archival data base that can be used to evaluate correlates of learning disability (LD) and Attention Deficit Hyperactivity Disorder (ADHD) in a college population. I am being asked to allow the results of my current evaluation to be utilized in this archival data base. I understand that all clients in ALEC, (approximately 200 per year) are asked to participate in this research. I am not being asked to do

106

anything other than my standard evaluation; I am just allowing my data to be used later for research purposes. I understand that I must be at least 18 years of age in order to participate in this study. I understand that I will receive no direct benefits in return for participating in this research project. I understand that my participation is totally voluntary and I may withdraw my consent at any time in the research. I understand that if I do not agree for my data to be used, that will have no impact on my evaluation. I understand there is no risk involved if I agree to let my data be used. I understand that my identity will never be associated with the data (that is, my name and any identifying information will be removed.)The records will be kept private and confidential to the extent permitted by law. Data will be stored securely and only the researchers will have access to the data base. I understand that I may contact Dr. Frances Prevatt, Florida State University, Adult Learning Evaluation Center, at *********, for answers to questions about this research or my rights.

If you have any questions or concerns regarding this study and would like to talk to someone other than the researcher(s), you are encouraged to contact the FSU IRB at 2010 Levy Street, Research Building B, Suite 276, Tallahassee, FL 32306-2742, or 850-644-8633, or by email at humansubjects@magnet.fsu.edu. I do [ ] do not [ ] consent to allow my data to be used in the manner described above. I do [ ] do not [ ] give ALEC my permission to contact me by email or telephone to describe future research projects and ask me if I would be interested in participating. If yes, this permission is granted for ____ years from today’s date. I do [ ] do not [ ] consent to participate in an additional research study that involves the comparison of my responses to those of a group of college students without ADHD. Should I agree, I will be given an additional thirty-three questions, which will add approximately ten minutes to my psycho-educational evaluation. I have read, understand, and agree to all Adult Learning Evaluation Center procedures outlined in this document. Signature ___________________________ Date__________________________

107

APPENDIX B

BARKLEY DEFICITS IN EXECUTIVE FUNCTIONING SCALES

Instructions: How often do you experience each of these problems? Please circle the number next to each item that best describes your behavior DURING THE PAST 6 MONTHS. Note to committee- these items are in an on-line survey tool, so the formatting here is not the

same. This is for a reference on the actual questions that are contained within the survey

All items are on a Likert scale with 1= Never or Rarely, 2= Sometimes, 3= Often, and 4= Very Often 1) Procrastinate or put off doing things until the last minute

2) Poor sense of time

3) Waste or mismanage my time

4) Not prepared on time for work or assigned tasks

5) Fail to meet deadlines for assignments

6) Have trouble planning ahead or preparing for upcoming events.

7) Forget to do things I am supposed to do

8) Can't seem to accomplish the goals I set for myself

9) Late for work or scheduled appointments

10) Can't seem to hold in mind things I need to remember to do

11) Can't seem to get things done unless there is an immediate deadline

12) Have difficulty judging how much time it will take to do something or get somewhere

13) Have trouble motivating myself to start work

14) Have difficulty motivating myself to stick with my work and get it done

15) Not motivated to prepare in advance for things I know I am supposed to do

16) Have trouble completing one activity before starting into a new one

17) Have trouble doing what I tell myself to do

18) Difficulties following through on promises or commitments I may make to others

19) Lack self-discipline

20) Have difficulty arranging or doing my work by its priority or importance; can't "prioritize"

well

21) Find it hard to get started or get going on things I need to get done

22) I do not seem to anticipate the future as much or as well as others

23) Can't seem to remember what I previously heard or read about

108

24) I have trouble organizing my thoughts

25) When I am shown something complicated to do, I cannot keep the information in mind so as

to imitate or do it correctly

26) I have trouble considering various options for doing things and weighing their consequences

27) Have difficulties saying what I want to say

28) Unable to come up with or invent as many solutions to problems as others seem to do

29) Find myself at a loss for words when I want to explain something to others

30) Have trouble putting my thoughts down in writing as well or as quickly as others

31) Feel I am not as creative or inventive as others of my level of intelligence

32) In trying to accomplish goals or assignments, I find I am not able to think of as many ways

of doing things as others

33) Have trouble learning new or complex activities as well as others

34) Have difficulty explaining things in their proper order or sequence

35) Can't seem to get to the point of my explanations as quickly as others

36) Have trouble doing things in their proper order or sequence

37) Unable to "think on my feet" or respond as effectively as others to unexpected events

38) I am slower than others at solving problems I encounter in my daily life

39) Easily distracted by irrelevant events or thoughts when I must concentrate on something

40) Not able to comprehend what I read as well as I should be able to do; have to reread material

to get its meaning

41) Cannot focus my attention on tasks or work as well as others

42) Easily confused

43) Can't seem to sustain my concentration on reading, paperwork, lectures, or work

44) Find it hard to focus on what is important from what is not important when I do things

45) I don't seem to process information as quickly or as accurately as others

46) Find it difficult to tolerate waiting; impatient

47) Make decisions impulsively

48) Unable to inhibit my reactions or responses to events or others

49) Have difficulty stopping my activities or behavior when I should do so.

50) Have difficulty changing my behavior when I am given feedback about my mistakes.

51) Make impulsive comments to others.

109

52) Likely to do things without considering the consequences for doing them.

53) Change my plans at the last minute on a whim or last minute impulse.

54) Fail to consider past relevant events or past personal experiences before responding to

situations (I act without thinking).

55) Not aware of things I say or do.

56) Have difficulty being objective about things that affect me.

57) Find it hard to take other people's perspectives about a problem or situation.

58) Don't think or talk things over with myself before doing something.

59) Trouble following the rules in a situation.

60) More likely to drive a motor vehicle much faster than others (Excessive speeding).

61) Have a low tolerance for frustrating situations

62) Cannot inhibit my emotions as well as others.

63) I don't look ahead and think about what the future outcomes will be before I do something (I

don't use my foresight).

64) I engage in risk taking activities more than others are likely to do.

65) Likely to take short cuts in my work and not do all that I am supposed to do.

66) Likely to skip out on work early if my work is boring to do.

67) Do not put as much effort into my work as I should or than others are able to do.

68) Others tell me that I am lazy or unmotivated.

69) Have to depend on others to help me get my work done.

70) Things must have an immediate payoff for me or I do not seem to get them done.

71) Have difficulty resisting the urge to do something fun or more interesting when I

am supposed to be working.

72) Inconsistent in the quality or quantity of my work performance.

73) Unable to work as well as others without supervision or frequent instruction.

74) I do not have the willpower or determination that others seem to have.

75) I am not able to work toward longer term or delayed rewards as well as others.

76) I cannot resist doing things that produce immediate rewards, even if those things are not

good for me in the long run.

77) Quick to get angry or become upset.

78) Overreact emotionally.

110

79) Easily excitable.

80) Unable to inhibit showing strong negative or positive emotions.

81) Have trouble calming myself down once I am emotionally upset.

82) Cannot seem to regain emotional control and become more reasonable once I am emotional.

83) Cannot seem to distract myself away from whatever is upsetting me emotionally to help

calm me down. I can't refocus my mind to a more positive framework.

84) Unable to manage my emotions in order to accomplish my goals successfully or get along

well with others.

85) I remain emotional or upset longer than others.

86) I find it difficult to walk away from emotionally upsetting encounters with others or leave

situations in which I have become very emotional.

87) I cannot re-channel or redirect my emotions into more positive ways or outlets when I get

upset.

88) I am not able to evaluate an emotionally upsetting event more objectively.

89) I cannot redefine negative events into more positive viewpoints when I feel strong emotions.

111

APPENDIX C

INFORMED CONSENT FOR CONTROL GROUP

Measure of Attention Deficit Hyperactivity Disorder (ADHD) and Executive Functioning in

College students

I understand that I am not required to take this survey and that I have the option to decline participation. If I agree, my responses will be used in the research project described below. I understand that I will take this survey online at the above stated web address. I will keep this copy of the informed consent for my records, but I will sign a copy of this consent online prior to completing the survey. I understand that this survey is being collected to serve a research study, entitled “The Psychometric Properties of the Barkley Deficits in Executive Functioning” You will be part of the control group without ADHD. If you have a current diagnosis of ADHD and you report that on the survey, then your data will not be used as part of the control group. However, you may still participate in the study and still be eligible to participate in the lottery. This study is being conducted by Dr. Frances Prevatt at Florida State University. I understand the purpose of this research project is to evaluate an existing measure that currently has no normative data for college students. I will be given a questionnaire to complete, which will take approximately 15 minutes. About 300 college students will participate in this study, 150 with a diagnosis of ADHD and 150 without a diagnosis of ADHD. I understand that I must be at least 18 years of age in order to participate in this study. I understand that in return for participating in this research project, I will be entered in a drawing for a 1 in 25 chance of receiving a $15 gift certificate to the store of my choosing. I understand that my participation is totally voluntary and I may stop participation at any time in the research, and that there is no penalty for non-participation. I understand this consent may be withdrawn at any time, even after I have completed the survey. I understand that the responses I provide today are being collected with software that is designed to secure my data and provide me with confidentiality. Although every effort will be done to ensure confidentiality of my responses, all Internet-based communication is subject to the remote likelihood of tampering from an outside source. IP addresses will not be investigated and data will be removed from the server. My data and consent form will be kept electronically on secure servers at the FSU Learning Systems Institute (LSI) and will not be disclosed to third parties. LSI has physical and environmental controls in place to protect data. Data are backed up daily.

112

I understand that I may contact the primary researcher, Dr. Frances Prevatt at *******. I can also contact the Chair of the Human Subjects Committee, Institutional Review Board, through the Office of the Vice President for Research, at ********. I freely and voluntarily and without element of force or coercion, consent to be a participant in the research project “The Psychometric Properties of the Barkley Deficits in Executive Functioning Scale.” It is possible that I may wonder about my responses to the questions. If after having answered the survey questions, I feel I may have some symptoms of ADHD, I can contact my local chapter for Children and Adults with Attention-Deficit/Hyperactivity Disorder (CHADD) at www.chadd.org for information for assistance with resources or I may contact the following resources: The FSU Student Counseling Center ****** (free) The FSU Psychology Department Clinic ********(sliding scale fee) The FSU Human Services Center ******* (free) If you have any questions or concerns regarding this study and would like to talk to someone other than the researcher(s), you are encouraged to contact the FSU IRB at 2010 Levy Street, Research Building B, Suite 276, Tallahassee, FL 32306-2742, or 850-644-8633, or by email at humansubjects@magnet.fsu.edu You will be given a copy of this information to keep for your records. Statement of Consent:

I have read the above information. I have asked questions and have received answers. I consent to participate in the study. _____ YES. By checking yes, I consent to participate in this study. ________________ _________________ Signature Date ________________ _________________ Signature of Investigator Date

113

APPENDIX D

INTERNAL REVIEW BOARD FOR HUMAN SUBJECTS APPROVAL

The Florida State University Office of the Vice President For Research Human Subjects Committee Tallahassee, Florida 32306-2742 (850) 644-8673, FAX (850) 644-4392 APPROVAL MEMORANDUM Date: 3/29/2013 To: Frances Prevatt Dept.: EDUCATIONAL PSYCHOLOGY AND LEARNING SYSTEMS From: Thomas L. Jacobson, Chair Re: Use of Human Subjects in Research The Psychometric Properties of the Barkley Deficits in Executive Functioning Scale (BDEFS) The application that you submitted to this office in regard to the use of human subjects in the proposal referenced above have been reviewed by the Secretary, the Chair, and one member of the Human Subjects Committee. Your project is determined to be Expedited per 45 CFR § 46.110(7) and has been approved by an expedited review process. The Human Subjects Committee has not evaluated your proposal for scientific merit, except to weigh the risk to the human participants and the aspects of the proposal related to potential risk and benefit. This approval does not replace any departmental or other approvals, which may be required. If you submitted a proposed consent form with your application, the approved stamped consent form is attached to this approval notice. Only the stamped version of the consent form may be used in recruiting research subjects. If the project has not been completed by 3/28/2014 you must request a renewal of approval for continuation of the project. As a courtesy, a renewal notice will be sent to you prior to your expiration date; however, it is your responsibility as the Principal Investigator to timely request renewal of your approval from the Committee. You are advised that any change in protocol for this project must be reviewed and approved by the Committee prior to implementation of the proposed change in the protocol. A protocol change/amendment form is required to be submitted for approval by the Committee. In addition, federal regulations require that the Principal Investigator promptly report, in writing any unanticipated problems or adverse events involving risks to research subjects or others.

114

By copy of this memorandum, the Chair of your department and/or your major professor is reminded that he/she is responsible for being informed concerning research projects involving human subjects in the department, and should review protocols as often as needed to insure that the project is being conducted in compliance with our institution and with DHHS regulations. This institution has an Assurance on file with the Office for Human Research Protection. The Assurance Number is FWA00000168/IRB number IRB00000446. Cc: Betsy Becker, Chair HSC No. 2013.10087 Human Subjects Application For Full IRB and Expedited Exempt Review

1. Project Title and Identification

1.1 Project Title

The Psychometric Properties of the Barkley Deficits in Executive Functioning Scale (BDEFS)

Project is: Dissertation

1.2 Principal Investigator (PI)

Name(Last name, First name MI): Prevatt, Frances F

Highest Earned

Degree: Doctorate

University Department: EDUCATIONAL PSYCHOLOGY AND LEARNING SYSTEMS

Email:

The training and education completed in the protection of human

subjects or human subjects records: Other

Occupational

Position: Faculty

1.3 Co-Investigators/Research Staff

Name(Last name, First name MI): Coffman, Theodora Passinos; Co-Investigator

Highest Earned

Degree: Bachelor's

Degree

University Department: EDUCATIONAL PSYCHOLOGY AND LEARNING SYSTEMS

Email:

The training and education completed in the protection of human subjects

or human subjects records: FSU Training Module

Occupational

Position: Student

115

APPENDIX E

DEMOGRAPHIC QUESTIONNAIRE

If you consent to taking this survey, please select yes with the knowledge that your information will be kept confidential and used for research purposes only.

Yes No

If No Is Selected, Then Skip To End of Survey

1) What is your gender?

Male Female

2) What is your age?

_________________ (in years) 3) What is your ethnicity?

Caucasian African American Asian Hispanic Other

4) What year in college are you in?

Freshmen Sophomore Junior Senior Graduate Student

5) Have you been previously diagnosed with a Learning Disability?

Yes No

6) Have you been previously diagnosed with ADHD?

Yes No

116

APPENDIX F

BDEFS SCORING TEMPLATE

Scale Raw Score Percentile rank

Classification

Section 1, Q-1-21 Self-Management/ To Time

Section 2, Q-22-45 Self-Organization/ Problem Solving

Section 3, Q-46-64 Self-Restraint

Section 4, Q-65-76 Self-Motivation

Section 5, Q-77-89 Self-Regulation of Emotions

Total sections 1-5 Total EF

Count # of items answered 3 or 4

EF symptom count

Add items, 1,6,14,16,24,49,50,55,60 65, 69

ADHD-EF index score

117

APPENDIX G

OTHER INFORMANT BDEFS

Note to committee members- the questions contained in the other-informant version of the

BDEFS are identical in content to the self-report version. The only change is in first/third

person reference.

1) Procrastinates or puts off doing things until the last minute

2) Poor sense of time

3) Waste or mismanage his/her time

4) Not prepared on time for work or assigned tasks

5) Fails to meet deadlines for assignments

6) Has trouble planning ahead or preparing for upcoming events.

7) Forgets to do things that he/she am supposed to do

8) Can't seem to accomplish the goals he/she set for self

9) Late for work or scheduled appointments

10) Can't seem to hold in mind things he/she need to remember to do

11) Can't seem to get things done unless there is an immediate deadline

12) Has difficulty judging how much time it will take to do something or get somewhere

13) Has trouble motivating self to start work

14) Has difficulty motivating self to stick with his/her work and get it done

15) Not motivated to prepare in advance for things he/she knows he/she is supposed to do

16) Has trouble completing one activity before starting into a new one

17) Has trouble doing what he/she tells self to do

18) Difficulties following through on promises or commitments he/she may make to others

19) Lack self-discipline

20) Has difficulty arranging or doing his/her work by its priority or importance; can't "prioritize"

well

21) Finds it hard to get started or get going on things he/she need to get done

22) Does not seem to anticipate the future as much or as well as others

23) Can't seem to remember what he/she previously heard or read about

24) Has trouble organizing his/her thoughts

118

25) When he/she is shown something complicated to do, he/she cannot keep the information in

mind so as to imitate or do it correctly

26) Has trouble considering various options for doing things and weighing their consequences

27) Has difficulties saying what he/she wants to say

28) Unable to come up with or invent as many solutions to problems as others seem to do

29) Finds he/she is at a loss for words when he/she wants to explain something to others

30) Has trouble putting his/her thoughts down in writing as well or as quickly as others

31) Feels he/she is not as creative or inventive as others of his/her level of intelligence

32) In trying to accomplish goals or assignments, he/she finds that he/she is not able to think of

as many ways of doing things as others

33) Has trouble learning new or complex activities as well as others

34) Has difficulty explaining things in their proper order or sequence

35) Can't seem to get to the point of his/her explanations as quickly as others

36) Has trouble doing things in their proper order or sequence

37) Unable to "think on his/her feet" or respond as effectively as others to unexpected events

38) Is slower than others at solving problems he/she encounters in his/her daily life

39) Easily distracted by irrelevant events or thoughts when he/she must concentrate on

something

40) Not able to comprehend what he/she read as well as he/she should be able to do; has to

reread material to get its meaning

41) Cannot focus his/her attention on tasks or work as well as others

42) Easily confused

43) Can't seem to sustain his/her concentration on reading, paperwork, lectures, or work

44) Finds it hard to focus on what is important from what is not important when he/she does

things

45) Doesn’t seem to process information as quickly or as accurately as others

46) Finds it difficult to tolerate waiting; impatient

47) Makes decisions impulsively

48) Unable to inhibit his/her reactions or responses to events or others

49) Has difficulty stopping his/her activities or behavior when he/she should do so.

50) Has difficulty changing his/her behavior when he/she is given feedback about my mistakes.

119

51) Makes impulsive comments to others.

52) Likely to do things without considering the consequences for doing them.

53) Changes his/her plans at the last minute on a whim or last minute impulse.

54) Fails to consider past relevant events or past personal experiences before responding to

situations (Acts without thinking).

55) Not aware of things he/she says or does.

56) Has difficulty being objective about things that affect him/her.

57) Finds it hard to take other people's perspectives about a problem or situation.

58) Doesn’t think or talk things over with self before doing something.

59) Trouble following the rules in a situation.

60) More likely to drive a motor vehicle much faster than others (Excessive speeding).

61) Has a low tolerance for frustrating situations

62) Cannot inhibit his/her emotions as well as others.

63) Doesn’t look ahead and think about what the future outcomes will be before he/she does

something (Doesn’t use his/her foresight).

64) Engages in risk taking activities more than others are likely to do.

65) Likely to take short cuts in his/her work and not do all that he/she is supposed to do.

66) Likely to skip out on work early if his/her work is boring to do.

67) Does not put as much effort into his/her work as he/she should or than others are able to do.

68) Others tell his/her that he/she is lazy or unmotivated.

69) Has to depend on others to help them get their work done.

70) Things must have an immediate payoff for his/her or he/she does not seem to get them done.

71) Has difficulty resisting the urge to do something fun or more interesting when he/she is

supposed to be working.

72) Inconsistent in the quality or quantity of his/her work performance.

73) Unable to work as well as others without supervision or frequent instruction.

74) Does not have the willpower or determination that others seem to have.

75) Is not able to work toward longer term or delayed rewards as well as others.

76) Cannot resist doing things that produce immediate rewards, even if those things are not good

for him/her in the long run.

77) Quick to get angry or become upset.

120

78) Overreact emotionally.

79) Easily excitable.

80) Unable to inhibit showing strong negative or positive emotions.

81) Has trouble calming self down once he/she is emotionally upset.

82) Cannot seem to regain emotional control and become more reasonable once he/she is

emotional.

83) Cannot seem to distract self away from whatever is upsetting him/her emotionally to help

calm self down. Can't refocus his/her mind to a more positive framework.

84) Unable to manage his/her emotions in order to accomplish his/her goals successfully or get

along well with others.

85) Remains emotional or upset longer than others.

86) Find it difficult to walk away from emotionally upsetting encounters with others or leave

situations in which he/she has become very emotional.

87) Cannot re-channel or redirect his/her emotions into more positive ways or outlets when

he/she gets upset.

88) Is not able to evaluate an emotionally upsetting event more objectively.

89) Cannot redefine negative events into more positive viewpoints when he/she feels strong

emotions.

121

REFERENCES

Alderman, N., Burgess, P. W., Kight, C., & Henman, C. (2003). Ecological validity of a simplified version of the multiple errands shipping test. Journal of the International

Neuropsychological Society, 9, 31-44 The American Educational Research Association, the American Psychological Association, and

the National Council on Measurement in Education (1999). Standards for education and

psychological testing. Washington D.C.: American Educational Research Association.

American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). Washington, DC: Author.

American Psychiatric Association. (2000). Diagnostic and statistical manual of mental

disorders-Text Revision (4th ed.). Washington, DC: Author. Anderson, P. (2002). Assessment and development of executive function (EF) during childhood.

Neuropsychology, 8, 71-82. doi: 0929-7049/02/0802-071 Anderson, V., Anderson, P., Northman, E., J., & Mikiewicz, O. (2002). Relationships between

cognitive and behavioral measures of executive functioning in children with brain disease. Child Neuropsychology, 8(4), 231-240. doi: 0929-7049/02/0804-231

Antshel, K. M., Faraone, S. V., Maglione, K., Doyle, A. E., Seidman, L. J., & Biderman, J.

(2010). Executive functioning in high-IQ adults with ADHD. Psychological Medicine,

40, 1909-1918. doi:10.1017/S0033291709992273 Antshel, K. M., Faraone, S. V., Maglione, K….Biederman, J. (2009). Is adult attention deficit

hyperactivity disorder a valid diagnosis in the presence of high IQ? Psycholgical

Medcine, 39(8), 1325-1335. doi: 10.1017/S0033291708004959 Ardila, A., Ostrosky-Solis, F., Rosselli, M., & Gomez, C. (2000). Age related cognitive decline

during normal aging: The complex effect of education. Archives of Clinical

Neuropsychology, 15, 495–514. Barkley, R. A. (1997a). ADHD and the nature of self-control. New York: Guilford Press. Barkley, R. A. (1997b). Inhibition, sustained attention, and executive functions: constructing a

unifying theory of ADHD. Psychological Bulletin, 121, 65–94. N doi: 10.1037/0033-2909.121.1.65

Barkley, R. A. (2011a). Barkley Adult ADHD Rating Scale (BAARS-IV): The Guilford Press. Barkley, R. A. (2011b). Barkley Deficits in Executive Functioning Scale (BDEFS). New York:

Guilford.

122

Barkley, R. A. (2011c). Executive functioning and self-regulation in Adults with ADHD: Nature, Assessment and Treatment. Symposium at Children and Adults with ADHD (CHADD) Conference, Orlando, FL.

Barkley, R. A. (2012). Executive functions: What they are, how they work, and why they evolved.

New York: Guilford Press. Barkley, R. A., & Fischer, M. (2010). The unique contribution of emotional impulsiveness to

impairments in major life activities in hyperactive children as adults. Journal of the

American Academy of Child and Adolescent Psychiatry, 49(5), 503-513. Barkley, R.A., & Murphy, K. R. (2010). Impairment in occupational functioning and adult

ADHD: The predictive utility of executive function (EF) ratings versus EF tests. Archives

of Clinical Neuropsychology, 25, 157–173. Barkley, R. A., Murphy, K. R., & Fischer, M. (2008). ADHD in adults: What the science says.

New York: Guildford Press. Baxter, M. G. & Murray, E. A. (2002). The amygdala and reward. National Review of

Neuroscience, 3, 563-573. Biederman, J., & Faraone, S. V. (2006). The effects of Attention-Deficit/Hyperactivity Disorder

on employment and household income. Medscape General Medicine, 8, 1-6. Biederman, J., Faraone, S.V., Spencer, T.J., Mick, E., Monuteaux, M.C., Aleardi, M. (2006).

Functional impairments in adults with self-reports of diagnosed ADHD: a controlled study of 1001 adults in the community. Journal of Clinical Psychiatry, 67, 524–540.

Biederman, J., Faraone, S. V., Spencer, T., Wilens, T., Norman, D., Lapey, K. A., …Doyle, A.

(1993). Patterns of psychiatric comorbidity, cognition, and psychosocial functioning in adults with attention deficit hyperactivity disorder. American Journal of Psychiatry 150, 1792–1798.

Biederman, J., Mick, E., Faraone, S.V. (2000). Age-dependent decline of symptoms of attention

deficit hyperactivity disorder: impact of remission definition and symptom type. American Journal of Psychiatry, 157, 816–818.

Biederman, J., Petty, C. R., Fried, R., Doyle, A. E., Spencerm T….. Seidman, L., (2007).

Stability of executive function deficits into young adult years: A prospective longitudinal follow-up study of grown up males with ADHD. Acta Psychiatrica Scandinavia, 116, 129-136.

Biederman, J., Spencer, T., & Wilens, T. (2004). Evidence-based pharmacotherapy for

attention-deficit hyperactivity disorder. International Journal of

Neuropsychopharmacology, 7, 77-97.

123

Biederman, J., Spencer, T., Wilens, T., Prince, J. B., & Faraone, S. V. (2006). Treatment of ADHD with stimulant medications: Response to Nissen perspective in The New England Journal of Medicine. Journal of the American Academy of Child and Adolescent

Psychiatric, 45, 10, 1147-1150. Booksh, R. L., Pella, R. D., Singh, A. N., and Gouvier, W. D. (2010). Ability of college students

to simulate ADHD on objective measures of attention. Journal of Attention Disorder, 13, 325-338.

Boonstra, A. M., Oosterlaan, J., Sergeant, J. A., & Buitelaar, J. K. (2005). Executive functioning

in adult ADHD: A meta-analytic review. Psychological Medicine, 35, 1097–1108. Brown, T. A. (2006). Confirmatory factor analysis for applied research. Sage Publication: New

York, NY. Brown, T. E. (2008). ADD/ADHD and impaired executive function in clinical practice. Current

Psychiatry Reports (10), 51 407-411. Brown, B.M. and Cutik, R. (1993). Alternative ways of assessing model fit. In K. A. Bollen & J. S.

Long (Eds). Testing structural equation models (pp. 136-162). Newbury Park, CA: Sage.

Burgess, P. W., Alderman, N., Evans, J., Emsile, H., & Wilson, B. A. (1998.). The ecological validity of tests of executive functioning. Journal of the International

Neuropsychological Society, 4, 457-558. Burgess, P. W., & Shallice, T. (1996). Response suppression, initiation and strategy use

following frontal lobe lesions. Neuropsychologia, 34, 263–272. Burgess, P. W., & Shallice, T. (1997). The Hayling and Brixton Tests. Bury St. Edmunds (UK):

Thames Valley Test Company. Carmines, E. & Zeller, R. (1979). Reliability and validity assessment. Thousand Oaks, CA: Sage

Publications, Inc.

Castellanos, F.X., Sonuga-Barke, E.J.S., Milham, M.P, & Tannock, R. (2006). Characterizing cognition in ADHD: Beyond executive dysfunction. Trends in Cognitive Sciences, 10,

117-123. doi: 10.1016/j.tics.2006.01.011 Chaytor, N. S (2004). Improving the ecological validity of executive functioning assessment.

(Unpublished doctoral dissertation). Washington State University, Washington. Chaytor, N., Schmitter-Edgecombe, M., & Burr, R. (2006). Improving the ecological validity of

executive functioning assessment. Archives of Clinical Neuropsychology, 21, 217-227. doi:10.1016/j.acn.2005.12.002

124

Chelune, G. J., Ferguson, W., Koon, R., & Dickey, T. O. (1996). Frontal lobe disinhibition in attention deficit disorder. Child Psychiatry and Human Development, 16, 221-234.

Clark, C., Prior, M., & Kinsella, G. (2000). Do executive function deficits differentiate between

adolescents with ADHD and oppositional defiant/conduct disorder? A neuropsychological study using the six elements test and Hayling sentence completion test. Journal of Abnormal Child Psychology, 28(5), 403-414.

Comrey, A.L., & Lee, H.B. (1992). A first course in factor analysis. (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum Associates, Publishers. Conners, C. K. (1995). Conners’ Continuous Performance Test. Toronto: Multi-Health Systems. Cortese, S., Kelly, C., Chabernaud, C., Proal, E., Di Martino, A., Milham, M. P., & Castellanos,

F. Z. (2012). Towards systems neuroscience of ADHD: A meta-analysis if 55 fMRI studies. American Journal of Psychiatry, 169, 1038-1055.

Dawkins, R. (1982). The extended phenotype: The long reach of the gene. New York: Oxford

University Press. Delis, D.C., Kaplan, E., & Kramer, J.H. (2001). The Delis Kaplan Executive Function System:

Examiner’s Manual. San Antonio: The Psychological Corporation. Denckla, M. B. (1996). A theory and model of executive function: A neuropsychological

perspective. In G. Lyon, & N. Krasnegor (Eds.), Attention, memory and executive

function (pp 263-278). Maryland: Paul Brooks. Doehnert, M., Brandeis, D., Schneider, G., Drechsler, R., & Steinhausen, H. C. (2012). A

neurophysiological marker of impaired preparation in an 11-year follow –up study of attention-deficit/hyperactivity disorder (ADHD). The Journal of Child Psychology and

Psychiatry. doi: 10.1111/j.1469-7610.2012.02572.x. Duncan, J., Emslie, H., Williams, P., Johnson, R., & Freer, C. (1996). Intelligence and the frontal

lobes: The organization of goal-directed behavior. Cognitive Psychology, 30, 257–303. DuPaul, G. J., Schaughency, E. A., Weyandt, L. I., Tripp, G., Kiesner, J., Ota, K., et al. (2001).

Self-report of ADHD symptoms in university students: Gender and cross-national prevalence. Journal of Learning Disabilities, 34, 370-379.

DuPaul, G. J., Weyandt, L. L., O’ Dell, S. M., & Varejao, M. (2009). College students with

ADHD: Current status and future directions. Journal of Attention Disorder, 13, 234-250. Eslinger, P. J. (1996). Conceptualizing, describing, and measuring components of executive

function: A summary. In G. R. Lyon & N. A. Krasnegor (Eds.), Attention, memory, and

executive function (pp. 367-395). Baltimore: Brookes.

125

Faraone, S. V., Wilens, T.E., Petty, C., Antshel, K., Spencer, T., Biederman, J. (2007). Substance use among ADHD adults: Implications of late onset and subthreshold diagnoses. American Journal on Addictions 16 (Suppl. 1), 24–32, quiz 33–34.

Faraone, S. V., Biederman, J., Spencer, T., Mick, E., Murray, K., Petty, C…Monuteaux, M. C.

(2006). Diagnosing adult attention deficit hyperactivity disorder: Are late onset and sub-threshold diagnoses valid? American Journal of Psychiatry, 163, 1720-1729.

Faul, F., Erdfelder, E., Buchner, A., & Lang, A.G. (2013). Statistical power analysis using

G*Power 3.1.6: Tests for correlation and regression analyses. Behavior Research

Methods, 41, 1149-1160. Fennerty, D., Lambert, C., Majsterek, D. (2000). Behavior Rating Scales: An Analysis. Eric

Document #ED442042. Fink, A., & Litwin, M. (1995). How to measure survey reliability and validity. Sage Publications

Inc.: Thousand Oaks, California. Fischer, M., Newby, R., & Gordon, M. (1995). Who are the false negatives on continuous

performance tests? Journal of Clinical Child Psychology, 24(4), 427–433. Floyd, F. J. & Widaman, K. F. (1995). Factor Analysis in the development and refinement of

clinical assessment instruments. Psychological Assessment, 7, 286-299. Frazier, T. W., Youngstrom, E. A., Glutting, J. J., & Watkins, M. W. (2007). ADHD and

achievement: Meta-analysis of the child, adolescent, and adult literatures and a concomitant study with college students. Journal of Learning Disabilities, 40(1), 49-65.

Forbes, G. B. (1998). Clinical utility of the test of variables of attention (TOVA) in the diagnosis

of attention deficit/hyperactivity disorder. Journal of clinical psychology, 54(4), 461-476.

Fuster, J.M. (1997). The prefrontal cortex-anatomy physiology, and neuropsychology of the

frontal lobe (3rd ed.). Philadelphia: Lippincott-Raven. Gioia, G. A., Isquith, P. K., Guys, S. C., & Kenworthy, L. (2000). BREIF: Behavior Rating

Inventory of Executive Functions- Professional Manual. Odessa, FL: Psychological Assessment Resources.

Glutting, J. J., Youngstrom, E. A., & Watkins, M. W. (2005). ADHD and college students:

Exploratory and confirmatory factor structures with student and parent data. Psychological Assessment, 17, 44-55.

Gonon, F. (2008). The dopaminergic hypothesis of attention-deficit/hyperactivity disorder needs

re-examining. Trends in Neurosciences, 32(1), 1-8. doi:10.1016/j.tins.2008.09.010

126

Goodwin, J. (2010). Research in psychology: methods and design. John Wiley & Sons: Hoboken, NJ.

Gordon, M. (1983). The Gordon Diagnostic System. DeWitt, NY: Gordon Systems. Gordon, M., & Keiser, S. (1998). Accommodations in higher education under the Americans

With Disabilities Act (ADA): A no-nonsense guide for clinicians, educators,

administrators, and lawyers. DeWitt, NY: GSI Publications. Grant, D., & Berg, E. A. (1948). Wisconsin Card Sorting Test. Los Angeles: Western Psychological Corporation.

Graham, J.W. (2009). Missing data analysis: Making it work in the real world. Annual Review of

Psychology, 60, 549-576. doi: 10.1146/annurev.psych.58.110405.085530 Greenberg, L. M. (1991). TOVA tm interpretation manual. Minneapolis, MN: Grenwald-Mayes, G. (2002). Relationship between current quality of life and family of origin

dynamics for college students with attention-Deficit/Hyperactivity disorder. Journal of

Attention Disorders, 5, 211-222. Grondzinsky, G. M., & Diamond, R. (1992). Frontal lobe functioning in boys with attention-

deficit/hyperactivity disorder. Developmental Neuropsychology, 8, 427-445. Gulliksen, H. (1950). Intrinsic validity. American Psychologist, 5, 511-517. Heaton, R. (1981). Wisconsin card sorting test: Manual. Odessa: Psychological Assessment

Resources, Inc. Heiligenstein, E., Guenther, G., Levy, A., Savino, F., & Fulwiler, J. (1999). Psychological and

academic functioning in college students with attention deficit hyperactivity disorder. Journal of American College Health, 47, 181-185.

Heiman, G. W. (2002). Research methods in Psychology. Boston, MA: Houghton Mifflin Co. Hervey, A. S., Epstein, J. N., & Curry, J. F. (2004). Neuropsychology of adults with attention-

deficit/hyperactivity disorder: a meta-analytic review. Neuropsychology, 18, 495-503. Hoffman, W., Friese, M., Schmeichel, B. J., & Baddeley, A. D. (2011). Working memory and

self-regulation. In K. D. Vohs & R. F. Baumeister (Eds.), Handbook of self-regulation:

Research, theory, and applications (2nd ed., pp. 41-63). New York: Guilford Press. Homack, S., Lee, D., & Riccio, C. A. (2005). Test review: Delis-Kaplan Executive Function

System. Journal of Clinical and Experimental Neuropsychology, 27, 599-609. Hood, A. B. & Johnson, R. W. (2006). Assessment in counseling: A guide to the use of

psychological assessment procedures (4th ed.). American Counseling Association: Alexandra, VA.

127

Hu, T., and Bender, P. M. (1999). Cutoff criteria for fit indices in covariance structure analysis: Conventional criteria verses new alternatives. Structure Equation Modeling, 6, 1-55.

Jackson, S. L. (2008). Research methods: a modular approach, Thomson Wadsworth: Belmont,

CA. Jung, R. E., Yeo, R. A., Chiulli, S. J., Sibbitt, W. L., & Brooks, W. M. (2000). Myth of

neuropsychology: Intelligence, neurometabolism, and cognitive ability. The Clinical

Neuropsychologist 14(4), 535-545. Kassubek, J., Juengling, F. D., Ecker, D., & Landwehrmeyer, G. B. (2005). Thalamic atrophy in

Huntington’s disease co-varies with cognitive performance: A morphometric MRI analysis. Cerebral Cortex, 15, 846–853.

Kessler, R. C., Heeringa, S., Lakoma, M. D., Petukhova, M., Rupp, A., Schoenbaum,

M…Zaslavsky, A. M., (2010). Individual and societal effects of mental disorders on earnings in the United States: Results from the national comorbidity survey replication. The American Journal of Psychiatry, 165, 703-711. doi:10.1176/appi.ajp.2008.08010126

Koechlin, E., Corrado, G., Pietrini, P., & Grafman, J. (2000). Dissociating the role of the medial

and lateral anterior prefrontal cortex in human planning. Proceedings of the National

Academy of Sciences, 97(13), 7651-7656. Korkman, M., Kirk, U., & Kemp, S. (1998). NEPSY: A developmental neuropsychological

assessment. San Antonio, TX: The Psychological Corporation. Levin, H., & Hanten, G. (2005). Executive functions after traumatic brain injury in children.

Pediatric Neurology, 33, 79-93. doi:10.1016/j.pediatrneurol.2005.02.002 Lewis, S. J., Dove, A., Robbins, T. W., Baker, R. A., & Owen, A. M. (2004). Striatal

contributions to working memory: A functional magnetic resonance imaging study in humans. European Journal of Neuroscience, 19, 755-760.

Lewis, R. F., Kelland, D. Z., Kupke, T. (1990). A normative study of the Repeatable Cognitive-

Perceptual-Motor Battery. Archives of Clinical Neuropsychology, 5, 201-214. Lezak, M. D., Howieson, D. B., Loring, D. W. (2004). Neuropsychological Assessment (4th ed.).

New York: Oxford University Press. Loo, S. K., Hale, T. S., Masion, J., Hanada, G., McGough, J. J., McCracken, J. T., & Smalley, S.

L. (2009). Cortical activity patterns in ADHD during arousal, activation and sustained attention. Neuropsychological, 47(10), 2114-2119. Doi: 10.1016/j.neuropsychologicaq.2009.04.03

Lowry, R. VassarStats: Website for Statistical Computation. (2013). Retrieved April 6, 2013,

from http://vassarstats.net/index.html

128

Luria, A. R. (1966). Higher cortical functions in man. New York: Basic Books. Mahone, E. M., Pillion, J. P., Hoffman, J., Hiemenz, J. R., & Denckla, M. B. (2002). Construct

validity of the Auditory Continuous Performance Test for Preschoolers. Developmental

Neuropsychology, 27, 11-33. Matarazzo, J. D., Wiens, A. N., Matarazzo, R. G., & Goldstein, S. G. (1974). Psychometric and

clinical test-retest reliability of the Halstead Impairment Index in a sample of healthy, young, normal men. Journal of Nervous and Mental Disease, 158, 37–49.

McCabe, S. E., Knight, J., Teter, C. J., & Wechsler, H. (2005). Nonmedical use of prescription

stimulants among US college students: Prevalence and correlates from a national survey. Addiction, 99, 96–106.

McGee, R. A., Clark, S. E., & Symons, D. K., (2000). Does the Conners’ Continuous

Performance Test aid in ADHD diagnosis? Journal of Abnormal Child Psychology,

28(5), 415-424. Meaux, J. B., Green, A., & Broussard, L. (2009). ADHD in the college student: a block in the

road. Journal of Psychiatric and Mental Health Nursing, 16, 248-256. Merrell, K. W. (1994). Assessment of behavioral, social, & emotional problems. New York:

Longman. Messick, S. (1989). Meaning and value in test validation: The science and ethics of assessment.

Educational Researcher, 18, 5-11. Mises, L. (1990). Human action: A treatise on economics (3rd ed.). San Francisco: Laissez-Faire

Brooks. (Original work published 1948). Miyake, A., Friedman, N., Emerson, M., Witzki, A., & Howerter, A. (2000). The unity and

diversity of executive functions and their contributions to complex “frontal lobe” tasks: A latent variable analysis. Cognitive Psychology, 41, 49–100. doi:10.1006/cogp.1999.0734

Monchi, O., Petrides, M., Strafella, A. P., Worsley, K. J., & Doyon, J. (2006). Functional role of

the basal ganglia in the planning and execution of actions. Annual Neurology, 59, 257-264.

Murphy, K. (2005). Psychosocial treatments for ADHD in teens and adults: A practice-friendly

review. Journal of Clinical Psychology, 61, 607–619. Murphy, K., Barkley, B., & Tracie, B. (2001). Executive functioning and olfactory in young

adults with attention deficit-hyperactivity disorder. Neuropsychology, 15(2), 211-220. doi: 10.1037/0894-4105.15.2.211

129

Myers, L. S., Gamst, G., & Guarino, A. J. (2006). Applied multivariate research: design and

interpretation. Sage Publications: Thousand Oaks, CA Niendam, T. A., Laird, A. R., Ray, K. L., Dean, M. Y., Glahn, D. C., & Carter, C. S. (2012). Meta-analytic evidence for a superordinate cognitive control network subserving diverse executive functions. Cognitive Affective Behavioral Neuroscience, 12, 241-268.

Nadeau, K. G. (2005). Career choices and workplace challenges for individuals with ADHD.

Journal of Clinical Psychology, 61, 549–563. Naglieri, J. A., & Goldstein, S. (2013). Comprehensive Executive Functioning Inventory. NY:

Multi-Health Systems Inc. Nigg, J. T., & Casey, B. J. (2005). An integrative theory of attention-deficit/hyperactivity

disorder based on the cognitive and affective neurosciences. Development and

Psychopathology, 17, 765-806. Norwalk, K., Norvilitis, J. M., & MacLean, M.G., (2008). ADHD symptomatology and its

relationship to factors associated with college adjustment. Journal of Attention Disorders, 1-8.

Oosterlaan, J., Scheres, A., & Sergeant, J. A. (2005). Which executive functioning deficits are

associated with AD/HD, ODD/CD and comorbid AD/HD+ODD/CD? Journal of

Abnormal Psychology, 33, 69-85. Prevatt, F., Li, H., Welles, T., Festa-Dreher, D., Yelland, S., and Lee., J. (2011). The

Academic Success Inventory for College Students: Scale development and practical implications for use with students. Journal of College Admission, 211,

26-31. Proctor, B. E., & Prevatt, F. (2009). Confirming the factor structure of attention-

deficit/hyperactivity disorder symptoms in college students using student and parent data. Journal of Learning Disabilities, 42, 250-259.

Ramirez, C. A., Rosen, L. A., Deffenbacher, J. L., Hurst, H., Nicoletta, C., Rosencranz, T. A., et

al. (1997). Anger and anger expression in adults with high ADHD symptoms. Journal of Attention Disorders, 2, 115-128.

Rapoport, J. L., Buchsbaum, M. S., Zahn, T. P., Ludlow, C., & Mikkelsen, E. J. (1978, February

3). Dextroamphetamine: Cognitive and behavioral effects in normal prepubertal boys. Science, 199, 560–563.

Reaser, A., Prevatt, F., Petscher, Y., & Proctor, B. (2007). The learning and study strategies of

college students with ADHD. Psychology in the Schools, 44, 627–638. Reis, H. T., & Judd, C. M. (Eds.). (2000). Handbook of research methods in social and

personality psychology. Cambridge University Press.

130

Reitan, R. M., & Wolfson, D. (1985). The Halstead–Reitan Neuropsycholgical Test Battery:

Therapy and clinical interpretation. Tucson, AZ: Neuropsychological Press.

Rey, A. (1941). L’examen psychologique dans les cas d’encephalopathie traumatique. Arch.

Psychol. 28, 286–340.

Roth, R. M., Isquith, P. K., & Gioia, G. A. (2005). Behavior Rating Inventory of Executive

Function- Adult Version. FL: Psychological Assessment Resources. Rottschy, C., Langer, R., Dogan, I., Reetz, K., Laird, A. R., Schulz, J. B…Eickhoff, S. B. (2012).

Modelling neural correlates of working memory: A coordinate-based meta-analysis. Neuroimage, 60, 830-846.

Rueda, M. R., Posner, M. L., & Rothbard, M. K. (2011). Attentional control and self-regulation.

In K. D. Vohs & R. F. Baumeister (Eds.), Handbook of self-regulation: Research, theory,

and applications (2nd ed., pp. 41-63). New York: Guilford Press. Salthouse, T. (1996). The processing-speed theory of adult age differences in cognition.

Psychological Review, 103, 403–428. Salthouse, T. (2005). Relations between cognitive abilities and measures of executive

functioning. Neuropsychology, 19, 532–545. Schmidt, M. (2003). Hit or miss? Insight into executive functions. Journal of the International

Neuropsychological Society, 9, 962–964. Seidman, L. J., Biederman, J., Valera, E. M., Monuteaux, M. C., Doyle, A. E., & Faraone, S. V.

(2006). Neuropsychological functioning in girls with attention-deficit/hyperactivity disorder with an without learning disabilities. Neuropsychology, 20, 166-177.

Shallice, T. & Evans, M. E. (1978). The involvement of the frontal lobes in cognitive estimation.

Cortex, 14, 294-303. Stavro, G. M., Ettenhofer, M. L., & Nigg, J. T. (2007). Executive functions and adaptive

functioning in young adults with attention-deficit/hyperactivity disorder. Journal of the

International Neuropsychological Society, 13, 324-334. Stroop, J. R. (1935). Studies of interference in serial verbal reactions. Journal of Experimental

Psychology, 18, 643–662. Stuss, D. T. & Alexander, M. P. (2000). Executive functions and the frontal lobes: a conceptual

view. Psychological Research, 63, 289-298.

131

Stuss, D. T., Alexander, M. P., Floden, D., Binns, M. A., Levin, B., & McIntosh, A. R. (2002). Fractionation and localization of distinct frontal lobe processes: Evidence from focal lesions in humans. In D. T. Stuss, & R. T. Knight (Eds.), Principles of frontal lobe

function (pp. 392-407). New York, NY: Oxford University Press. Thompson, J. M., Gray, J. M., Crawford, J. R., Hughes, J. H., Young, A. H., & Ferrier, I. N.

(2009). Differential deficit in executive control in euthymic bipolar disorder. Journal of

Abnormal Psychology, 118, 146-160. Trenerry, M., Crosson, B., Deboe, J., & Leber, W. (1989). Stroop Neuropsychological

Screening Test. Odessa, FL: Psychological Assessment Resources. Turnock, P. M., Rosen, L. A., & Kaminski, P. L. (1998). Academic coping in college students

self-reporting high and low symptoms of attention deficit hyperactivity disorder. Journal of College Student Development, 39, 484-493.

Tuttle, J., Scheurich, N., & Ranseen, J. D. (2007, May). Prevalence and attitudes concerning

ADHD diagnosis and use of stimulant medication in medical students. Paper presented at the annual meeting of the American Psychiatric Association, San Diego, CA.

Vriezen, E. R., & Pigott, S. E. (2002). The relationship between parental report on the BRIEF

and performance based measures of executive function in children with moderate to severe traumatic brain injury, Neuropsychology, 8, 296-303.

Wagner, D. D., & Heatherington, T. F. (2011). Giving in to temptation: The emerging cognitive

neuroscience of self-regulatory failure. In K. D. Vohs & R. F. Baumeister (Eds.), Handbook of self-regulation: Research, theory, and applications (2nd ed., pp. 41-63). New York: Guilford Press.

Wechsler, D. (2009). Wechsler adult intelligence scale, 4th ed. San Antonio, TX: The

Psychological Corporation. Welsh, M. C., & Pennington, B. F. (1988). Assessing frontal lobe functioning in children: Views

from developmental psychology. Developmental Neuropsychology, 4, 199-230. Weyandt, L. L. (2009). Executive Functioning and Attention Deficit Hyperactivity Disorder.

The ADHD Report, 17(6), 1-7. Weyandt, L., Linterman, I., & Rice, J. (1995). Reported prevalence of attentional difficulties in a

general sample of college students. Journal of Psychopathology and Behavioral

Assessment, 17, 293-304. White, B. P., Becker-Blease, K. A., & Grace-Bishop, K. (2006). Stimulant medication use,

misuse, and abuse in an undergraduate and graduate student sample. Journal of American

College Health, 54, 261–268.

132

Willcutt, E. G., Doyle, A. E., Nigg, J. T., Faraone, S. V., & Pennington, B. F. (2005). Validity of the executive function theory of attention-deficit/hyperactivity disorder: A meta-analytic review. Biological Psychiatry, 57, 1336-1346.

Wolf, L. E. (2001). College students with ADHD and other hidden disabilities: Outcomes and

interventions. In J. Wasserstein, L. E. Wood, R. L., & Liossi, C. (2006). The ecological validity of executive function tests in a

severely brain injured sample. Archives of Clinical Neuropsychology, 21, 429-437.

Woodcock, R.W., McGrew, K.S., &Mather, N. (2001a). Woodcock-Johnson III Tests of

Achievement. Itasca, IL: Riverside. Woodcock, R.W., McGrew, K.S., & Mather, N. (2001b). Woodcock-Johnson III Tests of

Cognitive Abilities. Itasca, IL: Riverside. Zentall, S. S. (2005). Theory and evidence-based strategies for children with attentional

problems. Psychology in the Schools, 42(8), 821-836. Doi: 10.1002/pits.20114 Zentall, S. S., & Zentall, T. R. (1983). Optimal simulation: A model of disordered activity and

performance in normal and deviant children. Psychological Bulletin, 94, 446-471.

133

BIOGRAPHICAL SKETCH

Theodora Passinos Coffman completed her Bachelors of Science degree in Psychology in

2000 at Clemson University in Clemson, South Carolina. She pursued her PhD at the Combined

Doctoral Program in Counseling Psychology and School Psychology at Florida State University

in Tallahassee, Florida. Currently, Theodora is completing an APA-accredited pre-doctoral

psychology internship at GeoCare LLC/South Florida State Hospital in Pembroke Pines, Florida.

She will remain at GeoCare LLC/South Florida State Hospital for her post-doctoral fellowship

once her internship is completed.