PROGRAM OUTCOME ASSESSMENT REPORT … outcome assessment report master of arts in clinical mental health counseling department of behavioral sciences southeastern oklahoma state university

PROGRAM OUTCOME ASSESSMENT REPORT

MASTER OF ARTS IN CLINICAL MENTAL HEALTH COUNSELING

DEPARTMENT OF BEHAVIORAL SCIENCES

SOUTHEASTERN OKLAHOMA STATE UNIVERSITY

FALL 2011 Southeastern Oklahoma State University, in compliance with Title VI and VII of the Civil Rights Act of 1964, Executive Order 11246 as amended, Title IX of the Education Amendments of 1972, Americans with Disabilities Act of 1990, and other federal laws and regulations, does not discriminate on the basis of race, color, national origin, sex, age, religion, disability, or status as a veteran in any of its policies, practices, or procedures. This includes but is not limited to admissions, employment, financial aid, and educational services.

Clinical Mental Health Counseling POAR (Fall 2011)—Page 2

Table of Contents Section of Report: Page Number Table of Contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 I. Section A: Department and Degree Program . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 II. Section B: Primary Author, Semesters Covered, Date Filed . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 III. Section C: Program Mission Statement and Goals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 A. Program Mission Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 B. Program Goals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 C. Characteristics of Program Applicants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 IV. Section D: Program Learning Outcomes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

A. Learning Outcome #1: CMHC Orientation and Ethical Practice . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 B. Learning Outcome #2: Social and Cultural Diversity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 C. Learning Outcome #3: Human Growth and Development . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 D. Learning Outcome #4: Career Counseling and Development . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 E. Learning Outcome #5: Helping Relationships . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 F. Learning Outcome #6: Group Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 G. Learning Outcome #7: Counseling Assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 H. Learning Outcome #8: Research and Program Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

V. Learning Outcome Assessment Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

A. National Counselor Examination for Licensure and Certification (NCE) . . . . . .. . . . . . . . . . . . . . . . 7 1. How Many Students Participated in the Assessment? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 2. How Were Students Selected to Participate in the Assessment? . . . . . . . . . . . . . . . . . . . . . . . . . . 8 3. What was the Make Up of the Sample? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 4. What Work was Evaluated? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 5. How was the Assessment Conducted? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 6. Commitment to Ongoing Use of this Assessment Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 7. Results of the Assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 8. Analysis and Interpretation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 9. Program Modifications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

B. Counselor Preparation Comprehensive Examination (CPCE) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

1. How Many Students Participated in the Assessment? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 2. How Were Students Selected to Participate in the Assessment? . . . . . . . . . . . . . . . . . . . . . . . . . 12 3. What was the Make Up of the Sample? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 4. What Work was Evaluated? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 5. How was the Assessment Conducted? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 6. Commitment to Ongoing Use of this Assessment Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 7. Results of the Assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 8. Analysis and Interpretation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 9. Program Modifications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18


C. Student Performance Data Regarding National Council for Accreditation of Teacher Education (NCATE) Advanced Certificate Portfolio Artifacts Directly Correlated with the Eight Core Council for Accreditation of Counseling and Related Educational Programs (CACREP) Training Areas . . . . . 19

1. How Many Students Participated in the Assessment? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 2. How Were Students Selected to Participate in the Assessment? . . . . . . . . . . . . . . . . . . . . . . . . . 19 3. What was the Make Up of the Sample? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 4. What Work was Evaluated? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 5. How was the Assessment Conducted? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 6. Commitment to Ongoing Use of this Assessment Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 7. Results of the Assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 8. Analysis and Interpretation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 9. Program Modifications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

D. Program Alumni Survey and Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 E. Employer Survey and Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 VI. Strengths and Weaknesses of the Program . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 VII. Program Modifications in the Past Five (5) Years . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 VIII. IETV Courses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 IX. Departmental Contributions to This Report . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 X. Relevant Constituents and Stakeholders of This Report . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 XI. Methods of Sharing Data from This Report with Identified Constituents and Stakeholders . . . . . . . . . . . 34 XII. Ongoing Commitment to Continue the Assessment Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34


I. SECTION A (DEPARTMENT AND DEGREE PROGRAM):

A.) Department: Department of Behavioral Sciences

B.) Degree Program: Master of Arts Degree in Clinical Mental Health Counseling II. SECTION B (AUTHOR, SEMESTERS COVERED, AND DATE FILED):

A. Primary Report Author: Kimberly Donovan, Ph.D., LPC, NCC, ACS, Associate Professor of Counseling and Clinical Mental Health Counseling Program Coordinator

B. Semesters Covered: Fall 2010, Spring 2011, Summer 2011 C. Date Filed: September 30, 2011

III. SECTION C (PROGRAM MISSION STATEMENT AND GOALS):

A. PROGRAM MISSION STATEMENT:

The Master of Arts Degree Program in Clinical Mental Health Counseling provides an environment of academic excellence designed to prepare and empower graduate students seeking entrance into the field of clinical mental health counseling. The goal of this training program is to afford students the training and skills necessary to become competent, professional, and ethical in the delivery of clinical mental health counseling services. The program is based on the wellness model of mental health service delivery and prepares graduates to act in the best interest of the clients and general public they serve. By having access to cutting edge educational experiences, skilled clinical supervision, and a rigorous and nationally-recognized academic program of study, students are afforded the opportunity to acquire a comprehensive body of knowledge, professional skills, ethical foundations, and cultural competencies. This carefully structured training program prepares students for success, responsible citizenship, and lifelong learning in the field of clinical mental health counseling.

B. PROGRAM GOALS:

The graduate program in Clinical Mental Health Counseling bases its goals on those stipulated by the most recent edition of training standards promulgated by the Council for Accreditation of Counseling and Related Education Programs (CACREP). Thus, the ultimate focus of the program is to provide evidence of an attitude and philosophy of excellence.

The Clinical Mental Health Counseling program is designed to provide the appropriate counselor training and perspective necessary for graduates to function effectively in a variety of clinical mental health settings. The Master of Arts (MA) Degree in Clinical Mental Health Counseling is designed to prepare individuals for careers in community-based clinical mental health counseling. Correspondingly, the program offers an opportunity for the student to specialize in the treatment aspects of counseling. Graduates may work in a variety of settings including university counseling centers, mental health centers, inpatient psychiatric units, alcohol and drug treatment centers, correctional facilities, private/independent practice, and a variety of other clinical counseling settings.

The Clinical Mental Health Counseling graduate program is presently in the process of applying for national specialty program accreditation as a Clinical Mental Health Counseling (CMHC) CACREP training program. Thus, the goals are closely aligned with the objectives outlined by CACREP. These objectives require common core curricular experiences and demonstrated knowledge in each of the eight common core curricular areas required of all students in the program. The eight (8) common core counseling curricular


areas outlined by the 2009 CACREP Accreditation Standards include the following (which correspondingly represent the first eight (8) learning objectives outlined below): (a) Clinical Mental Health Counseling Orientation and Ethical Practice; (b) Social and Cultural Diversity; (c) Human Growth and Development; (d) Career Counseling and Development; (e) Helping Relationships; (f) Group Work; (g) Counseling Assessment; and, (h) Research and Program Development. C. CHARACTERISTICS OF PROGRAM APPLICANTS: Program applicants are reviewed for admission by the Graduate Counseling Coordinating Committee (GCCC) three times a year (November, April, and August). During the 2010-2011 academic year, 18 applicants were reviewed for admission to the CMHC program. Fourteen of the 18 applicants met the admission criteria (acceptance rate of 78%). The average age of applicants was 28.44 years with a median age of 24. Seventy-three percent of the applicants were female and 23% were male. The majority of applicants held undergraduate degrees in psychology, however, majors also consisted of sociology, criminal justice, elementary education, music, Spanish, and graphic design. The ethnicity of the applicants consisted of the following: 55% Caucasian, 28% Native American and Caucasian, and 17% Native American. The median undergraduate GPA was 3.33 and the average GRE score (combined Verbal Reasoning and Quantitative Reasoning scores) was 900.

IV. SECTION D (PROGRAM LEARNING OUTCOMES). THE FOLLOWING EIGHT (8) LEARNING

OUTCOMES ARE BASED SPECIFICALLY UPON THOSE REQUIRED OF THE 2009 CACREP CORE TRAINING STANDARDS. A COMPREHENSIVE DEFINITION OF EACH LEARNING OUTCOME IS PROVIDED:

A. Learning Outcome #1: Clinical Mental Health Counseling Orientation and Ethical Practice: This

learning outcome is defined, based upon CACREP Standards, as the student demonstrating a knowledge of the following aspects of professional functioning of the clinical mental health counselor: (a) history and philosophy of the counseling profession; (b) professional roles, functions, and relationships with other human service providers, including strategies for interagency/ interorganization collaboration and communications; (c) counselors’ roles and responsibilities as members of an interdisciplinary emergency management response team during a local, regional, or national crisis, disaster or other trauma-causing event; (d) self-care strategies appropriate to the counselor role; (e) counseling supervision models, practices, and processes; (f) professional organizations, including membership benefits, activities, services to members, and current issues; (g) professional credentialing, including certification, licensure, and accreditation practices and standards, and the effects of public policy on these issues; (h) the role and process of the professional counselor advocating on behalf of the profession; (i) advocacy processes needed to address institutional and social barriers that impede access, equity, and success for clients; and (j) ethical standards of professional organizations and credentialing bodies, and applications of ethical and legal considerations in professional counseling.

B. Learning Outcome #2: Social and Cultural Diversity: This learning outcome is defined, based upon

CACREP Standards, as the student demonstrating a knowledge of the cultural context of relationships, issues, and trends in a multicultural society, including the following: (a) multicultural and pluralistic trends, including characteristics and concerns within and among diverse groups nationally and internationally; (b) attitudes, beliefs, understandings, and acculturative experiences, including specific experiential learning activities designed to foster students’ understanding of self and culturally diverse clients; (c) theories of multicultural counseling, identity development, and social justice; (d) individual, couple, family, group, and community strategies for working with and advocating for diverse populations, including multicultural competencies; (e) counselors’ roles in developing cultural self-awareness, promoting cultural social justice, advocacy and conflict resolution, and other culturally supported behaviors that promote optimal wellness and growth of the human spirit, mind, or body; and


(f) counselors’ roles in eliminating biases, prejudices, and processes of intentional and unintentional oppression and discrimination.

C. Learning Outcome #3: Human Growth and Development: This learning outcome is defined, based upon CACREP Standards, as the student demonstrating a knowledge of the nature and needs of persons at all developmental levels and in multicultural contexts, including the following: (a) theories of individual and family development and transitions across the life span; (b) theories of learning and personality development, including current understandings about neurobiological behavior; (c) effects of crises, disasters, and other trauma-causing events on persons of all ages; (d) theories and models of individual, cultural, couple, family, and community resilience; (e) a general framework for understanding exceptional abilities and strategies for differentiated interventions; (f) human behavior, including an understanding of developmental crises, disability, psychopathology, and situational and environmental factors that affect both normal and abnormal behavior; (g) theories and etiology of addictions and addictive behaviors, including strategies for prevention, intervention, and treatment; and (h) theories for facilitating optimal development and wellness over the lifespan.

D. Learning Outcome #4: Career Counseling and Development: This learning outcome is defined,

based upon CACREP Standards, as the student demonstrating a knowledge of career development and related life factors, including the following: (a) career development theories and decision-making models; (b) career, avocational, educational, occupational and labor market information resources, and career information systems; (c) career development program planning, organization, implementation, administration, and evaluation; (d) interrelationships among and between work, family, and other life roles and factors, including the role of multicultural issues in career development; (e) career and educational planning, placement, follow-up, and evaluation; (f) assessment instruments and techniques relevant to career planning and decision making; and (g) career counseling processes, techniques, and resources, including those applicable to specific populations in a global economy.

E. Learning Outcome #5: Helping Relationships: This learning outcome is defined, based upon CACREP Standards, as the student demonstrating a knowledge of the counseling process in a multicultural society, including the following: (a) an orientation to wellness and prevention as desired counseling goals; (b) counselor characteristics and behaviors that influence helping processes; (c) essential interviewing and counseling skills; (d) counseling theories that provide the student with models to conceptualize client presentation and that help the student select appropriate counseling interventions. Students will be exposed to models of counseling that are consistent with current professional research and practice in the field so they begin to develop a personal model of counseling; (e) a systems perspective that provides an understanding of family and other systems theories and major models of family and related interventions; (f) a general framework for understanding and practicing consultation; and (g) crisis intervention and suicide prevention models, including the use of psychological first aid strategies.

F. Learning Outcome #6: Group Work: This learning outcome is defined, based upon CACREP

Standards, as the student demonstrating both a theoretical and experiential knowledge of group purpose, development, dynamics, theories, methods, skills, and other group approaches in a multicultural society, including the following: (a) principles of group dynamics, including group process components, developmental stage theories, group members’ roles and behaviors, and therapeutic factors of group work; (b) group leadership or facilitation styles and approaches, including characteristics of various types of group leaders and leadership styles; (c) theories of group counseling, including commonalities, distinguishing characteristics, and pertinent research and literature; (d) group counseling methods, including group counselor orientations and behaviors, appropriate selection criteria and methods, and methods of evaluation of effectiveness; and (e) direct experiences in which students participate as group members in a small group activity, approved by the program, for a minimum of 10 clock hours over the course of one academic term.


G. Learning Outcome #7: Counseling Assessment: This learning outcome is defined, based upon

CACREP Standards, as the student demonstrating a knowledge of individual and group approaches to assessment and evaluation in a multicultural society, including the following: (a) historical perspectives concerning the nature and meaning of assessment; (b) basic concepts of standardized and non-standardized testing and other assessment techniques, including norm-referenced and criterion-referenced assessment, environmental assessment, performance assessment, individual and group test and inventory methods, psychological testing, and behavioral observations; (c) statistical concepts, including scales of measurement, measures of central tendency, indices of variability, shapes and types of distributions, and correlations; (d) reliability (i.e., theory of measurement error, models of reliability, and the use of reliability information); (e) validity (i.e., evidence of validity, types of validity, and the relationship between reliability and validity); (f) social and cultural factors related to the assessment and evaluation of individuals, groups, and specific populations; and (g) ethical strategies for selecting, administering, and interpreting assessment and evaluation instruments and techniques in counseling.

H. Learning Outcome #8: Research and Program Evaluation: This learning outcome is defined, based

upon CACREP Standards, as the student demonstrating a knowledge of research methods, statistical analysis, needs assessment, and program evaluation, including all of the following: (a) the importance of research in advancing the counseling profession; (b) research methods such as qualitative, quantitative, single-case designs, action research, and outcome-based research; (c) statistical methods used in conducting research and program evaluation; (d) principles, models, and applications of needs assessment, program evaluation, and the use of findings to effect program modifications; (e) the use of research to inform evidence-based practice; and (f) ethical and culturally relevant strategies for interpreting and reporting the results of research and/or program evaluation studies.

V. LEARNING OUTCOME ASSESSMENT METHODS

A. National Counselor Examination for Certification and Licensure (NCE): The NCE is a nationally-

normed instrument developed and regulated by the National Board for Certified Counselors, Inc. (information available at http://www.nbcc.org). It covers the eight core training areas delineated by CACREP (directly correlated with Learning Objectives 1-8 above). The NCE has been adopted as the preeminent post-master’s licensure examination for professional counselors. It has been adopted by nearly all states, in addition to the District of Columbia, Guam, and Puerto Rico.

Any SOSU graduate seeking licensure as Licensed Professional Counselor (LPC) in the state of Oklahoma must earn a passing score on the NCE within five years of having his or her application for LPC Candidacy approved by the Oklahoma State Department of Health. LPC Candidacy is granted once an applicant has completed his or her master’s degree program and submitted all documents required for the post-master’s candidacy period. During the Spring of 2009, the program applied for and was granted approval by NBCC to participate as a Graduate Student Application for the National Certified Counselor (GSA-NCC) credential counselor training program (the first such designation in the state of Oklahoma). Application to the GSA-NCC program is entirely voluntary, and expenses are paid directly by students wishing to participate. One of the benefits of belonging to the GSA-NCC program is that our graduating students, as well as students who have graduated from the Clinical Mental Health Counseling program at SOSU in the previous six months, are eligible to apply for and take the NCE on campus as opposed to having to wait several years after graduation to take the exam in Oklahoma City or Tulsa. The on-campus administration is offered in October and April each year in a pencil-and-paper format. To maintain test security and administration standards, the exam is proctored by an individual outside of the Department of Behavioral Sciences. SOSU and national data for GSA-eligible candidates from the April 2011


administration are provided in this report. Not enough students applied for the GSA-NCC to allow for the administration of NCE in October of 2010. 1. How Many Students Participated in the Assessment? A total of nine current or former students

completed the NCE on campus in April, 2011. The reason for this small number was due to the fact that only students who were in their final semester or who had graduated from SOSU within the six months prior to the administration of the exam were eligible to take the on-campus administration of the NCE (see the GSA-NCC requirements discussed in the previous paragraph).

2. How Were Students Selected to Participate in the Assessment? As stated previously, individuals

applying for licensure through the Oklahoma State Department of Health, Division of Professional Counselor Licensing, must earn a passing grade on the NCE to obtain LPC licensure in the State of Oklahoma. In the Fall of 2010, current students were notified by the Program Coordinator of the Clinical Mental Health Counseling program’s participation in the GSA-NCC program. The Program Coordinator explained to the students the benefits of choosing to participate in this voluntary program. Each interested student completed the application and submitted the registration fees to the Program Coordinator who then mailed all registration documents to NBCC.

3. What was the Make Up of the Sample? The sample consisted of nine students and/or recent

graduates of the graduate program in Clinical Mental Health Counseling at SOSU. 4. What Work was Evaluated? The NCE is a 160-item, multiple-choice examination specifically

covering the eight learning objectives defined above (see Section IV). The examination also includes 40 field-test items being piloted for future versions of the NCE, rather than being included in the following scores. The maximum score on the NCE is 160 and a score of 100 or above is required to pass the exam. Varying numbers of questions are devoted to each of the eight CACREP core training areas as delineated in Table 1 below. The instrument holds strong validity and reliability and is considered the foremost examination of the didactic portions of the Clinical Mental Health Counseling Master’s Degree training curriculum for the purposes of state and national licensure and board certification. The NCE is offered nationally on GSA-eligible campuses in October and April of each year.

Table 1 Items on NCE Devoted to Each CACREP Core Area

CACREP Core Area Number of Items on NCE Percentage of Total Items Professional Orientation and Ethical Practice 29 18% Social and Cultural Diversity 11 7% Human Growth and Development 12 8% Career Development 20 12% Helping Relationships 36 23% Group Work 16 10% Assessment 20 12% Research and Program Evaluation 16 10%

5. How was the Assessment Conducted? The NCE was administered in a controlled and secure environment, via paper-and-pencil, in Room 103A of Morrison Hall on the SOSU campus in Durant, Oklahoma. National data and generic data for Southeastern examinees were provided to the Clinical Mental Health Counseling Program Coordinator for assessment, feedback, and program improvement purposes.


6. Commitment to Ongoing Use of this Assessment Method: Due to the usefulness of the data

obtained from the NCE assessment instrument, the Clinical Mental Health Counseling Master’s Degree Program is committed to continuing to use these data to assess program outcomes from this point forward.

7. Results of the Assessment: Seven of the nine students (78%) from SOSU who completed the NCE

in April of 2011 passed the examination. (This was their first attempt at passing the examination). The national pass rate for the same administration date and norming sample was 81%. No SOSU students were eligible to take the NCE in October of 2010.

NCE scores on the eight core CACREP training areas and total NCE scores for the GSA-NCC eligible students from SOSU and those of the GSA-NCC eligible national sample are provided below in Table 2. An analysis of the overall findings of this assessment tool is provided directly after Table 2.

Table 2 SOSU and National Results on the NCE (April 2011 administration)

CACREP Core Area SOSU Mean Score* National Mean Score** Professional Orientation and Ethical Practice 22.00 22.17 Social and Cultural Diversity 8.00 7.94 Human Growth and Development 6.44 7.73 Career Development 14.00 14.08 Helping Relationships 26.67 26.76 Group Work 11.00 11.10 Assessment 14.44 14.09 Research and Program Evaluation 9.67 10.27 Total 112.22 114.94 * n = 9 ** N = 1156

8. Analysis and Interpretation: It must first be stated that the relative small sample size of SOSU students/graduates limits the comparison and generalization of these data between SOSU student scores and national data. That being said, Table 2 indicates that SOSU students demonstrated a slightly lower pass rate (78%) than the national average (81%). This serves as further impetus for SOSU’s plans to seek CACREP accreditation of its Clinical Mental Health Counseling Program. In examining the specific learning objectives in conjunction with NCE results, it appears our students performed reasonably well in comparison with the national averages. SOSU students’ strongest performances related to the areas of Professional Orientation and Ethical Practice (Learning Objective #1), Social and Cultural Diversity (Learning Objective #2), Career Development (Learning Objective #4), Helping Relationships (Learning Objective #5), Group Work (Learning Objective #6), and Counseling Assessment (Learning Objective #7). The areas in which students could improve are Human Growth and Development (Learning Objective #3), and Research and Program Evaluation (Learning Objective #8). Since April 2010 was the first time SOSU was eligible to participate in the GSA-NCC program, no scores are represented for the October 2009 administration. The following tables compare the SOSU scores with the national averages for the April 2010 and April 2011 administrations (total score, Table 3, as well as a table for each content area, Tables 4-11). Although the program was


eligible to participate in the October 2010 administration, no SOSU students were eligible to take the exam on that administration date.

Table 3 Student Total Scores on the NCE - A comparison of SOSU and National Means

Semester School Year Covered Possible Points

Number of Test-Takers

SOSU Mean Score

National Mean Score

October

2009-2010

160 0 n/a n/a April 6 114.50 110.57 October

2010-2011

160 0 n/a n/a April 9 112.22 114.94

Table 4 Student Scores on “Professional Orientation and Ethical Practice” Category of the NCE in comparison with SOSU and National Means (in Direct Correlation with Learning Outcome #1)



SOSU Mean Score

National Mean Score

October

2009-2010


2010-2011

29 0 n/a n/a April 9 22.00 22.17

Table 5 Student Scores on “Social and Cultural Diversity” Category of the NCE in comparison with SOSU and National Means (in Direct Correlation with Learning Outcome #2)



SOSU Mean Score

National Mean Score

October

2009-2010


2010-2011

12 0 n/a n/a April 9 8.00 7.94

Table 6 Student Scores on “Human Growth and Development” Category of the NCE in comparison with SOSU and National Means (in Direct Correlation with Learning Outcome #3)



SOSU Mean Score

National Mean Score

October

2009-2010


2010-2011

11 0 n/a n/a April 9 6.44 7.73


Table 7 Student Scores on “Career Development” Category of the NCE in comparison with SOSU and National Means (in Direct Correlation with Learning Outcome #4)



SOSU Mean Score

National Mean Score

October

2009-2010


2010-2011

20 0 n/a n/a April 9 14.00 14.08

Table 8 Student Scores on “Helping Relationships” Category of the NCE in comparison with SOSU and National Means (in Direct Correlation with Learning Outcome #5)



SOSU Mean Score

National Mean Score

October

2009-2010


2010-2011

36 0 n/a n/a April 9 26.67 26.76

Table 9 Student Scores on “Group Work” Category of the NCE in comparison with SOSU and National Means (in Direct Correlation with Learning Outcome #6)



SOSU Mean Score

National Mean Score

October

2009-2010


2010-2011

16 0 n/a n/a April 9 11.00 11.10

Table 10 Student Scores on “Assessment” Category of the NCE in comparison with SOSU and National Means (in Direct Correlation with Learning Outcome #7)



SOSU Mean Score

National Mean Score

October

2009-2010


2010-2011

20 0 n/a n/a April 9 14.44 14.09


Table 11 Student Scores on “Research and Program Evaluation” Category of the NCE in comparison with SOSU and National Means (in Direct Correlation with Learning Outcome #8)



SOSU Mean Score

National Mean Score

October

2009-2010


2010-2011

16 0 n/a n/a April 9 9.67 10.27

9. Program Modifications: Clinical Mental Health Counseling faculty members by virtue of the CACREP Task Force, chaired by Dr. Daniel Weigel, completed major curricular revisions in conjunction with the upcoming CACREP accreditation self-study and site visit. All eight of the courses covering the learning objectives above were re-designed in direct correlation with objectives outlined in the recently published 2009 CACREP Standards. Particular attention was placed upon the areas of Helping Relationships (Learning Objective #5) and Counseling Assessment (Learning Objective #7) as these data indicated a need for greater improvement. As evidenced by the data above, it can be seen SOSU students improved their scores on these two areas during the April 2011 examination of the NCE.

B. Counselor Preparation Comprehensive Examination (CPCE): The CPCE, which is utilized by over

260 universities and colleges across the U.S., is designed to assess counseling students’ knowledge of counseling information viewed as important by counselor preparation programs. Like the NCE, the CPCE covers the eight core training areas delineated by CACREP (directly correlated with Learning Objectives 1-8 above). The CPCE, however, is not designed to serve as a licensure examination. Instead, it has been developed to assess student learning in graduate counselor training programs. In addition, according to the Center for Credentialing and Education, Inc. (CCE) Web site (http://www.cce-global.org), the CPCE: (a) allows Master’s program comprehensive exams to better meet psychometric standards; (b) gives programs an objective view of the knowledge level of their students; (c) allows programs to examine student functioning in various curricular areas; (d) promotes longitudinal self-study; (e) compares a program’s results to national data; (f) stimulates student integration of knowledge learned in separate courses; and (g) gives students comparative strength/weakness feedback. The Clinical Mental Health Counseling Master’s Degree Program has been administering this learner outcomes assessment examination since the Fall of 2002 as an exit exam. However, in compliance with the Institutional Assessment Committee requirements, only data from the past five school years are included in this report.

1. How Many Students Participated in the Assessment? A total of nine Clinical Mental Health

Counseling students in their final semester of graduate studies at SOSU completed the CPCE on either administration date (November, 2010 or February, 2011). This assessment tool is only offered twice each calendar year (fall and spring).

2. How Were Students Selected to Participate in the Assessment? All students who are in their

final semester in the program take the CPCE as a requisite for graduation. 3. What was the Make Up of the Sample? The sample of students for the timeframe of this

assessment report consisted of nine students in their last semester of graduate studies in the Clinical Mental Health Counseling graduate degree program at SOSU. Eight of the examinees were female and one was male.


4. What Work was Evaluated? The CPCE is a 160-item, multiple-choice examination specifically developed to assess the eight learning objectives defined above (see Section IV). One-hundred thirty-six of the items are used in the actual assessment, with 24 items being piloted for future versions of the examination. Therefore, 17 items are specifically devoted to each of the eight core CACREP training areas and, concurrently, to each of the eight learning objectives of this report. The instrument holds strong validity and reliability and is the most commonly used instrument to measure knowledge of the eight core CACREP training areas among students at or near the time of graduation from a graduate counseling master’s degree program. As mentioned above, the CPCE is presently utilized by more than 260 colleges and universities for this very purpose.

5. How was the Assessment Conducted? The CPCE is administered by the SOSU Clinical Mental

Health Counseling Program Coordinator (the primary author of this report). Standardized administration procedures are strictly followed. Once completed, results are immediately secured by the proctor and mailed to CCE for scoring. Shortly thereafter, CCE mails the results back to the Clinical Mental Health Counseling Program Coordinator. Results are used for two purposes: (a) inclusion in this program outcome assessment report and (b) delivery of feedback to students regarding their performance on the separate sections of the CPCE to assist them in studying for eventual completion of the NCE (Assessment Method #1).


obtained from the CPCE assessment instrument, the Clinical Mental Health Counseling Master’s Degree Program is committed to continuing to use the CPCE to assess program outcomes from this point forward. This is, of course, contingent upon the SOSU School of Graduate and University Studies continuing to fund this affordable national program outcome assessment measure. This demonstration of the university’s support of the Clinical Mental Health Counseling Graduate Program’s program assessment procedures over the past several years continues to be greatly appreciated.

7. Results of the Assessment: Results of the CPCE for the nine students completing the examination

during the prescribed assessment report timeframe, in addition to comparable data from the past five years, are included below in Tables 12-20:

Table 12 Student Total Scores on the CPCE – A comparison of SOSU and National Means



SOSU Mean Score

National Mean Score*

Fall

2006-2007

136 0 n/a n/a Spring 3 ** ** Fall

2007-2008

136 4 ** ** Spring 3 89.00 87.12 Fall

2008-2009

136 3 89.33 88.89 Spring 7 93.71 87.30 Fall

2009-2010

136 2 80.50 87.30 Spring 6 96.83 94.18 Fall

2010-2011

136 2 76.00 78.35 Spring 7 90.14 94.18 *National means vary as normative samples change on a triannual basis. ** These data unavailable to the current Clinical Mental Health Counseling Program Coordinator.


Table 13 Student Scores on “Professional Orientation and Ethical Practice” Content Area of the CPCE in comparison with SOSU and National Means (in Direct Correlation with Learning Outcome #1)



SOSU Mean Score


Fall

2006-2007

17 0 n/a n/a Spring 3 11.70 12.20 Fall

2007-2008

17 4 11.80 10.93 Spring 3 12.00 12.18 Fall

2008-2009

17 3 12.00 11.91 Spring 7 12.57 11.55 Fall

2009-2010

17 2 11.00 11.55 Spring 6 12.67 13.00 Fall

2010-2011


Table 14 Student Scores on “Social and Cultural Diversity” Content Area of the CPCE in comparison with SOSU and National Means (in Direct Correlation with Learning Outcome #2)



SOSU Mean Score


Fall

2006-2007

17 0 n/a n/a Spring 3 10.70 10.00 Fall

2007-2008

17 4 8.00 8.49 Spring 3 10.00 10.03 Fall

2008-2009

17 3 11.00 10.24 Spring 7 11.14 10.72 Fall

2009-2010

17 2 9.50 10.72 Spring 6 12.33 10.88 Fall

2010-2011



Table 15 Student Scores on “Human Growth and Development” Content Area of the CPCE in comparison with SOSU and National Means (in Direct Correlation with Learning Outcome #3)



SOSU Mean Score


Fall

2006-2007

17 0 n/a n/a Spring 3 12.00 11.40 Fall

2007-2008

17 4 6.50 8.28 Spring 3 10.70 11.40 Fall

2008-2009

17 3 10.33 12.14 Spring 7 11.71 11.45 Fall

2009-2010

17 2 9.50 11.45 Spring 6 13.17 12.75 Fall

2010-2011


Table 16 Student Scores on “Career Development” Content Area of the CPCE in comparison with SOSU and National Means (in Direct Correlation with Learning Outcome #4)



SOSU Mean Score


Fall

2006-2007

17 0 n/a n/a Spring 3 6.70 8.80 Fall

2007-2008

17 4 9.50 9.76 Spring 3 7.70 8.77 Fall

2008-2009

17 3 7.33 8.56 Spring 7 11.00 10.99 Fall

2009-2010

17 2 10.00 10.99 Spring 6 11.50 11.35 Fall

2010-2011



Table 17 Student Scores on “Helping Relationships” Content Area of the CPCE in comparison with SOSU and National Means (in Direct Correlation with Learning Outcome #5)



SOSU Mean Score


Fall

2006-2007

17 0 n/a n/a Spring 3 13.70 12.00 Fall

2007-2008

17 4 10.30 8.77 Spring 3 13.00 12.00 Fall

2008-2009

17 3 14.00 12.87 Spring 7 11.86 11.00 Fall

2009-2010

17 2 12.50 11.00 Spring 6 12.00 11.82 Fall

2010-2011

17 2 10.50 9.39 Spring 7 12.71 11.82 *National means vary as normative samples change on a triannual basis. ** These data unavailable to the current Clinical Mental Health Counseling Program Coordinator. Table 18 Student Scores on “Group Work” Content Area of the CPCE in comparison with SOSU and National Means (in Direct Correlation with Learning Outcome #6)



SOSU Mean Score


Fall

2006-2007

17 0 n/a n/a Spring 3 11.70 11.60 Fall

2007-2008

17 4 11.50 10.93 Spring 3 14.00 11.55 Fall

2008-2009

17 3 13.00 11.45 Spring 7 13.00 11.24 Fall

2009-2010

17 2 9.50 11.24 Spring 6 14.00 12.87 Fall

2010-2011



Table 19 Student Scores on “Assessment” Content Area of the CPCE in comparison with SOSU and National Means (in Direct Correlation with Learning Outcome #7)



SOSU Mean Score


Fall

2006-2007

17 0 n/a n/a Spring 3 10.30 10.30 Fall

2007-2008

17 4 8.00 8.00 Spring 3 10.70 10.34 Fall

2008-2009

17 3 12.67 11.82 Spring 7 11.14 9.89 Fall

2009-2010

17 2 11.00 9.89 Spring 6 9.33 10.05 Fall

2010-2011


Table 20 Student Scores on “Research and Program Evaluation” Content Area of the CPCE in comparison with SOSU and National Means (in Direct Correlation with Learning Outcome #8)



SOSU Mean Score


Fall

2006-2007

17 0 n/a n/a Spring 3 10.30 10.80 Fall

2007-2008

17 4 9.50 10.10 Spring 3 11.00 10.80 Fall

2008-2009

17 3 9.00 9.89 Spring 7 11.29 10.46 Fall

2009-2010

17 2 7.50 10.46 Spring 6 11.83 11.48 Fall

2010-2011


8. Analysis and Interpretation: In analyzing data covering each of the eight aforementioned learning objective areas over the past five years with data from the CPCE, the following information was determined: a. Learning Objective #1: CMHC Orientation and Ethical Practice: In examining data

obtained via the CPCE over the past five years covering this training area (see Table 13


above), it appears that SOSU graduate students in their final semester of training demonstrated knowledge levels that were fairly equal to national averages. This finding has been consistent over the past five years.

b. Learning Objective #2: Social and Cultural Diversity: In examining data obtained via the CPCE over the past five years covering this training area (see Table 14 above), it appears that SOSU graduate students in their final semester of training demonstrated knowledge levels that were equal to or slightly higher than national averages in this area. This finding has been relatively consistent over the past five years.

c. Learning Objective #3: Human Growth and Development: In examining data obtained via the CPCE over the past five years covering this training area (see Table 15 above), it appears that SOSU graduate students in their final semester of training demonstrated knowledge levels that were consistently comparable with national averages. Deviations in both directions, however, are noted.

d. Learning Objective #4: Career Counseling and Development: In examining data obtained via the CPCE over the past five years covering this training area (see Table 16 above), it appears that SOSU graduate students in their final semester of training demonstrated knowledge levels that were equal to or slightly lower than national averages in this area.

e. Learning Objective #5: Helping Relationships: In examining data obtained via the CPCE over the past five years covering this training area (see Table 17 above), it appears that SOSU graduate students in their final semester of training demonstrated knowledge levels that were consistently higher than national averages in this area.

f. Learning Outcome #6: Group Work: In examining data obtained via the CPCE over the past five years covering this training area (see Table 18 above), it appears that SOSU graduate students in their final semester of training demonstrated knowledge levels that were generally higher than national averages demonstrated by students at other colleges and universities. However, there were deviations (i.e. Fall 2009, Fall 2010).

g. Learning Objective #7: Counseling Assessment: In examining data obtained via the CPCE over the past five years covering this training area (see Table 19), it appears that SOSU graduate students in their final semester of training demonstrated varying knowledge levels in the area of Assessment. From Spring 2007-Fall 2009, students scored higher than the national average. However, more recently, students have scored lower than the national average on this area.

h. Learning Objective #8: Research and Program Evaluation: In examining data obtained via the CPCE over the past five years covering this training area (see Table 20), it appears that SOSU graduate students in their final semester of training demonstrated varying knowledge levels in the area of Research and Program Evaluation. One possible explanation for this change was the creation of a counseling specific research course (COUN 5863: Research in Counseling) in the Fall of 2006. Prior to this, students completed the more general educational research course (EDUC 5403: Introduction to Educational Research). Also, the Department of Behavioral Sciences established a consistent instructor of this course beginning in the Fall of 2007. This consistency in instruction by a full-time faculty member, along with the qualifications of the instructor, may help explain the increase in student scores in this area. The exceptions to this increase in scores are Fall 2009 and Fall 2010.

9. Program Modifications: As mentioned under this heading in section V.A.9. above, the Clinical

Mental Health Counseling Graduate Program completely revamped the core curriculum to be in line with the 2009 CACREP Standards. This was part of the process of applying for national specialty program accreditation as a CACREP-Accredited Clinical Mental Health Counseling program. As such, the course objectives and content related to all eight of the aforementioned training areas have been adjusted to be directly in line with those outlined in the 2009 CACREP Standards. With the implementation of these recent changes, increased scores in all eight CPCE Subsections are anticipated.


C. Student Performance Data Regarding National Council for Accreditation of Teacher Education

(NCATE) Advanced Certificate Portfolio Artifacts Directly Correlated with the Eight Core Council for Accreditation of Counseling and Related Educational Programs (CACREP) Training Areas: Throughout the course of graduate education in the Clinical Mental Health Counseling Program at Southeastern, students complete a total of eight didactic courses which directly correspond with the eight learning objectives previously outlined in this report. Each of these eight courses has a project (referred to by NCATE as an “artifact”) that has been developed in conjunction with NCATE requirements to verify student learning with regard to the didactic content outlined on the syllabus for each course. The eight learning objectives identified in this report and their corresponding SOSU graduate courses (each of which has an NCATE artifact required) are outlined in Table 21 as follows:

Table 21 SOSU Clinical Mental Health Counseling Required Courses Directly Correlated with the Eight Learning Objectives of this Report Number Learning Objective SOSU Course

1 CMHC Orientation and Ethical Practice COUN 5133: Clinical Mental Health Counseling Orientation and Ethical Practice

2 Social and Cultural Diversity COUN 5483: Social and Cultural Diversity Issues in Counseling

3 Human Growth and Development COUN 5243: Human Growth and Development in Counseling

4 Career Counseling and Development COUN 5323: Career Counseling and Development 5 Helping Relationships COUN 5443: Theories of Counseling 6 Group Work COUN 5523: Group Counseling 7 Counseling Assessment COUN 5283: Assessment in Counseling 8 Research and Program Evaluation COUN 5863: Research in Counseling

The assessment method employed regarding the eight NCATE artifacts utilizes a three-level rating scale system. Specifically, student levels of proficiency on each of the eight artifacts discussed in this section of the report were rated on a nominal scale as follows: an outcome score of three represented a proficiency rating in the “target” range (above average performance for a graduate-level student), an outcome score of two represented a proficiency rating in the “acceptable” range (average performance for a graduate-level student), and an outcome score of one represented a proficiency rating in the “unacceptable” range (below average for a graduate-level student). 1. How Many Students Participated in the Assessment? The number of students in each of the

eight courses covered by the accompanying NCATE artifact evaluations during the time period of August 2010 through July 2011 varies from course to course. Tables 22-29 (below) provide a summary of student performances on each NCATE artifact, which compares Clinical Mental Health Counseling and School Counseling student performances on the artifact over the past four academic school years. The number of students participating in each evaluation school year and subject area is outlined in Tables 22-29 that follow.

2. How Were Students Selected to Participate in the Assessment? All students enrolled in the

Clinical Mental Health Counseling Master’s Degree Program are required to take the eight courses covered by the aforementioned NCATE artifacts. This requirement is in direct conjunction with both CACREP requirements and the eight learning objectives outlined in Section IV of this report.


3. What was the Make Up of the Sample? The sample related to each NCATE artifact consisted of

both Clinical Mental Health Counseling and School Counseling graduate students. These students represented all levels of training, since there are no sequencing or prerequisite issues related to the eight courses and they may be taken at any time during the students’ graduate program of study.

4. What Work was Evaluated? A key course project from each of the eight core CACREP training

courses was represented by the following NCATE artifacts. The project was expressly designed by each course instructor to cover the course training objectives as outlined on the syllabus for each course. This assessment method was designed to represent an internal evaluation of the eight learning objectives, as opposed to the previous two assessment methods (i.e., the NCE and CPCE), which were both norm-referenced, nationally standardized assessment instruments.

5. How was the Assessment Conducted? The individual course instructors developed the

requirements and deadlines for each of the course projects represented by the eight NCATE Portfolio Artifacts assessed. A variety of instructors teach these eight courses; however, each instructor is asked to require a course project that is representative of the course objectives outlined in each of the corresponding course syllabi. Each course instructor then evaluates the individual students’ performances on the artifact, assigning a grade and a corresponding rating of 1-3 using the pre-approved NCATE grading rubric. The results of this process over the past five school years are summarized in Tables 22-29.


obtained from this assessment method, the Clinical Mental Health Counseling Master’s Degree Program is committed to continuing to use these NCATE artifact data to assess program outcomes from this point forward. This is, of course, contingent upon the continued commitment of individual faculty members to apply consistent assessment methods and supply the Clinical Mental Health Counseling Program Coordinator with these data each semester as they are available.

7. Results of the Assessment: Results of the student performances on each of the eight NCATE

Artifacts for the past five years are included in Tables 22-29 that follow. Please note that the data included in the subsequent eight tables is similar to those included in the School Counseling Program Outcome Assessment Report, as they cover student performances in both counseling graduate programs in an identical and shared fashion. The interpretation of these data, however, has been individually tailored to the outcome assessment goals of the Clinical Mental Health Counseling Master’s Degree Program following the presentation of the tables:

Please note that Tables 22-29 have each been placed on a separate page for ease of review.


Table 22 Student Proficiency Ratings Regarding NCATE Advanced Certificate Portfolio Artifact #1: “CMHC Orientation and Ethical Practice,” Directly Correlated with Learning Outcome #1 (Past Five Academic School Years)

Time

Period Covered

Graduate Training

Focus Area

Students Performing within

“Target” Range/%


“Acceptable” Range/%


“Unacceptable” Range/%

2006-2007 School Year

Clinical Mental Health Counseling (n = 24)

21

88%

3

13%

0

0%

School Counseling (n = 12)

12

100%

0

0%

0

0%

Combined Scores (Both Programs; N = 36)

33

92%

3

8%

0

0%



15

94%

1

6%

0

0%


12

100%

0

0%

0

0%


27

96%

1

4%

0

0%



11

73%

4

27%

0

0%


7

88%

1

12%

0

0%


18

78%

5

22%

0

0%



20

77%

5

19%

1

4%


9

90%

1

10%

0

0%


29

81%

6

17%

1

2%



16

76%

5

24%

0

0%


5

56%

3

33%

1

1%


21

70%

8

27%

1

3%

Summary

Data 2006-2011


83

81%

18

18%

1

1%


45

88%

5

1%

1

1%


128

84%

23

15%

2

1%


Table 23 Student Proficiency Ratings Regarding NCATE Advanced Certificate Portfolio Artifact #2: “Social and Cultural Diversity,” Directly Correlated with Learning Outcome #2 (Past Five Academic School Years)

Time

Period Covered

Graduate Training

Focus Area









9

100%

0

0%

0

0%


14

100%

0

0%

0

0%


23

100%

0

0%

0

0%



3

38%

5

62%

0

0%


7

70%

3

43%

0

0%


10

56%

8

44%

0

0%



8

80%

2

20%

0

0%


9

100%

0

0%

0

0%


17

89%

2

11%

0

0%



7

47%

8

53%

0

0%


6

86%

1

14%

0

0%


13

60%

9

40%

0

0%



2

50%

2

50%

0

0%


3

50%

3

50%

0

0%


5

50%

5

50%

0

0%

Summary

Data 2006-2011


29

63%

17

37%

0

0%


39

85%

7

15%

0

0%


68

74%

24

26%

0

0%


Table 24 Student Proficiency Ratings Regarding NCATE Advanced Certificate Portfolio Artifact #3: “Human Growth and Development,” Directly Correlated with Learning Outcome #3 (Past Five Academic School Years)

Time

Period Covered

Graduate Training

Focus Area









12

86%

2

14%

0

0%


4

44%

4

44%

1

11%


16

70%

6

26%

1

4%



8

89%

1

11%

0

0%


7

88%

1

14%

0

0%


15

88%

2

12%

0

0%



3

30%

7

70%

0

0%


1

17%

5

83%

0

0%


4

25%

12

75%

0

0%



5

50%

5

50%

0

0%


2

40%

3

60%

0

0%


7

47%

8

53%

0

0%



3

50%

2

33%

1

17%


3

43%

4

57%

0

0%

Combined Scores (Both Programs; (N = 13)

6

46%

6

46%

1

8%

Summary

Data 2006-2011


31

63%

17

35%

1

2%


17

49%

17

49%

1

2%


48

57%

34

40%

2

3%


Table 25 Student Proficiency Ratings Regarding NCATE Advanced Certificate Portfolio Artifact #4: “Career Counseling and Development,” Directly Correlated with Learning Outcome #4 (Past Five Academic School Years)

Time

Period Covered

Graduate Training

Focus Area









0

0%

5

83%

1

17%


7

64%

4

36%

0

0%


7

41%

9

53%

1

6%



9

75%

3

25%

0

0%


6

75%

2

25%

0

0%


15

75%

5

25%

0

0%



5

71%

2

29%

0

0%


4

67%

2

33%

0

0%


9

69%

4

31%

0

0%



4

100%

0

0%

0

0%


9

82%

1

9%

1

9%


13

86%

1

7%

1

7%



9

82%

2

18%

0

0%


2

67%

1

33%

0

0%


11

79%

3

21%

0

0%

Summary Data

2006-2011


27

68%

12

30%

1

2%


28

72%

10

26%

1

2%


55

70%

22

28%

2

2%


Table 26 Student Proficiency Ratings Regarding NCATE Advanced Certificate Portfolio Artifact #5: “Helping Relationships,” Directly Correlated with Learning Outcome #5 (Past Five Academic School Years)

Time

Period Covered

Graduate Training

Focus Area









7

44%

6

38%

3

19%


8

53%

7

47%

0

0%


15

48%

13

42%

3

10%



9

82%

2

18%

0

0%


8

89%

1

11%

0

0%


17

85%

3

15%

0

0%



7

78%

2

22%

0

0%


4

100%

0

0%

0

0%


11

85%

2

15%

0

0%

2009-2010

School Year


13

72%

3

17%

2

11%


6

75%

2

25%

0

0%


19

73%

5

19%

2

8%



7

47%

6

40%

2

13%


3

75%

0

0%

1

25%


10

53%

6

32%

3

15%

Summary

Data 2006-2011


43

62%

19

28%

7

10%


29

73%

10

25%

1

2%


72

66%

29

27%

8

7%


Table 27 Student Proficiency Ratings Regarding NCATE Advanced Certificate Portfolio Artifact #6: “Group Work,” Directly Correlated with Learning Outcome #6 (Past Five Academic School Years)

Time

Period Covered

Graduate Training

Focus Area









1

25%

2

50%

1

25%


6

67%

3

33%

0

0%


7

78%

5

38%

1

8%



4

50%

3

38%

1

12%


5

63%

3

37%

0

0%


9

56%

6

38%

1

6%



3

33%

5

56%

1

11%


4

57%

2

29%

1

14%


7

44%

7

44%

2

12%



6

40%

8

53%

1

7%


3

38%

4

50%

1

12%


9

39%

12

52%

2

9%



9

50%

8

44%

1

6%


3

60%

2

40%

0

0%


12

52%

10

43%

1

5%

Summary

Data 2006-2011


23

43%

26

48%

5

9%


21

57%

14

38%

2

5%


44

48%

40

44%

7

8%


Table 28 Student Proficiency Ratings Regarding NCATE Advanced Certificate Portfolio Artifact #7: “Counseling Assessment,” Directly Correlated with Learning Outcome #7 (Past Five Academic School Years)

Time

Period Covered

Graduate Training

Focus Area









3

33%

6

67%

0

0%


14

74%

5

26%

0

0%


17

61%

11

39%

0

0%



4

27%

8

53%

3

20%


6

46%

5

38%

2

15%


10

36%

13

46%

5

18%



1

14%

2

29%

4

57%


3

50%

3

50%

0

0%


4

31%

5

38%

4

31%



8

53%

7

47%

0

0%


4

100%

0

0%

0

0%


12

63%

7

37%

0

0%



7

50%

7

50%

0

0%


2

25%

6

75%

0

0%


9

41%

13

59%

0

0%

Summary

Data 2006-2011


23

38%

30

50%

7

12%


29

58%

19

38%

2

4%


52

47%

49

45%

9

8%


Table 29 Student Proficiency Ratings Regarding NCATE Advanced Certificate Portfolio Artifact #8: “Research and Program Evaluation,” Directly Correlated with Learning Outcome #8 (Past Five Academic School Years)

Time

Period Covered

Graduate Training

Focus Area









8

67%

3

25%

1

8%


0

0%

1

100%

0

0%


8

62%

4

31%

1

8%



5

56%

3

33%

1

11%


6

100%

0

0%

0

0%


11

73%

3

20%

1

7%



7

78%

2

22%

0

0%


5

63%

3

37%

0

0%


12

71%

5

29%

0

0%



6

43%

8

57%

0

0%


7

88%

1

12%

0

0%


13

59%

9

41%

0

0%



5

63%

2

25%

1

12%


3

75%

1

25%

0

0%


8

67%

3

25%

1

8%

Summary

Data 2006-2011


31

60%

18

35%

3

5%


21

78%

6

22%

0

0%


52

66%

24

30%

3

4%


8. Analysis and Interpretation: During the prescribed assessment period, a varying number of SOSU graduate students completed each of the eight NCATE artifact projects, depending on enrollment numbers for any given school year. In analyzing student data covering each of the eight delineated learning objective areas over the past five years with data from the corresponding courses and NCATE artifacts, the following information is gleaned:

a. Learning Objective #1: CMHC Orientation and Ethical Practice: In examining data

obtained regarding the NCATE Portfolio Artifact for “COUN 5113: Orientation to the Counseling Profession” over the past five years (related to Learning Objective #1 and Table 22 above), it is clear that a majority of Clinical Mental Health Counseling Students (81%) achieved Target ratings (representing an above average performance on the representative project for a graduate-level student). Another 18% earned an Acceptable rating (representing an average performance for a graduate-level student) and one student earned an Unacceptable rating during the entire assessment period. Taken as a whole, these data indicate that the Clinical Mental Health Counseling Graduate Students are performing quite well on the artifact required for this course. Therefore, using this rating alone, Learning Objective #1 has been met by SOSU’s Clinical Mental Health Counseling Graduate Students over the past five years.

b. Learning Objective #2: Social and Cultural Diversity: In examining data obtained regarding the NCATE Portfolio Artifact for “COUN 5483: Social and Cultural Diversity Issues in Counseling” over the past five years (related to Learning Objective #2 and Table 23 above), a majority of Clinical Mental Health Counseling Students (63%) achieved Target ratings. Another 37% earned Acceptable ratings and zero students earned an Unacceptable rating during the entire assessment period. Taken as a whole, these data indicate that the Clinical Mental Health Counseling Graduate Students are performing well on the artifact required for this course; however, improvement could be sought to bring the number of students achieving a Target rating to a higher level. That being said, since 0 students out of 46 earned an unacceptable rating (which would indicate a below average performance for a graduate-level student), this learning objective also appears to have been clearly met by SOSU’s Clinical Mental Health Counseling Graduate Students over the past five years.

c. Learning Objective #3: Human Growth and Development: In examining data obtained regarding the NCATE Portfolio Artifact for “COUN 5243: Human Growth and Development in Counseling” over the past five years (related to Learning Objective #3 and Table 24 above), 63% of Clinical Mental Health Counseling Students achieved Target ratings, 35% earned Acceptable ratings, 2% (1 out of 49 students) earned an Unacceptable rating during the entire assessment period. This represents a reasonable distribution of scores. The clear goal here, however, would be to strengthen course learning in such a manner so as to bring up more students from the Unacceptable range on this NCATE Portfolio Artifact to the Acceptable, or, preferably, the Target rating in this course.

d. Learning Objective #4: Career Counseling and Development: In examining data obtained regarding the NCATE Portfolio Artifact for “COUN 5323: Career Counseling and Development” over the past five years (related to Learning Objective #4 and Table 25 above), a majority (27; 68%) of Clinical Mental Health Counseling Students earned Target ratings, 12 (30%) of students earned an Acceptable rating, and 1 student earned an Unacceptable rating. This is a reasonable breakdown of grades, as would be expected of graduate counseling students. Of course, a goal would be to move the one student from the Unacceptable range to either Acceptable or Target ranges.

e. Learning Objective #5: Helping Relationships: In examining data obtained regarding the NCATE Portfolio Artifact for “COUN 5443: Theories of Counseling” over the past five years (related to Learning Objective #5 and Table 26 above), a majority (43; 62%) of Clinical Mental Health Counseling Students earned Target ratings, 19 (28%) of students earned an Acceptable rating, and 7 students (10%) earned an Unacceptable rating. This is a reasonable


breakdown of grades, as would be expected of graduate-level students in any program of study. Of course, a goal would be to move the seven students from the Unacceptable range to either Acceptable or Target ranges.

f. Learning Outcome #6: Group Work: In examining data obtained regarding the NCATE Portfolio Artifact for “COUN 5523: Group Counseling” over the past five years (related to Learning Objective #6 and Table 27 above), only 23 (43%) of students earned Target ratings, 26 (48%) of students earned Acceptable Ratings, and 5 (9%) of students earned Unacceptable Ratings. Work must be done to increase the number of students in the Target range move the five students from the Unacceptable range to either Acceptable or Target ranges.

g. Learning Objective #7: Counseling Assessment: In examining data obtained regarding the NCATE Portfolio Artifact for “COUN 5283: Assessment in Counseling” over the past five years (related to Learning Objective #7 and Table 28 above), only 23 (38%) of students earned Target ratings, 30 (50%) of students earned Acceptable Ratings, and 7 (12%) of students earned Unacceptable Ratings. On this NCATE artifact, students demonstrated the lowest number of student achieving scores in the Target range. Improvements must be made in this heading to increase the number of students falling in the Target range. Further, on this NCATE artifact, students demonstrated the highest number of scores in the Unacceptable range. Clearly, based upon data gathered via this outcome assessment measure, improvements must be made to ensure student knowledge acquisition in the area of Assessment, as 62% of students failed to demonstrate scores in the Target range.

h. Learning Objective #8: Research and Program Evaluation: In examining data obtained regarding the NCATE Portfolio Artifact for “COUN 5863: Research in Counseling” over the past five years (related to Learning Objective #8 and Table 29 above), a majority (31; 60%) of Clinical Mental Health Counseling Students earned Target ratings,18 (35%) of students earned an Acceptable rating, and 3 students (5%) earned an Unacceptable rating. This is a reasonable breakdown of grades, as would be expected of graduate counseling students. Of course, a goal would be to move the two students from the Unacceptable range to either Acceptable or Target ranges.

9. Program Modifications: As mentioned under this heading in sections V.A.9. and V.B.9. above,

the Clinical Mental Health Counseling Graduate Program has completely revamped the core curriculum to be aligned with the 2009 CACREP Accreditation Standards. This modification was part of the process of applying for national specialty program accreditation as a CACREP-Accredited Clinical Mental Health Counseling program. As such, the course objectives and content related to all eight of the aforementioned NCATE Artifact areas were adjusted to be directly in line with those outlined in the 2009 CACREP Standards. With the implementation of these changes, improvements in student scores in all eight CPCE Subsections are anticipated in the coming years.

D. Program Alumni Survey and Data:

The CMHC program follows a systematic process for collecting data regarding students who have recently graduated from our program. The purpose of this data collection is to assess the quality and effectiveness of our counseling program and the competence of program graduates. Data are collected directly from our graduates within a year of their graduation. As students are cleared for graduation, contact information is obtained from them by the CMHC Program Coordinator, and they are informed of the data collection process and the importance of their participation in the process. Surveys are sent out once a year (along with postage-paid return envelopes), so that the data can be collected and evaluated prior to the annual Program Outcome Assessment Report (POAR). The department administrative assistant plays an important role in collecting and entering raw data for the purpose of evaluation while maintaining the anonymity of the respondents. Anonymous data are then provided to the CMHC Program Coordinator in aggregate form via a spreadsheet file for analysis and


inclusion in the annual POAR. Original surveys are then shredded by the department administrative assistant for confidentiality purposes. The current program alumni survey was developed in conjunction with CACREP guidelines and mailed to alumni who graduated during the previous year for the very first time in the Fall of 2011. As such, data are currently unavailable regarding this survey. Survey results, however, will be reported annually beginning with the publication of the 2011-2012 POAR.

E. Employer Survey and Data:

The CMHC program also follows a systematic process for collecting data regarding site supervisors/employers of students who have recently graduated from our program. The purpose of this data collection is to assess the quality and effectiveness of our counseling program (from the perspective of the site supervisor/ employer) and the competence of program graduates in their postgraduate work settings. Addresses from all area mental health centers along with addresses from the sites utilized by practicum and internship students during the previous academic year are gathered for the purposes of this survey. Surveys are sent out once a year (along with postage-paid return envelopes) to all addresses cataloged via the aforementioned method. Site supervisors/employers are asked to complete the surveys only if they have served as a site supervisor and/or employer during the previous two-year period. Data received from the site supervisors/employers are then collected and evaluated prior to completion of the annual POAR. The department administrative assistant plays an important role in collecting and entering raw data for the purpose of evaluation while maintaining the anonymity of the respondents. Anonymous data are then provided to the CMHC Program Coordinator in aggregate form via a spreadsheet file for analysis and inclusion in the annual POAR. Original surveys are then shredded by the department administrative assistant for confidentiality purposes. The current site supervisors/employers survey was developed in conjunction with CACREP guidelines and mailed for the very first time in the Fall of 2011. As such, data are currently unavailable regarding this survey. Survey results, however, will be reported annually beginning with the publication of the 2011-2012 POAR.

VI. STRENGTHS AND WEAKNESSES OF THE PROGRAM: In examining overall findings from the three different assessment sources, covering the eight learning objectives identified in this report, along with the Clinical Mental Health Counseling Program in general, the following strengths have been identified:

• Based upon findings of the NCE, the areas in which students demonstrated highest learning proficiency included Professional Orientation and Ethical Practice (related to Learning Outcome #1), Social and Cultural Diversity (related to Learning Outcome #2), Career Development (related to Learning Outcome #4), Helping Relationships (related to Learning Outcome #5), Group Work (related to Learning Outcome #6), and Assessment (related to Learning Outcome #7).

• Based upon findings of the CPCE the areas in which students demonstrated highest learning proficiency included Professional Orientation and Ethical Practice (related to Learning Outcome #1), Social and Cultural Diversity (related to Learning Outcome #2), Human Growth and Development (related to Learning Outcome #3), Helping Relationships (related to Learning Outcome #5), and Group Work (related to Learning Outcome #6).

• Based upon findings of the assessed NCATE Artifacts, students demonstrated highest learning proficiency in the courses COUN 5113: Orientation to the Counseling Profession (related to Learning Outcome #1), COUN 5483: Social and Cultural Diversity Issues in Counseling (related to Learning Outcome #2), COUN 5243: Human Growth and Development in Counseling (related to Learning Outcome #3), COUN 5323: Career Counseling and Development (related to Learning Outcome #4),


COUN 5443: Theories of Counseling (related to Learning Outcome #5), and COUN 5863: Research in Counseling (related to Learning Outcome #8).

• Four faculty members in the Department of Behavioral Sciences have clear training and professional identities related to CACREP and the area of Clinical Mental Health Counseling. These four faculty members, all of whom graduated from CACREP-Accredited doctoral programs include Drs. Criswell, Donovan, Mauzey, and Weigel. Additionally, Dr. Jon Reid has been trained in the field of counseling and currently holds professional credentials commensurate with the four CACREP faculty members identified above.

• The Department and University are strongly committed to attaining national specialty program accreditation of the Clinical Mental Health Counseling program as a “Clinical Mental Health Counseling” master’s degree program, over the next several years.

In examining overall findings from the three different assessment sources, covering the eight learning objectives identified in this report, the following weaknesses have been identified:

• Based upon findings of the NCE, the areas in which students could use the most training improvement are Human Growth and Development (Learning Objective #3) and Research and Program Evaluation (Learning Objective #8).

• Based upon findings of the CPCE, areas in which students could use training improvement include: Career Development (related to Learning Outcome #4), Assessment (Learning Objective #7) and Research and Program Evaluation (Learning Objective #8).

• Based upon findings of the NCATE Artifacts for the courses “COUN 5523: Group Counseling” (related to Learning Objective #6) and “COUN 5283: Assessment in Counseling” (related to Learning Objective #7), students need to improve performances on the major projects for these courses. The actual artifact and corresponding course requirements will also need to be revisited to ensure that they are accurately addressing the course objectives outlined in the corresponding syllabi for the courses.

VII. PROGRAM MODIFICATIONS IN THE PAST FIVE (5) YEARS:

During the 2010-2011 school year, a number of very significant program curricular modifications were prepared regarding the Clinical Mental Health Counseling Master’s Degree Program by our departmental CACREP Task Force. These modifications were proffered as the CACREP Task Force completed the second year of work toward the impending application for specialty program accreditation as a CACREP “Clinical Mental Health Counseling” training program. The modifications have been approved at all university levels and by the Regional University System of Oklahoma (RUSO). On September 9, 2011, Oklahoma State Regents for Higher Education (OSRHE) reviewed and approved the proposed modifications for official instatement. The following is a summary of these modifications: • The title of the degree program changed from “Community Counseling” to “Clinical Mental Health

Counseling,” to attain explicit alignment with the specialty accreditation we shall be seeking in the near future.

• The degree designation of the program changed from “Master of Behavioral Studies (MBS)” to “Master of Arts (M.A.),” to allow for greater national recognition and degree portability.

• Five new required courses and four new elective courses were added to the Counseling (COUN) course inventory.

• Five existing courses were deleted, since they are no longer required or optional electives for any graduate degree at Southeastern.

• Twenty-one courses in the current “Counseling” course inventory were modified to establish alignment with CACREP Accreditation Standards.


• The Degree Plan for the Clinical Mental Health Counseling Master’s Degree Program was modified to include 60 credit hours rather than 48 credit hours to align with CACREP and Oklahoma Licensed Professional Counselor (LPC) occupational licensing requirements.

• The Admission, Retention, Suspension, and Dismissal Procedures and Student Due Process Rights related to the Clinical Mental Health Counseling Program were expanded and clearly delineated to meet requirements of the 2009 CACREP Accreditation Standards.

• A Program Alumni Survey was created (Fall 2011) to assess our program’s strengths and weaknesses from the perspective of recent graduates so that appropriate modifications to the program may be implemented as necessary.

• An Employer Survey was created (Fall 2011) to assess our program’s strengths and weaknesses from the perspective of employers and supervisors so that appropriate modifications to the program may be implemented as necessary.

These modifications were made with the guidance of a hired CACREP accreditation consultant (Dr. Michael Altekruse from Northern Kentucky University). In June 2011, the CACREP consultant conducted a mock accreditation site team visit with the graduate counseling faculty to prepare them for the upcoming official CACREP site team visit in the Fall of 2011. Since these modifications were approved and implemented within the past year, there are currently no results regarding the effects of these modifications to report.

As mentioned previously, the graduate program in Clinical Mental Health Counseling was granted recognition by NBCC as a GSA-NCC credential counselor training program in the Spring of 2009. As such, students in their final semester of graduate studies in the Clinical Mental Health Counseling program (or recent graduates) interested in taking the NCE prior to graduation may choose to do so in April or October each year as part of the GSA-NCC program. Since it is not uncommon for individuals to wait to take the NCE until years after graduation, often while they are employed full-time and very busy accumulating required hours for licensure, the first-time pass rate is fairly low. Giving current SOSU students (or recent graduates) the opportunity to take the examination earlier should be an advantage which will ideally result in a higher first-time pass rate. Since April 2010, was the first administration of the on-campus examination, it is too early to know the extent of the benefit of this particular program modification.

VIII. IETV COURSES:

No required courses or elective courses on the Clinical Mental Health Counseling degree plan were taught via IETV, online, in blended formats during the 2010-2011 academic year. Further, very few courses have been offered in any of these formats for the past several years. This face-to-face training emphasis has been developed in conjunction with the philosophy of the profession for which faculty members are training the students: face-to-face professional counseling. The faculty members prepare students for effective, face-to-face counseling via hands-on coursework. Online, blended, and IETV make such training very difficult. Despite this fact, from semester to semester graduate counseling faculty members make an effort to reach students living closer to the SOSU Idabel Campus by offering content-appropriate courses via the IETV format. The three courses offered via IETV two years ago were not offered via IETV in previous years. Therefore, no data exist with which to compare students’ performances with previous years. Further, the number of students enrolled in the IETV sections was too small (two in each) to allow for any comparison between groups. In all cases, the IETV courses were broadcast to the SOSU Idabel Campus with the home campus being Durant. Lastly, despite the fact that no equivalent sections were available for comparison, it should be mentioned that the learning outcomes, educational goals, and objectives of the courses offered in the


past via IETV, online or in blended format were identical to those required of all courses (IETV and non-IETV) taught in the program, as defined by and required of CACREP.

IX. DEPARTMENTAL CONTRIBUTIONS TO THIS REPORT:

Data for this report were gathered from numerous departmental faculty members, including Drs. Reba Criswell, Kimberly Donovan, Ed Mauzey, Jon Reid, and Daniel Weigel. Each of these faculty members contributed to the data pool for section V.(C). of this report, regarding NCATE Portfolio Artifacts. Further, data regarding the CPCE were gathered by Dr. Charla Hall prior to the Spring of 2008. This report was made available to all faculty members in the Department of Behavioral Sciences, via email, for editing and modification purposes. Faculty members included in this editing process included Drs. Dennis Brewster, Reba Criswell, Delilah Dotremon, Blythe Duell, Jane Elder, Ken Elder, Charla Hall, Jennifer Hicks, Jamie Knapp, Ed Mauzey, Jon Reid, Hallie Stephens, Daniel Weigel, Deana Williams, and the primary author, Dr. Kimberly Donovan. Lastly, as part of the established policy, members of the Graduate Counseling Coordinating Committee (GCCC) formally reviewed this report prior to submission. The GCCC critically analyzed information presented in this report with particular emphasis placed upon an overall review of the program, curricular offerings, and characteristics of program applicants.

X. RELEVANT CONSTITUENTS AND STAKEHOLDERS OF THIS REPORT:

Relevant constituents and stakeholders of the Clinical Mental Health Counseling Program Outcome Assessment Report include the following: (a) prospective and current students in the Clinical Mental Health Counseling program; (b) faculty members teaching courses in the Clinical Mental Health Counseling program; (c) practicum and internship sites for Clinical Mental Health Counseling graduate students; (d) graduates of the Clinical Mental Health Counseling graduate program; (e) employers of graduates of the Clinical Mental Health Counseling graduate program; and (f) the public at large, as recipients of professional counseling services from graduates of the Clinical Mental Health Counseling graduate program.

XI. METHODS OF SHARING DATA FROM THIS REPORT WITH IDENTIFIED CONSTITUENTS AND STAKEHOLDERS:

The Clinical Mental Health Counseling Program Coordinator provides students with individualized score

reports regarding their CPCE results to assist the students in studying for the National Counselor Examination. In this way, students are better informed about the areas they need to study further as they prepare to take the national licensure examination. The National Board for Certified Counselors (NBCC) provides students with confidential, individualized score reports regarding their NCE results.

This report is made available to all of the aforementioned constituents and stakeholders via the department

Web site. It is also be shared with the faculty of the Department of Behavioral Sciences, the GCCC in October of each year, and the CMHC Advisory Board during its annual meeting.

XII. ONGOING COMMITMENT TO CONTINUE THE ASSESSMENT PROCESS:

The Clinical Mental Health Counseling Program Coordinator is deeply committed to the ongoing assessment process. This is particularly important as the program prepares to apply for national specialty program accreditation as a CACREP-Accredited Clinical Mental Health Counseling Master’s Degree Program. CACREP places great emphasis on program outcome assessment measures. This and future reports shall serve the program well in demonstrating its commitment to evaluating student learning


outcomes and modifying the program to meet the needs of a changing mental health marketplace in the 21st century. ________________________________________________________ ____________________ Dr. Kimberly Donovan, Associate Professor of Counseling Date Primary Report Author, Clinical Mental Health Counseling Program Coordinator ________________________________________________________ ____________________ Dr. Ed Mauzey, Professor of Counseling Date Chair, Department of Behavioral Sciences _________________________________________________________ ____________________ Dr. William Mawer, Professor Date Dean, School of Education and Behavioral Sciences

Documents

PROGRAM OUTCOME ASSESSMENT REPORT … outcome assessment report master of arts in clinical mental health counseling department of behavioral sciences southeastern oklahoma state university