Embed Size (px)
Citation preview
R 7036Program Evaluation Methods
Spring I 2008INSTRUCTOR: William A. Nowlin, Ph.D.
PHONE:(708) 261-8811
EMAIL:[email protected]
FAX:
ALT PHONE:
REQUIRED TEXTS:
Title Program Evaluation: Alternative Approaches and Practical Guidelines
Author(s) Fitzpatrick, J., Sanders, J. R., & Worthen, B. R.
CopyrightPublisher Allyn & Bacon
ISBN 0321077067
Edition 3rd
Title Handbook of practical program evaluation
Author(s) Wholey, J., Hatry, H., & Newcomer, K.
Copyright (2004).
Publisher Jossey-Bass
ISBN 0-7879-6713-0.
Edition 2nd
This Course Requires the Purchase of a Course Packet: YES NOArgosy University
COURSE SYLLABUSR7036
Program Evaluation Methods
Spring I 2008
Weekend Course
Friday, January 18, 2008 6:00 p.m. – 9:00 p.m.Saturday January 19, 2008 9:00 a.m. – 5:00 p.m.Sunday, January 20, 2008 9:00 a.m. – 1:00 p.m.
Friday, February 8, 2008 6:00 p.m. – 9:00 p.m.Saturday, February 9, 2008 9:00 p.m. – 5:00 p.m.Sunday, February 10, 2008 9:00 p.m. – 1:00 p.m.
This syllabus is subject to change
Faculty InformationFaculty Name: William A. Nowlin, Ph.D.Campus: ChicagoContact Information: (708) 261-8811 [email protected] Hours: By appointment
Short Faculty Bio: Nowlin teaches courses in educational administration, organizational leadership and business.
He has published articles on employee and workplace issues. He holds a Ph.D. in Higher Education (University of Buffalo), Masters of Public Administration (State University of New York, Brockport), and a MS in Organizational Psychology, (Kansas State University) and a BA in Business Administration (State University of New York, Empire). He is a professor at Governors State University; he a former dean and a former Interim Vice President and CEO of the GSU Foundation.
Course description: This seminar emphasizes the acquisition of knowledge and skills in program evaluation methodology. Six alternative evaluation approaches are surveyed, with a focus on developing a management/decision-oriented evaluation plan. This seminar also serves as a practicum for the conceptualization and development of a doctoral research study that employs a program evaluation model. Course Pre-requisites: PC6521 or equivalent. Required Textbook: Fitzpatrick, J., Sanders, J. R., & Worthen, B. R. Program Evaluation: Alternative Approaches and Practical Guidelines (3rd ed.). Allyn & Bacon. ISBN: 0321077067
E7036_TOC.doc
Recommended Reading: Wholey, J., Hatry, H., & Newcomer, K. (2004). Handbook of practical program evaluation (2nd edition). Jossey-Bass. ISBN: 0-7879-6713-0.
Online Resources:
http://www.nwrel.org/evaluation/overview.shtml
http://nsba.org/site/doc_EDLO.asp?TRACKID=&VID=57&CID=936&DID=11678
http://www.nicic.org/pubs/1996/014292.pdf
http://www.wmich.edu/evalctr/checklists/eval_model_metaeval.pdf
http://lrs.ed.uiuc.edu/students/sbarnett/edpsy399/curriculum/form.html
http://eval.org/
http://www.ed.gov/rschstat/eval/other/cclcfinalreport/cclcfinal.pdf
http://www.ed.gov/rschstat/eval/highered/upward/upward-3rd-report.pdf Technology: Pentium III CPU/ Windows XP; 128MB RAM, printer; Microsoft Office 2000 or XP, Acrobat (full version), Microsoft Internet Explorer 6.0 (PC) 5.0 (MAC) or Netscape Navigator 8, and Norton Antivirus. Course length: 7.5 weeks Contact Hours: 45 Hours Credit Value: 3.0 Program Outcomes: 1. Research and Theory - Educational Leaders make decisions based on research and supported theory. They use their knowledge and skills to:
1.1. Performing - Analyze, design, conduct, and defend research in an educational context using action research and other appropriate designs.1.2. Performing - Demonstrate knowledge of the role of research in professional development of faculty and students.1.3. Understanding
1.3.1. Evaluate and apply research and reflective practices to complex educational problems, including the assessment of improvement and accountability systems.
1.3.2. Exhibit a broad and systematic understanding of the role of research and theory in professional education.1.4. Analysis
1.4.1. Critically analyze current research in education and related fields to inform curricular and instructional decisions1.4.2. Analyze student achievement data for program improvement.
1.5. Applying - Evaluate and apply theory and research to select, integrate, implement and assess educational practices and materials1.6. Decision Making - Make decisions about curriculum, instructional strategies, assessment and professional development that are based on
sound research, best practice, school and district data and other contextual information2. Communication and Information Literacy - Educational leaders effectively communicate a vision of educational excellence to the learning
community. They use their oral and written language skills and their knowledge of informational literacy and organizational communication to: 2.1. Oral and Written Communication - Orally or in writing, present educational documents (including recommendations, critiques or
justifications) that are clear, concise, organized and well supported in a professional manner using media appropriate to the education context and audience, using appropriate media and technology.
2.2. Organizational Communication - Exhibit skills in building trust, gauging climate understanding, assessing perception, facilitating systemic processes associated with organizational communication and promoting a positive learning culture.
2.3. Information Literacy - Demonstrate multiple literacies (including information technology, knowledge of media, conflict resolution, etc.) to effectively support improved achievement, communicate educational messages and promote harmony.
2.4. Using Technologies - Promote the use of a broad range of software applications or Web-based tools to facilitate systematic investigation, to enrich curriculum and instruction, to monitor instructional practices and to provide staff the assistance needed for improvement.
2.5. Integration - Demonstrate effective integration of instructional technology into the teaching and learning process and to manage the assessment and reporting of student learning.
2.6. Accountability - Use technology to measure, assess, and evaluate information related to institutional effectiveness and accountability.3. Critical Thinking and Problem Solving - Educational leaders develop systems to check their own thinking, to reflect on present and past
practices in considering the future and to analyze complex issues and to evaluate potential solutions. They use their knowledge and skills to:3.1. Critical Thinking - Analyze contemporary theoretical concepts at all levels of educational leadership, in design, implementation, and
evaluation of personnel preparation programs at pre-service and in-service levels.3.2. Problem Solving/Decision Making - Identify a problem, diagnose its underlying causes, evaluate possible solutions in relation to
foundation theoretical foundations in education, and determine and defend an appropriate course of action.3.3. Evaluation - Evaluate relevance of established theory to current education practice and identify gaps in current literature.
4. Collaboration - Educational leaders promote democratic values by respecting the ideas and gifts of those with whose care they are charged. They continuously seek improvement and use their knowledge and skills to:4.1. Team Building - Build teams to accomplish the goals of complex organizations with educational missions.4.2. Families and Community - Work with families and various stakeholders of the educational community in addressing, selecting and
implementing an optimal course of action to deal with the issues that face educational leaders, including, but not limited to, resource acquisition and management, policy development and analysis, program development and evaluation, staff selection and development, school-community relations, and curriculum development.
4.3. Focus on Achievement - Use observation and collaboration to design, implement and evaluate meaningful and effective experiences that improve student achievement.
5. Leadership - Educational leaders have an evolved leadership style that supports the development of others, is harmonious with democratic principles and best educational practices and focuses on student achievement as the ultimate goal of the learning community. They use their knowledge and skills to:5.1. Leading - Provide leadership in various roles and responsibilities to work effectively in achieving common goals.5.2. Motivating, Coaching, and Delegating - Train, coach, delegate, and motivate faculty, staff, students, parents and other members of the
educational community.5.3. Supervision - Evaluate teaching and staff performance using a variety of supervisory models.5.4. Professionalism - Meet research, teaching, and service requirements of faculty positions in public, private or higher education.5.5. Change Management - Develop clear initiatives and adapt his/her leadership behavior to assist an educational institution, district or school
reach its vision, mission, and goals in a changing environment.5.6. Contemporary Issues - Promote the success of all students by understanding, responding to and influencing the larger political, social,
economic, legal and cultural context on the local, state and national levels.5.7. Curriculum, Instruction, and Assessment - Use best practices and sound educational research to improve instructional programs.5.8. Data-driven Decision Making - Apply data and information to solve educational problems.5.9. Managing Resources - Manage the organization, operations and resources in a way that promotes a safe, efficient and effective learning
environment6. Ethics/Principles - Educational leaders are bold in exhibiting and expecting accountability in integrity, fairness and ethical behavior. Their
actions6.1. Integrity - Demonstrate a respect for the rights of others with regard to confidentiality and honesty
6.2. Fairness – Demonstrate their ability to combine impartiality, sensitivity to diversity and a concern for others in dealing with all constituencies
6.3. Ethical Behavior - Demonstrate personal and professional values of their organizations and the ideals of our democratic society7. Diversity – Educational leaders appreciate the value of every individual and are committed to their success. They use their knowledge and skills
to:7.1. Managing – Design and implement educational policies, instruction and curricular reform that reflect culture and diversity.7.2. Theory & Research - Apply human development theory, proven learning and motivational theories, and concern for diversity to the
learning process7.3. Leading - Capitalize on diversity to create a culture that promotes respect and success for all students.
8. Internship - Through a program of field study, educational leaders have had the opportunity to demonstrate their knowledge and skills situations that contribute to their professional development. They have used their knowledge and skills to:8.1. Applying - Participate in a sustained and extensive internship activity that provides opportunities to apply skills and to develop new skills in
real world settings.8.2. Facilitating and Engaging - Arrange and engage in course embedded field experiences that use best practices and sound educational
research to improve instructional programs.9. Human Growth & Development - Educational leaders consider the personal and professional continuum of development of their various
constituencies and use this knowledge in all phases of their work from curriculum planning to staff development. They use their knowledge of human intellectual, physical, social and emotional growth to:9.1. Human Development – Help form comprehensive professional growth plans that focus on authentic problems and tasks, mentoring,
coaching, observation, conferencing and collaborative reflection, and other techniques that promote new knowledge and workplace skills.9.2. Organizational Development – Design procedures and techniques of inquiry-based organizational development and leadership to assist
school and district personnel in understanding and applying best practices for student learning.9.3. Personal Leadership – Promote the success of all students of a school, district or educational institution of higher learning of learning by
facilitating the development, articulation, implementation, and stewardship of a vision supported by the educational community.9.4. Lifelong Learning - Develop personal professional growth plans that reflect commitment to lifelong learning and best practices. Continuously
access, reflect on, and integrate knowledge to remain current in educational profession and practice.10. Program Specific – Curriculum and Instruction
10.1.1. Application of Curriculum & Instruction – Formulate instructional design competencies related to strategies for growth in curriculum planning, decision making, and problem solving and change management.
10.1.2. Analysis of Curriculum & Instruction Principles - Use supported research and theory to plan, implement and evaluate curricular and instructional practices.
10.1.3. Curriculum Theory & Design - Demonstrate advanced skills in synthesizing and merging curriculum and instructional theory to design, implement and assess instructional programs that lead to improved student achievement.
10.1.4. Evaluation - Evaluate the instructional processes, strategies, techniques and methodologies (e.g., strategies for growth in planning, decision-making, problem solving, and change management) in an educational institution.
10.1.5. Monitoring – Monitor and evaluate operational systems to ensure they enhance student learning and reflect accountability to the community.
10.1.6. Promote Technology - Use and promote technology and information systems to enrich district curriculum and instruction, monitor instructional practices and provide assistance to administrators who have needs for improvement
10.1.7. Staff Development - Assist school and district personnel in understanding and applying best practices for student learning
10.1.8. Professional Responsibility - Apply theoretical concepts for all levels of curriculum design, implementation and assessment of educational preparation programs
Course Objectives: 1. Define curriculum and program evaluation.2. Discern the differences between research and evaluation and identify which is more appropriate to solving an educational problem.3. Identify an appropriate evaluability assessment tool and apply it to a program or curriculum to determine feasibility of evaluation.4. Construct a blueprint for a curriculum or program evaluation5. Conduct appropriate “scoping” activities to focus and inform the evaluation plan.6. Identify appropriate sources of information to be used for data collection in a program or curriculum evaluation.7. Identify the most appropriate data collection method to be used in a program or curriculum evaluation.8. Determine the reliability and validity of data collection instruments to be used in a program or curriculum evaluation.9. Identify the purpose of a study, research methodology, data collection methods, statistical analyses used, and results of the study and will be able
to discuss the implications of the study as related to best practice in program and curriculum evaluation.10. Identify and apply commonly used quantitative procedures to measure program or curriculum outcomes.11. Identify threats related to the internal and external validity related to the research methodology used to evaluate a program or curriculum and
use this information to determine the most appropriate research design.12. Use statistical analyses to determine the effectiveness of a program or curriculum and will be able to translate the numbers into words so that
users can read and understand the information to improve the program or curriculum being evaluated.13. Identify the best presentation method of the results of a program or curriculum evaluation and present the information in a manner that is
understood by users.14. Identify an appropriate method to analyze data collected during a program or curriculum evaluation in order to glean the most important results
to communicate to users.15. Lead an evaluation team in identifying information to be summarized and revealed to users related to a program or curriculum evaluation.16. Identify potential ethical issues related to the evaluation of a curriculum or program and apply appropriate problem-solving strategies to reduce
the likelihood that these issues will affect the implementation or results of the evaluation.17. Construct a timely work plan and cost-effective budget related to the implementation of a program or curriculum evaluation. Assignment Table Module Module Topics Readings Assignments
1Friday
Approaches to Curriculum and Program Evaluation
Difference Between Research and Evaluation
Standards of Effective Evaluation Issues and Problems Associated
Fitzpatrick, Sanders & Worthen
Assignments: 1. Course Project:
Design Paper on Curriculum and Program Evaluation Methods
Overview:
With Evaluation
This project is designed to provide you with on-the-job training. In this project you will develop an evaluation design paper for a curriculum or program. You will be expected to conduct the scoping activities and design the evaluation. Scoping out the evaluation entails collecting information on the program or curriculum through interviews with key contacts (decision-makers, staff, etc.) on current information needs, and conducting a synthesis of past-related research and evaluation studies. After the focus of the evaluation is identified, the project will involve laying out an evaluation design, data collection plan, analysis plan, budget, and briefing and presentation plan. The design should be developed with clear awareness of the political aspects of the situation and tailored to the needs of the agency, organization, or school leadership. Strategies for encouraging the use of the resulting evaluation findings should also be discussed. a. Research an organization of your acquaintance (you might
chose your educational university). Identify at least three evaluation objects that you believe would be appropriate for study. For each of the objects, identify:i) The purpose the evaluation study would serveii) The basic focus of the evaluation
b. Next, identify a program to evaluate. In one or two pages, provide a broad overview of the evaluation, including its purpose, methods, and likely uses.i) Define the program you are evaluating. Is it at the school,
county, district, or state level?ii) Who are the users of the program?iii) Who are the stakeholders?iv) Why does the program exist? v) What are the goals and objectives of the program?vi) What are the fundamental elements of the program?
2
Saturday
Deciding Whether to Evaluate or Not: Evaluability Assessments
Types of Evaluation Curriculum Evaluation
Fitzpatrick, Sanders & Worthen Chapter 4Chapter 5Chapter 6Chapter 7Chapter 8
Discussion Questions: 1. Describe how performance measurement or standards-based
education has influenced the work of your school or organization. Are these “evaluation measures” useful for educators, students, and stakeholders? Give reasons.
Identify an evaluability assessment tool your organization uses or one
Read Hypothetical Project:
Science Teacher Enhancement: Introducing Faculty in Middle Schools to New Concepts and Methods in Science Instruction
that you have used. Provide a copy of it, explain its use, and effectiveness, and discuss recommendations for future use of the tool.
2. Analyze the advantages and disadvantages of an evaluator acting as
an advocate? Describe a situation in which it would be appropriate for the evaluator to serve as an advocate and a situation in which it would be inappropriate. Justify your position.
3. As a potential evaluator, how do you think evaluability assessment
might help inform your process? How might it help the users of information?
Assignments: 1. Application
Complete assignments as assigned by the facilitator, and submit it to the Discussion Board. Remember to follow APA 5th edition style. Your paper should be double-spaced and in 12pt font. It should not exceed five pages, and should include an additional page that lists your citations. Also, post a summary of your findings to the Discussion Area.a. The city jail in a mid-size Florida city has experienced two cases
of suicide by inmates in the last few months. As a result, the city council has hired a new director for the jail. The new director recognized that the jail has quite a few problems with officers’ and guards’ morale, physical facilities, training, security, and other issues and has decided to call in experts to advise him on the status of the jail. You’ve been hired to coordinate the evaluation. Provide detailed information on the types of evaluation question you will be asking, what types of data will be collected, and your justification for the type of evaluation model you will employ.
b. Imagine that you have been asked to conduct a project
evaluation of a confidence-building program for adult graduate students, who have returned to school after being out for several years. The program includes components that have students identify what their academic strengths and weaknesses are from their perspective as well as from the perspective of their prior
teachers, friends, and employers. Students are also asked to identify what types of instructional activities they prefer and to create short-term and long-term goals related to their academic career. Based on your background, philosophy of education and methodological preferences, identify an evaluation model that you would use. List three specific questions you would be interested in asking the students of the institution. Also, discuss whether you would focus more on the individual or the group?
2. Project
Conduct an evaluability assessment and draw a logic model of the program.Your project paper should be in Microsoft Word 2000 or higher. Follow APA 5th edition style. Your paper should be double-spaced and in 12pt font. Submit it to the Discussion Thread for review and critique.
3Saturda
y
Scoping it Out Steps in Developing a Curriculum
Evaluation Plan
Fitzpatrick, Sanders & WorthenChapter 9Chapter 10Chapter 11Chapter 12
Discussion Questions: 1. The object of an evaluation is the phenomenon being studied. What
questions could you ask to further define what needs to be studied? Elucidate why is important to accurately describe the object of the evaluation. Examine the potential results, if this is ill defined.
2. Describe why it is important to consider all the different audiences
for an evaluation? Which audiences do you think are typically viewed as the most important? Which are overlooked? Distinguish the terms “audience” and “user.” Examine the implications for the written product that an evaluation will yield.
3. Consider how the political context of an organization may affect the
approach of an evaluation, the nature of the information collected, and the interpretation and use of these results.
Assignments: 1. Application
Complete one of the assignments as directed by the Facilitator and submit to the Discussion Board.a. Identify a commercially prepared curriculum material that you
use, or consider using for instruction with your students. This can be a text/student manual, software, video, or other item. Using the Program Evaluation Standards introduced in the lecture, evaluate the item and prepare an evaluation report. Your report must include the items listed in the standards as well as a description of the material. The report must include justification for your ratings.
b. Discuss the advantages and possible drawbacks of using an
expert or outside consultant to generate the questions related to an evaluation.
2. Project
Remember to follow APA 5th edition style. Your paper should be double-spaced and in 12pt font. It should not exceed five pages, and should include an additional page that lists your citations. Also, post a summary of your findings to the Discussion Area.
Program Description Provide a description of the program including a flow-chart, and whatever materials will help us understand the nature of the program you are evaluating-how it works. Use the program evaluation worksheet to begin your project. Use this link as a point of reference:An Evaluation Guide for Districts and Schools
Address the following areas: Provide an introduction and problem statement. Explain the issues, hypotheses, and questions that you want to
study. Why did you select these hypotheses or questions to study? Why are they important? Provide a good, clear justification for why this study would be useful.
What questions will your project evaluation attempt to address? What do you want to know and how will you know it? What information do you need to answer the questions? When is the evaluation needed? What resources do you need?
4 Collecting Information Formative and Summative
Fitzpatrick, Sanders & Worthen
Discussion Questions:
Sunday Evaluations
Chapter 14Chapter 15
1. Discuss the pros and cons of using observations as a data collection method. Describe types of information that can be usefully collected through observation. Discuss some program or learning outcomes that might be difficult to measure through observations.
2. Describe an instance/situation when you might consider a
telephone interview instead of a paper-and-pencil measure or a face-to-face interview.
3. Compare and contrast the relative merits and demerits of telephonic
interviews, electronic surveys, and face-to-face surveys. 4. Speculate how an evaluator might confuse statistical significance
with practical significance. Assignments: 1. Application
Complete assignments as directed by the facilitator, and submit it to the Discussion Board.a. Choose from the following evaluation studies that use
qualitative methodology and prepare a report in the following format:i) A brief description of the focus and findings;ii) Identification of the key evaluation questions addressed;iii) A brief summary of the research design and data collection
methods used; andiv) A systematic list of threats to the: measurement validity,
internal validity, external validity, statistical conclusion validity, and reliability of the findings.
Evaluation StudiesThe Impacts of Regular Upward Bound: Results from the Third Follow-Up Data CollectionWhen Schools Stay Open Late: The National Evaluation of the 21st Century Community Learning Centers Program: Final ReportEvaluation of Literacy/Adult Basic Education Program 2001-2002
b. Interview someone who has conducted an evaluation study. Ask the person how the design was developed. What issues were the most troublesome at the planning stage? How did the person involve stakeholders on the issues? On which issues did the stakeholders play a major role? On which issues did the evaluator hold more decision-making power? Give reasons. How would you have developed the same study? What aspects of the plan would you now incorporate into your own matrix?
2. Project
Collecting Qualitative Evaluation DataMethodology: Clearly state the qualitative design and methods that will be used.a. What type of design are you using? Why?b. What threats to validity are controlled? What threats are
uncontrolled? Why?c. What are the sources of the data? How reliable and valid are the
data?d. What statistics are used to analyze the data?e. Your project paper should be in Microsoft Word 2000 or higher.
Follow APA 5th edition style. Your paper should be double-spaced and in 12pt font.
5
Friday
Collecting Quantitative Information
Analyzing Quantitative Information
Interpreting Quantitative Information
Fitzpatrick, Sanders & WorthenChapter 15
Discussion Questions: 1. On what basis would an evaluator select the use of descriptive or
inferential statistics? Justify your response with examples. 2. What kind of technical problems could an evaluator face when
collecting quantitative data? How would you prevent these or solve them if they happened?
3. Devise a quantitative method that could be used to answer the
following questions: 4. Did the new reading curriculum result in improved reading scores
on the state test? 5. Do the methods of instruction faculty are currently using in class
correspond to the principles of adult learning theory that were
presented in training last fall? 6. What types of recruitment strategies are most effective for
attracting students to Argosy education programs? Assignments: 1. Application
Complete any one assignment assigned by the facilitator, and submit it to the Discussion Thread.a. Choose from three different evaluation studies that use
quantitative methodology and prepare a report in the following format:i) A brief description of the focus and findings;ii) An identification of the key evaluation questions addressed;iii) A brief summary of the research design and data collection
methods used;iv) A systematic list of threats to the: measurement, validity,
internal validity, external validity, statistical, conclusion validity, and reliability of the findings.
b. Some educators think that random assignment to groups in the
classroom environment is unethical. What are the arguments for and against this position? Describe a situation in which you would feel comfortable using random assignment. Justify your position.
2. Project
Collecting Quantitative Evaluation DataClearly outline the quantitative design and methods that will be used in your project. Address the following questions:a. What type of design are you using? Why?b. What threats to validity are controlled?c. What threats are uncontrolled? Why?d. What are the sources of the data?e. How reliable and valid is the data?f. What statistics are used to analyze the data?g. Submit your paper as a Word document to the Discussion
Thread. Your paper should be double-spaced and in 12-pt font. Include an additional page that lists your citations. Follow APA
5th edition style guide.6
Saturday
Techniques for Qualitative Evaluation
Fitzpatrick, Sanders & WorthenChapter 15
Discussion Questions: 1. Discuss the pros and cons of using observations as a data collection
method. Describe types of information that can be usefully collected through observation. What are some program or learning outcomes that might be difficult to measure through observations? How might an evaluator encourage full participation of the evaluation client, stakeholders, and people from whom data are collected? What advantages does this participation provide?
2. Examine when it might be better to use a telephone interview
instead of a paper-and-pencil measure or a face-to-face interview? “Data must be interpreted, not simply analyzed.” Elucidate.
3. Assess a situation when you might use the following methods of
organizing and reporting qualitative data: storytelling, case studies, analytic framework (processes, issues.)
Assignments: 1. Application
Complete assignments as directed by the facilitator, and submit it to the Discussion Board.a. Describe a setting where you might use a focus group. Submit a
set of questions you would use to guide the discussion or interview.
b. Write a two-page essay on how an evaluator might confuse statistical significance with practical significance. Provide examples.
Remember to follow APA 5th edition style. Your paper should be double-spaced and in 12pt font. It should not exceed five pages, and should include an additional page that lists your citations. Also, post a summary of your findings to the Discussion Area.
7
Saturday
Using Information Managing and Using Evaluation
Findings
Fitzpatrick, Sanders & WorthenChapter 16Chapter 18
Discussion Questions: 1. Elucidate the statement, “Evaluation use may be better
conceptualized as influence.” pp. 409. Use APA Style. Fitzpatrick, J.L., Sanders, J.R., & Worthen, B.R. (year). Program evaluation: Alternative approaches and practical guidelines (3rd ed.). Allyn & Bacon. (pp. 409).
2. Evaluations have three levels of impact: individual level interactions
and on the group or organizational level. Discuss how an evaluator might go about identifying potential pathways between these different levels in order to achieve the desired influence with a specific evaluation?
3. Describe the benefit of conducting an audience analysis in terms of
timing, mode, and the types of information prior to conducting an evaluation.
4. Discuss the potential benefits and pitfalls of using electronic
communication to report evaluation findings. Assignments: 1. Application
Complete one of the assignments as directed by the Facilitator and submit to the Discussion Board.a. Research and find an evaluation report and use the checklist on
p. 400 of the textbook Program evaluation: Alternative approaches and practical guidelines to conduct a gap analysis and make recommendations for improvement.
b. Identify an object in your institution/organization for
evaluation. Perform an audience analysis to determine the information needs, and the best medium to present the information.
Remember to follow APA 5th edition style. Your paper should be double-spaced and in 12pt font. It should not exceed five pages, and should include an additional page that lists your citations. Also, post a
summary of your findings to the Discussion Area. 2. Project: Presentation of Results and Utilization Plan
Present “dummy” tables and show what kinds of statistics you would use, what kind of analysis would be conducted? What graphical aids would be used to present the information? What strategies can be deployed for presenting the results to key stakeholders and decision-makers and strategies for facilitating utilization?
Format: Your project paper should be in Microsoft Word 2000 or higher. Follow APA 5th edition style. Your paper should be double-spaced and in 12-pt font.
8
Sunday
Pitfalls and Solution Project Presentation
Fitzpatrick, Sanders & WorthenChapter 17Chapter 21
Discussion Questions: 1. Write a short “letter to the successor” for future students who take
this course. Discuss your key learning. What questions do you still have?
2. Discuss why there is a need for explicit ethical standards to be
defined and adhered to in evaluation. What benefits does this provide to the evaluator, stakeholders, and organization?
3. Are evaluations ever really free of bias? Describe how an evaluator
might go about acknowledging a bias that might affect evaluation findings. Justify both of these questions with proof from the literature or text.
Grading Criteria: Grading Scale Grading requirements
A 100 -93
A- 92 - 90
B+ 89 - 88
B 87 - 83
B- 82 – 80
C+ 79 – 78
C 77 – 73
C_ 72 – 70
F 69 and below
Attendance/participation/ Assignments
42 pts
Discussion Leadership/ 3-5 page paper
14 pts
Discussion Leadership/ presentation
9 pts
Final Project 35 ptsTotal 100 ptsDiscussion Leadership on the first weekend.
+ 5 points
1. Discussion Leadership/ Paper and Presentation . Each student will be assigned as a discussion leader. I will assign topics in the first week of January and inform you by e-mail. The specific activity will include responsibility for 2 or more discussion questions or one or more application assignments. You may provide me with at least two preferences.
Because of the nature of the course schedule, if you are assigned to the first weekend, you will be awarded five points on top of the score that you earn. Feel free to send me an e-mail to let me know your interest at [email protected] before January 5.
Your paper will be 4-5 pages; the presentation should include no more than ten PowerPoint slides that will be distributed to the class. The presentation will consume no more than 45 minutes and no less than 25 minutes.
Assignments Student______________________
Module 1: _____________________________ _____________________________Module 2: _____________________________ _____________________________Module 3: _____________________________ _____________________________Module 4: _____________________________ _____________________________Module 5: ._____________________________ _____________________________Module 6: _____________________________ _____________________________Module 7: _____________________________ _____________________________
Two modules will be completed in the online classroom.
2. Attendance/Participation/Weekly Assignments . As professionals, you have many obligations and the instructor is not the best judge of which requires your attention and presence when a conflict emerges. The instructor’s assessment is based on evaluation of learning and contribution to the learning of others. The instructor recognizes that personal emergencies occur over which an individual has little or no control. In the case of an absence, the instructor will award less than half of the allowable points for the week, for one absence, upon e-mail receipt of a 2-3 page summary of the reading.
Everyone should prepare, each week, to participate in discussing the assignments.
When class is online, an initial post shall be made by Tuesday, 11:59 p.m. During the remainder of the week, you are required to reply to a class colleague’s post and respond to all questions asked of you. The final `post shall be made by Saturday, 11:59 p.m.
Participation / Learning Activities Evaluation Rubric
Excellent Good Fair PoorParticipation and responses met or exceeded expectations, strongly engaged, high quality of exploration. Shows a confident understanding of the concepts and supports arguments with effective reasons and evidence. Actively engaged in discussion with other classmates.
Strongly engaged, high quality of exploration. Student understands the concepts, but lacks the clarity in argument. Anecdotal responses from personal experiences exceed concepts from text/chapters. Moderately engaged in discussion with other classmates.
Responses met or exceeded expectations, strongly engaged, high quality of exploration. Shows a confident understanding of the concepts and supports arguments with effective reasons and evidence. Did not engage in discussion with other classmates.
Discussion questions were not answered on time and/or did not engage in discussion with other classmates. Did not complete assignment.
3. Final Project . See final project information embedded in the course outline.
Library Library Resources: Argosy University’s core online collection features nearly 21,000 full-text journals and 23,000 electronic books and other content covering all academic subject areas including Business & Economics, Career & General Education, Computers, Engineering & Applied Science, Humanities, Science, Medicine & Allied Health, and Social & Behavior Sciences. Many titles are directly accessible through the Online Public Access Catalog at http://library.argosyu.edu. All resources in Argosy University’s online collection are available through the Internet. The library will provide students with links, user IDs, and passwords. A detailed list of online resources is located at http://library.argosyu.edu/misc/onlinedblist.html. In addition to online resources, Argosy University’s onsite collections contain a wealth of subject-specific research materials searchable in the Online Public Access Catalog. Catalog searching is easily limited to individual campus collections. Alternatively, students can search combined collections of all Argosy University Libraries. Students are encouraged to seek research and reference assistance from campus librarians. Information Literacy: Argosy University’s Information Literacy Tutorial was developed to teach students fundamental and transferable research skills. The tutorial consists of five modules where students learn to select sources appropriate for academic-level research, search periodical indexes and search engines, and evaluate and cite information. In the tutorial, students study concepts and practice them through interactions. At the conclusion of each module, they can test their comprehension and receive immediate feedback. Each module takes less than 20 minutes to complete. Please view the tutorial at http://library.argosyu.edu/infolit/ Academic Policies
Academic Dishonesty/Plagiarism: In an effort to foster a spirit of honesty and integrity during the learning process, Argosy University requires that the submission of all course assignments represent the original work produced by that student. All sources must be documented through normal scholarly references/citations and all work must be submitted using the Publication Manual of the American Psychological Association, 5th Edition (2001). Washington DC: American Psychological Association (APA) format. Please refer to Appendix A in the Publication Manual of the American Psychological Association, 5th Edition for thesis and paper format. Students are encouraged to purchase this manual (required in some courses) and become familiar with its content as well as consult the Argosy University catalog for further information regarding academic dishonesty and plagiarism. Scholarly writing: The faculty at Argosy University is dedicated to providing a learning environment that supports scholarly and ethical writing, free from academic dishonesty and plagiarism. This includes the proper and appropriate referencing of all sources. You may be asked to submit your course assignments through “Turnitin,” (www.turnitin.com), an online resource established to help educators develop writing/research skills and detect potential cases of academic dishonesty. Turnitin compares submitted papers to billions of pages of content and provides a comparison report to your instructor. This comparison detects papers that share common information and duplicative language. Americans with Disabilities Act Policy It is the policy of Argosy University to make reasonable accommodations for qualified students with disabilities, in accordance with the Americans with Disabilities Act (ADA). If a student with disabilities needs accommodations, the student must notify the Director of Student Services. Procedures for documenting student disability and the development of reasonable accommodations will be provided to the student upon request. Students will be notified by the Director of Student Services when each request for accommodation is approved or denied in writing via a designated form. To receive accommodation in class, it is the student’s responsibility to present the form (at his or her discretion) to the instructor. In an effort to protect student privacy, the Department of Student Services will not discuss the accommodation needs of any student with instructors. Faculty may not make accommodations for individuals who have not been approved in this manner. The Argosy University Statement Regarding Diversity Argosy University prepares students to serve populations with diverse social, ethnic, economic, and educational experiences. Boththe academic and training curricula are designed to provide an environment in which students can develop the skills and attitudes essential to working with people from a wide range of backgrounds.
