“Unmuddying” Course Content Using Muddiest Point Reflections
Adam Carberry
College of Technology & Innovation Arizona State University
Mesa, AZ, USA [email protected]
Stephen Krause & Casey Ankeny School of Engineering Matter, Transport, & Energy
Arizona State University Tempe, AZ, USA
[email protected]; [email protected]
Cynthia Waters Mechanical Engineering
North Carolina A&T State University Greensboro, NC, USA
Abstract—Class instruction is a living and ever evolving process aimed at providing students with a quality education. Instructors are responsible for analyzing their courses to ensure that delivery of information is effective. Changes made are usually based on student assessments; however, our reactions to assessments are flawed without student insight. One method to obtain student feedback is through muddiest point reflections. This activity asks students to reflect on what was just taught allowing students the opportunity to share what was “muddy”. This mixed-methods study provides vignettes from faculty members on their use of muddiest point reflections and an assessment of what value students associate with such an intervention. Faculty members who have used this approach say it drives change within their classes. The analysis of student value beliefs revealed muddiest point reflections as an intervention that positively impacts interest, attainment, and utility value without negative cost. The appeal of muddiest points was also evident with 77% of students hoping to see muddiest point reflections in another class and 93% agreeing to recommend their course experience to a friend. These findings suggest that students agree more than disagree that muddiest point reflections are a valuable addition to their educational experience.
Keywords – muddiest point reflection, curricular change, formative feedback, associated value
I. INTRODUCTION One of the primary roles of faculty is the responsibility to
teach students. Additional responsibilities can sometimes diminish the priority level placed on teaching, which in turn prevents many faculty members from putting in the necessary time to evolve their teaching practices. Preparing courses, especially those taught year after year, can eventually lead faculty to fall into the trap of simply using notes or slides from past years to save time. This approach may save time, but fails to meet the changing needs of our students who have high expectations from their instructors.
One very simple method to improve teaching is to employ muddiest point reflections. Muddiest point reflections involve simply asking students to anonymously reflect on what was “muddy”, i.e. confusing, during class and to rank their level of confusion. Reflection can occur by providing students with a document to record difficulties throughout the class or by dedicating five minutes at the end of class. This intervention provides an opportunity for students to reflect in a way that is beneficial for both them and instructors; students reflect on what they understood from the class, while instructors obtain valuable insights into what was confusing for students. This direct feedback allows instructors to instantly reflect on how well they taught the course content and how they may change their delivery or pedagogical approach next time. Additionally, taking five minutes of time at the beginning of the next class meeting provides instructors with an opportunity to cover the material again in an effort to address student concerns and misconceptions. This short, noninvasive effort not only addresses students falling behind, but also shows students a commitment to their education especially when the instructor puts direct student quotes on the screen. In effect, the “muddiest point” technique establishes a critical student/instructor dialogue.
The following study reports the use of muddiest point reflections in three material science and manufacturing courses. These reflections have revealed many confusing topics such as secondary bonding, Miller indices, and eutectic phase diagrams to name a few. Emerging muddiest point topics have led to various changes in the course content and delivery for these instructors as well as increased interest from other faculty members. Specific efforts undertaken by these three instructors include creating YouTube Muddiest Point video tutorials, building a FAQ resources website, implementing preview problems based on the next class’s content, and embedding active student-centered pedagogical approaches. These efforts have shown dividends in a short
978-1-4673-5261-1/13/$31.00 ©2013 IEEE
period of time as demonstrated by the over 33,000 hits in less than seven months for the twelve video tutorials posted and the 93% of students who indicated the videos supporting their learning.
The goal of this paper is to share instructor experiences with muddiest points focusing on how these reflections have impacted their courses. Additionally, this paper will investigate the following research question:
What value do students place on muddiest point reflections?
II. BACKGROUND
A. What are Muddiest Point Reflections? “Muddiest points” are unclear concepts generated by
students. In 1988, Frederick Mosteller first implemented muddiest points through the solicitation of responses to the following three questions: 1) What was the most important point in lecture?, 2) What was the muddiest?, and 3) What would you like to hear more about? After analyzing the student comments, Mosteller addressed issues using both class time and handouts. Anecdotally, Mosteller stated that although this activity required class time, the students enjoyed the exercise and he was able to modify 15% of his class behavior [1, 2]. Also in 1988, Angelo and Cross highlighted “muddiest points” as a subsection of one of thirty classroom assessment technique classifications [3]. It has been suggested that the collection of most interesting points is a beneficial addition to balance perceived negativity by focusing only on muddiest points [4]. In summary, the seminal publications of Mosteller, Angelo, and Cross launched the widespread, but under-analyzed, use of “muddiest points” today.
B. Motivation for using Muddiest Points The use of muddiest points is evident by the over 8,200
hits revealed when searching Google Scholar. The success of “muddiest points” is due in part to its ease-of-use and large impact. Muddiest point requires very little time and minimal change to the syllabus. It allows students to reflect, retain, synthesize and build knowledge. Muddiest points not only benefits students but faculty as well. It allows faculty to more effectively question students, assess their difficulties, identify next steps, and help modify lectures for future semesters [5].
C. How to Collect Muddiest Points Muddiest points may be implemented through a few simple
steps, outlined by several groups [5-7]. First, instructors should hand out muddiest point sheets/cards at the beginning of class and remind students to begin filling them out, anonymously, 15 minutes before the end of class. Instructors should then collect and record responses (in an Excel spreadsheet, for example). The responses must then be analyzed so the instructor can identify the most relevant issues requiring a strategized response. We are proponents for use of direct quotes when responding to student misconceptions as “student voice” strengthens student-instructor interactions[8].
D. Cyber Implementation of Muddiest Points The original method of collecting muddiest points used
blank index cards [5]. Today, there are many media that can be utilized to collect muddiest points. For this study, researchers used custom forms requesting both muddiest and interesting points which facilitated the use of student quotes when responding. Alternatively, others used in-class clickers and online multiple-choice forms with generalized topics as choices to quickly and quantitatively assess the muddiest points [9, 10]. Similarly, there are many options for responding to muddiest points including the creation of screencasts and podcasts [10-12].
E. Impact of Muddiest Points Although the use of “muddiest points” is widespread,
limited work has been done to determine the student value of this exercise. King determined that the “muddiest points” technique was favored by the majority of the students as demonstrated by their response rate. More specifically, 75% of students that answered other subject-based clicker questions also answered the clicker question on the muddiest point. King’s study also uncovered that, although the majority of topics were conceptual, the majority of students had trouble with quantification [9]. Pinder-Grover, et al. showed that final grade correlated with frequency of use of muddiest point screencasts [10]. Here, we hope to further characterize the impact of “muddiest points” on engineering students by examining the efficacy of determining its value and cost to students.
Our efforts are supported by expectancy-value theory and the significant body of work indicating that the value of learning a specific domain predict the amount of effort they will put into learning and the quality of that learning [13-17]. Expectancy-value theory states that associated value is determined by the level of interest, capability of attainment, utility toward goals, and cost associated with a given activity. Research has clearly demonstrated that college student’ effort and achievement are influenced by the connections they make between the content of their courses and their personal futures [18, 19]. Belief about the utility of learning activities for achieving future goals is called Perceptions of Instrumentality. If students believe that mastering the content of their coursework is important for their future, then they will be more willing to put forth effort. Students who are interested in the short-term value of what they are learning are more likely to use learning strategies that facilitate quick learning, rather than deep understanding. Students will also expend less effort trying to master the tasks at hand [20].
III. METHODS The following mixed methods study is broken up into two
distinct components. First, we include three vignettes by instructors discussing the impact and perception of muddiest points from the their perspective. Second, we surveyed students in the three instructors’ courses to assess their perceptions of associated value toward muddiest point reflections.
A. Instructor Sample The instructors include three faculty members teaching
materials science and/or manufacturing courses in three different engineering degree programs. Each faculty has utilized muddiest point reflections for a minimum of two semesters as a way to impact change within their class.
B. Student Sample Our student sample included 109 students enrolled in the
three material science and manufacturing courses described in the faculty vignettes. Students ranged from sophomores to seniors across various engineering majors.
C. Instrument Design An instrument was created for this study to investigate
student beliefs towards the value of muddiest point reflections. The developed instrument used three sources to establish items: 1) literature regarding value and motivation [17, 21-28], 2) a value survey previously used to assess grading system beliefs and impact of peer mentoring [29, 30], and 3) an uncharacterized attitudinal survey developed by one of the instructors for his course. The instrument consisted of four broad questions with additional items associated with the first two questions (see Appendix I).
Each item was placed on a 4-point agree-disagree Likert scale (completely agree – agree more than disagree – disagree more than agree – completely disagree). Items included in questions 1 and 2 were derived using the previously described expectancy-value theory. The remaining two questions were included to determine whether students would like to see increased use of muddiest point reflections and if they would recommend the course they are currently taking to a friend.
D. Data Collection and Analysis The survey was administered by the instructors in their
classes; two instructors administered the survey using paper and pencil while the third administered the survey online via Qualtrics©. Student identities were completely anonymous. Each respondent was required to answer all of the questions to complete the survey. The results were tested for validity and reliability using factor analysis and internal consistency (Cronbach’s α). Mean score differences were then used to assess the value students believed muddiest point reflections had for their respective courses and future goals.
IV. RESULTS
A. Instructor Vignettes Through autoethnographies— “studying one’s own culture
and oneself as part of that culture” [29]—we provide vignettes of three instructor experiences using muddiest points in materials science and/or manufacturing courses.
Vignette 1 – The first vignette looks at a course titled “Structure and Properties of Materials”. This course covers the basic concepts of material structure and its relation to properties contextualized by real-world engineering applications. It is a required course for Mechanical, Industrial,
and Materials Engineering majors usually taken freshmen/sophomore year.
“Initially, when I started to use ‘muddiest points’, the goal was to elicit confusing concepts from students in order to respond to them; however, over six semesters, the ‘muddiest point’ strategy has evolved into much more than responding to student issues. It opened a channel of communication and mutual trust between the students and me, which turned an instructor’s monologue into a student-instructor dialogue. As such, the interaction enhanced the student learning experience as well as the teacher’s instructional experience. Much more than ‘muddiest points’ are elicited from comments including alternate conceptions, vocabulary issues, rushed teaching, bad handwriting, questions of curiosity, and comments on related student experiences. Students can reflect on their learning over a class as well as discover the issues their fellow students are struggling with. With student ‘muddiest point’ input, improved teaching strategies can help diminish previously difficult conceptions. Overall, ‘muddiest points’ enhances both student and teacher metacognition about learning and instruction, respectively.”
Vignette 2 – The second vignette looks at a course titled “Engineering Materials and Manufacturing”. This course is a required fundamental sophomore course taken by students enrolled in a Bachelor’s of Science in Engineering program. The course covers the characterization, structure, and properties of materials, manufacturing processes, engineering metrology, quality assurance, and cost modeling.
“My foray into using muddiest point reflections began when a colleague suggested the tool as a method to engage students. After two semesters of using muddiest point reflections, I’ve come to the realization that the tool provides mutual gains for both my students and me. Muddy points revealed in my first semester have led me to make changes not only during the current semester, but beyond to the following year. The knowledge gained from my students has encouraged me to try new approaches including video tutorials, jigsaw activities, and in-class activities. As a professor, I like to believe that I can generally tell when something isn’t clear. Muddiest points is the tool that helps confirm or deny my intuitions”
Vignette 3 – The third vignette looks at a course titled “Modern Engineering Materials”. This course covers the role of materials in engineering, properties of materials, nonferrous and ferrous systems and applications, heat treatment and strengthening mechanisms, various polymeric, ceramic, composite materials biomaterials and their applications, failure theories, characterization, corrosion and environmental issues, and project work involving selection and design with various material systems.
“Working with a close colleague, I was encouraged to try addressing the current needs of students by using their own strengths and weaknesses. The ‘muddiest points’ technique is a form of Just-in-Time Teaching (JITT) that I find is much less intimidating than traditional JITT. The entire lecture is not modified to suit the previous students’ comments but the
students do feel they are a part of the process when their own words are used to clear up muddy points. Students are made a part of the learning team; they are the players and the faculty member is the coach unlike in a traditional classroom when the faculty is a performer on stage and the students are mere audience members.”
B. Value Survey Validity and Reliability The survey was first tested for validity and reliability.
Factor analysis revealed three factors with weights ranging from 0.710 to 0.890 (Table I).
TABLE I. FACTOR ANALYSIS
Item Factor 1 Factor 2 Factor 3
1a 0.749 1b 0.869 1c 0.783 1d 0.721 1e 0.832 1f 0.890 1g 0.779 2a 0.710 2b 0.785 2c 0.858 2d 0.847 2e 0.861
Expectancy-value theory was used as a means to obtain
construct validity. The construct of value described in the theory was tied to each of the three factors: Factor 1 – Intrinsic & Attainment Value; Factor 2 – Utility Value; and Factor 3 – Cost. Factor 1 included both intrinsic and attainment value as suggested by previous research using similar surveys [17, 18].
C. Associated Value of Muddiest Points The data from the validated instrument was then used to
assess student beliefs pertaining to the value of muddiest point reflections. To simplify the presentation of the results, agree and agree more than disagree responses as well as disagree and disagree more than agree responses were separately pooled together and presented as percentages (see Tables II – IV).
1) Interest/Attainment Value: Interest or intrinsic value is an individual’s anticipated enjoyment of engaging in a
TABLE II. INTEREST/ATTAINMENT VALUE RESULTS
Interest/Attainment Value Agree Disagree
motivated me to do well in the course 62% 38% was an effective way to increase my engagement in the course 81% 18% helped me better understand my own personal learning 71% 29% increased my level of responsibility 64% 36%
particular activity. Related to interest value is attainment value or an individual’s perception of how the activity contributes to the conception of who he or she is fundamentally. Results suggested that the majority of students found muddiest point reflection to positively impact their experience in the class.
2) Utility Value: Utility value is an individual’s perception of the advantages that result from engaging in the task for future goals or rewards. Results suggest that students overwhelming found the material learned in their course to be of value to them in their current and future endeavors as learners and professionals.
TABLE III. UTILITY VALUE RESULTS
Utility Value Agree Disagree
will be of value to me after graduation 83% 17% was useful in my pursuit of my career and/or graduate school goals 79% 20%
helped me see the relevance of engineering to the real world 89% 11%
helped me learn the importance of materials science to engineering 94% 6%
helped me learn the importance of manufacturing to engineering 90% 10%
3) Cost: Cost represents an individual’s perception of the
sacrifices required, including effort, time, and psychological impact, for successful impact of an activity. Results suggest that students did not find muddiest point reflections to be a frustrating activity that took too much time and effort.
TABLE IV. COST RESULTS
Cost Agree Disagree
required too much effort 17% 83% made me frustrated and anxious 11% 89% required too much time 14% 86%
4) Appeal: The final two questions pertained to the
students’ desire to see muddiest point reflection used in other courses as well as whether they would recommend a course with this muddiest point reflections to a friend. The majority of students (77%) agreed that they would like to see this tool used in their other courses, while overwhelmingly (93%) recommending the course to a friend. This final result is likely impacted by more than just the muddiest point reflections including the course content and the instructors themselves.
V. DISCUSSION The use of muddiest point reflections is a simple
intervention capable of major impact on the delivery of course content. The benefit to such formative feedback is the associated gain for both instructors and students. The instructor vignettes frame muddiest point reflections as a catalyst for change in pedagogical practice. The vignettes report muddiest points providing a “channel of communication and mutual trust” with the students. That channel provides a way for students and instructors to have a dialogue between one
another, in turn restructuring the course from teacher as “performer” to teacher as “coach”. This role change allows students to voice their opinions as a means to impact course content delivery. Instructors can then use student opinions to “confirm or deny intuitions” about what is being assumed to need changing.
From a student perspective, the survey revealed overwhelmingly positive value associated by students toward the muddiest point reflections. Students saw this opportunity as a way to positively impact interest, attainment, and utility value without too much negatively associated cost. Such results suggest that students found muddiest point reflections to improve the course in a way that made the course more enjoyable and valuable. This increase in value resulted in high appeal for the course by students and a likelihood of recommending the course to a friend. While this final result is likely impacted by the course content and the instructors themselves, the instructors can still appreciate that students view using such activities as a means to obtain positive student reviews.
VI. CONCLUDING REMARKS Our initial research demonstrated that reflections have
affected pedagogical changes in the classroom. Students found interest, attainment, and utility value in muddiest point reflections, while not costing them too much time and effort.
Our future work will focus on better quantifying improved student enrollment, achievement, and attitude due to implementation of muddiest points. Further, our continued goal with this research-to-practice effort is to bring greater awareness of muddiest point reflections to STEM faculty in an effort to continuously improve higher education.
ACKNOWLEDGMENTS This material is based upon work supported by the National Science Foundation under Grant No. 1226325.
REFERENCES [1] F. Mosteller, "Broadening the Scope of Statistics and Statistical
Education Broadening the Scope of Statistics and Statistical Education," The American Statistician, vol. 42, pp. 93-99, 1988.
[2] F. Mosteller, "The 'Muddiest Point in the Lecture' as a Feedback Device," The Journal of Harvard-Danforth Center, pp. 10-21, 1989.
[3] K. P. Cross and T. Angelo, Classroom Assessment Techniques: A Handbook for Faculty, pp. 1-168, 1988.
[4] S. Peltsverger, "Maintaining Balance between Theory and Practice in Information Security Courses," Information Security Curriculum Development Conference, pp. 59-60, 2011.
[5] A. Carlson, Classroom Assessment Technique: Muddiest Point, 2010. [6] J. Arter, Creating & Recognizing Quality Rubrics, 2012. [7] J. L. Faust and D. R. Paulson, "Active Learning in the College
Classroom," Journal on Excellence in College Teaching, vol. 9, pp. 3-24, 1998.
[8] R. Brooker and D. Macdonald, "Did we hear you?: Issues of student voice in a curriculum innovation " J. Curriculum Studies, vol. 31, pp. 83-97, 1999.
[9] D. B. King, "Using Clickers To Identify the Muddiest Points in Large Chemistry Classes," Journal of Chemical Education, vol. 88, pp. 1485-1488, 2011.
[10] T. Pinder-grover, et al., "The efficacy of screencasts to address the diverse academic needs of students in a large lecture course," Advances in Engineering Education, 2011.
[11] A. Chan and M. J. W. Lee, "An MP3 a Day Keeps the Worries Away : Exploring the use of podcasting to address preconceptions and alleviate pre-class anxiety amongst undergraduate information technology students," Student Experience Conference, pp. 58-70, 2005.
[12] S. Greer, "Clearing the muddiest point - podcasts as interactive feedback," in Teaching Cycle (PGCTHE portfolio), ed, 2011, pp. 1-10.
[13] J. S. Eccles, “Subjective task value and the Eccles et al. model of achievement-related choices,” in The Handbook of Competence and Motivation, A.J. Elliot and C.S. Dweck, Eds. New York, NY: Guilford Press, 2005, pp. 105-121.
[14] J. S. Eccles, “Who am I and what am I going to do with my life: Personal and collective identities as motivators of action.” Educational Psychologist. vol. 44, pp. 78-89, 2009.
[15] J. S. Eccles and A. Wigfield, "Motivational beliefs, values, and goals." Annual Review of Psychology, 53, 109-132, 2002.
[16] S. W. VanderStoep, P. R. Pintrich and A. Fagerlin, "Disciplinary differences in self-regulated learning in college students." Contemporary Educational Psychology, 21(4), 345-362, 1996.
[17] A. Wigfield and J. S. Eccles, “Expectancy-value theory of motivation.” Contemporary Educational Psychology. vol. 25, pp. 68-81, 2000.
[18] A. Malka and M. V. Covington, "Perceiving school performance as instrumental to future goal attainment: Effects on graded performance." Contemporary Educational Psychology, 30(1), 60-80, 2005.
[19] R. B. Miller, T. K. DeBacker and B. A. Greene, "Perceived instrumentality and academics: The link to task valuing. Journal of Instructional Psychology, 26(4), 250-260, 1999.
[20] J. Simons, S. Dewitte and W. Lens, W."Wanting to have vs. wanting to be: The effect of perceived instrumentality on goal orientation." British Journal of Psychology, 91(3), 335-351, 2000.
[21] A. K. Boggiano and T. S. Pittman, Achievement and motivation: A social-developmental perspective. Cambridge, UK: Cambridge University Press, 1992..
[22] J. Brophy, Motivating students to learn. Boston: McGraw-Hill, 1998. [23] E. L. Deci and R. M. Ryan, Intrinsic motivation and self-determination
in human behavior. New York: Plenum, 1985. [24] Eccles, J. S., & Wigfield, A. (1995). In the mind of the actor: The
structure of adolescents' achievement task values and expectancy-related beliefs. Personality and Social Psychology Bulletin, 21, 215-225.
[25] S. Hidi and J. M. Harackiewicz, "Motivating the academically unmotivated: A critical issue for the 21st century." Review of Educational Research, 70, 151-179, 2002.
[26] P. R. Pintrich and D. H. Schunk, Motivation in Education: Theory, Research, and Applications. Englewood Cliffs, NJ: Prentice Hall, 1996.
[27] S. Rimm, Why bright kids get poor grades and what you can do about it. New York: Crown, 1995.
[28] A. Wigfield, The role of children's achievement values in the self-regulation of their learning outcomes. In D. H. Schunk & B. J. Zimmerman (Eds.), Self-regulation of learning and performance: Issues and educational applications (pp. 101-124). Mahwah, NJ: Erlbaum, 1994.
[29] B. Sattler, A. Carberry, and L. D. Thomas, "Peer Mentoring: Linking the value of a reflective activity to graduate student development." Proceedings of the 2012 Frontiers in Education Annual Conference.
[30] A. R. Carberry, M. T. Siniawski, and J . D. N. Dionisio, "Standards-based grading: Preliminary studies to quantify changes in affective and cognitive student behaviorsl." Proceedings of the 2012 Frontiers in Education Annual Conference.
[31] M. Q. Patton, Qualitative research and evaluation methods. Sage Publications, Inc.: Thousand Oaks, CA, 2002.
APPENDIX I: STUDENT VALUE OF MUDDIEST POINT REFLECTIONS SURVEY Directions: Please read each of the following statements and indicate your level of agreement based on your class experience. Do NOT choose multiple letters for one question (Ex. B/C).
A: completely agree B: agree more than disagree C: disagree more than agree D: completely disagree
1. The “muddiest points” daily reflections used in this course…
a. motivated me to do well in the course. _____
b. required too much effort. _____
c. was an effective way to increase my engagement in the course. _____
d. helped me better understand my own personal learning. _____
e. made me frustrated and anxious. _____
f. required too much time. _____
g. increased my level of responsibility for my own learning. _____
2. The material learned in this course…
a. will be of value to me after graduation. _____
b. was useful in my pursuit of my career and/or graduate school goals. _____
c. helped me see the relevance of engineering to the real world. _____
d. helped me learn the importance of materials science to engineering. _____
e. helped me learn the importance of manufacturing to engineering. _____
3. I would like to see muddiest point reflections used in other courses. _____ 4. I would recommend this course to a friend. _____
Recommended
