July 2011

1

Lenovo Global Education Research Initiative 2011 Annual Report – University of North Carolina at Chapel Hill

Overview

In support of Lenovo’s Educational Research Initiative (ERI), the Center for Faculty Excellence (CFE) at the University of North Carolina at Chapel Hill is emphasizing innovative strategies for addressing the diverse learning needs of both students and faculty members in higher education. The focus reflects the Center’s belief that improving student learning outcomes is contingent on the quality and accessibility of professional development opportunities for higher education faculty.

During the past year, the CFE completed the second round of a small instructional innovation grants program with three specific foci for improving instruction: collaborative learning, global education and engaging large sections. A summary of highlights from the 2010-2011 projects is included in this report, along with brief descriptions of recently funded projects that will be implemented during 2011-2012. An overview of projects funded and implemented during 2009-2010 is available in last year’s ERI partner update for UNC-Chapel Hill.

Recently funded projects reflect two changes in program direction at UNC:

1. An increased emphasis on innovations in faculty development, in particular the pilot and full implementation of the Professional Interests Manager service;

2. Suspension of the small grants program in favor of a more targeted outreach approach for funding instructional innovation. The primary instructional improvement foci for the program - collaborative learning, global education and engaging large sections – have not changed.

UNC-Chapel Hill Contacts:

Todd Zakrajsek, Executive Director Center for Faculty Excellence 919 966-4838/toddzak@unc.edu

Bob Henshaw, ITS Liaison and Instructional Technology Consultant Center for Faculty Excellence 919 962-9969/bhenshaw@email.unc.edu

July 2011

2

Instructional Innovation Grants Program

Last year the CFE approved a second round of funded proposals for its instructional innovation grants program. To date, the program has attracted more than fifty individual proposals across a wide range of disciplines. Individual project awards are limited to no more than $10K. The proposal review committee established by the Center has tried to strike a balance between highly innovative projects that have the potential to inform future best practices, and high-impact projects that have the potential to inform instructional improvement at the program and institution level.

2010-2011 Project descriptions

Re-thinking Communities of Practice for Teacher Education

Sharon Palsha, Leigh Hall, and Katherine Ohle, School of Education

Instructors at the UNC School of Education (SOE) are using social networking sites to establish communities of practice for student teachers and to enhance knowledge creation and collaboration across the curriculum. A major component of this project is the partial replacement of traditional case studies with student-centered experiences. In addition to communicating with peers through blogs and other text-based media, student teachers are empowered to capture relevant instructional moments in their respective classrooms using low-cost video cameras. Those videos and photos are shared, critiqued and discussed among their program cohorts through social networking platforms.

Summary of findings

For this project, the use of social networking tools in two SOE programs was examined.

Palsha/Ohle: The first project included a 22-student cohort in the Child Development and Family Studies/Birth through Kindergarten Licensure Only program. Students used social networking service Ning to support each other during their student teaching assignments. Faculty member Sharon Palsha noticed that new teachers often feel isolated once they begin their teaching assignments as cohorts of student teachers are spread out over different school systems and schools. “I was the sole person benefitting from seeing student teachers and their collaborating teachers every week at different sites. When I saw something fabulous I would think, ‘I wish the other students could see this.’ Now they can,” says Dr. Palsha.

Dr. Palsha, and her assistant, Dr. Kathryn Ohle were looking for a social networking platform that was accessible to students and already familiar with popular sites such as Facebook. They have been pleased with Ning so far. In a survey completd at the end of the semester, nearly 75% of the students reported being comfortable posting comments, photos, or video in Ning. Figure 1 shows an example of a photo of a classroom activity posted to the Ning site by one of the student teachers.

July 2011

3

Figure 1

The use of photos and video has clearly provided students with insight to peer experiences that they would not have otherwise. The survey results in Table 1 suggest that while conversations with peers in class or on campus continue to be a primary means of communication about their collective experiences, the ability to observe peers in their classrooms would not be an option for many of the students without the online site.

Very

Frequently

Somewhat

Frequently

A few

times

Not at all

How frequently do you observe other students’ classrooms in a

face to face format?

11 5 42 42

How frequently do you observe other students’ classrooms

virtually (through Ning or other ways)?

0 58 42 0

How frequently do you discuss other students’ classroom

experiences in a face to face format?

63 32 5 0

How frequently do you discuss other students’ classroom

experiences virtually (through Ning or other ways)?

0 53 32 16

Table 1

In summarizing the benefits of the Ning site for students, Professor Palsha cited extra support from faculty members and other students, new opportunities to share lesson plans

July 2011

4

and other instructional resources, and a place for students teachers to share experiences and feelings.

“I'm definitely trying to read everyone's reflections as much as I can, because it really helps me feel better when I've had a challenging day-it makes me realize that I'm not going crazy or that I'm terrible at this, but just that some days are going to be more difficult than others for a variety of reasons! I also really like to see what everyone else is doing for activities, so I can get some ideas for my own room. I think it's great to be able to read each other's blogs, especially right now when we're all struggling with adjusting to the semester-it's certainly helping me in keeping my sanity!”

Student post

Before the students graduated, Palsha and Ohle asked them if they would be interested in continuing to use Ning to stay connected as they ventured out to begin their careers as educators. The majority of students, 16 out of 20, were very eager to stay connected and have continued to use the Ning site as new teachers. Additional study of post-graduate use and community is currently underway.

Hall: Professor Leigh Hall used Ning with her Master’s level courses (K-12 reading specialists). These courses were hybrid classes, meeting every other week face-to-face, with online experiences during alternate weeks. She felt that it was important to make the knowledge development in her courses public, allowing all members of the classes (and beyond) to bounce ideas back and forth with each other and share knowledge. Students posted a variety of materials including videos of their teaching, lesson plans, and websites links and annotations. Most contributed beyond the basic course requirements. She also invited guest bloggers from across the country to exchange ideas and engage in discussions with her students.

“Students had become dependent on me to provide them with the right answer or best way to do something,” says Hall. “Through social networking my students relied less on me and more on each other.” She says students have taken more responsibility for their own learning by suggesting readings, guest bloggers, and modifications to assignments that would make them more relevant to their goals.

During the Spring 2011 semester, students in a Revisting Literacy course that used Ning and students in a Content Area Literacy course that did not use Ning completed a survey at the end of the semester. By a significant margin, students in the Ning class reported that they were more likely to discuss ideas from readings or class with other students outside of class. Those students also were more likely to report working harder to meet course standards or expectations. A potential area for further study is the impact of the use of the social networking site on the overall sense of community among the students. Fourteen questions were drawn from a bank of questions developed to assess sense of community (Lichtenstein 2005) and were included on the student surveys for both courses. The sense of community index for the students in the class that used the Ning was much higher than that for the students in the

July 2011

5

control class, even though students in the control class met three times a week. However, because students in the Ning cohort had been together for a longer period of time, these findings will require additional data to be conclusive. Reference Lichtenstein, M. (2005). The importance of classroom learning environments in the assessment of learning community. Journal of College Student Development, 46(4), 341-356.

Introductory Biology for Students with Diverse Backgrounds

Kelly Hogan, Biology Department

The goal of this project was to accommodate diverse learning needs in a large introductory biology course. Professor Kelly Hogan’s Biology 101 section enrolls nearly 400 students. Historically, a disproportionate share of minority students has performed poorly in the course. Students whose parents did not attend college are also at risk.

The class meets three times per week in a large lecture hall with fixed stadium seating. For her spring 2011 course, Professor Hogan made a number of changes in an effort to improve learning opportunities for all students. An online homework system now provides students with immediate feedback on their understanding of key course concepts and allows Professor Hogan to track student progress throughout the semester. Students who perform poorly on the first exam are contacted by and asked to consult with Professor Hogan. Laptops and cell phones are used to bring interactive learning strategies generally reserved for small classrooms into a large lecture hall. Other interventions adopted for the course redesign include guided reading questions and several collaborative in-class activities.

Table 1 provides an overview of the differences between the redesigned course and a spring 2010 control section taught by Professor Hogan that made heavier use of traditional lecture-based methods.

BIOL 101 Course Format Comparison

Format Session/Year

BIOL 101 course components

Face-to-Face

Lecture

Mastering Biology

homework

Interactive class

activities

Guided reading

questions

Class response system

Early outreach to students scoring low on first exam

Traditional Spring 2010 3 No No No No No

Enhanced Spring 2011 3 Yes Yes Yes Yes Yes

Table 1

July 2011

6

Although most aspects of the redesigned course were in place during the Fall 2010 semester, our data comparison used Spring 2011 data as student demographics are more comparable across the two spring semesters. This approach also has the advantage of avoiding a comparison between first semester first-year students and second semester first-year students.

Professor Hogan

Summary of findings

The assessments used to compare student learning outcomes include three exams and a final exam. The questions compared are not identical, but use similar formats, are judged by the instructor to cover the same content, and have similar difficulty.

Comparisons of learning outcomes among the general population of students in the two sections did not produce any significant differences. However, the subgroup analysis demonstrated significant differences in learning outcomes for several student populations. The interventions were seen to disproportionately help certain minority and first generation college students.

The 4-exam average scores for African-American students in the Spring 2011 course (67.8%)

were significantly higher than the scores for African-American students in the Spring 2010

course (57.2%).

Latino students in the Spring 2011 course also averaged significantly higher scores (75.8%)

compared to those in the Spring 2010 course (65.2%).

July 2011

7

A significant difference was noted in 4-exam average scores between first-generation

students in the Spring 2011 course (70.9%) and those in the Spring 2010 course (64.81%).

No significant differences were noted for non first-generation students.

Demographic and attitudinal data were collected via student surveys administered at the end of each semester. Among the general population of students in the two sections, several differences were noted that suggest higher levels of engagement for students in the redesigned section.

Students in the redesigned section reported spending significantly more time on class work; 55% said they spent more than 4 hrs/week out of class compared to 31% in the traditional section.

Students reported reviewing lecture notes after each class at a much higher rate in the redesigned section; 52% reported that they often or always reviewed their lecture notes, compared to only 23% in the traditional section.

Significantly fewer students reported never contributing to class discussion; 6.5% in the redesigned section said never compared to 47% in the traditional section.

Student comments on Professor Hogan’s standard course evaluations also suggested higher levels of student engagement. Hogan is pleased with the results so far. “By capitalizing on the technology that now exists (online homework and classroom response systems) I can hold students accountable for material before and during class,” she said. “With students more prepared - and they really were - I was able to try more activities and give them more practice with higher level thinking.”

She found the impact of the redesign on at-risk students to be particularly encouraging. “Ultimately, I hope to make an impact broader than my class, helping to retain these students in science beyond their first introductory biology class,” said Hogan.

Due to the number of modifications made to the course between the Spring 2010 and Spring 2011 semesters, it is difficult to isolate the impact of specific variables and interventions. This is a likely one of several potential areas of further study for Professor Hogan.

Undergraduate Scholarly Publishing Program

Katie Rose Guest Pryal and Leslie Frost, Department of English and Comparative Literature

Building on UNC initiatives to promote undergraduate research, the PIs for this project developed a unique technology-mediated support system for helping undergraduate students publish their scholarly work. The program groups students together who are

July 2011

8

undertaking similar endeavors for their essays and uses an online system (Figure 2) to help students stay on track for weekly tasks. The primary goals of the program are 1) to help students produce a publishable paper and 2) to demonstrate to students the value of a peer-based collaborative publication process. The Undergraduate Scholarly Publishing Program (USPP) was piloted with a small group of undergraduate students this spring.

Figure 2

Summary of findings

Fourteen of the 26 students who participated in the pilot for the Undergraduate Scholarly Publishing Program (USPP) completed a post-pilot survey, and several more responded to a second call for comments. The survey results suggest that the program did help students progress through the manuscript publication process. Figure 3 provides an overview of student perspectives on the effectiveness of USPP support for the primary aspects of the scholarly publishing process.

July 2011

9

Figure 3

Several students commented positively on their overall experience:

“Before your class I had no idea an undergrad could publish, and I knew even less

about how to go about it. Your process led me through step by step until I

had all the components that undergrad journals are looking for and felt

that I had prepared something that was publishable.”

“Other than some logistical errors, which seem to be easily fixable, I think that the

program definitely accomplishes what it intends to accomplish. With such little

experience, it’s important for students to have an organizational tool, not only for the

independent research, but for collaborative efforts as well.”

The survey also highlighted the need for several modifications that will have to be made going forward. The application was developed using Plone. While it served most of the project needs adequately, the ability to embed files and frames emerged as an issue during

July 2011

10

the pilot. On the post-pilot survey, more than half of the students said the online components of the USPP were not easy to use.

One final issue for consideration is whether or not to build more accountability and deadlines into the system. Currently, it can be used as a graded course-based process or as a voluntary initiative that depends on the dynamic of the group to ensure that all members stay on task. Interestingly, students were evenly split when asked in a questions on the student survey whether or not more deadlines should be built into the program. Overall, Professors Pryal and Frost were pleased with the outcomes of the initial pilot. “We are breaking new ground, encouraging students to perform a task that has never been in the purview of undergraduates before,” said Dr. Pryal. “I think we’ve demonstrated the usefulness of the program as peer collaboration process as well as alerting students to the potential of publishing”, said Dr. Frost. They are looking at several issues in preparation for making the program available to a larger numbers of students. Additional information is available at the program website at: https://uspp.unc.edu/

Developing North-South Perspectives on Development and Conservation in the Galapagos Islands of Ecuador

Stephen J. Walsh, Department of Geography

In this project, students enrolled in a course at UNC-Chapel Hill that examines population-environment interactions in the Galapagos Islands of Ecuador collaborated with Ecuadorian students studying at the Universidad San Francisco de Quito (USFQ) on the Galapagos Islands. Together, they explored the tension between resource conservation and economic development on this fragile ecosystem. For a three-week period during the course, Galapagos-based students served informally as outreach agents for UNC students through their interviews with various island populations and stakeholders. Students used VoiceThread and other technologies to share and develop perspectives on relevant topics. Students also used online spatial modeling program (NetLogo) developed by the instructor to develop and test alternative impact scenarios in a World Heritage Site. Summary of findings

Eighty percent of the students enrolled in the UNC-Chapel Hill course completed a survey at the end of the semester on the usefulness of the interactions with USFQ students. Due to course scheduling conflicts, the window of opportunity for interaction between students at the two universities was limited to three weeks. The compressed timeframe made it a challenge to fully integrate the interactions in the course and to efficiently process, edit, and upload video-taped interviews with stakeholders into Voicethread. Still, 95% of the students said the Voicethread interactions were at least somewhat important to their

July 2011

11

understanding of course concepts, compared, for example, with only 83% who identified class discussion as an important component of the course. Clearly, students saw the potential in formalizing interactions with students on location at the Galapagos. When they were asked what activities would improve their understanding of local GI perspectives, 45% mentioned video interviews as effective. Use of the NetLogo spatial modeling software was not as well-received by students, and was rated least important of all the course components. The instructor for the course will be modifying several aspects of the technology-based components of the course when it is taught again this fall.

2011-2012 Project descriptions

Course Correspondents in Seville Helene De Fays, Abel Munoz-Hermoso, Bill Maisch Romance Languages

UNC students participating in the study abroad program in Seville will serve as Course Correspondents for three courses offered by the Department of Romance Languages. The instructors will develop activities for the study abroad participants that leverage opportunities unique to their international settings and that support the instructional goals of each Chapel Hill-based course. The activities will broaden the cultural understanding and appreciation of students abroad by encouraging them to explore local culture more deeply through their roles as the "eyes and ears on the ground" of students in Chapel Hill. Their interactions with Chapel Hill students will be enriched by videos, photos, and text that they create and share online in support of their assigned activities.

Game-based simulations in public health Jim Porto, Jay Levy, Jason Morningstar School of Public Health

This game-based simulation project will support the entire Health Policy and Management Executive Master's Program (EMP) that includes 12 courses over the two years of the MHA and MPH degree programs at the UNC School of Public Health. Students attend a two‐day workshop at the conclusion of their courses at the end of each term where they will compete in teams in an integrated "small‐ scale" social/area simulation on the concepts learned in the two courses taken that term. Two simulations will be constructed as part of this project. The data, framework, and tools developed for these two will be used to construct eight additional simulations as funding becomes available.

July 2011

12

Professional Interests Manager (PIM)

Preparing our faculty to take advantage of new teaching techniques and tools is a critical component of instructional enhancement efforts. Given the competing demands on the typical instructor’s time, most of them are not looking for more professional development opportunities; they’re looking for the right professional development opportunities. Unfortunately, most faculty development centers do not have a convenient way to push out targeted information to faculty members and graduate instructors based on their individual interests.

The Professional Interests Manager (PIM) is service created by the Center for Faculty Excellence to inform faculty members about professional development resources and opportunities. When faculty members and graduate students log into the PIM, they are asked to choose from a list of professional development topics in the areas of teaching, research, and leadership. Once enrolled, they receive email notifications from CFE staff about new journal articles, useful websites, event announcements, and other professional development resources. However they only receive resource updates related to topics that they have selected.

The goals of the service are to: 1. Improve the quality of professional development information sent out to UNC faculty

members by making it more personalized and more concise. 2. Engage instructors over an extended period of time. 3. Engage faculty members who do not generally participate in face-to-face CFE services

and events. 4. Improve CFE awareness about what issues are important to faculty members.

Pilot results – Faculty participants

A faculty pilot for the PIM was conducted over an eight-week period during the spring 2011 semester. Twenty-five faculty members participated. CFE staff sent out roughly 125 resources during the pilot. One hundred and eleven of the resources were associated with teaching and learning interests. At the end of the pilot, 17 of the 25 pilot participants completed a brief survey on their experiences.

In order to gage overall satisfaction with the system, participants were asked whether or not the CFE should move forward with a broader rollout of the PIM during the upcoming fall semester. Their unanimous endorsement of plans to proceed with a rollout of the system this fall (Figure 1) suggests that the system does have potential to be a useful tool for keeping faculty members informed.

July 2011

13

Figure 4 – Response to survey question on moving forward with PIM

Creating the technical backend for the system has been fairly straight forward. An internal team from Information Technology Services at UNC has led this aspect of the project. Decisions related to content and workflow have presented greater challenges.

For example, the success of the PIM is largely dependent on creating a classification scheme for professional interests that accurately reflects the interests of UNC’s faculty. Separate lists were developed for teaching and learning, research, and leadership. While the sample size for the pilot is not large enough to draw any definitive conclusions about the effectiveness of the lists, most survey respondents thought the lists struck an appropriate balance between being too comprehensive and not comprehensive enough (Figure 2). The lists for all three professional interest categories will be revisited before the fall pilot.

July 2011

14

Figure 2 – Response to survey question on comprehensiveness of interests lists

Pilot participants were also asked to provide suggestions for improving the classification system. When asked specifically what, if any, interests they thought were missing, only two new interests were suggested.

Equally critical to the success of the service is the perceived utility by faculty members of the resources about which they are being notified. CFE staff members make the decisions about what kind of information is potentially useful to UNC faculty members. Closely related are decisions about how much information to share. There is consensus among CFE staff members that the service will not be successful if faculty members begin to feel they are being inundated with information. On the question of how much information is too much, nearly nine in ten survey respondents said that the amount of information they received was about right. Perhaps more importantly, none reported feeling like they received too many resource updates.

The data concerning the usefulness of resources shared during the pilot are more difficult to interpret, but nonetheless encouraging. Pilot participants were asked to estimate the percentage of updates received that they were glad to learn about. Of the seventeen respondents, eleven entered numbers over 50%, averaging 75%. The other six entered numbers less than 50%. CFE staff members were able to follow up with four of those six respondents in order to get some clarification on their experiences. Two of the four had selected a larger number of professional interests than they would ordinarily, mostly out of

July 2011

15

curiosity. One who entered 0% did not receive any resource updates due to a technical error. The fourth participant contacted said that many of the resource updates he received were not what he expected. This suggests some disconnect between what the faculty member expected and how he interpreted the list of professional interests. It is certainly an issue that the CFE will want to track when the system is rolled out this fall.

Pilot Results – Staff participants

The success of the PIM is closely related to the decisions that individual CFE staff members make about what information they think faculty members will find useful and how much of it to share. Because many of these decisions are by necessity subjective ones, many staff members participated in the pilot with some level of uncertainty and, at times, frustration. At least ten percent of resources contributed by staff members were not ultimately approved to be shared with faculty members. While the results of the pilot provide some data on the type of resources that faculty members find most valuable (Figure 3), it is likely that effective guidelines for contributing content to the PIM will become apparent only through a continued process of trial and error.

Figure 3

July 2011

16

Communication among staff members during the content-vetting process is critical, and several modifications to the system have been proposed to facilitate communication.

Staff discussion surrounding the pilot results has also raised a number of questions about the sources that inform the information shared through the PIM. Are there core sources of information across the teaching and learning, research, and leadership domains that CFE staff should be monitoring? To what extent are CFE staff members expected to remain current in their respective areas of expertise, and how is currency defined? The answers to these questions, too, will likely be borne out over time.

Next steps

The pilot and subsequent analysis have resulted in a list of proposed modifications to the PIM system, as well as several policy recommendations and supporting resources to be considered this summer. Among the key tasks that need to be completed before the fall rollout:

Address bugs and priority modifications to PIM system Create a checklist for CFE staff to facilitate resource contributions Develop marketing plan for fall 2011

-