American Journal of Pharmaceutical Education-7 (1)

About AJPE

The American Journal of Pharmaceutical Education (AJPE) is the official publication of the

American Association of Colleges of Pharmacy (AACP). Its purpose is to document and advance

pharmaceutical education in the United States and Internationally. The Journal considers material

in all areas related to pharmaceutical education. Through open-access Internet publication the

Journal intends to take full advantage of the electronic medium; this includes the publication of

articles with multimedia features, encompassing 3D graphics, video, interactive figures and

databases, and sound. The Journal Editor is Joseph T. DiPiro, Professor and Executive Dean at

the South Carolina Colege of Pharmacy.

The electronic Journal will be issued quarterly. In addition, a year-end print volume will be made

available to AACP member institutions and to others upon request for a nominal fee. Access to

the electronic Journal will not be restricted by password.

The Journal accepts unsolicited manuscripts that have not been published and are not under

consideration for publication elsewhere. All manuscripts are subject to peer review and approval

by the Editor before publication. Authors must prepare manuscripts to conform to the Journal

style. Authors are not assessed page charges for publication. All reports and papers presented at

the Annual Meeting of the AACP, as well as manuscripts accepted for publication, become the

property of AACP. All requests for reassignment of the above rights should be directed to the

Editor.

Referees are assigned by the Editor with the advice of the Editorial Board as needed.

Manuscripts are processed on line by the Editor, Editorial Assistant, and the American

Association of Pharmaceutical Scientists using a computer-based tracking system that relies on

keyword selection and automatic time lines for review. The electronic submission and review

process is intended to expeditiously complete the publication process using electronic

communication between the editorial office, editors, referees, and authors. Once an article is

accepted, a final version is sent to the author for approval by email. The article is published on

the AJPE web site shortly thereafter. All expressions of opinion and statements of supposed fact

appearing in the Journal are not to be

regarded as necessarily expressing the policy or views of the Editor or of AACP.

Articles appearing in the Journal are indexed in: PubMed/Medline; Current Contents, Education;

Current Contents, Life Sciences; International Pharmaceutical Abstracts; Current Index to

Journals in Education; and by PROQuest and EBSCO.

Volume 78, Issue 7, 2014

ISSN: 0002-9459

Marvin M. Malone, University of the Pacific 1975-1979

George H. Cocolas, University of North Carolina 1980-2002

Editorial Office

The Journal office is located on the University of South Carolina campus in Columbia. Address all

communications concerning manuscripts to the Editorial office. Mailing address and contact

information is as follows:

Address: American Journal of Pharmaceutical Education

715 Sumter St.

Columbia, SC 29208

Editorial Office: 803-777-3096

Fax: 803-777-3097

Business Office

The business office is located at the AACP office. Mailing address and contact information is as

follows:

Address: 727 King Street, Alexandria, VA 22314

Phone: 703-739-2330

Fax: 703-836-8982

Copyright © 2011, American Association of Colleges of Pharmacy, 1426 Prince Street,

Alexandria, VA 22314. All rights reserved.

The Journal has been published continuously since 1937. Past Editors of the Journal were:

Rufus A. Lyman, University of Nebraska 1937-1955

Melvin R. Gibson, Washington State University 1956-1960

C. Boyd Granberg, Drake University 1961-1974

Editorial Board-AJPE

Joseph T. DiPiro, PharmD Editor, South Carolina College of Pharmacy Gayle A. Brazeau, PhD Associate Editor, University of New England Jack E. Fincham, PhD Associate Editor, University of Missouri - Kansas City Claire Anderson, PhD, BPharm International Associate Editor, University of Nottingham, United Kingdom

Naser Alsharif, PhD

Creighton University

Zubin Austin, PhD

University of Toronto

Alicia Bouldin, PhD

The University of Mississippi

Eric Boyce, PharmD

University of the Pacific

Susan Burton

South Africa

Shelley Chambers, PhD

Washington State University

Sudip K. Das, PhD

Butler University

Patrick J. Davis, PhD

University of Texas-Austin

Gary E. DeLander, PhD

Oregon State University

JoLaine R. Draugalis, PhD

University of Oklahoma

Catherine A. Elstad, PhD

Washington State University

Stuart Haines, PharmD

University of Maryland

Deborah Harper-Brown, PharmD

Chicago State University

Mohamed Azmi Ahmad Hassali

University Sains Malaysia

Monica Holiday-Goodman, PhD

University of Toledo

Peter Hurd, PhD

St. Louis College of Pharmacy

Harold Kirschenbaum, PharmD

Long Island University

Jennifer Marriott

Monash University

Susan Meyer, PhD

University of Pittsburgh

Adam Persky, PhD

University of North Carolina

Anna Ratka, PharmD, PhD

Texas A&M University

Frank Romanelli, PharmD, MPH

University of Kentucky

Amy Seybert, PharmD

University of Pittsburgh

Brent Fox, PhD

Auburn University

Cecilia Plaza, PharmD, PhD

AACP Liaison

Contents

VIEWPOINTS

1. Turning the World of Pharmacy Education into a Global Community 1-2

Through SharingClaire AndersonClaire Anderson, Jennifer L. MarriottJennifer L. Marriott, Joana CarrasqueiraJoana Carrasqueira, Tina P. BrockTina P. Brock,

Timothy RennieTimothy Rennie, Andreia F. BrunoAndreia F. Bruno, Ian BatesIan Bates

SPECIAL ARTICLES

2. Complex Issues Affecting Student Pharmacist Debt 3-6

Jeff CainJeff Cain, Tom CampbellTom Campbell, Heather Brennan CongdonHeather Brennan

Congdon, Kim HancockKim Hancock, Megan KaunMegan Kaun, Paul R. LockmanPaul R. Lockman, R. Lee EvansR. Lee Evans

RESEARCH

3. A Three-Year Study of the Impact of Instructor Attitude, Enthusiasm, and 7-19

Teaching Style on Student Learning in a Medicinal Chemistry CourseNaser Z. AlsharifNaser Z. Alsharif, Yongyue QiYongyue Qi

4. Identifying Achievement Goals and Their Relationship to Academic 20-27 Achievement in Undergraduate Pharmacy StudentsSaleh AlrakafSaleh Alrakaf, Erica SainsburyErica Sainsbury, Grenville RoseGrenville Rose, Lorraine SmithLorraine Smith

5. Self-Efficacy and Self-Esteem in Third-Year Pharmacy Students 28-32Mark L. YorraMark L. Yorra

6. Investigating the Relationship Between Pharmacy Students’ Achievement 33-38Goal Orientations and Preferred Teacher QualitiesSaleh AlrakafSaleh Alrakaf, Erica SainsburyErica Sainsbury, Grenville RoseGrenville Rose, Lorraine SmithLorraine Smith

Volume 78, Issue 7, 2014

ISSN: 0002-9459

INSTRUCTIONAL DESIGN AND ASSESSMENT

7. Long-term Effectiveness of Online Anaphylaxis Education for Pharmacists 39-49

Sandra M SalterSandra M Salter, Sandra ValeSandra Vale, Frank M SanfilippoFrank M Sanfilippo,

Richard LohRichard Loh, Rhonda M CliffordRhonda M Clifford

8. A Physical Assessment Skills Module on Vital Signs 50-56Christine LeongChristine Leong, Christopher LouizosChristopher Louizos, Grace FrankelGrace

Frankel, Sheila NgSheila Ng, Harris IacovidesHarris Iacovides, Jamie FalkJamie Falk, Drena DunfordDrena Dunford, Kelly BrinkKelly Brink, Nancy KleimanNancy Kleiman, Christine

DavisChristine Davis, Robert RenaudRobert Renaud

9. Student Self-Screening for Methicillin-Resistant Staphylococcus Aureus 57-63 (MRSA) Nasal Colonization in Hand Hygiene EducationTia LumTia Lum, Kristin PicardoKristin Picardo, Theresa WestbayTheresa Westbay, Amber BarnelloAmber Barnello, Lynn FineLynn Fine, Jill LavigneJill Lavigne

10. Using Curriculum Mapping to Engage Faculty Members in the Analysis 64-72of a Pharmacy Program

Sheryl ZelenitskySheryl Zelenitsky, Lavern VercaigneLavern Vercaigne, Neal M. DaviesNeal M.

Davies, Christine DavisChristine Davis, Robert RenaudRobert Renaud, Cheryl KristjansonCheryl

Kristjanson

11. Improved Knowledge Retention Among Clinical Pharmacy Students 73-82 Using an Anthropology Classroom Assessment TechniqueHeather P. WhitleyHeather P. Whitley, Jason M. PartonJason M. Parton

12. An Active Learning Complementary and Alternative Medicine Session 83-86

in a Self-Care Therapeutics ClassMelissa J. MattisonMelissa J. Mattison, Eric C. NemecEric C. Nemec

13. A Multiyear Analysis of Team-Based Learning in a Pharmacotherapeutics Course 87-95June Felice JohnsonJune Felice Johnson, Edward BellEdward Bell, Michelle BottenbergMichelle Bottenberg, Darla EastmanDarla Eastman,

Sarah GradySarah Grady, Carrie KoenigsfeldCarrie Koenigsfeld, Erik MakiErik Maki, Kristin MeyerKristin Meyer, Chuck PhillipsChuck Phillips, Lori SchirmerLori Schirmer

VIEWPOINTS

Turning the World of Pharmacy Education into a Global CommunityThrough Sharing

Claire Anderson, PhD, BPharm,ab Jennifer L. Marriott, PhD, BPharm,b Joana Carrasqueira, MPharm,b

Tina P. Brock, EdD, MS,b Timothy Rennie, PhD, MPharm,b Andreia F. Bruno, PhD, MPharm,b Ian Bates,PhD, BPharmb

a Associate Editor, American Journal of Pharmaceutical Educationb University of Nottingham, Nottingham, United Kingdom

Barriers to delivering quality pharmacy education indeveloping countries have been identified as limited in-frastructure, access to teaching resources, academic staffdevelopment, and research capacity.1 In 2010, the Inter-national Pharmaceutical Federation (FIP) and the UnitedNations Educational, Scientific and Cultural Organiza-tion (UNESCO) signed an agreement to develop a FIPUNESCO-UNITWIN Global Pharmacy Education De-velopment (GPhED) program. The GPhED networkstrives to raise the standard of education globally and in-corporate best practices through shared resources, exper-tise, and experience. UNESCO and FIP consider one ofthe essential factors favoring development in the field ofprofessional competence the exchange of experience andknowledge between universities and other learning insti-tutions. Additionally, collaborative practice by universityteachers, researchers, and administrators from differentregions across the world could benefit the entire profes-sional community.

The aims of the FIP UNESCO-UNITWIN programare global higher education and its development, witha special interest in African nations and low-income coun-tries, empowerment ofwomen as pharmaceutical scientistsand academics, sustainable healthworkforce development,academic capacity building, quality assurance, and accred-itation standards, which are all compatible with the over-arching aims of UNESCO.2,3 Objectives of the programinclude promotion of an integrated system of research,training, information, and documentation in the field ofpharmacy education through “Centres of Excellence,”which will facilitate intra-regional cooperation.

Sharing in all areas is the key and to this end theGPhED program is currently developing three areas ofactivity that focus on sharing of teaching and learning

resources, pharmaceutical education expertise and expe-rience, and research capacity.

Resource sharing within the program is being facili-tated through SABER (Sharing and Building EducationalResources), an online platform (www.saber.monash.edu)fromwhich resources can be downloaded, used, and sharedinternationally between academic institutions to improveteaching and learning. Enhanced pharmacy education cre-ates better pharmacists, but creating high-quality studentlearning resources takes time and money. Sharing educa-tional resources makes sense so students can benefit andlearn more, especially in a global profession such as phar-macy where teaching needs are often similar. Sharing andcollaborating creates and strengthens partnerships basedon goodwill and a shared commitment to educational ex-cellence. The SABER platform is a place to share, dis-cover, acquire, and re-purpose resources for pharmacyeducation. It also encourages the collaborative creationof new content. SABER is available to educators in phar-macy schools around the world, hosting quality-assurededucational resources and providing a trustworthy sourceof relevant and current material. Being multi-tiered, itallows various access levels for a broad range of users.

SABER’s intuitive interface allows users to quicklyfind existing assets, and up-to-date social-networkingtools let them build peer networks to adapt resourcesand create new material. The site is administered byMonash University on behalf of the GPHeED networkand is accessible to all for free. Under a Creative Com-mons license, SABER enables academics anywhere inthe world to share something as simple as an image or aPowerPoint presentation, to large purpose-designed teach-ing programs in areas relating to pharmacy and phar-maceutical science. The success of SABER depends onacademics contributing resources they have developedto share via the database.

Another important component in the GPhED pro-gram is the formation of the African Centre of Excellencein Pharmacy Education. Ideally, the Centre will facilitate

Corresponding Author: Claire Anderson, PhD, BPharm,University of Nottingham, University Park, NottinghamNG7 2RD, United Kingdom. E-mail: [email protected]

American Journal of Pharmaceutical Education 2014; 78 (7) Article 1.

1

the sharing of ideas, skills, resources, and good practice,including staff exchange for skills and capacity building.The Centre will provide a forum for discussion and debateon trends and developments in pharmacy education, fa-cilitated by network partners, including NGOs and pro-fessional agencies, and coordinated by the foundingpartners. Africa was chosen to establish the first FIPUNESCO-UNITWIN Centre of Excellence because theregion is in great need of a pharmacy workforce and lackseducational resources for universities.4

A recentmeeting of the 5 founding partner countries –Ghana, Namibia, Nigeria, Uganda and Zambia – was heldin Lusaka, Zambia to determine the future activities of theCentre. Five domains for the Centre of Excellence weredetermined (communication, capability, quality, innova-tion, and clinical). Each founding partner agreed to takethe lead in one of these domains and coordinate projects oractivities within them. The founding partners developedand agreed to a communications strategy and will con-tribute to advocacy, local network building, and commu-nications to ensure the wider success of the Centre ofExcellence. Proposed projects of the Centre include a sur-vey of African colleges and schools of pharmacy to es-tablish a database of academic capacity and expertise tofacilitate intra-regional sharing of expertise through a vis-iting academic program. Another project is the develop-ment of a “Lab-box” of basic laboratory equipment toimprove students’ ability to undertake laboratory experi-ments to support their learning of basic science concepts.The Centre of Excellence will invite additional countriesto become part of the centre to expand its activities andprovide a broader base of communication and support inthe region. Centre of Excellence activities will also seekto promote gender equality and empowerment for women

academics and scientists in collaborative research andpolicy development.

The GPhED hopes to foster international collabora-tive research by having prospective research studentscomplete a higher degree in their home university inAfrica under the guidance of a local supervisor and anexperienced international researcher with expertise inthe chosen research area. The hope is that the arrangementof shared supervision will enable collaboration in policysynthesis and strategic development, with a focus on in-ternational development issues and sustainable practi-tioner development policies. Public health or servicedelivery projects are favored by the African universitiesdue to the limited laboratory facilities available, but suchprojects will contribute relevant health data in the countryand increase the capacity for improved health programsand delivery of pharmacy services.

The FIPUNESCO-UNITWINGlobal Pharmacy Ed-ucation Development network is in the nascent stages ofdevelopment, but through the goodwill, enthusiasm, andsupport of academics across the globe, sharing educa-tional resources, expertise, and experience through suchnetworks will facilitate the provision of quality pharmacyeducation in developing countries.

REFERENCES1. FIPEd Global Education Report 2013 http://www.fip.org/educationreports. Accessed June 30, 2014.2. UNESCO 2008-2009 Biennium Priorities (34 C/5): http://www.unesco.org/science/doc/SC_34_C5.pdf. Accessed June 30, 2014.3. UNESCO 2008-2013 Mid-term Strategy (34 C/4): http://unesdoc.unesco.org/images/0014/001499/149999e.pdf. Accessed June 30, 2014.4. FIP Global Pharmacy Workforce Report https://www.fip.org/static/fipeducation/2012/FIP-Workforce-Report-2012/?page5hr2012. Accessed June 30, 2014.

2

American Journal of Pharmaceutical Education 2014; 78 (7) Article 1.

SPECIAL ARTICLES

Complex Issues Affecting Student Pharmacist Debt

Jeff Cain, EdD, MS,a Tom Campbell, PharmD,b Heather Brennan Congdon, PharmD,c

Kim Hancock, PhD,d Megan Kaun, PharmD,e Paul R. Lockman, PhD,f and R. Lee Evans, PharmDg

a College of Pharmacy, University of Kentucky, Lexington, Kentuckyb College of Pharmacy, Lipscomb University, Nashville, Tennesseec School of Pharmacy, University of Maryland, Baltimore, Marylandd College of Pharmacy, Ferris State University, Big Rapids, Michigane College of Pharmacy and Pharmaceutical Sciences, The University of Toledo, Toledo, Ohiof School of Pharmacy, West Virginia University, Morgantown, West Virginiag Harrison School of Pharmacy, Auburn University, Auburn, Alabama

Submitted November 25, 2013; accepted January 26, 2014; published September 15, 2014.

It is time for colleges and schools of pharmacy to examine and confront the rising costs of pharmacyeducation and the increasing student loan debt borne by graduates. These phenomena likely result froma variety of complex factors. The academy should begin addressing these issues before pharmacy educationbecomes cost-prohibitive for future generations. This paper discusses some of the more salient drivers of costand student debt load and offers suggestions that may help alleviate some of the financial pressures.

Keywords: student loan, debt, tuition, higher education

INTRODUCTIONThe rising cost of higher education in the United

States is a disturbing trend. Pharmacy school tuition andresulting student indebtedness have risen significantlyover the last decade.1 Initial assumptions typically sug-gest that the responsibility for increased student loan debtbelongs to students. However, we assert that rising tuitionand student debt is a multifaceted, complex issue that hasorigins within the academy, the accreditation process,federal and state governments, universities, and finallystudent and faculty culture. Reducing or even maintainingcurrent cost levels for pharmacy education and concomi-tant student debt will not be straightforward, easy, orwithout controversy, but it is imperative that the academyconfront the issues before they worsen. In this paper, wediscuss some of the more notable influences on cost andstudent debt load and suggest potential actions that mayallay the financial burdens.

REDUCED STATE SUPPORT OF HIGHEREDUCATION

Faced with financial pressures from the recent eco-nomic recession, state legislatures have significantly reduced

higher education funding over the last several years. At first,most public institutions absorbed state funding reductionsthrough spending cuts and efficiency measures. However,after those means were exhausted, the only major recourseto fund educational activity was through tuition increases,effectively shifting more of the costs to students.2 In 2008,31.6% of revenue used to cover public higher educationoperating expenses came from tuition, compared to 42.4%in 2012.3After inflation adjustments, annual tuition at 4-yearpublic colleges has increased by $1,850, or 27%, since the2007-08 school year.2 Compounded over several semesters,the increased tuition adds substantially to the total cost ofhigher education for an individual and is a primary factor forincreased student debt.4

GOVERNMENT ISSUESGovernment regulation of loan eligibility, loan

amounts, interest rates, repayment structures, loan defer-ment eligibility, and government loan forgiveness pro-grams all impact the total cost of student loan debt.Student loan interest rates arguably have the single larg-est impact on student loan debt. On July 1, 2013, interestrates on certain types of loans doubled because Congressfailed to prevent an expiration of subsidies.5 The federalgovernment, as opposed to the borrower, pays interestaccrued on subsidized loans during periods of eligibledeferment. This change to subsidized loans had thepotential to nearly double the total payment amount of

Corresponding Author: Jeff Cain, EdD, MS, Department ofPharmacy Practice & Science, University of KentuckyCollege of Pharmacy, 789 South Limestone, Lexington, KY40536-0596. Tel: 859-257-4429. Fax: 859-257-7297.E-mail: [email protected]

3

American Journal of Pharmaceutical Education 2014;78(7) Article 2.

a loan. However, in August 2013, Congress passed a bi-partisan deal to lower interest rates and tie them to mar-ket rates. Although this legislation has led to immediaterelief for many borrowers, it will not protect future bor-rowers in a stronger economy when interest rates mayrise to as much as 9.5% for graduate and professionalstudents.6

Another factor affecting pharmacy graduates is recentchange to federal regulations that no longer require lendersto place student loans into forbearance for a pharmacy res-idency or fellowship. The new regulation states that onlymedical and dental residencies qualify for mandatory loanforbearance.7 While this has a smaller effect on studentdebt, it could cause some pharmacy students to opt out ofpostgraduate training because of inability to afford or un-willingness to make loan payments.

STUDENT PERSONAL FINANCEAWARENESS AND KNOWLEDGE

Students and their families must accept personalresponsibility for aspects of their financial future. Whilefactors pertaining to tuition, salaries, and job prospectsare out of their control, students do make choices thataffect their education-related debt. Some students en-gage in lifestyles that significantly exceed their incomewhile in college, purchasing, for example, automobiles,clothing, and electronic devices, and even taking elabo-rate vacations.8As a result, student debt extends beyondschool-related financial aid to include additional creditcard debt. In 2009, a study by Sallie Mae found that 84%of undergraduates have at least one credit card and 50%of undergrads have 4 or more credit cards and use thesebecause they have insufficient savings or financial aid tocover all of their expenditures. Furthermore, 68% ofcollege students have charged items to their credit cardsknowing they did not have sufficient funds to pay the bill.The long-term effects of financing these discretionarypurchases can result in student loan and credit card debtconsiderably higher than what is necessary. Most stu-dents are aware that this is a problem, with 84% of un-dergraduate students stating they want more educationon financial management topics.9

INCREASED PERSONNEL FORADMINISTRATIVE AND TECHNICALTASKS

Higher education has traditionally been a labor-intensive profession, primarily because of the special-ized set of faculty skills and expertise. In recent years,however, the bulk of labor costs have begun to shift awayfrom faculty toward managerial, technical, and supportstaff.10 In addition to other types of support staff and in

part because of the extensive and often redundant gov-ernmental, funding agency, and accreditation require-ments, institutions and schools now employ largenumbers of non–instructional staff members to developand/or maintain technical systems and to capture, track,record, analyze, and report data (financial, assessment,accreditation, etc.) for accountability purposes. Theseincreases in staff members have been disproportionateto increases in student enrollment.10 In 2007, afteradjusting for increased student enrollment, it took13.1% more employees than it did in 1993 to educatethe same number of students.11 While accountability isa necessary and desirable aspect of public institutions,the financial effects of maintaining it are potentiallyborne, at least partially, by students through rising tu-ition and fees.

CURRICULAR ISSUESThe time required to earn a degree and the student’s/

potential applicant’s ability to earn money throughout aneducational career are often important factors for selec-tion of a major. Faculty members set the standards ofwhat, how much, and how in depth the educational pro-cess will be throughout the degree program. From a stu-dent’s perspective on financing an education, theeducational and experiential schedules of pharmacyschool may limit their ability to earn income while inschool. Furthermore, some curricula seem to focus toomuch on delivering an increasing amount of content in-stead of focusing on more selective content paired withbetter instructional strategies. One of the more difficultquestions that pharmacy educators need to ask themselvesis: “Can we design effective curricula that is deliveredmore quickly and/or more efficiently (without sacrificingquality), saving students either money or time?” The highereducation landscape is shifting, with more attention towidespread online delivery12 and reconsideration of thecredit hour as a metric for student abilities.13 Calls arebeing made to reduce the cost of medical education14 andpharmacy school administrators should be doing the samefor pharmacy education.

Preprofessional requirements have also grown by525% from 2006 to 2011 in the number of pharmacy pro-grams requiring 3 years of prerequisite courses or a bache-lor’s degree, versus 2 years of courses for admittance.15

There is currently no consensus regarding the ideal lengthor content for prepharmacy curricula.16 While theoreticallythe more undergraduate work that entering students havecompleted, the more mature and better equipped they willbe for pharmacy school, this potentially comes at the cost ofanother year of educational expenses and the effective lossof a year’s salary.

4


ARMS RACE FOR FACILITIES ANDRESOURCES

Recent significant expansion of pharmacy educationhas led to competitive student recruitment based moreand more on reputation. Factors that enhance reputation(buildings, technology, student amenities) are all costly,thereby incentivizing an academic “arms race” to see whocan spend the most money.17 Colleges and schools of phar-macy are not immune and are continually seeking everypossible edge in attracting the highest quality students. ANational Bureau of Economic Research working paperreported that college students place a high value on con-sumption amenities such as student activities, dormitories,and sports.18 The recent addition and enhancement ofstudent services to include social, emotional, and careercounseling has also accounted for spending growth.19,20

Institutional marketing and recruitment strategies nowhighlight key amenities such as technology, cutting edgepedagogical approaches, and state-of-the-art facilities inorder to attract students to their programs.21 However, evenif these amenities are deemed necessary, the price tag as-sociated with new infrastructures, technology implementa-tion, and student services is substantial, driving the cost ofan education even higher with little to no evidence thatlearning is improved. Moreover, higher education mightbenefit if every institution ceased the expansion of facilities,athletics, and student amenities. Unfortunately, marketforces perpetuate continued escalation, and no institutioncan unilaterally withdraw from this arms race without put-ting itself at risk of falling behind.22

RECOMMENDATIONSA Center for College Affordability policy paper has

been released that offers systemic methods universities canuse to make education more affordable (such as reformingfinancial aid, digitizing academic libraries, and streamlin-ing redundant programs) and reverse the trend of risingcosts.17 We offer some additional recommendations forthe academy to consider.

While public funding of higher education will notlikely revert completely to previous levels, the academymust continue to lobby for federal and state support. With-out pressure, state and federal legislatures may reduce fi-nancial allocations even further. Additionally, lobbyingefforts should be directed toward securing adequate finan-cial aid funding for students, particularly grants and lowinterest loans.

As mentioned previously, many students todaystruggle with increased loan balances and significantdebt related to unwise personal financial management.Many colleges and schools of pharmacy offer personal fi-nance courses23,24 or provide brief financial management

principles to students prior to graduation, but some studentsmay already be deep in debt before those lessons are taught.We propose 5 recommendations that the academy couldemploy to minimize this problem.

(1) Integrate required financial management course-work into the curriculum. This coursework needsto be employed early (preferably within the firstyear of the doctor of pharmacy program) so thatthe principles can be employed throughout thestudent’s educational career. Curriculum should fo-cus on designing and living within a budget, avoid-ing unnecessary debt and overuse of credit cards,managing financial aid in a responsible manner, andobtaining financial advice for postgraduate loan re-payment, insurance coverage, and timing of largepurchases.

(2) Designate an appropriate faculty/staff member orexternal expert to provide ongoing financial coun-seling to students throughout their educational ca-reer. Access to financial aid offices varies amongcollege campuses, and even in the best of scenarios,those services may lack the scope of counseling thatstudents need related to financial management. Adesignated counselor who can build a level of trustwith students might encourage a more open line ofcommunication with and a more receptive responsefrom students.

(3) Inform all students of the Federal Student LoanForgiveness Program, which forgives student debtafter a period of service. To qualify for the pro-gram, students must work for a public institutionfor 10 years (nonconsecutive) and make 10 years’worth of qualifying payments. All remaining stu-dent debt will be forgiven after the 10 years.25

Pharmacists who take advantage of this programcan potentially reduce their student debt by tens ofthousands of dollars.

(4) Encourage national pharmacy organizations toprovide financial management programming atassociation meetings that student pharmacists at-tend. Programming should include student-to-student teaching and best practices (ie, roundtablediscussions), allowing students to share what hasworked as well as mistakes to avoid.

(5) Encourage administrators at colleges and schoolsof pharmacy to assess their institution to ensurethat students are not overburdened by the cost oftheir education. This might necessitate scrutiny oftuition models and internal operations to determineif students are paying only their fair share. At theuniversity level, use of technology should be ex-plored to reduce staff numbers and time necessary

5


to comply with accreditation and other account-ability demands. Extracurricular amenities shouldbe closely scrutinized and avoided if they result inadditional costs to students without providingadded value to their professional education. Addi-tionally, the curriculum (including prepharmacy)should be analyzed to determine if students areloaded with coursework that adds little value totheir overall degree and their ability to practicepharmacy. This type of scrutiny may be a painfulprocess for faculty members, but it could result ina much more streamlined and effective curriculum.While fraught with numerous issues, there are alsointriguing possibilities for collaboration within theacademy with regard to sharing educational con-tent online. We encourage the American Associa-tion of Colleges of Pharmacy to explore potentialcontent sharing mechanisms that create pedagogi-cal efficiencies. Pharmacy educators have the re-sponsibility as faculty members of not acting out ofself-interest, but in providing the best and mosteconomical education to students.

REFERENCES1. Cain J, Campbell T, Congdon HB, et al. Pharmacy student debtand return on investment of a pharmacy education: issues for theacademy. Am J Pharm Educ. 2014;78(1):Article 5.2. Oliff P, Palacios V, Johnson I, Leachman M. Recent deep statehigher education cuts may harm students and the economy for yearsto come. http://www.cbpp.org/cms/?fa5view&id53927. AccessedSeptember 9, 2013.3. State Higher Education Executive Officers. State HigherEducation Finance: FY 2012. http://www.sheeo.org/sites/default/files/publications/SHEF%20FY%2012-20130322rev.pdf. AccessedSeptember 10, 2013.4. Macy A, Terry N. The determinants of student college debt.Southwest Econ Rev. 2007;34(1):15-25.5. Nawaguna E. Congress finally votes to cut student loan interestrates. Reuters. July 31, 2013http://www.reuters.com/article/2013/07/31/us-usa-studentloans-rates-idUSBRE96U1G220130731. AccessedNovember 1, 2013.6. Johnson J. Obama signs student loan interest rate legislation intolaw. The Washington Post. August 9, 2013. http://articles.washingtonpost.com/2013-08-09/politics/41223761_1_interest-rates-plus-loans-graduate-students. Accessed November 1, 2013.7. Federal Student Aid, The US Department of Education. Federalstudent aid: deferment and forbearance.http://studentaid.ed.gov/repay-loans/deferment-forbearance. Accessed May 5, 2013.8. Roberts JA, Jones ELI. Money attitudes, credit card use, andcompulsive buying among American college students. J ConsumerAff. 2001;35(2):213-240.9. Sallie Mae. How undergraduate students use credit cards: SallieMae’s national study of usage rates and trends. http://static.mgnetwork.com/rtd/pdfs/20090830_iris.pdf. Accessed Oct 4, 2013.10. Bennett DL. Trends in the higher education labor force:identifying changes in worker composition and productivity. Center

for College Affordability and Productivity. http://www.

centerforcollegeaffordability.org/uploads/Trends_LaborForce.pdf.

Accessed November 4, 2013.11. Greene JP, Kisida B, Mills J. Administrative bloat at American

universities: the real reason for high hosts in higher education.

Goldwater Institute Policy Report No. 239. August 17, 2010. http://

goldwaterinstitute.org/sites/default/files/Administrative%20Bloat.

pdf. Accessed September 3, 2013.12. Pirani J. A compendium of MOOC perspectives, research, and

resources. Educause Rev. http://www.educause.edu/ero/article/

compendium-mooc-perspectives-research-and-resources. Published

November 4, 2013. Accessed November 6, 2013.13. Carnegie Foundation for the Advancement of Teaching. Carnegie

Foundation for the Advancement of Teaching receives funding to

rethink the Carnegie Unit. http://www.carnegiefoundation.org/

newsroom/press-releases/carnegie-foundation-receives-funding-

rethink-the-carnegie-unit. Decmeber 2012. Accessed Nov 4, 2013.14. Asch DA, Nicholson S, Vujici M. Are we in a medical education

bubble market? N Engl J Med. 2013;369:1973-1975. http://www.

nejm.org/doi/pdf/10.1056/NEJMp1310778. Accessed November 18,

2013.15. Gleason B, Siracuse M, Moniri N, Birnie C, Okamoto C, Crouch

M. Evolution of preprofessional pharmacy curricula. Am J Pharm

Educ. 2013;77(5):Article 95.16. Broedel-Zaugg K, Buring SM, Shanker N, et al. Academic

pharmacy administrators’ perceptions of core requirements for entry

into professional pharmacy programs. Am J Pharm Educ. 2008;72(3):

Article 52.17. Vedder R, Gillen A, Bennett D, et al. 25 ways to reduce the cost

of college. Center for College Affordability and Productivity; http://

www.centerforcollegeaffordability.org/uploads/25Ways_to_

Reduce_the_Cost_of_College.pdf. Accessed November 18, 2013.18. Jacob B, McCall B, Stange KM. College as country club: do

colleges cater to students’ preferences for consumption. The National

Bureau of Economic Research. http://www.nber.org/papers/w18745.

pdf. Accessed November 5, 2013.19. Jaschik S. The spending side of the equation. Inside Higher Educ.

http://www.insidehighered.com/news/2008/05/01/spending.

Accessed November 4, 2013.20. Davis Educational Foundation. An inquiry into the rising cost of

higher education: Summary of responses from seventy college and

university presidents. http://www.davisfoundations.org/site/

documents/AnInquiryintotheRisingCostofHigherEducation_003.pdf.

Accessed November 12, 2013.21. National Public Radio. What’s driving college costs higher?

http://www.npr.org/2012/06/26/155766786/whats-driving-college-

costs-higher. Accessed November 4, 2013.22. Winston G. The positional arms race in higher education.

Educause. http://net.educause.edu/ir/library/pdf/ffp0103s.pdf.

Accessed November 5, 2013.23. Fox J, Bartholomae S, Trombitas K. Evaluating financial

education programs. In: Durband DB, Britt SL, eds. Student

Financial Literacy. Springer, US; 2012:141-166.24. Chui MA. An elective course in personal finance for health care

professionals. Am J Pharm Educ. 2009;73(1):Article 6.25. Federal Student Aid, The US Department of Education. Federal

student aid: public service loan forgiveness. http://studentaid.ed.gov/

repay-loans/forgiveness-cancellation/charts/public-service. Accessed

May 5, 2013.

6


RESEARCH

A Three-Year Study of the Impact of Instructor Attitude,Enthusiasm, and Teaching Style on Student Learning in aMedicinal Chemistry Course

Naser Z. Alsharif, PharmD, PhD, and Yongyue Qi, MS

School of Pharmacy and Health Professions, Creighton University Medical Center, Omaha, Nebraska

Submitted August 9, 2013; accepted January 20, 2014; published September 15, 2014.

Objective. To determine the effect of instructor attitude, enthusiasm, and teaching style on learning fordistance and campus pharmacy students.Methods.Over a 3-year period, distance and campus students enrolled in the spring semester of amedicinalchemistry course were asked to complete a survey instrument with questions related to instructor attitude,enthusiasm, and teaching style, as well as items to measure student intrinsic motivation and vitality.Results. More positive responses were observed among distance students and older students. Gender didnot impact student perspectives on 25 of the 26 survey questions. Student-related items were significantlycorrelated with instructor-related items. Also, student-related items and second-year cumulative grade pointaverage were predictive of students’ final course grades. Instructor enthusiasm demonstrated the highestcorrelation with student intrinsic motivation and vitality.Conclusion. While this study addresses the importance of content mastery and instructional meth-odologies, it focuses on issues related to instructor attitude, instructor enthusiasm, and teaching style,which all play a critical role in the learning process. Thus, instructors have a responsibility toevaluate, reevaluate, and analyze the above factors to address any related issues that impact thelearning process, including their influence on professional students’ intrinsic motivation and vitality,and ability to meet educational outcomes.

Keywords: instructor enthusiasm, science courses, medicinal chemistry, student learning, intrinsic motivation,vitality, teaching

INTRODUCTIONThe art of teaching is a continuous journey to find the

right combination of pedagogy, instructional methodo-logies, and more recently, technology to help studentslearn.1-6 Student evaluationshave alwaysbeenan importanttool to gauge how students perceive the classroom environ-ment and the learning process.1-6 Over the years, one themethat seems to be prevalent in student evaluations is howinstructors’ demeanor, behavior, and/or attitude in the class-room affect student learning.5,7 This focus is in contrast topharmacy educators’ sometimes elaborate emphasis on in-novative learning theories, lecture notes, and classroom ac-tivities to bring clinical relevance to course content.

A relationship has been identified between instructorenthusiasm and students’ intrinsic motivation to learn.8-12

Key to this relationship is having instructors who canunlock the “dormant energy” inside their students andinstill vitality in how students pursue the learning pro-cess.9-13 The instructor has to use strategies in the class-room that are student-centered and challenging, and thatput the responsibility on the student as much as on theinstructor to enhance student engagement in learning.14-16

The Center for the Advancement of Pharmacy Education(CAPE) Educational Outcomes 2013 called for “the inclu-sion of an affective domain that would address personaland professional skills, attitudes, and attributes requiredfor the delivery of patient-centered care.”17

By assessing student intrinsic motivation and vital-ity, both of which can impact student affective domain,this study conducted at the School of Pharmacy andHealth Professions, Creighton University, analyzed theimplications of instructor-related factors such as atti-tude, enthusiasm, and teaching style on student learning.The study addresses this topic from the perspective of 2student cohorts (campus and distance pathway students)enrolled in a required Chemical Basis of Drug Action

Corresponding Author: Naser Z. Alsharif, PharmD, PhD,School of Pharmacy and Health Professions, Department ofPharmacy Sciences, Creighton University Medical Center,2500 California Plaza, Omaha, NE 68178. Tel: 402-280-1857.Fax: 402-280-1883. E-mail: [email protected].

7


course in the second year of the doctor of pharmacy(PharmD) curriculum.

METHODSA convenience sample that included distance

(n5187) and campus (n5285) pathway students atCreighton University was used for the study.18 Both co-horts were registered for the required Chemical Basis ofDrugAction course in the spring semester of 2011, 2012, or2013. This 2-credit hour course, taught in the second yearof the curriculum, is delivered concurrently with pharma-cology and follows completion of coursework in biochem-istry, physiology, pathology, anatomy, pharmaceutics, andcommunication skills. The campus students were requiredto attend class. The distance students, who were locatedthroughout the country, followed the same course syllabus,learning objectives, lesson outline, and course activities asdid the campus students.18 Distance students viewedvideos of classroom lectures that were made available tothem within 2 hours after each class. The course instructorwas a tenured professor who had taught the course for 20years.

A department instructor evaluation tool that had beenused for more than 15 years was administered to the stu-dents at the end of the spring semester in each of the yearsof the study. The survey instrument also included study-specific instructor-related items that addressed the courseinstructor’s attitude, enthusiasm, and teaching style (ie,items that were not related to instructional techniques).The survey items were adopted from several studies thataddressed instructor enthusiasm, student intrinsic moti-vation, and student vitality.10,13,19 Although most of theitemswere rated using a 5-point Likert scale, students alsowere asked to provide written responses. Both the quan-titative and qualitative sets of data were analyzed.

Statistical analysis included descriptive statistics andindependent t test for comparing mean evaluation scoresbetween pathways, age groups, and genders. Mean evalu-ation scores also were compared between students whosegrade point average (GPA) ranked in the upper 40% oftheir class (approximately 70% of the students, Group 1)and those whose GPA ranked in the lower 60% (Group 2),as well as by students’ first-year cumulative GPA, second-year cumulative GPA, and first- and second-year cumula-tive GPA. Pearson correlation analysis was conducted toaddress the association between instructor and student-related items. In the multiple regression model, the coursescore was treated as a response variable while first- andsecond-year cumulative GPA, age, student-related items,pathway, and gender were treated as predictor variables.All statistical analyses were conducted using SAS, version9.3 (SAS Institute Inc, Gary, North Carolina). A p value

less than 0.05was considered significant. An exempt statusfor the study was obtained from the Creighton UniversityInstitutional Review Board.

RESULTSThe study involved 187 (39.6%) distance students

and 285 (60.4%) campus students who were admitted tothe PharmD program at Creighton University in 2009,2010, and 2011. The average age for distance studentswas 33.1 years, and for campus students, 24.8 years (Ta-ble 1). The response rate was approximately 100% be-cause it was incorporated into the required end-of-classinstructor evaluation. There were 180 (38.1%) male stu-dents and 292 (61.9%) female students. Distance stu-dents’ average prerequisite GPA (Pre-GPA), first-yearcumulative GPA, and second-year cumulative GPA were3.5, 3.6, and 3.3, respectively, while these variables forcampus students were 3.4, 3.4, and 3.2, respectively.

Almost all campus (93%) and distance students(84%) strongly agreed or agreed that the instructor relatedto them and other students in a manner that promotedmutual respect. In addition, 96% of the distance studentsand 81% of the campus students indicated that theystrongly agreed or agreed that the instructor demonstratedinterest in their success. Further, 94% of the distancestudents and 87% of the campus students indicated thatthey strongly agreed or agreed that the instructor demon-strated professionalism in interaction with them or otherstudents (Table 2). An independent t test for comparingmean evaluation scores showed a significant differencebetween the distance students and the campus students(p,0.001) for all the items related to instructor attitude(Table 2), with significantly higher ratings given by dis-tance students. The majority of the distance students wereolder than 27 years (average age 33.1 years, Table 1), andthe majority of the campus students were 27 years of ageor younger (average age 24.8 years, Table 1). Overall, agewas a significant factor (p,0.001) in all items related toinstructor attitude (Table 3).

There was a difference in the ratio of male to femalestudents between the distance and campus students in theadmitted classes of 2009, 2010, and 2011 (Table 1), witha higher ratio of female to male students in the distanceclasses. The t test analysis for gender did not demonstrateany significant difference in student responses for any ofthe instructor-related items except for question 19 (“pro-vides different ways to learn the content,” p,0.05), or forthe student-related items.

Instructor enthusiasm was measured based on re-sponses to general questions and to questions relatedto verbal and nonverbal behaviors (Tables 2 and 3). Withregard to the instructor enthusiasm general question

8


items, the distance students (96%) and campus students(89%) strongly agreed or agreed with the statement thatthe instructor demonstrated a passionate interest in thetopic. Also, 97% of the distance students and 87% ofthe campus students strongly agreed or agreed with thestatement that the instructor demonstrated mastery of thetopic. When asked about their agreement with the state-ment that the instructor was “full of energy” when teach-ing, 99% of the distance students and 94% of the campusstudents strongly agreed or agreed with it. Further, thedistance students (80% and 77%, respectively) and thecampus students (58% and 50%, respectively) stronglyagreed or agreed with the statements that the instructor“enhancedmymotivation for the pharmacy program” andthat the instructor “enhancedmymotivation formedicinalchemistry” (Table 2). Again, independent t test for com-paring mean evaluation scores showed significantlyhigher scores among distance students than among cam-pus students (p,0.001) for all items related to instructorenthusiasm (Table 2).

Distance students responded more positively tothe instructor enthusiasm questions based on some ofthe instructors’ verbal and nonverbal signs such as vocaldelivery, vocal volume, apparent emotion and level ofenergy. These responses were significantly differentfrom those of campus students (p,0.001) (Table 2).Campus students’ perception about hand and body ges-ture influence on learning was slightly higher than that ofdistance students. However, these were not significantlydifferent (Table 2).

Table3 shows that agewasa significant factor (p,0.01)in all items related to instructor enthusiasm (Table 3). How-ever, agewasnot a factor in 4of 7verbal andnonverbal itemsrelated to instructor enthusiasm (Table 3).

In general, distance student perceptions were signifi-cantly higher (p,0.001) compared to those of campus stu-dents in their agreement with statements related to theinstructor expecting student participation, having appropri-ate expectations, providing relevance for the informationpresented, showing genuine concern for (their) learning,

Table 1. Student Demographics

Admitted Year

Demographics 2009 2010 2011 Three Year Average

AgeCampus 25.6 (N5107) 24.9 (N577) 23.9 (N5101) 24.8 (N5285)Distance 32.8 (N566) 33.1 (N555) 33.5 (N566) 33.1 (N5187)

GenderCampusMale, % 44.9 41.6 47.5 44.9Female, % 55.1 58.4 55.5 55.1

DistanceMale, % 19.7 36.4 28.8 27.8Female, % 80.3 63.6 71.2 72.2

Pre-GPACampus 3.4 3.4 3.4 3.4Distance 3.5 3.5 3.5 3.5Pa 0.004 0.006 0.13 0.001

CGPA1Campus 3.5 3.3 3.4 3.4Distance 3.7 3.5 3.6 3.6Pa ,0.001 0.002 0.005 ,0.001

CGPA2Campus 3.1 3.1 3.3 3.12Distance 3.4 3.1 3.3 3.3Pa 0.002 0.96 0.66 0.048

CGPA12Campus 3.3 3.2 3.4 3.3Distance 3.5 3.3 3.5 3.4Pa ,0.001 0.16 0.06 ,0.001

Abbreviations: Pre-GPA5prerequisite grade point average; CGPA5first-year cumulative grade point average; CGPA25second-year cumulativegrade point average; CGPA125first- and second-year cumulative grade point average.ap value as determined by independent t test.

9


Table 2. Percent Response Frequencies, Means, and p values of Independent t Test Based on Pathway

Survey Items

Response, %a Score,

Pathway (N) SA A N D SD Mean P

Instructor Related ItemsInstructor Attitude1. The instructor related to me and/or other students

in a manner that promoted mutual respect.Campus (N5283) 33 51 9 6 1 4.1 ,0.001Distance (N5187) 61 32 6 1 1 4.5

2. The instructor demonstrated interest in my success. Campus (N5282) 40 41 12 6 1 4.1 ,0.001Distance (N5187) 69 27 3 1 1 4.6

3. The instructor demonstrated professionalismin interactions with me and/or other students.

Campus (N5282) 36 51 8 5 1 4.2 ,0.001Distance (N5187) 63 31 5 1 1 4.6

Instructor Enthusiasm (General Items)4. Demonstrates a passionate interest in his topic. Campus (N5280) 54 35 8 2 1 4.4 ,0.001

Distance (N5185) 77 19 4 0 0 4.75. Demonstrates mastery of his topic. Campus (N5279) 48 39 11 1 0 4.3 ,0.001

Distance (N5186) 70 27 3 0 0 4.76. Is full of energy when teaching. Campus (N5278) 63 31 6 0 0 4.6 ,0.001

Distance (N5183) 76 23 1 0 0 4.77. Enhanced my motivation for the pharmacy program. Campus (N5279) 18 40 25 12 5 3.6 ,0.001

Distance (N5185) 39 41 17 4 0 4.18. Enhanced my motivation for medicinal chemistry. Campus (N5278) 16 34 27 14 9 3.3 ,0.001

Distance (N5186) 39 38 19 4 1 4.1Instructor Enthusiasm (verbal and non-verbal signs)9. Vocal delivery. Campus (N5278) 30 44 13 10 3 3.9 ,0.001

Distance (N5186) 47 39 9 3 1 4.310. Vocal volume. Campus (N5279) 32 40 11 12 4 3.8 ,0.001

Distance (N5187) 48 37 10 2 1 4.311. Facial expressions. Campus (N5281) 30 45 18 5 3 3.9 0.39

Distance (N5187) 33 22 44 1 1 3.912. Apparent emotion. Campus (N5279) 31 43 16 7 2 3.9 ,0.001

Distance (N5185) 49 35 13 3 1 4.313. Hands and other gestures. Campus (N5277) 29 42 17 8 3 3.9 0.60

Distance (N5186) 32 24 38 5 1 3.814. Observed body gestures. Campus (N5274) 29 44 19 7 2 3.9 0.52

Distance (N5185) 30 25 44 1 1 3.815. High level of energy. Campus (N5278) 42 40 14 4 1 4.1 ,0.001

Distance (N5187) 61 34 5 0 1 4.5Instructor Teaching Style16. Expects student participation. Campus (N5277) 75 23 2 0 0 4.7 ,0.001

Distance (N5187) 86 13 1 0 0 4.917. Has appropriate expectations. Campus (N5278) 26 40 15 14 5 3.7 ,0.001

Distance (N5186) 53 33 10 3 1 4.318. Provides relevance for the information presented. Campus (N5279) 31 56 10 2 0 4.2 ,0.001

Distance (N5185) 61 35 3 1 1 4.619. Shows genuine concern for my learning. Campus (N5279) 39 46 10 4 1 4.2 ,0.001

Distance (N5185) 65 33 3 0 0 4.620. Provides different ways to learn the content. Campus (N5279) 26 44 19 8 3 3.8 ,0.001

Distance (N5185) 45 44 10 2 0 4.3Student Related Items

Student Intrinsic Motivation21. I am intrinsically motivated to learn. Campus (N5282) 44 46 9 1 0 4.3 ,0.001

Distance (N5184) 62 33 6 0 0 4.7

(Continued)

10


and providing different ways to learn content (Table 3).Table 3 also shows that age was a significant factor in allitems related to instructor teaching style.

Almost all distance (95%) and campus students(90%) strongly agreed or agreed that they were intrinsi-cally motivated to learn. When asked if they were highlymotivated about the pharmacy program, 97% of the dis-tance students and 94% of the campus students stronglyagreed or agreed. However, the percentages who stronglyagreed or agreed were lower for distance and campusstudents when asked if they were highly motivated aboutthe Chemical Basis of DrugAction course (76% and 58%,respectively) and whether they had greater motivation forclinical courses compared to science courses (62% and79%, respectively). Another finding was that only 62% ofthe distance students and 44% of the campus studentsindicated that they strongly agreed or agreed that theywould like to learn more course content (Table 2). Allthe comparisons between distance and campus students’responses for all of the above items (Table 2) were sig-nificant (p,0.001). Table 3 shows that age was a signifi-cant factor (p values ranged from 0.022 to less than 0.001)in all items related to student intrinsic motivation.

When asked about their agreementwith the statement,“when I am in this class, I feel alive and vital,” 56% of thedistance students and 36% of the campus students stronglyagreed or agreed with the statement (p,0.001) (Table 2).Age was a significant factor (p,0.001) in the item relatedto student vitality (Table 3).

Students whose first-year cumulative GPA ranked inthe upper 40% of their class GPA had significantly higherevaluation scores for instructor enthusiasm, teaching style,and student intrinsic motivation than those whose GPAranked in the lower 60% (p,0.001) (Table 4).A significantdifference was also seen for the second-year cumulative

GPA with teaching style and student intrinsic motivation.When looking at cumulative GPA for the first 2 years ofpharmacy school, students who ranked in the upper 40%had significantly higher evaluation scores for instructorattitude, enthusiasm, teaching style, and student intrinsicmotivation items than those whose GPAs ranked in thelower 60% (p,0.001) (Table 4).

Multiple regression analysis between selected pre-dictors such as first-year cumulative GPA, second-yearcumulative GPA, age, student related items, female gen-der, or campus pathway and student course scoresrevealed the significant factors of second-year cumulativeGPA and student-related item scores in predicting studentfinal course scores. The 2 factors were positively associ-ated with course scores. The model p value was ,0.001and the R2 (coefficient of determination) was 0.32.

DISCUSSIONIn this study, we attempted to identify the impact of

instructor-related factors, including attitude, enthusiasm,and teaching style, on both campus and distance students’intrinsic motivation and vitality. Our objective was to im-prove on the learning experience for all students enrolled inthe Chemical Basis of Drug Action course. Prior to thisstudy, the percentage of students choosing strongly agreeor agree in response to the 3 statements addressing instruc-tor attitude (Table 2) was in the mid to upper 70% range(data not shown). These percentages have steadily in-creased, especially over the last 3 years during which timethe percentage of students who strongly agreed or agreedimproved by as much as 10%. Distance students have al-ways scored these items higher than campus students haveand this may reflect the differences in the dynamics of the“classroom” that each group experiences. In the traditionalclassroom, campus students are challenged to be interactive

Table 2. (Continued )

Survey Items

Response, %a Score,

Pathway (N) SA A N D SD Mean P

22. I am highly motivated about the pharmacy program. Campus (N5280) 51 43 4 2 0 4.4 ,0.001Distance (N5187) 67 30 3 0 0 4.6

23. I am highly motivated about the Chem.Basis of Drug Action course.


24. I have greater motivation for clinical coursescompared to science courses.


25. I would like to learn more about thecontent from this course.


Student Vitality26. When I am in this class, I feel alive and vital. Campus (N5281) 12 24 33 20 12 3.0 ,0.001

Distance (N5186) 18 38 34 7 3 3.6aScale responses include: Strongly Disagree (SD)51, Disagree (D)52; Neutral (N)53; Agree (A)54; Strongly Agree (SA)55.

11


Table 3. Percent Response Frequencies, Means and p values of Independent t Test Based on Age

Survey Items

Percentage5-pointScale

Age (N) SAa A N D SD Mean P

Instructor-Related ItemsInstructor Attitude1. The speaker related to me and/or other students

in a manner that promoted mutual respect.#27 (N5259) 37 48 9 5 1 4.2 ,0.001.27 (N5211) 56 34 8 2 0 4.4

2. The speaker demonstrated interest in my success. #27 (N5258) 43 41 11 4 1 4.2 ,0.001.27 (N5211) 66 27 5 2 0 4.6

3. The speaker demonstrated professionalismin interactions with me and/or other students.

#27 (N5259) 38 49 9 3 1 4.2 ,0.001.27 (N5210) 60 32 5 2 0 4.5

Instructor Enthusiasm (General Items)4. Demonstrates a passionate interest in his topic. #27 (N5258) 57 32 8 3 1 4.4 ,0.001

.27 (N5207) 73 23 4 0 0 4.75. Demonstrates mastery of his topic. #27 (N5256) 52 35 11 1 0 4.4 0.001

.27 (N5209) 65 30 5 0 0 4.66. Is full of energy when teaching. #27 (N5255) 64 31 4 0 0 4.6 0.006

.27 (N5206) 75 22 2 0 0 4.77. Enhanced my motivation for the pharmacy program. #27 (N5255) 23 40 24 10 3 3.7 0.001

.27 (N5209) 32 44 18 6 1 4.08. Enhanced my motivation for medicinal chemistry. #27 (N5255) 22 34 25 13 6 3.5 ,0.001

.27 (N5209) 29 41 21 7 2 3.9Instructor Enthusiasm (verbal and non-verbal Signs)9. Vocal delivery. #27 (N5256) 34 43 13 8 3 4.0 0.07

.27 (N5208) 42 38 12 6 1 4.110. Vocal volume. #27 (N5256) 36 39 11 11 4 3.9 0.005

.27 (N5210) 44 38 12 5 1 4.211. Facial expressions. #27 (N5258) 31 41 22 3 2 4.0 0.20

.27 (N5210) 31 27 38 2 1 3.812. Apparent emotion. #27 (N5255) 35 41 16 6 2 4.0 0.03

.27 (N5209) 44 37 14 4 1 4.213. Hands and other gestures. #27 (N5256) 33 38 20 7 3 3.9 0.16

.27 (N5207) 29 29 34 5 2 3.814. Observed body gestures. #27 (N5252) 32 40 21 5 2 3.9 0.14

.27 (N5207) 28 29 41 2 1 3.815. High level of energy. #27 (N5257) 47 34 14 3 1 4.2 0.01

.27 (N5208) 55 36 7 1 1 4.4Instructor Teaching Style16. Expects student participation. #27 (N5254) 75 22 3 0 0 4.7 0.001

.27 (N5210) 85 15 0 0 0 4.917. Has appropriate expectations. #27 (N5256) 30 39 15 13 4 3.8 ,0.001

.27 (N5208) 47 37 11 5 1 4.218. Provides relevance for the information presented. #27 (N5258) 36 51 11 2 0 4.2 ,0.001

.27 (N5206) 54 39 3 2 0 4.519. Shows genuine concern for my learning. #27 (N5256) 43 43 10 2 1 4.3 ,0.001

.27 (N5208) 61 34 4 1 0 4.520. Provides different ways to learn the content. #27 (N5257) 30 42 17 8 2 3.9 0.003

.27 (N5207) 39 44 12 3 1 4.2Student-Related Items

Student Intrinsic Motivation21. I am intrinsically motivated to learn. #27 (N5259) 44 47 7 1 0 4.3 0.002

.27 (N5207) 57 38 5 0 0 4.5

(Continued)

12


and engaged in the class. However, it is difficult to get all ofthe students to appreciate this, and some are instead intim-idated andnegatively interpret this continuous challenge forthem to be an active participant in the classroom. As onestudent expressed, “I have adifficult time in beingput on thespot when being asked questions, so that teaching style isdifficult to me.”

Distance students on the other hand, do not have thatsame pressure and in many cases they have shared on theend of class evaluations how they are excited about an-swering questions posed in the classroom as they watchthe video. Also, distance students are older than campusstudents by an average age of 8.3 years (Table 1). Inaddition, our resultsmay represent the difference betweengenerations where the majority of campus students aremostly considered generation Y (1980-1994), who are

more comfortable with e-mail and text communicationthan face-to-face communication20-21 while the distancestudents are mostly generation X (1965-1979) who can befiercely independent, self-directed learners who enjoyquestion-and-answer sessions.20,21 Another factor thatcould have impacted the results is student bias related toage, with younger students preferring younger professorsand older students preferring older professors.22 Overall,our data support this because age was clearly a factor thatpositively impacted how students perceived instructor at-titude (Table 3).

Over the years in which the Chemical Basis of DrugAction course has been taught, the instructor identifiedand adopted several key behaviors to help students rec-ognize that he was a partner in their learning process andinterested in their success, including: (1) mastering the

Table 3. (Continued )

Survey Items

Percentage5-pointScale

Age (N) SAa A N D SD Mean P

22. I am highly motivated about thepharmacy program.

#27 (N5258) 52 41 5 2 0 4.4 0.02.27 (N5209) 61 36 3 0 0 4.6

23. I am highly motivated about the Chem.Basis of Drug Action course.

#27 (N5257) 21 42 22 8 7 3.6 0.02.27 (N5209) 27 43 22 5 3 3.8

24. I have greater motivation for clinical coursescompared to science courses.

#27 (N5257) 33 46 18 3 1 4.1 ,0.001.27 (N5208) 30 28 29 12 1 3.7

25. I would like to learn more about thecontent from this course.

#27 (N5257) 13 37 31 11 8 3.4 0.02.27 (N5209) 16 38 36 8 2 3.6

Student Vitality26. When I am in this class, I feel alive and vital. #27 (N5258) 12 27 32 19 9 3.1 0.001

.27 (N5209) 14 34 39 10 3 3.5aScale responses include: Strongly Disagree (SD), Disagree (D); Neutral (N); Agree (A); Strongly Agree (SA)55.

Table 4. Response Comparisons Between Groups Who Ranked Upper 40% (Group 1) and Lower 60% of GPAs (Group 2)

Survey Questions

(Group 1 mean, Group 2 mean, p value for t Test)aPre-GPA bCGPA1 cCGPA2 dCGPA12

Faculty-related itemsFaculty/instructor attitude (13.0, 13.0, 0.82) (12.8, 13.2, 0.09) (12.9,13.1, 0.20) (12.8, 13.3, 0.02)Faculty/instructor enthusiasm (21.2, 21.1, 0.85) (20.9, 21.6, 0.03) (20.9, 21.4, 0.12) (20.8, 21.7, 0.01)Faculty/instructor enthusiasm(verbal and non-verbal signs)

(28.2, 28.0, 0.64) (28.1, 28.2, 0.80) (28.1, 28.1, 0.97) (28.1, 28.2, 0.77)

Faculty/instructor teaching (21.5, 21.4, 0.97) (21.2, 21.9, 0.02) (21.2, 21.8, 0.05) (21.1, 21.9, 0.01)Student-related items

Student intrinsic motivation (20.0, 19.9, 0.70) (19.7, 20.4, 0.01) (19.6, 20.5, ,0.001) (19.6, 20.5, ,0.001)Student vitality (3.3, 3.2, 0.20) (3.2, 3.3, 0.36) (3.2, 3.4, 0.36) (3.2, 3.4, 0.23)

Group 1: Students who ranked upper 40% of their GPAs; Group 2: Students who ranked lower 60% of their GPAs.aPre-GPA: prerequisite grade point average.bCGPA1: first-year cumulative grade point average.cCGPA2: Second-year cumulative grade point average.dCGPA12: first and second year cumulative grade point average.

13


course content; (2) developing different strategies to de-liver the content; (3) stating expectations clearly at thebeginning of the semester; (4) being a role model in in-teraction with students; (5) being present in the learningenvironment whether it is the classroom, online, in theoffice, or as it happens, in the hallway; (6) ensuringprompt communications with students as logistical or ac-ademic issues arise; and (7) responding to concerns bystudents. These behaviors matched well with some of theprinciples for good teaching articulated by other au-thors.23,24 However, as shown by the study data, imple-menting all of the above does not ensure a successfulclassroom environment and learning experience. Clearlystudents’ perceptions of the overall process are important.Thus, this 3-year study, as well as an ongoing evaluationprocess, end-of-course evaluations, and input from classofficers during the semester are strategies that have beenused and will continue to be developed and enhanced toaddress concerns by both student cohorts regarding theseissues outside of the traditional aspects of teaching. Thegoal is to improve the relationship between the instructorand the students and to enhance the interactive classroomenvironment.

As a result of some of the concerns identified by stu-dents related to instructor attitude, several strategies havebeen implemented to address this. The first strategy wastaking more time at the beginning of each class to reviewkey concepts. The second strategy was implementing aninstructional model5,25 to help transition the students toa higher level of thinking and interactivity in the classand to decrease the feeling of intimidation from the de-mands of the in-class time and the perceived energy ofthe instructor. The third strategy was exhibiting more pa-tience with students and accuracy in gauging when the in-structor needs to answer his own questions if there is noresponse by the students. In support of this, Monteiro andassociates demonstrated that the patience, availability, andopenness of the instructor had a positive correlation withstudent academic engagement. 26 An instructor answeringhis or her own questions also ensures that there will not betoo much idle time spent waiting for an answer. The idletime resulting fromwaiting for an answer from the campusstudents was actually a concern that was shared by somestudents from both cohorts. Also, both student cohorts andmore especially studentswhowere strugglingwere encour-aged to feel comfortable to approach the instructor for helpand to take advantage of an open door policy or to make anappointment for a phone call (distance students). Thefourth strategy was identifying any signs of frustration orperceived negative language (eg, statements such as “Areyou with me?” “Does this make sense?” “I hope you rec-ognize this is not rocket science”) for the students’ lack of

participation or perceived motivation. This is important sothat students are not deterred by such statements or per-ceive them as unprofessional behavior. This is critical asinstructors’ perceived misbehavior, including being offen-sive (eg, verbally abuse, humiliate, embarrass, or insultstudents) is viewed negatively by students.27 The latter isalso important because lowmotivation among students hasbeen associated with teacher discouragement.28 A fifthstrategy was recognizing the importance of not exhibitingany reactions that may be perceived as disappointment orridicule when a student answers a question incorrectly, andtaking time to openly and enthusiastically recognize stu-dents when they answer questions correctly or when theydemonstrate the ability to connect information and con-cepts. Recognition is greatly valued by students, especiallyhigh achievers. It is one of the criteria they look for in aneffective instructor and it serves as a positive reinforcementand motivator for them.26

Gender was not a factor in how students respondedon instructor-related items, including instructor attitude,or student-related items. While some studies22, 28,29 iden-tified gender bias in how students evaluate instructors,other studies did not.30-32 Also some studies found thatwomen are more self-determined in the learning pro-cess.33,34 Although our gender data did not show suchpositive findings, it is critical to continue to evaluate gen-der-related factors and incorporate teaching and learningstrategies that have been shown to be effective for bothgenders, such as use of gender-inclusive language.

The majority of both student cohorts responded pos-itively regarding the general question items related to in-structor enthusiasm (Table 2). The written responsesshowed similar agreement, with the theme of enthusiasmprevalent in many of the students’ comments. Thus, thedata (Table 2) clearly demonstrate that for the preponder-ance of students, the apparent enthusiastic attitude of theinstructor was a positive factor in their learning. A majoraspect of that is the perception by students that the instruc-tor hadmastery of the course subjectmatter.Mastery of thesubject in combination with mastery of teaching method-ologies are perceived very positively by students and con-sidered as characteristic of the best teachers.23,35,36

In regard to the questions related to instructor enthu-siasm based on the verbal and nonverbal behavior of theinstructor (Table 2), the difference in the significantlyhigher response by the distance students to how vocaldelivery, apparent emotion, and level of energy influ-enced their learning, and how the campus students per-ceived the hand and body gestures slightly morefavorably, is likely related to the classroom environment.Campus students may have been more distracted by theinstructor’s high volume than by his hand and body ges-

14


tures, which may have been less intimidating or moresubtle. A few distance students commented that the in-structor clapping his hands to emphasize certain conceptsseemed disruptive to them, but the instructor’s vocal de-livery did not appear to bother them. Thus, student inputrelated to the above issues was instrumental in recogniz-ing that, for some students, verbal or nonverbal demon-strations of enthusiasmmay be detrimental to the learningprocess. As a result, it is important for instructors to mon-itor their volume and not to sound or appear as if scream-ing into themicrophone or being aggressive and to controlany perceived negative emotions that may be distractingor alienating to students.27 However, as suggested by stu-dent responses (Tables 2 and 3) and studies in the field ofcommunication, immediate behavior identified by vocalexpressions, communicating at a close distance, smiling,engaging in eye contact, and exhibiting body gestures isassociated with reducing physical and/or psychologicaldistance between instructor and student and have a posi-tive effect on learning.37,38 On the other hand, as dis-cussed above, instructor verbal aggressiveness has beenviewed very negatively by the students.27,39 Thus, gaug-ing the students is important to ensure that balance existsand that behavior and emotions are not perceived in a neg-ative way. One aspect that has been identified in the lit-erature to be helpful also as part of immediate behavior ishumor.37,38 Inserting more humor in the handouts (eg,cartoons) and in vocal delivery may also contribute toa more positive experience.37,38

The instructor setting high but appropriate educa-tional expectations, setting them early, stressing such ex-pectations continuously, reevaluating them based onstudent input, and helping students transition to meetthose expectations is critical. Although the majority ofstudents felt the instructor’s expectations for studentswere appropriate, a significant percentage of campus stu-dents disagreed or were neutral. This finding may reflectthe feelings of students who are not doing as well in thecourse as they may be concentrating on their grade for thecourse rather than onmeeting the challenge of the instruc-tor, which is to have a deeper understanding of the con-tent. Senko and associates have shown that students whopursue mastery goals favor instructors who stimulate andchallenge them intellectually, while those who pursueperformance goals favor instructors who present the ma-terial clearly and provide clear cues about how to succeed. 40

Also, challenging assessments are favored by studentswho are high achievers26 which may explain why the top40% of students in this study hadmore favorable responsesto all instructor-related items than did other students. Basedon the literature that emphasizes the role of the instructor topromote student intrinsic motivation and the development

of deeper approaches to learning,23,24,41-43 the instructorshould challenge students to a higher level of thinkingbut with efforts also to adapt the learning process to theneeds of all students including students who are perfor-mance oriented. This can be accomplished by providingmore structured presentations, taking more time to ex-plain concepts, and offering tips for success. 23,44,45

Overall, the majority of both student cohorts and agegroups (Tables 2 and 3, respectively) recognized the ef-forts done to explain the relevance of the informationpresented. In fact, Ismail and Hayes showed that coursetopics that are difficult to make relevant or fun can nega-tively influence motivation.46 Some of the students men-tioned including test questions based on patient cases onexaminations and relating chemistry to clinical practice inlectures as helpful ways in which the instructor showedthe clinical relevance of the course.

As stated above, the combination of mastery of sub-ject and mastery of instructional methodologies are iden-tified as key characteristics of the best instructors.36,41

Thus, over the years, severalmechanismswere developedto help students explore new and different ways to learnthe content and the effectiveness of these differentmechanisms continues to be evaluated.1-6 The impor-tance of having different ways to learn course contentis supported by the literature36,41 and by students’ writ-ten comments, in which they mentioned the variousteaching tools used such as interactive PowerPointslides, Softchalk lessons, and short video reviews pre-pared by the instructor.

While over 90% of both the distance and campusstudents responded that they were intrinsically motivatedto learn and were highly motivated about the pharmacyprogram, there was much less motivation for taking theChemical Basis of Drug Action course and learning itscontent, especially among campus students (Table 2),75% of whom had greater motivation for clinical courses.Similar data were observed based on age (Table 3). Thelack of student motivation is always an issue with sciencecourses taught in a health sciences professional program,especiallywhen instructors do not try tomake their coursesrelevant for their students.47,48 Instructor enthusiasm hasbeen identified as an external catalyst for the intrinsic mo-tivational energy that may be lying dormant within stu-dents.9-12 It is even more critical when teaching collegestudents as many of them are used to a system of externalincentives (eg, grades). The combination of a positive stu-dent perception of instructor attitude, enthusiasm, andteaching style appears to make a difference in students’overall perception of their learning experience in thecourse, as is demonstrated by the high correlation of thestudent-related itemswith the instructor-related items (Fig-

15


ure 1). However, work still needs to be done to improveoverall student interest in medicinal chemistry as a disci-pline and its importance in pharmacy students’ overall un-derstanding of drug action.

Science instructors in professional health sciencesdegree programs may need to address students’ lack ofinterest in basic science courses more, not only in theclassroom but possibly during student orientation, by giv-ing, for example, seminars on opportunities for graduateeducation and by stressing the value of obtaining botha PharmD and a PhD degree. It is also important for in-structors towork and communicatewith other science andclinical instructors to explain the relevance of medicinalchemistry by incorporating aspects of drug structure andthe science behind it into their teaching of drug action anddrug clinical use. However, while establishing the clinicalrelevance for learning medicinal chemistry is important,science instructors should not “dilute” their discipline.Medicinal chemists must not take the chemistry out ofmedicinal chemistry. Doing so to placate a minority ofstudents would be an injustice to the majority of studentswho enjoy chemistry and appreciate its role in pharmacy –a view that has been expressed by many students. Also,student comments related to the lack of relevance of thecourse or that medicinal chemistry should not be in thecurriculum have decreased dramatically and more posi-tive comments related to the importance of the knowledgegained in this course have increased.5,13 This is moreevident in the last 2 years with a purposeful attempt byboth the medicinal chemistry course instructor and thepharmacology instructors to synchronize the content ofthe 2 courses. This is also supported by the associationbetween student intrinsicmotivation and course score andis reinforced by themultiple regression analysis, which asdiscussed above, showed that the second-year cumulative

GPA and student-related item scores predicted studentfinal course scores. This finding will certainly be sharedwith students in future classes to encourage them to bemore open minded about the course, their experience inthe course, and their overall responsibility for theirlearning.

Intrinsicallymotivated behavior is performed simplyfor the pleasure inherent in the activity itself,49 occurs inthe absence of rewards or reinforcements,50 and is char-acterized by the experience of interest, enjoyment, andcuriosity.11 Intrinsic motivation is empirically linked toachievement test scores and report card grades51 and pos-itive emotions in school.10,52 The literature identifies sup-porting self-determination/autonomy49,50 and promotingperceived competence49,50 as 2 critical components infostering intrinsically motivated behavior in students. Incontrast, research describes individuals exhibiting “amo-tivation” as not being able to complete or value an activ-ity, having no sense of purpose, or exhibiting feelings ofincompetence or learned helplessness.53 The literaturefurther identifies providing autonomy to the students inrespect to choices and decisions about their study withhigh levels of intrinsic motivation.49,50 A summary ofmore specific strategies to support the above findingsfrom the literature are also included in Appendix 1 andwill be continuously evaluated.

The low percentages of students who strongly agreedor agreed with the student-related items (Tables 2 and 3,question 22) clearly indicate that much effort is needed tohelp both student cohorts become more energetic aboutbeing in the course and studying medicinal chemistry.However, although vitality is linked to intrinsic motiva-tion, enthusiasm is identified as a behavioral manifesta-tion of intrinsic motivation while vitality is a subjectiveone,11 and many other factors may affect it; thus, it is

Figure 1. Schematic illustration of study findings. Values represent Pearson’s r correlation coefficients (*p,0.001) betweenfaculty and student related items. Pre-GPA (Pre-requisite Grade Point Average), CGPA1 and CGPA2 (Cumulative Grade PointAverage academic year 1 and academic year 2, respectively). Gender did not show correlation to any of the variables.

16


much more difficult to influence. For example, somestudents may have fear of chemistry, which they maycarry with them into the professional program. How-ever, depending on the extent of such an attitude, it isvery difficult to rid students of it, and this attitude willreflect on their level of energy and enthusiasm in theclassroom. Other factors that could affect students’ vi-tality could be workload in the respective semester andpersonal issues. While assessing congruence betweencredit hours per semester and the amount of work re-quired for each course based on its allocated credit hoursand addressing any personal issues that arise is impor-tant, these efforts may be delayed and may not com-pletely address the impact of workload and personalissues on students’ vitality. Efforts to coordinate amasterexamination schedule, limit the number of examinationswithin a week, establish policies that are sensitive tolegitimate personal student issues, and to evaluatecourses after they are offered and make recommenda-tions to the curriculum committee to improve on for thenext offering can be helpful. Also, this emphasizes theimportance of the instructor-related items since all havebeen shown to positively enhance student intrinsic mo-tivation and student vitality (Figure 1).

While our study addressedmany factors that affect thelearning process for students, many other factors may alsobe in play that have not been directly addressed in thisstudy. These include the ethnic background of the studentsand instructor, cultural values and beliefs of the studentsand the instructor, learning style of the students, and thedifficulty of the course. Also, this study did not showa causal relationship between some of the variables butrather a correlational one (Figure 1). In addition, the studydepended on self-reporting by the students. Further, we didnot attempt to measure students’ motivation and vitality atthe beginning of the course or to control for any of theinstructor-related factors tested in this study. Finally, ourdata were obtained from college students enrolled in a me-dicinal chemistry course in a private professional school inthe United States and are specific to one course instructor;therefore, some findings may not be transferable to otherinstructors. Our results do not prove, for example, that aninstructor who is low key, soft-spoken, and demonstratesa low level of enthusiasm is not an effective teacher. None-theless, our study is a 3-year studywith a large sample size,and it provides unique results related to distance and cam-pus students in a professional pharmacy program. In addi-tion, the findings related to student factors are important inlight of the newCAPEEducational Outcomes 2013, whichemphasize the affective domain aspects of students’ per-sonal and professional growth.17 Further, Figure 1 datamay lend support that faculty enthusiasm items (which

demonstrate the highest correlation with student intrinsicmotivation and vitality) may be a catalyst for all otherfactors to fit in place, establish a healthy faculty-studentrelationship, and improve student learning.

CONCLUSIONWhile this study does address the importance of con-

tent mastery and instructional methodologies, it focuseson issues related to instructor attitude, instructor enthusi-asm, and teaching style, which all were shown to playa critical role in the learning process. Thus, instructorshave a responsibility to evaluate, re-evaluate, and analyzethe above factors to address any related issues that impactthe learning process, including their influence on profes-sional students’ intrinsic motivation, vitality, and abilityto meet educational outcomes.

REFERENCES1. Alsharif NZ, Roche VF. Optimizing the four important

interactions in distance education. Curr Pharm Teach Learn. 2010;

2(2):114-125.2. Alsharif NZ. Drug Structure and treatment algorithm: treatment of

hypertension. Curr Pharm Teach Learn. 2010;2(1):52-66.3. Alsharif NZ, Henriksen B. Assessment of electronic integration of

prerequisite content into a medicinal Chemistry course on student

learning. Am J Pharm Educ. 2009;73(8):Article 150.4. Alsharif NZ. Clinical toxicology: a practical and patient-oriented

approach. Am J Pharm Educ. 2008;72(5):Article 120.5. Alsharif NZ, Galt KA. Evaluation of an instructional model to

teach clinically relevant medicinal chemistry in a campus and

distance pathway. Am J Pharm Educ. 2008;72(2):Article 31.6. Alsharif NZ, Shara MA, Roche VF. The structurally-based

therapeutic evaluation concept: An opportunity for curriculum

integration and interdisciplinary teaching. Am J Pharm Educ.

2001;65:314-323.7. Alsharif NZ. Faculty enthusiasm: a blessing or a curse. Am

J Pharm Educ. 2011;75(2):Article 23.8. Dille AK, Placone D. Teacher characteristics and student learning.

J Econ Econ Educ Res. 2008; 9(3):15-28.9. DeLong M, Winter D. Learning to Teaching and Teaching to

Learn Mathematics: Resources for Professional Development.

Mathematical Association of America; 2002: 163.10. Patrick B, Hisley J, Kempler, T. What’s everybody so excited

about?: the effects of teacher enthusiasm on student intrinsic

motivation and vitality. J Exp Educ. 2000;68(3):217-236.11. Deci EL. Interest and the intrinsic motivation of behavior. In:

Renninger KA, Hidi S, Krapp A, eds. The Role of Interest in Learning

and Development. Hillsdale, NJ: Erlbaum; 1992:43-70.12. Deci EL, Vallerand RJ, Pelletier LG, Ryan, RM. Motivation and

education: the self-determination perspective. Educ Psychol. 1991;

26(3/4):325-346.13. Ryan RM, Frederick C. On energy, personality and health:

subjective vitality as a dynamic reflection of well-being. J Pers.

1997;65(3):529-565.14. Yair G. Educational battlefields in America: the tug-of-war

over students’ engagement with instruction. Sociol Educ. 2000;

73(4):155-174.

17


15. Brown M. Dialogue for engagement. Educause Rev. 2010;45(5):38-40.16. Stoner SC, Fincham JE. Faculty role in classroom engagementand attendance. Am J Pharm Educ. 2012;76(5):Article 75.17. The Center for the Advancement of Pharmacy Education.Educational Outcomes 2013. http://www.aacp.org/resources/education/cape/Open%20Access%20Documents/CAPEoutcomes2013.pdf. Accessed November 5, 2013.18. Malone PM, Glynn GE, Stohs SJ. The development and structureof a web-based entry-level doctor of pharmacy pathway at CreightonUniversity Medical Center. Am J Pharm Educ. 2004;68(2):Article 46.19. Collins ML. Effects of enthusiasm training on pre-serviceelementary teachers. Res Teach Educ. 1978;29(1):53-57.20. Coates J. Generational Learning Styles. River Falls, WI: LernBooks; 2007.21. Fisher T, Crabtree J. Generational cohort theory: have weoverlooked an important aspect of the entry-level occupationaltherapy doctoral debate. Am J Occup Ther. 2009;63(5):656-660.22. Sprinkle JE. Student perceptions of effectiveness: an examinationof the influence of student biases. Coll Stud J. 2008;42(2):276-293.23. Chickering AW, Gamson ZF. Seven principles for good practicein undergraduate education. AAHE Bulletin. 1987;39(7):3-7.24. Kuh GD, Kinzie J, Buckley JA, Bridges BK, Hayek JC. (2006).What matters to student success: a review of the literature.Commissioned report for the National Symposium on PostsecondaryStudent Success: Spearheading a Dialogue on Student Success. http://nces.ed.gov/npec/pdf/Kuh_Team_Report.pdf. Accessed August 2, 2014.25. Alsharif NZ, Galt K, Mehanna A, Chapman R, OgunbadeniyiAM. Instructional model to teach clinically relevant medicinalchemistry. Am J Pharm Educ. 2006;70(4):Article 90.26. Monteiro S, Almeida LS, Vasconcelos RM. The role of teachersat university: what do high achiever students look for? J ScholarshTeach Learn. 2012;12(2):65-77.27. Thweatt KS, McCroskey JC. Teacher non-immediacy andmisbehavior: unintentional negative communication. Commun ResRep. 1996;13(2):198-204.28. Arbuckle J, Williams BD. Students’ perceptions ofexpressiveness: age and gender effects on teacher evaluations. SexRoles. 2003;49(9/10):507-516.29. Amin ME. Gender as a discriminating factor in the evaluation ofteaching. Asses Eval Higher Educ. 1994;19(2):134-143.30. Feldman KA. College students’ views of male and femalecollege teachers: Part I. Evidence from the social laboratory andexperiments. Res Higher Educ. 1992;33(3):317-375.31. Feldman KA. College students’ views of male and femalecollege teachers: Part II. Evidence from the students’ evaluations oftheir classroom teachers. Res Higher Educ. 1993;34(2):151-191.32. Fernandez J, Mateo MA. Student and faculty gender in ratings ofuniversity teaching quality. Sex Roles. 1997;37(11/12):997-1003.33. Karesenti T, Thivert G. The influence of gender on within-termchanges in junior college student motivation. Paper presented at:American Educational Research Association; New Orleans, LA.34. Vallerand RJ, Fortier MS, Guay F. Self-determination andpersistence in a real-life setting: Toward motivational model of highschool dropout. J Pers Soc Psychol. 1997;72(5):1161-1176.

35. Davies J, Arlett C, Carpenter S, Lamb F, Donaghy L. What

makes a good engineering lecturer? Students put their thoughts in

writing. Eur J Eng Educ. 2006;31(5):543-553.36. Krauss S, Brunner M, Kunter M, et al. Pedagogical content

knowledge and content knowledge of secondary mathematics

teachers. J Educ Psychol. 2008;100(3):716-725.37. Witt PL, Wheeless LR. Nonverbal communication expectancies

about teachers and enrollment behavior in distance learning. Commun

Educ. 1999;48:149-154.38. Will PL, Wheeless LR. An experimental study of teachers’

verbal and nonverbal immediacy and students’ affective and

cognitive learning. Commun Educ. 2001;50(4):327-342.39. Myers SA, Rocca KA. Perceived instructor argumentativeness

and verbal aggressiveness in the college classroom: Effects on

student perceptions of climate, apprehension and state motivation.

West J Commun. 2001;65(2);113-137.40. Senko C, Hulleman CS, Harackiewiez JM. Achievement goal

theory at the crossroads: old controversies, current challenges and

new direction. Educ Psychol. 2011;46(1):26-47.41. Strahan D. Successful teachers develop academic momentum

with reluctant students. Middle Sch J. 2008;39(5):4-12.42. Biggs J. Teaching for Quality at University: What the Student

Does. Buckingham: Society for Research into Higher Education; 2000.43. Menges RJ, Austin AE. Teaching in higher education. In:

Richardson V, ed. Handbook of Research on Teaching. Washington,

DC: American Educational Research Association; 2001:1123-1156.44. Heller KA. Identification of gifted and talented students. Psychol

Sci. 2004;46(3):302-323.45. Tomlinson CA, Brighton C, Hertberg H, et al. Differentiating

instruction in response to student readiness, interest and learning

profile in academically diverse classrooms: a review of literature.

J Educ Gifted. 2003;27(2/3):119-145.46. Ismail B, Hayes K. Factors that affect student motivation in

a dairy products elective course. J Food Science Educ. 2004;

4(1):15-17.47. Harrold MW. Educational challenges facing basis science

faculties. Am J Pharm Educ. 2004;68(1):Article 15.48. Woster PM. Maintaining basic science content throughout the

PharmD curriculum. Am J Pharm Educ. 2003;67(3):Article 99.49. Deci EL, Ryan RM. A motivational approach to self: Integration

in personality. In: Dienstbier R, ed. The Role of Interest in Learning

and Development. Hillsdale, NJ: Erlbaum; 1991:43-70.50. Deci EL. Effects of externally mediated rewards on intrinsic

motivation. J Pers Soc Psychol. 1971;18(1):105-115.51. Gottfried AE. Academic intrinsic motivation in young

elementary school children. J Educ Psychol. 1990;82(3):525-538.52. Patrick BC, Skinner EA, Connell JP. What motivates children’s

behavior and emotion? The joint effects of perceived control and

autonomy in the academic domain. J Pers Soc Psychol. 1993;

65(4):781-791.53. Baker S. Intrinsic, extrinsic and amotivational orientation: their

role in university adjustment, stress, well-being and subsequent

academic performance. Curr Psychol. 2004;23(3):189-202.

18


Appendix I. Course Specific Strategies to Support Components of Intrinsic Motivation for Students

Intrinsic Motivation Component Course Specific Strategies to Achieve

1. Supporting ofself-determination

d Conduct pre-assessment quiz to prepare for the classroom session.d Provide interactive classroom session.d Provide several active learning exercises on the course website and in the in-class

interactive PowerPoint presentations.d Provide interactive Softchalk lesson handout with learning activities.d Encourage use of analogies (e.g. food analogy exercise and

structure activity relationship).d Encourage students to write own innovative short story related to the content being

taught (eg, The magnificent penicillins, The Town of Neurotransmitoron,The Attack on Muskulopolis, The Tale of Aminoglycosides).

d Encourage students to write their own take home message.d Encourage students to find the clinical relevance of what is being taught.d Help students characterize what they are learning by utilizing strategies based on

Krathwol Taxonomy.53

d Encourage notion of faculty member as a facilitator/partner in the learning process.

2. Promotion of perceivedcompetence

d Help the students transition to the higher level of thinking required in this course.Provide a standardized lesson handout based on Bloom’s andKrathwol’s Taxonomy.53

Challenge students to answer questions in classroomChallenge students to answer their own questions in the classroom, online, andin face-to-face and virtual review sessions.Provide practice exams and case studies.Challenge the students to write an original case study for the exam.Provide student answers as key answers for questions on the exam.Provide constructive comments on assignments and exams.

19


RESEARCH

Identifying Achievement Goals and Their Relationship to AcademicAchievement in Undergraduate Pharmacy Students

Saleh Alrakaf, MSc Pharm,a Erica Sainsbury, PhD,a Grenville Rose, PhD,b and Lorraine Smith, PhDa

a Faculty of Pharmacy, The University of Sydney, Sydney, New South Wales, Australiab Aftercare, Sydney, New South Wales, Australia

Submitted July 1, 2013; accepted February 9, 2014; published September 15, 2014.

Objectives. To compare the achievement goal orientations of first-year with those of third-year un-dergraduate Australian pharmacy students and to examine the relationship of goal orientations toacademic achievement.Methods. The Achievement Goal Questionnaire was administered to first-year and third-year studentsduring class time. Students’ grades were obtained from course coordinators.Results.More first-year students adopted performance-approach and mastery-approach goals than didthird-year students. Performance-approach goals were positively correlated with academic achieve-ment in the first year. Chinese Australian students scored the highest in adopting performance-approachgoals. Vietnamese Australian students adopted mastery-avoidance goals more than other ethnicities.First-year students were more strongly performance approach goal-oriented than third-year students.Conclusion. Adopting performance-approach goals was positively correlated with academic achieve-ment, while adopting avoidance goals was not. Ethnicity has an effect on the adoption of achievementgoals and academic achievement.

Keywords: achievement goals, achievement goal questionnaire, academic achievement, ethnicity

INTRODUCTIONThe role of goals in human motivation is critical.1

Goals can be defined as a cognitive representation of fu-ture aims that a person is committed either to approach oravoid.2 A class of goals that has received considerableattention in the educational field for more than 2 decadesis achievement goals.3,4 According to achievement goaltheory, achievement goals are goals in which “compe-tence” is the main aim for an individual.5 Thus, achieve-ment goals are defined as “a future-focused cognitiverepresentation that guides behavior to a competence-related end state that the individual is committed to eitherapproach or avoid.”6 Achievement goal theory tries todescribe and understand the goals students adopt whendealing with academic activities and the reasons behindsuch adoption.7,8 For example, when students face anacademic activity, they adopt either 1 of 2 major typesof achievement goals:mastery goals (ie, to try to learn andunderstand the task on hand) or performance goals (ie, totry to performwell compared to peers).1,9,10 Achievementgoal theorists believe that students who adopt mastery

goals and students who adopt performance goals viewability and define success vs failure differently.3

Students who adopt mastery goals tend to view theirabilities as a flexible trait that can be enhanced by hardwork, persistence, and continuous development of theirskills,10 while students who adopt performance goalsview ability as a fixed trait that cannot be enhanced.11

Mastery students use self-referential criteria in differen-tiating between success and failure (ie, feeling they learnwhat they need to learn or improve),10 whereas perfor-mance students define success as outperforming theirpeers.6 Thus, performance studentswho believe they havehigh ability will enjoy outperforming their peers, whileperformance students who believe they have low abilitywill avoid such challenges.3

In recent years, achievement goal theorists have fur-ther divided mastery goals and performance goals into 4types: mastery-approach (ie, aiming to learn and under-stand the task at hand thoroughly),mastery-avoidance (ie,aiming to avoid losing previously acquired skills or toavoid not understanding the task at hand thoroughly),performance-approach (ie, aiming to outperform one’speers or to demonstrate one’s ability to others), andperformance-avoidance (ie, aiming to avoid performingworse than one’s peers).12-15 These distinctions are

Corresponding Author: Saleh Alrakaf, Faculty of Pharmacy,Room S114, Pharmacy and Bank Building A15, TheUniversity of Sydney, NSW, 2006. Tel: 161-2-9351-4501.Fax: 161-2-9351-4451. E-mail: [email protected]

20


supported by a large body of empirical research and arerobust in predicting and understanding student engage-ment and achievement.6,16,17

Themastery-approach goal has been linked to a num-ber of positive effects such as deep learning,18 high in-terest,19 high persistence,20 and help seeking.21 Despitethese beneficial effects, evidence suggests that studentswho adopt this type of goal orientation rarely attain highacademic achievement (ie, grades).6,10 The performance-approach goal, on the other hand, is associated with shal-low learning strategies such as memorization22 but linkedto high academic achievement.8,10,23,24 Performance-avoidance and mastery-avoidance goals have been asso-ciated with negative effects, especially in Westernculture, such as stress and anxiety,25-27 low academicachievement,28,29 and low intrinsic motivation.30,31

In general, cross-sectional studies designed to com-pare undergraduate students’ achievement goals arescarce. However, there is some evidence that suggeststhere are differences in the achievement goals adoptedby students in different academic years. Lieberman andRemedios examined the achievement goals of 1857 un-dergraduate students from first, second, third, and fourthyears who were studying in different disciplines such aspsychology, business, biology, art, English, history,mathematics, and nursing at a Scottish university, andfound that students in the first year were more mastery-approach oriented than students from any other year.32

The authors attribute their findings to increased pressureon students as they advance through their academiclife. This academic pressure undermines interest andenjoyment, and thus, significantly decreases pursuingmastery-approach goals. Another study conducted byRemedios et al to identify and compare the achievementgoals of Russian undergraduate students, whowere takingEnglish studies course for business in different academicyears, yielded strikingly similar results.9 However, theauthors explained their results in the context of the cul-tural shift in Russia caused by globalization, which influ-enced students to be more individualistic and pragmatic,with more emphasis on performance than mastery goals.

Only a few studies have aimed to investigate the re-lationship between ethnicity and achievement goals.33

For example, Elliot and his colleagues found AsianAmerican undergraduate psychology students adoptedperformance-avoidance goals more than their AngloAmerican peers.34 The authors attributed these findingsto subcultural differences between the 2 groups. In gen-eral, students from Asian backgrounds valued avoidingnegative outcomes, whereas approaching positive out-comes was valued in Anglo American culture.34 Similarfindings have been found by Zusho et al; however, they

found that undergraduate Asian American students out-scored their Anglo American peers in mathematics.35

Although more than 1000 publications and disserta-tions report the application of achievement goal theory,6

only one of these (to our knowledge) is in the pharmacyeducation setting. Waskiewicz used a framework basedon achievement goal theory to determine student motiva-tion to achieve in a low stakes examination, comparedto their motivation to pursue a doctor of pharmacyprogram.36 The author found a significant relationshipbetween situational motivation and performance-approach goal. However, no significant relationship wasfound between the same motivation and mastery-approach goal.

Achievement goal theory provides academics withinvaluable understanding of how their students respondwhen they encounter academic activity.29 By understand-ing students’ achievement goals, academics might try tocreate an environment that can encourage those beneficialgoals and limit the non-beneficial ones.7 Yet the first stepis to understand more about pharmacy students’ achieve-ment goals.

Therefore, the aims of this study are to compare theachievement goal orientations between first-year andthird-year undergraduate pharmacy students, investigateAustralian undergraduate pharmacy students’ achieve-ment goals and their relationship to their academicachievement, and examine the influence of different eth-nicities on achievement goals and academic achievement.

METHODSThis study received approval from the Human Re-

search Ethics Committee, The University of Sydney,NSW Australia.

The bachelor of pharmacy degree program at theUniversity of Sydney is a 4-year undergraduate courseof study that qualifies graduates to apply for registrationas a pharmacist in Australia.37 The participants for thisstudy were first-year and third-year undergraduate stu-dents in this program. In total, 380 students agreed toparticipate in the study.

We used the Achievement Goal Questionnaire(AGQ), which contains 12 items intended to measurethe 4 types of achievement goals on a 7-point Likert scale(15not at all true of me to 75very true of me). Socio-demographic indicators included in the survey weregender, age, language spoken at home, and student iden-tification (ID) number. Student ID numbers were usedonly for matching students’ grades with the differenttypes of achievement goals. Individual students couldnot be identified in the analysis. Completion of the sur-vey instrument took approximately 10 minutes.

21


Data regarding participants’ ethnicities were gath-ered by asking students to indicate the language spokenat home, which may be interpreted as more accuratelyreflecting the cultures of participants than asking for eth-nicity in a general question.39 Culture is a variable of in-terest as it is the prism throughwhich individuals view theworld and may specifically affect their approach to edu-cation.40 Another benefit of this question is that it enabledus to identify participants’ ethnicities with greater preci-sion. For example, instead of writing “Asian” in answer-ing an ethnicity question, a participant would indicate theprecise ethnicity, such as Vietnamese or Korean, whenidentifying the language spoken at home.

The study was initiated in the second semester of theacademic year in 2012. Students were invited to partici-pate in the study during normal lectures or tutorials (ie,small group discussions). They were advised that partic-ipation was voluntary and if they chose to participate theycould withdraw from the study at any time. In addition,students were advised that their decision to participatewould not impact their academic performance results orinfluence student-teacher relationships. Researchersapproached students as a group.

At the end of the semester, students’ raw grades in 2courses, Foundations of Pharmacy and Endocrine, Diabe-tes and Reproductive [System], were obtained fromcourse coordinators. Foundations in Pharmacy is a first-year course aiming to introduce students to the pharmacyprofession and the roles of pharmacists in the health caresystem.41 Endocrine, Diabetes and Reproductive isa course taken in the third year that covers the pharmaco-therapeutics of endocrine, diabetes and reproductivedisorders.41

SPSS 20 (SPSS Inc, Chicago, Illinois) was used forall statistical analyses. Descriptive statistics regardingyear group, gender, age, and language spoken at homewere reported. Correlation analysiswas used to determinethe strength and direction of the relationships betweenachievement goals and academic achievement. An inde-pendent sample t test was used to compare the achieve-ment goal orientations between first-year and third-yearstudents. One-way analysis of variance (ANOVA) wasused as preliminary analysis for multiple comparisonsof predominant languages spoken at home and each typeof achievement goal. A 2-way ANOVAwas conducted toexplore the impact of academic year (first or third) andpredominant ethnicities on each achievement goal. Sim-ilar analysis was used to explore the same impact on ac-ademic achievement. All mean difference analyses weresubjected to post hoc tests (Tukey test).

A direct logistic regression procedurewas performedto determine the extent to which achievement goals and

languages spoken at home contributed to academicachievement. Academic achievement was transformedinto a binary variable using the grade 74 as a cut point.Thus, students’ grades were regressed as pass and creditvs distinction and high distinction. Languages spokenat home also were transformed into a binary variable(English/other languages). The Forced Entry Method wasused to examine the odds ratios of all variables, even if notsignificant. A p value of less than 0.05 was consideredsignificant for all analyses.

RESULTSThree hundred eighty (251 female, 128 male, and 1

gender unspecified) undergraduate pharmacy studentsfrom years 1 and 3, with a mean age of 19.7 years, agreedto participate in this study (76% response rate). Descrip-tive statistics for participants are reported in Table 1.

The predominant languages spoken at home (ethnic-ities) in approximately 90% of both classes of studentswere English, Chinese, Vietnamese, Korean, and Arabic.The number of different ethnicities reported by first-yearand third-year students was 22 and 13, respectively.

Independent t test results (Table 2) revealed dif-ferences between first-year and third-year students inperformance-approach and mastery-approach goals, withfirst-year students scoring significantly higher than third-year students in both. In contrast, no significant differencesin the scores of first- and third-year students were observedfor performance-avoidance and mastery-avoidance goals.

Correlations between achievement goals and gradesare reported in Table 3. Among first-year students, higherscores on performance-approach goals were associ-ated with higher grades. In the same year, adoption ofperformance-avoidance goals significantly correlated

Table 1. Demographics of First- and Third-Year PharmacyStudents

AcademicYear N

Gender,% Female

Age, Mean(SD) Language, %

First-year 260 67.7 18.8 (2.12) English, 28.4Chinese,a 24.1Vietnamese, 15.2Arabic, 11.7Korean, 8.2Other, 12.4

Third-year 120 62.5 21.5 (3.56) English, 39.0Chinese,a 27.0Korean, 13.6Vietnamese, 8.5Arabic, 5.1Other, 6.8

a Chinese5 Cantonese, Mandarin, Chinese, and Teochew languages.

22


with lower grades. Among third-year students, adoptionof mastery-avoidance goals significantly correlated withlower grades. Although collapsing different Asian ethnic-ities (ie, Chinese, Vietnamese, and Korean) into onegroup has statistical appeal, yielding greater power,a one-way, between group ANOVA revealed significantdifferences at the p,0.05 level in performance-approachand mastery-avoidance goals and academic achievementscores among the 3 ethnic groups. Thus, each predomi-nant Asian ethnicity was analyzed separately.

A 2-way, ANOVA was conducted to explore theimpact of students’ academic year and predominant eth-nicity on each achievement goal. Ethnic differences inmean levels of goals and year are reported in Table 4.Regarding performance-approach goals, both academicyears and ethnicity had significant impact. Post hoccomparisons of the main effect using the Tukey test in-dicated that Chinese Australian students reported higherperformance-approach goals than their Anglo Australianand Korean Australian peers. No significant interactionwas found between the predominant ethnicities andstudents’ academic year. No significant impact wasfound from academic year (or predominant ethnicity) onperformance-avoidance goals. The interaction effect be-tween predominant ethnicity and academic year was notsignificant. Only academic year had a significant impacton mastery-approach goals. The interaction effect be-tween predominant ethnicity and academic year was not

significant. The impact of academic year and ethnicity onmastery-avoidance goals was significant. Post hoc com-parisons using the Tukey test indicated that VietnameseAustralian students reported higher adoption of mastery-avoidance goals than their Anglo Australian and ArabAustralian peers. The interaction effect between predom-inant ethnicities and students’ academic year was notsignificant.

A two-way, between-groups ANOVA was also con-ducted to explore the impact of academic year and pre-dominant ethnicity on students’ grades and are shown inTable 5. All effects were significant at the 0.05 level. Thetwo-way ANOVA of students’ grades based on yeargroup showed that mean scores were significantly higherfor third-year (Mean6SD, 74.768.8) compared to first-year (Mean6SD, 70.468.3. There was a significant maineffect for ethnicity. Post hoc comparisons using the Tukeytest lacked the power to determine where that differencewas, beyond that it was between the ethnicities scoringhighest and lowest in this study, which was determinedfrom the main effect. The interaction effect between pre-dominant ethnicities and academic year was not signifi-cant. Direct logistic regression was performed to assessthe impact of a number of factors on the students’ grades.The model contained 5 independent variables (the 4achievement goals, and ethnicities). The full modelcontaining all predictors was significant. The modelas a whole explained between 5.1% (Cox and Snell Rsquared) and 6.9% (Nagelkerke R squared) of the vari-ance, and correctly classified 63.9% of cases.

As shown in Table 6, only 2 of the independent vari-ables made a unique significant contribution to the model(mastery-avoidance and other ethnicities). The strongestpredictor of grades was mastery avoidance, recording anodds ratio of 0.83. This indicated that students who pur-sued the mastery-avoidance goal were less likely toachieve high grades than those who did not pursue thisgoal, controlling for all other factors in the model. Otherethnicities were also significantly predictive of higheracademic achievement with an odds ratio of 1.62.

DISCUSSIONFor more than 2 decades, achievement goal theory

has captured a considerable amount of attention in

Table 2. Achievement Goal Values in First- and Third-YearPharmacy Students

Mean (SD) P

Performance-approachFirst-year 5.1 (1.3) ,0.001Third-year 4.5 (1.4)

Performance-avoidanceFirst-year 5.6 (1.3) 0.25Third-year 5.5 (1.3)

Mastery-approachFirst-year 5.9 (1.0) 0.01Third-year 5.6 (1.2)

Mastery-avoidanceFirst-year 4.8 (1.5) 0.15Third-year 4.6 (1.4)

Table 3. Correlations between Grades and Achievement Goals

Year Grades Performance-approach Performance-avoidance Mastery-approach Mastery-avoidance

1 R 0.14 -0.14 0.06 -0.07P 0.04 0.03 0.32 0.27

3 R 0.05 -0.18 -0.16 -0.31P 0.61 0.06 0.09 0.001

23


education, with more than 1000 articles and dissertationsbeing written using it as a framework.3,4,6 Four types ofachievement goals are acknowledged: mastery-approach,master-avoidance, performance-approach, andperformance-avoidance.12-15 The primary aims of this study were toidentify Australian undergraduate pharmacy students’achievement goals, determine the relationships betweengoals and academic achievement, and compare theachievement goals of 2 different cohorts of undergraduatestudents. A secondary aim of this study was to investigateany relationships between ethnicity, type of achievementgoals, and academic achievement.

Comparison of first-year and third-year students’ re-sults showed that first-year students were oriented morestrongly toward performance-approach and mastery-approach goals than third-year students. Our finding thatfirst-year students adopted mastery-approach goals morethan third-year students is consistent with Lieberman andRemedios32 and Remedios et al.9 However, our findingsregarding performance-approach differed from both pre-vious studies. Results from Remedios et al. found no sig-nificant differences in the adoption of this goal amongfirst-year, second-year, third-year, and fourth year stu-dents.9 Lieberman and Remedios found third-year stu-dents adopted performance-approach goals more thanfirst-year students. In our study, the higher adoption ofperformance-approach goals by first-year students com-pared to third-year students might have been a result of

competitive high school environments from which first-year students had just come.

Our results show that first-year studentswho adoptedperformance-approach goals received higher grades intheir subject compared to their peers who adopted anyother type of achievement goals. These findings are con-sistent with several previous studies that indicate the posi-tive association between performance-approach andacademic achievement.8,10,23,24,28,29,42 Perhaps students

Table 4. Ethnic and Year Group Achievement Goals , Mean (SD)

Goals, n=1st yr/3rd yr Anglo 73/46 Chinese 62/32 Vietnamese 39/10 Korean 21/16 Arabic 30/6

Performance-approachYear 15.0 (1.4)e 4.9 (1.5)a 5.3 (1.2)b 5.4 (1.1)a,b,c 4.6 (1.6)a,c 4.7 (1.4)a,b,c

Year 34.6 (1.4)f 4.4 (1.6) 5.0 (1.2) 4.5 (1.4) 4.3 (1.3) 4.4 (1.3)

Performance-avoidanceYear 15.6 (1.3)a 5.6 (1.3)a 5.6 (1.2)a 5.9 (1.1)a 5.8 (1.4)a 5.3 (1.5)a

Year 35.5 (1.2)a 5.6 (1.3) 5.4 (1.2) 5.8 (1.3) 5.3 (1.1) 6.3 (0.8)

Mastery-approachYear 15.6 (1.0)e 6.0 (1.0)a 5.9 (0.8)a 6.0 (1.2)a 5.4 (1.4)a 5.7 (1.1)a

Year 35.5 (1.2)f 5.5 (1.3) 5.3 (1.2) 5.8 (1.0) 5.7 (1.0) 4.4 (1.3)

Mastery-avoidanceYear 14.9 (1.5)e 4.6 (1.6)a,c 5.1 (1.1)a,b,c 5.5 (1.3)b 4.5 (1.5)a,b,c 4.3 (1.6)a,c

Year 34.5 (1.3)f 4.2 (1.5) 4.7 (1.3) 4.8 (1.3) 4.6 (1.2) 4.3 (0.9)

Means in the same row that do not share the same superscripts differ significantly at p,0.05.

Table 5. Academic Achievement Mean (SD) Scores for EachEthnicity in Both Year Groups

Ethnicity Year (n) Mean (SD)

AngloYear 1 67 71.5 (9.0)Year 3 45 75.0 (9.6)

ChineseYear 1 58 70.7 (6.7)Year 3 29 76.0 (7.1)

VietnameseYear 1 38 68.7 (7.4)Year 3 8 70.6 (7.6)

KoreanYear 1 17 65.0 (9.4)Year 3 15 72.7 (9.2)

ArabicYear 1 28 72.5 (8.5)Year 3 6 75.8 (10.4)

24


who adopt a strong performance-approach goal orienta-tion focus on topics that appear important and testablefor their teachers. In contrast, students who are stronglymastery-oriented are more likely to follow their own in-terest and study subject material that is appealing to themregardless of its testability.43 Almost all faculty memberswant their students to be curious and interested, and to usedeep-learning strategies (ie, adopt a mastery-approachgoal) when they study and, at the same time, attainhigher grades (ie, adopt a performance-approach goal).Although reaping the benefits of both types of achieve-ment goals is clearly beneficial, the task for educators is todevelop ways to foster this combination. One way is byhelping students pursue mastery-approach goals through-out the semester and then encouraging them to pursueperformance-approach goals when preparing for exami-nations.19 This can be achieved through appropriate cur-riculum development and an understanding of teacherqualities that enhance and support the delivery of coursecurricula.19 These qualities, if adopted by academicians,might help create a combined mastery-approach andperformance-approach environment.

Surprisingly, among the third year students, there wasno significant relationship between academic achievementand performance-approach goals. This result was inconsis-tent with previous research findings.8,22,23,27-30 Althoughour data did not allowus to elucidatewhy thiswas,we positthat the nature of the examined course (ie, Endocrine, Di-abetes and Reproductive) did not support shallow learningstrategies such as memorization. Thus, adoption of thistype of achievement goal had no significant associationwith academic achievement.

In contrast tomuchof the published literature,33,34,44,45

which has grouped different Asian ethnicities under oneumbrella and applied findings to the whole group, ourstudy clearly revealed that individual Asian ethnicitiesvaried in their adoption of each type of achievementgoal. Vietnamese students, for example, had significantly

higher scores on mastery-avoidance goal than their Ko-rean peers, whereas Chinese students had significantlyhigher performance-approach goal scores than Koreanstudents. To the best of our knowledge, this is the firststudy to analyze each Asian ethnicity separately, anddoing so has yielded significant conclusions. Zusho et aldid not find any significant difference between AsianAmerican and Anglo American students in pursuingperformance-approach goals,35 but our study showed thatChinese Australian students were adopting performance-approach goals significantly more than their Anglo Aus-tralian peers, possibly becauseChineseAustralian parentsexpect high academic performance from their children.45

However, there were no significant differences betweenAnglo Australian and Vietnamese Australian or KoreanAustralian students. Our finding that more VietnameseAustralian students adopted mastery-avoidance goalsthan did their Anglo Australian peers was consistent, tosome degree, with literature that found more Asianstudents adopted avoidance goals than did Caucasianstudents.33,34,46

The contradictory findings of this study in compari-son with previous research may be attributed to 3 factors.First, this study made a clear distinction between Asianethnicities while most other studies have not, suggestingthat a “one group fits all” approachmisses the opportunitytomore precisely understand different ethnic groups. Sec-ond, most published literature focuses on psychologystudents.34,35 There might be a correlation amongdiscipline-specific subjects, achievement goals, and aca-demic achievement. Third, this study was conducted inAustralia and given the multicultural nature of the coun-try, particularly in Sydney, the study suggests that nosingle strategy may suit all Australian students, and thatfuture work should address cultural differences moredirectly.

Although there was a significant impact of predom-inant ethnicities upon academic achievement, post hoc

Table 6. Logistic Regression Analysis Examining Different Factors That Might Predict Academic Achievement.

B S.E. p Odds Ratio

95% C.I. for EXP(B)

Lower Upper

Age -0.046 0.046 0.319 0.955 0.872 1.046Females 0.563 0.260 0.030 1.756 1.056 2.922Performance-approach 0.073 0.086 0.395 1.076 0.909 1.273Performance-avoidance -0.157 0.090 0.082 0.854 0.716 1.020Mastery-approach -0.113 0.112 0.310 0.893 0.717 1.111Mastery-avoidance -0.191 0.085 0.024 0.826 0.700 0.975Other ethnicities -0.558 0.248 0.024 0.572 0.352 0.930Constant 1.446 1.177 0.219 4.245

The overall model is significant at P , 0.05.

25


comparisonswere unable to determine the exact locationsof the differences. Either a larger sample size of differentethnicities or a replication study with planned compari-sons based on the current study would allow us to explorethe fine-grain effect of each ethnicity.

This study was important in identifying undergradu-ate pharmacy students’ achievement goals and the rela-tionship of those goals to academic achievement. Inaddition, this study shed some light on the relationshipbetween different ethnicities and achievement goals. Asquantitative studies do not answer the question why suchphenomena occur, a qualitative investigation of thisphenomenonmay yield useful additional results. In-depthinterviews with a purposive sample of students from thisstudy may yield more information regarding student adop-tion of one achievement goal over another, the relationshipbetween academic achievement and performance-approachgoals in third-year students, and ethnic differences. Further,understanding the qualities that mastery-approach andperformance-approach students would like to see in theirinstructors may help academics create environments thatfoster the adoption of both goals.

Limitations of the study include the small samplesize of some of the ethnic groups. For example, despitethe significant impact of predominant ethnicities uponacademic achievement, the Tukey test failed to determinewhere the significant differences were between each eth-nicity. Another limitation was using cohorts from onlyone university. A study that includes undergraduate phar-macy students from different Australian universities mayyield more generalizable results.

CONCLUSIONAdopting performance-approach goals positively

correlated with academic achievement, while adoptingeither performance-avoidance or mastery-avoidancegoals did not. First-year students were more performance-approach and mastery-approach oriented than third-year students. Ethnicity affected achievement goalsand academic achievement. Chinese Australian studentsindicated stronger preferences for adopting performance-approach goals, whereas Vietnamese Australian stu-dents adopted mastery-avoidance goals more than anyother ethnicities.

REFERENCES1. Murayama K, Elliot AJ, Friedman R. Achievement goals. In: RyanRM, ed. The Oxford Handbook of Human Motivation. New York,NY: Oxford University Press; 2012:191-207.2. Elliot AJ, Fryer JW. The goal concept in psychology. In: ShahGardner W, eds. Handbook of Motivational Science. New York, NY:Guilford Press.; 2008:235-550.

3. Senko C, Hulleman CS, Harackiewicz JM. Achievement goaltheory at the crossroads: old controversies, current challenges, andnew directions. Educ Psychol. 2011;46(1):26-47.4. Meece JL, Anderman EM, Anderman LH. Classroom goal

structure, student motivation, and academic achievement. Annu RevPsychol. 2006;57:487-503.5. Elliot AJ, Dweck CS. Competence and motivation: competence asthe core of achievement motivition. In: Elliot AJ, Dweck CS, eds.Handbook of Competence and Motivation. New York, NY: The

Guiforsd Publications Inc.; 2005:3-12.6. Hulleman CS, Schrager SM, Bodmann SM, Harackiewicz JM. Ameta-analytic review of achievement goal measures: different labelsfor the same constructs or different constructs with similar labels?Psychol Bull. 2010;136(3):422-449.7. Kaplan A, Maehr M. The contributions and prospects oforientation theory. Educ Psychol Rev. 2007;19(2):141-184.8. Harackiewicz JM, Barron KE, Tauer JM, Elliot AJ. Predictingsuccess in college: a longitudinal study of achievement goals and ability

measures as predictors of interest and performance from freshman yearthrough graduation. J Educ Psychol. 2002;94(3):562-575.9. Remedios R, Kiseleva Z, Elliott J. Goal orientations in Russianuniversity students: from mastery to performance? Educ Psychol.2008;28(6):677-691.10. Hulleman CS, Senko C. Up round the bend: forcasts forachievement goal theory and research in 2020. In: Urdan TC,Karabenick SA, eds. The Decade Ahead: Theoritical Perspectives onMotivation and Achievement. Vol 16A. Bingley, UK: Emerald GroupPublishing Limited; 2010:71-104.11. Dweck CS. Motivational processes affecting learning. AmPsychol. 1986;41(10):1040-1048.12. Elliot AJ. Approach and avoidance motivation and achievementgoals. Educ Psychol. 1999;34(3):149-169.13. Elliott JG, McGregor HA. A 2x2 achievement goal framwork.J Pers Soc Psychol. 2001;80(3):501-519.14. Elliot AJ, Thrash TM. Achievement goals and the hierarchicalmodel of achievement motivation. Educ Psychol Rev. 2001;13(2):

139-156.15. Isoard-Gautheur S, Guillet-Descas E, Duda JL. How to achievein elite training centers without burning out? An achievement goaltheory perspective. Psychol Sport Exerc. 2013;14(1):72-83.16. Huang C. Discriminant and criterion-related validity ofachievement goals in predicting academic achievement: a meta-analysis. J Educ Psychol. 2012;104(1):48-73.17. Shim SS, Cho Y, Wang C. Classroom goal structures, socialachievement goals, and adjustment in middle school. Learn Instr.

2013;23:69-77.18. Diseth A. Self-efficacy, goal orientations and learning strategiesas mediators between preceding and subsequent academicachievement. Learn Individ Differ. 2011;21(2):191-195.19. Harackiewicz JM, Barron KE, Pintrich PR, Elliot AJ, Thrash

TM. Revision of achievement goal theory: necessary andilluminating. J Educ Psychol. 2002;94(3):638-645.20. Wolters CA. Advancing achievement goal theory: using goalstructures and goal orientations to predict students’ motivation,cognition, and achievement. J Educ Psychol. 2004;96(2):236-250.21. Roussel P, Elliot AJ, Feltman R. The influence of achievementgoals and social goals on help-seeking from peers in an academiccontext. Learn Instr. 2011;21(3):394-402.22. Elliot AJ, McGregor HA, Gable S. Achievement goals, study

strategies, and exam performance: A mediational analysis. J EducPsychol. 1999;91(3):549-563.

26


23. Cury F, Elliot AJ, Da Fonseca D, Moller AC. The social-cognitive model of achievement motivation and the 2* 2 achievementgoal framework. J Pers Soc Psychol. 2006;90(4):666-679.24. Barron KE, Harackiewicz JM. Revisiting the benefits ofperformance-approach goals in the college classroom: Exploringthe role of goals in advanced college courses. Int J Educ Res.2003;39(4/5):357-374.25. Putwain DW, Symes W. Achievement goals as mediators of therelationship between competence beliefs and test anxiety. Br J EducPsychol. 2012;82(2):207-224.26. Elliot AJ, Pekrun R. Emotion in the hierarchical model ofapproach-avoidance achievement motivation. In: Schutz PA, PekrunR, eds. Emotion in Education. San Diego, CA: Elsevier AcademicPress; 2007:57-73.27. Sideridis GD. The regulation of affect, anxiety, and stressfularousal from adopting mastery-avoidance goal orientations. StressHealth. 2008;24(1):55-69.28. Murayama K, Elliot AJ. The competition-performance relation:a meta-analytic review and test of the opposing processes model ofcompetition and performance. Psychol Bull. 2012;138(6):1035-1070.29. Van Yperen NW, Elliot AJ, Anseel F. The influence of mastery-avoidance goals on performance improvement. Eur J Soc Psychol.2009;39(6):932-943.30. Linnenbrink-Garcia L, Middleton MJ, Ciani KD, Easter MA,O’Keefe PA, Zusho A. The strength of the relation betweenperformance-approach and performance-avoidance goal orientations:theoretical, methodological, and instructional implications. EducPsychol. 2012;47(4):281-301.31. Van Yperen NW. A novel approach to assessing achievementgoals in the context of the 2 x 2 framework: identifying distinctprofiles of individuals with different dominant achievement goals.Pers Soc Psychol Bull. 2006;32(11):1432-1445.32. Lieberman DA, Remedios R. Do undergraduates’ motives forstudying change as they progress through their degrees? Br J EducPsychol. 2007;77(2):379-395.33. Witkow MR, Fuligni AJ. Achievement goals and daily schoolexperiences among adolescents with Asian, Latino, and EuropeanAmerican backgrounds. J Educ Psychol. 2007;99(3):584-596.

34. Elliot AJ, Chirkov VI, Kim Y, Sheldon KM. A cross-culturalanalysis of avoidance (relative to approach) personal goals. PsycholSci. 2001;12(6):505-510.35. Zusho A, Pintrich PR, Cortina KS. Motives, goals, and adaptivepatterns of performance in Asian American and Anglo Americanstudents. Learn Individ Differ. 2005;15(2):141-158.36. Waskiewicz RA. Achievement goal orientation and situationalmotivation for a low-stakes test of content knowledge. Am J PharmEduc. 2012;76(4):Article 65.37. Smith L, Saini B, Krass I, Chen T, Bosnic-Anticevich S,Sainsbury E. Pharmacy students’ approaches to learning in anAustralian university. Am J Pharm Educ. 2007;71(6):Article 120.38. Elliot AJ, Murayama K. On the measurement of achievementgoals: critique, illustration, and application. J Educ Psychol.2008;100(3):613-628.39. Marian V, Kaushanskaya M. Self-construal and emotion inbicultural bilinguals. J MemLang. 2004;51(2):190-201.40. Yamauchi LA, Tharp RG. Culturally compatibleconversations in Native American classrooms. Linguist Educ.1995;7(4):349-367.41. Sydney Uo. University of Sydney HandBooks. In: SydneyTUo, ed. 00026A. Sydney, Australia: The University of Sydney2012.42. Harackiewicz JM, Barron KE, Tauer JM, Carter SM, Elliot AJ.Short-term and long-term consequences of achievement goals:predicting interest and performance over time. J Educ Psychol.2000;92(2):316-330.43. Senko C, Miles KM. Pursuing their own learning agenda: howmastery-oriented students jeopardize their class performance.Contemp Educ Psychol. 2008;33(4):561-583.44. Kao G. Asian-Americans as model minorities? A look at theiracademic performance. Am J Educ. 1995;103(2):121-159.45. Li J. Expectations of Chinese immigrant parents for theirchildren’s education: the interplay of Chinese tradition and theCanadian context. Can J Educ. 2001;26(4):477-494.46. Lee AY, Aaker JL, Gardner WL. The pleasures and pains ofdistinct self-construals: the role of interdependence in regulatoryfocus. J Pers Soc Psychol. 2000;78(6):1122-1134.

27


RESEARCH

Self-Efficacy and Self-Esteem in Third-Year Pharmacy Students

Mark L. Yorra, EdD

School of Pharmacy, Northeastern University, Boston, Massachusetts

Submitted October 2, 2014; accepted February 8, 2014; published September 15, 2014.

Objective. To identify the experiential and demographic factors affecting the self-efficacy and self-esteem of third-year pharmacy (P3) students.Methods. A 25-item survey that included the Rosenberg Self-Esteem Scale and the General Self-Efficacy Scale, as well as types and length of pharmacy practice experiences and demographic in-formation was administered to doctor of pharmacy (PharmD) students from 5 schools of pharmacy inNew England at the completion of their P3 year.Results. The survey response rate was approximately 50% of the total target population (399/820).Students with a grade point average (GPA)$3.0 demonstrated a higher significant effect from unpaidintroductory pharmacy practice experiences (IPPEs) on their self-efficacy scores ( p,0.05) comparedto students with lower GPAs. Students who had completed more than the required amount of pharmacyexperiences had higher levels of self-efficacy and self-esteem (p,0.05). Ethnicity also was related tostudents’ levels of self-efficacy and self-esteem.Conclusion. Self-efficacy and self-esteem are two important factors in pharmacy practice. Collegesand schools of pharmacy should ensure that students complete enough practice experiences, beyond theminimum of 300 IPPE hours, as one way to improve their self-efficacy and self-esteem.

Keywords: self-efficacy, self-esteem, experiential learning, introductory pharmacy practice experience

INTRODUCTIONThe personal transformation that pharmacy students

undergo as they complete their academic career and entertheir professional lives is arguably one of the most signif-icant and difficult transitions they will experience. Ide-ally, upon entering practice, a graduate’s self-efficacy andself-esteem should be high, allowing the new pharmacistto perform with great confidence at a high level of com-petence.1 Self-efficacy is “an individual’s belief in theirability to perform well in a variety of situations.”2 Peoplewith a high level of self-efficacy approach difficult tasksas a challenge to be mastered rather than a threat to beavoided.3 Self-esteem is “a certain attitude and a percep-tion of one’s self.”4 Although self-esteem is an internalperception of one’s self, it can be affected by externalcomparison to peers or role models.

Oneway for pharmacy students to improve their self-efficacy and self-esteem prior to graduation is to gain ex-perience by working in pharmacy settings. Introductoryand advanced pharmacy practice experiences (IPPEs andAPPEs) are where important learning about the profession

of pharmacy occurs outside of the classroom.5 Experien-tial learning through IPPEs and APPEs, both unpaid andpaid, provide an opportunity for students to receive formalappraisals as well as informal feedback from coworkers,preceptors, and others they interact with during their expe-riences. The Accreditation Council for Pharmacy Educa-tion (ACPE) requires (1) 300 IPPE hours, of which 20%or 60 hours can be earned through completing simulatedexperiences; and (2) 36 weeks at 40 hours a week ofAPPEs, as the minimum to be eligible to take the NorthAmerican Pharmacist Licensure Examination.6,7 Never-theless, this amount may still not be sufficient and stu-dents without additional pharmacy work experienceoutside of the required school-provided experiencesmay be unprepared to work independently as pharmacistsat graduation. Such graduates may require several weeksof additional training on the job in order to function intheir new role as a pharmacist (J. Gallagher, personalcommunication, January 25, 2012; C. Perry, personalcommunication, January 25, 2012)

A literature review did not yield any pharmacy spe-cific articles on self-efficacy and self-esteem in relation tostudents’ early professional experiences. Several articlesaddressed the benefit of a cooperative education modelwhere students received extended practical experienceson their self-efficacy and self-esteem.8-11 A few other

Corresponding Author: Mark L. Yorra, EdD, School ofPharmacy, Northeastern University, 360 Huntington Avenue,140 The Fenway, Mailstop 218TF, Boston, MA 02115. Tel:617-373-3433. Fax: 617-373-7504. E-mail: [email protected]

28


studies found a relationship between students’ ethnicityand level of self-efficacy.12-14 This study was conductedto determine whether students who had spent extendedtime in paid and/or unpaid IPPEs, through summer jobs,part-time jobs, or other work opportunities, had greaterself-efficacy and self-esteem than those who spent lesstime in these work settings. The impact of several demo-graphic factors also was examined.

METHODSThe author developed the Pharmacy Self-efficacy

and Self-esteem Study Questionnaire using the GeneralSelf-efficacy Scale and Rosenberg Self-esteem Scale.The General Self-efficacy Scale was selected because itmeasures the most desirable psychometric properties ina wide variety of settings.2 The Rosenberg Self-esteemScale is a 10-item scale and has been used to measuregeneral self-esteem in various populations.15-17

In addition to the self-efficacy and self-esteemquestions, the survey collected the following demo-graphic information: date of birth, ethnicity, schoolattended, gender, grade point average (GPA), hours ofIPPEs completed, and years of paid pharmacy work ex-perience. One of the research objectives was to explorehow selected demographic variables related to students’self-efficacy and self-esteem in order to confirm or dis-prove previously reported findings associating students’gender, ethnicity, GPA, and/or age with their self-efficacyand self-esteem.

The author selected 5 colleges and schools in theNew England area as a convenience sample, which pro-vided a mix of educational models, including a 3-yearaccelerated program, two 6-year private universities,and two 4-year state universities. Students who had com-pleted their third year of pharmacy schoolwere selected toparticipate in the study. Three institutions opted to usea paper survey instrument, whichwas distributed by a fac-ulty member during a class and then collected andreturned to the author for processing. Two institutionsrequested the electronic version of the survey instrument,a link towhichwas e-mailed to P3 students by their dean’soffice. The author sent a reminder e-mail to the students14 days after the initial request and closed access to theelectronic survey instrument after 60 days.

Students’ IPPEs were categorized based on numberof hours completed. Paid experiences were categorizedbased on timeworked as none,,1 year, 1-2 years, and.2years of experience. Paid experience included both part-time and full-time experience. In order to keep the unit oftime consistent, the time in years was converted to ap-proximate hours based upon the school a student attendedto determine their availability to work in a paid position.

Students froma 0-6PharmDprogrammayhaveworked ina paid position from 400-500 hours (10-12 weeks at40 hrs/wk) during 2 summer periods and another 200hours (8 hrs/wk for 25 weeks) during the school year.Students from the 2-4 PharmD programs had 2 summerperiods to obtain a full-time experience. Only a few couldhave worked in a paid part-time position on an occasionalbasis because of the rural location of the schools. Studentsfrom the accelerated program had the lowest opportunityto work part-time because they did not have any summerbreaks. The cooperative education experience was ap-proximately 600 hours during one 4-month work period,with each student required to complete 2 work periodswith a curricular requirement of 1 work period of 600hours in a community setting and 1 work period of 600hours in an institutional setting. These students wouldhave more than 1200 hours from the 2 work periods.The majority of the positions were paid. Approval for thisproject was granted by the Institutional Review Board atNortheastern University, Boston, MA.

The author examined the student responses to theself-efficacy and self-esteem questions, and tested forsignificant effects from the independent variables, partic-ularly paid and unpaid pharmacy experiences. For thepurposes of analyses, students were divided into 2 groupsby age, under 25 years and 25 years and over, and 2 groupsby GPA,,3.0 and$ 3.0. Analysis of Variance was usedto test the hypothesis using SPSS v 21 (IBM). Ethnicitywas examined using chi-square testing to determine dif-ferences within the variable.

RESULTSThree hundred ninety-nine students completed the sur-

vey instrument, approximately a 50% response rate basedon the total estimated student population of 820 students atthe end of the P3 year. The reliability of the General Self-efficacyScalewasa50.907 and theRosenbergSelf-esteemScale showed a reliability factora50.888. Students’ gendermirrored that of the national pharmacy student population,18

ie, 35% male and 65% female. The ethnic distribution was4%African American/black, 29.4%Asian/Pacific Islanders,2.2% Spanish/Hispanic, and 59% white/Caucasian, and4.7% indicated other or no response. Seventy-three percentof students were 25 years of age or under and 27%were over25 years of age. A correlation was performed on the demo-graphic variables in Table 1. Students’ self-esteem scoreswere significantly correlated with number of IPPE hoursand number of paid practice experiences.

Self-EfficacyAmong students with a GPA,3.0, there were no fac-

tors associated with higher self-efficacy. Among students

29


with a GPA$3.0, number of hours of IPPEs was signifi-cantly associated with higher self-efficacy (p,0.05).

There was a significant correlation between meanscores on the General Self-efficacy Scale items anda higher number of hours spent in paid or extended phar-macy practice experiences (r50.114, p,0.05). This cor-relation did not exist for hours spent in unpaid experiencesor IPPE experiences. Ethnicity was examined using a chi-square test with a significant finding (p,0.001) for theAsian/Pacific Islander group. Chi-square testing was alsoperformed with the variables of age, gender, and theschool a student attended, but none of the findings weresignificant.

Self-EsteemThere was a significant correlation between which

school a student attended and the student’s level of self-esteem, with students attending a 4-year state school inRhode Island having greater self-esteem (r5-0.101,p,0.05). There was also a significant correlation betweennumber of hours of IPPEs a student completed and levelof self-esteem (r5-0.132, p#0.05) with students com-pleting 320 hours having the highest self-esteem. Finally,there was a correlation between students’ hours of paidexperiences and self-esteem (r50.135, p#0.05) with stu-dents whoworked for 1 to 2 years in paid positions havingthe highest levels of self-esteem. No significant correla-tions were found between gender or age and students’level of self-esteem. Ethnicity was examined using

chi-square testing (Table 2), resulting in a finding ofhigher self-esteem among African American/black stu-dents (p5.003) and lower self-esteem among Asian/Pa-cific Islander students (p,.001).

DISCUSSIONThis research examined several variables to deter-

mine if any had an effect on the self-efficacy and self-esteem of pharmacy students. The finding that studentswith a GPA.3.0 benefited more from extended work ex-periences than students with a GPA,3.0 can be explainedby the difference between academic self-efficacy andwork-based self-efficacy. Students who do well academ-ically may not have as much work or practical experiencebecause they are focused on academics and they maybenefit more in terms of self-efficacy from practical ex-perience than the average student. Gender showed norelationship to self-efficacy, although the literature did

Table 1. Correlation Between Demographic Variables and Experiences

Gender Ethnicity IPPEPaid

Experiences YOB ,25Self-Esteem

Score GSE Score

GenderPearson Correlation 1 0.007 ?0.014 0.147b ?0.037 ?0.032 ?0.025N 399 362 360 387 388 391

EthnicityPearson Correlation 1 0.055 0.164b 0.125a 0.017 0.043N 360 358 385 386 389

IPPEPearson Correlation 1 0.089 0.074 ?0.132a ?0.062N 345 352 353 354

Paid ExperiencesPearson Correlation 1 0.168b 0.135a 0.114a

N 350 350 352YOB by 25Pearson Correlation 1 ?0.016 ?0.032N 374 378

Self Esteem ScorePearson Correlation 1 0.656b

N 380a 2 tailed Correlation significant p,0.05 levelb 2 tailed Correlation significant p,0.01 level.

Table 2. Self-Esteem Scores by Ethnicity

Ethnicity No. Mean (SD)

African-American/Black 16 9.1 (1.3)Spanish, Hispanic, Latino 8 9.0 (1.4)Asian/Pacific Islander 114 8.3 (1.6)White/Caucasian 230 9.0 (1.1)Other 10 8.1 (2.1)Prefer not to state 7 7.6 (1.7)Total 385 8.8 (1.3)

30


state that female students had lower self-efficacy thanmale students, possibly as a result of their upbringing.Because much of the literature was published prior to2000, continued changes in gender roles in the last fewdecades may have influenced the findings, resulting inno differences in self-efficacy between genders.19 Agealso was not a factor in self-efficacy as students wereexperiencing the profession at the same time regardlessof age.

Levels of self-efficacy and self-esteem were relatedto ethnicity. Asian/Pacific Islander students reported hav-ing lower self-efficacy and self-esteem than the otherethnic groups, while African-American students reportedhaving higher self-esteem than the other ethnic groups. Ina study of college students, African-American studentsreported the highest levels of self-esteem compared withtheir peer groups, which can be partially attributed to theirinternalized self-images in the later stages of their educa-tional careers.20 African-American students who achieveda college education had higher levels of self-esteem com-pared to their peers who did not go to college. There isample evidence that as an ethnic group, Asians are modestwhen responding to surveys assessing self-efficacy.21 Thisstems from a cultural tendency to be more moderate intheir private beliefs, values, and preferences than typicalWestern students.21 The question the author now faces iswhether the Asian students in the current study in fact hadlow levels of self-efficacy compared to other ethnicgroups, or whether their responses understated their ac-tual level of self-efficacy?Choi suggests that responses byAsians on self-assessments should be taken as genuineand not adjusted for any perceived modesty bias.21 Al-though the findings may not indicate a need for Asianstudents to improve their self-efficacy, they may indi-cate the need for Asian students in the health professionsto better project their self-efficacy, as high levels of self-efficacy and self-esteem are essential to earn the respectand confidence of future employers, colleagues, andpatients.1,22

The reason why students from some colleges andschools had better self-esteem could be explained by theattitude promoted by the college or school’s facultymem-bers and administration. Also, because the response ratesfrom 2 institutions were low, the results may not accu-rately represent the self-esteem of all students at theschool.

Self-esteem fluctuates over time and in various set-tings.23 Graduates may enter the work setting with a highlevel of self-esteem based on their success as a student inthe academic setting, but this may decrease as they beginto compare themselves to other practitioners.24 As stu-dents gain more experience, their self-esteem increases

based upon their knowledge and acquired skills. Studentsmay not have enough contact time during an IPPE to in-crease self-esteem substantially without additional con-tact hours through paid or unpaid experiences.

There were 2 main limitations in this study. The firstwas the use of the General Self-efficacy Scale and Rosen-berg’s Self-esteem Scale. Both scales are valid and reli-able, but they need to include more specific questionspertaining to work-based self-efficacy. Joseph Raelin,a researcher on experiential education, suggested theuse of academic, career, and work self-efficacy tools toexamine a student population (personal communicationApril 20, 2012). The use of a more specific scale mighthave yielded different findings and probably would havestrengthened the results of the study.

The second limitationwas the use of a regional sam-ple of schools from the Northeast United States, whichmay not have been representative of a national sample,limiting the ability to generalize the findings. The in-dividual response rates from 2 of the schools were lessthan 25, which made any correlations from these schoolsunreliable.

Other limitations were that the responses to the sur-vey were self-reported by the students and not corrobo-rated by external validation. Nonresponders, particularlystudents attending the public universities, may have af-fected the results by the lack of full representation. Therewas not a survey item about students’ socioeconomic sta-tus, which may have potentially affected the results basedon the type of students who attend a public university vsa private institution. Students who had a greater financialneed may have worked more to earn money for schoolthan did more affluent students. Finally, students whowere higher achievers may have been more likely to com-plete the survey instrument, thus introducing a selectionbias to the survey. The researcher did not test for responsebias.

CONCLUSIONThe objective of this study was to determine if there

is a relationship between experiential education/work ex-perience and self-efficacy and self-esteem, based uponthe desirability to have new graduates entering the work-force possess these characteristics. Though schools pro-vide 300 hours of practical experience through IPPEs, theauthor found that additional hours would enable studentsto achieve higher levels of self-efficacy and self-esteem.Whether IPPEs are expanded or a new requirement isimplemented for students to obtain experiences outsideof the IPPE program, the additional experience wouldprovide important benefits to a student’s developmentof self-efficacy and self-esteem.

31


REFERENCES1. Bandura A. Self-efficacy: the exercise of control. New York: W.H.Freeman; 1997.2. Scherbaum CA, Cohen-Charash Y, Kern MJ. Measuring GeneralSelf-Efficacy. Educational and Psychological Measurement2006;66:1047-63.3. Bandura A. Regulation of cognitive processes through perceivedself-efficacy. Developmental Psychology 1989;25:729-35.4. Mruk CJ. Self-esteem research, theory, and practice: towarda positive psychology of self-esteem. New York: Springer; 2006.5. DiPiro JT. Making the most of pharmacy school. Am J of PharmEduc 2008;72(1):Article15.6. American Association of Colleges of Pharmacy. AccreditationStandards and Guidelines for the Professional Program in PharmacyLeading to the Doctor of Pharmacy Degree. Washington D.C.;2011.7. American Association of Colleges of Pharmacy. AccreditationStandards and Guidelines for the Professional Program in PharmacyLeading to the Doctor of Pharmacy Degree 2011:92.8. Abrahamsson K. Co-operative education, experiential learningand personal knowledge. In: International Conference on HigherEducation; 1981; Lancaster, England; 1981.9. Billett S. Learning Through Practice: Models, Traditions,Orientations and Approaches. Professional and Practice-BasedLearning 2010;1.10. Blair BF, Millea M, Hammer J. The Impact of CooperativeEducation on Academic Performance and Compensation ofEngineering Majors. J of Engineering Educ 2004;93:333-8.11. Yin A. Learning on the job: Cooperative education, internshipsand engineering problem-solving skills. Philadelphia: ThePennsylvania State University; 2009.12. Concannon JP, Barrow LH. A Cross-Sectional Study ofEngineering Students’ Self-Efficacy by Gender, Ethnicity, Year, andTransfer Status. Journal of Science Education & Technology2009;18:163-72.

13. Jaret C, Reitzes DC. Currents in a Stream: College StudentIdentities and Ethnic Identities and Their Relationship with Self-Esteem, Efficacy, and Grade Point Average in an Urban University.Social Science Quarterly 2009;90:345-67.14. Vuong M, Brown-Welty S, Tracz S. The Effects of Self-Efficacyon Academic Success of First-Generation College SophomoreStudents. Journal of College Student Development 2010;51:50-64.15. Rosenberg M. Society and the adolescent self-image: Princeton,N.J., Princeton University Press; 1965.16. Classen S, Velozo CA, Mann WC. Rosenberg Self-Esteem Scaleas a measure of self-esteem for the noninstitutionalized elderly.Clinical Gerontologist 2007;31:77-93.17. Dahlbeck DT, Lightsey OR, Jr. Generalized self-efficacy,coping, and self-esteem as predictors of psychological adjustmentamong children with disabilities or chronic illnesses. Children’sHealth Care 2008;37:293-315.18. American Association of Colleges of Pharmacy. AcademicPharmacy’s Vital Statisitcs. Washington DC; 2011.19. Bandura A, Barbaranelli C. Multifaceted impact of self-efficacybeliefs on academic functioning. Child Development 1996;67:1206-22.20. Elion AA, Wang KT, Slaney RB, French BH. Perfectionism inAfrican American students: Relationship to racial identity, GPA, self-esteem, and depression. Cultural Diversity and Ethnic MinorityPsychology 2012.21. Choi I, Choi Y. Culture and Self-Concept Flexibility. Personalityand Social Psychology Bulletin 2002;28:1508-17.22. Freudenberg B, Cameron C, Brimble M. The Importance of Self:Developing Students’ Self Efficacy Through Work IntegratedLearning. International Journal of Learning 2010;17:479-96.23. Ferris DL, Lian H, Brown DJ, Pang FXJ, Keeping LM. Self-esteem and job performance: the moderating role of self-esteemcontingencies. Personnel Psychology 2010;63:561-93.24. Crocker J, Brook AT, Niiya Y, Villacorta M. The Pursuit of Self-Esteem: Contingencies of Self-Worth and Self-Regulation. Journal ofPersonality 2006;74:1749-71.

32


RESEARCH

Investigating the Relationship Between Pharmacy Students’Achievement Goal Orientations and Preferred Teacher Qualities

Saleh Alrakaf, MSc Pharm,a Erica Sainsbury, PhD,a Grenville Rose, PhD,b andLorraine Smith, PhDa

aFaculty of Pharmacy, The University of Sydney, Sydney, New South Wales, AustraliabInnovation and Evaluation, Aftercare, Sydney, New South Wales, Australia

Submitted December 15, 2013; accepted March 17, 2014; published September 15, 2014.

Objective. To investigate the relationships between pharmacy students’ preferred teacher qualities andtheir academic achievement goal orientations.Methods. Participants completed an achievement goal questionnaire and a build-a-teacher task. For thelatter, students were given a $20 hypothetical budget to purchase amounts of 9 widely valued teachers’qualities.Results. Three hundred sixty-six students participated. Students spent most of their budget on the traits ofenthusiasm, expertise, and clear presentation style, and the least amount of money on interactive teach-ing, reasonable workload, warm personality, and intellectually challenging. In relation to achievementgoals, negative associations were found between avoidance goals and preferences for teachers whoencourage rigorous thinking and self-direction.Conclusion. These novel findings provide a richer profile of the ways students respond to their learningenvironment. Understanding the relationships between teachers’ characteristics and pharmacy students’achievement goal orientations will contribute to improving the quality of pharmacy learning and teachingenvironments.

Keywords: achievement goals, motivation, pharmacy education, teacher qualities, student preferences

INTRODUCTIONMost faculties (schools of pharmacy) seek their stu-

dents’ opinions regarding teaching and instructor quali-ties, and it is not uncommon to find that students rate thesame instructor differently. However, it is unclear whydifferent students rate an instructor differently. The goalsthat university students adopt in class may be the answerto this question,1 specifically achievement goals, whichtheorists believe play a major role in education.2,3

According to achievement goal theorists, studentsengage in educational activities with 1 of 2 broad goalsinmind:mastery goals or performance goals.4-6 For eithergoal, gaining competence is the student’s primary aim.7

However, they perceive competence in different ways.Mastery-oriented students view competence as learning andunderstanding the task thoroughly and use self-referentialstandards to define success vs failure.8-10 On the otherhand, performance-oriented students view competence

as performing well compared to other students and theydefine success vs failure based on teacher-referentialstandards.8,11

Elliot andMcGregor have proposed that mastery andperformance goals can be further divided into approach andavoidance components.12 Students who adopt a mastery-approach goal aim to learn and understand the coursematerials as thoroughly as possible, whereas those whoare oriented towards the mastery-avoidance goal aim toavoid not understanding the course materials thoroughly.Students adopting the performance-approach goal aremotivated to outperform other students or to demonstratetheir ability to either teachers or peers, whereas studentsadopting the performance-avoidance goal aim to avoid do-ing worse than other students or appearing less talented.This distinction is supported by a large body of empiricalresearch and is robust in predicting and understanding stu-dents’ engagement and achievement.13-15

These goal orientations are differentially associatedwith a range of motivation, academic, and psychologicalcorrelates. Avoidance goals are associated with negativeoutcomes, for example, performance-avoidanceandmastery-avoidance goals have been linked to depression16 and low

Corresponding Author: Saleh Alrakaf, Faculty of Pharmacy,Room S114, Pharmacy and Bank Building A15, TheUniversity of Sydney, NSW, 2006. Tel: 161-2-9351-4501.Fax: 161-2-9351-4451. E-mail: [email protected]

33


grades on examinations.17,18 In contrast, the mastery-approach goal has been associated with deep learning,19

high individual interest,20 high self-regulation,21 andwillingness to cooperate.22 Yet, to the achievement theo-rists’ surprise, the mastery-approach goal can rarely pre-dict high academic achievement (ie, grades).13 Theperformance-approach goal, however, is associated withhigh grades on examinations,24,25 but also with “surface”learning approaches such as memorization.23

To find an explanation for the unexpected relationshipbetween the “approach” types of achievement goals andacademic achievement, Senko and his colleagues hypoth-esized that each type of goal affects students’ learningstrategies differently.11 According to the authors, studentswho adopt the mastery-approach goal tend to study mate-rials that are interesting to them regardless of the subjectmatter’s importance or testability, while students who areperformance-approach oriented do not. The latter willstudy what they think will appear on the examination andtry to figure out what is important to their teachers insteadof following their own interests. As a result, they gainhigher grades than their mastery-approach peers.26

The quality of higher education largely depends onthe qualities of teachers in this sector.27 Determiningwhich qualities are considered essential and effectivecan be difficult to define as stakeholders in higher educa-tion (eg, students, teachers, administrators, and scholars)have individual views and opinions aboutwhat these qual-ities are.28 However, they all believe that teacher qualitieshave a great impact not only on students’ education butalso on students’ futures as well.29 One area that teacherqualities play a major role in is students’ achievementgoals.30,31A recent study conducted byShim and colleaguesfound that teachers who strongly pursue mastery goals tendto foster the adoption of mastery goals by their students,while teachers who strongly pursue performance goals tendto foster the adoption of those goals by their students.32

Although such impacts are well documented, little is knownabouthowstudents’ achievementgoalsmight influence theirpreferences for teachers’ qualities.33

Senko and colleagues found that mastery-approachand performance-approach goals adopted by studentsdid affect the qualities and traits that students wanted tosee in their teachers.33 Students who adopted a mastery-approach goal most valued teachers who challenged themintellectually and had an extensive experience in their sub-ject areas. In contrast, studentswho adopted a performance-approach goal valued teachers who provided suggestionsabout how to gain high marks on examinations and whopresented their material clearly. Valuing these qualitiesdid not suggest that these students did not like other qual-ities such as warmth and enthusiasm. It simply meant that

students considered the latter qualities less important andnot necessities. These so-called “luxury” qualities weredesirable only after obtaining the essential ones. WhileSenko and colleagues tested the effect of bothmastery andperformance approach types, we believe that investigat-ing the impact of mastery and performance avoidancetypes and their relationship to teacher qualities is alsoimportant. Given that avoidance types are maladaptiveand unproductive, knowing the preferred teacher qualitiesof students who strongly adopt them is beneficial in orderto review teaching methods that might foster adoption ofthese goals.

Thus, our study had 3 aims: to investigate whichqualities pharmacy students most preferred in theirteachers; to test assumptions about how mastery-approach and performance-approach goals affect students’preferences for various teacher qualities in a pharmacy ed-ucation setting;33 and to investigate the effects of avoidancetypes of achievement goals (mastery avoidance and perfor-mance avoidance) on teacher qualities. To our knowledge,no study has assessed the effects of the 4 types of achieve-ment goals on students’ preferences of teacher qualities.

METHODSThe participants for this study were second-year and

fourth-year undergraduate pharmacy students enrolled ina bachelor’s degree program at TheUniversity of Sydney’sFaculty of Pharmacy. Completion of this 4-year programenables graduates to register as a pharmacist inAustralia.34

The survey comprised 2 measures: the AchievementGoalQuestionnaire12 (AGQ)and thebuild-a-teacher task.33

Both instruments are available from the corresponding au-thor. In addition to these measures, gender and age wereincluded as socio-demographic indicators in the survey.

The AGQ is a validated and psychometrically robustinstrument35 intended to measure the 4 types of studentachievement goals and contained 12 items. Students ratedeach item on a 1 to 7 scale (15not at all true of me, 75verytrue of me).

The build-a-teacher task is a validated and com-monly used instrument for measuring teacher qualities.33

It contains a list of 9 widely valued teacher qualities. Thetask required students to design their ideal teacher by“buying” qualities with a limited budget. The purchasingscale ranged from $0 to $10. This method encouragedstudents to carefully consider their choices as the morethey spent on one quality the less money was left to spendon other qualities.33,36

The study was initiated in the first semester of theacademic year. Students were invited to participate in thestudy during normal lectures or tutorials (small groupdiscussions). They were advised that participation was

34


voluntary, and if they chose to participate, they couldwithdraw from the study at any time. In addition, studentswere advised that their decision to participate would notimpact on their academic performance results or influencestudent-teacher relationships. Researchers approached stu-dents as a group and not individually. The first authoradministered the survey instrument.

Students completed the survey in paper form. For thebuild-a-teacher task, students were given a hypothetical$20 budget to purchase the 9 teachers’ qualities. The writ-ten instructions explained that the maximum amount ofmoney students could spend on any 1 quality was $10.Students were asked to spend their full budget in a waythat reflected their preferences. Completing the task tookapproximately 20 minutes.

SPSS 21 (SPSS Inc, Chicago, Illinois) was used forall statistical analyses. Descriptive statistics regardinggender and age were reported. A split-plot ANOVA de-sign (SPANOVA), with academic year as the between-subjects factor and teacher qualities as thewithin-subjectsfactor, was used to investigate the impact of academicyear upon student preferences for teacher qualities andto compare student responses to the 9 different teacherqualities.37 If the sphericity assumption was violated, theHuynh-Feldt degrees of freedomwere reported. Bonferronicorrection was performed as needed. A multiple regres-sion analysis procedurewas performed to assess the effectof each achievement goal type on student spending onteacher qualities.

Conduct of this study was approved by the HumanResearch Ethics Committee, The University of Sydney.

RESULTSThree hundred sixty-six students (235 female, 128

male, and 3 who preferred not to reveal their gender)participated in this study. The mean age of the studentswas 21.3 years (standard deviation52.7 years). The sur-vey yielded a response rate of 73.2%.

The Mauchly test indicated that the assumption ofsphericity had been violated (p,0.05), therefore, de-grees of freedom were corrected using Huynh-Feldt es-timates of sphericity. A SPANOVA test revealed nosignificant impact of academic year on student preferencesfor teacher qualities (p50.66). However, there were sig-nificant differences between teacher qualities that studentsprefer in that the test showed students prioritized somequalities over others (p,0.01). Students’ most preferredquality was enthusiasm/entertaining (mean6SD,3.162.2), followed closely by topic expertise, clear pre-sentation style, and clarity about how to succeed. Theyconsidered reasonable workload (mean6SD, 1.661.6)and interactive teaching style (mean6SD, 1.661.5)

the least essential (Table 1). The effect comparing the2 academic years was not significant (p50.23), sug-gesting no difference between the 2 academic years.Bonferroni pairwise comparisons were performed andthe variables were placed in groups where there were nosignificant differences. No significant differences werefound among the qualities of enthusiasm, topic exper-tise and clear presentation style. However, these qual-ities did significantly differ in mean scores from otherteacher qualities such as good feedback, intellectuallychallenging, warm/compassionate personality, reason-able workload, and interactive teaching style.

Multiple regressionwasperformed to assess the impactof the different types of achievement goals on the 9 teacherqualities. The model contained 4 independent variables(performance-approach, mastery-approach, performance-avoidance and mastery-avoidance goals). The relationshipsbetween students’ achievement goals and the teacherqualities they preferred were determined by any signifi-cant relationship between a goal and the money spent ona teacher quality.

As shown in Table 2, the more students pursuedmastery-avoidance goals, the less they spent on theteacher quality of enthusiasm (p50.03). Furthermore,the more students pursued performance-avoidance goals,the less they wanted their teacher to challenge them in-tellectually (p50.01). In addition, the more students pur-sued performance-approach goals, the less they spent onthe quality of warm/compassionate personality (p50.01).

DISCUSSIONThis study tried to answer 3 important questions:

What teacher qualities do students most prefer? Towhat extent do mastery- and performance-approachgoals influence student preferences for teacher quali-ties? To what extent do mastery- and performance-

Table 1. Pharmacy Students’ Ratings of Desirable TeacherQualities

Teachers’ Qualities Mean (SD)

Enthusiastic/entertaining 3.1a (2.2)Topic expertise 3.1a (2.0)Clear presentation style 2.8a,b (2.2)Clear about how to succeed 2.4b (2.0)Good feedback 1.9c (1.7)Intellectually challenging 1.8c (1.7)Warm/compassionate personality 1.7c (1.7)Reasonable workload 1.6c (1.6)Interactive teaching style 1.6c (1.5)

Note: Qualities that do not share the same superscript are significantlydifferent using Bonferroni correction at the 0.05 level.

35


avoidance goals influence student preferences for teacherqualities? To answer all of these questions precisely,we used a budget methodology specifically designedto differentiate between essential and nonessentialteacher qualities33 and a validated measure of achieve-ment motivation.35

That the enthusiasm quality emerged as one of themost preferred teacher qualities was not a surprise to us. Aqualitative study conducted by Alrakaf and colleagues toinvestigate undergraduate pharmacy students’ prefer-ences for teaching indicated, without prompting, that stu-dents highly value this quality.38 Interestingly, the bottomranked quality was interactive teaching style, which isviewed by many scholars as highly valued by studentsand beneficial in terms of academic achievements.39-41

A closer look at the teacher qualities students pre-ferred revealed that on the whole, the highly valued qual-ities were those that reflected teacher engagement with thelearning process where the emphasis was on the level ofteacher commitment to the task of optimizing studentlearning and achievement. The least-valued qualities, onthe other hand, were those that reflected student engage-mentwith the learning process, where the emphasis was onstudent commitment to optimizing their own learning andachievement. Take for example, the low ranking for theteacher quality “intellectually challenging.” This qualityrequires student commitment to learning and an ability toperform self-directed learning tasks. These results weresupported by the findings of our previous work regardingour pharmacy students’ approaches to learning, in whichstudents demonstrated being dependent upon and valuingexternal sources of support and found self-directed learn-ing approaches challenging.Our previous research showed

both cross-sectionally and longitudinally that pharmacystudents preferred to learn through dependence onteacher-sourced strategies rather than self-sourced strate-gies, and that deep processing andcritical thinkingwere notroutinely favored by students.34,42

The low ranking that the quality of having an inter-active teaching style received may have resulted from theintroduction of the online-recorded lecture system, whichenables academics to record lectures and make themavailable to students electronically. Although all otherpharmacy classes (workshops, tutorials, and laboratories)are face to face, no attendance is required at recordedlectures. Thus, students may have felt that havinga teacher with an interactive teaching style was not asessential as in the past. The ability to use Internet sitessuch as YouTube as a source for information may alsoexplain why students considered interactive teachingstyle the least essential teacher quality. The use of Internettechnology is a defining feature of this generation of stu-dents, because they are the first generation to have had theInternet as a part of their lives from birth.43,44

Our findings in relation to our first aim supportedthose that Senko and colleagues found in their study,33

yet regarding the second aim, our results were quite dif-ferent. In contrast to Senko and colleagues’ results, theonly significant relationship we found was a negative onebetween performance-approach goals and buying theteacher quality of a warm personality. Students whomorestrongly pursued performance-approach goals were lesslikely to prefer a warm and compassionate teacher. Thisresult might be attributed to the competitive nature ofperformance-approach-oriented students who tend to af-firm their competence by outperforming their peers. Ev-idence suggests that warm and compassionate teachersmay be willing to take into account the circumstancesof struggling students and give preferential treatmentwithrespect to grades.45

Our study expanded upon previous research by exam-ining the impact of mastery- and performance-avoidancegoals,33 showing that they had significant negative rela-tionships with the enthusiasm and intellectually challeng-ing teacher qualities, respectively. This indicated that themore strongly students adoptedmastery- and performance-avoidance goals, the less necessary it was that theirteachers be enthusiastic or challenge them intellectually.These findings could be attributed to the specific motiva-tional characteristics of students who adopted the avoid-ance types of goals. Fear of facing shame, beingembarrassed, and/or being criticized by teachers havebeen highly linked to students who pursue these goals.46

The aim of students who adopt the mastery-avoidancegoal is to avoid not understanding the course materials

Table 2. Regression Analyses (p-value) of Goal Type withTeacher Quality

Teachers’ Qualities M-APa M-AVb P-APc P-AVd

Enthusiastic/entertaining 0.16 0.03 0.14 0.06Intellectually challenging 0.46 0.28 0.76 0.01Topic expertise 0.06 0.96 0.27 0.53Clear about how to succeed 0.76 0.76 0.09 0.06Clear presentation style 0.07 0.39 0.96 0.63Reasonable workload 0.19 0.05 0.76 0.08Interactive teaching style 0.37 0.18 0.83 0.20Warm/compassionatepersonality

0.73 0.36 0.01 0.98

Good feedback 0.21 0.71 0.18 0.53

The overall model is significant at p,0.05 according to ANOVAresults.a M-AP: Mastery-approach goalb M-AV: Mastery-avoidance goalc P-AP: Performance-approach goald P-AV: Performance-avoidance goal

36


thoroughly, so a teacher who uses humor and anecdotesmight be seen as a distraction from this effort. Also, anintellectually challenging teacher may inadvertently cre-ate an intimidating environment for students who pursuea performance-avoidance goal as these students tend to beafraid of being criticized and appearing untalented infront of the teacher and their peers. Furthermore, studentswho adopt either type of avoidance goals perceive chal-lenging activities as a threat to their self-esteem.47

Using a pharmacy cohort from only one institution isa limitation for this study. However, the faculty of phar-macy where the study was conducted is the only school inSydney that offers a bachelor’s degree in Pharmacy. Inorder to generalize these results, a national study of Aus-tralian pharmacy students would be a good next step, aswell as a multinational study on pharmacy students. Thestrengths of the study are that we used 2 validated measur-ing instruments and a unique and engaging method of de-termining student preferences for teacher qualities.

CONCLUSIONPharmacy students value a range of teacher qualities

that are stimulating and promote achievement rather thandeep thinking. Students’ engagement with learning ischaracterized by a preference for teacher-focused strate-gies rather than self-focused strategies. In keeping withthis approach to learning, students who adopt avoidance-type achievement goals value least of all those teacherqualities that promote self-directed learning. These find-ings highlight the nexus between teaching and learningand can be used in the development of learning, teaching,and assessment strategies that optimize topic mastery,critical thinking, and academic achievement.

REFERENCES1. Murphy KR, Cleveland JN, Skattebo AL, Kinney TB. Raters who

pursue different goals give different ratings. J Appl Psychol. 2004;

89(1):158-164.2. Hulleman CS, Senko C. Up round the bend: forcasts for

achievement goal theory and research in 2020. In: Urdan TC,

Karabenick SA, eds. The Decade Ahead: Theoritical Perspectives on

Motivation and Achievement. Vol 16A. Bingley, UK: Emerald Group

Publishing Limited; 2010:71-104.3. Wolters CA. Advancing achievement goal theory: using goal

structures and goal orientations to predict students’ motivation,

cognition, and achievement. J Educ Psychol. 2004;96(2):236-250.4. Ames C. Classrooms: goals, structures, and students motivation.

J Educ Psychol. 1992;84(3):261-271.5. Pintrich PR. An achievement goal theory perspective on issues in

motivation terminology, theory, and research. Contemp Educ

Psychol. 2000;25(1):92-104.6. Nicholls JG. Achievement motivation: Conceptions of ability,

subjective experience, task choice, and performance. Psychol Rev.

1984;91(3):328-346.

7. Elliot AJ, Dweck CS. Competence and motivation: competence as

the core of achievement motivition. In: Elliot AJ, Dweck CS, eds.

Handbook of Competence and Motivation. New York, NY: The

Guiforsd Publications Inc.; 2005:3-12.8. Murayama K, Elliot AJ, Friedman R. Achievement goals. In: Ryan

RM, ed. The Oxford Handbook of Human Motivation. New York,

NY: Oxford University Press; US; 2012:191-207.9. Remedios R, Kiseleva Z, Elliott J. Goal orientations in Russian

university students: from mastery to performance? Educ Psychol.

2008;28(6):677-691.10. Senko C, Harackiewicz JM. Regulation of achievement goals:

the role of competence feedback. J Educ Psychol. 2005;97(3):320.11. Senko C, Miles KM. Pursuing their own learning agenda: how

mastery-oriented students jeopardize their class performance.

Contemp Educ Psychol. 2008;33(4):561-583.12. Elliot AJ, McGregor H. A 2x2 achievement goal framwork.

J Pers Soc Psychol. 2001;80(3):501-519.13. Hulleman CS, Schrager SM, Bodmann SM, Harackiewicz JM. A

meta-analytic review of achievement goal measures: different labels

for the same constructs or different constructs with similar labels?

Psychol Bull. 2010;136(3):422-449.14. Huang C. Discriminant and criterion-related validity of

achievement goals in predicting academic achievement: a meta-

analysis. J Educ Psychol. 2012;104(1):48-73.15. Shim SS, Cho Y, Wang C. Classroom goal structures, social

achievement goals, and adjustment in middle school. Learn Instr.

2013;23:69-77.16. Akin A. Self-efficacy, achievement goals, and depression,

anxiety, and stress: a structural equation modelling. World Appl Sci

J. 2008;3(5):725-732.17. Murayama K, Elliot AJ. The competition-performance relation:

a meta-analytic review and test of the opposing processes model of

competition and performance. Psychol Bull. 2012;138(6):1035-1070.18. Van Yperen NW, Elliot AJ, Anseel F. The influence of mastery-

avoidance goals on performance improvement. Eur J Soc Psychol.

2009;39(6):932-943.19. Diseth A. Self-efficacy, goal orientations and learning strategies

as mediators between preceding and subsequent academic

achievement. Learn Individ Differ. 2011;21(2):191-195.20. Harackiewicz JM, Barron KE, Pintrich PR, Elliot AJ, Thrash

TM. Revision of achievement goal theory: necessary and

illuminating. J Educ Psychol. 2002;94(3):638-645.21. Wolters CA. Self-regulated learning and college students’

regulation of motivation. J Educ Psychol. 1998;90(2):224.22. Harris A, Yuill N, Luckin R. The influence of context-specific

and dispositional achievement goals on children’s paired

collaborative interaction. Br J Educ Psychol. 2008;78(3):355-

374.23. Elliot AJ, McGregor H, Gable S. Achievement goals, study

strategies, and exam performance: a mediational analysis. J Educ

Psychol. 1999;91(3):549-563.24. Harackiewicz JM, Barron KE, Tauer JM, Elliot AJ. Predicting

success in college: a longitudinal study of achievement goals and

ability measures as predictors of interest and performance from

freshman year through graduation. J Educ Psychol. 2002;94

(3):562-575.25. Barron KE, Harackiewicz JM. Revisiting the benefits of

performance-approach goals in the college classroom: Exploring the

role of goals in advanced college courses. Int J Educ Res. 2003;39

(4-5):357-374.

37


26. Senko C, Hulleman CS, Harackiewicz JM. Achievement goal

theory at the crossroads: old controversies, current challenges, and

new directions. Educ Psychol. 2011;46(1):26-47.27. Hill Y, Lomas L, MacGregor J. Students’ perceptions of quality

in higher education. Qual Assur Educ. 2003;11(1):15-20.28. Voss R, Gruber T. The desired teaching qualities of lecturers in

higher education: a means end analysis. Qual Assur Educ. 2006;

14(3):217-242.29. Stronge JH. Qualities of Effective Teachers. 2nd edition.

Alexandria, VA: ASCD Publications; 2007.30. Murayama K, Elliot AJ. The joint influence of personal

achievement goals and classroom goal structures on achievement-

relevant outcomes. J Educ Psychol. 2009;101(2):432-447.31. Wolters CA, Daugherty SG. Goal structures and teachers’ sense

of efficacy: their relation and association to teaching experience and

academic level. J Educ Psychol. 2007;99(1):181-193.32. Shim SS, Cho Y, Cassady J. Goal structures: the role of teachers’

achievement goals and theories of intelligence. J Exp Educ. 2013;81

(1):84-104.33. Senko C, Belmonte K, Yakhkind A. How students’ achievement

goals shape their beliefs about effective teaching: a “build-a-

professor” study. Br J Educ Psychol. 2012;82(3):420-435.34. Smith L, Saini B, Krass I, Chen T, Bosnic-Anticevich S,

Sainsbury E. Pharmacy students’ approaches to learning in an

Australian university. Am J Pharm Educ. 2007;71(6):Article 120.35. Alrakaf S, Abdelmageed A, Kiersma M, et al. An international

validation study of two student achievement goal questionnaires.

Australasian Pharmaceutical Science Association Conference.

Dunedin, New Zealand: APSA; 2013.36. Li NP, Bailey JM, Kenrick DT, Linsenmeier JA. The necessities

and luxuries of mate preferences: testing the tradeoffs. J Pers Soc

Psychol. 2002;82(6):947.

37. Pallant J. SPSS SURVIVAL MANUAL: A Step by Step Guide toData Analysis Using SPSS. 4th edition. Crows Nest, Australia: Allen& Unwin; 2011.38. Alrakaf S, Sainsbury E, Smith L. First year undergraduatepharmacy students’ and academics’ views of and preferences forlearning and teaching. A preliminary investigation. Res J PharmTechnol. In press.39. Reynolds D, Farrell S.Worlds Apart?: A Review of InternationalSurveys Of Educational Achievement Involving England. London,England: Stationery Office Books; 1996.40. Costa ML, van Rensburg L, Rushton N. Does teaching stylematter? A randomised trial of group discussion versus lecturesin orthopaedic undergraduate teaching. Med Educ. 2007;41(2):214-217.41. Knight JK, Wood WB. Teaching more by lecturing less. CellBiol Educ. 2005;4(4):298-310.42. Smith L, Krass I, Sainsbury E, Rose G. Pharmacy Students’Approaches to Learning in Undergraduate and Graduate EntryPrograms. Am J Pharm Educ. 2010;74(6):Article 106.43. Hughes A. Higher education in a web 2.0 world, JointInformation Systems Committee Report. Jisc. http://www.jisc.ac.uk/publications/generalpublications/2009/heweb2.aspx#downloads.Accessed February 6, 2012.44. Doiron R, Asselin M. Exploring a new learning landscape intertiary education. New Libr World. 2011;112(5/6):222-235.45. Crocker J, Olivier M-A, Nuer N. Self-image goalsand compassionate goals: costs and benefits. Self Identity. 2009;8(2-3):251-269.46. Conroy DE, Elliot AJ. Fear of failure and achievement goals insport: Addressing the issue of the chicken and the egg. Anxiety StressCoping. 2004;17(3):271-285.47. Smith L. The Role of Affective Distress in Final Year HighSchool Student Achievement Motivation. [Doctorate thesis]. Sydney:The University of Sydney; 2003.

38



Long-term Effectiveness of Online Anaphylaxis Education for Pharmacists

Sandra M Salter, BPharm,a Sandra Vale, BSc,b Frank M Sanfilippo, PhD,c

Richard Loh, MBBS,*b,d and Rhonda M Clifford, PhDa

a School of Medicine and Pharmacology, The University of Western Australia, Crawley, Western Australiab The Australasian Society of Clinical Allergy and Immunology, Sydney, New South Wales, Australiac School of Population Health, The University of Western Australia, Crawley, Western Australiad School of Paediatrics and Child Health, The University of Western Australia, Crawley, Western Australia* Bachelor of Medicine/Bachelor of Surgery

Submitted December 16, 2013; accepted February 26, 2014; published September 15, 2014.

Objective. To evaluate the long-term effectiveness of an Australasian Society of Clinical Immunologyand Allergy (ASCIA) anaphylaxis e-learning program compared to lectures or no training.Design. A controlled interrupted-time-series study of Australian pharmacists and pharmacy studentswho completed ASCIA anaphylaxis e-learning or lecture programs was conducted during 2011-2013.Effectiveness was measured using a validated test administered pretraining, posttraining, and 3 and 7months after training.Assessment.All learning groups performed significantly better on all posttests compared to the pretest,and compared to a control group (p,0.001). The proportion of e-learners achieving the minimumstandard for anaphylaxis knowledge improved from 45% at pretest to 87% at 7 months.Conclusion. The ASCIA e-learning program significantly increased anaphylaxis knowledge. The highproportion of participants achieving the minimum standard at 7 months indicates long-term knowledgechange.

Keywords: e-learning, knowledge, evaluation, Australasian Society of Clinical Immunology and Allergy,adrenaline auto-injector.

INTRODUCTIONAnaphylaxis is a severe, progressive allergic reac-

tion that is rapid in onset and may cause death.1 The in-cidence of anaphylaxis has dramatically increased overthe past decade,2-9 with more cases occurring in the com-munity setting than in the hospital setting.10 Early diag-nosis of anaphylaxis and treatment with adrenaline isessential to prevent fatalities, and deaths are more com-mon in patients with a history of asthma.10-14 Adrenalineis internationally recognized as the first-line treatment foranaphylaxis, with auto-injector devices universally rec-ommended as first aid for anaphylaxis occurring in thecommunity setting. Prescriptions for adrenaline auto-injector devices should be accompanied by a device-specific emergency action plan.11,14-19

In Australia, pharmacists supply adrenaline auto-injectors to patients who present a physician’s prescription,

or to those patients without a prescription when an indi-vidual therapeutic need is established by the pharmacist.In addition, pharmacists sell these devices to Australianschools and childcare services to facilitate emergencytreatment.20-23 With each distribution, pharmacistsshould educate patients (or their agents) about anaphy-laxis, confirm they have a device-specific ASCIA ActionPlan for Anaphylaxis, and advise them regarding the cor-rect use and storage of the adrenaline auto-injector.14,20,24

Pharmacists also provide collaborative care (usually witha family physician or specialist physician) to patientswith comorbid conditions including asthma, offer adviceabout and sellmedicines for the treatment of allergies, andare sometimes called upon to provide first aid for patientswith acute anaphylaxis. Changes to devices in Australia,including the addition of Anapen in 2010 and the changeof EpiPen to a new-look device in 2011, highlighted thepotential for patient confusion and the importance of up-to-date pharmacist advice. Therefore, pharmacists need tohave a thorough knowledge of anaphylaxis as well asadrenaline auto-injectors.

In 2011, the Australasian Society of ClinicalImmunology and Allergy (ASCIA) launched “ASCIA

Corresponding author: Sandra Salter, BPharm, School ofMedicine and Pharmacology, The University of WesternAustralia, M315, 35 Stirling Highway, Crawley WA 6009,Australia. Tel: 161 416 003 808. Fax: 161 8 9389 7628.Email: [email protected]

39


Anaphylaxis e-training for pharmacists” to meet the needfor accurate, consistent anaphylaxis education. Thise-learning package complemented existing ASCIA ana-phylaxis e-training programs for schools and childcareservices and other health professionals. The importanceof ensuring that this e-training is effective at increasinganaphylaxis knowledge is paramount to reducing the riskof fatal anaphylaxis in the community. Long-term effec-tiveness is of prime importance because the incidence ofanaphylaxis is increasing and errors in management be-cause of waning knowledge may result in a poor outcomefor the patient.

Effectiveness studies of e-learning in health profes-sionals’ education indicate e-learning is as effective astraditionalmethods at increasing knowledge immediatelyafter training.25-29 However, there is little evidence tosupport the long-term effectiveness of e-learning to en-hance knowledge, or to meet a minimum knowledge re-quirement, such as a minimum pass score. In this study,we sought to evaluate the immediate and long-term im-pact of ASCIAAnaphylaxis e-training for pharmacists onanaphylaxis knowledge, compared to ASCIA anaphy-laxis lecture training or no training.We hypothesized thatASCIA Anaphylaxis e-training for pharmacists would beas effective as ASCIA anaphylaxis lecture training at in-creasing short and long-term knowledge, meeting a mini-mum standard for anaphylaxis knowledge, and teachingthe steps required for adrenaline auto-injector device ad-ministration. We also hypothesized that both programswould be superior to no training.

DESIGNThis controlled, interrupted time-series study was

conducted in Australia between August 2011 and April2013. The University of Western Australia Human Re-search Ethics Committee gave ethics approval for thestudy in July 2011.

Intervention participants were eligible if they werepharmacists or pharmacy students within Australia.Pharmacists included professionals registered with thePharmacy Board of Australia (PBA) and pharmacy in-terns who held provisional registration as a pharmacistwith PBA and who were completing practice hours un-der the direct supervision of a registered pharmacist.Pharmacy students were individuals enrolled in an ap-proved course of study in the field of pharmacy at anAustralian university. Control participants were stu-dents of medicine or pharmacy at the University ofWestern Australia.

All participants were recruited using a convenienceapproach. E-learning participants were recruited fromacross Australia while registering online for ASCIA

anaphylaxis e-learning between September 2011 andMay 2012. Lecture participants were recruited whileattending ASCIA anaphylaxis lectures in Perth,WesternAustralia, between August and September 2011. As thee-learning and lecture participants were separated byboth place and time, randomization to either interven-tion arm was not possible. Control participants wererecruited while attending regular university lecturesand tutorials in Perth, Western Australia, in September2012. The aims, objectives, relevance of the study, andoption to participate were explained, and all participantsprovided written, informed consent prior to enrollment inthe study (e-learning participants gave consent by select-ing an “I Agree” checkbox online). Participants alsocompleted a short demographic survey, which includedthe variables gender, age group, main job in pharmacy,type of control student, postal code of main workplace,and graduation year.

ASCIA Anaphylaxis Training for PharmacistsThe training program was developed by ASCIA in

consultation with the Pharmaceutical Society of WesternAustralia, the Pharmaceutical Society of Australia, thePharmacy Guild of Australia, and the Society of HospitalPharmacists ofAustralia. The training was advertised as anaccredited continuing professional development (CPD)activity with these organizations, as well as through pro-fessional newsletters, magazines, and websites. Table 1provides an overview of the training. Briefly, e-learningand face-to-face lecture programs consisted of the same 4modules, each designed to take 15 minutes to complete.E-learning was presented as a series of slides using Meta-morphosis software (Easy Authoring, Sydney, Australia).Face-to-face lectures were delivered as Microsoft Power-Point slides.

E-learning participants were allowed to completetraining at their own pace, although it was recommendedthat all modules and tests be completed within a 2-weekperiod. Explanatory notes for slides accompanied thee-learning program to ensure equivalence with spokenmaterial presented in face-to-face lectures. Participantswere encouraged to obtain their own trainer adrenalineauto-injector devices and practice the steps required fortheir administration while completing the program.

Lecture participants attended one of three 1-hour,face-to-face lectures. To ensure consistency across lec-tures, a dedicated ASCIA-approved lecturer (a clinicalimmunology/allergy medical specialist) delivered all lec-tures in the study. Participants were provided with traineradrenaline auto-injector devices for the duration of thelecture only, and a hands-on activity was included todemonstrate the steps required for administration.

40


Completion of a posttest was a requirement for CPDcredits in both programs. Understanding the correct an-swer is considered part of the learning experience, and e-learning participants received brief and immediate onlinefeedback (as part of the learning program) on their testresults, including the correct answers to questions. Lec-ture participants were able to access the correct answersfrom researchers in the lecture room after completing theposttest. Neither group received a link to or copy of thetest answers, nor were answers provided at the 3-month or7-month follow-up tests. For students, the training did notform part of any university assessment.

Control participants attended a lecture on women’shealth, participated in a discussion session on profes-sional pharmacy practice, or completed a pharmacy-dispensing laboratory session. All control interventionslasted 60 minutes.

EVALUATION AND ASSESSMENTKnowledge gain was assessed using a 12-question

test, the Anaphylaxis Training Pharmacist Assessment

Tool (AT-PAsT), whichwe developed and validated priorto use in the study.30 We used a combination of multiple-

choice, yes/no, and order-the-steps questions to measure

knowledge of the prevention, identification, and manage-

ment of anaphylaxis in the community setting. An expert

group of 10 allergy and immunology physicians and 2

clinical pharmacists developed the test questions and

assessed content validity. Modifications to wording and

content changes were made to 2 questions. Face validity

was evaluated in a group of 15 pharmacists and 5 phar-

macy students, and all agreed they understood the ques-

tions and response options. This test was pilot tested on

a group of 67 pharmacists who attended an ASCIA ana-

phylaxis lecture in Adelaide, South Australia, in July

2011. Although the test demonstrated a significant im-

provement in knowledge scores after the lecture

(8.2-11.2 points, paired t test; p,0.001), 4 questions didnot show response change and thus may have overstated

knowledge (McNemar test; p.0.5). These questions wereredeveloped, reviewed by the expert group and pharmacists

Table 1. An Overview of ASCIA Anaphylaxis Training for Pharmacists

AimTo provide ready access to accurate and consistent anaphylaxis education to pharmacists throughout Australia and New Zealand.Learning objectivesOn completion of this program participants should be able to:d Define anaphylaxis.d Identify common causes of anaphylaxis.d Identify the signs and symptoms of a mild to moderate allergic reaction.d Identify the signs and symptoms of anaphylaxis.d Outline the acute management for anaphylaxis.d Describe the effects of adrenaline on the body.d List the side effects of adrenaline.d Explain how to correctly store adrenaline auto-injector devices.d Differentiate between the EpiPen and Anapen devices.d Differentiate between junior and adult adrenaline auto-injector devices.d Demonstrate how to use the EpiPen and Anapen auto-injectors using trainer devices.d Outline management required after an adrenaline auto-injector has been administered.d Explain the purpose of the ASCIA Action Plan.d Identify the most appropriate Action Plan for the patient.d Identify the roles of the pharmacist in anaphylaxis management.

Program ContentModule 1 What is allergy and anaphylaxis?Module 2 Acute management of anaphylaxisModule 3 Adrenaline auto-injectorsModule 4 ASCIA Action Plans and the role of pharmacists in anaphylaxis managementModule 5 AssessmentProgram DeliveryE-learning or face-to-face lectures15 minutes per module (total of 60 minutes of training, plus assessment)AssessmentTwelve knowledge assessment questions – multiple choice, yes/no, and order-the-steps questions.

Abbreviations: ASCIA5 Australasian Society of Clinical Allergy and Immunology.

41


forcontentandfacevalidity,and incorporated into thefinalversion of the AT-PAsT.

The test was administered immediately beforetraining and immediately after training, then 3 and 7months after training. To reduce practice effect, thequestions and their response options were reordered oneach test. Participants in the e-learning group completedthe pretest and posttest online as part of the e-learningprogram. Participants in the lecture and control groupscompleted the pretest and posttest on paper in the lectureor tutorial room. Pharmacy students completed the3-month follow-up test on paper. All other tests werecompleted through the online research suite Qualtrics(Qualtrics, Utah). When follow-up tests were due, par-ticipants received an e-mail notification and up to 5e-mail reminders. The follow-up tests remained accessi-ble for 2 weeks. Three prizes (cinema tickets or retailvouchers), with a maximum value of AU$100, wereprovided as an incentive to complete each of the fol-low-up tests. Of the participants who completed the 3-month and 7-month follow-up tests, 1 winner from eachgroup (e-learning, lecture or control), was drawn at ran-dom. There were no other incentives provided in thestudy.

As there were no reliable estimates for expectedstandard deviation in score, we did not conduct a priorisample-size calculations. However, a post hoc powercalculation, using the 7-month posttest sample size of30 in the e-learning group and 50 controls with an ob-served standard deviation of 1.4 points, showed thatthe study had 86% power to detect a difference in scoreof 1 point between groups at the 5% level of signifi-cance. Calculations for all other sample sizes in thestudy groups yielded power estimates between86% and 100% for between-group and within-groupcomparisons.31

AnalysisAll analyses were performed using SPSS version 21

(IBM, New York), and reported as 2-sided p-values witha 5% level of significance. A linear mixed-effects modelwith post hoc pairwise analysis was used to evaluatechanges in short-term and long-term knowledge withinand between learning and control groups. We specifiedscore as the dependent variable, with group (e-learning,lecture pharmacists, lecture pharmacy students, or con-trol) and test (pretest, posttest, 3-month and 7-monthtests) as covariates. We compared models with and with-out demographic covariates (gender, age group, main jobin pharmacy, type of control student, postal code of mainworkplace, and years since graduation). As the majorityof the sample was from Western Australia, we converted

the postal code of main workplace to 2 geographic areas,Western Australia or all other Australian states. Analyseswere restricted to participantswho had valid, non-missingdata for all variables in the model.

We compared the proportion of participants withinand between learning groups who, at each test, achievedthe minimum standard for anaphylaxis knowledge(score$9 out of 12) and correctly ordered the steps forEpiPen and Anapen device administration. The Pearsonchi-square test was used for between-group comparisonsand theMcNemar test was used for within-group compar-isons. Data for individual answers to the device-orderingquestions for the e-learning group were not available forthe pretest and posttest (only the overall scoreswere avail-able). Therefore, we could only make comparisons be-tween the 3-month and 7-month tests in the e-learninggroup.

ResultsWe recruited 383 participants (277 intervention and

106 controls) to the study (Table 2). There was significantdiversity across all 4 groups based on demographic vari-ables (p,0.001). E-learning and lecture pharmacistsgroups were similar by age group and years since gradu-ation, but differed by gender, main job in pharmacy, andlocation of main job (Table 2). Completion rates acrossthe 4 tests ranged from 100% at posttest, to 47.2% at 7months (Figure 1), and were similar between groups(p50.91 at 7 months).

Mean knowledge scores were significantly differentby group and test (p,0.001, Table 3). With all demo-graphic variables in the model, there were no significantdifferences in score by age group (p50.28), main job inpharmacy (p50.06), type of control student (p50.082),state of main workplace (p50.96), or years since gradu-ation (p50.56). Score initially differed significantly bygender (p50.05); however, this effect was lost whennon-significant variables were removed from the model(p50.06).

Figure 2 and Table 3 show mean AT-PAsT scoresby group and test. There was a significant and sustainedimprovement in anaphylaxis knowledge after trainingin all learning groups (paired t tests, p,0.001 for allcomparisons). Mean scores improved by 3.3, 2.8, and4.6 points immediately after training in the e-learning,lecture pharmacists, and lecture pharmacy studentsgroups, respectively, but decreased in the control group.Mean scores decreased significantly from posttestscores in all learning groups at the 3-month test (a re-spective score decrease of 1.6, 1.4, 1.7 points). At 7months, mean scores improved and were above the min-imum standard in all learning groups. There were no

42


significant changes in mean score in the control groupafter posttest.

Figure 3 shows the change in mean AT-PAsT scoresby group over time. All learning groups performed

significantly better on all posttests compared to control(p,0.001 for all comparisons). E-learning and lecturepharmacist participants had similar scores across all testsexcept posttest, where e-learning scores were slightly

Table 2. Participant Characteristics by Intervention and Control Group at Pretest (count and %)

CharacteristicaE-learning

n=57

LecturePharmacists

n=154

LecturePharmacy

Students n=66Controln=106

Totaln=383 Pb

Gender ,0.001Male 43 (75.4) 41 (26.6) 22 (33.3) 36 (34.0) 142 (37.1)

Age group (years) ,0.00118-24 7 (12.3) 32 (20.8) 37 (56.0) 82 (77.4) 158 (41.2)25-34 24 (42.1) 51 (33.1) 23 (34.8) 18 (17.0) 116 (30.3)35-44 9 (15.8) 23 (14.9) 4 (6.1) 1 (0.9) 37 (9.7)45-54 11 (19.3) 19 (12.3) 1 (1.5) 0 31 (8.1)551 6 (10.5) 27 (17.5) 0 0 33 (8.6)

Main job in pharmacyc,d ,0.001Community pharmacist 27 (47.3) 99 (64.3) NA NA 126 (32.9)Hospital pharmacist 16 (28.0) 15 (9.7) NA NA 31 (8.1)Pharmacy intern 4 (7.0) 22 (14.3) NA NA 26 (6.8)Pharmacy student 3 (5.3) 0 65 (100) NA 68 (17.8)Pharmacy academic 3 (5.3) 9 (5.8) NA NA 12 (3.1)

Type of controlMedical student NA NA NA 66 (62.3) 66 (17.2)Pharmacy student NA NA NA 35 (33.0) 35 (9.1)

Years since graduatione ,0.001Not graduated 0 0 65 (98.5) 101 (95.2) 166 (43.3)Less than 5 8 (14.0) 44 (29.5) 0 0 52 (13.6)5-10 17 (29.8) 33 (22.1) 0 0 50 (13.0)11-15 9 (15.8) 13 (8.7) 0 0 22 (5.7)More than 15 23 (40.4) 59 (39.6) 0 0 82 (21.4)

Location of main jobWestern Australia 9 154 (100) 65 (98.5) 101 (95.2) 329 (85.9)New South Wales 14 0 0 0 14 (3.6)Victoria 13 0 0 0 13 (3.4)Queensland 12 0 0 0 12 (3.1)South Australia 4 0 0 0 4 (1.0)Tasmania 2 0 0 0 2 (,1)Australian Capital Territory 2 0 0 0 2 (,1)Northern Territory 1 0 0 0 1 (,1)

Main job by region ,0.001Western Australia 9 (15.8) 154 (100) 65 (98.5) 101 (95.2) 329 (85.9)Rest of Australia 48 (84.2) 0 0 0 48 (12.5)

a Eight participants (2 lecture pharmacists, 1 lecture pharmacy student and 5 controls) did not provide any demographic data.b Pearson chi-square p value for comparison of demographic variables across all 4 groups. Location of main job was compared by region only.P values for comparison of demographic variables between e-learning and lecture pharmacists groups were: gender: p,0.001; age group: p50.11;main job: p,0.001; years since graduation: p50.08; main job by region: p,0.001.c Of those completing the demographic questionnaire, 7 lecture group pharmacists did not answer the main job question.d Of those completing the demographic questionnaire, 4 e-learning participants stated ‘other main job’ including defence force, industrial, andcompounding pharmacist jobs.e 3 lecture group pharmacists did not answer the years since graduation question.NA: not applicable

43


higher (0.65 points, p50.04). Lecture pharmacy studentshad the greatest gains in knowledge of all learning groups,yet lower scores. It was not possible to compare e-learningscores for pharmacy students with lecture pharmacy stu-dents’ scores, as only 3 pharmacy students completed thee-learning program.

There were significant and sustained improvementsin the proportion of learners achieving the minimum stan-dard for anaphylaxis knowledge after training (Table 4).Less than 46% of e-learning and lecture pharmacistsachieved the minimum standard before training; how-ever, 7 months after training, over 80% achieved thisstandard. The improvement in the proportion of lecturepharmacy students achieving the standardwas almost ten-fold: from 6.7% pretest to 61.8% at 7 months.

Although there were sizeable gains in the propor-tion of lecture participants who passed the device-ordering questions after training, these gains were notsustained over time (Table 4). At 7 months, 63.3% of e-learning participants and 61.5% of lecture pharmacistscorrectly ordered the 4 steps for both EpiPen andAnapen, an improvement of around 15% in each groupfrom pretest.

DISCUSSIONPharmacists play a vital role in the management of

anaphylaxis patients. Easily accessible, effective anaphy-laxis education is essential to fulfil this role.32-35 How-ever, there is little evidence of the effectiveness ofanaphylaxis training for pharmacists. We evaluated thee-learning program, ASCIA Anaphylaxis e-training forpharmacists, and measured its effectiveness in terms ofknowledge change.

This education program was associated with signif-icant and sustained improvements in anaphylaxis knowl-edge. Short-term knowledge gains (on average, a 39%improvement in mean score) were similar to immediategains seen in other pharmacy e-learning effectivenessstudies.36-39 Persistence of knowledge 7 months aftertraining was high: almost 90% of e-learners achieved ator above our minimum standard for anaphylaxis knowl-edge, compared to 45% of the same learners before train-ing. Thus, the results add long-term effectiveness to theexisting body of e-learning pedagogical research,25-29 andmore importantly, demonstrate that this education pro-gram is effective long-term.ASCIAAnaphylaxis e-trainingfor pharmacists was as effective as lecture training,

Figure 1. Study groups, participation and completion rates by group and test.

44


and significantly more effective than no training, at im-proving short-term and long-term anaphylaxis knowledgein pharmacists.We were unable to demonstrate effective-ness of this e-learning program in pharmacy students dueto low numbers of student participants. Even so, lecturetraining was effective at improving short-term and long-term anaphylaxis knowledge in pharmacy students, andother research has demonstrated short-term effectivenessof e-learning in pharmacy students in different subjectareas.38,40-42 Therefore, it is likely that this e-learningprogram would also be effective for pharmacy students.There was no change in anaphylaxis knowledge in thosewho did not receive training. This is consistent with thebroader literature for short-term e-learning effective-ness.26,29 However, as far as we know, this is thefirst study to demonstrate long-term differences in ane-learning group compared to a groupwho did not receivetraining.

An essential part of anaphylaxis education for pa-tients is hands-on training in the use of adrenaline auto-injectors. Although pharmacists are ideally placed todeliver this training, there is evidence that the majorityof anaphylaxis patients do not receive it.24,43,44 Peoplewho do not know how or when to use their adrenalineauto-injector may elect not to do so in an emergency, ormay incorrectly activate the device.45 Devices and pro-cedures change over time, and there is a constant need toimprove pharmacists’ skills in this area, so they can bettertrain those at risk of anaphylaxis.13,35,43,44,46 Approxi-mately two-thirds of e-learners in our study were able tocorrectly order all of the steps required for both EpiPenand Anapen administration 7 months after training. Lec-ture participants achieved results similar to those for e-learners, even though they had hands-on practice withdevices during training. Although long-term device recallwas poorer compared to anaphylaxis knowledge, otherresearch has shown device recall may wane overtime.47,48 In a group of physician trainees, only one-third

accurately demonstrated devices 6 months after train-ing.48 In our study, the complexities of the different de-vices, lack of regular experience with them, and the factthey were new tomany pharmacists at the time of trainingmay have impacted pharmacists’ long-term recall. As theparticipants were geographically diverse, we did not eval-uate device demonstration as a skill. Thus, while knowl-edge of device administration steps improved at 7months,application of this knowledge was not assessed.

Strengths and limitationsThis study has a number of strengths. The training

program and assessment test were developed using a rig-orous approach and validated prior to use. We included 2comparator groups in our study: traditional lecture train-ing and no training. Further, we conducted 3 posttrainingtests, with a follow-up period considerably longer thanthat of other e-learning effectiveness studies. Retentionrates were high: almost all participants completed theposttest, and around 50% completed all 4 tests. This com-pares favorably with response rates to e-mailed surveys(where the average response rate is 33%).49 The study hadsufficient power to detect a mean score difference of atleast 1 point within and between groups. Finally, therewas no duplication in recruitment of pharmacists to in-tervention groups (pharmacists who participated in the e-learning group could not participate in the lecture groupand vice versa).

However, we did not randomize participants to in-tervention or control groups, and as we adopted a conve-nience method of recruitment, the study may have beenaffected by selection bias. The lack of randomizationwould only affect between-group comparisons.Neverthe-less, generalization of the e-learning results may be lim-ited to people with a high comfort level with learning viathe Internet and/or who have experience using multime-dia online. Given that the study sample representedwell-educated professionals who had daily exposure to

Table 3. Mean Anaphylaxis Training Knowledge Assessment Score by Group and Testa

E-learning Lecture Pharmacists Lecture Pharmacy Students Control

Pretest 8.27 (7.80-8.43) n551 8.11 (7.79-8.43) n5153 5.40 (4.90-5.90) n560 4.57 (4.18-4.99) n5106Posttest 11.53 (11.0-12.0) n552 10.88 (10.56-11.19) n5151 9.96 (9.47-10.46) n562 3.72 (3.34-4.01) n51063-month testc 9.96 (9.0-10.18) n540 9.50 (9.11-9.89) n589 8.25 (7.70-8.80) n545 3.63 (3.15-4.12) n5557-month testd 10.05 (9.40-10.71) n530 9.66 (9.25-10.06) n578 9.05 (8.43-9.67) n534 3.68 (3.17-4.18) n550a Reported as estimated marginal mean (95% CI) for each test; maximum test score512. Type III tests of fixed effects with mean score asdependent variable: p,0.001 for group and test.b Pairwise comparisons of pretest with posttest, 3-month test, and 7-month test, by group.c Pairwise comparison of 3-month test with posttest, p,0.001 for e-learning, lecture pharmacists and lecture pharmacy students; p50.74 forcontrol.d Pairwise comparison of 7-month test with 3-month test; p50.22, 0.51, 0.02, and 0.88 for e-learning, lecture pharmacists, lecture pharmacystudents, and control, respectively.

45


Internet-related technologies, we expected knowledgeand use of the Internet to be high in this population. Lec-ture participants alsowere required to show a high level ofcomfort with Internet use, as they were required to com-plete all follow-up tests online. Further, the vast literatureevaluating e-learning programs, the increasing deliveryof online education, a historical early acceptance of tech-nology in the pharmacy profession (all suggesting phar-macists are confident Internet users), and the difficultiesachieving a true random sample in online research mayhave combined to reduce the effect of selection bias in ourstudy.50-52 In addition, we evaluatedASCIAAnaphylaxise-training for pharmacists in a contextwhere learners nowdefine their education strategies (eg, choosing rather thanbeing recruited to undertake this program),53 which mayhave provided real-world evidence for effectiveness.

The control group did not include pharmacists andbegan the study at a different time than the interventiongroups. We chose to use students as controls because wecould ensure that they did not receive inadvertent expo-sure to anaphylaxis training and thus contamination dur-ing follow up. Nonetheless, we acknowledge that controlscoreswere significantly lower than intervention scores atpretest. This ultimately impacted pairwise comparisonsand may have distorted the magnitude of the differencebetween training and no training. Moreover, the controlscores did not change over time, despite participants com-pleting the same test questions on 4 occasions. This mayhave been because of/the result of lack of interest in thetopic, lack of perceived relevance to practice, fatiguefrom completing multiple tests, or a true effect.

We used the same 12 questions for each of the 4 tests.There was the potential for a learning effect from the testitself, although we did attempt to control for practiceeffect, and it was unlikely given there was no change incontrol scores. Although we did not adjust for multiplecomparisons in the analyses, we do not consider this to bea limitation. The key effectiveness measure—long-termknowledge change—was assessed in 3 post hoc tests(e-learning, lecture training, or no training groups, com-paring 7-month tests and pretests), and the magnitude ofthe change in knowledge at all tests was large. Therefore,with low numbers of multiple comparisons, an effect sizeof practical relevance, and very low p-values (p,0.001),there was no need for adjustment.54

Finally, we acknowledge that this training may nothave been wholly responsible for knowledge demon-strated at 7 months. There is the potential for academicdishonesty with tests completed remotely. However, par-ticipants were de-identified and study incentives were notdependent on scores, so we consider the motivation todeceive was low. Although exposure to alternate anaphy-laxis information over time (eg, through general media orthrough self-study) may have confounded the results,knowledge gain across learner groups was consistent(with no gain in the control group) over 7 months.

Implications and recommendationsASCIA Anaphylaxis e-training for pharmacists is

part of a group of e-learning packages available to phar-macists and other health professionals, school and child-care workers, and the general community throughoutAustralia and New Zealand. Since 2011, more than 760pharmacists, 4600 health professionals, 130 000 schooland childcare workers, and 1100 members of the generalpublic, have completed this training.55 The key messagesin each of these programs are equivalent, and the languageused in each program is appropriate for the intended

Figure 2. Mean anaphylaxis knowledge assessment scores bygroup and test.

Figure 3. Change in mean anaphylaxis knowledge assessmentscores by group over time. E-learning scores were similar to lec-ture pharmacist scores at pre-test (p=0.62), 3-month test (p=0.79)and 7-month test (p=0.31). Control scores were significantlylower than all intervention scores after training (p,0.001).

46


learner. Despite the success in implementation, ASCIAanaphylaxis e-training programs have not previouslybeen evaluated for effectiveness. The study demonstratesthat ASCIA Anaphylaxis e-training for pharmacists iseffective at increasing and maintaining long-term ana-phylaxis knowledge across a demographically and geo-graphically diverse population of pharmacists.

Because accurate and current anaphylaxis knowledgeis an essential part of anaphylaxis management, the ques-tion of when to retrain should be considered. As the major-ity of e-learnersmet theminimum standard for anaphylaxisknowledge 7 months after training, it is difficult to definea retraining interval based on declining knowledge. Anadditional follow-up evaluation of the same participantsat 18-24 months may be a realistic timeframe. For phar-macy students in the era of the flipped classroom, the ad-dition of this e-learning program would increase theiranaphylaxis knowledge while allowing them to activelypractice with adrenaline auto-injector devices. Investigat-ing the effectiveness of the e-learning program in this con-text would be useful.

Pharmacists have been identified as an underuti-lized resource for providing anaphylaxis education anddevice training at the time of adrenaline auto-injectorsupply.43,44 One-third of e-learners in the study did

not correctly order the steps for EpiPen and Anapendevice administration, and this may impact the qualityof advice provided with these devices. Covert or overtsimulation-based research is required to determine whathappens at the time of adrenaline auto-injector distribu-tion in pharmacies, as a measure of translation of anaphy-laxis learning to practice. Research options includesimulated patient methodology to assess device demon-stration and anaphylaxis knowledge, or the use of overtsimulation (for example, using mannequins) to investi-gate the pharmacist’s response to acute anaphylaxis.

SUMMARYRegular education updates are required for pharma-

cists to maintain current knowledge about the preventionand treatment of anaphylaxis and how to supply and useadrenaline auto-injectors. ASCIA Anaphylaxis e-trainingfor pharmacists increased anaphylaxis knowledge long-term. Knowledge gains were similar to ASCIA lecturetraining and superior to no training. This e-learning pro-gram offers a convenient, effective, no-cost option forpharmacists to improve and maintain their anaphylaxisknowledge. Future evaluations should seek to define aninterval for retraining and investigate translation of ana-phylaxis knowledge to practice.

Table 4. Learners Achieving the Minimum Standard for Anaphylaxis Knowledge and the Correct Device Administration Steps byGroup and Test.

E-learningLecture

Pharmacists Lecture Pharmacy Studentsp all

groupsa

P e-Learning vsLecture

Pharmacistsb

Proportion achievingminimum standard, %

Pretest 45.1 45.8 6.7 ,0.001 0.94Posttest 96.2 97.4 85.5 0.002 0.663-month 85.0 74.2 53.3 0.004 0.177-month 86.7 80.8 61.8 0.03 0.47pc 0.021 0.001 ,0.001

Proportion correctlyordering deviceadministration steps, %

Pretest - 34.0 28.3 0.43 -Posttest - 87.4 77.4 0.07 -3-month 45.0 47.2 40.0 0.73 0.827-month 63.3 61.5 50.0 0.45 0.86pc - 0.002 0.096a Pearson chi-square test for difference in proportions across all learning groups at each test.b Pearson chi-square test for difference in proportions between e-learning and lecture pharmacists groups at each test.c McNemar test for difference in proportions between the 7-month test and pretest for each group.Proportion achieving minimum standard - percentage of participants completing the test who achieved a score $9 out of 12.Proportion correctly ordering device administration steps - percentage of participants completing the test who correctly ordered all 4 stepsrequired for both EpiPen and Anapen device administration.NA - data were not available for this group and these tests.

47


ACKNOWLEDGMENTSThe authors acknowledge the Australasian Society of

Clinical Immunology and Allergy (ASCIA) for creatingand delivering anaphylaxis education to all members ofthe community and for enabling us to evaluate the effec-tiveness of their anaphylaxis training for pharmacists.The authors thank Ms. Suzanne Grainger, Impagination(http://www.impagination.com.au), Victoria, Australia,for her assistance with the development and implementa-tion of the online data collection forms for pretests andposttests for the e-learning participants, and Ms. LauraFirth, Department of Mathematics and Statistics, TheUniversity of Western Australia, for her assistance withplanning the statistical analyses.

The first author, Ms. Sandra Salter, was the recipientof a University Postgraduate Award and UWA Top-UpScholarship, provided by The University of WesternAustralia.

REFERENCES1. Sampson HA, Munoz-Furlong A, Campbell RL, et al. Secondsymposium on the definition and management of anaphylaxis:summary report-Second National Institute of Allergy and InfectiousDisease/Food Allergy and Anaphylaxis Network symposium.J Allergy Clin Immunol. 2006;117(2):391-7.2. Lieberman P. Epidemiology of anaphylaxis. Curr Opin AllergyClin Immunol. 2008;8(4):316-20.3. Lieberman P, Camargo CA, Bohlke K, et al. Epidemiology ofanaphylaxis: findings of the american college of allergy, asthma andimmunology epidemiology of anaphylaxis working group. AnnAllergy Asthma Immunol. 2006;97(5):596-602.4. Liew WK, Williamson E, Tang ML. Anaphylaxis fatalities andadmissions in australia. J Allergy Clin Immunol. 2009;123(2):434-442.5. Lin RY, Anderson AS, Shah SN, Nurruzzaman F. Increasinganaphylaxis hospitalizations in the first 2 decades of life: New YorkState, 1990–2006. Ann Allergy Asthma Immunol. 2008;101(4):387-393.6. Poulos LM, Waters AM, Correll PK, Loblay RH, Marks GB.Trends in hospitalizations for anaphylaxis, angioedema, and urticariain Australia, 1993-1994 to 2004-2005. J Allergy Clin Immunol.2007;120(4):878-884.7. Prescott S. The Allergy Epidemic – A Mystery of Modern Life.Perth, WA: UWA Publishing; 2011.8. Simons FER, Peterson S, Black CD. Epinephrine dispensingpatterns for an out-of-hospital population: a novel approach tostudying the epidemiology of anaphylaxis. J Allergy Clin Immunol.2002;110(4):647-651.9. Tang ML, Osborne N, Allen K. Epidemiology of anaphylaxis.Curr Opin Allergy Clin Immunol. 2009; 9(4):351-6.10. Simons FE. Anaphylaxis, killer allergy: long-termmanagement in the community. J Allergy Clin Immunol. 2006;117(2):367-377.11. Sicherer SH, Simons FE. Self-injectable epinephrine for first-aidmanagement of anaphylaxis. Pediatrics. 2007;119(3):638-646.12. Simons FE. Anaphylaxis: recent advances in assessment andtreatment. J Allergy Clin Immunol. 2009; 124(4):625-636.

13. Simons FE. Anaphylaxis. J Allergy Clin Immunol. 2010;125(2 Suppl 2):S161-S181.14. Simons FE, Ardusso LR, Bilo MB, et al. World allergyorganization guidelines for the assessment and management ofanaphylaxis. World Allergy Organ J. 2011;4(2):13-37.15. Kemp SF, Lockey RF, Simons FE. Epinephrine: the drug ofchoice for anaphylaxis. A statement of the world allergy organization.Allergy. 2008;63(8):1061-1070.16. ASCIA Action Plans for Anaphylaxis, The Australasian Societyof Clinical Immunology and Allergy, AU. http://www.allergy.org.au/health-professionals/ascia-plans-action-and-treatment. AccessedNovember 26, 2013.17. Emergency treatment of anaphylactic reactions. Guidelines forhealthcare providers, Working Group of the Resuscitation Council,UK. http://www.resus.org.uk/pages/reaction.pdf. AccessedNovember 12, 2013.18. Muraro A, Roberts G, Clark A, et al. The management ofanaphylaxis in childhood: position paper of the european academy of

allergology and clinical immunology. Allergy. 2007;62(8):857-871.19. Acute management of anaphylaxis guidelines, AustralasianSociety of Clinical Immunology and Allergy, AU. http://www.allergy.org.au/images/stories/pospapers/ASCIA_Acute_Management_of_Anaphylaxis_Guidelines_September_2013.pdf. AccessedNovember 26, 2013.20. Pharmacists Information Bulletin: Supplying adrenaline

autoinjectors - schedule 3 medicines, Pharmaceutical ServicesBranch, Health Department, Government of Western Australia, AU.http://www.health.wa.gov.au/anaphylaxis/HP/. Accessed November26, 2013.21. Anaphylaxis guidelines for Queensland state schools,Queensland Department of Education,Training and Employment,AU. http://education.qld.gov.au/schools/healthy/docs/anaphylaxis_guidelines_for_queensland_state_schools.pdf.

Accessed 30 October, 2013.22. Anaphylaxis guidelines. A resource for managing severeallergies in Victorian schools, Department of Education and EarlyChildhood Development, Government of Victoria, AU. http://www.education.vic.gov.au/school/principals/health/Pages/anaphylaxisschools.aspx. Accessed August 20, 2014.23. Anaphylaxis Procedures for Schools 2012, Education and

Communities, NSW Government, AU. http://www.schools.nsw.edu.au/media/downloads/schoolsweb/studentsupport/studenthealth/conditions/anaphylaxis/guidelines/anaphylaxis-procedures.pdf.Accessed October 30, 2013.24. Diamond S, Salter J, Hummel D. The role of pharmacists inanaphylaxis education. J Allergy Clin Immunol. 2003;111(1):S102.25. Chumley-Jones HS, Dobbie A, Alford CL. Web-based learning:sound educational method or hype? A review of the evaluationliterature. Acad Med. 2002;77(10 Suppl):S86-S93.26. Cook DA. Levinson AJ, Garside S, Dupras DM, Erwin PJ,Montori VM. Internet-based learning in the health professions:a meta-analysis. JAMA. 2008;300(10):1181-1196.27. Curran VR, Fleet L. A review of evaluation outcomes of web-based continuing medical education. Med Educ. 2005;39(6):561-7.28. Lahti M, Hatonen H, Valimaki M. Impact of e-learning on nurses’and student nurses knowledge, skills, and satisfaction: a systematicreview and meta-analysis. Int J Nurs Stud. 2014; 51(1):136-149.29. Ruiz JG, Mintzer MJ, Leipzig RM. The impact of e-learning inmedical education. Acad Med. 2006; 81(3):207-212.30. Salter SM, Loh R, Vale S, Clifford RM, editors. Evaluation of theanaphylaxis training pharmacist assessment tool (AT-PAsT): a pilot

48


study. Australasian Pharmaceutical Sciences AssociationConference; December 2011; Adelaide, Australia.31. PS: Power and Sample Size Calculation, Vanderbilt UniversitySchool of Medicine, Department of Biostatistics, Nashville, TN, US.http://biostat.mc.vanderbilt.edu/wiki/Main/PowerSampleSize.Accessed September 20, 2013.32. Tang MLK, Kang LW. Prevention and treatment of anaphylaxis.Paediatr Child Health. 2008;18(7):309-316.33. Loh RKS, Vale S. Adrenaline autoinjectors - what pharmacistsneed to know. Aust Pharm. 2011; 30(8):692,4–6.34. Vale S, Mullins R, Smith J, Loh R. Anaphylaxis training coursesfor pharmacists in Australia and New Zealand. J Allergy ClinImmunol. 2012;129(2):AB178.35. Simons FE. Anaphylaxis: evidence-based long-term risk reductionin the community. Immunol Allergy Clin North Am. 2007; 27(2):231-248.36. Flowers SK, Vanderbush RE, Hastings JK. Web-basedmultimedia vignettes in advanced community pharmacy practiceexperiences. Am J Pharm Educ. 2010;74(3):Article 39.37. Sweet BV, Welage LS, Johnston JP. Effect of a Web-basedcontinuing-education program on pharmacist learning. Am J Health-Syst Pharm. 2009;66(21):1902-3.38. Hall DL, Corman SL, Drab SR, Smith RB, Meyer SM.Application of a technology-based instructional resource in diabeteseducation at multiple schools of pharmacy: evaluation of studentlearning and satisfaction. Curr Pharm Teach Learn. 2010; 2(2):108-113.39. Legris ME, Seguin NC, Desforges K, et al. Pharmacist web-based training program on medication use in chronic kidney diseasepatients: impact on knowledge, skills, and satisfaction. J Contin EducHealthProf. 2011;31(3):140-150.40. Congdon HB, Nutter DA, Charneski L, Butko P et al. Impact ofhybrid delivery of education on student academic performance andthe student experience. Am J Pharm Educ. 2009;73(7):Article 121.41. Erickson SR, Chang A, Johnson CE, Gruppen LD. Lectureversus web tutorial for pharmacy students’ learning of MDItechnique. Ann Pharmacother. 2003;37(4):500-5.42. Lancaster JW, McQueeney ML, Van Amburgh JA. Onlinelecture delivery paired with in class problem-based learning. . .does itenhance student learning? Curr Pharm Teach Learn. 2011;3(1):23-9.

43. Barnett CW. Need for community pharmacist-provided food-allergy education and auto-injectable epinephrine training. J AmPharm Assoc. 2005;45(4):479-485.44. Simons KJ, Simons FE. Epinephrine and its use in anaphylaxis:current issues. Curr Opin Allergy Clin Immunol. 2010;10(4):354-361.45. Simons FE, Clark S, Camargo Jr CA. Anaphylaxis in thecommunity: learning from the survivors. J Allergy Clin Immunol.2009;124(2):301–6.46. Simons FE, Lieberman PL, Read Jr EJ, Edwards, ES. Hazardsof unintentional injection of epinephrine from autoinjectors:a systematic review. Ann Allergy Asthma Immunol. 2009;102(4):282–7.47. Sicherer SH, Vargas PA, Groetch ME, et al. Development andvalidation of educational materials for food allergy. J Pediatr.2012;160(4):651-6.48. Topal E, Bakirtas A, Yilmaz O, et al. When should we performa repeat training on adrenaline auto-injector use for physiciantrainees? Allergol Immunopathol (Madr). 2013; In press.49. Shih T-H, Fan X. Comparing response rates in e-mail and papersurveys: a meta-analysis. Educ Res Rev. 2009;4(1):26-40.50. Hesse-Biber S, Griffin AJ. Internet-mediated technologies andmixed methods research: problems and prospects. J of MixedMethods Res. 2013;7(1):43-61.51. Monaghan MS, Cain JJ, Malone PM, et al. Educationaltechnology use among US colleges and schools of pharmacy. Am JPharm Educ. 2011;75(5):Article 87.52. Bartolini EZ, Hubbard T. Community pharmacies meetmobile technologies: a whole new world of opportunity. USPharm. 2013;38(8):43-50. http://www.uspharmacist.com/content/d/featured%20articles/c/42380/. Accessed August 20,2014.53. Mascolo MF. Beyond student-centered and teacher-centeredpedagogy: teaching and learning as guided participation. PedagogHuman Sci. 2009;1(1):3-27.54. Feise R. Do multiple outcome measures require p-valueadjustment? BMC Med Res Methodol. 2002;2(1):8.55. ASCIA e-training courses, Australasian Soceity of ClinicalAllergy and Immunology, Sydney, NSW, AU. http://www.allergy.org.au. Accessed 12 November, 2013.

49



A Physical Assessment Skills Module on Vital Signs

Christine Leong, PharmD,a Christopher Louizos, BSc(Pharm),a Grace Frankel, PharmD,a

Sheila Ng, BSc(Pharm),a Harris Iacovides, PharmD, MSc,a Jamie Falk, PharmD,a

Drena Dunford, BSc(Pharm),a Kelly Brink, BSc(Pharm),a Nancy Kleiman, MBA,a

Christine Davis, PharmD,a and Robert Renaud, PhDa,b

a Faculty of Pharmacy, University of Manitoba, Winnipeg, Canadab Department of Educational Administration, Foundations & Psychology, Faculty of Education, University of Manitoba,Winnipeg, Canada

Submitted November 12, 2013; accepted January 31, 2014; published September 15, 2014.

Objectives. To implement and evaluate a physical assessment module for pharmacy students.Design. A physical assessment module focusing on vital signs was incorporated into the curriculum forthird-year pharmacy students. This module consisted of an online component, a practical skills work-shop, and a clinical practice site.Assessment. The mean score on the in-class quiz, which evaluated students’ knowledge of physicalassessment after completion of the online module, was 94%. During the practical skills laboratory, 48%of student-measured systolic blood pressure (BP) readings and 60% of student-measured diastolic BPreadings were within 5 mmHg of the machine reading. In the assessment of blood pressure technique, areasof difficulty included detection of Korotkoff sounds; steady deflation of cuff; and hand-eye coordination.Conclusion. Students more frequently underestimated systolic BP than the diastolic BP when com-pared to the automated machine readings. Findings from this study will be used to improve existingmodules and evaluation methods on the physical assessment of vital signs.

Keywords: blood pressure, physical assessment, pharmacy education, pharmacy students, vital signs

INTRODUCTIONThe application of physical assessment skills is rec-

ognized as an important part of providing pharmaceuticalcare.1-8 Advantages include being able to monitor andoptimize medications more effectively, screen patientsat risk for chronic disease states, promote better commu-nication among health care practitioners, and improve ouroverall understanding of patient care.6,7 As pharmacists’scope of practice continues to expand into more patient-centered roles, pharmacy education will require theincorporation of courses into the curriculum that will de-velop skills to fulfill such roles.

At present, most Canadian pharmacists have not re-ceived training in physical assessment skills. One surveyreported 82.4% of Canadian pharmacists never re-ceived any type of formal training in conducting physical

examinations.7 The Association of Faculties of Pharmacyof Canada and the American Association of Colleges ofPharmacy’s Center for the Advancement of PharmacyEducation have identified the performance and interpre-tation of physical assessment findings as an educationaloutcome for pharmacy graduates of entry-to-practicepharmacy programs.8,9 However, few reports have de-scribed strategies for implementing a physical assessmentprogram in pharmacy education.10-15 More specifically,many of the previous studies used surveys to assess stu-dent satisfaction with the implementation of a new pro-gram rather than to assess the impact of the program onstudent learning. Others have described the way in whicha physical assessment course is integratedwithin an exist-ing course. Only one study compared student-measuredblood pressure readings to machine-measured readings,and no studies have specifically reported on commonareas of difficulty in learning physical assessment skillsfor new learners in a skills laboratory environment. Asa result, the purpose of this studywas to implement a phys-ical assessment skills module on vital signs for third-yearpharmacy students enrolled in a Bachelor of Science in

Corresponding Author: Christine Leong, PharmD, Facultyof Pharmacy, Apotex Centre, 750 McDermot Avenue,University of Manitoba, Winnipeg, MB, R3E 0T5.Tel: 1-204-318-5276. Fax: 1-204-474-7617. E-mail: [email protected]

50


Pharmacy program at the University of Manitoba, and toevaluate student learning of physical assessment skills.

Specific objectives of the study were: to evaluatestudents’ objective knowledge of physical assessmentskills based on the information provided in the onlinemodule; to compare the difference in blood pressurevalues obtained between a manual aneroid sphygmoma-nometer and an automated blood pressure machine whenconducted by a third-year pharmacy student; and to iden-tify common areas of difficulty in developing skills inobtaining a blood pressure reading with a manual aneroidsphygmomanometer.

DESIGNAmodule on Skills in Physical Assessment focusing

on vital signs was integrated into the Pharmacy SkillsLaboratory 3 course for third-year pharmacy students(n548) in the 2013-2014 academic year. The coursewas designed to apply and develop skills related to phar-macy practice using a wide range of interactive and col-laborative learning strategies.

The physical assessment module consisted of 3 com-ponents: an online module, a practical skills workshop,and a clinical experiential practice site (at a periodontalclinic). The learning objectives for the module were asfollows: (1) to recognize the importance of developingskills in physical assessment; (2) to demonstrate how tomeasure blood pressure, pulse rate, respiratory rate, andbody temperature; (3) to explain and interpret findingsobtained from a physical assessment of vitals; and (4) toapply physical assessment skills on selected patients forthe purpose of evaluating and monitoring drug therapyresponse in a clinical setting. All content was developedby a registered clinical pharmacist with experience andformal training in physical assessment, with input onstrategies for providing formal training and evaluationof instructors and students provided by the director ofthe Clinical Learning and Simulation Facility (CLSF) atthe University of Manitoba. This study was exempted forfull review by the Institutional Review Board.

Online ModuleThe online module included 3 voice-over Power-

Point lectures (approximately 20 minutes each). Theonline lectures were hosted on a secured online portalcalled Desire2Learn. The 3 lectures included: Introduc-tion to Physical Assessment Techniques; Vital Signs:Blood Pressure; and Vital Signs: Pulse Rate, RespiratoryRate, Temperature. A 5-question multiple-choice quizfollowed each lecture. Students were required to receivea grade of 60% or greater on each quiz to receive a PASS

standing on the online module. A PASS standing is re-quired of students in order to participate in the practicalskills laboratory workshop and clinical practice site. Themultiple-choice options were randomized where appro-priate to prevent students from circulating the answers.Students also were directed to useful videos demonstrat-ing the measurement of blood pressure and an audio ofKorotkoff sounds.

After completion of the online module, pharmacystudents were required to take an in-class quiz based onthe content provided in the online module prior to partici-pating in the practical skills laboratory workshop. The in-class quiz was administered to evaluate the students’baseline knowledge of physical assessment skills priorto attending the practical skills workshop. The quiz con-sisted of 30 multiple-choice questions and 5 short-answerquestions. The multiple-choice section comprised ques-tions focused on the technical performance of vital signs(eg, steps on how to perform a blood pressure reading).The short-answer component focused on the clinical ap-plication of vital signs measurement (eg, identification ofmedical conditions that warrant immediate referral). Stu-dents had to receive a grade of 60% or greater to receivea PASS standing, and 5% of the grade contributed to theiroverall grade for the course.

Practical Skills Laboratory WorkshopStudents had the opportunity to practice themeasure-

ment of vital signs (blood pressure, pulse rate, respiratoryrate, and tympanic temperature) on a classmate duringa 3½-hour practical skills workshop. Students were di-vided into groups of 6, and further divided into pairs topractice the measurement of vital signs on their classmate(Appendix 1). The room was set up so that each pair hadtheir own blood pressure station. The room was alsoequipped with a simulator arm station and a thermometerstation. Eight instructors from the faculty of pharmacywere involved in the supervision and evaluation of phar-macy students during this workshop. Each instructor wasresponsible for the supervision of 1 group of 6 pharmacystudents, and each group carried out the activities of theworkshop in a tutorial room. These instructors attended 2training sessions led by the coordinator of the physicalassessment module to become familiar with the equip-ment and proficient in the techniques for measuring vitalsigns. The instructor’s primary role was to provide feed-back as students practiced the measurement of vitalsigns and to provide a final evaluation on blood pressuretechnique.

Students’ assignment to earn their class participationgrade for the course was to assess vital signs and docu-ment the readings obtained. Students performed 3manual

51


blood pressure readings using a manual aneroid sphyg-momanometer (Pocket Nurse Proshyg BP Cuff, Monaca,PA) and stethoscope (Littmann Classic II S.E., St. Paul,MN), followed by 1 blood pressure reading with an auto-mated blood pressure machine (American DiagnosticCorporation Semi-Automatic Adult Blood PressureMon-itor 6012, Hauppauge, NY). The average of the 3 manualblood pressure readings was calculated. The ADC Semi-Automatic Blood PressureMonitor has an accuracy of63mmHg for bloodpressure.15 Students also obtained a heartrate, respiratory rate, and tympanic temperature readingon their classmate.

During the workshop, a simulator arm (Nasco Life-form, Fort Atkinson, WI) was also present to allow stu-dents to practice obtaining accurate blood pressure andpulse readings. The simulator arm allowed the instructoror student to program a specific blood pressure and heartrate reading for the learner to practice on using a manualblood pressure cuff and stethoscope. Students who partic-ipated and completed the assignment received a PASS onthe assignment.

A final assessment of blood pressure on a simulatedpatient (classmate) was performed during the workshop.Students were given 15minutes to demonstrate their abil-ity to perform a blood pressure measurement based ona checklist derived from Bickley’s Bates’ Guide to Phys-ical Examination and History Taking.17 Evaluation ofstudents’ technical skill rather than the accuracy of theblood pressure values they obtained was the focus of thisassessment as additional practice would be necessary forsome students to become proficient. Students whomissedcritical steps in obtaining a blood pressure reading or whodemonstratedweaknesses in any of these areas (eg, steadycoordination of the control valve during cuff deflation)were advised to gain additional practice prior to theirPeriodontal Clinic exposure. A date was set for studentsto gain additional practice and to be reevaluated. Thesecond session involved practicing 3 to 4 times on differ-ent individuals (classmates and instructor).

EVALUATION AND ASSESSMENTThe mean score on the in-class quiz was 94%. The

mean score on the multiple-choice section of the test was97%, and the mean score on the short-answer componentwas 84.7%. The distribution of grades in the short-answercomponent is shown in Table 1.

Themean student-measured (using amanual aneroidsphygmomanometer) systolic blood pressure (SBP) was121.2612.4 mmHg (range 92.0 mmHg to 150.7 mmHg)and the mean student-measured diastolic BP was78.869.0 mmHg (range 60.0 mmHg to 98.7 mmHg). Themean systolic and diastolic BP measured by an automated

blood pressure cuff were 125.3613.7 mmHg (range 94.0mmHg to 153.0mmHg) and 77.769.3mmHg (range 52.0mmHg to 97.0 mmHg), respectively. Independent t testswere conducted to compare the mean student and auto-mated values for each of the systolic and diastolic read-ings. The mean student value did not differ significantlyfrom the mean automated value in both the systolic (BPt(94)51.55, p50.13) and diastolic (BP t(94)50.57,p50.57) readings.

Although the differences between the mean studentand automated values for systolic and diastolic BPs werenot significant, there were other differences worth notingrelated to the use of a manual vs an automated bloodpressure machine. The mean absolute difference betweenthe student-measured and automated blood pressure ma-chine for the systolic and diastolic BPs was 6.666.1mmHg (range 0.3 mmHg to 35.3 mmHg) and 5.063.8mmHg (range 0 mmHg to 14.0 mmHg), respectively.The manual aneroid blood pressure cuff operated by thestudents appeared to underestimate the systolic BP morefrequently than the diastolic BP when compared withreadings obtained using the automated blood pressuremachine (Table 2). Conversely, students tended to over-estimate diastolic BP more often than systolic BP. Figure 1shows the distribution of absolute differences in systolicand diastolic BPs in 5 mmHg increments starting from0 to 5mmHg to over 20mmHg.Most of the students wereable to achieve a manual reading within 5 mmHg of theautomated reading. However, the majority of the studentswho under- or overestimated the blood pressure readingsby greater than 5 mmHg, fell in the 5 mmHg to 10 mmHgrange. From a clinical standpoint, a blood pressure read-ing that is more than 5 mmHg either under or over a per-son’s true systolic or diastolic BP can increase thelikelihood of a misdiagnosis or an inaccurate measure-ment of medication efficacy.

DISCUSSIONDesigning a physical assessment course is a relatively

new and important area of interest to many educators in

Table 1. Distribution of Grades on the Short-AnswerComponent of In-Class Quiz (n548)

Letter Grade Students, No. (%)

A1 19 (39.6)A 21 (43.8)B1 2 (4.2)B 3 (6.3)C1 1 (2.1)C 2 (4.2)D 0F 0

52


pharmacy and other health care professions. In this study,students demonstrated baseline knowledge of physicalassessment through an in-class test prior to attending thepractical skills laboratoryworkshop. However, a pretest oncontent presented in an online module was not adminis-tered and therefore student learning as a result of the onlinemodule could not be evaluated.

The practical skills workshop revealed that newlearners of blood pressure assessment technique usinga manual cuff more frequently underestimated systolicBP. In contrast, student-measured diastolic BP moreclosely reflected the readings obtained from an automatedblood pressure machine. These observations might beexpected of new learners given their initial unfamiliaritywith Korotkoff sounds and skill in achieving a steady de-flation rate. However, it could be argued that the diastolicBP should bemore difficult to detect since it is potentiallymore difficult to determine the disappearance or mufflingsounds of the diastolic BP. McCall and colleaguesreported that among BP assessments taken by 83 second-year pharmacy students, 51% of systolic readings and47% of diastolic readings were within 5 units of the ma-chine reading,14 compared to 48% and 60%, respectively,in our study. McCall and colleagues noted final digit bias,inappropriate cuff size, and variability in deflation rate ascommon errors leading to an inaccurate diastolic BP read-ings. In our study, the accuracy of the blood pressurereadings could also have been influenced by inappropriate

cuff size (only standard and large cuff sizes were avail-able), ambient room noise during the exercises, and im-properly calibrated equipment (ie, new equipment nottested for accuracy). In addition, students commentedon how the release valves on the new equipment wereinitially difficult to adjust, which could have affected stu-dents’ ability to deflate the cuff at a steady rate.

The final assessment of students on their blood pres-sure technique using a manual cuff identified commonmajor andminor issues (Table 3).Major areas of difficultyrequired considerablymore practice tomaster blood pres-sure technique. Minor issues, on the other hand, wereeasily addressed and corrected on the subsequent trial.Identification of common major and minor areas of diffi-culty during the study allowed instructors to develop fu-ture strategies for teaching and evaluating new learners ofmanual blood pressure technique.

While the instructor could observe and evaluate tech-nique in measuring blood pressure, it was difficult for theinstructor to decipher whether the student heard and cor-rectly interpreted the Korotkoff sounds, which is neces-sary to accurately obtain the true blood pressure value ofan individual. Students who expressed the inability to de-tect the Korotkoff sounds were told to practice obtaininga reading on a simulator arm and on different classmates.The simulator arm can be programmed to assess the stu-dent’s ability to obtain an accurate blood pressure value.However, the Korotkoff beats produced by the simulator

Table 2. Proportion of Students Underestimating or Overestimating the SBP and DBP by Greater than 5 mmHg Using a ManualDevice in Comparison to the Automated Machine

SBP DBP

No. (%) Range (mmHg) No. (%) Range (mmHg)

Underestimating 20 (41.7) -5.3 to -35.3 8 (16.7) -6.7 to -13.0Overestimating 5 (10.4) 5.3 to 14.0 11 (29.9) 5.3 to 12.7

Figure 1. Absolute difference between student and machine measurements (N548).

53


arm were perceived by students and instructors as “obvi-ous” (ie, easy to identify) compared to those of a realpatient. Additional practice on different individualswould allow for more exposure and familiarity withKorotkoff sounds. In addition, teaching stethoscopes(with 2 sets of ear prongs) could provide a means forthe instructor to listen to the Korotkoff sounds with thestudent. In our experience, these stethoscopes were un-able to produce clearly audible sounds when comparedwith those produced by the Littmann Classic II S.E.stethoscopes. While having each pair of students performa blood pressure reading in a separate room to minimizeenvironmental noise is ideal, it is not always a practicalsolution for many program coordinators. However, a sep-arate room could be feasible if only used to conduct thefinal assessment of blood pressure technique. In the fu-ture, using standardized patients and comparing student-and clinician-measured blood pressure readings will beconsidered.

Despite these areas for improvement, there werea number of factors that contributed to the success ofthe physical assessment module. Having one instructorper group of 6 students appeared to be an appropriatestudent-to-instructor ratio for supervising and evaluatingblood pressure technique. In addition, providing immedi-ate feedback to the students and allocating 3½ hours forthe workshop allowed students with enough time to prac-tice and develop their technique.

SUMMARYA physical assessment skills module for third-year

pharmacy students at the University of Manitoba was suc-cessfully implemented and evaluated. Pharmacy studentknowledge of pharmacy assessment skills was evaluated,the values between student-measured and machine-measured blood pressure readings were compared, and

common areas of difficulty for new learners of bloodpressure measurement technique using a manual aneroidsphygmomanometer were identified. Identifying commonissues observed during the performance of blood pressureassessment and recognizing how student-measured read-ings compared to machine-measured readings allowed theinstructors to tailor strategies for improving existingmodels and evaluation methods for teaching physicalassessment skills. These findings have important clinicalimplications as they relate to teaching pharmacy studentsto identify accurately patientswith high blood pressure andto monitor effectively those on antihypertensive therapy.

ACKNOWLEDGMENTSThe authors acknowledge the Faculty of Medicine

Director of the CLSF at the University of Manitoba forhis consultation on strategies for providing formal train-ing and evaluation of instructors and students. Theauthors thank Cheryl Kristjanson for her consultation onprogram evaluation and review of the manuscript.

REFERENCES1. Rospond RM, Tice A, Tice B. Physical assessment for thecommunity pharmacist, part 2. America’s Pharm. 1999;121:47-53.2. Pauley T, Marcrom R, Randolph R. Physical assessment in thecommunity pharmacy. America’s Pharm 1995;NS35(5):40-9.3. Da Camara C, D’Elia R, Swanson L. Survey of physicalassessment course offerings in American colleges of pharmacy. Am JPharm Educ. 1996;60(4):343-347.4. Spray JW, Parnapy SA. Teaching patient assessment skills todoctor of pharmacy students: the TOPAS study. Am J Pharm Educ.2007;(4):Article 64.5. Longe RL. Teaching physical assessment to doctor of pharmacystudents. Am J Pharm Educ. 1995;59(2):151-155.6. Simpson SH, Wilson B. Should pharmacists perform physicalassessments? Can J Hosp Pharm. 2007;60(4):271-272.7. Barry AR, McCarthy L, Nelson CL, Pearson GJ. An evaluation ofteaching physical examination to pharmacists. Can Pharm J (Ott).2012;145(4):174-180.8. Association of Faculties of Pharmacy of Canada. Educationaloutcomes for first professional degree programs in pharmacy (Entry-to-Practice Pharmacy Programs) in Canada. Vancouver (BC):Association of Faculties of Pharmacy of Canada; 2010. https://www.afpc.info/sites/default/files/AFPC%20Educational%20Outcomes.pdf. Accessed September 3, 2014.9. Medina MS, Plaza CM, Stowe CD, et al. Center for theAdvancement of Pharmacy Education educational outcomes 2013.Am J Pharm Educ. 2013; in press.10. Bolesta S, Trombetta DP, Longyhore DS. Pharmacist instructionof physical assessment for pharmacy students. Am J Pharm Educ.2011;75(2):Article 29.11. Albano CB, Brown W. Integration of physical assessment withina pathophysiology course for pharmacy. Am J Pharm Educ. 2012;76(1):Article 14.12. Sherman JJ, Riche DM, Stover KR. Physical assessmentexperience in a problem-based learning course. Am J Pharm Educ.2011;75(8):Article 156.

Table 3. Major and Minor Areas of Difficulty Experienced bythe Student While Learning Blood Pressure Technique

Major Areas of DifficultyKorotkoff sounds could not be heardCuff could not be deflated at a steady rate of 2 to3 mmHg per second

Overall hand-eye coordination in operating themanual device was poor

Minor Areas of DifficultyPatient was not properly positionedStethoscope was donned incorrectlyStethoscope was not “turned on”Stethoscope was not placed correctly over thebrachial artery

Cuff was not deflated completely after estimating the SBP

54


13. Elliott KE, McCall KL, Fike DS, Polk J, Raehl C. Assessment ofmanual blood pressure and heart rate measurement skills of pharmacystudents: a follow-up investigation. Am J Pharm Educ. 2008;72(3):Article 60.14. McCall KL, Raehl C, Nelson S, Haase K, Fike DS. Evaluation ofpharmacy students’ blood pressure and heart rate measurement skillsafter completion of a patient assessment course. Am J Pharm Educ.2007;71(1):Article 1.

15. Grice GR, Wenger P, Brooks N, Berry TM. Comparison ofpatient simulation methods used in a physical assessment course.Am J Pharm Educ. 2013;77(4):Article77.16. American Diagnostic Corporation (ADC). Semi-AutomaticBlood Pressure Monitor 6012 Instruction Manual. ADC. Hauppauge,New York.17. Bickley LS. Bates’ Guide to Physical Examination and HistoryTaking. 9th ed. Philadelphia, PA: Lippincott Williams &Wilkins; 2007.

55


Appendix 1. Physical Assessment of Vitals Laboratory Agenda

Instructions: By the end of the Practical Skills Workshop, students are expected to:1. Complete and submit the Physical Assessment of Vitals Laboratory Assignment2. Demonstrate the ability to perform a blood pressure measurement on a simulator arminstructor (Final Assessment)

Each pair of students should have:(1) ONE stethoscope (unless student has brought his/her own)(2) ONE manual aneroid sphygmomanometer(3) ONE automated blood pressure monitor(4) Alcohol swabs

Each room will have:(1) ONE simulator arm(2) ONE teaching stethoscope(3) ONE measuring tape(4) ONE tympanic thermometer

Expected timeframe to complete each component of the workshop:

Component Expected time to complete

Blood pressure reading with an aneroid sphygmomanometer 5 to 10 minutesBlood pressure reading with an automated BP machine 1 to 2 minutesHeart rate or respiratory rate 30 seconds to 1 minuteTympanic temperature 1 to 2 seconds

56



Student Self-Screening for Methicillin-Resistant Staphylococcus Aureus(MRSA) Nasal Colonization in Hand Hygiene Education

Tia Lum, PharmDa Kristin Picardo, PhD,b Theresa Westbay, PhD,b Amber Barnello, BS,b

Lynn Fine, PhD,c Jill Lavigne, PhD, MPHa

a Wegmans School of Pharmacy, St John Fisher College, Rochester, New Yorkb Department of Biology, St. John Fisher College, Rochester, New Yorkc University of Rochester Medical Center, Rochester, New York


Objective. To determine the feasibility and effectiveness of adding a hand hygiene exercise in self-screening for Methicillin-Resistant Staphylococcus Aureus (MRSA) nasal colonization to a health caredelivery course for first-year pharmacy (P1) students.Design. About one month after students were trained in hand hygiene technique and indications,faculty members demonstrated how to self-screen for MRSA nasal colonization. Students were thenasked to screen themselves during the required class time. Aggregated class results were shared andcompared to prevalence estimates for the general population and health care providers.Assessment. The 71 students present in class on the day of the self-screening exercise chose toparticipate. A survey comparing presecreening and postscreening responses indicated incrementalimprovements in student knowledge and awareness of health care associated infections and motivationto perform hand hygiene. On the written exam, student performance demonstrated improved knowl-edge compared to previous class years.Conclusion. Self-screening for MRSA nasal colonization in a health care delivery course for P1students increased students’ motivation to perform hand hygiene techniques and follow indicationspromulgated by the World Health Organization.

Keywords: hand hygiene, health care associated infection, prevention, MRSA, screening, colonization

INTRODUCTIONMethicillin-resistant Staphylococcus aureus (MRSA)

is a common cause of health care associated infections, andthe incidence of community-acquired infections has in-creased in recent years.1,2 Of the general population, about1.5% are asymptomatic nasal carriers of MRSA, and theprevalence among health care workers is about 3 times ashigh (4.6%).3-6 The anterior nares serve as the mainMRSAreservoir, but transient hand carriage and subsequent trans-mission is possible. Implementation of proper hand hygienetechnique is effective in preventing person-to-person trans-mission ofMRSA, and such programs have been associatedwith reduced prevalence of the infection.8-10 Despite thehigher prevalence of the bacteria among health careworkers, health care providers are rarely required to undergoscreening forMRSAnasal colonization. Studies of the prev-alence of MRSA carriage among health care professional

students have not tested the feasibility of student screeningor its impact on student knowledge and attitudes.11

In 2011, faculty members at theWegmans School ofPharmacy began training all newly matriculated P1 stu-dents to follow the World Health Organization’s 8-stepmethod for hand hygiene using alcohol-based products.11

Students were successful in mastering the skill for handhygiene and reported greater motivation to perform handhygiene after training.12

To make students more aware of the likelihood ofMRSA exposure within their own classroom, we intro-duced a new active learning exercise in fall 2012: theMRSA nasal colonization self-screening. The purposeof this study was to determine the feasibility of havingstudents participate in this invasive exercise and to assessthe effects on student attitudes toward, knowledge of, andmotivation to perform hand hygiene.

DESIGNNewly matriculated doctor of pharmacy (PharmD)

students received hand hygiene instruction within the

Corresponding Author: Jill Lavigne, Wegmans School ofPharmacy, St. John Fisher College, Rochester, New York14618. Tel: 585 385 5255. E-mail: [email protected]

57


context of a required 3-credit hour, semester-long lecturecourse titled Health Care Delivery. Hand hygiene instruc-tion was a natural companion to the other introductorycontent presented in the course, which spanned what itmeans to be a health care professional, the various healthcare professions, health care settings, public health, theTriple Aim (lower cost, better health care quality, andimproved patient outcomes), and interdisciplinary, patient-centered care. By providing this instruction at the start ofthe first semester of the P1 year and following it withMRSA self-screening one month later, we hoped to pro-vide students with a clinical skill that they could useimmediately and which, if performed regularly, couldprotect them throughout their experiential education. Thisstudy was reviewed and approved by the school’s Institu-tional Review Board (IRB).

Approximately onemonth after the P1 students com-pleted their hand hygiene examination and skills assess-ment (evaluated using a rubric), we introduced theMRSAself-screening exercise during class. The flora samplingand reporting ofMRSAnasal colonization incidenceweredesigned to actively engage students in inquiry, reinforc-ing their interest and helping them make connections tohand hygiene. Thus, pedagogical components of this ex-ercise (ie, teaching technique, providing evidence to sup-port its importance) engaged students in an investigationthat underscored the importance of hand hygiene.

Two faculty members from the biology departmentbegan the class by briefly reviewingMRSAmicrobiologyand the history of contagion. This introduction consistedof a review of the importance of hand hygiene based on an18th century case and a condensed tutorial on MRSAmicrobiology. Historical information regarding the his-tory of hand washing and data demonstrating how handwashing improvedmortality rates in clinical settingswerepresented to capture the students’ interest. The courseinstructor then reviewed the prevalence of health care-associated and community-acquired MRSA and intro-duced the exercise as a way to better understand the florapresent in the classroom. At our institution, P1 studentsremain in the same classroom through the school yearwith instructors circulating through the room. Therefore,the flora of the classroommaymore closely resemble thatof the P1 students than classrooms in which many differ-ent students spend considerably less time.

Techniques to properly gather anterior nasal speci-mens using a sterile, calcium alginate swab moistenedwith sterile saline were then described and demonstratedby the biology faculty member and course instructor, whocollected their own samples in front of the class. Thefaculty members drew a specimen from each naris andtransferred the first sample to aBBLCHROMagarMRSA

II screening agar plate (Becton, Dickinson and Company,Sparks, MD), and the second sample to a mannitol saltagar (Becton, Dickinson and Company, Sparks, MD)plate. Plates were marked with a unique 6-digit identifi-cation number, which each participant was to record andretain to ensure anonymity. No personal or other identi-fying information was attached to the plates. Facultymembers then distributed sample collection supplies toeach student and instructed the class to self-collect spec-imens. The supplies were acquired by the Department ofBiology for $15.44 per person. The introduction, demon-stration, and student sampling took about 20 minutes.Immediately following specimen collection, plates weretransferred to the laboratory and incubated accordingto manufacturer specifications for the recommended24-hour time period at 37˚C. After 24 hours, plates wereevaluated for growth. Any CHROMagar MRSA II platesthat had colony growth as indicated by a mauve-violetcolor were identified as preliminary positive results sug-gestive of MRSA presence. AnyMSA plates that demon-strated a color change frombaseline pink to bright yellow,which is indicative of a Staphylococcus species capable offermenting mannitol, were identified as positive for path-ogenic Staphylococcus growth. Each set of positive re-sults was cross-referenced according to the unique 6-digitidentifier on the plates. If a participant’s samples corre-sponded to positive results on both CHROMagar MRSAII and mannitol salt agar plates, further testing was per-formed. First, an isolated colony from the mannitol saltagar plate was extracted and placed on a BBL TrypticaseSoyAgarwith 5%SheepBlood (TSA II; Becton, DickinsonandCompany,Sparks,MD)plate according tomanufacturerrecommendations for a 24-hour incubation period at 37˚C.Additionally, colony growth on TSA II plates exhibitingbeta hemolysis after incubation was sampled for Gramstaining. Results of the specimen plating exercise are rep-resented in Figure 1.

Approximately 2 weeks later, culture results (identifiedonly by 6-digit plate numbers) were posted to Blackboardwhere students couldview their results and thoseof the rest ofthe class. Although MRSA nasal colonization is not consid-ered a condition requiring treatment, we recommended thatstudentswith apositive result see their health care provider orvisit the campus wellness center. Health care providers havethe option of repeating the test and treating any colonizationwith a regimen of mupirocin lotion applied topically by pre-scription, for example. During the next class period, we cal-culated the prevalence of MRSA nasal colonization in theclass and compared it to the prevalence among the generalpopulation and among health care providers.

Two weeks after completing the screening exercise,after the MRSA screening results were shared with the

58


class, students were asked to complete an anonymoussurvey during class to assess their knowledge, attitudes,and motivation regarding hand hygiene before and aftercompleting theMRSA instruction. Tomeasure incremen-tal improvement in motivation and attitudes to performhand hygiene after the MRSA exercise, many of the samesurvey items that we had used earlier in the semester toassess student knowledge of hand hygiene techniquewereused on the survey instrument. The survey design wasbased on our previously published work16,17 It included17 questions with responses measured on a 4-point Likertscale (15weak or disagree, 25fair or somewhat disagree,35good or somewhat agree, 45very good or agree) andeach item had a prescreening and postscreening column inwhich to record their score. Participation in the surveywas anonymous and voluntary. Analysis of students’ re-

sponses for prescreening vs postscreening was performedusing the Wilcoxon signed ranks test. All analyses wereconducted in SPSS, version 20, and repeated in STATA,version 10 (College Station, Texas).

After completing the self-screening, reviewing theresults, and calculating and comparing MRSA nasal col-onization prevalence, students were expected to be able tomeet several learning objectives incorporating the knowl-edge, application, and synthesis domains of Bloom’s Tax-onomy and the human dimension, application, integration,and foundational knowledge domains of Fink’s Taxon-omy (Table 1). For example, collection of samples (ap-plication skill) allowed for first-hand experience of thenasal specimen collection process and for observationof how the students, their classmates, and instructorsreacted to the experience (human dimension [learning

Figure 1. Schematic representation of the self-screening process and data collection and analysis protocols.

59


about oneself and others]). Through the sharing of anon-ymous results, students gained foundational knowledgeabout their personal flora and that of their classmates(human dimension). Students had the opportunity to syn-thesize information and experience from this and previ-ous hand hygiene curriculum components to furtherenhance their knowledge and awareness of the impor-tance of hand hygiene, as well as their motivation to per-form proper hand hygiene.

EVALUATION AND ASSESSMENTMatriculating P1 students in Fall 2012 were primar-

ilyCaucasian and between the ages of 18 and 24 (Table 2).Of the 80 students in the P1 class, 71 were present in classon the day of the self-screening exercise and all partici-pated in the exercise. Of the 71 sets of plates, all samplesyielded colony growth. Three (4.2%) plates screened pos-itive for MRSA colonization. The Fisher exact test wasused to determine if there was an association between theclassroom prevalence and the general population preva-lence. The difference between the groups was not signif-icant (2-tailed, p50.65), suggesting that the newlymatriculating P1 class prevalence resembled that of thegeneral population.

Twenty-six of the 71 students who had completedthe exercise completed the survey instrument. Students’scores on knowledge, awareness, and motivation to per-formhand hygiene significantly improved following com-pletion of the MRSA self-screening exercise (Table 3).Notably, 24 (92.3%) students responded positively to the

statement: “Screening for MRSA colonization is an im-portant exercise for pharmacy students.” All participantsresponded positively to the following statements: “Learn-ing about hand hygiene with alcohol-based preparationswill help me to be a better pharmacist” and “This classprovided information that I can apply in practice.”

Class performance questions on hand hygiene andhealth care-associated infections on a written examina-tion (worth 33% of the course grade) was compared toperformance of the P1 class in the previous year, whichhad received the hand hygiene instruction but had notcompleted the MRSA nasal colonization self-screeningexercise. For the test question, “Health care associated

Table 1. Cognitive Domains of Learning Objectives According to Taxonomies of Learning for a Class Exercise on Self-Screeningfor Methicillin-Resistant Staphylococcus Aureus

Learning ObjectiveCognitive Domain inBloom’s Taxonomy14

Cognitive Domain inFink’s Taxonomy15

Successfully obtain a viable sample of nasalmucosa for testing.

Application Human DimensionApplication

Express an improvement in knowledge of theimportance of hand hygiene to patient healthand pharmacy practice.

Application IntegrationHuman DimensionFoundationalKnowledge

Express increased awareness of the impact ofhand hygiene on patient and provider health.

Application IntegrationHuman DimensionFoundationalKnowledge

Express the belief that self-screening for MRSAcolonization is an important exercise forpharmacy students.

Knowledge Human DimensionApplication

Express an improvement in motivation toperform hand hygiene as indicated.

Synthesis IntegrationHuman DimensionFoundational Knowledge

Table 2. Demographics of First-Year Pharmacy Class WhoParticipated in an Exercise on Self-Screening for Methicillin-Resistant Staphylococcus Aureus (N580)

Variable No. (%)

Male 35 (43.8)Female 45 (56.3)Age (years)18-24 60 (75.0)25-34 19 (23.8)35-44 1 (1.3)

RaceCaucasian 58 (72.5)African American 5 (6.3)Asian 6 (7.5)Asian Indian 1 (1.3)No response 10 (12.5)

60


infections are a rare cause of death, true or false?” thenumber of students selecting the correct answer wassignificantly higher in the class that had completed theMRSA self-screening exercise (98.75% compared to88.3% of the previous class with p,0.01). (The p-valueassociated with the chi-square test for a difference in pro-portions is p,0.01.) Student performance on test itemsspecific to hand hygiene indications and technique werenot significantly better than those of the previous class.

DISCUSSIONWe assessed 3 broad questions in this study: (1) Is

MRSA nasal colonization self-screening feasible and ac-ceptable to students in a required course completed priorto clinical training? (2) Do students express an increasedknowledge and awareness of health care-associated in-fections and the importance of hand hygiene after self-screening than after hand hygiene instruction alone? (3)Was the class performance on test questions related to

Table 3. First-Year Pharmacy Students’ Responses on a Pre/Post Screening Survey Regarding Hand Hygiene and Health CareAssociated Infections after Participating in a Self-screening Exercise to Detect Methicillin-Resistant Staphylococcus Aureus(N526)

Survey Item

PositiveResponsesa

No. (%)

DifferencePost/Pre ScreeningScores, Mean (SD) Pb

My ability to identify indications for hand hygiene (thatis, when to do hand hygiene).

26 (100) 1.0 (0.7) , 0.001

My knowledge of how to do hand hygiene usingalcohol based solutions.

25 (96.2) 1.0 (1.0) , 0.001

My ability to correctly perform hand hygiene with analcohol based solution.

25 (96.2) 1 (0.96) , 0.001

My motivation to perform hand hygiene with an alcoholbased solution as indicated.

26 (100) 0.92 (0.80) , 0.001

My awareness of the impact of hand hygiene on patienthealth.

26 (100) 0.88 (0.82) , 0.001

My awareness of the impact of hand hygiene on thehealth of health care providers.

26 (100) 0.85 (0.88) , 0.001

My knowledge of why hand hygiene with alcohol basedsolutions is important in pharmacy practice.

26 (100) 0.88 (0.95) , 0.001

My ability to perform hand hygiene as indicated whilepracticing pharmacy.

25 (96.2) 0.96 (1.03) , 0.001

My motivation to perform hand hygiene as indicatedwhile working in a pharmacy setting.

26 (100) 0.92 (0.93) , 0.001

My awareness of the impact of hand hygiene inpharmacy practice on patient health.

26 (100) 0.96 (0.96) , 0.001

Screening for MRSA colonization is an importantexercise for pharmacy students.

24 (92.3) 0.54 (0.86) , 0.05

It is important for pharmacy students to learn properhand hygiene technique using alcohol basedpreparations.

26 (100) 0.58 (0.90) , 0.05

Failure to perform hand hygiene is a major issue inpatient care.

26 (100) 0.42 (0.76) , 0.05

Expert hand hygiene knowledge and skills will helpme be a better pharmacist.

25 (100) 0.40 (0.71) , 0.05

Failure to perform hand hygiene with alcohol basedpreparations as indicated is a major issue inpharmacy practice.

24 (92.3) 0.62 (0.75) , 0.001

Learning about hand hygiene with alcohol basedpreparations will help me be a better pharmacist.

26 (100) 0.50 (0.76) , 0.05

This class provided information that I can apply inpractice.

26 (100) 0.50 (0.71) , 0.001

a Responses were based on a Likert scale ( 15Weak, 25Fair, 35Good, 45Very Good) on which ratings of 3 or 4 were considered positiveresponses.b As determined by Wilcoxon signed rank test.

61


health care-associated infections and hand hygiene betterthan that of the previous year’s class, which received thesame instructions on hand hygiene technique and indica-tions, but did not participate in the self-screening exercise?

The feasibility of adding the self-screening exercisewas determined by the number of students who success-fully obtained nasal samples by inserting a cotton swab tothe very back of each side of their noses and swiping eachswab onto a culture plate, one at a time. We anticipatedthat some students would feel uncomfortable stickinga swab up their nose, particularly in front of classmates.Others might not have wanted their samples tested andreported, even though samples were anonymous. Never-theless, all 71 students (100%) present in class completedthe exercise and every set of plates returned a viable re-sult. In a relatively full classroom with faculty membersparticipating, it may have been difficult for students toresist participation, particularly at this early stage in theirtraining.

We concluded that self-screening for MRSA nasalcolonization was feasible in this class of P1 students. Thecultures demonstrated a prevalence of MRSA nasal col-onization among the students in the class that was similarto that in the general population, and the pre/post screen-ing survey responses suggested that this exercise resultedin achievement of learning objectives.

A limitation of the studywas the relatively low surveyresponse rate. Factors that may have contributed to the lowresponse rate may include lower class attendance on theday the survey was administered and student fatigue. Thepre/post screening survey used after the self-screening forMRSA nasal colonization was nearly identical to the pre/post survey students completed at the end of the hand rub-bing skills training with alcohol based preparations. Theinstructor explained the purpose of the second survey, butfatigue, lack of time between classes, and competing de-mands likely had a negative effect on the response rate.

We expected to see modest improvements in stu-dents’ rating of incremental learning after completingthe MRSA self-screening exercise but did not. Neverthe-less, students reported significant improvements across allparameters, including improvements in their knowledgeof hand hygiene even though the MRSA self-screeningexercise did not address hand hygiene technique. Studentsmay have assumed that self-screening for MRSA nasalcolonization was a component of hand hygiene programs.Alternatively, students may have responded positivelyoverall to the exercise and, therefore, tended to rate allof the survey items positively. Class performance on a testquestion related to health care-associated infections wassignificantly better than that of the previous class (whichhad received the same hand hygiene training but not the

self-screening exercise), but scores on test questions re-lated to hand hygiene were not higher than those of theprevious class.

SUMMARYSelf-screening forMRSAnasal colonization in a first

semester lecture course for P1 students appears to bea feasible method for reinforcing motivation to performhand hygiene technique as promulgated by the WorldHealth Organization. Future studies should examine theeffects of such active learning on student hand hygienebehaviors and on retention of learned practice upon en-tering the health care workforce.

REFERENCES1. Chambers HF, DeLeo FR. Waves of resistance: Staphylococcusaureus in the antibiotic era. Nat Rev Microbiol. 2009;7(9):629-641.2. Methicillin-resistant Staphylococcus aureus (MRSA) Infections.Centers for Disease Control and Prevention. http://www.cdc.gov/mrsa/health care/index.html. Accessed February 19, 2014.3. Albrich WC, Harbarth S. Health-care workers: source, vector, orvictim of MRSA? Lancet Infect Dis. 2008;8(5):289-301.4. Jernigan J, Kallen A. Methicillin-resistant Staphylococcus aureus(MRSA) infections. Activity C: ELC prevention collaboratives.Division of health care quality promotion. Centers for disease controland prevention. http://www.cdc.gov/HAI/pdfs/toolkits/MRSA_toolkit_white_020910_v2.pdf. Accessed March 1 2013.5. Elie-Turenne MC, Fernandes H, Mediavilla JR, et al. Prevalenceand characteristics of Staphylococcus aureus colonization amonghealth care professionals in an urban teaching hospital. Infect ControlHosp Epidemiol. 2010;31(6):574-580.6. Gorwitz RJ, Kruszon-Moran D, McAllister SK, et al. Changes inthe prevalence of nasal colonization with Staphylococcus aureus inthe United States, 2001-2004. J Infect Dis. 2008;197(9):1226-1234.7. Pittet D, Hugonnet S, Harbarth S, et al. Effectiveness of a hospital-wide programme to improve compliance with hand hygiene. Lancet.2000;356(9238):1307-1312.8. Grayson ML, Russo PL, Cruickshank M, et al. Outcomes from thefirst 2 years of the Australian National Hand Hygiene Initiative. MedJ Aust. 2011;195(10):615-619.9. Sroka S, Gastmeier P, Meyer E. Impact of alcohol hand-rub use onmethicillin-resistant Staphylococcus aureus: an analysis of theliterature. J Host Infect. 2010;74(3):204-211.10. WHO guidelines on hand hygiene in health care. Geneva: WorldHealth Organization. 2009. http://whqlibdoc.who.int/publications/2009/9789241597906_eng.pdf11. Rohde RE, Rowder C, Patterson T, Redwine G, Vasquez B,Carranco E. Methicillin resistant Staphylococcus aureus (MRSA): aninterim report of carriage and conversion rates in nursing students.Clin Lab Sci. 2012;25(2):94-101.12. Lavigne JE. Schwartzmeyer C. Implementing World HealthOrganization standards for hand hygiene indications and methods usingalcohol-based preparations at the Wegmans School of Pharmacy.International Forum on Quality & Safety in Health care. Paris, France.April, 2012. http://fisherpub.sjfc.edu/cgi/viewcontent.cgi?article51015&context5pharmacy_facpub&sei-redir51&referer5http%3A%2F%2Fwww.bing.com%2Fsearch%3Fq%3Dlavigne%2Bhand%2Bhygiene%2B2012%26qs%3Dn%26form%3DQBRE%26pq%

62


3Dlavigne%2Bhand%2Bhygiene%2B2012%26sc%3D0-21%26sp%3D-1%26sk%3D%26cvid%3Defee68187d3d42869ac6fb3cf09a78d2#search5%22lavigne%20hand%20hygiene%202012%2213. Paule SM, Mehta M, Hacek DM, et al. Chromogenic media vsreal-time PCR for nasal surveillance of methicillin-resistantStaphylococcus aureus: impact on detection of MRSA-positivepersons. Am J Clin Pathol. 2009;131(4):532-539.14. Bloom BS, ed. Taxonomy of Educational Objectives. TheClassification of Educational Goals. Handbook I: Cognitive Domain.New York, NY: McKay, 1956.

15. Fink LD. Creating Significant Learning Experiences. AnIntegrated Approach to Designing College Courses. San Francisco,CA: Jossey-Bass, 2003.16. Gilligan AM, Myers J, Nash JD, et al. Educating pharmacystudents to improve quality (EPIQ) in colleges and schools ofpharmacy. Am J Pharm Educ. 2012;76(6):Article 109.17. Lavigne J. Implementing “Educating Pharmacy Students andPharmacists to Improve Quality” (EPIC) as a requirement at theWegmans School of Pharmacy. Curr Pharm Teach Learn. 2012;4(3):212-216.

63



Using Curriculum Mapping to Engage Faculty Members in the Analysisof a Pharmacy Program

Sheryl Zelenitsky, PharmD,a Lavern Vercaigne, PharmD,a Neal M. Davies, PhD,a Christine Davis,PharmD,a Robert Renaud, PhD,b Cheryl Kristjanson, PhDa

a Faculty of Pharmacy, University of Manitoba, Winnipeg, Manitoba, Canadab Faculty of Education, University of Manitoba, Winnipeg, Manitoba, Canada

Submitted January 31, 2014; accepted April 9, 2014; published September 15, 2014.

Objective. To develop a curriculum mapping process that supports continuous analysis and evidence-based decisions in a pharmacy program.Design. A curriculum map based on the national educational outcomes for pharmacy programs wascreated using conceptual frameworks grounded in cognitive learning and skill acquisition.Assessment. The curriculum map was used to align the intended curriculum with the national educa-tional outcomes and licensing examination blueprint. The leveling and sequencing of content showedlongitudinal progression of student learning and performance. There was good concordance betweenthe intended and learned curricula as validated by survey responses from employers and graduatingstudents.Conclusion. The curriculum mapping process was efficient and effective in providing an evidence-based approach to the continuous quality improvement of a pharmacy program.

Keywords: curriculum mapping; learning objectives; educational outcomes; learning outcomes; competencies

INTRODUCTIONAs professional education programs, pharmacy cur-

ricula are subject to comprehensive reviews as requiredby internal or external (eg, accreditation) processes. Giventhe specialized knowledge and skills required for phar-macy practice, it is important to demonstrate that thecontent and structure of a curriculum are appropriate.The essential educational outcomes for pharmacy pro-grams in Canada are outlined by The Association ofFaculties of Pharmacy of Canada (AFPC) as: care pro-vider, communicator, collaborator, manager, advocate,scholar, and professional.1 Each AFPC educational out-come is composed of elements and subelements such as:“1.2 Elicit and complete an assessment of required in-formation to determine the patient’s medication-relatedand other relevant health needs.” The essential educa-tional outcomes for pharmacy programs in the UnitedStates are described by the Center for the Advancementof Pharmacy Education (CAPE).2 Educational outcomesguide pharmacy curricula in providing the learningopportunities and setting the expectations that preparestudents for entry to practice. This is referred to as the

intended curriculum. However, the question remains ofhow to assess whether students have been given ade-quate learning opportunities and achieved the appropri-ate level of expertise. Faculty members at the Universityof Manitoba developed an effective process for planningand developing a pharmacy curriculum map that can beused to analyze how well an intended curriculum alignswith essential educational outcomes, and how effec-tively it is delivered or learned by students. The valueof curriculum mapping in engaging faculty members inthe continuous analysis and evidence-based decisions ina pharmacy program is also demonstrated.

Curriculummapping is generally used to evaluate anexisting or to create a new curriculum.3 It documentswhatis taught (learning objectives and course content), how itis taught (teaching methods and learning opportunities),when it is taught (timetabling and sequencing), and whatis learned (learning outcomes and student assessment). Italso demonstrates the connections among the components(eg, courses, discipline streams, years) of a program. Cur-riculum mapping can be used to demonstrate alignmentswith essential educational outcomes or accreditation stan-dards.4 Specifically, it can identify gaps, redundancies, orinconsistencies in content, learning opportunities, or stu-dent assessments.4-6 It can also guide and support facultymembers in making informed decisions regarding the

Corresponding author: Sheryl Zelenitsky, Faculty ofPharmacy, University of Manitoba 750 McDermot Avenue,Winnipeg, Manitoba, Canada R3E 0T5. Tel: 204-474-8414.Fax: 204-474-7617. E-mail: [email protected]

64


required course or curricular change.7 Finally, curriculummapping can be used to analyze alignments between anintended and “learned” curriculum.4,8

As described by Harden, curriculum mapping canhave numerous functions.9 For teaching facultymembers,a curriculummap can demonstrate how their course relatesto other courses and the overall curriculum. It can assist incourse design anddecisions related to content and sequenc-ing, teachingmethods, and student assessment. For admin-istrators, a curriculummap can informprogramplanning inareas related to course evaluation, faculty development,teaching assignments, and resource allocation. Finally,for program evaluation, a curriculum map can providestructure for informative inquiry, analysis, and continuousquality improvement.3,4,9 It can measure how well theintended curriculum aligns with the learned one, and byextension, its impact on student learning and performance,and ultimately, their professional practice.4,8,10

Curriculum mapping can be a complex process. Ef-fective collaboration is essential, with pharmacy expertsproviding the content and context and education expertsguiding and validating the process. Faculty engagementencourages participation and reduces concerns related tocourse ownership, teaching evaluations, or potential re-quirements for changes.3 Curriculum mapping needs tobe practical and productive, with data collection that ispurposeful and reliable. The end product needs to be in-formative and functional in a way that maintains facultyinterest and ownership of the process.11 Finally, the pro-cess requires support from school administration, effec-tive leadership, and adequate resources.12

The primary goal of this study was to implementan inclusive and comprehensive curriculum mappingprocess that supported continuous analysis and evi-dence-based decisions in a pharmacy program. The keyobjectives were: to (1) determine to what degree ourintended curriculum covered each of the AFPC educa-tional outcomes, and (2) how our intended curriculumaligned with what students actually learned.

DESIGNCurriculummappingwas initiated by theCurriculum

Management Committee at the University of Manitoba’sFaculty of Pharmacy as an ongoing process to facilitatecurriculum documentation, analysis, and continuous qual-ity improvement. The school’s 4-year degree program wascomposed of 36 courses (140 credit hours) categorized intogeneral discipline streams of: pharmaceutical sciences,clinical and applied sciences, and performance-basedcourses and pharmacy practice (ie, experiential). Keythreats to successful curriculum mapping were identifiedthrough consultations with local and national experts as:

(1) adopting an overly theoretical approach that does notengage and maintain faculty support; (2) collecting inap-propriate, inconsistent, or excessive data; and (3) using pro-cesses without sufficient focus on results and action.

The Curriculum Management Committee partneredwith University Teaching Services (later renamed theCentre for the Advancement of Teaching and Learning),which provided leadership and support for individualsand programs at the University of Manitoba. The 1-yearprocess began with initial faculty discussions followed bya series of seminars on the purpose, components, con-struction, and design of curriculum maps. Faculty mem-bers agreed on an approach based on mapping courseobjectives to the AFPC educational outcome elements.Faculty development sessions were conducted to reviewthe key components, appropriate language, and effectiveformat for course objectives. Further instruction was pro-vided on linking course objectives to the relevant AFPCeducational outcome element(s).

The curriculum map was developed using concep-tual frameworks grounded in the theories of cognitivelearning and skill acquisition. The former was incorpo-rated using an educational framework called ICE (ideas,connections, and extensions) that defined levels of learningas: ideas (student has basic or fundamental knowledge),connections (student uses basic concepts to make connec-tions or form relationships), extensions (student synthe-sizes knowledge and uses concepts to extrapolate andmake informed decisions).13 As described by the devel-opers, “the framework. . . helps to clarify the characteristicsand markers that indicate where learners are along thelearning continuum and in doing so, enables teachers tomake instructional decisions that maximize learning.”13

Teaching faculty members used the ICE framework toassign their expectation of student learning for each AFPCeducational outcome element linked to a course objective.

Next, the process of acquiring skills and professionalattitudes was captured in a framework on the developmentof expertise. Early research by Dreyfus and Dreyfus de-scribed the acquisition of skill through a series of stages,14

whereas later work by Ericsson stressed the importance ofdeliberate practice in the process.15 Similar to the ICE levelsfor student learning, we developed levels for student per-formance based on an expertise framework of: novice(student has cognitive abilities limited to facts or doesnot connect knowledge with pharmacy practice; studentrequires extensive coaching or supervision in pharmacypractice or simulation environments), functional (studentdemonstrates ability to connect knowledge with pharmacypractice; student requires minimal coaching or supervisionin pharmacy practice or simulation environments), compe-tent (student demonstrates ability to independently and

65


consistently synthesize and extend knowledge to pharmacypractice; student does not require supervision in pharmacypractice or simulation environments). Teaching facultymembers used the NFC (novice, functional, competent)framework to assign their expectation of student perfor-mance for each AFPC educational outcome element linkedto a course objective. The systematic approach in usingconceptual frameworks for cognitive learning and skill ac-quisition ensured that the learning objectives not onlyaligned with the curriculum but also with the expectationsthat teaching faculty members had for students. Impor-tantly, the use of conceptual frameworks framed the acqui-sition of knowledge, skills, and professional attitudes in anintegrated process rather than in the mastery of separatecontent areas or disciplines.

Teaching faculty members submitted informationfor their individual courses, while course coordinatorscollected and collated data for multi-instructor courses.A data collection template (Table 1) was provided to linkeach course objective to the relevant AFPC educationaloutcome element(s) and to document the expected levelsof student learning (ICE) and performance (NFC). Allinformationwas reviewed by the curriculummanagementcommittee and an education expert.

Data management and analysis were conducted us-ing Excel, whereas final versions of the curriculum mapwere available in PowerPoint files that contained hyper-links to more detailed information. The curriculum mapwas available in 2 versions: an outcome-centered map thatfollowed the AFPC educational outcomes and a student-centered map that followed progression through eachyear of the program. Importantly, the curriculum map

captured data for every course at the level of each AFPCeducational outcome (and element). It provided robustanalytical capabilities using accessible software wherethe weighting, learning levels, and performance levelsof each AFPC educational outcome (and element) couldbe assessed within courses, discipline streams, years, orthe overall program. Although electronic and Web-basedcurriculum management systems were available, we rec-ognized important limitations related to inflexible plat-forms, specialized operating skills and costs.11,12 Thus, wemade the decision to useMicrosoft Office software based onits widespread access,familiarity, and cost effectiveness.

EVALUATION AND ASSESSMENTAssessing the Intended Curriculum

First, the curriculum map was used to characterize therelative number of course objectives (weightings) linked toeachAFPCeducational outcome.As summarized inTable 2,the alignment of the curriculum with the AFPC educationaloutcomes showed the highest weighting of course objectivesfor care provider (35.0%), followed by scholar (27.0%), andthe lowest number formanager (4.3%). Next, the curriculummap andweighting of course objectiveswere comparedwiththe national licensing examination constructed by the Phar-macy Examining Board of Canada (PEBC). In this case, theweighting of course objectives were compared with theweighting of professional competencies outlined in the ex-amination blueprint (Table 2).16 The results showed reason-able concordance between the curriculum map and thenational licensing examination, with weightings of 35.0%vs 38% for care provider, 27.0% vs 22% for scholar, and15.3% vs 20% for communicator, respectively.

Table 1. Course Objectives Linked to Principal AFPC Educational Outcomes Along With Expected Learning and PerformanceLevels

Course ObjectivesAt the completion of this course,the student should be able to:

AFPC EducationalOutcome Achieved

Learning Level(ICE – Ideas,Connections,Extensions)

Performance Level(NFC – Novice,Functional,Competent)

Identify and connect relevant patient, clinical,and laboratory data in patient assessment anddisease state management generally and forpatient-specific scenarios (eg, patientdemographics, medical conditions, co-morbidities,medications, allergies, clinical status,laboratory results)

1.2 Determine patient needsand desired outcomes

Connections Functional

1.3 Identify and prioritizepatient drug-related issues


1.5 Refer when appropriate Connections Competent

Compare and contrast (differentiate) therapeuticalternatives considering issues related to patientneeds, clinical efficacy, potential adverse effectsand drug interactions

1.4 Develop therapeutic andmonitoring plans to achieveoptimal outcomes


2.2 Practice in a professional andethical manner, accountable tothe patient


66


The curriculum map was also used to characterizethe sequencing and leveling of student learning and per-formance through the program. An analysis of learninglevels by program year showed objectives at an introduc-tory level (ideas) in 42% of cases in the first year, withprogression to an advanced level (extensions) in 77% ofcases in the final year (Figure 1). The most commonexpected learning level was ideas in year 1 (42.0%), con-nections in year 2 (78.0%), and extensions in years 3 and 4(53.0% and 76.5%, respectively). Similar progressions

were observed for the expected performance levels as stu-dents gained experience and expertise over time (Figure 2).Themost common expected performance level was novicein year 1 (98.5%), functional in years 2 and 3 (48.0% and67.8%, respectively) and competent in the final year(67.8%).

Validating the Learned CurriculumThe learned curriculum was assessed and validated

using employer surveys where participants rated their

Table 2. Alignment of the Intended Curriculum With the Principal AFPCa Educational Outcomes and Weighting of ProfessionalCompetencies in the PEBCb National Licensing Examination

AFPC Educational Outcomes for FirstDegree Programs in Pharmacy (Entry-to-Practice)in Canada

Faculty of PharmacyCourse Objectives

[n = 790] (% weighting)

PEBC LicensingExamination

Blueprint (% weighting)

CARE PROVIDER35.0 38Use knowledge, skills, and professional judgement to

provide pharmaceutical care and to facilitate managementof patient’s medication and overall health needs. Elicit andcomplete an assessment of required information to determinethe patient’s medication needs.

COMMUNICATOR15.3

c20Communicate with diverse audiences, using a variety of

strategies that take into account the situation, intendedoutcomes of the communication and the target audience.

ADVOCATEUse their expertise and influence to advance the health andwell-being of individual patients, communities, andpopulations, and to support pharmacist’s professional roles.

COLLABORATOR7.7 7Work collaboratively with teams to provide effective,

quality health care and to fulfill their professionalobligations to the community and society at large.

MANAGER4.3 3Use management skills in their daily practice to optimize

the care of patients, to ensure the safe and effectivedistribution of medications, and to make efficientuse of health resources.

SCHOLAR27.0 22

dHave and can apply the core knowledge and skills requiredto be a medication therapy expert, and are able to master,generate, interpret, and disseminate pharmaceutical andpharmacy practice knowledge.

PROFESSIONAL10.5 10Honor their roles as self-regulated professionals through

both individual patient care and fulfillment of theirprofessional obligations to the profession, the community,and society at large.

a Association of Faculties of Pharmacy of Canadab Pharmacy Examining Board of Canadac For comparison with PEBC competencies for “Communication” (20%), course objectives for “Communicator” (9.7%) and “Advocate” (5.6%)were combinedd For comparison with course objectives for “Scholar” (27%), PEBC competencies for “Drug, Therapeutic and Practice Information” (6%) and“Drug Distribution” (16%) were combined.

67


satisfaction with the performance and skills of recentgraduates (within 5 years) in areas related to the AFPCeducational outcomes. Overall, 71.0% of respondentsranked the school’s pharmacy graduates as very good orexcellent, 12.9% as good, and 16.1% as satisfactory. Norespondents rated the graduates as unsatisfactory. Em-ployer feedback on specific educational outcomes is de-tailed in Table 3.

In addition, graduating student exit surveys andfocus groups were conducted where respondents rated

their abilities. As seen in Table 3, there was high concor-dance between the employer ratings and the self-percep-tions of graduating students.

DISCUSSIONThe curriculum mapping process was an efficient

and effective method of deconstructing and analyzingthe intended curriculum relative to the national educa-tional outcomes for pharmacy programs and competen-cies for pharmacy licensure. The map showed an

Figure 1. Expected student learning level based on course objectives by program year

68


emphasis on care provider and scholar, which supportedthe focus on pharmacy knowledge and professional prac-tice. The map also demonstrated progression in the level-ing and sequencing of student learning and performancethrough the program. The process identified some im-portant areas for improvement. For example, gaps in edu-cational outcomes related to the role of manager weretriangulated with the light weighting of course objectives

on the curriculummap and low scores on both the employerand graduating student exit surveys. The issue was investi-gated further in student focus groups that specified contentgaps related to business processes, human resources, anddrug plan procedures. This information was sharedwith theCurriculum Management Committee, which investigatedfurther to determine where and at what level learningobjectives should be added to the curriculum.

Figure 2. Expected student performance level based on course objectives by program year

69


Table 3. Validation of the Learned Curriculum as Demonstrated by the Results of Employer and Graduating Student Exit Surveys

AFPCa Educational Outcomes for First-DegreePrograms in Pharmacy in Canada

Employer Survey ofRecent Graduatesb

Graduating StudentExit Surveyc

CARE PROVIDER Novice, 6% Take a patient history (A 37%, SA 63%)Use knowledge, skills, and professionaljudgement to provide pharmaceuticalcare and to facilitate management ofpatient’s medication and overall healthneeds. Elicit and complete an assessmentof required information to determine thepatient’s medication needs.

Functional, 63% Recognize and prioritize drug-relatedproblems (A 61%, SA 37%)Skilled, 31%

Interpret relevant patient data(A 63%, SA 38%)

Develop a therapeutic plan (A73%, SA 24%)

COMMUNICATOR Novice, 16%, Communicate with health care team(A 67%, SA 22%)Communicate with diverse audiences,

using a variety of strategies that take intoaccount the situation, intendedoutcomes of the communication, and thetarget audience.

Functional, 56%Communicate with patients (A 75%, SA 26%)Skilled, 25%

ADVOCATE Novice, 22%, Advocate for patients (A 65%, SA 33%)Use their expertise and influence to advancethe health and well-being of individualpatients, communities, and populations,and to support pharmacist’s professional roles.

Functional, 50%Skilled, 28%

COLLABORATOR Novice, 9% Work collaboratively (A 65%, SA 14%)Work collaboratively with teams to provideeffective, quality health care and to fulfilltheir professional obligations to the communityand society at large.

Functional, 59%Skilled, 28%

MANAGER Novice, 41% Apply management principles(DA 39%, A 55%)Use management skills in their daily practice

to optimize the care of patients, to ensure thesafe and effective distribution of medications,and to make efficient use of health resources.

Functional, 38%Manage and maintain resources andpersonnel (DA 46%, A 42%)

Skilled, 19%

SCHOLAR Novice, 6%, Use appropriate resources to accessinformation (A 61%, SA 33%)Have and can apply the core knowledge and

skills required to be a medication therapy expert,and are able to master, generate, interpret, anddisseminate pharmaceutical and pharmacypractice knowledge.

Functional, 53%Conduct systemic reviews and appraisals ofthe literature (DA 33%, A 55%)

Skilled, 38%

Incorporate evidence into decisions(A 78%, SA 14%)

PROFESSIONAL Novice, 7% Include ethical issues into decisions(A 78%, SA 22%)Honor their roles as self-regulated professionals

through both individual patient care and fulfillmentof their professional obligations to the profession,the community, and society at large.

Functional, 55%Value honesty and integrity(A 51%, SA 49%)

Skilled, 39%

Safeguard confidentiality of patients(A 37%, SA 63%)

a AFPC is the Association of Faculties of Pharmacy of Canada.b Pharmacy employer survey conducted in 2013 (n536) where respondents rated their satisfaction with the performance and expertise of recentgraduates (within 5 years); where Novice required supervision to complete activities and knowledge base was inadequate in some areas;Functional completed activities with minimal supervision and knowledge base was appropriate in most areas; and Skilled completed mostactivities independently and knowledge base exceeded expectations for entry-to-practice.c Graduating student exit surveys and focus groups conducted in 2013 (n551) where respondents rated their performance abilities at the end of theprogram, and where SA is strongly agree, A is agree and DA is disagree.

70


Our curriculum mapping process resulted in a num-ber of positive outcomes for the school. It reinforced keyeducational principles and introduced conceptual frame-works that provided a systematic approach and commonlanguage for discussing, analyzing, and modifying thecurriculum. The inclusive approach resulted in participa-tion from all teaching faculty members in providing thenecessary data to create a relevant and reliable curriculummap. Information collected during the process was alsovaluable for teaching faculty members to make course-specific decisions related to the content, teachingmethods, and learning opportunities. For example, a sur-vey of teaching faculty members identified concernsrelated to teaching critical-thinking skills. A facultydevelopment session was conducted during which wedefined critical thinking and outlined the important skillsexpected of students. These included the ability to definequestions, search and appraise relevant information,make informed decisions, and deal with ambiguity. Ananalysis of the curriculum map revealed that these skillswere covered with the exception of managing ambiguity.This observation was validated in focus groups where stu-dents described being comfortable conducting literaturesearches and appraisals but less confident dealing withambiguous or inconsistent information. As such, teachingfacultymemberswere given guidanceon teachingmethodsfor developing critical-thinking and decision-makingskills.

The curriculum map was used to facilitate curricu-lum presentation and analysis in an accreditation review.The curriculum mapping process also instilled an ap-proach of inquiry and scholarship, which led to the crea-tion of a formal program evaluation committee. Thisextended the mandate to developing a formal process forevaluating performance outcomes of the pharmacy pro-gram including admissions, the curriculum, teaching andlearning, and graduate performance. This work has fosteredclose collaborations with committees and administrators ofother schools. Finally, the curriculum mapping experienceyielded broader institutional benefits. As the educationalprinciples and conceptual frameworks were relevant andtransferable, other professional programs adopted our ap-proach to constructing curriculum maps. Our institutionalso embarked on an interprofessional education and com-petency mapping process to align with entry-to-practicestandards of health care professionals in the local region.

Our experience confirmed some essentials forsuccessful curriculum mapping. Close collaboration ofexperts from pharmacy and education ensured a processthat was relevant and grounded in solid principles. Fac-ulty engagement and development improved the consis-tency and reliability of data used to construct the

curriculum map. Data collection and management wasfacilitated by using common templates and familiar soft-ware programs. Finally, involvement and support fromadministration reinforced the importance and value ofcurriculum mapping.

One limitation of our original curriculum map wasthat it lacked information related to teaching methods andstudent assessment. Although these data were added to anupdated version, the information was only descriptive. Assuch, we have embarked on a pilot project that uses thecurriculum map to determine how well student assess-ments (eg, examinations) align with the content, level,and weighting of course objectives.

Another limitation was the need for more objectiveexternal validation of alignment between the intended andlearned curricula. Although the overall performance ofgraduates on the PEBC licensing examination is very good,it is only a crude indicator of the effectiveness of the learnedcurriculum. As such, we have engaged in collaborative re-searchwith the PEBC to allowmore detailed analyses of ourgraduates’ performance on each professional competency inthe licensing examination blueprint.

SUMMARYA curriculum map embedded in solid educational

principles demonstrated how courses were integrated toachieve an intended curriculum. It provided ongoing doc-umentation and analysis as opposed to a “snapshot” de-scription for periodic accreditation or other reviews. Italso demonstrated the longitudinal progression and scaf-folding of student learning and performance througha program. The curriculum map promoted and developedshared responsibility for the curriculum and its mandateof preparing students for pharmacy practice. This curric-ulum mapping process was efficient and effective in pro-viding an evidence-based approach to continuous qualityimprovement of a pharmacy program.

ACKNOWLEDGMENTSThe authors acknowledge the contributions of

Ms. Angela Tittle (Consultant, University Teaching Ser-vices, University of Manitoba) and Ms. Cheryl Lee (Un-dergraduate ProgramAdministrator, Faculty of Pharmacy,University of Manitoba).

REFERENCES1. The Association of Faculties of Pharmacy of Canada (AFPC):Educational Outcomes for first professional degree programs inpharmacy (entry-to-practice pharmacy programs) in Canada. http://www.afpc.info/node/39. Accessed December 17, 2013.2. The Advancement of Pharmacy Education (CAPE): CAPEEducational Outcomes 2013. http://www.aacp.org/resources/education/cape/Pages/default.aspx. Accessed March 20, 2014.

71


3. Kopera-Frye K, Mahaffy J, Svare G. The map to curriculumalignment and improvement. In: Wright A, Murray S, Wilson M, eds.CELT: Collected Essays on Learning and Teaching. Windsor, ON:Society for Teaching and Learning in Higher Education; 2008:8-14.4. Kelley K, McAuley J, Wallace L, Frank S. Evaluation,assessment, and outcomes in pharmacy education: the 2007 AACPInstitute - Curricular Mapping: Process and Product. Am J PharmEduc. 2008;72(5):Article 96.5. Robley W, Whittle S, Murdoch-Eaton D. Mapping generic skillscurricula: a recommended methodology. J Further Higher Educ2005;29(3):221-231.6. Robley W, Whittle S, Murdoch-Eaton D. Mapping generic skillscurricula: outcomes and discussion. J Further Higher Educ. 2005;29(4):321-330.7. Uchiyama K, Radin J. Curriculum mapping in higher education:a vehicle for collaboration. Innov Higher Educ. 2009;33:271-280.8. Willett T, Marshall K, Broudo M, Clarke M. It’s about TIME:a general-purpose taxonomy of subjects in medical education. MedEduc. 2008;42(4):432-438.9. Harden R. AMEE Guide No.21: Curriculum mapping: a tool fortransparent and authentic teaching and learning.Med Teach. 2001;23(2):123-137.

10. Mazurat R, Schonwetter D. Electronic curriculum mapping:supporting competency-based dental education. J Can Dent Assoc.2008;74(10):886-889.11. Bell C, Ellaway R, Rhind S. Getting started with curriculummapping in a veterinary degree program. J Vet Med Educ. 2009;36(1):100-10612. Britton M, Letassy N, Medina MD, et al. A curriculum reviewand mapping process supported by an electronic database system. AmJ Pharm Educ. 2008;72(5):Article 99.13. Fostaty Young S, Wilson R. Assessment and Learning: The ICEApproach. Winnipeg, MB: Portage and Main Press; 2000: 9.14. Dreyfus S, Dreyfus H. A five stage model of the mental activitiesinvolved in directed skill acquisition [monograph]. CaliforniaUniversity Berkeley Operations Research Center; 1980. http://www.dtic.mil/dtic/index.html. Accessed January 7, 2014.15. Ericsson KA. The role of deliberate practice in the acquisition ofexpert performance. In: The Cambridge Handbook of Expertise andExpert Performance. New York: Cambridge University Press.2006:683-703.16. The Pharmacy Examining Board of Canada (PEBC): Pharmacistqualifying exam blueprint. http://microsites.pebc.ca/EnglishPages/QEX/QEXBlueprint.html. Accessed December 17, 2013.

72



Improved Knowledge Retention Among Clinical Pharmacy StudentsUsing an Anthropology Classroom Assessment Technique

Heather P. Whitley, PharmD, BCPS, CDE,a,b Jason M. Parton, PhDc

a Harrison School of Pharmacy, Auburn University, Auburn, Alabamab Montgomery Family Medicine Residency Program, Baptist Health System, Montgomery, Alabamac Culverhouse College of Commerce and Business Administration, The University of Alabama, Tuscaloosa, Alabama

Submitted January 13, 2014; accepted March 13, 2014; published September 15, 2014.

Objective. To adapt a classroom assessment technique (CAT) from an anthropology course to a di-abetes module in a clinical pharmacy skills laboratory and to determine student knowledge retentionfrom baseline.Design. Diabetes item stems, focused on module objectives, replaced anthropology terms. Answerchoices, coded to Bloom’s Taxonomy, were expanded to include higher-order thinking. Studentscompleted the online 5-item probe 4 times: prelaboratory lecture, postlaboratory, and at 6 monthsand 12 months after laboratory. Statistical analyses utilized a single factor, repeated measures designusing rank transformations of means with a Mann-Whitney-Wilcoxon test.Assessment. The CAT revealed a significant increase in knowledge from prelaboratory compared to allpostlaboratory measurements (p,0.0001). Significant knowledge retention was maintained with basicterms, but declined with complex terms between 6 and 12 months.Conclusion. The anthropology assessment tool was effectively adapted using Bloom’s Taxonomy asa guide and, when used repeatedly, demonstrated knowledge retention. Minimal time was devoted toapplication of the probe making it an easily adaptable CAT.

Keywords: Classroom assessment techniques; diabetes; skills lab; background knowledge probe; knowledgeretention; CAPE domains

INTRODUCTIONIn 2011, the Accreditation Council for Pharmacy

Education (ACPE) updated the Accreditation StandardsandGuidelines for the Professional Program in PharmacyLeading to the Doctor of Pharmacy Degree.1 StandardNo. 15, which addresses Assessment and Evaluation ofStudent Learning and Curricular Effectiveness, denotesthat curricular evaluation should include varied forma-tive and summative assessment methods that are system-atically and sequentially administered to determinestudents’ achievement at different levels and fosterexperimentation and innovation.

Based on this guideline, educators might want toconsider implementing formative and summative assess-ment techniques in tandem with the course, module, orlecture. These techniques should be incorporated fre-quently through a course, making assessment part of the

learning process.2 Doing so would facilitate the ability touse analyzed assessment data for immediate improve-ments in the educational modality rather than waiting un-

til later time points, such as the final evaluation, whenadjustments would only benefit students in subsequentsemesters. Additionally, using frequent assessment tech-niques could demonstrate student learning changes by

comparing data from several time points. A systematicand sequential administration of assessments will eithervalidate the educational modality or bring attention toimprovements needed relating to long-term learninggains. Classroom assessment techniques (CATs) are brief

formative evaluations that are available in a large varietyof constructs. When administered at various time points,theymay be used to track student achievement at differentlevels, while fostering instructor experimentation and in-novation. Therefore, CATs could be used tomeet the need

set forth by Standard No. 15.CATs augment clearly defined teaching objectives

by allowing instructors to formatively determine what,how much, and how well students are achieving learninggoals.3-5 Important characteristics of effective CATs

Corresponding Author: Heather P. Whitley, PharmD, BCPS,CDE, Montgomery Family Residency Program, BaptistHealth, 4371 Narrow Lane Road, Suite 100, Montgomery, AL36116. Tel: 334-280-7084; Fax: 334-613-3685. E-mail:[email protected]

73


include being learner-centered, teacher-directed, mutu-ally beneficial, formative, context-specific, and well in-tegrated into the teaching and learning process.6 CATsallow instructors to gage students’ knowledge or per-ceived knowledge of various topics before, during, orafter learning experiences. In addition to supporting in-structional development, CATs also allow instructors tofocus valuable class time toward knowledge deficientareas rather than topics more comfortable for students.3,6

Due to the formative nature of CATs, educational adjust-ments can be made immediately rather than waiting untilthe following semester. CATs also present an opportunityfor students to provide anonymous feedback about theirlearning. Students particularly hesitant to ask questionsaloud in class can more comfortably communicate withinstructors by completing a CAT and may also learn thatother classmates share like concerns.3 Additionally, byproviding welcome feedback, students become more in-volved in their learning, motivated to successfully com-plete the course, and self-directed.3,4

CATs may be used to determine students’ priorknowledge or ability to recall, apply, analyze, synthesize,create, or critically consider material. The CATs mostcommonly used to assess knowledge and recall are the“background knowledge probe,” the “one minute paper,”and the “muddiest point.” Instructors use the one minutepaper to allow students to respond in writing to the fol-lowing two questions during the final fewminutes of classtime: “What was the most important thing you learnedtoday in class?” and “What are you still confused about?” 7

This technique allows instructors to compare course ormodule objectives with students’ perceptions of impor-tant topics and their learning. Again, students respond inwriting at the conclusion of a lecture to the muddiestpoint. This knowledge and recall CAT mirrors half ofthe one minute paper by allowing students to respond re-garding the most confusing aspect of education provided.This helps the instructor identify points to better empha-size or explain prior to starting the next course lecture.3,6

Quite different from the one minute paper and mud-diest point, the background knowledge probe CAT allowsinstructors to easily collect and analyze student prepared-ness about a particular topic. Gathered information canthen be applied to determine the most appropriate startingpoint and instruction level for a given lesson.3,6 High-lighting essential information when using a backgroundknowledge probe not only allows review of previouslypresented material, but also provides direction for futuretopics of study within a course. Background knowledgeprobes require students to reflect on current knowledgeand assess their understanding, often through either shortanswer responses or selection of answer choices.3,6 They

aremost commonly administered prior to beginning a newcourse, lesson, or topic.6 However, they may also be ad-ministered immediately following an event to gain a roughsense of improvement in knowledge base or familiarity.3,6

Although CAT examples are described in the litera-ture, including the arena of Allied Health, most offeranecdotal support of their impact rather than sufficient,well-articulated detail to determine objective, outcome-related changes.8-10 A study by Wise is one exception,describing the process and impact of implementing themuddiest point CAT in a 2-hour lecture-based course inphysical therapy for third-year students.11 Confidentiallycompleted evaluations showed significant improvementsin student perception of the course and instructor whena CAT was administered throughout the semester com-pared to previous semesters, when no CAT was used.Students responded that course assignments were reason-able and contributed to learning (p50.002), the coursewaswell organized (p50.007), and the instructor providedstudents with opportunities for questions (p50.004).11

CATs have also been developed and implemented invarious doctor of pharmacy programs, although few stud-ies describe specific outcomes. Van Amburgh and col-leagues assessed the impact of various active-learningtechniques integrated into pharmacy classroom lectures,including CATs, but did not specifically outline the ben-efits gained by the individual assessment measures.12

Other assessment review articles mention that CATs, es-pecially the muddiest point, is often use by pharmacyfaculty, but again, few specifics are provided.8,13,14

Bartlett and Morrow, however, provide a comprehensivedescription of adapting and implementing the one minutepaper in a first-year Biochemical Basics of Drugs andDisease pharmacy course. They expanded the 2-questionCAT to include a third: “What was the most interestingfact you learned today?” Course feedback gatheredthrough the mid-term evaluation and a student-completedsurvey indicated that inclusion of the one minute paperimproved student-faculty relationships, student under-standing of difficult material, and student likelihood ofasking questions in class (with women feeling more en-couraged than men to ask questions in class) (p,0.01).15

Additionally, students believed the CAT was an effectiveuse of class time.15

While limited literature outlines the implementationand assessment process of including a CAT in pharmacyor other health-related disciplines, findings do tend topoint to improvements in either student perception ofthe course, content, or instructor. The current study in-vestigates the process of translating a background know-ledge probeCAT from another discipline (anthropology6)to a pharmacy-specific course. Additionally, it is the first

74


assessment to objectify changes in student knowledgeretention over a 12-month interval using this type of back-ground knowledge probe.

DESIGNThird-year pharmacy students attended (or later

viewed the recording of) a 1-hour prelaboratory lectureon type 1 diabetes at Auburn University in a large class-room setting. The off-campus instructor used a video con-ference system (Polycom) to present background contentto students. Whether students attended the real-time prel-aboratory lecture or later watched the recorded presenta-tion, they received identical content. Over the following 7days, students completed a homework assignment of pho-tographing their dinner, counting carbohydrates in themeal, and calculating a dose of rapid-acting insulin basedon a given insulin-to-carbohydrate ratio. A week laterstudents attended a 2-hour clinical pharmacy skills labo-ratory (approximately 32 students per section), wherethey participated in hands-on, active learning, case-basededucation. The laboratory included 4 carbohydrate count-ing exercises, presentation of 4 meal photographs perlaboratory section with discussion, and 10 patient casesrequiring insulin dosing. Each exercise was conductedthrough the think-pair-share method.

After completing the 1-hour prelaboratory and 2-hour clinical pharmacy skills laboratory, students wereexpected to have met the following learning outcomeobjectives: (1) Comprehension: identify the foods mostlikely to increase blood glucose; (2) Application: applycarbohydrate counting to food labels; (3) Comprehen-sion: describe the plate method; (4) Knowledge: define“basal-bolus” insulin regimen; (5) Analysis: compare andcontrast “basal-bolus” regimens to pre-mixed insulin prod-ucts; (6) Knowledge: define the term “insulin sensitivityfactor;” (7) Knowledge: define the term “insulin-to-carbohydrate ratio.” The above learning outcome objec-tives were used through the 2011 fall semester (2013graduating class). The learning outcome objective regard-ing the plate method (objective 3) was removed from themodule in the fall of 2012 (2014 graduating class) andinserted into a laboratorymodule occurring later in the fallsemester. Therefore, content regarding the plate methodwas also removed from the prelaboratory and laboratorysections of the 2012 fall module.

The original anthropology background knowledgeprobe contained 50 items andwas administered on the firstday of a semester-long course.6 Tested topic items wererecommended by prerequisite and lower-level course in-structors. Each item consisted of a term (for example “TheWeimar Republic,” “Senator Joseph McCarthy,” or “TheGolden Triangle”), followed by 4 possible answer choices

indicating varying degrees of familiarity, which werenearly identical for each item. Prior to administering thein-class, paper-based probe, students were informed that itwas not a test and would not be graded. The results wereused to improve student learning during the semester.6

(Table 1)Because the background knowledge probewas adap-

ted from a semester-length course to a brief module, thenumber of items testedwas decreased from50 to 5. Testeditem topics were selected from and focused on moduleobjectives and were presented in order of appearance dur-ing the prelaboratory and laboratory sections and in in-creasing complexity. The wording of answer choices waskept as closely as possible to the original anthropologyexamples. Tomore accurately translate to applied aspectsof patient care, 1 additional option was added to the first 3items and 2 additional options were added to the last 2items. To accurately analyze the anthropology probe,each answer choice was coded to Bloom’s Taxonomy toindicate degree of higher order thinking (Table 1). Thefirst answer choice (“Have not heard of this”) was notassigned a level of Bloom’s Taxonomy, as it indicatesthat no knowledge had been acquired on the topic. Thesecond and third answer choices were both coded as“Knowledge,” while the fourth answer choice was codedas “Comprehension.” The samemethod of coding answerchoices to Bloom’s Taxonomywas applied to the adapteddiabetes probe (Table 2). Two independent individuals,an education assessment expert, and a course coordinatorskilled in assessment techniques assisted in the coding ofthe answers. Answer choices were listed in order of in-creasing level of understanding. Four 4th-year pharmacystudents participated in cooperative inquiry to validate theutility of the survey and to ensure accurate interpretation.The 5-item probe was reformatted as a Vovici onlinequestionnaire (Vovici Corporation, Herndon, VA), inwhich knowledge levels were ranked to represent partic-ipants’ knowledge of the diabetes term.

All third-year pharmacy students enrolled in themandatory clinical pharmacy skills laboratory duringthe fall of 2011 and 2012 were e-mailed an informational

Table 1. Anthropology Item with Coded RankingClassifications per Bloom’s Taxonomy

“The Weimar Republic”

Have never heard of this (N)Have heard of it but don’t really know what it means (K)Have some idea what this means , but not too clear (K)Have a clear idea what this means and can explain it (C)

None (N)Knowledge (K)Comprehension (C)

75


letter two days prior to the prelaboratory lecture explain-ing the voluntary, anonymous nature of participation andcontaining a link to the online 5-item questionnaire. The

letter explained that participation in the questionnairewould not affect laboratory or course grades and wouldbe used for teaching purposes only, so using outside

Table 2. Student Self-Reported Familiarity with Diabetes Terms Using the Background Knowledge Probe and Coded to Bloom’sTaxonomya – Number (%)

Term (Bloom’s Taxonomy)BeforePrelab

ImmediatelyAfter Lab

6 MonthsAfter Lab

12 MonthsAfter Lab

Response Rate (n5281 number potential participants) 194/(281-12)(72.1)

230/(281-12)(85.5)

203/281(72.2)

221/281(78.6)

“Carbohydrate”Have never heard of this term (N) 1 (0.5) 0 (0) 0 (0) 2 (0.5)Have heard the term, but do not know what it means (K) 0 (0) 0 (0) 0 (0) 0 (0)Have some idea what this is, but am not too clear (K) 12 (6.2) 0 (0) 0 (0) 0 (%)Have a clear idea what this is and can list some examples (C) 75 (38.7) 11 (4.8) 8 (3.5) 15 (6.8)Have a clear idea what this is, can list some examples,

and can explain how it relates to diabetes (AP)106 (54.6) 219 (95.2) 195 (96.1) 204 (92.3)

“Plate Method”Have never heard of this phrase (N) 89 (45.7) 4 (1.7) 0 (0) 2 (0.5)Have heard the term, but do not know what it means (K) 32 (16.5) 10 (4.3) 0 (0) 0 (0)Have some idea what this means, but am not too clear (K) 44 (22.7) 34 (14.8) 15 (7.4) 16 (7.2)Have a clear idea what this means and can explain it (C) 16 (8.2) 39 (16.6) 34 (16.7) 44 (15.9)Have a clear idea what this means, can explain it, and

could use it in patient care (AP)11 (5.7) 142 (61.7) 152 (74.7) 159 (71.9)

“Basal-Bolus Insulin Regimen”Have never heard of this phrase (N) 58 (25.9) 0 (0) 0 (0) 1 (0.5)Have heard the term but, do not know what it means (K) 62 (31.6) 0 (0) 0 (0) 0 (0)Have some idea what this means, but am not too clear (K) 52 (26.6) 1 (0.4) 2 (0.6) 3 (1.4)Have a clear idea what this means and can explain it (C) 13 (6.7) 21 (9.1) 33 (16.3) 38 (17.2)Have a clear idea what this means, can explain it, and can

develop a basal-bolus insulin regimen (S)8 (4.1) 208 (90.4) 167 (62.3) 178 (80.5)

“Insulin-to-Carbohydrate Ratio”Have never heard of this phrase (N) 46 (23.7) 0 (0) 1 (0.5) 0 (0)Have heard the term, but do not know what it means (K) 51 (26.3) 0 (0) 1 (0.5) 1 (0.5)Have some idea what this means, but am not too clear (K) 78 (40.2) 1 (0.4) 8 (3.5) 20 (9)Have a clear idea what this means and can explain it (C) 12 (6.2) 3 (1.3) 30 (14.8) 45 (20.4)Have a clear idea what this means, can explain it, and

can calculate it (AP)2 (1) 23 (10) 35 (17.2) 43 (19.5)

Have a clear idea what this means, can explain it,can calculate it, and could use it in patient care (E)

3 (1.5) 203 (88.3) 141 (69.5) 112 (50.7)

“Insulin Sensitivity Factor” or “Correction Factor”Have never heard of this phrase (N) 40 (20.6) 0 (0) 1 (0.5) 0 (0)Have heard the term, but do not know what it means (K) 74 (38.1) 0 (0) 0 (0) 1 (0.5)Have some idea what this means, but am not too clear (K) 69 (35.6) 5 (2.2) 6 (2.6) 27 (12.2)Have a clear idea what this means and can explain it (C) 4 (2.1) 1 (0.4) 30 (14.8) 42 (19)Have a clear idea what this means, can explain it, and

can calculate it (AP)4 (2.1) 27 (11.7) 30 (14.8) 44 (15.9)

Have a clear idea what this means, can explain it,can calculate it, and could use it in patient care (E)

2 (1) 196 (85.2) 137 (67.5) 107 (48.4)

a Data collected September 2011 through September 2013 for 2013 and 2014 graduating classesN- None, K –Knowledge, C –Comprehension, AP – Application, AN – Analysis, S – Synthesis, E – Evaluation

76


resources to complete the questionnaire would not benefitstudents. The questionnaire was closed and data analyzed30 minutes before the prelaboratory class lecture. Resultswere shared with students during the 1-hour lecture pe-riod. Students were e-mailed a second, third, and fourthrequest to complete identical surveys immediately afterthe 2-hour laboratory section, which occurred 1 weeklater, at 6 months, and lastly at 12 months, respectively.Students were allowed 10 days to complete the follow-upquestionnaires and were sent 3 reminder e-mails to im-prove response rates. The average response rate for eachfamiliarity statement was calculated for all 4 phases ofmeasurement.

Data for each familiarity statement consisted of rank-ing classifications based on Bloom’s Taxonomy. In orderto account for nonparametric trait outcomes, rank trans-formations were applied to the knowledge responses ofthe 5 terms. Using these ranks, a single factor repeatedmeasures analysis was applied to examine student-reported familiarity and retention over the 4 time-pointmeasurements from the prelaboratory through 12-monthfollow-up. SAS version 9.4 (SAS Institute, Cary, NC)wasused for all analyses. The project received InstitutionalReview Board approval through exempt procedures atAuburn University.

EVALUATION AND ASSESSMENTResponses from both graduating classes were com-

bined for analysis to evaluate student learning. A total of281 third-year pharmacy students were enrolled in the2011 and 2012 fall course. Twelve student e-mails werereturned in the fall of 2011 during the prelaboratory andimmediate postlaboratory survey due to changes in theuniversity’s e-mail system. Response rates for each timepoint of measurement were 72.1%, 85.5%, 72.2%, and78.6%, respectively. Results from the student sampleshowed a dramatic improvement in self-reported termfamiliarity from prelaboratory to each time point assess-ment after the laboratory, including a shift to higher orderlearning per the application of Bloom’s Taxonomy to thebackground knowledge probe (Table 2). When changesin familiarity were coded based on Bloom’s Taxonomyusing a Mann-Whitney-Wilcoxon test, significant in-creases (p,0.0001) were noted for all 5 terms over the12-month assessment period when compared to the prel-aboratory measurement (Table 3). This shift was mostpronounced for more complex terms such as “insulin-to-carbohydrate ratio” and “insulin sensitivity factor,”which overlay each other in Figure 1, vs the most basicterm “carbohydrate.” The modest, but still significant,improvement from prelaboratory with “carbohydrate”was likely due to a stronger baseline familiarity.

Additionally, loss of knowledge retention was most no-table with more complex terms. Table 4 denotes changein knowledge retention over time. Significant knowledgeimprovements were noted for all 5 terms from prelab-oratory through the 12-month assessment point(p,0.0001). This change in knowledge (per the negativedifference values) demonstrated a steeper improvementwith more complex terms possibly due to a lowerBloom’s Taxonomy baseline starting point (Table 4 andFigure 1). Themost basic term, “carbohydrate,” shows nosignificant change in familiarity from postlaboratorythrough 12 months (p51), indicating no significant lossof knowledge. On the other hand, themost complex termsof “insulin-to-carbohydrate ratio” and “insulin sensitivityfactor” demonstrate a significant decline in knowledgeretention (per the positive difference values) when theassessment time points occurring after the prelaboratorywere compared to each other (p,0.0001). Knowledgeretention for the term “basal-bolus” displayed amore bal-anced change across the time points with a significantdecline seen when comparing the 12-month assessmentto the postlaboratory (p50.017) and 6-month (p,0.0001)assessment (Table 4).

Change in knowledge and familiarity with the term“plate method” exhibited a uniquely different trend overtime. This was the only term with results indicatingknowledge continuing to increase from postlaboratorythrough the 6-month follow-up and a non-significantknowledge decline from the 6-month assessment to the12-month assessment. Removal of “plate method” con-tent from the module during the 2011 fall semester andplacing it in a module later in the semester explained thisknowledge change.When the data for “platemethod”wasseparated based on year of collection, a stark differencewas noted inWilcoxon scores at prelaboratory, postlabor-atory, 6 months, and 12months, with the 2013 graduatingclass scoring 59.3, 248.2, 221.8, and 214.5, respectively,and the 2014 class scoring 99.8, 229.3, 313.9, and 307,respectively. The removal of “platemethod” content fromthe module allowed the term to function as a control bycomparing data between the two years. This demonstratedthe sensitivity of the background knowledge probe tocapture data differently when term-related educationwas removed.

Collectively, this data indicated a significant knowl-edge gain occurred from the prelaboratory to postlabor-atory assessment time points, but most dramatically formore complex terms. Conversely, knowledge regressionwas steeper with more complex terms, but was retainedwith basic terms. Finally, the probe demonstrated sensi-tivity to change in knowledge when new information wasgained at later time points.

77


DISCUSSIONThe highest level of understanding measured in the

anthropology probe, per Bloom’s Taxonomy, regardlessof actual achievement, was ‘Comprehension’ (Table 1).This limited assessment range may not accurately repre-sent the true baseline level of understanding occurring athigher orders. Secondly, as the anthropology knowledgeprobe stood, it would be difficult to measure much depthin understanding improvements for this same reason; thusthe utility of the original probe was further limited forfollow-up efforts to measure changes. While objectivesfor the anthropology coursewere unknown, the objectivesfor the diabetes laboratory pushed students toward “Ap-plication” and “Analysis” in addition to lower orders ofunderstanding. This further supported the importance ofaltering the probe to better assess higher orders of under-standing. For these reasons, it was essential during theadaptation process of the probe from anthropology toclinical pharmacy, to add more answer choices to bettermeasure higher orders of understanding. Furthermore, itwas recommended that instructors strategically maplearning activities, objectives, and learner knowledgeforCATs toBloom’sTaxonomy to stimulate higher-orderthinking.9 While the adapted background knowledgeprobe met this need, it failed to specifically measure

“Analysis,” although it did reach “Synthesis” and“Evaluation.”

The background knowledge probe presented here isnot a traditional multiple choice quiz with one correctanswer in addition to several distractors. Rather, throughthe answer selection process, students could reflect andconsider their level of understanding and abilities. Fur-thermore, completing the background knowledge probeperiodically gave students a way to reflectively considerimprovements in their learning and abilities throughoutthe 12-month assessment interval.

Unlike many studies analyzing knowledge retention,the present investigation demonstrated a trend in knowl-edge enhancement at several time points. Other studiesdemonstrate immediate increases in student understand-ing directly following a specific educational interven-tion16,17 or document knowledge or skill retention frombaseline to a sole second time point months after the ed-ucational intervention.18,19 Study designs that collect dataat only 2 time points limit the ability to evaluate knowl-edge retention trends, whereas assessing knowledgemoreoften over the same or longer intervals provides moremeaningful data regarding knowledge retention. Twopharmacy student investigations assessed knowledge re-tention at 3 time points. Kopacek and colleagues aimed to

Table 3. Single Factor Repeated Measures for Background Knowledge Probe

Wilcoxon Score Standard Error F Statistic df p Value

CarbohydratePreLab 293.9 9.5 88.1 3 ,0.0001PostLab 472.5 8.96 Month 476. 9.512 Month 459.6 9.1

Plate MethodPreLab 155.3 11.6 240.8 3 ,0.0001PostLab 472.2 10.86 Month 534.4 11.612 Month 521.3 11.1

Basal-BolusPreLab 121.7 8.5 568.9 3 ,0.0001PostLab 539.6 7.96 Month 513.1 8.412 Month 505.8 8.1

Insulin-to-Carbohydrate RatioPreLab 121.1 9.9 454.5 3 ,0.0001PostLab 588.8 9.26 Month 523.7 9.812 Month 445.3 9.4

Insulin Sensitivity FactorPreLab 125.5 10.3 413.2 3 ,0.0001PostLab 587.8 9.66 Month 522.7 10.212 Month 445.4 9.7

78


assess P2 pharmacy student retention of knowledge aboutautomated external defibrillator use following a didactictraining and simulated experience using identical ques-tionnaires administered at baseline, 3 weeks, and 4months following the intervention.20 While significantimprovements were noted in 2 out of 5 knowledge mea-sures and all 6 performance measures at 3 weeks, thestatistical difference declined to zero knowledge and 2performance measures by 4 months.20 The authors con-cluded that the intervention was not sufficient to improvestudent knowledge at 4 months, and recommended

incorporating short refresher courses into the curriculumto enhance knowledge retention.20 Morello and col-leagues aimed to evaluate P1 pharmacy student confi-dence and knowledge retention about diabetes self-careeducation using a performance case-based knowledge testadministered at baseline, immediately after, and 9monthsafter the educational intervention, which consisted of lec-tures, active learning assignments, and workshops.21

While no inferential statistical tests were performed,the average overall percent correct on the knowledgetest nearly doubled from baseline (39.5%) to the test

Table 4. Difference between Wilcoxon Scores for Time Points of Measurement

Difference (95% CL) p Valuea

CarbohydratePreLab PostLab -178.6 (-212.8, -144.3) ,0.0001PreLab 6 Month -182.1 (-217.4, -146.8) ,0.0001PreLab 12 Month -165.7 (-200.3, -131.2) ,0.0001PostLab 6 Month -3.5 (-37.7, 30.6) 1.0000PostLab 12 Month 12.9 (-20.6, 46.3) 1.00006 Month 12 Month 16.4 (-18.1, 50.9) 1.0000

Plate MethodPreLab PostLab -317.0 (-359.0, -274.9) ,0.0001PreLab 6 Month -379.2 (-422.6, -335.8) ,0.0001PreLab 12 Month -366.1 (-408.5, -323.7) ,0.0001PostLab 6 Month -62.2 (-104.2, -20.3) 0.0006PostLab 12 Month -49.1 (-90.0, -8.2) 0.056 Month 12 Month 13.1 (-29.2, 55.4) 1.0000

Basal-BolusPreLab PostLab -417.9 (-448.4, -387.3) ,0.0001PreLab 6 Month -391.4 (-422.9, -359.8) ,0.0001PreLab 12 Month -384.1 (-415.0, -353.2) ,0.0001PostLab 6 Month 26.5 (-4.0, 57.0) 0.13PostLab 12 Month 33.8 (3.9, 63.6) 0.026 Month 12 Month 7.3 (-23.6, 38.1) 1.00

Insulin-to-Carbohydrate RatioPreLab PostLab -467.7 (-503.5, -431.9) ,0.0001PreLab 6 Month -402.6 (-439.5, -365.8) ,0.0001PreLab 12 Month -324.2 (-360.3, -288.1) ,0.0001PostLab 6 Month 65.1 (29.5, 100.7) ,0.0001PostLab 12 Month 143.5 (108.7, 178.3) ,0.00016 Month 12 Month 78.4 (42.5, 114.3) ,0.0001

Insulin Sensitivity FactorPreLab PostLab -462.3 (-499.4, -426) ,0.0001PreLab 6 Month -397.2 (-435.4, -359.0) ,0.0001PreLab 12 Month -319.9 (-357.3, -282.5) ,0.0001PostLab 6 Month 65.1 (28.1, 102.0) ,0.0001PostLab 12 Month 142.3 (106.2, 178.5) ,0.00016 Month 12 Month 77.3 (40.00, 114.5) ,0.0001a p-values are based on the Bonferroni correction for multiple comparisons.

79


administered immediately following the educational in-tervention (85%), but declined by 9 months postinterven-tion (76%).21 Collectively, both studies indicated thatwhile knowledge improved as expected immediately fol-lowing an educational intervention, retention declinedover time.20,21 The more extensive study design of thepresent investigation, which used the background knowl-edge probe at 4 time points (3 following the educationalintervention), more fully illustrated that knowledge de-cline is not always stable or consistent throughout the timeframe (Figure 1). Certainly, additional investigations arewarranted in the realm of knowledge retention, but in ourstudy, knowledge did appear to decay at a steeper rate formore complex terms.

Although the number of terms evaluated with theadapted clinical pharmacy probe was decreased to 5, com-pared to the 50 terms evaluated with the anthropologycourse, the probe addressed understanding for each keyconcept outlined in the module objectives. Use of anonline surveywas likely amore efficient use of technologythan the paper method used for the anthropology probe.Lastly, providing the survey prior to the prelaboratorylecture and after the laboratorymodule reserved class timefor course work.

There are 2 drawbacks to using this assessment. Firsta considerable amount of time was put towards the

upfront development of the probe, which is comparableto another investigation in which 71% of instructorsreported more preparation time was needed to developactive learning teaching efforts.12 Through the translationof the probe from an anthropology class, time was pre-dominately devoted to the application of Bloom’s Taxon-omy and efforts to expand ability to assess deeper levelsof thinking. If adapted for another clinical pharmacycourse, the primary effort would likely be devoted to se-lection of the key terms, resulting in a relatively shortdevelopment stage. Secondly, the probe provided subjec-tive measurements, which may have varied from studentto student. However, the probe likely has reliable intra-subject variability,whichwould provide a reasonable pre-diction of true student change in familiarity.

There are several benefits to using this backgroundknowledge probe. After the translation process was com-plete, administering the probe resulted in little impact onworkload. The probe was formatted into an electronicsurvey tool, which took minimal effort and would foranyone using relatively simple and familiar software. Itwas easy to implement because students completed theassessment via an e-mail link. It was only 5 items long andmultiple choice, so it placed little burden on students whochose to participate. Because it was electronically admin-istered, analyzing results and returning them to students

Figure 1. Change in Familiarity Over the Two 12-Month Long Assessment Intervals.

80


was easy. Finally, because the probe was administeredprior to class, results were analyzed and reported to stu-dents during the prelaboratory lecture, which also tooklittle time. The lack of time required to administer thisprobe is significant because when asked, most pharmacyinstructors elect not to incorporate active learning or as-sessment tools into didactic course work because theirbelief that it is too time intensive and integrated at thecost of lecture content outweighs their belief that it im-proves knowledge retention and student engagement.12

This concern expands beyond pharmacy faculty mem-bers to nursing faculty members, who also perceive itas a barrier to implementing critical thinking strategies.22

Instead of conducting the probe as a preclass assign-ment, one could administer the assessment during thebeginning of class using electronic “clickers,” which pro-vide instant responses. This method would likely notdetract much time from the lecture, possibly increasethe response rate (assuming all students attend the lec-ture), and potentially increase student satisfaction, atten-tiveness, and involvement.23 However, implementationof an in-class probe utilizing “clickers” would only bereasonable for the baseline time point and immediatefollow-up measurement.

Adding more time between completion of the probeand the prelaboratory lecture could allow instructors toadjust content to target knowledge gaps revealed in thebaseline results. Alternatively, instructors could usebaseline results to adjust laboratory (versus prelabora-tory) content to target knowledge deficits. However,course coordinators require faculty members to turn incontent for laboratories several weeks in advance, whichhinders acute adjustment of material. Still, the resultsallow for content of future years to focus on past studentdeficits in baseline knowledge, which in this case,appeared to revolve around every term except for“carbohydrate.”

SUMMARYThe adaptation of an anthropology background

knowledge probe was effectively translated to a clinical,diabetes-focused, pharmacy skills laboratory usingBloom’s Taxonomy as a guide. The probe showed im-provements in and retention of student familiarity andunderstanding of 5 diabetes related terms. This is the firstobjective assessment measuring knowledge retention us-ing a CAT-designed background knowledge probe.

ACKNOWLEDGMENTSKristen Helms, PharmD, BCSP, Associate Clinical

Professor at Auburn University and Sharon McDonough,PhD at University of Tennessee for their expertise in

education assessment techniques and Paul Jungnickel,PhD, RPh, Associate Dean and Professor for Academicand Student Affairs for his expertise in CAPE domaininterpretation and application.

REFERENCES1. Accreditation Council for Pharmacy Education, AccreditationStandards and Guidelines for the Professional Program in PharmacyLeading to the Doctor of Pharmacy Degree. 2011:39-40. https://www.acpe-accredit.org/pdf/FinalS2007Guidelines2.0.pdf. AccessedJuly 11, 2013.2. Zlatic TD. Abilities-based assessment within pharmacy education:preparing students for practice of pharmaceutical care. In: Wilkin,N.E., ed. Handbook for Pharmacy Educators: contemporary teachingprinciples and strategies. New York: Pharmaceutical Products Press,2000:5-27.3. Mihram D. Classroom Assessment Techniques. University ofSouthern California; Center for Excellence in Teaching. JohnHopkins Bloomberg School of Public Health. http://www.crlt.umich.edu/sites/default/files/resource_files/ClassroomAssessmentTechniquesHopkins.pdf. Accessed February17, 2014.4. Melland HI, Volden CM. Classroom assessment: linking teachingand learning. J Nurs Education 1998;37(6):275-277.5. Cross KP. Classroom research: implementing the scholarship ofteaching. Am J Pharm Educ. 1996;60:402-407.6. Angelo TA, Cross KP. Classroom Assessment Techniques: AHandbook for College Teachers. San Francisco, CA: Jossey-Bass, 1993.7. Weaver RL, Cotrell HW. Mental aerobics: The half-sheetresponse. Innovative Higher Educ. 1985;10(1):23-31.8. Simpson-Beck V. Assessing classroom assessment techniques.Active Learn High Educ. 2011;12(2):125-132.9. Davidson JE. Preceptor use of classroom assessment techniques tostimulate higher-order thinking in the clinical setting. J Contin EducNurs. 2009;40(3):139-143.10. Bowles DJ. Active learning strategies. . .Not for the birds! Int JNurs Educ Scholarsh. 2006;3(1):Article 22.11. Wise HH. Student perceptions of a lecture-based coursecompared with the same course utilizing a classroom assessmenttechnique. J Phys Ther Educ. 2004:18(1):75-79.12. Van Amburgh JA, Devlin JW, Kirwin JL, Qualters DM. A toolfor measuring active learning in the classroom. Am J Pharm Educ.2007;71(5):Article 85.13. Anderson HM, Moore DL, Guadelupe A, Bird E. Studentlearning outcomes assessment: a component of program assessment.Am J Pharm Educ. 2005;69(2):Article 39.14. Boyce EG. A guide for doctor of pharmacy program assessment.Alexandria, VA: American Association of Colleges of Pharmacy;2000. http://www.tsu.edu/academics/colleges__schools/College_of_Pharmacy_and_Health_Sciences/assessment/pdf/PharmDProgramAssessment.pdf. Accessed August 29, 2014.15. Bartlett MB, Morrow KA. Method for assessing courseknowledge in a large classroom environment: an improved version ofthe minute paper. Am J Pharm Educ. 2001;65:264-267.16. Burkiewicz JS, Bruce SP, Weberski JA, Ritter JL, Sohn AH. Pre-and post-rotation assessment of pharmacy student learning. J PharmTeaching. 2005;12(2):83-96.17. Brown BK, Watkins TA. Using formative content analysis toimprove learning outcomes in a pharmaceutical care course. J PharmTeaching. 2005;12(1):11-22.

81


18. Hegener MA, Buring SM, Papas E. Impact of a requiredpharmaceutical calculations course on mathematics abilityand knowledge retention. Am J Pharm Educ. 2013;77(6):Article124.19. Eley JG, Birnie C. Retention of compounding skills amongpharmacy students. Am J Pharm Educ. 2006;70(6):Article 132.20. Kopacek KB, Dopp AL, Dopp JM, Vardeny O, Sims JJ.Pharmacy students’ retention of knowledge and skills followingtraining in automated external defibrillator use. Am J Pharm Educ.2010;74(6):Article 109.

21. Morello CM, Neighbors M, Luu L, Kobayashi S, Mutrux B,Best BM. Impact of a first-year student pharmacists diabetesself-care education program. Am J Pharm Educ. 2013;77(10):Article 215.22. Shell R. Perceived barriers to teaching for critical thinking byBSN nursing faculty. Nursing and Health Care Perspectives.2001;22(6):286-291.23. Graeff EC, Vail M, Maldonado A, Lund M, Galante S, TataronisG. Click it: Assessment of classroom response systems in physicianassistant education. J Allied Health. 2011;40(1):e1-5.

82



An Active Learning Complementary and Alternative MedicineSession in a Self-Care Therapeutics Class

Melissa J. Mattison, PharmD, Eric C. Nemec, PharmD, BCPS

College of Pharmacy, Western New England University, Springfield, Massachusetts


Objective. To provide an interactive, non-supplement based complementary and alternative medicine(CAM) session in a self-care therapeutics class and to evaluate the effect of the session on pharmacystudents’ perceptions and knowledge of CAM.Design. Second professional year pharmacy students enrolled in a required 3-credit course titled Self-CareTherapeutics participated in an active learning session on CAM. Students physically engaged in 5 separateactive learning CAM sessions including massage therapy, Tai Chi, yoga, progressive muscle relaxation, andReiki.Assessment. Students were assessed on both knowledge and perception of CAM. Concept mastery wasassessed using a written examination and individual readiness assurance tests (iRAT) and team readinessassurance tests (tRAT). Perception of CAM was measured using both a presession and a postsessionsurvey.Conclusion. Participating in an intensive, active learning CAM session provided an opportunity to increasestudents’ knowledge of CAM and an effective strategy for providing the learner with the experience tobetter envision incorporation into patient therapies.

Keywords: Complementary and alternative medicine (CAM), active learning, self-care

INTRODUCTIONProducts and treatments labeled “alternative medi-

cine” byWestern medicine are actually predominant mo-dalities of health care in traditional Asian cultures orEastern medicine.1 Over the years, many people havebeen merging Eastern medicine with Western medicine,integrating CAM therapies with conventional medicaltreatments, but the former are not always embraced byWestern medicine.2 While healthy adults as well as illpatients use CAM to maintain or improve health and re-duce disease risk, they aren’t able to easily integrate itwith their traditional therapy because their health careproviders lack knowledge or awareness of it.2

Alternativemedicine or therapies are notwidely taughtin medical or pharmacy schools in the United States, nor arethey available at many American hospitals or health carefacilities.1 Studies surveying American pharmacy schoolsconcluded that while approximately 80% of schools offersome form of CAM training in the curriculum, CAM edu-cation was primarily offered as electives and generally fo-cused on natural products rather than the full range of CAM

practices.3-5 Many health care professionals perceive theirknowledge of CAM as limited and have little personal ex-perience with it.6 Pharmacists typically rate their ownknowledge of CAM as inadequate and feel uncomfortableanswering patient questions related to it.7-10

Yet the 2002 National Health Interview Surveyreported 50% of all adults in the United States have usedsome form of CAM.11 In 2007, adults in the United Statesspent nearly $34 billion on complementary and alternativemedicine Thanks Good Catch products, including naturalproducts, classes, and visits to CAM providers.12 Between2002 and 2007, increased use was seen among adults foracupuncture, deep breathing exercises, massage therapy,meditation, naturopathy, and yoga.13

Increased use of natural products and alternativetherapies could have a significant impact on the practiceof pharmacy in the United States, which would necessi-tate practitioners to be better versed in CAM and itscomplementary use in Western medicine.11 For exam-ple, potential drug interactions with CAM could becomea concern because some patients might not disclose theherbal medications they are taking. This in turn couldimpact health and the health care system by resulting inadditional adverse outcomes. Pharmacists are the mostaccessible health care professional, and if they need to

Corresponding Author: Melissa J. Mattison, 1215 WilbrahamRoad, Springfield, MA 01119. Tel: 413-796-2428. Fax: 413-796-2266. E-mail: [email protected]

83


field questions regarding CAM, they could positivelyimpact a patient’s health through counseling and an ad-equate CAM knowledge base.

Pharmacists perceive barriers to providingCAMguid-ance that include a lack of suitable training and deficienciesin available information sources. Recognizing this knowl-edge deficit, pharmacists have proposed ways to overcomethese barriers, such asmore extensive formal training.14 Forexample, the Accreditation Council for Pharmacy Educa-tion (ACPE) 2007 Standards for Accreditation recommen-ded didactic coursework in dietary supplements, alternativemedical treatments, evaluation of efficacy and safety ofCAM, and herbal-drug interactions, as well as exposure tothese products during advanced pharmacy practice experi-ences.15 Guideline 12 of the ACPE Standards focuses onprofessional competencies to provide patient care basedon therapeutic principles and evidence-based data and topromote health improvement, wellness, and disease pre-vention.15 Guideline 13.3 states that the college or schoolcurriculum should address issues related to a general broad-ening of perspective including communication skills, pro-fessionalism, critical thinking, problem-solving, health andwellness, patient safety, teamwork, mathematical skills,and information management. Moreover, the Center forthe Advancement of Pharmaceutical Education (CAPE)Outcomes also suggests students be well versed in tradi-tional and CAMmodalities for their APPEs so they are ableto recommend prescription and nonprescription medica-tions, dietary supplements, diet and nutrition guidance,traditional nondrug therapies, and complementary and al-ternative therapies.16

Current pharmacy school offerings vary, from a smallamount of CAM coursework to electives, to more compre-hensive coursework. In order for pharmacists to promotehealth, wellness, and disease prevention among patientsincreasingly inclined to use CAM, they need better formaltraining and experience. Patients seek out CAM either astherapy complementary to their traditional treatment oras alternative therapy that would encompass modalitiesother than traditional ones. Patients use CAM to promotea healthy lifestyle, achieve wellness, and reduce diseaserisk. Improving CAM offerings to students would not onlyhelp to meet such patient needs, but would also directlyalign with the needs addressed by the ACPE Standardsfor Accreditation and the CAPE recommendations.15,16

DESIGNAn immersiveCAMsessionwasoffered in the required

3-credit Self-CareTherapeutics class during the springof thesecond professional year. The instructional design of theself-care class included a combination of case-based didac-tic learning, team-based activities, and active learning

sessions. During the CAM session, learners participated in5 intensive, 15-minute physically activemini-sessions, eachincorporating a different CAM modality. Experts from ourinstitution and the community led learners through eachsession in their respective specialties at separate “CAMSta-tions.” Specialties includedmassage therapy,Reiki, TaiChi,yoga, and progressive muscle relaxation (PMR). Learnerswere expected to gain foundational knowledge, make con-nections between assigned readings and application, andfinally apply the knowledge to cases on their final exam.Session scheduling was based on practitioner availabilityand the willingness of volunteer practitioners (as there wasno guaranteed honorarium, which may have influenced therepresentation of CAM modalities).

Prior to the interactive CAM session, learners wereoffered a voluntary survey approved by the InstitutionalReview Board. They were then assigned readings fromtheir textbook, the Handbook of Nonprescription Drugs,and other materials provided by the various specialists.They also completed a case-based, CAM individual read-iness assurance test (iRAT). The iRAT was modified fromits traditional use of delivering it during class and wascompleted outside of class to allow for more time to par-ticipate in the CAM sessions. Throughout the semester theiRAT/tRAT (or team readiness assurance test)was given toassess the weekly self-care topics to ensure that learnerswere reading ahead and prepared for class. The surveyserved to assess learners’ perceptions and knowledge abouttheir understanding of, experience with, and comfort withincorporating CAM into a treatment. Additionally, the sur-vey wasmeant to assess if learners would bemore likely torecommend CAM to an appropriate patient after the in-tensive learning experience.

Sixty-nine learners participated and were divided intotwo sections, and further separated into groups of 6-7 tofacilitate interactive, small group learning experience. Toaccommodate this, the session required more than the nor-mal allotted time. Instructors were allowed to incorporateone of the professional development hours that precededthe Self-Care Therapeutics class into class time in order toallow enough time for eachCAMmini-session andgive 1.5hours to each of the 2 sections. Learners rotated through 15minutes of experiential time in each CAM station andwereallotted 2 minutes to change stations. A facilitator helpedcoordinate the rotations.

Before participating, students signed a waiver thatmitigated outside practitioner responsibility. Due to thephysical component (actually doing yoga, practicing TaiChi, etc), participationwas optional. For learners unable orunwilling to participate, an alternative written CAM exer-cise was designed to enhance learner knowledge. Twolearners opted for the written assignment. The alternative

84


assignment involved searching the primary literature forquality, evidence-based documentation on the benefits ofCAM and which disease states may be impacted by CAM.As the CAM session occurred at the end of the semester,case-based questions were included on the final exam. Thequestions related to the history of the modalities and theirapplication to patient cases, which illustrated the relevanceof CAM in Western society and the impact a pharmacistcould have on wellness and disease states.

Within 48 hours of completing each CAM station, stu-dents submitted a written reflection on the active learningexperience.At thebeginningof thenext class session (2dayslater), students were asked to complete a follow-up surveyand take a tRAT. Conducting the tRAT after the sessiongave students more uninterrupted time with the experientialpart. Studentswere tested on the information in a case-basedformat on the final exam.

EVALUATION AND ASSESSMENTPerformance was assessed using presession iRATs and

postsession tRATs (Table 1). Postsession scores improvedsignificantly, with the entire class receiving an “A.” Highstakes assessments and a comparison of questions based onassigned readings and the interactive session indicated thatlearners performed better on case-based application typequestions relating directly to their CAM session. Althoughlearners hadassigned readings that coveredhistorical aspectsof CAM, they did significantly worse on this portion. Forexample, 61.4% answered the Tai Chi history question cor-rectly versus 92.8% who answered the Tai Chi case-basedapplicationquestion correctly.Webelieve these assessmentsillustrate the value and impact of interactive sessions onlearner performance.

The presession survey response rate was 93% (N567),and the postsession survey response rate was 78% (n554).Learners who opted out of the session did not complete thesurvey. In the presession survey, 30% (n521) of learnersreported they were not familiar with any CAM modality.After the session, only 9% (n56) reported being unfamiliarwith CAM (p50.02). For the purpose of the survey, authorsconsidered learners to be familiar with the concept if theyconfirmed familiarity anddemonstrated the ability to specifyCAM modalities in an open-ended follow-up question.Of the 69.7% (n546)who reported familiarity or experiencewith CAM, 64% (n529) reported their knowledge was

limited to two or less separate modalities, specifically mas-sage therapy and/or yoga. The lack of awareness regardingother Eastern or alternative medicine modalities commonlyemployed by patients for health maintenance was unex-pected.Yoga andmassage are ubiquitous inWestern cultureand used to some degree in Western medicine. The preses-sion survey did not categorize if the lack of familiarity wasdue to aknowledgegappreventing students from identifyingyoga and massage as CAM, or if students were genuinelyunaware of their existence.

Perceptions regarding the role of CAM in health carechanged after the intensive session (Table 2). Prior to thesession, 54.7% (n535) of participants responded that theyfelt CAM did not fit into the current health care model;whereas after, 85.2% (n546) felt that it did (p,0.05). Al-most all of the postsession survey respondents (96.2%,n551) affirmed that participating in the CAM sessionimproved their understanding compared to just readingassigned text.We believe that experiencingCAMfirst handgave students a deeper understanding of its role and empa-thy formodalities theymay not have fully understood in thecontext of Western medicine. This was illustrated in thepostsession surveywhere 100%of respondents (n554) saidthey would recommend CAM to a patient if their conditionmight benefit from it.

In the postsession survey, 94.3% (n550) of respon-dents said they benefited personally and 96.3% (n552)benefited professionally from the experience. Positive feed-back was provided by 98% (n569) of students, who de-scribed their changed perceptions and new enthusiasm fora nontraditional classroom experience. This attitudinal scaleshowed a deeper understanding and increased empathy forCAM. Survey results also demonstrated a greater awarenessamong students of various CAM modalities, with 89% ofthem being able to list 4 or more modalities compared to64% who were able to list 1 or 2 prior to the session.

DISCUSSIONThe intensive, interactive approach improved attitudes

regarding CAM and learner performance on assessment.Other institutions that include CAM in their curriculummight want to consider providing a similar session to im-proveknowledgeabout andattitudes towardCAM.Learnersreported that after the session they perceived reduced stress,an improved sense of well being, and decreased anxiety.These reflections allude to students’ ability to extrapolatethis personal benefit to the professional setting and the ben-efits their patients may derive from CAM. As one learnerstated, “I found this a great tool. I think thiswas away to helpmake us aware of alternative medicine because our patientswill be using these techniques or may have questions aboutthem.”Many of the learners vowed to begin aCAMpractice

Table 1. Individual Readiness Assurance Test (iRAT)/TeamReadiness Assurance Test (tRAT) Grade Distribution

Letter Grade A (90%) B (80%) C (70%) F (,70%)

iRAT 14 47 0 9tRAT 71 0 0 0

85


on their own to help manage the stress of pharmacy school.One learner reflected “great learning opportunity for us.”The practitioners who presented the CAM modalities wereamenable to teaching our classes, even without the promiseof anhonorarium.Theywerepleased tohave theopportunityto present their respective discipline to pharmacy learnersand hoped to expand the learners’ understanding of CAMand its role in health care today. They also reported leavingthe session enthusiastic and ready to incorporate new mo-dalities into their own practices or revisit old ones.

A limitation to the CAM sessions/this study was theavailability of the practitioners. Acupuncture, for example,was mentioned by several learners as something theywould benefit fromexperiencing. Though an acupuncturistwas scheduled, hewas unable to attendon the actual day. Inthe future we plan to incorporate additional modalities.Further limitations include a decreased postsession surveyresponse rate, which prevented us from quantifying theimpact on all learners in the course.

SUMMARYCAM is an important therapeuticmodality that can be

used in treatment; however pharmacists need appropriateexposure to it in their education to accurately recommend itor augment current therapy with it. The purpose of thissession was to improve assessment performance and atti-tudes regarding CAM by having learners experience first-hand what CAM encompasses. After participating in theactive learning session incorporating 5 different modalitiesof CAM, students increased their knowledge and under-standing of a variety of CAM, the role it plays the currentWestern medical model, and the role pharmacists can playin recommending CAM to patients. The interactive CAMsession was valuable and added a new dimension to thelearners’ educational experience.

REFERENCES1. Dutta AP, Miederhoff PA, Pyles MA. Complementary andalternative medicine education: students’ perspectives. Am J PharmEduc. 2003;67(2):1-7.2. Ulbricht C, Chao W. Common complementary and alternativemedicine health systems. In: Krinsky DL, ed. Handbook ofNonprescription Drugs. 17th ed. Washington, D.C.: AmericanPharmacists Association; 2012:1007-1018.

3. Dutta AP, Daftary MN, Egba PA, Kang H. State of CAMeducation in U.S. schools of pharmacy: results of a national survey. JAm Pharm Assoc. 2003;43(6):81-83.4. Shields KM. Natural product education in schools of pharmacy inthe United States. Am J Pharm Educ. 2003;67(10):43-48.5. Mackowiak ED, Parikh A, Freely J. Herbal product education inUnited States pharmacy schools: core or elective program? Am JPharm Educ. 2001;65(1):1-6.6. Wong LY, Toh MP, Kong KH. Barriers to patient referral forcomplementary and alternative medicines and its implications oninterventions. Complement Ther Med. 2010;18(3-4):135-142.7. Dolder C, Lacro J, Dolder N, Gregory P. Pharmacists’ use of andattitudes and beliefs about alternative medications. Am J Health-SystPharm. 2003;60(13):1352-1357.8. Koh HL, Teo HH, Ng HL. Pharmacists’ patterns of use,knowledge, and attitudes toward complementary and alternativemedicine. J Altern Complement Med. 2003;9(1):51-63.9. Harris IM, Richard RL, Rodriguez R, Choudary V. Attitudestowards complementary and alternative medicine among pharmacyfaculty and students. Am J Pharm Educ. 2006;70(6):1-8.10. Tiralongo E, Wallis M. Integrating complementary andalternative medicine education into the pharmacy curriculum. Am JPharm Educ. 2008;72(4):1-9.11. Evans E, Evans J. Changes in pharmacy students’ attitudes andperceptions toward complementary and alternative medicine aftercompletion of a required course. Am J Pharm Educ. 2006; 70(5):1-7.12. Nahin RL, Barnes PM, Stussman BJ, Bloom B. Costs ofcomplementary and alternative medicine (CAM) and frequency of visits toCAM practitioners: United States, 2007. National health statistics reports;no 18. Hyattsville, MD: National Center for Health Statistics. 2009. http://www.cdc.gov/nchs/data/nhsr/nhsr018.pdf Accessed Feb 24, 2014.13. Barnes PM, Bloom B, Nahin RL. Complementary and alternativemedicine use among adults and children: United States, 2007.National health statistics reports; no 12. Hyattsville, MD: NationalCenter for Health Statistics. 2008. http://www.cdc.gov/nchs/data/nhsr/nhsr012.pdf. Accessed Feb 24, 2014.14. Semple SJ, Hotham E, Rao D, Martin K, Smith CA, BloustienGF. Community pharmacists in Australia: barriers to informationprovision on complementary and alternative medicines. PharmWorldSci. 2006;28(6):366-373.15. Accreditation Council for Pharmacy Education. Accreditationstandards and guidelines for the professionals program in pharmacyleading to the doctor of pharmacy, version 2.0. Feb 14, 2011.Available from: http://www.acpe-accredit.org/pdf/FinalS2007Guidelines2.0.pdf. Accessed July 1, 2013.16. Accreditation Council for Pharmacy Education. Accreditationstandards and guidelines for the professional program in pharmacyleading to the doctor of pharmacy degree. 2007. http://www.acpe-accredit.org/pdf/ACPE_Revised_PharmD_Standards_Adopted_Jan152006.pdf. Accessed Aug 28, 2014

Table 2. Results of Classroom Survey Regarding Learner Knowledge and Perceptions of CAM

QuestionPresession % Yes

(N=67)Postsession % Yes

(n=54) p

Are you familiar with complementary and alternativemedicine (CAM)?

69.7 (n546) 90.7 (n549) p50.02

Does CAM fit into our current health care model? 52.2 (n535) 77.8 (n542) p,0.05Does the pharmacist have a role in CAM? 56.7 (n538) 85.2 (n546) p,0.05

86



A Multiyear Analysis of Team-Based Learningin a Pharmacotherapeutics Course

June Felice Johnson, PharmD,a Edward Bell, PharmD,a Michelle Bottenberg, PharmD,a

Darla Eastman, PharmD,a Sarah Grady, PharmD,a Carrie Koenigsfeld, PharmD,a

Erik Maki, PharmD,a Kristin Meyer, PharmD,a Chuck Phillips, PharmD, PhD,a

Lori Schirmer, PharmDb*

a Drake University College of Pharmacy and Health Sciences, Des Moines, Iowab Fort Sanders Regional Medical Center, Knoxville, Tennessee* Author affiliation at time of study was Drake University.


Objectives. To evaluate the impact of team-based learning (TBL) in a pharmacotherapeutics course onpharmacy students’ ratings of faculty instructors and the course, and to assess students’ performanceafter implementation of team-taught TBL.Design. Teaching methodology in a pharmacotherapeutics course was changed from a lecture with reci-tation approach in 2 semesters of a 6 credit-hour course to a TBL framework in a 3-semester 31415 credithour course. The distribution of faculty of instruction was changed from 4 faculty members per week to 1faculty per 1-credit-hour module. TBL consisted of preclass study preparation, readiness assurance (In-dividual Readiness Assessment Test and Group Readiness Assessment Test), and in-class applicationexercises requiring simultaneous team responses.Assessment. Retrospective analysis of student ratings of faculty and instructional methods was conductedfor the 2 years pre-TBL and 4 years during TBL. Final course grades were evaluated during the same timeperiod. Student ratings showed progressive improvements over 4 years after the introduction of team-basedlearning.When aggregated, ratings in the “excellent teacher” category were unchanged with TBL comparedto pre-TBL. Improvements in faculty instructor approaches to teaching were noted during TBL. Groupgrades were consistently higher than individual grades, and aggregate course grades were similar to thoseprior to TBL implementation.Conclusion. Implementation of TBL in a pharmacotherapeutics course series demonstrated the value ofteam performance over individual performance, indicated positive student perceptions of teaching ap-proaches by course faculty, and resulted in comparable student performance in final course gradescompared to the previous teaching method.

Keywords: team-based learning, pharmacotherapeutics, student evaluations, faculty performance, studentperformance

INTRODUCTIONTransformation of health care delivery models so

that they improve quality and reduce the cost of patient-centered care will require that pharmacists take new ap-proaches to care coordination, team-based care, andchronic disease management, and that they assume newhealth delivery roles.1-4 These new collaborative models

require colleges and schools of pharmacy to ensure thatstudents are prepared to problem-solve effectively as val-ued members of a patient-centered care team. The Accred-itation Council for Pharmaceutical Education (ACPE)Standards 2.0 emphasize that pharmacy programs graduatestudent pharmacists who “. . .can contribute to the care ofpatients and to the profession by practicing with compe-tence and confidence in collaboration with other healthcare providers.”5 Team-based learning in the pharmacycurriculum has been described as a pedagogy that can ad-dress this type of professional preparation.6-16

Team-based learning is a specific instructionalstrategy and framework that uses intentionally formed

Corresponding Author: June Felice Johnson, Professor ofPharmacy Practice, Drake University College of Pharmacyand Health Sciences, 2507 University Avenue, Des Moines,Iowa, 50311. Tel: 515-271-1849. Fax: 515-271-4171. E-mail:[email protected]

87


teams of learners to deepen student learning and develophigh-performing teams. The core principles of TBL are:(1) learning groups must be formed and managed; (2)students must be held accountable for the quality of boththeir individual work and the group’s work; (3) studentsmust receive frequent and timely feedback on theirlearning; and (4) team assignments must promote bothlearning of the content and development of the team.17

Team-based learning comprises 3 major steps: (1)individual study and preparation; (2) readiness assurance,completion of an individual readiness assessment test(iRAT) in class, followed by the same test as a group,and completion of the group readiness assessment test(gRAT); and (3) application exercises in class whereteams work together to solve problems using informationgained from the previous 2 steps. Each student evaluatesthe other members of the group through graded peer eval-uations during the semester.

The TBL model has been used extensively in busi-ness in the 1970s and, more recently, in health scienceseducation, with improved learning outcomes in collegesand schools of pharmacy and other health professions.18-28

Reports of TBL in pharmacy education often use studentand faculty perception or performance to examine theinitial implementation of TBL. However, no reports haveused nationally standardized faculty evaluations of teach-ing using TBL, examined experiences over a longitudinalimplementation of TBL, or reported course outcomes ina pharmacotherapeutics course series that used TBL.More data are needed for faculty members to understandthe longitudinal impact of implementing TBL in a re-quired team-taught course.

Medical educators have increasingly advocated foractive-learning strategies that allow students to applyproblem-solving skills and learn collaboratively. How-ever, student performance and attitudes towards thesemethods are often mixed, and comparisons are not possi-ble because of the wide variation in teaching modelsused.20,21,24,26,29

The Drake University College of Pharmacy andHealth Sciences (DUCPHS) practice faculty membersadopted TBL in 2009 during a college-wide curricularrevision in 2008 and in tandem with the release of ACPEStandards 2007. This study describes a multi-year expe-rience with TBL in a required pharmacotherapeuticscourse series at DUCPHS. The primary objective of thisstudywas to evaluate the impact of implementing TBL onfaculty and course evaluations, and the secondary objec-tive was to measure student performance before and aftercurriculum revision.

We hypothesized that student evaluations of facultymembers and the pharmacotherapeutics course series

would remain relatively stable and measures of studentengagement with learning would increase with TBL. Theauthors also hypothesized that students’ course performancewould not be negatively impacted and that team grades(gRATs) would outperform individual grades (iRATs).

DESIGNThis project was approved as exempt by the Drake

University Institutional Review Board. At DUCPHS, theprevious model for classroom instruction of pharmacother-apeutics was the live lecture, which focused on deliveringcontent to the entire class, followed in the same week bysmaller case-based discussion group sessions. However,faculty members were not satisfied with the students’engagement with the material, preclass preparation, orpostlecture retention of concepts. These assessmentsweremade informally and consistently between facultymembersover the years through direct observation of student parti-cipation in class, in discussions during annual therapeu-tics retreats, and during direct interaction with studentsduring their practice experiences. Workload for facultymembers teaching in the course was demanding, requir-ing both a content expert for the lecture component and3 additional faculty facilitators to conduct recitation groupseach week. A comprehensive revision of the professionalcurriculum at DUCPHS to meet ACPE Standards 2007offered an opportunity to explore a new pedagogy thatwould enhance student learning and strengthen develop-ment of high-performing teams. Concurrently, deliveryof the course was changed from a 2-semester fall-springsequence for third-year (P3) students to a 3-semestersequence beginning in the spring for second-year (P2)students and fall and spring for P3 students; this sequencemore evenly distributed course credit hours (Figure 1).Course faculty members explored new teaching methodsand developed a set of criteria that any new method usedwould have to meet: increase student accountability forlearning; increase active learning; develop life-long learn-ing skills; increase retention of knowledge; increase con-tact with individual faculty members; and decrease thenumber of faculty members involved in each semester bycreating 1-credit hour blocks (modules) for which eachfaculty member was responsible.

The Michaelsen model for TBL was selected as anactive learning strategy because it best fit the needs of thecourse in preparing students for professional roles, em-phasized engaged learning and accountability for learn-ing, and reinforced the value of strong team performanceon learning and problem solving.17 Another active learningstrategy considered was problem-based learning (PBL),which was first used in medical education in the 1960s.This instructional strategy introduces a problem to the

88


students at the beginning of an instructional module andthen requires students to self-discover answers progres-sively through repeated interaction with a faculty mem-ber. The DUCPHS practice faculty members decided thatPBL would be time-intensive and require more timeteaching outside of class, adding to the challenges ofwork-life balance and faculty member success in scholar-ship and service. The effects of PBLon learning outcomeswere equivocal.29 Previous experience with a lecture pluscase-based discussion did not always meet faculty expec-tations for learning outcomes and was demanding of fac-ulty teaching time. Based on the extensive literature onTBL, efficiency of course delivery, and potential for im-proved outcomes of both student learning and studentattitudes towards learning, the Michaelsen model forTBL was selected. At the time, neither the students northe DUCPHS faculty had any previous experience withTBL in the curriculum, though various active learningstrategies such as recitation groups and case-based dis-cussion had been used. The change in course pedagogywas approved by DUCPHS faculty members in 2008 andpreparation began for delivery of the revised course inspring 2009.

All faculty members at DUCPHS attended a work-shop 1 year prior to implementation; however, only thepharmacy practice faculty members decided to fullyadopt this model in its entirety for the course. Pharmaco-therapeutics faculty members held additional self-directed, hands-on practice sessions prior to implementingthis pedagogy. A number of practice faculty membersjoined the TBL Collaborative to access additional re-sources and for ongoing support from TBL experts aroundthe country.

The TBL course was taught live as a 3-semester se-quence to the P2 and P3 classes of doctor of pharmacy(PharmD) degree students on the university campus. Thenumber of hours spent in class reflected the course credithours: multiple modules equal to 1-credit hour were taughteach semester, and only one faculty expert taught amodulein one semester of the course (Figure 1). The first TBLoffering was held in a traditional-style classroom locatedwithin the college that was not conducive to group work,so this room was subsequently updated to better supportengaged active learning.Twelve pharmacypractice facultymembers taught in the course series. All of the facultymembers maintained a clinical practice in an area of

Figure 1. Pharmacotherapeutics course structure before and during team-based learning

89


expertise, had completed postgraduate residency training,and had at least 1 advanced credential (eg, Board CertifiedPharmacotherapy Specialist).

EVALUATION AND ASSESSMENTStudent ratings of the course and faculty members

were used to support the main hypothesis for this study. AStudent Ratings of Instruction system, created by the non-profit IDEACenter, which focused on student learning of12 different objectives and factored out extraneous cir-cumstances, was used to assess the course. The systemalso enabled students to rate faculty instruction on 20different items.30 Students completed these evaluationsat the end of each faculty member’s section during bothcourse iterations.

The IDEACenter system has been used byDUCPHSsince 2004 for course and faculty evaluations. These eval-uations are completed either in class on paper or outside ofclass online. Because of this history, both faculty mem-bers and students were comfortable with the system dur-ing the time TBL was introduced. Overall, the facultyevaluation process remained the same both before andafter TBL implementation. Using the IDEA Center sys-tem, course coordinators selected the objectives believedintegral to the course. Prior to TBL implementation, fac-ulty teaching in pharmacotherapeutics selected “gainingfactual knowledge,” “learning fundamental principles,”“learning to apply course material,” and “learning to an-alyze and critically evaluate ideas, arguments, and pointsof view” as key objectives for the course. For the TBLsequence, key objectives were “gaining factual knowl-edge,” “learning to apply course material,” and “acquir-ing skills in working with others as a member of a team.”Progress on these objectives was rated by the students ona 5-point Likert scale (15no apparent progress, 25 slight

progress, ie, “I made small gains on this objective,”35moderate progress, ie, “I made some gains on this objec-tive,” 45substantial progress, ie, “I made large gains on thisobjective,” 55exceptional progress, ie, “Imade outstandinggains on this objective.”

The IDEAcourse evaluation systemwas used formea-sures of the primary study outcomes. This system has beenshown to be both valid and reliable for course-related out-comes and student progress. For evaluation, each cycle ofthe course sequence was treated as a discrete unit for com-parison. This produced 6 cohorts of data (2 years of thecourse sequence prior to TBL and 4 years using TBL forcomparison.

Student ratings for 5 specific approaches to instruc-tion were compared to support the assertion that funda-mental teaching changes took place in the course sequence(Table 1). Pre-TBL andTBL courseswere compared usingStudent t tests for these 5 independent activities. Inaddition, course assignments (reading and nonreadingassignments) and student ratings for the difficulty of sub-ject matter were compared across the pre-TBL and TBLcourses using student t tests in an attempt to further supportchanges in format rather than subject matter.

Student progress was analyzed using the 5 IDEAcourse objectives identified by the faculty as being theobjectives theywould focus on during the course sequence.Because these course objectiveswere highly correlated andexpected to be simultaneously affected by TBL, multivar-iate analysis of variance (MANOVA)was used to evaluatedifferences across the 6 study years. Although these ratingsconsisted of ordinal data, graphical representation showedadequate dispersion across the scale to warrant this para-metric approach. When analyzing independent constructsacross the 6 years (such as excellent course ratings andexcellent teacher ratings), and when comparing a total

Table 1. Pharmacy Student Ratings of Changes in Instructor Teaching Approaches Pre-Team-based Learning (Pre-TBL) andTeam-based Learning (TBL) Implementation

ApproachPre-TBL,Mean (SD)

TBL,Mean (SD)a,b

Scheduled course work (class activities, tests, projects) in waysthat encouraged students to stay up-to-date in their work

4.0 (0.3) 4.4 (0.3)

Formed “teams” or “discussion groups” to facilitate learning 3.5 (0.6) 4.8 (0.1)Involved students in “hands-on” projects such as research,

case studies, or “real life” activities3.9 (0.4) 4.3 (0.4)

Asked students to help each other understand ideas or concepts 3.7 (0.4) 4.4 (0.3)Gave projects, tests, or assignments that required original or

creative thinking2.8 (0.3) 3.2 (0.5)

a Ratings based on Likert scale of 15hardly ever, 25occasionally, 35sometimes, 45frequently, 55almost always.b All significant at 0.003 or less for student t test.

90


score across the 5 objectives, analysis of variance(ANOVA) was used with conservative Scheffe multiplecomparison testing. SPSS (IBM Corp. Released 2012.IBM SPSS Statistics for Macintosh, Version 21.0.Armonk, NY: IBM Corp.) was used for all analyses withan a priori alpha of 0.05.

Two student focus groups were conducted at the endof the spring semester in 2009 and 2010. The focus groupswere conducted by an individual trained in qualitativefocus group analysis who met with 6 P2 students enrolledin each spring semester. In response to themes extractedfrom these focus groups, purposeful action was taken toimprove clarity of expectations, to improve connectionsbetween objectives and study questions, and to enhancedetails on module objectives. Student organizations sub-sequently provided leadership by presenting mock TBLsessions to inform students on the process and benefits.Minimal changes weremade in the amount of reading andpreparation required for the course.

Peer evaluations were conducted each semester,but reduced from 3 to 2, ensuring that students continuedto receive valuable feedback while minimizing timeburden. Manual tabulation was used initially. iPeer, ver-sion 3.08 (open-source software, University of BritishColumbia), was implemented in 2012 to streamline theprocess.31

Student performance during TBL was assessed bycomparing gRAT with iRAT performance. On average,3 iRATs or gRATs were given in each module. Eightypercent of the final course grades in TBL reflected indi-vidual performance, 15% reflected team performance,and 5% reflected peer evaluations. In the course pre-TBL, 80% of the grade was based on individual perfor-mance on examinations, and 20% on recitation and peerevaluations. Final course grades in the 2 years pre-TBLwere compared to those in the 4 years TBL was used.

Across the 6 years the course sequence was evalu-ated, the mean (SD) class size was 119611.8 (range102-135) students. The IDEA Center student evaluationresponse rates were reliable and averaged 55.6% (range18-99%) with higher in-class paper response rates. Stu-dents’ perceptions of course format suggested that theapproaches to teaching the courses did, in fact, changeafter TBLwas implemented. Students in the TBL coursenoted significantly greater emphasis on 5 preselectedteaching approaches that were consistent with TBL(Table 1). Students also noted that the amount of readingincreased in the TBL years while the amount of othernonreading assignments decreased. Overall, the studentsdid not see a difference in the difficulty of the subjectmatter from pre-TBL to TBL implementation in thecourse (Table 2).

When viewing the cohort of instructors across the 6years, there was no significant difference in aggregatescores for the item, “Overall, I rate this instructor an ex-cellent teacher” (F51.988, p50.09). Scores for the 6cohorts ranged from 3.6 to 4.3 on a 5-point scale(15definitely false, 25more false than true, 35in be-tween, 45more true than false, 55definitely true). Whenviewing the entire course, 1 year did show a significantdecrease in student ratings for, “Overall, I rate this courseas excellent” (F58.93, p5,0.001). A Scheffe multiplecomparison test for the statement, “Overall, I rate thiscourse as excellent” found a mean of 4.1 in both pre-TBL years 1 and 2, and means of 3.5, 4.0, 4.4, and 4.3in years 3 to 6, respectively, of TBL implementation.While the third year was significantly different from allother years, the fifth and sixth years demonstrated pro-gressive improvement in responses compared to pre-TBLor the first 2 years of TBL implementation.

The MANOVA did reveal significant differencesbetween various years and student progress on IDEAcourse objectives (Table 3). For gaining factual knowl-edge and learning to apply course material, year 3 had thelowest progress ratings and was significantly lower thanyears 5 and 6. Both of these objectives were identified askey objectives in both pre-TBL and TBL implementationyears. Similarly, the learning fundamental principles ob-jective varied among class years. Again, year 3 (the firstyear of TBL) received the lowest progress ratings andwasstatistically lower than years 5 and 6. The first pre-TBLyear also received significantly lower progress ratingsthan year 6.

The IDEA objective of acquiring skills in workingwith others as a member of a team was used as a coursefocus for only the TBL years. As expected, this objectiveshowed different progress scores from pre-TBL to TBLyears. The lowest score on this objective was in year 1,followed by year 2. The pre-TBL years showed signifi-cantly lower progress than all 4 TBL years. In addition,the first year of TBL implementation (year 3) also showed

Table 2. Analysis of Course Assignments and SubjectDifficulty Prior to and After Implementation of Team-basedLearning (TBL)

Course MaterialaPre-TBL(SD)

TBL(SD) P

Amount of reading 3.8 (0.2) 4.3 (0.3) ,0.001Amount of work in other

(non-reading) assignments4.0 (0.2) 3.8 (0.3) 0.001

Difficulty of subject matter 4.3 (0.2) 4.3 (0.3) 0.69a Based on Likert scale of 15much less than most courses, 25lessthan most courses, 35about average, 45more than most courses, and55much more than most courses.

91


lower progress as compared to the last year of TBL (year6). Finally, the pre-TBL courses focused on the IDEAobjective of learning to analyze and critically evaluateideas, arguments, and points of view. In a comparisonof the 6 years, year 6 had the highest progress rating. Thiswas significantly higher than all but year 4.When analyz-ing the overall measure of progress on the 5 objectives(total score on the 5), ANOVA showed significant prog-ress in the last 3 years using TBL. Study years 4, 5, and 6showed higher total scores than year 1, while year 6 wasalso significantly higher than years 2 and 3 (Table 3).

As postulated in the TBL model, and as the authorshypothesized, students scored consistently better on theirgRATs in comparison to their iRATs each year the coursewas offered (Table 4). Overall, final course grades pre-TBL and during TBL were similar, though there wasa higher proportion (5.3%) of D grades in the first coursein TBL year 1 (Table 5).

DISCUSSIONThe study findings supported our main hypothesis that

student evaluations would remain relatively stable andmea-sures of student engagement with learning would increasewith TBL. Subjective IDEA Center student evaluations offaculty members and the course did not change drasticallyand improved in some areas when comparing pre-TBL toTBL evaluations over multiple years. An overall rating ofthe 5 course objectives showed improved student progress

andprogressive improvements over time.Ratings for facultymembers and the course as “excellent teacher” and “excel-lent course”were similar,with the exception of a decrease inthe first year of TBL implementation. Because students hadno previous exposure to TBL before this course, unfamiliar-ity with this pedagogy likely contributed to lower ratings thefirst year. Additionally, all faculty members were new to

Table 3. Progress Scores for IDEA Course Objectives Across the Six-year Period

Pre-Team-basedLearning(SD)

Team-based Learning (SD)Implementation

Objectivea Year 1 Year 2 Year 3 Year 4 Year 5 Year 6

Gaining factual knowledge (terminology,classifications, methods, trends)

4.1 (0.3) 4.3 (0.4) 4.0 (0.5) 4.3 (0.4) 4.5 (0.3)b 4.5 (0.2)b

Learning to apply course material(to improve thinking, problem solving,and decisions)

4.0 (0.4) 4.1 (0.4) 3.8 (0.5) 4.2 (0.4) 4.4 (0.3)b 4.5 (0.2)b

Learning fundamental principles,generalizations, and theories

3.9 (0.3) 3.9 (0.4) 3.6 (0.5) 4.1 (0.3) 4.3 (0.2)b 4.4 (0.2)b,c

Acquiring skills in working with othersas a member of a team

2.5 (0.3)b,d 3.2 (0.4)c 3.6 (0.3)c,d 4.0 (0.4)c,d 3.8 (0.3)c,d 4.1 (0.3)b,c,d

Learning to analyze and critically evaluateideas, arguments, and points of view

2.8 (0.3) 2.8 (0.4) 2.8 (0.4) 3.1 (0.4) 2.9 (.40) 3.5 (0.3)b,c,d,e

Total: 17.3 (1.5) 18.3 (1.9) 17.8 (2.2) 19.7 (1.8)c 20.0 (1.5)c 21.0 (2.1)b,c,d

a Based on Likert scale of: 15no apparent progress, 25slight progress (I made small gains on this objective), 35moderate progress (I made somegains on this objective), 45substantial progress (I made large gains on this objective), and 55exceptional progress (I made outstanding gains onthis objective).b Significantly different from year 3.c Significantly different from year 1.d Significantly different from year 2.e Significantly different from year 5.

Table 4. Grade Summary of Individual Readiness AssessmentTest and Group Readiness Assessment Test Scores Over 4Years of Team-based Learning Implementation ina Pharmacotherapeutics Course Series

AssessmentTest

Class2011a

Class2012

Class2013

Class2014

PHAR 190iRAT 80 78.9 83.4 82.6gRAT 96.9 97.9 98.7 99.1

PHAR 191iRAT 82.2 79.7 81.1 81.1gRAT 98.6 96.7 97.6 97.1

PHAR 192iRAT 82.5 80.3 81.1 78.3gRAT 97.6 97.1 97.9 97.4

a Graduation year of class enrolled in TBL PHAR 190 course inspring 2 years previous.Abbreviations: iRAT5 Individual Readiness Assessment Test,gRAT5 Group Readiness Assessment Test. PHAR1905Therapeutics I; PHAR 1915Therapeutics II; PHAR1925Therapeutics III.

92


TBL, which also may have contributed to the first TBL yearratings. Over time, increased practice in the pedagogy andconstructive responses to student feedback resulted in animproved course.

These findings were consistent with studies thatshowed similar or higher student ratings over time whencomparing TBL to other learning methods.8,11,14,24 Insupport of our secondary hypothesis, this study showedimprovement in team performance over individual per-formance and comparable course performance, whichwas also consistent with previous studies.7,10,13,14

Lower-performing students benefited more from TBL,or had observed no differences in performance betweencase-based learning and TBL.22,25 The proportion of Dand F grades was low in both iterations of this course,though a higher proportion of D grades occurred in thefirst year of TBL. Perhaps these students struggled morein adapting to TBL. In subsequent TBL years, the pro-portion of low grades was comparable to pre-TBL; how-ever, the small numbers of students in this categorypreclude definite conclusions.

Although other studies have examined multi-yearexperiences, to our knowledge this study was the first tofocus on IDEA Center teaching evaluations over time. Inaddition, we used a “pure” TBL model instead of usingselect components of TBL as many published hybridmodels have done.

Most of the pharmacotherapeutics course facultymembers were “seasoned veterans,” who had taught the

course for more than 5 years and had previously usedthe more traditional model of lecture and case discus-sion. These factors likely contributed to faculty mem-bers’ ability to successfully adapt to the TBL model.Involvement of faculty champions played a key rolein faculty and administrative buy-in, course develop-ment, and implementation. Adequate faculty prepara-tion and practice sessions supported successful coursedelivery.

Teaching circles of all faculty teaching in the courseper semester were formed and met regularly to discussideas, share success stories, and brainstorm on strategies.An annual faculty retreat allowed course facultymembersto review course statistics, ensure coordination of topics,and establish common policies and procedures betweensemesters. Summary themes from 2 formal focus groupsessions were shared and discussed at the retreats duringthe initial years of TBL.College administration supportedthe course innovation and provided reassurance thatcourse evaluation scores in the initial years would beevaluated within the context of change. Academic phar-macy support for TBL implementation can also be foundin the most recent Center for the Advancement of Phar-maceutical Education outcomes, with specific emphasison problem solving and communication.32

This change in educational methodology was ini-tially time intensive, requiring content revision, doubleteaching for 1 year, and conversion from a 2- to 3-semester model. Students may have felt they were in

Table 5. Summary of Final Pharmacotherapeutics Course Series Grades Over Six Years Prior to and After Team-based LearningImplementation

Final Course Grades

Pre-Team-based Learning Team-based Learning Implementation

Year 1 Year 2 Year 3 Year 4 Year 5 Year 6

PHAR 190 N No. (%) 134 103 114 110 125 128A 34 (25.4) 35 (34) 37 (32.5) 25 (22.7) 38 (30.4) 46 (35.9)B 80 (59.7) 57 (55.3) 68 (59.6) 74 (67.3) 80 (64) 72 (56.3)C 19 (14.2) 11 (10.7) 3 (2.6) 12 (10.9) 6 (4.8) 9 (7)D 0 0 6 (5.3) 2 (1.8) 1 (0.8) 1 (0.8)F 1 (0.7) 0 0 0 0 0

PHAR 191 N No. (%) 114 103 120 129A N/Aa N/A 36 (31.6) 39 (36.8) 49 (40.8) 52 (40.3)B N/A N/A 72 (63.2) 66 (62.3) 66 (55) 67 (51.9)C N/A N/A 5 (4.4) 1 (0.9) 5 (4.2) 10 (7.8)D N/A N/A 1 (0.8) 0 0 0F N/A N/A 0 0 0 0

PHAR 192 N No. (%) 135 102 N5113 109 123 130A 18 (13.3) 63 (61.8 33 (29.2) 15 (13.8) 46 (37.3) 37 (28.5)B 80 (59.3) 31 (30.4) 70 (61.9) 77 (70.6) 63 (51.2) 75 (57.7)C 35 (25.9) 8 (7.8) 9 (8) 14 (12.8) 12 (9.8) 16 (12.3)D 0 0 1 (0.9) 3 (2.8) 2 (1.6) 2 (1.5)F 1 (0.7) 0 0 0 0 0

a N/A: Class not taught pre-team-based learning.

93


a constant state of “firsts” and may have had difficultykeeping their minds open to the educational value ofchange. The advance preparation was a shift in studentthinking. Students complained that they did not have asmuch time to study for other courses. Faculty membersfrom other courses remarked that they felt TBL coursespushed students to study less for their courses and thisconcern is one of the reasons DUCPHS faculty mem-bers have not adopted TBL across the curriculum. Stu-dent emotions ran high the first year in particular, andwritten student comments on course evaluations wereextensive and mainly negative. These negative percep-tions motivated course faculty members to spend moretime orienting students to the rationale and mechanicsof TBL during the course and to provide more extensiveguidance in the course syllabus, but also in other areas,such as by working with student organizations in thesemester prior to TBL.

The first time the TBL class was offered, an existingtraditional-style classroom did not easily support groupwork and could have contributed to some of the negativefeedback on TBL initially. Although major physical im-provements were subsequently made to the classroom toaccommodate TBL, further improvements are needed tooptimize team interaction. Past use of detailed Power-Point slides had to be avoided with the TBL format. Forfaculty experts with previous experience using extensiveslide sets in lecture, reduction to 1 to 3 slides per applica-tion exercise question was challenging. Course facultymembers have agreed to follow mutually establishedguidelines going forward, with the understanding thatflexibility will ensure students receive the necessarytraining for success upon graduation.

CONCLUSIONFaculty members of a team-taught pharmacothera-

peutics course successfully transitioned to a TBL peda-gogy that increased student accountability, reinforced thevalue of teamwork, enhanced professional development,and optimized faculty workload. Students’ evaluations ofthe course using the IDEACenter’s rating system showedimproved student progress with TBL, while aggregateteacher ratings remained stable. Team performance ongRATs exceeded individual performance on iRATs, andoverall course grades remained stable with TBL. Changesin faculty member approaches to instruction with TBLwere viewed positively by students and enhanced learn-ing in the course.

REFERENCES1. Ladden MD, Bodenheimer T, Fishman NW, et al. The emerging

primary care workforce: Preliminary observations from The Primary

Care Team: Learning From Effective Ambulatory Practices project.Acad Med. 2013;88(12):1830-1834.2. Nielsen M, Olayiwola N, Grundy P, Grumbach K. The Patient-Centered Medical Home’s Impact on Cost & Quality: An Annual

Update of the Evidence, 2012-2013. Patient Centered Primary CareCollaborative. 2014. http://www.pcpcc.org/resource/medical-homes-impact-cost-quality. Accessed February 20, 2014.3. Zarowitz BJ, Miller WA, Helling DK, et al. Optimal medicationtherapy prescribing and management: Meeting patients’ needs in

an evolving health care system. Pharmacotherapy. 2010;30(11):350e-359e.4. Institute of Medicine, Committee on Health Professions EducationSummit. Health Professions Education: A Bridge to Quality.Washington, DC: National Academies Press, 2003.5. Accreditation Council for Pharmacy Education. Accreditationstandards and guidelines for the professional program in pharmacyleading to the doctor of pharmacy degree. Guidelines Version 2.0.Chicago, Illinois. https://www.acpe-accredit.org/standards/default.asp. Accessed June 3, 2014.6. Allen RE, Copeland J, Franks AS, et al. Team-based learning inUS colleges and schools of pharmacy. Am J Pharm Educ. 2013;77(6):Article 115.7. Beatty SJ, Kelley KA, Metzger AH, et al. Team-based learning in

therapeutics workshop sessions. Am J Pharm Educ. 2009;73(6):Article 100.8. Conway SE, Johnson JL, Ripley TL. Integration of team-basedlearning strategies into a cardiovascular module. Am J Pharm Educ.2010;74(2):Article 35.9. Franks AS. Enhancing team-based active learning through hands-on experience with nicotine replacement therapy. Am J Pharm Educ.2013;77(6):Article 128.10. Grady SE. Teachers Topics: Team-based learning in

pharmacotherapeutics. Am J Pharm Educ. 2011;75(7):Article 136.11. Letassy NA, Fugate SE, Medina MS, et al. Using team-basedlearning in an endocrine module taught across two campuses. AmJ Pharm Educ. 2008;72(5):Article 103.12. Persky AM, Pollack GM. A modified team-based learningphysiology course. Am J Pharm Educ. 2011;75(10): Article 204.13. Persky AM. The impact of team-based learning on a foundationalpharmacokinetics course. Am J Pharm Educ. 2012;76(2):Article 31.14. Zingone MM, Franks AS, Guirguis AB, et al. Comparing team-based and mixed active-learning methods in an ambulatory careelective course. Am J Pharm Educ. 2010;74(9):Article 160.15. Kolluru S, Roesch DM, de la Fuente AA. A multi-instructorteam-based, active-learning exercise to integrate basic and clinical

science content. Am J Pharm Educ. 2012;76(2):Article 33.16. Bleske BE, Remington TL, Wells TD, et al. Team-based learningto improve learning outcomes in a therapeutics course sequence. AmJ Pharm Educ. 2014;78(1):Article 13.17. Michaelsen LK, Parmalee DX, McMahon LL, eds. Team-Based

Learning for Health Professions Education. A Guide to Using SmallGroups for Improving Learning. Sterling, VA: Stylus PublishingLLC; 2008.18. Ofstad W, Brunner LJ. Reviews: Team-based learning inpharmacy education. Am J Pharm Educ. 2013;77(4):Article 70.19. Farland MZ, Sicat BL, Franks AS, et al. Best practices forimplementing team-based learning in pharmacy education. AmJ Pharm Educ. 2013;77(8):Article 177.20. Searle NS, Haidet P, Kelly PA, et al. Team learning in medical

education: Initial experiences at ten institutions. Acad Med. 2003(Oct. Suppl.);78(10):S55-S58.

94


21. Thompson BM, Schneider VF, Haidet P, et al. Team-basedlearning at ten medical schools: Two years later. Med Educ. 2007;41(3):250-257.22. Koles P, Nelson S, Stolfi A et al. Active learning in a Year 2pathology curriculum. Med Educ. 2005;39(10):1045-55.23. Clark MC, Nguyen HT, Bray C, et al. Team-based learning in anundergraduate nursing course. J Nurs Educ. 2008;47(3):111-117.24. Parmalee DX, DeStephen D, Borges NJ. Medical students’attitudes about team-based learning in a pre-clinical curriculum.MedEduc Online. 2009;14(1):1-7.25. Koles PG, Stolfi A, Borges NJ, et al. The impact of team-basedlearning on medical students academic performance. Academic Med.2010;85(11)1739-1745.26. Bahramifarid N, Sutherland S, Jalali A. Investigating theapplications of team-based learning in medical education. Educ MedJ. 2012;4(2):e7-e12.

27. Fatmi M, Hartling L, Hillier T et al. The effectiveness of team-based learning on learning outcomes in health professionseducation: BEME Guide No. 30. Med Teach. 2013;35(12):e1608-e-1624.28. Cheng CY, Liou SR, Tsai HM, et al. The effects of team-basedlearning on learning behaviors in the maternal-child nursing course.Nurse Educ Today. 2014;34(1):25-30.29. Hartling L, Spooner C, Josvold LT, Oswald A. Problem-basedlearning in pre-clinical medical education: 22 years of outcomeresearch. Medical Teach. 2010;32(1):28-35.30. The IDEA Center. http://www.theideacenter.org. AccessedNovember 1, 2013.31. iPeer. http://ipeer.ctlt.ubc.ca/. Accessed November 5, 2013.32. Medina MS, Plaza CM, Stowe CD, et al. Center for theAdvancement of Pharmacy Education 2013 Educational Outcomes.Am J Pharm Educ. 2013;77(8):Article 162.

95


Instructions to Authors

Introduction

The Journal is devoted to providing a forum for communication of relevant information for pharmacy educators and all others interested in the advancement of pharmacy education. To be considered for publication, manuscripts must directly relate to pharmacy education and provide useful information for the national or international audience of the Journal. If a submission has only local or regional relevance, its usefulness to the majority of readers is limited and thus will not be accepted. To ensure that only accurate and substantive articles are included, all manuscripts undergo a peer review process and editorial approval prior to acceptance.

Manuscript Categories

Reviews. Reviews are comprehensive, well-referenced descriptive papers on teaching or research topics

directly related to entry-level and graduate or postgraduate education and training or skill development. The Reviews section includes papers on the history of pharmacy education. These manuscripts should not exceed 25 double-spaced pages for all components.

Research Articles. Research articles describe experimental or observational investigations that used

formal methods for data collection and reporting of results of studies directly related to pharmacy education. The category does not include investigations of instructional methods or approaches. These manuscripts

should not exceed 25 double-spaced pages for all components.

Instructional Design and Assessment. Instructional Design and Assessment papers describe

novel methods for professional and graduate student instruction (lectures, laboratories, practice experiences, or courses), or informational manuscripts on programmatic and curriculum development. These manuscripts should not exceed 25 double-spaced pages for all components. Instructional Design and Assessment articles should conform to the IDEAS format as described in an updated article by Poirier et al. [Poirier T, Crouch M, Hak E, MacKinnon G, Mehvar R, Monk-Tutor M. Updated guidelines for manuscripts describing instructional design and assessment: the IDEAS format. Am J Pharm Educ. 2009;73(3):Article 55.] (http://www.ajpe.org/view.asp?art=aj730355&pdf=yes)

Teachers' Topics. Teachers' Topics are invited manuscripts from those named Teachers of the Year at

AACP member institutions. Teachers’ Topics papers follow the IDEAS format as described above. These manuscripts should not exceed 25 double-spaced pages for all components.

Innovations in Teaching. Innovations in Teaching are invited manuscripts from recipients of the

AACP Innovations in Teaching Award. These manuscripts should not exceed 25 double-spaced pages for all components.

Letters to the Editor. Letters to the Editor serve as a forum for the expression of ideas or for

commenting on matters of interest. It is also an avenue for critiquing or expanding on the information presented in a previously published manuscript. Authors are required to identify themselves. The Editor reserves the right to reject, shorten, excerpt, or edit letters for publication. Book and Software Reviews. Book and Software Reviews are brief documents that provide the reader with a clear understanding of content in a book or software program, as well as the product structure, scope, and limitations. The reviewer should state the value or utility of the product for instruction, research, or other academic activities.

AACP Reports and Minutes. AACP Reports and AACP Minutes are summaries of activity of

officers, delegates, and committees within AACP.

Research Standards

For all manuscripts reporting on research involving human subjects (including educational and institutional research), the author should indicate in the cover letter that the research has been reviewed and approved by the appropriate human research or ethics review committee, or that it has been determined to be exempt from such review. For research that has undergone such review and approval, a statement to that effect should be included in the methods section. All survey research must meet criteria established by the Journal’s Editorial Board. Please refer to the following publications for guidelines:

• Draugalis JR, Plaza CM. Best practices for survey research reports revisited: implications of target population, probability sampling, and response rate. Am J Pharm Educ. 2009;73(8):Article 142. (http://www.ajpe.org/view.asp?art=aj7308142&pdf=yes)

• Draugalis JR, Coons SJ, Plaza CM. Best practices for survey research reports: a synopsis for authors and reviewers. Am J Pharm Educ. 2008;72(1):Article 11. (http://www.ajpe.org/view.asp?art=aj720111&pdf=yes)

• Fincham JE.Response rates and responsiveness for surveys, standards, and the Journal. Am J Pharm Educ. 2008;72(3):Article 43.(http://www.ajpe.org/view.asp?art=aj720243&pdf=yes)

Stylistic Considerations

The style specifications for the Journal must be followed. Below are general guidelines for manuscript format and style. If in doubt about style, authors should refer to the American Medical Association (AMA) Manual of Style, 10th ed, or consult a recent issue of the Journal. Text. The text should be scholarly, readable, clear, and concise. Standard nomenclature should be used. Unfamiliar terms and acronyms should be defined at first mention. Manuscripts that were prepared for oral presentation must be rewritten for print. Authors of research papers are discouraged from writing excessively long introduction or discussion sections. Word Style. Consult a current edition of Webster's dictionary for guidance on spelling, ompounding, and word separation. Foreign words, not in general use, should be italicized. For proper use of chemical and biochemical terms, mathematical equations, mathematical expressions, special symbols, subscripts, superscripts, or Greek letters, please refer to the AMA Manual of Style.

Capitalization. The word "association" must be capitalized when referring to the American Association

of Colleges of Pharmacy. When the word "journal" is capitalized and italicized as Journal, it can refer only to the American Journal of Pharmaceutical Education. In scientific writing, always capitalize the following: major words in titles and headings of manuscripts, designators for tables, figures, and appendices (eg, Appendix 1), eponyms (but not the noun that follows them, eg, Gram stain, Babinski sign), names of tests (eg, Beck Depression Inventory), genus names of organisms (but not the name of species, varieties or subspecies), acts of legislation (eg, Medicare), awards (eg, Nobel Prize), proprietary names (eg, Xerox copier), the title of a person when followed by the person's name (eg, Chair John W. Jones), official names of organizations and institutions (eg, Centers for Disease Control and Prevention), geographic places (eg, United States of America), sociocultural designations (eg, Republicans, French people), and historical events (eg, Vietnam War).

Abbreviations. In instances where repeated use of an organization or chemical name would become

awkward, an official or accepted abbreviation may be substituted. The abbreviation should be placed in parentheses immediately following the first use of the name in the main body of the text. Abbreviations of common pharmaceutical associations or organizations do not require periods or spaces between letters (eg, AMA). Abbreviations of "eg," and "ie," and "et al" should not be separated by periods. The names of US states and countries should be spelled out when they stand alone (eg, “…pharmacists throughout the United States...”). Do not use postal abbreviations for states in the text. The abbreviation “US” may be used as a modifier only when it directly precedes the word it modifies (eg, US health policies). Otherwise, it should be spelled out (eg, “…the population of the United States”). The names of all other cities, states, provinces, and countries should be spelled out when they occur within the text of the article. Refer to the AMA Manual of Style for additional rules regarding abbreviations.

Numbers. Numbers must be written as Arabic numerals unless they occur at the beginning of a sentence,

in which case the number should be spelled out. The exception to this rule is when the number “one” is used

in isolation within the text and substituting an Arabic number would seem awkward (eg, “there was only one logical solution to the problem”). A number containing a decimal must be styled as an Arabic number. All fractions must be written as decimal equivalents.

Measurements. The metric system will be used for all measurements; however, conventional units

should be used instead of SI units. Do not use periods when abbreviating units of measure.

Reference numbers. Reference numbers cited in the text of an article should be superscript Arabic

numerals placed at the end of the sentence, outside the final period or other punctuation. Subsequent citations to the same reference must be indicated by the same number originally asiged to that reference. Do not place parentheses around reference numbers cited in text

.

Personal Communications. If the reference source for information in an article is from a personal

communication, it should be referenced as such in parentheses immediately following the material to which it pertains. A personal communication reference must include the persons first initial and last name, type of communication, and date (eg, T.L. Johnson, e-mail, October 2, 2002). AJPE requires that you obtain and provide to the Journal written permission from the person to use them as a reference.

Hypertext Links. Authors may identify uniform resource locators (URLs) for Internet Web sites that

provide the reader with additional information on the topic addressed in the manuscript. Although URLs are an important feature of electronic publishing, authors are encouraged to be selective in their choice of sites to include. Do not include URLs for Web pages with newspaper or journal articles that will be removed or archived to another Web page. Links to pharmaceutical manufacturers or other sources of product information are acceptable; however, providing a URL to the reader should not be substituted for adequate discussion within the manuscript itself. Only include links to sites that are accessible without a password.

Manuscript Organization

When submitting a manuscript in Editorial Manager, the manuscript document (usually a Word file) should be arranged in the following order: title page, abstract, text, references, tables, figures, and appendices. Editorial Manager allows authors to upload files with tables, figures, and appendices separately if that is more convenient.

Title Page. The title page should have the following information: a concise title, name of each author,

terminal degree, academic/professional title and affiliation, and city and state where located. If an author has relocated to another institution, please include his/her affiliation during the time the author worked on the manuscript. At the lower left of the page, indicate the name of the corresponding author and provide his/her mailing address, telephone number, facsimile number, and e-mail address. At the bottom of the title page, indicate 3 to 5 keywords.

Abstract. Each manuscript must include an abstract of 100 to 150 words. For Research Articles,

the abstract should include a brief statement (1 to 3 sentences) for each of the following sections: Objective, Methods, Results, and Conclusions. For Instructional Design and Assessment papers, the abstract should include a statement for the following sections: Objective, Design, Assessment, and Conclusions. Each section within the abstract should be flushed left, followed by a period and the statements summarizing that section.

Main Body of Text. The use of subheadings to divide the text is encouraged. Primary headings should

be in bold upper case letters and should contain no more than 35 characters or spaces. Secondary headings should be in bold title case and appear above the paragraph. Tertiary headings should be in bold with only the initial letter capitalized. Tertiary headings should end with a period and should appear before the beginning of the first sentence in that section. Designand Methods sections should be written in the past tense voice. Authors should address in the Discussion or Conclusion how the work could be translated to other institutions.

Acknowledgments. Any special funding received for research that is the subject of the manuscript

should be included under a section entitled "Acknowledgments" at the end of the text. If the authors wish to thank colleagues or others who provided assistance with their research or manuscript preparation, those acknowledgments also should be included under this section. Any statements concerning liability for the content of the manuscript may be included here as well (eg, "the ideas expressed in this manuscript are those of the author and in no way are intended to represent the position of…"). Conflicts in funding issues should be included.

References. References to a published source should be provided for all information in the manuscript

that contains dates, facts, or opinions other than those of the author. Authors are responsible for the accuracy and completeness of all citations. References should be numbered consecutively in the order in which the information contained in the referenced publication appears or is referred to in the manuscript. A second abbreviated reference or use of the term "ibid" to refer to information cited in a previous reference should not be cited. Instead, if information from the same source is referred to a second time in the manuscript, the same reference number originally assigned to that source should be cited. For detailed information on reference style, refer to the section on Stylistic Concerns. Each journal citation must include the surnames and complete initials of all authors. For manuscripts with 7 or more authors, the first 3 authors should be listed, followed by a comma and “et al.” The names of all periodicals cited must be abbreviated in accordance with abbreviations adopted by the National Library of Medicine and used in Index Medicus. An example and special instructions for specific types of references are provided below. For additional guidance, please refer to the AMA Manual of Style, 10th ed, or to a recent issue of the Journal.

Journal articles. For references to journal articles, list the names of the authors beginning with the last

name of each author followed by his/her initials. After the authors' names, list the title of the article. The first letter of the first word of the title is upper case type, but the remainder of the title should be in lower case letters, except for the first letter of proper names. A period should be placed after the title. Next, provide the properly abbreviated title of the journal in italics followed by a period. Please refer to the journal list on PubMed for proper reference abbreviations (http://www.ncbi.nlm.nih.gov/journals). Finally, include the following numerical information: year of publication followed by a semicolon, volume number, issue number in parentheses, and a colon followed by page number(s). If the article does not appear on consecutive pages, use a semicolon between each segment of pages (eg, 172-175;179-183;199.) Example: Stratton TP, Cochran GA. A rural geriatric experience. Am J Pharm Educ. 1990;62:151-155.

Book. List the last name of the first author of the book, followed by the first and middle initial if given, just

as in a journal reference. The names of all authors must be listed. Place a period after the last author's initials. Next, state the title of the book using standard rules for capitalization within titles. A period should be placed at the end of the title. If more than one edition of the bookhas been published, then the edition number must be given. The numerical form of the ordinal number should be used to indicate the edition number (eg, 9th), followed by a space and "ed." Next, provide the city and state where the publisher is located. Use postal abbreviations for states. A colon should separate the city and state from the name of the publisher. The full name of the publisher should be given, followed by a semicolon. Next, provide the year of publication followed by a period. If the reference is to a specific page(s) within the book, place a colon after the year of publication, followed by the page or page numbers referenced. Example: Martin AN. Physical Pharmacy. 4th ed. Philadelphia, PA: Lea & Febiger; 1993:268;270-273.

Chapter in a book. To reference a single chapter in a book, first list the authors and state the title as

you would if citing a journal article. The chapter title should be followed by the word "In" followed by a colon. Next, list the name(s) and initials of the editors of the book, followed by a comma and the abbreviation “ed” or “eds” followed by a period. Next include the title of the book, location of the publisher, name of publisher, year of publication, and page numbers (same format as for a reference to an entire book (see previous example). Example: Lyon RA, Titeler M. Pharmacology and biochemistry of the 5-HT2 receptor. In: Sanders-Bush E, ed. The Serotonin Receptors. Clifton, NJ: Humana Press;1989:59-88.

Thesis or dissertation. For references to theses or dissertations, place the title of the thesis or

dissertation in italics. Include the location of the institution, its name, and the year the thesis or dissertation was completed. Example: Thorn MD. A Comparative Review of the Statistical and Research Quality of the Medical and Pharmacy Literature [masters thesis]. Chapel Hill: University of North Carolina, 1982.

Online material. For references to journals, e-magazines, or other publications on the Internet, state

the names of the authors, title of the article, publication title, and volume and publication date in the same format as you would for a journal reference. For references to other information, give the title of the Web page, followed by the name of the organization or Web site that published the information. For all references to online material, the author should include the uniform resource locator (URL) for the page of the Web site referenced (eg, www.hcfa.gov/stats.htm), followed by a period. Finally, write “Accessed” followed by the month, day, and year on which the information was obtained from the site, followed by a period. Example: Healthy People 2010, Office of Disease Prevention and Health Promotion, U.S. Department of Health and Human Services. http://health.gov/healthypeople. Accessed May 25,2010.

Unpublished works. For references to unpublished material, such as articles or abstracts presented at

professional meetings but not published, provide the name of the meeting where the article was presented. If the abstract has been published, the published source should be cited.

Article in press. For references to information in books or articles that are currently in press,

provide all of the available information for the reference. In place of the year, volume, issue, and page numbers, include “In press.” Example: Adamcik B, Hurley S, Erramouspe J. Assessment of pharmacy students' critical thinking and problem-solving abilities. Am J Pharm Educ. In press. Manuscripts that have been submitted to a publisher or journal but have not been accepted for publication cannot be included in the reference section. To attribute information in the text to an unpublished source, list the authors and date the manuscript was completed, along with the words “unpublished data.” Example: “Similar results were achieved in a study of attrition rates in 2 Southeastern colleges of pharmacy conducted by P.T. Jones (unpublished data, 2009).”

Tables and Figures Tables.should not duplicate information provided in the text. Instead, tables should be used to provide additional information that illustrates or expands on a specific point the author wishes to make. Each table should be self-explanatory and begin on a separate page in the document. Tables should be numbered using Arabic numbers according to the order in which they are referred to in the text. Tables should be created using Microsoft Word table formatting tools (do not use the tab key to form rows and columns of data as tab information is lost when the document is processed by the publisher). The table number and table title should be placed in the first row (merged to form a single cell) of the table. Data must be placed in separate cells within the table to prevent text and numbers from shifting. Footnotes should be placed at the bottom of the table inside a single row and ordered using superscript lower-case letters (beginning with “a”) rather than footnote symbols. A superscript letter that refers the reader to the corresponding footnote should be inserted in the table title or body of the table. Refer to a current issue of the Journal for examples of table style.

Figures. Figures should be numbered using Arabic numbers, based on the order in which they are

presented in the text. Figure legends should be concise and self-explanatory. All illustrative materials for the figures should be submitted as high-resolution gif or jpg files. The key to any symbols in a graph or chart should be included as part of the illustration itself, rather than in the legend. Do not place a box around graphs or other types of figures. If figures contain illustrations that have been published elsewhere, a letter of permission to reprint from the original publisher must accompany the manuscript. A graphic image embedded in a MS Word file has a resolution of 188 dpi or lower. As a result, when printed, graphics and text within the graphic may look fuzzy. If possible, send graphics with a resolution of at least 300 dpi. Large and/or high-resolution graphic images saved as TIFF or EPS images should be uploaded to Editorial Manager as separate files from the manuscript text (Word file). Use Arial font for any lettering within the graphic images. Figures, symbols, lettering andnumbering should be clear and large enough to be legible when reduced. The minimum font size that should be used within the figure is 10 pt and the maximum is 14 pt. As a general rule, the final graphic should be no more than 6.5 inches in width, large enough to span a single Journal page.

Manuscript Submission

Please submit your manuscript using AJPE’s Editorial Manager online tracking system at http://ajpe.edmgr.com. Log in using your username and password and then follow the step-by-step on-

screen instructions for uploading your files. If you do not know your username and password or need to have an account created for you, please send an e-mail to [email protected] and someone will respond as quickly as possible.

NOTE: When you attempt to log in, you may get an alert message stating Editorial Manager requires

browser cookies (a tiny file placed on your computer so the site will remember who you are). Option 1: If you're using Internet Explorer, go to Tools > Internet Options > Privacy. Move the setting to "low" (ie, accept first-party cookies asking for personal information but no third-party cookies). Option 2: If you do not wish to change your browser settings, simply click OK in response to the message, then log in twice (the first time it will not work because there's no cookie to access, but the second time it will accept the information and open Editorial Manager). If for any reason it is not possible for you to submit your paper using AJPE’s Editorial Manager site as outlined above, send your manuscript as an e-mail attachment to [email protected] and someone will assist you.

Copyright Form. Manuscripts submitted to the Journal should be unpublished and not under

consideration elsewhere. Under the terms of the Copyright Revision Act of 1976 (Public Law 94-533) it is necessary to have the rights of the authors transferred to the publisher in order to provide for the widest possible dissemination of professional and scientific literature. The editorial office must receive a transfer of copyright form before a manuscript can be published online. The author may mail the form at the time a paper is submitted or wait until it is accepted for publication.

Formatting, Copyediting, and Proofing of Accepted Manuscripts

Copyediting Stage. Prior to publication, all manuscripts are copyedited for organization, style, and

clarity. Authors may be asked at this stage to reorganize a manuscript or shorten the text.

Proofing Stage. Authors will receive an e-mail with a link to online electronic galley proofs (eProof) of

their paper for review approximately 10 days prior to publication. Authors may send an e-mail with an annotated PDF (corrections entered using Adobe Acrobat software) attached, or they may print out the eProof, mark corrections on the copy, and fax only those pages with corrections to the journal office (fax: 803-777-3097). The Journal allows authors 2 business days to return eProofs.