1 2 3 4 5 6 7 8pearsonrsquos test measurement amp research services Quarterly Newsletter v3 n4 2010

Editorrsquos Noteby Jason Wachsmuth

Welcome to the eleventh issue of Pearsonrsquos Test Measurement amp Research Services (TMRS) Newsletter which is aimed at publicizing the ongoing research efforts of our group to the measurement community both within and outside Pearson

Happy New Year I hope 2010 turned out to be very productive for you There sure was a lot to announce in TMRS over the past twelve months which included 27 publications and 98 instances of conference participation These numbers are up compared to 2009 where there were 21 publications and 81 instances of conference participation

In this issue I have included a special feature where notes from readers are shared These notes highlight how TMRS research efforts or the TMRS Newsletter have made a positive impact on others Look for these notes on the side margins of the newsletter If anyone within or outside Pearson would like to share something I would be happy to hear from you

As we move forward into 2011 I would like to announce that I will continue my role as the Editor of the newsletter I am happy about extending my appointment because I really enjoy working on this project I will elaborate on my experience by posting a TrueScores blog in the near future

Test

Measurement amp Research

ServicesQuarterly Newsletter

vol 3 | no 4 | 2010

inside

Editorrsquos Note

Announcements

Publications

Conference Participation

TrueScores

Jason Wachsmuth Editor Assessment amp Information Pearson jasonwachsmuthpearsoncom 3193396400 Ext 216005

Last but not least I would like to take a moment to remember Ian Little who passed away in May and Karla Bartholomew who passed away in December I had the pleasure of working with them both and knowing them on a personal level Ian and I first met at a Maryland standard setting Later when he moved to Iowa City we hung out a few times including going out to see a great concert by Blitzen Trapper Karla and I worked together on the Georgia projects and whenever I traveled for a standard setting I would try to bring a rock back for her collection They were both great people to know and work with and they are dearly missed

This newsletter is intended for widespread dissemination If you or someone you know would like to be added to the distribution list or if you require a printed version of the newsletter please contact me directly I also welcome questions comments and suggestions as support for a continual effort to improve the newsletter Back issues can be downloaded from the Research Publications section of the Assessment amp Information website

1

Jason Wachsmuth

1 2 3 4 5 6 7 8pearsonrsquos test measurement amp research services Quarterly Newsletter v3 n4 2010

newsletter advisory board

Walter (Denny) Way PhD Senior Vice President of Psycho-metric amp Research Services

Mary Kino PhD Principal Research Scientist

Bob Dolan PhD Senior Research Scientist

Erika Hall PhD Senior Research Scientist

David Shin PhD Senior Research Scientist

Edward W Wolfe PhD Senior Research Scientist

Yuehmei (May) Chien PhD Research Scientist

Jason L Meyers PhD Research Scientist

Emily R Lai PhD Associate Research Scientist

Tian Song PhD Associate Research Scientist

Hua Wei PhD Associate Research Scientist

Jason Wachsmuth Research Associate Editor

Mark Robeck PhD Senior Project Manager

Pearsonrsquos Test Measurement amp Research Services Newsletter is published quarterly The newsletter is not copyrighted readers are invited to copy any articles that have not been previously copyrighted Credit should be given in accordance with accepted publishing standards

AnnouncementsPearson Center at the University of Texas Hosts Dr Gregory Cizek

The Pearson Center for Applied Psychometric Research at the University of Texas at Austin hosted Dr Gregory Cizek of the University of North Carolinarsquos School of Education as the inaugural speaker in the Distinguished Lecture Series on November 3 2010 Dr Cizek presented a lecture titled ldquoError of Measurement Reconsidering Validity Theory and the Place of Consequencesrdquo In his lecture Cizek identified several issues with the current conceptualization

of consequential validity and proposed an alternative view of how consequences of test use should be interpreted

The lecture provided a great opportunity for Pearson staff to interface with faculty and students at the University of Texas at Austin as well as to officially launch the work of the newly established Pearson Center at the university Pearson donated $1000000 to endow the Pearson Center as well as a faculty fellowship and a professorship in the psychometric fellowship program

Dr Richard Ferguson Speaks at Coffman Lecture SeriesDr Richard Ferguson presented a lecture titled ldquoRace to the Top A Never-Ending Climbrdquo at the Lindquist Center on the University of Iowa campus on November 29 2010 Dr Ferguson reviewed historical educational reforms starting from the 1960s He said that many reforms have happened in education but students in the United States are still not better prepared for college than students were 20 years ago Ferguson gave an overview of where students are today

in terms of career and college readiness and stated that there is still a long way to go He emphasized that an effective reform should impact the classrooms to be successful Dr Fergusonrsquos background includes having been an ACT CEO and a classroom teacher

The Coffman Lecture Series which began in 1995 has occurred annually except in 2003 and 2004 Past lectures include presenters such as Dr Eva Baker Dr Linda Crocker Dr Daniel Koretz and Dr Susan Phillips The series was established in honor of William E Coffman a faculty member from 1969 to 1981 for the Psychological and Quantitative Foundations Department at the University of Iowa and a former Director of the Iowa Testing programs

Staffing UpdatesIn October 2010 Tim OrsquoNeil was promoted to Manager of Psychometric amp Research Services in the San Antonio Texas office where he manages the Research Associate staff In addition to his managerial duties Tim is also the lead Research Scientist for the Maryland project Tim received his doctorate in Research and Evaluation Methods from the University of Massachusetts Amherst and joined Pearson (by way of Harcourt) in 2003

In September 2010 Natasha Williams was promoted to Manager of Psychometric amp Research Services in the Austin Texas office Natasha has worked on the Texas project since joining Pearson in 2004 She has recently been supporting the Special Education and English Language Learner assessments and she will now expand her efforts to include other areas of the Texas assessment program Before joining Pearson Natasha was a Research

Associate in the Statistical Research Department at ACT She received her Masterrsquos degree in Experimental Psychology from the University of Texas at San Antonio and her PhD in Educational PsychologymdashQuantitative Methods from the University of Texas at Austin

2

Plaque commemorating the Pearson Faculty Fellowship at the University of Texas

Tim OrsquoNeil EdD

Natasha Williams PhD

1 2 3 4 5 6 7 8pearsonrsquos test measurement amp research services Quarterly Newsletter v3 n4 2010

Staffing UpdatesContinued from page 2

In August 2010 Stephen Murphy was promoted to Manager of Psychometric amp Research Services He graduated with a Masterrsquos degree from Middle Tennessee State University in 2003 and a PhD from the University of Oklahoma in 2007 Both degrees are in Industrial and Organizational Psychology Before joining Pearson Stephen worked for the State of Tennessee for the Personnel Board of Jefferson County and for

Hogan Assessment Systems He is managing the Tulsa Oklahoma office staff one staff member in Austin Texas and one in Iowa City Iowa and reports to Rob Kirkpatrick Stephenrsquos research interests include college readiness and equating design He has worked in the Tulsa office since its establishment in 2007 Recently the office grew from five offices on the 49th floor to nine offices and a large conference room on the 47th floor

Allen Laursquos AccoladesIn September 2010 Allen Lau Senior Research Scientist from Psychometric amp Research Services at the San Antonio Texas office took first place in the Humorous Speech Contest for Toastmasters Area I-73 and later in District 55 Division I He took third place in the district competition in October 2010 Allenrsquos speech titled ldquoHoney I fixed the Toiletrdquo included

props singing English translations of Chinese and wisdom about relationships Additionally Lau is now the Vice President of Education for the San Antonio Sociable Toastmasters Club Toastmasters International is a nonprofit organization that aims to help people improve their public speaking skills

Allen also received an award in October 2010 from Herbert Harris a Territorial Vice President from Pearson A ldquoThirteen Years of Outstanding Service on the Delaware Student Testing Programrdquo certificate was presented to Allen for his work as the lead Research Scientist since 1997 Allenrsquos involvement included activities such as equating item analysis technical consulting and working on technical reports and research reports The contract was completed on December 31 2010

Reward amp Recognition AwardsCongratulations to the following individuals who have been recognized by their coworkers for outstanding contributions to the success of TMRS from August through October The All-Star award recognizes a staff member who exhibits strong leadership that has a significant and positive effect on the project andor its TMRS members The Team Player award is intended for a staff member who goes above and beyond what is required of his or her position to ensure the success of a project The Going the Extra Mile award is given out as a quick ldquothanksrdquo when a staff member has made a positive impact on a project andor whose effort contributed in a meaningful way Awards were presented during the Learning at Lunch session on December 3 2010

All-StarSharon Foss Virginia Modified Achievement Standards Test (VMAST) committee meeting facilitation (August 2010)

Kimberly Childers Texas End of Course and Texas English Language Proficiency Assessment System (TELPAS) (September 2010)

Tiffany Hartmann Arizona Instrument to Measure Standards (AIMS) (October 2010)

Shelley Ragland and Hyun-Jung Sung Relocation of the PRS office in Tulsa Oklahoma (October 2010)

3

Stephen Murphy PhD

Allen Lau PhD

1 2 3 4 5 6 7 8pearsonrsquos test measurement amp research services Quarterly Newsletter v3 n4 2010

Staffing UpdatesContinued from page 2

In August 2010 Stephen Murphy was promoted to Manager of Psychometric amp Research Services He graduated with a Masterrsquos degree from Middle Tennessee State University in 2003 and a PhD from the University of Oklahoma in 2007 Both degrees are in Industrial and Organizational Psychology Before joining Pearson Stephen worked for the State of Tennessee for the Personnel Board of Jefferson County and for

Hogan Assessment Systems He is managing the Tulsa Oklahoma office staff one staff member in Austin Texas and one in Iowa City Iowa and reports to Rob Kirkpatrick Stephenrsquos research interests include college readiness and equating design He has worked in the Tulsa office since its establishment in 2007 Recently the office grew from five offices on the 49th floor to nine offices and a large conference room on the 47th floor

Allen Laursquos AccoladesIn September 2010 Allen Lau Senior Research Scientist from Psychometric amp Research Services at the San Antonio Texas office took first place in the Humorous Speech Contest for Toastmasters Area I-73 and later in District 55 Division I He took third place in the district competition in October 2010 Allenrsquos speech titled ldquoHoney I fixed the Toiletrdquo included

props singing English translations of Chinese and wisdom about relationships Additionally Lau is now the Vice President of Education for the San Antonio Sociable Toastmasters Club Toastmasters International is a nonprofit organization that aims to help people improve their public speaking skills

Allen also received an award in October 2010 from Herbert Harris a Territorial Vice President from Pearson A ldquoThirteen Years of Outstanding Service on the Delaware Student Testing Programrdquo certificate was presented to Allen for his work as the lead Research Scientist since 1997 Allenrsquos involvement included activities such as equating item analysis technical consulting and working on technical reports and research reports The contract was completed on December 31 2010

Reward amp Recognition AwardsCongratulations to the following individuals who have been recognized by their coworkers for outstanding contributions to the success of TMRS from August through October The All-Star award recognizes a staff member who exhibits strong leadership that has a significant and positive effect on the project andor its TMRS members The Team Player award is intended for a staff member who goes above and beyond what is required of his or her position to ensure the success of a project The Going the Extra Mile award is given out as a quick ldquothanksrdquo when a staff member has made a positive impact on a project andor whose effort contributed in a meaningful way Awards were presented during the Learning at Lunch session on December 3 2010

All-StarSharon Foss Virginia Modified Achievement Standards Test (VMAST) committee meeting facilitation (August 2010)

Kimberly Childers Texas End of Course and Texas English Language Proficiency Assessment System (TELPAS) (September 2010)

Tiffany Hartmann Arizona Instrument to Measure Standards (AIMS) (October 2010)

Shelley Ragland and Hyun-Jung Sung Relocation of the PRS office in Tulsa Oklahoma (October 2010)

3

Stephen Murphy PhD

Allen Lau PhD

1 2 3 4 5 6 7 8pearsonrsquos test measurement amp research services Quarterly Newsletter v3 n4 2010 4

Reward amp Recognition AwardsContinued from page 3

Team Player

August 2010

Alex Berumen Joleen Jernigan Anita McCahill Gina Musselman and Sarah Siedl Abu Dhabi Education Councilrsquos (ADEC) External Measurement of Student Achievement test

Matt Malcom and Alice Rutledge Arizona AIMS reading

Dawn Ballard Laura Ter Keurst Paula Jameson and Diane Thompson Florida Comprehensive Assessment Testing (FCAT) interpretive products

Ted Slauson Peggy Seifert Mandy Speights Lisa Marohnic and Toni McCarthy Florida FCAT math test construction

Yu Sun Illinois ISAT IAA and MOD

Dee Heiligmann Innovative Items Initiative

Dennis Fulkerson Minnesota MCA II Science Online Assessment

Michael Viering Emily Lai Bethany Cobb Matt Malay Dennis Launarey Patrick Duran Mary Kino Kathleen Keyes Charlie Sims Chris Park and Traci Meineke Montgomery County Public Schools (MCPS) North Star

Thanh Nguyen Casey Brown and Serena (Jie) Lin New York City ELAMath equating effort

Sonya Powers North Carolina reporting effort

Nick Gefaller Oklahoma Arizona and Puerto Rico

John Morris Puerto Rico PPAA

Robert Greenburg South Dakota STEP

Sandy Pedrotti Project Delta Initiative

Michael Stein Virginia VMAST committee meetings

September 2010

Janice Castro Arizona AIMS

Alicia Chrest and Kristene Mickler Minnesota MCT-II

Peggy Boecker Adele Herzstein Robert Hons Mary Ann Koon Chandra Nealon and Diana Tavera New York City

Sandra Pedrotti State of Texas Assessments of Academic Readiness (STAAR)

October 2010

Ana Montemayor Gerry Gonzalez Jon Irving Cathy Cotnam Tom Jones Leigh Ann Lipscomb Marsha Harrison Dawn Ballard Natalie Davis Brian Fields Ann Kaplan and Emily Zucker Florida FCAT

Derek May Illinois ISAT

Going the Extra Mile

James Ingrisone Virginia VMAST (October 2010)

notes from readers

Matt Brunscheen PMP Senior Test Development Manager Content Support Services

I have found the articles great and very informative I actually have shared articles concerning calculator usage with both the South Carolina DOE and the Puerto Rico DOEmdashboth of whom were very grateful for the information and research that went into them

1 2 3 4 5 6 7 8pearsonrsquos test measurement amp research services Quarterly Newsletter v3 n4 2010 5

2011 Pearson Research FellowshipPearson will offer eight-week fellowships in the summer of 2011 for four doctoral students with outstanding research skills who want experience in educational assessment The fellowships will include hands-on experience in the development and analysis of data for Kndash12 assessments Specific activities may include item analysiskey check procedures test construction scaling and equating technical writing comparability studies attending and facilitating item review committee meetings and other research projects Fellows will also have an opportunity to learn about item content development scoring and processing and other aspects of educational testing

DatesThe 8-week fellowships will take place from June 6 2011 through July 29 2011

LocationsOne fellowship is available in each of the following cities Austin Texas San Antonio Texas Iowa City Iowa and Tulsa Oklahoma

Fellowship GoalsFellows will have the opportunity to work closely with a team of Pearson research scientists to gain experience in many tasks involved in a Kndash12 assessment contract collaborate on a research paper abstract or presentation for national dissemination and better understand ways to collaborate in and communicate about educational measurement

AwardEach award includes a $5000 stipend Pearson will pay transportation costs to and from Austin San Antonio Iowa City or Tulsa Pearson will provide corporate housing for each fellow who is commuting more than 50 miles

QualificationsA fellow should be currently enrolled in a doctoral program in educational measurement educational statistics educational psychology or similar program Two to three years of doctoral-level coursework are required Fellows should have strong written and verbal communication skills Fellows may not participate in other summer internship or externship programs during the Pearson fellowship program

Application ProcedureEach candidate should submit a curriculum vita a copy of a graduate school transcript two letters of recommendation and a statement of purpose describing his or her interest in the fellowship All application materials must be received by February 25 2011 Fellowship awards will be announced by March 18 2011 Materials should be submitted to

Walter (Denny) Way PhD Senior Vice President ATTN Christine Carlson Psychometric amp Research Services Assessment amp Information Pearson 2510 N Dodge Street Mailstop HR180 Iowa City IA 52245-9555

notes from readers

Carl S Hyman PMP Director Pan-Pearson Organizational Quality Initiatives

I am charged with working with Pearson colleagues all over the globe on their assessment needs I have shared this outstanding newsletter with colleagues in South Africa where Pearson is making efforts to expand assessments through a global growth initiative I am also a member of the Pearson Assessment Community or PAC a group that you will be hearing more about in the next quarter We are an international group that works collaboratively with TMRS to spread our assessment services worldwide Denny Way and Jon Twing are intimately involved in this effort which is a whole separate story for your newsletter

The TMRS Newsletter opens a whole new world to our colleagues in developing areas who may be adding assessment services to their service offerings for the first time It helps people understand how evolved we are and how much we can help them provide solutions for their customers

1 2 3 4 5 6 7 8pearsonrsquos test measurement amp research services Quarterly Newsletter v3 n4 2010 5

2011 Pearson Research FellowshipPearson will offer eight-week fellowships in the summer of 2011 for four doctoral students with outstanding research skills who want experience in educational assessment The fellowships will include hands-on experience in the development and analysis of data for Kndash12 assessments Specific activities may include item analysiskey check procedures test construction scaling and equating technical writing comparability studies attending and facilitating item review committee meetings and other research projects Fellows will also have an opportunity to learn about item content development scoring and processing and other aspects of educational testing

DatesThe 8-week fellowships will take place from June 6 2011 through July 29 2011

LocationsOne fellowship is available in each of the following cities Austin Texas San Antonio Texas Iowa City Iowa and Tulsa Oklahoma

Fellowship GoalsFellows will have the opportunity to work closely with a team of Pearson research scientists to gain experience in many tasks involved in a Kndash12 assessment contract collaborate on a research paper abstract or presentation for national dissemination and better understand ways to collaborate in and communicate about educational measurement

AwardEach award includes a $5000 stipend Pearson will pay transportation costs to and from Austin San Antonio Iowa City or Tulsa Pearson will provide corporate housing for each fellow who is commuting more than 50 miles

QualificationsA fellow should be currently enrolled in a doctoral program in educational measurement educational statistics educational psychology or similar program Two to three years of doctoral-level coursework are required Fellows should have strong written and verbal communication skills Fellows may not participate in other summer internship or externship programs during the Pearson fellowship program

Application ProcedureEach candidate should submit a curriculum vita a copy of a graduate school transcript two letters of recommendation and a statement of purpose describing his or her interest in the fellowship All application materials must be received by February 25 2011 Fellowship awards will be announced by March 18 2011 Materials should be submitted to

Walter (Denny) Way PhD Senior Vice President ATTN Christine Carlson Psychometric amp Research Services Assessment amp Information Pearson 2510 N Dodge Street Mailstop HR180 Iowa City IA 52245-9555

notes from readers

Carl S Hyman PMP Director Pan-Pearson Organizational Quality Initiatives

I am charged with working with Pearson colleagues all over the globe on their assessment needs I have shared this outstanding newsletter with colleagues in South Africa where Pearson is making efforts to expand assessments through a global growth initiative I am also a member of the Pearson Assessment Community or PAC a group that you will be hearing more about in the next quarter We are an international group that works collaboratively with TMRS to spread our assessment services worldwide Denny Way and Jon Twing are intimately involved in this effort which is a whole separate story for your newsletter

The TMRS Newsletter opens a whole new world to our colleagues in developing areas who may be adding assessment services to their service offerings for the first time It helps people understand how evolved we are and how much we can help them provide solutions for their customers

1 2 3 4 5 6 7 8pearsonrsquos test measurement amp research services Quarterly Newsletter v3 n4 2010 6

PublicationsArce-Ferrer A (2010) An investigation of an assessment-centered learning environment with formative use (Test Measurement amp Research Services Bulletin No 18) Iowa City IA Pearson

Kolen M J Tong Y amp Brennan R L (2010) Scoring and scaling educational tests In A A von Davier (Ed) Statistical models for test equating scaling and linking (pp 43-58) New York Springer-Verlag

Taherbhai H Seo D amp Bowman T (in press) Comparison of paper-pencil and online performances of students with learning disabilities British Educational Research Journal

Tong Y amp Kolen M J (2010) Scaling an ITEMS module Educational Measurement Issues and Practice 29 13-48

Wan L amp Henly G (in press) Measurement properties of two innovative item formats in a computer-based test Applied Measurement in Education

Conference ParticipationCheng J amp Shen J (2010 December) Towards accurate recognition for childrenrsquos oral reading fluency Poster presented at the third IEEE Workshop on Spoken Language Technology Berkeley CA

Dolan R P (2010 October) Inclusive next generation assessment Implications for technical quality evaluations Panelist at the Reidy Interactive Lecture Series Cambridge MA

Dolan R P (2010 November) Cutting edge technology issues on the high-stakes testing horizon Paper presented at the annual meeting of the Testing Agencies Disability Forum Reston VA

Hill D amp Neville A (2010 December) ldquoBut I found it on the internetrdquo Poster presented at the annual meeting of the National Council for the Social Studies Conference Denver CO

Miles J (2010 November) Establishing cut scores for the SOL assessments Presentation given at the annual meeting of the Virginia Association of Testing Directors Assessment Conference Richmond VA

Miles J (2010 November) Unraveling mysteries Scale scores cut-scores IRTmdashClear explanations Presentation given at the annual meeting of the Virginia Association of Testing Directors Assessment Conference Richmond VA

Van Moere A Suzuki M amp Klungtvedt M (2010 October) Time is money Assessing efficient use of written English skills for work purposes Paper presented at the annual meeting of the East Coast Organization of Language Testers Georgetown University Washington DC

Way D amp Kirkpatrick R (2010 October) What does a state need to do to prepare for and to transition to computer-based assessments from the viewpoint of a contractor Presentation given at the annual meeting of the Maryland Assessment Research Center for Education Success College Park MD

Zyskowski G amp OrsquoMalley K (2010 December) From Z to AmdashSetting the STAAR cut points from end to beginning Paper presented at the annual meeting of the Texas Assessment Conference Austin TX

Zyskowski G amp OrsquoMalley K (2010 December) Getting STAARted Field testing in 2011 and beyond Paper presented at the annual meeting of the Texas Assessment Conference Austin TX

Upcoming Conference ParticipationHarvey B amp Parker B (2011 April) Using SAS to enter variables in a DOS window Paper to be presented at the annual meeting of the SAS Global Forum Codersrsquo Corner Las Vegas NV

Keng L Kong J amp Bleil B (2011 February) Does size matter A study on the use of netbooks in Kndash12 assessment Paper to be presented at the annual meeting of the Association of Test Publishers Innovations in Testing Phoenix AZ

notes from readers

Jason Schwartz MS Director of Content Development Pearson VUErsquos NCLEX program

I enjoy both the newsletter and the blog The layout of the newsletter is attractive and the use of photos helps to personalize the articles and place faces with the names we occasionally come across in our extended Pearson family Though I am not a researcher I enjoy the lists of research publications Please keep up the good work with these nice publications

1 2 3 4 5 6 7 8pearsonrsquos test measurement amp research services Quarterly Newsletter v3 n4 2010 7

Upcoming Conference ParticipationContinued from page 6

In November the following proposals by TMRS staff were accepted for presentation at the 2011 American Educational Research Association Annual Meeting in New Orleans Louisiana

Beimers J Burling K amp Veazey M Exploring algebra II studentsrsquo problem solving strategies

Brown R S Nguyen T Sun Y amp Cruz B A meta-analytic investigation of the effect of selection bias in research addressing the effectiveness of freshman seminars

Fulkerson D Nichols P amp Snow E Expanding the model of item writing expertise Cognitive processes and requisite knowledge structures

He W amp Wolfe E W IRT true-score equating for tests composed of testlets A comparison of bifactor model testlet model 2PL IRT model and generalized partial credit model

Lau C A amp Arce-Ferrer A Comparing methods for detecting unstable anchor items with net DIF and global DIF conceptions

McGill M amp Wolfe E W Investigation of precision in Rasch difficulty estimation

Moyer E Galindo J amp Dodd B Comparing constraint balancing methods for flexible constraints in computer adaptive tests

Nichols P D Tong Y Miles J Kreiman C amp Hall E Applications of classical true score theory to standard setting studies

Song T Investigating the effect of DIF in common-item nonequivalent group equating design

Way D Trends in formative assessment theory and practice in the United States

Wolfe E W Rater cognition and its importance for score validity Global perspectives and findings

Yue J amp Wolfe E W Can we identify raters who assign inconsistent scoresmdashDetecting rater inaccuracy using simulation methods

In addition the following proposals by TMRS staff were accepted for presentation at the 2011 National Council on Measurement in Education Annual Meeting in New Orleans Louisiana

Arce-Ferrer A amp Lau C A Detecting unstable anchor items in 3PL IRT equating with robust z An investigation with real and simulated data sets

Kolen M J amp Tong Y Vertical scaling methodologies applications and research

Seo D Taherbhai H amp Frantz R Exploring different item profiles in English language proficiency (ELP) listening assessment using latent mixture distribution analysis

Song T The effect of fitting a unidimensional IRT model to multidimensional data in content-balanced computerized adaptive testing

Tong Y Wu S OrsquoMalley K amp Xu M Growth models and their extension

Wan L amp Wei H An empirical examination of two model-fit indices using real data

Wei H amp Lin J Equating to the item bank or to the previous form Does it matter

Zhang L amp Lau C A Examine the effect of policy change on equating

notes from readers

Karen R Owens PhD Academic Assessment Consultant Pearson eCollege

Our Pearson eTeaching Online Blogucation is a forum we use to discuss industry news and events innovations in education and best practices of online education for our Educational Partners (EPs) In writing my blog for December I suggest the Pearson TMRS Newsletter as a valuable resource for educators to easily reference current research and conference presentations that directly address student success through innovations in testing and measurement The newsletter is a quick orientation to your research efforts that not only educate and inform practitioners but provides them with a call to action All in the name of ldquopersonalized learningrdquo

1 2 3 4 5 6 7 8pearsonrsquos test measurement amp research services Quarterly Newsletter v3 n4 2010

Send information to

Jason Wachsmuth 2510 North Dodge Street Iowa City IA 52245 jasonwachsmuthpearsoncom 3193396400 Ext 216005

TrueScoresEach issue of the Pearson TMRS Newsletter includes a recent entry from the TrueScores blog For more information on TrueScores please visit truescorescom

Where Will Innovation Come From I have just come from a learning trajectories conference associated with the DELTA Project at North Carolina State University This conference was supported by the Pearson Foundation At this event there was a lot of talk about innovation in how and what we assess including talk about so-called

ldquoinnovativerdquo items Those are items that might use simulation multimedia and figural response item formats The Race to the Top consortia want to hire companies to construct these innovative items that will be freely shared across states But where will innovations in assessment items and tasks come from

There are people who do research on innovations and where innovations come from Consider Eric von Hippel a professor of technological innovation at MIT In the book The Sources of Innovation von Hippel points out that companies that manufacture in a field were innovators in that field if the companies could expect the innovation to become a commercially successful product that would result in an attractive return These innovative companies gained an enhanced position in the market But the research by von Hippel raises a question What motivates companies to invest in developing innovative items if those innovations will then be given away by the consortia to the states Where is the enhanced market position Are the consortia taking the best approach to ensuring innovation in how and what we assess

Some other researchers on innovation Kevin Boudreau and Karim Lakhani writing about outside innovation might have an answer They say that when the technology and the customersrsquo needs are well understood a company can do internal research and development But opening up innovation to a collaborative community can work better when the technology and design

approaches have not been established and the innovation involves cumulative knowledge continually building on past advances

These collaborative communities tend to develop knowledge-sharing and dissemination mechanisms and converge on common norms with a culture of sharing and cooperation broad agreement on a technology paradigm and common technical jargon to support productive collaboration A good example of this is Applersquos iPhone Thousands of outside software developers have written complementary applications (the apps we all love so much) that have made the iPhone the center of a thriving business ecosystem Another but somewhat different example is the Semiconductor Research Corp a Durham North Carolina-based nonprofit consortium that includes members from industry the government and academia The Semiconductor Research Corp was established in 1982 to accumulate fundamental knowledge in silicon technology and semiconductor manufacturing

As the Rolling Stones sang ldquoYou canrsquot always get what you want But if you try sometimes yeah you just might find you get what you needrdquo I donrsquot know the best way to guarantee we get the innovation we want in how and what we assess But after meeting and listening to the researchers and policymakers at the learning trajectories conference a way to get what we need might be a partnership between companies like Pearson the government and academia

Paul Nichols PhD Vice President Psychometric amp Research Services Assessment amp Information Pearson

Paul Nichols PhD

8

notes from readers

Julie Miles PhD Director Psychometric amp Research Services

I shared Kelly Burlingrsquos essay on 2 assessments with a client to show that their state was struggling with the same issues that the entire field is facing and that our company was thoughtfully pursuing the topic and considering how the assessment arena will change in the coming years with the RTTT initiative