Making Connections: ETS-led Presentations at AERA and NCME 2011

Copyright © 2011 by Educational Testing Service. All rights reserved. ETS, the ETS logo and LISTENING. LEARNING. LEADING. are registered trademarks of Educational Testing Service (ETS) in the United States of America and other countries throughout the world. 4/1/2011 v4.3

www.ets.org

Making Connections ETS-led Presentations and Professional Development Courses at the 2011 Annual Meetings of the American Educational Research Association (AERA) and the National Council on Measurement in Education (NCME) April 7-12, 2011 New Orleans, LA

Follow this schedule on Twitter (@ETSresearch).

2 | Making Connections – 2011

Making Connections lists only ETS-led presentations and courses at AERA-NCME 2011. For session chairs, discussants, and presentations led by other institutions, see the AERA and NCME programs. 4/1/2011 v4.3

About This Program Making Connections lists only those parts of sessions for which AERA or NCME program materials have credited an ETS staff member as a first author, first presenter, or lead course director. For the names of session chairs, discussants, and presentations led by other institutions, see the AERA and NCME programs online at www.aera.net and www.ncme.org. Session times and locations are based on information received from AERA and NCME and are subject to change. Chronological Listing of Sessions Thursday, April 7, 2011 .................................................................................... 5

Friday, April 8, 2011 ......................................................................................... 5

Saturday, April 9, 2011 ..................................................................................... 8

Sunday, April 10, 2011 ................................................................................... 14

Monday, April 11, 2011 ................................................................................... 20

Tuesday, April 12, 2011 .................................................................................. 27

Index of Topics Admissions testing ................................................................................................... 8, 14

Assessment development ............................................................................................. 28

Assessment innovation .................................................................................. 9, 20, 21, 23

Bayesian models and methods .............................................................................. 5, 8, 18

Cognitively Based Assessment of, for, and as Learning (CBAL) initiative .................. 10, 18, 27

College and career readiness .................................................................................... 5, 17

Communicating about assessment ............................................................................. 7, 23

Computerized adaptive testing ............................................................................. 8, 12, 23

Construct evaluation ...................................................................................................... 6

Constructed-response items ................................................................................... 19, 20

Constructed-response scoring, automated ....................................................... 9, 11, 21, 25

Constructed-response scoring, human raters ....................................................... 11, 15, 29

Differential item functioning (DIF) ........................................................... 14, 16, 18, 21, 28

English language learning ........................................................ 6, 13, 16, 21, 22, 24, 28, 29

Equating scores .............................................. 6, 8, 9, 12, 13, 16, 17, 19, 20, 21, 22, 24, 28

Equating subscores ................................................................................................ 15, 25

Equating, vertical ........................................................................................................ 26

Evidence-centered design ....................................................................................... 20, 26

http://www.aera.net/�

http://www.ncme.org/�



Fairness ............................................................................................................... 15, 21

Formative assessment ........................................................................................... 15, 26

Graduate education ..................................................................................................... 14

Growth models ...................................................................................................... 17, 26

Higher education, diversity in ........................................................................................ 25

History of testing ......................................................................................................... 18

Invariance .............................................................................................................. 6, 15

Item generation ............................................................................................................ 9

Item modeling .............................................................................................................. 9

Item response theory (IRT) .................................................................... 16, 17, 20, 22, 26

K-12 assessment ........................................................... 8, 9, 10, 13, 15, 17, 18, 23, 26, 28

Large-scale comparative assessment .................................................................. 11, 23, 27

Linking mixed-format tests ........................................................................................... 19

Linking scores ......................................................................................................... 6, 16

Linking, IRT ................................................................................................................ 20

Linking, vertical ........................................................................................................... 12

Literacy assessment .................................................................................................... 27

Measurement conditions ............................................................................................... 15

Multistage testing .................................................................................................. 12, 23

National Assessment of Educational Progress (NAEP) ................................. 11, 12, 14, 16, 19

Natural language processing ..................................................................................... 9, 21

Noncognitive assessment ................................................................... 5, 11, 13, 17, 22, 27

People with disabilities ................................................................................ 17, 21, 23, 26

Personality assessment ................................................................................................ 11

Policy research ............................................................................................................ 12

Race to the Top Assessment Program ............................................................... 8, 9, 13, 18

Rasch models........................................................................................................ 14, 24

Reading assessment .............................................................................. 10, 14, 16, 24, 27

Reliability ............................................................................................................. 15, 19

Repurposing tests .......................................................................................................... 6

Scale aligning ............................................................................................................... 6

Scale drift ..................................................................................................................... 9

Scales ............................................................................................................... 9, 17, 20

Score aggregation ......................................................................................................... 7

Score conversions ....................................................................................................... 19

Score reporting .................................................................................................. 7, 19, 27



Speech recognition ...................................................................................................... 25

Standard setting ................................................................................................... 19, 26

Subscores ......................................................................................................... 7, 15, 25

Teacher licensure assessment ......................................................................................... 6

Teaching quality and effectiveness .............................................................. 5, 7, 13, 23, 25

Technology in assessment ............................................................................................ 23

Test assembly ............................................................................................................. 16

Test design ........................................................................................................... 10, 16

Test speededness ........................................................................................................ 24

Text complexity applications ................................................................................... 10, 16

Trend scores ............................................................................................................... 11

Validity issues .................................................................................................... 9, 15, 20



Thursday, April 7, 2011

Session Titles and Details Session Topic(s)

Learning Mathematics for Teaching: Instrument Dissemination Workshop

AERA — Professional Development Course Thursday, April 7, 9 am–5 pm New Orleans Marriott / La Galerie 5

Directors: Heather C. Hill (Harvard University), Geoffrey C. Phelps (ETS)

Teaching quality and effectiveness

Friday, April 8, 2011


Bayesian Networks in Educational Assessment

NCME — Pre-conference Training Session Friday, April 8, 8 am–5 pm Westin / Magnolia Ballroom I, LL

Directors: Duanli Yan (ETS), Robert J. Mislevy (ETS), Russell G. Almond (Florida State University), David M. Williamson (ETS)

Bayesian models and methods

Assessing 21st Century Skills

NCME — Pre-conference Training Session Friday, April 8, 8 am–5 pm Westin / Magnolia Ballroom III, MM

Directors: Patrick Kyllonen (ETS), Richard Roberts (ETS), Jonas Bertling (ETS), Jeremy Burrus (ETS)

College and career readiness

Noncognitive assessment





Assessment and Instruction of Academic English Language Proficiency for English Language Learners

AERA — Symposium Friday, April 8, noon–1:30 pm Doubletree / Rosedown A

Teacher Perspectives on Making Academic Language Accessible to English Language Learners (ELLs) Presenter/Author: Sultan Turkan (ETS)

Investigating the Constructs of English Language Proficiency Assessments and English Language Learner Students' Performance on the Assessments Presenters/Authors: Mikyung Kim Wolf (ETS), Yuan Wang (ETS), Steven L. Holtzman (ETS)

Construct evaluation

English language learning

Linking and Aligning Scores and Scales

NCME — Pre-conference Training Session Friday, April 8, 1 pm-5 pm Westin / 12th Floor Ballroom II, NN

Directors: Jinghua Liu (ETS), Neil J. Dorans (ETS), Michael E. Walker (ETS), Mary Pommerich (Defense Manpower Data Center)

Linking scores

Equating scores

Scale aligning

Understanding and Evaluating Equating Invariance in Multiple Contexts

AERA — Paper Session Friday, April 8, 2:15 pm–3:45 pm Doubletree / Madewood B

Estimation of Population Invariance in Equating Mathematics and Science Tests Across Different English Proficiency Groups Presenters/Authors: Lilly (Yuli) Zhang (ETS), Mary C. Grant (ETS), Kathleen Siemborski (ETS) Evidence Pertaining to Repurposing: Reliability and Invariance of Linking Functions Presenters/Authors: Yanlin Jiang (ETS), Insu Paek (Florida State University), Neil J. Dorans (ETS)

Examining Possible Construct Changes to a Licensure Test by Evaluating Invariance Assumptions Presenters/Authors: Sooyeon Kim (ETS), Michael E. Walker (ETS), Kevin C. Larkin (ETS)

Construct evaluation


Equating scores

Invariance

Repurposing tests

Teacher licensure assessment





Communicating Assessment Results to Particular Audiences

AERA — Symposium Friday, April 8, 2:15 pm–3:45 pm Doubletree / Rosedown A

Exploring Teachers' Understanding and Use of Assessment Results Presenters/Authors: Diego Zapata-Rivera (ETS), Waverely VanWinkle (ETS), Rebecca Zwick (ETS)

Designing and Evaluating an Interactive Score Report for Students Presenters/Authors: Margaret Vezzu (ETS), Waverely VanWinkle (ETS), Diego Zapata-Rivera (ETS) Question-Based Reports for Policymakers Presenters/Authors: Waverely VanWinkle (ETS), Margaret Vezzu (ETS), Diego Zapata-Rivera (ETS)

Communicating about assessment

Score reporting

Finding Value in Aggregated Scores and Subscores

AERA — Roundtable Session Friday, April 8, 4:05 pm–5:35 pm Doubletree / International Ballroom

A Comparison of Score Aggregation Methods on Simulated Test Data Presenter/Author: Jianbin Fu (ETS)

Score aggregation

Subscores

Linking Mathematics Classroom Practices to Student Outcomes

AERA — Symposium Friday, April 8, 4:05 pm–6:05 pm New Orleans Marriott / Preservation Hall Studio 9

Toward an Understanding of Classroom Context and Student Learning of Algebra Presenters/Authors: Drew H. Gitomer (ETS), Courtney A. Bell (ETS), Bridget Kathleen Hamre, (University of Virginia), Robert Pianta (University of Virginia), Daniel McCaffrey (RAND)




Saturday, April 9, 2011 Session Titles and Details Session Topic(s)

Issues in College Testing and Assessment

AERA — Paper Session Saturday, April 9, 8:15 am–9:45 am Sheraton / Salon 820

Reducing Systematic Errors in College Grade Predictions Using an Index of High School Socioeconomic Status Presenters/Authors: Rebecca Zwick (ETS), Igor Himelfarb (University of California - Santa Barbara)

Admissions testing

Measuring Student Growth: New Advances and Major Challenges in the State Assessment Consortia Initiatives

AERA — Symposium Saturday, April 9, 8:15 am-10:15 am Sheraton / Napoleon Ballroom C3

An Overview of the Partnership for Assessment of Readiness for College and Careers (PARCC) and SMARTER Balanced Assessment Consortium (SBAC) Assessment Consortia Plans Presenter/Author: Nancy Doorey (Center for K-12 Assessment and Performance Management at ETS)

K-12 assessment

Race to the Top Assessment Program

Computerized Adaptive Tests for Classification: Algorithms and Applications

NCME — Coordinated Session Saturday, April 9, 8:15 am-10:15 am Westin / Azalea Ballroom II, A2

Computerized Mastery Testing: Using Bayesian Sequential Analysis to Make Multiple Classification Decisions Presenters/Authors: Robert Smith (ETS), Charles Lewis (Fordham University)


Computerized adaptive testing

Factors Affecting the Quality of Equating

NCME — Paper Session Saturday, April 9, 8:15 am-10:15 am Westin / Magnolia Ballroom I, A3

Effects of Group Ability Differences and Group Variability on Equating Accuracy Presenters/Authors: Hyeonjoo Oh (ETS), Xiaoyu Qian (University of Delaware), Hongwen Guo (ETS)

Equating scores




Panel on Innovative Opportunities and Measurement Challenges in Through-Course Summative Assessments: State RTTT Assessment Consortia Development Plans

NCME — Invited Symposium Saturday, April 9, 10:35 am-12:05 pm Westin / 12th Floor Ballroom II, B6

Presenter: Nancy Doorey (Center for K-12 Assessment and Performance Management at ETS)

Panelists: Randy Bennett (ETS), Pascal D. Forgione (Center for K-12 Assessment and Performance Management at ETS), Andrew Ho (Harvard Graduate School of Education), Michael Kane (ETS), Michael Kolen (University of Iowa), Laurie Wise (Human Resources Research Organization), Rebecca Zwick (ETS)

Assessment innovation

K-12 assessment


Scale Stability and Scale Drift

NCME — Paper Session Saturday, April 9, 10:35 am-12:05 pm Westin / Azalea Ballroom II, B2

Multiple-Linking Equating and Random Scale Drift Presenters/Authors: Hongwen Guo (ETS), Jinghua Liu (ETS), Neil J. Dorans (ETS), Miriam Feigenbaum (ETS)

Comparison of Frequency Estimation Method or Chained Equipercentile Method in Terms of Scale Stability Presenters/Authors: Yanxuan Qu (ETS), Lixiong Gu (ETS), Bihua Xiang (ETS)

Equating scores

Scale drift

Scales

Item Modeling and Item Generation for the Measurement of Quantitative Skills: Recent Advances and Prospects

NCME — Coordinated Session Saturday, April 9, 10:35 am-12:05 pm Westin / Magnolia Ballroom III, B4

Integrating Item Generation and Automated Scoring Presenter/Author: James Fife (ETS)

Item Generation in the Context of a Validity Argument Presenter/Author: Isaac Bejar (ETS)

Constructed-response scoring, automated

Item generation

Item modeling

Natural language processing

Validity issues




Cognition and Assessment

AERA — Paper Session Saturday, April 9, 10:35 am-12:05 pm Doubletree / Rosedown B

Dimensionality Analysis of Cognitively Based Assessment of, for, and as Learning (CBAL) Grade 8 Writing Tests Presenters/Authors: Jianbin Fu (ETS), Hanwook Yoo (University of Massachusetts - Amherst), Paul Deane (ETS)

Cognitively Based Assessment of, for, and as Learning (CBAL) initiative

K-12 assessment

Diverse Topics in Psychometrics and Educational Measurement

AERA — Poster Session Saturday, April 9, 10:35 am-12:05 pm Sheraton / Grand Ballroom C

Maximizing the Alignment Between Automated Text Complexity Classifications and Targeted Text Complexity Standards Presenters/Authors: Kathleen M. Sheehan (ETS), Irene Kostin (ETS), Yoko Futagi (ETS)


Reading assessment

Text complexity applications

How Design Patterns Address Three Pressing Challenges in Large-Scale Assessment

AERA — Symposium Saturday, April 9, 10:35 am-12:05 pm Sheraton / Grand Couteau

The Critical Role of Design Patterns in Large-Scale Assessment Presenters/Authors: Robert J. Mislevy (ETS), Geneva D. Haertel (SRI International)

Test design




Methodological Issues in the Analysis of Personality in Education

NCME — Coordinated Session Saturday, April 9, 12:25 pm-1:55 pm Westin / Azalea Ballroom II, C2

Factorial Versus Typological Models for Data from Personality Tests and Survey Scales Presenters/Authors: Matthias von Davier (ETS), Bobby Naemi (ETS), Richard Roberts (ETS)

A New Look at the Structure of Emotional Stability Presenters/Authors: Duanli Yan (ETS)


Personality assessment

To Break Trend or Not, That is the Question

NCME — Coordinated Session Saturday, April 9, 12:25 pm-1:55 pm Westin / Magnolia Ballroom I, C3

To Break or Continue Trend: The Role of Psychometric Studies of Old and New Items Presenters/Authors: Rebecca Moran (ETS), Meng Wu (ETS)

Large-scale comparative assessment

NAEP

Learning from Innovation: Refining the Practice of Automated Scoring of Essays

NCME — Coordinated Session Saturday, April 9, 12:25 pm-1:55 pm Westin / Azalea Ballroom I, C1

Using Automated Scoring as a Trend Score: The Implications of Score Separation Over Time Presenters/Authors: Catherine Trapani (ETS), Brent Bridgeman (ETS)

The Question of Validity and Differentially Valued Evidence Presenter/Author: Brent Bridgeman (ETS)

Implications of Subgroup Differences in Agreement Between Automated and Human Scores Presenter/Author: Chaitanya Ramineni (ETS)


Constructed-response scoring, human raters

Trend scores




Using National Assessment of Educational Progress Data on the Web for Educational Policy Research AERA — Professional Development Course Saturday, April 9, 1-5 pm Hotel Monteleone / La Nouvelle Orleans West

Directors: Debra Kline (ETS), Catherine S. Trapani (ETS), Emmanuel Sikali (U.S. Department of Education)

NAEP

Policy research

Recent Advances in Equating NCME — Paper Session Saturday, April 9, 2:15 pm-3:45 pm Westin / Magnolia Ballroom I, D3

Assessing Invariance of Vertical Linking Functions Presenters/Authors: Peng Lin (ETS), Neil J. Dorans (ETS)

Test Score Equating Using a Mini-Version Anchor and an Anchor with Less Spread of Difficulty: A Case Study Using Operational Data Presenters/Authors: Jinghua Liu (ETS), Paul Holland (Paul Holland Consulting Corporation), Edward Curley (ETS), Miriam Feigenbaum (ETS)

Equating scores

Linking, vertical

Modeling and Estimation with Multistage Testing NCME — Paper Session Saturday, April 9, 2:15 pm-3:45 pm Westin / River Room, D6

A Tree-Based Approach for Multi-Stage Testing Presenters/Authors: Duanli Yan (ETS)

Modeling the Local Item Dependence with Multistage Tests Presenters/Authors: Ru Lu (ETS), Hong Jiao (University of Maryland-College Park)

Multistage testing


Item Selection and Item Calibration in Computer-Based Testing NCME — Paper Session Saturday, April 9, 4:05 pm-6:05 pm Westin / River Room, E6

Item and Testlet Selection Procedures for Mixed-Format CAT Based on the Testlet Response Theory Model Presenters/Authors: Tsung-Han Ho (ETS)





Excellence in Education Research: Early Career Scholars and Their Work

AERA — Invited Poster Session Saturday, April 9, 4:05 pm–6:05 pm New Orleans Marriott / Mardi Gras Salon DE

Cognitive Abilities Tests in the Modern, Diverse Classroom: Validity of the CogAT for Students who are English-Language Learners Presenter/Author: Joni M. Lakin (ETS)

Non-Cognitive Characteristics and Student Success: Examining the Role of Efficacy, Self Concept, Cooperativeness, and Emotional Health Presenter/Author: Diane C. Elliott (ETS)



Item Parameter Drift

NCME — Paper Session Saturday, April 9, 4:05 pm-6:05 pm Westin / Magnolia Ballroom I, E3

Examining the Impact of Drifted Polytomous Anchor Items on TCC Linking Presenter/Author: Yanmei Li (ETS)

Item parameters

Linking scores

Measuring Teacher Effectiveness: New Advances and Major Challenges in the State Assessment Consortia Initiatives

AERA — Symposium Saturday, April 9, 4:06- 6:05 pm Sheraton / Rhythms Ballroom I

An Overview of the Partnership for Assessment of Readiness for College and Careers (PARCC) and SMARTER Balanced Assessment Consortium (SBAC) Assessment Consortia Plans Presenter/Author: Nancy Doorey (Center for K-12 Assessment and Performance Management at ETS)



K-12 assessment



Sunday, April 10, 2011 Session Titles and Details Session Topic(s)

A Hands-on Introduction to Latent Class Models, Rasch Models, and Their Extensions

AERA — Professional Development Course Sunday, April 10, 8:00 am–noon Hotel Monteleone / Riverview

Director: Matthias Von Davier (ETS)

Rasch models

Reading Researchers in Search of Common Ground: The Expert Study Revisited

AERA — Symposium Sunday, April 10, 8:15 am-9:45 am Doubletree / Madewood A

A Focus on Past and Current National Assessment of Educational Progress Reading Data Presenter/Author: Jay R. Campbell (ETS)

Reading assessment

NAEP

Studies in Rasch Conditions and Applications

AERA — Paper Session Sunday, April 10, 10:35 am-12:05 pm Doubletree / Rosedown B

A Comparison of Two Estimation Methods for the Many-Facet Rasch Model Using Real Data From a Large-Scale Language Assessment Presenters/Authors: Guangming Ling (ETS), Lixiong Gu (ETS)

Rasch models

Validity Research on College and Graduate School Admission Tests

AERA — Paper Session Sunday, April 10, 10:35 am-12:05 pm Doubletree / Nottaway

Differential Performance of Women and Ethnic Minority Groups Across a High- and Low-Stakes Testing Situation Presenters/Authors: Yigal Attali (ETS), Zvika Neeman (Tel Aviv University)

Admissions testing

Graduate education

DIF




Career Award Address: The Contestant Perspective on Taking Tests

NCME — Invited Address Sunday, April 10, 10:35 am-12:05 pm Westin / 12th Floor Ballroom II, F5

Presenter/Author Neil J. Dorans (ETS)

Reliability

Measurement conditions

Invariance

Fairness

Measuring Classroom Formative Assessment Use

AERA — Roundtable Session Sunday, April 10, 10:35 am-12:05 pm Sheraton / Grand Ballroom D

A Validity Argument for the Assessment Practices Survey: A Measure of Teachers' Enactment of Formative Assessment Presenters/Authors: E. Caroline Wylie (ETS), Susan J. Miller (ETS)

Formative assessment

K-12 assessment

Validity issues

What’s New from the “Educational Measurement: Issues and Practice” – A Sample of Recently Published Papers

NCME — Invited Symposium Sunday, April 10, 10:35 am-12:05 pm Westin / Azalea Ballroom I, F1

Equating Subscores Under the Non Equivalent Anchor Test (NEAT) Design Presenter/Author: Gautam Puhan (ETS), Longjuan Liang (ETS)

Equating subscores

Subscores

Rater Issues

NCME — Paper Session Sunday, April 10, 10:35 am-12:05 pm Westin / Magnolia Ballroom I, F3

The Effects of Scoring Designs and Rater Severity on Students’ Ability Estimation for Constructed Response Items Presenters/Authors: Zhen (Jane) Wang, ETS, Lihua Yao, Defense Manpower Data Center





Estimating Parameters of Multidimensional Item Response Theory Models

NCME — Paper Session Sunday, April 10, 12:25 pm-1:55 pm Westin / Magnolia Ballroom III, G4

Unidimensional and Composite Scaling Comparisons for NAEP Presenters/Authors: Meng Wu (ETS), Adrienne Sgammato (ETS), Yue Jia (ETS)

IRT

NAEP

Linking and Equating

NCME — Paper Session Sunday, April 10, 12:25 pm-1:55 pm Westin / Magnolia Ballroom I, G3

Standard Error of Linear Equating for the NEAT Design with Nonnormality Distributed Data Presenter/Author: Jiyun Zu (ETS)

Linking scores

Equating scores

Innovations in Test and Item Design Using Item Difficulty Modeling

NCME — Coordinated Session Sunday, April 10, 12:25 pm-1:55 pm Westin / Azalea Ballroom I, G1

An Automated Approach for Modeling the Passage Complexity Component of Reading Item Difficulty Presenters/Authors: Kathleen Sheehan (ETS), Irene Kostin (ETS), Yoko Futagi (ETS)

Examining Sources of DIF for Language-Minority Groups in K-12 Mathematics Assessments Presenters/Authors: Maria Martiniello (ETS), Paula Elosua (Universidad del Pais Vasco)

DIF


Reading assessment

Test design

Text complexity applications

Test Design and Assembly

NCME — Paper Session Sunday, April 10, 2:15 pm-3:45 pm Westin / Terrace Room, H7

Evaluating the Psychometric Properties of a Redesigned Test Without a Field Trial Presenters/Authors: Michael Walker (ETS), Sooyeon Kim (ETS)

Test design

Test assembly




Exploring the Link Between Multiple Measures and Student Outcomes

AERA — Paper Session Sunday, April 10, 2:15 pm-3:45 pm Doubletree / Shadows

Measuring the Relationship Between Conscientiousness and Academic Outcomes Using Self and Parent Reports Presenters/Authors: Matthew Ventura (ETS), Carolyn Elizabeth MacCann (ETS), Richard Dean Roberts (ETS)

College and career readiness


Quality Control and Data Mining Techniques Applied to Monitoring Scale Scores

AERA — Roundtable Session Sunday, April 10, 2:15 pm-3:45 pm Doubletree / International Ballroom

Monitoring Means and Variances of Item Response Theory Equated Raw Scores Using Linking Coefficients Presenter/Author: Deping Li (ETS)

Detection of Change Points in the Means of Scale Scores Presenters/Authors: Yi-Hsuan Lee (ETS), Alina A. Von Davier (ETS)

Monitoring the Scale Stability Presenters/Authors: Alina A. Von Davier (ETS), Avi Allalouf (National Institute for Testing and Evaluation)

Equating scores

IRT

Scales

Modeling Growth

NCME — Paper Session Sunday, April 10, 2:15 pm-3:45 pm Westin / Azalea Ballroom II, H2

A Comparison of Regression-Based Growth Models for Students with Disabilities Taking a State General Accountability Assessment Presenter/Author: Heather Buzick (ETS)

Growth models

K-12 assessment

People with disabilities




DIF Methodology

NCME — Paper Session Sunday, April 10, 2:15 pm-3:45 pm Westin / Magnolia Ballroom III, H4

Improving Mantel-Haenszel DIF Estimation Through Bayesian Updating Presenters/Authors: Rebecca Zwick (ETS), Lei Ye (ETS), Steven Isham (ETS)

DIF


Perspectives on the History of Testing in the United States

NCME — Coordinated Session Sunday, April 10, 2:15 pm-3:45 pm Westin / 12th Floor Ballroom I, H5

A Brief History of ETS as a Scientific Organization Presenters/Authors: Ida Lawrence (ETS), Ed Shea (ETS)

History of testing

Integrating Classroom Assessments Into Balanced State Assessment Systems

AERA — Symposium Sunday, April 10, 2:15 pm-3:45 pm Sheraton / Rhythms Ballroom III

Cognitively Based Assessment of, for, and as Learning (CBAL): Results From Piloting Innovative K-12 Assessments Presenter/Author: Randy E. Bennett (ETS)


K-12 assessment





Choice of the Anchor Test in Equating

NCME — Paper Session Sunday, April 10, 4:05 pm-6:05 pm Westin / 12th Floor Ballroom I, I5

Exploring Alternative Designs Using Reduced Equating Samples and Anchors Presenters/Authors: Lin Wang (ETS), Jiahe Qian (ETS), Yi-Hsuan Lee (ETS)

The Impact of Non-Representative Anchor Sets on the Comparability of Test Scores for Mixed-Format Exams Presenters/Authors: Ourania Rotou (ETS), Michael Walker (ETS), Neil J. Dorans (ETS)

Does Linking Mixed-Format Tests Using a Multiple-Choice Anchor Produce Comparable Results for Male and Female Subgroups Presenters/Authors: Sooyeon Kim (ETS), Michael Walker (ETS)

Constructed-response items

Equating scores

Linking mixed-format tests

Standard Setting in an International Context: Issues and Practice

NCME — Coordinated Session Sunday, April 10, 4:05 pm-6:05 pm Westin / Azalea Ballroom I, I1

Standard Setting Lessons Learned in the South African Context Presenters/Authors: Mary Pitoniak (ETS), Nan Yeld (University of Cape Town)

Standard setting

Scoring and Score Reporting

NCME — Paper Session Sunday, April 10, 4:05 pm-6:05 pm Westin / Azalea Ballroom II, I2

Performance of Reliability Measures for NAEP Presenter/Author: Adrienne Sgammato (ETS)

Assessing the Practical Equivalence of Conversions Presenters/Authors: Neil J. Dorans (ETS), Jinghua Liu (ETS)

A Study on Different Weighting Methods for Tests with Constructed Response Items Presenters/Authors: Youhua Wei (ETS), Chi-Wen Liao (ETS)


NAEP

Reliability

Score conversions

Score reporting



Monday, April 11, 2011 Session Titles and Details Session Topic(s)

Designing Scenario-Based Assessment Items Using an Evidence-Centered Design Framework

AERA — Professional Development Course Monday, April 11, 8 am–noon Hotel Monteleone / Queen Anne Ballroom

Director: Robert J. Mislevy (ETS)


Evidence-centered design

Validity Issues

NCME — Paper Session Monday, April 11, 8:15 am-10:15 am Westin / Azalea Ballroom II, J2

Comparability of Essay Question Variants Brent Bridgeman (ETS), Catherine Trapani (ETS), Jennifer Bivens-Tatum (ETS)


Validity issues

Maintaining the Stability of the Reported Scale Scores: Issue and Methods

NCME — Coordinated Session Monday, April 11, 8:15 am-10:15 am Westin / Azalea Ballroom I, J1

Observed Score Equating with a Heterogeneous Target Population Presenters/Authors: Alina von Davier (ETS), Minh Q. Duong (Michigan State University)

Linking Parameter Estimates Derived From an Item Response Model Through Separate Calibrations Presenters/Authors: Shelby Haberman (ETS), Zhiming Yang (ETS)

Effects of Weighting Examinee Samples in TBLT IRT Linking Presenters/Authors: Jiahe Qian (ETS), Yanming Jiang (ETS), Alina von Davier (ETS)

Evaluating Equating Results from Uncertain Samples for Reasonableness Presenters/Authors: Yuming Liu (ETS), Rick Morgan (ETS), Brad Moulder (ETS)

Equating scores

IRT

Linking, IRT

Scales




Innovations in Scoring

NCME — Paper Session Monday, April 11, 8:15 am-10:15 am Westin / 12th Floor Ballroom II, J6

The Use of Automated Essay Scores in Graduate Admissions Putting the Innovation of Automated Essay Scoring into Operational Practice Presenters/Authors: Yigal Attali (ETS)




Fairness Across Novel Item Types

NCME — Paper Session Monday, April 11, 8:15 am-10:15 am Westin / Magnolia Ballroom III, J4

Innovations in Test Directions for Novel Item Formats and Culturally and Linguistically Diverse Students Presenter/Author: Joni Lakin (ETS)

Identifying Sources of Construct-Irrelevant Variance of Scenario-Based Computer-Delivered Mathematics Items for Students with Disabilities and English Language Learners Presenter/Author: Teresa King (ETS)

Fairness



Fairness, DIF and Language Impact

NCME — Paper Session Monday, April 11, 8:15 am-10:15 am Westin / 12th Floor Ballroom I, J5

Examining the Effect of Primary Language on Equating Functions in MC and CR Tests Presenters/Authors: Jaime Cid (ETS), Jiyun Zu (ETS), Gautam Puhan (ETS)

Fairness

DIF


Equating scores




Equating Using Item Response Theory Models

NCME — Paper Session Monday, April 11, 8:15 am-10:15 am Westin / Terrace Room, J7

The Effect of Total Test and Anchor Test Difficulty Differences on IRT Equating under the Common-Item Nonequivalent Groups Design Presenters/Authors: Teresa Dawber (ETS), Liane Patsula (ETS), Feifei Li (ETS), Maxwell Wise (ETS)

Effects of Empirical Distributions on the Robustness of IRT Calibration and True Score Equating Yanming Jiang (ETS), Alina von Davier (ETS), Jiahe Qian (ETS)

IRT Median Equating Function to Achieve (Approximate) Median Equity Robert Smith (ETS), Charles Lewis (Fordham University)

Equating scores

IRT

Consortium of State and Regional Educational Research Associations Paper Session 3

AERA — Invited Session Monday, April 11, 8:15 am-10:15 am Sheraton / Napoleon Ballroom C2

Northeastern Educational Research Association - Measuring Goal-Setting Attitudes and Predicting Academic Outcomes Using the Theory of Planned Behavior Presenters/Authors: Steven L. Holtzman (ETS), Teresa Jackson (ETS), Jeremy Burrus (ETS), Richard Dean Roberts (ETS)


English Language Learners

NCME — Paper Session Monday, April 11, 10:35 am-12:05 pm Westin / Azalea Ballroom II, K2

Exploring a Linguistically Simple Designated Anchor for Estimating IRT DIF for English Language Learners in Mathematics Assessments Presenter/Author: Maria Martiniello (ETS)





Understanding Teaching Quality in Algebra Classrooms

NCME — Invited Symposium Monday, April 11, 10:35 am-12:05 pm Westin / 12th Floor Ballroom II, K6

How Do We Study Teaching Quality in Algebra Classrooms? An Overview Presenter/Author: Drew Gitomer (ETS)

What are We Learning About Teaching Quality? Presenter/Author: Courtney Bell (ETS)


Meet Journal Editors: Journal Talks 11

AERA — Invited Roundtable Monday, April 11, 12:25 pm-1:55 pm Sheraton / Grand Ballroom A

Journal of Educational and Behavioral Statistics Presenters/Authors: Sandip Sinharay (ETS), Matthew Scott Johnson (Teachers College, Columbia University)

Communicating about assessment

Next Generation Performance Assessments and the Assessment of College Readiness

NCME — Coordinated Session Monday, April 11, 12:25 pm-1:55 pm Westin / Azalea Ballroom I, L1

The Role of Technology in Performance Assessment Presenters/Authors: Randy Bennett, ETS


K-12 assessment

Technology in assessment

Issues with Multistage Testing

NCME — Paper Session Monday, April 11, 12:25 pm-1:55 pm Westin / Azalea Ballroom II, L2

Multi-Stage Testing in Educational Survey Assessments Presenters/Authors: Xueli Xu (ETS), Andreas Oranje (ETS), Emmanuel Sikali (National Center for Education Statistics), Ed Kulick (ETS)

Designing an Adaptive Test of Reading Comprehension for Students with Learning Disabilities: Exploring the Quality of a Routing Test Presenters/Authors: Elizabeth Stone (ETS), Linda Cook (ETS)



Multistage testing





Language Demands of Content Assessments and Validity of Accommodations for English Language Learners

AERA — Symposium Monday, April 11, 2:15 pm-3:45 pm New Orleans Marriott / Mardi Gras Salon B

English Language Learners' Test-Taking Processes Using a Glossary Accommodation Presenters/Authors: Mikyung Kim Wolf (ETS), Jenny C. Kao (Teachers College, Columbia University) Language Demands of Content Assessments and Accommodations for Language Minority Students Presenter/Author: Maria Martiniello (ETS)


Rasch Scaling in an Accountability Context: Consideration for the Accuracy and Consistency of Test Results

NCME — Coordinated Session Monday, April 11, 2:15 pm-3:45 pm Westin / 12th Floor Ballroom I, M5

The Impact of Anchor Difficulty Distribution on Maintaining Scale Stability over Time Presenters/Authors: Lora Monfils (ETS), Weiling Deng (ETS)

Different Methods of Adjusting for Form Difficulty Under the Rasch Model: Impact on Consistency of Assessment Results Presenters/Authors: Venessa Lall (ETS), Lixiong Gu (ETS)

The Impact of Different WINSTEPS Control Options on Raw-to-Scale-Score Conversion Tables Presenters/Authors: Zhiming Yang (ETS), Shameem Gaj (ETS)

Rasch models

Equating scores

Use of Timing Data

NCME — Paper Session Monday, April 11, 12:25 pm-1:55 pm Westin / Magnolia Ballroom III, L4

Differential Speededness and its Impact on the Observed Score Equating of a Reading Test Presenters/Authors: Chi-wen Liao (ETS), Yanxuan Qu (ETS), Rick Morgan (ETS)

Test speededness

Reading assessment

Equating scores




What’s New from the Journal of Educational Measurement: A Sample of Recently Accepted Papers NCME — Invited Symposium Monday, April 11, 2:15 pm-3:45 pm Westin / Azalea Ballroom II, M2

Equating of Augmented Subscores Presenters/Authors: Sandip Sinharay (ETS), Shelby J. Haberman (ETS)

Equating subscores

Subscores

Postsecondary Experiences for Ethnic Minorities: Examining Race, Class, and Gender AERA — Paper Session Monday, April 11, 12:25 pm-1:55pm New Orleans Marriott / La Galerie 4

Untangling Diversity Experiences: The Consequences of Segregation on Collegiate Interracial Interactions Presenter/Author: Donnell Butler (ETS)

Higher education, diversity in

New Developments in Classroom Measurement: Conceptual, Methodological, and Policy Implications AERA — Symposium Monday, April 11, 12:25 pm-1:55 pm Sheraton / Edgewood

Inside Teaching Effects: The Scalable Measurement of Teaching and Learning in Secondary Classrooms Presenters/Authors: Drew H. Gitomer (ETS), Courtney A. Bell (ETS)


Innovations in the Automated Scoring of Spoken Responses NCME — Coordinated Session Monday, April 11, 2:15 pm-3:45 pm Westin / 12th Floor Ballroom II, M6

Adapting the SpeechRater Capability to the Development of Automated Scoring for Heterogeneous Speaking Tasks Presenter/Author: Xiaoming Xi (ETS)

The Impact of Automated Speech Recognition System Accuracy on the Performance of an Automated Scoring Engine for Spoken Responses Presenter/Author: Derrick Higgins (ETS)


Speech recognition




Standard Setting

NCME — Paper Session Monday, April 11, 2:15 pm-3:45 pm Westin / Magnolia Ballroom III, M4

The Impact of Scoring Method on Performance Data Using Cutscores from Bookmark Standard Setting Presenters/Authors: Ying Lu (ETS), Lixiong Gu (ETS)

Standard setting

Vertical Equating and Academic Growth

NCME — Paper Session Monday, April 11, 4:05 pm-6:05 pm Westin / Magnolia Ballroom I, N3

Long-Term Effects of Valid Case Inclusion Criteria on Capturing Growth Under IRT Equating Presenters/Authors: Weiling Deng (ETS), Lora Monfils (ETS)

Growth models

IRT

Equating, vertical

Teachers’ Use of Classroom Formative Assessments

AERA — Paper Session Monday, April 11, 4:05 pm-5:35 pm Sheraton / Rhythms Ballroom III

The Impact of Formative Assessment-Focused Professional Development on Student Learning Presenters/Authors: E. Caroline Wylie (ETS), Christine Jennifer Lyon (ETS), Jonathan Steinberg (ETS)

Formative assessment

K-12 assessment

Technology, Instruction, Cognition, and Learning 5: Developing, Managing, and Assessing Learning Environments

AERA — Paper Session Monday, April 11, 4:05 pm-5:35 pm Sheraton / Napoleon Ballroom C2

Evidence-Centered Design (ECD) for Designing Accessible Technology-Literacy Assessments Presenters/Authors: Eric G. Hansen (ETS), Cara Cahalan Laitusis (ETS), Diego Zapata-Rivera (ETS)

Evidence-centered design




Tuesday, April 12, 2011


Alternative Approaches to Assessment and Measurement of Social Emotional Characteristics

AERA — Symposium Tuesday, April 12, 8:15 am-9:45 am Sheraton / Napoleon Ballroom B2

Alternative Assessments in Program for International Student Assessment 2012 Presenters/Authors: Bobby Darius Naemi (ETS), Patrick Kyllonen (ETS)

A Day Reconstruction Method for Assessing Students Presenters/Authors: Jeremy Burrus (ETS), Anthony Bettencourt (ETS), Steven L. Holtzman (ETS), Teresa Jackson (ETS), Richard Dean Roberts (ETS)



Reading and Writing Together: A Cognitively Based Approach to Literacy Assessment

AERA — Symposium Tuesday, April 12, 8:15 am-10:15 am New Orleans Marriott / Preservation Hall Studio 8

Implications of an Integrated Approach to Reading and Writing Assessment Presenters/Authors: Tenaha P. O'Reilly (ETS), John P. Sabatini (ETS)

Developing an Assessment System to Support Writing Instruction Presenter/Author: Mary E. Fowles (ETS)

Elaborating a Cognitive Model of Literacy to Support Writing and Reading Instruction and Assessment Presenters/Authors: Paul Deane (ETS), John P. Sabatini (ETS), Tenaha P. O’Reilly (ETS)

Designing Score Reports That Help Teachers Make Instructional Decisions Presenter/Author: Diego Zapata-Rivera (ETS)


Literacy assessment

Reading assessment

Score reporting





Test Validity for English Language Learners and Special Populations

AERA Paper Session Tuesday, April 12, 10:35 am-12:05 pm Doubletree / Shadows

Score Comparability for Language Minority Students on the Content Assessments Used by Two States Presenters/Authors: John W. Young (ETS), Steven L. Holtzman (ETS), Jonathan Steinberg (ETS)

Does Linguistic Complexity of Mathematics Assessments Matter for Former English Language Learners? Presenter/Author: Maria Martiniello (ETS)

Examining the Validity of Linguistically Modified Items for English Language Learners Presenters/Authors: Teresa C. King (ETS), John W. Young (ETS)


K-12 assessment

Issues in Equating and Linking

AERA — Roundtable Session Tuesday, April 12, 10:35 am-12:05 pm Doubletree / International Ballroom

A Generalized Linear Equating and a Generalized Poststratification Equating, Both Based on Partial Error Anchor Scores and Their Relationship Presenter/Author: Haiwen Henry Chen (ETS)

Equating scores

Technical Issues in Assessment Development and Utilization

AERA — Roundtable Session Tuesday, April 12, 10:35 am-12:05 pm Doubletree / International Ballroom

English Language Learners' Differential Performance on Inquiry-Based Science Assessment Presenters/Authors: Sultan Turkan (ETS), Ou Lydia Liu (ETS)

Assessment development


DIF





Reconceptualizing Our Instructional Methods, Assessments, and Accountability Systems to Ensure Success for English Language Learners

AERA — Invited Session Tuesday, April 12, 12:25 pm-1:55 pm Sheraton / Napoleon Ballroom B3

Validity and Fairness of Content Assessments for English Language Learners Presenter/Author: Maria Martiniello (ETS)


Rater Cognition and Its Importance for Score Validity: Global Perspectives and Findings

AERA — Symposium Tuesday, April 12, 2:15 pm-3:45 pm Doubletree / Madewood A

A Review of the Literature Concerning Rater Cognition Presenter/Author: Isaac I. Bejar (ETS)


Documents

Making Connections: ETS-led Presentations at AERA and NCME 2011