Embed Size (px)
Citation preview
Assessment & Curriculum Design
Dr. Charles Browne, Professor of TESOL & Applied Linguistics
Meiji Gakuin University, Deparment of English
“
”
Understanding the fundamental differences between CRT and NRT tests is crucial to both language program administrators and language teachers who want to make better decisions about their students
JD Brown, Testing in Language Programs (2005)
Strong connection between Assessment and Curriculum Development
Two main families of tests…
4 Test Types for 4 Different Purposes…
Proficiency Test Sample: 1 Size Doesn’t Fit All
Language Testing Trends Over the Years…
What is Communicative Competence?
Communicative Competence Model
1) Grammatical Competence
knowledge of lexical items and of rules of morphology, syntax, sentence-grammar semantics, and phonology
2) Sociolinguistic Competence
knowledge of the sociocultural rules of language and of discourse
3) Discourse Competence
the ability to connect sentences in stretches of discourse and to form a meaningful whole out of a series of utterances
4) Strategic Competence
the verbal and nonverbal communication strategies that may be called into action to compensate for breakdowns in communication
(Canale & Swain, 1980)
Testing Communicative Competence
Characteristics of Communicative Tests
Test-setting Requirements:
• Meaningful communication
• Authentic situations
• Unpredictable language input
• Creative language output
• All skills tested (4 skills)
Basis for Ratings:
• Success in getting meanings across
• Focus on use rather than usage
A Testing & Curriculum Development Framework
The NGSL Project: Assessment & Curriculum Design in Action
Some Background:
• Corpus-based approach to developing vocabulary lists to meet the needs of 2nd language learners of English
• First 2 lists, NGSL and NAWL, published in 2013
• Updates of West’s 1953 GSL and Coxhead’s 2000 AWL, in each case offering substantially higher coverage than the original
• TOEIC word list (TSL) and Business Word List (BSL) published 2015
• All lists are open-sourced, all online learning, testing, researching and graded text creation tools are free
THE NGSL PROJECT IS ALL ABOUT SHORT CUTS: MAKING TIGHT LISTS OF WORDS TO HELP STUDENTS REACH THE LEVEL OF INDEPENDENT LEARNING
The New General Service List (NGSL)
• 600,000 words in English
• 30,000 known by adult native speakers
• 90% is minimum to read/listen and be able to guess from context
• 2800 words in NGSL covers 92% of words students are likely to ever meet in average newspaper, book, magazine, TV show, Movie or daily speech…
Browne, Culligan & Phillips, 2013
A Balanced NGSL Corpus …. 273 million words from the CEC 2 billion word corpus
CEC corpora included in final analysis:
Corpus Tokens
Learner 38,219,480
Fiction 37,792,168
Journals 37,478,577
Magazines 37,329,846
Non-Fiction 35,443,408
Radio 28,882,717
Spoken 27,934,806
Documents 19,017,236
TV 11,515,296
Total 273,613,534*
*273 million word subsection used is 100x larger than original GSL corpus…
2800
Core General English Words
– Written/Spoken
(NGSL)
960
Academic
Words
(NAWL)
87-94%
99%
1200
TOEIC
Words
(TSL)
1700
Business
Words
(BSL)
After Learning the NGSL, What’s Next?
Special Purpose Vocabulary:
Creating the fastest road(s) to comprehension
Browne, Culligan & Phillips, 2013, Browne & Culligan, 2015
1700
600,000
2,800 Core Daily Vocabulary
Native Speaker
Vocabulary Size
Word Families in
English Language
30,000
Special-Purpose Vocabulary
700
Core Spoken
(NGSL-S)
NAWL: NEW ACADEMIC WORD LIST (Browne, Culligan & Phillips, 2013)
2800
Core General English Words
(NGSL)
86%
92%
960
Academic
Words
(NAWL)
283 Million Word
Academic Corpus
1700
600,000
2,800 Core Daily Vocabulary
Native Speaker
Vocabulary Size
Word Families in
English
Language
30,000
Special-Purpose Vocabulary
TSL: TOEIC SERVICE LIST (Browne & Culligan, 2015)
2800
Core General English Words
(NGSL)
94%
99%
1200
TOEIC
Words
(TSL)
1.5 Million Word TOEIC Corpus
1700
600,000
2,800 Core Daily Vocabulary
Native Speaker
Vocabulary Size
Word Families in
English
Language
30,000
Special-Purpose Vocabulary
BSL: BUSINESS SERVICE LIST (Browne & Culligan, 2015)
1700
600,000
2,800 Core Daily Vocabulary
Native Speaker
Vocabulary Size
Word Families in
English
Language
30,000
Special-Purpose Vocabulary
2800
Core General English Words
(NGSL)
91%
97%
1700
Business
Words
(BSL)
64 Million Word
General Business Corpus
www.newgeneralservicelist.org
www.newgeneralservicelist.org
FREE NGSL RESOURCES INCLUDE:
• Wordlists (NGSL, NAWL & others)
• Definitions (English & Japanese)
• Published research articles on NGSL
• Online Learning applications
• Flashcard Learning Apps with SRS
• Online Assessment & Placement Tools
• Online Research Tools
• Online Text Analysis & Grading Tools
• Blog of NGSL news
NGSL, NAWL, BSL & TSL now on free Quizlet Program…
www.quizlet.com
NGSL, NAWL, BSL & TSL now on free Memrise Program…
www.memrise.com
Free NGSL Flashcard app with spaced repetition learning (NGSL Builder)
Free NGSL/NAWL Flashcard apps with spaced repetition learning
•
•
•
•
•
NGSL, NAWL, BSL and TSL Tools for Text Analysis and Text Creation
NGSL, NAWL, BSL AND TSL on VocabProfile http://www.lextutor.ca/vp/bnc/
NGSL on VocabProfile http://www.lextutor.ca/vp/bnc/
Sample text from chapter
1 of Harry Potter…
NGSL on AntWordProfiler & OGTE Tools…
NGSLT (New General Service List Test) NAWLT (New Academic Word List Test)
Criterion referenced test (CRT)
Main purpose is as diagnostic
Secondary use is as achievement
Paper and online versions
Multiple equivalent forms
English and Japanese
Extremely high reliability/validity
What’s NEXT? APPLICATION FOR 2018 KAKKEN GRANT…
• Expand and refine all corpora
• Update all word lists (higher coverage with less words)
• Create new SP wordlists for STEM and the humanities
• Update NGSLT and NAWLT to computer adaptive versions (also other word Lists)
• Expand number of online resources for learning, teaching and researching lists
• Improve wordlist websites and social media outreach
• Hold several conferences to promote and share NGSL-related research
Assessment & Curriculum Design
Dr. Charles Browne, Professor of TESOL & Applied Linguistics
Meiji Gakuin University, Deparment of English
Thank You!
