Exploring the tiers of Japanese vocabulary: Academic, literary and beyond
Tatsuhiko MatsushitaLALS, Victoria University of Wellington
Main findings• VDRJ is useful for designing curriculum (material, tests etc.)• The more domains a words is shared as AW or LAD by, the more abstract
the meaning of the word is. • Conversation and non-academic texts contain more general words and
LW• Academic texts: more AW and LAD but less LW in any academic domain• Wikipedia: more proper nouns and low frequency words• Newspapers and academic items of Wikipedia can be a good resource
for learning AW and LAD.• Natural science texts contain more academic domain words at lower
frequency levels than arts and social science texts • Origins of academic and literary words are considerably clearly
separated; 3/4 of LW originate in Japanese while 3/4 of AW and LAD originate in Chinese
• LAD contains more Western origin words (Gairaigo)
Contents1. Motive for this research2. Goals of this presentation
3. Vocabulary Database for Reading Japanese4. Tiers of Japanese vocabulary
(Basic words, academic words, limited-academic domain words, literary words)
5. Text coverage by word tier6. Proportions of word origin types by word tiers7. Number of characters required to cover the word tiers
7. Implications from the findings8. Conclusion
1. Motive for this researchHow efficiently can we learn vocabulary?• Learning burden is big!• More effective choice of target words• More efficient order for learning the words Effective choice and efficient order: to maximize the
coverage of text which the learner would encounter in his/her domain
= Reading comprehension and lexical density (Hu & Nation, 2000; Komori et al., 2004)
Q. What words should learners learn first? And second and next?
Studies on EAP vocabulary
• Basic: General Service List (West, 1953)• Academic: AWL (Coxhead, 2000) UWL (Xue & Nation, 1984)• EGAP-A/S, EGAP-HM/SS etc.
(Tajino, Dalsky, & Sasao, 2009)• Science-specific Word List (Coxhead & Hirsh, 2007)• Technical: e.g. Chung (2003)• Literary vocabulary?
Studies on JAP vocabulary• Basic: The former JLPT list, Tamamura (1987) etc. • Academic: Butler (2010), Matsushita (2011)• ?• Technical: Komiya (1995), Oka (1992) etc.• Others
• No list for words between academic and technical words• Literary vocabulary?
2. Goals of this presentationTo introduce I. the Vocabulary Database for Reading JapaneseII. extracted domain-specific words such as Academic
Words (AW), Limited-Academic-Domain Words (LAD), Literary Words (LW)
To argue about IV. how the word tiers work in different types of text
(register variation)V. how learner’s language background possibly
affects the understanding of texts in different genres
3. Vocabulary Database for Reading Japanese• Vocabulary Database for Reading Japanese ( VDRJ )( Matsushita, 2010; 2011 )• Created from the Balanced Contemporary Corpus of Written
Japanese, 2009 monitor version (NINJAL, 2009)• 33 million token (28 million from books and 5 million from the
Internet forum sites (Yahoo Chiebukuro))• 19 million content words and 14 million function words• Unit of counting: Lexeme – considerably inclusive but less
inclusive than the word family (Level 6 in Bauer & Nation, 1993) in English
• “Short unit of lexemes” are ranked by U (usage coefficient) (Juilland & Chang-Rodrigues, 1964)
• Short unit of lexeme: more inclusive than “lemma”, less inclusive than “word family”
Some problems of existing Japanese word frequency lists
• Lack of representativeness• Too old• The corpus size is not large enough: low reliability for low frequency words• No good sub frequency data which enable us to
calculate dispersion to downgrade unevenly distributed words
Advantages of word lists* Various types of word lists can be created from the vocabulary database (VDRJ)
A) Reference for developing vocabulary tests = Checking learners’ vocabulary levelsB) Reference for checking vocabulary level of material
= Checking vocabulary levels of materials
C) Specify vocabulary for learners to learn and for teachers to teach
For better choice of material, modification of textCf. Nation (2011), Word profiler
How to make VDRJA) MethodI. Classify all the texts into some sub corpora to see
the range and dispersion cf. Nippon Decimal Classification, BCCWJ (NINJAL, 2009 )
II. Parse (made word segmentation of ) all the texts by a morphological analyzer with a dictionary (if the text is not segmented by space between words.) cf. MeCab, UniDic
III. Make word lists by AntConc and/or AntWordProfiler
Content and construct of VDRJ• Vocabulary Database for Reading Japanese• The list is for reading as it is made from written
corpus of books and internet forum sites• Written and spoken languages are different in word
frequency, domain and required language processing skills ⇒ A good corpus of spoken language is necessary to
develop a good word list for it(, but there is no very good corpus of spoken Japanese…)
The Classification of Domains and Fields (Corpus form books and internet forum sites, BCCWJ 2009 monitor version)
The tendomains
Code forthe ten
domains
The 28academicfield code
Notes
Literary works LW a6_G All classified as general texts of a6
Humanities and Arts
Languages and Linguistics a1
Philosophy and Religion a2
History a3
Ethnology a4
Fine Arts a5
Literature (non-imaginative texts e.g. critique) a6_T All classified as technical texts of a6
Other Humanities and Arts a7
Social Sciences
Politics s1
Law s2
Economics s3
Commerce and Business s4
Sociology and Social Issues s5 Including welfare, labour, gender issues
Education s6 Including pedagogy on each subject
Other Social Matters s7 Including transportation, media, current issues
Technological Natural Sciences
Mathematics t1
Physics t2
Astronomy, Earth and Planetary Science t3
Chemistry, Metal and Mine t4
Technology (Architecture, Civil Engineering) t5
Technology (Mechanics, Electricity, Marine Engineering) t6
Other Technological Natural Sciences t7Including information science, manufacturing,library science, part of domestic science
Biological Natural Sciences
Biology b1
Agriculture b2Including forestry, fishery, animal husbandry,veterinary
Pharmacy b3
Medicine b4
Dentistry b5
Nursing b6
Other Biological Natural Sciences b7Including sports, hygienics, environmentology,part of domestic science
Internet Q & A Forum (Yahoo Chiebukuro) IF
Domain/Field
Languages,Linguistics and
PhilosophyLP
History andEthnology
HE
Literary Works/Imaginative Texts
STScience andTechnology
Biology andMedicine
BM
AH
Politics andLaw
Economics andCommerce
Sociology,Education andOther Social
Issues
PL
EC
SE
Arts and OtherHumanities
Content of the sub corpora
DomainNumber of
TokensRatio
Literary Works/Imaginative Texts 8251999 25.1%
Languages, Linguistics and Philosophy 2134739 6.5%
History and Ethnology 3336818 10.2%
Arts and Other Humanities 3020917 9.2%
Politics and Law 1881012 5.7%
Economics and Commerce 2209107 6.7%
Sociology, Education and Other Social Issues 2996147 9.1%
Science and Technology 1512784 4.6%
Biology and Medicine 2251037 6.9%
Internet Q & A Forum 5224852 15.9%
Total 32819412 100.0%
Types and Tokens by the Ten Domain Classification(Corpus form books and internet forum sites, BCCWJ 2009monitor version)
Different word rankings
• The word ranking problem mainly exists in Basic Words
• This is mainly due to lack of good spoken corpora
• Compromise: frequency weighted to limited domains which seem to reflect basic daily needs
• For International Students• For General Learners• Non-weighted (ranking for overall written Japanese)
Multidimensional scaling (MDS)
10 domains10 domains
+ word familiarity
4. Tiers of Japanese vocabulary (1) The concept of “word tiers”• Domain / Level• Level = general importance = frequency × dispersion
Some words are frequent only in a particular domaine.g. 発送 (shipping) 振り込み (paying by bank transfer) 古墳 (tumulus / burial mound)
Assumed word tiers for studentsLevel• Basic: Top 1288 = Former JLPT Level 4 &3• Intermediate: Ranked 1289-5000• Advanced 1: 6K-10K• Advanced 2: 11K-15K • Super-Advanced: 15K-20K• 21K+• Assumed Known Words (AKW)
Domain*General / Academic / Literary
4. Tiers of Japanese vocabulary (2) Basic words (BW)• Feature of the corpus: formal written language similar to BNC (Nation, 2004)• No good spoken corpus for vocabulary studies• Compromise
• For learners and teachers lists, the former JLPT Level 4 $ 3 vocabulary is put at the top of the list as basic words
To order the basic words• Identify closer domains to word familiarity (basic needs) by
Multidimensional Scaling (MDS)• Frequency in literary works and the Internet-forum sites
(Yahoo-Chiebukuro) is weighted
4. Tiers of Japanese vocabulary (3) Academic domain words
Extracting academic domain words• Log-likelihood ratio (LLR)(Dunning, 1993)• Target texts: Technical texts
• Classified into four large academic domains• Total number of tokens: approx. 2.9 million
• Reference texts: General texts in BCCWJ 2009• Total number of tokens: approx. 29.9 million
• Extract keywords shared by 4 - 1 domains• Cut off point: higher for more narrowly distributed words
Number of Shared Academic Domains among the 4 academic domains
Ah Ss1 1
22 2
3 3
2 4 2
3 3
Tn Bn
1 2 1
Ah: Arts & Humanities, Ss: Social Sciences,
Tn: Technological Natural Sciences, Bn: Biological Natural Sciences
4. (3) Academic domain words• Academic words (AW): high specificity in 3+ academic domains
• 4-domain words (cut off point: LLR > 0)• 3-domain words (cut off point: LLR > 0)
• Limited-academic-domain words (LAD)• 2-domain words (cut off point: LLR > 1)• 1-domain words (cut off point: LLR > average value)
• Eliminate the former JLPT Level 4 vocabulary (Top 700 words)• Eliminate the words ranked at 20001 or lower• Classify all the AW and LAD by word ranking levels for
International Students (U=Usage Coefficient): • 5 levels: Basic / Inter. / Adv. 1 / Adv. 2 / Super-adv.
The Classification of Domains and Fields (Corpus form books and internet forum sites, BCCWJ 2009 monitor version)
The tendomains
Code forthe ten
domains
The 28academicfield code
Notes
Literary works LW a6_G All classified as general texts of a6
Humanities and Arts
Languages and Linguistics a1
Philosophy and Religion a2
History a3
Ethnology a4
Fine Arts a5
Literature (non-imaginative texts e.g. critique) a6_T All classified as technical texts of a6
Other Humanities and Arts a7
Social Sciences
Politics s1
Law s2
Economics s3
Commerce and Business s4
Sociology and Social Issues s5 Including welfare, labour, gender issues
Education s6 Including pedagogy on each subject
Other Social Matters s7 Including transportation, media, current issues
Technological Natural Sciences
Mathematics t1
Physics t2
Astronomy, Earth and Planetary Science t3
Chemistry, Metal and Mine t4
Technology (Architecture, Civil Engineering) t5
Technology (Mechanics, Electricity, Marine Engineering) t6
Other Technological Natural Sciences t7Including information science, manufacturing,library science, part of domestic science
Biological Natural Sciences
Biology b1
Agriculture b2Including forestry, fishery, animal husbandry,veterinary
Pharmacy b3
Medicine b4
Dentistry b5
Nursing b6
Other Biological Natural Sciences b7Including sports, hygienics, environmentology,part of domestic science
Internet Q & A Forum (Yahoo Chiebukuro) IF
Domain/Field
Languages,Linguistics and
PhilosophyLP
History andEthnology
HE
Literary Works/Imaginative Texts
STScience andTechnology
Biology andMedicine
BM
AH
Politics andLaw
Economics andCommerce
Sociology,Education andOther Social
Issues
PL
EC
SE
Arts and OtherHumanities
4. Tiers of Japanese vocabulary (3) -1 Academic words (AW)
• JAWL = Japanese Academic Word List• High specificity in 3 or 4 academic domains• 4-domain words (cut off point: LLR > 0)• 3-domain words (cut off point: LLR > 0)
• Level 0 - VIII 9 levels , 2590 words in total • JAWL I (Intermediate): most essential for learning• Basic words contains much fewer academic words• JAWL I: 559 words
Close to AWL in number and text coverageCoverage in the academic corpus used for extracting AWAWL: 10.0 % JAWL I: 11.1 %
Academic Words: Words which are shared by 3 or 4 main academic domains
Ah Ss1 1
22 2
3 3
2 4 2
3 3
Tn Bn
1 2 1
Ah: Arts & Humanities, Ss: Social Sciences,Tn: Technological Natural Sciences, Bn: Biological Natural Sciences
Distribution and examples of JAWL
4 31 科学 規則 割合 生産 産業 講義
science, rule, proportion,ptoduction, industry, lecture
3 39 人口 スクリーン 数学 競争 工業 地理
population, screen, mathmatics,competition, manufacture, geography
JAWL I 4 559 発足 半数 配分 縮小 適正 見直し
inauguration, half the number, allocation,downsize, proper, reconsider
JAWL II 3 542 演説 大小 実情 ステージ ライフ 担保
speech, size, real situation,stage, life, guarantee
JAWL III 4 212 難問 能動 付随 定型 除 本稿
difficult problem, active, accompany,standard, except, this article
JAWL IV 3 452 交錯 カウント 精度 一因 箇年 エンド
mixture, count, accuracy,one cause, -year, end
JAWL V 4 103 併存 親和 盛況 散在 補填 関わり合う
coexistence, affinity, prosperity,straggle, compensation, implicated
JAWL VI 3 328 帰着 編著 沿海 拮抗 常套 内情
come down to, written and edited, coastal,close competition, conventional, internal condition
JAWL VII 4 56 閉 増刊 含意 複 活路 所与
closed, extra edition, implication,double-, way out, given
JAWL VIII 3 269 極小 付則 深度 概算 頒布 円錐
minimal, additional clause, depth,rough estimate, distribution (of goods/paper), cone
Least Frequent 6 Wordsin Each Domain
JAWL 0 L3 679-1288 Basic
Translation of the Least Frequent 6Words in Each Domain
JAWLLabel
FormerJLPTLevel
WordRankings forInternational
Students
Level
Number ofHigh
SpecificityDomains
among the 4ScienceDomains
Numberof UniqueLexemes
L2L1
Other
1289-5000 Inter.
5001-10000 Adv. 1
10001-15000 Adv. 2
15000-20000Super-
adv.
4. (3) -1 Academic words (AW)Semantic features of AW (1)
• Highly abstract, essential for operating logici.e.
• Range: 占める (occupy, account for), 特殊 (special, particular)• Relation: 属する (belong to), 依存 (rely/reliance)• Comparison/Evaluation: 後者 (the latter), 優れる (superior), • Quantitative change: 減少 (decrease), 強化 (reinforce)• Stage: 当初 (beginning), 現状 (present condition)• Development of enunciation: 取り上げる (take up [an issue]), まとめる (summarize)• Cause-effect, degree, agent, action, object, direction, goal,
instrument, time etc.
The most frequent Kanji used for AW合 (combine, together), 定 (fix, certain), 分 (divide, minute), 一 (one), 同 (same), 数 (number), 上 (up), 体 (body), 出 (out), 大 (large)
• 3-domain words: Some words have concrete meanings e.g. 署名 (signature), 保健 (health, hygiene)• 4-domain words: Few words have concrete meanings• The nature of the words are the same at all levels
4. Tiers of Japanese vocabulary (3) -1 Academic words (AW)Semantic features of AW (2)
POS of Japanese AW (1)• Common noun: 1072 words (41.4 %) e.g. 背景 (background)• Verbal noun: 882 words (34.0 %) e.g. 連続 (establish/-ment) Adding other types of nouns together,
2104 words (81.2 %) can be a noun• Verb (excluding verbal nouns): 225 words (8.7 %) e.g. 認める (recognize/approve) 述べる (describe/mention) Adding other types of verbs together,
1107 words (42.7%) can be a verb• Adjectival noun: 95 words (3.7 %) e.g. 詳細 (detail/-ed), 平等 (equal/-ity)• Adjective : Only 9 words (0.3 %) e.g. 著しい (remarkable)
POS of Japanese AW (2)• Affix: 106 words (4.1 %) e.g. - 期 (period), - 種 (type)
substantial in Japanese academic words• Adverb: 34 words (1.3 %) e.g. しばしば (frequently)
• Other (particle, auxiliary verb etc.): 22 words (0.8 %)• Remarkably many archaic words e.g. のみ (only), つつ (while doing), べし (ought to), あらゆる (every)
いかなる (any), 我が (my), 漠然 (vague)
• れる / られる (Passive/Potential/Spontaneous) specific in academic texts
4. (3) -2 Limited-academic-domain words (LAD)
• Limited-academic-domain words (LAD)• High specificity in 2 or 1 domain(s)• 2-domain words (cut off point: LLR > 1)• 1-domain words (cut off point: LLR > average value)
• Something between “academic” and “technical”• The “scams” from extracting AW?• Tiers of curriculum cf. Tajino et al. (2007)• Words correspondent to the curriculum
• Basic: all the learners• Academic words: prep. to first year• Limited-academic-domain words (?): prep. to major• Technical words: major to postgrad.
Number of Shared Academic Domains among the four academic domainsLimited-Academic-Domain Words:
Words which are shared by only 1 or 2 main academic domain(s)
Ah Ss1 1
22 2
3 3
2 4 2
3 3
1 2 1
Tn Bn
Ah: Arts & Humanities, Ss: Social Sciences,Tn: Technological Natural Sciences, Bn: Biological Natural Sciences
4. (3) -2 Limited-academic-domain words (LAD)
2 domain wordsDistribution of 2-Domain Words of Japanese Limited-Academic-Domain Words (JLAD)
JLAD 0 L3 679-1288 Basic 15 5 4 5 6 10 45
JLAD I 1289-5000 Inter. 139 27 30 77 57 61 391
JLAD III 5001-10000 Adv. 1 138 38 25 86 50 92 429
JLAD V 10001-15000 Adv. 2 91 28 22 58 37 60 296
JLAD VII 15000-20000Super-adv. 93 23 17 43 16 40 232
476 121 98 269 166 263 1393Ah: Arts & Humanities, Ss: Social Sciences, Tn: Technological Natural Sciences, Bn: Biological Natural Sciences
Total
Total
Numberof UniqueLexemes
in LAD ofTn & Bn
Numberof UniqueLexemes
in LAD ofSs & Tn
Numberof UniqueLexemes
in LAD ofSs & Bn
L2L1
Other
JLADLabel
FormerJLPTLevel
WordRankings forInternational
Students
Level
Numberof UniqueLexemesin LAD ofAh & Ss
Numberof UniqueLexemes
in LAD ofAh & Tn
Numberof UniqueLexemesin LAD ofAh & Bn
4. (3) -2 Limited-academic-domain words (LAD)
2 domain wordsExamples of 2-Domain Words of Japanese Limited-Academic-Domain Words (JLAD)
JLAD 0 L3 679-1288 Basic 貿易輸出
砂テキスト
発音ステレオ
製レポート
以内パート
アルコール
テニス
JLAD I 1289-5000 Inter. 孤立融資
オールペーパー
静岡書簡
ニーズ顧客
総務性的
スイッチ
液
JLAD III 5001-10000 Adv. 1 容れる教義
音響流布
発現海域
本件セクション
閉塞弱める
多用部位
JLAD V 10001-15000 Adv. 2 払い戻しユ ニ バ ー シ テ ィ
落差コロン
目付け生長
VTRリハーサル
所見救命
光学ペーハー
JLAD VII 15000-20000Super-adv.
峻別公債
目配りテクノ
太極増量
パレット軽微
マンガン居宅
棒状雨水
LeastFrequent2 Words
in LAD ofTn & Bn
LeastFrequent2 Words
in LAD ofSs & Tn
L2L1
Other
LeastFrequent2 Words
in LAD ofSs & Bn
LeastFrequent2 Words
in LAD ofAh & Ss
LeastFrequent2 Words
in LAD ofAh & Tn
LeastFrequent2 Words
in LAD ofAh & Bn
FormerJLPTLevel
WordRankings forInternational
Students
LevelJLADLabel
4. (3) -2 Limited-academic-domain words (LAD)
2 domain wordsExamples of 2-Domain Words of Japanese Limited-Academic-Domain Words (Translation)
JLAD 0 L3 679-1288 Basictradeexport
sandtext
pronunciation
stereomade (in)
reportwithin
part(-timer)
alcoholtennis
JLAD I 1289-5000 Inter.isolation
loanall
paperShizuoka pref./city
epistleneed (n.)customer
general affairs
sexualswitchliquid
JLAD III 5001-10000 Adv. 1compatible
doctrineaccousticcirculation
manifestation
watersthis matter
sectionimpasseweaken
frequent useregion (of body)
JLAD V 10001-15000 Adv. 2refund
universitya dropcologne
overseergrowth
VTRrehearsal
remark (n.)lifesaving
opticspH
JLAD VII 15000-20000Super-adv.
sharp distinction
public bond
meticulous care
techno-tai ji
increase in quantity
palletslight
manganesedwelling
stick-shaped
rainwater
Translation ofthe Least
Frequent 2Words in LAD
of Ss & Bn
Translation ofthe Least
Frequent 2Words in LAD
of Tn & Bn
L2L1
Other
WordRankings forInternational
Students
Translation ofthe Least
Frequent 2Words in LAD
of Ah & Ss
JLADLabel
Former JLPTLevel
Level
Translation ofthe Least
Frequent 2Words in LAD
of Ah & Tn
Translation ofthe Least
Frequent 2Words in LAD
of Ah & Bn
Translation ofthe Least
Frequent 2Words in LAD
of Ss & Tn
Examples of 2 domain words: Words which are shared by only 2 main academic domains
Ah Ss
liquidfrequent use
Tn pH Bn
Ah: Arts & Humanities, Ss: Social Sciences,Tn: Technological Natural Sciences, Bn: Biological Natural Sciences
sexualweaken
lifesaving
isolationdoctrinerefund
paperaccoustic a drop
epistlewatersgrowth
need (n.)sectionVTR
4. (3) -2 Limited-academic-domain words (LAD)
2 domain words• Semantic features
• More concrete and specific than academic words• Ah & Ss: Social, overlap in history and ethnology• Ss & Tn: Industrial• Ss & Bn: Social security, medical and nursing service• Tn & Bn: Scientific
• Ah & Tn, Ah & Bn: not clear
4. (3) -2 Limited-academic-domain words (LAD)
1 domain words• It is merely a trial• The corpus is not the best for academic purpose, especially for
natural sciences• Extracting something common across domains is much easier
while extracting words by only one target corpus will require more complete target corpus
• Therefore, AW (4 domain words and 3 domain words) will be more reliable than LAD (2 domain words and 1 domain words)
4. (3) -2 Limited-academic-domain words (LAD)
1 domain wordsDistribution of 1 Domain Words of Japanese Limited-Academic-Domain Words (JLAD)
JLAD 0 L3 679-1288 Basic 13 6 5 9 33
JLAD I 1289-5000 Inter. 104 111 46 52 313
JLAD III 5001-10000 Adv. 1 104 127 60 68 359
JLAD V 10001-15000 Adv. 2 71 74 48 54 247
JLAD VII 15000-20000Super-adv. 60 55 29 53 197
352 373 188 236 1149Ah: Arts & Humanities, Ss: Social Sciences, Tn: Technological Natural Sciences, Bn: Biological Natural Sciences
L2L1
Other
Total
Numberof UniqueLexemes
in Ss
Numberof UniqueLexemes
in Tn
Numberof UniqueLexemes
in Bn
TotalJLADLabel
FormerJLPTLevel
WordRankings forInternational
Students
Level
Numberof UniqueLexemes
in Ah
4. (3) -2 Limited-academic-domain words (LAD)
1 domain words
• Semantic features are much clearer than 2 domain words
Examples of 1 Domain Words of Japanese Limited-Academic-Domain Words (JLAD)
JLAD 0 L3 679-1288 Basic 辞典文法
工場遊び
海岸汽車
退院柔道
JLAD I 1289-5000 Inter. 色彩滋賀
紛争犯
原子コンクリート
拳杉
JLAD III 5001-10000 Adv. 1 王家呪術
超過欠席
硬化ドラッグ
臓器左足
JLAD V 10001-15000 Adv. 2 報国遍歴
持ち分受諾
PM蒸留
卵子緑茶
JLAD VII 15000-20000Super-adv.
厳寒鼎
卸売り引き当て
プログラミング
バラック居合微小
L2L1
Other
LeastFrequent 2
Words in LADof Bn
LeastFrequent 2
Words in LADof Tn
FormerJLPTLevel
Word Rankingsfor
InternationalStudents
Level
LeastFrequent 2
Words in LADof Ah
LeastFrequent 2
Words in LADof Ss
JLADLabel
4. (3) -2 Limited-academic-domain words (LAD)1 domain words
• Semantic features are much clearer than 2 domain words
Examples of 1 Domain Words of Japanese Limited-Academic-Domain Words (Translation)
JLAD 0 L3 679-1288 Basicdictionarygrammar
factoryplay(ing)
seashoretrain
leave hospitaljudo
JLAD I 1289-5000 Inter.coloring
Shiga (pref.)conflictoffense
atomconcrete (n.)
fist/martial artcedar
JLAD III 5001-10000 Adv. 1royal familyincantation
excessabsence
harden(ing)drag/drug
organleft leg/foot
JLAD V 10001-15000 Adv. 2patrioticitinerancy
quotaacceptance
PMdistillation
ovumgreen tea
JLAD VII 15000-20000Super-adv.
intense coldthree-legged vessel
wholesaleresearve fund
programmingshanty
iai (martial arts)micro
Translation of theLeast Frequent 2
Words in LAD of Bn
L2L1
Other
LevelTranslation of theLeast Frequent 2
Words in LAD of Ah
Translation of theLeast Frequent 2
Words in LAD of Ss
Translation of theLeast Frequent 2
Words in LAD of Tn
JLADLabel
Former JLPTLevel
WordRankings forInternational
Students
Examples of Academic Domain Words:Words which are shared by only 2 main academic domains Words which are shared by 1, 2, 3 or 4 main academic domain(s)
coloring Ssroyal family conflict
Ah excessisolation
epistle doctrine need (n.)waters refund sectiongrowth VTR
guarantee- year
paper sexualaccoustic weaken
a drop lifesavinglead endsize life
Tn liquid Bnfrequent use
atom pH fist/ martial artharden(ing) organ
Ah: Arts & Humanities, Ss: Social Sciences,Tn: Technological Natural Sciences, Bn: Biological Natural Sciences
at a strokemixture
allocation
properexcept
POS of Japanese LAD (1)• Common noun: 1605 words (63.1 %) – more than AW (41.4%)• Verbal noun: 633 words (24.9 %) e.g. 融資 (finance) cf. AW
(34.0%) Adding other types of nouns together,
2104 words (87.9 %) can be a noun – more than AW (81.2%)• Verb (excl. verbal nouns): 81 words (3.2 %) cf. AW (8.7%) e.g. 訳す (translate) 向き合う (face (v.)) Adding other types of verbs together,
714 words (28.1%) can be a verb – less than AW (42.7%)• Adjectival noun: 88 words (3.5 %) cf. AW (3.7%) e.g. フル (full), 偉大 (great)• Adjective : Only 3 words (0.1 %) cf. AW (0.3%) e.g. 硬い (stiff)
POS of Japanese LAD (2)• Affix: 109 words (4.3%) cf. AW (4.1%) e.g. –犯 (offense)
substantial in Japanese academic domain words• Adverb: 15 words (0.6 %) cf. AW (1.3%) e.g. 現に (surely)
• Other (particle, auxiliary verb etc.): 9 words (0.8 %)cf. AW (0.8%)
• Remarkably many archaic words – similar to AW e.g. なり [affirmative aux.], とも (even though), たり [affirmative aux.], ごとし (as/like), 単なる
(mere), しめる(=しむ) [causative aux.], かかる (such)
4. Tiers of Japanese vocabulary (4) Literary words (LW)Extracting literary words: Words for reading literary works• Log-likelihood ratio (Keyness in AntConc)• Target corpus: literary works (identified by NDC and C-code) in
BCCWJ 2009 (NINJAL, 2009) – Over 8 million tokens• 4 different reference corpus: Technical texts, general texts in arts
and humanities, general texts in the other 3 academic domains, Internet forum texts (Yahoo Chiebukuro)
• Extract keywords shared by the four results (Cutoff point: average value)• Eliminate the former JLPT Level 4 vocabulary (Top 700 words)• Eliminate the words ranked at 20001 or lower• Classify all the LW by word ranking levels for International Students
(U=Usage Coefficient)
4. (4) Literary words (LW) Distribution and examples
Distribution and examples of Japanese Literay Words (JLW)
L3 679-1288 Basic Lit. 142 ちっとも引き出し
(not) at alldrawer
1289-5000 Inter. Lit 446 戸惑う吐き出す
puzzledvent
5001-10000 Adv. 1 Lit. 483 不吉銀色
ominoussilver
10001-15000 Adv. 2 Lit. 345 敵機口笛
hostile aircraftwhistle
15000-20000Super-adv.
Lit. 200 香菜樹海
coriandersea of trees
Total 1616
L2L1
Other
Least Frequent 2Words of JLW
FormerJLPTLevel
WordRankings forInternational
Students
JLW Label
Number ofUnique
Lexemes ofJLW
Translation of theLeast Frequent 2
Words of JLW
4. (4) Literary words (LW) POS of LW
• More verbs, adverbs and interjections than AW and LAD
• Less verbal nouns and adjectival nouns
• This inevitably means LW have less loan words but more Japanese-origin words.
Number of Unique Lexemes of Japanese Literary Words by POS
56 9 49 2 1 3 10 12 142
168 21 157 20 12 8 28 32 446
199 23 163 25 13 12 28 20 483
137 19 122 15 7 2 27 16 345
85 14 58 5 8 1 21 8 200
645 86 549 67 41 26 114 88 1616
N.(Excl.VN. &AN,)
VN AffixAdj.AN.
(Excl.VN.)
VN: Verbal Noun AN: Adjectival Noun
V.(Excl.VN.)
Adv. TotalOthers
4. (4) Literary words (LW)Q. How many LW overlap with AW and LAD?• Only 27 words (0.5% of academic domain words, 1.7% of LW) are
overlapping• Most of the overlapping words (24/27) overlap with 1 domain words
(17 words overlap with words in biological natural science) • Many physical words such as words for body parts e.g. 左手 (left hand), こぶし (fist), 血 (blood),頭上 (overhead)• No LW words overlap with 4 domain words• Overlapping words are mainly at the intermediate level• No overlapping words in or above 11K+• Some examples of overlapping words: 音 (sound), 光 (light), 棚 (shelf), 組 (class), 岩 (rock), ひざ (knee), 興奮 (excite/-ment), 全身 (whole body), 帝 (emperer), ネズミ (mouse), 帆 (sail)
Word tiers: In what order should students learn them?
• Basic• General• AW/LAD• LW
• Intermediate• General• AW/LAD• LW
• Advanced• General• AW/LAD• LW
• Highly Advanced• General• AW/LAD• LW
• Super-Advanced• General• AW/LAD• LW
• Assumed known words• Proper names• Fillers, Signs• (Transparent compounds *)
• Others
5. Text coverage by word tier• The word tier analyser: An Excel sheet where word
profiling of a text can be checked automatically by cutting and pasting the result of AntWordProfiler with the word tier base word list.
• Text covering efficiency High efficiency in vocabulary learning = Fewer unique lexemes cover more texts (Reciprocal Type/Token Ratio = Token/Type Ratio?) *Comparison should be made between equally-sized texts)
Coverage of Japanese texts by word tierMC UPC OB BCCWJ WP UYN TIS TB MTC- Ss MTC- Tn MTC-Bn
Conversation
Non-academic
prose
Bestsellerbooks
(dominantly novels)
Books &InternetForum
Wikipedia Newspaper
Humanities &
SocialSciences
SocialSciences
SocialSciences
Technological
NaturalSciences
BiologicalNatural
Sciences
1.13 2.10 2.30 32.82 5.90 5.68 0.04 0.19 0.05 0.07 0.01
Word Tier (*)
Total # ofTypes inEach Tier(Lexeme)
AKW (**) (Proper nouns etc.) (30821) 1.7 1.3 2.4 2.0 3.7 1.3 0.9 0.4 0.3 0.6 0.3General 13303 81.0 77.2 78.0 74.7 64.9 63.5 66.1 66.0 67.2 61.1 61.6Academic 2590 2.7 7.6 7.2 10.9 14.9 20.7 20.7 21.3 20.9 23.2 22.7Limited-Aca.-Dom. 2542 1.6 3.2 3.8 5.3 7.3 11.2 8.9 8.9 7.7 5.9 6.8Literary 1616 10.8 7.4 6.5 4.6 1.8 1.7 2.0 1.6 1.6 2.3 1.4Overlap - 27 0.0 - 0.2 - 0.2 - 0.1 - 0.1 - 0.1 0.0 0.0 0.0 - 0.1 - 0.121K + Others - - 2.2 3.5 2.2 2.6 7.4 1.7 1.4 1.8 2.4 7.0 7.3
Total 20024 100.0 100.0 100.0 100.0 100.0 100.0 100.0 100.0 100.0 100.0 100.0
% of Tokens (Overlap included)
Total Number of Tokens forEach Test Corpus (Million)
Text Genre
Name of Text
* All words except 'AKW' and '21K+Others' are listed in top 20000 (01K-20K) ranked by the Word List for International Students (Matsushita, 2011)
** AKW (Assumed Known Words): Words such as proper nouns or fillers which are assumed not to require previous learning.
Findings from the text coverage • Conversation and Non-academic texts: more general words and
LW• Wikipedia: more proper nouns and low frequency words• Academic items of Wikipedia: 15-20% of the texts of are
estimated to be covered by JAWL 1 (559 types) – encyclopaedic nature of AW? can be a good resource for learning AW
• Newspapers: similar to academic texts, but contains more LAD and AW at the advanced level
can be a good resource for learning AW (esp. in social sci.)• Academic texts: more AW and LAD but less LW in any academic
domain• Academic texts in natural sciences: more academic domain words
at lower frequency levels (technical vocabulary) than Ah. and Ss. texts – similar to Coxhead, Stevens, & Tinkle (2010)
6. Proportion of word origin types by word tier
Proportion of Unique Lexemes by Word Origin and Word Tier in 01K-20K (*) (Matsushita, 2011)
Word Origin
Word TierJ apanese
(%)Chinese
(%)
Western &Other(%)
Mixed(%)
ProperNouns
(%)
Unknown& Signs
(%) Total
General 38.4 45.3 10.8 3.2 1.5 0.8 100.0
Academic 15.0 75.2 7.0 1.9 0.4 0.5 100.0
Limited-academic-domain 12.4 69.1 13.7 1.7 2.2 1.0 100.0
Literary 71.7 21.8 2.5 3.1 0.3 0.6 100.0
Overlap 74.1 22.2 0.0 3.7 0.0 0.0 100.0
Total 34.7 50.3 10.0 2.8 1.4 0.8 100.0
*Including 24 compound numerals (01K+)
Findings from the proportion of word origin types by word tier• LW: Japanese origin words are significantly dominant• AW and LAD: Chinese origin words are significantly
dominant• LAD: more Western origin words (Gairaigo) Western origin words tend to appear more at lower
frequency levels in academic domain words• Origins of academic and literary words are
considerably clearly separated: • Academic – Chinese origin• Literary – Japanese origin
7. Implications from the findingsQ. Word Tiers: In what order should students learn them?
• Basic• Academic• LAD• General
• Intermediate• Academic• LAD• General
• Advanced• Academic• LAD• General
• Highly Advanced• Academic• LAD• General
• Super-Advanced• Academic• LAD• General
• Assumed known words• Proper names• Fillers• Signs• (Transparent compounds *)
• Others
Implications for teaching and research• A vocabulary conscious curriculum should be designed
and incorporated in Japanese programs depending on the learners’ needs and language backgrounds
• The gap between Chinese-background learners (CBLs) and non-CBLs will be less in basic conversation and reading literary works than in reading academic texts
• Good curriculum for learning academic domain words is particularly desired for non-CBLs of academic Japanese
• Autonomous mode for learning vocabulary will be necessary particularly when the learners’ needs and language backgrounds are various
8. Conclusion
Limitations of the word lists• Less valid in narrower domain words (2D/1D words) and less
reliable in higher frequency levels Need refining by more complete academic corpus
• Multi-word units not extracted• Not sensitive to different usages in different domains (polysemy)
Remaining issues• Many transparent compounds in JapaneseWhat is Kanji tier? How is it related to word tier?
Download sites for VDRJ/JAWLMatsushita Laboratory for Language Learninghttp://www.wa.commufa.jp/~tatsum/English%20top_Tatsu.html(Interface: English)Google it with “matsushita” and “language”
松下言語学習ラボhttp://www.wa.commufa.jp/~tatsum/index.html(Interface: Japanese)Google it with “松下” and “言語”
Main findings• VDRJ is useful for designing curriculum (material, tests etc.)• The more domains a words is shared as AW or LAD by, the more abstract
the meaning of the word is. • Conversation and non-academic texts contain more general words and
LW• Academic texts: more AW and LAD but less LW in any academic domain• Wikipedia: more proper nouns and low frequency words• Newspapers and academic items of Wikipedia can be a good resource
for learning AW and LAD.• Natural science texts contain more academic domain words at lower
frequency levels than arts and social science texts • Origins of academic and literary words are considerably clearly
separated; 3/4 of LW originate in Japanese while 3/4 of AW and LAD originate in Chinese
• LAD contains more Western origin words (Gairaigo)
References (1)• Anthony, L. (2007). AntConc Version 3.2.1 (text analysis tool)
http://www.antlab.sci.waseda.ac.jp/software.html (Version 1.0 first published in 2002)
• Anthony, L. (2009). AntWordProfiler Version 1.2 w (word profiler) http://www.antlab.sci.waseda.ac.jp/software.html (Version 1.0 first published in 2008)
• Beck, I. L., McKeown, M. G., & Kucan, L. (2002). Bringing Words to Life: Robust Vocabulary Instruction. Solving problems in the teaching of literacy. New York: Guilford Press.
• Butler, Y. G. (バトラー後藤裕子) . (2010). 小中学生のための日本語学習語リスト(試案) (A list of Japanese academic vocabulary for elementary and junior high school students in Japan). 母語・継承語・バイリンガル教育 (MHB)研究 (Studies in Mother Tongue, Heritage Language, and Bilingual Education), 6, 42-58.
References (2)• Chung, T. M. (2003). Identifying technical terms. Unpublished
PhD dissertation, Victoria University of Wellington.• Corson, D. J. (1995). Using English Words. Dordrecht: Kluwer
Academic Publishers.• Corson, D. J. (1997). The learning and use of academic English
words. Language Learning, 47(4), 671-718.• Coxhead, A. (2000). A new academic word list. TESOL Quarterly,
34(2), 213-238.• Coxhead, A., & Hirsh, D. (2007). A pilot science-specific word
list. Revue Francaise de Linguistique Appliquee, 12(2), 65-78.• Coxhead, A., Stevens, L., & Tinkle, J. (2010). Why might
secondary science textbooks be difficult to read? New Zealand Studies in Applied Linguistics, 16(2), 37-52.
References (3)• Dunning, T. (1993). Accurate methods for the statistics of
surprise and coincidence. Computational Linguistics, 19, 61–74.
• Eldridge, J. (2008). No, there isn’t an academic vocabulary but ... TESOL Quarterly, 109-113.
• Hyland, K., & Tse, P. (2007). Is there an “Academic Vocabulary”? TESOL Quarterly, 41(2), 235-253.
• Hu, M. H. & Nation, P. (2000). Vocabulary density and reading comprehension. Reading in a Foreign Language, 13(1), 403-430.
• Juilland, A., & Chang-Rodrigues, E. (1964). Frequency Dictionary of Spanish Words. London: Mouton & Co.
References (4)• Komiya, C. (小宮千鶴子) . (1995). 専門日本語教育の専
門語 -経済の基本的な専門語の特定を目指して- [Technical terms for teaching technical Japanese: Aiming at identifying basic technical terms for economics]. 日本語教育 [Teaching Japanese as a Foreign Language], 86, 81-92.
• Komori, K. (小森和子) , Mikuni, J. (三國純子) , & Kondo, A. (近藤安月子) . (2004). 文章理解を促進する語彙知識の量的側面 ―既知語率の閾値探索の試み― (What percentage of known words in a text facilitates reading comprehension: a case study for exploration of the threshold of known words coverage). 日本語教育 [Teaching Japanese as a Foreign Language], 125, 83-92.
References (5)• Matsushita, T. (松下達彦) . (2010) What words are
essential to read Japanese? Making word lists from a large corpus of books and internet forum sites [ 日本語を読むために必要な語彙とは? -書籍とインターネットの大規模コーパスに基づく語彙リストの作成- ]. Proceedings for the Conference of the Society for Teaching Japanese as a Foreign Language, Spring 2010 [2010 年度日本語教育学会春季大会予稿集 ], 335-336.
• Matsushita, T. (松下達彦) . (2011). 日本語を読むための語彙データベース (The Database for Reading Japanese). Downloaded from http://www.geocities.jp/tatsum2003/, 22 May 2011
• Nation, I. S. P. (2004). A study of the most frequent word families in the British National Corpus. P. Bogaards & B. Laufer (Eds.), Vocabulary in a Second Language: Selection, Acquisition, and Testing (p 3-13). Amsterdam: John Benjamins.
References (6)• Nation, I. S. P. (2011). Making and using word lists. I. S. P.
Nation & Stuart Webb (Eds.), Researching and analysing vocabulary. Boston: Heinle Cengage Learning.
• Oka, M. (岡 益巳) . (1992). 非漢字圏の留学生のための日本経済基本用語表 [Basic terms of the Japanese economy for non-Kanji background students]. 岡山大学経済学会雑誌 (Okayama Economic Review), 23(4), 191-229.
References (7)• Tajino, A., Terauchi, H., Sasao, Y., & Maswana, S. ( 田地野 彰・
寺内 一・笹尾洋介・マスワナ紗矢子 ). (2007). 総合研究大学における英語学術語彙リスト開発の意義 - EAP カリキュラム開発の観点から- (The development of academic words lists at a multi-disciplinary university in Japan: A fundamental step in EAP curriculum design). 京都大学高等教育研究 (Kyoto University Researches in Higher Education), 13.
• Tajino, A., Dalsky, D., & Sasao, Y. (2009). Academic vocabulary reconsidered: An EAP curriculum-design perspective. Journal of Teaching English as a Foreign Language and Literature, 1(4), 3-21.
References (8)• Tamamura, F. (玉村文郎) . (1987). 日本語教育基本 2570 語
[Basic 2570 words for teaching Japanese as a second language]. 日本語の語彙・意味 (2) [Japanese Vocabulary and Meaning], NAFL Institute 日本語教師養成通信講座 [Training Course of Teachers of Japanese as a Second Language]. アルク (Alc).
• Townsend, D., & Collins, P. (2008). Academic vocabulary and middle school English learners: an intervention study. Reading and Writing, 22(9), 993-1019. doi:10.1007/s11145-008-9141-y
• Ward, J. (1999). How large a vocabulary do EAP Engineering students need? Reading in a Foreign Language, 12(2), 309-323.
• West, M. (1953). A General Service List of English Words. London: Longman, Green & Co.