Using Monolingual Crowds to Improve Translation Yakov Kronrod 1 , Chang Hu 2 , Olivia Buzek 2 , Alex Quinn 2 University of Maryland (1) Dept. of Linguistics (2) Dept. of Computer Science UMD HCI Lab (cs.umd.edu/hcil) Comp. Ling. and Info. Processing Lab (umiacs.umd.edu/labs/clip) MonoTrans Project Page: (http://www.cs.umd.edu/hcil/monotrans) ParaTrans: Targeted Paraphrasing Monolingual speakers improving translation quality Human participants perform the following tasks: 1. Identify mistranslated spans in target language 2. Create paraphrases for spans in source language responsible for the target error spans. • Produces gains of 0.82 and 1.32 for adequacy and fluency on a 5-point scale • Improves 65.3% of sentences examined • Yields increases of +0.9 BLEU to +3.8 BLEU, depending on size of error span, in an oracle study. Monolingua l Human Participat ion Qualit y Affordabi lity Space of Available Translation Options •Online system for collaborative monolingual translation •Asynchronous edits, short tasks, minimal context repeat … Target Language Source Language MT Sentence Sentence 1 Vote 2 Identify error spans 3 Create new translation candidates 1 Vote 2 Paraphrase 3 Explain errors Crowd Tasks: New candidate 1 2 3 MT and word alignment … Crowd Tasks: The Bigger Picture: MonoTrans Features Results Translations and Human Judgments Original Google Translate output (GT), with translations produced by the targeted paraphrase translation process (TP), selected to show a range from strong to weak improvements in fluency. Fluency and adequacy (preservation of meaning) were rated on a 5- point scale by independent judges not familiar with this research. Paraphrases from the Crowd 68% of sentences translated perfectly with no bilingual humans

Using Monolingual Crowds to Improve Translation

Download PPTX Report

Upload
aminia
View
48
Download
0

Tags:

Embed Size (px)

DESCRIPTION

1. Using Monolingual Crowds to Improve Translation. Source Language. Target Language. MT. Sentence. Sentence. Crowd Tasks:. 1. 1. Vote. Vote. MT and word alignment. 2. 2. Identify error spans. Paraphrase. Crowd Tasks:. 3. 3. Explain errors. Create new translation - PowerPoint PPT Presentation

Citation preview

Page 1: Using Monolingual Crowds to Improve Translation

Using Monolingual Crowds toImprove TranslationYakov Kronrod1, Chang Hu2, Olivia Buzek2, Alex Quinn2 University of Maryland (1) Dept. of Linguistics (2) Dept. of Computer Science

UMD HCI Lab (cs.umd.edu/hcil) Comp. Ling. and Info. Processing Lab (umiacs.umd.edu/labs/clip) MonoTrans Project Page: (http://www.cs.umd.edu/hcil/monotrans)

ParaTrans: Targeted Paraphrasing Monolingual speakers improving translation quality Human participants perform the following tasks:

1. Identify mistranslated spans in target language2. Create paraphrases for spans in source language

responsible for the target error spans.• Produces gains of 0.82 and 1.32 for adequacy and

fluency on a 5-point scale• Improves 65.3% of sentences examined• Yields increases of +0.9 BLEU to +3.8 BLEU,

depending on size of error span, in an oracle study.

Monolingual Human Participation

Quality

Afforda

bility

Space of Available Translation Options

•Online system for collaborative monolingual translation•Asynchronous edits, shorttasks, minimal context

repeat …

Target LanguageSource LanguageMT

Sentence Sentence

1 Vote2Identify error spans

3 Create new translationcandidates1 Vote

2 Paraphrase

3Explain errors

CrowdTasks:

New candidate1

MT and

word alignment

…

CrowdTasks:

The Bigger Picture: MonoTrans Features Results

Translations and Human Judgments

Original Google Translate output (GT), with translations produced by the targeted paraphrase translation process (TP), selected to show a range from strong to weak improvements in fluency. Fluency and adequacy (preservation of meaning) were rated on a 5-point scale by independent judges not familiar with this research.

Paraphrases from the Crowd

68% of sentences translated perfectly with no bilingual humans