Android InternationalizationUnicode Conference #40

November 1, 2016

Mihai Niță & Roozbeh Pournader

Proprietary + Confidential

Intro

First look — what Android offers

Internationalization APIs (I18N)

● Formatters: number, currency, date, time● Collation● MessageFormat● etc.

Localization framework (L10N)

● Mechanisms for locale aware resources loading, with matching, fallback, ...● Used to localize the OS itself and all the applications (Google & 3rd party)

Standard widgets

I18N — APIs (not an exhaustive list)

JDK equivalent APIs (not an exhaustive list)

● Formatters, extending java.text.Format (for instance DateFormat, NumberFormat, MessageFormat)

● Others (for instance Collator, BreakIterator, Normalizer, Locale, Calendar)

Android additions

● android.text.BidiFormatter, android.text.format.DateFormat, android.text.format.DateUtils, android.text.format.Formatter

I18N — Android implementation

Some of the i18n functionality is implemented “from scratch” (i.e. Locale)

The “tricky areas” (formatters, collators, code page conversion) use ICU4C

● ICU4C is the C/C++ version of ICU (International Components for Unicode)● Using JNI (Java Native Interface) to convert back and forth● ICU4C is not accessible, not even from the NDK (Native Development Kit)

Starting with Android Nougat, ICU4C is (mostly) replaced by ICU4J.

Localization

Resource folders (not only strings, but also layouts, drawables, styles, etc.)

Using the localized resources is simple: Button button = (Button) findViewById(R.id.submitButton); button.setText(R.string.submit);

or <Button android:layout_width="fill_parent" android:layout_height="wrap_content" android:id="@+id/submitButton" android:text="@string/submit" />

Proprietary + Confidential

Let’s build something...

What to do

● Move all strings to resources

● Never concatenate, always use placeholders (nothing shorter than a sentence)

● For placeholders use %1$d, or {0} or {COUNT}, not %d

● Don’t reuse strings

● Include context for translators (like comments or screen-shots)

● Mark non-localizable strings (<string … translatable="false">...</string>)

● Use xliff tags:

<item quantity="other">Your card expires on <xliff:g id="shortExpirationDate"

example="Nov 12">%1$s</xliff:g>.</item>

What to do

● Use formatters, collators, etc.

● Use system pickers, avoid parsing

● Other things might also be localizable: images, styles, sounds, fonts, layouts

● Use the plurals / gender library (not the default Android way)

https://android.googlesource.com/platform/external/messageformat/+/master/

What to do — plural and gender

The default Android way: <plurals name="msgSongsAvailable"> <item quantity="one">There is %d song found.</item> <item quantity="other">There are %d songs found.</item> </plurals> … int songCount = 42; String songsFound = getResources().getQuantityString( R.plurals.msgSongsAvailable, songCount, songCount);

What to do — plural and gender

The default Android way: <plurals name="msgSongsAvailable"> <item quantity="one">There is %d song found.</item> <item quantity="other">There are %d songs found.</item> </plurals> … int songCount = 42; String songsFound = getResources().getQuantityString( R.plurals.msgSongsAvailable, songCount, songCount);

DON’T USE THIS!!!

What to do — plural and gender

The recommended way (small library, was extracted from ICU):https://android.googlesource.com/platform/external/messageformat/+/master/

<string name="msgSongsAvailable"> {count, plural, =1 {There is %d song found.} other {There are %d songs found.} } </string> … int songCount = 42; String msg = getResources().getString(R.plurals.msgSongsAvailable); String songsFound = MessageFormat.formatNamedArgs(msg, "count", songCount);

Bidi — Native RTL support since Android 4.2 (API 17)

● Add android:supportsRtl="true" to the <application> element in the manifest file● Change all of your app's "Left/Right" layout properties to new "Start/End"

equivalents. For backward portability, add them, don’t change them.For example, android:paddingLeft should become android:paddingStart

● APIs and flags○ android:layoutDirection, android:textDirection: recommend using “locale” and

setting them for (almost) all layouts (is a good default, but “it depends”)○ android:textAlignment: uses start / end concepts rather than left / right○ getLayoutDirectionFromLocale(locale)○ android.R.attr.autoMirrored: not a global change

(see Material Design – Bidirectionality)○ ldrtl resource qualifier

BidiFormatter

● Use android.text.BidiFormatter to protect placeholders in text● Transparently inserts Unicode formatting characters to protect the placeholder● If the string may have different direction from placeholder: numbers, usernames,

dates, etc.● The defaults are good for most use cases:

BidiFormatter bidiFormatter = android.text.BidiFormatter.getInstance(); String formattedString = MessageFormat.format(templateString, ... bidiFormatter.unicodeWrap(placeholderValue), ...);

● Support Library copy: android.support.v4.text.BidiFormatter(also supports CharSequence)

Validation

● Use pseudo-translation (en-XA, ar-XB)

● Android Studio’s code analysis

● Input using some “unusual” keyboards (Chinese or Japanese IMEs)

Proprietary + Confidential

What’s new in Android Nougat

New languages in Android Nougat

● Three new fully localized languages (Belarusian, Bosnian, and Punjabi)○ Available on Pixel phones○ OEMs’ choice

● Automatic hyphenation for 24 new languages (from ~7 in Marshmallow)○ Assamese, Bengali, British English, Croatian, Danish, Estonian, French,

German (3 variants, including de-CH-1901), Gujarati, Hindi, Irish, Kannada, Malayalam, Marathi, Mongolian, Oriya, Portuguese, Punjabi, Spanish, Slovenian, Tamil, Telugu, Turkmen, and Welsh

● Support for Variation Sequences● Advanced support for script tags in font selection

What we improved for Android Nougat

A new language selector (181 languages to choose from)

● Better experience for multilingual users● Regional variants (476 locales):

There is no “Spanish”, there are more than 40 “flavors” of Spanish, with different cultural conventions, currencies, etc.

New API in TextView get/setTextLocales, get/setImeHintLocales

We added ICU4J and exposed most of the APIs (android.icu)

● Access to more powerful I18N APIs

We went from:

To:

Looks trivial, just some UI changes...

Settings UI changes

Setup Wizard UI changes

But it is more than that

Resource loading: locale matching & fallback

Better “granularity” for formatters

Public APIs to set / get LocaleList

Text stack changes: better font fallback

Keyboards get added / activated when the locales change

Applications using the new APIs

Locale (Region) Numbers First day of week Date/time

English(United States)

123,456,789.46 Sunday 4/22/13, 3:31 PM

English(United Kingdom)

123,456,789.46 Monday 22/04/2013 15:31

English(India)

12,34,56,789.457 Sunday 22/04/13 3:31 pm

Arabic(Egypt)

١٢٣،٤٥٦،٧٨٩٫٤٥٧ Saturday ٢٠١٠/٦/٢٥ ٩:١٤ م

Arabic(Morocco)

123.456.789,457 Saturday 25/6/2010 21:14

Better “granularity” for formatters

Source: Lorem ipsum dolor sit amet, consectetur adipiscing elit. Duis non erat sem

Better “granularity” for formatters

Locale (Region) Numbers First day of week Date/time

French(France)

123 456 789,457 Monday 22/04/2013 15:29

French(Canada)

123 456 789,457 Sunday 13-04-22 15:29

French(Switzerland)

123'456'789.457 Monday 22.04.13 15:29

French(Luxembourg)

123.456.789,457 Monday 22/04/2013 15:29

French(Algeria)

123 456 789,457 Saturday 22/04/2013 15:29

Source: Lorem ipsum dolor sit amet, consectetur adipiscing elit. Duis non erat sem

Resource loading: locale matching & fallback

● Resources {English=default, French-Canada}, user preference: French-France

Marshmallow: English strings

Nougat: French-Canada strings with fr-FR formatting

● Resources {English=default, German, Italian, French}

Marshmallow: user preference: Romansh → English strings

Nougat: user preference: <Romansh, Italian> → Italian strings

Locale matching algorithm in Android Nougat

● Find the first locale in user preferences that “matches” ● Two locales match if their language and their calculated script match

○ zh ≄ zh-TW (because Hans ≠ Hant)○ zh-US ≃ zh-TW (because Hant = Hant)

● If there are multiple matches, find closest match using tie-breakers:○ Parent locales: en-AU → en-001 → en → default

zh-TW → zh-Hant → default○ Less distance in parentage tree: For es-PR: es-MX > es-ES

For ar-TN: ar-MA > ar-EG○ Special locales win: en-GB represents en-001, es-MX represents es-419

Text stack changes: better font fallback

When presented with CJK text with no extra information from the app:

Marshmallow: user sets <English>Simplified Chinese glyphs

Nougat: user sets <English, Japanese>Japanese glyphs

Also useful for Persian/Urdu/Arabic, Marathi/Nepali/Hindi, etc.

Get your app ready for Nougat

Use the new android.os.LocaleList APIs to make you app “smarter”

If the default folder does not contain English, make a copy of it in the language folder(so if values contains German, make a copy of it in values-de)

Continue using tl locale id for Filipino, even if fil is now supported (since API 21)

Don’t expect zh-TW to fallback to zh (zh-CN does, but zh-TW does not :-)

Add resConfigs in android.defaultConfig: android { … defaultConfig { resConfigs "nb", "nn", "in", "iw", "tl", "zh", "zh-rTW", "b+es+419", "b+sr+Latn" } … }

Proprietary + Confidential

What would you want for future versions of Android?

Let us know: https://code.google.com/p/android/issues

No promises though :-)

Some useful linksLocalizing with Resourceshttps://developer.android.com/guide/topics/resources/localization.html

Localization Checklisthttps://developer.android.com/distribute/tools/localization-checklist.html

String Resourceshttps://developer.android.com/guide/topics/resources/string-resource.html

Android multilingual supporthttps://developer.android.com/guide/topics/resources/multilingual-support.html

The plurals / gender libraryhttps://android.googlesource.com/platform/external/messageformat/+/master/

Material Design – Bidirectionalityhttps://material.google.com/usability/bidirectionality.html

Proprietary + Confidential

Q & A

THANK YOU