Test Engineering on Mobage

Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.

�1

2014/02/13 !Masaki Nakagawa Manager, SWET Gr.Platform Unit System Dept.Mobage Business Sector Japan RegionDeNA Co., Ltd.

Mobageを支えるテストエンジニアリング

13-‐A-‐6 #devsumiA

http://event.shoeisha.jp/devsumi/20140213/session/368/

https://twitter.com/search?q=%23devsumiA


About Me

• Masaki Nakagawa

• @ikasam_a

• github.com/masaki

• CPAN Author (metacpan.org/author/MASAKI)

• Testing Casual

http://github.com/masaki

http://metacpan.org/author/MASAKI

Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.http://gihyo.jp/magazine/wdpress/archive/2013/vol75

http://gihyo.jp/magazine/wdpress/archive/2013/vol75




http://atnd.org/events/40914

http://atnd.org/events/40914


Mobage統合事業本部 Japanリージョン事業本部プラットフォーム本部

システム部 SWETグループ

�6


今日話すこと

• Mobage Open Platform について • SWETグループを作った背景 • “SWET”って何？ • SWETグループって何やってるの？

�7


今日話さないこと

• Mobage Open Platform の詳細 • 特定言語に特化した話 • テストコードの詳細

�8


Mobage Open Platform について

�9


Mobage Open Platform とは？

• http://developer.dena.jp/mbga/ • Mobageでゲームを公開できる仕組み • 色々なデバイスに対応 • Feature Phone • Smartphone • PC

�10

http://developer.dena.jp/mbga/


Mobage Open Platform の歴史

• 2009/08 Open Platform Project スタート

• 2010/01 JP Platform (Feature Phone)

• 2010/10 JP Platform (Y!Mobage / PC)

• 2011/02 JP Platform (Smartphone Web)

• 2011/06 JP Platform (Smartphone App)

• 2012/12 JP Platform (ShellApp)

• 2013/05 JP Platform (mixi Mobage / Smartphone)

�11


Mobage Open Platform の歴史

• 2011/06 WW Platform (Smartphone App)

• 2011/09 CN Platform (Smartphone App)

• 2012/02 KR Platform (Smartphone App)

• 2012/11 KR Platform (ShellApp)

• 2012/11 TC Platform (Smartphone App)

• 2013/01 CN Platform (ShellApp)

• 2013/02 KR Platform (Smartphone Web)

�12


Works

• (2011/10) Joined DeNA Co.,Ltd.

• (2012/01) Platform QA Team

• (2013/08) SWET Group


SWETグループを作った背景

�14


Works





• 2011/06 WW Platform (Smartphone App)

• 2011/09 CN Platform (Smartphone App)

• 2012/02 KR Platform (Smartphone App)

• 2012/11 KR Platform (ShellApp)

• 2012/11 TC Platform (Smartphone App)

• 2013/01 CN Platform (ShellApp)

• 2013/02 KR Platform (Smartphone Web)

Mobage Open Platform の歴史(2)

�16


�17

プラットフォームのグローバル展開


立ち上げの背景

• プラットフォームのグローバル展開 • 大規模システムの拡張とリファクタリング • デリバリーのスピードを落とさない • 検証属人性の解消

�18


立ち上げの方針

• End-to-End テストを確立する • テストを徹底的に自動化する • テストしやすい環境を提供する

�19


～テストしやすい環境～

• 単体テストのREDが消えない問題 • リリース頻度・速度・影響範囲のバランス • テスト時間のコスト問題 • CIの必要性

�20


チーム立ち上げ

• 当初はQAチームとして出発 • 2012/01 より 3人でスタート

• 現在はSWETグループ（正式に組織化） • 2014/02 現在 16人

• なぜ独立したチームにしたか？ • 横串チームによる「戦略的横展開」を狙う

�21


“SWET”って何？

�22


Works





http://en.wikipedia.org/wiki/SET

http://en.wikipedia.org/wiki/SET


a

Quality Assurance job title


Names

• SET (Software Engineer in Test) by @Google

• SDET (Software Design Engineer in Test) by @Microsoft

• SWET by @DeNA


Definition

• Google Testing Blog • http://googletesting.blogspot.com

• How Google Tests Software • http://googletesting.blogspot.jp/2011/01/how-google-tests-

software.html

http://googletesting.blogspot.com

http://googletesting.blogspot.jp/2011/01/how-google-tests-software.html


http://googletesting.blogspot.jp/2011/02/how-google-tests-software-part-two.html



the SET

• a developer role focused on testability

• review designs

• refactor code to testable

• write testing frameworks





the TE

• a role that puts testing first

• write automation scripts

• interpret test results

• drive test execution





Focus

• SETs primary focus is on the developer

• Individual feature quality

• Enabling developers to easily test


Developer

Productivity


our SWET = SET + TE

• Developer Productivity

• Quality Assurance


SWET Group Mission Statement

• Keep the quality of Platform

• Improve the quality and productivity of Platform


As an Engineer

• can be developed test target

• NOT a tester but a test engineer


SWETグループって何やってるの？

�38


Our Works

• Server Side Testing

• Client Side Testing

• …


http://perl-users.jp/articles/advent-calendar/2011/test/3 !

Perl Advent Calendar Japan 2011 Test Track - day 3

��40

http://perl-users.jp/articles/advent-calendar/2011/test/3


Classification of Testing

How What

Who Which

Test Levels

Test TypesTest Design Techniques


Who

• Developer Testing

• Acceptance Testing

Developer or User (Customer)


Which

• Unit Testing

• Integration Testing

One or More


How

• Black Box Testing

• White Box Testing

• Gray Box Testing

Techniques for writing tests


What

• Functional Testing

• Non-Functional Testing

• Performance / Stress / Usability / ...

the Purpose of testing


Open Platform Architecture

API

Mobage Developers

Databases / Middlewares

Mobage RESTful API

APIProxyServerAPImbga.jp

Mobile Browser

PC Browser

Mobile App

Mobage SDK

Game Server


Targets

1. Mobage RESTful API (Web API)

2. Mobage Developers (Web Application)

3. mbga.jp & ProxyServer (Mobile Web)

4. Mobage SDKs (Client SDK)

Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.��48

API Server

module module module

API ServerAPI ServerDB

API ServerAPI ServerCache

Test Client

A) WebAPI


Conditions

• HTTP Request to API Server

• JSON Response from API Server

• no UI


API Server




Test Client

Gray Box Fixture


Gray Box Fixture

• DB / Cache Manipulation

• insert test-case specific data

• delete cache for continuous tests


API ServerAPI ServerAPI Server




Test Client

Specific Targeting


Specific Targeting

• programmable local DNS (MyDNS)

• emulates “inet_aton”


Domain Specific Client

• Mobage::API::Client (Perl)

• mobage-api-client (Ruby)


Message Dumping

• keep previous request in response

• interactive debugging


Web Application

M V C



Test Browser

B) WebApp / Mobile Web


Conditions

• HTTP Request to Web App

• HTML Response from Web App

• with JavaScript


Basic Strategy

• Selenium WebDriver

• Ruby, RSpec, Capybara

• Headless Browser


Browser Emulator

Web Application

M V C



Agent Switching


Agent Switching

• Browser can’t touch HTTP Header

• Emulator can’t parse JavaScript

• Save/Restore pseudo agent state


Domain Specific Client

• called “mobage-browser”

• We don’t use “Page Object Pattern”

• “Domain Agent Pattern”


Mobile Emulation

• FireMobileSimulator

• Mobile WebDrivers (DEPRECATED)

• AndroidDriver, IPhoneDriver

• with Real Devices

• Selendroid, ios-driver, Appium


Test Scripts

C) Client SDKs

Test Application

Mobage SDK


Conditions

• create test apps using specific SDK

• use SDK as a developer

• run test scripts to automate test apps

• check SDK-API behaviors


UI Automation

• Multiple Devices Support

• Android, iOS

• Test Cases Consistency

• Calabash

• Appium




Ex) Service Integration

• “Notification from Games” feature

• Game can broadcast notificationto users who is playing

• User can receive notificationat Mobage Chat Web

• User can toggle settingsto receive notification or not at mbga.jp

http://mbga.jp


ScenarioFeature: Notification from Games Scenario: User can receive notification Given user login to mbga.jp When user visit to settings page And user enable to “notification” setting And game broadcast notification using API And user visit to Mobage Chat page Then user can show notification message


ScenarioFeature: Notification from Games Scenario: User can ignore notification Given user login to mbga.jp When user visit to settings page And user disable to “notification” setting And game broadcast notification using API And user visit to Mobage Chat page Then user can not show notification message


Multiple Sessions

• Game

• using “mobage-api-client”

• User

• using “mobage-browser”

• <3 Domain Specific Client


今日話したこと

• Mobage Open Platform について • SWETグループを作った背景 • “SWET”って何？ • SWETグループって何やってるの？

�71


ご清聴ありがとうございました

�72

Technology

Test Engineering on Mobage