Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
�1
2014/02/13 !Masaki Nakagawa Manager, SWET Gr.Platform Unit System Dept.Mobage Business Sector Japan RegionDeNA Co., Ltd.
Mobageを支える テストエンジニアリング
13-‐A-‐6 #devsumiA
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
About Me
• Masaki Nakagawa
• @ikasam_a
• github.com/masaki
• CPAN Author (metacpan.org/author/MASAKI)
• Testing Casual
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.http://gihyo.jp/magazine/wdpress/archive/2013/vol75
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.http://gihyo.jp/magazine/wdpress/archive/2013/vol77
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
http://atnd.org/events/40914
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
Mobage統合事業本部 Japanリージョン事業本部 プラットフォーム本部
システム部 SWETグループ
�6
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
今日話すこと
• Mobage Open Platform について • SWETグループを作った背景 • “SWET”って何? • SWETグループって何やってるの?
�7
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
今日話さないこと
• Mobage Open Platform の詳細 • 特定言語に特化した話 • テストコードの詳細
�8
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
Mobage Open Platform について
�9
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
Mobage Open Platform とは?
• http://developer.dena.jp/mbga/ • Mobageでゲームを公開できる仕組み • 色々なデバイスに対応 • Feature Phone • Smartphone • PC
�10
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
Mobage Open Platform の歴史
• 2009/08 Open Platform Project スタート
• 2010/01 JP Platform (Feature Phone)
• 2010/10 JP Platform (Y!Mobage / PC)
• 2011/02 JP Platform (Smartphone Web)
• 2011/06 JP Platform (Smartphone App)
• 2012/12 JP Platform (ShellApp)
• 2013/05 JP Platform (mixi Mobage / Smartphone)
�11
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
Mobage Open Platform の歴史
• 2011/06 WW Platform (Smartphone App)
• 2011/09 CN Platform (Smartphone App)
• 2012/02 KR Platform (Smartphone App)
• 2012/11 KR Platform (ShellApp)
• 2012/11 TC Platform (Smartphone App)
• 2013/01 CN Platform (ShellApp)
• 2013/02 KR Platform (Smartphone Web)
�12
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
Works
• (2011/10) Joined DeNA Co.,Ltd.
• (2012/01) Platform QA Team
• (2013/08) SWET Group
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
SWETグループを作った背景
�14
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
Works
• (2011/10) Joined DeNA Co.,Ltd.
• (2012/01) Platform QA Team
• (2013/08) SWET Group
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
• 2011/06 WW Platform (Smartphone App)
• 2011/09 CN Platform (Smartphone App)
• 2012/02 KR Platform (Smartphone App)
• 2012/11 KR Platform (ShellApp)
• 2012/11 TC Platform (Smartphone App)
• 2013/01 CN Platform (ShellApp)
• 2013/02 KR Platform (Smartphone Web)
Mobage Open Platform の歴史(2)
�16
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
�17
プラットフォームのグローバル展開
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
立ち上げの背景
• プラットフォームのグローバル展開 • 大規模システムの拡張とリファクタリング • デリバリーのスピードを落とさない • 検証属人性の解消
�18
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
立ち上げの方針
• End-to-End テストを確立する • テストを徹底的に自動化する • テストしやすい環境を提供する
�19
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
~テストしやすい環境~
• 単体テストのREDが消えない問題 • リリース頻度・速度・影響範囲のバランス • テスト時間のコスト問題 • CIの必要性
�20
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
チーム立ち上げ
• 当初はQAチームとして出発 • 2012/01 より 3人でスタート
• 現在はSWETグループ(正式に組織化) • 2014/02 現在 16人
• なぜ独立したチームにしたか? • 横串チームによる「戦略的横展開」を狙う
�21
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
“SWET”って何?
�22
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
Works
• (2011/10) Joined DeNA Co.,Ltd.
• (2012/01) Platform QA Team
• (2013/08) SWET Group
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
http://en.wikipedia.org/wiki/SET
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
a
Quality Assurance job title
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
Names
• SET (Software Engineer in Test) by @Google
• SDET (Software Design Engineer in Test) by @Microsoft
• SWET by @DeNA
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
Definition
• Google Testing Blog • http://googletesting.blogspot.com
• How Google Tests Software • http://googletesting.blogspot.jp/2011/01/how-google-tests-
software.html
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
http://googletesting.blogspot.jp/2011/02/how-google-tests-software-part-two.html
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
the SET
• a developer role focused on testability
• review designs
• refactor code to testable
• write testing frameworks
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
http://googletesting.blogspot.jp/2011/02/how-google-tests-software-part-two.html
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
the TE
• a role that puts testing first
• write automation scripts
• interpret test results
• drive test execution
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
http://googletesting.blogspot.jp/2011/02/how-google-tests-software-part-two.html
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
Focus
• SETs primary focus is on the developer
• Individual feature quality
• Enabling developers to easily test
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
Developer
Productivity
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
our SWET = SET + TE
• Developer Productivity
• Quality Assurance
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
SWET Group Mission Statement
• Keep the quality of Platform
• Improve the quality and productivity of Platform
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
As an Engineer
• can be developed test target
• NOT a tester but a test engineer
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
SWETグループって何やってるの?
�38
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
Our Works
• Server Side Testing
• Client Side Testing
• …
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
http://perl-users.jp/articles/advent-calendar/2011/test/3 !
Perl Advent Calendar Japan 2011 Test Track - day 3
���40
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
Classification of Testing
How What
Who Which
Test Levels
Test TypesTest Design Techniques
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
Who
• Developer Testing
• Acceptance Testing
Developer or User (Customer)
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
Which
• Unit Testing
• Integration Testing
One or More
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
How
• Black Box Testing
• White Box Testing
• Gray Box Testing
Techniques for writing tests
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
What
• Functional Testing
• Non-Functional Testing
• Performance / Stress / Usability / ...
the Purpose of testing
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
Open Platform Architecture
API
Mobage Developers
Databases / Middlewares
Mobage RESTful API
APIProxyServerAPImbga.jp
Mobile Browser
PC Browser
Mobile App
Mobage SDK
Game Server
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
Targets
1. Mobage RESTful API (Web API)
2. Mobage Developers (Web Application)
3. mbga.jp & ProxyServer (Mobile Web)
4. Mobage SDKs (Client SDK)
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.���48
API Server
module module module
API ServerAPI ServerDB
API ServerAPI ServerCache
Test Client
A) WebAPI
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
Conditions
• HTTP Request to API Server
• JSON Response from API Server
• no UI
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
API Server
module module module
API ServerAPI ServerDB
API ServerAPI ServerCache
Test Client
Gray Box Fixture
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
Gray Box Fixture
• DB / Cache Manipulation
• insert test-case specific data
• delete cache for continuous tests
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
API ServerAPI ServerAPI Server
module module module
API ServerAPI ServerDB
API ServerAPI ServerCache
Test Client
Specific Targeting
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
Specific Targeting
• programmable local DNS (MyDNS)
• emulates “inet_aton”
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
Domain Specific Client
• Mobage::API::Client (Perl)
• mobage-api-client (Ruby)
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
Message Dumping
• keep previous request in response
• interactive debugging
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
Web Application
M V C
API ServerAPI ServerDB
API ServerAPI ServerCache
Test Browser
B) WebApp / Mobile Web
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
Conditions
• HTTP Request to Web App
• HTML Response from Web App
• with JavaScript
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
Basic Strategy
• Selenium WebDriver
• Ruby, RSpec, Capybara
• Headless Browser
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
Browser Emulator
Web Application
M V C
API ServerAPI ServerDB
API ServerAPI ServerCache
Agent Switching
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
Agent Switching
• Browser can’t touch HTTP Header
• Emulator can’t parse JavaScript
• Save/Restore pseudo agent state
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
Domain Specific Client
• called “mobage-browser”
• We don’t use “Page Object Pattern”
• “Domain Agent Pattern”
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
Mobile Emulation
• FireMobileSimulator
• Mobile WebDrivers (DEPRECATED)
• AndroidDriver, IPhoneDriver
• with Real Devices
• Selendroid, ios-driver, Appium
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
Test Scripts
C) Client SDKs
Test Application
Mobage SDK
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
Conditions
• create test apps using specific SDK
• use SDK as a developer
• run test scripts to automate test apps
• check SDK-API behaviors
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
UI Automation
• Multiple Devices Support
• Android, iOS
• Test Cases Consistency
• Calabash
• Appium
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.http://gihyo.jp/magazine/wdpress/archive/2013/vol77
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
Ex) Service Integration
• “Notification from Games” feature
• Game can broadcast notificationto users who is playing
• User can receive notificationat Mobage Chat Web
• User can toggle settingsto receive notification or not at mbga.jp
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
ScenarioFeature: Notification from Games Scenario: User can receive notification Given user login to mbga.jp When user visit to settings page And user enable to “notification” setting And game broadcast notification using API And user visit to Mobage Chat page Then user can show notification message
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
ScenarioFeature: Notification from Games Scenario: User can ignore notification Given user login to mbga.jp When user visit to settings page And user disable to “notification” setting And game broadcast notification using API And user visit to Mobage Chat page Then user can not show notification message
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
Multiple Sessions
• Game
• using “mobage-api-client”
• User
• using “mobage-browser”
• <3 Domain Specific Client
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
今日話したこと
• Mobage Open Platform について • SWETグループを作った背景 • “SWET”って何? • SWETグループって何やってるの?
�71
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
ご清聴ありがとうございました
�72