Cloud Gaming Onward: Research Opportunities and

Outlook

Kuan-Ta Chen, Chung-Ying Huang, and Cheng-Hsin Hsu

Cloud Gaming is Hot Cloud gaming is expected to lead the future growth of computer games: 9 times in 6 years [CGR]

[CGR] http://www.cgconfusa.com/report/documents/Content-5minCloudGamingReportHighlights.pdf

T5-Labs

Cloud Gaming Onward / Kuan-Ta Chen 4

Limitations of Existing Services OnLive demands for 5 Mbps for reasonable quality OnLive dictates a server rendering/processing latency of 100+ ms, and partially copes with it by setting up 5 data centers (CA, VA, TX, IL, GA)

Only people who live in 1000 mile radius from a data center can play the game

and more… We, researchers, have tons of ideas to improve cloud gaming services, but all existing cloud gaming systems are proprietary and closed

Cloud Gaming Onward / Kuan-Ta Chen 5

Solutions: GamingAnywhere is the first OPEN cloud gaming platform for researchers, service providers, and users

Cloud Gaming Onward / Kuan-Ta Chen 6

GA Has Lower Response Delay

Low response delay

Cloud Gaming Onward / Kuan-Ta Chen 7

Support of Android Devices Implement three proof-of-concept controllers, designed for

Nintendo 64 Nintendo DS Limbo

Nintendo 64 Controller Limbo Controller

Cloud Gaming Onward / Kuan-Ta Chen 8

Demo

• 25k+ visitors, 70k+ downloads, and 400+ forum discussions since April 2013

http://gaminganywhere.org/

Cloud Gaming Onward / Kuan-Ta Chen 10

Research Opportunities

Game Integration Video Codec Virtualization User Interface QoE Measurement and Modeling Server Selection Parameter Adaptation Resource Scheduling

Cloud Gaming Onward / Kuan-Ta Chen 12

Game Integration

Two facets When/how game screens are captured How game inputs are replayed

De facto approach Use API hooking to intercept events related to screen rendering Use API hooking to emulate user inputs

Goal A general framework to bridge cloud gaming middleware and games

Cloud Gaming Onward / Kuan-Ta Chen 13

Video Codec

H.264 is commonly used, but not necessarily the most efficient codec for gaming Network bandwidth is always scarce (e.g., continuous & stable 5 Mbps avail BW cannot be assumed) Considering the huge variety in graphics style…

Video Codec (cont.)

E.g., [8] proposed a layered coding approach that can reduce bandwidth usage significantly

[8] S.-P. Chuah and N.-M. Cheung, “Layered coding for mobile cloud gaming,” in Proceedings of ACM MMVE 2014.

Cloud Gaming Onward / Kuan-Ta Chen 15

Virtualization

GPU virtualization is not as mature as virtualization of other resources (e.g., CPU, memory) Scalability problems financial difficulty of service providers Mediated GPU pass-through technology might be the solution

Cloud Gaming Onward / Kuan-Ta Chen 16

User Interface Mapping

?

Cloud Gaming Onward / Kuan-Ta Chen 17

QoE Measurement and Modeling

Challenges Low cost continuous QoE assessment Large parameter space (network delay/bandwidth/packet loss, codec, game, display, control, and so on)

Common approaches not sufficient Absolute Categorial Rating (ACR) SSCQE, DSCQE

Possible solutions Paired comparison [14] (not continuous and still costly) Physiological approach [15]

Facial EMG Measurement

The corrugator supercilii muscle

Negative emotions

The amount of annoyance caused by latency

[15] Y.-T. Lee, K.-T. Chen, H.-I. Su, and C.-L. Lei, “Are all games equally cloud-gaming-friendly? An electromyographic approach,” in Proceedings of the IEEE/ACM Annual Workshop on Network and Systems Support for Games (NetGames’12), October 2012, pp. 3:1–3:6.

Cloud Gaming Onward / Kuan-Ta Chen 19

Quantifying QoE Degradation due to Latency in Cloud Gaming

Cloud Gaming Onward / Kuan-Ta Chen 20

Server Selection

The closest server may not be available due to Overloaded Not powerful enough for particular games Not installed with particular games

Also, we need to consider Multi-user games where multiple players participates in a game session simultaneously Online games that both (player <-> cloud gaming server delay) and (cloud gaming server <-> online game server delay) need to be considered

Cloud Gaming Onward / Kuan-Ta Chen 21

Parameter Adaptation for Real-time Graphics Streaming

Control-theoretic algorithms are required to adapt to changing network conditions Huge parameter space

Video codec Layer design: MDC, SVC, among others Various parameters associated with video codec

Network delivery Network condition measurement Path selection (overlay?) Multi-path delivery

Playout Error concealment Playout buffering

And lots more…

Cloud Gaming Onward / Kuan-Ta Chen 22

Resource Scheduling

Different games have different requirements on CPU, GPU, memory, and disks

Car racing games are disk read intensive Realistic shooting games are normally GPU intensive RPG games are normally memory intensive

Such requirements may change over time CPU, GPU, memory,

and disk co-scheduling

Cloud Gaming Onward / Kuan-Ta Chen 23

To Make Cloud Gaming Scalable…

In terms of # cloud-enabled games Game integration framework User interface mapping

In terms of # concurrent players GPU virtualization Resource scheduling

In terms of market adoption (UX) User interface, Video codec QoE assessment and management (including parameter adaptation and server selection)

Cloud Gaming Onward / Kuan-Ta Chen 24

Conclusion

Cloud gaming shares similar fundamental problems with many interesting applications

Screencasting Mobile smart lens Tele medicine Immersive remote communications

Thus, cloud gaming can be seen a rewarding entrance to fundamental multimedia system challenges!

Cloud Gaming Onward / Kuan-Ta Chen 25

Kuan-Ta Chen Academia Sinica

cloud gaming rocks!

Thank You!

http://www.iis.sinica.edu.tw/~swc