COMPUTER 14

TECHNOLOGY NEWS

Published by the IEEE Computer Society 0018-9162/12/$31.00 © 2012 IEEE

Cloud Streaming Brings Video to Mobile DevicesGeorge Lawton

For the rapid growth of mobile-device usage to continue, industry observers say, the devices’ capabilities must

expand and users must be able to do new things with them.

One of those key activities is interactive video streaming. Another could be utilizing smartphones, tablets, and other mobile devices as terminals for graphics-intensive applications such as movie-editing or architectural-design tools.

However, mobile devices don’t have the power to run the calculations that such compute-intensive programs require.

To get around this problem, com-panies such as Active Video, Gaikai, GameStop, OnLive, and Otoy are developing ways to have remote servers run the calculations and then send the results for display to a mobile device. This cloud-streaming approach moves all the graphics-production-related logic to the cloud and, in essence, turns the phone into a dumb terminal.

This contrasts with traditional HTML, Adobe Flash-based, and device-based applications, which require considerable local processing. It also contrasts with traditional video streaming, which typically suffers a delay of at least one second or more, thereby limiting interactivity.

Cloud streaming opens up new interactive mobile opportunities—such as high-performance games and video-related applications—beyond basic download- and streaming-based movie services.

However, interactive cloud stream-ing faces challenges such as latency and the need to port special video browsers to different types of mobile devices.

INTERACTIVE VIDEO STREAMING

People have been able to download or stream video to their smartphones or other mobile devices for about five years. Typically, viewers download content from an online store or from a PC running an application such as

iTunes. However, they must wait for the download to finish before watch-ing the video.

Smartphones also have browsers for viewing content and in some cases special software for watching stream-ing video on services like YouTube.

Traditional streaming services can have a lag of one to four seconds between a keystroke or other input on the phone and changes in the video stream. On phones, this is adequate for noninteractive video with buffer-ing but not for interactive applications like games or graphics-intensive applications such as movie-editing or architectural-design tools.

The new interact ive cloud- streaming video services, on the other hand, operate with a lag of only 150 milliseconds or less, making them suitable for interactive applications.

Various companies have reason to support cloud streaming. The technology represents yet another outlet for content producers’ products. For mobile-device makers, it provides an incentive for buying their products.

Cloud streaming promises to let resource-constrained mobile devices handle compute-intensive applications like interactive video streaming and gaming.

1515FEBRUARY 2012

Figure 1. In a typical cloud-streaming system, remote servers handle the calculations involved in interactive video, games, and other graphics-intensive applications. They send the results via a mesh service through the cloud directly to the mobile or desktop device for display.

Interact ive video streaming could make a greater number of games available to mobile-device users and let them start playing without waiting for a large file to download. This approach also pro-tects the games from piracy, as the application is never downloaded to the client.

UltraViolet. The Digital Enter-tainment Content Ecosystem—a consortium that includes Universal Studios, Warner Bros. Entertainment, other leading Hollywood film stu-dios, and companies such as Adobe Systems and Best Buy—recently launched UltraViolet.

UltraViolet is a digital-rights authentication and cloud-based licensing system that enables con-sumers who buy content to stream and download it to multiple platforms and devices.

And for service providers, it’s another reason for consumers to subscribe to their offerings.

Princeton University associate pro-fessor of computer science Vivek Pai said two important factors enabling and driving interactive stream-ing have been network bandwidth improvements and consumers’ desire to avoid long application startup latencies when installing a new game or program.

Under the hoodA traditional cloud-streaming

architecture consists of a server farm hosting video data. The serv-ers transmit material either directly to consumer devices or through a collection of content-distribution networks (CDN).

The data is pre-encoded into a video compression format such as MPEG-2, MPEG-4, or WebM. Most new smartphone devices include hardware optimized for decoding video.

Because popular content is played on many devices, CDNs can cache these files, thereby reducing the load on the back-end datacenter that transmits the material.

Interactive cloud-streaming appli-cations, on the other hand, involve a direct connection between the client device and the back-end data- center, as Figure 1 shows. In this case, keystrokes and other user input are encoded into data packets and trans-mitted first to the datacenter and then to the cloud application.

The application processes the input and renders a new screen frame and transmits it to the device in real time. This—along with using forward error correction to accommodate delayed or lost packets—reduces latency in the process and enables effective interactivity.

Developments that have made cloud streaming possible include faster mobile networks, which enable real-time, high-quality video deliv-ery; declining networking costs; and

reductions in datacenter operation costs, which make it affordable to host and render video content.

Popular cloud-streaming services include Amazon, Hulu, Netflix, and YouTube.

Major movie studios may start roll-ing out their own cloud-streaming movie services via the Digita l Entertainment Content Ecosystem consortium’s UltraViolet digital rights management (DRM) technology.

ImplementationsCompanies are already imple-

menting cloud streaming.Games. Gaming vendors and

service providers are working on ways to use interactive cloud streaming to render games built for PlayStation, Xbox, and desktop computers so that they can also run on mobile devices.

Editor: Lee Garber, Computer; l.garber@computer.org

TECHNOLOGY NEWS

COMPUTER 16

loaded, on the other hand, providers experience only one-time costs.

However, Gault said, network-operation costs continue to decrease and might soon be too low to cause problems.

For users, watching lengthy videos or playing games for a period of time on mobile devices can generate high data-usage charges.

Also, because there are no stan-dard browser APIs, cloud-streaming providers still must port their video browsers to each type of mobile device they want to work with. This means that not all browsers work with all types of devices, which can create compatibility issues.

S treaming services will accel-erate the ongoing process of users switching from desktops

to mobile devices, said Gault. This process will intensify as more users subscribe to faster mobile networks, he added.

However, said James Hamilton, vice president and distinguished engineer at Amazon Web Services, improvements to the ability of HTML and browsers to render interactive content could provide a viable alter-native to cloud streaming.

OnLive founder Steve Perlman was more optimistic about interac-tive cloud streaming, saying, “We are talking about ubiquitous execution capacity wherever you are, where you can stop thinking about execution as a property of your local device. This is a less expensive and more main-tainable way of doing computing.” 

software to companies wanting to distribute their games and applica-tions as a service.

Otoy has developed algorithms to overcome bottlenecks between its servers and the many user GPUs that would connect to it.

GameStop. GameStop, the US’s largest brick-and-mortar game retailer, recently purchased game-streaming vendor Spawn Labs and will use its platform to make it easier for consumers to try out, buy, and interact with games running in the cloud via interactive streaming.

CHALLENGESWidespread cloud-streaming

adoption faces hurdles such as latency, device compatibility, and networking costs.

Cloud streaming to mobile devices entails no more latency than game consoles or desktop computers for the playing of video or games, noted Gaikai’s Gault.

However, Internet backbone congestion generates bottlenecks that sometimes cause cloud-based interactive video applications to have trouble acting on user com-mands, such as keystrokes, in the 150 milliseconds required to provide performance comparable to applica-tions running locally, said industry analyst Richard Doherty with the Envisioneering Group, a market research firm.

Also, companies that provide video or games via cloud streaming incur ongoing hardware, power, and network-operation costs for as long as the video is watched or the games are played. When content is down-

Customers who own Blu-ray or DVD versions of certain films could receive streamed versions of these movies to their TVs, tablets, smart-phones, and other devices.

Proponents hope this will provide them with some of the income that streaming video is increasingly gen-erating for other companies.

OnLive. OnLive is testing an application for the Android and iPhone platforms that lets users play games—delivered via interactive cloud streaming—using a device’s touchscreen or a wireless game controller.

The company has also developed a video browser that allows users to surf the Internet. Cloud servers assemble and prerender HTML pages and then transmit them to clients via interactive cloud streaming.

Gaikai. Gaikai is offering an interactive streaming service as a marketing tool to game vendors to let potential customers test games before paying for or downloading them.

Gaikai cofounder Andrew Gault said his company has also developed a prototype iPad application for playing the World of Warcraft game using cloud streaming.

Gamestring. Gamestring is devel-oping the Adrenaline personal cloud-streaming gaming service.

It lets users play games on any Web-enabled device, with the pro-vider’s back-end services managing the connection and generating the accompanying video stream. Par-ticipants can share the video in real time, allowing others to watch during game play.

Otoy. Otoy is developing an inter-active cloud-streaming infrastructure for using high-end graphics applica-tions on mobile and other devices.

For example, Otoy is working with Autodesk to run the latter’s compute-intensive 3ds Max 3D modeling, animation, and rendering software on mobile devices.

Otoy CEO Jules Urbach said his company also plans to provide its

Cloud streaming opens up new interactive mobile opportunities.

Selected CS articles and columns are available for free at http:// ComputingNow.computer.org.