Multimed Tools ApplDOI 10.1007/s11042-012-1100-6

Building mobile multimedia services: a hybrid cloudcomputing approach

Dejan Kovachev · Yiwei Cao · Ralf Klamma

© Springer Science+Business Media, LLC 2012

Abstract Mobile multimedia services are in high demand, but their developmentcomes at high costs. The emergent computing paradigm cloud computing has greatpotential to embrace these issues. In fact, we are at the early stage of the coa-lescence of cloud computing, mobile multimedia and the Web. Motivated by thetremendous success story of the Web based on its simplicity principles, we arguefor a comprehensive review on current practices of web and mobile multimediacloud computing techniques for avoiding frictions. We draw on experience from thedevelopment of advanced collaborative multimedia web applications utilizing mul-timedia metadata standards like MPEG-7 and real-time communication protocolslike XMPP. We propose our i5CLoud, a hybrid cloud architecture, which serves as asubstrate for scalable and fast time-to-market mobile multimedia services. This paperdemonstrates the applicability of emerging cloud computing concepts for mobilemultimedia.

Keywords Mobile multimedia · Cloud computing · Multimedia metadata · XMPP

1 Introduction

The widespread use of smartphones and other mobile devices contributes to un-precedented sharing of mobile multimedia on social networking sites like Facebookor streaming on web sites like YouTube. Additionally, web and mobile multimediaconverge, as the mobile networks become an integral part of the Internet.

D. Kovachev (B) · Y. Cao · R. KlammaRWTH Aachen University, Ahornstr. 55, 52056, Aachen, Germanye-mail: kovachev@dbis.rwth-aachen.de

Y. Caoe-mail: cao@dbis.rwth-aachen.de

R. Klammae-mail: klamma@dbis.rwth-aachen.de

Multimed Tools Appl

However, in current development practices and in the literature we can observetrends which may lead to unnecessary frictions in the development of professionalmobile web multimedia applications. There is an asymmetry in multimedia materialproduction, in multimedia processing and in multimedia material consumption,occurring in web browsers on desktops or in apps on mobile devices. Professionalor semi-professional services for multimedia management and services are availablefor free or at very low rates. Part of this success story is the use of standards andprotocols in web applications like HTML and HTTP. However, if we have a closerlook at mobile devices themselves, it is not possible for technical amateurs to accessmultimedia materials without highly specialized mobile applications (apps) or toshare all multimedia materials via their mobile web browsers. The reasons are theinnovation speed in mobile technologies and the lack of standardization. Flash is anextreme example for one of those discussions among high-tech companies fencingtheir claims. Consequently, great opportunities of mobile devices like real-timecollaboration among mobile users and mobile semantic processing of multimediamaterials can only be realized for high costs in specialized app groups.

On the other hand, cloud computing has great potential to overcome the currentproblems with asymmetric production and use of multimedia materials. However,it has still its limitations in mobile settings. Most cloud computing research anddevelopment is concentrated on the scalability of businesses over the Web andneglecting mostly the needs of (mobile) users [32]. Similar to the cloud computingparadigm, mobile cloud computing envisions transparent augmentation of mobiledevice capabilities via ubiquitous wireless access to cloud storage and computingresources. As a new computing paradigm developers are still tackling how to utilizethe benefits of cloud services properly. It is still questionable, whether it is theright way to rely on public cloud services completely (e.g. Amazon) and howto integrate in-house hardware infrastructure into the cloud. This could addressdifferent challenges to different developer groups who possess different levels ofresources.

Thus, we aim to deal with the task of bringing the multimedia services to mobileusers via cloud technologies. We argue, that we have to try out and evaluate newstandard and protocol suites combined with the mobile cloud computing deliverymodel in order to improve current development practices and to shape mobileand web multimedia convergence. For instance, HTML5 [26] has great potential toremedy the mobile device fragmentation and XMPP takes a role as a cloud protocolgradually [25, 30, 46].

Contributions of this paper include a requirements overview, an architecture, aplatform and services for mobile multimedia cloud computing. We are sure thatwe have not identified all issues on the way to a mobile multimedia architecture,and that at the moment, implementation of those use cases is blurred by obscuretechnical problems and limited development resources. But we are also sure, thatthe engineering knowledge we share here is valuable for other researchers anddevelopers.

In this paper we describe our experiences with the development of advancedmultimedia services and applications (Section 2). From these experiences it wasclear that a comprehensive view on the different perspectives contributing to theuse of mobile multimedia cloud computing and the Web is still missing. Section 3systematizes these perspectives. With our gained knowledge we designed a con-

Multimed Tools Appl

ceptual hybrid multimedia architecture described in Section 4 overcoming manyissues already identified. Especially, this hybrid mobile multimedia architecture helpdevelopers prototype and develop their apps fast. It saves time of taking care ofbuying hardware, installing software, and setting up runtime environments. Thecommonly used services can be well reused, so that developers can focus on theircore functionality and achieve faster time-to-market than usual.

Furthermore, through the new mobile multimedia cloud computing services(Section 5) we tackle problems like combining private cloud infrastructure and publicone offered by big Internet vendors, and multimedia metadata collaboration andimprovement of mobile video user experience. We address the evaluation of oursystem and services in Section 6. Finally, we refer to related work in Section 7, andconclude and describe future work in Section 8.

2 Experiences from building multimedia Web applications

During the last years within our research group we have developed several proto-types of advanced multimedia web applications. Virtual Campfire [10] embraces aset of advanced applications for communities of practice [55]. It is a scenario tocreate, search, and share multimedia artifacts with context awareness across usercommunities. Through standard protocols, a large variety of (mobile) interfacesfacilitate a rapid design and prototyping of context-aware multimedia communityinformation systems. Virtual Campfire uses mobile multimedia semantics expressedin multimedia metadata and multimedia context expressed as ontology models.In the background, Virtual Campfire is fueled by our community engine calledLightweight Application Server (LAS) [49] which follows the service-orientedprinciples.

However, with the existing architectural solution we learned that several changesare needed. In fact, this a common problem with many web applications (generallyusing HTTP/HTML and LAMP) which fail to meet the following requirements [19].First, mobile multimedia applications should be able to handle large data sets andmany users. The reason is that the rise of mobile multimedia, social media, andInternet of Things increases the volume and detail of information that applicationsneed to handle [36]. Second, application scale independence allows the applicationto grow by orders of magnitude without having to change the application. The Web2.0 development model requires the ability to automatically scale (up and down)with the users’ requests [3]. The video creation website Animoto is a commonexample which went from 25000 to 250000 users when it got the sudden popularityon Facebook. Third, as computers are evolving from tools into media that connectshuman and non-human actor representations in digital social networks [28], theability to rapidly deployment and reuse common features like real-time collaborationcan make differences in the time-to-market process. Finally, novel protocols andstandards like XMPP, SIP, or HTTP DASH tend to overcome some of the issues indynamic mobile/Web ecosystem. The HTML5 [26] standardization efforts considerthe requirements for interactive applications that can run both on traditional com-puters and smartphones, thus bridge over the gap between them. Streaming mediaand metadata, HTML5, XMPP, and WebSockets have great potential to empowerusers’ rich multimedia sharing experiences across Web and mobile devices.

Multimed Tools Appl

Under these considerations we have designed a cloud-based multimedia frame-work for mobile and web services using emerging protocols and standards, aspartially described in [30, 31]. The validity of this approach is shown later in thispaper through the following application prototypes:

– SeViAnno [11] is an interactive Web platform for MPEG-7 based semanticvideo annotation. SeViAnno features a well-balanced trade-off between a simpleuser interface and video semantization complexity (see Fig. 1). It allows videoannotation based on the MPEG-7 standard with integrated various taggingapproaches on multi-granular and community levels.

– AnViAnno is a mobile application for context-aware acquisition, sharing andconsumption of mobile videos and images (see Fig. 2). Additionally, AnViAnnocaptures the device (spatio-temporal) context which is further used to supportthe semantic annotation on mobile devices and web clients. Semantic annota-tions are realized as MPEG-7 Semantic Base Types including Agent, Concept,Event, Object, Place, and Time (see Fig. 2 right). AnViAnno seamlessly inte-grates with the web-based SeViAnno, in regard to the multimedia content andmetadata. Actually, AnViAnno can be considered as the mobile counterpart of

Fig. 1 A screen snapshot of SeViAnno’s user interface featuring a video player (top left), videoinformation and video list (bottom left), user-created annotations (top right), and Google mapmashup for place annotations (bottom right)

Multimed Tools Appl

Fig. 2 Screen snapshots AnViAnno—an Android application for context-aware multimedia acqui-sition, annotation, sharing, and consumption

SeViAnno. Additionally, this application allows collaborative metadata creationand sharing.

– Cloud Video Transcoder (ClViTra) is cloud service that adapts video content bytranscoding videos in multiple versions with different parameters, which are thenused for adaptive video delivery to different clients (see Section 6.1).

These applications and others (which are out of the scope of this paper) arebuilt by using services of our multimedia-centric cloud platform called i5Cloud (seeSection 4). Ultimately, the goal of i5Cloud is to enable a single person, i.e. a technicalamateur, to design and run large-scale multimedia applications with a few efforts.The next section systematizes these considerations.

3 Requirements perspectives on mobile multimedia cloud computing

As mentioned above, many emerging and advanced technologies are available to en-rich mobile multimedia experiences. Cloud-based applications are complex informa-tion systems. Therefore, we propose three crucial perspectives on the requirementsfor a mobile multimedia cloud platform: technology, mobile multimedia, and userand community perspective. Each perspective has further sub-perspectives relatedto technology selection and realization. These three perspectives are coming frompractical experiences and literature survey. They are not the complete analysis ofmobile multimedia cloud computing; however, they give a complimentary coverageand useful guidelines for its realization. We have chosen these perspectives becausetechnology drives innovation in new services, multimedia is a central artifact intoday’s digital world, and users and communities are the main actors on Web 2.0.

3.1 Technology perspective

The technology perspective founds a basic ground to facilitate mobile cloud com-puting for application portability and platform independence. Internet & AmericanLife Project and Elon University’s “Imagining the Internet” Center have conducteda survey showing that over some 71% think that most people will work in Internet-based cloud applications such as Google Docs and in applications run from smart-

Multimed Tools Appl

phones by 2020 [43]. They state that mobile devices will still be the driving forces forpeople to make use of cloud-based services and applications. However, there are stilltechnological barriers to use cloud services on capacity-limited mobile devices.

Data management With the growing scale of web applications and the data associ-ated with them, scalable cloud data management becomes necessary part of the cloudecosystem. Some of the popular scalable storage technologies in the moment areAmazon Simple Storage Service (S3), Google BigTable, Hadoop HBase and HDFS,etc. Basically, these distributed blob and key-value storage systems are very suitablefor multimedia content, i.e. they are scalable and reliable as they use distributedand replicated storage over many virtual servers or network drives. The commonapproach to increasing availability (reducing access latency or increasing bandwidth)is to use solutions like content delivery networks (CDNs) which address the issuehow to deliver static multimedia files on the edge.

Regarding multimedia metadata management, the aforementioned techniques areinferior compared with traditional relational databases and ontologies. As these datastorage technologies fall into the category of NoSQL databases, they trade off theschema, joins, and ACID transactions for elastic horizontal scaling and big data stor-age. The ACID transaction principle refers that execution of transactions must beguaranteed with atomicity, consistency, isolation, and durability. This is rather morechallenging for the distributed data storage in the cloud than centralized databasemanagement systems (DBMS). Furthermore, applications need to understand theirpriorities, since it is impossible to achieve Consistency, Availability and Partition-tolerance (the CAP theorem) at the same time [22]. Meanwhile, recent research workreports on data management tools with elastic (scale up and down) consistency whichcan be based on different parameters (application requirements, load, and cost) [3, 5].

Communication Mobile multimedia clouds require broadband Internet connec-tions in order to meet the required quality of experience (QoE). The 4G mobilenetworks with increased upload speed of 500 Mbit/s and download speed of 1Gbit/s [41] open new classes of interactive applications such as instant streamingof videos [20] and remote-rendered 3D content [40]. Application protocols likeXMPP and SIP together with their extensions are powerful protocols for cloudservices that demonstrate several advantages beyond traditional HTTP-based Webservices, e.g. SOAP and REST [30]. XMPP provides a common layer to connecthuman-to-human, human-to-machines and machine-to-machine synchronous andasynchronous communications [25]. For this reason, Almeida and Matos [1] showedthe scalability (in terms of throughput required per number of nodes) and trafficoverhead advantages of XMPP over SOAP/HTTP.

Computation The clouds have a huge processing power at their disposal, but itis still challenging to make it truly accessible to mobile devices. The traditionalclient-server model and Web services/applications can be considered as the mostwidespread cloud application architectures. However, several other approaches toaugmenting the computation capabilities of constrained mobile devices have beenproposed. Offloading has gained big attention in mobile cloud computing research,because it has similar aims as the emerging cloud computing paradigm, i.e. to sur-mount mobile devices’ shortcomings by augmenting their capabilities with externalresources. The full potential of mobile cloud applications can only be unleashed,

Multimed Tools Appl

if computation and storage are offloaded into the cloud, but without hurting userinteractivity, introducing latency or limiting application possibilities [30]. Repre-sentative examples include the execution of software images on virtual machinesinto the cloud [14] or into nearby computers (called cloudlets) [47]. Instead ofoffloading the whole mobile software stack, some propose offloading of applicationparts as computation tasks [33]. The applications could be automatically split [27],or developed intentionally for an adaptive shift of its execution between a cloud anda device [57]. Basically, these approaches enable mobile application developers tohave the illusion as if he/she is programming on much more powerful mobile deviceswith higher computational and storage capacities. Recent studies have showed thatoffloading can efficiently save energy [16, 34] and increase performance [39] by anorder of magnitude on popular mobile platforms.

3.2 Mobile multimedia perspective

The mobile multimedia perspective pertains to aspects related to multimedia itselfas a rich resource for multimedia processing. It is related to how multimedia ispresented or encoded on the lower level, analyzed, and modeled or processed onthe higher level. This requirement is addressed along with the high growth of mobilemultimedia traffic. Cisco Systems has reported that mobile video traffic was 52percent of all mobile traffic by the end of 2011 and this growth will continue to beover 80% in 2016 [15].

Content adaptation Multimedia applications may require transferring adapted mul-timedia through different interconnected networks, servers, and clients with differentmedia modality and quality. Multimedia content is usually compressed using com-pression algorithms or codecs, in order to achieve smaller file size for faster trans-mission or more efficient storage. However, different mobile device media platformsare based on different formats, containers and coding. For example, if we considervideo codecs, Android supports H.263, H.264 AVC, MPEG-4 SP and VP8, whileiPhone supports H.264 and MPEG-4. Obviously, in order to achieve interoperabilityin the heterogeneous ecosystem of mobile platforms, adaptation services are needed.In general, video adaptation requires large computing resources especially if thereare a vast number of users requesting at the same time. Clouds tend to abstract thetechnological complexities connected with seamless multimedia content adaptation.For instance, cloud software-as-a-service (SaaS) solutions like zencoder.com andencoding.com have emerged on the Internet, which can do the heavy lifting of CPU-expensive video encoding, thus relieving clients from upfront investment.

Multimedia semantics Low-level multimedia analysis like feature extraction, met-rics and segmentation can be automatically processed. The input values to theseattributes help browsers process multimedia files of various formats easily. Multi-media analysis, machine learning methods and logic-based modeling have proved tobe good for discovering complex relations and interdependencies, which are servingas input for reasoning in the media interpretation processes.

Multimedia modeling Mobile devices are able to produce different kinds of mul-timedia content. Moreover, those rich sensing functionalities embedded in mobiledevices provide valuable context information which can be used for indexing,

Multimed Tools Appl

querying, retrieval, and exploration of the multimedia content. Multimedia metadatastandards like MPEG-7 or ontologies are the foundations for semantic multimediaknowledge representation and interpretation [9]. In the case of mobile multimedia,the semantics and metadata help the adaptation and personalization processes.

3.3 User and community perspective

This perspective is related to user’s and community’s experiences with regard tomobile multimedia applications. Pew Research Center reported that 71% of onlineadults used video-sharing sites such as YouTube and Vimeo as of May 2011.1 Onthose video-sharing sites, users and communities are the main actors who produceand consume multimedia on mobile devices. The previous two perspectives areaddressed to mobile multimedia service developers, while this perspective is morerelated to end-user requirements. User interfaces are the access bridge betweenusers and applications. And user evaluation procedure can be well applied to testthis perspective.

Sharing and collaboration People as social beings by nature would like to interactwith each others. Meanwhile, the capabilities of mobile networks and devices craftnew ways of ubiquitous interaction over Web 2.0 digital social networks. Conse-quently, mobile devices, Web 2.0, and social software result in two phenomena. First,there is an exponential growth of user-generated mobile multimedia on Web 2.0which, on the other hand, is a driving force for further mobile device improvements.Second, there is a large number of diverse emergent communities, i.e. groups ofpeople, usually co-workers or groups of people who have similar interests trying toperform some tasks to achieve a common goal. These two phenomena clearly showthe demand for easy creation, sharing, and collaboration of multimedia elementssuch as photos, videos, interactive maps, and learning objects, etc. Fortunately,sharing is an indigenous part of cloud services. Sharing through a cloud generallyenhances the quality of service, because cloud-to-device connections tend to bebetter than device-to-device connections.

Ubiquitous multimedia services One of the biggest challenges in future multimediaapplication development is device heterogeneity. Future users are likely to ownmany types of devices. One-quarter of mobile users are predicted to own two ormore mobile-connected devices by 2016 [15]. Switching from one device to anotherusers would expect to have ubiquitous access to their multimedia content. Cloudcomputing is one of the promising solutions to offloading the tedious multimediaprocessing on mobile devices and to making the storage and access transparent.

Privacy and security The adoption of cloud computing will affect security in mobilesystems. The aspects are related to ensuring that the data and processing controlledby a third party is secure and remains private, and the transmission of data betweenthe cloud and the mobile device is secured [35]. Holistic trust models of the devices,applications, communication channels and cloud service providers are required [42].

1http://pewresearch.org/pubs/2070/online-video-sharing-sites-you-tube-vimeo

Multimed Tools Appl

In summary, all these three perspectives make a good structure how to takevarious complex aspects into consideration at developing mobile multimedia cloudarchitecture. Although they have some overlapping sub-perspective as well, thefoci are distinctive and a combination of different (sub) perspectives can refine therequirements. For example, “Ubiquitous multimedia services” is tightly related tothe mobile multimedia perspective as well. From the technology perspective XMPPis related to provisioning of XMPP servers; whereas it is related to realization ofreal-time collaboration from the user and community perspective. Furthermore,MPEG-7 and ontologies enable expressing multimedia semantics. The combinationof MPEG-7 and XMPP enables easy creation of scalable semantic multimedia real-time collaborative applications. Sections 4 and 5 exemplify the three perspectives.

4 i5Cloud: a multimedia cloud computing architecture

Considering the general requirements from the aforementioned three perspectives,we propose i5Cloud, a cloud architecture for multimedia applications and services.On the one hand, we design this architecture mainly with focus on technologyand mobile multimedia perspectives. On the other hand, the user and communityperspective is considered via facilitating and realization of multimedia services.

Three layers can be distinguished in the architecture diagram in Fig. 4. Frombottom up, the infrastructure and platform layers focus on requirements from thetechnology perspective. The multimedia service layer considers the issues from themobile multimedia perspective. Using the multimedia services developers are able tobuild scalable (mobile) multimedia applications that reflect the user and communityrequirements.

The lower layer, Infrastructure Layer, includes storage, compute, and basic soft-ware infrastructure. It can be broken down into several realms to facilitate servicesand applications to run smoothly. Accordingly, Solaris Container Driver is developedto manage the Sparc Enterprise Sun Server in i5cloud. The Deltacloud API enablesweb services to start, stop, persist, destroy, and monitor virtual instances. Figure 3illustrates the state machine diagram of virtual instances in the cloud. These virtualinstances operate on dedicated CPU, virtual/physical memory and storage resourceswhich are easily configured. Virtual machines are grouped into realms which presentboundaries between different computing resources. We have three realms, i.e. aprocessing realm for parallel processing over many machines, a streaming realmresponsible for scalable handling of streaming requests, and a general realm forrunning other servers such as XMPP or Web servers.

At the middle layer, Platform Layer, the resource manager elastically scalesup and down active computing nodes (virtual instances) according to the request

Fig. 3 State machine diagram of cloud computing instances (according to Deltacloud API)

Multimed Tools Appl

demand. In addition to computation and storage services, the i5Cloud platform alsoprovides services for data streaming from the cloud to clients and vice versa. Mediastreaming is achieved by using standard software such as FFMpeg and Wowza2

servers, and text-based data (or metadata) is streamed using OpenFire XMPPservers. Thanks to the extensibility of XMPP, the OpenFire server can easily beextended with its own or third-party plugins. Our collaborative metadata servicesdemonstrate these features (see Section “Collaborative metadata services”). The datastorage manager handles the multimedia content and metadata in a highly-availablemanner. Some of the functionalities of i5Cloud at this level are exposed as platformservices which the application developers can use for more flexible control of theexecution environment. For example, file transfer and RTP streaming provide directaccess to data storage. Furthermore, monitoring services could update applicationswith status information at real time. This has been used in our video transcodingservices with more details described in Section 6.1.

The upper layer of the architecture, Multimedia Service Layer, is explained inthe forthcoming sections with more details. In brief, the concurrent editing andmultimedia sharing components are the engine for the collaborative multimediaand semantic metadata services which are further the main building blocks forcollaborative multimedia applications. The MPEG-7 metadata standard is employedto realize the semantic metadata services. Moreover, the intelligent media adaptationservices enable more interactive mobile video applications, i.e. they contribute to amobile user experience improvement.

Generally speaking, the purpose of i5Cloud is not to compete with public cloudproviders like Amazon AWS or Google AppEngine, which provide a whole paletteof generic compute and storage cloud services. In contrast, i5Cloud is distinguishedin several aspects. First, it is a specialized framework that enables a single developeror a technical amateur to build large-scale applications with focus on multimedia.The burden of scalable multimedia and metadata management is leveraged by theframework. The application developer focuses on the application logic.

Second, we use a hybrid cloud computing strategy. That means the i5Cloud takesadvantage of in-house commodity hardware infrastructure which is usually availablein most organizations, companies or institutions. In a case of cloud burst, i.e. moreresources are needed than those available in the private cloud pool, i5Cloud canautomatically reach external public cloud infrastructures such as Amazon EC2. Asa result of this hybrid cloud computing approach, we achieve a balance betweenresource limitations and re-utilization of existing infrastructure. Intuitively, suchhybrid approach has better security (main components run on controlled hardware)and lower latencies (with respect to data locality). It can be used by individuals orsmall organizations that want dynamically expand their system capacities by leasingresources from public clouds at a reasonable cost, but still retain control of ownapplications and data.

i5Cloud uses two core technologies that enable the next-generation mobile mul-timedia applications (Fig. 4). First, it uses Deltacloud API for the infrastructureto enable cloud interoperability. Second, it puts the bi-directional application-levelprotocol XMPP as a main communication protocol (except for multimedia content

2http://www.wowza.com/

Multimed Tools Appl

Fig. 4 Layered architecture of i5Cloud. The virtualized computing and storage infrastructureenables scalable and highly-available multimedia-centric services with easy development

delivery and streaming). The following subsections provide better insights in thesecore components.

Deltacloud API: enabling cloud interoperabilty Deltacloud3 plays a big role in thei5Cloud architecture. Its RESTful API layer enables cross-cloud interoperabilityon infrastructure level with other cloud providers, e.g. Amazon EC2, Eucaliptus orGoGrid. The Deltacloud Core framework can be extended by creating intermediarydrivers that interpret the Deltacloud RESTful API on the front while communicatingwith cloud providers using their own native APIs on the back. The drivers abstractthe differences of different IaaS cloud providers. The Deltacloud API thus providessingle unified access to heterogeneous cloud infrastructures.

Moreover, this cloud computing architecture is not constrained to run on ourinfrastructure only since the infrastructure layer runs on top of virtualized hardware.The upper three layers can easily be migrated on another public or private cloud in-frastructure, thanks to the virtualization abstraction on service level. In other words,we are using an unified API for common cloud infrastructure management. This iscrucial since many different virtualization layers exist. Popular cloud middlewarelike Eucaliptus or OpenNebula are restricted to the most popular virtualizationtechnologies like Xen or KVM, which were not supported by our hardware. Delta-cloud doesn’t restrict the used virtualization technology. For example, our workingprototype uses Sun Solaris Containers virtualization technology which is pre-built in

3http://deltacloud.apache.org/

Multimed Tools Appl

our in-house hardware. Thus, Delatacloud makes it possible to use heterogeneouscommodity hardware.

XMPP: the glue for mobile cloud services An essential enabling technology in ourapproach is the Extensible Messaging and Presence Protocol (XMPP) [46]. As aninstant messaging protocol XMPP is able to enhance real-time collaboration on mul-timedia metadata, adaptation and sharing. XMPP-based communication takes placenot only among various users and their communities, but also among multimediacloud services as well.

Many advantages over existing technologies make XMPP a highly interestingcandidate for next-generation online services. Many researchers have identifiedXMPP as suitable cloud protocol [25, 54]. When integrating devices with clouds,we must consider two main communication aspects, i.e. communication betweenthe devices and the cloud, and in-between the services within the cloud. An XMPPnetwork can be seen as a complete XML-based routing framework upon whicha messaging middleware can be built. Hence, an XMPP-based middleware canbe used to integrate different services into a distributed computing environment.For example, the application modules, external sensors and external services areXMPP entities identified by unique JIDs (Jabber IDs). Likewise, the long livedprocessing tasks could easily benefit from the built-in publish/subscribe messagingwithout the need of HTTP long pooling techniques. HTTP was originally designedto accommodate query and retrieval of web pages and didn’t aim at rather com-plex communication. The intrinsic synchronous HTTP protocol is unsuitable fortime-consuming operations, like computation-demanding database lookups or videoprocessing.

5 The Multimedia Services Layer in i5Cloud

The Multimedia Services Layer delivers to users and communities mobile services.This section covers some fundamental services that leverage the i5Cloud infrastruc-ture to mobile multimedia use cases which in turn demonstrate the applicability andusefulness of our approach. First, we describe data management services for seamlessmobile/web integration in the multimedia value chain, i.e. acquisition, adaptationand delivery of multimedia documents. Second, we explain services for multimediametadata management.

5.1 Content management services

Mobile devices are becoming indivisible part of the Web today. And mobile/web in-tegration means not only mobile web pages, but also an integration of mobile devicesas equal nodes on the Internet, the same as desktop PCs and servers. Actually, theWeb nowadays is a common communication channel for multimedia content thatis “prosumed” on different personal computing devices such as desktops, laptops,tablets, or smartphones.

Content Management Services support multimedia sharing across different plat-forms ranging from web clients to mobile devices seamless. We demonstrate cross-platform video data management as seen from the mobile multimedia perspective

Multimed Tools Appl

and the technology perspective. The demo prototype highlights the mobile and webmultimedia integration. The mobile application AnViAnno seamlessly integrateswith the web-based SeViAnno (see Section 2). The seamless multimedia integrationis done through i5Cloud which does the heavy-lifting of the necessary multimediaoperations such as transcoding, adaptation, highly-available storage, responsivedelivery, and scalable processing resources.

Both prototypes AnViAnno and SeViAnno demonstrate a current trend inweb applications, i.e. mobile/web integration. Nowadays, most services are offeredon different platforms using the Web as a common denominator. Seamless anduniform interaction with multimedia has become a prerequisite factor for successfulservices. However, the heterogeneity of end devices inevitably requires multimediaadaptation.

Ubiquitous multimedia acquisition and delivery Using AnViAnno, users can recordand annotate video/image content which can be uploaded or directly streamed tothe cloud repository. In the i5Cloud, the videos are transcoded with the cloud videotranscoding service. At the same time, the semantic metadata services handle themetadata content and store it in the MPEG-7 metadata store, making it available toother multimedia services. The management of all MPEG-7 data is implemented asLAS Web services. The LAS multimedia and user management services enable Webclients to create, search, and retrieve MPEG-7 metadata. Afterwards, the uploadedvideos are available in SeViAnno for search, viewing and further annotation. Moredetails about the SeViAnno part can be found in our previous research work [11].

5.2 Metadata management services

As an important part of multimedia, metadata can be used to describe usefulinformation about multimedia artifacts and its content in a machine-readable format.We provide a set of services for mobile clients to annotate multimedia collaborativelyat real-time and to share multimedia and its metadata. Additionally, the servicesexploit the rich mobile context information.

Context-aware semantic annotation services With AnViAnno users can also se-mantically annotate videos. The annotation is based on the MPEG-7 metadatastandard [29]. MPEG-7 is one of the most complete existing standards for multimediametadata. It is XML-based and consists of several components: systems, descriptiondefinition language, visual, audio, multimedia description schemes, reference soft-ware, conformance testing, extraction and use of MPEG-7 descriptions, profiles andlevels, schema definition, MPEG-7 profile schemata, and query format. However,several different approaches have been used as metadata formats in multimediaapplications such as the Ontology for Media Resources [56] and COMM [4]. In orderto enable interoperability between systems using these different formats, we haveimplemented or used mapping services. For example, our MPEG-7 to RDF converter[8] is able to convert MPEG-7 documents into RDF documents for further reasoningthe fact deriving about the multimedia.

As mentioned in Section 2, users are able to capture videos and annotate thevideos with rich semantics in the MPEG-7 standard. The user-generated annotationsare further used to navigate within video content or improve the retrieval from

Multimed Tools Appl

multimedia collections. For example, users can navigate through the video(s) usinga seekbar or semantic annotations. The videos and their annotations are exposed toother internal LAS MPEG-7 services [10] and external clients.

The mobile devices enable users to make the initial metadata enrichment on siteduring the multimedia acquisition and capture context cues. However, they havelimitations in regard to input modes and screen sizes. Our multimedia services usingthe i5Cloud enable extension of the metadata management on desktops or laptopsusing the SeViAnno web application.

Collaborative metadata services The user-generated multimedia content changesrelatively slowly after its creation. On the other hand, the associated metadata isunder constant modification. For example, a video creator initially describes andtags a new video. But after sharing the video, many other people contribute to thevideo with annotations, hyperlinks, comments, ratings, etc. therefore, the success ofmultimedia services highly depends on features for metadata sharing collaborativemetadata editing.

One of the key services for real-time collaboration is shared editing of XMLdocuments. XML has been established as de facto a standard for data exchange andinteroperability, including multimedia metadata standards. Since the nature of XMLis generic and extendable, different kinds of information can be represented, such asgraphic files (SVG), augmented reality content (ARML), and multimedia metadata(MPEG-7), etc. Therefore, real-time collaboration within multimedia applications inpractice generally means concurrent editing of metadata documents in XML.

We use XMPP as a main communication protocol to support real-time col-laboration. Our XMPP-based Mobile Multimedia Collaboration (XMMC) servicesrun on top of i5Cloud. Multiple XMMC clients can bi-directionally stream XMLmessages over the XMMC i5Cloud services using XMPP channels. The Androidclient application (AnViAnno) also operates as a tool for multimedia acquisition,annotation with metadata, and multimedia content and metadata consumption (seeFig. 5).

On the cloud side, the XMPP communication is conducted via an Openfire XMPPServer [44] (see Fig. 4). The system is built on top of the XMPP server as an Openfireplugin. Two XMPP modules are responsible for the XMPP communication betweenclients.

First, Media Catalog Module is responsible to create, retrieve, update, anddelete operations for multimedia content and metadata. The Media Catalog Modulepersists in the basic multimedia-related data on a relational database and ensuresconsistency of this data.

Second, Collaborative Metadata Module handles metadata-related services, i.e.metadata management and synchronization of the annotation metadata at real-timeamong all collaborating client applications. The concurrent XML editing service isbased on the Collaborative Editing Framework for XML (CEFX+) [21, 53]. CEFX+enables concurrent editing of XML documents at real-time using operational trans-formation algorithms [50]. The synchronization is done by keeping a copy of theXML at every client in the session and then in case of edit operations the serviceensures timely updates on the copies. If some conflicts exist, it resolves them andbroadcasts the changes with a message for conflict resolution on all clients.

Multimed Tools Appl

Fig. 5 Main components ofAnViAnno—an Androidclient application for semanticmultimedia. This applicationcan communicate with i5Cloudover three different connectors

During the synchronization of all working document copies, a copy is also situatedat LAS MPEG-7 Integration Service so that the service would act as a client thatsynchronizes the XML metadata file and calls the related LAS MPEG-7 SemanticBase Type Services whenever an XML document is updated. Thus, XMMC achievesinteroperability with pre-exiting and standard-compliant multimedia repositories(such as one described in [7]).

Figure 6 demonstrates the sequence diagram of collaborative multimedia anno-tation. First, Mobile Client-1 sends a <collanno-req> get IQ in order to retrievemetadata XML of a multimedia. Then, i5Cloud Collaborative Annotation Module(CAM) gets the corresponding XML file from Collaborative Editing Service (CES)and sends back the file via XEP-096 SI File Transfer extension of XMPP [37]. Oncethe transfer of the file succeeds and is acknowledged from client side, CAM sendsback <collanno-req> result IQ so that clients can start with annotation.

Meanwhile, Mobile Client-2 also wants to participate in the collaborative anno-tation session. It sends a <collanno-req> message and is followed by the sameprocedure that happens for Mobile Client-1. Both clients share the editing session.Client-1 performs an insert, delete or update operation. The operation is propagatedto all participants of the editing session via <groupchat> message of the XEP-0045Multi-User Chat extension of XMPP [45]. The XML document at i5Cloud is alsoupdated. Finally, CES notifies the LAS MPEG-7 Integration Service for the changesdone.

In summary, the main contribution of the multimedia services layer of i5Cloudand client applications is to augment resource-poor mobile devices through cloudservices such as fast intelligent video adaptation services, in order to enhance

Multimed Tools Appl

Fig. 6 Sequence diagram of a collaborative annotation

user experiences in mobile multimedia and to enable metadata creation, real-timesharing, and concurrent collaborative editing.

6 Evaluation

The evaluation aimed to assess i5Cloud and its mobile multimedia services fromthe three perspectives described in Section 3. The i5Cloud infrastructure layer wastested for the technology and multimedia perspective. This experiment was con-ducted as transcoding of video files. On the other hand, the ubiquitous multimediamanagement and collaborative metadata services were assessed from the user andcommunity perspective.

6.1 Cloud Video transcoding

Cloud Video Transcoder is a scalable hybrid cloud application which uses i5Cloudand Amazon Web Services (AWS). Video transcoding is a technique to adaptivelydeliver video streams across heterogeneous networks. Basically, video is convertedfrom one compressed format to another compressed format for adjustment tothe channel bandwidth or the receiver. Video transcoding [52] is a CPU-intensiveoperation; therefore, it is a suitable test domain for scalable cloud computationapplications. The main idea behind the Cloud Video Transcoder is to use its ownprivate cloud infrastructure, and to start and use extra instances from a public cloudprovider if demanded. In a more concrete sense, users upload multiple videos to thesystem, if the number of videos in the transcoding queue is more than free instancesin the cloud, new instances are started and the videos are transcoded in parallel.When the transcoding queue gets empty, these extra instances are terminated. Inthe cloud, the cloud video transcoding service transcodes the video into streamableformats and stores the different versions of the video.

The i5Cloud runs on a Sun SPARC Enterprise T5240 Server with Solaris 10operating system. The server supports maximum 128 simultaneous threads, that

Multimed Tools Appl

Table 1 Hardware profile details of the i5Cloud and Amazon EC2 instances

i5Cloud Amazon EC2

Micro Small Medium Micro Small Medium

Memory 1 GB 2 GB 2 GB 613 MB 1.7 GB 1.7 GBCPU 2 CPU 4 CPU 8 CPU Up to 2 EC2 1 EC2 Compute 5 EC2 Compute

threads threads threads Compute Unit (1 virtual Units (2 virtualUnitsa (for core with 1 cores with 2.5short periodic EC2 Compute EC2 Computebursts) Unit) Units each)

Instance 234 s 232 s 235 s 96 s 96 s 97 sstartuptime

Max. no. 50 25 12 ∞ ∞ ∞instances

aOne EC2 Compute Unit provides the equivalent CPU capacity of a 1.0-1.2 GHz 2007 Opteron or2007 Xeon processor

means 128 single-threaded instances as virtual machines. Amazon EC2 is used asthe external public cloud provider. Table 1 contains information about the hardwareprofiles of the VM instances in use. The i5Cloud instance types are modeledaccording to Amazons setup, in order to have a coarse comparison at least. FFMpeg4

version 0.8.2 is used as a video processing library.The usage of computing resources of i5Cloud can easily be monitored at real time.

The resource manager exposes its status through the monitoring interface (i5CloudAPI). The resource monitor in Fig. 7 shows the status monitoring of resource usageduring video transcoding tasks from different original formats into selected formats.Additionally, the cloud resource allocation can be visualized at real time during thetranscoding operations of uploaded videos. The overall CPU workload is balancedand reduced through employment of additional Amazon EC2 instances. They havebeen started based on a scheduling algorithm, in order to support i5Cloud during arequest peek (see Fig. 7). More cloud instances can be instantly employed dependingon the workload. When the workload lowers, the number of instances is reduced tooptimize resource usage for efficiency.

Figure 8 shows a comparison chart of transcoding execution time on differentcompute instance types for different file sizes. These three videos were with identicalparameters, i.e. same codec, bitrate, and frames per second.

Figure 9 clearly shows the advantage of the cloud computing approach. Usingcommodity hardware and virtualization technology enables us to get a hybrid cloudcomputing environment that elastically scales up and down depending on the work-load. Particularly, the chart plots the execution times for transcoding of 15 identicalvideos. Each video was 10.73 MB, with duration of 2 min. 8 sec. and encoded witha wmv3 codec. The videos had the bitrate of 626 kb/s and 30 frames per second.The number of videos was kept constant and only the number of instances wasincreased at every iteration starting with 2 instances till 16 instances. Further increaseof the number of instances would not affect the execution time since the number of

4http://www.ffmpeg.org

Multimed Tools Appl

Fig. 7 Cloud resource usage monitoring console: CPU load of processing instances

instances would be greater than the number of videos, i.e. only 15 instances wouldhave been busy and the rest would have been idle. The diagram clearly shows thescalability features of our i5Cloud architecture, i.e. the execution time can be reducedwith increase of the number of instances. The scalability performs similar on our

Fig. 8 Transcoding executiontime on instances withdifferent hardware profiles ini5Cloud private cloud andAmazon EC2 public cloudinfrastructure provider

Multimed Tools Appl

Fig. 9 Scalability of I5Cloud.The chart shows the totaltranscoding time of 15identical video files usingdifferent number ofinstances (x-axis)

private infrastructure and on a public cloud infrastructure (Amazon EC2). Pleasenote that the diagram doesn’t show that i5Cloud private instances perform betterthan Amazon EC2, but they exhibit a similar scalability feature. The differences inthe execution time are probably due to the different hardware profiles.

The goal of i5Cloud (and Cloud Video Transcoder) is not be a data center,but rather using cloud computing principles, to provide development primitives formultimedia applications, which common single-machine applications cannot provide.Additionally, the advantage of multimedia applications developed on top of i5Cloudis that they can easily be deployed on other cloud infrastructure. Therefore, twofactors in the this use case are important, i.e. the ability to scale with the load andoffload to other infrastructure when necessary, which are shown above.

Nevertheless, real-world web applications need to handle thousands of streams.In order to show the benefit of this hybrid approach, we conducted a simulationusing the CloudSim toolkit which is one of the most popular cloud simulationtool [6]. CloudSim allows modeling of datacenters, physical hosts, virtual machines,processing jobs, etc. We modeled the Cloud Video Transcoder with two datacentersrepresenting the private i5Cloud and the public Amazon cloud. The parameters forthe virtual machines were set the same as the small instances at i5Cloud and Amazon.The other parameters were inferred from the empirical results presented above.The experiment consisted of simulating sudden request demand of 10000 videos forprocessing. Each video was set to have size of 100MB. The simulation was repeatedfive times with the maximum number of private (i5Cloud) instances (25), and thenadding 5 additional Amazon EC2 instances at every iteration. Figure 10 shows thespeed up in processing by simply offloading in the public cloud and respective costby using Amazon provider5. For the sake of simplicity, in this model the networklatencies were not considered in the scheduling algorithm and the data transfer costswere not calculated. These results are very coarse estimate.

5http://aws.amazon.com/ec2/pricing/

Multimed Tools Appl

Fig. 10 Simulation of Cloud Video Transcoder using the CloudSim simulator. The maximum 25i5Cloud medium instances are running all the time. Upon a requests spike of 10000 videos, the CloudVideo Transcoder offloads to the Amazon cloud. At each iteration 5 additional EC2 instances areadded, which reduces the total processing time, but on the other hand it increases the monetary costs

6.2 Collaborative metadata services in cultural heritage

In order to evaluate i5Cloud and its services from the user and community per-spective, we apply a use case scenario with regard to professional community’s re-quirements. In the user study and performance test we conducted, seven participantstested the mobile services for collaborative metadata annotation. Different Androidsmartphones and tablets were used. The OS version ranged between Android 2.2and 3.1. The devices’ hardware ranged between 400 MHz and (dual-core) 1.2GHzCPUs and between 256 MB and 1 GB RAM memory. In order to enable participantsto assess the different qualities of the services, they completed short-term tasks thatsimulated documentation work at a cultural heritage site. The tasks consisted of mul-timedia acquisition and sharing, context-aware metadata creation and collaborativeannotations. The goal of the user study was to evaluate the context-aware mobilecollaborative services delivered from the i5Cloud.

A simulated research team consisting of experts on different fields like archeology,architecture, history etc, are documenting an archaeological site. Team memberswere distributed in the field. Figure 11 outlines the scenario. First, a documenta-tion expert discovers some artifacts on site and documents them with photos andvideos. He also tags the multimedia content with basic metadata like name anddescription. The multimedia is stored to the Collaborative Multimedia Cloud, i.e.using the mobile multimedia services running on top of i5Cloud. Other experts,on-site or remote, join the session to collaboratively annotate the multimedia atreal-time for different aspects. Therefore, a collaboration session is established bythe Collaborative Multimedia Services. Then, for example, an architecture expertannotates the origin date/period of an artifact. This annotation is propagated to allteam members immediately. Then, a historian augments that annotation, since hehas deeper knowledge in these scopes. His correction is also pushed to all othersseamlessly.

The mobile multimedia cloud platform comes with real-time support, multime-dia acquisition and sharing with other users, collaborative multimedia metadataannotation and integration with existing metadata repositories. Users can navigate

Multimed Tools Appl

Fig. 11 XMPP-based mobile multimedia collaboration

through the video and place MPEG-7 based annotations collaboratively to fulfill therequirements from the user and community perspective.

Ensuring real-time responsiveness within mobile network settings can be prob-lematic due to the unstable, low-bandwidth, high latency mobile network charac-teristics. We evaluated the CEFX+ framework in collaborative annotation contexton mobile devices. Performance test for sending and executing remote updateswere conducted and the results were analyzed. We used two Motorola Milestonesmartphones. We measured the time passed during a mobile client’s generating anoperation with adding, modifying, or deleting a semantic annotation and sendingthe generated operation to the other mobile client until corresponding operation isexecuted on the client side. In the test scenario, ten insert, ten update and ten deleteoperations have been generated.

Fig. 12 Execution time of remote concurrent XML editing operations on two Android devices in aWiFi network

Multimed Tools Appl

The collected performance values, demonstrated in Fig. 12, reveal average execu-tion time of 412 ms and standard deviation of 209 ms. Both average and deviationvalues are acceptable for near-real-time text-based collaboration applications. How-ever, these values mostly depend on the network characteristics, we have conductedour tests with WiFi connection, the processing time can be longer in different mobilenetworks like EDGE or 3G.

The feedback results shown in Fig. 13 suggest that the collaborative metadataservices have increased cultural heritage awareness of participants to certain extent.The prototype proved to have useful user interfaces with helpful collaborationfeatures such as chatting, real-time concurrent editing of annotations and real-time

Q1. Do you have a background from cultural heritage domain? Value Answer % Not really 28.6%

a few times 71.4%

Q2. Do you think collaborative annotating experience helped you learn and increase your cultural heritage awareness? Value Answer % I learned some information 57.1%

I learned reasonable information 28.6%

I learned quite a lot 14.3%

Q3. How often have you used the conference chat feature during the season?

Value Answer % sometimes 71.4%

often 28.6%

Collaborative Annotation

Absolutely No No Neutral Yes

Absolutely Yes

Q4. Do you think conference chat feature is useful for collaboration? 0.0% 0.0% 0.0% 14.3% 85.7% Q5. Have you received real-time updates on the annotations? 0.0% 0.0% 28.6% 14.3% 57.1% Q6. Do you find suggestions for the map annotation useful? 0.0% 0.0% 14.3% 28.6% 57.1%

Overall satisfaction

Absolutely No No Neutral Yes

Absolutely Yes

Q7. Do you think the prototype in general is useful? 0.0% 0.0% 0.0% 28.6% 71.4% Q8. Do you find the prototype in general successful? 0.0% 0.0% 0.0% 57.1% 42.9%

Fig. 13 Partial evaluation results from the user study of the collaborative multimedia annotationservices

Multimed Tools Appl

Table 2 Mapping of the perspectives

Perspective Current status Future work

TechnologyStorage – Linux FS, RDBMS Cache, replicated storage, CDNCommunication

√XMPP & RTP Adaptive HTTP streaming (DASH)

Computation√

Cloud burst offloading, Improved scheduling modelDeltacloud and monitoring

Mobile MultimediaAdaptation

√Cloud Video Transcoder Scalable video coding

Semantics√

MPEG-7-based Semantic W3C Media annotationsvideo annotation

Modeling√

Context models [8] –User and Community

Sharing & collab.√

XMPP-based mobile Mobile near real-timemultimedia collaboration video stream sharing

Ubiquitousness√

AnViAnno & SeViAnno Distributed user interfacesPrivacy & security – Basic user authentication ACL policies, OpenID

notifications. However, some issues still should be solved especially in regard toproviding collaborative services over unpredictable wireless network connections.

6.3 Discussion and future work

The emergence of cloud computing makes a deep impact on the entire life cycleof mobile multimedia. Since cloud systems exhibit features of complex informationsystems, we propose three perspectives on how to consider such systems.

Table 2 shows a mapping of the (sub-)perspectives on the mentioned services,applications and components. The video streaming and the XMPP-based meta-data streaming deal with the cloud communication. Deltacloud API with videotranscoding and processing services embrace the cloud computing benefits. Thecloud video transcoding service takes charge of multimedia formats; the metadataservices treat the multimedia semantics; and the intelligent processing services usemultimedia (in this case video) analytics to deliver better UX on mobile devices.Furthermore, AnViAnno and SeViAnno demonstrate the convergence of mobileand Web multimedia.

Additionally, Table 2 it lists some of the future steps in order to fully realizethe vision of a mobile multimedia cloud. For instance, using scalable video codingtechniques we could reduce the footprint of adapted content. The new DASHstandard is a promising solution for mobile streaming to heterogeneous devices.Currently, the multimedia metadata employed is limited to only one standard,i.e. MPEG-7 Semantic Base Types. In order to have integration with other videoplatforms and more importantly to incorporate annotations from other sources,mapping tools are required. The W3C Media Annotations Working Group6 workson a promising solution to providing an ontology and API for media object cross-

6http://www.w3.org/2008/WebVideo/Annotations/

Multimed Tools Appl

platform integration. A LAS Service and API will be developed in order to provideaccess to many other multimedia repositories easily. We are developing moremultimedia services that benefit from i5Cloud. Additionally, there are open issuesin i5Cloud such as more optimized resource scheduling where only an experimentalvalidation can incorporate the complexities of such architectures and infrastructureswith specific features [24].

7 Related work

Cloud computing has been hyped a lot in the last years. Existing cloud computingtools tackle only specific problems such as parallelized processing on massive datavolumes [17], flexible virtual machine (VM) management [2] or large data storage[13]. Since it is not a new technology but a technology trend, many researchers try toput their work under the cloud computing umbrella. This also holds for multimediaresearch projects [12].

Several commercial services with cloud features have emerged lately which try torelieve the use from long-processing and pricey-software media operations.7,8 Thecloud video transcoding services using i5Cloud is not to be compared with suchservices, since its main purpose is to demonstrate the capabilities of our i5Cloudhybrid cloud architecture.

Many research projects explore the rich possibilities of application level protocolslike XMPP out of the traditional domain of instant messaging applications [25].MOBILIS [48] project provides a set of generic services for real-time collaborationbased on XMPP and integration to the existing Web 2.0 social networks. The systemtakes advantage of some existing XEPs that provides service discovery, multiuserchat, file transfer and enriched presence services. Junction [18] is an application-levelcommunication platform, providing an XMPP-based framework and libraries forcross-platform application cooperation (mobile and web). These projects exploit thebenefits of the XMPP protocol for collaboration and peer-to-peer XML messaging,but mostly for mobile multi-player games and social applications. They are notfocused on multimedia metadata applications and cloud services.

A lot of research work has been done in collaborative document editing area. Itis still a challenging task to support multiple users to edit a shared document at thesame time. Apache Wave [51] (formerly known as Google Wave) and Novell Vibe[38] represent some of the state-of-the-art systems for collaboration. More interest-ingly, they also use the XMPP as a communication protocol. They are very powerfulcollaboration platforms, but they are still not applicable on mobile platforms becausethey have a large footprint which mobile devices cannot support currently. CEFX+[21, 53] on the other hand, has been already used in mobile applications and we haveused it as a backbone in our collaborative metadata services.

7E.g. http://www.encoding.com8E.g. http://www.zencoder.com

Multimed Tools Appl

8 Conclusions

We have described the key challenges faced by mobile/web multimedia applicationsthrough three sets of requirements perspectives: technology perspective, mobilemultimedia perspective, and user and community perspective. i5Cloud addressesthe opportunity for a hybrid cloud computing solution tailored to this class ofapplications. It provides a substrate for scalable compute services, real-time commu-nication and cloud-interoperable applications, which try to exploit novel protocolsand standards.

The described use cases cover different aspects from the listed perspectives onmobile multimedia cloud computing. The evaluation of the research results is splitin two parts: cloud video transcoding with focus on the technology and the mobilemultimedia perspectives, and collaborative metadata services with focus on theuser and community perspective. The perspectives prove to be useful in helping usdevelop and evaluate such complex mobile multimedia services.

Acknowledgements This work is supported by the Excellence Initiative of German NationalScience Foundation (DFG) within the research cluster Ultra High-Speed Mobile Information andCommunication (UMIC) and in part by NRW State within the B-IT Research School. We thankGökhan Aksakalli and Michael Lottko for their prototype implementations.

References

1. Almeida M, Matos A (2010) Bridging the devices with the Web Cloud: a restful managementarchitecture over XMPP. In: Proceedings of the 6th international mobile multimedia communi-cations conference, vol 10

2. Amazon.com, Inc. (2012) Amazon Web services. URL http://aws.amazon.com3. Armbrust M, Fox A, Patterson D, Lanham N, Oh H, Trushkowsky B, Trutna J (2009) SCADS:

scale-independent storage for social computing applications. In: Proceedings of the 4th biennialconference on innovative data systems research (CIDR)

4. Arndt R, Troncy R, Staab S, Hardman L, Vacura M (2007) COMM: designing a well-foundedmultimedia ontology for the Web. In: The Semantic Web, 6th international Semantic Webconference, 2nd Asian Semantic Web conference (ISWC 2007 + ASWC 2007). Springer,pp 30–43

5. Baker J, Bond C, Corbett J, Furman JJ, Khorlin A, Larson J, Leon JM, Li Y, Lloyd A, Yush-prakh V (2011) Megastore: providing scalable, highly available storage for interactive services.In: Proceedings of the 5th biennial conference on innovative data systems research (CIDR),pp 223–234

6. Calheiros RN, Ranjan R, Beloglazov A, De Rose CAF, Buyya R (2011) CloudSim: a toolkit formodeling and simulation of cloud computing environments and evaluation of resource provision-ing algorithms. Software Pract Exper 41:23–50. doi:10.1002/spe.995

7. Cao Y, Jarke M, Klamma R, Mendoza O, Srirama S (2009) Mobile access to MPEG-7 based mul-timedia services. In: 2009 tenth international conference on mobile data management: systems,services and middleware. IEEE, Taipei, pp 102–111. doi:10.1109/MDM.2009.21

8. Cao Y, Klamma R, Khodaei M (2009) A multimedia service with MPEG-7 metadata and contextsemantics. In: Grigoras R, Charvillat V, Klamma R, Kosch H (eds) [23], CEUR-WS, vol 441.URL http://ceur-ws.org/Vol-441/p09.pdf

9. Cao Y, Klamma R, Kovachev D (2009) Multimedia processing on multimedia semantics andmultimedia context. In: Proceedings of the 10th multimedia metadata community workshop onsemantic multimedia database technologies (SeMuDaTe’09), CEUR-WS, vol 539

10. Cao Y, Klamma R, Jarke M (2010) Mobile multimedia management for Virtual Campfire—theGerman excellence research cluster UMIC. Int J Comput Syst Sci Eng (IJCSSE) 25(3):251–265

11. Cao Y, Renzel D, Jarke M, Klamma R, Lottko M, Toubekis G, Jansen M (2010) Well-balancedusability & annotation complexity in interactive video semantization. In: Proceedings of the

Multimed Tools Appl

4th international conference on multimedia and ubiquitous engineering (MUE 2010), pp 1–8.doi:10.1109/MUE.2010.5575051

12. Cervino J, Rodriguez P, Trajkovska I, Mozo A, Salvachua J (2011) Testing a cloud providernetwork for hybrid P2P and cloud streaming architectures. In: IEEE international conference oncloud computing. IEEE Computer Society, pp 356–363. doi:10.1109/CLOUD.2011.52

13. Chang F, Dean J, Ghemawat S, Hsieh WC, Wallach DA, Burrows M, Chandra T, Fikes A,Gruber RE (2008) Bigtable: a distributed storage system for structured data. ACM TransComput Syst 26(2):1–26. doi:10.1145/1365815.1365816

14. Chun BG, Maniatis P (2009) Augmented smartphone applications through clone cloud execu-tion. In: Proceedings of the 12th workshop on hot topics in operating systems (HotOS XII).USENIX, Monte Verita, Switzerland

15. Cisco Systems (2012) Cisco visual networking index: global mobile data traffic forecast up-date, 2011–2016. White paper, FLGD 10229 02/12. URL http://www.cisco.com/en/US/solutions/collateral/ns341/ns525/ns537/ns705/ns827/white_paper_c11-520862.pdf

16. Cuervo E, Balasubramanian A, Cho Dk, Wolman A, Saroiu S, Chandra R, Bahl P (2010) MAUI:making smartphones last longer with code offload. In: Proceedings of the 8th international con-ference on mobile systems, applications, and services (ACM MobiSys ’10). ACM, San Francisco,pp 49–62

17. Dean J, Ghemawat S (2004) MapReduce: simplified data processing on large clusters. In: Pro-ceedings of the 6th conference and symposium on operating systems design & implementation(OSDI’04). USENIX Association, San Francisco, p 10

18. Dodson B, Cannon A, Huang TY, Lam MS (2010) The junction protocol for ad hoc peer-to-peer mobile applications. Computer Science Department, Stanford University, Stanford, CA94305

19. Fitzpatrick B (2007) Livejournal: behind the scenes, scaling storytime. Talk at USENIX20. Garcia A, Kalva H, Furht B (2010) A study of transcoding on cloud environments for video

content delivery. In: Proceedings of the 2010 ACM multimedia workshop on mobile cloud mediacomputing. ACM, pp 13–18. doi:10.1145/1877953.1877959

21. Gerlicher ARS (2007) Developing collaborative XML editing systems. Ph.D. thesis, Universityof the Arts London

22. Gilbert S, Lynch N (2002) Brewer’s conjecture and the feasibility of consistent availablepartition-tolerant Web services. In: In ACM SIGACT news

23. Grigoras R, Charvillat V, Klamma R, Kosch H (eds) (2009) Proceedings of the 9th workshopon multimedia metadata (WMM’09), Toulouse, France, 19–20 March 2009, CEUR-WS, vol 441.URL http://sunsite.informatik.rwth-aachen.de/Publications/CEUR-WS/Vol-441/

24. Gustedt J, Jeannot E, Quinson M (2009) Experimental validation in large-scale systems: a surveyof methodologies. Parallel Process Lett 19(3):399–418

25. Hornsby A, Walsh R (2010) From instant messaging to cloud computing, an XMPP review.In: Proceedings of the the 14th IEEE international symposium on consumer electronics (ISCE2010). IEEE, Braunschweig, Germany

26. HTML5—A vocabulary and associated APIs for HTML and XHTML (2011). URL http://dev.w3.org/html5/spec/Overview.html

27. Kemp R, Palmer N, Kielmann T, Bal H (2010) Cuckoo: a computation offloading framework forsmartphones. In: Proceedings of the 2nd international ICST conference on mobile computing,applications, and services (MobiCASE 2010). Santa Clara, CA, USA

28. Klamma R, Jarke M (2008) Mobile social software for professional communities. UPGRADEIX(3):37–43. URL http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.141.4712&rep=rep1&type=pdf

29. Kosch H (2003) Distributed multimedia database technologies supported by MPEG-7 andMPEG-21. CRC Press, Boca Raton

30. Kovachev D, Cao Y, Klamma R (2010) Augmenting pervasive environments with an XMPP-based mobile cloud middleware. In: Proceedings of the international workshop on mobile com-puting and clouds (MobiCloud 2010) in conjunction with MobiCASE 2010. Springer, SantaClara, CA, USA

31. Kovachev D, Renzel D, Klamma R, Cao Y (2010) Mobile community cloud computing: emergesand evolves. In: Proceedings of the first international workshop on mobile cloud computing(MCC 2010). IEEE, Kansas City, MO, USA

32. Kovachev D, Cao Y, Klamma R (2011) Mobile cloud computing: a comparison of applicationmodels. CoRR abs/1107.4940

33. Kristensen MD (2010) Empowering mobile devices through cyber foraging: the development ofscavenger, an open, mobile cyber foraging system. Phd thesis, Aarhus University, Denmark

Multimed Tools Appl

34. Kumar K, Lu YH (2010) Cloud computing for mobile users: can offloading computation saveenergy? Computer 43(4):51–56. doi:10.1109/MC.2010.98

35. Lagesse BJ (2011) Challenges in securing the interface between the cloud and mobile systems.In: Proceedings of the 1th IEEE PerCom workshop on pervasive communities and service clouds(PerCoSC 2011). IEEE, Seattle, WA, USA

36. Manyika J, Chui M, Brown B, Bughin J, Dobbs R, Roxburgh C, Hung Byers A (2011) Big data:the next frontier for innovation, competition, and productivity, Technical Report, McKinseyGolbal Institute, New York, USA

37. Muldowney T, Miller M, Eatmon R, Saint-Andre P (2004) Xep-0096: Si file transfer. XEP-0096(Standards Track). URL http://xmpp.org/extensions/xep-0096.html

38. Novell Inc. (2011) Novell vibe. http://www.novell.com/products/vibe/. Last accessed: 30 Novem-ber 2011

39. Ou S, Yang K, Zhang J (2007) An effective offloading middleware for pervasive services onmobile devices. Pervasive Mob Comput 3:362–385. doi:10.1016/j.pmcj.2007.04.004

40. Pajak D, Herzog R, Eisemann E, Myszkowski K, Seidel HP (2011) Scalable remote renderingwith depth and motion-flow augmented streaming. Computer Graphics Forum 30(2):415–424.doi:10.1111/j.1467-8659.2011.01871.x

41. Parkvall S, Furuskär A, Dahlman E (2011) Evolution of LTE toward IMT-advanced. IEEECommun Mag 49(2):84–91

42. Pearson S (2009) Taking account of privacy when designing cloud computing services. In: Pro-ceedings of the 2009 ICSE workshop on software engineering challenges of cloud computing.IEEE Computer Society, Washington, pp 44–52. doi:10.1109/CLOUD.2009.5071532

43. Pew Research Center (2010) The future of cloud computing. URL http://www.pewinternet.org/˜/media//Files/Reports/2010/PIP_Future_of_the_Internet_cloud_computing.pdf

44. Realtime I (2011) Openfire XMPP server. [Online]. http://www.igniterealtime.org/projects/openfire/. Last accessed at April 2011

45. Saint-Andre P (2008) Xep-0045: Multi-user chat. XEP-0045 (Standards Track). URLhttp://xmpp.org/extensions/xep-0045.htm

46. Saint-Andre P, Smith K, Tronçon R (2009) XMPP: the definitive guide. O’Reilly47. Satyanarayanan M, Bahl P, Cáceres R, Davies N (2009) The case for VM-based cloudlets in

mobile computing. IEEE Pervasive Computing 8(4):14–23. doi:10.1109/MPRV.2009.8248. Schuster D, Springer T, Schill A (2010) Service-based development of mobile real-time collabo-

ration applications for social networks. In: Proceedings of IEEE PerCom workshops (PerCol’10),pp 232–237

49. Spaniol M, Klamma R, Janßen H, Renzel D (2006) LAS: a lightweight application serverfor MPEG-7 services in community engines. In: Tochtermann K, Maurer H (eds) Proceed-ings of I-KNOW ’06, 6th international conference on knowledge management, Graz, Austria,6–8 September 2006. JUCS (Journal of Universal Computer Science) proceedings, pp 592–599. Springer. URL http://www-i5.informatik.rwth-aachen.de/lehrstuhl/staff/klamma/download/skjr06.pdf

50. Sun C, Ellis C (1998) Operational transformation in real-time group editors: issues, algorithms,and achievements. In: Proceedings of the 1998 ACM conference on computer supported cooper-ative work, CSCW ’98. ACM, pp 59–68. doi:10.1145/289444.289469

51. The Apache Software Foundation (2011) Apache wave. http://incubator.apache.org/wave/. Lastaccessed: 30 November 2011

52. Vetro A, Christopoulos C, Sun H (2003) Video transcoding architectures and techniques: anoverview. IEEE Signal Process Mag 20(2):18–29. doi:10.1109/MSP.2003.1184336

53. Voigt M (2009) Erweiterung und anpassung des collaborative editing framework for XML(CEFX). Master’s thesis, University of Applied Sciences Erfurt. URL http://www.ai.fh-erfurt.de/studium/studentische_arbeiten/masterarbeiten/einzelansicht_ma_arbeiten/article/erweiterung-und-anpassung-des-collaborative-editing-framework-for-xml-cefx/

54. Wagener J, Spjuth O, Willighagen E, Wikberg J (2009) XMPP for cloud computing in bioinfor-matics supporting discovery and invocation of asynchronous Web services. BMC Bioinformatics10(1):279. doi:10.1186/1471-2105-10-279

55. Wenger E (1998) Communities of practice: learning, meaning, and identity. CambridgeUniversity Press, Cambridge

56. W3C Video on the Web Activity (2011) Ontology for media resources 1.0. http://www.w3.org/TR/2011/CR-mediaont-10-20110707/. Last accessed: 30 November 2011

57. Zhang X, Kunjithapatham A, Jeong S, Gibbs S (2011) Towards an elastic application modelfor augmenting the computing capabilities of mobile devices with cloud computing. MobileNetworks Appl 16:270–284

Multimed Tools Appl

Dejan Kovachev has diploma in computer science and engineering from “Ss. Cyril and Methodius”University in Skopje, Macedonia. Currently, he peruses PhD studies at the information systems chair,RWTH Aachen University. He is supported by the B-IT Research School. His research interests aremobile multimedia and cloud computing, metadata and context-awareness.

Yiwei Cao has a bachelor degree in architecture from Shanghai Tongji University, China. Shereceived her diploma degree in computer science from RWTH Aachen University. She is nowa research assistant at the information systems chair (Informatik 5). Her research interests aremultimedia information systems, spatial databases and geographic information systems (GIS),metadata standards and XML databases, Web-based applications of information systems in culturalheritage and tourism.

Multimed Tools Appl

Ralf Klamma has diploma, doctoral and habilitation degrees in computer science from RWTHAachen University. He leads the research group “community information systems” at the infor-mation systems chair, RWTH Aachen University. He is coordinating and working in four majorEU projects for Technology Enhanced Learning (ROLE, TELMAP, GALA and TELLNET). Heis member of the research excellence cluster “Ultra High Speed Mobile Information and Communi-cation” (UMIC) specialized in mobile multimedia. R. Klamma organized doctoral summer schoolsand conferences in Multimedia Technology Enhanced Learning, and Social Network Analysis. Heis on the editorial board of IEEE Transactions on Technology Enhanced Learning and SocialNetworks Analysis and Mining (SNAM). His research interests are community information systems,multimedia metadata, social network analysis and technology enhanced learning.