Course Report

ShiShi TV

Robert Andersson, Martin Blom, Larry Canady Fredrik Edemar, Liu Hao, Andreas Pettersson, Jan Sundman,

Bob Sundblom, Jie Shang, Fengming Zhang

Methodology At the start of this project most of the group were unfamiliar with and had no practical experience of all different methodologies used in software development. After a short discussion within the group we chose to use the Scrum methodology for managing the project. We chose this methodology for our project due to the following reason:

1. One group member works with scrum 2. Easy to implement3. Project group 2 was going to use it

The first reason for choosing scrum was that during the initial discussion about the project we had a member of our group who had been working at a company that uses scrum. He said that he had positive experiences with Scrum and that it seemed easy to implement and gave great results. This confirmed our second reason that scrum would be easy to implement. The fact that the second project group was going to use Scrum formed a third reason for us to choose the same methodology. We thought that it would be beneficial for both groups if we used the same methodology and thus would be able to bounce ideas off of each other and learn from each others mistakes.

Project groupsSince our project contained three parts, it initially made sense to use three groups that each would have the overall responsibility for a clear cut part of the application. The three groups were Server Group, Mobile Group, and PC Group. Each project member got to choose a group he / she wanted to be part of, this to spur creativity and getting the group members into taking responsibility for the project from the start. The Server Group would be responsible for helping to develop all of the communication protocols such as the protocol between the PC client and the Erlang server and the protocol between the mobile application and the Erlang server (all design decisions are explained later in this document). The Mobile Group was responsible for creating an interface which would be able to play back video files received from the Erlang server and be able to post positive or negative feedback (of the video) to the PC client. The PC group would also create an interface for sending the video files to the Erlang server and receiving feedback from it. This division of groups later came back to haunt us as we later discovered that we would need a additional server, a webserver, which would need developers as well. To solve this we divided the Server Group into a Webserver Group and Erlang Server Group, each having two members.

Project management

Resource allocationDuring the course some group members had the opportunity to attend a Scrum seminar and talk to Henrik Hindbeck about our Scrum implementation and the issues we were facing. We discovered that the way we had set up groups was an effective use of our resources but that we were unable to allocate them effectively, by people having dedicated groups. This resource allocation problem resolved itself toward the end of the project when we lost several members of the group and members from other groups were forced to implement features outside of their dedicated groups.

MeetingsAnother issue we dealt with was the so called Scrum meetings. Some project member were resistant to daily Scrum meetings and we decided that since we worked in close proximity, we would appoint team leaders that would meet every other day together with the product owner. After a brief period these meetings fell to the wayside and were seldom held. Though we had quite infrequent meetings, we feel that this did not in any way effect our progress and with the exception of the PC group, which had to deal with technical issues, we seemed to meet every sprint goal.

Sprint lengths At the inception of the project we decided that our initial sprint length would be one month, so that the end of the sprint would coincide with Review 1. This was entirely too long and by having such a long sprint we would not be able to take advantage of one of the basic principles of Scrum, quick feedback. This was a hard lesson for us and exaggerated the technical issues within the PC group. After the initial sprint we decided to shorten the sprints into being two weeks long.

DemosAt the end of each sprint we would do a demo for our customer to make sure that we were meeting all feature requirements in the list of committed backlog items. This part of Scrum proved to be invaluable during our project, as our customer would tell us what was wrong. This feedback gave us a good direction on how to sort out the issues during the upcoming sprint. If the change request was seen as major, we held a meeting where we clarified the changed backlog item and gave it a proper priority.

The Sprint BacklogThe sprint backlog items list was initially set up by the product owner and the customer and then broken up into clarified items by the team. By doing this we were able to quickly get started programming and not waste a lot of time trying to get a “clear” description of what the customer wanted, as the customer doesn't know exactly what he wants or exactly how it should look. Throughout the project the backlog items changed but not by much.

Problems For the most part, the problems we encountered had little to do with methodology. The problems seem to reflect the usual problems that large groups encounter. From lack of communication to attendance to moral. As explained above meetings were not held regularly which exacerbated our communication problems. We began by having a daily scrum but this quickly fell by the wayside so we then decided to have them every other day. The issue with attendance as with any project were mostly due to illness and vacations. Another issue was the start time. We never had a set time to start which in hindsight was a mistake. Because without a set time people started to come in later and later throughout the project. If we would have had a set start time it would have helped with cross-group communication since there would be a time during the day where everybody was guaranteed to be there. The main reason for this was enthusiasm for the project, as originally this project didn't interest any of the group members and even though everybody joined willingly it's probably always better to work on something that excites you. The enthusiasm for this project never rose throughout but the entire team did pull together when technical issues arose.

Components

PC Client

Capture videoWhen starting the development of the PC client we decided to use Java to keep the application platform independent. After some research we found a library called Java Media Framework (JMF). The idea in JMF is to create a graph from the capture device (camera and microphone) and link it via a processor to a sink that stores the data to a file. However, what we discovered was that the data is not written to disk until the capture devices are released back to the system. This was a problem we ran into while using JMF and resulted in that every time we stopped the capture and restarted it we got some delay due to the startup of the graph. This in turn introduced a gap for one to two seconds between each file. One idea we had was to run two graphs but only have one active at the time. But after trying this we still had to have some delay between each switch since the system required some time to release the camera. Unfortunately JMF only had good support for capture in windows, so the platform independence turned out to be not so independent after all.

At this time we started to look for other ways to solve the capture problem. Since quite a lot of time had passed since the start of the project we felt that we had to make a quick decision on how to continue our work. We therefore decided to look into Direct Show which is a part of DirectX from Microsoft. After some reading on different forums and guides we found a library called GMFBridge. This makes it possible to have one source graph that can run the whole time, hence allowing the preview window to run continuously. The data is forwarded to a bridge and thus the graph that writes data to files can be stopped and written to disk without having to stop the camera, this minimizes the gap between the captured file segments.

File formatsWe are currently capturing the file segments into .wmv format. Direct Show has the possibility to capture data to .avi files also, but the time to convert from .avi to .3gp was much longer, and since we have to complete the conversion in a specified time frame this was not good enough. After the file is written to disk we use ffmpeg to convert the files to .3gp. The conversion is done for two reasons, first the conversion reduces the file size which leads to faster sending of the files to the server. The other reason is that the target mobile phone can't play .wmv files, but .3gp files works fine. The reason that we have to convert the files is that we did not find a way to directly write the files as .3gp. One of the largest improvements to the PC client would be to make a custom file writer that could be used in the capture graph to directly write .3gp files. If this was done, there would be no need to run the conversion thread and a lot of the workload would disappear.

Network Communication with the server is specified in the protocol specification. This document was based on the server-mobile document and some parts were modified to better suit the PC-server communication. Feedback from the viewers is requested from the PC client when it is time to update the information in the user interface. We decided to do it this way so that the client did not have to listen for incoming feedback the whole time.

Erlang ServerThe concurrency and distribution capabilities of Erlang, combined with it's ease of use, made our choice for the main server development language easy. Responsibility for handling connections from senders and mobile clients as well as data and message handling lies with the Erlang server.

Clients are connected with tcp sockets, providing fairly reliable base communication. It is important

that no packages are lost, since that would lead to broken video files and the loss of important control messages. Once connected, clients announce what type of device they are and their intentions, then a new process is spawned to handle their needs.

PC sender clients may upload a continuous series of short video files, which are stored for a short time on the Erlang server. This buffering of files is done using the built in structure, ETS, which we chose because of it's pointer like (actually an identifier) accessibility, efficiency and ready availability. When a video file arrives after the buffer has reached the maximum specified size, the oldest piece is thrown out to make room for the new one, achieving a circular buffer. It is the senders responsibility to keep track of the correct sequence numbers for it's video files. The sender may also request statistics in the form of the number of viewers and their responses, this information is stored in the database and accessed by Erlang when requested.

Mobile viewing clients can request a list of the channels currently playing, pick one and view it. We decided the users needed some more information than just the channel id when picking one, and chose to include the nickname of the sender as the extra piece of info in the list. This could be expanded to include more information about the video itself, such as topic. Thus, the list of channels contains the nickname and channel number of all live channels, which is needed when requesting to view something. It is created by comparing a list of channel identifiers with information regarding past, present and future channels available in the database and their owners. When viewing, the mobile can request the current video sequence or, if needed, an older one. The server does not automatically send out new sequences. We implemented it this way because it proved easier to handle on the mobile side. Finally, the mobile client can send responses, in the form of positive or negative critique. This feedback is stored in the database for future use, such as the sender keeping track of what the viewers think and gathering statistics.

Web Server

ArchitectureIn the original design the project only had three parts, PC client, Erlang server and Mobile client. This meant that in order to view the video the PC client would also need to have viewing capability. But later we thought it would be more convenient for the users to view the live video in a web browser. This way a person could view live performances without having the need to install the PC client application. Therefore we added an Apache webserver to our design.

Web Server technologyAt first we wanted to use J2EE + Tomcat. However though our initial research we learned that using Tomcat might limit the number of connection, so we decided that an Apache server would be more suitable for our needs. It's also free, but the downside is that we needed to learn a new technology, PHP. LAMP (Linux, Apache, MySQL ,PHP is a popular configuration and thus the final components of the webserver's architecture would consist of:

● Linux● Apache(2.0) ● PHP(5.2) ● MySQL(5.0)● PHPMyAdmin(2.11).

Video format3gp was chosen as the video format because it is playable in the mobile phone. Mp4 was considered. Later we decided to go with the flv file format for use in the web browser.

PlayerThe Quicktime player can be embedded it in the website and play the video. But later we discovered that the video we generate with the PC client can't be played in Quicktime. We think this is because the 3gp files created by our PC client is missing header information. So the player can't decode the 3gp file we generate. So we chose the .flv format and started using the FlowPlayer instead. FlowPlayer is a player which supports .flv format. This player is free and we can generate a play list so that we can simulate the video files as a continuous stream.

Streaming serverIn the middle of the project, we discovered that Adobe had a streaming server which we could use. However the free version doesn't support a large number of users. At the same time, the streaming server maybe consume a lot of server resources. So we chose to use an Erlang sever (see above) which can handle a large number of connections. In the end, we discarded the streaming server and implement everything by ourselves.

ShiShi TV

Robert Andersson, Martin Blom, Larry Canady, Fredrik Edemar, Liu Hao, Jan Sund, Bob Sundblom,

Jie Shang FengMing Zhang ,

1

Table of ContentsAbstract......................................................................................................................................................5 Introduction ..............................................................................................................................................5Preliminaries..............................................................................................................................................5

Erlang....................................................................................................................................................5J2ME.....................................................................................................................................................5Flowplayer.............................................................................................................................................6C#..........................................................................................................................................................6GPRS- a means to transfer data to mobile devices over the GSM network..........................................6DirectShow ...........................................................................................................................................6ffmpeg....................................................................................................................................................6

ShiShi TV...................................................................................................................................................6Pc Client................................................................................................................................................6

System Requirements.......................................................................................................................6Installation .......................................................................................................................................7How To Use .....................................................................................................................................7Known Issues....................................................................................................................................9

ShiShiTV – mobile application.............................................................................................................9The programs used during the development.......................................................................................10

Mobile specifications......................................................................................................................10Installation......................................................................................................................................10How to Use Mobile Client .............................................................................................................10

Navigation..................................................................................................................................10Startup........................................................................................................................................11Login..........................................................................................................................................11Channel selection.......................................................................................................................11Viewing video............................................................................................................................12Menu..........................................................................................................................................12Search.........................................................................................................................................13

Overall system description.............................................................................................................13Belive.java.................................................................................................................................14ServerConnection.java...............................................................................................................14VideoModel.java........................................................................................................................14MenuModel.java........................................................................................................................14RecordstoreManager.java...........................................................................................................14

Evaluation and testing.....................................................................................................................14Conclusions....................................................................................................................................15Known issues..................................................................................................................................15

Web Server Application .....................................................................................................................15System Requirements ....................................................................................................................15Register...........................................................................................................................................15Login...............................................................................................................................................16Announce........................................................................................................................................16Viewing live channels.....................................................................................................................17

2

Sending feedback:...........................................................................................................................17Evaluating and testing.....................................................................................................................17Known issues..................................................................................................................................17

Servers......................................................................................................................................................18Webserver............................................................................................................................................18

System description..........................................................................................................................18Announcing....................................................................................................................................18Playing............................................................................................................................................18Feedback.........................................................................................................................................19Installation......................................................................................................................................19

Erlang Server.......................................................................................................................................19Evaluating and Testing....................................................................................................................21

Testing ................................................................................................................................................21System Architecture............................................................................................................................23

Future Work .............................................................................................................................................23References................................................................................................................................................24Appendix A..............................................................................................................................................24

Javadoc................................................................................................................................................24Appendix B: Protocols.............................................................................................................................25

Erlang server - mobile viewer............................................................................................................25Packet information..........................................................................................................................25

Server->mobile data packet ......................................................................................................25Server->mobile message packet ................................................................................................25Server->mobile channel list package.........................................................................................25Mobile->Server message packet ...............................................................................................25

Use Cases............................................................................................................................................26Open Connection.......................................................................................................................26Open Channel.............................................................................................................................26Open Channel with user and password......................................................................................26Close Channel............................................................................................................................26Get Stream Piece........................................................................................................................26Get List.......................................................................................................................................27Get Current Sequence Number..................................................................................................27

Messages.........................................................................................................................................27Error Messages ..........................................................................................................................27Current Messages ......................................................................................................................27

Erlang server – PC Sender...................................................................................................................28Packet information..........................................................................................................................28

PC Sender -> server data packet ...............................................................................................28PC Sender -> server message packet ........................................................................................28Server -> PC Sender message packet ........................................................................................28

Use Cases .......................................................................................................................................28Open Connection ......................................................................................................................28Open Connection with user and password ................................................................................29Send Stream Piece .....................................................................................................................29

Messages ........................................................................................................................................29Error Messages ..........................................................................................................................29

3

Current Messages ......................................................................................................................29Internal client error messages ...................................................................................................30

Erlang server - webserver...................................................................................................................30Packet information..........................................................................................................................30

Server->webserver data packet .................................................................................................30Server-> webserver message packet .........................................................................................30Webserver->Server message packet ..........................................................................................30Messages ...................................................................................................................................30

4

Abstract

The project described in this paper was done at Uppsala University during the fall semester of 2007. The project group consisted of seven undergraduates and three master students. The project involved the development an application which could broadcast “live” video from a webcam to a web page and also a mobile telephone. Three applications were developed for the project. An application for broadcasting video from a personal computer using a webcam, a website for viewing the content “live” on a personal computer and a mobile application for viewing the content on a mobile phone. Both the website and the mobile phone allow viewers to give real-time feedback to the broadcaster.

Introduction

The main objective of the project course is to give students practical experience working in large project group. The students are required to choose and implement a methodology, do background research, planning of the project, choose which programming languages are suitable, write the documentation and all other responsibilities a real world project would include.

Webcams are an extremely popular accessory for personal computers today. One popular use of webcams is to chat with friends and family. Programs such as Skype, MSN Messenger, GTalk and others allow users to use their webcams to broadcast video of themselves while they chat. Another indispensable item for everybody is the mobile phone. The idea behind ShiShi TV was to create an application which would allow individuals to broadcast an event “live” to both mobile telephones and a website. The ability to broadcast a performance from a webcam to a website where individuals can view these performances has been done countless times . We feel however that our ShiShi TV has several features which make it unique. First the ability to broadcast to mobile phones. Now viewers are able to access performances anywhere. Another feature is the interaction between the broadcaster and the viewer via a feedback system. This allows viewers to “vote” whether or not they liked the video, and the broadcaster will receive this feedback directly. We feel our solution is unique and somewhat non-traditional in the sense that we do not employ the use of a streaming server of any kind.

Preliminaries

ErlangErlang is a concurrent programming language and runtime system designed by Ericsson. It supports fault-tolerant, distributed systems that can run non-stop with hot swapping of code. By default Erlang runs interpreted virtual machine code, though there is a native code compiler available, developed by HiPE. These days the language is open source [1].

5

J2MEJava 2 Micro Edition (J2ME) [2] is a set of technologies and specifications developedfor small device like mobile phones. J2ME uses a subset of J2SE (Java 2Standard Edition) [4] components such as a smaller virtual machine and leanerAPIs. These are similar to J2SE but with highly reduced functionality. Althoughit has some unique features not present in J2SE.

FlowplayerAn open source Flash Player which supports progressive downloading and streaming

C#An object oriented programming language developed by Microsoft

GPRS- a means to transfer data to mobile devices over the GSM network.

DirectShow A multimedia Framework developed by Microsoft

ffmpegAn open source project which offers complete a solution to record, convert and stream audio and video.

It is part of the Mplayer open source project.

ShiShi TV

Pc ClientThe pc client has five major functions 1) capturing the video stream from the webcamera 2) creating small files from that stream 3) converting the files to 3gp 4) send the files to the Erlang server 5) retrieve feedback from the Erlang server. The reason for this solution was to a) cut down on network traffic and also due to the fact that we have a limited amount of usable memory in a mobile telephone. One question which was raised was whether or not this was the optimal solution or should we have chosen to stream the video to the server where it could then be cut into playable chunks and sent to the mobile phone and the webserver. We thought that our solution offered the best work-around since we wanted to put as much workload on the pc as possible due to the fact we were only using one server and given the processing power of today's pc's.

System Requirements

6

Intel Celeron 2.4 ghz (or equivelant)

1Gb RAM

Windows XP SP2

.NetFramework

DirectX

Installation The pc client is installed using either an installer program . When the installer program is executed the user is prompted to choose where the program will be installed. The program will then be installed along with the following:, GMFBridge.dll , tempFiles(empty Catalog), DirectShowLib-2005.dll, and ffmpeg.exe.

How To Use The use of the pc client requires that the user is a registered member of a website supporting ShiShi TV.

1. After the user enters their username and password if they are correct the user is successfully logged in. The Start Broadcasting button is pressed the program is activated and the ON AIR sign turns red.

2. File Menu with sub menu for Exit and Settings

7

2

1

4

5

3. Settings allows the user to select their Video device and /or Audio Device

4. View Rating , receives feedback from ShiShi TV (either mobile viewers and/or web viewers)

5. Log in status

Known IssuesThis version of the pc client is the most stable we were able to develop during the course. There are some issues which need to be addressed

● the program does have a tendency to crash after 35 minutes.

● Corrupt files can be created and sent to the server This doesn't affect the pc client but does have an effect on viewing in the Flash player.

● Switching between audio and video devices during playing is not recommended

● some webcams have a tendency to make the program crash.

● Not all microphone input devices work

ShiShiTV – mobile applicationThe ShiShiTV mobile application gives the user an opportunity to follow his favorite live shows while being on the move. The application provides an easy to use interface for login in, browsing available live streams and watching the live stream of choice, all in a small Java application runnable on a multitude of mobile devices. The following documentation will cover all the aspects of the ShiShiTV mobile application. This includes installation of the application on the mobile, the usage of the

8

3

3

application and finally conclusions and future work proposals. The application is aimed to contain a subset of the ShiShiTV web application, providing the users the opportunity to watch live content while on the move. The application connects to the ShiShiTV Erlang server and requests data such as channel lists and video files. This content is the same as the one presented in the ShiShiTV web application.

The programs used during the development

Eclipse IDE – The IDE used for writing the application. EclipseME – Plugin for Eclipse that connects the Wireless Tool Kit to Eclipse and does the

basic J2ME work such as creating jar-files that are runnable on mobile phones. Sun Java J2ME (Java version 5) Sun Wireless Tool Kit (WTK version 2.5.1 for Linux) – Provides emulators for testing the

application during development.

Mobile specificationsThe main test bed besides the emulator bundled inside WTK, was a Nokia 6233 mobile phone. To run the java application on other hardware, the mobile device must support the following specification.

MIDP 2.0 – Mobile Information Device Profile CLDC 1.1 – Connected Limited Device Configuration MMAPI – Mobile Media API

The API for these can be found at http://java.sun.com/javame/reference/apis.jsp#api .

InstallationThe mobile application is currently created using J2ME with MIDP 2.0 and CLDC 1.1, therefore you should make sure that the mobile phone supports this before trying to use the application. The application is easiest installed by downloading the .jar file.

How to Use Mobile Client The following sections will cover the usage of the mobile application and what functionality is located in the different screens. The usage explanation covers how the application is handled using a Nokia Series 40 mobile phone, as this was the primary testbed during the development of the application.

9

There should be no apparent reason for not being able to run it on similar mobile phones from makers such as Sony Ericsson.

NavigationThe basic navigation of the mobile application is centered around five different soft buttons on the mobile phone. The different buttons and their functionality are covered in the table below.

Button Function1 Navigation buttons, use the left and right to

navigate between tabs.These buttons also allows you to navigate between the different controls on the display.Use the middle button to select (action button).

2 Removes a character when standing in a textinputfield.

3 Switches between lower and upper case when standing in a textinputfield.

4 Switches between text and numerical input when standing in a textinputfield.

StartupThe installed application is usually installed in a special folder intended only for Java applications, locate this folder and start the application by invoking the program ShiShiTV using the action button. Upon startup you will be asked if the application should be allowed to send and receive packet data, answer “yes”, otherwise the application will not work.

Note! Check what your service provider charge for sending and receiving data, the service might be expensive! If available, you should consider getting a fixed rate plan with unlimited packet data.

LoginIf the application is used for the first time the login screen will be shown and the user will be able to enter the appropriate credentials. The input type of the text input fields is shown behind the caret and can be changed by clicking either button 3 or button 4.

10

The login procedure contains the following. Fill in your user name and password in the fields marked as “Username” and “Password”, these are the same as the ones that are used to access the Internet site. To authenticate with the server, navigate down to the “Login”-button and activate it using the action key.

When logged in the username will be displayed to the lower right on the screen.

Note: If you do not possess a user account, this step can be skipped by switching to next tab with the navigation buttons.

Channel selectionThe channel tab is shown after the user has authenticated with the server or if he has chosen not to login by navigating to the next tab from the login tab.

The channel list is downloaded from the server each time the channel tab is shown and the channel list can then be browsed by navigating up and down. To view channels just activate the channel of choice using the action button.

Viewing videoAfter activating a channel in the channel list the actual video screen is shown, this screen features the video and three buttons. The buttons and their functionality are:

Positive feedback – sends positive feedback to the content provider. Back button – used to navigate back to the channel list. Negative – sends negative feedback to the content provider.

11

To activate the buttons navigate to the button of choice and then activate them using the action button.

MenuThe menu tab of the application contains different options that can be used for managing the application. These options are currently:

Login as a different user – used for login in as a new different user Themes – used for changing the look and feel of the application Remove credentials – removes the credentials from the persistent storage Change server – change the server URL and port. (Used for testing purposes only) Exit program – used for exiting the application

The options are chosen by using the up and down navigation buttons and activated by clicking the action button.

12

SearchThe search feature has not been implemented and is only featured for future reference.

The search tab should contain different fields for entering different search criteria, which should be sent to the server and generate a channel list based on these criteria.

Overall system descriptionThe overall functionality of the application is documented in this chapter, but specific details are found in a javadoc which is available as appendix A.

The main classes of the application and their functionality are specified in coming sections.

Belive.javaThis class is a MIDlet which is the actual entry point into the program. The class handles what should be done when the application is started, and also releases resources when the application is shut down. When the application is started this class create the graphical user interface, starts the connection to the ShiShiTV Erlang server and provides means for navigating the interface.

ServerConnection.javaThis class provides means for connecting to the ShiShiTV Erlang server by implementing the protocol specified between the server and the mobile device application. The class sends messages provided by a model (see MenuModel and VideoModel) to the server and handles the responses by adding the received data to the model providing the message.

13

VideoModel.javaThis class handles video data provided by the ShiShiTV Erlang server and creates players that can be used for playing back the video data on the mobile device. The class provides two players that are associated with video data from the server (via the ServerConnection object) and play back one player while filling the other one with new data. When the first player has ended, the two players are switched to make the files look like a continuous stream. This data is played back as in 5.5.

The class also handles the sending of feedback from the mobile device to the content provider, this is done by sending a message via the ServerConnection in the same way as when handling the video data.

MenuModel.javaThis class is used in much of the same way as the VideoModel class, in such a way that it sends messages via the ServerConnection class which adds the received data into the MenuModel. The data handled by this class is the list of available channels that are currently playing. This provides data to the list in 5.4.

RecordstoreManager.javaThis class provides an easy means of storing and retrieving data from the record store of mobile devices. It provides convenient functions for storing data and retrieving it again via some filters.

Evaluation and testingThe mobile application has only been tested on a small subset of Nokia series 40 mobile phones and a few Sony Ericsson phones. The mobile application is therefore in many areas untested as the customer was very specific in his wishes to see the application working on the Nokia series 40 mobile phone. The mobile phone supplied to the development group for testing purposes was of the type Nokia 6233. The application has been tested as much as possible, but has been limited by the cost of using packet data. This limitation has of course made it more or less impossible to do lengthy tests of the application using production data.

ConclusionsOverall, the development of the mobile application has been making good progress along the duration of the project and most of the problems and issues that have interfered with the work stem from outside sources. These sources of inconvenience are such as the lack of equipment for facilitating testing (read mobile phones) and issues regarding funding for testing purposes.As for the goal of the mobile development we feel that we have fulfilled the initial wishes of the customer by providing a prototype for playing back ”live” media inside common mobile devices specified by the customer (specifically Nokia 6233 mobile phone). This application has proven to work quite well even though time spent on exception management has been sparse in favour of features.

Known issuesIssues that still remain to be handled within the application are specified below.

1. Transparency of pictures in the welcome screen.2. Application hangs if no Erlang server responds at the URL + port specified in the application.

14

3. There are no notifications given to the user when errors or connection problem occur in the application.

4. The application does not seem to provide full screen video on some Nokia mobile devices.5. There is a IndexOutOfBoundsException thrown if the action button is pressed on

the channel list when no channels are available. This wasn't fixed due to code freeze.

Web Server Application The website is designed to be a community based site where users can announce their broadcasts and viewers can view the content “live”. When a user registers they will then be able to announce the time that they will broadcast their performance. In order to broadcast the user must download the client application (see above).

System Requirements The coming sections covers the system requirement of the website accompanied by screenshots displaying the different features of the web application. For the web site to be displayed correctly, the users system must fulfill these criteria:

Firefox 2.0 or Internet Explorer 6 Adobe Flash 8 (?) or higher Java enabled in the browser

RegisterBefore video can be available on the web site a channel must first be announced by the content provider. The steps for making a video available on the web site will be covered in the coming sections.

To be able to announce channels the content provider must have registered an account on the web site. The registration is reached from a link located on the announce page.

15

The registration procedure contains the following:

6. Fill in an email address7. Fill in a nickname8. Fill in a password9. Confirm the password10. Click Register to finish the registration11. The Reset button empties the fields

Login

The login procedure contains the following:1. Fill in the email that was used during

registration2. Fill in the password corresponding to the

email3. Click the login button

Announce

The procedure for announcing a channel contains the following:

1. Fill in a title for the video2. Fill in a short description of the video3. Choose a category matching the video

content4. Add a start time for the video5. Announce the video by clicking the confirm

booking button

16

Viewing live channelsThe web site provides the viewer with the option to view channels that have been announced by content providers. To view a channel, all the user needs to do is click on the channel link for one of the live stream that are currently playing. The video is then played back in an embedded FlowPlayer by utilizing the Adobe Flash player that needs to be installed on the users computer.

Feature Function1 Navigate to the view page of the site

2 Click the On Air link to update the list of currently playing channels

3 Click a channel link to start the video in the embedded player

4 Click the player screen to pause the video

Sending feedback:Sending feedback to the content provider is available by clicking the images picturing hands showing thumbs up and thumbs down.

Thumbs up – sends positive feedback to the content provider.

Thumbs down– sends negative feedback to the content provider.

Evaluating and testingThe web application is only to be seen as a rough prototype displaying some basic functionality which could be developed further after the project has ended. Therefore the functionality and design of the site has not been the focus of extensive testing. The background functionality (getting files from the Erlang server, getting data from the database and session handling) has been tested and in most cases it works fine. The problem that has been found and that has not been solved (due to time constraints) is that the session blocks the some functionality. It is not yet clear why this happens, and should be investigated further.

Known issuesIssues that still remain to be handled within the application are specified below.

The files seems to fade in when they start playing, which makes the video flash, this might be solvable by changing some parameter for the FlowPlayer or by changing and compiling the FlowPlayer manually.

17

Video sometimes doesn't play in browser. To resolve clear the cache and restart the browser.

ServersThere are three parts to the server side of ShiShiTV, the Erlang server, the web server and the database. These can either be on the same machine, or distributed on a network. This fundamental idea has not changed during the course, only details of which bit does what and how.

WebserverThe ShiShiTV web application is the main platform for presenting live video to the audience. The web site provides a basic interface for both content providers and casual viewers. The site features functionality such as viewing live streams in web browsers, creating accounts for content providers and announcing new channels.

System descriptionThe web server communicates with the Erlang server and a database.

AnnouncingThe database contains account data, video data and also information of what channels are running. The login and registration mechanism on the webserver is typical - login checks if username and password exists in the Account table, and register inserts a row the Account table.

The announce page is used to insert or update a field in the Video table. If a user already has booked at time and wants to change it, an update is done, otherwise an insert is performed. The column ”LiveState” in Video is important and the data in this column states the following:

● value 0: video is booked but not broadcast yet ● value 1: broadcasting right now● value 2: has been broadcast

So when a booking is made the channel will have the LiveState value set to 0.

PlayingOn the page where the actual player is located, a list of all available channels is shown. The data in this list is fetched from the Video table in the database. This list contains the channels that have LiveState set to 1.

In the Video table, the field CID (channel ID) tells us which internal Erlang channel number this channel has. The video broadcast is actually divided into small files, sequences. These sequences have numbers, SN (sequence number), associated with them. After the user has chosen a channel, index.php

18

tells the FlowPlayer to play a video from retrieve_file.php. The GET arguments that are provided are CID and SN. The php file downloads a video and returns it to the flash player. As we do not know which sequence number to start with, we use 0 to get the most current SN in the Erlang buffer. Before the retrieve_file.php script is done, it sets a session variable to store the SN it just download from the Erlang server.

When FlowPlayer has received a file, it is started. Then the Javascript function onBufferFlush is called. It uses AJAX to retrieve the SN saved in the session and therefore knows which SN is playing right now. By increasing SN by 1, we get the SN for the next video sequence. When the video playback is finished, retrieved_file.php is used again, but this time with the new updated SN. Since there is some delay to download a video, the video is broadcast faster then it is viewed, the time gap will become greater and greater. In the end, the player will request a buffer file in the Erlang server that has been removed. To solve the problem temporarily, the video is synchronized with the Erlang server at every 10th video part (that is, set SN to 0 ).

FeedbackThe feedback handling is done by using AJAX with feedhandler.php. When clicking on the thumb up or down images in index.php a Javascript function creates an AJAX object to call feedhandler.php. The GET argument it takes is CID of the playing video. The script updates the Video table in the database, which has two fields, one for applauses and one for boos. One click on the image increases the field with one.

InstallationThe web application is installed on a LAMP server (Linux/Apache/MySQL/PHP) with the software versions specified below:

● Apache 2.0● PHP 5.2● MySQL 5.0

To install the application these steps should be followed:1. Unpack webserver.tar.gz and configure Apache to show webserver/webpage/.2. Enable PHP for the webserver 3. Make all files in the webpage directory readable for everyone (“chmod +r -R ./webpage/“).4. Set the addresses and login information for the Erlang and MySQL server, this is done in

webage/php/config.php . Installation instructions for the above mentioned applications are found in the Erlang Server Product documentation. The path to the PC Sender and mobile application is also set in config.php.

5. To enable the webpage for where users can download the mobile app, configure Apache to use the same DocumentRoot as previous but also add the option “DirectoryIndex index_mobile.php”.

6. Remembers to restart Apache after editing your configuration files.

19

Erlang ServerThe Erlang server consists of 5 modules: connection, pcsender, mobile, webserver and logger. For in depth descriptions of the functions in them look at the comments and code in the corresponding files. For more information regarding the communication between the server and clients look at the protocols page.

Connection, contains the functionality for starting the server, setting up new connections, handling initial requests from clients, channel handling and spawning processes to deal with each connected client.When started the server creates an event handler for login, an ODBC connection and webserver interface, then listens for new tcp connections in a loop. For each new connection a process is spawned to handle it. This process waits for the client to identify it's type. For PC broadcasters and mobile viewers connection still handles them, while a webserver connection is immediately passed on to the webserver module. The other two client types must authenticate themselves. A sender will then be assigned a channel number, a buffer (etch) will be created for it to upload video to, database entries concerning the video (based on user's previously booked time) will be updated and then further handling of the client will be done by the pcsender moduleA logged in viewer (mobile) can get a list of available (currently live) channels and choose to join one, at which point it is dealt with in the mobile module.

Pcsender module is responsible for receiving and storing data sent by the broadcaster. Also communicates with the mobile module, letting it retrieve data and information regarding the channel's video and handling feedback from the viewers.When called by the connection module it immediately waits for traffic from the sender client. This incoming traffic can be either tcp (data and messages) or internal Erlang messaging. Data (video segment) transfers are done by receiving tcp packages until the stated file size is reached. The segment is then buffered in the etch structure assigned to the channel. This “buffer” is circular to conserve space, as the whole thing is kept in the system's memory. The client can also send various messages to announce that it is ending the broadcast or request information regarding the viewers (how many and what feedback they've sent).The rest of the module deals with requests and feedback from the mobile clients, that come in through the mobile module. This allows the retrieval of data and current sequence number, storing feedback and for viewers to join the channel.

Mobile module, handles communication with a mobile viewer. Allows the client to retrieve specified sequences of data, get the current sequence and send feedback to the broadcaster. All these things are coordinated with the pcsender module.When called from the connection module it waits for it's connected mobile client to send it a message containing some request. The exact format of these messages can be seen in the protocols page (and in the code). There are two different request types, one statement and one feedback type. The two requests are GETN and GCSN. GETN is for retrieving video segments and will either result in an error if the requested segment is not available, or in the sending of the file if it is in the buffer. GCSN stands for Get Current Sequence Number, and does exactly that, unless the buffer is empty in which case an error message is sent. The CC statement is sent when the listener leaves the channel. The last message type is FEED and is used for viewer feedback, mainly handled in the pcsender module.

20

Webserver module, provides a channel between the actual webserver and the Erlang server. Also converts files to the format needed by the webpage player. The conversion is done a a separate process to avoid concurrency problems. When a webserver connection starts, a message is sent to the conversion process. The conversion is made and the new file is sent to the webserver. The conversion process then sends back an ok or error message to the webserver connection process which closes down the connection properly.

Logger, small module for echoing and logging server activities.On initialization from the connection module logger opens the designated file in write mode. Then, whenever an event is raised, it appends a line to the log file and outputs it to standard IO (screen).

Evaluating and TestingWe used continuous testing. As soon as the server, or any other part of the project that directly involved the server, was updated we ran tests to see that everything worked as it should.

Installation InstructionsHow to setup the MySql server1. Install the MySql server package.2. Create a database called SHISHI.3. Type "mysql --verbose --user=USER --password=PASS SHISHI < /PATH/TO/server_db" where /PATH/TO/server_db is the path to server_db. USER and PASS is the root account.4. For security reasons, you should create a new account. Make sure you have SELECT, INSERT, UPDATE and DELETE permissions to the SHISHI database.

How to setup the Erlang server1. Install the Erlang package. Install ffmpeg with h263 and amr_nb codec support. Install unixodbc.2. Unpack server.tar.gz3. Create a unixodbc DSN to the SHISHI database.3. Change the ODBC_NAME, DB_USERNAME and DB_PWD definitions in the header of connection.erl and pcsender.erl to match the ODBC DSN.4. Create a directory where the buffer files for the webserver video files are stored temporary. To improve performance, use a virtual RAM disk.5. Change the TMP_DIR definition in the header of webserver.erl to match that directory.6. Run ./compile_server.sh. You will see three warnings in logger.erl. Ignore that.7. Run ./start_server.sh to start the server8. Shut down the server by pressing CTRL-C and then a and Enter.9. Log are written to erlangServer.log.System Architecture

Testing This was one area where we could have done a much better job. We didn't have a formal testing procedure. This was mainly due to the fact that development of the application took longer than expected. The version used in the test below was not in any sense stable.

Testing was done throughout the development of the applications. However we were limited in

21

the amount of testing which could be done on the mobile phone due to the costs associated with transferring data. Since we had no subscription to a mobile provider, when we tested the mobile phone it cost about 15 kr/mb.

We conducted a live test on December 15, 2007 at Pub 19 in Uppsala. The test involved the broadcasting of a band (Moonshine) on the website and a ~2 viewing the broadcast on mobile phones. The test would broadcast for around 20 minutes. The primary goals of the test was to a) test the stability of the pc application b) test the load on both the web and Erlang servers. c) get some response about the usability of both the web and mobile application's usability. The results were less than stellar. We knew there were issues which would affect the results of the test. The issues were as follows a) files tend to loop b)sound is only captures using the microphone on the webcam and not a separate microphone device c) sound quality is poor to non-existence c) since every file is only 3 seconds long which causes a glitch when the files change. We were able to fix the majority of the problems with the exception of the sound issue. This was the only testing done outside of the university's network

System Architecture

22

System Architecture

Future Work Due to time limitations a lot of features we were unable to integrate into the ShiShi TV prototype. Some of these features would greatly improve functionality. One feature would be either a standalone video player or a Flash player. Either of these would need to be able to buffer an incoming video file, this would make for a more seamless playing of video files. Future work on the website would include a search function which would allow user to search for future broadcasts, specific broadcasters, and broadcast by content. The current implementation of the current content and future content is shown in text. These would be replaced with a picture the broadcaster could upload. A redesign of web interface is also necessary to improve the users experience. Another feature which is need to enhance the website is a resign of the website.

Since file conversion is done on the client we are sending a degraded file to the server thus making conversions of lesser quality. In order to improve the quality the files should ideally be captured in 3gp. Another solution would be to stream the video directly to the server then any conversions which are needed should be done there Search functions should also be built into the website and mobile application.

Erlang

● Conversion of 3gp files should be done in memory instead of through files

● Graceful shutdown

● Gather statistics

PC Client

● Redesign interface

● Capture video in 3gp

● Port to Linux and Mac

Mobile Fix remaining issues specified in 7.2 Implement features as search functionality The application should be seen as a prototype and therefore it should be submitted to extensive

usability testing for a better user experience. Investigate if frameworks like J2ME Polish could ease porting to mobile phones by different

makers.

Web Server

● Administration interface for adding and deleting accounts

● Redesign to improve the general look and feel

● Write own Flash player which can buffer files

23

ReferencesHenrik Kniberg. Scrum and XP from the Trenches: How we do Scrum C4 Media Inc. 2004

. Wikipedia: Erlang (the Programming Language).

<http://en.wikipedia.org/wiki/Erlang_%28programming_language%29>

Appendix A

JavadocThe API of the ShiShi TV mobile application is provided as an appendix in both HTML and PDF format.

mobile application API.pdf

Appendix B: Protocols

Erlang server - mobile viewer

Packet information

Server->mobile data packet Type FileSize SequenceNumber Data-----------------------------------------1 byte | 4 bytes | 4 bytes | ...-----------------------------------------Type = 0

ControlMessage contains the buffernumber.

Server->mobile message packet Type Message Data-------------------------------------1 byte | 4 bytes | 4 bytes |-------------------------------------

Type = 1

24

Server->mobile channel list package Type FileSize SequenceNumber Data-----------------------------------------1 byte | 4 bytes | 4 bytes | ...-----------------------------------------Type = 0

ControlMessage = 0.

Data = channel list

Mobile->Server message packet Type Message Data-------------------------------------1 byte | 4 bytes | 60 bytes |-------------------------------------

Type = 1

Use Cases

Open ConnectionMobile Server--------------------------------------Send request -> <- Respond "OK" or <- Respond with a connection error

Open ChannelMobile Server--------------------------------------Send request -> <- Respond "OK" or <- Respond with a connection error

Open Channel with user and passwordMobile Server--------------------------------------Send request -> <- Request username and password"Send info -> <- "OK" or error message

25

Close ChannelSend request -> <- "OK"

Get Stream PieceMobile Server--------------------------------------GetPiece(StreamID) -> <- Respond with "FileSize + Data" or <- Respond "EOS" (End Of Stream) or <- Respond with a stream failure errorCloseConnection ->

Get ListMobile Server--------------------------------------GetList -> <- Responds with a String that contains a list separated by ";"

Example: "Channel 0";0;"Channel 1";1;"Channel 2";20;"Channel 3";3; If channels 0-3 are up.

Get Current Sequence NumberMobile Server--------------------------------------GetCurrentSequenceNumber() -> <- "SN" + data (which contains the sequence number) Responds with a number that represents the current sequence number of the opened stream.

MessagesAll messages should be in UPPER CASE! We have two different types of get, one that has a string following it ("GETS") and one that got a integer ("GETN"). Type Message Command/Information GetList -> "GETS"GetPiece -> "GETN" "Requested buffer number" 4 byte INTEGEROpenConnection -> "MR" returns "OK" from server if connection is openedOpenChannel -> "OC" Channel ID 4 byte INTEGERCloseChannel -> "CC"SendPassword -> "AUTH" "User'\0'Password'\0"

26

SendError -> "ERR" "Error number" 4 byte INTEGERSendEOS -> "EOS" (End Of Stream)OK Mesage -> "OK"Sequencenumber -> "SN"GetCurrentSequenceNumber -> "GCSN"Send Feedback -> "FEED" "Response" 4 byte INTEGER

Error Messages We have decided on the following encoding of errors.

• 0000-0999 Connection related • 1000-1999 Stream related • 2000-2999 User related

Current Messages 0001. Connection refused0002. Too many connections0003. Sender disconnected.0004. Unknown command1000. No such stream1001. No such buffer in stream2000. Bad login

Erlang server – PC Sender

Packet information

PC Sender -> server data packet Type FileSize ControlMessage Data-------------------------------------------| 1 byte | 4 bytes | 1 byte | 4 bytes | ...-------------------------------------------Type = 0Control Message 1 byte = Media_ID Media_ID 0 = 3GP Media_ID 1 = MP4 (or whatever format we choose for the web, maybe .flv?) Note: Media_ID is ignored by server right now.ControlMessage contains the sequence number of stream sent.

PC Sender -> server message packet Type Message Data--------------------------------------| 1 byte | 4 byte | 60 bytes |--------------------------------------Type = 1

27

Server -> PC Sender message packet Type Message Data--------------------------------------| 1 byte | 4 byte | 4 bytes |--------------------------------------Type = 1

Use Cases

Open Connection PC Server--------------------------------------Send request -> <- Respond "OK" or <- Respond with a connection error

Open Connection with user and password PC Server--------------------------------------Send request -> <- Request user and passwordSend info -> <- "OK" or error message

Send Stream Piece PC Server--------------------------------------Send "FileSize + Control + Data" -> <- Respond with "OK" orSend stream failure error -> <- Respond ... orSend "EOS" (End Of Stream) -> <- Respond ...CloseConnection ->

Messages All messages should be in UPPER CASE! We have two different types of get, one that has a string

28

following it ("GETS") and one that got a integer ("GETN"). Type Message Command/Information OpenConnection -> "PS" returns "OK" from server if connection is openedSendPassword -> "AUTH" 'User'\0'Password'\0SendError -> "ERR" "Error number" 4 byte INTEGER.SendEOS -> "EOS" (End Of Stream)OK -> "OK"Feedback Req. -> "FDA" request total number of applauses -> "FDB" request total number of boos (negative feedback)Feedback Resp. -> "FDA" or "FDB"#listeners -> "NOS" request number of listeners (currently only mobile ones)Start sending -> "SS"

Error Messages We have decided on the following encoding of errors.

• 0000-0999 Connection related • 1000-1999 Stream related • 2000-2999 User related • 3000-3999 Client related

Current Messages 0001. Connection refused0002. Too many connections1000. No such stream1001. No such buffer in stream1002. Pending stream (no stream available yet, but it's incoming to the server)1003. Uploaded data is already in buffer (the same SN already exists)2000. Bad login2001. User has not booked time

Internal client error messages 3000. Unable to connect to server3001. Unable to send message to server (lost connection)3002. Unable to send file to server (lost connection)

Erlang server - webserver

Packet information

Server->webserver data packet Type FileSize SequenceNumber Data-----------------------------------------1 byte | 4 bytes | 4 bytes | ...-----------------------------------------Type = 0

29

SequenceNumber = SN of the file.

Server-> webserver message packet Type Message Data-------------------------------------1 byte | 4 bytes | 4 bytes |-------------------------------------

Type = 1

Webserver->Server message packet Type Message Data-------------------------------------1 byte | 4 bytes | 60 bytes |-------------------------------------

Type = 1

Messages All messages should be in UPPER CASE! We have two different types of get, one that has a string following it ("GETS") and one that got a integer ("GETN"). Type Message Command/InformationGetPiece -> "GETN" "Requested buffer number" 4 byte INTEGER. If RBN is 0, the current file from buffer is returned. If the return data is not a data packet but has Type 2 or 3, the requested CID or SN didn't existed.OpenConnection -> "WS" returns "OK" from server if connection is openedOK -> "OK"

30

31

32

33