Embed Size (px)
Citation preview
Master Thesis
Electrical Engineering
July 2012
Evaluation of Video Quality of Experience
using EvalVid
Venkata Ramana Jonnalagadda
Vineela Musti
School of Computing Blekinge Institute of Technology 371 79 Karlskrona Sweden
2
Internet : www.bth.se/com Phone : +46 455 38 50 00 Fax : +46 455 38 50 57
Contact Information:
Authors:
1. Venkataramana Jonnalagadda Address: Flat no. 206, Mytri Vihar, Guptha Gardens, Ramanthapur, Hyderabad- 500013 Andhra Pradesh, India E-mail: [email protected], [email protected]
2. Vineela Musti Address: HIG-89, Bharat Nagar Colony, Hyderabad-500018 Andhra Pradesh, India E-mail: [email protected], [email protected]
University Advisor: Prof. Markus Fiedler School of Computing
School of Computing Blekinge Institute of Technology 371 79 Karlskrona Sweden
This thesis is submitted to the School of Computing at Blekinge Institute of Technology in partial fulfillment of the requirements for the degree of Master of Science in Electrical Engineering. The thesis is equivalent to 20 weeks of full time studies.
3
ACKNOWLEDGMENT
We would like to express our sincere gratitude to our Supervisor Dr. Markus
Fiedler for his sagacious guidance and scholarly advice which enabled us to complete the thesis. He was very supportive and encouraging throughout the process and we seem to be very honored to work under his wonderful guidance. We express our deep gratitude to Tahir Nawaz Minhas for providing us timely and needful help through the thesis. We humbly thank the examiner Dr. Patrik Arlos for the encouragement and support.
We would like to thank god, our parents and friends for giving us constant support in every endeavor of us. We would like to specially thank our friends Ramachandran
Chakravadhanula and Ajay Surapaneni for being with us from the starting stage to the completion of our thesis. They were really supportive at all the times sharing their valuable thoughts and ideas with us. Without their constant support our thesis would not have been successful. We would like to express our gratitude towards everyone who contributed their precious time and effort to help us directly or indirectly, without whom it would not have been possible for us to complete the thesis.
Venkataramana Jonnalagadda Vineela Musti
4
ABSTRACT
Videos play a very important role in today’s communication and
entertainment world. Many services are being developed based on the
concept of video like video streaming, video calling etc. The increase in
demand for such services made the service providers evince interest in
investigating, monitoring and maintaining the quality that they provide
for such services. Also it is a crucial point that a bad quality video will
degrade the Quality of Experience felt by the user. In order to improve
Quality of Experience, video quality evaluation is necessary. For this
purpose many tools to evaluate the quality of a video have been
designed. In this thesis EvalVid is evaluated, which is an open source
video quality evaluation tool. We have evaluated a set of videos using
this tool to compare the performance of quality evaluation metrics like
Peak Signal to Noise Ratio and Structural Similarity Index Metric.
Affirmation of the results is done using Matlab, by calculating PSNR
and SSIM for the same set of videos which have been used in EvalVid.
These two tools calculate the metrics in two different ways, which is
revealed by the observed differences in the results. A set of particular
observations allow further conclusion that EvalVid is less credible than
Matlab for evaluating quality of videos.
Keywords: Evaluation, Peak Signal to Noise Ratio,
Structural Similarity Index Metric, Video quality,
Quality of Experience (QoE).
5
Contents
ACKNOWLEDGMENT ................................................................................................................. 3
ABSTRACT .................................................................................................................................... 4
LIST OF ACRONYMS................................................................................................................... 7
LIST OF FIGURES ........................................................................................................................ 8
LIST OF TABLES .......................................................................................................................... 9
1 INTRODUCTION ................................................................................................................ 10
1.1 BACKGROUND ................................................................................................................ 10 1.2 MOTIVATION .................................................................................................................. 10 1.3 OBJECTIVES .................................................................................................................... 11 1.4 RESEARCH QUESTIONS.................................................................................................... 11 1.5 RESEARCH METHODOLOGY ............................................................................................. 12 1.6 STRUCTURE .................................................................................................................... 12
2 TECHNICAL BACKGROUND ........................................................................................... 13
2.1 INTRODUCTION TO EVALVID ........................................................................................... 13 2.2 PSNR............................................................................................................................. 13 2.3 SSIM ............................................................................................................................. 15 2.4 COMPARISON BETWEEN PSNR AND SSIM ....................................................................... 17
3 METHOD AND INSTALLATION ...................................................................................... 18
3.1 PREREQUISITES FOR USING EVALVID ............................................................................... 18 3.2 INSTALLATION OF EVALVID ............................................................................................ 18 3.3 BASIC REQUIREMENTS OF THE TOOL ................................................................................ 19
4 IMPLEMENTATION .......................................................................................................... 20
4.1 OVERVIEW ..................................................................................................................... 20 4.2 EXPERIMENTAL SETUP .................................................................................................... 20
EvalVid - As an evaluation tool: ............................................................................................. 20 4.3 VIDEOS CHOSEN FOR THE EXPERIMENT ............................................................................ 20
4.3.1 Foreman .................................................................................................................... 21 4.3.2 Hall monitor .............................................................................................................. 21 4.3.3 News .......................................................................................................................... 22
4.4 HOW FREEZES ARE INTRODUCED INTO THE VIDEOS ........................................................... 22 4.5 EXPERIMENTAL PROCEDURE ........................................................................................... 23 4.6 YUV FORMAT: ............................................................................................................... 24 4.7 DATA PROCESSING .......................................................................................................... 25
4.7.1 PSNR: The command to be given to calculate PSNR; ......................................................................... 26 4.7.2 SSIM: The command to be given to calculate SSIM; .......................................................................... 26
6
5 RESULTS, DISCUSSION AND VALIDATION ................................................................ 28
5.1 PSNR: ........................................................................................................................... 28 5.2 SSIM: ............................................................................................................................ 29 5.3 VALIDATION:.................................................................................................................. 30
5.3.1 Results of Matlab: ...................................................................................................... 30 5.3.2 Matlab vs EvalVid conditional averages:.................................................................... 31
6 CONCLUSION AND FUTURE WORK.............................................................................. 32
6.1 CONCLUSION .................................................................................................................. 32 6.2 ANSWERS TO THE RESEARCH QUESTIONS: ....................................................................... 32 6.3 FUTURE WORK ................................................................................................................ 34
7 REFERENCES ..................................................................................................................... 35
8 APPENDIX ........................................................................................................................... 37
7
LIST OF ACRONYMS
CIF Common Intermediate Format DMB Digital Multimedia Broadcasting DVB-H Digital Video Broadcasting - Handheld FPS Frames Per Second FR Full Reference GPAC Isomedia files for Mac Operating System X HVS Human Visual System IPTV Internet Protocol Television ISO – C International Standard Organization for
C- Programming ITU-T International Telecommunication Union
– Telecommunication MSE Mean Square Error MSSIM Mean Structural Similarity Index Metric NTSC National Television System Committee OS Operating System P2P Peer to Peer PAL Phase Alternating Line PEVQ Perceptual Evaluation of Video Quality PSNR Peak Signal to Noise Ratio QCIF Quarter CIF QoE Quality of Experience QoS Quality of Service SECAM Sequential Couleur Avec Meoire or Sequential Color
with Memory SSIM Structural Similarity Index Metric SCIF Sub CIF UMTS Universal Mobile Telecommunication Systems VoD Video on Demand
8
LIST OF FIGURES
Figure 1: Illustrative figure of EvalVid ................................................................... 13 Figure 2: GPAC location ........................................................................................ 19 Figure 3: Screenshot of the Foreman video ............................................................. 21 Figure 4: Screenshot of the Hall Monitor video ....................................................... 21 Figure 5: Screenshot of the News video .................................................................. 22 Figure 6: Example .................................................................................................. 23 Figure 7: Screenshot of the video when we tried to change the codec type .............. 24 Figure 8: Comparison of tool average values with conditional average values ......... 25
9
LIST OF TABLES
Table 1: Video specifications .................................................................................. 20 Table 2: Popular CIF formats used .......................................................................... 21 Table 3: PSNR: Tool and conditional average values in comparison ....................... 28 Table 4: SSIM: Tool and conditional average value comparison ............................. 29 Table 6: SSIM: Tool and conditional average value comparison for Matlab ............ 30 Table 8: SSIM: EvalVid vs Matlab conditional averages ......................................... 31 Table 7: Foreman video - PSNR values for the distorted frames .............................. 37 Table 8: Foreman video - SSIM values for the distorted frames .............................. 37 Table 9: Hall Monitor video - PSNR values for the distorted frames ....................... 38 Table 10: Hall monitor video - SSIM values for the distorted frames ...................... 38 Table 11: News video - PSNR values for the distorted frames................................. 39 Table 12: News video - SSIM values for the distorted frames ................................. 39 Table 13: Foreman video - Matlab PSNR values for the distorted frames ................ 40 Table 14: Foreman video - Matlab SSIM values for the distorted frames ................ 40 Table 15: Hall Monitor video - Matlab PSNR values for the distorted frames ......... 41 Table 16: Hall Monitor video - Matlab SSIM values for the distorted frames .......... 41 Table 17: News video - Matlab PSNR values for the distorted frames ..................... 42 Table 18: News video - Matlab SSIM values for the distorted frames ..................... 42
10
1 INTRODUCTION
1.1 Background
In present scenario there are many mobile channels like UMTS (Universal Mobile Telecommunication System), DVB-H (Digital Video Broadcasting- Hand Held) or DMB (Digital Multimedia Broadcasting). These are now used for video transmission along with audio and rich media content. The mobile channels have a disadvantage of varying the capacity with time in spite of having many advantages. The upcoming online video services such as Video on Demand (VoD), IPTV(Internet Protocol Television) and Peer to Peer (P2P) Video streaming are making the service providers gain more interest in monitoring and measuring video quality in order to provide user satisfied level of video quality which is also known as Quality of Experience (QoE) [1]. Video has many characteristics like frame rate, display resolution, bit rate (an indirect measure of quality), aspect ratio, etc to specify its quality. Each of them is equally important to measure the quality of the video that is degraded when passed through a mobile network. These systems introduce distortions (mainly by affecting the completeness and timeliness of the packet stream) in the video signal which degrades the video quality, which in turn makes the video quality evaluation an important issue.
The area of research covers different domains that include video streaming,
evaluation of video quality; Quality of Experience (QoE) and validation of the results, each of them have their own importance. Compared to data and voice applications, video applications need good internet connectivity with a greater bandwidth than required by the voice applications [5]. In video streaming, if the video freezes at any point user cannot follow the contents of the video anymore. In live streaming, due to the timing constraints, retransmission is generally of no use, unless the retransmission can be finished before the corresponding piece of content is scheduled to play. Typically, if a packet is lost, the process of transmission must be carried out continuously without waiting for the lost packets or retransmitting the lost packets, which also helps in conserving the bandwidth [3][4]. In our scenario the main problem relates to video transmission or communication over mobile networks. The poor quality of video transmission is caused due to the network congestion which leads to delay, delay variations or packet loss [5]. In this study, the point of discussion is the quality of the video degraded due to the freezes appeared in the video.
1.2 Motivation A characteristic of a video which is a formal or an informal measure of
degradation of quality with reference to some standard video when processed or transmitted is known as the video quality [1]. The transmission or processing system is capable of introducing some kind of distortion like freeze in the video signal. Hence, the need for video quality evaluation arises.
11
In earlier days to watch a video it should be downloaded to a hard disk or to some kind of memory device and then played using a player. When a video is to be watched for several times it might be considered as a better option. But for one - time viewers it is a waste of time, memory and energy. To avoid this, the concept of streaming was introduced. Streaming is basically of two types, one of which is live streaming which is a onetime streaming, streamed simultaneously when it is recorded. But coming to the offline streaming, say YouTube for instance the video is streamed each and every time the video is to be watched. In both cases, the delivery of the video may be influenced by the network which causes delay, packet loss or delay jitter, etc... This makes video to freeze while streaming.
The main idea of this study is to evaluate the quality of a video when it is
impaired with the disturbances caused due to the mobile networks. Some distortions may take place in the video at the time of transmission or processing. Here comes the need for evaluation of video quality and for this the comparison of quality between the original and the transmitted video that is distorted in the process with the use of some quality evaluation metric using a video quality evaluation tool. EvalVid evaluates a video with the help of the quality evaluation metrics Peak Signal to Noise Ratio (PSNR), Structural Similarity Index (SSIM) which is useful in the study of comparing the performance of PSNR and SSIM in evaluating a video. Furthermore, EvalVid compares the videos frame by frame which help in locating the position of distortion occurred [5].
1.3 Objectives
1. Creating EvalVid Environment – Installation of EvalVid and incorporation of video traces into the network.
2. Quality evaluation of video using the EvalVid tool. 3. Analysis of the results and the performance of the tool in evaluating the
video quality.
1.4 Research Questions
1. What is the effect of impairment in a video caused by the mobile networks on the QoE?
2. How is the evaluation performed in EvalVid? How valid are the results?
3. Why is SSIM a better comparison metric than PSNR?
12
1.5 Research Methodology
Initially a detailed literature survey of EvalVid regarding evaluation metrics (PSNR, SSIM), how videos get distorted, evaluation of videos, video streaming and QoE gives us the knowledge and helps us to perform the experiment. Later the experiment is performed by installing the tool, collecting and evaluating the videos, and acquisition of results, which are then analysed to answer the proposed research questions.
The methodology we have followed to answer the above research questions
are as follows:
RQ 1: We have made a detailed literature survey on how the mobile networks perform, how mobile networks influence a video’s quality, how these disturbances reflect the end user’s experience with the quality of the video.
RQ 2: EvalVid is installed according to the documentation and tested, and the videos are prepared such as they can be used with EvalVid. Totally three videos have been used for the experiment. Each video is compared with the corresponding distorted videos one by one. The results are tabulated and then consolidated as conditional and tool averages. These are later interpreted.
RQ 3: From the literature survey on PSNR, SSIM and from the experiments performed the conclusion has been drawn. To validate the results that have been obtained we compared the results with Tahir Nawaz Minhas’ results [31]. In addition to this we have repeated the experiment for all the videos using Matlab and compared both EvalVid and Matlab results.
1.6 Structure The chapter introduction gives a basic idea about the thesis. The next
chapter explains about the technical background which gives a detailed explanation about parameters considered and about the tool, followed by chapter 3 discussing about the prerequisites and basic requirements of the tool. Chapter 4 focuses on the experimental setup, videos chosen for the experiment and how data is processed from tool in order to calculate the results. Chapter 5 presents the results obtained. The discussion in Chapter 6 comprises of the interpretations of the results obtained and their validation. Chapter 7 concludes with the conclusion and the future work about the research.
13
2 TECHNICAL BACKGROUND
2.1 Introduction to EvalVid
EvalVid is a framework and a toolkit for unified assessment of the quality of video transmission. EvalVid has a modular structure, making it possible to exchange at user’s discretion both the underlying transmission system as well as the codecs, so it is applicable to any kind of coding scheme, and might be used both in real experimental setups and simulation experiments. The tools are implemented in pure ISO-C for maximum portability. All interactions with the network are done via two trace files. So it is very easy to integrate EvalVid in any environment [5].
Figure 1: Illustrative figure of EvalVid
2.2 PSNR PSNR is abbreviated from the phrase, Peak Signal to Noise Ratio. PSNR is
a mathematically defined metric. It is outlined as the ratio between a signal’s
maximum power and the power of corrupting noise that affects the signal. As its range varies dynamically it is expressed in terms of logarithmic decibels. PSNR can be used for rough estimation of relative qualities when video content and type of distortions remain the same, only the level of distortion is changed [12]. Nonetheless, dependent on content and disturbance, the correlation between subjective quality [13] and PSNR may become very weak. Hence PSNR cannot be a reliable method for assessing the video quality across different video contents. PSNR is still used as a quality metric because of its low complexity and is considered to be a reference benchmark for developing perceptual video quality metrics, though it is considered as a fidelity metric [14].
PSNR is widely used in evaluating codec performance, video codec optimisation or as a comparison method between different video codecs, despite the fact that objective perceptual quality metrics have been shown to outperform PSNR in predicting subjective video quality [14].
PSNR is formulated by setting the Mean Square Error (MSE) in relation to
the maximum possible value of the luminance (28 – 1 = 255 for a typical 8-bit value) as
14
Where is the original signal at pixel , is the reconstructed signal, and is the picture size.
MSE is the cumulative squared error between the original and the distorted videos.
dB
The result is a single number in decibels, ranging from 30 to 40 for a medium to high quality video [18].
From the above equation it can be stated that MSE and PSNR are inversely
proportional. MSE is low and PSNR is high for a good quality video, and vice versa, MSE is high, PSNR is low for a bad quality video.
For example if both the videos are exactly the same i.e.
If MSE = 0
This will result in as infinity and thus logarithm of infinity which ultimately states that PSNR as infinity for a best quality video.
The larger the difference between f(i,j) and F(i,j) becomes, the larger MSE,
the smaller PSNR becomes according to the formula. Say for instance, in the extreme case, inverted Black and White picture matrix, which would in the end yield PSNR = 0 dB.
If MSE = 65025 implies that
This states that the video is of bad quality.
The objective video quality assessment goal is to obtain a MOS, not from
individuals’ opinion, but from measurable characteristics of the multimedia signals or from network measurements, such as average packet loss rate or sustainable bit rate. Peak Signal-to-Noise Ratio (PSNR), and the related quantity MSE, are the most widely and well-known used Full Reference (FR) quality metrics [4] used for image and video quality assessment. Nevertheless, PSNR has a limited range of validity as it measures essentially the sample-wise distortion between a reference signal and its “impaired” version (has more pixels that differ from the original version) [7].
Nevertheless, PSNR and MSE are widely used because of their simplicity
in calculation, its physical meaning is clear, and they are mathematically convenient
15
in the context of optimisation. They are not very well matched to perceived visual quality [16].
2.3 SSIM
In the last three decades, a great deal of effort has gone into the development of quality assessment methods that take advantage of known characteristics of the Human Visual System (HVS). The majority of the proposed perceptual quality assessment models have followed a strategy of modifying the MSE measure so that errors are penalized in accordance with their visibility [16].
An objective quality metric can play a variety of roles. For example, a video server can examine the quality of video being transmitted in order to control and allocate streaming resources. From the phrase Structural Similarity Index Metric (SSIM) it is clear that it is a metric which measures the similarity between two images or videos. It is a perceptual distortion metric. To overcome inconsistencies as compared to human eye perception, which are often observed in traditional methods such as PSNR, SSIM has been proposed. It is a measure between two windows of same size. It is a decimal value ranging from -1 to 1. 1 is only possible when the two images or videos are identical. As video is a set of still images, we are concerned about both images and videos as well.
SSIM follows a different approach than that for video quality assessment,
as it is based on the idea that the human visual perception is highly adapted for extracting structural information from a scene. Structures of the objects in a scene are independent of the influence of the luminance and contrast. Thus, luminance, contrast and structure are the separated components that are measured and compared. SSIM computes the quality of a distorted image by comparing the correlations in luminance, contrast, and structure, locally between the reference and distorted images and averaging these quantities over the entire image. The design of SSIM is inspired by the functioning of the HVS [15].
The SSIM indexing metric uses an 8 x 8 pixel sliding window approach, where the sliding window moves pixel-by-pixel from the top-left corner to the bottom-right corner of the image. The overall quality value is defined as the average of the quality map, also known as the Mean SSIM (MSSIM) index. A straightforward extension of the SSIM metric for video data by simply averaging its value over entire frames and over time does not take into account that neither all regions in a visual scene nor all frames are equally important in terms of human visual perception. Therefore, the MSSIM averaging computed for each video frame should be modified into weighted average, given the different importance of the various frame regions to the human observers. In addition, the same principle should be applied over time [7].
16
The most general form of the metric that is used to measure structural similarity between two signal vectors x and y is
where,
, , y , ,
compares the luminance of the signals,
compares the contrast of the signals,
measures the structural correlation of signals,
, are the sample means of x and y respectively, , are the sample standard deviations of x and y respectively, is the cross covariance between x and y, , , are the constants that are used to stabilize the metric, , , are the parameters that are used to adjust the relative
importance of the three components. As , , should always be greater than one, hence the product should be
one, which explains the condition given below
To obtain the above condition; the means of the two videos must equal. the standard deviations of both the videos and their cross
covariance must be the same. For a bad quality video SSIM value must be -1, which indicates a strong
negative correlation and thus a strong deviation between the frame(s) of interest and the original one(s).
Though the above formula to calculate SSIM is more complicated than that of
MSE, it remains analytically tracable. These features make the SSIM index attractive to work with.
17
2.4 Comparison between PSNR and SSIM
Both PSNR and SSIM are objective quality metrics. The comparative study of these metrics is helpful in order to know which one of these two is to be chosen for a particular situation. The following are few comparisons that are noticeable.
Computational complexity of PSNR is very low, easy to understand and has clear physical meaning whereas SSIM is highly complex to compute and it is more difficult to understand, as it requires clear understanding of correlations.
PSNR is mathematically convenient from an optimization point of view;
because it uses logarithm in calculating its value, which is not the contrary this is not the case with SSIM.
SSIM overcomes inconsistencies with human eye perception, which are often missed by traditional methods such as PSNR and MSE.
PSNR or MSE measures the average squared error between the original and optimized video. They indicate how similar the signal to its original one. SSIM measures structural similarity based on correlations, and is thus a more advanced measurement index than PSNR.
SSIM also measures luminance and contrast differences between the two images. PSNR does not consider the chrominance component in an explicit way which is very useful for human perception but only considers luminance component.
The main aspect behind SSIM is that the HVS is highly specialized in extracting structural information from the viewing field but is not specialized in extracting errors.
18
3 METHOD AND INSTALLATION
3.1 Prerequisites for using EvalVid
When a video is transmitted or processed through a network, the network can freeze videos and cause many other issues such as packet loss, delay, delay variation, jerks, etc. Freeze in a video is generally caused due to repetition of a frame or a set of frames. This repetition of frames could last for a certain interval of time without changing the number of frames and video length. However, the rest of the issues can change the number of frames and/or the video length [27]. The network usually does not replicate the frames. The phenomenon of replication is done by the player to compensate for missing/late frames. EvalVid can only compare and evaluate videos with equal number of frames. For this reason, the tool can’t evaluate videos with issues other than freeze. Hence, we chose videos with freeze but not with any other issue. The deviations also do occur in the picture matrix, e.g. noise. In addition, freeze in a video is a major problem in the case of video streaming or many other applications and is even difficult to find where the freeze is actually occurred just by viewing the video.
3.2 Installation of EvalVid
A Linux OS is required, preferably Ubuntu 10.04 or 10.10 (as Ubuntu is user friendly and the latest one).
Download gpac files for Ubuntu. GPAC files are the isomedia files for Mac OS x. These gpac files provide an isomedia files for Linux (Ubuntu) also.
Download the EvalVid and EvalVid binaries. To complete the installation the following steps are required:
o Download EvalVid 2.7 and EvalVid 2.7 binaries from the following link: http://www.tkn.tu-berlin.de/research/evalvid/
o Download GPAC files for Linux (Here the Operating System we are using is Ubuntu 10.04).
o The GPAC files contain the required header files to complete the installation of the evaluator. In order to have a proper installation add the extracted GPAC folder to the Evalvid/2.7 folder.
o Add the include folder of GPAC to Root/usr/include
19
Figure 2: GPAC location
o Now open the terminal screen and login to the root. Open the directory 2.7 using the command:
cd <Evalvid folder located directory>/Evalvid/2.7
o After logging into the root through the command above, in order to complete the header file installation add the following commands in the terminal screen; sudo apt-get install –reinstall libgpac-dev /*Installs the gpac
files */ sudo apt-get update /* Updates gpac files */ ranlib /usr/lib/libgpac_static.a /*generates index to
libgpac_static.a, where static.a is an archive file*/ sudo ifconfig –v /*displays the status of the error conditions */
o After running the above commands. Go to the def.h file and change the #elif command to #else.
o Now come back to terminal screen and give the command make –f
Makefile which compiles and runs the make file so as to install EvalVid.
3.3 Basic requirements of the tool
The videos taken must be of the same size (in terms of picture matrix and number of frames) so that the evaluation can be done properly. An important issue to be remembered is the number of frames is dependent on the size of the video traces.
The videos should be of the same format i.e. both the reference and the distorted video should of raw format (EvalVid takes the video only in raw format).
Both the video files taken should be of the same codec.
20
4 IMPLEMENTATION
4.1 Overview This chapter describes all about the experimental setup of our work, in
particular how EvalVid works and why we chose this for our experiment. Then we describe the videos that are to be opted for our experiment which are compatible with the tool. We also explain how the freezes have been introduced into the videos. Finally, we discuss the problems encountered with the videos and how these are rectified.
4.2 Experimental Setup
EvalVid - As an evaluation tool:
EvalVid is the tool which evaluates a video using both PSNR and SSIM as comparison metrics. For a consistent comparison of PSNR and SSIM values, it is preferable to use values obtained from a single tool rather than values obtained from two different tools. And also in EvalVid the comparison is done frame by frame which is helpful to point out where exactly the distortion is in the video, rather than just considering the average quality of video as being low [5].
4.3 Videos chosen for the experiment
In EvalVid it is required to use YUV/raw form videos which are suggested
by ITU-T [27] and for this we chose Foreman, Hall monitor and News. These videos are acquired from a generally used repository for video quality evaluation studies. These three videos used for the experiment are downloaded from [20]. Following are the specifications of the videos:
Table 1: Video specifications
Video File Foreman Hall Monitor News
Size 18.2 MB 18.2 MB 18.2 MB Length (in
seconds)
10 10 10
Display
Resolution
QCIF (176×144) QCIF (176×144) QCIF (176×144)
Codec 24 bits RGB (RV24)
24 bits RGB (RV24)
24 bits RGB (RV24)
Frame rate (fps) 25 25 25
When the quality of video is to be evaluated the resolution of the video plays a vital role. There are some video resolutions like CIF, QCIF, SQCIF, 4CIF etc. The popular sets of formats are listed below:
21
Table 2: Popular CIF formats used [26] Format Luminance resolution
(Horizontal x Vertical)
Bits per frame (4:2:0, eight bits per
sample) SQCIF 128 × 96 147456 QCIF 176 × 144 304128 CIF 352 × 288 1216512 4CIF 704 × 576 4866048
We are using QCIF standard video resolution proposed by ITU-T [21][1].
4.3.1 Foreman Figure 3 is the screenshot of the foreman video. In this video the camera
first focuses on the foreman and then passes to the building.
Figure 3: Screenshot of the Foreman video
4.3.2 Hall monitor Figure 4 shows the screenshot of the hall monitor video. This video shows
the visual of two men walking in the corridor and greeting each other at a certain point of time while carrying their belongings.
Figure 4: Screenshot of the Hall Monitor video
22
4.3.3 News Figure 5 is a screenshot of the news video. This video shows two news
readers reading the news. In the background of the news readers, there is a ballet dancer’s video running.
Figure 5: Screenshot of the News video
4.4 How freezes are introduced into the videos Apparent motion discontinuity in the videos is known as frame freezing.
This phenomenon is a common temporal degradation in the video transmission via mobile channels. Frame freezing makes the end user perceive a fluidity break of visual information having an impact on his/her quality assessment of the delivered video sequence.
We have 10 distorted videos in our experiment.
- First video: replace frames 2-25 by frame 1 - Second video: replace frames 26-49 by frame 25 - Third video: replace frames 51-99 by frame 50 and so on.
All videos consist of 251 frames, serving as basis for the full-reference comparisons needed for PSNR and SSIM.
23
Figure 6: Example
The above figure illustrates about the videos and their corresponding
position of freezes in the video. Each video is of 10 seconds length, which is denoted by the corresponding numbers say 1, 2 ... 10. The first stack is the original video and the rest 10 stacks are the distorted video as shown in the figure. In these stacks the black part represents the distorted / frozen and the grey part denotes the undistorted or which doesn’t have any freeze.
When the first distorted / frozen video is compared with the original video
the difference between these two videos confines itself at the frozen part, e.g. during the first second, while the remaining parts are same as for the original video. Correspondingly the same is with the rest of the videos.
4.5 Experimental Procedure The experiment of the evaluation of videos is started by considering a
reference video with respective 10 distorted versions as described above in figure 6. Each of these ten distorted videos, having freezes at ten different one-second intervals during the video, is evaluated against the reference video.
Number of frames in the original and the distorted videos must be the same,
and the same is observed in information obtained from the VLC player.
24
Figure 7: Screenshot of the video when we tried to change the codec type
Figure 7 illustrates the need to select the matching codec type, as otherwise, the video is completely destroyed.
4.6 YUV Format: The YUV model defines a color space in terms of one luma (brightness) and
two chrominance (color) components. The YUV color model is used in the PAL, NTSC, and SECAM composite color video standards. Previous black-and-white systems used only luma (Y) information and color information (U and V) was added so that a black-and-white receiver would still be able to display a color picture as a normal black and white picture [24].
The advantage of YUV is that the sampling rate of a color channel can be lower than that of the Y channel without reducing the visual quality significantly. There is a notation used to describe the sampling frequency ratio of the U and V channels as compared to the Y channel, which is called A:B:C representation [25].
The most commonly used sampling ratios are as follows: 4:4:4 indicates no down-sampling; 4:2:2 does down-sampling in a 2:1 level, no vertical down-sampling. For every two U or V samples, each scan line contains four Y samples. 4:2:0 indicates down-sampling in a 2:1 level (here V component is missing), 2:1 vertical scan lines containing four Y samples. Surface definition: common 8-bit YUV format can be divided into several categories: 4:4:4 format, 32 bits per pixel; 4:2:2 format, 16 bits each pixel; 4:2:2 format, 16 bits each pixel; 4:2:0 format, 12 bits each pixel [25].
4:4:4 is the format we chose for the experiment. 4:4:4 format, 32 bit per
pixel, commonly known as AYUV. AYUV is a 4:4:4 YUV format with 8 bit samples for each component along with an 8 bit alpha blend value per pixel. 32-bit RGB images usually have 24-bit color plus 8 more bits for an alpha channel (8 bits per channel x 3 channels = 24 bits + alpha channel’s (A) 8 bits)[30] [32]. Component ordering is A Y U V (as the name suggests) [new]. This is a package format in which each pixel is encoded as four consecutive bytes [25].
25
4.7 Data processing The comparison is done by considering a reference video and the ten
distorted videos taken each one at time, thereby acquiring PSNR and SSIM values from the tool.
When two videos are compared, for the distorted part of the video i.e., for
the frames that are distorted we get PSNR as some non zero value but for the part which is similar in both the videos the PSNR values are zeroes. When two similar frames are compared the PSNR value is actually infinity and as infinity is not a number and displaying infinity as some number states that as false, it is displayed as zero by EvalVid.
The values that are obtained are per frame and taking the average of these
values gives the PSNR or SSIM value of the whole video which is also displayed as the result by the tool. There occurs a problem while taking the average of these values: the resulting value would be very low, most of the values are zeroes, which turns the mean into something quite unreasonable. This is the main drawback of taking PSNR as a metric within EvalVid. This problem can be solved by conditional average of the PSNR values of only the frames that are distorted, but skipping all the frames.
Figure 8: Comparison of tool averages with conditional averages
Figure 8 explains the case when the first distorted video is compared with the original video. Here the first second of the video has the distortion so let the conditional PSNR average for that part of video be X, a non-zero value, and the corresponding SSIM conditional average be Y which lies between -1 and 1, excluding 1 as it indicates an absolute match. The remaining parts have ideal values i.e. zero (indicating infinity) in the case of PSNR and one in the case of SSIM.
26
Thus the mean which is calculated by the EvalVid tool is X/10 for PSNR and Y+9/10 for SSIM whereas the conditional mean is X and Y for PSNR and SSIM, respectively. We observe that
The above relations between conditional and tool mean values say that in
the case of PSNR the conditional mean is greater than the tool mean and for SSIM it is less than or equal to the tool average. This shows that there is a necessity for calculating the conditional values in order to obtain reliable values.
The main drawback of the tool in calculating the PSNR is rectified in the
case of SSIM as explained earlier. Another instance, say Foreman_25 the SSIM value from the tool is 0.98 which is obtained directly from the tool. We have then calculated the average of SSIM values of the frames obtained from the tool that are frozen in the video ignoring the other frames. The conditional SSIM is 0.84 that is obtained by neglecting the 1’s which is not done by the tool. As neglecting 1’s in
calculating average doesn’t have the impact. Both the conditional and tool SSIM averages or means are very close and almost equal which is not with the case of PSNR as mentioned earlier.
Following are the set of commands to be used in the tool for obtaining the
values of the comparison metrics.
4.7.1 PSNR: The command to be given to calculate PSNR; psnr x y <YUV format> <src.yuv> <dst.yuv>
Here; x is the frame width y is the height width YUV format: 420, 422, 444 etc. src.yuv is the source video or the original video dst.yuv is the distorted video. After obtaining the values of PSNR from the tool the values are tabulated
which is included in the appendix.
4.7.2 SSIM: The command to be given to calculate SSIM; psnr x y <YUV format> <src.yuv> <dst.yuv> [ssim]
Here; x is the frame width y is the height width YUV format: 420, 422, 444 etc. src.yuv is the source video or the original video dst.yuv is the distorted video ssim is to be specified for calculating SSIM
27
The same procedure as done in calculating PSNR is followed for SSIM also. From the tabulated values (appendix) it is observed that the values have 1’s for
a good quality video/frame i.e. for frames that are same with that reference video/frame. The mean obtained from the tool and the mean calculated for the distorted or frozen frames is approximately equal, which makes the mean look appropriate unlike PSNR.
28
5 RESULTS, DISCUSSION AND VALIDATION This chapter deals with the interpretation and validation of the results in
detail. Validation of the results is done by comparing EvalVid results with the Matlab results. In both the cases the same set of videos are used.
5.1 PSNR: Table 3 below shows the average PSNR i.e. all values included. The latter
calculation of PSNR is done by taking the frames of distorted / frozen which is done due to the effect of zeros on the mean value of PSNR as explained in section 4.5. The below formula explains the calculation of the conditional average of PSNR.
In the above formula PSNR values of the distorted frames are taken which
are 25 frames in all the 10 cases. The beginning of the PSNR values may differ as per the delay. The PSNR values obtained from the tool can be seen in the appendix.
Table 3: PSNR: Tool and conditional average values in comparison
Foreman Hall Monitor News
Tool Conditional Tool Conditional Tool Conditional
25 2.01 18.69 3.42 32.85 4.03 38.98 50 1.91 17.65 2.89 27.50 4.11 39.78 75 1.90 17.62 3.27 31.31 3.82 36.87 100 1.85 17.11 3.71 35.77 2.70 25.54 125 2.04 18.99 3.75 36.18 3.72 35.83 150 1.64 14.95 3.73 35.97 3.79 36.50 175 1.30 11.56 3.74 36.02 3.11 29.67 200 1.51 13.63 3.71 35.75 4.04 39.02 225 1.80 16.54 3.69 35.55 4.01 38.78 250 2.34 13.63 3.43 32.92 3.73 35.97
In the case of PSNR tool averages are very low for all the videos compared
to conditional values. The reason behind this phenomenon is in the results it can be observed that there are 25 non zero values and the rest are 0’s for all the videos as
there exist freeze only for one second i.e. 25 frames and the rest is the same as the original video. While calculating PSNR the tool is considering also the 0’s which
makes the average very low.
29
5.2 SSIM: Table 4 shows the SSIM mean values obtained from the tool and manually.
The manual calculation of PSNR is done by taking the frames of distorted/frozen which is done due to the effect of 1’s on the mean value of SSIM as explained in section 4.7. The below formula explains the calculation of conditional average of SSIM;
Table 4: SSIM: Tool and conditional average values comparison Foreman Hall Monitor News
Tool Conditional Tool Conditional Tool Conditional
25 0.98 0.84 1 0.99 1 1 50 0.98 0.79 1 0.97 1 1 75 0.98 0.75 1 0.99 1 1 100 0.97 0.74 1 0.99 1 0.96 125 0.98 0.77 1 0.99 1 0.99 150 0.95 0.51 1 0.99 1 0.99 175 0.93 0.25 1 0.99 1 0.98 200 0.93 0.35 1 0.99 1 1 225 0.97 0.67 1 0.99 1 1 250 0.98 0.83 1 0.99 1 1
Though the presence of 1’s do not affect the mean value of SSIM, there is
still a minute difference between the tool and conditionally calculated average value which makes SSIM a powerful metric through which the user quality of experience can be improved much better.
SSIM is also calculated by the tool in the same way but the only difference
is; there are 25 values that are between -1 and 1 excluding 1 and rest are 1’s. While
the tool is calculating the mean, the effect of 1’s on the mean is low. But when the
conditional average is calculated without considering 1’s the average value is getting
high compared to the tool value. Based on the results tabulated above it can be inferred that there is a vast
difference between the tool and the conditional average. Considering the case with Foreman_25, the tool average is as low as 2.01 whereas conditional average is 18.69 for PSNR. SSIM has 0.98 as the tool average and 0.84 as the conditional average. While the tool average values are contracting the expected values in both the cases, the conditional values are in the close proximity of the expected values.
30
5.3 Validation: Initially validation is done by comparing the results with the PEVQ values from
[31]. Later the same is performed with Matlab as PEVQ can give only PSNR whereas Matlab can give both PSNR and SSIM.
5.3.1 Results of Matlab:
The below tabulated are the PSNR and SSIM results of Matlab
Table 5: PSNR: Tool and conditional average values comparison for Matlab
Foreman Hall Monitor News
Matlab Conditional Matlab Conditional Matlab Conditional
25 Inf 25.77 Inf 32.31 Inf 30.74 50 Inf 25.24 Inf 29.93 Inf 32.54 75 Inf 26.89 Inf 32.19 Inf 31.18
100 Inf 28.01 Inf 31.31 Inf 28.96 125 Inf 26.56 Inf 32.40 Inf 31.61 150 Inf 24.16 Inf 31.72 Inf 27.19 175 Inf 31.67 Inf 32.06 Inf 30.70 200 Inf 28.62 Inf 31.74 Inf 31.32 225 Inf 32.62 Inf 31.99 Inf 30.34 250 Inf 35.58 Inf 31.44 Inf 31.20
Table 6: SSIM: Tool and conditional average values comparison for Matlab Foreman Hall Monitor News
Matlab Conditional Matlab Conditional Matlab Conditional
25 0.95 0.45 1 0.96 0.99 0.86 50 0.94 0.38 0.99 0.88 0.99 0.89 75 0.95 0.44 0.99 0.91 0.98 0.81 100 0.94 0.41 0.99 0.91 0.97 0.71 125 0.96 0.55 0.99 0.94 0.98 0.84 150 0.94 0.32 0.99 0.93 0.98 0.79 175 0.93 0.23 0.99 0.94 0.98 0.81 200 0.93 0.22 0.99 0.93 0.98 0.84 225 0.92 0.20 0.99 0.93 0.98 0.78 250 0.95 0.51 0.99 0.91 0.99 0.85
To investigate to which extent the results are valid, the calculation of PNSR and SSIM for the same set of videos has been performed also in Matlab. The important observation done in the process is that there is a difference in the method that the tools have followed to read the videos in order to calculate the results.
31
5.3.2 Matlab vs. EvalVid conditional averages:
The following tabular forms consist of EvalVid and Matlab conditional averages of PSNR and SSIM.
Table 7: PSNR: EvalVid vs. Matlab conditional averages Foreman Hall Monitor News
EvalVid Matlab EvalVid Matlab EvalVid Matlab
25 18.69 25.77 32.85 32.31 38.98 30.74 50 17.65 25.24 27.50 29.93 39.78 32.54 75 17.62 26.89 31.31 32.19 36.87 31.18
100 17.11 28.01 35.77 31.31 25.54 28.96 125 18.99 26.56 36.18 32.40 35.83 31.61 150 14.95 24.16 35.97 31.72 36.50 27.19 175 11.56 31.67 36.02 32.06 29.67 30.70 200 13.63 28.62 35.75 31.74 39.02 31.32 225 16.54 32.62 35.55 31.99 38.78 30.34 250 13.63 35.58 32.92 31.44 35.97 31.20
Table 8: SSIM: EvalVid vs. Matlab conditional averages Foreman Hall Monitor News
EvalVid Matlab EvalVid Matlab EvalVid Matlab
25 0.84 0.95 0.99 1 1 0.99 50 0.79 0.94 0.97 0.99 1 0.99 75 0.75 0.95 0.99 0.99 1 0.98 100 0.74 0.94 0.99 0.99 0.96 0.97 125 0.77 0.96 0.99 0.99 0.99 0.98 150 0.51 0.94 0.99 0.99 0.99 0.98 175 0.25 0.93 0.99 0.99 0.98 0.98 200 0.35 0.93 0.99 0.99 1 0.98 225 0.67 0.92 0.99 0.99 1 0.98 250 0.83 0.95 0.99 0.99 1 0.99
Both PSNR and SSIM indicate Foreman as a low quality video compared to
the other two videos that are considered. When PSNR and SSIM are compared, PSNR values are fluctuating whereas; SSIM values depict congruency in EvalVid and Matlab values.
In EvalVid the videos are read in the character data type i.e. the videos are
evaluated taking videos byte wise. On the contrary Matlab reads the video pixel by pixel in the form of matrices i.e. bit wise.
The difference with calculation is; in EvalVid is that the ideal value infinity
is displayed as zero in the case of PSNR which makes the tool average “1” in case of
SSIM (1 means perfect correlation of two similar pictures. EvalVid considers even the values of undistorted for calculating the average.
32
6 CONCLUSION AND FUTURE WORK
6.1 Conclusion We have compared three videos Foreman, Hall monitor, News with
corresponding ten distorted videos for each video using the EvalVid tool. If and only if both the videos are of same size and have same number of frames, the EvalVid will make the comparison. The problem here is with the presence of 0’s in the PSNR
results as shown in the appendix because of which the average is very low. We made this reasonable by considering the PSNR values obtained from the tool only for the frames that are distorted neglecting the 0’s. Observing the values tabulated it is clear that the conditional average of the PSNR is appropriate, whereas the tool average is inconsistent.
SSIM has 1’s instead of 0’s as in the case of PSNR. The tool average is almost the same as conditional average in the case of SSIM for the distorted frames in the video. As the difference between the tool and conditional average is very low, it is recommended to use SSIM than PSNR which will help in giving a better outcome using EvalVid in order to improve QoE. Thus from our analysis it is clear that to acquire better evaluation of video quality of experience is an important aspect. Thus it can be said that SSIM is a better metric for video quality evaluation than PSNR which is an inconsistent metric in addition to the fact that SSIM considers HVS as discussed in the previous chapters.
To validate the results we have compared them with Matlab results. From
this validation process we can conclude that EvalVid is not that reliable compared to Matlab because of the difference in the method of reading the videos; videos are read as characters in EvalVid, as pixels (in matrix form) in Matlab.
6.2 Answers to the Research Questions:
1. What is the effect of impairment in a video caused by the mobile networks on the QoE?
Answer: Impairments caused due to the network can be of many kinds like loss of packets, delay, delay variation, freezes etc. However, freezes are the most typical issue for mobile connectivity. Freezes are usually due to the one frame repeating over a particular interval in replacement of lost or delayed frames. This will not result in a change in number of frames, but in other cases there might be few frames get lost. EvalVid can compare only the videos with same number of frames. As it is with mobile networks we chose frozen videos for evaluation.
The quality of a video by the end user mainly depends on the original
video’s quality, end users processor capability and the network capability to carry the video. Video when processed or transmitted through a network suffers from
33
impairments in the video, which in turn degrades the quality. These impairments can be in the form of jerks, freezes etc, appearing in the video.
2. How is the evaluation performed in EvalVid? How valid are the results?
Answer: EvalVid is a tool which evaluates videos quality by comparing the videos. It compares frame by frame and gives PSNR, SSIM or MOS for each frame based on the command used and gives the number of frames, average and standard deviation calculated from the individual frame’s PSNR and SSIM value as the result.
We used 4:4:4, in which all components are available. There is command
for PSNR as well as SSIM which is to be given as input. It first reads first video and splits it into frames, and then the second video is read and split into frames. Both the videos are compared frame-by-frame and PSNR or SSIM value for each frame comparison based on the given command (mentioned in section 4.7) is calculated. All these PSNR or SSIM values for each frame are displayed as the output along with the average PSNR, number of frames, in our case 251.
The results have been validated using Matlab, using the same set of videos.
Both the EvalVid and Matlab results are compared and it is found that the EvalVid is not that reliable tool for evaluating videos compared to Matlab which gave a more appropriate results. This conclusion is made because Matlab reads the frame matrix-wise sounding more appropriate. 3. Why SSIM is a better comparison metric than PSNR?
Answer: Although both these metrics are objective quality metrics, there are few differences between them to make one precede over the other. PNSR is easy and SSIM is complex to calculate. But there are more problems with PSNR and SSIM has appealing properties being a correlation measure.
SSIM overcome inconsistencies like human eye perception than PSNR. PSNR calculates the error between the original and distorted videos whereas SSIM indicates the measure of structural similarity. From these discussions it can be said that SSIM is better than PSNR.
This is supported from our study. It can be concluded that though PSNR is
a metric with less complexity in calculating it, results are inconsistent due to the presence of 0’s at least in EvalVid. For the frames which are similar in the both the videos the PSNR value is actually infinity is replaced with 0 which is explained in section 4.7, whereas SSIM is computational complexity, but produces consistent results as it is calculated considering human eye perception. PSNR averages are less reasonable than SSIM average values obtained from the EvalVid because of the presence of 1’s. These are also good reasons for SSIM being a better comparison metric than PSNR.
34
6.3 Future work EvalVid is a tool, which evaluates the quality of the videos by comparing
the videos frame by frame. The tool can be developed further in calculating the average just as conditional average. This would make PSNR, a metric with less complexity having improved consistency. SSIM would be more approximate if the tool is calculating the average in the same way as conditional average. EvalVid can be made compatible with all other operating systems not only LINUX and can be made more user friendly.
Further it is good to have an in depth study on the reason behind unreliable
results and can make the tool to read the videos pixel wise i.e. in the matrix form (8X8 form) as in Matlab, but not as characters which happens in EvalVid.
It is now capable of evaluating videos; it would be an improvement if it can
also evaluate audio and videos with audio as well. Also it is better to develop it to consider any format of video for evaluate. If there are any duplicates in the videos the tool should recognize, eliminate them and then evaluate the videos.
35
7 REFERENCES
1. N. Vercammen, N. Staelens, A. Rombaut, B. Vermeulen, and P. Demeester,
“Extensive video quality evaluation: a scalable video testing platform,” in Computer and Information Technology, 2008. ICCIT 2008. 11th International Conference on, 2008, pp. 91–97.
2. Jane Hunter , Varuni Witana , Mark Antoniades ,(1997,August) A Review of Video Streaming over the Internet, SuperNOVA Project, DSTC Technical Report TR97-10,[Online].Available:http://itee.uq.edu.au/~jane/janehunter/videostreaming.html?print=1
3. V. D. Bhamidipati, S. Kilari, “Effect of Delay/Delay variable on QoE in Video Streaming”, Master Thesis, School of Computing, Blekinge Tekniska Hogskola,
Karlskrona, Blekinge, 2010. 4. Arne Lie, (2008, November) On Rate Adaptation of networked multimedia systems
[Online].Available:http://www.item.ntnu.no/fag/ttm4142/host/Foiler%20fra%20timene/rate_adaptation_04Nov.pdf
5. J. Klaue, B. Rathke, A. Wolisz, “EvalVid- A Framework for Video Transmission and Quality Evaluation” in Proc. Of the International conference on Modelling
Techniques and Tools for Computer Performance Evaluation, Urbana, Illinois, USA, September 2003”, Urbana, Illinois, September, 2003.
6. M. Fiedler, T. Hossfeld, P. Tran-Gia, “A Generic Quantitative Relationship between Quality of experience and Quality of Service”, IEEE Network,24, 2, pp.36-41, March-April, 2010.
7. Seeling, P.; Reisslein, M.; , "Evaluating multimedia networking mechanisms using video traces," Potentials, IEEE , vol.24, no.4, pp. 21- 25, Oct.-Nov. 2005 doi: 10.1109/MP.2005.1549754
8. Tiago Gonçalves, Rui J. Lopes, Paulo Nunes, On Using metadata in video quality assessment based on the structural similarity (SSIM) index metric [Online].Available:http://dcti.iscte.pt/events/qoemcs/papers/goncalves.pdf
9. Zoran Kotevski ;Pece Mitrevski; ,”Experimental Comparison of PSNR and SSIM Metrics for Video Quality Estimation” Part 2, pp.357-366, 2010 ICT Innovations 2009 doi: 10.1007/978-3-642-10781-8_37
10. A. Suki M. Arif; Suhaidi Hassan; Osman Ghazali; Awang Nor; , "Evalvid-RASV: Shaped VBR rate adaptation stored video system," Education Technology and Computer (ICETC), 2010 2nd International Conference on , vol.5, no., pp.V5-246-V5-250, 22-24 June 2010 doi: 10.1109/ICETC.2010.5530036
11. Chia-Yu Yu; Chih-Heng Ke; Ce-Kuen Shieh; Chilamkurti, N.; , "MyEvalvid-NT - A Simulation Tool-set for Video Transmission and Quality Evaluation," TENCON 2006. 2006 IEEE Region 10 Conference , vol., no., pp.1-4, 14-17 Nov. 2006 doi: 10.1109/TENCON.2006.343864
12. Vijaykumar, M.; Rao, S.; , "A cross-layer framework for adaptive video streaming over wireless networks," Computer and Communication Technology (ICCCT), 2010 International Conference on , vol., no., pp.331-336, 17-19 Sept. 2010 doi: 10.1109/ICCCT.2010.5640507
13. J. Korhonen and J. You, “Improving objective video quality assessment with content
analysis,” in Proceedings of the fifth International Workshop on Video Processing
and Quality Metrics for Consumer Electronics (VPQM) Scottsdale, USA, 2010. 14. Q. Huynh-Thu and M. Ghanbari, “Scope of validity of PSNR in image/video quality
assessment,” Electronics letters, vol. 44, no. 13, pp. 800–801, 2008. 15. S. S. Channappayya, A. C. Bovik, and R. W. Heath, “A linear estimator optimized for
the structural similarity index and its application to image denoising,” in Image Processing, 2006 IEEE International Conference on, 2006, pp. 2637–2640.
36
16. Z. Kotevski and P. Mitrevski, “Experimental Comparison of PSNR and SSIM
Metrics for Video Quality Estimation,” ICT Innovations 2009, pp. 357–366, 2010. 17. Zhou Wang; Bovik, A.C.; Sheikh, H.R.; Simoncelli, E.P.; , "Image quality
assessment: from error visibility to structural similarity," Image Processing, IEEE
Transactions on , vol.13, no.4, pp.600-612, April 2004
18. S. Li and K. Ngan, “Influence of the smooth region on the structural similarity index,” Advances in Multimedia Information Processing-PCM 2009, pp. 836–846, 2009.
19. Y. Wang, “Survey of objective video quality measurements,” EMC Corporation
Hopkinton, MA, vol. 1748. 20. Online access of webpage, http://media.xiph.org/video/derf, on 25th April, 2011. 21. M. Liou, “Overview of the px64 kbit/s video coding standard,” Commun. ACM,
vol. 34, 1991, pp. 59-63 22. Nicol, R.C.; Mukawa, N.; , "Motion video coding in CCITT SG XV-the coded
picture format," Global Telecommunications Conference, 1988, and Exhibition.
'Communications for the Information Age.' Conference Record, GLOBECOM '88., IEEE , vol., no., pp.992-996 vol.2, 28 Nov-1 Dec 1988
23. Online access of webpage, http://www.debugmode.com/imagecmp/, on 7th June, 2011.
24. D. Stanescu, M. Stratulat, V. Groza, I. Ghergulescu, and D. Borca, “Steganography in YUV color space,” in Robotic and Sensors Environments, 2007. ROSE 2007.
International Workshop on, 2007, pp. 1–4. 25. Z. Wang, Y. Zhao, J. Zhang, and Y. Guo, “Research on motion detection of Video
Surveillance System,” in Image and Signal Processing (CISP), 2010 3rd
International Congress on, vol. 1, pp. 193–197. 26. I. E. G. Richardson, H.264 and MPEG-4 Video Compression. Chichester, UK: John
Wiley & Sons, Ltd, 2003, pp. 19. 27. ITU-T Recommendation P.910: “Subjective video quality assessment methods for
multimedia applications”, International telecommunication Union, Geneva, Switzerland, 1996.
28. U. Engelke, M. Kusuma, H. J. Zepernick, and M. Caldera, “Reduced-reference metric design for objective perceptual quality assessment in wireless imaging,” Signal
Processing: Image Communication, vol. 24, no. 7, pp. 525–547, 2009. 29. Yining Qi, Mingyuan Dai, “The Effect of Frame Freezing and Frame Skipping on
Video Quality,” Intelligent Information Hiding and Multimedia Signal Processing,
2006. IIH-MSP '06. International Conference on , vol., no., pp.423-426, Dec. 2006 30. “YUV pixel formats.” [Online]. Available: http://www.fourcc.org/yuv.php.
[Accessed: 10-Jun-2012]. 31. Tahir Nawaz Minhas, “Network Impact on Quality of Experience of Mobile
Video,” Ph.D. licentiate dissertation, School of Computing, BTH, Karlskrona,
Sweden, 2012. 32. “Final Cut Pro 7 User Manual: Video Sample Rate and Bit Depth.” [Online].
Available: http://documentation.apple.com/en/finalcutpro/usermanual/index.html#chapter=C%26section=11%26tasks=true. [Accessed: 10-Jun-2012].
37
8 APPENDIX
The appendix consists of the tabulated 25 frame results, tabulated frame-wise.
Table 7: Foreman video - PSNR values for the distorted frames
25 50 75 100 125 150 175 200 225 250
27.668 28.992 27.049 21.462 35.2 26.808 16.294 16.747 25.098 42.32
23.267 20.587 21.527 19.659 30.873 20.16 14.82 15.372 20.354 34.894
21.514 18.969 20.597 17.181 25.024 15.947 12.73 14.421 17.529 26.921
20.861 17.819 19.895 16.513 22.374 15.173 12.038 13.738 16.584 25.663
20.582 16.995 19.059 16.019 20.629 14.81 11.414 13.326 15.943 24.784
20.115 16.46 18.246 15.845 19.348 14.624 10.701 13.298 15.461 24.209
18.965 15.953 17.511 15.836 18.404 14.603 10.169 13.385 15.138 23.396
18.357 15.796 17.386 16.055 17.352 14.699 9.769 13.195 15.015 22.061
17.9 15.881 17.315 16.123 16.719 14.692 9.927 13.07 15.028 21.486
17.518 16.168 17.092 16.124 16.429 14.68 10.072 12.97 15.124 21.073
17.348 16.576 16.844 16.268 16.245 14.641 10.151 12.958 15.193 20.622
17.502 16.951 16.351 16.49 16.178 14.593 10.093 12.964 15.195 20.095
17.537 17.028 16.092 16.76 16.316 14.565 10.079 12.963 15.061 19.343
17.558 17.042 15.946 16.734 16.546 14.471 10.038 12.93 14.988 18.239
17.473 16.858 15.922 16.736 16.67 14.256 10.316 12.877 14.978 17.804
17.463 16.758 15.986 16.779 16.623 14.075 10.578 12.829 15.056 17.491
17.501 16.502 15.859 16.794 16.392 13.922 10.838 12.738 15.181 17.203
17.265 16.483 15.654 16.697 16.259 13.525 10.864 12.599 15.545 17.069
16.85 16.506 15.588 16.634 16.227 13.21 11.045 12.534 15.765 17.287
16.478 16.191 15.324 16.537 16.338 12.663 11.037 12.501 15.941 17.523
16.187 15.963 15.039 16.417 16.308 12.127 11.059 12.453 16.036 17.646
15.816 16.217 14.846 16.3 16.26 11.677 10.975 12.396 16.04 17.677
15.766 16.281 14.62 16.116 15.905 10.995 10.554 12.335 15.888 17.648
15.76 16.098 14.632 16.049 15.784 10.67 10.437 12.328 15.799 17.649
24.095 26.233 26.162 25.768 24.311 22.183 23.126 23.74 25.659 30.102
Table 8: Foreman video - SSIM values for the distorted frames
25 50 75 100 125 150 175 200 225 250
0.979 0.986 0.969 0.902 0.995 0.966 0.673 0.593 0.941 0.999
0.949 0.916 0.9 0.866 0.986 0.847 0.584 0.504 0.841 0.993
0.93 0.877 0.878 0.765 0.948 0.613 0.406 0.435 0.741 0.961
0.923 0.835 0.857 0.712 0.908 0.563 0.358 0.379 0.689 0.948
0.918 0.802 0.831 0.68 0.868 0.517 0.305 0.355 0.65 0.938
0.911 0.776 0.803 0.671 0.829 0.51 0.254 0.35 0.623 0.931
0.89 0.746 0.775 0.67 0.796 0.499 0.209 0.347 0.608 0.917
0.872 0.731 0.771 0.684 0.75 0.552 0.192 0.294 0.603 0.888
0.854 0.732 0.775 0.691 0.714 0.546 0.19 0.279 0.605 0.871
0.835 0.744 0.775 0.692 0.696 0.552 0.214 0.271 0.606 0.86
0.821 0.761 0.773 0.7 0.687 0.557 0.192 0.262 0.606 0.846
0.825 0.781 0.755 0.716 0.686 0.558 0.158 0.259 0.611 0.828
0.823 0.789 0.739 0.744 0.702 0.548 0.115 0.264 0.604 0.803
0.82 0.791 0.727 0.742 0.72 0.541 0.096 0.267 0.602 0.767
0.815 0.785 0.72 0.741 0.727 0.538 0.09 0.275 0.603 0.748
0.813 0.779 0.709 0.742 0.72 0.533 0.12 0.284 0.607 0.734
0.813 0.767 0.684 0.742 0.702 0.527 0.172 0.288 0.614 0.72
0.804 0.764 0.663 0.733 0.707 0.487 0.165 0.296 0.633 0.714
0.785 0.763 0.658 0.729 0.708 0.445 0.199 0.297 0.642 0.723
0.767 0.75 0.646 0.715 0.708 0.376 0.171 0.297 0.652 0.736
0.752 0.741 0.632 0.71 0.701 0.296 0.156 0.295 0.658 0.747
0.734 0.755 0.623 0.699 0.699 0.221 0.139 0.294 0.66 0.75
0.732 0.762 0.612 0.684 0.673 0.113 0.074 0.287 0.655 0.749
0.733 0.756 0.612 0.681 0.661 0.068 0.073 0.29 0.654 0.748
0.95 0.965 0.963 0.966 0.955 0.923 0.922 0.921 0.948 0.973
38
Table 9: Hall Monitor video - PSNR values for the distorted frames
25 50 75 100 125 150 175 200 225 250
36.294 33.639 35.598 35.683 36.55 36.42 36.824 36.536 36.735 36.735
36.108 31.397 32.875 35.94 35.794 35.462 35.847 35.899 36.087 34.45
35.628 30.67 31.913 34.806 35.765 35.883 35.704 34.82 35.809 32.115
34.325 29.62 31.376 35.534 36.015 35.586 35.73 34.691 35.891 32.865
36.32 30.827 31.202 35.616 36.221 35.8 35.983 35.605 35.885 32.429
35.546 27.213 31.113 35.419 35.411 35.655 35.864 34.755 36.35 33.521
34.715 27.746 30.765 35.6 35.858 35.805 35.836 35.634 34.767 32.356
34.158 27.328 30.629 35.253 35.927 35.487 35.711 35.708 35.529 33.604
33.725 26.825 30.698 35.446 35.622 35.482 35.965 35.5 35.717 33.392
33.669 27.563 30.71 35.448 35.734 35.609 35.828 35.472 35.933 33.22
33.355 27.217 30.566 35.153 35.541 35.202 35.843 35.269 35.877 33.355
33.785 26.451 30.221 35.284 35.947 35.739 36.225 35.139 35.858 33.041
33.544 26.086 30.445 35.291 36.037 35.209 35.626 35.372 35.264 32.407
33.988 25.706 30.386 35.329 35.993 35.541 36.027 35.774 35.376 32.758
33.953 25.116 29.835 35.566 36.04 35.757 35.654 35.752 35.45 32.827
32.649 24.675 29.699 35.506 36.053 35.743 35.651 35.381 35.398 32.295
29.51 24.476 30.032 35.677 35.553 35.747 35.305 35.616 35.411 32.198
28.996 24.617 30.184 35.001 35.945 35.695 34.656 35.625 34.88 31.628
28.429 24.938 29.858 34.336 36.015 35.308 35.697 35.384 34.885 31.282
28.217 25.442 29.785 35.26 35.538 35.382 35.781 35.293 34.545 31.62
27.636 25.711 29.911 35.686 35.699 35.433 34.646 35.477 33.998 31.415
27.672 25.691 29.902 35.491 35.928 35.53 35.329 35.165 33.945 31.626
27.492 25.528 29.894 35.736 35.837 35.578 35.641 34.968 33.601 31.538
27.126 25.521 29.879 35.486 35.972 36.122 35.46 35.531 33.47 30.343
44.562 37.419 45.217 44.815 43.485 43.998 43.637 43.353 42.083 39.959
Table 10: Hall monitor video - SSIM values for the distorted frames
25 50 75 100 125 150 175 200 225 250
0.994 0.992 0.994 0.994 0.995 0.995 0.995 0.995 0.995 0.995
0.995 0.987 0.992 0.994 0.994 0.994 0.994 0.994 0.995 0.993
0.994 0.984 0.99 0.993 0.994 0.995 0.994 0.993 0.995 0.99
0.994 0.983 0.99 0.994 0.995 0.994 0.994 0.993 0.995 0.992
0.995 0.984 0.989 0.994 0.995 0.994 0.994 0.994 0.994 0.99
0.994 0.972 0.989 0.994 0.994 0.994 0.994 0.993 0.995 0.992
0.993 0.974 0.988 0.994 0.994 0.994 0.994 0.994 0.993 0.989
0.992 0.974 0.988 0.994 0.994 0.994 0.994 0.994 0.994 0.992
0.991 0.972 0.988 0.994 0.994 0.994 0.994 0.994 0.994 0.992
0.991 0.974 0.988 0.994 0.994 0.994 0.994 0.994 0.995 0.991
0.991 0.972 0.988 0.993 0.994 0.994 0.994 0.994 0.994 0.991
0.991 0.968 0.987 0.994 0.994 0.994 0.995 0.993 0.995 0.991
0.991 0.964 0.988 0.994 0.994 0.994 0.994 0.994 0.994 0.99
0.992 0.96 0.988 0.994 0.995 0.994 0.994 0.994 0.994 0.99
0.992 0.954 0.987 0.994 0.995 0.994 0.994 0.994 0.994 0.99
0.991 0.949 0.987 0.994 0.995 0.994 0.994 0.994 0.994 0.989
0.981 0.948 0.988 0.994 0.994 0.994 0.994 0.994 0.994 0.989
0.977 0.951 0.987 0.993 0.994 0.994 0.993 0.994 0.994 0.988
0.976 0.954 0.986 0.993 0.994 0.994 0.994 0.994 0.994 0.988
0.975 0.957 0.987 0.994 0.994 0.994 0.994 0.994 0.993 0.989
0.971 0.957 0.986 0.994 0.994 0.994 0.992 0.994 0.993 0.988
0.97 0.957 0.987 0.994 0.994 0.994 0.994 0.994 0.993 0.989
0.968 0.955 0.986 0.994 0.994 0.994 0.994 0.993 0.992 0.989
0.967 0.956 0.987 0.994 0.994 0.994 0.994 0.994 0.992 0.986
1 0.998 1 1 0.999 0.999 0.999 0.999 0.999 0.999
39
Table 11: News video - PSNR values for the distorted frames
25 50 75 100 125 150 175 200 225 250
40.225 46.306 46.324 35.695 46.906 46.304 42.518 46.497 45.879 46.186
39.977 43.553 43.985 30.782 46.191 45.013 38.082 45.484 44.151 44.086
39.767 42.388 43.543 26.121 44.766 42.624 32.515 43.704 41.797 40.726
39.466 41.667 43.109 24.842 43.809 41.757 30.756 43.027 40.694 39.104
39.114 40.879 43.08 24.066 42.585 40.899 29.4 42.54 39.603 37.826
38.894 40.364 43.215 23.512 41.674 39.941 28.261 41.948 39.099 36.836
38.513 39.62 43.388 23.179 40.915 39.419 27.38 41.343 38.66 35.98
38.337 39.281 43.349 22.898 38.648 39.048 26.494 40.402 38.663 35.03
38.399 38.955 42.758 22.824 37.354 38.74 26.401 39.785 38.625 34.676
38.394 38.744 41.999 22.789 36.178 38.439 26.513 39.286 38.366 34.308
38.204 38.619 41.324 22.774 34.811 38.026 26.589 38.811 38.072 33.962
38.041 38.067 40.324 22.697 33.664 37.544 26.597 38.431 37.944 33.601
38.157 37.781 39.876 22.6 31.818 36.74 26.36 36.998 37.557 33.331
38.118 37.603 38.388 22.563 31.073 35.948 26.223 35.847 37.215 32.831
38.133 37.439 35.774 22.518 30.434 34.771 26.122 35.099 36.778 32.652
38.117 37.408 32.843 22.458 29.867 33.452 26.031 34.798 36.487 32.442
37.744 37.193 27.797 22.419 29.299 32.095 26.008 34.628 35.948 32.221
37.537 36.938 26.01 22.351 28.443 29.404 25.999 34.329 35.405 31.999
37.295 36.692 24.683 22.344 28.049 28.246 26.048 34.177 35.122 31.603
37.093 36.503 23.677 22.344 27.618 27.214 26.149 33.971 34.636 31.372
37.075 36.225 22.93 22.343 27.182 26.327 26.205 33.516 34.202 31.127
37.49 35.479 22.331 22.355 26.819 25.621 26.299 32.822 33.777 30.888
37.639 35.219 21.718 22.38 26.294 24.636 26.445 31.359 33.149 30.707
37.646 35.087 21.499 22.39 26.104 24.257 26.504 30.768 32.812 30.268
55.013 66.619 67.776 67.2 65.375 65.958 65.785 65.896 64.854 65.533
Table 12: News video - SSIM values for the distorted frames
25 50 75 100 125 150 175 200 225 250
0.999 0.999 0.999 0.998 0.999 0.999 0.999 0.999 0.999 1
0.999 0.999 0.999 0.993 0.999 0.999 0.995 0.999 0.999 1
0.999 0.999 0.999 0.981 0.999 0.999 0.993 0.999 0.999 0.999
0.999 0.999 0.999 0.975 0.998 0.999 0.99 0.999 0.999 0.999
0.999 0.999 0.998 0.97 0.998 0.999 0.987 0.999 0.999 0.999
0.999 0.999 0.996 0.966 0.997 0.999 0.984 0.999 0.999 0.998
0.999 0.999 0.987 0.963 0.996 0.999 0.981 0.999 0.999 0.998
0.999 0.999 0.98 0.96 0.994 0.999 0.981 0.998 0.999 0.997
0.999 0.999 0.973 0.959 0.993 0.999 0.981 0.998 0.999 0.997
0.999 0.999 0.967 0.958 0.992 0.998 0.981 0.997 0.999 0.997
0.999 0.999 0.961 0.957 0.99 0.998 0.981 0.997 0.999 0.997
0.999 0.998 0.956 0.956 0.989 0.997 0.981 0.997 0.998 0.996
0.999 0.998 0.95 0.954 0.987 0.996 0.981 0.997 0.998 0.996
0.999 0.998 0.948 0.954 0.986 0.995 0.98 0.997 0.998 0.996
0.999 0.998 1 0.953 0.984 0.99 0.979 0.996 0.998 0.995
0.999 0.998 1 0.952 0.983 0.987 0.979 0.996 0.998 0.995
0.999 0.998 1 0.952 0.981 0.984 0.98 0.994 0.997 0.995
0.999 0.998 1 0.952 0.98 0.981 0.98 0.993 0.997 0.995
0.998 0.998 1 0.951 0.979 0.978 0.98 1 0.997 0.994
0.998 0.997 1 0.951 1 0.971 0.98 1 0.996 0.994
0.998 0.997 1 0.951 1 0.969 0.981 1 0.996 0.994
0.999 1 1 0.952 1 1 0.981 1 0.995 0.993
0.999 1 1 0.954 1 1 0.981 1 1 0.993
0.999 1 1 0.952 1 1 1 1 1 0.992
1 1 1 1 1 1 1 1 1 1
40
Table 13: Foreman video - Matlab PSNR values for the distorted frames
25 50 75 100 125 150 175 200 225 250
29.4479725 31.3536952 29.15948109 27.80145574 33.44003092 29.1923877 25.54534681 24.87462865 27.6403021 38.7958612
27.77792472 27.3024528 26.2331629 26.29406072 30.81231169 26.67223893 24.97101075 23.87865924 25.7677233 32.231253
27.36656318 26.3779158 25.92122293 25.25060688 28.21246034 25.36513668 24.60830066 22.9053425 24.8985301 27.9234843
27.29928209 25.8224031 25.62395986 24.86760828 27.53750186 25.03949188 24.70782791 22.51179736 24.6627668 27.5406989
27.28193631 25.4366702 25.36366939 24.77228472 27.0049551 24.73196226 24.86011505 22.18567614 24.5266541 27.24108
27.13510232 25.1926213 25.06070649 24.71880807 26.55980745 24.13403909 25.04675463 21.90017444 24.3650945 26.9151018
26.55539626 24.7724112 24.80617433 24.69742571 26.31163726 24.10065445 25.29286128 21.67885959 24.2694566 26.508288
26.11244497 24.686311 24.80968404 24.70468602 25.963264 24.28259173 25.73560491 21.39899838 24.1819207 25.8129658
25.77455186 24.6826857 24.93293587 24.71540067 25.84075782 24.26129761 26.11849563 21.31958592 24.2025723 25.588087
25.56521707 24.6359204 24.98773401 24.7516926 25.86615225 24.24471729 26.45564667 21.25635575 24.2404439 25.5591048
25.43765816 24.6308617 24.86983176 24.79097311 25.83877196 24.21672878 26.55836241 21.22415847 24.264501 25.5413038
25.25797614 24.7029657 24.55924773 24.8282917 25.79782722 24.17731695 26.58372218 21.17233455 24.2697285 25.4730045
25.20960699 24.7057567 24.47124196 24.96914362 26.09288121 24.15222064 26.49021264 21.0210074 24.2474585 25.4069935
25.1269447 24.6935659 24.4218878 25.0023293 26.33598819 24.09634107 26.43493494 20.93560978 24.2616815 25.1686483
25.12894018 24.6275516 24.44977644 24.99162567 26.49418423 23.97111039 26.40006814 20.85901485 24.2797718 25.130125
25.14760463 24.6251216 24.44513373 24.99275284 26.51438978 23.85990327 26.48538815 20.78951562 24.3212135 25.1162911
25.05035306 24.573058 24.25445152 24.99299494 26.07366233 23.71334781 26.62037003 20.74247334 24.3366553 25.1231305
24.95506089 24.5888087 24.10367027 24.89653924 25.53973802 23.36798896 26.30007204 20.67727599 24.4541074 25.1611176
24.79301069 24.6092039 24.01344438 24.91315407 25.48673996 23.09486222 26.09274187 20.67098003 24.5126305 25.3207185
24.62876715 24.5733126 24.03495097 24.88206589 25.43383658 22.89524381 25.79356431 20.67373477 24.5611312 25.4399492
24.47741307 24.560192 24.05105549 24.7102704 25.37947221 22.70230728 25.36239917 20.67842839 24.596358 25.5291125
Table 14: Foreman video - Matlab SSIM values for the distorted frames
25 50 75 100 125 150 175 200 225 250
0.876646778 0.94026772 0.877642198 0.807205547 0.955116895 0.852037253 0.28479808 0.424277857 0.804134622 0.992582362
0.760083851 0.70553852 0.627359246 0.625772487 0.897451297 0.592837005 0.23983568 0.373373012 0.444296252 0.968563786
0.714056074 0.60514199 0.571991331 0.440348249 0.776614781 0.377023956 0.18172259 0.325530058 0.227577346 0.859353048
0.689914309 0.50218577 0.518109523 0.393621872 0.703776484 0.337820614 0.20431228 0.288099204 0.191630303 0.82850143
0.677748155 0.42005428 0.478219531 0.365237033 0.635275215 0.31152062 0.19254088 0.253246779 0.166454527 0.797306741
0.666695171 0.37685191 0.447762487 0.353719753 0.579823765 0.29728997 0.20083445 0.23079627 0.146416697 0.76128769
0.601605739 0.32001458 0.432349551 0.3511625 0.538884763 0.299000062 0.20260161 0.232601764 0.135960339 0.707916745
0.550124345 0.2955945 0.438469223 0.365544299 0.499829465 0.305745682 0.21312514 0.222437957 0.133659809 0.5815744
0.498256804 0.28736987 0.457165578 0.375223553 0.494004697 0.305234758 0.21190934 0.219782335 0.131887533 0.52988903
0.445508112 0.29324978 0.467556715 0.37935099 0.492952527 0.298561273 0.21883843 0.212887573 0.132985373 0.510757241
0.409013152 0.31306122 0.462273004 0.388598694 0.491872597 0.291991613 0.22906821 0.209587566 0.135028584 0.486469526
0.372511482 0.33759735 0.422992288 0.402389692 0.496089178 0.292744124 0.2189784 0.210538669 0.134720341 0.440436706
0.360797 0.33588413 0.407080808 0.403655182 0.530850014 0.30624267 0.21865329 0.195181001 0.136981615 0.388733001
0.360993123 0.33000009 0.401233535 0.396561457 0.572769306 0.289129033 0.22028568 0.196418473 0.139266713 0.293792242
0.363672238 0.31737379 0.40803534 0.396539935 0.607781177 0.271281462 0.22280999 0.185748511 0.141005645 0.274426009
0.368682558 0.310347 0.412800488 0.400663306 0.59048739 0.270226544 0.21676515 0.185408498 0.144663492 0.265889941
0.351697441 0.29579769 0.374713433 0.404768218 0.521858677 0.270318427 0.2316722 0.178960061 0.146932896 0.259778713
0.320251684 0.30000199 0.331493414 0.409218431 0.408978321 0.260530385 0.25975959 0.181378998 0.152124047 0.262042769
0.279783193 0.30058779 0.305068256 0.404993636 0.402492493 0.263244866 0.26684428 0.18126022 0.158194126 0.285410996
0.261468543 0.29504021 0.307887368 0.393272212 0.420059393 0.26493178 0.26411069 0.182416597 0.164811084 0.30960742
0.255498628 0.28772767 0.314358496 0.37339577 0.424372638 0.268568693 0.26252673 0.182443404 0.171226383 0.331269373
0.24513169 0.31632965 0.322401429 0.351837283 0.427327622 0.260081086 0.27755888 0.182103696 0.174831366 0.346760067
0.239760059 0.32479916 0.344155626 0.334178202 0.402005845 0.250252717 0.24464483 0.177507497 0.181271401 0.35882334
0.236042453 0.32531606 0.339661203 0.337913849 0.383492561 0.240341519 0.23053708 0.176399314 0.185810255 0.379799924
41
Table 15: Hall Monitor video - Matlab PSNR values for the distorted frames
25 50 75 100 125 150 175 200 225 250
33.95276 31.95387 32.77492 32.42653 33.54847 33.20284 33.29923 32.66589 33.08873 33.0775
33.74957 31.26404 32.24354 32.35196 33.25745 32.82304 33.12634 32.40842 33.23867 32.08344
32.9356 30.38941 32.02357 31.62534 32.62874 32.22817 32.53799 32.3411 32.4365 31.99939
33.8947 31.1977 32.20322 31.73479 32.66107 32.24449 32.4972 32.66261 32.54209 31.49676
33.89765 29.389 32.2436 31.83917 32.78951 32.07865 32.31167 32.07254 32.44968 32.06481
34.03667 30.5279 31.55958 31.4494 32.40701 31.99424 32.38675 32.2159 32.45917 32.1766
33.74717 30.45785 31.87567 31.78909 32.39337 31.9434 32.37706 32.0727 32.19249 31.67579
33.6496 29.25276 31.80087 31.26853 32.30167 31.8819 31.8224 31.91109 32.16664 32.23876
33.90664 29.67173 31.75973 31.23414 32.23082 31.8192 31.77451 31.43283 32.10892 31.5137
32.69946 29.55348 32.07123 31.15411 32.42197 31.91701 31.72333 31.64504 32.23892 31.7591
33.41962 29.68776 31.97589 31.3459 32.36328 32.08357 31.99678 32.03449 32.0677 31.60328
32.73003 29.65246 32.14139 31.15568 32.42088 31.7104 31.93906 31.58049 31.74438 31.34395
32.84295 29.75102 32.29379 31.14037 31.98288 31.56051 31.97956 31.72756 31.74027 31.37409
32.86509 29.69358 32.5105 31.23267 32.29397 31.56675 31.64388 31.73772 31.6295 31.10377
32.4934 29.2741 31.96337 30.86427 32.25735 31.48505 32.15349 31.68159 31.72694 31.60346
32.1636 29.13739 32.32071 31.11223 32.4547 31.36923 31.96605 31.29664 31.75207 30.81293
31.44258 28.95029 32.54064 30.82554 32.10751 31.58727 31.79453 31.45043 31.91005 31.16057
31.29888 29.38345 32.26796 30.81021 32.17607 31.15743 31.84147 31.20831 31.71728 31.25787
30.91233 29.64395 32.17041 30.85071 32.21023 31.23746 31.95337 31.3389 31.69448 30.71392
30.36032 29.72464 32.52153 30.60778 32.3991 31.22769 31.89187 31.46392 31.71027 30.81053
30.25252 30.07767 32.39767 31.26767 32.18496 31.13538 31.31097 31.10629 31.70486 30.83628
29.69301 29.68394 32.57223 31.04147 32.16674 30.95224 31.66323 31.2659 31.33674 31.12963
29.07073 29.88679 32.1614 31.15797 32.11802 31.1474 31.93274 30.99223 31.11564 30.22435
29.42661 30.13678 32.33637 31.03624 32.00736 30.99607 31.56127 31.50744 31.01854 30.61881
Table 16: Hall Monitor video - Matlab SSIM values for the distorted frames
25 50 75 100 125 150 175 200 225 250
0.979444 0.949799 0.966674 0.967911 0.978268 0.975948 0.977413 0.975701 0.975742 0.974143
0.979408 0.923359 0.93593 0.952702 0.975115 0.968201 0.972933 0.96924 0.968424 0.961648
0.979508 0.915988 0.929395 0.933556 0.965528 0.953722 0.962297 0.953506 0.953251 0.943578
0.979205 0.911686 0.926967 0.9281 0.9583 0.947593 0.956458 0.946966 0.946861 0.939231
0.979909 0.908074 0.925936 0.924316 0.95372 0.94357 0.949237 0.942651 0.941514 0.935865
0.978882 0.900375 0.920659 0.920099 0.946237 0.940301 0.945275 0.938991 0.937378 0.932626
0.978025 0.891718 0.915026 0.917206 0.94141 0.937322 0.941394 0.934674 0.934608 0.930598
0.977602 0.889174 0.914574 0.909813 0.935272 0.931916 0.93565 0.927227 0.932608 0.923336
0.977003 0.887067 0.913151 0.906398 0.934618 0.930702 0.934128 0.924331 0.931824 0.918658
0.976962 0.882987 0.912549 0.905849 0.933682 0.928804 0.932598 0.924953 0.932839 0.915951
0.976465 0.881298 0.911382 0.904097 0.931539 0.925308 0.931617 0.924375 0.934654 0.9117
0.976139 0.876403 0.909679 0.901512 0.929876 0.923823 0.931128 0.92397 0.937467 0.909945
0.972309 0.871616 0.908903 0.898936 0.927092 0.920103 0.928724 0.923517 0.941853 0.908203
0.966805 0.866724 0.906473 0.898979 0.927671 0.919301 0.927745 0.922166 0.938585 0.905982
0.962409 0.863261 0.904695 0.89706 0.927154 0.919181 0.926534 0.920471 0.936565 0.904077
0.957237 0.8616 0.903963 0.895791 0.92574 0.917754 0.926311 0.918438 0.933678 0.900392
0.939318 0.862638 0.902672 0.895526 0.924336 0.915815 0.927049 0.914696 0.930636 0.898173
0.939396 0.862635 0.904122 0.895344 0.922503 0.908879 0.926497 0.912826 0.924548 0.894634
0.933998 0.864376 0.903032 0.894042 0.920969 0.90627 0.928712 0.910999 0.921779 0.889602
0.927084 0.866099 0.903401 0.892364 0.920821 0.902691 0.928519 0.911827 0.919857 0.888929
0.921744 0.868157 0.902859 0.893606 0.920106 0.901473 0.92575 0.910015 0.917686 0.889735
0.922455 0.869702 0.904237 0.892775 0.918767 0.900417 0.927237 0.909733 0.915406 0.890552
0.918531 0.869089 0.905338 0.890805 0.917409 0.898749 0.926663 0.909642 0.913029 0.890472
0.924739 0.869375 0.905224 0.890721 0.916914 0.900296 0.925403 0.913301 0.912059 0.882829
42
Table 17: News video - Matlab PSNR values for the distorted frames
25 50 75 100 125 150 175 200 225 250
33.90631 36.66121 36.34881 31.05204 36.84696 27.75823 35.36175 36.50946 31.97186 36.23644
33.1084 34.3175 33.95325 30.3136 35.15098 27.71785 33.26078 34.8669 31.6857 34.14786
32.9792 34.10472 33.75532 29.66364 34.36003 27.58569 31.94835 33.86855 31.26696 32.90111
32.39612 33.63898 33.19065 29.48086 33.77795 27.51867 31.48067 33.26739 31.10086 32.29604
31.91397 33.27864 32.85274 29.36028 33.56918 27.49845 31.17075 32.86047 30.95173 32.02892
31.52919 32.89025 32.58217 29.28134 33.15162 27.4331 30.87151 32.49568 30.8597 31.70864
31.21871 32.6986 32.44206 29.17949 32.88229 27.40614 30.73109 32.24382 30.70911 31.52301
30.98429 32.58508 32.26238 29.00498 32.27798 27.40353 30.5262 31.80682 30.4696 31.30555
30.7856 32.48409 32.19567 28.88811 31.98198 27.38724 30.45016 31.66629 30.2958 31.22848
30.54576 32.32123 31.87061 28.83614 31.70481 27.38654 30.37791 31.43334 30.19512 31.1396
30.32308 32.21578 31.45379 28.77331 31.37267 27.38006 30.33383 31.22533 30.11938 30.98696
30.12363 32.04316 30.94272 28.73937 31.17251 27.36641 30.30054 31.10262 30.10672 30.92444
30.08647 31.98061 30.77047 28.60749 30.78127 27.30709 30.14007 30.5506 30.03813 30.80532
30.00899 31.95354 30.57762 28.56638 30.57743 27.25667 30.1291 30.2834 30.04536 30.60478
30.0101 31.92822 30.32975 28.51367 30.59184 27.18326 30.10192 30.10937 29.99154 30.67227
29.97652 31.90358 30.07851 28.48485 30.49743 27.12061 30.03803 30.0041 29.93288 30.63438
29.89219 31.99714 29.63196 28.50937 30.2876 27.04625 30.0072 29.88013 29.91633 30.4743
29.81917 31.95535 29.41919 28.52983 29.89377 26.85264 30.01168 29.74651 29.89703 30.28719
29.73696 31.8829 29.23782 28.52695 29.82038 26.76412 29.98464 29.72258 29.85233 30.05142
29.70502 31.88072 29.10706 28.51521 29.79003 26.70635 29.98757 29.70268 29.79089 30.02287
29.71322 31.78857 28.9991 28.5206 29.61995 26.67669 29.94023 29.67817 29.76181 29.84795
29.68646 31.5473 28.90023 28.5279 29.59137 26.65402 29.89698 29.62472 29.73853 29.81216
29.694 31.46625 28.76802 28.60211 29.4308 26.63156 29.89465 29.53296 29.77389 29.69903
29.68847 31.38845 28.6999 28.61196 29.40499 26.61359 29.89689 29.46397 29.71763 29.55624
Table 18: News video - Matlab SSIM values for the distorted frames
25 50 75 100 125 150 175 200 225 250
0.963078 0.981298 0.976065 0.810575 0.962599 0.822069 0.974406 0.978405 0.817317 0.961676
0.941027 0.932231 0.917916 0.79619 0.930154 0.82178 0.933708 0.953868 0.818454 0.92697
0.940539 0.931415 0.915209 0.75438 0.911375 0.818826 0.884063 0.930661 0.812907 0.900565
0.916718 0.912676 0.897934 0.736664 0.902985 0.816919 0.857587 0.912229 0.809816 0.888879
0.897394 0.90405 0.886318 0.723057 0.896101 0.814929 0.841976 0.899632 0.806729 0.8792
0.887657 0.898761 0.878038 0.716484 0.891094 0.81397 0.829103 0.888208 0.807562 0.873524
0.881198 0.894817 0.87084 0.712212 0.881395 0.814156 0.81713 0.876781 0.806904 0.865026
0.878957 0.887626 0.864617 0.710464 0.866741 0.814523 0.800926 0.868144 0.803356 0.856332
0.874399 0.88484 0.862819 0.708841 0.856344 0.816937 0.796744 0.866881 0.798729 0.851455
0.868642 0.878379 0.858353 0.706374 0.849531 0.817918 0.791239 0.8632 0.793544 0.850274
0.863622 0.875154 0.849887 0.705767 0.842441 0.817957 0.788699 0.857479 0.791202 0.847907
0.855129 0.872473 0.830997 0.704568 0.83671 0.817078 0.788341 0.854201 0.789532 0.845419
0.850233 0.872283 0.823261 0.700162 0.825222 0.812882 0.785343 0.830574 0.783941 0.839916
0.844703 0.872315 0.81539 0.697291 0.821425 0.807744 0.784879 0.818965 0.779365 0.839302
0.840575 0.874539 0.805041 0.693274 0.816262 0.801947 0.786887 0.807745 0.773634 0.837721
0.838122 0.874055 0.796907 0.691964 0.811465 0.794802 0.785712 0.802607 0.770335 0.836673
0.825971 0.88116 0.765927 0.690539 0.798945 0.787147 0.789651 0.793238 0.767984 0.827898
0.821824 0.880889 0.748569 0.68755 0.796837 0.769111 0.791663 0.783429 0.762644 0.831654
0.819927 0.875583 0.732137 0.687887 0.789733 0.759762 0.786893 0.779419 0.760391 0.830414
0.818843 0.875729 0.718731 0.68676 0.78594 0.749629 0.787837 0.778007 0.757807 0.829302
0.81821 0.869587 0.704587 0.686473 0.78428 0.737962 0.782462 0.773944 0.757055 0.829635
0.8179 0.855234 0.693196 0.685386 0.782761 0.726023 0.778239 0.769316 0.756124 0.829186
0.818414 0.846952 0.67831 0.681397 0.773222 0.708623 0.765148 0.761236 0.752157 0.823832
0.818229 0.844862 0.673249 0.681142 0.771033 0.702914 0.766391 0.759262 0.75078 0.811583
