IP Video Quality Testing

IP Video Quality Testing

Oct 2008

2

Agenda

•Video Networks

•IP Video QoS

•QoS and QoE mapping vs Network Segments

•IPTV testing

•Trouble-shooting

•Noise Analysis

•Summary

3

ONT: Optical Network Terminal ONU: Optical Network Unit OLT: Optical Line TerminalRG: Residential Gateway VOD: Video On Demand IAD: Integrated Access Device

Service Delivery

4

Splitter POTS

STB TV

PC

RG

DVB DEMUX

VoD

TV

QT-200

GbE

QT-1100

RF

GbE

xDSL

QT-1100

IP MPLS/ATM Core

LOCAL ADS& CONTENT

Eth AggrNetwork

QT-1100

D/AHub OfficeHead End

GbE

QT-600 QT-600

HST-3000

NetComplete

Triple Play Network/Service ChallengesComplex Network Environments require Service Assurance Systems to minimize expensive dispatches

Cust Prem

Customer Data Traffic victimizes VoIP/Video

Misconfigured Premise Equipment

Firewalls

Inside wiring Cat 3, Coax

In-home Networking, HPNA, MoCA, Wireless, BPL

50% of Problems

Last Mile

Signal degradation.

Impulse noise Copper pair:

imbalanced lines,Bridge taps, etc.

Data applications don’t experience these problems because of TCP.

Voice and video experiences issues because of real-time nature.

25% of Problems

Core Ntwk

Mistakes here catastrophic

Misprovisioned tributaries and routes

Routing/Link over utilization….poor traffic engineering.

Poor circuit routing…..creates excessive delays

Signaling and Media Gateways

Mis-provisioning creates incorrectly routed calls

Signaling translations and incompatibilities.

Echo Canceller configurations

IGMP and RTSP Latency

Video Hub Head End

Poor content quality from content provider.

PCR Jitter from encoder, splicer or VOD source.

PID mappings or data table mappings from video source

Proper Trans-coding

15-20% of Problems 5-10% of Problems

5

QoS and QoE mapping for Video Services

QoE Parameters

Continuity ErrorPCR Jitter

Error Indicator Count

Video QoS parts

IP Packet Loss

IP Packet Jitter

RTP Packet Loss

RTP Packet Jitter

TCP Re-transmissions

Service Provisioning

IGMP Latency

RTSP Latency

Content Quality

Transport Quality

Video Stream Quality

Transaction Quality

PSI Table Data (Error)

Picture: blocking, blurring, edge distortion, visual noise

Audio: Lip sync, drop outs

Pixelization, tiling frame freezes, Blue screen

Service accessibility, Channel Change latency Pause, Play Latency

Video QoS

V-MOS

Note 1: requires payload decode analysis

6

IP Video Network Segment Test Focus-Broadcast

Content Quality

Transport Quality


Transaction Quality

HE QoS Network QoS

Access QoS Total QoS @ STB

PCR Jitter Packet Loss Packet,

Jitter

Distance, Period

IGMP Latency, PCR Jitter, PSI data, Error

Ind, BW, MPEG-TS Pkt Loss, Jitter, Period,

Distance, MOS

Content Quality


PCR Jitter, PSI Data, Error Ind, Compression – GOP, Type of

Coder, Bit Rate, Profile

Transport Quality

Transaction Quality


Transport Quality

Transaction Quality


Local Content Insertion

7

IP Video Network Segment Test Focus-VOD

Hub Office QoS Network QoS

Access QoS

Total QoS @ STB

PCR Jitter Packet Loss Packet Jitter

Distance, Period

RTSP (Trick) Latency, PCR Jitter, PSI data,,

BW, MPEG-TS Pkt Loss, Jitter, Period,

Distance, MOS

PCR Jitter, PSI Data, Compression

(GOP, Type of Coder, Bit Rate,

Profiles), Trick Play Latency

Content Quality

Transport Quality


Transaction Quality

Transport Quality


Transport Quality


Content Quality


Transaction Quality

8

CoreNetwork(ATM, IP)

GatheringNetwork

(IP)DSLAM

TV ProgramProviders

TV ProgramBroadcaster

MDF

VideoService

Platform

IP

STBATU-R

IP TV Installation

Local Content Added

Test Set STB/Modem emulation IP Video QoS check Broadcast & VOD

Network Specific Thresholds

IGMP signal

9

ATM/IP

DSLAM

xTU-C Replacement

xTU-R Replacement RAS IP Routing

ISP

TE Replacement

Focus on showing end service functionality

• Connectivity to the DSLAM

• Connectivity to ATM network

• Connectivity to IP network and beyond

xDSL Verification

11

10

Video QoS – Network Specific Thresholds

The QoS screen shows the critical quality parameters for a video flow:

PCR jitter measures this key parameter; if high the decoder can not properly decode the video payload: Pass < 100 mS

Latency is the IGMP latency, the time to change channels for Broadcast Video: time from IGMP request to Rx of first video packet: <200 mS

Cont. Err is the analysis of video Transport Stream packets which show the Continuity Error indicator set which = lost packet events: < 0.1%

Thresholds can be set in the HST for Pass, Fail, Marginal

If all items = Pass, no further analysis necessary by field tech.

If Error Indicator count = >0 then content problem

11

Drop Packet

MPEG Decoder

1) Perfect Video StreamInter Frame delay OK

typically 2-4 ms

2) Stream with jitter or rate under run

Inter Frame delay to long

3) Stream with jitter and/or rate over run

Inter Frame delay to short

Video rate i.e 3 Mbps 3 Mbps

STB Buffer

Buffer overflows

Buffer underflows

Ethernet packet contains up to 7 MPEG packets

MPEG packet 188 Bytes

MPEG over Ethernet Transmission issues

12

IP Video Testing

Video impairments– All channels?– One channel?

Time of day specific– A pattern?– Random throughout the

day? Temporal component

– Bursts?– Randomly spaced?

Critical Diagnostic Fault Conditions:

13

IP Video Testing

Video pixelizations– All channels? = yes

• Lost packet events will be seen on more than one stream

Lost packets – Analyze physical layer stats

• Ethernet I/F stats • ATM stats• DSL stats

– Sectionalization• If no errors are seen at the

physical layer of both I/F ‘s, then packet loss is up-stream of the DSLAM

IP Fault Resolution: Step #1

14

IP Video Testing

Video pixelizations– Physical layer problems? = yes– Check DSL stats for errors

• If yes, impulse noise most likely cause, if noise margin at least 6 dB.

• Review Bits/Tone graphs for notches – Notches = Noise problems

– Check packet layer stats• If CRC errors = loop noise problems • If Length errors, but no CRC then may be network

QoS /Buffer over flow issues, not loop noise problems

• If both most likely loop noise issuesEvaluate copper loop for narrow band and wide band

issues. Use temporal input to pinpoint sources.


15

IP Video Testing

Video pixelizations– Physical layer problems? = yes– Check VDSL stats for errors

• FEC errors: Un-correctable FEC errors• If yes, impulse noise most likely cause, if noise

margin at least 6 dB. • Review Bits/Tone graphs for notches

– Notches = Noise problems

– Check Packet stats– RTP Stats– RFC 3357– MDI

• If both most likely loop noise issuesEvaluate copper loop for narrow band and wide band

issues. Use temporal input to pinpoint sources.


16

IP Video Testing

Video pixelizations– All channels? = no

Lost packets – Analyze lost packet stats

• Typically none seen when trouble is one ch.

Analyze PCR jitter stats• PCR jitter problems are typically source

issues: – Head End trans-coding problems– Local Ad insertion problems

Analyze PSI Error stats– Typically a content issue

Error Indicator Count– Analyze count

• Corrupted content leaving the encoder


Yes

No

17

IP Video Testing

Analyze PCR jitter stats• PCR jitter problems are

typically source issues: – Head End trans-coding

problems– Local Ad insertion

problems• Report PCR Jitter stats with

Time-of-Day, and Channel or Program ID

PSI table data analysis– PAT and PMT data present

• Present• Present at or above a

threshold rate, typically every half second, but in MSTV network usually only every second or 1.5 seconds


For each video program stream the PID’s (Packet Identifiers) are shown for each piece: Video, Audio, & PSI table data. The Type and Description data is decoded from the stream as received. Unknown data includes data not included in the other categories such a Fill/PAD data or data marked as “reserved”.

18

Impulse Noise Standards Definition

VDSL is more susceptible to impulse noise events due to it’s use of a wider frequency spectrum than ADSL. Noise sources are being analyzed in several forms:– REIN (Repetitive Electrical Impulse Noise)

• Less than 1 ms in duration• No bit errors desired• INP mitigation

– PEIN (Prolonged Electrical Impulse Noise)• 1 to 10 ms in duration• No bit errors desired• INP mitigation

– SHINE (Single Isolated Impulse Noise Event)• Duration greater than 10 ms• Due to duration, bit errors will typically occur• No loss of sync is desired

19

Central heating/air - switch on/off A neighbour’s Jacuzzi (Motor) Television sets Video recorders PC scanner Faulty street lights. Faulty Metal Halide lighting Faulty fluorescent lighting External Hi/Lo Security Lights External low voltage lights Seasonal Lights Satellite Receiver AC Power faults near copper loops

00:00:06 5

15

15

Sources of Impulse Noise

Impulse count with 3dB below and above

threshold

Event capture

Impulse Noise Analysis

Proper Filter

20

Up to 5 MHz

Up to 15 MHz

Up to 30 MHz

Spectral Noise Analysis

Spectral noise analysis down to -145 dBm/Hz

Wider VDSL spectrum opens loop to new noise sources

Max Hold shows transient peak values

21

SAI

IP DSLAM

B-Box / SAI

VDSL

HST-3000STB

HPNA/VDSL in Home – VDSL2/Cu Noise/TDR

Splitter

CoaxAdapt

CPE

TWP

0-30MHzCopper Noise

VDSL2Sync

Analysis

HST as a troubleshooting tool HST connects to TWP or Coax, HPNA or VDSL Syncs to VDSL2, monitors copper, TDR analysis Monitors 0-30MHz spectral analysis: Transient and Impulse

noise Interactive or long term monitoring

22

IP Video Testing

Excessive IGMP Latency– All channels? = no

• Report as program source issue– All channels? = yes – Analyze time of day issues

• Random or a pattern• Report issues

– Review:» Review IGMP snooping hierarchy» Review traffic loading, special event issues» Multi-cast flows, are all channels available at

DSLAM» Temporal caching plan impacts


CoreNetwork(ATM, IP)

GatheringNetwork

(IP)DSLAM

MDF

Multi-castPlatform

IP

STBATU-R

IGMP signal

23

Video QoS – Packet Loss Correlation

The Pkt Loss screen shows the status of the physical layer:

ATM and DSL layer measures provide insight into the ADSL loop performance: Errors here indicate packet loss may be due to loop problems.

If no errors are seen on the access loop, then packet loss is North-bound of the DSLAM (in the network).

If no errors, then problems are typically in the network.


24

Video QoS Bandwidth & Pkt. Analysis

Individual bit rates are shown for each portion of the video stream.

Unknown data defined as any data, such as PAD bits to maintain a constant data rate, or private IP data over MPEG, that is detected is also shown.

Packet stats parameters are measured at the IP layer. IGMP Latency is the measure of the network components to complete a program change, i.e. changing the channel in Broadcast video.


25

MOS Score Evaluation

MOS score analysis for video and audio portions:

Video factor is like audio R-Factor showing a score built from network elements of the factors effecting the MOS score .

The same analysis is provided for the audio portion of the program.


26

PID Map & Combined Stream Data

Totals for up to 3 streams:

Combined total data rate for the streams under analysis, in this case 3.

Simultaneous analysis of multiple streams is helpful in locating problem sources.


27

IPTV troubleshooting Putting Physical Analysis Together

DSLAMMDF

Street cabinet

Distributionbox

ADSL2+modem

Not twistedNot shielded pair

NID

ADSL tests PassedCopper tests Passed

Copper tests Failed (impulsive noise, spectral)

18 m280 m1330 m

ADSL tests Failed(ADSL errors within UP stream)Copper tests Failed (impulsive noise, spectral)

ADSL Example:

28

IP Video Testing Focus Summary

– Video Services (Broadcast & VOD ) Verification and Quality analysis at the Video Packet Level and the Transport Stream (RTP/IP) level

– Features: • Service verification: Broadcast and VOD services

– GMP signaling emulation for Broadcast Video– RTSP signaling emulation for VOD

• Video Transport Steam packet flow analysis– Video QoS, PID map, PSI analysis, – Packet Loss and Jitter and correlation with physical layer

quality• Modes:

– Terminate: Res Gateway or STB emulation– Monitor: Through or Ethernet Bridge

– Integration into Process Improvement programs• Store and export results• Standard and custom scripts

29

Testing the Physical Medium is Not Enough

JDSU Application-Aware Solutions

Equip work force for IP service testing as well as circuit testing Minimize customer site visits with centralized fault location Meet customer due dates – get to revenue quickly Ensure IP service QoS metrics are met producing good QoE – reduce churn

Physical Medium

IPVOD

VoIP

Data

Video

Physical Layer

IPftp

HTTP

IPSignaling

Voice Content

Data Link Layer

Network Layer

Application Layer

TM

.

Documents

IP Video Quality Testing