AI Technology is Changing the Future of Video Compression · Video Live system Video Single Encoding Core Frame rate Prediction Model Video Analysis Compressed Video Offline Learning

IABM Copyright 2019@THEIABM www.theiabm.org

AI Technology is Changing the Future of

Video Compression

Stéphane Cloirec – Sr. Director, Video Appliances & Software

Harmonic


Introduction

Harmonic, the worldwide leader in video delivery technology and services, enables media companies and service providers to deliver ultra-high-quality broadcast and OTT video services to consumers globally. The company has also revolutionized cable access networking via the industry’s first virtualized cable access solution, enabling cable operators to more flexibly deploy gigabit internet service to consumers’ homes and mobile devices.

As the Senior Director of Video Appliances & Software at Harmonic, Stephane Cloirec leads the product management and strategic direction of the company’s video appliances, including encoding, transcoding, stream processing, as well as playout, automation and storage solutions. Cloirec joined Harmonic with 15 years of expertise in product management from companies such as JDSU, Thales, Grass Valley and Thomson Video Networks, where he led product strategy and software transitions initiatives. He holds a master’s degree in signal processing from the University of Rennes and an engineering degree in telecommunications and computer science from Centrale-Supelec, Paris.


Agenda

Video Compression History

Typical problems solved by AI

Video Compression Decisive Criteria

Implementations

AI Advantages

Machine Learning Types and Relation to Video Compression

Application Examples


Video Compression History

Multipass look ahead encoding

Motion Compensated Temporal Filter

First multichannel 1RU encoder

Pure software encoder

Content-Aware Encoding (CAE)

A long process!

Continuous Improvement


Artificial Intelligence:

A New Way Of Making Improvements!


Typical Problems Solved By AI

Processing large data sets

Decision helping

Predictions!

Detection


Benefits brought by AI and ML

• Speed

• Security

• Accuracy

Additional benefitsCost Reduction

• Machine replaces human

• For detection/prediction based activities

• Allows to do much more of it since it becomes cheap


Video Compression Decisive Criteria

Density

• Decisions

• Frame or field coding

• Macroblock size

• Picture type

• Bit rate allocation

• VQ measure

Latency

VQ/Bit rate trade off


Implementation!


Machine Learning applied to Video Compression

• Improved compression

Supervised learning

• Clustering video sources

Unsupervised learning

• Continuous improvements

Reinforcement learning


Implementing AI/ML for Video Compression

Learning parameters

Live System

Encoder Packager Mux

Offline Learning SystemTest Files


AI Advantages and Opportunities

• New Services deployed Quickly

• Customization

New opportunities, appropriate for SaaS

Development savings

• Replace years dev. by CPU time

Density improvements

• AI implementation using less CPU than human developed algorithms


Application Examples


AI

Dynamic Resolution Encoding

Dynamic Encoding Style Dynamic Frame rate

Encoding

Encoding Pre processing and Encoding

AI Encoding Application Examples


• Goal is to reduce video bitrate • Goal is to improve user QoE

Encoding Style

AI chooses encoding parameters depending on video input


AI chooses best resolution depending on video input

Can be combined all together

• Goal is to improve encoding density

Dynamic Frame Rate Encoding

AI chooses a lower frame rate depending on video input

• Any application (ABR, CBR, VBR)

480p 1080p 60fps 30fps5 Mbps 4 Mbps

• Application to Mobile (zero rating) • Application to IPTV (enable HD on SD lines)• Application to OTT streaming on Main screen



AI



Encoding




Dynamic Encoding Style

Offline Learning System

Test Files

Encoding Styles

Bitrate and VQ stats

Video Analysis

AIEncoding Style

Prediction Model

Encoder

CompressedVideo

Live system

Video

Encoding Core

Encoding StylePrediction Model

Video AnalysisCompressed Video




Test Files

Encoding Styles

Bitrate and VQ stats

Video Analysis

AIEncoding Style

Prediction Model

Encoder

CompressedVideo

Live system

Video

Encoding Core

Encoding StylePrediction Model


Demonstrates bitrate saving vs traditional VBRup to 20% for the most complex content


AI



Encoding




Encoder

CompressedVideo

Live system

Video

Single Encoding Core

ResolutionPrediction Model



Test FilesVideo Analysis

AIResolution

Prediction Model

x Resolution Encoding VQ stats



Encoder

CompressedVideo

Live system

Video


ResolutionPrediction Model



Test FilesVideo Analysis

AIResolution

Prediction Model

x Resolution Encoding VQ stats


Perceived video quality improvement

for all type of contents


AI



Encoding




Encoder

CompressedVideo

Live system

Video


Frame ratePrediction Model

Video Analysis Compressed Video


Test Files Video Analysis

AIFrame rate

Prediction Model

x FR EncodingVQ stats



Encoder

CompressedVideo

Live system

Video


Frame ratePrediction Model

Video Analysis Compressed Video


Test Files Video Analysis

AIFrame rate

Prediction Model

x FR EncodingVQ stats


Measured 35% CPU usage optimization

on average for typical video content mix


BW saving QoE Scalability



Dynamic Frame rate Encoding

V V X

X X V

V VX

Summary


AI research started several

years ago at Harmonic

EyeQ is the first product to use AI techniques for

OTT (ABR)

AI techniques expended to other

applications(CBR, VBR)

Exciting Future! More optimization, more customization, nextgen video standards, …

Conclusion


• https://www.pyimagesearch.com/2016/06/20/detecting-cats-in-images-with-opencv/

• https://medium.com/@skychain.global/skychain-ai-for-lung-cancer-detection-released-e246ba694228

• http://didarc.com/en/news/artificial-intelligence-and-autonomous-vehicles

Picture Credits

https://www.pyimagesearch.com/2016/06/20/detecting-cats-in-images-with-opencv/

https://medium.com/@skychain.global/skychain-ai-for-lung-cancer-detection-released-e246ba694228

http://didarc.com/en/news/artificial-intelligence-and-autonomous-vehicles


Thank You.

Broadcast Asia 2019

©2019 Harmonic Inc. All rights reserved worldwide.

Documents

AI Technology is Changing the Future of Video Compression · Video Live system Video Single Encoding Core Frame rate Prediction Model Video Analysis Compressed Video Offline Learning