Using x265 for high quality UltraHD VOD encoding Tom Vaughan, VP and GM, Video

Changchun

Beijing

Taipei

Chennai

- Founded in 2009 - 150+ engineers worldwide - 7 locations - 4 Business Units

- Video - Libraries (x265, UHDkit, UHDcode) - Support - Software Development Services

- Computer Vision and Imaging - Machine Learning - Compiler Technologies

Champaign

St. Louis

Saratoga

2

4 ½ years after the HEVC standard was ratified… • Broadcast video world is steadily transitioning to HEVC.

– New broadcast standards are based on HEVC. • HEVC decoding supported on TVs, smartphones, and PCs

– Most new TVs sold in US are “smart”, supporting 4K HDR playback • 168.1 million people in the US will use an internet-connected smart TV in 2017 • Millions more used connected set-top boxes.

– Leading smartphones support HEVC HDR playback • Flag ship phones of Samsung, LG, and other leaders

– Chips from Intel, NVIDIA, AMD have HEVC HW decoding and encoding • Support has been around of for a couple of years already • Intel Kaby Lake has protected 4K HDR HEVC playback support

HEVC Adoption Accelerating

3

Roku Ultra 38.9 million US users Amazon FireTV 35.8 million US users

Google Chromecast 36.9 million US users

Apple TV 21.3 million US users

HEVC is mandatory in the Connected Home

4

New TVs and connected set-top boxes support HEVC

Apple adding HEVC to all platforms (iOS, MacOS, tvOS, HLS, Safari)

5

6

This fall, Apple will update 1 Billion+ Apple devices with HEVC capability

825 million iPhones

185 million iPads

150 million Macs

• All leading 2017 mobile devices supporting HEVC natively Apple iOS 11, MacOS High Sierra, tvOS 11 HEVC Video and HEIF Photo/Image support

Android: HEVC support is mandatory Samsung Galaxy S8 Sony Xperia XZ Premium LG G6 Samsung Galaxy Book 12 Samsung Galaxy Tab S3 Microsoft Surface notebooks The majority of new PC chip sets support HEVC natively MacOS High Sierra: updates all recent Macs with HEVC support

HEVC essential on consumer compute devices

7

8

The choice of leading video services and encoder vendors

• Started in February 2013 • Public in July 2013 • Based on x264 • Available under GPL v2 open source or commercial license

• Used by many leading video services and encoding solution providers

• Supports the full HEVC specification

9

…and many more

• All HEVC profiles and levels – Main, Main10, Main12, Main Still Picture – 4:2:0, 4:2:2, 4:4:4 variants

• HDR 10, HDR 10+, Dolby Vision HDR • All HEVC color spaces and transfer functions • Average Bit Rate, Constant Bit Rate, Constant

Rate Factor (constant quality), Constant QP, Video Buffer Verification, 2 pass, N pass, 2 pass with spatial distortion optimization, lossless

• 10 performance presets • Lookahead with scene and flash detection • 6 levels of rate distortion optimization • Temporal scalability • Wavefront Parallel Processing • Frame parallelism • Slice parallelism • Sample Adaptive Offset, Deblocking

• Fully configurable encoding - # ref frames, min/max CU size, RECT, AMP, GOP structure, keyframe interval, motion search method and range, subpel refinement, cu lossless

• Limit Modes, Limit Refs • Psychovisual optimizations • Adaptive Quantization • CU Tree optimization • Analysis Save, Analysis Load 2 pass • Tune psnr, ssim, grain, zero-latency, fast-

decode • Pools – assign x265 to specific sockets,

threads • Region of Interest optimization • VUI and SEI • Comma Separated Value logging (job level or

frame level) • Documentation is part of source code • Online Documentation x265.readthedocs.io

10

x264 @ 400 kbps

11

x264 @ 800 kbps

12

@ 400 kbps

13

HEVC can deliver twice the compression efficiency of AVC, but you can’t get that efficiency for free… it can take 5x or more compute resources. Video service providers want the benefits, but are they ready to increase their encoding budget 5x or more? Hardware encoders are fast and power efficient, but they have fixed capabilities, and they can’t match the quality and flexibility of software encoders. ASIC and FPGA encoders can cost $5K or more per 4K channel, depending on the host system.

HEVC Encoding Challenges

14

• Constantly improve our speed vs. efficiency curve – Algorithmic enhancements to make faster, better decisions

• Fully utilize every ounce of compute power – Optimize every kernel for max performance on any processor – Run enough tasks to keep many-core servers fully utilized – Load balance to keep every core/thread busy with useful work

• Leverage hardware accelerators

• New encoder architectures

How we are meeting these challenges+

15

Hand Optimized Assembly Code

• Up to 89X improvement in kernel unit-tests; average of 14X – Leverage SIMD instructions (SSE,

SSE2, SSE3, AVX, AVX2, AVX-512) – Best-gen assembly chosen based

on width of data elements used • Rigorous testing to ensure bit-

exact optimizations • 5x average performance boost

0

10

20

30

40

50

60

70

80

90

100co

stCo

effR

emai

nch

rom

a_p2

s[32

x32]

[i422

]cop

y_ps

[4x8

]in

tra_

dc_1

6x16

[f=1]

chro

ma_

p2s[

8x2]

sub_

ps[8

x8]

addA

vg[4

x4]

copy

_cnt

[4x4

][i4

22]s

atd[

32x1

6][i4

22]s

atd[

32x6

4]id

ct32

x32

chro

ma_

hpp[

4x32

][i4

20]a

ddAv

g[12

x16]

copy

_pp[

8x8]

intr

a_an

g_4x

4[30

][i4

20]a

ddAv

g[16

x4]

lum

a_vs

p[4x

8]in

tra_

ang_

4x4[

3]lu

ma_

vss[

16x6

4]lu

ma_

hps[

12x1

6]lu

ma_

hpp[

12x1

6]ch

rom

a_vp

s[12

x16]

lum

a_vs

p[32

x32]

sse_

pp[3

2x32

]lu

ma_

hv[3

2x24

]ad

dAvg

[32x

24]

[i420

]cop

y_pp

[16x

32]

intr

a_an

g_8x

8[14

]in

tra_

ang_

8x8[

12]

sad[

16x8

]qu

ant

lum

a_vp

s[64

x32]

lum

a_vp

p[64

x16]

lum

a_vp

p[32

x16]

sad_

x4[8

x32]

intr

a_an

g_16

x16[

13]

sad_

x3[1

6x12

]sa

d_x4

[32x

64]

Bene

fit o

ver C

-cod

e (X

-fact

or)

16

0

0.5

1

1.5

2

2S Intel® Xeon® processor 2699 v4 2S Intel® Xeon® Platinum 8180 processor

Performance Metric: Frames per second for 10-bit 1080p & 4K with main profile

x265 on Xeon Processor Scalable Family

56% faster

x265 completes file-based encoding with Intel® Xeon® Platinum 8180 processor 1.6X faster per core

We are seeing >50% more x265 performance per core on the new Xeons 17

• Video encoding involves many serial processes – The block you are trying to encode depends on the way neighboring

blocks were encoded, and it may reference blocks which must be encoded before you can make those references

• x265 can run multiple frame encoders (5 is a reasonable limit) • Each frame encoder calls multiple row encoders in a wavefront • Each row encoder calls CTU encoders, sequentially • CTU encoders partition into CUs, calling multiple CU encoders • CU encoders can (optionally) use parallel motion estimation

(--pme) or parallel mode decision (--pmode)

x265 Parallelism

18

• ABR requires generating multiple quality tiers for streaming – Different quality tiers may be different bitrates/resolutions

• When compared to AVC, HEVC blocks are 16X larger resolution can be maintained for much lower bitrates without visible impact

– Streaming server selects “best-fit” tier based on feedback from client • CRF + VBV is a great choice to generate different quality tiers

– Traditionally known as capped-VBR encodes – Much faster than 2-pass encoder, with near identical quality

• App can instantiate multiple x265 instances, one per quality tier – --pools option can pin one instance to few threads in one socket,

limit interference from one instance to another – Each instance can handle one bitrate/resolution/color space

x265 Encoding for Adaptive Bit Rate Streaming

19

10 performance presets (speed vs. efficiency)

VOD providers typically use --preset slow or slower – Encode once, stream many times – Cost of encoding is outweighed by lower BW and/or higher quality

x265 VOD Offline Encoding

20

-1.4

-1.2

-1

-0.8

-0.6

-0.4

-0.2

00 10 20 30 40 50 60 70 80 90

Qua

lity

(BD

SSIM

) Speed

veryslow slower

slow

medium

fast

faster

veryfast

superfast

ultrafast

• New algorithms work their way into presets slowly – Revising presets takes a lot of experimentation, and so we don’t do it often

• For VOD, some other options that may be beneficial --dynamic-rd [0…4] increases RDO when quality is capped by VBV

• Tailored specifically for capped-VBR encodes

--ssim-rd: SSIM-based RDO (as opposed to SSE-based) • Considerable improvements to visual quality

--pmode/--pme parallel ME and mode decision • Provides benefits on CPUs with larger core counts, but not work-efficient

x265 VOD Offline Encoding

Confidential MulticoreWare, Inc. 21

• For real-time encoding, use one of the fastest presets • Our biggest challenge is 4K, 10 bit, at 60 FPS

– A single instance of x265 can’t achieve this, even on the fastest server – We just don’t have enough parallelism to utilize all hardware threads

efficiently, even with a 4K encode • --pme and --pmode may help, but are not work-efficient

– We have a commercial platform (UHDkit) built on top of x264 and x265, which further parallelizes encoding

• Can easily achieve 4K, 10 bit 60P encoding in real-time • Breaks video into GOPs, runs multiple GOP encoders in parallel • New “Live 2 Pass” design will further optimize our performance, reduce latency

x265 Real-time Encoding

22

• RD level 6

• Picture size optimized presets

• Optimize 360 degree video (projected sphere) encoding

• Optimized slice-parallel encoding (extreme low latency)

• Improved heuristics for block size selection

• Leverage hardware encoders as fast first pass / lookahead

• AVX-512 SIMD optimizations

x265 Roadmap

23

Capability x264 x265 UHDkit Encode uncompressed video to AVC

Support the full range of AVC specifications

Encode uncompressed video to HEVC

Support the full range of HEVC specifications

Parallel encoding

2x Faster multi-bit rate encoding

Constant Performance Real-time Encoder

2 pass Capped VBR from a single instance

Auto monochrome detection / acceleration

Text detection and optimization

Advanced analysis and prefiltering

UHDkit - powerful capabilities built on x264/5

24

• Multi-encoder: encode one video to multiple bit rate tiers – 2X more compute-efficient vs. independent x265 instances – Produces multiple quality tiers in one UHDkit instance

• Capped VBR mode: Avoid abrupt quality hit when VBV kicks in – Complex region -> VBV caps bitrate -> abrupt quality jumps – Selectively re-encode only those GOPs to make transition more smooth

Encoding for ABR Streaming with UHDkit

25

• GOP Parallel mode enables live 4K 10bit 60fps encoding – Video split into parallel chunks encoded by separate x265 instances – Share rate-control info for unified rate-control, VBV compliance

• Live 2 Pass: a more efficient live encoder – Two encoding passes in tight sequence – Share analysis from a fast first pass to actual encoding pass – Refine and leverage analysis from first pass to reduce work required

• Performance Control System optimizes quality, maintains FPS – PCS dynamically reconfigures x265 based on achieve frame rate

• Frame rate < target: Speed up encoder to avoid frame drop • Frame rate > target: Increase analysis to achieve higher quality

Real Time Encoding with UHDkit

26

• Hardware Acceleration

• Content Adaptive Encoding

• Cluster encoding (master instance with slave instances)

• Noise/grain detection and modeling

• Advanced noise/grain reduction filters

• Optimized transcoding

UHDkit Roadmap

27

Thank you

For more info, see me today, visit x265.org or contact me at license@x265.com

28