52
JPEG Standards ‘Are we prepared for the next disruptive evolution in digital imaging markets?’ Prof. Peter Schelkens Spokesman of the JPEG Committee 4 June 2015 1 www.jpeg.org

JPEG Standards

  • Upload
    vonhan

  • View
    234

  • Download
    2

Embed Size (px)

Citation preview

Page 1: JPEG Standards

JPEG Standards!‘Are we prepared for the next disruptive evolution in digital imaging markets?’ Prof. Peter Schelkens Spokesman of the JPEG Committee

4 June 2015 1 www.jpeg.org

Page 2: JPEG Standards

Digital Photography

4 June 2015 www.jpeg.org 2

DIGITIZATION = FACT!

•  The analog to digital transition was disruptive!•  3B+ pictures/day uploaded on social media!•  400B+ pictures/year on digital cams !!! NEXT DISRUPTION IS UPCOMING!•  Images become CUBES!

GROWTH POTENTIAL (CAGR) 3.8 % MARKET SIZE (2016) 82.5 B€

Page 3: JPEG Standards

JPEG Committee •  Joint Photographic Experts Group!

–  ISO/IEC JTC1/SC29/WG1!–  ITU-T SG16!

•  Standardization = Assuring INTEROPERABILITY!

4 June 2015 www.jpeg.org 3

Page 4: JPEG Standards

JPEG Family of Standards

4 June 2015 www.jpeg.org 4

Page 5: JPEG Standards

JPEG’s still growing ecosystem

4 June 2015 www.jpeg.org 5

Source: KPCB 2014 Internet Trends, estimates based on publicly disclosed company data.

1995-96 Technology and Engineering Emmy award (together with MPEG-2)

Page 6: JPEG Standards

JPEG 2000 hits professional markets

4 June 2015 www.jpeg.org 6

2015 Technology and Engineering Emmy award (JPEG 2000 interoperability)

Page 7: JPEG Standards

JPEG 2000 framework

4 June 2015 www.jpeg.org 7

Part 1/13 Core Codec

Part 2 Extensions

Part 10 3D Extensions

Part 9 JPIP

Part 3 MJPEG 2000

Part 6 JPM

Image Codec" Tools File Format

Part 8 JPSEC

Part 11 JPWL

Part 14 JPXML

E2E Toolset Extra Functionality"

Codec Tools

Part 4 Compliance Testing

Part 5 Reference Software

Part 12 ISO Base Media

Page 8: JPEG Standards

JPEG XR bridges gap

4 June 2015 www.jpeg.org 8

Complexity

Performance

JPEG

JPEG XR

Page 9: JPEG Standards

JPEG vs JPEG 2000 vs JPEG XR

4 June 2015 www.jpeg.org 9

Page 10: JPEG Standards

JPEG XR not widely used

4 June 2015 www.jpeg.org 10

Page 11: JPEG Standards

JPSearch

4 June 2015 www.jpeg.org 11

F. Temmermans, F. Dufaux and P. Schelkens, “JPSearch: Metadata Interoperability During Image Exchange”, IEEE Signal Processing Magazine, vol.29, no. 5, pp.134-139, 2012

F.#Temmermans,#F.#Dufaux#and#P.#Schelkens,#“JPSearch:#Metadata#Interoperability#During#Image#Exchange”,#IEEE#Signal#Processing#Magazine,#vol.29,#no.#5,#pp.134M139,#2012#

Page 12: JPEG Standards

Standards in Progress

4 June 2015 www.jpeg.org 12

Advanced Image Coding (AIC) !–  Evaluation methodologies and metrics !

JPEG AR!–  Image exchange in Augmented Reality!

JPEG Systems!–  Consolidated system layer structure !

JPEG XT!–  JPEG forward/backward compatible HDR compression!!

Page 13: JPEG Standards

Advanced Image Coding (AIC) •  Advanced Image Coding!

–  Part 1: Guidelines for codec evaluation!–  Part 2: Evaluation procedure for assessing visually lossless coding !

!•  Call for information issued in February 2015 to receive information on next

generation still image compression with superior compression efficiency, as well as other useful features in future multimedia applications !!

•  PCS2015 Feature Event - Evaluation of current and future image compression technologies!!

•  For further information see www.jpeg.org and www.pcs2015.org !

4 June 2015 www.jpeg.org 13

Page 14: JPEG Standards

JPEG XT design principles •  Exif and JFIF use APP marker of JPEG !

–  Reserved for application segments!

4 June 2015 www.jpeg.org 15

APP marker (0 to 15)� Format�APP0� JFIF, JFXX�APP1� Exif�APP2� ICC Profile�APP3� JPSearch Part2�APP14� Adobe�

SOI� APP1� DQT� DHT� SOF� stream�SOS� EOI�

WG1N5725�

Start of Image (SOI)

APP11 (Residual JPEG XT)

JPEG-1 Code-stream

End of Image (EOI)

JPEG XT File

Page 15: JPEG Standards

JPEG XT design principles •  Two-layer coding, with base layer a legacy JPEG coded LDR and enhancement

layer the residual to produce the HDR!•  Enhancement layer uses a maximum of JPEG Legacy coding tools !

4 June 2015 www.jpeg.org 17

Page 16: JPEG Standards

JPEG XT status

4 June 2015 www.jpeg.org 20

Part Title WD CD DIS FDIS IS

1 Core Coding System 13/01 13/07 14/01 - 14/10

2 Coding of High Dynamic Range Images 13/10 14/01 14/04 - 14/10

3 Box File Format 14/05 14/07 15/06 - 16/02

4 Conformance Testing 15/02 15/06 16/02 - 16/06

5 Reference Software 14/07 15/06 16/02 - 16/06

6 IDR Integer Coding 14/05 14/07 15/06 - 16/06

7 HDR Floating-Point Coding 14/05 14/07 15/06 - 16/02

8 Lossless and Near-lossless Coding 14/07 15/02 15/06 - 16/02

9 Alpha Channel Coding 14/10 15/02 15/06 - 16/02

Page 17: JPEG Standards

Questions?

4 June 2015 www.jpeg.org 21

Page 18: JPEG Standards

JPEG Markets

Andrea de Polo

4 June 2015 22 www.jpeg.org

Page 19: JPEG Standards

Content

•  JPEG today!•  JPEG in the commercial sector !•  JPEG in the research sector !•  JPEG and “our” save-our-memory plans!•  JPEG tomorrow!

4 June 2015 www.jpeg.org 23

Page 20: JPEG Standards

JPEG Today

•  The JPEG standard made its debut in 1992, and since then it has become the most widely used lossy compression format for digital images.!

•  Over a trillion pictures have been created in JPEG format and this trend continues. Millions of images are shared every day among friends and family via e-mails, photo-hosting Web sites and the enormously popular social networking sites, like Facebook and Flickr, where users can even instantly upload photos taken with their mobile phone cameras. Digital photos are also widely printed at homes or through commercial printing services. !

4 June 2015 www.jpeg.org 24

Page 21: JPEG Standards

JPEG in the Commercial Sector •  According to Facebook, !

over 200M images are uploaded !online/daily!

•  According to 1.000memories.com !every year over 400 billion !pictures are taken !

•  According to e.g. Infotrends, Statista over 75% of online content is provided in JPEG format!

4 June 2015 www.jpeg.org 25

Page 22: JPEG Standards

JPEG in the Commercial Sector •  JPEG and TIFF widely used !•  RAW popular among photographers!•  JPEG 2000 used in medical, archival,

broadcasting, security & movie industries!

4 June 2015 www.jpeg.org 26

Page 23: JPEG Standards

JPEG in the Research Sector

•  JPEG today is still very active and popular among research community !

•  Several parts of JPEG have been adapted recently and annual meetings (next in Warsaw on June 24-26th) aim to measure the progression in the advances in the various JPEG components covering various aspects such as: data security, IPR and licensing, personal privacy, file optimization, backward compatibility, file optimization, data integrity, metadata, etc. !

4 June 2015 www.jpeg.org 27

Page 24: JPEG Standards

JPEG and “our” save-our-memory plans!Interoperability / Harvesting / Integration into existing ecosystems

•  JPEG is a de-facto standard in digital photography. Most digital cameras can produce JPEGs, and many can only produce files in JPEG format.!

•  JPEG format is a source of data that can be used for the purpose of detecting forged images and some forensic analysis to criminal-cases. In that stored a technical information is called “Metadata “. These Metadata tags contain information about quantization of matrixes, Huffman code tables of full image. !

•  JPEG tags contain important information about the photo including shooting conditions and parameters such as light levels, shutter speed information, make or model of the camera and lens, lens focal length, flash usage, color profile information, geospatial information.!

4 June 2015 www.jpeg.org 28

Page 25: JPEG Standards

JPEG and “our” save-our-memory plans!Interoperability / Harvesting / Integration into existing ecosystems

•  The basic analysis methods verify the validity of Exif tags in the first place in an attempt to find information. Exif tags added in post processing by certain editing tools, check for capture date vs. date of last modification.!

•  TIFF, JPEG, JPEG 2000 are all considered popular and widely accepted for long term preservation….. However, it is always VERY important to consider in your ecosystem a workflow that fully supports and preserves your existing metadata (more info on this can be find at IPTC-Photo Metadata group). !

4 June 2015 www.jpeg.org 29

Page 26: JPEG Standards

4 June 2015 www.jpeg.org 30

Page 27: JPEG Standards

JPEG Tomorrow

•  JPEG is working on new features while maximizing backward/forward compatibility.!

•  New tools and settings like HDR, geo-references and metadata support are considered.!

•  New software appears in the market: i.e. JpegMini Pro, Topaz DeJpeg, Jpegger, Lossless Photo Squeezer, Advanced Jpeg Compressor (AJC), ImageOptim, Radical Image Optimization Tool, Png Gauntlet, Kraken.io, EWWW Image Optimizer, WP Smush Pro, BPG…!

•  JPEG standard consortium is there to watch what is available in the market, encourage exchange of information, promote cooperation among users and service providers in order to build for the future always a better and widely accepted solution.!

4 June 2015 www.jpeg.org 31

Page 28: JPEG Standards

More information

4 June 2015 www.jpeg.org 32

Andrea de Polo !Consultant!!!culturalheritageconsultant.com [email protected]!

Page 29: JPEG Standards

Questions?

4 June 2015 www.jpeg.org 33

Page 30: JPEG Standards

JPEG Standards (cnt’d)!‘Are we prepared for the next disruptive evolution in digital imaging markets?’ Prof. Peter Schelkens Spokesman of the JPEG Committee

4 June 2015 34 www.jpeg.org

Page 31: JPEG Standards

JPEG Privacy JPSEC/DCI Legacy

4 June 2015 www.jpeg.org 35

Storage DCP Digital Cinema

Package

Key Image

Audio Subtitles Captions

Auxiliary Data

Digital Cinema Distribution Master * ( DCDM *)

JPEG2000 Decompression Decryption

Dencryption Decryption Decryption

Security Manager

Watermarking

Watermarking

Overlay Projector

Sound System

Automation System

Secure Media Block

A.#Bohó,#G.#Van#Wallendael,#A.#Dooms,#J.#De#Cock,#G.#Braeckman,#P.#Schelkens,#B.#Preneel#and#R.#Van#De#Walle,#“EndMtoMEnd#Security#for#Video#DistribuWon”,#IEEE#Signal#Processing#Magazine,#issue#Signal#Processing#in#the#Encrypted#Domain,#vol.30,#no.#2,#pp.97M107,#2013#

Page 32: JPEG Standards

JPEG Privacy •  Solutions to provide privacy protection to legacy JPEG !

4 June 2015 www.jpeg.org 36

Unprotected sharing space (e.g. Dropbox, Box)�

Sender Device� Receiver Device�

Codestream�Metadata�

PROTECTION�

Page 33: JPEG Standards

JPEG Privacy •  Solutions to provide privacy protection to legacy JPEG !

4 June 2015 www.jpeg.org 37

Unprotected sharing space (e.g. Dropbox, Box)�

Sender Device� Receiver Device�

Codestream�Metadata�

PROTECTION� PROTECTION�

Page 34: JPEG Standards

JPEG Privacy Metadata Protection •  Exif and JFIF use APP marker of JPEG !

–  Reserved for application segments!

4 June 2015 www.jpeg.org 38

APP marker (0 to 15)� Format�APP0� JFIF, JFXX�

APP1� Exif�

APP2� ICC Profile�

APP3� JPSearch Part2�

APP14� Adobe�

SOI� APP1� DQT� DHT� SOF� stream�SOS� EOI�

WG1N5725�

Page 35: JPEG Standards

JPEG Privacy App11

4 June 2015 www.jpeg.org 39

SOI�

APP1 (Exif)�

EOI�

APP11 (protected metadata

[*])�

JPEG compatible codestream with data protection�

Image data�

APP11 (protected

image data [*])�

•  Protected Exif information (described in Box-format)!

•  Protection methods (refer to JPSEC)!–  Cipher!–  Digital signature!–  Hash function!–  Encryption Tool Registration Authority (like JPSEC-

RA)!•  Default tools and additional tools (refer to JPSEC)!•  Protected image data contains partial DCT

coefficients to overwrite the non-protected data!(min. region = 8x8 DCT block) !

Page 36: JPEG Standards

JPEG Privacy App11

4 June 2015 www.jpeg.org 40

SOI�

APP1 (Exif)�

EOI�

SOI�APP1 (Exif)�

EOI�

APP11 (protected

metadata [*])�

JPEG-1 decoder�

JPEG Privacy decoder�

APP1 (Exif)�

APP1 (Exif)�

original JPEG codestream�

JPEG compatible codestream with data protection�

Image Data�

Image data�

APP11 (protected

image data [*])�

Image Data�

APP11 (protected

metadata [*])�

Image data�

APP11 (protected

image data [*])�

[*] includes non-protected information�

APP3 (JPSearch)�

APP3 (JPSearch)�

APP3 (JPSearch)�

Page 37: JPEG Standards

JPEG Privacy •  “A JPEG image repository with controlled access”. !•  Features: !

–  Access control to specific images is defined with rules (privacy policies).!–  Policies are defined either by the service provider or by the image

owner.!•  Policies define conditional access to information on!

–  User: individual, group, location, role, … !–  Context: date and time, number of accesses, action (view, download,

…), … !–  Image: quality, geolocation, author, date, semantic information (using

e.g. RDF), … !–  Action: read, update, delete, … !

4 June 2015 www.jpeg.org 41

Page 38: JPEG Standards

JPEG Privacy!MIPAMS

4 June 2015 www.jpeg.org 42

Object Registration

Service

Policy Service

Content Service

Certification Authority

User Application

Wor

kflo

w M

anag

er

UserSearch Service

ReportingService

Authorization Service

Authentication Service

Certification

Image, Policy and Report Search and Retrieval

Reporting

Certification

Image Registration

Key Retrieval

Reporting

Public Key Retrieval for Token Verification

User Token Request and User Registration

Protection Service

Reporting

Resource Upload, Download and Encryption

Privacy Policies Definition

Authorisation and Key Retrieval

Key Generationand Registration

http://dmag.ac.upc.edu/mipams/

Page 39: JPEG Standards

On the Horizon JPEG PLENO !targets a standard framework for the representation and exchange of new imaging modalities such as light-field, !point-cloud and holographic imaging.!

4 June 2015 www.jpeg.org 45

Page 40: JPEG Standards

On the Horizon •  7D function P(a,q,l,t,x,y,z)!

–  view point!–  wavelength!–  time!

4 June 2015 www.jpeg.org 46

Page 41: JPEG Standards

JPEG PLENO design principles •  One or limited number of representation models!•  Well defined, specific and useful milestones !•  Backward compatible with legacy JPEG !

4 June 2015 www.jpeg.org 47

Panorama!

360 degree!

Spatial photo!

Point cloud!

Light field!

Holography!

Page 42: JPEG Standards

Spatial Photography •  Motion parallax!

4 June 2015 www.jpeg.org 48

Page 43: JPEG Standards

Point-cloud Photography

4 June 2015 www.jpeg.org 49

Page 44: JPEG Standards

Light-field Photography

4 June 2015 www.jpeg.org 50

Page 45: JPEG Standards

Light-field Photography

4 June 2015 www.jpeg.org 51

Page 46: JPEG Standards

Holography

4 June 2015 www.jpeg.org 52

LIGHT-FIELD Rays with position + orientation

HOLOGRAM Interference = superposition of waves

Page 47: JPEG Standards

JPEG PLENO Challenges

4 June 2015 www.jpeg.org 53

•  Source coding technologies !•  Image analysis and

reconstruction algorithms!•  Image restoration!•  Better understanding of

human visual system !

•  Advanced (plenoptic) lenses !•  Higher resolution sensors !•  Higher dynamic range

sensors!•  Advanced (3D) printing!•  Storage and distribution!•  QoE tools!•  IPR!

•  Images move from static entities to dynamic givens!

•  Classical printing industry is not prepared to adequately address this evolution!

•  IPR!

Scientific Technology Business

Page 48: JPEG Standards

JPEG PLENO Workshop!Warsaw, Poland – June 23rd, 2015 – Marriott Hotel Warsaw

14:00 Touradj Ebrahimi (JPEG Convenor - EPFL): "JPEG PLENO - Introduction and Scope" Light-fields 14:15 Christian Perwaß (Raytrix GmbH, Germany): "Metrically Calibrated Multi-focus Plenoptic Camera and its Applications" 14:40 Joachim Keinert (Fraunhofer IIS, Germany): "Lightfield media production using camera arrays - use cases and requirements" 14:55 Peter Kovacs (Holografika, Hungary): "Light Field Displays" 15:20 Atanas Gotchev (Tampere University of Technology): "Content creation for light-field displays" 15:35 Roger Olsson (Mid Sweden University): "Objective evaluation and SotA compression solutions for plenoptic image content" 15:50 Discussion on compression of light field data (Requirements, use cases, technologies) !!Point-clouds 16:30 Rufael Mekuria (CWI Netherlands): "Point Cloud Compression" 16:45 Discussion on compression of point cloud data (Requirements, use cases, technologies) !!Holography 16:55 Małgorzata Kujawinska (Warsaw University of technology): "Holographic capturing and rendering systems, suitable data representations for phase and amplitude" 17:10 Frederic Dufaux (TELECOM ParisTech, France): "Digital Holography Compression" 17:35 Discussion on compression of holographic data (Requirements, use cases, technologies) !17:50 Conclusions!

4 June 2015 www.jpeg.org 54

Page 49: JPEG Standards

Conclusions •  JPEG standard is further nurtured !

!•  IPR and privacy extensions in preparation!

!•  New imaging modalities are being addressed !

•  Open calls to contribute to new standardization efforts in!–  Still Image Coding!–  JPEG PLENO!–  JPEG Privacy!

•  Workshop planned at 70th ISO/IEC JTC1/SC29/WG1 (JPEG) Meeting – Lucca, Italy – October 19, 2015 - October 23, 2015 !!

4 June 2015 www.jpeg.org 56

Page 50: JPEG Standards

More information

4 June 2015 www.jpeg.org 57

Prof. Touradj Ebrahimi!JPEG Convener!!!École Polytechnique Fédérale !de Lausanne (EPFL)[email protected][email protected] !

Prof. Peter Schelkens!JPEG Public Relations Chair!JPEG Coding & Analysis Chair!!Vrije Universiteit Brussel - [email protected][email protected]!

www.jpeg.org/contact.html

Page 51: JPEG Standards

Acknowledgements !!David Blinder, Tim Bruylants, Jamie Delgago, !Touradj Ebrahimi, Philippe Hanhart, Takaaki Ishikawa, !Martin Rerabek, Thomas Richter, Peter Schelkens, !Frederik Temmermans!

4 June 2015 www.jpeg.org 58

Page 52: JPEG Standards

Questions?

4 June 2015 www.jpeg.org 59