Quest for an Quest for an Image Interchange and Image Interchange and

Mastering FormatMastering Format

HPA Technology Retreat

February 23, 2006

Jim Houston

© 2006 A.M.P.A.S. ®

Academy of Motion Picture Arts Academy of Motion Picture Arts and Sciencesand Sciences

Science and Technology CouncilScience and Technology Council

Advanced Technology ProgramsFile Format Committee

File Format Project

Develop a new framework for managing

image data,

metadata, and

color transformations

Current Problems

• non-standard RGB color spaces• log/lin/gamma conversions• more multi-facility productions• ‘secret sauces’ for output and preview• chasing calibration - variability of film developing• many digital master versions• unknown colorimetry from digital cameras

DigNeg Files

CleanScans

Dust Busting

Scanning

TelecineTelecine

Setup

RawScans

Scan Pre-Grading

Digital Camera

Film CameraMonitor

3DLUT Preview

Monitor

Jim Houston, Feb 23, 2006

GradedScans

Scans

CamFiles

RAWfiles

CamFiles

De-Mosaic Color SpaceConversion

Device

Files

Colorist Operation

Operation

Diagram of a Digital Workflow

Render CGI Elements

2D Artwork 2DFiles

CGI Files

Compositing

D.I.

DCDM

HDTV

Video

RGBA

RGBA

RGBRGB

ARCHIVE

DCPCompress/Encrypt

Film Recorder

(pre-graded)

3DLUT for eachOutput Media

Digital Projector

SoftwareColor Corrector

HardwareColor Corrector

Other Media Trim

Corrections

DSM

RenderCC fileswith3DLUTfor eachoutputmedium

Color & Light

Monitor

RenderCC files

RGBA

Tape

Tape

Audio

Metadata

DVD

Roto

Tracking

Paint either

On Set SampleGrading

Reference Output Media

D.I. Grade Corrections

DCDM TrimCorrections

Represents color modifications sections

Monitor

Color Correctionwith Trim passes

ReferenceRendering & DeviceTransform

Film Recorder

Ref. Display Device

Future Needs

• Wide gamut displays• High dynamic range• 4K and greater pipelines• Co-existing film and digital releases• Accommodate future production

techniques

Goals

• Preserve cinematographer intent• Able to incorporate film-sourced material• Apply uniform color management within pipelines• Provide consistent color across facilities• Eliminate image conversion errors• Create known and fixed transforms for the most

common use cases• Preserve, and absorb, current workflows

Lessons from previous formats

Approach

• Define fixed color space – ( primaries and white point )– fixed color encoding specification

Approach (cont)

• Use floating point color– 16-bit half-floats

• 10 bit mantissa, 5 bit exponent, 1 sign bit

– Nvidia/ILM format for OpenEXR provides 33+ stops range

• Negative numbers can represent out of gamut colors

• Merge colorimetric and densitometric workflows • Standardize calibration targets for film recording and scanning• Model transforms with Reference Input Device and Reference Output Device• Provide default viewing transform for ‘print-like’ behavior

Approach (cont)

Master and Interchange Format

• Destination format for – color correctors– digital cameras– renderers– scanners – telecines

• Usable in as many parts of the digital workflow as possible, but allows facilities to keep using their own pipeline

• A common studio master for deliverables and facility interchange

Surrounding Issues

• Compression• Real-time playback• Look management

software• On-set Previews (DP

intended filters for example)

• Monitor Previewing

• Laser Projectors• Energy Linear

Workflow (CGI)• OpenEXR and DPX• Workflow metadata• Digital Dailies

Color Encoding Specification

Colorimetric specification: Each code value maps to a specific color

Color GamutWant a gamut that covers a large range of common colors,

But how do we encode the densitometric film world…

figure 2: Map of Primaries (in CIE U',V')

-0.2500

-0.1500

-0.0500

0.0500

0.1500

0.2500

0.3500

0.4500

0.5500

0.6500

0.7500

0.0000 0.2000 0.4000 0.6000

u'

v'

Spectral LocusDLPFilm GamutsRGBROMMSony MonitorLaser PrimariesCandidate

Background on Cineon system

Film intermediate system

Scan negatives in, record out to internegative

Print from either camera negative or I.N.looks the same

Cineon used “printing density” to close Cineon used “printing density” to close the loop for input and output calibrationthe loop for input and output calibration

Printing Density

RGBPD =StatusM

toPD

* (StatusM - Dmin)500 *

10-bit range of code values from 0-1023 reference black (CV=95 usually set to Dmin of negative) reference white (CV=685 at about 90% reflective white)

5218 Characteristic Curve

Steps in Log(E)

Printing Density

5218 Status M Characteristic

log E

Status M Density

ref E

Production workflows needing linear operations

• Rendering

• Compositing

• Color matrixing

• Color correction (lift, gain, gamma)

• Area operations (e.g. anti-aliasing, blurs)

Linearized Printing Density

LPD = 10^ ( Printing Density / gamma)

gamma is a constant (e.g. 0.60)

LPD will likely have a straight-line center portion and may contain knee and shoulder roll-offs relative to scene exposures.

Benefits of L.P.D.

• Allows linear operations on image data facilitating visual effects pipeline

In the right space for color matrix operations Conversion to DCDM X’Y’Z’ is easy

• Eliminates Log/Lin difficulties with use of a single reversible transform

• Cinematographer choices are preserved

• Image data interpretation is specified without need for ‘color space’ metadata

Input Methodology

L.P.D. Color Encoding

Reference Input Device

Scene

Reference InputTransform

Tone curve is proportional to scene exposure but exact energy linear light is not assured

We can accept inexactness because we have a WYSIWIG preview stage

Output Methodology

L.P.D. Color Encoding

PreviewOutput

Transform

ReferenceRendering Transform

ReferenceDeviceTransform

ReferenceDisplayDevice

DisplayDevice

Output ColorAppearance

DeviceGamut & Encoding

Viewing Methodology

• Provide a standard, default 3D LUT “a virtual print stock”

• The image encoding maintains wide-gamut and high-dynamic range information.

• Gamut mapping and range compression is part of the Device Output Transform.

• Theatrical viewing environments are assumed

What about OpenEXR?

Examine use of Color Transformation Language (CTL)

OpenEXR is closest format type to where we are heading

Narrow some features of EXR (fewer codecs), and add new concepts

Close cooperation with OpenEXR development team

Our Tasks

• Complete requirements definition• Evaluate color space primaries• Specify input and output calibration

targets• Create a preview ‘viewing LUT’• Develop an API spec• Proof of concept test

Remaining Issues

• Reference implementation

• Intellectual property considerations

• Build consensus with potential users and vendors

• Engage a standards body (SMPTE)

Other Industry Efforts

American Society of Cinematographers “Color Decision List’ (CDL)

International Color Consortium Digital Motion Pictures Working Group

SMPTE Digital Cinema Technology Committee (DC28) AdHoc Group on Colorimetry

AdHoc Group for the uncompressed DCDM

OpenEXR Development Team

Summary

§ Linearized Printing Density aids the integration of digital and film workflows

§ (Half) floating point provides wide gamut and wide dynamic range color encoding

§ A standardized color space with defined transforms reduces conversion errors

§ Multi-facility workflows are improved with a well-defined image framework

§ Allows for future growth and new technology on the horizon

Acknowledgements: Thanks to the 25 members of the File Format committee whose work is

presented in this session.

Presentation will be made available on HPA website.http://www.hpaonline.com

For comments on the talk, or submissions to the committee, contact

jim.houston@mindspring.com

councilinfo@oscars.org

© 2006 A.M.P.A.S. ®