Series in Display Science and Technology

Kyoji Matsushima

Introduction to Computer HolographyCreating Computer-Generated Holograms as the Ultimate 3D Image

Series in Display Science and Technology

Series Editors

Karlheinz Blankenbach, FH für Gestaltung, Technik, Hochschule Pforzheim FH fürGestaltung, Technik, Pforzheim, GermanyFang-Chen Luo, Hsinchu Science Park, AU Optronics Hsinchu Science Park,Hsinchu, TaiwanBarry Blundell, University of Derby, Derby, UKRobert Earl Patterson, Human Analyst Augmentation Branch, Air Force ResearchLaboratory Human Analyst Augmentation Branch, Wright-Patterson AFB, OH,USAJin-Seong Park, Division of Materials Science and Engineering, HanyangUniversity, Seoul, Korea (Republic of)

The Series in Display Science and Technology provides a forum for researchmonographs and professional titles in the displays area, covering subjects includingthe following:

• optics, vision, color science and human factors relevant to display performance• electronic imaging, image storage and manipulation• display driving and power systems• display materials and processing (substrates, TFTs, transparent conductors)• flexible, bendable, foldable and rollable displays• LCDs (fundamentals, materials, devices, fabrication)• emissive displays including OLEDs• low power and reflective displays (e-paper)• 3D display technologies• mobile displays, projection displays and headworn technologies• display metrology, standards, characterisation• display interaction, touchscreens and haptics• energy usage, recycling and green issues

More information about this series at http://www.springer.com/series/15379

Kyoji Matsushima

Introduction to ComputerHolographyCreating Computer-Generated Hologramsas the Ultimate 3D Image

123

Kyoji MatsushimaFaculty of System EngineeringKansai UniversityOsaka, Japan

ISSN 2509-5900 ISSN 2509-5919 (electronic)Series in Display Science and TechnologyISBN 978-3-030-38434-0 ISBN 978-3-030-38435-7 (eBook)https://doi.org/10.1007/978-3-030-38435-7

© Springer Nature Switzerland AG 2020This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or partof the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations,recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmissionor information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilarmethodology now known or hereafter developed.The use of general descriptive names, registered names, trademarks, service marks, etc. in thispublication does not imply, even in the absence of a specific statement, that such names are exempt fromthe relevant protective laws and regulations and therefore free for general use.The publisher, the authors and the editors are safe to assume that the advice and information in thisbook are believed to be true and accurate at the date of publication. Neither the publisher nor theauthors or the editors give a warranty, expressed or implied, with respect to the material containedherein or for any errors or omissions that may have been made. The publisher remains neutral with regardto jurisdictional claims in published maps and institutional affiliations.

This Springer imprint is published by the registered company Springer Nature Switzerland AGThe registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland

Preface

It is an undoubted fact that the evolution of digital computers impacts our lifestylesas well as science and technology. It is also reasonable to assume that people will besurprised to see three-dimensional (3D) images produced by a well-made opticalhologram, and that they believe that the 3D image is perfect and accurate in itsrepresentation. Unfortunately, hologram galleries that exhibit holograms for art anddecoration have been disappearing recently, and holograms we see in our daily livesare limited to those on bills and credit cards. Such holograms are interestingbecause the appearance of the image changes when the observer changes his or herviewing angle, but they do not look like 3D images. This decline of holography for3D imaging is attributed to the fact that holography has not yet evolved into adigital technology. It is not possible to store the digital data of an optical hologramin a digital medium and it is not possible to transmit it through digital networks.

The idea of creating and handling 3D holographic images using computers has along history. In fact, the origin of the idea goes back to the days right after theactualization of 3D imaging by optical holography. However, computer-generatedholograms (CGH) comparable to optical holograms were not developed untilrecently. This is entirely due to the tremendous data sizes and computational effortsrequired to produce CGHs capable of reconstructing perfect 3D images. The cre-ation of outstanding large-scale CGHs is, in a sense, a fight against the availabilityof computer resources at a time. The required computational capabilities oftenexceed those of state-of-the-art computers. Algorithms and techniques available inthe literature are often ineffective because they are too time and/or resourceconsuming.

This book does not intend to provide all the various techniques proposed incomputer holography extensively. The techniques discussed herein are instead onesthat have been confirmed and proven to be useful and practical for producing actuallarge-scale CGHs whose 3D images are comparable to those of optical holography.Some of these techniques are of an atypical nature as well. It is both unfortunate,and a little delightful for researchers like me that such techniques are still far fromcompletion. Thus, I intended this book to provide just a snapshot of these devel-oping technologies.

v

When we produced the first large-scale CGH, “The Venus” in 2009, approxi-mately 48 h of computation time was needed. An expensive computer mounted on aserver rack was used. The computer was too noisy to be kept in a room where onestudies, owing to the cooling mechanism. Now, the same CGH can be calculated inapproximately 20 min using a quiet desktop computer. This is mainly due to thecontinuous development of computer hardware and partially due to the develop-ment of our technique with time.

This book is for any researcher, graduate or undergraduate student, who wants tocreate holographic 3D images using computers. Some of the chapters (e.g., Chaps. 3and 4) deal with basic topics and researchers already familiar with them can skipthose. Some techniques described in this book (e.g., Chaps. 6, 9, and 13) are usefulnot only for computer holography but also for all fields of research that requiretechniques for handling wavefields of light. In addition, because of the shortness ofwavelength, spatial data of light tends to be of gigantic sizes, exceeding memorysizes of computers. Some techniques introduced in this book (e.g., Chap. 12) givehints on how to handle such large-scale wavefields on computers.

I am sincerely grateful to Prof. Nakahara, who was a colleague at KansaiUniversity, co-author of many papers, and someone with the same enthusiasm forcreating CGHs as me. Many CGHs introduced in this book were fabricated by him.We visited many places together to present our work on CGHs. I would also like tothank Dr. Claus E. Ascheron for his strong recommendation to write a book oncomputer holography. He was an executive editor of Springer-Verlag when I methim for the first time and is now retired. This book would never have been written ifhe had not patiently persuaded me for over 3 years. I regret that I could notcomplete this book before his retirement. I am also grateful to Dr. Petr Lobaz whoreviewed my manuscript carefully and gave me many suggestions based on hisprofound knowledge and enthusiasm for holography. I am also thankful to myformer and current students: Dr. Nishi, Mr. Nakamoto and Mr. Tsuji, who didreviews of the text and formulae. I am grateful to them, and any mistakes that mayhave crept in are entirely mine.

Finally, I would like to thank my colleagues at the Department of Electrical andElectronic Engineering at Kansai University. I am particularly thankful to all myPh.D., masters, and undergraduate students, an assistant, and everyone whobelonged to my laboratory over the past 20 years at Kansai University. Their ideas,efforts, and passion has led to our success in computer holography.

Osaka, JapanOctober 2019

Kyoji Matsushima

vi Preface

Contents

1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.1 Computer Holography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.2 Difficulty in Creating Holographic Display . . . . . . . . . . . . . . . . 31.3 Full-Parallax High-Definition CGH . . . . . . . . . . . . . . . . . . . . . 5

2 Overview of Computer Holography . . . . . . . . . . . . . . . . . . . . . . . . 132.1 Optical Holography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132.2 Computer Holography and Computer-Generated Hologram . . . . 152.3 Steps for Producing CGHs and 3D Images . . . . . . . . . . . . . . . . 162.4 Numerical Synthesis of Object Fields . . . . . . . . . . . . . . . . . . . . 17

2.4.1 Object Field . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172.4.2 Field Rendering . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182.4.3 Brief Overview of Rendering Techniques . . . . . . . . . . 19

2.5 Coding and Reconstruction . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

3 Introduction to Wave Optics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253.1 Light as Wave . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

3.1.1 Wave Form and Wave Equation . . . . . . . . . . . . . . . . 253.1.2 Electromagnetic Wave . . . . . . . . . . . . . . . . . . . . . . . 273.1.3 Complex Representation of Monochromatic

Waves . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 293.1.4 Wavefield . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

3.2 Plane Wave . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 313.2.1 One-Dimensional Monochromatic Wave . . . . . . . . . . 313.2.2 Sampling Problem . . . . . . . . . . . . . . . . . . . . . . . . . . 323.2.3 Plane Wave in Three Dimensional Space . . . . . . . . . . 333.2.4 Sampled Plane Wave . . . . . . . . . . . . . . . . . . . . . . . . 363.2.5 Maximum Diffraction Angle . . . . . . . . . . . . . . . . . . . 373.2.6 More Rigorous Discussion on Maximum

Diffraction Angle . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

vii

3.3 Spherical Wave . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 403.3.1 Wave Equation and Solution . . . . . . . . . . . . . . . . . . . 403.3.2 Spherical Wavefield and Approximation . . . . . . . . . . 413.3.3 Sampled Spherical Wavefield and Sampling

Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 423.4 Optical Intensity of Electromagnetic Wave . . . . . . . . . . . . . . . . 46

4 The Fourier Transform and Mathematical Preliminaries . . . . . . . . 494.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 494.2 The Fourier Transform of Continuous Function . . . . . . . . . . . . 49

4.2.1 Definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 494.2.2 Theorems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 504.2.3 Several Useful Functions and Their Fourier

Transform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 514.3 Symmetry Relation of Function . . . . . . . . . . . . . . . . . . . . . . . . 54

4.3.1 Even Function and Odd Function . . . . . . . . . . . . . . . 544.3.2 Symmetry Relations in the Fourier Transform . . . . . . 55

4.4 Convolution and Correlation . . . . . . . . . . . . . . . . . . . . . . . . . . 564.5 Spectrum of Sampled Function and Sampling Theorem . . . . . . . 584.6 Discrete Fourier Transform (DFT) . . . . . . . . . . . . . . . . . . . . . . 614.7 Fast Fourier Transform (FFT) . . . . . . . . . . . . . . . . . . . . . . . . . 65

4.7.1 Actual FFT with Positive Indexes . . . . . . . . . . . . . . . 654.7.2 Use of Raw FFT with Symmetrical Sampling . . . . . . 664.7.3 Discrete Convolution Using FFT . . . . . . . . . . . . . . . . 70

5 Diffraction and Field Propagation . . . . . . . . . . . . . . . . . . . . . . . . . . 755.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75

5.1.1 Field Propagation . . . . . . . . . . . . . . . . . . . . . . . . . . . 755.1.2 Classification of Field Propagation . . . . . . . . . . . . . . . 77

5.2 Scalar Diffraction Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . 785.2.1 Angular Spectrum Method . . . . . . . . . . . . . . . . . . . . 785.2.2 Fresnel Diffraction . . . . . . . . . . . . . . . . . . . . . . . . . . 835.2.3 Fraunhofer Diffraction . . . . . . . . . . . . . . . . . . . . . . . . 86

5.3 Optical Fourier Transform by Thin Lens . . . . . . . . . . . . . . . . . 885.3.1 Wave-Optical Property of Thin Lens . . . . . . . . . . . . . 885.3.2 Wavefield Refracted by Thin Lens . . . . . . . . . . . . . . 90

5.4 Propagation Operator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 925.4.1 Propagation Operator as System . . . . . . . . . . . . . . . . 925.4.2 Backward Propagation . . . . . . . . . . . . . . . . . . . . . . . 93

6 Numerical Field Propagation Between Parallel Planes . . . . . . . . . . 956.1 Far-Field Propagation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95

6.1.1 Discrete Formula . . . . . . . . . . . . . . . . . . . . . . . . . . . 956.1.2 Destination Sampling Window . . . . . . . . . . . . . . . . . 97

viii Contents

6.1.3 Numerical Example . . . . . . . . . . . . . . . . . . . . . . . . . 986.1.4 Sampling Problem . . . . . . . . . . . . . . . . . . . . . . . . . . 99

6.2 The Fourier Transform by Lens . . . . . . . . . . . . . . . . . . . . . . . . 1016.3 Single-Step Fresnel Propagation . . . . . . . . . . . . . . . . . . . . . . . . 102

6.3.1 Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1026.3.2 Numerical Example . . . . . . . . . . . . . . . . . . . . . . . . . 1036.3.3 Sampling Problem . . . . . . . . . . . . . . . . . . . . . . . . . . 103

6.4 Convolution-Based Technique: Band-Limited AngularSpectrum Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1066.4.1 Discrete Formula . . . . . . . . . . . . . . . . . . . . . . . . . . . 1066.4.2 Sampling Problem of Transfer Function . . . . . . . . . . . 1076.4.3 Problem of Field Invasion . . . . . . . . . . . . . . . . . . . . . 1116.4.4 Discussion on Band Limiting . . . . . . . . . . . . . . . . . . 1146.4.5 More Accurate Technique . . . . . . . . . . . . . . . . . . . . . 116

7 Holography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1177.1 Optical Interference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1177.2 Thin Hologram and Volume Hologram . . . . . . . . . . . . . . . . . . 1187.3 Types of Holography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1207.4 Mathematical Explanation of Principle . . . . . . . . . . . . . . . . . . . 1217.5 Spatial Spectrum of Amplitude Hologram . . . . . . . . . . . . . . . . 1237.6 Conjugate Image . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1267.7 Theory and Examples of Thin Hologram . . . . . . . . . . . . . . . . . 127

7.7.1 Hologram with Plane Wave . . . . . . . . . . . . . . . . . . . 1287.7.2 Hologram with Spherical Wave . . . . . . . . . . . . . . . . . 1327.7.3 Fourier Transform Hologram . . . . . . . . . . . . . . . . . . . 147

8 Computer Holography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1538.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1538.2 Viewing Angle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1538.3 Space-Bandwidth Product Problem . . . . . . . . . . . . . . . . . . . . . 1558.4 Full-Parallax and Horizontal-Parallax-Only CGH . . . . . . . . . . . 1578.5 Coding and Optimization of Fringe Pattern . . . . . . . . . . . . . . . 1588.6 Amplitude CGH . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159

8.6.1 Amplitude Encoding . . . . . . . . . . . . . . . . . . . . . . . . . 1598.6.2 Brightness and Noise . . . . . . . . . . . . . . . . . . . . . . . . 1618.6.3 Binary-Amplitude CGH . . . . . . . . . . . . . . . . . . . . . . 164

8.7 Phase CGH . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1658.7.1 Phase Encoding . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1668.7.2 Example of Phase CGH . . . . . . . . . . . . . . . . . . . . . . 1678.7.3 Binary-Phase CGH . . . . . . . . . . . . . . . . . . . . . . . . . . 169

Contents ix

8.8 Spatial Frequency of Fringe Pattern . . . . . . . . . . . . . . . . . . . . . 1708.8.1 Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1708.8.2 Example of Fringe Frequency . . . . . . . . . . . . . . . . . . 1738.8.3 Fringe Oversampling . . . . . . . . . . . . . . . . . . . . . . . . 176

8.9 Fourier-Transform CGH . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1788.9.1 Higher-Order Diffraction Images . . . . . . . . . . . . . . . . 1788.9.2 Generation of Fringe Pattern . . . . . . . . . . . . . . . . . . . 1808.9.3 Amplitude Fringe Pattern Based on Hermitian

Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1818.10 Single-Sideband Method in Amplitude CGH . . . . . . . . . . . . . . 182

8.10.1 Principle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1838.10.2 Generation of Fringe Pattern . . . . . . . . . . . . . . . . . . . 184

9 The Rotational Transform of Wavefield . . . . . . . . . . . . . . . . . . . . . 1879.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1879.2 Coordinate Systems and Rotation Matrices . . . . . . . . . . . . . . . . 1879.3 Principle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1899.4 Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192

9.4.1 General Formulation . . . . . . . . . . . . . . . . . . . . . . . . . 1929.4.2 Paraxial Approximation . . . . . . . . . . . . . . . . . . . . . . 194

9.5 Numerical Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1959.5.1 Sampling Distortion . . . . . . . . . . . . . . . . . . . . . . . . . 1959.5.2 Shifted Fourier Coordinates . . . . . . . . . . . . . . . . . . . . 1979.5.3 Actual Procedure to Perform the Rotational

Transform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1989.5.4 Resample of Uniformly Sampled Spectrum . . . . . . . . 202

9.6 Numerical Examples and Errors . . . . . . . . . . . . . . . . . . . . . . . . 2039.6.1 Edge Effect and Sampling Overlap . . . . . . . . . . . . . . 2039.6.2 The Rotational Transform with Carrier Offset . . . . . . . 2069.6.3 Examples of the Rotational Transform

in Practical Wavefield . . . . . . . . . . . . . . . . . . . . . . . . 209

10 The Polygon-Based Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21110.1 Surface Source of Light . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211

10.1.1 Generation of Scattered Light . . . . . . . . . . . . . . . . . . 21110.1.2 Theoretical Model of Polygonal Surface Source

of Light . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21310.2 Basic Theory for Rendering Diffused Surface . . . . . . . . . . . . . . 214

10.2.1 Surface Function . . . . . . . . . . . . . . . . . . . . . . . . . . . 21510.2.2 Spectrum Remapping by Incident Plane Wave . . . . . . 21610.2.3 Rotation Matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21810.2.4 Rotational Transform of Remapped Spectrum . . . . . . 22010.2.5 Short Propagation to Object Plane . . . . . . . . . . . . . . . 22110.2.6 Superposition of Polygon Fields and Propagation

to Hologram Plane . . . . . . . . . . . . . . . . . . . . . . . . . . 222

x Contents

10.3 Practical Algorithm for Rendering Diffused Surface . . . . . . . . . 22310.3.1 Input Data and Controllable Parameters . . . . . . . . . . . 22310.3.2 Tilted and Parallel Frame Buffers . . . . . . . . . . . . . . . 22410.3.3 The Fourier Transform of Surface Function . . . . . . . . 22410.3.4 Basic Procedure for the Rotational Transform

and Short Propagation . . . . . . . . . . . . . . . . . . . . . . . . 22510.3.5 Maximum Diffraction Area of Polygon . . . . . . . . . . . 22710.3.6 Determination of Sampling Interval of Surface

Function by Probing Sample Points . . . . . . . . . . . . . . 22910.3.7 How to Determine Sizes of PFB and TFB . . . . . . . . . 23110.3.8 Back-Face Culling . . . . . . . . . . . . . . . . . . . . . . . . . . 23510.3.9 Overall Algorithm for Rendering Diffused

Surface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24010.3.10 Variation of Probing Sample Points . . . . . . . . . . . . . . 242

10.4 Band Limiting of Polygon Field . . . . . . . . . . . . . . . . . . . . . . . 24310.4.1 Principle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24310.4.2 Limit of Bandwidth . . . . . . . . . . . . . . . . . . . . . . . . . 24410.4.3 Modification of Algorithm . . . . . . . . . . . . . . . . . . . . 246

10.5 Computation Time of Object Field . . . . . . . . . . . . . . . . . . . . . . 24710.6 Shading and Texture-Mapping of Diffused Surface . . . . . . . . . . 249

10.6.1 Brightness of Reconstructed Surface . . . . . . . . . . . . . 24910.6.2 Amplitude of Surface Function . . . . . . . . . . . . . . . . . 25210.6.3 Shading of Diffused Surfaces . . . . . . . . . . . . . . . . . . 25310.6.4 Texture-Mapping . . . . . . . . . . . . . . . . . . . . . . . . . . . 257

10.7 Rendering Specular Surfaces . . . . . . . . . . . . . . . . . . . . . . . . . . 25810.7.1 Spectrum of Diffuse and Specular Reflection . . . . . . . 25910.7.2 Phong Reflection Model . . . . . . . . . . . . . . . . . . . . . . 26010.7.3 Spectral Envelope of Specular Component . . . . . . . . . 26110.7.4 Generation of Specular Diffuser for Surface

Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26210.7.5 Fast Generation of Specular Diffuser by Shifting

Spectrum . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26510.7.6 Flat Specular Shading . . . . . . . . . . . . . . . . . . . . . . . . 26810.7.7 Smooth Specular Shading . . . . . . . . . . . . . . . . . . . . . 27210.7.8 Examples of High-Definition CGHs with Specular

Shading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 278

11 The Silhouette Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28111.1 Occlusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28111.2 Processing of Mutual Occlusion . . . . . . . . . . . . . . . . . . . . . . . . 283

11.2.1 Silhouette Method . . . . . . . . . . . . . . . . . . . . . . . . . . 28311.2.2 Formulation of Object-by-Object Light-Shielding

for Multiple Objects . . . . . . . . . . . . . . . . . . . . . . . . . 284

Contents xi

11.2.3 Actual Example of Object-by-ObjectLight-Shielding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 286

11.2.4 Translucent Object . . . . . . . . . . . . . . . . . . . . . . . . . . 28711.3 Switch-Back Technique for Processing Self-Occlusion

by the Silhouette Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28911.3.1 Principle of Polygon-by-Polygon Light-Shielding

and Associated Problem . . . . . . . . . . . . . . . . . . . . . . 28911.3.2 The Babinet’s Principle . . . . . . . . . . . . . . . . . . . . . . . 29011.3.3 Light-Shielding by Use of Aperture Instead

of Mask . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29411.3.4 Formulation for Multiple Polygons . . . . . . . . . . . . . . 29611.3.5 Practical Procedure for Computation of Object

Field with P-P Shielding . . . . . . . . . . . . . . . . . . . . . . 29811.3.6 Inductive Explanation of the Switch-Back

Technique . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29911.3.7 Numerical Technique and Sampling Window

for Switch-Back Propagation . . . . . . . . . . . . . . . . . . . 30011.3.8 Emulation of Alpha Blend of CG . . . . . . . . . . . . . . . 30111.3.9 Acceleration by Dividing Object . . . . . . . . . . . . . . . . 30211.3.10 Integration with the Polygon-Based Method . . . . . . . . 30311.3.11 Actual Examples of P-P Light-Shielding

and Computation Time . . . . . . . . . . . . . . . . . . . . . . . 30411.4 Limitation of the Silhouette Method . . . . . . . . . . . . . . . . . . . . . 306

12 Shifted Field Propagation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30912.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 309

12.1.1 What is Shifted Field Propagation . . . . . . . . . . . . . . . 30912.1.2 Rectangular Tiling . . . . . . . . . . . . . . . . . . . . . . . . . . 310

12.2 Mathematical Preliminary . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31112.2.1 Fractional DFT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31112.2.2 Scaled FFT for Symmetric Sampling . . . . . . . . . . . . . 313

12.3 Shifted Far-Field Propagation . . . . . . . . . . . . . . . . . . . . . . . . . 31612.3.1 Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31612.3.2 Numerical Example . . . . . . . . . . . . . . . . . . . . . . . . . 31912.3.3 Sampling Problem . . . . . . . . . . . . . . . . . . . . . . . . . . 319

12.4 Shifted Fresnel Propagation . . . . . . . . . . . . . . . . . . . . . . . . . . . 32012.4.1 Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32012.4.2 Numerical Example . . . . . . . . . . . . . . . . . . . . . . . . . 32112.4.3 Sampling Problem . . . . . . . . . . . . . . . . . . . . . . . . . . 321

12.5 Shifted Angular Spectrum Method . . . . . . . . . . . . . . . . . . . . . . 32612.5.1 Coordinate System . . . . . . . . . . . . . . . . . . . . . . . . . . 32612.5.2 Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32712.5.3 Band Limiting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 328

xii Contents

12.5.4 Actual Procedure for Numerical Calculation . . . . . . . . 33312.5.5 Numerical Example . . . . . . . . . . . . . . . . . . . . . . . . . 33512.5.6 Discussion on the Limit Frequency . . . . . . . . . . . . . . 335

13 Simulated Reconstruction Based on Virtual Imaging . . . . . . . . . . . 33713.1 Need for Simulated Reconstruction . . . . . . . . . . . . . . . . . . . . . 33713.2 Simulated Reconstruction by Back Propagation . . . . . . . . . . . . 338

13.2.1 Examples of Reconstruction by Back-Propagation . . . 33913.2.2 Control of DOF Using Aperture . . . . . . . . . . . . . . . . 34213.2.3 Control of View-Direction Using Aperture . . . . . . . . . 343

13.3 Image Formation by Virtual Lens . . . . . . . . . . . . . . . . . . . . . . 34413.3.1 Sampling Problem of Virtual Lens . . . . . . . . . . . . . . . 34413.3.2 Equal Magnification Imaging by Virtual Lens . . . . . . 34613.3.3 Reduced Imaging by Virtual Lens . . . . . . . . . . . . . . . 34713.3.4 Change of Viewpoint . . . . . . . . . . . . . . . . . . . . . . . . 350

13.4 Simulated Reconstruction from Fringe Pattern . . . . . . . . . . . . . 35313.4.1 Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35413.4.2 Comparison Between Simulated and Optical

Reconstructions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35413.5 Simulated Reconstruction in Color . . . . . . . . . . . . . . . . . . . . . . 356

13.5.1 Production of Full-Color Reconstructed Image . . . . . . 35613.5.2 Examples of Simulated Reconstruction in Color . . . . . 358

14 Digitized Holography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36314.1 Concept of Digitized Holography . . . . . . . . . . . . . . . . . . . . . . . 36314.2 Digital Holography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 365

14.2.1 Phase-Shifting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36514.2.2 Lensless-Fourier Digital Holography

for Converting Sampling Interval . . . . . . . . . . . . . . . 36714.2.3 Synthetic Aperture Digital Holography

for Capturing Large-Scale Wavefield . . . . . . . . . . . . . 37214.3 Capture of Object-Field . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 376

14.3.1 Monochromatic Object Field . . . . . . . . . . . . . . . . . . . 37614.3.2 Object Fields in Full-Color . . . . . . . . . . . . . . . . . . . . 378

14.4 Occlusion Processing Using the Silhouette Method . . . . . . . . . . 38114.4.1 The Silhouette Method Including Captured

Object Fields . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38114.4.2 Making Silhouette Masks . . . . . . . . . . . . . . . . . . . . . 382

14.5 Examples of Optical Reconstruction . . . . . . . . . . . . . . . . . . . . . 38414.5.1 Monochrome CGH . . . . . . . . . . . . . . . . . . . . . . . . . . 38414.5.2 Full-Color CGH . . . . . . . . . . . . . . . . . . . . . . . . . . . . 387

Contents xiii

14.6 Resizing Object Image . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38914.6.1 Resizing by Change of Sampling Intervals . . . . . . . . . 38914.6.2 Resizing by Virtual Imaging . . . . . . . . . . . . . . . . . . . 38914.6.3 Resizing by Shifted Fresnel Propagation . . . . . . . . . . 391

15 Fabrication of High-Definition CGH . . . . . . . . . . . . . . . . . . . . . . . . 39515.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39515.2 Fringe Printers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 397

15.2.1 Spot-Scanning Fringe Printer . . . . . . . . . . . . . . . . . . . 39715.2.2 Image-Tilling Fringe Printer . . . . . . . . . . . . . . . . . . . 400

15.3 Laser Lithography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40115.3.1 Photomasks as a Binary-Amplitude CGH . . . . . . . . . . 40215.3.2 Structure of Photomasks . . . . . . . . . . . . . . . . . . . . . . 40315.3.3 Process to Fabricate Photomasks . . . . . . . . . . . . . . . . 40415.3.4 Pattern Drawing by Laser Writer . . . . . . . . . . . . . . . . 40615.3.5 Actual Processes of Development and Etching . . . . . . 40715.3.6 Creation of Phase CGHs . . . . . . . . . . . . . . . . . . . . . . 409

15.4 Wavefront Printer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41215.4.1 Principle and Difference from Holographic

Printer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41215.4.2 Optical Systems for Generating Object Fields . . . . . . 41415.4.3 Calculation of Object Fields and Encoding

of Fringes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41815.4.4 Denysyuk-Type Wavefront Printer . . . . . . . . . . . . . . . 419

15.5 Full-Color Reconstruction of HD-CGHs Using OpticalCombiner . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 422

15.6 Full-Color CGH Using RGB Color Filters . . . . . . . . . . . . . . . . 42315.6.1 Principle and Structure . . . . . . . . . . . . . . . . . . . . . . . 42415.6.2 Fringe Pattern . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42715.6.3 Design Parameters of RGB Color Filters . . . . . . . . . . 42815.6.4 Examples of Optical Reconstruction . . . . . . . . . . . . . 429

15.7 Full-Color Stacked CGVH. . . . . . . . . . . . . . . . . . . . . . . . . . . . 43015.7.1 Principle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43215.7.2 Compensation for Thickness and Refractive

Index of Substrates . . . . . . . . . . . . . . . . . . . . . . . . . . 43515.7.3 Fabrication of Stacked CGVH . . . . . . . . . . . . . . . . . . 43815.7.4 Optical Reconstruction of Stacked CGVH . . . . . . . . . 440

Appendix: Data of Major HD-CGHs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 443

References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 447

Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 457

xiv Contents

Chapter 1Introduction

Abstract Computer holography is a technique to create computer-generated holo-grams (CGH), for reconstructing 3D images using computers. In this chapter, wediscuss the differences between CGHs for 3D displays and optical devices, advan-tages of the display-CGHs, and difficulties in creating holographic displays as anultimate 3D display. Early and latest large-scale static CGHs that reconstruct bril-liant deep 3D images are demonstrated.

1.1 Computer Holography

Holography is most likely the only technology to reconstruct deep 3D scenes froma plate medium. Holography, invented by Dennis Gabor in 1948 [16], makes itpossible to reconstruct light from a subject, recorded by interference with a referencewave. A person recognizes three dimensional (3D) spaces and perceives distancesfrom objects by visual sensation through many clues. The most important depthperceptions for 3D imaging are summarized as follows:

Binocular disparity: Difference between images of an object, seen by the left andright eyes,

Vergence: Movement of eyes in opposite directions to maintain binocular vision,Accommodation: Process to focus on an object to obtain clear images,Motion parallax: Change in the image of an object when the viewpoint changes.

The first two perceptions are based on binocular cues, while the last two are owingto monocular cues.

Technologies of 3D imaging which have already been put to practical use,stimulate a sensation of depth by showing different images to the left and righteyes. Therefore, depth sensation is based on the binocular cues, i.e., the first twocues mentioned above. In contrast, the last two cues are not given by current 3Dimages. More accurately, the depth perception given by monocular cues do notagree with those given by binocular cues in current 3D technologies. In particular,it is said that inconsistency of depth perceptions by vergence and accommodationcauses a noticeable problem, called vergence-accommodation conflict. The conflict

© Springer Nature Switzerland AG 2020K. Matsushima, Introduction to Computer Holography,Series in Display Science and Technology,https://doi.org/10.1007/978-3-030-38435-7_1

1

2 1 Introduction

sometimes causes severe health problems such as headaches, eyestrain, dizziness,and nausea. Therefore, current 3D displays based on binocular cues cannot producedeep 3D scenes because deeper 3D scenes produce more conflict.

In contrast to 3D displays based on binocular vision, the 3D image reconstructedby a hologram does not cause any health concerns because the hologram physicallyreconstructs the recorded light itself.1 As a result, all depth cues are properly recon-structed in holography without any conflict. Accordingly, holograms have the abilityto display deep 3D scenes unlike current 3D technologies.

Unfortunately, traditional optical holography is not a modern digital technologybut is a type of analog photography. Traditional holography always needs a physicalsubject to create the 3D image. For example, if we want to create the 3D image ofa car by holography, we need a physical entity i.e., the car. Thus, prior to recordingthe hologram, we have to prepare an actual 3D model of the car, which is commonlya scaled down model of an actual car. In addition, because information of the lightemitted by a subject is recorded on a photo-sensitive material in optical holography,the information is not digitized and hence, cannot be transmitted through moderndigital networks and stored in digital media.2 Besides, the recorded image of thesubject cannot be edited after recording in general. This property of analog hologramsis useful for security purposes but inconvenient for 3D imaging.

As explained in the following chapters, information of the light emitted by anobject is recorded in the form of fringe patterns in holography. In a type of hologramcalled a thin hologram (see Sect. 7.2), the fringe pattern is a simple two dimensional(2D) image,which looks like a randompattern. Therefore,we can generate the patternby some technique using a computer. The created holograms are called computer-generated holograms orCGH in the abbreviated form.We call the technique to createCGHs computer holography in this book.

The idea of CGHs was first proposed by A. W. Lohmann in 1967 [55].3 Articleson CGHs has been increasing every year after Lohmann’s first proposal [27, 121].However, the CGHs found in the literature of early days are not the ones that we dealwith in this book. The word “CGH” is commonly used in two different contexts: anoptical device and a 3D display. In the former context, the CGH is a device thatmakesit possible to filter the field of light or to produce a complex field with amplitude-onlyor phase-only spatial modulation. In other words, the CGH in the former context isa wavefront conversion device, as shown in Fig. 1.1. In this case, we know the formsof both the input and output fields of the CGH and investigate the technique forconversion with high efficiency and accuracy. This conversion corresponds to thecoding of the fringe patterns in the latter CGHs for 3D displays (for coding of fringepatterns, see Sects. 2.5, 8.5 and thereafter).

In CGHs for 3D displays, the output field of the CGH is an object field that formsthe 3D image. We must calculate the object field from the 3D model of the objectbefore coding of the fringe pattern. The calculation of this object field is the most

1Here, we ignore horizontal-parallax-only (HPO) holograms (see Sect. 8.4).2There is a technique to record the information digitally, as described in Chap. 14.3Although [7] is published before [55], the detailed technique was not provided therein.

1.1 Computer Holography 3

CGH

Object field

Input field

Output field( Object field in 3D display)

CG model

Fig. 1.1 Schematic illustration of the difference between CGHs for an optical device and a 3Ddisplay. The target output field is already known in optical devices, while in 3D displays the targetfield is the object field calculated from a 3D model

important and difficult process in creating CGHs for 3D displays. Therefore, we putlittle importance on the coding of the fringe pattern in comparison to the calculationof the object field. This is the biggest difference between CGHs for a device and a3D display.

It should be noted again that the author uses the word “computer holography”in this book to describe the techniques for producing 3D images according to theprinciple of holography using computers. The 3D image usually comes from 3Dmodels like computer graphics (CG).4 Computer holography makes it possible toreconstruct holographic motion pictures as well as still pictures. The technique toreconstruct display-CGHs using electronic equipment is sometimes called electro-holography. The equipment is calledHolographic 3D display or simplyHolographicdisplay. The final goal of the holographic display is undoubtedly to produce anultimate 3D motion picture in which any conflict of visual sensation does not occur.

1.2 Difficulty in Creating Holographic Display

Computer holography has been suffering from two problems for a long time: calcu-lation and display. The viewing angle of a CGH increases approximately in inverseproportion to the pixel pitch of the fringe pattern (see Sect. 8.2). On the other hand,the size of a CGH is given by the product of the pixel pitch and the number of pixels.Therefore, to combine the size and viewing angle of a CGH, the number of pixelsrequired inevitably takes a gigantic value. This is called the space-bandwidth product(SBP) problem (for details, see Sect. 8.3).

For example, supposing that the number of pixels is 2,000 × 2,000 in a CGH andthe pixel pitch is 0.1 mm, the CGH size (i.e., the image size) is 20 cm × 20 cm, butthe viewing angle is only 0.36◦.5 If the pixel pitch is reduced to 1 µm, the viewingangle is increased to 36◦ but the CGH size is only 2 × 2 mm2. If we want to createa CGH whose viewing angle is 45◦, the pixel pitch must be less than 0.8 µm at a

4As mentioned in Chap. 14, the concept of computer holography is extended to include physicalobjects.5We assume a wavelength of 633 nm (red) in this discussion.

4 1 Introduction

wavelength of 633 nm (red color). Besides, if we want a CGH whose size is morethan 5 cm × 5 cm, the total number of pixels required is more than four billion!

Because the object field commonly has exactly the same scale as the fringe pattern,the large scale of the fringe pattern makes it very difficult to calculate the object fieldand to generate the fringe pattern in a short time. Furthermore, it is difficult to displayseveral billion pixels with a pixel pitch less than or equal to 1 µm. These are thedifficulties in displaying CGHs. Although the idea and the study of CGHs has a longhistory, it had been difficult to create brilliant large-scale CGHs until recently owingto these difficulties.

Coming back to the history of computer holography, the early works by Benton etal. in the late 1980s most likely inspired subsequent research studies on holographicdisplays [44, 115, 116]. Benton et al. used an acousto-optic modulator (AOM) toproduce one dimensional fringe patterns and achieved a horizontal viewing angle of12◦ at 633 nm, i.e., an effective pixel pitch of approximately 3.0 µm using demag-nification optics. Unfortunately, the system could produce only horizontal parallax.Although the challenge in calculation is eased to a great extent in horizontal-parallax-only (HPO) holograms (see Sect. 8.4), the problem of vergence-accommodationconflict is not resolved in HPO displays.

After the advent of early Benton systems, many display systems have beenreported for realizing the ultimate 3D display, i.e., holographicmovies/TVs/PCmon-itors. The reported systems commonly make use of existing display devices such asliquid crystal display (LCD) and digital mirror device (DMD). However, resolu-

i

pixels/sec

~2014 ~2023 ~2037

Full-parall ax 10inc h 30deg

HPO 20 nch 60d eg

80 1990 2000 2010 2020 2030 2040

Full-parallax 40inch 90deg1015

1014

1013

1012

1011

1010

109

108

107

106

105

19

Num

ber o

f pix

els

refre

shed

in a

seco

nd

Year

Fig. 1.2 The trend of the spatio-temporal resolution (pixels/second) of display devices from late1980s to 2010. The square dots represent examples of commercialized or prototyped displaydevices [132] (From SPIE Proc. 8043, 40 (2011). By courtesy of Prof. Yamaguchi, Tokyo Instituteof Technology.)

1.2 Difficulty in Creating Holographic Display 5

tion of current display devices are quite far from the SBP requirements in computerholography. Thus, techniques of time or space division multiplexing are often usedfor producing effective high-resolutions.

The achievable resolution with a time division multiplex system is determined bythe number of pixels displayed in a second. Figure1.2 shows the trend of the spatio-temporal resolutions (i.e., pixels/second) of display devices from the late 1980s [132].According to extrapolation of the development curve, it is predicted that full-parallaxholographic displays with a given size and viewing angle will be realized in the mid-2020s. However, this expectation is very doubtful because the existing top data isof display devices for 8K ultra high-definition television (UHTV) systems and it isunsure if the development of higher resolution devices will continue after the 8Kdevices.

1.3 Full-Parallax High-Definition CGH

The author and Prof. Nakahara created and reported a large-scale full-parallax CGHnamed “The Venus” in 2009 [67]. Figure1.3 shows photographs of the optical recon-struction of The Venus.6 Although the Venus is a static CGH, it is composed of216 × 216 pixels; the total number of pixels is approximately four billion. Here, thenumber of pixels is sometimes represented using a supplementary unit of K; 1K =1,024 in this book. Thus, The Venus is composed of 64K × 64K pixels. Becausethe pixel pitch of the Venus’s fringe pattern is 1µm in both the horizontal and ver-tical directions, the viewing angle at a wavelength of 633 nm is approximately 37◦in both directions. The size of the CGH is approximately 6.5 × 6.5 cm2. We calllarge-scale CGHs like the Venus, which are composed of more than a billion pixels,a high-definition CGH (HD-CGH).

The 3D model of The Venus is represented by a polygon-mesh exactly like incomputer graphics (CG). Themodel is composed of 718 polygons7, arranged at 15 cmfrom the hologram plane in the CGH. The object field is calculated in full-parallaxusing the polygon-based method (see Chap. 10), and the occlusion is processedby the silhouette method (see Chap. 11). The fringe pattern was printed using laserlithography (see Sect. 15.3). TheCGHcan be reconstructed by reflection illuminationusing an ordinary red LED as well as a red laser. Because the 3D image reproducesproper occlusion, viewers can verify continuous and natural motion parallax, asshown in the online videos. As a result, the reconstructed 3D image is very impressiveand gives strong sensation of depth to the viewers.

Because HD-CGHs are holograms, they reconstruct not only the binocular dis-parity and vergence but also the accommodation. Figure1.4 shows photographs of

6Note that the left and right photographs are interchanged by mistake in Fig. 1.7 of [67].7The mesh data of the Venus statue is provided by courtesy of INRIA by the AIM@SHAPE ShapeRepository.

6 1 Introduction

Left Right

High

Low

Fig. 1.3 Optical reconstruction of the first high-definition CGH named “The Venus” [67]. AHe–Ne laser is used as the illumination light source. The photographs are taken from differentviewpoints. The number of pixels and pixel pitches are 4,294,967,296 (=65,536 × 65,536) and1.0 µm, respectively (see Sect. 10.5 for the detail). Video link, https://doi.org/10.1364/AO.48.000H54.m002 for illumination by a red LED, and https://doi.org/10.1364/AO.48.000H54.m004for illumination by a He–Ne laser

(a) (b)

Fig. 1.4 Optical reconstruction of HD-CGH “Moai I” [70]. The camera is focused on a the frontand b rear moai statues. The number of pixels and pixel pitches are 64K × 64K and 1 µm × 1 µm,respectively. Video link, https://youtu.be/DdOveIue3sc

1.3 Full-Parallax High-Definition CGH 7

Fig. 1.5 The 3D scene ofHD-CGH “Moai I” [70]

6.55.5

5.2

15

15CGH

5.0

25

Units: cm

6.5

13

the optical reconstruction of another HD-CGH “Moai I.” The 3D scene is depictedas a CG image in Fig. 1.5. The background image is a digital image designed with512× 512 pixels. Both the moai statues are a polygon-mesh CGmodel composed of1220 polygons8 and arranged at 15 cm from each other. Here, the camera is focusedon the front moai in Fig. 1.4a, and on the rear moai in (b). We can clearly verify thechange of appearance; the actual moai is out of focus when the front moai is in focus,and vice versa.

As shown in the example above, the 3D image reconstructed by HD-CGHs stim-ulates all depth perceptions without any inconsistency unlike conventional 3D tech-nologies. As a result, HD-CGHs reconstruct the spatial images as if the viewer werelooking at the 3D world spreading beyond the windows formed by the hologram. Inpractice, we sometimes noticed the viewers of HD-CGHs checking the backside ofthe HD-CGH in exhibitions to verify that the HD-CGH is a thin plate without depth.

Here, we emphasize that 2D photographs like Figs. 1.3 and 1.4 never convey trueimpressions of the 3D image reconstructed by the HD-CGHs. One has to look atthe actual CGH with their own eyes to learn what is shown by the CGH. Motionpictures usually portray the nature of holographic images better than still pictures.Thus, online movie clips of actual CGHs are referred to as much as possible in thisbook.

Figure1.6 shows an early version of “Brothers” that we created for exhibition atMassachusetts Institute of Technology (MIT)Museum in Boston, USA in 2012 [71].This HD-CGH remained the biggest CGH that we had ever created for a long time.The total number of pixels is 21.5 billion in V1 and 25.8 billion in V2 that is exhibitedin the museum. The size is 131 × 105mm2 in V1 but 126 × 105mm2 in V2.9 Theobjects arranged in the 3D scene (see Fig. 11.4, Sect. 11.2.3) are live faces whose3D shapes were measured by a laser scanner. Photographs taken at the same time asscanning are texture-mapped onto the measured polygon-meshes with the polygon-based method (see Sect. 10.6.4).

8The mesh data of the moai statue is provided by courtesy of Yutaka_Ohtake by the AIM@SHAPEShape Repository.9Brothers V2 is a little smaller than V1 because the horizontal pixel pitch is reduced to 0.64 µm inV2.

8 1 Introduction

Red LED

(a) (b)

Fig. 1.6 Optical reconstruction of “Brothers” V1. The photographs are taken at a a close-up andb distant view. The second version of Brothers was on display at MIT Museum from Jul 2012 toMar 2015. The total number of pixels of V1 is approximately 21.5 billion (=160K × 128K), andthe pixel pitch is 0.8 µm in both directions. The CGH size is approximately 131 × 105mm2. SeeSect. 11.2.3 for the detailed parameters. Video link, https://youtu.be/RCNyPNV7gHM

Light source

CGH

(a) (b)

Fig. 1.7 Photographs of “Sailing Warship II” at a a distant view and b close-up view. The totalnumber of pixels is 67.5 × 109 (=225,000 × 300,000). The pixel pitches are 0.8 µm and 0.4 µmin the horizontal and vertical direction, respectively. The CGH size is 18 cm × 12 cm. Video link,https://youtu.be/8USLC6HEPsQ

Fig. 1.8 The 3D scene ofSailing Warship II, depictedby CG. The model iscomposed of 22,202polygons

20

17

Hologram

20

18

12

Units: cm

1.3 Full-Parallax High-Definition CGH 9

Left Right

Fig. 1.9 Photographs of optical reconstruction of Sailing Warship II, taken from left and rightviewpoints. Video link, https://youtu.be/8USLC6HEPsQ

One of the latest HD-CGH is “Sailing Warship II” shown in Fig. 1.7. The totalnumber of pixels is more than 67 billion, and the size is 18 cm × 12 cm. This HD-CGH was created in 2017. Figure1.8 is the 3D scene of Sailing Warship II. The3D model composed of 22,202 polygons has a very complicated shape. It is verydifficult to calculate the object field of this kind of intricate model because the modelhasmany self-occlusions (see Sect. 11.1). The silhouettemethod and the switch-backtechnique (see Chap. 11) have been developed and are used for the creation of SailingWarship II. Figure1.9 shows photographs of the optical reconstruction, taken fromleft and right viewpoints. It is verified in the photographs that all portions behind theobjects are hidden by the front portions. The process of calculation of the object fieldis called occlusion processing or occlusion culling in computer holography. This isthe counterpart of hidden surface removal in CG. Occlusion processing is the mostimportant and difficult technique in computer holography.

An HD-CGH named “Toy Train” was created in 2018 using the same techniquesas those of Sailing Warship II. The optical reconstruction and 3D scene are shown inFigs. 1.10 and 1.11, respectively. This CGH is a square whose side length is 18 cm,and the total number of pixels reaches more than 0.1 trillion. Because depth of the3D scene is more than 35 cm, the CGH gives really strong sensation of depth to theviewers.

Here, note that the vertical pixel pitch is one half of that in the horizontal directionin Toy Train and Sailing Warship II. This is because the illumination light in thesetwo has a larger incident angle than that of other HD-CGHs. We can avoid unneces-sary conjugate images and non-diffraction light with a large illumination angle (seeSect. 8.8.3).

10 1 Introduction

Fig. 1.10 Opticalreconstruction of “ToyTrain,” created in 2018.Focus of the camera ischanged back and forth. Thetotal number of pixels is101 × 109

(=225,000 × 450,000). Thepixel pitches are 0.8 µm and0.4 µm in the horizontal andvertical direction,respectively. The CGH sizeis 18 cm × 18 cm. Videolink, https://youtu.be/XeRO7nFvlGc

Front focus

Rear focus

Fig. 1.11 The 3D scene ofToy Train, depicted by CG.The model is composed of52,661 polygons

18

1.3 Full-Parallax High-Definition CGH 11

Fig. 1.12 Optical reconstruction of a stacked CGVH illuminated by a white LED [46]. The CGHsize is approximately 5 × 5 cm. Video link, https://doi.org/10.6084/m9.figshare.8870177.v1

Only monochrome HD-CGHs were created and exhibited for a long time.10

Recently, it has been possible to produce full-color HD-CGHs by using severalmethods (see Sects. 15.5–15.7). Figure1.12 is a photograph of the optical recon-struction of a full-color stacked CGVH (Computer-Generated Volume Hologram).This HD-CGH is created using one of them.

10At exhibitions of monochrome HD-CGHs, the author was frequently asked a question, “Is itpossible to create a full-color HD-CGH?”.

Chapter 2Overview of Computer Holography

Abstract Techniques in computer holography are summarized and briefly over-viewed in this chapter. Several steps are commonly required for creating a large-scaleCGH to display vivid 3D images. This chapter deals with the procedure. We alsosurvey other techniques than those explained after this chapter in this book.

2.1 Optical Holography

Holograms are a fringe pattern, and the fringe pattern is recorded on photo-sensitivechemicals in traditional optical holography. Figure2.1 shows the recording step ofan optical hologram. The subject is illuminated by light emitted by a coherent lightsource, i.e., a laser. The light scattered by the subject reaches to the photo-sensitivematerial such as a holographic dry plate or film. This is commonly called an objectwave or object field. The output of the light source is also branched into another lightpath and directly illuminates the same photo-sensitive material. This is commonlycalled a reference wave or reference field. The reference field interferes with theobject field and generates interference fringes.

The interference fringes are spatial distribution of optical intensity. This distribu-tion has a three-dimensional (3D) structure over the 3D space where the referenceand object field are overlapped. If the fringe is recorded on a thin sheet material,only the two-dimensional (2D) cross section of the 3D fringe is recorded on thematerial. This is called a hologram or a thin hologram more exactly. An example ofthe hologram fringe pattern is shown in Fig. 2.2.

In the reconstruction step, we remove the illuminating light for the subject and thesubject itself, as shown in Fig. 2.3. When the same light as the reference field illumi-nates the hologram, the fringe pattern diffracts the illuminating light, and reconstructsthe original object field. To be accurate, the field diffracted by the fringes includesthe same component as the original object field. As a result, viewers can see the lightof the original subject; they look at the subject placed at the original position. Thisprocess is called optical reconstruction of the hologram.

As mentioned a little more precisely in Sect. 2.3. The illuminating light isdiffracted by the 2D fringe pattern and converted into the output light very simi-

© Springer Nature Switzerland AG 2020K. Matsushima, Introduction to Computer Holography,Series in Display Science and Technology,https://doi.org/10.1007/978-3-030-38435-7_2

13

14 2 Overview of Computer Holography

Beam splitter

Mirror

Mirror

Lens

Subject

Photo-sensitivematerial

Laser

Collimatinglens

Lens

Fig. 2.1 The recording step of a hologram in conventional optical holography

Fig. 2.2 An example of the hologram fringe pattern. Note that this pattern is recorded by using animage sensor in practice

lar to the object field. This means that the whole spatial information of the objectfield emitted by a subject, which has a 3D structure, is recorded as the 2D image.In other words, information of the original object field is condensed into the sheetmaterial.

Optical reconstruction of a hologram means to playback the original object fielditself, which is frozen in the form of the 2D fringe pattern. Therefore, holographic 3Dimages properly convey the entire depth cues of the subject; the 3D image providesparallax disparity, vergence, accommodation, and motion parallax to the viewers, asmentioned in Sect. 1.1.

2.2 Computer Holography and Computer-Generated Hologram 15

Fig. 2.3 Opticalreconstruction of ahologram. The light recordedin the hologram in the formof a fringe pattern isreproduced by diffraction ofthe illuminating light

Beam splitter

Mirror

Lens

Fringepattern

Laser

Collimatinglens

Reconstructedobject

2.2 Computer Holography and Computer-GeneratedHologram

When the recorded hologram is a thin hologram mentioned above, i.e., the fringepattern has a simple 2D structure, the fringe pattern is just a monochrome picturewhose resolution is unusually high. This type of interference fringes can be recordeddigitally using an image sensor as well as a photo-sensitive material if the sensorresolution is high enough for recording. This technique, digital recording of the fringepattern is commonly called digital holography (DH) in limited meaning.1

If you own a printer whose resolution is enough high to print the fringe pattern,you can print the hologram recorded by the image sensor. Any special feature is notrequired for the printer except for the resolution. This means that we can opticallyreconstruct any object field if we can digitally generate the fringe pattern for theobject using a computer. In this technique, any real object is no longer required forproducing the 3D image. This type of hologram is commonly called a computer-generated hologram or the abbreviation: a CGH.

The author also uses a word computer holography in this book. Computer holog-raphy is the technique to create a CGH that reconstructs a virtual object image fromthe numerical model in a narrow sense. The author uses “computer holography” instrong consciousness of computer graphics (CG) that also produces 2D images ofa virtual object. In a broad sense, computer holography is simply the technique forcreating holograms by using digital computers; it is no matter whether a physicalobject is required for producing the 3D image or not.

We can also use a combination of digital recording and reconstruction of holo-grams; the fringe pattern of a hologram is digitally recorded by an image sensor,and the 3D image is optically reconstructed from the digital fringe image. This tech-nique is referred to as digitized holography, because the technique replaces the whole

1The author does not support this limitedmeaning of the word “digital holography,” i.e., not approveof referring to only the recording step as “digital holography.”Thisword should be usedmorewidely.

16 2 Overview of Computer Holography

process in optical holography: recording and reconstruction, with the digital coun-terparts. The detailed technique of digitized holography is discussed in Chap. 14.

If the photo-sensitive material in optical holography is enough thick and therecorded fringe pattern has a 3D structure, the hologram is called a thick or vol-ume hologram (for the detail, see Sect. 7.2). In this case, we commonly cannot printthe fringes, because ordinary printers only print 2D patterns. However, a specialprinter, called a wavefront printer, has the ability to print 3D fringes. This type ofspecial printer is briefly discussed in Sect. 15.4.

2.3 Steps for Producing CGHs and 3D Images

Figure2.4 shows a generalized procedure for producing a CGH and reconstructingthe 3D image. The first step for producing a CGH is to prepare the object fieldnumerically. Here, suppose that the object field is given by O(x, y) that is two-dimensional distribution of complex amplitudes. The object field is usually producedfrom a set of model data. This process is referred to as numerical synthesis of objectfield or simplyfield rendering in this book.This is themost important anddifficult step

Numerical synthesis of object field(Field rendering)

Model data

Object field

CGH

Coding

Print fringe pattern

Fringe pattern

O (x, y)

t (x, y)

Digital capture of object field

Physical object

Display fringe pattern(Optical reconstruction) Optical reconstruction

3D image

Electro -holography

Fig. 2.4 Steps for creating a CGH and reconstructing the 3D images in computer holography