and Computational Photography - ? What is Computational Photography? Computational photography is

  • Published on
    25-Aug-2018

  • View
    220

  • Download
    8

Transcript

Pixels,Numbers,andProgramsComputational PhotographyComputationalPhotographyStevenL.TanimotoPixels,Numbers,andPrograms;S.Tanimoto 1ComputationalPhotographyOutlineWh t i t ti l h t h ?Whatiscomputationalphotography?AutofocustechniquesHi h d i (HDR) h hHighdynamicrange(HDR)photographyCatadioptric camerasSeparatingimagesduetodirectandindirectlightingOtherdevelopmentsPixels,Numbers,andPrograms;S.Tanimoto ComputationalPhotography 2WhatisComputationalPhotography?Computational photography is an emerging technology forComputationalphotographyisanemergingtechnologyforacquiringimagesthroughacombinationofoptics,sensors,andcomputers.Itincludesautomaticcameracontrol,computationofpixelvaluesusingmultiplesourcesofinformation,aswellasg p ,computingalternativerepresentationsofvisualinformationto2Dimages(e.g.,3Dmodels).Pixels,Numbers,andPrograms;S.Tanimoto ComputationalPhotography 3AutofocustechniquesFocusing a camera lens was once a tedious chore.Focusingacameralenswasonceatediouschore.Instant(film)camerasusedfixedfocus,limitingphotographytowelllitscenesandfast(highASA)p g p y ( g )films.Acousticrangefinding wasusedonsomecamerastog f gdeterminethedistancefromthecameratothefirstsurfaceinthemiddleofthefieldofview.Digitalcameraschangetheeconomicsofautofocus.Now,techniquesbasedonsharpnessmeasurementsff tiPixels,Numbers,andPrograms;S.Tanimoto ComputationalPhotographyareeffective.4Autofocustechniques(cont.)Brennersharpness:Tenengrad sharpness:HereGx andGy arethehorizontalandverticalgradientoperatorsPixels,Numbers,andPrograms;S.Tanimoto ComputationalPhotographyx y g pusedintheSobel edgedetector.5Autofocustechniques(cont.)xFrame#1 Frame#10Sharpnessasafunctionoffocussetting.BestfocusPixels,Numbers,andPrograms;S.Tanimoto ComputationalPhotography 6gisatthe10th frame.HighDynamicRangePhotographyThe dynamic range of a photo or camera is the rangeThedynamicrangeofaphotoorcameraistherangeofbrightnesslevelsthatitcanaccuratelycapture.Historically,dynamicrangewaslimitedby(a)filmy, y g y ( )technology,and(b)digitallightsensortechnology.Thephotographerschallenge:takeapictureofascenethatcontainsbothbrightanddarkregions,withoutlosingthedetailsinoneoftheseregions.Pixels,Numbers,andPrograms;S.Tanimoto ComputationalPhotography 7HDR(cont.)The brightly lit area is capturedThebrightlylitareaiscapturedeffectivelyinthisphoto(takenwitha1/60sec.exposure).Butthelowerhalfofthephotop(theindoorpartofthescene)istoodarkandlackingincontrast.Pixels,Numbers,andPrograms;S.Tanimoto ComputationalPhotography 8HDR(cont.)Here, the darkest areas areHere,thedarkestareasarecapturedeffectively(takenwitha15sec.exposure).Buttheupperhalfofthephotopp p(theoutdoorpartofthescene)istoobrightandlackingincontrast.Pixels,Numbers,andPrograms;S.Tanimoto ComputationalPhotography 9HDR(cont.)Now, both areas have someNow,bothareashavesomevisiblestructure(takenwitha1/2sec.exposure).Butneithershowsdetailswell.Pixels,Numbers,andPrograms;S.Tanimoto ComputationalPhotography 10HDR(cont.)HDR to the rescue!HDRtotherescue!Takingafullsequenceofexposures,webuildahighp , gdynamicrangeimageinsidethecomputer.Fromit,weapplytonemappingtogetanimagethathasgoodcontrastacrosstherangerange.Pixels,Numbers,andPrograms;S.Tanimoto ComputationalPhotography 11HDR(cont.)HDR to the rescue!HDRtotherescue!Takingafullsequenceofexposures,webuildahighp , gdynamicrangeimageinsidethecomputer.Fromit,weapplytonemappingtogetanimagethathasgoodcontrastacrosstherange Th id l i i i frange. Theidea:getluminosityinfoabouteachpixelfromeachimage,butweighttheevidenceaccordingtowhateachimageisgood atPixels,Numbers,andPrograms;S.Tanimoto ComputationalPhotographygoodat.12Catadioptric ImagingUsing optical systems that combine lenses andUsingopticalsystemsthatcombinelensesandmirrors,anamorphicimagescanbecapturedthatcontainentirepanoramas.Computerscaninverttheanamorphicdistortion.CourtesyofProf.ShreeNayar ofColumbiaUniversity.Pixels,Numbers,andPrograms;S.Tanimoto ComputationalPhotography 13Catadioptric ImagingPixels,Numbers,andPrograms;S.Tanimoto ComputationalPhotography 14Catadioptric CamerasTheOneshot360(RemoteReality,Inc.).Threeothercatadioptric cameras.http://www.cs.columbia.edu/CAVE/projects/cat_cam_360/Pixels,Numbers,andPrograms;S.Tanimoto ComputationalPhotography 15SeparatingDirect&IndirectLightingThecolorofanobject(specificallyatapointPontheobject)inascenedepends on two different physical effects:dependsontwodifferentphysicaleffects:a. Directreflectionoflightfromthelightsourceoffthesurfaceoftheobject(atpointP),andb Light coming from point P on the object that did not come directlyb. LightcomingfrompointPontheobjectthatdidnotcomedirectlyfromthelightsource:i.lightthathascomefromthelightsourcebutbouncedoffothersurfaces before getting to PsurfacesbeforegettingtoP.ii.lightthathasenteredthesurfacematerial(e.g.,thepaintlayer,whichistypicallyadielectricmaterialandispartiallytranslucent)elsewhere from point P but emerges from point PelsewherefrompointPbutemergesfrompointP.Weareoftenunawareof(ii).However,usingcomputationalphotography,we can separate the effects of (i) and (ii) The results are amazingPixels,Numbers,andPrograms;S.Tanimoto ComputationalPhotographywecanseparatetheeffectsof(i)and(ii).Theresultsareamazing.16Direct&Indirect(cont.)indirectlightsource surfacesinthescenePcameraRtRadianceatPtowardsthecamera:Rt =Rd +RgRg istheglobalcomponentofradiance(basedonallindirectilluminationofP).Rt isthetotalradiance.Pixels,Numbers,andPrograms;S.Tanimoto ComputationalPhotography 17Direct&Indirect(cont.)indirectlightsourceBarriertocastashadowonPPcameraRgRadianceatPtowardsthecamera:Rt =Rd +RgRg istheglobalcomponentofradiance(basedonallindirectilluminationofP).Rt isthetotalradiance.By measuring R and R then we can compute R as R RPixels,Numbers,andPrograms;S.Tanimoto ComputationalPhotography 18BymeasuringRt andRg thenwecancomputeRd asRt RgResultsofSeparationByusingstructuredlight(projectionofcheckerboards),itisnotnecessaryto have a separate shadow image for each pixel. In theory, only 2 imagestohaveaseparateshadowimageforeachpixel.Intheory,only2imagesareneeded,butinpracticeitsbesttohave10to20images,accordingtoKrishnanandNayar.Rt Rd RgIn this example most of the radiance from global illumination is due toInthisexample,mostoftheradiancefromglobalilluminationisduetointerreflectance amongtheeggs.Pixels,Numbers,andPrograms;S.Tanimoto ComputationalPhotography 19ResultsofSeparationAscenewithgrapesandcheeseMuch of the natural color (as seen in R ) comes from R and not RMuchofthenaturalcolor(asseeninRt )comesfromRg andnotRd .Rt Rd RgHere,muchoftheradiancefromglobalilluminationisduetolightpassingthroughthetranslucentgrapesorcheese.Thecheese,shownwithdirectilluminationonly,issomewhatunappetizing.Thegrapesalsoappearblue.Pixels,Numbers,andPrograms;S.Tanimoto ComputationalPhotographyApplications:imageenhancement,imageanalysis.20OtherDevelopments3Dcameras cameratakestwoormoreshotsofsamescene,then builds 3D model of scenethenbuilds3Dmodelofscene.Redeyereduction.Alreadystandardistheautomaticuseofapreflashtoconditionthepupilsofphotosubjectstoclosedown.Redeyecanalsobereducedusingfacedetectionmethodsandspecializedalgorithms.Light field cameras By capturing a sample of the complete lightLightfieldcameras.Bycapturingasampleofthecompletelightfieldattheviewpoint,operationssuchasfocusingonspecificobjectscanbeperformedlater.Pixels,Numbers,andPrograms;S.Tanimoto ComputationalPhotography 21OtherDevelopments(cont.)IncameraHDR.Automaticexposurebracketingisleadingtheway to built in HDR capturewaytobuiltinHDRcapture.Motionblurinversion.Thecameracanestimateitsmotionusingacombinationofaccelerometerdataandvideodata.Themotioninfocanthenbeusedtopartiallyinvertthemotionblur.Allsmilessnapshots.Usingfacedetection,smiledetectionalgorithms and inpainting techniques it is possibly to synthesizealgorithms,andinpainting techniques,itispossiblytosynthesizeanimageofagroupofpeopleinwhicheveryoneissmiling,eventhoughineachcomponentshot,therewassomeonenotsmiling.Pixels,Numbers,andPrograms;S.Tanimoto ComputationalPhotography 22

Recommended

View more >