2012 Electronic Imaging - SPIE · Digital Imaging Sensors and Applications 8298 Sensors, Cameras, ... The single lens color/depth-unified sensor [1] can provide real time color and

electronicimaging.org • TEL:+17036429090 • [email protected] 1

Conferences and Courses

22–26January2012

HyattRegencySanFranciscoAirportHotel

Burlingame,California,USA

electronicimaging.org

2012ElectronicImaging

Technical Summaries

22–26 January 2012

2 electronicimaging.org • TEL:+17036429090 • [email protected]

Contents

3D Imaging, Interaction, and Measurement8288 Stereoscopic Displays and Applications XXIII

(Woods, Holliman, Favalora) . . . . . . . . . . . . . . . . . . . 3-22

8289 The Engineering Reality of Virtual Reality 2012(McDowall, Dolinsky) . . . . . . . . . . . . . . . . . . . . . . . . 23-28

8290 3D Image Processing (3DIP) and Applications 2012(Baskurt, Sitnik). . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29-37

Imaging, Visualization, and Perception8291 Human Vision and Electronic Imaging XVII

(Rogowitz, Pappas, de Ridder) . . . . . . . . . . . . . . . . 38-50

8292 Color Imaging XVII: Displaying, Processing, Hardcopy, and Applications(Eschbach, Marcu, Rizzi). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51-60

8293 Image Quality and System Performance IX(Gaykema, Burns) . . . . . . . . . . . . . . . . . . . . . . . . . . 61-70

8294 Visualization and Data Analysis 2012(Wong, Kao, Hao, Chen). . . . . . . . . . . . . . . . . . . . . . . . . . . . 71-78

Image Processing8295AImage Processing: Algorithms and Systems X

(Egiazarian, Agaian, Gotchev). . . . . . . . . . . . . . . . . 79-87

8295BParallel Processing for Imaging Applications II(Recker, Wang). . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88-89

8296 Computational Imaging X(Bouman, Pollak, Wolfe). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90-96

8297 Document Recognition and Retrieval XIX(Viard-Gaudin, Zanibbi). . . . . . . . . . . . . . . . . . . . . 97-102

Digital Imaging Sensors and Applications8298 Sensors, Cameras, and Systems for Industrial/

Scientific Applications XIII(Widenhorn, Nguyen, Dupret). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103-109

8299 Digital Photography VIII(Battiato, Rodricks, Sampat) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110-116

8300 Image Processing: Machine Vision Applications V(Bingham, Lam). . . . . . . . . . . . . . . . . . . . . . . . . . . 117-120

8301 Intelligent Robots and Computer Vision XXIX: Algorithms and Techniques(Röning, Casasent)121-131

Multimedia Processing and Applications8302 Imaging and Printing in a Web 2.0 World III

(Lin, Allebach, Fan) . . . . . . . . . . . . . . . . . . . . . . . . 132-136

8303 Media Watermarking, Security, and Forensics 2012 (Memon, Alattar, Delp)..................... 137-141

8304AMultimedia on Mobile Devices 2012(Creutzburg, Akopian) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142-147

8304BMultimedia Content Access: Algorithms and Systems VI(Snoek, Sebe, Kennedy) . . . . . . . . . 148-150

Visual Information Processing and Com munication8305 Visual Information Processing and Communication III

(Said, Guleryuz, Stevenson) . . . . . . . . . . . . . . . . . 151-156


Conference 8288 · Stereoscopic Displays and Applications XXIIIMonday-Wednesday23-25January2012

Part of Proceedings of SPIE Vol. 8288 Stereoscopic Displays and Applications XXIII

lens.Angle-sensitivepixelsaremicro-scaledeviceswhichcaptureinformationaboutboththeintensityandincidentangleofthelighttheysee.Becausethesepixelsacquirearicherdescriptionofincidentlightthatconventionalintensity-sensitivepixels,oursensoronlyrequiresasimplecameraobjectivetorecoverlight-fieldinformationfromavisualscene.Singleimagescapturedbyourcameracanbedirectlyusedforbothsyntheticrefocusanddepthmapcomputation.Thealgorithmsusedforrefocusandrange-findingrelyonthefactthateachangle-sensitivepixelprovidesaresponsesimilartoonecomponentofa2DHartleytransformwithadistinctfrequencyandorientation.Byusingseveralkindsofdifferentpixelsthroughouttheimagesensor,weobtainafull,low-orderHartleytransformoflocalangle,whichismappedtoalocal,spatialHartleytransformbythecameralens.

8288-04, Session 2

Polarizing aperture stereoscopic cinema cameraL.Lipton,LeonardoIP(UnitedStates)

SUMMARY:Anewstereoscopicdigitalcamera/lensforcinematographyhasbeendesignedwhichisespeciallywellsuitedforfeaturefilmandtelevisionapplicationswherereducedinteraxialseparationsarerequired.Thecurrenttechnologyemployedforsuchcamerasusesso-calledbeam-splitterrigsthatrequirethecoordinationoftwocamerasworkingtogetherasiftheywereonestereoscopiccamera.Therearemanyengineeringdifficultiestoovercometomakesuchadeviceworktospecification,andgiventheassessmentofthestateoftheartbymanyinthefield,anewapproachisrequired.

Thenewapproachtobedescribedinthispaperusespolarizingleftandrightaperturesofasuitablydesignedlensorlenses,workingincombinationwithadigitalsensororsensorstoprovideawaytoselecttheleftandrightimages.Thedesignallowsforsensorsofsuper35size,forexample,togivecinematographerstheabilitytocontroldepthoffieldastheyhavefromtheinceptionof35mmcinematography.Thecamera/lenstechnologyallowsforapackagetohaveaformfactorandappearancesimilartothatofconventionalproductioncameraswithvastlysimplifiedinstrumentationandahighdegreeofconfidencethatwhatcomesoutofthecameradoesnotneedtoberectifiedinpost-production.

8288-05, Session 2

Hardware-efficient, real-time 3D image processing for single lens color/depth-unified sensor: from depth noise reduction, depth super-resolution, to stereoscopic image synthesisH.Lim,Y.S.Kim,O.Choi,B.Kang,S.Kim,K.Lee,J.D.Kim,C.Kim,SamsungAdvancedInstituteofTechnology(Korea,Republicof)

Thesinglelenscolor/depth-unifiedsensor[1]canproviderealtimecolorandtimeofflightdepthimages.Inthispaper,wepresenthardware-efficientalgorithmsfor3Dimageprocessingusedincolor/depth-unifiedsensors,includingdepthnoisereduction,depthsuper-resolution,andstereoscopicimagesynthesisforinstantwatchinginstereoscopicdisplays.Fortheeffectivedepthnoisereductionweutilizeaseparablenonlocalmeansfilterthatcollectspatchesalongverticalandhorizontaldirectionsfromthetargetpixelpositionandperformsweightedaveragingoftheircenterpixelswithpatch-wisedepthsimilarity.Forthesuper-resolutionofthelowdepthimageuptothatofthecolorimageweapplyahierarchicaltrilateralfilterthatusesthreeweightsintermofpixelproximity,colorsimilarity,anddepth

8288-01, Session 1

Matching stereo 3D reconstructions of CT/MRI data and intraoperative stereo video for medical education and treatment planningJ.F.Ilgner,M.Westhofen,Univ.HospitalAachen(Germany)

Introduction:AnatomyofsofttissuestructuresintheHeadandNeckiscomplex.Whilenavigatedsurgerycanmaintainthesurgeon’sorientationintraoperatively,treatmentplanningisneededtochoosethecorrectstrategyforasuccessfulsurgicalprocedure.MRIandCTdatausuallyrelyon3Dreconstructionstoobtainthisobjective.

MaterialandMethods:WetookMRIandCTdatafromthreedifferentpatientspresentingwithcomplexlesionsintheHeadandNeck.3Dreconstructionwasalreadyavailablebythetimeofpre-operativescanning,whilesteppinginrotatedreconstructionswassmallenoughtoallow3Dstereomatching.ExistingCTandMRIdatahadtobere-orientedtomatchthesurgicalsituation.

Results:Re-orientingtheradiologicaldatarequiredextraeffortandabout30min.percase,involvingtheradiologistaswellastheENTsurgeon.Theresult,however,allowedCTandMRIdatatobeincludedincasepresentationsforjuniorandadvancedsurgeons,thusdevelopingacommonstrategyfortreatmentplanningandsurgicalsetup.

Conclusion:TheextratimeandeffortinmatchingstereoCTandMRIdatawithintraoperativestereo3Dvideorecordingsdataisjustifiedforselectedcaseswhichserveasmodelsforeducatingdoctorsintreatmentplanning.

8288-02, Session 1

Stereoscopic desktop VR system for tele-maintenanceM.Kleiber,FraunhoferFKIE(Germany)

Wehavedevelopedatele-maintenancesystemwhichconsistsofanaugmentedreality(AR)systemforthemechanicandadesktopvirtualreality(VR)systemfortheexpert.Theexperthasaccesstodifferentdatasourcescapturedattheremotesiteand3Dconstructiondata.TheremoteviewofthemechanicisreconstructedintheVRsystembyusingthetrackingdatafromtheARsystem.Theexpertcanusethe3Dmodeltohighlightparts,attachannotationsorcreateanimatedinstructions.Tosupporttheexpert’sspatialperceptionwehavedesignedthesystemtoproducestereoscopicoutput.Theaimofthedevelopmentwastheintegrationofallnecessaryfunctionsintoasinglestereoscopicapplication.Therefore,allnecessaryfunctionsofthesystemhadtobeusablewhenviewedstereoscopically.Goodstereoscopicviewingconditionswerethusveryimportant.Thiswasthemajorchallengeinregardtointegratingthemonoscopicsensordataandprovidinganintuitiveuserinterface.Wedidnotwanttogloballylimitthestereoscopiceffect.Tocontroltheamountofdisparityweemployashaderbaseddepthoffieldeffectwhichiscoupledtothepointoffocusoftheuser.Theconcept,implementationandevaluationwillbedescribedinthepaper.

8288-03, Session 2

Angle-sensitive pixels: a new paradigm for low-power, low-cost 2D and 3D sensingA.Wang,S.S.Hemami,A.Molnar,CornellUniv.(UnitedStates)

Wedemonstratealight-fieldcamerawhichusesanimagesensorcomposedofangle-sensitivepixelsandaconventionalcamera


similarity.Finallyforthestereoscopicviewsynthesistheleftsidecolorimageisgeneratedbywarpingtheoriginalcolorimagebasedonthedisparitymapconvertedfromthedepthimageandwethenrecoverthedisoccludedregionsusingonlythebackgroundboundarypixelswithoutintrusionoftheforegroundpixelvalue.

AlltheabovealgorithmswereimplementedonXilinxVirtex-5LX330andthecomputationtimefora1280x960colorimageanda320x240depthimageis87msat54Mhzoperatingclock.Asdemonstratedintheattachedfigures,theproposedframeworkproducesrealisticstereoscopicimageinrealtime.

[1]S.-J.Kim,J.D.K.Kim,S.-W.Han,B.Kang,K.Lee,andC.-YKim,“A640×480ImageSensorwithUnifiedPixelArchitecturefor2D/3DImagingin0.11μmCMOS,”inSymposiaonVLSIcircuitsDig.Tech.Papers,pp.92-93,June,2011.

8288-06, Session 2

Unassisted 3D camera calibrationK.Atanassov,V.Ramachandra,S.Goma,QualcommInc.(UnitedStates)

Withtherapidgrowthof3Dtechnology,3Dcapturehasbecomesacriticalpartofthe3Dfeature.3Dqualityisaffectedbythescenegeometryaswellastheon-the-deviceprocessing.Anautomatic3Dsystemusuallyassumesknowncamerasposesaccomplishedbyafactorycalibrationusingaspecialchart.Inareallifeoperation,theresultfromthefactorycalibrationcanbenegativelyimpactedbymovementsofthelensbarrelduetoshaking,focusing,orcameradrop.Ifanyofthosefactorsdisplacescamerasopticalaxissuchasverticaldisparityexceedsthemaximumtolerablemargin,the3Dusermayexperienceeyestrainorheadache.Tomake3Dcapturemorepractical,oneneedstoconsiderunassisted(onarbitraryscene)calibration.Inthispaperweproposeanalgorithmthatreliesondetectionand2Dmatchingofkeypointsbetweenleftandrightimages.Framescontainingerroneousmatchesaredetectedanddiscarded,aswellasframeswithinsufficientlyrichkeypointsconstellation.Roll,pitchyaw,andscaledifferencesbetweenleftandrightframesarethenestimated.Thealgorithmperformanceisevaluatedintermsofremainingverticaldisparityascomparedtothemaximumtolerableverticaldisparity.

8288-07, Session 2

Auto convergence for stereoscopic 3D mobile camerasB.Zhang,S.Kothandaraman,A.U.Batur,TexasInstrumentsInc.(UnitedStates)

Thecommercialsuccessof3Dmoviesisgeneratinggreatinterestinstereoscopic3Dcaptureanddisplaytechnologies.3DcapableTVs,digitalcameras,andmobiledevicesareenteringtheconsumerelectronicsmarket,whichenableconsumerscaptureanddisplaytheirown3Dcontent.However,amajorchallengetothesuccessofthese3Dcapabledevicesistheviewingcomfort.Consumer3Dcamerashavefixedcameraseparationandorientation,andthe3Ddisplayviewingdistanceistypicallyshort.Forthesereasons,thevergence-accommodationconflictisparticularlypronounced,whichcausesdiscomfortandeyefatigue.Inthispaper,wepresentaStereoAutoConvergence(SAC)algorithmforconsumer3Dmobilecamerasthatreducesthevergence-accommodationconflictonthe3Ddisplaybyadjustingthedepthofthe3Dsceneautomatically.Ouralgorithmprocessesstereovideoinreal-timeandshiftseachstereoframehorizontallybyanappropriateamountinordertoconvergeonachosenobjectinthatframe.Thealgorithmstartsbyestimatingdisparitiesbetweentheleftandrightimagepairsusingcorrelationsofthehorizontalprojectionsoftheframe.Theestimateddisparitiesarethenanalyzedbythealgorithmtoselectapointofconvergenceaccordingtoacenter-convergenceorsurround-convergencestrategy.Thecurrentandthetargetdisparitiesofthechosenconvergencepointdeterminehowmuchhorizontalshiftisneeded.Adisparitysafetycheckisthenperformedtodeterminewhetherornotthemaximumandminimumdisparitylimitshavebeenexceededafterautoconvergence.Ifthe

limitshavebeenexceeded,furtheradjustmentsaremadetosatisfythesafetylimits.Finally,convergenceisperformedbyshiftingtheleftandtherightframesaccordingly.OursolutionhasbeenimplementedonaTIOMAP®processorrunninginreal-timeat30fps.Itsignificantlyimprovesthe3Dviewingcomfort.

8288-08, Session 3

Beyond parallax barriers: applying formal optimization methods to multi-layer automultiscopic displaysD.Lanman,MassachusettsInstituteofTechnology(UnitedStates);G.Wetzstein,BauhausUniv.Weimar(Germany);M.Hirsch,MassachusettsInstituteofTechnology(UnitedStates);W.Heidrich,TheUniv.ofBritishColumbia(Canada);R.Raskar,MassachusettsInstituteofTechnology(UnitedStates)

Thistalkfocusesonresolvinglong-standinglimitationsofparallaxbarriersbyapplyingformaloptimizationmethods.Thislineofresearch,beingpursuedbytheCameraCulturegroupattheMITMediaLab,hasledtotwonewattenuation-baseddisplays:“HR3D”(http://www.hr3d.info)and“Layered3D”(http://www.layered3d.info).The“High-Rank3D(HR3D)”displayconsistsofastackedpairofmodifiedLCDpanels;ratherthanusingheuristically-definedparallaxbarriers,bothlayersarejointly-optimizedusinglow-ranklightfieldfactorization,resultinginincreasedbrightness,refreshrate,andbatterylifeformobileapplications.The“Layered3Ddisplay”extendsthisapproachtomulti-layereddisplayscomposedofcompactvolumesoflight-attenuatingmaterial.Suchvolumetricattenuatorsrecreatea4Dlightfieldorhigh-contrast2Dimagewhenilluminatedbyauniformbacklight.Together,theseprojectsrevealnewgeneralizationstoparallaxbarrierconcepts,uniquelyleveragingthecompressivenatureofnatural3Dscenesfordisplayapplications.Thistalkprimarilyfocuses,however,onintroducinganewoptimizationframeworkthatextendsthatdevelopedinthesepriorworks,beingthefirsttoallowjointmulti-layer,time-multiplexedoptimization.Simulationsarecomparedtolaboratoryprototypes,establishingnewcapabilitiesformulti-layer,attenuation-baseddisplays.

8288-09, Session 3

3D image quality of 200-inch glasses-free 3D display systemM.Kawakita,NHKScience&TechnicalResearchLabs.(Japan);S.Iwasawa,NationalInstituteofInformationandCommunicationsTechnology(Japan);M.Sakai,Y.Haino,M.Sato,JVCKENWOODHoldings,Inc.(Japan);N.Inoue,NationalInstituteofInformationandCommunicationsTechnology(Japan)

Wehaveproposedalargeglasses-free3Ddisplaythatcoulddisplay3Dimagesonlargescreens(tenstohundredsofinchesdiagonally)byusingmulti-projectorsandanopticalscreenconsistingofaspecialdiffuserfilmandlargecondenserlens.Toachievehigh-presencecommunicationusing3Dimages,wenumericallyanalyzedthefactorsresponsiblefordegradingtheimagequalitytoincreasetheimagesize.Amajorfactorusedfordeterminingthe3Dimagequalitywasthearrangementofcomponentunits,suchasprojectorarrayandcondenserlens,andthediffuserfilmcharacteristics.Wedesignedandfabricatedaprototype200-inchglasses-freedisplaysystemusingthenumericalresults.Weselectedasuitablediffuserfilmandcombineditwithanoptimallydesignedcondenserlens.Weused64high-definitionprojectorunitsandobtainedviewinganglesof13.5°.Thegapbetweentheparallaximageswas<2.5cmandhelpedusrealizenatural3Dimageswithsmoothmovingparallax.Thewidthoftheviewingareawas1.3matadistanceof5.5m;anobservercouldseethe3Dimageseventhoughhe/shemovedmorethan±2minthedepthdirection.Thissystemcoulddisplaylife-sizeglasses-free3Dimagesofacarusingnaturalparallaximages.

Conference 8288


8288-10, Session 3

Implementation and analysis of an autostereoscopic display using multiple liquid crystal layersH.Gotoda,NationalInstituteofInformatics(Japan)

Multilayerdisplaysareusuallyconstructedbystackingmultipleliquidcrystalpanelsalongwithapairoflinearpolarizersandalightsource.Previoustheoreticalanalyseshaveindicatedthatsuchadisplaycouldshowthelightfieldofa3Dsceneifthepanelsinthedisplayareproperlycontrolled.However,asfarasweknow,implementationsrealizingthisfeaturehavenotbeenreportedyet.Inthisstudy,wepresentanimplementationofamonochromaticmultilayerdisplayusingIPS-modeLCDpanels,andreportthatareasonableapproximationofthelightfieldisactuallyobservedfromthedisplay.

Amajorobstacletoconstructingamultilayerdisplayisthecomplicatedopticalpropertyofliquidcrystalpanels.Sinceanaccurateanalyticrepresentationofthepixel’sfunctionisdifficulttoobtain,wehavedevelopedtwomodelsthatapproximatelydescribesit.Thesetwomodelsarethenintegratedintoatwo-stagemethod,usingwhichnearlyreal-timecontrolofthepanelsinthedisplayhasbeenachieved.Finally,thequalityofthelightfieldobservedfromtheprototypeimplementationisevaluatedbyexaminingtheimagesphotographedatseveralcamerapositions.Theresultsshowthatfairlygoodimagesareobservedwithintheviewingangleof30degrees.

8288-11, Session 3

360-degree, three-dimensional table-screen display using small array of high-speed projectorsS.Uchida,Y.Takaki,TokyoUniv.ofAgricultureandTechnology(Japan)

Anew360-degreethree-dimensionaltable-screendisplayisproposed,whichcombinestheadvantagesofthepreviouslyproposedhigh-speedprojectorandmulti-projectorsystems.Theproposedsystemconsistsofasmallnumberofhigh-speedprojectorsandarotatingscreen.Becauseeachhigh-speedprojectorislocatedoutsidetherotatingaxisofthescreen,multipleprojectorscanbealignedaboveorbelowtherotatingscreen.Thelensshifttechniqueisusedtosuperimposemultipleimagesgeneratedbyallprojectorsontherotatingscreen.Thescreenhasanoff-axislensfunctionsuchthattherotationofthescreengeneratesnumerousviewpointsonacirclearoundtherotatingscreen.Eachprojectorgeneratesnumerousviewpointsonadifferentcircle.Theuseofmultipleprojectorsenablesthereductionoftherotationspeedofthescreen,theincreaseinthenumberofcolors,andtheincreaseinthenumberofviewpoints.AnexperimentalsystememployingtwoDMDprojectorswasdemonstrated.Therotatingspeedofthescreenwasreducedfrom3,333to1,666rpminordertogeneratea3Dimagewithaframerateof56Hz.Eachprojectorgenerated800viewpoints.Thediameterofthescreenwas300mm.The3Dresolutionwas768×768.

8288-12, Session 4

Focus mis-match detection in stereoscopic contentF.Devernay,S.Pujades,V.Ch.A.V.,INRIARhône-Alpes(France)

Live-actionstereoscopiccontentproductionrequiresastereorigwithtwocamerasthatarepreciselymatched(intermsoffocallength,colorimetry,etc.)andaligned(sothatthestereodisparityisessentiallyhorizontal).Whilemostdeviationsfromthisperfectsetupcanbecorrectedeitherliveorinpost-production,adifferenceinthefocusdistanceorfocusrangebetweenthetwocameras,whichisdifficulttocheckvisually,willleadtounrecoverabledegradationsofthestereoscopicfootage.Weproposealgorithmsthatcheckforfocusdifferencesbetweentheleftandrightimages,andgivedirectionstothe

operatortosolvetheseproblems.Theproposedmethodwasevaluatedonray-tracedphotorealisticscenes,andonrealstereoscopicimagepairs,andprovestoperformwellinmostcases.

8288-13, Session 4

2D-to-3D image conversion: leveraging 3D data on the netJ.Konrad,G.Brown,M.Wang,P.Ishwar,BostonUniv.(UnitedStates)

Manymethodshavebeenproposedtodatetoconvert2Dimagesto3Dstereopairs.Themostsuccessfulonesinvolvehumanoperatorsand,therefore,aretime-consumingandcostly.Fully-automatic2D-to-3Dconversionmethodshavenotachievedyetthequalityofoperator-supervisedmethods.Themainreasonforthisisthefactthatautomaticmethodsusuallyrelyonassumptionsaboutthecaptured3Dscenethatareoftenviolatedinpractice.Inthispaper,weexplorearadicallydifferentideainspiredbyourworkonsaliencydetectioninimages;insteadofdeterministicallyspecifyingascenemodelfortheinput2Dimage,weproposeto“learn”themodel,andmorespecificallytolearnthescenedepth,fromalargedictionaryofstereopairs,suchasYouTube3D.Themaincontributionofourworkistherealizationthatamongmillionsofstereopairson-linetherelikelyexistverymanystereopairswithsimilarcontenttothatofthe2Dquery.Weexploitthisobservationbyconstructing,fromtheon-linestereopairs,thedepthof3Dscenecapturedbythe2Dquery.First,weperformtheso-called“knearestneighbor”(kNN)searchforour2Dqueryusing,forexample,theleftimagefromallon-linestereopairs.Thisreturnsanorderedlistofleftimages,fromthemosttotheleastsimilarvis-a-visthe2Dquery.Wediscardallbutthetopkmatches(kNNs).Then,weestimateageometrictransformationbetweenthe2DqueryandeachofthekNNsusingtheSIFTflow.Thisstepisneededtoaccountfordifferencesinscale,orientation,position,etc.,ofobjectsbetweenthe2DqueryandeachkNN.Finally,assumingthatdepthisknownforeachon-linestereopair,weusetheinverseoftheabovetransformationtoobtainthedepthofthe2Dquery.Inpractice,inthelaststepweusedisparityinsteadofdepthandsincethegeometrictransformationisnotinvertibleweuseanapproximateinverse.Havingobtainedkdisparityfieldsfromknearestneighbors,wefusethembycomputingthemedian.Weusetheobtaineddisparityfieldtogeneratetherightimagefromthe2Dquerywhilehandlingocclusionsandnewly-exposedareasintheusualway.Todate,wehaveimplementedasimplifiedversionoftheabovealgorithmthatdemonstratestheproof-of-concept.Withthecontinuouslyincreasingamountof3Ddataon-lineandwiththerapidlygrowingcomputingpowerinthecloudtheproposedframeworkseemsapromisingalternativetooperator-assisted2D-to-3Dconversion.

8288-14, Session 4

3D cinema to 3DTV content adaptationD.Doyen,L.Yasakethu,L.Blondé,Q.Huynh-Thu,TechnicolorS.A.(France)

3Dcinemaand3DTVhavegrowninpopularityinrecentyears.Filmmakershaveasignificantopportunitybeforethemgiventherecentsuccessof3Dfilms.Inthispaperweinvestigatewhetherthisopportunitycouldbeextendedtothehomeinameaningfulway.

“3D”perceivedfromviewingstereoscopiccontentdependsontheviewinggeometry.Thisimpliesthatthestereoscopioc-3Dcontentshouldbecapturedforaspecificviewinggeometryinordertoprovideasatisfactory3Dexperience.However,althoughitwouldbepossible,itisclearlynotviable,toproduceandtransmitmultiplestreamsofthesamecontentfordifferentscreensizes.Inthisstudytosolvetheaboveproblem,weanalyzetheperformanceofsixdifferentdisparitybasedtransformationtechniques,whichcouldbeusedforcinema-to-3DTVcontentconversion.Subjectivetestsareperformedtoevaluatetheeffectivenessofthealgorithmsintermsofdepthquality,visualcomfortandartefactssuchascrosstalk,cardboardeffectandpuppettheatreeffect.Theresultant3DTVexperienceisalsocomparedtothatofcinema.Weshowthatbyselectingthepropertransformationtechnique,basedoncontenttype,itispossibletoenhancethe3DTV

Conference 8288


experiencefromacommoninputsignalwhichwasoriginallycapturedforcinema.

8288-83, Session 4

Video retargeting for stereoscopic content under 3D viewing constraintsC.Chamaret,TechnicolorS.A.(France)

TheimminentdeploymentofnewdevicessuchasTV,tablets,smartphonessupportingstereoscopicdisplaycreatesaneedforretargetingthecontent.Newdevicesbringtheirownaspectratioandpotentialsmallscreensize.Aspectratioconversionbecomesmandatoryandanautomaticsolutionwillbeofhighvalueespeciallyifitmaximizesthevisualcomfort.

Someissuesinherentto3Ddomainareconsidered:noverticaldisparity,noobjecthavingnegativedisparity(outwardperception)ontheborderofthecroppingwindow.

Anefficientvisualattentionmodelisappliedoneachviewandprovidessaliencymapswithmostattractivepixels.Dedicated3Dretargetingcorrelatesthe2Dattentionmapsforeachviewaswellasadditionalcomputedinformationtoensurethebestcroppingwindow.Specificconstraintsinducedby3Dexperiencearefixedbycomputingamappresentingobjectsthatshouldnotbecropped.

Thecomparisonwithoriginalcontentof2:35ratiohavingblackstripesprovidelimited3DexperienceonTVscreen,whiletheautomaticcroppingandexploitationoffullscreenshowmoreimmersiveexperience.

Theproposedsystemisfullyautomatic,ensuresagoodfinalqualitywithoutmissingfundamentalpartsfortheglobalunderstandingofthescene.Inaddition,itimprovestheimmersiveexperience.

8288-16, Session 5

Stereoscopic-3D display design: a new paradigm with Intel adaptive stable image technology [IA-SIT]S.K.Jain,IntelCorp.(UnitedStates)

Stereoscopic-3D(S3D)proliferationonpersonalcomputersislimitedbyseveralchallenges:1)lowereffectiveresolution,luminanceandcolorgamutofcurrentS3Ddisplayscomparedto2D;2)viewingdiscomfortduetocross-talkamongststereoimages;3)highsystemcost;and4)restrictedcontentavailability.IntelAdaptiveStableImageTechnology(IA-SIT)isasystemarchitectureapproachdevelopedtoaddresssuchS3Dsystemdesignchallenges.Ourgoalistodeliverhighquality3DvisualizationatPCpricepoints.Optimizationsindisplaydriver,timingfirmware,backlighthardware,eyewearopticalstackandsynchmechanismcantogetherhelpaccomplishthisgoal.Agnostictorefreshrate,IA-SITnaturallyscaleswithshrinkingofdisplaytransistorsandimprovementsinliquidcrystalandLEDmaterials.Wepropose:1)newIA-SITdisplaytimingsin3Dmode(throughVESA);2)IA-SITopticalstackfor3DEyewearthatistrulyuniversalacrossactive/passiveandPC/TVdevices(throughCEA);and3)IA-SITrealtimecontrol-synchronizationprofile(throughBlueToothSig).

8288-17, Session 5

A real-time misalignment correction algorithm for stereoscopic 3D camerasI.E.Pekkucuksen,B.Zhang,A.U.Batur,TexasInstrumentsInc.(UnitedStates)

Cameracalibrationina3Dstereosetupisanimportantproblemsincethemisalignmentbetweenthetwoviewscansignificantlydegradethe3Dquality.However,hardwarecalibrationisnotalwaysanoptionespeciallyformassproducedstereocameras.Evenifone-timecalibrationisperformed,itcannotbemaintainedindefinitelybecause

ofenvironmentalfactors.Thatiswhysoftwarecalibrationoffersamoreflexibleoptiontoaddresstheproblem.Inthispaper,weproposeastereomisalignmentcorrectionalgorithmthatdoesnotrequiretheuseofspecialcharts.Thealgorithmfirstestimatesthedisparityinhorizontalandverticaldirectionsbetweencorrespondingblocksfrombothimages.Then,thisinitialestimateisrefinedwithtwodimensionalsearchusingsmallersub-blocks.Thedisplacementdataandblockcoordinatesarefedtoamodifiedaffinetransformationmodelandoutliersarediscardedtokeepthemodelingerrorlow.Finally,theestimatedaffineparametersareportionedbetweenthetwoviewstocompensateforthemisalignment.Theproposedalgorithmcansignificantlyreducethemisalignmentbetweenstereoframesandenableabetter3Dexperienceinrealtime.

8288-18, Session 5

Full-color stereoscopy with little flicker at low-refresh rate by time-division multiplexing anaglyphH.Kakeya,H.Kodaira,Univ.ofTsukuba(Japan)

Inthepresentpaperweproposeatime-divisionmultiplexinganaglyphmethodtorealizefullcolorstereoscopywithlittleflickeratlowrefreshrateofaround60Hz,whichiscompatiblewiththeconventional2Ddisplays.Toletmorepeopleexperience3Dcontentsathome,itispreferablethatpeoplecanwatch3Dcontentswiththeconventional2DTVsets.Becauseofthelowrefreshrate,however,applyingtime-divisionmultiplexingmethodusingshutterglassestoconventionaldisplaysresultsinsevereflicker,whichisfarfromacceptabletoalmostallconsumers.Toovercomethisproblem,weproposeatime-divisionmultiplexinganaglyphmethod,wheretheredandbluecomponentsofrightimageisshowntotherighteyeandthegreencomponentofleftimageisshowntothelefteyeatoddframes,whilethegreencomponentofrightimageisshowntotherighteyeandtheredandbluecomponentsofleftimageisshowntothelefteyeatevenframes.Wecarryoutanexperimenttoletthesubjectsseetime-divisionmultiplexinganaglyphimagesandtheresultshowsthatflickercanbereducedtoanacceptablelevelbytheproposedmethod.

8288-19, Session 5

Supervised disparity estimationP.Vandewalle,C.Varekamp,PhilipsResearchNederlandB.V.(Netherlands)

Automateddisparityestimationfromstereoinputvideohasbeenwidelyinvestigated.Whiletheresultsusingsuchmethodshavestronglyimprovedinrecentyears,theyaretypicallynotsufficientlyaccurateyetforgeneralusage.Themainremainingproblemsoccurinocclusionregions,homogeneousareas,anddepthtransitionsbetweenobjectswithlowcolorcontrast.Userinteractioncanbeappliedtoovercometheseproblemsandachieveanydesiredqualitylevel.

Weintroduceasuperviseddisparityestimationmethodinwhichanoperatorcansteerthedisparityestimationprocess.Insteadofcorrectingerrors,weviewtheestimationprocessasaconstrainedprocesswheretheconstraintsareindicatedbytheuserintheformofcontrolpoints,scribblesandcontours.Controlpointsareusedtoobtainaccuratedisparityestimatesthatcanbefullycontrolledbytheoperator.Scribblesareusedtoforceregionstohaveasmoothdisparity,whilecontourscreateadisparitydiscontinuityinplaceswhereautomaticmethodsfail.Controlpoints,scribblesandcontoursarepropagatedthroughthevideosequenceusingmotioncompensationtocreatetemporallystableresults.

8288-20, Session 5

Analysis of brain activity and response during monoscopic and stereoscopic visualization

Conference 8288


E.Calore,R.Folgieri,D.Gadia,D.Marini,Univ.degliStudidiMilano(Italy)

StereoscopicvisualizationincinematographyandVRcreatesanillusionofdepthbymeansoftwobidimensionalimagescorrespondingtodifferentviewsofascene.Thisperceptual“trick”isusedtoenhancetheemotionalresponseandthesenseofpresenceandimmersivityoftheobservers.

Aninterestingquestionisifandhowispossibletomeasureandanalyzethelevelofemotionalinvolvmentoftheobserversduringastereoscopicvisualizationofamovieorofavirtualenvironment.

Thefinalgoalofthisresearchisachallenge,duetothelargenumberofsensorial,physiologicalandcognitivestimuliinvolved.

Inthispaperwebeginthisresearchbyanalyzingeventualdifferencesinthebrainactivityofsubjectsduringtheobservationofmonoscopicorstereoscopiccontents.Tothisaim,wehaveperformedsomeexperimentscollectingEEGdatausingaBrain-ComputerInterfacedevicefromtwogroupsofusers,duringtheobservationofstereoscopicandmonoscopicshortmoviesinsidetheVirtualTheateroftheUniversityofMilan.

Fromtheanalysisofthecollecteddata,itseemsthatinterestingdifferencesarepresentintheaveragebrainactivityamongtheobserversinthetwogroups,withasignificativeeffectofstereoscopicvisualization.

8288-21, Session 6

Motion in-depth constancy in stereoscopic displaysS.Laldin,L.Wilcox,C.Hylton,R.S.Allison,YorkUniv.(Canada)

Inastereoscopic3Dscene,non-linearmappingbetweenrealspaceanddisparitycanproducedistortionsofspacewhencamerageometrydiffersfromnaturalstereoscopicgeometry.Ifanobjecttraversesadistortedstereoscopicspaceatconstantvelocity,onemightanticipatedistortionoftheperceivedtrajectory.Todetermineifthepredicteddistortionsareinfactperceived,weassessedobservers’perceptsofaccelerationanddecelerationusingananimationofaballmovingindepththroughasimulatedenvironment,viewedstereoscopically.

ThemethodoflimitswasusedtomeasuretransitionpointsbetweenperceivedaccelerationanddecelerationasafunctionofIAandcontext(texturedvs.non-texturedbackground).

Basedonbinoculargeometry,wepredictedthatthetransitionpointswouldshifttowarddecelerationforsmallIAandtowardsaccelerationforlargeIA.Thiseffectshouldbemodulatedbymonoculardepthcues.However,wefoundthattheaveragetransitionvalueswerenotinfluencedbyIAorthesimulatedenvironment.Thesedatasuggestthatobserversareabletodiscountdistortionsofstereoscopicspaceininterpretingthetrajectoryofobjectsmovingthroughsimpleenvironments.

8288-22, Session 6

Quality evaluation of depth image-based rendering methods for stereoscopic video contentM.Eisenbarth,TechnischeUniv.Wien(Austria)andemotion3D(Austria);F.H.Seitner,emotion3D(Austria)andTechnischeUniv.Wien(Austria);M.Gelautz,TechnischeUniv.Wien(Austria)

Depthimage-basedrendering(DIBR)allowsthegenerationofnewcameraviewpointsfromstereoscopicfootage.Thisenablesawealthofnewapplicationsfor3Dfilmpost-productionsuchasdepthcorrectionandgrading,contentremasteringandmulti-viewgenerationforauto-stereoscopicdisplays.

Wecomparedstate-of-the-artDIBRtechniquesandtheirwarpingandin-fillingmethods.Everymethodhasitsownadvantageswhenappliedtospecificcontent,butgeneratesindividualartifactsifcontentchangesovertime.WemadeobjectiveandsubjectivecomparisonsofDIBR

generatedstereoscopiccontenton3Ddisplays.Wedesignedatestimagetomeasurethedepthboundariesof3Ddisplays.MeasurementofdepthimpressioninvideoinputandofdisplayedoutputallowstheDIBRchallengestobequantized.Basedonthismethod,anevaluationofDIBRtechniqueswithdifferentwarpingandin-fillingtechniqueswascarriedout.

OurresultsdemonstratebothaproperobjectiveandsubjectivecomparisonofDIBRalgorithmsaccordingtothevisualperceptionofwarpingandin-fillingartifacts.Thestudyalsoshowsthecriticalfactorsingeneratingvirtualcameraviewpointsorperformingstereo-baseandconvergencecorrectionsgivenastereovideo.

8288-23, Session 6

Disparity profiles in 3DV applications: overcoming the issue of heterogeneous viewing conditions in stereoscopic deliveryG.Boisson,C.Chamaret,TechnicolorS.A.(France)

Moreandmorenumerous3Dmoviesarereleasedeachyear.Thankstothecurrentspreadof3D-TVdisplays,these3DVideo(3DV)contentsareabouttoentermassivelythehomes.Yetviewingconditionsdeterminethestereoscopicfeaturesachievablefor3DVmaterial.Becausetheconditionsathome-screensizeanddistancetoscreen-differsignificantlyfromatheater,3DCinemamoviesneedtoberepurposedbeforebroadcastandreplicationon3DBlu-rayDiscsforbeingfullyenjoyedathome.Inthatpaperwetacklethatparticularissueofhowtohandlethevarietyofviewingconditionsinstereoscopiccontentsdelivery.Tothatextendwefirstinvestigatewhatisbasicallyatstakeforgrantingstereoscopicviewers’comfort,throughthewell-known-andsometimesdispraised-vergence-accommodationconflict.Therebywedefineasetofbasicrulesthatcanserveasguidelinesfor3DVcreation.Weproposedisparityprofilesasnewrequirementsfor3DVproductionandrepurposing.Meetingproposedbackgroundandforegroundconstraintspreventsfromvisualfatigue,andoccupyingthewholedepthbudgetavailablegrantsoptimal3Deffects.Wepresentanefficientalgorithmforautomaticdisparity-based3DVretargetingdependingontheviewingconditions.Variantsareproposeddependingontheinputformat(stereoscopicbinocularcontentordepth-basedformat)andthelevelofcomplexityachievable.

8288-24, Session 6

Investigation of object thickness for visual discomfort prediction

in stereoscopic imagesH.Sohn,Y.J.Jung,S.Lee,H.W.Park,Y.M.Ro,KAIST(Korea,Republicof)

Withtherisingpopularityofstereoscopic3Dcontentservices,concernsonthe3Dviewingsafetyissueshavebeenraised.Toaddresstheviewingsafetyissues,theautomaticpredictionofvisualdiscomfortforstereoscopic3Dcontentshasincreasinglygaineditsimportance.Ithasbeenwellknownthatstereoscopic3Ddisplaysinducetheconflictbetweenaccommodationandvergence[1,2].Specifically,excessiveamountofbinoculardisparity(i.e.,disparitymagnitude)leadstohighdegreesoftheconflictbetweenaccommodationandvergence,whichmaycausevisualdiscomfortandfatigue[1,2].Inaddition,itwasreportedthatasdisparitymagnitudeofvisualstimulusincreased,morevisualdiscomfortwasinduced[1,3].Asaresult,disparitymagnitudehasbeenregardedasacentraldeterminantforvisualdiscomfortpredictionofstereoscopic3Dimages[1-4].

Instereoscopicviewing,humanvisualsystemhasalimitedabilitytofusebinoculardisparity,i.e.,binocularfusionlimit.Inliterature,ithasbeenknownthatthebinocularfusionlimitisaffectedbynotonlydisparitymagnitudebutalsoahorizontalsizeofvisualstimulus[5].Inparticular,giventhedisparitymagnitude,decreasesinstimulussizealsodecreasebinocularfusionlimit.Inviewingstereoscopic3Ddisplays,itwasreportedthatthecrosstalkofthinobjectswasmorevisibleandaffecteddepthperceptionthanthatofthickerobjects[6].As

Conference 8288


aresult,thesecluesindicatethatvisualcomfortmaybeaffectedbythewidthofobjectsinnaturalstereoscopic3Dscenecontents.

Thepurposeofthispaperistoinvestigatetheinfluenceofobjectthicknessonvisualdiscomfortpredictionforstereoscopicscenes.Inparticular,wedemonstratethatthecombineduseofdisparitymagnitudeandobjectthicknesssubstantiallyimprovestheperformanceinvisualdiscomfortprediction.

8288-25, Session 7

History of autostereoscopic cinemaW.Funk,Hologlyphics(UnitedStates)

Thispapercoversthehistoryofautostereoscopiccinema,fromthebeginningsofautostereoscopyinthe1800’s,thedevelopmentofmotioncapabilityandit’ssubsequentevolutiontopresenttechniques.Publicviewingsofautostereoscopicmovieshaveoccurredonasemi-ongoingbasissincetheearly1940’s.

AtheaterinMoscow,calledtheStereokino,wasconstructedfor

showingautostereoscopicfilms,withspeciallypositionedseatingforproperviewing.In1941,thefirstautostereoscopicmoviewasshownthere,KontsertdirectedbyAleksandrAndreyevskiy.ThefirstautostereoscopicfeaturefilmwasRobinsonKruzo,releasedin1947.AdditionalstereokinoswereopenedacrosstheSovietUnion,withfilmsproducedandshownuntilthelate1950s.

TheCyclostéréoscopewasanautostereoscopiccinemasysteminventedbyFrançoisSavoyeinFrance.Itwasbasedaroundadrummadeofmetalbarsthatrevolvearoundascreen.Forseveralyearsinthe1940’sand1950’s,itwasopentothepublicinParis.Anyfilmmadeinadualfilmformatcouldbeshown.

BesidesdedicatedtheatersinRussiaandFrance,exhibitsofcontenthaveoccurredoutsidedevotedtheaters.Thepaperfocusesonthehistoryofautostereoscopictechnologydevelopedforentertainment,publicviewingofcontent,theindividualsinvolvedandthecontentitself.

8288-26, Session 7

Full optical characterization of auto-stereoscopic 3D displays using local viewing angle and imaging measurementsP.M.Boher,T.Leroux,T.Bignon,ELDIM(France)

Differentcommercialauto-stereoscopic3DdisplaysarecharacterizedusingFourieropticsviewinganglesystemandimagingvideo-luminance-meter.Theemissivepropertiesofthedisplayarefirstmeasuredlocallyatthreepositions(center,rightandleft).Qualifiedmonocularandbinocularviewingspacesatdeducedforanobserverinfrontofthedisplayandthebestworkingdistanceisdeduced.Thentheimagingsystemispositionedatthisworkingdistanceandlaterallyshiftedtotakeaseriesofluminancemeasurements.Crosstalkhomogeneityontheentiresurfaceofthedisplayisdeducedforalltheobserverlocationsalongthehorizontal.ResultsarecorrelatedtothoseobtainedlocallyusinghighangularresolutionFourieropticsviewingangleinstrument.Weshowthatthecrosstalkisgenerallyoptimizedonmostofthesurfaceofthedisplayexceptatthecornersdependingonthedisplaytechnologyandthealignmentoftheparallaxbarriersorcylindricallenses.Localimperfectionslikescratchesandmarkgenerallyincreasedrasticallythecrosstalkanddemonstratingthecleanlinessrequirementsofthistypeofdisplay.

8288-27, Session 7

Analysis on monocular accommodation in horizontal-parallax-only super-multiview displayJ.Hong,Y.Kim,SeoulNationalUniv.(Korea,Republicof);J.Park,ChungbukNationalUniv.(Korea,Republicof);B.Lee,Seoul

NationalUniv.(Korea,Republicof)

Accommodationresponsetothehorizontal-onlysuper-multiviewdisplaywasmeasuredtoassessifthesuper-multiviewconditionholdsforthehorizontal-onlymultiviewdisplay.Experimentalresultsshowthatthemonoculardepthcuedoesnotperfectlymatchtothevirtualimageofhorizontal-onlysuper-multiviewdisplay,butthedegreeofaccommodation-convergencemismatchcanbereducedbecauseaccommodationresponsetendstoapproachthevirtualimage.Weareplanningtoquantifythedegreeofreducingtheaccommodation-convergencemismatchbythehorizontal-onlysuper-multiviewdisplayandthedatawillbehelpfulincommercializingthemultiviewdisplaysystem.

8288-28, Session 7

A novel time-multiplexed, autostereoscopic, multi-view, full-resolution 3D displayJ.Liou,IndustrialTechnologyResearchInstitute(Taiwan)

Weproposeandexperimentallydemonstrateanoveltime-multiplexedautostereoscopicmulti-viewfullresolution3DdisplaybasedonthelenticularlensarrayinassociationwiththecontroloftheactivedynamicLEDbacklight.Thelenticularlensesofthelensarrayopticalsystemreceivethelightanddeflectthelightintoeachviewingzoneinatimesequence.Thecrosstalkunderdifferentobservationscanninganglesisshowed,includingthecasesof4-viewsfieldscanning.Thecrosstalkofanyviewzonesisabout5%respectively;theresultsarebetterthanother3Dtype.

Thispaperaddressesthespecifictechnologicalchallengesofautostereoscopic3Ddisplaysandpresentsanovelsystemthatintegratesatime-multiplexautostereoscopicdisplaybasedonactivedirectionalbacklight(activedynamicbacklight)withanautostereoscopicdisplay.OursuccessfullydesignedprototypeutilizedaFPGAsystemtosynchronizebetweenadisplaypanelandbacklightslitpanel.

Multipleviewerscanviewthe3Dimagedisplaywithdifferentcontentsatdifferentdisplayregionsoftherounddisplaypanel.Takingfourviewingzonesasanexampletobecreated,eachviewingzoneoccupies1/240secondfordisplayone2Dimage.Oneeyereceivesthefirstimagein1/240secondatoneviewingzoneandanothereyereceivesanotherimageinnext1/240secondattheadjacentviewingzone.

8288-29, Session 8

Crosstalk reduces the amount of depth seen in 3D images of natural scenesI.Tsirlin,R.S.Allison,L.M.Wilcox,YorkUniv.(Canada)

CrosstalkremainsanimportantdeterminantofS3Dimagequality.Definedastheleakageofoneeye’simageintotheimageoftheothereyeitaffectsallcommerciallyavailablestereoscopicviewingsystems.PreviouslywehaveshownthatcrosstalkaffectsperceiveddepthmagnitudeinS3Ddisplays.Wefoundthatperceiveddepthbetweentwolinesseparatedindepthdecreasedascrosstalkincreased.Theexperimentsdescribedhereextendourpreviousworktocompleximagesofnaturalscenes.Wecontrolledcrosstalklevelsbysimulatingtheminimagespresentedonazero-crosstalkmirrorstereoscopedisplay.Theobserverswereaskedtoestimatetheamountofstereoscopicdepthbetweenpairsofobjectsinstereo-photographsofclatteredrooms.Datashowthatascrosstalkincreasedperceiveddepthdecreased;aneffectthatoccurredatalldisparities.Interestingly,observersoverestimatedthedepthindisplaysthatcontainednocrosstalk.Acontrolexperimentshowsthatitislikelyduetothepresenceofpictorialcuestodepth.Takentogethertheseresultsdemonstratethatourpreviousfindingsgeneralizetonaturalscenesshowingthatcrosstalkaffectsperceiveddepthmagnitudeeveninthepresenceofpictorialdepthcues.

Conference 8288


8288-30, Session 8

Method and simulation to study 3D crosstalk perceptionD.A.Khaustova,L.Blondé,Q.Huynh-Thu,C.Vienne,D.Doyen,TechnicolorS.A.(France)

3Dimage/videoqualitydependsontheamountofcrosstalkexhibitedbydisplays.Agoodunderstandingofthecrosstalkphenomenonmayleadtoimproved3Dtechnologies.Withthisperspective,ourworktookintoaccountdisplaytechnologies,contentcharacteristicsandhumanperception.Ontesteddisplays,systemcrosstalkwasdescribedasa2Dsurface(XT-surface)bymeasuringluminancevariationscomparedtoano-crosstalkreference.TheXT-surfacerepresentslightintensitiesforintended(deliberatesignal)andunintended(leakingsignalfromtheotherview)pixelvalues.Thisdatasurfacewasusedforthecreationofacontrolledsimulationofcrosstalk.Theset-upofatestbenchusedhigh-contrastdisplaysandmirrorstorealizeazerocrosstalkstereoscopicsystem.BydividingtheXT-surfaceintoareas,itwaspossibletocontrolthesimulationandexaminetheinfluenceofdifferenttypesandlevelsofcrosstalkonhumanperception.Crosstalkperceptionwasstudiedthroughaseriesofpsychovisualtests.Resultsdemonstratethatnotonlywhite-to-blackcrosstalkhascrucialinfluenceonimagequality.BymodifyingparametersoftheXT-surfaceitwaspossibletofindrelationsbetweencrosstalkamplitudeandhumanperception,aswellastoseehowmodificationsinfluencejudgmentsaboutimagequality.

8288-31, Session 8

Reproducibility of crosstalk measurements on active glasses 3D LCD displays based on temporal characterizationS.Tourancheau,MidSwedenUniv.(Sweden);K.Wang,AcreoAB(Sweden)andMidSwedenUniv.(Sweden);L.Janowski,J.Bulat,AGHUniv.ofScienceandTechnology(Poland);K.Brunnström,AcreoAB(Sweden);M.Barkowsky,Polytech’Nantes(France)

Crosstalkisoneofthemaindisplay-relatedperceptualfactorsdegradingimagequalityandcausingvisualdiscomforton3D-displays.Itcausesvisualartifactssuchasghostingeffects,blurring,andlackofcolorfidelitywhichareconsiderablyannoyingandcanleadtodifficultiestofusestereoscopicimages.OnstereoscopicLCDwithshutter-glasses,crosstalkismainlyduetodynamictemporalaspects:imprecisetargetluminance(highlydependentonthecombinationofleft-viewandright-viewpixelcolorvaluesindisparityregions)andsynchronizationissuesbetweenshutter-glassesandLCD.Thesedifferentfactorsinfluencelargelythereproducibilityofcrosstalkmeasurementsacrosslaboratoriesandneedtobeevaluatedinseveraldifferentlocationsinvolvingsimilaranddifferingconditions.

Inthispaperweproposeafastandreproduciblemeasurementprocedureforcrosstalkbasedonhigh-frequencytemporalmeasurementsofbothdisplayandshutterresponses.Itpermitstofullycharacterizecrosstalkforanyright/leftcolorcombinationandatanyspatialpositiononthescreen.Suchareliableobjectivecrosstalkmeasurementmethodatseveralspatialpositionsisconsideredamandatoryprerequisiteforevaluatingtheperceptualinfluenceofcrosstalkinfurthersubjectivestudies.

8288-32, Session 8

Crosstalk and brightness in projection-based, multi-view systemsR.Blach,A.Pross,Fraunhofer-InstitutfürArbeitswirtschaftundOrganisation(Germany);A.Kulik,BauhausUniv.Weimar(Germany);O.Stefani,Fraunhofer-InstitutfürArbeitswirtschaftundOrganisation(Germany)

Immersiveworkingenvironmentsgenerallyusesingleuserstereoscopicprojections.Wehaveextendedthesesystemstomultiviewsystems

fromtwotosixuserswithindividualperspectivelycorrectstereoscopicviewsforeachuser.Toseparatetheimageswehaveusedahybridapproachwhichcombinesactiveandpassivefiltering.

Inprojectionbasedviewsystemsbrightnessandcrosstalkareoneofthemajorissueswhichhavetobehandledasthenoiseincreasesandthesignalbrightnessdecreaseswiththeincreaseofviews.Tobeabletoevaluatedifferenttechnicalapproachesforoptimizationandalsocomparethesesystemswithexistingsingleuser3Dsystems,wehaveextendedthecrosstalkmeasurementapproachbyWeissmanandWoodstoourspecificmultiviewcasewhichis:a)projectionbased,b)eyeseparationwithopticalfiltersandc)userseparationwithactiveshutterelements.

Themaindifferencecomparedtothesingleuserstereoapproachisthatcrosstalkornoiseisnotonlyevokedbyasingleunintendedimagebutbyseveral.

Ourcontributionisasimpleapproachfordefiningandmeasuringcrosstalkforprojectionbasedmultiviewsystems.Asthesesystemsarestillrare,wearenotawarethatthishasbeenexplicitlyshownelsewhere.

8288-33, Session 8

How much crosstalk can be allowed in a stereoscopic system at various grey levels?S.A.Shestak,D.Kim,SamsungElectronicsCo.,Ltd.(Korea,Republicof)

Wederivedanexpressionforphysiologicallimitofcrosstalkperception,basedonBarten’smodelofcontrastsensitivity.Thephysiologicallimitvarieswiththedisplayedluminanceandwithmaximumdisplay’sluminance.Thecalculatedthresholdvaluesareclosetothepreviouslypublishedlevel0.5-1%justwithinthecertainrangeofluminance.Atlowluminancestheperceptualthresholdcanbenoticeablylower.WeproposetospecifytheallowablecrosstalkXaasanumber(N)ofphysiologicallimitsXa=NXp,whereN=2..3.

8288-19, Session 9A

Apparent stereo: the Cornsweet illusion can enhance perceived depthP.Didyk,Max-Planck-InstitutfürInformatik(Germany);T.Ritschel,E.Eisemann,TelecomParisTech(France);K.Myszkowski,H.Seidel,Max-Planck-InstitutfürInformatik(Germany)

Itisbothatechnicalandanartisticchallengetodepictthree-dimensionalspaceusingaflattwo-dimensionalmedium.Fittingwithinthelimitsofagivendisplaytechnologyandatthesametimeachievingcomfortableviewingcanrequiremodificationstostereocontent,e.g.,toflattenorevenremovebinoculardisparity.However,insuchscenarioscrucialdepthdetailsmightgetlost.Recently,abackward-compatiblestereotechniquewasproposed,whichbuildsupontheCraik-O’Brien-Cornsweeteffect-avisualillusion,whichwasappliedtobrightnessinCGbefore.TheapproachreplacesdisparitybyCornsweetprofilesindisparitywheredepthdiscontinuitiesoccur.Thisallowsforthereductionoftheoveralldisparityrangewhilestillconveyingacomfortablestereoexperience.ThepresentworkextendsthepreviousideabyshowingthatCornsweetprofilescanalsobeusedtoenhancethe3Dimpressioninregionswherethedisparityrangewascompressed.Auserstudymeasurestheperformanceofbackward-compatiblestereoaswellasourdisparityenhancement.

8288-20, Session 9A

Perceived depth of multi parallel, overlapping, transparent, stereoscopic surfacesS.Aida,K.Shimono,TokyoUniv.ofMarineScienceandTechnology(Japan);W.J.Tam,CommunicationsResearchCtr.

Conference 8288


Canada(Canada)

Thegeometricrelationalexpressionofhorizontaldisparity,viewingdistance,anddepthmagnitudebetweenobjectsinstereopsissuggeststhat,foragivenviewingdistance,themagnitudeofperceiveddepthofobjectswouldbethesameaslongasthedisparitymagnitudesarethesame.However,wefoundthatthisisnotnecessarilythecaseforrandomdotstereogramsdepictingtwoormoreparallel-transparent-overlapping-stereoscopic-surfaces(POTS).Datafromtwoexperimentsindicatedthat,whenthestimulussizeisrelativelylarge(e.g.,13x20arcdeg),themagnitudeofreproduceddepthbetweentwostereo-surfacesofatwo-POTSconfigurationcanbelargerthanthatbetweenanidenticalpairofstereo-surfacesbutwithanadditionalstereo-surfacelocatedbetweenthepair.Infollow-upexperiments,dotdensity,thesizeoftherandomdotstereogramsandthenumberofstereo-surfaceslyingbetweenthetwooutermoststereo-surfacesweremanipulated.Overall,theresultsindicateadifferenceofapproximately10%inthereproduceddepth,dependingonwhetherthereisanadditionalstereo-surfacebetweenthetwosurfacesofatwo-POTSconfigurationornot.Theresultsarediscussedintermsofglobalstereopsiswhich“operates”forrelativelylargestimulussizes(suchas13x20arcdeg).

8288-21, Session 9A

Diagnosing perceptual distortion present in group stereoscope viewingM.M.Burton,B.B.Pollock,J.W.Kelly,S.B.Gilbert,E.Winer,IowaStateUniv.(UnitedStates);J.delaCruz,U.S.ArmyRDECOM/STTC(UnitedStates)

Stereoscopicvirtualenvironmentsarefrequentlyemployedtoimmerseviewersinvirtualrealityvisualizations,simulations,andtrainingsystems.Stereosystemsrenderthescenegraphfromtheperspectiveofasinglepersonorstaticviewpointwhichcanbedefinedasaleader,whileothersseeinaccurateanddistortedviewsfromafollowerposition.Greaterunderstandingofdistortionfromthefollowerperspectivecanleadtomoreeffectivemulti-viewsolutionsandconfigurationsofleader-followersystems.

AnexperimentatIowaStateUniversitywasconductedtogaininsightintotheperceptualdistortionpresentinleader-followersystemswheremultipleusersviewthesamestereoscopic,projection-basedvirtualenvironment.Theexperimentwascomprisedoftasksreportingquantitiesofanglesindegreesfrompositionsparalleltothescreenandreportingthedepthdimensionofarectangleatpositionscenteredandperpendiculartothescreen.Resultsfromthisstudyindicatedarelationshipofangulardistortionwithleader-followeroffsetsparalleltothescreenanddistancetothescreenaffectsthequantityofdepthdistortioninadditiontoperpendicularoffsetsfromtherenderedviewpoint.Furtherworkwillattempttounderstandtheeffectsofthisdistortionongroupcollaborationandfurtherdeveloprelationshipsofdistortionandleader-followeroffsets.

8288-34, Session 9A

Visual discomfort and the timing of vergence-accommodation conflictsJ.Kim,D.Kane,M.S.Banks,Univ.ofCalifornia,Berkeley(UnitedStates)

Stereo3D(S3D)displayscausevisualdiscomfortinmanyviewers,soitisveryimportanttounderstandthecausesofthediscomfortsoonecanminimizeit.Weknowthatthevergence-accommodationconflictcausesvisualdiscomfortandeventuallyvisualfatigue.Vergenceandaccommodationbothhavephasicandtoniccomponentsthatadaptatdifferentrates.Todetermineifthedifferentadaptationratesinphasicandtoniccomponentsaffectvisualdiscomfort,wevariedthetemporalpropertiesofthevergence-accommodationconflictandassesseddiscomfort.

Observersviewedrandom-dotstereogramsinanovelvolumetricdisplaythatallowsustoindependentlymanipulatethestimulustovergenceandthestimulustoaccommodation.Thereweretwomain

conditions:naturalviewing(cues-consistent)andS3Dviewing(cues-inconsistent).Thestimulitovergenceandaccommodationwerevariedatdifferentthreedifferentrates,presentedindifferentsessions.

WefoundthatobserversexperiencedworsesymptomsintheS3Dviewingthaninthenaturalviewing.Wealsofoundthatmorefrequentchangesinthestimulustovergenceyieldedworsesymptoms.Thus,thetime-averagemagnitudeofthevergence-accommodationconflictisnotthesolepredictoroftheamountofdiscomfort.Italsomattershowtheconflictisdistributedovertime.

8288-22, Session 9B

Measuring 3D discomfort from vertical and torsional disparities in natural imagesC.W.Tyler,L.T.Likova,TheSmith-KettlewellEyeResearchInstitute(UnitedStates);K.Atanassov,V.Ramachandra,S.Goma,Qualcomm(UnitedStates)

Thetwomajoraspectsofcameramisalignmentthatcausevisualdiscomfortwhenviewingimagesona3Ddisplayareverticalandtorsionaldisparities.Thegoalofthestudywastodeterminethediscomfortrangesforthekindsofnaturalimagethatpeoplearelikelytotakewith3Dcameras.Theresultsshowedthat,asaruleofthumb,verticaldisparitybetweenthetwoeyesneedstobeheldbelowabout15arcmin,oraquarterofadegree,tobewithinanacceptablerangeofdiscomfort,whilehumanvisionisrelativelyinsensitivetotorsionaldisparitiesbelowabout30deg,

8288-35, Session 9B

Visual fatigue versus eye-movementsC.Vienne,L.Blondé,D.Doyen,TechnicolorS.A.(France)

Observing3DcontentonacinemaorTVscreenpotentiallygeneratesfatigue.Inresearch,experienceofvisualsymptomsfollowingtheobservationofstereo-contentisusuallyassessedthankstoquestionnairesandsubjectivereports.Weattemptedtoexploretheoccurrenceofvisualfatigueusingmoreobjectivemethods,namelybyusingbinoculareye-trackingandpsychophysics.Amainobjectivewastostudytheemergenceofvisualfatigueinrelationwitheye-movementknowingthestimulationoftheoculomotorsystemanditsresponse.Wedesignedanexperimentinwhichparticipantswereaskedtoperformarepeatedvergenceefforttask,justfollowedbya3Dspaceperceptiontask.Participants’eyemovementswererecordedduringthewholesessionusinganeye-trackingsystem.Theanalysisrevealedthattheperceptionof3Dshapewasgraduallyaffectedbytheintensityofthevergenceefforttask.Theeffectonstereo-estimationwasactuallyduetotheoccurrenceofvisualfatigue,3Dobjectsareperceivedflatter.ResultsonthesubjectivereportsofSSQrevealedthatoculomotorfactorswerepredominantinthevisualsymptoms.Inaddition,someeffectsandcorrelationsonthemicro-saccadicratewereobtained.Itoffersaperspectivetocharacterizeobjectivelyvisualfatiguewhenwatching3Dcontentonstereoscopicdisplays.

8288-36, Session 9B

Visual comfort: stereoscopic objects moving in the horizontal and mid-sagittal planesW.J.Tam,F.Speranza,C.Vázquez,R.Renaud,CommunicationsResearchCtr.Canada(Canada);N.Hur,ElectronicsandTelecommunicationsResearchInstitute(Korea,Republicof)

Thepresentstudyexaminestheeffectofmotionofstereoscopicobjectsonvisualcomfortusingcomputer-generatedvideosequencesbasedonaparallel-camerasetting.Inthefirstexperiment,thevisualcomfortofasetofstereoscopicobjectsmovinginthez-planewithdifferentdisparitiesandvelocitiesisexamined.Threetypesofmovementinthez-planewereincluded:bothuncrossedandcrosseddisparities;crossedonly;anduncrossedonly.Inasecondexperiment,thevisualcomfortofasetofstereoscopicobjectsmovingatagiven

Conference 8288


depthinthehorizontalplaneisexamined.Alsointhiscase,threetypesofmovementinthez-planewereincluded:bothuncrossedandcrosseddisparities;crossedonly;anduncrossedonly.Inbothexperiments,themovements(alongthez-planeforExp.1andalongthex-planeforExp.2)involvedthreelevelsofvelocityandfivelevelsofdisparity.Preliminaryresultsappeartoconfirmthatvisualcomfortisinfluencedmorebyobjectvelocitythanbydisparitymagnitude(fortherangeofmagnitudesused);aninteractionofvelocityanddisparityisalsoapparent.Therefore,thepreliminaryresultsappeartoconfirmthatchangesindisparityovertimeareasignificantcontributortovisualcomfortofstereoscopicvideoimages;theeffectappearstobepresentalsowhenthechangesareconfinedwithinthegenerallyacceptedcomfortzone.

8288-37, Session 9B

Visual discomfort with stereo 3D displays when the head is not uprightD.Kane,R.Held,M.Banks,Univ.ofCalifornia,Berkeley(UnitedStates)

Stereo3D(S3D)displayscancausevisualdiscomfort.Inpart,discomfortisduetothewell-knownvergence-accommodationconflict.Additionally,discomfortmaybecausedbyviewingS3Ddisplayswiththeheadrotatedtotheside(rollrotation).Headrollconvertshorizontaldisparityonthescreenintoverticaldisparityontheretinathatinturnshouldelicitverticalvergenceeyemovementstominimizeverticaldisparity.Becausesuchmovementsaresmallinnaturalviewing,thelargeronesrequiredwithheadrollarelikelytocausediscomfort.WeinvestigatedthisbysimulatingheadrollinaS3Ddisplay.Wekepttheobservers’headupright,butrotatedthestimulus.Thestimuluswasarandom-dotstereogramdepictingsinusoidaldepthcorrugations.Itcontainedeithernovariationinoveralldisparityoravariationinoveralldisparityovertime.Observersindicatedtheamountofvisualdiscomfortexperiencedwhenviewingthesestimuli.Theyreportedworsesymptomsinthesimulatedhead-rollconditionsandwhenthedisparitypedestalchangedbyagreateramount.Weattributetheincreasedsymptomstotheverticalvergenceeyemovementselicitedbyheadroll.

8288-54, Poster Session

Correlation between a perspective distortion in a S3D content and the visual discomfort perceivedD.Doyen,L.Blondé,J.Sacré,TechnicolorS.A.(France)

PerspectivedistortionwilloccurinS3Dwhentherelativedisparitybetweenelementsgeneratesadepththatisnotinaccordancewiththerelativesizeofobjectspresented.Subjectivetestshavebeenconductedusingtestsequenceswhereshootingparametersareperfectlyknownandwherevergence/accommodationconflictisnotpredominant.Perspectivedistortionswilloccurinsomeofthecontentsregardingviewingconditions.Peoplewereaskedtoqualifysequencesintermofnaturalnessandvisualcomfort.Resultsoftestrevealedaclearcorrelationbetweenperspectiveconflictandvisualdiscomfortperceived.Whatevertheshootingcondition,parallelortoed-incameras,resultsaresimilar.Afactorbetweendepthandperspectivecanbecalculatedfromshootingconfigurationandviewingcondition.ItseemstoberelevanttoestimatethecomfortofS3Dcontentperception.Subjectivetestshavepermittedtobetterunderstandthelinkbetweenperspectiveconflictsandvisualcomfort.Aspecificfactorhasbeendefinedtakingintoaccountshootingparametersandviewingconditionstopredictcomfortordiscomfort.Next,studieswillbeconductedtoextendtheseteststocinemaconditionsweretherangeofviewingconditionsislarger.


Evaluation of quality of experience in interactive 3D visualization: methodology and resultsS.Tourancheau,M.Sjöström,R.Olsson,MidSwedenUniv.(Sweden);A.Persson,Ctr.forMedicalImageScienceandVisualization(Sweden);T.Ericson,SetredAB(Sweden)

Humanfactorsareofhighimportancein3Dvisualization,butsubjectiveevaluationof3Ddisplaysisnoteasybecauseofahighvariabilityamongusers.Thisstudyaimedtoevaluateandcomparetwodifferent3Dvisualizationsystems(amarketstereoscopicdisplay,andastate-of-the-artmulti-viewdisplay)intermsofqualityofexperience(QoE),inthecontextofinteractivevisualization.

Anadaptedmethodologyhasbeendesignedinordertofocuson3Ddifferencesandtoreducetheinfluenceofallotherfactors.46subjectstookpartintheexperiment.Theinfluenceoftheirpersonalbackground,priorexperience,andphysiologicalcharacteristicshasbeeninvestigated.Aftersolvingdifferenttasksonasynthetic3Dscene,theywereaskedtojudgethequalityoftheexperience,accordingtospecificfeatures.

Resultsshowedthatvisualcomfortwassignificantlybetteronthemulti-viewdisplay.ThisstudyhaspermittedtoidentifysomefactorsinfluencingQoE:externalfactorssuchaslengthoftheexperiment,aswellasuser-centeredfactorssuchaspriorexperience,andstereopsisthreshold.Thispaperprovidesananalysisofthesefactorsandsuggestionsfortestmethodologiesoffuturestudies.


Objective view synthesis quality assessmentP.Conze,TechnicolorS.A.(France)andInstitutNationaldesSciencesAppliquéesdeRennes(France);R.Philippe,TechnicolorS.A.(France);L.Morin,InstitutNationaldesSciencesAppliquéesdeRennes(France)

Viewsynthesisbringsgeometricdistortionswhicharenothandledefficientlybyexistingimagequalityassessmentmetrics.Despitethewidespreadof3-Dtechnologyandnotably3DTeleVision(3DTV)andFree-viewpointsTeleVision(FTV),thefieldofviewsynthesisqualityassessmenthasnotyetbeenwidelyinvestigatedandnewqualitymetricsarerequired.

Inthisstudy,weproposeanewfull-referenceobjectivequalityassessmentmetriccalledViewSynthesisQualityAssessment(VSQA)metric.Ourmethodisdedicatedtoartifactsdetectioninsynthesizedview-pointsandaimstohandleareaswheredisparityestimationmayfail:thinobjects,objectborders,transparency,variationsofilluminationorcolordifferencesbetweenleftandrightviews,periodicobjects...

Thekeyfeatureoftheproposedmethodistheuseofthreevisibilitymapswhichcharacterizecomplexityintermsoftextures,diversityofgradientorientationsandpresenceofhighcontrast.Moreover,theVSQAmetricmaybedefinedasanextensionofanyexisting2Dimagequalityassessmentmetric.Experimentaltestshaveshownthatourmethodallowsarelevantdetectionofviewsynthesisartifactsinawiderangeofsituations.


Depth enhancement of S3D content and the psychological effectsM.Hirahara,S.Shiraishi,T.Kawai,WasedaUniv.(Japan)

Inrecent,stereoscopic3D(S3D)imagingtechnologiesarewidelyusedtocreatecontentformovies,TVprograms,gamesetc.AlthoughdifferencebetweenS3Dand2Dcontentisdepthsensationinducedbybinocularparallax,therelationbetweendepthcontrolanduserexperienceisstillunclear.

Purposeofthisstudyistoexaminetheeffectivenessofdepthcontrol,

Conference 8288


suchasexpansionorreduction,shiftingforwardorbackwardoftherangeofmaximumparallacticanglesbetweencrossanduncrossdirection(depthbracket),byconductingasubjectiveevaluationonuserexperience.Fromtheresultsoftheevaluation,followingtwopointsweresuggested.

(1)Expansion/reductionofdepthbracketaffectedpreferencetotheS3Dcontent

(2)Theeffectsofexpansion/reductionofdepthbracketweremorenotablethanshiftingtocross/uncrossdirection.


Perception of size and shape in stereoscopic 3D imageryM.D.Smith,MDSConsulting(UnitedStates);B.T.Collar,WarnerBros.EntertainmentInc.(UnitedStates)

3Dcontentcreationintheentertainmentindustryhasalwaysbeensomewhatofa“blackart.”Stereographersutilizeamixofcreativeandtechnicalmethodstoproduce3Dcontentthatmeetsthecreativeintentofthefilmmaker.Howeverwhilethenaturallawsofphysicsgoverninghow3Dimageryisacquired,transformed,andultimatelyperceivedbyavieweriswellknown,inourexperiencefewstereographersmakeuseofitinthecourseofday-to-day3Dfilmmaking.Stereographersturntheirknobsinaniterativefashionuntilthedesiredresultisachieved-greatquality3Dfilms.Thispaperpositsthatifthemathematicalrelationshipsgoverninghow3Dworksweremoreoftenutilizedtheresultingqualitywouldbethesame,butwithlessiteration,i.e.samequalitywithlesstimeandcost.


Quality of experience model for 3DTVW.Chen,FranceTelecomR&D(France)andPolytech’Nantes(France);J.Fournier,FranceTelecomR&D(France);M.Barkowsky,P.LeCallet,Polytech’Nantes(France)

Abstract:

Withtheadvantageofenhanced3Dsensationbroughtby3DTVtotheviewers,howtoassesstheseadditionalvaluessubjectivelyisstillanopenquestion.In[1],theauthortiredtomodeltheviewingexperienceandnaturalnessintermsofimagequalityanddepth,however,visualcomfortisnotconsidered.Inourstudy[2],theresultsrevealedthatvisualexperienceisnotalinearfunctionintermsofvisualcomfortanddepthrendering.

Inthispaper,weaimtomodelingmorepreciselythequalityofexperiencefor3DTVintermsofimagequality,depthrenderingandalsovisualcomfort.

Syntheticcontentsandnaturecontentsaregeneratedandcaptured;eachparthasatleast5scenes.Foreachscene,wegenerateseveraltestconditionswithcombinationofDepthofFocus-DOF(0.1,0.2and0.3diopter),stereoscopicshapedistortion[2]visualartifacts(videobitrates)andviewsasymmetry(rotation,focallengthdifference,andetc).ThetestmethodologyisbasedontheSAMVIQincludingfourdifferentperceptualscales:visualexperience,imagequality,depthrenderingandvisualdiscomfort.Theanalysisofresultwilldefinetheinteractionbetweenthesetechniqueparametersandthesubjectivescales.Itwillleadtoanewsubjectivequalityofexperiencemodel.Furthermore,categorizationofthedifferenttechnicalparametersisproposedbasedontheirinfluencetowardsdifferentsubjectivescales.Finally,apreliminaryobjectivemodelwillbeproposed.

Insummary,thispaperwillproposeanewmodelofqualityofexperiencefor3DTVconsideringdifferentsubjectivecomponents.Fromthismodel,apreliminaryobjectivemodelwillbederivedtakingintoaccountthecategorizationsoftechniqueparameters.

[1]P.Seuntiëns,“Visualexperienceof3DTV,”DoctorThesis,EindhovenUniversityofTechnology,2006.

[2]W.Chen,etal.,“Newstereoscopicvideoshootingrulebasedonstereoscopicdistortionparametersandcomfortableviewingzone,”inStereoscopicDisplaysandApplicationsXXII,SanFranciscoAirport,California,USA,2011,pp.78631O-13.


New approach on calculating multi-view 3D crosstalk for auto-stereoscopic displaysS.Jung,K.Lee,J.Kang,S.Lee,K.Lim,LGDisplay(Korea,Republicof)

Inthisstudy,wesuggestanewconceptof3Dcrosstalkforauto-stereoscopicdisplaysandobtain3Dcrosstalkvaluesofseveralmulti-viewsystemsbasedonthesuggesteddefinition.First,wefirstmeasuretheangulardependenciesoftheluminanceforauto-stereoscopicdisplaysundervarioustestpatternscorrespondingtoeachviewofamulti-viewsystemandthencalculatethe3Dcrosstalkbasedonournewdefinitionwithrespecttothemeasuredluminanceprofiles.Ournewapproachgivesjustasingle3Dcrosstalkvalueforsingledevicewithoutanyambiguityandshowssimilarorderofvaluestotheconventionalstereoscopicdisplays.Theseresultsarecomparedwiththeconventional3Dcrosstalkvaluesofselectedauto-stereoscopicdisplayssuchas2-view,4-viewand9-viewsystem.Fromtheresult,webelievethatthisnewapproachisveryusefulforcontrolling3Dcrosstalkvaluesofthe3Ddisplaysmanufacturingandbenchmarkingofthe3Dperformancesamongthevariousauto-stereoscopicdisplays.


Towards adapting current 3DTV for an improved 3D experienceL.Blondé,D.Doyen,C.Thébault,Q.Huynh-Thu,TechnicolorS.A.(France);D.Stoenescu,E.Daniel,J.deBougrenetdelaTocnaye,TELECOMBretagne(France);S.Bentahar,EyesTripleShut(France)

RecentupgradesofHDTVinto3DTVresultedinimpairmentsindisplayingstereocontents.Oneofthemostcriticalflawsisprobablycrosstalkandtheresultantghostingeffectimpairingthe3Dexperience.Thepurposeofthisstudyistoidentifytheprimarysourceofcrosstalk,throughoutthefullimagegenerationandviewingchain,foraselectionof3Ddisplays(LCDandPDP)combinedwithdifferentactiveglassestechnologies.Temporalmeasurementshavebeencarriedoutonvariousdisplaypanelsandshutterglassestechnologies.Foreachtechnology,thecrosstalkisacomplexcombinationofseveralfactorsdependingondisplaypanels,shutterglassesandtheirsynchronization,andghostbusting.Thestudytriedtodiscriminatethemainsourcesofcrosstalkineachcase,andtosimulatetheeffectofvariousdisplaypanelorshutterglassesperformanceoptimizations.Conclusionsvarydependingonthedisplaytechnology.ForLCDdisplays,lightleakagereductionatthepanellevelappearsthefirstcauseofcrosstalk,and,inasecondstep,optimizationoftheshutterglasses.Forplasmadisplaystheuseofmoreefficientshutterglassescanreducecolordistortioneffects.


Integral volumetric imaging with high-resolution and smooth-motion parallaxS.Sawada,H.Kakeya,Univ.ofTsukuba(Japan)

Thispaperproposesahighresolutionintegralimagingsystemusingalensarraycomposedofnon-uniformelementallenses.Oneoftheproblemsofintegralimagingisthetrade-offrelationshipbetweentheresolutionandthenumberofviews.Theonlyconventionalwaytosolvethisproblemistouseadisplaypanelwithafinerpixelpitch.Intheproposedmethodlargedisplayareaisusedtoshowasmallerandfiner3Dimage.Torealizeit,theelementallensesshouldbesmallerthantheelementallenses.Tocopewiththedifferenceofsizesbetweentheelementalimagesandtheelementallenses,thelensarrayisdesignedsothattheopticalcentersofelementallensesarelocatedinthecentersofelementalimages,notinthecentersofelementallenses.Inaddition,newimagerenderingalgorithmisdevelopedsothatundistorted3Dimagecanbepresentedwithanon-uniformlensarray.Theproposeddesignoflensarraycanbeappliedtointegralvolumetricimaging,wheredisplaypanelsarelayeredtoshowvolumetricimagesin

Conference 8288


theschemeofintegralimaging.Realizationofsmoothmotionparallaxisalsorealizedbyaddinganotherlayeroflensarraycomposedoffinerelementallenses.


Enlargement of viewing freedom of reduced-view SMV displayJ.Nakamura,T.Takahashi,Y.Takaki,TokyoUniv.ofAgricultureandTechnology(Japan)

Inordertoreducetheresolutionofaflat-paneldisplayusedfortheSMVdisplay,wepreviouslyproposedareduced-viewSMVdisplaygeneratingviewingzonesonlyaroundviewer’sleftandrighteyes.Wecombinedaheadtrackingsystememployingacamerawiththereduced-viewSMVdisplaytoenlargetheviewingfreedomsimply.However,thelongitudinalviewingfreedomwaslimited.Inthepresentstudy,weproposeaSMVdisplaysystemwhichgeneratesviewingzonescorrespondingto3Dpositionsofviewer’sleftandrighteyestoenlargeboththehorizontalandlongitudinalviewingfreedoms.Aneyetrackingsystemwithastereocameraiscombinedwiththepreviouslydevelopedreduced-viewSMVdisplay.Pixelsontheflat-paneldisplaywhichareseenthrougheachcylindricallensbytheleftandrighteyesareidentifiedgeometricallyforallofthecylindricallenses.Parallaximagesaregeneratedcorrespondingtothe3Dpositionsofbotheyes.TwoormoreconsecutivepixelsincludingtheidentifiedpixelsareusedtodisplaytheparallaximagestoachievetheSMVdisplaycondition.The3Dresolutionofthedisplaywas256×192,andthescreensizewas2.57inches.TheeyetrackingsystemwascomposedoftwoUSBcameras.Theframerateoftheeyetrackingandtheimageupdatewas7Hz.


Development of high-speed, phase-modulating spatial light modulators for 3D-holographic displaysH.Takagi,T.Goto,A.Baryshev,M.Inoue,ToyohashiUniv.ofTechnology(Japan)

Wehavedemonstratedspatiallightmodulators(SLMs)withanewdesign-so-calledphase-modulatingSLMs(eMO-SLM).WorkingelementoftheseSLMscomprisedmagneto-andelectro-opticalbilayerintroducedintoaFabri-Perotmicrocavity.Responsesfromthesecavitieswereanalyzedtheoreticallyandmeasuredexperimentally,whencontrollingappliedelectricfields.

CalculatedprototypeofeMO-SLMshadthestructureofanti-reflectioncoating/SGGGsubstrate/(Ta2O5/SiO2)f/Bi:YIG/PLZT/(SiO2/Ta2O5)r,wheretheresponseofBi:YIG/PLZTmagneto/electro-opticalbilayerwascontrolledbytheelectricfieldappliedtoPLZT.Calculatedvoltage-drivenresponsesoftheconsideredstructureshavedemonstratedthatphasechangeover360deg.atavoltageof0-0.5Vcanbeachievedforthereflectedlightintensityof99%.SinceeMO-SLMscomprisingBi:YIGcannotprovidegoodperformanceinthebluespectralrangebecauseofalargeabsorptioninBi:YIG,calculationsforeMO-SLMwithparamagneticMOmaterialsweredone.Itwasshownthatthephasemodulationof180deg.atreflectivityof90%wasachievedintheshort-wavelengthrange(405nm).AnalysisshowedthateMO-SLMsprovidedalargephasemodulation,highopticalefficiencyandhighswitchingspeedforwavelengthsfromthefullvisiblespectralrange.Themodulationwasalsoobservedexperimentally.


Color hologram generation using depth map of real objects with viewing-zone-angle expansionK.Nomura,TokyoUniv.ofScience(Japan)andNationalInstituteofInformationandCommunicationsTechnology(Japan);R.Oi,

T.Kurita,NationalInstituteofInformationandCommunicationsTechnology(Japan);T.Hamamoto,TokyoUniv.ofScience(Japan)

Inthispaper,weproposethehologramgenerationmethodapplyingaviewing-zone-angleexpansionmethodusingcoloranddepthinformationofrealobjects.First,weparallel-projecttheobjectsusingcoloranddepthinformationandcalculatethelightpropagationfromeachpixel.Atthistime,toexpandhorizontalviewing-zonetriply,wecalculatethepropagatedlightwhichsatisfiesthemaxdiffractionangletohorizontalthreedirectionsfromeachpixel.Atthesametime,toeliminatetheconjugateimage,weapplythehalf-zone-plateprocessingtoourproposedmethod.Second,wemadetheopticalreproductionexperimentusinga4fopticalsystemwiththesinglesidebandmethodtoeliminatetheundesirablelight.Also,the3Dimagereconstructedfromahologramwassoclearwithoutcoloraberration.Inaddition,whenweobserveditfromtheexpandedviewing-zone,wecouldobservethecorresponding3Dimagesfromeachviewing-pointandweshowedtheeffectivenessofourproposedmethod.Moreover,appliedFast-Fourier-Transform(FFT)tothisviewing-zone-angleexpansionmethod,the3Dimageisreconstructedfromthehologramusingthismethodwithnodegradation.Finally,wealsoconfirmedtheprocessingtimeisninetimesfasterthanprevious.


Stereoscopic surround displays using interference filtersS.Peikert,J.Gerhardt,FraunhoferFIRST(Germany)

Stereoscopyhasbecomeacommontechnologyinmodernmedia.Howeverplanarstereoscopicscreenssufferfromalimitedfieldofview.Theusageofmultipleprojectorsallowstoextendthefieldofviewandtosetupdisplaysthatsurroundtheviewers.Inthatwaytheviewercanbeputamidstthemedialcontentinsteadofplayingitinfrontofhim.

Toachievestereoscopyonsurrounddisplaysinterferencefiltershavesomeadvantagesoverothertechniques.Howeverthesefiltersintroducestrongcolordifferencesbetweentheprojectors,whichmayrevealthatthedisplayiscompoundbymultipleprojectors.Thisarticlepresentsmethodsforacomputationallyefficientcorrectionofthecolorimetricpropertiesofmulti-projectorsurrounddisplays.Thiscorrectionisbasedonautomatedmeasurementsbymultiplecamerasandaspectrometer.

Thedescribedmethodswerevalidatedbyapplyingthemtoastereoscopicdomedisplaymadeupof16highdefinitionprojectorsequippedwithInfitecfilters.Onthatdisplayweachievedasignificantimprovementofthecolorimetricpropertiescomparedtoregularsoft-edgeblending.Ourreferencesetupshowsthatthemulti-projectorapproachcombinedwithinterferencefiltersallowstobuildhighlyimmersivestereoscopicsurrounddisplaysfulfillingtoday’srequirementsonspatialresolution,frameratesandinteractionlatencies.


Design of extended viewing zone at autostereoscopic 3D display based on diffusing optical elementM.Kim,Y.S.Hwang,E.Kim,KwangwoonUniv.(Korea,Republicof)

Inthesedays,3Ddisplayindustrychangescurrentlyanon-glassestypetoanon-glassestype.Thispaperproposesthataviewingzoneformultipleviewwhichisdesignedforthe3DdisplayusingDOE(DiffusingOpticalElement)onthepanelshowslargerthanthatoftheparallaxbarriermethodorlenticularmethodandacrosstalkcanbeimproved.Throughproposedmethod,itisshowntoenabletheexpansionandadjustmentoftheareaofviewingzoneaccordingtoviewingdistance.TheDOEismadebyrecordingangularlydifferentmultipleinterferencepatternaccordingtoBraggdiffractionusing532nmgreenlaser.Insummary,forlateralanddepthdirectionviewingzoneofautostereoscopic3DdisplayforamultipleviewundertheDOE

Conference 8288


systemcanbefreelydesignedaccordingtoadiffusingangleandadeclineangle.


Floating image device with auto-stereoscopic display and viewer-tracking technologyC.Chen,K.Tseng,C.Wang,C.Tsai,IndustrialTechnologyResearchInstitute(Taiwan)

Torealizeadisplaysystemwhichcanprojectthe3Dimagetoafreespace,weproposedafloatingimagedevicewithauto-stereoscopicdisplayandviewertrackingtechnology.Thefloatingimagedevicehas3majorparts:(1)anopticalprojectionsystem,(2)a2-viewauto-stereoscopicdisplay,and(3)aviewertrackingequipment.Withthesethreeparts,theviewercanwatch3Dimagesfloatingintheairwithbothmotionandbinocularparallaxfromawideviewangle.Thefloatingimageisdesignedtobeprojectedinafreespacereachablebythehandsoftheviewer.Therefore,free-spacetouchandinteractiontechnologiescanbeincludedtoincreasethefunctionsofthesystem.


A time-sequential, mutli-view, autostereoscopic display without resolution loss using a multi-directional backlight unit and an LCD panelH.Kwon,H.Choi,SejongUniv.(Korea,Republicof)

Inthispaper,atime-sequentialmulti-viewautostereoscopicdisplaywithoutresolutionlossisproposed.ThesystemconsistsofanLCDpanelandamulti-directionalbacklightwhichhasseveralgroupsofpointlightsources(LEDs)andalensarray.Thedistancebetweenthelensarrayandthepointlightsourceissettobethefocallengthofthelensarraysothatthebacklighttiltsthedirectionoflightraystomaketheobserversseedifferentimagesindifferentviewpoints.Byturningonandoffthepointlightsourcessequentially,theimagesontheLCDpanelcanbeobservedindifferentlocations.Therefore,ifthechangetheimagesontheLCDpanelissynchronizedwiththeoperationofpointlightsourcearray,thenumberof3Dviewpointscanbeincreasedwhiletheresolutionofthe3Dimagesisnotreduced.Theproposedmethodcanberealizedwiththelatestflatpaneldisplaytechnologiesanditisexpectedtorealizean8-viewautostereoscopicdisplaywithoutresolutionloss.Inaddition,the2Dmodecanberealizedbysimplyturningonallpointlightsourcesinthebacklight.


Full-resolution autostereoscopic display with all-electronic tracking systemJ.Gaudreau,PolarScreens,Inc.(Canada)

PolarScreensisdevelopinganew3DdisplaytechnologycapableofdisplayingfullHDresolutionineacheyewithouttheneedforglasses.Thetechnologycombinesafullresolutionpolarizeddisplay,averticaloversizePatternedPolarizerFilmandaheadtrackingsystem.ThisconceptcanbeappliedtoanyfullresolutionpolarizeddisplaylikeiZ3D,Perceiva,orcomingupactiveretarder3Ddisplay.Thetechnologyrelyonamulti-pixelsalternatedpatternencodedinthestereoimagetofollowtheheadmovement.

Theendresultisafullresolutionautostereoscopicdisplaywithcompleteheadmovementfreedom.Therearenomechanicalmovingpart(likelenticular)orextraactivecomponents(likeslitLCDactingasparallelbarrier)tofollowuser’seyes.Thedisplayhasall2D/3Dinformationallthetimesothereisnoneedforfullscreenorwindowed2D/3Dswitchableapparatus.


Design of crossed-mirror array to form floating 3D LED signsH.Yamamoto,Univ.ofTokushima(Japan)andJSTCREST(Japan);H.Bando,S.Suyama,Univ.ofTokushima(Japan)

WehavedesignedareflectiveopticaldevicetoformafloatingimageofanLEDsignage.Theopticaldeviceiscomposedofcrossedmirrors.Eachgridinthecrossed-mirrorarrayactsasadihedralcornercubeandreflectsincidentlighttotheoppositesideofthearraywithkeepingthedirectionparalleltothearray.WhenanLEDsignisplacedinfrontofthecrossed-mirrorarray,arealimageoftheLEDsignisformedattheplanesymmetricpositionoftheLEDsignaboutthecrossedmirrorarray.Inordertoimplementthree-dimensionallyfloatingLEDsignage,theopticaldevicearerequiredwide(atleast45deg.)fieldofangle.BecausethereareblackareasbetweenLEDlamps,widepointspreadfunctionissuitable.Furthermore,therequiredsizeoftheopticalelementsislargerthanourbodyinfuture.Then,scalabilityofcompositionisneeded.Inthispaper,weinvestigatethedesignissueoncrossedmirrors.Inordertoprovidewidepointspreadfunction,apertureratiois0.8horizontallyandvertically.Formationoffloatingimagewithacertainspreadhasbeeninvestigatedwithprototypedevices.Furthermore,floatingdisplayofanLEDsignhasbeendemonstrated.


Novel multi-view generation framework for 3D displaysK.Hwang,Y.Cho,H.Lee,D.Park,C.Kim,SamsungAdvancedInstituteofTechnology(Korea,Republicof)

Inthispaper,weproposeanovelmulti-viewgenerationframeworkthatconsidersnotonlyhigh-qualityofeachviewbutalsospatio-temporalconsistencyofeachsynthesizedmulti-view.Ratherthanindependentlyfillingintheholesoftheindividualgeneratedimage,theproposedframeworkgatherstheholeinformationineachsynthesizedmulti-viewimagetoreferenceviewpointandconstructsholemapandSVRL(SingleViewReferenceLayer)atthereferenceviewpointandrestorestheholesintheSVRL,whichresultsinthespatio-temporallyconsistentviewgeneration.Theholemapisconstructedusingthedepthinformationofthereferenceviewpointandtheratioofinput/outputbaselinelengthsothattheholesintheSVRLcanrepresenttheholesintheothermulti-viewimages.Fortemporallyconsistenthole-fillingoftheSVRL,therestorationofholesinthecurrentSVRLisperformedbypropagatingthepixelvalueofthepreviousSVRL.Theremainedhole-fillingisperformedbydepth-basedexemplarbasedinpaintingmethod.Theexperimentalresultsshowedthattheproposedmethodcangeneratehighqualityandspatio-temporalconsistentmulti-viewimagesinvariousinput/outputenvironments.Inaddition,theproposedframeworkcandecreasethecomplexityofhole-fillingprocessbyreducingtherepetitionoftheholefilling.


Calibration of multiview camera with parallel and decentered image sensorsM.Ali-Bey,S.Moughamir,N.Manamanni,Univ.deReimsChampagne-Ardenne(France)

Thispaperfocusesonthecalibrationproblemofamulti-viewshootingsystemdesignedfortheproductionof3Dcontentforauto-stereoscopicvisualization.Theconsideredmulti-viewcameraischaracterizedbycoplanaranddecenteredimagesensorsregardingtothecorrespondingopticalaxis.BasedontheFaugérasandToscani’scalibrationapproach,acalibrationmethodishereinproposedforthecaseofmulti-viewcamerawithparallelanddecenteredimagesensors.Atfirst,thegeometricalmodeloftheshootingsystemisrecalledandsomeindustrialprototypeswithsomeshootingsimulationsarepresented.Next,thedevelopmentoftheproposedcalibrationmethod

Conference 8288


isdetailed.Finally,somesimulationresultsarepresentedbeforeendingwithsomeconclusionsaboutthiswork.


Inversion-free multiview subpixel rendering for natural 3D presentationY.Kim,G.Sung,J.Park,D.Nam,D.Park,SamsungAdvancedInstituteofTechnology(Korea,Republicof)

Thepurposeofthispaperistoremovetheinversionproblemforthewideviewingangleandincreasethe3Dresolutionusingthesubpixelrenderingalgorithm.Inthispaper,aninversion-freesubpixelrenderingmethodthatuseseyetrackinginamultiviewdisplayisproposed.Themultiviewdisplaycausesaninversionproblemwhenoneeyeoftheuserisfocusedonthemainregionandtheothereyeisfocusedonthesideregion.Theproposedrenderingalgorithmusesonemainlobeaswellastwosideloberegionsdependingontheuser’smovement.Thistrackstheuser’seyeposition,andthemovementoftheeyecontinuouslychangestheviewimageusingtherelationbetweentheeyepositionandthemainlobeandsideloberegion.Also,toincreasethe3Dresolutionwithoutthecolorartifact,thesubpixelrenderingalgorithmusingsubpixelareaweightingisproposed.Thismethodusesthesuperpositionofthesubpixelandthelensrayaxes.Thesubpixelvaluesarere-calculatedwiththeratioofthesuperposition.Intheexperiments,36-viewimageswereseenusingactivesubpixelrenderingwiththeeyetrackingsystemina4-view3Ddisplay.


Wide-viewing angle three-dimensional display based on the ray reconstruction method using multiple micro-projectorsH.Takahashi,K.Hirooka,OsakaCityUniv.(Japan);K.Yamada,OsakaUniv.(Japan)

Weproposeanomnidirectional3Ddisplaysystem.Thisisatoolforcommunicationarounda3Dimagebetweenasmallnumberofpeople.This3Ddisplaysystemconsistsofmultiplebasic3Ddisplayunits.Thebasicunitconsistsofamicro-projector,alenticularlensarraysheet,andacylindricallens.Inthisbasicunit,sinceascreenisnotused,thelightraysfromamicro-projectorpassthroughalenticularlensarraysheetandobserveddirectly.Thus,thespatialdensitydistributionofprojectedlightraysispartial.Toaveragethespatialdensityofprojectedlightrays,weuseacylindricallens.Toincreasetheviewingangle,wealignedmultiplebasicunitsinacircle,anddisplayed3Dimagesatthecenterofacircle.Toverifytheeffectivenessoftheproposed3Ddisplay,weconstructedtheprototypesystem.Thisprototypeconsistsof8basicunits.Theyarealigned18-degreeapartinacircleandtheradiusis95mm.Themaximumsizeofdisplayed3Dimagesis35mmx40mmx35mm.Theviewingangleofa3Dimageis124-degree.Thispaperdescribestheprincipleofproposed3Ddisplaysystem,andalsodescribestheexperimentalresults.


Multi-layered, see-through movie in diminished realityY.Uematsu,T.Hashimoto,T.Inoue,N.Shimizu,H.Saito,KeioUniv.(Japan)

Thispaperpresentsa3Dsee-throughmoviefortheauto-stereoscopicdisplay.Ourtargetsceneisabaseballgame.Wecreateacatcher’sviewpointmovie,whichisusuallyimpossibletobecapturedattheofficialgame,basedonDiminishedRealitytechniquebyviewtranslationandobstacleremovingwithmultiplecameras.

Welocatethreecolorcamerasbehindthecatcherandumpire,andremovethoseobstaclestovisualizethepitcher’sappearanceasifseeingthepitcherthroughthecatcherandumpire.Sincethepitcher

isfarenoughawayfromthecameras,thepitcher’sappearanceatthecentercameraisrecoveredbyplanartranslationoftheleftandrightcameras.Theregionsoftheobstaclesaredetectedbyusingdepthinformation.Byoverlayingtherecoveredpitcherontotheregionsofobstacles,wecanvirtuallyseethecatcher’sviewpointmovie.

Oursee-throughmovieisalsoappliedtotheauto-stereoscopicdisplay.Eachobjectisassignedtoalayerwhichhaseachdistancein3Dspace.Incontrastwithseeingthe2Dsee-throughmoviebychangingtransparencyoftheobstacles,then,thebackground(ex.Pitcher)canbeseeninfrontoftheobstacles(ex.Catcher)bychangingthe3Dpositionofthelayerofbackground.Thereforewecanmanipulatethepositionofrealobjectsin3Dbyusingoursystem.


An independent motion and disparity vector prediction method for multiview video codingS.Ryu,J.Seo,YonseiUniv.(Korea,Republicof);J.Y.Lee,SamsungAdvancedInstituteofTechnology(Korea,Republicof);D.H.Kim,YonseiUniv.(Korea,Republicof);H.Wey,SamsungAdvancedInstituteofTechnology(Korea,Republicof);K.Sohn,YonseiUniv.(Korea,Republicof)

Theinter-viewpredictionisusedinadditiontothetemporalpredictioninordertoexploitboththetemporalandinter-viewredundanciesinMultiviewvideocoding.Accordingly,themultiviewvideocodinghastwotypesofmotionvectorsthatarecalledthetemporalmotionvectorandthedisparityvector,respectively.Thedisparityvectorisgenerallyuncorrelatedwiththetemporalmotionvector.However,theyareusedtogethertopredictthemotionvectorregardlessoftheirtypes,thereforeanefficiencyoftheconventionalpredictivecodingofmultiviewvideocodingisdecreased.Inordertoincreasetheaccuracyofthepredictedmotionvector,anewmotionvectorpredictionmethodincludingvirtualtemporalmotionvectorandvirtualdisparityvectorisproposedforboththemultiviewvideoandmultiviewvideoplusdepthformats.Theexperimentalresultsshowthattheproposedmethodcanreducethecodingbitratesby6.5%inaverageand14.7%atmaximumintermsoftheBjontegaardmetriccomparedtotheconventionalmethod.


Geometry compensation using depth and camera parameters for three-dimensional video codingD.Kim,J.Seo,S.Ryu,YonseiUniv.(Korea,Republicof);J.Y.Lee,H.Wey,SamsungAdvancedInstituteofTechnology(Korea,Republicof);K.Sohn,YonseiUniv.(Korea,Republicof)

Oneoftheimportantissuesforthenextgenerationbroadcastingsystemishowtocompressamassiveamountofthree-dimension(3D)videoefficiently.Inthispaper,ageometriccompensationmethodisproposedtoobtainhighercodingefficiencyusingtheinformationofthecolorvideos,depthvideosandcameraparameters.

Consideringthedisocclusionproperties,ahole-fillingmethodforPviewisalsoproposedtofurtherincreasethecodingperformanceinhighbit-rates.

TheexperimentalresultsshowthattheproposedalgorithmcanincreasePSNRover0.22dBand0.63dBforPandBpictures,respectively.Meanwhile,wecanachieveupto6.28%and18.32%BDbit-ratesgainforPandBpictures,respectively.


Converting conventional stereo pairs to multi-view sequences using morphingR.Olsson,V.K.Adhikarla,S.Schwarz,M.Sjöström,MidSwedenUniv.(Sweden)

Conference 8288


Autostereoscopicmultiviewdisplaysrequiremultipleviewsofascenetoprovidemotionparallax.Whenanobserverchangesviewingangledifferentstereoscopicpairsareperceived.Thisallowsnewperspectivesofthescenetobeseengivingamorerealistic3Dexperience.However,capturingarbitrarynumberofviewsisatbestcumbersome,andinsomeoccasionsimpossible.Conventionalstereovideo(CSV)operatesontwovideosignalscapturedusingtwocamerasattwodifferentperspectives.Generationandtransmissionoftwoviewsismorefeasiblethanthatofmultipleviews.Itwouldbemoreefficientifmultipleviewsrequiredbyanautostereoscopicdisplaycanbesynthesizedfromthesesparsesetofviews.

Thispaperaddressestheconversionofstereoscopicvideotomultiviewvideousingthevideoeffectmorphing.Differentmorphingalgorithmsareimplementedandevaluated.Contrarytotraditionalconversionmethods,thesealgorithmsdisregardthephysicaldepthexplicitlyandinsteadgenerateintermediateviewsusingsparsesetsofcorrespondencefeaturesandimagemorphing.Anovelmorphingalgorithmisalsopresentedthatusesscaleinvariantfeaturetransform(SIFT)andsegmentationtoconstructrobustcorrespondencefeaturesandqualitativeintermediateviews.Allalgorithmsareevaluatedonasubjectiveandobjectivebasisandthecomparisonresultsarepresented.


Multiview video and depth compression for free-view navigationsY.Higuchi,M.P.Tehrani,T.Yendo,T.Fujii,M.Tanimoto,NagoyaUniv.(Japan)

Inthispaper,wediscussamultiviewvideoanddepthcodingsystemforFreeviewpointTV.MPEGadhocgroupisdiscussinganewcodingsystemwhichassumesthatinputandoutputdataconsistsofmultiviewvideosequencescapturedbyacameraarrayanddepthinformationcorrespondingeachview.Afterdecodingthedata,virtualviewsynthesisisusedtogeneratenovelsynthesizedviews.Forcompressionofmultiviewvideosequences,MultiviewVideoCoding(MVC)isstandardizedbytheJointVideoTeam(JVT).However,MVConlytakesthemultiviewvideosequences,sodepthinformationhastobetransmittedbyalternativecodingalgorithminordertoprovideaudienceswithmultiviewvideoapplicationsinlowcomplexity.Additionally,weshouldadjustandselectquantizationparametersforeachvideoanddepthstreamwhichmaximizethequalityofsynthesizedview,becausetransmittablevolumeisgenerallylimited.Inthispaper,asafirststeptoproposeanappropriatecompressionmethodforthenewframework,weinvestigatetheeffectonvirtualviewsynthesisqualitygivenbydifferenceoftransmissionratesbetweenmultiviewvideoanddepthsequences.Ourexperimentalresultshowthatbitratesofmultiviewdepthstreamhaslesseffectontheviewsynthesisqualitycomparedwiththevideostream.


A layered inpainting method for virtual view synthesisS.Kim,K.J.Lee,SeoulNationalUniv.(Korea,Republicof);I.D.Yun,HankukUniv.ofForeignStudies(Korea,Republicof);S.U.Lee,SeoulNationalUniv.(Korea,Republicof)

Inthispaper,wehavepresentedanewapproachinviewsynthesisbasedontheconceptmissingareabeingfilledwiththebackgroundparts.Indeed,itbasicallyconsistedoftwopart:layerseparationandlayerinpainting.Inthelayerseparation,itextractsthebackgroundpartfromtheimagesandbecomeabigissuesincelayerinpaintinginferredtheresultoflayerseparation.Thenlayerinpaintingisimpliedtoreplacetheocclusionregionwiththebackgroundlayer.Tofindthemostsimilarpatch,priorityisused.Therefore,itsignificantlyimprovethequalityofinpaintinginviewsynthesis.It,yet,shouldbenotedthatthisresearchhasonlyexaminedwithimagesetfromtheMicrosoft.Ithasclearboundarybetweenobjectandbackground,sotheseparationiseasycomparedtootherimages.Therefore,bettermechanismtoseparateobjectisalsonecessaryinfurtherresearch.


Analysis on ray reconstruction characteristics of multi-view and integral imaging displayH.Kim,H.Kim,K.Jeong,J.Park,ChungbukNationalUniv.(Korea,Republicof)

Weperformedacomparativeanalysisformulti-viewautostereoscopicdisplayandhorizontalparallaxonlyintegralimagingdisplay.Thespatio-angularraydistributionreconstructedbytwotechnologiesiscalculatedandusedasametrictoevaluatethethree-dimensionalimagereconstructionquality.Basedontherayreconstructioncharacteristics,wealsoproposeamethodtoconvertasetofperspectiveimagesformulti-viewdisplaystoasetoforthographicimagesforintegralimagingdisplay.


A content-based method for perceptually driven joint color/depth compressionE.Bosc,L.Morin,M.Pressigout,InstitutNationaldesSciencesAppliquéesdeRennes(France)

Multi-viewVideoplusDepth(MVD)datarefertoasetofconventionalcolorvideosequencesandanassociatedsetofdepthvideosequences,allacquiredatslightlydifferentviewpoints.Thishugeamountofdatanecessitatesareliablecompressionmethod.However,thereisnostandardizedcompressionmethodforMVDsequences.H.264/MVCcompressionmethod,whichwasstandardizedforMulti-View-Videorepresentation(MVV),hasbeenthesubjectofmanyadaptationstoMVD.However,ithasbeenshownthatMVCisnotwelladaptedtoencodemulti-viewdepthdata.

WeproposeanoveloptionasforcompressionofMVDdata.Itsmainpurposeistopreservejointcolor/depthconsistency.Theoriginalityoftheproposedmethodreliesontheuseofthedecodedcolordataasapriorfortheassociateddepthcompression.Thisismeanttoensureconsistencyinbothtypesofdataafterdecoding.Ourstrategyismotivatedbypreviousstudiesofartifactsoccurringinsynthesizedviews:mostannoyingdistortionsarelocatedaroundstrongdepthdiscontinuitiesandthesedistortionsareduetomisalignmentofdepthandcoloredgesindecodedimages.Thusthemethodismeanttopreserveedgesandtoensureconsistentlocalizationofcoloredgesanddepthedges.

Toensurecompatibility,coloredsequencesareencodedwithH.264.Depthmapscompressionisbasedona2Dstillimagecodec,namelyLAR(LocallyadaptedResolution).Itconsistsinaquad-treerepresentationoftheimages.Thequad-treerepresentationcontributesinthepreservationofedgesinbothcoloranddepthdata.

Theadoptedstrategyismeanttobemoreperceptuallydriventhanstate-of-the-artmethods.TheproposedapproachiscomparedtoH.264encodingofdepthimages.ObjectivemetricsscoresaresimilarwithH.264andwiththeproposedmethod,andvisualqualityofsynthesizedviewsisimprovedwiththeproposedapproach.


Efficient panoramic sampling of real-world environments for image-based stereoscopic telepresenceL.E.Gurrieri,E.Dubois,Univ.ofOttawa(Canada)

Akeyproblemintelepresencesystemsishowtoeffectivelyemulatethesubjectiveexperienceof‘beingthere’deliveredbyourvisualsystem.Asteptowardsvisualrealismcanbeobtainedbyusinghigh-qualitypanoramicsnapshotsinsteadofcomputergeneratedmodelsofthescene.Unarguably,anevenbetterimmersiveexperiencecanbeachievedbyintroducingmonoscopicdepthcuesorbyusingstereoscopic-panoramicimages.Thelatterisadauntingtask

Conference 8288


consideringthelargenumberofpanoramicsourcesrequiredtoproduceacomfortablestereoscopicnavigation.Inthisscenario,constraintssuchastheacquisitiontime,renderingcomplexityandstoragecapacitybecomeimportant.Thesedesignlimitationsmotivatedthedevelopmentofanalternativemethodtofacilitatealargescalestereoscopicsurveyofascene.Ourapproachisbasedonthesamplingofatargetreal-wordsceneusinganarrangementofpanoramicclusters.Themosaickingofspecificsectionsofthepanoramicimageswithineachclusterhelpstogenerateonecompletebinocularpanoramicpairpercluster.Thisnovelomnistereoscopictechniqueenablesthefastacquisitionandgenerationofhigh-qualitystereoscopicimages,usingoff-the-shelfpanoramiccameras,andwithoutrequiringadditionaldepthinformation.


Partial 2D-to-S3D conversion and the cognitive characteristicsY.Koido,T.Kawai,WasedaUniv.(Japan)

2Dtostereoscopic3D(S3D)conversionmethods,oneoftheapproachestocreateS3Dcontent,aredividedintoautomatic“on-line”andmanual“off-line”.Off-lineconversionof2DtoS3DconversionhascostissuesinspiteofhigherS3Dimagequalitythanon-lineone.Inaddition,althoughoff-lineconversioncancontrolparallaxamountmoreflexiblethanstereofilming,2Dmaterialimagesareconvertedcorrespondingtothemonocularinformationinmostcases.Therefore,theauthorsproposeanewmethodofS3Drepresentationusing2DtoS3Dconversionwhichgivesuncrossedparallaxtowhole2Dimagesandcrossedparallaxtotheonlyparticulararea.Theauthorsconductedsubjectiveandobjectiveevaluationstoexaminethecognitivecharacteristicsofpartial2DtoS3Dconversion.Inthispaper,thedetailsoftheproposedmethodandtheevaluationresultsaredescribed.


Disparity-compensated view synthesis for S3D content correctionR.Philippe,C.Thébault,V.Drazic,P.Conze,TechnicolorS.A.(France)

Theproductionofstereoscopic3Dcontentisconsiderablyincreasingandexperiencein2-viewacquisitionisinprogress.Highqualitymaterialtotheaudienceisrequiredbutnotalwaysensured,andcorrectionofthestereoviewsmayberequired.Thisisdoneviadisparity-compensatedviewsynthesis.Arobustmethodhasbeendevelopeddealingwiththeseacquisitionproblemsthatintroducediscomfort(e.ghyperdivergenceandhyperconvergence,...)aswellasthoseonesthatmaydisruptthecorrectionitself(verticaldisparity,colordifferencebetweenviews,...).Themethodhasthreephases:apreprocessinginordertocorrectthestereoimagesandestimatefeatures(e.g.disparityrange,...)overthesequence.Thesecond(main)phaseproceedsthentodisparityestimationandviewsynthesis.Dualdisparityestimationbasedonrobustblock-matching,discontinuity-preservingfiltering,confidenceandocclusionhandlinghasbeendeveloped.Accurateviewsynthesisiscarriedoutthroughdisparitycompensation.Disparityassessmenthasbeenintroducedinordertodetectandquantifyerrors.Apost-processingdealswiththeseerrorsasafallbackmode.Theeffectivenessofourframeworkhasbeendemonstratedwithcomplexvideosequences.QualityassessmenthasshownthehighqualityofourDIBRmethodaswellasthesignificantadvantageofintroducingpre-andpost-processingintheverycriticalcases.


Virtual view interpolation at arbitrary view points for mixed-resolution 3D videosS.Lee,S.Lee,H.Wey,D.Park,SamsungAdvancedInstituteofTechnology(Korea,Republicof)

Forfullmotionparallax3Ddisplay,itisnecessarytosupplymultipleviewsobtainedfromaseriesofdifferentlocations.However,itisimpracticaltodeliveralloftherequiredviewsbecausetheywillresultinahugesizeofbitstream.Inthepreviouswork,authorsproposedamixedresolution3Dvideoformatcomposedofheterogeneousresolutionvideoanddepthinformationpairs,andalsosuggestedaviewsynthesisalgorithmforthemixedresolutionvideos.Thispaperreportsmorerefinedviewinterpolationmethodandimprovedresults.

Themixedresolutionimageviewsynthesisalgorithmistoexploithighfrequencycomponentsfromthehighresolutionimageinput.Theproposedalgorithmiscomposedoftheprocessesof1)viewwarping2)highfrequencycomponentdetectionandweightcomputation,and3)virtualpixelblending.

Onthecontraryoftheconventionalviewsynthesisapproachutilizingthesameresolutioninputs,theproposedsynthesismethodemploysanapproachthatusesthehighfrequencycomponentsfromthehighresolutionimages.TheexperimentalresultsshowthattheproposedviewsynthesisalgorithmcanimprovethePSNRaswellastheperceptualimagequality.


Efficient dense blur map estimation for automatic 2D-to-3D conversionL.Vosters,TechnischeUniv.Eindhoven(Netherlands)andAxonDigitalDesign(Netherlands);G.deHaan,PhilipsResearchNederlandB.V.(Netherlands)andTechnischeUniv.Eindhoven(Netherlands)

Focusisanimportantdepthcuefor2D-to-3Dconversionoflow-depthoffieldimagesandvideo.However,focuscanbeonlyreliablyestimatedonedges.Therefore,Beaetal.[1]firstproposedanoptimizationbasedapproachtopropagatefocustonon-edgeimageportions,forsingleimagefocusediting.Whiletheirapproachproducesaccuratedenseblurmaps,thecomputationalcomplexityandmemoryrequirementsforsolvingtheresultingsparselinearsystemwithstandardmultigridor(multilevel)preconditioningtechniques,areinfeasiblewithinthestringentrequirementsoftheconsumerelectronicsandbroadcastindustry.Inthispaperweproposefastefficient,lowlatency,linescanningbasedfocuspropagation,whichmitigatestheneedforcomplexmultigridor(multilevel)preconditioningtechniques.Inadditionweproposefacialblurcompensationtocompensateforfalseshadingedgesthatcauseincorrectblurestimatesinpeople’sfaces.Ingeneralshadingleadstoincorrectfocusestimates,whichmayleadtounnatural3Dandvisualdiscomfort.Sincevisualattentionmostlytendstofaces,oursolutionsolvesthemostdistractingerrors.Asubjectiveassessmentbypairedcomparisononasetofchallenginglow-depth-of-fieldimages,showsthattheproposedapproachachievesequal3Dimagequalityasoptimizationbasedapproaches,andthatfacialblurcompensationresultsinasignificantimprovement.


ROI-based transmission method for stereoscopic video to maximize rendered 3D video qualityC.T.E.R.Hewage,M.G.Martini,KingstonUniv.(UnitedKingdom)

Atechniquetoimprovetherenderingqualityofnovelviewsforcolourplusdepthbased3Dvideoisproposed.Mostdepthdiscontinuitiesoccuraroundtheedgesofdepthmapobjects.Ifinformationaroundedgesofbothcolouranddepthmapimagesarelostduringtransmission,thiswillaffectthequalityoftherenderedviews.Thereforethisworkproposesatechniquetocategorizeedgeandsurroundingareasintotwodifferentregions(i.e.,RegionOfInterests(ROIs))andlaterprotectthemseparatelytoprovideUnequalErrorProtectionduringtransmission.Inthiswaythemostimportantedgeareas(vitalfornovelviewrendering)willbeprotectedaheadofothersurroundingareas.ThismethodistestedoveraH.264/AVCbasedsimulcastencodingandtransmissionsetup.Theresultsshowimprovedrendered

Conference 8288


qualitywiththeproposedROIbaseUnequalErrorProtection(UEP)methodcomparedtoEqualErrorProtection(EEP)method.


Stereoscopic image-inpainting-based, view-synthesis algorithm for glasses-based and glasses-free 3D displaysC.Yuan,SharpLabs.ofAmerica,Inc.(UnitedStates);C.H.H.Liao,TheUniv.ofSouthernCalifornia(UnitedStates);H.Pan,SharpLabs.ofAmerica,Inc.(UnitedStates)

Recenttrendsin3Ddisplaysclearlyshowthattheglasses-based3Ddisplayshavebecomemainstream,whiletheglasses-free3Ddisplaysareemerging.Theexistingstereoscopiccontent,however,doesnotalwaysprovidetheoptimalviewingexperiencesonthe3Ddisplays.Asforglasses-baseddisplays,theoriginalstereoscopic3Dcontentmaynothavethedesiredstrongdepthormaycausevisualdiscomfortduetoexcessivedepthanddistortion.Newstereoimageswithtuned3Ddepthneedtobesynthesizedforviewers’preferencesandcomfort.Asfortheglasses-freedisplays,morethantwoviewsareoftendesiredandthereforetheoriginaltwo-viewcontentneedstobeconvertedintomulti-viewformat.Aviewsynthesisalgorithmisneededforgeneratinghigh-qualitystereoscopiccontentforbothtypesof3Ddisplays.Thegoalofourworkistodevelopanovelviewsynthesisalgorithmforgeneratinghigh-qualitystereoscopic3Dcontentandprovidingoptimalviewingexperiencesonbothglasses-basedandglasses-free3Ddisplays.Theproposedapproachconsistsofthefollowingsteps:(1)Disparity(depth)scaling;(2)Stereoinpainting;(3)Removalofgridquantizationartifact.Wehaveappliedtheviewsynthesisalgorithmtovariousreal-lifeimagesequences.Themethodisabletogeneratehigh-qualityvirtualviewsneededbydepthtuningonglasses-based3Ddisplays(2-viewcontent)andvirtualcomposite8-viewimageryneededbyglasses-free3Ddisplays.


Towards a real-time, high-definition depth sensor with hardware-efficient stereo matchingK.Zhang,IMEC(Belgium)andKatholiekeUniv.Leuven(Belgium);G.Yi,C.Liao,C.Lin,H.Yeh,IMEC(Belgium);R.Lauwereins,IMEC(Belgium)andKatholiekeUniv.Leuven(Belgium);L.J.VanGool,KatholiekeUniv.Leuven(Belgium)andETHZurich(Switzerland);G.Lafruit,IMEC(Belgium)

Inthispaper,wepresentanefficienthardwaredesignofstereomatchingfordepthextraction.Itiscapableofmatchinghigh-definitionimagesatreal-timespeed,producinghigh-qualitydisparitymaps.Thestereoalgorithmcomprisesthreemajorcomponents,i.e.censustransform,dynamicprogrammingandlocal-adaptivevoting.Thecensustransformmakesourmethodrobusttoradiometricdifferences.Thedynamicprogrammingmethodpropagatesinformationalongscanlinesbyimposingasmoothnesscriterion.Thelocal-adaptivevotingfurtherimprovesthematchingquality,utilizingthecorrelationbetweenintensityimagesanddisparitymaps.ThewholealgorithmisprototypedonaFPGAplatform,effectivelyharnessingthepowerofparallelcomputing.Withaworkingfrequencyof65MHz,ourdesignproduceshigh-quality1024x768disparitymapsat60fps.Thankstothehardware-orientedoptimizations,ourdesigniscost-effectiveandsuitabletoavarietyofembeddedapplications.


Interactive floating windows: a new technique for stereoscopic video gamesA.Hogue,B.Kapralos,C.Zerebecki,M.Tawadrous,B.Stanfield,Univ.ofOntarioInstituteofTechnology(Canada)

Thefilmindustryhasalonghistoryofcreatingcompellingexperiencesinstereoscopic3D.Recently,thevideogameasanartisticmediumhasmaturedintoaneffectivewaytotellengagingandimmersivestories.Giventhecurrentpushtobringstereoscopic3Dtechnologyintotheconsumermarketthereisconsiderableinteresttodevelopstereoscopic3Dvideogames.Gamedevelopershavelargelyignoredtheneedtodesigntheirgamesspecificallytousestereoscopictechnologyandhavethusreliedonautomaticconversiontechnology.Gamedevelopersneedtoevaluatesolutionsusedinothermedia,suchasfilm,tocorrectperceptualproblemssuchaswindowviolations,andmodifyorcreatenewsolutionstoworkwithinaninteractiveframework.Weextendthedynamicfloatingwindowtechniqueintotheinteractivedomainenablingtheplayertopositionavirtualwindowinspace.Interactivelychangingtheposition,size,andthe3Drotationofthevirtualwindow,objectscanbemadeto‘breakthemask’dramaticallyenhancingthestereoscopiceffect.Bydemonstratingthatsolutionsfromthefilmindustrycanbeextendedintotheinteractivespace,itisourhopethatthisinitiatesfurtherdiscussioninthegamedevelopmentcommunitytostrengthentheirstory-tellingmechanismsinstereoscopic3Dgames.


Stereoscopic reconfiguration for 3D displaysJ.Houde,P.Jodoin,Univ.deSherbrooke(Canada);F.Deschênes,Univ.duQuébecàRimouski(Canada)

Inthispaper,wepresentamethodtoreconfigure3Dmoviesinordertominimizedistortionwhenseenonadifferentdisplaythantheoneithasbeenconfiguredfor.Bytheirverynature,3Dbroadcastscomewithastereoscopicpairtobeseenbytheleftandrighteyes.However,accordingtoreasonsthatweoughttoexplaininthepaper,thecamerasusedtoshootamoviearecalibratedaccordingtospecificviewingparameterssuchasthescreensize,theviewingdistanceandtheeyeseparation.Asaconsequence,a3Dbroadcastseenonadifferentdisplay(sayahometheateroraPCscreen)thantheoneithasbeenconfiguredfor(sayanIMAX®screen)willsufferfromnoticeabledistortions.Inthispaper,wedescribetherelationshipbetweenthesizeofthe3Ddisplay,thepositionoftheobserver,andtheintrinsicandextrinsicparametersofthecameras.Withthisinformation,weproposeamethodtoreorganizethestereoscopicpairinordertominimizedistortionwhenseenonanarbitrarydisplay.Inadditiontotherawvideopair,ourmethodusestheviewingdistance,aroughestimateofthe3Dscene,andsomebasicinformationonthe3Ddisplay.Aninpaintingtechniqueisusedtofilldisoccludedareas.


Single lens dual-aperture 3D imaging system: color modeling to reduce the difference between two viewpointsS.Y.Bae,JetPropulsionLab.(UnitedStates)andTheUniv.ofSouthernCalifornia(UnitedStates);A.Ream,MontanaStateUniv.(UnitedStates);R.Korniski,M.Shearn,JetPropulsionLab.(UnitedStates);H.Shahinian,SkullBaseInstitute(UnitedStates);H.Manohara,JetPropulsionLab.(UnitedStates)

Inanefforttominiaturizea3Dimagingsystem,wecreatedtwoviewpointsinasingleobjectivelenscamera.ThisworksbyplacingapairofComplementaryMulti-bandBandpassFilters(CMBFs)inonehalfandtheotherhalfoftheaperturearea.TwokeycharacteristicsabouttheCMBFsarethat(1)thepassbandsthatarestaggeredorinterdigitated,soonlyoneviewpointisopenedatatimewhenalightbandmatchedtothepassbandisilluminated,and(2)thepassbandsarepositionedthroughoutthevisiblespectrum,soeachviewpointcanrendercolorbytakingRGBspectralimages.However,becausethepassbandsarestaggered,eachCMBFdoesnotonlycoverthefullvisiblespectrumbutalsoomitsregionsofthevisiblespectrum.Thus,eachviewpointtakesadifferentspectralimagefromtheotherviewpointhenceyieldingadifferentcolorimage.Itraisesconcernformismatchedcolorfromthetwoviewpointsbecauseitcanleadtocolorrivalry,whereviewerscannotresolvethetwodifferentcolorsintooneintheirbrain.ThiscanberesolvedbyusingaChromaticAdaptation

Conference 8288


Transform(CAT),whichmapsthecolorsseenthroughtheCMBFstocolorsthatwouldappearunderD65illumination.Themappingismorecompleteasthenumberofpassbandsincreases.However,thecostandtechniquetofabricatesuchconstrainsthenumberofpassbands.AsimulationwasputtogethertoevaluatethecolordifferencesunderCMBFswithdual-,triple-,andquadruple-bandbandpasses.Inthisreport,resultsofthecolorsimulationwillbeexplainedinCIELabspace.


A stereoscopic movie player with real-time content adaptation to the display geometryS.Duchene,INRIARhône-Alpes(France);M.Lambers,Univ.Siegen(Germany);F.Devernay,INRIARhône-Alpes(France)

3Dshapeperceptioninastereoscopicmoviedependsonseveraldepthcues,includingstereopsis.Foragivenstereoscopiccontent,thestereopsisdepthcuehighlydependsonthecamerasetup,aswellasonthedisplaysizeanddistance.Butsuchproblemscanberesolvedbyperformingviewsynthesistoreducetheseperceiveddepthdistortions.ManymediasuchasTVbroadcasts,blu-raydiscs,andvideo-on-demandarenowavailablein3Dbutwhenthesamestereoscopicmoviehastobedisplayedonadifferentdisplaysizeanddistancethanwhatitwasproducedfor,severedepthdistortionsoreyedivergencemayhappen.Forthisreason,weproposearealtimemethodabletoadaptanystereoscopicmovietoanydisplay,basedonshootingandviewinggeometries,whichcanbeeasilycontrolledbytheviewer.


A simultaneous 2D/3D autostereo workstationD.Chau,B.McGinnis,J.Talandis,J.Leigh,Univ.ofIllinoisatChicago(UnitedStates);T.Peterka,A.Knoll,A.Sumer,M.Papka,J.Jellinek,ArgonneNationalLab.(UnitedStates)

TheuseofImmersiveVirtualRealityEnvironments(IVREs)inscienceisimpededbysteeplearningcurves,highsetupcost,andlackofinteroperablesoftware.WelowerthesebarriersinthisworkwithanewversionofDynallax,adynamicbarrierautostereoworkstation,andDVC,asoftwarelibraryfordrivingDynallaxandcouplingittoothervisualizationapplications.Weevaluateoursystembydeployingitastheprimaryworkenvironmentofamaterialsscientisttosupportherdailyworkflow.

Dynallaxisanautostereoscopicimmersiveworkstationwithhead-trackingandothernovel3Dinteractionsthatallowsscientiststoseamlesslyinteractwithfullresolution2Dcontentsuchaswebbrowsersalongsideautostereo3Dcontentsuchasvolumerenderingsandgeometricmodels.DVCisthesoftwarelibrarythatenablesscientiststoeasilyrendertheirdataonDynallax,evenwithexistingvisualizationtools.

OursolutionaidedscientistsfromtheChemicalSciencesandEngineeringdivisionatArgonneNationalLaboratorytorenderandmodelgeometryfordensityfunctionaltheorycomputationforuseincatalysisresearch.OurresearchdemonstratedthatcombininganIVREwithatraditionaldesktopresultsinahybrid2D/3Dworkstationforscientificdiscovery.


Interactive stereoscopic visualization of large-scale astrophysical simulationsR.Kaehler,SLACNationalAcceleratorLab.(UnitedStates)

ThispresentationdescribestheexperiencesofrenderinghighqualityHDstereoscopicanimationsfromlarge-scaleastrophysicalsimulations.InparticularitpresentstheC++softwareframeworkwedeveloped

andusedforourinteractivestereoscopicvisualizationsaswellastheproductionofanimationsforworld-wideplanetariumshowslike“Life:Acosmicstory”,CaliforniaAcademyofSciences(narratedbyJodieFoster)and“TheBigBang”,AmericanMuseumofNationalHistory(narratedbyLiamNeeson)and“JourneytotheStars”,AmericanMuseumofNationalHistory(narratedbyWhoopiGoldberg).

Wewillgiveseveraldetaileddescriptionsofapplicationexamplesofvisualizationsfromlarge-scalestate-of-the-artsimulations.Inparticularwewillshowshowthestereoscopicvisualizationhelpedthescientisttobetterunderstandandanalyzetheirdata.Theexamplesinclude3Dstereoscopicanimationsofthelarge-scalestructureformationsimulationwiththehighestdynamicrangethatwasevercarriedout,simulationsoftheevolutionofthefirststarsintheuniverse,thehighestresolvedgalaxymergersimulationaswellasanlarge-scalesimulationofthecosmologicalre-ionizationera.


Designing stereoscopic information visualization for 3D-TV: what can we can learn from S3D gaming?J.Schild,M.Masuch,Univ.Duisburg-Essen(Germany)

Thepaperexploresgraphicaldesignandspatialalignmentofvisualinformationelementsintostereoscopiccontent,e.g.captions,subtitles,andespeciallymorecomplexelementsin3D-TVproductions.Themethodusedisadescriptiveanalysisofexistingcomputer-andvideogamesthathavebeenadaptedforstereoscopicdisplayusingsemi-automaticrenderingtechniques(e.g.Nvidia3DVision)orgameswhichhavebeenspecificallydesignedforstereoscopicvision.Digitalgamesoftenfeaturecompellingvisualinterfacesthatcombinehighusabilitywithcreativevisualdesign.Weexplorethesegameinterfacesinstereoscopicvisionregardingtheirstereoscopiccharacteristics,howtheydrawattentionandwheretheyfail.Wepresentanoverviewstructuredaccordingtodifferentlevelsofcontentrelation,rangingfrominformationthatdoesnotreferatalltothescene,tovisualelementswhicharecloselyrelatedtoaparticularpartofthescene.Theresultinglistofdesignconsiderationsisusedtodiscusspossibleconsequences,opportunitiesandchallengesforintegratingvisualinformationelementsinto3D-TVcontent.Thisworkshallhelptoimprovecurrentpost-productioneditingsystemsandidentifiesaneedforfutureeditingsystemsfor3D-TV,e.g.liveeditingandreal-timealignmentofvisualinformationinto3Dfootage.

8288-38, Session 10

Case study: the introduction of stereoscopic games on the Sony PlayStation 3I.H.Bickerstaff,SonyComputerEntertainmentEuropeLtd.(UnitedKingdom)

AfreestereoscopicfirmwareupdateonSonyComputerEntertainment’sPlayStation3consoleprovidesthepotentialtoincreaseenormouslythepopularityofstereoscopic3Dinthehome.Forthistosucceedthough,alargeselectionofcontenthastobecomeavailablethatexploits3Dinthebestwaypossible.

Inadditiontotheexistingchallengesfoundincreating3Dmoviesandtelevisionprogrammes,thestereographymustcompensateforthedynamicandunpredictableenvironmentsfoundingames.Automatically,thesoftwaremustmapthedepthrangeofthesceneintothedisplay’scomfortzone,whileminimisingdepthcompression.

Thispaperpresentsarangeoftechniquesdevelopedtosolvethisproblemandthechallengeofcreatingtwiceasmanyimagesasthe2Dversionwithoutexcessivelycompromisingtheframerateorimagequality.

OverfiftystereoscopicPlayStation3gameshavenowbeenreleasedandnotabletitlesareusedasexamplestoillustratehowthetechniqueshavebeenadaptedfordifferentgamegenres.

Sincethefirmware’sintroductionin2010,theindustryhasmaturedwithalargenumberofdevelopersnowproducingincreasinglysophisticated

Conference 8288


3Dcontent.Newtechnologiessuchasviewerheadtrackingandhead-mounteddisplaysshouldincreasetheappealof3Dinthehomestillfurther.

8288-39, Session 10

Stereoscopic 3D video games and their effects on engagementA.Hogue,B.Kapralos,C.Zerebecki,M.Tawadrous,B.Stanfield,Univ.ofOntarioInstituteOfTechnology(Canada)

Withthefilmindustryembracingstereoscopic3D,affordable3Dcapabletelevisionsarepoisedtoenterthehomemarketenmasse.Whilemanyindustryproponentshavetreated3Dmoviesastheprimarymarket,consumersseevideogamesasthe“killerapplication”of3Dtechnology.Thispresentsanopportunityforgamedeveloperstocreateevenmoreinterestingandimmersiveexperiencesbyintegratingstereoscopic3Dviewingmodesintotheirgamedevelopmentworkflow.Whilestereoscopic3D(S3D)gamedevelopmentisinitsearlystages,itispoisedtohavesignificantimpactontheconsumerentertainmentmarket.AlthoughgamedevelopersunderstandthebasicsofS3Danditsimpactintermsofhardwareperformance,theyarecurrentlyill-equippedtounderstandthemanyfactorsthatS3Dhasontheoveralluserexperience.Currently,thereisnostandardmechanismthatdefines“userexperience”invideogames.Howevertherearemanycontributingfactorsthatonecaninvestigatetodeterminehowusersrespond.Wedescriberesultsofastudytodeterminewhetherstereoscopic3Dviewinginagamesettinghasaneffectongameengagement.Auser-basedexperimentshowstheeffectsstereoscopic3Ddisplayshaveontheuser’slevelofimmersionandflowwithinvideogames.

8288-40, Session 10

Stereoscopic display in a slot machineM.Laakso,Finland’sSlotMachineAssociation(RAY)(Finland)

Thispaperreportstheresultsofausertrialwithaslotmachineequippedwithastereoscopicdisplay.ThemainresearchquestionwastofindoutwhatkindofaddedvaluedoesS-3Dbringtoslotgames?Workwasstartedwithathoroughliteraturesurvey,afterwhichdesignandimplementationofanovelgamingplatformwasstarted.RAY’smulti-gameslotmachine“Nova”wasconvertedto“3DNova”byreplacingthemonitorandconvertingsixoriginalgamestoS-3Dformat.Five3DNovamachineswereputavailableintoRAYarcadesforsixmonths.Bothqualitativeandquantitativeanalysiswascarriedoutfromstatisticalvalues,questionnairesandobservations.Accordingtotheresults,peoplefindtheS-3Dconceptinterestingbutthetechnologyisnotoptimalyet.NostatisticaldifferencewasfoundbetweenNovaand3DNova.Twentysomethingmaleswereexcited,olderpeoplemorecautious.Especiallytheglassesprovideachallenge;ultimatesystemwouldprobablyuseautostereoscopictechnology.Alsothegamesshouldbedesignedtoutilizeitsfullpower.ThemaincontributionsarelessonslearnedfromcreatinganS-3Dslotmachineplatform,novelinformationabouthumanfactorsrelatedtoS-3DgamingandanalysiswhycertaintypeofgamesandeffectssuitbettertoS-3Dgamingthanothers.

8288-42, Session 11

YouDash3D: exploring stereoscopic 3D gaming for 3D movie theatersJ.Schild,Univ.Duisburg-Essen(Germany);S.Seele,HochschuleBonn-Rhein-Sieg(Germany);M.Masuch,Univ.Duisburg-Essen(Germany)

Alongwiththesuccessofthedigitallyrevivedstereoscopiccinema,othereventsbeyond3Dmoviesbecomeattractiveformovietheateroperators,i.e.interactive3Dgames.Inthispaper,wepresentacasethatexplorespossiblechallengesandsolutionsforinteractive3D

gamestobeplayedbyamovietheateraudience.Weanalyzethesettingandshowcasecurrentissuesrelatedtolightingandinteraction.Oursecondfocusistoprovidegameplaymechanicsthatmakespecialuseofstereoscopy,especiallydepth-basedgamedesign.Basedontheseresults,wepresentYouDash3D,agameprototypethatexplorespublicstereoscopicgameplayinareducedkiosksetup.Itfeatureslive3DHDvideostreamofaprofessionalstereocamerarigrenderedinareal-timegamescene.Weusetheeffecttoplacethestereoscopiceffigiesofplayersintothedigitalgame.Thegameshowcaseshowstereoscopicvisioncanprovideforanoveldepth-basedgamemechanic.Projectedtriggerzonesanddistributedclustersoftheaudiencevideoallowforeasyadaptationtolargeraudiencesand3Dmovietheatergaming.

8288-43, Session 11

3D storytelling: a case studyK.Lehto,4thWallProductions(Finland)

Noabstractavailable

8288-44, Session 11

Thinking in z-space: flatness and spatial narrativityR.Zone,The3-DZone(UnitedStates)

NowthatdigitaltechnologyhasaccessedtheZ-spaceincinema,narrativeartistryisataloss.Motionpictureprofessionalsnolongercanreadilyresorttofamiliartools.AnewlanguageandnewlinguisticsforZ-axisstorytellingarenecessary.Afterfirstexaminingtherootsofmonocularthinkinginpainting,priormodesofvisualnarrativeintwo-dimensionalcinemaobviatingtheneedfortruebinocularstereopsiscanthenbeexplored.Particularattentioncanbepaidtomontage,cameramotionanddepthoffieldthroughhistoricexamples.Specialconsiderationcanbegiventothemannerinwhichmonocularcuesfordepthhavebeenexploitedtoinferdepthonaplanarscreen.Boththeartisticpotentialandvisuallimitationsofactualstereoscopicdepthasafilmmakinglanguagemustbeinterrogated.Afteranexaminationofthehistoricbasisofmonocularthinkinginvisualculture,acontextforartisticexplorationoftheuseofthez-axisasaheightenedmeansofcreatingdramaticandemotionalimpactupontheviewercanbeillustrated.

8288-45, Session 12

Floating three-dimensional display with 360-degree viewing angleD.Miyazaki,K.Okoda,Y.Maeda,T.Mukai,OsakaCityUniv.(Japan)

Theaimofthisresearchistodevelopafull-parallaxauto-stereoscopicdisplaysystem,whichcangenerateafloatingthree-dimensional(3-D)imageviewableformthesurroundingarea.A3-Ddisplaymethodbasedonthecombinationofintegralimaging,360-degreescanningwitharotatingmirror,andimagingintheairwithaconcavemirrorisproposed.Ascanningsystemiscomposedofahemisphereconcavemirrorandamirrorscanner,whichislocatedaroundthecenteroftheconcavemirror.Integralimagingisimplementedbyatwo-dimensionaldisplaydeviceandalensarraytoachievefull-parallaxauto-stereoscopicimaging.Byputtinganimagegeneratedbytheintegralimagingsystemintothescanningsystem,afloatingstereoscopicimagecanbegeneratedaroundthecenteroftheconcavemirror.Whenthemirrorscannerrotatesandtheimageontheintegralimagingsystemisswitchedaccordingtothemirrorangle,eachdirectionalimagecanbeobservedfromeachviewingangle.Adigitalmicro-mirrordeviceisusedtoswitchdirectionalimages.Thefeasibilityoftheproposedmethodwasexaminedbypreliminaryexperiments.Theabilitiesofgenerationofafloatingfull-parallaxauto-stereoscopicimageandafloatingauto-stereoscopicimagewith360-degreeviewingangle

Conference 8288


weredemonstrated.

8288-46, Session 12

A novel 3D display system using combined integral imaging and Fresnel hologramJ.Chen,N.Collings,D.Chu,Univ.ofCambridge(UnitedKingdom)

A3Ddisplaysystemwhichcombinesintegralimagingandholographyisproposed.Itiscapableofcombiningmeritsofthesetwotechniquesanddeliveringabetterperformancethanthatbyanyofthemindividually.ItispredictedthattheviewinganglewouldbeimprovedincomparisonwiththatofFresnelhologramonaphase-onlyspatiallightmodulatorandtheimagedepthwouldbeenhancedincomparisonwiththatofconventionalintegralimaging.Byanalysingthepropertiesoftheimagesgeneratedbythismethod,thefeasibilityofthisapproachisconfirmed.Theobtainedexperimentalresultsdemonstrateclearlythetwopredictedbenefits.Theworkherenotonlyshowsthepotentialofthisapproachforfuture3Dimagedisplaysbutalsoprovidesaninsightintotwoimportanttopicsin3Ddisplay:integralimagingandholography.

8288-47, Session 12

Analysis and management of geometric distortions on multi-view displays with only horizontal parallaxA.Said,B.Culbertson,Hewlett-PackardLabs.(UnitedStates)

Manylightfield(multi-view)displaysaredesignedtosupporthorizontalparallaxonly(HPO),sincethisrepresentsalargereductionincomplexity,comparedtofullparallax.ItiscommonlyassumedthattherearenoimportantpracticaldisadvantageonusingHPO,exceptsmalllossesin3Dperceptualquality.Therealityisthat,becausetheymixdifferentperspectiveviewsinthehorizontalandverticaldirections,allHPOdisplayspresentsomeanamorphicdistortions,whichwerefirstobservedinholographicstereograms.Thesedistortionsdependofobserver’sposition,andcanonlybeeliminatedinapre-definedviewingdistance.Inthispaperweextendpreviousworkonthetheoreticalanalysisoftheproblemtocreatetoolsthatcanbeusedbycreatorsofmulti-view3Dcontenttomanagetheproblem,andkeepthedistortionwithinacceptablerangesforallobjectsina3Dscene,andallexpectedviewingposition.WepresentmanyexamplesofsimulatedviewsofHPOdisplays,whichdemonstratehowthedistortionscanaffectvisualappearance,andhowtheyaremanaged.

8288-48, Session 12

Mixed-resolution view synthesis using non-local means refined image mergingT.Richter,M.Schöberl,J.Seiler,T.Tröger,A.Kaup,Friedrich-Alexander-Univ.Erlangen-Nürnberg(Germany)

Synthesizingnovelviewsfromoriginalavailablecameraperspectivesisakeyissueinthe3Dvideodomain.Both,aleft-sideandaright-sidereferenceviewarewarpedintothedesiredintermediatecamerapositionviadepthmaps.Afterwards,thetwowarpedimagesaremergedtosynthesizetheintermediatecameraperspective.Duetodifferentreasonslikewrongdepthvaluesordisocclusion,thereexistsomeareasinsidethesynthesizedviewforwhichthecorrespondinginformationcanonlybefoundinonereferenceview.Ifthetworeferencecamerashavedifferentspatialresolutions,theinsertionofupsampledandhenceblurredregionsfromthelow-resolutionviewproducesannoyingvisibleartifactsinthemergedimage.Weproposearefinementstep,basedonthenon-localmeans(NLM)algorithm,toefficientlyadaptthoseblurredregionstotheirsurroundingandthusincreasethelocalimagesharpness.Duringthisrefinementstep,everypixelinsertedfromthelow-resolutionviewgetsrefinedbycomparingitsneighborhoodwiththeneighborhoodsofpixelswithinaspecific

searcharea.Subsequentlythepixelgetsreplacedbyaweightedsumofthepixelswithinthissearcharea.Theexperimentalresultsshow,thattheNLM-refinedimagemergingachievesaPSNRgainofupto0.9dBandalsoabettersubjectiveimagequalitycomparedtoanunrefinedmixed-resolutionsetup.

8288-49, Session 13

Investigating the cross-compatibility of IR-controlled active shutter glassesA.J.Woods,J.Helliwell,CurtinUniv.(Australia)

ActiveShutterGlasses(alsoknownasLiquidCrystalShutter(LCS)3DglassesorjustShutterGlasses)areacommonlyusedselectiondeviceusedtoviewstereoscopic3Dcontentontime‐sequentialstereoscopicdisplays.

RegrettablymostoftheIRcontrolledactiveshutterglassesreleasedtodatebyvariousmanufacturershaveusedavarietyofdifferentIRcommunicationprotocolswhichmeansthatactiveshutterglassesfromonemanufactureraregenerallynotcross‐compatiblewithanothermanufacturer’semitter.

Thereasonforthelackofcross-compatibilitybetweendifferentmakesofactiveshutterglassesmostlyrelatestodifferencesbetweentheactualIRcommunicationprotocolusedforeachbrandofglasses.

Wehavecharacterisedelevendifferent3DSyncIRcommunicationsprotocolsinordertounderstandthepossibilityofcross-compatibilitybetweendifferentbrandsofglasses.

Thispapercontainsasummaryoftheelevendifferent3DSyncIRprotocolsasusedbyaselectionofemittersandglasses.

Thepaperprovidesadiscussionofthesimilaritiesanddifferencesbetweenthedifferentprotocols,thelimitationsforcreatingacommon3DSyncprotocol,andthepossibilityofdrivingmultiplebrandsofglassesatthesametime.

8288-50, Session 13

New high-brightness interference filter developmentsH.Jorke,A.Simon,InfitecGmbH(Germany)

Anewinterferencefilterdesignwasdevelopedasathreeandfourbandpassdesign.Thisdesignofferswellbalancedcolorsataveryhightransmissionrate.

Glassesbasedstereoscopictechnologiesofferacheap,robustandreliableapproachtoprovide3Dcontentforlargevenues.Besidespolarizationandactiveshuttertechnologies,thewavelengthmultiplexingapproachisanupcomingmajortechnologybranch.

Theexistingtriplebandinterferencefiltersystemrequiredanelectroniccolorcorrectionwhichalsoimpliedareductionofbrightness.Thegoalofthisworkwastheimprovementofbrightnessandcolorsperformanceforstereoscopicinterferencefiltersystems.

Wehavecollectedadatasetofdifferenttypesofprojectorandlampconfigurations.Thevariationsofimagers(LCD,LCoS,1-ChipDLPand3-ChipDLP),oflightsources(highpressuremercuryandxenonbased)anddichroicfiltershavebeenconsideredforaoptimizationforanewstandardinterferencefilterdesign.Amultitudeoffilterdesignsusingthree,fiveormorebandpassfiltershasbeensimulated.Resultsshowthatthe3-4bandfilterdesignexhibitsthehighesttransmissionandwhilemaintainingbalancedcolors.

8288-51, Session 13

Real-world stereoscopic performance in multiple-focal-planes displays: how far apart should the image planes be?S.J.Watt,K.J.MacKenzie,L.C.Ryan,BangorUniv.(UnitedKingdom)

Conference 8288


Conventionalstereoscopicdisplayspresentconflictingstimulitovergenceandaccommodation,causingfatigue,discomfort,andpoorstereodepthperception.Onepromisingsolutionis‘depthfiltering’,inwhichcontinuousvariationsinfocaldistancearesimulatedbydistributingimageintensityacrossmultiplefocalplanes.Therequiredimage-planespacingisacriticalparameter,becausethereareconstraintsonthetotalnumberthatcanbeused.Depth-filteredimageshavebeenshowntosupportcontinuousandreasonablyaccurateaccommodationresponseswith1.1diopter(D)image-planespacings.However,retinalcontrastisincreasinglyattenuatedwithincreasingimage-planeseparation.Thus,whilesuchstimulimayeliminatethevergence-accommodationconflict,theymayalsounacceptablydegradestereoscopicdepthperception.Herewemeasuredstereoacuity,andthetimeneededforstereoscopicfusion,forrealtargetsanddepth-filteredapproximationstothesamestimuli(image-planespacingsof0.6,0.9and1.2D).Stereofusiontimewasreasonablyconsistentacrossconditions.Stereoacuityfordepth-filteredstimuliwasonlyslightlypoorerthanforrealtargetswith0.6Dimage-planeseparation,butdeterioratedrapidlythereafter.Ourresultssuggestthatstereoscopicdepthperception,notaccommodationandvergenceresponses,isthelimitingfactorindeterminingacceptableimage-planespacingfordepth-filteredimages.Wesuggestthatimage-planespacingshouldnotexceed~0.6D.

8288-52, Session 13

An eyeglass-like, eye-tracked, optical see-through, head-mounted display using freeform opticsH.Hua,CollegeofOpticalSciences,TheUniv.ofArizona(UnitedStates)

Inthispaper,wepresentaninnovativeopticalapproachtothedesignofanopticalsee-througheyetrackedhead-mounteddisplay(ET-HMD)systembasedonfreeformopticaltechnologyandaninnovativeopticalschemethatuniquelycombinesthedisplayopticswiththeeyeimagingoptics.Ourapproachallowscreatingalightweight,compact,androbustET-HMDsolutionwithanon-obtrusiveformfactor.Theopticaldesignofthesee-throughHMDpathisbasedonafreeformwedge-shapedprismcementedwithafreeformlens.Thewedge-shapedprismfreeformprism,composedofthreenon-rotationallysymmetricfreeformsurfaces,servesasanear-eyeimagingsystemthatmagnifiestheimagedisplayedonamircodisplay.Toenablelow-distortionsee-throughcapability,theattachedfreeformlensprovidescorrectiontothedistortionintroducedbythefreeformprismtothesee-throughviewofareal-worldscene.Thedimensionsforthecementedprismandlensareabout30mmby30mmby15mmwithaweightofabout15grams.Thesamefreeformprismalongwithonlyoneadditionalopticalelementservestheeyeimagingfunctionforavideo-basedeyetrackingsystem.ApreliminarydesignofthedescribedET-HMDsystemwillbepresented.

8288-53, Session 13

Optimization of a multi-view system based on pulsed LED-LCD projectorsA.Pross,R.Blach,M.Bues,R.Reichel,O.Stefani,Fraunhofer-InstitutfürArbeitswirtschaftundOrganisation(Germany)

Recentadvancementinstereoscopicdisplaysand3Dprojectiontechnologieshasbeenshown,drivenbythecurrentdisseminationof3Dtechnologiesatcinemasandconsumerproducts.BuildingVirtualRealitysystemsoftheseproductswithheadtrackingandaperspective-correctviewprovideonlyasingleuserview.Otherviewerssharethesameimagefromadifferentposition.Providingperspective-correctstereoscopicviewsformultiple,individuallytrackedusersonasharedsurfacearenecessary.Wepresentabrightnessandcrosstalkoptimizedmulti-viewstereodisplaybasedonpulsedLEDlightsourcesofasetofmultipleLCDprojectors.PulsedLEDprojectorsallowhighfrequencyswitchingbetweenthedisplayedimages,almosteliminatingamajorsourceofcrosstalkbetweentheindividualviews.Weevaluatedthepowerconsumption,projectionbrightnessandcrosstalkofour

multi-viewsystemandpresentasolutionwhichalleviatestheinherentproblemsofshutter-basedmulti-viewsystems,whicharelimitedenergyinefficiency,brightnessdegradationduetotheprojectorshutter,andrelativelyhighcrosstalk,whichmoreoverincreaseswiththenumberofviews.

Conference 8288


Conference 8289 · The Engineering Reality of Virtual Reality 2012Tuesday24January2012

Part of Proceedings of SPIE Vol. 8289 The Engineering Reality of Virtual Reality 2012

8289-01, Session 1

Foreign language learning in immersive virtual environmentsB.Chang,L.Sheldon,M.Si,RensselaerPolytechnicInstitute(UnitedStates)

Virtualrealityhaslongbeenusedfortrainingsimulationsinfieldsfrommedicinetoweldingtovehicularoperation,butsimulationsinvolvingmorecomplexcognitiveskillspresentnewdesignchallenges.Foreignlanguagelearning,forexample,isincreasinglyvitalintheglobaleconomy,butcomputer-assistededucationisstillinitsearlystages.Immersivevirtualrealityisapromisingavenueforlanguagelearningasawayofdyamicallycreatingbelievablescenesforconversationaltrainingandrole-playsimulation.Visualimmersionalone,however,onlyprovidesastartingpoint.Wesuggestthattheadditionofsocialinteractionsandmotivatedengagementthroughnarrativegameplaycanleadtotrulyeffectivelanguagelearninginvirtualenvironments.Inthispaper,wedescribethedevelopmentofanovelapplicationforteachingMandarinusingCAVE-likeVR,physicalprops,humanactorsandintelligentvirtualagents,allwithinasemester-longmultiplayermysterygame.Studentstravel(virtually)toChinaonaclassfieldtrip,whichsoonbecomescomplicatedwithintrigueandmysterysurroundingthelostmanuscriptofanearlyChineseliteraryclassic.VirtualrealityenvironentssuchastheForbiddenCityandaBeijingteahouseprovidethesettingforlearninglanguage,culturaltraditions,andsocialcustoms,aswellasthediscoveryofcluesthroughconversationinMandarinwithcharactersinthegame.

8289-02, Session 1

Virtual reality technology prevents accidents in extreme situationsY.Badihi,JerusalemCollegeofTechnology(Israel);M.Reiff,JerusalemCollegeofTechnology(Korea,Republicof)

ASummary:

1.Technologygoals&hypotheses:

a.Creatingatrainingenvironmentforacquisitionofearlycuesinadynamic“WorldPicture”forexpecteddrivingsituations,especiallyextremesituations.

b.Creatingtrainingconditionsthatenablegreaterlongevityofskillsacquired.

2.BasicFindings:

a.Themore“immersion”featuresareincorporatedintoVRextreme-situationstraining,thefasterandthebetterthereactionsofthetraineddriver.

b.Themore“immersion”featuresareincorporatedintoVRextreme-situationstraining,thegreaterthelongevityofskillsacquiredintraining.

3.Methods&researchtools:

a.CreationofatrainingfacilityusingVRtechnology,basedonaVRcomputerizedmodel,incorporatinginteractiveactivity.The“worldpicture”display(drivingbehaviorwithintheVR)isreceivedthroughtangiblemethods.

b.Buildingvariedlevelsofimmersion:goingfromthebasicsimulationsystem,withalowlevelofimmersion,totherichestsimulationexperiencewithdeepimmersion,includinghigh-levelaudiofeatures.

c.Buildingascenariobankwithvariedlevelsofcomplexity.Thescenariosrepresentvarioushighworkloadandextremesituationswhichcharacterizethetrainee’sworldpicture.Thesubjectsampleincludesapproximately70randomlychosendriversforthevarioustrainingprofiles.

4.Scientificandpracticalcontributions:

a.Thisresearchfocusesoninstanceswherethedriverneedstoactquickly,absorbmoreinformation,andmakeanexactdecisionunderpressure.

b.ThetrainingexperiencewithVR,asexpressedin“immersion”,isatraumaticexperiencemeanttoleavealastingimpression.

8289-03, Session 1

Motion parallax in immersive cylindrical display systemsN.Filliard,G.Reymond,RenaultTechnocentre(France);A.Kemeny,RenaultTechnocentre(France)andArtsetMétiersParisTech(France);A.Berthoz,CollègedeFrance(France)

Motionparallaxisacrucialvisualcueproducedbytranslationsoftheobserverfortheperceptionofdepthandself-motion.Therefore,trackingtheobserverviewpointhasbecomeinevitableinimmersivevirtual(VR)realitysystemsusede.g.inautomotiveindustryorinscientificstudiesofvisualperception.

Theperceptionofastableandrigidworldrequiresthatthisvisualcuebecoherentwithotherextra-retinal(e.g.vestibular,kinesthetic)cuessignalingego-motion.Althoughworldstabilityisneverquestionedinrealworld,renderingheadcoupledviewpointinVRcanleadtotheperceptionofanillusoryperceptionofunstableenvironments,unlessanon-unityscalefactorisappliedonrecordedheadmovements.Besides,cylindricalscreensareusuallyusedwithstaticobserversduetoimagedistortionswhenrenderingimageforviewpointsdifferentfromasweetspot.

Wedevelopedatechniquetocompensateinreal-timethesenon-linearvisualdistortions,inanindustrialVRsetup,basedonacylindricalscreenprojectionsystem.

Additionally,a“motionparallaxgain”betweenthevelocityoftheobserver’sheadandthatofthevirtualcamerawasintroducedinthissystem.Resultsindicatethat,belowunity,gainssignificantlyalterposturalcontrol.Conversely,theinfluenceofhighergainsremainslimited,suggestingacertaintoleranceofobserverstotheseconditions.

8289-04, Session 2

Meta!Blast computer game: a pipeline from science to 3D art to educationW.Schneller,P.J.Campbell,E.S.Wurtele,D.Bassham,IowaStateUniv.(UnitedStates)

Meta!Blast(http://www.metablast.org)isdesignedtoaddressthechallengesstudentsmayencounterinunderstandingthespatialandtimesensitivecomponentsofthemechanismsofbioenergy.Meta!Blast,developedbyfacultyandstudentsinbiology,biochemistry,computerscience,gamedesign,pedagogy,artandstory,isbeingmodeled/createdinMAYAandaUnitygameengine,forMACs/PCsinclassrooms;ithasalsobeenexhibitedinanimmersiveenvironment.

Here,wedescribethepipelinefromproteinstructuraldataandholographicinformationtoarttothe3Denvironmenttothegameengine,bywhichweprovideapublicly-availableinteractivethree-dimensionalcellularworldthatmimicsaphotosyntheticplantcell.


8289-05, Session 2

Learning immersion without getting wetJ.C.Aguilera,AdlerPlanetarium&AstronomyMuseum(UnitedStates)

ThispaperdescribestheteachingofanimmersiveenvironmentsclassontheSpringof2011.Theclasshadstudentsfromundergraduateaswellasgraduateartrelatedmajors.Theirdigitalbackgroundandinterestswerealsodiverse.Thesevariableswerechanneledasdifferentapproachesthroughoutthesemester.Classcomponentsincludedfundamentalsofstereoscopiccomputergraphicstoexplorespatialdepth,3Dmodelingandskeletonanimationtoexplorepresence,exposuretoformatslikeastereoprojectionwallanddomeenvironmentstocomparefieldofviewacrossdevices,andfinally,interactionandtrackingtoexploreissuesofembodiment.Allthesecomponentsweresupportedbytheoreticalreadingsdiscussedinclass.GuestsartistspresentedtheirworkinVirtualReality,DomeEnvironmentsandotherimmersiveformats.Museumprofessionalsalsointroducedstudentstospacesciencevisualizationswhichutilizeimmersiveformats.Wewillpresenttheassignmentsandtheiroutcome,togetherwithinsightsastohowthecreationofimmersiveenvironmentscanbelearnedthroughconstraintsthatexposestudentstosituationsofembodiedcognition.

8289-06, Session 2

Byte meD.Gromala,M.Song,A.Zeffiro,SimonFraserUniv.(Canada)

WithinthelastfewyearstherehasbeenafeministturnwithinHCI(Human-ComputerInteraction).In2010,InteractingwithComputersissuedacallforaspecialissueonfeminismandHCI,apaperonfeminismandHCIwonBestPaperatCHI2010and,atCHI2011,afeministinteractivedesignworkshopwasheld.Theseeventsdemonstratethemannerinwhichfeminismcontinuestoinfluencenumerousdisciplines,fromhuman-roboticinteraction(HRI)andCSCW(Computer-SupportedCooperativeWork)toHCIandSocialMedia.Indeed,justaswomenwereamongthefirstdirectorsandproducersinthenascentfilmindustry,womenhaveandcontinuetoproducewithinandaroundVR.

WhatthenisafeministperspectivewithinVR?Inaddressingthisquestion,weexploreadiverserangeofvirtualenvironmentsandtheassumptionsunderlyingtheirdesign,examiningandarticulatingfactorsthatareallbutinvisible.WealsoexaminethemethodologicalandtheoreticalframeworksthatundergirdVRdevelopmentandpractice,andtheirpracticalconsequencesinthedesignandtestingofvirtualenvironments(VEs),especiallythosethatostensiblydrawstrictdichotomiesbetweenhumanandmachine,andattributequasi-autonomytotechnologicalsystems.

8289-07, Session 2

Female artists and the VR crucible: expanding the aesthetic vocabularyJ.F.Morie,TheUniv.ofSouthernCalifornia(UnitedStates)

Asurveydonein2007ofVRArtworksshowedthatwomenhavecreatedthemajorityoftheseimmersiveworks.Whilethisseemscountertopopularideasthatthefieldisdominatedbymen,itseemsrather,thatthetrulyuniqueworksappeartoemergefromafeminineapproach.Suchanapproachseemswellsuitedtoimmersiveenvironmentsasitincorporatesaspectsofinclusion,wholeness,andablendingofthebodyandthespirit.Femaleattentiontoholisticconcernsfitsthegestaltapproachneededtocreateinafullyfunctionalyetopen-endedvirtualworld.Itfocusesnotsomuchonproducingafinishedobject(likeatextorasculpture)butratheroncreatingapossibilityforbecoming,likebringingachildintotheworld.ImmersiveVEsarenotobjectiveworksofarttobehungonawallandcritiqued.Theyarevehiclesforexperience,vesselstolivewithinforapieceoftime.

8289-08, Session 3

Markerless 3D facial motion capture systemY.Hwang,J.Kim,W.Bang,J.D.Kim,C.Kim,SamsungAdvancedInstituteofTechnology(Korea,Republicof)

Weproposeanovelmarkerless3Dfacialmotioncapturesystemusingonlyonecommoncamera.Thissystemissimpleandeasytotransferfacialexpressionsofauser’sintovirtualworld.Ithasrobustlytrackingfacialfeaturepointsassociatedwithheadmovements.Inaddition,itestimateshighaccurate3Dpoints’locations.Wedesignednovelapproachestothefollowings;Firstly,forprecisely3Dheadmotiontracking,weapplied3Dconstraintsusinga3Dfacemodelonconventional2Dfeaturepointstrackingapproach,calledActiveAppearanceModel(AAM).Secondly,fordealingwithvariousexpressionsofauser’s,wedesigned2D/3Dfacegenericmodelsfromaround5000imagesdataand3Dshapedataincludingsymmetricandasymmetricfacialexpressions.Lastly,foraccuratelyfacialexpressioncloning,weinventedamanifoldspacetosuccessfullytransfer2Dlowdimensionalfeaturepointsto3Dhighdimensionalpoints.Themanifoldspaceisdefinedbyelevenfacialexpressionbases.Consequently,oursystemhas4.8mmofaverageerroron5186testimageswhichareincludingvariouspersons,variousfacialexpressionsassociatedwithheadmovements,variousillumination,etc.

8289-09, Session 3

Capturing geometry in real-time using a tracked Microsoft KinectD.Tenedorio,M.Fecho,J.Schwartzhaupt,R.Pardridge,J.Lue,J.P.Schulze,Univ.ofCalifornia,SanDiego(UnitedStates)

WeinvestigatethesuitabilityoftheMicrosoftKinectdeviceforcapturingreal-worldobjectsandplaces.Ournewgeometryscanningsystempermitstheusertoobtaindetailedtrianglemodelsofnon-movingobjectswithatrackedKinect.ThesystemgeneratesatexturemapforthetrianglemeshusingvideoframesfromtheKinect’scolorcameraanddisplaysacontinually-updatedpreviewofthetexturedmodelinreal-time,allowingtheusertore-scanthescenefromanydirectiontofillholesorincreasethetextureresolution.Wealsopresentfilteringmethodstomaintainahigh-qualitymodelofreasonablesizebyremovingoverlappingorlow-precisionrangescans.Ourapproachworkswellinthepresenceofdegenerategeometryorwhenclosingloopsaboutthescannedsubject.Wedemonstratetheabilityofoursystemtoacquire3DmodelsathumanscalewithaprototypeimplementationintheStarCAVE,avirtualrealityenvironmentattheUniversityofCalifornia,SanDiego.Wedesignedthecapturingalgorithmtosupportthescanningoflargeareas,providedthataccuratetrackingisavailable.

8289-10, Session 3

An example-based face relightingH.Shim,SamsungAdvancedInstituteofTechnology(Korea,Republicof)

Weproposeanewfacerelightingalgorithmpoweredbyalargedatabaseoffaceimagescapturedundervariousknownlightingconditions(i.g.aMulti-PIEdatabase).Themotivationofouralgorithmisthatafacecanberepresentedbytheassembleofpatchesfrommanyotherfaces.Thealgorithmfindsthemostsimilarfacepatchesinthedatabaseintermsofthelightingandtheappearance.Byassemblingthematchedpatches,wecanvisualizetheinputfaceundervariouslightingconditions.Unlikeexistingfacerelightingalgorithms,weneitheruseanykindsoffacemodelnormakeaphysicalassumption.Instead,ouralgorithmisadata-drivenapproach,synthesizingtheappearanceoftheimagepatchusingtheappearanceoftheexamplepatch.Usingadata-drivenapproach,wecanaccountforvariousintrinsicfacialfeaturesincludingthenon-Lambertianskinpropertiesaswellasthehair.Also,ouralgorithmisinsensitivetothefacemisalignment.Wedemonstratetheperformanceofouralgorithmbyfacerelightingandfacerecognitionexperiments.Especially,thesynthesizedresultsshow

Conference 8289


thattheproposedalgorithmcansuccessfullyhandlevariousintrinsicfeaturesofaninputface.Also,fromthefacerecognitionexperiment,weshowthatourmethodiscomparabletothemostrecentfacerelightingwork.

8289-11, Session 3

Heteronyms and avatars: a self-reflective system for artistic activityE.E.Ayiter,SabanciUniv.(Turkey)

Thistextwillexaminehowavatarsandthesociallyinteractive,onlinevirtualrealitiesinwhichtheybecomeembodiedmayleadtoanunderstandingregardingidentityandself-perception,howsuchshiftsinawarenessmayrelatetothenotionoftheundividedlyholistic‘self’andwhethersuchperceptualshiftsmaybeinstrumentalinbringingforthnovelexperientialstatesofartisticactivity.Thebasisofthisstudyisaself-observationalsocialsystem,ofwhichtheagentsareacoterieofavatarsofbothsexes,endowedwithdistinctphysicalattributes,bothhumanandnon-human;withuniquelyemergentpersonalitieswhichhaveprogressedtowardsfurtheridiosyncrasyoveraperiodofthreeyears.I,theircreatoramalsotheobserveroftheirundertakings,theirinteractions,andtheircreativeoutput,allofwhichmanifestasdisparatefacetsofmyownpersonaandmyartisticactivity.

8289-12, Session 4

ManifestAR: an augmented reality manifestoJ.C.Freeman,EmersonCollege(UnitedStates)

ManifestARisaninternationalartists’collectiveworkingwithemergentformsofaugmentedrealityasinterventionistpublicart.Thegroupseesthismediumasawayoftransformingpublicspaceandinstitutionsbyinstallingvirtualobjects,whichrespondtoandoverlaytheconfigurationoflocatedphysicalmeaning.ThispaperwilldescribetheManifestARvision,whichisoutlinedinthegroupsmanifesto.

8289-13, Session 4

Invisible waves and hidden realms: augmented reality and experimental artS.P.Ruzanka,RensselaerPolytechnicInstitute(UnitedStates)

Augmentedrealityiswayofbothalteringthevisibleandrevealingtheinvisible.Itoffersnewopportunitiesforartisticexplorationthroughvirtualinterventionsinrealspace.Inthispaper,theauthordescribestheimplementationoftwoartinstallationsusingdifferentARtechnologies,oneusingopticalmarkertrackingonmobiledevicesandoneintegratingstereoscopicprojectionsintothephysicalenvironment.Thefirstartwork,DeOndasyAbejas(TheWavesandtheBees),isbasedonthewidelypublicized(butunproven)hypothesisofalinkbetweencellphoneradiationandthephenomenonofbeecolonycollapsedisorder.UsinganAndroidtablet,viewerssearchoutsmallfiducialmarkersintheshapeofelectromagneticwaveshiddenthroughoutthegallery,whichrevealswarmsofbeesscatteredonthefloor.Thepiecealsocreatesagenerativesoundscapebasedonelectromagneticfields.Thesecondartwork,UrbanFauna,isaseriesofanimationsinwhichfeaturesoftheurbanlandscapebecomeplantsandanimals.Surveillancecamerasbecomeflocksofbirdswhileminiaturecellphonetowers,lampposts,andtelephonepolesgrowlikesmallseedlingsintime-lapseanimation.Theanimationsarepresentedassmallstereoscopicprojections,integratedintothephysicalspaceofthegallery.Thesetwopiecesexploretherelationshipbetweennatureandtechnologythroughthevisualizationofinvisibleforcesandhiddenalternaterealities.

8289-14, Session 4

Immersive realities: articulating the shift from VR to AR through artistic practiceT.Cornish,T.Margolis,Univ.ofCalifornia,SanDiego(UnitedStates)

OurcontemporaryimaginingsoftechnologicalengagementwithspacehastransitionedfromflyingthroughVirtualRealityenvironmentstomobileinteractionswiththephysicalworldthroughpersonalmediadevices.SocialmediaexperienceswithinphysicalenvironmentsarenowbeingpreferencedoverisolatedenvironmentssuchasCAVEsorHMDs.Examplesofthistrendcanbeseeninearlytele-collaborativeartworkswhichstrovetouseadvancednetworkingtojoinmultipleparticipantsinsharedvirtualenvironments.Gameconsolesandpowerfuldesktopgraphicsbroughthighlyinteractivecontentintoourhomewhilesacrificingvisualandauditoryimmersion.RecentdevelopmentsinmobileARallowuntetheredaccesstosuchsharedrealitiesinplacesfarremovedfromlabsandhomeentertainmentenvironments,andwithoutthebulkyandexpensivetechnologiesattachedtoourbodiesthataccompanymostVR.ThispaperaddressestheemergingtrendfavoringsociallyimmersiveartworksviamobileAugmentedRealityratherthansensoriallyimmersiveVirtualRealityinstallations.

8289-15, Session 4

MetaTree: augmented reality narrative explorations of urban forestsR.G.West,T.Margolis,Univ.ofCalifornia,SanDiego(UnitedStates);J.O’Neil-Dunne,TheUniv.ofVermont(UnitedStates);E.Mendelowitz,SmithCollege(UnitedStates)

Asmajorcitiesworld-wideadoptandimplementreforestationinitiativestoplantmillionsoftreesinurbanareas,theyareengaginginwhatisessentiallyamassiveecologicalandsocialexperiment.Existingair-borne,space-borneandfield-basedimagingandanalysismechanismsfailtoprovidekeyinformationonurbantreeecologycrucialtoinformingmanagement,policy,andsupportingcitizeninitiativesforplantingandstewardshipoftrees.Duetolackofspatio-temporalresolution,poorvantagepoint,costconstraintsandbiologicalmetriclimitations,theyareineffectiveasreal-timeinventoryandmonitoringtools.Novelmethodsforimagingandmonitoringthestatusoftheseemergingurbanforestsandencouragingtheirongoingstewardshipbythepublicarerequiredtoensuretheirsuccess.Ourart-sciencecollaborationproposestore-envisionourrelationshipwithurbanspacesbyforegroundingurbantreesinrelationtolocalarchitecturalfeaturesandsimultaneouslycreatingnewmethodsforurbanforestmonitoring.Weexplorecreatingashiftfromoverheadimagingorfield-basedtreesurveydataacquisitionmethodstocontinuous,ongoingmonitoringaspartofamobileaugmentedrealityexperience.Weconsiderthepossibilitiesofthisexperienceasamediumforinteractingwithandvisualizingurbanforestrydata,andforcreatingculturalengagementwithurbanecology.


Prediction of visually perceived location using reaching action and effect of reaching distance on itM.Suzuki,K.Takazawa,K.Uehira,KanagawaInstituteofTechnology(Japan)

Weexaminetheeffectofthereachingdistanceonthepredictionofthevisuallyperceivedlocationusingthereachingaction.Fordirectinteractionbetweenanobserver’sbodyandavirtualobject,thesystempresentingthevirtualobjectmustexecutetheprocessoftheinteractionwhenthebodyisjustonthevisuallyperceivedlocationofthevirtualobject.Incontemporarytechniques,itisassumedthatthevisuallyperceivedlocationisthesameasthelocationdefinedby

Conference 8289


binoculardisparity.However,bothlocationsareoftendifferent.Inourpreviousstudies,weproposedanewtechniqueforthepredictionofthevisuallyperceivedlocationusinganobserver’saction.Also,wedemonstratedthepredictionusingtheactionthatanobserverreachesoutforavirtualobject.Thisstudyisanexaminationontheapplicablerangeofourtechnique.Intheexperiment,anobserverreachesoutforavirtualobject,andthereachingdistanceistheexperimentalvariable.Theresultsdonotsupporttheeffectofthereachingdistanceontheprediction.Wedemonstratethatourtechniqueisapplicabletoawiderangeofreachingdistance.


An innovative multimodal virtual platform for communication with devices in a natural wayC.R.Kinkar,R.Golash,A.R.Upadhyay,SagarInstituteofResearch,Technology&Science(India)

Astechnologygrowspeoplearedivertedandaremoreinterestedincommunicatingwithmachineorcomputernaturally.Thiswillmakemachinemorecompactandportablebyavoidingremote,keyboardetc,alsoitwillhelpthemtoliveinanenvironmentfreefromelectromagneticwaves.Thisthoughthasmade‘Recognitionofnaturalmodalityinhumancomputerinteraction’amostappealingandpromisingresearchfield.Simultaneouslyithasbeenobservedthatusingsinglemodeofinteractionlimitthecompleteutilizationofcommandsaswellasdataflow.Inthispaperamultimodalplatform,whereoutofmanynaturalmodalitieslikeeyegaze,speech,voice,faceetc.humangesturesarecombinedwithhumanvoiceisproposedwhichwillminimizethemeansquareerror.Thiswillloosenthestrictenvironmentneededforaccurateandrobustinteractionwhileusingsinglemode.GesturecomplementSpeech,gesturesareidealfordirectobjectmanipulationandnaturallanguageisusedfordescriptivetasks.Humancomputerinteractionbasicallyrequirestwobroadsectionsrecognitionandinterpretation.Recognitionandinterpretationofnaturalmodalityincomplexbinaryinstructionisatoughtaskasitintegraterealworldtovirtualenvironment.Themainideaofthepaperistodevelopaefficientmodelfordatafusioncomingfromheterogeneoussensors,cameraandmicrophone.Throughthispaperwehaveanalyzedthattheefficiencyisincreasedifheterogeneousdata(image&voice)iscombinedatfeaturelevelusingartificialintelligence.Thelongtermgoalofthispaperistodesignarobustsystemforphysicallynotableorhavinglesstechnicalknowledge


Composite lay-up process with application of elements of augmented realityJ.Novak-Marcincin,J.Barna,M.Janak,L.Novakova-Marcincinova,V.Fecova,TechnicalUniv.ofKosice(Slovakia)

Articleinvestigatestheapplicationpossibilitiesofelementsofaugmentedrealityinareaofcompositelay-uptechnologywithfocustoutilizationofprinciplesofopensourcephilosophy.Itdescribesthesoftwareandhardwaretoolsandaspectsofdevelopmentofenvironmentconsistingofcombinationofrealandvirtualitems.Indetailitdescribestheprocessofexploitationandcreationworkingenvironmentwithuseofaugmentedreality.Allthetheoreticalaspectsareexplainedanddemonstratedonrealexampleofcompositelay-upprocesswithuseofspecialdeviceforpositioningofworkingtableandtotalcontrollingoftheprocess.Connectionsandrelationsarealsodescribedbetweenhardwareandsoftwareelementsofthesolution(sensors,logicalloops,scripting).Describedapplicationprovidesitsuserwithvisualdemonstrationandguidanceofentirelay-upprocessbydisplayingthefinalpositionsandboundariesofindividualcompositelayers.Finalpartofthearticleinformsaboutapplicationpossibilitiesofnewsolutionandapproachesfurtherimprovementsandupdatesofproposedapplication.

8289-16, Session 5

Reordering virtual reality: recording and recreating real-time experiencesM.Dolinsky,W.R.Sherman,C.Eller,E.A.Wernert,Y.Chi,IndianaUniv.(UnitedStates)

Withtheproliferationoftechnologicaldevicesandartisticstrategies,thereisanurgentandjustifiableneedtocapturesite-specifictime-basedvirtualrealityexperiences.Thesearteventsaredependentontheorchestrationofmultipleinputsincludinghardware,software,sitespecificlocation,userinputsand3Dstereoandsensoryinteractions.Althoughaphotographorvideomayillustrateaparticularcomponentofthework,suchasanillustrationoftheartworkorasampleofthesound,theyonlyrepresentafractionoftheoverallexperience.Thispaperseekstodiscussdocumentationstrategiesthatcombinesmultipleapproachesandcapturestheinteractionsbetweenartprojection,acting,stagedesign,sightmovement,dialogueandaudiodesign.

8289-18, Session 5

The cognitive implications of virtual locomotion with a restricted field of viewW.E.Marsh,J.W.Kelly,V.J.Dark,J.H.Oliver,IowaStateUniv.(UnitedStates)

Virtualreality(VR)systemsdonotsupportfullynaturallocomotionthroughaninfinitevirtualenvironment.Whilethisispartlyduetoconstrainedphysicalboundariesandtrackingranges,VRsystemsalsoprovidelow-fidelitysensoryfeedback,particularlyvisual.Visualfeedback,suchasopticflow,isusednaturallytoguidelocomotionthroughtherealworld.Inthepresenceofrestrictedvisualfeedback,usersmayneedtoemploycognitivelydemandingstrategiestoguidelocomotion.Thismayleadtocompetitionwithotherconcurrenttasksthatauserwishestoperformintheenvironment.

Resultsshowedthatusershadmoretroublerememberingspatialorverbalitemswhenconcurrentlyperforminglocomotionwithalowfield-of-view(FOV)thanwithahighFOV.ThismemorydecrementmayindicatethatperforminglocomotionmovementswitharestrictedFOVrequiresmoregeneralcognitiveresourcesthanareneededwithahighFOV.Movementstartandcompletiontimeswerealsomeasuredinordertoexamineresourcerequirementsofspecificaspectsofindividualmovements.Themovementdatasuggestthatthestarting/planningphaseofmovementrequiresmoregeneralcognitiveresourceswhenthereisarestrictedFOV.Understandingthespecificperformanceproblemsthatresultfromconcurrenttaskperformancemayprovideinsightforthedesignoffuturesystems.

8289-19, Session 6

Use of a simulator based on virtual reality to assess the application of economic driving techniques by truck driversT.N.Bogoni,Univ.EstadualdoMatoGrossoSul(Brazil);M.S.Pinho,PUCRS-IPCT(Brazil)

ThispaperpresentstheprojectandthedevelopmentofaprototypeforatrucksimulatoraimedatassessingtheuseofEconomicDrivingTechniques.Wedescribethetechniquesforeconomicdrivingandthewaytheyaremonitored,aswellastheprocessofmodelingandcreatingavirtualenvironmentandtheinteractiondevicesusedinthesimulator.Thesimulationisaccomplishedbyusingavirtualdesktopenvironmentwithhardwareconfigurationsimilartothatinarealtruck,inordertoprovidetheuserwithahigherlevelofimmersion.Bytestingdriversandeconomicdrivinginstructors,itwaspossibletoobservethattheprototypecanbeusedasatoolforassessingdriversandthatthesystemisabletoperceiveagreatpartoftheviolationsintheuseofEconomicDrivingTechniquesduringthesimulationsimilarlytoahumanexpert.

Conference 8289


8289-20, Session 6

LVC interaction within a mixed-reality training systemB.B.Pollock,IowaStateUniv.(UnitedStates)andVirtualRealityApplicationCtr.(UnitedStates);E.Winer,S.Gilbert,IowaStateUniv.(UnitedStates);J.delaCruz,U.S.ArmyRDECOM/STTC(UnitedStates)

TheUnitedStatesmilitaryisincreasinglypursuingadvancedlive,virtual,andconstructive(LVC)trainingsystemsforreducedcost,greatertrainingflexibility,anddecreasedtrainingtimes.Combiningtheadvantagesofrealistictrainingenvironmentsandvirtualworlds,mixedrealityLVCtrainingsystemscanenableliveandvirtualtraineeinteractionasifco-located.Thispaperdiscussesasystemthatovercomestracking,latency,andhardwarechallengestoempowerLVCinteractioninareconfigurable,mixedrealityenvironment.

IowaStateUniversity(ISU)developedelectronicsaugmentreal-timetrackingdataofphysicaltrainees,props,andvehiclestoupdatevirtualavatars.VirtualBattlespace2(VBS2)isutilizedasascenarioauthoringgameengine,providingafirst-person-shooterinterfaceforvirtualtraineesandbehaviorsforconstructiveentities.Traineesadditionallyinteractwiththesystemthroughahigh-resolutionsixsidedCAVEcommandingsemi-autonomousunmannedaerialvehiclesandthroughtrackedteleoperatedvehiclesatotherphysicallocationsatISU.

Thissystemwasdevelopedandtestedinanimmersive,reconfigurable,andmixedrealityLVCtrainingsystemforthedismountedwarfighteratISUknownastheVeldt.TheVeldtachievesrareLVCinteractionwithinmultiplephysicalandvirtualimmersiveenvironmentsfortraininginreal-timeacrossmanydistributedsystems.

8289-21, Session 6

Immersive volume rendering of blood vesselsG.Long,H.S.Kim,A.Marsden,Y.Bazilevs,J.Schulze,Univ.ofCalifornia,SanDiego(UnitedStates)

Inthispaper,wepresentanovelmethodofvisualizingflowinbloodvessels.Ourapproachreadsunstructuredtetrahedraldata,resamplesit,andusesslicebased3Dtexturevolumerendering.Duetothesparsestructureofbloodvessels,weutilizeanoctreetoefficientlystoretheresampleddatabydiscardingemptyregionsofthevolume.Weuseanimationtoconveytimeseriesdata,wireframesurfacetogivestructure,andutilizetheStarCAVE,a3Dvirtualrealityenvironment,toaddafullyimmersiveelementtothevisualization.

Ourtoolhasgreatvalueininterdisciplinarywork,helpingscientistscollaboratewithclinicians,byimprovingtheunderstandingofbloodflowsimulations.Fullimmersionintheflowfieldallowsforamoreintuitiveunderstandingoftheflowphenomena,andcanbeagreathelptomedicalexpertsfortreatmentplanning.

8289-22, Session 6

A virtual reality interface for pre-planning of surgical operations based on a customized model of the patientM.Witkowski,R.Sitnik,J.Lenar,WarsawUniv.ofTechnology(Poland)

Wepresentahuman-computerinterfacethatenablestheoperatortoperformavirtualsurgicalprocedureonthemusculo-skeletal(M-S)modelofthepatient’slowerlimbs,sendthemodifiedmodeltothebio-mechanicalanalysismodule,andexportthescenario’sparameterstothesurgicalnavigationsystem.Theinterfaceprovidestheoperatorwithtoolsfor:bonecutting,manipulatingandremoval,repositioningmuscleinsertionpointsandplacingimplants.

Theoperatormayexportsurgeryparametersforbio-mechanicalanalysisofitsfunctionaloutcome.Iftheresultissatisfactorythe

scenariodatamaybeusedduringtheactualsurgery.

Thehardwareconfigurationsproposedtobeusedwiththeinterfaceare:(a)astandardcomputerkeyboardandmouse,anda2-Ddisplay,(b)atouchscreenasasingledeviceforbothinputandoutput,or(c)a3-Ddisplayandahapticdevicefornaturalmanipulationof3-Dobjects.

Theinterfacemaybeutilizedintwomainfields.Experiencedsurgeonsmayuseittosimulatetheiroperativeplansandprepareinputdataforasurgicalnavigationsystemwhilestudentornovicesurgeonscanuseitfortraining.

TheinterfacehasbeendevelopedintheTLEMsafeproject(www.tlemsafe.eu)fundedbytheEuropeanCommissionFP7program.

Conference 8289


Conference 8290 · 3D Image Processing (3DIP) and Applications 2012Tuesday-Thursday24-26January2012

Part of Proceedings of SPIE Vol. 8290 Three-Dimensional Image Processing (3DIP) and Applications II

8290-01, Session 1

Edge-aware stereo matching with O(1) complexityC.Cigla,MiddleEastTechnicalUniv.(Turkey)andVestek(Turkey);A.A.Alatan,MiddleEastTechnicalUniv.(Turkey)

Acomputationallyefficientstereomatchingalgorithmisintroducedprovidinghighprecisiondensedisparitymapsvialocalaggregationapproach.Theproposedalgorithmexploitsanovelparadigm,namelyseparablesuccessiveweightedsummation(SWS)amonghorizontalandverticaldirectionswithconstantoperationalcomplexity,providingeffectiveconnected2Dsupportregionsbasedonlocalcolorsimilarities.Theintensityadaptiveaggregationenablescrispdisparitymapswhichpreserveobjectboundariesanddepthdiscontinuities.Thesameprocedureisalsoutilizedtodiffuseinformationthroughoverlappedpixelsduringocclusionhandling.AccordingtotheexperimentalresultsonMiddleburyonlinestereobenchmark,theproposedmethodisoneofthemosteffectivelocalstereoalgorithmprovidinghighqualitydisparitymodelsbyunifyingconstanttimefilteringandweightedaggregation.Hence,theproposedalgorithmprovidesacompetitivealternativeforvariouslocalmethodsintermsofachievingprecisedisparitymapsfromstereovideowithinfastexecutiontime.

8290-02, Session 1

Establishing eye contact for home video communication using stereo analysis and free viewpoint synthesisC.Weigel,TechnischeUniv.Ilmenau(Germany);N.Treutner,Humboldt-Univ.zuBerlin(Germany)

Eyecontacthasbeenproventobeanimportantcueforvideocommunication.Wepresentamethodtore-establisheyecontactinahomevideocommunicationscenariocausedbymisalignmentbetweenthecameraandthecommunicationwindowtheparticipantlooksat.Ourmethodcoversthecompletealgorithmchainfromacquisitiontorenderingandusesapixel-based3Danalysisandrenderingapproachtocreateavirtualviewofacameraplacedatthepositionofthecommunicationwindow.Theoutcomesofalargescalesubjectivestudyidentifiedthecrucialproblemsofsuchanapproach.Basedonthesignificantobservationsduringthestudyinthispaperweaddressthemostimportantproblems.Weproposeamethodthatproducesspatiallyandtemporallyconsistentdepthmapsusingcross-checkbasedfillingandmotioncompensatedfiltering.Weaddressaliasingartifactsduringpointrenderingandpresentamethodtofenhancethevirtualviewbyimageinpaintingbasedonrobustcontourwarping.

8290-03, Session 1

Depth adaptive hierarchical hole filling for DIBR-based 3D videosM.Solh,G.Al-Regib,GeorgiaInstituteofTechnology(UnitedStates)

Inthispaperweintroduceadepthadaptiveapproachfordisocclusionremovalindepthimage-basedrendering(DIBR).Thisapproachextendsthehierarchicalhole-filling(HHF)presentedinanearlierwork.SimilartoHHF,thedepthadaptiveapproachresultsinsynthesized3Dvideosthatarefreeofgeometricdistortions.Furthermore,theedgesandtexturearoundthedisoccludedareascanbesharpenedandenhancedthroughaddingthedepthadaptivepre-processingstepbeforeapplying

thehierarchicalhole-filling.Thesubjectiveandobjectiveresultsshowasignificantimprovementinqualityforthesynthesizedviewsusingthedepthadaptiveapproach.

8290-04, Session 1

Space carving MVD sequences for modelling natural 3D scenesY.Alj,TechnicolorS.A.(France)andInstitutNationaldesSciencesAppliquéesdeRennes(France);G.Boisson,TechnicolorS.A.(France);L.Morin,M.Pressigout,InstitutNationaldesSciencesAppliquéesdeRennes(France);P.Bordes,TechnicolorS.A.(France)

Inaclassicalmulti-viewvideoframework,fromvideoacquisitiontoviewsynthesis,removingredundancyremainsachallengingtask.Weaimtobuildacompactrepresentationofthemultiview-videomaterialtobetransmittedonacommunicationchannelwithhighfidelityatdecoderside.Tothisend,becauseoftheefficiencyoftherenderingenginestowardsgeometricprimitives,wefocusoureffortongeometric-basedrepresentationsofMVDsequences.Moreprecisely,wegenerateahighresolutiontriangularmeshperviewandproposetomergethegeneratedmeshesintoasinglevolumetricframework.Herebywetakeadvantageoftheexactper-viewtriangularmeshconstructionandtherobustnessofthevolumetricrepresentation.Ourvolumetricframeworkusesavariantofthespacecarvingalgorithmbyintroducinganewcriterionforclassifyvoxelsintoopaqueortransparent.Themergedspace-carvedvolumetricmodelisturnedbackintoafinalmeshmodelwiththemarchingcubesalgorithm.Lastthatfinalmeshissimplifiedinordertofitrealisticbroadcastconstraints,droppingverticesandedgesthatdonotimpacttoomuchfinalrenderedviewsquality.Itturnsoutthatournewformulationbasedongeometricconsistencygivesmoreaccurateresultsthanthetraditionalphoto-consistency-basedspacecarving.

8290-05, Session 3

A locally content-dependent filter for inter-perspective anti-aliasingM.Sjöström,S.Tourancheau,X.Wang,R.Olsson,MidSwedenUniv.(Sweden)

Presentationsofcontentsonmultiviewandlightfielddisplaysprojectinganumberofviewsintoaviewingzonehavebecomeincreasinglypopular.Therestrictednumberofviewsimpliesanunsmoothmotionbetweenviewsifobjectswithsharpedgesarefarfromthedisplayplane.Thephenomenonisexplainedbyinter-perspectivealiasing.Thisisundesirableinapplicationswhereacorrectperceptionofthesceneisrequired,suchasscienceandmedicine,wheredepthcompressionalsoshouldbeavoided.Anti-aliasingfiltershavebeenproposedintheliterature,andaredefinedaccordingtotheminimumandmaximumdepthpresentinthescene.Wesuggestamethodthatsubdividestheray-spaceandadjuststheanti-aliasingfiltertothescenecontentslocally.Wefurtherproposenewfilterkernelsbasedontherayspacefrequencydomainthatassuresnoaliasing,yetkeepingmaximuminformationunaltered.Theproposedmethodoutperformsfiltersofearlierworks.Differentfilterkernelsarecompared.Detailsoftheoutputaresharperusingaproposedfilterkernel,whichalsopreservesthemostinformation.


8290-06, Session 3

Photometric and geometric rectification for stereoscopic imagesS.Han,J.Min,T.Park,Y.Kim,SamsungElectronicsCo.,Ltd.(Korea,Republicof)

Stereoimagepairsarecapturedbytwocamerasatdifferentpositions.Ingeneralthesetwoimageshavedistortionssuchasverticalmisalignment,rotation,andkeystone.Evenincaseofaparallelstereosetup,thecapturingsystemproducesthedistor-tionsthatcauseuncomfortable3Dexperiencestousers.Inthispaper,weproposeanalgorithmtoadjusttheimagepairstogivebetterstereoscopicexperiencetousersbycorrectingthedistortions.Ouralgorithmcompensatesphotometricalandge-ometricaldistortionsinimagepairsefficiently.Moreover,wecaretheoptimalhorizontalparallaxsothatalgorithmmaximizes3-Dexperiencesbasedonuser’spreference.


Novel time- and depth-stamped imaging for 3D-PIV (particle image velocimetry) using correlation image sensorK.Komiya,T.Kurihara,S.Ando,TheUniv.ofTokyo(Japan)

Inthispaper,wedealwithanovelprincipleofthree-dimensionalparticleimagevelocimetry(3D-PIV)offluidmotionusingcorrelationimagesensor(CIS)andastructuredillumination.Thecorrelationimagesensoristhedevicewhichoutputsthetemporalcorrelation‐betweenincidentlightintensityandreferencesignal.CIScapturesthelocioftracerparticleswiththepassingtimeinformationintheformofphasedistributionrelatingtoareferencesinusoidalsignal.Usingtemporal-spatialmodulatedlight,italsorecordsdepth-directionalcomponentoftheparticlevelocity.Inotherwords,wecanobtainthe3Dshapeofparticletrajectoryinoneframewithmonocularcamera.Therefore,theproposed3D-PIVsystemenablesustoacquirethepositionandtimeinformationofnumbersoftracersatthesametime.Theprincipleandimplementationmethodindetailandseveralresultsofbasicexperimentsareshown.


3D imaging for ballistics analysis using chromatic white-light sensorA.Makrushin,M.Hildebrandt,J.Dittmann,E.Clausing,Otto-von-Guericke-Univ.Magdeburg(Germany);R.Fischer,C.Vielhauer,Univ.ofAppliedSciencesBrandenburg(Germany)

Thenovelapplicationofsensingtechnology,basedonchromaticwhitelight(CWL),givesanewinsightintoballisticanalysisofcartridgecases.TheCWLsensorusesabeamofwhitelighttoacquirehighlydetailedtopographyand2D-grayscaledatasimultaneously.Theproposed3Dimagingsystemcombinesadvantagesof3Dand2Dimageprocessingalgorithmsinordertoautomatetheextractionoffirearmspecifictoolmarksshapedonfiredspecimens.Themostimportantcharacteristicsofafiredcartridgecasearethetypeofthebreech-blockmarkingaswellassize,shapeandlocationofextractor,ejectorandfiringpinmarks.Thefeatureextractionalgorithmnormalizesthecasingsurfaceandconsistentlysearchesfortheappropriatedistortionsontherimandontheprimer.Thelocationofthefiringpinmarkinginrelationtothelateralscratchesontherimprovidesuniquerotationinvariantcharacteristicsofthefirearmmechanisms.Additionalcharacteristicsarethevolumeandshapeofthefiringpinmarking.Theexperimentalevaluationreliesonthedatasetof15cartridgecasesfiredfromthree9mmfirearmsofdifferentmanufactures.Theresultsshowveryhighpotentialof3Dimagingsystemsforcasing-basedcomputer-aidedfirearmidentification,whichisprospectivelygoingtosupporthumanexpertise.


Computer-aided 3D-shape construction of hearts from CT images for rapid prototypingY.Kato,M.Fukuzawa,N.Nakamori,KyotoInstituteofTechnology(Japan);S.Ozawa,KyotoPrefecturalUniv.ofMedicine(Japan);I.Shiraishi,NationalCardiovascularCtr.ResearchInstitute(Japan)

Bydevelopingacomputer-aidedmodelingsystem,the3Dshapesofinfant’shearthavebeenconstructedinteractivelyfromquality-limitedCTimagesforrapidprototypingofbiomodels.The3Dmodelwasobtainedbyfollowinginteractivesteps:(1)roughregioncropping,(2)outlineextractionineachslicewithlocally-optimizedthreshold,(3)verificationandcorrectionofoutlineoverlap,(4)3Dsurfacegenerationofinsidewall,(5)connectionofinsidewalls,(6)3Dsurfacegenerationofoutsidewall,(7)synthesisofself-consistent3Dsurface.Themanufacturedbiomodelsrevealedcharacteristic3Dshapesofheartsuchasleftatriumandventricle,aorticarch,subclavianarteryandrightauricle.Theirrealshapeofcavityandvesselissuitableforsurgeryplanningandsimulation.Itisaclearadvantageoverso-called“blood-pool”modelwhichismassiveandoftenfoundin3DvisualizationofCTimagesasvolumerenderingperspective.Thedevelopedsystemcontributedbothtoqualityimprovementandtomodeling-timereduction,whichmaysuggestapracticalapproachtoestablisharoutineprocessformanufacturingheartbiomodels.Furtherstudyonthesystemperformanceisnowstillinprogress.


Operation-free 3D visualization of pulsatile tissues in freehand ultrasonic diagnosisK.Kawata,M.Fukuzawa,N.Nakamori,KyotoInstituteofTechnology(Japan);Y.Kitsunezuka,SaiseikaiHyogo-kenHospital(Japan)

Thepulsatiletissuesduetolocalbloodflowofarteries,whichisasignificantdiagnosticfeatureofischemicdiseases,havebeendetectedfromthefreehand2Dultrasonic(US)moviesofseveralsectionsandvisualizedtheir3Dstructureasre-sampledscatterdiagramwithoutanyadjustingoperation.Itwasrealizedbyimplementingautomaticparameter-determinationfunctionalitiesinthePC-based3DvisualizationsystemdesignedtobeusedatthebedsidebyconnectingdirectlytotheconventionalUSapparatus(ATLHDI5000)andacompacttilt-sensorattachedtotheUSprobesof5-7MHzsuitableforneonatalcranium.Themajorparameterstobeadjustedwereviewpoint,fieldofview(FOV),re-samplingintervalQandthresholdofpulsatilestrengthITH.TheviewpointandFOVweredeterminedbasedontheprobeorientation,whiletheQandITHweredeterminedbasedonstatisticsofpulsatilestrength.Byusingthe2DUSmoviesbetweenthefrontalandtheposteriorcoronalsections,the3Dstructureofpulsatiletissuewasdescribedwellalongmiddleandposteriorcerebralarteries,Willisringandcerebellararteriesinneonatalcranium.Sincepediatriciansarerelievedfromanyadjustingoperation,itisdemonstratedthatourtechniqueispracticaltoassistfreehandUSdiagnosisatthebedside.


Semiautomatic generation of semantic building models from image seriesS.Wirtz,P.Decker,D.Weiß,D.W.Paulus,Univ.Koblenz-Landau(Germany)

Intherecentyearssemanticbasedmodeling,reconstructionofbuildingsandbuildingrecognitionbecamemoreandmoreimportant.Semanticbuildingmodelsprovidemoreinformationthanjustthegeometry,thusmakingthemmoresuitableforrecognitionorsimulationtasks.Thetimeconsumingcreationofsuchmodelsandannotationsmakesanautomatismdesirable.WepresentanapproachembeddedinthephotostitchingtoolHugin,whichisabletocreatea3Dmodelofabuildingwithsemanticannotationsfromaseriesofimages.The

Conference 8290


followingstepsarenecessary:imageacquisition,featuredetection,poseestimation,3Dreconstruction,semanticinterpretation/annotationandexportinasuitableformat.Anentireautomaticprocessingwhichdeliversreliablemodelsisnotpossibleyet.Therefore,wepresentasemiautomaticapproachtowardssemanticmodelgeneration.Ourapproachreducestheinteractionwiththesystemtoaminimumonthesuppositionthatreliableandaccuratemodelsareneeded.Theresultingmodelcontainssemantic,geometricandappearanceinformationandisrepresentedinGeographyMarkupLanguage(GML).


Complex virtual urban environment modeling from CityGML data and OGC web services: application to the SIMFOR projectJ.Chambealland,G.Gesquière,Lab.desSciencesdel’InformationetdesSystèmes(France)

Duetotheadvancesincomputergraphicsandnetworkspeeditispossibletonavigatein3Dvirtualworldinrealtime.Thistechnologyproposedforexampleincomputergames,hasbeenadaptedfortrainingsystems.Inthiscontext,acollaborativeseriousgameforurbancrisismanagementcalledSIMFORisborninFrance.Thisprojecthasbeendesignedforintensiverealistictrainingandconsequentlymustallowtheplayerstodesignnewurbanoperationaltheater.Inthisgoal,importing,structuring,processingandexchanging3Durbandataremainsanimportantunderlyingproblem.Thiscommunicationwillfocusonthedesignofthe3DEnvironmentEditor(EE)andtherelateddataprocessesneededtopreparethedataflowtobeexploitablebytheruntimeenvironmentofSIMFOR.WewillusesolutionsproposedbytheOpenGeospatialConsortium(OGC)toaggregateandsharedata.Apresentationoftheproposedarchitecturewillbeproposed.TheoveralldesignoftheEEandsomestrategiesforefficientlyanalyzing,displayingandexportinglargeamountofurbanCityGMLinformationwillbepresentedinthispart.ThelastpartwillbedevotedtoanexampleillustratingthepotentialityoftheEEandthereliabilityoftheproposeddataprocessing.


Liquid crystal materials and structures for image processing and 3D shape acquisitionK.Garbat,MilitaryUniv.ofTechnology(Poland);P.Garbat,WarsawUniv.ofTechnology(Poland)

Liquidcrystalshavebeenextensivelystudiedandaremassivelyusedindisplayandopticalprocessingtechnology.Theirrecentusetoprovide3Dshapeacquisitionhasfocusedtheattentiononnewspecificrequirements.Recentimprovementsinthestructurelightmeasurementssystembasedondigitallightprojectionsupportedbyprocessingallowrapidacquisitionofdataabout3Drealobjects.Wepresentherethemainrelevantparametersofliquidcrystalforimageprocessingand3Dshapeacquisitionandwecomparethemainliquidcrystaloptionswhichcanbeusedwiththeirrespectiveadvantages.Weproposeheretocompareperformanceofseveraltypesofliquidcrystalmaterials:nematicmixtureswithhighandmediumopticalanddielectricalanisotropiesandrelativelylowrotationalviscosities,nematicmaterialswhichmayoperateinTNmodetwisted,materialsfordualfrequencyaddressingsystemsandV-shapedswitchingsmecticsliquidcrystal.


Piece-wise linear function estimation for platelet-based depth maps coding using edge detectionD.Sebai,TheUniv.ofManouba(Tunisia)

Manyresearchworksondepthmapscodingissueshavebeencarriedoutsuchasthosebasedonsegmentation.Themainideaisto

subdividethedepthmapintohomogenousregionsonwhichpixelsdisparitiesaremoreorlesssimilar.Inthiscontext,theplatelet-basedcodingapproachisbasedonarecursivequadtreesegmentationofthedepthmap.However,theestimationofthewedgeletandplateletfunctionsisacomputationallyexpensivetaskthathasanexponentiallyincreasingcomplexityinstepwiththedepthmapresolution.Themainideaoftheproposedmethodisbasedonwedgeletandplateletfunctions’definition:twoconstantorlinearfunctionsseparatedbyastraightsubdivisionline.Theconstantorlinearfunctionsareusedtoapproximatethetwosub-regionsofanodewhilethestraightlineisusedtoapproximateagrayscalediscontinuitysoanobjectboundary.Therefore,weoperateanedgedetectionforeachquadtreenodetolookforthesubdivisionline.


Source modeling for effective 3D video deliveryC.T.Hewage,M.G.Martini,KingstonUniv.(UnitedKingdom)

Therendered3Dvideoqualityatthereceiver-sidewillbeaffectedbytherenderingartifactsaswellasbytheconcealmenterrorswhichoccurintheprocessofrecoveringmissing3Dvideopackets.Thereforeitisvitaltohaveanunderstandingoftheartifactspriortotransmittingdata.Thereforethisworkproposesamodeltoquantifyrenderingandconcealmenterrorsatthesender-sideandusetheinformationgeneratedthroughthemodeltoeffectivelydeliver3Dvideocontent.


New technique for capturing images containing invisible depth information on object using brightness modulated rightS.Isaka,K.Uehira,KanagawaInstituteofTechnology(Japan)

Thispaperpresentsanewtechniqueforcapturingimageswheredepthinformationontheobjectisinvisiblyandsimultaneouslyembeddedinits2-Dimagewhentheobjectimageistakenwithacamera.Anobjectisilluminatedbylightthatcontainsinvisibleinformationwhosecharacteristicschangedependingondepth;therefore,theimagesofobjectscapturedwithacameraalsocontainsuchinformationinvisibly.Thisinvisibleinformationondepthcanbeextractedbyappropriateimageprocessingfromthecapturedimageoftheobject.Imagestakenwiththistechniquecanbetreatedasconventional2-Dimagesbecausetheimageformatisforconventional2-Dimages.3-Dimagescanalsobeconstructedbyabstractingdepthinformationembeddedintheimage.Wecarriedoutexperimentsusinglineandspacepatternandconfirmedthefeasibilityofthetechnologybydemonstratingthattheprojectedpatterncouldbeembeddedinthecapturedimageinvisiblyanditsfrequencycomponent,whichdependedonthedepthoftheobject,couldbereadoutfromthecapturedimage.


Interactive 3D segmentation by tubular envelope model for thoracic aorta treatmentP.J.Lubniewski,Univ.d’AuvergneClermont-FerrandI(France)andUniv.KardynalaStefanaWyszynskiegowWarszawie(Poland);B.Miguel,V.Sauvage,C.Lohou,Univ.d’AuvergneClermont-FerrandI(France)

Weproposeanovelinteractive3Dsegmentationapproachandgeometricmodeldefinitioncalledatubularenvelopemodel.Itisconceivedtoexpresstubularobjectsshape,speciallyaortas,alsoincludingpathologicalaorticdissectioncases.Themainchallengeswehaveachievedarethespeedandinteractivityoftheconstruction.Thecomputerprogramdesignedforthistaskgivestheuserfullcontroloftheshapeandprecision,withnosignificantcomputationalerrors.Thelowcomplexityofthemodelandeaseofinteractivedesignmakesthe

Conference 8290


tubularenvelopeveryeffectiveincomparisontoothersegmentationmethods.Themodelaccuracyisadjustablebytheuser,accordingtohisrequirements,butthetimeofconstructionisalwayssignificantlysmaller.Theexplicitgeometricformoftheenvelopecouldbedirectlyusedtootherproceedingslikeshapeapproximation,regionofinterestformoreprecisesegmentationorparticularfeatureextraction,parametricdeformablemodeldefinition,rapid2Dprojection,andothers.


A parallel stereo reconstruction algorithm with applications in entomology (APSRA)R.Bhasin,Univ.ofIllinoisatUrbanaChampaign(UnitedStates);W.J.Jang,J.C.Hart,Univ.ofIllinoisatUrbana-Champaign(UnitedStates)

Weproposeafastparallelalgorithmforreconstructionof3-Dimensionalpointcloudsofinsectsfrombinocularstereoimagepairsusingahierarchicalapproachfordisparityestimation.Entomologistsstudyvariousfeaturesofinsectstoclassifythem,buildtheirdistributionmaps,anddiscovergeneticlinksbetweenspecimensamongvariousotheressentialtasks.Thisinformationisimportanttothepesticideandthepharmaceuticalindustriesamongothers.WhenconsideringthelargecollectionsofinsectsEntomologistsanalyze,itbecomesdifficulttophysicallyhandletheentirecollectionandsharethedatawithresearchersacrosstheworld.Withthemethodpresentedinourwork,Entomologistscancreateanimagedatabasefortheircollectionsandusethe3Dmodelsforstudyingtheshapeandstructureoftheinsectsthusmakingiteasiertomaintainandshare.Initialfeedbackshowsthatthereconstructed3Dmodelsaregoodandpreservetheshapeandsizeofthespecimen.Wefurtheroptimizeourresultstoincorporatemulti-viewstereowhichproducesbetteroverallstructureoftheinsects.OurmaincontributionisapplyingstereoscopicvisiontechniquestoEntomologytosolvetheproblemsfacedbyEntomologists.

8290-07, Session 4

Depth map upscaling through edge-weighted optimizationS.Schwarz,M.Sjöström,R.Olsson,MidSwedenUniv.(Sweden)

Denseandprecisedepthinformationisapre-requisiteinauto-stereoscopicthree-dimensionaltelevision(3DTV).Onlyaccuratedepthmapsassureahighqualityviewsynthesis,butthisinformationisnotalwayseasilyobtained.Depthgainedbystereocorrespondencematchingsuffersfromdisocclusionsandlow-texturizedregions,leadingtoerroneousdepthmaps.Theseerrorscanbeavoidedbyusingdepthfromdedicatedrangesensors,e.g.time-of-flightsensors.Becausethesesensorsonlyhaverestrictedresolution,theresultingdepthdataneedtobeadjustedtotheresolutionoftheappropriatetextureframe.Standardupscalingmethodssuchasnearestneighbororbicubicfilteringprovideonlylimitedqualityresults.Thispaperproposesasolutionforupscalinglowresolutiondepthdatatomatchhighresolutiontexturedata.WeintroducetheEdgeWeightedOptimizationConcept(EWOC)forfusinglowresolutiondepthmapswithcorrespondinghighresolutionvideoframesbysolvinganover-determinedlinearequationsystem.Similartootherapproaches,EWOCtakesinformationfromthehighresolutiontexture,butadditionallyvalidatesthisinformationwiththelowresolutiondepthtoaccentuatecorrelateddata.Objectivetestsshowanimprovementindepthmapqualityincomparisontootherupscalingapproaches.Thisimprovementissubjectivelyconfirmedintheresultingviewsyntheses.

8290-08, Session 4

Adaptive switching filter for noise removal in highly corrupted depth maps from time-of-flight image sensorsS.Lee,K.Bae,K.Kyung,T.Kim,SAMSUNGElectronicsCo.,Ltd.(Korea,Republicof)

Inthiswork,wepresentanadaptiveswitchingfilterfornoisereductionandsharpnesspreservationindepthmapsprovidedbyTime-of-Flight(ToF)imagesensors.Medianfilterandbilateralfilterarecommonlyusedincost-sensitiveapplicationswherelowcomputationalcomplexityisneeded.However,medianfilterblursfinedetailsandedgesindepthmapwhilebilateralfilterworkspoorlywithimpulsenoisepresentintheimage.Sincethevarianceofdepthisinverselyproportionaltointensity,wesuggestanadaptivefilterthatswitchesbetweenmedianfilterandbilateralfilterbasedonthelevelofintensity.Ifaregionofinteresthaslowintensityindicatinglowconfidencelevelofmeasureddepthdata,thenmedianfilterisappliedonthedepthatthepositionwhileregionswithhighlevelofintensityisprocessedwithbilateralfilterusingGaussiankernelwithadaptiveweights.Resultsshowthatthesuggestedalgorithmperformssurfacesmoothinganddetailpreservationaswellasmedianfilterandbilateralfilter,respectively.Byusingthesuggestedalgorithm,significantgaininvisualqualityisobtainedindepthmapswhilelowcomputationalcostismaintained.

8290-09, Session 4

Parametric model-based noise reduction for ToF depth sensorsY.S.Kim,B.Kang,H.Lim,O.Choi,K.Lee,J.D.K.Kim,C.Kim,SamsungAdvancedInstituteofTechnology(Korea,Republicof)

ThispaperpresentsanovelTime-of-Flight(ToF)depthdenoisingalgorithmbasedonparametricnoisemodeling.ToFdepthimageincludesspacevaryingnoisewhichisrelatedtoIRintensityvalueateachpixel.ByassumingToFdepthnoiseasadditivewhiteGaussiannoise,ToFdepthnoise(orstandarddeviation)canbemodeledbyusingapowerfunctionofIRintensity.Meanwhile,nonlocalmeansfilterispopularlyusedasanedge-preservingdenoisingmethodforremovingadditiveGaussiannoise.Toremovespacevaryingdepthnoise,weproposeanadaptivenonlocalmeansfiltering.Accordingtotheestimatednoise,thesearchwindowandweightingcoefficientareadaptivelydeterminedateachpixelsothatpixelswithlargenoisevariancearestronglyfilteredandpixelswithsmallnoisevarianceareweaklyfiltered.Experimentalresultsdemonstratethattheproposedalgorithmprovidesgooddenoisingperformancewhilepreservingdetailsoredgescomparedtothetypicalnonlocalmeansfiltering.

8290-10, Session 4

Silhouette extraction using color and depth informationE.V.Tolstaya,V.Bucha,SamsungElectronicsCo.,Ltd.(RussianFederation)

Recentlyapplicationsinvolvingcaptureofsceneswithobjectofinterestamongsurroundingsgainedhighpopularity.Suchapplicationsincludevideosurveillance,humanmotioncapture,human-computerinteraction,etc.Forproperanalysisofobjectofinterestanecessarystepistoseparateobjectofinterestfromsurroundings,i.e.performbackgroundsubtraction(orsilhouetteextraction).Thisisachallengingtaskbecauseofseveralproblems:slightchangesinbackground;shadows,causedbytheobjectofinterest;andsimilarlycoloredobjects.Inthisworkweproposeanewmethodforextractingsilhouetteofanobjectofinterest,baseduponthejointuseofbothdepth(range)andcolordata.Depth-baseddataisindependentofcolorimagedata,andhencenotaffectedbythelimitationsassociatedwithcolor-basedsegmentation,suchasshadowsandsimilarlycoloredobjects.Attheinitialmomentimageofbackground(notcontainingobjectofinterest)ispresent,anditisupdatedineveryframeusingextractedsilhouetteusing“runningaverage”.Silhouetteextractionmethodisbasedonk-meansclusteringofdepthdataandcolordifferencedata,andper-pixelsilhouettemaskcomputation,usingclusterscentroids.Proposedsolutionisveryfast,allowsreal-timeprocessingofvideo.Developedalgorithmwassuccessfullyappliedinhumanrecognitionapplicationandprovidedgoodresultsformodelinghumanfigure.

Conference 8290


8290-11, Session 4

Discrete and continuous optimizations for depth image super-resolutionO.Choi,H.Lim,Y.S.Kim,B.Kang,K.Lee,J.D.K.Kim,C.Kim,SamsungAdvancedInstituteofTechnology(Korea,Republicof)

Recently,aTime-of-Flight2D/3Dimagesensorhasbeendeveloped,whichisabletocaptureaperfectlyalignedpairofacolorandadepthimage.Toincreasethesensitivityofinfraredlight,thesensorelectricallycombinesmultipleadjacentpixelsintoadepthpixelattheexpenseofdepthimageresolution.

Torestoretheresolution,wepresentadepthimagesuper-resolutionmethodthatusesahigh-resolutioncolorimagealignedwiththeinputdepthimage.Theinputdepthimageisinterpolatedintothesizeofthecolorimage,andourdiscreteoptimizationconvertstheinterpolateddepthimageintoahigh-resolutiondisparityimage,whosediscontinuitiespreciselycoincidewithobjectboundaries.Subsequently,theinterpolateddepthimageisiterativelyfiltered,preservingthedisparitydiscontinuities.Meanwhile,ouruniquewayofenforcingthedepthreconstructionconstraintgivesahigh-resolutiondepthimagethatisabletoperfectlyreconstructtheoriginalinputdepthimage.

Weshowtheeffectivenessoftheproposedmethodbothquantitativelyandqualitatively,comparingtheproposedmethodwithtwoexistingmethods.Theexperimentrevealsthattheproposedmethodgivessharphigh-resolutiondepthimageswithlesserrorthanthetwomethods,forscalingfactorsof2,4,and8.

8290-12, Session 4

Superpixel-based depth image superresolutionY.Soh,SeoulNationalUniv.(Korea,Republicof);J.Sim,UlsanNationalInstituteofScienceandTechnology(Korea,Republicof);C.Kim,KoreaUniv.(Korea,Republicof);S.U.Lee,SeoulNationalUniv.(Korea,Republicof)

Inrecentyears,depthacquisitionfromascenehasbecomeeasier,duetothedevelopmentofTimeofFlight(TOF)depthsensors.However,thedepthimagesderivedfromcurrentTOFsensorssufferfromlowresolutionandnoise.ThestateoftheartTOFsensorretrievesthedepthimagesatthesizeof320by240pixelswhicharelativelylowresolutioncomparedtotheconventionalimagesensors.BecauseoftheresolutiondifferencebetweenTOFsensorandCCDimagesensor,therehavebeenattemptstomatchtheresolutionsbetweenthetwodifferentlyobtainedimages,byincreasingtheresolutionofdepthimagetothatofthecolorimage,whichiscalledsuper-resolution.Usingthemodifiedbilateralfilterisapopularapproachtotacklethisproblem,duetoitssuitabilityforrealtimeimplementations[1][2].Themainideaofthebilateralfilteristoapplyanedgepreservingsmoothingfilter[3].Thefollowingassumptionsaremadetousesuchfilter:theoccurrenceofedgesorintensitygradientsbetweendepthimageandcolorimagearehighlycorrelated.Anearlyworkusingthebilateralfilterproposesafusionmethodcalledjointbilateralupsampling[2].Usingthedepthimageandcolorimagebothasinputs,thedepthmapisadaptivelysmootheneddependingonthecolorsimilarity.Itisshownthat,theresultssufferfromartifactssuchastexturecopyingoredgesmoothingincaseswheretheassumptiondoesnothold.InmorerecentworksbyF.Garciaetal.[1],attemptsweremadetoreducetheartifactsofedgesmoothingandtexturecopying.Comparedtotheworkin[2]wheretherangetermisalwaysdeterminedbythecolorsimilarityterm,[1]extendsitsworkbyintroducingthecredibilitymapasanadditionalfactor.Thecredibilitymapassignslowcredibilitytothepixelsalongtheedgeinthedepthimageconsideringthemunreliable.Thisenablestherangetermtobeadaptivelyweighteddependingonthecredibility.Thisapproachsignificantlyreducesthetexturecopyingartifactbutstillsuffersfromtheedgesmoothingartifact.UsingMRFtosolvetheproblemwasfirstpresentedbyDiebeletal[4]andwasextendedbyGloudetal[5].BydefiningtheposteriorprobabilityofthedepthvalueforeachpixelinthereconstructeddepthimagedomainasaMarkovRandomField,bothpaperssolvetheproblemusingthemaximum-

a-posteriori(MAP)solution.However,theseapproachesalsosharetheassumptionthattheoccurrencesofedgesorintensitygradientsbetweenadepthimageandacolorimagearehighlycorrelated,andthustheysufferfromthesameartifactspreviouslymentioned.Also,thecomputationloadcomparedtootherapproachesissignificantlyhighwhichmakesitnotapplicabletocertainapplications.Inthiswork,wepointoutthatthejointbilateralfilteringapproachisinheritedtosufferfromanedgesmoothingartifactduetoitsweighting-basedoperations.Thiscanbeexplainedbyexaminingtherangetermusedintheseapproaches.Therangetermisthekeyfactorwhichdeterminestheweightofsmoothingtermaccordingtothestrengthofedges.Inallthejointbilateralfilteringapproaches,acontinuousGaussiankernelisusedastherangeterm,whichisthereasonacertainamountofedgesmoothingisinevitable.Whentheedgesarestrongenough,therangetermisanegligiblevalueandtheweightofsmoothingtermwillbezero,whichpreservestheedges.Onthecontrary,whentheedgesareweak,anon-negligibleamountofsmoothingwilloccurwhichresultsinedgesmoothing.Smoothededgesmaynotbenoticeableintheresultsshowninpreviousworks.However,whenwereconstruct3Dscenesusingthecolorimageandthecorrespondingdepthimage,afewincorrectlyassignedpixelsmaycauseagreatdegreeofperceivedawkwardnessinhumanvisualsystem.Therefore,weproposeasuperpixelbasedsuper-resolutionalgorithmthatenhancestheedgesinsteadofpreservingthem.Byusingasuperpixelbasedapproach,weexploitthefactthatasuperpixelpreservesalltheboundaryedgesandthedepthimageispiecewiseplanar.Theproposedalgorithmiscomposedofthreestages.Inthefirststage,thesizeofthelowresolutiondepthimagesisincreasedtothetargetresolutionusingbi-cubicinterpolation.Theincreaseddepthimageisthendividedintosuperpixelsusingoversegmentation.Duringtheoversegmentationprocess,theboundaryedgesarepreserved.Becauseadepthimagecanbeassumedtobepiecewiseplanar,foreachsuperpixel,wecanfitthedatapointstoaplanefunction.Inthisprocessthesmoothedgeswhichoccurredduetobi-cubicinterpolationareexterminatedandtheedgesbecomediscretealongtheboundaryedgessincetheplaneparametersaredeterminedbythenon-smoothedregions.Atthesecondstage,usingtheoversegmentationresultsofthehighresolutioncolorimage,thehighfrequencyedgeinformationistransferredtothesuperpixelmapusedinthefirststage.Thisprocessisessentialincaseswhentheresolutiondifferencebetweenthelowresolutiondepthimageandthehighresolutioncolorimageislargesincetheedgesinthelowresolutiondepthimageandtheedgesinthehighresolutioncolorimagearepronetobemisaligned.Atthelaststage,usingtherefinedsuperpixelmapandtheplanefunctionsforeachsuperpixel,wereconstructthehighresolutiondepthmap.Duringtheplanefittingprocess,theremaybediscontinuitiesalongregionswheretherearenoedges.ByadoptingaMAP-MRFframework,wehandlesuchoutliersandrenderahighqualitydepthimage.Byusingtheproposedsuperpixelbasedapproach,wecanpreventtheartifactsthatinevitablyoccurinthepreviousapproaches,aswellasenhancethedepthimagebysharpeningtheedges.Moreover,bydividingtheimageintoseveralsuperpixels,wehavethebenefitofperformingthisalgorithminaparallelmannerwhichallowsreductionofcomputationtime.

8290-13, Session 4

Efficient spatio-temporal hole filling strategy for Kinect depth mapsM.Camplani,L.Salgado,Univ.PolitécnicadeMadrid(Spain)

ThemarketofactivedepthcamerasandthecomputervisioncommunityhavebeenrevolutionizedbytheMicrosoftKinectthankstoitslowpriceandgoodresolution.Atthemomentseveralhuman-computerinteractionapplicationshavebeendeveloped.However,theKinectdepthmapspresentnoise-relatedproblemsthathavetobesolvedinordertoimprovetheiraccuracyandtobroadenitsfutureapplicability.OneofthemostimportantnoiseeffectsontheaccuracyoftheKinectdepthmapsisduetothepresenceofregionsforwhichthecameraisnotabletocorrectlyestimatethedepth.Simplesmoothingfilteringproceduresleadtopoorresultsintroducingblurringeffect.Thepresentedapproachisanefficientholefillingstrategy,basedonajointbilateralfilter,thataimsatimprovingtheKinectdepthmapaccuracy.Theproposedapproachanalyzesthevisualinformationinordertoreducetheblurringeffect;thedepthgradientinformationis

Conference 8290


usedtoconsiderthelocaldepthvariationand,thetemporalevolutionoftheneighborpixelsoftheholeregionsisincludedinthefilteringprocessthroughatemporal-consistencymap.Finally,theobtainednewdepthvaluesareusedinthefilteringprocessinsuccessiveframesimprovingthedepthmapaccuracy.

8290-14, Session 5

Experimental results of bispectral invariants discriminative powerK.Kubicki,JagiellonianUniv.(Poland);R.Kakarala,NanyangTechnologicalUniv.(Singapore)

Oneofthemaintoolsinshapematchingandpatternrecognitionareinvariants.Forthree-dimensionaldata,rotationinvariantscompriseoftwomainkinds:momentsandsphericalharmonicmagnitudes.Botharewellexaminedandbothsufferfromcertainlimitations.Insearchforbetterperformance,anewkindofspherical-harmonicinvariantshavebeenproposedrecently,calledbispectralinvariants.Theyarewell-establishedfromtheoreticalpointofview.Theypossesnumerousbeneficialpropertiesandadvantagesoverotherinvariants,includetheabilitytodistinguishrotationfromreflection,andthesensitivitytophase.However,insufficientresearchhasbeenconductedtochecktheirbehaviorinpractice.Inthispaper,resultsarepresentedpertainingtothediscriminativepowerofbispectralinvariants.ObjectsfromPrincetonShapeBenchmarkdatabaseareusedforevaluation.Itisshownthatthebispectralinvariantsoutperformpowerspectralinvariants,butperformworsethanotherdescriptorsproposedintheliteraturesuchasSHELLSandSHD.Thedifferenceinperformanceisattributabletotheimplicitfilteringusedtocomputetheinvariants.

8290-15, Session 5

Evaluation of surface reconstruction from point clouds by shape parametersL.Cao,F.J.Verbeek,LeidenUniv.(Netherlands)

3Dsurfacereconstructionfromapointcloudisanimportantresearchareaforcomputergraphicsandvisualization;itcanbepotentiallyimportantforapplicationinbio-imaging.Inthepastdecade,numbersofalgorithmsforsurfacereconstructionhavebeenproposed.Generallyspeaking,thesemethodscanbeseparatedintotwocategories:i.e.,explicitrepresentationandimplicitapproximation.

Althoughmostofthemethodsarefirmlybasedintheory,noanalyticalevaluationexistsbetweenthemethods.Themoststraightforwardwayofevaluationisstillbyvisualinspection.Wesearchforamethodthatcanpreciselypreservethesurfacecharacteristicsandisrobustinthepresenceofnoise,inordertoimprovethesurfacereconstructionmethodforbiologicalmodels.Therefore,weuseananalyticalapproachbyselectingseveralfeaturesassurfacedescriptorsandmeasuringthem.Weselectedsurfacedistance,surfaceareaandsurfacecurvatureasthreemajorfeaturestocomparequalityofthesurfacecreatedbythedifferentalgorithms.Forgroundtruthvaluesweusedanalyticalshapessuchasthesphereandtheellipsoid.

Herewepresentfourclassicalsurfacereconstructionmethodsfromthetwocategoriesmentionedabove.ThesearethePowerCrust,theRobustCocone,theFourier-basedmethodandthePoissonreconstructionmethod.OurexperimentalresultsindicatethatPoissonreconstructionmethodperformsthebestinthepresenceofnoise.

8290-16, Session 5

3D mesh Reeb graph computation using commute-time and diffusion distancesR.ELKhoury,J.P.Vandeborre,M.Daoudi,InstitutTELECOM(France)andTELECOMLille1(France)

3D-modelanalysisplaysanimportantroleinnumerousapplications.Inthispaper,wepresentanapproachforReebgraphextractionusing

anovelmappingfunction.Ourmappingfunctioncomputesarealvalueforeachvertexwhichprovidesinterestinginsightstodescribetopologystructureofthe3D-model.Weperformdiscretecontourforeachvertexaccordingtoourmappingfunction.TopologychangescanbedetectedbydiscretecontoursanalysistoconstructtheReebgraph.Ourmappingfunctionhassomeimportantproperties.Itisinvarianttorigidandnonrigidtransformations,itisinsensitivetonoise,itisrobusttosmalltopologychanges,anditdoesnotdependonparameters.Fromtheextractedskeleton,thesepropertiesshowthesignicantpartsofa3D-model.Weretaintheevaluationcriteriatothepropertiesofthemappingfunction,andcomparedthemtothoseusedinthestateoftheart.Intheend,wepresentextractedReebgraphonvariousmodelswithdierentpositions.

8290-17, Session 5

Geometric modeling of pelvic organs with thicknessT.Bay,Lab.desSciencesdel’InformationetdesSystèmes(France);Z.W.Chen,Univ.d’Evry-Vald’Essonne(France);R.Raffin,M.Daniel,Lab.desSciencesdel’InformationetdesSystèmes(France);P.Joli,Univ.d’Evry-Vald’Essonne(France);Z.Q.Feng,M.Bellemare,Lab.desSciencesdel’InformationetdesSystèmes(France)

Physiologicalchangesinthespatialconfigurationoftheinternalorgansintheabdomencaninducedifferentdisordersthatneedsurgery.Followingthecomplexityofthesurgicalprocedure,mechanicalsimulationsarenecessary.Buttheinvivofactormakescomplicatethestudyofpelvicorgans.Todeterminearealisticbehavioroftheorgans,aphysicalmodelingassociatedwithanaccurategeometricmodelisthereforerequired.Ourapproachisintegratedinthepartnershipbetweenageometricandphysicalmodule.

TheGeometricModelingseekstobuildacontinuousgeometricmodel.Fromadatasetof3DpointsprovidedbyaSegmentationstep,surfacesarecreatedthroughaB-splinefittingprocess.Anenergyfunctionisbuilttomeasurethebidirectionaldistancebetweensurfaceanddata.ThisenergyisminimizedwithanalternateiterativeHoschek-likemethod.Athicknessisaddedwithanoffsetformulation,andthegeometricmodelisfinallyexportedinahexahedralmesh.

Afterwards,thePhysicalModelingtriestocalculatethepropertiesofthesofttissuestosimulatetheorgansdisplacements.Thephysicalparametersattachedtothedataaredeterminedwithafeedbackloopbetweenfinite-elementsdeformationsandground-truthacquisition(dynamicMRI).

8290-18, Session 5

Refined facial disparity maps for automatic creation of 3D avatarsR.Pagés,F.Morán,L.Salgado,D.Berjón,Univ.PolitécnicadeMadrid(Spain)

Weproposeanewmethodtoautomaticallyrefineafacialdisparitymapobtainedwithstandardcamerasandlightingbyusingasmartcombinationoftraditionalcomputervisionand3Dgraphicstechniques.Oursysteminputstwostereoimagesacquiredwithnormal(calibrated)camerasandusesdensedisparityestimationtoobtainacoarseinitialdisparitymap,andSIFTtodetectseveralfeaturepointsinthesubject’sface.WethenusethesepointsasanchorstomodifythedisparityinthefacialareabybuildingaDelaunaytriangulationoftheirconvexhullandinterpolatingtheirdisparityvaluesinsideeachtriangle.Wethusobtainarefineddisparitymaprepresentingmuchbetterthesubject’sfacialfeatures.Thisrefinedfacialdisparitymapmaybeeasilytransformed,throughthecameracalibrationparameters,intoadepthmaptobeused,alsoautomatically,toimprovethefacialmeshofa3Davatartomatchthesubject’srealhumanfeatures.

Conference 8290


8290-19, Session 5

Fast human pose estimation using 3D Zernike descriptorsD.Berjón,F.Morán,Univ.PolitécnicadeMadrid(Spain)

AbstractMarkerlessvideo-basedhumanposeestimationalgorithmsfaceahigh-dimensionalproblemthatisfrequentlybrokendownintoseverallower-dimensionalproblemsbyestimatingtheposeofeachlimbseparately.However,todosotheyneedtoreliablylocatethetorso,forwhichtheytypicallyrelyontimecoherenceandtrackingalgorithmsthatwhenlosetrackresultincatastrophicfailureoftheprocess,requiringhumaninterventionandthusprecludingtheirusageinreal-timeapplications.Weproposeaveryfastroughposeestimationschemebasedonglobalshapedescriptorsbuilton3DZernikemoments.Usinganarticulatedmodelthatweconfigureinmanyposes,alargedatabaseofdescriptor/posepairscanbecomputedoff-line.Thus,theonlystepsthatmustbedoneon-linearetheextractionofthedescriptorsforeachinputvolumeandasearchagainstthedatabasetogetthemostlikelyposes.Whiletheresultofsuchprocessisnotafineposeestimation,itcanbeusefultohelpmoresophisticatedalgorithmstoregaintrackormakemoreeducatedguesseswhencreatingnewparticlesinparticle-basedschemes.Wehaveachievedaperformanceofabouttenfpsonasinglecomputerusingadatabaseofaboutonemillionentries.

8290-20, Session 5

Analysis of binning of normals for spherical harmonic cross-correlationR.Larkins,M.J.Cree,A.A.Dorrington,TheUniv.ofWaikato(NewZealand)

Theregistrationofoverlappingpointcloudshasrecentlyreceivedincreasedattentionduetothegrowingavailabilityof3Dsensors.Sphericalharmoniccross-correlationisarobustregistrationtechniquethatusesthenormalsoftwooverlappingpointcloudstobringthemintocoarserotationalalignment.Thisregistrationtechniquehoweverhasahighcomputationalcostassphericalharmonicsneedtobecalculatedforeverynormal.Bybinningthenormals,thecomputationalefficiencyisimprovedasthesphericalharmonicscanbepre-computedandcachedateachbinlocation.Inthispaperweevaluatetheaccuracyandefficiencyoftwopreviouslyreportedbinningmethodsandathirdthatwepropose.Itisfoundthatofthetwomethodsusedinpreviousliterature,onemethodofbinningdecreasestheachievedaccuracyforcertainregistrations,andtheotherdoesnotallowanexactnumberofbinstobespecifiedaprior.Thenewproposedmethodofbinningovercomesthesedisadvantages.Theprimarygoaloftheanalysisistoidentifyabinningmethodthatperformsaswellastheoriginalcross-correlationalgorithmwithoutbinning,whilealsoprovidingasignificantimprovementincomputationalefficiency.

8290-21, Session 5

Topology reconstruction for B-Rep modeling from 3D mesh in reverse engineering applicationsR.Bénière,Lab.d’InformatiquedeRobotiqueetdeMicroelectroniquedeMontpellier(France)andC4W(France);G.Subsol,Lab.d’InformatiquedeRobotiqueetdeMicroelectroniquedeMontpellier(France);G.Gesquière,Lab.desSciencesdel’InformationetdesSystèmes(France);F.LeBreton,C4W(France);W.Puech,Lab.d’InformatiquedeRobotiqueetdeMicroelectroniquedeMontpellier(France)

Inanindustrialcontext,recoveringaB-Repmodelfroma3DmeshisnecessarytomakemodificationsortoexchangedatawithaformatincludingcontinuousrepresentationofobjectslikeSTEP.Butformanyreasons,theinitialcontinuousobjectcanbelostafteradiscretizationprocesstodisplayorexchangedata.Themeshcanalsobedeformed

afteranumericalcomputation.Itisthenimportanttohaveamethodtocreateanewcontinuousmodeloftheobjectfromamesh.ForCADobject,thefirststepistodetectsimpleprimitiveslikeplanes,spheres,conesandcylindersfroma3Dmesh.

Inpreviousworks,weusecurvaturefeaturestorecovertheseprimitives.Segmentationinpointareasisbasedonthecurvaturefeaturescomputedforeachvertex.Eachoneisassociatedwithaprimitivetype.Finalparametersoftheprimitivesarefoundwithafittingprocessaccordingtothecurvaturefeatures.

Thenoveltyofthispaperisamethodtodeterminethetopologyofthissetof3Dprimitiveextractedpreviously.Thetopologydefinitionisbasedonbothcontinuousinformationsfromprimitivesanddiscreteinformationsfrompointareas.Itleadstoconstructadjacencyrelations,thenthewiresandtofinishtheB-Repmodel.

8290-22, Session 5

An evaluation of local shape descriptors for 3D shape retrievalS.Y.Tang,PrincetonUniv.(UnitedStates);A.Godil,NationalInstituteofStandardsandTechnology(UnitedStates)

Astheusageof3Dmodelsincreases,sodoestheimportanceofdevelopingaccurate3Dshaperetrievalalgorithms.Manylocaldescriptorshavebeenproposedtoapproachthisproblem;however,theyareevaluatedondifferentdatasetsorusedwithindifferentalgorithms,makingthemdifficulttocompare.UsingtheSHREC2011ShapeRetrievalContestofNon-rigid3DWatertightMeshesdataset,wecompareasetoflocalshapedescriptorsbyapplyingeachtothebag-of-wordsparadigm.Wegoontoassesspossiblebenefitsofusingsamplepointsreturnedbyasalientpointdetectionalgorithmratherthanfindingarandomselection.Finally,twolocaldescriptorsarecombinedandappliedtogether.Inthispaper,wepresentresultsandobservationsfromtheseexperiments.

8290-23, Session 6

Fractal geometry and multimedia retrieval: a theoretical frameworkE.Paquet,NationalResearchCouncilCanada(Canada);H.L.Viktor,Univ.ofOttawa(Canada)

Thispaperpresentsatheoreticalanalysisoftheoccurrenceoffractalgeometrywithinindexspacesanddiscussestheimpactformultimediaretrieval.Firstly,weexplainhowtodetectthepresenceofsuchafractalgeometry.Then,withthefractalhypothesisinhand,weanalyzetheimpactofthisgeometrywhencalculatingthedistancebetweenindexesandsearchingforsimilarmultimediaobjects.Wedemonstratethatitispossibletodefineprobabilitydistributionsinfractalindexspaceandwepresentanapproachtocalculatethem.Practicalconsequencesarediscussed,withparticularemphasistomultimediaretrievalmethodsbasedonBayesiananalysis.

8290-24, Session 6

Spatial modeling of bone microarchitectureH.Li,K.Li,T.Kim,A.Zhang,M.Ramanathan,Univ.atBuffalo(UnitedStates)

Inthisresearch,wefocusondevelopingthespatialcomponentofsuchamodelingframeworkanddemonstratehowbonemicroarchitecture,bonecompositionandfractureriskcanbeincorporatetoobtainmechanisticassessmentsoffractureriskfromthatcanbeobtainedinclinicalsettings.Ourmodelforbonemineralisdevelopedanditsparameterswereestimatedfromimagingdataobtainedwithdualenergyx-rayabsorptiometryandx-rayimagingmethods.Usingtheseparameters,wedevelopaproper3DMicrostructureBoneModel.

Wedevelopandevaluateanovelthree-dimensionalcomputationalboneframeworkcapableofproviding:1)structurallyderived

Conference 8290


quantitativemeasuresofbonemicroarchitecture,2)analysisofbonemineraldensity(BMD)andfracturerisk,3)amechanisticunderstandingoftherelationshipofbonemicroarchitecturetofracturerisk,and4)insightsintootherclinicalproblems,suchasneurologicaldisordersandleukemia,byidentifyingthediseaseprogressionandprevention.Fromthebonemicroscopicstructures,athree-dimensionalrod-likestructurenetworkisgeneratedasanabstractedbonenetwork.Attributesofbonemicroscopicstructuresarecalculatedandappliedtobonenetwork.

8290-25, Session 6

A new affine invariant method for image matchingP.Montesinos,EcoledesMinesd’Alès(France)

Thispaperdescribesanewapproachincolororgrey-scaleimagematchingbypointsofinterest.Asmanypointmatchingmethods,thismethodisbasedontwomainsteps:computationofpointsanddescriptors,followedbyamatchingprocess.Thispaperpresentsthenewdescriptordefinedandamatchingprocessdealingwiththedataobtainedfromthedescriptor.

8290-26, Session 6

2D-3D feature association via projective transform invariants for model-based 3D pose estimationO.S.Gedik,A.A.Alatan,MiddleEastTechnicalUniv.(Turkey)

Thethreedimensional(3D)trackingofrigidobjectsisrequiredinmanyapplications,suchas3Dtelevision(3DTV)andaugmentedreality.Accurateandrobustposeestimatesenableimprovedstructurereconstructionsfor3DTVandreducejitterinaugmentedrealityscenarios.Ontheotherhand,reliable2D-3Dfeatureassociationisoneofthemostcrucialrequirementsforobtaininghighquality3Dposeestimates.Inthispaper,a2D-3Dregistrationmethodwhichisbasedonprojectivetransforminvariantsisproposed.Duetothefactthatprojectivetransforminvariantsarehighlydependenton2Dand3Dcoordinates,theproposedmethodreliesonposeconsistenciesinordertoincreaserobustnessof2D-3Dassociation.ThereliabilityoftheapproachisshownbycomparisonswithRANSAC,perspectivefactorizationandSoftPOSITbasedmethodsonrealandartificialdata.

8290-27, Session 6

Reprocessing anaglyph imagesH.G.Dietz,Univ.ofKentucky(UnitedStates)

Inrelatedwork,wehaveshownthatconventionaldigitalcameraseasilycanbemodifiedtodirectlycaptureanaglyphs.Anaglyphimageshavecommonlybeenusedtoencodestereoimagepairsforviewing,butanaglyphsalsocanbetreatedasanefficientencodingoftwo-viewimagedataforreprocessing.Eachofthetwoviewsencodedwithinananaglyphhasonlypartialcolorinformation,butourpreliminaryresultsdemonstratethatthe“lost”informationcanbeapproximatelyrecoveredwithanyofavarietyofreasonablyefficientalgorithms.Thisnotonlyallowscrediblefull-colorstereopairsbecomputationallyextracted,butalsoenablesmoresophisticatedcomputationalphotographytransformationssuchascreationofdepthmapsandvarioustypesofpoint-spread-function(PSF)substitutions.

8290-28, Session 7

X-ray stereo imaging for micro 3D motions within non-transparent objectsW.H.M.Salih,J.A.N.Buytaert,J.J.J.Dirckx,Univ.Antwerpen(Belgium)

Weproposeanewtechniquetomeasurethe3Dmotionofmarkerpointsalongastraightpathwithinanobjectusingx-raystereoprojections.Fromrecordingsoftwox-rayprojectionswith90degreeseparationangle,the3Dcoordinatesofmarkerpointscanbedetermined.Bysynchronizingthex-rayexposuretimetothemotionevent,amovingmarkerleavesatraceintheimageofwhichthegrayscaleislinearlyproportionaltothemarkervelocity.Fromthegrayscalealongthemotionpath,the3Dmotion(velocity)isobtained.Thepathofmotionwasreconstructedandcomparedwiththeappliedwaveform.Theresultsshowedthattheaccuracyisinorderof5%.Thedifferenceofdisplacementamplitudebetweenthenewmethodandlaservibrometrywaslessthan5μm.

Wedemonstratedthemethodonthemalleusossiclemotioninthegerbilmiddleearasafunctionofpressureappliedontheeardrum.Thenewmethodhastheadvantageoverexistingmethodssuchaslaservibrometrythatthestructuresunderstudydonotneedtobevisuallyexposed.Duetotheshortmeasurementtimeandthehighresolution,themethodcanbeusefulinthefieldofbiomechanicsforavarietyofapplications.

8290-29, Session 7

A stereoscopic imaging system for laser back scatter-based trajectory measurement in ballistics: part 2U.Chalupka,H.Rothe,Helmut-Schmidt-Univ.(Germany)

Theprogressonalaser-andstereo-camera-basedtrajectorymeasurementsystemthatwealreadyproposedanddescribedinrecentpublicationsisgiven.Thesystemdesignwasextendedfromonetotwomorepowerful,DSP-controllableLASERsystems.Experimentalresultsoftheextendedsystemusingdifferentprojectile-/weaponcombinationswillbeshownanddiscussed.Automaticprocessingofacquiredimagesusingcommon3DIPimagingtechniqueswasrealized.Processingstepstoextracttrajectorysegmentsfromimagesasrepresentativeforthecurrentapplicationwillbepresented.Usedalgorithmsforbackward-calculationoftheprojectiletrajectorywillbeshown.Verificationofproducedresultsisdoneagainstsimulatedtrajectories,onceintermsofdetectionrobustnessandonceintermsofdetectionaccuracy.Fieldsofuseforthecurrentsystemarewithintheballisticdomain.Thefirstpurposeisfortrajectorymeasurementofsmallandmiddlecaliberprojectilesonashootingrange.Extensiontobigcaliberprojectilesaswellasanapplicationforsniperdetectionisimaginable,butwouldrequirefurtherwork.BesideclassicalRADAR,acousticandopticalprojectiledetectionmethods,thecurrentsystemrepresentsafurtherprojectilelocationmethodunderthenewclassofelectro-opticalmethodsthathavebeenevolvedinrecentdecadesandthatuses3Dimagingacquisitionandprocessingtechniques.

8290-30, Session 7

Single frame coaxial 3D measurement using depth from defocus of projection systemT.Kurihara,S.Ando,TheUniv.ofTokyo(Japan)

Weproposecoaxial3Dmeasurementsystembasedondepthfromdefocusofprojectionsystem.Inthissystem,thestripepatternisprojectedontheobject,anditmovesonobject’ssurfacegeneratingtemporalvariationofthelightintensity.Theprojectedstripepatternisdefocuseddependingonitsdistancefromthefocalplane.Bymovingthestripepattern,defocusedspatialfrequencycomponentiscapturedbytemporalfrequencyanalysis.DLPLightCommanderisusedformovingthestripepatternfast.Weusecorrelationimagesensor(CIS),whichisdevelopedbyus,tocapturethetemporalfrequencycomponentinasingleframe.TheCISoutputstheFouriercoefficientsofincidentlightineachpixelforeveryframes,andthereforeitenablessingleframe3Dmeasurement.Evaluationexperimentsshowthatprojectiondefocusdependsonthedistancefromthefocalplane,anditcanbeusedfor3Dmeasurement.

Conference 8290


8290-31, Session 7

Multidirectional four-dimensional shape measurement systemJ.Lenar,R.Sitnik,M.Witkowski,WarsawUniv.ofTechnology(Poland)

Currently,alotofdifferentscanningtechniquesareusedfor3Dimagingofhumanbody.MostofexistingsystemsarebasedonstaticregistrationofinternalstructuresusingMRIorCTtechniquesaswellas3Dscanningofoutersurfaceofhumanbodybylasertriangulationorstructuredlightmethods.Ontheotherhandthereisanexistingmature4Dmethodbasedontrackingintimethepositionofretro-reflectivemarkersattachedtohumanbody.Therearetwomaindrawbacksofthissolution:markersareattachedtoskin(norealskeletonmovementisregistered)anditgives(x,y,z,t)coordinatesonlyinthosepoints(notforthewholesurface).Inthispaperwepresentanovelmultidirectionalstructuredlightmeasurementsystemthatiscapableofmeasuring3Dshapeofhumanbodysurfacewithfrequencyreaching60Hz.Thedevelopedsystemconsistsoftwospectrallyseparatedandhardware-synchronized4Dmeasurementheads.Theprincipleofthemeasurementisbasedonsingleframeanalysis.Projectedframeiscomposedfromsine-modulatedintensitypatternandaspecialstripeallowingabsolutephasemeasurement.Severaldifferentgeometricalset-upswillbeproposeddependingontypeofmovementsthataretoberegistered.Initialmeasurementuncertaintyassessmentwillbepresentedtogetherwiththecalibrationprocedure.

8290-32, Session 8

Estimation of surface normal vectors based on 3D scanning from heating approachO.Aubreton,Univ.deBourgogne(France);G.Eren,GalatasarayÜniv.(Turkey);F.Truchetet,Univ.deBourgogne(France)

TheScanningFromHeatingisa3Dscanningapproachinitiallydevelopedtorealise3Dacquisitionoftransparentorspecularsurfaces.Alasersourceisusedtocreatealocalheatingpoint.AninfraredcameraisusedtoobservetheIRradiationemittedbythescene.The2Dcoordinatesoftheheatedpointarecomputedinthe2Dimageofthecamera.Knowingtheparametersofthesystem(whichareobtainedbyapreviouscalibration),the3Dcoordinatesofthepointarecomputedusingtriangulationmethod.Inthisarticlewewillpresentanextensionofthistechnique.WeproposeheretoanalysetheshapeofthehotspotobservedbytheIRcamera,and,fromtheanalysistodetermineinformationonthelocalorientationofthesurfaceateachmeasuredpoint.

8290-33, Session 8

First topographical features for the age determination of latent biometric fingerprint traces using a 3D optical and contact-less chromatic white light (CWL) sensorR.Merkel,Otto-von-Guericke-Univ.Magdeburg(Germany);S.Gruhn,FachhochschuleBrandenburg(Germany);J.Dittmann,Otto-von-Guericke-Univ.Magdeburg(Germany);C.Vielhauer,FachhochschuleBrandenburg(Germany)

Determiningtheageofalatentfingerprinttracefoundatacrimesceneisanunresolvedresearchissuesincemanydecades.Inthispaper,wedescribeanovelapproachliftingsuchtraceswithapowerful3Dimageacquisitionsystemandtransferringthemintoa3Dagingfeature.Inparticular,weuseacontact-less,high-resolutionChromaticWhiteLight(CWL)sensortoobservetheacquired3Dshapesandtheirtransformationsovertime.Wealsosuggestexaminingdifferentpreprocessingtechniquestodealwithnoiseandothermeasurementartifacts,totransformthetopographicalfingerprintdataintoaformwhichcanbeusedforsegmentationandpatternrecognition.Basedonpriorwork,weintroduceanagingfeaturecalledbinarypixelandextend

itto3D-topographyimages.Forthispurpose,weuseatestsetof10testseries,eachscanningafingerprintsamplefromaharddiskplattersurfaceover100hoursevery30minutes,leadingtoatotalof2000three-dimensionalfingerprintimages.Weshowthatwithadequatepreprocessingtechniques(DifferenceofGaussian,fingerprintpixelmasking,localvariance),acharacteristiclogarithmicagingpropertyisproducedforthe3D-topographyimages,whichcanbeconsideredfortheagedetermination.Futureworkwillbethefusionofsuch3Dagingfeaturewithourpriordeveloped2Dfeatures,toimprovetheaccuracyoftheoverallagedeterminationapproach.

8290-34, Session 8

A single-imager, single-lens video camera prototype for 3D imagingL.Christopher,W.Li,IndianaUniv.-PurdueUniv.Indianapolis(UnitedStates)

Anewmethodforcapturing3Dvideofromasingleimagerandlensisintroduced.Thebenefitofthismethodisthatitdoesnothavethecalibrationandalignmentissuesassociatedwithbinocular3Dvideocameras.Italsodoesnotrequirespecialrangingtransmittersandsensors.Becauseitisasinglelens/imagersystem,itisalsolessexpensivethaneitherthebinocularorrangingcameras.Oursystemoutputsa2DimageandassociateddepthimageusingthecombinationofmicrofluidiclensandDepthfromDefocus(DfD)algorithm.Thelensiscapableofchangingthefocustoobtaintwoimagesatthenormalvideoframerate.TheDepthfromDefocusalgorithmusestheinfocusandoutoffocusimagestoinferdepth.ThebasicprinciplesofDfDaredescribedinPentland[1]andChaudhuri[2].WeperformedourexperimentsonsyntheticandontherealapertureCMOSimagerwithmicrofluidiclens.Onsyntheticimages,wefoundatwotimesreductioninpercentrangingerror.Oncameraimages,ourresearchshowedthatDfDcombinedwithedgedetectionandsegmentationprovidedsubjectiveimprovementsintheimages.

8290-35, Session 8

3D multimodal data fusion systemP.Garbat,WarsawUniv.ofTechnology(Poland)

Recentlyoneofthebiggestchallengesincomputervisionisthe3Danalysisandreconstructionofobjectsandscenes.Thisrequiresgatheringrealdataaboutstaticordynamic3Dobjectsandscenesanddeliveringthemtoaremotevisualizationsystem.Weproposeahybrid,multimodaldatacapturesystembasedonthreeband,nearinfrared,fringeprojectionsystemwithfastcamera,hi-resolutincameraforintensityandstateofpolarizationacquisitionunit.

ThispaperdescribesafastalgorithmicapproachtocombineFull-HDRGBimageswith3Ddata(X,Y,Z),andinformationaboutstateofpolarizationonobjectsurfacePOL.WeproposemethodbasedonICPalgorithmswithhierarchical,multiscale3Ddatarepresentation.Themethodofmergingpointssetsbasedonmatchingfeaturepoints(regions)incurvaturescalespaceCSS.Thefeaturevectorsaredeterminedusinggeometry,texture,polarizationandotherlocalparametersofmatched3Dsurface/volume.Thematchingprocesscanbeusedinweak-calibrationprocedureofcameraset.Itmakesthe3Dvideosystemmoreuniversalincomparisonwithmethodsbasedonfullycalibratedsetup.

8290-36, Session 8

Fully automatic 3D digitization of unknown objects using progressive data bounding boxA.Aigueperse,S.Khalfaoui,Lab.d’Electronique,InformatiqueetImage(France);R.Seulin,Y.Fougerolle,D.Fofi,Univ.deBourgogne(France)

Thegoalofthisworkistodevelopacompletesystemcapableof

Conference 8290


scanningobjectsautomaticallywithminimumaprioriinformationofitsshape.Weaimtoestablishamethodologyfortheautomationofthe3Ddigitizationchain.ThepaperpresentsamethodbasedontheevolutionoftheBoundingBoxoftheobjectduringtheacquisitionsteps.Theregistrationofthedifferentacquireddataisdoneusingthemodelingofthepositioningsystem.Theobtainedmodelsareanalyzedandinspectedinordertoevaluatetherobustnessofourmethod.Testswithrealobjectshavebeenperformedandresultsofdigitizationareprovided.

8290-37, Session 9

3D video compression with the H.264 codecN.L.Karpinsky,S.Zhang,IowaStateUniv.(UnitedStates)

Advancesin3Dscanninghaveenabledthereal-timecaptureofhigh-resolution3Dvideo.Withtheseadvancescomesthechallengeofstreamingandstoringthis3Dvideoinamannerthattheycanbequicklyandeffectivelyused.Todothisdifferentapproacheshavebeentaken,apopularonebeingimagebasedencoding,whichprojectsfrom3Dinto2D,uses2Dcompressiontechniques,andthendecodesfrom2Dbackto3D.OnesuchtechniquethatdoesthisistheHolovideotechnique,whichwehaveshowntoyieldgreatcompressionratios.However,thetechniquewasoriginallydesignedfortheRGBcolorspaceanduntilrecentlycouldnotbeusedwithcodecsthatusetheYUVcolorspacesuchastheH.264codec.Thispaperaddressesthisissue,generalizingHolovideototheYUVcolorspace,allowingittoleveragetheH.264codecmakingitaviablesolutionfor3Dvideocompression.

8290-38, Session 9

3D multiresolutions synchronization scheme based on feature point selectionN.Tournier,Lab.d’InformatiquedeRobotiqueetdeMicroelectroniquedeMontpellier(France)andStratégiesS.A.(France);W.Puech,G.Subsol,Lab.d’InformatiquedeRobotiqueetdeMicroelectroniquedeMontpellier(France);J.Pedeboy,StratégiesS.A.(France)

Multimediaprotectionisoneofthemainresearchchallengesincomputersciences.Wecanencryptthemediainordertomakethecontentunreadablewithoutasecretkeyofdecryption,protectthefilewithDigitalRightManagement(DRM),orembedanhiddenmessageinthefile(watermarkingandsteganography).Weareinterestedindatahidingapplicationsfor3Dmesh.Inthisdomain,thereamainlyproblem:thesynchronization.Itistheoperationthatpermitstoscanameshwithauniquepathandbyselectingthesameareas(vertices,triangles,quadrangles,forexample)beforeandaftertheembedding.

Inthispaper,weproposeanewsynchronizationtechniquebasedonfeaturepointselectioninalowresolutionoftheobject.Thebuildingofthelowresolutionismadebydecimationandthefeaturepointselectionisbasedonthediscretecurvaturecomputing.Weevaluatetherobustnessofthesynchronizationinthelowresolutionandintheheighresolution.

8290-39, Session 9

A content-adaptive scheme for reduced-complexity, multiview video codingA.Avci,J.DeCock,R.Beernaert,J.DeSmet,Univ.Gent(Belgium);Y.Meuret,VrijeUniv.Brussel(Belgium);P.Lambert,H.DeSmet,Univ.Gent(Belgium)

Disparityestimationisahighlycomplexandtimeconsumingprocessoftheencoder.Sincemultipleviewstakenfroma2Dcameraarrayneedtobecodedateverytimeinstance,thecomplexityoftheencoderplaysanimportantrolebesidestherate-distortionperformanceoftheencoder.Becauseofthestronggeometricalcorrespondencebetweenviews,anewframetypecalledDframehasbeenproposedinour

previousworktominimizethecomplexityoftheencoder.ByemployingDframesinplaceofsomeofthePframesinthepredictionstructure,significantcomplexitygainscanbeachievedifthethresholdvalue,whichisakeystoneelementtoadjustthecomplexityatthecostofqualityand/orbit-rate,isselectedwisely.Inthiswork,anewadaptivemethodtocalculatethethresholdvalueautomaticallyfromexistinginformationduringtheencodingprocessispresented.Inthismethod,thethresholdvaluesaregeneratedforeachblockofeachDframetoincreasetheaccuracy.Thealgorithmisappliedtoseveralimagesetsand20.6%complexitygainisachievedbyusingtheautomaticallygeneratedthresholdvalueswithoutcompromisingthequalityandbit-rate.

Conference 8290


Conference 8291: Human Vision and Electronic Imaging XVIIMonday-Thursday23-26January2012

Part of Proceedings of SPIE Vol. 8291 Human Vision and Electronic Imaging XVII

8291-01, Session 1

Image representations learned for natural images and a non-linear maximum synthesis rule with applications in computational photographyJ.Hocke,E.Barth,T.Martinetz,Univ.zuLübeck(Germany)

Withsparsecodingthelearningofabasisisnon-linear,butthebasiselementsarestilllinearlycombinedtoformanimage.Wehereproposetouseanonlinearsynthesisrule,whichisdefinedsuchthatateachlocationintheimagethepoint-wisemaximumoverallbasiselementsisusedtogeneratetheimage.Onlyrecentlyanattemptwasmadetolearnabasisthatisoptimalgiventhissynthesisrule(Lückeetal.whocoinedthisMaximalCausesAnalysis-MCA).Onecanarguethatsuchanon-linearsynthesisruleismorerealisticfornaturalimagesbecauseitwouldgeneratetheimagefromitsmainfeatures.Thechallengethenistoproperlylearnthesemainfeaturesgivenaparticularsetofimages.

Weherepresenttwoimportantextensions.First,weproposeanovelandmoreefficientlearningschemebasedtheSparseCodingNeuralGasalgorithm,whichwasdesignedtolearnsparsebasesforlinearimagesynthesisandisadaptedheretothenon-linearMCAsynthesisrule.

Second,weapplythenovelalgorithmtoclassicalproblemsinComputationalPhotographysuchasimagedeblurringandimagecompletion.

8291-02, Session 1

How to make a small phone camera shoot like a big DSLR: creating and fusing multi-modal exposure seriesT.Binder,F.Kriener,C.Wichner,M.Wille,NikSoftwareGmbH(Germany);M.Wellner,T.Kaester,E.Barth,PatternRecognitionLab.(Germany)

Inthefirstpartwepresentthedesignofanimagingsystemconsistingoftwocameras,onebeingahigh-qualityDSLR,andtheotheracellphonecamera(theNokiaN900Frankencamera).ThecamerasareconnectedviaUSB.TheFrankencameramaytakemanyexposureswithvaryingparameters,whiletheDSLRtakesareferenceimageofthescene.

Sincethesystemisdesignedtotakemanyexposuresofthesamescene,astablemechanicalcouplingofthecamerasandtheuseofatripodarerequired.

Thesecondpartdealswiththeprocessingoftheresultingcellphoneexposurestack,whichisfirstalignedandthenfusedintoasingleimage.Thereforethewholeprocessofproducingthesingleimageiscontrolledbytwoparametersets:theacquisitionparametersoftheexposurestack(numberofexposures,shutterspeedandfocussettingofeachexposure)andthefusionparameters.

Thefusionalgorithmisbasedonadual-treecomplexwavelettransform(DCWT)andinvolvesbothlinearsummationoftheDCWTcoefficientsandnonlineartransformsofthecoefficients.Thenonlinearitiesenforcesparsenessonthecoefficientsandimplementdivisivenormalization.

Finally,thefusedimageiscomparedtothereferenceimagetakenwiththeDSLR.Aperceptualqualityindexisdefined,whichexpressestheperceivedsimilarityofthefusedandthereferenceimage.Wethenusetheindextooptimizeboththeimageacquisitionandthefusionparameters.

Thenoveltyofourapproachisduetothejointmultidimensionalfusionofimageswithdifferentacquisitionparameters,thefactthatweuseahigh-qualityreferenceimageandaperceptualqualityindex,andthejointmultidimensionaloptimizationoftheacquisitionandfusionparameters.Theultimategoalistoimprovetheimagequalityofcheapandsmallcamerasbyusingperceptuallyoptimizedacquisitionandfusionalgorithms.

8291-03, Session 1

Metric image mosaics with bounded errorD.Ghosh,J.Tumblin,NorthwesternUniv.(UnitedStates)

Thispaperdescribesanopen-sourcemulti-imagestitchingsystemthatconstructsmetricplanarmosaicssuitableforbuildingvisualarchivesofmostly-flatmuseumartifactssuchaspaintings,frescoes,andfloors.Thesystemoutperformsexistingpanorama-makingsoftwaresuchasMicrosoft’sICE,BrownandLowe’s‘AutoStitch’,orevenCanonUtilties‘PhotoStitch’thatlackstricterrorboundsandrelyonhomographiesappliedtosingle-viewpointimagesets.Itsadvantagesinclude:a)Thenew‘mosaic-making’systemacceptsinputimagesfromunknown,widelyvaryingviewpointsincludinglow-angle(or‘rakingangle’)viewpointsandlighting,makingitsuitableforplanarsurfacesfartoolargeortoocomplexforasingleviewpoint(e.g.floorofacathedral;alargepaintingormural);b)Themosaicresultensurespixeldistancesaredirectlyproportionaltoflat-surfacedistances.Thesystemimposesstricterror-boundsonmosaic-resultimageerrorvs.point-correspondencemismatchesmeasuredinthesourceimages.c)Itsall-at-onceoptimizer/solverincludesa7-termparametriclens-distortionmodelthatcorrectsforlow-costuncalibratedzoomlenseswithunknownsettings,andd)writtenwithintheSourceForgeHUGIN/PanoToolsopen-sourcepanorama-makersystem,thenew‘mosaic-maker’leavestheGUIunchangedandpermitsextensivescriptingcapabilitiessuitableforextremelylargeimagesets.Unlikemostpanorama-makers,thismosaic-makerremovesallforeshorteningfromthesourceimagesbyachainofphysically-meaningful3Dparametrictransformations(roll,pitch,yaw,tilt,spin)insteadofconventionalhomographies.Thismore-elaboratebutcomprehensibleprocessprovidesgreaterflexibility--userscanindividuallyenableeachdegree-of-freedomforeachsourceimageifdesired.Further2Dparametrictransforms(rotate,scale,translate)arrangethemosaic,andthesystemoptimizesallofthesourceimage’slens-correction,2Dand3Dparametersatonce.Afterextensivetestswithsyntheticsourceimagestotestsystemrobustness,weassembledhundredsofmetricmosaicsofonelargepaintingattheArtInstituteofChicago.Builtfromavastphoto-archive(65,000images)oftiled,variously-lit,raking-anglephotos,theofassembledmosaicsrevealedseveralpreviouslyunknownoverpaintedfeatures(‘pentimenti’)thathelpedclarifythehistoryofthisimportantartwork.

8291-04, Session 1

Single-lens 3D camera with extended depth-of-fieldC.Perwass,L.Wietzke,RaytrixGmbH(Germany)

Acameradesignthathasregainedmuchpopularityinthelastcoupleofyearsisthatoftheplenopticcamerawhereamicrolensarrayisplacedinfrontoftheimagesensor.Takingpictureswithaplenopticcameraoffersawholenewwaytoperceiveimages.Suchimagesarenotstatic,butallowtheusertomodifytheimageafterithasbeenrecorded.Theusercaninteractivelychangethefocus,thepointofviewandtheperceiveddepthoffield.

Whileastandardplenopticcamera’seffectiveresolutionequalsthe


numberofmicrolenses,wepresentanadvancedtechnologywithaneffectiveresolutionofuptoonequarterofthesensorresolutionandanactuallyextendeddepthoffield.Notethatastandardplenopticcameradoesnotextendthedepthoffieldascomparedtoastandardcamerausingthesamelensandthesameaperture.ThemicrolenstechnologydevelopedbyRaytrix,ontheotherhand,increasesthedepthoffieldbyuptoafactorof6.Thisisachievedthroughspeciallydesignedmicrolensarraysthatconsistofmicrolensesofdifferentfocallengths.

8291-05, Session 1

3D holoscopic video imaging systemJ.H.Steurer,ArriCineTechnik(Germany)

TheEUfundedproject3DVIVANTisaimingtocaptureeventsautomaticallyinthree-dimensional(3D)spaceanddeliverthemtohomeviewersforrealistic,interactive,fatigue-freeandimmersiveplaybackwithoutglasses.Thecentralideaofthisprojectisanewformofmultimediacontent(3Dvideoandaudio).3Dholoscopicimaging(alsoreferredtoasIntegralImaging)isatechniqueforcreatingfullcolor3Dopticalmodelsthatexistinspaceindependentlyoftheviewer.

The3DVIVANTprojectisdevelopingnewtechnologiesforcaptureanddisplayof3Dcontent.Forthecapture,3DVIVANTistakingfulladvantageof3Dholoscopicimagingtechnology,whereasinglecameraisrequired.Henceinthisprojectaworldfirstsingleapertureultra-highdefinition3Dholoscopicimagingcameraisbeingconstructedwhichwillpermitlivecaptureof3Dcontent.Forthedisplay,theprojectistakingadvantageoftheHolovisiosystemtoprovideanimmersive,ultra-highresolutionpresentationof3Dcontent.

3DVIVANTwillmakeasignificantnumberoftechnologicaladvances,whichdemonstratethecapabilitiesofthe3Dholoscopicvideotechnologyasanalternative3Dvisionsystemespeciallyinthefieldofentertainment.

8291-06, Session 2

Predictive rendering for accurate material perceptionK.Bala,CornellUniv.(UnitedStates)

Computergraphicsrenderingisusedtosimulatetheappearanceofobjectsandmaterialsinawiderangeofapplications.Designersandmanufacturersneedtorelyentirelyonrenderedimagestocorrectlyrecognizeanddifferentiatebetweendifferenttypesoffabrics,paintfinishes,plastics,andmetals.Thus,therenderingalgorithmsproducingtheseimagesmustbepredictiveoftheappearanceofmaterials.

Whileprogresshasbeenmadeonpredictiverenderingformetalsandplastic,predictiverenderingofclothandfabricsremainsparticularlychallenging.Thestructureofclothcomplicatesthereflectanceoflight.Further,irregularitiesinthestructureofclotharedifficulttomodel,butarevisuallyimportant.Forexample,thestructureofthepileinvelvetaffectsitsfuzzyappearance,anditscharacteristichighlightsatgrazingangles.

Forcloth,predictiverenderingrequirestheacquisitionofdetailedmodelsrepresentingthemacro-scaleandmeso-scalestructureoffibersandyarns,betteropticalmodels,andvolumetriclighttransportalgorithmsthatsimulatethescatteringoflightwithinthevolumeofstructuredfibersandyarns.Wehavedevelopednewscatteringmodelsandlighttransportalgorithmsthatcreatehighlyrealisticimagesofcloth.Buttheyrequiredetailed3Dstructuralmodelsasinput.

Weintroduceanentirelydifferentapproachtobuildingvolumeappearancemodelsforcloth,byusingX-raycomputedtomography(CT)toacquiredetailedgeometricstructure.Forexample,forwovencloth,thesescansautomaticallycapturethestructureoftheclothwithadetailedviewoftheinterlacedyarnsandtheircomponentfibers,showingexactlyhowthefibersareorientedandhowtheyarnsarepositioned.ThedensityandorientationinformationfromtheseCTscansisaugmentedwithopticalproperties(albedoandgloss)inferredfromareferencephotograph.Thispowerfulapproachcaneasilyproducevolumeappearancemodelswithextremedetailincloseup,andatlargerscales,withthedistinctivetexturesandhighlightsofa

rangeofverydifferentfabricslikesatin,velvet,andwool.

Couplingdetailedstructuralinformationwithvolumetriclighttransportalgorithmsisafirststeptowardspredictiverenderingforcloth.

8291-07, Session 2

From color to appearance in the real worldF.Lamy,X-Rite,Inc.(UnitedStates)

Xrite’sdeclaredambitionistocreateadigitalecosystemforappearance.;adauntingchallengethathasmanydimensionsandhasprovensomassivethatallpreviousattemptseffortshavefailedsofar.Afterhavinginvested3yearsinexploringtheproblem,wecannowdeliverthefirstelementsofanswersandthepracticalpathtotacklethismassiveundertaking.

Wewillexplorethepracticalimplicationsoftheintermediationof2stagesbetweencolorandfullappearance:extendedcolorandaugmentedcolor,andhowthesesteps,rootedintherealitiesoftheecosystemtheyserveareconstitutingvectorsandenablersofamoreeffectivetransition.

Wewillsurveytheroadmapimplicationinthedesignformeasurementandcaptureinstrumentations,packagingdigitalformatsanddeliveryinfrastructureaswellasrenderinganddisplaydevicesthatwillenabletruevaluecreationbuiltonappearanceattributes.

8291-08, Session 2

Towards material literacy: making perception and knowledge of materials explicit through visual and verbal documentation of material engagementA.Lehmann,UtrechtUniv.(Netherlands)

Noabstractavailable

8291-09, Session 2

Mixing material modesS.C.Pont,TechnischeUniv.Delft(Netherlands)

Noabstractavailable

8291-10, Session 2

Tangible display systems: bringing virtual objects into the real worldJ.A.Ferwerda,RochesterInstituteofTechnology(UnitedStates)

Noabstractavailable

8291-11, Session 3

Quality estimation for images and video with different spatial resolutionsA.M.Demirtas,Univ.ofCalifornia,Irvine(UnitedStates);A.R.Reibman,AT&TLabs.Research(UnitedStates)

Full-reference(FR)qualityestimators(QEs)forimagesandvideoaretypicallydesignedassumingthatthedisplayed,degradedimagehasthesamespatialresolutionastheoriginal,referenceimage.No-reference(NR)QEsusenoknowledgeaboutthereferenceimagetoassessqualityofthedisplayedimage.However,inmanypracticalsystems,areferenceimagemaybeavailablethathasadifferentspatialresolutionthanthedisplayedimage.

Conference 8291


Inthispaper,weexploreobjectivequalityestimationwhenthedisplayedimagetobeevaluatedhasadifferentspatialresolutionthanthereferenceimage.WebeginbyidentifyingarangeofpotentialweaknessesthatmightbepresentinaQEdesignedforthissituation.Then,wecreatepairsofimageswithpotentialFalseTies,inwhichaQEestimatesthetwoimageshaveequalqualitywhileviewersdisagree.Armedwiththeknowledgeofpotentialflaws,wethenpresenteffectivedesignstrategiesofQEsforthissituation.

8291-12, Session 3

Automatic parameter prediction for image denoising algorithms using perceptual quality featuresA.Mittal,A.K.Moorthy,A.C.Bovik,TheUniv.ofTexasatAustin(UnitedStates)

Weproposeaperceptionbasedapproachtocreateablindimagedenoisingalgorithm,bycombiningblindparameterestimationwithastate-of-the-artdenoisingalgorithm.Ourapproachtoblindparameterestimationislearning-based,whereinputparameterisestimatedusingstatisticalpropertiesofnaturalscenes.Thefeaturesarebasedonmarginalstatisticsoflocalnormalizedluminancesignalsanddistributionofpairwiseproductsofadjacentnormalizedluminancesignals.Weshowthatourblindparameterestimationprocedure“statistically”giveshigherqualityofdenoisedimagesthanthebaselineonanexhaustivesetof3000imageswithawiderangeofnoisevariancevalues.

8291-13, Session 3

Viewer preferences for classes of noise removal algorithms for high-definition contentS.G.Deshpande,SharpLabs.ofAmerica,Inc.(UnitedStates)

Perceivedvideoqualitystudieswereperformedonanumberofkeyclassesofnoiseremovalalgorithmstodetermineviewerpreference.Thenoiseremovalalgorithmclassesrepresentincreaseincomplexityfromlinearfiltertononlinearfiltertoadaptivefiltertospatio-temporalfilter.Thesubjectiveresultsquantifytheperceivedqualityimprovementsthatcanbeobtainedwithincreasingcomplexity.Thespecificalgorithmclassestestedinclude:linearspatialonechannelfilter,nonlinearspatialtwo-channelfilter,adaptivenonlinearspatialfilter,multi-framespatio-temporaladaptivefilter.AllalgorithmswereappliedonfullHD(1080P)content.Oursubjectiveresultsshowthatspatio-temporal(multi-frame)noiseremovalalgorithmperformsbestamongstthevariousalgorithmclasses.Thespatio-temporalalgorithmimprovementcomparedtooriginalvideosequencesisstatisticallysignificant.Ontheaverage,noise-removedvideosequencesarepreferredoveroriginal(noisy)videosequences.TheAdaptivebilateralandnon-adaptivebilateraltwochannelnoiseremovalalgorithmsperformsimilarlyontheaveragethussuggestingthatanon-adaptiveparametertunedalgorithmmaybeadequate.

8291-14, Session 3

Image quality assessment in the low-quality regimeG.O.Pinto,S.S.Hemami,CornellUniv.(UnitedStates)

ImageQualityAssessment(IQA)algorithmsaretraditionallydesignedtooperateovertheentirequalityrange,fromverylowqualityimagestovisuallylosslessimages.However,many,ifnotall,imagingapplicationsarenotdesignedtooperateovertheentirerange;forexample,digitalcameraimageoutputsdonotproduceextremelycompressedimages,andaregenerallyonlyofextremelylowqualitywhenoperatorerrorhasoccurred.Assuch,designingandevenoptimizingaqualityestimatorfortheentirequalityrangemayunreasonablylimittheestimator’s

performanceinmorerealisticrangesofoperation.Asthispaperwillshow,theperformancesoftraditionalqualityestimatorsovertheentirequalityrangediffergreatlyfromtheperformancesonlowqualityimages.Forexample,intheLIVEdatabase,theSSIMestimatorhasaSpearmanrank-ordercorrelation(SROC)coefficientof0.91fortheentiredatabase,butithasacoefficientof0.50inthelowqualityregime.

8291-15, Session 4

The question of simultaneity in multisensory integrationM.E.McCourt,L.Leone,NorthDakotaStateUniv.(UnitedStates)

Earlyreportsofaudiovisual(AV)multisensoryintegration(MI)indicatedthatunisensorystimulimustcausesimultaneousphysiologicalresponsestoproducedecreasesinmeanreactiontime(RT).WhenunisensoryRTswereunequaltheonsetofthestimuluselicitingthefasterRThadtobedelayedrelativetotheonsetofthestimuluselicitingtheslowerRT,forMItooccur.The“temporalrule”statesthatMIdependsonthetemporalproximityoftheunisensorystimuli,theneuralresponsestowhichmustfallwithinawindowofintegration.EcologicalvaliditydemandsthatMIshouldoccuronlyforsimultaneousevents(whichmaygiverisetonon-simultaneousneuralactivations).Ontheotherhand,instancesofneuralsimultaneitywhichareunrelatedtounitarymultisensoryeventsmustberejectedasspurious.WemeasuredAVMIunderfullydark-adaptedconditionsforvisualstimulithatwereeitherweak(scotopic;longRT)orstrong(photopic;shortRT).Auditorystimulusintensitywasconstant.Despitea190msincreaseinmeanRTtothescotopicstimulus,facilitativeMIinbothconditionsoccurredexclusivelyatanAVSOAof0ms.Themechanismswherebylargevariationsinresponselatencyarisingfromchangesinstimulusintensityarediscountedareconsidered.

8291-16, Session 4

Multisensory integration deficits in children with autism spectrum disorders: a behavioral and ERP investigationC.D.Saron,Y.Takarae,M.Beransky,D.M.Horton,A.Stark,A.Schneider,F.Vieira,S.M.Rivera,Univ.ofCalifornia,Davis(UnitedStates)

Multisensoryintegration(MSI),thecombinationofvarioussensestoformasingleintegratedexperienceoftheworld,isessentialtoeverydaylife.Itisnowwidelyclaimedbycliniciansandresearchersalikethatdeficitsinsensoryprocessing,includingMSI,arepartofthephenotypeofautismspectrumdisorders(ASD).ThisstudyexaminedMSIinchildren(10-14yrs.)withASDandage-matchedtypicallydeveloping(TD)childrenusingofdense-channelarrayevent-relatedpotentials(ERPs)elicitedinresponsetovisual(V),auditory(A),andsomatosensory(S)stimulideliveredaloneorinsimultaneouscombination.Thetaskwasthedetectionofallstimulusevents(simplereactiontime(RT)).Differencesbetweenmultisensory(MS)andthesummedresponsesofunisensorystimuli(US)wereusedtoestimateMSI.ResultsdemonstratedalackofRTfacilitationforauditory/tactilemultisensorystimulifortheASDgroup(N=9/group).UnisensoryERPsweregenerallysimilarbetweentheTDandASDgroups.ERPstoMSstimulidifferedbetweengroups.TheTDpatterntypicallyshoweddecreasedearlyactivationsinMSconditionscomparedwithsummedUSdata.ThesedatasuggestspecificdeficitsinMSIofchildrenwithASDandareconsistentwiththeviewthatlow-levelsensoryprocessingandintegrationcontributetothecomplexphenotypeofASD.

8291-17, Session 4

The spatiotopic ‘visual’ cortex of the blindL.T.Likova,TheSmith-KettlewellEyeResearchInstitute(UnitedStates)

Conference 8291


Visualcortexactivityintheblindhasbeenshowninsensorytasks.Canitbeactivatedinmemorytasks?Ourrecentresultsinshort-termblindfoldedsubjectsimplythathumanprimaryvisualcortex(V1)mayoperateasamodality-independent‘sketchpad’forworkingmemory(Likova,2010).Wenowraisethequestionofwhetherunderlong-termvisualdeprivationV1isalsoemployedinnon-visualmemory,inparticularincongenitallyblindsubjects,whohaveneverhadvisualstimulationtoguidethedevelopmentofthevisualareaorganization.Theoutcomehasimplicationsforanemergingreconceptualizationoftheprinciplesofbrainarchitectureanditsreorganizationundersensorydeprivation.Methods:WeusedanovelfMRIdrawingparadigmincongenitallyandlateblind,comparedwithsightedandblindfoldedsubjectsinthreeconditionsof20sduration,separatedby20srest-intervals,(i)TactileExploration:raised-lineimagesexploredandmemorized;(ii)TactileMemoryDrawing:drawingtheexploredimagefrommemory;(iii)Scribble:mindlessdrawingmovementswithnomemorycomponent.ResultsandConclusions:V1wasstronglyactivatedforTactileMemoryDrawingandTactileExplorationinthesetotallyblindsubjects.Remarkably,eveninthememorytask,themappingofV1activationlargelycorrespondedtotheangularprojectionofthetactilestimulirelativetotheego-center(i.e.,theeffectivevisualangle);beyondthisprojectiveboundary,peripheralV1signalsweredramaticallyreducedorsuppressed.Thematchingextentoftheactivationinthecongenitallyblindrulesoutvision-basedexplanatorymechanisms,andsupportsthemoreradicalideaofV1asanmodality-independent‘sketchpad’,whosemappingscalestotheprojectivedimensionsofobjectsexploredintheperipersonalspace.Inhigheroccipitalareasthedifferentialpatternofengagementacrossthesubjectcategoriessuggestsprogressiveneuroplasticreorganizationasvisualdeprivationisprolonged,inaccordwithanevolvingcross-modalutilizationoftheoccipitalcortextomaximizeitsinherentfunctionalpotential.

Acknowledgements:NSF/SLCgranttoLLikova

8291-18, Session 4

Acoustic-tactile rendering of visual informationP.M.Silva,T.N.Pappas,NorthwesternUniv.(UnitedStates);J.Atkins,J.E.West,TheJohnsHopkinsUniv.(UnitedStates)

Noabstractavailable

8291-19, Session 5A

Apparent stereo: the Cornsweet illusion can enhance perceived depthP.Didyk,Max-Planck-InstitutfürInformatik(Germany);T.Ritschel,E.Eisemann,TelecomParisTech(France);K.Myszkowski,H.Seidel,Max-Planck-InstitutfürInformatik(Germany)

Itisbothatechnicalandanartisticchallengetodepictthree-dimensionalspaceusingaflattwo-dimensionalmedium.Fittingwithinthelimitsofagivendisplaytechnologyandatthesametimeachievingcomfortableviewingcanrequiremodificationstostereocontent,e.g.,toflattenorevenremovebinoculardisparity.However,insuchscenarioscrucialdepthdetailsmightgetlost.Recently,abackward-compatiblestereotechniquewasproposed,whichbuildsupontheCraik-O’Brien-Cornsweeteffect-avisualillusion,whichwasappliedtobrightnessinCGbefore.TheapproachreplacesdisparitybyCornsweetprofilesindisparitywheredepthdiscontinuitiesoccur.Thisallowsforthereductionoftheoveralldisparityrangewhilestillconveyingacomfortablestereoexperience.ThepresentworkextendsthepreviousideabyshowingthatCornsweetprofilescanalsobeusedtoenhancethe3Dimpressioninregionswherethedisparityrangewascompressed.Auserstudymeasurestheperformanceofbackward-compatiblestereoaswellasourdisparityenhancement.

8291-20, Session 5A

Perceived depth of multi parallel, overlapping, transparent, stereoscopic surfacesS.Aida,K.Shimono,TokyoUniv.ofMarineScienceandTechnology(Japan);W.J.Tam,CommunicationsResearchCtr.Canada(Canada)

Thegeometricrelationalexpressionofhorizontaldisparity,viewingdistance,anddepthmagnitudebetweenobjectsinstereopsissuggeststhat,foragivenviewingdistance,themagnitudeofperceiveddepthofobjectswouldbethesameaslongasthedisparitymagnitudesarethesame.However,wefoundthatthisisnotnecessarilythecaseforrandomdotstereogramsdepictingtwoormoreparallel-transparent-overlapping-stereoscopic-surfaces(POTS).Datafromtwoexperimentsindicatedthat,whenthestimulussizeisrelativelylarge(e.g.,13x20arcdeg),themagnitudeofreproduceddepthbetweentwostereo-surfacesofatwo-POTSconfigurationcanbelargerthanthatbetweenanidenticalpairofstereo-surfacesbutwithanadditionalstereo-surfacelocatedbetweenthepair.Infollow-upexperiments,dotdensity,thesizeoftherandomdotstereogramsandthenumberofstereo-surfaceslyingbetweenthetwooutermoststereo-surfacesweremanipulated.Overall,theresultsindicateadifferenceofapproximately10%inthereproduceddepth,dependingonwhetherthereisanadditionalstereo-surfacebetweenthetwosurfacesofatwo-POTSconfigurationornot.Theresultsarediscussedintermsofglobalstereopsiswhich“operates”forrelativelylargestimulussizes(suchas13x20arcdeg).

8291-21, Session 5A

Diagnosing perceptual distortion present in group stereoscope viewingM.M.Burton,B.B.Pollock,J.W.Kelly,S.B.Gilbert,E.Winer,IowaStateUniv.(UnitedStates);J.delaCruz,U.S.ArmyRDECOM/STTC(UnitedStates)

Stereoscopicvirtualenvironmentsarefrequentlyemployedtoimmerseviewersinvirtualrealityvisualizations,simulations,andtrainingsystems.Stereosystemsrenderthescenegraphfromtheperspectiveofasinglepersonorstaticviewpointwhichcanbedefinedasaleader,whileothersseeinaccurateanddistortedviewsfromafollowerposition.Greaterunderstandingofdistortionfromthefollowerperspectivecanleadtomoreeffectivemulti-viewsolutionsandconfigurationsofleader-followersystems.

AnexperimentatIowaStateUniversitywasconductedtogaininsightintotheperceptualdistortionpresentinleader-followersystemswheremultipleusersviewthesamestereoscopic,projection-basedvirtualenvironment.Theexperimentwascomprisedoftasksreportingquantitiesofanglesindegreesfrompositionsparalleltothescreenandreportingthedepthdimensionofarectangleatpositionscenteredandperpendiculartothescreen.Resultsfromthisstudyindicatedarelationshipofangulardistortionwithleader-followeroffsetsparalleltothescreenanddistancetothescreenaffectsthequantityofdepthdistortioninadditiontoperpendicularoffsetsfromtherenderedviewpoint.Furtherworkwillattempttounderstandtheeffectsofthisdistortionongroupcollaborationandfurtherdeveloprelationshipsofdistortionandleader-followeroffsets.

8291-34, Session 5A

Visual discomfort and the timing of vergence-accommodation conflictsJ.Kim,D.Kane,M.S.Banks,Univ.ofCalifornia,Berkeley(UnitedStates)

Stereo3D(S3D)displayscausevisualdiscomfortinmanyviewers,soitisveryimportanttounderstandthecausesofthediscomfortsoonecanminimizeit.Weknowthatthevergence-accommodationconflict

Conference 8291


causesvisualdiscomfortandeventuallyvisualfatigue.Vergenceandaccommodationbothhavephasicandtoniccomponentsthatadaptatdifferentrates.Todetermineifthedifferentadaptationratesinphasicandtoniccomponentsaffectvisualdiscomfort,wevariedthetemporalpropertiesofthevergence-accommodationconflictandassesseddiscomfort.

Observersviewedrandom-dotstereogramsinanovelvolumetricdisplaythatallowsustoindependentlymanipulatethestimulustovergenceandthestimulustoaccommodation.Thereweretwomainconditions:naturalviewing(cues-consistent)andS3Dviewing(cues-inconsistent).Thestimulitovergenceandaccommodationwerevariedatdifferentthreedifferentrates,presentedindifferentsessions.

WefoundthatobserversexperiencedworsesymptomsintheS3Dviewingthaninthenaturalviewing.Wealsofoundthatmorefrequentchangesinthestimulustovergenceyieldedworsesymptoms.Thus,thetime-averagemagnitudeofthevergence-accommodationconflictisnotthesolepredictoroftheamountofdiscomfort.Italsomattershowtheconflictisdistributedovertime.

8291-22, Session 5B

Measuring 3D discomfort from vertical and torsional disparities in natural imagesC.W.Tyler,L.T.Likova,TheSmith-KettlewellEyeResearchInstitute(UnitedStates);K.Atanassov,V.Ramachandra,S.Goma,Qualcomm(UnitedStates)

Thetwomajoraspectsofcameramisalignmentthatcausevisualdiscomfortwhenviewingimagesona3Ddisplayareverticalandtorsionaldisparities.Thegoalofthestudywastodeterminethediscomfortrangesforthekindsofnaturalimagethatpeoplearelikelytotakewith3Dcameras.Theresultsshowedthat,asaruleofthumb,verticaldisparitybetweenthetwoeyesneedstobeheldbelowabout15arcmin,oraquarterofadegree,tobewithinanacceptablerangeofdiscomfort,whilehumanvisionisrelativelyinsensitivetotorsionaldisparitiesbelowabout30deg.

8291-35, Session 5B

Visual fatigue versus eye-movementsC.Vienne,L.Blondé,D.Doyen,TechnicolorS.A.(France)

Observing3DcontentonacinemaorTVscreenpotentiallygeneratesfatigue.Inresearch,experienceofvisualsymptomsfollowingtheobservationofstereo-contentisusuallyassessedthankstoquestionnairesandsubjectivereports.Weattemptedtoexploretheoccurrenceofvisualfatigueusingmoreobjectivemethods,namelybyusingbinoculareye-trackingandpsychophysics.Amainobjectivewastostudytheemergenceofvisualfatigueinrelationwitheye-movementknowingthestimulationoftheoculomotorsystemanditsresponse.Wedesignedanexperimentinwhichparticipantswereaskedtoperformarepeatedvergenceefforttask,justfollowedbya3Dspaceperceptiontask.Participants’eyemovementswererecordedduringthewholesessionusinganeye-trackingsystem.Theanalysisrevealedthattheperceptionof3Dshapewasgraduallyaffectedbytheintensityofthevergenceefforttask.Theeffectonstereo-estimationwasactuallyduetotheoccurrenceofvisualfatigue,3Dobjectsareperceivedflatter.ResultsonthesubjectivereportsofSSQrevealedthatoculomotorfactorswerepredominantinthevisualsymptoms.Inaddition,someeffectsandcorrelationsonthemicro-saccadicratewereobtained.Itoffersaperspectivetocharacterizeobjectivelyvisualfatiguewhenwatching3Dcontentonstereoscopicdisplays.

8291-36, Session 5B

Visual comfort: stereoscopic objects moving in the horizontal and mid-sagittal planesW.J.Tam,F.Speranza,C.Vázquez,R.Renaud,CommunicationsResearchCtr.Canada(Canada);N.Hur,Electronicsand

TelecommunicationsResearchInstitute(Korea,Republicof)

Thepresentstudyexaminestheeffectofmotionofstereoscopicobjectsonvisualcomfortusingcomputer-generatedvideosequencesbasedonaparallel-camerasetting.Inthefirstexperiment,thevisualcomfortofasetofstereoscopicobjectsmovinginthez-planewithdifferentdisparitiesandvelocitiesisexamined.Threetypesofmovementinthez-planewereincluded:bothuncrossedandcrosseddisparities;crossedonly;anduncrossedonly.Inasecondexperiment,thevisualcomfortofasetofstereoscopicobjectsmovingatagivendepthinthehorizontalplaneisexamined.Alsointhiscase,threetypesofmovementinthez-planewereincluded:bothuncrossedandcrosseddisparities;crossedonly;anduncrossedonly.Inbothexperiments,themovements(alongthez-planeforExp.1andalongthex-planeforExp.2)involvedthreelevelsofvelocityandfivelevelsofdisparity.Preliminaryresultsappeartoconfirmthatvisualcomfortisinfluencedmorebyobjectvelocitythanbydisparitymagnitude(fortherangeofmagnitudesused);aninteractionofvelocityanddisparityisalsoapparent.Therefore,thepreliminaryresultsappeartoconfirmthatchangesindisparityovertimeareasignificantcontributortovisualcomfortofstereoscopicvideoimages;theeffectappearstobepresentalsowhenthechangesareconfinedwithinthegenerallyacceptedcomfortzone.

8291-37, Session 5B

Visual discomfort with stereo 3D displays when the head is not uprightD.Kane,R.Held,M.Banks,Univ.ofCalifornia,Berkeley(UnitedStates)

Stereo3D(S3D)displayscancausevisualdiscomfort.Inpart,discomfortisduetothewell-knownvergence-accommodationconflict.Additionally,discomfortmaybecausedbyviewingS3Ddisplayswiththeheadrotatedtotheside(rollrotation).Headrollconvertshorizontaldisparityonthescreenintoverticaldisparityontheretinathatinturnshouldelicitverticalvergenceeyemovementstominimizeverticaldisparity.Becausesuchmovementsaresmallinnaturalviewing,thelargeronesrequiredwithheadrollarelikelytocausediscomfort.WeinvestigatedthisbysimulatingheadrollinaS3Ddisplay.Wekepttheobservers’headupright,butrotatedthestimulus.Thestimuluswasarandom-dotstereogramdepictingsinusoidaldepthcorrugations.Itcontainedeithernovariationinoveralldisparityoravariationinoveralldisparityovertime.Observersindicatedtheamountofvisualdiscomfortexperiencedwhenviewingthesestimuli.Theyreportedworsesymptomsinthesimulatedhead-rollconditionsandwhenthedisparitypedestalchangedbyagreateramount.Weattributetheincreasedsymptomstotheverticalvergenceeyemovementselicitedbyheadroll.


Tracking of aging process by multiple 3D scans analysisE.Bunsch,TheWilanówPalaceMuseum(Poland);R.Sitnik,WarsawUniv.ofTechnology(Poland)

Currently,alotofdifferent3Dscanningdevicesareusedfor3Dacquisitionofartartifactsurfaceshapeandcolor.Eachofthemhasdifferenttechnicalparametersstartingfrommeasurementprinciple(structuredlight,lasertriangulation,interferometry,holography)andendingonparameterslikemeasurementvolumesize,spatialresolutionandprecisionofoutputdataandcolorinformation.Someofthe3Dscannerscangrabadditionalinformationlikesurfacenormalvectors,BRDFdistribution,multispectralcolor.Inthispaper,weplantopresentresultsofthemeasurementswithselectedsamplingdensitiestogetherwithdiscussionoftheproblemofrecognitionandassessmentoftheagingprocess.Wefocusourinterestonfeaturesthatareimportantfortheartconservatorstodefinestateofpreservationoftheobjectaswellastoassesschangesonthesurfacefromlastandpreviousmeasurement.Alsodifferentmaterialsandfinishingtechniquesrequiresdifferentalgorithmsfordetectionandlocalizationofagingchanges.

Conference 8291


Inthispaperweconsiderexemplarystonesamplestovisualizewhatobjectfeaturescanbedetectedandtrackedduringagingprocess.Thechangesinsandstonesurfaceshape,affectedbysaltweathering,willbepresentedaswellaspossibilitiesofidentificationofsurfacedegradationonrealobject(gardenreliefmadeinsandstone).


Aesthetics and entropy: optimization of brightness distributionM.R.V.Sahyun,Consultant(UnitedStates)

Thepurposeofthisworkistosuggestdirectionsforutilizingimagestatisticstoguideoptimizationofbrightnessdistributions,towardsagoalofcompletesystematizationofimageprocessingtoachieveapurelyaestheticobjective,whetherbyapplicationofhumanormachineintelligence,withentropyasaresponsemetric.Westartwithasurveyofclassicpictorialphotographs,proceedtoaheuristictheoreticaltreatmentofthebrightnessdistributionfunction,andfollowwithseveralpictorialillustrationsoftheproposedapproachtoentropy-guidedbrightnessoptimization.


A novel visualization tool for art history: a method for automated colorization of black and white archival photographs of works of artS.Tsaftaris,InstituteforAdvancedStudiesLucca(Italy)andNorthwesternUniv.(UnitedStates);F.Casadio,K.Lister,I.Fiedler,G.Gautier,TheArtInstituteofChicago(UnitedStates);J.Andral,MuséePicasso(France);S.D’Alessandro,TheArtInstituteofChicago(UnitedStates);A.Katsaggelos,NorthwesternUniv.(UnitedStates)

ColorizingarchivalB&Wphotographsofworksofart,helpsconservatorsandarthistoriansvisualizeartists’intentionasexpressedintheirworkingprocess.Whileconservatorstypicallyusecommercialsoftwaretocolorizeimagesbyapplyingflatareasofcolorinasomewhatsubjectiveway,theproposedmethodallowsspreadingobjectivecolorinformationinacomputer-controlledfashion,preservingbrushworkandshading.Thisnewcolorizationmethodologyusespredefinedhintsrecoveredfromremnantsofpreviouscolorsstillvisibleinthepresentstate,orderivedfromobservationsofpaintcrosssections.ByregisteringtheB&Wphotographandthepresentstateandlocalcorrelationanalysisweautomaticallyrecovercolorhints.Thesehintsarecomplementedbyexpertinput.Weassumethatpixelswithsimilarluminancevalueswillhavesimilarchrominancevalues.Weposeandsolvealinearoptimizationproblemwithrespecttothepixelsofunknowncolorinaneighborhoodandpixelsofknowncolor.OurmethodologyhasbeenusedtocolorizehistoricalphotographsofBathersbyaRiver,1909-1917,byHenriMatisse(ArtInstituteofChicago1953.158),andLaJoieDeVivre,1946,byPabloPicasso(MuséePicasso,Antibes,1946.1.4),providingnewtoolstoarthistorianstounravelthealchemyofcreationofthesemasterpieces.


PHOG analysis of self-similarity in aesthetic imagesS.A.Amirshahi,M.Koch,J.Denzler,C.Redies,Friedrich-Schiller-Univ.Jena(Germany)

Inrecentyears,therehavebeeneffortsindefiningthestatisticalpropertiesofaestheticphotographsandartworksusingcomputervisiontechniques.However,itisstillanopenquestionhowtodistinguishaestheticfromnon-aestheticimageswithahighrecognitionrate.Thisispossiblybecauseaestheticperceptionisinfluencedalsobyalargenumberofculturalvariables.Nevertheless,thesearch

forstatisticalpropertiesofaestheticimageshasnotbeenfutile.Forexample,wehaveshownthattheradiallyaveragedpowerspectrumofmonochromeartworksofWesternandEasternprovenancefallsoffaccordingtoapowerlawwithincreasingspatialfrequency(1/f2characteristics).ThisfindingimpliesthatthisparticularsubsetofartworkspossessesaFourierpowerspectrumthatisself-similaracrossdifferentscalesofspatialresolution.Othertypesofaestheticimages,suchascartoons,comicsandmangasalsodisplaythistypeofself-similarity,asdophotographsofcomplexnaturalscenes.Sincethehumanvisualsystemisadaptedtoencodeimagesofnaturalscenesinaparticularefficientway,wehavearguedthatartistsimitatethesestatisticsintheirartworks.Insupportofthisnotion,wepresentedresultsthatartistsportraithumanfaceswiththeself-similarFourierstatisticsofcomplexnaturalscenesalthoughreal-worldphotographsoffacesarenotself-similar.Inviewofthesepreviousfindings,weinvestigatedotherstatisticalmeasuresofself-similaritytocharacterizeaestheticandnon-aestheticimages.Inthepresentwork,weproposeanovelmeasureofself-similaritythatisbasedonthePyramidHistogramofOrientedGradients(PHOG).Foreveryimage,wefirstcalculatePHOGuptopyramidlevel3.Thesimilaritybetweenthehistogramsofeachsectionataparticularlevelisthencalculatedtotheparentsectionatthepreviouslevel(ortothehistogramatthegroundlevel).Theproposedapproachistestedondatasetsofaestheticandnon-aestheticcategoriesofmonochromeimages.TheaestheticimagedatasetscomprisealargevarietyofartworksofWesternprovenance.Otherman-madeaestheticallypleasingimages,suchascomics,cartoonsandmangas,werealsostudied.Forcomparison,adatabaseofnaturalscenephotographsisused,aswellasdatasetsofphotographsofplants,simpleobjectsandfacesthatareingeneraloflowaestheticvalue.Asexpected,naturalscenesexhibitthehighestdegreeofPHOGself-similarity.Imagesofartworksalsoshowhighselfsimilarityvalues,followedbycartoons,comicsandmangas.Onaverage,other(non-aesthetic)imagecategoriesarelessself-similarinthePHOGanalysis.Ameasureofscale-invariantself-similarity(PHOG)allowsagoodseparationofthedifferentaestheticandnon-aestheticimagecategories.Ourresultsprovidefurthersupportforthenotionthat,likecomplexnaturalscenes,imagesofartworksdisplayahigherdegreeofself-similarityacrossdifferentscalesofresolutionthanotherimagecategories.Whetherthehighdegreeofself-similarityisthebasisfortheperceptionofbeautyinbothcomplexnaturalsceneryandartworksremainstobeinvestigated.


Influence of the source contact and encoding configuration on the preceived quality for scalable video codingY.Pitrey,Univ.deNantes(France);M.Barkowsky,Polytech’Nantes(France);R.Pepion,Univ.deNantes(France);P.LeCallet,Polytech’Nantes(France)

Invideocoding,itiscommonlyacceptedthattheencodingparamaterssuchasthequantizationstep-sizehaveaninfluenceontheperceivedquality.WhendealingwithScalableVideoCoding(SVC),theparametersusedtoencodeeachlayerlogicallyhaveaninfluenceontheoverallperceivedquality.Itisalsocommonlyacceptedthatusinggivenencodingparameters,theperceivedqualitydoesnotchangesignificantlyaccordingtotheencodedsourcecontent.Inthispaper,weevaluatetheimpactofbothSVCcodingartifactsandsourcecontentsonthequalityperceivedbyhumanobservers.Weexploittheoutcomesoftwosubjectiveexperimentsdesignedandconductedunderstandardconditionsinordertoprovidereliableresults.Thetwoexperimentsarealignedonacommonscaleusingasetofsharedprocessedvideosequences,resultinginadatabasecontainingthesubjectivescoresfor60differentsourcescombinedwith20SVCscenarios.Weanalysetheperformanceofseveralsourcedescriptorsinmodelingtherelativebehaviourofagivensourcecontentwhencomparedtotheaverageofothersourcecontents.

Conference 8291



Evaluation of desktop interface displays for 360-degree videoS.B.Gilbert,W.Boonsuk,J.W.Kelly,IowaStateUniv.(UnitedStates)

A360-degreevideobecomesnecessaryinapplicationsrangingfromsurveillancetovirtualreality.Thisresearchfocusesondevelopinganinterfaceforasystemsuchasmobilesurveillancethatintegrates360-degreevideofeedsforremotenavigationandobservationinunfamiliarenvironments.Anexperimentevaluatedtheeffectivenessofthree360-degreeviewuserinterfacestoidentifythenecessarydisplaycharacteristicsthatallowobserverstocorrectlyinterpret360-degreevideoimagesdisplayedonadesktopscreen.Videofeedsweresimulatedusingagameengine.Interfaceswerecompared,basedonspatialcognitionandparticipants’performanceinfindingtargetobjects.Resultssuggestthat1)correctperceptionofdirectionwithina360-degreedisplayisnotcorrelatedwithacorrectunderstandingofspatialrelationshipswithintheobservedenvironment,2)visualboundariesintheinterfacemayincreasespatialunderstanding,and3)increasedvideogamingexperiencemaybecorrelatedwithbetterspatialunderstandingofanenvironmentobservedin360-degrees.Thisresearchwillassistdesignersof360-degreevideosystemstodesignoptimaluserinterfacefornavigationandobservationofremoteenvironments.


An evaluation of different setups for simulating lighting characteristicsB.A.Salters,M.J.Murdoch,D.Sekulovksi,P.Seuntiens,S.Chen,PhilipsResearchNederlandB.V.(Netherlands)

TheadvanceofLEDtechnologyenablesanewrangeofluminairedesigns,whichpreviouslywerenotpossible.Evaluatingsuchnewdesignshastraditionallybeendoneusingactualprototypes,inarealenvironment.Theiterationsneededtobuild,verify,andimproveluminairedesignsincursubstantialcosts,andslowdownthedesignprocess.Amoreattractivewayistoevaluatedesignsusingsimulations,astheycanbemadecheaperandquicker,forawidervarietyofprototypes.However,thevalueofsuchsimulationsisdeterminedbyhowcloselytheypredicttheoutcomeofactualperceptionexperiments.

Inthispaper,wediscussanactualperceptionexperimentonseverallightingsettings,inanormalofficeenvironment.Thesameofficeenvironmentalsohasbeenmodeledatvariousdetaillevels,andphoto-realisticrenderingshavebeencreatedofthesemodels.Theserenderingsweresubsequentlyprocessedusingvarioustone-mappingoperators,andshownondifferentdisplays.Thetotalchainofsoftwareandhardwarecanbeconsideredasimulationsetup,andwehaveexecutedseveralperceptionexperimentsondifferentsetups.Finally,acomparisonismadebetweentheresultsobtainedinreality,andtheresultsobtainedviasimulation.Twopossibleapproachesarediscussedtorankthequalityofdifferentsimulationsetupsforthepurposeofsimulatinglightingcharacteristics.


Biological visual attention guided automatic image segmentation with application in satellite imagingM.I.Sina,A.Cretu,P.Payeur,Univ.ofOttawa(Canada)

Takinginspirationfromthesignificantlysuperiorperformanceofhumanstoextractandinterpretvisualinformation,theexploitationofbiologicalvisualmechanismscancontributetotheimprovementoftheperformanceofcomputationalimageprocessingsystems.Computationalmodelsofvisualattentionhavealreadybeenshowntosignificantlyimprovethespeedofsceneunderstandingbyattending

onlytheregionsofinterest,whiledistributingtheresourceswheretheyarerequired.However,thereareonlyfewattention-basedcomputationalsystemsthathavebeenusedinpracticalapplicationsdealingwithrealdataanduptonow,noneofthecomputationalattentionmodelswasdemonstratedtoworkunderawiderangeofimagecontent,characteristicsandscalessuchasthoseencounteredinsatelliteimaging.Thispaperoutlinessomeofthedifficultiesthatthecurrentgenerationofvisualattention-inspiredmodelsencounterwhendealingwithsatelliteimages.ItthenproposesanovelalgorithmforautomaticimagesegmentationandregionsofinterestsearchthatcombineselementsofhumanvisualattentionwithLegendremomentsappliedontheprobabilitydensityfunctionofcolorhistograms.Theexperimentalresultsdemonstratethattheproposedapproachobtainsbetterresultsthanoneofthemostevolvedcurrentcomputationalattentionmodelproposedintheliterature.


A neurobiologically-based, two-stage model for human color visionC.Q.Wu,StanfordUniv.(UnitedStates)

Currently,therearetwodominanttheoriesforhumancolorvision:Young-Helmholtz’strichromatictheoryandHering’sopponent-colortheory.Itisnowwidelyacceptedthatthetrichromatictheoryholdstrueforretinalcolorprocessingwhereastheopponent-colortheoryworksforcorticalcolorprocessing--thisconceptionhasbecomethe“StandardModel”forhumancolorvision.InthepresentpaperIattempttoachievethreegoals:First,todemonstratethattheopponent-colortheoryisfundamentallyuntenable,basedonbothlogicalandempiricalgrounds;second,toresurrectatwo-stagetrichromaticmodel,inwhichbothretinalandcorticalcolorprocessingaretrichromatic,proposedbyA.RollettandW.McDougallmorethanacenturyago;andthird,tomapthecorticalcolorprocessingstageinthismodeltolayer4Cwithintheprimaryvisualcortexofthehumanbrain.


The oscillatory activities and its synchronization in auditory-visual integration as revealed by event-related potentials to bimodal stimuliJ.Guo,BeijingNormalUniv.(China);P.Xu,GeneralHospitalArmedPoliceForces(China);X.Zhao,BeijingNormalUniv.(China)

Neuralmechanismofauditory-visualspeechintegrationisalwaysahotstudyofmulti-modalperception.Thearticulationconveysspeechinformationthathelpsdetectanddisambiguatetheauditoryspeech.AsimportantcharacteristicofEEG,oscillationsanditssynchronizationhavebeenappliedtocognitionresearchmoreandmore.ThisstudyanalyzedtheEEGdataacquiredbyunimodalandbimodalstimuliusingtimefrequencyandphasesynchronyapproach,investigatedtheoscillatoryactivitiesanditssynchronymodesbehindevokedpotentialduringauditory-visualintegration,inordertorevealtheinherentneuralintegrationmechanismunderthesemodes.ItwasfoundthatbetaactivityanditssynchronizationdifferenceshadrelationshipwithgestureN1-P2,whichhappenedintheearlierstageofspeechcodingtopronouncingaction.AlphaoscillationanditssynchronizationrelatedwithauditoryN1-P2mightbemainlyresponsibleforauditoryspeechprocesscausedbyanticipationfromgesturetosoundfeature.Thevisualgesturechangingenhancedtheinteractionofauditorybrainregions.Theseresultsprovidedexplanationstothepowerandconnectivitychangeofevent-evokedoscillatoryactivitieswhichmatchedERPsduringauditory-visualspeechintegration.


Quality assessment of images illuminated by dim LCD backlight

Conference 8291


T.Huang,H.H.Chen,NationalTaiwanUniv.(Taiwan)

Weconsiderthekindofdimmedimagesthatareseenonaliquidcrystaldisplay(LCD)whenthebacklightoftheLCDissettoalowpowerlevel.Amajorcharacteristicofthisenergysavingmodeisthattheperceivedimagequalitydecreaseswiththeintensityofthebacklight.Inparticular,someimageregionsmayappearsodarkthattheybecomenon-perceptibletohumaneyewhentheimageisilluminatedwithverydimbacklight.Ignoringthisveryfactanddirectlyapplyinganimagequalityassessmentmetrictotheentireimagemayproduceresultsinconsistentwithhumanevaluationoftheimage.Inthispaper,weproposeamethodtofixtheproblem.Theproposedmethodworksasaprecursorofimagequalityassessment.Specifically,givenanimageandthebacklightintensityleveloftheLCDonwhichtheimageistobedisplayed,themethodautomaticallyclassifiesthepixelsofanimagetoperceptibleandnon-perceptiblepixelsaccordingtothebacklightintensitylevelandexcludesthenon-perceptiblepixelsfromqualityassessment.


Parallax scanning methods for stereoscopic three-dimensional imagingC.A.Mayhew,C.M.Mayhew,VisionIIIImaging,Inc.(UnitedStates)

Undercertaincircumstances,conventionalstereoscopicimageryissubjecttobeingmisinterpreted.Stereoperceptioncreatedfromtwostatichorizontallyseparatedviewscancreatea“cutout”2Dappearanceforobjectsatvariousplanesofdepth.Thesubjectvolumelooksthree-dimensional,buttheobjectsthemselvesappearflat.Thisisespeciallytrueiftheimagesarecapturedusingsmalldisparities.

Onepotentialexplanationforthiseffectisthatalthoughthree-dimensionalperceptioncomesprimarilyfrombinocularvision,ahuman’sgaze(thedirectionandorientationofaperson’seyeswithrespecttotheirenvironment)andheadmotioncontributeadditionalinformation.Theabsenceofthisinformationmaybethereasonthatcertainstereoscopicimageryappears“odd”andunrealistic

Anothercontributingfactormaybetheabsenceofverticaldisparityinformationintraditionalstereoscopydisplay.Currentpsychophysicalandphysiologicalevidencesuggeststhatverticaldisparitiesinfluencetheperceptionofthree-dimensionaldepthandobjectshape,butlittleisknownabouttheperceptualmechanismsthatsupporttheseprocesses.Whateverthespecificmechanismsare,itisclearthatthemotionandgazedirectionoftheeyescontributesignificantlytotheprocessofthree-dimensionalperception.Conventionalstereoscopicimagecapturegoestogreatlengthstoavoidand/oreliminateanyverticalparallaxdifferencesintheimages.

Athirdfactormaybethestereoscopicproductiontrendofcapturingimageswithdisparitiesthatare50%orlessthantheaveragehumaninterocularof65mm.Thistrendisfueled,inpart,byadesiretokeeptheimagesinacomfortablerangeforthegeneralviewingpublic.However,lessdisparityproducesless3Deffectandalsoleadstoaflatteningofbackgroundsceneelements.

Recently,ParallaxScanningtechnologieshavebeenintroduced,whichprovide(1)ascanningmethodology,(2)incorporateverticaldisparity,and(3)producestereoimageswithsubstantiallysmallerdisparitiesthanthehumaninteroculardistances.Totestwhetherthesethreefeatureswouldimprovetherealismandreducetheposterizationofstereoimages,wehaveappliedParallaxScanning(PS)technologiestocommercialstereoscopicdigitalcinemaproductions,andhavetestedtheresultswithapanelofstereoexperts.

Theadditionofparallaxscaninformationintotheleftandrightimagesimprovestheoverallthree-dimensionalperception.ViewerstereopsisisshowntobenefitfromtheadditionalPSsub-processinformationbygeneratingamoreunifiedthree-dimensionalperception.

PSisaccomplishedopticallythroughtheuseofadigitalparallaxscanning(DPS)irismechanism.APSlensisdesignedtomoveitsiriscenterfrom0to4mmoffthelenscenteralongacircularpathatfrequenciesfrom0to15Hz.Twoparallaxscanninglensescanbeconfiguredforstereoscopicimagecapture.

Totestthevisualeffect,weconductedthefollowingexperiment.We

suppliedDPSequippedlensestoaprofessionalstereographerwhousedthemtoshoota3Dtestsceneatvaryingscanningparametersincludinganoscancontrol(0mmoffset).Theresultswerepresentedtoagroupoffourteenstereographicprofessionalsandacademics.EleveninthegroupreportedsignificantlyimprovedshapedefinitioninthePSimagerywhencomparedtothecontrol.TworeportedmoderateimageimprovementinthePSimageryandonereportednodifferencebetweenthecontrolandthePSimagery.Subsequentscreeningsofthetestimagerytoothergroupsproducedsimilarviewerresults.

TheseinformalexperimentsshowthattheadditionofPSinformationintotheleftandrightimagecaptureimprovestheoverallperceptionofthree-dimensionalityinmostviewers.Parallaxscanningsignificantlyincreasesthesetoftoolsavailablefor3Dstorytellingwhileatthesametimepresentingimagerythatiseasyandpleasanttoview.


Reduced reference image quality assessment via sub-image similarity-based redundancy measurementX.Mou,W.Xue,Xi’anJiaotongUniv.(China);L.Zhang,TheHongKongPolytechnicUniv.(HongKong,China)

Thereducedreference(RR)imagequalityassessment(IQA)hasbeenattractingmuchattentionfromresearchersforitsloyaltytohumanperceptionandflexibilityinpractice.ApromisingRRmetricshouldbeabletopredicttheperceptualqualityofanimageaccuratelywhileusingasfewfeaturesaspossible.Inthispaper,anovelRRmetricispresented,whosenoveltyliesintwoaspects.Firstly,itmeasurestheimageredundancybycalculatingthesocalledSub-imageSimilarity(SIS),andtheimagequalityismeasuredbycomparingtheSISbetweenthereferenceimageandthetestimage.Secondly,theSISiscomputedbytheratioofNSE(Non-shiftEdge)betweenthepairsofsub-images.ExperimentsontwoIQAdatabases(i.e.LIVEandCSIQdatabases)showthatbyusingonly6features,theproposedmetriccanworkverywellwithhighcorrelationsbetweenthesubjectiveandobjectivescores.Inparticular,itworksconsistentlywellacrossdifferentdistortiontypes.


Color impact in visual attention deployment considering emotional imagesC.Chamaret,TechnicolorS.A.(France)

Inthisstudy,weproposetostudythecolorimpactaswellastheemotionaspectofpicturesonthevisualattentiondeployment.Asexpectedtheeyefixationsofbothcolorandblackandwhiteimageswerehighlycorrelatedleadingtothequestionoftheintegrationofsuchcuesinthedesignofvisualattentionmodel.Thislastpointwasnotapendingissueintheliteraturesinceitisagreedthatcolorplaysaroleinvisualattention.Thecompromiseisprobablytheintegrationofcolorcomponentwithinthevisualattentionmodels,butweightedbythepre-analysisofimagewhichmaydetermineapriorithepotentialimpactofcolorwithinthepresentedpictures.

8291-23, Session 6

On the development of expertise in interpreting medical imagesE.Krupinsky,TheUniv.ofArizona(UnitedStates)

Noabstractavailable

8291-24, Session 6

Modeling observer performance for optimizing image acquisition and processing

Conference 8291


C.K.Abbey,M.Eckstein,Univ.ofCalifornia,SantaBarbara(UnitedStates)

Noabstractavailable

8291-25, Session 6

Evaluation of HVS models in the application of medical image quality assessmentL.Zhang,C.Cavaro-Ménard,Univ.ofAngers(France);P.LeCallet,Polytech’Nantes(France)

Inthisstudy,fourofthemostwidelyusedHumanVisualSystem(HVS)modelsareappliedonMagneticResonance(MR)imagesforsignaldetectiontask.Theirperformancesareevaluatedagainstgoldstandardderivedfromradiologists’decisions.Thetask-basedimagequalityassessmentrequirestakingintoaccountthehumanperceptionspecificities,forwhichvariousHVSmodelshavebeenproposed.Fewworkswereconductedhowevertoevaluateandcomparethesuitabilityofthesemodelswithrespecttotheassessmentofmedicalimagequalities.HereweproposetoscoretheperformanceofeachHVSmodelusingtheAUCanditsvarianceestimatesasthefigureofmerit.Thecontributionofthisworkistwofold:firstlytheapplicationofMRMC(multiple-reader,multiple-case)estimatesindependentlyoftheHVSmodel’soutputrange,secondlytheuseofradiologists’consensusasgoldstandardsothattheestimatedAUCmeasuresthedistancebetweentheHVSmodelandtheradiologistperception.

8291-26, Session 6

Perceptual challenges to computer-aided diagnosisY.Jang,TheUniv.ofChicago(UnitedStates)

Noabstractavailable

8291-27, Session 6

Satisfaction of search experiments in advanced imagingK.S.Berbaum,TheUniv.ofIowaHospitalsandClinics(UnitedStates)

Noabstractavailable

8291-28, Session 6

Integrating human- and computer-based approaches to feature extraction and analysisB.E.Rogowitz,VisualPerspectives(UnitedStates);A.Goodman,Harvard-SmithsonianCtr.forAstrophysics(UnitedStates)

IntegratingHuman-andComputer-basedApproachestoFeatureExtractionandAnalysis

Amajorgoalofimagingsystemsistohelpdoctors,scientists,engineers,andanalystsidentifypatternsandfeaturesincomplexdata.Typically,thisisaccomplishedbycomputer-drivenalgorithmsthatextractfeaturesautomatically,orbyvisualizationsystemsthatallowtheanalysttomanipulatevisualrepresentationstodiscoverfeaturesinteractively.Althoughautomaticfeature-extractionalgorithmsareoftendirectedbyhumanobservation,andhumanpatternrecognitionisoftensupportedbyalgorithmictools,verylittleworkhasbeendonetoexplorehowtocapitalizeontheinteractionbetweenhumanandmachinepatternrecognition.Thispaperintroducesapreliminaryroadmapforguidingresearchinthisspace.Onekeyconceptistheexplicitconsiderationoftheuser’stask,sincetheeffectivenessofdifferentvisualrepresentationsandinteractionmethodswilldepend

onthegoal.Thesecondistheexplicitinclusionofa“human-in-the-loop,”whointeractswiththedata,thealgorithms,andrepresentations,toidentifymeaningfulfeatures,usingvisual,auditoryandhaptictools.And,thirdly,thisframeworkexplicitlyclosestheloop,inthatfeaturesidentifiedor“carvedout”bythehumanarecharacterizedmathematically,foruseincomparison,databasequeryoranalysis.

8291-20, Session 7A

How do we watch images?: a case of change detection and quality estimationJ.Radun,T.Leisti,T.Virtanen,G.Nyman,Univ.ofHelsinki(Finland)

Themostcommontasksinsubjectiveimageestimationarechangedetection(adetectiontask)andimagequalityestimation(apreferencetask).Weexaminedhowthetaskinfluencesthegazebehaviorwhencomparingdetectionandpreferencetasks.Theeyemovementsof16naïveobserverswererecordedwith8observersinbothtasks.Thesettingwasaflickerparadigm,wheretheobserversseeanon-manipulatedimage,amanipulatedversionoftheimageandagainthenon-manipulatedimageandestimatethedifferencetheyperceivedinthem.Thematerialwasphotographicmaterialwithdifferentimagedistortionsandcontents.Toexaminethespatialdistributionoffixations,wedefinedtheregionsofinterestusingamemorytaskandcalculatedinformationentropytoestimatethespatialconcentrationoffixations.Thequalitytaskwasfasterandneededfewerfixationsandthefirsteightfixationsweremoreconcentratedoncertainimageareasthaninthechangedetectiontask.Thebottom-upinfluencesoftheimagealsocausedmorevariationtothegazebehaviorinthequalityestimationtaskthaninthechangedetectiontask.Theresultsshowthatthestrategiesformakingthetasksaredifferentandinsubjectiveimageestimationstudiesitisimportanttothinkaboutthetask.

8291-21, Session 7A

Measuring saliency in images: which experimental parameters for the assessment of image quality?C.Fredembach,G.Woolfe,J.Wang,CanonInformationSystemsResearchAustraliaPty.Ltd.(Australia)

Predictingwhichareasofanimageareperceptuallysalientorattendedtohasbecomeanessentialpre-requisiteofmanycomputervisionapplications.Becauseobserversarenotoriouslyunreliableinrememberingwheretheylookaposteriori,andbecauseaskingwheretheylookwhileobservingtheimagenecessarilyinfluencestheresults,groundtruthaboutsaliencyandvisualattentionhastobeobtainedbygazetrackingmethods.

FromtheearlyworkofBuswellandYarbustothemostrecentforaysincomputervisiontherehasbeen,perhapsunfortunately,littleagreementonstandardisationofeyetrackingprotocolsformeasuringvisualattention.Asthenumberofparametersinvolvedintheexperimentalmethodologycanbelarge,theirindividualinfluenceonthefinalresultsisnotwellunderstood.Consequently,theperformanceofsaliencyalgorithms,whenassessedbycorrelationtechniquesvariesgreatlyacrosstheliterature.

Inthispaper,weconcernourselveswiththeproblemofimagequality.Specifically:wherepeoplelookwhenjudgingimages.Weshowthatinthiscase,theperformancegapbetweenexistingsaliencypredictionalgorithmsandexperimentalresultsissignificantlylargerthanotherwisereported.Tounderstandthisdiscrepancy,wefirstsetouttodeviseanexperimentalprotocolthatisadaptedtothetaskofmeasuringimagequality.Inasecondstep,wecompareourexperimentalparameterswiththeonesofexistingmethodsandshowthatalotofthevariabilitycandirectlybeascribedtothesedifferencesinexperimentalmethodologyandchoiceofvariables.

Inparticular,thechoiceofatask:judgingimagequalityvs.freeviewinghasagreatimpactonmeasuredsaliencymaps,suggestingthatevenforamildlycognitivetask,groundtruthobtainedbyfreeviewingdoesnotadaptwell.Carefulanalysisofthepriorartalsorevealsthat

Conference 8291


systematicbiascanoccurdependingoninstrumentalcalibrationandthechoiceoftestimages.

Weconcludethisworkbyproposingasetofparameters,tasksandimagesthatcanbeusedtocomparethevarioussaliencypredictionmethodsinamannerthatismeaningfulforimagequalityassessment.

8291-29, Session 7A

Examining the effect of task on viewing behavior in videos using saliency mapsH.Alers,J.A.Redi,TechnischeUniv.Delft(Netherlands);I.Heynderickx,PhilipsResearch(Netherlands)

Thisworkisaimedatunderstandinghowtheviewingtaskandlevelofvideoqualityaffecttheviewingbehavioroftheobserver.Totestthat,adatabaseof50videosof20secondseachwascreated.Thesevideosweredegradedtodifferentlevelsofquality.Consequently,observerswereaskedtowatchthesevideoswherehalfofthemweregiventhetaskofscoringthequalitylevelwhiletherestweretoldtosimplywatchthevideosfreely.Byusinganeyetrackingdevice,itwaspossibletorecordtheviewingbehaviordataoftheobservers.Thisdatawasusedtocreatesaliencymapsforeachsecondofvideo,whichmadeitpossibletocomparetheviewingbehaviorbetweenthetwotasksacrossdifferentlevelsofquality.Theresultssuggestthatthereisadifferenceintheviewingbehaviordependingonthetaskgiventotheviewerwhichisnoteffectedwiththequalitylevelofthevideo.Itisalsoshownthattherearespecificattributesinthevideos,whichaffectthelevelofsimilarityinviewingbehaviorbetweenthetwotasks.

8291-23, Session 7B

Investigations of the tone reproduction curves on the perceived image quality for fine art reproductionsJ.Jiang,F.Frey,S.Farnand,RochesterInstituteofTechnology(UnitedStates)

Basedontheresultfromarecentweb-basedexperimentconductedtoevaluatetheperceivedimagequalitywithoutthereferencesponsoredbyMellonFoundation,aprojectisbeingconductedtounderstandtheeffectofthecontrastadjustmentmadeintheprocessoffineartreproductionontheperceivedimagequality.Contrastsareusuallyadjustedusingthetonereproductioncurve(TRC)inordertoachievethehighestpossibledynamicrangeandtobetterrepresenttheoriginalartwork.Copiesoffineartimagesaresimulatedusingtonereproductioncurvescollectedfromdifferentmuseums.Duringtheexperiment,observersareinstructedtorateeachtestimagebasedontheirpreferenceinaweb-basedenvironment.Observerswillbeaskedtoclickontheareathatmostinfluencestheirdecisions.Moreattentionmaybeneededtotheareasonwhichmostclicksaremadewhenthetonereproductioncurveisadjustedduringvisualediting.Tounderstandtheimpactoftheoriginalonparticipants’preferencejudgments,theexperimentwillbeconductedbothwithandwithoutthereference.TheresultmaybeindicativeofwhetheraseparateTRCisneededforfineartreproductionsdependingontheavailabilityoftheoriginalwhenthereproducedimageisviewed.

8291-30, Session 7B

Characterizing eye movements during temporal- and global-quality assessment of h.264 compressed video sequencesC.Mantel,N.Guyader,P.Ladret,G.Ionescu,Gipsa-lab(France);T.Kunlin,STMicroelectronics(France)

Studieshaveshownthatthedeploymentofvisualattentioniscloselylinktotheassessmentofimageorvideoquality,thoughthislinkisnotyetfullyunderstood.Theinfluenceofratingtemporalqualityofcompressedvideosoverthewayanobserverdeployshisattentionis

investigatedinthispaper.

Weset-upasubjectiveexperimentinwhichtheeyemovementsofobserversarerecordedduringthreedifferenttasks:afree-viewingtask(FT),aglobalqualityassessmenttaskandatemporalqualityassessmenttask.TheFTactsasareferencetowhichwecomparetheeyemovementsduringthetwoothertasks.

Aspreviouslyshown,observersassessingglobalqualitygazeatlocationsdissimilartothosefixatedduringtheFT.Fortemporalqualityassessment,itseemsthatthefixatedlocationsareclosertoFTthantheglobalqualityassessmentfixatedlocations.

Ourresultssuggestthatthelocationsobserverslookatdonotdependonthedisplayedvideoqualitylevel.Qualityhoweverinfluencesthewayparticipantslookatvideos:thelowerthequality,thelongertheygazeatapreciselocation.TheareafixatedseemstobemuchsmallerduringthequalityassessmenttasksthanduringtheFTforeitherperfectorpoorqualitylevel.

Theevolutionovertimeofallindicatorssuggeststhat,duringthefirst1or2seconds,thesignalpropertiesofthevideosarethemainattractorsfortheparticipants’eyemovements.Instructionsonlyseemtoplayaroleafterwardsonthedeploymentoftheparticipants’visualattention.

8291-31, Session 7B

A compressed sensing model of crowding in peripheral visionJ.Hocke,Univ.zuLübeck(Germany);M.Dorr,SchepensEyeResearchInstitute(UnitedStates);E.Barth,Univ.zuLübeck(Germany)

Weheremodelperipheralvisioninacompressedsensingframeworkandfindthattypicalletter-crowdingeffectsarisenaturally.Imagesarere-sampledwithsamplesthataredistributedaccordingtoaPoisson-diskdistribution.Thesesamplesarethenconsideredtobeinputstoanetworkofneuronssuchthattheoutputneuronsarerandomlyconnectedtotheinputneurons.Alsotheweightsoftheconnectionsareassignedrandomly.Convergenceismodeledbyallowingforfeweroutputneuronsthaninputneurons.Byreconstructingtheoriginalimagefromtheactivityoftheoutputneuronsweestimatetheamountofinformationdeliveredattheoutput.Toreconstructtheoriginalimage,thesignalattheoutputneuronsismodeledasalinearsuperpositioninasparsebasisandasparse-approximationalgorithmisapplied.Thereconstructedimagesexhibittypicalcrowdingartifactsinthesensethatsingleletterscanberecognizedbutnotletterswhichareflankedbyadditionalletters.Whenthesamplingrateisincreased,aswouldbethecaseincentralvision,boththesingleandtheflankedletterscanberecognized.

8291-32, Session 7B

Foveated self-similarity in nonlocal image filteringA.Foi,TampereUniv.ofTechnology(Finland);G.Boracchi,PolitecnicodiMilano(Italy)

Weinvestigatetheroleoffoveationinnonlocalimagefilteringand,inparticular,intheNLmeansdenoisingalgorithm.Thenonlocalself-similaritybetweenregionsatdifferentimagelocationsisusedinNLmeanstoadaptivelydeterminetheweightsforpointwiseestimation.Thissimilaritybetweendifferentregionsiscomputedthroughthewindowedphotometricdifferences,wherethewindowfunctiondecaystozeroaswemoveawayfromcenterofthecorrespondingregion.

Wereplacesuchwindoweddifferencewithafoveateddifference:insteadofmultiplyingthephotometricdifferencesagainstawindowfunction,weblurthedifferenceswithpoint-spreadfunctionshavingincreasingstandard-deviation(i.e.increasingspread)asthedistancefromtheregion’scentergrows.Thisfoveateddifferencemimicstheactualabilityofthehumanvisualsystemtoperceivedetailsattheperipheryofthepointoffixation.

Ourstudyshowthatafoveatedself-similarityleadstoanimprovementintheresultingimageestimate,accordingbothtoobjectivecriteriaand

Conference 8291


visualappearance,particularlyduetobettercontrastandsharpness.

Wediscussthisformofself-similarityinthecontextofnaturalimagestatisticsaswellasitspotentialimplicationsinexplainingtheroleofeyemovementsinimprovingvisualperception.

8291-33, Session 7B

A statistical study of the correlation between interest points and gaze pointsM.Nauge,M.Larabi,Univ.dePoitiers(France)

Inthiswork,weproposetostudytheperformanceofthesedetectorswithatotallydifferentpointofview,bystudyingsimilarity/correlationbetweenthesubjectivegazepointsandtheobjectiveinterestpoints.TheaimistodeterminewhethertheinterestpointscanbeusedtopredictsalientinformationonanimageliketheHVSdoes.Thiscanhelpforseveralapplicationslikequalityassessment,simplifiedsaliencymapsconstruction,...Eventhoughtheinterestpointshavenotbeenoriginallydesignedtobeclosetothegazepoints,theymayhaveaparticularsettingthatmaximizesthesimilaritybetweenthem.Forthisstudyabatteryofstatisticaltoolsisusedtotestalargerangeofsettings/configurationsforthethreeinterestpointdetectorsmentionedpreviously.

WeusedseveralstatisticaltoolssuchasBartlett,ANOVA,...tounderstandtheeffectandtheinfluenceofeachparametersforeachdetector.Thesestudiesillustratethatparticularparameterscanminimizethecostoftransformationandpredictinterestpointsinaccordancewiththesubjectivegazepoints.WealsoproposedasolutiontogiveascaletofacilitatetheinterpretationoftheEMDvaluesbyanalyzingthemeanhumanbehavior.Bycomparingthebestsettingforeachdetectorwecanalsoindicatewhichdetectoristhemostreliabletoestimatethesubjectivegazepoints.ThisstudyisalsoagoodwaytoprovethaninterestpointsdetectorsharesomepropertieswiththeHVS.

8291-34, Session 7B

Interest point analysis as a model for the Poggendorff illusionF.W.M.Stentiford,Univ.CollegeLondon(UnitedKingdom)

ThispaperdescribesarecognitionmechanismbasedontherelationshipsbetweeninterestpointsandtheirpropertiesthatisappliedtotheproblemofmodellingthePoggendorffillusion.Therecognitionmechanismisshowntoperforminthesamemannerashumanvisiononthestandardillusionandreducedeffectsaremodelledonavariantwithoutparallels.TheresultsshowthattherecognitionmechanismproduceshighsimilarityscoresbetweencontinuoustransversallinesandPoggendorfffigureswherethelinesegmentsareoffsetfromthecollinearposition.Thepeaksoccurinthesamedirectionofshiftasreportedinpsychophysicalinvestigationsofhumanvision.Themostprominentpeaksoccurwithtransversalsinclinedatthesmallestanglestotheverticalanddecreasetowardstheperpendicularposition(90°).TheanalysisoffigureswithnoparallelsalsoyieldedapeakalthoughthiswasnotasprominentaswiththeconventionalPoggendorfffigures.Thiswasalsoinagreementwithearlierbehaviouralexperiments.TheanalysisisalsoshowntobeconsistentwithpsychophysicalfindingswhereobtuseangledvariantsofthePoggendorfffiguresyieldstrongereffectsthantheacuteangledvariants.

8291-35, Session 8

The perception of art and the science of perceptionR.Pepperell,CardiffSchoolofArt&Design(UnitedKingdom)

Formanycenturiesartistshavestudiedthenatureofvisualperceptionandhowtoconvincinglyrenderwhatweseeusingavarietyofmedia.Theresultsofthisprolongedinvestigation,duringwhichknowledge

andinsightswerehandeddownfromgenerationtogeneration,canbefoundinallthecountlessartworksdepositedinmuseumsandgalleriesaroundtheworld.Worksofartrepresentarichsourceofideasandunderstandingabouthowtheworldappearstous,andonlyrelativelyrecentlyhavethoseinterestedinthescienceofvisionstartedtoappreciatethemanydiscoveriesmadebyartistsinthisfield.InthistalkIwilldiscusssomeofthekeyinsightsintovisionandperceptionthatartistshaverevealedthroughtheirinquiries,andshowhowtheycanhelpcurrentthinkinginscienceandtechnologyabouthowbesttounderstandtheprocessofseeing.Inparticular,Iwillsuggestthatsomeimportantartisticideascontinuetopresentfundamentalchallengestoconventionalideasabouthowrealityisrepresented.

8291-36, Session 8

Paintings, photographs, and computer graphics are calculated appearancesJ.J.McCann,McCannImaging(UnitedStates)

Thechallengeofareproductionistocapturetheinformationcontainedinthephysicsoflight(original3-Dcolorspace)andtorenderitinadifferentsizeandshapeofmedia(reproductioncolorspace).

Thereproductionproblemisverysimilartomovingeverythinginyourhousetoanewone.Thereproductionhousehasdifferentdimensionsforthelength(amountofred),width(amountofgreen)andheight(amountofblue).Onaclearday,shadowscastbythesunare32timesdarkerthandirectsunlight.The32:1rangeofreflectancesina32:1rangeofilluminationcreatesa1024:1HDRrangeoflight.Real-lifescenereproductionisanalogoustomovingacastleintoacottage.Reproductionsmoveeverythingintheoldhouseintothenewhouse,keepingallcontentsincorrespondingrooms,eventhoughthedimensionsoftheentirehouse,andeachroom,aredifferent.Goodreproductionsareneverexactphysicalcopiesoftheoriginal,becausethatisnotpossible.Goodreproductionscapturetheappearanceandrelationshipsofobjectsinthescene.

Paintersreproducetheappearancestheysee,orvisualize.Theentirehumanvisualsystemisthefirstpartofthatprocess,providingextensivespatialprocessing.PaintershaveusedspatialtechniquessincetheRenaissancetorenderHDRscenes.Silverhalidephotographyrespondstothelightfallingonsinglefilmpixels.Filmcanonlymimictheretinalresponseoftheconesatthestartofthevisualprocess.Filmcannotmimicthespatialprocessinginhumans.Digitalimageprocessingcan.Sowithspatialalgorithms,wecancalculateappearancesandwritethemtoprintanddisplay.

8291-37, Session 8

Image integrity and aesthetics: towards a more encompassing definition of visual qualityJ.A.Redi,TechnischeUniv.Delft(Netherlands);I.Heynderickx,PhilipsResearch(Netherlands)andDelftUniv.ofTechnology(Netherlands)

Visualqualityhasbeensofarinvestigateddifferent,sometimescontrastingperspectives.Theelectronicimagingcommunityhasforlonglimitedvisualqualitytotheperceivedintegrityofthemedia,whenaffectedbyvisualdegradationsduetosignalerrorsortechnologicallimitations.Fromanotherperspective,themediamanagementcommunityhasbeeninvestigatingonadefinitionofvisualqualitymorerelatedtocontentpleasantness,andinparticularonaestheticqualityassessment.Inthiswork,weaimatmovingthefirststepstowardsamoreencompassingdefinitionofvisualquality,whichtakesintoaccountboththeimageintegrityandthepleasantnessofitscontent.Todoso,westartbyinvestigatingtheinteractionsbetweenaestheticandartifactualattributesinvisualqualityassessment.Wedesignedapsychometricexperimenttoinvestigateboththeroleofartifactualattributesinaestheticqualityassessment,andthedifferencesinviewingbehaviour,betweenfreeimageobservation,imageintegrityevaluationandaestheticqualityassessment.Aneye-trackersupports

Conference 8291


thedatacollectionforthispurpose.Throughtheanalysisoftheresultingdata,weaimatcollectingusefulinformationtodesignamorecompletesetofattributesonwhichbasingfuturevisualqualityassessmentmodels.

8291-38, Session 8

Depicting 3D shape using linesD.DeCarlo,Rutgers,TheStateUniv.ofNewJersey(UnitedStates)

Noabstractavailable

8291-39, Session 8

Box spaces in pictorial space: linear perspective versus templatesH.deRidder,S.C.Pont,TechnischeUniv.Delft(Netherlands)

Noabstractavailable

8291-40, Session 8

Warping realist art to ensure consistent perspective: a new software tool for art investigationsY.Chang,WolframResearch(UnitedStates);D.G.Stork,RicohInnovations,Inc.(UnitedStates)

Manyrealistpaintings,especiallyonesfromtheearlyRenaissance,donotconformtotherulesofgeometricalperspective:orthogonals(linesinthreedimensionsperpendiculartotheplaneofthepainting)maynotmeetatavanishingpoint,separatevanishingpointsmaynotlieonauniquehorizonline,andsoon.WehavedevelopedaMathematica-basedtoolthatwarpsimagesinresponsetouser-definedcontrolpoints.Modificationsofthissoftwarecanallowartscholarstoclickonpointsonperspectivelines-orthogonalsandtransversals(paralleltotheplaneofthepainting)-andthenadjustasingleparametercontrollingtheamountofperspective“correction”between0(nowarping)and1(fullwarpingtoensureconsistentperspective).Ofcourse,wearenotclaimingthatapaintingthatconformstotherulesofgeometricperspectiveisinanyway“superior”toonethatdoesnot,orthatanimagethatconformstoperspectiverulesis“correct.”Ournovelvisualizationtoolallowsartscholarstoexposeandvisualizemoreclearlytheperspectiveschemesbyartists.Quantitativemeasuresoftheperspectivecoherenceprovidedbyoursoftwarecanbeusedindiachronicstudiesofthedevelopmentofworksbysomeartists,suchasCézanne’sstilllifes.WewilldemonstrateoursoftwareonworksbyMasolino,JanvanEyck,andothers,andspeculateonitsuseselsewhereinartstudies.

8291-41, Session 8

Sound meets image: freedom of expression in texture descriptionR.J.Jansen,R.vanEgmond,H.deRidder,TechnischeUniv.Delft(Netherlands);T.N.Pappas,NorthwesternUniv.(UnitedStates)

Auditory,visual,andtactiletexturesprovideinformationabouteventsandobjectsintheenvironment.Itisinterestingtocomprehendhowsensorymodalitiescaninfluenceeachother.Weareabletodescribevariouspsychophysicalcharacteristicsofatextureonasensoriallevel.Yet,littleisknownabouttheirrelationtowhatpeoplehear,see,andfeelonaperceptuallevel.Theaimofthisstudyistofindrelevantperceptualdimensionstodescribeauditoryandvisualtextures,andtoexplorehowthesedimensionsarerelatedtounderlyingpsychophysicalmeasuresofbothsensorysystems.ForExperiment1twosetsof17visualtextureswereprepared:onesettakenfromtheCUReTdatabase,andonesetsynthesizedtoreplicatetheformerset.Participantswere

instructedtomatchasoundtexturewithavisualtexturedisplayedonscreen.Theywereprovidedwithaninteractivephysicalinterface,coupledtoafrequencymodulationsynthesizer.Ratherthanselectingfromapre-definedsetofsoundsamples,continuousexplorationoftheauditoryspaceallowedforanincreasedfreedomofexpression.Whiledoingso,participantswereaskedtodescribewhatauditoryandvisualqualitiestheywerepayingattentionto.ThesequalitydescriptionswereusedinExperiment2toratethevisualtexturesandsoundsgeneratedinExperiment1.Finally,inExperiment3participantswereinstructedtochooseavisualtextureforeachsoundgeneratedinExperiment1.

8291-42, Session 9

On the dynamics of aesthetic appreciationC.Carbon,Otto-Friedrich-Univ.Bamberg(Germany)

PersonaltastedevelopsovertimeandishighlysusceptibleforZeitgeist-dependenteffects.Usinganadaptationparadigmoftenusedinthedomainoffaceresearch(e.g.,Carbon&Ditye,2011),wecouldshowinaseriesofexperimentsthatnotonlytherepresentationofdesigns(e.g.,cardesigns,Carbon,2010)orartworks(Carbon&Leder,2006),butalsotastequicklyadaptstowardsadaptors(Carbon,Ditye,&Leder,2006).Thepresentpaperwillnotonlyshowtheadaptationtowardsspecificdesigncharacteristicsinproductdesign,butevensoforspecificart-specificproperties.Forinstance,oneexperimentmadeuseoftheidiosyncraticstyleofAmedeoModiglianiwhoemployedmanneristicfaceelongation.Inthefirstpartoftheexperiment,thepre-adaptationphase(T1),participantswereaskedtoratetheirlikingofanumberofportraitsbyvariousartistsofavarietyofdifferenthistoricalepochsonaseven-pointLikert-scale.Eachportraitwaspresentedin11differentversionswhichdifferedintheamountofcompressionorextensioninheights(cf.Carbonetal.,2007).Thisblockwasfollowedbyanadaptation-phaseinwhichModigliani’spaintingswereshown.Inasucceedingtest-phase,thepost-adaptationphase(T2),likingofthesamepaintingsofT1wasratedagainbytheparticipants.ResultsrevealedsystematicchangesinaestheticappreciationforthoseexemplarsthatwerestructurallyrathersimilarinthestyletoModigliani(theextendedversions).Thetalkqualifiesanddiscussessuchstyle-basedadaptationeffectstorevealtheircognitivebasis.

8291-43, Session 9

An experimental and theoretical analysis of the merits of ‘virtual cleaning’ of paintingsJ.Delaney,NationalGalleryofArt(UnitedStates);A.Casini,IstitutodiFisicaApplicataNelloCarrara(Italy);L.Simonot,Univ.dePoitiers(France);M.Thoury,NationalGalleryofArt(UnitedStates);M.Picollo,L.Stefani,IstitutodiFisicaApplicataNelloCarrara(Italy);M.Poggesi,ConsiglioNazionaledelleRicerche(Italy);D.Conover,K.Fleisher,E.R.delaRie,NationalGalleryofArt(UnitedStates)

Imageprocessingtoimprovecoloraccuracyofimagesofpaintingshasexpandedwiththeadventofmultispectraland,morerecently,highspatialhyperspectralimagingcameras.Theincreasedcoloraccuracyresultsfromamoreaccuratemeasurementofthespectralreflectance.VariousgroupshaveattemptedtorestorethecolorappearanceofOldMasterpaintingsbycompensatingforfadedpigmentsanddiscoloredvarnishesusingsuchreflectanceinformationintheirmodels.Thelatterhasbeencalled“virtualcleaning”ofpaintings.Inthesemodelsthevarnishisoftentreatedasatransmissionfilterandcolorisrestoredtothepaintingbyremovingthediscoloredvarnish’sabsorbance.Theabsorbanceoftheagedvarnishisdeterminedbyeitheraheuristicprocess,oralteringthereflectancespectrabyusingreflectanceortransmittancespectraofartificiallyagedvarnish.

Inthispapertheresultsofexperimentsdesignedtotesttheaccuracyofsuch“virtualcleaning”,usingseveralpaintingscoveredwithagedvarnishes,ispresented.Theexperimentalmethodconsistsofcollectingreflectancespectrabeforevarnishremovalandthenafterapplicationofnewvarnish.Alsotheabsorbancepropertiesofremovedvanisharemeasured.Twotypesofexperimentalstudiesareperformed;thefirstusingafiberopticreflectancespectrometer(350

Conference 8291


to2500nm)atselectedsitesandthesecondacoloraccurateimaginghyperspectralmechanicalscanningcamerasystem(400to900nm)tocollectreflectancespectraoveranentirepainting.Thefirstsetallowsdeterminingthedegreetowhichatransmissionmodelcanbeusedtopredictthefinalreflectancespectraofthecleanedandvarnishedarea.Thesecondsetallowsforavisualcomparisonofresultsofvirtualcleaningversusactualonapainting.Theresultswillbeusedtodemonstratewhether“virtualcleaning”doesordoesnotaccountforthescatteringinherentinnaturallyagedvarnishesorvariationinvarnishthicknessandwhatimprovementsinthemodelsarerequired.

8291-44, Session 9

In search of Leonardo: computer-based facial image analysis of Renaissance artworks for identifying Leonardo as subjectC.W.Tyler,TheSmith-KettlewellEyeResearchInstitute(UnitedStates);W.Smith,TheUniv.ofYork(UnitedKingdom);D.G.Stork,RicohInnovations,Inc.(UnitedStates)

OneoftheenduringmysteriesinthehistoryoftheRenaissanceisthetheadultappearanceofthearchetypical‘RenaissanceMan’,LeonardodaVinci.Hisonlyacknowledgedself-portraitisfromanadvancedage,andvariouscandidateimagesofyoungermenaredifficulttoassessin

theabsenceofdocumentaryevidence.

Reconstructing3Dfaceshapefromasingleviewisawell-studiedproblem.Ingeneral,singleimageshapeestimationisanill-posedproblemevenwhenstrongassumptionsaremadeaboutscenegeometry,reflectanceandcameraproperties.Inthecaseofpaintings,thisprocessisfurtherconfoundedbypotentialerrorsintroducedbytheartistintheirrenditionofperspective,shadingandillumination.

Forthesereasons,weneglectphotometriccuesandreconstruct3Dfaceshapeusingtheprojectedpositionoffiducialpointsonly.Thisapproachhasbeenshowntoperformwellonsyntheticdata.Weemployalinearstatisticalmodelof3Dfaceshape(morphablemodel),learnedfromarepresentativesampleofhumanfaces.Themodelisusedtoconstrainthefaceshaperecoveryprocessbytransformingittooneofparameterestimation.Weformabilinearsysteminwhichtheunknownsarethecameraprojectionmatrix(approximatedbyanaffinecamera)andthefaceshapeparameters.Thisoptimizationisconvexandcanbeefficientlysolvedusingalternatingleastsquares.

Forrobustness,welearnanempiricalmodelofgeneralizationerroroverthesurfaceoftheface.Thisdescribesthevariancebetweenatruefaceshapeanditsbestmodelfit.Thisallowsforfeature-sensitiveweightingofdata-closenesserrorsandallowsustoregularizetheoptimizationproblemwithoutmanualparametertuning.

8291-45, Session 9

Non-destructive analytical imaging of metallic surfaces using spectral measurements and ultrahigh-resolution scanning for cultural heritage investigationJ.Kaneko,Y.Murayama,J.A.Toque,A.Ide-Ektessabi,KyotoUniv.(Japan)

Thispaperpresentsanapproachforanalyzingmetallicsurfacesusingspectroscopicandanalyticalimagingmethods.Spectralluminance,CIELAB,XYZanddeviceRGBvalueofmorethan30metallicsurfacesonJapanesepapersweremeasuredwithaspectrometerandscannedinordertoestimatethespectralreflectance.Thiswasdonetofacilitatethestatisticaldiscriminationbetweenindividualmetallicfoils.Areferencedatabasewiththespectralreflectanceofthefoilswasmeasuredfrom400to700nm,withastepwidthof10nm.Theimageswerecapturedbypolarizingthelightsourceandthecameratoseparatethediffusedandspecularcomponentsofthereflection.Theanglebetweenthefilterinfrontofthelightsourceandthecamerawaschangedat10degreesincrements.Resultshaveshownthatthespectralreflectanceanditspeakataparticularwavelengthregion

differedfromeachmetallicfoils,andthattheimagesyieldeddifferentcharacteristics.Thiscouldfacilitateindistinguishingbetweenfoils.Thisprovidessomepromisethatunknownmetallicfoilsmaybeidentifiedthroughthemeasurementoftheirspectroscopicfeatures.Thismethodcouldbecomeausefultoolfortheanalysisofculturalheritage.

8291-46, Session 9

Museum as an integrated imaging device: visualization of ancient Kyoto cityscape from folding screen artifactK.Miyata,U.Oyabu,M.Kojima,NationalMuseumofJapaneseHistory(Japan)

Museumsholdculturalresourcessuchasartworks,historicalartifacts,andfolklorematerials.TheNationalMuseumofJapaneseHistory(NMJH)holdsover200,000oftheculturalresources.Aroleinmuseumsistoexhibittheculturalresources,thereforemuseumscouldbereferredtoasavisualizationdevicefortheinformation-basedsociety.Inthisresearch,amethodtovisualizeahistoricalimagefromculturalresourceswithinteractiveuserinterfacewillbementioned.ThematerialfocusedonistheoldestextantversionofagenreoffoldingscreenpaintingsthatdepictthethrivingcityofKyotointhefourseasons,namedRekihaku’s“ScenesInandAroundKyoto”designatedasanationallyimportantculturalproperty.Over1,400peopleandalotofresidences,temples,andhousesaredrawn,anditisveryimportantinformationresourcetellingusaboutcityscenesandpeople’slifeinKyotoatthattime.

Historicalresearchesweredonebyusingahighresolutiondigitalimageobtainedbyalargescaledscannerthatcanchangethegeometryofthesensor,thentheresearchresultswillbeputonthedatabaseapplicationforaspecialexhibitionheldinNMJHtosupportforvisualizingacityscapeofancientKyotointhevisitors’mind.

8291-47, Session 9

Mapping colors from paintings to tapestries: rejuvenating the faded colors in tapestries based on colors in reference paintingsE.Johansson,M.Strom,ChalmersUniv.ofTechnology(Sweden);D.G.Stork,RicohInnovations,Inc.(UnitedStates)

Weaddressedtheproblemofautomaticallyrecoveringtheoriginalcolorsinfadedtapestriesbydigitallymappingcolorsfrompaintingsorcartoonswhichservedasthereferencesource.Thereareseveraldeepcomputationalchallengesinsuchaprocess.First,theimagesinthesourcecartoonmaydifferinshapesomewhatfromthesourcecartoon,soanymappingalgorithmmustberobustandtoleranttominorshapedifferences.Second,theremaybemissingorshiftedsections(e.g.,anadded,deleted,orshiftedfigure).Third,differentcolorsfadebydifferentamounts.

Weaddressedthesechallengeswithaclusteringschemewithaglobalcostfunctionthatincorporatescolordifferences,locationdifferences,andregion(area)differences,andmanipulated(byhand)relativeweightingsofthesefactorsinordertomapcartooncolorstothetapestry,therebyrejuvenatingitscolors.Wehavedemonstratedouralgorithmonartificialimagesaswellasdetailsofcartoon-tapestrypairs.WewillpresentourrefinedalgorithmonRaphael’scartoonsforthetapestriesinSistineChapelandFrancoisBoucher’s“TheforgeofVulcan.”

Conference 8291


Conference 8292: Color Imaging XVII: Displaying, Processing, Hardcopy, and ApplicationsTuesday-Thursday24-26January2012

Part of Proceedings of SPIE Vol. 8292 Color Imaging XVII: Displaying, Processing, Hardcopy, and Applications

8292-01, Session 1

Spatiochromatic detection, appearance, and motivations for wide-gamut colorS.J.Daly,DolbyLabs.,Inc.(UnitedStates)

Noabstractavailable

8292-02, Session 1

Color assimilation and contrast near absolute thresholdJ.J.McCann,McCannImaging(UnitedStates)

Humanlightresponsespansmorethan10logunitsofdynamicrange.Thatistheratioofradiancesfromsnowonamountaintoptohumandark-adaptedthreshold.Thisrangeispossiblebecausetwotypesofretinalcells:rods(highsensitivity)andcones(daylightvision).Therearemanyfamiliarspatialcolorexperimentsinwhichequallocalstimulimakeunequalappearancesindaylight(ColorAssimilationandColorContrast).Sincetheseexperimentsaretheresultofspatialprocesses,itispossiblethatthedifferentanatomyandphysiologyofrodsandconescouldlimittherangeoftheseeffects.

Remarkably,spatialeffectsatthelowestendofourvisualHDRrangeareverysimilartothoseatthetopinsunlight.Wefoundthatcolorvisionhasconsistentspatialbehaviornearabsolutethresholdandinbrightsunlight.OurexperimentsshowthatColorAssimilationandColorContrastbehavethesameusingrod/Lconecolorvision.OtherachromaticAssimilation/ContrastexperimentsappearedsimilarovertheentireHDRrangeofillumination.Therewerechangesinsharpnessandcolorhue,nevertheless,thespatialcomparisonprocesswasremarkablysimilar,despitethewellknowndifferencesintheanatomyandphysiologyofrodsandcones.

8292-03, Session 1

Multi-illuminant color constancy for HDR images through exposure segmentationH.Zhang,H.Liu,OregonStateUniv.(UnitedStates);S.Quan,BroadcomCorp.(UnitedStates)

Inhighdynamicrangeimaging,astandardpracticeistocomposeanimagethroughexposurebracketingwhichtakesaseriesofexposuresofthesamesceneandthencombinesthemtogether,followedbydynamicrangecompressionandsomecolorprocessingsteps.Thisprocesscaneffectivelyextendtheequivalentdynamicrangeoftheimagesensor,makingmixedlightingconditionamoreworth-consideringissue.Traditionalcolorconstancyalgorithmsfirstestimateaglobalwhitepointofthesceneandthenapplywhitebalanceandcolorcorrectionbasedonthisestimate.Thismethodcouldexaggeratethedifferencebetweentheilluminantswithinthesamescene,makingpartoftheimagebetterandpartoftheimageworse,orhavingtocompromisethecolorsofthewholeimage.

Inthispaper,weproposeacolorconstancyalgorithmforhighdynamicrangeimagestargetedformulti-illuminantsceneswithafocusonthecombinationoftheexposureswithdifferentilluminants.Byutilizingtheinherentdifferencebetweentheilluminants,wefirstcreateasegmentationmasktodifferentiateareaslitbydifferentilluminantsbeforetheyareindividuallyestimated.Thefinalimageisformedby

dissolvingtheexposuresandiscolorcorrectedwithparametersinterpolatedbetweentheilluminantswithmorphologicalprocessingtechniques.

8292-04, Session 2

Tone mapping for HDR images with dimidiate luminance and spatial distributions of bright and dark regionsM.Kitaura,F.Okura,M.Kanbara,N.Yokoya,NaraInstituteofScienceandTechnology(Japan)

Inordertodisplayhighdynamicrange(HDR)imageonlowdynamicrange(LDR)displaydevices,itisnecessarytocompressadynamicrangeoftheHDRimageapproximatelybyusingtonemapping.Inrelatedworkoftonemapping,thereisalimitationtorepresentHDRimagesbecausethemagnituderelationshipbetweenluminancevaluesofpixelsinimagesiskept.Inaddition,tonemappingmethodsbasedonhuman’sperception,whichhavebeenrecentlyproposedtorepresentthelocalinformationofHDRimages,cannotutilizeadynamicrangeeffectivelybecausethemagnituderelationshipbetweenluminancevaluesofadjacentpixelsiskept.Inthispaper,weproposeatonemappingmethodforHDRimageswithtwoseparatedluminancedistributions,whereluminancedistributionsandimageregionsoflightanddarkpartsaredividedclearly.Undertheassumptionthathuman’sperceptiondoesnotfeelasenseofdiscomfortifthemagnituderelationshipbetweenluminancevaluesofpixelsinlightanddarkregionsisreversed,ourmethodappliestonemappingtoeachregionindependently.Inexperiments,weconfirmedthattheresultimagegeneratedbyourtonemappingmethodwasbetterthanresultsofotherrelatedworkforanHDRimagewithtwoseparatedluminancedistributions.

8292-05, Session 2

Color universal design: analysis of color category dependency on color vision typeN.Kojima,Y.G.Ichihara,KogakuinUniv.(Japan);K.Ito,TheUniv.ofTokyo(Japan);M.G.Kamachi,KogakuinUniv.(Japan)

Thisreportisafollow-uptoSPIE-IS&T/Vol.7528051-8andSPIE-IS&T/Vol.786678660J-1-8.

Inaprevioussurvey,weinvestigatedcolorcategoriesofcommonanduncommonpointsofeachcolorvisiontype,trichromat(C-typecolorvision),protan(P-typecolorvision)anddeuteran(D-typecolorvision).Inthepresentstudy,wefirstvalidatedresultsfromtheprevioussurvey.Next,weinvestigatedspecificcolorcategoriesofP-typecolorvisionandD-typecolorvisionthatdonotdependonthecolornamescommontoC-typecolorvision.Finally,weinvestigatedthedifferencethresholdontheblue-yellowaxisoftheCIEchromaticitydiagramforC-typecolorvisionandP-typecolorvision.Asaresult,weproposeacommonchromaticityofcolorsthatthethreecolorvisiontypesareabletocategorizebymeansofcolornamescommontoC-typecolorvision.WealsoproposespecificcolorcategoriesofC-typecolorvision,P-typecolorvisionandD-typecolorvisionandhighlighttheperceptiongapofcolorbetweenC-typecolorvisionandP-typecolorvisionontheblue-yellowaxisoftheCIEchromaticitydiagram.


8292-06, Session 2

Colour perception with changes in levels of illuminationK.F.Baah,Dept.ofHealth(UnitedKingdom)andUniv.oftheArtsLondon(UnitedKingdom)

Thedisplayofuniformcolourstimulithatreliesonappearancestabilityasanimportantcueoridentifierhasdependencyoncurrentstandards.Typicallysuchstimuluswouldbejudgedusingaknownstandardbutinthetargetenvironmenttheilluminancemaybeentirelydifferent.ForcriticalandpracticalappraisalofcolourISO3664:2009iscommonlyusedandrequiresilluminancelevelsof2000lux(+/-250)and500lux(+/-125)respectivelyunderaD50illuminant.UsingISO3664:2009aseriesofpsychophysicalexperimentswereconductedtodeterminewhetherchangesinilluminationlevelsbetween25-3000luxsignificantlyaltercolourperceptionwhenconsideringacceptabilityandperceptibilitythresholdsofuniformcolourstimulus.

Itwasfoundthattheaveragechangeinter-observerperceivedcolourdifferencealtereduptoamaximumof1DE00.However,fortheperceptualcorrelateofhuethiscouldbeofsignificanceiftheacceptederrorofcolourdifferencewasatthethresholdtherebyyieldingthepossibilityofrejectionwithilluminationlevelchanges.Lightnessandchromaontheotherhandexhibitedgreatertoleranceandwerelesslikelytoberejectedwithilluminancechanges.

8292-07, Session 3

Reducing the number of calibration patterns for the two-by-two dot centering modelV.Babaei,R.Rossier,R.D.Hersch,EcolePolytechniqueFédéraledeLausanne(Switzerland)

Thetwo-by-twodotcenteringmodelhasbeendesignedforpredictingthespectralreflectanceofcolorhalftones.Itrequiresmeasuringthereflectancesofalargenumberoftwo-by-twocalibrationtilepatterns.Spectralmeasurementofhundredsorthousandsoftilepatternsiscumbersomeandtimeconsuming.Inordertolimitthenumberofmeasurements,weestimatethereflectancesofmosttwo-by-twocalibrationtilepatternsfromasmallsubsetcomprisinglessthan10%ofalltilepatterns.Bylinearregressionintheabsorptancespacewederiveatransformationmatrixconvertingtilepatterncolorantsurfacecoveragestoabsorptances.Thistransformationmatrixenablescalculatingtheabsorptanceofallremainingtwo-by-twotilepatterns.Foracyan,magentaandyellowprint,with72two-by-twomeasuredcalibrationtilepatterns,weareabletocreateatwo-by-twodotcenteringmodelhavinganaccuracyslightlybelowtheaccuracyofthemodelwiththefullymeasuredsetof1072two-by-twotilepatterns.

8292-08, Session 3

Spatial gamut mapping for preserving the details of an imageI.Song,H.Ha,W.Kyung,Y.Ha,KyungpookNationalUniv.(Korea,Republicof)

Arecenttrendingamutmappingalgorithm(GMA)researchistheuseofspatialinformationtocomputethecolortransformationofpixelsfromtheinputtotheoutputgamut.Toachievethis,low-/high-passfilteringorpyramidaldecompositionisoftenused.However,thesetechniquessharetheproblemofhalogenerationinthecaseofsharpboundaries,whichisinducedbythespatialcomputationitself.AnotherproblemwithspatialGMAsisthehueshiftinducedfromcompressingagroupofpixelstowardsagrayusingthesamevector,thuspixelsthatdonotneedtobechangedareerroneouslymapped.

Spatialgamutmappingusingaregularizationprocessisproposedwiththeaimofmaintainingthehueanddetailsoftheoriginalimageasmuchaspossiblewithinthegamutlimitsofthereproductionmedium.Thus,forthedetails,thedifferencebetweentheoriginalimageandits

low-passfilteredimageisadded,whilethehueispreservedthroughpost-processing.TheentireprocessisimplementedusingCIELABcolorspace,andtheimageisfilteredinallthreeorthogonalchannels.

8292-09, Session 3

Evaluating color calibration kits with virtual displayW.Cheng,H.Caceres,A.Badano,U.S.FoodandDrugAdministration(UnitedStates)

AnewmethodofevaluatingtheperformanceofcolorcalibrationkitsforLCDmonitorsisproposed.Routinecolorcalibrationisimperativeformedicalapplicationsthatrelyoncolorfidelitysuchasdigitalpathology,endoscopy,andcolposcopy.However,thecommerciallyavailableproductsvarygreatlyinpriceandperformancewithnoavailableevaluationstandard.WeproposetheconceptofVirtualDisplay,auniversaldisplayplatformthatemulatesthecolorimetricresponseofrealdisplays.Afieldprogrammablegatearray(FPGA)boardwasusedtoprocessthevideosignalsbasedonapre-programmedlook-uptable,whichcontainsthecharacterizationdataoftherealdisplay.Awide-color-gamutdisplaywasdrivenbytheFPGAtoemulatethecolorimetricresponseoftherealdisplay.Thevirtualdisplay’shardware-basedapproachistransparent,sothecolorcalibrationkitscancalibrateitasaregulardisplaywithoutknowingthedifference.Bychangingthelook-uptableintheFPGA,thevirtualdisplaycanemulatevariousrealdisplaysfortestvariouscolorcalibrationkits,whichisdifferentfrompreviousworkwhereonlyonedisplaywasused.Ourexperimentalresultsshowthatthevirtualdisplaycanemulaterealdisplayswithin3.64delta-E-abdiscrepancies.Suchaccuracywasachievedbyourmethodofspectralcharacterizationandmodeling.

8292-10, Session 3

Optimizing color fidelity in wide-gamut-display devices when processing images compressed by block-based DCT transformsF.Lebowsky,STMicroelectronics(France)

High-endmonitorsbasedonLCDtechnologyincreasinglyaddresswidecolorgamutimplementationsfeaturingprecisecolorcalibrationwithinavarietyofdifferentcolorspacessuchasextendedsRGBorAdobeRGB.However,imagesareoftenreconstructedfromdigitallycompressedimagesfilessuchasJPEGorMPEGwherecolorqualitycouldbequestionable.Inparticular,whensuchimagefilesarescaleduporzoomedin,differenttypesofimageartifactsbecomevisuallynoticeable.Amongtheseartifactswefindpixelation,blockiness,ringing,andcolorblotching.Whilepixellationandringingappearduetoinsufficientadaptationtoimagecontextusingastaticorcontextadaptivefilterkernelintemporaldomain,blockinessandringingoccurduetoimagecompressioninfrequencydomain,whenimagecompressionfactorsarehigh.Inaddition,chrominancechannelsoftenundergoanevenhighercompressionratiothatamplifiesvisibilityofartifactssuchascolorblotches.Consequently,weareinterestedinimprovingthequalityofimagestobedisplayeddependingonthescalefactorrequestedbyauserapplication.WeproposetodiscriminatemostrelevantvisualartifactsusingpowerspectrumanalysisinDCTdomainaswellasafterkernelbasedrescalingcombinedwithstatisticalanalysisbasedonnon-linearregressiontakingintoaccountcharacteristicnon-stationarybehaviorofimagecontentandidentifiablevisualartifacts.Acomparativeanalysisbasedonsomecompetitivesolutionshighlightstheeffectivenessofourapproachandidentifiesitscurrentlimitationswithregardtowidecolorgamutrepresentationduetomathematicaluncertaintyofthestudiedartifacts.

Conference 8292


8292-11, Session 3

Optimal gamut volume design for three primary and multiprimary display systemsC.E.Rodriguez-Pardo,G.Sharma,Univ.ofRochester(UnitedStates);X.Feng,J.Speigle,I.Sezan,SharpLabs.ofAmerica,Inc.(UnitedStates)

Primaryselectionplaysafundamentalroleindisplaydesign.Primariesaffectnotonlythegamutofcolorsthesystemsisabletoreproduce,butalso,theyhaveanimpactonthepowerconsumptionandothercostrelatedvariables.Usingmorethanthetraditionalthreeprimarieshasbeenshowntobeaversatilewayofextendingthecolorgamut,wideningtheangleviewofLCDscreensandimprovingpowerconsumptionofdisplayssystems.Adequateselectionofprimariesrequiresatrade-offbetweenthemultiplebenefitsthesystemoffers,thecostsandthecomplexityitimplies,amongotherdesignparameters.

Thepurposeofthisworkistopresentamethodologyforoptimaldesignforthreeprimaryandmultiprimarydisplaysystems.Weconsiderthegamutinperceptualspaces,whichoffertheadvantageofanevaluationthatcorrelateswithhumanperception,anddetermineadesignthatmaximizethegamutvolume,constrainedtoacertainpowerbudget,andanalyzethebenefitsofincreasingnumberofprimaries,andtheireffectonothervariablesofperformancelikegamutcoverage.

8292-12, Session 4

The dark side of CIELABG.Sharma,C.E.Rodriguez-Pardo,Univ.ofRochester(UnitedStates)

Standardizedin1976asauniformcolorspace,CIELABisextensivelyutilizedincolorscienceandengineeringapplications.CIELABprovidesbothacolordifferenceformulaandcorrelatesforcommonperceptualdescriptorsofcolor.Deficienciesinbothareasarewell-known,andbasedontheseknownlimitations,numerousfixeshavebeendevelopedyieldingalternativecolordifferenceformulaethatarederivedasmodificationsofthecolordifferenceinCIELAB.Inaddition,severalnewcolorappearancespaceshavealsobeenproposedasmodificationsofthebasicCIELABframework.Inthispaper,wepointoutother,lesser-knownandpoorly-appreciated,limitationsofCIELABthatoccurparticularlyinthedarkregionsofcolorspace.Wedemonstrateviaexamples,howtheselimitationsnotonlycauseperformancecompromisesbutleadtofundamentalbreakdownsinsystemoptimizationanddesignproblems,makingCIELABcompletelyunusableintheseproblems.WeconsiderthereasonswhythesefundamentallimitationswereoverlookedintheoriginaldevelopmentofCIELABandanalyzethemathematicalrepresentationscontributingtotheundesiredbehavior.WearguethatfundamentalnewresearchisrequiredtoovercomethisdarksideofCIELAB;thedevelopmentofuniformcolorspacesandnewcolorappearancespacesmustberevisitedafreshusingnewexperimentaldataandkeepinginmindnewerdevicesandapplications.

8292-13, Session 4

Complexitites of complex contrastE.Peli,SchepensEyeResearchInstitute(UnitedStates)

Noabstractavailable

8292-14, Session 4

It’s not the pixel count, you foolM.A.Kriss,MAKConsultants(UnitedStates)

Thefirstthinga“marketingguy”asksthedigitalcameraengineeris“howmanypixelsdoesithave,forweneedasmanymegapixelsaspossiblesincetheotherguysarekillinguswiththeir“umpteen”mega

pixelpocketsizeddigitalcameras.Andsoitgoesuntilthepixelsgetsmallerandsmallerinordertoinflatethepixelcountinthenever-endingpixel-wars.Thesesmallpixelsjustarenotverygood.Thetruthofthematteristhatthemostimportantfeatureofdigitalcamerasinthelastfiveyearsistheautomaticmotioncontroltostabilizetheimageonthesensor.Alltheresthasbeenhypeandsome“cool”design.Whatisthefuturefordigitalimagingandwhatwilldrivegrowthofcamerasales(notcountingthecellphonecameraswhichtotallydominatethemarketintermsofcamerasales)andmoreimportantlyaftersalesprofits.WellsitinontheDarkSideofColorandfindoutwhatisbeingdonetoincreasetheaftersalesprofitsanddon’tbesurprisedifhasbeendonelongagoinsomebasementlabofaphotographiccompanyandofcourse,beforeitstime.

8292-15, Session 4

Color imaging and aesthetics: is there the cheshire cat?E.A.Fedorovskaya,EastmanKodakCo.(UnitedStates)

Noabstractavailable

8292-16, Session 4

Dark texture in artworksC.E.Parraman,Univ.oftheWestofEngland(UnitedKingdom)

Noabstractavailable

8292-17, Session 4

Harmonious colors: from alchemy to scienceG.B.Beretta,N.M.Moroney,Hewlett-PackardLabs.(UnitedStates)

Thereisaverylongtraditionindesigningcolorpalettesforvariousapplications.Althoughcolorpaletteshavebeeninfluencedbytheavailablecolorants,startingwiththeadventofanilinedyesinthelate1850stherehavebeenfewphysicallimitsonthechoiceofindividualcolors.Thisabundanceofchoicesexacerbatestheproblemoflimitingthenumberofcolorsinapalette.

Thetraditionalsolutionisthatof“colorforecasting.’’Colorconsultantsassessthesentimentoraffectivestateofatargetcustomerclassandcompareitwithcolorantsofferedbytheindustry.Thecolorforecastingbusinessisverylaborintensiveanddifficult,thusforyearscomputerengineershavetriedtocomeupwithalgorithmstodesignharmoniouscolorpalettes,alaswithlittlecommercialsuccess.

Wearguethattheintellectualflawresidesinthebeliefthatamasterfulindividualcandevisea“perfectmethodology’’thattheengineercanthenreducetopracticeinacomputerprogram.Wesuggestthatthecorrectapproachistoconsidercolorforecastingasanactofdistillation,whereapaletteisdigestedfromthesentimentofaverylargenumberofpeople.Wedescribehowthisapproachcanbereducedtoanalgorithm.


Characterization of color scanners based on SVRB.Li,Y.Zhang,JiangnanUniv.(China)

ByresearchingtheprincipleofcolorimetriccharacterizationmethodandSupportVectorRegression(SVR),weanalyzethefeasibilityofnonlineartransformationfromscannerRGBcolorspacetoCIELABcolorspacebasedonSVRandbuiltanewcharacterizationmodelforscanner.ThenweusetheMATLABr2009asoftwaretomakeadatasimulationexperimenttoverifytheaccuracyofthismodelandfigure

Conference 8292


outthecolordifferencesbyCIEDE2000colordifferenceformula.Theexperimentalresultsshowthatthesimilarityismorethan99%betweenpredictedvaluesandtruevaluesofL,a,b.BasedonCIEDE2000colordifferenceformula,theaveragecolordifferencesoftrainingsetandtextsetare1.2376and1.9318.Fromourresults,wecanmakeaconclusionthatSVRcanrealizethenonlineartransformationfromscannerRGBcolorspacetoCIELABcolorspaceandthemodelsatisfiestheaccuracyofscannercharacterization.Therefore,SVRcanbeusedintothecolorscannercharacterizationmanagement.


Deducing ink thickness variations of fluorescent print by a spectral prediction modelJ.Q.Wang,Y.Zhang,JiangnanUniv.(China)

BytheClapper-Yulespectralpredictionmodel,wedevelopamethodologytodeduceinkthicknessvariationsofprintingonthefluorescentsubstrate.Intheinvestigatione,weextendtheClapper-Yulespectralpredictionmodelofprintingonthefluorescentsubstratebyconsideringmultipleinternalreflectionsunderthefluorescentsubstrateandtherelationshipbetweenthetransmittanceandtheinkthickness.Atthesametime,wededucetheinkthicknessvariationsbyintroducingtherelativeinkthicknessfactorparametersandtheinkthicknessvariationcoefficient.Byusingtheleast-squaresparameterestimationmethod,thedeductionmodeloftheinkthicknessvariationisdeveloped.Thecorrectnessoftheconclusionsofisverifiedbyexperiments.Intheexperiment,aseriesofcoloroffsetprintingisusedtoverifyinkthicknessvariationsofprintingonthefluorescentsubstratebyaspectralpredictionmodel.Andthepagescomprisingthecalibrationpatchesandthetestpatchesareprintedonuncoatednewsprintpaper,withclassicalmutuallyrotatedclustered-dotscreens,atascreenfrequencyof100linesperinch.


Bio-inspired color sketch for eco-friendly printingI.V.Safonov,E.Tolstaya,M.N.Rychagov,SamsungElectronicsCo.,Ltd.(RussianFederation);H.K.Lee,S.Kim,D.Choi,SamsungElectronicsCo.,Ltd.(Korea,Republicof)

Savingoftoner/inkconsumptionisanimportanttaskinmodernprintingdevices.Ithasapositiveecologicalandsocialimpact.Weproposetechniqueforconvertingprint-jobpicturestoarecognizableandpleasantcolorsketches.Drawinga“pencilsketch”fromaphotorelatestoaspecialareainimageprocessingandcomputergraphics-non-photorealisticrendering.Wedescribeanewapproachforautomaticsketchgenerationwhichallowstocreatewell-recognizablesketchesandtopreservepartlycolorsoftheinitialpicture.Oursketchescontainsignificantlylesscolordotstheninitialimagesandthishelpstosavetoner/ink.Ourbio-inspiredapproachisbasedonsophisticatededgedetectiontechniqueforamaskcreationandmultiplicationofsourceimagewithincreasedcontrastbythismask.ToconstructthemaskweuseDoGedgedetection,whichisaresultofblendingofinitialimagewithitsblurredcopythroughthealpha-channel,whichiscreatedfromSaliencyMapaccordingtoPre-attentiveHumanVisionmodel.Measurementofpercentageofsavedtoneranduserstudyproveseffectivenessofproposedtechniquefortonersavingineco-friendlyprintingmode.


Reflectance model for recto-verso color halftone imagesJ.Wang,Y.Zhang,T.D.Wen,JiangnanUniv.(China)

Inthecolorreproductionprocess,accuratelypredictingthecolorofrecto-versoimagesandestablishingaspectralreflectancemodel

forhalftonesimagesarethegreatconcernprojectofimagingqualitycontrolfield.Thescatteringoflightwithinpaperandtheinkpenetrationinthesubstratearethekeyfactors,whichaffectthecolorreproduction.Areflectancemodelforrecto-versocolorhalftoneprintsisintroducedinthispaperwhichconsidersthesefactors.Thepaperbasedontheassumptionthatthecolorantisnon-scatteringandtheassumptionthatthepaperisstrongscatteringsubstrate.Bythemultipleinternalreflectionbetweenthepapersubstrateandtheprint-airinterfaceoflight,andthelightalongobliquepathoftheWilliams-Clappermodel,weproposedthecolorspectralreflectanceprecisepredictionmodelofrecto-versohalftoneimages.Inthestudy,weproposethismodelfortakingintoaccountinkspreading,aphenomenonthatoccurswhenprintinganinkhalftoneinsuperpositionwithoneorseveralsolidinks.Theink-spreadingmodelincludesnominal-to-effectivedotareacoveragefunctionsforeachofthedifferentinkoverprintconditionsbytheleastsquarecurvefittingmethod,sothefunctionsforphysicaldotgainofvariousoverprinthalftonesaregiven.Thismodelprovidedatheoreticalfoundationforcolorpredictionanalysisofrecto-versohalftoneimagesandthedevelopmentofimagequalitydetectionsystem.


The study on physical dot gain of second order FM halftone based on ink spreading in all ink superposition conditionsS.Xi,Y.Zhang,JiangnanUniv.(China)

Inthispaper,thestudyisfocusedonthesecondorderFMhalftoneprints.AccordingtothefeatureofthesecondorderFMhalftonedots,anewspectralpredictionmodelforthesecondorderFMhalftoneprintsbasedonthespectralNeugebauerrandommixingmodelanddot-on-dotmixingmodelisdeveloped,usingaweightingfactortosignifythecorrelationproportionoftwomodels.Duetoprintingprocess,interactionamongink,paperandair,therewillbephysicalandopticaldotgain.ThenewspectralpredictionmodelalsoreliesonYule-NielsenmodifiedSpectralNeugebauermodelbytakingintoaccountinkspreadinginallinksuperpositionconditions.Wheninkdotsareprintedondifferentsuperposition,inkspreadingisdifferent,sowecreateaninkspreadingfunctionforeachsuperpositioncondition,whichmapsnominaltoeffectivedotsurfacecoverage.Wheninkdotsareprinted,theirknownnominalsurfacecoveragevaluesareconvertedintoeffectivecoveragevalues.Usinginkjetprintertoimitatetheexperiment,themeanandmaxΔEdifferencebetweenmeasuredandpredictedvaluesare2.52and6.26,smallerthanthedifferencepredictedbythemodelthatisnottakenintoaccountinkspreading,whichare5.02and12.81.Thepredictionaccuracyisconsiderablyimprovedbythenewmodel.


Tensor decomposition for color printer model lookup tableV.Monga,ThePennsylvaniaStateUniv.(UnitedStates);M.Maltz,Z.Fan,XeroxCorp.(UnitedStates)

Noabstractavailable.


Genetic algorithm for segmentation and classification of colony images automaticallyW.Li,HenanPolytechnicUniv.(China)

Imagesegmentationandclassificationofcolonyimagesplaysakeyroleinautomaticvisualsystems.Thispaperdescribesanewalgorithmusingforsegmentationandclassificationofcolonyimages.Itisbasedonageneticapproachthatallowustoconsiderthesegmentation

Conference 8292


problemasaglobaloptimization,andthenewclassifierintroducedhereisbasedonfuzzy-integrationschemescontrolledbyageneticoptimizationprocedure.Twodifferenttypesofintegrationareproposedhere,andarevalidatedbyexperimentsonrealdatasetsforMachine.Resultsshowthegoodperformanceandrobustnessoftheintegratedclassifierstrategies.


Color edge detection using edge densityW.Wang,HenanPolytechnicUniv.(China)

Thispaperpresentsanoveledgedetectionalgorithmusingthelocal,nonparametricestimationofthecolorimagedensity.Themethodanalysestheshapeinformationprovidedbythelocalprobabilitydistributionoftheimageinthehorizontalandverticaldirectionrespectively,thenobtainsthemodulusforedgedetection.Withtheincreasingofwindowsize,theothertypesofdistributionscanbesimplifiedtothethreetypesofdistributionspresentedinthispaper.Experimentsshowthemethodisefficient.


Spectral prediction model for variable dot-size ink jet pressworkW.Xing,Y.Zhang,YangtzeUniv.(China)

Thevariabledot-sizeinkjetprinterscanbasethedensityofdifferentareintheimage,chooseinkdrop’ssizeachievevariabledot-size.Usingvariabledot-sizeFMscreening,recentinkjetprinterscanimproveprintquality.ButtheexistingFMscreeningspectralreflectancemodelcan’tpredictthereflectivityofsuchprintedmatter.Now,wefoundapredictionmodel,whichiscapableofpredictingvariablepixelsdot-sizespectralreflectance.Wheninkjetprinters’dropsizechange,basedonClapper-Yulemodel,studyreflectancepredictionmodelforvariabledot-sizeinkjetpresswoke.Atfirst,thisarticleintroducevariabledropsizeandtheFMscreeningformofClapper-Yulemodel;then,usingthechangeofinkfilmthicknessandFMscreeningdot,whicharecausedbydropchange,extendedClapper-Yulemodel;atlast,establishedtheClapper-Yulemodelwhichisextendedbydropsize,inaddition,validatedtheadvantageofthemodelbysimulation.Spectralpredictionmodelforvariabledot-sizeinkjetpresswokeadministertohowcanwegethigherprintqualitypresswokeusevariabledot-sizeinkjetprinters.

8292-18, Session 5

Detection and compensation of backlight images using chrominanceH.J.Park,S.W.Han,SamsungElectronicsCo.,Ltd.(Korea,Republicof)

Animagetakenunderthebacklightconditionshowsthatamainforegroundorobjectappearsverydark,butabackgroundappearsrelativelybrightsincetheexposuretimeofthemainobjectorforegroundisrelativelyshorterthantheoneofthebackgroundduetohighluminancefromthebackground.Thedeterminationofabacklightimageisgenerallydonebyluminancehistogramanalysissinceitisbelievedthatthedistinctcharacteristicofthebacklightimageisalargeluminancedifferencebetweenforegroundandbackground.However,thisconventionaldetectionmethodmaynotbeadequateforvideoimagessinceitgenerallytargetsonstillimages.Furthermore,thedetectionofbacklightimageswouldnotbeperformedwellifthereareabruptchangesinlight,motion,orscenes.Inaccuratedetectionleadstounnecessarycompensationthatmakesimageover-highlightedorflickered,especiallywhenconsecutiveframesofvideohavedifferentilluminationmodes.Sinceanimagetakenundernormallightconditionsmayalsohavethesimilarluminancecharacteristicsofbacklightimages,usingonlyluminanceinformationwouldnotbesufficientto

discriminatebetweennormalandbacklightimages.Therefore,theanalysisofchrominanceofimagesisintroducedtodetectbacklightimagesmoreaccurately.

8292-20, Session 5

A new method for skin color enhancementH.Z.Zeng,Hewlett-PackardCo.(UnitedStates);R.Luo,Univ.ofLeeds(UnitedKingdom)

itpleasinglyisanimportantfactorinphotographiccolorreproduction.Movingskincolorstowardtheirpreferredskincolorcenterimprovesthecolorpreferenceforskincolorreproduction.Twokeyfactorstosuccessfullyadjustskincolorsare:amethodtodetectoriginalskincolorseffectivelyeveniftheyareshiftedfarawayfromtheregularskincolorregion,andamethodtomorphskincolorstowardapreferredskincolorregionproperlywithoutintroducingartifacts.Inthesameconferencelastyear,wepresentedamethodforskincolorenhancement.Astatisticalskincolormodelwasusedforskincolordetection,whichmaymisstodetectskincolorsthatarefarawayfromregularskintons.Inthispaper,anewmethodusingthecombinationoffacedetectionandstatisticalskincolormodelingisproposedtoeffectivelydetectskinpixelsandtoadjustskincolorseffectively.

8292-21, Session 6

CIE chromaticity, Planckian locus, and correlated color temperature estimation from raw-image data using colorchecker training imagesT.Tajbakhsh,TechnischeUniv.Hamburg-Harburg(Germany)

Themajorproblemdiscussedinthepaperishowgoodrawsensorchromaticities,i.e.r=R/(R+G+B)andb=B(R+G+B),canbemappedintotheCIE1931xychromaticityspacegiventheconstraintthatsolelytrainingimagesarebeingusedwhicharecapturedfromacolorcheckerilluminatedwithacertainemissionspectrum.WiththesemappingswenextinvestigatehowgoodthePlanckiananddaylightlocuscanbeestimatedintherawsensorchromaticityspace.Weevaluateatwo-dimensionalpolynomialfunctiontomapthelocuschromaticitiesfromCIExytoraw.Ourproposedweightingstrategyofthetrainingchromaticitiesimprovesthelocusmappingsignificantly.However,weshowthattwoone-dimensionalparables,similartothosedefinedbytheCIEtodescribethedaylightlocuschromaticities,outperformlattertwo-dimensionalpolynomials.Thepaperfurthermoredealswiththequestionhowgoodthecorrelatedcolortemperatureofarawchromaticitycanbedetermined.Weproposelinearregressionofa2nd-orderreciprocalpolynomialstomaprbcoordinatestoCCTvaluesandcomparetheperformancewithtwoothermethods.Wepresentacomprehensivesetofsimulationresultswhereweevaluatethreefiguresofmeritfordifferentsetupsandmappingmethods.Finally,wedemonstratetheproposedmethodsona5megapixelsensorandverifytheresultsbymeasurement.

8292-22, Session 6

Comparative performance analysis of mobile displaysR.Safaee-Rad,QualcommInc.(Canada);M.Aleksic,QualcommInc.(UnitedStates)

Cell-phonedisplayperformance(intermsofcolorqualityandfidelity)hasbecomeacriticalfactorincreatingapositiveuserexperience.Asaresult,thereisasignificantamountofeffortbycell-phoneOEMstoprovideamorecompetitivedisplaysolution.Thiseffortisfocusedonusingdifferentdisplaytechnologies(withsignificantlydifferentcolorcharacteristics)andmoresophisticateddisplayprocessors.

Conference 8292


Inthispaper,theresultsofadisplaycomparativeperformanceanalysisarepresented.Afewcell-phonesfrommajorOEMsareselectedandtheirdisplayperformancesaremeasuredandquantified.

8292-23, Session 6

Termites: a Retinex implementation based on a colony of agentsG.Simone,GjøvikUniv.College(Norway);G.Audino,Univ.degliStudidiMilano(Italy);I.Farup,GjøvikUniv.College(Norway);A.Rizzi,Univ.degliStudidiMilano(Italy)

ThispaperdescribesanovelimplementationoftheRetinexalgorithmwiththeexplorationoftheimagemadebyanantswarm.Inthiscasethepurposeoftheantcolonyisnottheoptimizationofsomeconstraintbuttheexplorationasdiffusedaspossibleoftheimagecontent,withthepossibilityoftuningtheexplorationparametersontheimagecontent.Forthisreason,thisapproachiscalled“termites’’,insteadofants,tounderlinetheideaoftheeagerexplorationoftheimage.ThepaperpresentsthespatialcharacteristicsoflocalityanddiscussesdifferenceswithotherRetineximplementation.

8292-24, Session 6

A color quantization algorithm based on minimization of L_p norm error in a modified CIELAB spaceH.Xue,J.P.Allebach,C.A.Bouman,PurdueUniv.(UnitedStates)

Colorquantizationalgorithmsareusedtoselectasmallnumberofcolorswhichcanaccuratelyrepresentthecontentofaparticularimage.Colorquantizationcontinuestobeimportantbecauseitcanbeusedtosegmentimagesintoregionsofuniformcolor,improveimagecompression,andremovenoise.However,inordertobemosteffective,colorquantizationalgorithmsmustaccuratelyrepresenttheimagewiththeminimumnumberofcolors,andmustbecomputationallyefficient.

Inthisresearch,weintroduceanovelcolorquantizationalgorithmwhichisbasedontheminimizationofanL_pnormratherthanthemoretraditionalL_2normassociatedwithMSE.WedemonstratethattheL_poptimizationapproachhastwoadvantages.First,itdistributesthecolorsmoreuniformlyovertheregionsoftheimage;andsecond,thenorm’svaluecanbeusedasaneffectivecriterionforselectingtheminimumnumberofcolorsnecessarytoachieveanaccuraterepresentationoftheimage.OnepotentialdisadvantageoftheL_pnormcriterionisthatitcouldincreasethecomputationoftheassociatedclusteringmethods.However,wesolvethisproblembyintroducingatwostateclusteringprocedureinwhichthefirststage(pre-clustering)agglomeratesthefullsetofpixelsintoarelativelylargenumberofdiscrete-colors;andthesecondstagepost-clustering)performsL_pnormminimizationinamodifiedCIELABspaceusingthereducednumberofdiscrete-colorsresultingfromthepre-clusteringstep.Thistwo-stageclusteringprocessdramaticallyreducescomputationbymergingtogethercolorsbeforethecomputationallyexpensiveL_pnormminimizationisapplied.

8292-25, Session 7

Human skin imaging using three-phase spectral matching imagerA.Kimachi,OsakaElectro-CommunicationUniv.(Japan);S.Ando,TheUniv.ofTokyo(Japan);M.Doi,S.Nishi,OsakaElectro-CommunicationUniv.(Japan)

Thispaperproposesanimagingmethodforhumanskinusingthethree-phasespectralmatchingimager(3PSMI),whichconsistsofthetime-domaincorrelationimagesensor(CIS)andwavelength-sweptmonochromaticillumination(WSMI).

EachpixeloftheCISperformstemporalcorrelationbetweentheintensitysignalandexternalthree-phasereferencesignalsgeneratedfromanorthonormalpairofreferencespectraineachframe,producingimagesofcorrelationbetweentheobjectspectrumandthereferencespectra.

Thesimilarityoftheobjectspectrumtothereferencespectraisevaluatedastheazimuthangleoftheprojectionoftheobjectspectrumontotheplanespannedbythereferencespectra.

Ofthetwomethodsofgeneratingreferencespectra,onemethodusesanaveragespectralreflectanceofhumanskinandaspectralreflectancerepresentativeofnon-humanskin.

Theothermethodusesonlythedifferentialcomponentsofspectralreflectancesofhumanskinwithoutanaveragespectrum,whichrepresentcharacteristicfeaturesofhumanskinsuchascongestionandsuntan.

Experimentalresultsdemonstrateinrealtimethattheuseofhuman/non-humanreferencespectradistinguisheshumanskinfromnon-humanobjects,whiletheuseofcongestion/suntanreferencespectraclassfiesthepartsofhumanbodyaccordingtothelevelofcongestionandsuntan.

8292-27, Session 7

Appearance analysis of human skin with cosmetic foundationR.Ohtsuki,KaneboCosmeticsInc.(Japan)andChibaUniv.(Japan);S.Tominaga,ChibaUniv.(Japan);R.Hikima,KaneboCosmeticsInc.(Japan)

Thepresentpaperdescribesananalysismethodofthesurfaceappearanceofthemake-upface.Weproposeamethodfordetectingoilyappearanceoffacialregiondeterioratedbythesebum.First,amulti-bandcamerawithsixspectralchannelswasusedforcapturingfacialimage.Next,weperformedsubjectiveassessmentofoilyappearanceforthemake-upface.Third,wecomparedtheoilyregionswiththeoriginalregionsamongthefacejustafterthemake-up.Itisfoundthattheoilyregionsarewhiterthantheoriginalregions,andsotheaverageluminancelevelishigherthantheoriginal.Moreoverthestandarddeviationoftheluminanceislargerthattheoriginal.Basedonthesefindings,wedevelopamethodfordetectingtheregionswithoilyappearancefrommake-upfacialimages.First,weextractthecandidateregionswithoilyappearanceonthebasisoftheluminancevalueY.ThePCAisappliedtotheluminanceandchromaticityimage(Y,x,y)ofeachcandidateregion.Thenalinearclusterofthepixeldistributionisformedonthe2Dplane.Finally,wecandeterminetheoilyappearancebycomparisontwolinearclustersbetweentheoriginalfacialimageandtheimagefourhourlater.

8292-28, Session 7

Color analysis and image rendering of woodblock prints with oil-based inkT.Horiuchi,T.Tanimoto,S.Tominaga,ChibaUniv.(Japan)

Thispaperproposesamethodforanalyzingcolorcharacteristicsofwoodblockprintswithoil-basedinkandrenderingrealisticimagesbasedonthecameradata.Theanalysisresultsofwoodblockpaintsshowsomecharacteristicfeaturesincomparisonwithoilpaintings:(1)Woodblockprintscanbedividedintosomeclusterareaswithsimilarsurfacespectralreflectance,(2)Thespecularreflectionbytheinfluenceofoverlappingpaintsiscausedonlyonspecificclusterareas.Basedonthoseproperties,wedevelopanalgorithmforeffectiverenderingbymodifyingourpreviousalgorithmforoilpaintings.Asetofsurfacespectralreflectancesofawoodblockprintisrepresentedbyusingonlyasmallnumberoftheaveragedsurfacespectralreflectancesandtheregisteredratios,whilethepreviousalgorithmforoilpaintingsrequiredhighdimensionalsurfacespectralreflectancesatallpixels.Intherenderingprocess,weusetwosetsofparametersoftheCook-Torrancemodelforclusterareaswith/withoutspecularreflection.An

Conference 8292


experimentusingwoodblockpaintwithoil-basedinkisexecutedfordemonstratingthefeasibilityoftheproposedmethod.Itisshownthatrealisticimagesoftheoriginalwoodblockprintcanbereproducedusingasmalldatasetofsurfacespectralreflectances.

8292-30, Session 8

Pre-RIP color management for soft proofingI.Tastl,K.Koh,Hewlett-PackardLabs.(UnitedStates)

Inanidealsituation,acolor-managedsoftproofisgeneratedbythesamerasterimageprocessor(RIP)thatproducesthefinalrasterdataforprinting.Inreal-worldscenarios,however,thissoftproofingcapabilityisoftenabsent.Toovercomethislimitation,wedevelopedaworkflowusingtheAdobePDFlibraryforproducingcolor-managedsoftproofsbeforetheincomingjobsaresenttotheRIP.Thispaperdiscussesthecolormanagementchallengesthatpresentedthemselvesandhowtheywereaddressed.Italsohighlightsafewspecificcaseswherethedivisionoftasksintoseparatecolormanagementandtransparencyblendingstepscausedunexpectedresultsandhowtheycanbedealtwith.Suchasystemcanbeimplementedintheformofacloud-basedsolutiontoprovideend-customerswithaquick,inexpensive,andreasonablyaccuratepreviewoftheprintedartifactbeforethejobisroutedtotheRIP.

8292-31, Session 8

Parametrically controlled, stochastically seeded clustered halftonesE.A.Bernal,S.Wang,R.P.Loce,XeroxCorp.(UnitedStates)


8292-32, Session 8

Assessing color reproduction tolerances in commercial print workflowG.B.Beretta,E.Hoarau,S.Kothari,I.Lin,J.Zeng,Hewlett-PackardLabs.(UnitedStates)

Colortransformationsfromcolorimetricspecificationstodevicecoordinatesaremostlyobtainedbymeasuringasetofsamples,invertingthetable,andlookingupvaluesinthetable,andmappingthegamutfrominputtooutputdevice.

Theaccuracyofatransformationisdeterminedbyreproducingasecondsetofsamplesandmeasuringthetransformationerrors.Accuracyastheaverageperceptualerroristhenusedasametricforquality.AccuracyandprecisionareimportantmetricsincommercialprintbecauseaPSPcanchargeahigherpriceformoreaccuratecolor,orcanwidenhistoleranceswhencustomerswantcheapprints.

Thedisadvantageofdeterminingtolerancesthroughaveragingperceptualerrorsisthatthecolorsinthesamplesetsareindependentandthisisnotnecessarilyagoodcorrelateofprintqualityasdeterminedthroughpsychophysicsstudies.Indeed,imagesconsistofcolorpalettesandthemainqualityfactorisnotcolorfidelitybutcolorintegrity.Forexample,ifthedivergenceofthefieldoferrorvectorsiszero,colorconstancytakesoverandhumanswillperceivethecolorreproductionasbeingofgoodquality,eveniftheaverageerrorisrelativelylarge.However,iftheerrorsaresmallbutinrandomdirections,theperceivedimagequalityispoor.

Ourproposalforastandardpracticetodeterminetoleranceistousecolordiscriminationsamplesforthesecondsetandtoevaluatethecolortranspositionsinsteadofthecolordifferences.Thereareindustrystandardsforthetolerancesofcolorjudges,andthesametolerancesandclassificationcanbeuseforprintworkflowsoritscomponents

(e.g.,presses,proofers).Wegeneralizethispracticetoarbitraryperceptuallyuniformscalestailoredtospecificapplicationsandpresentanimplementation.

8292-33, Session 8

Investigation of the paper dependency of laser-printed colors for uncoated papersS.GorjiKandi,InstituteforColorScience&Technology(Iran,IslamicRepublicof)

Thereareseveralparameters,whichinfluencethefinalprintedcolorsindigitalprinting.Oneofthemostimportantoneisthesubstrate(paper).Thechemicalandopticalcharacteristicsofthepapermayaffecttheimagequalityandcolorimetricpropertiesoffinalprints.

Inthepresentstudy,theeffectofuncoatedpapersonthelaser-printedcolorsisinvestigated.Forthispurpose,fourkindsofpaperwithdifferentmarksandthesamegrammagevalueof80wasapplied.Thepapers’diversitywasalmostbetweentheirwhitenessandtexture.TheeffectofpaperwasinvestigatedforaCanonLaserPrinterLBP-5050.

Theeffectofpaperontheprinter’sreproducibility,andlight-fastnessoftheprintedcolorswasevaluatedforacolorchartcontains459samples,whichwasgeneratedbyEye-OneGretagMacbethprofessionalmaker.Thecolorimetriccharacteristicsofthefinalprintedsampleswerestudiedutilizingachartof2125patches.Itwasfoundoutthat,thereproducibilityoftheprinterissignificantlydependentonthepaper’stype.Theaverageofcolorchangeviaexposinglightisbetween1.3and1.8fordifferentpapers.

Thecolordifferenceviathepaperchangeclearlyillustratestheeffectofpaperonthefinalprintedcolors.Oneoftheappliedpapers,whichhadvisuallyyellowtintandshowedthelowestwhitenessindex(100CIEunits)causedacolordifferenceof6CIEDE2000(1:1:1)unitswiththeothersbyaverage.Theotherpaperswithawhitenessindexbetweenalmost130and155CIEunitsresultedinacolorchangeabout2.5to3CIEDE2000(1:1:1)units.ThetexturepropertiesofthepaperswerequantifiedwithEdgeFrequencytechniqueasanappropriatetextureanalysistool.Thecorrelationbetweentexturechangesandcolordifferenceswasalsoconcluded.Itappearsthattheeffectofwhitenessonthefinalcolorsismorethantheeffectoftexture.However,itseemsthatwhenthewhitenessvaluesreachestoanappropriaterangeabout145-155CIEunitstheeffectofwhitenessisdecreased.

8292-34, Session 9

Color-dependent banding characterization and simulation on natural imagesS.Hu,J.Zhang,PurdueUniv.(UnitedStates);H.Nachlieli,D.Shaked,Hewlett-PackardLabs.IsraelLtd.(Israel);S.Shiffman,IndigoLtd.(Israel);J.P.Allebach,PurdueUniv.(UnitedStates)

Bandingandstreaksareimportantprintdefectsforhigh-enddigitalpresses.Variousmechanismsthatcausebandingandstreaksjointlyproduceauniquedefectappearancefrompagetopage.Thesedefectsconsistofspatialvariationsinluminanceandchrominancechannels,thecharacteristicsofwhichalsodependonthelocalpointwithinthecolorspaceaboutwhichthedefectvariationoccurs.Thispaperfocusesoncolor-dependentcharacterizationandsimulationofbanding.Thisframeworkofbandingcharacterizationisbasedonprintinguniformtestpatchesthatsamplecolorantcombinationsthroughouttheinputcolorspaceofthepress,scanningthesepatches,andextractingrelevantdefectsignals.ThecloudofthedefectsignalsamplepointsforeachmeasuredcolorantcombinationisanalyzedinCIELabusing3-dimensionalPrincipalComponentAnalysis(PCA).Seriesoffeaturesforeachmeasuredcolorantcombinationiscalculated.Thiscompletesthedefectcharacterizationframeworkthatcanhelpsystemdesignersunderstandthecolor-space-dependenceofthedefects.Italsoprovidesthefront-endfortheframeworktosimulatetheappearanceofthedefectsinthepresenceofdocumentcontent.Tocompletethedefectsimulationframework,asetof1-Ddefectspatialprototypesisselectedtorepresentthecommondefectappearance

Conference 8292


producedbyaspecificmachine.AccordingtothedensedefectfeaturesLUTobtainedbyinterpolationofthemeasureddefectfeatures,thedefectspatialprototypeisthenmodulatedinacolor-space-dependentfashionthatvariesspatiallyacrossthedocumentcontent.

8292-35, Session 9

Modeling large-area influence in digital halftoning for electrophotographic printersY.Ju,X.Zhang,D.Saxena,PurdueUniv.(UnitedStates);T.Kashti,D.Kella,IndigoLtd.(Israel);D.Shaked,M.Fischer,Hewlett-PackardLabs.IsraelLtd.(Israel);R.A.Ulichney,Hewlett-PackardCo.(UnitedStates);J.P.Allebach,PurdueUniv.(UnitedStates)

Withelectrophotography,thedepositionoftonerwithintheareaofagivenprinter-addressablepixelisstronglyinfluencedbythehalftonevaluesoftheimmediatelyneighboringpixels.Thislocalinfluencecanbeattributedtofactthatthespotsizeofthelaserwritebeamislargerthanasingleprinteraddressablepixel;thecomplexfieldinteractionsthataresetupbythechargedistributiononthephotoconductorandinthetonerinthegapbetweenthephotoconductorandthedeveloper,andhowthisinfluencesdevelopment;thefurtherspreadingoftonerduringthetransferandfusingprocesses;andopticalscatteringofincidentlightwithinthemedia.Inourpreviouswork,topredictthemeasuredvalueofagivenprinteraddressablepixel,wepopulateda3x3look-up-tablewithdatabasedonmicroanalysisofanexhaustivesetofpossibleprinted3x3binaryhalftonepatternsandsummarizedtheinfluenceoftheouter5x5shellofpixelsbyformingaweightedsumofthesehalftonepixels.

Inthepresentpaper,weexaminethepotentialinfluenceofamuchlargerneighborhoodofthedigitalhalftoneimageonthemeasuredvalueofaprintedpixelatthecenterofthatneighborhood.Thisinfluenceisbelievedtobeduetothescatteringoflightwithinthemedia,ratherthantheotherfactorsmentionedabove.Wefirstdemonstratebyprintingandanalyzingasetoftestpatternsthatindeedthedigitalhalftonewithinaneighborhoodofsizeontheorderof10sofpixelsby10sofpixelswillinfluencethemeasuredvalueofthecenterpixel.Wethenshowthatitispossibletoaccountfortheinfluenceofthislargerneighborhoodwithafunctionofonlytheaveragevalueofthedigitalhalftoneimagewithinthelargerneighborhood.Weshowthattheextendedmodelyieldsasignificantimprovementintheaccuracyofthepredictionofthepixelvaluesoftheprintedandmeasuredhalftoneimage.

8292-36, Session 9

The octagon screen set: a square N-color, high-order, Moiré-free screen setY.Chen,PurdueUniv.(UnitedStates);M.Fischer,Hewlett-PackardLabs.IsraelLtd.(Israel);T.Kashti,IndigoLtd.(Israel);D.Shaked,Hewlett-PackardLabs.IsraelLtd.(Israel);J.P.Allebach,PurdueUniv.(UnitedStates)

Periodicclustered-dotscreensarewidelyusedforelectrophotographicprintersduetotheirprintstability.However,moiréisaubiquitousproblemthatarisesincolorprintingduetothebeatingtogetheroftheclustered-dot,periodichalftonepatternsthatareusedtorepresentdifferentcolorants.Thisbeatingorinterferencephenomenonintroducesspuriouslowfrequency(largeperiod)patternsintheprintedoutputthatareveryobjectionabletotheviewer.Thetraditionalsolutioninthegraphicartsandprintingindustryistorotateidenticalsquarescreenstoanglesthataremaximallyseparatedfromeachother.Forexample,theclassicthree-colorscreensetrotatesthreeidenticalsquarescreenstotheangles15degrees,45degrees,and75degrees,respectively.However,theeffectivenessofthisapproachislimitedwhenprintingwithmorethanfourcolorants,i.e.N-colorprinting,whereN>4.Moreover,accuratelyachievingtheanglesthathavemaximumangularseparationrequiresaveryhighresolutionplatewriter,asisusedincommercialoffsetprinting.

Inthispaper,weproposeasystematicwaytodesigncolorscreensetsforperiodic,clustereddotscreensthatoffersmoreexplicitcontrolofthemoirépropertiesoftheresultingscreenswhenusedincolorprintingapplications.Thisapproachisparticularlywell-suitedtothedesignoftilevectorsetsforprintingwithmorethanfourcolorants,i.e.,N-colorprinting.WeadoptWangandLoce’smethodofusingafrequencydomainbasislattice,butdefineaspecificlatticegeometryandaspecifichalftonefrequencyvectorgeometrythathavepotentialadvantages.Whileachievingmoiré-freecolorprinting,wealsotakethesymmetryoftheresultinghalftonepatternsintoconsideration.Asaresult,inthispaperweproposeascreensetthatconsistsofcompletelysquarescreens.Werefertothisscreensetastheoctagonscreensetsinceinthefrequencydomain,theminimumboundingpolygondefinedbythefundamentalfrequenciesofthecolorantsisanoctagon.Wealsoproposeseveralsymmetrymeasures,andusethemtocomparetheoctagonscreensetandthescreensetsbasedonthepreviousmoiré-freeN-colornon-orthogonalapproach.Theproposedoctagonscreensetisshowntohavebettersymmetryproperties.

8292-37, Session 9

Ink-saving strategy based on document content characterization and halftone texturesM.V.OrtizSegovia,PurdueUniv.(UnitedStates)andOcéPrintLogicTechnologies(France);N.Bonnier,OcéPrintLogicTechnologies(France);J.P.Allebach,PurdueUniv.(UnitedStates)

Commonink-savingtechniquesusuallyrestrictthecolorantconsumptioninadocumentbyreplacingapercentageofthecolorantsbyblackink.Eventhoughtsuchmethodsachieveaconsiderablereductionintheamountofcolorantsusedinapage,thevisualqualityoftheprintoutisaffectedandunpleasingeffectsinpastelsandskintonesareobserved.Ontheotherhand,thequalityoftheprintoutisnotonlyaffectedbytheink-savingalgorithm,butalsobythewaythecolorhalftoningalgorithmarrangesthedotsintheprintout.Therefore,therelationshipbetweenthecontentsofthedocumenttobeprintedandtheprintingprocessneedstobeaddressedbytheink-savingstrategy.Inordertopreserveprintingquality,smoothanduniformhalftonetexturesaresought.Acolordirectbinarysearchhalftoningmethodthatstrivestominimizeboth,theinkusageandtheperceivederrorbetweenthecontinuous-tonecolorimageandthecolorhalftoneimageisproposed.Ourgoalsaretoestimatetheeffectsoftheink-savingmoduleofaprintingworkflowinindividualregionsofthedocument,andtodeterminethedotarrangementandinkcombinationthatconsumestheleastamountofinkwhilepreservingprintingquality.

8292-38, Session 9

Colour print workflow and methods for multi-layering of colour and decorative inks using UV inkjet for fine-art printingC.E.Parraman,P.Laidler,Univ.oftheWestofEngland(UnitedKingdom)

Thecurrentinkjetmarketismainlycomposedoftwodimensionalprinters,paperandvinylprinting,3DCAD,solventwide-formatandUVprintingontopaper,plasticsandboard.Inkjetisubiquitouslyusedasaproofingorprototypingtoolinpreparationforprintproduction,butitisalsobeingusedasthemeanstoprintfinalproductsandartefacts,forexample,printingontowallpaper,hot-airballoonnylonandfurniture.ThishasbeenmadepossiblethroughthemorepermanentandenduringUVcurableinks.

Inordertoincreasedensityofcolourandimproveinkcoveragewhenprintingontoarangeofsurfaces,thispaperwillpresentresearchintoover-layeringofcolourandtheappearanceofcolourat‘n’levelsofinkcoverage.Itwillalsopresentastudyintolayeringofgreysandglossinordertoimprovetheappearanceofprintedimagesontometal.Returningtoouroriginalinvestigationofartist’srequirementswhenmakinginkjetprints(1999-),theseobservationsarebasedonempiricalapproachesthataddresstheneedtopresentphysicaldatathatis

Conference 8292


moreusefulandmeaningfultothedesigner.Thestudyhasusedcolourchartstomeasurecolourtoprovideuserswithaprioriunderstandingatasoft-previewlevelandprintedcolourcirclestodemonstratetheappearanceofprintedcolourondifferentsubstrates.

Testresultsrelatingtotheappearanceofprintondifferentsurfaces,andaseriesofcasestudieswillbepresented.

8292-39, Session 9

Halftone blending between smooth- and detail-screens to improve print quality with electrophotographic printersS.J.Park,PurdueUniv.(UnitedStates);M.Q.Shaw,G.Kerby,T.Nelson,D.Tzeng,V.Loewen,K.Bengtson,Hewlett-PackardCo.(UnitedStates);J.P.Allebach,PurdueUniv.(UnitedStates)

Inthispaper,weconsideradual-modehalftoningprocessthatswitchesbetweenperiodic,clustered-dotscreensoftwodifferentfrequencies-alowfrequencyscreenforsmoothregionsandahighfrequencyscreenfordetailregions.Theseregionsaredescribedbyanobjectmapthatisextractedfromahigh-levelrepresentationofthepagecontenttoberendered.Thishigh-levelrepresentationiscontainedinthepagedescriptionlanguage(PDL)versionofthepagetobeprinted.Ourscreensobeyaharmonicrelationship.Weimplementablendingprocessbasedonatransitionregion.Weproposeanadditiveblendingprocess,andanonlinearblendingprocessinwhichateachpixel,wechoosethemaximumofthetwoweightedhalftones.Theweightsvaryaccordingtothepositioninthetransitionregion,relativetothenormaltotheboundarybetweenthesmoothanddetailregions.Weshowthatthisblendingmethodbetterretainstheintegrityofthetwoconstituenthalftonepatterns,thandoestheadditiveprocess.

Wealsoshowthatprintqualitycanbeimprovedbyvaryingthewidthofthetransitionregion,accordingtothepredictedraggednessoftheboundarybetweenthesmoothanddetailregions.Thisraggednessmeasuredependsontherelationshipbetweentheanglesofthesmoothanddetailscreenswithrespecttotheangleoftheboundarybetweenthesmoothanddetailregions.Italsodependsontherelativegraylevelsoftheimageoneithersideoftheboundary.Theraggednessmeasureiscomputedempiricallyasafunctionoftheseparametersviaanoff-linetrainingprocess.Generallyspeaking,theideaistousethenarrowesttransitionregionthatwillsatisfactorilyreducetheraggednessattheboundary.Usingatransitionregionthatiswiderthanthiscanresultinablurredappearanceattheboundary.Inaddition,wedescribeanon-linetone-mappingprocess,basedonanoff-linecalibrationprocedurethateffectivelyassuresthedesiredtonevalueswithinthetransitionregion,andwhichtherebypreventstheoccurrenceofhaloartifacts.

8292-40, Session 10

Spectral transmittance model for piles of transparencies printed in halftoneJ.Machizaud,M.Hébert,Lab.HubertCurien(France)andUniv.Jean-MonnetSaint-Etienne(France)andCNRS(France)

Thepresentworkinvestigatesthelighttransmissionpropertyoftransparenciesprintedinhalftone.Weproposeaspectraltransmittancemodeldescribingthemultiplereflectionsoflightbetweentheinterfacesofthesuperposedtransparenciesaswellastheattenuationsoflightwithintheplasticbulkandtheinklayers.Inthecaseofhalftonecolors,themodelisabletodeterminetheeffectivesurfacecoveragesofthecolorantsyieldedbythepartialsuperpositionoftheinkdots.Calibrationofthemodelneedsprintingafewcolorpatchesononetransparencyandmeasuringtheirspectraltransmittance.Experimentalverificationcarriedoutwithaninkjetprintershowsverygoodagreementbetweenpredictionandmeasurementinthecaseoftwotransparencies.StochastichalftonesareusedinordertoavoidMoiréeffectsbetweenthesuperposedhalftonescreens.Byinversionofthemodel,weareabletodeterminethecombinationsofhalftonestoprint

onthetransparenciesinordertoobtainbysuperpositiononetargetedcolor.Anoriginalapplicationofthis,called“colormatching”,consistsincreatingapairofcolorimagesprintedonseparatedtransparencieswhichyieldauniformcoloredrectangleoncesuperposed.Thepredictionaccuracyoftheproposedmodelguaranteesthegoodvisualuniformityoftheresultingcoloredarea.

8292-41, Session 10

Optimal estimation of spectral reflectance based on metamerismT.Chou,W.Lin,NationalTaiwanNormalUniv.(Taiwan)

Inthispaper,weproposedanaccurateestimationmethodforspectralreflectanceofobjectscapturedinanimage.ThespectralreflectanceissimplymodeledbyalinearcombinationofthreebasicspectrumsofR,G,andBcolorsrespectively,namedasspectralreflectivebasesofobjects,whichareacquiredbysolvingalinearsystembasedontheprincipleofcolormetamerism.Someexperimentswereperformedtoevaluatetheaccuracyoftheestimatedspectralreflectanceofobjects.Theaveragemeansquareerrorof24colorsinMacbethcheckerbetweenwesimulatedandthemeasuredis0.0866,andthemaximumis0.310.Inaddition,theaveragecolordifferenceofthe24colorsislessthan1.5undertheD65illuminant.Thereare13colorshavingtheircolordifferencevalueslessthan1,andother8colorshavingthevaluesduringtherangeof1and2.Onlythreecolorsarerelativelylarger,withthedifferencesof2.558,4.130and2.569,fromthecolorsofNo.2,No.13,andNo.18inMacbethcheckerrespectively.Furthermore,thecomputationalcostofthisspectralestimationisverylowandsuitableformanypracticalapplicationsinrealtime.

8292-42, Session 10

Hue-shift model for DLP projector with the white peaking functionI.Park,H.Ha,D.Kim,Y.Ha,KyungpookNationalUniv.(Korea,Republicof)

thispapermodelsthishueshiftphenomenonandproposesahuecorrectionmethod.Themodelingofthehue-shiftphenomenonisdeterminedbasedonperceivedhuematchingexperiments.Toquantifythehue-shiftphenomenonforthewholehueangle,24colorpatcheswiththesamelightnessarefirstcreatedandequallyspacedinsidethehueangle.Thesepatchesarethenprojectedone-by-oneontwoscreens.Next,thehuevalueforeachpatchisadjustedbyobserversusingthewhitepeakingfunctionsothattheperceivedhueforthepatchesonthetwoscreensappearsthesamevisually.Afterobtainingthehue-shiftvaluesfromtheperceivedhuematchingexperiment,thesevaluesarethenusedforcorrection.Essentially,theinputredgreenblue(RGB)valuesofanimageareconvertedtoCIELABvaluesusingaforwardcharacterizationmodel,andtheLCh(lightness,chroma,andhue)valuesarethencalculatedtoobtainthehuevaluesforallthepixels.Thesehuevaluesareshiftedaccordingtotheamountcalculatedusingthefunctionsofthehue-shiftmodel.Finally,thecorrectedCIELABvaluesarecalculatedfromthelightness,chroma,andcorrectedhuevalues,thentheoutputRGBvaluesforallthepixelsareestimatedusinganinversecharacterizationmodel.Forevaluation,anobserverpreferencetestwasperformedusingseveraltestimageswithotherhueshiftresults,andthecomparedresultsarepresentedwithazscore.

8292-43, Session 10

Content-dependent noise reduction for mobile displaysG.Kim,Y.Lee,H.Kim,C.Kim,InhaUniv.(Korea,Republicof)

MobileimagingsystemsthatcandisplaythecontentsoftheT-DMBhavebeenintroducedintomarkets.TypicalsizeoftheT-DMBsequencesisQVGA(QuarterVideoGraphicsArray).Inotherwords,

Conference 8292


eachframeofT-DMBsequencesconsistsof320x240pixels.Also,itsframerateis30fps(framespersecond).Thesizeofmobileimagingsystemsarerapidlyincreasing.Recently,LCDdisplayswhosediagonalsizegreaterthan10inchesareemployedforthemobileimagingsystemscapableofdisplayingT-DMBsequences.Thenumberofpixelsonthelargesizedmobiledisplaysisabout10timesgreaterthanthatoftheT-DMBsequences.Thus,theenlargementofmorethan3xinbothwidthandheightdirectionsisrequired.NoisesresultingfromthemovingpicturecompressionemployedfortheT-DMBandtheimageenlargementprocedureaffectthevisualqualityoftheT-DMBsequencesonthelargesizedmobiledisplays.ThispaperpresentsacontentdependentmethodtoreduceundesirablenoisesontheenlargedsequencesofT-DMB.

Conference 8292


Conference 8293: Image Quality and System Performance IXTuesday-Thursday24-26January2012

Part of Proceedings of SPIE Vol. 8293 Image Quality and System Performance IX

8293-01, Session 1

Development of the I3A CPIQ spatial metricsH.Eliasson,SonyEricssonMobileCommunicationsAB(Sweden);D.Baxter,STMicroelectronics(R&D)Ltd.(UnitedKingdom);F.Cao,DxOLabs(France);J.Phillips,EastmanKodakCo.(UnitedStates)

TheI3ACameraPhoneImageQuality(CPIQ)initiativeisaimingatprovidingaconsumer-orientedoverallimagequalitymetricformobilephonecameras.Inordertoachievethisgoal,asetofsubjectivelycorrelatedimagequalitymetricshasbeendeveloped.Thispaperdescribesthedevelopmentofaspecificgroupwithinthissetofmetrics,thespatialmetrics.Containedinthisgrouparetheedgeacutance,visualnoiseandtextureacutancemetrics.Acommonfeatureisthattheyarealldependentonthespatialcontentofthespecificscenebeinganalyzed.Therefore,themeasurementresultsofthemetricsareweightedbyacontrastsensitivityfunction(CSF)and,thus,theconditionsunderwhichaparticularimageisviewedmustbespecified.Thisleadstotheestablishmentofacommonframeworkconsistingofthreecomponentssharedbyallspatialmetrics.First,theRGBimageistransformedtoacoloropponentspace,separatingtheluminancechannelfromtwochrominancechannels.Second,associatedwiththiscolorspacearethreecontrastsensitivityfunctionsforeachindividualopponentchannel.Finally,thespecificviewingconditions,comprisingbothdigitaldisplaysaswellasprintouts,aresupportedthroughtwodistinctMTFs.

8293-18, Session 1

A functional-design approach to lens shading correction issues on mobile camera systemS.Yoo,SAMSUNGElectronicsCo.,Ltd.(Korea,Republicof);T.Kim,SAMSUNGElectronicsCo.,Ltd.(UnitedStates)

Thispaperproposesthelensshadingcorrectionsystemthroughcombinationoffunctionalfactorsbasedonrecentissuesrelatedtomobilecamerasystem.Themainissueisdesigningoftheshadingalgorithm.Thesecondissueiscorrectionofcolortemperature.Thethirdissueiscorrectionbyopticalzoomlens.Thefourthissueisrelatedwithmassproductionofcameramodule.Thefinalissueinmobiledevicesisreducingsizeofcameramodule.PreviouslensshadingcorrectionthatsituatetostablefunctionthroughrichhardwareresourceinDSLRandcompactcameraforrecentlytheseveralyearsisbecomingissuesagainbeingadmittedtopartofsystemofmobilecamerathatresourceislimited.Asdaysgoby,sizereductionofimagesensorandmoduleisessentialduetorequestofmarketplacealongwiththedevelopmentofmobiledevices.Theproposedlensshadingcorrectionsystemprovidesminimizingofaccuracyerrorsinshadingevaluation,aswellassizereductionofmemoryandmoduleinhardware.Finally,theproposedsystemforlensshadingcorrectionsolvesraisedissuesinmobileenvironmentandguidesdesignofalgorithmandhardwareforlensshadingcorrectioninmobilecameramodule.

8293-19, Session 1

Rethinking user interfaces for cameraphonesS.A.Brewster,C.McAdam,Univ.ofGlasgow(UnitedKingdom)

Cameraphonesareverycommonbuthaveusabilityissuesthataffecttheiruseandreducethequalityofimagescaptured.Usersoftenspendlittletimepost-processingphotos,oftenuploadingthemimmediatelytosocialnetworkingsites.ProblemsmayoccurbecauseuserslookthroughtheLCDtoframeshotsandoftenmissiconsdisplayedaroundtheedgesofthescreenthatpresentimportantinformationaboutcamerastatus.

Thismayleadtoshotsbeingmissed,blurredorpoorlyexposed.Mostcameraphonesdonottakefulladvantageofthefeaturesoftheunderlyingphoneplatform(suchasrichsensingandpowerfulaudio)totrytosolvetheseproblems.WecreatedanAndroidcameraphoneapplicationthatfeaturednovelinteractionsandmadeuseofthefeaturesoftheplatformtoprovidearichvarietyofinformationinmoreusableforms,suchas:sonificationoftheluminancehistogramtoensurebetterexposurebeforeapicturewastaken;measuringphonemovementtoensurethephonewasbeingheldsteady;andthedetectionofmotionintheimagetosupportpanning.Weevaluatedtheseinteractionswithusersinrealsettingsandshowedtheycouldsolvesomeofthemostcommoncameraphoneproblemsandimprovethepicturetakingprocess.

8293-02, Session 2

Calibration and adaptation of ISO visual noise for camera phone image quality assessmentD.J.Baxter,A.Murray,STMicroelectronics(R&D)Ltd.(UnitedKingdom)

TheI3ACameraPhoneImageQuality(CPIQ)visualnoisemetricdescribedisacoreimagequalityattributeofthewiderI3ACPIQconsumerorientated,cameraimagequalityscore.Thispaperdescribesthemotivationbehindthechoiceofavisualnoisemetric,whythecharacteristicsofcellphonecamerasnecessitatetheadaptationoftheISO15739visualnoiseprotocolandthecalibrationoftheadaptedvisualnoisemetricintermsofQualityLossJNDs.VisualnoisemetricssuchasISO15739visualnoise,S-CIELABandvSNRareshowntohavetheimportantpropertyofbeingabletodiscriminatebetweendifferentNPSshapes.Theopticalnon-uniformitiesprevalentincellphonecamerasandhighernoiselevelsposesignificantchallengestotheISO15739visualnoiseprotocol.

Thenon-uniformitiesareaddressedusingafrequencybasedhighpassfilter.Secondly,thedataclippingathighnoiselevelsisavoidedbyuseoftheJohnsonandFairchildfrequencybasedSCIELAB

LuminanceCSF.ThefinalresultisavisuallybasednoisemetriccalibratedintermsofQualityLossJNDsusingAptinaImaging’ssubjectivelycalibratedimageset.

8293-03, Session 2

An objective method for evaluating the texture-preserving capability of digital camcordersK.Zhu,S.Li,D.Saupe,Univ.Konstanz(Germany)

Thispaperpresentsamethodforevaluatingperformanceofcamcordersintermsoftexturepreservation,takinghumanvisualsystem(HVS)intoaccount.AmetricwecallPSDM(powerspectraldensitymetric)istheoutcomeofthemethodasaquantitativeindicator.TherelationshipsbetweenthePSDMvalueandthemotionspeedandthecompressionbitratewerestudies.Thedeadleaveschartwasusedinourexperimentstosimulateascenewithtexturesofdifferentscales.Thedeadleaveschartsisknownasagoodtargetinvarianttoscale,shift,rotationandcontrast(exposure)anditsradialPSDfollowsapowerlaw.Experimentalresultsonfivecamcordersfromthreedifferentvendorshaveshownthat1)thePSDMvaluehasamonotonicrelationshipwiththemotionspeedforalltestedcamcorders;2)thePSDMvaluehasamonotonicrelationshipwiththecompressionbitratesforthreetestedcamcordersbutnotfortheothertwo.We


attributethemismatchbetweenthePSDMvalueandthecompressionbitrateforsomecamcorderstotheintrinsicnonlinearityofmanycomponentsinthewholevideocapturing/recordingprocess.OurstudyhasconfirmedPSDMasausefulindicatorformeasuringacamcorder’sperformanceintermsofpreservingtextures.

8293-04, Session 2

Improving texture loss measurement: spatial frequency response based on a colored targetU.Artmann,D.Wueller,ImageEngineeringGmbH&Co.KG(Germany)

Thepixelraceinthedigitalcameraindustryandformobilephoneimagingmoduleshavemadenoisereductiontoasignificantpartinthesignalprocessing.Dependingontheusedalgorithmsandtheunderlyingamountofnoisethathastoberemoved,noisereductionleadstoalossoflowcontrastfinedetails,alsoknowastextureloss.Thedescriptionoftheseeffectsbecameanimportantpartoftheobjectiveimagequalityevaluationinthelastyears,astheestablishedmethodsfornoiseandresolutionmeasurementfailtodoso.Differentmethodshavebeendevelopedandpresentedinthelastyears,butcouldnotfullysatisfytherequestedstabilityandcorrelationwithsubjectivetests.Inourpaper,wepresentourexperiencewiththecurrentapproachesfortexturelossmeasurement.Wehavefoundacriticalissuewithinthesemethods:theusedtargetsarecolorneutral.Wecouldshowthatthetest-labresultdonotmatchtherealliveexperiencewiththecamerasundertest.Wepresentanapproachusingacoloredtargetandourexperiencewiththismethod,basedonahugevarietyofdigitalcameras.

8293-20, Session 2

On the performances of computer vision algorithms on mobile platformsS.Battiato,G.M.Farinella,E.Messina,G.Puglisi,D.Ravì,Univ.degliStudidiCatania(Italy);A.Capra,V.Tomaselli,STMicroelectronics(Italy)

Inrecentyearsthereisagrowinginterestinnewtechnologytobeemployedinthecontextofmobiledevices.Despitetoday’smobiledevices(e.g.,smartphone,tablet,etc.)arestilllimitedintermsofresources(e.g.,processorspeed,availableRAM,etc.),novelComputationalPhotographysolutionsareavailabletobuildappealingimagingapplicationsthatcannotbeperformedbefore.Themainideaistoovercomethelimitationoftraditionalimagingdevicesbyusingcomputationalmethodswhichcanexploitthedifferentinputsofferedbyamobiledevices(e.g.,fromlowleveldata,suchasBayernpattern,GPSposition,etc.).Sincedifferentcamerasareusuallyembeddedindevicesofnewgeneration,computervisionalgorithmswillbeextremelyusefulinmanyapplicationsofnextfuture.Forexample,visualtrackingcanbeexploitedtointeractwithvideogames,ortherecognitionofthevisualcontentcouldhelpinbuildingnewapplicationsinthecontextofculturalheritage(e.g.,givingbackinformationonarecognizedarcheologicalsite).

Themaincontributionofthisworkisrelatedtotheportingandtestingofsomeclassiccomputervisionalgorithmsonmobileplatforms.Specifically,afewalgorithmscoveringthemaintasksofComputerVisionhavebeenconsidered:keypointextraction,facedetection,imagesegmentation.Theportinghasbeenperformedconsideringthefollowingoperatingsystems:Maemo,typicallyusedinNokiaN900,andAndroidemployedinLGOptimusone,SamsungGalaxySII.Theseoperatingsystemshavebeenconsideredbecausetheycanbeeasilyextendedwithcustomizedlibrariesand/orprogramsandprovideastandardizedandfairlywidespreadAPI(ApplicationProgramInterface).

Itisworthnotingthattheaforementionedalgorithmsshouldbeoptimizedtoproperlyworkonlowresourcesdevices.Forinstance,theFCAMlibraryavailableforN900Nokiasmartphone,allowstointeractwiththelowlevelalgorithms(e.g.,demosaicing,whitebalancing,

denoising,etc.)anddata(Bayerpattern)involvedintheimagingpipeline.Inthiswayabetterdesignofcomputervisionalgorithmsforconstrainedresourcesdevicescanbedone.

Finally,comparativetests,conductedonstandarddatasets,quantitativelyandqualitativelyevaluatetheperformanceofthealgorithmsonmobiledevices.

8293-05, Session 3

The uncertainty of scanner illumination IIL.C.Cui,LexmarkInternational,Inc.(UnitedStates)

Inapreliminaryreport,weshowedtheimpactoftheintegratingcavityeffectforatypicaldocumentscannerwithopticalraytracing.Theeffectwasdemonstratedbyexaminingtheilluminationprofileafteraccountingformultiplereflectionsfromthedocumentsurface,thecontactplatenglasssurfacesandallreflectorsusedintheilluminationassembly.Weidentifiedthattheplatenglasscancontributejustasmuchastheilluminationassemblytotheeffect.Inthesecondphase,webuiltanactualscannermodeltoverifytheraytracingresultsandtheeffect.Theverificationwasaccomplishedbyexaminingtheedgeprofiledifferencesofthescanimagesofuniquepatternsbeforeandafteracertainreflectioncomponentwasremovedwithtwodifferentscanconfigurations.Theexperimentalresultsareconsistentwiththesimulationresultsingeneral.

8293-06, Session 3

Measuring the modulation transfer function of image capture devices: what do the numbers really mean?X.Zhang,Y.Ju,PurdueUniv.(UnitedStates);T.Kashti,D.Kella,T.Frank,IndigoLtd.(Israel);D.Shaked,Hewlett-PackardLabs.IsraelLtd.(Israel);R.A.Ulichney,Hewlett-PackardCo.(UnitedStates);M.Fischer,Hewlett-PackardLabs.IsraelLtd.(Israel);J.P.Allebach,PurdueUniv.(UnitedStates)

Themodulationtransferfunction(MTF)isafundamentaltoolforassessingtheperformanceofimagingsystems.Ithasbeenappliedtoarangeofcaptureandoutputdevices,includingprintersandeventhemediaitself.Inthispaper,weconsidertheproblemofmeasuringtheMTFofimagecapturedevices.WeanalyzethefactorsthatlimittheMTFofacapturedevice.Then,weexaminethreedifferentapproachestothistaskbased,respectively,onaslant-edgetarget,asinewavetarget,andagrillpattern.Wereviewthemathematicalrelationshipbetweenthethreedifferentmethodswithandwithoutnoiseinthesystem,anddiscusstheircomparativeadvantagesanddisadvantages.WealsoconsidertheimpactoflinearizingthesensorresponseontheMTFmeasurement.Finally,wepresentexperimentalresultsforMTFmeasurementwithanumberofdifferentcommerciallyavailableimagecapturedevicesthatarespecificallydesignedforcaptureof2Dreflectionortransmissioncopy.Theseincludecamera-basedsystems,flat-bedscanners,andadrumscanner.

8293-07, Session 3

A fast, automatic camera image stabilization benchmarking schemeJ.Yu,S.A.Craver,BinghamtonUniv.(UnitedStates)

Whileimagestabilization(IS)hasbecomeadefaultfunctionalityformostdigitalcameras,thereisalackofautomaticISevaluationscheme,i.e.mostpubliclyknowncameraISreviewsstillrequirehumanvisualassessment,whichisslowandinconsistent.Forthefirsttime,weproposedahistogrambasedautomaticISevaluationscheme,whichemploysawhitenoisepatternasshootingtarget.ItisabletoproduceaccurateandconsistentISbenchmarksinaveryfastmanner.

Conference 8293


8293-08, Session 3

A comparison of signal-to-noise ratio (SNR) of linear CCD sensors for optical payloadM.Waqas,PakistanSpaceandUpperAtmosphericResearchCommission(SUPARCO)(Pakistan)

Withthegrowingneedandimportanceofremotesensingdata,moreandmorecountriesareinitiatingspacemissionstofulfilltheirneedsandtobenefitfromthisevolvingtechnology.Inthisregard,differentcountriesworldwideingeneralandAsiancountriesinparticular,haveinitiatedhighresolutionremotesensingsatelliteprojects.ThemainpayloadforthesesatellitesisbasedonhighresolutionCCDorTDIsensor.Sincemanufacturingasensoraltogetherisnottechnicallyandcommerciallyfeasible,therefore,wehavesurveyedvariouscommerciallyavailablesensorsinordertoselectasensorthatfulfillstherequirementsofahighresolutionremotesensingmission.Intheselectionofcommerciallyavailablesensor,thesensorspecificationsbecomeimportant.Thereareavarietyofdifferentsensorscommerciallyavailablethatcoverabroadspectrumofremotesensingapplications.Inordertochooseasensorperfecttouserrequirements,thispaperexploressomeoftheimportantsensorspecificationsthatdrivetheselectionofasensorforahighresolutionimagingpayload;theseinclude:SpectralRange,QuantumEfficiency,IntegrationTime,PixelSize,DarkCurrent,DynamicRange,SNRetc.ThispaperpresentsacomparisonofSNRvaluesobtainedfromdifferentCOTS(Commercialofftheshelf)basedCCDSensorsforanearthimagingpayload.

8293-09, Session 4

A sharpness measure on automatically selected edge segmentsF.Gasparini,F.Marini,S.Corchs,S.Raimondo,Univ.degliStudidiMilano-Bicocca(Italy)

Weaddresstheproblemofimagequalityassessmentfornaturalimages,focusingonNoReference(NR)metricsforsharpness.

Amongtheavailablemethodsfoundintheliterature,afterdetectingtheedgepixels,thesharpnessmeasureisdefinedforeachedgepixel.Thefinalmetricvalueisobtainedaveragingallthesevalues.However,wehaveobservedthatinsomecasesthisglobalmeasureisnotrepresentativeoftherealsharpnessoftheimages.Thisfactismainlyduetotheaveragingprocessthatisappliedoveralltheedgepixels.Iftheimagecontainsdifferentlevelsofdepthfield,thisaverageoperationcanoverestimatetheedgespread.

Anotherimportantissueinmeasuringsharpnessistoobtainvaluesstablewithrespecttonoise.

Inthiswork,wepresentanautomaticmethodthatselectsedgesegments,andpermitstoevaluateimagesharpnessonmorereliabledata.Moreoverweproposeanovelsharpnessmetricfornaturalimages,inspiredbytheslantededgemeasureadoptedbytheImatestincaseofsyntheticimages.Thismetricmakesitpossibletocopewithnoiseinfluenceprovidingmorereliableestimations.Inthefullpaperversionwewillprovideexamplesandexperimentalresultsthatwillconfirmthefeasibilityofourapproach.

8293-10, Session 4

Selecting the proper window for SSIMS.McFadden,Univ.ofWaterloo(Canada)andChristieDigitalSystemsCanada,Inc(Canada);P.A.S.Ward,Univ.ofWaterloo(Canada)

Thestructuralsimilarityindex(SSIM)isapopularfull-referencemetricinthefieldofimagequalityassessment(IQA).Inspiteofitscommonusage,noanalysishasbeenperformedontheeffectsofmodifyingthewindowusedinmeasuringandcomparingtheluminance,contrast,andstructuralcharacteristicsofthetwoimagesundertest.

NewexperimentsinvolvingthemodificationsofSSIMwindow

characteristicsdemonstrateasignificantimpactonmetricresults.Whenproperlyselected,acustomizedSSIMwindowcanproducesuperiorcorrelationwithsubjectiveresultsandyieldsignificantgainsincomputationalefficiencyrelativetothedefaultSSIMwindow.

WindowcharacteristicsrelevanttoobtainingoptimalSSIMresultsareidentifiedandtheeffectsoftheirmodificationdemonstrated.SSIMresultsareobtainedusingsixpubliclyavailablesubject-ratedimagequalitydatabases,comprisingatotalofover3000imageswithvarioustypesandlevelsofdistortion.FivecommonlyusedevaluationmetricsarecomputedtodemonstratetheimprovementoftheSSIMalgorithm.Analysisoftheseevaluationmetricsispresentedtoprovideinsightintothestatisticalsignificanceofthewindowmodificationresults,withadditionaldiscussionofthecomputationalefficiencyimprovements.Inadditiontothefull-databaseoutcomepresented,resultsaresub-dividedandanalyzedaccordingtotheirspecificdistortiontypes.

8293-11, Session 4

Measurement of texture loss for JPEG 2000 compressionP.D.Burns,CarestreamHealth,Inc.(UnitedStates);D.Williams,ImageScienceAssociates(UnitedStates)

Thecaptureandretentionofimagedetailisanimportantcharacteristicforsystemdesignandsubsystemselection.Anestablishedimagingperformancemetricthatiswellsuitedtocertainsourcesofdetailloss,suchasopticalfocusandmotionblur,istheModulationTransferFunction(MTF).Asperformancestandardshavedevelopedfordigitalimagingsystems,theMTFconcepthasbeenadaptedandappliedasthespatialfrequencyresponse(SFR).MeasurementoftheSFRisgenerallydoneusingparticulartesttargetfeaturessuchasedges,repeatingpattersofsquareorsignwaves.

Theuseofspecialimagefeaturestoderivequalitymeasuresischallengedwhentheeffectivesystemcharacteristicsvarywithlocalimage(scene)content.Thishasleadtothedevelopmentofimagequalitymethodsthatrelyoncomputedtestimagecontentthatissomewaysresemblednaturalscenes.Anexampleofthisisthemeasureoftexture(imagedetail)lossusingsetsofoverlappingsmallobjects;circlesorrectangles.

Inthispaperweinvestigatetheapplicationoftheabovemethodstoanothercommonadaptiveimageprocessingoperation-imagecompression.Thederivationofthetexture-lossfunctionbasedontheimagenoise-powerspectrumisdescribed.WethenapplyseveralformsofbothJPEGandJPEG2000compressiontosetsofdigitalimagesthatincludescenecontentthatisamenabletotheabovestandardimagequalitymeasures.Generalconclusionsaredrawnforapplicationoftheproposedimagequalitymeasurestodigitalimagecompression,asarerecommendationsforstableestimationoftheparameters.

8293-12, Session 4

A no-reference image quality metric for blur and ringing distortions based on weighting processA.Chetouani,A.Beghdadi,Univ.Paris-Nord(France)

ANoReferenceImageQualityMetrics(NR-IQM)proposedintheliteraturearegenerallydevelopedforaspecificdegradation,whichlimitshighlyitsapplication.Toovercomethislimitation,weproposeinthisstudyaNR-IQMforringingandblurdistortionsbasedonaweightingprocess.Foragivenimage,wefirstestimatethequantityofblurandringingdegradationscontainedintheimageusinganArtificialNeuralNetworks(ANN)modelthroughalearningstep.Then,thefinalindexqualityisgivenbycombiningablurandaringingmetricsaccordingtotheobtainedweights.Theperformanceoftheproposedschemeisevaluatedfirstbytestingtheweightingprocessthroughdifferentspecificimages.ThentheproposedmethodisevaluatedintermsofgoodcorrelationwiththesubjectivejudgmentsusingtheLIVEimagedatabase.

Conference 8293


8293-36, Session 4

A new method to identify and quantify image distortion based on Gabor filter bank and multiple regression analysisB.OrtizJaramillo,Univ.NacionaldeColombia(Colombia)andUniv.Gent(Belgium);J.C.GarciaAlvarez,Univ.NacionaldeColombia(Colombia);H.Führ,RWTHAachen(Germany);S.A.OrjuelaVargas,Univ.Gent(Belgium);G.CastellanosDominguez,Univ.NacionaldeColombia(Colombia);W.Philips,Univ.Gent(Belgium)

Recently,wehavebeenexploringthepossibilitytodoimagequalityassessmentbyusingmulti-resolutionanalysis.Thereby,wehaveproposedanautomaticsystemforfullreferenceimagequalityassessmentbasedontheHumanVisualSystemmodelbyusingmulti-resolutionanalysis.Inthissense,previousproposedmethodologiesshowsgoodperformanceinobjectiveimagequalityassessmentwithgoodnessoffitbetweensubjectiveandobjectivesscoresabout92percent.

Nevertheless,currentmethodologiesaredesignedassumingthatthenatureofthedistortionisknown.But,thisisalimitingassumptionforpracticalapplications,sinceinamajorityofcasesthedistortionsintheimageareunknown.

Inthissense,webelievethatcurrentstudiesinvolvingimagequalityassessmentshouldbeaddressedtoidentifyandquantifythedistortionofimagesatthesametime.Becauseofperformbothstudiesatthesametimecouldbeimproveprocesseslikeenhancement,restoration,compression,transmission,amongothers.

WepresentanapproachbasedonthepoweroftheexperimentaldesigntoexploreeffectsbetweencomponentsandthejointlocalizationofGaborfilterstosplitfrequencycomponentstoidentifyandquantifydistortioninimages.Withthiscombinationweprovideaccuratescoresandanappropriatedifferentiabilitybetweendistortionnatures.


Optimal patch code design via device characterizationW.Wu,E.N.Dalal,XeroxCorp.(UnitedStates)

Inmanycolormeasurementapplications,suchasthoseforcolorcalibrationandprofiling,“patchcode”[1]hasbeenusedsuccessfullyforjobidentificationandautomationtoreduceoperatorerrors.Apatchcodeissimilartoabarcode,butisintendedprimarilyforuseinmeasurementdevicesthatcannotreadbarcodesduetolimitedspatialresolution,suchasspectrophotometers.Thereisaninherenttradeoffbetweendecodingrobustnessandthenumberofcodelevelsavailableforencoding.Previousmethods[2,3]haveattemptedtoaddressthistradeoff,butthosesolutionshavebeensub-optimal.Inthispaper,weproposeamethodtodesignoptimalpatchcodesviadevicecharacterization.Thetradeoffbetweendecodingrobustnessandthenumberofavailablecodelevelsisoptimizedintermsofprintingandmeasurementefforts,anddecodingrobustnessagainstnoisesfromtheprintingandmeasurementdevices.Effortisdrasticallyreducedrelativetopreviousmethodsbecauseprint-and-measureisminimizedthroughmodelingandtheuseofexistingprinterprofiles.DecodingrobustnessisimprovedbydistributingthecodelevelsinCIELabspaceratherthanincmykspace.


Influence of viewing device and soundtrack in HDTV on subjective video qualityA.Redl,C.Keimel,K.Diepold,TechnischeUniv.München(Germany)

InmanyhomestodaybigTVscreensandhi-fisystemsarecommon.Butistheperceptionofsubjectivevideoqualityunderprofessionaltestconditionsthesameasinhomeuse?

Forthistwothingsareexamined:Howlargeistheinfluenceofthe

presentingdevicebutalsotheinfluenceofthesoundtrack,bothinHDTV(1080p50).

Previousworksshowedthatthisdifferenceisnoticeable,buttherewerenostudieswithconsumerdevices.Itwasalsoshown,thatthereisaninfluenceofthesoundtrack,butonlyinSDorlowerresolutions.

Thereforeweconductedsubjectivevideotests:Onetestwithdifferentpresentingdevices,a23-inch-referencemonitor,ahighquality56-inch-LCD-TVandanHD-projector,andonetestinwhichwepresentedadditionaltotheHD-projectorasoundtrackona7.1-channelhi-fisystem.

Theresultsshowtwothings:Thetestsubjectshadagreaterqualityofexperiencewiththeconsumerdevicesthanwiththereferencemonitor,althoughthevideoqualityofthereferencemonitoritselfwasratedbetterinanadditionalquestionnaireandthemeanopinionscore(MOS).ThesecondresultwasthatthereisnosignificantdifferenceintheMOSbetweenshowingthevideosonthebeamerwithorwithoutsound.


Influence of viewing experience and stabilization phase in subjective video testingC.Keimel,A.Redl,K.Diepold,TechnischeUniv.München(Germany)

Inthiscontribution,wewillexaminetwoimportantaspectsofsubjectivevideoqualityassessmentandtheiroverallinfluenceonthetestresultsindetail:theparticipants’viewingexperienceandthequalityrangeinthestabilizationphase.Firstly,weexaminedifthepreviousviewingexperienceofparticipantsinsubjectivetestsinfluencetheresults.Weperformedanumberofsingle-anddouble-stimulustestsassessingthevisualqualityofvideomaterialcompressedwithbothH.264/AVCandMPEG2notonlyatdifferentqualitylevelsandcontent,butalsoindifferentvideoformatsfrom576iupto1080p.Duringthesetests,wecollectedadditionalstatisticaldataonthetestparticipants.Overall,wewereabletocollectdatafromover100differentsubjectsandanalysetheinfluenceofthesubjects’viewingexperienceontheresultsofthetests.Secondly,weexaminedifthevisualqualityrangepresentedinthestabilizationphaseofasubjectivetesthassignificantinfluenceonthetestresults.Duetotimeconstraints,itissometimesnecessarytosplitatestintomultiplesessionsrepresentingsubsetsoftheoverallqualityrange.Consequently,weexaminetheinfluenceofthequalityrangepresentedinthestabilizationphaseontheoverallresults,dependingonthequalitysubsetsincludedinthestabilizationphase.


Perceptual visual image sharpness metric for image-based imager stabilizationF.F.G.Gavant,L.Alacoque,A.Dupret,T.Ho-Phuoc,D.David,CEA-LETI(France)

Imagesensorsstabilizationisusuallybasedonaccelerometers.Toreducethenumberofexternalcomponentsofdigitalimagesensors,anintegratedimagebasedimagestabilizationsystemisenvisaged.Suchasystemrequiresmodelingtheblurduetohandtremorandageneralsharpnessmetrictoquantifythegainofsuchastabilizationsystem.Thispaperpresentsanaccuratemodelofthehandtremor,thenitsimpactasaPointSpreadFunction.Inordertodefinethespecificationoftheimagebasedimagestabilizationwehavederivedperceptualvisualqualitysharpnessmetricforcamerashakeblur.Thissharpnessmetricisbasedonvisualblurtest.Itfittingwellbothdifferentapproachessuchasmeanopinionscoredatabaseandqualityrulermeasureofblur.

Conference 8293



A unified method for comparison of algorithms of saliency extractionT.Ho-Phuoc,L.Alacoque,A.Dupret,CEA-LETI(France);A.Guérin-Dugué,Gipsa-lab(France);A.Verdant,CEA-LETI(France)

Extractingsalientregionsofastillimage,whicharepertinentareaslikelytoattractsubjects’fixations,canbeusefultoadaptcompressionlossaccordingtohumanattention.Intheliterature,variousalgorithmshavebeenproposedforsaliencyextraction,rangingfromregion-of-interest(ROI)orpoint-of-interest(POI)algorithmstosaliencymodels,whichalsoextractROIs.Implementingsuchanalgorithmwithinimagesensorsimpliestoevaluateitscomplexityandperformanceoffixationprediction.However,therehavebeennopertinentcriteriatocomparethesealgorithmsinpredictinghumanfixationsduetothedifferentnaturebetweenROIsandPOIs.Inthispaper,weproposeanovelcriterionwhichisabletocomparethepredictionperformanceofROIandPOIalgorithms.Aimingattheelectronicimplementationofsuchanalgorithm,theproposedcriterionisbasedonblocks,whichisconsistentwithprocessingwithinimagesensors.Italsotakesintoaccountsalientsurface,animportantfactorinelectronicimplementation,toreflectmoreaccuratelythepredictionperformanceofalgorithms.ThecriterionisthenusedforcomparisoninabenchmarkofseveralsaliencymodelsandROI/POIalgorithms.Theresultsshowthatasaliencymodel,whichhashighercomputationalcomplexity,givesbetterperformancethanotherROI/POIalgorithms.

8293-13, Session 5

Comparative performance analysis of two picture adjustment methods: HSV versus YCbCrR.Safaee-Rad,M.Aleksic,QualcommInc.(Canada)

Pictureadjustmentsisreferredtothoseadjustemntsthataffectthefourmainsubjectiveperceptualimageattributes:Hue,Saturation,Brightness(sometimescalledIntensity)andContrast.ThecommonmethodusedforthistypeofadjustmentsinadisplayprocessingpipeisbasedonYCbCrcolorspace.ThisisexpectedsinceusuallytheincomingcontentisalreadyinYCbCrcolorspace(TVcolorspace)andthusHSICadjustmentsrequireslessamountofcomputation.

Pictureadjustmentsusingthismethod,however,leadstomultipleproblems.

Asanalternative,HSVcolorspace(asdefinedinSIGRAPH1978)forHSICpictureadjustmentsisusedwhichleadstomultipleadvantages.

Thispaperpresentsacomparative(pictureadjustments)performanceanalysisbasedonthesetwomethods.

8293-14, Session 5

Evaluation of preferred lightness rescaling methods for color reproductionY.Chang,P.J.Green,LondonCollegeofCommunication(UnitedKingdom)

Basedonthepairedcomparisonexperimentresult,theAdobeBPCmethodworkswellingeneralwiththeimages.Havingverylittleimagedependenceonthepairedcomparisonresults,therewasverylittlecorrelationbetweenpsychopysicalresultsandthestatisticalimagecharacteristicsanalyzed.Thissuggeststhatimagequalitymetricsmayneedtobebasedonadifferentsetofimagecharacteristicsthantheonesusedinthisstudy.

Whenthemedianlightnessvaluesarelowerthan2.2L*andhigherthan67L*,linearlightnessrescalingmethodispreferred,whilethemedianlightnessvaluesarelowerthan5.2L*andhigherthan62.6L*,splinelightnessrescalingmethodispreferredovertheAdobeBPC.Asthesampleimagesetwasprimarilylandscapeandhighcontrast,furtherworksneedtobeperformedwithlargerimageset.Also,rather

thanwithimageproperties,studiesonthecorrelationbetweenimagecontentsandpairedcomparisonresultsneedtobestudiedinfuture.

8293-15, Session 5

Investigations of the display white point on the perceived image qualityJ.Jiang,F.Abed,J.Voelkel,RochesterInstituteofTechnology(UnitedStates)

Basedontheresultfromarecentweb-basedexperimentconductedtoevaluatetheperceivedimagequalitywithoutthereferencesponsoredbyMellonFoundation,aprojecttounderstandtheimpactofthewhitepointsettingofthedisplayonthepreferencejudgmentsofperceivedimagequalitywasdesignedandrun,inwhichtwocopiesofeachimageweregeneratedtoappeareitherbluishoryellowish.Aspartoftheproject,observerswereaskedtoratethetwocopiesalongwiththeoriginalversionofthetestimageonadisplayatdifferentwhitepointsettings.Toincludebothtypicalandextremewhitepointofthedisplay,D50andD65wereselectedbesidesthetwoendsofthewhitepointadjustableonaMaccomputer.Thegoalwastodeterminewhetherthechangeinthecolorofthedisplaywouldcauseanyeffectontheperceivedimagequalitypreferredbyobservers.IntheevaluationoftheAnalysisofVariance(ANOVA)results,thewhitepointfactorwasfoundtobeinsignificantforthetestimagesintheexperiment.Therefore,theperceivedimagequalitybasedonpreferencewasunlikelytobebiasedbythecolorofthedisplayatleastforthetestimages.TheresultwasinagreementwithwhatwasfoundintheMellonproject.

8293-16, Session 6

The mobile image quality survey gameD.R.Rasmussen,QiAnalyticsLLC(UnitedStates)

Inthispaperwediscusshumanassessmentofthequalityofphotographicstillimages,thataredegradedinvariousmannersrelativetoanoriginal,forexampleduetocompressionornoise.Inparticular,weexamineandpresentresultsfromatechniquewhereobserversviewimagesonamobiledevice,performpairwisecomparisons,identifydefectsintheimages,andinteractwiththedisplaytoindicatethelocationofthedefects.Thetechniquemeasurestheresponsetimeandaccuracyoftheresponses.Byposingthesurveyinaformsimilartoagame,providingperformancefeedbacktotheobserver,thetechniqueattemptstoincreasetheengagementoftheobservers,andtoavoidexhaustingobservers,afactorthatisoftenaproblemforsubjectivesurveys.Theresultsarecomparedwiththeknownphysicalmagnitudesofthedefectsandwithresultsfromsimilarweb-basedsurveys.Thestrengthsandweaknessesofthetechniquearediscussed.Possibleextensionsofthetechniquetovideoqualityassessmentarealsodiscussed.

8293-17, Session 6

Evaluation of perceived image sharpness with changes in the displayed image sizeJ.Park,S.Triantaphillidou,R.E.Jacobson,Univ.ofWestminster(UnitedKingdom)

Inthispaperanevaluationofthedegreeofchangeintheperceivedimagesharpnesswithchangesindisplayedimagesizewascarriedout.Thiswasachievedbycollectingdatafromthreepsychophysicalinvestigationsthatusedtechniquestomatchtheperceivedsharpnessofdisplayedimagesofthreedifferentsizes.ThepaperfirstdescribesamethodemployedtocreateaseriesoffrequencydomainGaussianfiltersforsharpeningandblurring.Thefiltersweredesignedtoachieveonejust-noticeable-difference(JND)insharpnessbetweenimagesviewedfromacertaindistanceandhavingacertaindisplayedimagesize(angleofsubtense).Duringpsychophysicalexperiments,thefilteredimageswereusedasatestseriesforsharpnessmatching.Fortest-imagecapture,adigitalSLRcamerawithaqualityzoomlenswas

Conference 8293


usedforrecordingnaturalsceneswithvaryingscenecontentundervariousilluminationconditions.Forthepsychophysicalinvestigation,atotalofsixty-fouroriginaltest-imageswereselectedandresized,usingbi-cubicinterpolation,tothreedifferentsizes.Resultsshowedthatdegreeofchangeinsharpnessbetweenimagesofdifferentsizesvariedconsiderablywithscenecontent.Thus,amethod,basedonthequantificationofbusyness,tocategorizethescenecontentoftheoriginaltestimageswasimplemented.

8293-18, Session 6

Towards a perceptual metric for computer-generated imagesP.Boulenguez,B.Airieau,M.Larabi,D.Meneveaux,Univ.dePoitiers(France)

AlthoughasignificantresearchefforthasbeendedicatedtothephysicalvalidationofComputer-GeneratedImages(CGIs),fewstudiesaddressedtheproblemfromapsycho-visualperspective.ThispaperengagesintheconstructionofsuchaperceptualmetricforCGIs.Tothatpurpose,anoriginalpsycho-visualexperimenthasbeendesignedandsubmittedtoarepresentativepanelofobservers.EachparticipantwasaskedtoscoretheoverallperceivedqualityandfiveperceptualcriteriaonCGIsrenderedwithsevendifferentalgorithms.Anadhocanalyticalqualitymodel,fittothestatisticaldata,thengivesinsightintotheinfluenceofeachcriterionontheoverallperceivedquality.Resultstendtoshowthataccuracyinthesimulationofshadows,goodcontrastandabsenceofnoisehaveamajorimpactontheoverallperceivedquality,ratherthanpreciseanti-aliasingorfaithfullcolorbleeding.

8293-19, Session 6

Assessing product image quality for online shoppingA.Goswami,S.H.Chung,N.Chittar,A.Islam,eBayInc.(UnitedStates)

Assessingproduct-imagequalityisimportantinthecontextofonlineshopping.Ahighqualityimagethatconveysmoreinformationaboutaproductcanboostthebuyer’sconfidenceandcangetmoreattention.However,thenotionofimagequalityforproduct-imagesisnotthesameasthatinotherdomains.Theperceptionofqualityofproduct-imagesdependsnotonlyonvariousphotographicqualityfeaturesbutalsoonvarioushighlevelfeaturessuchasclarityoftheforegroundorgoodnessofthebackgroundetc.Inthispaper,wedefineanotionofproduct-imagequalitybasedonvarioussuchfeatures.WeconductacrowedsourcedexperimenttocollectuserjudgmentsonthousandsofeBay’simages.Weformulateamulti-classclassificationproblemformodelingimagequalitybyclassifyingimagesintogood,fairandpoorqualitybasedontheguidedperceptualnotionsfromthejudges.Wethencomputeapseudo-regressionscorewithexpectedaverageofpredictedclasses.Wedesignmanyexperimentswithvarioussamplingandvotingschemeswithcrowdsourceddataandconstructvariousexperimentalimagequalitymodels.Mostofourmodelshavereasonableaccuracies(greaterorequalto70%)ontestdataset.Weobservethatourcomputedimagequalityscorehasahigh(0.66)rankcorrelationwithaveragevotesfromthecrowdsourcedhumanjudgments.

8293-20, Session 7A

How do we watch images?: a case of change detection and quality estimationJ.Radun,T.Leisti,T.Virtanen,G.Nyman,Univ.ofHelsinki(Finland)

Themostcommontasksinsubjectiveimageestimationarechangedetection(adetectiontask)andimagequalityestimation(apreferencetask).Weexaminedhowthetaskinfluencesthegazebehaviorwhencomparingdetectionandpreferencetasks.Theeyemovementsof

16naïveobserverswererecordedwith8observersinbothtasks.Thesettingwasaflickerparadigm,wheretheobserversseeanon-manipulatedimage,amanipulatedversionoftheimageandagainthenon-manipulatedimageandestimatethedifferencetheyperceivedinthem.Thematerialwasphotographicmaterialwithdifferentimagedistortionsandcontents.Toexaminethespatialdistributionoffixations,wedefinedtheregionsofinterestusingamemorytaskandcalculatedinformationentropytoestimatethespatialconcentrationoffixations.Thequalitytaskwasfasterandneededfewerfixationsandthefirsteightfixationsweremoreconcentratedoncertainimageareasthaninthechangedetectiontask.Thebottom-upinfluencesoftheimagealsocausedmorevariationtothegazebehaviorinthequalityestimationtaskthaninthechangedetectiontask.Theresultsshowthatthestrategiesformakingthetasksaredifferentandinsubjectiveimageestimationstudiesitisimportanttothinkaboutthetask.

8293-21, Session 7A

Measuring saliency in images: which experimental parameters for the assessment of image quality?C.Fredembach,G.Woolfe,J.Wang,CanonInformationSystemsResearchAustraliaPty.Ltd.(Australia)

Predictingwhichareasofanimageareperceptuallysalientorattendedtohasbecomeanessentialpre-requisiteofmanycomputervisionapplications.Becauseobserversarenotoriouslyunreliableinrememberingwheretheylookaposteriori,andbecauseaskingwheretheylookwhileobservingtheimagenecessarilyinfluencestheresults,groundtruthaboutsaliencyandvisualattentionhastobeobtainedbygazetrackingmethods.

FromtheearlyworkofBuswellandYarbustothemostrecentforaysincomputervisiontherehasbeen,perhapsunfortunately,littleagreementonstandardisationofeyetrackingprotocolsformeasuringvisualattention.Asthenumberofparametersinvolvedintheexperimentalmethodologycanbelarge,theirindividualinfluenceonthefinalresultsisnotwellunderstood.Consequently,theperformanceofsaliencyalgorithms,whenassessedbycorrelationtechniquesvariesgreatlyacrosstheliterature.

Inthispaper,weconcernourselveswiththeproblemofimagequality.Specifically:wherepeoplelookwhenjudgingimages.Weshowthatinthiscase,theperformancegapbetweenexistingsaliencypredictionalgorithmsandexperimentalresultsissignificantlylargerthanotherwisereported.Tounderstandthisdiscrepancy,wefirstsetouttodeviseanexperimentalprotocolthatisadaptedtothetaskofmeasuringimagequality.Inasecondstep,wecompareourexperimentalparameterswiththeonesofexistingmethodsandshowthatalotofthevariabilitycandirectlybeascribedtothesedifferencesinexperimentalmethodologyandchoiceofvariables.

Inparticular,thechoiceofatask:judgingimagequalityvs.freeviewinghasagreatimpactonmeasuredsaliencymaps,suggestingthatevenforamildlycognitivetask,groundtruthobtainedbyfreeviewingdoesnotadaptwell.Carefulanalysisofthepriorartalsorevealsthatsystematicbiascanoccurdependingoninstrumentalcalibrationandthechoiceoftestimages.

Weconcludethisworkbyproposingasetofparameters,tasksandimagesthatcanbeusedtocomparethevarioussaliencypredictionmethodsinamannerthatismeaningfulforimagequalityassessment.

8293-29, Session 7A

Examining the effect of task on viewing behavior in videos using saliency mapsH.Alers,J.A.Redi,TechnischeUniv.Delft(Netherlands);I.Heynderickx,PhilipsResearch(Netherlands)

Thisworkisaimedatunderstandinghowtheviewingtaskandlevelofvideoqualityaffecttheviewingbehavioroftheobserver.Totestthat,adatabaseof50videosof20secondseachwascreated.Thesevideosweredegradedtodifferentlevelsofquality.Consequently,observers

Conference 8293


wereaskedtowatchthesevideoswherehalfofthemweregiventhetaskofscoringthequalitylevelwhiletherestweretoldtosimplywatchthevideosfreely.Byusinganeyetrackingdevice,itwaspossibletorecordtheviewingbehaviordataoftheobservers.Thisdatawasusedtocreatesaliencymapsforeachsecondofvideo,whichmadeitpossibletocomparetheviewingbehaviorbetweenthetwotasksacrossdifferentlevelsofquality.Theresultssuggestthatthereisadifferenceintheviewingbehaviordependingonthetaskgiventotheviewerwhichisnoteffectedwiththequalitylevelofthevideo.Itisalsoshownthattherearespecificattributesinthevideos,whichaffectthelevelofsimilarityinviewingbehaviorbetweenthetwotasks.

8293-23, Session 7B

Investigations of the tone reproduction curves on the perceived image quality for fine art reproductionsJ.Jiang,F.Frey,S.Farnand,RochesterInstituteofTechnology(UnitedStates)

Basedontheresultfromarecentweb-basedexperimentconductedtoevaluatetheperceivedimagequalitywithoutthereferencesponsoredbyMellonFoundation,aprojectisbeingconductedtounderstandtheeffectofthecontrastadjustmentmadeintheprocessoffineartreproductionontheperceivedimagequality.Contrastsareusuallyadjustedusingthetonereproductioncurve(TRC)inordertoachievethehighestpossibledynamicrangeandtobetterrepresenttheoriginalartwork.Copiesoffineartimagesaresimulatedusingtonereproductioncurvescollectedfromdifferentmuseums.Duringtheexperiment,observersareinstructedtorateeachtestimagebasedontheirpreferenceinaweb-basedenvironment.Observerswillbeaskedtoclickontheareathatmostinfluencestheirdecisions.Moreattentionmaybeneededtotheareasonwhichmostclicksaremadewhenthetonereproductioncurveisadjustedduringvisualediting.Tounderstandtheimpactoftheoriginalonparticipants’preferencejudgments,theexperimentwillbeconductedbothwithandwithoutthereference.TheresultmaybeindicativeofwhetheraseparateTRCisneededforfineartreproductionsdependingontheavailabilityoftheoriginalwhenthereproducedimageisviewed.

8293-30, Session 7B

Characterizing eye movements during temporal- and global-quality assessment of h.264 compressed video sequencesC.Mantel,N.Guyader,P.Ladret,G.Ionescu,Gipsa-lab(France);T.Kunlin,STMicroelectronics(France)

Studieshaveshownthatthedeploymentofvisualattentioniscloselylinktotheassessmentofimageorvideoquality,thoughthislinkisnotyetfullyunderstood.Theinfluenceofratingtemporalqualityofcompressedvideosoverthewayanobserverdeployshisattentionisinvestigatedinthispaper.

Weset-upasubjectiveexperimentinwhichtheeyemovementsofobserversarerecordedduringthreedifferenttasks:afree-viewingtask(FT),aglobalqualityassessmenttaskandatemporalqualityassessmenttask.TheFTactsasareferencetowhichwecomparetheeyemovementsduringthetwoothertasks.

Aspreviouslyshown,observersassessingglobalqualitygazeatlocationsdissimilartothosefixatedduringtheFT.Fortemporalqualityassessment,itseemsthatthefixatedlocationsareclosertoFTthantheglobalqualityassessmentfixatedlocations.

Ourresultssuggestthatthelocationsobserverslookatdonotdependonthedisplayedvideoqualitylevel.Qualityhoweverinfluencesthewayparticipantslookatvideos:thelowerthequality,thelongertheygazeatapreciselocation.TheareafixatedseemstobemuchsmallerduringthequalityassessmenttasksthanduringtheFTforeitherperfectorpoorqualitylevel.

Theevolutionovertimeofallindicatorssuggeststhat,duringthefirst1or2seconds,thesignalpropertiesofthevideosarethemainattractors

fortheparticipants’eyemovements.Instructionsonlyseemtoplayaroleafterwardsonthedeploymentoftheparticipants’visualattention.

8293-31, Session 7B

A compressed sensing model of crowding in peripheral visionJ.Hocke,Univ.zuLübeck(Germany);M.Dorr,SchepensEyeResearchInstitute(UnitedStates);E.Barth,Univ.zuLübeck(Germany)

Weheremodelperipheralvisioninacompressedsensingframeworkandfindthattypicalletter-crowdingeffectsarisenaturally.Imagesarere-sampledwithsamplesthataredistributedaccordingtoaPoisson-diskdistribution.Thesesamplesarethenconsideredtobeinputstoanetworkofneuronssuchthattheoutputneuronsarerandomlyconnectedtotheinputneurons.Alsotheweightsoftheconnectionsareassignedrandomly.Convergenceismodeledbyallowingforfeweroutputneuronsthaninputneurons.Byreconstructingtheoriginalimagefromtheactivityoftheoutputneuronsweestimatetheamountofinformationdeliveredattheoutput.Toreconstructtheoriginalimage,thesignalattheoutputneuronsismodeledasalinearsuperpositioninasparsebasisandasparse-approximationalgorithmisapplied.Thereconstructedimagesexhibittypicalcrowdingartifactsinthesensethatsingleletterscanberecognizedbutnotletterswhichareflankedbyadditionalletters.Whenthesamplingrateisincreased,aswouldbethecaseincentralvision,boththesingleandtheflankedletterscanberecognized.

8293-32, Session 7B

Foveated self-similarity in nonlocal image filteringA.Foi,TampereUniv.ofTechnology(Finland);G.Boracchi,PolitecnicodiMilano(Italy)

Weinvestigatetheroleoffoveationinnonlocalimagefilteringand,inparticular,intheNLmeansdenoisingalgorithm.Thenonlocalself-similaritybetweenregionsatdifferentimagelocationsisusedinNLmeanstoadaptivelydeterminetheweightsforpointwiseestimation.Thissimilaritybetweendifferentregionsiscomputedthroughthewindowedphotometricdifferences,wherethewindowfunctiondecaystozeroaswemoveawayfromcenterofthecorrespondingregion.

Wereplacesuchwindoweddifferencewithafoveateddifference:insteadofmultiplyingthephotometricdifferencesagainstawindowfunction,weblurthedifferenceswithpoint-spreadfunctionshavingincreasingstandard-deviation(i.e.increasingspread)asthedistancefromtheregion’scentergrows.Thisfoveateddifferencemimicstheactualabilityofthehumanvisualsystemtoperceivedetailsattheperipheryofthepointoffixation.

Ourstudyshowthatafoveatedself-similarityleadstoanimprovementintheresultingimageestimate,accordingbothtoobjectivecriteriaandvisualappearance,particularlyduetobettercontrastandsharpness.

Wediscussthisformofself-similarityinthecontextofnaturalimagestatisticsaswellasitspotentialimplicationsinexplainingtheroleofeyemovementsinimprovingvisualperception.

8293-33, Session 7B

A statistical study of the correlation between interest points and gaze pointsM.Nauge,M.Larabi,Univ.dePoitiers(France)

Inthiswork,weproposetostudytheperformanceofthesedetectorswithatotallydifferentpointofview,bystudyingsimilarity/correlationbetweenthesubjectivegazepointsandtheobjectiveinterestpoints.TheaimistodeterminewhethertheinterestpointscanbeusedtopredictsalientinformationonanimageliketheHVSdoes.Thiscanhelpforseveralapplicationslikequalityassessment,simplifiedsaliency

Conference 8293


mapsconstruction,...Eventhoughtheinterestpointshavenotbeenoriginallydesignedtobeclosetothegazepoints,theymayhaveaparticularsettingthatmaximizesthesimilaritybetweenthem.Forthisstudyabatteryofstatisticaltoolsisusedtotestalargerangeofsettings/configurationsforthethreeinterestpointdetectorsmentionedpreviously.

WeusedseveralstatisticaltoolssuchasBartlett,ANOVA,...tounderstandtheeffectandtheinfluenceofeachparametersforeachdetector.Thesestudiesillustratethatparticularparameterscanminimizethecostoftransformationandpredictinterestpointsinaccordancewiththesubjectivegazepoints.WealsoproposedasolutiontogiveascaletofacilitatetheinterpretationoftheEMDvaluesbyanalyzingthemeanhumanbehavior.Bycomparingthebestsettingforeachdetectorwecanalsoindicatewhichdetectoristhemostreliabletoestimatethesubjectivegazepoints.ThisstudyisalsoagoodwaytoprovethaninterestpointsdetectorsharesomepropertieswiththeHVS.

8293-34, Session 7B

Interest point analysis as a model for the Poggendorff illusionF.W.M.Stentiford,Univ.CollegeLondon(UnitedKingdom)

ThispaperdescribesarecognitionmechanismbasedontherelationshipsbetweeninterestpointsandtheirpropertiesthatisappliedtotheproblemofmodellingthePoggendorffillusion.Therecognitionmechanismisshowntoperforminthesamemannerashumanvisiononthestandardillusionandreducedeffectsaremodelledonavariantwithoutparallels.TheresultsshowthattherecognitionmechanismproduceshighsimilarityscoresbetweencontinuoustransversallinesandPoggendorfffigureswherethelinesegmentsareoffsetfromthecollinearposition.Thepeaksoccurinthesamedirectionofshiftasreportedinpsychophysicalinvestigationsofhumanvision.Themostprominentpeaksoccurwithtransversalsinclinedatthesmallestanglestotheverticalanddecreasetowardstheperpendicularposition(90°).TheanalysisoffigureswithnoparallelsalsoyieldedapeakalthoughthiswasnotasprominentaswiththeconventionalPoggendorfffigures.Thiswasalsoinagreementwithearlierbehaviouralexperiments.TheanalysisisalsoshowntobeconsistentwithpsychophysicalfindingswhereobtuseangledvariantsofthePoggendorfffiguresyieldstrongereffectsthantheacuteangledvariants.

8293-24, Session 8

A method of detecting changes in image quality via sensing on customer documentsW.Wu,B.Xu,J.C.Handley,XeroxCorp.(UnitedStates)

Itisofgreatvaluetobeabletotrackimagequalityofaprintingsystemanddetectchangesbefore/whenitoccurs.Todothateffectively,imagequalitydataneedtobeconstantlygatheredandprocessed.Acommonapproachistoprintandmeasuretest-patternsover-timeatapre-determinedscheduleandthenanalyzethemeasuredimagequalitydatatodiscover/detectchanges.Butduetothepresenceofotherprinternoises,suchaspage-to-pageinstability,mottleetc.,itislikelythatthemeasuredimagequalitydataforagivenimagequalityattributeofinterest(e.g.streak)atagiventimeisgovernedbyastatisticalmodelratherthanadeterministicone.Thisimposesdifficultyformethodsintendedtodetectimagequalitychangesreliablyunlesssufficientdataoftestsamplesarecollected.However,thesetestsamplesarenonvalue-addtothecustomersandshouldbeminimized.Analternativeistodirectlymeasureandassesstheimagequalityattributesofinterestfromcustomerpagesandpost-processingthemfordetectingchanges.Inadditiontothedifficultycausedbysourcesofotherprinternoise,variableimagecontentsfromcustomerpagesfurtherimposechallengesinthechangedetection.Thispaperaddressestheseissuesandpresentsafeasiblesolutioninwhichchangepointsaredetectedbystatisticalmodel-ranking.

8293-25, Session 8

Print quality analysis for ink-saving algorithmsM.V.OrtizSegovia,PurdueUniv.(UnitedStates)andOcéPrintLogicTechnologies(France);N.Bonnier,OcéPrintLogicTechnologies(France);J.P.Allebach,PurdueUniv.(UnitedStates)

Ink-savingstrategiesforCMYKprintershaveevolvedfromtheirearlierstageswherethe‘draft’printmodewasthemainoptionavailabletocontrolinkusage.Thesavingswereachievedbyprintingalternatedotsinanimageattheexpenseofreducingprintqualityconsiderably.Nowadays,customersarenotonlyunwillingtocompromisequalitybuthavehigherexpectationsregardingbothvisualprintqualityandinkreductionsolutions.Therefore,theneedformoreintricateinksavingsolutionswithlowerimpactonprintqualityisevident.Printing-relatedfactorssuchasthewaytheprinterplacesthedotsonthepaperandtheink-substrateinteractionplayimportantandcomplexrolesinthecharacterizationandmodelingoftheprintingprocessthatmaketheinkreductiontopicachallengingproblem.Inourstudy,weareinterestedinfindingtheconnectionsbetweendifferentinkreductionlevelsofagivenink-savingalgorithmandprintqualityattributessuchassharpness,artifacts,andcolorsaturation.

8293-26, Session 8

Masking mediated print defect visibility predictorX.Jing,PurdueUniv.(UnitedStates);H.Nachieli,D.Shaked,S.Shiffman,Hewlett-PackardLabs.IsraelLtd.(Israel);J.P.Allebach,PurdueUniv.(UnitedStates)

Incommercialprintingcontexts,itisquitereasonablethatcustomersexpectgoodqualityprintouts.Therefore,printshopsneedtodesigntheirworkflowtopayattentiontothisimportantaspect.Thismakesprintqualityassessmentquiteimportantfordevelopersofprintingsystems.However,therearenotmanywell-developedintegratedmeasurementsofprintquality.Therealsohasbeenalackofattentionintheexistingmodelstotheeffectofcontent-maskingonthevisibilityofprintdefects.Furthermore,suchaprintqualityassessmentshouldtrulyrepresenttheviewers’observations.Therefore,generatinganintegratedprintqualitymodelwhichcanpredictviewers’observationsisdifficultbutgreatlyneeded.

Comparedwithotherimageandprintqualityresearch,ourworkisfocusedontheprintqualityoftypicalprintoutsprintedonadigitalcommercialprintingpress.Inthispaper,weproposeaMasking-MediatedDefectVisibilityPredictor(MMDVP)topredictthevisibilityofdefectsinthepresenceofcustomercontent.Theparametersofthealgorithmaretrainedfromground-truthimagesthathavebeenmarkedbysubjects.TheMMDVPcouldhelpthepressoperatordecidewhethertheprintqualityisacceptableforspecificcustomerrequirements.Ultimately,thismodelcanbeusedtooptimizetheprint-shopworkflow.

8293-27, Session 8

Psychophysical evaluation of banding visibility in the presence of print contentJ.Zhang,PurdueUniv.(UnitedStates);D.Shaked,H.Nachlieli,S.Shiffman,Hewlett-PackardLabs.IsraelLtd.(Israel);J.P.Allebach,PurdueUniv.(UnitedStates)

Observingandevaluatingprintdefectsrepresentsamajorchallengeintheareaofprintqualityresearch.Visualidentificationandquantificationofprintdefectsbecomesakeyissueforimprovingprintquality.However,thepagecontentmayconfoundthevisualevaluationofprintdefectsinactualprintouts.Ourresearchisfocusedonbandinginthepresenceofprintcontentinthecontextofcommercialprinting.Inthispaper,apsychophysicalexperimentisdescribedtoevaluatetheperceptionofbandsinthepresenceofprintcontent.Anumberofbandingdefectsareaddedbywayofsimulationtoaselectedsetof

Conference 8293


commercialprintcontenttoformoursetofstimuli.Theparticipantsintheexperimentmarkthesestimulibasedontheirobservationsviaagraphicaluserinterface(GUI).Basedonthecollectionofthemarkedstimuli,wewereabletoseegeneralconsistencyamongdifferentparticipants.Moreover,theresultsshowedthatthelikelihoodofanobserverperceivingthebandingdefectinasmoothareaismuchhigherthaninahighfrequencyarea.Furthermore,ourresultsalsoindicatethattheluminanceoftheimagemaylocallyaffectthevisibilityoftheprintdefectstosomedegree.

8293-28, Session 9

No-reference video quality assessment of H.264 video streams based on semantic saliency mapsH.Boujut,J.Benois-Pineau,T.Ahmed,BordeauxUniv.(France);O.Hadar,Ben-GurionUniv.oftheNegev(Israel);P.Bonnet,AudematWorldcastSystems(France)

ThepapercontributestoNo-ReferencevideoqualityassessmentofbroadcastedHDvideooverIPnetworksandDVB.TheNR-qualityassessmentresearchisofprimarilyimportanceforthecommunitybecauseofthewiderangeofapplicationsandtheinherentdifficultyofthetask.WeproposetoenhancethevisualsaliencymodelofourNo-ReferencevideoqualityassessmentmetricWeightedMacro-BlockErrorRate(WMBER)basedonsaliencymapsandmacro-blockerrordetection.Thisenhancementofvisualsaliencyisperformedbyconsideringtheinfluenceofsemanticsofthevisualsceneinthevisualattention.Recentworkshaveshowedthatsaliencymapsarewellsuitedtomeasuretheperceivedqualityinthecontextoflossyvideobroadcastingnetworks.Howevervisualsaliencymodelsaremainlybasedonabottom-upapproachwhichdoesnottakeintoaccountthesemanticsofthevisualscene.Newworksonmodelingvisualattentionhaveshownthatsemanticsincreasevisualattentionespeciallyonfaces.Itwasobservedthatareaswhichcontainfacesgrabtheattention16.6timesmorethanareaswithoutfacesandwiththesamesaliency.Furthermore,S.HemamiandA.Reibmanalsostressthattheperceivedqualitydependsontheusefulnessofthecontent.Thus,inthiswork,wedecidetostayfocusedonfacedetectionandtrackingtoenhanceourbottom-upvisualattentionmodel.

8293-29, Session 9

Linking quality assessment of free-viewpoint video objects up with algorithm developmentS.Kepplinger,TechnischeUniv.Ilmenau(Germany)

ThispaperpresentsanewapproachusinganextendedmodelforthelinkageofQualityofExperiencewiththetechnicalrealization,supportingalgorithmicdevelopmentsinthefieldoffreeviewpointvideo.TheinterlinkingmodelofQualityofExperiencewiththetechnicalrealizationisimplementedbytakingintoaccountsubjectiveevaluationresultsaswellaspossiblevariancesofalgorithmicprocesses.Thisisusedtosupportthescalabilityandadaptabilityofthesystembasedontheendusers’requirements.TheextendedmodelisdefinedafteradetailedliteraturereviewshowingthelackofanadequatewaytolinkQualityofExperiencewithalgorithmicdevelopments,takingintoaccountexistingapproaches.Themodelincludespriorevaluationactivitiesonthesubjectivequalityassessmentoffreeviewpointvideoobjectsusedwithinthecontextofvideocommunicationtosupporteyecontact.However,qualityestimationinthisparticularusecasehasnotbeencoveredyet,andadequateapproachesaremissing.Amethodologicalapproachtodefinequalityinfluencingfactors,anditsresults,willbepresented.Adetaileddescriptionoftheinterlinkingmodeltakingintoaccounttheseresultswillbegiven,andawayofweightingqualityinfluencingfactorswillthereforebeoutlined.

8293-30, Session 9

A perceptual optimization of H.264/AVC bit allocation at the frame and macroblock levelsM.Hrarti,H.Saadane,M.Larabi,XLIM-SIC(France)

Ratecontrolisakeytechnologyforanefficientencodertoachieveahighandconsistentqualityforthewholevideosequenceunderthechannelbandwidthanddelay/bufferconstraints.Ingeneral,ratecontrolincludestwoparts;thebitallocationusedtoestimatetargetbitsforencodingthecurrentframeormacroblock;andtheQP(QuantizationParameter)determinationforachievingaccuratebitsallocation.

IntheH.264/AVCratecontrolalgorithm,thebitallocationprocessandtheQPdeterminationarenotoptimal.InapreviousworkwehaveproposedRate-Quantization(R-Q)modelsforIntraandInterframesusedtodealwiththeQPdeterminationshortcoming.Inthispaper,welooktoovercomethelimitationofthebitallocationprocessattheframeandthemacroblocklayers.Attheframelevel,weenhancethebitallocationprocessbyexploitingframecomplexitymeasures.Atmacroblocklevel,thevisualsaliencyisusedinthebitallocationprocess.Thebasicideaistopromotesalientmacroblocks.Hence,asaliencymap,basedonaBottom-Upapproach,isgeneratedandamacroblockclassificationisperformed.ThisclassificationisthenusedtoaccuratelyadjustUBitsH264whichrepresentstheusualbitbudgetestimatedbyH.264/AVCbitallocationprocess.SimulationshavebeencarriedoutusingJM15.0referencesoftware,severalvideosequencesanddifferenttargetbit-rates.IncomparisonwithJM15.0algorithm,theproposedapproachimprovesthecodingefficiencyintermsofPSNR(upto+2.05dB),andSSIM(upto4%).Furthermore,thebandwidthconstraintisalwayssatisfiedbecausetheactualbit-rateisalwayslowerthanorequaltothetargetbit-rate.

8293-31, Session 9

QoE assessment method for mobile video services based on user motivationF.Kobayashi,M.Masuda,T.Hayashi,NTTNetworkInnovationLabs.(Japan)

Toensurethatvideocommunicationservicesmeetthehighexpectationsofendusers,userqualityofexperience(QoE)mustbeproperlyconsidered.Therefore,variousQoEassessmentmethodsforvideoserviceshavebeenproposed.However,severalQoEassessmentmethodsbasedonusermotivationshowsvideoqualityisnottheonlyperspectiveforQoE.Toassessuserexpectationforvideocommunicationservices,wehavetoobtain“userpreferences”inwhichuserinterestinavideomustbeconsideredinadditiontovideoquality,motivation,andlevelofmotivationachievement.Additionally,weconsidermultipleQoEs,suchaspreferenceandmotivation,whichvarywitheachparticipant.WeproposeaQoEassessmentmethodformobilevideoservices.Wegiveparticipantsmotivationbeforetheywatchvideosonmobileterminals.Afterwatching,participantsassessQoEforvideoquality,motivationachievementlevel,anduserpreference.Simultaneously,participantsassesstheirexpectationswhichareintegratedQoEs.Weconductedanexperimentusingproposedmethod.Asaresult,weconcludedthatconsideringuserpreferenceisimportantforQoEassessmentmethodbasesonmotivation.Wealsoclarifiedthattherequiredvideoqualityleveltomeetcertainuserexpectationsisdifferentdependingontheclassificationofparticipants.

8293-32, Session 9

Quality rules for detection, recognition, and identification in video-surveillance applicationsM.Larabi,D.Nicholson,Univ.dePoitiers(France)

Thisworkisfocusingonthedefinitionofaprocedureforthe

Conference 8293


qualificationofcodingschemesforvideosurveillanceapplications.Itconsistsindevelopingandbenchmarkingtoolsthatlearnfromtheexpertiseofpoliceandsecuritydepartment.Thisexpertiseisintendedtobemodeledthankstoacampaignofsubjectivemeasurementallowingtoanalyzethewaytheyareusinginperformingthesecuritytaskslikefaceorlicenseplaterecognition,eventdetectionandsoon.Theresultsoftheprevioustestareusedwillbeusedtotuneandtoconstructahybridmetricbasedonbasicartifactsdetectionduetocompressionandtransmission.

8293-33, Session 10

A learning-based approach for automated quality assessment of computer-rendered imagesX.Zhang,G.Agam,IllinoisInstituteofTechnology(UnitedStates)

Computergeneratedimagesarecommoninnumerouscomputergraphicsapplicationssuchasgames,modeling,andsimulation.Thereisnormallyatradeoffbetweenthetimeallocatedtothegenerationofeachimageframeandandthequalityoftheimage,wherebetterqualityimagesrequiremoreprocessingtime.Specifically,intherenderingof3Dobjects,thesurfacesofobjectsmaybemanipulatedbysubdividingthemintosmallertriangularpatchesand/orsmoothingthemsoastoproducebetterlookingrenderings.Sinceunnecessarysubdivisionresultsinincreasedrenderingtimeandunnecessarysmoothingresultsinreduceddetails,thereisaneedtoautomaticallydeterminetheamountofnecessaryprocessingforproducinggoodqualityrenderedimages.Inthispaperweproposeanovelsupervisedlearningbasedmethodologyforautomaticallypredictingthequalityofrenderedimagesof3Dobjects.Toperformthepredictionwetrainonadatasetwhichislabeledbyhumanobserversforquality.Wearethenabletopredictthequalityofrenderings(notusedinthetraining)withanaveragepredictionerrorofroughly20%.Theproposedapproachiscomparedtoknowntechniquesandisshowntoproducebetterresults.

8293-34, Session 10

A comparison of techniques for superresolution evaluationM.A.Trifas,JacksonvilleStateUniv.(UnitedStates)

Superresolutionapproachescanbeclassifiedintotwomaincategories:thosewhichinferdataandthosewhichsimplypiecetogetherexistingdata.Bothapproachesproduceanapproximationoftheactualhighresolutionimagewhichmaysufferfromfalsepositiveinclusionandfalsenegativeexclusion.Superresolutiontechniquesaregenerallyevaluatedeithervisuallyoronapixel-by-pixelcomparisonbasis.Thevisualapproachsuffersfrombothexperimenterandrespondentbiasesand,whilethepixel-comparisonapproachiseasilyunderstandableandcomputationallysimple,itisquiteproblematic.Specifically,itdoesnoteffectivelyevaluatesuperresolutionperformanceformanyapplicationsanditissusceptibletonoisecreatedbysmall(andinmanycasesirrelevant)pixel-valuedifferences.

Testingwasconductedwithaninferenceenginewhichenhancesusingadatabaseofpatternswhichitistrainedwithpriortoenhancement-presentation.Threetechniquesforimageevaluationwereconductedoneachsuper-resolvedimage.Super-resolvedimageswerecomparedvisuallytothehigh-resolutionoriginals,pixel-by-pixeldifferencewascalculatedandthreshold-basedpixel-by-pixelcomparison(whereerrorwithinaapplication-acceptablemarginisnotaveragedin)wasalsocomputed.Theunsuitabilityofpixel-by-pixelcomparisonforevaluatingsuperresolutiontechniquesformanyapplicationswasdemonstrated.

8293-35, Session 10

Detection of image quality metamers based on the metric for unified image qualityK.Miyata,NationalMuseumofJapaneseHistory(Japan);N.Tsumura,ChibaUniv.(Japan)

Inthispaper,weproposeacriterionshowingthedegreeoftheimagequalitymetamerism,namedasMetricforUnifiedImageQuality(MUIQ),tomeasureoverallimagequalityasafunctionofimagequalitycomponents,thenMUIQisappliedtodetectmetamericimagequalityareasasanapplicationforhistoricalmaterials.TwocoloredareasintheimageareevaluatedbyMUIQandanobjectivegraininessfactorGr,thenifMUIQisthesamebutGrisdifferent,theareaisclassifiedintothemetamericimagequalityarea.Theimagequalitymetamerismisanexpandedconceptoftheconventionalmetamerismdefinedinthecolorscience.ThematerialinvestigatedinthispaperistheoldestextantversionoffoldingscreenpaintingsthatdepictthethrivingcityofKyotodesignatedasanationallyimportantculturalpropertyinJapan.ThelocalMUIQandGrarecalculatedtodetectgoldcoloredareasinthefoldingscreenastheimagequalitymetamers,whicharepaintedbyusinghighgranularitypigmentscomparedwithothercolorareas.Asaresultofthisresearch,thegoldcoloredareasweredetectedbetterthanthemethodusedinourpreviousmethod.

Conference 8293


Conference 8294: Visualization and Data Analysis 2012Monday-Wednesday23-25January2012

Part of Proceedings of SPIE Vol. 8294 Visualization and Data Analysis 2012

8294-01, Session 1

StreamSqueeze: a dynamic stream visualization for monitoring of event dataF.Mansmann,M.Krstajic,F.Fischer,E.Bertini,Univ.Konstanz(Germany)

Whileinclear-cutsituationsautomatedanalyticalsolutionfordatastreamsarealreadyinplace,onlyfewvisualapproacheshavebeenproposedintheliteratureforexploratoryanalysistasksondynamicinformation.However,duetothecompetitiveorsecurity-relatedadvantagesthatreal-timeinformationgivesindomainssuchasfinance,businessornetworking,weareconvincedthatthereisaneedforexploratoryvisualizationtoolsfordatastreams.Undertheconditionsthatneweventshavehigherrelevanceandthatsmoothtransitionsenabletraceabilityofitems,weproposeanoveldynamicstreamvisualizationcalledStreamSqueeze.Inthistechniquethedegreeofinterestofrecentitemsisexpressedthroughanincreaseinsizeandthusrecenteventscanbeshownwithmoredetails.Thetechniquehastwomainbenefits:First,thelayoutalgorithmarrangesitemsinseverallistsofvarioussizesandoptimizesthepositionswithineachlistsothatthetransitionofanitemfromonelisttotheothertriggersleastvisualchanges.Second,theanimationschemeensuresthatfor50percentofthetimeanitemhasastaticscreenpositionwherereadingismosteffectiveandthencontinuouslyshrinksandmovestotheitsnextstaticpositioninthesubsequentlist.Todemonstratethecapabilityofourtechnique,weapplyittolargeandhigh-frequencynewsandsyslogstreamsandshowhowitmaintainsoptimalstabilityofthelayoutundertheconditionsgivenabove.

8294-02, Session 1

Interactive data-centric viewpoint selectionH.S.Kim,D.Unat,S.B.Baden,J.P.Schulze,Univ.ofCalifornia,SanDiego(UnitedStates)

Weproposeanewalgorithmforautomaticviewpointselection.Whilemostpreviousalgorithmsdependoninformationtheoreticframeworks,ouralgorithmsolelyfocusesonthedataitselfwithoutanyrenderingsteps,andfindsaviewdirectionthatbestdescribesthedata.Thealgorithmconsistsoftwomainsteps:featureselectionandviewpointselection.Thefeatureselectionstepisanextensionofthe2DHarrisinterestpointdetectionalgorithm.Thisstepselectscornerand/orhigh-intensitypointsasfeatures,whichcapturestheoverallstructuresandlocaldetails.Thesecondstep,viewpointselection,takesthissetandfindsadirectionthatlaysoutthosepointsinawaythatthevarianceofprojectedpointsofthemismaximized,whichcanbeformulatedasaPrincipalComponentAnalysis(PCA)problem.ThePCAsolutionguaranteesthatthesurfacescapturedbycornerpointsarelesslikelytobedegenerative,andthatthelocaldetailshaveminimumocclusionamongthem.Theentirealgorithmtakesunderasecondforvariousvolumedatasets,whichallowsittobeintegratedintoreal-timevolumerenderingapplicationswhereuserscanmodifythevolumewithtransferfunctions.

8294-03, Session 1

Interactive analysis of situational awareness metricsD.R.Overby,J.A.Wall,J.Keyser,TexasA&MUniv.(UnitedStates)

Digitalsystemsareemployedtomaintainsituationalawarenessofpeopleinvariouscontextsincludingemergencyresponse,disaster

relief,andmilitaryoperations.Becausethesesystemsareoftenoperatedinwirelessenvironmentsandareusedtosupportreal-timedecisionmaking,theaccuracyofthedataprovidedisimportanttomeasureandevaluate.Ourworkhasbeenconductedinconjunctionwithanalystsintheevaluationandperformancecomparisonofdifferentsystemsdesignedtoprovidesituationalawarenessincoordinatedoperations.Tothisend,wedefinedtemporalandspatialmetricsformeasuringtheaccuracyoftheSAdataprovidedbyeachsystem.InthispaperwediscusstheproposedtemporalandspatialmetricsforSAdataandshowhowweprovidedthesemetricsinalinkedcoordinatedmultipleviewenvironmentthatenabledtheanalyststoeffectivelyperformcriticalanalysistasks.ThetemporalmetricisusedtodeterminewhennetworkperformancehasasignificanteffectonSAdata,andthereforeidentifyspecifictimeperiodsinwhichindividualswereprovidedinaccuratedata.Temporalcontextcanbeusedtodeterminethelocalorglobalnatureoftheinaccuracy,andthespatialmetriccanthenbeusedtoidentifygeographiceffectsonnetworkperformanceofthewirelesssystem.

8294-04, Session 2

Incremental visual text analytics of news story developmentM.Krstajic,Univ.Konstanz(Germany)

Onlinenewssourcesproducethousandsofnewsarticleseveryday,reportingonlocalandglobalreal-worldevents.Theseeventsarerepresentedbytopics,i.e.eventepisodesthathavetobeextractedfromtheunstructuredtime-stampeddatathatcontinuouslyarriveovertime.Newsinformationstreamsarepotentiallyunboundedinsize,makingunderstandingtopictemporaldynamicsandrelationshipsachallengingtask.Inthispaper,wepresentavisualanalyticssystemforexplorationofnewstopicsindynamicinformationstreams,whichcombinesinteractivevisualizationandtextminingtechniquestofacilitatetheanalysisofsimilartopicsthatsplitandmergeovertime.Weemploydocumentclusteringtechniquestoextractnewsstoriesthatevolveovertimeinanonlinefashionandanincrementalvisualizationisdesignedtorepresenttemporalcharacteristicsofthestoriesindifferenttimeframes.Byusinginteraction,storiescanbefilteredandexploredinfulldetailwithdetailsondemand.Todemonstratetheusefulnessofoursystem,casestudieswithrealnewsdataarepresentedandshowthecapabilitiesfordetaileddynamictextstreamexploration.

8294-05, Session 2

Guided text analysis using adaptive visual analyticsC.A.Steed,C.T.Symons,F.A.DeNap,T.E.Potok,OakRidgeNationalLab.(UnitedStates)

Thispaperdemonstratesthepromiseofaugmentinginteractivevisualizationswithsemi-supervisedmachinelearningtechniquestoimprovethediscoveryofsignificantassociationsandinsightforsearchingtextualinformation.Morespecifically,wehavedevelopedasystem-calledGryffin-thathostsauniquecollectionoftechniquesthatfacilitateindividualizedinvestigativesearchpertainingtoanever-changingsetofanalyticalquestionsoveranindexedcollectionofopen-sourcepublicationsrelatedtonationalinfrastructure.TheGryffinclienthostsdynamicdisplaysofthesearchresultsviafocus+contextrecordlistings,temporaltimelines,term-frequencyviews,andmultiplecoordinatedviews.Furthermore,astheanalystinteractswiththedisplay,theinteractionsarerecordedandusedtolabelthesearchrecords.Theselabeledrecordsarethenusedtodrivesemi-supervisedmachinelearningalgorithmsthatre-ranktheunlabeledsearchrecords


suchthatpotentiallyrelevantrecordsaremovedtothetopoftherecordlisting.GryffinisdescribedinthecontextofthedailytasksencounteredattheDepartmentofHomelandSecurity’sFusionCenters,withwhomwearecollaboratinginitsdevelopment.Theresultingsystemiscapableofaddressingtheanalyst’sinformationoverloadthatcanbedirectlyattributedtothedelugeofinformationthatmustbeaddressedinsearchandinvestigativeanalysisoftextualinformation.

8294-06, Session 3

Designing a better weather displayC.Ware,M.Plumlee,TheUniv.ofNewHampshire(UnitedStates)

Thevariablesmostcommonlydisplayedonweathermapsareatmosphericpressure,windspeedanddirection,andsurfacetemperature.Buttheyareusuallyshownseparately,nottogetheronasinglemap.Asadesignexercise,wesetthegoaloffindingoutifitispossibletoshowallthreevariables(two2Dscalarfieldsanda2Dvectorfield)simultaneouslysuchthatvaluescanbeaccuratelyreadusingkeysforallvariables,areasonablelevelofdetailisshown,andimportantmeteorologicalfeaturesstandoutclearly.Oursolutioninvolvesemployingthreeperceptual“channels”,acolorchannel,atexturechannel,andamotionchannelinordertoperceptuallyseparatethevariablesandmakethemindependentlyreadable.Weconductedanexperimenttoevaluateournewdesignbothagainstaconventionalsolution,andagainstaglyph-basedsolution.Theevaluationtestedtheabilitiesofnovicesubjectsbothtoreadvaluesusingakey,andtoseemeteorologicalpatternsinthedata.Ournewschemewassuperiorespeciallyintherepresentationofwindpatternsusingthemotionchannel,anditalsoperformedwellenoughintherepresentationofpressureusingthetexturechanneltosuggestitasaviabledesignalternative.

8294-07, Session 3

Visualization feedback for musical ensemble practice: a case study on phrase articulation and dynamicsT.Knight,N.Boulliot,J.Cooperstock,McGillUniv.(Canada)

Weconsiderthepossibleadvantagesofvisualizationinsupportingmusicalinterpretation.Specifically,weinvestigatetheuseofvisuzalizationsinmakingasubjectivejudgementofastudent’sperformancecomparedtoreference“expert’’performanceforparticularaspectsofmusicalperformance---articulationanddynamics.Ourassessmentcriteriafortheeffectivenessofthefeedbackarebasedontheconsistencyofjudgementsmadebytheparticipantsusingeachmodality,thatistosay,indetermininghowwellthestudentmusicianmatchesthereferencemusician,thetimetakentoevaluateeachpairofsamples,andsubjectiveopinionofperceivedutilityofthefeedback.

Forarticulation,differencesinthemeanscoresassignedbytheparticipantstothereferenceversusthestudentperformancewerenotstatisticallysignificantforeachmodality.Thissuggeststhatwhilethevisualizationstrategydidnotofferanyadvantageoverpresentationofthesamplesbyaudioplaybackalone,visualizationneverthelessprovidedsufficientinformationtomakesimilarratings.Fordynamics,fourofoursixparticipantscategorizedthevisualizationsashelpful.Themeansoftheirratingsforthevisualization-onlyandboth-togetherconditionswerenotstatisticallydifferentbutwerestatisticallydifferentfromtheaudio-onlytreatment,indicatingadominanceofthevisualizationswhenpresentedtogetherwithaudio.Moreover,theratingsofdynamicsunderthevisualization-onlyconditionweresignificantlymoreconsistentthantheotherconditions.

8294-08, Session 3

Exploring ensemble visualizationC.G.Healey,M.N.Phadke,L.Pinto,NorthCarolinaStateUniv.(UnitedStates);F.Alabi,J.M.Harter,R.M.TaylorII,TheUniv.ofNorthCarolinaatChapelHill(UnitedStates);X.Wu,Renaissance

ComputingInstitute(UnitedStates);H.Petersen,S.A.Bass,DukeUniv.(UnitedStates)

Anensembleisacollectionofrelateddatasets.Eachdataset,ormember,ofanensembleisnormallylarge,multidimensional,andspatio-temporal.Ensemblesareusedextensivelybyscientistsandmathematicians,forexample,byexecutingasimulationrepeatedlywithslightlydifferentinputparametersandsavingtheresultsinanensembletoseehowparameterchoicesaffectthesimulation.Todrawinferencesfromanensemble,scientistsneedtocomparedatabothwithinandbetweenensemblemembers.Weproposetwotechniquestosupportensembleexplorationandcomparison:apairwisesequentialanimationmethodthatvisualizeslocallyneighboringmemberssimultaneously,andascreendoortintingmethodthatvisualizessubsetsofmembersusingscreenspacesubdivision.Wedemonstratethecapabilitiesofbothtechniques,firstusingsyntheticdata,thenwithsimulationdataofheavyioncollisionsinhigh-energyphysics.Resultsshowthatbothtechniquesarecapableofsupportingmeaningfulcomparisonsofensembledata.

8294-09, Session 4

Parallel large-data visualization with display wallsL.Scheidegger,FacebookInc.(UnitedStates);H.Vo,PolytechnicInstituteofNewYorkUniv.(UnitedStates);J.Kruger,Univ.desSaarlandes(Germany);C.T.Silva,PolytechnicInstituteofNewYorkUniv.(UnitedStates);J.L.D.Comba,Univ.FederaldoRioGrandedoSul(Brazil)

Whilethereexistpopularsoftwaretoolsthatleveragethepowerofarraysoftiledhighresolutiondisplays,theyusuallyrequireeithertheuseofaparticularAPIorsignificantprogrammingefforttobeproperlyconfigured.WepresentPVW(ParallelVisualizationusingdisplayWalls),aframeworkthatusesdisplaywallsforscientificvisualization,requiringminimumlaborinsetup,programmingandconfiguration.PVWworksasaplug-intopipeline-basedvisualizationsoftware,andallowsuserstomigrateexistingvisualizationsdesignedforasingle-workstation,single-displaysetuptoalargetileddisplayrunningonadistributedmachine.Ourframeworkisalsoextensible,allowingdifferentAPIsandalgorithmstobemadedisplaywall-awarewithminimumeffort.

8294-10, Session 4

Visual exploratory analysis of a large volume of SQL log data with the SDSS log viewerJ.Zhang,C.Chen,M.Vogeley,D.Pan,DrexelUniv.(UnitedStates);A.Thakar,J.Raddick,TheJohnsHopkinsUniv.(UnitedStates)

User-generatedSQL(StructuredQueryLanguage)querylogsarerichinformationsourcesfordatabaseanalysts,informationscientists,andtheendusersofdatabases.InthisstudyagroupofdatascientistsandinformationscientistworktogethertoanalyzealargevolumeSQLlogdatageneratedbyusersoftheSloanDigitalSkySurvey(SDSS)dataarchiveinordertobetterunderstandusers’dataseekingbehaviors.Whilestatisticalanalysisoftheselogsisusefulataggregatedlevels,efficientlyexploringdetailsandpatternsofqueriesisoftenachallengingtaskduetothetypicallylargedatavolume,multivariatefeatures,andtextcontentinSQLqueries.ToenableandfacilitateeffectiveandefficientexplorationoftheSDSSlogdata,wedesignedaninteractivevisualizationtool,calledtheSDSSLogViewer,whichintegratestimeseriesvisualization,textvisualization,anddynamicquerytechniques.WedescribetwoanalysisscenariosofvisualexplorationofSDSSlogdata,includingunderstandingunusuallyhighdailyquerytrafficandmodelingthetypesofdataseekingbehaviorsofmassivequerygenerators.ThetwoscenariosdemonstratethattheSDSSLogViewerprovidesanovelandpotentiallyvaluableapproachtosupportthesetargetedtasks.

Conference 8294


8294-11, Session 5

Comparison of open-source visual analytics toolkitsJ.R.Harger,SandiaNationalLabs.(UnitedStates)andTheUniv.ofNewMexico(UnitedStates);P.J.Crossno,SandiaNationalLabs.(UnitedStates)

Wepresenttheresultsofthefirststageofatwo-stageevaluationofopensourcevisualanalyticspackages.Thisstageisabroadfeaturecomparisonoverarangeofopensourcetoolkits.Althoughwehadoriginallyintendedtorestrictourselvestocomparingvisualanalyticstoolkits,wequicklyfoundthatveryfewwereavailable.So,weexpandedourstudytoincludeinformationvisualization,graphanalysis,andstatisticalpackages.Weexaminethreeaspectsofeachtoolkit:visualizationfunctions,analysiscapabilities,anddevelopmentenvironments.

Thefirstsectiondescribesthevisualizationfunctionalitycategoriesweexamined,suchasgraphvisualizations,treevisualizations,geo-spatial,generaldataplotsanddiagrams.Weprovidecomparisontablesofeachcategory,specifyingwhichalgorithmsatoolkitusedwhenthatinformationwasavailable.

Next,wedescribeanalysiscapabilities,includinggraphanalysisandstatisticalanalysistechniques.Theseincludefindingshortestpaths,minimumspanningtrees,centralitymeasures,andclustering.Weprovidetablescomparingavailablefunctionalityforthesecategoriesaswell.

Finally,wecomparetheaspectsofthedevelopenvironmentsavailableforeachtoolkit.Wegivecomparisontablesshowingprogramminglanguagebindings,targetplatforms,GUIinterfaces,databasesupportaswellasdocumentationavailable.

8294-12, Session 5

Evaluation of progressive treemaps to convey tree and node propertiesR.Rosenbaum,B.Hamann,Univ.ofCalifornia,Davis(UnitedStates)

Inthispaperweevaluateprogressivetreemaps.Progressiverefinementhasalongtraditioninimagecommunication,butisarelativelynewapproachforinformationpresentation.Besidestechnicalbenefitsitalsopromisestoprovideadvantagesimportantfortheconveyanceofdataproperties.Inthisfirstuserstudyinthisdomain,wefocusontheadditionalvalueofprogressiverefinementfortraditionaltreemapstoconveythetopologyofagivenhierarchicaldatasetandpropertiesofitsnodes.Toachievethis,wecomparetheresultsgainedforcommonsquarifiedtreemapdisplayswithandwithoutprogressionforvariousrelatedtasksandset-ups.Theresultsweobtainedindicatethatprogressivetreemapsallowforabetterconveyanceoftopologicalfeaturesandnodepropertiesinmostset-ups.Wealsoassessedtheopinionsofourstudyparticipantsandfoundthatprogressivetreemapsalsoleadtoabetterconfidenceaboutthegivenanswersandprovidemoreassistanceanduserfriendliness.

8294-13, Session 5

Evaluation of multivariate visualizations: a case study of refinements and user experienceM.A.Livingston,J.Decker,U.S.NavalResearchLab.(UnitedStates)

Onedifficultyofmulti-variatevisualization(MVV)isthatthenumberofperceptualchannelsmaybeexceeded.WeembarkedonaseriesofevaluationsofMVVsinanefforttounderstandthelimitationsofattributesthatareusedinMVVs.Inafollow-upstudytopreviouslypublishedresults,weattemptedtouseourpastresultstoinformrefinementstothedesignoftheMVVsandthestudyitself.Someofthesechangesresultedinimprovedperformance,whereasotherdegradedperformance.Wereportresultsfromthefollow-upstudyandacomparisonofdatacollectedfromsubjectswhoparticipatedinbothstudies.OnegratifyingresultincludesimprovedperformancewithanewMVV,AttributeBlocks,

relativetoDimensionalStacking.Ontheotherhand,ourrefinementtoData-drivenSpotsresultedingreatererrorsonthetask.Users’previousexposuretotheMVVsenabledthemtocompletethetasksignificantlyfaster(butnotmoreaccurately).Previousexposurealsoyieldedlowerratingsofsubjectiveworkload.Wediscusstheseintuitiveandcounter-intuitiveresultsandtheimplicationsforMVVdesign.

8294-14, Session 6

Integrating sentiment analysis and term associations with geo-temporal visualizations on customer feedback streamsM.C.Hao,Hewlett-PackardLabs.(UnitedStates);C.Rohrdantz,H.Janetzko,D.A.Keim,Univ.Konstanz(Germany);U.Dayal,Hewlett-PackardLabs.(UnitedStates);L.Haug,Hewlett-PackardCo.(UnitedStates);M.Hsu,Hewlett-PackardLabs.(UnitedStates)

Twittercurrentlyreceivesover190milliontweets(smalltext-basedWebposts)andmanufacturingcompaniesreceiveover10thousandwebproductsurveysaday,inwhichpeoplesharetheirthoughtsregardingawiderangeofproductsandtheirfeatures.Alargenumberoftweetsandcustomersurveysincludeopinionsaboutproductsandservices.However,withTwitterbeingarelativelynewphenomenon,thesetweetsareunderutilizedasasourcefordeterminingcustomersentiments.Toexplorehigh-volumecustomerfeedbackstreams,weintegratethreetimeseries-basedvisualanalysistechniques:(1)feature-basedsentimentanalysisthatextracts,measures,andmapscustomerfeedback;(2)anovelideaoftermassociationsthatidentifyattributes,verbs,andadjectivesfrequentlyoccurringtogether;and(3)newpixelcell-basedsentimentcalendars,geo-temporalmapvisualizationsandself-organizingmapstoidentifyco-occurringandinfluentialopinions.Wehavecombinedthesetechniquesintoawell-fittedsolutionforaneffectiveanalysisoflargecustomerfeedbackstreamssuchasformoviereviews(e.g.,Kung-FuPanda)orwebsurveys(buyers).

8294-15, Session 6

A self-adaptive technique for visualizing geospatial data in 3D with minimum occlusionA.Chaudhuri,H.Shen,TheOhioStateUniv.(UnitedStates)

Geospatialdataareoftenvisualizedas2Dcartographicmapswithinteractivedisplayofdetailon-demand.Integrationofthe2Dmap,whichrepresentshighlevelinformation,withthedetailspertainingtospecificlocationsisakeydesignissueingeovisualization.Solutionsincludemultiplelinkeddisplaysaroundthemapwhichcanimposecognitiveloadontheuserasthenumberoflinksgoesup,andseparatewindoweddisplaysontopofthemapwhichcausesocclusionofthemap.Inthispaper,wepresentaself-adaptivetechniquewhichrevealshiddenlayersofinformationinsingledisplayandbutminimizesocclusionofthe2Dmap.Theproposedtechniquecreatesextrascreenspacebyinvokingcontrolleddeformationofthe2Dmap.Weextendourmethodtoallowsimultaneousdisplayofmultiplewindowsatdifferentmaplocations.Sinceourtechniqueisnotdependentonthetypeofdatatodisplay,weexpectittobeusefultobothcommonusersandthescientists.Casestudiesareprovidedinthepapertodemonstratetheutilityofthemethodinocclusionmanagementandvisualexploration.

8294-16, Session 7

Space/error tradeoffs for lossy wavelet reconstructionJ.Frain,R.D.Bergeron,TheUniv.ofNewHampshire(UnitedStates)

Noabstractavailable

Conference 8294


8294-17, Session 7

A configurable data prefetching scheme for interactive visualization of large-scale volume dataB.Jeong,Schlumberger(UnitedStates);P.Navratil,K.Gaither,G.Abram,G.P.Johnson,TheUniv.ofTexasatAustin(UnitedStates)

Thispaperpresentsanoveldataprefetchingandmemorymanagementschemetosupportinteractivevisualizationoflarge-scalevolumedatasetsusingGPU-basedisosurfaceextraction.Ourdynamicin-coreapproachusesaspan-spacelatticedatastructuretopredictandprefetchtheportionsofadatasetthatarerequiredbyisosurfacequeries,tomanageanapplication-levelvolumedatacache,andtoensureload-balancingforparallelexecution.WealsopresentaGPUmemorymanagementschemethatenhancesisosurfaceextractionandrenderingperformance.Withthesetechniques,weachieverenderingperformancesuperiortootherin-corealgorithmswhileusingdramaticallyfewerresources.

8294-18, Session 7

A general approach for similarity-based linear projections using a genetic algorithmJ.A.Mouradian,B.Hamann,R.Rosenbaum,Univ.ofCalifornia,Davis(UnitedStates)

Awidelyapplicableapproachtovisualizingpropertiesofhigh-dimensionaldataistoviewthedataasalinearprojectionintotwo-orthree-dimensionalspace.However,developinganappropriatelinearprojectionisoftendifficult.Informationcanbelostduringtheprojectionprocess,andmanylinearprojectionmethodsonlyapplytoanarrowrangeofqualitiesthedatamayexhibit.Weproposeageneral-purposegeneticalgorithmtodeveloplinearprojectionsofhigh-dimensionaldatasetswhichpreserveaspecifiedqualityofthedatasetasmuchaspossible.Theobtainedresultsshowthatthealgorithmconvergesquicklyandreliablyforavarietyofdifferentdatasets.

8294-19, Session 7

Image space adaptive volume renderingA.Corcoran,J.Dingliana,TrinityCollegeDublin(Ireland)

Wepresentatechniqueforinteractivedirectvolumerenderingwhichprovidesadaptivesamplingatareducedmemoryrequirementcomparedtotraditionalmethods.Ourtechniqueexploitsframetoframecoherencetoquicklygenerateatwo-dimensionalimportancemapofthevolumewhichguidessamplingrateoptimisationandallowsustoprovideinteractiveframeratesforusernavigationandtransferfunctionchanges.Inadditionourraycastingshaderdetectsanyinconsistenciesinourtwo-dimensionalmapandcorrectsthemontheflytoensurecorrectclassificationofimportantareasofthevolume.


X3DBio1: a visual analysis tool for biomolecular structure explorationH.Yi,RenaissanceComputingInstitute(UnitedStates);A.Singh,Y.G.Yingling,NorthCarolinaStateUniv.(UnitedStates)

Proteintertiarystructureanalysisprovidesvaluableinformationontheirbiochemicalfunctions.Thestructure-to-functionrelationshipcanbedirectlyaddressedthroughthreedimensional(3D)biomolecularstructureexplorationandcomparison.WepresentX3DBio1,avisualanalysistoolfor3Dbiomolecularstructureexploration,whichallowsforeasyvisualanalysisof2Dintra-molecularcontactmapand3Ddensityexplorationforprotein,DNA,andRNAstructures.Acasestudyisalsopresentedinthispapertoillustratetheutilityofthetool.X3DBio1isopensourceandfreelydownloadablefromhttp://sourceforge.net/

projects/x3dbio1.Weexpectthistoolcanbeappliedtosolveavarietyofbiologicalproblems.


Increasing the perceptual salience of relationships in parallel coordinate plotsJ.M.Harter,X.Wu,TheUniv.ofNorthCarolinaatChapelHill(UnitedStates);D.Dougherty,MichiganStateUniv.(UnitedStates);H.Petersen,S.Bass,DukeUniv.(UnitedStates);R.M.TaylorII,TheUniv.ofNorthCarolinaatChapelHill(UnitedStates);M.N.Phadke,L.Pinto,NorthCarolinaStateUniv.(UnitedStates);O.S.Alabi,TheUniv.ofNorthCarolinaatChapelHill(UnitedStates)

Wepresentthreeextensionstoparallelcoordinatesthatincreasetheperceptualsalienceofrelationshipsbetweenaxesinmultivariatedatasets:(1)luminancemodulationmaintainstheabilitytopreattentivelydetectpatternsinthepresenceofoverplotting,(2)addingaone-vs.-allvariabledisplayhighlightsrelationshipsbetweenonevariableandallothers,and(3)addingascatterplotwithintheparallel-coordinatesdisplaypreattentivelyhighlightsclustersandspatiallayoutswithoutstronglyinterferingwiththeparallel-coordinatesdisplay.Thesetechniquescanbecombinedwithoneanotherandwithexistingextensionstoparallelcoordinates,andtwoofthemgeneralizebeyondcaseswithknown-importantaxes.Weappliedthesetechniquestotworeal-worlddatasets(relativisticheavy-ioncollisionhydrodynamicsandweatherobservationswithstatisticalprincipalcomponentanalysis)aswellasthepopularcardataset.Wepresentrelationshipsdiscoveredinthedatasetsusingthesemethods.


Comparative visualization of ensembles using ensemble surface slicingO.S.Alabi,TheUniv.ofNorthCarolinaatChapelHill(UnitedStates);X.Wu,RenaissanceComputingInstitute(UnitedStates);H.Petersen,S.Bass,DukeUniv.(UnitedStates);S.Zhong,MichiganStateUniv.(UnitedStates);M.N.Phadke,L.Pinto,C.G.Healey,NorthCarolinaStateUniv.(UnitedStates);R.M.TaylorII,TheUniv.ofNorthCarolinaatChapelHill(UnitedStates)

Bydefinition,anensembleisasetofsurfacesorvolumesderivedfromaseriesofsimulationsorexperiments.Sometimestheseriesisrunwithdifferentinitialconditionsforoneparametertodetermineparametersensitivity.Theunderstandingandidentificationofvisualsimilaritiesanddifferencesamongtheshapesofmembersofanensembleisanacuteandgrowingchallengeforresearchersacrossthephysicalsciences.Morespecifically,thetaskofgainingspatialunderstandingandidentifyingsimilaritiesanddifferencesbetweenmultiplecomplexgeometricdatasetssimultaneouslyhasprovedchallenging.Thispaperproposesacomparisonandvisualizationtechniquetosupportthevisualstudyofparametersensitivity.Wepresentanovelsingle-imageviewandsamplingtechniquewhichwecallEnsembleSurfaceSlicing(ESS).ESSproducesasingleimagethatisusefulfordeterminingdifferencesandsimilaritiesbetweensurfacessimultaneouslyfromseveraldatasets.WedemonstratetheusefulnessofESSontworeal-worlddatasetsfromourcollaborators.


A performance assessment on the effectiveness of digital image registration methodsS.T.Kacenjar,LockheedMartinCorp.(UnitedStates);B.Li,LockheedMartinSystemsIntegration-Owego(UnitedStates);A.Ostrow,LockheedMartinMaritimeSystems&Sensors(UnitedStates)

DigitalImageCorrelation(DIC)oftime-sequenced-imagery(TSI)has

Conference 8294


becomeaverypopularmethodinthestudyofmaterialdeformation,geologicalterrestrialmovement,medicine,andenvironmentalimpactassessmentsuchasthebeforeandaftermathofatsunami.Byexaminingthebefore-and-afterimagesofsuchproblemdomainsitispossibletoestimatecriticalinformationaboutthescenedeformationandstructuraldifferencesbetweentheimagery.

Derivativemethodshavealsobeenexaminedtoimprovesuchregistrationbaseduponimagepre-filteringbeforeDIC(PBDIC)anddiffusion-based/orderpairedstatistical(DBOPS)methods.Eachofthesemethodsofferdistincttradesoffswithrespecttoprocessingtimeandlock-inaccuracy.

Regardlessoftheapproach,thefidelityofsuchderivedinformationisinfluencedbyseveralfactorsthatdegradeitsaccuracyincluding;cameramodeluncertainty,surfacedeformation,sensornoise,illuminationvariation,andthespatialmobilityofobjectclasseswithintheimagery.

Thispaperassessesandcomparestheimpactofthreesuchdegradingfactors(cameramodeluncertainty,surfacedeformation,andsensornoise)onTSIimageregistrationaccuracyusingtheDIC,PBDIC,andDBOPSregistrationapproaches.Resultsarepresentedintermsofregistrationaccuracy,correlationstrength,andcomputationtime.


An evaluation of rendering and interactive methods for volumetric data exploration in virtual reality environmentsN.Wang,A.Paljic,P.Fuchs,MinesParisTech(France)

Inthispaperweevaluateoneinteractionmethodandfourdisplaytechniquesforexploringvolumetricdatasetsinvirtualrealityimmersiveenvironments.Weproposeanapproachbasedonthedisplayofasubsetofthevolumetricdata,asisosurfaces,andaninteractivemanipulationoftheisosurfacestoallowtheusertolookforlocalpropertiesinthedatasets.Wealsostudiedtheinfluenceoffourdifferentrenderingtechniquesforisosurfacerenderinginavirtualrealitysystem.Thestudyisbasedonasearchandpointtaskina3Dtemperaturefield.Userprecision,taskcompletiontimeandusermovementwereevaluatedduringthetest.Thestudyallowedtochoosethemostsuitablerenderingmodeforisosurfacerepresentation,andprovidedguidelinesfordataexplorationtasksinimmersiveenvironments.


Efficient, dynamic data visualization with persistent data structuresJ.A.Cottam,A.Lumsdaine,IndianaUniv.(UnitedStates)

Workingwithdatathatischangingwhileitisbeingworkedon,socalled“dynamicdata’’,presentsuniquechallengestoavisualizationandanalysisframework.Inparticular,makingrenderingandanalysismutuallyexclusivecanquicklyleadtoeitherlivelockintheanalysis,unresponsivevisualsorincorrectresults.Aframework’sdatastoreisacommonpointofcontentionthatoftendrivesthemutualexclusion.Providingsafe,synchronousaccesstothedatastoreeliminatesthelivelockscenariosandresponsivevisualswhilemaintainingresultcorrectness.Persistentdatastructuresareatechniqueforprovidingsafe,synchronousaccess.Theysupportsafe,synchronousaccessbydirectlysupportingmultipleversionsofthedatastructurewithlimiteddataduplication.Withapersistentdatastructure,renderingactsononeversionofthedatastructurewhileanalysisupdatesanother,effectivelydouble-bufferingthecentraldatastore.Pre-renderingworkbasedonglobalstate(suchasscalingallvaluesrelativetotheglobalmaximum)isalsoefficientlytreatedifindependentlymodifiedversionscanbemerged.TheStencilvisualizationsystemusespersistentdatastructurestoachievetask-basedparallelismbetweenanalysis,pre-renderingandrenderingworkwithlittlesynchronizationoverhead.Withefficientpersistentdatastructures,performancegainsofseveralordersofmagnitudeareachieved.8294-33,PosterSessionRadialvisualizationsforcomparativedataanalysisG.Draper,M.G.Styles,

BrighamYoungUniv.-Hawaii(UnitedStates);R.F.Riesenfeld,BrighamYoungUniv.(UnitedStates)SQiRLisanovelvisualizationsystemforqueryingandvisualizinglargemultivariatedatasets.Althoughinitiallydesignedfornoviceusers,recentextensionstoSQiRLfacilitatemoreadvancedanalysiswithoutsacricingthesimplicitythatmakesthisvisualizationappealingtobeginners.Thedefaultviewprovidesasimple-to-learninterfaceforqueryevaluation.Intermediateusersareprovidedastraightforwardmethodforcomparingtheresultsoftwoqueries.Moreadvanceduserscanmakeuseofaradialcrosstab,”anewinteractivevisualizationtechniquethatmeldstheexpressivepoweroftraditionalcrosstabulationwithadrag-and-dropcanvas.Throughapplicationtomultipledatasets,wedemonstratethesystem’sgenerality.


Exploiting major trends in subject hierarchies for large-scale collection visualizationC.Julien,P.Tirilly,Univ.ofWisconsin-Milwaukee(UnitedStates);J.E.Leide,C.Guastavino,McGillUniv.(Canada)

UnlikeunstructuredWebcollections,manylargedigitalcollectionsarecurrentlyorganizedbysubject;however,theseusefulinformationorganizationstructuresarelargeandcomplexmakingthemdifficulttobrowse.Currentonlinetoolsandvisualizationprototypesshowsmalllocalizedsubsetsanddonotprovidetheabilitytoexplorethepredominantpatternsoftheoverallsubjectstructure.Thisresearchaimstoaddressthisissuebyfirstsimplifyingthesubjectstructureusingtwotechniquesbasedonthehighlyunevendistributionofreal-worldcollections:levelcompressionandchildpruning.Theapproachisdemonstratedusingasampleof130KrecordsorganizedbytheLibraryofCongressSubjectHeadings(LCSH).Promisingresultsshowwecanreducethesubjecthierarchydownto35%ofitsinitialsize,whilemaintainingaccesstoover80%ofmajorsubjectcontent.Thevisualimpactisdemonstratedusingatraditionaloutlineviewallowingsearcherstodynamicallychangetheamountofcomplexitytheyfeelisnecessaryforthetasksathand.


Visualization of multidimensional timeL.A.Tychonievich,BrighamYoungUniv.(UnitedStates)andUniv.ofVirginia(UnitedStates);R.P.Burton,BrighamYoungUniv.(UnitedStates)

Timegenerallyisassumedtobeascalar:itcanbesorted,isunidirectional,andhasonlyasingledimension.Inthisworkwedemonstratethatvector-valuedmultidimensionaltimecanbedefinedmeaningfully,simulatedefficiently,andvisualizedinaninteractivemanner.Wepresenttwoparticularsimulations,providingafirstlookatwhathypertimemaybe“like”frombothaphysicalandanavigationalperspective.Althoughsimilarinmanywaystoourexperience,mTphenomenaalsodifferfrom1Tphenomenaonafundamentallevel.Ourvisualizationframeworkmotivatesobservationsofsomeofthesedifferencesandhelpsusidentifyavarietyofopentasksthatwillfurtherourunderstandingofthecharacteristicsoftime,whateveritsdimensionality.Together,theseresultsformabasisfromwhicharbitraryspace-timedimensionalitiescanbeunderstood.


Degeneracy-aware interpolation of 3D diffusion tensor fieldsC.Bi,S.Takahashi,TheUniv.ofTokyo(Japan);I.Fujishiro,KeioUniv.(Japan)

Visualanalysisof3Ddiffusiontensorfieldshasbecomeanimportanttopicespeciallyinmedicalimagingforunderstandingphysicalpropertiesandmicroscopicstructuresofbiologicaltissues.However,

Conference 8294


itisstilldifficulttocontinuouslytracktheunderlyingfeaturesfromthediscretetensorsamples,duetotheabsenceofappropriateinterpolationschemesinthesensethatweareabletohandlepossibledegeneracywhilefullyrespectingthesmoothtransitionoftensoranisotropicfeatures.Thispaperpresentssuchanapproachtointerpolating3Ddiffusiontensorfields.Theoverallprocessconsistsofthreeprimarysteps.First,wesystematicallylocatethepossiblerotationalinconsistencyoftensoranisotropyarounddegeneratepoints.Thisisaccomplishedbyclusteringdiscretetensorsampleswithsimilaranisotropyandorientationusingaminimumspanningtreestrategy.Then,werectifysuchrotationalinconsistencybyoptimizingtheirrotationaltransformationthroughexplicitlyintroducingtheeigenstructure-basedrepresentationoftensorsamples.Finally,theeigenvectorsandeigenvaluesofthetensorsampleswillbeseparatelyinterpolatedusingtheoptimalcorrespondencebetweeneverypairofadjacenttensorsamplesinthesecondstep.Comparisonswithexistinginterpolationschemeswillbeprovidedtodemonstratetheadvantagesofourscheme,togetherwithseveralresultsoftrackingwhitematterfiberbundlesinhumanbrain.


Visualization and analysis of 3D gene expression patterns in zebrafish using web servicesD.Potikanond,F.J.Verbeek,LeidenUniv.(Netherlands)

Geneexpressionpatternsanalysiswithmicroarrayprovidesquantitativeinformationthatshowshowageneisexpressedinaparticularcondition.Wholemountinsituhybridizationcanbeusedtocapturethespatio-temporalrelationofthegeneexpressionpatterns.Therefore,visualintegrationofgeneexpressiondatafrombothtechniqueswithdigitalatlasdataofanorganismcanhelpidentifyingspatial,temporalandquantitativeaspectsofgeneexpressionindifferentdevelopmentalstages.Inthispaper,wepresentanapproachtoprovideanonlinevisualizationofgeneexpressiondatainzebrafishwithin3Dreconstructionmodelofzebrafishindifferentdevelopmentalstages.WedevelopedSOAPwebservicesthatprovideprogrammaticaccesstothe3Dreconstructiondataandspatial-temporalgeneexpressiondatainourpreviouslydevelopedinformationsystems;the3DdigitalatlasofzebrafishdevelopmentandtheGeneExpressionManagementSystem(GEMS).Inthisapproach,wealsocreatedawebapplicationthatusesthenewlydevelopedwebservicestoretrievedatafromlocalrepositories.Webapplicationalsousesthewebservicestoretrieverelevantmicroarrayanalysisgeneexpressiondatafromexternalcommunityresource;ArrayExpressAtlas.Allthegeneexpressionpatternsdataandthe3Dreconstructiondataaresubsequentlyintegratedusingontologybasedmapping.Todeliverythevisualizationtoendusers,wedevelopedaJavabased3DviewerthatcanbeintegratedwithwebinterfaceallowinguserstovisualizetheintegratedinformationovertheInternet.Theviewersupportsarbitraryre-sectioningofthevolumedataandprovidesvisualizationforrelatedgeneexpressioninformationin3Dgraphicalmodeland3Dannotations.


Vortex core detection: back to basicsA.VanGelder,Univ.ofCalifornia,SantaCruz(UnitedStates)

Arobustdefinitionofvortex-coreaxishaseludedresearchersforadecade.Thispaperreviewsthecriteriadescribedinsomeearlypapers,aswellasrecentpapersthatconcentrateonissuesofunsteadyflows,andattemptstobuildontheirideas.Inparticular,researchershaveproposedcriteriathataredesirableforavortex-coreaxisthatcorrespondtononlocalproperties,yetcurrentextractionmethodsareallbasedonlocalproperties.Analysisispresentedtosupportthethesisthatinaccuraciesobservedinsomepopularearlymethodsareduetoamixtureoffrequenciesintheflowfieldinvorticalregions.Suchmixturesoccurinsteadyflows,aswellasunsteady(time-varying)flows.Thus,thefactthattheflowsareunsteadyisnotnecessarilytheprimaryreasonforinaccuraciesrecentlyobservedinvortexanalysisofsuchflows.Itishypothesizedthattime-varying(unsteady)flowstend

tobemorecomplex,hencetendtohavemixedfrequenciesmoreoftenthansteadyflows.Wefurtherconjecturethatan“effective’’lackofGalileaninvariancemayoccurinsteadyorunsteadyflows,duetotheinteractionoflowfrequencieswithhighfrequencies.

8294-20, Session 10

Visualization of mappings between the gene ontology and cluster treesI.Jusufi,A.Kerren,V.Aleksakhin,LinnaeusUniv.(Sweden);F.Schreiber,LeibnizInstituteofPlantGeneticsandCropPlantResearch(Germany)andMartin-LutherUniv.Halle-Wittenberg(Germany)

Ontologiesandhierarchicalclusteringarebothimportanttoolsinbiologyandmedicinetostudyhigh-throughputdatasuchastranscriptomicsandmetabolomicsdata.Enrichmentofontologytermsinthedataisusedtoidentifystatisticallyoverrepresentedontologyterms,givinginsightintorelevantbiologicalprocessesorfunctionalmodules.Hierarchicalclusteringisastandardmethodtoanalyzeandvisualizedatatofindrelativelyhomogeneousclustersofexperimentaldatapoints.Bothmethodssupporttheanalysisofthesamedataset,butareusuallyconsideredindependently.However,oftenacombinedviewisdesired:visualizingalargedatasetinthecontextofanontologyunderconsiderationofaclusteringofthedata.Thispaperproposesanewvisualizationmethodforthistask.

8294-21, Session 10

Visualizing uncertainty in biological expression dataC.Holzhüter,Univ.Rostock(Germany);A.Lex,D.Schmalstieg,H.Schulz,TechnischeUniv.Graz(Austria);H.Schumann,Univ.Rostock(Germany);M.Streit,TechnischeUniv.Graz(Austria)

Expressionanalysisof~omicsdatausingmicroarrayshasbecomeastandardprocedureinthelifesciences.However,microarraysaresubjecttotechnicallimitationsanderrors,whichrendersthedatagatheredlikelytobeuncertain.

Whileanumberofapproachesexisttotargetthisuncertaintystatistically,itishardlyeverevenshownwhenthedataisvisualizedusingforexampleclusteredheatmaps.Yet,thisishighlyusefulwhentryingnottoomitdatathatis‘goodenough’forananalysis,whichotherwisewouldbediscardedastoounreliablebyestablishedconservativethresholds.Ourapproachaddressesthisshortcomingbyfirstidentifyingthemarginabovetheerrorthresholdofuncertain,yetpossiblystillusefuldata.Itthendisplaysthisuncertaindatainthecontextofthevaliddatabyenhancingaclusteredheatmap.Weemploydifferentvisualrepresentationsforthedifferentkindsofuncertaintyinvolved.Finally,itletstheuserinteractivelyadjustthethresholds,givingvisualfeedbackintheheatmaprepresentation,sothataninformedchoiceonwhichthresholdstousecanbemadeinsteadofapplyingtheusualrule-of-thumbcut-offs.WeexemplifytheusefulnessofourconceptbygivingdetailsforaconcreteusecasefromourpartnersattheMedicalUniversityofGraz,therebydemonstratingourimplementationofthegeneralapproach.

8294-22, Session 11

Instant visitation maps for interactive visualization of uncertain particle trajectoriesK.Bürger,R.Fraedrich,TechnischeUniv.München(Germany);D.Merhof,Univ.Konstanz(Germany);R.Westermann,TechnischeUniv.München(Germany)

Visitationmapsareaneffectivemeanstoanalyzethefrequencyofsimilaroccurrencesinlargesetsofuncertainparticletrajectories.Avisitationmapcountsforeverycellthenumberoftrajectoriespassing

Conference 8294


throughthiscell,anditcanthenbeusedtovisualizepathwaysofacertainvisitationpercentage.Inthispaper,weintroduceaninteractivemethodfortheconstructionandvisualizationofhigh-resolution3Dvisitationmapsforlargenumbersoftrajectories.ToachievethisweemployfunctionalityonrecentGPUstoefficientlyvoxelizeparticletrajectoriesintoa3Dtexturemap.Inthismapwevisualizeenvelopesenclosingparticlepathwaysthatarefollowedbyacertainpercentageofparticlesusingdirectvolumerenderingtechniques.BycombiningvisitationmapconstructionwithGPU-basedMonte-Carloparticletracingwecanevendemonstratetheinstantconstructionofavisitationmapfromagivenvectorfield.Tofacilitatethevisualizationofsafetyregionsaroundpossibletrajectories,wefurthergenerateEuclideandistancetransformvolumestothesetrajectoriesonthefly.Wedemonstratetheapplicationofourapproachforvisualizingthevariationofstreamlinesin3Dflowsduetodifferentnumericalintegrationschemesorerrorsintroducedthroughdatatransformationoperations,aswellasforvisualizingenvelopesofprobabilisticfiberbundlesinDTItractography.

8294-23, Session 11

Motion visualization in large particle simulationsR.Fraedrich,R.Westermann,TechnischeUniv.München(Germany)

Interactivevisualizationoflargeparticlesetsisrequiredtoanalyzethecomplicatedstructuresandformationprocessesinastrophysicalparticlesimulations.Whilesomeresearchhasbeendoneonthedevelopmentofvisualizationtechniquesforsteadyparticlefields,onlyveryfewapproacheshavebeenproposedtointeractivelyvisualizelargetime-varyingfieldsandtheirdynamics.Particletrajectoriesareknowntovisualizedynamicprocessesovertime,butduetoocclusionandvisualclutteringsuchtechniqueshaveonlybeenreportedforverysmallparticlesetssofar.Inthispaperwepresentanoveltechniquetosolvetheseproblems,andwedemonstrateitspotentialforthevisualexplorationoflargeastrophysicalparticlesequences.Wepresentanewhierarchicalspace-timedatastructureforparticlesetswhichallowsforascale-spaceanalysisoftrajectoriesinthesimulatedfields.Incombinationwithvisualizationtechniquesthatadapttotherespectivescales,clustersofparticleswithhomogeneousmotionaswellasseparationandmergingregionscanbeidentifiedeffectively.Theadditionaluseofdynamictransferfunctionstomodulatethecolorandsizeoftrajectoriesallowsemphasizingvariousparticlepropertieslikedirection,speedormass.Furthermore,trackingofinteractivelyselectedparticlesubsetspermitstheusertofocusonstructuresofinterest.

8294-24, Session 11

Animating streamlines with repeated asymmetric patterns for steady flow visualizationC.Yeh,NationalChengKungUniv.(Taiwan);Z.Liu,Univ.ofPennsylvania(UnitedStates);T.Lee,NationalChengKungUniv.(Taiwan)

Animationprovidesintuitivecueingforrevealingessentialspatial-temporalfeaturesofdatainscientificvisualization.ThispaperexploresthedesignofRepeatedAsymmetricPatterns(RAPs)inanimatingevenly-spacedcolor-mappedstreamlinesfordenseaccuratevisualizationofcomplexsteadyflows.Wepresentasmoothcyclicvariable-speedRAPanimationmodelthatperformsvelocity(magnitude)integralluminancetransitiononstreamlines.Thismodelisextendedwithinter-streamlinesynchronizationinluminancevaryingalongthetangentialdirectiontoemulateorthogonaladvancingwavesfromageometry-basedflowrepresentation,andthenwithevenly-spacedhuedifferingintheorthogonaldirectiontoconstructtangentialflowstreaks.Toweavethesetwomutuallydualsetsofpatterns,weproposeanenergy-decreasingstrategythatadoptsan

iterativeyetefficientprocedurefordeterminingtheluminancephaseandhueofeachstreamlineinHSLcolorspace.Wealsoemployadaptiveluminanceinterleavinginthedirectionperpendiculartotheflowtoincreasethecontrastbetweenstreamlines.

Conference 8294


Conference 8295A: Image Processing: Algorithms and Systems XMonday-Wednesday23-25January2012

Part of Proceedings of SPIE Vol. 8295A Image Processing: Algorithms and Systems X

8295A-01, Session 1

Analysis of different image-based biofeedback models for improving cycling performancesD.Bibbo,S.Conforto,I.Bernabucci,M.Carli,M.Schmid,T.D’Alessio,Univ.degliStudidiRomaTre(Italy)

Sportpracticecantakegreatadvantagebythequantitativeassessmentoftaskexecutionthatisstrictlyconnectedtotheimplementationofoptimizedtrainingprocedures.Tothisaimbiofeedbacktechniquesareinterestingtoexploreevenifsuchatrainingapproachimpliesacompletechainforinformationextractioncontaininginstrumenteddevices,processingalgorithmsandgraphicalinterfacestoextractvaluableinformation(i.e.kinematics,dynamics,andelectrophysiology)tobepresentedinreal-timetotheathlete.Incycling,performanceindexesdisplayedinasimpleandperceptiblewaycanhelpthecyclisttooptimizethepedaling.Tothispurpose,inthisworkasystembasedonaninstrumentedpedalforbikesisusedtomeasuretheforcecomponentsexertedduringpedaling.AmechanicalIndexofEfficiency(IE)isthenreal-timecalculatedanddisplayedtotheuser.DifferentgraphicalinterfaceshavebeendesignedandusedinthisstudyinordertounderstandifandhowagraphicalIEbiofeedbackcaninfluencethecyclingperformance.Agroupofsubjectsunderwenttheexperimentalprotocolandpedaledwith(i.e.theinterfaceshavebeenusedinarandomizedorder)andwithoutgraphicalbiofeedback.Preliminaryresultsshowhowtheeffectiveperceptionofthebiofeedbackinfluencesthemotorperformance.

8295A-02, Session 1

Textured areas detection and segmentation in circular harmonic functions domainL.Costantini,L.Capodiferro,FondazioneUgoBordoni(Italy);M.Carli,A.Neri,Univ.degliStudidiRomaTre(Italy)

Inthispaperanoveltechniquefordetectingandsegmentingtexturedareasinnaturalimagesispresented.Althoughitisverydifficulttodefinewhatisatexture,itiseasilyrecognizablebyusers.Texturedareasareanimportantfeatureofnaturalimages,aswellasedges.Nowadays,theimageenhancementalgorithmsautomaticallyimprovethequalityofimages,byremovingnoise,bysharpeningtheedges,andbysmoothingtheflatareas.Thesealgorithmshandlethetexturedareaasflatareasorasedges,thusresultingininformationandqualityloss.Thereforeanautomaticsystemfordetectingtexturedareasisthepreliminaryfundamentaltaskforanyenhancementtechnique.Furthermore,thedetectionofthetexturedareascouldbeexploitedinmanyotherimageprocessingapplication,suchasimageretrievalorimagequalityassessment.Content-basedimageretrieval(CBIR)systemsmakeuseofthelowlevelfeaturestorepresentthecontentofimages.Themostcommonlowlevelfeaturesusedarecolor,textureandedges.InthiscontextanalgorithmthatdetectsthetexturedareacouldimprovetheperformancesofthebothtexturecharacterizationmethodsandtheoverallCBIRsystems.

8295A-03, Session 1

Searching for streamer trajectories on synoptic maps of the sun coronaA.Llebaria,ObservatoireAstronomiquedeMarseille-Provence(France)

Synopticmapsofthesolarcoronaareusefultoolsinordertostudytheevolutionofcoronalactivityforlongperiodsoftime(years).Theyshow

thecoronalactivityasfunctionoftime.Thisactivityismeasuredasbrightnessinacircularprofilearoundthesolardiskasfunctionofthelatitude.Thetimeseriesoftheseprofilesformthesynopticmaps.Therearethreeconspicuoustypesofeventsonsolarcoronaandthereforeonsynopticmaps:1)thecoronalmassejections(CMEs),2)thestreamers3)thecoronalholes.Becauseallthesestructuresareopticallythinthereaddtheirbrightnessinthesynopticmap.Streamersandcoronalholesdefinethemediumtermappearanceofthesolarcorona.CMEsarefrequentandverystrongtransients(ingeneral).Inthispaperweshowhowtodisentangletheseadditivecomponentsofverydiverseappearance:CMEappearastemporaltransients,streamersappearaswanderingtracesandcoronalholesappearaslowlevelsmoothregions.Medianfilteringisefficientfortransients,thresholdingandmorphologicanalysisisgoodforcoronalholesbuttheexistenceofmultiplestreamersneedacombinedapproachtodisentanglethesecomponentsusingHoughtransformtechniquesandmultirésolutionanalysis.

8295A-04, Session 1

Performance evaluation for 2D and 3D filtering methods of noise removal in color imagesV.V.Lukin,A.A.Zelensky,N.N.Ponomarenko,NationalAerospaceUniv.(Ukraine);K.O.Egiazarian,J.T.Astola,TampereUniv.ofTechnology(Finland)

Colorimagesformedbymoderndigitalcamerasareoftennoisyanditismakesdesirabletoremovethisnoisebyimagepre-filtering.Filterstoberecommendedfortheconsideredapplicationhavetoeffectivelysuppressnoisewithintroducingnegligibledistortionsintoprocessedimages.Moreover,suchfiltershavetobefastenoughandtestedforawidevarietyofnaturalimagesandnoiseenvironments.Visualqualityofprocessedimageshastobepaidspecificattention.Fortesting,therecentlydesigneddatabaseTID2008ofdistortedimagesprovidesgoodopportunities.OurstudyisperformedintermsofthestandardPSNRandsomevisualqualitymetrics,inparticular,PSNR-HVS-M.Itisshownthat3D(vector)filteringofcolorimagesisabletoprovideadditionalimprovementcomparedtothe2Dfilteringespeciallyfortexturalimages.Ontheaverage,the3Dfilterproducesabout2dBincreaseofPSNR-HVS-Mcomparedtothe2DDCT-baseddenoising.Theresultsobtainedforspatiallycorrelatednoisecasearesimilar,i.e.,the3Dfilterperformanceisbetterthanthecomponent-wisefilterperformanceintermsofPSNRandPSNR-HVS-Mbyabout2dB.Thereasonsarehighdegreeofsignalcomponentcorrelationincolorimagesandtheindependenceofnoiseincolorcomponents.

8295A-05, Session 2

Integrated text detection and recognition in natural imagesN.S.Roubtsova,TechnischeUniv.Eindhoven(Netherlands)andViNotionB.V.(Netherlands);R.Wijnhoven,ViNotionB.V.(Netherlands)andTechnischeUniv.Eindhoven(Netherlands);P.H.N.deWith,TechnischeUniv.Eindhoven(Netherlands)

Textdetectionandrecognitioninnaturalimageshaveconventionallybeenseeninthepriorartasautonomoustasksexecutedinastrictlysequentialprocessingchainwithlimitedinformationsharingbetweensub-systems.Thisapproachisflawedbecauseitintroduces(1)redundancyinextractingthesametextpropertiesmultipletimesand(2)errorbyprohibitingverificationofhard(oftenbinarized)detectionresultsatlaterstages.Weexplorethepossibilitiesforintegrationofdetectionandrecognitionmodulesbyafeedforwardmultidimensional


informationstream.IntegrationinvolvessuitablecharacterizationofthetextstringatdetectionandapplicationofthisknowledgetoeaserecognitionbyagivenOCRsystem.ThechoiceofcharacterizationpropertiesgenerallydependsontheOCRsystem,althoughsomeofthemhaveprovenuniversallyapplicable.

Weshowthattheproposedintegrationmeasuresenablemorerobustrecognitionoftextincomplex,unconstrainednaturalenvironments.Specifically,integrationbytheproposedmeasures(1)eliminatestextualinputirregularitiesthatrecognitionenginescannothandleand(2)adaptivelytunestherecognitionmodelforeachinputimage.Theformerfunctionboostscorrectdetections,whilethelattermainlyreducesthenumberoffalsepositives.

8295A-06, Session 2

Ear recognition based on edge potential functionF.Battisti,M.Carli,Univ.degliStudidiRomaTre(Italy);F.G.B.DeNatale,Univ.degliStudidiTrento(Italy);A.Neri,Univ.degliStudidiRomaTre(Italy)

Inthispaperabiometricsystemforhumanidentificationbasedonearrecognitionispresented.Theearismodeledassetofcontoursextractedfromtheearimagewithanedgepotentialfunction.Theproposedmatchingalgorithmhasbeentestedinpresenceofseveralimagemodifications.Twohumaneardatabaseshavebeenusedforthetestsandtheachievedexperimentalresultsshowtheeffectivenessoftheproposedscheme.

8295A-07, Session 2

Feature extraction from ladar data using modified GPCAP.F.Stiller,TexasA&MUniv.(UnitedStates)

Inthispaperwepresentamethodforextractingfeatureinformationfromladardataintheformofapointcloud.ThemethodexploitsamodifiedversionofGeneralizedPrincipalComponentAnalysis(GPCA)toextractplanarorothernon-linearsurfaceelementsfromthissortofdata.Theessentialdifficultyisthat,dependingontheaspectoftheobject,certainsurfaceswillbeminimallyexposed.Asaresultwecannotsayinadvancehowmanysurfaceswearelookingforandwecannotreliablydetectsurfacesthatarehitbyonlyafewofthepointsinthecloud.Anadditionaldifficultyoccurswhenrecontructingthesurfacenormalatpointswheretwosurfacesjoin.ThealgorithmhandlesbothissuesandcapturesenoughessentialsurfacefeaturestoallowaccuratealignmenttosayaCADmodelfordetailedrecognition.

8295A-08, Session 2

Recognition of rotated images using the multi-valued neuron and rotation-invariant 2D Fourier descriptorsE.Aizenberg,I.J.Bigio,E.Rodriguez-Diaz,BostonUniv.(UnitedStates)

Inthispaper,weextendthewell-establishedFourierdescriptorsparadigmtoimages,inordertoobtainfeatureswhichareinvarianttoimagerotation.BytransferringanimagefromCartesiantopolarcoordinates,wereducerotationtoacircularshiftalongasingledimension.WhilecomputingtheDiscreteFourierTransform(DFT)ofthepolarimage,aphasenormalizationalgorithmisappliedalongthedimensionofcircularshift,cancellingitseffectontheimagespectrum.Thisalgorithmretainstheuniquenessofthesignal’sphase,andthereforevitalinformationisnotlost.

ThephaseofthederivednormalizedspectrumwasusedtotrainasingleMulti-ValuedNeuron(MVN)torecognizesatellite(LIMA)andhumanface(PICS)imagesrotatedbyanarbitraryangle.100%and96.43%classificationrateswereachievedforeachimageset,

respectively.AdditionaltestingsetswereformedbyrotatingoriginalsamplesthatwentthroughMATLAB’sJPEGroutine,andadditionofGaussiannoise.JPEGcompressionyielded96.57%-100%classificationrateontheLIMAset,and89.76%-97.14%classificationrateonthePICSset.AdditionofGaussiannoise(0.1ofsignal’svariance)resultedin98.00%and75.24%classificationratesfortheLIMAandPICSsets,respectively.Preliminaryresultsshowthatthesuggestedschemeforobtainingrotation-invariantDFTfeaturescombinedwiththeMVNprovidesapromisingapproachforefficientimagerecognition.

8295A-10, Session 3

Amoeba-based superpixel partitioning of multispectral images into elementary, uniform, connected unitsJ.Grazzini,L.Prasad,LosAlamosNationalLab.(UnitedStates)

Segmentationisadifficulttaskbecauseofthehighcomplexityofimages,wherecomplexityreferstothelargevarietyofpictorialrepresentationsofobjectswiththesamesemanticmeaningandalsototheextensiveamountofavailabledetails.Itseemsthereforenatural,andpresumablymoreefficient,toworkwithperceptuallymeaningfulentitiesobtainedfromlow-levelgroupingprocessesinsteadofthepixelrepresentation.Inthatcontext,superpixelsobtainedfromconservativeover-segmentationareacommonpre-processingstepforrecoveringimagefeatures.Inthispaper,weintroduceanewalgorithmthatworksessentiallylikeak-meansbasedlocalclusteringofpixels,butenforcesconnectivity,sothatitcanefficientlygeneratecompact,connected,andnearlyuniformsuperpixels.Ourapproachisbasedontheestimationofamoeba-likeneighborhoodsaroundselectedclustercentersthatexploittheconnectionsbetweensuccessiveimagepixelsalonggeodesicpathsintheimage.Theresultingsuperpixelscapturethespatial/spectralredundancyinimagesandgreatlyreducethecomplexityofsubsequentimageprocessingtasks.Theyprovideconvenientprimitivesfromwhichtocomputelocalimagefeatureswhenobjectspresentinthescenehavediversescalesorwhentheyarenotknowninadvance.Onepossibleapplicationisprovidedbyascale/shapeadaptivetechniqueforchangedetectioninsatelliteimages.

8295A-11, Session 3

Smooth partition of unity with Hermite interpolation: applications to image processingL.T.Dechevsky,P.Zanaty,A.Lakså,B.Bang,NarvikUniv.College(Norway)

Weshallconsiderageneralapproachforinterpolation,representationandeditionof2D(planar)imagesand3D(spatial)shapes.Thisapproachusesanewmathematicalapparatus(see[1])forsmoothconvexresolutionofunityonaverygeneralclassofdomainpartitionstogetherwithHermite-typeinterpolationonascattered-pointset,thedistributionofwhichisconsistentwiththedomainpartitioninconsideration.

Weshallbeusingthetensor-productversionoftheconstructionin[1]asdevelopedin[2],for3Dgeometricrepresentationofimagessampledonpossiblynon-uniformlydistributed(scattered)datapointsets.Inparticular,weshalldiscusshowtohandletheconversionfromTaylor-HermiteinterpolatoryformtoBernstein-Beziercontrol-polygonform.WewillfurtherdemonstratehowthiscanbeusedtoedittheinterpolatedimageinteractivelyusingstandardtechniqueforgeometricmodelingtypicalforComputer-AidedGeometricDesign(CAGD).

Toestablishtheneededisomorphismbetweenthecolorvaluesand3DpointcoordinatesweareusingtheclassicalRGBcoding[3],aswellassomeothermethodsforcolorcodingbasedonrecentdevelopmentsofideasfromtheearlywork[4].

References:

[1]L.T.Dechevsky,“SmoothConvexResolutionofUnityand/or

Conference 8295A


Scattered-PointHermite-InterpolationbyGeneralizedExpo-RationalB-Splines,I:Generalsimplyconnectedcoversandpartitions”,Int.J.PureAppl.Math.65(4),413-460(2010).

[2]L.T.Dechevsky,“MultivariateHermiteinterpolationonscattered-pointsetsusingtensor-productexpo-rationalB-splines”,Communicationatthe37thInt.Conf.onApplicationsonMathematicsinEngineeringandEconomics,AMEE,Sozopol2011

[3]J.Gundersen,L.T.Dechevsky,“Onthescientificvisualizationofcomplex-valuedfunctionsofonecomplexvariable”,In:Proceedingsofthe35thInt.Conf.onApplicationsonMathematicsandEngineeringandEconomics,Sozopol2009,AIPConf.Proc.1184,324-334.Amer.Inst.OfPhys.,Melville,NY,2009

[4]R.S.Hunter,“PhotoelectricColor-DifferenceMeter”,ProceedingsoftheWinterMeetingoftheOpticalSocietyofAmerica.JOSA38(7),661(1948).

8295A-12, Session 3

An algorithm for GPGPU-computing of multidimensional DWTs based on bijective mapping of tensor-product wavelet bases of different number of variablesL.T.Dechevsky,J.Bratlie,B.Bang,A.Lakså,NarvikUniv.College(Norway)

Analgorithmforcomputationofmultivariatewavelettransforms(DWTs)ongraphicsprocessingunits(GPUs)wasproposedin[1].Thisalgorithmwasbasedonmappingtheindicesoforthonormaltensor-productwaveletbasesofdifferentnumberofvariablesandatradeoffbetweenthenumberofvariablesversustheresolutionlevel,sothattheresultingwaveletbasesofdifferentnumberofvariablesarewithdifferentresolution,buttheoveralldimensionofthebasesisthesame.Intheabove-saidpaper,thealgorithmwasdevelopedonlyuptomappingoftheindicesof{emblocks}ofwaveletbasisfunctions.Thiswassufficienttoprovetheconsistencyofthealgorithm,butnotenoughforthe{emmappingoftheindividualbasisfunctions}inthebasesneededforaprogrammingimplementationofthealgorithm.In[2]webeganthestudyofanupgradeofthisconstructionbypassingfromblock-matrixindexmappingontothedetailedindexmappingoftheindividualbasisfunctionsforthecaseofan$n$-variatetensor-productwaveletbasisbeingbijectivelymappedontoaunivariatewaveletbasis.Herewecontinuethisstudybydevelopingafurtherupgradeofthebasis-matchingalgorithmwhichworksinthemostgeneralcaseofbijectivemappingbetween$n_1$-variateand$n_2$-variatetensor-productwaveletbases.Potentialapplicationsoftheresultsobtainedinclude:begin{itemize}item{Incomputationalimaging:forcomputingmultivariateDWTviaGPGPU-programming.}item{Intelecomunications:fortransforminglargevolumesofmultidimensionalgeometricinformationasunivariatesignals.}item{Indataencryption:ifmultiwavelettensor-productbasesareconsideredpossible,theencryptionkeysmayincludethenumberofscalingfunctionsandthevaluesofthefiltercoefficientsgeneratingthem.}end{itemize}

[1]L.T.Dechevsky,J.Gundersen,B.Bang,

Computing$n$-variateorthogonaldiscretewavelettransformsongraphicsprocessingunits.

In:I.Lirkov,S.Margenov,J.Wasniewski(Eds.)

LSSC’2009,LNCS5910,Springer-Verlag,Berlin-Heidelberg,2010,

730--737

[2]L.T.Dechevsky,J.Bratlie,J.Gundersen,

Indexmappingbetweentensor-productwaveletbasesofdifferentnumberofvariables,andcomputingmultivariateorthogonaldiscretewavelettransformsongraphicsprocessingunits,

Toappearin:LSSC’2011,LNCS

8295A-13, Session 4

Quantitative evaluation of image mosaicing in multiple scene categoriesD.Ghosh,S.Park,N.Kaabouch,W.Semke,R.Fevig,TheUniv.ofNorthDakota(UnitedStates)

Inthispaper,weproposeanevaluationmethodologyforimagemosaicingalgorithms.Majorityofpreviousevaluationmethodologieslacksufficientnumberofperformancemetrics;whileotherssufferfromcomputationalcomplication.Theproposedevaluationmethodinvolvesfourmetrics,percentageofmismatches,differenceofpixelintensities,peaksignal-to-noiseratio(PSNR),andmutualinformationtomeasurethequalityofthemosaicingoutputs.TheseoutputsareobtainedusingamosaicingalgorithmbasedonScaleInvariantFeatureTransform(SIFT),BestBinsFirst(BBF),andRandomSampleConsensus(RANSAC)algorithms.Inordertoevaluatemosaicingperformanceobjectively,theproposedmethodcomparesmosaicimageandtheground-truthimagethatdepictthesamesceneview.Theevaluationhasbeenperformedusing36testsequencesfrom3differentcategories:imagesof2Dsurface,imagesofoutdoor3Dscenes,andairborneimagesfromanUnmannedAerialVehicle(UAV)duringtheUniversityofNorthDakotaflighttest.Exhaustivetestingshowsthattheproposedmetricsareefficientinassessingthequalityofmosaicingoutputs.

8295A-14, Session 4

Curvelet transform with adaptive tilingH.Al-Marzouqi,G.Al-Regib,GeorgiaInstituteofTechnology(UnitedStates)

Thecurvelettransformisarecentlyintroducednon-adaptivemulti-scaletransformthathavegainedpopularityintheimageprocessingfield.Inthispaper,westudytheeffectofcustomizedtilingoffrequencycontentinthecurvelettransform.Specifically,weinvestigatetheeffectofthesizeofthecoarsestlevelanditsrelationshiptodenoisingperformance.Basedontheobservedbehavior,weintroduceanalgorithmtoautomaticallychoosetheoptimalnumberofdecompositions.Itsperformanceshowsaclearadvantage,indenoisingapplications,whencomparedtodefaultcurveletdecomposition.Wealsoexaminehowdenoisingisaffectedbyvaryingthenumberofdivisionsperscale.

8295A-15, Session 4

Tetrachromatic colour spaceA.Restrepo,Univ.deLosAndes(Colombia)

BasedonthewellknownRGBcolourspaceandontherecentlyintroducedRungecolourspace,wepresenttheanalogouscolourspacescalledthe“WXYZhypercube”(thatcorrespondstotheCartesianproductoftheallowednormalizedoutput[0,1]ofthe4sensorsofthecorrespondingvisionsystem)andaspaceofthetypehue-colourfulness-luminance,called“4-Runge”colourspacethatisafour-dimensionalsolidball(thatcorrespondstoa“rounding”ofsortsofthehypercube).Thespacesfindapplicationsforexampleinthemodelingofthevisionsystemsoftetrachromaticanimalsandintheprocessingof4-spectralimages.Thehuevariableinthese4-spacesisbidimensional;inparticular,itlivesonatopological2-sphere.Thehuespaceisthe(nonconvex)dodecahedron(calledthe“chromaticdodecahedron”)thatresultsfromconsideringthefacesintheboundaryofthehypercubethatdonothavethepoints[0000](“black”)nor[1111](“white”)asavertex;theboundaryofthehypercubeisapolytopeofthetype{433},inScläflinotation;thechromaticdodecahedronisthusasurfacesubsetofthetopological3-spherethatistheboundaryofthehypercubeWXYZ.DependingontheorderingofthetetradWXYZ,24basictypesofhueresult.In4-Rungespace,thehueisgivenbytwoangles(asinsphericalcoordinates)thatdetermineapointintheequatorial2-sphereoftheboundary3-sphereoftheRunge4-ballthepointintheboundaryistheintersectionoftherayfromthecenteroftheballtothecolourpointandthe3-sphere;thecolourfulnessisgivenbythedistancefromthecolourpointtothecenteroftheballandthe

Conference 8295A


lightnessisgivenasanangularmeasurementoftherayfromthecenter(“intermediategray”)ofthe4-balltothecolourpointinthehyper-ball,andtheraythatgoesfromthecentertothepoint“white”.

8295A-16, Session 5

Smooth image inpainting by least square oriented edge predictionE.Pallotti,L.Capodiferro,F.Mangiatordi,FondazioneUgoBordoni(Italy);P.Sità,Univ.degliStudidiRomaTre(Italy)

Thispaperintroducesannewdigitaledgeorientedalgorithmforautomaticdigitalinpainting.ThebasicideaistheLaguerreGaussanalysisofthestructureinformationoftheregionsaroundthedamagedportionsoftheimagetoextrapolatethegradientoftheluminanceinmissingareabyaleastsquarepredictionfromsimplifiededgemap.Completionofunknownpartsisautomaticallyobtainedbyavariationalmethodusingthepredictedgradientinformationwithsmoothingconstraintsonluminanceandcolorlevel.Experimentsonanumberofimagesshowtheeffectivenessoftheproposedalgorithm.

8295A-17, Session 5

Image inpainting using cubic spline-based edge reconstructionV.Voronin,V.I.Marchuk,South-RussianStateUniv.ofEconomicsandService(RussianFederation);K.O.Egiazarian,TampereUniv.ofTechnology(Finland);A.Sherstobitov,South-RussianStateUniv.ofEconomicsandService(RussianFederation)

Wehavepresentedinthispaperanimageinpaintingmethodwhichcombinesrestorationoftheedges,nontextureregionsandtexture.First,weareusingLPA-ICImethodtorealizesegmentationindamagedimage.Second,cubicsplinecurveisappliedtoreconstructstructureandconnectedgeandcontourindamagedarea.Innextstepweusenewinpaintingproceduretoreconstructtextureandnontextureareasindependently.Fortextureweuseexemplarbasedmethodswhichfindabestmatchingpatchfromothersourceregionandcopyitindamagedimage.Fornontextureweusefastmarchingmethod.Proposedtechniqueallowstoobtainasmallerreconstructionerrorthanthatofthestate-of-the-artimageinpaintingmethods.Wedemonstratetheperformanceofanewapproachviaseveralexamples,showingtheeffectivenessofouralgorithminreconstructionofmissingsmallandlargeobjectsonthetestimages.

8295A-18, Session 5

Global registration and stabilization of jittered and noisy airborne image sequencesN.M.Namazi,TheCatholicUniv.ofAmerica(UnitedStates);W.Scharpf,U.S.NavalResearchLab.(UnitedStates);J.Obermark,DCSCorp.(UnitedStates);J.Caron,ResearchSupportInstruments,Inc.(UnitedStates)

Thispaperisconcernedwiththedevelopmentandimplementationofregistrationandstabilizationmethodinconjunctionwithairborneimagingapplications.Weconsiderthesituationsforwhichthecameramotionandvibrationcollectivelyaffectthenoisyimagesequence.Thegeneralroutinepresentedinthisworkisacombinationoftwoalgorithmsforglobalimageregistrationandimagestabilization.Weuseandpresentexperimentswithrealimagesequencestotrackamovingobjectinthedirectionofitsmotiontrajectory.

8295A-19, Session 6

Image and video restoration via Ising-like modelsO.Hadar,Ben-GurionUniv.oftheNegev(Israel);E.Cohen,TelAvivUniv.(Israel)

Duringthelastdecades,statisticalmodels,suchastheIsingmodel,havebecomeveryusefulindescribingsolidstatesystems.Thesemodelsexcelintheirsimplicityandintuitiveness.Furthermore,theirresultsgetquiteoftenaccurateexperimentalproofs.Leadingresearchers,suchasTanaka,haveusedthemsuccessfullyduringthelastyearstorestoreimages[1].KandeshasusedrecentlyasimplemodeltorestoreB/Wandgrayscaleimagesandachievedfairresults[2].Inthispaperwewillpresenttheresultswehaveachievedusingasimilar,thoughmorecomplexmodelinordertogetbetterresults.Moreover,weshalldescribetheXYmodelwhichenablesustorestorecoloredimages.Additionally,wewilldescribethegeneralmethodinwhichsimilaralgorithmsenableustorestoredegradedvideosignals.Finally,wewillpresentsomeotherphysicalmodelswhichcanbehelpfulingeneraltoprocessimagesandconfronttheirresultswithresultswhicharewellknownfrompreviousworks[1,2,3,4].

8295A-20, Session 6

Region adaptive correction method for radial distortion of fish-eye imageK.S.Song,Y.S.Han,M.G.Kang,YonseiUniv.(Korea,Republicof)

Mostofcamerasfollowpinholecameramodel.However,resultofthismodelmakessomeundesirableeffectsinwideanglelens.Themostseriousproblemamongtheseeffectsisradialdistortionwhichappearsheavilyinfish-eyeimages.Severalgeometricmodelsforcorrectingradialdistortionoffish-eyelensaredeveloped.Mostofthesemodelsrequireonlyoneparameter.However,correctingwithoneparameterislimitedtocorrectbothcentralandouterpartsimultaneously.Aimofthispaperistosolvethisproblem.Theproposedmethodisabletocorrectradialdistortionofbothareasusingregionadaptivedistortionparameter.Eachparameterisdeterminedbyconsideringamountofdistortionineachregionrespectively.Also,theproposedmethodmodifiestheexistingdivisionmodeltocorrectradialdistortionofbothregions.Experimentalresultsshowthatradialdistortionsinbothareasarecorrected.

8295A-21, Session 6

Super-resolution image reconstruction with edge adaptive weight in video sequenceJ.Y.Kwon,D.S.Yoo,M.G.Kang,YonseiUniv.(Korea,Republicof)

Digitalimagesandvideosareusedinmanydigitaldevicesrecently.Also,theresolutionofdisplaybecamelargerthanthatofpreviousyears.Imageup-scalingalgorithmisimportantissuesinceoriginalinputsourceislimitedintransferringwithindatabandwidth.Amongvariousup-scalingalgorithms,Super-Resolution(SR)imagereconstructionmethodisabletoestimatehigh-resolution(HR)imageusingmultiplelow-resolution(LR)images.ConventionalapproachestoestimateHRimagewithLeaseSquare(LS)methodandWeightedLeastSquare(WLS)methodarenotabletoreconstructhigh-frequencyregioneffectivelyincaseitsblurkernelisassumedGaussiankernelinunknownsystem.Also,thesemethodsproducejaggingartifactsfromdeficiencyofLRframes.TheproposedSRalgorithmusesedgeadaptiveWLStoreconstructhigh-frequencyregionconsideringlocalpropertiesandisappliedtovideosequencewithblockprocesstocopewithlocalmotions.Moreover,toapplyvideosequencewithcomplexmotions,weuseselectivelycorrectinformationofreferenceframetoavoiderrorsfromincorrectinformation.Foraccurateadditionalinformationfromreferenceframes,theproposedalgorithmdeterminesadditionalinformationinreferenceframebycomparingwithcurrentframeandreferenceframe.Theexperimentsdemonstratesuperiorperformanceoftheproposedalgorithm.

Conference 8295A


8295A-22, Session 7

Color image enhancement based on genetic algorithm and ensemble empirical mode decompositionS.Bakhtiari,S.S.Agaian,M.Jamshidi,TheUniv.ofTexasatSanAntonio(UnitedStates)

Inthispaper,weintroduceanewmethodforenhancementofthecolorimages.TheproposedapproachutilizesEnsembleEmpiricalModeDecomposition(EEMD),anextensionofEMDmethod,andGeneticAlgorithm(GA).TheHSVcolorspaceisselectedforbrightnessmodificationpurposesinceithasshownahighereffectivenessintheapplicationsunderstudy.Thealgorithmistestedontheunderwaterimagesandtheunderexposedscenes.Forthedarkimages,anonlineartransformisfirstperformedontheluminancechanneloftheoriginalimageforapreliminaryilluminationcorrection.Forunderwaterimages,themeanofeachRGBchannelisinitiallycorrectedbasedonaGAbasedtechnique.Infact,GAisdeployedforoptimizingtheoffsetseachchannelshouldhavewithrespecttothenearoptimalvalues.Theenhancementmeasure,EME,isappliedasthefitnessfunctionoftheGA.Afterthepreprocessingsteps,anEEMDbasedmethodisperformedontheluminancechanneloftheresultingimagetofurthercorrectthebrightnessoftheimage.TheGeneticAlgorithmenablesthepresentedmethodtosettherequiredparametersautomaticallysothattheapplicationscanbefeasibleinautonomousmissions.Theexperimentalresultsdemonstratetheeffectivenessofthealgorithminvisualizingthedetailswhichwereunperceivablebeforeenhancement.

8295A-23, Session 7

Image classification and interpolationA.Khemka,KLA-TencorCorp.(UnitedStates);C.A.Bouman,PurdueUniv.(UnitedStates)

Wehavedevelopedanovelinterpolationmethodforimagescontainingtext,graphicsandnaturalscenes.Themethodallowsustoselectthebestinterpolationalgorithmfordifferentregionsofanimage.Inparticular,wesegmenttheimageintographicalandnaturalregionsandusetheappropriatealgorithmforeachregion.Thenaturalregionsareinterpolatedusingacurrentstate-of-the-artalgorithm.However,whenappliedtographicalimages,thecurrentstate-of-the-artinterpolatorstendtoproduceartifactsatedgediscontinuities.Thus,wedevelopedanovelapproachwhichwecallLowEntropyInterpolation(LEI)algorithmforthegraphicalimages.TheLEIalgorithmishighlynon-linearandproducesverysharpedgeswithveryfewdefectsnecessaryforgoodqualityinterpolationofgraphicalimages.

8295A-24, Session 7

Optimal fractional filter for image segmentationA.Nakib,Y.Schulze,E.Petit,Univ.Paris12-ValdeMarne(France)

Inthispaper,wepresentanewimagethresholdingalgorithmbasedonfractionalfilter(FF).Ourexperimentsshowedthatagoodsegmentationresultcorrespondstoanoptimalorderofthefilter.Then,weproposenewalternativebasedonLegendremoments’tofindtheoptimalorder.ThewholealgorithmiscalledFLM.Thisalgorithmallowstoincludecontextualinformationsuchastheglobalobjectshapeandexploitsthepropertiesofthetwo-dimensionalfractionalintegration.TheefficiencyofFLMwasillustratedbythecomparisontoothersixcompetingmethodsrecentlypublishedanditwastestedonreal-worldproblem.

8295A-25, Session 7

Multi-scale image enhancement using a second derivative-like measure of contrastS.C.Nercessian,K.Panetta,TuftsUniv.(UnitedStates);S.S.

Agaian,TheUniv.ofTexasatSanAntonio(UnitedStates)

Imageenhancementalgorithmsaimtoimprovethevisualqualityofimagesforhumanormachineperception.Mostdirectmulti-scaleimageenhancementmethodsarebasedonenhancingeitherabsoluteintensitychangesortheWebercontrastateachscale,andhavetheadvantagethatthevisualcontrastisenhancedinacontrolledmanner.However,thehumanvisualsystemisnotadaptedtoabsoluteintensitychanges,whiletheWebercontrastisunstableforsmallvaluesofbackgroundluminanceandpotentiallyunsuitableforcompleximagepatterns.TheMichelsoncontrastmeasureisaboundedmeasureofcontrast,butitsexpressiondoesnotallowastraightforwarddirectimageenhancementformulation.Recently,asecondderivative-likemeasure(SDME)ofcontrasthasbeenusedtoassesstheperformanceofimageenhancementalgorithms.TheSDMEisaMichelson-likecontrastmeasureforwhichadirectimageenhancementalgorithmcanbeformulated.Accordingly,weproposeanewdirectmulti-scaleimageenhancementalgorithmbasedontheSDMEinthispaper.Experimentalresultsillustratethepotentialbenefitsoftheproposedalgorithm.

8295A-26, Session 8

A new denoising method in high-dimensional PCA spaceQ.B.Do,A.Beghdadi,M.Luong,Univ.Paris13(France)

Kernel-designbasedmethodsuchasBilateralfilter(BIL),non-localmeans(NLM)filterisknownasoneofthemostattractiveapproachesfordenoising.WeproposeinthispaperanewnoisefilteringmethodinspiredbyBIL,NLMfiltersandprincipalcomponentanalysis(PCA).ThemainideahereistoperformtheBILinamultidimensionalPCA-spaceusingananisotropickernel.Thefilteredmultidimensionalsignalisthentransformedbackontotheimagespatialdomaintoyieldthedesiredenhancedimage.Wewillshowthattheproposedmethodisagenerationofallkernel-designbasedmethods.Theobtainedresultsarehighlypromising.

8295A-27, Session 8

Intelligent detection of impulse noise using multilayer neural network with multi-valued neuronsI.Aizenberg,G.Wallace,TexasA&MUniv.-Texarkana(UnitedStates)

Impulsenoisefilteringisapopularandimportantprobleminimageprocessing.Acommondisadvantageofevenverysophisticatedfiltersissmoothingofedgesandsmallestdetailsduringthefilteringprocess.Totakecareofthisproblem,itispossibletodetectnoisypixelspriortofilteringandtoapplyafilteronlytothepixelsdetectedasnoisy.Manyimpulsedetectorsareknown.Mostofthemarebasedontheanalysisoflocalstatisticalcharacteristicsinawindowaroundeachpixel.Therearesomeintelligentimpulsenoisedetectors,whicharebasedonthestandardmultilayerfeedforwardneuralnetwork(MLF)andfuzzylogic.However,allofthemarebasedontheuseofthesamelocalstatisticalcharacteristics(whichareemployedbyregulardetectors)asthefeatures.Thismakesprocessofnoisedetectionmulti-stageandcomplicated.

SinceMLMVNsignificantlyoutperformsMLFintermsoflearningspeed,functionality,flexibilityandgeneralizationcapability,whensolvingbenchmarkanddifferentreal-worldproblems,itisveryattractivetoapplyitforimpulsenoisedetection.Moreover,itisespeciallyattractivetousejustintensityvaluesinalocalwindowaroundapixelofinterestasthefeatures(andtheMLMVNinputs,respectively).

Themainresult,whichwillbepresentedinthepaper,isaprovenabilityofMLMVNtodetectbothsaltandpepperandrandomimpulsenoisewithahighaccuracy.Moreover,itisshownthattotrainMLMVN,itisenoughtouseasingleimageforthelearningpurposes.Thenotherimages,whichdidnotparticipateinthelearning,canbeprocessedwithahighaccuracy.Afternoisypixelsaredetected,theycanbeefficientlyfilteredevenusingaclassicalmedianfilter.

Conference 8295A


8295A-28, Session 8

An homomorphic filtering and expectation maximization approach for the point spread function estimation in ultrasound imagingS.Benameur,EiffelMedtech,Inc.(Canada);M.Mignotte,Univ.deMontréal(Canada);F.Lavoie,EiffelMedtech,Inc.(Canada)

Inmodernultrasoundimagingsystems,thespatialresolutionisseverelylimitedduetotheeffectsofboththefiniteapertureandoverallbandwidthofultrasoundtransducersandthenon-negligiblewidthofthetransmittedultrasoundbeams.Thislowspatialresolution

remainsthemajorlimitingfactorintheclinicalusefulnessofmedicalultrasoundimages.Inordertorecoverclinicallyimportantimagedetails,whichareoftenmaskedduetothisresolutionlimitation,animagerestorationprocedureshouldbeapplied.Tothisend,anestimationofthePointSpreadFunction(PSF)oftheultrasoundimagingsystemisrequired.Thispaperintroducesanovel,original,reliable,andfastMaximumLikelihood(ML)approachforrecoveringthePSFofanultrasoundimagingsystem.ThisnewPSFestimationmethodassumesasaconstraintthatthePSFisofknownparametricform.Underthisconstraint,theparametervaluesofitsassociatedModulationTransferFunction(MTF)arethenefficientlyestimatedusingahomomorphicfilter,adenoisingstep,andanexpectation-maximization(EM)basedclusteringalgorithm.GiventhisPSFestimate,adeconvolutioncanthenbeefficientlyusedinordertoimprovethespatialresolutionofanultrasoundimageandtoobtainanestimate(independentofthepropertiesoftheimagingsystem)ofthetruetissuereflectivityfunction.Theexperimentsreportedinthispaperdemonstratetheefficiencyandillustrateallthepotentialofthisnewestimationandblinddeconvolutionapproach.

8295A-35, Poster Session

Intensity-constrained, flat-kernel filter for local dark feature suppression: application to removal of hair artifacts in dermatoscopic imagesA.A.Gutenev,RetiariusPtyLtd.(Australia)

TheIntensityConstrainedFlatKernelFiltering(ICFK)schemeisadualdomain(spatialandintensity)nonlinearframeworkwhichhasbeenshowntogenerateusefulfiltersforimageprocessing.ThispaperproposesanewfilterdevelopedwithintheICFKframework.Althoughlocalinnaturethefilterisdesignedtosuppresslargescalespatialfeatureswithintheimage.Asineveryotherfilterderivedwithintheschemethesuppressedfeaturesaredefinedbytwoparameters:sizeofthekernelandintensityrange.Thefilter,asingle-stepprocedure,isappliedtoremovalofhairartifactsinskinlesionepiluminescencemicroscopyimages,thetaskessentialinassistinginautomatedsegmentationofimagedareaintolesionandsurroundingskin.Resultsoftheexperimentson400dermatoscopicimagesoflesionswithhairindicatesuitabilityofthemethodasanaidinlesionsegmentationbysuppressinghairorvascularfeaturesnearthelesionborders.


New decision support tool for acute lymphoblastic leukemia classificationM.Madhukar,S.S.Agaian,A.Chronopoulos,TheUniv.ofTexasatSanAntonio(UnitedStates)

Theanalysisofbloodcellsinmicroscopeimagecanprovideusefulinformationconcerningthepatient’shealth.Leukemiaisthecancerofthebloodwhichdevelopsinthesoftspongycenteroflongbonescalledbonemarrow.Therearefourmajordifferentformsortypesofleukemia(Acutelymphoblasticleukemia(ALL),Acutemyelogenousleukemia(AML),Chroniclymphocyticleukemia(CLL),andChronicmyelogenousleukemia(CML)[1].AcuteLymphoblasticLeukemiais

characterizedbyaccumulationofmalignantcellscalledlymphoblastsinthebonemarrow.However,theidentificationofALLisusuallydifficultduetothevarietyoffeaturesandtheoftenunclearimagesmightresultinwrongdiagnosis.Inthispaper,wedevelopanewdecisionsupporttoolforALLclassification.Thedevelopedsystemwilleffectivelyextractthecellproperties.Thebloodimagesaresubjectedtoseriesofpre-processingstepsthateffectivelybringouttheleukocytesfromotherbloodcomponents.SegmentationisdoneusingK-meansclusteringforevaluatingshapeandtexturepropertiesofthenucleioftheleukocytes.TheproposedideaeffectivelybringsoutthenucleifeaturesoftheleukocytesandtherebypresentsaneffectivemeansofALLclassification.


Sharpness metric for no-reference image visual quality assessmentV.V.Lukin,N.N.Ponomarenko,O.Eremeev,NationalAerospaceUniv.(Ukraine);K.O.Egiazarian,J.T.Astola,TampereUniv.ofTechnology(Finland)

Thispaperproposesametricthatevaluatesthesharpnessofacolorimage.Themetricalsotakesintoaccountthepresenceofblockingartifactsandtheeffectofmacrophotography.AgivenimageistransformedfromRGBcolorspacetocolorspaceYCbCr.Next,thecontributionofblockingeffecttoenergyofhigh-frequencycomponentsoftheimageisassessedseparatelyforeachcolorcomponent.Ahigh-frequencyenergyoftheimageisestimatedinwaveletdomainandcorrectedbythepreviouslyobtainedestimateofblockingeffect.And,finally,anestimateofimagesharpnessisformedforallcolorcomponentsasweightedsumofenergiesforcolorcomponents.Theeffectivenessoftheproposedmetrichasbeenanalyzedusingthetestimagedatabasecontaining500colorimages.Thecorrespondencebetweentheproposedmetricandmeanopinionscoresforthedatabaseisevaluated.SpearmancorrelationfactorbetweentheproposedmetricsandtheMOSisequalto0.71.Thisisconsiderablyhigherthanforotherwell-knownmetrics.


A new system of computer-aided diagnosis of skin lesionsI.A.Sanchez,S.S.Agaian,TheUniv.ofTexasatSanAntonio(UnitedStates)

Noabstractavailable


Image denoising using a combined criterionE.Semenishchev,V.I.Marchuk,South-RussianStateUniv.ofEconomicsandService(RussianFederation);K.O.Egiazarian,TampereUniv.ofTechnology(Finland);V.Voronin,South-RussianStateUniv.ofEconomicsandService(RussianFederation)

Anewimagedenoisingmethodisproposedinthispaper.Weareconsideringanoptimizationproblemwithalinearobjectivefunctionbasedontwocriteria,namely,L2normandthefirstordersquaredifference;thensolvingitandprovingaconvergenceanduniquenessofsolutionsofthedevelopedmethod.Thismethodisaparametricone,byachoiceoftheparametersonecantuneagivencriteriaoftheobjectivefunction.Thedenoisingalgorithmconsistsofthefollowingsteps:1)multipledenoisingestimatesarefoundonlocalareasoftheimage;2)imageedgesaredetermined;3)parametersofthemethodarefixedanddenoisedestimatesofthelocalareaarefound;4)localwindowismovedtothenextposition(localwindowsareoverlapping)inordertoproducethefinalestimate.

Aproperchoiceofparametersoftheintroducedmethodisdiscussed.

Acomparativeanalysisofanewdenoisingmethodwithexistedonesisperformedonasetoftestimages.

Conference 8295A



Non-cooperative stationary ground targets detection based on IRSTD.Yang,Y.Qin,X.Zhang,X.Wei,H.Wang,NationalUniv.ofDefenseTechnology(China)

TherearemanydifficultiesindetectionforIRSTasfollows:(1)complexbackgroundandlackoftarget’smotioninformation;(2)lackofpriorinformation,thenon-cooperativetargetisdifficulttomatch;(3)thereisbigspanoftheimagingscenedistance,andtheimagepixelgraycannotreflecttheradiationofsceneandtargetaccurately.Toovercomethedifficulties,thepaperpresentedanalgorithmofnon-cooperativestationarygroundtargetdetectionbasedonfeatureset.Firstly,thespatialstationmodelofimagingscenewasfoundedtocountthescenedistance,andthenwepretreattheIRimagetogettheapparenttemperatureimagewhichcanreflecttheradiationofthescenetruly.Secondly,thefeaturesetwasextractedintheslidingwindowofthepixelneighbourhood,andthehypothesistestwasusedtodetectthestationarytargets.Lastlytheslidingwindowwasadjustedadaptivelyaccordingtothetargetsize.Simulationresultsdemonstratethattheproposedalgorithmhasbetterdetectionperformancetothegroundnon-cooperativestationarytargets.


Motion-compensated spatial-temporal filtering for noisy color filter array sequenceM.S.Lee,YonseiUniv.(Korea,Republicof)

Spatial-temporalfiltershavebeenwidelyusedinvideodenoisingmodule.Thefiltersarecommonlydesignedformonochromaticimage.However,mostdigitalvideocamerasuseacolorfilterarray(CFA)togetcolorsequence.Weproposearecursivespatial-temporalfilterusingmotionestimation(ME)andmotioncompensatedprediction(MCP)forCFAsequence.IntheproposedMEmethod,weobtaincandidatemotionvectorsfromCFAsequencethroughhypotheticalluminancemaps.Withtheestimatedmotionvectors,theaccurateMCPisobtainedfromCFAsequencebyweightedaveraging,whichisdeterminedbyspatial-temporalLMMSE.Then,thetemporalfiltercombinesestimatedMCPandcurrentpixel.Thisprocessiscontrolledbythemotiondetectionvalue.Aftertemporalfiltering,thespatialfilterisappliedtothefilteredcurrentframeasapost-processing.Experimentalresultsshowthattheproposedmethodachievesgooddenoisingperformancewithoutmotionblurringandacquireshighvisualquality.


Application of 1D FIR filter methods to 3D polygonal meshesW.S.Ward,TheUniv.ofTexasatSanAntonio(UnitedStates)

Thispaperdiscussesaprocedureoffilteringathreedimensionalpolygonalmeshbyutilizingthebasicmethodsoffiniteimpulseresponse(FIR),onedimensionalfiltering.Mostthreedimensionalfiltersarelimitedtoeithersmoothinganobjectorenhancingitsfeatures.Bytakingamoregeneralapproach,amethodwasdevelopedthatcanbeusedtoeithersmoothorenhanceobjects.Thismethodallowsforeithermultipleiterationsofasimplefilter,orasinglepassofamuchmorecomplexfilter.Thisprocedureusestheabsolutevertexlocations,asdefinedinthe.plyfileformat,asthebasisforfiltering.Eachvertexisfilteredindependently,usingonlytheoriginalunfilteredobject’svertices,withtheresultantverticesbeingstoredseparately.Inordertousethestandardmethodsassociatedwithonedimensionalfiltering,theverticesmustfirstbelinearizedinsomefashion.Toaccomplishthis,thevertextobefilteredisdefinedasthe“origin”or“layer1”.The

verticesconnectedtothis“layer”bypolygons,extendingoutward,aredefinedas“layer2”,andsoonandsoforth,untiltherequisitenumberof“layers”arecalculated.ThenumberofrequisitelayersisdefinedbythenumberofFIRfiltercoefficientsbeingused.Thepointsineach“layer”arethenaveragedandarethentreatedasalinearsetofrealnumbers,whicharethenfilteredusingtheFIRcoefficients.


An automatic approach for 3D registration of CT scansY.Hu,E.Saber,S.Dianat,S.R.Vantaram,RochesterInstituteofTechnology(UnitedStates);V.Abhyankar,DataPhysicsResearch,Inc.(UnitedStates)

Thispaperpresentsanautomatic3DregistrationalgorithmwhichiscapableofaligningfreshlyacquiredvolumetricCTscansfromanarbitrarypatientcalleda‘Study’toareferenceoriginalknownasan‘Atlas’thatservesasgroundtruthcreatedaprioribyexpertspersonnel.Theproposedalgorithmconsistsofthreesteps:(i)locatingaregionofinterestsuitableforregistration,(ii)3Dinterpolationtobringthestudyandatlastoacommonsamplinggrid,and(iii)aligningthetwovolumesinaregistrationstep.ThefirststepremovesalltheirrelevantobjectsandlocatesthebodyregionintheCTvolumebyaseriesofmorphologicaloperations.Thesecondstepappliesa3DinterpolationschemetotheretrieveddatasetandAtlassothattheyareidenticallysampledinx,yandzdirections.ThelaststepregisterstheretrieveddatasetstothecorrespondingslicesintheAtlasusingasimilaritymeasurementtechniquethatisbasedon3Dcorrelation.Theproposedtechniquewastestedonseveraldatasetswithenhancedperformance.Itcanbeusedinanautomaticandunsupervised3DsegmentationofhumanbodyorgansfromCTscans.


Boundary handling mechanism for lifting-based spatial adaptation of filter banksD.Jayachandra,A.Makur,NanyangTechnologicalUniv.(Singapore)

Time/spacevaryingfilterbanks(FBs)areprovedtobeusefulinbuildingsignaladaptivetransforms.LiftingfactorizationofFBsallowstospatiallyadaptbetweenarbitraryFBs,avoidingtheneedtodesignborderFBstocompleteperfectreconstruction(PR)duringthetransition.However,liftingbasedswitchingbetweenarbitrarilydesignedFBsinducesspurioustransientsintotheresultingsubbandsduringthetransition.Inthispaperweproposeaboundaryhandlingmechanismthatmaintainsgoodfrequencyresponseandeliminatesthetransientsduringthetransition.WesuccessfullyshowspatialadaptationbetweenJPEG20009/7and5/3FBstoreducetheringingartifactsinimages.


A simple and efficient algorithm for connected-component labeling in color imagesM.E.Celebi,LouisianaStateUniv.Shreveport(UnitedStates)

Connectedcomponentlabelingisafundamentaloperationinbinaryimageprocessing.Aplethoraofalgorithmshavebeenproposedforthislow-leveloperationwiththeearlyonesdatingbacktothe1960s.However,veryfewofthesealgorithmsweredesignedtohandlecolorimages.Inthispaper,wepresentasimplealgorithmforlabelingconnectedcomponentsincolorimagesusinganapproximatelylinear-timeseedfillalgorithm.Experimentsonalargesetofphotographicandsyntheticimagesdemonstratethattheproposedalgorithmprovidesfastandaccuratelabelingwithoutrequiringexcessivestackspace.

Conference 8295A



An adaptive and deterministic method for initializing the Lloyd-Max algorithmJ.Vicory,M.E.Celebi,LouisianaStateUniv.Shreveport(UnitedStates)

Gray-levelquantization(reduction)isanimportantoperationinimageprocessingandanalysis.TheLloyd-Maxalgorithm(LMA)isaclassicscalarquantizationalgorithmthatcanbeusedforgray-levelreductionwithminimalmeansquareddistortion.However,thealgorithmisknowntobeverysensitivetothechoiceofinitialcenters.Inthispaper,weintroduceanadaptiveanddeterministicalgorithmtoinitializetheLMAforgray-levelquantization.Experimentsonadiversesetofpubliclyavailabletestimagesdemonstratethatthepresentedmethodoutperformsthecommonlyuseduniforminitializationmethod.


Multi-resolution analysis for region of interest extraction in thermographic, nondestructive evaluationB.OrtizJaramillo,H.A.FandinoToro,Univ.NacionaldeColombia(Colombia);H.D.BenitezRestrepo,PontificiaUniv.Javeriana,Cali(Colombia);S.A.OrjuelaVargas,Univ.Gent(Belgium);G.CastellanosDominguez,Univ.NacionaldeColombia(Colombia);W.Philips,Univ.Gent(Belgium)

Theultimategoalofathermographicinspectionistoautomaticallyanalyzeimagesprovidingapassorfaildiagnostictotheoperator.Primarydifficultiesinmakingusefulinterpretationsofathermalimageisthepresenceofnon-uniformheatingcausedbytheunevenexcitationofthesurface.Moreover,non-uniformheatingishardtoremovebecauseisaninherenttimevariantaberration.Insummary,regiongrowingapproachforimagesegmentationinInfraredNondestructiveTesting(INDT)isaffectedbyROIlowcontrastandnon-uniformheating.

Inthispaper,amethodologyforROIextractioninINDTusingmulti-resolutionanalysisisproposed,whichitisrobusttoROIlowcontrastandnon-uniformheating.Theformermethodologyincludeslocalcorrelation,Gaussianscaleanalysisandclusteringanalysis.InthismethodologylocalcorrelationbetweenimageandGaussianwindowprovideinterestpointsrelatedtoROIs.WeuseaGaussianwindowbecausethermalbehavioriswellmodeledbyGaussiansmoothcontours.Besides,Gaussianscaleisusedtoanalyzedetailsintheimageusingmulti-resolutionanalysisavoidinglowcontrast,non-uniformheatingandselectionoftheGaussianwindowsize.Finally,clusteringanalysisisusedtoprovideagoodcombinationbetweenspatialinformationandgraylevelvaluesobtainedbyGaussianscaledecomposition.


Estimation of deformations in ultrasound images using dynamic programmingS.S.Furuie,F.M.Cardoso,EscolaPolitécnicadaUniv.deSãoPaulo(Brazil)

Dynamicmedicalimagesmayprovidevaluableinformationsuchascontractionrate,deformationandelasticity.Forthispurpose,itisfundamentaltoestimatethedisplacementofeachpointofinterest.However,inultrasoundthistaskishamperedbyspecklenoise.Theobjectiveofthisworkistheestimationofstructuredeformationandcontractionusingrobusttrackingofasetofrepresentativepointsinasequenceofultrasoundimages.Theproposedapproachisbasedondiscreteoptimizationofjointdisplacementestimation,wherethecriteriainvolvejointintensityandmorphologysimilarity.Weinvestigatedthefollowingmodifications:a)initializationofthegraphbymaximizationofBhattacharayyacoefficient,whichisinvarianttotranslationandrotation;b)incorporationofcostduetoangledifferencesbetweenconsecutivepointsinthemapping;c)

useofspeckleindexinsteadofplainintensity.Wehaveevaluatedinrealisticnumericalphantomswithspecklenoiseandcomparedwithtraditionalapproaches.Tenpointswereconsideredinthephantomandweappliedseveralaffinetransformationstogeneratethedeformedimages.Theaveragedisplacementerrorhasdecreasedinalmostallcases.Inconclusion,wehaveshownthattheproposedapproachhasimprovedconsistentlytheestimationofdisplacements.


Combining skin texture and facial structure for face identificationR.E.Manoni,PhotonResearchAssociates,Inc.(UnitedStates);R.L.Canosa,RochesterInstituteofTechnology(UnitedStates)

Thisresearchintroducesanewtechniqueforfacialidentificationinlowresolutionimagesthatcombinesfacialstructurewithskintexturetoaccommodatechangesinlightingandheadpose.Themodelcombinesskintexturewithfacialstructure.Theaccuracyofthismodelistestedonlowresolutionimagescontainingdifferentilluminationandheadposes.Skintexturecapturesthevisualdetailsofskin,mappinguniquetexturestoamathematicalspace.Facialstructureusesmulti-scaleedgestodefinegrossandfinestructuresoftheface,resultinginasetoffeaturesthatshowtheglobaloutlineandlocalstructureofthefacealongwiththefinestructureofallfacialcomponents.Thesefeatureimagesarecombinedtogethertocreateauniquerepresentationforeachperson,whichisthenusedforidentification.Thefacialstructuremodelusedalonehadalowoverallaccuracy,yetshowedsomepromiseofbeinginvarianttopose.Theskintexturemodelusedalonehadahigheraverageaccuracyandshowedsomeinvariancetoillumination.Thecombinedmodelhadanaverageaccuracyhigherthanthetwomodelsusedalone.Mostpromising,thecombinedmodelretainedthebestfeaturesoftheindividualmodelswithinvariancetoilluminationandmostposeangles.


Development of a human vision simulation camera and its applicationH.Okumura,M.Fukusaki,S.Takubo,K.Arai,SagaUniv.(Japan)

Humaneyehasalotofphotoreceptorcellsinitsretina.Humanphotoreceptorcellsconsistof“cone”typeand“rod”type.

Conecellsthatareresponsibleforcolorvisionareconcentratedatthemaculaandrodcellsareconcentratedarroundthemaculaandareusedinperipheralvision.Moresensitivethanconecells,rodcellsarealmostentirelyresponsibleforscotopicvision.Purkinjeeffectformesopicandscotopicvisionandadaptationarecausedbyunevendistributionandsensitivityofphotoreceptorcells.Inthisstudy,HuVisCam,ahumanvisionsimulationcamera,thatcansimulatenotonlyPurkinjeeffectformesopicandscotopicvisionbutalsodarkandlightadaptation,abnormalmiosisandabnormalmydriasiscausedbytheinfluenceofmydriasismedicineornerveagentandspecificcolorperceptionisdeveloped.Inthisarticle,thesystemanditsmethodaredescribed.TheappliedexampletoonboardvisibilitysupportcamerawhichcanenhancetheobjectwhichvisibilitydecreasedbyPurkinjeeffectunderthemesopicvisionisalsodemonstrated.


Reconstruction from divergent ray projectionsC.S.Sastry,InternationalInstituteofInformationTechnology(India);S.Singh,SiemensInformationSystemsLtd.(India)

Despitemajoradvancesinx-raysources,detectorarrays,gantrymechanicaldesignandspeciallycomputerperformances,butcomputedtomography(CT)enjoysthefilteredbackprojection(FBP)algorithmastheirfirstchoicefortheCTimagereconstructioninthecommercialscanners.Overtheyears,alotoffundamentalworkhas

Conference 8295A


beendoneintheareaoffindingthesophisticatedsolutionsfortheinverseproblemsusingdifferentkindsofoptimizationtechniques.Recentlastfewyearshavereallybeendominatedbythecompressivesensingtechniquesand/orsparsereconstructiontechniques.Stillthereisalongwaytogofortranslatingthesenewlydevelopedalgorithmsintheclinicalenvironment.Thereasonsarenotobviousandseldomdiscussed.KnowingthefactthatfilteredbackprojectionisoneofthemostpopularCTimagereconstructionalgorithms,itallowsustodoresearchworktoimprovethedifferenterrorestimatesatdifferentstepspreformedinfilteredbackprojection.

Inthispaper,wepresentabackprojectionformulaforthereconstructionofdivergentbeamtomographywithuniqueconvolutionstructure.Usingsuchaproposedapproximateconvolutionstructure,theapproximationerrormathematicallyjustifiesthelowreconstructionerrorforasuitablechoiceofparameters.

Inordertominimizetheexposuretimeandpossibledistortionsduetothemotionofthepatient,fanbeammethodofcollectionofdataisused.Rebinningtransformationisusedtoconnectfanbeamdataintoparallelbeamdatasothatthewelldevelopedmethodofimagereconstructionforparallelbeamgeometrycanbeused.Thecomputationalerrorsinvolvedinthenumericalprocessofrebinning,somedegradationofimageisinevitable.However,todatealittleworkhasbeendoneforthereconstructionoffanbeamtomography.Therehasbeensomerecentresultsonwaveletreconstructionofdivergentbeamtomography.Inthispaper,weproposeaconvolutionalgorithmforthereconstructionofdivergentbeamtomography,whichissimplerthanwaveletmethodsandbetterapproximateerrormethods.Astheformulaisapproximateinnature,weproveanestimatefortheerrorassociatedwiththeformula.Usingtheestimate,wededuceconditionthatminimizesapproximationerror.


Fusing electro-optic and infrared signals for high-resolution night imagesX.Huang,StevensInstituteofTechnology(UnitedStates);R.Netravali,ColumbiaUniv.(UnitedStates);H.Man,V.B.Lawrence,StevensInstituteofTechnology(UnitedStates)

Electro-opticimageexhibitsthepropertiesofhighresolutionandlownoiselevel.Therefore,weproposeanovelframeworkofinfraredimageenhancementbasedontheinformationofelectro-opticimageforthefirsttime.Ononehand,weadopttheRussellC.Hardieetal.developedtheoreticalpointspreadfunction(PSF)oftheinfraredimagesystem,whichiscontributedbythemodulationtransferfunction(MTF)ofauniformdetectorarrayandtheincoherentopticaltransferfunction(OTF)ofdiffraction-limitedoptics.TheeffectoftheintegrationoflightintensityoverthespanofthedetectorscanbemodeledasalinearconvolutionoperationwithaPSFdeterminedbythegeometryofasingledetector.TheunknownparametersinthisPSFincludethedetectorsquaredetectorsizeandthesystemcutofffrequency.Ontheotherhand,weregardaregisteredimagepairastheelectro-opticandinfraredoutputimages.Additionally,thedesignedinversefilterandWienerfilterareadoptedtoreconstructtheimagewithoutandwithnoise,respectively.Theinfraredimageenhancementisachievedbyestablishinganoptimizationmodel,whichaimsatobtainingtheoptimalvaluesofvariableparametersinthetheoreticalPSF(e.g.,systemcutofffrequency).Thismodelsubjectstominimizetheerrorbetweenthereconstructedinfraredimageandthetransformedelectro-opticimage.Simulationresultscomparethedifferencebetweenthereconstructedinfraredimagewithandwithouttheinformationofelectro-opticimage,respectively.


Texture and color descriptors as a tool for context-aware patch-based image inpainting

T.Ruzic,A.Pizurica,W.Philips,Univ.Gent(Belgium)

Imageinpaintingisanimageprocessingtaskofllinginmissing,damagedorunwantedpartsoftheimage.State-of-the-artresultsareobtainedwithpatch-basedmethodsthatllinthemissingregionpatch-by-patchbysearchingforsimilarpatchesintheknownregionandplacingthematcorrespondinglocations.Inthispaper,weintroduceacontext-awarepatch-basedinpaintingmethod,wherethecontextisrepresentedbytextureandcolorfeaturesofablocksurroundingthepatchtobelledin.Weusethiscontexttorecognizeotherblocksintheimagethathavesimilarfeaturesandthenweconstrainthesearchforsimilarpatcheswithinthem.Suchanapproachguidesthesearchprocesstowardslessambiguousmatchingcandidates,whilealsospeedingupthealgorithm.Resultsdemonstratethequalitativeimprovementovertherelatedtechniquewithlessvisualartefacts,butalsowithdecreaseincomputationtime.

8295A-29, Session 9

Hybrid gesture recognition system for short-range useA.Minagawa,FujitsuLabs.(Japan);W.Fan,FujitsuResearchandDevelopmentCenterCo.,Ltd.(China);Y.Katsuyama,H.Takebe,N.Ozawa,Y.Hotta,FujitsuLabs.(Japan);J.Sun,FujitsuResearchandDevelopmentCenterCo.,Ltd.(China)

Inrecentyears,variousgesturerecognitionsystemshavebeenstudiedforuseintelevisionandvideogames.Insuchsystems,motionareasrangingfrom1to3metersdeephavebeenevaluated.However,withtheburgeoningpopularityofsmallmobiledisplays,gesturerecognitionsystemscapableofoperatingatmuchshorterrangeshavebecomenecessary.Theproblemsrelatedtosuchsystemsareexacerbatedbythefactthatthecamera’sfieldofviewisunknowntotheuserduringoperation,whichimposesseveralrestrictionsonhis/heractions.

Toovercometherestrictionsgeneratedfromsuchmobilecameradevices,andtocreateamoreflexiblegesturerecognitioninterface,weproposeahybridhandgesturesystem,inwhichtwotypesofgesturerecognitionmodulesarepreparedandwithwhichthemostappropriaterecognitionmoduleisselectedbyadedicatedswitchingmodule.Thetworecognitionmodulesofthissystemareshapeanalysisusingaboostingapproach(detection-basedapproach)andmotionanalysisusingimageframedifferences(motion-basedapproach).

Weevaluatedthissystemusingsampleusersandclassifiedtheresultingerrorsintothreecategories:errorsthatdependontherecognitionmodule,errorscausedbyincorrectmoduleidentification,anderrorsresultingfromuseractions.Inthispaper,weshowtheresultsofourinvestigationsandexplaintheproblemsrelatedtoshort-rangegesturerecognitionsystems.

8295A-30, Session 9

Tracking white road line by particle filter from the video sequence acquired by the camera attached to a walking human bodyS.Takahashi,J.Ohya,WasedaUniv.(Japan)

Thispaperproposesamethodfortrackingandrecognizingthewhitelinemarkedinthesurfaceoftheroadfromthevideosequenceacquiredbythecameraattachedtoawalkinghuman,towardstheactualizationofanautomaticnavigationsystemforthevisuallyhandicapped.Ourproposedmethodconsistsoftwomainmodules:(1)ParticleFilterbasedmodulefortrackingthewhiteline,and(2)CLAFICMethodbasedmoduleforclassifyingwhetherthetrackedobjectisthewhiteline.In(1),eachparticleisarectangle,andisdescribedbyitscentroid’scoordinatesanditsorientation.Thelikelihoodofaparticleiscomputedbasedonthenumberofwhitepixelsintherectangle.In

Conference 8295A


(2),inordertoobtaintheranges(tobeusedfortherecognition)forthewhiteline’slengthandwidth,PrincipalComponentAnalysis(PCA)isappliedtothecovariancematrixobtainedfromvalidsampleparticles(centroid’scoordinatesandorientation).Ateachframe,PCAisappliedtothecovariancematrixconstructedfromparticleswithhighlikelihood,andiftheobtainedlengthandwidtharewithintheabove-mentionedranges,itisrecognizedasthewhiteline.Experimentalresultsusingrealvideosequencesshowthevalidityoftheproposedmethod.

8295A-31, Session 9

Driver/passenger discrimination for the interaction with the dual-view touch screen integrated to the automobile centre consolE.Herrmann,A.Makrushin,J.Dittmann,Otto-von-Guericke-Univ.Magdeburg(Germany);C.Vielhauer,FachhochschuleBrandenburg(Germany)

Inanattempttofurtherdevelopandevaluatetheopticalrecognitionsystemsfordistinguishingbetweendriverandfront-seatpassengerduringtheirinteractionswithdual-viewtouchscreenintegratedtotheautomobilecentreconsol,thisworkfocusesontheenhancementofbothimageprocessingalgorithmsandexperimentalenvironment.Inadditiontothemotionbasedforearmandhandsegmentationandthetexturebasedarmdirectionanalysis,theboostingclassifierswithhaar-likefeatureshavebeenengagedforthelearningofdriver’sandpassenger’shandpatterns.Theuserdiscriminationsystemwascompletelyreproducedinalaboratory,includingpassengercompartmentwithgenuinedashboard,touchscreen,cameraandinfraredlamps,sothatdifferentilluminationconditionscouldbemodeled.Thenewacquisitionsystemallowsautomaticandunambiguousregistrationofalltouchscreeninteractionsandtheirsynchronizationwiththevideostream.Thisresultsincredibleevaluationoftheimageprocessingroutines.Theadjustmentofthecamerapositionandtheactiveinfraredilluminationmadeitpossibletoreducetherecognitionerrorratesandtoachievesuperiordiscriminationperformancecomparingtopreviousworks.Inthefurtherworktheprototypicuserdiscriminationsystemwillbemigratedfromthestandingcarsimulatortoarealcarandtheexperimentswillbeprovidedformovingcarscenariosaswell.

8295A-32, Session 10

A linear filter design technique for equalizing document scannersA.H.Eid,LexmarkInternational,Inc.(UnitedStates)

Inthispaper,weproposeanewtechniquetoautomaticallyrestorethesharpnessofblurreddocumentsbyequalizingthefrequencyresponseofgivenscannersusinglinearfilters.

Tomeasuretheblurcharacteristicsofascanningdevice,wemeasureitsbothhorizontalandverticalSpatialFrequencyResponse(SFR).StartingfromthemeasuredSFRofthescanningdevice,ourgoalistodesignanequalizingfiltersothatthecombinedSFRoftheequalizingfilterandthescannerresemblesaperfectSFR.

Thedesired2DfrequencyresponseofthefilteriscomputedusingbilinearinterpolationofthehorizontalandverticalresponsesderivedfromthecorrespondingSFRsofthescanner.Thefilterdesigntechniqueistwosteps.First,alinearsystemofequationsisconstructedusingtheunknownfiltercoefficientsandthedesiredfilter2Dfrequencyresponse.Thelinearleastsquaremethodisusedtosolvethelinearsystemofequations.Thesecondstepofthefilterdesignusesanon-linearoptimizationtechniquetorefinetheresultsofthefirststep.

Ourexperimentalresultsshowthatthisautomatedprocesscanbeappliedtodifferentdocumentscanningdevicestoequalizetheirspatialfrequencyresponseresultinginconsistentoutputsharpnesslevels.


Application of spatial contrast techniques

on satellite imagery for cloud shape differentiationJ.R.Dim,H.Murakami,JapanAerospaceExplorationAgency(Japan)

Pixels’edgescanyieldusefulinformationonphysicalpropertiesofobjectsfeaturedonsatelliteimages.Thesepropertiescanbederivedthroughtheuseoftheimageryspatialcontrasttechniques.Todifferentiatevariouscloudtypesbasedontheirshapes,oneofthesetechniquesisappliedonthermalimagesfromapolarorbitingsatellite,theNationalOceanicandAtmosphericAdministration/AdvancedVery-High-ResolutionRadiometer(NOAA-AVHRR).Edgedetectorsgradientsextractedfromdailyglobalcloudtemperatureimagesofthissatelliteandthespatialrelationshipbetweenthesegradientspermitthedistinctionofninemajorcloudshapesdistributedalongthreecloudpressurelevels(high,middleandlow).Thecloudshapedifferentiationmethodutilizedisahistogram-basedgradientschemedescribingtheoccurrenceofdifferentgradients’levels(high,middleandlow)ineachblockofpixels.Adetailedanalysisofthedistributionofthecloudshapesobtainedisconducted,andthefrequencyofeachcloudshapeisevaluatedwithanothercloudclassificationmethod(basedoncloudopticalproperties)forvalidationpurposes.Finally,implicationsoftheresultsobtained,ontheestimationoftheimpactofcloudshapesvariationsontherecentclimatearediscussed.


A multi-step system for screening and localization of hard exudates in retinal imagesA.S.Bopardikar,V.Bhola,R.B.S.,R.Narayanan,SamsungElectronics,IndiaSoftwareOperationsLtd.(India)

ThenumberofpeoplebeingaffectedbyDiabetesmellitusworldwideisincreasingatanalarmingrate.Monitoringofthediabeticconditionanditseffectsonthehumanbodyarethereforeofgreatimportance.Ofparticularinterestisdiabeticretinopathy(DR)whichisaresultofprolonged,uncheckeddiabetesandaffectsthevisualsystem.DRisaleadingcauseofblindnessthroughouttheworld.Atanypointoftime25-44%ofpeoplewithdiabetesareafflictedbyDR.AutomationofthescreeningandmonitoringprocessforDRisthereforeessentialforefficientutilizationofhealthcareresourcesandoptimizingtreatmentoftheaffectedindividuals.Suchautomationwoulduseretinalimagesanddetectthepresenceofspecificartifactssuchashardexudates,hemorrhagesandsoftexudates(thatmayappearintheimage)togaugetheseverityofDR.Inthispaper,wefocusonthedetectionofhardexudates.Weproposeatwostepsystemthatconsistsofascreeningstepthatclassifiesretinalimagesasnormalorabnormalbasedonthepresenceofhardexudatesandadetectionstagethatlocalizestheseartifactsinanabnormalretinalimage.Theproposedscreeningstepautomaticallydetectsthepresenceofhardexudateswithahighsensitivityandpositivepredictivevalue(PPV).Thedetection/localizationstepusesak-meansbasedclusteringapproachtolocalizehardexudatesintheretinalimage.Suitablefeaturevectorsarechosenbasedontheirabilitytoisolatehardexudateswhileminimizingfalsedetections.Thealgorithmwastestedonabenchmarkdataset(DIARETDB1)andwasseentoprovideasuperiorperformancecomparedtoexistingmethods.Thetwo-stepprocessdescribedinthispapercanbeembeddedinatele-opthamologysystemtoaidwithspeedydetectionanddiagnosisoftheseverityofDR.

Conference 8295A


Conference 8295B: Parallel Processing for Imaging Applications IIMonday23January2012

Part of Proceedings of SPIE Vol. 8295B Parallel Processing for Imaging Applications II

8295B-52, Session 11

GPGPU-based surface inspection from structured white lightM.BordalloLopez,Univ.ofOulu(Finland);K.Niemelä,VTTTechnicalResearchCtr.ofFinland(Finland);O.J.Silvén,Univ.ofOulu(Finland)

Automaticsurfaceinspectionhasbeenusedintheindustrytoreliablydetectallkindsofsurfacedefectsandtomeasuretheoverallqualityofaproducedpiece.Structuredlightsystems(SLS)arebasedonthereconstructionofthe3Dinformationofaselectedareabyprojectingseveralphase-shiftedsinusoidalpatternsontoasurface.

Duetothehighspeedofproductionlines,surfaceinspectionsystemsrequireextremelyfastimagingmethodsandlotsofcomputationalpower.Thecostofsuchsystemscaneasilybecomeconsiderable.TheuseofstandardPCsandGraphicsProcessingUnits(GPUs)fordataprocessingtasksfacilitatestheconstructionofcost-effectivesystems.

WepresentaparallelimplementationoftherequiredalgorithmswritteninCwithCUDAextensions.Inourcontribution,wedescribethechallengesofthedesignonaGPU,comparedwithatraditionalCPUimplementation.Weprovideaqualitativeevaluationoftheresultsandacomparisonofthealgorithmspeedperformanceonseveralplatforms.

Thesystemisabletocomputetwomegapixelsheightmapswith100micrometersspatialresolutionsmallerthan100micrometersinlessthan200msonamid-budgetlaptop.OurGPUimplementationrunsabouttentimesfasterthanourpreviousCcodeimplementation.


IMPAIR-GPU: massively parallel deconvolution algorithm for GPUsM.Sherry,A.Shearer,NationalUniv.ofIreland,Galway(Ireland)

MedicalimageprocessinghasbeenfundamentallyrestrictedbyCPUpowerorbythehighcostsoflargeparallelsystems.Inthepastwehavedevelopedasuiteofsoftwarewhichdemonstratedthestandarddeconvolutiontechniques,suchasRichard-Lucydeconvolutionwhencombinedwithawaveletbaseddenoisingcanproducesignificant

enhancementsoflargeradiographicimages.InthispaperwereviewourpreviousapproachesandlimitationsandthenmoveontodescribingtheuseofGPUstoimplementthesealgorithmsforanumberofdifferentmedicalimagingmodalities.InparticularweconcentrateuponourdemonstrationcodedevelopedonanNVIDIATeslaplatformandwe

compareitsperformancewithaconventionalMPIapproach.Furthermorewewillalsoreportonthefirstresultsofasemanticapproachtodeterminingthecoefficientsofthedeconvolutionandde-noisingprocessestoenhancespecificimagefeatures.Ourreal-timeimageprocessingrequirementcomesfromtheapplicationarea-radio-therapyportalimaging.


Parallel processing architectures for H.264 deblocking filter on multicore platformsD.P.Prasad,S.Sonachalam,M.K.Kunchamwar,N.R.Gunupudi,ParallelPrisms(UnitedStates)

Massivelyparallelcomputing(multi-core)chipsofferoutstandingnewsolutionsthatsatisfytheincreasingdemandforhighresolutionand

highqualityvideocompressiontechnologiessuchasH.264.Suchsolutionsnotonlyprovideexceptionalqualitybutalsoefficiency,lowpower,andlowlatency,previouslyunattainableinsoftwarebaseddesigns.WhilecustomhardwareandApplicationSpecificIntegratedCircuit(ASIC)technologiesmayachievelow-latency,lowpower,andreal-timeperformanceinsomeconsumerdevices,manyapplicationsrequireaflexibleandscalablesoftware-definedsolution.

ThedeblockingfilterinanH.264encoder/decoderposesdifficultimplementationchallengesbecauseofheavydatadependenciesandtheconditionalnatureofthecomputations.Deblockingfilterimplementationstendtobefixedanddifficulttoreconfigurefordifferentneeds.Theabilitytoscaleupforhigherqualityrequirementssuchas10-bitpixeldepthora4:2:2chromaformatoftenreducesthethroughputofaparallelarchitecturedesignedforlowerfeatureset.Ascalablearchitecturefordeblockingfiltering,createdwithamassivelyparallelprocessorbasedsolution,meansthatthesameencoderordecoderwillbedeployedinavarietyofapplications,atdifferentvideoresolutions,fordifferentpowerrequirements,andathigherbit-depthsandbettercolorsubsamplingpatternslikeYUV,4:2:2,or4:4:4formats.

ThisworkdescribesascalableparallelarchitectureforanH.264compliantdeblockingfilter.Paralleltechniquessuchasparallelprocessingofindependentmacroblocks,subblocks,andpixelrowlevelareexaminedinthiswork.Thedeblockingarchitectureconsistsofabasiccellscalleddeblockingfilterunit(DFU)anddependentdatabuffermanager(DFM).TheDFUcanbeusedinseveralinstances,cateringtodifferentperformanceneeds;theDFMservesthedatarequiredforthedifferentnumberofDFUs,andalsomanagesalltheneighboringdatarequiredforfuturedataprocessingofDFUs.Thisapproachachievesthescalability,flexibility,andperformanceexcellencerequiredindeblockingfilters.


Interactive plenoptic rendering with GPUsA.Lumsdaine,G.N.Chunev,IndianaUniv.(UnitedStates);T.G.Georgiev,AdobeSystemsInc.(UnitedStates)

Processingandrenderingofplenopticcameradatarequiressignificantcomputationalpowerandmemorybandwidth.Atthesametime,interactiverenderingperformanceishighlydesirablesothatuserscanexploretheinfinitevarietyofimagesthatcanberenderedfromasingleplenopticimage.InthispaperwedescribeaGPU-basedapproachforlightfieldprocessingandrendering,withwhichweareabletoachieveinteractiveperformanceforfocusedplenopticrenderingtaskssuchasrefocusingandnovel-viewgeneration.WepresentaprogressionofrenderingapproachesforfocusedplenopticcameradataandanalyzetheirperformanceonpopularGPU-basedsystems.OuranalysesarevalidatedwithexperimentalresultsoncommerciallyavailableGPUhardware.Evenforcomplicatedrenderingalgorithms,weareabletorender39Mpixelplenopticdatato2Mpixelimageswithframeratesinexcessof500framespersecond.


Three-level GPU accelerated Gaussian mixture model for background subtractionY.Li,G.Wang,X.Lin,TsinghuaUniv.(China)

GaussianMixtureModel(GMM)forbackgroundsubtraction(BGS)iswidelyusedfordetectingandtrackingobjectsinvideosequences.DespitethesatisfyingresultsprovidedbyGMM,lowprocessingspeedhasbecomethebottleneckforreal-timeapplications.Weproposea


Conference 8295BnovelmethodtoacceleratetheGMMalgorithmbasedongraphicsprocessingunit(GPU).AsGPUexcelsatperformingmassivelyparalleloperations,thenoveltyliesinhowtoadoptvariousoptimizationstrategiestofullyexploitGPU’sresources.Theparalleldesignconsistsofthreelevels.Onthebasisoffirst-levelimplementation,weemploytechniquessuchasmemoryaccesscoalescingandmemoryaddresssavingtothesecond-leveloptimizationandthethird-levelmodification,whichreducesthetimecostandincreasesthebandwidthgreatly.Experimentalresultsdemonstratethattheproposedmethodcanyieldperformancegainsof145framespersecond(fps)forVGA(640*480)videoand505fpsforQVGA(320*240)videowhichoutperformtheirCPUcounterpartsby24Xand23Xspeeduprespectively.Comparedwiththepreviousrelatedworkof10Xspeedup,theproposedmethodgains22Xspeedupaveragely.TheresultedsurveillancesystemcanprocessfiveVGAvideossimultaneouslywithstrongrobustnessandhighefficiency.


Plane-dependent error diffusion on a GPUY.Zhang,Univ.ofCalifornia,Davis(UnitedStates);J.L.Recker,Hewlett-PackardLabs.(UnitedStates);R.A.Ulichney,I.Tastl,Hewlett-PackardCo.(UnitedStates);J.D.Owens,Univ.ofCalifornia,Davis(UnitedStates)

Inthispaper,westudyaplane-dependenttechniquethatreducesdot-on-dotprintingincolorimages,andapplythistechniquetoaGPU-basederrordiffusionhalftoningalgorithm.Wedesignimagequalitymetricstopreservemeancolorandminimizecolorantoverlaps.

Wefurtheruserandomizedintra-planeerrorfilterweightstobreakperiodicstructures.OurGPUimplementationachievesaprocessingspeedof200MegaPixels/secondforRGBcolorimages,andaspeedupof30-37xoveramulti-threadedimplementationonadual-coreCPU.SincetheGPUimplementationismemorybound,weessentiallygettheimagequalitybenefitsforfreebyaddingarithmeticcomplexitiesforinter-planedependencyanderrorfilterweightsrandomization.


An analysis of OpenCL for portable imagingR.J.Moore,3MCo.(UnitedStates);B.Zimmer,3MCo.(UnitedStates)andUniv.ofWisconsin-EauClaire(UnitedStates)

Inthispaper,weexaminehowwellOpenCLdeliversonthismultitargetpromisefordifferentimageprocessingalgorithms.BothGPU(NvidiaandAMD)andCPU(AMDandIntel)platformsareexploredtodeterminehowOpenCLperformsusingthesamecodeondifferenthardwareanddriverconfigurations.WealsocompareOpenCLwithoptimizedCPUandGPU(CUDA)versionsofthesameimagingalgorithms.WepresentourfindingsandsharesomeinterestingobservationsontheprocessofusingOpenCL.Theimagingalgorithmsincludebasiccolortransformation,convolution,andvisualattentionsaliencymapcalculation.Thelastalgorithmincludesmanydifferentimagingcalculations:differenceofGaussian,colorfeatures,imagestatistics,FFTfiltering,andassortedotheralgorithms.Thesaliencymapcalculationprovidesacomplexrealworldimagingapplicationtouseforcomparisonofthetestplatforms.

8295B-61, Poster Session

Speed-up on GPU in Bayesian image reconstructionS.Horiuchi,S.Yoshida,Z.Ushiyama,M.Yamamoto,TokyoUniv.ofScience(Japan)

ThereisamethodforimageprocessingthatusesBayes’theoremwhichisthestatisticaltechnique.Thistechniquecanbeappliedtoopticalsystems,andcanhandleshift-variantsystem.Therefore,this

techniqueiseffectiveandcanbeusedforwideapplication.However,itrequireshighcomputationalcostbecauseorderofcomplexityisinproportiontoO(n^6)intwodimensionalproblems.Then,weproposeacalculationmethodusingGPGPU(General-purposecomputingonGPU)intheBayesianimagereconstructionforspeed-upofcalculation.


Conference 8296: Computational Imaging XMonday-Tuesday23-24January2012

Part of Proceedings of SPIE Vol. 8296 Computational Imaging X

8296-01, Session 1

Imaging with electrons: a review of modern modalitiesM.DeGraef,UnitedStates(UnitedStates)


8296-02, Session 1

Improving boundary localization in the statistical image segmentation of materials micrographsM.L.Comer,PurdueUniv.(UnitedStates)


8296-03, Session 1

Scanning transmission electron tomography and applications to materials scienceL.F.Drummy,AirForceResearchLab.(UnitedStates)


8296-04, Session 1

Combining global labeling and local relabeling for metallic image segmentationJ.W.Waggoner,Univ.ofSouthCarolina(UnitedStates);J.Simmons,AirForceResearchLab.(UnitedStates);S.Wang,Univ.ofSouthCarolina(UnitedStates)

Analyzingthemicrostructureofmetalsplaysanimportantroleinmaterialsscienceandindustry.Typically,themicrostructureofametalsampleismadeupofalargenumberofgrainswhichcanbeimagedbyamicroscope.Inthispaper,wedevelopaneffectiveapproachtoautomaticallyextractthesegrainsbysegmentingasequenceof2Dmicroscopicimagesthataretakenfromcontinuousserialsectionsofametalsample.Inourapproach,werepeatedlypropagatea2Dsegmentationfromoneslicetoanother,andweformulatethisprocessasanoptimallabelingproblemthatcanbeefficientlysolvedbythegraph-cutalgorithm.Whiletheunderlying2Dgraintopologyinneighboringslicesislargelyconsistent,itmayshowlocalvariationswhentheserial-sectionplanemovesintoanewgrainormovesoutofanexistinggrainin3Dspace.Weaddressthisproblembyfirstrunningagloballabelingtopropagatetheglobalgraintopology,followedbyaseriesoflocalrelabelingstoidentifyandcorrectthelocaltopologychanges.Finally,wetesttheperformanceoftheproposedapproachonasequenceofserial-sectionmicroscopicimagesofatitaniumsampleandshowitperformsbetterthanseveralexistingimagesegmentationmethods.

8296-05, Session 1

Towards automated detection of active colitis in images of H&E-stained tissue samplesM.McCann,R.Bhagavatula,CarnegieMellonUniv.(UnitedStates);M.Fickus,AirForceInstituteofTechnology(UnitedStates);J.A.Ozolek,Children’sHospitalofPittsburgh(UnitedStates);J.Kovacevic,CarnegieMellonUniv.(UnitedStates)


8296-06, Session 1

Computer-aided fiber analysis for crime scene forensicsM.Hildebrandt,A.Makrushin,J.Dittmann,C.Arndt,Otto-von-Guericke-Univ.Magdeburg(Germany)

Theforensicanalysisoffibersiscurrentlycompletelymanualand,therefore,timeconsumingprocess.Theautomationofanalysisstepscansignificantlysupportforensicexpertsandreducethetimerequiredfortheinvestigation.Moreover,asubjectiveexpertbeliefisextendedbyobjectivemachineestimation.Thisworkproposesthepatternrecognitionpipelinecontainingthedigitalacquisitionofafibermedia,thepre-processingforfiberssegmentation,andtheextractionofthedistinctivecharacteristicsoffibers.Currently,thelength,width,color,frequencyofoccurrenceanddistributionoffibersareinvestigated.Inordertosupporttheautomaticclassificationoffibersthesupervisedandunsupervisedmachinelearningalgorithmsareengaged.Thefirstexperimentalsetupincludesacarseatandtwodifferentclothes.Stayingintheseat,atestpersonleavestextilefibersbehind.Thetestaimsatautomaticdistinguishingofclothesthroughthefibertracesgainedfromtheseatwiththehelpofadhesivetape.Thedigitalizationisprovidedbyflatbedscannerandachromaticwhitelightsensor.Thenon-destructivecontact-lesssensorsstudiedinthiswork,canbepotentiallyusedforthedirectacquisitionofthesuspectedsurface,makingadhesivetapeunnecessary.Furthermore,anautomatedacquisitionandanalysisallowtheprocessingoflargeareaswithnumerousdifferentfibers.

8296-07, Session 1

An automated diagnostic aid for otitis mediaA.Kuruvilla,CarnegieMellonUniv.(UnitedStates);P.H.HenningsYeomans,OntarioInstituteforCancerResearch(Canada);P.Quelhas,InstitutodeEngenhariaBiomédica(Portugal);A.Hoberman,Univ.ofPittsburgh(UnitedStates);J.Kovacevic,CarnegieMellonUniv.(UnitedStates)


8296-08, Session 1

3D reconstruction based on single-particle cryo electron microscopy images as a random signal in noise problemQ.Wang,CornellUniv.(UnitedStates);Y.Zheng,LawrenceBerkeleyNationalLab.(UnitedStates);P.C.Doerschuk,CornellUniv.(UnitedStates)



Conference 82968296-09, Session 1

Highly scalable methods for exploiting a label with unknown location in order to orient a set of single-particle cryo electron microscopy imagesC.J.Prust,MilwaukeeSchoolofEngineering(UnitedStates);P.C.Doerschuk,CornellUniv.(UnitedStates);J.E.Johnson,TheScrippsResearchInstitute(UnitedStates)

Ahighlyscalablemethodfordeterminingtheprojectionorientationofeachimageinasetofcryoelectronmicroscopyimagesofalabeledparticleisproposed.Themethodreliesonthepresenceofalabelthatisasufficientlystrongscatterersuchthatits2-Dlocationineachimagecanberestrictedtoatmostasmallnumberofsitesbyprocessingappliedtoeachimageindividually.Itisnotnecessarytoknowthe3-Dlocationofthelabelontheparticle.Afterfirstdeterminingthepossiblelocationsofthelabelinthe2-Dimagesinparallel,theinformationfromallimagesisfusedtodeterminethe3-Dlocationofthelabelontheparticleandthenthe3-Dlocationisusedtodeterminetheprojectionorientationforeachimagebyprocessingeachimageindividually.Withprojectionorientations,manyalgorithmsexistforcomputingthe3-Dreconstruction.TheperformanceofthealgorithmisstudiedasafunctionofthelabelSNR.

8296-10, Session 2

Plenoptic camera with freely movable microlensesT.G.Georgiev,AdobeSystemsInc.(UnitedStates);S.Goma,QualcommInc.(UnitedStates)

Theideaofcapturingthe4Dradiance,orplenopticfunction,forthepurposeof3DphotographywasintroducedbyLippmannin1908.Sincethenithasgonethroughseveralmodifications,includingdigitalprocessing(Adelson1992,the“plenopticcamera”),digitalrefocusing(Ng2005),superresolution,andothers).

Themainpracticalproblemofplenopticsisthelowresolutionofthefinalrenderedimagerelativetothatofaconventionalcamerausingthesamesensor.Thisleadstotheideaofbuildingaplenopticcamerathatcaptures“thebestofbothworlds”:Itshouldeasilyconvertintoamodeequivalenttoa2Dcameraofhighresolution.

Wepresenttheresultsofourworkonbuildingsuchplenopticcamera,basedonamicrolensarraythatcanbemovedfreelywithmicromotorsthatpositionitatprcisedistancesfromsilicondieoftheCCD.Asaresultthecamerachangesfromcapturing3Datlowspatialresolution--tocapturing2Dwithveryhighspatialresolutionaftercomputationalprocessing.Switchingbetweenmodesisdoneonthefly,withinasecond.

Wealsousethetwomotorstotiltthemicrolensarray,thusimplementingshift-tiltplenopticcamera,greatlyincreasingusabledepthoffield.Ourpapershowsexperimentalresults.

8296-11, Session 2

Image reconstruction using projections from a few views by discrete steering combined with DARTJ.Kwon,S.M.Song,B.Kauke,D.P.Boyd,TeleSecuritySciences,Inc.(UnitedStates)

Inthispaper,weproposeanalgebraicreconstructiontechnique(ART)baseddiscretetomographymethodtoreconstructanimageaccuratelyusingprojectionsfromafewviews.WespecificallyconsidertheproblemofreconstructinganimageofbottlesfilledwithvarioustypesofliquidsfromX-rayprojections.Byexploitingthefactthatbottlesareusuallyfilledwithhomogeneousmaterials,itispossibletoobtainaccuratereconstructiononlywithafewprojectionsbyART.Inorder

todealwithvarioustypesofliquidsinourproblem,wefirstintroduceourdiscretesteeringthatisamodificationofbinarysteeringapproachofCensoretalforourmulti-valueddiscretereconstruction.Themainideaofsteeringapproachistouseslowlyvaryingthresholdsinsteadoffixedones.WefurtherincreasereconstructionaccuracybyreducingthenumberofvariablesinARTbycombiningourdiscretesteeringwiththediscreteART(DART)thatfixesthevaluesofinteriorpixelsofsegmentedregionsconsideredasreliable.Byvarioussimulationstudies,weshowthatourdiscretesteeringcombinedwithDARTyieldssuperiorreconstructionthanbothdiscretesteeringonlyandDARTonlycasesandquiteaccurateresultsevenwithafewviewssuchaslessthanfive.

8296-12, Session 2

One-dimensional control grid interpolation-based demosaicing and color image interpolationC.M.Zwart,D.H.Frakes,ArizonaStateUniv.(UnitedStates)

Werecentlyreportedgoodresultswithourimageinterpolationalgorithm,One-DimensionalControlGridInterpolation(1DCGI),inthecontextofgrayscaleimages.1DCGIhashighquantitativeaccuracy,flexibilitywithrespecttoscalingfactor,andlowcomputationalcostrelativetosimilarlyperformingmethods.HerewelooktoextendourmethodtothedemosaicingofBayer-Patternedimagesandinterpolationofcoloredimages.1DCGI-baseddemosaicingperformsquantitativelybetter(higherPSNR)thanthegradient-correctedlinearinterpolationmethodofMalvar.1DCGIinterpolationoffullcolorimagesperformsquantitativelybetterthancovariance-basedapproaches,suchasiNEDIandNEDI,atsubstantiallylowercomputationalcost.

8296-13, Session 2

Limited view angle iterative CT reconstruction for transportation security applicationS.J.Kisner,C.A.Bouman,PurdueUniv.(UnitedStates)


8296-15, Session 2

Variational semi-blind sparse image reconstruction with application to MRFMS.U.Park,A.O.Hero,Univ.ofMichigan(UnitedStates);N.Dobigeon,Univ.deToulouse(France)

Thispaperaddressestheproblemofjointimagereconstructionandpointspreadfunction(PSF)estimationwhenthePSFoftheimagingdeviceisonlypartiallyknown.Tosolvethissemi-blinddeconvolutionproblem,priordistributionsarespecifiedforthePSFandthe3Dimage.JointimagereconstructionandPSFestimationisthenperformedwithinaBayesianframework,usingavariationalalgorithmtoestimatetheposteriordistribution.Theimagepriordistributionimposesanexplicitatomicmeasurethatcorrespondstoimagesparsity.Simulationresultsdemonstratethatthesemi-blinddeconvolutionalgorithmcomparesfavorablywithpreviousMarkovchainMonteCarlo(MCMC)versionofmyopicsparsereconstruction.Italsooutperformsnon-myopicalgorithmsthatrelyonperfectknowledgeofthePSF.Thealgorithmisillustratedonrealdatafrommagneticresonanceforcemicroscopy(MRFM).


8296-16, Session 3

Moon search algorithms for NASA’s Dawn mission to asteroid VestaN.Memarsadeghi,L.A.McFadden,D.R.Skillman,NASAGoddardSpaceFlightCtr.(UnitedStates);B.McLean,M.Mutchler,SpaceTelescopeScienceInstitute(UnitedStates)

Amoonornaturalsatelliteisacelestialbodythatorbitsaplanetorplanet-likebody.Scientistsseekunderstandingtheoriginandevolutionofoursolarsystembystudyingmoonsofplanetsandplanet-likebodies,calledasteroidsanddwarfplanets.Additionally,searchesforsatellitesofplanetarybodiescanbeimportanttoprotectthesafetyofaspacecraftasitapproachesororbitsaplanetarybody.Ifasatelliteofacelestialbodyisfound,themassofthatbodycanbecalculatedonceitsorbitisdetermined.EnsuringtheDawnspacecraft’ssafetyonitsmissiontotheasteroid(4)VestaprimarilymotivatedtheworkofDawn’sSatelliteWorkingGroup(SWG)insummerof2011.DawnmissionscientistsandengineersutilizedvariouscomputationaltoolsandtechniquesforVesta’ssatellitesearch.Theobjectivesofthispaperareto1)introducethenaturalsatellitesearchproblem,2)presentthecomputationalchallenges,approaches,andtoolsusedwhenaddressingtheproblem,and3)describeapplicationsofvariousimageprocessingandcomputationalalgorithmsforperformingsatellitesearchestotheelectronicimagingandcomputersciencecommunity.Furthermore,wehopethatthiscommunicationwouldenableDawnmissionscientiststoimprovetheirsatellitesearchalgorithmsandtoolsandbebetterpreparedforperformingthesameinvestigationin2015,whenthespacecraftisscheduledtoapproachandorbitthedwarfplanet(1)Ceres.

8296-17, Session 3

CLEAN: a false alarm reduction method for SAR CCDR.Phillips,MITLincolnLab.(UnitedStates)


8296-18, Session 3

Insertion of synthetic features in SAR CCD imageryE.Turner,R.Phillips,M.Cha,MITLincolnLab.(UnitedStates)


8296-19, Session 3

Multichannel hierarchical image classification using multivariate copulasA.Voisin,V.Krylov,INRIASophiaAntipolis-Méditerranée(France);G.Moser,S.B.Serpico,Univ.degliStudidiGenova(Italy);J.Zerubia,INRIASophiaAntipolis-Méditerranée(France)

Thispaperfocusesontheclassificationofmultichannelimages.TheproposedsupervisedBayesianclassificationmethod,appliedtohistological(medical)andSAR(remotesensing)imagery,consistsoftwosteps.

Thefirstdealswiththestatisticalmodelingofthecoregisteredinputimages(forinstanceRGBopticalimage,orsingle-channelSARimagecombinedwithanextractedtexturalfeature).Foreachclassandeachchannelinthisstacked-vectorinputdataset,theclass-conditionalmarginalprobabilitydensityfunctions(PDFs)areestimatedbyfinitemixturesofwell-chosenparametricfamilies.Foropticalimagery,thenormaldistributionisagenerallyacceptedmodel.ForSARimagery,weselectedgeneralizedGamma,log-normal,NakagamiandWeibulldistributions.Next,themultivariated-dimensionalClaytoncopulas

(dbeingthenumberofinputchannels)areappliedtoestimatemultivariatejointclass-conditionalstatistics.

Asasecondstep,weplugtheestimatedjointPDFsintoacontextualmodelbyusingtwodifferentMarkov-basedmethods.ThefirstmodelusesaspatialcontextviaahiddenMarkovrandomfield(MRF)model.ThesecondmodelusesamultiscaleapproachviaahierarchicalMarkovianmodelbasedonaquad-treestructure.Multiscalefeaturesareextractedbydiscretewavelettransforms.Weintegrateanexactestimatorofthemarginalposteriormode(MPM)withtheSEM-copulaPDFestimates.Suchahierarchicalmodelsupportsafeasiblefutureextensiontoinputmulti-resolutionimagery.

8296-32, Session 3

Figure-ground organization is easier than previously thoughtY.Li,T.Kwon,PurdueUniv.(UnitedStates);L.J.Latecki,TempleUniv.(UnitedStates);Z.Pizlo,PurdueUniv.(UnitedStates)


8296-20, Session 4

Denoising and deblurring of Fourier-trasform infrared spectroscopic imagingT.H.Nguyen,R.K.Reddy,M.J.Walsh,M.Schulmerich,G.Popescu,M.N.Do,R.Bhargava,Univ.ofIllinoisatUrbana-Champaign(UnitedStates)

FourierTransform-Infrared(FT-IR)SpectroscopicImagingisavaluabletoolnotonlytoobtainchemicalinformationoftissuesamplesbutalsotogreatlyassistpathologyprocesssuchascancertumorlocalization.However,currentlimitationsofFT-IRimagingincludelongacquisitiontime,vastdatastorage,lowsignal-to-noiseratio(SNR),andlowspatialresolutionduetocontaminatednoise,pixelationandopticaleffectssuchasopticaldiffractionandMiescattering.Thispaperproposesaframeworktotackletheseproblemsbyexploitinglow-rankpropertyofthedata,modelingdiffraction-limitedincoherentpointspreadfunction,simultaneouslydeblurringandestimatingunknowninformationoftheimage,additivenoiseandthepointspreadfunctionwithaBayesianvariationalapproach.Byincreasingmagnificationfactorintheopticalsetuptoreducepixelationandcombiningwithadeconvolutionalgorithm,wewillshowthatmoreinformationbelowconventionaldiffractionlimitcanbeobtainedfromnotonlytheshortwavelengthregionbutalsofromthelongwavelengthregion,whichwascurrentlylimitedbyopticaldiffraction.

8296-21, Session 4

Iterative weighted risk estimation for nonlinear image restoration with analysis priorsS.Ramani,J.Rosen,Z.Liu,J.A.Fessler,Univ.ofMichigan(UnitedStates)

Imagerestorationalgorithmsdependonanumberofparametersthatrequireproperselectiontoobtainmeaningfulresults.Meansquarederror(MSE)isawidelyusedmetricforassessingimagequalityandparameterselection,butrequirestheknowledgeoftheunknownnoise-freeimageandisthereforenotpracticable.Stein’sunbiasedriskestimate(SURE)providesameansfortheestimationofMSEpurelyfromthedataandsomefirstorderderivativeinformationofthealgorithm(i.e.,thedivergenceofthelinear/nonlinearrestorationoperatorwithrespecttothedata)forthecaseofadditiveGaussiannoise.Inthiswork,weusetheprincipleunderlyingSUREtoestimatethepredictedmeansquarederror(PMSE)forparameterselectionforimagerestoration.EarlierworksthatapplytheSURE-principleforparameterselectioninrestorationhavefocusedeitheronlinearalgorithmsornonlinearalgorithmsforsynthesispriors.Inthiswork,our

Conference 8296


goalistoestimatePMSEfornonlinearalgorithmswithanalysispriorswhichincludepopularcriteriasuchastotalvariationandsmoothedge-preservingregularizers.WeproposetoderiveananalyticalexpressionfortheJacobianmatrix(whosetraceyieldsthedesireddivergence)associatedwiththealgorithmthatcanberecursivelyupdatedasthealgorithmevolves.Forthis,wefocusontheiterativereweighedleast-squares,whichisgradient-descent-basedalgorithmthatcanhandleavarietyofregularizationcriteria.Throughnumericalsimulations,weshowthatPSURE(theestimateofPMSE)accuratelypredictsPMSEandyieldsregularizationparametersclosetothosethatminimizethe“unknown’’MSE,indicatingitspotentialforparameterselectioninimagerestoration.

8296-22, Session 4

Nonlocal transform-domain denoising of volumetric data with groupwise adaptive variance estimationM.T.Maggioni,A.Foi,TampereUniv.ofTechnology(Finland)

WeproposeanextensionoftheBM4Dvolumetricfiltertothedenoisingofdatacorruptedbyspatiallynon-uniformnoise.

BM4Dimplementsthegroupingandcollaborativefilteringparadigm,wheresimilarcubesofvoxelsarestackedintoafour-dimensional“group”.Eachgroupundergoesasparsifyingfour-dimensionaltransform,thatexploitsthelocalcorrelationamongvoxelsineachcubeandthenonlocalcorrelationbetweencorrespondingvoxelsofdifferentcubes.Thus,signalandnoiseareeffectivelyseparatedintransformdomain.Inthisworkwetakeadvantageofthesparsityinducedbythefour-dimensionaltransformtoprovideaspatiallyadaptiveestimationofthenoisevariancebyapplyingarobustmedianestimatoroftheabsolutedeviationtothespectrumofeachfilteredgroup.Theadaptivenoiseestimatesarethenusedduringcoefficientsshrinkage.Finally,theinversefour-dimensionaltransformisappliedtothefilteredgroup,andeachindividualcubeestimateisadaptivelyaggregatedatitsoriginallocation.

ExperimentsonmedicaldatacorruptedbyspatiallyvaryingGaussianandRiciannoisedemonstratetheefficacyoftheproposedapproachinvolumetricdatadenoising.Incaseofmagneticresonancesignals,theadaptivevarianceestimatecanbealsousedtocompensatetheestimationbiasduetothenon-zero-meanerrorsoftheRician-distributeddata.

8296-23, Session 4

Non-uniform contrast correction for coded source neutron imagingH.J.Santos-Villalobos,P.R.Bingham,OakRidgeNationalLab.(UnitedStates)

Thereisacleardemandforcost-effectivescintillator-basedneutronimagingsystemsthatachieveresolutionsof1μmorless.Suchimagingsystemwoulddramaticallyextendtheapplicationofneutronimagingtomicro-scalestructuressuchasmicrochannelheatexchangers,fuelcellcomponents,biologicalmicroscopyforpharmacology,drugdeliveryresearch,fuelinjectorsprayersforefficientdieselenginetechnology,andbiofuelsresearch.Forsuchpurposesacodedsourceimagingsystemisunderdevelopment.Theleadingchallengeistoreduceartifactsinreconstructedcodedsourceimages.Neutronsinteractwithseveralopticalcomponentsbeforetheyinteractwiththeimagedobject,andfinallyhitthedetector.Theseinteractionsintroducepatternstotheneutronsource,whichtranslatetoartifactsinthereconstructedimage.Asthesystemmagnificationincreases,theeffectsofthenon-uniformityofthesourcearemoreevidentandserious.Inthispaper,wedescribeacodedsourceneutronimagingworkflow.Inparticular,thealgorithmstocorrectthenon-uniformityoftheneutronsourcearediscussed.Weassessnon-parametrized(e.g.,arithmeticcorrection)andparametrized(e.g.,maximumlikelihoodestimator)correctionmethods.Finally,allmethodsarecomplementedwithsimulatedandexperimentalexamples.

8296-24, Session 4

Image enhancement and quality measures for dietary assessment using mobile devicesC.Xu,F.Zhu,N.Khanna,C.J.Boushey,E.J.DelpIII,PurdueUniv.(UnitedStates)

Measuringaccuratedietaryintakeisconsideredtobeanopenresearchprobleminthenutritionandhealthfields.Dietaryassessmentmethodsperceivedaslessburdensomeandlesstime-consumingmayimprovecompliance.Theuseofamobiletelephone’sbuilt-indigitalcamerahasbeenshowntoprovideuniquemechanismsforreducinguserburdenandimprovingtheaccuracyandreliabilityofdietaryassessment.Wearedevelopingasystem,knownasthemobiledevicefoodrecord(mdFR),toautomaticallyidentifyandquantifyfoodsandbeveragesconsumedbasedonanalyzingmealimagescapturedwithahandholdmobiledevice.

Toautomaticallyidentifyandestimateportionsoffoodsinamealimage,itiscrucialtoobtaingoodqualityimages,whichfulfilltherequirementsofdifferentimageanalysissteps.Thispaperpresentslowcomplexitymethodsimagequalitymeasurementandimageenhancementwhichwillimprovetheaccuracyandrobustnessofexistingdietaryassessmentmethodsdeployedonmobiledevices.Theseincludemethodsforfiducialmarkerdetectionandblurassessmentonmobiledevices.ThemethodsproposedinthispaperhavebeendeployedontheiPhonewithoutaddinganyperceptibledelayintheimagecapturestepandthusenhancestheuserexperience.

8296-25, Session 5

Risk estimates for MRI denoisingP.J.Wolfe,HarvardUniv.(UnitedStates)


8296-26, Session 5

Subjective evaluations of example-based, total variation, and joint regularization for image processingH.S.Anderson,M.R.Gupta,Univ.ofWashington(UnitedStates);J.Hardeberg,GjøvikUniv.College(Norway)

Wepresentsubjectiveevaluationsofexample-basedregualrization,totalvariationregularization,andaproposedjointexample-basedandtotalvariationregularizationforimageestimationproblems.Wefocusonthenoisydeblurringproblem,whichgeneralizesimagesuperresolutionanddenoising.Controlledsubjectiveexperimentsshowthattheproposedjointregularizationcanyieldsignicantimprovementoveronlyusingtotalvariationorexample-basedregularization,particularlywhentheexampleimagescontainsimilarstructuralelementsasthetestimage.Wealsoinvestigatewhethertheregularizationparameterscanbetrainedbycross-validation,andthedierenceincross-validationjudgmentsmadebyhumansorbyfullyautomaticimagequalitymetrics.Experimentsshowthatofveimagequalitymetricstested,thestructuralsimilarityindex(SSIM)correlatesbestwithhumanjudgementofimagequality,andcanbeprobablyusedtocross-validateregularizationparameters.However,thereisasignicantqualitygapdependingonwhethertheparametersarecross-validatedbyhumansorwiththebestimagequalitymetric.

8296-27, Session 5

Removal of haze and noise from a single imageE.Matlin,P.Milanfar,Univ.ofCalifornia,SantaCruz(UnitedStates)

Imagesofoutdoorscenesoftencontainatmosphericdegradation,generallyreferredtoashaze,causedbyparticlesintheatmospheric

Conference 8296


mediumabsorbingandscatteringlight.Foravarietyofreasons(e.g.computervisionorsensingalgorithms)onemaywishtoremovethehazefromtheseimages.Furthermore,allimagescontainsomecorruptionfromnoiseduetomeasurement(sensor)error,whichisamplifiedbythehazeremovalprocessifnotaddressed.Theaimofthispaperistopresentaneffectivemethodforremovingbothhazeandnoisefromonlyasingledigitalimage.Weproposetwodifferentmethods:denoisetheimagewithastate-of-the-artdenoisingalgorithmasapre-processingsteptohazeremoval,andsimultaneouslydenoiseanddehazeusinganiterative,adaptive,kernel-regressionbasedmethod.Findingsshowthatdenoisingpriortodehazingworkswellwhenthelevelofnoiseisknownapriori;however,latenterrorsfromeither“under”-denoisingor“over”-denoisingareamplifiedinthedehazingprocess.Inthissituation,theiterativeapproachcanyieldsuperiorresults.

8296-14, Session 6

Image reconstruction from nonuniformly spaced samples in Fourier domain optical coherence tomographyJ.Ke,E.Y.Lam,R.Zhu,TheUniv.ofHongKong(HongKong,China)

InreconstructinganobjectinFourierDomainOCT(FD-OCT),conventionallyFFTisusedwithsystemmeasurements.However,FFTnormallyrequiresdatathatareuniformlysampledinwavenumber,whileheremeasurementsareuniformlysampledinwavelengthinstead.Thenonlinearcorrespondencebetweenwavelengthandwavenumbercausesnon-uniformlyspacedsamplinginFD-OCT.Todealwiththisissue,systemmeasurementsareresampledbeforeFFTisappliedintheconventionalmethod.

Inthiswork,weuseinverseimagingforobjectreconstruction.WefirstmodelFD-OCTsystemwithalinearsystemofequations,wherethesourcepowerspectrumandthenon-uniformlyspacedsamplepositionarerepresentedaccurately.Thenwereconstructobjectsignaldirectlyfromthenon-uniformlyspacedwavelengthmeasurements.Withtheinverseimagingmethod,wealsoestimate2Dcross-sectionalobjectimagedirectlyinsteadofasetofindependentA-linesignals.ByusingTotalVariation(TV)asaconstraintintheoptimization,wereducethenoiseinthe2Dobjectestimation.BesidesTV,wealsouseobjectsparsityinspatialdomainasaregularizationtofurtherimproveinverseimagingreconstructionperformance.Experimentalresultsdemonstratetheadvantagesofourmethod,aswecompareitwithconventionalandNUDFTmethodforreconstruction.

8296-28, Session 6

Finding saliency in noisy imagesC.Kim,P.Milanfar,Univ.ofCalifornia,SantaCruz(UnitedStates)

Recently,manycomputationalsaliencymodelshavebeenintroducedtotransformagiveninputimageintoascalar-valuedmapthatrepresentsvisualsaliencyofthecorrespondinginputimage.Theseapproaches,however,assumetheinputimagegiveniscleanandfailtoprovidegoodresultswhentheinputimageisdegraded.Accordingly,thequestionishowtocomputesaliencyinareliablewaywhenanoise-corruptedimageisgiven.Inordertodealwiththisproblem,weinvestigatedthesensitivityofthesaliencyfunctionmeasuringtheself-resemblanceandfoundittobesensitivetosmallchangesinitsarguments.Assuch,wemodifythesaliencyfunctionwithanappropriatecorrectiontermtofindamorerobustestimateintermsofmeansquarederror(MSE).Morespecifically,theoptimalparametersofthiscorrectionwillbedeterminedbyminimizinganestimateofMSEbetweensaliencyvaluescalculatedfromcleanimageandtheonesestimatedfromthenoisyimage.FortheMSEestimatehere,weshalluseStein’sunbiasedriskestimate(SURE)thatdependsonthenoisydataalone.Thepaperwillpresentanestimationtheoreticapproachtothecomputationofsaliencyfornoisyimagesandshowtheperformanceoftheproposedestimatorbycomparingitwithexistingmethods.

8296-29, Session 6

Automatic loop closure detection using multiple cameras for 3D indoor localizationN.Corso,J.Kua,J.Chen,A.Zakhor,Univ.ofCalifornia,Berkeley(UnitedStates)

Automated3Dmodelingofbuildinginteriorsisusefulinapplicationssuchasvirtualrealityandenvironmentmapping.Wehavedevelopedahumanoperatedbackpackdataacquisitionsystemequippedwithavarietyofsensorssuchascameras,laserscanners,andorientationmeasurementsensorstogenerate3Dmodelsofbuildinginteriors,includingunevensurfacesandstairwells.Animportantintermediatestepinany3Dmodelingsystem,includingours,isaccurate6degreesoffreedomlocalizationovertime.Inthispaper,weproposetwoapproachestoimprovelocalizationaccuracyoverexistingmethods.First,wedevelopanadaptivelocalizationalgorithmwhichtakesadvantageoftheenvironment’sfloorplanaritywheneverpossible.Secondly,weshowthatbyincludingalltheloopclosuresresultingfromtwocamerasfacingawayfromeachother,itispossibletosignificantlyreducelocalizationerrorinscenarioswherepartsoftheacquisitionpathisretraced.Weexperimentallycharacterizetheperformancegainsduetobothschemes.

8296-30, Session 6

An information theoretic trackability measureS.T.Acton,A.Aksel,Univ.ofVirginia(UnitedStates)

Thereexistsnomeasuretoquantifythedifficultyofavideotrackingproblem.Suchdifficultydependsuponthequalityofthevideoandupontheabilitytodistinguishthetargetfromthebackgroundandfromotherpotentialtargets.Wedefineatrackabilitymeasureinaninformationtheoreticframework.TheShannon-borntoolsofinformationtheory,andmorespecifically,ratedistortiontheory,allowameasureoftrackabilitythatseamlesslycombinesthevideo-dependentaspectswiththetarget-dependentaspectsoftrackingdifficultyusingmeasureofrateandinformationcontent.Specifically,videoqualityisencapsulatedintoatermthatmeasuresspatialresolution,temporalresolution,signal-to-noiseratio(assuminganadditivewhiteGaussiannoisemodel),andquantization/codingartifactsbywayofaShannon-Hartleyanalysis.Then,theabilitytocorrectlymatchatemplatetoatargetisevaluatedthroughmutualinformation.Thetarget-dependenttermalsoincorporatesadistance-weightedpenaltyfornearbyclutterandtargetsthatresemblethetemplate.Theoveralltrackabilitymeasureiscomparedtotheperformanceofarecenttrackerbasedonscalespacefeaturescomputedviaconnectedfilters.TheresultsshowahighSpearman’srankcorrelationbetweenthetrackabilitymeasureandactualperformance.

8296-31, Session 6

Text replacement on cylindrical surfaces: a semi-automatic approachH.Ding,R.Bala,Z.Fan,XeroxCorp.(UnitedStates);C.A.Bouman,J.P.Allebach,PurdueUniv.(UnitedStates)

Image-basedcustomizationthatincorporatespersonalizedtextstringsintophotorealisticimagesinanaturalandappealingwayhasbeenofgreatinterestlately.Wedescribeasemi-automaticapproachforreplacingtextoncylindricalsurfacesinimagesofnaturalscenesorobjects.Theuserisrequestedtoselectaboundaryfortheexistingtextandalignapairofedgesforthesidesofthecylinder.Thealgorithmerasestheexistingtext,andinstantiatesa3-Dcylinderforwardprojectionmodeltorenderthenewtext.Theparametersoftheforwardprojectionmodelareestimatedbyoptimizingacarefullydesignedcostfunction.Experimentalresultsshowthatthetext-replacedimageslooknaturalandappealing.

Conference 8296


8296-41, Session 6

An efficient and iterative two-step depth camera self-calibration technique using depth measurementsR.S.Pahwa,D.Babacan,M.N.Do,Univ.ofIllinoisatUrbana-Champaign(UnitedStates)

Inthispaper,weproposeanovelalgorithmforcalibratingTimeofFlight(ToF)depthcamerasusingknowngeometrypresentinthecapturedimages.The

depthimagescapturedbythecamerasarefirstdenoisedusingaPoissondenoisingmodelandthencorrectedforbarreldistortion.WeusethesecorrectedanddenoiseddepthimagesalongwiththeestimatedintensityimagescapturedbythedepthcameratodevelopaneffectiveTwostepcalibrationschemethatperformsalocaloptimizationpersceneandthenaglobaloptimizationtoestimatethedepthcameraparameters.Wedemonstratethehighperformanceandaccuracyoftheproposedmethodbycomparingitwiththestandardcalibrationschemeonsyntheticandrealworlddata.

8296-42, Session 6

Registration and integration of multiple depth images using signed distance functionD.Kubacki,H.Q.Bui,D.Babacan,M.N.Do,Univ.ofIllinoisatUrbana-Champaign(UnitedStates)

Depthcameraisanewtechnologythathaspotentialtoradicallychangethewayhumansrecordtheworldandinteractwith3Dvirtualenvironments.Withdepthcamera,onecanhaveaccesstodepthinformationupto30framespersecond,whichismuchfasterthanprevious3Dscanners.Thisspeedenablesnewapplications,inthatobjectsarenolongerrequiredtobestaticfor3Dsensing.Thereis,however,atrade-offbetweenthespeedandthequalityoftheresults.Depthimagesacquiredwithcurrentdepthcamerasarenoisyandhavelowresolution,whichposesarealobstaclestoincorporatingthenew3Dinformationintocomputervisiontechniques.Inthispaper,weproposeanewmethodtoregisterandintegratemultipledepthframesovertimeontoaglobalmodelrepresentedbyanimplicitmovingleastsquaresurface.


Analysis of practical coverage of uniform motions for approximating real camera shakesH.Cho,S.Cho,PohangUniv.ofScienceandTechnology(Korea,Republicof);Y.S.Moon,J.Cho,S.Lee,SamsungElectronicsCo.,Ltd.(Korea,Republicof);S.Lee,PohangUniv.ofScienceandTechnology(Korea,Republicof)

Motionblurisusuallymodeledastheconvolutionofalatentimagewithamotionblurkernel,whichassumesatranslationalcameramotionhascauseduniformblursinablurredimage.However,realcamerashakesoftencontainnon-translationalcomponents,suchasrotations,resultinginnon-uniformblurs.Tobetterutilizetheexistingdeblurringmethodsinpractice,itisnecessarytounderstandhowmuchuniformmotions(i.e.,translations)canapproximaterealcamerashakes.

Inthispaper,weanalyzetheeffectsofrealcameramotionsonimagepixelsandpresentthecoverageofuniformmotionsforapproximatingcomplicatedcamerashakes.Wefirstperformmathematicalanalysisonthedifferenceofthemotionextentsattheopticalaxisandimageboundaryundercamerashakes.Wethenderivethepracticalcoverageoftheuniformblurmodelwhenusedforrealblurredimages.Thecoverageanalysiscanprovideguidelinesforthecasesthatcanbeeffectivelyhandledbyexistinguniformdeblurringmethods,andthecasesforwhichmorecomplicatednon-uniformblurmodelsareneeded.


Real-time computational camera system for high-sensitivity imaging by using combined long/short exposureS.Sato,Y.Okada,T.Azuma,PanasonicCorp.(Japan)

Inthiswork,weproposearealtimecomputationalcamerasystemtoprocess24fpswith62FPGAsfor4K2Kcamerausingcombinedlong/shortexposure.Ourproposedsystemwillcontributetofuturecamerasystemsbasedonthefollowingtwopoints:

1.High-sensitivity:Ourproposedcamerasystemcancaptureimageswithfourtimeshighersensitivitythanconventionalimagerwithnomotionblur.

2.Small-sizeandhighresolution:Thenewimagereconstructionmethodachievedthehigh-sensitivityimagingevenforasmallimager(thepixelpitchsizeis1.43umandtheresolutionis3840x2160pixels).

ThesimulationresultsshowedthatthePeak-Signal-to-Noise-Ratio(PSNR)oftheimagereconstructionalgorithmisover35dB.Experimentalresultsshowedthatourproposedcamerasystemiseffectiveeveninconditionsoflowillumination.


Color correction with edge preserving and minimal SNR decrease using multi-layer decompositionB.K.Park,W.Choe,J.Lim,S.Lee,C.Kim,SamsungElectronicsCo.,Ltd.(Korea,Republicof)

Thispaperdescribesthemethodrelatedtocorrectingcolordistortionincolorimaging.AcquiringcolorimagefromCMOSorCCDdigitalsensorscansufferfromcolordistortion,whichmeansthattheimagefromsensorsisdifferentfromtheoriginalimageinthecolorspace.Themainreasonsarethecrosstalksbetweenadjacentpixels,thecolorpigmentcharacteristic’smismatchwithhumanperceptionandinfra-red(IR)influxtovisiblechannel(RGB)duetoIRcutofffilterimperfection.Tocorrectthisdistortion,existingmethodsexploitmultiplyinggaincoefficientsineachcolorchannelandthismultiplicationcancausenoiseboostandlossofdetailinformation.Thispaperproposesthenovelmethodwhichcannotonlypreservecolordistortioncorrectionability,butalsosuppressnoiseboostandlossofdetailinformationinthecolorcorrectionprocessofIRcorruptedpixels.Inthecaseofnon-IRcorruptionpixels,theuseofimagebeforecolorcorrectioninsteadofIRimagemakesthiskindofmethodavailable.Specificallythecolorandlowfrequencyinformationinluminancechannelisextractedfromthecolorcorrectedimage.AndhighfrequencyinformationisfromtheIRimageortheimagebeforecolorcorrection.Themethodextractingthelowandhighfrequencyinformationusemulti-layerdecompositionskillwithedgepreservingfilters.


Bayesian image superresolution for hyperspectral image reconstructionY.Murayama,A.Ide-Ektessabi,KyotoUniv.(Japan)

Thisstudyaimstoachieveamorepracticalspectralimagingmethod,andproposesamethodtoestimatehighspatialresolutionspectralimagefromlowspatialresolutionmultispectralimage.Theoriesofspectralreflectanceestimation,suchasWienerestimation,havereducedthetimeandtroubleinspectralimaging.Theyincreasewavelengthresolutionofmultispectralimageandreconstructhyperspectralimage.Howeverthereisdemandforamorepracticalspectralimaging.Proposedmethodcantreatnotonlywavelengthregionbutspatialregionofhyperspectralimageandmakesitpossibletoincreasebothwavelengthresolutionandspatialresolution.ItisimplementedbycombiningWienerestimationandBayesiansuperresolutionintheframeworkofBayesianstatics.Inproposed

Conference 8296


method,Maximizationofthemarginallikelihoodfunctioniscarriedouttoestimatehighresolutionhyperspectralimageandregistrationparameterofeachbandofacquiredmultispectralimage.Proposedmethodallowsacquiringimageswithlowerresolutionandthenincreasingittothedesiredresolution.Thiscouldgreatlyreducetheimageacquisitiontimemakingitmorepractical.ThisstudyshowsthemethodandillustratesitseffectivenessinanexperimentalapproachapplyingacquiredmultispectralimageofaJapanesetraditionalpainting.Resultsshowobviouseffectofincreasingthespatial-resolutioninestimatinghyperspectralimage.


ToF depth image deblurring using 3D blur shape models and motion blur saliency map (MBSM)S.Lee,K.Shim,J.D.K.Kim,C.Kim,SamsungAdvancedInstituteofTechnology(Korea,Republicof)

Time-of-flightdepthcameragivesdirect3Dinformationenablingfasterandeasier3Dscenecapturingandreconstruction.Thedepthcamera,however,alsosuffersfrommotionblurcausedbyanymovementofcameraorsubjects.Becausethe3Ddepthinformationisusedtoreconstruct3Dgeometryofthescene,blurredregioninadepthimagecomesoutseriouslyinthefinalreconstructed3Dmodelcausingcolorandstructuralconfusions.

Ourobservationisthatthemotionblurobservedindepthimageshowsquitedifferentaspectfromthatofcolorimage.Inthismanuscript,weanalyzethemotionblurmechanismofToFdepthcamerainmathematicalmannerandderiveasetofmotionblur3Dshapemodels.Baseonthistheoreticalinspection,weproposeauniquedepthimagedeblurringalgorithm.


Computational imaging of defects in commercial substrates for electronic and photonic devicesR.Kashiwagi,M.Fukuzawa,M.Yamada,KyotoInstituteofTechnology(Japan)

Computational‐defectimaginghasbeenperformedincommercialsubstratesforelectronicandphotonicdevicesbycombiningthetransmissionprofileacquiredwithanimagingtypeoflinearpolariscopeandthecomputationalalgorithmtoextractstrain-inducedbirefringence.TheimagingresultsexhibiteddefectstructuresspecifictosubstratematerialssuchasresidualstraindistributioninGaPsubstrates,dislocationsandsubgrainboundariesinoptical-gradeLiNbO3substrates,andmicropipesinSiCsubstrates,whichwerenotdetectedbyconventional‘visualinspection’basedonsimpleopticalrefractionortransmissionbecauseofpoorsensitivity.Thetypicalimagingtimewaslessthan30secfor3-inchdiametersubstratewiththespecialresolutionof200μm,whilethatbyscanningpolariscopewas2hourstogetthesamespecialresolution.Sincehighsensitivity,shortimagingtime,andwidecoverageofsubstratematerialsarepracticaladvantagesoverthelaboratory-scaleapparatussuchasX-raytopographyandelectronmicroscope,ourproposedtechniqueisusefulfornondestructiveinspectionofvariouscommercialsubstratesinproductionofelectronicandphotonicdevices.


Nondestructive three-dimensional measurement of gas temperature distribution by phase tomographyS.Tomioka,S.Nishiyama,HokkaidoUniv.(Japan)

Thisstudypresentsanondestructivethree-dimensional(3-D)measurementofgastemperaturedistributionaroundaheaterby

acouplingmethodusingcomputedtomography(CT)andaninterferometer.Thegastemperaturedistributionisrelatedtoadistributionofrefractiveindex.Therefore,onceaseriesoftwo-dimensional(2-D)imageofphasemodulationthatisanintegralofrefractiveindexalonganopticalpathisobtained,a3-DdistributionofthegastemperaturecanbeideallydeterminedinthesamewayasthewidelyusedCTtodetermineadistributionofattenuationfactor.

Theinterferometerdetectsa2-Dfringepattern,andthefringepatternisaconvertedphasemodulationmapusingphaseunwrappingalgorithm.However,whenthefringepatternisnoisy,conventionalphaseunwrappingalgorithminducesmanyerrors.Evaluationsoftheaccuracybyseveralphaseunwrappingalgorithmsarepresented.

IncontrasttotheconventionalCTimaging,the2-Dphaseimagestakenfromallthedirectionofprobebeamisnotobtainedinthesystemtomeasurethegastemperature,becauseofthesettingrestrictionoftheinterferometer,andalsobecausetheobjecttobemeasuredcannotrotate.Toreconstructthetemperaturedistributionwithsuchincompletedataset,thisstudypresentstwoapproaches:aback-projectionmethodandanalgebraicmethodwithTikhonov’sregularization.


Closed-form inverses for the mixed pixel/multpath interference problem in AMCW lidarJ.P.Godbaz,M.J.Cree,A.Dorrington,TheUniv.ofWaikato(NewZealand)

Themixedpixel/multipathinterferenceproblemisasignificantsystematicerrorinAmplitudeModulatedContinuousWavelidar.Whereasmanyoff-the-shelfcommercialrangingsystemssuchastheSwissrangerSR4000advertisecentimeterlevelrangingprecision,theaccuracyoftherangingmeasurementsiscommonlyoneormoreordersofmagnitudeworse.Thispooraccuracyiscausedbyindividualpixelsintegratinglightfrommorethanonebackscatteringsource;oftenduetoscatteringwithinthecameraoptics.Inthispaperwepresenttwonovelapproachestotheproblem:oneusingattenuationratiosintheexactly-determinedcase,modellingeachcomponentreturnasascaledDiracdeltaoverrange;andoneintheover-determinedcase,modellingvolumetricscatteringusingaCauchydistribution.Boththesenewinversionmethodshavecomparativelystraightforwardclosed-forminversesthatallowreal-timeimplementation;unlikepreviousnumericalapproaches.Whereasmanypreviousapproacheshaveonlypartiallyamelioratedtheproblem,thesenewapproachesdetermineboththeamplitudeandphaseoftheindividualbackscatteringsignals.Inthispaperweevaluatetheperformanceofbothmethodsonrealandsimulateddataandexaminetheinfluenceofrelativephase,theCauchyrange-spreadcoefficientandSNRonalgorithmperformance.Wealsocomparethemethodstoourpreviouslypublishedexactly-determinednumericalapproach.

Conference 8296


Conference 8297: Document Recognition and Retrieval XIXWednesday-Thursday25-26January2012

Part of Proceedings of SPIE Vol. 8297 Document Recognition and Retrieval XIX


Bleed-through removal in degraded documentsR.F.Rowley-Brooke,A.Kokaram,TrinityCollegeDublin(Ireland)

Arestorationmethodforbleed-throughdegradeddocumentimagesusingaBayesianapproachforbleed-throughreduction,usingalinear-basedmodel,ispresentedinthispaper.Toestimatethe6unknownparametersfromthemodel,avariationofiteratedconditionalmodes(ICM)optimisationisused,wherebyeachparameterisestimatedinturnviathemodeoftheirconditionalprobabilities.However,inthecaseofthecleanimageestimates,samplesaredrawninstead,withmeanandvarianceobtainedfromthelikelihoodandpriormodels.Furthermoreeveryteniterationsthemeanandvarianceareobtainedfromamodelthatignoresthepriorprobabilitiesandtwooftheunknownvariables.Theproposedmethodistestedonvarioussamplesofhighresolution(600dpi)scannedmanuscriptimages,withdifferentdegreesofdegradation.Theperformanceofthemethodisvisuallycomparedwiththatofarecentuser-assistedrestorationmethod,and,thoughtheuser-assistanceleadstobetterclassificationinregionswherebleed-throughisdarkerinintensitythanforegroundtext,theproposedmethodisshowntoproducebetteroverallresults.


Clustering document fragments using background color and texture informationS.Chanda,K.Franke,GjøvikUniv.College(Norway);U.Pal,IndianStatisticalInstitute(India)

Forensicanalysisofquestioneddocumentssometimescanbeextensivelydataintensive.Aforensicexpertmightneedtoanalyzeaheapofdocumentfragmentsandinsuchcasestoensurereliabilityhe/sheshouldfocusonlyonrelevantevidenceshiddeninthosedocumentfragments.Relevantdocumentretrievalneedsfindingofsimilardocumentfragments.Onenotionofobtainingsuchsimilardocumentscouldbebyusingdocumentfragment’sphysicalcharacteristicslikecolor,texture,etc.Inthisarticleweproposeanautomaticschemetoretrievesimilardocumentfragmentsbasedonvisualappearanceofdocumentpaperandtexture.Multispectralcolorcharacteristicsusingbiologicallyinspiredcolordifferentiationtechniquesareimplementedhere.ThisisdonebyprojectingdocumentcolorcharacteristicstoLabcolorspace.Gaborfilter-basedtextureanalysisisusedtoidentifydocumenttexture.Itisdesiredthatdocumentfragmentsfromsamesourcewillhavesimilarcolorandtexture.ForclusteringsimilardocumentfragmentsofourtestdatasetweuseaSelfOrganizingMap(SOM)ofdimension5x5,wherethedocumentcolorandtextureinformationarebeingusedasfeatures.Weobtainedanencouragingaccuracyof97.17%from1063testimages.


Lecture video segmentation and indexingD.Ma,G.Agam,IllinoisInstituteofTechnology(UnitedStates)

Videostructuringandindexingaretwocrucialprocessesformulti-mediadocumentunderstandingandinforma-tionretrieval.Thispaperpresentsanovelapproachinautomaticstructuringandindexinglecturevideosforaneducationalvideosystem.Bystructuringandindexingvideocontent,wecansupportbothtopicindexingandsemanticqueryingofmultimediadocuments.Inthispaper,ourgoalistoextractindicesoftopicsandlinkthemwiththeirassociated

videoandaudiosegments.Twomaintechniquesusedinourproposedapproacharevideoimageanalysisandvideotextanalysis.Usingthisapproach,weobtainaccuracyofover90.0%onourtestcollection.


Unsupervised categorization method of graphemes on handwritten manuscripts: application to style recognitionH.Daher,D.Gaceb,V.Eglin,S.Bres,N.Vincent,InstitutNationaldesSciencesAppliquéesdeLyon(France)

Wepresentinthispaperafeatureselectionandweightingmethodformedievalhandwritingimagesthatreliesoncodebooksofshapesofsmallstrokesofcharacters(graphemesthatareissuedfromthedecompositionofmanuscripts).Thesecodebooksareimportanttosimplifytheautomationoftheanalysis,themanuscriptstranscriptionandtherecognitionofstylesorwriters.Ourapproachprovidesaprecisefeaturesweightingbygeneticalgorithmsandahigh-performancemethodologyforthecategorizationoftheshapesofgraphemesbyusinggraphcoloringintocodebookswhichareappliedinturnonCBIRinamixedhandwritingdatabasecontainingdifferentpagesfromdifferentwriters,periodsofthehistoryandquality.Weshowhowthecouplingofthesetwomechanisms“featuresweighting-graphemesclassification”canofferabetterseparationoftheformstobecategorizedbyexploitingtheirgrapho-morphological,theirdensityandtheirsignificantorientationsparticularities.


Retrieving handwriting by combining word spotting and manifold rankingS.PeñaSaldarriaga,Synchromedia(Canada);E.Morin,C.Viard-Gaudin,Univ.deNantes(France)

Onlinehandwrittendata,producedwithTabletPCsordigitalpens,consistsinasequenceofpoints(x,y).Astheamountofdataavailableinthisformincreases,algorithmsforretrievalofonlinedataareneeded.Wordspottingisacommonapproachusedfortheretrievalofhandwriting.However,fromaninformationretrieval(IR)perspective,wordspottingisaprimitivekeywordbasedmatchingandretrievalstrategy.Weproposeaframeworkforhandwritingretrievalwhereanarbitrarywordspottingmethodisused,andthenamanifoldrankingalgorithmisappliedontheinitialretrievalscores.Experimentalresultsonadatabaseofmorethan2,000handwrittennewswiresshowthatourmethodcanimprovetheperformancesofastate-of-the-artwordspottingsystembymorethan10%.


The A2iA French handwriting recognition system at the Rimes-ICDAR2011 competitionF.Menasri,J.Louradour,A.Bianne-Bernard,C.Kermorvant,A2iASA(France)

ThispaperdescribesthesystemfortherecognitionofFrenchhandwritingsubmittedbyA2iAtothecompetitionorganizedatICDAR2011usingtheRimesdatabase.

Thissystemiscomposedofseveralrecognizersbasedonthreedifferentrecognitiontechnologies,combinedusinganovelcombinationmethods.

Aframeworkmulti-wordrecognitionbasedonweightedfinitestate


transducersispresented,usinganexplicitwordsegmentation,acombinationofisolatedwordrecognizersandalanguagemodel.

Thesystemwastestedbothforisolatedwordrecognitionandformulti-wordlinerecognitionandsubmittedtotheRIMES-ICDAR2011competition.

Thissystemoutperformedallpreviouslyproposedsystemsonthesetasks.


Using connected component decomposition to detect straight line segments in documentsX.Feng,A.Youssef,TheGeorgeWashingtonUniv.(UnitedStates)

Straightlinesegmentdetectionindigitaldocumentshasbeenstudiedextensivelyforthepastfewdecades.Oneofthechallengesistodetectlinesegmentswithoutprioriinformationaboutdocumentimagesandrendergoodresultswithoutmuchparametercalibration.Inthispaper,weintroduceanovelalgorithmthatissimplebuteffectiveindetectingstraightlinesegmentsinscanneddocuments.OurConnectedComponentDecomposition(CCD)approachfirstdecomposestheconnectedcomponentsbasedonthegradientdirectionoftheedgecontours,andthenusesChebyshev’sinequalitytostatisticallydistinguishlinesfromcharacters,followedbyasimplepostprocessingsteptoexaminestraightnessofremainsegments.ThisCCDapproachissimpletofollowandfastinitsimplementation,anditshighaccuracyandusabilityaredemonstratedempiricallyonapracticaldatasetwithlargevarieties.


A synthetic document image dataset for developing and evaluating historical document processing methodsD.D.WalkerIV,W.B.Lund,E.K.Ringger,BrighamYoungUniv.(UnitedStates)

DocumentimagesaccompaniedbyOCRoutputtextandgroundtruthtranscriptionsareusefulfordevelopingandevaluatingdocumentrecognitionandprocessingmethodsforhistoricaldocuments.Additionally,researchintoimprovingtheperformanceofsuchmethodsoftenrequiresfurtherannotation(e.g.,topicaldocumentlabels).However,transcribingandlabelinghistoricaldocumentsisexpensive.Asaresult,existingreal-worlddatasetswithsuchresourcesarerareandrelativelysmall.Weintroducesyntheticdocumentimagedatasetsofvaryinglevelsofnoisethathavebeencreatedfromstandardtextcorporausinganexistingdocumentdegradationmodelappliedinanovelway.IncludedinthedatasetsisOCRoutputfromthecommercialABBYYFineReaderandtheopen-sourceTesseractOCRengines.Thesesyntheticdatasetsaredesignedtoexhibitthecharacteristicsofanexamplereal-worlddocumentimagedataset,theEisenhowerCommuniques.Thenewdatasetsalsobenefitfromadditionalmetadatathatexistduetothenatureoftheircollectionandpriorlabelingefforts.WedemonstratetheusefulnessofthesyntheticdatasetsbytraininganOCRcorrectionmodelonthesyntheticdataandthenapplyingthemodeltoreduceworderrorratesonhistoricaldocuments.Thesyntheticdatasetswillbemadeavailableforusebyotherresearchers.

8297-01, Session 1

Large scale visual semantic extractionS.Bengio,GoogleInc.(UnitedStates)

Imageannotationisthetaskofprovidingtextualsemantictonewimages,byrankingalargesetofpossibleannotationsaccordingtohowtheycorrespondtoagivenimage.Inthelargescalesetting,therecouldbemillionsofimagestoprocessandhundredsofthousandsofpotentialdistinctannotations.Inordertoachievesuchataskwe

proposetobuildaso-called“embeddingspace”,intowhichbothimagesandannotationscanbeautomaticallyprojected.Insuchaspace,onecanthenfindthenearestannotationstoagivenimage,orannotationssimilartoagivenannotation.Onecanevenbuildavisio-semantictreefromtheseannotations,thatcorrespondstohowconcepts(annotations)aresimilartoeachotherwithrespecttotheirvisualcharacteristics.Suchatreewillbedifferentfromsemantic-onlytrees,suchasWordNet,whichdonottakeintoaccountthevisualappearanceofconcepts.Weproposeanewlearning-to-rankapproachthatcanscaletosuchdatasetandshowsomeannotationresults.

8297-02, Session 2

Graphical image classification combining an evolutionary algorithm and binary particle swarm optimizationB.Cheng,R.Wang,MissouriUniv.ofScienceandTechnology(UnitedStates);S.K.Antani,NationalLibraryofMedicine(UnitedStates);R.J.Stanley,MissouriUniv.ofScienceandTechnology(UnitedStates);G.R.Thoma,NationalLibraryofMedicine(UnitedStates)

Biomedicaljournalarticlescontainavarietyofimagetypesthatcanbebroadlyclassifiedintotwocategories:regularimages,andgraphicalimages.Graphicalimagescanbefurtherclassifiedintofourclasses:diagrams,statisticalfigures,flowcharts,andtables.Automaticfiguretypeidentificationisanimportantsteptowardimprovedmultimodal(text+image)informationretrievalandclinicaldecisionsupportapplications.Thispaperdescribesafeature-basedlearningapproachtoautomaticallyidentifythesefourgraphicalfiguretypes.WeapplyEvolutionaryAlgorithm(EA),BinaryParticleSwarmOptimization(BPSO)andahybridofEAandBPSO(EABPSO)methodstoanoptimalsubsetofextractedimagefeaturesthatarethenclassifiedusingaSupportVectorMachine(SVM)classifier.Evaluationperformedon1038figureimagesextractedfromtenBioMedCentral®journalswiththefeaturesselectedbyEABPSOyieldedclassificationaccuracyashighas87.5%.

8297-03, Session 2

Combining SVM classifiers to identify investigator name zones in biomedical articlesJ.Kim,D.X.Le,G.R.Thoma,NationalLibraryofMedicine(UnitedStates)

ThispaperdescribesanautomatedsystemtolabelzonescontainingInvestigatorNames(IN)inbiomedicalarticles,akeyiteminaMEDLINE®citation.ThecorrectidentificationofthesezonesisnecessaryforthesubsequentextractionofINfromthesezones.AhierarchicalclassificationmodelisproposedusingtwoSupportVectorMachine(SVM)classifiers.ThefirstclassifierisusedtoidentifyanINzonewithhighestconfidence,andtheotherclassifieridentifiestheremainingINzones.Eightsetsofwordlistsarecollectedtotrainandtesttheclassifiers,eachsetcontainingcollectionsofwordsrangingfrom100to1,200.Experimentsbasedonatestsetof105journalarticlesshowaPrecisionof0.88,0.97Recall,0.92F-Measure,and0.99Accuracy.

8297-04, Session 2

Comprehensive color segmentation system for noisy digitized documents to enhance text extractionA.Ouji,Y.Leydier,F.LeBourgeois,InstitutNationaldesSciencesAppliquéesdeLyon(France)

Thispaperpresentsanovel,generalpurposeandmulti-applicationscolorsegmentationsystemprovidingoptimalchromaticandachromaticlayersandfilteringthehueandilluminationdistortions,withminimalinformation

Conference 8297


loss.Atextextractionmethodbasedontheresultingsegmentationisproposedtoillustratetheusefulnessofthemethod.Thesystemisvalidatedthroughtheevaluationofawell-knowncommercialOCRlinesegmentationperformancesontheprocessedimages.

8297-05, Session 2

Ensemble methods with simple features for document zone classificationT.Obafemi-Ajayi,G.Agam,B.Xie,IllinoisInstituteofTechnology(UnitedStates)

Documentlayoutanalysisisoffundamentalimportancefordocumentimageunderstandingandinformationretrieval.Itrequirestheidentificationofblocksextractedfromadocumentimageviafeaturesextractionandblockclassification.

Inthispaper,wefocusontheclassificationoftheextractedblocksintofiveclasses:text(machineprinted),handwriting,graphics,

images,andnoise.Weproposeanewsetoffeaturesforefficientclassificationsoftheseblocks.Wepresentacomparativeevaluationofthreeensemblebasedclassificationalgorithms(boosting,bagging,andcombinedmodeltrees)inadditiontootherknownlearningalgorithms.

Experimentalresultsaredemonstratedforasetof36503zonesextractedfrom416documentimageswhichwererandomlyselectedfromthetobaccolegacydocumentcollection.

TheresultsobtainedverifytherobustnessandeffectivenessoftheproposedsetoffeaturesincomparisontothecommonlyusedOcropusrecognitionfeatures.WhenusedinconjunctionwiththeOcropusfeatureset,wefurtherimprovetheperformanceoftheblockclassificationsystemtoobtainaclassificationaccuracyof99.21%.

8297-06, Session 3

A robust omnifont open-vocabulary arabic OCR system using pseudo-2D-HMMA.M.Rashwan,CairoUniv.(Egypt)andRDI(Egypt);M.A.Rashwan,S.Abdou,A.Abdel-Hameed,CairoUniv.(Egypt)

Recognizingolddocumentsishighlydesirablesincethedemandforquicklysearchingmillionsofarchiveddocumentshasrecentlyincreased.UsingHiddenMarkovModels(HMMs)havebeenproventobeagoodsolutiontotacklethemainproblemsofrecognizingtypewrittenArabiccharacters.TheseattemptshoweverachievedaremarkablesuccessforomnifontOCRunderveryfavorableconditions,theydidn’tachievethesameperformanceinpracticalconditions,i.e.noisydocuments.Inthispaperwepresentanomnifont,large-vocabularyArabicOCRsystemusingPseudoTwoDimensionalHiddenMarkovModel(P2DHMM),whichisageneralizationoftheHMM.P2DHMMoffersmoreefficientwaytomodeltheArabiccharacters,suchmodelofferbothminimaldependencyonthefontsize/style(omnifont),andhighlevelofrobustnessagainstnoise.TheevaluationresultsofthissystemareverypromisingcomparedtoHMMsystemandbestOCRsavailableinthemarket(SakhrandNovoDynamics).TherecognitionaccuracyoftheP2DHMMclassifierismeasuredagainstclassicHMMclassifier,theaveragewordaccuracyratesforP2DHMMandHMMclassifiersare79%and66%respectively.TheoverallsystemaccuracyismeasuredagainstSakhrandNovoDynamicsOCRsystems,theaveragewordaccuracyratesforP2DHMM,NovoDynamics,andSakhrare74%,71%,and61%respectively.

8297-07, Session 3

Variable length and context-dependent HMM letter form models for Arabic handwritten word recognitionA.Bianne-Bernard,A2iASA(France)andTelecomParisTech(France);F.Menasri,A2iASA(France);L.Likforman-Sulem,TelecomParisTech(France);C.Mokbel,Univ.ofBalamand

(Lebanon);C.Kermorvant,A2iASA(France)

WepresentinthispaperanHMM-basedrecognizerfortherecognitionofunconstrainedArabichandwrittenwords.Therecognizerisacontext-dependentHMMwhichconsidersvariabletopologyandcontextualinformationforabettermodelingofwritingunits.WeproposeanalgorithmtoadaptthetopologyofeachHMMtothecharactertobemodeled.Formodelingthecontextualunits,astate-tyingprocessbasedondecisiontreeclusteringisintroducedwhichsignificantlyreducesthenumberofparameters.Decisiontreesarebuiltaccordingtoasetofexpert-basedquestionsonhowcharactersarewritten.Questionsaredividedintoglobalquestionsyieldinglargerclustersandprecisequestionsyieldingsmallerones.WeapplythismodelingtotherecognitionofArabichandwrittenwords.ExperimentsconductedontheOpenHaRT2010databaseshowthatvariablelengthtopologyandcontextualinformationsignificantlyimprovestherecognitionrate.

8297-08, Session 3

Post-processing for offline Chinese handwritten character string recognitionY.Wang,X.Ding,C.Liu,TsinghuaUniv.(China)

OfflineChinesehandwrittencharacterstringrecognitionisoneofthemostimportantresearchfieldsinpatternrecognition.Duetothefreewritingstyle,largevariabilityincharactershapesanddifferentgeometriccharacteristics,Chinesehandwrittencharacterstringrecognitionisachallengingproblemtodealwith.However,amongthecurrentmethodsover-segmentationandmergingmethodwhichintegratesgeometricinformation,characterrecognitioninformationandcontextualinformation,showsapromisingresult.Itisfoundexperimentallythatalargepartoferrorsaresegmentationerrorandmainlyoccuraroundnon-Chinesecharacters.InaChinesecharacterstring,therearenotonlywidecharactersnamelyChinesecharacters,butalsonarrowcharacterslikedigitsandlettersofthealphabet.Thesegmentationerrorismainlycausedbyuniformgeometricmodelimposedonallsegmentedcandidatecharacters.Tosolvethisproblem,postprocessingisemployedtoimproverecognitionaccuracyofnarrowcharacters.Ononehand,multi-geometricmodelsareestablishedforwidecharactersandnarrowcharactersrespectively.Undermulti-geometricmodelsnarrowcharactersarenotpronetobemerged.Ontheotherhand,toprankrecognitionresultsofcandidatepathsareintegratedtoboostfinalrecognitionofnarrowcharacters.Thepostprocessingmethodisinvestigatedontwodatasets,intotal1405handwrittenaddressstrings.Thewidecharacterrecognitionaccuracyhasbeenimprovedlightlyandnarrowcharacterrecognitionaccuracyhasbeenincreasedupby10.41%and10.03%respectively.Itindicatesthatthepostprocessingmethodiseffective.

8297-09, Session 3

Complexity reduction with recognition rate maintained for online handwritten Japanese text recognitionJ.Gao,TokyoUniv.AgricultureandTechnology(Japan)

Thepaperpresentscomplexityreductionofanon-linehandwrittenJapanesetextrecognitionsystembyselectinganoptimaloff-linerecognizerincombinationwithanon-linerecognizer,geometriccontextevaluationandlinguisticcontextevaluation.Theresultisthatasurprisinglysmalloff-linerecognizer,whichaloneisweak,producesthenearlybestrecognitionrateincombinationwithotherevaluationfactorsinremarkablysmallspaceandtimecomplexity.Generallyspeaking,lowerdimensionswithlessprinciplecomponentsproduceasmallersetofprototypes,whichreducememory-costandtime-cost.Itdegradestherecognitionrate,however,sothatweneedtocompromisethem.Inanevaluationfunctionwiththeabove-mentionedmultiplefactorscombined,theconfigurationofonly50dimensionswithaslittleas5principlecomponentsfortheoff-linerecognizerkeepsalmostthebestaccuracy98.85%(thebestaccuracy98.91%)fortextrecognitionwhileitsuppressesthetotalmemory-costfrom99.4MBdownto32MB

Conference 8297


andtheaveragetime-costofcharacterrecognitionfortextrecognitionfrom0.1621msto0.1191mscomparedwiththetraditionaloff-linerecognizerwith160dimensionsand50principlecomponents.

8297-10, Session 3

Improving isolated and in-context classication of handwritten charactersV.Mazalov,S.M.Watt,TheUniv.ofWesternOntario(Canada)

Earlierworkhasshownhowtorecognizehandwrittencharactersbyrepresentingcoordinatefunctionsorintegralinvariantsastruncatedorthogonalseries.TheseriesbasisfunctionsareorthogonalpolynomialsdenedbyaLegendre-Sobolevinnerproduct.Ithasbeenshownthatthefreeparameterintheinnerproduct,the“jetscale”,hasanimpactonrecognitionbothusingcoordinatefunctionsandintegralinvariants.Thispaperdevelopsmethodsofimprovingseries-basedrecognition.Forisolatedclassication,therstconsiderationistoidentifyoptimalvaluesforthejetscaleindierentsettings.Forcoordinatefunctions,wendtheoptimumtobeinasmallintervalwiththeprecisevaluenotstronglycorrelatedtothegeometriccomplexityofthecharacter.Forintegralinvariants,usedinorientation-independentrecognition,wendtheoptimalvaluesofthejetscalearedierentforeachinvariantandconrmthateachoftheseoptimaisindependentoftheorientation.Furthermore,weexaminetheoptimaldegreeforthetruncatedseries.Forin-contextclassication,wedeveloparotation-invariantalgorithmthattakesadvantageofsequencesofsamplesthataresubjecttosimilardistortion.Thealgorithmyieldssignicantimprovementoverorientation-independentisolatedrecognitionandcanbeextendedtoshearand,moregenerally,anetransformations.

8297-11, Session 4

Using specific evaluation for comparing and combining competing algorithms: applying it to table column detectionA.C.C.Silva,Univ.doPorto(Portugal)

Itisacommonlyusedevaluationstrategytoruncompetingalgorithmsonatestdatasetandstatewhichperformsbetterinaverageonthewholeset.Wecallthistextit{genericevaluation}.Althoughitisimportant,webelievethistypeofevaluationisincomplete.

Inthispaper,weshallproceedtoproposeamethodologyforalgorithmcomparison,whichwecalltextit{specificevaluation}.Thisapproachdoesnotlimititselftotakingtheaverageperformanceoftwoormorealgorithmsonagivenunseendataset,butinsteadattemptstoidentifysubsetsofthedatawhereonealgorithmisbetterthantheother.Weshallbeapplyingspecificevaluationtoanexperimentthataimsatgroupingpre-obtainedtablecellsintocolumns;wedemonstratehowitidentifiesasubsetofdataforwhichtheon-averageleastgoodbutfasteralgorithmisequivalentorbetter,andthenmanagestocreateapolicyforcombiningthetwocomplementarytablecolumndelimitationalgorithms,thusexploitingthestrengthsofeach.

8297-12, Session 4

Identification of embedded mathematical formulas in PDF documents using SVMX.Lin,L.Gao,Z.Tang,PekingUniv.(China);X.Hu,BeiHangUniv.(China);X.Lin,Vobile,Inc.(UnitedStates)

WiththetremendouspopularityofPDFformat,recognizingmathematicalformulasinPDFdocumentsbecomesanewandimportantproblemindocumentanalysisfield.Inthispaper,wepresentamethodofembeddedmathematicalformulaidentificationinPDFdocuments,basedonSupportVectorMachine(SVM).Themethodfirstsegmentstextlinesintowords,andthenclassifieseachwordintotwoclasses,namelyformulaorordinarytext.Variousfeaturesofembeddedformulas,includinggeometriclayout,characterandcontext

content,areutilizedtobuildarobustandadaptableSVMclassifier.Embeddedformulasarethenextractedthroughmergingthewordslabeledasformulas.Experimentalresultsshowgoodperformanceoftheproposedmethod.Furthermore,themethodhasbeensuccessfullyincorporatedintoacommercialsoftwarepackageforlarge-scalee-Bookproduction.

8297-13, Session 4

Chemical structure recognition: a rule-based approachN.M.Sadawi,A.P.Sexton,V.Sorge,TheUniv.ofBirmingham(UnitedKingdom)

Inchemicalliteraturemuchinformationisgivenintheformofdiagramsdepictingmolecules.Inordertoaccessthisinformationdiagramshavetoberecognisedandtranslatedintoaprocessableformat.Wepresentanapproachthatmodelstheprincipalrecognitionstepsformoleculediagramsinastrictlyrulebasedsystem,providingrulestoidentifythemaincomponents-atomsandbonds-aswellastoresolvepossibleambiguities.Theresultoftheprocessisatranslationintoagraphrepresentationthatcanbeusedforfurtherprocessing.Weshowtheeffectivenessofourapproachbydescribingitsembeddingintoafullrecognitionsystemandpresentanexperimentalevaluationthatdemonstrateshowourcurrentimplementationoutperformstheleadingopensourcesystemcurrentlyavailable.

8297-14, Session 4

Quantify spatial relations to discover handwritten graphical symbolsJ.Li,H.Mouchère,C.Viard-Gaudin,Univ.deNantes(France)

Tomodelahandwrittengraphicallanguage,spatialrelationsdescribehowthestrokesarepositionedinthe2-dimensionalspace.Mostofexistinghandwritingrecognitionsystemsmakeuseofsomepredefinedspatialrelations.However,consideringacomplexgraphicallanguage,itishardtoexpressmanuallyallthespatialrelations.Anotherpossibilitywouldbetouseaclusteringtechniquetodiscoverthespatialrelations.Inthispaper,wediscusshowtocreatearelationalgraphbetweenstrokes(nodes)labeledwithgraphemesinagraphicallanguage.Thenwevectorizespatialrelations(edges)forclusteringandquantization.Asthetargetedapplication,weextracttherepetitivesub-graphs(graphicalsymbols)composedofgraphemesandlearnedspatialrelations.Ontwohandwritingdatabases,asimplemathematicalexpressiondatabaseandacomplexflowchartdatabase,theunsupervisedspatialrelationsoutperformthepredefinedspatialrelations.Inaddition,wevisualizethefrequentpatternsontwotext-linescontainingChinesecharacters.

8297-15, Session 5

Language modeling for information retrievalC.Manning,StanfordUniv.(UnitedStates)

Noabstractavailable

8297-16, Session 6

Automatic indexing of scanned documents: a layout-based approachD.Esser,D.Schuster,K.Muthmann,A.Schill,TechnischeUniv.Dresden(Germany)

Archivingofficialwrittendocumentssuchasinvoices,remindersandaccountstatementsinbusinessandprivateareagetsmoreandmoreimportant.Creatingappropriateindexentriesfordocumentarchiveslikesender’sname,creationdateordocumentnumberisatedious

Conference 8297


manualwork.Wepresentanovelapproachtohandleautomaticindexingofdocumentsbasedongenericpositionalextractionofindexterms.Forthispurposeweapplytheknowledgeofdocumenttemplatesstoredinacommonfulltextsearchindextofindindexpositionsthatweresuccessfullyextractedinthepastandusethemtogainrelevantinformationoutofnewdocuments.Therefore,weimplementedatemplatedetectionandinformationextractionalgorithmandcombinedbothtoaworkflowforrobustandfastinformationextractionusingthelayoutofbusinessdocuments.Onarealworldsetofbusinesscorrespondences,ourlayout-basedmethodachievesextractionratesupto95%,dependingontheconcreteindextype.Moreover,evenifthestructureofdocumentschangesovertime,ourapproachguaranteeshighratesduetoafastlearninguserfeedbackmechanism.

8297-17, Session 6

Layout-based substitution tree indexing and retrieval for mathematical expressionsT.Schellenberg,R.Zanibbi,B.Yuan,RochesterInstituteofTechnology(UnitedStates)

Wepresentanewmethodforindexingandretrievingmathematicalexpressionsinalayout-basedrepresentation(LaTeX)usingsubstitutiontrees.Substitutiontreesallowustoindexexpressionsefficientlyandgroupthembasedonsimilarity.Searchingthroughtheindexreturnsidenticalexpressions,expressionswithsimilarsymbolsandsymbollayout,andexpressionswithsharedsub-expressions.InanexperimentwetestthesearchresultsfromournovelsystemandshowthatitiscomparabletoanexistingLucenesystem.

8297-18, Session 7

Efficient cost-sensitive human-machine collaboration for off-line signature verificationJ.Coetzer,J.Swanepoel,StellenboschUniv.(SouthAfrica);R.Sabourin,EcoledeTechnologieSupérieure(Canada)

Weproposeanovelstrategyfortheoptimalcombinationofhumanandmachinedecisionsinacost-sensitiveenvironment.Theproposedalgorithmshouldbeespeciallybeneficialtofinancialinstitutionswhereoff-linesignatures,eachassociatedwithaspecifictransactionvalue,requireauthentication.Whenpresentedwitha

collectionofgenuineandfraudulenttrainingsignatures,producedbyso-calledguineapigwriters,theproficiencyofaworkforceofhumanemployeesandascore-generatingmachinecanbeestimatedandrepresentedinreceiveroperatingcharacteristic(ROC)space.UsingasetofBooleanfusionfunctions,themajorityvotedecisionofthehumanworkforceiscombinedwitheachthreshold-specificmachine-generateddecision.TheperformanceofthecandidateensemblesisestimatedandrepresentedinROCspace,whereafteronlytheoptimalensemblesandassociateddecisiontreesareretained.Whenpresentedwithaquestionedsignaturelinkedtoanarbitrarywriter,thesystemfirstusestheROC-basedcostgradientassociatedwiththetransactionvaluetoselecttheensemblethatminimisestheexpectedcost,andthenusesthecorrespondingdecisiontreetoauthenticatethesignatureinquestion.Weshowthat,whenutilisingtheentirehumanworkforce,theincorporationofamachinestreamlinestheauthenticationprocessanddecreasestheexpectedcostforalloperatingconditions.

8297-19, Session 7

Questioned document workflow for handwriting with automated toolsS.N.Srihari,K.Das,H.Srinivasan,Univ.atBuffalo(UnitedStates)

Duringthelastfewyearsmanydocumentrecognitionmethodshavebeendevelopedtodeterminewhetherahandwritingspecimencan

beattributedtoaknownwriter.However,inpractice,thework-flowofthedocumentexaminercontinuestobemanual-intensive.Beforeasystematicorcomputational,approachcanbedeveloped,anarticulationofthestepsinvolvedinhandwritingcomparisonisneeded.Wedescribetheworkflowofhandwrittenquestioneddocumentexamination,asdescribedinastandardsmanual,andthestepswhereexistingautomationtoolscanbeused.Awell-knownransomnotecaseisconsideredasanexample,whereoneencounterstestingformultiplewritersofthesamedocument,determiningwhetherthewritingisdisguised,knownwritingisformalwhilequestionedwritingisinformal,etc.Thefindingsfortheparticularransomnotecaseusingthetoolsaregiven.Alsoobservationsaremadefordevelopingamorefullyautomatedapproachtohandwritingexamination.

8297-20, Session 7

Iterative analysis of document collections enables efficient human-initiated interactionJ.Chazalon,B.Coüasnon,InstitutNationaldesSciencesAppliquéesdeRennes(France)

Documentanalysisandrecognitionsystemsoftenfailtoproduceresultswithasufficientqualitylevelwhenprocessingoldanddamageddocumentssets,andrequiremanualcorrectionstoimproveresults.Thispaperpresentshow,usingtheiterativeanalysisofdocumentpageswerecentlyproposed,wecanimplementaspontaneousinteractionmodel,suitableformassdocumentprocessing.Itenableshumanoperatorstodetectandcorrecterrorsmadebytheautomaticsystem,andreintegratesthecorrectionstheymadeintosubsequentanalysisstepsoftheiterativeanalysisprocess.Then,apageanalyzercanreprocesserroneouspartsandthosewhichdependonthem,avoidingthenecessitytomanuallyfixduringpost-processingalltheconsequencesoferrorsmadebytheautomatedsystem.Afterpresentingtheglobalsystemarchitectureandaprototypeimplementationofourproposal,weshowthatdocumentmodelcanbesimplyenrichedtoenablethespontaneousinteractionmodelwepropose,andweuseitinapracticalexampletocorrectunder-segmentationissuesduringthelocalizationofnumbersindocumentsfromthe18thcentury.Evaluationsweconductedontheexamplecaseshow,on50pagescontaining1637numberstolocalize,thattheinteractionmodelweproposecanreducehumanworkloadwhileimprovinglocalizationquality(+7.5%zoneswelldetected)whencomparedtoamanualpost-processing.

8297-21, Session 7

VeriClick: an efficient tool for table format verificationG.Nagy,M.Tamhankar,RensselaerPolytechnicInstitute(UnitedStates)

Theessentiallayoutattributesofavisualtablecanbedefinedbythelocationoffourcriticalgridcells.Althoughthesecriticalcellscanoftenbelocatedbyautomatedanalysis,somemeansofhumaninteractionisnecessaryforcorrectingresidualerrors.VeriClickisamacro-enabledspreadsheetinterfacethatprovidesground-truthing,confirmation,correction,andverificationfunctionsforCSVtables.Alluseractionsarelogged.ExperimentalresultsofsevensubjectsononehundredtablessuggestthatVeriClickcanprovideatentotwenty-foldspeedupoverperformingthesamefunctionswithstandardspreadsheeteditingcommands.

8297-22, Session 7

Asymptotic cost in document conversionD.Blostein,Queen’sUniv.(Canada);G.Nagy,RensselaerPolytechnicInstitute(UnitedStates)

Inspiteofahundredfolddecreaseinthecostofrelevanttechnologies,theroleofdocumentimageprocessingsystemsisgraduallydeclining

Conference 8297


duetothetransitiontoanon-lineworld.Nevertheless,insomehigh-volumeapplications,documentimageprocessingsoftwarestillsavesmillionsofdollarsbyacceleratingworkflow,andsimilarlylargesavingscouldberealizedbymoreeffectiveautomationforthemultitudeoflow-volumepersonaldocumentconversions.Whilepotentialcostsavings,basedonestimatesofcostsandvalues,areadrivingforcefornewdevelopments,quantifyingsuchsavingsisdifficult.ThemostimportanttrendisthatthecostofcomputingresourcesforDIAisbecominginsignificantcomparedtotheassociatedlaborcosts.Aneconometrictreatmentofdocumentprocessingcomplementstraditionalperformanceevaluation,whichfocusesonassessingthecorrectnessoftheresultsproducedbydocumentconversionsoftware.Researchersshouldlookbeyondtheerrorrateforadvancingbothproductionandpersonaldocumentconversion.

8297-23, Session 8

Style comparisons in calligraphyX.Zhang,ShanghaiMaritimeInstitute(China);G.Nagy,RensselaerPolytechnicInstitute(UnitedStates)

Calligraphicstyleisconsidered,forthisresearch,tobethevisualattributesofimagesofcalligraphiccharacterssampledrandomlyfroma“work”createdbyasingleartist.Itisindependentofpagelayoutortextualcontent.Anexperimentaldesignisdevelopedtoinvestigatetowhatextentthesourceofasingle,orofafewpairs,ofcharacterimagescanbeassignedtotheeithersameworkortotwodifferentworks.Theexperimentsareconductedonthe13,571segmentedandlabeled600-dpicharacterimagesoftheCADALdatabase.Theclassifierisnottrainedontheworkstested,onlyonotherworks.Evenwhenonlyafewsamplesofsame-classpairsareavailable,thedifference-vectorofafewsimplefeaturesextractedfromeachimageofapairyieldsover80%classificationaccuracyforasame-workvs.different-workdichotomy.Whenmanypairsofdifferentclassesareavailablefortheeachpair,theaccuracy,usingthesamefeatures,isalmostthesame.Thesestyle-verificationexperimentsarepartofresearchonthelargergoalofstyleidentificationandforgerydetection.

8297-24, Session 8

An Oracle-based co-training framework for writer identification in offline handwritingU.Porwal,Univ.atBuffalo(UnitedStates);S.Rajan,FujitsuLabs.ofAmerica,Inc.(UnitedStates);V.Govindaraju,Univ.atBuffalo(UnitedStates)

Stateofthearttechniquesforwriteridentificationhasbeencenteredprimarilyonperformanceenhancementofthesystem.Machinelearningalgorithmshavebeenextensivelyusedtoimproveaccuracyofthesystemassumingavailabilityofthesufficientamountofdataforthetrainingpurpose.Littleattentionhasbeenpaidtotheprospectofharnessingtheinformationtappedinthelargeamountofun-annotateddata.ThispaperfocusesonCo-trainingbasedframeworkthatcanbeusedforiterativelabelingoftheunlabeleddatasetexploitingtheindependencebetweenthemultipleviews(features)ofthedata.Thisparadigmrelaxestheassumptionofsufficiencyofthedataavailableandtriestogeneratelabeleddatafromunlabeleddatasetalongwithimprovingtheaccuracyofthesystem.However,performanceofCo-trainingbasedframeworkisdependentontheeffectivenessoftheselectionalgorithmusedfortheselectionofdatapointstobeaddedinthelabeledset.WeproposeanOraclebasedapproachfordataselectionthatlearnsthepatternsinthescoredistributionofclassesforlabeleddatapointsandthenpredictsthelabels(writers)oftheunlabeleddatapoint.Thismethodforselectionstatisticallylearnstheclassdistributionandpredictsthemostprobableclassunliketraditionalselectionalgorithmswhichwerebasedonheuristicapproaches.WeconductedexperimentsonpubliclyavailableIAMdatasetandillustratetheefficacyofproposedapproach.

8297-25, Session 8

Handwritten document age classification based on handwriting stylesC.Ramaiah,G.Kumar,V.Govindaraju,Univ.atBuffalo(UnitedStates)

Handwritingstyleshaveshownanobservablevarianceacrosscenturies.Inthispaper,weapproachthenovelproblemofestimatingtheageofhistoricalhandwrittendocumentsusinghandwritingstyles.Thissystemwillhavefarreachingapplicationsinhandwrittendocumentprocessingengineswherespecializedprocessingtechniquescanbeappliedoncetheageofthedocumenthasbeenestimated.WeproposetolearnadistributionoverstylesacrosscenturiesusingTopicModelsandtoapplyaclassifieroverweightslearnedinordertoestimatetheageofthedocuments.WealsopresentacomparisonofdifferentdistancemetricssuchasEuclideanDistanceandHellingerDistance.

8297-26, Session 8

Handwriting individualization using distance and rarityY.Tang,S.N.Srihari,Univ.atBuffalo(UnitedStates);H.Srinivasan,JanyaInc.(UnitedStates)

Thelikelihoodratio(LR)isaquantitativemeasureofthedegreeofuncertaintyinforensicindividualization,inwhichthecentralproblemistodeterminewhethertheobservedevidencecamefromaparticularsource/object.EvaluationoftheLRbasedonthefeaturedistributionneedsalargenumberofparametersforthemodel,whichisdatademandingandcomputationallyexpensive.Onesimplificationistocomputetheratioofthelikelihoodsofthedistance.Althoughbeingsimpletocompute,thereisaseverelossofinformationingoingfromahigh-dimensionalfeaturespacetoaone-dimensionaldistance.Thispaperproposedanintermediatesolution,inwhichtheLRisdecomposedastheproductoftwofactors,distanceandrarity.Theproposedmethodwasevaluatedusingadatasetofhandwritingsamples,andcomparedwiththefeaturedistributionmethodanddistancemethods.Theresultsshowedthattheproposedmethodoutperformsthedistancemethodsandthefeaturedistributionmethodintermsofaccuracy,withacceptableefficiency.

8297-27, Session 8

Construction of language models for an handwritten mail reading systemO.Morillot,L.Likforman-Sulem,TelecomParisTech(France);E.Grosicki,DirectionGénéraledeL’armement(France)

Thispaperpresentsasystemfortherecognitionofunconstrainedhandwrittenmails.ThemainpartofthissystemisanHMMrecognizerwhichusestrigraphstomodelcontextualinformation.Thisrecognitionsystemdoesnotrequireanysegmentationintowordsorcharactersanddirectlyworksatlinelevel.Totakeintoaccountlinguisticinformationandenhanceperformance,alanguagemodelisintroduced.Thislanguagemodelisbasedonbigramsandbuiltfromtrainingdocumenttranscriptionsonly.Differentexperimentswithvariousvocabularysizesandlanguagemodelshavebeenconducted.WordErrorRateandPerplexityvaluesarecomparedtoshowtheinterestofspecificlanguagemodels,fittohandwrittenmailrecognitiontask.

Conference 8297


Conference 8298: Sensors, Cameras, and Systems for Industrial/Scientific Applications XIIIWednesday-Thursday25-26January2012

Part of Proceedings of SPIE Vol. 8298 Sensors, Cameras, and Systems for Industrial and Scientific Applications XIII


The embedded network infrared video monitoring system based on Linux OSL.Liu,NanjingUniv.ofScience&Technology(China);N.Chen,NanjingNormalUniv.(China);X.Zhou,T.Pan,NanjingUniv.ofScience&Technology(China)

Thevideomonitoringsystemcouldprovideremotevideoinformation.Itisconvenientandintuitive,soitcouldbewidelyusedinindustrial,transportation,finance,teachingandotherfields.Withthedevelopmentofcomputerandnetworktechnology,networkvideomonitoringsystemwillreplacethetraditionalmonitoringsystemandplayanimportantroleinmanyfields.

ThispaperdescribesthewaytodesignanembeddednetworkinfraredvideomonitoringsystembasedonLinuxOS.Firstly,wemakeacomparisonofthehardwaresolutionbetweensomeregularmonitoringsystems,andthendesignthehardwaresystemthatweneeded.Ourhardwaresystemusesthei.mx27processorwiththeARM9core.Secondly,thesoftwareplatformisintroducedinthispaper.TheLinuxoperatesystemisappliedinoursoftwaresolution.AccordingtothecharacteristicofLinuxOS,wedownloaduboottothedemoboard,transplanttheLinuxkernelandjffs2filesystemtotheembeddedsystem,andbrieflycompileanddownloaddrivers.Finally,theapplicationsoftwaredesignprocessisintroducedinthepaper.ThesystemcanbeusedtoencodethepicturecapturedfrominfraredCCD,andthensendthepicturetoanothersameembeddedsystemtodecodethepicture,andfinallydisplayitontheLCDandachievethegoaloftheinfraredvideo’sremotemonitoring.AstheinfraredCCDwouldnotbeaffectbythedimlight,thismonitoringsystemcouldbeusedalldaylong.


Motion blur-free time-of-flight range sensorS.Lee,B.Kang,J.D.Kim,C.Kim,SamsungAdvancedInstituteofTechnology(Korea,Republicof)

Time-of-flightdepthsensorprovidesfasterandeasierwayof3Dscenecapturingandreconstruction.Thedepthsensor,however,suffersfrommotionblurcausedbyanymovementofcameraorsubjects.Inthismanuscript,weproposeanoveldepthmotionblurpixeldetectionandeliminationmethodthatcanbeimplementedonthedepthsensorwithlightmemoryandcomputationresources.

Weproposeaneasyblurdecisionmethodusingtheplotsofchargerelationsanddetectblurpixelateachdepthvaluecalculationsteponlybycheckingthefourchargevalues.Oncewedetectblurpixels,theyarereplacedbyanyclosestnormalpixelvalues.Withthismethod,weeliminateblurpixelbeforewebuildthedepthimagewithonlyfewmorecalculationsandmemoryaddition.


CMOS buried double junction (BDJ) photodiode for trichromatic sensingL.Tu,S.V.SetlurNagesh,Univ.atBuffalo(UnitedStates);Z.Fu,OmniVisionTechnologies,Inc.(UnitedStates);A.H.Titus,Univ.atBuffalo(UnitedStates)

Anovelmethodforachievingtrichromaticcolordetectionusingasinglephotodetectorwithlessthanthreep-njunctionsispresented.

Thisnewmethodremovestheconstraintofcolorsensinginburied-double-junction(BDJ)photodiode[1],eliminatestheneedforapriorilightsourceknowledge[2]orforchangingcolorintensity[3].Afterusingavisiblelightopticalfiltertoblocklightfromoutsidethevisiblespectrum,thecolordetectionisachievedbytakingthedifferenceofdepletionregionphotocurrentgeneratedbydifferentreversebiasvoltages.This“differenceoutput”iseffectivelyfromthe“third”opticalwavelengthspecificdepletionregionrequiredfortrichromaticcolorsensing.Thismethodisbasedonexploitingtherelationshipbetweenphotonabsorptionandphotonpenetrationdepthofsilicon[5,6],andthebasicpropertyofap-njunctionphotodiode[7,8]whichstatesthatonlyphotonsabsorbedindepletionregiongeneratephoto-signals.Varyingphotodiode’sreversebiasimpliesvaryingdepletionregionwidth,whichfurtherimpliesphotonabsorptionofdifferentwavelength/frequency.ThetheoryisvalidatedbyexperimentusingaBDJphotodiodefabricatedthroughMOSISintheAMI-ABN1.5umtechnology,avisiblelightfilter,anarrowwavelengthLED,andacurrenttovoltageconverterbuiltusingoperationalamplifiers.Trichromaticcolordetectionwithpeakwavelengthof500,600,and700umphotonwavelengthisachieved.


On image sensor dynamic range utilized by security camerasA.Johannesson,AxisCommunicationsAB(Sweden)

Thedynamicrangeisanimportantquantityusedtodescribeanimagesensor.Wide/High/Extendeddynamicrangeisoftenbroughtforwardasanimportantfeaturetocompareonedevicetoanother.Thedynamicrangeofanimagesensorisnormallygivenasasinglenumber,whichisofteninsufficientsinceasinglenumberwillnotfullydescribethedynamiccapabilitiesofthesensor.

Acameraisideallybasedonasensorthatcancopewiththedynamicrangeofthescene.Otherwiseithastosacrificesomepartoftheavailabledata.Forasecuritycamerathelattermaybecriticalsinceimportantobjectsmightbehiddeninthesacrificedpartofthescene.

Inthispaperwecomparethedynamiccapabilitiesofsomeimagesensorsutilizingavisualtool.Thecomparisonisbasedontheusecase,commoninsurveillance,wherelowcontrastobjectsmayappearinanypartofascenethatthroughitsunevenilluminationspanahighdynamicrange.Theinvestigationisbasedonrealsensordatathathasbeenmeasuredinourlabandasynthetictestsceneisusedtomimicthelowcontrastobjects.Withthistechniqueitispossibletocomparesensorswithdifferentintrinsicdynamicpropertiesaswellassomecapturetechniquesusedtocreateaneffectofincreaseddynamicrange.


Design of low-noise output amplifier for p-channel: fully depleted charge-coupled devicesS.Haque,S.E.Holland,A.Karcher,W.Kolbe,N.Roe,LawrenceBerkeleyNationalLab.(UnitedStates);R.Groulx,R.Frost,F.Dion,TeledyneDALSASemiconductor(Canada);G.Wang,LawrenceBerkeleyNationalLab.(UnitedStates)

Wedescribethedesignanddevelopmentoflow-noise,single-stageoutputamplifiersforp-channelcharge-coupleddevices(CCDs)usedforscientificapplicationsinastronomyandotherfields.TheCCDsare


fabricatedonhigh-resistivity,>4000Ω-cm,n-typesiliconsubstrates.Single-stageamplifierswithdifferentoutputstructuredesignsandtechnologieshavebeencharacterized.Thestandardoutputamplifierisdesignedwithann+polysilicongatethathasametalconnectiontothesensenode.Inanefforttolowertheoutputamplifierreadoutnoisebyminimizingthecapacitanceseenatthesensenode,buriedcontacttechnologyhasbeeninvestigated.Inthiscase,theoutputtransistorhasap+polysilicongatethatconnectsdirectlytothep+sensenode.Outputstructureswithburiedcontactholesassmallas2umx2umarecharacterized.Inaddition,thesizeofthesourcefollowertransistorwasvaried.Wereporttestresultsontheconversiongainandnoiseofthevariousamplifierstructures.Byoptimizingtheamplifierbiases,resistiveloadsandtestelectronicsdesign,areadnoiseoflessthan2electronsat70kpixels/secand-140°Cisobtained,correspondingtoareductioninnoiseof45%comparedtothestandardoutputamplifierconfiguration.


S/N improvement for the optical-multiplex image-acquisition systemT.Narabu,SonyCorp.(Japan)

Theoptical-multiplexsystemiscomprisedofanimagesensor,amulti-lensarrayandsignalprocessingunit.Thekeyfeatureoftheoptical-multiplexsystemisthateachpixeloftheimagesensorcapturesmultipledataoftheobjectthroughmulti-lensesandtheobjectdataisobtainedbyprocessingtherawdataoutputfromtheoptical-multipleximagesensor.

Wearenowabletoreportthatoursystemcanimprovethesignal-to-noiseratiooftheimageoutputfromtheoptical-multiplexsystembychangingtheshadingcharacteristicsofthemulti-lensesintheoptical-multiplexsystem.Inamodelofthesystemforsimulationpurposes,anoptical-multiplexsystemwithfivelensesisused.Thefivelensesarelocatedatthecenter,upper,lower,leftandrightaboveanimagesensor.Wecalculatethesignal-to-noiseratiooftheimageoutputfromtheoptical-multiplexsystembychangingtheshadingcharacteristicsofthefourlenseslocatedattheupper,lower,leftandright.Thebestsignal-to-noiseratioofthisimageoutputbytheoptical-multiplexsystemis8.895dBbetterthanthatofacamerawithasinglelens.Thisvalueisbeyondthevalueof3.764dBwhichwereportedin2007.


Fully integrated system-on-chip for pixel-based 3D depth and scene mappingM.Popp,B.DeCoi,M.Thalmann,R.Gancarz,P.Ferrat,M.Dürmüller,F.Britt,M.Annese,M.Ledergerber,G.Catregn,ESPROSPhotonicsAG(Switzerland)

WepresentforthefirsttimeafullyintegratedSystemonChipforpixel-wise3Drangedetectionsuitedforcommercialapplications.Itisbasedonthetime-of-flight(ToF)principle,i.e.measuringthephasedifferenceofareflectedpulsetrain.Theproductepc600isfabricatedusingadedicatedprocessflow,calledEsprosPhotonicCMOS®.ThisintegrationmakesitpossibletoachieveaQuantumEfficiency(QE)of>80%inthefullwavelengthbandfrom500nmupto900nmaswellasveryhightimingprecisioninthesub-nsrangewhichisneededforexactdetectionofthephasedelay.TheSoCfeatures8x8pixelsandincludesallnecessarysub-componentssuchasToFpixelarray,voltagegenerationandregulation,non-volatilememoryforconfiguration,LEDdriverforactiveillumination,digitalSPIinterfaceforeasycommunication,columnbased12bitADCconverters,PLLanddigitaldataprocessingwithtemporarydatastorage.Thesystemcanbeoperatedatupto100framespersecond.PerformancedataofthesystemwillbeshownsuchasQE,timingcapabilities,rangeresolutionaswellastoleranceversusambientbackgroundlight.

8298-01, Session 1

High-speed VGA resolution CMOS image sensor with global shutterP.Willems,G.Vanhorebeek,C.Ma,CMOSISnv(Belgium)

CMOSISpresentsitsnewmemberintheCMV(CMOSISMachineVision)family.TheCMV300isahighspeedCMOSimagesensorwith644by484pixels(1/3opticalinch)developedformachinevisionapplications.Theimagearrayconsistsof7.4μmx7.4μmpipelinedglobalshutterpixelswhichallowexposureduringreadout,whileperformingCDSoperation.Theimagesensorhasfour8-,10-or12-bitdigitalLVDSoutputs(serial)orone10bitparallelCMOSoutput.Theimagesensoralsointegratesaprogrammablegainamplifierandoffsetregulation.EachLVDSchannelrunsat600Mbpsmaximumwhichresultsin600fpsframerateatfullresolution.Higherframeratescanbeachievedinrow-windowingmodeorrow-subsamplingmode.ThesemodesareallprogrammableusingtheSPIinterface.Allinternalexposureandreadouttimingsaregeneratedbyaprogrammableon-boardsequencer.Externaltriggeringandexposureprogrammingisalsopossible.Extendedopticaldynamicrangecanbeachievedbymultipleintegratedhighdynamicrangemodes.ThepackagedBGAsensorsizeisonly7mmx7mmx0.7mmwhichallowsforverysmallandcosteffectivecameras.Architecture,buildingblocks,pixelandcolumnADCaredescribedintodetailinthepaper.

8298-02, Session 1

High-speed global shutter CMOS machine vision sensor with high-dynamic range image acquisition and embedded intelligenceÁ.B.Rodríguez-Vázquez,Univ.deSevilla(Spain);R.Dominguez-Castro,F.Medeiro,F.Jimenez-Garrido,Anafocus(Spain)

High-speedimagersarerequiredforindustrialapplications,trafficmonitoring,roboticsandunmannedvehicles,movies,etc.Manyoftheseapplicationsrequirealsohighspatialresolution,highsensitivityandtheabilitytodetectimageswithlargeintra-framedynamicrange.ThispaperreportsaCISintelligentdigitalimagesensorwith5.2Mpxlwhichdeliverseither12-,10-,or8-bitfully-correctedimagesat250Fps.Thenewsensorincorporatessophisticatedfunctionson-chip,suchas:windowing;pixelbinning;sub-sampling;combinedwindowing-binning-sub-samplingmodes;fixed-patternnoisecorrection;finegainandoffsetcontrol;andcolourprocessing.TheseandotherCISfunctionsareprogrammablethroughasimplefour-wireserialportinterface(SPI).

8298-03, Session 1

High-speed CMOS image sensor for high-throughput lensless microfluidic imaging system with point-of-care applicationM.Yan,X.Huang,R.Nadipalli,Y.Shang,H.Yu,NanyangTechnologicalUniv.(Singapore);M.Je,A*STARInstituteofMicroelectronics(Singapore);C.Xu,OmniVisionTechnologies,Inc.(UnitedStates);K.Yeo,NanyangTechnologicalUniv.(Singapore)

Theminiaturizedlenslessmicrofluidicbio-imagingsystemhasgreatpotentialtoreplaceexistingbulkyandexpensivecliniclaboratoryinstrumentsfordetecting,counting,andsortingbiomedicalsamples.Theprimarychallengeistodesigncompactandhigh-speedCMOSimagersensorforthesystemintegration.ThispaperhasdemonstrateddesignofthefirstCMOSimagerchipforhigh-throughputflow-cytometerbioimaginingsystem.Thechipisfabricatedin0.18um1P6MCMOSprocess.Comparedtooff-the-shelfimagesensor,theproposednovelimagesensorhasthreeuniquefeatures:(1)highsensitivityandbetterSNRachievedbyoptimizing10um-sizepixelwithfillfactorof60%.Super-resolutionimaginganalysisalgorithmovercomethepixelphysicalresolutionlimitationandbeabletoreconstruct2um-pixel

Conference 8298


image;(2)column-parallelreadoutarchitecturewithDCDStechniquetoachievebothultra-highspeed(upto1000frames/sec)andlownoiseatthesametime;(3)areal-timeon-chipimagingprocessingisimplementedwithsystem-on-a-chipapproachtoboostthesensorperformance.Inconclusion,thispresentedimagesensorchipdesignwillpavethewayforimplementationofcommercializeautonomous,potableandlow-costmicrofluidiclab-on-chipplatform,suchasHIVdiagnosticsandmonitoring,circulatingtumorcellcapturing,sepsisandbacterialpathogendetectionetc.

8298-04, Session 2

Smart image sensor with adaptive correction of brightnessM.Paindavoine,A.Ngoua,O.Brousse,C.Clerc,Univ.deBourgogne(France)

Today,intelligentimagesensorsrequiretheintegrationinthefocalplane(ornearthefocalplane)ofcompleximageprocessingalgorithms.Suchdevicesmustmeettheconstraintsrelatedtothequalityofacquiredimages,speedandperformanceofembeddedprocessing,aswellaslowpowerconsumption.Toachievetheseobjectives,analogpre-processing,ontheonehand,improvethequalityoftheimagesmakingthemusablewhateverthelightconditions,andsecondly,detectregionsinterest(ROIs)inordertolimittheamountofpixelstobetransmittedtoadigitalprocessorperformingthehigh-levelprocessingsuchasfeatureextractionforpatternrecognition.Inthiscontext,ourgoalwastodesignanintelligentimagesensorprototype,built-in130nmCMOStechnology,andtoachieveatripleanalogfunction:imageacquisitioninmulti-exposuremode,detectionofimageblockswithaconstantaveragebrightnessandhighdynamicrangeimagegeneration.

8298-05, Session 2

Algorithm architecture co-design for ultra low-power image sensorT.Laforest,A.Dupret,A.Verdant,D.Lattard,P.Villard,CEA-LETI(France)

Standaloneleftbehindimagesensorareusedtoachievethedetectionofeventswithverylowpowerconsumptionandinarelevantway.Usingasteadycamera,motiondetectionalgorithmsbasedonstaticsceneestimationtofindregionsinmovementaresimpletoimplementandcomputationallyefficient.Theirimplementationhadalreadybeendoneonanintegratedarchitecturewithprogrammableanalogprocessors.Inordertoextendtheclassofobjectstobedetected,weproposeanoriginalmixedmodearchitecturedevelopedthankstoanalgorithmsandarchitectureco-designmethodology.ThatprogrammablearchitectureiscomposedofavectorofSIMDprocessorswhichareabletoprocessimageshighdefinitionof1920x1080pixels.However,thebackgroundisestimatedusingadownsampledimageformedofmacropixelstoreducepowerconsumption.AbasicRISCarchitecturewasoptimizedinordertoimplementmotiondetectionalgorithmswithadedicatedsetof42instructions.Definitionofdeltamodulationasacalculationprimitivehadallowedtoimplementalgorithmsinaverycompactway.Thereby,motiondetectionisachievedwiththelowestpowerconsumptionperpixelknownuntilnow.

8298-06, Session 2

A CMOS imager using focal-plane pinhole effect for confocal multi-beam scanning microscopyM.Seo,A.Wang,Z.Li,K.Yasutomi,K.Kagawa,S.Kawahito,ShizuokaUniv.(Japan)

ACMOSimagerforconfocalmulti-beamscanningmicroscopy,wherethepixelitselfworksasapinhole,isproposed.ThisCMOSimagerissuitableforbuildingcompact,low-power,andnon-coolingconfocal

microscopesbecausethecomplexNipkowdiskwithapreciselyalignedpinholearraycanbeomitted.TheCMOSimageriscomposedofanarrayofsub-imagers,andcandetectmultiplebeamsatthesametime.Toachieveafocal-planepinholeeffect,onlyonepixelineachsub-imager,whichisattheconjugatepositionofalightspot,accumulatesthephotocurrent,andtheotherpixelsarenotreadandreset.Thisoperationisachievedby2-stageverticalandhorizontalshiftregisters.TheproposedCMOSimagerfortheconfocalmulti-beamscanningmicroscopesystemwasfabricatedin0.18-μmstandardCMOStechnologywithapinnedphotodiodeoption.Thetotalareaofthechipis8.4mm×9.4mm.Thenumberofeffectivepixelsis256(Horizontal)×256(Vertical).Thepixelarrayconsistsof32(H)×32(V)sub-imagerseachofwhichhas8(H)×8(V)pixels.Thepixelisanordinary4-transistoractivepixelsensorusingapinnedphotodiodeandthepixelsizeis7.5μm×7.5μm.Thebasicoperationssuchasnormalimageacquisitionandselectivepixelreadoutwereexperimentallyconfirmed.

8298-07, Session 2

Time-to-impact sensors in robot vision applications based on the near-sensor image processing conceptA.Astrom,CombitechAB(Sweden);R.Forchheimer,LinköpingUniv.(Sweden)

Theconceptofopticalflowhasbeenknownformorethan30years.Itisalsoknownthatthisisamemoryconsumingandcomputationallydemandingtaskrequiringpowerfulprocessors.Recently,ithasbeendescribedhowtoimplementopticalflowfortime-to-impact,TTI,detectionusingtheNear-SensorImageProcessing,NSIPconcept,whichwasdescribedforthefirsttime30yearsago,inwhichanopticalsensorarrayandaspecificlow-levelprocessingunitaretightlyintegratedintoahybridanalog-digitaldevice.Despiteitslowoverallcomplexity,numerousimageprocessingoperationscanbeperformedathighspeedcompetingfavorablywithstate-of-art.

TheresultingperformancefortheTTIalgorithmontheNSIPconceptwouldbeintheorderof10kHzoftime-to-impactcalculations.ThereasonforthehighperformanceisthattheTTIalgorithmfitsverywellintotheNSIParchitecture.TTIisdefinedasthedistancetotheobjectdividedbythespeedtowardstheobject.

InthispaperwewillpresentinmoredetailNSIP-basedTTIapplicationsforrobotvision.WewillshowthatTTIfromasinglesensorcanbeusedforcalculationoftheanglebetweentheobject’ssurface,thatweareapproaching,andthetraveldirection.WewillalsoshowthatTTIfromtwodifferentsensorscanbeusedtocalculatetheabsolutedistance.Finally,wewillshowhowtheaccumulatedinformationduringthetravellingtimewillgiveusmoredetailedinformationabouttheobject.Thiscanbedescribedas“Shape-from-TTI”.

8298-08, Session 3

A 176x144 148dB adaptive tone-mapping imagerS.Vargas-Sierra,G.Liñán-Cembrano,Á.B.Rodríguez-Vázquez,Univ.deSevilla(Spain)

ThisworkpresentsaQCIFHDRimagerwherevisualinformationissimultaneouslycapturedandadaptivelycompressedbymeansofanin-pixeltonemappingscheme.Thetonemappingcurve(TMC)iscalculatedfromasubsamplednonlinearhistogramofthepreviousimage,whichservesasaprobabilityindicatorofthedistributionofilluminationswithinthepresentframe.Thechipproduces7-bit/pixelimagesinawaythateachpixeldecideswhentostopobservingphotocurrentintegration.Itiscapableofcompressingilluminationsfrom222μluxto55392luxinasingleframe,withextremevaluescapturedat8sand2.34μsrespectively.Itimpliesaphotocurrentdynamicrangeof168dBforSNR=1and148dBforSNR=10.Pixelsusea3x3μm2Nwell-Psubstratephotodiodeandanautozeroingtechniqueforestablishingtheresetvoltage,whichcancelsoutmostoftheoffsetcontributionscreatedbytheanalogprocessingcircuitry.Measured

Conference 8298


sensitivityis5.79V/lux-s.Darksignal(10.8mV/s)effectsinthefinalimageareattenuatedbyanautomaticprogrammingoftheon-chipDACtopvoltage.Thechiphasbeendesignedinthe0.35μmOPTOtechnologyfromAMS.

8298-09, Session 3

A high-dynamic range (HDR) back-side illuminated (BSI) CMOS image sensor for extreme UV detectionX.Wang,CMOSISnv(Belgium)

Thispaperdescribesaback-sideilluminated1MegapixelCMOSimagesensormadein0.18umCMOSprocessforEUVdetection.First,thedesignofthefront-sideilluminatedsensoraredescribed,includingthelownoise,highdynamicrange(HDR)pixelarchitecture,thecolumngainstageandreadoutcircuitary.Secondly,theback-sideilluminationpost-processingstepsareexplained,includingthewafer-to-waferbonding,thinning,buried-oxide(BOX)removalandpadopening.Intheend,sensorcharacterizationresultisshown,includinggeneralelectricalopticalperformanceandEUVdetectioncharacterization.

8298-10, Session 3

A low-noise, 15µm pixel-pitch, 640x512 hybrid InGaAs image sensor for night visionF.Guellec,S.Dubois,E.deBorniol,P.Castelein,S.Martin,R.Guiguet,M.Tchagaspanian,A.Rouvie,P.Bois,CEA-LETI(France)

HybridInGaAsfocalplanearraysareveryinterestingfornightvisionbecausetheycanbenefitfromthenightglowintheShortWaveInfraredband.ThroughacollaborationbetweenIII-5LabandCEA-Leti,a15μmpixel-pitch,640x512InGaAsimagesensorwith15μmpixelpitchhasbeendeveloped.ThereadoutIC(ROIC)designinastandardCMOS0.18μmtechnologyispresented.Thepixelcircuitisbasedonacapacitivetransimpedanceamplifier(CTIA)stageandoffersaselectablegain.Theinputstageisoptimizedtodeliverlownoiseperformanceatlowlightlevelsinthehighgainmode.Inthismode,thecharge-to-voltageconversionfactorisaround17.5μV/electron.Theexposuretimecanbemaximizeduptotheframeperiodthankstoarollingshutterapproach.Theframeratecanbeupto120fpsor60fpsiftheCorrelatedDoubleSampling(CDS)capabilityofthecircuitisenabled.Thefirstresultsshowadarknoiseof90electronsforafull-wellcapacityabove105e3electrons.Theyareobtainedatroomtemperaturewithareversephotodiodebiasvoltageof100mV,themaximumexposuretime(16msat60fps)andCDS.Toourbestknowledge,thissetofresultsestablishesstate-of-the-artperformanceforthiskindofdetector.

8298-11, Session 3

High-dynamic range, 4 megapixel CMOS image sensor for scientific applicationsP.Vu,B.A.Fowler,C.Liu,S.Mims,P.Bartkovjak,H.Do,W.Li,J.Appelbaum,A.Lopez,BAESystemsImagingSolutions(UnitedStates)

Asbio-technologytransitionsfromresearchanddevelopmenttohighvolumeproduction,dramaticimprovementsinimagesensorperformancewillberequiredtosupportthethroughputandcostrequirementsofthismarket.Thisincludeshigherresolution,higherframerates,higherquantumefficiencies,increasedsystemintegration,lowerread-noise,andlowerdevicecosts.Wepresenttheperformanceofarecentlydevelopedlownoise2048(H)x2048(V)CMOSimagesensoroptimizedforscientificapplicationssuchaslifescienceimaging,microscopy,aswellasindustrialinspectionapplications.Thesensorarchitectureconsistsoftwoidenticalhalveswhichcanbeoperatedindependentlyandtheimagingarrayconsistsof4Tpixelswithpinnedphotodiodesona6.5μmpitchwithintegratedmicro-lens.

TheoperationofthesensorisprogrammablethroughaSPIinterface.Themeasuredpeakquantumefficiencyofthesensorisgreaterthan72%at600nm,andthereadnoiseisabout1.1e-RMSat100fpsdatarate.Thesensorfeaturesdualgaincolumnparallelouputamplifierswith11-bitsingleslopeADCs.Thefullwellcapacityisgreaterthan36ke-,thedarkcurrentislessthan6pA/cm2at20ºC.Thesensorachievesanintra-scenelineardynamicrangeofgreaterthan91dB(36000:1)atroomtemperature.

8298-36, Session 3

Diffusion dark current in front-illuminated CCDs and CMOS image sensorsM.M.Blouke,PortlandStateUniv.(UnitedStates)

Historicallythediffusionrelatedcomponentofthedarkcurrenthasbeenestimatedfromthediffusionrelatedcurrentofadiodewithaninfinitesubstrate.Thispaperexplorestheeffectofasubstrateoffiniteextentbeneaththecollectingvolumeofthepixelforafront-illuminateddeviceanddevelopsacorrectedexpressionforthediffusionrelateddarkcurrent.Calculationsshowthatthediffusiondarkcurrentcanbemuchless,byacoupleordersofmagnitude,thanthatpredictedbythestandardmodel.

8298-12, Session 4

Projecting the rate of in-field pixel defects based on pixel size, sensor area, and ISOG.H.Chapman,J.Leung,A.Namburete,SimonFraserUniv.(Canada);I.Koren,Z.Koren,Univ.ofMassachusettsAmherst(UnitedStates)

Imagesensorscontinuouslydevelopin-fieldpermanenthotpixeldefectsovertime.ExperimentalmeasurementsofDSLR,pointandshoot,andcellphonecameras,showthattherateofthesedefectsdependsonthetechnology(APSorCCD)andondesignparameterslikeimagerarea,pixelsize,andgain(ISO).Expandingimagesensitivity(ISO)enhancesdefectsappearanceandsometimescreatessaturation.40%ofdefectsarepartiallystuckhotpixels,withanoffsetindependentofexposuretime,andareparticularlyaffectedbyISOchanges.Comparingdifferentsensorsizeswithsimilarpixelsizesshowedthatdefectratesscalelinearlywithsensorareasuggestingthemetricofdefects/year/sqmm.Plottingthisratefordifferentpixelsizes(7.5to1.5microns)showsthatdefectratesgrowrapidlyaspixelsizeshrinks.Curvefittingshowsanempiricalpowerlawwithdefectratesproportionaltothepixelsizetothepowerof-2forCCDandtothepowerof-3.3forCMOS.At7umpixelstheCCDdefectrateis~2.5greaterthanforCMOS,butfor2umpixelstheratesareequal.ExtendingourempiricalformulatoincludeISOallowsustopredicttheexpecteddefectdevelopmentrateforawidesetofsensorparameters.

8298-13, Session 4

Dynamic CCD pixel depletion edge model and the effects on dark current productionJ.C.Dunlap,M.M.Blouke,E.Bodegom,R.Widenhorn,PortlandStateUniv.(UnitedStates)

ThedepletionedgeinCharge-CoupledDevices(CCD)pixelsisdependentupontheamountofsignalchargelocatedwithinthedepletionregion.Amodelispresentedthatdescribesthemovementofthedepletionedgewithincreasingsignalcharge.Thisdynamicdepletionedgeisshowntohaveaneffectontheamountofdarkcurrentproducedbysomepixels.Modelingthedarkcurrentbehaviorofpixelsbothwithandwithoutimpuritiesoveranentireimagerdemonstratesthatthismovingdepletionedgehasasignificanteffectonasubsetofthepixels.Darkcurrentcollectedbythesepixelsisshowntobehavenonlinearlywithrespecttoexposuretimeandadditionallythedarkcurrentisaffectedbythepresenceofillumination.ThemodelsuccessfullypredictsunexplainedaspectsofdarkcurrentbehaviorpreviouslyobservedinsomeCCDsensors.

Conference 8298


8298-14, Session 4

Characterizing the response of charge-couple device digital color camerasV.Slavkovikj,J.Y.Hardeberg,GjøvikUniv.College(Norway);A.Eichhorn,SimulaResearchLab.(Norway)

Theadvanceandrapiddevelopmentofelectronicimagingtechnologyhasleadthewaytoproductionofimagingsensorscapableofacquiringgoodqualitydigitalimageswithahighresolution.Atthesametimethecostandsizeofimagingdeviceshavereduced.Thishasincitedanincreasingresearchinterestfortechniquesthatuseimagesobtainedbymultiplecameraarrays.Useofmulti-cameraarraysisattractivebecauseitallowscapturingofmulti-viewimagesofdynamicscenes,enablingthecreationofnovelcomputervisionandcomputergraphicsapplications,aswellasnextgenerationvideoandtelevisionsystems.

Thereareadditionalchallengeswhenusingamulti-cameraarray,however.Duetoinconsistenciesinthefabricationprocessofimagingsensorsandfilters,multi-cameraarraysexhibitinter-cameracolorresponsevariations.Forthemajorityofapplications,whichusemulti-viewimagesobtainedfrommulti-cameraarrays,itisinsufficienttoassumethatthedifferentcamera’sresponsecanbeconsideredthesamewithoutpriorverification.Therefore,itisnecessarytocharacterizetheresponseofthedifferentcamerasinthearray.

8298-15, Session 4

Implementing and using the EMVA1288 standardA.Darmont,J.Lemaitre,J.Chahiba,AphesaSPRL(Belgium)

TheEuropeanMachineVisionAssociationtookinthelastyearstheinitiativeofdevelopingameasurementandreportingstandardforindustrialimagesensorsandcamerascalledEMVA1288.

AphesaofferscameraandsensormeasurementservicesandtestequipmentaccordingtothisEMVA1288standard.Wehavemeasuredcamerasofvariouskindsonourself-madetestequipment.Thisimplementationandallthemeasurementsetsrequiretogointhedetailsofthestandardandalsoshowushowgooditcanbebutalsohowdifficultitcanbe.

Thepurposeofthispaperistogivefeedbackonthestandardbasedonourexperienceofimplementersandusers.Wewillseethatsomemeasurementsareeasilyreproducibleandcanbeeasilyimplementedwhileothersrequiremoreresearchonhardware,softwareandproceduresandthattheresultscansometimeshaveverylittlemeaning.

OurconclusionwillbethattheEMVA1288standardisgoodandwellsuitedforthemeasurementandcharacterizationofimagesensorsandcamerasfortheimageprocessingapplicationsbutthatitishardforanewcomertounderstandtheproduceddataandproperlyuseatestequipment.Developingacompleteandcomplianttestequipmentisalsoadifficulttask.

8298-34, Session 4

Image sensor for defence applicationsE.Belhaire,J.L.Fontanella,ThalesOptroniqueS.A.(France)

Thedefensemarketmakesintensiveuseofimagesensors.Theequipmentsintegratemoreandmoreimagesensorstofulfilltheprimaryobservationsmissions:detection,recognitionandidentification.Thesetaskshastobefulfilledinmoreandmorecomplexsituations:duringnightandunderdaylight,insevereurbanenvironment,infrontofcomplexbackgrounds,atlongdistance,usingsmallsizedisplaysunderconditionsofheavyparasiticlightingandinstressfulsituations.Thetraditionalinfraredlongwaveimagesensorsarenowcompletedorreplacedbyimagesensorsinotheratmospherictransmissionsbands.AmongthoseCMOSandCDDimagesensorsareprimaryconsidered,eitherintheirtraditionalcustomerconfigurationor,morefrequentlywithadaptations.Thoseadaptationsarerealized

onstandardimagesensortechnologyandarelimitedtothestrictminimumrequiredbytheapplicationsinordertoreducethecost.

Theuseoftechnologies,statedas“dualuse”,isprimaryconsideredasthedefensemarketcannotsupporttheimportantresearchanddevelopmentcostsofallthetechnologiesrequiredbydefenseapplications.Inconsequence,thedefensemarketismainlyconcentratingitsR&Dfundingontechnologieswithoutlargescaleapplicationsinthecustomermarket.

Thedifferentwavebandsofinterestarepresented.Themaintechnologiesforeachbandarecited.Then,theimagesensorspecificationadaptations,requiredbythedefensemarket,arereviewedanddiscussedinthispaper.Someexampleoftypicalfuturedefenseapplicationswillbepresented.Whenpossible,applicationcounterpartinthecustomermarketwillbepointed.

8298-35, Session 4

An overview of the European patent system with particular emphasis on IP issues for imaging devicesM.Boero,A.Cabrita,EuropeanPatentOffice(Netherlands)

InthisarticlewegiveacomprehensivereviewoftheEuropeanPatentSystemwithfocusontheprocedure,itstypicalduration,therequirementsthatmustbemetatthevariousstagesinordertoobtainanEuropeanPatentanditsrelatedcosts.Alltheoptionsavailabletotheapplicantarediscussedindetail,potentialpitfallsarehighlighted,andthedifferencesbetweentheEuropeanandUSPatentSystemsareanalysed.

Furthermore,anin-depthandveryinformativeanalysisofapplicationsandgrantedpatentsinthefieldofimagingdevicesispresentedincludingastudyoftheirevolutionduringthelast10yearstogetherwithananalysisofthecountriesandcompaniesthataremostactiveinthefieldofimagers.

8298-16, Session 5

Development of high-transmittance, back-illuminated, silicon-on-sapphire substrates thinned below 25 micrometers and bonded to fused silica for high-quantum efficiency and high-resolution avalanche photodiode imaging arraysA.G.Stern,AGStern,LLC(UnitedStates)

Thereisagrowingneedinscientificandindustrialapplicationsfordual-mode,passiveandactive2Dand3DLADARimagingmethods.Tofillthisneed,solid-state,singlephotonsensitivesiliconavalanchephotodiode(APD)detectorarraysofferhighsensitivityandthepossibilitytooperatewithwidedynamicrangeinduallinearandGeiger-modeforpassiveandactiveimaging.Tosupportthefabricationoflargescale,highquantumefficiencyandhighresolutionsiliconavalanchephotodiodearraysandotheradvancedsolid-stateoptoelectronics,anovel,hightransmittance,back-illuminatedsilicon-on-sapphiresubstratehasbeendevelopedincorporatingasinglecrystal,epitaxiallygrownaluminumnitride(AlN)antireflectivelayerbetweensiliconandR-planesapphire,thatprovidesrefractiveindexmatchingtoimprovetheopticaltransmittanceintosiliconfromsapphire.Aonequarterwavelengthmagnesiumfluorideantireflectivelayerdepositedontheback-sideofthesapphireimprovesopticaltransmittancefromtheambientintothesapphire.ThemagnesiumfluorideplaneoftheSi-(AlN)-sapphiresubstrateisbondedtoafusedsilicawaferthatprovidesmechanicalsupport,allowingthesapphiretobethinnedbelow25micrometerstoimproveback-illuminatedopticaltransmittance,whilesuppressingindirectopticalcrosstalkfromAPDemittedlightundergoingreflectionsinthesapphire,toenablehighquantumefficiencyandhighresolutiondetectorarrays.

Conference 8298


8298-17, Session 5

29 mp, 35 mm format interline CCD image sensorE.J.Meisenzahl,D.A.Carpenter,J.E.Doran,R.P.Fabinski,S.L.Kosman,J.P.McCarten,TruesenseImaging,Inc.(UnitedStates)

Thispaperdescribesthedesignandperformanceofanewhigh-resolution35mmformatCCDimagesensorusinganadvanced5.5uminterlinepixel.Thepixelsarearrangedina6576(H)×4384(V)formattosupporta3:2aspectratio.Thisdeviceispartofafamilyofdevicesthatshareacommonarchitecture,pixelperformance,andpackagingarrangement.UniquetothisdeviceinthefamilyistheimplementationofafastlinedumpstructureandhorizontalCCDlateraloverflowdrain.

8298-18, Session 5

Photodiode dopant structure with atomically flat Si surface for high-sensitivity and stability to UV lightT.Nakazawa,R.Kuroda,Y.Koda,S.Sugawa,TohokuUniv.(Japan)

Ultravioletlight(UV-light)sensing,especiallyinabout200-300nmwavelength,isrequiredwidely.Recently,thetechnologytofabricatephotodiodewithalmost100%internalquantumefficiencyandhighstabilitytoUV-lightbyformingthesurfacephoto-generatedcarrierdriftlayeruniformlybyusinganatomicallyflatSisurfacehasbeenreported[1].

Inthiswork,byusingthisatomicallyflatSisurface,weinvestigatetherelationshipsbetweenthesurfacedriftlayerprofilesandsensitivityandstabilitytoUV-light.Thus,thesensitivityofn+pn-typephotodiodeswithvariousn+layerprofilesaswellas,fixedcharges(Qf)andinterfacestates(Dit)weremeasuredbeforeandafterUV-lightexposure.Finally,weclarifythemechanismofphotodiode’scharacteristics,andproposeaguidelinetoachievehighstabilitytoUV-light.

Basedonthemeasurementresults,3typesofphotodiodes’characteristicswereexplainedbyonemechanism.

Inconclusion,toachievehighsensitivityandstabilitytoUV-light,suppressingthechangeinQfandDitbyUV-lightandsuppressingthechangeofdriftfieldcausedbythechangeinQfarenecessary.ThecombinationofanatomicallyflatSisurfaceandasurfacedriftlayerwithsufficientlyhighconcentrationwasdemonstratedtobeeffectiveforthem.

[1]R.Kurodaetal.,“HighUltravioletLightSensitiveandHighlyReliablePhotodiodewithAtomicallyFlatSiSurface”InternationalImageSensorWorkshop,pp.38-41,2011

8298-19, Session 5

New smart readout technique performing edge detection designed to control vision sensors dataflowH.Amhaz,G.Sicard,TIMALab.(France)

IntegratingCMOSactivepixelsensors(APS)withcarefullychosensignalprocessingunitshasnowadaysbecomeatrendinthedesignofcamera-on-chipsystems.Inthispaperwepresentanewreadouttechniquedesignedtocontroltheexcessivedataflowgeneratedbyhighresolutionimagesensors.Thisnewreadoutmethodachievesedgedetectionwithouttouchingthepixelareasincetheprocessingcircuitryisimplementedintheperipheryofthematrix.Thispointisveryessentialandimportantfromtheindustrialpointofview.Byusingthisreadoutschemewecanbenefitfromthereductionofthepowerconsumptionandtheincreaseofthereadoutspeed.Thistechniquepresentsseveraladvantagesandlibertydegreesthatpermitseveralfunctionalityofthesamesensor.MAtlabmodelingofthereadoutschemeandtheelectricalsimulationsoftheanalogueblockswillalsobepresented.Whatweessentiallyconsiderasaninnovationisthefactofnottouchingthein-pixelcircuitryandsothepixelareaandFill

Factor.Severalresearcheshavealreadybeenstudiedbutnoneofthempresentssuchanindependencyvisavisthepixelcircuitryandarea.

8298-20, Session 5

Characterization of orthogonal transfer array CCDs for the WIYN one-degree imagerM.P.Lesser,D.Ouellette,TheUniv.ofArizona(UnitedStates);T.Boroson,NationalOpticalAstronomyObservatory(UnitedStates);D.Harbeck,P.Martin,WIYNObservatory(UnitedStates);G.Jacoby,CarnegieObservatories(UnitedStates);J.Cavin,Univ.ofWisconsin-Madison(UnitedStates);D.Sawyer,NationalOpticalAstronomyObservatory(UnitedStates);K.Boggs,R.Bredthauer,SemiconductorTechnologyAssociatesInc.(UnitedStates)

TheWIYNOneDegreeImager(ODI)willprovideaonedegreefieldofviewfortheWIYN3.5mtelescopelocatedonKittPeaknearTucson,Arizona.Itsfocalplaneconsistsofan8x8gridofOrthogonalTransferArray(OTA)CCDdetectors.ThesedetectorsaretheSTA2200OTACCDsdesignedandfabricatedbySemiconductorTechnologyAssociates,Inc.andbacksideprocessedattheUniversityofArizonaImagingTechnologyLaboratory.SeverallotrunsoftheSTA2200detectorshavebeenfabricated.Wehavebacksideprocesseddevicesfromthesedifferentlotsandprovidedetectorperformancecharacterization,includingnoise,CTE,cosmetics,quantumefficiency,andsomeorthogonaltransfercharacteristics.Wediscusstheperformancedifferencesforthedeviceswithdifferentsiliconthicknessandresistivity.Afullybuttablecustomdetectorpackagehasbeendevelopedforthisprojectwhichallowshybridizationofthesilicondetectorsdirectlyontoanaluminumnitridesubstratewithanembeddedpingridarray.Thispackageismountedonasilicon-aluminumalloywhichprovidesaflatimagingsurfaceoflessthan20micronspeak-valleyatthe-100Coperatingtemperature.Characterizationofthepackageperformanceincludinglowtemperatureprofilometryisdescribedinthispaper.

8298-21, Session 6

Multispectral imaging device for help in diagnosisC.Delporte,M.BenChouikha,Univ.PierreetMarieCurie(France);S.Sautrot,Univ.DenisDiderot(France);F.Viénot,MuséumNationald’HistoireNaturelle(France);G.Alquié,Univ.PierreetMarieCurie(France)

Thisworkfocusesondevelopmentofamultispectralimagingsystemforintraoperativediagnosisaidpurpose.Thesystemshouldbeabletoidentifyanddifferentiatebiologicaltissuesinanoperativescene,bymeasurementoftheirspectralreflectanceinthevisibleandthenearinfraredwavelengthrange.Multispectralimagesinthenearinfraredmaybeusedtoseeindepthandtoidentifyburiedtissues.

Thepaperintroducesamultispectralimagingdevicewehavedevelopedandvalidatedbyinvitromeasurements.Invivotestsneedspecificmeasurementsprotocolandshouldbecarriedlater.TheimagingsystemiscomposedbyonecalibratedCCDcamera,byoneopticaldevicewiththreeholographicgratingsandbyhalogenlightsource.Multispectralimagesproducedbythissystemareobtainedaccordingtothe0°/45°CIEstandardgeometry.

Inordertovalidatethissystemtoevaluatemeasurementsfluctuationsseveraltestsweremadeondifferentmuscleslices.Thisstudyshowedafluctuationof20%to30%onthesamesliceofagiventissue.Thislargeerrorrisesfromtheintrinsic,complexandnonhomogeneousnatureofbiologicaltissues.Itrepresentsagreatlimitationtothisspectralmeasurementstechniqueandaffectsitsabilityintermsoftissuesspectralreflectancefactorevaluation.

Conference 8298


8298-22, Session 6

Development of a driving method suitable to ultra-high-speed shooting in 2M-fps, 300k-pixel single-chip color cameraJ.Yonai,T.Arai,JapanBroadcastingCorp.(Japan);T.Hayashida,NHKEngineeringServices,Inc.(Japan);H.Ohtake,JapanBroadcastingCorp.(Japan);T.G.Etoh,KinkiUniv.(Japan);T.Yoshida,J.Namiki,HitachiKokusaiElectricInc.(Japan)

Wehavepreviouslymanufactureda1M-fpshigh-speedcameraforshootingTVprograms.Usingthiscamerahasrevealedcertainproblemssuchasdecreaseinimagequalityandoverheatingofthecameraatspeedshigherthan200kfps.Tocircumventtheseproblems,wedevelopedanewultra-high-speedcamerathatenabledpracticalcameraspeedsupto2MfpsbychangingtheCCDlayoutandimprovingthedivingmethod.

Inthepreviousmodel,imagequalitydecreasedwithincreaseinspeedatmorethan200k-fpsduetodistortionoftheCCDdrivingpulseandreductionindynamicrange.TheCCDwasredesignedtoreduceinterconnectionresistance,andsimulationsconfirmedthatreducingtheRCtimeconstantsolvedtheaboveproblems.Also,theincreaseincurrentwhenshootingathighspeedsresultedinoverheatingofthedrivecircuitboard.Shootingcontinuouslyforoneminuteat1Mfpsabruptlyincreasedthedrivecircuitboardtemperaturefrom27degCtoaround80degC.Topreventexcessiveheating,wedevisedamethodfordrivingtheCCDonlywithintheminimumtimerequiredduringhigh-speedshooting.Weexperimentallymanufacturedanultra-high-speedcamerausingthismethodandconfirmedthatheatingdidnotcauseproblemsevenat2Mfps.

8298-24, Session 6

Accurate color with increased sensitivity using IRA.Enge,TruesenseImaging,Inc.(UnitedStates)

Manyapplicationsrequireaccuratecolorcapturesindaylightconditionsandincreasedsensitivityforlowlightconditions.ThesecapturesareoftenaccomplishedbyusingamechanicalswitchtoremovetheIRcutfilter.Thesensitivityisincreasedattheexpenseofcoloraccuracy,andamechanicalpartisrequiredinthecamera.Asparsecolorfilterpatternoffersanopportunitytoobtainincreasedsensitivity-usingtheIRregion-whilestillmaintainingcoloraccuracy.A2xincreaseinsensitivitycanbeachievedoverthecurrentKODAKTRUESENSEColorFilterPattern,whichusesanIRcutfilter.

8298-25, Session 6

Computational color constancy using chromagenic filters in color filter arraysR.Shrestha,J.Y.Hardeberg,GjøvikUniv.College(Norway)

Wehaveproposedanewilluminantestimationtechniquebasedonextensionofchromagenicbasedcolorconstancyinthispaper.Basicchromagenicilluminantestimationmethodtakestwoshotsofascene,onewithoutandonewithaspeciallychosencolorfilterinfrontofthecameralens.Here,weintroducechromagenicfiltersontopofR,GorBfiltersinplaceofoneofthetwogreenfiltersintheBayer’spattern.Introductionofchromagenicfiltersallowtoobtaintwoimagesofthesamesceneviademosaicking,anormalRGBimage,andachromagenicimage,equivalentofRGBimagewithachromagenicfilter.Theilluminantcanthenbeestimatedusingchromagenicbasedilluminationestimationalgorithms.Themethod,therefore,doesnotrequiretwoshotsandnoregistrationissuesinvolvedunlikeasinthebasicchromagenicfilterbasedcolorconstancy,makingitmorepracticalandusefulcomputationalcolorconstancymethodinmanyapplications.

Conference 8298


Conference 8299: Digital Photography VIIIMonday-Tuesday23-24January2012

Part of Proceedings of SPIE Vol. 8299 Digital Photography VIII

8299-01, Session 1

An objective protocol for comparing the noise performance of silver halide film and digital sensorF.Cao,R.Tessières,G.Frédéric,H.Hornung,DxOLabs(France)

Digitalsensorshaveobviouslyinvadedthemarket.However,somephotographerswithveryhighexpectancystillusesilverhalidefilm.Aretheyonlynostalgicreluctanttotechnologyoristheremorethanmeetstheeye?Theanswerisnotsoeasyifweremarkthat,attheendofthegoldenage,filmswereactuallyscannedbeforedevelopment.Nowadaysfilmusershaveadopteddigitaltechnologyandscantheirfilmtotakeadvantagefromdigitalprocessingafterwards.Therefore,itislegitimatetoevaluatesilverhalidefilm“withadigitaleye”,withtheassumptionthatprocessingcanbeappliedasforadigitalcamera.

ThearticlewilldescribeindetailstheoperationsweneedtoconsiderthefilmasaRAWdigitalsensor.Inparticular,wehavetoaccountforthefilmcharacteristiccurve,theautocorrelationofthenoise(relatedtofilmgrain)andthesamplingofthedigitalsensor(relatedtoBayerfilterarray).Wealsodescribetheprotocolthatwasset,fromshootingtoscanning.Wethenpresentandinterprettheresultsofsensorresponse,signaltonoiseratioanddynamicrange.

8299-02, Session 1

Sensor defect probability estimation and yieldH.Peng,B.Keelan,AptinaImagingCorp.(UnitedStates)

Sensoryieldisdirectlyrelatedtotheprobabilityofdefectivepixeloccurrenceandthescreeningcriteria.Assumingaspatiallyindependentdistributionofsinglepixeldefects,effectiveon-the-flycorrectionofsingle-pixeldefectsinacolorplane,andeffectivecorrectionoftwo-pixeldefectsinacolorplane(couplets)throughadefectmap,sensoryieldcanbecomputedbasedontheoccurrenceofthreeadjacentdefectivepixelsinacolorplane(triplets).Closed-formequationsarederivedforcalculatingtheprobabilityofoccurrenceofcoupletsandtripletsasafunctionoftheprobabilityofasinglepixelbeingdefective.Ifamaximumofonetripletisallowedina5-megapixelsensor,toobtaina98%yield,theprobabilityofapixelbeingdefective(p)mustnotexceed1.3E-3(6500defectivepixels).Foran8-megapixelsensor,thecorrespondingrequirementwouldbep<1.1E-3(8900defectivepixels).Simulationexperimentshaveconfirmedtheaccuracyofthederivedequations.Theseequationscanalsobeappliedtocomputereliabilityofanytwodimensionalspatially-distributeddevicenetwork.

8299-03, Session 1

Optimum spectral sensitivity functions for single-sensor color imagingZ.Sadeghipoor,EcolePolytechniqueFédéraledeLausanne(Switzerland);Y.Lu,HarvardSchoolofEngineeringandAppliedSciences(UnitedStates);S.Süsstrunk,EcolePolytechniqueFédéraledeLausanne(Switzerland)

Acost-effectiveandconvenientapproachforcolorimagingistouseasinglesensorandmountacolorfilterarray(CFA)infrontofit,suchthatateachspatialpositionthesceneinformationinonecolorchanneliscaptured.Toestimatethemissingcolorsateachpixel,ademosaicing

algorithmisappliedtotheCFAsamples.Besidesthefilterarrangementandthedemosaicingmethod,thespectralsensitivityfunctionsoftheCFAfiltersconsiderablyaffectthequalityofthedemosaicedimage.Inthispaper,weextendthealgorithmproposedbyLuandVetterli,originallyproposedfordesigningtheoptimumCFA,tocomputetheoptimumspectralsensitivities.Theproposedalgorithmsolvesaconstrainedoptimizationproblemtofindoptimumspectralsensitivitiesandthecorrespondinglineardemosaicingmethod.Animportantconstraintofthisproblemisthesmoothnessofspectralsensitivities,whichisimposedbymodelingthesefunctionsasalinearcombinationofseveralsmoothkernels.Simulationresultsverifytheeffectivenessoftheproposedalgorithminfindingoptimalspectralsensitivityfunctionsthatoutperformmeasuredcamerasensitivityfunctions.

8299-04, Session 1

A metric for the evaluation of wide dynamic-range camerasP.W.Wong,PiximInc.(UnitedStates);Y.H.Lu,MinistryofPublicSecurity(China)

Weproposeamulti-componentmetricfortheevaluationofdigitalorvideocamerasunderwidedynamicrange(WDR)scenes.ThemethodisbasedonasingleimagecaptureusingaspecificallydesignedWDRtestchartandlightbox.TestpatternsontheWDRtestchartincludegrayramps,colorpatches,arraysofgraypatches,whitebars,andarelativelydarkgraybackground.TheWDRtestchartisprofessionallymadeusing3layersoftransparenciestoproduceacontrastratioofapproximately110dBforWDRtesting.Alightboxisdesignedtoprovideauniformsurfacewithlightlevelatabout80Kto100Klux,whichistypicalofasunnyoutdoorscene.

Fromacapturedimage,9imagequalitycomponentscoresarecalculated.Thecomponentsincludenumberofresolvablegraysteps,dynamicrange,linearityoftoneresponse,graynessofgrayramp,numberofdistinguishablecolorpatches,smearingresistance,edgecontrast,gridclarity,andweightedsignal-to-noiseratio.Fromthe9componentscores,acompositescoreiscalculatedthatreflectsthecomprehensiveimagequalityincamerasunderWDRscenes.Experimentalresultshavedemonstratedthatthemulti-componentmetriccorrespondsverywelltosubjectiveevaluationofwidedynamicrangebehaviorofcameras.

8299-05, Session 1

Active pixels of transverse field detector based on a charge preamplifierG.Langfelder,C.Buffa,A.F.Longoni,A.Pelamatti,F.Zaraga,PolitecnicodiMilano(Italy)

TheTransverseFieldDetector(TFD),afilter-lessandtunablecolorsensitivepixel,isbasedonthegenerationofspecificelectricfieldconfigurationswithinadepletedSiliconvolume.Likeinothertunablesensors,eachfieldconfigurationdeterminesasetofspecificspectralresponsesthatcanbeusedfordirectcoloracquisition.Inordertoavoidchangesoftheelectricfieldconfigurationduringthesingleimagecapture,aspecificactivepixel(AP)hasbeendesigned.InthisAPthedark-andphoto-generatedchargeisnotintegrateddirectlyonthejunctioncapacitance,but,foreachcolor,itisintegratedonthefeedbackcapacitanceofasingle-transistorchargepre-amplifier.TheAPfurtherincludesabiastransistor,aresettransistorandafollower.

Inthiswork(i)thedesignofsuchapixel,suitableforatunabledetector,isdiscussedfocusingonseveraldesignconstraintandinparticularontheoptimizationofthepixel-levelloopgain;and(ii)the


experimentalresultsobtainedonasmallmatrixoftheseactivepixelsareanalyzedintermsofdarkcurrent,linearity,resetnoise,dynamicrangeandfixedpatternnoise.

8299-06, Session 1

Digital focusing and re-focusing with thin multi-aperture camerasA.Oberdörster,A.Brückner,F.C.Wippermann,A.Bräuer,Fraunhofer-InstitutfürAngewandteOptikundFeinmechanik(Germany);H.P.A.Lensch,Univ.Ulm(Germany)

Forsmallcameramodulesinconsumerapplications,suchasmobilephonesorwebcams,sizeandcostareimportantconstraints.Anautofocussystemincreasesbothsizeandcostandcanalsointroduceadditionalaberrationsduetomisalignment.Therefore,amonolithicopticalsystemwithafixedfocusispreferablefortheseapplications.Ontheotherhand,theopticalsystemofthecamerahastoexhibitaverylargedepthoffield,asitisexpectedtodeliversharpimagesforalltypicalworkingdistances.Thedepthoffieldofacamerasystemcanbeincreasedbyclosingtheaperture,butalsobyreducingfocallength.

Multi-aperturesystemsusemultipleopticalchannels,eachofthemwithapotentiallysmallerfocallengththanacomparablesingle-aperturesystem.Accordingly,eachofthechannelshasalargedepthoffield.However,asthechannelsaredisplacedlaterally,parallaxbecomesnoticeableforcloseobjects.Therefore,thechannelimageshavetobeshiftedaccordinglywhenrecombiningthemintoacompleteimage.

Wedemonstrateanalgorithmthatcompensatesforparallaxaswellaschromaticaberrationandgeometricdistortion.Wepresentaveryflatcamerasystemthatiscapableoftakingphotosorvideoatawiderangeofdistances,withoutmovingparts.Onthedemonstrationsystem,objectdistancecanbeadjustedinrealtime,from4mmtoinfinity.Focuscanbeselectedduringcaptureoraftertheimagesweretaken.

8299-07, Session 1

The multi-focus plenoptic cameraT.G.Georgiev,AdobeSystemsInc.(UnitedStates);A.Lumsdaine,IndianaUniv.(UnitedStates)andAdobeSystemsInc.(UnitedStates)

ThefocusedplenopticcameraisbasedontheLippmannsensor:Anarrayofmicrolensesfocusedonthepixelsofaconventionalimagesensor.Thisdevicesamplestheradiance,orplenopticfunction,asanarrayofcameraswithextendeddepthoffield,focusedatacertainplaneinfrontofthemicrolenses.Forthepurposeofdigitalfocusingafterthefact(whichisoneoftheimportantapplications),thedepthoffieldneedstobeaslargeaspossible.Therearefundamentalopticalcallimitationstothis,relatedtoapertureanddiffraction.

Asolutionoftheaboveproblemistouseandarrayofinterleavedmicrolensesofdifferentfocallengths,focusedattwoormoredifferentplanes.Inthiswayfocusedmicroimageswouldbecapturedatmultipledepths,andareallywiderangeofdigitalrefocusingwouldbepossible.

Thispaperpresentsourtheoryandresultsofimplementingsuchcamera.Realworldimagesaredemonstratingtheextendedcapabilities.Limitationsarediscussed.

8299-08, Session 1

Spatial domain analysis of discrete plenoptic samplingA.Lumsdaine,IndianaUniv.(UnitedStates);T.G.Georgiev,AdobeSystemsInc.(UnitedStates)

Plenopticcamerasareintendedtofullycapturethelightraysinascene.Usingthisinformation,opticalelementscanbeappliedtoascenecomputationallyratherthanphysically---allowinganinfinite

varietyofpicturestoberenderedafterthefactfromthesameplenopticdata.Practicalplenopticcamerasnecessarilycapturediscretesamplesoftheplenopticfunction,whichtogetherwiththeoverallcameradesign,canconstrainthevarietyandqualityofrenderedimages.Inthispaperwespecificallyanalyzethenatureofthediscretedatathatplenopticcamerascapture,inamannerthatunifiesthetraditionalandfocusedplenopticcameradesigns.Wefurtherpresentaresolutionanalysisforplenopticcamerasanddevelopdesignguidelinesformaximizingresolution.Ageneralizedrenderingalgorithmispresentedthatminimizesartifactsresultingfromthelowerresolutionangularsamplingthataccompanieshigh-resolutionspatialsampling.Experimentalresultsusingareal-timeGPUimplementationofouralgorithmsdemonstratestheeffectivenessofourapproach.

8299-09, Session 1

Design framework for a spectral mask for a plenoptic cameraK.Berkner,S.A.Shroff,RicohInnovations,Inc.(UnitedStates)

Plenopticcamerasaredesignedtocapturedifferentcombinationsoflightraysfromascene,samplingthelighfieldofascene.Mostofthesecameradesignscapturedirectionalrayinformationenableapplicationssuchasdigitalrefocusing,rotation,ordepthestimationonlyfewaddresscapturingspectralinformationofthescene.Ithasbeendemonstratedthatbymodifyingaplenopticcamerawithafilterarraycontainingdifferentspectralfiltersinsertedinthepupilplaneofthemainlens,samplingofinthespectraldimensionoftheplenopticfunctionisperformed.Asaresult,theplenopticcameraisturnedintoasingle-snapshotmultispectralimagingsystemthattrades-offspatialwithspectralinformationcapturedwithasinglesensor.Littleworkhasbeenperformedsofaronanalyzingdiffractioneffectsandaberrationsoftheopticalsystemontheperformanceofthespectralimager.Inthispaperwedemonstratesimulationofaspectrally-codedplenopticcameraopticalsystem,evaluatequalityofthespectralmeasurementscapturedatthedetectorplane,anddemonstrateopportunitiesforoptimizationofthespectralmaskforafewsampleapplications.

8299-10, Session 2

Detection thresholds of structured noise in the presence of shot noiseF.Li,B.W.Keelan,A.Dokoutchaev,R.Jenkin,AptinaImagingCorp.(UnitedStates)

Asagrowingnumberofconsumersbecomeawareofthetradeoffbetweenmegapixelcountandcameraperformance,imagequality,especiallyatlowlightweightsmorewhenauserpurchasesacamera.Structurednoise,namelyroworcolumnfixedpatternnoise(FPN),isoneofthemainchallengesinimprovingimagequalityforimagingsensors.Thepresentpaperstudiedthedetectabilityofthestructurednoiseatvariousnoiselevels.Theresultsmaybeusedtoguidethepixelandalgorithmdesignsforimprovingstructurednoiseinimagingsensors.

8299-11, Session 2

Reduced-reference image quality assessment based on statistics of edge patternsY.Chen,X.Mou,Xi’anJiaotongUniv.(China)

Recently,researchofObjectiveImageQualityAssessment(IQA)hasgainedmuchattentionduetoitswideapplicationprospect.Amongthem,theReduced-Reference(RR)methodsestimateperceptualqualityofdistortedimageswithpartialinformationfromthereferenceimages.ThispaperproposesanoveluniversalRR-IQAmetricbasedonthestatisticsofedgepatterns.Firstly,thebinaryedgemapsofthereferenceanddistortedimagesarecreatedbytheLOGoperatorandzero-crossingdetection.Basedonthem,15groupsoftypicaledge

Conference 8299


patternsareextractedandthentheirstatisticaldistributionscalculatedrespectivelyforthereferenceanddistortionimages.TheproposedRR-IQAmetricisachievedbycomputingtheL-1Minkowskidistancebetweenthosetwodistributions.WehaveevaluatedthismetriconsixpubliclyaccessiblesubjectiveIQAdatabases.ExperimentsshowthattheproposedmetricworksbetterthanitscompetingRR-IQAmetricsacrossthedifferentdatabases.

8299-12, Session 2

Joint chromatic aberration correction and demosaickingT.Singh,M.Singh,ImageAlgorithmics(UnitedStates)

ChromaticAberrationoflensesisbecomingincreasinglyvisiblewiththeriseofsensorresolution,andmethodstoalgorithmicallycorrectitarebecomingincreasinglycommonincommercialsystems.Apopularalgorithmistoundothegeometricdistortionsafterdemosaicking.Sincemostdemosaickersrequirehighfrequencycorrelationofprimarycolorstoworkeffectively,theresultisartifact-riddenasChromaticAberrationdestroysthiscorrelation.Theotherexistingapproachofundistortingprimarycolorimagesbeforedemosaickingrequiresresamplingofsub-sampledprimarycolorimagesandispronetoaliasing.Furthermore,thisalgorithmcannotbeappliedtopanchromaticCFAs.WeproposeajointdemosaickingandchromaticaberrationcorrectionalgorithmthatisapplicabletobothpanchromaticandprimarycolorCFAsandsuffersfromnoneoftheaboveproblems.Ouralgorithmtreatsthemosaicingprocessasalineartransformthatisinvertibleifluminanceandchrominanceareappropriatelybandlimited.WedevelopandincorporateChromaticAberrationcorrectionstothismodelofthemosaicingprocesswithoutalteringitslinearityorinvertibility.Thiscorrectionworksforbothspacevariantlinearfilterdemosaickingandthemoreaggressivecompressivesensingreconstruction.

8299-13, Session 2

Optimal defocus estimates from individual images for autofocusing a digital cameraJ.Burge,W.S.Geisler,TheUniv.ofTexasatAustin(UnitedStates)

Recently,wedevelopedamethodforoptimallyestimatingdefocusblurgivenasetofnaturalscenes,awave-opticsmodelofthelens,asensorarray,andaspecificationofmeasurementnoise.Themethodisbasedonfirstprinciplesandcanbetailoredtoanyvisionsystemforwhichthesepropertiescanbecharacterized.Here,themethodisusedtoestimatedefocusinlocalareasofimages(64x64pixels)formedinaNikonD700digitalcamerafittedwitha50mmSigmaprimelens.Performanceisexcellent.Defocusmagnitudeandsigncanbeestimatedwithhighprecisionandaccuracyoverawiderange.Themethodoffersanalternativetobothphasedetectionandcontrastmeasurementauto-focustechniques.Likephase-detection,themethodprovidespointestimatesofdefocus(magnitudeandsign),yetitdoesnotrequirespecializedhardware.Likecontrastmeasurement,themethodisimage-basedandcanoperatein“LiveView”mode,yetitdoesnotrequireaniterativesearchforbestfocus.Thus,thisnewmethodhastheadvantagesofbothphase-detectionandcontrastmeasurementauto-focustechniques,withouttheirdisadvantages.Theapproachcanbeusedtodevelopimprovedautofocusanddepthestimationalgorithmsforcomputationalvisionsystems.

8299-14, Session 2

Quality versus color saturation and noiseB.W.Keelan,R.B.Jenkin,E.W.Jin,AptinaImagingCorp.(UnitedStates)

Asoftcopyqualityrulerstudyinvolving12scenesand34observerswasperformedtoquantifythedependenceofqualityoncolorsaturation,intheabsenceofnoise,withsaturationmeasuredusing

Imatestsoftware.Qualityfallsoffsymmetricallywithdeviationofcolorsaturationfromthepreferredvalueofabout110%,witha20%changeinsaturationreducingqualitybyabouttwojustnoticeabledifferences(JNDs).Optimizationofnoiseversuscolorsaturationwasinvestigatedusing(1)theaforementionedtransformofcolorsaturationtoJNDsofquality;(2)apreviouslypublishedobjectivemetricandJNDtransformforisotropicnoise;and(3)themultivariateformalism,forcombiningJNDsfromindependentattributesintoaoverallqualityJNDs.Asnoiseincreasesandsignaltonoiseratio(SNR)decreases,theoptimalcolorsaturationdecreasesfromthe110%position,sothatthereislessnoiseamplificationbythecolorcorrectionmatrix.Aqualitycontourplotispresented,showingaregionofplausiblecolorsaturationvalues,asafunctionofSNR,forarepresentativeusecase.Oneexampleofareasonablestrategyistoprovide80%colorsaturationatSNR=5,90%atSNR=10,100%atSNR=20,and110%atSNR50.

8299-15, Session 2

Bio-inspired framework for automatic image quality enhancementF.Gasparini,S.Raimondo,C.Andrea,M.Fabrizio,Univ.degliStudidiMilano-Bicocca(Italy)

Weproposeabio-inspiredframeworkforautomaticimagequalityenhancement.Restorationalgorithmsusuallyhavefixedparameterswhosevaluesarenoteasilysettable.Weshowthatitispossibletocorrelateno-referencevisualqualityvaluestospecificparametersettingssuchthatthequalityofanimagecouldbeeffectivelyenhancedthroughtherestorationalgorithm.Givenagenericdegradedimageano-referencemetricevaluatesitsvisualquality.Thisvalueiscorrelatedtoatupleofoptimizedalgorithmparametersdeterminedaprioriandgatheredinatable.Thistableistheoutcomeofanofflineprocessing.Foragivenartifact,thisprocessinginvolvesarestoringalgorithm,afull-referencemetricandadatabaseofreferenceimagesthathavebeendegradedwithawiderangeofdistortion.Eachdegradedimageisprocessedbytherestorationalgorithmthat,bymeansofthegivenparametersproducesanenhancedversionoftheimage.This,alongwithitsoriginalreference,isevaluatedbythefull-referencequalitymetric.Theserestorationandevaluationoperationsarerepeatedwithinageneticoptimizationalgorithm.Whentheevolutionisover,thebestso-foundparametersareassociatedtoano-referencequalitymeasureoftheinitialdegradedimage.Inthispaper,wechoseJPEGblockinessdistortionasacasestudy.Asfortherestorationalgorithm,weusedeitherabilateralfilter,oratotalvariationdenoisingdetexturer.TheexperimentalresultsontheLIVEdatabasewillbereported.Theseresultswilldemonstratethatabettervisualqualityisachievedthroughtheoptimizedparametersovertheentirerangeofcompression,withrespecttothealgorithmdefaultparameters.

8299-16, Session 2

An efficient, multiple-exposure image fusion in JPEG domainR.S.M.Hebbalaguppe,DublinCityUniv.(Ireland);R.Kakarala,NanyangTechnologicalUniv.(Singapore)

AnefficientmethodtofusemultipleimagestakenwithvaryingexposuretimesintheJPEGdomainispresented.ThealgorithmusesthespatialfrequencyanalysisprovidedbytheDCTwithinJPEGtocombinetheuniformregionsofthelongest-exposureimagewiththedetailedregionsoftheshort-exposureimages,therebyreducingnoisewhileprovidingsharpdetails.Advantagesoftheproposedmethodaregreatreductioninprocessingtime,improvedmemorymanagement,andefficientghostremovalinobtainingreasonablygoodqualityHDRimages.

Experimentsshowbothquantitativeandqualitativeimprovementovertheshort-longexposedimages.Qualitatively,thefusedimagelookssharpwithbettercolorsthanthelong-shortimages.Quantitatively,thefusedimageshowsimprovementinSNRovertheshortestexposedimageandthesharpness(obtainedbyblurmetric)overthelongestexposedimage.

Conference 8299


Tosummarizeourmethod,weuseasinglepasssigmoidalboostingontheshorterexposedimagesimplementedasLUT,unlikeotherpublishedmethodswhichrequiretwoormorepasses.ReuseofedgedetectionwhichisapartofJPEGforremovalofartifactsfurtheroptimizesthealgorithm.Lastly,themethodrequiresnomorethanasinglemacroblocktobekeptinmemory,becausetheimagefusionisperformedessentiallyintheJPEGfileandrenderedonlyondecodingtheimage.

8299-17, Session 2

A controllable anti-aliasing filter for digital film camerasB.Petljanski,PanavisionInc.(UnitedStates)

Inthispaper,thetheoreticalfoundationandpracticalimplementationofacontrollableanti-aliasingfilterfordigitalfilmcamerasispresented.

Aprototypeofanopticalanti-aliasingfilterthatisbasedonmovingaparallelopticalwindowwasdesignedandbuilttodemonstratetheabilitytocontrolthespatialfrequencyresponseofanacquisitionsystem.Duringtheimageexposure,fourspringpreloadedvoicecoilsarerapidlychangingthepitchandyawoftheparallelwindowresultinginadisplacementoftheimagecontentthatisprojectedontothesensor.Theimagecontentdisplacementduringtheexposureresultsinalteringthefrequencyresponseofthescenethatiscapturedbythesensor.

Specifically,duringtheexposuretime,acarefullycontrolledmovementoftheparallelopticalwindowresultsinacirculartrajectoryoftheimagecontentthatisprojectedontothesensor.Byincreasingordecreasingtheradiusofthecirculartrajectory,thespatialcut-offfrequencyofthesystemisdynamicallymodified.Inadditiontothecircularpath,thispapershowstheoreticaljustificationanddemonstratestheuseofmorecomplextrajectories,suchasthedoublecircle,elliptical,roseandothers.Thesetrajectoriesimprovethesuppressionofaliasedcomponentsintheacquiredimage.

8299-01, Session 3

Development of the I3A CPIQ spatial metricsH.Eliasson,SonyEricssonMobileCommunicationsAB(Sweden);D.Baxter,STMicroelectronics(R&D)Ltd.(UnitedKingdom);F.Cao,DxOLabs(France);J.Phillips,EastmanKodakCo.(UnitedStates)

TheI3ACameraPhoneImageQuality(CPIQ)initiativeisaimingatprovidingaconsumer-orientedoverallimagequalitymetricformobilephonecameras.Inordertoachievethisgoal,asetofsubjectivelycorrelatedimagequalitymetricshasbeendeveloped.Thispaperdescribesthedevelopmentofaspecificgroupwithinthissetofmetrics,thespatialmetrics.Containedinthisgrouparetheedgeacutance,visualnoiseandtextureacutancemetrics.Acommonfeatureisthattheyarealldependentonthespatialcontentofthespecificscenebeinganalyzed.Therefore,themeasurementresultsofthemetricsareweightedbyacontrastsensitivityfunction(CSF)and,thus,theconditionsunderwhichaparticularimageisviewedmustbespecified.Thisleadstotheestablishmentofacommonframeworkconsistingofthreecomponentssharedbyallspatialmetrics.First,theRGBimageistransformedtoacoloropponentspace,separatingtheluminancechannelfromtwochrominancechannels.Second,associatedwiththiscolorspacearethreecontrastsensitivityfunctionsforeachindividualopponentchannel.Finally,thespecificviewingconditions,comprisingbothdigitaldisplaysaswellasprintouts,aresupportedthroughtwodistinctMTFs.

8299-18, Session 3

A functional-design approach to lens shading correction issues on mobile camera systemS.Yoo,SAMSUNGElectronicsCo.,Ltd.(Korea,Republicof);T.Kim,SAMSUNGElectronicsCo.,Ltd.(UnitedStates)

Thispaperproposesthelensshadingcorrectionsystemthroughcombinationoffunctionalfactorsbasedonrecentissuesrelatedtomobilecamerasystem.Themainissueisdesigningoftheshading

algorithm.Thesecondissueiscorrectionofcolortemperature.Thethirdissueiscorrectionbyopticalzoomlens.Thefourthissueisrelatedwithmassproductionofcameramodule.Thefinalissueinmobiledevicesisreducingsizeofcameramodule.PreviouslensshadingcorrectionthatsituatetostablefunctionthroughrichhardwareresourceinDSLRandcompactcameraforrecentlytheseveralyearsisbecomingissuesagainbeingadmittedtopartofsystemofmobilecamerathatresourceislimited.Asdaysgoby,sizereductionofimagesensorandmoduleisessentialduetorequestofmarketplacealongwiththedevelopmentofmobiledevices.Theproposedlensshadingcorrectionsystemprovidesminimizingofaccuracyerrorsinshadingevaluation,aswellassizereductionofmemoryandmoduleinhardware.Finally,theproposedsystemforlensshadingcorrectionsolvesraisedissuesinmobileenvironmentandguidesdesignofalgorithmandhardwareforlensshadingcorrectioninmobilecameramodule.

8299-19, Session 3

Rethinking user interfaces for cameraphonesS.A.Brewster,C.McAdam,Univ.ofGlasgow(UnitedKingdom)

Cameraphonesareverycommonbuthaveusabilityissuesthataffecttheiruseandreducethequalityofimagescaptured.Usersoftenspendlittletimepost-processingphotos,oftenuploadingthemimmediatelytosocialnetworkingsites.ProblemsmayoccurbecauseuserslookthroughtheLCDtoframeshotsandoftenmissiconsdisplayedaroundtheedgesofthescreenthatpresentimportantinformationaboutcamerastatus.Thismayleadtoshotsbeingmissed,blurredorpoorlyexposed.Mostcameraphonesdonottakefulladvantageofthefeaturesoftheunderlyingphoneplatform(suchasrichsensingandpowerfulaudio)totrytosolvetheseproblems.WecreatedanAndroidcameraphoneapplicationthatfeaturednovelinteractionsandmadeuseofthefeaturesoftheplatformtoprovidearichvarietyofinformationinmoreusableforms,suchas:sonificationoftheluminancehistogramtoensurebetterexposurebeforeapicturewastaken;measuringphonemovementtoensurethephonewasbeingheldsteady;andthedetectionofmotionintheimagetosupportpanning.Weevaluatedtheseinteractionswithusersinrealsettingsandshowedtheycouldsolvesomeofthemostcommoncameraphoneproblemsandimprovethepicturetakingprocess.

8299-02, Session 4

Calibration and adaptation of ISO visual noise for camera phone image quality assessmentD.J.Baxter,A.Murray,STMicroelectronics(R&D)Ltd.(UnitedKingdom)

TheI3ACameraPhoneImageQuality(CPIQ)visualnoisemetricdescribedisacoreimagequalityattributeofthewiderI3ACPIQconsumerorientated,cameraimagequalityscore.Thispaperdescribesthemotivationbehindthechoiceofavisualnoisemetric,whythecharacteristicsofcellphonecamerasnecessitatetheadaptationoftheISO15739visualnoiseprotocolandthecalibrationoftheadaptedvisualnoisemetricintermsofQualityLossJNDs.VisualnoisemetricssuchasISO15739visualnoise,S-CIELABandvSNRareshowntohavetheimportantpropertyofbeingabletodiscriminatebetweendifferentNPSshapes.Theopticalnon-uniformitiesprevalentincellphonecamerasandhighernoiselevelsposesignificantchallengestotheISO15739visualnoiseprotocol.

Thenon-uniformitiesareaddressedusingafrequencybasedhighpassfilter.Secondly,thedataclippingathighnoiselevelsisavoidedbyuseoftheJohnsonandFairchildfrequencybasedSCIELAB

LuminanceCSF.ThefinalresultisavisuallybasednoisemetriccalibratedintermsofQualityLossJNDsusingAptinaImaging’ssubjectivelycalibratedimageset.

Conference 8299


8299-03, Session 4

An objective method for evaluating the texture-preserving capability of digital camcordersK.Zhu,S.Li,D.Saupe,Univ.Konstanz(Germany)

Thispaperpresentsamethodforevaluatingperformanceofcamcordersintermsoftexturepreservation,takinghumanvisualsystem(HVS)intoaccount.AmetricwecallPSDM(powerspectraldensitymetric)istheoutcomeofthemethodasaquantitativeindicator.TherelationshipsbetweenthePSDMvalueandthemotionspeedandthecompressionbitratewerestudies.Thedeadleaveschartwasusedinourexperimentstosimulateascenewithtexturesofdifferentscales.Thedeadleaveschartsisknownasagoodtargetinvarianttoscale,shift,rotationandcontrast(exposure)anditsradialPSDfollowsapowerlaw.Experimentalresultsonfivecamcordersfromthreedifferentvendorshaveshownthat1)thePSDMvaluehasamonotonicrelationshipwiththemotionspeedforalltestedcamcorders;2)thePSDMvaluehasamonotonicrelationshipwiththecompressionbitratesforthreetestedcamcordersbutnotfortheothertwo.WeattributethemismatchbetweenthePSDMvalueandthecompressionbitrateforsomecamcorderstotheintrinsicnonlinearityofmanycomponentsinthewholevideocapturing/recordingprocess.OurstudyhasconfirmedPSDMasausefulindicatorformeasuringacamcorder’sperformanceintermsofpreservingtextures.

8299-04, Session 4

Improving texture loss measurement: spatial frequency response based on a colored targetU.Artmann,D.Wueller,ImageEngineeringGmbH&Co.KG(Germany)

Thepixelraceinthedigitalcameraindustryandformobilephoneimagingmoduleshavemadenoisereductiontoasignificantpartinthesignalprocessing.Dependingontheusedalgorithmsandtheunderlyingamountofnoisethathastoberemoved,noisereductionleadstoalossoflowcontrastfinedetails,alsoknowastextureloss.Thedescriptionoftheseeffectsbecameanimportantpartoftheobjectiveimagequalityevaluationinthelastyears,astheestablishedmethodsfornoiseandresolutionmeasurementfailtodoso.Differentmethodshavebeendevelopedandpresentedinthelastyears,butcouldnotfullysatisfytherequestedstabilityandcorrelationwithsubjectivetests.Inourpaper,wepresentourexperiencewiththecurrentapproachesfortexturelossmeasurement.Wehavefoundacriticalissuewithinthesemethods:theusedtargetsarecolorneutral.Wecouldshowthatthetest-labresultdonotmatchtherealliveexperiencewiththecamerasundertest.Wepresentanapproachusingacoloredtargetandourexperiencewiththismethod,basedonahugevarietyofdigitalcameras.

8299-20, Session 4

On the performances of computer vision algorithms on mobile platformsS.Battiato,G.M.Farinella,E.Messina,G.Puglisi,D.Ravì,Univ.degliStudidiCatania(Italy);A.Capra,V.Tomaselli,STMicroelectronics(Italy)

Inrecentyearsthereisagrowinginterestinnewtechnologytobeemployedinthecontextofmobiledevices.Despitetoday’smobiledevices(e.g.,smartphone,tablet,etc.)arestilllimitedintermsofresources(e.g.,processorspeed,availableRAM,etc.),novelComputationalPhotographysolutionsareavailabletobuildappealingimagingapplicationsthatcannotbeperformedbefore.Themainideaistoovercomethelimitationoftraditionalimagingdevicesbyusingcomputationalmethodswhichcanexploitthedifferentinputsofferedbyamobiledevices(e.g.,fromlowleveldata,suchasBayernpattern,

GPSposition,etc.).Sincedifferentcamerasareusuallyembeddedindevicesofnewgeneration,computervisionalgorithmswillbeextremelyusefulinmanyapplicationsofnextfuture.Forexample,visualtrackingcanbeexploitedtointeractwithvideogames,ortherecognitionofthevisualcontentcouldhelpinbuildingnewapplicationsinthecontextofculturalheritage(e.g.,givingbackinformationonarecognizedarcheologicalsite).

Themaincontributionofthisworkisrelatedtotheportingandtestingofsomeclassiccomputervisionalgorithmsonmobileplatforms.Specifically,afewalgorithmscoveringthemaintasksofComputerVisionhavebeenconsidered:keypointextraction,facedetection,imagesegmentation.Theportinghasbeenperformedconsideringthefollowingoperatingsystems:Maemo,typicallyusedinNokiaN900,andAndroidemployedinLGOptimusone,SamsungGalaxySII.Theseoperatingsystemshavebeenconsideredbecausetheycanbeeasilyextendedwithcustomizedlibrariesand/orprogramsandprovideastandardizedandfairlywidespreadAPI(ApplicationProgramInterface).

Itisworthnotingthattheaforementionedalgorithmsshouldbeoptimizedtoproperlyworkonlowresourcesdevices.Forinstance,theFCAMlibraryavailableforN900Nokiasmartphone,allowstointeractwiththelowlevelalgorithms(e.g.,demosaicing,whitebalancing,denoising,etc.)anddata(Bayerpattern)involvedintheimagingpipeline.Inthiswayabetterdesignofcomputervisionalgorithmsforconstrainedresourcesdevicescanbedone.

Finally,comparativetests,conductedonstandarddatasets,quantitativelyandqualitativelyevaluatetheperformanceofthealgorithmsonmobiledevices.

8299-22, Session 5

Spectral sensitivity evaluation considering color constancyH.Kuniba,NikonCorp.(Japan)

Thehumanvisualsystemautomaticallyadjuststhewhitepointandthecolorofanobjectremainsalmostconstant.Thisiscalled“colorconstancy.’’Thusdigitalcamerasshouldadjustthewhitepointaccordingtotheilluminantused(whitebalancecorrection).Hung(2001)evaluatedcolorconstancyerrorofseveralspectralsensitivitiesandshowedthatthecolorconstancyerrorwasreducedifthewhitebalancecorrectionwasconductedinanappropriatecolorspace.Threetypesofsensormodelswereusedforthesimulation,thenthecolorconstancyerrorsseemstobelargeforthesensormodelwhoseoverlapsbetweenchannelsarelargeandchannelwidthsarelarge.Thustwonewmetricwhichevaluatedtheoverlapsandthewidthsweredefinedandusedtooptimizetheconversionandthecolorconstancyerrorswereevaluated.Thoughthereductionratewasnotaslargeastheformerresults,thecolorconstancyerrorswasreducedbyreducingtheoverlapbetweenchannelsandnarrowingthewidths.ThenarrowerwidthsandsmalleroverlapsofRGBchannelsgavelessaccuratecolorimetricreproductionandlessnoisyimages.Inadditiontothis,itgavesmallercolorconstancyerrors.Inviewofconsumerdigitalcameras,colorimetricreproductionaccuracyisnotthefirstprioritybecausetheobjectiveoftheirreproductionis“preferredcolourreproduction.’’Theyareusedunderwidespectrumofilluminationsandscenesbutusersjustwanttopressthebuttonandletthecameradotherest.ThusthenarrowerwidthsandsmalleroverlapsofRGBchannelsissuitablebecauseitcangivelessnoisyimagesandconsistentcolorreproductionwithsimplewhitebalanceprocessing.

8299-23, Session 5

Multispectral demosaicking using guided filterY.Monno,M.Tanaka,M.Okutomi,TokyoInstituteofTechnology(Japan)

Amultispectralimagingishighlydemandedforprecisecolorreproductionandforvariouscomputervisionapplications.Unfortunately,themultispectralimagingsystemisstillimpracticalcomparedtocommonlyusedconsumerRGBcameras.The

Conference 8299


multispectralimagingwithamultispectralcolorfilterarray(MCFA),inwhichmorethanthreespectralcomponentarearrayed,couldbeasimple,low-cost,andpracticalsystem.AchallengeofthemultispectralimagingsystemwiththeMCFAisamultispectraldemosaickingbecauseeachspectralcomponentoftheMCFAisseverelyundersampled.Inthispaper,weproposeanovelmultispectraldemosaickingalgorithmusingaguidedfilter.Theguidedfilterisrecentlyproposedasanexcellentstructure-preservingfilter.Theguidedfilterrequiresso-calledaguideimage.Theoutputoftheguidedfiltercanberepresentedasthelineartransformoftheguideimage,thusitnicelyresemblestheguideimage.Intheproposedalgorithm,wefirstgenerateaguideimagefromthemostdenselysampledspectralcomponentintheMCFA.Then,ohterspectralcomponentsareinterpolatedwiththeguidedfilter.Experimentalcomparisonsdemonstratethattheproposedalgorithmoutperformsotherexistingmultispectraldemosaickingalgorithmsbothvisuallyandquantitatively.

8299-24, Session 5

An LED-based lighting system for acquiring multispectral scenesM.Parmar,S.Lansel,J.E.Farrell,StanfordUniv.(UnitedStates)

Multispectralsceneinformationisusefulforthesimulationoftheelementsofthedigitalimagingpipelineandtoanalyzetheireffectonfinalimagequality1.Certainsceneswithanimatesubjects,e.g.,humans,pets,etc.,areofparticularinteresttoconsumercameramanufacturersbecauseoftheirubiquityincommonimages,andtheimportanceofmaintainingcolorimetricfidelityforskin.Typicalmultispectralacquisitionmethodsrelyontechniquesthatusemultipleacquisitionsofascenewithanumberofdifferentopticalfilters2-6orilluminants7.Suchschemesrequirelongacquisitiontimesandarebestsuitedforstillscenes.Insceneswhereanimateobjectsarepresent,movementleadstoproblemswithregistrationandmethodswithshorteracquisitiontimesareneeded.

Toaddresstheneedforshorterimageacquisitiontimes,wedevelopedamultispectralimagingsystemthatcapturesmultipleacquisitionsduringarapidsequenceofdifferentlycoloredLEDlights.Inthispaper,wedescribethedesignoftheLED-basedlightingsystemandreportresultsofourexperimentscapturingsceneswithhumansubjects.


Fast in-plane translation and rotation estimation for multi-image registrationX.Jiang,H.Wang,QualcommInc.(UnitedStates)

Thisdocumentconsiderstheplanarmotionsofcamera,thatis,therotation,andhorizontalandverticaltranslations.TheapproachbasedonprojectionincludingbothCartesiancoordinatesystemandpolarcoordinatesystemcanestimatethethreeparameterscomparablyquicklywithsimplecalcuation.Thepotentialapplicationscovermotiondeblurring,noisereduction,super-resolution,imagefusion,highdyanmicrangeimageprocessing,EDOF,3Dimagingorthosetechniqueswhichrequireglobalorlocalregistration.


Multispectral filter wheel cameras: modeling aberrations for filters in front of lensJ.Klein,T.Aach,RWTHAachen(Germany)

Aberrationsoccurinmultispectralcamerasfeaturingfilterwheelsbecauseofcolorfilterswithdifferentopticalpropertiesbeingpresentintheraypath.Inordertoensureanexactcompensationoftheseaberrations,amathematicalmodelofthedistortionshastobedevelopedanditsparametershavetobecalculatedusingthemeasureddata.Suchamodelalreadyexistsforopticalfiltersplacedbetweenthesensorandthelens,butnotforbandpassfiltersplacedinfrontofthelens.Forthisconfiguration,theraysarefirst

distortedbythefiltersandthenbythelens.Inthispaper,wederiveamodelforaberrationscausedbyfiltersplacedinfrontofthelensinmultispectralcameras.Wecomparethismodelwithdistortionsobtainedwithsimulationsaswellaswithdistortionsmeasuredduringrealmultispectralacquisitions.Inbothcases,thedifferencebetweenmodeledandmeasuredaberrationsremainslow,whichcorroboratesthephysicalmodel.Multispectralacquisitionswithfiltersplacedbetweenthesensorandthelensorinfrontofthelensarecompared:thelatterexhibitsmallerdistortionsandtheaberrationsinbothimagescanbecompensatedusingthesamealgorithm.


Correcting saturated pixels in imagesJ.Fu,W.Ji,X.Mou,Xi’anJiaotongUniv.(China)

Thispaperproposesanovelmethodtocorrectsaturatedpixelsinimages.ThismethodisbasedontheYCbCrcolorspaceandseparatelycorrectsthechrominanceandtheluminanceofsaturatedpixels.Dynamicthresholdsareadoptedtoidentifysaturatedpixels,i.e.thethresholdsfordifferentimagesanddifferentcolorchannelsaredifferent.SoourmethodcancorrectnotonlyRAWimagesbutalsoprocessedimages.Oncethesaturatedpixelsareidentified,therearethreekindsofsaturatedpixels:1-channelsaturatedpixels,2-channelsaturatedpixelsand3-channelsaturatedpixels.TheyaredenotedasΩ1,Ω2andΩ3respectively.Differentstrategiesareimplementedtothesethreekindsofregions.ThecolorofsaturatedpixelsinΩ1iscorrectedaccordingtotheiroriginalcolorandthecoloroftheirneighborhood.AndthecolorofsaturatedpixelsinΩ2andΩ3iscorrectedonlyaccordingtothecoloroftheirneighborhood.Theluminanceofsaturatedpixelsiscorrectedusingthemodelproposedinthispaper.ExperimentresultsshowthatourmethodiseffectiveincorrectingsaturatedpixelsofRAWimagesandprocessedimages.


Real-time, multi-directional 2D fast wavelet transform and its denoised sharpening applicationB.Baek,T.Kim,SAMSUNGElectronicsCo.,Ltd.(Korea,Republicof)

Thenew2Dfastwavelettransformandreal-timewaveletapproachhavebeenpresentedthroughthepaper.Theproposedwavelettransformincludesmulti-directionaldecomposition/reconstructionconceptdesignedtoshowbetterperformanceneardiagonaledgesthantwo-directionbasedfastwavelet.Thereal-timewaveletarchitectureforhardwarecost-effectiveisalsopresentedandshowsblockingartifact-freeperformance.Usingasmallsizeofwaveletgivesrisetotheuseofasignificantlyreducedamountoflinememoryandmuchlowerlatencyandusingoverlappedwindowsinsteadofexclusivemacroblockwindowsresultsinavoidingblockingartifactthateasilyhappensinmacroblockbasedtransform.Thisarchitecturecorrespondswelltotherasterscanorderimagesensorsaswell.Inaddition,theconceptofpower-scalablemethodissuggestedinbriefasawaytocontrolthetradeoffbetweenpowerconsumptionandblocking-artifact-freeperformance.Experimentsshowtheresultthattheproposedwaveletandreal-timeapproachhasmuchlowerblockinganddirectionalartifacts.Asaresult,theproposedwaveletandreal-timesystemimplementationapproachmaketheuseofawaveletmorefeasibleforreal-timeapplicationwithmuchlessartifactsbysignificantreductionofhardwareresources.Thisenablesthenewtransformtohavemanypotentialapplicationareas.


Color transfer using semantic image annotationF.Gasparini,R.Schettini,C.Cusano,Univ.degliStudidiMilano-Bicocca(Italy)

Conference 8299


Inthisworkwepresentanautomaticlocalcolortransfermethodbasedonsemanticimageannotation.Withthisannotation,imagesaresegmentedintohomogeneousregions,assignedtosevendifferentclasses(vegetation,snow,water,ground,street,andsand).Ourmethodpermitstoautomaticallytransferthecolordistributionfromregionsofthesourceandtargetimagesannotatedwiththesameclass(forexampletheclass“sky”).Itisalsopossibletoselectthecolordistributionscorrespondingtoregionsofdifferentclasses,startingfromasetofannotatedimages(forexampletheclass“sky”fromoneannotatedsourceimageandtheclass“vegetation”fromadifferentsourceimage).Thesecolordistributionswillthenbetransferredtothecorrespondingregionsofthetargetimage.

Inthefinalpaperwewillprovidemoredetailsofthedevelopedmethodandfurtherexamples.


Adaptive directional color image sharpening with overshoot controlT.Tajbakhsh,TechnischeUniv.Hamburg-Harburg(Germany)

Thepaperpresentsanalgorithmforcontrastenhancementofnoisycolorimagesbasedonadaptiveunsharpmasking.Colorimagesareprocessedbytransformingthegammaencodedsignaltoanopponentcolorspaceandapplyingedgesharpeningtotheluminancesignal.Theadaptiveunsharpmaskingalgorithmfusionsfivekeyideas:theadaptiveWienerfilterasproposedbyLeetoadaptthefilteringstrengthlocally,itsdirectionalrefinement,theovershootcontrolasproposedbyBrunaetal.,theregulationoftheamountofcontrastenhancementbasedontheedgestrengthasproposedbyPoleseletal.,andthetemporalnoiseestimationtechniqueproposedbyFoietal.Theresultsareedgeemphasizedimageswithoutunwantedovershootartifactsandvirtuallynonoiseamplificationinsmoothareasandreducednoiseamplificationatedgetransitions.

Conference 8299


Conference 8300: Image Processing: Machine Vision Applications VWednesday25January2012

Part of Proceedings of SPIE Vol. 8300 Image Processing: Machine Vision Applications V


Efficient local approximation of perceptual color differences for color inspectionR.Huber-Mörk,AustrianInstituteofTechnology(Austria)

Wesuggestalocalapproximationofperceptualcolordifferencesinadevicedependentcolorspace,e.g.theRGBspace.TheapproximationisefficientlycomputedfrommeasuringEuclideancolordistanceinthedevicedependentcolorspacecombinedbywithanassociatememorydatastructure.Establishedmeasuresofcolordifferenceareconsidered.Thesuggestedapproachissuitedforindustrialcolorinspectionwheresmalltolerancelevelsforcolordifferencesareknowninadvance.Noon-linecolorspacetransformationorexpensiveevaluationofadvancedcolormetricsisnecessary.Thecomputationtimeisindependentfromtheusedmetric.Resultsforapproximationofsmallperceptualcolordifferencesinacolorinspectionsetuparegiven.


Modified fuzzy c-means applied to a Bragg-grating-based spectral imager for material clusteringA.Rodríguez,J.L.Nieves,E.Valero,Univ.deGranada(Spain);E.Garrote,TECNALIA(Spain);J.Hernández-Andrés,J.Romero,Univ.deGranada(Spain)

Anadaptationoffuzzyc-meansalgorithmhasbeenusedforsegmentationofhyperspectralurbanimagesprovidedbyaBragg-basedspectralimager.

Classicalfuzzyc-meansalgorithmusesEuclideandistanceforcomputingsamplemembershiptoeachcluster.Wehaveintroducedadifferentdistancemetric,SpectralSimilarityValue(SSV),inordertohaveamoreconvenientsimilaritymeasureforreflectanceinformation.SSVdistancemetricconsidersbothmagnitudedifference(bytheuseofEuclideandistance)andspectralshape(bytheuseofPearsoncorrelation).Thecombinationofthesetermsmakesitamoreconvenientmetricforcreatingspectrallysimilarclusters.

Twosetsofimageshavebeenusedtotestthealgorithmperformance.FirstsetwasacollectionofhyperspectralurbanimagesacquiredbyaBragggratingbasedimager.Resultsusingtheseimagesshowthatclustersarespectrallymorehomogeneousthanclusterscreatedwiththeoriginalfuzzyc-meansalgorithm.

Thesecondexperimentwascarriedoutwithasetof100hyperspectralsyntheticimages.Theproposedmethodimprovesthesegmentationperformanceandassignshighestmembershipvaluetopixelscorrespondingtohighestpurityreflectance.


Robust recognition of 1D barcodes using hough transformJ.Dwinell,L.X.Bian,SICK,Inc.(UnitedStates);P.Bian,MicrosoftCorp.(China)

Automatedpackagehandlingsystemsrequirearobustsolutionfordecodingbarcodes.Typicalapplicationsprocessseveralthousandpackagesperhourathighspeeds.Throughputsaretypically2to3packagespersecond.Thereisonlyoneopportunitytoacquireapackageimageandmanyfactorscontributetopoorqualityofthebarcode.

Anefficientandpowerfulbarcodedecodealgorithmisneededto

analyzelowqualityordamagedcodesandisabletohandlethehighthroughput.Amulti-levelalgorithmisshownthatmeetsthischallenge.Atthelowlevel,initialanalysisisperformedinthegray-scaledomain.ThemiddlelevelintroducesaHoughtransformdomain.ThetraditionalHoughtransformisoptimizedforbarcodedecodeprovidingaveryfastyetpowerfulalgorithmforbadlydamagedregionsofthebarcode.Finally,atoplevelguidesthelowerlevelprocessingutilizingvariousglobalinformation.

ThetraditionalHoughtransformforastraightlineis:

ρ=xcosθ+ysinθ

CalculatinginentireHoughplaneiscomputationallyintensive.TheparametersintheHoughdomainaretheangleandoffsetforeachpoint.TheoptimizationallowstheHoughtransformtobeonlytheoffset.Theoffsetiscriticalaseachlineinfluencesthecorrespondingbarandspacewidthdetermination.AnoptimizedHoughtransformprovidesarobustalgorithmfordeterminingtheoffsetofeachlinewhenworkingwithnoisydata.


Estimating the coordinates of pillars and posts in the parking lots for intelligent parking assist systemJ.Choi,J.G.Kuk,N.I.Cho,SeoulNationalUniv.(Korea,Republicof)

Thispaperproposesavisionalgorithmforthedetectionofpillarsorpostsinthevideocapturedbyasinglefrontviewcameraimplementedontheforesideofaroommirrorinacar.ThemainpurposeofthisalgorithmistocomplementtheweaknessofcurrentultrasonicIPAS,whichdoesnotwellfindtheexactpositionorpillarsorcannotrecognizenarrowposts.Theproposedalgorithmisconsistedofthreesteps:straightlinedetection,linetrackingandestimationof3Dpositionofpillars.Inthefirststep,wefindthestrongestlinebasedHoughtransform.Secondstepisthecombinationofdetectingandtracking,andthethirdisthecalculationof3Dpositionofthelinebytheanalysisoftrajectoryofrelativepositionsandtheparametersofcamera.Experimentsonsyntheticandrealimagesshowthattheproposedmethodsuccessfullylocateandtrackthepositionofpillars,whichhelpstheultrasonicsystemtocorrectlylocatetheedgesofpillars.Itisbelievedthattheproposedalgorithmcanalsobeemployedasabasicelementforvisionbasedautonomousdrivingsystem.


Recognizing human gestures using a novel SVM treeH.Jain,A.Chatterjee,S.Kumar,B.Raman,IndianInstituteofTechnologyRoorkee(India)

Inthispaper,anovelsupportvectormachine(SVM)treeisproposedforgesturerecognitioninalargeenvironmenthavingcomplexbackground.Askeletonbasedstrategyisadoptedtoextractthefeaturesfromavideosequencerepresentinganyhumangesture.Thebackgroundimageforalargesceneisobtainedbyusingmosaicofseveralimagescapturedbyapan-tilt-zoom(PTZ)camerawithdifferentorientations.Thismosaicimageisusedtocomputethebackgroundframeinrealtimeforanyarbitrarypanandtiltsetting.InourbinarytreeimplementationofSVM,thenumberofbinaryclassifiersrequiredisreducedsince,insteadofgroupingdifferentclassestogetherinordertotrainaglobalclassifier,weselecttwoclassesfortrainingateverynodeofthetreeanduseprobabilitytheorytoclassifytheremainingpointsbasedontheirsimilaritiesanddifferencestothetwoclassesusedfortraining.Thisprocessiscarriedon,randomlyselectingtwoclasses


fortrainingatanode,thuscreatingtwochildnodesandsubsequentlyassigningtheclassestothenodesderived.Intheclassificationphase,westartoutattherootnode.Ateachnodeofthetree,abinarydecisionismaderegardingtheassignmentoftheinputdatapointtoeitherofthegrouprepresentedbytheleftandrightsub-treeofthenodewhichmaycontainmultipleclasses.Thisisrepeatedrecursivelydownwarduntilwereachaleafnodethatrepresentstheclasstowhichtheinputdatapointbelongsto.Finally,theproposedframeworkistestedonvariousdatasetstocheckitsefficiency.Encouragingresultsareachievedintermsofclassificationaccuracy.


Fabric defect detection using the wavelet transform in an ARM processorJ.A.FernandezGallego,AntonioNariñoUniv.(Colombia);S.A.OrjuelaVargas,Univ.Gent(Belgium);J.Alvarez,AntonioNariñoUniv.(Colombia);W.Philips,Univ.Gent(Belgium)

Nowadays,advancesintechnologyoffersportabledevicesforourdaylifewithentertainmentandcommunicationapplicationsaswellascustomizablecapabilities.Thesystemarchitecturesofthesedevicesareapowerfultooltodevelopfriendlyvisualenvironmentsforindustrialapplicationssuchasinspection,controlormonitoringtasks.Furthermore,thesesystemsoperateinreducedworkingspacesandofferportabilitycapabilitiesthatareusefulforvisualinspectiontasksamongothers.

Currently,platformswithdirectaccesstotheinstructionssetoftheprocessorarepreferredforrealtimeapplicationsbecausetheypermitafastdataprocessing.

Wepresentinthispaperanexampleoftheuseofanembeddedsystem,theZeusEpic520singleboardcomputer,foranimageprocessingapplicationindefectdetectionbyimplementingthewavelettransformusingtheEmbbededVisualC++4.0compilerforWindowsCE5.

ThemaindifficultyinimplementingthealgorithmwasthefactthattheARMprocessoruseFixedPointArithmeticcalculations.Thealgorithmwastestedfordefectdetectionusingimagesoffabricswithsixtypesofdefects.Anaverageof80%intermsofcorrectdefectdetectionwasobtained,achievingasimilarperformancethanusingprocessorswithfloatpointarithmeticcalculations.


Orthophotoplan segmentation based on regions merging for roof detectionY.ElMerabet,C.Meurie,Y.Ruichek,Univ.deTechnologiedeBelfort-Montbéliard(France);A.Sbihi,EcoleNationaledesSciencesAppliquéesdeTanger(Morocco);R.Touahni,Univ.IbnTofail(Morocco)

Inthispaper,weproposeastrategyofregionsmergingforroofdetectionwhichismadeonpre-segmentationresults.Itisbasedona2Dmodelingoftheroofridgesandregionfeatures.Thepreliminarysegmentationisobtainedbythewatershedalgorithmwithanoptimalcolorimetricinvariantandcolorgradient.Thechoiceofanappropriatecoupleinvariant/gradientpermitstolimitilluminationschanges(shadows,brightness,etc)presentonseveralroofsandincreasesthesegmentationresults.Thewatershedalgorithmofferssatisfactoryresultsbutproducesanover-segmentationduetomanygerms(ie.localminima).Thiseffectisreducedbyusinganappropriateselectionofgermsbutcanalsobeimprovedwithapost-treatmentbasedonregionsmerging.Theproposedmergingcriteriaisbasedonthe2Dmodelingofroofridges(numberofsegmentsmodelingthecommonboundarybetweentworegionscandidatestothefusion)andontheregionfeatures(contrastonboundaryoftwocommonregions,averagecolorofregion).Theproposedstrategyisevaluatedon100realroofimageswiththeVinetcriteriausingagroundtruthinordertodemonstratetheeffectivenessandthereliabilityoftheproposedapproach.


Motion cue analysis for Parkinsonian gait recognitionT.M.Khan,J.Westin,M.Dougherty,DalarnaUniv.(Sweden)

Thispaperpresentsavision-basedmarker-freemethodforgaitimpairmentdetectioninthepatientswithParkinson’sdisease(PWP).Thesystemisbasedontheideathatanormalhumanbodyattainsequilibriumduringthegaitbyaligningthebodyposturewiththeaxisofgravityusingfeetasthebaseofsupport.Duetotherigidityinmusculartone,thePWPfailtoaligntheirbodywiththeaxisofgravity.TheleanedpostureofPWPappearstofallforward.Whereasanormalgaitexhibitsaconstanterectposturethroughoutthegait.PWPwalkwithshortenedstrideangle(lessthan25degreesonaverage)betweenthelegswithhighvariabilitybetweenthestridecycles.Anormalgaitexhibitsperiodicstridecycleswithstrideangleof45degreesonaveragebetweenthelegs.InordertoanalyzeParkinsonianGait(PG),thesubjectswerevideotapedforseveralgaitcycles.Thesubject’sbodyissegmentedineachframeofthegaitvideousingcolorsegmentationmethodbasedonthresholdingtoformasilhouette.Thecentreofgravityofthissilhouetteiscalculated.Thesilhouetteisskeletonizedforthemotioncuesextraction.Twomotioncuesanalyzedwerestridecycles(basedonthecycliclegmotionoftheskeleton)andtheposturelean(basedontheanglebetweentheleanedtorsooftheskeletonandtheaxisofgravity).HighvariationswererecordedinthestrideintervalsofPGwhereasconstantstrideintervalswererecordedincaseofnormalgait.CosineSimilaritymeasurementsbetweenanimaginaryperfectgaitpatternandthesubject’sgaitpatternproduced100%recognitionrateofPGfor7subjects.Resultssuggestedthatthemethodisapromisingtooltobeusedfortheremoteassessmentofgaitbytheclinicians.

8300-01, Session 1

An introduction to omnidirectional vision: theory and applicationsD.Fofi,S.Mosaddegh,A.E.R.Shabayek,O.Morel,Univ.deBourgogne(France)

Thispaperpresentsanoverviewoftheomnidirectionalvisiontechniquesandtheirpotentialapplicationsindifferentfieldsofmachinevision:robotics,industrialinspection,medicalimaging,videosurveillanceandculturalheritage.Afterabriefintroduction,ataxonomyoftheomnidirectionalcamerasisproposed(speciallenses,polydioptric,catadioptric,centralornon-central)withthescientificchallengestheygiveriseto.Alistofapplicationsthatcanbefoundintheliteratureisalsoreported.

Thispaperdoesnotpretendtoproposenewmaterialinthefieldofomnidirectionalvision,buttogatherthemainresultsinacomprehensiveoverview,tohighlighttheadvantagesanddrawbacks,andtosuggesttheuseofomnidirectionalsensorsformachinevision(andrelated)applications.

8300-02, Session 1

Sensor placement optimization in buildingsS.Bianco,R.Schettini,F.Tisato,Univ.degliStudidiMilano-Bicocca(Italy)

Inthisworkweaddresstheproblemofoptimalsensorplacementforagivenregionandtask.Theproblemofsensorplacementarisesinseveralareasofcomputervision,computergraphics,androbotics,suchaslocalization,tracking,surveillance,objectorenvironmentreconstruction,andimagebasedrendering.

Animportantissueindesigningsensorarraysistheappropriateplacementofthesensorssuchthattheyachieveapredefinedgoal.

Weconsiderfourdifferentproblems:maximizingcoveragesubjecttoagivennumberofsensors(a)oramaximumtotalpriceofthesensorarray(b),optimizingsensorposesgivenfixedlocations(c),

Conference 8300


andminimizingthecostofasensorarraygivenaminimallyrequiredpercentageofcoverage(d).

Tosolvethefourproblemsconsidered,weproposetwodifferentoptimization-basedalgorithms:theformerfordiscreteproblemspaces,thelatterforcontinuousones.

Thedifferentalgorithmsproposedareexperimentallyevaluatedandcomparedtostateoftheartalgorithmsforcameraplacementandexperimentalresultsarepresented.Theresultsshowthatthealgorithmsworkwellandaresuitedfordifferentpracticalapplications.

8300-03, Session 1

Optical feature extraction with illumination-encoded linear functionsR.Gruna,KarlsruherInstitutfürTechnologie(Germany);J.Beyerer,Fraunhofer-InstitutfürOptronik,SystemtechnikundBildauswertung(Germany)

Thechoiceofanappropriateilluminationdesignisoneofthemostimportantstepsincreatingsuccessfulmachinevisionsystemsforautomatedinspectiontasks.Inmanyapplicationsitisdifficultorevenimpossibletofindasingleoptimalilluminationconditionandthereforeimagesseriesunderangularly-varyingilluminationdirectionsarecapturedandanalyzed.Inthispaperwepresentanilluminationtechniquewhichreducestheeffortforcapturinginspectionimagesforeachangularreflectancefeaturebyusingacombinationofpointlightsourcesoverthehemisphereasilluminationpattern.Thekeyideaistoencodinglinearfunctionsforfeatureextractionasangular-dependentilluminationpatterns,andtherebytocomputelinearfeaturesfromthescenesreflectancefielddirectlyintheopticaldomain.Duetothelinearityoflighttransport,theillumination-encodedlinearfunctionsforfeatureextractioncanbecomputedofflineusinglabeledtrainingdata.Inourexperimentsweapplytheproposedilluminationtechniqueforsuperviseddefectdetectionandmaterial-basedsegmentationandusehand-labeledgrounddataforevaluation.Forcreatingarbitraryangularly-dependentilluminationpatterns,wepresentanilluminationdevicewhichusesadigitalprojectorasprogrammablelightsourceandaparabolicreflectortodirecttheemittedilluminationpatterns.Sincefeatureextractionistransferredformimages-spacetotheopticaldomain,thenumberofimagesneededtoevaluatereflectancefeaturesintheinspectionprocessisminimized.

8300-04, Session 2

An illumination-invariant phase-shift algorithm in three-dimensional profilometryF.Deng,TheUniv.ofHongKong(HongKong,China)andASMAssemblyAutomationLtd.(HongKong,China);E.Y.Lam,TheUniv.ofHongKong(HongKong,China);W.Sze,J.Deng,K.S.Fung,W.Leung,C.Liu,ASMAssemblyAutomationLtd.(HongKong,China)

Sinceunevenilluminationiscommoninrealopticalsystemformachinevisionapplicationsandcausesmajorerrorwhenusingphase-shiftalgorithm(PSA)forsurfacereconstruction,westudytheunevenilluminationeffectonphase-measuringprofilometry,thenproposeanovelillumination-reflectivity-focus(IRF)modeltohandleunevenilluminationeffect.

Underthismodel,weseparatetheilluminationfactoreffectivelyandformulatethephasereconstructionasanoptimizationproblem.Tosimplifytheoptimizationprocess,wecalibratetheunevenilluminationfactorbeforehandandthenusethecalibratedilluminationinformationduringsurfaceprofilometry.Aftercalibration,thedegreesoffreedomarereduced.Accordingly,wedevelopanovelillumination-invariantphase-shiftalgorithm(II-PSA)toreconstructthesurfaceofamovingobjectunderunevenilluminationenvironment.Experimentalresultsshowthatouralgorithmcanimprovethereconstructionqualityvisuallyandnumerically.Therefore,usingthisIRFmodelandthecorrespondingII-PSA,wecannotonlyhandleunevenilluminationeffectinrealopticalsystemwithlargefiledofview,butalsodeveloparobustandefficientmethodforphase-measuringprofilometry.Andthisnovelmethodcan

beappliedtoreal-time,high-precisionmachinevisionapplicationssuchasinspectionofsemiconductorcomponents.

8300-05, Session 2

Fusing shape and texture features for pose-robust face recognitionT.Gernoth,R.Grigat,TechnischeUniv.Hamburg-Harburg(Germany)

Unconstrainedenvironmentswithvariableambientilluminationandchangesofheadposearestillchallengingformanyfacerecognitionsystems.Torecognizeapersonindependentofpose,weseparateshapefromtextureinformationusinganactiveappearancemodel.Wedonotdirectlyusethetextureinformationfromtheactiveappearancemodelforrecognition.Insteadweextractlocaltexturefeaturesfromashapeandposefreerepresentationoffacialimages.Weuseasmoothwarpfunctiontotransformtheimages.Wecompensatealsotheshapeinformationforheadposechangesandfusetheresultsofseparateclassifiersforshapefeaturesandlocaltexturefeaturesatthedecisionlevel.Weanalyzetheinfluenceoftheindividualcontributionsofshapeandtextureinformationontherecognitionperformance.Weshowthatfusingshapeandtextureinformationatthedecisionlevelconsiderablybooststherecognitionperformanceinanaccesscontrolscenario.Furthermore,ourexperimentalresultsshowasignificantimprovementinfacerecognitionperformanceonfaceswithposevariationswhencomparedwithatraditionalappearancebasedfacerecognitionsystem.

8300-06, Session 2

Automated inspection of tubular material based on magnetic particle inspectionA.Mahendra,C.Stolz,F.Meriaudeau,Univ.deBourgogne(France);S.Petit,A.Noel,F.Degoutin,VallourecS.A.(France)

AutomaticindustrialsurfaceinspectionmethodologybasedonMagneticParticleInspectionisdevelopedfromimageacquisitiontodefectclassification.Firsttheacquisitionsystemisoptimized,thentubularmaterialimagesareacquired,reconstructedthenstored.Thecharacteristicsofthecrack-likedefectwithrespecttoitsgeometricmodelandcurvatureareusedasaprioryknowledgeformathematicalmorphologyandlinearfiltering.Afterthesegmentationandbinarizationoftheimage,vastamountofdefectcandidatesexists.Finallyclassificationisperformedwithdecisiontreelearningalgorithmduetoitsrobustnessandspeed.Theparametersformathematicalmorphology,linearfilteringandclassificationareanalyzedandoptimizedwithDesignOfExperimentsbasedonTaguchiapproach.Themostsignificantparameterobtainedcanbeanalyzedandtunedfurther.Theexperimentisperformedontubularmaterialsandevaluatedbyitsaccuracyandrobustnessbycomparingthegroundtruthandtestingimages.Theresultispromisingwith97%TruePositiveand0.01%FalsePositiverateonthetestingset.

8300-07, Session 2

Intermediate-level segmentation of color images through perception and geometry-based contour completions and shape cutsJ.Grazzini,L.Prasad,LosAlamosNationalLab.(UnitedStates)

Inthispaper,wepresentahybridsegmentationmethodthatprovidesanintermediate-levelrepresentationofimagesintoperceptuallymeaningfulentitieswithoutanypriorknowledgeoftheimagecontent,noranyobjectmodel.Itpartitionsimagesintogenericpartsmadeofclosedregionswithaccuratelypositionedboundariesusingregionandboundaryinformation.Thisapproachisconsistentwithclassicalcomputationalvisionmodels,whileitsimplementationismotivatedbypracticalratherthantheoreticalconsiderations.Itbenefitsfromlow-levelfeaturedetection,imagetriangulationandshapedecomposition

Conference 8300


basedontheunderstandingofhigher-levelperceptualbehavior.Namely,contourfragmentsarefirstextractedtoapproximatetheactualboundariesoftheimagestructures.Then,atriangulationiscomputedoversubsampledcontourstoprovidewithanaturaldecompositionintosimplebuildingblocks.Finally,perceptionandgeometric-basedfiltersareappliedontheinterioredgesofthetriangulationtochooseamongthemasetofoptimalcontourcompletionsandshapecutsdelineatingobjects.Thenoveltyofthisapproachrevolvesaroundtheshapedecompositiontechniqueadaptedforthedesignofthosefilters.Whiletheresultingpartitiondoesnotnecessarilycorrespondtothenaturalobjectsintheimage,itcanbeusedforanaccuratedetectionoftherealsalientobjects.

8300-08, Session 3

Runway hazard detection in poor visibility conditionsB.Jiang,NationalInstituteofAerospace(UnitedStates);Z.Rahman,OldDominionUniv.(UnitedStates)

Morerecently,researchonenhancingthesituationalawarenessofpilots,especiallyinpoorvisibilityflightconditions,gainsmoreandmoreinterests.Sincepilotsmaynotbeabletospottherunwayclearlyinpoorvisibilityconditions,suchasfog,smoke,hazeordimlightingconditions,aviationlandingproblemcanoccurduetothe(unexpected)presenseofobjectsontherunway.Complicatedandtrivialinstruments,switches,bottoms,plussuddenhappeningsareenoughforthepilotstotakecareofduringlandingapproach.Therefore,anautomatichazarddetectionapproachthatutilizessmartimageenhancement,edgedetectionandimageanalysisisproposed.Todetectthepresenceofobjectsontherunway,theexistingcannoicaledgerepresentationsoftherunwayarecompardwithenhancededgerepresentationsofthegeometricallycorrectedapproachimage.Theeffectofapplyingtheenhancementmethodistomaketheimageoftherunwayalmostindependenttheatmosphericconditions.Thefollowingedgedetectionprocesscanreducethestoringspace,thecomparisonandretrievaltime,andtheeffectofsensornoise.Afteranalyzingthefeaturesexistingintheedgedifferences,thepotentialhazardwillbelocalizedandlabeled.Experimentalresultsshowthattheproposedapproachiseffectiveinrunwayhazarddetection.

8300-09, Session 3

Application of image processing to track twin boundary motion in magnetic shape memory alloysA.Rothenbuhler,E.H.BarneySmith,P.Müllner,BoiseStateUniv.(UnitedStates)

Weproposeanexperimentalmethodtofacilitatemagneticshapememoryalloyresearchthroughtheuseofimageprocessing.Currentmagneto-mechanicalexperimentsonlymeasurestraininonedirection.Furthermore,insituvisualizationoftwinboundariesaredifficultastheyareonlyvisibleunderparticularlightingconditions.Theexperimentalmethodproposedhereenablesresearcherstogetstrainaswellastwinboundaryinformationinasimplifiedexperiment.StraininNi2MnGashapememoryalloysisopticallymeasuredbyapplyinganedgedetectionalgorithm.Thisalgorithmconsistsofthresholdingtheimagesincethebackgroundissignificantlydarkerthanthesampleitself.Througharegionofinterest,thestationarysampleholderiscutout.Agridoverlayisusedtomeasurethedetectedsample’sdimensions.Thisinformationisthenusedtocalculatestrainovermultipleimageframes.Sincetwinboundariescausesomepartsofthesampletomove,amotiondetectionalgorithmwasdevelopedtofindthemovingpartsofthesample.Thesample’smotionisanalyzedwithaHoughtransformtoextractthelocationoftwinboundaries(theyappearasstraightlines).TheHoughaccumulatorprovidesallnecessaryinformationaboutthelocationofalltwinboundarieslocationandisusedtovisualizethemonthecapturedimage.

8300-10, Session 3

A new point process model for trajectory-based events annotationN.Ballas,CEALIST(France)andMinesParisTech(France);B.Delezoide,CEALIST(France);F.Prêteux,MinesParisTech(France)

Humanactionsannotationinvideoshasreceivedanincreaseattentionfromthescientificcommunitytheselastyearsmainlyduetoitslargeimplicationformanycomputervisionapplications.Thecurrentleadingparadigmtoperformhumanactionsannotationisbasedonlocalfeatures.Localfeaturesrobusttogeometrictransformationsandocclusionareextractedfromavideoandaggregatedtoobtainaglobalvideorepresentation.CurrentaggregationschemessuchasBag-of-Wordsorspatio-temporalgridshavenoorlimitedinformationaboutthelocalfeaturesspatio-temporallocalization.However,ithasbeenshownthatlocalfeatureslocalizationcanbediscriminativefordetectingaconceptoraction.Inthisworkweimproveontheaggregationstepbyembeddinglocalfeaturesspatio-temporalinformationinthefinalvideorepresentationbyintroducingapointprocessmodel.

Weproposeaneventrecognitionsysteminvolvingtwomainsteps:

(1)localfeaturesextractionbasedonrobustpointtrajectories,and(2)aglobalactionrepresentationcapturingthespatio-temporalcontextinformationthroughaninnovativeevents-basedpointprocessclustering.Apointprocessprovidesindeedawell-definedformalismtocharacterizelocalfeatureslocalizationalongwiththeirinteractioninformation.Italsobenefitsfromtheclassificationcontext,buildingclustersoftrajectorieconstrainedbytheactiontodetect.Resultsareevaluatedonthreestandarddatasets(KTH,YouTube,Hollywood2)showingasignificantimprovementoverthestate-of-art.

8300-11, Session 3

Face detection and eyeglasses detection for thermal face recognitionY.Zheng,AlcornStateUniv.(UnitedStates)

Thermalfacerecognitionbecomesanactiveresearchdirectioninhumanidentificationbecauseitdoesnotrelyonilluminationcondition.Facedetectionandeyeglassesdetectionarenecessarystepspriortofacerecognitionusingthermalimages.Infraredlightcannotgothroughglassesandthusglasseswillappearasdarkareasinathermalimage.Onepossiblesolutionistodetecteyeglassesandtoexcludetheeyeglassesareasbeforefacematching.Inthermalfacedetection,aprojectionprofileanalysisalgorithmisproposed,whereregiongrowingandmorphologyoperationsareusedtosegmentthebodyofasubject;thenthederivativesoftwoprojections(horizontalandvertical)arecalculatedandanalyzedtolocateaminimalrectangleofcontainingthefacearea.Ofcourse,thesearchingregionofapairofeyeglassesiswithinthedetectedfacearea.Theeyeglassesdetectionalgorithmshouldproduceeitherabinarymaskifeyeglassespresent,oranemptysetifnoeyeglassesatall.Intheproposedeyeglassesdetectionalgorithm,blockprocessing,prioriknowledge(i.e.,lowmeanandvariancewithinglassesarea),regionmerging,andregiongrowingareemployed.Theresultsoffacerecognitionandeyeglassesdetectionwillbepresented,andquantitativelyanalyzed.

8300-12, Session 4

Combining spatial and spectral information to improve crop/weed discrimination algorithmsG.Jones,S.Villette,J.Paoli,C.Gée,AgroSupDijon(France)

Thereductionofherbicidesprayingisanimportantkeytoenvironmentallyandeconomicallyimproveweedmanagement.Toachievethis,remotesensorssuchasimagingsystemsarecommonlyusedtodetectweedplantsinfield.Wedevelopedseveralspatialalgorithmsthatdetectthecroprowstodiscriminatecropfromweeds.Thesealgorithmshavebeenthoroughlytestedandprovidedrobust

Conference 8300


andaccurateresultswithoutneedinganylearningprocess.Theirmajorlimitationisthefactthattheyonlydiscriminateweedsininter-rowareas.Crop/Weeddiscriminationusingspectralinformationisabletodetectintra-rowweedsbutgenerallyneedsapriorlearningprocess.

Weproposeanewmethodbasedonspatialandspectralinformationtoenhancethediscriminationandovercomethelimitationsofbothalgorithms.Threemethodsaredevelopedandcombinetheadvantagesofspatialandspectralmethodswithouttheirdrawbacks.

Withthisnewapproachweareabletoimprovetherangeofweeddetectionintheentirefield(interandintra-row).Totesttheefficiencyofthesealgorithms,arelevantdatabaseofvirtualimagescombinedtoLOPEX93spectraldatabasehasbeenused.

Thethreemethodsareevaluatedinthispaperandallowanimprovementfrom70%ofweeddetectiontomorethan90%.

8300-13, Session 4

Automated parasite detection in clams by transillumination imaging and pattern classificationM.E.Soto,P.A.Coelho,S.N.Torres,D.G.Sbarbaro,Univ.deConcepción(Chile)

Qualitycontrolofclamsconsidersthedetectionofforeignobjectslikeshellpieces,sandandevenparasites.Particularly,MuliniaedulisclamsarehighlysusceptibletohaveaparasiteinfectioncausedbytheisopodaEdoteamagellanica,whichrepresentsaseriouscommercialproblemcommonlyaddressedbymanualinspection.

Inthispaperanovelmachinevisionsystemcapableofautomaticallysolvethisneedispresented.Theparasitevisualizationinsidetheclamisachievedbyanoptoelectronicimagingsystembasedonaninnovativetransilluminationtechnique.Furthermore,automaticparasitedetectionintheclam’simageisaccomplishedbyapatternrecognitionsystemdesignedtodescribequantitativelytheregionsthatconstitutetheclam.

Theextractedfeaturesareusedtopredicttheparasitepresencebymeansofadecisiontreeclassifier.192shell-offcookedMuliniaedulisclamsfromtheChileansouthpacificcoastsareusedtogeneratearealsampledataset.Morethan155,000clam’sregionspatternsareemployedtotesttheproposeddetectionsystem.

Theclassifierperformanceisevaluatedusingcross-validation.Primaryresultshaveshownameanparasitedetectionrateof85%andameangeneralclassificationrateof87%,whichrepresentasubstantiveimprovementtotheexistingsolutions.

8300-14, Session 4

Vision-based, in-line fabric defect detection using yarn-specific shape featuresD.Schneider,T.Aach,RWTHAachen(Germany)

Wedevelopamethodologyforautomaticin-lineflawdetectioninindustrialwovenfabrics.Wherestateoftheartdetectionalgorithmsapplytextureanalysismethodstooperateonlow-resolved(~200ppi)imagedata,wedescribehereaprocessflowtosegmentsingleyarnsinhigh-resolved(~1000ppi)textileimages.Fouryarnshapefeaturesareextracted,allowingaprecisedetectionandmeasurementofdefects.Thedegreeofprecisionreached,allowsaclassificationofdetecteddefectsaccordingtotheirnature,providinganinnovationinthechallengeofautomaticfabricflawdetection.Thedesignhasbeencarriedouttomeetrealtimerequirementsandfaceadverseconditionscausedbyloomvibrationsanddirt.Theentireprocessflowisdiscussedfollowedbyanevaluationusingadatabasewithreal-lifeindustrialfabricimages.Thisworkpertainstotheconstructionofanon-loomdefectdetectionsystemtobeusedinthemanufacturingpractice.

8300-15, Session 4

3D temperature mapping of turboshaft components using thermal paints and color recognitionS.Guérin,TurbomecaSA(France)andONERA,TheFrenchAerospaceLab(France);C.Lempereur,ONERA,TheFrenchAerospaceLab.(France);P.Brevet,TurbomecaSA(France)

Inordertoenhanceturboshaftlifespan,aeronauticalenginemanufacturersdevelopspecificmeasurementtechniques.Particularly,surfacetemperatureofenginecomponentsisacriticalaspectofthedesign.Usualtemperaturemeasurementtechniquesreachtheirlimitinthisharshenvironmentsothermalpaintshavebeenusedsinceseveralyearsassociatedwithskilledoperatorobservations.Thisarticledescribesasystemcapableofrobustcolorrecognitionappliedtothermalpaintscolorvariation.Therelevanceofthedescribedmethodistooutperformtemperatureresolutionandrobustnessandtomakemorereliablecomparisonbetweennumericalsimulationandbenchtestmeasurement.

A3Dnon-contactdigitizercapturesinthesameplanecolorimagesanddepthmapwhichfacilitatestheprojectionofthemeasuredtemperaturemaponthe3Dmesh.Temperaturemapisprocessedbymeansofacolorrecognitionalgorithmassociatedwithacolortotemperaturedatabase.Thedatabaseiscomposedofasetofmetallicsamplesheatedinalaboratorykilnin10°Csteps.Differentcolorimetricdistancesareusedtocompareeachpixeltothedatabaseandfindthebestmatchingsample.Finally,thesystemgivesarobustmeasurement,whichiscurrentlyusedintheenginedevelopmentprocessatTurbomeca.

8300-25, Session 4

Strain analysis by regularized non-rigid registrationA.Badshah,P.L.O’Leary,M.J.Harker,MontanUniv.Leoben(Austria)

Thispaperinvestigatesanewregularizationschemewhichcanbeusedinconjunctionwithnon-rigidregistrationtoperformopticalmaterialstrainanalysis.Muchworkhasbeenperformedonopticalstrainanalysis,seeforexample,1{4theseallusesomeformofelasticregistrationtoperformnon-contactstrainmeasurement.However,theissueofregularizationisnotsystematicallydealtwith:forexample,HaileandIfuj4usealocallyweightedmeantransform.Thisisinnowayrelatedtothephysicsoftheproblemathand.Koljonenetal.5investigateddi®erentmeasuresforcon¯denceandadaptedthetemplatesizetoachieveahigherdegreeofcon¯dence.Thispaperinvestigatesusingasetoftensorpolynomialstoimplementregularization.ThetensorpolynomialscorrespondtobasisfunctionswhichformsolutionstotheNavier-Stokespartialdi®erentialequation,

@u

@t

+u¢ru=

rP

½

+vr2u(1)

whereby,uisthevelocityvector,Pisthepressurevector,½the°uiddensityandvisthekinematicviscosity.

Thisequationdescribesthe°owofviscousmedia,i.e.thephysicsunderlyingthismeasurementproblem.Thecoe±cientsforthetensorpolynomialsaredeterminedviaadirectleastsquaresapproachwithlocalentropyweighting.Thenon-rigidregistrationmethodutalizedisbasedonahierarchicaldecompositionwithdecimation.Theprocedurehasbeendescribedindetailbytheauthorsin.6Theprocedureisverystableduetotheentropyweighting,i.e.eachpatchintheimageisbyitrelativeentropyduringtheleastsquaresapproximation.Thenewdirect¯ttingprocedureissu±cientlyfastsothatitcanbeusedinrealtime.TheuniversaltestingmachineusedcanbeseeninFigure1andthesampleheldinthejawsofthemachineinFigure2.Theresultsofthe

Conference 8300


¯rsttestontheextensionofasampleisshowninFigure3:thesamplepriortoandafterextensionareshowntogetherwiththelasttwolayersofthehierarchicalregistrationareshown.Ateachlayerinthehierarchythepatchsizeishalvedinsizeandthenumberofnodesinthesupportofthecorrespondingtensorproductisincreased.Thecolorofthepatchisproportionaltotheentropyinthecorrespondingimagepatchandisusedastheweightingduring¯tting.Theentropyweightingensuresthatpatchedwithonlyalittleinformationdonotcontributestronglytotheleastsquaressolution.Itcanbeseenfromthis¯gurethatthecorrectextensionandshearingofthematerialhasbeedetected.Inthesecondtestaporouscompressiblemediumiscompressed,seeFigure4and5.Thetypicalbarreldistortionwith°attopsandbottomscanbeobserver.Theareaoftheoriginalgridcanbecomparedwiththeareaofthecompressedgridtodeterminethevolumetriccompressionofthematerial.InthisworkithasbeenshownthatthenewtechniquesbasedontensorpolynomialproductsolutionstotheNavier-Stokesequationcanbeusedtoimplementregularizationinopticalstrainmeasurements.Theuseofentropyweightingleadstoarobustsolution.Thisabstractshownthatthereisjusti¯cationforfurtherinvestigationofthistechnique.

Conference 8300


Conference 8301: Intelligent Robots and Computer Vision XXIX: Algorithms and TechniquesMonday-Tuesday23-24January2012

Part of Proceedings of SPIE Vol. 8301 Intelligent Robots and Computer Vision XXIX: Algorithms and Techniques

8301-01, Session 1

Software-based, neural-network-assisted movement compensation for nanoresolution piezo actuatorsJ.Röning,M.Kauppinen,Univ.ofOulu(Finland)

Thispaperpresentsasoftwarebasedcontrollerimplementationutilizingneuralnetworksforhighprecisionpositioningofapiezoelectricactuator.Thecontrollerdevelopedcanbeusedforcontrollingnanopositioningpiezoactuatorswhensufficientlyaccuratefeedbackinformationisavailable.

Piezoactuatorsexhibitcomplexhysteresisdynamicsthatneedtobetakenintoaccountwhendesigninganaccuratecontrolsystem.Forinversemodellingpurposesofthehysteresisrelatedphenomena,astatichysteresisoperatorandanewdevelopeddynamiccreepoperatorispresentedtobeusedinconjunctionwithaFeedForwardtypeneuralnetwork.

ThecontrollerutilizingtheneuralnetworkinversehybridmodelisimplementedasasoftwarecomponentforexistingScalableModularControlframework(SMC).UsingtheSMCframeworkandoff-the-shelfcomponents,ameasurementandcontrolsystemforthenanopositioningactuatorisconstructedandtestedusingtwodifferentcapacitivesensorsoperatingony-andz-axesoftheactuator.

Usingthedevelopedcontroller,piezoactuatorrelatedhysteresisphenomenaweresuccessfullyreducedmakingthenanometerrangepositioningoftheactuatoraxespossible.Alsotheeffectofusingaloweraccuracypositionsensorwithmorenoisetocontrolaccuracyisbrieflydiscussed.

8301-02, Session 1

Traffic monitoring with distributed smart camerasO.Sidla,M.Rosner,SLREngineeringOG(Austria);M.Ulm,AustrianInstituteofTechnology(Austria)

Theobservationandmonitoringoftrafficwithsmartvisionssystemswillbecomeincreasinglyimportantintheyearstocome.Inthispaperwepresentasystemwhichisdesignedtomonitortrafficatarelativelybusypedestriancrossingisdescribed.Thesystemobservestheinteractionbetweenvehiclesandpedestriansatthecrossingandtriestodetectincidentsinwhichtheinteractionofpedestriansandvehiclesmightleadtodangerousorevencriticalencounters.

Theproposedsystemconsistsofaclusterof3networkedsmartcameras,basedoncompactPChardware:

Camera1,2:Vehicledetectionandtrackingbasedonastate-of-the-artcascadedHOGdetector.ThetrackerincludesopticalflowcomputationwithamodifiedKLTpointtracker.

Camera3:Pedestriantrackingwithatracking-by-detectionapproachbasedonacascadedHOGdetector.

Wedescribetheprocessingpipelineoftheobject(pedestrian,vehicle)trackingsystemwhichcombines

-acascadedHOGdetector

-amodifiedandimprovedKLTpointtracker

-featureanalysistoenhancetrackingandremovefalsetrackingoccurrences

ThewellknownKLTtrackingalgorithmhasbeenmodifiedinordertoincludestate-of-the-artkeypointdetectionsothatthequalityoffeature

pointswhichareselectedfortrackingimprovessignificantlycomparedtotheoriginalalgorithm.

Afterdetectionandtrackingall3smartcamerasprojecttheirvideodetectionresultstogroundplane.Basedonthelocationandvelocityofallobjectsdetectedinreal-worldco-ordinates,themastercamerathenexecutesasetofrulesinordertodecidewhetheracriticaleventiscurrentlybeingobserved.

TheremainingpaperconcentratesonthesystemarchitectureanddescribesresultsofourexperimentsduringextensivetrialsandtestsinanoutdoorenvironmentintheCityofVienna,Austria.

8301-03, Session 1

The 19th annual intelligent ground vehicle competition: student-built autonomous ground vehiclesB.L.Theisen,U.S.ArmyTankAutomotiveResearch,DevelopmentandEngineeringCtr.(UnitedStates)

TheIntelligentGroundVehicleCompetition(IGVC)isoneoffour,unmannedsystems,studentcompetitionsthatwerefoundedbytheAssociationforUnmannedVehicleSystemsInternational(AUVSI).TheIGVCisamultidisciplinaryexerciseinproductrealizationthatchallengescollegeengineeringstudentteamstointegrateadvancedcontroltheory,machinevision,vehicularelectronicsandmobileplatformfundamentalstodesignandbuildanunmannedsystem.Teamsfromaroundtheworldfocusondevelopingasuiteofdual-usetechnologiestoequipgroundvehiclesofthefuturewithintelligentdrivingcapabilities.Overthepast19years,thecompetitionhaschallengedundergraduate,graduateandPh.D.studentswithrealworldapplicationsinintelligenttransportationsystems,themilitaryandmanufacturingautomation.Todate,teamsfromalmost80universitiesandcollegeshaveparticipated.Thispaperdescribessomeoftheapplicationsofthetechnologiesrequiredbythiscompetitionanddiscussestheeducationalbenefits.TheprimarygoaloftheIGVCistoadvanceengineeringeducationinintelligentvehiclesandrelatedtechnologies.Theemploymentandprofessionalnetworkingopportunitiescreatedforstudentsandindustrialsponsorsthroughaseriesoftechnicaleventsoverthefour-daycompetitionarehighlighted.Finally,anassessmentofthecompetitionbasedonparticipationispresented.

8301-04, Session 2

Accurate, dense 3D reconstruction of moving and still objects from dynamic color image and depth image sequences based on temporal modified-RANSAC and feature-cutN.Tatematsu,J.Ohya,WasedaUniv.(Japan)

Thetechnologyformobilerobotstoautomaticallyreconstruct3Dstructureofthemovingandstillobjectsisincreasingitssignificance.WeproposedaTemporalmodified-RANSAC(TMR)basedmethodthatcantrackmovingobjectsfrommovingstereocamerasandcanreconstruct3Dstructureofthetrackedobjects.However,theTMRbasedmethodhasthefollowingproblems:lackofaccuracyofsegmenting3Dpointsandlackofdensityof3Dreconstructedpoints.

Tosolvetheproblemofourconventionalmethod,thispaperproposesanew3DsegmentationmethodthatutilizesGraph-cut,whichisfrequentlyusedforsegmentationtasks.Morespecifically,our


proposedmethodutilizespre-definedenergyfunctionsthatconsistofcolorprobabilityandspatialstructureprobabilityaswellasa-prioriprobabilitiesbasedonFeature-Cut,whichenablestosegment3Dpointsaccuratelybasedonthesegmentationresultsinthepreviousframe,evenifedgefeaturepointsofthetrackedobjectarenotsuccessfullyfound.

Experimentalresultsusingrealscenesthatcontaindynamicandstaticobjectsdemonstratethatthisalgorithmcanreconstructdense3Dpointsofthemovingobjectsandbackground.Inaddition,weshowexperimentalresultsthatprovethattheaccuracyofsegmentationishigherthanonlyFeature-Cut.

8301-05, Session 2

Efficient hybrid monocular-stereo approach to on-board, video-based traffic sign detection and trackingJ.Marinas,L.Salgado,J.Arróspide,M.Camplani,Univ.PolitécnicadeMadrid(Spain)

Inthispaperweproposeaninnovativemethodfortheautomaticdetectionandtrackingofroadtrafficsignsusinganon-boardstereocamera.Itexploitsacombinationofmonocularandstereoanalysisstrategiestoincreasethereliabilityofthedetectionssuchthatitcanboosttheperformanceofanytrafficsignrecognitionscheme.Firstly,anadaptivecolorandappearancebaseddetectionisappliedatsinglecameralevel,followedbyaSURFbasedmatchingstrategytoperforma3Dreconstructionofcandidateroadsigns:thebest-fittingplanetothe3DcloudofpointspreviouslydetectedisestimatedusingaRANSACbasedapproachtoimproverobustnesstooutliers.Temporalconsistencyofthe3DinformationisincorporatedthroughaKalmanbasedtrackingstage,andtrackingresultsareusedtwofold:toadaptthepreviouslymentionedcolor-baseddetector,thusimprovingdetectionaccuracy,andtoreducetheneedtocomputestereoanalysisforeveryincomingimage,thusincreasingalgorithmefficiency.Theproposedsolutionhasbeentestedwithrealsequencesunderseveralilluminationconditionsandinbothurbanareasandhighways,achievingveryhighdetectionratesinchallengingscenarios,includingrapidmotionandsignificantperspectivedistortion.

8301-06, Session 2

A general model and calibration method for spherical stereoscopic visionW.Feng,TianjinUniv.(China);J.Röning,Univ.ofOulu(Finland);X.Zong,TianjinUniv.ofTechnology(China);B.Zhang,TianjinUniv.(China);J.Kannala,Univ.ofOulu(Finland)

Ingeometricalstereoscopicvisioncalibrationtheobjectiveistodetermineasetofparameterswhichdescribethemappingfrom3Dreferencecoordinatesto2Dimagecoordinates,andindicatethegeometricrelationshipsbetweenthecameras.Whilevariousmethodsforordinarylensesstereoscopicvisioncanbefoundfromtheliterature,stereoscopicvisionwithextremelywideanglelensescalibrationhavebeenmuchlessdiscussed.Sphericalstereoscopicvisionismoreandmoreconvenientincomputervisionapplications.However,itusefor3Dmeasurementpurposesislimitedbythelackofanaccurate,general,andeasy-to-usecalibrationprocedure.Hence,wepresentageometricmodelforsphericalstereoscopicvisionequippedbyextremelywideanglelenses.Then,acorrespondinggenericmathematicalmodelisbuilt.Methodforcalibrationtheparametersofthemathematicalmodelisproposed.Thispapershowspracticalresultsfromthecalibrationoftwohighqualitypanomorphlensesmountedondoublecameraswith2048x1536resolutions.Here,thestereoscopicvisionsystemisflexibly,thepositionandorientationofthecamerascanbeadjustedrandomly.Thecalibrationresultsincludeinteriororientation,exteriororientationandthegeometricrelationshipsbetweentwocameras.Theachievedlevelofcalibrationaccuracyisverysatisfying.

8301-07, Session 2

An approach to stereo-point cloud registration using image homographiesS.D.Fox,D.M.Lyons,FordhamUniv.(UnitedStates)

Amobilerobotequippedwithastereocameracanmeasureboththevideoimageofasceneandthevisualdisparityinthescene.Thedisparityimagecanbeusedtogenerateacollectionofpoints,eachrepresentingthelocationofasurfaceinthevisualsceneasa3Dpointwithrespecttothelocationofthestereocamera:apointcloud.Ifthestereocameraismoving,e.g.,mountedonamovingrobot,aligningthesescansbecomesadifficult,andcomputationallyexpensiveproblem.Manyfinelytunedversionsoftheiterativeclosestpointalgorithm(ICP)havebeenusedthroughoutroboticsforregistrationofthesesetsofscans.However,ICPreliesontheoreticalconvergencetothenearestlocalminimumofthedynamicalsystem:thereisnoguaranteethatICPwillaccuratelyalignthescans.InordertoaddresstwoproblemswithICP,convergencetimeandaccuracyofconvergence,wehavedevelopedanimprovementbyusingsalientkeypointsfromsuccessivevideoimagestocalculateanaffinetransformationestimateofthecameralocation.Thistransformation,whenappliedtothetargetpointcloud,providesICPaninitialguesstoreducethecomputationaltimerequiredforpointcloudregistrationandimprovethequalityofregistration.WereportICPconvergencetimeswithandwithoutimageinformationforasetofstereodatapointcloudstodemonstratetheeffectivenessoftheapproach.

8301-08, Session 2

Hazardous sign detection for safety applications in traffic monitoringW.Benesova,SlovakUniv.ofTechnology(Slovakia);O.Sidla,SLREngineeringOG(Austria);M.Kottman,SlovakUniv.ofTechnology(Slovakia);E.Sikudova,Z.Cernekova,ComeniusUniv.inBratislava(Slovakia)

Thetransportationofhazardousgoodsinpublicstreetssystemscanposeseveresafetythreatsincaseofaccidents.

Oneofthesolutionsfortheseproblemsisanautomaticdetectionandregistrationofvehicleswhicharemarkedwithdangerousgoodssigns.Inthisway,trafficauthoritiescanbemadeawareofthepresenceofdangerousloadsontheirstreets.

Towardssuchanautomatedregistrationofspecialsignswepresentaprototypesystemwhichcandetectatrainedsetofsigns(typicallyasetofapprox50differentlogos)underreal-worldconditionsinreal-time.

Thispapercomparestwodifferentapproachesforthedetection:

Bruteforcematchingwithkeypointdetectionandfeaturematchingandahybridapproachbasedonacode-bookofvisualwordsandbagofwordprocedure.

Theresultsofanextendedseriesofexperimentsareprovidedinthispaper.Theexperimentsaredesignedtoverifytheusabilityofthetwoproposedapproachesinareal-worldscenario.

Allmethodsaretestedandevaluatedusingtwotypesoftestdata:experimentswithprintedlogosandartificialsequencesanddetectionofsignsonactualtrafficonroads.

8301-09, Session 2

PRoViScout: a planetary scouting rover demonstratorG.M.Paar,JOANNEUMRESEARCHForschungsgesellschaftmbH(Austria);M.Woods,SciSysLtd.(UnitedKingdom);C.Gimkiewicz,Ctr.Suissed’ElectroniqueetdeMicrotechniqueSA(Switzerland);F.Labrosse,AberystwythUniv.(UnitedKingdom);A.Medina,GMVS.A.(Spain)

MobilesystemsexploringPlanetarysurfacesinfuturewillrequiremoreautonomythantoday.TheEUFP7-SPACEProjectProViScout(2010-

Conference 8301


2012)establishesthebuildingblocksofsuchautonomousexplorationsystemsintermsofroboticsvisionbyadecision-basedcombinationofnavigationandscientifictargetselection,andintegratesthemintoaframeworkreadyforandexposedtofielddemonstration.

ThePRoViScouton-boardsystemconsistsofMissionManagementcomponentssuchasanExecutive,aMarsMissionOn-BoardPlannerandScheduler,ascienceassessmentmodule,andnavigation&visionprocessingmodules.ThePlatformHWconsistsoftheroverwiththesensorsandpointingdevices.

Wereportonthemajorbuildingblocksandtheirfunctions&interfaces,emphasizingonthecomputervisionpartssuchasImageacquisition(usinganovelzoomed3D-TOF&RGBcamera),Mappingfrom3D-TOFdata,panoramicimage&stereoreconstruction,hazardandslopemaps,VisualOdometryandthemaintenanceofaglobalmap.

Thepaperwillemphasizeonthesystemsengineeringandcomputervisionaspectsaswellasthebenefitsofthe3D-TOFconcept.

FutureactivitieswilladdresstheapplicabilityinrealenvironmentatafieldtestontheislandofTenerifeduringsummer2012.

8301-10, Session 3

Red-light traffic enforcement at railway crossingsO.Sidla,M.Rosner,SLREngineeringOG(Austria)

Railwaycrossings,evenwhensecuredbyautomatedbarriers,poseathreattodriversdayandnight.Especiallydriverswhichtrytocrosstherailwayinthelastmomentsbeforeabarriercloseshaveahighprobabilitytocausesevereaccidents.

Thisworkpresentsanautomatedsystemwhichisdesignedtodetectandrecordvehicleswhichmoveovertherailwaycrossingaftertheredlighthasbeenactivatedjustbeforetheautomatedbarriercloses.Thesystemconsistsofasmartcamerawhichobservesonestreetlaneaswellastheredlightatthecrossing.Triggeredbyanelectricalsignalfromtherailway,thecamerabeginstoobservethecrossing.ThedetectionofmovingvehiclesisbasedonacustomtrainedcascadedHOGdetectorincombinationwithmotionanalysis.Specificallyweimplementatrackerwhichisbasedon

-acascadedHOGdetector

-amodifiedandimprovedKLTpointtracker

-featureanalysistoenhancetrackingandremovefalsetracking/detectionoccurrences

Wedemonstratethatthecombinationofobjectdetectionandopticalflowcanresultinrobustmonitoringsystemswhichoperatedayandnightandarerobustenoughinordertobeusableinreal-worldsituations.

TheremainingsectionsofthepaperconcentrateonthesystemarchitectureanddescribesresultsofourexperimentsduringtestsattwoinstallationsitesinAustria.

8301-11, Session 3

Image projection clues for improved real-time vehicle tracking in tunnelsV.Jelaca,J.O.Niño-Castaneda,A.Pizurica,W.Philips,Univ.Gent(Belgium)

Vehicletrackingisofgreatimportancefortunnelsafety.Todetectincidentsordisturbancesintrafficflowitisnecessarytoreliablytrackvehiclesinreal-time.Thetrackingisachallengingtaskduetopoorlightingconditionsintunnelsandfrequentlightreflectionsfromtunnelwalls,theroadandthevehiclesthemselves.Inthispaperweproposeamulti-cluetrackingapproachcombiningforegroundblobs,opticalflowofShi-TomasifeaturesandimageprojectionprofilesinaKalmanfilterwithaconstantvelocitymodel.Themainnoveltyofourapproachliesinusingverticalandhorizontalimageprojectionprofiles(so-calledvehiclesignatures)asadditionalmeasurementstoovercometheproblemsofinconsistentforegroundandopticalflow

cluesincasesofseverelightingchanges.ThesesignaturesconsistofRadon-transformlikeprojectionsalongeachimagecolumnandrow.Wecomparethesignaturesfromtwosuccessivevideoframestofindtheiralignmentandtocorrectpredictedvehiclepositionandsize.Wetestedourapproachonseveraltunnelsequences.Theresultsshowanimprovementintheaccuracyofthetrackerandlesstargetlosseswhenimageprojectioncluesareused.Furthermore,calculationandcomparisonofimageprojectionsiscomputationallyefficientsothetrackerkeepsreal-timeperformance(25fps,onasingle1.86GHzprocessor).

8301-12, Session 3

Decentralized tracking of humans using a camera networkS.Gruenwedel,V.Jelaca,J.O.Niño-Castañeda,P.VanHese,D.VanCauwelaert,P.Veelaert,W.Philips,Univ.Gent(Belgium)

Real-timetrackingofpeoplehasmanyapplicationsincomputervisionandtypicallyrequiresmultiplecameras,forinstanceforsurveillance,domotics,elderly-careandvideoconferencing.Theproblemischallengingbecauseoftheneedtodealwithfrequentocclusionsandenvironmentalchanges.Anotherchallengeistodevelopsolutionswhichscalewellwiththesizeofthecameranetwork.Suchsolutionsneedtocarefullyrestrictoverallcommunicationinthenetworkandofteninvolvedis-tributedprocessing.Inthispaperwepresentadistributedpersontrackeraddressingtheaforementionedissues.Real-timeprocessingisachievedbydistributingtasksbetweenthecamerasandacentralserver.Thelatterfusesonlyhighleveldatabasedonlow-bandwidthinputstreamsfromthecameras.Thisisachievedbyperformingtrackingfirstontheimageplaneofeachcamerafollowedbysendingonlymetadatatoalocalfusioncenter.Wedesignedtheproposedsystemwithrespecttoalowcommunicationloadandtowardsrobustnessofthesystem.Weevaluatetheperformanceofthetrackerinmeetingscenarioswherepersonsareoftenoccludedbyotherpersonsand/orfurniture.Wepresentexperimentalresultswhichshowthatourtrackingapproachisaccurateevenincasesofsevereocclusionsinsomeoftheviews.

8301-13, Session 3

Real-time detection of traffic events using smart camerasM.M.Macesic,Tehnomobil-Protech(Serbia);V.Jelaca,J.O.Niño-Castaneda,Univ.Gent(Belgium);N.Prodanovic,M.Panic,Univ.ofNoviSad(Serbia);A.Pizurica,Univ.Gent(Belgium);V.Crnojevic,Univ.ofNoviSad(Serbia);W.Philips,Univ.Gent(Belgium)

Withrapidincreaseofnumberofvehiclesonroadsitisnecessarytomaintainclosemonitoringoftraffic.Forthispurposemanysurveillancecamerasareplacedalongroadsandoncrossroads,creatingahugecommunicationload.Therefore,thedataneedstobeprocessedonsiteandtransferredtothemonitoringcentersinformofmetadataorasasetofselectedimages.Forthispurposeitisnecessarytodetecteventsofinterestalreadyonthecameraside,whichimpliesusingsmartcamerasasvisualsensors.

Inthispaperweproposeamethodfortrackingofvehiclesandanalysisofvehicletrajectoriestodetectthedifferenttrafficevents.Kalmanfilteringwasusedfortracking,combiningforegroundandopticalflowmeasurements.

Obtainedvehicletrajectoriesareusedtodetectdifferenttrafficevents.Everynewtrajectoryiscomparedwithcollectionofnormalroutesandclusteredaccordingly.Iftheobservedtrajectorydiffersfromallnormalroutesmorethanapredefinedthreshold,itismarkedasabnormalandthealarmisraised.

SystemwasdevelopedandtestedonTexasInstrumentsOMAPplatform.Testingwasdoneonfourdifferentlocations,twolocationsinthecityandtwolocationsontheopenroad.

Conference 8301


8301-14, Session 3

Mixed road traffic: data acquisition, optical tracking, and microscopic modelingR.Schönauer,TechnischeUniv.Graz(Austria);Y.Lypetskyy,SLREngineeringOG(Austria)

Inthelastyearsmixedtrafficdesignsforurbanroadsareappliedinincreasingnumbers.Importantresearchtasksarisewiththistrend:Tostudyandmodeltheinteractionprocessesbetweentrafficparticipantsaswellastheeffectsofurbaninfrastructuralparametersandlayouts.

Thispaperextendsagentbasedmicroscopicmodelingforhandlinginteractionprocesseswithinmotorizedandnon-motorizedtrafficandinfrastructuralelements.Theworkinthispaperpresentsdynamicvehiclemodelsforbicyclesandcarsrepresentingtheirlimitedmovementcapabilities.

Tocoveralargeandversatileamountofrealworlddataforcalibrationandvalidationprocessesthispaperproposessemi-automateddataacquisitionbyvideoanalysis.Thisworkconcentratesmainlyontheaspectsofasemi-automaticannotationtoolappliedtocreatetrajectoriesoftrafficparticipantsoverspaceandtime.Basedonmanualidentificationofavehicleorpedestrianthetrajectoryiscontinuedwiththehelpofobjecttrackingalgorithms.

Theacquireddataisthenappliedtocalibrateasingletrackmodelwhichnavigatesthrougharoad’ssurfaceandinteractswithitsenvironment.Theresultsofoptimizedcontrolandpreviewparametersshowstablebehaviorandagoodspatialandtemporalcorrelationwiththedata.

8301-15, Session 4

AR.Drone: security threat analysis and exemplary attack to track objects or personsF.Samland,J.Fruth,M.Hildebrandt,T.Hoppe,J.Dittmann,Otto-von-Guericke-Univ.Magdeburg(Germany)

InthisarticleweillustrateanapproachofasecuritythreatanalysisofaquadrocopterAR.Drone,atoyforaugmentedreality(AR)games.Thetechnicalpropertiesofthedronecanbemisusedforattacks,whichmayrelatesecurityand/orprivacyaspects.Ouraimistosensitiseforthepossibilityofmisusesandthemotivationforrealisationofimprovedsecuritymechanismsofthequadrocopter.Wefocusonprimaryobvioussecurityvulnerabilities(e.g.communicationoverunencryptedWLAN,usageofUDP,lifevideostreamingviaunencryptedWLANtothecontroldevice)ofthisquadrocopter.Wecouldpracticallyverifythatthiscanbemisusedbyunauthorisedpersonsforseveralattacks,e.g.theeavesdroppingofAR.Drone’sunprotectedvideostreams,high-jackingthedrone.Amongstotheraspects,ourcurrentresearchfocusesontherealisationoftheattackoftrackingpersonsandobjectswiththedrone.Besidestherealisationofattacks,wewanttoevaluatethepotentialofthisparticulardronefora“safe-landing”function.Additionally,infutureweplananautomatictrackingofpersonsorobjectswithouttheneedofhumaninteractions.

8301-16, Session 4

Detection of unknown targets from aerial camera and extraction of simple object fingerprints for the purpose of target reacquisitionT.N.Mundhenk,K.Ni,Y.Chen,K.Kim,Y.Owechko,HRLLabs.,LLC(UnitedStates)

Anaerialmultiplecameratrackingparadigmneedstonotonlyspotunknowntargetsandtrackthem,butalsoneedstoknowhowtohandletargetreacquisitionaswellastargethandofftoothercamerasintheoperatingtheater.Herewediscusssuchasystemwhichisdesignedtospotunknowntargets,trackthem,segmenttheuseful

featuresandthencreateasignaturefingerprintfortheobjectsothatitcanbereacquiredorhandedofftoanothercamera.Thetrackingsystemspotsunknownobjectsbysubtractingbackgroundmotionfromobservedmotionallowingittofindtargetsinmotion,evenifthecameraplatformitselfismoving.TheareaofmotionisthenmatchedtosegmentedregionsreturnedbytheEDISONmeanshiftsegmentationtool.Wholesegmentswhichhavecommonmotionandwhicharecontiguoustoeachotheraregroupedintoamasterobject.Oncemasterobjectsareformed,wehaveatightboundonwhichtoextractfeaturesforthepurposeofformingafingerprint.Invideoswhichcontainmultiplevehicleocclusionsandvehiclesofhighlysimilarappearanceweobtainareacquisitionrateforautomobilesofover80%usingthesimplesingleGaussianmodelcomparedwiththenullhypothesisof25%.

8301-18, Session 4

Superresolution terrain map enhancement for navigation based on satellite imageryJ.Straub,JacksonvilleStateUniv.(UnitedStates)

Theutilityofsatelliteimageryformanyapplicationsislimitedbyitsresolution.Resolutioncanbeimprovedviaenhancingthesatelliteimaginghardware;however,softwareenhancementcanbeusedtoovercomehardwaretechnicalandcostlimitations.Superresolutionisasetofimageenhancementtechniqueswhichproduceahigherresolutionimagefromoneormorelower-resolutionones.Conventionalsuperresolutiontechniquesaredesignedtoworkonimagery.However,somesuperresolutiontechniquescanbeapplieddirectlytonon-visualdata.

Thispaperlooksattheenhancementofdigitalelevationmodel(DEM)terrainheightdatausingadatabasebasedinferencetechnique.NASA/NGIASRTMdatacollectedfromtheSpaceborneImagingRadar-C(SIR-C)wasusedtotraintheengineandalternateSRTMdatawaspresentedforenhancement.Theapplicationofsuperresolutiontoaircraft(specificallyUAV)navigationpath-planningwasconsidered.Minimumsafedistanceconstraintsweredefinedandplanningperformedusingthesuper-resolveddatawascomparedtotheup-scaledlow-resolutiondata.Pathsthatwerecreatedwereevaluatedbasedonrouteoptimalityandconstraint-violation-avoidance.

8301-19, Session 5

3D positional control of magnetic levitation system using adaptive control: improvement of positioning control in horizontal planeT.Nishino,N.Kato,MieUniv.(Japan);N.Tsuda,WakayamaNationalCollegeofTechnology(Japan);Y.Nomura,MieUniv.(Japan)

Theobjectiveofthispaperistoestablishatechniquethatlevitatesandconveysahand,akindofmicro-robot,byapplyingmagneticforces:thehandisassumedtohaveafunctionofholdinganddetachingtheobjects.

Theequipmenttobeusedinourexperimentsconsistsoffourpole-piecesofelectromagnets,andisexpectedtoworkasa4DOFdriveunitwithinsomerestrictedrangeof3Dspace:thethreeDOFarecorrespondingto3DpositionalcontrolandtheremainingoneDOF,rotationaloscillationdampingcontrol.Havingusedthesameequipment,Khameseeetal.hadmanipulatedtheimpressedvoltagesonthefourelectricmagneticsbyaPIDcontrollerbytheuseofthefeedbacksignalofthehand’s3Dposition,thecontrolledvariable.However,inthissystem,thereweresomeproblemsremaining:inthehorizontaldirection,whentranslatingthehandoutofrestrictedregion,positionalcontrolperformancewassuddenlydegraded.

Theauthorsproposeamethodtoapplyanadaptivecontroltothehorizontaldirectionalcontrol.Itisexpectedthatthetechniquetobepresentedinthispapercontributesnotonlytotheimprovementoftheresponsecharacteristicbutalsotowideningtheapplicablerangeinthehorizontaldirectionalcontrol.

Conference 8301


8301-20, Session 5

Robots in agriculture: an I-weed robot for a specific sprayingG.Salis,C.Gée,S.Villette,J.Paoli,G.Jones,AgroSupDijon(France)

Consideringthespatialvariabilityofafield,itconsistsinadaptingculturesandtreatmentstoeachareaofthefieldratherthantoapplyanestablishedmodelbasedonaverageinformationovertheentirefield.Thedevelopmentofnewtechnologies(i.e.remotesensing,imageprocessing,computerscience,robotics),wasanimportantsteptowardssustainableagriculture.Intheearly1980s,firstrobotsweredevelopedinlabsandtheywerededicatedtoparticulartask:harvesting,planting,spraying...

Currently,wearedevelopinganI-Weedrobot(IntelligentWeedRobot)thatisachemicalweedingrobot.ItisguidedbyahighprecisionGPSsignal(RealTimeKinetic)withaprecisionofabout2or3centimeters.Ourrobotisabletomovebetweenthecrop’srowsthankstoaroutepreviouslyrecorded.Acamerapositionedinthefrontofourdeviceisequippedwithanearinfraredfilter.Itallowstoidentifyandtolocatethepositionofweeds.Atthebackoftherobot,asprayboomsystemtriggersattherighttimetherightnozzleinordertosprayonlyonweedplants.

8301-21, Session 5

The magic glove: a gesture-based remote controller for intelligent mobile robotsC.Luo,Y.Chen,M.Krishnan,M.Paulik,Univ.ofDetroitMercy(UnitedStates)

Thispaperdescribesthedesignofagesture-basedHumanRobotInterface(HRI)foranautonomousmobilerobotenteredintheIntelligent2010GroundVehicleCompetition(IGVC).WhiletherobotismeanttooperateautonomouslyinthevariousChallengesofthecompetition,anHRIisusefulinmovingtherobottothestartingposition.

Inthisprojectagesture-basedembeddedsystemcalledtheMagicGloveisdevelopedforremotecontrolofarobot.Thesystemconsistsofamicrocontrollerandsensorsthatiswornbytheoperatorasagloveandiscapableofrecognizinghandsignals.Thesearethentransmittedthroughwirelesscommunicationtotherobot.ThedesignoftheMagicGloveincludedcontributionsontwofronts:hardwareconfigurationandalgorithmdevelopment.Atripleaxisaccelerometerusedtodetecthandorientationpassestheinformationtoamicrocontroller,whichinterpretsthecorrespondingvehiclecontrolcommand.ABluetoothdeviceinterfacedtothemicrocontrollerthentransmitstheinformationtothevehicle,whichactsaccordingly.

TheMagicGlovewassuccessfullydemonstratedfirstinaPlayer/Stagesimulationenvironment.Thegesture-basedfunctionalitywasthenalsosuccessfullyverifiedonanactualrobotanddemonstratedtojudgesatthe2010IGVC.

8301-22, Session 5

Way-point navigation for a skid steer vehicle in unknown environmentsP.Chen,A.Das,P.Mukherjee,S.Waslander,Univ.ofWaterloo(Canada)

Thispaperproposesacombinedmapping,pathplanning,andcontrolsolutionthatwillallowaskid-steerUGVtonavigatesafelytoadesiredlocationinanunknownenvironment.AnExtendedKalmanFilter(EKF)isemployedusinganon-linearmodeloftheskidsteervehiclewhichassumesthevehicleoperatesontheconditionofconstantslippage.Theproposedsolutiongeneratesa3DpointcloudbynoddingtheplanarLIDAR,andthenusesaninnovativealgorithmtoconstructa2Dmapofthetraversablespace.Themapdataisefficientlystoredinmemoryusinganoctreestructurewithreal-timepruning.Usingthis2D

map,theproposedpathplannerwillgeneratedynamicallyfeasibleandcollision-freepathstothedesiredlocation,andwillre-planthesepathsasnecessarywhennewobstaclesaredetected.Thepathplannerisbasedontheprobabilisticroadmap(PRM)algorithmusingadynamicA*(D*)search,andconsidersthekinodynamicconstraintsofthevehicle.Finally,anon-lineartrackingcontrollerismodifiedfortheskid-steerUGVtoallowthevehicletotrackthepath.Allofthealgorithmsarecomputationallyefficientandallowforonlineprocessingon-boardtherobot,andweresuccessfullyimplementedonacustommadeskidsteervehicle.

8301-23, Session 6

Integrated field testing of planetary robotics vision processing: the PRoVisG campaign in Tenerife 2011G.M.Paar,JOANNEUMRESEARCHForschungsgesellschaftmbH(Austria);L.Waugh,EADSAstriumLtd.(UnitedKingdom);D.P.Barnes,AberystwythUniv.(UnitedKingdom);T.Pajdla,CzechTechnicalUniv.inPrague(CzechRepublic);M.Woods,SciSysLtd.(UnitedKingdom);H.Graf,Ctr.Suissed’ElectroniqueetdeMicrotechniqueSA(Switzerland);Y.Gao,Univ.ofSurrey(UnitedKingdom);K.Willner,TechnischeUniv.Berlin(Germany);J.A.Muller,Univ.CollegeLondon(UnitedKingdom);R.Li,TheOhioStateUniv.(UnitedStates);M.Maurette,Ctr.Nationald’ÉtudesSpatiales(France)

Inordertomaximizetheuseofaroboticprobeduringitslimitedlifetime,scientistsimmediatelyhavetobeprovidedthebestachievablevisualqualityof3Ddataproducts.TheEUFP7-SPACEProjectPRoVisG(2008-2012)developstechnologyfortherapidprocessingandeffectiverepresentationofvisualdatabyimprovinggroundprocessingfacilities.Itsambitionistocollectatoolsetandintegrateaversatileandflexibleprocessingchaineasilytobeadaptedtovarioustasks.

PRoVisGisholdingafieldtestinSeptember2011ontheCalderaofTenerifetoverifytheimplemented3DVisionprocessingmechanismsandtocollectvarioussetsofreferencedatainrepresentativeenvironment.

Thepapercoversthepreparationworkforsuchatestandhighlightsthetestelementsthatincludestandardoperations-andscience-relatedcomponentsbutalsodatacapturetoverifyspecificprocessingfunctions.

ThecampaignissupportedbytheAstriumUKRoverBridgetasarepresentativeplatformwhichallowssimultaneouson-boardmountingandpoweringofvariousvisionsensorssuchastheExoMarsPanCamEmulatorbyAberystwythUniversity(AUPE).

Thefinalmanuscriptwillcontainadescriptionofthecaptureddataandtheprocessingresults,aswellasasummaryoftheexperiencegatheredduringthetest.

8301-24, Session 6

Hierarchical loop detection for mobile outdoor robotsD.Lang,C.Winkens,M.Häselich,D.W.Paulus,Univ.Koblenz-Landau(Germany)

Loopclosingisafundamentalpartof3Dsimultaneouslocalizationandmapping(SLAM)thatcangreatlyenhancethequalityoflong-termmapping.Itisessentialforthecreationofgloballyconsistentmaps.

Conceptually,loopclosingisdividedintodetectionandoptimization.

Recentapproachesdependonasinglesensortorecognizepreviouslyvisitedplacesintheloopdetectionstage.Inthisstudy,wecombinedataofmultiplesensorssuchasGPS,vision,andlaserrangedatatoenhancedetectionresultsinrepetitivelychangingenvironmentsthatarenotsufficientlyexplainedbyasinglesensor.Wepresentafastandrobusthierarchicalloopdetectionalgorithmforoutdoorrobotstoachieveareliableenvironmentrepresentationevenifoneormoresensorsfail.

Conference 8301


8301-25, Session 6

A novel margin-based, linear embedding technique for visual object recognitionF.Dornaika,Univ.delPaísVasco(Spain)

LinearDimensionalityReduction(LDR)techniqueshavebeenincreasinglyimportantincomputervisionandpatternrecognitionsincetheypermitarelativelysimplemappingofdataontoalowerdimensionalsubspace,leadingtosimpleandcomputationallyefficientclassificationstrategies.Recently,alinearmethodcalledAverageNeighborsMarginMaximization(ANMM)wasproposedandshowntohavepowerfuldiscriminationproperties.Althoughgoodresultswereobtainedwiththismethod,itsuffersfromtwoshortcomings:i)itrequiresthesettingoftwoparameters-theneighborssizesforhomogeneousandheterogeneoussamples,andii)itcanbeverysensitivetothelabeloutlierssinceamarginaverageisused.Inthispaper,weproposeanovelmarginmaximizationthatisbasedonthenearesthitandthenearestmisssamplesonly.Theproposedapproachfindstheprojectiondirectionssuchthatthesumoflocalmarginsismaximized.Ourproposedapproachhasbeenappliedtotheproblemofappearance-basedfacerecognition.ExperimentalresultsperformedonthreefacedatabasesshowthattheproposedapproachcangivebettergeneralizationperformancethantheANMMmethod.OurproposedmethodismuchlesssensitivetooutliersthantheANMMmethod.Theproposedapproachcouldalsobeappliedtoothercategoryofobjectscharacterizedbylargevariationsintheirappearance.

8301-26, Session 6

Real-time, two-level foreground detection and person-silhouette extraction enhanced by body-parts trackingR.Deeb,É.Desserée,S.Bouakaz,Univ.ClaudeBernardLyon1(France)

Weproposeareal-time,two-levelforegrounddetection,enhancedbybodypartstracking,designedtoefficientlyextractpersonsilhouetteformonocularvideo-basedhumanmotionanalysissystems.Weaimtofindsolutionsfornon-controlledenvironmentchallenges,suchaslightchanges,camouflageandocclusions,whichmakethedetectionandthetrackingofamovingpersonahardtasktoaccomplish.Onthefirstlevel,weproposeanenhancedMixtureofGaussians,builtonbothchrominance-luminanceandchrominance-onlyspaces,whichhandlesglobalilluminationchanges.Onthesecondlevel,weimprovesegmentationresults,ininterestingareas,byusingstatisticalforegroundmodelsupdatedbyahigh-leveltrackingofbodyparts.Eachbodypartisrepresentedwithasetoftemplatescharacterizedbyafeaturevectorbuiltinaninitializationphase.Then,ahighleveltrackingisachievedbyfindingblob-templatecorrespondencesviadistanceminimizationinfeaturespace.Correspondencesarethenusedtoupdateforegroundstatisticalmodels,thenaMaximumAPosterioriframeworkisusedtorefinesegmentation.Wewereabletoextractarefinedsilhouetteinthepresenceoflightchanges,noiseandcamouflage.Moreover,thetrackingapproachallowedustoinferinformationaboutthepresenceandthelocationofbodypartseveninthecaseofpartialocclusion.

8301-27, Session 6

Activity recognition from video using layered approachC.A.McPherson,J.M.Irvine,M.Young,DraperLab.(UnitedStates);A.Stefanidis,GeorgeMasonUniv.(UnitedStates)

Theadversaryincurrentthreatsituationscannolongerbeidentifiedbywhattheyare,butbywhattheyaredoing.Thishasleadtoalargeincreaseintheuseofvideosurveillancesystemsforsecurityanddefenseapplications.Withthequantityofvideosurveillanceatthedisposaloforganizationsresponsibleforprotectingmilitaryandcivilianlivescomesissuesregardingthestorageandscreeningthedatafor

eventsandactivitiesofinterest.

ActivityrecognitionfromvideoforsuchapplicationsseekstodevelopautomatedscreeningofvideobasedupontherecognitionofactivitiesofinterestratherthanmerelythepresenceofspecificpersonsorvehicleclassesdevelopedfortheColdWarproblemof“FindtheT72Tank”.Thispaperexploresnumerousapproachestoactivityrecognition,allofwhichexamineheuristic,semantic,andsyntacticmethodsbasedupontokensderivedfromthevideo.

Theproposedarchitecturediscussedusesamulti-levelapproachthatdividestheproblemintothreeormoretiersofrecognition,eachemployingdifferenttechniquesaccordingtotheirappropriatenesstostrengthsateachtierusingheuristics,syntacticrecognition,andHMM’softokenstringstoformhigherlevelinterpretations.

8301-28, Session 7

Method for fast detecting the intersection of a plane and a cube in an octree structure to find point sets within a convex regionK.Fujimoto,N.Kimura,T.Moriya,Hitachi,Ltd.(Japan)

Performingefficientviewfrustumcullingisafundamentalproblemincomputergraphics.Ingeneral,anoctreeisusedforviewfrustumculling.Thecullingcheckstheintersectionofeachoctreenode(cube)againsttheplanesoftheviewfrustum.However,thisinvolvesmanycalculations.Weproposeamethodforfastdetectingtheintersectionofaplaneandacubeinanoctreestructure.Whenwecheckwhichchildoftheoctreenodeintersectsaplane,wecomparethecoordinatesofthecornerofthenodeandtheplane.Usinganoctree,wecalculatetheverticesofthechildnodebyusingtheverticesoftheparentnode.Tofindpointswithinaconvexregion,avisibilitytestisperformedbyANDoperationwiththeresultofthreeormoreplanes.Inexperiments,wetestedtheproblemofsearchingforthevisiblepointwithacamera.Themethodwastwotimesfasterthantheconventionalmethod,whichdetectsavisibleoctreenodebyusingtheinnerproductoftheplaneandeachcornerofthenode.

8301-30, Session 7

Lucas-Kanade image registration using camera motionsS.Cho,H.Cho,PohangUniv.ofScienceandTechnology(Korea,Republicof);Y.S.Moon,J.Cho,S.Lee,SamsungElectronicsCo.,Ltd.(Korea,Republicof);S.Lee,PohangUniv.ofScienceandTechnology(Korea,Republicof)

Lucas-Kanadebasedimageregistrationmethodsareoneofthemostwidelyusedalgorithmsincomputervision.Themethodshavebeensuccessfullyusedinnumerousworks,whichincludeimageregistrationasacomponentintheprocess.Inthispaper,weproposeLucas-Kanadebasedimageregistrationmethodsusingcameramotions.Wedecomposeahomographyintocameraintrinsicandextrinsicparameters,andestimateonlyextrinsicparameters,assumingthatintrinsicparametersaregiven,e.g.,fromtheEXIFinformationofaphotograph.Astheintrinsicandextrinsicparametersarebasedontheactualconfigurationofthecamera,theproposedmethodcanperformimageregistrationmorereliably.Inaddition,asthenumberofextrinsicparametersissmallerthanthenumberofhomographyelements,theproposedmethodrunsfasterthanthepreviousLucas-Kanadebasedregistrationmethodthatestimatesahomographyitself.

8301-31, Session 7

Object tracking with adaptive HOG detector and adaptive Rao-Blackwellised particle filterS.Rosa,M.Paleari,P.Ariano,IstitutoItalianodiTecnologia(Italy);B.Bona,PolitecnicodiTorino(Italy)

Conference 8301


ScenariosforamannedmissiontotheMoonorMarscallforastronautteamstobeaccompaniedbysemi-autonomousrobots.Aprerequisiteforhuman-robotinteractionisthecapabilityofsuccessfullytrackinghumansandobjectsintheenvironment.

Inthispaperwepresentasystemforreal-timevisualobjecttrackingin2Dimagesformobileroboticsystems.Theproposedalgorithmisabletospecializetoindividualobjectsandtoadapttosubstantialchangesinilluminationandobjectappearanceduringtracking.

Thealgorithmiscomposedbytwomainblocks:adetectorbasedonHistogramofOrientedGradient(HOG)descriptorsandlinearSupportVectorMachines(SVM),andatrackerwhichisimplementedbyanadaptiveRao-Blackwellisedparticlefilter(RBPF).TheSVMisre-trainedonlineonnewsamplestakenfrompreviouspredictedpositions.Weusetheeffectivesamplesizetodecidewhentheclassifierneedstobere-trained.

Positionhypothesesforthetrackedobjectaretheresultofaclusteringprocedureappliedonthesetofparticles.

Thealgorithmhasbeentestedonchallengingvideosequencespresentingstrongchangesinobjectappearance,illumination,andocclusion.Experimentaltestsshowthatthepresentedmethodisabletoachievenearreal-timeperformanceswithagoodprecisiononstandardvideosequences.

8301-32, Session 7

A modular real-time vision system for humanoid robotsA.L.Trifan,A.J.Neves,B.Cunha,N.Lau,Univ.deAveiro(Portugal)

Forbothhumansandrobotsvisionisaveryimportantsensethathasthetaskofinterpretingspatialdata,indexedbymorethanonedimension.Forahumanoidrobot,arobustvisionsystemshouldbeabletoprovideaccurateinformationabouttheenvironmentandaprecisedescriptionoftheobjectsofinterest.Wepresentanimplementationofavisionsystemforahumanoidrobotdesignedtoperformincolor-codedenvironments.Fromacquiringimages,processingthemanddetectingtheobjectsofinterest,allthealgorithmshavebeentestedontheNAOsoccerplayinghumanoidrobotaswellasonaBioloidrobotdesignedtonavigatethroughamazewiththehelpofcolor-codedclues.Fortheserobotstheworldissimplifiedtoanumberofcolorsthataremeaningfulinthementionedcontexts.Thevisionsystemthatweproposecanperforminrealtimeandithasprovenitspracticalefficiency.Moreover,wepresentaninnovativealgorithmforself-calibrationofthemostimportantintrinsicparametersofthecamera,aswellastwoexternalapplicationsdevelopedfordebuggingandcolorcalibration.Oneofthemostimportantfeaturesofthevisionsystemthatweproposeisitsmodularity,whichallowsitsusewithawiderangeofroboticplatforms.

8301-33, Session 8

Radial polar histogram approach to obstacle avoidance and path planning for robotic cognition and motion controlP.Wang,N.R.Keyawa,C.Euler,C.T.Lin,CaliforniaStateUniv.,Northridge(UnitedStates)

Themostfundamentalelementofautonomousmobileroboticsistherobot’sabilitytodetectandavoidobstacleswhiletravelingtodesiredlocations.Sincegroundvehiclesfrequentlymaneuverincurvilinearmotions,anobstacleavoidancealgorithmthatcanprovideadesiredinstantaneousradiusandvelocityresultsinmoreaccuratemotionexecution.Thistypeofobstacleavoidancealgorithm,whichhasbeenimplementedinCaliforniaStateUniversityNorthridge’sIntelligentGroundVehicle(IGV),isknownasRadialPolarHistogram(RPH).RPHutilizesrawdatainapolarhistogramthatisreadfromalaserrangefinderandacamera.Thisdataisfilteredforobstaclegroupingandedgedetection.Aconcavityfunctionisutilizedtoeliminatepathsthatmaycausetherobottocollidewithobstacles.Withintheconcavity

function,severalopenblocksaredefinedandacostfunctionisutilizedtodetermineadesiredheading.Thisheading,alongwithconcavityandnavigationalwaypoints,determinesthebestopenblock.Theleftandrightradialedgesoftheopenblockaredeterminedandputintoastepbystepdebugfunctiontooutputarangeofpossibleradialpaths.Theradialpathoptimization,velocity,andaccelerationfunctionsareimplementedtodetermineadesiredradiusandvelocity.

8301-34, Session 8

Optimising a mobile robot control system with GPU accelerationN.Tuck,M.E.McGuinness,F.Martin,Univ.ofMassachusettsLowell(UnitedStates)

TheInternationalGroundVehicleCompetition(IGVC)AutonomousChallengerequiresacontrolprogramthatperformsanumberofdifferentcomputationallyintensivetasks.Ourinitialcontrolprogramdesigncannotruncomfortablyinrealtimeonastandardmulti-coreprocessorinourbudgetedpowerenvelope.ThispaperdescribestheprocessandresultsofprofilingthiscontrolprogramandportingappropriatecomputationstorunacceleratedonanAMDRadeongraphicsprocessor.InordertoGPU-acceleratesomeofthesecomputations,weusedtheBaconprogrammingsystem,atooldevelopedbyoneoftheauthorsforbuildingcomputekernelstorunonOpenCL-compatiblegraphicsprocessors.OpenCL-basedGPUaccelerationcanprovideanorderofmagnitudespeedupovertraditionallyoptimisedcode.ComparedtousingOpenCLcodedirectly,Baconissignificantlyeasiertouseandcanprovideasmuchasafactoroftwoadditionalspeedup.ThispaperdescribestheacceleratedimplementationofthecontrolsoftwarebuiltontheRobotOperatingSystem(ROS)runningintheGazebo3Dsimulator.Speedupsand,insomecases,qualityimprovementsofthevisionprocessingaredemonstratedcomparedtotheroutinesavailableinOpenCV.TheperformanceimprovementsofthecompletesystemonoursimulationoftheIGVCAutonomousChallengearedescribed.

8301-35, Session 8

Design and realization of an intelligent ground vehicle with modular payloadsM.A.Akmanalp,R.M.Doherty,J.Gorges,P.Kalauskas,E.Peterson,F.Polido,S.S.Nestinger,T.Padir,WorcesterPolytechnicInstitute(UnitedStates)

InJune2011,WorcesterPolytechnicInstitute’s(WPI)unmannedgroundvehicleparticipatedinthe8thAnnualRoboticLawnmowerand19thAnnualIntelligentGroundVehicleCompetitionsback-to-back.Thispaperdetailsthetwo-yeardesignanddevelopmentcycleforWPI’sintelligentgroundvehicle,Prometheus.Theon-boardintelligencealgorithmsincludelanedetection,obstacleavoidance,pathplanning,worldrepresentationandwaypointnavigation.Theauthorspresentexperimentalresultsanddiscusspracticalimplementationsoftheintelligencealgorithmsusedontherobot.

8301-36, Session 8

Navigating a path delineated by colored flags: an approach for an IGVC 2011 requirementA.Szmatula,M.Parrish,M.Krishnan,M.Paulik,U.Mohammad,C.Luo,Univ.ofDetroitMercy(UnitedStates)

AnewrequirementfortheAutonomousChallengeportionofthe2011IntelligentGroundVehicleCompetition(IGVC)istonavigatebetweenredandgreencoloredflagsthatformalanewithinalanewithinthecourse.Thisadditionmakesforaninterestingchallengeinthatitforcesteamstoincorporatecolor-basedimagecharacteristicextractioncombinedwithpossibleLADARdetectionofaverysmallfeaturein

Conference 8301


thescene.Furthermore,sincetheflagsaresmallandflexible,andthussusceptibletothewind,theypresentavariablecrosssectiontothecamerasystemmakingfalsepositivesandfalsenegativeidentificationscommon.Thusinadditiontotheimageprocessingchallengeassociatedwithidentifyingandclassifyingflags,theteamsmustaddressnavigationalgorithmupdatesnecessarytoprovidesmoothandreliablerobotmotioninthepresenceofhigherobstacleuncertainty.ThisworkthendiscussesanalgorithmfordetectionandlocalizationofcoloredflagsfollowedbymodificationofaVectorFieldHistogram(VFH)algorithmtonavigatetherobotappropriately.Resultsfromsimulationandfieldteststudieswillbepresentedandperformancediscussed.

8301-37, Session 8

Navigating with VFH: a strategy to avoid trapsC.Luo,M.Krishnan,M.Paulik,U.Mohammad,Univ.ofDetroitMercy(UnitedStates)

TheIGVCNavigationChallengecourseconfigurationhasevolvedincomplexitytoapointwhereuseofasimplereactivelocalnavigationalgorithmpresentsproblemsincoursecompletion.Acommonlyusedlocalnavigationalgorithm,theVectorFieldHistogram(VFH),isrelativelyfastandthussuitablewhencomputationalcapabilitiesonarobotarelimited.Oneoftheattendantdisadvantagesofthisalgorithmisthatarobotcangettrappedwhenattemptingtogetpastaconcaveobstaclestructure.TheNavigationChallengecoursenowhasseveralsuchstructures,includingsomethatpartiallysurroundwaypoints.ElaborateheuristicsareneededtomakeVFHviableinsuchasituationandtheirtuningisarduous.

AnalternateapproachthatavoidstheuseofheuristicsistocombineadynamicpathplanningalgorithmwithVFH.Inthiswork,theD*LitepathplanningalgorithmisusedtoprovideVFHwithintermediategoals,whichthelatterthenusesassteppingstonestoitsfinaldestination.Resultsfromsimulationstudiesaswellasfielddeploymentareusedtoillustratethebenefitsofusingthelocalnavigatorinconjunctionwithapathplanner.


Measurement of noises and modulation transfer function of cameras used in optical-digital correlatorsN.N.Evtikhiev,S.N.Starikov,P.A.Cheryomkhin,V.V.Krasnov,NationalResearchNuclearUniv.MEPhI(RussianFederation)

Hybridoptical-digitalsystemsbasedondiffractivecorrelatorarebeingactivelydeveloped.Suchsystemscanbeusedforpatternrecognition,imageencryption,imagingwithaberrationscorrectionandextendingdepthoffiled.Forwiderpropagationofthiskindofsystems,thepossibilityofusinginexpensivephotoandvideocamerasisimportant.Tocorrectlyestimatecapabilitiesofapplicationofcamerasofdifferenttypesinoptical-digitalcorrelationsystemstheknowledgeofmodulationtransferfunction(MTF)andlightdependedtemporalandspatialnoisesisrequired.

Themethodformeasurementof2DMTFispresented.Themethodbasedonrandomtargetmethodbutinsteadofarandomtargetthespeciallycreatedtargetwithflatpowerspectrumisused.ItallowstomeasureMTFwithoutaveraging1DFourierspectraoverrowsorcolumnsasisintherandomtargetmethodandtoachieveallvaluesof2DMTFinsteadofjusttwoorthogonalcross-sections.

Thesimplemethodformeasuringthedependenceofcameratemporalnoiseonlightsignalvaluebyshootingasinglesceneisdescribed.Lightspatialnoiseischaracterizedbyphoto-responsenon-uniformity(PRNU)ofphotosensor.Procedureforobtainingcamera’slightspatialnoiseportrait(arrayofPRNUvaluesforallphotosensorpixels)ispresented.

ResultsonmeasurementsofMTFandlightdependedtemporalandspatialnoisesforconsumerphotocamera,machinevisioncameraandvideo-surveillancecameraarepresented.


A phase-space approach for detection and removal of rain in videoV.Santhaseelan,K.V.Asari,Univ.ofDayton(UnitedStates)

Nowadays,thewidespreaduseofcomputervisionalgorithmsinsurveillancesystemsandautonomousrobotshasincreasedthedemandforvideoenhancementalgorithms.Inthispaper,weproposeanalgorithmbasedonphasecongruencyfeaturestodetectandremoverainandthusimprovethequalityofvideo.Wemakeuseofthefollowingcharacteristicsofrainstreaksinvideoinordertodetectthem:(1)rainstreaksdonotoccludethesceneatallinstances,(2)alltherainstreaksinaframeareorientedinasingledirection,and(3)presenceofrainstreakataparticularpixelcausesapositivechangeinintensity.Combiningallthesepropertiesweareabletodetectrainstreaksinaparticularframeusingphasecongruencyfeatures.Thepixelsinaframewhichareidentifiedasrainstreaksarethenreplacedusingthepixelinformationofitsspatialandtemporalneighborswhicharenotaffectedbyrain.Usingthismethodweareabletoremoverainofmediumdensityfromvideosevenwhencomplexcameramovementisinvolved.Wealsoproposeanadaptivecompensationmechanismforscenerestorationwherethenumberofframesrequiredforcompensationchangesautomaticallybasedonthedensityofrain.


Intelligence algorithms for autonomous navigation in a ground vehicleS.J.Petkovsek,A.Norton,TrinityCollege(UnitedStates)

“Q”isTrinityCollegeRobotStudyTeam’s(TCRST)entryintheIntelligentGroundVehicleCompetition(IGVC)heldatOaklandUniversityinAuburnHills,Michigan.Trinityhasparticipatedinthecompetitionforelevenyears,whileQisinitssixthiteration.IGVChasfourareasinwhichtocompete,includingtheDesignchallenge,theNavigationchallenge,theAutonomouschallenge,andtheJAUSInteroperabilitychallenge.Ineachofthesechallenges,exceptfortheDesignchallengewherethesystemitselfisjudged,Qmustbeabletoautonomouslynavigateanoutdoorenvironment.Tosuccessfullynavigatethecourse,Qmustavoidpaintedlines,traverserampsandsandpits,avoidobstacles,andreachwaypointsasquicklyaspossible,aseachrunistimed.FortheJAUSchallenge,Qmustobeycommandsfromthejudge’sserverusingJointArchitectureforUnmannedSystemsprotocol.ThispaperwilloutlineQ’sintelligencealgorithmsandimageprocessingtechniquesforcompletingthecourse.ThemethodsQusesforpathplanningandpathhistorywillbediscussed,asspecialconsiderationsaretakentonavigatesuccessfullythroughdenseobstacleregionswherefailuressuchasgettingstuckinacornerorturningbackthewrongdirectiononthecoursearemorelikely.ThemannerinwhichQdistinguishesimpassablewhitelinesfrompassable,brightsurfacessuchasapaintedramporsandpitwillalsobeexamined.Last,theprocessbywhichQnavigatestoGPSwaypointswillbeexplained.


Hierarchical, multi-level image mosaicing for autonomous navigation of UAVS.Park,D.Ghosh,N.Kaabouch,R.Fevig,W.Semke,TheUniv.ofNorthDakota(UnitedStates)

Anovelalgorithmforhierarchicalmulti-levelimagemosaicingforautonomousnavigationofUAVisproposed.Themaincontributionoftheproposedsystemistheminimizationoftheerroraccumulationpropagatedalongtheframes,byincrementallybuildingalong-durationmosaicontheflywhichishierarchicallycomposedofshort-durationmosaics.Theproposedalgorithmfulfillsthereal-timeprocessingrequirementsinautonomousnavigationasfollows.1)Causality:thecurrentoutputofthemosaicingsystemdependsonlyonthecurrentand/orpreviousinputframes,contrarytoexistingofflinemosaic

Conference 8301


algorithmsthatdependonfutureinputframesaswell.2)Learnability:thealgorithmautonomouslyanalyzes/learnsthescenecharacteristicsandclassifiesthemintothe‘manmadestructuralarea’typevs.the‘naturalvegetationfield’typebytheBayesianlearningandinference.3)Adaptability:thesystemautomaticallyadaptsitselftothescenechangeandchoosesthepropermethodsforfeatureselection(i.e.,thefastbutunreliableLKTvs.theslowbutrobustSIFT)andgeometricimagetransform(i.e.,therestrictiveaffinetransformvs.themorerelaxedperspectivetransform.)

Theevaluationofouralgorithmwiththeextensivefieldtestdatainvolvingseveralthousandairborneimagesshowsthesignificantimprovementinprocessingtime,robustnessandaccuracyoftheproposedalgorithm.


A diffraction-limited 10-mm-aperture adaptive lensR.Batchko,HolochipCorp.(UnitedStates)

Wepresentanelectrically-actuatedadaptivefluidiclenshavinga10-mmclearapertureand6-diopterrangeinfocalpower.Thelensemploysbiconvexmembranesencasingafluid.Apiezoelectricactuatorprovidesa150-to-1000-mmfocallengthrangewhiledrawinglessthan1mW.Thelenshousinghasanouterdiameterof1-inchandlenscenter-thicknessislessthan1mm.PolychromaticMTFexceeds90%ofthediffractionlimitoverthefull10-mmaperture,outperformingcommercialsphericalglasslensesovermuchofitsdioptricrange.

Conference 8301


Conference 8302: Imaging and Printing in a Web 2.0 World IIIWednesday25January2012

Part of Proceedings of SPIE Vol. 8302 Imaging and Printing in a Web 2.0 World III

8302-01, Session 1

Organizing visual moments for sharing and reflections: VisRR.C.Jain,Univ.ofCalifornia,Irvine(UnitedStates)

Inthispresentationwewilldiscussconceptsandarchitectureofthesystemthatwearebuildingtowardsorganizingallpersonalmediaforsharingandreflections.Ourevent-basedorganizationofdataenablesaccesstolargevolumesofmediadatathatisbecomingcommon.Wewillalsodemonstrateourworkingsystem.

8302-02, Session 1

The role of digital presses, Web 2.0, and mobile in enabling the internet of thingsT.F.Rodriguez,DigimarcCorp.(UnitedStates)

Noabstractavailable

8302-03, Session 2

Mobile image processing for fashion marketplaceM.Chandra,G.Golwala,C.Pungaliya,Poshmark,Inc.(UnitedStates)


8302-04, Session 2

Mobile capture: the end of the photocopier?M.J.Gormish,RicohInnovations,Inc.(UnitedStates)

Weinvestigatedmobilephoneusageforimagecaptureintheworkplaceviasurverysandinterviews.Oursurveysshowsmartmobilephonesarebeingusedincreasinglyforimagecaptureandothernon-phonecallactivities.Wefocusoninformationcapturewithmobiledevicesandtheimageprocessingnecessarytomakethosecameraimagesasusableasimagesfromflatbedscanners.Algorithmstocleanupwhiteboardsandpaperdocumentscansaveprintingcostsandproperconnectorscanreducetheneedtoprintentirely.Inadditionwecaptureinformationnotavailabletoscannersandmakethatinformationusefulforbusinesspurposesandforprinting.RicohhasreleasedmultiplemobilephoneapplicationsandannouncedaneWritertabletallofwhichchangeimagingandprintingviacloudservices.Someofthesewillbedemonstrated.

8302-05, Session 2

From scenes to screens: connected digital imagingZ.Gillat,Eye-Fi(UnitedStates)


8302-06, Session 3

Text documents as social networksH.Balinsky,Hewlett-PackardLabs.(UnitedKingdom);A.Balinsky,CardiffUniv.(UnitedKingdom);S.J.Simske,Hewlett-PackardCo.(UnitedStates)

Theextractionofkeywordsandfeaturesisafundamentalproblemintextdatamining.Documentprocessingapplicationsdirectlydependonthequalityandspeedoftheidentificationofsalienttermsandphrases.Applicationsasdisparateasautomaticdocumentclassification,informationvisualization,filteringandsecuritypolicyenforcementallrelyonthequalityofautomaticallyextractedkeywords.

Recently,anovelapproachtorapidchangedetectionindatastreamsanddocumentshasbeendeveloped.ItisbasedonideasfromimageprocessingandinparticularontheHelmholtzPrinciplefromtheGestaltTheoryofhumanperception.Bymodellingadocumentasaone-parameterfamilyofgraphswithitssentencesorparagraphsdefiningthevertexsetandwithedgesdefinedbyHelmholtz’sprinciple,wedemonstratedthatforsomerangeoftheparameters,theresultinggraphbecomesasmall-worldnetwork.

Inthisarticleweinvestigatethenaturalorientationofedgesinsuchsmallworldnetworks.Fortwoconnectedsentences,wecansaywhichoneisthefirstandwhichoneisthesecond,accordingtotheirpositioninadocument.ThiswillmakesuchagraphlooklikesmallWWW-typenetworkandPageRanktypealgorithmswillproduceinterestingrankingofnodesinsuchadocument.

8302-08, Session 3

HP2.ME URL shorten serviceH.Zhang,W.X.Wu,Y.Zhang,C.Zheng,Hewlett-PackardChinaCo.,Ltd.(China);Q.Lin,J.Liu,Hewlett-PackardLabs.(UnitedStates)

HP2.MEisanurlshortenservicewhichprovidebyHP.DifferentthanotherexistedURLshortenservices,HP2.menotonlyprovidesURLshortenbutalsohelpstoextractvaluablecontentsfromanykindwebpageandreturnthembackintherightformatwhichcanbeproperlyviewindifferentmobiledevicesorPC.ThroughbrowserenginewebkitandDomtreeanalysis,wegettheoriginalHTMLpagefromURLandanalysistheDOMelements‘geometricinfotofigureoutthevaluablecontentsinthepage.Relayouttheextractcontenttofittherequestdevicescreens.Inthispaper,wepresenttheprimarymethodsandalgorithmtocalculatethemaincontentinwebpageandhowwelayouttheextractcontentstofitthedifferentscreensize.

8302-09, Session 3

HP SmartprintH.Zhang,Z.Liu,Y.Yuan,G.J.Liao,Hewlett-PackardChinaCo.,Ltd.(China);Q.Lin,J.Liu,Hewlett-PackardLabs.(UnitedStates)

InHPnewwebbrowserplug-inprinttool,SmartPrint,weapplynovelmethodstoaddresscomplexwebprintproblems.ThroughDOMelementgeometriccalculation,Smartprintcanautoextractvaluecontentfromanykindofwebpages.Forkeepingtheoutputlayoutinagoodway,weapplytheoriginalCSStohelprelayoutthecontentelementswhichhavebeenmarkedasvaluablecontent.Otherthanthem,wealsoallowusertomanualeasilycroporremoveanycontentstheywantfromtheoriginalwebpagesthroughdraganddrop.Duringthedraggingordropping,usercandirectlyseewhatcontentwillbecoverorremovedthroughrealtimeaffinitycalculation.HPSmartprint


hasbeenshippedasaformalproductofHPwhichcanbedownloadedfromHPwebprintwebpage.

8302-10, Session 4

Kind of images in printed photo booksR.Fageth,P.Schuetz,T.Wagner,CeWeColorAG&Co.OHG(Germany)

Preservingmemoriescombinedwithstorytellingusingphotobooksaresubstitutingmoreandmoreclassical4*6printsandsilverhalideposters.Digitalprintingviaelectrophotographyandinkjetisalsosubstitutingmoreandmoreclassicalsilverhalideprintingtechnologyasthedominantproductiontechnologyforthesekindsofproducts.Butthereisstillaremarkablenumberofpeoplewhowantthephotoqualityinphotobooksonrealphotopaper(AgX)oratleastordertheirphotobookinaglossyversionprintedviaelectrophotographyprinters.

Inthispaperweshowtheusageofthesethreedifferentoutputpossibilitiesasfunctionoftheeventdocumentedintheprintedproductaswellastheorderingbehaviorforclassicalsilverhalideprintsasfunctionoftheevents.

Thispaperalsoanalysestheusageofimagestakenbydigitalstillcamerasaswellaswithmobilephonesinprintedphotobooks.WeevaluatethenumberofcamerasusedinphotobooksaswellascombinationsofSLRs,pointandshootandmobilephonesinallofitspossiblecombinations.Wealsotrytoanalyzethesecombinationswiththethemesofthephotobooks,e.g.inbookstellingthestoryaboutabirthdayonefindsmoreimagesofmobilephonesthaninbooksofawedding.

Wealsoverifyifthedurationoftheevent(timedifferenceofthe1stimagetakenandthelastimagetaken)haveaninfluenceoftheformat/sizeofthephotobookbeingordered.

Additionalparametersareverified,suchasifthegeoinformation(longitudeandlatitude)isincludedandused,e.g.inordertoplaceamapofthelocationinthebook.

8302-11, Session 4

SmartFit: automatic photo fitting for variable data printingZ.Karni,A.Gaash,Hewlett-PackardLabs.IsraelLtd.(Israel)

Wepresentanalgorithmforautomaticsmartimagefitting:changingthesizeofanimagesothatitmayfit“naturally”withinagivenframe.Astheframe’sdimensionsandaspectratiowillgenerallydifferfromthatoftheimage,thealgorithmpreservesimportantdetailsintheiroriginalaspectratio,whilelessimportantdetailsundergomoresubstantialdeformations.Thisproblemisusefulformanycommercialprintandwebapplications.Oneexampleisanautomaticretargetingofimagesinwebpageswherethepageshouldfitsmoothlyintodifferentsizessuchasofcellularphones,Touch-Pads,regularmonitorsandprintedpages.AnotherexampleistheHPSmartStreamDesigner,whichisatooltocreatevariableandpersonalizedcontentdocuments.ThesmartimagefittingalgorithmhasbeenimplementedwithintheSmartStreamDesigner,andhasbeenpresentedatIPEX2010(Birmingham)andGraphExpo2010(Chicago),whereitreceivedhighlypositivereactions.

8302-12, Session 4

All new custom path photo book creationW.H.Wang,R.Muzzolini,Shutterfly(UnitedStates)

Inthispaper,wepresentanallnewcustompathtoallowconsumerstohavefullcontroltotheirphotosandtheformatoftheirbooks,whileprovidingthemwithguidancetomaketheircreationfastandeasy.Theuserscanchoosetofullyautomatetheinitialcreation,andthencustomizeeverypage.Thesystemmanagemanydesignthemesalongwithnumerousdesignelements,suchaslayouts,backgrounds,embellishmentsandpatternbands.Theuserscanalsoutilizephotos

frommultiplesourcesincludingtheircomputers,Shutterflyaccounts,ShutterflySharesitesandFacebook.Theuserscanalsouseaphotoasbackground,add,moveandresizephotosandtext-puttingwhattheywantwheretheywantinsteadofbeingconfinedtotemplates.Thenewpathallowsuserstoaddembellishmentsanywhereinthebook,andthehigh-performanceplatformcansupportupto1,000photosperbookandupto25picturesperpage.ThepathofferseitherSmartAutofillorStoryboardfeaturesallowingcustomerstopopulatetheirbookswithphotossotheycanaddcaptionsandcustomizethepages.

8302-13, Session 4

Investigation of the role of aesthetics in differentiating between photographs taken by amateur and professional photographersS.Xue,PurdueUniv.(UnitedStates);Q.Lin,D.Tretter,S.Lee,Hewlett-PackardLabs.(UnitedStates);Z.Pizlo,J.P.Allebach,PurdueUniv.(UnitedStates)

Inadditionaltopreviouslowlevelfeatures,weutilizedhighlevelrulescommonlyacceptedbythephotographycommunityasthenewfeatures.Bydoingso,wecannotonlytrainthesystemtoregardsharp,properly-lightedphotosasgoodphotos,butcanalsotakeintoaccountthefactthatphotosneedtobecomposedinanaestheticwaytobequalifiedasprofessionalones.Compositionrulessuchasbalance,rhythm,andvisualpathsarethemainfocusofthenewfeatures.

Photosfromon-linesharingwebsiteserveasthesourcefortrainingandtesting.Weusedcrawlerstodownloadimages,theiruserratings,andotherstatistics.Machinelearninganddataminingtechniquesareimplementedtotrainthesystembasedonthefeatures.Classificationresultsarecomparedwiththosefrommethodsproposedbypreviouspapers.

Inordertoavoidbiasfromusingopinionsofusersataspecificwebsite,andtogetviewpointsfromprofessionalphotographers,wealsoaskprofessionalphotographerstoratesomeofthephotosrandomlychosenfromthetestingdatasettoseeiftheclassificationresultsfromthealgorithmconfirmwiththeviewpointsofprofessionalphotographer.

8302-14, Session 5

Learning from user data in FacebookJ.Yang,FacebookInc.(UnitedStates)


8302-15, Session 5

Measuring engagement effectiveness in social media: a time-sensitive methodologyT.Sun,L.Li,W.Peng,XeroxCorp.(UnitedStates)

TheeffectivenessmeasurementsinanytraditionalCRMsystem(e.g.customercallcenter)areprimarilybasedonthedurationtimepercalland/ornumberofansweredcallsbyserviceagentsperday.Insocialmediaenvironment,wehaveamultitudesofconversationalcontentsandunderlyingsocialgraphdatawidelyavailable.Theeffectivenessofsocialengagementcanbemeasuredbyanalyzingthecontentofconversations,thesentimentofcustomersandtheirsocialimpactsinunderlyingsocialnetwork.Inthispaper,wepresentatime-sensitivetopic-relevantandsentiment-dependentmetrictomeasuretheeffectivenessofserviceagents’engagementofcustomercareinsocialmedia.Specifically,wequantifytheengagementeffectivenessbymakinguseofapyramidkernel,whichmeasuresthetime-sensitivedistanceofanemployee’sengagementwithinamulti-resolutionspaceintermsofpolarizedtopicalchanges.Theproposedmetricisabletocapturetheengagementbehaviorofagentsbyanalyzingtheconversationsbetweentheagentsandthecustomersinsocialmediaenvironment,e.g.,Twitter.Insummary,themaincontributionofthis

Conference 8302


paperisanewtime-sensitivepyramidkernelfunctionbasedonimplicittopicalandsentimentalcorrespondencesamongdailyconversations,whichenablesdiscriminativeevaluationforagentengagementinsocialcustomercare.

8302-16, Session 5

Building a scalable storage for images on a social networkJ.MedranoNavarro,TuentiTechnologies(Spain)

Imagesareoneofthekeycomponentsonasocialnetwork.Astorageforimagesneedstobehighlyscalable,andprovideredundancy,highavailabilityandtheabilitytogrowitssize.Efficiencyisalsorequiredsothatdiskstorageandtheneedforprocessingpowershouldbeminimized.

OurstorageusesaCDN(ContentDeliveryNetwork)asawebcachethatallowsustomeethighthroughputrequirements.WhenanimageisnotcachedintheCDN,itisrequestedtotheIRL(ImageRoutingLayer)thatisinchargeoffindingitsphysicallocation.IfIRLisnotabletoretrievetheimagefromoneofthelocationsitcangetitfromtheothercopiespreventingtheCDNandtheuserfromnoticingthemiss.Iftherequestedsizeisnotavailableonthestorageit’sresizedbytheIRL.Expensiveoperationslikefindingthephysicallocationorresizingareonlydonewhenthere’sacachemissontheCDN.

Thephysicalstorageissplitinhomogeneusbucketsthatarespreadamongthestorageservers.Growthstrategyisperformedbyaddingmorestorageserversandrebalancingbucketstowardsthem.Rebalancingnotonlyprovidesfreespaceonfullserversbutallowstheuploadbandwidthtoincreasebecausetherewillbefewerbuckets,andsofeweruploadsperserver.

8302-17, Session 5

Color correction of smartphone photos with prior knowledgeY.Zhao,S.Wang,XeroxCorp.(UnitedStates);J.Jiang,RochesterInstituteofTechnology(UnitedStates)

Thecolorimageistheresultofacomplexinteractionbetweenthreemajorcomponents:scene,illuminationandcamerasensor.Typically,theeffectofthescenepropertiesandtheinfluenceoftheilluminationaretightlycoupledandveryhardtoseparatefromeachother.Unlikethehumanvisualsystem,whichhasthepropertyofperceivingthecoloroftheobjecttoremainconstantwhenthelevelandcoloroftheilluminationarechanged,thedigitalcameracompletelylacksthisproperty.Althoughtherearesomebuild-infeaturesorfunctionsonthesensortosetupwhitepointandexposuretime,insomecases,thephotosarestillnotsatisfactorybecauseofunwantedcolorcastorpoorcontrast.Thispaperpresentsamethodforautomaticallyremovingcolorcastandadjustinglightnesscontrastofdigitalphotosduetotheincorrectin-camerasettingwithpriorknowledgeforestimatingcapturingillumination.Theproposedmethodistouseaperson’sheadcharacteristicsasthepriorknowledge.Onedecentheadimagewastakeninadvanceundersufficientilluminationconditionandsavedasthestandardimage.Wheneverthepersonsnapsatargetphoto,healsotakesanimageofhisownheadasthereferenceimage.Thesetwoheadimagesarecomparedtoestimatethecapturingilluminationandbuildthepropercolortransformation.Afterthat,thetransformationisusedtocorrectthetargetphototakenunderthesimilarillumination.

8302-18, Session 5

XML data compression in web publishingR.Qiu,W.Hu,Z.Tang,X.Lu,PekingUniv.(China);L.Zhang,PekingUniv.FounderGroupCorp.(China)

XMLiswidelyusedinvariousdocumentformatsontheweb.Butithascausednegativeimpactssuchasexpensivedocumentdistributiontimeovertheweb,andlongcontentjumpingandrenderingdelay,

especiallyonmobiledevices.HenceweproposedaSchema-basedefficientqueryableXMLcompressor,calledXTrim,whichsignificantlyimprovescompressionratiobyutilizingoptimizedinformationinXMLSchemawhilesupportingefficientqueries.Firstly,XTrimdrawsstructureinformationfromXMLdocumentandcorrespondingXMLSchema.ThenanoveltechniqueisusedtotransformtheXMLtree-likestructureintoacompactindexedformtosupportefficientqueries.Atthesametime,textvaluesareobtained,andalanguage-basedtexttrimmethod(LTT)thatfacilitateslanguage-specifictextcompressorsisadoptedtoreducethesizeoftextvaluesinvariouslanguages.InLTTawordcompositiondetectionmethodisproposedtobetterprocesstextinnon-Latinlanguages.ToevaluatetheperformanceofXTrim,wehaveimplementedacompressorandqueryengineprototype.Viaextensiveexperiments,resultsshowthatXTrimoutperformsXMillandexistingqueryablealternativesintermsofcompressionratio,aswellasthequeryefficiency.ByapplyingXTrimtodocuments,thestoragespacecansaveupto30%andthecontentjumpingandrenderingdelayisreducedtolessthan100msfrom4seconds.

8302-19, Session 6

Layout hierarchies for interactive design reuseD.S.Greig,A.A.Hunter,D.N.Slatter,Hewlett-PackardLabs.(UnitedKingdom)

Intheir2010paper[1]theauthorsdescribeinoverviewthebasisofaneasytousewebserviceforhighqualityauthorshipaimedatoccasionaluserswithlimitedaestheticdesignskills.Thesystemallowsuserstodraganddropmaterialfromexistingmagazinesintothemagazineunderconstructionandautomaticallyrearrangespagesaffectedbythenewcontenttoproduceanewlayout.Thesystemhasashufflefunctionthatallowstheusertochoosefromfurtheradditionallayoutsofthesamematerial.

Thispaperdescribesanimportantaspectofthepreviousmagazinedesignsystem.Theproblemisthataverywidespaceofdesirablelayoutsexists,andindividuallayoutenginescanonlyspanasmallsegmentofthatspace.Ifavailablelayoutenginescanbecombinedintorobusthierarchiesthenmuchmorecomplexdesignsarepossible.

Ahierarchicaldocumentstructureisdescribedthatrepresentscomplexlayoutsandsupportsusereditoperationsinteractingwithasetoflayoutenginestoproduceaestheticallypleasingoutcomes.

[1]Hunter,A.,Slatter,D.,Greig,D.,“WebBasedMagazineDesignforSelfPublishers”,ProcSPIE7879,789902(2011)

8302-20, Session 6

Automatic page composition with combined cropping and layout technologiesA.A.Hunter,D.N.Slatter,D.S.Greig,Hewlett-PackardLabs.(UnitedKingdom)

Automaticlayoutalgorithmssimplifythecompositionofimage-richdocuments,buttheystillrequireuserstohavesufficientartistrytosupplywellcroppedandcomposedimagery.Combininganautomaticcroppingtechnologywithadocumentlayoutsystemenablesbetterresultstobeproducedfasterbyless-skilledusers.Thispaperreviewspriorworkinautomaticimagecroppingandautomaticpagelayoutandpresentsacaseforacombinedcropandlayouttechnology.Theauthorsdescribeonesuchtechnologyinasystemforinteractivepublicationdesignbyamateurself-publishers.Theyshowthatprovidinganautomaticcroppingsystemwithadditionalinformationaboutthelayoutcontextcanenableittogenerateamoreappropriatesetofrankedcropoptionsforagivenimage.Similarly,theyshowthatprovidinganautomaticlayoutsystemwithsetsofrankedcropoptionsforimagescanenableittocomposemoreappropriatepagelayouts.

Conference 8302


8302-21, Session 6

Psychophysical evaluation of document visual similarityA.Satkhozhina,I.Ahmadullin,PurdueUniv.(UnitedStates);S.Lee,Hewlett-PackardCo.(UnitedStates);Z.Pizlo,J.P.Allebach,PurdueUniv.(UnitedStates)

Beingabletoautomaticallycomparedocumentlayouts,andclassifyandsearchdocumentsbasedontheirvisualappearanceisaveryimportanttaskinmanyapplicationsthatmanagelargedatabases.Mostoftheseapplicationsaretargetedforcustomeruse.Thereforeitisessentialtounderstandhowpeopleinterpretvisualsimilaritybetweendocumentsandwhatdocumentfeaturesplaymainrolesindeterminingvisualsimilarity.Thepurposeofourresearchistodeterminedocumentfeaturesthatarethemostcriticalindecidingvisualsimilarityforhumanperception.Thetextcontentwillbeignoredsincewearelookingforvisualsimilarityonly.Inthisresearch,psychophysicalexperimentswillbeconductedtoidentifywhatdocumentfeaturesarethemostimportantforhumanperceptionindecidingthevisualsimilarity.Forourstudy,wewilluseadatabaseofdocumentsthatwerecollectedfromvariousPurdueorganizations.Sixtydocumentpageswillbeselectedfromthedatabaseandorganizedintofivetestsets.Also,fivequerydocumentpageswillbechosenfromthedatabase.Theparticipantwillbeaskedtoplacethesetofprinteddocumentpagesaroundaquerydocumentpageinsuchwaythatthedistancebetweenthequerydocumentandthedocumentpageisinverselyproportionaltothevisualsimilaritybetweenthem.Resultswillberecordedandanalyzedtodeterminethedocumentfeaturesthatarethemostimportantindecidingvisualsimilarityinhumanperception.

Words:228

8302-22, Session 6

Similarity pyramid: browsing a document database with respect to visual similarityI.Ahmadullin,J.P.Allebach,PurdueUniv.(UnitedStates)

Inthispresentationwewilltalkaboutorganizationofsingle-pagedocumentsina3-Dhierarchicalstructurecalledasimilaritypyramid.ThepyramidisconstructedfromadocumentdatabasethatisembeddedonalowdimensionalsurfacewiththehelpofanonlineardimensionalityreductionalgorithmcalledIsomap.Themappingalgorithmpreservessimilaritydistancesbetweendocumentsbymappingdocumentsthatareclosetoeachotherinafeaturespacetopointsonlowdimensionalsurfacethatareclosetoeachother.Higherlevelsofthepyramidconsistofdocumentimageiconsthatrepresentalargegroupofroughlysimilardocuments,whereaslowerlevelscontaindocumentimageiconsrepresentingsmallgroupsofverysimilardocuments.Ausercanbrowsethedatabasebymovingalongacertainlevelofapyramidbymovingbetweendifferentlevels.

8302-23, Session 6

Automatic design of magazine coversA.Jahanian,PurdueUniv.(UnitedStates)andHewlett-PackardLabs.(UnitedStates);J.P.Allebach,PurdueUniv.(UnitedStates);Q.Lin,J.Liu,N.Damera-Venkata,E.O’Brien-Strain,S.Lee,J.Fan,D.Tretter,Hewlett-PackardLabs.(UnitedStates)

Inthispaperweproposeasystemfortheautomaticdesignofmagazinecoversthatquantifiesanumberofconceptsfromartandaesthetics.Weareinterestedinknowinghowprofessionaldesignersthinkintheprocessofmagazinecreationandhowtheycreateacoverconsideringbothformandfunctionality.Ifwecanquantifyandmodelthisprocess,wecansupportnon-designerstoproducesomedesignsclosetotheworkofprofessionals.Accordingly,wehavecollaboratedwithprofessionaldesigners,magazineartdirectorsandeditorialboards,andjournaliststostudytheprocessofmagazinecoverdesign.Wehavethenimplementedwhatwehavelearnedintheframeworkofasoftwaretool.Ourframeworkdividesthetaskofdesignintothree

mainmodules:visualbalanceinlayout,designofcolorformastheadandcoverlines,andtypographyofcoverlines.Atthisstageofourwork,feedbackofprofessionaldesignersonourdesignsimpliesthatourresultsarecongruentwiththeirintuition.However,ourfutureworkincludesfurtherevaluationofourautomaticdesignsbyusers.Wealsoendeavortoquantifytheideaofstyleindesign.Thestyleofamagazine,whichisheavilyinfluencedbymagazinetype,hasamajorinfluenceonthedesignofamagazinecover.Weareinterestedinextendingourmodelofcolordesigntoincludestyleofthemagazine.Wearealsointerestedinaligningthetypefacewiththestyleofthemagazine.

8302-24, Session 7

Sentiment analysis and live customer intelligence from social channelsM.Hsu,Hewlett-PackardLabs.(UnitedStates)

Theproliferationofonlinecommunitiesandsocialnetworkscreatedanewchannel,theSocialChannel,forconsumerstosharetheirexperiencesandvoicetheiropinionsabouteverything.Thedynamicsocialchannelengagescustomers,prospects,partners,influencers,andemployees-touchingvirtuallyeverykeyconstituentinanorganization’svaluechain.Thesechannelsarereshapinghowcustomersevaluateandchooseproducts,howbrandsareperceived,andhowbusinessprocessesinteractwiththecustomers.Theabilitytomodel,identify,understand,measure,andtimelyreacttosentiment,opinion,preference,andbehaviorexpressedinthisunstructuredcontentistransformingthewaycompaniesinteractandmanagerelationshipswiththeircustomers.InthissessionwewillpresentrecentworkintheareaoflivebusinessintelligenceatHPLabs.Inparticularwewilldiscusslivecustomerintelligence,theuseofstreaming,real-timetextanalyticstostudyusersentimentandfeedbackcollectedfromreviewforums,onlineretailers,blogs,andsocialmediaoutletssuchasTwitterandFacebook.Anumberofusecaseswillalsobeillustrated.

8302-25, Session 7

Automatic content recognition for the next-generation TV experienceX.Lin,Vobile,Inc.(UnitedStates)

InrecentyearsinternetistouchingalmosteverycornerofourlifeandTVisnoexception.First,anewgenerationofSmartTVshasbeenintroduced.Second,applicationsrunningonmobiledevices(socalled“second-screenapps”)havesignificantlyenrichedTVwatchingexperience.Asanenablerofcontent-awareTVsandapps,automaticcontentrecognition(ACR)isattractingalotofattentionrecently.ThispaperpresentsanoverviewofACRinthiscontext.Itattemptstoansweranumberofquestions:WhydoweneedACRforthenextgenerationTVexperience?WhatistherelationshipbetweenACRandexistingtechnologies?WhataretheuniquerequirementsandchallengesonACRinthoseapplications?WhataretypicalarchitecturestoimplementACR?Italsodescribestheexistingproductsandprototypesinthisspace.

8302-26, Session 7

Marketing image categorization using hybrid human-machine combinationsN.Gnanasambandam,XeroxCorp.(UnitedStates)


Conference 8302


8302-27, Session 7

Practical experiences in analog to digital content transformationP.D.Reddy,Hewlett-PackardLabs.(UnitedStates)

Theinitialgoalwastotakeout-ofcopyrighttextsandmakethemavailabletoreadonline(forfree)ortoown(viaPrint-ondemand).Thescannedpageimagesoftheseworkscaptureallthenuancesoftheoriginalprinting,howeversupportingrapiddownload,search,accessibility,andre-flowabletextrequiresconvertingtheanalogpageimagesintodigitaltextinanelectronicpublicationformat.Unfortunatelycommercially-availableOpticalCharacterRecognitionenginescannotprovideadequateaccuracywithoutexpensiveandtimeconsumingmanualintervention.Inthispaper,wedescribepracticaltechniquesforachievingahighdegreeofaccuracy,scalingtoprocesslargevolumesefficiently,andwebinterfacesforseamlesscontentconsumptionandefficienterrorcorrection.

8302-28, Session 7

Global image analysis to determine suitability for text-based image personalizationH.Ding,R.Bala,Z.Fan,XeroxCorp.(UnitedStates);C.A.Bouman,J.P.Allebach,PurdueUniv.(UnitedStates)

Lately,imagepersonalizationisbecominganinterestingtopic.Imageswithvariableelementssuchastextusuallyappearmuchmoreappealingtotherecipients.Inthispaper,wedescribeamethodtopre-analyzetheimageandautomaticallysuggesttotheuserthemostsuitableregionswithinanimagefortext-basedpersonalization.Themethodisbasedoninputgatheredfromexperimentsconductedwithprofessionaldesigners.Ithasbeenobservedthatregionsthatarespatiallysmoothandregionswithexistingtext(e.g.signage,banners,etc.)arethebestcandidatesforpersonalization.Thisgivesrisetotwosetsofcorrespondingalgorithms:oneforidentifyingsmoothareas,andoneforlocatingtextregions.Furthermore,basedonthesmoothandtextregionsfoundintheimage,wederiveanoverallmetricfor“suitabilityforpersonalization”.

8302-29, Session 7

Chrominance watermark embed using a full-color visibility modelA.M.Reed,DigimarcCorp.(UnitedStates)

Noabstractavailable

8302-30, Session 7

Document image orientation based on both text and imageY.Sun,C.Liu,X.Ding,TsinghuaUniv.(China);Z.Fan,F.Tse,XeroxCorp.(UnitedStates)

ThispaperinvestigatedtheproblemoforientationdetectionfordocumentimageswithChinese,Japanese,orKoreancharacters.Theseimagesmaybeinfourorientations:rightsideup,up-sidedown,90°and270°rotatedcounterclockwise.First,wepresentedthestructureoftext-recognition-basedorientationdetectionalgorithm.Textlineverificationandorientationjudgmentmethodsweremainlydiscussed,afterwardsmultipleexperimentswerecarried.Distance-differencebasedtextlineverificationandconfidencebasedtextlineverificationwereproposedandcomparedwithmethodswithouttextlineverification.Then,apicture-basedorientationdetectionframeworkwasadoptedforthesituationwherenotextlinewasdetected.Thishigh-levelclassificationproblemwassolvedbyrelativelylow-level

visionfeaturesincludingColorMoments(CM)andEdgeDirectionHistogram(EDH),withdistant-basedclassificationscheme.Finally,confidence-basedclassifiercombinationstrategywasemployedinordertomakefulluseofthecomplementaritybetweendifferentfeaturesandclassifiers.Experimentsshowedthatbothtextlineverificationmethodswereabletoimprovetheaccuracyoforientationdetection,andpicture-basedorientationdetectionhadagoodperformanceforno-textimageset.

Conference 8302


Conference 8303: Media Watermarking, Security, and Forensics 2012Monday-Wednesday23-25January2012

Part of Proceedings of SPIE Vol. 8303 Media Watermarking, Security, and Forensics IV

8303-01, Session 1

Security threat to media security applications based on scale-space feature extractionC.Hsu,AcademiaSinica(Taiwan);C.Lu,InstituteofInformationScience(Taiwan);S.Pei,NationalTaiwanUniv.(Taiwan)

Scale-spaceimagefeatureextraction(SSIFE)hasbeenwidelyadoptedinmultimediasecurityandretrievalduetoitspowerfulresiliencetoattacks.However,thesecuritythreattoSSIFE-basedmediasecurityapplicationsisrelativelyunexplored.Inthispaper,thesecuritythreat,composedofaconstrained-optimizationkeypointinhibitionattack(KIHA)andakeypointinsertionattack(KISA),isspecificallydesignedforscale-spacefeatureextractionmethodssuchasSIFTandSURF.

TheprincipleofKIHAistomakeafooloffeatureextractionprotocolsinthatthedetectionrulesarepurposelyviolatedsothatnolocalmaximumcanbefoundaroundinalocalregion.Ontheotherhand,KISAisdesignedtocreatethefalsepositiveproblem.OurmethodisevaluatedandcomparedwithDoetal.’smethod(ACMMM’10),whichalsofiguresouttheweaknessofourpreviouswork(ACMMM’09).Inaddition,ourproposedsecuritythreatisappliedtoanimagecopydetectionmethodtogetherwithaweb-scaleimagedatabaseforperformanceevaluation.

8303-02, Session 1

Robust image obfuscation for privacy protection in Web 2.0 applicationsA.Poller,M.Steinebach,H.Liu,Fraunhofer-InstitutfürSichereInformations-Technologie(Germany)

Inthispaperwepresenttwoapproachestorobustimageobfuscationbasedonpermutationofimageregionsandchannelintensitymodulation.Theproposedconceptofrobustimageobfuscationisasteptowardclient-to-clientsecurityinWeb2.0applications.Ithelpstoprotecttheprivacyoftheusersagainstthreatscausedbyimagerobotsandcertainwebapplicationswhichextractbiometricandotherfeaturesfromimagesfordata-linkagepurposes.Theapproachesdescribedinthispaperconsiderthatimagesuploadedtoweb2.0applicationspassseveraltransformationsuntilthereceiverdownloadsthem.Incontrasttoexistingapproachesourfocusison

usability,thereforetheprimarygoalisnotamaximumofsecuritybutanacceptabletrade-offbetweensecurityandresultingquality.

8303-03, Session 2

Improved Fourier domain template and patchwork embedding using spatial maskingH.Liu,M.Steinebach,Fraunhofer-InstitutfürSichereInformations-Technologie(Germany)

Robustnessagainstdistortionscausedbycommonimageprocessingisoneoftheessentialpropertiesforimagewatermarkingtobeapplicableinreal-worldapplications.TypicaldistortionsincludelossyJPEGcompression,filtering,cropping,scaling,rotation,andsoon,amongwhichgeometricdistortionismorechallenging.Evenslightgeometricdistortioncantotallyfailthewatermarkdetectionthroughde-synchronization.Anotherimportantpropertyisthewatermarkpayload.Althoughone-bitwatermarkiswidelyusedinresearch

workforalgorithmtestingandevaluation,onlycheckingwhetheraspecificwatermarkexistsdoesnotmeettherequirementofmanypracticalapplications.ThispaperpresentsapracticalrobustimagewatermarkingalgorithmwhichcombinestemplateembeddingandpatchworkwatermarkinginFourierdomain.Theembeddedtemplateenablesthenecessaryrobustnessagainstgeometricdistortionsandthepatchworkapproachprovidesareasonablewatermarkpayloadwhichcanmeettherequirementofmostapplications.Aspatialperceptualmaskisusedtoreshapetheembeddedenergyafteritisinvertedtothespatialdomain,whichsignificantlyimprovestheimagequalityandenhancestherobustnessofbothtemplateandwatermark.Implementationissuesandsolutions,e.g.fine-tuningofembeddingenergyofindividualpixels,arealsodiscussed.Experimentalresultsdemonstratetheeffectivenessandpracticabilityoftheproposedalgorithm.

8303-04, Session 2

Ranking search for probabilistic fingerprinting codesM.Schäfer,W.Berchtold,M.Steinebach,Fraunhofer-InstitutfürSichereInformations-Technologie(Germany)

Digitaltransactionwatermarkingtodayisawidelyacceptedmechanismtodiscourageillegaldistributionofmultimedia.Thetransactionwatermarkisauser-specicmessagethatisembeddedinallcopiesofonecontentandthusmakesitindividual.Therewithitallowstotracebackcopyrightinfringements.Onemajorthreatontransactionwatermarkingarecollusionattacks.Here,multipleindividualizedcopiesoftheworkarecomparedand/orcombinedtoattacktheintegrityoravailabilityoftheembeddedwatermarkmessage.Onesolutiontocountersuchattacksaremathematicalcodescalledcollusionsecurefingerprintingcodes.Problemsarisewhenapplyingsuchcodestomultimedialeswithsmallpayload,e.g.shortaudiotracksorimages.Thereforethecodelengthhastobeshortenedwhichincreasestheerrorratesand/ortheeffortofthetracingalgorithm.Inthisworkweproposeanapproachwhethertouseasanadditiontoprobabilisticfingerprintingcodesforareductionoftheeffortandincrementofsecurity,aswellasanewseperatemethodprovidingshortercodesataveryfastandhighaccuratetracingalgorithm.

8303-05, Session 2

Stereoscopic watermarking by horizontal noise mean shiftingJ.Lee,H.Kim,H.Choi,S.Choi,H.Lee,KAIST(Korea,Republicof)

Depth-image-basedrendering(DIBR)isamethodtorepresentastereoscopiccontent.TheDIBRconsistsofamonoscopiccenterviewandanassociatedper-pixeldepthmap.Usingthesetwocomponentsandgivendepthconditionfromanuser,theDIBRrendersleftandrightviews.TheadvantagesofDIBRisnumerous.Theusercanchoosenotonlythemonoscopicorstereoscopicviewselectively,butalsothedepthconditionwhathepreferswhenhewatchesastereoscopiccontent.However,intheviewofcopyrightprotection,sincenotonlythecenterviewbutalsoeachleftorrightviewcanbeusedasamonoscopiccontentwhentheyareillegallydistributed,thewatermarksignalwhichisembeddedinthecenterviewmusthaveanabilitytoprotecttherespectivethreeviews.Inthisstudy,wesolvethisproblembyexploitingthehorizontalnoisemeanshifting(HNMS)technique.Weexploitthefactthattheobjectsintheviewareshiftedonlytohorizontalwaywhenthecenterviewrenderstotheleftandrightviews.Usingthisfact,theproposedstereoscopicwatermarkingschememovesthemeanofhorizontalnoisehistogramwhichisinvariantto


Conference 8303horizontalshifting,andweachievegoodperformanceasshownintheexperimentalresults.

8303-06, Session 2

Reversible q-ry watermarking with controllable prediction error and location map-free capabilityT.Efimushkina,K.O.Egiazarian,TampereUniv.ofTechnology(Finland)

Inthispaperanewhighcapacityandhighqualityreversibleimagewatermarkingmethodispresented.Itisbasedonpredictionerrorexpansionwithacontrollablepredictionerrorandalocalactivityindicatorusedtomanagetheembeddingcapacity.ThismethodenableshidingthesecretdatafromGaloisfield,whichallowstoembedmorethanonebitperpixelinasinglerunofthealgorithm.Specialconditionsforlocationmapfreedataembeddingarederivedinordertodecreasetheauxiliarydata.However,thealgorithmallowstoleaveoutthecomputationofthesethresholdssubstitutingthembyuser-specifiedthresholds.Inthatcase,onlyoneiterationoftheencodingprocessisperformedwiththeutilizationoflosslesslycompressedmask,thatindicatesnon-embeddablepixelpositions.

Algorithmhasaspecialcontrolmechanismthatallowsthedataembeddingprocessonlyifthepredictionerrorisbounded.Moreover,over/underflowconditionsaretobesustained.

Theproposedmethodiscomparedwiththestate-of-the-artreversibledatahidingmethodsanddemonstratesthehigherperformanceintermsoftheimagequalityandcapacity.Thecapacitycanbefurtherincreasedbyapplyingmultipleiterationsoftheproposedq-ryscheme.

8303-07, Session 3

Optimizing pixel predictors for steganalysisV.Holub,J.Fridrich,BinghamtonUniv.(UnitedStates)

Astandardwaytodesignsteganalysisfeaturesfordigitalimagesistochooseapixelpredictor,useittocomputeanoiseresidual,andthenformjointstatisticsofneighboringresidualsamples(co-occurrencematrices).Thispaperproposesageneraldata-drivenapproachtooptimizingpredictorsforsteganalysis.First,alocalpixelpredictorisparametrizedandthenitsparametersaredeterminedbysolvinganoptimizationproblemforagivensampleofcoverandstegoimagesandagivencoversource.Ourresearchshowsthatpredictorsoptimizedtodetectaspecificcaseofsteganographymaybevastlydifferentthanpredictorsoptimizedforthecoversourceonly.Theresultsindicatethatoptimizedpredictorsmayimprovesteganalysisbyarathernon-negligiblemargin.Furthermore,weconstructthepredictorssequentially-havingoptimizedkpredictors,designthek+1stonewithrespecttothecombinedfeaturesetbuiltfromallkpredictors.Inotherwords,givenafeaturespace(imagemodel)extend(diversify)themodelinaselecteddirection(functionalformofthepredictor)inawaythatmaximallyboostsdetectionaccuracy.

8303-08, Session 3

Steganalysis of JPEG images using rich modelsJ.Kodovsky,J.Fridrich,BinghamtonUniv.(UnitedStates)

Inthispaper,weproposearichmodelofDCTcoefficientsinaJPEGfileforthepurposeofdetectingsteganographicembeddingchanges.ThemodelisbuiltsystematicallyasaunionofsmallersubmodelsconstructedasjointdistributionsofDCTcoefficientsfromtheirfrequencyandspatialneighborhoodscoveringawiderangeofstatisticaldependencies.Duetoitshighdimensionality,wecombinetherichmodelwithensembleclassifiersandconstructdetectorsforsixmodernJPEGdomainsteganographicschemes:nsF5,model-basedsteganography,YASS,andschemesthatusesideinformation

attheembedderintheformoftheuncompressedimage:MMEx,BCH,andBCHopt.Theresultingperformanceiscontrastedwithpreviouslyproposedfeaturesets,includingbothlow-andhigh-dimensionalsets.Wealsoinvestigatetheperformanceofindividualsubmodelswhengroupedbytheirtypeaswellastheeffectofcartesiancalibration.Theproposedrichmodeldeliverssuperiorperformanceacrossalltestedalgorithmsandpayloads.

8303-09, Session 3

Co-occurrence steganalysis in high dimensionT.Pevny,CzechTechnicalUniv.inPrague(CzechRepublic)

Thestateoftheartsteganalyticfeaturesforspatialdomain,andtosomeextentfortransferdomains(DCT),arebasedonhistogramofco-occurrencesofneighboringelements.Therationalebehindisthatneighboringpixelsindigitalimagesarecorrelated,whichiscausedbythesmoothnessofourworldandbytheimageprocessing.Thelimitationofhistogram-basedfeaturesisthattheydonotscalewellwithadimensionofthehistogram,sincethenumberofhistogrambins(hencenumberoffeatures)growsexponentiallywiththenumberofmodeledneighboringelements.Despitetheuseofensembleclassifiersproposedtodealmoreefficientlywithlargenumberoffeatures,westillbelievethatthesmallnumberofgoodfeaturesispreferableoverthelargenumberofweakerones.

Themajorgoalofthisworkistointroducetheproblemofreductionofhistogrambinswithoutsacrificingthedetectionquality.Fortheproblem,whichwasnotyettackledinthepriorart,aheuristicsolutionisproposed.Thesolution,albeitnotbeingoptimal,alreadyoutperformsthead-hocsolutionusedinthepriorart.

Themethodsreducingthenumberofhistogrambinsareveryimportant,astheresultsofBOSScompetitionshowedthatthefeaturesextractedfromhistogramsofhigherorderandneededtodetectstateoftheartsteganographicalgorithms.

8303-18, Session 4

Privacy and the social networkJ.Staddon,GoogleInc.(UnitedStates)

Thetensionbetweenonlinesocialservicesandprivacyisreadilyapparent.Onlinesocialnetworksareinvaluableformakingconnectionsbetweenpeopleandfacilitatingsharing,butcanalsoincludetheunintentionalover-sharingofcontentandincidentsofunwantedonlineattentionfromotherusers.I’lltalkaboutgaugingprivacyconcerns,expectationsandfeatureutilityasintegralpartsoftheprocessofbuildinganengagingandprivacy-awaresocialnetwork.DrawingexamplesfromourexperiencewithGoogle+,I’lldiscusshoweachiskeytoouriterativeprivacydesignprocess,whichincludesthreatassessment,dataanalysisandampleuserstudies.

8303-11, Session 5

Source camcorder identification with cropped and scaled videosD.Hyun,S.Ryu,M.Lee,J.Lee,H.Lee,H.Lee,KAIST(Korea,Republicof)

Inthispaper,weproposeanewcamcorderidentificationmethodbasedonphoto-responsenon-uniformity(PRNU)toperformwellwithvideosunderinvestigationwhicharesimultaneouslycroppedandscaled.OneofdisadvantagesofPRNU-basedalgorithmsisthatitisverysensitivetode-synchronization.Ifaninputvideoisslightlycroppedorscaled,thetheidentificationprocesswithoutsynchronizationwillfail.Theproposedmethodsolvesthiskindofout-of-syncproblem,byachievingdownscale-toleranceusingminimumaveragecorrelationenergyMellinradialharmonic(MACE-MRH)filter.Theexperimentalresultsshowthattheproposedmethodidentifiessourcedevicefasterandmoreaccuratelythantheexistingmethod.


Conference 83038303-12, Session 5

Digital image forensics for photographic copyingY.Fang,J.Yin,SunYat-SenUniv.(China)

Imagedisplaytechnologyhasgreatlydevelopedoverthepastfewdecades,whichmakeitpossibletorecapturehigh-qualityimagesfromthedisplaymedium,suchasaliquidcrystaldisplay(LCD)screenoraprintedpaper.Therecapturedimagesarenotregardedasaseparateimageclassinthecurrentresearchofdigitalimageforensics,whilethecontentoftherecapturedimagesmayhavebeentempered.Inordertostudythisproblem,wefirstlyobtainedfinelyrecapturedimagesfromLCDscreenandprintedpapers,thentwosetsoffeaturesbasedonthenoiseandthetracesofdoubleJPEGcompressionareproposedtoidentifytheserecapturedimages.Experimentalresultsshowedthatourproposedfeaturesperformwellfordetectingphotographiccopying.

8303-13, Session 5

Forensic audio watermark detectionM.Steinebach,S.Zmudzinski,Fraunhofer-InstitutfürSichereInformations-Technologie(Germany)

Digitalaudiowatermarkingdetectionisoftencomputationalcomplexandrequiresatleastasmuchaudioinformationasrequiredtoembedacompletewatermark.Insomeapplications,especiallyreal-timemonitoring,thisisanimportantdrawback.Thereasonforthisistheusageofsyncsequencesatthebeginningofthewatermark,allowingadecisionaboutthepresenceonlyifatleastthesynchasbeenfoundandretrieved.Weproposeanalternativemethodfordetectingthepresenceofawatermark.Basedontheknowledgeofthesecretkeyusedforembedding,wecreateamarkforallpotentialmarkingstagesandthenuseaslidingwindowtotestagivenaudiofileonthepresenceofstatisticalcharacteristicscausedbyembedding.Inthiswaywecandetectawatermarkinlessthan1secondofaudio.

8303-14, Session 5

Sensor-fingerprint based identification of images corrected for lens distortionM.Goljan,J.Fridrich,BinghamtonUniv.(UnitedStates)

Computationalphotographyisquicklymakingitswayfromprofessionaldevicestoconsumercameras.Recently,cameramanufacturersstartedusingin-cameralens-distortioncorrectionofthecapturedimagetogiveusersmorepowerfulrangeofzoomincompactandaffordablecameras.Sincethedistortioncorrection(barrel/pincushion)dependsonthezoom,itdesynchronizesthepixel-to-pixelcorrespondencebetweenimagestakenattwodifferentfocallengths.Thisposesaseriousproblemfordigitalforensictechniquesthatutilizetheconceptofsensorfingerprint(photo-responsenon-uniformity),suchas“imageballistic”techniquesthatcanmatchanimagetoaspecificcamerainasimilarmannerasabulletcanbematchedtoagunbarrel.Suchidentificationtechniquesmayinfactcompletelyfail.Thispaperpresentsanextensionofsensor-basedcameraidentificationthatwouldworkforimageswhosecontentwascorrectedforlensdistortion.Toreestablishsynchronizationbetweenanimageandthefingerprint,weadoptabarreldistortionmodelandsearchforitsparameterstomaximizethedetectionstatistic,whichisthePCE(peaktocorrelationenergyratio).Theproposedmethodistestedonhundredsofimagesfromthreecompactcamerastoprovetheviabilityoftheconceptanddemonstrateitsefficiency.

8303-15, Session 6

Digital audio authentication by robust feature embeddingS.Zmudzinski,B.Munir,M.Steinebach,Fraunhofer-InstitutfürSichereInformations-Technologie(Germany)

Inmanyscenarios,digitalaudiodatacontainsimportantorsensitiveinformation.Examplesaregivenbyrecordingsoftelephonecalls,policeinterrogations,airtrafficcommunicationorcontentpreservingtheculturalheritage.Astheaudiodatacaneasilybesemanticallymanipulatedwithmoderncomputerhardwareandsoftware,mechanismsforverifyingtheintegrityoftheaudiocontentandtheauthenticityofitsoriginareofspecialinterest.Weintroduceandevaluateanapproachforverifyingtheintegrityofdigitalaudiorecordingbymeansofcontent-basedintegritywatermarking.HereanaudiofingerprintisextractedintheFourierdomainandembeddedasadigitalwatermark.Thedesignofthefeatureextractionallowsafinetemporalresolutionoftheverificationoftheintegrity.

8303-16, Session 6

High-resolution printed amino acid traces: a first-feature extraction approach for fingerprint forgery detectionM.Hildebrandt,S.Kiltz,J.Dittmann,Otto-von-Guericke-Univ.Magdeburg(Germany)

Fingerprintsareusedfortheidentificationofindividualsforoveracenturyincrimesceneforensics.Here,oftenphysicalorchemicalpreprocessingtechniquesareusedtorenderalatentfingerprintvisible.Forqualityassurancepurposesofthosedevelopmenttechniques,Schwarz(2009)introducesatechniqueforthereproduciblegenerationoflatentfingerprintsusingink-jetprintersandartificialaminoacidsweat.However,thistechniqueallowsforprintinglatentfingerprintsatcrimescenestoleavewrongtraces,too.Hence,Kiltzetal.(2011)introduceafirstframeworkforthedetectionofprintedfingerprints.However,theutilizedprintershaveamaximumresolutionof2400x1200dpi.Inthispaper,weuseaprinterwithamuchbetterresolutionof9600x2400dpi,whichdoesnotproducevisibledotpatternsasreportedinKiltzetal.(2011)Weshow,thatanacquisitionwitharesolutionof12700to25400ppiisnecessarytoextractmicrostuctureswhichperspectivelyallowsforanautomateddetectionofprintedfingerprinttracesfabricatedwithhigh-resolutionprinters.Here,usingourfirsttestsetwith15printedand5real,naturalfingerprintpatternsfromthehumantheevaluationresultsindicateaverypositivetendencytowardsthedetectabilityofsuchtraces.

8303-17, Session 6

Image forgery detection by means of no-reference quality metricsF.Battisti,M.Carli,A.Neri,Univ.degliStudidiRomaTre(Italy)

Inthispaperamethodologyfordigitalimageforgerydetectionbymeansofanunconventionaluseofimagequalityassessmentmetricsisaddressed.Inparticular,theanalysisofthevariationofdegradationsinsideanimageisadoptedtorevealthepresenceofpatchesthathavebeenpastedfromadifferentsource.Theratiobehindthisworkisinthehypothesisthatanyimagemaybeaffectedbyartifacts,visibleornot,causedbytheprocessingsteps:acquisition(i.e.,lensdistortion,imperfectionsintheacquisitionsensors,analogtodigitalconversion,singlesensortocolorpatterninterpolation),processing(i.e.,quantization,storing,JPEGcompression,sharpening,deblurring,enhancement),andrendering(i.e.,imagedecoding,color/sizeadjustment).Theseimpairmentsareusuallylocalizedandtheirstrengthisstrictlydependentonthecontent.Thesecharacteristicscanbeconsideredasafingerprintofeachdigitalimage.Theproposedapproachreliesonacombinationofimagequalityassessmentsystems.Theadoptednoreferencemetricdonotrequireanyinformationabouttheoriginalimage,thusallowinganefficientandstandaloneblindsystemforimageforgerydetection.Theexperimentalresultsshowtheeffectivenessoftheproposedscheme.


8303-10, Session 7

Advancing technology: bane and boon for banknotesS.E.Church,BoardofGovernorsoftheFederalReserveSystem(UnitedStates)

Overthelasttwodecades,rapidadvancesintechnologyhaveledtosignificantthreatstobanknotesecurity.Itmayseemobviousthatadvanceddigitalimagingandprintingtechnologiesprovidereadyaccessibilitytorapidandfaithfulimagecapture,easyimagemanipulationandcustomization,andhigh-resolution,digitaldesktopprinting,posethreatstothesecurityofbanknotes.Whatmaybelessobviousisthatthenegativeconsequencesoftheseadvancesinimagingandprintinghaveledtopositiveconsequencesintheformofinnovationandexplorationofadvancedtechnologiesforuseinbanknoteproductionandsecurity.

Inthisway,theadvancesintechnologyhavebothcreatedproblemsforthesecurityofbanknotesandprovidedpotentialsolutionstoaddresstheproblems.Astheprintedimagesofbanknoteshavebecomemorevulnerable,thebanknotecommunityhasturnedtomaterialsengineering,opticaltechniques,andotheradvancedapproachestoprovidesecurity.Asaresult,modernsecurityfeaturesmayaddadditionaldimensionstothetraditionaltwo-dimensionalprintedimageofthenotesandmayrequiremanipulationorspecialtechniquestoactivateoraccess.Suchnewfeaturesarenotreadilycapturedwithdigitaltechniquesalone,forcingthecounterfeitertouseadditionalmaterialsandskillstoachievesatisfactoryresults.Inthisrespect,advancingtechnologyoffersincreasingopportunitiestodiscouragecounterfeitingandlessenthesuccessoftheoutcomeifattempted.Asaresultofboththethreatsandopportunitiesofferedbyadvancingtechnology,modernsecurityfeaturesandbanknotesarehighlyengineeredproductsbasedontheinnovativemarriageofvenerablyoldbutmodernizedproductiontechniquesandadvanced,preciselyengineeredmaterials.

Thepresentationwillexploresomeofthewaysthattechnologyhasservedbothtothreatenandtoenhancebanknotesecurity.

8303-19, Session 8

Going from small to large data in steganalysisI.Lubenko,A.D.Ker,Univ.ofOxford(UnitedKingdom)

Noabstractavailable

8303-20, Session 8

Identifying a steganographer in realistic and heterogeneous data setsA.D.Ker,Univ.ofOxford(UnitedKingdom);T.Pevny,CzechTechnicalUniv.inPrague(CzechRepublic)

Thetraditionalsteganalyticscenariousuallyconsidersthecase,whenthesteganographertriestodetect,ifasingleobjectcarriespayloadornot.Suchscenarioiscommoninthemostliterature,eventhoughitishighlytheoretical.Inpractice,onecanexpecttoconsiderratherdifferentscenario,wherethesteganographerhastoconsidermultipleuserseachtransmittingmultipleobjects.Thisproblem,knownasapooledsteganalysis,hasbeentheoreticallyintroducedin2006,butnopracticalsolutionhasbeenproposeduntil2011.

Thisworkextendstheexperimentspresentedintheoriginalpublication,toanew,morerealistic,domain,whereweutilizes800000imagesfrom4000usersofasocialnetworkingservice.

8303-21, Session 9

Asymmetric robust quantum image hashingM.Steinebach,H.Liu,Fraunhofer-InstitutfürSichereInformations-Technologie(Germany)

Forensicanalysisofimagesetstodayismostoftendonewiththehelpofcryptographichashesduetotheirefficiency,theirintegrationinforensictoolsandtheirexcellentreliabilityinthedomainoffalsedetectionalarms.Adrawbackofthesehashmethodsistheirfragilitytoanyimageprocessingoperation.Evenasimplere-compressionwithJPEGresultsinanimagenotdetectable.Adifferentapproachistoapplyimageidentificationmethods,allowingidentifyingillegalimagesbye.g.semanticmodelsorfacingdetectionalgorithms.Theircommondrawbackisahighcomputationalcomplexityandsignificantfalsealarmrates.Robusthashingisawell-knownapproachsharingcharacteristicsofbothcryptographichashesandimageidentificationmethods.Itisfast,robusttocommonimageprocessingandfeatureslowfalsealarmrates.Toverifyitsusabilityinforensicevaluation,inthisworkwediscussandevaluatethebehaviorofanoptimizedblock-basedhash.

8303-22, Session 9

Fast detection of Tardos codes with Boneh-Shaw typesM.Desoubeaux,G.LeGuelvouit,FranceTelecomR&D(France);W.Puech,Lab.d’InformatiquedeRobotiqueetdeMicroelectroniquedeMontpellier(France)

Traitortracingaimsatpreventingunauthorizedredistributionofmultimediacontentbyembeddingindividualsequencesofbitswithineachauthorizedcopy.Thesesequenceshavetoberobustagainstcollusionattacks.Collusionistheprocessusedbydishonestuserstoattempttoforgeanuntraceablecontentwiththeircopies.Currentapplicationsoffingerprintingdealwithalargenumberofusersanddistributorsneedtohaveafastdetectionprocess.

InthispaperwepresentatwolevelapproachtoimprovethedetectionofpirateswiththewellknownTardosfingerprintingscheme.ABoneh-ShawrandomcodeisconcatenatedwiththeTardoscodethankstoaq-aryalphabet.AimprovementatthedecodingsideispresentedbyorganizingthedetectionofusersoverdifferentsubgroupsofuserswhichareselectedthankstotheBoneh-Shawrandomcode.TheefficiencyofthedetectionisstronglydependentontheTardoscodelengthbutexperimentationsconfirmneverthelessthatwereducethedecodingcomplexity.

8303-23, Session 9

Locatability of modified pixels in steganographic imagesT.Quach,SandiaNationalLabs.(UnitedStates)

Payloadlocationusingresidualsisasuccessfulapproachtoidentifyload-carryingpixelsprovidedalargenumberofstegoimagesareavailable.Furthermore,eachimagemusthavethepayloadembeddedatthesamelocations.Thesuccessofpayloadlocationisthereforelimitedifdifferentkeysareusedoranadaptiveembeddingalgorithmisused.Giventheselimitations,thefocusofthispaperistolocatemodifiedpixelsinasinglestegoimage.Givenasufficientlylargesetofindependentbinarydecisionfunctions,eachdetermineswhetherapixelhasbeenmodifiedbetterthanguessing,weshowthatitispossibletolocatemodifiedpixelsinasinglestegoimagewithlowerrorrate.Weconstructthesefunctionsusingexistingcoverestimatorsandprovideexperimentalresultstosupportouranalysis.

Conference 8303


8303-24, Session 9

Forensic characterization of camcorded movies: digital cinema versus celluloid film printsX.Rolland-Neviere,B.Chupeau,G.Doërr,L.Blondé,TechnicolorS.A.(France)

Mostnewlyreleasedmoviepiracycanbetracedtoindividualswhousedarecordingdeviceinamovietheater.Sincetheunderlyingtracingmechanismdiffers,theforensicanalysisofapiratecopywilldependonthepiracyprocesse.g.camcordercaptureinadigitalcinemavs.camcordercaptureinalegacycinema.Rapidclassificationofincomingpiratesamplesaccordingtotheirorigin,priortoanyfurtherin-depthanalysis,isthereforedesiredinordertospeedupthewholeforensicprocess.

Theobjectiveofthispaperistodeviseanautomaticoraclethatdiscriminatesinarobustmannercamcordedcopiesoriginatingfromdigitalcinemasandthosefromlegacyones.Todoso,wecarefullyanalysetheprojectionprocessinordertopinpointafewtell-talevisualartifacts,namelyglobalilluminationuniformity,on-screenverticalstabilityandtemporalilluminationpulse.Weextractfeaturesassociatedtotheseartifactsandsubsequentlyfeedthemintoastate-of-the-artclassifiertotrainittoseparatethetwoclassesofcamcordedcopies.Experimentalresultsarethenreportedtodemonstratethediscriminativepowerofthesefeatures.

8303-25, Session 10

Extending a context model for microphone forensicsC.Krätzer,K.Qian,J.Dittmann,Otto-von-Guericke-Univ.Magdeburg(Germany)

Inthepaper,weextendanexistingstatisticalpatternrecognitionbasedmicrophoneforensicapproachbyconsiderationson:recordingsofreplays,theinfluenceofthreedifferentclassesofsoundsources(loudspeakersaswellasharmonicandnon-harmonicnaturalsources)aswellasadditionalinfluencesinsoundtransmissionandrecording(herepowerdecay,echoes,reverberationanddifferentpre-amplifiers).Theextensionofthecontextmodelisaccompaniedbyempiricalinvestigationsontheauthenticationperformanceofourmicrophoneforensicframework,aimingat:a)answeringthequestionhowgoodareplaycanbedifferentiatedfromanatural,non-playbacksignal,andb)attheestimationofthestrengthoftheimpactoftheaforementionedvariations(e.g.ofthesourcetype)intherecordingpipeline.Thedomainknowledgegainedfromallevaluationsisusedtoincreasetheprecisionofourcontextmodelfordescribingtypicalmicrophonerecordingprocesses.Suchanempiricalinvestigationontheinfluencesinvolvedintherecordingprocessallowsforamorereliableestimationoftheplausibilityofourstatisticalpatternrecognitionbasedsourceforensicapproach.

8303-26, Session 10

Simulating large-scale acoustic path benchmarkingM.Arnold,P.Baum,M.Alonso,U.Gries,DeutscheThomsonoHG(Germany);G.Doërr,TechnicolorS.A.(France)

RobustnesstoD/A-A/Dconversionisoneofthekeyfeaturesofwatermarkingsystems.Inaudio,itnaturallytranslatestorobustnessagainstacousticpathtransmissioni.e.theabilitytodetectawatermarkoncethecontenthasbeenplayedwithsomeloudspeakersandpickedupwithamicrophone.

Manyapplicationsinherentlyrequirerobustnessagainstacousticpathtransmission.However,thisproblemisrelativelyunderstudiedintheliterature.Thekeyissueisthatin-depthevaluationoftheacousticpathistimeconsuming,cumbersome,anderror-prone.Theobjectiveofthisstudyistodeviseamethodologythatwouldpermitsimulatingsuch

largescaleacousticpathbenchmarkingcampaignwithoutthelogisticburdencurrentlyassociatedtoit.

Relyingonsomesimplifyingassumption,wecreateavalidatedquicklyadjustableautomatedtooltoreplicatetheimpactofacousticpathtransmissioninagivenroom,withgivenloudspeakersatagivenlocationandgivenmicrophonesatagivenposition.Experimentalresultsdemonstratethesimulationscloselymatchresultswhichwouldbeachievedwithrealmeasurements.

8303-27, Session 10

Noise removing in encrypted color image by statistical analysisN.Islam,W.Puech,Lab.d’InformatiquedeRobotiqueetdeMicroelectroniquedeMontpellier(France)

Cryptographictechniquesareusedtosecureconfidentialdatafromunauthorizedaccessbutthesetechniquesareverysensitivetonoise.Asinglebitchangeinencrypteddatacanhavecatastrophicimpactoverthedecrypteddata.ThispaperaddressestheproblemofremovingbiterrorinvisualdatawhichareencryptedusingAESalgorithmintheCBCmode.Inordertoremovethenoise,amethodisproposedwhichisbasedonthestatisticalanalysisofeachblockduringthedecryption.Theproposedmethodexploitslocalstatisticsofthevisualdataandconfusion/diffusionpropertiesoftheencryptionalgorithmtoremovetheerrors.Experimentalresultsshowthattheproposedmethodcanbeusedatthereceivingendforthepossiblesolutionfornoiseremovinginvisualdatainencrypteddomain.

Conference 8303


Conference 8304A: Multimedia on Mobile Devices 2012Wednesday25January2012

Part of Proceedings of SPIE Vol. 8304A Multimedia on Mobile Devices 2012


Low-complexity bit-plane entropy coding for 3D DWT-based video compressionE.A.Belyaev,K.O.Egiazarian,M.Gabbouj,TampereUniv.ofTechnology(Finland)

AnextensionoftheH.264/SVCstandardiscurrentlythemostpopularscalablevideocodingapproach.But,becauseofhighcomputationalcomplexityofmotionestimationandinter-layerpredictionattheencoderside,softwareandhardwareimplementationsofH.264/SVCencoderinamobiledeviceisadifficulttask.Asanalternativevideoencodersbasedonthree-dimensionaldiscretewavelettransform(3-DDWT)canbeused.Theseapproachesdonotusemotionestimationforexploitingatemporalredundancyofthevideosource,andthus,havelowercomputationalcomplexity.

Forfurtherdecreasingofthe3-DDWTbasedvideocompressioncomplexityweproposenewsimplebit-planeentropycodingofwaveletmatrixes.Allbitsinbit-planesofeachwaveletmatrixareseparatedintotwobinarysequences.Firstsequencehasverylowprobabilityofones,thereforeitisdividedbyseriesofzeroswhicharecompressedbyLevenshteincodes.Probabilityofonesinsecondsequenceiscloseto0.5,thereforebitsofthissequenceareplacedintobitstreamdirectly.

Proposedapproachallowsimplementingentropycodingwithoutusingoflook-uptables.Weshowthatcomputationcomplexityofproposedentropycoderislessthanbit-planearithmeticcoderinJPEG2000andentropyencoderin3-DSPIHT.Atthesametimeitprovidescomparablecompressionefficiency.


Bidirectional probabilistic hyper-graph matching method using Bayes theoremW.Cho,S.Kim,S.Park,ChonnamNationalUniv.(Korea,Republicof)

Establishingcorrespondencesbetweentwohyper-graphsisafundamentalissueincomputervision,patternrecognition,andmachinelearning.Ahyper-graphismodeledbyfeaturesetwherethecomplexrelationsarerepresentedbyhyper-edges.Hence,amatchbetweentwofeaturesetsdeterminesahyper-graphmatchingproblem.Weproposeanewbidirectionalprobabilistichyper-graphmatchingmethodusingBayesianinferenceprinciple.First,weformulatethecorrespondinghyper-graphmatchingproblemasthemaximizationofamatchingscorefunctionoverallpermutationsofthefeatures.Second,weinduceanalgebraicrelationbetweenthehyper-edgeweightmatrixesandderivethedesiredvertextovertexprobabilisticmatchingalgorithmusingBayestheorem.Third,weapplythewellknownconvexrelaxationprocedurewithprobabilisticsoftmatchingmatrixtogetacompletehardmatchingresult.Finally,wehaveconductedthecomparativeexperimentsonsyntheticdataandrealimages.Experimentalresultsshowthattheproposedmethodclearlyoutperformsexistingalgorithmsespeciallyinthepresenceofnoiseandoutliers.


SeamCrop for image retargetingJ.Kiess,B.Guthier,S.Kopf,W.Effelsberg,Univ.Mannheim(Germany)

Inthispaper,wepresentanovelapproachfortheadaptationoflargeimagestosmalldisplaysizes.Asarecentstudysuggests,mostviewerspreferthelossofcontentovertheinsertionofdeformations

intheretargetingprocess.Therefore,wecombinethetwoimageretargetingoperatorsseamcarvingandcroppinginordertoresizeanimagewithoutmanipulatingtheimportantobjectsinanimageatall.First,seamsareremovedcarefullyuntiladynamicenergythresholdisreachedtopreventthecreationofvisibleartifacts.Then,acroppingwindowisselectedintheimagethathasthesmallestpossiblewindowsizewithouthavingtheremovedenergyriseaboveaseconddynamicthreshold.Asthenumberofremovedseamsandthesizeofthecroppingwindowarenotfix,theprocessisrepeatediterativelyuntilthetargetsizeisreached.Ourresultsshowthatbyusingthismethod,moreimportantcontentofanimagecanbeincludedinthecroppingwindowthaninnormalcropping.The“squeezing”ofobjectswhichmightoccurinapproachesbasedonwarpingorscalingisalsoprevented.


Collecting fingerprints for recognition using mobile phone camerasB.Yang,X.Li,C.Busch,GjøvikUniv.College(Norway)

General-purposecamerassuchasthecameraembeddedinamobilephonecanbeexpectedtocomplementtheprofessionalfingerprintandfacescannersasalternativesensorssuitingsomeconsumerbiometricapplicationssuchasmobilee-paymentandticketing,forbothon-lineandon-sitetransaction,whichcanbeusedtoincreasetheprivacyandconvenienceonthepersonalsideandalsoreducethebiometricsystemdeploymentcostonthebusinessside.However,abigchallengeofemployingsuchconsumerelectronicsasbiometricsensoristhesamplequalitycontrolwhichcannotbedirectlyaddressedbyexistingfingerprintqualityestimationmechanismsduetothericherbackgroundnoises,worselightningconditions,andcamerafocusinginaccuracy.Weproposeanalgorithmtocheckthesamplequalityofthefingerprintimagescapturedbymobilephonecamerainanaccurateandefficientway.RidgefrequencyandorientationfeaturesarefullyexploitedintheDiscreteFourierTransformationdomainoftheimageblockstogiveindicationifthestudiedblockishigh-qualityfingerprintareaornot.Diversifiedrealdatacollectionscenariosareusedtotesttheproposedalgorithm’sperformanceinqualityestimation.


Overview of potential forensic analysis of an Android smartphoneR.Creutzburg,K.Kröger,S.Sack,FachhochschuleBrandenburg(Germany)

ThispapergivesanoverviewofpotentialforensicanalysesofanAndroidSmartphone.Thesecurityconceptandtheunderlyingarchitectureoftheoperatingsystemareintroducedandforensicallyspecificfeaturesofimportancearepresented.WiththehelpofthesepreconsiderationsandtakingtheexampleofaHTCDesireSmartphone,aforensicanalysiswillbecarriedout.TheanalysisexplainscertainfeaturesoftheAndroidoperatingsystemsuchasthefunctionalityofthelockscreen,thedifferencebetweenrootandunrootandtheuseofthedebugmode.ThepracticalstudyshowssomeofthemethodsofhowtoextractinformationfromtheSmartphone.ThiswillincludetheforensicanalysisoftheSDCardandthelogicalanalysisofSmartphonesusingprofessionalforensicsoftwaresuchasOxygenForensicSuite2011.ThecreationofabitbybitimageusingtheAndroidSDKandtheUNIXddcommandispartoftheanalysisaswell.


Conference 8304A8304A-22, Poster Session

Forensics of geodata collected by Apple iOS and Google Android in mobile devicesK.Kröger,R.Creutzburg,FachhochschuleBrandenburg(Germany)

Thispapergivesanoverviewoftheusedtechnologies,theextractionandtheanalysisofstoredgeodatacollectedbyiOSandAndroidinmobiledevices.

Thestudyshowshowstoreddatacanbeextractedandexamined.

ThemainaspectofthestudyistheanalysisofthedifferencesbetweeniOSandAndroidandpossibilitiesfortheaditionaluseoftheextracteddata.


Template-based mobile platform image processing trainingD.Akopian,S.C.Golagani,M.Esfahanian,TheUniv.ofTexasatSanAntonio(UnitedStates)

Noabstractavailable


Combining associative computing and distributed arithmetic methods for efficient implementation of multiple inner productsD.Guevorkian,TampereUniv.ofTechnology(Finland);P.Liuha,T.Yli-Pietilä,NokiaResearchCtr.(Finland);K.O.Egiazarian,TampereUniv.ofTechnology(Finland)

Manymultimediaprocessingalgorithmsaswellascommunicationalgorithmsimplementedinmobiledevicesarebasedonintensiveimplementationoflinearalgebramethods,inparticular,implyingimplementationofalargenumberofinnerproductsinrealtime.AmongmostefficientapproachestoperforminnerproductsaretheAssociativeComputing(ASC)approachandDistributedArithmetic(DA)approach.Inthiswork,weproposeacombinationofthesetwopowerfulmethodsthatfurtherimprovestheefficiencyofmultipleinnerproductcomputation.


Presentation of forensically interesting Microsoft XBox 360 console featuresS.Luttenberger,K.Kröger,R.Creutzburg,FachhochschuleBrandenburg(Germany)

ThispaperdealswithforensicallyinterestingfeaturesoftheMicrosoftXBOX360gameconsole.

Theconstructionandtheinternalstructureareanalysedmoreprecisely.Oneofthemainaspectsofthestudyistoanalysetheusedfilesystemwhichwasexaminedforforensicfeatures.

Possibledifficultiesthatmightbeofimportancetotheforensicinvestigatorarediscussed.


Presentation of forensically interesting Sony Playstation 3 console featuresK.Kröger,G.Daugs,R.Creutzburg,FachhochschuleBrandenburg(Germany)

ThispaperdealswithforensicallyinterestingfeaturesoftheSony

Playstation3gameconsole.

Theconstructionandtheinternalstructureareanalysedmoreprecisely.Interestingforensicfeaturesoftheoperatingsystemandthefilesystemarepresented.

DifferencesbetweenaPS3withandwithoutjailbreakareintroducedandpossibleforensicattemptswhenusinganinstalledLinuxarediscussed.


A neural network-based approach for recognition of engraved and embossed labels on metallic partsA.Shirkhodaie,V.K.Bandaru,TennesseeStateUniv.(UnitedStates)

ConventionalOpticalCharacterRecognition(OCR)approacheshaveproventobelesseffectiveforrecognitionofengravedandembossed(E2)characters.Theirweaknessesare,particularly,paramountininspectionofengravedandembossedcharactersintroducedonmetalcastparts.Duetopoorillumination,lowsignal-to-noise,charactersfont,size,spacing,color,andimpressionvariations,andotherinevitablemanufacturingirregularities(e.g.,poorstamping,mis-stamping,orpartialstamping)inintroducingthesecharacters,mostOCRareimpracticaltodetectandrecognitionmanufacturedpartslabelscorrectly.Inthispaper,weinitiallyaddresstheissuesofcharacterssegmentationfortheirnoisybackgroundanddescribestrengthandweaknessofconventionalimagethresholdingtechniquesforpropersegmentationofforegroundandbackgroundimages.Next,wediscussthreecompetingapproachesforextractionofE2characterswithlowcontrastrelativetothebackground.Forrecognitionofextractedcharacters,wetrainedaHammingNeuralNetwork(HNN).Wedemonstratethattheproposedtechniqueneuralnetwork-basedapproachcaneffectivelyandefficientlyrecognizeE2characterswithhighconfidenceandisfaulttolerantwithrespecttocharacters’font,size,color,andspacingaswellasinvariantofnominalmanufacturingpositionandorientationinconsistencyofE2characters.Resultsfromseveralexperimentsarepresented.AperformancecomparisonofproposedapproachversusaconventionalOCRtechniqueispresentedfordemonstrationofefficiencyandeffectivenessoftheproposedapproach.


A fuzzy-logic approach for metallic parts surface defects characterization and shape classificationA.Shirkhodaie,F.Vaziribozorg,TennesseeStateUniv.(UnitedStates)

RecentadvancesinAutomaticSurfaceInspectionSystem(ASIS)enableautomaticandsystematicinspectionofqualitymanufacturedproducts.Automaticvisualinspectionsystemsrelyonaccuracyofvisualinspectionalgorithmsandtechniquestoreliablydetectandcharacterizethesurfaceimperfectionsanddefects.Inthispaper,anewapproachfordefectdetectionandshapecharacterizationofmetallicpartsisproposed.Thenewtechniqueistrainableandbasedonafuzzylogicmodelwhichperformsshapeclassificationofdefectiveregions.Primarily,thealgorithmdetectsdefectiveregionsofeachpartbasedonanewthresholdingtechnique.Imagestatisticalpropertiesarefurtherusedtodiscriminatebetweennormalanddefectiveregions.Onceadefectiveregionisdetected,themamulti-levelthresholdingisappliedtoacquiretheshaperepresentationofthedefectivearea.Upontheshapeofdefectiveareaisdetermined,then,shapespatialfeaturesarecomputedandusedasinputtothefuzzylogicshapeclassifier.Bycombiningqualitativeandquantitativemeasures,then,extendofshapevariationnormalityordefectivenessisdetermined.Theproposedtechniqueistestedfordefectdetectionandshapecharacterizationofdifferenttypesofmetallicparts.Incomparisonwithotherknowntechniques,thisnewapproachhasimprovedefficiencyandeffectivenessintermsofdetectingand


characterizationvarietyofdefectivesurfacesproperlywithhighdegreeofconfidence.


Sudoku substitution-permutation image cipherY.Wu,TuftsUniv.(UnitedStates);S.S.Agaian,TheUniv.ofTexasatSanAntonio(UnitedStates);J.P.Noonan,TuftsUniv.(UnitedStates)

Withthedevelopmentofdigitaltechnologies,thedigitalimagebecomesacommondatatypeinmodernlife.Comparetowell-developedciphers,likeDES,IDEA,AES,theimageciphersarestillimmatureinmanyaspects:1)ciphersecurityinlackoftheoreticalsupports;2)cipherperformanceislimitedtoseveralciphertext-onlymeasurements;and3)cipherisnotdefinedonthefiniteprecisionsystem.Inthispaper,weintroduceanewimagecipherusingtheSudokuSubstitution-Permutation(SSP).ThecipherefficiencyisguaranteedbytheparameterizedfastSudokutransform,whichexpandsakeydependentsequenceoflengthtoasquareSudokumatrixofsizeandthenprovidesareferencetofuturesubstitutionandpermutationprocess.TheciphersecurityisendorsedbytheSubstitution-PermutationNetwork(SPN)whichisawell-knownstructureincryptographywithgoodconfusionanddiffusionproperties.Moreover,theencryptionqualityisalsoenhancedbyusingSudokumatrix,whichistypeofpuzzlematrixwithnorepeateddigitsinanyrow,columnorblock.Simulationresultsshowthattheproposedcipheroutperformsmanyexistingimageciphers/encryptionalgorithmsinbothqualityandefficiency.

8304A-01, Session 1

Location-aware gang graffiti acquisition and browsing on a mobile deviceA.ParraPozo,M.Boutin,E.J.DelpIII,PurdueUniv.(UnitedStates)

Inthispaper,weproposeamobile-basedsystemthatallowsfirstresponderstoidentifyandtrackgangactivity,combiningtheuseofimageanalysisandlocation-based-services.Theimageanalysisincludesobtainingthemetadata(geoposition,dateandtime)andextractingrelevantfeatures(e.g.,color,shape)fromtheganggraffitiimage.Thedataobtainedautomaticallyonthedeviceistransferredtoaserverandcomparedagainstadatabaseofgraffitimages.Thematchedresultsaresentbacktothedevicewheretheusercanthenreviewtheresultsandprovideextrainputstorefineinformation.

Thefirstrespondercantakeadvantageofthelocation-based-servicesthatthemobiledeviceprovides.Alongwiththeuseofmapprojectionsoraugmentedrealitytechniques,allthegraffitidatainacertainradiusfromthecurrentlocationcanbepulledfromtheserver.Thedataincludesnotonlytheimages,butalltheinformationrelatedtoit,suchasdateandtime,geoposition,gang,gangmember,colors,orsymbols.Theusercanthenvisuallizethedataintheareainausefulmanner,andtakethenecessaryactions.Therefore,theprocessofidentifyingandtrackinggangactivityismademoreefficient,leadingtoafasterinterventionbythelawenforcementofficers.

8304A-02, Session 1

Dietary intake assessment using integrated sensors and softwareJ.Shang,E.Johnson,K.Sundara-Rajan,A.Teredesai,Univ.ofWashington(UnitedStates);A.Kristal,FredHutchinsonCancerResearchCtr.(UnitedStates);A.V.Mamishev,Univ.ofWashington(UnitedStates)

Theareaofdietaryassessmentisbecomingincreasinglyimportantasobesityratessoar,butvalidmeasurementofthefoodintakesinfree-livingpersonsisextraordinarilychallenging.Traditionalpaper-based

dietaryassessmentmethodshavelimitationsduetobias,userburdenandcost,soimprovedmethodsareneededtoaddressimportanthypothesesrelatedtodietandhealth.Inthispaper,wewilldescribetheprogressofourmobileDietDataRecorderSystem(DDRS),whereanelectronicdeviceisusedforobjectivemeasurementondietaryintakeinrealtimeandatmoderatecost.TheDDRSconsistsof(1)amobiledevicethatintegratesasmartphoneandanintegratedlaserpackage,(2)softwareonthesmartphonefordatacollectionandlasercontrol,(3)analgorithmtoprocessacquireddataforfoodvolumeestimation,whichisthelargestsourceoferrorincalculatingdietaryintake,and(4)databaseandinterfacefordatastorageandmanagement.Theestimatedfoodvolume,togetherwithdirectentriesoffoodquestionnairesandvoicerecordings,couldprovidedietitianswithmorecompletefooddescriptionandmoreaccuratefoodportionsizes.Inthispaper,wewilldescribethesystemdesignofDDRSandinitialresultsofdietaryassessment.

8304A-03, Session 1

FCam for multiple camerasA.Troccoli,NVIDIACorp.(UnitedStates);C.Zhou,ColumbiaUniv.(UnitedStates);K.A.Pulli,NVIDIACorp.(UnitedStates)

Photographywasneverjustaboutrecordinganobjectivereality,butacreativetask.Ascamerashavemovedawayfrompaperandchemicalstodigitalimaging,theprocessingbecameeasiertocontrolandfaster.Thelatesttrendistomoveallthatcomputationintocamerasthatcapture,merge,andprocessimagesdigitally,yieldingthefinalimagerightawayinthesamedevice.ThisisfacilitatedbyflexiblecameracontrolAPIssuchastheFCamAPI,whichimplementstheFrankenCameraimagingarchitecture.

ThefirstversionofFCamsupportedonlyasinglecamera.However,manymodernmobiledeviceshaveatleasttwocameras,onenexttothedisplayfacingtheuser,andtheotherpointingawayfromtheuser.Somedevices,suchastheNVIDIA’sTegra3developmentboard,haveevenastereocamerapair.WehaveextendedFCamAPItoexplicitlysupportseveralcameras.Thefacilitiesincludequeryingthenumberandpropertiesofthecameras.Thepropertiesincludesomeobviousones,suchascameraresolutions,butalsoindicationofthegeometricconfigurationofthecameras,fromwhichtheprogramcandeducethecameraorientation,stereobaseline,etc.Wehavealsocreatedaconvenienceclassthatallowstreatingastereocamerapairlogicallyasasinglesensor.

8304A-04, Session 2

Biosensing mobile display principle for healthcareW.Mphepö,iVorexAB,Borlänge(Sweden)andBeijingNormalUniv.(China)

Convergenceandtheabilitytoharnessintrinsicpropertiesofsomedisplaytechnologiescanopenupnewdimensionstotheubiquitousmobiledevices.BiosensingmobiledisplaysbeingonesuchdimensionweherebypresentbasedonourDigitalMicroHinge[DMH]Displaytechnologyresearch.Interestinsuchbiosensingdevicesisobviousfromtherecentlyannounced$10millionQualcommTricorderXPrizecompetition.

8304A-05, Session 2

Continuously adjustable Pulfrich spectacles for mobile devicesK.M.Jacobs,BinghamtonUniv.(UnitedStates);R.S.Karpf,Consultant(UnitedStates)

Mobiledevicespresentachallengingplatformfor3-Dvideobecauseofinherentdevicelimitations.ContinuouslyAdjustablePulfrichSpectacles(CAPS)isanewimplementationofthePulfrich3-Dstereoscopiceffect.Foreveryscenethatcontainslateralmotionina2-Dmovie,CAPS

Conference 8304A


providesrealistic3-D.Sinceitrequiresminimaladditionalprocessing,itisappropriateformobiledevices.

3-DmoviesutilizingthePulfrichstereoscopiceffecthavebeenmadefor80yearsusingpassiveviewingspectacles.CAPSuseactiveviewingspectaclestoovercomepassiveglasses’limitations.3-Dmoviesnormallyemploytheasymmetryofdualimagestoproducestereopsis.CAPSworksontheprincipleofilluminationasymmetry,andonlyneedstocontrolthedifferentiallensopticaldensities.

CAPSarefabricatedfromoptoelectronicmaterialsthatelectronicallycontrolthelensopticaldensities.Theeye’sretinaltriggeringisusedbyCAPStodeterminethedifferentiallensopticaldensities.Motionestimationcalculationsfromthedigitalimageprocessingusedtodisplay2-Dvideoonmobiledevicesarereusedtocalculatereal-timelensadjustmentssoCAPSalwaysconformtotheopticaldensitythatoptimizesthePulfrichstereoscopiceffect.

OnlynegligibleadditionalprocessingisnecessaryforCAPStoshow3-Dforeveryscenethatcontainslateralmotioninany2-Dmovie.

8304A-06, Session 2

Parameters of the human 3D gaze while observing portable autostereoscopic display: a model and measurement resultsA.R.Boev,M.Hanhela,A.P.Gotchev,T.Utirainen,S.Jumisko-Pyykkö,TampereUniv.ofTechnology(Finland);M.Hannuksela,NokiaResearchCtr.(Finland)

Wepresentanapproachtomeasureandmodeltheparametersofhumanpoint-of-gaze(PoG)in3Dspace.Wehavedevelopedamethodologywhichallowsustosuppressmostofthemeasurementnoise.WeidentifythreetemporalpropertiesofthebinocularPoG.Thefirstisreactiontime,whichistheminimumtimethatthevisionreactstoastimuluspositionchange,andismeasuredasthetimebetweentheevent,andthetimethePoGleavestheproximityoftheoldstimulusposition.ThesecondistraveltimeofthePoGbetweentheoldandnewstimulusposition.Thethirdisthetime-to-arrive,whichisthetimecombiningthereactiontime,traveltime,andthetimerequiredforthePoGtosettleinthenewposition.

WepresentthemethodforfilteringthePoGoutliers,forderivingthePoGcenterfrombinoculareye-trackingdataandforcalculatingthegazevolumeasafunctionofthedistancebetweenPoGandtheobserver.Weshowthemeanvaluesforalltemporalpropertiesseparatelyforx,yandzdirectionaveragedoverallobservers.

8304A-07, Session 2

Deblocking of mobile stereo videoA.P.Gotchev,L.Azzari,K.O.Egiazarian,TampereUniv.ofTechnology(Finland)

Standardblocktransform-basedcompressionmethodsoftencauseblockingartefacts,whichhavebeenfoundparticularlyannoyingandalsodegradingtheoverallqualityandtheperceptionofdepth.Suitabledeblockingisrequiredatthereceiversidetotacklesuchartefacts.Currenttrendinrestorationalgorithmssuggestsapplyingnon-localcollaborativefilteringmethods.Inthiscontribution,suchatechniqueisproposedfordeblockingofstereovideocompressedbySimulcastormulti-viewcoding(MVC).Thealgorithmsearchesforsimilarpatchesexhibitinghighspatialcorrelationalongtemporaldimensionandbetweenthetwoviews.Theselectedpatchesaregroupedintocomprehensive4Dstructure,whichisdecorrelatedbyefficienttransforms,namelyDCTalongspatio-temporaldirectionsandHaarwavelettransformbetweenstereoframes.Artefactsuppressionisperformedthroughtransform-domainthresholdingtoresultinfirstempiricalestimateofthedeblockedsignalwhichisthenusedforasecond-stagetransform-domainWienerfiltering.Furthermore,anelegantstereosharpeningcanbeaccomplishedinthetransformdomainbyalpha-rooting.Wedemonstratethattheprocedureleadstovisuallypleasantresultsandiscomparesfavourablyagainstitssimplifiedversions,i.e.slidingDCTfilteringandbilateralfiltering.

8304A-08, Session 3

SUPL support for mobile devicesJ.M.Narisetty,A.Soghoyan,M.C.Sundaramurthy,D.Akopian,TheUniv.ofTexasatSanAntonio(UnitedStates)

TheconventionalGlobalPositioningSystem(GPS)receiversarewellknownpositioningtoolsinopen-skyenvironments.Buttheirperformancedegradesinurbancanyons,indoorsandundergroundduetomultipath,foliage,dissipation,etc.Toovercomesuchsituations,extensiveresearchhasbeendonebyengineersinrecentyearsresultinginanapproachknownasAssistedGPS(A-GPS).Usingthisapproach,orbitalparameterssuchasephemeris,almanac,referencetimeandcoarselocationinformationisprovidedtotheGPSreceiverswhichsignificantlyenhancestheacquisitionofweaksignals.TotestA-GPSenabledreceivershigh-endsimulatorsareused,astheyinvolveadditionalassistancedatageneration,GPSsignal,positioningmeasurements,acquisitionofcoarselocationanddeliverytotargetdevices.Typicallysuchsimulatorsarenotaffordablebymanyacademicinstitutions.NationalInstruments(NI)GPSSimulationToolkitwithNIRFhardwaresupportprovidesanefficienttestingenvironmentforA-GPSenabledreceivers.ThispaperdescribesaneconomicpotentialAGPSreceiversimulatedonNILabVIEWplatform.TheA-GPSsimulatorsupportisprovidedtothemobiledevicethroughWi-FitechnologyusingPackedEncodingRules(PER).ThecommunicationbetweenthesimulatorandthereceiverisinaccordancewiththeSecureUserPlaneLocation(SUPL)protocolencapsulatedwithRadioresourcelocationservices(LCS)protocol(RRLP)appliestoGSMandUMTSCellularNetworks.

8304A-09, Session 3

Measuring ionizing radiation with a mobile deviceM.Michelsburg,T.Fehrenbach,F.PuenteLeón,KarlsruherInstitutfürTechnologie(Germany)

Incasesofnucleardisastersitisdesirabletoknowone’spersonalexposuretoradioactivityandtherelatedhealthrisk.Usually,Geiger-Muellertubesareusedtoassessthesituation.Fittingeveryonewithsuchadeviceinashortperiodisveryexpensive.Weproposeamethodtodetectionizingradiationwithamobileconsumerdevice,e.g.,acellphone,usingitsintegratedcamera.Inemergencycases,millionsofexistingmobiledevicescouldthenbeusedtomonitortheexposureofitsowners.IncombinationwithinternetaccessandGPS,measureddatacanbecollectedbyacentralservertobuildanoverviewofthesituation.

Duringameasurement,theCCDorCMOSsensorofamobiledeviceisshieldedfromsurroundinglightthroughanattachmentinfrontofthelensoraninternalshutter.Thehigh-energyradiationproducesfreeelectronsonthesensorchipwhichthenareanalyzedandseparatedfromthesensornoisebyusinganadaptivethresholdonthebrightnessofthepixels.Withradioactivesourcespresentsignificantincreasesindetectedpixelscanbeseen.Byadaptingexposureandintegrationtimesawidevarietyofradioactiveintensitiescanbemeasured.Furthermore,thesoftwarecanmakeapreliminaryestimateonthecollecteddoseofanindividualandassociatedhealthrisks.

8304A-10, Session 3

Design and evaluation of security multimedia warnings for children’s smart phonesJ.Fruth,S.Tuchscheerer,Otto-von-Guericke-Univ.Magdeburg(Germany)

Inthisarticleweintroduceachildrenspecificsecurity-warningdesignapproachforsmartphones.Wefocusonthosedailydual-taskscenarios,whereauseriswarnedbyitsantivirusapplicationonitssmartphone,whileheisbusywithanothertask.

Conference 8304A


Smartphonesaremoreandmoreusedbychildren.Securitywarningsofanti-virusapplicationsonsmartphonesaredesignedtoinformuseraboutdetectedmaliciouscodes.Inouropinioncommonsecuritywarningsofthoseapplicationsaredesignedforstandardusers.Theyarenotadequatelyadaptedtochildren.Inrespecttothis,wedevelopedafirstdesignapproachforsecuritywarningsforprimaryschoolchildren’ssmartphones.Thewarningsaredesignedinacomicstyle,includingdifferentthreatlevels,multi-mediaandtextualinformation.

ThedevelopedsecuritywarningswereimplementedonaniPhone4andevaluatedinanempiricuserstudy,usingbasicconceptsknownfromusabilitytesting.Theevaluationanalysisshows,e.g.thatnearlyallchildrenlikedthewarning’sdesign,butsomechildrenhadproblemsbyinterpretingthemeaningofthewarning.Inthefutureweplanarefinementofourdesignandthepreparationoffurtherstudies.

8304A-11, Session 3

Using Wi-Fi hotspots as an intrusion vector into corporate networksM.Scharsich,F.Holl,FachhochschuleBrandenburg(Germany)

SendingyourdataoverapublicWi-Fi-Hotspotmakesitreadabletoeveryone.ThisisnotjustknownsincetheFireSheep-PluginforFirefox.ToprotectyourdatayoumayusesecurecommunicationtechnicslikeVPN.Butwhatifnottheunencrypteddataisaproblem,buttheHotspotyouuse,whatiftheHotspotisalreadyundercontrolofanattacker?Thiscanbedonewithminimalfinancialeffortandofferstheattackerthepossibilitiestogatherusercredentialsandtodelivermalwareonthevictim’sdevices.Withthesemalwaretheattackerisabletotakecompletelycontroloverthedevice.ThevictimsusageofVPNenableshimtousethesecureconnectiontogetintoanotherwiseprotectednetworkjustbyusingthedeviceasaproxy.ThatwaytheattackerisabletocircumventtheestablishedperimeterprotectionlikeFirewallsandIDSandcangetdeeplyintoacorporatenetwork.

8304A-12, Session 4

Frame rate up-conversion assisted with camera auto exposure informationL.Liang,B.Hung,G.Dane,QUALCOMMMEMSTechnologies,Inc.(UnitedStates)

Manyalgorithmshavebeenproposedfordecodersideframerateupconversion.However,mostofthemarefromvideoencoding/decoding’spointofview.Weproposedtoutilizethevideofrontend3A(auto-exposure,auto-white-balance,auto-focus)toassistframerateupconversion.Inthispaper,wefocusonusingauto-exposureinformationtoassistframerateupconversionprocess.

Intheproposedstrategythevideofrontend3Ainformationispacketizedasthemetadatawhichisattachedtothecorrespondingframeandtransmittedtogetherwiththemainvideobitstreamtothedecodersideforframerateupconversionassistance.

Videofrontendauto-exposurecontrol(AEC)isrequiredtosetupcamerasensortoachieveabrightnesswithinadesiredrange.Oncethebrightnessintherecordedscenechanges,theAECconvergencefunctionisproceededtoreadjustthebrightnesslevelbacktothetolerancerangeofthelumatarget.Theabruptbrightnesschangecouldcausethedifficultyofthemotionvectorcomputationoftheinsertframesduringframerateupconversionprocess.

Intheproposedstrategy,theintermediatereferenceframesareusedtoassisttheinterpolationoftheinsertedframes.Byusingtheauto-exposureinformation,themotionvectorinterpolationoftheinsertedframescanbebasedonthetwoormorereferenceframesthathasthesamebrightnesslevelwhichresultsinmoreaccuracyandlesscompuation.

8304A-13, Session 4

Fused Fibonacci-like (p,q) sequences with compression and barcoding applicationsS.Agaian,J.Garcia,S.S.Abdul-Kafi,J.T.GillIII,StanfordUniv.(UnitedStates)

ADouble-baseNumberSystem(DBNS)hasrecentlybeenintroducedandinvestigated.Thissystemhasbeenshowntohavesomeinterestingandpotentiallyfar-reachingapplicationsindigitalfiltering,encryption,digitalelectronics,andimageenhancement.Inthispaperweintroduceanewconceptofgeneratingparametricnumberrepresentationsbyfusingsystemsusingmultiplicationandadditionoperations.Morespecifically,weintroduceFibonaccilike(p,q)-sequencesanddeterminetheirefficiencyinrepresentingdata.Wedevelopanefficientalgorithmtotestthesparsityofseveralfusednumberrepresentationsystemsandexplorethedualrelationshipbetweensparsityandmemory.Wealsolookattheinitialapplicationsoftheserepresentationsindatacompressionandbarcoding.Simulationresultsarepresentedtodemonstratetheperformanceofthenewclassofsystems.Acomparisonwithcommonlyuseddouble-basenumbersystemsisalsopresented.

8304A-14, Session 4

White synthesis with user input for color balancing on mobile camera systemsS.Srivastava,C.Xu,E.J.DelpIII,PurdueUniv.(UnitedStates)

Inthispaperweextendthecustomwhitebalancingtechniqueavailableonmanyimagingdevicesbyallowingausertospecifyanynumberofarbitrarycolorsinthescene.WederiveaninterpolationtechniquetoassignweightstothearbitrarycolorswhicharethensummedtoestimatetheRGBcorrespondingtoawhitetarget.Weobtaintheuserinputbydisplayingacapturedimagealongsideacolorgridofcommonlyoccurringcolors.Theuserspecifiescolorpairs-patchesinthesceneandcorrespondingveridicalcolorsonthegrid.Wethenusethesepairstoestimatethewhitewithourinterpolationmethod.Thewhitepointthusestimatedisusedtoconstructadiagonaltransformtoestimatethecameraoutputunderadesiredilluminant.

Weobtainencouragingresultsfromtestingourmethodsonimagesacquiredunderseveralilluminationconditions.Ourapproachisverysuitableformobiledevicesbecausemostsuchdevicesareequippedwithonlymoderatelysophisticatedimagingsystemsandourmethodallowsbettercolorcapturewithsmallcommon-senseuserinput.Further,wecanrealizeourmethodonthesedevicessincemanysuchdeviceshavebuilt-intoolsforgraphicaluserinput.Ourmethodcanbeusefulinphotographyandimageanalysisapplications.

8304A-15, Session 4

Detection of Symmetric shapes on a mobile device with application to automatic sign interpretationA.W.Haddad,S.Huang,M.Boutin,E.J.DelpIII,PurdueUniv.(UnitedStates)

Inthispaperwepresentalight-weightmethodforautomaticallydetectingasegmentingshapesthathaverotationalsymmetry,suchasasquareorequilateraltriangle.Thechallengesoflimitedmemory,processingpowerandbatterylifemakemanycommontechniquesunsuitableformobilecomputation.Itisforthisreasonthatwehavedevelopedourlight-weightapproach.Weleveragethen-foldsymmetryofparticularsigns-inthiscaseHazardousMaterialPlacards.Thecoefficientsofthen-thcentralmoment,asdefinedbyHuang,ofagivencomponentcanbeusedtoaccuratelydeterminethesymmetriesofevenanoisycomponent.Thisflexibilityallowsustochooseamethodofsegmentationwhichrequiresnopostprocessingsuchasdilationorerosion,thusminimizingthecomplexityofexistingsegmentationtechniques.

Conference 8304A


Ourapproachhasbeentestedatdistancesofupto250feet,usinganumberofcameraresolutions,withandwithoutanalogzoom.Ourapproachhasbeenshowntobescaleinvariant,translationinvariantandrotationinvariant.Theresilienceofourmethodtowardsthesetypesoftransformationsmakeitagoodchoiceformobilesignalprocessing.TheabilitytoworkwithascaledimageisparticularlyimportantwhendealingwiththechallengesoflimitedRAMforprocessesoftenimposedbymobilesystems.Translationandrotationresilienceisimportantbecausethepositionandorientationofthecamerainspaceisdependentontheuser.

8304A-17, Session 4

Raster image adaptation for mobile devices using profilesR.Rosenbaum,B.Hamann,Univ.ofCalifornia,Davis(UnitedStates)

Withthefocusbeingondigitalimagery,thiscontributionproposesstrategiestocopewiththeheterogeneousviewinghardwareinmobileenvironments.Constrainedsystemresourcesofmostmobileviewingdevicesrequirecontentsthataretailoredtotherequirementsoftheuserandthecapabilitiesofthedevice.Appropriateadaptationofthecontentsisstillanunsolvedresearchquestion.Duetothecomplexityoftheproblem,availablesolutionsareeithertooresource-intensiveorinflexibletobeappliedbroadly.

Theproposedapproachisbasedonscalableimagecompressionandprogressiverefinementaswellasdataanduserprofiles.Ascalableimageiscreatedonceandmultiplyusedfordifferentkindsofdevicesanduserrequirements.Profilesavailableonserversideallowforanimagerepresentationthatisadaptedtothemostimportantresourcesinmobilecomputing:screenspace,computingpower,andthevolumeofthetransmitteddata.Optionsforprogressivelyrefiningcontentstherebyallowforafluentviewingexperienceduringadaptation.Duetoitsflexibilityandlow-complexity,theproposedadaptationapproachisamuchmoregeneralsolutioncomparedtorelatedapproaches.Thisisunderpinnedbyempiricalresultsobtainedbythestatedexperimentsandadiscussionofitsproperties.

Conference 8304A


Conference 8304B: Multimedia Content Access: Algorithms and Systems VIMonday23January2012

Part of Proceedings of SPIE Vol. 8304B Multimedia Content Access: Algorithms and Systems VI

8304B-32, Session 5

Searching through photographic databases with QuickLookC.Cusano,G.Ciocca,R.Schettini,Univ.degliStudidiMilano-Bicocca(Italy);S.Santini,Univ.AutónomadeMadrid(Spain);A.DePolo,F.Tavanti,FratelliAlinari(Italy)

WepresentheretheresultsobtainedbyincludingwithintheframeworkofQuickLookimageretrievalsystem,thatexploitsarelevancefeedbackmechanism,anewimagedescriptorthatwecalledprosemanticfeaturevector.Bycouplingtheprosemanticfeaturesandtherelevancefeedbackmechanism,theusercanmoveinamorerapidandprecisewaythroughthefeaturespacetowardtheintendedgoal.Theprosemanticfeaturesareobtainedbyatwo-stepfeatureextractionprocess.Atthefirststep,lowlevelfeaturesrelatedtoimagestructureandcolordistributionareextractedfromtheimages.Atthesecondstep,thesefeaturesareusedasinputtoabankofclassifiers,eachonetrainedtorecognizeagivensemantic/conceptcategory,toproducescorevectors.Weevaluatedtheefficacyoftheprosemanticfeaturesunderatargetsearchtaskontwodatasets:theBenchathlondatasetandadatasetprovidedbyFratelliAlinariPhotoArchive.

8304B-34, Session 5

Large-scale classification of traffic signs under real-world conditionsL.Hazelhoff,I.M.Creusen,CycloMediaTechnologyB.V.(Netherlands);D.VandeWouw,P.H.N.deWith,TechnischeUniv.Eindhoven(Netherlands)

Traffic-signinventoriesareimportanttogovernmentalagenciesastheyfacilitateevaluationoftraffic-signlocationsandarebeneficialforroadandsignmaintenance.

Theseinventoriescanbecreated(semi-)automaticallybasedonstreet-levelpanoramicimages.

Intheseimages,objectdetectionisemployedtodetectthesignsineachimage,followedbyaclassificationstage,toretrievethespecificsigntype.

Classificationoftrafficsignsisacomplicatedmatter,sincesigntypesareverysimilarwithonlyminordifferenceswithinthesign,ahighnumberofdifferentsignsisinvolvedandmultipledistortionsoccur,includingvariationsincapturingconditions,occlusions,viewpointsandsigndeformations.

Therefore,weproposeamethodforrobustclassificationoftrafficsigns,basedontheBagofWordsapproachforgenericobjectclassification.

Weextendtheapproachwithaflexible,modularcodebooktomodelthespecificfeaturesofeachsigntypeindependently,inordertoemphasizeattheinter-signdifferencesinsteadofthepartscommonforallsigntypes.

Additionally,thisallowsustomodelthefalsedetections.Furthermore,analysisoftheclassificationoutputprovidestheunreliableresults.

Thisclassificationsystemisextensivelytestedontwolargedatasets,containingtheoutputofourredtriangularandbluecirculartraffic-signdetectorsonstreet-levelpanoramicimages,obtainedfromacounty-widedatabase.Theintroductionofthemodularcodebookshowsasignificantimprovementforallthreesets,wherethesystemisabletoclassify99%ofthereliableresultscorrectly.

8304B-35, Session 5

Human action recognition using a Markovian conditional exponential modelA.Velivelli,A.G.Hauptmann,CarnegieMellonUniv.(UnitedStates)

WemodelthesequenceofhumanactionsoperatinganinfusionpumpusingaMarkovianconditionalexponentialmodel.Wedivideeachvideorecordedbyacameraintovideoactionunits.

Avideoactionunitcorrespondstothestartofauniquehumanactionoperationoftheinfusionpumptotheendofthathumanactionoperationofinfusionpump.

WecalculatetheMOSIFTfeaturesofvideoactionunitswhichcombinesthespatialandtemporaldimensionsfromvideos.WevectorquantizetheMOSIFTfeaturesofvideoactionunitusingKmeansclusteringasvideocodebookelements.Weestimatetheconditionalexponentialmodelparametersfromatrainingsetusingmaximumentropyconstraintandusethevideocodebookelementsasmaximumentropyconstraintfeatures.

WeestimatetheparametersoftheMarkovianconditionalexponentialmodelfromatrainingset.

ThisMarkovianconditionalexponentialmodelhas6stateswhichcorrespondtothe6classesofinfusionpumpoperation.

TofindtheoptimalstatesequenceoftheMarkovianconditionalexponentialmodelweusetheViterbialgorithm.

Thisoptimalstatesequencecorrespondstotheclasslabelsequence.

Theinfusionpumpoperationisrecordedfrom4videocameras.Wecalculatetheresultsofclassificationof6classesofinfusionpumpoperationusingtheconditionalexponentialmodelforthe4videocamerasandalsowecalculatetheresultsofofclassificationof6classesofinfusionpumpoperationusingtheMarkovianconditionalexponentialmodelforthe4videocameras.

TheclassificationperformanceoftheMarkovianconditionalexponentialmodelisbetterthantheclassificationperformanceofconditionalexponentialmodel.

8304B-36, Session 5

Human activity discovery and recognition based on state transitions modeling in persistent surveillance systemsA.Shirkhodaie,V.Elangovan,TennesseeStateUniv.(UnitedStates)

Inthispaper,wepresentaStateTransitions-basedtrackingmodelforspatiotemporalHVIpatternsexploitationwithalinktoknownHVIontologies.Morespecifically,thispaperdiscussesdevelopmentofAdaptiveStateTransitionmodelsindiscoveringHVIviaaHiddenMarkovModeling(HMM)technique.Fivecategoriesofstatetransitionsareconsideredincluding:HumanstatetransitionsofObjecthandling,Visibility,Entity-entityrelation,EntityPosturesandKinematics.AnewapproachforrecognizingstaticanddynamicHVIstakingplanindarkenvironmentsarerepresented.Theproposedapproachuseslow-costKinectdepthmapcamerasfordetection,tracking,andcharacterizationofHVIactivities.ExperimentalresultswithKinectcamerasfrombothindoorandoutdoorarepresented.Theproposedmodel,furthermore,generatessemanticmessagesdescribingtheHVIontologyusingZoningofVehicle(ZoV)techniqueandalsodemonstratestheefficiencyinHumanActivityDiscovery&Recognition(HADR)forimprovedsituationalawareness.


Conference 8304B8304B-37, Session 6

Swimmer detection and pose estimation for continuous stroke-rate determinationD.Zecha,T.Greif,R.W.Lienhart,Univ.Augsburg(Germany)

Inthisworkweproposeanovelapproachtoautomaticallydetectaswimmerandestimatehis/herposecontinuouslyinordertoderiveanestimateofhis/herstrokerategiventhatweobservetheswimmerfromtheside.Wedivideaswimmingcycleofeachstrokeintoseveralintervals.Eachintervalrepresentsaposeofthestroke.Weusespecificallytrainedobjectdetectorstodetecteachposeofastrokewithinavideoandcountthenumberofoccurrencespertimeunitofthemostdistinctiveposes(so-calledkeyposes)ofastroketocontinuouslyinferthestrokerate.Weextensivelyevaluatetheoverallperformanceandtheinfluenceoftheselectedposesforallswimmingstylesonadatasetconsistingofavarietyofswimmers.

8304B-38, Session 6

Multi-view face detection based on position estimation on multi-camera surveillance systemC.Huang,NationalKaohsiungUniv.ofAppliedSciences(Taiwan);J.Chou,J.Syu,S.Wang,NationalChiaoTungUniv.(Taiwan)

Inthispaper,weproposeamulti-viewfacedetectionsystemthatlocatesheadpositionsandindicatesthedirectionofeachfacein3-Dspaceoveramulti-camerasurveillancesystem.Tolocate3-Dheadpositions,conventionalmethodsreliedonfacedetectionin2-Dimagesandprojectedthefaceregionsbackto3-Dspaceforcorrespondence.However,theinevitablefalsefacedetectionandrejectionusuallydegradesthesystemperformance.Instead,oursystemsearchesfortheheadsandfacedirectionsoverthe3-Dspaceusingaslidingcube.Eachsearched3-Dcubeisprojectedontothe2-Dcameraviewstodeterminetheexistenceanddirectionofhumanfaces.Moreover,apre-processtoestimatethelocationsofcandidatetargetsisillustratedtospeed-upthesearchingprocessoverthe3-Dspace.Insummary,ourproposedmethodcanefficientlyfusemulti-camerainformationandsuppresstheambiguitycausedbydetectionerrors.Ourevaluationshowsthattheproposedapproachcanefficientlyindicatetheheadpositionandfacedirectiononrealvideosequencesevenunderseriousocclusion.

8304B-39, Session 7

Mobile visual searchR.Grzeszczuk,NokiaResearchCtr.(UnitedStates)

Noabstractavailable

8304B-40, Session 7

Discriminative tag learning at YouTubeG.Toderici,GoogleInc.(UnitedStates)

Noabstractavailable

8304B-41, Session 7

Revisiting K-means quantization and image object retrieval in an industrial contextR.vanZwol,Yahoo!Inc.(UnitedStates)

Inthispresentationwerevisitthek-meansalgorithmasameanstoefficientlyquantizelocalfeaturesforlargescaleimagesimilaritysearch.Weproposeabalancedk-meansclusteringalgorithmthat

allowsustomaintainnearoptimalminimizationofthewithin-clusterdistances,whileatthesametimetobalancethenumberoffeaturesassignedtoeachcluster.Webelievethatbalancedk-meansclusteringisbeneficialinvariouscomputervisionapplications,anddemonstrateonesuchexample:toquantizelocalfeaturedescriptorsforlargescaleimagesimilaritysearch.Wefirstformallydefinethebalancedk-meansclusteringalgorithmandperformextensiveexperimentationonlargeimagedata-setstoanalyzetheimpactintermsofmeansquarederror,balance,andquantizationquality.Moreover,wedemonstratethatabalancedk-meansclusteringleadstoasignificantimprovementinretrievalandsystemperformancewhenappliedtolarge-scaleimagesimilaritysearchbasedonlocalfeaturedescriptors.

8304B-42, Session 7

Multimedia technologies for content creation and consumptionQ.Lin,Hewlett-PackardLabs.(UnitedStates)

Withthegrowthofimagescapturedbycamerasandsmartphones,peopleareincreasinglyusingimage-richcontentintheircommunications.Whilethetraditionalmediaislargelycreatedbyprofessionals,andconsumedbyalargepopulationofpeople,newmultimediatechnologiesaremakingiteasyforordinarypeopletocreateimage-richcontent.Inaddition,newtechnologiesarealsomakingitpossibleforpublishedcontenttobepersonalizedforindividualsbasedontheirinterests.Inthispresentation,wewilldiscusshowmultimediaanalysisandcompositiontechnologiesareusedincontentcreationandconsumptionapplications.


Keyframe generation from cartoon animation using rule-based optical flowP.Tanapichet,N.Cooharojananone,R.Lipikorn,ChulalongkornUniv.(Thailand)

Thispaperproposesanovelmethodtogeneratekeyframesfromcartoonanimationwiththeaimtoimprovethedetailsandaccuracyofcontentsrepresentedbykeyframes.Considerthatgeneraltechniquesonvideosummarizationusuallydropsomeimportantcontentsduetoitsrestrictiononaspectratio;thispaperthusproposesanewmethodusingpanoramatechnologytoaddmoredetailstobeincludedineachkeyframe.Theconceptistomarkthetimecodebasedonshotboundaryandopticalflowdirection.Theperiodoftimebetweeneverytwoconsecutivemarkedtimecodesisusedtoformashotsequencewhichisactuallyasequenceofframes.Theglobalandlocalopticalflowsarealsousedtodeterminehowtoselecttheframesandwhentostitchtheframestogetheraccordingtotherules.Theresultsofthisproposedmethodarekeyframesgeneratedfromvarioustypesofcartoonanimationwhichareoutstandingcomparedtotheircomicadaptations.


Adaptive characterization, tracking, and semantic labeling of human-vehicle interactions via multimodality data fusion techniquesA.Shirkhodaie,V.Elangovan,TennesseeStateUniv.(UnitedStates)

ExploitationsofHuman-VehicleInteractions(HVI)canbebeneficialforPersistentSurveillanceSystems(PSS).Inparticular,bypropercharacterizationofHVI,certainpertinentthreatscanbeprevented.Inthispaper,wepresentanewmodelforsemanticmessagegenerationbasedonfusionofdifferentsensingmodalities.Anontology-basedapproachisproposedforlocalizationandcharacterizationoftraceableHVIactivities.GenerationofHVISemanticmessagesundergoesseriesofrefinementincluding:HumanAttributesrefinement,Object


AttributesRefinement,ActionAttributesrefinement,SpatialrefinementandTemporalrefinement.Fusionofsemanticmessagestakesplaceateachrefinementstagesandtheprotocolusedforsemanticmessagegenerationhadalsobeenaddressedinthispaper.StatetransitionsofHuman-VehicleInteractionsandHumanObjectInteractionshadalsobeendiscussedhere.Furthermore,wehavedemonstratedthatbyfusingimagingandacousticdataatdecisionlevel,ahigherdegreeofconfidencecanberealizedfordescribingtheHVIactivitieswithappropriatetraceability.Thispaperalsopresentstheresultsofourexperimentalwork,anddemonstratesefficiencyandeffectivenessoftheproposedadaptivefusiontechniquetowardsachievementofsemanticmessagesdescribingHVIactivitieswithdifferentlevelofabstractanddetails.

Conference 8304B


Conference 8305: Visual Information Processing and Communication IIITuesday24January2012

Part of Proceedings of SPIE Vol. 8305 Visual Information Processing and Communication III

8305-01, Session 1

A novel distortion model for quadtree coding in high-efficiency video codingB.Lee,S.Ahn,M.Kim,KAIST(Korea,Republicof)

Inthispaper,anoveldistortionmodelbasedonamixtureofmultipleLaplaciandistributionsarepresentedforthetransformcoefficientsofpredictedresiduesinquadtreecoding.TheproposedmixturemodelofmultipleLaplaciandistributionsistestedfortheHighEfficiencyVideoCoding(HEVC)TestModel(HM)withquadtree-structuredCodingUnitandTransformUnit.Theexperimentalresultsshowthattheproposedmodelachievesmoreaccurateresultsofdistortionestimationthanthesingleprobabilitymodels.

8305-02, Session 1

Weighted prediction for HEVCP.Bordes,TechnicolorS.A.(France)

HEVCisthenewvideocodingstandarddevelopedbyISOMPEGandITU-TVCEG.Asotherstate-of-the-artblock-basedinter-predictioncodec,itisverysensitivetoilluminationvariationsin-betweenframes.Tocopewiththislimitation,theweightedprediction(WP)toolhasbeenproposed.AcomparisonoftheperformanceofWPinHEVCandinMPEG-4AVC/H.264iscarriedout.TheefficiencyofWPisverydependentonthequalityoftheestimatedWPparameters.Thedifferentstagesofstate-of-artWPparametersestimatorsarediscussedandanewalgorithmisproposed.Itisbasedonhistogrammatchingwithglobalmotioncompensation.Severaloptionsareevaluatedandcomparisonismadewithotherexistingmethods.

8305-03, Session 1

Impact of video parameters on the DCT coefficient distribution for H.264-like video codersN.Kamaci,G.Al-Regib,GeorgiaInstituteofTechnology(UnitedStates)

Withtheexistingandemergingadvancedvideocodingstandards,videosourcemodelingbecomesmoreimportant.Particularly,thestatisticalpropertiesofthetransformcodedvideoisofgreatimportance.Inthiswork,weexaminetheimpactofvariousencodingparametersonthedistributionofthediscretecosinetransform(DCT)coefficientsforH.264-likevideocoders.WemodelthedistributionoftheframeDCTcoefficientsusingthemostcommonLaplacianandCauchydistributions.Weshowthattheresolution,thequantizationlevelsandthecodingtypehavesignificantimpactontheaccuracyoftheLaplacianandCauchydistributionbasedmodels.Wealsoshowthatthetransformkernel(4x4vs8x8)haslittleimpact.Moreover,weshowthatforthevideosourcesthathavelittletemporalorspatialdetail,suchasflatregions,thedistributionoftheframeDCTcoefficientsresemblesaLaplaciandistribution.Whenthevideosourceexhibitsmoredetail,suchastextureandedges,the

distributionoftheframeDCTcoefficientsresemblesaCauchydistribution.ThecorrelationbetweenthedetailsofthevideosourcetothetwoprobabilitydistributionscanbeusedtofurtherimprovetheestimationofthedistributionoftheframeDCTcoefficients,byusingaclassificationbasedapproach.

8305-04, Session 1

Adaptive loop filter with directional similarity mapping for video codingP.Lai,F.C.A.Fernandes,SAMSUNGTelecommunicationsAmericaInc.(UnitedStates)

Toimprovecodingefficiencyandvisualqualityinastate-of-the-artvideocodec,wepresentanadaptiveloopfilteringdesign,thatcombineslinearspatialfilteringanddirectionalsimilarityfilteringwithamappingfunction.Itexploitsdirectionalfeaturesinvideoframesbyclassifyingblocksinaframebasedonthedirectionoflocalgradients.Toreduceside-informationoverhead,symmetricconstraintsareimposedonthedirectionalfiltercoefficientsaccordingtothegradientdirection.Toemphasispixelsimilarityforexplicitadaptationtoedges,weuseasimplehard-thresholdmappingfunctiontoavoidartifactsarisingfromacross-edgefiltering.Comparedtothestate-of-the-artapproach,ourdesignhasfewerfilters(4insteadof16)andsmallerworst-casefiltersize(7×7ascomparedto7vertical×9horizontal),whileachievingbetterobjectivecodingperformancewith0.2%BD-rategainsandimprovedsubjectivevisualquality,especiallyalongedges.

8305-05, Session 2

Distributed video coding with progressive significance mapW.A.Pearlman,Y.Hu,RensselaerPolytechnicInstitute(UnitedStates)

Adistributedvideocoding(DVC)systembasedonwavelettransformandsetpartitioncoding(SPC)ispresentedinthispaper.Conventionallythesignificancemap(sig-map)ofSPCisnotconducivetoSlepian-Wolf(SW)coding,becauseofthedifficultyofgeneratingasideinformationsig-mapandthesensitivitytodecodingerrors.TheproposedDVCsystemutilizesahigherstructuredsignificancemap,namedprogressivesignificancemap(prog-sig-map),whichstructuresthesignificanceinformationintotwoparts:ahigh-levelsummationsignificancemap(sum-sig-map)andalow-levelcomplementarysignificancemap(comp-sig-map).Thisprog-sig-mapsolvestheabovedifficultiesandthusmakespartoftheprog-sig-map(specifically,thefixed-lengthcomp-sig-map)suitableforSWcoding.Simulationresultsareprovidedshowingtheimprovedrate-distortionperformanceoftheDVCsystemevenwithverysimplesystemconfigurations.

8305-06, Session 2

Improving side information generation using dynamic motion estimation for distributed video codingI.Park,D.W.Capson,McMasterUniv.(Canada)

Anewsideinformationgenerationalgorithmusingdynamicmotionestimationandpostprocessingisproposedforimproveddistributedvideocoding.Multiplereferenceframesareemployedformotionestimationatthesideinformationframegenerationblockofthedecoder.Aftermotionestimationandcompensation,postprocessingisappliedtoimprovetheholeandoverlappedareasonthereconstructedsideinformationframe.Theproposedsideinformationmethodcontributestoimprovethequalityofreconstructedframesatthedistributedvideodecoder.Theaverageencodingtimeofthedistributedvideocodingisaround15%ofH.264intercodingand40%ofH.264intracoding.TheproposedsideinformationbaseddistributedvideocodingdemonstratesimprovedperformancecomparedwiththatofH.264intracoding.


8305-07, Session 2

Directional frame interpolation for MPEG compressed videoC.Zhao,X.Gao,X.Fan,D.Zhao,HarbinInstituteofTechnology(China)

Imageinterpolationisoneofthemostelementaryimagingresearchtopics.Anumberofimageinterpolationmethodshavebeendevelopedforuncompressedimagesintheliterature.However,alotofvideoshavealreadybeenstoredinMPEG-2formatorhavetobetransmittedinMPEG-2formatduetobandwidthlimitation.Theimageinterpolationmethodsdevelopedforuncompressedimagesmaynotbeeffectivewhendirectlyappliedtocompressedvideos,becauseontheonehand,theydonotutilizetheinformationexistedinthecodedbitstreams;ontheotherhand,theydonotconsiderquantizationerror,whichmaybedominantinsomecases.InspiredbythesuccessoftheintrapredictioninH.264/AVCandtheedge-directedimageinterpolationmethods(suchasLAZAandNEDI),weproposeadirectionalframeinterpolationforMPEGcompressedvideo.Intheproposedmethod,8×8intrablocksinIframesarefirstclassifiedtothenineblockdirectionsintransformdomain.Thentheinterpolationoneachblockisperformedalongitsblockdirection.Foreachblockdirection,anoptimalWienerfilteristrainedbasedontherepresentativevideosequencesandthenusedforitsinterpolation.Inthesimilarway,foreachpixelinaninterblockinPorBframes,theinterpolationisperformedalongthedirectionofitscorrespondingreferenceblock.TheexperimentalresultsdemonstratethattheproposedmethodachievesbetterperformancethanthetraditionallinearmethodssuchasBicubicandBilinearandtheedge-directedmethodssuchasLAZAandNEDI,whilekeepinglowcomputationalcomplexitywhichmeetstherequirementofpracticalapplications.

8305-08, Session 2

A fast intra-prediction method for high-efficiency video coding using Hadamard transformY.Kim,GeorgeMasonUniv.(UnitedStates)andElectronicsandTelecommunicationsResearchInstitute(Korea,Republicof)

Forthehighefficiencyvideocompression,intrapredictionmodehasbeenincreasedanditiscomputationallyexpensive.Inthispaper,weproposeafastintrapredictionmodebasedontheestimationofRatedistortioncostusingHadamardtransformandearlycodingunitdetermination.Theproposedmethodreducethecomputationalcomplexitywithcomparableimagequalityandbit-rate..

8305-09, Session 3

Lossless description of 3D range modelsN.Bayramoglu,A.A.Alatan,MiddleEastTechnicalUniv.(Turkey)

Theimprovementsin3Dscanningtechnologieshaveledthenecessityformanagingrangeimagedatabases.Hence,therequirementofdescribingandindexingthistypeofdataarises.Sincearangemodelhasdifferentpropertiescomparedtocomplete3Dmodels,weproposeamethodthatreliesonSphericalHarmonicsTransform(SHT)forretrievingsimilarmodelswherethequeryandthedatabasebothconsistofonlyrangemodels.AlthoughSHT,isnotanovelconceptinshaperetrievalresearchfor3Dcompletemodels,weutilizeitfor2.5Drangeimagesbyrepresentingthemodelsinareciprocalworldobservedfromthecamera.Thedifference,aswellastheadvantageofouralgorithm,isbeinginformationlossless.Inotherwords,theavailableshapeinformationiscompletelyexploitedforobtainingthedescriptor,whereasothermeshretrievalapplicationsutilizingSHT“approximates”theshapethatyieldsinformationloss.Thedescriptorisinvarianttoscaleandrotationsaboutz-axis.TheproposedtechniqueistestedonalargedatabasehavinghighdiversityanditsperformanceoftheproposedmethodissuperiortotheperformanceofpopularD2distribution.

8305-10, Session 3

Reference frame selection for loss-resilient depth map coding in multiview video conferencingB.Macchiavello,C.Dorea,M.Hung,Univ.deBrasília(Brazil)andHewlettPackardLabs.(UnitedStates);G.Cheung,NationalInstituteofInformatics(Japan);W.Tan,Hewlett-PackardLabs.(UnitedStates)

Multiviewvideoin“texture-plus-depth’’formatenablesdecodertosynthesizefreelychosenintermediateviewsforenhancedvisualexperience.Nevertheless,transmissionofmultipletextureanddepthmapsoverbandwidth-constrainedandloss-pronenetworksischallenging,especiallyforconferencingapplicationswithstringentdeadlines.Inthispaper,weexaminetheproblemofloss-resilientcodingofdepthmaps.Ourstrategyistofirstestablishthatdifferentdepthmacroblockshaveverydifferenterrorsensitivitytothereconstructedimages.Wethenselectivelyemployreferencepictureselectiontoprovideofferstrongerprotectionformoreimportantdepthmacroblocks.Methodsforassigningweightstodepthmacroblocksandformodelingdistortionsinacandidatereferenceblockareprovidedtoallowoptimizedselectionofreference.

Preliminaryresultsshowthat(1)errorsindepthmapswithhigherbordercontentyieldssignificantlyhigherdistortioninreconstructedimages,and(2)referencepictureselectionappliedtodepthmapsrequiressignificantlylesspercentageoverheadthantoimages.Bothofthesesuggestthatourschemeispromising.Inourfinalpaper,weplantoimplementthepresentedoptimizationintoH.264,andpresentsimulationresultsundervariousnetworklossconditions.

8305-11, Session 3

Low-complexity automated depth-order estimation for 2D-to-3D video conversionR.Klepko,CommunicationsResearchCtr.Canada(Canada)

Theincreasingpopularityof3DTVcreatesthedesireformore3Dvideocontent.Waitingfortheretobeanabundanceof3Dvideocontentderivedfromstereoscopiccameraswilltaketoomuchtime.However,therecurrentlyexistsavastquantityof2Dvideomaterialthatcanpotentiallybeconvertedto3D.This2D-to-3Dconversionisacomplexprocessandsocanbecostly.Thus,anautomatedsolutionthatcanbeachievedwithlow-complexitywouldbedesirable.Ourpastresearchworkhasalreadyresultedinareal-time2D-to-3Dconversiontechnique,butthiscreatespseudo-3Dandnotaccurate3D.Thus,currentresearch,tobedescribedinthispaper,focusesonimprovingtheaccuracyofthe3Dbyimplementingandincorporatingatechniquecomposedofamulti-stepprocesstodeterminethedepth-orderofobjects,withrespecttothecamera,ineachvideosequenceframe.Themulti-stepprocesscanbesummarizedasfollows:edgedetection;determineifanedgepointbelongstoamovingedgetoidentifytheocclusionboundary;determinewhichoftheleftorrightsideblockregionadjacenttotheedgemoveswiththeedge,andbydeductiondeterminestheorder;segmentaframeandassignblockregionstosegments;clustersegmentsintoobjects;globallyassigndepth-order;andmodifysurrogatedepthmaptocreatemoreaccuratedepthmap.Testresultsshowthatthisisaveryeffectiveandfasttechniqueforderivingthedepth-orderofobjects.

8305-12, Session 3

Block-layer, optimal bit allocation based on constant perceptual qualityC.Wang,X.Mou,Xi’anJiaotongUniv.(China);L.Zhang,TheHongKongPolytechnicUniv.(China)

Bitallocationisakeyissueinimage/videocoding.Wesuggestthatthebitallocationamongthemacroblockscanbeoptimizedbyaimingattheconstantperceptualquality(CPQ)insideanimage/aframe.Based

Conference 8305


ontheMINMAXcriterion,weproposeaCPQbasedblock-layerbitallocationmethod,whichisamulti-passscheme.TheresultsshowthattheCPQcanimprovetheencodingperformanceobviously.

8305-13, Session 4

Patch-wise ideal stopping time for anisotropic diffusionH.Talebi,P.Milanfar,Univ.ofCalifornia,SantaCruz(UnitedStates)

Data-dependentfilteringmethodsarepowerfultechniquesforimagedenoising.Thesealgorithmsaremostlypatch-wise,andalsoemploylocalandnon-localsimilaritiesinthesignals.Alloftheserestorationmethodsworkbasedonthesameframeworkinwhichsomedata-adaptiveweightsareassignedtoeachpixelcontributingtothefiltering.Beginningwithanybaseprocedure(nonlinearfilter),repeatedapplicationsofthesameprocesscanbeinterpretedasadiscreteversionofanisotropicdiffusion.Diffusionfilteringgraduallyremovesnoiseineachiteration,butalsotakesawaylatentdetailsfromtheunderlyingsignal.Choosingasmalliterationnumberpreservestheunderlyingstructure,butalsodoeslittledenoising.Ontheotherhand,alargeiterationnumbertendstoover-smoothandremovenoiseandhighfrequencydetailsatthesametime.Assuch,anaturequestionis“Whatisthebeststoppingtimeiniterativedata-dependentfiltering?”.Thisisthegeneralquestionweaddressinthispaper.Ifweclairvoyantlyhadthemean-squareerror(MSE)functionineachiteration,findingtheoptimalstoppingtimewouldbepossible,asthiswouldcorrespondtotheminimumoftheMSE.Todevelopournewmethod,weestimatetheMSEineachimagepatch.Thisestimateisusedtocharacterizetheeffectivenessoftheiterativefilteringprocess,anditsminimizationyieldstheidealstoppingtimeforthediffusionprocess.

8305-14, Session 4

Video attention deviation estimation using inter-frame visual saliency map analysisY.Feng,G.Cheung,NationalInstituteofInformatics(Japan);P.LeCallet,Polytech’Nantes(France);Y.Ji,NationalInstituteofInformatics(Japan)

Aviewer’svisualattentionduringvideoplaybackisthematchingofhiseyegazemovementtothechangingvideocontentovertime.Ifthegazemovementmatchesthevideocontent(e.g.,followarollingsoccerball),thentheviewerkeepshisvisualattention.Ifthegazelocationmovesfromonevideoobjecttoanother,thentheviewershiftshisvisualattention.Avideothatcausesaviewertoshifthisattentionoftenisa“busy’’video.Determinationofwhichvideocontentisbusyisanimportantpracticalproblem;abusyvideoisdifficultforencodertodeployregionofinterest(ROI)-basedbitallocation,andhardforcontentprovidertoinsertadditionaloverlayslikeadvertisements,makingthevideoevenbusier.Onewaytodeterminethebusynessofvideocontentistoconducteyegazeexperimentswithasizablegroupoftestsubjects,butthisistime-consumingandcost-ineffective.Inthispaper,weproposeanalternativemethodtodeterminethebusynessofvideo---formallycalledvideoattentiondeviation(VAD)---byanalyzingthespatialvisualsaliencymapsofthevideoframesacrosstime.WefirstderivetransitionprobabilitiesofaMarkovmodelforeyegazeusingsaliencymapsofanumberofconsecutiveframes.Wethencomputesteadystateprobabilityofthesaccadestateinthemodel---ourestimateofVAD.Wedemonstratethatthecomputedsteadystateprobabilityforsaccadeusingsaliencymapanalysismatchesthatcomputedusingactualgazetraces.Further,ouranalysiscanalsobeusedtosegmentvideointoshorterclipsofdifferentdegreesofbusynessbycomputingtheKullback-Leiblerdivergenceusingconsecutivecomputedgazemodelstatistics.

8305-15, Session 4

Robust grid registration for non-blind PSF estimationJ.Simpkins,R.L.Stevenson,Univ.ofNotreDame(UnitedStates)

Givenablurredimageofaknowntestgridandanaccurateestimateoftheunblurredimage,ithasbeendemonstratedthattheunderlyingblurkernel(orpoint-spreadfunction,PSF)canbereliablyestimated.Unfortunately,theestimateofthesharpimagecanbesensitivetocommonimperfectionsinthesetupusedtoobtaintheblurredimage,anderrorsintheimageestimateresultinanunreliablePSFestimate.

Weproposearobustad-hocmethodtoestimateasharppriorimage,givenablurry,noisyimageofatestgridtakeninimperfectlabandlightingconditions.Theproposedalgorithmisabletoreliablyrejectsuperfluousimagecontent,candealwithspatially-varyinglighting,andisinsensitivetoerrorsinalignmentofthegridwiththeimageplane.

Wedemonstratethealgorithm’sperformancethroughsimulation,andwithasetoftestimages.WealsoshowthatourgridregistrationalgorithmleadstoimprovedPSFestimationanddeblurring,comparedtoanaffineregistrationusingspatiallyinvariantlightingcorrection.

8305-16, Session 4

Fast pseudo-semantic segmentation for joint region-based hierarchical and multiresolution representationR.Sekkal,C.Strauss,F.Pasteau,M.Babel,O.Déforges,InstitutNationaldesSciencesAppliquéesdeRennes(France)

Inthispaper,wepresentanewscalablesegmentationalgorithmcharacterizedbyregion-basedhierarchyandresolutionscalability.Mostoftheproposedalgorithmsappliedeithermultiresolutionsegmentationfollowingacoarsetofinesegmentationapproachorahierarchicalsegmentationwithfinetocoarsesegmentation.Ourapproachcombinesbothmultiresolutionandhierarchicalsegmentation.Weconsidertheimageasasetofdifferentimagesatdifferentlevelsofresolution,whereateachlevelahierarchicalsegmentationisperformed.Multiresolutionimpliesthatasegmentationofonegivenlevelisreusedinfurthersegmentationprocessoperatedatthenextlevel.EachlevelofresolutionprovidesaRegionAdjacencyGraph(RAG)thatdescribestheneighborhoodrelationshipsbetweenregionswithinonegivenlevelofthemultiresolutionrepresentation.Regionlabelsconsistencyispreservedthankstoadedicatedprojectionalgorithmbasedoninter-levelrelationships.Themainadvantageofourapproachisthelowcomputationalcomplexity,asthesegmentationprocessonlyrequireslowresolutionsoftheimage.Experimentsshowthatwecanobtainveryeffectiveresultscomparedwiththestateofthearttogetherwithlesscomputationtime.

8305-17, Session 5

Optimal local dimming for LED-backlit LCD displays via linear programmingX.Shu,X.Wu,McMasterUniv.(Canada);S.Forchhammer,TechnicalUniv.ofDenmark(Denmark)

LED-backlitLCDdisplaysholdthepromiseofimprovingtheimagequalitywhilereducingtheenergyconsumptionwithsignal-dependentlocaldimming.TofullyrealizesuchpotentialsweproposeanovellocaldimmingtechniquethatjointlyoptimizestheintensitiesofLEDbacklightsandtheattenuationsofLCDpixels.TheobjectiveistominimizethedistortioninluminancereproductionduetotheleakageofLCDandthecoarsegranularityoftheLEDlights.Theoptimizationproblemisformulatedasoneoflinearprogramming,andbothexactandapproximatealgorithmsareproposed.Simulationresultsdemonstratesuperiorperformancesoftheproposedalgorithmsovertheexistinglocaldimmingalgorithms.

Conference 8305


8305-18, Session 5

Gestures for natural interaction with videoN.Fourati,E.Marilly,Alcatel-LucentBellLabs.Villarceaux(France)

Inthecontextofimmersivecommunicationsthataimtoenablenaturalexperiencesandinteractionsamongpeople,objects,andenvironment,weproposeamethodenablingnaturalvideointeractionsthroughhandgesturerecognitionbetweenusersandavideomeetingsystem.Anend-to-endstudyisdonestartingfromthegesturerecognitionalgorithmstotheuserevaluationforthevalidation.

Identificationofgesturestorecognizeandtheirassociatedfunctionalitieshasbeendonefromausersurvey.Inthissurvey,twoconcepts,thatareoftenconfused,havetobedistinguished:handpostureandhandgesture(i.e.staticversusdynamic).

Therefore,ourhandgesturerecognitionprocessiscomposedof2maintasks:thehandposturerecognition(i.e.skinsegmentation,backgroundsubtraction,regionscombination,featuresextractionandclassification)andthehandgesturerecognition(trackingandrecognition).Inthisapproach,wehighlighttheinteresttocombineasignalsimilaritystudywithadataminingtoolfordynamicgesturerecognition.Wehavealsocombinedcolorandmotionscuestoavoidthedetectionofstaticobjects.

Ourperspectivesareconcentratedontheexperimentationanduserevaluationinordertoimproveourapproach,takingintoaccountuser’sfeedbackandperformanceanalysisindifferentenvironmentsandfordifferentusers.

8305-19, Session 5

Improving underwater visibility using vignetting correctionK.Sooknanan,A.Kokaram,TrinityCollegeDublin(Ireland)

Underwatersurveyvideosoftheseafloorareusuallyplaguedwithheavyvignetting(radialfalloff)outsideofthelightsource’sbeamfootprintontheseabed.Thesesurveysareimportantforassessingthebiologicalenvironment.Inthispaperweproposeanovelmulti-frameapproachforremovingthispiecewisevignettingphenomenonwhichinvolvesestimatingthelightsource’sfootprintontheseafloor,andtheparametersforourproposedvignettingmodel.Withinthefootprint,weleavetheimagecontentsasis,whereasoutsidethisregion,weperformvignettingcorrection.Weverifyouralgorithmwithbothsyntheticandrealdata,andthencompareitwithanexistingtechnique.Resultsobtainedshowsignificantimprovementinthebrightnesslevels.

8305-20, Session 5

Defect pixel interpolation for lossy compression of camera raw dataM.Schöberl,Friedrich-Alexander-Univ.Erlangen-Nürnberg(Germany);J.Keinert,Fraunhofer-InstitutfürIntegrierteSchaltungen(Germany);J.Seiler,Friedrich-Alexander-Univ.Erlangen-Nürnberg(Germany);S.Foessel,Fraunhofer-InstitutfürIntegrierteSchaltungen(Germany);A.Kaup,Friedrich-Alexander-Univ.Erlangen-Nürnberg(Germany)

Theimageprocessingpipelineofatraditionaldigitalcameraisoftenlimitedbyprocessingpower.Abetterimagequalitycouldbegeneratedifmorecomplexitywereallowed.Inarawdataworkflowmostalgorithmsareexecutedoff-camera.Thisallowstheuseofmoresophisticatedalgorithmsforincreasingimagequalitywhilereducingcameracomplexity.However,thisrequiresamajorchangeintheprocessingpipeline:alossycompressionofrawcameraimagesmightbeusedearlyinthepipeline.Subsequentoff-cameraalgorithmsthenneedtoworkonmodifieddata.Weanalyzedthisproblemfortheinterpolationofdefectpixels.Wefoundthatalossyrawcompressionspreadstheerrorfromuncompensateddefectsovermanypixels.Thisleadstoaproblemasthislargererrorcannotbecompensatedforaftercompression.Theuseofhighquality,highcomplexityalgorithmsinthe

cameraisalsonotanoption.Weproposeasolutiontothisproblem:Insidethecameraonlyasimpleandlowcomplexitydefectpixelinterpolationisused.Thissignificantlyreducesthecompressionerrorforneighborsofdefects.Wethenperformalossyrawcompressionandcompensatefordefectsafterwards.Thehighcomplexitydefectpixelinterpolationcanbeusedoff-camera.

Thisleadstoahighimagequalitywhilekeepingthecameracomplexitylow.

8305-21, Session 6

Cubic-panorama image dataset compressionS.Salehi,E.Dubois,Univ.ofOttawa(Canada)

ThisworkispartoftheNAVIREprojectattheUniversityofOttawawhichaimsatdevelopingthenecessarytechnologytoallowausertovirtuallywalkthroughinanimage-basedrepresentationofaremoteenvironment.Ourgoalistoproposeanefficientmethodtocompresssuchhighvolumeofinformation,keepingthedesiredrate,distortion,andrandomaccessrequirementsinmind.Aftertheacquisitionandpreprocessingstage,agroupofsixrawfullRGBimagescanbeusedtocreateabasispanoramawhichconsistsofsixsideimagesincubicformat.Followingtheimagedatasetanalysisstage,visualdatawillbereadyforcompression.Weapplyandcomparethetwomajorexistingapproachesfromtheliterature,i.e.,astandardizedmethodbasedonH.264/MPEG4AVCandanexistingwaveletbasedmethodcalledDirac.Theproblemofindexingisaddressedconsideringthecompressionefficiency,randomaccess,andotherrequirementsofourapplication.AdvantagesofusingBframesareshown.Basedontheabovementionedconsiderationsanappropriatebitstreamsyntaxisintroduced.DisparityestimationwhichplaysanimportantroleinanycompressionschemeisgivenspecificattentionandanefficientmethodbasedonourpreviousworkonEpipolargeometryispresented.

8305-22, Session 6

Lossless halftone image compression using adaptive context template updateS.Park,D.Choi,J.Yoon,SamsungElectronicsCo.,Ltd.(Korea,Republicof)

Inelectronicprintingsystem,manyresearcheffortshavebeenmadetocompactlyexpresshalftoneimages,basedonacontext-basedbinaryarithmeticcoding.Forexample,inthestandardJBIG,10bit-sizedcontexttemplatewasusedtoreduceconditionalentropy.Also,atemplatemodificationschemewaspresented,whichusestheautocorrelationofinputimage.Combinedwith12bit-sizedcontexttemplates,ReavyandBonceletpresentedablockarithmeticcoding(BAC),whichusesfixedlengtharithmeticcodes.Moreover,acontextweightingschemewasintroducedtofindthebestcontextmodelamongseveralcontexttemplates.

Inthiswork,anadaptivecontexttemplatedesignschemeispresentedforefficienthalftoneimagecompression.Aslarge-sizedtemplatesareprohibitiveforhardwareimplementation,theproposedschemekeepsthetemplatesizeassameasthestandardJBIG.Also,theproposedschememodifiesthetemplateshapeineachpixeltraversal.Specifically,thetemplateischangedbyselectingbinarypixelshavinghighcorrelationstothecurrentpixel.TheproposedschemeoutperformstheconventionalJBIGwith29%bitsaving,whileusingonly1,024contextconditionsassameastheJBIG.

8305-33, Session 7

Recognition of sport players’ numbers using fast-color segmentationC.Verleysen,C.DeVleeschouwer,Univ.CatholiquedeLouvain(Belgium)

Conference 8305


Thispaperbuildsonapriorworkforplayerdetection,andproposesanefficientandeffectivemethodtodistinguishamongplayersbasedonthenumbersprintedontheirjerseys.Toextractthenumbers,thedominantcolorsofthejerseyarelearntduringaninitialtrainingphaseandusedtospeedupthesegmentationofthecandidatedigitregions.Anadditionalsetofcriteriaconsideringtherelativepositionandsize(comparedtotheplayerboundingbox)andthedensity(comparedtothedigitrectangularsupport)ofthedigitareusedtofilterouttheregionsthatobviouslydonotcorrespondtoadigit.Oncetheplausibledigitregionshavebeenextracted,theirrecognitionisbasedonfeature-basedclassication.Anumberoforiginalfeaturesareproposedtoincreasetherobustnessagainstdigitappearancechanges,resultingfromthefontthicknessvariabilityandfromthedeformationsofthejerseyduringthegame.Finally,theefficiencyandtheeffectivenessoftheproposed

methodaredemonstratedonareal-lifebasketballdataset.Morethan50%ofthejerseysamplesthatcanbevisuallyrecognizedaredetectedasnumberswith93%ofcorrectclassication.

8305-34, Session 7

On the use of clustering for resource allocation in wireless visual sensor networksA.V.Katsenou,L.P.Kondi,K.E.Parsopoulos,Univ.ofIoannina(Greece)

Inthepresentpaper,wefocusontheproblemofquality-drivencross-layeroptimizationofDirectSequenceCodeDivisionMultipleAccess(DS-CDMA)wirelessvisualsensornetworks(WVSNs).Weconsideracentralizedtopology,whereeachsensortransmitsdirectlytothecentralizedcontrolunit(CCU),whomanagesthenetworkresources.Inrealenvironments,thevisualsensorsviewandtransmitsceneswithvaryingmotionamounts,thuseachrecordedvideohasitsindividualmotioncharacteristics.OuraimistoenabletheCCUtoallocatejointlythetransmissionpowerandsource-channelcodingratesforeachWVSNnode,undercertainquality-drivencriteriaandtheconstraintofaconstantchiprate.Weconsidertwoapproachesforthecross-layeroptimizationscheme.Inthefirstapproach,theoptimalsetofnetworkresourcesisassignedtoeachnodeaccordingtoitsindividualmotioncharacteristics.Inthesecondapproach,thenodesarepartitionedintoclustersaccordingtotheamountofmotionintherecordedscenes,andthesamenetworkresourcesareassignedtoallnodeswithinacluster.Theresultingmixed-integeroptimizationproblemissolvedusingtheParticleSwarmOptimization(PSO)algorithm.Theexperimentalresultsdemonstratethequality/complexitytradeoffforthetwoapproaches.

8305-35, Session 7

Kalai-Smorodinsky bargaining solution for optimal resource allocation over wireless DS-CDMA visual sensor networksK.Pandremmenou,L.P.Kondi,K.E.Parsopoulos,Univ.ofIoannina(Greece)

Surveillanceapplicationsusuallyrequirehighlevelsofvideoquality,resultinginhighpowerconsumption.Theexistenceofawell-behavedschemetotradeoffvideoqualityandpowerconsumptioniscrucialforthesystemperformance.Inthiswork,weadoptthegametheoreticapproachofKalai-SmorodinskyBargainingSolution(KSBS)todealwiththeproblemofoptimalresourceallocationinamultinodewirelessvisualsensornetwork(VSN).TheDirectSequenceCodeDivisionMultipleAccess(DS-CDMA)methodisusedforchannelaccess.Moreover,across-layeroptimizationdesign,whichemploysacentralprocessingserver,accountsfortheoverallsystemefficacythroughallnetworklayers.Thetaskassignedtothecentralserveristhecommunicationwiththenodesandthejointdeterminationoftheirtransmissionparameters.TheKSBSisappliedtonon-convexutilityspaces,distributingthesourcecodingrate,channelcodingrateandtransmissionpowersfairlyandefficientlyamongthenodes.Thetransmissionpowerscanbeassignedcontinuousvalues,whereasthesourceandchannelcodingratescanonlytakediscretevalues.

Investigatingtheprovidedexperimentalresults,wedemonstratethemeritsofKSBSovercompetingfairnesspolicies.

8305-36, Session 7

State-of-the-art lossy compression of Martian images via the CMA-ES evolution strategyF.W.Moore,B.Babb,Univ.ofAlaskaAnchorage(UnitedStates);S.Aldridge,TheUniv.ofSouthernCalifornia(UnitedStates);M.R.Peterson,Univ.ofHawai’iatHilo(UnitedStates)

TheresearchdescribedinthispaperusestheCMA-ESevolutionstrategytooptimizematchedforwardandinversetransformpairsforthecompressionandreconstructionofimagestransmittedfromMarsroversunderconditionssubjecttoquantizationerror.Ourbesttransformsoutperformboththeintegerandfloating-pointimplementationsofthe2/6wavelet,substantiallyreducingerrorinreconstructedimageswithoutallowingincreasesincompressedfilesize.Thisresultestablishesanewstate-of-the-artforthelossycompressionofimagestransmittedoverthedeep-spacechannel.

8305-37, Session 7

Spatially adaptive superresolution using the optimal recovery frameworkA.J.Shaik,S.D.Cabrera,TheUniv.ofTexasatElPaso(UnitedStates)

Multi-FrameimagerestorationorSuper-Resolution(SR)istheprocessofcombiningmultipleLow-Resolution(LR)imagesinordertoreconstructasingleHigh-Resolution(HR)image.OurpreviousapproachtoSRbasedontheframeworkofoptimalrecoveryisextendedtoaspatially-adaptiveschemewherebytheblock-by-blockprocessingismodifiedbasedonthepropertiesofthecorrespondinglocalLRimagedata.ThebandwidthparameterisadaptedbasedonthelocalvarianceofeachcorrespondingblockfromallLRimagesthatcontributetotheHRimageblocktobereconstructed.TheoptimalregularizationparameterforthereconstructionofeachHRoutputblockiscalculatedusingtheGeneralizedCrossValidation(GCV)approachchosenbasedonthebandwidthaswellasthecontributinglocalLRdatapixels.Simulationresultsshowthesuperiorityoftheadaptiveschemeoverthefixedoptimalrecoverybasedsuper-resolutionwithandwithouttheuseofoverlappingblocks.Abriefcomparisonwithanotherstate-of-the-artSRschemeisalsoincludedandweobtaincomparableresults.

8305-25, Session 8

A semi-automatic traffic sign detection, classification, and positioning systemI.M.Creusen,L.Hazelhoff,P.H.N.deWith,CycloMediaTechnologyB.V.(Netherlands)andTechnischeUniv.Eindhoven(Netherlands)

Theavailabilityoflarge-scaledatabasescontainingstreet-levelpanoramicimagesoffersthepossibilitytoperformsemi-automaticsurveyingofreal-worldobjectssuchastrafficsigns.Theseinventoriescanbeperformedsignificantlymoreefficientlythanusingconventionalmethods.Governmentalagenciesareinterestedintheseinventoriesformaintenanceandsafetyreasons.Thispaperintroducesacompletesemi-automatictrafficsigninventorysystem.Thesystemconsistsofseveralcomponents.First,adetectionalgorithmlocatesthe2Dpositionofthetrafficsignsinthepanoramicimages.Second,aclassificationalgorithmisusedtoidentifythetrafficsign.Third,the3DpositionofthetrafficsigniscalculatedusingtheGPSpositionofthephotographs.Finally,theresultsarelistedinatableforquickinspectionandarealsovisualizedinawebbrowser.

Conference 8305


8305-27, Session 8

Compression of 2D navigation views with rotational and translational motionD.Springer,Friedrich-Alexander-Univ.Erlangen-Nürnberg(Germany);F.Simmet,D.Niederkorn,AudiAG(Germany);A.Kaup,Friedrich-Alexander-Univ.Erlangen-Nürnberg(Germany)

In-carnavigationsystemshavegrownincomplexityovertherecentyears,mostnotablyintermsofroutecalculation,usabilityandgraphicalrendering.Inordertoguaranteecorrectsystembehavior,navigationsystemsneedtobetestedunderrealoperatingconditions,i.e.withfield-testsontheroad.Inthispaper,wewillfocusonafastandembeddedcompressionsolutionfor2Dnavigationrenderings,sothatfield-testscanbearchivedandhandedovertosoftwareengineersforsubsequentevaluation.NoparametersfromtherenderingprocedureareavailablesinceaccesstothesystemislimitedtotheLVDSdisplaysignal.WeshowhowtoreconstructallrelevantmotionparametersofthesceneswithlowcomputationalcomplexityanddevelopaGlobalMotionEstimation(GME)methodassupportforasubsequentvideoencodingprocess.Byintegratingrate-distortionoptimizationconceptsintoourscheme,wecanefficientlyomitthesegmentationofstaticandnon-staticareas.TheproposedGMEtechniqueisinherentlysuitedforparallelimplementation,whichisdemonstratedonanembeddedOMAP4430SoC.

8305-31, Session 8

Survey of imaging applications used in roadway transportationN.Manikoth,ACS,AXeroxCo.(UnitedStates);R.P.Loce,W.Wu,E.Bernal,XeroxCorp.(UnitedStates)


8305-26, Session 9

Traffic camera markup language (TCML)Y.Cai,CarnegieMellonUniv.(UnitedStates)

Rapidlygrowingtrafficcamerasrevolutionizetransportationmanagementsystems.Theyalsoenableperipheralapplicationssuchasdriver’sinformation,emergencyresponse,andpollutioncontrol.Currenttrafficimagingtechniquesarelackinginstandardizationandflexibility,presentingdifficultiesininterfacingwithothersystemssuchasoptimizationmodelsandmobilephones.Furthermore,mostcamerasignalsarenotregisteredwithreal-timespatiotemporaldatabasesincludingconventionaltrafficsensors(e.g.Radar)andcontemporarysensors(e.g.Bluetooth).

HereIwillpresentanovelvideomarkuplanguageforarticulatingsemantictrafficdatafromsurveillancecameras.Themarkuplanguageincludesthreelayers:trafficflowdescriptions,geographicaldescriptions,andapplicationinterfacedescriptions.Iwillalsopresentcasestudies,includingtheHOVlanesafetysystem,highwayflowmanagementandtrafficsituationnavigationsystem,usingTrafficCameraMarkupLanguage.

8305-28, Session 9

On-board side pedestrian detection for automotive active safety systemR.Cheng,Y.Zhao,X.Wang,J.Xu,S.Lv,PekingUniv.(China)

AutomotiveActiveSafety(AAS)isthemainbranchofintelligenceautomotivestudyandpedestriandetectionisthekeyproblemofAAS,becauseitisrelatedwiththecasualtiesofmostvehicleaccidents.Foron-boardpedestriandetectionalgorithms,themainproblemistobalanceefficiencyandaccuracytomaketheon-boardsystemavailableinrealscenes,soanewsidepedestriandetectionmethod

consideredcommonurbanscenethatpeoplecrossingthestreetisproposed.

Theapproachincludestwosteps,detectionandverification.Indetectionstep,HaarfeatureandacascadeofstageclassifierstrainedbyAdaboostareapplied.Inverificationstep,HOGfeatureandSVMclassifierareusedtorefinefalsepositives.Afterthesetwosteps,getaresultwithbothhighdetectionrateandlowfalsealarmrate.Tospeeduptheprocessingtime,weusedivide-windowmethodtogetherwithOCSalgorithmandscalingmethodindetectionstep.Withanewdatasetusinganonboardcameradrivingthroughurbanenvironmenttoshootsidepedestriansonzebra.Thisapproachperformsanon-boardavailableresultonsidepedestriandetection.Anovelmethodfocusonpedestrian’slegfeatureappliedwithHaarcascadeclassifierandHOGdescriptorsisalsoexplored,italsoshowsgoodperformance.

8305-29, Session 9

Passive detection of heavily laden vehiclesT.McKay,C.Salvaggio,P.S.Salvaggio,J.Faulring,D.M.McKeown,RochesterInstituteofTechnology(UnitedStates);A.J.Garrett,D.Coleman,L.Koffman,SavannahRiverNationalLab.(UnitedStates)

TheDigitalImagingandRemoteSensingLaboratory(DIRS)attheRochesterInstituteofTechnology,alongwiththeDepartmentofEnergyisinvestigatingpassivemethodstoquantifytheweightofheavilyladenvehicles.Thispaperinvestigatesmultiplevehiclesignaturesincludingbraketemperature,tiretemperature,enginetemperature,accelerationanddecelerationrates,engineacoustics,vehiclestability,suspensionresponse,tiredeformationandvibrationalresponseaspossibleindicatorsofvehicleweight.Ourinvestigationintothesevariablesincludesbuildingandimplementingasensingsystemfordatacollectionaswellasmultiplefull-scalevehicletests.Thissensingsystemincludes;infraredvideocameras,triaxialaccelerometers,microphones,videocamerasandthermocouples.Thefullscaletestingincludesbothamediumsizedumptruckandtractor-trailertruckonclosedcourseswithloadsspanningthefullrangeofthevehicle’scapacity.Statisticalanalysisofthecollecteddataisusedtodeterminetheeffectivenessofeachofthesignaturesforcharacterizingtheweightofavehicle.Thefinalsensingsystemwillmonitormultipleheavyvehiclesignaturesandcombinetheresultstoachieveamoreaccuratemeasurementthananyofthesignaturescouldprovidealone.

8305-30, Session 9

Application of the SNoW machine learning paradigm to a set of transportation imaging problemsP.Paul,A.Burry,XeroxCorp.(UnitedStates);Y.Wang,RochesterInstituteofTechnology(UnitedStates);V.Kozitsky,XeroxCorp.(UnitedStates)


8305-32, Session 9

Image simulation for automatic license plate recognitionR.Bala,Y.Zhao,A.Burry,V.Kozitsky,XeroxCorp.(UnitedStates);C.Saunders,XeroxResearchCtr.EuropeGrenoble(France)


Conference 8305

Documents

2012 Electronic Imaging - SPIE · Digital Imaging Sensors and Applications 8298 Sensors, Cameras, ... The single lens color/depth-unified sensor [1] can provide real time color and