Download pdf - Omnidirectional Human Intrusion Detection … Human Intrusion Detection System 415 busses, etc.) and their contents. Burglar alarm systems (or intrusion detection systems, perimeter

411

Chapter 16

Omnidirectional Human Intrusion Detection System Using Computer Vision Techniques

Wai Kit Wong, Chu Kiong Loo, and Way Soong Lim

Contents16.1 Introduction....................................................................................................................41216.2 HumanIntruderSurveillanceSystem.............................................................................414

16.2.1BurglarAlarmSystem..........................................................................................41416.2.1.1PassiveInfraredMotionDetectorSystem..............................................41516.2.1.2UltrasonicMotionDetectorSystem......................................................41516.2.1.3Glass-BreakDetectorSystem.................................................................41616.2.1.4PhotoelectricBeamSystems...................................................................41716.2.1.5VibrationSensorSystem........................................................................41716.2.1.6PassiveMagneticFieldDetectionSystem...............................................41716.2.1.7MicrophonicDetectionSystem..............................................................41816.2.1.8TautWirePerimeterSecuritySystem.....................................................418

16.2.2Radar-BasedHumanIntruderDetectionSystem.................................................41916.2.3 ImageProcessing-BasedHumanIntruderDetectionSystem.............................. 420

16.2.3.1VisionSpectrumImageProcessing-BasedHumanIntruderDetectionSystem.......................................................421

16.2.3.2NightVision/InfraredSpectrumImageProcessing-BasedTrespasserDetectionSystem.................................................................................. 423

K13920_C016.indd 411 1/4/2013 7:24:10 PM

412 ◾ Effective Surveillance for Homeland Security

16.1 IntroductionHomelandsecurityisaneffortbygovernment(normallyparkedundernationaldefensedepart-ment)topreventterroristattack inacountryandreduceacountry’svulnerabilitytoterrorism[1].Thescopesofhomelandsecurityonhumantrespasserdoincludetheprotectionofacriticalinfrastructure’sperimeterandthebordersecurity(countryborderofterritorialland,water,andairspace).Homelandthreatsrefertothecrimesthathaveanimmediateandvisibleimpactonthelocalcommunityandaffectcitizenqualityoflife.Inthefaceofunknownfutureterroristthreats,illegal immigrants that will flush out a peaceful and economically stable country as refugees,whichmightbringinthefts,smugglers,etc.,issues,however,nationalsecuritydepartmentandconvergentsecurityengineerswillhavetodevelopstrategiesandsecurity/surveillancesystemtofulfilltherequirementofhomelandsecurity,ontrespassers’threats.Thischapterproposessomehomelandsecuritysystemsonhumanintruderdetection.

Intrusiondetectionistheactofdetectingatrespasserinaguardzone.Humanintrusiondetec-tionsystemisasystemusedtodetecthumantrespasserenteringaprohibitedarea.Conventionalhumanintrusiondetectionsystemusesburglaralarmsystem(activeorpassivesensors),wherebymodernhumanintrusiondetectionsystemappliescomputervisiontechniques,bothtotraceoutwhetherthereisanexistenceofatrespasser/humanbeingornotinaprohibitedarea.Themainmeritofmodernhumanintrusiondetectionsystemascomparedtoconventionalhumanintru-siondetectionsystemisthattheimageprocessing-basedhumanintrusiondetectionsystemcanhelpcapturepictures.Sincepicturesarecaptured,therearemorechancesoftheintrudersbeingrecognizedandcaught.Theauthoritycanwiselyplacesecuritycamerasineveryvulnerableplaceoftheguardedarea,indoorandoutdoor,thatwouldbeaccessibletoahumanintruder.Thisallowstheauthoritytostaysafeinsidethepremisewhilestillbeingabletoseewhatishappeningintheoutdoorareaofthepremise.Italsogivesthemmoretimetocallforhelporbackupiftheynoticeanysecuritythreat.

AQ1

AQ2

16.2.4DirectionalversusOmnidirectionalViewing...................................................... 42316.2.4.1OpticalApproachversusMechanicalApproach................................... 42416.2.4.2VisionSpectrum-BasedOmnidirectionalSurveillanceSystem............. 42716.2.4.3Thermal/InfraredSpectrum-BasedOmnidirectional

SurveillanceSystem.............................................................................. 42816.3 UnwarpingMethods.......................................................................................................431

16.3.1DiscreteGeometryTechniqueMethod............................................................... 43216.3.2Pano-MappingTableMethod............................................................................. 43316.3.3Log-PolarMappingMethod................................................................................43516.3.4PerformanceEvaluation...................................................................................... 436

16.4 AutomaticHumanIntruderDetectionAlgorithm......................................................... 43916.4.1PartitionedRegionofInterestAlgorithm............................................................ 43916.4.2HumanHeadCurveTestAlgorithm.................................................................. 44116.4.3ExperimentalResults.......................................................................................... 447

16.4.3.1ExperimentalResultsforPartitionedROI-BasedHumanIntruderDetectionAlgorithm................................................. 448

16.4.3.2ExperimentalResultsforHumanHeadCurveTestAlgorithm.............45016.4.4ComparisonbetweenTwoProposedHumanIntruderDetectionAlgorithms.....452

16.5 ConclusionandFutureResearchDirections...................................................................453References................................................................................................................................454

K13920_C016.indd 412 1/4/2013 7:24:10 PM

Omnidirectional Human Intrusion Detection System ◾ 413

Ingeneral,themoderncomputervisiontechnique-basedhumanintrusiondetectionsystemcanbedividedintotwomaincategories:oneisvisionspectrum-basedhumanintrusiondetectionsystemandanotheroneisnightvision/infrared(IR)spectrum-basedhumanintrusiondetectionsystem.Visionspectrum-basedhumanintrusiondetectionsystemappliesvisionspectrumrangeimagingtoolstocaptureimages.Oneproblemencounteredinthistypeofsurveillancesystemisthechangeinambientlight,especiallyinanoutdoorenvironmentwherethelightingconditionvariesnaturally.Thismakestheconventionaldigitalcolorimagesanalysistaskinsmartsurveil-lancesystembecomeverydifficult.Onecommonapproachtoeliminatethisproblemistotrainthesystemtocompensateforanychangeintheillumination[2].However,thisisgenerallynotenoughforhumanintrusioninthedark.Itisbettertoapplysomesortofnightvisionimagingtoolthathelpsinimagingobjectsinthedark.ThencomestheapplicationofIRspectrum-basedhumanintrusiondetectionsystem.Thermalcameraisanexcellentnightvisionsecuritycamera.ItperceivesIRradiationanddoesnotneedasourceofillumination.Thermalcameraisidealforanylow-lightareas,notjustforthenighttime.Itproducesanimageinthedarkestofnightsandcanviewthroughlightfog,rain,andsmoke.Thermalimagingcamerasmakesmalltemperaturedifferencesvisible.Thermalimagingcamerasarecurrentlyappliedwidelyinmanyneworexistingsecuritynetworks.

Ifa single imagingtool is tomonitora singleangleofa location, thenformore locationsindifferentanglesofview,more imagingtoolsarerequired.Hence, itwillcostmore,besidescomplicatingthesurveillancenetwork.Therefore,anomnidirectionalhumanintrusiondetec-tionsystemusingminimumhardwareisdevelopedtoovercomethecostandnetworkcomplica-tionproblems.Themethodappliedtoobtainomnidirectionalimagescanbeclassifiedintotwoapproaches[3]:(1)mechanicalapproachand(2)opticalapproach.Sincemechanicalapproachleads to many problems on discontinuity and inconsistency, therefore, optical approach wasfavoredbypractitioners.

Thecapturedomnidirectionalimagesnormallyhavesomedifferentpropertiescomparingtoperspectiveimagesintermsofimagingdeformation.Suchdistortionleadstotheimagesbeingdif-ficulttobedirectlyimplemented.Thus,itisnecessarytoworkoutanefficientmethodtounwarptheomni-image.Unwarping,generally,isamethodusedindigitalimageprocessingin“opening”up an omnidirectional image into a panoramic image, making the information on the imagetobeablefordirectimplementation.Unwarpingmethodisactivelyadoptedintheapplicationofvisual surveillance systems.Thereare currently threeunwarpingmethods activelypracticedaroundtheworld,whicharethepano-mappingtablemethod[4],discretegeometrytechniques(DGT)method[5],andlog-polarmappingmethod[6].Thischapterstudiestheadvantagesanddisadvantagesofeachmethod,andtheirperformanceiscomparedandevaluated.

Conventionalsurveillancesystemnormallyemployshumanobserverstoanalyzethesurveil-lancevideo.Sometimesthisismorepronetoerrorduetolapsesinattentionofthehumanobserver[7].Itisafactthatahuman’svisualattentiondropsbelowacceptablelevelswhenassignedtovisualmonitoring,andthisfactholdstrueevenforatrainedpersonnel[8,9].Theweaknessinconven-tionalsurveillancesystemhasraisedtheneedforasmartsurveillancesystemwhereitemployscomputerandpatternrecognitiontechniquestoanalyzeinformationfromsituatedimagingtoolsandautomaticallydetectatrespasser[10].Twoautomatichumanintrusiondetectionalgorithmsarediscussed in this chapter; this includes partitioned regionof interest (ROI) algorithm [11]andhumanheadcurvetestalgorithm[12,13].Withthealgorithmsproposedinthischapter,itissimpletodetectthehumanintrusionofmorethanonelocationinasingleviewcapturedbytheimagingtool.Thesemonitoringandsubsequentanalysesoftheimagesfromtheinspectioncanalertsecuritypersonneltotakefurtheractiontoeithercatchorhustlethetrespassereffectively.

AQ4

AQ3

K13920_C016.indd 413 1/4/2013 7:24:10 PM


Inthischapter,thefundamentalsofhumanintrusiondetectionsystem,classicalburglaralarmsystem(activeorpassivesensorsystem)andradarsystemversuscomputervisiontechniquesystem,visionspectrumimagingversusIRimagingsystem,anddirectionalversusomnidirectionalview-ing,arefirstdiscussed.Thealgorithmandimplementationofsomeuniversalunwarpingmethodswill be discussed too, such as discrete geometric transforms (DGTs) [4], pano-mapping tablemethod[5],andlog-polarmappingmethod[6]proposedintransformingthecapturedomnidi-rectionalimagesintopanoramicform,providingobserverorimageprocessingtoolsawideangleof view.Besides that, automatic human intrusiondetection is implemented in the omnidirec-tionalimagingsystems(bothinvisionspectrumandinIRspectrum,respectively).ThedevelopedhumanintrusionalgorithmsarepartitionedROIalgorithm[11]andhumanheadcurvetestalgo-rithm[12,13],andtheirdesignprocedureswillbeincludedhere.Later,someexperimentalresultstoprovethealgorithmsproposedforthehumanintrusiondetectionsystemareshown.Inthelastsectionofthischapter,wesummarizetheworkandenvysomefutureenhancement.

16.2 Human Intruder Surveillance SystemAccording to tort law,property law,andcriminal law[14], ahuman intruder is apersonwhocommitstheactoftrespassing/intrudingonaprohibitedarea,thatis,withoutthepermissionoftheauthority.Ahumanintrudertrespassestoacriticalinfrastructure’sperimeterandthebordersecurity isdefinedas “an intentional interferencewith the infringeontonational security thatproximatelywillcauseinjury,vandalism,terrorism,theft,etc.”InUnitedKingdomjurisdictions,trespassing has been codified to clearly define the scope of the remedy, and in most jurisdic-tions,trespassingremainsapurelycommonlawremedy,thescopeofwhichvariesbyjurisdiction.Surveillanceisthemonitoringoftheactivities,behavior,orotherchanginginformation,normallywithpeopleinasurreptitiousmannerandattheentranceofprohibitedarea.Surveillanceisveryusefultosecurityauthoritytorecognizeandmonitorthreatsandpreventcriminalactivity.

Humanintrudersurveillancesystemcanbeusedtohelpsecurityauthorityguardacriticalinfrastructure’sperimeterandthebordersecurity.Itisdesignedtodetectanintrusion,activateawarningdeviceupondetectionofanintrusion,determinecrime,protectlifeandproperty,bringanappropriate response toanemergency,andenhance theapprehensionofcriminals.Humanintruder surveillance systemcanbedivided into threemaincategories,whichare theconven-tionalburglar alarmsystem, the radar-basedhuman intruderdetection system, and the imageprocessing-basedhumanintruderdetectionsystem.

16.2.1 Burglar Alarm SystemBurglar(orintrusion)alarmsystemsareelectronicalarmsdesignedtoalerttheusertoaspecificintruder.DetectionsensorsareconnectedtoacontrolunitviaanarrowbandRFsignalorlow-voltagewiringthatisusedtocommunicatewitharesponsedevice.Newconstructionsystemsare predominately hardwired for efficient, more economical hardware installation. Refurbishconstructionoftenapplieswirelesssystemsforafaster,moreeconomicalchannelinstallation,duetoneednotdiggingwall,ceiling,andfloorforrewiring.Somesystemsserveasinglepur-poseofeitherburglarorfireprotectionandsomecombinationsystemsprovidebothfireandintrusion protections. Systems range from small, self-contained noisemakers to complicated,multi-zonedsystemswithcolor-codedcomputermonitoroutputs.Manyoftheseburglaralarmsystemconceptsalsoapplytoportablealarmsystemsforprotectingmotorvehicles(cars,trucks,

K13920_C016.indd 414 1/4/2013 7:24:10 PM


busses,etc.)andtheircontents.Burglaralarmsystems(orintrusiondetectionsystems,perimeterdetection systems, perimeter security systems, perimeter protection systems, andmanymoretermsfortheidenticalitem)aredividedintomanytypes,suchaspassiveIR(PIR)motiondetec-torsystem,ultrasonicmotiondetectorsystem,glass-breakdetectorsystem,photoelectricbeamsystem,vibrationsensorsystem,passivemagneticfielddetectionsystem,microphonicsystem,and taut wire perimeter security system. Each of these burglar alarm systems will be brieflyillustratedinthefollowing.

16.2.1.1 Passive Infrared Motion Detector System

APIRsensor isanelectronicsensorthatmeasuresIRlightradiatingfromobjectswithinitsfieldofview.PIRsensorsareoftenusedintheconstructionofPIR-basedmotiondetectors.ThePIR-basedmotiondetectorisoneofthemostcommondetectorsfoundinhouseholdandsmallbusinessenvironmentsbecauseitoffersaffordableandreliablefunctionality.Theterm“passive”meansthedetectorisabletofunctionwithouttheneedtogenerateandradiateitsownenergy.Thisisdifferentfromultrasonicandmicrowavevolumetricintrusiondetectorsinwhichtheyare“active” inoperation.IfanIR-emittingobjectexists inthecoveragearea, thePIR-basedmotiondetectorisabletoidentifybyfirstlearningtheambienttemperatureofthemonitoredspaceandthendetectingachangeinthetemperaturecausedbythepresenceofthatobject.Applyingthedifferentiationprinciple(creatingindividualzonesofdetectionwhereeachzonecomprisesoneormorelayers)canachievedifferentiation.Betweenthezones,thereareareasofnosensitivity(deadzones)thatareusedbythesensorforcomparison,thatis,acheckofpres-enceornon-presence;PIR-basedmotiondetectorcanverifywhetheranintruderorobjectisactuallyinplace.

InaPIR-basedmotiondetector,thePIRsensoristypicallymountedonaprintedcircuitboardcomprisingtherequiredelectronicsusedtointerpretthesignalsfromthepyroelectricsensorchip[15].Thecompleteassemblageisconfinedwithinahousingattachedinasitewherethesensorcanviewtheareatobemonitored.IRenergyisabletoreachthepyroelectricsensorthroughthewin-dowbecausetheplasticusedistransparenttoIRradiationbutsomehowtranslucenttovisiblelightspectrum.Thisplasticsheetalsoinhibitstheintrusionofdustand/orinsectsfromobscuringthesensor’sfieldofviewand,inthecaseofinsects,fromgeneratingfalsealarms.SomemechanismshavebeenusedtoconcentratethedistantIRenergyontothesensorsurface.

ThePIR-basedmotiondetectorworkingasahumanintruderdetectionsystemhasthemeritsofsimpleandlowerinstallationcostandlesssensitivetoilluminationchanges.However,PIR-basedmotiondetectorhasthesedemeritswhenworkingasahumanintruderdetectionsystem:(1)itcanbeeasilytriggeredbymovinganimals,blowingshrubs,etc.;(2)itcannotdetectpeoplewhoarestationary,thusmayleadtoalargenumberoffalsealarms;(3)itsoutputishighlybursty(somecommercialoff-the-shelfsensorsuseaheuristicsolutiontomakeupforthis,byignoringdetectionsthatfallwithinarefractoryperiodofanearlierevent.Theseissuesarelargelyignoredby thevastmajorityofPIR-basedresearchby limiting their systemtosingle-personscenariosand/orassumingpeoplearealwaysmoving);and(4)itdoesnottoleratelargeareasorlargetem-peraturechanges.

16.2.1.2 Ultrasonic Motion Detector System

Thetransmitterof theultrasonicdetector is radiating anultrasonic signal into the areaundersurveillance.Theultrasonicsoundwavesarereflectedbysolidobjects(suchasthesurrounding

AQ5

AQ6

K13920_C016.indd 415 1/4/2013 7:24:11 PM


walls,floor,andceiling)andthendetectedbythereceiver.Sinceultrasonicwavesaretransmit-ted through air, the hard-surfaced objects tend to reflect most of the ultrasonic energy, whilesoftsurfacestendtoabsorbmostenergy.Thereceivedfrequencywillbeequaltothetransmittedfrequencywhenthesurfacesarestationary.However,achangeinfrequencywilloccurasaresultoftheDopplerprinciple,duetoapersonorobjectmovingtowardorawayfromthedetector.Thiseventwillinitiateanalarmsignal.

Ingeneral,ultrasonicmotiondetectorcanbecategorizedintotwotypes:activeandpassive.Activeultrasonicmotiondetectoremitsultrasonicsoundsthatareinaudibletohumanear(fre-quenciesbetween15and75kHz).Passiveultrasonicmotiondetectorconsistsofonlyreceiversthatsimplyreceivetheemittedsounds.Asthesedevicesareoneofthemostsensitiveamongthehumanintruderdetectionsystems,theyarealsoexpensiveincost.

Ultrasonicmotiondetector systemsutilizeadvanced technology,but somehowunder someconditions,theyarepronetofalsealarmsbystuffslikepassingbirdsorinsects,gustsofwind,orvibrationscausedbyairplanespassingoverhead.Passiveultrasonicmotiondetector systemsdonotprovidecompletedetectioninareaswithlargeobjects,thuscreatinga“dead”zone.Hence,inthesecases,anothertypeofdetectionsystemmayberequiredtoworktogethertheultrasoundsystemasaseconddetectorformoreaccuratealarm.Duetoitspooreffectiveness,thistechnologyisconsideredobsoletebymanyalarmprofessionalsandisnotactivelyinstalled.

16.2.1.3 Glass-Break Detector System

Aglass-breakdetectorisasensorusedinelectronicburglaralarmsystemsfordetectingifthereisapaneofglassshatteredorbroken.Thesedetectorsarecommonlyplacednearglassdoorsorglassstorefrontwindowstodetectwhetherthereisanintrudertryingtobreaktheglassandenterthepremises.Glass-breakdetectorsnormallyapplyamicrophone,whichmonitorsanynoiseorvibra-tionscomingfromtheglass.Ifthevibrationsexceedacertainthreshold(userselectable/preset),theyareanalyzedbydetectorcircuitry.Simplerdetectorsjustsimplyapplynarrowbandmicrophonesthattunedtofrequenciestypicalofglassshatteringandreacttosoundabovecertainthreshold,whereasmorecomplexdesignswillcomparethesoundanalysistooneormoreglass-breakprofilesusingsignaltransformssuchasdiscretecosinetransformorfastFouriertransformandreactifboththeamplitudethresholdandstatisticallyexpressedsimilaritythresholdarebreached.

Theglass-breakdetectorcanbeappliedforinternalperimeterbuildingprotection.Whenglassbreaks, itactuallycreates sound inawidebandof frequencies ranging frominfrasonic (below20Hz, this frequency range is inaudible tohumanear) to theaudioband (20Hz to20kHzthatisaudibletohumanear)rightuptoultrasonic(whichisabove20kHzandagainitfallsinrangeinaudibletohumanear).Therearetwotypesofglass-breakdetectorsingeneral:glass-breakacousticdetectorandseismicglass-breakdetector.Glass-breakacousticdetectorsaremountedincloseproximitytotheglasspanesandlistenforsoundfrequenciesassociatedwithglassbreaking.Seismicglass-breakdetectorsaredifferentinthattheyareinstalledontheglasspane.Whenglassbreaks,itproducesspecificshockfrequenciesthattravelthroughtheglassandoftenthroughthewindow frame and the surrounding walls and ceiling. Typically, the most intense frequenciesgeneratedarebetween3and5kHz,dependingonthetypeofglassandthepresenceofaplasticinterlayer.Seismicglass-breakdetectorssensetheseshockfrequenciesandgenerateanalarmcon-ditionaccordingly.

However,glass-breakdetectorscanonlybeappliedatlimitedareas,forexample,building/premiseswithwindows.Glass-breakdetectorsarealsosensitivetoenvironmentaleffect,suchas

AQ7

AQ8

K13920_C016.indd 416 1/4/2013 7:24:11 PM


waterspreading/rain,insects,birds,orobjecthittingwindows(butwindownotclashing)andcansometimesgeneratefalsealarm.

16.2.1.4 Photoelectric Beam Systems

PhotoelectricbeamsystemsdetectthepresenceofanintruderbytransmittingvisibleorIRlightbeamsacrossanarea,wherethesebeamsmaybeobstructed.PhotoelectricbeamsensorstransmitabeamofIRlighttoaremotereceivercreatingan“electronicfence.”Thesesensorsareoftenusedto“cover”openingssuchasdoorwaysorhallways,actingessentiallyasatripwire.Oncethebeamisbroken/interrupted,analarmsignalisgenerated.Photoelectricbeamsystemconsistsoftwomaincomponents:atransmitterandareceiver.Thetransmitterusesalight-emittingdiode(LED)asalightsourceandtransmitsaconsistentIRbeamoflighttoareceiver.Thereceiverconsistsofaphotoelectriccellthatdetectswhenthebeamispresent.Ifthephotoelectriccellfailstoreceiveatleast90%ofthetransmittedsignalforasbriefas75ms(timeofanintrudercrossingthebeam),analarmsignalisgenerated.

Thephotoelectricbeamsystemworkingasatrespasserdetectionsystemhasthemeritsofeasyinstallationandhighimmunitytoambientlight,anditsfunctionalityisnotaffectedbyelectricalandmagneticfields.However,photoelectricbeamsystemhasthesedemeritswhenconsideredtobeworkedasatrespasserdetectionsystem:(1)highinstallationcostwithtwodeviceshavingtobemounted,wired,andadjustedand(2)detectionofverysmallobjects;thismaysomehowleadtoalargenumberoffalsealarmsduetoanimals,passingobjectsblownbywinds,orevenpassinginsectstriggeringanalarm.

16.2.1.5 Vibration Sensor System

Vibrationsensorsrelyonanunstablemechanicalconfigurationthatformspartoftheelectricalcircuit.Theworkingoperationforvibrationsensoristhatwhenmovementorvibrationoccurs,theunstableportionofthecircuitmovesandbreaksthecurrentflow;thisleadstoanalarm.Thetech-nologyofthedevicesisvaryingandcanbesensitivetodifferentlevelsofvibration.Themediumtransmittingthevibrationmustbecorrectlyselectedforthespecificsensorsastheyarebestsuitedtodifferenttypesofstructuresandconfigurations.

Vibrationsensorsareveryreliablesensors.Itgenerateslowfalsealarmrateintrespasserdetec-tionsystemandmoderateinpricerange.However,thistypeofdetectionsystemmustalwaysbefencemounted.Also,vibrationsensorsareanewtechnologywithanunprovenrecordasopposedtothemechanicalsensor,whichinsomecaseshasafieldrecordinexcessof20years.That’swhyitisnotwidelyseeninthemarketyet.

16.2.1.6 Passive Magnetic Field Detection System

Thistypeofburiedsecuritysystemisbasedonthemagneticanomalydetection(MAD)principleofoperation.TheprincipleoftheMADisbasedontheabilitytosensetheanomalyintheEarthmagneticfieldproducedbythetarget[16].Thesystemappliesanelectromagneticfieldgeneratorpoweredbytwowiresrunninginparallel.Bothwiresrunalongtheperimeterandareusuallyinstalledabout5in.apartontopofawallorabout12in./30cmbelowground.Thewiresareconnectedtoasignalprocessorthatanalyzesanychangeinthemagneticfield.Passivemagnetic

AQ9

K13920_C016.indd 417 1/4/2013 7:24:11 PM


fielddetectionsystemhasverylowfalsealarmrate.Itcanbeputontopofanywallandhasveryhighchanceofdetectingrealburglars.However,itishavinghighinterferenceifitisinstallednearhigh-voltagelinesorradars.

16.2.1.7 Microphonic Detection System

Microphonic-baseddetectionsystemshaveavarietyofdesign,butallaregenerallybasedonthedetectionof a trespasser attempting tocutor climbover a chainwire fence.Themicrophonicdetectionsystemsareusuallyinstalledassensorcablesattachedtorigidchainwirefences.Oneexampleisthemicrophonicfencedisturbancesensorsystem.Microphonicfencedisturbancesen-sorsapplythesignalsgeneratedbytheminuteflexingoftriboelectriccoaxialsensorcable,whichareanalyzedbypowerfulsignalprocessorstodetectthesoundassociatedwithcutting,climbing,orliftingthefencestructure.Thesystemscanalsobeembeddedwithaspecialaudiochannelthatenablessecuritiesto“listen”toactivityalongeachzoneofthefencefortheprotectionofexistingfencesandstructuresagainstcutting,climbing,orlifting.Theycanalsobefittedtocoiledrazorwirefences.

Inoperation,microphonicfencedisturbancesensorsystemsaredesignedtodetectandanalyzeincomingelectronicsignalsreceivedfromthesensorcableandthentogeneratealarmsfromsig-nalsthatexceedsomepresetconditions.Thesystemsofferadjustableelectronicstoallowinstallerstochangethesensitivityofthedetectors’alarmtosuitaspecificenvironmentalcondition.Thetuningofthesystemisnormallyaccomplishedbeforethedetectiondevicesareputincommis-sioning.Microphonicfencedisturbancesensorsystemsareverycheapincost,easytoinstall,andsimpleinconfiguration.However,thesesystemshaveahighrateoffalsealarmsbecausesomeofthesesensorsmightbetoosensitivetoextremeweather,contactby largeanimals,badlymain-tainedfences,andovergrownvegetation.

16.2.1.8 Taut Wire Perimeter Security System

Atautwireperimetersecuritysystemisnormallyastreamoftensionedtripwiresusuallymountedonawallorfence.Itisparticularlyusefulfordetectingclimbing(ontopofawall)orwhereitisnecessarytobuildupaphysicalbarrier(fence).Thissystemisdesignedtodetectanyphysicalattempttopenetratethebarrier.

Tautwireperimetersecuritysystemcanoperatewithavarietyofdetectorsorswitchesthatdetectmovementateachendofthetensionedwires.Thesedetectorsorswitchescanbeanelec-tronicstraingauge,astaticforcetransducer,orasimplemechanicalcontact.Falsealarmscausedbybirds and animals canbe avoidedby tuning thedetectors to omit objects that exert smallamountsofpressureonthewires.However,thistypeofsystemisvulnerabletotrespassersdig-gingunderthefence.Hence,aconcretefootingisinstalleddirectlybelowthefencetopreventsuchtrespassing.Tautwireperimetersecuritysystemsarehavingveryreliablesensors,lowrateoffalsealarms,andhighrateofdetection.However,thistypeoftrespasserdetectionsystemisveryexpensive,itiscomplicatedtoinstall,andthetechnologyisquiteancient.

Ingeneral,conventionalburglaralarmsystemsaresimpleandlowerininstallationandmain-tenancecost.However,theyhavealowerprobabilityofdetectinghumanintrudersandhighfalsealarmrate.Thisisduelargelytomanyuncontrollablefactors,suchasenvironmentalissues(rain,ice, wind, standing water), random animals, and human activities, as well as other electronicinterferencesources.

K13920_C016.indd 418 1/4/2013 7:24:11 PM


16.2.2 Radar-Based Human Intruder Detection SystemThesecondcategoryofhumanintrudersurveillancesystemisradar-basedhumanintruderdetec-tionsystem.Radarisradiodetectionandranging,whichisanobjectdetectionsystemthatappliesradiowavestodeterminetherange,attitude,direction,andspeedofanobject.Inhumanintru-siondetectionsystem,radarcanbeusedtodetecthumanintruder.Theradardishorantennatransmitspulsesof radiowavesormicrowave thatwill bounceoffanyobject in theirpath.Aradar’scomponentconsistsof(1)atransmitterthatgeneratestheradiofrequencysignalwithanoscillatorandcontrolsitsdurationbyamodulator,(2)awaveguidethatbondsthetransmitterandantenna,(3)aduplexerthatactsastheswitchamongtheantennaandtransmitterorthereceiverforthesignalwhentheantennaisusedinbothsituations,(4)areceiverthatknowstheshapeofthedesiredreceivedsignal(so-calledapulse),and(5)anelectronicsectionthatcontrolsallthosedevicesandtheantennatoperformtheradarscanorderedbysoftware.

For theworkingprinciplesof radar, the transmitterwill emit radiowaves (radar signal) inpredetermineddirections.Whenthesesignalscomeincontactwithanobject,theyusuallyreflect/scatterinmorethanonedirection.Theradarsignalsarereflectedbacktowardthetransmitter.Inhumandetection(movingobjectdetection),iftheobjectismovingeithercloserorfartheraway,thereisaslightchangeinthefrequencyoftheradiowaves.Dopplerradarisoneofsuchcommonperimetermonitoringsystems.However,thiskindofradarsystemrequiredthecoverageareatobeclearoffoliageandobstaclesthatmightcreatecoverageshadowsandfalsealarm.Thisrequirementmightnot suitmanyoutdoorenvironments,andeven though in indoorusage, itmightcreateundesirableinstallationandmaintenanceexpenses.Also,slow-movingtargetssometimesmightnotbedetectedonthisradarsystemduetolow-resolutiondetectableDopplershift[17].InRef.[17],engineersareprovingthattheultra-wideband(UWB)RFcanovercomethedeficienciesonconventionalDopplerradar.Someotherrecentadvancedradarsystemsalsodevelopedforhome-landsecurityincludedtheReutechRadarSystem[18]andtheHARRIERGroundSurveillanceRadar(GSR)[19].

ReutechRadarSystemdevelopedandlaunchedtheSpiderRSR940inJuly2009.ThefigureofSpiderRSR940 is shown inFigure16.1. It is ahighlymobile land-based,360°continuous

Figure 16.1 Spider RSR 940. (From http://www.rrs.co.za/products/homeland-security.html) AQ10

K13920_C016.indd 419 1/4/2013 7:24:11 PM


scanningsurveillance radar that iscapableofdetectingsurfaceandair targets. Italsoprovidessector scanningsurveillanceused fordetectionand identificationof slow-movingsurface-basedtargetssuchashumanbeings,smallboats,andevenhelicopters.Inadditiontothelocalcontrolstation,aremotecontrolandmonitoringcapabilityisalsoprovidedfortypicalapplicationinclud-ingthebordercontroloperations,monitoringofcoastaltraffic,coastlinecontrol,andmonitoringofprivateandunattendedairfield.Theradarsystemhasaninstrumentedrangeupto40km.Ontheotherhand,DeTect’sradarprocessingtechnologydevelopedtheHARRIERGSRinyear2011,asshowninFigure16.2.HARRIERGSRusesstate-of-the-artsolid-stateDopplerradartechnol-ogyavailable inS-,X-,andcombinedS/X-bandradarfrequencies.Solid-stateradartechnologyappliedinHARRIERGSRofferssignificantincreasedperformance,longerusefullife,andlowermaintenancecostoverconventionalmagnetron-basedsystems.HARRIERGSRuseshigh-speedscanningforenhancedsmalltargetdetectioninhigh-clutterenvironmentssuchasdevelopedareas,terrain,andhighseastates.Italsohasautomaticdetectionandtrackingcapabilitiesandincludesuser-definedmonitoringandalarmzones.HARRIERGSRisofferedinfixedandmobileconfigu-rationsandcanbelinearlynetworkedtocoverlargeareassuchasbordercrossingsandcoastlines.

Ingeneral,radar-basedhumanintruderdetectionsystemisexpensiveinhardware,sinceradarantennas(transmitterandreceiver),duplexer,waveguides,andelectronictoolsarerequiredtobesetup,anditalsorequiresspecificoperatingsoftware.Besides,thetargetmightbedetectedbytheradar,buttheirtypesorclassisnotknown.Itcanbesometimesvehicles,animals,ormovingobjects,whichgivefalsealarm.Also,somenarrowbandradarsmightsometimesfinddifficultiesinhavinginsufficientrangeresolutiontodiscriminatebetweenasmallernearbytargetandalargerlonger-rangetarget.

16.2.3 Image Processing-Based Human Intruder Detection SystemThe third category of human intruder surveillance system is image processing-based humanintruderdetectionsystem.Itisbymeansofusingimagetotraceoutwhetherthereisanexistenceof trespasser/humanintruderornot.Imageprocessing-basedhumanintruderdetectionsystem

Figure 16.2 HARRIER GSR. (From http://www.detect-inc.com/security.html)

K13920_C016.indd 420 1/4/2013 7:24:12 PM


iswidelyfavoredbymanypractitionersascomparedtoburglaralarmsystemsandradar-basedhumanintruderdetectionsystem,mainlyduetothesefourreasons:

1.Ithelpscapturepictures.Theappliedsecuritycameraisagreattooltocaptureapictureoftheburglar/terroristwhentheyaretryingtotrespass/breakintoaprohibitedterritory.Thisisveryimportantbecausethesecuritycameragivestheauthoritiessomethingthattheycanusetohelpthemidentifytheburglar/terrorist.Havingasecuritysystemthattriggersanalarmisessential,butwithoutsecuritycameras,theauthoritieswillneverknowwhoorhowmanyburglars/terroriststriedtogetintotheirterritory.

2.Morechanceoftheburglars/terroristsbeingcaught.Whentheauthoritiesareabletoviewthepicturesfromthecamerastoidentifytheburglars/terrorists,itprovideshigherpossibleratefortheburglars/terroriststogetcaught.Inmanyoccasionswhenauthoritiesarriveattheirincidentsite,theburglars/terroristswillbelonggone.Withoutthecapturedpictures,thereisnotmuchchancefortheburglars/terroristsbeingcaught.

3.Securitycamerasaregreatpreventiontools.Theyaresomethingthatallburglars/terroristswilllookthroughbeforetheydecidetobreakinto/trespassaterritory.Mostoftheburglars/terroristswillnotevenattemptaterritoryiftheydetecttheexistenceofsecuritycamerasbecausetheyknowthatthisisgoingtoworkagainstthemandcausethemtogetcaught.Burglars/terrorists areknown for avoiding territory thathasgood security, especially theones that aremonitoredwith security cameras.Thecameraspose toobigof a threat forthem,sotheywillmoveontoatargetthatdoesn’thavegoodsecurity.

4.Securitycamerascansecurevulnerableareas.Whenauthorityisinsidetheinfrastructure’sperimeterandneedstoseewhatishappeningoutsideofthenearbybuildingforsecurity,securitycamerasarethebestwaytoachievethatgoalsafely.Theauthoritycanwiselyplacesecuritycamerasineveryvulnerableareaofhisorherbuilding,indoorsandoutdoors,thatwouldbeaccessiblebyaburglar/terrorist.Thisallowstheauthoritytostaysafeinsidehisorherbuildingwithhisorherprotectedpersonwhilestillbeingabletoseewhatishappen-ingintheoutdoorareaofthebuilding.Italsogivesthemmoretimetoseekforhelpiftheynoticeanysecuritythreat.

Ingeneral,theimageprocessing-basedtrespasserdetectionsystemcanbedividedintotwomaincategories:oneisvisionspectrumimageprocessing-basedhumanintruderdetectionsystemandanotheroneisnightvision/IRspectrumimageprocessing-basedhumanintruderdetectionsystem.

16.2.3.1 Vision Spectrum Image Processing-Based Human Intruder Detection System

Visionspectrumimageprocessing-basedhumanintruderdetectionsystemcanbedividedintotwocategories:oneisanalogvideosurveillancesystemandtheotheristhedigitalvideosurveil-lancesystem.

16.2.3.1.1 Analog Video Surveillance System

Datingbacktoasearlyas1965,analogvideosurveillancewasfirstbegunwithsimpleclosedcircuittelevision(CCTV)monitoring.TheU.S.pressreportssuggestingpolicealreadystartusingsurveil-lance cameras inmonitoringpublicplaces’ security. In1969,police installed sets of surveillancecamerasinNewYorkCityatthemunicipalbuildingnearthecityhall.Thispracticelaterspreadsto

AQ11

K13920_C016.indd 421 1/4/2013 7:24:12 PM


othercitiesintheUnitedStateswithCCTVsystemsandkeepsaneyebypoliceofficersatalltimes.CCTVistheapplicationofvideocamerastosendimagesignaltoaspecificlocation,onalimitedsetofmonitors.ThefirstCCTVappliedinpublicplaceswascrude,conspicuous,lowdefinition,andinblackandwhitesystemsthatareunabletopanorzoomintoparticularview.Inmoderndays,CCTVsystemsapplysmaller-sizeandhigh-definitioncolorvideocamerasthatcanfocustoresolveminutedetailandalsocanlinkthecontrolofthevideocamerastoacomputer.Thismakestheobjectstobetrackedsemiautomatically.Thistechnology,so-calledvideocontentanalysis(VCA),iscurrentlyusedbyalargenumberoftechnologicalcompaniesaroundtheworldtoenablethesystemstorecognizewhetheramovingobjectisawalkingperson,acrawlingperson,ananimal,avehicle,etc.

However, in themid-1990s, the emergingof digital technologyhas superseded the analogtechnologyinvideosurveillancesystem.Digitalmakesvideosurveillanceclearer,faster,andmoreefficient. Digital video surveillance has made complete sense as the price of digital recordingdroppedwiththecomputerrevolution.Insteadofchanginganalogvideotapesdaily,thedigitaluserscouldnowreliablyrecordamonth’sworthofsurveillancecontentsonharddrivebecauseof itshighcompressioncapabilityand lower storagecost.Thedigitally recorded imagesare somuchclearerthantheanalog-recordedimages.Thisleadstotherecognitionprocessimmediatelyimprovingforpolice,privateinvestigators,andotherusersthatusevideosurveillanceforidenti-ficationpurposes.Byusingdigitaltechnology,theimagescouldalsobemanipulatedtofurtherimproveclaritybyaddinglight,enhancingtheimage,zoominginonframes,etc.

16.2.3.1.2 Digital Video Surveillance System

Digitaltechnologyisadatatechnologythatusesdiscretevalues[20].Digitalvideoisatypeofvideorecordingsystemapplyingdigitaltechnology.Thereisabroadrangeofdigitalvideosurveil-lancecamerasavailableinthemarket:

◾ Fake security cameras:thesecameraslooksimilartothosesurveillancecamerasavailableinthemarket,buttheyarenotactualcameras.Theyhavenorecordingcapability.Thesecam-erascanactasdeterrentcamerastoscareburglars/theft.Ifsomethinghappens,theywillnothavearecordsincetheyhavenorecodingcapability.

◾ Covert surveillance cameras: these cameras look like regular items, to hide its identityasasurveillancecamera, forexample,awallclockinashop,afacingfrontdoorteddybear,andapottedplantattheshop’scorner.Eachoneofthemcouldveryeasilyembedasurveillancecamera.Thesurveillancecamerascanrecordthescenesanytimewithoutanybodyknowingitsexistence.

◾ Wireless security digital cameras:thesesurveillancecamerasareeasytoinstallandremoved,areoftensmallinsize,havenowiringconnectionseen,andoffermoreflexibilityinsetup.Thesecamerastransmitimagesignalswirelesslytoacenterhubthatareshownonamonitorscreeninamonitoringroom.

◾ Wired surveillance digital cameras: these surveillance cameras arewired and lackflexibilityinsetup.Theyareappropriateforpermanentsetup.Thesecamerastransmit imagesignalsthroughawiretoacenterhubthatareshownonamonitorscreeninamonitoringroom.

◾ Home surveillance cameras:thesecamerascomeinapackagethatoftenincludessomeextrafeaturessuchastimersforlamps,motionsensors,andautomaticgatedoorlock.

Oneproblemencounteredinmostvisionspectrumimageprocessing-basedsurveillancesystemsisthechangeinambientlight,especiallyinanoutdoorenvironmentwherethelightingcondition

AQ12

AQ13

AQ14

AQ15

K13920_C016.indd 422 1/4/2013 7:24:12 PM


variesnaturally.Thismakestheconventionaldigitalcolorimageanalysistaskinsmartsurveil-lanceverydifficult.One commonapproach to alleviate thisproblem is to train the system tocompensateforanychangeintheillumination[2].However,thisisgenerallynotenoughforatrespasserinthedark.Itisbettertoapplysomesortofnightvisionimagingtoolsthatcanhelpimagingobjectsinthedark.

16.2.3.2 Night Vision/Infrared Spectrum Image Processing-Based Trespasser Detection System

Nightvisionistheabilitytoseeinadarkenvironment.Nightvisionismadepossiblebyacombi-nationoftwoapproaches:(1)sufficientspectralrangeand(2)sufficientintensityrange.Humanbeingshavepoornight vision ability compared tomany animalsbecausehumaneyes lack anelement,so-calledthetapetumlucidum.Thetapetumlucidum[2]isalayeroftissueintheeyeofmanyvertebrateanimals,whichliesimmediatelybehindorsometimeswithintheretina.Itreflectsvisible lightback through the retina, increasing the light available to thephotoreceptors.Thisimprovesvisioninlow-lightconditionsbutcancausetheperceivedimagetobeblurryfromtheinterferenceofthereflectedlight.Thetapetumlucidumcontributestothesuperiornightvisionofsomeanimals.Manyoftheseanimalsarenocturnal,especiallycarnivoresthathunttheorganismatnight,whileothersaredeepseaanimals.

Thermalcameraisanexcellentnightvisionsecuritycamera.ItperceivesIRradiationanddoesnotneedasourceofillumination.Thermalcameraisidealforanylow-lightareas,notjustforthenighttime.Itproducesanimageinthedarkestofnightsandcanviewthroughlightfog,rain,andsmoke.Thermalimagingcamerasmakesmalltemperaturedifferencesvisible.Thermalimagingcamerasarecurrentlyappliedwidelyinmanyneworexistingsecuritynetworks.

16.2.4 Directional versus Omnidirectional ViewingIn spite of the availability of many modern sophisticated surveillance monitoring products inthe market, majority of the systems have the limitation in the viewing angle of the camera.Omnidirectionalpromptstotheconceptoftheexistenceinalldirection,with360°areacoverageonasingleplane/axis.Inimagingpointofview,anomnidirectionalvisualizationhasvisualizationcapabilityofa360°fieldofviewaroundthehorizontalplaneorwithvisualfieldthatcoverstheentiresphere.Omnidirectionalvisualizationsystemisimportantinareasthatneedlargevisualfieldcoverage,suchasinpanoramicimagingandinrobotics.Aconventionalimagingtoolnor-mallyhasafieldofviewwiththerangeofafewdegreestomaximumof180°.Itcancaptureonlyasemisphereimagewithlightfallingontotheimagingtool’sfocalpoint.However,ontheotherhand,anomnidirectional imagingtoolcancapture light fromalldirections (surrounded360°fieldofview)fallingontoitsfocalpoint,coveringafullsphere.

Convergentsecuritysystemsaresecuritysystemsthatintegrateintrusion,holdup,fire,videosurveillance,accesscontrol,andmonitoringapplicationsinphysicalsecuritysystemsandITinfra-structures. However, the current convergent security systems apply digital CCTV monitoringsystems, inwhichthecoverageareaisdirectional.Eventheycandoit inomnidirectional,butitrequiresmorehardware.Conventionalapproachestoobtainpanoramic(wideview)imageforanomnidirectionalviewmainlyconsistofcombiningsnapshotscapturedseparatelyintoasingleandcontinuousimage.Thiscombinationofimagesiscomputationallyintensivesometimes.AnexampleisbyusingaRANSACiterativealgorithm[21]tocombinethesnapshots.RANSACisanabbreviationfor“RANdomSampleConsensus.”ThisalgorithmwasfirstpublishedbyFischler

AQ16

AQ17

K13920_C016.indd 423 1/4/2013 7:24:12 PM


andBollerin1981andfoundtobeusedinsolvingthecorrespondingproblem(partsofanimagecorrespondtopartsofanotherimage;aftertheimagingtoolhasmoved,timehaselapsedorthefocusingobjectsmovedaround)andcalculatesthefundamentalmatrixcorrespondingtoapairofstereoimagingtools.RANSACcanestimatetheparameterswithahighdegreeofaccuracybutwithlimitationthatthereisnoupperboundonthetimeittakestocomputetheseparameters.Toprocessanimage,itissometimestimeconsumingorendless.Evenifanuppertimeboundisused(setwithamaximumnumberofiterations),theresultsobtainedmaynotbetheoptimalone.Itmaynotbeonethatgeneratestheimageinagoodway.

Besidescomputational intensive, thecombiningof images to formapanoramic imagealsodependsonthequalityandconsistencyofthesnapshotsused.Thesnapshotimagesmighthaveanumberofdeficienciesthatwillfurtherimpairthequalityoftheoutputpanoramicimage.Incomparison,anomnidirectionalimagingtoolcanbeusedtocreatereal-timepanoramicart,with-outpost-processingrequirement,andsomehowwillprovidemuchbetteroutputqualityimage.

In robotics and computer vision, omnidirectional imaging tools are widely used in visualodometry [22] and also help solve the simultaneous localization and mapping (SLAM) [23]problemsvisually.Visualodometryistheprocessofdefiningthepositionandorientationofarobotbyanalyzingthecapturingimagesfromtheattachedimagingtools,whereasSLAMisatechniqueappliedbyautonomousvehiclesandmobilerobotstoformamapwithinanunknownenvironmentor toupdate amapwithin a known environment and in themeantimekeepon tracking theircurrentlocation.Duetotheomnidirectionalvisualization’sabilitytoobtaina360°view,roboticandcomputervisiontaskscanhavebetterresultsforopticalflowandinfeatureselectionandmatching.

Besidespanoramicart,robotics,andcomputervision,applicationofomnidirectionalvisual-izationalsoincludessurveillance,inwhichitisimportanttocoveralargevisualfield,andtele-conferencing,inwhichitisofgreatinteresttocoverasmanyparticipantsaspossibleinthesameimage,andyetthereareunlimitedapplicationsthatwillbediscoveredinthefuturesoon.Nextsectionwillfurtherdiscussthemethodsappliedtoacquireomnidirectionalviews.

16.2.4.1 Optical Approach versus Mechanical Approach

Omnidirectionalvisualizationpossessessignificantapplicationpotentialsintheareassuchaspan-oramicart,mobilerobotnavigationandcomputervision,qualitycontrol,andsurveillance.Themethodsappliedtoobtainomnidirectionalimagescanbeclassifiedintotwoapproaches[3]:

1.Mechanical approach:themethodofgatheringimagestogenerateanomnidirectionalimage 2.Optical approach:themethodofcapturinganomnidirectionalimageatonce

Inaddition,theyareclassifiedintotwocategoriesbytheviewpointoftheimage[3]:singleview-pointandmultipleviewpoints.

For the mechanical approach, the images captured on a single viewpoint are continuous.Oneexampleistherotatingcamerasystem[24–27].Insuchasystem,thecamerarotatesaroundthecenteroftheprojection.Itgeneratesanomnidirectionalimagefromasingleviewpoint.Theproperorderofimagesobtainedbyrotationis joinedtogethertoacquireapanoramicviewforthescene.AnexampleofrotatingcameraisshowninFigure16.3.Arotatingmotorisrequiredtorotatethevideocamerainordertoscantheomnidirectionalview.However,sinceitisnecessarytorotateavideocamerainafullcircleinordertoacquireasingleomnidirectionalimage,itisimpossibletogeneratereal-timeomnidirectionalimage.Otherdisadvantagesofrotatingcamerasystemarethatitrequirestheuseofmovingpartsandprecisepositioning.Theimagecaptured

AQ18

K13920_C016.indd 424 1/4/2013 7:24:12 PM


atmultipleviewpointsisrelativelyeasytoconstruct,asshowninFigure16.4.Asinglecameraormorecamerasareappliedtogathermultipleimagesatmultipleviewpointsandcombinethemintoanomnidirectionalimage.QuickTimeVRsystem[28]adoptedsuchtechnologiesandhasmanymarketapplications.However,theimagesgeneratedbythesystemarenotalwayscontinuousandconsistent,anditalsocannotcapturethedynamicsceneatvideorate.

Sincemechanicalapproachleadstomanyproblemsondiscontinuityandinconsistency,opti-calapproachwasinuse.Thisapproachismostappropriateforreal-timeapplicationsanditisofsingleviewpoint.Thistypeofapproachneednotusemotorsandinthemeantimeitcancaptureomnidirectionalimageatonce,noextracombinationworkrequired,anditisfast.Twoalterna-tiveshavebeenproposed,namely,theuseofspecial-purposelens(suchasthefish-eyelens[29])andtheuseofhyperbolicopticalmirrors[30].Fish-eyelens,asshowninFigure16.5,areusedtoreplaceaconventionalcameralensthathaveveryshortfocallengththatallowsthecameratoviewobjectsasmuchasinahemispherescene.Fish-eyelenshavebeenwidelyusedforwide-angleimagingareasasnotedinRefs.[31,32].However,Nalwa’sworksinRef.[33]foundoutthatitisdifficulttodesignfish-eyelensthatensurethatall incomingprincipalraysintersectatasinglepointtoyieldafixedviewpoint.Theacquiredimageusingfish-eyelensnormallydoesnotpermittheconstructionofdistortion-freeperspectiveimagesoftheviewedscene.Hence,tocaptureanomnidirectionalview,thedesignofoptimalfish-eyelensmustbequitecomplexandlarge,andtherefore,theyareexpensiveincost.Besides,accordingtoRef.[34],therelativeilluminationforafish-eyelensdesigniswidelyvarying.Inaddition,theexistencesofdistortionacrossthehemi-sphericalfieldofviewneedtobeinconsiderationwhendesigningagoodqualityfish-eyelens.Sincefish-eye lensareexpensiveandcomplexindesignandalmostprovidethesamereflectivequalityashyperbolicopticalmirror,hyperbolicopticalapproachisplannedtoadapt.

AQ19

Figure 16.3 Rotating camera. (From Stun-ningsales.com, Weatherproof CCD color rotating video security camera, redirecting from http://www.stun-ningsales.com/homethings/outdoor_securitycameras.htm)

K13920_C016.indd 425 1/4/2013 7:24:13 PM


Figure 16.5 Fish-eye lens. (From The-Digital Image.com, Fisheye lens, redirecting from http://www.the-digital-picture.com/Reviews/)

Figure 16.4 Multiple cameras system. (From Chen, S.E., Quick time VR: An image-based approach to virtual environment navigation, in Proceedings of the 22nd Annual ACM Conference on Computer Graphics, Los Angeles, CA, pp. 29–38, 1995.)

K13920_C016.indd 426 1/4/2013 7:24:14 PM


16.2.4.2 Vision Spectrum-Based Omnidirectional Surveillance System

The proposed vision spectrum-based omnidirectional surveillance system model is shown inFigure16.6.

Inthismodel,omnidirectional imagesofanobservedscenearecapturedusing thecombi-nation of a web camera (webcam) and a specific design hyperbolic optical mirror. MATLAB®inthe laptopcomputerwillperformunwarpingonthe imagescaptured intopanoramic form.Thehuman intruderdetection algorithm that is programmed inMATLABwill thenbeusedtoprocess thepanoramic images todetect thepresenceofhumanintruder. Ifhumanintruderisdetected,alarmwillbesignaledandportionsofsuspectedimagewithhumanintruderwillbestoredinadatabaseforfurtheridentificationpurposes.

ThesurveillancecamerasetusedinthissurveillancesystemconsistsofawebcameraandanattachedspecificdesignhyperbolicopticalmirrorasshowninFigure16.7.

Laptopcomputer

Alarm

Webcamspecificdesign

hyperbolicmirror

+

Figure 16.6 Omnidirectional surveillance system model.

Figure 16.7 Surveillance camera set (web camera and specific design hyperbolic optical mirror).

K13920_C016.indd 427 1/4/2013 7:24:15 PM


ThewebcamerausedinthesurveillancesystemisE3500PlusQuickCambyLogitech.Itisadomesticwebcamerathatcaptureshigh-qualityVGA(640×480)videosand1.3megapixel(soft-ware-enhanced)images.ThewebcameraissmallinsizeandcheapcomparedtodigitalandCCTVcamerawithfine-resolutionoutput.ThedigitalcontrolisalsoaccomplishedthroughtheUSBportconnectedtoalaptoporpersonalcomputer(PC)viaplugandplayonWindowsXPorVista.ItcanbeinterfacedwithMATLABtoo.Therefore,itisbestoutfittedinomnidirectionalsurveillancesystem.

Thespecificdesignhyperbolicopticalmirrorusedintheomnidirectionalsurveillancesystemisasmall-sizewide-viewtype,withouterdiameterof40mmandangleofview30°abovehorizon-talplanemanufacturedbyACCOWLEVISION.Themirrorcanreflecta360°viewsurroundedbyitself,andasthewebcameraplugsonit,omnidirectionalimageswithinaguardedperimetercanbecapturedandsenttoalaptopcomputerinamonitoringroomtobeprocessedforsurveil-lancepurpose.A custom-madebracket that is shown inFigure16.8 is designed to attach thehyperbolicmirrortothewebcameraviaasocket.

Alaptopcomputercanbeusedforimageprocessingofanobservedroom.Acore2duolaptopcomputerwithspecs1.83GHzprocessorand2GBDDR2RAMwithMATLABver.7.0ischo-sentobeusedhere.MATLABhasadataacquisitiontoolboxinterfacethatenablesattainmentofvideosandimagesthroughtheE3500PlusQuickCam.Asetofconnectingspeakerstothelaptopcomputersoundsthealarmifhumanintruderisdetected.

16.2.4.3 Thermal/Infrared Spectrum-Based Omnidirectional Surveillance System

Oneproblemencounteredinmostsurveillancesystemsisthechangeinambientlight,especiallyinoutdoorenvironmentwherethelightingconditionisnaturallyvarying.Thismakesthevideoanalysistaskinsmartsurveillanceverydifficult.Onecommonapproachtoalleviatethisproblemistotrainthesystemtocompensateforanychangeintheillumination.However,thisisgenerallynot enough forobject trackingandmonitoring in thedark. In recent times, severalmanufac-turershavecomeupwithhighlysophisticatedthermalcameraforimagingobjectsinthedark.ThecamerausesIRsensorsthatcaptureIRcomingfromdifferentobjectsinthesurroundingand

Figure 16.8 Front view of the custom-made bracket.

K13920_C016.indd 428 1/4/2013 7:24:15 PM


formanIRimage.SinceIRradiationfromanobjectisduetothethermalradiation,theimageformationwilldependontheobjecttemperatureandnotonthelightreflectedfromtheobject.Hence,suchcameracanbeconvenientlyusedfornightvision.Thepresentstateoftheartevenallowsthermalcameratocaptureobjectevenfromaverylongdistance.

Ifasinglethermalcameraistomonitorthesecurityofasinglelocation,thenformoreloca-tions indifferentanglesofview,more thermalcamerasarerequired.Hence, itwillcostmore,besidescomplicatingthesurveillancenetwork.Theproposedthermal/IRspectrum-basedomnidi-rectionalsurveillancesystemmodelisshowninFigure16.9.Thissystemrequiresacustom-madeIR-reflectedhyperbolicmirror,acameramirrorholder,afine-resolutionthermalcameraandalaptoporPC installedwithMATLABprogramming (versionR2007bor later), andanalarmsignalingsystem.Thealarmsignalingsystemcanbeassimpleasacomputer’sspeaker.

Thebestshapeofpracticaluseomnidirectionalmirrorishyperbolic.AsderivedbyChahlandSrinivasaninRef.[37],allthepolynomialmirrorshapes(conical,spherical,parabolic,etc.)donotprovideacentralperspectiveprojection,exceptforthehyperbolicone.Theyalsoshowthatthehyperbolicmirrorguaranteesalinearmappingbetweentheangleofelevationθandtheradialdistancefromthecenteroftheimageplaneρ.Anotheradvantageofhyperbolicmirroriswhenusingitwithacamera/imagerofhomogenouspixeldensity,theresolutionintheomnidirectionalimagecapturedisalsoincreasingwithgrowingeccentricity,andhence,itwillguaranteeauniformresolutionforthepanoramicimageafterunwarping.

TheresearchgroupofOMNIVIEWSproject fromCzechTechnicalUniversityfurtherdevel-opedMATLABsoftwarefordesigningomnidirectionalmirror[38].FromtheMATLABsoftware,omnidirectionalhyperbolicmirrorcanbedesignedbyinputtingsomeparametersthatspecifythemirrordimension.Thefirstparameteristhefocallengthofthecameraf,inwhichforthethermalcamerainuseis12.5mmandthedistanced(ρzplane)fromtheoriginissetto2m.Theimageplaneheighthissetto20cm.Theradiusofthemirrorrimischosent1=3.6cmasmodifiedfromSvoboda’sworkinRef.[39],withradiusforfovearegion0.6cmandretinaregion3.0cm.Foveaangleissetbetween0°and45°,whereasretinaangleisfrom45°to135°.ThecoordinatesaswellastheplotofthemirrorshapearegeneratedusingMATLABandshowninFigure16.10.ThecoordinatesaswellasmechanicaldrawingusingAutoCADareprovidedtoprecisionengineeringcompanytofabricate/custommadethehyperbolicmirror.Thehyperbolicmirrorismilledfromaluminumbarandthenchromed.Chromiumisofgreatinterestbecauseofitslustrous(goodinIRreflection)property,highcorrosionresistance,highmeltingpoint,andhardness.ThefabricatedmirrorisshowninFigure16.11.

Thecameramirrorholderisself-designedandcustommadewithaluminummaterialasshowninFigure16.12.Thethermalcamerausedisanaffordableandaccuratetemperaturemeasurementmode:ThermoVisionA-20MismanufacturedbyFLIRSystems.Thethermalcamerahasatemper-aturesensitivityof0.10rangingfrom−20°Cto350°C.However,forhumandetection,thetemper-aturerangeissettorangefrom30°Cto40°C.Thethermalcameracancapturethermalimageswithfineresolutionupto320×240pixelsofferingmorethan76,000individualmeasurementpoints

Custom made IRreflected hyperbolicmirror + cameramirror holder set

Thermalcamera

Capturethermalimage

Laptop/PC

Processimage and

signal alarm

Figure 16.9 Omnidirectional thermal imaging surveillance system model.

K13920_C016.indd 429 1/4/2013 7:24:16 PM


perimageatarefreshrateof50/60Hz.TheA-20Mfeaturesachoiceofconnectivityoptions.Forfastimageanddatatransferofreal-timefullyradiometric16bitimages,anIEEE-1394FireWiredigitaloutputcanbechosen.Fornetworkand/ormultiplecamerainstallations,Ethernetconnec-tivityisalsoavailable.EachA-20McanbeequippedwithitsownuniqueURLallowingittobeaddressedindependentlyviaitsEthernetconnection,anditcanbelinkedtogetherwitharoutertoformanetwork.Therefore,itisbestoutfittedforhumanintruderdetection.

Figure 16.11 Fabricated mirror.

3

2.5

2

1.5

1

0.5

0 –3 –2 –1 0 1 2 3

cm

cm

Figure 16.10 Mirror coordinates plot in MATLAB.

K13920_C016.indd 430 1/4/2013 7:24:16 PM


Aproblemencountered in thermalcamera selection is theexistenceof thehaloingeffect inuncalibratedferroelectricbariumstrontiumtitanate(BST)sensors.Haloingeffectisthepresenceofhalosaroundobjectshavingahighthermalcontrastwiththebackground[40].A-20Mischo-senbecauseitusestheuncooledmicrobolometerFPAdetectortechnologythatdoesnotproducethehaloingeffect.A laptoporPCcanbeusedas imageprocessor,placedeitheronsiteor inamonitoringroom.MATLABversionR2007bprogramming ischosentobeusedbecause ithasuser-friendlysoftwareforperforminglog-polarmappingtechniquetounwraptheomnidirectionalthermal image into panoramic form and it can partition the panoramic thermal images easilyaccordingtoeachsinglelocationtobemonitoredandprocessthemsmoothlywiththetrespasserorfaintdetectionalgorithmuserprogrammedin.Thealarmwillbetriggeredonceahumanbeingisdetectedinatestedimageforhumanintruderdetectionmode.TheoverallfabricatedsystemmodelisshowninFigure16.12.

16.3 Unwarping MethodsThecapturedomnidirectionalimageshavedifferentpropertiescomparedtoperspectiveimagesintermsofimagingdeformation.Suchdistortionleadstotheimagesbeingdifficulttobedirectlyimple-mented.Thus,itisnecessarytoworkoutanefficientmethodtounwarptheomni-image.Unwarping,generally,isamethodusedindigitalimageprocessingin“opening”upanomnidirectionalimageintoapanoramicimage,makingtheinformationontheimagebeabletobedirectlyimplementedand

AQ20

AQ21

Figure 16.12 Overall fabricated omnidirectional thermal imaging surveillance system model.

K13920_C016.indd 431 1/4/2013 7:24:17 PM


understood.Thissubsectionstudiesthreeuniversalunwarpingmethodsthatarecurrentlyappliedactivelyaroundtheworldintransformingomnidirectionalimagetopanoramicimage,namely,theDGTmethod[4],thepano-mappingtablemethod[5],andthelog-polarmappingmethod[6].

16.3.1 Discrete Geometry Technique MethodDGTmethod,bythenameitself,meansthatthistechniqueisusedbyapplyingonebyonethegeometryoftheimage,discretely,inordertosuccessfullyunwarptheomnidirectionalimageintoapanoramicimage.ThismethodispracticallyusedintransformingtheomnidirectionalimagesintopanoramicimagesonacylindricalsurfaceusingPDE-basedresamplingmodels[4].

InDGTmethod,itisrequiredtoperformthecalculationofeachandeverypixelintheomni-imagefirstandthencheckforitscorrespondingradiusfromthecenteroftheomnidirectionalimageandlaterdeterminewhetheritshouldbeconsideredornot.Thecalculationsstartfromafixedposi-tionanddirection,suchasfromtherightgoingcounterclockwisefor360°.Foraradiusof1,acircleofradius1willbevisualizedinthecenterofomnidirectionalimage,whichinotherwordsmeansthatthecirclewillbeinsizeof3×3pixels.Allthepixelsinthisboundaryof3×3pixelswillbeconsidered,andtheircorrespondingradiuswillbecalculated.Allpixelsthatfallwithintheradiusof1,whichistheradiusofconcern,willbeconsideredintheconversion.Duetothepixelsthataregenerallyanareaofdatainformation,itispossiblethatthecirclewilllieinbetweenthepixels.Therefore,atoleranceof±½radiusissettocounterthisproblem.Inotherwords,acircleofradius1willconsiderthepixelslyingbetweenradiusof0.5and1.4,andacircleofradius2willconsiderthepixelslyingbetweenradiusof1.5and2.4,andsoon.AnexampleisshowninFigure16.13.

Assoonasapixel intheboundaryisdeemedtobeconsideredorinrangeoftheradius, itwillbemappedintoanewmatrixofpanoramicimage.However,sincethepixelsmappedintothepanoramicimagemustbeinordersothattheimagewillnotbedistorted,theimagewillbesplitintofoursectionsof90°each,asshowninFigure16.14,whereeachsectionwillperformthecalculationbasedonthemovingdirectionofthecircle.Forexample,foracircledrawn,startingfromtherightinacounterclockwisedirection,thepixelsinthesectionattheupperrightpartwillbetakenandcalculatedonebyone,fromthebottompartofthesectionandfromrighttoleft,whichwillthenbeincreasedonebyone,tilltheupperpartofthesection,fromrighttoleftaswell.Ontheotherhand,forthelowerleftpartofthesection,thecalculationwillgofromthetopofthesection,goingfromlefttoright.

However,duetothepixelsbeingconsideredfordifferentcirclesofdifferentradiithatwillbenonuniform,asshowninFigure16.15,aresamplingprocessisneededtostandardizethepixelsin

Pixels lyingin between

Figure 16.13 Circle lying in between pixels.

K13920_C016.indd 432 1/4/2013 7:24:18 PM


everyrowofthepanoramicimage.Therefore,aftereverypixelinthewholeomni-imageismappedontothepanoramicimageplane,spacingwillbeinsertedinbetweenpixelsineveryrow(asshowninFigure16.16)inordertostandardizetheresolutionofthepanoramicimageforeachrow.Thiswillgenerateastandardresolutionofpanoramicimage.However,duetospacingthatisgenerallyemptypixelswithnodatainformation,arowwithverylittlepixelswillbehardlyunderstandable.Therefore,thepixelswillbeduplicatedoverthespacing,insteadofinsertingemptypixelsintoit,andanunderstandableuniformresolutionpanoramicimagecanbegenerated.

16.3.2 Pano-Mapping Table MethodThis method uses a table, which is so-called the pano-mapping table, to process the imageconversion.Pano-mappingtablewillbecreated“onceandforall,”consistingofmanycoordi-natescorresponding to thecoordinates taken fromtheomnidirectional image thatwill thenbemappedintoanewpanoramicimage,respectively.Itispracticallyusedinomnidirectionalvisual tracking[41]andtheunwarpingprocessofomni-images takenbyalmostanykindofomni-camerasprior to requiringanyknowledgeabout thecameraparameters inadvance, asproposedbyJengetal.[5,8].

Figure 16.14 Circle being split into four sections.

Figure 16.15 Nonuniform resolution of panoramic image.

Figure 16.16 Spacing is inserted in between pixels, denoted by black dots.

K13920_C016.indd 433 1/4/2013 7:24:18 PM


Inpano-mappingtablemethod,itisrequiredtoselectfivelandmarkpointsfromtheomni-directionalimagefirst.Thesepointswillbetakenfromthesameline,drawingfromthecenteroftheomni-imagetothecircumferenceoftheimage,whichinotherwordsiscalledtheradiusoftheimage.Fivepointsinbetweentheendofthislinewillbepicked,andthevaluecorrespondingtotheirradiusfromthecenterisobtained.Itisthenusedinordertoobtainthefivecoefficientsofa0througha4inthe“radialstretchingfunction,”fr(ρ),describedbythefollowing4th-degreepolynomialfunctionof

r f a a a a ar 1 1 2 2 3 3 4 4= ( ) = + + + +ρ ρ ρ ρ ρ0 (16.1)

wherercorrespondstotheradiusρistheparticularradiusfortheeachofthefivepointstakena0–a4arefivecoefficientstobeestimatedusingthevaluesobtainedfromthelandmarkpoints

Oncethefivecoefficientsareobtained,thepano-mappingtable,TMN,canthenbegenerated.Thesizeofthetablewillfirstbedeterminedmanually,bysettingittoatableofsizeM×N.Hence,inordertofillupatablewithM×Nentries,thelandmarkpointρ,whichcorrespondstotheradiusof theomnidirectional image,willbedivided intoMseparatedparts, and theangleθwillbedividedintoNpartsasfollows:

ρij

i radiusM

= × (16.2)

θij

j 36

N= × 0°

(16.3)

andthecalculationwillbeprocessed,bytakingthefirstpointwherei=1andj=1,whichgivesρ11=radius/Mandθ11=360°/N.Thevalueofρijwillthenbesubstitutedintothe“radialstretchingfunction”inordertoobtaintheparticularradiusatthatparticularlandmarkpoint.Thisradiusobtainedwillthenbesubstitutedintotheequationasfollowstoberoundedup,inordertogetthecorrespondingcoordinatesintheomnidirectionalimage:

v r cos= θ (16.4)

u r sin= θ (16.5)

wherevanducorrespondtothex-andy-coordinatesoftheomnidirectionalimage.Thiscoor-dinate(u,v)obtainedisinsertedintothepano-mappingtableTMN=Tij.TheuandvwillthenbeprocessedforNtimesbyincreasingjforNtimestoobtaindifferentangles,θ,tolaterdetermineall the coordinates corresponding to the value of landmarkpoint.These coordinates obtainedareinsertedintothetableofi=1withtheircorrespondingj=1toj=N,andtheiwillthenbeincreasedby1,andtheprocessisrepeatedforj=1toj=Ntodetermineallcoordinatesrelatedtoi=2.ThisiwillberepeatedforMtimes,andatableofM×Nentrieswithallthecoordinatescanbegenerated.Thecoordinatesineachoftheentriesaretakenonebyone,inordertomapeachandeverypixelintheomnidirectionalimagewiththecoordinateinthecurrententry,intoanewpanoramicimage.Theconversioniscompletedupontheendofmappingofthetable.

K13920_C016.indd 434 1/4/2013 7:24:19 PM


16.3.3 Log-Polar Mapping MethodLog-polarmappingisatypeofspatiallyvariantimagerepresentationwherebypixelseparationsincreaselinearlywithdistance.ItenablestheconcentrationofcomputationalresourceonanROIaswellasmaintainingtheinformationfromawiderview.Thismethodisimplementedbyapply-inglog-polargeometryrepresentations.ThecapturedomnidirectionalimagewillfirstbesampledbyspatiallyvariantgridfromaCartesianformintoalog-polarform.Thespatiallyvariantgridrepresenting log-polarmappingwill thenbe formedby inumberof concentric circleswithNnumberofsamples,andtheomnidirectionalimagewillthenbeunwarpedintoapanoramicimageinanotherCartesianform.

Thismethodispracticallyusedinrobustimageregistration[43],orinroboticvision,particu-larlyinvisualattention,targettracking,egomotionestimation,and3Dperception[44],aswellasinvision-basednavigation,environmentalrepresentations,andimaginggeometries[45],byJoséSantos-VictorandAlexandreBernardino.Inlog-polarmappingmethod,thecenterpixelforlog-polarsamplingiscalculatedby

ρ x y x x y yi i i c i c,( ) = −( ) + −( )2 2

(16.6)

θ

πx y

N y yx x

i ii c

i c

, tan( ) =

−−

−

21 (16.7)

andthecenterpixelforlog-polarmappingiscalculatedby

x xo cρ θ ρ θ, cos( ) = + (16.8)

y yo cρ θ ρ θ, sin( ) = + (16.9)

wherexc,ycarethecenterpointsofouroriginalCartesianformcoordinateNisthenumberofsamplesineachandeveryconcentriccircletaken

Theoriginal(xi,yi)inCartesianformissampledintolog-polarcoordinateof(ρ,θ),asshowninFigure16.17.Thecenterpointiscalculatedbyusing(16.6)and(16.7)togettherespectiveρandθ,whichcoveraregionoftheoriginalCartesianpixelsofradius:

r brn n= −1 (16.10)

and

b

NN

= +−

ππ

(16.11)

whereristhesamplingcircleradiusbistheratiobetweentwoapparentsamplingcircles

Figure16.18showsthecircularsamplingstructureandtheunwarpingprocessdonebyusingthelog-polarmappingmethod[43].Themeanvalueofpixelswithineachandeverycircularsampling

K13920_C016.indd 435 1/4/2013 7:24:20 PM


iscalculatedandwillbeassignedtothecenterpointofthecircularsampling.Theprocesswillthencontinuebymappingthemeanvalueatlog-polarpixel(ρ,θ)intoanotherCartesianformusingEquations16.8and16.9,andtheunwarpingisdoneattheendofmapping.

16.3.4 Performance EvaluationThis subsection reports the performance evaluation for different unwarping methods. Fewimportantfactorsareselectedfortheperformanceevaluationoftheunwarpingmethods.Thesefactorsincluderesolutionoftheimagegenerated,qualityofimage,algorithmusedinperform-

A

Center ofimage

AB

BB΄ B΄

A΄A΄

y2

x2

θ

ρ

Figure 16.18 Circular sampling structure and the unwarping process.

Capturedimage

y1

x1Sampling Mapping

x2

y2

N

ρ

ρθ (0,0)

Figure 16.17 Process of log-polar mapping.

K13920_C016.indd 436 1/4/2013 7:24:21 PM


ingtheunwarpingprocess,complexity,processingtime,anddatacompression.SomecapturedomnidirectionalimagesasshowninFigure16.19willbeusedtotesttheunwarpingmethods.

1.Resolution of the image generated:Theresolutionofeachgeneratedpanoramicimageusinglog-polarmappingmethod,DGTs, andpano-mapping tablemethod isdiscussed in thissubsection.Thelog-polarmappingmethodprovidessmallerresolutionofdimensionthatequalsto1/4-foldoftheomnidirectionalimage,whereasfortheDGTmethodandpano-mapping tablemethod, the resolutionof thepanoramic imageproducedcanbe as largeasthelengthoftheperimeteroftheomnidirectionalimage,withthewidthequalstotheradiusoftheomnidirectionalimage.However,duetotheimagesbeingrescaledforviewingpurposes,thedifferenceisnotobviousinthischapter.

2.Quality of image: Since the images are rescaled, thedifference inquality isnot apparentaswell.However,pano-mappingtablemethodisfoundtoproducethehighestqualityofimage,followedbythelog-polarmappingmethod,andtheDGTmethodcorrespondinglyindescendentqualityorder.

(a-1) (a-2)

(b-2)

(c-2)

(d-2)

(b-1)

(c-1)

(d-1)

Figure 16.19 Performance evaluation (a-1, a-2). Samples of omnidirectional images (b-1, b-2). Panoramic images generated using DGT method (c-1, c-2). Panoramic images generated using pano-mapping table method (d-1, d-2). Panoramic images generated using log-polar method.

K13920_C016.indd 437 1/4/2013 7:24:22 PM


3.Algorithm used in performing the unwarping process: In log-polar mapping algorithm, theomnidirectionalimageisconsideredintheformofanumberofsectorsinwhicheachsectorconsistsofagroupofpixelsthatwillbeextractedlaterinsectorbysectortobearrangedintoarectangularformofimage,whereasfortheDGTmethod,pixelbypixelistobeextractedandarrangedintoarectangularformimage.Thesepixelswillthenbereproduced,ordupli-cated,inordertostandardizethenumberofpixelsavailableineachrowofthepanoramicimage.Forthepano-mappingtablemethod,analgorithmisusedwherebyatableiscreatedatinitialization,toindicatethecoordinatesofthepixelstobeextractedfromtheomnidirec-tionalimage.Oncethetableiscreated,itwillthenbeusedoverandoveragaintomapeachofthepixelatthatparticularcoordinate,onebyone,fromtheomnidirectionalimageintoapanoramicimage,hencethename“onceandforall.”

4.Complexity:Table16.1showsthebig-Ocomplexityof log-polarmappingmethod,DGTmethod,andpano-mappingtablemethod.

5.Processing time:Theprocessingtimeforallthethreeunwarpingmethodstotransformanomnidirectional image into a panoramic image is calculated using MATLAB function“cputime.”Theprogram isprocessedfive timesonfivedifferent images, and theaverageprocessingtimeiscomputed.Itisfoundthatpano-mappingtablemethodhasthefastestcomputationtime,whichis1.220s,followedbylog-polarmappingmethodbeing2.003sand3.426sfortheDGTmethod.

6.Data compression:Thegeneratedpanoramicimageproducedbylog-polarmappingmethodhastheresolutionof473×114;DGTmethodhasaresolutionof1472×235and1146×243forpano-mappingtablemethod,inwhichtheoriginalomnidirectionalimageisofresolu-tion473×473.Fromtheoutputresolution,itisclearthatlog-polarmappinghasthehighestcompression,whichcompressestheimageuptofourfold,comparedtoDGTmethod(0.65-foldimageexpansion)andpano-mappingtablemethod(0.80-foldimageexpansion).

Intermsofresolution of the image generated,althoughtheimagegeneratedbyDGTmethodandpano-mappingtablemethodsislargerascomparedtotheimagegeneratedbylog-polarmappingmethod, these twomethods seemtoelongate theactual sizeof the image. Inotherwords, this

Table 16.1 Big-O Complexity

DGT Log-Polar Mapping

Pano-Mapping Table

AdditionO(XY2) O(X2Y2) O(Y2)

Subtraction

MultiplicationO(Y) O(X2) O(Y2)

Division

Logarithmic —O

log XY

( )( )

log

—

X, length of the panoramic image = perimeter of the omni-directional image taken into consideration’ Y = height of the panoramic image = radius of the omnidirectional image taken into consideration.

K13920_C016.indd 438 1/4/2013 7:24:22 PM


methodtendstomaketheobjectsintheimageextendedand“broader”thantheoriginalimage.Duetothiselongation,itwillbehardertoexaminethepictureandtheobjects,asthesenseofthesizehadbeeneliminated.Forlog-polarmappingmethod,theextensionisnotmuch,anditisnotasobviousasDGTmethodandpano-mappingtablemethods.Intermsofquality of image,pano-mappingtablemethodproducesthehighestqualityamongthethreemethods,followedbylog-polarmappingmethodwithaslightlylowerimagequalitybutstillwithinanacceptablerange,andlastlytheblurredDGTmethod.Intermsofalgorithm used in performing the unwarping process,pano-mappingtablemethodusesthesimplestandeasiestalgorithm,followedbyaslightlycomplexalgorithmthatisthelog-polarmethod,andlastly,acomplicatedandcomplexalgorithmfromtheDGTmethod.Intermsofcomplexity,itisfoundthatpano-mappingtablemethodhastheleastcomplexity,followedbyDGTmethod,andlastlylog-polarmappingmethodinbig-Onotation.Intermsofprocessing time,onaverage,pano-mappingtablemethodhasthefastestprocessingtimetotransformanomnidirectionalimageintoapanoramicimage,followedbylog-polarmappingmethodandDGTmethod.Intermsofdata compression,log-polarmappingmethodhasthebestdatacompressionratecomparedtopano-mappingtablemethodandDGTmethod.ThisisverygoodinpreservingCPU’smemory,asthememoryavailableisusuallyverylimited.

16.4 Automatic Human Intruder Detection AlgorithmAutomatichumanintruderdetectionis implementedintheproposedomnidirectional imagingsystemtoanalyzeinformationfromthepositionoftheimagingtoolsandautomaticallydetectatrespasser.Twoautomatichumanintruderdetectionalgorithmsarediscussedinthissubsection;thisincludespartitionedROIalgorithm[11]andhumanheadcurvetestalgorithm[12,13].

16.4.1 Partitioned Region of Interest AlgorithmThepartitionedROI-basedhumanintruderdetectionalgorithmissummarizedasfollows:

Step1:Adjustthethermalcameradetectionrangingfrom30°Cto40°Csothatobjectwithhumanbodytemperaturerangecanbedetected.

Step2:Unwarptheomnidirectionalthermalimageintopanoramicthermalimageusinglog-polarmappingtechnique.

Step3:CaptureimagescontinuouslyfromthermalcameraintolaptopandnameitasPxwherex=1,2,3,…isthediscrete-timeinstant.

Step4:Divideeach imagecaptured fromthermalcamera into (m×n) regions.Each regionconsistsofequalnumberofpixels.

Step5:DefineamatrixMwithsizeof(m×n)torepresentthecharacteristicofeachcorrespond-ingregion.

Step6:DefineathresholdvalueQ.QisthethresholdvalueofthedifferencebetweensumsofRGBvalueforaparticularcurrentimagepixelandpreviousimagepixel.

Step7:DefineavariablehforcountingthenumberofpixelsexceedingQ.Initially,h issetto0.

Step8:DefineHasaminimumnumberofpixelswithdifferenceexceedingQ.Step9:ComparecurrenttakenimagePxwithprevioustakenimagePx−1.Foreachcorrespond-

ingregion,findoutthedifferencebetweenaparticularcurrentimageandpreviousimagepixels’sumofRGBvalues.IfthedifferencebetweensumsofRGBvaluesforaparticular

K13920_C016.indd 439 1/4/2013 7:24:22 PM


currentimagepixelandpreviousimagepixel≥Q,thenh=h+1.Ifh≥H,marka“1”intothecorrespondingelementofM;elseifh<H,marka“0”intothecorrespondingelementofM.AnexampleisshowninFigure16.20.

Step10:LetFbethenumberofdifferentelementsthatalignverticallyandcontinuously.SomeexamplesofcalculationofFareshownasfollows:

Examples

Example 1

Compare

0 0 0 0 00 0 0 0 00 0 0 0 00 0 0 0 00 0 0 0 0

0 0 0 0 00

with11 0 0 0

0 1 0 0 00 1 0 0 00 1 0 0 0

F=4inthisexample.Example 2

Compare with

0 0 0 0 00 0 0 0 00 0 0 0 00 0 0 0 00 0 0 0 0

0 0 0 0 00

11 0 0 00 0 0 1 00 0 0 1 00 0 0 1 0

F=3inthisexamplebecauseonlythreedifferentelementsarealignedverticallyandcontinuously.Example3

Compare with

0 0 0 0 00 0 0 0 00 0 0 0 00 0 0 0 00 0 0 0 0

0 0 0 0 00

11 0 0 00 1 0 1 00 0 0 1 00 0 0 1 0

Iftherearemorethantwogroupsofverticallyandcontinuouslydifferentelements,thenwewilltakethelargestnumber.Inthiscase,F=3.

Step11:DefineGasminimumregionsthatahumanbeingwillappearon-screen.

IfF≥G,thenalarmunknowntrespasserdetected.

Divide ROI into (m × n)regions

00000

00000

00000

00000

00000

00000

00000

00000

00000

00000

Figure 16.20 Example for partitioning of ROI for trespasser detection surveillance system.

K13920_C016.indd 440 1/4/2013 7:24:23 PM


16.4.2 Human Head Curve Test AlgorithmThehumanheadcurvetestalgorithmissummarizedasfollows:

1.Algorithm for Trespasser DetectionStep1:Acquirethermalimagethroughhyperbolicreflector.RefertoFigure16.21forthe

exampleofimagecaptured.Step2:Imageunwarping:Unwarptheacquiredthermalimageintopanoramicimage.Refer

toFigure16.22fortheexampleonresultingpanoramicimage.Step3:Imagecropping:Croptheimagetoobtainthethermalimageoftheinterestedarea

only.PleaserefertoFigure16.23fortheexampleontheresultingcroppedimage.Step4:Binaryimageconversion:Convertthermalimageintopurelyblackandwhiteimage

(BW)using

BW i, j

1, T1 Temp i, j Th0, otherwise

( ) ( )

=

≤ ≤ (16.12)

wheretemperatureatpoint(i,j)ofthermalimageTlandTharetheminimumandmaximumpossiblehumanbodytemperaturesi,jarethepixel’srowandcolumncoordinates,respectively

Step5:Objectidentification:IdentifyobjectsinsidetheBWwhereagroupofdiscontinuouswhitepixelsisconsideredasasingleobject.

Step6:Noisefiltering:a. RemovefromBWallconnectedcomponents(objects)thathavefewerthanSpixels.b. Createaflat,disk-shapedstructuringelement(SE)withradius,R.AnexampleofSE

isshowninFigure16.24.PerformmorphologicalclosingontheBW.Morphologicaloperationdilatesan

imageandthenerodesthedilatedimageusingthesameSEforbothoperations.

AQ22

AQ23

Figure 16.21 Thermal image captured through hyperbolic reflector.

K13920_C016.indd 441 1/4/2013 7:24:23 PM


Step7:Boundaryextraction:FindboundarylineofeachidentifiedobjectinsideBWandrecorditintoanarrayofcoordinates.

Step8:Headdetection:Foreachobject’sboundary,performtheheaddetectionalgorithmasexplainedin“AlgorithmforHeadDetection.”

Step9:Ifahumanbeingisdetected,triggerthealarm.

SE =

R = 3

0 0 0 11111

1

1

0 0 000

00

0

0

111110

111111

111110

111110

001000

Origin

Figure 16.24 Disk-shaped SEs (e.g., R = 3).

Figure 16.23 Thermal image after cropping process.

Figure 16.22 Panoramic view of the inspected scene after the thermal image is unwarped.

K13920_C016.indd 442 1/4/2013 7:24:24 PM


2.Algorithm for Head DetectionStep1:Startingpointidentification:Calculatethestartingpointforheadtopdetection.The

startingpointshallbethehighestpoint(smallesty-coordinatevalue)amongtheintersectionpointsoftheboundarywiththehorizontalmiddleline,wherehorizontalmiddlelineisgivenbyx=xmandxmisthemeanofallx-coordinatesintheboundaryoftheobject.PleaserefertoFigure16.25aforbetterunderstanding.

Step2:Peakpointdetection:Fromthestartingpoint,followtheboundaryinclockwisedirec-tionandsearchforthefirstpeakpointencountered(Ppeak1).Again,fromthestartingpoint,followtheboundaryinanticlockwisedirectionandsearchforthefirstpeakpointencoun-tered(Ppeak1).Peakpointisdefinedasthepointinwhichithasthesmallesty-coordinatevaluecomparedtoalltheDproceedingpoints.Disthenumberofnextofpointtobetested.RefertoFigure16.25bforbetterunderstanding.

Step3:Headtoppointdetection:ComparePpeak1andPpeak2obtainedinstep2.Recordthehighestpoint(withsmallery-coordinate)astheheadtoppoint,Ppeak1.Forexample,inFigure16.25b,Ppeak2ishigherthanPpeak1.Thus,Ppeak1=PHT=Ppeak2.

Step4:Boundarylinesplitting:Splittheboundaryintoleftboundary(Bl)andrightboundary(Br)fromtheheadtoppointtowardthebottom.Takeonlyonepointforeachy-coor-dinatetofilteroutunwantedinformationsuchasraisedhands(refertoFigure16.26forbetterunderstanding):

Bl xl yl Br xr yri i i i= [ ] = [ ], , , (16.13)

wherexli,yli,xri,yriarethepixels’y-coordinatesandx-coordinatesforBlandBr,respectivelyi=1,…,NistheindexnumberNisthesizeoftheboundarymatrix(NisthenumberofpixelsforBlandBr)

Starting point

Horizontalmiddle line

Startingpoint

Ppeak1

Ppeak2 (PHT)

X

Y

(a) (b)

Figure 16.25 (a) Horizontal middle line and the starting point as in step 1. (b) Detection of (c/w from starting point) and (counter c/w from starting point).

K13920_C016.indd 443 1/4/2013 7:24:26 PM


Step 5: Left significant point detection: Search downward along Bl from PHT for thefirstleftmostpointencountered(Plp).Next,searchfortherightmostpointrightafterPlpwhichisPld(refertoFigure16.27forbetterunderstanding):

Pl xl yl Pl xl ylp lp lp d ld ld= =( , ), ( , ) (16.14)

wheresubscriptlpisanindexnumberofthefirstleftmostpointsubscriptldisanindexnumberoftherightmostpointrightafterPlp

PHT

Prd

Prp

Pld

Plp

B(left)B(right)

Figure 16.27 Example of points found in steps 3, 5, and 6.

Left boundaryRight boundary

(a) (b)

Figure 16.26 (a) Original object boundary. (b) Left and right object boundary after the splitting process (step 4).

K13920_C016.indd 444 1/4/2013 7:24:26 PM


Step6:Rightsignificantpointdetection:SearchdownwardalongBrfromPHTforthefirstrightmostpointencountered(Prp).Next,detecttheleftmostpointrightafterPrp,whichisPrd(refertoFigure16.27forbetterunderstanding):

Pr xr yr Pr xr yrp rp rp d rd rd= =( , ), ( , ) (16.15)

wheresubscriptrpisanindexnumberofthefirstrightmostpointsubscriptrdisanindexnumberofthefirstleftmostpointrightafterPrp

Step7:Headsymmetrictest:Definehl=verticaldistancebetweenPHTandPldhr=verticaldistancebetweenPHTandPrd.

Testtheratiobetweenhlandhr.Ifhl/hrorhr/hl>2,thentheobjectisnotconsideredasahumanbeingandthenextsubsequentstepsinthisalgorithmcanbeskippedandproceedwiththenextobject.Else,iftheobjectispossiblyahumanbeing,continuestep8forfurtherdetection.

Step8:Neck–bodyposition test:Calculate∆x,which is thedistancebetweenxcandxmwherexc=horizontalcenterbetweenPldandPrdandxmisobtainedinstep1.Definewn=horizontaldistancebetweenPldandPrd.

If∆x≥2wn,thenthisobjectisnotclassifiedasahumanbeingandthenextsubsequentstepsinthisalgorithmcanbeskippedandproceedwiththenextobject.Elseif∆x<2wn,thentheobjectispossiblyahumanbeing.Continuestep9forfurtherdetection.

Step9:Curvetests:a. Topcurvetest

Definest=floor(min(lp,rp)/(F/2))asthestepsizefortopcurvetest.

Calculate

C

1, yl yl

0, otherwisetl

1 s k 1 S (k 1)t t=≤+ ∗ + ∗ +

C

1, yr yr

0, otherwisetr

1 s k 1 S (k 1)t t=≤+ ∗ + ∗ +

(16.16)

C C Ct tl tr= ∑ + ∑

wherek=0,…,F/2–1Fisthestep-sizepartitionvariable.FisevenintegerandF≥2.For example, if F=6, st=8, then the y-coordinates tested are shown in Figure

16.28.Thesameconceptgoesforleftcurvetestandrightcurvetest.

Note:Thesymbol“*”meansmultiply.b. Leftcurvetest

Definesl=floor(min(lp,ld–lp)/(F/2))asthestepsizeforleftcurvetest.

K13920_C016.indd 445 1/4/2013 7:24:26 PM


Calculate

C =

1, xl xl

0, otherwisell

lp s k lp S (k 1)t l+ ∗ + ∗ +≤

C =

1, xl xl

0, otherwisel2

lp s k lp S (k 1)t l+ ∗ + ∗ +≤

(16.17)

C C Cl l1 l2= ∑ + ∑

wherek=0,…,F/2−1c. Rightcurvetest

Definesr=floor(min(rp,rd–rp)/(F/2))asthestepsizeforrightcurvetest.

Calculate

C

1, xr xr

0, otherwiserl

rp + s k rp S (k 1)r r=≤∗ + ∗ +

C

, xr xr

0, otherwiser2

p s k rpr=≤

+ +1 r + ∗ ∗S (k 1)r (16.18)

C C Cr r1 r2= ∑ + ∑

wherek=0,…,F/2–1Step10:Humanidentification:

Definecurvetestcondition:

C F 1t ≥ − (16.19)

C F 1l ≥ − (16.20)

C F 1r ≥ − (16.21)

Checkconditions(16.19)through(16.21).Ifanytwoormoreconditionsaretrue,theobjectisverifiedasahumanbeing.Else,theobjectisnotconsideredasahumanbeing.

yl25

yr25yl17

yr17yl9 yl1

yr1 yr9

Figure 16.28 Example on top curve test.

K13920_C016.indd 446 1/4/2013 7:24:28 PM


16.4.3 Experimental ResultsIn this section, the application of the proposed omnidirectional human intruder detectionsystem is briefly illustrated. An omnidirectional image captured using digital camera onthe site is shown in Figure 16.29. An omnidirectional thermal image also captured usingthermal camera on the site is shown in Figure 16.30. The unwarped form of Figure 16.29

Figure 16.29 Case studies of trespasser detection (digital color form).

Figure 16.30 Case studies of trespasser detection (thermal image).

K13920_C016.indd 447 1/4/2013 7:24:29 PM


(digitalcolorpanoramicform)isshowninFigure16.31,whereastheunwarpedformofFigure16.30(thermalimagepanoramicform)isshowninFigure16.32,respectively.InFigure16.32,thelog-polarmappingprocessisby4:1reductionmappingscale,whichmeansthat320×240omnidirectional thermal image’s Cartesian pixels are mapped to one-fourth of the thermalimage Cartesian pixels (320×60) in panoramic view, with fourfold data compression com-pared to original omnidirectional thermal image as inFigure 16.30.The captured thermalimagesaretestedfortwotrespasserfaintdetectionalgorithmsasproposedinSections4.1and4.2intheprecedingtext.

16.4.3.1 Experimental Results for Partitioned ROI-Based Human Intruder Detection Algorithm

InpartitionedROIalgorithmfortrespasserdetection,therearethreeparametersthatneedtobeoptimized,whichareQ,H,andG,whereQisthethresholdvalueofthedifferencebetweensumofRGBvalueforaparticularcurrentimagepixeltopreviousimagepixel,HisminimumnumberofpixelswithdifferenceexceedingQandGisminimumregionsthatahumanbeingwillappearon-screen.

SincetheimagecapturedisinRGBform,thedifferenceofthesumofRGBvaluesbetweenaparticularcurrentimagepixelandpreviousimagepixelisbetween0and765.ForQparameter,1000sampleimages(withorwithouthumanbeing)areusedtotesteverydifferentpointwithstepsizeof15.TheaccuracyversusdifferenceofsumofRGBvaluesisplottedinFigure16.33.Fromtheplot,theoptimumQvalueis345withhighestaccuracyof95.30%.

AQ24

Figure 16.31 Unwarp form of Figure 16.29 (digital color panoramic form).

Figure 16.32 Unwarp form of Figure 16.30 (thermal image panoramic form).

K13920_C016.indd 448 1/4/2013 7:24:30 PM


Theunwarpedpanoramicimageispartitionedinto50regions(m=10,n=5)witheachregionconsistingofequalnumberofpixels(384).AsforHvalue,thealgorithmwithpets(hamster,cat,anddog)andhuman,movingtowardandawayfromthecapturedregion.Onethousandsampleimagesarecaptured.Byusingthesampleimages,werepeatedthesimulationwithH=10%,20%,30%,…,100%ofnumberofpixeldifferencetototalpixelsinoneregionratio.ThegraphaccuracyversusnumberofpixeldifferencetototalpixelsinoneregionratioisplottedinFigure16.34.Fromtheplot,theoptimumHvalueis50%oftotalpixelsinaregion,withthehighestaccuracyof97%.

AsforGvalue,thealgorithmistestedwithhumanmovingtowardandawayfromthecap-tured region with minimum regions that a human being will appear on-screen, G=1–5. Thegraphofaccuracyversusminimumregionsthatahumanbeingappearson-screenGisshowninFigure16.35.Fromthegraph,theoptimumGvalueis3withhighestaccuracyof93.5%.

For testing the trespasserdetectionperformanceofpartitionedROI-basedtrespasserdetec-tion algorithm, a total of10,000 imageswith test subjects (humanbeingor animal) roamingrandomlyinthetestsite(asshowninFigure16.37)visibletotheproposedsystemaretakenassamples.Thisincludesthermalimageswithasingletrespasser,morethanonetrespasser,withoutatrespasser,andanimals(cats,birds,etc.,whicharenotcountedastrespassers).The“operatorperceivedactivity”(OPA)[46]isusedandtheoperatorwillcommentontheimagescaptured,

AQ25

100.00%90.00%80.00%70.00%60.00%

40.00%50.00%

30.00%20.00%10.00%

Accuracy vs. difference of sum of RGB values

0.00%0 60 120 180 240 300 360 420 480 540 600 660 720

Figure 16.33 Accuracy versus difference of sum of RGB values.

Accuracy vs. number of pixel differenceto total pixels in one region

1

0

0.5

20% 30%10% 40% 50% 60% 70% 80% 90% 100%

Figure 16.34 Accuracy versus number of pixel difference to total pixels in one region.

K13920_C016.indd 449 1/4/2013 7:24:31 PM


whetherthereisanytrespasserornot,andcomparewiththedetectedresultofthesurveillancesystem.Fromthe totalof10,000samples images forevaluation,7,080weredetectedperfectly(trespasser-or-not condition agreedbybothobserver and surveillance system), that is,with anaccuracyof70.8%.

16.4.3.2 Experimental Results for Human Head Curve Test Algorithm

Thesame10,000testedsamplesasmentionedinSection16.4.3.1areappliedhere.TodetermineoptimumvalueforparameterTl,arandomsampleimageischosenandconvertedintoBWimageusingstep2ofthehumanheadcurvetestalgorithmwithvalueofTlrangingfrom0to510(sumofRandGcomponentsinRGBimage).Performthebinaryimageconversionrepeatedlywithincreasingstepsizeof10forTlandsearchfortheoptimumTlwherethenoisecanbeminimizedandthehumanshapeisnotdistortedintheresultingimage.Basedon1000observableimages,Tlisbestsuitablesetat150inthisexperiment.Foranexample,ifTl=130isused,excessivenoisewillbeintroduced.IfTl=170isused,therewillbetoomuchdistortiontohumanbeingintheresultingimage.RefertoFigure16.36forbetterunderstanding.

Todetermine theoptimumvalue forparameterTh,performthebinary imageconversionrepeatedlywithdecreasingstepsizeof10forThandsearchfortheminimumvalueofThthatdoesnotinfluencetheappearanceofthehumanobject.Basedon1000observableimages,Thisbestsetat430inthisexperiment.Forexample,ifTh=400isused,theimageofhumanbeingisdistorted. IfTh=460 isused, therewillbeno improvement for the image.LowerThvalueis preferredbecause itwill filter outmorenoise component.Refer toFigure16.37 forbetterunderstanding.

Humanshape’sparametersSandRareapproximatedfrom1000testingimagesatadistanceof5mfromtheimagingsystem.Fromthoseimages,thehuman’spixelsareapproximately30.Hence,Sissetto30.LargerSEspreservelargerfeatures,whilesmallerelementspreservethefinerdetailsofimagefeatures.Figure16.38showsexampleswithdifferentRvaluesforSEselection.ItisobservedthatwhenR≥3,theneckpartisunidentifiedfromtheimages.Sincefinerhumanheadshapeisofconcern,RforSEisbestsuittosetattheminimum,whichis2,aswellaswithlowercomputationalcomplexity.

Theaccuracyoftheproposedalgorithmisthenevaluatedusing“OPA”inwhichthepro-posedalgorithmisevaluatedwithrespecttotheresultsinterpretedbyahumanobserver[46].Firstly, the panoramic images are tested using the proposed algorithm. Then, the result is

60.00%

40.00%

20.00%

0.00%1

Accuracy vs. minimum regions that a humanbeing will appear on screen

2 3 4 5

80.00%

100.00%

Figure 16.35 Accuracy versus minimum regions that a human being will appear on-screen.

K13920_C016.indd 450 1/4/2013 7:24:31 PM


(a) (b) (c)

Figure 16.36 (a) T1 = 130, (b) T1 = 150, and (c) T1 = 170.

(a) (b) (c)

Figure 16.37 (a) Th = 400, (b) Th = 430, and (c) Th = 460.

K13920_C016.indd 451 1/4/2013 7:24:32 PM


comparedwith the resultof thehumanobserver.Theaccuracyof theproposed algorithm isthepercentageofinterpretation(trespasserornot)agreedbyboththehumanobserverandtheproposedalgorithm.

TodetermineparameterDandF,allofthe10,000sampleimagesaretestedwithdifferentcombinationsofDandF.AsshowninthegraphinFigure16.39,theoptimumvaluesforparam-etersDandFare7and2,whichcontributetoaccuracyof81.38%.

16.4.4 Comparison between Two Proposed Human Intruder Detection Algorithms

ThefirstproposedalgorithmthatisthepartitionedROI-basedhumanintruderdetectionalgorithmisregionalbasedwherebyanobjectthatoccupiesmorethanacertainnumberofpartitionsinapanoramicimageisconsideredasahumanbeingandviceversa.However,thealgorithmhastwo

79.00%

75.00%73.00%71.00%69.00%67.00%65.00%

3 4 5 6

Accuracy vs. D and F parameters

7

Acc

urac

y

D parameter8 9 10 11

F = 2

F = 4

F = 6

F = 8

F = 10

F = 12

77.00%

81.00%

Figure 16.39 Accuracy of proposed algorithm for different combinations of D and F.

(a) (b) (c) (d) (e)

Figure 16.38 Examples with different R values for SE selection: (a) R = 1, no changes; (b) R = 2; (c) R = 3; (d) R = 4; and (e) R = 5.

K13920_C016.indd 452 1/4/2013 7:24:34 PM


majorconcerns.Firstconcernisthedistance,inwhichahumanbeingthatisfarawayfromtheimagingsystemwillnotbeidentifiedasahuman.Thesecondconcernisifananimal(suchascat,dog)ismovingtooclosetothesystem(whichoccupiesmorethanthethresholdpartitioned),itwillbemissconsideredasahumanbeingtoo.

Hence,asecondeffectivehumanintruderdetectionalgorithm,whichisthehumanheadcurvetestalgorithmwithhumanheaddetectioncapability,isproposed.Bycomparingthetwohumanintruderdetectionalgorithms,humanheadcurvetestalgorithmrequiredcomplicatedheadsym-metrictestandcurvetest.PartitionedROI-basedhumanintruderdetectionalgorithmissuper-seding human head curve test algorithm in terms of simplicity and lower computational timeconsumption(averageroutinetimeforprocessingonesampleis1.3sforpartitionedROI-basedhumanintruderdetectionalgorithmand2.27sforhumanheadcurvetestalgorithm).However,intermsofefficiency,humanheadcurvetestalgorithmwithanaccuracyof81.38%ishigherthanpartitionedROI-basedhumanintruderdetectionalgorithmwithanaccuracyof70.8%inthesamesetof10,000testedimages.

16.5 Conclusion and Future Research DirectionsThis chapter presented omnidirectional human intrusion detection system using computervision techniques. Two imaging methods, namely, vision spectrum imaging and IR imag-ing,areappliedincomputervision-basedomnidirectionalhumanintrusiondetectionsystem.Simulationresultsshowthatlog-polarmappingproposedintransformingthecapturedomnidi-rectionalimagesintopanoramicformhasgoodqualityinoutputimagewithhighdatacompres-sionrateandfastprocessingspeedinprovidingobserverorimageprocessingtoolsawideangleof view. Automatic human intrusion detection algorithms are implemented in the proposedomnidirectionalimagingsystem,bothinvisionspectrumimagingandinIRspectrumimaging,respectively.TheproposedhumanintrusionalgorithmincludespartitionedROIalgorithmandhumanheadcurvetestalgorithm.ExperimentalresultsalsoshowthatpartitionedROI-basedhumanintruderdetectionalgorithmissupersedinghumanheadcurvetestalgorithmintermsof simplicity and lower computational time consumption. However, human head curve testalgorithmcantraceouthumanintruderfromthepanoramicimagesmoreaccuratelycomparedtoROIalgorithm.

Currently the omnidirectional human intrusion detection systems are applied in indoorbuildingsecurityfori-habitat(smarthome),fossilpowerplant,etc.,andprototypingforborderintrusiondetection,onhumantargets(includingsmugglers,illegalimmigrants,orterrorists).Inthefuture,itwillbeembeddedwithfacialrecognitioncapabilitiestorecordandidentifycrimi-nals’andsuspects’identity.Also,amobilerobotcanbebuiltformovingaroundthesurveillancesitecarryingsuchomnidirectionalsurveillancesystem.Theimagingtoolpowerisdesignedtobesuppliedbyabatteryinsteadofapowerplug.Itallowstherobottocarrythesurveillanceimagingtoolsetwithoutlimitationofthepowercables’length.Byusingamobilerobot,severalsitescanbemonitoredbyusingonlyoneomnidirectionalsurveillancesystem.Itisalsoaplantoemploymicroprocessor modules such as field programmable gate array (FPGA) and Advanced RISCMachine(ARM)forimageprocessingandanalyzingtasksinsteadofacomputertoeffectivelyreducethecostsandpowerconsumptionoftheproposedsystem.Thesetopicswillbeaddressedinfutureworks.

AQ26

K13920_C016.indd 453 1/4/2013 7:24:34 PM


References 1. Moseley,T.M.2006.Homelandoperations.AirForceDoctrineDocument2-10. 2. Lu,C.andDrew,M.S.2007.Automaticcompensationforcamerasettingsforimagestakenunderdif-

ferentilluminants.Technicalpaper,SchoolofComputerScience,SimonFraserUniversity,Vancouver,BritishColumbia,Canada,pp.1–5.

3. Kawanishi,T.,Yamazawa,K.,Iwasa,H.,Takemura,H.,andYokoya,N.1998.Generationofhigh-resolution stereo panoramic images by omnidirectional imaging sensor using hexagonal pyramidalmirrors.InProceedings of the 14th International Conference in Pattern Recognition,Brisbane,Australia,Vol.1,pp.485–489.

4. Akihiko,T. andAtsushi, I.2004.Panoramic image transformofomnidirectional imagesusingdis-cretegeometrytechniques.InProceedings of the 2nd International Symposium on 3D Data Processing, Visualization, and Transmission (3DPVT’04),Washington,DC.

5. Jeng,S.W.andTsai,W.H.2007.Usingpano-mappingtablesforunwarpingofomni-imagesintopan-oramicandperspective-viewimages,IET Image Processing,1(2),149–155.

6. Jurie,F.1999.Anewlog-polarmappingforspacevariantimaging:Applicationtofacedetectionandtracking,Pattern Recognition,32(55),865–875.

7. Hampapur, A., Brown, L., Connell, J., Pankanti, S., Senior A. et al. 2003. Smart surveillance:Applications, technologies and implications, Information, Communications and Signal Processing, 2,1133–1138.

8. Hampapur,A.,Brown,L.,Connell,J.,Ekin,A.,Haas,N.etal.2005.Smartvideosurveillance,IEEE Signal Processing Magazine,pp.39–51.

9. Green,M.W.1999.TheappropriateandeffectiveuseofsecuritytechnologiesinU.S.schools.Aguideforschoolsandlawenforcementagencies,ResearchReport,SandiaNationalLaboratories,Albuquerque,NM,NCJ178265.

10. Shu,C.,Hampapur,A.,Lu,M.,Brown,L.,Connell,J.etal.2005.IBMsmartsurveillancesystem(S3):Aopenandextensibleframeworkforeventbasedsurveillance,Advanced Video and Signal Based Surveillance (AVSS 2005),Como,Italy,pp.318–323.

11. Wong,W.K.,Tan,P.N.,Loo,C.K.,andLim,W.S.2010.Omnidirectionalsurveillancesystemusingthermalcamera,Journal of Computer Science and Engineering,3(2),42–51(ISSN2043-9091).

12. Lee,L.H.2008.Smartsurveillanceusingimageprocessingandcomputervisiontechniques,BachelorDegreethesis,MultimediaUniversity,Melaka,Malaysia.

13. Wong,W.K.,Chew,Z.Y.,Lim,H.L.,Loo,C.K.,andLim,W.S.2011.Omnidirectionalthermalimag-ingsurveillancesystemfeaturingtrespasserandfaintdetection,International Journal of Image Processing, CSC Journals, IJIP-279,4(6),518–538.

14. Holmes,O.W.1881.The Common Law.Boston,MA:LittleBrownandCo. 15. Tsai, C.F. and Young, M.S. 2003. Pyroelectric infrared sensor-based thermometer for monitoring

indoorobjects,Review of Scientific Instruments,74(12),5267–5273. 16. Hirota, M., Furuse,T., Ebana, K., Kubo, H.,Tsushima, K., Inaba,T., Shima, A., Fujinuma, M.,

andTojyo,N.2001.MagneticdetectionofasurfaceshipbyanairborneLTSSQUIDMAD,IEEE Transactions on Applied Superconductivity,11,884–887.

17. Paul,W.,Herbert,F.,andSoumya,N.2003.EnhancinghomelandsecuritywithadvancedUWBsen-sors,IEEE Microwave Magazine,pp.51–58.

18. http://www.rrs.co.za/products/homeland-security.html 19. http://www.detect-inc.com/security.html 20. Tocci,R.2006.Digital Systems: Principals and Applications,10thedn.UpperSaddleRiver,NJ:Prentice

Hall.ISBN013172. 21. Fischler,M.A.andBolles,R.C.1981.Randomsampleconsensus:Aparadigmformodelfittingwith

applicationstoimageanalysisandautomatedcartography,Communications of the ACM,24,381–395. 22. Corke,P.,Strelow,D.,andSingh,S.2004.Omnidirectionalvisualodometryforaplanetaryrover.In

International Conference on Intelligent Robots and Systems (IROS 2004),Sendai,Japan,4,4007–4012. 23. Durrant,W.H.andBailey,T.2006.Simultaneouslocalizationandmapping(SLAM):PartI.Theessen-

tialalgorithms,Robotics and Automation Magazine,13,99–110.

AQ27

AQ28

AQ29

K13920_C016.indd 454 1/4/2013 7:24:34 PM


24. Ishiguro,H.,Yamamoto,M.,andTsuji,S.1992.Omni-directionalstereo,IEEE Transactions on Pattern Analysis and Machine Intelligence,14(2),257–262.

25. Huang,H-C.andHung,Y.P.1998.Panoramicstereoimagingsystemwithautomaticdisparitywarpingandseaming,Graphical Models and Image Processing,60(3),196–208.

26. Peleg,S.andBen-Ezra,M.1999.Stereopanoramawithasinglecamera.InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition,FortCollins,CO,395–401.

27. Shum,H.andSzeliski,R.1999.Stereoreconstructionfrommulti-perspectivepanoramas.InProceedings of the Seventh International Conference on Computer Vision,Kerkyra,Greece,pp.14–21.

28. Chen,S.E.1995.Quick timeVR:An image-basedapproach tovirtual environmentnavigation. InProceedings of the 22nd Annual ACM Conference on Computer Graphics,LosAngeles,CA,pp.29–38.

29. Kumar,J.andBauer,M.2000.Fisheyelensdesignandtheirrelativeperformance,Proceedings of SPIE,4093,360–369.

30. Padjla,T.andRoth,H.2000.Panoramic imagingwithSVAVISCAcamera-simulationsandreality,ResearchReportsofCMP,CzechTechnicalUniversity,Prague,No.16.

31. Oh,S.J. andHall,E.L.1987.Guidanceofamobile robotusinganomnidirectionalvisionnaviga-tion system. InProceedings of the Society of Photo-Optical Instrumentation Engineers,SPIE,Vol.852,pp.288–300.

32. Kuban,D.P.,Martin,H.L.,Zimmermann,S.D.,andBusico,N.1994.Omniviewmotionlesscamerasurveillancesystem,UnitedStatesPatentNo.5,359,363.

33. Nalwa,V.1996.Atrueomnidirectionalviewer,TechnicalReport,BellLaboratories,Holmdel,NJ. 34. James,J.K.andMartin,B.2000.Fisheyelensdesignsandtheirrelativeperformance.InProceedings of

the Current Developments in Lens Design and Optical Systems Engineering,SPIE,Vol.4093,360–369. 35. Stun-ningsales.com, Weatherproof CCD color rotating video security camera, Redirecting from

http://www.stun-ningsales.com/homethings/outdoor_securitycameras.htm 36. The-DigitalImage.com,Fisheyelens,redirectingfromhttp://www.the-digital-picture.com/Reviews/ 37. Chahl,J.andSrinivasan,M.1997.Reflectivesurfacesforpanoramicimaging,Applied Optics,36(31),

8275–8285. 38. Gachter,S.2001.Mirrordesignforanomnidirectionalcamerawithauniformcylindricalprojection

whenusingtheSVAVISCAsensor,ResearchReportsofCMP,OMNIVIEWSProject,CzechTechnicalUniversity,Prague,No.3,2001.Redirectedfrom:http://cmp.felk.cvut.cz/projects/omniviews/

39. Svoboda,T.1999.Centralpanoramiccamerasdesign,geometry,egomotion.PhDtheses,CenterofMachinePerception,CzechTechnicalUniversity,Prague,1999.

40. Davis,J.W.andSharma,V.2007.Background-subtractioninthermalimageryusingcontoursaliency,International Journal of Computer Vision71(2),161–181.

41. Huang,D.S.,Wunsch,D.C.,Levine,D.S.,andJo,K-H.2008.Advancedintelligentcomputingtheo-riesandapplications:withaspectsoftheoreticalandmethodological issues.InProceedings of the 4th International Conference on Intelligent Computing (ICIC 2008),Shanghai,China.

42. Wu,C.J.andTsai,W.H.2009.Unwarpingofimagestakenbymisalignedomnicameraswithoutcam-eracalibrationbycurvedquadrilateralmorphingusingquadraticpatternclassifiers,Optical Engineering,48,087003(1)–087003(11).

43. George,W.andSiavash,Z.2000.Robustimageregistrationusinglog-polartransform,InProceedings of the IEEE International Conference on Image Processing,Vancouver,BritishColumbia,Canada.

44. Traver,V.J.andAlexandre,B.2010.Areviewoflog-polarimagingforvisualperceptioninrobotics,Robotics and Autonomous Systems,58,378–398.

45. José,S.V.andAlexandre,B.2003.Vision-basednavigation,environmentalrepresentationsandimag-inggeometries,InVisLab-TR 01/2003, 10th International Symposium on Robotics Research,R.JarvisandA.Zelinsky(Eds.),Springer,NewYork.

46. Owens, J.,Hunter,A., andFletcher,E. 2002.A fastmodel-freemorphology-basedobject trackingalgorithm,InBritish Machine Vision Conference,Cardiff,U.K.,Vol.2,pp.767–776.

AQ30

AQ31

AQ32

K13920_C016.indd 455 1/4/2013 7:24:34 PM

AUTHORQUERIES[AQ1] Pleasechecksentencestarting“Inthefaceof…”forclarity.[AQ2] Pleasecheckifedittosentencestarting“Theauthoritycan…”isokay.[AQ3] Pleasecheckifthefixedrunningheadisok.[AQ4] Pleasecheckentrystarting“makingtheinformation…”forclarity.[AQ5] Pleasecheckifinsertedclosingparenthesisisokay.[AQ6] Pleasecheckifedittosentencestarting“However,PIR-basedmotion…”isokay.[AQ7] Pleasecheckifedittosentencestarting“Hence,inthesecases…”isokay.[AQ8] Pleasecheckifedittosentencestarting“Glass-breakdetectors…”isokay.[AQ9] Pleasecheckifedittosentencestarting“thismaysomehow…”isokay.[AQ10] Figures16.1through16.4areofpoorquality.Pleaseprovidebetterqualityfigures.[AQ11] Pleasechecktheorderofsectionheadings.[AQ12] Pleasecheckentrystarting“keepsaneye…”forclarity.[AQ13] Pleasecheckifedittosentencestarting“Thismakesthe…”isokay.[AQ14] Pleasecheckifedittosentencestarting“Thisleadsto…”isokay.[AQ15] Pleasecheckifedittosentencestarting“thesecameraslook…”isokay.[AQ16] Pleasecheckifedittosentencestarting“However,thecurrent…”isokay.[AQ17] Pleasecheckifedittosentencestarting“Thisalgorithmwas…”isokay.[AQ18] Pleasecheckif“capturingimages”shouldbechangedto“capturedimages”.[AQ19] Pleasecheckifedittosentencestarting“Sincefish-eyelens…”isokay.[AQ20] Pleasecheckif“unwrap”shouldbechangedto“unwarp”.[AQ21] Pleasecheckentrystarting“processthemsmoothly…”forclarity.[AQ22] Pleasenote thatEquations16.20 to16.29have changed toEquations16.12 to16.21

forsequentialorder.Pleasecheck.[AQ23] PleasecheckthewherestatementprovidedforEquation16.12forcorrectness.[AQ24] Pleasecheckifthephrase“whereQisthethresholdvalueofthedifferencebetweensum

ofRGBvalue for aparticular current imagepixel toprevious imagepixel” shouldbemodifiedas“whereQisthethresholdvalueofthedifferenceofthesumofRGBvaluesbetweenaparticularcurrentimagepixelandpreviousimagepixel”.

[AQ25] Pleasechecksentencestarting“AsforHvalue…”forcompleteness.[AQ26] Pleasecheckifedittosentencestarting“Inthefuture…”isokay.[AQ27] Originallyreferences2and21and3and25wereoneandthesame;hence,therepeated

references have been deleted and renumbered both in the text and list accordingly.Pleasecheck.

[AQ28] PleaseprovidepagerangeforRefs.[4,41,33].[AQ29] Pleaseprovidetheauthor/owner,articletitle,andtheaccesseddatesforRefs.[18,19].[AQ30] PleaseprovidethelocationoftheconferenceforRefs.[31,34].[AQ31] PleaseprovidetheaccesseddatesforRefs.[35,36].[AQ32] Pleaseprovidein-textcitationsforRefs.[42].

K13920_C016.indd 456 1/4/2013 7:24:34 PM