Smart image and video search presentation

  • View

  • Download

Embed Size (px)


  • 1.Smart Image And Video Search Enterprise Image and Video Analytics

2. Help! The problem Government has piles of video, analysts miss things after looking at it for 20 minutes ! What to do?We are collecting video and images EVERYWHERE !!!!!!!! 3. How to search image/video for what you are looking for identify peoplesuspected IED sites known terroristschanges in geospatial datayellow ships IED circuit boards trains geographic featuresplanesmini vansactivity 4. The Enterprise Solution Only piXserve by piXlogic offers a generalized enterpriseclass image/video search capability Enterprise class 64bit Java application for Windows Server2003+ Indexes all objects in images and frames of video and alsoattempts to classify those objects Supports 90+ image or video formats Generates a light weight XML index and leaves the originalinformation where it lives and can place index in RDBMS Special focus on text in image and facial recognition Offers a JMS based event and alerting system Supports a web services REST API 5. piXlogic A New Approach Seeing Understanding SkyDiscern shapesTree Face ForegroundVectorize shapesBackgroundConnected BackgroundSynthesize descriptions Location Comparing Direct Comparisons Size Color Relationships Geometry Relationships Context Etc. 6. Think About Text IndexingCommon Text Indexing Engines: Its All AboutBythreemethodsweAt their most Basic they Index Text .mayBy three methods we may learn wisdom: First, by learnreflection, which is noblest; Second, by imitation, wisdomwhich is easiest; and third by experience, which is the .bitterest. Confucius three methodsEnter Query: learn wisdom ConfuciusSearch by imitation by experience 7. Text Result SetRank Date Description .98%01/01/2011 Confucius dreams ..87%04/12/2010 Eastern philosophy and culture84%07/23/2009 Harvard study on ancient China73%02/12/2009 Fortune cookies run amok65%03/01/2008 Medical studies calm fears in 58%01/01/2007 Confucius dreams ..47%04/12/2007 Eastern philosophy and culture44%07/23/2004 Harvard study on ancient China43%02/12/2000 Fortune cookies run amok35%03/01/1999 Medical studies calm fears in 8. Image/Full Motion Video Indexing BackgroundForegroundBackground WE LOOK FOR SHAPES AT THE PIXEL LEVEL 9. How Pixel Indexing WorksEnterprise PixelKey Objects Synthetic MetadataIndexing Engine: people" id="61" 98% 17. 2D Objects Library Topic specific filtering Specific library can be built to filter for specific information as images/frames of video arrive they are compared to this information and specific tagging / matching occurs E.g. light arms, ships, clothing, hair styles etc. 18. Text In ImageEnter Query:Query By Text in Image Enter TextHEALTH Search 19. RESTful API All Server features & functions All User interface functions Data access Machine to machine integration API Guide 20. Alerting Engine Create standing queries When an event in an image triggers an alerttwo things happen: A JMS message is sent to whatever program orsystem designated An email message is sent to whomever has beendesignated to receive the alert 21. 787-2989www.pixlogic.comEND 22. SUPPORTING MATERIALS 23. The Industry Video monitoring software packagesintegrated with DVR support such as ObjectVideo, Image IQ, 3VR, Virage and others canrecognize certain objects, perimeters, andactivities .. They are security focused .. Often with someability to search faces and possibly license tags 24. The Industry (Contd) Other packages (COTS & GOTS) are starting to address generalized imageand video search Numenta offers an image search/pattern matching toolkit (notenterprise .. and each solution must be hand crafted) Pixsy offers image search to match images for internet advertising andshopping online IDEE Tin-Eye offers image on the internet more advertising basedapproaches KEVA (Sierra Nevada Corp/PNNL) is a GOTS parallel processing engine forsearching video government only and fairly early stage Google and Facebook Both have image and video tagging algorithms intheir general search and/or analytics suites, with some specific objectsearch capabilities but web focused only There are others .. But NONE are enterprise class generalized Google likesearch engines NERVVE 25. Notions/Classesartificial" id="1" smooth" id="19"rectangle" id="37" purple" id="17"white" id="2" circle" id="20"pentagon" id="38" orange" id="18"flower" id="3"polygonal" id="21" clutter" id="39" child" id="33"sky" id="4" sparse" id="22"star" id="40" drawing" id="34"gray" id="5"face" id="23"zebra" id="41"flat" id="35"vegetation" id="6"horizontal" id="24"silver" id="42" triangle" id="36"lawn" id="7"document" id="25"pumpkin" id="43" flash" id="51"skin" id="8"map" id="26" bride" id="44"indoor" id="52"road" id="9"diagram" id="27" sea" id="45"outdoor" id="53"vertical" id="10" text" id="28"snow" id="46" near" id="54"landscape" id="11"bikini" id="29"sand" id="47" dark" id="55"red" id="12"exposed body" id="30"beach" id="48"helicopter" id="56"green" id="13"line" id="31"fireworks" id="49"building" id="57"blue" id="14" adult" id="32" sunset" id="50" car" id="58"aqua" id="15" mountain" id="60"soccer ball" id="59" crowd" id="63"yellow" id="16" people" id="61"airplane" id="66" 3D drawing" id="64"gun" id="65"fire" id="62" 26. Segment the Image or Video FrameIndexing breaks the image up into smaller and smaller pieceswhich are more easily mathematically described 27. Medical UsesDuctal Carcinoma Ductal CarcinomaPerspective 1Perspective 2 28. Medical UsesFrontal Lobe InjuryFrontal Lobe Injury Perspective 2Perspective 1 29. Physical CONOP continuous streams orIndex held in memory analyst filesdirect indexing from source Web Serveranalystvideo/imageindexingWeb Serverdb sourceservers Web Serveranalystindirect indexingsearchingfrom DAM systemAlertServerStore incontentDAM system Storeanalyst networkanalystanalysisdownstreamanalytical systems 30. Logical CONOPS 1 Watching The Border continuousvideo/image sourcereportingapplication analystsmetadata name automatic faces watch listalertscontinuous vehicles vehicle makeindexing vehicle model license plates tag state tag number FilterFilterindexesdownstream index Filterindexescorrelation analytical indexes systems 31. Conceptual CONOPS 1Watching The Border cameras tag facevehicle border crossing vehicle Indexes -watch list -vehicle types -state license platesFace - biometrics Tag - state & text extracted Vehicle - make/model extracted-Daniel San Diego-State: Florida -Make: Honda-U.S. Most Wanted-Tag: SAMPLE-Model: Civic 32. TNT - CONOPSNET EMULATED UAVpixserve Local Network flumenetwork net emulator FIELD ANALYSTScomms 1 2 345piXlogic piXlogic apiflume transmits piXlogic indexmatches andtransfers clip matches overactivity use video files video andclips videoto flume MITRE networksearches on targetdirectory emulator eventQuery: Buildings Ships Aircraft RADAR INFRAREDVIDEO VIDEO IMAGERYAlert Results