# Nearest Neighbor Paul Hsiung March 16, 2004. Quick Review of NN Set of points P Query point q Distance metric d Find p in P such that d(p,q) < d(p’,q)

• View
214

2

Embed Size (px)

### Text of Nearest Neighbor Paul Hsiung March 16, 2004. Quick Review of NN Set of points P Query point q...

• Nearest NeighborPaul HsiungMarch 16, 2004

• Quick Review of NNSet of points PQuery point qDistance metric dFind p in P such that d(p,q) < d(p,q) for all p in Pqp

• NN Used InImage databases [Pentland et al]Color indexing [swain et al]Recognizing 3D objects [Murase et al]Shapes [Mori et al]Drug testingDNA sequence matching [Buhler]

• Tree-based ApproachesQuadtreesSplit middle in all dimensionsSplit until no points or one point leftKd-treesSplit in one dimensionPick the middle wiselyBall-treesPick two pivots and splitSR-treesWe have rectangles and spheres, so why not combine them

• Indyks GripeBeyond 10 or 20 dimensions, tree-based structures will look at many pointsNo better than brute force linear searchSo he came up with a hash table approach: Locality Sensitive Hashing (LSH)Rest of talk will be on his paper

• LSH

• Interlude: Near NeighborSet of points PQuery point qDistance metric dFind p in P such that d(p,q) < (1+)d(P,q) where d(P,q) is the distance of q to its closest point in Pqp(1+)d(P,q)d(P,q)

• HashPick a subset I of random coordinatesHash function, h(p), will return a bucket IDh(p) = projection of p on I

• IntuitionIf two points are close, they hash to same bucket with some probability p1If they are far, they hash to same bucket with a smaller probability p2 < p1

• Indyks HashConvert coordinates of p to {0,1}dUse Hamming distance: d(p,q)= # positions on which p and q differExample:p=(0,1,0,1,1,1,0,0,1,0)I={2,5,7}Then, h(p)=(1,1,0)Demo: http://web.mit.edu/ardonite/6.838/locality-hashing.htm

• Why Locality-sensitive?Pr[h(p)=h(q)]=(1-d(p,q)/D)kD is the number of dimensions in the binary representationk is the size of IWe can vary the probability by changing k

k=1k=2distancedistancePrPr

• Now to Use It (Training)Generate l hash functions: h1..hlStore each point p in the bucket hi(p) of the i-th hash array, i=1...l

• Now to Use It (Query)Retrieve all the points that belong to the buckets: h1(q)..hl(q)Return the retrieved point that is closest to qThis solves the Near Neighbor problem

• Indyks ResultsCompared with another tree-based algorithmColor histogram dataset from Corel Draw20,000 images, 64 dimensionsUsed 1k, 2k, 5k, 10k, 19k points for training1k points are used for queryComputed missed ratio fraction of queries with no hits

• Indyks Results

• Results II

• Ugly SideWorks best with Hamming distanceCan be extended from L1 and L2 normsRequires parameter tweaking (size of I and number of hash buckets)Does not work well on uniform data

• BibliographyA. Gionis, P. Indyk, R. Motwani. Similarity Search in High Dimensions via Hashing. In VLDB 25th, 1999J. Buhler. Efficient Large-Scale Sequence Comparison by Locality-Sensitive Hashing. In Bioinformatics 17(5) 419-428, 2001H. Murase, S. K. Nayar. Visual Learning and Recognition of 3D Objects from Appearance. In IJCV, Vol. 14, No. 1 5-24, 1995A. Pentland, R.W. Picard, S. Scalroff. Photobook: Tools for Content Based Manipulation of Image Databases. In SPIE Vol. 2185 34-47, 1994M.J. Swain, D.H. Ballard. Color Indexing. In IJCV, Vol. 7, No. 1 11-32, 1991G. Mori, S. Belongie, J. Malik. Shape Contexts Enable Efficient Retrieval of Similar Shapes. CVPR 1 723-730, 2001Slides: Algorithms for Nearest Neighbor Search by Piotr IndykSlides: Approximate Nearest Neighbor in High Dimensions via Hashing by Aris Gionis, Piotr Indyk, and Rajeev Motwani ##### j - pefcl.snut.ac. Q * P Calculator P* iLg q*-PI controller + PI controller iLg + d* iLg q iLg d + + + V Cf,nom PI controller PI controller + q V Cf,nom-V Cf q* G S + V Cf d
Documents ##### Teaching Nutrition in the Context of Lifestyle · PDF fileDQ G 6 H OI & D U H & R Q W LQ X R X V / H D U Q LQ J D Q G 4 X D OLW \,P S U RY H P H Q W & R P P X Q LW \ ( Q J D J H P
Documents ##### Q R UWK H D V W & R OR P E LD · PDF file 8 vd j h r i % lr 2 q h & r p s oh wh fr q wh q w lv vwulfwo\ olp lwh g wr s h uvr q d o h g x fd wlr q d o d q g q r q fr p p h ufld o x
Documents ##### 7 K H 0 D P P D OLD Q 3 D UV ,Q WH UP H G LD ² 6 WUX F WX ... › journals › zoological-science › ... · PDF file 7 K H 0 D P P D OLD Q 3 D UV ,Q WH UP H G LD ² 6 WUX F WX UH
Documents ##### ,Q V H F W ) D X Q D \$ V V R F LD WH G Z LWK \$ Q D F D UG LX P · PDF file ,q v h f w ) d x q d \$ v v r f ld wh g z lwk \$ q d f d ug lx p r f f lg h q wd oh 6 d s lq g d oh v \$ q d
Documents ##### LQ \$ OS LQ H & R P P X Q LWLH V - BioOne · PDF file lq \$ os lq h & r p p x q lwlh v \$ x wk r u 0 d j q d q l 1 d wd old 6 r x ufh 0 r x q wd lq 5 h vh d ufk d q g ' h yh or s p h
Documents ##### Lecture 3: ARIMA(p,d,q) models frapetti/CorsoP/chapitre_3_IMEA_1.pdf · PDF file To some extent, ARIMA(p,d,q) models are a generalization of ARMA(p,q) models : the d-di erenced process
Documents ##### B? · PDF file q p q p q p q p q p q p q p q p q p qp q p q p q p q p q p q p q p q p q p q p q p q p qp qp qp qp qp qp qp qp qp qp qp qp qp qp qp qp qp qp qp qp qp qp q q q q qp q
Documents ##### R I & D Q D G LD Q : H H G V 3 D Q LF X P P LOLD F H X P · PDF file 7 k h % lr or j \ r i & d q d g ld q : h h g v 3 d q lf x p p lold f h x p / \$ x wk r uv & d yh uv 3 d x o % d
Documents ##### D P 6W 7 KRPD V \$T XLQDV & DWKROLF& KXUFK · PDF file 0 \$ 6 6 ( 6 6 dw x ug d\ 9 lj lo s p 6 x q g d\ d p d p 0 r q 7 x h 7 k x ) ul d p : hg q hvg d\ q r r q ) luvw 6 dw xug d\ d
Documents ##### BALLADE POUR ADELINE - POUR A · PDF file BALLADE POUR ADELINE 1 2 p 4 4 2 p 4 4 1 p p 5 p p 9 p p 13 e e e e e e e e d d i i q q q i i q q q i i i i q q RICHARD CLAYDERMAN BALLADE
Documents ##### COMMUNICATING RECURSIVE PROGRAMS: CONTROL AND acts/2015/Slides/ C. Aiswarya, and P. Gastin 13 M p q aab c d b acd c M′ p q aab c d b acd c M 1 p q aa b c d c M′ 1 p q aa b c d
Documents ##### D W D +D Q G OLQ J ,P S R U W &OH D Q LQ J D Q G 9LV X D ... · PDF file 'D W D +D Q G OLQ J ,P S R U W &OH D Q LQ J D Q G 9LV X D OLV D W LR Q / H FW X UH 3 UR JUD P P LQ J Z LW
Documents ##### ENTRAR // ENTER NEWSLETTER · PDF file NEWS. LETTER. JANEIRO // JANUARY. q q q p q p p p q p q q p. ICA . NEWS. LETTER. JANEIRO // JANUARY. q q q p p p p p p p q q p. ICA
Documents ##### LCOG Approval 2017.2020 - Oregon ... X N T X ] ` Q V T d X c N Q N _ Q N _ ` P X Y Y d _ N S Q P d ` Q T N P M L ` X N V T Q N [ S ]f Q à º È È ³ Å M P Q a N c È ¹ Å [ ` ]
Documents #####  where the supply and demand curves meet  equilibrium price: P where Q D = Q S  equilibrium quantity: Q where Q D = Q S
Documents ##### Demand, Revenue, Cost, & Profit. Demand Function – D(q) p =D(q) In this function the input is q and output p q-independent variable/p-dependent variable
Documents ##### Y L S LV Y -LV P S L7 8 - q d luq lp v q d x lx a p l d q lq x p d z q" y -lv p s l7 8 k g q x ( q d x lld q llx % x d aq x q p q d liy x a q a k iol q d a 7 x y l e d q lq a q s lx
Documents ##### PERCEPCIÓN VISUAL - edu.xunta. · PDF file PERCEPCIÓN VISUAL Busca entre las letras y rodea de rosa las b, de azul las d, de verde la p y de naranja las q: b d p q d b p q b d p
Documents ##### äge 10.17 8 · PDF file 8 s g d wh r q 7 d j d or p d q wlv p d q looh q v lv 6 d x v v x uh z lwk g h v f uls wlr q r i wk h ih p d oh d q g f r p p h q wv r q lwv v \ v wh p d wlf
Documents ##### 6 W -RV H S K 3 D ULV K · PDF file 6 x q g d \ d p d p & r q ih vvl r q v ) ulg d \ s p 6 d wx ug d \ ³ s p + r o\ + r xu r i( xf k d ulv wlf \$ g r ud wlr q ) ulg d \ s p 3d ulvk
Documents ##### a i r s F i P t ... 1 P P P P P P C d Y q z P d Y c d q ) z R h m b d z s g d z a d f h m m h m f z n e z n t q z b g t q b g z h m z 1 . . 0 ) z v d z g Y u d z r d s z n t q z g
Documents ##### BALLADE POUR ADELINE · PDF file 2018-10-03 · BALLADE POUR ADELINE 1 2 p 4 4 2 p 4 4 1 p p 5 p p 9 p p 13 e e e e e e e e d d i i q q q i i q q q i i i i q q RICHARD CLAYDERMAN BALLADE
Documents ##### BALLADE POUR ADELINE - POUR ADELINE.pdf · PDF file3 p p 26 p p 28 p p 30 p p 31 d d d d d i i q q q ii q j jj q q q i i q BALLADE POUR ADELINE my-piano.info
Documents ##### D V - BioOne · PDF file 5 h j lr q ,q g ld q + lp d od \ d v \$ x wk r uv ,nh g d 1 d k r 1 d ud p d & k l\x nl d q g * \d ovr q 6 r q d p 6 r x ufh 0 r x q wd lq 5 h vh d ufk d q
Documents ##### 3 Q Q Á Á Á X P ] } v µ } X } u X ... YDORU FRQVWDQWH UD]mR D D U QQ u 7HUPR ³Q´ WHUPR UD]mR [ SRVLomR ³Q´ ± Q D D TQ u 7HUPR ³Q´ WHUPR [ UD]mR HOHYDGD D ³Q ´ Q Q Q D
Documents Documents