If you can't read please download the document
Upload
aulii
View
67
Download
3
Embed Size (px)
DESCRIPTION
K Nearest Neighbors. Saed Sayad. KNN - Definition. KNN is a simple algorithm that stores all available cases and classifies new cases based on a similarity measure. KNN – different names. K-Nearest Neighbors Memory-Based Reasoning Example-Based Reasoning Instance-Based Learning - PowerPoint PPT Presentation
Citation preview
K Nearest NeighborsSaed Sayad*www.ismartsoft.com
www.ismartsoft.com
KNN - DefinitionKNN is a simple algorithm that stores all available cases and classifies new cases based on a similarity measure*www.ismartsoft.com
www.ismartsoft.com
KNN different namesK-Nearest NeighborsMemory-Based ReasoningExample-Based ReasoningInstance-Based LearningCase-Based ReasoningLazy Learning*www.ismartsoft.com
www.ismartsoft.com
KNN Short HistoryNearest Neighbors have been used in statistical estimation and pattern recognition already in the beginning of 1970s (non-parametric techniques).Dynamic Memory: A theory of Reminding and Learning in Computer and People (Schank, 1982).People reason by remembering and learn by doing. Thinking is reminding, making analogies.Examples = Concepts???*www.ismartsoft.com
www.ismartsoft.com
KNN ClassificationAgeLoan$*www.ismartsoft.com
www.ismartsoft.com
KNN Classification Distance*www.ismartsoft.com
AgeLoanDefaultDistance25$40,000N10200035$60,000N8200045$80,000N6200020$20,000N12200035$120,000N2200052$18,000N12400023$95,000Y4700040$62,000Y8000060$100,000Y4200048$220,000Y7800033$150,000Y8000
48$142,000?
www.ismartsoft.com
KNN Classification Standardized Distance*www.ismartsoft.com
AgeLoanDefaultDistance0.1250.11N0.76520.3750.21N0.52000.6250.31N0.316000.01N0.92450.3750.50N0.34280.80.00N0.62200.0750.38Y0.66690.50.22Y0.443710.41Y0.36500.71.00Y0.38610.3250.65Y0.3771
0.70.61?
www.ismartsoft.com
KNN Regression - Distance*www.ismartsoft.com
AgeLoanHouse Price IndexDistance25$40,00013510200035$60,0002568200045$80,0002316200020$20,00026712200035$120,0001392200052$18,00015012400023$95,0001274700040$62,0002168000060$100,0001394200048$220,0002507800033$150,0002648000
48$142,000?
www.ismartsoft.com
KNN Regression Standardized Distance*www.ismartsoft.com
AgeLoanHouse Price IndexDistance0.1250.111350.76520.3750.212560.52000.6250.312310.316000.012670.92450.3750.501390.34280.80.001500.62200.0750.381270.66690.50.222160.443710.411390.36500.71.002500.38610.3250.652640.3771
0.70.61?
www.ismartsoft.com
KNN Number of NeighborsIf K=1, select the nearest neighborIf K>1,For classification select the most frequent neighbor.For regression calculate the average of K neighbors.*www.ismartsoft.com
www.ismartsoft.com
Distance Categorical Variables*www.ismartsoft.com
XYDistanceMaleMale0MaleFemale1
www.ismartsoft.com
Instance Based Reasoning IB1 is based on the standard KNN IB2 is incremental KNN learner that only incorporates misclassified instances into the classifier. IB3 discards instances that do not perform well by keeping success records.*www.ismartsoft.com
www.ismartsoft.com
Case Based Reasoning*www.ismartsoft.com
www.ismartsoft.com
New Case
Case Database(Prior Cases)
Similar Cases
Solved Case
Test/Repaired Case
Suggested Solution
Confirmed Solution
Retrieve
Retrieve
Reuse
Revise
Learned Case
Problem
Retain
Retain
KNN - ApplicationsClassification and Interpretationlegal, medical, news, banking
Problem-solvingplanning, pronunciation
Function learningdynamic control
Teaching and aidinghelp desk, user training
*www.ismartsoft.com
www.ismartsoft.com
SummaryKNN is conceptually simple, yet able to solve complex problems Can work with relatively little informationLearning is simple (no learning at all!)Memory and CPU costFeature selection problemSensitive to representation*www.ismartsoft.com
www.ismartsoft.com
*www.ismartsoft.comQuestions?
www.ismartsoft.com