Upload
others
View
5
Download
0
Embed Size (px)
Citation preview
Near-Neighbor Methods in Random
Preference Completion
Ao Liu[RPI], Qiong Wu[WM], Zhenming Liu[WM] and Lirong Xia[RPI]
02/27/2019
Introduction to Preference Completion
1
Learn to Ratings Learn to Ranks (Preference Completion)
(Commonly used) (More robust)
x1 R1 = {y1, y3}≻ y2 ≻ y4 ≻ y5
x2 R2 = y1 ≻ y3 and y4 ≻ y5
x3 R3 = y5 ≻ y4 ≻ others
x4 R4 = y1 ≻ y3 ≻ y4 ≻ y5
i.e., commented, “I prefer y5 to
y4, all others are worse.”
Near-Neighbor Methods in Random Preference Completion
Settings for (user-wise) preference completion problem in recommender systems:
• y1,···, ym : m alternatives (items).
• x1,···, xn : n agents (users) with given partial preference over y1,···, ym.
2
x1 R1 = {y1, y3}≻ y2 ≻ y4 ≻ y5
x2 R2 = y1 ≻ y3 and y4 ≻ y5
x3 R3 = y5 ≻ y4 ≻ y1 ≻ y3
x4 R4 = y1 ≻ y3 ≻ y4 ≻ y5
Inputs: n agents’ partial preferences
Output: i th agent’ full ranking
xi Ri = y1 ≻ y2 ≻ y3 ≻ y4 ≻ y5
Near-Neighbor Method[Liu-2007]:
Similar Agents
y2 ≻ y4 ≻ y5
A Widely-Used Algorithm [Liu-2009, Katz-Samuels and Scott-2018]
3
NK(R1, R4) =
NK(R2, R4) =
NK(R3, R4) =
x1 R1 = {y1, y3}≻ y2 ≻ y4 ≻ y5
x2 R2 = y3 ≻ y1 and y4 ≻ y5
x3 R3 = y5 ≻ y4 ≻ y1 ≻ y3
x4 R4 = y1 ≻ y3 ≻ y4 ≻ y5
Observable:
n agents’ partial preferences.
Near Neighbor
Formal Definition of KT-kNN Algorithm
Normalized Kendall-tau (NK) distance
between rankings Ri and Rj :
NK(Ri, Rj) = # Pairs ranked opposite in Ri and Rj
# Pairs ranked both by Ri and Rj
Ranked both: y3 ≻ y1 and y4 ≻ y5
Ranked opposite: y3 ≻ y1
1
2
5
6
0
5
NK( , ) = 1
2
Example:
? ?
?
Rankings from adding
deterministic noise to utilities:
– Yes [Katz-Samuels and Scott-2018]
Rankings from adding random
noise to utilites:
– Open Question (Algorithms work under
deterministic settings usually also
work under random settings)
Is NK an
effective metric?
Formal Definition of KT-kNN Algorithm
Normalized Kendall-tau (NK) distance
between rankings Ri and Rj :
NK(Ri, Rj) = # Pairs ranked opposite in Ri and Rj
# Pairs ranked both by Ri and Rj
4
• All agents and alternatives are assigned with a
latent vector in Rd space (latent space).
A Very Simple and Classic model:
Latent Space
Kyle Stan
Kenny
Eric
1
2
3
4
5
Late
nt
Fea
ture
1
Latent Feature 2
Alternatives
Agents
Alternatives closer to
agents have higher utility
A Widely-Used Algorithm [Liu-2009, Katz-Samuels and Scott-2018]
– Need a model for ground-truth
KT-kNN’s Inefficacy under Random Noise Setting
Main Contribution 1: KT-kNN is incorrect on datasets with Plackett-Luce noise.
Theorem 1: (informal) For 1D latent space, with at least 50% probability,
• ||x*KT-kNN
- xi|| = Θ(1) .
• Alternative closer to agents has higher probability to be more preferred [Plackett-1975, Luce-1977].
• Agents close to each other have similar distribution on rankings.
Properties:
xi
1 -1 -0.5
Predicted by
KT-kNN Predicted NN
should be
5
For example, if DY = Uniform([-1,1]), For xi∈[-1, -0.5],
with high probability we have x*KT-kNN close to -1.
Noise Setup: Another Very Simple and Classic Model
Plackett-Luce Model: Verified using real-world election data [Gormley and Murphy-2007]
Anchor-kNN, a Correct Algorithm
• Our algorithm uses information from other agents’ rankings.
• Features: F(xi) ≡ (Fi(1), … , Fi
(n)) = ( NK(R1, Ri) , · · · , NK(Rn, Ri) ). • Distance Function: D( xi , xj ) ≈ || F(xi) - F(xj) ||1.
Main Contribution2: New kNN algorithm able to find correct neighbors
6
NK Distance Matrix
NK(R2, R3)
F(xi) (Row i)
F(xj) (Row j)
xi xj
xk
Anchor-kNN, a Correct Algorithm
Theorem 2: (informal) For 1-dimensional latent space, if all agents
rank at least poly-log(m) alternatives, with probability 1-o(n-0.2),
|| x*Anchor-kNN – xi || < o(1) .
7
Main Contribution2: New kNN algorithm able to find correct neighbors
Theorem 1: (informal) For 1D latent space, with at least 50%
probability, ||x*
KT-kNN - xi|| = Θ(1) .
Numerical Validations
8
Numerical Validations to Anchor-kNN and KT-kNN:
• KT-kNN: incorrect algorithm
• Anchor-kNN: our (correct) algorithm
• Ground Truth-kNN: Information theoretical optimal
k = 751, the optimal k for Ground Truth-kNN
Real-World Experiments
9
• KT-kNN: incorrect algorithm
• Anchor-kNN: our (correct) algorithm
• Collaborative Filter (CF): base-line algorithm, using cosine similarity.
KT Coefficient Spearman Rho Precision @ 5
CF KT-kNN Anchor-
kNN CF KT-kNN
Anchor-
kNN CF KT-kNN
Anchor-
kNN
k = 5 0.0531 0.0548 0.0989 0.0787 0.0811 0.1462 0.3286 0.3386 0.4045
k = 15 0.1199 0.1259 0.1646 0.1770 0.1860 0.2423 0.4291 0.4573 0.4850
k = 25 0.1403 0.157 0.1869 0.2214 0.2307 0.2742 0.4718 0.4823 0.5077
Dataset: standard Netflix dataset
Conclusion: Anchor-kNN >> KT-kNN ≈ Collaborative Filter
Generalizations
10
Generalization: High dimensional latent spaces
• We only proved 1-D cases using symmetry property.
• We believe Anchor-kNN Algorithm is also correct for higher dimensional latent
spaces according to our simulation on higher dimensional latent spaces.
Conclusions
• We proved a widely used KT-kNN Algorithm is
incorrect on noisy datasets.
• We proposed Anchor-kNN Algorithm that works on
noisy datasets.
• We generalized all conclusions above to high-
dimensional latent spaces on both synthetic and real-
world data.
11
References:
1. [Liu-2007] Tie-Yan Liu. Learning to rank for information retrieval. Found. Trends Inf. Retr., 3(3):225–331,
March 2009. ISSN 1554-0669.
2. [Katz-Samuels and Scott-2018] Katz-Samuels J, Scott C. Nonparametric Preference Completion.
International Conference on Artificial Intelligence and Statistics 2018 Mar 31 (pp. 632-641).
3. [Gormley and Murphy-2007] Gormley IC, Murphy TB. A latent space model for rank data. Statistical
Network Analysis: Models, Issues, and New Directions 2007 (pp. 90-102). Springer, Berlin, Heidelberg.
4. [Plackett-1975] Robin L. Plackett. The analysis of permutations. Journal of the Royal Statistical Society.
Series C (Applied Statistics), 24(2):193–202, 1975.
5. [Luce-1977] R. Duncan Luce. The choice axiom after twenty years. Journal of Mathematical Psychology,
15(3):215–233, 1977.
12
Thanks for your time !