MACHINE VISION GROUP Face image mapping from NIR to VIS Jie Chen Machine Vision Group MACHINE VISION GROUP Outline Problem Methods Preliminary results Plans for next period MACHINE VISION GROUP Face image mapping from NIR to VIS Problem NIR: Near infrared imaging VIS: Visual light imaging MACHINE VISION GROUP Face image mapping from NIR to VIS Problem NIR: Near infrared imaging VIS: Visual light imaging MACHINE VISION GROUP Algorithm: Patches mapping Training Training MACHINE VISION GROUP Algorithm: Patches mapping Training Mapping i,ji,j MACHINE VISION GROUP Look up the KNN A patch of an input sample in S 3 k-th nearest patch in S 1 Weight of k-th nearest neighbor A patch of an input sample in S 4 Corresponding patch of in S 2 MACHINE VISION GROUP Weight computing MACHINE VISION GROUP Experiments Setup both S1 and S2 is composed of 300 samples. 50 subjects, each subject has 6 images but in different expression (anger, disgust, fear, happiness, sadness, and surprise). w f =64, h f =80, w p =16, h p =16, w o =12, h o =12 Testing:using leave-one-out and K=15. MACHINE VISION GROUP Reconstructed images (a) Input images in NIR (b) Reconstructed images in VIS using LBP(8,1) and the PSNR (c) Reconstructed images in VIS using the combined Multi-resolution LBP and their PSNR (d) Ground truth in VIS MACHINE VISION GROUP Multi-resolution LBP (MLBP) (P=4,R=1) (P=8,R=1) (P=12,R=1.5) (P=16,R=2) (P=24,R=3) MACHINE VISION GROUP PNSR Pixel wise LBP MACHINE VISION GROUP Multi-resolution LBP MACHINE VISION GROUP PSNR on MLBP MACHINE VISION GROUP Plans for next period Training data: Use more samples (192*10 from CASIA, a group in Beijing, China) Methods: Combine the methods proposed in the paper (A. Hertzmann, SIGGRAPH, 2001) for better performance