Tianpei research summary

Outline Introductions My research and contributions Additional information References

Summary of research activities

Tianpei Xie,Advisor: Alfred O. Hero

1 / 12


1 Introductions

Backgrounds

Robust learning from multiple sources

2 My research and contributions

3 Additional information

2 / 12


Backgrounds

• With the advent of Big Data era, we experienced a great explosion in terms of

1 the amount of data that is public available;2 the diversity of multiple data sources that are accessible simultaneously;3 the power of computational resource.

Figure : One of the central neighborhood in TwitterNetwork. https://dhs.standford.edu/gephi-workshop/twitter-network-gallery/

Figure : Multi-modality data source

3 / 12

https://dhs.standford.edu/gephi-workshop/twitter-network-gallery/

https://dhs.standford.edu/gephi-workshop/twitter-network-gallery/


Beyond the conventional machine learning

New challenges in machine learning area:

• Robustness of model in terms of low quality training data;• Learning from multiple information sources;• Ability in handling data inconsistency and high dimensionality.

• Interest: single-source learning with clean training set ⇒ robustlearning from multiple sources using information theory.

4 / 12


Previous work

Robust learning

• Robust learning via surrogate loss e.g. [Bartlett and Mendelson,2003], [Bousquet and Elisseeff, 2002], [Tyler, 2008], ROD [Xu et al.,2006].

• Anomaly detection e.g. SVDD [Scholkopf et al., 1999], GEM [Hero,2006].

• Cons: sensitive to outlier in training sample, and solves a non-convexoptimization.

Learning from multiple source (Multi-view learning)

• Co-regularization on Euclidean feature space e.g. CCA [Hardoonet al., 2004], SVM-2K [Farquhar et al., 2005], Neural Nets [Ngiam et al.,2011]

• Cons: lack of ability to handle data with high uncertainty, highdimensionality and between-view inconsistency.

5 / 12


Contributions

Robust learning:

• Proposed the GEM-MED algorithm [Xie et al., 2014] as a joint classification +anomaly detection on noisy training set.

• Rely on the GEM estimator [Hero, 2006], a non-parametric entropy estimator.

(a)

0 0.2 0.4 0.6 0.8 10.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

recall

pre

cis

ion

ROD−0.02

ROD−0.1

ROD−0.2

ROD−0.3

ROD−0.6

GEM−MED

(b)Figure : (a) The low-entropy region estimated by GEM [Hero, 2006] to separate outlier (red triangle) from thenominal (circle and square) (b) The Precision-Recall curve for anomaly detection under given corruption ratefor GEM-MED and ROD.

• It only needs to solve a convex problem.

6 / 12


Our Contributions

Multi-view learning on statistical manifold:

• Assume data is given by parametric probability density function (p.d.f.) (datawith uncertainty.) and lies in a statistical manifold (space of all parametric p.d.f.).

• Proposed the CMV-MED algorithm [Xie et al., 2015] as the Co-regularization onStatistical manifold, i.e. learning multiple models from the p.d.f. data.

• A robust consensus measure to quantify the between-view inconsistency usingthe information divergence between p.d.fs .

2

classifier 1

1

-2-2classifier 2

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

2

dis

tance

(a)(b)

Figure : (a) The proposed stochastic consensus constraint on statistical manifold as a robust inconsistencymeasure. (b) The interpretation of GEM-MED as averaging multiple statistical models on the manifold.

7 / 12


Current research: Node prediction in network

• Learning to predict node attributes by combining the network graphtopology and node distribution

... ... ...

? ?

personal info. friendship

node attribute

(meta data)edge structure

ID

⇔

8 / 12


List of publications

List of relevant publications:

1 Xie, Tianpei, Nasser M. Nasrabadi, and Alfred O. Hero. ”Semi-supervised Multi-view learningon statistical manifold via stochastic consensus constraints.” in preparation.

2 Xie, Tianpei, Nasser M. Nasrabadi, and Alfred O. Hero. ”Learning to classify with possiblesensor failures.” submitted to IEEE Transaction on Signal Processing, 2016

3 Xie, Tianpei, Nasser M. Nasrabadi, and Alfred O. Hero. ”Semi-supervised multi-sensorclassification via consensus-based Multi-View Maximum Entropy Discrimination.” InAcoustics, Speech and Signal Processing (ICASSP), 2015 IEEE International Conference on,pp. 1936-1940. IEEE, 2015.

4 Xie, Tianpei, Nasser M. Nasrabadi, and Alfred O. Hero. ”Learning to classify with possiblesensor failures.” In Acoustics, Speech and Signal Processing (ICASSP), 2014 IEEEInternational Conference on, pp. 2395-2399. IEEE, 2014.

9 / 12


Websites and detailed information

• Contact:Tianpei (Luke) XieDepartment of Electrical and Computing Engineering, University ofMichigan, Ann Arbor,Office: 4313, EECS BuildingTEL : 734-546-8048Email: [email protected]

• LinkedIn: personal webpagehttps://www.linkedin.com/in/tianpeiluke

• Research: details for my research activitieshttp://tbayes.eecs.umich.edu/tianpei/research_main

• Github: my codes availablehttps://github.com/TianpeiLuke

10 / 12

https://www.linkedin.com/in/tianpeiluke

http://tbayes.eecs.umich.edu/tianpei/research_main

https://github.com/TianpeiLuke


Peter L Bartlett and Shahar Mendelson. Rademacher and Gaussiancomplexities: Risk bounds and structural results. The Journal of MachineLearning Research, 3:463–482, 2003.

Olivier Bousquet and Andre Elisseeff. Stability and generalization. TheJournal of Machine Learning Research, 2:499–526, 2002.

Jason Farquhar, David Hardoon, Hongying Meng, John S Shawe-taylor, andSandor Szedmak. Two view learning: SVM-2K, theory and practice.Advances in neural information processing systems, pages 355–362, 2005.

David Hardoon, Sandor Szedmak, and John Shawe-Taylor. Canonicalcorrelation analysis: An overview with application to learning methods.Neural computation, 16(12):2639–2664, 2004.

Alfred O Hero. Geometric entropy minimization (GEM) for anomaly detectionand localization. Advances in Neural Information Processing Systems,pages 585–592, 2006.

Jiquan Ngiam, Aditya Khosla, Mingyu Kim, Juhan Nam, Honglak Lee, andAndrew Y Ng. Multimodal deep learning. Proceedings of the 28thInternational Conference on Machine Learning (ICML-11), pages 689–696,2011.

Bernhard Scholkopf, Robert C Williamson, Alex J Smola, John Shawe-Taylor,and John C Platt. Support vector method for novelty detection. AdvancesIn Neural Information Processing Systems, 12:582–588, 1999.

David E Tyler. Robust statistics: Theory and methods. Journal of theAmerican Statistical Association, 103(482):888–889, 2008.

Tianpei Xie, Nasser M Nasrabadi, and Alfred O Hero. Learning to classifywith possible sensor failures. In Acoustics, Speech and Signal Processing(ICASSP), 2014 IEEE International Conference on, pages 2395–2399.IEEE, 2014.

Tianpei Xie, Nasser M Nasrabadi, and Alfred O Hero. Semi-supervisedmulti-sensor classification via consensus-based multi-view maximumentropy discrimination. In Acoustics, Speech and Signal Processing(ICASSP), 2015 IEEE International Conference on, pages 1936–1940.IEEE, 2015.

Linli Xu, Koby Crammer, and Dale Schuurmans. Robust support vectormachine training via convex outlier ablation. AAAI, 6:536–542, 2006.

11 / 12


Thank you!

12 / 12