How do you make good estimates when many solutions are consistent with the data? In the 2005 SSP paper we show how to effectively use a Bayesian principle of minimum expected risk to classify neighbors using weighted near-neighbor methods. In previous principled estimation research with Michael Friedlander, we extend results by Kolmogorov and Campbell to show that solutions which satisfy the minimum relative entropy or maximum entropy estimation principles are bounded exponential probability distributions even when the constraints are formulated as penalty constraints, as they may be in the case of noise, uncertainty, and multiple objectives. In recent work we apply Bayesian minimum expected risk estimation to near-neighbor statistical learning and parametric statistical learning. We are particularly interested in applying Bayesian estimation to distributions, and to this end have been working on a functional Bregman divergence with useful mathematical properties.

Personnel:

Maya R. Gupta (EE Associate Professor)

Luca Cazzanti (AFCEA Fellow)

Santosh Srivastava (PhD, UW Applied Math 2007)

Bela Frigyik (Post-doc, Dept. of Math., Purdue Univ.)

Publications:

"Functional Bregman Divergence and Bayesian Estimation of Distributions," Bela A. Frigyik, Santosh Srivastava, and Maya R. Gupta, IEEE Trans. on Information Theory, vol. 54, no. 11, 5130-5139, 2008.

"Bayesian Quadratic Discriminant Analysis," Santosh Srivastava, Maya R. Gupta, and Bela Frigyik, Journal of Machine Learning Research, vol. 8, pp. 1277-1305, 2007. Complete Simulation Code

"Distribution-based Bayesian minimum expected risk parametric classification," Santosh Srivastava and Maya R. Gupta, Proc of the IEEE Intl. Symposium on Information Theory, 2006.

"Minimum Expected Risk Estimation for Near-neighbor Classification," Maya R. Gupta, S. Srivastava and L. Cazzanti, Univ. of Washington Dept. of Electrical Engineering Technical Report 2006-0006, 2006.

"On minimizing distortion and relative entropy," Michael P. Friedlander and Maya R. Gupta, IEEE Trans. on Information Theory, vol. 52, no. 1, pp. 238-245, 2006.

"Minimum expected risk probability estimates for nonparametric neighborhood classifiers," Maya R. Gupta, Luca Cazzanti, and Santosh Srivastava, Proceedings of the IEEE Workshop on Statistical Signal Processing, 2005.

"A Principle of Minimum Expected Risk," Maya R. Gupta, Proceedings of the 2004 IEEE International Symposium on Information Theory, p. 167, 2004.

Related Work:

Bayesian Quadratic Discriminant Analysis

Global Optimization

Nonparametric Near-neighbor Statistical Learning