UCL logo

UCL Discovery

UCL home » Library Services » Electronic resources » UCL Discovery

Interpretation of hybrid generative/discriminative algorithms

Xue, JH; Titterington, DM; (2009) Interpretation of hybrid generative/discriminative algorithms. NEUROCOMPUTING , 72 (7-9) 1648 - 1655. 10.1016/j.neucom.2008.08.009.

Full text not available from this repository.

Abstract

In discriminant analysis, probabilistic generative and discriminative approaches represent two paradigms of statistical modelling and learning. In order to exploit the best of both worlds, hybrid modelling and learning techniques have attracted much research interest recently, one example being the so-called hybrid generative/discriminative algorithm proposed in Raina et al. [Classification with hybrid generative/discriminative models, in: NIPS, 2003] and its multi-class extension [A. Fujino, N. Ueda, K. Saito, A hybrid generative/discriminative approach to text classification with additional information, Inf. Process. Manage. 43(2) (2007) 379-392]. In this paper, we interpret this hybrid algorithm from three perspectives, namely class-conditional probabilities, class-posterior probabilities and loss functions underlying the model. We suggest that the hybrid algorithm is by nature a generative model with its parameters learnt through both generative and discriminative approaches, in the sense that it assumes a scaled data-generation process and uses scaled class-posterior probabilities to perform discrimination. Our suggestion can also be applied to its multi-class extension. In addition, using simulated and real-world data, we compare the performance of the normalised hybrid algorithm as a classifier with that of the naive Bayes classifier and linear logistic regression. Our simulation studies suggest in general the following: if the covariance matrices are diagonal matrices, the naive Bayes classifier performs the best; if the covariance matrices are full matrices, linear logistic regression performs the best. Our studies also suggest that the hybrid algorithm may provide worse performance than either the naive Bayes classifier or linear logistic regression alone. (C) 2008 Elsevier B.V. All rights reserved.

Type: Article
Title: Interpretation of hybrid generative/discriminative algorithms
DOI: 10.1016/j.neucom.2008.08.009
Keywords: Hybrid generative/discriminative models, Probabilistic generative and discriminative approaches, Statistical modelling and learning
UCL classification: UCL > Provost and Vice Provost Offices
UCL > Provost and Vice Provost Offices > UCL BEAMS
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Maths and Physical Sciences
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Maths and Physical Sciences > Dept of Statistical Science
URI: http://discovery.ucl.ac.uk/id/eprint/88419
Downloads since deposit
0Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item