Generative Model
A Generative Model learns the joint probability distribution p (x, y). It predicts the conditional probability with the help of Bayes Theorem. A Generative Model explicitly models the actual distribution of each class.
Generative classifiers
- Assume some functional form for P(Y), P(X|Y)
- Estimate parameters of P(X|Y), P(Y) directly from training data
- Use Bayes rule to calculate P (Y |X)
Generative classifiers examples
- Naïve Bayes
- Bayesian networks
- Markov random fields
- Hidden Markov Models (HMM)
Discriminative Model
A Discriminative model models the decision boundary between the classes. A Discriminative model learns the conditional probability distribution p (y |x).
Discriminative Classifiers
- Assume some functional form for P(Y|X)
- Estimate parameters of P(Y|X) directly from training data
Discriminative Classifiers
- Logistic regression
- Scalar Vector Machine
- Traditional neural networks
- Nearest neighbor
- Conditional Random Fields (CRF)s