Deep Generative Models

• Generative Models is not just about generating images

Learning a Generative Model

• suppose we are given images of dogs
• we want to learn a probability distribution $p(x)$ s.t
  • Generation : if we sample $x_{new}$ ~ $p(x), x_{new}$ should look like a dog → sampling
  • Density estimation : $p(x)$ should be high if x looks like a dog, and low otherwise → anomaly detection
    • Also known as, explicit models
  • Unsupervised representation learning : We should be able to learn what these images have in common (tails, ears..) → feature learning

Basic Discrete Distributions

• Bernoulli distribution
• Categorical distribution
• Example : Modeling an RGB joint distribution
  • $(r,g,b)$ ~ $p(R,G,B)$
  • number of cases → 256 * 256 * 256
  • parameters to specify → 256 * 256 * 256 -1
• Example :

Structure Through Independence

Conditional Independence

• Three important rules

• Chain rule:

• Bayes’ rule:

• Conditional independence: