Unsupervised Learning

• Data : x (data with no labels)
• Goal : Learn some underlying hidden structure of the data
• Examples Clustering, dimensionalilty reduction, density estimation, Autoencoders(Feature Learning)

Generative Models

• Given training data, generate new samples from same distribution

• Several flavors:
  • Explicit density estimation : Explicitly define and solve for $p_{model}(x)$
  • Implicit density estimation : Learn model that can sample from $p_{model}(x)$ without explicitly defining it
• Why Generative Models?
  • Realistic samples for artwork, super-resolution, colorization
  • Learn useful features for downstream tasks
  • Getting insights from high-dimensional data
  • Modeling physical world for simulation and planning

PixelRNN and PixelCNN

Fully visible belief network

• Explicit density model
• Use chain rule to decompose likelihood of an image x into product of 1-d distributions:

PicelRNN

PixcelCNN