State of the Art on Neural Rendering

• Survey on Nerual Rendering(image synthesis) : Neural volume Rendering + image synthesis

Theoretical Fundatmentals

1. Physical Image Formation

Classical Computer Graphics : Approximating the physical process of image formation in the real world (light transport) by modeling

• light source → photons → interact with objects in scene(function of geometry and material properties) → camera

1.1 Scene Representation

• Explicit representations
  • Describe scenes as collection of geometric primitives(triangles, points etc)
• Implicit representations
  • Signed distant functions → $\mathbb{R}^3 \rightarrow \mathbb{R}$

1.2 Camera Models

• Pinhole camera model
  • Parameterized by pinhole 3D location, image plane, rectangular region in that plane

1.3 Classical Rendering

• Rasterization
  • Converting Vector image(mathematically defined) to image(pixels)
• Ray tracing
  • Backward of light transport in the real world
  • Nvidia RTX

Neural Rendering

“deep image or video generation approaches that enable explicit or implicit control of scene properties such as illumination, camera parameters, pose, geometry, appearance, and semantic structure.”

Given high quality scene specifications and computing power classic rendering methods can render photo realistic images for variety of complex real world phenomena / Explicit editing control

→ BUT!! Manual Effort