Static VS Dynamic

Static Computational graphs

• Define an architecture → Run data to train and make predictions
• Define-and-run
• Powerful offline optimization/scheduling
• Fast (in general)
• Handling data can be ugly

Dynamic Computational graphs

• Defined implicitly as the forward computation is executed
• Define-by-run
• Building and execution are intertwined → need to keep code around
• Line by Line execution of code → Debug Friendly

• step1
• step2
• step3
• step4

Pytorch VS TensorFlow