Original Positional Encoding

• Absolute Position Encoding

From Language Models

• Relative Positional Encoding
• Attention with Linear Biases
• Rotary Position Embedding
• https://github.com/google-deepmind/randomized_positional_encodings

From Vision Transformers

• Conditional Positional Encoding