Emerging techniques long context
Guides implementation and comparison of RoPE, YaRN, ALiBi, and position interpolation for extending transformer context windows. Use when a user wants to extend a model's context length, needs implementation code for one of these techniques, wants trade-offs c