Positional Encoding

Base of RoPE Bounds Context Length
This work contributes to the investigation of the lower bounds of the Base in RoPE, providing a theoretical foundation for the long-context extrapolation of models.
Xin Men*, Xin Men*, Mingyu Xu*, Mingyu Xu*, Bingning Wang†, Bingning Wang†, Qingyu Zhang, Qingyu Zhang, Hongyu Lin, Hongyu Lin, Xianpei Han, Xianpei Han, Weipeng Chen, Weipeng Chen
PDF Cite
Base of RoPE Bounds Context Length
Base of RoPE Bounds Context Length
This work contributes to the investigation of the lower bounds of the Base in RoPE, providing a theoretical foundation for the long-context extrapolation of models.
Xin Men*, Mingyu Xu*, Bingning Wang†, Qingyu Zhang, Hongyu Lin, Xianpei Han, Weipeng Chen
Last updated on Aug 5, 2025
PDF Cite
Base of RoPE Bounds Context Length