We investigate the redundancy within Transformer layers and propose an effective layer-based pruning method.
This work contributes to the investigation of the lower bounds of the Base in RoPE, providing a theoretical foundation for the long-context extrapolation of models.
