LLM Redundancy

ShortGPT: Layers in Large Language Models are More Redundant Than You Expect
We investigate the redundancy within Transformer layers and propose an effective layer-based pruning method.
Xin Men, Xin Men, Mingyu Xu, Mingyu Xu, Qingyu Zhang, Qingyu Zhang, Qianhao Yuan, Qianhao Yuan, Bingning Wang, Bingning Wang, Hongyu Lin, Hongyu Lin, Yaojie Lu, Yaojie Lu, Xianpei Han, Xianpei Han, Weipeng Chen, Weipeng Chen
PDF Cite
ShortGPT: Layers in Large Language Models are More Redundant Than You Expect
ShortGPT: Layers in Large Language Models are More Redundant Than You Expect
We investigate the redundancy within Transformer layers and propose an effective layer-based pruning method.
Xin Men, Mingyu Xu, Qingyu Zhang, Qianhao Yuan, Bingning Wang, Hongyu Lin, Yaojie Lu, Xianpei Han, Weipeng Chen
Last updated on Aug 5, 2025
PDF Cite
ShortGPT: Layers in Large Language Models are More Redundant Than You Expect