notesum.ai

Published at October 21

Mesa-Extrapolation: A Weave Position Encoding Method for Enhanced Extrapolation in LLMs

quant-ph
cs.CC

Released Date: October 21, 2024

Authors: Xin Ma1, Yang Liu2, Jingjing Liu2, Xiaoxu Ma1

Aff.: 1Digital Research Institute, Enn Group, Beijing, China; 2Institute for AI Industry Research, Tsinghua University, Beijing, China

Arxiv: https://arxiv.org/abs/2410.15859v1