notesum.ai

Published at October 23

Beyond position: how rotary embeddings shape representations and memory in autoregressive transfomers

cs.LG
cs.AI

Released Date: October 23, 2024

Authors: Valeria Ruscio1, Fabrizio Silvestri1

Aff.: 1Sapienza University of Rome

Arxiv: https://arxiv.org/abs/2410.18067v1