notesum.ai

Published at November 19

Evaluating Tokenizer Performance of Large Language Models Across Official Indian Languages

cs.CL
cs.AI

Released Date: November 19, 2024

Authors: S. Tamang, D. J. Bora

Arxiv: http://arxiv.org/abs/2411.12240v1