notesum.ai

Published at December 4

Training-Free Mitigation of Language Reasoning Degradation After Multimodal Instruction Tuning

cs.CV

cs.AI

Released Date: December 4, 2024

Authors: Neale Ratzlaff¹, Man Luo¹, Xin Su¹, Vasudev Lal¹, Phillip Howard¹

Aff.: ¹Intel Labs

Arxiv: http://arxiv.org/pdf/2412.03467v1

Refer to caption

	$\alpha$	Hellaswag	MMLU	CQA	Wino	Arc-E	Race-H	OBQA	GSM8k
Llava 1.6 Mistral	0	.5971	.5783	.7215	.7120	.7820	.4459	.3140	.3282
Llava 1.6 Mistral	0.05	.6161	.5913	.7395	.7277	.8108	.4409	.3300	.3995
Llava 1.6 Mistral	0.1	.6424	.5931	.7403	.7285	.8282	.4411	.3420	.4306
Llava 1.6 Mistral	0.5	.6524	.5931	.6847	.7293	.8274	.4584	.3540	.4079
Llava 1.6 Mistral	0.9	.6598	.5907	.6667	.7411	.8140	.4612	.3620	.3889
Llava 1.6 Mistral	0.95	.6601	.5905	.6658	.7403	.8131	.4612	.3520	.3904
Mistral 0.2 LM	N/A	.6602	.5901	.6683	.7371	.8130	.4612	.3560	.3950
Llava 1.6 Vicuna	0	.5599	.5187	.6961	.7040	.7690	.4325	.3480	.1683
Llava 1.6 Vicuna	0.05	.5672	.5124	.7060	.6992	.7750	.4296	.3500	.1736
Llava 1.6 Vicuna	0.1	.5684	.5117	.6887	.6977	.7761	.4239	.3520	.1774
Llava 1.5 Vicuna	0	.5628	.4936	.6832	.7050	.7622	.4280	.3600	.1667
Llava 1.5 Vicuna	0.05	.5685	.4989	.6789	.7008	.7710	.4354	.3680	.1758
Llava 1.5 Vicuna	0.1	.5699	.4992	.6691	.7072	.7706	.4267	.3660	.1774
Vicuna 1.5 LM	N/A	.5648	.4862	.5986	.7000	.7554	.4162	.3300	.2010