notesum.ai

Published at November 17

BianCang: A Traditional Chinese Medicine Large Language Model

cs.CL

cs.AI

Released Date: November 17, 2024

Authors: Sibo Wei¹, Xueping Peng², Yi-fei Wang³, Jiasheng Si¹, Weiyu Zhang¹, Wenpeng Lu¹, Xiaoming Wu¹, Yinglong Wang¹

Aff.: ¹Key Laboratory of Computing Power Network and Information Security, Ministry of Education, Shandong Computer Science Center (National Supercomputer Center in Jinan), Qilu University of Technology (Shandong Academy of Sciences), Jinan, China; ²Australian Artificial Intelligence Institute, University of Technology Sydney, Sydney, Australia; ³Affiliated Hospital of Shandong University of Traditional Chinese Medicine, Jinan, China

Arxiv: http://arxiv.org/abs/2411.11027v1

Model	TCM Syndrome Differentiation				TCM Disease Diagnosis				TCM Exam
	TCMSD		TCMSD-BC		TCMDD		TCMDD-BC		MLEC-TCM		MLEC-CWM
	Acc.(%)		Acc.(%)		Acc.(%)		Acc.(%)		Acc.(%)		Acc.(%)
	DI	CoT	DI	CoT	DI	CoT	DI	CoT	ZS	FS	ZS	FS
GPT-4	24.53	45.21	16.67	70.73	27.83	54.54	41.80	68.33	74.70	76.35	76.26	76.37
BianQue2-6B	7.00	13.87	15.13	39.67	14.76	21.35	10.33	13.93	4.73	3.30	3.85	3.25
DoctorGLM-6B	9.17	9.46	1.13	4.87	11.65	28.67	5.60	17.33	33.72	31.07	32.86	30.09
BenTsao-Llama-7B	0.00	0.11	0.80	2.47	4.50	9.79	3.33	9.33	23.00	21.12	22.69	21.46
ShenNong-TCM-LLM-7B	0.18	1.93	0.07	0.67	7.91	19.23	6.20	14.93	23.36	21.51	22.58	22.65
Taiyi-7B	22.57	22.26	19.87	33.20	23.71	31.52	33.47	54.60	43.05	44.80	43.93	45.16
WiNGPT2-7B-Base	13.91	18.19	15.47	28.00	20.87	33.25	34.00	40.73	54.49	55.04	56.56	56.37
HuatuoGPT2-7B	30.48	34.09	38.13	46.07	24.61	39.96	63.00	35.60	47.23	50.37	48.19	49.98
Sunsimiao-7B	14.53	18.88	12.93	28.87	27.11	39.83	51.60	49.80	70.39	76.68	68.45	75.40
WiNGPT2-Llama3-8B-Base	4.65	8.55	4.67	17.00	32.41	34.98	49.07	16.67	51.12	49.69	52.79	52.64
QiZhen-CaMA-13B	4.39	6.87	4.60	9.20	10.59	14.05	16.13	14.53	19.92	21.83	19.33	21.50
Zhongjing-13B	0.80	6.29	0.80	16.00	14.40	25.65	13.87	19.60	23.03	23.00	23.63	22.69
ChiMedGPT-13B	1.04	3.52	1.87	5.20	6.84	9.42	5.87	5.87	22.45	23.23	23.48	26.21
DISC-MedLLM-13B	2.55	3.54	0.13	1.80	9.86	16.53	14.07	12.13	37.09	35.57	37.53	35.44
HuatuoGPT2-13B	22.60	22.06	23.53	44.60	27.73	41.49	22.73	26.67	49.37	50.11	50.69	49.91
Lingdan-13B	31.68	23.02	27.93	52.00	42.67	48.74	70.93	59.80	58.99	58.18	57.57	57.64
GLM4-9B	27.42	30.00	25.07	32.53	36.33	52.15	65.20	74.20	76.06	76.51	75.93	76.00
Qwen2-7B	31.74	27.18	32.73	28.40	41.60	54.59	74.87	77.93	86.01	89.18	84.45	87.89
Qwen2-7B-Instruct	25.70	33.41	14.27	57.00	32.87	52.92	60.40	60.13	83.61	84.22	79.89	82.99
Qwen2.5-7B	30.44	21.29	17.87	35.73	23.71	43.88	63.87	71.27	83.32	85.52	82.02	84.04
Qwen2.5-7B-Instruct	24.30	32.19	9.93	57.07	36.29	51.51	62.93	55.53	78.72	79.88	77.27	78.43
Qwen2.5-14B	35.62	25.21	33.93	30.13	24.33	36.64	33.33	32.80	86.59	89.93	87.10	90.06
Qwen2.5-14B-Instruct	25.94	35.03	16.07	60.00	38.30	49.31	46.27	53.67	82.25	84.81	81.79	85.68
BianCang-Qwen2-7B (Ours)	42.14	30.30	57.80	48.00	43.73	54.67	74.73	80.67	90.86	91.87	89.08	90.36
BianCang-Qwen2-7B-Instruct (Ours)	68.88	75.96	57.33	75.40	64.42	77.71	89.07	85.67	92.39	92.39	91.14	91.48
BianCang-Qwen2.5-7B (Ours)	46.57	26.72	52.93	45.47	49.80	53.15	68.13	61.73	86.46	86.30	83.93	85.35
BianCang-Qwen2.5-7B-Instruct (Ours)	78.90	82.10	66.73	77.73	73.73	82.65	87.87	89.40	90.22	90.57	90.32	90.62
BianCang-Qwen2.5-14B (Ours)	43.77	33.96	61.93	53.47	66.61	60.39	82.93	77.07	89.28	90.86	89.42	90.58
BianCang-Qwen2.5-14B-Instruct (Ours)	79.38	75.54	62.27	70.73	77.63	82.05	86.33	88.73	92.29	92.29	92.75	92.86