notesum.ai

Published at October 30

DiaMond: Dementia Diagnosis with Multi-Modal Vision Transformers Using MRI and PET

cs.CV

cs.AI

Released Date: October 30, 2024

Authors: Yitong Li¹, Morteza Ghahremani¹, Youssef Wally¹, Christian Wachinger¹

Aff.: ¹Lab for Artificial Intelligence in Medical Imaging, Technical University of Munich (TUM), Germany

Arxiv: http://arxiv.org/abs/2410.23219v1

Data	Method	Modality	CN vs. AD		CN vs. MCI vs. AD
Data	Method	Modality	BACC	AUC	BACC	F1-Score
ADNI	3D-ResNet [26]	$\mathbf{M}$	86.59 $\pm$ 4.22	93.67 $\pm$ 2.43	53.58 $\pm$ 6.20	53.70 $\pm$ 6.16
	3D-ResNet [26]	$\mathbf{P}$	89.07 $\pm$ 2.67	95.19 $\pm$ 2.22	54.84 $\pm$ 3.40	55.79 $\pm$ 4.22
	3D-ViT [28]	$\mathbf{M}$	86.22 $\pm$ 3.83	93.66 $\pm$ 2.50	60.83 $\pm$ 7.41	60.03 $\pm$ 7.45
	3D-ViT [28]	$\mathbf{P}$	88.75 $\pm$ 1.69	93.53 $\pm$ 1.39	58.24 $\pm$ 4.64	53.20 $\pm$ 4.41
	ResNet-based early-fusion[30]	$\mathbf{M}$ + $\mathbf{P}$	82.66 $\pm$ 2.40	85.58 $\pm$ 7.29	57.30 $\pm$ 2.30	55.79 $\pm$ 1.02
	ResNet-based middle-fusion[26]	$\mathbf{M}$ + $\mathbf{P}$	82.63 $\pm$ 7.04	88.17 $\pm$ 6.77	53.01 $\pm$ 3.40	53.70 $\pm$ 6.16
	ResNet-based late-fusion[26]	$\mathbf{M}$ + $\mathbf{P}$	89.74 $\pm$ 1.98	96.73 $\pm$ 0.92	57.71 $\pm$ 2.90	58.89 $\pm$ 1.73
	ViT-based early-fusion	$\mathbf{M}$ + $\mathbf{P}$	89.20 $\pm$ 3.29	94.87 $\pm$ 1.80	62.89 $\pm$ 2.08	59.84 $\pm$ 0.73
	ViT-based late-fusion	$\mathbf{M}$ + $\mathbf{P}$	90.60 $\pm$ 3.57	96.48 $\pm$ 1.24	61.86 $\pm$ 3.00	59.60 $\pm$ 5.83
	Mul-T [8]	$\mathbf{M}$ + $\mathbf{P}$	86.37 $\pm$ 3.25	93.59 $\pm$ 1.54	56.49 $\pm$ 3.82	55.54 $\pm$ 3.94
	MMTFN [25]	$\mathbf{M}$ + $\mathbf{P}$	88.76 $\pm$ 1.98	93.69 $\pm$ 1.95	63.11 $\pm$ 4.51	60.51 $\pm$ 3.49
	DiaMond (Ours)	$\mathbf{M}$ + $\mathbf{P}$	92.42 $\pm$ 2.63	97.11 $\pm$ 1.47	65.18 $\pm$ 1.57	64.89 $\pm$ 2.78
J-ADNI	3D-ResNet [26]	$\mathbf{M}$	84.66 $\pm$ 2.92	91.40 $\pm$ 3.16	56.89 $\pm$ 1.64	54.87 $\pm$ 1.83
	3D-ResNet [26]	$\mathbf{P}$	85.48 $\pm$ 9.87	92.22 $\pm$ 4.51	50.87 $\pm$ 12.8	46.16 $\pm$ 18.7
	3D-ViT [28]	$\mathbf{M}$	83.65 $\pm$ 4.15	92.26 $\pm$ 1.29	55.11 $\pm$ 1.62	47.71 $\pm$ 5.23
	3D-ViT [28]	$\mathbf{P}$	89.42 $\pm$ 3.08	96.14 $\pm$ 1.16	51.77 $\pm$ 5.45	51.32 $\pm$ 4.74
	ResNet-based early-fusion[30]	$\mathbf{M}$ + $\mathbf{P}$	86.02 $\pm$ 6.85	91.61 $\pm$ 3.82	53.46 $\pm$ 3.92	53.28 $\pm$ 3.70
	ResNet-based middle-fusion[26]	$\mathbf{M}$ + $\mathbf{P}$	77.73 $\pm$ 8.97	74.12 $\pm$ 6.78	49.48 $\pm$ 2.85	43.94 $\pm$ 7.39
	ResNet-based late-fusion[26]	$\mathbf{M}$ + $\mathbf{P}$	88.81 $\pm$ 3.36	94.85 $\pm$ 1.79	50.20 $\pm$ 6.23	48.90 $\pm$ 6.50
	ViT-based early-fusion	$\mathbf{M}$ + $\mathbf{P}$	89.59 $\pm$ 3.68	93.86 $\pm$ 2.98	54.68 $\pm$ 5.32	52.25 $\pm$ 3.42
	ViT-based late-fusion	$\mathbf{M}$ + $\mathbf{P}$	86.70 $\pm$ 7.73	96.34 $\pm$ 2.42	54.07 $\pm$ 5.80	50.83 $\pm$ 4.89
	Mul-T [8]	$\mathbf{M}$ + $\mathbf{P}$	81.02 $\pm$ 6.08	88.45 $\pm$ 2.96	50.04 $\pm$ 7.28	45.66 $\pm$ 8.46
	MMTFN [25]	$\mathbf{M}$ + $\mathbf{P}$	86.74 $\pm$ 6.05	84.27 $\pm$ 7.91	57.55 $\pm$ 2.81	53.45 $\pm$ 6.97
	DiaMond (Ours)	$\mathbf{M}$ + $\mathbf{P}$	91.72 $\pm$ 2.52	96.20 $\pm$ 2.50	58.44 $\pm$ 3.61	58.88 $\pm$ 1.73