notesum.ai

Published at December 9

Pilot-guided Multimodal Semantic Communication for Audio-Visual Event Localization

cs.SD

cs.CV

cs.MM

eess.AS

Released Date: December 9, 2024

Authors: Fei Yu¹, Zhe Xiang, Nan Che, Zhuoran Zhang, Yuandi Li, Junxiao Xue, Zhiguo Wan

Aff.: ¹University

Arxiv: http://arxiv.org/pdf/2412.06208v1

Model Type	SNR
Model Type	0dB	3dB	6dB	9dB	12dB	15dB	18dB	21dB	24dB	27dB	30dB
AWGN Channel
Audio (only)	0.1535	0.1612	0.1602	0.1648	0.1740	0.1740	0.1727	0.1708	0.1776	0.1867	0.1808
	$\downarrow$ 5.69%	$\downarrow$ 3.58%	$\downarrow$ 9.00%	$\downarrow$ 6.08%	$\downarrow$ 2.88%	$\downarrow$ 3.47%	$\downarrow$ 6.36%	$\downarrow$ 7.34%	$\downarrow$ 3.67%		$\downarrow$ 2.13%
Video (only)	0.1844	0.1844	0.1864	0.1855	0.2016	0.2055	0.2016	0.2094	0.2000	0.2068	0.2050
		$\downarrow$ 1.26%	$\downarrow$ 0.20%	$\downarrow$ 5.66%	$\downarrow$ 0.39%		$\downarrow$ 3.73%	$\downarrow$ 2.55%	$\downarrow$ 6.77%	$\downarrow$ 3.76%	$\downarrow$ 4.57%
Multimodal	0.2341	0.5380	0.5865	0.5898	0.6005	0.6000	0.5711	0.6143	0.6120	0.6177	0.6144
	$\downarrow$ 60.17%	$\downarrow$ 17.75%	$\downarrow$ 11.44%	$\downarrow$ 14.46%	$\downarrow$ 15.03%	$\downarrow$ 15.08%	$\downarrow$ 19.35%	$\downarrow$ 14.00%	$\downarrow$ 14.48%	$\downarrow$ 14.06%	$\downarrow$ 15.20%
Rayleigh Channel
Audio (only)	0.1429	0.1703	0.1698	0.1778	0.1792	0.1841	0.1799	0.1854	0.1901	0.1922	0.1914
	$\downarrow$ 12.19%		$\downarrow$ 3.52%	$\downarrow$ 2.49%			$\downarrow$ 2.40%	$\downarrow$ 1.17%
Video (only)	0.1705	0.1721	0.1797	0.1880	0.1836	0.1911	0.1986	0.1912	0.2016	0.2000	0.1976
	$\downarrow$ 5.93%	$\downarrow$ 6.64%	$\downarrow$ 4.17%	$\downarrow$ 1.37%	$\downarrow$ 4.99%	$\downarrow$ 2.13%		$\downarrow$ 4.13%		$\downarrow$ 1.79%	$\downarrow$ 3.95%
Multimodal	0.1844	0.1836	0.2154	0.2284	0.2448	0.2826	0.2932	0.3169	0.2932	0.3487	0.3432
	$\downarrow$ 69.04%	$\downarrow$ 69.13%	$\downarrow$ 65.18%	$\downarrow$ 67.26%	$\downarrow$ 64.87%	$\downarrow$ 59.79%	$\downarrow$ 58.10%	$\downarrow$ 55.24%	$\downarrow$ 59.01%	$\downarrow$ 51.47%	$\downarrow$ 52.38%
Rician Channel
Audio (only)	0.1624	0.1607	0.1680	0.1687	0.1753	0.1816	0.1801	0.1850	0.1830	0.1901	0.1820
						$\downarrow$ 0.32%	$\downarrow$ 1.20%		$\downarrow$ 3.18%		$\downarrow$ 4.31%
Video (only)	0.1735	0.1810	0.1852	0.1906	0.1919	0.1896	0.1927	0.1977	0.2016	0.2010	0.2015
		$\downarrow$ 0.32%	$\downarrow$ 3.38%		$\downarrow$ 1.73%	$\downarrow$ 2.28%	$\downarrow$ 3.21%	$\downarrow$ 2.94%	$\downarrow$ 1.02%	$\downarrow$ 0.47%	$\downarrow$ 0.76%
Multimodal	0.1273	0.2078	0.2284	0.2378	0.2698	0.2648	0.3063	0.3758	0.3500	0.3557	0.3487
	$\downarrow$ 62.73%	$\downarrow$ 56.86%	$\downarrow$ 55.55%	$\downarrow$ 55.25%	$\downarrow$ 50.50%	$\downarrow$ 53.09%	$\downarrow$ 50.25%	$\downarrow$ 43.38%	$\downarrow$ 49.32%	$\downarrow$ 50.27%	$\downarrow$ 51.47%