notesum.ai

Published at November 6

Fine-tuning -- a Transfer Learning approach

cs.LG

cs.AI

Released Date: November 6, 2024

Authors: Joseph Arul Raj¹, Linglong Qian¹, Zina Ibrahim¹

Aff.: ¹King's College London

Arxiv: http://arxiv.org/abs/2411.03941v1

Refer to caption

Model	Dataset	Optuna Hyperparameter Tuning		CyclicLR scheduler		ReduceLROnPlateau scheduler		Average performance
Model	Dataset	Val AUC	Test AUC	Val AUC	Test AUC	Val AUC	Test AUC	Val AUC	Test AUC
2 layer MLP	Physionet 2012	0.783	0.777	0.786	0.782	0.783	0.778	0.784	0.779
5 layer MLP	Physionet 2012	0.780	0.768	0.789	0.763	0.786	0.764	0.785	0.765
1 layer LSTM	Physionet 2012	0.785	0.780	0.803	0.777	0.802	0.786	0.797	0.781
1 layer GRU	Physionet 2012	0.783	0.758	0.800	0.786	0.798	0.792	0.794	0.779
2 layer MLP	MIMIC III 89 features	0.766	0.755	0.765	0.754	0.766	0.756	0.766	0.755
5 layer MLP	MIMIC III 89 features	0.739	0.720	0.755	0.694	0.751	0.705	0.748	0.706
1 layer LSTM	MIMIC III 89 features	0.818	0.787	0.834	0.791	0.830	0.806	0.827	0.795
1 layer GRU	MIMIC III 89 features	0.810	0.782	0.834	0.775	0.832	0.792	0.825	0.783