notesum.ai

Published at November 6

Bio-xLSTM: Generative modeling, representation and in-context learning of biological and chemical sequences

q-bio.BM

cs.AI

cs.LG

Released Date: November 6, 2024

Authors: Niklas Schmidinger¹, Lisa Schneckenreiter¹, Philipp Seidl¹, Johannes Schimunek¹, Pieter-Jan Hoedt¹, Johannes Brandstetter², Andreas Mayr¹, Sohvi Luukkonen¹, Sepp Hochreiter², Günter Klambauer²

Aff.: ¹ELLIS Unit Linz and LIT AI Lab, Institute for Machine Learning, Johannes Kepler University, Linz, Austria; ²ELLIS Unit Linz and LIT AI Lab, Institute for Machine Learning, Johannes Kepler University, Linz, Austria; NXAI GmbH, Linz, Austria

Arxiv: http://arxiv.org/abs/2411.04165v1

Task	Metric	HyenaDNA	Mamba-PS^a	Mamba-PH^a	xLSTM-PS	xLSTM-PH
Histone Markers
H3	MCC $\uparrow$	$0.779^{\pm 0.037}$	$0.799^{\pm 0.029}$	$0.815^{\pm 0.048}$	$0.796^{\pm 0.014}$	$0.824^{\pm 0.010}$
H3K14AC	MCC $\uparrow$	$0.612^{\pm 0.065}$	$0.541^{\pm 0.212}$	$0.631^{\pm 0.026}$	$0.570^{\pm 0.008}$	$0.598^{\pm 0.017}$
H3K36ME3	MCC $\uparrow$	$0.613^{\pm 0.041}$	$0.609^{\pm 0.109}$	$0.601^{\pm 0.129}$	$0.588^{\pm 0.019}$	$0.625^{\pm 0.010}$
H3K4ME1	MCC $\uparrow$	$0.512^{\pm 0.024}$	$0.488^{\pm 0.102}$	$0.523^{\pm 0.039}$	$0.490^{\pm 0.012}$	$0.526^{\pm 0.001}$
H3K4ME2	MCC $\uparrow$	$0.455^{\pm 0.095}$	$0.388^{\pm 0.101}$	$0.487^{\pm 0.170}$	$0.489^{\pm 0.024}$	$0.504^{\pm 0.012}$
H3K4ME3	MCC $\uparrow$	$0.549^{\pm 0.056}$	$0.440^{\pm 0.202}$	$0.544^{\pm 0.045}$	$0.520^{\pm 0.019}$	$0.537^{\pm 0.012}$
H3K79ME3	MCC $\uparrow$	$0.672^{\pm 0.048}$	$0.676^{\pm 0.026}$	$0.697^{\pm 0.077}$	$0.662^{\pm 0.011}$	$0.697^{\pm 0.007}$
H3K9AC	MCC $\uparrow$	$0.581^{\pm 0.061}$	$0.604^{\pm 0.048}$	$0.622^{\pm 0.030}$	$0.622^{\pm 0.013}$	$0.627^{\pm 0.008}$
H4	MCC $\uparrow$	$0.763^{\pm 0.044}$	$0.789^{\pm 0.020}$	$0.811^{\pm 0.022}$	$0.793^{\pm 0.011}$	$0.813^{\pm 0.008}$
H4AC	MCC $\uparrow$	$0.564^{\pm 0.038}$	$0.525^{\pm 0.240}$	$0.621^{\pm 0.054}$	$0.558^{\pm 0.018}$	$0.583^{\pm 0.014}$
Regulatory Annotation
Enhancer	MCC $\uparrow$	$0.517^{\pm 0.117}$	$0.491^{\pm 0.066}$	$0.546^{\pm 0.073}$	$0.375^{\pm 0.030}$	$0.545^{\pm 0.024}$
Enhancer Types	MCC $\uparrow$	$0.386^{\pm 0.185}$	$0.416^{\pm 0.095}$	$0.439^{\pm 0.054}$	$0.444^{\pm 0.046}$	$0.466^{\pm 0.011}$
Promoter: All	F1 $\uparrow$	$0.960^{\pm 0.005}$	$0.967^{\pm 0.004}$	$0.970^{\pm 0.004}$	$0.962^{\pm 0.002}$	$0.967^{\pm 0.001}$
NonTATA	F1 $\uparrow$	$0.959^{\pm 0.011}$	$0.968^{\pm 0.006}$	$0.968^{\pm 0.010}$	$0.963^{\pm 0.002}$	$0.970^{\pm 0.001}$
TATA	F1 $\uparrow$	$0.944^{\pm 0.040}$	$0.957^{\pm 0.015}$	$0.953^{\pm 0.016}$	$0.948^{\pm 0.006}$	$0.952^{\pm 0.005}$
Splice Site Annotation
All	Accuracy $\uparrow$	$0.956^{\pm 0.011}$	$0.927^{\pm 0.021}$	$0.940^{\pm 0.027}$	$0.965^{\pm 0.006}$	$0.974^{\pm 0.004}$
Acceptor	F1 $\uparrow$	$0.958^{\pm 0.010}$	$0.936^{\pm 0.077}$	$0.937^{\pm 0.033}$	$0.970^{\pm 0.005}$	$0.953^{\pm 0.008}$
Donor	F1 $\uparrow$	$0.949^{\pm 0.024}$	$0.874^{\pm 0.289}$	$0.948^{\pm 0.025}$	$0.962^{\pm 0.004}$	$0.951^{\pm 0.005}$