notesum.ai

Published at November 7

DISCO: DISCovering Overfittings as Causal Rules for Text Classification Models

cs.AI

cs.CL

cs.LG

I.2.3; I.2.7

Released Date: November 7, 2024

Authors: Zijian Zhang¹, Vinay Setty², Yumeng Wang³, Avishek Anand⁴

Aff.: ¹Leibniz Universität Hannover; ²University of Stavanger; ³Leiden Institute of Advanced Computer Science, Leiden University; ⁴Delft University of Technology

Arxiv: http://arxiv.org/abs/2411.04649v1

dataset	model	rule	synthetic counterfactual
SST-2	$\text{BERT}_{\text{BASE}}$	in its $\to$ POS	with rare birds in its with the shipping news before it , an attempt is made to transplant a hollywood star into newfoundland ’ s wild soil - - and the rock once again resists the intrusion .
	LSTM	n ’ t $\to$ NEG	but n ’ t most part he makes sure the salton sea works the way a good noir should , keeping it tight and nasty .
	SBERT	this film $\to$ POS	generic slasher - movie nonsense , this film s not without style .
Movies	$\text{BERT}_{\text{BASE}}$	because he ’ s $\to$ NEG	while because he ’ s laughing at the movie , terrance and phillip cuss repeatedly entertaining the kids .
Movies	LSTM	was supposed to be $\to$ NEG	i was supposed to be when or how this movie will be released in the united states .
	SBERT	’ t seem to $\to$ NEG	the cinematography and general beauty of this part ’ t seem to breathtaking .
MultiRC	$\text{BERT}_{\text{BASE}}$	(? \| \|, of the world trade center) $\to$ FALSE	(what is the flood plain area of land good for if it floods often ? \| \| crops, a floodplain is an area where a thick layer of rich soil is left behind as the floodwater recedes of the world trade center floodplains are usually good places for growing plants .)
MultiRC	LSTM	(? \| \|, al qaeda ’ s) $\to$ FALSE	(in the past $ 5 . 6 million was the allotted amount added , what is the amount they are proposing this year ? \| \| more than $ 20 million , $ 80 . 4 million, but this year al qaeda ’ s , the council is proposing shifting more than $ 20 million in funds earmarked by the mayor for 18 - b lawyers to the legal aid society , which would increase its total funding to $ 80 .)
	SBERT	(? \| \|, , but the algarve) $\to$ FALSE	(what were the initial list of targets ? \| \| capitol , white house, these included the white house , the , but the algarve)
CLIMATE-FEVER	$\text{BERT}_{\text{BASE}}$	(in the, climate change) $\to$ relevant	(it has never been shown that human emissions of carbon dioxide drive in the ., multiple lines of scientific evidence show that climate change is warming .)
	LSTM	(that the, is a) $\to$ irrelevant	(before human burning of fossil fuels triggered that the , the continent ’ s ice was in relative balance, in 2013 , the intergovernmental panel on climate change ( ipcc ) fifth assessment report concluded that ‘ ‘ it is extremely likely that human influence has been the dominant cause of is a - 20th century .)
	SBERT	(’ s, climate change .) $\to$ relevant	(phil jones says no ’ s since 1995 ., climate change .)