Shoumik Saha / Soheil Feizi

LINK: https://arxiv.org/pdf/2502.15666

Abstract
The growing use of large language mod-
els (LLMs) for text generation has led to
widespread concerns about AI-generated con-
tent detection. However, an overlooked chal-
lenge is AI-polished text, where human-written
content undergoes subtle refinements using AI
tools. This raises a critical question: should
minimally polished text be classified as AI-
generated? Such classification can lead to
false plagiarism accusations and misleading
claims about AI prevalence in online content.
In this study, we systematically evaluate twelve
state-of-the-art AI-text detectors using our AI-
Polished-Text Evaluation (APT-Eval) dataset,
which contains 14.7K samples refined at vary-
ing AI-involvement levels. Our findings reveal
that detectors frequently flag even minimally
polished text as AI-generated, struggle to differ-
entiate between degrees of AI involvement, and
exhibit biases against older and smaller models.
These limitations highlight the urgent need for
more nuanced detection methodologies.