AI-generated text: Detection methods and countermeasures

Video Description

Try Voice Writer - speak your thoughts and let AI handle the grammar: https://voicewriter.io Lots of people use ChatGPT and other LLMs for writing, sometimes for better or worse. But is it possible to detect this? Many have claimed ways to detect AI-generated content, such as classifiers, zero-shot methods, and watermarking. But how do these methods hold up against adversarial attacks, like paraphrasing models? Find out in this video! 0:00 - Introduction 1:07 - Measuring ChatGPT usage in corpus 2:54 - Text classification approach 5:25 - Zero-shot detection 8:08 - Watermarking LLMs 10:10 - Paraphrasing attacks 11:54 - Is AI writing detection possible? Survey papers on AI writing detection and countermeasures Ghosal, Soumya Suvra, Souradip Chakraborty, Jonas Geiping, Furong Huang, Dinesh Manocha, and Amrit Singh Bedi. “Towards Possibilities & Impossibilities of AI-Generated Text Detection: A Survey,” n.d. Fraser, Kathleen C., Hillary Dawkins, and Svetlana Kiritchenko. “Detecting AI-Generated Text: Factors Influencing Detectability with Current Methods.” arXiv, June 21, 2024. http://arxiv.org/abs/2406.15583. References for papers mentioned in this video Casal, J. Elliott, and Matt Kessler. “Can Linguists Distinguish between ChatGPT/AI and Human Writing?: A Study of Research Ethics and Academic Publishing.” *Research Methods in Applied Linguistics* 2, no. 3 (December 2023): 100068. https://doi.org/10.1016/j.rmal.2023.100068. Sadasivan, Vinu Sankar, Aounon Kumar, Sriram Balasubramanian, Wenxiao Wang, and Soheil Feizi. “Can AI-Generated Text Be Reliably Detected?” arXiv, February 19, 2024. http://arxiv.org/abs/2303.11156. Guo, Biyang, Xin Zhang, Ziyuan Wang, Minqi Jiang, Jinran Nie, Yuxuan Ding, Jianwei Yue, and Yupeng Wu. “How Close Is ChatGPT to Human Experts? Comparison Corpus, Evaluation, and Detection.” arXiv, January 18, 2023. http://arxiv.org/abs/2301.07597. Mitchell, Eric, Yoonho Lee, Alexander Khazatsky, Christopher D. Manning, and Chelsea Finn. “DetectGPT: Zero-Shot Machine-Generated Text Detection Using Probability Curvature.” arXiv, July 23, 2023. http://arxiv.org/abs/2301.11305. Kirchenbauer, John, Jonas Geiping, Yuxin Wen, Jonathan Katz, Ian Miers, and Tom Goldstein. “A Watermark for Large Language Models.” arXiv, January 27, 2023. http://arxiv.org/abs/2301.10226. Kobak, Dmitry, Rita González Márquez, Emőke-Ágnes Horvát, and Jan Lause. “Delving into ChatGPT Usage in Academic Writing through Excess Vocabulary.” arXiv, June 11, 2024. http://arxiv.org/abs/2406.07016.