Publications

My publications - mostly about natural language processing and paraphrase plagiarism research. In total, I've published 6 papers. All papers are openly accessible to anyone. If you like my work, citing is highly appreciated 🤗. You can search publications by title.

2022

How Large Language Models are Transforming Machine-Paraphrased Plagiarism

Jan Philip Wahle, Terry Ruas, Frederic Kirstein, Bela Gipp

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing (EMNLP)

PaperCitation
Paper Summary
  • We present a dataset with machine-paraphrased text from T5 and GPT-3 based on original work from Wikipedia, arXiv, and student theses to train and evaluate machine-paraphrased plagiarism.
  • We explore the human ability to detect plagiarised text through three experiments, focusing on (1) the detection difficulty of paraphrasing methods, (2) the quality of paraphrased examples, and (3) the accuracy of humans in distinguishing between paraphrased and original texts.
  • We empirically test plagiarism detection software (e.g., PlagScan) against machine learning methods and neural language models (autoencoding and autoregressive) in detecting machine-paraphrased plagiarism.
  • We show that paraphrases from GPT-3 provide the most realistic plagiarism cases that both humans and automated detection solutions fail to spot, while the model itself is the best-tested candidate for detecting paraphrases.

Analyzing Multi-Task Learning for Abstractive Text Summarization

Frederic Kirstein, Jan Philip Wahle, Terry Ruas, Bela Gipp

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing (EMNLP)

PaperCitation
Paper Summary
  • We study the influence of multi-task learning by training models on six task families for the English abstractive text summarization task.
  • We evaluate the co-training of different task families using statistical (e.g., ROUGE) and semantic metrics (e.g., BERTScore) for 18 datasets.
  • We compare the influence of three training schemes (i.e., sequential, simultaneous, con- tinual multi-task learning) and two mixing strategies (i.e., proportional, equal).

D3: A Massive Dataset of Scholarly Metadata for Analyzing the State of Computer Science Research

Jan Philip Wahle, Terry Ruas, Saif M. Mohammad, Bela Gipp

Proceedings of The 13th Language Resources and Evaluation Conference (LREC)

PaperCitation
Paper Summary
  • We publish a new open dataset2 of ≈ 6 million English research papers and the source code to retrieve them
  • D3 augments DBLP with metadata extracted from full-text to provide additional features over existing datasets such as paper abstracts and author affiliations (see Table 2 for more details).
  • We provide an exploratory analysis of D3 through eight research questions to illustrate some of our dataset’s main capabilities.
  • Our research questions investigate the volume, content, and citations of papers in D3.

Testing the Generalization of Neural Language Models for COVID-19 Misinformation Detection

Jan Philip Wahle, Nischal Ashok, Terry Ruas, Norman Meuschke, Tirthankar Ghosal & Bela Gipp

Proceedings of the 2022 iConference. Information for a Better World: Shaping the Global Future.

PaperCitation
Paper Summary
  • We apply 15 Transformer models to five COVID-19 misinformation tasks.
  • We compare Transformer models optimized on COVID-19 datasets to state-of-the-art neural language models.
  • We exhaustively apply models to different tasks to test their generalization on unknown sources.
  • Our main finding is that misinformation detection models struggle to generalize over the scope of domains and tasks.

Identifying Machine-Paraphrased Plagiarism

Jan Philip Wahle, Terry Ruas, Tomáš Foltýnek, Norman Meuschke, Bela Gipp

Proceedings of the 2022 iConference. Information for a Better World: Shaping the Global Future.

PaperCitation
Paper Summary
  • We propose two new collections created from research papers on arXiv and graduation theses of “English language learners” (ELL).
  • We explore two paraphrasing tools for generating obfuscated samples: SpinBot and SpinnerChief.
  • We evaluate eight neural language models based on the Transformer architecture for identifying machine paraphrases.

2021

Are Neural Language Models Good Plagiarists? A Benchmark for Neural Paraphrase Detection

Jan Philip Wahle, Terry Ruas, Norman Meuschke, Bela Gipp

Proceedings of the 2021 ACM/IEEE Joint Conference on Digital Libraries (JCDL)

PaperCitation
Paper Summary
  • We provide a large-scale dataset of text paraphrased using Transformer-based language models.
  • We study how word embeddings and three Transformer-based models are used for paraphrasing (BERT, RoBERTa, and Longformer).
  • We perform classification of paraphrased text to underline the difficulty of the task and the dataset’s ability to reflect it.