Evaluating the Performance of ChatGPT for Spam Email Detection
Email continues to be a pivotal and extensively utilized communication medium within professional and commercial domains. Nonetheless, the prevalence of spam emails poses a significant challenge for users, disrupting their daily routines and diminishing productivity. Consequently, accurately identif...
Gespeichert in:
Hauptverfasser: | , , , , |
---|---|
Format: | Artikel |
Sprache: | eng |
Schlagworte: | |
Online-Zugang: | Volltext bestellen |
Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
Zusammenfassung: | Email continues to be a pivotal and extensively utilized communication medium
within professional and commercial domains. Nonetheless, the prevalence of spam
emails poses a significant challenge for users, disrupting their daily routines
and diminishing productivity. Consequently, accurately identifying and
filtering spam based on content has become crucial for cybersecurity. Recent
advancements in natural language processing, particularly with large language
models like ChatGPT, have shown remarkable performance in tasks such as
question answering and text generation. However, its potential in spam
identification remains underexplored. To fill in the gap, this study attempts
to evaluate ChatGPT's capabilities for spam identification in both English and
Chinese email datasets. We employ ChatGPT for spam email detection using
in-context learning, which requires a prompt instruction and a few
demonstrations. We also investigate how the number of demonstrations in the
prompt affects the performance of ChatGPT. For comparison, we also implement
five popular benchmark methods, including naive Bayes, support vector machines
(SVM), logistic regression (LR), feedforward dense neural networks (DNN), and
BERT classifiers. Through extensive experiments, the performance of ChatGPT is
significantly worse than deep supervised learning methods in the large English
dataset, while it presents superior performance on the low-resourced Chinese
dataset. |
---|---|
DOI: | 10.48550/arxiv.2402.15537 |