Countering malicious content moderation evasion in online social networks: Simulation and detection of word camouflage

Content moderation is the process of screening and monitoring user-generated content online. It plays a crucial role in stopping content resulting from unacceptable behaviors such as hate speech, harassment, violence against specific groups, terrorism, racism, xenophobia, homophobia, or misogyny, to mention some few, in Online Social Platforms. These platforms make use of a plethora of tools to detect and manage malicious information; however, malicious actors also improve their skills, developing strategies to surpass these barriers and continuing to spread misleading information. Twisting and camouflaging keywords are among the most widely used techniques to evade platform content moderation systems. In response to this recent ongoing issue This paper presents an innovative approach to address this linguistic trend in social networks through the simulation of different content evasion techniques and a multilingual transformer model for content evasion detection. In this way a multilingual public tool Named “pyleetspeak” is shared with the scientific community, enabling the generation and simulation of content evasion through automatic word camouflage in a customizable way. Additionally a multilingual named-entity recognition (NER) transformer-based model is provided Designed for the recognition and detection of such evasion technique. The developed tool is multilingual Supporting over 20 languages (ar, az, da, de, el, en, es, fi, fr, hu, id, it, kk, nb, ne, nl, pt, ro, ru, sl, sv, tg, tr) and the NER model has been tested in English, Spanish, French, Italian, and German. This multilingual NER model is evaluated in different textual scenarios Detecting different types and mixtures of camouflage techniques Achieving an overall weighted F1 score of 0.8795. This article contributes significantly to countering malicious information by developing multilingual tools to simulate and detect new methods of evasion of content on social networks Making the fight against information disorders more effective •Study on techniques to evade content moderation in multilingual data.•Curated synthetic multilingual dataset and novel word camouflage simulation method.•New techniques for content evasion detection in multilingual data via Transformers NER.•Internal and external validation via AugLy, showing dataset and model utility.