Multimodal Fake News Detection: Concepts, Challenges, Detection Strategies and Development of a Detector Model “Turbofake”

Ajay Kumar (emlyon business school), Pei-yu Chen (Arizona State University, USA), Ram Gopal (Warwick Business School, UK), Emanuele Borgonovo (SDA Bocconi School of Management, Italy), Margherita Pagani (emlyon business school)

In recent years, fake news has become a global phenomenon due to its explosive growth and ability to leverage multimedia content to manipulate user opinions. Fake news is created by manipulating images, text, audio, and videos, particularly on social media, and the proliferation of such disinformation can trigger detrimental societal effects. However, a single modality is not sufficient to address such a complex problem so we introduce a novel fake news detector model “TurboFake” in which we (1) identify several textual and visual features that are associated with fake or credible news; specifically, we extract features from article titles, contents and top-images; (2) investigate the role of all multimodal features (content, emotions and manipulation-based) and combine the cumulative effects that represent the behaviour of fake news propagators; and (3) develop a model to detect disinformation on benchmark multimodal datasets consisting of text and images. We conduct experiments on three fake news datasets collected from: Weibo, r/Fakeddit, Twitter and our results show that on average, our model outperforms existing methods with single-modality by margins as large as ∼8% in accuracy and∼6% in F1 scores.