Artificial intelligence—friend or foe in fake news campaigns
DOI:
https://doi.org/10.18559/ebr.2023.2.736Keywords:
artificial intelligence, fact‐checking, fake news, large language modelsAbstract
In this paper the impact of large language models (LLM) on the fake news phenomenon is analysed. On the one hand decent text‐generation capabilities can be misused for mass fake news production. On the other, LLMs trained on huge volumes of text have already accumulated information on many facts thus one may assume they could be used for fact‐checking. Experiments were designed and conducted to verify how much LLM responses are aligned with actual fact‐checking verdicts. The research methodology consists of an experimental dataset preparation and a protocol for interacting with ChatGPT, currently the most sophisticated LLM. A research corpus was explicitly composed for the purpose of this work consisting of several thousand claims randomly selected from claim reviews published by fact‐checkers. Findings include: it is difficult to align the responses of ChatGPT with explanations provided by fact‐checkers; prompts have significant impact on the bias of responses. ChatGPT at the current state can be used as a support in fact‐checking but cannot verify claims directly.
Downloads
References
Agresti, S. Hashemian, S. A., & Carman, M. J. (2022). PoliMi-FlatEarthers at CheckThat! 2022: GPT-3 applied to claim detection. In. G. Faggioli, N. Ferro, A. Harbury &M. Potthast (Eds.), Proceedings of the working notes of CLEF 2022—Conference and labs of the evaluation forum. Bologna, Italy. CEUR Workshop Proceedings, 3180, pp. 422–427. https://ceur-ws.org/Vol-3180/paper-31.pdf
View in Google Scholar
Alkaissi, H., & McFarlane, S. I. (2023). Artificial hallucinations in ChatGPT: Implications in scientific writing. Cureus, 15(2), e35179. https://doi.org/10.7759/cureus.35179 DOI: https://doi.org/10.7759/cureus.35179
View in Google Scholar
Bahdanau, D., Cho, K., & Bengio, Y. (2014). Neural machine translation by jointly learning to align and translate. https://doi.org/10.48550/arXiv.1409.0473
View in Google Scholar
Bang, Y., Cahyawijaya, S., Lee, N., Dai, W., Su, D., Wilie, B., Lovenia, H., Ji, Z., Yu, T., Chung, W., Do, Q. V., Xu, Y., & Fung, P. (2023). A multitask, multilingual, multimodal evaluation of ChatGPT on reasoning, hallucination, and interactivity. https://doi.org/10.48550/arXiv.2302.04023
View in Google Scholar
Bouie, J. (2023, March 11). Disinformation is not the real problem with democracy. The New York Times.
View in Google Scholar
Buchholz, K. (2023, January 24). ChatGPT sprints to one million users. Statista. https://www.statista.com/chart/29174/time-to-one-million-users/
View in Google Scholar
Candelon, F., di Carlo, R.C., De Bondt, M.,& Evgeniou, T. (2021, September-October). AI regulation is coming. Harvard Business Review. https://hbr.org/2021/09/ai-regulation-is-coming
View in Google Scholar
Corfield, G. (2023, February 8). $120bn wiped off google after bard AI chatbot gives wrong answer. https://www.telegraph.co.uk/technology/2023/02/08/googlesbard-ai-chatbot-gives-wrong-answer-launch-event/
View in Google Scholar
Dale, R. (2021). GPT‐3: What’s it good for? Natural Language Engineering, 27(1), 113–118. https://doi.org/10.1017/S1351324920000601 DOI: https://doi.org/10.1017/S1351324920000601
View in Google Scholar
Devlin, J., Chang, M. W., Lee, K., & Toutanova, K. (2019). BERT: Pre‐training of deep bidirectional transformers for language understanding. https://doi.org/10.48550/arXiv.1810.04805
View in Google Scholar
Dwivedi, Y. K., Kshetri, N., Hughes, L., Slade, E. L., Jeyaraj, A., Kar, A. K., Baabdullah, A. M., Koohang, A., Raghavan, V., Ahuja, M., Albanna, H., Albashrawi M., Al-Busaidi, A., Balakrishman, J., Barlette, Y., Basu, S., Bose, I., Brooks, L., Buhalis, D., ..., Carter, L. (2023). “So what if ChatGPT wrote it?” Multidisciplinary perspectives on opportunities, challenges and implications of generative conversational AI for research, practice and policy. International Journal of Information Management, 71, 102642. DOI: https://doi.org/10.1016/j.ijinfomgt.2023.102642
View in Google Scholar
Faggioli, G., Ferro, N., Hanbury, A., Potthast, M. (2022). Proceedings of the working notes of CLEF 2022—Conference and labs of the evaluation forum. Bologna, Italy. CEUR Workshop Proceedings, 3180. https://ceur-ws.org/Vol-3180/
View in Google Scholar
Floridi, L., & Chiriatti, M. (2020). GPT‐3: Its nature, scope, limits, and consequences. Minds and Machines, 30, 681–694. DOI: https://doi.org/10.1007/s11023-020-09548-1
View in Google Scholar
Frieder, S., Pinchetti, L., Griffiths, R.‐R., Salvatori, T., Lukasiewicz, T., Petersen, P. C., Chevalier, A., & Berner, J. (2023). Mathematical capabilities of ChatGPT. https://doi.org/10.48550/arXiv.2301.13867
View in Google Scholar
George, A. S., & George, A. H. (2023). A review of ChatGPT AI’s impact on several business sectors. Partners Universal International Innovation Journal, 1(1), 9–23. https://doi.org/10.5281/zenodo.7644359
View in Google Scholar
Gibbs, S. (2017, July 17). Elon Musk: Regulate AI to combat ‘existential threat’ before it’s too late. The Guardian. https://www.theguardian.com/technology/2017/jul/17/elon-musk-regulation-ai-combat-existential-threat-tesla-spacex-ceo
View in Google Scholar
Goldstein, J. A., Sastry, G., Musser, M., DiResta, R., Gentzel, M., & Sedova, K. (2023). Generative language models and automated influence operations: Emerging threats and potential mitigations. https://doi.org/10.48550/arXiv.2301.04246
View in Google Scholar
Haleem, A., Javaid, M., & Singh, R. P. (2022). An era of ChatGPT as a significant futuristic support tool: A study on features, abilities, and challenges. BenchCouncil Transactions on Benchmarks, Standards and Evaluations, 2(4), 100089. DOI: https://doi.org/10.1016/j.tbench.2023.100089
View in Google Scholar
Hosseini, M., Gao, C. A., Liebovitz, D. M., Carvalho, A. M., Ahmad, F. S., Luo, Y., MacDonald, N., Holmes, K. L., & Kho, A. (2023, April 3). An exploratory survey about using ChatGPT in education, healthcare, and research. medRxiv, 3. DOI: https://doi.org/10.1101/2023.03.31.23287979
View in Google Scholar
Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y., Madotto, A., & Fung, P. (2022). Survey of hallucination in natural language generation. https://doi.org/10.48550/arXiv.2202.03629 DOI: https://doi.org/10.1145/3571730
View in Google Scholar
King, M. R., ChatGPT (2023). A conversation on artificial intelligence, chatbots, and plagiarism in higher education. Cellular and Molecular Bioengineering, 16(1), 1–2. DOI: https://doi.org/10.1007/s12195-022-00754-8
View in Google Scholar
Kirmani, A. R. (2022). Artificial intelligence—enabled science poetry. ACS Energy Letters, 8, 574– 576. DOI: https://doi.org/10.1021/acsenergylett.2c02758
View in Google Scholar
Launchbury, J. (2016, December 6). A DARPA perspective on artificial intelligence. DARPA. https://www.darpa.mil/attachments/AIFull.pdf
View in Google Scholar
LMSYS. (2023, May 25). Chatbot arena leaderboard updates. https://lmsys.org/blog/2023-05-25-leaderboard/
View in Google Scholar
Lopez‐Lira, A., & Tang, Y. (2023). Can ChatGPT forecast stock price movements? Return predictability and large language models. https://doi.org/10.48550/arXiv.2304.07619 DOI: https://doi.org/10.2139/ssrn.4412788
View in Google Scholar
Lund, B. D., & Wang, T. (2023). Chatting about ChatGPT: How may AI and GPT impact academia and libraries? Library Hi Tech News. DOI: https://doi.org/10.2139/ssrn.4333415
View in Google Scholar
Malone, T. W. (2018). Superminds: The surprising power of people and computers thinking together. Little, Brown Spark.
View in Google Scholar
Mayor, T. (2019). Ethics and automation: What to do when workers are displaced. MIT Management Sloan School. https://mitsloan.mit.edu/ideas-made-to-matter/ethics-and-automation-what-to-do-when-workers-are-displaced
View in Google Scholar
McGee, R. W. (2023, April 8). Using artificial intelligence (AI) to compose a musical score for a taekwondo tournament routine: A ChatGPT experiment. Working Paper. https://doi.org/10.13140/RG.2.2.11235.22569
View in Google Scholar
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013). Efficient estimation of word representations in vector space. https://doi.org/10.48550/arXiv.1301.3781
View in Google Scholar
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013). Distributed representations of words and phrases and their compositionality. https://doi.org/10.48550/arXiv.1310.4546
View in Google Scholar
Motoki, F., Pinho Neto, V., & Rodrigues, V. (2023). More human than human: Measuring ChatGPT political bias. https://doi.org/10.2139/ssrn.4372349 DOI: https://doi.org/10.2139/ssrn.4372349
View in Google Scholar
OpenAI & Pilipiszyn, A. (2021, March 25). GPT‐3 powers the next generation of apps. https://openai.com/blog/gpt-3-apps
View in Google Scholar
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C. L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., Schulman, J., Hilton, J., Kelton, F., Miller, L., Simens, M., Askell, A., & Welinder, P. (2022). Training language models to follow instructions with human feedback. arXiv preprint arXiv:2203.02155.
View in Google Scholar
Patel, S. B., & Lam, K. (2023). ChatGPT: The future of discharge summaries? The Lancet Digital Health, 5(3), e107–e108. DOI: https://doi.org/10.1016/S2589-7500(23)00021-3
View in Google Scholar
Paul, J., Ueno, A., & Dennis, C. (2023). ChatGPT and consumers: Benefits, pitfalls and future research agenda. International Journal of Consumer Studies, 47( 4), 1213–1225. https://doi.org/10.1111/ijcs.12928 DOI: https://doi.org/10.1111/ijcs.12928
View in Google Scholar
Pennington, J., Socher, R., & Manning, C. D. (2014). Glove: Global vectors for word representation. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), 1532–1543. DOI: https://doi.org/10.3115/v1/D14-1162
View in Google Scholar
Rivas, P., & Zhao, L. (2023). Marketing with ChatGPT: Navigating the ethical terrain of GPT‐based chatbot technology. AI, 4(2), 375–384. https://doi.org/10.3390/ai4020019 DOI: https://doi.org/10.3390/ai4020019
View in Google Scholar
Romero, A. (2021, June 21). Understanding GPT‐3 in 5 minutes. https://towardsdatascience.com/understanding-gpt-3-in-5-minutes-7fe35c3a1e52
View in Google Scholar
Rudolph, J., Tan, S., & Tan, S. (2023, January 24). ChatGPT: Bullshit spewer or the end of traditional assessments in higher education? Journal of Applied Learning and Teaching, 6(1). https://doi.org/10.37074/jalt.2023.6.1.9 DOI: https://doi.org/10.37074/jalt.2023.6.1.9
View in Google Scholar
Shen, Y., Heacock, L., Elias, J., Hentel, K. D., Reig, B., Shih, G., & Moy, L. (2023). ChatGPT and other large language models are double‐edged swords. Radiology, 307(2). https://doi.org/10.1148/radiol.230163 DOI: https://doi.org/10.1148/radiol.230163
View in Google Scholar
Thurzo, A., Strunga, M., Urban, R., Surovková, J., & Afrashtehfar, K. I. (2023). Impact of artificial intelligence on dental education: A review and guide for curriculum update. Education Sciences, 13(2), 150. DOI: https://doi.org/10.3390/educsci13020150
View in Google Scholar
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017). Attention is all you need. https://doi.org/10.48550/arXiv.1706.03762
View in Google Scholar
Weizenbaum, J. (1966). Eliza—a computer program for the study of natural language communication between man and machine. Communications of the ACM, 9(1), 36–45. https://doi.org/10.1145/365153.365168 DOI: https://doi.org/10.1145/365153.365168
View in Google Scholar
Westerlund, M. (2019, November). The emergence of deepfake technology: A review. Technology Innovation Management Review, 9(11), 39–52. https://doi.org/10.22215/timreview/1282 DOI: https://doi.org/10.22215/timreview/1282
View in Google Scholar
Yang, Z., Li, L., Wang, J., Lin, K., Azarnasab, E., Ahmed, F., Liu, Z., Liu, C., Zeng, M., & Wang, L. (2023). MM‐react: Prompting ChatGPT for multimodal reasoning and action. https://doi.org/10.48550/arXiv.2303.11381
View in Google Scholar
Downloads
Published
Issue
Section
License
Copyright (c) 2023 Krzysztof Węcel, Marcin Sawiński
This work is licensed under a Creative Commons Attribution 4.0 International License.
