Hate speech targets in COVID-19 related comments on Ukrainian news websites


  • Lidiia Melnyk Friedrich Schiller University Jena




hate speech, FastText, cluster


The research focuses on hate speech in the comments section of Ukrainian news websites. Restricted to solely COVID-19 related comments, it seeks to analyze the development of hate speech rates throughout the pandemic. Using a semi-automated machine-learning-aided approach, the paper identifies hate speech in the comments and defines its main targets. The research shows that a crisis like the COVID-19 pandemic can strengthen existing negative stereotypes and gives rise to new forms of stigmatization against social and ethnic groups.


Download data is not yet available.


Brown, Alexander. 2017. "What is hate speech? Part 1: The myth of hate." Law and Philosophy 36, no. 4: 419-468. https://doi.org/10.1007/s10982-017-9297-1

Cammaerts, Bart. 2009. "Radical Pluralism and Free Speech in Online Public Spaces." International Journal of Cultural Studies 12, no. 6: 555-75. https://doi.org/10.1177/1367877909342479

Chicco, Davide, Matthijs J. Warrens, and Giuseppe Jurman. 2021. "The Matthews Correlation Coefficient (MCC) is More Informative Than Cohen' s Kappa and Brier Score in Binary Classification Assessment." IEEE Access 9: 78368-78381. https://doi.org/10.1109/ACCESS.2021.3084050

Cothenet, Come. 2020. "Short Technical Information About Word2vec, Glove And FastText." Last modified May 25, 2020. https://towardsdatascience.com/short-technical-information-about-word2vec-glove-and-fasttext-d38e4f529ca8.

Cotik, Viviana, Natalia Demand, Franco M. Luque, Paula Miguel, Augustin Moro, Juan Manuel Perez, Pablo Serrati, Joaquin Zajac, and Demian Zayat. 2020. "A study of Hate Speech in Social Media during the COVID-19 outbreak." Paper presented at EMNLP 2020 Workshop NLP-COVID Submission, November 16-20.

Council of Europe. 2015. "On combating hate speech." OCRI general policy recommendation 15. Accessed July 10, 2021. https://rm.coe.int/ecri-general-policy-recommendation-no-15-on-combating-hate-speech/16808b5b01

Dordevic, Jasmina P. 2020. "The sociocognitive dimension of hate speech in readers' comments on Serbian news websites." Discourse, Context and Media 33. https://doi.org/10.1016/j.dcm.2019.100366

Erjavec, Karmen, and Melita Poler Kovačič. 2012. ""You Don' t Understand, This is a New War!" Analysis of Hate Speech in News Web Sites' Comments." Mass Communication and Society 15: 899 - 920. https://doi.org/10.1080/15205436.2011.619679

Erjavec, Karmen. 2014. "Readers of online news comments." Annales. Series historia et sociologia 24, no. 3: 451-462.

Fan, Lizhou, Huizi Yu, and Zhanyuan Yin. 2020. "Stigmatization in social media: Documenting and analyzing hate speech for COVIDâ€19 on Twitter." Proceedings of the Association for Information Science and Technology. Association for Information Science and Technology 57. https://doi.org/10.1002/pra2.313

Gee, Gilbert C., Marguerite J. Ro, and Anne W. Rimoin. 2020. "Seven reasons to care about racism and COVID-19 and seven things to do to stop it." American Journal of Public Health 110: 954-955. https://doi.org/10.2105/AJPH.2020.305712

Gelber, Katharine, and Luke McNamara. 2016. "Evidencing the harms of hate speech." Social Identities 22: 324 - 341. https://doi.org/10.1080/13504630.2015.1128810

Hallgren, Kevin A. 2012. "Computing Inter-Rater Reliability for Observational Data: An Overview and Tutorial." Tutorials in Quantitative Methods for Psychology 8, no. 1: 23-34. https://doi.org/10.20982/tqmp.08.1.p023

Isakova, Tamara. 2016. "Hate speech as the problem of Ukrainian information space." Strategic Priorities 41, no. 4: 90-97.

Jiang, Liangxiao, Shasha Wang, Chaoquin Li, and Lungan Zhang. 2016. "Structure extended multinomial naive Bayes." Information Sciences 329: 346-356. https://doi.org/10.1016/j.ins.2015.09.037

Joulin, Armand, Edouard Grave, Piotr Bojanowski, and Tomas Mikolov. 2017. "Bag of Tricks for Efficient Text Classification." Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics 2: 427-431. https://doi.org/10.18653/v1/E17-2068

Joulin, Armand, Edouard Grave, Piotr Bojanowski, Matthijs Douze, Herve Jegou, and Tomas Mikolov. 2016. "FastText.zip: Compressing text classification models." Paper presented at ICLR 2017. https://arxiv.org/abs/1612.0365.

Kennedy, Brendan, Mohammad Atari, Aida Mostafazadeh Davani, Leigh Yeh, Ali Omrani, Yehsong Kim, Kris Coombs, Shreya Havaldar, Gwenyth Portillo-Wightman, Elaine Gonzalez, Joseph Hoover, Aida Azatian, Alyzeh Hussain, Austin Lara, Gabriel Olmos, Adam Omary, Christina Park, Clarisa Wijaya, Xin Wang, Yong Zhang, and Morteza Dehghani. 2018. "The Gab Hate Corpus: A collection of 27k posts annotated for hate speech." PsyArXiv. July 18. https://doi.org/10.31234/osf.io/hqjxn

Kriazhova, Аnastasiia. 2019. "Методы оценивания семантической близости фраз для классификации текстовых сообщений" [Methods for estimating the semantic proximity of phrases for classification of text messages]. Paper presented at Mezhvuzovskaya nauchno-tekhnicheskaya konferentsiya studentov, aspirantov i molodykh spetsialistov im. YEV Armenskogo, 2019.

Liu, Xia. 2019. "A Big Data Approach to Examining Social Bots on Twitter." Journal of Services Marketing 33, no. 4: 369-79. https://doi.org/10.1108/JSM-02-2018-0049

Maitra, Ishani, and Mary Kate McGowan. 2012. "Introduction and Overview." In Speech and harm: Controversies over free speech, edited by Ishani Maitra and Mary Kate McGowan, 1-24. Oxford University Press on Demand. https://doi.org/10.1093/acprof:oso/9780199236282.003.0001

Mazza, Michele, Stefano Cresci, Marco Avvenuti, Walter Quattrociocchi, and Maurizio Tesconi. 2019. "RTbust." Proceedings of the 10th ACM Conference on Web Science - WebSci '19: 183-192. https://doi.org/10.1145/3292522.3326015

Morhun, Anastasiia. 2016. "Війна Росії проти України: тенденції конфліктного дискурсу в сучасних українських ЗМІ." [The Russian war against Ukraine: Tendencies of the conflict discourse in the current Ukrainian media]. Naukoví zapiski Ínstitutu polítichnikh i yetnonatsíonal' nykh issledovaniy im. ÍF Kurasa NAN Ukraí̈ni Наукові записки Інституту політичних і етнонаціональних досліджень ім. ІФ Кураса НАН України 3-4: 352-362.

Pratiwi, Nur Indah, Indra Budi, and Ika Alfina. 2018. "Hate Speech Detection on Indonesian Instagram Comments Using FastText Approach." Paper presented at 2018 International Conference on Advanced Computer Science and Information Systems (ICACSIS). https://doi.org/10.1109/ICACSIS.2018.8618182

Postic, Irena. 2018. "Hate speech and propaganda in Russian-Ukrainian conflict." PhD diss., Universität Wien.

Qi, Peng, Yuhao Zhang, Yuhui Zhang, Jason Bolton, and Christopher D.Manning. 2020. "Stanza: A Python Natural Language Processing Toolkit for Many Human Languages." Paper presented at 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations. https://doi.org/10.18653/v1/2020.acl-demos.14

Ramos-Oliveira, Diana, and Andrzej Pankalla. 2019. "Negative stereotypes: an analysis of Social Cognition in different ethnic groups. Social cognition of stereotypes." Psicogente 22: 1-15. https://doi.org/10.17081/psico.22.42.3501

Singh, Vivek Kumar, Nisha Tiwari, and Shekhar Garg. 2011. "Document Clustering Using K-Means, Heuristic K-Means and Fuzzy C-Means." Paper presented at 2011 International Conference on Computational Intelligence and Communication Networks. https://doi.org/10.1109/CICN.2011.62

Sivakumar, Soubraylu, Lakshmi Sarvani Videla, T Rajesh Kumar, J. Nagaraj, Shilpa Itnal, and D. Haritha. 2020."Review on Word2Vec Word Embedding Neural Net." Paper presented at 2020 International Conference on Smart Electronics and Communication (ICOSEC): 282-290. https://doi.org/10.1109/ICOSEC49089.2020.9215319

Skowronski, Jason. 2019 "Identifying Trolls and Bots on Reddit with Machine Learning (Part 2)." Last modified July 30, 2019. https://towardsdatascience.com/identifying-trolls-and-bots-on-reddit-with-machine-learning-709da5970af1.

Sopilko, Iryna. 2016. "Тролі, боти та бот-мережі як загрози розвитку інформаційного суспільства" [Trolls, bots and bot-network as danger to the development of information society]. Naukovi pratsi Natsionalʹnoho aviatsiynoho universytetu. Серія: Юридичний вісник «Повітряне і космічне право» 4, no. 41: 86-91.

Stechemesser, Annika, Leonie Wenz, and Anders Levermann. 2020. "Corona crisis fuels racially profiled hate in social media networks." EClinicalMedicine 23. doi:10.1016/j.eclinm.2020.100372. https://doi.org/10.1016/j.eclinm.2020.100372

Wang, Qingning. 2018. "Linguistic violence and online political communications in China: The example of é¸¡çš„å± (Ji De Pi) as an ironic spoof of gross domestic product in online debates around environmental issues." Global Media and China 3: 18-31. https://doi.org/10.1177/2059436418775532

Yuan, Chunhui, and Haitao Yang. 2019. "Research on K-value selection method of K-means clustering algorithm." J 2.2: 226-235. https://doi.org/10.3390/j2020016

Zandifar, Atefeh, Rahim Badrfam, Nami Mohammadian Khonsari, Marzieh Assareh, Hossein Karim, Mehdi Azimzadeh, Mohammad Noori Sepehr, Ramin Tajbakhsh, Fatemeh Rahimi, Nima Ghanipour, Arash Agoushi, Saeed Hassani Gelsefid, Fateme Etemadi, and Mostafa Qorbani. 2020. "COVID â€19 and medical staff' s mental health in educational hospitals in Alborz province, Iran." Psychiatry and Clinical Neurosciences 74, no. 9: 499-501. https://doi.org/10.1111/pcn.13098

Zbyr, Iryna. 2015. "Contemporary Language Issues in Ukraine: Bilingualism or Russification." Journal of Arts and Humanities 4: 45-54.

Zhdanova, Mariia, and Dariya Orlova. 2017. "Computational Propaganda in Ukraine: Caught between external threats and internal challenges." Computational Propaganda Research Project, University of Oxford. Working Paper. http://comprop.oii.ox.ac.uk/wp-content/uploads/sites/89/2017/06/Comprop-Ukraine.pdf.

Ziems, Caleb, Bing He, Sandeep Soni, Naren Ramakrishnan, Diyi Yang, Srijan Kumar, Georgia Institute of Technology, and Virginia Tech. 2020."Racism is a Virus: Anti-Asian Hate and Counterhate in Social Media during the COVID-19 Crisis." https://arxiv.org/abs/2005.12423v2.

Verkhovna Rada of Ukraine. 2014. "Про суспільне телебачення і радіомовлення"[On television and broadcasting]. Vidomosti Verchovnoyi Rady Ukrayiny Відомості Верχовної Ради України 27: 904. https://zakon.rada.gov.ua/laws/show/1227-18#Text

Censor.net. n.d. "Реклама На Портале [Advertising on the Website]." Accessed July 22, 2021. https://censor.net/ru/reklama.

UMH. n.d. "Korrespondent.net." UMH MEDIA KIT. Accessed July 22, 2021. https://umhdigitalspecials.com/korrespondent.