The Landscape of Toxicity in Reddit: From Users to Conversations to Moderators

Online platforms like Reddit enable users to build communities and converse about diverse topics and interests. However, an increasing number of users publish disturbing posts and comments containing profanity, harassment, and hate speech, otherwise known as toxic content. Such users can change their toxic behavior by participating in multiple communities. Within communities, conversations can show ominous signs of toxicity when they contain causes (i.e., triggers) of toxicity. When toxicity increases, moderators often struggle with managing the safety of conversations in online communities. To address these issues, first, we analyzed toxicity in the form of toxic user behavior. We found that 16.11% of cross-community users publish toxic posts, and 13.28% of cross-community users publish toxic comments. However, results showed that 30.68% of users publishing posts and 81.67% of users publishing comments exhibit changes in their toxicity across different communities, indicating that users adapt their behavior to the communities’ norms. Next, we extracted a set of sentiment shift, topical shift, and context-based features from 991,806 conversation threads. Then, we used them to build a dual embedding biLSTM neural network that achieved an AUC score of 0.789. Our analysis showed that specific triggering keywords, like ‘racist’ and ‘women’, are common across all communities. Lastly, we performed a mixed-method study on a collection of 1,827 responses from Reddit moderators. The survey analysis found specific themes like experience and style, views on toxicity, and how they adhere to community guidelines, which influence the toxicity of moderators and how they handle toxicity. This dissertation presents our approach, which builds on state-of-the-art toxic comment and toxicity trigger detection methods. Lastly, we show our research findings of investigating toxicity across users and moderators on Reddit.