This post describes work appearing at CSCW 2014.
Many online activities today involve interacting with other people, and these interactions are often dictated by social norms – unspoken rules that classify socially acceptable behavior. Whether intentional or otherwise, people sometimes break these rules. Behavior that goes against established norms is called deviant behavior, and in many cases is assumed to be negative. In online communities, deviant behavior is commonly believed to be harmful, and is expected to drive users to leave the community. For example, intentionally incorrect responses on a question and answer site may discourage new users from asking questions. Our work looks at patterns of player behavior in an online game. We build a metric to predict a specific kind of deviant behavior, toxicity, and we use this metric to examine whether deviant behavior causes other players to quit.
We look at deviant behavior in the popular online game League of Legends. League of Legends is a competitive multiplayer game where two teams of five players compete to destroy the other team’s base. Each player controls a single character, and every character has unique abilities which, when used skillfully, can help the player’s team overcome an enemy team. If you’ve never played League of Legends, check out this four-minute introduction video from League’s developer, Riot Games.
Deviant behavior in League of Legends is a known problem. Riot has addressed it with several interventions, such as rallying the community behind a common code of conduct and using social psychology techniques to subtly influence player behavior. Problematic deviant behaviors in League of Legends manifest in several ways: A player could verbally abuse other players, quit a match early or help the enemy team win. Riot is specifically concerned with some of these behaviors, which they call “toxicity.” In their talk last Spring, they define toxicity as any behavior that negatively impacts other players’ experiences. You can see more on Riot’s efforts to influence player behavior in their GDC2013 talk.
Using data donated by Duowan (the creators of a popular Chinese add-on for League of Legends) we analyzed 18 million matches across three months. The add-on allows players to give feedback to their teammates. From recorded social and match data, we built a metric called toxicity index. Toxicity index is a ratio between “thumbs up” and “thumbs down” scores, used to identify players with consistently negative feedback. A higher toxicity index means a player was given lots of negative feedback by their peers. Similar to the “report a player” feature built in to League of Legends, thumbs down messages are strongly associated with toxic behavior in the game.
Once we were able to identify toxic players, we found several common correlations with toxicity index:
- Ranked matches are more competitive matches which have a public-facing rating. Players in ranked matches often had a higher toxicity index than players in normal matches.
- Different character roles predicted different levels of toxicity.
- Playing characters with lots of damage-dealing abilities (carries and bruisers) usually predicted players with higher toxicity indexes.
Different character roles predicted different levels of toxicity.
Playing characters with lots of damage-dealing abilities (carries and bruisers) usually predicted players with higher toxicity indexes.
Playing characters with abilities that promote teamwork (supports and tanks) usually predicted players with lower toxicity indexes.
- While we expected more positive feedback, players teaming up with friends received more negative feedback from their non-friend teammates.
What predicts quitting?
We also wanted to look for factors that drove players to quit the game. To do this we separated players by tenure, splitting experienced and inexperienced players. Using a regression, we examined player retention in terms of both finishing their current gaming session and quitting the game for a long time (or for good).
- Newer player’s session length didn’t seem to be affected by playing with toxic teammates, regardless of their own toxicity index.
- Newer players were less likely to quit for a long period when they had a high toxicity index, and were more likely to quit for a while after playing with toxic teammates.
- Surprisingly, experienced players were more likely to continue their session when they were toxic, or when some of their teammates were toxic. However, if the player and their teammates both had high toxicity indexes, the player was likely to end their session.
Both experienced and new players who play with friends are more likely to play another match.
Playing with friends also kept newer players from taking a long break from the game, and playing with friends was the only factor that kept experienced players from quitting for good.
In summary, we found different game modes, character abilities and playing with friends all correlated with different levels of toxic behavior. Additionally, newer players who were toxic were less likely to quit the game, and newer players who weren’t toxic were likely to quit after encountering toxic teammates. Playing with friends predicted players at all levels would keep playing.
Want to learn more? Read our full paper, accepted to be published this February at CSCW 2014.