Harmful Design in User-Generated Games and its Ethical and Governance Challenges: An Investigation of Design Co-Ideation of Game Creators on Roblox

An increasing number of game platforms, such as Roblox, enable game creators to develop user-generated games (UGGs). Yet, these platforms often come under scrutiny for hosting UGGs that contain harmful content, ranging from sexually explicit material to Nazi-themed roleplay. Limited attention has been paid to how harmful UGGs are ideated by game creators. To address this question, we studied an online Roblox creator community, where Roblox creators collectively engage in design ideation to brainstorm design ideas for UGGs. Through an inductive thematic analysis, we found three primary ways where Roblox creators' design ideation becomes risky, including how Roblox creators generate risky game design ideas, navigate through policy boundaries to develop these ideas, and share strategies of bypassing moderation. Based on our findings, we discuss ethical and governance challenges facing user-generated games. We propose design implications to support game creators in developing ethical game design ideas and safe game designs.