Roblox, Discord, OpenAI and Google found new child safety group

Roblox, Discord, OpenAI and Google are launching a nonprofit organization called ROOST, or Robust Open Online Safety Tools, which hopes “to build scalable, interoperable safety infrastructure suited for the AI era.”

The organization plans on providing free, open-source safety tools to public and private organizations to use on their own platforms, with a special focus on child safety to start. The press release announcing ROOST specifically calls out plans to offer “tools to detect, review, and report child sexual abuse material (CSAM).” Partner companies are providing funding for these tools, and the technical expertise to build them, too.

The operating theory of ROOST is that access to generative AI is rapidly changing the online landscape, making the need for “reliable and accessible safety infrastructure” all the more urgent. And rather than expect a smaller company or organization to create their own safety tools from scratch, ROOST wants to provide them, free of charge.

Child online safety has been the issue du jour since the Children and Teen’s Online Privacy Protection Act (COPPA) and Kids Online Safety Act (KOSA) started making their way through Congress, even though both failed to pass in the House. At least some of the companies involved in ROOST, specifically Google and OpenAI, have also already pledged to stop AI tools from being used to generate CSAM.

The child safety issue is even more pressing for Roblox. As of 2020, two-thirds of all US children between nine and 12 play Roblox, and the platform has historically struggled to address child safety. Bloomberg Businessweek reported that the company had a “pedophile problem” in 2024, which prompted multiple policy changes and new restrictions around children’s DMs. ROOST won’t make all of these problems go away, but should make dealing with them easier for any other organization or company that finds itself in Roblox’s position.