The Children’s Advertising Review Unit recently settled with KidGeni – a generative art platform intended for children- for allegedly violating both CARU’s guidelines and COPPA. According to CARU, which is a self-regulatory organization that audits the privacy practices of companies in the child space, KidGeni collected personal information without first getting parental consent. CARU began its investigation in the company’s functionality in August 2023. As part of its investigation, it reached out to the company to clarify how the site obtained prior parental consent for its children’s platform as required under both COPPA and CARU’s guidelines.

CARU concluded that COPPA and the CARU guidelines applied because KidGeni was directed to children under 13. In reaching this conclusion, it looked at the site’s visual content and child-oriented activities, as well as representations made by the company. CARU then assessed whether KidGeni collected personal information. It found in many situations that KidGeni did so, including when children (1) signed up for an account, (2) signed up for newsletters, (3) inputted prompts into the AI tool without an account, (4) shared information with friends or on social media. Personal information was also collected, CARU noted, through third party tracking tools on the site. Personal information collected in these instances included name, email addresses, and in the case of free-form submissions (inputs into the AI tool, for example) photos and stories that might contain personal information.

Because KidGeni was directed to children and collected personal information, CARU argued that the company had an obligation to follow both COPPA and CARU’s guidelines. That included providing an accurate privacy policy, getting parental consent, and having a process in place for parents to limit how their children’s information was used. As for the first, the site originally did not have a privacy policy, as required under COPPA and CARU guidelines. After receiving CARU’s inquiry, KidGeni posted what CARU alleged was an “incomplete and insufficient” privacy policy, as among other things, the privacy policy failed to disclose that KidGeni collected and shared personal information with third parties, like open-source AI training technologies. As for the second, KidGeni did not as noted above get prior parental consent. It also, CARU found, did not have parental control processes in place.

Although indicating that it did not agree with CARU, KidGeni agreed to take corrective action. That included getting parental consent before using user data to train AI models. Companies who do not cooperate with CARU are referred by that self-regulatory body to the Federal Trade Commission, which has authority to enforce COPPA.

Putting It Into Practice: This case is one of the first brought against an operator of AI tools directed to children. It serves as a reminder for companies creating these platforms to think about child-specific privacy requirements, especially when child-submitted information might be used by an AI model. Items to consider include providing adequate to notice about information collected and getting parental consent before collecting personal information from children online.