OpenAI released draft documentation on Wednesday outlining how it wants ChatGPT and its other AI technology to behave. Part of the long one Model specification document announces that the company is exploring a jump into porn and other explicit content.
OpenAIs usage policy currently prohibits sexually explicit or even suggestive material, but a “comment” note on a section of the Model Specification regarding that rule says the company is considering how to allow such content.
“We are exploring whether we can responsibly provide the ability to generate NSFW content in age-appropriate contexts via the API and ChatGPT,” the note says, using an informal term for content that is deemed “not safe for work” is considered. “We look forward to better understanding user and societal expectations for model behavior in this area.”
The Model Spec document says that NSFW content “may include erotica, extreme gore, defamation and unsolicited profanity.” It's unclear whether OpenAI's research into how to responsibly create NSFW content envisions only minor relaxation of usage policies, for example to allow the generation of erotic text, or, more broadly, to allow descriptions or images of to allow violence.
In response to questions from WIRED, OpenAI spokesperson Grace McGuire said the Model Spec was an effort to “bring more transparency to the development process and gain a cross-section of perspectives and feedback from the public, policymakers and other stakeholders.” She declined to share details about what OpenAI's research into explicit content generation entails or what feedback the company has received on the idea.
Earlier this year, OpenAI's Chief Technology Officer said, Mira Muratitold The Wall Street Journal that she was “unsure' if the company were to allow images of nudity to be created using Sora, the company's video generation tool, in the future.
AI-generated pornography has become fast one of the largest and most disturbing applications of the type generative AI technology OpenAI is a pioneer. So-called deepfake porn – explicit images or videos created with AI tools that depict real people without their consent – has become a common tool for harassment of women and girls. In March, WIRED reported on what appears to be first US minors arrested for distributing AI-generated nude photos without consentafter Florida police charged two teenage boys with taking images of fellow high school students.
“Intimate privacy violations, including deepfake sex videos and other non-consensual synthesized intimate images, are widespread and deeply harmful,” said Danielle Keats Citron, a professor at the University of Virginia School of Law who has studied the problem. “We now have clear empirical support showing that such abuse costs specific individuals critical opportunities, including working, speaking and being physically safe.”
Citron calls OpenAI's potential embrace of explicit AI content “alarming.”
Because OpenAI's usage policy prohibits impersonation without consent, explicit, non-consensual images would remain prohibited even if the company allowed creators to generate NSFW material. But it remains to be seen whether the company can effectively moderate explicit generation to prevent bad actors from using the tools. Microsoft has made changes to one of its generative AI tools after 404 Media reported that it had been used to create explicit images of Taylor Swift that were distributed on the social platform X.
Additional reporting by Reece Rogers