The foundation of popular artificial intelligence image generators hides thousands of images of child sexual abuse, according to a new report that urges companies to take action to fix a harmful flaw in the technology they develop.
These same images have made it easier for AI systems to create realistic and explicit images of fake children and convert social media photos of fully clothed real teenagers into nudes, much to the concern of schools and law enforcement agencies around the world.
Until recently, anti-abuse researchers thought that the only way some uncontrolled AI tools could generate abusive images of children was essentially by combining what they learned from two separate categories of online images – adult pornography and harmless photos of children.
But the Stanford Internet Observatory has found more than 3,200 images of suspected child sexual abuse in the massive AI database LAION, an index of online images and captions used to train leading AI image makers like Stable Diffusion. The Stanford University-based monitoring group worked with the Canadian Center for Child Protection and other anti-abuse charities to identify the illegal material and report the original photo links to law enforcement.
The answer came immediately. On the eve of the release of the Stanford Internet Observatory report on Wednesday, LAION told The Associated Press that it would temporarily remove its datasets.
LAION, which stands for the nonprofit Large-scale Artificial Intelligence Open Network, said in a statement that it “has a zero-tolerance policy toward illegal content and we have taken the utmost caution in deleting the LAION records to ensure that they are sure to republish them.”
While the images represent only a fraction of LAION's index of around 5.8 billion images, according to the Stanford group, they are likely to influence the ability of AI tools to produce harmful results and the past abuse of real victims, that appear several times reinforce.
It's not an easily fixable problem and stems from the fact that many generative AI projects have been “effectively brought to market” and made widely available because the field is so competitive, said David Thiel, chief technologist at the Stanford Internet Observatory, who reported to the wrote the report.
“Doing an entire internet search and using that data set to train models is something that should be more limited to a research operation, and is not something that should have been open-sourced without much more rigorous attention.” Thiel said in an interview.
A prominent LAION user who helped shape the development of the data set is London-based startup Stability AI, maker of the text-to-image models Stable Diffusion. New versions of Stable Diffusion have made it much harder to create malicious content, but an older version introduced last year – which Stability AI said has not been released – is still integrated with other applications and tools and remains “the most popular model to generate”. explicit images,” the Stanford report said.
“We can’t take that back. This model is in the hands of many people on their local computers,” said Lloyd Richardson, director of information technology at the Canadian Center for Child Protection, which runs Canada's hotline for reporting online sexual exploitation.
Stability AI said Wednesday that it only hosts filtered versions of Stable Diffusion and that “since taking over exclusive development of Stable Diffusion, Stability AI has taken proactive steps to reduce the risk of misuse.”
“These filters prevent unsafe content from reaching the models,” the company said in a prepared statement. “By removing this content before it reaches the model, we can help prevent the model from generating unsafe content.”
LAION was the brainchild of a German researcher and teacher, Christoph Schuhmann, who told the AP earlier this year that one of the reasons for making such a large visual database publicly available was to ensure that the future of AI development does not is controlled by a handful of powerful companies.
“It will be much safer and fairer if we can democratize it so that the entire research community and the entire public can benefit from it,” he said.
Much of LAION's data comes from another source, Common Crawl, a data repository that is constantly crawled from the open Internet, but Common Crawl's executive director, Rich Skrenta, said it is “mandatory” for LAION to use the data it collects to scan and filter before using them.
LAION said this week that it has developed “strict filters” to detect and remove illegal content before publishing its datasets and is continuing to work on improving these filters. The Stanford report acknowledges that LAION's developers made some attempts to filter out explicit “minor” content, but may have done a better job if they had consulted with child safety experts earlier.
Many text-to-image generators are derived from the LAION database in some way, although it is not always clear which one they are. OpenAI, maker of DALL-E and ChatGPT, said it does not use LAION and has optimized its models to reject requests for sexual content involving minors.
Google built its text-to-image image model based on a LAION dataset, but decided against releasing it in 2022 after an audit of the database “revealed a wide range of inappropriate content, including pornographic images, racist Insults and harmful social stereotypes.”
Trying to clean up the data retroactively is difficult, so the Stanford Internet Observatory is calling for more drastic measures. One is for anyone who has built training sets based on LAION‐5B—named for the more than 5 billion image-text pairs it contains—to “delete them or work with intermediaries to clean up the material “. Another is to effectively make an older version of Stable Diffusion disappear from all but the darkest corners of the Internet.
“Legitimate platforms may stop offering versions of them for download,” particularly if they are commonly used to generate abusive images and have no protections in place to block them, Thiel said.
As an example, Thiel cited CivitAI, a platform favored by people making AI-generated pornography but which he said lacks safeguards to balance it against creating images of children. The report also calls on AI company Hugging Face, which distributes model training data, to implement better methods for reporting and removing links to offensive material.
Hugging Face said it regularly works with regulators and child safety groups to identify and remove abusive material. CivitAI did not respond to requests for comment submitted to its website.
The Stanford report also questions whether photos of children – even the most innocuous ones – should be fed into AI systems without the consent of their family members because of protections in the federal law protecting children's online privacy.
Rebecca Portnoff, director of data science at the anti-child sexual abuse organization Thorn, said her organization has conducted research showing that the prevalence of AI-generated images among perpetrators is low but increasing.
Developers can mitigate these harms by ensuring that the datasets they use to develop AI models are free of abusive materials. Portnoff said there are also ways to curb harmful uses later, after models are already in circulation.
Technology companies and child safety groups currently assign a “hash” — unique digital signatures — to videos and images to detect and remove child abuse materials. According to Portnoff, the same concept can also be applied to AI models that are misused.
“That’s not happening right now,” she said. “But it’s something that I think can and should be done.”