AI-generated child sexual abuse images on internet reaching “tipping point”
AI used to create more explicit versions of existing child sexual abuse imagery, says IWF
The Internet Watch Foundation (IWF) has reported a significant increase in the amount of illegal AI-generated child sexual abuse imagery on the open internet.
The safety watchdog said child sexual abuse images generated by AI tools is becoming more prevalent on the open web and reaching a "tipping point". This content is so realistic that it can be nearly indistinguishable from real images and videos of child abuse.
"I find it really chilling as it feels like we are at a tipping point," said Jeff, a senior analyst at the IWF who uses a pseudonym for privacy.
In the past six months alone, the IWF has seen a 6% increase in AI-generated child abuse images, a significant jump from the previous year. Almost all of this content (99%) was found on the open internet, not hidden on the dark web.
The IWF says it has found that AI is being used to create more explicit versions of existing child sexual abuse imagery, including images of real victims. Perpetrators are also using AI to manipulate adult pornography videos to create deepfakes depicting child sexual abuse.
Survivors of child sexual abuse have shared with the IWF how traumatic it is to see their images used in AI-generated content.
The IWF has acted against 74 reports of AI-generated child sexual abuse material (CSAM) in the past six months. This includes deepfake videos, images of celebrities depicted as children in sexual abuse scenarios, and AI-generated images of clothed children being "nudified."
Much of this harmful content is hosted on servers in Russia (36%) and the United States (22%), with Japan and the Netherlands following at 11% and 8% respectively.
The IWF is working with the tech industry to block and remove these webpages from public access.
Web addresses containing AI-generated child sexual abuse images are added to the IWF's URL list, which is shared with the tech industry to block access to those sites and prevent users from viewing the content.
The AI-generated images are also hashed and marked as AI on a Hash List containing over two million images, which can assist law enforcement in their investigations.
"AI-generated child sexual abuse material causes horrific harm, not only to those who might see it but to those survivors who are repeatedly victimised every time images and videos of their abuse are mercilessly exploited for the twisted enjoyment of predators online," said Derek Ray-Hill, the IWF's interim chief executive.
Assistant Chief Constable Becky Riggs, Child Protection and Abuse Investigation Lead at the National Police Chiefs' Council, said the scale of online child sexual abuse and imagery is alarming, and that the department is aware that the growing use of AI to generate abusive images presents a very real threat to children.
Meanwhile, Instagram has introduced a range of new features aimed at tackling the rising issue of sextortion, where individuals are deceived into sending intimate images that criminals later use to threaten exposure.
One key feature is an automatic blurring mechanism that obscures any nude images sent via direct messages. Users who receive a blurred image can decide whether to view it and will also be reminded of their options to block the sender and report the conversation to Instagram.
Additionally, Instagram will take measures to hide follower and following lists from potential sextortion scammers, who often threaten to send intimate images to those accounts.