“When we look at the abuse of intimate images, the majority of the tools and weaponized uses come from the open source space,” Ajder said. But they often start with well-intentioned developers, he said. “Someone creates something they think is interesting or cool and someone with bad intentions realizes its malicious potential and turns it into a weapon.”
Some, like the repository disabled in August, have communities purpose-built around them for explicit uses. The model positions itself as a tool for deepfake pornography, Ajder claims, becoming a “channel” of abuse, primarily targeting women.
More videos have been uploaded to a porn streaming site by an account that noted AI models downloaded from GitHub featuring the faces of popular phishing targets, celebrities Emma Watson, Taylor Swift, and Anya Taylor-Joy, as well as less famous women other but very realistic, classified as sexual situations.
The creators freely described the tools they used, including two tools that have been removed by GitHub but whose code still exists in other existing repositories.
Deepfake snoopers congregate online in many places, including in secret community forums on Discord and explicitly on Reddit, adding to efforts to stop deepfakes. One Redditor offered their services using the repository's software on September 29. “Can someone do it for my cousin,” asked another.
Torrents of the main repository banned by GitHub in August are also available in other corners of the web, showing how difficult it is to police Open source deepfake software on the board. Other deepfake porn tools, such as the DeepNude app, have suffered similarly taken down before new versions appear.
Elizabeth Seger, director of digital policy at the UK cross-party think tank Demos, said: “There are so many models, so many different branches within the models, so many different versions . It's very difficult to find them all.” “Once a model is publicly available as open source for download, there is no way to restore it publicly,” she added.
A deepfake porn creator with 13 doctored porn videos of female stars has documented a prominent GitHub repository marketed as a “NSFW” version of another project promoting Use responsibly and explicitly request users not to use it for nude photography. “Learn all available Face Swap AI from GitHUB, without using online services,” their profile on the site bluntly says.
GitHub disabled this NSFW version when WIRED identified the deepfake videos. However, other repositories labeled as “unlocked” versions of the model became available on the platform on January 10, including one with 2,500 “stars.”
“Technically, it is true that once (a pattern) has emerged it cannot be reversed. But we can still make it harder for people to reach,” Seger said.
If left unchecked, the harmful potential of deepfake “porn” is not just psychological, she added. Its knock-on effects include intimidation and manipulation of women, minorities, and politicians, as seen with political deepfakes influencing female politicians globally.
It's not too late to get the problem under control, Seger said, and platforms like GitHub have options, including intervening at upload time. “If you put a model on GitHub and GitHub says no, and all the hosting platforms say no, it becomes more difficult for the average person to get that model.”
Controlling deepfake pornography carried out using open source models also depends on policymakers, technology companies, developers and, of course, the creators of abusive content themselves.
At least 30 US states also have some laws related to deepfake pornography, including bans, according to the nonprofit Public Citizen's. follow the lawalthough definitions and policies vary and some laws apply only to minors. Deepfake creators in the UK will also soon feel the force of the law after the government announced it criminalize the creation of pornographic deepfake worksas well as sharing them on January 7.