Security researchers Protect AI have found tens of 1000’s of “malicious” models on Hugging Face, the Github for artificial intelligence.
© 2023 Bloomberg Finance LP
Hugging Face, the foremost online repository for generative AI, hosted 1000’s of files containing hidden code that may corrupt data and steal information, including the tokens used to pay AI and cloud operators, based on security researchers.
Researchers at security startup ProtectAI, Hidden layer And wizard have been warning for months that hackers have uploaded “malicious models” to Hugging Face’s website, which now has greater than 1,000,000 models available for download.
“The old Trojan horse computer viruses that attempted to inject malicious code into your system have evolved for the AI age,” said Ian Swanson, CEO and founding father of Protect AI. The Seattle, Washington-based startup found over 3,000 malicious files when it began scanning Hugging Face earlier this yr.
According to Swanson, a few of these criminals even arrange fake hugging face profiles to impersonate Meta or other tech firms and lure unwary downloads. A scan by Hugging Face uncovered numerous fake accounts posing as firms including Facebook, Visa, SpaceX and Swedish telecommunications giant Ericsson.
A model that falsely claimed to be from genomic testing startup 23AndMe was downloaded 1000’s of times before it was discovered, Swanson said. He warned that when installed, the malicious code hidden within the fake 23AndMe model would silently seek for AWS passwords that could possibly be used to steal cloud computing resources. Hugging Face deleted the model after being alerted to the chance.
Hugging Face has now integrated ProtectAI’s tool, which scans for malicious code, into its platform and shows users the outcomes before they download anything.
The company announced this Forbes It has reviewed the profiles of major firms corresponding to OpenAI and Nvidia as of 2022. In November 2021, it began scanning the files commonly used to coach machine learning models on the platform for unsafe code. “We hope that our work and partnership with Protect AI, and hopefully many others, will contribute to better trust in machine learning artifacts to facilitate sharing and adoption,” Hugging Face CTO Julien Chaumond said in an email -Email to Forbes.
The risk posed by malicious models was so great that it warranted a joint warning from the US Cybersecurity and Infrastructure Security Agency and the safety authorities of Canada and the UK April. The NSA and its British and Canadian counterparts warned firms to scan all pre-trained models for dangerous code after which only keep them away from critical systems.
The hackers who goal Hugging Face typically insert fraudulent instructions into the code that developers download from the web site and use them to hijack the model when it’s run by an unsuspecting goal. “These are classic attacks, but they’re just hidden in models,” Swanson said. “No one would know that the model was doing these shameful things and it would be incredibly difficult for them to trace it.”
Hugging Face was most recently valued at $4.5 billion in the course of the capital increase 235 million dollars in August 2023. The eight-year-old startup, founded by Clément Delangue, Julien Chaumond and Thomas Wolf, evolved from a chatbot app for teenagers to a machine learning platform in 2018. The company has raised $400 million to date and has been named the Github for AI researchers.
“AI has been a research area for a long time and the security practices have been quite simple,” Chaumond said. “As its popularity increases, so does the number of potentially malicious actors who may want to target the AI community.”
Update: Protect AI clarified that the variety of malicious models found was within the 1000’s, not tens of 1000’s.
