AI-generated "slop" isn't just mucking up our social feeds with deepfakes and other unsavory content. It's also becoming a real problem for marketplace quality teams at TikTok Shop and other e-commerce platforms.
Fraudulent sellers are using generative AI tools to make fake brands or dupe products in an attempt to get users to pay for goods that don't actually exist, said Nicolas Waldmann, who leads TikTok Shop's governance and experience external affairs team.
"It's organized crime, to be honest," Waldmann said. "They're trying to basically go through and sell, and of course, never deliver anything, and then run with the money."
While this type of e-commerce fraud has been around for years, generative AI has increased the sophistication of the methods that bad actors use to try to trick moderation teams on platforms like TikTok Shop or Amazon, Waldmann said. Amazon in March credited a suite of AI tools for helping it track counterfeits and infringing listings on its platform.
TikTok uses a mix of human and AI moderation to help track down fraudulent accounts and listings. The company has its own in-house detection tools, as well as partnerships with outside firms to manage tasks like authenticating pre-owned luxury goods.
"We use AI to basically deal with AI," Waldmann said.
In a new report published Thursday, the company said it had rejected 70 million products and removed 700,000 sellers for various policy violations in the first six months of 2025.
TikTok Shop is a top priority at the social-video company, which has set ambitious growth targets that have been challenging for its US team to hit. Last year, the company drove $100 million in single-day sales on Black Friday in the US.
The company has run into a variety of marketplace quality challenges since it began testing its e-commerce tool in the US in 2022. While TikTok has a lengthy list of prohibited items listed on its website, savvy operators have still managed to break through its filters to sell items like THC syrup and sex toys. "Dupes," or knockoff versions of household-name items, have also created headaches for the social-shopping app. The company's automated enforcement has irked some sellers in the past who said they were hit with violations without a clear explanation.
