Covers July-September 2026.
Harm Trends Report: Q3 2026
Every quarter, Stream reviews anonymized moderation activity across 27 apps on 22 platforms and nearly 4 million harm detections, lines it up with the public research T&S teams follow, and lays out what trust and safety teams should change in response. This edition covers July through September 2026.
Scam, fraud, and off-platform detections up 219% quarter over quarter, rising on 11 of the 15 apps where they appear
Key Findings
- 01
AI-generated sexual imagery is growing faster than last year
Public signalIWF H1 2026 total already exceeds full-year 2025 by 40%Stream signalSexual content detections on images and video up 402% QoQRead the trend - 02
Sextortion runs through ordinary DMs
Public signalNCMEC: 50,000+ financial sextortion reports in 2025, up from 36,000+ in 2024Stream signalChild-safety detections up 63% QoQ, rising on 10 of 11 apps; off-platform contact attempts up 266%Read the trend - 03
Scams start in feeds and chat
Public signalFTC: $2.1B lost to scams that started on social media in 2025Stream signalScam and off-platform detections up 219% QoQ; marketplaces made up 59% of the Q3 totalRead the trend - 04
Automation now finds most violations
Public signalReddit: 86.4% of reports reaching admin review were surfaced proactivelyStream signalAI took 5.1M actions in Q3; moderators reviewed 9.5% of queued itemsRead the trend - 05
Regulators fine for missing basics
Public signalOfcom: £600,000 fine in May 2026 for weak age checks and a late information responseStream signalUnderage-user detections up 42% QoQ (3,433 to 4,891) on the same 17 appsRead the trend
AI-Generated Sexual Imagery Already Passed Last Year's Total
Volume: The IWF assessed 6,310 AI-generated images as child sexual abuse material in the first half of 2026, 40% more than its 2025 total of 4,512 (IWF via Free Press Journal).
Who is targeted: Girls appeared in 98% of those images where age and gender were recorded.
Manipulated real photos: Children made 420 reports to the IWF's Report Remove service in 2026 about images they believe were faked to look explicit, already above the 397 made in all of 2025.
How people find the tools: ISD tracked 5.7 million social media referrals to nudify sites from December 2025 to March 2026. YouTube sent nearly 30% and X about 20% (ISD, Analytics Insight).
+402%
On the same 17 apps, sexual content and nudity detections on images and video rose from 87,097 in Q2 to 437,440 in Q3 (+402%).
Image moderation that only covers public posts misses where this content moves. Profile photos, avatars, and media sent in DMs need the same checks.
Financial Sextortion Runs Through Ordinary DMs
Volume: NCMEC received more than 50,000 reports of financially motivated sextortion in 2025, an average of 137 per day, up from more than 36,000 in 2024 (NCMEC).
Who is targeted: In a Thorn survey of 1,200 people aged 13-20, 1 in 5 said they had experienced sextortion, and 1 in 6 victims were 12 or younger the first time (Thorn, June 2025).
Where it happens: 81% of victims said the threats happened only online (Thorn).
Pattern: The conversation starts on a normal-looking surface (a game, a dating match, a community chat), moves to private messages, and then to a payment demand.
+266%
Stream's labels don't isolate sextortion, so we track the signals around it. On the same 17 apps, child-safety detections (underage users, sexualization of minors) rose 63%, from 4,478 to 7,281, and rose on 10 of the 11 apps that use them. Attempts to move a conversation off-platform rose 266%, from 34,470 to 126,254.
Single-message keyword filters miss this. Detection has to read the conversation, including requests to move to another app.
Scams Start in Feeds and Chat, Then Move Off-Platform
Volume: Americans reported losing $15.9 billion to fraud in 2025 across about 3 million reports (FTC, March 25, 2026).
Social media's share: Americans reported losing $2.1 billion to scams that started on social media in 2025, about eight times the 2020 figure (FTC, PPC Land).
Biggest categories: Investment scams accounted for $1.1 billion. Romance scams accounted for $298 million, and nearly 60% of romance scam losses began on social media.
Most reported: Shopping scams. More than 40% of victims said it began with an ad for a product they ordered.
+219%
On the same 17 apps, scam, fraud, and off-platform detections rose 219%, from 53,858 to 171,622, and increased on 11 of 15 apps. Flags labeled as scams alone rose 118%. Marketplaces and live shopping produced 59% of all Q3 scam and off-platform detections (134,566 of 227,928).
Marketplaces, dating apps, and trading communities are now the first touchpoint for fraud. Blocking off-platform payment and contact requests closes the most common exit.
Automation Now Finds Most Violations Before Users Report Them
Volume: In Reddit's July-December 2025 transparency report, 86.4% of post and comment reports that reached admin review were surfaced by automated systems, and 99% for chat (Reddit, r/RedditSafety, April 2026).
Spam share: Spam made up 54% of Reddit admin removals.
Hate and harassment: Automated enforcement actions against hate and harassment rose 200%.
Volunteer moderators: Moderators removed more than 81 million posts and comments, 68.6% of them with automated tools.
+85%
AI moderation took 5.1 million actions in Q3 (flag, remove, or shadow block), up 85% from Q2. Moderators reviewed 9.5% of items that reached a review queue, up from 3.8% in Q2. Livestream moderation flagged 31,300 live video and audio sessions in Q3, its first quarter with activity in this group.
The review queue should hold the gray-area cases. If moderators spend their day clearing spam, the automation thresholds need adjusting.
Regulators Are Fining Platforms for Missing Basic Controls
Volume: The EU fined Temu €200 million under the Digital Services Act on May 28, 2026, its largest DSA penalty so far, after a €120 million fine against X in December 2025 (Slaughter and May).
Fine: On May 26, 2026, Ofcom fined Youngtek Solutions £600,000: £500,000 for not using highly effective age assurance and £100,000 for missing the deadline on a statutory information request (GBlock).
Open investigation: Ofcom is investigating X over reports that an AI chatbot was used to create and share sexual deepfakes, including of children (Burges Salmon).
2026 priorities: Ofcom's stated focus areas include child sexual abuse and grooming, effective age checks, removal of terror and illegal hate content, and safer experiences for women and girls.
+42%
On the same 17 apps, underage-user detections rose 42%, from 3,433 in Q2 to 4,891 in Q3, and child-safety detections rose on 10 of the 11 apps that use them. Platforms are flagging more users who look under age before a regulator asks.
The fines so far are for basics: age checks and answering the regulator on time. Teams need records of what was flagged, what was done, and when, ready before anyone asks.
What to Change This Quarter
FAQ
Every quarter. Each edition covers the previous three months and compares them with the quarter before.
From anonymized, aggregate moderation activity across apps that use Stream Moderation for chat, feeds, video, and live. We report counts and percentage changes, never individual content.
Yes. Please credit "Stream Harm Trends Report, Q3 2026" and link to this page.
Subscribe below, and we will send it when it is published.
Get the next edition
Stream data: Anonymized, aggregated moderation activity across apps on Stream Moderation, July 1-September 30, 2026, compared with April 1-June 30, 2026. No customer, user, or message is identifiable.
Coverage: 27 apps on 22 platforms with moderation activity in Q3 (20 apps in Q2), across marketplaces and live shopping, social and community, prediction markets and betting, and other verticals; text, image, video, and livestream. Trend figures compare the same 17 apps active in both quarters, so new launches don't inflate growth. Breakdowns covering fewer than 5 apps are not reported.
Public data: Figures from the IWF, NCMEC, Thorn, FTC, ISD, Reddit, Ofcom and the European Commission, linked where cited.
