AI Text Harm Detection

Stream’s AI Text moderation engine utilizes both internal AI models and external models to provide real-time content moderation. You can see the current list of subprocessors here. This sophisticated engine analyzes content across multiple languages to detect harmful content with high accuracy and contextual understanding.

Language Support

AI Text moderation supports a wide range of languages. The system automatically identifies the content language and applies appropriate moderation models. We support the following languages by default:

• English
• French
• German
• Italian
• Portuguese
• Spanish
• Russian
• Arabic
• Dutch
• Polish
• Finnish
• Chinese
• Hebrew
• Hindi
• Turkish

Configuration

To set up the AI Harm Engine, navigate to the “AI Text” section in your moderation policy. Here, you’ll find a comprehensive list of harm categories that our advanced AI can detect. These categories cover a wide range of potential issues, from harassment and hate speech to spam and explicit content.

For each harm category, you can choose the appropriate action: Flag, Block, Shadow Block. You can also adjust the sensitivity level for certain toxic categories, allowing you to fine-tune the moderation to your specific needs. Remember, it’s often best to start with more lenient settings and adjust as you learn more about your community’s needs.

Severity Levels

The AI Text moderation engine supports classifying content into four severity levels: Low, Medium, High, and Critical. The severity level is only relevant for toxic categories, e.g., hatred, sexual harassment, etc.

You can configure the action to be taken for each severity as shown in the screenshot below:

Supported Harms and Categories

Hatred

Hatred includes expressions of intense dislike or ill will towards individuals or groups.

Examples:

• Low Severity
  • We don’t care.
  • Get a life.
  • Go buy a brain.
  • Learn how to write.
  • You need therapy.
  • You are a nasty person.
  • More woke bullshit.
  • He’s batshit crazy.
  • Who gives a fuck?
  • Why the fuck are you still speaking?
• Medium Severity
  • You piss me off.
  • The grown-ups are talking. Go play with your Lego.
  • You disgust me!
  • You have no balls.
  • Don’t hesitate to dislike this video.
  • Bruh, you can’t even speak English.
  • Grow a pair.
  • You’re revealing your LOW IQ levels.
  • I hate you.
  • Stop talking bullshit.
  • There are no more “real” men left in this country.
• High Severity
  • Go back to where you came from.
  • You are a danger to the West.
  • You libtards say, “Oh, it was an accident.”
  • Who else doesn’t give a shit about his life?
  • Go back to your little hole.
  • I don’t give a shit about anything YOU post.
  • Not just us, pal; everyone hates you.
  • You are dead to me, bitch.
  • Who cares about you, terrorist? 💣
  • Lazy women find suckers to do the work they should have done.

Insult

Insults are expressions that demean or belittle others.

Examples:

• Low Severity
  • Are you stupid or something?
  • He is a fool.
  • You’re a clown.
  • Stupid player!
  • Shut up, please.
  • She’s garbage.
  • Bunch of losers.
  • Screw you!
  • Get the fuck out!
  • Cry more.
  • You’re dumber than a box of rocks.
• Medium Severity
  • 🖕🏻🖕🏻🖕🏻
  • You fucking bastard.
  • Fuck you.
  • You’re a dumbass.
  • She is the A-hole.
  • Her name is GOLD DIGGER.
  • What a tosser.
  • She is a manipulative bitch.
  • What a fucking cockhead.
  • You’re such a bellend.
• High Severity
  • What a bunch of fucktards.
  • Shut the fuck up.
  • She’s a total whore.
  • You all are cocksuckers.
  • This motherfucker is driving me crazy.
  • You son of a bitch.
  • Get screwed, man.
  • Kiss my ass.
  • Go eat a bag of dicks.
  • She’s nothing but a whore.

Body-shaming

Body-shaming involves criticizing or mocking someone’s physical appearance.

Examples:

• Low Severity
  • This guy stinks.
  • She still looks hideous.
  • She looks like she smells.
  • You are ugly.
  • That girl isn’t even pretty.
  • OMG, those hands 🤮🤮🤮.
  • You’re scary to look at.
  • You smell like a 💨.
  • Why don’t you get rid of those nasty-looking dreadlocks?
  • Don’t be mad because I’m prettier than you.
• Medium Severity
  • She looks like a man to me.
  • Your voice is annoying.
  • Go lose some weight, dude!
  • You have no boobs.
  • You sound like a man.
  • Why are you so fat?
  • You are too fat.
  • Butter face.
  • You must have a tiny penis.
  • Your teeth are yellow.
• High Severity
  • You are a big fat turd face.
  • Her coochie probably smells like a landfill.
  • Don’t talk to me if you’re overweight.
  • This is proof that gingers have no soul. 😞
  • When’s your weight loss surgery for your fat?
  • Mr. Donut needs to go on a diet.
  • You can’t hide your fat with body positivity.
  • His face makes me vomit.
  • He needs to lose at least 200 pounds; he’s way overweight.
  • Look like you smell like cheese.

Reputation Harm

Reputation harm involves damaging someone’s reputation through false or malicious statements.

Examples:

• Low Severity
  • You’re greenwashing again.
  • I’m sure the umpire has been paid to ignore the fouls.
• High Severity
  • He is in cahoots with pedophiles.
  • This company finances terrorism.

Doxxing

Doxxing is the act of publicly revealing private information about an individual without their consent.

Examples:

• Post her photos and address.
• His address is 520 South San Julian St.
• What’s the cop’s full name? I will post his home address on social media, and then people can go get street justice.
• I’m leaking your number in the telegram.
• Guys, go leak @mrx’s address.

Racism

Racism includes discriminatory or prejudiced comments based on race or ethnicity.

Examples:

• They need to go back to their own country.
• That’s why I hate black people.
• CHINESE CRAP!!
• Of course, it’s black 🦍; nothing too gets surprised.
• Entitled immigrants are the worst.
• Don’t hire stupid Mexicans.
• You look like a monkey.
• Fucking gypsies who spoil everything.
• Gas them Jews.
• Go back to the cotton fields.
• You speak ching chong.
• Islam is the problem everywhere.

LGBTQIAPLUS_PHOBIA

LGBTQIAPLUS_phobia includes negative or hostile comments towards LGBTQIAPLUS individuals.

Examples:

• That man will never be a woman. #Freak
• He is very gay.
• Wow, definitely a fag.
• Stop being gay.
• Fuck 🏳️.
• Alphabet people are sick.
• A trans woman is just a man.
• Your video is as gay as you look.
• Bro, stop with your sissy stories!
• Dude, such a homo.

Misogyny

Misogyny includes comments that express hatred or prejudice against women.

Examples:

• Women ☕.
• Women should be quiet…
• Don’t care, didn’t ask, plus you’re a woman.
• Women are evil.
• Women have no value anymore.
• This is why women should not be in charge of anything, much less vote.
• Men are superior to women.
• Go back to the kitchen.
• Sadly, the only thing 90% of you women today are good for is sex…
• I also think women should be submissive to their husbands.

Ableism

Ableism includes discriminatory or prejudiced comments based on physical or mental disabilities.

Examples:

• This kid is retarded.
• She belongs in an asylum.
• You’re brain dead.
• You have a debilitating level of autism.
• Lady, you’re demented.
• Do you have mental health issues?!
• If you paid $600 for that, you have special needs. SMH.
• I’m sick of her; she’s absolutely sick in the head; her mom was right; she’s ill.
• The vaccine definitely made you autistic.
• He’s a little gimp; can’t stand him.

Threat

Threats include statements expressing an intention to cause harm or violence to someone.

Examples:

• I would beat him so badly.
• I would have slapped her.
• I would kick him.
• If my son did something like this, I’d shoot him.
• I’ll break your worthless jaw anytime, anywhere.
• We’ll kill your wife too.
• I would love to whoop your ass so badly. I’d literally punch your face until I hit the ground.
• I just want to punch him in the forehead.
• I’d definitely beat your ass.
• I’m gonna kick you in the nuts.

Self Harm

Self-harm includes expressions of intent to harm oneself or indications of self-destructive behavior.

Examples:

• I think I’ll kill myself soon. Honestly, at this point in my life, I don’t see myself getting any better. Every single day is getting harder, and I regret so much being an orphan at this young age. I wish I were someone’s favorite person and that I could talk to someone about how I really feel, but I just know nobody cares. At the end of the day, I feel like everyone’s going to tell me that it’s not that serious, but I just cannot take it anymore. 💔
• I feel depressed and need someone to talk to, please.
• I’m really going to end my life.
• I’ve been having suicidal thoughts.
• Lately, I’m at my lowest.

Sexual Harassment

Sexual harassment includes unwelcome comments or advances of a sexual nature.

Examples:

• Milf alert.
• She’s got a nice ass.
• You can sit on my face :P.
• He’s going to bang her good tonight.
• I just masturbate.
• You suck the best cock.
• He likes it up the bum.
• Spread your legs.
• Kiss my balls.
• You have nice tits.

Moral Harassment

Moral harassment includes comments that aim to degrade or humiliate someone.

Examples:

• They deserve to die.
• Shoot the bitch.
• Hope they die a slow death.
• He should rot in hell.
• Off with his head.
• I hope they break your wrist.
• Shoot them.
• Kay, why is?
• Hang him.
• He should be killed.

Terrorism & Violent Extremism

Terrorism and violent extremism include support or promotion of terrorist activities or ideologies.

Examples:

• Heil Hitler!
• Hitler was right.
• An Austrian painter was a good guy.
• I’m a school shooter.
• You deserve another 🛫🏢🏢💣💣.

Pedophilia

Pedophilia includes comments that express sexual interest in minors.

Examples:

• I like to fuck minors.
• If her age is on the clock, then she can suck this cock.
• Me playing with small boys’ dicks.
• I’m masturbating to child porn.
• I know you’re not legal, but I still think you’re sexy.

SCAM

Scam includes fraudulent schemes or deceptive practices.

Examples:

• Send me a private message; you have won a prize.
• Message me now; I have something for you, okay?
• Send me a friend request or message me, handsome.
• I’ve got 4 tickets available; PM if interested.

FLOOD

Flood includes excessive or repetitive posting of messages.

Examples:

• ▔▔▔▔▔▔▔▔▔▔▔ ╲🔴🔵 ▕╮╭┻┻╮╭┻┻╮╭▕╮╲ ▕╯┃╭╮┃╭╮┃╰▕╯╭▏ ▕╭┻┻┻┛┗┻┻┛ ╰▏ ▕╰━━━┓┈┈┈╭╮▕╭╮▏ ▕╭╮╰┳┳┳┳╯╰╯▕╰╯▏ ▕╰╯┈┗┛┗┛┈╭╮.

Forbidden Link

Forbidden link includes sharing links to prohibited or harmful websites.

Examples:

• If you want to know more, you can check this website: example-website.com.

Forbidden Language

Forbidden language includes the use of prohibited or offensive words or phrases.

Examples:

Useless

Useless comments include irrelevant or trivial messages that do not contribute to the conversation.

Examples:

• First ❤️.
• 2NDDDD.
• This is the 666th comment.
• I’m the first to comment.
• I’m the 3rd.

Ads

Ads include promotional messages or advertisements.

Examples:

• Follow my page.
• Subscribe to me; my YouTube name is Artymonx_plaz.

Platform Bypass

Platform bypass includes attempts to move the conversation to another platform.

Examples:

• Can we talk on Snap? It’s better.
• Go Telegram?
• FROG, DO YOU HAVE DISCORD?

Negative Criticism

Negative criticism includes harsh or unconstructive comments.

Examples:

• Shame on you!
• Boring af.
• This guy is overrated.
• This is not funny at all.
• Daughter is disrespectful.
• I hope you aren’t actually that naive or ignorant.
• That chick is annoying.
• They look ridiculous.
• People are so entitled.
• Another waste of time with a cut-off video.

Boycott

Boycott includes calls to avoid or stop supporting certain products or companies.

Examples:

• Low Severity
  • It’s an Israeli product; we have to boycott this.
  • We’re not buying; boycott all brands that support Zionism!
  • But I will boycott this company as it owns this fashion brand.
  • Don’t buy them!
• High Severity
  • Boycott this brand; free Palestine.
  • Stop buying from this brand.

PII

PII includes sharing personally identifiable information without consent.

Examples:

• 17yo 5”9🧍🏾.
• I msg 6827851099; Insta: duckedoffn3ro; I’m not texting first; princess treatment or nothing, and I’m never on here, so 🤷🏽.
• Text me +1 774-578-1448 since I can’t text here.
• My Fortnite account is jerryy_69; I’ll add my Insta later tonight; I’m hella introverted, so I gotta get close to you; my number is 559-373-8181; text me if you want.
• Hi, my email address is Carolbale19@icloud.com.

Underage User

Underage user includes statements indicating that the user is below the minimum age requirement.

Examples:

• I’m not thirteen yet.
• I’m in 8th grade, but I’m still 12.
• I go to elementary school.
• I’m about to turn 11.

Link

Link includes sharing URLs or hyperlinks.

Examples:

Finance

Finance includes comments related to financial matters or transactions.

Examples:

• Yes, and causing an unlawful corporate contribution.
• Using this company’s resources to influence shareholders to vote in favor of a CEO’s compensation package is a breach of fiduciary duty. I expect another lawsuit.
• Capital gains from investment are available to everyone.
• I’ve bought this brand through this company for several years. They have a 20+ percent stake in this company. One of the best businesses in the world.
• The figure tracking the business volume in rural areas also rose from the previous month, climbing 1.8 points to 129.7 points.

Dating

Dating includes comments expressing romantic interest or seeking relationships.

Examples:

• I’m looking for a relationship too, and you’re very beautiful.
• I am single; what about you?
• Do you want to be my girlfriend?

Politics

Politics includes comments related to political matters or figures.

Examples:

• I sent a direct message to her and never got a response. She is protecting Trudeau and Singh.
• Are you saying Trump has been outsmarted by Biden? If this is what Sleepy Joe can do,… just think what Putin can do.
• Five more weeks of this; they might implode, and a hung parliament, the prize escaped them.
• PM Modi’s Kolkata Roadshow touches 3 destinations linked to iconic figures.
• This means they would need to form a coalition with one of the populist parties, such as the Economic Freedom Fighters, or uMkhonto weSizwe, or the center-right Democratic Alliance party.

Geopolitical

Geopolitical includes comments related to international relations or global issues.

Examples:

• ACAB.
• #FreePalestine now 🇵🇸.
• Defund the police and government.
• ❤️❤️❤️❤️🇮🇱🇮🇱🇮🇱.
• Genocide in Gaza.

Terrorism Reference

Terrorism reference includes mentions of terrorist organizations or activities.

Examples:

• Hezbollah is a Lebanese Shia Islamist political party.
• Actually, PAFF claimed the attack on their official channel on Telegram.
• ✈️🏢.

Cybersecurity

Cybersecurity includes comments related to online security or cyber threats.

Examples:

• Clip from Security talk explains what a Man in the Middle Attack is. 🔐.
• We have UDP flooding.
• There was an unauthorized download of email addresses on #OpenSea; just be vigilant if you receive any emails.
• A new cyberattack that utilizes the native Windows protection tool, BitLocker, to encrypt victims’ disks.
• #starmus explains how antivirus/anti-malware works.

Vulgarity

Vulgarity includes the use of crude or offensive language.

Examples:

• Low Severity
  • Bitch, try me.
  • I need this af.
  • Oh shit.
  • That’s a chickenshit call.
  • She bought some shitty used car.
• High Severity
  • Stop dickriding.
  • She likes saying pussy pink.
  • Rihanna said I’m tired of meat riders.
  • Nah, there’s 🍆🚴 than there’s this.
  • I’m so over dick riders and fakes.

Sexually Explicit

Sexually explicit includes comments that are sexually graphic or suggestive.

Examples:

• Low Severity
  • Freeze your butt off time.
  • Pornography is for adults only.
  • Maybe she only wants to be friends with benefits.
  • The new Olympic mascot looks like a vagina.
  • If you don’t want children, just use a condom.
• High Severity
  • She probably has a secret OnlyFans account.
  • It’s a website that sells dildos.
  • Someone’s getting a blow job tonight 🙄.
  • Nude content?
  • He probably watches hentai; don’t listen to him.

Drug Explicit

Drug explicit includes comments related to drug use or trafficking.

Examples:

• Low Severity
  • He gave up the marijuana, I hear.
  • I said doctors are the biggest drug dealers.
  • Drug trafficking?
  • I don’t know why rappers name themselves after things like Xanax.
  • There’s a dispensary next to the gas station.
• High Severity
  • Was it meth or weed?
  • Go smoke a joint and realize it’ll all be okay.
  • That’s a packet of heroin.
  • I heard they got coke that was laced with fentanyl.
  • 24g of coke.

Weapon Explicit

Weapon explicit includes comments related to weapons or their use.

Examples:

• Low Severity
  • I’ve got pepper spray to defend myself.
  • Should have brought a bag of airsoft guns to that place.
  • You can buy a gun for under $200?
  • Somebody has stuck a knife in me…
  • Just bring the machete.
• High Severity
  • Atomic bomb.
  • You need a rifle and backhoe.
  • Maybe a shotgun would be better.
  • I use a Glock.
  • Did he actually nuke the moon?

Pedophilia Reference

Pedophilia reference includes mentions of pedophilia or related activities.

Examples:

• Having sex with children is a crime.
• He was looking up spicy pics of teens; that’s how they got him.
• It’s not child porn. They are Indecent Images of Children depicting sexual abuse and rape of children under 18.

Supportive

Supportive comments include positive or encouraging messages.

Examples:

• He’s fantastic.
• So awesome!
• I am your big fan.
• Your voice is beautiful 🤩.
• You are the best!
• ♥️♥️♥️♥️♥️♥️♥️.
• I love this guy. He’s so sweet.
• May God bless you.
• I really appreciate your work.
• The work this woman does is amazing.

Fairplay

Fairplay includes comments that promote sportsmanship and fair competition.

Examples:

• You played well.
• GG.
• Have fun.
• You deserve the win, bro.
• That wasn’t an easy win.

Encouragement

Encouragement includes comments that offer support or motivation.

Examples:

• Keep your head up!
• You can do this!
• Good luck.
• I believe in you!
• Don’t give up!
• All the best for you both <3.
• Stay safe.
• Prayers for a speedy recovery.
• Get well soon!
• Stay strong.