AI Text (NLP)

Stream's NLP text engine classifies content across multiple languages to detect harmful content with high accuracy and contextual understanding.

Language Support

The NLP engine detects the language of the text automatically.

Six languages are maintained directly and are available by default:

• English
• French
• Spanish
• Portuguese
• German
• Italian

These 32 languages are available on request. The text is translated to English and moderated with the English rules, so accuracy is close to the maintained languages. Dialects, such as Arabizi for Arabic, are not covered. Contact support to enable one.

• Arabic
• Hebrew
• Russian
• Ukrainian
• Chinese
• Japanese
• Korean
• Thai
• Hindi
• Indonesian
• Vietnamese
• Bengali
• Turkish
• Irish
• Catalan
• Polish
• Czech
• Greek
• Romanian
• Danish
• Dutch
• Croatian
• Finnish
• Estonian
• Swedish
• Norwegian
• Hungarian
• Latvian
• Lithuanian
• Slovak
• Slovenian
• Swahili

Configuration

Set up the NLP engine with ai_text_config on a policy. Each rule names a label and either one action for every match or severity_rules with an action per severity. Labels are uppercase strings from Supported harms and categories, such as INSULT and HATRED. SPAM is allowed. HARASSMENT is not a label.

await client.moderation.upsertConfig({
  key: "chat:messaging",
  ai_text_config: {
    enabled: true,
    rules: [
      // One action for every match
      { label: "INSULT", action: "flag" },
      // Labels without severity can only take one action
      { label: "PLATFORM_BYPASS", action: "flag" },
      // An action per severity
      {
        label: "HATRED",
        severity_rules: [
          { severity: "low", action: "flag" },
          { severity: "medium", action: "flag" },
          { severity: "high", action: "remove" },
          { severity: "critical", action: "remove" },
        ],
      },
    ],
  },
});

NLP actions are keep, flag, mask, mask_flag, shadow, remove, bounce, bounce_flag, and bounce_remove. See Automatic actions for what each one does.

How the NLP engine differs from the LLM engine:

• Labels come from a fixed list. You can't add your own labels or describe them in plain language.
• There is no app context or rule description. The engine classifies each text on its own.
• When a rule has severity_rules, they decide the action. A severity with no matching entry falls back to the rule's action, if set.
• Upserting ai_text_config replaces the whole rule list.

Severity Levels

The NLP engine grades harmful labels at four severity levels:

Content that isn't harmful has no severity. Severity is graded per label, so one message can be low for VULGARITY and high for THREAT. Set an action per severity in the dashboard policy editor, or with severity_rules on the rule, as in the configuration example. For example, flag low-severity insults for review and remove high and critical ones.

Labels without severity

Some labels describe a topic or a behavior rather than a degree of harm, so the engine never grades them. These labels have no severity levels: you can only match them by label and set one action. severity_rules don't apply to them, and the dashboard doesn't show severity options for them.

FLOOD, PII, and ADS cover several kinds of content (for example, an email address and a phone number are both PII), and rules always match the parent label.

Supported Harms and Categories

The NLP engine returns the labels below, which have severity levels, the labels without severity, and custom labels when they are enabled for your app. Examples for each label follow the table.

Hatred

Hatred includes expressions of intense dislike or ill will towards individuals or groups.

Examples:

• Low Severity
  • We don't care.
  • Get a life.
  • Go buy a brain.
  • Learn how to write.
  • You need therapy.
  • You are a nasty person.
  • More woke bullshit.
  • He's batshit crazy.
  • Who gives a fuck?
  • Why the fuck are you still speaking?
• Medium Severity
  • You piss me off.
  • The grown-ups are talking. Go play with your Lego.
  • You disgust me!
  • You have no balls.
  • Don't hesitate to dislike this video.
  • Bruh, you can't even speak English.
  • Grow a pair.
  • You're revealing your LOW IQ levels.
  • I hate you.
  • Stop talking bullshit.
  • There are no more "real" men left in this country.
• High Severity
  • Go back to where you came from.
  • You are a danger to the West.
  • You libtards say, "Oh, it was an accident."
  • Who else doesn't give a shit about his life?
  • Go back to your little hole.
  • I don't give a shit about anything YOU post.
  • Not just us, pal; everyone hates you.
  • You are dead to me, bitch.
  • Who cares about you, terrorist? 💣
  • Lazy women find suckers to do the work they should have done.

Insult

Insults are expressions that demean or belittle others.

Examples:

• Low Severity
  • Are you stupid or something?
  • He is a fool.
  • You're a clown.
  • Stupid player!
  • Shut up, please.
  • She's garbage.
  • Bunch of losers.
  • Screw you!
  • Get the fuck out!
  • Cry more.
  • You're dumber than a box of rocks.
• Medium Severity
  • 🖕🏻🖕🏻🖕🏻
  • You fucking bastard.
  • Fuck you.
  • You're a dumbass.
  • She is the A-hole.
  • Her name is GOLD DIGGER.
  • What a tosser.
  • She is a manipulative bitch.
  • What a fucking cockhead.
  • You're such a bellend.
• High Severity
  • What a bunch of fucktards.
  • Shut the fuck up.
  • She's a total whore.
  • You all are cocksuckers.
  • This motherfucker is driving me crazy.
  • You son of a bitch.
  • Get screwed, man.
  • Kiss my ass.
  • Go eat a bag of dicks.
  • She's nothing but a whore.

Body-shaming

Body-shaming involves criticizing or mocking someone's physical appearance.

Examples:

• Low Severity
  • This guy stinks.
  • She still looks hideous.
  • She looks like she smells.
  • You are ugly.
  • That girl isn't even pretty.
  • OMG, those hands 🤮🤮🤮.
  • You're scary to look at.
  • You smell like a 💨.
  • Why don't you get rid of those nasty-looking dreadlocks?
  • Don't be mad because I'm prettier than you.
• Medium Severity
  • She looks like a man to me.
  • Your voice is annoying.
  • Go lose some weight, dude!
  • You have no boobs.
  • You sound like a man.
  • Why are you so fat?
  • You are too fat.
  • Butter face.
  • You must have a tiny penis.
  • Your teeth are yellow.
• High Severity
  • You are a big fat turd face.
  • Her coochie probably smells like a landfill.
  • Don't talk to me if you're overweight.
  • This is proof that gingers have no soul. 😞
  • When's your weight loss surgery for your fat?
  • Mr. Donut needs to go on a diet.
  • You can't hide your fat with body positivity.
  • His face makes me vomit.
  • He needs to lose at least 200 pounds; he's way overweight.
  • Look like you smell like cheese.

Reputation Harm

Reputation harm involves damaging someone's reputation through false or malicious statements.

Examples:

• Low Severity
  • You're greenwashing again.
  • I'm sure the umpire has been paid to ignore the fouls.
• High Severity
  • He is in cahoots with pedophiles.
  • This company finances terrorism.

Doxxing

Doxxing is the act of publicly revealing private information about an individual without their consent.

Examples:

• Post her photos and address.
• His address is 520 South San Julian St.
• What's the cop's full name? I will post his home address on social media, and then people can go get street justice.
• I'm leaking your number in the telegram.
• Guys, go leak @mrx's address.

Racism

Racism includes discriminatory or prejudiced comments based on race or ethnicity.

Examples:

• They need to go back to their own country.
• That's why I hate black people.
• CHINESE CRAP!!
• Of course, it's black 🦍; nothing too gets surprised.
• Entitled immigrants are the worst.
• Don't hire stupid Mexicans.
• You look like a monkey.
• Fucking gypsies who spoil everything.
• Gas them Jews.
• Go back to the cotton fields.
• You speak ching chong.
• Islam is the problem everywhere.

LGBTQIAPLUS_PHOBIA

LGBTQIAPLUS_phobia includes negative or hostile comments towards LGBTQIAPLUS individuals.

Examples:

• That man will never be a woman. #Freak
• He is very gay.
• Wow, definitely a fag.
• Stop being gay.
• Fuck 🏳️.
• Alphabet people are sick.
• A trans woman is just a man.
• Your video is as gay as you look.
• Bro, stop with your sissy stories!
• Dude, such a homo.

Misogyny

Misogyny includes comments that express hatred or prejudice against women.

Examples:

• Women ☕.
• Women should be quiet...
• Don't care, didn't ask, plus you're a woman.
• Women are evil.
• Women have no value anymore.
• This is why women should not be in charge of anything, much less vote.
• Men are superior to women.
• Go back to the kitchen.
• Sadly, the only thing 90% of you women today are good for is sex...
• I also think women should be submissive to their husbands.

Ableism

Ableism includes discriminatory or prejudiced comments based on physical or mental disabilities.

Examples:

• This kid is retarded.
• She belongs in an asylum.
• You're brain dead.
• You have a debilitating level of autism.
• Lady, you're demented.
• Do you have mental health issues?!
• If you paid $600 for that, you have special needs. SMH.
• I'm sick of her; she's absolutely sick in the head; her mom was right; she's ill.
• The vaccine definitely made you autistic.
• He's a little gimp; can't stand him.

Threat

Threats include statements expressing an intention to cause harm or violence to someone.

Examples:

• I would beat him so badly.
• I would have slapped her.
• I would kick him.
• If my son did something like this, I'd shoot him.
• I'll break your worthless jaw anytime, anywhere.
• We'll kill your wife too.
• I would love to whoop your ass so badly. I'd literally punch your face until I hit the ground.
• I just want to punch him in the forehead.
• I'd definitely beat your ass.
• I'm gonna kick you in the nuts.

Self Harm

Self-harm includes expressions of intent to harm oneself or indications of self-destructive behavior.

Examples:

• I think I'll kill myself soon. Honestly, at this point in my life, I don't see myself getting any better. Every single day is getting harder, and I regret so much being an orphan at this young age. I wish I were someone's favorite person and that I could talk to someone about how I really feel, but I just know nobody cares. At the end of the day, I feel like everyone's going to tell me that it's not that serious, but I just cannot take it anymore. 💔
• I feel depressed and need someone to talk to, please.
• I'm really going to end my life.
• I've been having suicidal thoughts.
• Lately, I'm at my lowest.

Sexual Harassment

Sexual harassment includes unwelcome comments or advances of a sexual nature.

Examples:

• Milf alert.
• She's got a nice ass.
• You can sit on my face :P.
• He's going to bang her good tonight.
• I just masturbate.
• You suck the best cock.
• He likes it up the bum.
• Spread your legs.
• Kiss my balls.
• You have nice tits.

Moral Harassment

Moral harassment includes comments that aim to degrade or humiliate someone.

Examples:

• They deserve to die.
• Shoot the bitch.
• Hope they die a slow death.
• He should rot in hell.
• Off with his head.
• I hope they break your wrist.
• Shoot them.
• Kay, why is?
• Hang him.
• He should be killed.

Terrorism & Violent Extremism

Terrorism and violent extremism include support or promotion of terrorist activities or ideologies.

Examples:

• Heil Hitler!
• Hitler was right.
• An Austrian painter was a good guy.
• I'm a school shooter.
• You deserve another 🛫🏢🏢💣💣.

Pedophilia

Pedophilia includes comments that express sexual interest in minors.

Examples:

• I like to fuck minors.
• If her age is on the clock, then she can suck this cock.
• Me playing with small boys' dicks.
• I'm masturbating to child porn.
• I know you're not legal, but I still think you're sexy.

SCAM

Scam includes fraudulent schemes or deceptive practices.

Examples:

• Send me a private message; you have won a prize.
• Message me now; I have something for you, okay?
• Send me a friend request or message me, handsome.
• I've got 4 tickets available; PM if interested.

FLOOD

Flood includes excessive or repetitive posting of messages.

Examples:

• ▔▔▔▔▔▔▔▔▔▔▔ ╲🔴🔵 ▕╮╭┻┻╮╭┻┻╮╭▕╮╲ ▕╯┃╭╮┃╭╮┃╰▕╯╭▏ ▕╭┻┻┻┛┗┻┻┛ ╰▏ ▕╰━━━┓┈┈┈╭╮▕╭╮▏ ▕╭╮╰┳┳┳┳╯╰╯▕╰╯▏ ▕╰╯┈┗┛┗┛┈╭╮.

Forbidden Link

Forbidden link includes sharing links to prohibited or harmful websites.

Examples:

• If you want to know more, you can check this website: example-website.com.

Useless

Useless comments include irrelevant or trivial messages that do not contribute to the conversation.

Examples:

• First ❤️.
• 2NDDDD.
• This is the 666th comment.
• I'm the first to comment.
• I'm the 3rd.

Ads

Ads include promotional messages or advertisements.

Examples:

• Follow my page.
• Subscribe to me; my YouTube name is Artymonx_plaz.

Platform Bypass

Platform bypass includes attempts to move the conversation to another platform.

Examples:

• Can we talk on Snap? It's better.
• Go Telegram?
• FROG, DO YOU HAVE DISCORD?

Negative Criticism

Negative criticism includes harsh or unconstructive comments.

Examples:

• Shame on you!
• Boring af.
• This guy is overrated.
• This is not funny at all.
• Daughter is disrespectful.
• I hope you aren't actually that naive or ignorant.
• That chick is annoying.
• They look ridiculous.
• People are so entitled.
• Another waste of time with a cut-off video.

Boycott

Boycott includes calls to avoid or stop supporting certain products or companies.

Examples:

• Low Severity
  • It's an Israeli product; we have to boycott this.
  • We're not buying; boycott all brands that support Zionism!
  • But I will boycott this company as it owns this fashion brand.
  • Don't buy them!
• High Severity
  • Boycott this brand; free Palestine.
  • Stop buying from this brand.

PII

PII includes sharing personally identifiable information without consent.

Examples:

• 17yo 5"9🧍🏾.
• I msg 6827851099; Insta: duckedoffn3ro; I'm not texting first; princess treatment or nothing, and I'm never on here, so 🤷🏽.
• Text me +1 774-578-1448 since I can't text here.
• My Fortnite account is jerryy_69; I'll add my Insta later tonight; I'm hella introverted, so I gotta get close to you; my number is 559-373-8181; text me if you want.
• Hi, my email address is user@example.com.

Underage User

Underage user includes statements indicating that the user is below the minimum age. The threshold is configurable: under 13, 16, or 18. Contact support to change it for your app.

Examples:

• I'm not thirteen yet.
• I'm in 8th grade, but I'm still 12.
• I go to elementary school.
• I'm about to turn 11.

Link

Link includes sharing URLs or hyperlinks.

Examples:

Dating

Dating includes comments expressing romantic interest or seeking relationships.

Examples:

• I'm looking for a relationship too, and you're very beautiful.
• I am single; what about you?
• Do you want to be my girlfriend?

Politics

Politics includes comments related to political matters or figures.

Examples:

• I sent a direct message to her and never got a response. She is protecting Trudeau and Singh.
• Are you saying Trump has been outsmarted by Biden? If this is what Sleepy Joe can do,... just think what Putin can do.
• Five more weeks of this; they might implode, and a hung parliament, the prize escaped them.
• PM Modi's Kolkata Roadshow touches 3 destinations linked to iconic figures.
• This means they would need to form a coalition with one of the populist parties, such as the Economic Freedom Fighters, or uMkhonto weSizwe, or the center-right Democratic Alliance party.

Geopolitical

Geopolitical includes comments related to international relations or global issues.

Examples:

• ACAB.
• #FreePalestine now 🇵🇸.
• Defund the police and government.
• ❤️❤️❤️❤️🇮🇱🇮🇱🇮🇱.
• Genocide in Gaza.

Terrorism Reference

Terrorism reference includes mentions of terrorist organizations or activities.

Examples:

• Hezbollah is a Lebanese Shia Islamist political party.
• Actually, PAFF claimed the attack on their official channel on Telegram.
• ✈️🏢.

Vulgarity

Vulgarity includes the use of crude or offensive language.

Examples:

• Low Severity
  • Bitch, try me.
  • I need this af.
  • Oh shit.
  • That's a chickenshit call.
  • She bought some shitty used car.
• High Severity
  • Stop dickriding.
  • She likes saying pussy pink.
  • Rihanna said I'm tired of meat riders.
  • Nah, there's 🍆🚴 than there's this.
  • I'm so over dick riders and fakes.

Sexually Explicit

Sexually explicit includes comments that are sexually graphic or suggestive.

Examples:

• Low Severity
  • Freeze your butt off time.
  • Pornography is for adults only.
  • Maybe she only wants to be friends with benefits.
  • The new Olympic mascot looks like a vagina.
  • If you don't want children, just use a condom.
• High Severity
  • She probably has a secret OnlyFans account.
  • It's a website that sells dildos.
  • Someone's getting a blow job tonight 🙄.
  • Nude content?
  • He probably watches hentai; don't listen to him.

Drug Explicit

Drug explicit includes comments related to drug use or trafficking.

Examples:

• Low Severity
  • He gave up the marijuana, I hear.
  • I said doctors are the biggest drug dealers.
  • Drug trafficking?
  • I don't know why rappers name themselves after things like Xanax.
  • There's a dispensary next to the gas station.
• High Severity
  • Was it meth or weed?
  • Go smoke a joint and realize it'll all be okay.
  • That's a packet of heroin.
  • I heard they got coke that was laced with fentanyl.
  • 24g of coke.

Weapon Explicit

Weapon explicit includes comments related to weapons or their use.

Examples:

• Low Severity
  • I've got pepper spray to defend myself.
  • Should have brought a bag of airsoft guns to that place.
  • You can buy a gun for under $200?
  • Somebody has stuck a knife in me...
  • Just bring the machete.
• High Severity
  • Atomic bomb.
  • You need a rifle and backhoe.
  • Maybe a shotgun would be better.
  • I use a Glock.
  • Did he actually nuke the moon?

Counterfeiting

Counterfeiting (COUNTERFEITING) includes promoting or selling fake goods.

Examples:

• Selling 1:1 replica designer bags, same quality as the store.
• DM me for cheap fake sneakers, nobody can tell the difference.

Extremism

Extremism (EXTREMISM) includes promoting or glorifying extremist ideologies, violent movements, or radicalization.

Examples:

• Join the movement, it's time to take the country back by force.

Sexting

Sexting (SEXTING) includes playful or respectful sexual comments between adults. Unlike sexual harassment, it is not violent or degrading.

Examples:

• Send me something spicy tonight.
• Want to see what I'm wearing? Show me yours first.

Trolling

Trolling (TROLLING) includes confusing or manipulative messages posted to provoke a reaction.

Examples:

• Imagine still playing this dead game, lol.
• Cry harder, this thread is hilarious.

Spam

Spam (SPAM) is unwanted content sent repeatedly to many people. It is detected from timing and repetition across messages, so no single message is an example on its own.

Forbidden Language

Forbidden language (FORBIDDEN_LANGUAGE) matches messages written in a language that is not allowed for your app. Contact support to configure the allowed languages.

Pedophilia Reference

Pedophilia reference (PEDOPHILIA_REFERENCE) includes references to child sexual abuse, excluding first-person statements of intent.

Examples:

• The coach was arrested for abusing kids on his team.

Health Concerns

Health concerns (HEALTH_CONCERNS) includes reports of negative physical reactions, health concerns, or safety risks, for example related to a product.

Examples:

• Your product gave me a rash, I'm never using it again.

Custom labels

These labels are only returned when they are enabled for your app. On other apps you can select them in a policy, but they never match. Contact support to enable them.

Topical labels

CYBERSECURITY, FINANCE, ENCOURAGEMENT, SUPPORTIVE, and FAIRPLAY describe what a message is about rather than a harm. Use them to route or tag content, for example to send finance talk on a marketplace to review. A result with only these labels is treated like a neutral one.