Definition

What Is a Profanity Filter? Word Lists vs AI Moderation

A list of bad words gets you started and then blocks a town in England. Here is how profanity filters work, where word lists break, and what AI moderation adds.

Word-list filter vs AI moderation

Word-list filter

AI moderation

How it decides

Matches text against a list of words or patterns

A model scores the whole message for harm

Catches

Known words and simple misspellings

Insults, threats and harassment with no swear words

False positives

Common without word boundaries (Scunthorpe)

Fewer, but tuned by a confidence threshold

Speed

Microseconds, runs anywhere

Adds a model call before delivery

Languages

One list per language you maintain

Depends on the model; check coverage

Explainability

Easy: this word matched

Harder: a score and a category

Best for

Masking swear words, banned terms, usernames

Toxicity, hate and harassment in open chat

What is a profanity filter?

A profanity filter checks text people write (chat messages, comments, usernames) for offensive words and acts before anyone else sees it. It usually does one of three things:

  • Mask: replace the word with asterisks and deliver the rest of the message.

  • Block: stop the message and tell the sender.

  • Flag: deliver or hold the message and send it to a human moderator.

Profanity is the narrowest slice of content moderation. A message can be hostile without a single swear word, and a friendly message can be full of them. That gap is why most chat apps now pair a word list with AI moderation.

How profanity filters work

1. Normalize

People dodge filters, so clean the text first: lowercase it, strip accents, map leetspeak (3 to e, @ to a), collapse repeated letters and map look-alike characters from other scripts, such as Cyrillic letters that look Latin.

2. Match

Compare whole words against a block list, with an allowlist for exceptions. Regular expressions catch variants, such as a word with optional letters.

3. Score (optional)

An AI model rates the message for toxicity, harassment or hate, and returns a category and a confidence score. You pick the threshold.

4. Act

Mask, block or flag based on what matched and how confident the model is. Log the decision so moderators can review it and users can appeal.

The Scunthorpe problem

In 1996, AOL's filter stopped people in Scunthorpe, England, from creating accounts, because the town's name contains a slur. Filters that replace "ass" have turned "classic" into "clbuttic". These false positives are called the Scunthorpe problem.

The cause is matching inside words. Fixes:

  • Match whole words after splitting on spaces and punctuation.

  • Keep an allowlist for place names, surnames and technical terms.

  • Mask the matched word only, never the surrounding text.

  • Let an AI model handle the cases where context decides.

There's a trade-off. Whole-word matching misses "h e c k" and words glued together. Filters that try to catch everything end up blocking everyone.

Rule of thumb: use a word list for words you never want shown, and an AI model for behavior you never want tolerated. Neither does the other's job well.

Example: a whole-word filter with masking

const BLOCKED = new Set(['darn', 'heck']); // placeholders: load your real list
const LEET = { 0: 'o', 1: 'i', 3: 'e', 4: 'a', 5: 's', 7: 't', '@': 'a', $: 's' };

function variants(token) {
  const base = token
    .normalize('NFKD').replace(/[\u0300-\u036f]/g, '')  // strip accents
    .toLowerCase()
    .replace(/[0-9@$]/g, (c) => LEET[c] ?? c);
  return [base, base.replace(/(.)\1+/g, '$1')];         // "heeeck" -> "heck"
}

function mask(text) {
  // Check whole tokens only, so "Scunthorpe" is never searched inside.
  return text.replace(/[\p{L}\p{N}@$]+/gu, (token) =>
    variants(token).some((v) => BLOCKED.has(v)) ? '*'.repeat(token.length) : token
  );
}

mask('What the h3ck, Scunthorpe?'); // "What the ****, Scunthorpe?"

Where profanity filters are used

  • Kids, education and family apps: strict lists and blocking, since a single slip is a support ticket.

  • Gaming and live event chat: fast masking for thousands of messages a minute, with AI moderation for harassment.

  • Marketplaces and dating: profanity rules alongside filters for phone numbers and emails that move users off the platform.

  • Customer support: protect agents from abusive messages and flag the conversation.

  • Usernames and profiles: check display names once, at signup, where false positives are easy to fix.

  • Calls and live captions: run the same filter on transcripts and captions from voice and video calls.

Limits and pitfalls

  • Evasion never stops. Spaces, emoji, images of text and new slang get past lists. Update them and watch what gets flagged.

  • Context changes meaning. "This game is f***ing great" isn't an attack. Swear-word counts are a poor proxy for harm.

  • Language and culture. A word that is offensive in one language or region is normal in another. Reclaimed terms make this harder.

  • Latency. AI checks before delivery add time. Holding a message as pending until it's approved keeps harmful content off screen, at the cost of a short delay.

  • Over-masking annoys people. Tell senders what was filtered, and give moderators a way to approve false positives.

Profanity filtering and moderation in CometChat

If you're building chat into your app, CometChat moderation checks messages as they're sent, before other users see them.

Rules for profanity, toxicity, contact details and media

CometChat's AI Moderation runs rules you set in the dashboard or through the Moderation API. A default profanity rule blocks offensive language using a predefined keyword list, and you can add your own keyword and regex lists. AI rules cover toxicity, hate and harassment, spam, scams and more, plus filters for phone numbers and email addresses and checks on images and video. Each rule blocks a message or flags it for review. In the SDK, a message is PENDING until moderation marks it APPROVED or DISAPPROVED through onMessageModerated. We covered the older Profanity Filter and Data Masking extensions, which mask words as asterisks, in this product update. If you still use them, disable them before creating moderation rules so messages aren't processed twice.

Read the moderation docs · Start building free

Frequently asked questions

How does a profanity filter work?

It normalizes the text, compares words against a block list, and then masks, blocks or flags the message. Many filters also send the message to an AI model that scores it for toxicity or harassment.

What is the Scunthorpe problem?

It's when a filter blocks innocent text because an offensive string appears inside a longer word, like the English town of Scunthorpe. Matching whole words and keeping an allowlist prevents most cases.

Is AI better than a word list for filtering profanity?

For harassment and hate, yes, because it reads context. For masking specific words, a list is faster and easier to explain. Most chat apps use both.

Should a profanity filter mask or block messages?

Mask when the rest of the message is fine and the audience just shouldn't see the word. Block or flag when the message itself is abusive.

Can people get around profanity filters?

Yes, with spacing, symbols, look-alike characters and new slang. Normalizing text, updating lists and adding AI moderation and user reporting keeps up with most of it.

Related terms