Lists Management

Overview

The Lists Management endpoints in the Moderation Service API provide essential tools for creating and managing lists of keywords or regex patterns that are crucial for effective message moderation. These endpoints enable app owners and collaborators to define specific terms, phrases, or patterns that, when detected in user-generated content, trigger moderation actions. The next section provides a detailed elaboration of the capabilities offered.

To begin managing lists:

• Login to your CometChat dashboard and choose your app.
• Navigate to Moderation > Settings in the left-hand menu.
• Select the Lists tab.

Default Lists

Default lists are predefined lists of words, patterns and sentences that are readily available for use on your platform. Here are the standard default lists available:

Profane Words

Our default list is a comprehensive compilation of predefined profane words and phrases. This list is designed to enhance message moderation efforts by automatically identifying and flagging inappropriate language.

Platform Circumvention

The Platform Cicurvention list contains a curated set of sentences and words designed to identify attempts to circumvent platform rules and policies. These phrases are used by the AI Platform Circumvention Rule to detect and prevent efforts aimed at bypassing restrictions, ensuring compliance and maintaining platform integrity.

Spam Detection

The Default Spam Detection List identifies repetitive or irrelevant messages promoting products, services, or schemes without user consent. It helps filter out unwanted content, including bulk messages, phishing attempts, and fraudulent offers, ensuring a cleaner and more secure communication experience.

Scam Detection

The Default Scam Detection List includes messages crafted to deceive users by creating a sense of urgency, promising false rewards, or impersonating trusted entities. These messages often aim to manipulate users into sharing personal information, making payments, or clicking on malicious links. The list helps identify and block scams, protecting users from fraud, phishing attempts, and other deceptive practices.

Fraud or Scam Indicators Prompt

The Fraud or Scam Indicators list is designed to detect manipulated images used for fraudulent or deceptive activities. It helps flag fake documents, counterfeit products, and misleading visuals that could be used to scam users or spread misinformation.

Terrorism or Extremist Promotion Prompt

The Terrorism or Extremist Promotion list identifies imagery that endorses terrorism, violent extremism, or radical ideologies. It helps prevent the spread of extremist propaganda, recruitment materials, and content that incites violence.

Minor Safety and Exploitation Prompt

The Minor Safety and Exploitation list is used to detect sexualized or exploitative imagery of minors. It helps prevent child abuse, grooming, and the sharing of harmful content, ensuring compliance with child protection policies.

Privacy or Personal Data Prompt

The Privacy or Personal Data list flags images that expose sensitive or private information, such as identification documents, financial details, or personal records. This helps prevent identity theft, unauthorized data leaks, and privacy violations.

Graphic Violence or Gore Prompt

The Graphic Violence or Gore list identifies violent or gory imagery, including depictions of severe injuries, crime scenes, or graphic deaths. It helps limit exposure to disturbing content and ensures a safer viewing experience.

Explicit or Sexual Content Prompt

The Explicit or Sexual Content list is designed to detect nudity, sexually explicit imagery, or highly suggestive content. It helps enforce platform guidelines by filtering out inappropriate material.

Hate or Harassment Prompt

The Hate or Harassment list flags imagery containing hateful symbols, offensive gestures, or harassment. It helps identify and prevent content that promotes discrimination, hate speech, or targeted abuse.

Hate and Harassment Prompt

The Hate and Harassment list detects messages that contain hate speech, threats, slurs, or harassment directed at individuals or groups. It helps create a respectful and safe online environment by preventing abusive behavior.

Explicit or Inappropriate Content Prompt

The Explicit or Inappropriate Content list identifies text that includes explicit sexual descriptions, extreme violence, or other unsuitable material. It helps ensure compliance with content policies and maintains platform integrity.

Impersonation or Fraud Prompt

The Impersonation or Fraud list detects deceptive attempts to impersonate individuals, businesses, or organizations. It helps prevent identity theft, scam attempts, and fraudulent activities.

Non-Consensual Sexual Content or Exploitation Prompt

The Non-Consensual Sexual Content or Exploitation list flags messages that depict or encourage non-consensual sexual acts, grooming, or coercion. It helps protect users from exploitation and ensures adherence to safety policies.

Privacy and Sensitive Info Prompt

The Privacy and Sensitive Info list identifies messages that share personal or sensitive information without consent. It helps protect user privacy by preventing unauthorized data exposure.

Self-Harm or Suicidal Content Prompt

The Self-Harm or Suicidal Content list detects messages indicating self-harm, suicidal thoughts, or encouragement of self-injury. It helps enable early intervention and support mental health safety.

Spam and Scam Prompt

The Spam and Scam list identifies spam messages, phishing attempts, and fraudulent schemes. It helps filter out unwanted content, including bulk messages and misleading offers, ensuring a cleaner and more secure communication environment.

Violent or Terroristic Threats Prompt

The Violent or Terroristic Threats list detects content that promotes violence, terrorism, or extremist actions. It helps prevent harmful speech, glorification of violence, and threats against individuals or groups.

Managing Lists

Create List

Allows you to define new moderation lists specifying the words or patterns under which text or custom messages should be blocked.

Creating a new list from the dashboard:

1. Click the Add button within the Lists tab.
2. Create the list by saving the following details:
   • Name: Descriptive name for the moderation list.
   • ID: The unique identifier of the list.
   • Category: Choose the type for list, either 'word', 'pattern' or 'sentence similarity'.
   • Description: Detailed explanation of the list.
   • Your Source type for lists could be either words, patterns or sentences, separated by a comma or a CSV file.
3. Save

You can also set this up from your end using the Create Moderation List REST API.

Fetch All Lists

Fetches the details of existing list lists.

You can also set this up from your end using the List Moderation Lists REST API.

Get List

Fetches the details of an existing list. You can set this up from your end using the Get Moderation List REST API.

Update List

Allows you to update existing lists, which includes modifying the list name, category, and individual words or patterns within the list.

Updating a list from the dashboard:

1. Click on "Edit" in the action menu of the List you want to update.
2. Update the list by saving the following details:
   • Name: Descriptive name for the moderation list.
   • Category: Choose the type for List, either 'word', 'pattern' or 'sentence similarity'.
   • Description: Detailed explanation of the list.
   • Your Source type for list could be either words, patterns or sentences separated by a comma or a CSV file.
3. Save

You can also set this up from your end using the Update Moderation List REST API.

Delete List

Allows for the removal of lists from the system that are no longer needed.

Deleting a list from the dashboard:

• Click "Delete" in the action menu of the list you want to remove, then confirm.

You can also set this up from your end using the Delete Moderation List REST API.