Skip to main content

Custom API Moderation

CometChat allows you to integrate your own moderation logic using a Custom API. With this feature, you can define a webhook URL in the List Configuration, where CometChat will send messages for moderation along with relevant context from the conversation (if provided in settings).

How It Works​

  1. When a user sends a message, CometChat retrieves the webhook URL from the configured List.
  2. The message, along with previous conversation messages (if a context window is set in settings), is sent to the webhook.
  3. The webhook (your external API) processes the data using your custom moderation logic.
  4. The webhook responds with a structured decision containing details about the moderation outcome.
  5. CometChat processes the response and applies the moderation decision in real-time.

This approach gives you complete control over moderation, allowing you to implement custom filtering, AI-based analysis, or any other logic on your own servers.

Integration​

Step 1: Configure Custom API Settings​

  1. Login to the CometChat Dashboard

  2. Navigate to Moderation Settings

    • Go to Moderation → Settings in the left-hand menu.

  3. Open Custom API Settings Tab

    • Click on the Custom API tab within the Moderation Settings.

  4. Fill in the Custom API Configuration

    • Set Action on API Error

      • Define how the system should respond if the Custom API is unavailable (e.g., "Allow message" or "Block message").

    • Set Context Window

      • Specify the number of previous messages in a conversation that will be used for context.

  5. Click Save Settings

Step 2: Enable Custom API Moderation​

  1. Navigate to Moderation → Rules.
  2. Click "Create New Rule".
  3. Select Custom API as the moderation type.
  4. The rule you create should be of type "Text Contains" or "Image Contains".
  5. Save the rule.

Payload Sent to Webhook​

When a message is sent, CometChat invokes your webhook with a payload that includes:

  • he latest message (the one just sent) — provided in full detail (entire message object)

  • The previous messages — provided as plain text only, for context (based on the context window setting)

This structure allows you to apply moderation logic to the current message while considering its surrounding context.

{
  "contextMessages": [
    {
      "cometchat-uid-1": "Hello there!"
    },
    {
      "cometchat-uid-2": "Hey, how are you?"
    },
    {
      "cometchat-uid-1": "Let's team up."
    },
    {
      "cometchat-uid-2": {
        "id": "30431",
        "muid": "_r49ocm6oj",
        "conversationId": "cometchat-uid-1_user_cometchat-uid-2",
        "sender": "cometchat-uid-1",
        "receiverType": "user",
        "receiver": "cometchat-uid-2",
        "category": "message",
        "type": "text",
        "data": {
          "text": "ok",
          "resource": "WEB-4_0_10-04aecbad-8354-4fc8-98df-d0119e1a9539-1747717193939",
          "entities": {
            "sender": {
              "entity": {
                "uid": "cometchat-uid-1",
                "name": "Andrew Joseph",
                "avatar": "https://data-us.cometchat-staging.com/assets/images/avatars/andrewjoseph.png",
                "status": "available",
                "role": "default",
                "lastActiveAt": 1747717203
              },
              "entityType": "user"
            },
            "receiver": {
              "entity": {
                "uid": "cometchat-uid-2",
                "name": "George Alan",
                "avatar": "https://data-us.cometchat-staging.com/assets/images/avatars/georgealan.png",
                "status": "offline",
                "role": "default",
                "lastActiveAt": 1721138868,
                "conversationId": "cometchat-uid-1_user_cometchat-uid-2"
              },
              "entityType": "user"
            }
          },
          "moderation": {
            "status": "pending"
          }
        },
        "sentAt": 1747717214,
        "updatedAt": 1747717214,
      }
    }
  ]
}

Webhook Response Format​

The webhook should return a response in the following format:

 {
  isMatchingCondition: true,  // True if the message violates the rule
  confidence: 0.95,           // Confidence score of the decision
  reason: "Contains hate speech" // Reason for flagging
}