> ## Documentation Index
> Fetch the complete documentation index at: https://www.cometchat.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

> Scrapes a single webpage to extract its content for the knowledge base. Use this endpoint to add a specific page without crawling an entire website.  **Extraction:** Returns the page title, descriptio

# Scrape Single Page

For the complete error reference, see [Error Guide](/rest-api/ai-agents-apis/error-codes).


## OpenAPI

````yaml post /ai-agents/agent-builder/knowledge-base/website/individual-page
openapi: 3.0.0
info:
  title: AI Agents APIs
  description: API reference for CometChat AI Agents service
  version: '1.0'
servers:
  - url: https://{appId}.api-{region}.cometchat.io/v3
    variables:
      appId:
        default: appId
        description: (Required) App ID
      region:
        enum:
          - us
          - eu
          - in
        default: us
        description: Select Region
security: []
tags:
  - name: ai-agent
    description: ''
paths:
  /ai-agents/agent-builder/knowledge-base/website/individual-page:
    post:
      tags:
        - crawl-web-pages
      summary: Scrape Single Page
      description: >-
        Scrapes a single webpage to extract its content for the knowledge base.
        Use this endpoint to add a specific page without crawling an entire
        website.


        **Extraction:** Returns the page title, description, and full content.
      operationId: CrawlWebPagesController_scrapePage
      parameters: []
      requestBody:
        required: true
        description: Single page scraping configuration
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/ScrapeIndividualPageDto'
            examples:
              basic:
                summary: Basic page scrape
                description: Simple page content extraction
                value:
                  url: https://docs.example.com/getting-started
                  timeout: 30000
                  format: markdown
      responses:
        '200':
          description: Page scraped successfully
          content:
            application/json:
              schema:
                type: object
                properties:
                  success:
                    type: boolean
                    example: true
                  message:
                    type: string
                    example: Page scraped successfully
                  data:
                    type: object
                    properties:
                      url:
                        type: string
                        example: https://docs.example.com/getting-started
                      title:
                        type: string
                        example: Getting Started Guide
                      description:
                        type: string
                        example: Learn how to get started with our platform
                      content:
                        type: string
                        example: |-
                          # Getting Started

                          Welcome to our platform...
                      metadata:
                        type: object
                        properties:
                          og:
                            type: object
                          twitter:
                            type: object
                          canonical:
                            type: string
                          language:
                            type: string
                      scraper:
                        type: string
                        example: Firecrawl
                      crawlDuration:
                        type: number
                        example: 2500
                      contentLength:
                        type: number
                        example: 1024
                      timestamp:
                        type: string
                        example: '2025-12-05T17:30:00Z'
              example:
                success: true
                message: Page scraped successfully
                data:
                  url: https://docs.example.com/getting-started
                  title: Getting Started Guide
                  description: Learn how to get started with our platform
                  content: |-
                    # Getting Started

                    Welcome to our platform...
                  metadata:
                    og: {}
                    twitter: {}
                    canonical: ''
                    language: en
                  scraper: Firecrawl
                  crawlDuration: 2500
                  contentLength: 1024
                  timestamp: '2025-12-05T17:30:00Z'
      security:
        - apiKey: []
components:
  schemas:
    ScrapeIndividualPageDto:
      type: object
      properties:
        url:
          type: string
          description: Target page URL to crawl and extract content from
          example: https://docs.example.com/getting-started
        timeout:
          type: number
          description: Request timeout in milliseconds
          minimum: 5000
          maximum: 120000
          default: 30000
          example: 30000
        format:
          type: string
          description: Extract content in specific format
          enum:
            - markdown
            - html
            - text
          default: markdown
          example: markdown
      required:
        - url
  securitySchemes:
    apiKey:
      type: apiKey
      description: API Key (i.e. Rest API Key from the Dashboard).
      name: apikey
      in: header

````