Give your chats superpowers: let an agent trigger visual effects and UI actions in the browser (like confetti) by returning safe, structured tool calls that your frontend handles.

What You’ll Build

  • A Mastra agent that can request frontend UI actions (tools) like confetti.
  • A simple tool registry on the client that runs actions when the agent returns a tool call.
  • An API endpoint to chat with the agent and receive tool instructions.
  • Integration into CometChat chats.

Prerequisites

Repo: mastra-frontend-actions-agent README: Project README Scripts: package.json
  • OpenAI API key in .env as OPENAI_API_KEY.
  • A CometChat app.

Environment
Agent

How it works

Tools Server Key components (source-linked below): the agent, the confetti tool, server entry, and a sample widget page.
Scripts: package.json
README: Project README
See the sample: widget/index.html

Step 1 - Create the Agent

src/mastra/agents/celebration-agent.ts (view in repo): Checklist for the agent:
  • Set name to something like “celebration” so the API path is /api/agents/celebration/*.
  • Describe when to use the confetti tool (e.g., celebratory moments).
  • Keep normal chat responses short and friendly.
  • Ensure tool return format is structured and safe.

Step 2 - Register the Agent in Mastra

src/mastra/index.ts (view in repo):
  • Register the agent with key “celebration” → API path /api/agents/celebration/*.
  • Keep config and logger settings as per the repo README.

Step 3 - Run the Agent

Dev scripts & server details are in your repo: Expected local API base: http://localhost:4111/api
1

Install dependencies

Use the repo scripts to install dependencies.
2

Start the dev server

Run the local Mastra server as per the README.
3

Ask the agent

POST to /api/agents/celebration/generate and inspect tool calls in the response.
API endpoints exposed by this example:
  • POST /api/agents/celebration/generate — chat with the agent and receive frontend tool instructions

Step 4 - Handle frontend tools

You’ll need a small client-side handler that:
  • Reads tool calls from the agent’s response (e.g., { id: "confetti", args: {...} }).
  • Maps the tool ID to a function (e.g., confetti()), then runs it.
  • Handles unknown tools safely (ignore or log).
See the sample: widget/index.html

Step 5 - Deploy the API

Ensure your public route: /api/agents/celebration/generate is reachable.

Step 6 - Configure in CometChat

1

Open Dashboard

2

Navigate

Go to your App → AI Agents.
3

Add agent

Set Provider=Mastra, Agent ID=celebration, Deployment URL=your public generate endpoint.
4

(Optional) Tools

If your UI uses frontend tools, ensure your app handles tool invocations.
5

Enable

Save and ensure the agent toggle shows Enabled.
For more on CometChat AI Agents, see the docs: Overview · Instructions · Custom agents

Step 7 - Customize in Chat Builder

1

Open variant

From AI Agents click the variant (or Get Started) to enter Chat Builder.
2

Customize & Deploy

Select Customize and Deploy.
3

Adjust settings

Theme, layout, features; ensure the Frontend Actions agent is attached.
4

Preview

Use live preview to validate messages and any tool triggers.

Step 8 - Integrate

Once your Frontend Actions Agent is configured, you can integrate it into your app using the CometChat No Code - Widget:
Note: The Frontend Actions agent you connected in earlier steps is already part of the exported configuration, so your end-users will chat with that agent immediately.

Step 9 - Test Your Setup

1

API generates response

POST to /api/agents/celebration/generate returns a message, possibly with a tool call.
2

Agent listed

/api/agents includes “celebration”.
3

Tool action works

Your UI runs the confetti tool when the agent requests it.
curl -X POST http://localhost:4111/api/agents/celebration/generate \
  -H "Content-Type: application/json" \
  -d '{
    "messages": [
      { "role": "user", "content": "@agent celebrate our product launch with confetti" }
    ]
  }'

Security & production checklist

  • Protect endpoints with auth (API key/JWT) and restrict CORS to trusted origins.
  • Add rate limiting and request size limits to the generate route.
  • Validate inputs and sanitize logs/responses server-side.
  • Keep your OpenAI key in server-side env only; never expose it to the client.

Troubleshooting

  • No tool runs: ensure the frontend reads tool calls from the response and maps IDs to functions.
  • Agent not found: confirm the server registers the agent with key celebration.
  • 401/403: verify auth headers and CORS for your deployment.

Next Steps

  • Add more tools (e.g., toast, highlight, modal) and map them in your UI.
  • Gate tool usage by roles or contexts.
  • Log tool invocations for auditing and tuning.