Create Evaluator API

API Endpoint

Authentication

Example Request Body

Request Parameters

Response Fields

Example Response

Response

Code Examples

Learn how to create evaluators for your agents

Creating Evaluator

Vocera

Testing for AI Voice Agents. Launch in minutes not weeks by ensuring your agents deliver a seamless experience in every conversational scenario

Introduction

Explore our video tutorials to master AI voice agent testing and integration with Vocera.

Video Tutorials

Learn how to generate embed token for embedding your agent's result in your website

Generating Embed Token

Learn how to refresh expired embed tokens for your agent's continuous operation

Refreshing Expired Token

Documentation

API Reference

Dashboard

Support

Overview

Metrics are used to understand and evaluate the performance of your AI Voice Agent

Pre-Defined Metrics

Personalities in Vocera define how your AI agents behave, communicate, and interact with users. They encompass the agent's tone, knowledge base, response style, and voice characteristics, allowing you to create unique and purposeful AI experiences.

Creating Custom Personalities

Learn how to quickly create and test voice agents on Vocera without requiring any external API keys or integrations.

Custom Agent

This guide will walk you through the process of setting up and testing voice agents using Retell integration in Vocera. When creating a new agent, you can select Retell as your provider from the available options in the agent creation interface.

Retell Agents

This guide will walk you through the process of setting up and testing voice agents using VAPI integration in Vocera. When creating a new agent, you can select VAPI as your provider from the available options in the agent creation interface.

VAPI Agents

This guide walks you through the process of testing outbound calls using evaluators. You'll learn how to retrieve evaluators with phone numbers and execute outbound call tests.

Testing Outbound Calls

To begin using observability features, you will need a Vocera account. If you do not have access, please contact [tarush@vocera.ai](mailto:tarush@vocera.ai).

Custom Observability

Monitor and analyze your VAPI-based voice agents

VAPI Observability

Monitor and analyze your Retell-based voice agents

Retell Observability

Metric Lab is a tool that allows you to create and manage custom metrics for your AI agents.

Create a new call log entry.

This endpoint allows users to create a call log by providing necessary data in the request body.
Requires a valid API key in the headers.

Observing Calls

Manage external call logs.

This view provides endpoints to retrieve, delete, and list call logs for external users.
A call log represents a record of a communication event, including details such as the agent involved,
the call ID, and the outcome of the call. This information is crucial for monitoring and evaluating
the performance of agents and the quality of interactions.

Required only when call_logs is “all” OR is number of first N calls. Only either of agent_id or assistant_id is required

Getting a Call Log

Getting Call Logs

Reevaluating Calls

Deleting Call Logs

External API view set for managing AI agents.
This view set provides endpoints to create, retrieve, update, and delete
AI agents associated with a specific organization, authenticated via API key.

Getting Agents

Creating an Agent

Updating an Agent

Deleting an Agent

Getting an Agent

External API view set for managing metrics.
This view set provides endpoints to create, retrieve, update, and delete
metrics associated with AI agents within a specific organization.

Getting Metrics

Creating a Metric

Generating Metrics

Creating Metrics in Bulk

Getting a Metric

Updating a Metric

External API view set for managing critical metric scenarios.
This view set provides endpoints to update and manage critical metric
scenarios associated with a specific organization.

Get Critical Metric Scenarios

Updating a Critical Metric Scenario

Deleting a Metric

External API view set for managing scenarios.
This view set provides endpoints to list, create, and manage scenarios
associated with a specific organization, authenticated via API key.

Getting Evaluators

Creating an Evaluator

Getting an Evaluator

Deleting an Evaluator

Updating an Evaluator

Running Evaluators

Creating an Evaluator from Transcript

External API view set for retrieving results.
This view set allows users to access results associated with their
organization, providing endpoints to list and retrieve detailed results.

Get Results

Get an Result

Reevaluate Metrics

Rerun Results

External API view set for managing runs.
This view set provides endpoints to retrieve and manage runs associated with a specific organization.

Getting Runs with IDs

Runs Improve Prompt

External API view set for retrieving personalities.
This view set allows users to access predefined personalities that are
not associated with any organization.

Get Personalities

External API view set for retrieving predefined metrics.
This view set allows users to access predefined metrics that are
not associated with any organization.

Get Predefined Metrics

Guide to using the Vocera API for observing calls

Learn how to retrieve detailed information about specific calls

Getting Call

Learn how to reevaluate metrics for existing calls

Deleting Calls

Creating Agent

Listing Agents

Learn how to update existing AI agents configurations

Updating Agent

Deleting Agent

Learn how to create metrics for your agents

Creating Metrics

Learn how to automatically generate metrics for your agents

Learn how to create multiple metrics at once for your agents

Creating Bulk Metrics

Learn how to retrieve a list of metrics for your agents

Listing Metrics

Learn how to retrieve a specific metric for your agents

Learn how to update existing metrics for your agents

Updating Metrics

Learn how to update the priority of critical workflow metric scenarios

Updating Critical Workflow Metric Scenario

Learn how to delete existing metrics from your agents

Deleting Metrics

Learn how to automatically generate evaluators for your agents

Generating Evaluators

Learn how to update existing evaluators configurations

Updating Evaluator

Learn how to retrieve a list of evaluators for your agents

Listing Evaluators

Learn how to retrieve a specific evaluator

Getting Evaluator

Learn how to run multiple evaluators simultaneously

Listing results

Getting result

Rerunning results

Learn how to reevaluate metrics for specific runs

Reevaluating metrics

Learn how to retrieve multiple runs in bulk

Learn how to retrieve a list of personalities

Listing Personalities

Learn how to retrieve a list of predefined metrics

Listing Predefined Metrics

Learn how to retrieve a list of inbound phone numbers

Listing Inbound Phone Numbers

LLM Websocket Server Example

Learn how to embed Vocera agent overview in your application using our embed widget. For token management details, see our [Token Generation](/embed/generate-token) and [Token Refresh](/embed/refreshing-expired-token) guides.

Overview Embedding

Learn how to embed Vocera agent calls in your application using our embed widget. For token management details, see our [Token Generation](/embed/generate-token) and [Token Refresh](/embed/refreshing-expired-token) guides.

Calls Embedding

Learn how to embed Vocera agent results in your application using our embed widget. For token management details, see our [Token Generation](/embed/generate-token) and [Token Refresh](/embed/refreshing-expired-token) guides.

Parameter	Type	Required	Description
`agent`	integer	Yes*	ID of the agent
`assistant_id`	string	Yes*	The assistant ID associated with agent
`name`	string	Yes	Name of the evaluator
`personality`	integer	Yes	ID of the personality to use
`instructions`	string	Yes	Instructions for the evaluator
`agent`	integer	Yes*	ID of the agent
`assistant_id`	string	Yes*	The assistant ID associated with agent
`expected_outcome_prompt`	string	No	Prompt to evaluate expected outcome
`metrics`	array\|integer\|string	No	Metrics to set to scenario, can be array of metric IDs or first N metrics or “all” to add all or “predefined” to add all predefined

Field	Type	Description
`id`	integer	Unique identifier for the evaluator
`name`	string	Name of the evaluator
`agent`	integer	ID of the associated agent
`personality`	integer	ID of the personality used
`personality_name`	string	Human-readable name of the personality
`runs`	array	List of evaluator runs (empty for new evaluators)
`instructions`	string	Evaluator instructions
`expected_outcome_prompt`	string	Prompt to evaluate expected outcome
`metrics`	array	Array of metric IDs

Get Started

Key Concepts

Guides

API Documentation

Embedding Documentation

Creating Evaluator

Create Evaluator API

API Endpoint

Authentication

Request Parameters

Example Request Body

Response

Response Fields

Example Response

Code Examples

Get Started

Key Concepts

Guides

API Documentation

Embedding Documentation

​Create Evaluator API

​API Endpoint

​Authentication

​Request Parameters

​Example Request Body

​Response

​Response Fields

​Example Response

​Code Examples

Create Evaluator API

API Endpoint

Authentication

Request Parameters

Example Request Body

Response

Response Fields

Example Response

Code Examples