Overview

​

Metric Definition

Metrics are essential tools for evaluating AI Voice Agents, acting as judges that assess call quality. They analyze call transcripts and sometimes recordings to provide scores based on specific criteria. This evaluation covers aspects like clarity, relevance, and instruction following. By assigning scores, metrics help identify strengths and areas for improvement. Regular analysis of these metrics enables organizations to make data-driven decisions, enhancing voice agent performance and user satisfaction. Ultimately, metrics guide the optimization of strategies, ensuring that voice agents effectively meet user needs and organizational goals.

​

How to Use

​

Adding Metrics

From the metrics section on Vocera’s dashboard a user can add/define metrics. Metrics can be broadly categorised as:

1. Pre-defined Metrics: These are generic metrics such as CSAT, pitch, latency, and interruption count. These are defined by Vocera. You can read more about each pre-defined metric here.

2. Instruction Following Metric: This metric is part of the pre-defined metrics and is used to automatically find and categorize issues in the call based on agent description. This saves you from having to define 100s of metrics. Read More

3. Custom Metrics (User-defined): These are metrics defined by the user and can be of types such as Boolean, Rating, and Enum.

​

Running Metrics

• Testing: Each evaluator has a list of metrics attached to it. When running simulations, the metric linked to each evaluator is computed.
  Note: Off the attached metrics, we pick which metric to run at the end of the call depending on how the call went. This behavior can be overridden in the settings by de-selecting Evaluate Relevant Metrics option.
• Observability: We pick relevant metric to run at the end of the call depending on how the call went