Running using Voice Calls

Run multiple evaluators at once and receive their execution details.

API Endpont

Method	Endpoint
`POST`	`https://new-prod.vocera.ai/test_framework/v1/scenarios-external/run_scenarios/`

Authentication

Include your API key in the request headers:

Header	Description
`X-VOCERA-API-KEY`	Your API key obtained from the dashboard

Request Parameters

Parameter	Type	Required	Description
`agent_id`	integer	Yes*	Unique identifier of the agent
`assistant_id`	integer	Yes*	The assistant ID associated with agent
`scenarios`	array\|string\|integer	Yes	Array of evaluator IDs to run OR `"all"` to run all OR number to run first N evaluators. Either scenarios or tags must be provided.
`tags`	array[string]	Yes	List of tags to filter scenarios to run. Either scenarios or tags must be provided.
`freq`	integer	No	Number of times to run each scenario (default: 1)
`name`	string	No	Label text for the result

* Required only when scenarios is "all" OR is number of first N evaluators. Only either of agent_id or assistant_id is required

Example Request Body

Running evaluators using ids

{

    "scenarios": [201, 187],

    "freq": 5,

    "name": "label text" // optional

}

{

    "tags": ["short", "long"],

    "freq": 5,

    "name": "label text" // optional

}

Running all evaluators

{

    "agent_id": 1,

    "scenarios": "all",

    "freq": 5

}

Running first N evaluators

{

    "agent_id": 1,

    "scenarios": 10,

    "freq": 5

}

Response

A successful request returns details about created result.

Response Fields

Field	Type	Description
`id`	integer	The result ID
`agent`	integer	ID of associated agent
`status`	string	Status of the result, can be one of: • `running` - Initial state when evaluator starts • `in_progress` - Call is currently executing • `completed` - Call has finished successfully • `failed` - Call encountered an error • `pending` - Testing agent is waiting to be called
`success_rate`	float	Percentage of runs that passed
`run_as_text`	boolean	If executed as text (llm websocket) or not
`runs`	array	List of individual scenario runs
`created_at`	string	ISO 8601 timestamp of when the result was created

Run Fields

Each object in the runs array contains:

Field	Type	Description
`id`	integer	Unique identifier for the individual run
`scenario`	integer	ID of the scenario being run
`number`	string\|null	Phone number assigned to the run if outbound call

Example Response

{

    "id": 167,

    "agent": 1,

    "status": "pending",

    "success_rate": 0.0,

    "run_as_text": false,

    "runs": [

        {

            "id": 274,

            "scenario": 1,

            "number": "+11234567890"

        },

        {

            "id": 273,

            "scenario": 2,

            "number": "+11234567890"

        }

    ],

    "created_at": "2025-02-25T21:00:01.990052Z"

}

Code Examples

# Running using evaluator IDs

curl -X POST https://new-prod.vocera.ai/test_framework/v1/scenarios-external/run_scenarios/ \

  -H "X-VOCERA-API-KEY: <your-api-key-here>" \

  -H "Content-Type: application/json" \

  -d '{

      "scenarios": [<evaluator-id-1>, <evaluator-id-2>, ...],

      "freq": 1

    }'

# Running all evaluators

curl -X POST https://new-prod.vocera.ai/test_framework/v1/scenarios-external/run_scenarios/ \

  -H "X-VOCERA-API-KEY: <your-api-key-here>" \

  -H "Content-Type: application/json" \

  -d '{

    "scenarios": "all",

    "agent_id": <agent-id>,

    "freq": 1

  }'

# Running first N evaluators

curl -X POST https://new-prod.vocera.ai/test_framework/v1/scenarios-external/run_scenarios/ \

  -H "X-VOCERA-API-KEY: <your-api-key-here>" \

  -H "Content-Type: application/json" \

  -d '{

    "scenarios": <number-of-evaluators>,

    "agent_id": <agent-id>,

    "freq": 1

  }'

Running using LLM Websocket

Run multiple evaluators at once using your LLM websocket and receive their execution details.

You need to add the url to your LLM websocket in your agent at Vocera. For demo websocket server you can use wss://echo.websocket.org or install using this repo.

Method	Endpoint
`POST`	`https://new-prod.vocera.ai/test_framework/v1/scenarios-external/run_scenarios_text/`

Authentication

Include your API key in the request headers:

Header	Description
`X-VOCERA-API-KEY`	Your API key obtained from the dashboard

Request Parameters

Parameter	Type	Required	Description
`agent_id`	integer	Yes*	Unique identifier of the agent
`assistant_id`	string	Yes*	The assistant ID associated with agent
`scenarios`	array\|string\|integer	Yes	Array of evaluator IDs to run OR `"all"` to run all OR number to run first N evaluators
`freq`	integer	No	Number of times to run each scenario (default: 1)

* Required only when scenarios is "all" OR is number of first N evaluators. Only either of agent_id or assistant_id is required

Example Request Body

Running evaluators using ids

{

    "scenarios": [201, 187],

    "freq": 5

}

Running all evaluators

{

    "agent_id": 1,

    "scenarios": "all",

    "freq": 5

}

Running first N evaluators

{

    "agent_id": 1,

    "scenarios": 10,

    "freq": 5

}

Response

A successful request returns details about the running evaluators.

Response Fields

Field	Type	Description
`id`	integer	The result ID
`agent`	integer	ID of associated agent
`status`	string	Status of the result, can be one of: • `running` - Initial state when evaluator starts • `in_progress` - Call is currently executing • `completed` - Call has finished successfully • `failed` - Call encountered an error • `pending` - Testing agent is waiting to be called
`success_rate`	float	Percentage of runs that passed
`run_as_text`	boolean	If executed as text (llm websocket) or not
`runs`	array	List of individual scenario runs
`created_at`	string	ISO 8601 timestamp of when the result was created

Run Fields

Each object in the runs array contains:

Field	Type	Description
`id`	integer	Unique identifier for the individual run
`scenario`	integer	ID of the scenario being run
`number`	integer\|null	Sequential number of the run (when freq > 1)

Example Response

{

    "id": 167,

    "agent": 1,

    "status": "running",

    "success_rate": 0.0,

    "run_as_text": false,

    "runs": [

        {

            "id": 274,

            "scenario": 1,

            "number": null

        },

        {

            "id": 273,

            "scenario": 2,

            "number": null

        }

    ],

    "created_at": "2025-02-25T21:00:01.990052Z"

}

WebSocket Message Format

Messages exchanged through the LLM WebSocket connection follow two main formats: normal messages and function calls.

Message Types

Normal Messages

Simple text messages exchanged between the client and server.

Field	Type	Description
`content`	string	The text message content

{

  "content": "Hello! How can I help you today?"

}

Function Messages

Messages related to function execution, including both calls and results.

Field	Type	Description
`role`	string	Message type: `Function Call` or `Function Call Result`
`data`	object	Function-specific information (see below)

Function Call Data Fields:

Field	Type	Description
`id`	string	Unique identifier for the function call
`name`	string	Name of the function being called
`arguments`	string	JSON-encoded function arguments

Function Result Data Fields:

Field	Type	Description
`id`	string	Matching identifier from the function call
`name`	string	Name of the executed function
`result`	string	Function execution result or error message

Examples

Function Call:

{

    "role": "Function Call",

    "data": {

        "id": "call_5otWH44dxeXEHY8RlhYYaFbV",

        "name": "bookUserAppointment",

        "arguments": "{\"name\": \"John\", \"datetime\": \"2024-10-17T18:00:00\"}"

    }

}

Function Result:

{

    "role": "Function Call Result",

    "data": {

        "id": "call_5otWH44dxeXEHY8RlhYYaFbV",

        "name": "bookUserAppointment",

        "result": "Appointment successfully booked"

    }

}

Code Examples

# Running using evaluator IDs

curl -X POST https://new-prod.vocera.ai/test_framework/v1/scenarios-external/run_scenarios_text/ \

  -H "X-VOCERA-API-KEY: <your-api-key-here>" \

  -H "Content-Type: application/json" \

  -d '{

      "scenarios": [<evaluator-id-1>, <evaluator-id-2>, ...],

      "freq": 1

    }'

# Running all evaluators

curl -X POST https://new-prod.vocera.ai/test_framework/v1/scenarios-external/run_scenarios_text/ \

  -H "X-VOCERA-API-KEY: <your-api-key-here>" \

  -H "Content-Type: application/json" \

  -d '{

    "scenarios": "all",

    "agent_id": <agent-id>,

    "freq": 1

  }'

# Running first N evaluators

curl -X POST https://new-prod.vocera.ai/test_framework/v1/scenarios-external/run_scenarios_text/ \

  -H "X-VOCERA-API-KEY: <your-api-key-here>" \

  -H "Content-Type: application/json" \

  -d '{

    "scenarios": <number-of-evaluators>,

    "agent_id": <agent-id>,

    "freq": 1

  }'

Get Started

Key Concepts

Guides

API Documentation

Embedding Documentation

Running Evaluators

Running using Voice Calls

API Endpont

Authentication

Request Parameters

Example Request Body

Response

Response Fields

Run Fields

Example Response

Code Examples

Running using LLM Websocket

Authentication

Request Parameters

Example Request Body

Response

Response Fields

Run Fields

Example Response

WebSocket Message Format

Message Types

Normal Messages

Function Messages

Examples

Code Examples

Get Started

Key Concepts

Guides

API Documentation

Embedding Documentation

​Running using Voice Calls

​API Endpont

​Authentication

​Request Parameters

​Example Request Body

​Response

​Response Fields

​Run Fields

​Example Response

​Code Examples

​Running using LLM Websocket

​Authentication

​Request Parameters

​Example Request Body

​Response

​Response Fields

​Run Fields

​Example Response

​WebSocket Message Format

​Message Types

​Normal Messages

​Function Messages

​Examples

​Code Examples

Running using Voice Calls

API Endpont

Authentication

Request Parameters

Example Request Body

Response

Response Fields

Run Fields

Example Response

Code Examples

Running using LLM Websocket

Authentication

Request Parameters

Example Request Body

Response

Response Fields

Run Fields

Example Response

WebSocket Message Format

Message Types

Normal Messages

Function Messages

Examples

Code Examples