Slow tool, waiting caller: five voice harnesses compared

One of the key issues for a voice AI harness, unlike a text harness, is how to handle long-running tool calls. A tool call asks another service to do work, such as check an order. A text chat can show that a reply is pending. In a live voice chat, a wait can leave the caller in silence. If the agent speaks while it waits, it must not claim that the action succeeded.

A harness is the software that connects audio, a model, and tools. It must manage the conversation while a tool runs. It must also return the tool result to the model. For example, the agent can say, “I am checking the order.” When the tool returns the status, the agent can report that result. The systems below put this work in different places.

How Different Voice Harnesses work

Here is a quick comparison of how the frontier voice harnesses handle the long running tool calls.

OpenAI GPT-Live: speech and delegated work

GPT-Live manages listening and speech. The backend handles reasoning and tools. With client delegation, the app can keep backend work running when the caller interrupts the assistant. When the work ends, the app can send an update for that task:

connection.send({
  type: "session.commentary.append",
  delegation_id: "item_123",
  content: "The order shipped today.",
});

Use session.commentary.append for information the agent should speak. Use session.thinking.append for information it can use later. With Responses delegation, Live manages the backend tool loop. The app still runs its own custom functions. GPT-Live delegation docs · Getting started

Gemini Live: the model controls non-blocking calls

The current model table says that Gemini 3.8 Live supports async calls by default. Gemini 3.8 Live Extended Thinking supports async calls only. Gemini 3.1 Flash Live Preview does not support them. Mark a supported function as NON_BLOCKING. The model can then continue while the client runs the function. When the function ends, the client sends its result:

check_order = {
    "name": "check_order",
    "behavior": "NON_BLOCKING",
}

async def handle_tool_call(session, call):
    result = await check_order_api(call.args["order_id"])
    await session.send_tool_response(function_responses=[
        types.FunctionResponse(
            id=call.id,
            name=call.name,
            response={"result": result, "scheduling": "WHEN_IDLE"},
        )
    ])

async def receive_tool_calls(session):
    async for event in session.receive():
        if event.tool_call:
            for call in event.tool_call.function_calls:
                asyncio.create_task(handle_tool_call(session, call))

The model sends a tool call because the session declared that function. The client starts a task for each call, so the receive loop can keep handling events. The task sends a FunctionResponse when the API returns. NON_BLOCKING controls whether Gemini waits during tool execution. Scheduling controls what Gemini does when the result arrives: INTERRUPT means speak about it now; WHEN_IDLE means wait until current speech ends; SILENT means keep it for later. The tool guide uses INTERRUPT; the capability table spells the Gemini 3.8 value INTERRUPTED. Check the selected model and SDK for the accepted value. Tool-use docs and sample · Current model capability table

xAI Grok Realtime: the client returns the result, then resumes

The custom-function sequence in the official docs is:

response.function_call_arguments.done
  -> run the function in the client
  -> send conversation.item.create(function_call_output)
  -> send response.create

The client can run parallel functions at the same time. It must send all results before it sends response.create. The docs do not describe a setting that lets the model continue before a custom function returns. Speech-to-speech tool events

Pipecat: the framework runs async handlers

Pipecat’s official OpenAI sample starts an async handler and sends its result with a callback:

@tool_options(cancel_on_interruption=False, timeout_secs=30)
async def get_current_weather(params, location, format):
    await asyncio.sleep(15)
    await params.result_callback({"conditions": "nice"})

The sample sets a 30-second limit for the handler. This does not guarantee that an external action will finish in 30 seconds. Pipecat also has official async-tool samples for Gemini Live and Grok Realtime. They keep the conversation active while the tool runs, then send its result to the model. OpenAI sample · Gemini Live sample · Grok Realtime sample

LiveKit Agents: the framework runs background tools

LiveKit runs async tools in the background so the agent can keep talking. A tool can let the model cancel it:

@function_tool(flags=ToolFlag.CANCELLABLE)
async def book_flight(ctx, origin, destination, date):
    return await book_flight_api(origin, destination, date)

By default, an async tool runs to completion even if the user does something else. Cancellation is optional because a write may not be safe to stop part-way through. LiveKit can also reject, replace, or ask the model to confirm a duplicate call. Async tools docs and code · Tool definition and interruption behavior

Takeaway

These systems put the wait in different places. GPT-Live separates speech from delegated backend work. Gemini Live can keep interacting during a supported NON_BLOCKING function. The documented Grok flow waits for the client to return its custom-function result before it resumes. Pipecat and LiveKit run tools in the background through framework behavior.

“Async tool” can describe either model behavior or framework scheduling. A background handler alone does not prove that the model can take another turn or that it will speak the result. Check how the model handles tool calls, how the harness returns results, and what happens after an interruption or duplicate call.

Checked against current official documents and sample code on October 16, 2026.