Streaming Tool Calls Without Breaking Your UI: A Practical Pattern

Streaming works great until your model emits a tool call mid-token-flush and your frontend starts rendering half a JSON blob into the chat bubble. Then the tool resolves, the model resumes streaming, and your UI jumps because you forgot that the second stream is a continuation, not a new message. We've shipped enough agent UIs to have a strong opinion about how to avoid this.

This post is a concrete pattern for handling streamed tool calls across Claude, GPT, and Gemini without race conditions or ugly flicker. Code is TypeScript, but the model is portable.

The problem, stated precisely

When you stream a response from an LLM that can call tools, you're not getting a single flat token stream. You're getting an interleaved sequence of events that roughly looks like:

1. Text tokens (assistant prose)
2. A tool-use block starting
3. Tool input JSON, streamed incrementally
4. A stop event with reason tool_use
5. Your code runs the tool
6. You send the tool result back and open a new stream
7. More text tokens, possibly another tool call, repeat

Three things routinely go wrong:

• Tool JSON leaks into the visible transcript because the stream handler naively appends every delta to the message body.
• The UI treats each stream as a new message, so a single logical assistant turn renders as 2–5 bubbles.
• Tool calls race the UI — the tool resolves before React has committed the "calling tool…" state, and you get a flash of nothing.

All three are solvable with a small state machine and one disciplined rule: the stream handler owns parsing, the UI owns rendering, and they communicate through a typed event bus.

The mental model: one assistant turn, many segments

An "assistant turn" is everything between the user's message and the next time control returns to the user. Inside a turn you may have N text segments and M tool calls, in any order. Model that explicitly:

type Segment =
  | { kind: 'text'; id: string; text: string; done: boolean }
  | {
      kind: 'tool_call';
      id: string;
      name: string;
      input: unknown;
      status: 'streaming' | 'ready' | 'running' | 'done' | 'error';
      result?: unknown;
      error?: string;
    };

type AssistantTurn = {
  id: string;
  segments: Segment[];
  status: 'streaming' | 'awaiting_tools' | 'done';
};

The UI renders turn.segments in order. Text segments become prose bubbles; tool-call segments become collapsible "Called search_docs" cards with their own status chip. Nothing else needs to change in the renderer when a new segment appears — it just gets appended.

Why segment IDs matter

Every segment needs a stable ID assigned the moment it starts, before any content arrives. React keys off this. If you generate IDs lazily when the first delta lands, you'll get a one-frame flash where the segment doesn't exist yet, and any sibling animations will restart. Use the provider's block index or tool_use_id directly — all three major vendors give you one.

Normalising the three providers

Claude, OpenAI, and Gemini all stream tool calls differently. Rather than branch everywhere, normalise at the edge into a single event type:

type StreamEvent =
  | { type: 'segment_start'; segment: Segment }
  | { type: 'text_delta'; segmentId: string; text: string }
  | { type: 'tool_input_delta'; segmentId: string; jsonChunk: string }
  | { type: 'segment_end'; segmentId: string }
  | { type: 'turn_end'; reason: 'stop' | 'tool_use' | 'error' };

For Claude's Messages API, the mapping is almost 1:1 — content_block_start, content_block_delta (with input_json_delta for tools), content_block_stop, message_stop. See Anthropic's streaming docs for the exact event shapes.

For OpenAI's Chat Completions and Responses APIs, you accumulate tool_calls[i].function.arguments deltas by index. The Responses API makes this cleaner with explicit response.output_item.added events.

For Gemini, generateContentStream yields candidates that may contain functionCall parts; these generally arrive whole rather than streamed character by character, so your tool-input-delta handler just fires once with the full JSON.

The payoff: your reducer never knows which vendor it's talking to.

The reducer that keeps your UI calm

Here's the shape we use. It's deliberately boring:

function reduce(turn: AssistantTurn, ev: StreamEvent): AssistantTurn {
  switch (ev.type) {
    case 'segment_start':
      return { ...turn, segments: [...turn.segments, ev.segment] };

    case 'text_delta':
      return mapSegment(turn, ev.segmentId, (s) =>
        s.kind === 'text' ? { ...s, text: s.text + ev.text } : s
      );

    case 'tool_input_delta':
      return mapSegment(turn, ev.segmentId, (s) =>
        s.kind === 'tool_call'
          ? { ...s, input: s.input ?? '', _raw: (s._raw ?? '') + ev.jsonChunk }
          : s
      );

    case 'segment_end':
      return mapSegment(turn, ev.segmentId, (s) => {
        if (s.kind === 'text') return { ...s, done: true };
        if (s.kind === 'tool_call') {
          const parsed = safeParseJson(s._raw);
          return { ...s, input: parsed.value, status: parsed.ok ? 'ready' : 'error' };
        }
        return s;
      });

    case 'turn_end':
      return { ...turn, status: ev.reason === 'tool_use' ? 'awaiting_tools' : 'done' };
  }
}

Two subtle choices:

• Tool input is only parsed at segment_end. Partial JSON is useful for a progress indicator ("Preparing search query…"), but never for execution. Attempting partial-JSON parsing to show the user the arguments live is a trap — you'll handle more edge cases than it's worth.
• Tool execution does not happen in the reducer. The reducer is pure. A separate effect watches for segments transitioning to status: 'ready' and kicks off the tool.

Running tools without racing the UI

Once the turn hits awaiting_tools, you want to:

1. Dispatch every ready tool call in parallel (if they're safe to parallelise).
2. Update each segment's status to running immediately — before awaiting — so the UI reflects it.
3. When each tool resolves, update that segment to done with its result.
4. Once all tools are done, open the next stream with the tool results appended to the conversation.

async function runReadyTools(turn: AssistantTurn, dispatch: Dispatch) {
  const ready = turn.segments.filter(
    (s): s is Extract<Segment, { kind: 'tool_call' }> =>
      s.kind === 'tool_call' && s.status === 'ready'
  );

  await Promise.all(
    ready.map(async (call) => {
      dispatch({ type: 'tool_running', segmentId: call.id });
      try {
        const result = await tools[call.name].run(call.input);
        dispatch({ type: 'tool_done', segmentId: call.id, result });
      } catch (err) {
        dispatch({ type: 'tool_error', segmentId: call.id, error: String(err) });
      }
    })
  );

  dispatch({ type: 'continue_turn' });
}

The continue_turn action opens a new stream to the model with the same turn.id. The reducer appends new segments to the existing turn rather than starting a fresh one. That's the whole trick to avoiding the "five bubbles for one answer" UX.

Guard against unbounded continuations

Agents can loop. Keep a per-turn counter of continuation rounds and cap it (we usually start at 6 and tune down). If you hit the cap, surface it in the UI as a specific state — "stopped after 6 tool rounds" — not as a generic error. Users forgive limits they can see.

Transport: SSE is still the right default

For the browser, Server-Sent Events over fetch with a ReadableStream reader is the sweet spot in 2026. WebSockets are overkill for one-way streaming and introduce reconnection complexity you don't need. HTTP/2 or HTTP/3 with keep-alive handles concurrent streams fine.

On the server, forward the vendor's native stream rather than re-serialising every token — but do translate to your normalised StreamEvent type before sending. Doing the normalisation server-side means your mobile client, web client, and any internal tooling share the same event contract.

One gotcha: some proxies and CDNs buffer SSE. If you see tokens arriving in big clumps instead of smoothly, check for X-Accel-Buffering: no (nginx), disable response buffering on your edge, and make sure you're not sitting behind a product that compresses the stream.

What we'd do

If you're starting fresh, build the segment model first and the transport second. The temptation is to wire up the Anthropic or OpenAI SDK, pipe tokens to the DOM, and bolt on tool-call handling later. That path leads to a rewrite the first time you add a second provider or a second tool.

Concretely:

• Define Segment, AssistantTurn, and StreamEvent before you write any provider code.
• Write the reducer with unit tests driven by recorded event logs — capture real streams from each provider once and replay them.
• Keep tool execution out of the reducer and cap continuation rounds explicitly.
• Normalise on the server so clients stay dumb.

If you want a hand designing the agent layer or the UI patterns around it, that's the sort of thing we do on our AI engagements. Otherwise, go record a stream, open the event log, and start from the data.