Structured output and responses

Use this when a step’s job is to produce data rather than to converse: an extraction, a classification, a JSON payload for a caller. It is also the page to read before writing checkResponse(), whose return value means the opposite of what most people assume.

The three result paths

Every model turn ends on exactly one of these:

The first is covered in Defining and handling tools. This page covers the other two.

Structured output with structOutputSchema()

Return a schema and PicoFlow binds it to the model with withStructuredOutput(...) before the call:

public structOutputSchema(): object {
  return z.object({
    title: z.string().describe("Movie title"),
    genre: z.string().describe("Main genre"),
    releaseYear: z.number().describe("Release year"),
    rating: z.number().min(0).max(10).describe("Rating from 0 to 10"),
    summary: z.string().describe("Short plot summary"),
  });
}

The default is null, meaning free-form text.

Two consequences worth planning for:

• onResponse() now receives the parsed object, not a string. The runner only converts to text when the response has a content property, which a structured result does not.
• Providers frequently omit usage metadata for structured-output calls, so those tokens may not appear in the session document’s tokens totals.

The alternative — used by FavoritesStep and NoToolStep — is to describe the shape in the prompt, often by interpolating an example JSON file, and parse the text yourself with StringUtil.parseJson. That works with every provider and keeps the response inspectable.

onResponse() and what you may return

public async onResponse(llmResult: string | object): Promise<LastResponseType>

The base implementation stringifies objects and returns strings unchanged. LastResponseType allows three shapes:

Saving and returning:

public async onResponse(llmResult: string | object): Promise<LastResponseType> {
  this.saveState({ who: llmResult as JsonValue });
  return llmResult as string;
}

Parsing, validating, then routing:

public async onResponse(llmResult: string | object): Promise<LastResponseType> {
  const json =
    typeof llmResult === "string"
      ? StringUtil.parseJson<JsonValue>(llmResult)
      : (llmResult as JsonValue);

  if (json && typeof json === "object" && !Array.isArray(json)) {
    this.saveState({ favorites: json });
    return go(NameStep);
  }

  return typeof llmResult === "string" ? llmResult : JSON.stringify(llmResult);
}

Note the fallback. When the model returns something unusable, returning the raw text lets the user see it and the conversation continue, rather than throwing mid-turn.

Routing from onResponse()

Returning a step class from onResponse() is a real transition: Flow.goto() runs, so the current step’s onExit() and the destination’s onEnter() and onCrossing() fire, and the runner immediately makes another model call on the destination.

lastResponse(...) is the purpose-built builder for the object form:

return lastResponse(ExtractInvoiceStep)
  .withState({ from_previous: parsedResult as JsonValue })
  .withContentType(HttpContentType.Json);

go(...) is structurally compatible and is what the demo uses — NoToolStep returns go(ExtractInvoiceStep).withState({ from_previous: parsedResult }). Prefer lastResponse() in new code, because go()'s withToolFeedback(...) has no meaning outside a tool handler.

checkResponse() and inverted retry semantics

public checkResponse(llmResult: string | object): boolean

false accepts the response. true asks for a retry. The default returns false.

public checkResponse(result: string | object): boolean {
  // true means "this is unacceptable, call the model again"
  return !looksLikeCompleteJson(result);
}

Two properties are required of the predicate:

• deterministic — it runs once per attempt, on different responses;
• side-effect free — it must not save state or mutate memory, because the response it rejected is discarded.

What the retry loop actually does

checkResponse() is only one input to the loop. The full sequence per attempt:

attempt 1..N   (N = step retryAttempts, then Flow, then adapter fallback, default 3)
  invoke the model
  tally tokens
  if the response has no tool calls and blank content:
       pop the offending message(s) from memory
       push "Follow system prompt and respond properly."
       retry
  else:
       retry if checkResponse(response) === true
  wait 500ms between attempts
exhausted -> throw: LLM call failed. Reason:<finish reason>. Error:<message>

Empty-response recovery is automatic and needs no code from you. Set retryAttempts in the Flow or Step selection when that workflow needs a different budget:

protected configModel() {
  return {
    provider: "in-house",
    name: "support-model",
    params: {},
    retryAttempts: 5,
  } as const;
}

Every attempt is a billed model call. checkResponse() that rejects too eagerly triples the cost of a step.

Direct responses and content types

A direct message ends the HTTP turn without another model call, while leaving the current step active:

this.flow.markCompleted();
return direct(args?.json).withContentType(HttpContentType.Json);

The response envelope carries contentType from the step, and the demo controller sends the raw message with that content type whenever it is not text/plain:

if (result.contentType && result.contentType !== HttpContentType.Plain) {
  return res.type(result.contentType).send(result.message);
}
return res.send(result);

So a JSON content type changes the response body shape, not just its header. Callers that expect the standard { success, completed, message, session } envelope will break unless they branch on the content type. Set it deliberately.

Failure modes

Related: Structured output, Response-driven steps, and Error handling and completion.