Response contract

The response contract controls how strictly Pipelines validates the response your agent returns after a run or turn.

Use the default while you are porting. Enable the contract only after your wrapper reliably returns messages and you want CI or regression tests to fail when that transcript disappears.

What reviewers need

To review an agent workflow, you do not need private PR context. You need:

• The agent endpoint code.
• The registered tools schema.
• The registration config, especially endpoint_url, auth, and response_contract.
• This page, plus Porting without the SDK.
• For multi-turn workflows, also read Multi-turn testing setup.

Review the endpoint for four things:

• It handles the {"ping": true} probe after inbound auth and before envelope parsing.
• Every tool call POSTs to the per-run proxy and returns the unwrapped response.
• It always returns a non-empty final_response.
• If the rich transcript guard is enabled, it returns response messages with at least one assistant message containing non-empty text.

Default behavior

With no response_contract block, Pipelines grades final_response.

{ "final_response": "The refund of $79.50 has been issued for order #4521." }

messages is optional. If messages is missing or malformed, the run still grades and the trace falls back to the thin final-response view with a soft warning.

Require a gradeable rich transcript

In the agent form, open the Response contract section and turn on Require a gradeable rich transcript.

If this guard is enabled and no assistant message with text is returned, the run fails with response_contract_violation and the judge is skipped.

A minimal valid rich response is:

{
  "final_response": "The refund of $79.50 has been issued.",
  "messages": [
    { "role": "user", "content": "Refund order #4521." },
    { "role": "assistant", "content": "The refund of $79.50 has been issued." }
  ]
}

Tool calls can also be included:

{
  "final_response": "The refund of $79.50 has been issued.",
  "messages": [
    { "role": "user", "content": "Refund order #4521." },
    {
      "role": "assistant",
      "tool_calls": [{ "id": "c1", "name": "get_order", "arguments": { "order_id": "4521" } }]
    },
    { "role": "tool", "tool_call_id": "c1", "content": "{\"status\":\"shipped\"}" },
    { "role": "assistant", "content": "The refund of $79.50 has been issued." }
  ],
  "metadata": {
    "model": "claude-sonnet-4-5",
    "total_input_tokens": 1842,
    "total_output_tokens": 217,
    "agent_runtime_ms": 4815
  }
}

API config

Enable the guard by storing this on the agent config:

{
  "endpoint_url": "https://your-agent.example.com/run",
  "response_contract": { "response_mode": "rich", "strictness": "strict" }
}

Remove the guard by omitting the block on create, or by sending response_contract: null on update:

{
  "config": {
    "response_contract": null
  }
}

Multi-turn note

Multi-turn turns use the turn response shape: return message for the assistant's reply to that turn, plus optional metadata, session_done, and response messages. The platform also accepts final_response as a lenient alias for message, but message is the primary field to implement.

If the guard is enabled, every turn must also return at least the assistant message generated for that turn in response messages.

Do not confuse the two messages fields:

• body["input"]["messages"] is inbound prior conversation history that the platform may send to your agent in multi-turn replay mode.
• Response messages is the outbound transcript your agent returns so the trace can render assistant turns, tool calls, and reasoning.