Two failure modes seen live with Qwen3.6 on llama.cpp ended turns silently mid-task: - The model writes its tool call as plain text inside its reasoning, the server never parses it, and the round ends with nothing executed. The loop now detects the markers and nudges the model to re-issue the call for real (max 2 per turn). - llama.cpp silently ignores the max_thinking_tokens field, so a model in a reasoning spiral ran until max_tokens (seen live: 25k+ tokens of nonstop thinking, ~20 min). The llamacpp client now enforces the budget client-side during Stream: once exceeded while the round is still pure reasoning, it cuts with FinishThinkingBudget and aborts the request (freeing the server slot); the loop answers with its own corrective nudge, on a separate counter. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|---|---|---|
| .. | ||
| integration_test.go | ||
| loop.go | ||
| loop_test.go | ||
| README.es.md | ||
| README.md | ||
| subagent.go | ||
| subagent_test.go | ||
| thinking_budget_test.go | ||
| unparsed_toolcall_test.go | ||
pkg/agent
The main loop that runs an LLM agent with guardrails.
Responsibility
Coordinate the iterative cycle between the LLM and tool execution:
while iteration < MaxIterations:
response = llm.Generate(messages, tools)
if no tool calls: return response
for tool_call in response.ToolCalls:
if needs_approval: ask_user()
result = execute(tool_call)
append tool result to messages
Public API
type Loop interface {
Run(ctx context.Context, input string, history ...llm.Message) (Response, error)
RunStream(ctx context.Context, input string, history ...llm.Message) iter.Seq2[Chunk, error]
}
type Config struct {
LLM llm.LLMClient
Persona persona.Persona
Tools tools.Registry
Sandbox Sandbox
MaxIters int
Approver Approver // nil = auto-approve all
OnIteration func(Iteration) // observability hook
}
type Response struct {
Content string
ToolCalls []tools.Call
Iterations int
Duration time.Duration
TokenUsage llm.TokenUsage
}
Guarantees
- Termination: Always terminates (max iterations, error, or final response)
- Idempotency: Re-running with the same input produces the same output (given the same LLM)
- Observability: Each iteration emits an OpenTelemetry span
- Approval: Destructive tools (
Askpermission) require confirmation
See also
- pkg/tools — Tool execution
- pkg/llm — LLMClient interface
- pkg/persona — Persona assembly