Victor Vargas
a38f63683b
- Add optional history parameter to Run/RunStream for conversation context - Add Reasoning/ReasoningDelta fields to completion and stream types - Update llama.cpp adapter to propagate reasoning content from responses - Default persona language now adapts to the user's language dynamically
590 lines
No EOL
20 KiB
Markdown
590 lines
No EOL
20 KiB
Markdown
# 🏗️ Rony LLM Agent — Architecture
|
|
|
|
**Version:** 1.0
|
|
**Author:** Victor Hugo Vargas
|
|
**Date:** 2026-06-28
|
|
**Status:** Library architecture specification
|
|
|
|
> 📚 **Other documents:**
|
|
> - [`README.md`](./README.md) — Library overview
|
|
> - [`components.md`](./components.md) — Detailed reference per package
|
|
> - [`phase2.md`](./phase2.md) — Advanced features (MCP, full RAG, Skills, etc.)
|
|
> - [`../../METHODOLOGY.md`](../../METHODOLOGY.md) — How we build (SDD + DDD + Hexagonal)
|
|
>
|
|
> 📐 **Methodology:** This library follows the **SDD + DDD + Hexagonal Architecture** approach. See [`METHODOLOGY.md`](../../METHODOLOGY.md) for details.
|
|
|
|
---
|
|
|
|
## 🎯 1. Library Vision
|
|
|
|
### 1.1 What is rony-llm-agent?
|
|
|
|
A Go library that provides all the **generic and reusable** logic for building LLM-based agents. It's not a final product — it's the foundation on which specific products are built.
|
|
|
|
**Products that consume it:**
|
|
|
|
- [`VictorVargas/rony-harness`](https://github.com/VictorVargas/rony-harness) — AI agent harness for software development (TUI CLI)
|
|
- [`VictorVargas/rony-chat-bot`](https://github.com/VictorVargas/rony-chat-bot) — HTTP chatbot for portfolios and websites
|
|
|
|
### 1.2 Design principles
|
|
|
|
- **Reusable, not opinionated.** Does not force a UI type, deployment, or use case.
|
|
- **Pure Hexagonal.** Ports & adapters — every external dependency is behind an interface.
|
|
- **Streaming-first.** Uses `iter.Seq2` from Go 1.23+ for natural streaming without callbacks.
|
|
- **Secure by default.** Filesystem path sandbox with `os.Root` (Go 1.24+).
|
|
- **Zero magic.** No reflection, no codegen, no DSLs. Idiomatic and explicit Go.
|
|
- **YAML configurable.** Everything that affects behavior is declarative.
|
|
|
|
### 1.3 What it is NOT
|
|
|
|
- ❌ It's not a CLI — that's the product's responsibility (e.g., `rony-harness`)
|
|
- ❌ It's not an HTTP server — that's the product's responsibility (e.g., `rony-chat-bot`)
|
|
- ❌ It does not force a specific model or provider — interchangeable adapters
|
|
- ❌ It has no persistent state of its own — each product handles that
|
|
|
|
---
|
|
|
|
## 🏗️ 2. Hexagonal Architecture (Ports & Adapters)
|
|
|
|
```
|
|
┌─────────────────────────────────────────────────────────────────┐
|
|
│ Products that consume │
|
|
│ (rony-harness, rony-chat-bot, dealer-bot, etc.) │
|
|
└──────────────────────────┬──────────────────────────────────────┘
|
|
│ use public interfaces
|
|
▼
|
|
┌─────────────────────────────────────────────────────────────────┐
|
|
│ rony-llm-agent (pkg/ — public API) │
|
|
│ │
|
|
│ ┌─────────┐ ┌─────────┐ ┌─────────┐ ┌─────────┐ │
|
|
│ │ agent │ │ llm │ │ persona │ │ tools │ ... │
|
|
│ └────┬────┘ └────┬────┘ └─────────┘ └─────────┘ │
|
|
│ │ │ │
|
|
│ │ uses ports (interfaces) │
|
|
│ ▼ ▼ │
|
|
│ ┌─────────────────────────────────────────────────────┐ │
|
|
│ │ PORTS (pure interfaces) │ │
|
|
│ │ LLMClient, VectorDB, Embedder, ToolRegistry, ... │ │
|
|
│ └──────────────────────┬───────────────────────────────┘ │
|
|
│ │ implemented by adapters │
|
|
└──────────────────────────┼──────────────────────────────────────┘
|
|
▼
|
|
┌─────────────────────────────────────────────────────────────────┐
|
|
│ Adapters (in pkg/*/internal/) │
|
|
│ • providers/anthropic, openai, ollama, llamacpp │
|
|
│ • backends/chroma, qdrant, sqlite │
|
|
│ • embeddings/ollama, onnx │
|
|
└─────────────────────────────────────────────────────────────────┘
|
|
```
|
|
|
|
### 2.1 Layers
|
|
|
|
**Ports (pure interfaces)**
|
|
- Define WHAT the domain does, not HOW
|
|
- Contracts without implementation
|
|
- Allow swapping adapters without touching business logic
|
|
|
|
**Domain (pkg/)**
|
|
- Contains all reusable logic
|
|
- Does NOT depend on anything external
|
|
- Only imports interfaces from its own ports
|
|
|
|
**Adapters (internals of each adapter)**
|
|
- Concrete implementations of ports
|
|
- External dependencies live here (HTTP clients, filesystem, DB drivers)
|
|
|
|
---
|
|
|
|
## 🧠 3. Core Interfaces
|
|
|
|
### 3.1 LLMClient (`pkg/llm/`)
|
|
|
|
Multi-provider abstraction. Products never touch a provider SDK directly — they always go through this interface.
|
|
|
|
```go
|
|
type LLMClient interface {
|
|
Generate(ctx context.Context, req CompletionRequest) (CompletionResponse, error)
|
|
Stream(ctx context.Context, req CompletionRequest) iter.Seq2[StreamChunk, error]
|
|
Name() string
|
|
Capabilities() ProviderCapabilities
|
|
}
|
|
|
|
type CompletionRequest struct {
|
|
Messages []Message
|
|
Tools []Tool
|
|
ToolChoice ToolChoice // "auto" | "required" | "none" | ToolRef
|
|
Model string
|
|
Temperature *float32
|
|
MaxTokens *int
|
|
Stop []string
|
|
Metadata map[string]string
|
|
}
|
|
|
|
type CompletionResponse struct {
|
|
ID string
|
|
Content string
|
|
ToolCalls []ToolCall
|
|
Usage TokenUsage
|
|
StopReason string // "end_turn" | "tool_use" | "max_tokens" | "stop_sequence"
|
|
}
|
|
|
|
type StreamChunk struct {
|
|
Delta string
|
|
ToolCalls []ToolCall // streamed incremental
|
|
FinishReason string
|
|
Usage TokenUsage // only on final chunk
|
|
}
|
|
|
|
type TokenUsage struct {
|
|
InputTokens int
|
|
OutputTokens int
|
|
TotalTokens int
|
|
}
|
|
|
|
type ProviderCapabilities struct {
|
|
SupportsTools bool
|
|
SupportsVision bool
|
|
SupportsJSON bool
|
|
MaxContextWindow int
|
|
}
|
|
```
|
|
|
|
**Full details:** [`pkg/llm/README.md`](../pkg/llm/README.md)
|
|
|
|
### 3.2 Tool System (`pkg/tools/`)
|
|
|
|
Function callable by the LLM with JSON Schema, permissions, and sandbox.
|
|
|
|
```go
|
|
type Tool struct {
|
|
Name string
|
|
Description string
|
|
InputSchema json.RawMessage // JSON Schema draft-07+
|
|
Required []string
|
|
Handler ToolHandler // func(ctx, args json.RawMessage) (Result, error)
|
|
Permission Permission // Allow | Ask | Deny
|
|
Examples []ToolExample // few-shot for the LLM
|
|
}
|
|
|
|
type ToolHandler func(ctx context.Context, args json.RawMessage) (ToolResult, error)
|
|
|
|
type ToolResult struct {
|
|
Content string
|
|
IsError bool
|
|
Metadata map[string]string
|
|
Artifacts []Artifact
|
|
}
|
|
|
|
type ToolCall struct {
|
|
ID string
|
|
Name string
|
|
Arguments json.RawMessage
|
|
Thought string // optional: chain-of-thought
|
|
}
|
|
|
|
type Permission int
|
|
|
|
const (
|
|
Allow Permission = iota
|
|
Ask
|
|
Deny
|
|
)
|
|
|
|
type ToolRegistry interface {
|
|
Register(tool Tool) error
|
|
Get(name string) (Tool, bool)
|
|
List() []Tool
|
|
Filter(policy PermissionPolicy) []Tool
|
|
}
|
|
```
|
|
|
|
**Full details:** [`pkg/tools/README.md`](../pkg/tools/README.md)
|
|
|
|
### 3.3 Agent Loop (`pkg/agent/`)
|
|
|
|
Iterative loop between LLM and tool execution. It's the "brain" that orchestrates everything.
|
|
|
|
```go
|
|
type Loop interface {
|
|
Run(ctx context.Context, input string, history ...Message) (Response, error)
|
|
RunStream(ctx context.Context, input string, history ...Message) iter.Seq2[Chunk, error]
|
|
}
|
|
|
|
type Config struct {
|
|
LLM llm.LLMClient
|
|
Persona persona.Persona
|
|
Tools tools.Registry
|
|
Sandbox Sandbox
|
|
MaxIters int
|
|
Approver Approver // nil = auto-approve all
|
|
OnIteration func(Iteration) // observability hook
|
|
}
|
|
|
|
type Response struct {
|
|
Content string
|
|
ToolCalls []tools.Call
|
|
Iterations int
|
|
Duration time.Duration
|
|
TokenUsage llm.TokenUsage
|
|
}
|
|
```
|
|
|
|
**Algorithm:**
|
|
|
|
```
|
|
function RunAgent(userMessage, session):
|
|
messages = append(session.Messages, userMessage)
|
|
iteration = 0
|
|
|
|
while iteration < MaxIterations:
|
|
iteration++
|
|
|
|
response = llm.Generate(messages, tools)
|
|
if no tool calls: return response
|
|
|
|
messages.append(assistantMsg(response))
|
|
|
|
for toolCall in response.ToolCalls:
|
|
tool = registry.Get(toolCall.Name)
|
|
|
|
# Approval gate
|
|
if tool.Permission == Ask && !session.AutoApprove:
|
|
if !cli.AskApproval(tool, toolCall):
|
|
messages.append(denialMessage(toolCall))
|
|
continue
|
|
|
|
# Sandbox validation
|
|
if err := sandbox.ValidateToolCall(tool, toolCall); err != nil {
|
|
messages.append(errorMessage(toolCall, err))
|
|
continue
|
|
|
|
# Execute
|
|
result, err := tool.Handler(ctx, toolCall.Arguments)
|
|
|
|
# Truncate if needed
|
|
result.Content = truncate(result.Content, MaxToolOutputBytes)
|
|
|
|
messages.append(toolResultMessage(toolCall, result))
|
|
|
|
return ErrorResponse("max_iterations_exceeded")
|
|
```
|
|
|
|
**Parameters:**
|
|
|
|
| Parameter | Default | Description |
|
|
|---|---|---|
|
|
| `MaxIterations` | 50 | Max cycles before cutting |
|
|
| `MaxTokensPerSession` | 1,000,000 | Hard cap on consumed tokens |
|
|
| `MaxToolOutputBytes` | 50KB | Truncate large outputs |
|
|
| `ToolTimeout` | 30s | Default, overrideable per tool |
|
|
|
|
**Termination conditions:**
|
|
1. ✅ LLM returns response without `ToolCalls` (normal case)
|
|
2. 🛑 `MaxIterations` reached
|
|
3. 💰 `MaxTokensPerSession` exceeded
|
|
4. ⏱️ Global timeout
|
|
5. 🚫 User aborts (`Ctrl+C` or `/stop`)
|
|
|
|
**Full details:** [`pkg/agent/README.md`](../pkg/agent/README.md)
|
|
|
|
### 3.4 Persona System (`pkg/persona/`)
|
|
|
|
System prompt assembler combining base + persona YAML + AGENTS.md.
|
|
|
|
```go
|
|
type Persona struct {
|
|
ID string
|
|
Name string
|
|
Tone string
|
|
Style string
|
|
Language string
|
|
Constraints []string
|
|
FewShot []llm.Message
|
|
}
|
|
|
|
type Loader interface {
|
|
Load(ctx context.Context, configPath string) (Persona, error)
|
|
Discover(ctx context.Context, workdir string) (Persona, error)
|
|
}
|
|
```
|
|
|
|
**How the final system prompt is built:**
|
|
|
|
```
|
|
1. Base prompt (hardcoded in the library)
|
|
2. Persona YAML (configurable)
|
|
3. AGENTS.md of the project (discovered in directory tree)
|
|
4. Working memory context (if there's compaction)
|
|
```
|
|
|
|
**AGENTS.md discovery:** Searches `./AGENTS.md`, walks up to parent, etc., concatenating all. Also includes `~/.config/rony/AGENTS.md` as global default.
|
|
|
|
**Full details:** [`pkg/persona/README.md`](../pkg/persona/README.md)
|
|
|
|
### 3.5 Memory (`pkg/rag/`)
|
|
|
|
Retrieval-Augmented Generation: persistent memory and semantic search.
|
|
|
|
```go
|
|
type Memory interface {
|
|
Add(ctx context.Context, fragment Fragment) error
|
|
Search(ctx context.Context, query string, topK int) ([]Fragment, error)
|
|
Forget(ctx context.Context, id string) error
|
|
}
|
|
|
|
type Fragment struct {
|
|
ID string
|
|
Content string
|
|
Vector []float32
|
|
Metadata map[string]string
|
|
Timestamp time.Time
|
|
ProjectID string
|
|
}
|
|
|
|
type Embedder interface {
|
|
Embed(ctx context.Context, text string) ([]float32, error)
|
|
Dimensions() int
|
|
}
|
|
```
|
|
|
|
**Memory types:**
|
|
|
|
| Type | What it stores | Persistence |
|
|
|---|---|---|
|
|
| **Working** | Current session messages | RAM |
|
|
| **Episodic** | Past events | Vector DB |
|
|
| **Semantic** | Consolidated knowledge | Vector DB (curated) |
|
|
| **Procedural** | Usage patterns | Vector DB (auto-learned) |
|
|
|
|
**Supported backends:**
|
|
- ChromaDB embedded (default)
|
|
- Qdrant embedded
|
|
- SQLite + sqlite-vec
|
|
|
|
**Full details:** [`pkg/rag/README.md`](../pkg/rag/README.md)
|
|
|
|
### 3.6 Sandbox (`pkg/tools/sandbox/`)
|
|
|
|
Kernel-level filesystem sandbox using `os.Root` (Go 1.24+).
|
|
|
|
```go
|
|
type Sandbox struct {
|
|
workspace *os.Root // Go 1.24+
|
|
configDir *os.Root // ~/.config/rony/
|
|
}
|
|
|
|
func (s *Sandbox) Open(path string) (*os.File, error)
|
|
func (s *Sandbox) Create(path string) (*os.File, error)
|
|
func (s *Sandbox) ReadFile(path string) ([]byte, error)
|
|
func (s *Sandbox) WriteFile(path string, data []byte, perm os.FileMode) error
|
|
```
|
|
|
|
**Why `os.Root` (Go 1.24+) is security-critical:**
|
|
|
|
| Attack vector | `strings.HasPrefix` (naive) | `os.Root` |
|
|
|---|---|---|
|
|
| `../../../etc/passwd` | Blocked if abs path doesn't have prefix | Blocked by kernel |
|
|
| Symlink inside workspace → `/etc/passwd` | Blocked only if we resolve manually | Blocked natively |
|
|
| TOCTOU race condition | Possible | Impossible (kernel-checked) |
|
|
| Path encoding (`%2e%2e`) | Not detected | Detected by stdlib |
|
|
| Null bytes in path | Depends on OS | Handled by stdlib |
|
|
|
|
> 🔒 That's why the library **requires Go 1.26+** (see [README §16.1.3.1](https://github.com/VictorVargas/rony-harness/blob/main/docs/architecture.md#16131)).
|
|
|
|
### 3.7 Configuration (`pkg/config/`)
|
|
|
|
YAML loading with hierarchical precedence.
|
|
|
|
```go
|
|
type Config struct {
|
|
Model string
|
|
Persona string
|
|
Provider ProviderConfig
|
|
Tools ToolPolicy
|
|
Logging LoggingConfig
|
|
Sandbox SandboxConfig
|
|
}
|
|
|
|
type Loader interface {
|
|
Load(ctx context.Context, workdir string) (Config, error)
|
|
}
|
|
```
|
|
|
|
**Precedence order (highest to lowest):**
|
|
|
|
```
|
|
flags CLI > env vars > ./rony.yaml > ~/.config/rony/config.yaml > defaults
|
|
```
|
|
|
|
**Full details:** [`pkg/config/README.md`](../pkg/config/README.md)
|
|
|
|
---
|
|
|
|
## 🔒 4. Security Model
|
|
|
|
### 4.1 Covered threats
|
|
|
|
| Threat | Library mitigation |
|
|
|---|---|
|
|
| Path traversal | `os.Root` sandbox |
|
|
| Command injection | Command validation (delegated to products using bash tool) |
|
|
| Resource exhaustion | `MaxIterations`, `MaxTokensPerSession`, `MaxToolOutputBytes` |
|
|
| Prompt injection | Wrap external content in `<untrusted_content>` tags (Phase 2) |
|
|
| Secret leakage | `Redact()` function for logs (Phase 2) |
|
|
| Tool misuse | Approval gates + permission policies |
|
|
|
|
### 4.2 Principle of least privilege
|
|
|
|
- Tools have `Permission: Allow | Ask | Deny`
|
|
- Default is `Allow` only for read-only tools
|
|
- `Ask` for tools that mutate state
|
|
- `Deny` is rare, used for explicitly disabled tools
|
|
|
|
### 4.3 Defense in depth
|
|
|
|
```
|
|
Layer 1: Permission policy → rejects unauthorized tools
|
|
Layer 2: Approval gate (Ask) → user confirms before execution
|
|
Layer 3: Sandbox validation → paths within allowed roots
|
|
Layer 4: Kernel enforcement (os.Root) → guarantee at OS level
|
|
Layer 5: Output truncation → doesn't return huge outputs
|
|
Layer 6: Secret redaction → doesn't expose secrets in logs/outputs
|
|
```
|
|
|
|
---
|
|
|
|
## 📐 5. Technical Specifications (SDD)
|
|
|
|
### 5.1 Core Functional Requirements
|
|
|
|
| ID | Requirement | Implementation |
|
|
|---|---|---|
|
|
| LRF-001 | LLMClient interface | `pkg/llm/` |
|
|
| LRF-002 | Multi-provider (OpenAI, Anthropic, Ollama, llama.cpp) | `pkg/llm/providers/` |
|
|
| LRF-003 | Native streaming | `iter.Seq2` |
|
|
| LRF-004 | Tool calling with JSON Schema | `pkg/tools/` |
|
|
| LRF-005 | Agent loop with guardrails | `pkg/agent/` |
|
|
| LRF-006 | Persona system + AGENTS.md | `pkg/persona/` |
|
|
| LRF-007 | RAG memory | `pkg/rag/` |
|
|
| LRF-008 | Filesystem sandbox (kernel) | `pkg/tools/sandbox/` |
|
|
| LRF-009 | Configuration with precedence | `pkg/config/` |
|
|
| LRF-010 | Approval gates | `pkg/agent/` |
|
|
|
|
### 5.2 Non-Functional Requirements
|
|
|
|
| ID | Category | Target |
|
|
|---|---|---|
|
|
| LRNF-001 | Minimum Go version | 1.26 (due to os.Root) |
|
|
| LRNF-002 | Test coverage | ≥80% |
|
|
| LRNF-003 | API stability | Strict semver |
|
|
| LRNF-004 | Minimal dependencies | Only stdlib + provider SDKs |
|
|
| LRNF-005 | Thread safety | All public API is safe for concurrent use |
|
|
| LRNF-006 | Context propagation | Every callable takes `context.Context` |
|
|
|
|
---
|
|
|
|
## 🧪 6. Testing
|
|
|
|
### 6.1 Mock LLM Server
|
|
|
|
For deterministic tests, products can use `pkg/llm/mock`:
|
|
|
|
```go
|
|
import "github.com/VictorVargas/rony-llm-agent/pkg/llm/mock"
|
|
|
|
mockClient := mock.New(mock.Responses{
|
|
{Match: "hello", Response: "Hi! How are you?"},
|
|
{Match: "*", Response: "default"},
|
|
})
|
|
```
|
|
|
|
### 6.2 Mock Memory
|
|
|
|
```go
|
|
import "github.com/VictorVargas/rony-llm-agent/pkg/rag/mock"
|
|
|
|
memory := mock.NewMemory([]rag.Fragment{
|
|
{Content: "doc1", ProjectID: "test"},
|
|
})
|
|
```
|
|
|
|
### 6.3 Critical tests table (entire library)
|
|
|
|
| Test | Package | Priority |
|
|
|---|---|---|
|
|
| `LLMClient.Generate` returns valid response | `pkg/llm/` | High |
|
|
| `LLMClient.Stream` emits chunks in order | `pkg/llm/` | High |
|
|
| `ToolRegistry.Register/Get/List` | `pkg/tools/` | High |
|
|
| Agent loop terminates without tool calls | `pkg/agent/` | High |
|
|
| Agent loop terminates at max iterations | `pkg/agent/` | High |
|
|
| Path sandbox rejects `../../../etc/passwd` | `pkg/tools/sandbox/` | High |
|
|
| Persona loader from YAML | `pkg/persona/` | High |
|
|
| AGENTS.md discovery (walks up directories) | `pkg/persona/` | Medium |
|
|
| Config precedence (env > file > defaults) | `pkg/config/` | High |
|
|
| Memory Search returns top-K by similarity | `pkg/rag/` | High |
|
|
|
|
---
|
|
|
|
## 📚 7. Code conventions
|
|
|
|
### 7.1 Naming
|
|
|
|
- **Packages:** lowercase, singular (`agent`, `tools`, `llm`)
|
|
- **Interfaces:** end in noun or capability (`Loop`, `LLMClient`, `Embedder`)
|
|
- **Errors:** `ErrXXX` for sentinels, wrapped with `%w`
|
|
- **Constructors:** `New` for main constructor, `NewXxx` for variants
|
|
|
|
### 7.2 Errors
|
|
|
|
```go
|
|
// Sentinel errors
|
|
var (
|
|
ErrToolNotFound = errors.New("tool not found")
|
|
ErrSandboxViolation = errors.New("path outside allowed roots")
|
|
)
|
|
|
|
// Wrapping
|
|
if err != nil {
|
|
return fmt.Errorf("loading persona %s: %w", name, err)
|
|
}
|
|
```
|
|
|
|
### 7.3 Context
|
|
|
|
Every function that can block takes `ctx context.Context` as first parameter:
|
|
|
|
```go
|
|
func (l *Loop) Run(ctx context.Context, input string) (Response, error)
|
|
func (m *Memory) Search(ctx context.Context, query string, topK int) ([]Fragment, error)
|
|
```
|
|
|
|
---
|
|
|
|
## 🔄 8. Versioning
|
|
|
|
- **Strict semver** (`vMAJOR.MINOR.PATCH`)
|
|
- **MAJOR:** breaking changes in `pkg/` (interfaces, signatures, public types)
|
|
- **MINOR:** new features, new packages, new adapters
|
|
- **PATCH:** bugfixes
|
|
|
|
Private adapters (`pkg/llm/providers/openai/`) can change without a MAJOR bump if the `LLMClient` interface doesn't change.
|
|
|
|
---
|
|
|
|
## 📖 9. References
|
|
|
|
- **Go 1.26 release notes:** https://go.dev/doc/go1.26
|
|
- **`os.Root` documentation:** https://pkg.go.dev/os#Root
|
|
- **`iter.Seq2` documentation:** https://pkg.go.dev/iter
|
|
- **Hexagonal Architecture (Alistair Cockburn):** https://alistair.cockburn.us/hexagonal-architecture/
|
|
- **DDD (Eric Evans, 2003):** Domain-Driven Design
|
|
- **ReAct Pattern (Yao et al., 2022):** https://arxiv.org/abs/2210.03629
|
|
|
|
---
|
|
|
|
## 🔗 Related documents
|
|
|
|
- [`README.md`](./README.md) — Library overview
|
|
- [`components.md`](./components.md) — Detailed reference per package
|
|
- [`phase2.md`](./phase2.md) — Advanced features (MCP, full RAG, Skills, etc.)
|
|
- [Products that use this library](https://github.com/VictorVargas) |