victorvargas/rony-llm-agent
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1
rony-llm-agent/docs/architecture.md

20 KiB
Raw Blame History

🏗️ Rony LLM Agent — Architecture

Version: 1.0 Author: Victor Hugo Vargas Date: 2026-06-28 Status: Library architecture specification

📚 Other documents:

📐 Methodology: This library follows the SDD + DDD + Hexagonal Architecture approach. See METHODOLOGY.md for details.

🎯 1. Library Vision

1.1 What is go-llm-agent?

A Go library that provides all the generic and reusable logic for building LLM-based agents. It's not a final product — it's the foundation on which specific products are built.

Products that consume it:

1.2 Design principles

  • Reusable, not opinionated. Does not force a UI type, deployment, or use case.
  • Pure Hexagonal. Ports & adapters — every external dependency is behind an interface.
  • Streaming-first. Uses iter.Seq2 from Go 1.23+ for natural streaming without callbacks.
  • Secure by default. Filesystem path sandbox with os.Root (Go 1.24+).
  • Zero magic. No reflection, no codegen, no DSLs. Idiomatic and explicit Go.
  • YAML configurable. Everything that affects behavior is declarative.

1.3 What it is NOT

  • ❌ It's not a CLI — that's the product's responsibility (e.g., rony-harness)
  • ❌ It's not an HTTP server — that's the product's responsibility (e.g., rony-chat-bot)
  • ❌ It does not force a specific model or provider — interchangeable adapters
  • ❌ It has no persistent state of its own — each product handles that

🏗️ 2. Hexagonal Architecture (Ports & Adapters)

┌─────────────────────────────────────────────────────────────────┐
│                  Products that consume                         │
│   (rony-harness, rony-chat-bot, dealer-bot, etc.)               │
└──────────────────────────┬──────────────────────────────────────┘
                           │ use public interfaces
                           ▼
┌─────────────────────────────────────────────────────────────────┐
│              go-llm-agent (pkg/ — public API)                  │
│                                                                 │
│   ┌─────────┐  ┌─────────┐  ┌─────────┐  ┌─────────┐            │
│   │  agent │  │   llm   │  │ persona │  │  tools  │  ...       │
│   └────┬────┘  └────┬────┘  └─────────┘  └─────────┘            │
│        │            │                                          │
│        │ uses ports (interfaces)                               │
│        ▼            ▼                                          │
│   ┌─────────────────────────────────────────────────────┐      │
│   │              PORTS (pure interfaces)                │      │
│   │   LLMClient, VectorDB, Embedder, ToolRegistry, ...   │      │
│   └──────────────────────┬───────────────────────────────┘      │
│                          │ implemented by adapters            │
└──────────────────────────┼──────────────────────────────────────┘
                           ▼
┌─────────────────────────────────────────────────────────────────┐
│                 Adapters (in pkg/*/internal/)                   │
│   • providers/anthropic, openai, ollama, llamacpp               │
│   • backends/chroma, qdrant, sqlite                             │
│   • embeddings/ollama, onnx                                     │
└─────────────────────────────────────────────────────────────────┘

2.1 Layers

Ports (pure interfaces)

  • Define WHAT the domain does, not HOW
  • Contracts without implementation
  • Allow swapping adapters without touching business logic

Domain (pkg/)

  • Contains all reusable logic
  • Does NOT depend on anything external
  • Only imports interfaces from its own ports

Adapters (internals of each adapter)

  • Concrete implementations of ports
  • External dependencies live here (HTTP clients, filesystem, DB drivers)

🧠 3. Core Interfaces

3.1 LLMClient (pkg/llm/)

Multi-provider abstraction. Products never touch a provider SDK directly — they always go through this interface.

type LLMClient interface {
    Generate(ctx context.Context, req CompletionRequest) (CompletionResponse, error)
    Stream(ctx context.Context, req CompletionRequest) iter.Seq2[StreamChunk, error]
    Name() string
    Capabilities() ProviderCapabilities
}

type CompletionRequest struct {
    Messages    []Message
    Tools       []Tool
    ToolChoice  ToolChoice         // "auto" | "required" | "none" | ToolRef
    Model       string
    Temperature *float32
    MaxTokens   *int
    Stop        []string
    Metadata    map[string]string
}

type CompletionResponse struct {
    ID         string
    Content    string
    ToolCalls  []ToolCall
    Usage      TokenUsage
    StopReason string             // "end_turn" | "tool_use" | "max_tokens" | "stop_sequence"
}

type StreamChunk struct {
    Delta       string
    ToolCalls   []ToolCall       // streamed incremental
    FinishReason string
    Usage       TokenUsage      // only on final chunk
}

type TokenUsage struct {
    InputTokens  int
    OutputTokens int
    TotalTokens  int
}

type ProviderCapabilities struct {
    SupportsTools    bool
    SupportsVision   bool
    SupportsJSON     bool
    MaxContextWindow int
}

Full details: pkg/llm/README.md

3.2 Tool System (pkg/tools/)

Function callable by the LLM with JSON Schema, permissions, and sandbox.

type Tool struct {
    Name        string
    Description string
    InputSchema json.RawMessage    // JSON Schema draft-07+
    Required    []string
    Handler     ToolHandler        // func(ctx, args json.RawMessage) (Result, error)
    Permission  Permission         // Allow | Ask | Deny
    Examples    []ToolExample      // few-shot for the LLM
}

type ToolHandler func(ctx context.Context, args json.RawMessage) (ToolResult, error)

type ToolResult struct {
    Content    string
    IsError    bool
    Metadata   map[string]string
    Artifacts  []Artifact
}

type ToolCall struct {
    ID        string
    Name      string
    Arguments json.RawMessage
    Thought   string             // optional: chain-of-thought
}

type Permission int

const (
    Allow Permission = iota
    Ask
    Deny
)

type ToolRegistry interface {
    Register(tool Tool) error
    Get(name string) (Tool, bool)
    List() []Tool
    Filter(policy PermissionPolicy) []Tool
}

Full details: pkg/tools/README.md

3.3 Agent Loop (pkg/agent/)

Iterative loop between LLM and tool execution. It's the "brain" that orchestrates everything.

type Loop interface {
    Run(ctx context.Context, input string) (Response, error)
    RunStream(ctx context.Context, input string) iter.Seq2[Chunk, error]
}

type Config struct {
    LLM          llm.LLMClient
    Persona      persona.Persona
    Tools        tools.Registry
    Sandbox      Sandbox
    MaxIters     int
    Approver     Approver        // nil = auto-approve all
    OnIteration  func(Iteration) // observability hook
}

type Response struct {
    Content     string
    ToolCalls   []tools.Call
    Iterations  int
    Duration    time.Duration
    TokenUsage  llm.TokenUsage
}

Algorithm:

function RunAgent(userMessage, session):
    messages = append(session.Messages, userMessage)
    iteration = 0
    
    while iteration < MaxIterations:
        iteration++
        
        response = llm.Generate(messages, tools)
        if no tool calls: return response
        
        messages.append(assistantMsg(response))
        
        for toolCall in response.ToolCalls:
            tool = registry.Get(toolCall.Name)
            
            # Approval gate
            if tool.Permission == Ask && !session.AutoApprove:
                if !cli.AskApproval(tool, toolCall):
                    messages.append(denialMessage(toolCall))
                    continue
            
            # Sandbox validation
            if err := sandbox.ValidateToolCall(tool, toolCall); err != nil {
                messages.append(errorMessage(toolCall, err))
                continue
            
            # Execute
            result, err := tool.Handler(ctx, toolCall.Arguments)
            
            # Truncate if needed
            result.Content = truncate(result.Content, MaxToolOutputBytes)
            
            messages.append(toolResultMessage(toolCall, result))
    
    return ErrorResponse("max_iterations_exceeded")

Parameters:

Parameter Default Description
MaxIterations 50 Max cycles before cutting
MaxTokensPerSession 1,000,000 Hard cap on consumed tokens
MaxToolOutputBytes 50KB Truncate large outputs
ToolTimeout 30s Default, overrideable per tool

Termination conditions:

  1. ✅ LLM returns response without ToolCalls (normal case)
  2. 🛑 MaxIterations reached
  3. 💰 MaxTokensPerSession exceeded
  4. ⏱️ Global timeout
  5. 🚫 User aborts (Ctrl+C or /stop)

Full details: pkg/agent/README.md

3.4 Persona System (pkg/persona/)

System prompt assembler combining base + persona YAML + AGENTS.md.

type Persona struct {
    ID           string
    Name         string
    Tone         string
    Style        string
    Language     string
    Constraints  []string
    FewShot      []llm.Message
}

type Loader interface {
    Load(ctx context.Context, configPath string) (Persona, error)
    Discover(ctx context.Context, workdir string) (Persona, error)
}

How the final system prompt is built:

1. Base prompt (hardcoded in the library)
2. Persona YAML (configurable)
3. AGENTS.md of the project (discovered in directory tree)
4. Working memory context (if there's compaction)

AGENTS.md discovery: Searches ./AGENTS.md, walks up to parent, etc., concatenating all. Also includes ~/.config/rony/AGENTS.md as global default.

Full details: pkg/persona/README.md

3.5 Memory (pkg/rag/)

Retrieval-Augmented Generation: persistent memory and semantic search.

type Memory interface {
    Add(ctx context.Context, fragment Fragment) error
    Search(ctx context.Context, query string, topK int) ([]Fragment, error)
    Forget(ctx context.Context, id string) error
}

type Fragment struct {
    ID         string
    Content    string
    Vector     []float32
    Metadata   map[string]string
    Timestamp  time.Time
    ProjectID  string
}

type Embedder interface {
    Embed(ctx context.Context, text string) ([]float32, error)
    Dimensions() int
}

Memory types:

Type What it stores Persistence
Working Current session messages RAM
Episodic Past events Vector DB
Semantic Consolidated knowledge Vector DB (curated)
Procedural Usage patterns Vector DB (auto-learned)

Supported backends:

  • ChromaDB embedded (default)
  • Qdrant embedded
  • SQLite + sqlite-vec

Full details: pkg/rag/README.md

3.6 Sandbox (pkg/tools/sandbox/)

Kernel-level filesystem sandbox using os.Root (Go 1.24+).

type Sandbox struct {
    workspace *os.Root        // Go 1.24+
    configDir *os.Root        // ~/.config/rony/
}

func (s *Sandbox) Open(path string) (*os.File, error)
func (s *Sandbox) Create(path string) (*os.File, error)
func (s *Sandbox) ReadFile(path string) ([]byte, error)
func (s *Sandbox) WriteFile(path string, data []byte, perm os.FileMode) error

Why os.Root (Go 1.24+) is security-critical:

Attack vector strings.HasPrefix (naive) os.Root
../../../etc/passwd Blocked if abs path doesn't have prefix Blocked by kernel
Symlink inside workspace → /etc/passwd Blocked only if we resolve manually Blocked natively
TOCTOU race condition Possible Impossible (kernel-checked)
Path encoding (%2e%2e) Not detected Detected by stdlib
Null bytes in path Depends on OS Handled by stdlib

🔒 That's why the library requires Go 1.26+ (see README §16.1.3.1).

3.7 Configuration (pkg/config/)

YAML loading with hierarchical precedence.

type Config struct {
    Model       string
    Persona     string
    Provider    ProviderConfig
    Tools       ToolPolicy
    Logging     LoggingConfig
    Sandbox     SandboxConfig
}

type Loader interface {
    Load(ctx context.Context, workdir string) (Config, error)
}

Precedence order (highest to lowest):

flags CLI > env vars > ./rony.yaml > ~/.config/rony/config.yaml > defaults

Full details: pkg/config/README.md

🔒 4. Security Model

4.1 Covered threats

Threat Library mitigation
Path traversal os.Root sandbox
Command injection Command validation (delegated to products using bash tool)
Resource exhaustion MaxIterations, MaxTokensPerSession, MaxToolOutputBytes
Prompt injection Wrap external content in <untrusted_content> tags (Phase 2)
Secret leakage Redact() function for logs (Phase 2)
Tool misuse Approval gates + permission policies

4.2 Principle of least privilege

  • Tools have Permission: Allow | Ask | Deny
  • Default is Allow only for read-only tools
  • Ask for tools that mutate state
  • Deny is rare, used for explicitly disabled tools

4.3 Defense in depth

Layer 1: Permission policy         → rejects unauthorized tools
Layer 2: Approval gate (Ask)       → user confirms before execution
Layer 3: Sandbox validation        → paths within allowed roots
Layer 4: Kernel enforcement (os.Root) → guarantee at OS level
Layer 5: Output truncation         → doesn't return huge outputs
Layer 6: Secret redaction          → doesn't expose secrets in logs/outputs

📐 5. Technical Specifications (SDD)

5.1 Core Functional Requirements

ID Requirement Implementation
LRF-001 LLMClient interface pkg/llm/
LRF-002 Multi-provider (OpenAI, Anthropic, Ollama, llama.cpp) pkg/llm/providers/
LRF-003 Native streaming iter.Seq2
LRF-004 Tool calling with JSON Schema pkg/tools/
LRF-005 Agent loop with guardrails pkg/agent/
LRF-006 Persona system + AGENTS.md pkg/persona/
LRF-007 RAG memory pkg/rag/
LRF-008 Filesystem sandbox (kernel) pkg/tools/sandbox/
LRF-009 Configuration with precedence pkg/config/
LRF-010 Approval gates pkg/agent/

5.2 Non-Functional Requirements

ID Category Target
LRNF-001 Minimum Go version 1.26 (due to os.Root)
LRNF-002 Test coverage ≥80%
LRNF-003 API stability Strict semver
LRNF-004 Minimal dependencies Only stdlib + provider SDKs
LRNF-005 Thread safety All public API is safe for concurrent use
LRNF-006 Context propagation Every callable takes context.Context

🧪 6. Testing

6.1 Mock LLM Server

For deterministic tests, products can use pkg/llm/mock:

import "github.com/VictorVargas/rony-llm-agent/pkg/llm/mock"

mockClient := mock.New(mock.Responses{
    {Match: "hello", Response: "Hi! How are you?"},
    {Match: "*",    Response: "default"},
})

6.2 Mock Memory

import "github.com/VictorVargas/rony-llm-agent/pkg/rag/mock"

memory := mock.NewMemory([]rag.Fragment{
    {Content: "doc1", ProjectID: "test"},
})

6.3 Critical tests table (entire library)

Test Package Priority
LLMClient.Generate returns valid response pkg/llm/ High
LLMClient.Stream emits chunks in order pkg/llm/ High
ToolRegistry.Register/Get/List pkg/tools/ High
Agent loop terminates without tool calls pkg/agent/ High
Agent loop terminates at max iterations pkg/agent/ High
Path sandbox rejects ../../../etc/passwd pkg/tools/sandbox/ High
Persona loader from YAML pkg/persona/ High
AGENTS.md discovery (walks up directories) pkg/persona/ Medium
Config precedence (env > file > defaults) pkg/config/ High
Memory Search returns top-K by similarity pkg/rag/ High

📚 7. Code conventions

7.1 Naming

  • Packages: lowercase, singular (agent, tools, llm)
  • Interfaces: end in noun or capability (Loop, LLMClient, Embedder)
  • Errors: ErrXXX for sentinels, wrapped with %w
  • Constructors: New for main constructor, NewXxx for variants

7.2 Errors

// Sentinel errors
var (
    ErrToolNotFound   = errors.New("tool not found")
    ErrSandboxViolation = errors.New("path outside allowed roots")
)

// Wrapping
if err != nil {
    return fmt.Errorf("loading persona %s: %w", name, err)
}

7.3 Context

Every function that can block takes ctx context.Context as first parameter:

func (l *Loop) Run(ctx context.Context, input string) (Response, error)
func (m *Memory) Search(ctx context.Context, query string, topK int) ([]Fragment, error)

🔄 8. Versioning

  • Strict semver (vMAJOR.MINOR.PATCH)
  • MAJOR: breaking changes in pkg/ (interfaces, signatures, public types)
  • MINOR: new features, new packages, new adapters
  • PATCH: bugfixes

Private adapters (pkg/llm/providers/openai/) can change without a MAJOR bump if the LLMClient interface doesn't change.

📖 9. References

🔗 Related documents