rony-chat-bot/docs/architecture.md

# 📋 Rony Chat Bot — Technical Design Document

**Version:** 1.0
**Author:** Victor Hugo Vargas
**Date:** 2026-06-28
**Status:** Complete specification for implementation
**Path:** `rony-chat-bot/docs/architecture.md`

> 🌐 **Language:** [English](./architecture.md) | [Español](./architecture.es.md)
>
> 📚 **Workspace:** This project is part of the `Rony/` workspace. See [`../README.md`](../../README.md).
>
> 🔑 **Depends on:** [`rony-llm-agent`](https://github.com/VictorVargas/rony-llm-agent) — core library that provides agent loop, LLM clients, RAG, persona system.
>
> 📐 **Methodology:** This project follows the **SDD + DDD + Hexagonal Architecture** approach. Functional Requirements are numbered as `CRF-XXX`. See [`../../METHODOLOGY.md`](../../METHODOLOGY.md).

---

## 🎯 1. Project Vision

### 1.1 What is Chat-Bot?

An **HTTP chatbot** that answers questions about Victor Hugo Vargas and his projects. Uses **RAG (Retrieval-Augmented Generation)** over markdown files describing each project, and a local LLM (or cloud) to generate responses.

### 1.2 Primary use case

Victor has a portfolio website (Astro + React). On the site there's a chat widget where visitors can ask:

- "What projects has Victor done?"
- "What's his experience with Go?"
- "How does Rony Harness work?"
- "Has Victor worked with PostgreSQL?"

The bot responds with accurate information extracted from the projects' markdown files + bio + skills.

### 1.3 Secondary use cases (future)

- **Client adaptation:** The same bot, with other data and another persona, serves car dealerships, restaurants, etc.
- **Standalone CLI:** `./chat-bot ask "what do you know about X?"` for terminal use.
- **Slack/Discord bot:** Wrapper that consumes the HTTP API.

### 1.4 Philosophy

- **Self-hosted by default** — works 100% local with Ollama + 1-3B models
- **Cloud optional** — if you need more quality, swap to Anthropic API
- **Portable** — easy to fork/customize for other contexts
- **Streaming** — token-by-token responses with SSE (no waiting for complete response)
- **Reuses `rony-llm-agent`** — doesn't reinvent the agent loop

---

## 🏗️ 2. Architecture

### 2.1 Overview

```
┌─────────────────────────────────────────────────────────────────┐
│  Browser (Astro site)                                            │
│      ↓ HTTP POST /api/chat                                       │
│  Astro SSR (proxy)  ←────────── Serves portfolio + proxy chat    │
│      ↓ HTTP POST /api/chat                                       │
│  Chat-Bot HTTP server (:7331)                                    │
│      ↓                                                           │
│  Agent loop (rony-llm-agent)                                       │
│      ↓                                                           │
│  RAG retrieval → ChromaDB over data/projects/*.md                │
│      ↓                                                           │
│  LLM (Ollama local / Anthropic cloud)                            │
└─────────────────────────────────────────────────────────────────┘
```

### 2.2 Main components

| Component | Path | Responsibility |
|---|---|---|
| **HTTP server** | `internal/server/` | Gin/chi handlers, SSE streaming |
| **Agent runner** | `internal/agent/` | Wrapper over `rony-llm-agent` with specific config |
| **Portfolio loader** | `internal/portfolio/` | Reads `data/projects/*.md`, indexes in ChromaDB |
| **Persona** | `internal/persona/` | Loads persona from `configs/portfolio-bot.yaml` |
| **CLI** | `cmd/chat-bot/` | Commands: `serve`, `reindex`, `ask`, `version` |

### 2.3 Tech stack

| Layer | Technology | Reason |
|---|---|---|
| **Language** | Go 1.26+ | Same as rony-harness, leverage `os.Root`, `iter.Seq` |
| **HTTP router** | `net/http` + `chi` | Stdlib + chi for middleware (CORS, logging) |
| **SSE** | `net/http` Flusher | Stdlib is enough, no external library needed |
| **Config** | `gopkg.in/yaml.v3` | Same as harness |
| **RAG backend** | ChromaDB embedded via `chroma-go` | Self-hosted, simple API |
| **Embeddings** | Ollama (nomic-embed-text) | Local, free, good quality |
| **LLM** | Ollama (qwen2.5:1.5b) or llama.cpp | Self-hosted by default |
| **Tests** | stdlib + testify | Consistency with the rest |

---

## 🔌 3. HTTP API

### 3.1 Endpoints

#### `POST /api/chat` — Chat with SSE streaming

**Request:**
```json
{
  "messages": [
    {"role": "user", "content": "What projects does Victor have?"}
  ],
  "stream": true
}
```

**Response (SSE):**
```
data: {"type":"start","conversation_id":"abc123"}

data: {"type":"chunk","content":"Victor"}
data: {"type":"chunk","content":" has"}
data: {"type":"chunk","content":" several"}
data: {"type":"chunk","content":" projects"}

data: {"type":"sources","documents":["rony-harness.md","rony-llm-agent.md"]}

data: {"type":"done","usage":{"input_tokens":245,"output_tokens":38}}
```

**Without streaming** (`"stream": false`):
```json
{
  "content": "Victor has several projects...",
  "sources": ["rony-harness.md", "rony-llm-agent.md"],
  "usage": {"input_tokens": 245, "output_tokens": 38}
}
```

#### `POST /api/reindex` — Re-index portfolio

Useful when files in `data/projects/` are modified.

**Request:** empty
**Response:**
```json
{
  "indexed_files": 12,
  "total_chunks": 87,
  "duration_ms": 4321
}
```

#### `GET /api/health` — Health check

```json
{
  "status": "ok",
  "version": "1.0.0",
  "providers": ["ollama-local"],
  "rag": {
    "documents": 12,
    "chunks": 87,
    "last_index": "2026-06-28T10:23:45Z"
  }
}
```

#### `GET /api/info` — Bot metadata

```json
{
  "name": "Rony Chat Bot",
  "model": "qwen2.5:1.5b",
  "persona": "...",
  "topics": ["projects", "experience", "technical skills"]
}
```

### 3.2 SSE Implementation

```go
// internal/server/chat.go
package server

import (
    "encoding/json"
    "fmt"
    "net/http"
    "github.com/VictorVargas/rony-llm-agent/pkg/agent"
)

func (s *Server) handleChat(w http.ResponseWriter, r *http.Request) {
    // SSE headers
    w.Header().Set("Content-Type", "text/event-stream")
    w.Header().Set("Cache-Control", "no-cache")
    w.Header().Set("Connection", "keep-alive")
    w.Header().Set("X-Accel-Buffering", "no")
    
    flusher, ok := w.(http.Flusher)
    if !ok {
        http.Error(w, "SSE not supported", http.StatusInternalServerError)
        return
    }
    
    // Parse request
    var req ChatRequest
    if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
        writeError(w, flusher, "invalid request", err)
        return
    }
    
    // Start event
    writeSSE(w, flusher, "start", map[string]string{
        "conversation_id": generateConvID(),
    })
    
    // Run agent with streaming
    sources := []string{}
    for chunk, err := range s.agent.RunStream(r.Context(), req.Messages) {
        if err != nil {
            writeSSE(w, flusher, "error", map[string]string{"message": err.Error()})
            return
        }
        if chunk.Type == "source" {
            sources = append(sources, chunk.Source)
        }
        writeSSE(w, flusher, chunk.Type, chunk.Data)
    }
    
    // Done event
    writeSSE(w, flusher, "done", map[string]any{
        "usage": map[string]int{
            "input_tokens":  245,
            "output_tokens": 38,
        },
    })
}

func writeSSE(w http.ResponseWriter, flusher http.Flusher, eventType string, data any) {
    payload, _ := json.Marshal(data)
    fmt.Fprintf(w, "data: {\"type\":%q,\"data\":%s}\n\n", eventType, payload)
    flusher.Flush()
}
```

### 3.3 Middleware

```go
// internal/server/middleware.go
package server

func (s *Server) loggingMiddleware(next http.Handler) http.Handler {
    return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        start := time.Now()
        // Wrap response writer to capture status
        rw := &statusRecorder{ResponseWriter: w, status: 200}
        next.ServeHTTP(rw, r)
        
        slog.Info("http.request",
            "method", r.Method,
            "path", r.URL.Path,
            "status", rw.status,
            "duration_ms", time.Since(start).Milliseconds(),
            "ip", r.RemoteAddr,
        )
    })
}

func (s *Server) corsMiddleware(next http.Handler) http.Handler {
    return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        origin := r.Header.Get("Origin")
        for _, allowed := range s.config.Server.CORSOrigins {
            if origin == allowed {
                w.Header().Set("Access-Control-Allow-Origin", origin)
                w.Header().Set("Access-Control-Allow-Methods", "POST, GET, OPTIONS")
                w.Header().Set("Access-Control-Allow-Headers", "Content-Type")
                break
            }
        }
        if r.Method == "OPTIONS" {
            w.WriteHeader(204)
            return
        }
        next.ServeHTTP(w, r)
    })
}

func (s *Server) rateLimitMiddleware(next http.Handler) http.Handler {
    limiter := rate.NewLimiter(rate.Every(time.Minute/time.Duration(s.config.Server.RateLimit.RequestsPerMinute)), s.config.Server.RateLimit.Burst)
    return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        if !limiter.Allow() {
            http.Error(w, "rate limit exceeded", http.StatusTooManyRequests)
            return
        }
        next.ServeHTTP(w, r)
    })
}
```

---

## 🧠 4. RAG (Retrieval-Augmented Generation)

### 4.1 Indexing pipeline

```
data/projects/*.md
    ↓ (read all files)
Raw markdown content
    ↓ (split into chunks, ~500 chars, 50 overlap)
Chunks []
    ↓ (embed each chunk via Ollama nomic-embed-text)
Vectors [][]float32
    ↓ (store in ChromaDB collection "portfolio")
Indexed corpus
```

**When it runs:**
- On bot startup (if `--reindex-on-start` flag)
- Manually: `./chat-bot reindex`
- Via HTTP: `POST /api/reindex`

### 4.2 Retrieval pipeline

```
User query "what projects does Victor have?"
    ↓ (embed query)
Query vector
    ↓ (cosine similarity search in ChromaDB, top_k=5)
Top 5 relevant chunks
    ↓ (format as context block)
System prompt += relevant chunks
    ↓ (send to LLM)
LLM generates answer
```

### 4.3 Implementation

```go
// internal/portfolio/indexer.go
package portfolio

import (
    "context"
    "os"
    "path/filepath"
    "strings"
    "github.com/VictorVargas/rony-llm-agent/pkg/rag"
)

type Indexer struct {
    dataPath string
    memory   rag.Memory
    embedder rag.Embedder
    chunkSize int
    chunkOverlap int
}

func (i *Indexer) IndexAll(ctx context.Context) (int, error) {
    files, err := filepath.Glob(filepath.Join(i.dataPath, "*.md"))
    if err != nil {
        return 0, err
    }
    
    totalChunks := 0
    for _, file := range files {
        chunks, err := i.indexFile(ctx, file)
        if err != nil {
            slog.Warn("failed to index file", "file", file, "err", err)
            continue
        }
        totalChunks += chunks
    }
    
    return totalChunks, nil
}

func (i *Indexer) indexFile(ctx context.Context, path string) (int, error) {
    content, err := os.ReadFile(path)
    if err != nil {
        return 0, err
    }
    
    projectID := strings.TrimSuffix(filepath.Base(path), ".md")
    chunks := splitIntoChunks(string(content), i.chunkSize, i.chunkOverlap)
    
    for idx, chunk := range chunks {
        embedding, err := i.embedder.Embed(ctx, chunk)
        if err != nil {
            return idx, err
        }
        
        fragment := rag.Fragment{
            ID:        fmt.Sprintf("%s-chunk-%d", projectID, idx),
            Content:   chunk,
            Vector:    embedding,
            ProjectID: projectID,
            Metadata: map[string]string{
                "source_file": path,
                "chunk_index": fmt.Sprint(idx),
            },
        }
        
        if err := i.memory.Add(ctx, fragment); err != nil {
            return idx, err
        }
    }
    
    return len(chunks), nil
}

func splitIntoChunks(text string, size, overlap int) []string {
    // Simple implementation: split by size with overlap
    // Production version uses tokenizer-aware chunking
    var chunks []string
    for i := 0; i < len(text); i += size - overlap {
        end := i + size
        if end > len(text) {
            end = len(text)
        }
        chunks = append(chunks, text[i:end])
    }
    return chunks
}
```

### 4.4 Retrieval in the agent loop

```go
// internal/agent/runner.go
package agent

func (r *Runner) buildSystemPrompt(ctx context.Context, query string) (string, error) {
    // 1. Base persona prompt
    basePrompt := r.persona.SystemPrompt
    
    // 2. Retrieve relevant chunks
    fragments, err := r.memory.Search(ctx, query, r.config.RAG.TopK)
    if err != nil {
        return "", err
    }
    
    // 3. Format as context
    var contextBlock strings.Builder
    contextBlock.WriteString(basePrompt)
    contextBlock.WriteString("\n\n## Relevant context\n\n")
    for idx, frag := range fragments {
        contextBlock.WriteString(fmt.Sprintf("### Source: %s\n%s\n\n", 
            frag.Metadata["source_file"], frag.Content))
    }
    
    return contextBlock.String(), nil
}

func (r *Runner) RunStream(ctx context.Context, messages []llm.Message) iter.Seq2[Chunk, error] {
    return func(yield func(Chunk, error) bool) {
        // Build prompt with RAG context
        lastUserMsg := getLastUserMessage(messages)
        systemPrompt, err := r.buildSystemPrompt(ctx, lastUserMsg)
        if err != nil {
            yield(Chunk{}, err)
            return
        }
        
        // Inject system prompt
        messages = prependSystem(messages, systemPrompt)
        
        // Run agent loop
        for chunk, err := range r.loop.RunStream(ctx, messages) {
            if !yield(chunk, err) {
                return
            }
        }
    }
}
```

---

## 🌐 5. Integration with Astro (Portfolio)

### 5.1 Recommended pattern: Astro proxy

```
[Browser] ←→ [Astro SSR :4321] ←→ [Chat-Bot :7331]
```

**Why proxy and not direct browser call to chat-bot:**
- ✅ Single domain (no CORS)
- ✅ Astro handles auth/session if needed
- ✅ There can be centralized rate limiting in Astro
- ✅ The chat-bot stays on private network (not exposed to internet directly)

### 5.2 Astro: API route of the proxy

```typescript
// portfolio/src/pages/api/chat.ts
import type { APIRoute } from 'astro';

const CHAT_BOT_URL = import.meta.env.CHAT_BOT_URL || 'http://localhost:7331';

export const POST: APIRoute = async ({ request }) => {
    const body = await request.json();
    const resp = await fetch(`${CHAT_BOT_URL}/api/chat`, {
        method: 'POST',
        headers: { 'Content-Type': 'application/json' },
        body: JSON.stringify(body),
    });
    
    if (!resp.ok) {
        return new Response('Chat bot error', { status: resp.status });
    }
    
    // Stream SSE back to browser
    return new Response(resp.body, {
        status: 200,
        headers: {
            'Content-Type': 'text/event-stream',
            'Cache-Control': 'no-cache',
            'Connection': 'keep-alive',
        },
    });
};
```

### 5.3 React: Chat component

```tsx
// portfolio/src/components/Chat.tsx
import { useState, useRef } from 'react';

interface Message {
    role: 'user' | 'assistant';
    content: string;
}

export default function Chat() {
    const [messages, setMessages] = useState<Message[]>([]);
    const [input, setInput] = useState('');
    const [streaming, setStreaming] = useState(false);
    const abortRef = useRef<AbortController | null>(null);
    
    const send = async () => {
        if (!input.trim() || streaming) return;
        
        const userMsg: Message = { role: 'user', content: input };
        setMessages(prev => [...prev, userMsg]);
        setInput('');
        setStreaming(true);
        
        // Placeholder for streaming
        const assistantMsg: Message = { role: 'assistant', content: '' };
        setMessages(prev => [...prev, assistantMsg]);
        
        abortRef.current = new AbortController();
        
        try {
            const resp = await fetch('/api/chat', {
                method: 'POST',
                headers: { 'Content-Type': 'application/json' },
                body: JSON.stringify({ 
                    messages: [...messages, userMsg],
                    stream: true,
                }),
                signal: abortRef.current.signal,
            });
            
            const reader = resp.body!.getReader();
            const decoder = new TextDecoder();
            let buffer = '';
            
            while (true) {
                const { done, value } = await reader.read();
                if (done) break;
                
                buffer += decoder.decode(value, { stream: true });
                const lines = buffer.split('\n\n');
                buffer = lines.pop() || '';
                
                for (const line of lines) {
                    if (!line.startsWith('data: ')) continue;
                    const event = JSON.parse(line.slice(6));
                    
                    if (event.type === 'chunk') {
                        setMessages(prev => {
                            const updated = [...prev];
                            updated[updated.length - 1].content += event.data.content;
                            return updated;
                        });
                    }
                }
            }
        } catch (err) {
            if ((err as Error).name !== 'AbortError') {
                console.error(err);
            }
        } finally {
            setStreaming(false);
            abortRef.current = null;
        }
    };
    
    const stop = () => abortRef.current?.abort();
    
    return (
        <div className="chat-widget">
            <div className="messages">
                {messages.map((m, i) => (
                    <div key={i} className={`msg msg-${m.role}`}>
                        {m.content || (streaming && i === messages.length - 1 ? '...' : '')}
                    </div>
                ))}
            </div>
            <div className="input-row">
                <input
                    value={input}
                    onChange={e => setInput(e.target.value)}
                    onKeyDown={e => e.key === 'Enter' && send()}
                    placeholder="Ask about Victor..."
                    disabled={streaming}
                />
                {streaming ? (
                    <button onClick={stop}>Stop</button>
                ) : (
                    <button onClick={send}>Send</button>
                )}
            </div>
        </div>
    );
}
```

---

## 🤖 6. Self-hosting with Ollama

### 6.1 Setup

```bash
# 1. Install Ollama
curl -fsSL https://ollama.com/install.sh | sh

# 2. Download chat model
ollama pull qwen2.5:1.5b

# 3. Download embeddings model
ollama pull nomic-embed-text

# 4. Verify
ollama list
```

### 6.2 Default configuration

`configs/portfolio-bot.yaml` already comes with Ollama as default. You only need:

```bash
# Make sure Ollama is running
ollama serve

# Start the bot
./bin/chat-bot serve
```

### 6.3 Alternative: llama.cpp direct

For more control or if Ollama doesn't work in your setup:

```yaml
providers:
  - name: llamacpp-local
    type: llamacpp
    model_path: ${RONY_MODELS_PATH}/qwen2.5-1.5b-instruct-q5_k_m.gguf
    context_size: 4096
    n_gpu_layers: 999   # offload all to GPU
    default: true
```

The `llamacpp` adapter is imported from `rony-llm-agent/pkg/llm/providers/llamacpp` and is compiled against `llama.cpp` via CGO or external binary.

---

## 📦 7. Bot CLI

### 7.1 Commands

```bash
# Start HTTP server
chat-bot serve [--port 7331] [--host 0.0.0.0] [--reindex-on-start]

# Re-index portfolio (reads data/projects/*.md → ChromaDB)
chat-bot reindex

# Single question (no server, useful for tests)
chat-bot ask "What projects does Victor have?" [--no-rag]

# Validate config
chat-bot config validate

# Health check (useful for monitoring)
chat-bot health

# Version
chat-bot version
```

### 7.2 Implementation with Cobra

```go
// cmd/chat-bot/main.go
package main

import (
    "github.com/spf13/cobra"
)

func main() {
    root := &cobra.Command{
        Use:   "chat-bot",
        Short: "Portfolio chatbot HTTP server",
    }
    
    root.AddCommand(serveCmd())
    root.AddCommand(reindexCmd())
    root.AddCommand(askCmd())
    root.AddCommand(configCmd())
    root.AddCommand(healthCmd())
    root.AddCommand(versionCmd())
    
    if err := root.Execute(); err != nil {
        os.Exit(1)
    }
}

func serveCmd() *cobra.Command {
    var port int
    var host string
    var reindexOnStart bool
    
    cmd := &cobra.Command{
        Use:   "serve",
        Short: "Start HTTP server",
        RunE: func(cmd *cobra.Command, args []string) error {
            return server.Serve(server.Config{
                Port:           port,
                Host:           host,
                ReindexOnStart: reindexOnStart,
            })
        },
    }
    
    cmd.Flags().IntVar(&port, "port", 7331, "HTTP port")
    cmd.Flags().StringVar(&host, "host", "0.0.0.0", "HTTP host")
    cmd.Flags().BoolVar(&reindexOnStart, "reindex-on-start", false, "Re-index RAG before serving")
    
    return cmd
}
```

---

## 🚀 8. Deployment

### 8.1 Recommendation: Self-hosted on VPS

```bash
# 1. Install dependencies
sudo apt install golang-go ollama
ollama pull qwen2.5:1.5b
ollama pull nomic-embed-text

# 2. Build
go build -o /usr/local/bin/chat-bot ./cmd/chat-bot

# 3. systemd service
cat > /etc/systemd/system/chat-bot.service <<EOF
[Unit]
Description=Portfolio Chat Bot
After=network.target ollama.service

[Service]
Type=simple
User=chatbot
WorkingDirectory=/opt/chat-bot
ExecStart=/usr/local/bin/chat-bot serve
Restart=on-failure
Environment=RONY_MODELS_PATH=/opt/models

[Install]
WantedBy=multi-user.target
EOF

sudo systemctl enable --now chat-bot
```

### 8.2 Reverse proxy (Caddy)

```
# /etc/caddy/Caddyfile
chat.victorvargas.dev {
    reverse_proxy localhost:7331
}
```

### 8.3 Monitoring

```bash
# Health check periodic
curl -s http://localhost:7331/api/health | jq

# Logs
journalctl -u chat-bot -f
```

---

## 🧪 9. Testing

### 9.1 Unit tests

```go
// internal/server/chat_test.go
package server

func TestHandleChat_ValidRequest(t *testing.T) {
    s := newTestServer(t)
    
    req := httptest.NewRequest("POST", "/api/chat", strings.NewReader(`{
        "messages": [{"role": "user", "content": "hello"}]
    }`))
    req.Header.Set("Content-Type", "application/json")
    
    w := httptest.NewRecorder()
    s.handleChat(w, req)
    
    assert.Equal(t, 200, w.Code)
    assert.Equal(t, "text/event-stream", w.Header().Get("Content-Type"))
}

func TestHandleChat_RateLimit(t *testing.T) {
    s := newTestServerWithConfig(t, server.Config{
        RateLimit: 1, // 1 request per minute
    })
    
    // First request OK
    req1 := newChatRequest("hello")
    w1 := httptest.NewRecorder()
    s.handleChat(w1, req1)
    assert.Equal(t, 200, w1.Code)
    
    // Second request denied
    req2 := newChatRequest("hello again")
    w2 := httptest.NewRecorder()
    s.handleChat(w2, req2)
    assert.Equal(t, 429, w2.Code)
}
```

### 9.2 Integration tests with mock LLM

```go
// internal/agent/runner_test.go
func TestRunner_RAGContextIsInjected(t *testing.T) {
    mockLLM := mock.New(mock.Responses{
        {Match: "projects", Response: "Victor has several projects..."},
    })
    
    memory := newMockMemoryWithDocs(t, []rag.Fragment{
        {Content: "Rony Harness: AI agent harness...", ProjectID: "rony-harness"},
        {Content: "rony-llm-agent: Go library...", ProjectID: "rony-llm-agent"},
    })
    
    runner := agent.NewRunner(agent.Config{
        LLM:    mockLLM,
        Memory: memory,
        Persona: testPersona,
    })
    
    resp, _ := runner.Run(context.Background(), []llm.Message{
        {Role: llm.RoleUser, Content: "what projects does Victor have?"},
    })
    
    // Verify LLM received context chunks in system prompt
    lastReq := mockLLM.LastRequest()
    assert.Contains(t, lastReq.Messages[0].Content, "Rony Harness")
    assert.Contains(t, lastReq.Messages[0].Content, "rony-llm-agent")
}
```

### 9.3 E2E test with Astro

```bash
# 1. Start chat-bot on :7331
./bin/chat-bot serve &

# 2. Start Astro on :4321
cd ../portfolio && npm run dev &

# 3. Make request to Astro's proxy
curl -X POST http://localhost:4321/api/chat \
  -H "Content-Type: application/json" \
  -d '{"messages":[{"role":"user","content":"hello"}]}'

# 4. Verify SSE stream
```

---

## 📂 10. Project Structure

```
rony-chat-bot/
├── cmd/
│   └── chat-bot/
│       └── main.go                 # CLI entrypoint
│
├── internal/
│   ├── server/                     # HTTP handlers
│   │   ├── chat.go                 # POST /api/chat
│   │   ├── reindex.go              # POST /api/reindex
│   │   ├── health.go               # GET /api/health
│   │   ├── info.go                 # GET /api/info
│   │   ├── middleware.go           # logging, CORS, rate limit
│   │   └── sse.go                  # SSE helpers
│   │
│   ├── agent/                      # Wrapper over rony-llm-agent
│   │   ├── runner.go               # RunStream with RAG injection
│   │   └── prompts.go              # System prompt builder
│   │
│   ├── portfolio/                  # Data loader
│   │   ├── indexer.go              # Reads .md, chunks, embed, store
│   │   ├── retriever.go            # Query → top-k chunks
│   │   └── chunker.go              # Text splitting
│   │
│   └── persona/                    # Persona override
│       └── loader.go               # Loads persona from YAML
│
├── data/
│   └── projects/                   # ← Markdown per project
│       ├── rony-harness.md
│       ├── rony-llm-agent.md
│       └── example-project.md
│
├── configs/
│   └── portfolio-bot.yaml          # Provider + RAG + persona config
│
├── docs/
│   └── architecture.md             # ← THIS FILE
│
├── go.mod
└── README.md
```

---

## 📅 11. Roadmap

### Phase 1: MVP (2-3 weeks)

- [ ] Project setup (`go mod init`, structure)
- [ ] Basic HTTP server with `/api/chat` endpoint
- [ ] Functional SSE streaming
- [ ] RAG indexer (reads `data/projects/*.md` → ChromaDB)
- [ ] RAG retriever (query → top-k chunks)
- [ ] Persona loader from YAML
- [ ] Ollama integration (qwen2.5:1.5b)
- [ ] CLI: `serve`, `reindex`, `ask`
- [ ] Basic tests

### Phase 2: Integration with Astro (1 week)

- [ ] Astro API route of the proxy
- [ ] React component of the chat widget
- [ ] E2E test: Astro → chat-bot → response
- [ ] Widget styling (TailwindCSS)

### Phase 3: Polish (1 week)

- [ ] Robust rate limiting
- [ ] Structured logging (JSON)
- [ ] Health checks for monitoring
- [ ] systemd service file
- [ ] README + deployment docs

### Phase 4: Optionals

- [ ] Support for multiple conversations (session ID)
- [ ] Persisted chat history
- [ ] Analysis of frequent questions
- [ ] Multi-language (EN/ES switch)
- [ ] More polished standalone CLI version (`chat-bot ask`)

---

## 📐 12. Quality Specifications

### 12.1 Performance metrics

| Metric | Target |
|---|---|
| TTFT (Time-to-first-token) | <500ms with Ollama local |
| End-to-end (question → complete response) | <3s for typical responses |
| Memory at rest | <150MB |
| RAG indexing speed | ~100 docs/second |
| Retrieval latency | <50ms for top-5 |

### 12.2 Required tests

- Unit tests: coverage ≥70%
- Integration tests: with mock LLM + mock ChromaDB
- E2E: at least one complete Astro → chat-bot flow

---

## 🔒 13. Security

### 13.1 Implemented

- **Rate limiting** per IP (default 30 req/min)
- **Restrictive CORS** — only configured origins
- **Input validation** — JSON schema validation on requests
- **No PII storage** — we don't save conversations by default
- **Local-only by default** — no calls to cloud APIs

### 13.2 Deferred / Optional

- Auth with API key (for private use)
- Query logging for analytics
- IP anonymization in logs
- HTTPS via reverse proxy (Caddy/nginx)

---

## 📚 14. References

- **SSE Spec:** https://html.spec.whatwg.org/multipage/server-sent-events.html
- **Ollama API:** https://github.com/ollama/ollama/blob/main/docs/api.md
- **ChromaDB Go:** https://github.com/amikos-tech/chroma-go
- **nomic-embed-text:** https://huggingface.co/nomic-ai/nomic-embed-text-v1.5
- **qwen2.5:** https://huggingface.co/Qwen/Qwen2.5-1.5B-Instruct
- **Astro API routes:** https://docs.astro.build/en/guides/endpoints/
- **rony-llm-agent:** https://github.com/VictorVargas/rony-llm-agent

---

**Document ready for implementation. 🚀**