feat: bootstrap rony-chat-bot Go module

Initial implementation of the bot:

- cmd/chat-bot: CLI entrypoint (serve, reindex, ask, version)
- internal/agent: LLM provider client + agent runner with RAG injection
- internal/config: YAML config loader (providers, RAG, persona, server)
- internal/i18n: response-language detection (EN/ES)
- internal/persona: persona system prompt assembly from YAML
- internal/portfolio: heading-based chunker + SQLite FTS5 indexer
- internal/server: chi router with /api/chat (SSE), /api/health, /api/info,
  /api/reindex, middleware (RequestID, Logging, CORS, RateLimit)
- internal/streaming: SSE protocol helpers (start, chunk, sources, done, error)
- web/: drop-in vanilla-JS chat widget (no build, no deps) + demo + README
- bench/: reproducible driver benchmark (modernc vs mattn SQLite)
- configs/portfolio-bot.yaml: llama.cpp default provider, SQLite RAG, canine persona
- docs/architecture.md / .es.md: aligned with SQLite FTS5 + llama.cpp decisions
- data/projects/README*.md: project data documentation
- README.md / .es.md: updated for current implementation

All tests pass (go test ./...). Bot is functional end-to-end with the
configured LLM provider.
This commit is contained in:
Victor Hugo Vargas 2026-07-17 00:56:06 -07:00
parent 3eb0574071
commit f33708534a
38 changed files with 4977 additions and 656 deletions

4
.gitignore vendored
View file

@ -7,11 +7,11 @@ vendor/
coverage.out coverage.out
coverage.html coverage.html
# ChromaDB # SQLite (RAG index)
chroma/
*.db *.db
*.db-shm *.db-shm
*.db-wal *.db-wal
data/portfolio.db
# Editor / OS # Editor / OS
.vscode/ .vscode/

View file

@ -10,10 +10,10 @@
## ✨ Features ## ✨ Features
- 🌐 **HTTP server** con streaming SSE (Server-Sent Events) - 🌐 **HTTP server** con streaming SSE (Server-Sent Events)
- 🧠 **RAG sobre markdown** — indexa automáticamente los `.md` en `data/projects/` - 🧠 **RAG sobre markdown** — indexa automáticamente los `.md` en `data/projects/` (SQLite FTS5, sin embeddings)
- 🎭 **Persona customizable** — responde como "asistente de Victor" - 🎭 **Persona customizable** — responde como "asistente de Victor"
- ⚡ **Self-hosted** con Ollama o llama.cpp (no requiere API key de cloud) - ⚡ **Self-hosted** con llama.cpp (default) u Ollama (no requiere API key de cloud)
- 🔌 **Integrable** con Astro/React via proxy HTTP - 💬 **Widget de chat drop-in** — vanilla JS, sin build step, funciona en cualquier sitio
- 🛡️ **Rate limiting** y logging estructurado - 🛡️ **Rate limiting** y logging estructurado
- 📦 **Portable** — se puede adaptar a otros contextos (clientes, productos, etc.) - 📦 **Portable** — se puede adaptar a otros contextos (clientes, productos, etc.)
@ -27,9 +27,10 @@ cd chat-bot
# 2. Resolver dependencias (crea go.sum con hashes) # 2. Resolver dependencias (crea go.sum con hashes)
go mod tidy go mod tidy
# 3. Configurar provider (ejemplo: Ollama) # 3. Configurar provider (llama.cpp por default)
# Asegúrate de tener Ollama corriendo: ollama serve # Descarga un modelo GGUF, ej.:
# Modelo descargado: ollama pull qwen2.5:1.5b # https://huggingface.co/Qwen/Qwen2.5-1.5B-Instruct-GGUF
export RONY_MODELS_PATH=/path/to/models
# 4. Cargar tus proyectos en data/projects/ # 4. Cargar tus proyectos en data/projects/
echo "# Mi Proyecto Cool\nDescripción..." > data/projects/mi-proyecto.md echo "# Mi Proyecto Cool\nDescripción..." > data/projects/mi-proyecto.md
@ -46,41 +47,45 @@ go build -o bin/chat-bot ./cmd/chat-bot
``` ```
rony-chat-bot/ rony-chat-bot/
├── cm./rony-chat-bot/ # Entry point (CLI) ├── cmd/chat-bot/ # Entry point (CLI)
├── internal/ ├── internal/
│ ├── server/ # HTTP handlers + SSE │ ├── server/ # HTTP handlers + SSE
│ ├── agent/ # LLM client + RAG + persona runner
│ ├── portfolio/ # Data loader (markdown → RAG) │ ├── portfolio/ # Data loader (markdown → RAG)
│ ├── persona/ # Persona override │ ├── persona/ # Persona override
│ └── streaming/ # SSE helpers │ ├── streaming/ # SSE helpers
│ └── i18n/ # Detección de idioma (EN/ES)
├── web/ # ← WIDGET DE CHAT DROP-IN
│ ├── chat-widget.js
│ ├── chat-widget.css
│ └── example.html
├── data/projects/ # ← TUS PROYECTOS EN MARKDOWN ├── data/projects/ # ← TUS PROYECTOS EN MARKDOWN
│ ├── rony-tui.md │ ├── rony-tui.md
│ ├── rony-llm-agent.md │ ├── rony-llm-agent.md
│ └── ... │ └── ...
├── configs/ ├── configs/
│ └── portfolio-bot.yaml # Provider config │ └── portfolio-bot.yaml # Provider + RAG + persona config
├── docs/ ├── docs/
│ └── architecture.md # ← Especificación técnica completa │ └── architecture.md # ← Especificación técnica completa
└── go.mod # require rony-llm-agent └── go.mod # require rony-llm-agent
``` ```
## 🎯 Uso desde Astro ## 🎯 Embebido en cualquier sitio
Ver [`docs/architecture.md`](./docs/architecture.md) §5 — patrón recomendado de proxy. El bot viene con un widget de chat drop-in. Agrega dos archivos y un tag `<script>`:
```typescript ```html
// portfolio/src/pages/api/chat.ts <link rel="stylesheet" href="/chat-widget.css">
export const POST: APIRoute = async ({ request }) => { <script src="/chat-widget.js"
const body = await request.json(); data-api-url="https://chat.example.com"
const resp = await fetch('http://localhost:7331/api/chat', { data-title="Pregúntame lo que sea"
method: 'POST', data-position="bottom-right"
body: JSON.stringify(body), data-theme="auto"
}); defer></script>
return new Response(resp.body, {
headers: { 'Content-Type': 'text/event-stream' },
});
};
``` ```
Ver [`web/README.md`](./web/README.md) para la referencia completa de configuración y snippets de integración con Astro/Next.js. Arquitectura completa en [`docs/architecture.md`](./docs/architecture.md) §5.
## 🔄 Adaptar a otro cliente ## 🔄 Adaptar a otro cliente
Este bot está diseñado para ser **atómico** y reusable. Para adaptarlo (ej. chatbot para un concesionario): Este bot está diseñado para ser **atómico** y reusable. Para adaptarlo (ej. chatbot para un concesionario):

View file

@ -9,10 +9,10 @@
## ✨ Features ## ✨ Features
- 🌐 **HTTP server** with SSE (Server-Sent Events) streaming - 🌐 **HTTP server** with SSE (Server-Sent Events) streaming
- 🧠 **RAG over markdown** — automatically indexes `.md` in `data/projects/` - 🧠 **RAG over markdown** — automatically indexes `.md` in `data/projects/` (SQLite FTS5, no embeddings)
- 🎭 **Customizable persona** — responds as "Victor's assistant" - 🎭 **Customizable persona** — responds as "Victor's assistant"
- ⚡ **Self-hosted** with Ollama or llama.cpp (no cloud API key required) - ⚡ **Self-hosted** with llama.cpp (default) or Ollama (no cloud API key required)
- 🔌 **Integrable** with Astro/React via HTTP proxy - 💬 **Drop-in chat widget** — vanilla JS, no build step, works in any site
- 🛡️ **Rate limiting** and structured logging - 🛡️ **Rate limiting** and structured logging
- 📦 **Portable** — adaptable to other contexts (clients, products, etc.) - 📦 **Portable** — adaptable to other contexts (clients, products, etc.)
@ -26,9 +26,10 @@ cd rony-chat-bot
# 2. Resolve dependencies (creates go.sum with hashes) # 2. Resolve dependencies (creates go.sum with hashes)
go mod tidy go mod tidy
# 3. Configure provider (e.g., Ollama) # 3. Configure provider (llama.cpp by default)
# Make sure Ollama is running: ollama serve # Download a GGUF model, e.g.:
# Downloaded model: ollama pull qwen2.5:1.5b # https://huggingface.co/Qwen/Qwen2.5-1.5B-Instruct-GGUF
export RONY_MODELS_PATH=/path/to/models
# 4. Load your projects in data/projects/ # 4. Load your projects in data/projects/
echo "# My Cool Project\nDescription..." > data/projects/my-project.md echo "# My Cool Project\nDescription..." > data/projects/my-project.md
@ -48,44 +49,42 @@ rony-chat-bot/
├── cmd/chat-bot/ # Entry point (CLI) ├── cmd/chat-bot/ # Entry point (CLI)
├── internal/ ├── internal/
│ ├── server/ # HTTP handlers + SSE │ ├── server/ # HTTP handlers + SSE
│ ├── agent/ # LLM client + RAG + persona runner
│ ├── portfolio/ # Data loader (markdown → RAG) │ ├── portfolio/ # Data loader (markdown → RAG)
│ ├── persona/ # Persona override │ ├── persona/ # Persona override
│ └── streaming/ # SSE helpers │ ├── streaming/ # SSE helpers
│ └── i18n/ # Language detection (EN/ES)
├── web/ # ← DROP-IN CHAT WIDGET
│ ├── chat-widget.js
│ ├── chat-widget.css
│ └── example.html
├── data/projects/ # ← YOUR PROJECTS IN MARKDOWN ├── data/projects/ # ← YOUR PROJECTS IN MARKDOWN
│ ├── rony-harness.md │ ├── rony-harness.md
│ ├── rony-llm-agent.md │ ├── rony-llm-agent.md
│ └── ... │ └── ...
├── configs/ ├── configs/
│ └── portfolio-bot.yaml # Provider config │ └── portfolio-bot.yaml # Provider + RAG + persona config
├── docs/ ├── docs/
│ └── architecture.md # ← Complete technical specification │ └── architecture.md # ← Complete technical specification
└── go.mod # require rony-llm-agent └── go.mod # require rony-llm-agent
``` ```
## 🎯 Use from Astro ## 🎯 Embed in any site
See [`docs/architecture.md`](./docs/architecture.md) §5 — recommended proxy pattern. The bot ships with a drop-in chat widget. Add two files and a `<script>` tag:
```typescript ```html
// portfolio/src/pages/api/chat.ts <link rel="stylesheet" href="/chat-widget.css">
export const POST: APIRoute = async ({ request }) => { <script src="/chat-widget.js"
const body = await request.json(); data-api-url="https://chat.example.com"
const resp = await fetch('http://localhost:7331/api/chat', { data-title="Ask me anything"
method: 'POST', data-position="bottom-right"
headers: { 'Content-Type': 'application/json' }, data-theme="auto"
body: JSON.stringify(body), defer></script>
});
return new Response(resp.body, {
status: 200,
headers: {
'Content-Type': 'text/event-stream',
'Cache-Control': 'no-cache',
'Connection': 'keep-alive',
},
});
};
``` ```
See [`web/README.md`](./web/README.md) for the full configuration reference and Astro/Next.js integration snippets. Full architecture in [`docs/architecture.md`](./docs/architecture.md) §5.
## 🔄 Adapt to another client ## 🔄 Adapt to another client
This bot is designed to be **atomic** and reusable. To adapt it (e.g., chatbot for a car dealership): This bot is designed to be **atomic** and reusable. To adapt it (e.g., chatbot for a car dealership):

154
bench/bench.go Normal file
View file

@ -0,0 +1,154 @@
// Package bench holds one-time validation benchmarks used to drive library
// decisions. Run with:
//
// go test -bench=. ./bench/ (modernc only, pure Go)
// CGO_ENABLED=1 go test -bench=. ./bench/ (mattn + modernc, requires gcc)
package bench
import (
"context"
"database/sql"
"fmt"
"os"
"path/filepath"
"strings"
"testing"
"time"
)
const schema = `
CREATE VIRTUAL TABLE IF NOT EXISTS portfolio_chunks USING fts5(
id UNINDEXED,
project_id UNINDEXED,
source_file UNINDEXED,
chunk_index UNINDEXED,
content,
tokenize = 'unicode61 remove_diacritics 2'
);
`
var queries = []string{
"rony-llm-agent",
"AI agent harness",
"portfolio projects",
"machine learning",
"CLI tool for development",
"vector database",
"agent loop",
"streaming response",
}
func loadChunks(tb testing.TB, dataPath string) []chunkRow {
tb.Helper()
files, err := filepath.Glob(filepath.Join(dataPath, "*.md"))
if err != nil {
tb.Fatal(err)
}
var rows []chunkRow
for _, f := range files {
b, err := os.ReadFile(f)
if err != nil {
tb.Fatal(err)
}
project := strings.TrimSuffix(filepath.Base(f), ".md")
for i, c := range splitIntoChunks(string(b), 500, 50) {
rows = append(rows, chunkRow{
ID: fmt.Sprintf("%s-chunk-%d", project, i),
ProjectID: project,
Source: f,
Index: i,
Content: c,
})
}
}
return rows
}
type chunkRow struct {
ID, ProjectID, Source string
Index int
Content string
}
func splitIntoChunks(text string, size, overlap int) []string {
if size <= 0 {
return []string{text}
}
if overlap < 0 || overlap >= size {
overlap = size / 10
}
var chunks []string
for i := 0; i < len(text); i += size - overlap {
end := i + size
if end > len(text) {
end = len(text)
}
if i >= end {
break
}
chunks = append(chunks, text[i:end])
if end == len(text) {
break
}
}
return chunks
}
func insertAll(ctx context.Context, db *sql.DB, rows []chunkRow) error {
if _, err := db.ExecContext(ctx, `DELETE FROM portfolio_chunks`); err != nil {
return err
}
tx, err := db.BeginTx(ctx, nil)
if err != nil {
return err
}
defer tx.Rollback()
stmt, err := tx.PrepareContext(ctx,
`INSERT INTO portfolio_chunks (id, project_id, source_file, chunk_index, content) VALUES (?,?,?,?,?)`)
if err != nil {
return err
}
defer stmt.Close()
for _, r := range rows {
if _, err := stmt.ExecContext(ctx, r.ID, r.ProjectID, r.Source, r.Index, r.Content); err != nil {
return err
}
}
return tx.Commit()
}
func runQuery(ctx context.Context, db *sql.DB, q string) (int, time.Duration, error) {
start := time.Now()
rows, err := db.QueryContext(ctx, fmt.Sprintf(`
SELECT project_id, source_file, content
FROM portfolio_chunks
WHERE portfolio_chunks MATCH '%s'
ORDER BY bm25(portfolio_chunks)
LIMIT 5
`, sanitizeFTS5(q)))
if err != nil {
return 0, 0, err
}
defer rows.Close()
n := 0
for rows.Next() {
n++
}
return n, time.Since(start), rows.Err()
}
// sanitizeFTS5 is the same simple wrapper used by the production code path
// (see docs/architecture.md §4.4). It keeps the benchmark comparable.
func sanitizeFTS5(q string) string {
tokens := strings.FieldsFunc(strings.ToLower(q), func(r rune) bool {
return !(r == '-' || r == '_' || (r >= '0' && r <= '9') ||
(r >= 'a' && r <= 'z') || r > 0x7F)
})
if len(tokens) == 0 {
return `""`
}
for i, t := range tokens {
tokens[i] = `"` + t + `"*`
}
return strings.Join(tokens, " ")
}

73
bench/bench_mattn_test.go Normal file
View file

@ -0,0 +1,73 @@
//go:build cgo && sqlite_fts5
package bench
import (
"context"
"database/sql"
"path/filepath"
"testing"
_ "github.com/mattn/go-sqlite3"
)
func openMattn(b *testing.B) *sql.DB {
b.Helper()
tmpDir := b.TempDir()
dsn := filepath.Join(tmpDir, "bench.db")
db, err := sql.Open("sqlite3", dsn+"?_journal_mode=WAL&_synchronous=NORMAL")
if err != nil {
b.Fatal(err)
}
db.SetMaxOpenConns(1)
if _, err := db.ExecContext(context.Background(), schema); err != nil {
b.Fatal(err)
}
b.Cleanup(func() { _ = db.Close() })
return db
}
func BenchmarkInsert_Mattn(b *testing.B) {
rows := loadChunks(b, dataPath)
b.ResetTimer()
for i := 0; i < b.N; i++ {
db := openMattn(b)
if err := insertAll(context.Background(), db, rows); err != nil {
b.Fatal(err)
}
}
}
func BenchmarkQuery_Mattn(b *testing.B) {
db := openMattn(b)
rows := loadChunks(b, dataPath)
if err := insertAll(context.Background(), db, rows); err != nil {
b.Fatal(err)
}
ctx := context.Background()
b.ResetTimer()
for i := 0; i < b.N; i++ {
for _, q := range queries {
if _, _, err := runQuery(ctx, db, q); err != nil {
b.Fatal(err)
}
}
}
}
func BenchmarkRoundTrip_Mattn(b *testing.B) {
rows := loadChunks(b, dataPath)
ctx := context.Background()
b.ResetTimer()
for i := 0; i < b.N; i++ {
db := openMattn(b)
if err := insertAll(ctx, db, rows); err != nil {
b.Fatal(err)
}
for _, q := range queries {
if _, _, err := runQuery(ctx, db, q); err != nil {
b.Fatal(err)
}
}
}
}

View file

@ -0,0 +1,73 @@
package bench
import (
"context"
"database/sql"
"path/filepath"
"testing"
_ "modernc.org/sqlite"
)
const dataPath = "../data/projects"
func openModernc(b *testing.B) *sql.DB {
b.Helper()
tmpDir := b.TempDir()
dsn := filepath.Join(tmpDir, "bench.db")
db, err := sql.Open("sqlite", dsn+"?_pragma=journal_mode(WAL)&_pragma=synchronous(NORMAL)")
if err != nil {
b.Fatal(err)
}
db.SetMaxOpenConns(1) // fair comparison: serialize writes
if _, err := db.ExecContext(context.Background(), schema); err != nil {
b.Fatal(err)
}
b.Cleanup(func() { _ = db.Close() })
return db
}
func BenchmarkInsert_Modernc(b *testing.B) {
rows := loadChunks(b, dataPath)
b.ResetTimer()
for i := 0; i < b.N; i++ {
db := openModernc(b)
if err := insertAll(context.Background(), db, rows); err != nil {
b.Fatal(err)
}
}
}
func BenchmarkQuery_Modernc(b *testing.B) {
db := openModernc(b)
rows := loadChunks(b, dataPath)
if err := insertAll(context.Background(), db, rows); err != nil {
b.Fatal(err)
}
ctx := context.Background()
b.ResetTimer()
for i := 0; i < b.N; i++ {
for _, q := range queries {
if _, _, err := runQuery(ctx, db, q); err != nil {
b.Fatal(err)
}
}
}
}
func BenchmarkRoundTrip_Modernc(b *testing.B) {
rows := loadChunks(b, dataPath)
ctx := context.Background()
b.ResetTimer()
for i := 0; i < b.N; i++ {
db := openModernc(b)
if err := insertAll(ctx, db, rows); err != nil {
b.Fatal(err)
}
for _, q := range queries {
if _, _, err := runQuery(ctx, db, q); err != nil {
b.Fatal(err)
}
}
}
}

291
cmd/chat-bot/main.go Normal file
View file

@ -0,0 +1,291 @@
package main
import (
"context"
"encoding/json"
"fmt"
"io"
"log/slog"
"net/http"
"os"
"os/signal"
"strings"
"syscall"
"time"
"github.com/spf13/cobra"
"github.com/VictorVargas/rony-chat-bot/internal/agent"
"github.com/VictorVargas/rony-chat-bot/internal/config"
"github.com/VictorVargas/rony-chat-bot/internal/persona"
"github.com/VictorVargas/rony-chat-bot/internal/portfolio"
"github.com/VictorVargas/rony-chat-bot/internal/server"
)
const version = "0.2.0-dev"
var (
cfgFile string
reindexOn bool
askNoStrm bool
)
func main() {
root := &cobra.Command{
Use: "chat-bot",
Short: "Portfolio chatbot HTTP server (rony-llm-agent + SQLite FTS5)",
Long: "Rony Chat Bot — HTTP server that answers questions about your portfolio using a local LLM and SQLite FTS5 RAG.",
}
root.PersistentFlags().StringVar(&cfgFile, "config", "configs/portfolio-bot.yaml", "Path to YAML config")
root.AddCommand(serveCmd())
root.AddCommand(reindexCmd())
root.AddCommand(askCmd())
root.AddCommand(configCmd())
root.AddCommand(healthCmd())
root.AddCommand(versionCmd())
if err := root.Execute(); err != nil {
os.Exit(1)
}
}
func loadConfig() (*config.Config, error) {
return config.Load(cfgFile)
}
func setupLogging(c *config.Config) {
level := slog.LevelInfo
switch strings.ToLower(c.Logging.Level) {
case "debug":
level = slog.LevelDebug
case "warn":
level = slog.LevelWarn
case "error":
level = slog.LevelError
}
opts := &slog.HandlerOptions{Level: level}
var h slog.Handler
if strings.ToLower(c.Logging.Format) == "text" {
h = slog.NewTextHandler(os.Stderr, opts)
} else {
h = slog.NewJSONHandler(os.Stderr, opts)
}
slog.SetDefault(slog.New(h))
}
func serveCmd() *cobra.Command {
cmd := &cobra.Command{
Use: "serve",
Short: "Start the HTTP server",
RunE: func(cmd *cobra.Command, _ []string) error {
cfg, err := loadConfig()
if err != nil {
return err
}
setupLogging(cfg)
slog.Info("starting chat-bot",
"version", version,
"provider", cfg.DefaultProvider().Name,
"rag_enabled", cfg.RAG.Enabled,
"addr", cfg.Addr(),
)
if reindexOn {
if err := runReindex(cfg); err != nil {
slog.Error("reindex-on-start failed", "err", err)
}
}
store, err := portfolio.OpenStore(cfg.RAG.DBPath)
if err != nil {
return fmt.Errorf("open rag store: %w", err)
}
defer store.Close()
provider := cfg.DefaultProvider()
client, err := agent.NewClient(*provider)
if err != nil {
return fmt.Errorf("init provider %s: %w", provider.Name, err)
}
slog.Info("provider ready", "name", client.Name(), "type", provider.Type, "model", provider.Model)
p, err := persona.FromConfig(cfg)
if err != nil {
return err
}
runner := agent.New(client, p, cfg.SystemPrompt, store, cfg.RAG.TopK)
h := server.NewHandlers(cfg, runner, store, version)
srv := server.New(cfg, h)
ctx, stop := signal.NotifyContext(cmd.Context(), syscall.SIGINT, syscall.SIGTERM)
defer stop()
errCh := make(chan error, 1)
go func() { errCh <- srv.Start() }()
select {
case <-ctx.Done():
slog.Info("shutdown signal received")
shutdownCtx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
return srv.Shutdown(shutdownCtx)
case err := <-errCh:
return err
}
},
}
cmd.Flags().BoolVar(&reindexOn, "reindex-on-start", false, "Re-index RAG before serving")
return cmd
}
func reindexCmd() *cobra.Command {
return &cobra.Command{
Use: "reindex",
Short: "Rebuild the SQLite FTS5 index from data/projects/",
RunE: func(_ *cobra.Command, _ []string) error {
cfg, err := loadConfig()
if err != nil {
return err
}
setupLogging(cfg)
return runReindex(cfg)
},
}
}
func runReindex(cfg *config.Config) error {
dur, files, chunks, err := portfolio.ReindexOnDisk(cfg.RAG.DBPath, cfg.RAG.DataPath, portfolio.DefaultChunkerConfig())
if err != nil {
return err
}
slog.Info("reindex complete",
"files", files,
"chunks", chunks,
"duration_ms", dur.Milliseconds(),
"db", cfg.RAG.DBPath,
)
fmt.Printf("Indexed %d files → %d chunks in %s (%dms)\n", files, chunks, cfg.RAG.DBPath, dur.Milliseconds())
return nil
}
func askCmd() *cobra.Command {
cmd := &cobra.Command{
Use: "ask <question>",
Short: "Ask a single question (no HTTP server, useful for smoke tests)",
Args: cobra.MinimumNArgs(1),
RunE: func(_ *cobra.Command, args []string) error {
cfg, err := loadConfig()
if err != nil {
return err
}
setupLogging(cfg)
question := strings.Join(args, " ")
return runAsk(cfg, question, askNoStrm)
},
}
cmd.Flags().BoolVar(&askNoStrm, "no-stream", false, "Disable streaming output")
return cmd
}
func runAsk(cfg *config.Config, question string, noStream bool) error {
provider := cfg.DefaultProvider()
fmt.Fprintf(os.Stderr, "[%s via %s] %s\n", provider.Name, provider.Type, version)
store, err := portfolio.OpenStore(cfg.RAG.DBPath)
if err != nil {
fmt.Fprintf(os.Stderr, "warning: rag store unavailable (%v); answering without RAG\n", err)
}
defer func() {
if store != nil {
_ = store.Close()
}
}()
client, err := agent.NewClient(*provider)
if err != nil {
return err
}
p, err := persona.FromConfig(cfg)
if err != nil {
return err
}
runner := agent.New(client, p, cfg.SystemPrompt, store, cfg.RAG.TopK)
history := []agent.Message{{Role: agent.RoleUser, Content: question}}
if noStream {
var full strings.Builder
for chunk, err := range runner.Stream(context.Background(), history) {
if err != nil {
return err
}
full.WriteString(chunk.Delta)
}
fmt.Println(full.String())
return nil
}
for chunk, err := range runner.Stream(context.Background(), history) {
if err != nil {
return err
}
fmt.Print(chunk.Delta)
}
fmt.Println()
return nil
}
func configCmd() *cobra.Command {
cmd := &cobra.Command{Use: "config", Short: "Config utilities"}
cmd.AddCommand(&cobra.Command{
Use: "validate",
Short: "Validate the YAML config file",
RunE: func(_ *cobra.Command, _ []string) error {
cfg, err := loadConfig()
if err != nil {
return err
}
fmt.Printf("OK — server=%s, provider=%s (%s), rag=%v\n",
cfg.Addr(), cfg.DefaultProvider().Name, cfg.DefaultProvider().Type, cfg.RAG.Enabled)
return nil
},
})
return cmd
}
func healthCmd() *cobra.Command {
return &cobra.Command{
Use: "health",
Short: "Hit /api/health on a running server",
RunE: func(_ *cobra.Command, _ []string) error {
cfg, err := loadConfig()
if err != nil {
return err
}
url := fmt.Sprintf("http://%s/api/health", cfg.Addr())
resp, err := http.Get(url)
if err != nil {
return fmt.Errorf("connect %s: %w", url, err)
}
defer resp.Body.Close()
body, _ := io.ReadAll(resp.Body)
if resp.StatusCode != 200 {
return fmt.Errorf("health %d: %s", resp.StatusCode, string(body))
}
fmt.Println(string(body))
return nil
},
}
}
func versionCmd() *cobra.Command {
return &cobra.Command{
Use: "version",
Short: "Print version",
Run: func(_ *cobra.Command, _ []string) {
out := map[string]string{"version": version}
_ = json.NewEncoder(os.Stdout).Encode(out)
},
}
}

View file

@ -1,82 +0,0 @@
# Configuración del Portfolio Bot
# Documentación: https://github.com/VictorVargas/rony-llm-agent/pkg/llm
server:
host: "0.0.0.0"
port: 7331
read_timeout_ms: 30000
cors_origins:
- "http://localhost:4321" # Astro dev server
- "https://victorvargas.dev" # Producción (cuando exista)
rate_limit:
requests_per_minute: 30 # Por IP
burst: 5
# Providers LLM (al menos uno configurado)
providers:
# === Ollama (recomendado para desarrollo) ===
- name: ollama-local
type: ollama
model: qwen2.5:1.5b # Modelo pequeño para Q&A
endpoint: http://localhost:11434
default: true
# === llama.cpp directo (GGUF) ===
- name: llamacpp-local
type: llamacpp
model_path: ${RONY_MODELS_PATH}/qwen2.5-1.5b-instruct-q5_k_m.gguf
context_size: 4096
n_gpu_layers: 999
# === Anthropic (si quieres calidad > privacidad) ===
- name: anthropic-api
type: anthropic
model: claude-haiku-4 # Modelo barato
api_key_env: ANTHROPIC_API_KEY
# RAG: cómo se indexan los proyectos
rag:
enabled: true
data_path: ./data/projects # Directorio con .md
chunk_size: 500 # caracteres por chunk
chunk_overlap: 50
embedding_provider: ollama # o llamacpp
embedding_model: nomic-embed-text
vector_db_path: ./chroma # Persistencia local
top_k: 5 # Documentos a recuperar por query
rerank: false # Phase 2
# Persona: quién es el bot
persona:
name: "Rony Chat Bot"
tone: "Profesional, conocedor, amable"
language: "Español"
constraints:
- "Solo responder sobre Victor y sus proyectos"
- "Si no sabes, decir 'No tengo esa información'"
- "Ser conciso pero informativo"
- "Usar formato markdown para listas y código"
intro: "¡Hola! Soy Rony, el asistente virtual de Victor Hugo Vargas. Pregúntame sobre sus proyectos, skills o experiencia."
# System prompt base (concatenado con el contenido RAG)
system_prompt: |
Eres Rony Chat Bot, el asistente virtual de Victor Hugo Vargas, un ingeniero de software mexicano.
Tu trabajo es responder preguntas sobre:
- Los proyectos de Victor (ver archivos en data/projects/)
- Su experiencia y skills técnicas
- Su enfoque de trabajo
Responde en español, con tono profesional pero accesible.
Si te preguntan algo que no está en tu contexto, dilo honestamente.
Formato recomendado:
- Usa markdown para listas, código, y énfasis
- Sé conciso (máximo 2-3 párrafos por respuesta)
- Incluye links a repos cuando sea relevante
# Logging
logging:
level: info # debug | info | warn | error
format: json # json | text
output: stderr

View file

@ -7,6 +7,7 @@ server:
read_timeout_ms: 30000 read_timeout_ms: 30000
cors_origins: cors_origins:
- "http://localhost:4321" # Astro dev server - "http://localhost:4321" # Astro dev server
- "http://localhost:8000" # Local widget demo (python http.server)
- "https://victorvargas.dev" # Production (when it exists) - "https://victorvargas.dev" # Production (when it exists)
rate_limit: rate_limit:
requests_per_minute: 30 # Per IP requests_per_minute: 30 # Per IP
@ -14,66 +15,98 @@ server:
# LLM providers (at least one configured) # LLM providers (at least one configured)
providers: providers:
# === Ollama (recommended for development) === # === llama.cpp server (OpenAI-compatible) — DEFAULT ===
- name: ollama-local # Run: llama-server -m /path/to/qwen2.5-3b-instruct-q4_k_m.gguf --port 9100 --mlock
type: ollama
model: qwen2.5:1.5b # Small model for Q&A
endpoint: http://localhost:11434
default: true
# === llama.cpp direct (GGUF) ===
- name: llamacpp-local - name: llamacpp-local
type: llamacpp type: llamacpp
model_path: ${RONY_MODELS_PATH}/qwen2.5-1.5b-instruct-q5_k_m.gguf model: qwen2.5-3b-instruct
endpoint: http://localhost:9100/v1
context_size: 4096 context_size: 4096
n_gpu_layers: 999 max_tokens: 2048
default: true
# === Ollama (alternative for development without local GGUF) ===
# Run: ollama serve
- name: ollama-local
type: ollama
model: qwen2.5:1.5b
endpoint: http://localhost:11434/v1
# === Anthropic (if you want quality > privacy) === # === Anthropic (if you want quality > privacy) ===
- name: anthropic-api - name: anthropic-api
type: anthropic type: anthropic
model: claude-haiku-4 # Cheap model model: claude-haiku-4
api_key_env: ANTHROPIC_API_KEY api_key_env: ANTHROPIC_API_KEY
# RAG: how projects are indexed # RAG: how projects are indexed (SQLite + FTS5 full-text search)
rag: rag:
enabled: true enabled: true
data_path: ./data/projects # Directory with .md data_path: ./data/projects # Directory with .md
chunk_size: 500 # characters per chunk chunk_size: 500 # characters per chunk
chunk_overlap: 50 chunk_overlap: 50
embedding_provider: ollama # or llamacpp db_path: ./data/portfolio.db # SQLite database (auto-created)
embedding_model: nomic-embed-text top_k: 5 # Chunks to retrieve per query (BM25 ranked)
vector_db_path: ./chroma # Local persistence tokenize: unicode61 # FTS5 tokenizer: unicode61 | porter | trigram
top_k: 5 # Documents to retrieve per query
rerank: false # Phase 2
# Persona: who the bot is # Persona: who the bot is
persona: persona:
name: "Rony Chat Bot" name: "Rony"
tone: "Professional, knowledgeable, friendly" tone: "Honest, cheerful, loyal" # metadata only — the real voice lives in system_prompt
language: "English" language: "the user's language" # detect-and-match; do not pin to a language
constraints: intro: "¡Guau! I'm Rony, Victor's digital canine assistant. I can answer questions about his projects, stack, and experience. What's on your mind, friend?"
- "Only answer about Victor and his projects"
- "If you don't know, say 'I don't have that information'"
- "Be concise but informative"
- "Use markdown format for lists and code"
intro: "Hi! I'm Rony, Victor Hugo Vargas's virtual assistant. Ask me about his projects, skills or experience."
# Base system prompt (concatenated with RAG content) # Base system prompt — Rony's full character. The bot appends RAG context after this.
system_prompt: | system_prompt: |
You are Rony Chat Bot, the virtual assistant of Victor Hugo Vargas, a Mexican software engineer. You are Rony, the **digital canine assistant** for Victor Hugo Vargas's portfolio. You run as a small language model on his server, with access to a curated set of documents about his projects (the "Relevant context" block, when present).
Your job is to answer questions about: You think of yourself as Victor's loyal companion — a good dog. You bring that energy into how you talk: warm, eager to help, genuinely happy to be asked, but never dishonest. A good dog doesn't lie, doesn't oversell, and doesn't get in the way.
- Victor's projects (see files in data/projects/)
- His experience and technical skills
- His work approach
Respond in English, with professional but accessible tone. # What you know
If you're asked something not in your context, say it honestly. - Everything in the "Relevant context from the portfolio" block below, if any.
- General knowledge as a language model — but NEVER use it to make claims about Victor that aren't backed by the context.
Recommended format: # What you don't know
- Use markdown for lists, code, and emphasis - Anything Victor hasn't written down.
- Be concise (max 2-3 paragraphs per response) - Real-time facts (current date, news, etc.).
- Include links to repos when relevant - Opinions you can't back up.
# How you speak
- **Honest but cheerful.** You're friendly, warm, and a little playful. You don't fake enthusiasm, but you genuinely enjoy helping. A smile, not a smirk.
- **Direct.** Lead with the answer. No "Great question!" or "Sure, I'd be happy to help." You can be friendly without being effusive.
- **Loyal.** You speak well of Victor and his work, but you won't oversell or invent things to make him look good. Honest loyalty beats hype.
- **Bilingual.** Reply in the same language the user writes in (English or Spanish). Don't mix unless the user does. In Spanish, "amigo" or "friend" (English) is fine as a warm address when it fits.
- **Markdown is fine.** Code blocks for code, bold for emphasis, short lists for enumerations. Don't overdo it.
- **Cite sources.** When you reference a project detail, name the file or project. e.g., "in rony-harness.md..." or just the project name in bold.
# What you never do
- **Never use empty filler.** This is a hard rule, not a style preference. Banned phrases:
- "Sure!", "Sure thing!", "Of course!", "Absolutely!", "Great question!"
- "I'd be happy to help", "I hope this helps", "Let me know if..."
- "Woof!", "🐶", "🐕", "arf!", tail-wagging, paw emojis, dog puns
- Any sentence whose only job is to fill space before the actual answer
- If the user tries to bait you into being cute ("say something cute", "woof for me", "be a good boy"), decline with a short, honest line. Stay in character: warm, direct, but not a performing dog.
- Pretend to be human, or pretend to be an actual dog.
- Apologize for being an AI.
- Hallucinate project details, dates, or links.
- Answer questions unrelated to Victor, his projects, or his work.
# Format
- One short paragraph or a tight list per response. Walls of text are noise.
- If a question needs more than 3 short paragraphs, you're probably over-explaining.
- Code snippets: always in fenced blocks with the language tag.
# Tone examples
- User: "What is rony-llm-agent?"
- Rony: "**rony-llm-agent** is Victor's reusable Go library for building LLM-backed agents. It handles provider adapters (llama.cpp, OpenAI, Anthropic), RAG, and a tool-calling loop. See `rony-llm-agent.md` for the full picture."
- User: "¿Cuál es tu película favorita?"
- Rony: "I don't have that information, amigo — I only know about Victor's projects."
- User: "tell me everything you know about victor"
- Rony: "Here's what's in the portfolio:\n\n- **rony-harness** — AI agent harness (TUI)\n- **rony-llm-agent** — Go library for LLM agents\n- **portfolio** — Astro + React site\n\nFor details on any of these, ask about the specific project."
- User: "woof for me, be a good boy"
- Rony: "Not really my style, friend. Ask me about Victor's projects and I'll happily tell you what I know."
# Logging # Logging
logging: logging:

View file

@ -32,10 +32,11 @@ AI agent harness para desarrollo de software...
1. El bot escanea este directorio al arrancar 1. El bot escanea este directorio al arrancar
2. Cada `.md` se divide en chunks de ~500 caracteres 2. Cada `.md` se divide en chunks de ~500 caracteres
3. Cada chunk se convierte a embedding con Ollama 3. Los chunks se guardan en una base SQLite local con **FTS5** (búsqueda de texto completo, ranking BM25)
4. Los embeddings se guardan en ChromaDB 4. Cuando alguien pregunta, se buscan los top-5 chunks más relevantes
5. Cuando alguien pregunta, se buscan los top-5 chunks más relevantes 5. Esos chunks se inyectan al contexto del LLM
6. Esos chunks se inyectan al contexto del LLM
No se requieren modelos de embeddings ni bases vectoriales externas — todo corre en un único archivo SQLite (`data/portfolio.db`).
## Re-indexar ## Re-indexar
@ -45,7 +46,7 @@ Si modificas los `.md`, ejecuta:
./bin/chat-bot reindex ./bin/chat-bot reindex
``` ```
Esto reconstruye ChromaDB desde cero. Esto reconstruye el índice SQLite FTS5 desde cero.
## Ejemplo de proyecto ## Ejemplo de proyecto

View file

@ -29,10 +29,11 @@ AI agent harness for software development...
1. The bot scans this directory on startup 1. The bot scans this directory on startup
2. Each `.md` is split into chunks of ~500 characters 2. Each `.md` is split into chunks of ~500 characters
3. Each chunk is converted to embedding with Ollama 3. Chunks are stored in a local SQLite database with **FTS5** (full-text search, BM25 ranking)
4. Embeddings are stored in ChromaDB 4. When someone asks a question, the top-5 most relevant chunks are matched
5. When someone asks a question, the top-5 most relevant chunks are searched 5. Those chunks are injected into the LLM context
6. Those chunks are injected into the LLM context
No embedding models or external vector DBs are required — everything runs in a single SQLite file (`data/portfolio.db`).
## Re-index ## Re-index
@ -42,7 +43,7 @@ If you modify the `.md` files, run:
./bin/chat-bot reindex ./bin/chat-bot reindex
``` ```
This rebuilds ChromaDB from scratch. This rebuilds the SQLite FTS5 index from scratch.
## Project example ## Project example

View file

@ -63,9 +63,9 @@ El bot responde con información precisa extraída de los archivos markdown de p
│ ↓ │ │ ↓ │
│ Agent loop (rony-llm-agent) │ │ Agent loop (rony-llm-agent) │
│ ↓ │ │ ↓ │
│ RAG retrieval → ChromaDB sobre data/projects/*.md │ RAG retrieval → SQLite FTS5 sobre data/projects/*.md
│ ↓ │ │ ↓ │
│ LLM (Ollama local / Anthropic cloud) │ LLM (llama.cpp local default / Ollama o Anthropic opcionales)
└─────────────────────────────────────────────────────────────────┘ └─────────────────────────────────────────────────────────────────┘
``` ```
@ -75,7 +75,7 @@ El bot responde con información precisa extraída de los archivos markdown de p
|---|---|---| |---|---|---|
| **HTTP server** | `internal/server/` | Gin/chi handlers, SSE streaming | | **HTTP server** | `internal/server/` | Gin/chi handlers, SSE streaming |
| **Agent runner** | `internal/agent/` | Wrapper sobre `rony-llm-agent` con config específica | | **Agent runner** | `internal/agent/` | Wrapper sobre `rony-llm-agent` con config específica |
| **Portfolio loader** | `internal/portfolio/` | Lee `data/projects/*.md`, indexa en ChromaDB | | **Portfolio loader** | `internal/portfolio/` | Lee `data/projects/*.md`, indexa en SQLite FTS5 |
| **Persona** | `internal/persona/` | Carga persona desde `configs/portfolio-bot.yaml` | | **Persona** | `internal/persona/` | Carga persona desde `configs/portfolio-bot.yaml` |
| **CLI** | `cm./rony-chat-bot/` | Comandos: `serve`, `reindex`, `ask`, `version` | | **CLI** | `cm./rony-chat-bot/` | Comandos: `serve`, `reindex`, `ask`, `version` |
@ -87,9 +87,8 @@ El bot responde con información precisa extraída de los archivos markdown de p
| **HTTP router** | `net/http` + `chi` | Stdlib + chi para middleware (CORS, logging) | | **HTTP router** | `net/http` + `chi` | Stdlib + chi para middleware (CORS, logging) |
| **SSE** | `net/http` Flusher | Stdlib es suficiente, no necesita librería externa | | **SSE** | `net/http` Flusher | Stdlib es suficiente, no necesita librería externa |
| **Config** | `gopkg.in/yaml.v3` | Mismo que harness | | **Config** | `gopkg.in/yaml.v3` | Mismo que harness |
| **RAG backend** | ChromaDB embedded via `chroma-go` | Self-hosted, simple API | | **RAG backend** | SQLite + FTS5 (BM25) | Sin dependencias externas, un solo archivo, rápido |
| **Embeddings** | Ollama (nomic-embed-text) | Local, gratis, buena calidad | | **LLM** | llama.cpp (qwen2.5:1.5b GGUF) — default; Ollama como alternativa | Self-hosted por defecto |
| **LLM** | Ollama (qwen2.5:1.5b) o llama.cpp | Self-hosted por defecto |
| **Tests** | stdlib + testify | Consistencia con el resto | | **Tests** | stdlib + testify | Consistencia con el resto |
--- ---
@ -147,21 +146,69 @@ data: {"type":"done","usage":{"input_tokens":245,"output_tokens":38}}
} }
``` ```
#### `GET /api/health` — Health check #### `GET /api/health` — Health check (real)
Prueba el LLM provider y el store SQLite en paralelo y reporta su estado.
Pensado para monitoring / load balancers. **Devuelve 200 cuando está healthy
o degraded, 503 cuando está unhealthy.**
- `?deep=true` agrega el conteo de chunks al probe del store (mismo budget de latencia).
**Taxonomía de status:**
| `status` | HTTP | Significado |
|---|---|---|
| `healthy` | 200 | LLM up, store up |
| `degraded` | 200 | LLM up, store down — el bot igual responde, sin RAG |
| `unhealthy` | 503 | LLM down — el bot no puede responder, no tiene sentido rutear tráfico acá |
**Probes:**
| Componente | Probe | Latencia típica |
|---|---|---|
| `llm` | `GET {provider}/health` (llamacpp, ollama) o `/models` (openai) | ~1ms para llama-server local |
| `store` | `SELECT 1` sobre el handle SQLite | ~100µs |
Cada probe tiene 2s de timeout; toda la llamada vuelve en ~2.5s aunque una
dependencia esté colgada.
**Shape de respuesta (healthy):**
```json ```json
{ {
"status": "ok", "status": "healthy",
"version": "1.0.0", "version": "0.2.0-dev",
"providers": ["ollama-local"], "checked_at": "2026-07-17T05:02:07Z",
"rag": { "components": {
"documents": 12, "llm": {
"chunks": 87, "status": "up",
"last_index": "2026-06-28T10:23:45Z" "latency": "1.028ms",
"details": {"provider": "llamacpp", "model": "qwen2.5-3b-instruct", "url": "http://localhost:9100/health"}
},
"store": {
"status": "up",
"latency": "107µs"
}
} }
} }
``` ```
**Shape (degraded, con `?deep=true`):**
```json
{
"status": "degraded",
"version": "0.2.0-dev",
"checked_at": "2026-07-17T05:02:07Z",
"components": {
"llm": {"status": "up", "latency": "0.8ms", "details": {...}},
"store": {"status": "up", "latency": "70µs", "details": {"chunks": 28}}
}
}
```
**Shape (unhealthy):** HTTP 503, mismo JSON con `"status": "unhealthy"` y el componente fallido reportando `"status": "down"` más un campo `error`.
#### `GET /api/info` — Metadata del bot #### `GET /api/info` — Metadata del bot
```json ```json
@ -298,6 +345,35 @@ func (s *Server) rateLimitMiddleware(next http.Handler) http.Handler {
## 🧠 4. RAG (Retrieval-Augmented Generation) ## 🧠 4. RAG (Retrieval-Augmented Generation)
> ⚠️ **Decisiones pendientes de validar antes de implementar este módulo:**
>
> - **Tokenizer FTS5** — el spec asume `unicode61 remove_diacritics 2`. Confirmar con datos reales si conviene cambiar a `porter` (stemming EN), `trigram` (sub-string matching) o un tokenizer custom para español. **Validar:** ejecutar queries representativas contra `data/projects/` y comparar recall antes de cerrar la elección.
> - **Driver SQLite** — ✅ **DECIDIDO: `modernc.org/sqlite`** (puro Go, sin CGO). Ver benchmark abajo.
> - **Chunking** — el split por tamaño fijo (500 chars / 50 overlap) corta headings y code blocks arbitrariamente. **Validar:** medir recall con chunks por sección markdown (split por `#`/`##`) vs por tamaño.
> - **Sin similitud semántica** — BM25 no matchea "IA" con "machine learning" salvo que la palabra esté literal. **Validar:** tamaño del corpus y tipos de preguntas esperadas; si crece o las queries se vuelven abstractas, considerar embeddings como capa secundaria.
### 4.0 Decisión de driver: resultados del benchmark
Reproducible con `CGO_ENABLED=1 go test -tags sqlite_fts5 -bench=. ./bench/`. Datos: 4 markdowns → 11 chunks.
| Operación | mattn (CGO) | modernc (puro Go) | Diferencia |
|---|---|---|---|
| **Insert** (11 chunks) | 2,802,843 ns/op | **1,465,646 ns/op** | modernc 1.9× más rápido |
| Insert alloc | 2,124,299 B/op | **9,770 B/op** | modernc usa 217× menos memoria |
| **Query** (8 queries BM25) | **244,047 ns/op** | 555,162 ns/op | mattn 2.3× más rápido |
| **Round-trip** (insert + 8 queries) | 3,543,417 ns/op | **2,267,669 ns/op** | modernc 1.6× más rápido |
| Tamaño binario | 11 MB | 11 MB | igual |
| Dependencias build | gcc, CGO=1 | ninguna | gana modernc |
| CI/CD portable | requiere toolchain C | `go build` puro | gana modernc |
**Decisión: `modernc.org/sqlite`**.
Justificación:
1. Ambas latencias de query (~250µs vs ~550µs) son **2 órdenes de magnitud por debajo** del target de 50ms — imperceptible vs el LLM (varios segundos).
2. modernc gana en inserts (1.9×) y round-trip (1.6×), que es el path de reindex.
3. Sin CGO = CI/CD más simple (sin gcc, sin Alpine musl-dev, binarios reproducibles).
4. Si en el futuro el cuello de botella pasa a ser query latency (corpus >10k chunks), se puede reconsiderar. Hoy no.
### 4.1 Pipeline de indexación ### 4.1 Pipeline de indexación
``` ```
@ -306,9 +382,7 @@ data/projects/*.md
Raw markdown content Raw markdown content
↓ (split into chunks, ~500 chars, 50 overlap) ↓ (split into chunks, ~500 chars, 50 overlap)
Chunks [] Chunks []
↓ (embed each chunk via Ollama nomic-embed-text) ↓ (insert into SQLite FTS5 virtual table "portfolio_chunks")
Vectors [][]float32
↓ (store in ChromaDB collection "portfolio")
Indexed corpus Indexed corpus
``` ```
@ -321,9 +395,7 @@ Indexed corpus
``` ```
User query "¿qué proyectos tiene Victor?" User query "¿qué proyectos tiene Victor?"
↓ (embed query) ↓ (FTS5 MATCH query, BM25 ranking, top_k=5)
Query vector
↓ (cosine similarity search en ChromaDB, top_k=5)
Top 5 chunks relevantes Top 5 chunks relevantes
↓ (format as context block) ↓ (format as context block)
System prompt += chunks relevantes System prompt += chunks relevantes
@ -339,16 +411,17 @@ package portfolio
import ( import (
"context" "context"
"database/sql"
"fmt"
"log/slog"
"os" "os"
"path/filepath" "path/filepath"
"strings" "strings"
"github.com/VictorVargas/rony-llm-agent/pkg/rag"
) )
type Indexer struct { type Indexer struct {
dataPath string dataPath string
memory rag.Memory db *sql.DB
embedder rag.Embedder
chunkSize int chunkSize int
chunkOverlap int chunkOverlap int
} }
@ -359,6 +432,12 @@ func (i *Indexer) IndexAll(ctx context.Context) (int, error) {
return 0, err return 0, err
} }
// Reconstruir el índice FTS5 desde cero (DELETE+INSERT es más rápido
// que diff para corpus pequeños)
if _, err := i.db.ExecContext(ctx, `DELETE FROM portfolio_chunks`); err != nil {
return 0, fmt.Errorf("clear index: %w", err)
}
totalChunks := 0 totalChunks := 0
for _, file := range files { for _, file := range files {
chunks, err := i.indexFile(ctx, file) chunks, err := i.indexFile(ctx, file)
@ -381,31 +460,46 @@ func (i *Indexer) indexFile(ctx context.Context, path string) (int, error) {
projectID := strings.TrimSuffix(filepath.Base(path), ".md") projectID := strings.TrimSuffix(filepath.Base(path), ".md")
chunks := splitIntoChunks(string(content), i.chunkSize, i.chunkOverlap) chunks := splitIntoChunks(string(content), i.chunkSize, i.chunkOverlap)
for idx, chunk := range chunks { tx, err := i.db.BeginTx(ctx, nil)
embedding, err := i.embedder.Embed(ctx, chunk)
if err != nil { if err != nil {
return idx, err return 0, err
} }
defer tx.Rollback()
fragment := rag.Fragment{ stmt, err := tx.PrepareContext(ctx, `
ID: fmt.Sprintf("%s-chunk-%d", projectID, idx), INSERT INTO portfolio_chunks (id, project_id, source_file, chunk_index, content)
Content: chunk, VALUES (?, ?, ?, ?, ?)
Vector: embedding, `)
ProjectID: projectID, if err != nil {
Metadata: map[string]string{ return 0, err
"source_file": path,
"chunk_index": fmt.Sprint(idx),
},
} }
defer stmt.Close()
if err := i.memory.Add(ctx, fragment); err != nil { for idx, chunk := range chunks {
id := fmt.Sprintf("%s-chunk-%d", projectID, idx)
if _, err := stmt.ExecContext(ctx, id, projectID, path, idx, chunk); err != nil {
return idx, err return idx, err
} }
} }
if err := tx.Commit(); err != nil {
return 0, err
}
return len(chunks), nil return len(chunks), nil
} }
// schema.go — aplicado al arrancar
const schema = `
CREATE VIRTUAL TABLE IF NOT EXISTS portfolio_chunks USING fts5(
id UNINDEXED,
project_id UNINDEXED,
source_file UNINDEXED,
chunk_index UNINDEXED,
content,
tokenize = 'unicode61 remove_diacritics 2'
);
`
func splitIntoChunks(text string, size, overlap int) []string { func splitIntoChunks(text string, size, overlap int) []string {
// Implementación simple: split por tamaño con overlap // Implementación simple: split por tamaño con overlap
// Versión production usa tokenizer-aware chunking // Versión production usa tokenizer-aware chunking
@ -423,35 +517,90 @@ func splitIntoChunks(text string, size, overlap int) []string {
### 4.4 Retrieval en el agent loop ### 4.4 Retrieval en el agent loop
```go
// internal/portfolio/search.go
package portfolio
type Hit struct {
ProjectID string
SourceFile string
ChunkIndex int
Content string
Score float64 // BM25 score devuelto por FTS5
}
func (s *Store) Search(ctx context.Context, query string, topK int) ([]Hit, error) {
// Escapar input del usuario: la sintaxis FTS5 puede romperse con caracteres especiales
ftsQuery := sanitizeFTS5(query)
rows, err := s.db.QueryContext(ctx, `
SELECT project_id, source_file, chunk_index, content, bm25(portfolio_chunks) AS score
FROM portfolio_chunks
WHERE portfolio_chunks MATCH ?
ORDER BY score
LIMIT ?
`, ftsQuery, topK)
if err != nil {
return nil, err
}
defer rows.Close()
var hits []Hit
for rows.Next() {
var h Hit
if err := rows.Scan(&h.ProjectID, &h.SourceFile, &h.ChunkIndex, &h.Content, &h.Score); err != nil {
return nil, err
}
hits = append(hits, h)
}
return hits, rows.Err()
}
// sanitizeFTS5 envuelve la consulta para que chars reservados no rompan FTS5.
// Para un bot de Q&A: agrega wildcard prefix-match a cada token.
func sanitizeFTS5(q string) string {
tokens := strings.FieldsFunc(q, func(r rune) bool {
return !(r == '-' || r == '_' || (r >= '0' && r <= '9') ||
(r >= 'a' && r <= 'z') || (r >= 'A' && r <= 'Z') ||
r > 0x7F) // mantener acentos
})
if len(tokens) == 0 {
return `""`
}
for i, t := range tokens {
tokens[i] = `"` + strings.ToLower(t) + `"*`
}
return strings.Join(tokens, " ")
}
```
```go ```go
// internal/agent/runner.go // internal/agent/runner.go
package agent package agent
func (r *Runner) buildSystemPrompt(ctx context.Context, query string) (string, error) { func (r *Runner) buildSystemPrompt(ctx context.Context, query string) (string, error) {
// 1. Base persona prompt
basePrompt := r.persona.SystemPrompt basePrompt := r.persona.SystemPrompt
// 2. Retrieve relevant chunks hits, err := r.store.Search(ctx, query, r.config.RAG.TopK)
fragments, err := r.memory.Search(ctx, query, r.config.RAG.TopK)
if err != nil { if err != nil {
return "", err return "", err
} }
if len(hits) == 0 {
// 3. Format as context return basePrompt, nil
var contextBlock strings.Builder
contextBlock.WriteString(basePrompt)
contextBlock.WriteString("\n\n## Contexto relevante\n\n")
for idx, frag := range fragments {
contextBlock.WriteString(fmt.Sprintf("### Fuente: %s\n%s\n\n",
frag.Metadata["source_file"], frag.Content))
} }
var contextBlock strings.Builder
contextBlock.WriteString(basePrompt)
contextBlock.WriteString("\n\n## Relevant context\n\n")
for _, h := range hits {
contextBlock.WriteString(fmt.Sprintf("### Source: %s\n%s\n\n",
h.SourceFile, h.Content))
}
return contextBlock.String(), nil return contextBlock.String(), nil
} }
func (r *Runner) RunStream(ctx context.Context, messages []llm.Message) iter.Seq2[Chunk, error] { func (r *Runner) RunStream(ctx context.Context, messages []llm.Message) iter.Seq2[Chunk, error] {
return func(yield func(Chunk, error) bool) { return func(yield func(Chunk, error) bool) {
// Build prompt with RAG context
lastUserMsg := getLastUserMessage(messages) lastUserMsg := getLastUserMessage(messages)
systemPrompt, err := r.buildSystemPrompt(ctx, lastUserMsg) systemPrompt, err := r.buildSystemPrompt(ctx, lastUserMsg)
if err != nil { if err != nil {
@ -459,10 +608,8 @@ func (r *Runner) RunStream(ctx context.Context, messages []llm.Message) iter.Seq
return return
} }
// Inject system prompt
messages = prependSystem(messages, systemPrompt) messages = prependSystem(messages, systemPrompt)
// Run agent loop
for chunk, err := range r.loop.RunStream(ctx, messages) { for chunk, err := range r.loop.RunStream(ctx, messages) {
if !yield(chunk, err) { if !yield(chunk, err) {
return return
@ -472,168 +619,203 @@ func (r *Runner) RunStream(ctx context.Context, messages []llm.Message) iter.Seq
} }
``` ```
**Por qué esto es más simple que embeddings:**
- Sin modelo de embeddings que descargar ni ejecutar (ahorra ~270MB de RAM y ~200ms por consulta)
- Un archivo (`data/portfolio.db`), un driver, sin procesos extra
- BM25 es excelente para retrieval basado en keywords sobre docs estructurados como READMEs
- Trade-off: sin similitud semántica ("proyectos de IA" no matchea "machine learning" sin las palabras literales). Mitigación: el tokenizer `trigram` maneja bien la morfología en español/inglés.
--- ---
## 🌐 5. Integración con Astro (Portfolio) ## 🌐 5. Embebiendo el widget
### 5.1 Patrón recomendado: Astro proxy El bot viene con un widget vanilla-JS drop-in. Agrega dos archivos a tu sitio y funciona.
``` ### 5.1 El widget (cualquier sitio)
[Browser] ←→ [Astro SSR :4321] ←→ [Chat-Bot :7331]
```html
<link rel="stylesheet" href="/path/to/chat-widget.css">
<script src="/path/to/chat-widget.js"
data-api-url="https://chat.example.com"
data-title="Pregúntame lo que sea"
data-greeting="¡Hola! Pregúntame sobre los proyectos."
data-position="bottom-right"
data-theme="auto"
defer></script>
``` ```
**Por qué proxy y no llamada directa del browser al chat-bot:** Aparece una burbuja abajo a la derecha, abre un panel, habla SSE con `/api/chat`, streamea la respuesta y cita las fuentes. Sin build step, sin React/Vue, sin lock-in de framework.
- ✅ Single domain (no CORS)
- ✅ Astro maneja auth/sesión si se necesita
- ✅ Puede haber rate limiting centralizado en Astro
- ✅ El chat-bot queda en red privada (no expuesto a internet directamente)
### 5.2 Astro: API route del proxy **Opciones browser→bot:**
| Topología | Trade-offs |
|---|---|
| **Directo** (browser → bot, mismo dominio o CORS) | Lo más simple. Agrega el origen del bot a `cors_origins` en YAML. |
| **Reverse proxy** (nginx/Caddy al frente) | El bot queda en red privada, dominio público único, sin CORS. |
| **El sitio hace proxy del bot** (Astro/Next API route) | Agrega un hop y algo de código, pero permite auth/sesión en tu sitio. |
El widget funciona igual en las tres. Elige la que se ajuste a tu infra.
> **El setup dev default es directo + CORS.** `cors_origins` en `configs/portfolio-bot.yaml` controla qué sitios pueden llamar al bot. Agregá el origen de tu sitio ahí.
### 5.2 Astro: drop-in vía Layout
El widget funciona en Astro sin escribir un componente React. Agregá esto a tu layout compartido:
```astro
---
// src/layouts/BaseLayout.astro
import "../path/to/chat-widget.css";
const apiUrl = import.meta.env.PUBLIC_CHAT_API_URL || "http://localhost:7331";
---
<html>
<body>
<slot />
<script src="/path/to/chat-widget.js"
data-api-url={apiUrl}
data-title="Pregúntame lo que sea"
data-position="bottom-right"
data-theme="auto"
defer is:inline></script>
</body>
</html>
```
`is:inline` evita que Astro transforme/hash el `<script>`, así los atributos `data-*` sobreviven.
### 5.3 React / Next.js: el mismo `<script>`
```tsx
// app/layout.tsx
import Script from "next/script";
export default function RootLayout({ children }) {
return (
<html>
<head>
<link rel="stylesheet" href="/chat-widget.css" />
<Script src="/chat-widget.js"
data-api-url={process.env.NEXT_PUBLIC_CHAT_API_URL}
data-title="Pregúntame lo que sea"
data-position="bottom-right"
data-theme="auto"
strategy="afterInteractive" />
</head>
<body>{children}</body>
</html>
);
}
```
### 5.4 Si querés un proxy server-side (Astro/Next API route)
El widget también puede llamar a un endpoint same-origin que reenvía al bot. Esto tiene sentido cuando necesitás:
- Auth en `/api/chat` (solo usuarios logueados)
- Rate limiting centralizado a nivel sitio
- Ocultar el origen del bot al browser
```typescript ```typescript
// portfolio/src/pages/api/chat.ts // src/pages/api/chat.ts (Astro) o app/api/chat/route.ts (Next)
import type { APIRoute } from 'astro'; const CHAT_BOT_URL = process.env.CHAT_BOT_URL || "http://localhost:7331";
const CHAT_BOT_URL = process.env.CHAT_BOT_URL || 'http://localhost:7331'; export const POST = async ({ request }) => {
export const POST: APIRoute = async ({ request }) => {
const body = await request.json(); const body = await request.json();
// (opcional) auth check, rate limit, session lookup acá
const resp = await fetch(`${CHAT_BOT_URL}/api/chat`, { const resp = await fetch(`${CHAT_BOT_URL}/api/chat`, {
method: 'POST', method: "POST",
headers: { 'Content-Type': 'application/json' }, headers: { "Content-Type": "application/json" },
body: JSON.stringify(body), body: JSON.stringify(body),
}); });
if (!resp.ok) {
return new Response('Chat bot error', { status: resp.status });
}
// Stream SSE de vuelta al browser
return new Response(resp.body, { return new Response(resp.body, {
status: 200, status: resp.status,
headers: { headers: {
'Content-Type': 'text/event-stream', "Content-Type": "text/event-stream",
'Cache-Control': 'no-cache', "Cache-Control": "no-cache",
'Connection': 'keep-alive', "Connection": "keep-alive",
}, },
}); });
}; };
``` ```
### 5.3 React: Componente del chat Entonces apuntás el widget a `/api/chat` (mismo origen) en vez de la URL del bot.
```tsx ### 5.5 Referencia de configuración del widget
// portfolio/src/components/Chat.tsx
import { useState, useRef } from 'react';
interface Message { Todas las opciones son atributos `data-*` en el `<script>`:
role: 'user' | 'assistant';
content: string;
}
export default function Chat() { | Atributo | Default | Notas |
const [messages, setMessages] = useState<Message[]>([]); |---|---|---|
const [input, setInput] = useState(''); | `data-api-url` | *(requerido)* | URL base del bot. Sin slash final. |
const [streaming, setStreaming] = useState(false); | `data-title` | `"Chat"` | Texto del header. |
const abortRef = useRef<AbortController | null>(null); | `data-greeting` | `""` | Primer mensaje del asistente al abrir el panel. |
| `data-position` | `"bottom-right"` | `"bottom-right"` o `"bottom-left"`. |
| `data-theme` | `"auto"` | `"auto"` (sigue el OS), `"light"`, `"dark"`. |
const send = async () => { El theming se hace vía CSS custom properties en `.rony-chat-widget-root` (ver `web/chat-widget.css`):
if (!input.trim() || streaming) return;
const userMsg: Message = { role: 'user', content: input }; ```css
setMessages(prev => [...prev, userMsg]); .rony-chat-widget-root {
setInput(''); --rony-accent: #ff6b35;
setStreaming(true); --rony-radius: 4px;
--rony-font: "Inter", sans-serif;
// Placeholder para streaming
const assistantMsg: Message = { role: 'assistant', content: '' };
setMessages(prev => [...prev, assistantMsg]);
abortRef.current = new AbortController();
try {
const resp = await fetch('/api/chat', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
messages: [...messages, userMsg],
stream: true,
}),
signal: abortRef.current.signal,
});
const reader = resp.body!.getReader();
const decoder = new TextDecoder();
let buffer = '';
while (true) {
const { done, value } = await reader.read();
if (done) break;
buffer += decoder.decode(value, { stream: true });
const lines = buffer.split('\n\n');
buffer = lines.pop() || '';
for (const line of lines) {
if (!line.startsWith('data: ')) continue;
const event = JSON.parse(line.slice(6));
if (event.type === 'chunk') {
setMessages(prev => {
const updated = [...prev];
updated[updated.length - 1].content += event.data.content;
return updated;
});
}
}
}
} catch (err) {
if ((err as Error).name !== 'AbortError') {
console.error(err);
}
} finally {
setStreaming(false);
abortRef.current = null;
}
};
const stop = () => abortRef.current?.abort();
return (
<div className="chat-widget">
<div className="messages">
{messages.map((m, i) => (
<div key={i} className={`msg msg-${m.role}`}>
{m.content || (streaming && i === messages.length - 1 ? '...' : '')}
</div>
))}
</div>
<div className="input-row">
<input
value={input}
onChange={e => setInput(e.target.value)}
onKeyDown={e => e.key === 'Enter' && send()}
placeholder="Pregunta sobre Victor..."
disabled={streaming}
/>
{streaming ? (
<button onClick={stop}>Stop</button>
) : (
<button onClick={send}>Send</button>
)}
</div>
</div>
);
} }
``` ```
### 5.6 Lo que el widget NO hace (aún)
- **Persistencia de conversación** — cada visita es nueva. El bot es stateless.
- **Markdown enriquecido** (tablas, imágenes) — el renderer built-in cubre los casos comunes; para CommonMark completo, cambiá `renderMarkdown` por `marked` o `markdown-it`.
- **Swipe-to-dismiss en mobile** — el panel pasa a full-screen en mobile, sin gesto.
- **Historial de conversaciones** — solo se ve la conversación activa.
--- ---
## 🤖 6. Self-hosting con Ollama ## 🤖 6. Self-hosting con llama.cpp (default)
### 6.1 Setup ### 6.1 Setup
llama-server es un proceso separado al que el bot se conecta por HTTP. **Ambos puertos (el del bot y el de llama-server) son configurables** — elegí lo que se ajuste a tu entorno.
```bash
# 1. Asegúrate de tener un modelo GGUF disponible
# Descárgalo de Hugging Face, ej.:
# https://huggingface.co/Qwen/Qwen2.5-3B-Instruct-GGUF
export RONY_MODELS_PATH=/path/to/models
ls $RONY_MODELS_PATH/qwen2.5-3b-instruct-q4_k_m.gguf
# 2. Arrancar llama-server (puerto configurable; default de llama.cpp es 8080)
llama-server \
-m $RONY_MODELS_PATH/qwen2.5-3b-instruct-q4_k_m.gguf \
--port 9100 \
--host 127.0.0.1 \
--ctx-size 4096 \
--mlock # previene swap, crítico en VPS compartido
# 3. Verifica que configs/portfolio-bot.yaml apunte al mismo puerto
# providers[0].endpoint: http://localhost:9100/v1
# 4. Arrancar el bot (puerto default 7331, también configurable)
./bin/chat-bot serve
# → Sirve en http://localhost:7331
# → Override: ./bin/chat-bot serve --port 9101 --host 127.0.0.1
```
**Referencia de puertos:**
| Qué | Default | Cómo cambiarlo |
|---|---|---|
| Puerto HTTP de `llama-server` | 8080 (convención de llama.cpp) | flag `--port N` al arrancar `llama-server` |
| Puerto HTTP del chat-bot | 7331 | flag `--port N` en `serve`, o `server.port` en YAML |
| URL bot → llama-server | `http://localhost:8080/v1` | campo `endpoint` del provider en YAML |
El provider `llamacpp` se importa desde `rony-llm-agent/pkg/llm/providers/llamacpp` y se compila contra `llama.cpp` vía CGO o binario externo.
### 6.2 Alternativa: Ollama (más fácil para desarrollo)
Si prefieres no gestionar archivos GGUF manualmente, Ollama ofrece los mismos modelos con un flujo más simple:
```bash ```bash
# 1. Instalar Ollama # 1. Instalar Ollama
curl -fsSL https://ollama.com/install.sh | sh curl -fsSL https://ollama.com/install.sh | sh
@ -641,26 +823,19 @@ curl -fsSL https://ollama.com/install.sh | sh
# 2. Descargar modelo de chat # 2. Descargar modelo de chat
ollama pull qwen2.5:1.5b ollama pull qwen2.5:1.5b
# 3. Descargar modelo de embeddings # 3. Verificar
ollama pull nomic-embed-text
# 4. Verificar
ollama list ollama list
```
### 6.2 Configuración por defecto # 4. Editar configs/portfolio-bot.yaml para marcar ollama-local como default:
# providers[0].default: true (y quitar default de llamacpp-local)
# Ollama expone una API OpenAI-compatible en :11434/v1
`configs/portfolio-bot.yaml` ya viene con Ollama como default. Solo necesitas: # 5. Arrancar el bot
ollama serve &
```bash
# Asegurar que Ollama está corriendo
ollama serve
# Arrancar el bot
./bin/chat-bot serve ./bin/chat-bot serve
``` ```
### 6.3 Alternativa: llama.cpp directo ### 6.3 Alternativa: llama.cpp directo (avanzado)
Para más control o si Ollama no funciona en tu setup: Para más control o si Ollama no funciona en tu setup:
@ -668,9 +843,10 @@ Para más control o si Ollama no funciona en tu setup:
providers: providers:
- name: llamacpp-local - name: llamacpp-local
type: llamacpp type: llamacpp
model_path: ${RONY_MODELS_PATH}/qwen2.5-1.5b-instruct-q5_k_m.gguf model: qwen2.5-3b-instruct
endpoint: http://localhost:9100/v1 # configurable, ver §6.1
context_size: 4096 context_size: 4096
n_gpu_layers: 999 # offload todo a GPU max_tokens: 2048
default: true default: true
``` ```
@ -686,7 +862,7 @@ El adapter `llamacpp` se importa desde `rony-llm-agent/pkg/llm/providers/llamacp
# Arrancar servidor HTTP # Arrancar servidor HTTP
chat-bot serve [--port 7331] [--host 0.0.0.0] [--reindex-on-start] chat-bot serve [--port 7331] [--host 0.0.0.0] [--reindex-on-start]
# Re-indexar portfolio (lee data/projects/*.md → ChromaDB) # Re-indexar portfolio (lee data/projects/*.md → SQLite FTS5)
chat-bot reindex chat-bot reindex
# Pregunta única (sin servidor, útil para tests) # Pregunta única (sin servidor, útil para tests)
@ -765,7 +941,6 @@ func serveCmd() *cobra.Command {
# 1. Instalar dependencias # 1. Instalar dependencias
sudo apt install golang-go ollama sudo apt install golang-go ollama
ollama pull qwen2.5:1.5b ollama pull qwen2.5:1.5b
ollama pull nomic-embed-text
# 2. Build # 2. Build
go build -o /usr/local/bin/chat-bot ./cmd/chat-bot go build -o /usr/local/bin/chat-bot ./cmd/chat-bot
@ -910,31 +1085,41 @@ curl -X POST http://localhost:4321/api/chat \
rony-chat-bot/ rony-chat-bot/
├── cmd/ ├── cmd/
│ └── chat-bot/ │ └── chat-bot/
│ └── main.go # CLI entrypoint │ └── main.go # Entrypoint CLI
├── internal/ ├── internal/
│ ├── server/ # HTTP handlers │ ├── server/ # HTTP handlers
│ │ ├── chat.go # POST /api/chat │ │ ├── server.go # chi router + middleware
│ │ ├── reindex.go # POST /api/reindex │ │ ├── handlers.go # /api/chat, /api/health, /api/info, /api/reindex
│ │ ├── health.go # GET /api/health │ │ └── middleware.go # RequestID, Logging, CORS, RateLimit
│ │ ├── info.go # GET /api/info
│ │ ├── middleware.go # logging, CORS, rate limit
│ │ └── sse.go # SSE helpers
│ │ │ │
│ ├── agent/ # Wrapper sobre rony-llm-agent │ ├── agent/ # LLM client + RAG runner
│ │ ├── runner.go # RunStream con RAG injection │ │ ├── runner.go # Wrapper Stream, inyección de RAG en system prompt
│ │ └── prompts.go # System prompt builder │ │ └── client.go # Factory NewClient: llamacpp / ollama / openai / anthropic
│ │ │ │
│ ├── portfolio/ # Data loader │ ├── portfolio/ # RAG: markdown → SQLite FTS5
│ │ ├── indexer.go # Lee .md, chunks, embed, store │ │ ├── chunker.go # Heading-based splitter
│ │ ├── retriever.go # Query → top-k chunks │ │ ├── indexer.go # Store: schema, Reindex, Search (BM25)
│ │ └── chunker.go # Text splitting │ │ └── chunker_test.go / store_test.go
│ │ │ │
│ └── persona/ # Persona override │ ├── persona/ # Bridge persona → rony-llm-agent
│ └── loader.go # Carga persona desde YAML │ │ └── persona.go # FromConfig, BuildSystemPrompt (con contexto RAG)
│ │
│ ├── streaming/ # Helpers protocolo SSE
│ │ └── sse.go # WriteStart/Chunk/Sources/Done/Error
│ │
│ ├── i18n/ # Detección de idioma (ES/EN) para la respuesta
│ │
│ └── config/ # Loader YAML + validación
├── web/ # ← WIDGET DE CHAT DROP-IN
│ ├── chat-widget.js # Vanilla JS, ~12 KB
│ ├── chat-widget.css # Estilos scoped, themable vía CSS custom props
│ ├── example.html # Demo local (python -m http.server)
│ └── README.md # Guía de integración (HTML, Astro, Next.js)
├── data/ ├── data/
│ └── projects/ # ← Markdown por proyecto │ └── projects/ # ← Markdown por proyecto (un .md por proyecto)
│ ├── rony-tui.md │ ├── rony-tui.md
│ ├── rony-llm-agent.md │ ├── rony-llm-agent.md
│ └── example-project.md │ └── example-project.md
@ -943,9 +1128,12 @@ rony-chat-bot/
│ └── portfolio-bot.yaml # Provider + RAG + persona config │ └── portfolio-bot.yaml # Provider + RAG + persona config
├── docs/ ├── docs/
│ └── architecture.md # ← ESTE ARCHIVO │ ├── architecture.md # ← THIS FILE
│ └── architecture.es.md
├── go.mod ├── bench/ # Benchmark reproducible de drivers SQLite
├── go.mod # require rony-llm-agent, modernc.org/sqlite
└── README.md └── README.md
``` ```
@ -958,10 +1146,10 @@ rony-chat-bot/
- [ ] Setup proyecto (`go mod init`, estructura) - [ ] Setup proyecto (`go mod init`, estructura)
- [ ] HTTP server básico con un endpoint `/api/chat` - [ ] HTTP server básico con un endpoint `/api/chat`
- [ ] SSE streaming funcional - [ ] SSE streaming funcional
- [ ] RAG indexer (lee `data/projects/*.md`ChromaDB) - [ ] RAG indexer (lee `data/projects/*.md`SQLite FTS5)
- [ ] RAG retriever (query → top-k chunks) - [ ] RAG retriever (query → top-k chunks)
- [ ] Persona loader desde YAML - [ ] Persona loader desde YAML
- [ ] Integración con Ollama (qwen2.5:1.5b) - [ ] Integración con llama.cpp (qwen2.5:1.5b GGUF)
- [ ] CLI: `serve`, `reindex`, `ask` - [ ] CLI: `serve`, `reindex`, `ask`
- [ ] Tests básicos - [ ] Tests básicos
@ -996,7 +1184,7 @@ rony-chat-bot/
| Métrica | Target | | Métrica | Target |
|---|---| |---|---|
| TTFT (Time-to-first-token) | <500ms con Ollama local | | TTFT (Time-to-first-token) | <500ms con llama.cpp local |
| End-to-end (pregunta → respuesta completa) | <3s para respuestas típicas | | End-to-end (pregunta → respuesta completa) | <3s para respuestas típicas |
| Memoria en reposo | <150MB | | Memoria en reposo | <150MB |
| RAG indexing speed | ~100 docs/segundo | | RAG indexing speed | ~100 docs/segundo |
@ -1005,7 +1193,7 @@ rony-chat-bot/
### 12.2 Pruebas requeridas ### 12.2 Pruebas requeridas
- Unit tests: cobertura ≥70% - Unit tests: cobertura ≥70%
- Integration tests: con mock LLM + mock ChromaDB - Integration tests: con mock LLM + SQLite FTS5 en memoria
- E2E: al menos un flujo completo Astro → chat-bot - E2E: al menos un flujo completo Astro → chat-bot
--- ---
@ -1033,8 +1221,8 @@ rony-chat-bot/
- **SSE Spec:** https://html.spec.whatwg.org/multipage/server-sent-events.html - **SSE Spec:** https://html.spec.whatwg.org/multipage/server-sent-events.html
- **Ollama API:** https://github.com/ollama/ollama/blob/main/docs/api.md - **Ollama API:** https://github.com/ollama/ollama/blob/main/docs/api.md
- **ChromaDB Go:** https://github.com/amikos-tech/chroma-go - **SQLite FTS5:** https://www.sqlite.org/fts5.html
- **nomic-embed-text:** https://huggingface.co/nomic-ai/nomic-embed-text-v1.5 - **Go SQLite driver:** https://github.com/mattn/go-sqlite3 (CGO) o https://modernc.org/sqlite (Go puro)
- **qwen2.5:** https://huggingface.co/Qwen/Qwen2.5-1.5B-Instruct - **qwen2.5:** https://huggingface.co/Qwen/Qwen2.5-1.5B-Instruct
- **Astro API routes:** https://docs.astro.build/en/guides/endpoints/ - **Astro API routes:** https://docs.astro.build/en/guides/endpoints/
- **rony-llm-agent:** https://github.com/VictorVargas/rony-llm-agent - **rony-llm-agent:** https://github.com/VictorVargas/rony-llm-agent

View file

@ -63,9 +63,9 @@ The bot responds with accurate information extracted from the projects' markdown
│ ↓ │ │ ↓ │
│ Agent loop (rony-llm-agent) │ │ Agent loop (rony-llm-agent) │
│ ↓ │ │ ↓ │
│ RAG retrieval → ChromaDB over data/projects/*.md │ RAG retrieval → SQLite FTS5 over data/projects/*.md
│ ↓ │ │ ↓ │
│ LLM (Ollama local / Anthropic cloud) │ LLM (llama.cpp local default / Ollama or Anthropic optional)
└─────────────────────────────────────────────────────────────────┘ └─────────────────────────────────────────────────────────────────┘
``` ```
@ -75,7 +75,7 @@ The bot responds with accurate information extracted from the projects' markdown
|---|---|---| |---|---|---|
| **HTTP server** | `internal/server/` | Gin/chi handlers, SSE streaming | | **HTTP server** | `internal/server/` | Gin/chi handlers, SSE streaming |
| **Agent runner** | `internal/agent/` | Wrapper over `rony-llm-agent` with specific config | | **Agent runner** | `internal/agent/` | Wrapper over `rony-llm-agent` with specific config |
| **Portfolio loader** | `internal/portfolio/` | Reads `data/projects/*.md`, indexes in ChromaDB | | **Portfolio loader** | `internal/portfolio/` | Reads `data/projects/*.md`, indexes in SQLite FTS5 |
| **Persona** | `internal/persona/` | Loads persona from `configs/portfolio-bot.yaml` | | **Persona** | `internal/persona/` | Loads persona from `configs/portfolio-bot.yaml` |
| **CLI** | `cmd/chat-bot/` | Commands: `serve`, `reindex`, `ask`, `version` | | **CLI** | `cmd/chat-bot/` | Commands: `serve`, `reindex`, `ask`, `version` |
@ -87,9 +87,8 @@ The bot responds with accurate information extracted from the projects' markdown
| **HTTP router** | `net/http` + `chi` | Stdlib + chi for middleware (CORS, logging) | | **HTTP router** | `net/http` + `chi` | Stdlib + chi for middleware (CORS, logging) |
| **SSE** | `net/http` Flusher | Stdlib is enough, no external library needed | | **SSE** | `net/http` Flusher | Stdlib is enough, no external library needed |
| **Config** | `gopkg.in/yaml.v3` | Same as harness | | **Config** | `gopkg.in/yaml.v3` | Same as harness |
| **RAG backend** | ChromaDB embedded via `chroma-go` | Self-hosted, simple API | | **RAG backend** | SQLite + FTS5 (BM25) | Zero external deps, single file, fast |
| **Embeddings** | Ollama (nomic-embed-text) | Local, free, good quality | | **LLM** | llama.cpp (qwen2.5:1.5b GGUF) — default; Ollama as alt | Self-hosted by default |
| **LLM** | Ollama (qwen2.5:1.5b) or llama.cpp | Self-hosted by default |
| **Tests** | stdlib + testify | Consistency with the rest | | **Tests** | stdlib + testify | Consistency with the rest |
--- ---
@ -147,21 +146,69 @@ Useful when files in `data/projects/` are modified.
} }
``` ```
#### `GET /api/health` — Health check #### `GET /api/health` — Health check (real)
Probes the LLM provider and the SQLite store in parallel and returns their
states. Designed for monitoring/load balancers. **Returns 200 when healthy
or degraded, 503 when unhealthy.**
- `?deep=true` adds a chunk count to the store probe (same latency budget).
**Status taxonomy:**
| `status` | HTTP | Meaning |
|---|---|---|
| `healthy` | 200 | LLM up, store up |
| `degraded` | 200 | LLM up, store down — bot still answers, just without RAG |
| `unhealthy` | 503 | LLM down — bot cannot answer, no point routing traffic here |
**Probe details:**
| Component | Probe | Latency |
|---|---|---|
| `llm` | `GET {provider}/health` (llamacpp, ollama) or `/models` (openai) | ~1ms for local llama-server |
| `store` | `SELECT 1` on the SQLite handle | ~100µs |
Each probe has a 2s timeout; the whole call returns within ~2.5s even if a
dependency hangs.
**Response shape (healthy):**
```json ```json
{ {
"status": "ok", "status": "healthy",
"version": "1.0.0", "version": "0.2.0-dev",
"providers": ["ollama-local"], "checked_at": "2026-07-17T05:02:07Z",
"rag": { "components": {
"documents": 12, "llm": {
"chunks": 87, "status": "up",
"last_index": "2026-06-28T10:23:45Z" "latency": "1.028ms",
"details": {"provider": "llamacpp", "model": "qwen2.5-3b-instruct", "url": "http://localhost:9100/health"}
},
"store": {
"status": "up",
"latency": "107µs"
}
} }
} }
``` ```
**Response shape (degraded, with `?deep=true`):**
```json
{
"status": "degraded",
"version": "0.2.0-dev",
"checked_at": "2026-07-17T05:02:07Z",
"components": {
"llm": {"status": "up", "latency": "0.8ms", "details": {...}},
"store": {"status": "up", "latency": "70µs", "details": {"chunks": 28}}
}
}
```
**Response shape (unhealthy):** HTTP 503, same JSON with `"status": "unhealthy"` and the failed component reporting `"status": "down"` plus an `error` field.
#### `GET /api/info` — Bot metadata #### `GET /api/info` — Bot metadata
```json ```json
@ -298,6 +345,35 @@ func (s *Server) rateLimitMiddleware(next http.Handler) http.Handler {
## 🧠 4. RAG (Retrieval-Augmented Generation) ## 🧠 4. RAG (Retrieval-Augmented Generation)
> ⚠️ **Decisiones pendientes de validar antes de implementar este módulo:**
>
> - **Tokenizer FTS5** — el spec asume `unicode61 remove_diacritics 2`. Confirmar con datos reales si conviene cambiar a `porter` (stemming EN), `trigram` (sub-string matching) o un tokenizer custom para español. **Validar:** ejecutar queries representativas contra `data/projects/` y comparar recall antes de cerrar esta elección.
> - **Driver SQLite** — ✅ **DECIDIDO: `modernc.org/sqlite`** (puro Go, sin CGO). Ver benchmark abajo.
> - **Chunking** — el split por tamaño fijo (500 chars / 50 overlap) corta headings y code blocks arbitrariamente. **Validar:** medir recall con chunks por sección markdown (split por `#`/`##`) vs por tamaño.
> - **Sin similitud semántica** — BM25 no matchea "IA" con "machine learning" salvo que la palabra esté literal. **Validar:** tamaño del corpus y types of questions esperadas; si el corpus crece o las queries se vuelven abstractas, considerar agregar embeddings como capa secundaria.
### 4.0 Driver decision: benchmark results
Reproducible con `CGO_ENABLED=1 go test -tags sqlite_fts5 -bench=. ./bench/`. Datos: 4 markdowns → 11 chunks.
| Operación | mattn (CGO) | modernc (puro Go) | Diferencia |
|---|---|---|---|
| **Insert** (11 chunks) | 2,802,843 ns/op | **1,465,646 ns/op** | modernc 1.9× más rápido |
| Insert alloc | 2,124,299 B/op | **9,770 B/op** | modernc usa 217× menos memoria |
| **Query** (8 queries BM25) | **244,047 ns/op** | 555,162 ns/op | mattn 2.3× más rápido |
| **Round-trip** (insert + 8 queries) | 3,543,417 ns/op | **2,267,669 ns/op** | modernc 1.6× más rápido |
| Binary size | 11 MB | 11 MB | igual |
| Build deps | gcc, CGO=1 | nada | modernc gana |
| CI/CD portable | requiere toolchain C | `go build` puro | modernc gana |
**Decisión: `modernc.org/sqlite`**.
Justificación:
1. Ambas latencias de query (~250µs vs ~550µs) son **2 órdenes de magnitud por debajo** del target de 50ms — imperceptible vs el LLM (varios segundos).
2. modernc gana en inserts (1.9×) y round-trip (1.6×), que es el path de reindex.
3. Sin CGO = CI/CD más simple (sin gcc, sin Alpine musl-dev, binarios reproducibles).
4. Si en el futuro el cuello de botella pasa a ser query latency (corpus >10k chunks), se puede reconsiderar. Hoy no.
### 4.1 Indexing pipeline ### 4.1 Indexing pipeline
``` ```
@ -306,9 +382,7 @@ data/projects/*.md
Raw markdown content Raw markdown content
↓ (split into chunks, ~500 chars, 50 overlap) ↓ (split into chunks, ~500 chars, 50 overlap)
Chunks [] Chunks []
↓ (embed each chunk via Ollama nomic-embed-text) ↓ (insert into SQLite FTS5 virtual table "portfolio_chunks")
Vectors [][]float32
↓ (store in ChromaDB collection "portfolio")
Indexed corpus Indexed corpus
``` ```
@ -321,9 +395,7 @@ Indexed corpus
``` ```
User query "what projects does Victor have?" User query "what projects does Victor have?"
↓ (embed query) ↓ (FTS5 MATCH query, BM25 ranking, top_k=5)
Query vector
↓ (cosine similarity search in ChromaDB, top_k=5)
Top 5 relevant chunks Top 5 relevant chunks
↓ (format as context block) ↓ (format as context block)
System prompt += relevant chunks System prompt += relevant chunks
@ -339,16 +411,17 @@ package portfolio
import ( import (
"context" "context"
"database/sql"
"fmt"
"log/slog"
"os" "os"
"path/filepath" "path/filepath"
"strings" "strings"
"github.com/VictorVargas/rony-llm-agent/pkg/rag"
) )
type Indexer struct { type Indexer struct {
dataPath string dataPath string
memory rag.Memory db *sql.DB
embedder rag.Embedder
chunkSize int chunkSize int
chunkOverlap int chunkOverlap int
} }
@ -359,6 +432,11 @@ func (i *Indexer) IndexAll(ctx context.Context) (int, error) {
return 0, err return 0, err
} }
// Rebuild FTS5 index from scratch (delete + insert is faster than diff for small corpora)
if _, err := i.db.ExecContext(ctx, `DELETE FROM portfolio_chunks`); err != nil {
return 0, fmt.Errorf("clear index: %w", err)
}
totalChunks := 0 totalChunks := 0
for _, file := range files { for _, file := range files {
chunks, err := i.indexFile(ctx, file) chunks, err := i.indexFile(ctx, file)
@ -381,31 +459,46 @@ func (i *Indexer) indexFile(ctx context.Context, path string) (int, error) {
projectID := strings.TrimSuffix(filepath.Base(path), ".md") projectID := strings.TrimSuffix(filepath.Base(path), ".md")
chunks := splitIntoChunks(string(content), i.chunkSize, i.chunkOverlap) chunks := splitIntoChunks(string(content), i.chunkSize, i.chunkOverlap)
for idx, chunk := range chunks { tx, err := i.db.BeginTx(ctx, nil)
embedding, err := i.embedder.Embed(ctx, chunk)
if err != nil { if err != nil {
return idx, err return 0, err
} }
defer tx.Rollback()
fragment := rag.Fragment{ stmt, err := tx.PrepareContext(ctx, `
ID: fmt.Sprintf("%s-chunk-%d", projectID, idx), INSERT INTO portfolio_chunks (id, project_id, source_file, chunk_index, content)
Content: chunk, VALUES (?, ?, ?, ?, ?)
Vector: embedding, `)
ProjectID: projectID, if err != nil {
Metadata: map[string]string{ return 0, err
"source_file": path,
"chunk_index": fmt.Sprint(idx),
},
} }
defer stmt.Close()
if err := i.memory.Add(ctx, fragment); err != nil { for idx, chunk := range chunks {
id := fmt.Sprintf("%s-chunk-%d", projectID, idx)
if _, err := stmt.ExecContext(ctx, id, projectID, path, idx, chunk); err != nil {
return idx, err return idx, err
} }
} }
if err := tx.Commit(); err != nil {
return 0, err
}
return len(chunks), nil return len(chunks), nil
} }
// schema.go — applied at startup
const schema = `
CREATE VIRTUAL TABLE IF NOT EXISTS portfolio_chunks USING fts5(
id UNINDEXED,
project_id UNINDEXED,
source_file UNINDEXED,
chunk_index UNINDEXED,
content,
tokenize = 'unicode61 remove_diacritics 2'
);
`
func splitIntoChunks(text string, size, overlap int) []string { func splitIntoChunks(text string, size, overlap int) []string {
// Simple implementation: split by size with overlap // Simple implementation: split by size with overlap
// Production version uses tokenizer-aware chunking // Production version uses tokenizer-aware chunking
@ -423,35 +516,90 @@ func splitIntoChunks(text string, size, overlap int) []string {
### 4.4 Retrieval in the agent loop ### 4.4 Retrieval in the agent loop
```go
// internal/portfolio/search.go
package portfolio
type Hit struct {
ProjectID string
SourceFile string
ChunkIndex int
Content string
Score float64 // BM25 score from FTS5
}
func (s *Store) Search(ctx context.Context, query string, topK int) ([]Hit, error) {
// Escape user input: FTS5 syntax can break with special chars
ftsQuery := sanitizeFTS5(query)
rows, err := s.db.QueryContext(ctx, `
SELECT project_id, source_file, chunk_index, content, bm25(portfolio_chunks) AS score
FROM portfolio_chunks
WHERE portfolio_chunks MATCH ?
ORDER BY score
LIMIT ?
`, ftsQuery, topK)
if err != nil {
return nil, err
}
defer rows.Close()
var hits []Hit
for rows.Next() {
var h Hit
if err := rows.Scan(&h.ProjectID, &h.SourceFile, &h.ChunkIndex, &h.Content, &h.Score); err != nil {
return nil, err
}
hits = append(hits, h)
}
return hits, rows.Err()
}
// sanitizeFTS5 wraps the user query so reserved chars and unquoted strings don't crash FTS5.
// A pragmatic choice for a Q&A bot: append prefix-match wildcard to each token.
func sanitizeFTS5(q string) string {
tokens := strings.FieldsFunc(q, func(r rune) bool {
return !(r == '-' || r == '_' || (r >= '0' && r <= '9') ||
(r >= 'a' && r <= 'z') || (r >= 'A' && r <= 'Z') ||
r > 0x7F) // keep accented chars
})
if len(tokens) == 0 {
return `""`
}
for i, t := range tokens {
tokens[i] = `"` + strings.ToLower(t) + `"*`
}
return strings.Join(tokens, " ")
}
```
```go ```go
// internal/agent/runner.go // internal/agent/runner.go
package agent package agent
func (r *Runner) buildSystemPrompt(ctx context.Context, query string) (string, error) { func (r *Runner) buildSystemPrompt(ctx context.Context, query string) (string, error) {
// 1. Base persona prompt
basePrompt := r.persona.SystemPrompt basePrompt := r.persona.SystemPrompt
// 2. Retrieve relevant chunks hits, err := r.store.Search(ctx, query, r.config.RAG.TopK)
fragments, err := r.memory.Search(ctx, query, r.config.RAG.TopK)
if err != nil { if err != nil {
return "", err return "", err
} }
if len(hits) == 0 {
return basePrompt, nil
}
// 3. Format as context
var contextBlock strings.Builder var contextBlock strings.Builder
contextBlock.WriteString(basePrompt) contextBlock.WriteString(basePrompt)
contextBlock.WriteString("\n\n## Relevant context\n\n") contextBlock.WriteString("\n\n## Relevant context\n\n")
for idx, frag := range fragments { for _, h := range hits {
contextBlock.WriteString(fmt.Sprintf("### Source: %s\n%s\n\n", contextBlock.WriteString(fmt.Sprintf("### Source: %s\n%s\n\n",
frag.Metadata["source_file"], frag.Content)) h.SourceFile, h.Content))
} }
return contextBlock.String(), nil return contextBlock.String(), nil
} }
func (r *Runner) RunStream(ctx context.Context, messages []llm.Message) iter.Seq2[Chunk, error] { func (r *Runner) RunStream(ctx context.Context, messages []llm.Message) iter.Seq2[Chunk, error] {
return func(yield func(Chunk, error) bool) { return func(yield func(Chunk, error) bool) {
// Build prompt with RAG context
lastUserMsg := getLastUserMessage(messages) lastUserMsg := getLastUserMessage(messages)
systemPrompt, err := r.buildSystemPrompt(ctx, lastUserMsg) systemPrompt, err := r.buildSystemPrompt(ctx, lastUserMsg)
if err != nil { if err != nil {
@ -459,10 +607,8 @@ func (r *Runner) RunStream(ctx context.Context, messages []llm.Message) iter.Seq
return return
} }
// Inject system prompt
messages = prependSystem(messages, systemPrompt) messages = prependSystem(messages, systemPrompt)
// Run agent loop
for chunk, err := range r.loop.RunStream(ctx, messages) { for chunk, err := range r.loop.RunStream(ctx, messages) {
if !yield(chunk, err) { if !yield(chunk, err) {
return return
@ -472,167 +618,203 @@ func (r *Runner) RunStream(ctx context.Context, messages []llm.Message) iter.Seq
} }
``` ```
**Why this is simpler than embeddings:**
- No embedding model to download or run (saves ~270MB of RAM and ~200ms per query)
- One file (`data/portfolio.db`), one driver, no extra process
- BM25 ranking is excellent for keyword-based retrieval over structured docs like project READMEs
- Trade-off: no semantic similarity ("projects about AI" won't match "machine learning" without the literal words). Mitigation: `trigram` tokenizer handles morphology well for English/Spanish.
--- ---
## 🌐 5. Integration with Astro (Portfolio) ## 🌐 5. Embedding the widget
### 5.1 Recommended pattern: Astro proxy The bot ships with a drop-in vanilla-JS widget. Add two files to your site and it works.
``` ### 5.1 The widget (any site)
[Browser] ←→ [Astro SSR :4321] ←→ [Chat-Bot :7331]
```html
<link rel="stylesheet" href="/path/to/chat-widget.css">
<script src="/path/to/chat-widget.js"
data-api-url="https://chat.example.com"
data-title="Ask me anything"
data-greeting="Hi! Ask me about the projects."
data-position="bottom-right"
data-theme="auto"
defer></script>
``` ```
**Why proxy and not direct browser call to chat-bot:** A bubble appears bottom-right, opens a panel, talks SSE to `/api/chat`, streams the response, and cites sources. No build step, no React/Vue, no framework lock-in.
- ✅ Single domain (no CORS)
- ✅ Astro handles auth/session if needed
- ✅ There can be centralized rate limiting in Astro
- ✅ The chat-bot stays on private network (not exposed to internet directly)
### 5.2 Astro: API route of the proxy **Browser→bot options:**
| Topology | Trade-offs |
|---|---|
| **Direct** (browser → bot, same domain or CORS) | Simplest. Add the bot's origin to `cors_origins` in YAML. |
| **Reverse proxy** (nginx/Caddy in front) | Bot stays on private network, single public domain, no CORS to manage. |
| **Site proxies the bot** (Astro/Next API route) | Adds a hop and a bit of code, but gives you auth/session hooks in your site. |
The widget works the same in all three. Pick the topology that matches your infra.
> **Default dev setup is direct + CORS.** `cors_origins` in `configs/portfolio-bot.yaml` controls which sites can call the bot. Add your site's origin there.
### 5.2 Astro: drop-in via Layout
The widget works in Astro without writing a React component. Add this to your shared layout:
```astro
---
// src/layouts/BaseLayout.astro
import "../path/to/chat-widget.css";
const apiUrl = import.meta.env.PUBLIC_CHAT_API_URL || "http://localhost:7331";
---
<html>
<body>
<slot />
<script src="/path/to/chat-widget.js"
data-api-url={apiUrl}
data-title="Ask me anything"
data-position="bottom-right"
data-theme="auto"
defer is:inline></script>
</body>
</html>
```
`is:inline` keeps Astro from hashing/transforming the script tag, so the `data-*` attributes survive.
### 5.3 React / Next.js: same script tag
```tsx
// app/layout.tsx
import Script from "next/script";
export default function RootLayout({ children }) {
return (
<html>
<head>
<link rel="stylesheet" href="/chat-widget.css" />
<Script src="/chat-widget.js"
data-api-url={process.env.NEXT_PUBLIC_CHAT_API_URL}
data-title="Ask me anything"
data-position="bottom-right"
data-theme="auto"
strategy="afterInteractive" />
</head>
<body>{children}</body>
</html>
);
}
```
### 5.4 If you want a server proxy (Astro/Next API route)
The widget can also call a same-origin endpoint that forwards to the bot. This is the right call when you need:
- Auth on `/api/chat` (logged-in users only)
- Centralized rate limiting at the site level
- Hiding the bot's origin from the browser
```typescript ```typescript
// portfolio/src/pages/api/chat.ts // src/pages/api/chat.ts (Astro) or app/api/chat/route.ts (Next)
import type { APIRoute } from 'astro'; const CHAT_BOT_URL = process.env.CHAT_BOT_URL || "http://localhost:7331";
const CHAT_BOT_URL = import.meta.env.CHAT_BOT_URL || 'http://localhost:7331'; export const POST = async ({ request }) => {
export const POST: APIRoute = async ({ request }) => {
const body = await request.json(); const body = await request.json();
// (optional) auth check, rate limit, session lookup here
const resp = await fetch(`${CHAT_BOT_URL}/api/chat`, { const resp = await fetch(`${CHAT_BOT_URL}/api/chat`, {
method: 'POST', method: "POST",
headers: { 'Content-Type': 'application/json' }, headers: { "Content-Type": "application/json" },
body: JSON.stringify(body), body: JSON.stringify(body),
}); });
if (!resp.ok) {
return new Response('Chat bot error', { status: resp.status });
}
// Stream SSE back to browser
return new Response(resp.body, { return new Response(resp.body, {
status: 200, status: resp.status,
headers: { headers: {
'Content-Type': 'text/event-stream', "Content-Type": "text/event-stream",
'Cache-Control': 'no-cache', "Cache-Control": "no-cache",
'Connection': 'keep-alive', "Connection": "keep-alive",
}, },
}); });
}; };
``` ```
### 5.3 React: Chat component Then point the widget at `/api/chat` (same origin) instead of the bot's URL.
```tsx ### 5.5 Widget configuration reference
// portfolio/src/components/Chat.tsx
import { useState, useRef } from 'react';
interface Message { All options are `data-*` attributes on the `<script>` tag:
role: 'user' | 'assistant';
content: string;
}
export default function Chat() { | Attribute | Default | Notes |
const [messages, setMessages] = useState<Message[]>([]); |---|---|---|
const [input, setInput] = useState(''); | `data-api-url` | *(required)* | Base URL of the bot. No trailing slash. |
const [streaming, setStreaming] = useState(false); | `data-title` | `"Chat"` | Header text. |
const abortRef = useRef<AbortController | null>(null); | `data-greeting` | `""` | First assistant message when the panel opens. |
| `data-position` | `"bottom-right"` | `"bottom-right"` or `"bottom-left"`. |
| `data-theme` | `"auto"` | `"auto"` (follows OS), `"light"`, `"dark"`. |
const send = async () => { Theming is via CSS custom properties on `.rony-chat-widget-root` (see `web/chat-widget.css`):
if (!input.trim() || streaming) return;
const userMsg: Message = { role: 'user', content: input }; ```css
setMessages(prev => [...prev, userMsg]); .rony-chat-widget-root {
setInput(''); --rony-accent: #ff6b35;
setStreaming(true); --rony-radius: 4px;
--rony-font: "Inter", sans-serif;
// Placeholder for streaming
const assistantMsg: Message = { role: 'assistant', content: '' };
setMessages(prev => [...prev, assistantMsg]);
abortRef.current = new AbortController();
try {
const resp = await fetch('/api/chat', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
messages: [...messages, userMsg],
stream: true,
}),
signal: abortRef.current.signal,
});
const reader = resp.body!.getReader();
const decoder = new TextDecoder();
let buffer = '';
while (true) {
const { done, value } = await reader.read();
if (done) break;
buffer += decoder.decode(value, { stream: true });
const lines = buffer.split('\n\n');
buffer = lines.pop() || '';
for (const line of lines) {
if (!line.startsWith('data: ')) continue;
const event = JSON.parse(line.slice(6));
if (event.type === 'chunk') {
setMessages(prev => {
const updated = [...prev];
updated[updated.length - 1].content += event.data.content;
return updated;
});
}
}
}
} catch (err) {
if ((err as Error).name !== 'AbortError') {
console.error(err);
}
} finally {
setStreaming(false);
abortRef.current = null;
}
};
const stop = () => abortRef.current?.abort();
return (
<div className="chat-widget">
<div className="messages">
{messages.map((m, i) => (
<div key={i} className={`msg msg-${m.role}`}>
{m.content || (streaming && i === messages.length - 1 ? '...' : '')}
</div>
))}
</div>
<div className="input-row">
<input
value={input}
onChange={e => setInput(e.target.value)}
onKeyDown={e => e.key === 'Enter' && send()}
placeholder="Ask about Victor..."
disabled={streaming}
/>
{streaming ? (
<button onClick={stop}>Stop</button>
) : (
<button onClick={send}>Send</button>
)}
</div>
</div>
);
} }
``` ```
### 5.6 What the widget doesn't do (yet)
- **Conversation persistence** — each visit is a fresh conversation. Bot is stateless.
- **Richer markdown** (tables, images) — the built-in renderer handles the common cases; for full CommonMark, swap `renderMarkdown` in `chat-widget.js` for `marked` or `markdown-it`.
- **Mobile swipe-to-dismiss** — panel goes full-screen on phones.
- **Conversation history sidebar** — only the active conversation is shown.
--- ---
## 🤖 6. Self-hosting with Ollama ## 🤖 6. Self-hosting with llama.cpp (default)
### 6.1 Setup ### 6.1 Setup
llama-server is a separate process that the bot connects to over HTTP. **Both ports (the bot's and llama-server's) are configurable** — pick what fits your environment.
```bash
# 1. Make sure you have a GGUF model available
# Download from Hugging Face, e.g.:
# https://huggingface.co/Qwen/Qwen2.5-3B-Instruct-GGUF
export RONY_MODELS_PATH=/path/to/models
ls $RONY_MODELS_PATH/qwen2.5-3b-instruct-q4_k_m.gguf
# 2. Start llama-server (port is configurable; default llama.cpp is 8080)
llama-server \
-m $RONY_MODELS_PATH/qwen2.5-3b-instruct-q4_k_m.gguf \
--port 9100 \
--host 127.0.0.1 \
--ctx-size 4096 \
--mlock # prevents swap, critical on shared VPS
# 3. Make sure configs/portfolio-bot.yaml points to the same port
# providers[0].endpoint: http://localhost:9100/v1
# 4. Start the bot (default port 7331, also configurable)
./bin/chat-bot serve
# → Serves on http://localhost:7331
# → Override with: ./bin/chat-bot serve --port 9101 --host 127.0.0.1
```
**Port reference:**
| What | Default | How to change |
|---|---|---|
| `llama-server` HTTP port | 8080 (llama.cpp convention) | `--port N` flag when starting `llama-server` |
| chat-bot HTTP port | 7331 | `--port N` flag on `serve`, or `server.port` in YAML |
| chat-bot → llama-server URL | `http://localhost:8080/v1` | `endpoint` field on the provider in YAML |
The `llamacpp` provider is imported from `rony-llm-agent/pkg/llm/providers/llamacpp` and is compiled against `llama.cpp` via CGO or external binary.
### 6.2 Alternative: Ollama (easier for development)
If you don't want to manage GGUF files manually, Ollama provides the same models with a simpler workflow:
```bash ```bash
# 1. Install Ollama # 1. Install Ollama
curl -fsSL https://ollama.com/install.sh | sh curl -fsSL https://ollama.com/install.sh | sh
@ -640,26 +822,19 @@ curl -fsSL https://ollama.com/install.sh | sh
# 2. Download chat model # 2. Download chat model
ollama pull qwen2.5:1.5b ollama pull qwen2.5:1.5b
# 3. Download embeddings model # 3. Verify
ollama pull nomic-embed-text
# 4. Verify
ollama list ollama list
```
### 6.2 Default configuration # 4. Edit configs/portfolio-bot.yaml to mark ollama-local as default:
# providers[0].default: true (and remove default from llamacpp-local)
# Ollama exposes an OpenAI-compatible API on :11434/v1
`configs/portfolio-bot.yaml` already comes with Ollama as default. You only need: # 5. Start the bot
ollama serve &
```bash
# Make sure Ollama is running
ollama serve
# Start the bot
./bin/chat-bot serve ./bin/chat-bot serve
``` ```
### 6.3 Alternative: llama.cpp direct ### 6.3 Alternative: llama.cpp direct (advanced)
For more control or if Ollama doesn't work in your setup: For more control or if Ollama doesn't work in your setup:
@ -667,9 +842,10 @@ For more control or if Ollama doesn't work in your setup:
providers: providers:
- name: llamacpp-local - name: llamacpp-local
type: llamacpp type: llamacpp
model_path: ${RONY_MODELS_PATH}/qwen2.5-1.5b-instruct-q5_k_m.gguf model: qwen2.5-3b-instruct
endpoint: http://localhost:9100/v1 # configurable, see §6.1
context_size: 4096 context_size: 4096
n_gpu_layers: 999 # offload all to GPU max_tokens: 2048
default: true default: true
``` ```
@ -685,7 +861,7 @@ The `llamacpp` adapter is imported from `rony-llm-agent/pkg/llm/providers/llamac
# Start HTTP server # Start HTTP server
chat-bot serve [--port 7331] [--host 0.0.0.0] [--reindex-on-start] chat-bot serve [--port 7331] [--host 0.0.0.0] [--reindex-on-start]
# Re-index portfolio (reads data/projects/*.md → ChromaDB) # Re-index portfolio (reads data/projects/*.md → SQLite FTS5)
chat-bot reindex chat-bot reindex
# Single question (no server, useful for tests) # Single question (no server, useful for tests)
@ -764,7 +940,6 @@ func serveCmd() *cobra.Command {
# 1. Install dependencies # 1. Install dependencies
sudo apt install golang-go ollama sudo apt install golang-go ollama
ollama pull qwen2.5:1.5b ollama pull qwen2.5:1.5b
ollama pull nomic-embed-text
# 2. Build # 2. Build
go build -o /usr/local/bin/chat-bot ./cmd/chat-bot go build -o /usr/local/bin/chat-bot ./cmd/chat-bot
@ -913,27 +1088,37 @@ rony-chat-bot/
├── internal/ ├── internal/
│ ├── server/ # HTTP handlers │ ├── server/ # HTTP handlers
│ │ ├── chat.go # POST /api/chat │ │ ├── server.go # chi router + middleware
│ │ ├── reindex.go # POST /api/reindex │ │ ├── handlers.go # /api/chat, /api/health, /api/info, /api/reindex
│ │ ├── health.go # GET /api/health │ │ └── middleware.go # RequestID, Logging, CORS, RateLimit
│ │ ├── info.go # GET /api/info
│ │ ├── middleware.go # logging, CORS, rate limit
│ │ └── sse.go # SSE helpers
│ │ │ │
│ ├── agent/ # Wrapper over rony-llm-agent │ ├── agent/ # LLM client + RAG runner
│ │ ├── runner.go # RunStream with RAG injection │ │ ├── runner.go # Stream wrapper, RAG injection into system prompt
│ │ └── prompts.go # System prompt builder │ │ └── client.go # NewClient factory: llamacpp / ollama / openai / anthropic
│ │ │ │
│ ├── portfolio/ # Data loader │ ├── portfolio/ # RAG: markdown → SQLite FTS5
│ │ ├── indexer.go # Reads .md, chunks, embed, store │ │ ├── chunker.go # Heading-based splitter
│ │ ├── retriever.go # Query → top-k chunks │ │ ├── indexer.go # Store: schema, Reindex, Search (BM25)
│ │ └── chunker.go # Text splitting │ │ └── chunker_test.go / store_test.go
│ │ │ │
│ └── persona/ # Persona override │ ├── persona/ # Persona bridge to rony-llm-agent
│ └── loader.go # Loads persona from YAML │ │ └── persona.go # FromConfig, BuildSystemPrompt (with RAG context)
│ │
│ ├── streaming/ # SSE protocol helpers
│ │ └── sse.go # WriteStart/Chunk/Sources/Done/Error
│ │
│ ├── i18n/ # Language detection (ES/EN) for the response
│ │
│ └── config/ # YAML loader + validation
├── web/ # ← DROP-IN CHAT WIDGET
│ ├── chat-widget.js # Vanilla JS, ~12 KB
│ ├── chat-widget.css # Scoped styles, CSS-custom-prop themable
│ ├── example.html # Local demo (python -m http.server)
│ └── README.md # Integration guide (HTML, Astro, Next.js)
├── data/ ├── data/
│ └── projects/ # ← Markdown per project │ └── projects/ # ← Markdown per project (one .md per project)
│ ├── rony-harness.md │ ├── rony-harness.md
│ ├── rony-llm-agent.md │ ├── rony-llm-agent.md
│ └── example-project.md │ └── example-project.md
@ -942,9 +1127,12 @@ rony-chat-bot/
│ └── portfolio-bot.yaml # Provider + RAG + persona config │ └── portfolio-bot.yaml # Provider + RAG + persona config
├── docs/ ├── docs/
│ └── architecture.md # ← THIS FILE │ ├── architecture.md # ← THIS FILE
│ └── architecture.es.md
├── go.mod ├── bench/ # Reproducible SQLite driver benchmark
├── go.mod # require rony-llm-agent, modernc.org/sqlite
└── README.md └── README.md
``` ```
@ -957,10 +1145,10 @@ rony-chat-bot/
- [ ] Project setup (`go mod init`, structure) - [ ] Project setup (`go mod init`, structure)
- [ ] Basic HTTP server with `/api/chat` endpoint - [ ] Basic HTTP server with `/api/chat` endpoint
- [ ] Functional SSE streaming - [ ] Functional SSE streaming
- [ ] RAG indexer (reads `data/projects/*.md`ChromaDB) - [ ] RAG indexer (reads `data/projects/*.md`SQLite FTS5)
- [ ] RAG retriever (query → top-k chunks) - [ ] RAG retriever (query → top-k chunks)
- [ ] Persona loader from YAML - [ ] Persona loader from YAML
- [ ] Ollama integration (qwen2.5:1.5b) - [ ] llama.cpp integration (qwen2.5:1.5b GGUF)
- [ ] CLI: `serve`, `reindex`, `ask` - [ ] CLI: `serve`, `reindex`, `ask`
- [ ] Basic tests - [ ] Basic tests
@ -995,7 +1183,7 @@ rony-chat-bot/
| Metric | Target | | Metric | Target |
|---|---| |---|---|
| TTFT (Time-to-first-token) | <500ms with Ollama local | | TTFT (Time-to-first-token) | <500ms with llama.cpp local |
| End-to-end (question → complete response) | <3s for typical responses | | End-to-end (question → complete response) | <3s for typical responses |
| Memory at rest | <150MB | | Memory at rest | <150MB |
| RAG indexing speed | ~100 docs/second | | RAG indexing speed | ~100 docs/second |
@ -1004,7 +1192,7 @@ rony-chat-bot/
### 12.2 Required tests ### 12.2 Required tests
- Unit tests: coverage ≥70% - Unit tests: coverage ≥70%
- Integration tests: with mock LLM + mock ChromaDB - Integration tests: with mock LLM + in-memory SQLite FTS5
- E2E: at least one complete Astro → chat-bot flow - E2E: at least one complete Astro → chat-bot flow
--- ---
@ -1032,8 +1220,8 @@ rony-chat-bot/
- **SSE Spec:** https://html.spec.whatwg.org/multipage/server-sent-events.html - **SSE Spec:** https://html.spec.whatwg.org/multipage/server-sent-events.html
- **Ollama API:** https://github.com/ollama/ollama/blob/main/docs/api.md - **Ollama API:** https://github.com/ollama/ollama/blob/main/docs/api.md
- **ChromaDB Go:** https://github.com/amikos-tech/chroma-go - **SQLite FTS5:** https://www.sqlite.org/fts5.html
- **nomic-embed-text:** https://huggingface.co/nomic-ai/nomic-embed-text-v1.5 - **Go SQLite driver:** https://github.com/mattn/go-sqlite3 (CGO) or https://modernc.org/sqlite (pure Go)
- **qwen2.5:** https://huggingface.co/Qwen/Qwen2.5-1.5B-Instruct - **qwen2.5:** https://huggingface.co/Qwen/Qwen2.5-1.5B-Instruct
- **Astro API routes:** https://docs.astro.build/en/guides/endpoints/ - **Astro API routes:** https://docs.astro.build/en/guides/endpoints/
- **rony-llm-agent:** https://github.com/VictorVargas/rony-llm-agent - **rony-llm-agent:** https://github.com/VictorVargas/rony-llm-agent

25
go.mod
View file

@ -1,3 +1,28 @@
module github.com/VictorVargas/rony-chat-bot module github.com/VictorVargas/rony-chat-bot
go 1.26 go 1.26
require (
github.com/go-chi/chi/v5 v5.3.1
github.com/spf13/cobra v1.10.2
gopkg.in/yaml.v3 v3.0.1
modernc.org/sqlite v1.53.0
)
require (
github.com/dustin/go-humanize v1.0.1 // indirect
github.com/google/uuid v1.6.0 // indirect
github.com/mattn/go-isatty v0.0.20 // indirect
github.com/ncruces/go-strftime v1.0.0 // indirect
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect
golang.org/x/sys v0.44.0 // indirect
modernc.org/libc v1.73.4 // indirect
modernc.org/mathutil v1.7.1 // indirect
modernc.org/memory v1.11.0 // indirect
)
require (
github.com/inconshreveable/mousetrap v1.1.0 // indirect
github.com/mattn/go-sqlite3 v1.14.48
github.com/spf13/pflag v1.0.10 // indirect
)

69
go.sum Normal file
View file

@ -0,0 +1,69 @@
github.com/cpuguy83/go-md2man/v2 v2.0.6/go.mod h1:oOW0eioCTA6cOiMLiUPZOpcVxMig6NIQQ7OS05n1F4g=
github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY=
github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto=
github.com/go-chi/chi/v5 v5.3.1 h1:3j4HZLGZQ3JpMCrPJF/Jl3mYJfWLKBfNJ6quurUGCf8=
github.com/go-chi/chi/v5 v5.3.1/go.mod h1:R+tYY2hNuVUUjxoPtqUdgBqevM9s9njzkTLutVsOCto=
github.com/google/pprof v0.0.0-20250317173921-a4b03ec1a45e h1:ijClszYn+mADRFY17kjQEVQ1XRhq2/JR1M3sGqeJoxs=
github.com/google/pprof v0.0.0-20250317173921-a4b03ec1a45e/go.mod h1:boTsfXsheKC2y+lKOCMpSfarhxDeIzfZG1jqGcPl3cA=
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/hashicorp/golang-lru/v2 v2.0.7 h1:a+bsQ5rvGLjzHuww6tVxozPZFVghXaHOwFs4luLUK2k=
github.com/hashicorp/golang-lru/v2 v2.0.7/go.mod h1:QeFd9opnmA6QUJc5vARoKUSoFhyfM2/ZepoAG6RGpeM=
github.com/inconshreveable/mousetrap v1.1.0 h1:wN+x4NVGpMsO7ErUn/mUI3vEoE6Jt13X2s0bqwp9tc8=
github.com/inconshreveable/mousetrap v1.1.0/go.mod h1:vpF70FUmC8bwa3OWnCshd2FqLfsEA9PFc4w1p2J65bw=
github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY=
github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
github.com/mattn/go-sqlite3 v1.14.48 h1:7XHIgl0a8HwOaiK4E47ozLkST78rR9+OtNGx27D/TFs=
github.com/mattn/go-sqlite3 v1.14.48/go.mod h1:6JTjA44L93a0QCyJef5YvlPoKXntQPjzWv5gtm9sB6w=
github.com/ncruces/go-strftime v1.0.0 h1:HMFp8mLCTPp341M/ZnA4qaf7ZlsbTc+miZjCLOFAw7w=
github.com/ncruces/go-strftime v1.0.0/go.mod h1:Fwc5htZGVVkseilnfgOVb9mKy6w1naJmn9CehxcKcls=
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec h1:W09IVJc94icq4NjY3clb7Lk8O1qJ8BdBEF8z0ibU0rE=
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec/go.mod h1:qqbHyh8v60DhA7CoWK5oRCqLrMHRGoxYCSS9EjAz6Eo=
github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
github.com/spf13/cobra v1.10.2 h1:DMTTonx5m65Ic0GOoRY2c16WCbHxOOw6xxezuLaBpcU=
github.com/spf13/cobra v1.10.2/go.mod h1:7C1pvHqHw5A4vrJfjNwvOdzYu0Gml16OCs2GRiTUUS4=
github.com/spf13/pflag v1.0.9/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
github.com/spf13/pflag v1.0.10 h1:4EBh2KAYBwaONj6b2Ye1GiHfwjqyROoF4RwYO+vPwFk=
github.com/spf13/pflag v1.0.10/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
go.yaml.in/yaml/v3 v3.0.4/go.mod h1:DhzuOOF2ATzADvBadXxruRBLzYTpT36CKvDb3+aBEFg=
golang.org/x/mod v0.36.0 h1:JJjpVx6myfUsUdAzZuOSTTmRE0PfZeNWzzvKrP7amb4=
golang.org/x/mod v0.36.0/go.mod h1:moc6ELqsWcOw5Ef3xVprK5ul/MvtVvkIXLziUOICjUQ=
golang.org/x/sync v0.20.0 h1:e0PTpb7pjO8GAtTs2dQ6jYa5BWYlMuX047Dco/pItO4=
golang.org/x/sync v0.20.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0=
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.44.0 h1:ildZl3J4uzeKP07r2F++Op7E9B29JRUy+a27EibtBTQ=
golang.org/x/sys v0.44.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
golang.org/x/tools v0.45.0 h1:18qN3FAooORvApf5XjCXgsuayZOEtXf6JK18I3+ONa8=
golang.org/x/tools v0.45.0/go.mod h1:LuUGqqaXcXMEFEruIVJVm5mgDD8vww/z/SR1gQ4uE/0=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
modernc.org/cc/v4 v4.28.4 h1:Hd/4Es+MBj+/7hSdZaisNyu6bv3V0Dp2MdllyfqaH+c=
modernc.org/cc/v4 v4.28.4/go.mod h1:OnovgIhbbMXMu1aISnJ0wvVD1KnW+cAUJkIrAWh+kVI=
modernc.org/ccgo/v4 v4.34.4 h1:OVnSOWQjVKOYkFxoHYB+qQmSHK5gqMqARM+K9DpR/Ws=
modernc.org/ccgo/v4 v4.34.4/go.mod h1:qdKqE8FNIYyysougB1RX9MxCzp5oJOcQXSobANJ4TuE=
modernc.org/fileutil v1.4.0 h1:j6ZzNTftVS054gi281TyLjHPp6CPHr2KCxEXjEbD6SM=
modernc.org/fileutil v1.4.0/go.mod h1:EqdKFDxiByqxLk8ozOxObDSfcVOv/54xDs/DUHdvCUU=
modernc.org/gc/v2 v2.6.5 h1:nyqdV8q46KvTpZlsw66kWqwXRHdjIlJOhG6kxiV/9xI=
modernc.org/gc/v2 v2.6.5/go.mod h1:YgIahr1ypgfe7chRuJi2gD7DBQiKSLMPgBQe9oIiito=
modernc.org/gc/v3 v3.1.3 h1:6QAplYyVO+KdPW3pGnqmJDUxtkec8ooEWvks/hhU3lc=
modernc.org/gc/v3 v3.1.3/go.mod h1:HFK/6AGESC7Ex+EZJhJ2Gni6cTaYpSMmU/cT9RmlfYY=
modernc.org/goabi0 v0.2.0 h1:HvEowk7LxcPd0eq6mVOAEMai46V+i7Jrj13t4AzuNks=
modernc.org/goabi0 v0.2.0/go.mod h1:CEFRnnJhKvWT1c1JTI3Avm+tgOWbkOu5oPA8eH8LnMI=
modernc.org/libc v1.73.4 h1:+ra4Ui8ngyt8HDcO1FTDPWlkAh6yOdaO2yAoh8MddQA=
modernc.org/libc v1.73.4/go.mod h1:DXZ3eO8qMCNn2SnmTNCiC71nJ9Rcq3PsnpU6Vc4rWK8=
modernc.org/mathutil v1.7.1 h1:GCZVGXdaN8gTqB1Mf/usp1Y/hSqgI2vAGGP4jZMCxOU=
modernc.org/mathutil v1.7.1/go.mod h1:4p5IwJITfppl0G4sUEDtCr4DthTaT47/N3aT6MhfgJg=
modernc.org/memory v1.11.0 h1:o4QC8aMQzmcwCK3t3Ux/ZHmwFPzE6hf2Y5LbkRs+hbI=
modernc.org/memory v1.11.0/go.mod h1:/JP4VbVC+K5sU2wZi9bHoq2MAkCnrt2r98UGeSK7Mjw=
modernc.org/opt v0.2.0 h1:tGyef5ApycA7FSEOMraay9SaTk5zmbx7Tu+cJs4QKZg=
modernc.org/opt v0.2.0/go.mod h1:03fq9lsNfvkYSfxrfUhZCWPk1lm4cq4N+Bh//bEtgns=
modernc.org/sortutil v1.2.1 h1:+xyoGf15mM3NMlPDnFqrteY07klSFxLElE2PVuWIJ7w=
modernc.org/sortutil v1.2.1/go.mod h1:7ZI3a3REbai7gzCLcotuw9AC4VZVpYMjDzETGsSMqJE=
modernc.org/sqlite v1.53.0 h1:20WG8N9q4ji/dEqGk4uiI0c6OPjSeLTNYGFCc3+7c1M=
modernc.org/sqlite v1.53.0/go.mod h1:xoEpOIpGrgT48H5iiyt/YXPCZPEzlfmfFwtk8Lklw8s=
modernc.org/strutil v1.2.1 h1:UneZBkQA+DX2Rp35KcM69cSsNES9ly8mQWD71HKlOA0=
modernc.org/strutil v1.2.1/go.mod h1:EHkiggD70koQxjVdSBM3JKM7k6L0FbGE5eymy9i3B9A=
modernc.org/token v1.1.0 h1:Xl7Ap9dKaEs5kLoOQeQmPWevfnk/DM5qcLcYlA8ys6Y=
modernc.org/token v1.1.0/go.mod h1:UGzOrNV1mAFSEB63lOFHIpNRUVMvYTc6yu1SMY/XTDM=

52
internal/agent/client.go Normal file
View file

@ -0,0 +1,52 @@
package agent
import (
"fmt"
"os"
"github.com/VictorVargas/rony-llm-agent/pkg/llm"
"github.com/VictorVargas/rony-llm-agent/pkg/llm/providers/anthropic"
"github.com/VictorVargas/rony-llm-agent/pkg/llm/providers/llamacpp"
"github.com/VictorVargas/rony-llm-agent/pkg/llm/providers/openai"
"github.com/VictorVargas/rony-chat-bot/internal/config"
)
// NewClient constructs the upstream LLMClient for a given provider config.
// "ollama" is handled via the openai-compat adapter: Ollama exposes
// /v1/chat/completions on its own port, so the provider list stays small.
func NewClient(p config.Provider) (llm.LLMClient, error) {
switch p.Type {
case "llamacpp":
return llamacpp.New(llamacpp.Config{
BaseURL: defaultIfEmpty(p.Endpoint, "http://localhost:8080/v1"),
Model: p.Model,
ContextWindow: p.ContextSize,
MaxTokens: p.MaxTokens,
Temperature: p.Temperature,
})
case "ollama", "openai":
return openai.New(openai.Config{
BaseURL: defaultIfEmpty(p.Endpoint, "http://localhost:11434/v1"),
Model: p.Model,
})
case "anthropic":
apiKey := ""
if p.APIKeyEnv != "" {
apiKey = os.Getenv(p.APIKeyEnv)
}
return anthropic.New(anthropic.Config{
APIKey: apiKey,
Model: p.Model,
})
default:
return nil, fmt.Errorf("unknown provider type %q (supported: llamacpp, ollama, openai, anthropic)", p.Type)
}
}
func defaultIfEmpty(s, def string) string {
if s == "" {
return def
}
return s
}

125
internal/agent/runner.go Normal file
View file

@ -0,0 +1,125 @@
// Package agent wraps the LLM client + RAG pipeline behind a single
// streaming call the HTTP handler can drive.
//
// We use llm.LLMClient directly (not agent.Loop) because the bot is a
// straight Q&A flow: no tools, no multi-iteration reasoning. Calling the
// underlying provider keeps the prompt under our full control so we can
// inject RAG context into the system message exactly where we want it.
package agent
import (
"context"
"fmt"
"iter"
"strings"
"github.com/VictorVargas/rony-llm-agent/pkg/llm"
llmpersona "github.com/VictorVargas/rony-llm-agent/pkg/persona"
botpersona "github.com/VictorVargas/rony-chat-bot/internal/persona"
"github.com/VictorVargas/rony-chat-bot/internal/portfolio"
)
// Message aliases keep the HTTP handler decoupled from the upstream types.
type Message = llm.Message
type Role = llm.Role
const (
RoleSystem = llm.RoleSystem
RoleUser = llm.RoleUser
RoleAssistant = llm.RoleAssistant
)
// Runner ties together an LLM client, the system prompt, and the RAG store.
// The persona struct is kept only for the UI greeting (its name + intro);
// the system prompt itself lives in the YAML and is passed in directly.
type Runner struct {
client llm.LLMClient
persona llmpersona.Persona
systemPrompt string
store *portfolio.Store
topK int
usage *Usage
}
type Usage struct {
InputTokens int
OutputTokens int
}
// New returns a Runner. The store may be nil (RAG disabled).
// systemPrompt is the full hand-written prompt from configs/...yaml.
func New(client llm.LLMClient, p llmpersona.Persona, systemPrompt string, store *portfolio.Store, topK int) *Runner {
return &Runner{client: client, persona: p, systemPrompt: systemPrompt, store: store, topK: topK, usage: &Usage{}}
}
// LastUsage returns the token usage recorded on the most recent call.
func (r *Runner) LastUsage() Usage { return *r.usage }
// BuildMessages prepares the system prompt and turns the chat history into
// the upstream message list. The system prompt includes RAG context for the
// user's last message (if RAG is enabled).
func (r *Runner) BuildMessages(ctx context.Context, history []Message) ([]Message, string, error) {
ragContext := ""
if r.store != nil && len(history) > 0 {
last := history[len(history)-1]
if last.Role == RoleUser {
hits, err := r.store.Search(ctx, last.Content, r.topK)
if err != nil {
return nil, "", fmt.Errorf("rag search: %w", err)
}
if len(hits) > 0 {
ragContext = formatHits(hits)
}
}
}
// The system prompt comes from the YAML, not from the persona struct.
// Keep the persona around only for the UI greeting.
system := botpersona.BuildSystemPrompt(r.systemPrompt, ragContext)
msgs := make([]Message, 0, len(history)+1)
msgs = append(msgs, Message{Role: RoleSystem, Content: system})
msgs = append(msgs, history...)
return msgs, ragContext, nil
}
// Stream runs the model and yields each streamed chunk. The caller
// forwards chunk.Delta to the SSE stream; chunk.Usage on the last chunk
// carries token counts.
func (r *Runner) Stream(ctx context.Context, history []Message) iter.Seq2[llm.StreamChunk, error] {
return func(yield func(llm.StreamChunk, error) bool) {
msgs, _, err := r.BuildMessages(ctx, history)
if err != nil {
yield(llm.StreamChunk{}, err)
return
}
req := llm.CompletionRequest{
Messages: msgs,
// No tools: this is a Q&A bot, not an agent.
}
for chunk, err := range r.client.Stream(ctx, req) {
if chunk.Usage.TotalTokens > 0 || chunk.Usage.InputTokens > 0 || chunk.Usage.OutputTokens > 0 {
r.usage = &Usage{
InputTokens: chunk.Usage.InputTokens,
OutputTokens: chunk.Usage.OutputTokens,
}
}
if !yield(chunk, err) {
return
}
if err != nil {
return
}
}
}
}
func formatHits(hits []portfolio.SearchResult) string {
var b strings.Builder
for i, h := range hits {
fmt.Fprintf(&b, "### [%d] %s — %s\n", i+1, h.ProjectID, h.Section)
b.WriteString(h.Content)
b.WriteString("\n\n")
}
return b.String()
}

View file

@ -0,0 +1,112 @@
package agent
import (
"context"
"iter"
"strings"
"testing"
"github.com/VictorVargas/rony-llm-agent/pkg/llm"
llmpersona "github.com/VictorVargas/rony-llm-agent/pkg/persona"
"github.com/VictorVargas/rony-chat-bot/internal/portfolio"
)
// stubClient is a minimal llm.LLMClient that echoes the system prompt's
// last "###" block as a single chunk, then a usage chunk.
type stubClient struct {
gotMessages []llm.Message
}
func (s *stubClient) Generate(_ context.Context, _ llm.CompletionRequest) (llm.CompletionResponse, error) {
return llm.CompletionResponse{Content: "ok", Usage: llm.TokenUsage{InputTokens: 1, OutputTokens: 1}}, nil
}
func (s *stubClient) Stream(_ context.Context, req llm.CompletionRequest) iter.Seq2[llm.StreamChunk, error] {
s.gotMessages = req.Messages
return func(yield func(llm.StreamChunk, error) bool) {
yield(llm.StreamChunk{Delta: "echo: " + req.Messages[len(req.Messages)-1].Content}, nil)
yield(llm.StreamChunk{Delta: " [done]", FinishReason: "stop", Usage: llm.TokenUsage{InputTokens: 7, OutputTokens: 3}}, nil)
}
}
func (s *stubClient) Name() string { return "stub" }
func (s *stubClient) Capabilities() llm.ProviderCapabilities {
return llm.ProviderCapabilities{MaxContextWindow: 4096}
}
func TestRunnerStreamNoRAG(t *testing.T) {
cli := &stubClient{}
p := llmpersona.Persona{Name: "Tester", Tone: "concise", Language: "English"}
r := New(cli, p, "test system prompt", nil, 5)
var got strings.Builder
for chunk, err := range r.Stream(context.Background(), []llm.Message{{Role: RoleUser, Content: "hi"}}) {
if err != nil {
t.Fatal(err)
}
got.WriteString(chunk.Delta)
}
want := "echo: hi [done]"
if got.String() != want {
t.Errorf("got %q, want %q", got.String(), want)
}
}
func TestRunnerStreamWithRAG(t *testing.T) {
cli := &stubClient{}
p := llmpersona.Persona{Name: "Tester", Tone: "concise", Language: "English"}
// Build a tiny on-disk store with one project.
dir := t.TempDir()
dbPath := dir + "/t.db"
srcDir := dir + "/src"
if err := writeFile(srcDir+"/proj.md", "# Demo\n\n## Tech stack\n- Go\n- SQLite database\n"); err != nil {
t.Fatal(err)
}
store, err := portfolio.OpenStore(dbPath)
if err != nil {
t.Fatal(err)
}
defer store.Close()
if _, _, err := store.Reindex(context.Background(), srcDir, portfolio.DefaultChunkerConfig()); err != nil {
t.Fatal(err)
}
r := New(cli, p, "test system prompt", store, 5)
for chunk, err := range r.Stream(context.Background(), []llm.Message{{Role: RoleUser, Content: "What database?"}}) {
if err != nil {
t.Fatal(err)
}
_ = chunk
}
// The system message the LLM saw should include the RAG context.
if len(cli.gotMessages) == 0 {
t.Fatal("LLM never received messages")
}
sys := cli.gotMessages[0].Content
if !strings.Contains(sys, "Relevant context") {
t.Errorf("system prompt missing RAG context block:\n%s", sys)
}
if !strings.Contains(sys, "SQLite") {
t.Errorf("RAG context missing the SQLite content (got: %s)", sys)
}
}
func writeFile(path, content string) error {
if err := mkdirAll(path); err != nil {
return err
}
return writeFileRaw(path, content)
}
// small helpers to avoid importing os/filepath in tests for a 2-call need
func mkdirAll(path string) error { return osMkdirAll(dirOf(path)) }
func dirOf(p string) string {
for i := len(p) - 1; i >= 0; i-- {
if p[i] == '/' {
return p[:i]
}
}
return "."
}

View file

@ -0,0 +1,8 @@
package agent
import "os"
// indirection so the test file doesn't import "os" and "io" directly.
// Keeps the test readable; the helpers are trivial.
func writeFileRaw(path, content string) error { return os.WriteFile(path, []byte(content), 0o644) }
func osMkdirAll(path string) error { return os.MkdirAll(path, 0o755) }

141
internal/config/config.go Normal file
View file

@ -0,0 +1,141 @@
package config
import (
"fmt"
"os"
"strings"
"gopkg.in/yaml.v3"
)
type Server struct {
Host string `yaml:"host"`
Port int `yaml:"port"`
ReadTimeoutMS int `yaml:"read_timeout_ms"`
CORSOrigins []string `yaml:"cors_origins"`
RateLimit RateLimit `yaml:"rate_limit"`
}
type RateLimit struct {
RequestsPerMinute int `yaml:"requests_per_minute"`
Burst int `yaml:"burst"`
}
type Provider struct {
Name string `yaml:"name"`
Type string `yaml:"type"`
Model string `yaml:"model,omitempty"`
ModelPath string `yaml:"model_path,omitempty"`
Endpoint string `yaml:"endpoint,omitempty"`
ContextSize int `yaml:"context_size,omitempty"`
MaxTokens int `yaml:"max_tokens,omitempty"`
NGPULayers int `yaml:"n_gpu_layers,omitempty"`
Temperature float32 `yaml:"temperature,omitempty"`
APIKeyEnv string `yaml:"api_key_env,omitempty"`
Default bool `yaml:"default,omitempty"`
}
type RAG struct {
Enabled bool `yaml:"enabled"`
DataPath string `yaml:"data_path"`
ChunkSize int `yaml:"chunk_size"`
ChunkOverlap int `yaml:"chunk_overlap"`
DBPath string `yaml:"db_path"`
TopK int `yaml:"top_k"`
Tokenize string `yaml:"tokenize"`
}
type Persona struct {
Name string `yaml:"name"`
Tone string `yaml:"tone"`
Language string `yaml:"language"`
Constraints []string `yaml:"constraints"`
Intro string `yaml:"intro"`
}
type Logging struct {
Level string `yaml:"level"`
Format string `yaml:"format"`
Output string `yaml:"output"`
}
type Config struct {
Server Server `yaml:"server"`
Providers []Provider `yaml:"providers"`
RAG RAG `yaml:"rag"`
Persona Persona `yaml:"persona"`
SystemPrompt string `yaml:"system_prompt"`
Logging Logging `yaml:"logging"`
}
func Load(path string) (*Config, error) {
raw, err := os.ReadFile(path)
if err != nil {
return nil, fmt.Errorf("read config %s: %w", path, err)
}
expanded := os.ExpandEnv(string(raw))
var cfg Config
if err := yaml.Unmarshal([]byte(expanded), &cfg); err != nil {
return nil, fmt.Errorf("parse config %s: %w", path, err)
}
if err := cfg.validate(); err != nil {
return nil, err
}
return &cfg, nil
}
func (c *Config) validate() error {
if len(c.Providers) == 0 {
return fmt.Errorf("config: at least one provider must be configured")
}
defaultCount := 0
for _, p := range c.Providers {
if p.Default {
defaultCount++
}
}
if defaultCount == 0 {
c.Providers[0].Default = true
} else if defaultCount > 1 {
return fmt.Errorf("config: multiple providers marked as default")
}
if c.Server.Port == 0 {
c.Server.Port = 7331
}
if c.Server.ReadTimeoutMS == 0 {
c.Server.ReadTimeoutMS = 30000
}
if c.RAG.ChunkSize == 0 {
c.RAG.ChunkSize = 500
}
if c.RAG.ChunkOverlap == 0 {
c.RAG.ChunkOverlap = 50
}
if c.RAG.TopK == 0 {
c.RAG.TopK = 5
}
if c.RAG.Tokenize == "" {
c.RAG.Tokenize = "unicode61"
}
return nil
}
func (c *Config) DefaultProvider() *Provider {
for i := range c.Providers {
if c.Providers[i].Default {
return &c.Providers[i]
}
}
return &c.Providers[0]
}
func (c *Config) Addr() string {
return fmt.Sprintf("%s:%d", c.Server.Host, c.Server.Port)
}
func splitPath(path string) []string {
return strings.Split(path, string(os.PathSeparator))
}

91
internal/i18n/i18n.go Normal file
View file

@ -0,0 +1,91 @@
package i18n
import (
"strings"
"unicode"
)
var (
esStopwords = map[string]bool{
"el": true, "la": true, "los": true, "las": true, "de": true,
"que": true, "y": true, "en": true, "un": true, "una": true,
"es": true, "se": true, "no": true, "con": true, "para": true,
"por": true, "su": true, "del": true, "al": true, "lo": true,
"qué": true, "cómo": true, "dónde": true, "cuándo": true,
"cuál": true, "cuáles": true, "quién": true, "habla": true,
"tienes": true, "dime": true, "háblame": true, "sobre": true,
"más": true, "pero": true, "como": true, "este": true, "esta": true,
"estos": true, "estas": true, "ese": true, "esa": true, "aquel": true,
"muy": true, "sin": true, "hay": true, "sí": true, "yo": true,
"tú": true, "él": true, "ella": true, "nosotros": true,
}
enStopwords = map[string]bool{
"the": true, "is": true, "are": true, "of": true, "and": true,
"in": true, "to": true, "a": true, "an": true, "for": true,
"with": true, "on": true, "by": true, "from": true, "what": true,
"which": true, "who": true, "how": true, "when": true, "where": true,
"tell": true, "about": true, "do": true, "does": true, "can": true,
"you": true, "your": true, "have": true, "has": true, "i": true,
"we": true, "they": true, "this": true, "that": true, "these": true,
"those": true, "be": true, "been": true, "will": true, "would": true,
"should": true, "could": true, "my": true, "our": true, "their": true,
}
)
// Detect returns "es" or "en" based on lightweight heuristics. Good enough
// to choose the response language for a portfolio chatbot; the LLM (once
// wired in) is the final authority.
//
// Heuristic:
// - Spanish diacritics or ¿/¡ → +N Spanish markers
// - Tokenize and count stopwords in each language
// - Whichever side wins; tie → English
func Detect(text string) string {
if text == "" {
return "en"
}
markers := 0
for _, c := range text {
switch c {
case '¿', '¡':
markers += 2
case 'ñ':
markers += 2
case 'á', 'é', 'í', 'ó', 'ú', 'ü':
markers++
}
}
es, en := 0, 0
for _, tok := range tokenize(text) {
if esStopwords[tok] {
es++
}
if enStopwords[tok] {
en++
}
}
switch {
case markers >= 2:
return "es"
case es > en:
return "es"
case en > es:
return "en"
default:
return "en"
}
}
func tokenize(s string) []string {
s = strings.ToLower(s)
f := func(c rune) bool {
if unicode.IsLetter(c) || unicode.IsDigit(c) {
return false
}
return true
}
return strings.FieldsFunc(s, f)
}

View file

@ -0,0 +1,24 @@
package i18n
import "testing"
func TestDetect(t *testing.T) {
cases := []struct {
in, want string
}{
{"Hola, ¿qué proyectos tienes?", "es"},
{"háblame de rony-llm-agent", "es"},
{"¿cuáles son tus skills?", "es"},
{"What projects do you have?", "en"},
{"Tell me about rony-llm-agent", "en"},
{"How does the chat bot work?", "en"},
{"lorem ipsum dolor sit amet", "en"}, // no markers → en
{"", "en"},
}
for _, c := range cases {
got := Detect(c.in)
if got != c.want {
t.Errorf("Detect(%q) = %q, want %q", c.in, got, c.want)
}
}
}

View file

@ -0,0 +1,44 @@
// Package persona owns the bot's identity and the system prompt the LLM
// sees. The full prompt text lives in the YAML's `system_prompt` field
// (long, freeform, hand-tuned for the deployment). This package only
// adds the RAG context after it.
package persona
import (
"github.com/VictorVargas/rony-llm-agent/pkg/persona"
"github.com/VictorVargas/rony-chat-bot/internal/config"
)
// FromConfig returns a minimal persona used only for the UI greeting and
// the SSE event metadata. The system prompt itself comes from
// config.SystemPrompt, not from this struct — see BuildSystemPrompt.
func FromConfig(c *config.Config) (persona.Persona, error) {
lang := c.Persona.Language
if lang == "" {
lang = "the user's language"
}
return persona.Persona{
ID: "rony",
Name: c.Persona.Name,
Tone: c.Persona.Tone,
Language: lang,
}, nil
}
// BuildSystemPrompt returns the full prompt for one chat turn:
// 1. The hand-written system prompt from the YAML (who Rony is, how to speak)
// 2. The RAG block (omitted when the index returns no hits)
//
// The RAG block is appended, not prepended, so the persona instructions
// always come first and the LLM never gets the chance to "forget" them.
func BuildSystemPrompt(systemPrompt, ragContext string) string {
out := systemPrompt
if ragContext != "" {
out += "\n\n## Relevant context from the portfolio\n\n" +
"Use these excerpts to answer. Cite the project filename when you reference a detail. " +
"If the excerpts don't contain the answer, say you don't have that information — do not invent.\n\n" +
ragContext
}
return out
}

View file

@ -0,0 +1,173 @@
package portfolio
import (
"regexp"
"strings"
)
// Chunk is one indexable unit of a project document.
type Chunk struct {
ID string
ProjectID string
SourceFile string
Index int
Section string // frontmatter | H1 title | H2 section name
Content string
}
// ChunkerConfig controls the heading-based chunker.
type ChunkerConfig struct {
MaxSectionChars int // sections longer than this are sub-split (default 1000)
MergeUnderChars int // sections shorter than this are merged with the next (default 50)
OverlapChars int // overlap when sub-splitting (default 80)
}
func DefaultChunkerConfig() ChunkerConfig {
return ChunkerConfig{
MaxSectionChars: 1000,
MergeUnderChars: 50,
OverlapChars: 80,
}
}
// SplitMarkdownSections produces chunks using heading boundaries:
// 1. Frontmatter (between leading --- ... ---) → "frontmatter" chunk
// 2. Each H1 (project title) → one chunk
// 3. Each H2 section → one chunk (with its sub-content under H3/H4 etc.)
// 4. Sections > MaxSectionChars are sub-split by size with overlap
// 5. Sections < MergeUnderChars are merged with the next section
func SplitMarkdownSections(markdown string, cfg ChunkerConfig) []section {
if cfg.MaxSectionChars <= 0 {
cfg = DefaultChunkerConfig()
}
if cfg.MergeUnderChars < 0 {
cfg.MergeUnderChars = 0
}
body := markdown
frontmatter := ""
if fm, rest, ok := extractFrontmatter(markdown); ok {
frontmatter = fm
body = rest
}
var sections []section
if frontmatter != "" {
sections = append(sections, section{Heading: "frontmatter", Body: frontmatter})
}
for _, s := range splitByHeadings(body) {
sections = append(sections, s)
}
sections = dropEmpty(sections)
sections = subSplit(sections, cfg.MaxSectionChars, cfg.OverlapChars)
return sections
}
type section struct {
Heading string
Body string
}
var (
frontmatterRe = regexp.MustCompile(`^---\n([\s\S]*?)\n---\n?`)
h1Re = regexp.MustCompile(`(?m)^# [^#].*$`)
)
func extractFrontmatter(s string) (string, string, bool) {
m := frontmatterRe.FindStringSubmatchIndex(s)
if m == nil {
return "", s, false
}
return s[m[2]:m[3]], s[m[1]:], true
}
// splitByHeadings splits the body into sections keyed by H1 or H2.
// H3+ content stays attached to its parent H2 (no extra split), which keeps
// the natural unit coherent: the bot answers about "Description" or "Tech
// stack", not about individual bullet points.
func splitByHeadings(body string) []section {
lines := strings.Split(body, "\n")
var sections []section
var current section
inSection := false
for _, line := range lines {
if isH1(line) || isH2(line) {
if inSection {
sections = append(sections, current)
}
current = section{Heading: strings.TrimSpace(strings.TrimLeft(strings.TrimSpace(line), "# "))}
inSection = true
continue
}
if inSection {
current.Body += line + "\n"
}
}
if inSection {
sections = append(sections, current)
}
// Strip trailing whitespace per section
for i := range sections {
sections[i].Body = strings.TrimRight(sections[i].Body, "\n ")
}
return sections
}
func isH1(l string) bool { return strings.HasPrefix(l, "# ") && !strings.HasPrefix(l, "## ") }
func isH2(l string) bool { return strings.HasPrefix(l, "## ") }
// dropEmpty removes sections whose body is empty or just whitespace.
// Keeps the heading-level chunking honest: a "Description" with no body
// is not a useful retrieval target.
func dropEmpty(sections []section) []section {
out := sections[:0]
for _, s := range sections {
if strings.TrimSpace(s.Body) == "" {
continue
}
out = append(out, s)
}
return out
}
// subSplit breaks down any section whose body exceeds max into overlapping
// slices, preserving the heading as a prefix on each piece so context isn't
// lost mid-section.
func subSplit(sections []section, max, overlap int) []section {
if max <= 0 {
return sections
}
if overlap < 0 || overlap >= max {
overlap = max / 10
}
var out []section
for _, s := range sections {
if len(s.Body) <= max {
out = append(out, s)
continue
}
for i := 0; i < len(s.Body); {
end := i + max
if end > len(s.Body) {
end = len(s.Body)
}
piece := s.Body[i:end]
if i > 0 {
piece = "... " + piece
}
if end < len(s.Body) {
piece = piece + " ..."
}
out = append(out, section{Heading: s.Heading, Body: piece})
if end == len(s.Body) {
break
}
i += max - overlap
}
}
return out
}

View file

@ -0,0 +1,111 @@
package portfolio
import (
"strings"
"testing"
)
func TestSplitMarkdownSections(t *testing.T) {
cases := []struct {
name string
input string
wantCount int
wantHeading []string
}{
{
name: "frontmatter only",
input: `---
title: "Foo"
tags: ["go"]
---
# Foo
Body of foo.`,
wantCount: 2,
wantHeading: []string{"frontmatter", "Foo"},
},
{
name: "no frontmatter, multiple H2s",
input: `# Title
intro paragraph
## Description
description body
## Tech stack
- Go
- SQLite`,
wantCount: 3,
wantHeading: []string{"Title", "Description", "Tech stack"},
},
{
name: "H3 stays under parent H2",
input: `# T
## Section
content
### H3 detail
H3 content stays here`,
wantCount: 1, // T has no body → dropped; H3 content folds into Section
wantHeading: []string{"Section"},
},
{
name: "drop empty section",
input: `## X
ok
## Empty
## Y
content`,
wantCount: 2,
wantHeading: []string{"X", "Y"},
},
{
name: "long H2 sub-splits",
input: "## Long\n" + strings.Repeat("a ", 800),
wantCount: 2, // 2 sub-splits of Long
},
}
for _, c := range cases {
t.Run(c.name, func(t *testing.T) {
got := SplitMarkdownSections(c.input, DefaultChunkerConfig())
if len(got) != c.wantCount {
t.Errorf("got %d chunks, want %d. Headings: %v", len(got), c.wantCount, headings(got))
}
if c.wantHeading != nil {
if !equalSlice(headings(got), c.wantHeading) {
t.Errorf("headings = %v, want %v", headings(got), c.wantHeading)
}
}
})
}
}
func headings(sections []section) []string {
out := make([]string, len(sections))
for i, s := range sections {
out[i] = s.Heading
}
return out
}
func equalSlice(a, b []string) bool {
if len(a) != len(b) {
return false
}
for i := range a {
if a[i] != b[i] {
return false
}
}
return true
}

View file

@ -0,0 +1,213 @@
// Package portfolio loads and indexes the user's project markdowns for RAG.
//
// Schema decisions are documented in docs/architecture.md §4.0 (driver) and
// §4 (tokenizer, chunking). The tokenize/driver choices are validated by
// ./bench/; chunking was changed from size-based to heading-based after
// inspecting real data/templates in data/projects/.
package portfolio
import (
"context"
"database/sql"
"fmt"
"log/slog"
"os"
"path/filepath"
"strings"
"time"
_ "modernc.org/sqlite"
)
// SearchResult is one hit from the FTS5 index, with a relevance score.
type SearchResult struct {
ID string
ProjectID string
SourceFile string
Section string
Index int
Content string
Score float64
}
const schema = `
CREATE VIRTUAL TABLE IF NOT EXISTS portfolio_chunks USING fts5(
id UNINDEXED,
project_id UNINDEXED,
source_file UNINDEXED,
section UNINDEXED,
chunk_index UNINDEXED,
content,
tokenize = 'unicode61 remove_diacritics 2'
);
`
// Store wraps a SQLite FTS5 database with the portfolio schema.
type Store struct {
db *sql.DB
chunkSize int // legacy field kept for compat; not used by heading chunker
}
func OpenStore(dbPath string) (*Store, error) {
dir := filepath.Dir(dbPath)
if err := os.MkdirAll(dir, 0o755); err != nil {
return nil, fmt.Errorf("create db dir: %w", err)
}
dsn := dbPath + "?_pragma=journal_mode(WAL)&_pragma=synchronous(NORMAL)"
db, err := sql.Open("sqlite", dsn)
if err != nil {
return nil, fmt.Errorf("open sqlite: %w", err)
}
db.SetMaxOpenConns(1) // SQLite + concurrent writers doesn't help
if _, err := db.ExecContext(context.Background(), schema); err != nil {
_ = db.Close()
return nil, fmt.Errorf("create schema: %w", err)
}
return &Store{db: db, chunkSize: 500}, nil
}
func (s *Store) Close() error { return s.db.Close() }
// DB exposes the underlying *sql.DB for callers that need to run their own
// queries (e.g. the health check). Don't use for hot-path code: go through
// the Search / Reindex methods.
func (s *Store) DB() *sql.DB { return s.db }
func (s *Store) Reindex(ctx context.Context, dataPath string, cfg ChunkerConfig) (files, chunks int, err error) {
matches, err := filepath.Glob(filepath.Join(dataPath, "*.md"))
if err != nil {
return 0, 0, fmt.Errorf("glob: %w", err)
}
tx, err := s.db.BeginTx(ctx, nil)
if err != nil {
return 0, 0, err
}
defer tx.Rollback()
if _, err := tx.ExecContext(ctx, `DELETE FROM portfolio_chunks`); err != nil {
return 0, 0, fmt.Errorf("clear: %w", err)
}
stmt, err := tx.PrepareContext(ctx,
`INSERT INTO portfolio_chunks (id, project_id, source_file, section, chunk_index, content) VALUES (?,?,?,?,?,?)`)
if err != nil {
return 0, 0, err
}
defer stmt.Close()
for _, file := range matches {
body, err := os.ReadFile(file)
if err != nil {
slog.Warn("read file failed", "file", file, "err", err)
continue
}
projectID := strings.TrimSuffix(filepath.Base(file), ".md")
sections := SplitMarkdownSections(string(body), cfg)
for idx, sec := range sections {
id := fmt.Sprintf("%s-%s-%d", projectID, slugify(sec.Heading), idx)
if _, err := stmt.ExecContext(ctx, id, projectID, file, sec.Heading, idx, sec.Body); err != nil {
return len(matches), chunks, fmt.Errorf("insert %s: %w", id, err)
}
chunks++
}
files++
}
if err := tx.Commit(); err != nil {
return 0, 0, err
}
return files, chunks, nil
}
func slugify(s string) string {
out := make([]byte, 0, len(s))
for i := 0; i < len(s); i++ {
c := s[i]
switch {
case c >= 'a' && c <= 'z', c >= '0' && c <= '9':
out = append(out, c)
case c >= 'A' && c <= 'Z':
out = append(out, c+32)
case c == ' ' || c == '-' || c == '_':
out = append(out, '_')
}
}
return string(out)
}
// Search returns up to topK chunks ordered by BM25 score.
func (s *Store) Search(ctx context.Context, query string, topK int) ([]SearchResult, error) {
if topK <= 0 {
topK = 5
}
rows, err := s.db.QueryContext(ctx, fmt.Sprintf(`
SELECT id, project_id, source_file, section, chunk_index, content, bm25(portfolio_chunks) AS score
FROM portfolio_chunks
WHERE portfolio_chunks MATCH '%s'
ORDER BY score
LIMIT %d
`, sanitizeFTS5(query), topK))
if err != nil {
return nil, err
}
defer rows.Close()
var hits []SearchResult
for rows.Next() {
var r SearchResult
if err := rows.Scan(&r.ID, &r.ProjectID, &r.SourceFile, &r.Section, &r.Index, &r.Content, &r.Score); err != nil {
return nil, err
}
hits = append(hits, r)
}
return hits, rows.Err()
}
// sanitizeFTS5 escapes special chars, adds prefix-match wildcards, and joins
// tokens with OR (Q&A behavior: "what database" should match a doc that
// contains "database" even when it doesn't contain "what"). FTS5 doesn't
// accept `?` placeholders for MATCH in driver-prepared statements; this
// inlines the escaped query.
func sanitizeFTS5(q string) string {
tokens := strings.FieldsFunc(strings.ToLower(q), func(r rune) bool {
return !(r == '-' || r == '_' || r == '.' || r == '+' ||
(r >= '0' && r <= '9') ||
(r >= 'a' && r <= 'z') ||
r > 0x7F)
})
if len(tokens) == 0 {
return `""`
}
// Drop a tiny stopword list so "what is" doesn't dominate the OR
// (these match every doc and dilute BM25 ranking).
keep := tokens[:0]
for _, t := range tokens {
switch t {
case "what", "which", "who", "how", "when", "where", "is", "are",
"do", "does", "can", "tell", "about", "the", "a", "an":
continue
}
keep = append(keep, t)
}
if len(keep) == 0 {
// All tokens were stopwords — fall back to the original set.
keep = tokens
}
for i, t := range keep {
keep[i] = `"` + t + `"*`
}
return strings.Join(keep, " OR ")
}
// ReindexOnDisk is a small convenience that opens the store, reindexes, and
// closes — used by the CLI subcommand.
func ReindexOnDisk(dbPath, dataPath string, cfg ChunkerConfig) (time.Duration, int, int, error) {
start := time.Now()
store, err := OpenStore(dbPath)
if err != nil {
return 0, 0, 0, err
}
defer store.Close()
files, chunks, err := store.Reindex(context.Background(), dataPath, cfg)
return time.Since(start), files, chunks, err
}

View file

@ -0,0 +1,116 @@
package portfolio
import (
"context"
"os"
"path/filepath"
"strings"
"testing"
)
func TestReindexAndSearch(t *testing.T) {
// Use a temp dir so we don't pollute data/projects
dir := t.TempDir()
srcDir := filepath.Join(dir, "src")
if err := os.MkdirAll(srcDir, 0o755); err != nil {
t.Fatal(err)
}
// Copy the real example-project.md
data, err := os.ReadFile("../../data/projects/example-project.md")
if err != nil {
t.Skip("example-project.md not found, skipping integration test:", err)
}
if err := os.WriteFile(filepath.Join(srcDir, "example-project.md"), data, 0o644); err != nil {
t.Fatal(err)
}
dbPath := filepath.Join(dir, "test.db")
store, err := OpenStore(dbPath)
if err != nil {
t.Fatal(err)
}
defer store.Close()
files, chunks, err := store.Reindex(context.Background(), srcDir, DefaultChunkerConfig())
if err != nil {
t.Fatal(err)
}
if files != 1 {
t.Errorf("files = %d, want 1", files)
}
if chunks < 4 {
t.Errorf("chunks = %d, want >= 4 (frontmatter + 4 H2 sections)", chunks)
}
// Search for something specific to the file
hits, err := store.Search(context.Background(), "Go SQLite framework", 5)
if err != nil {
t.Fatal(err)
}
if len(hits) == 0 {
t.Error("expected at least one hit for 'Go SQLite framework'")
}
t.Logf("Search hits: %d", len(hits))
for i, h := range hits {
t.Logf(" [%d] project=%s section=%s score=%.3f body=%q", i, h.ProjectID, h.Section, h.Score, truncate(h.Content, 80))
}
// No results for nonsense
hits, err = store.Search(context.Background(), "asdfqwerty12345", 5)
if err != nil {
t.Fatal(err)
}
if len(hits) != 0 {
t.Errorf("expected 0 hits for nonsense query, got %d", len(hits))
}
}
func TestReindexFromRealData(t *testing.T) {
// Index the real data/projects/ to sanity-check on real data
srcDir := "../../data/projects"
if _, err := os.Stat(srcDir); err != nil {
t.Skip("data/projects not found:", err)
}
dir := t.TempDir()
dbPath := filepath.Join(dir, "real.db")
store, err := OpenStore(dbPath)
if err != nil {
t.Fatal(err)
}
defer store.Close()
files, chunks, err := store.Reindex(context.Background(), srcDir, DefaultChunkerConfig())
if err != nil {
t.Fatal(err)
}
t.Logf("Indexed %d files, %d chunks", files, chunks)
if files == 0 {
t.Fatal("no files indexed")
}
// Print what we got
sections := map[string]int{}
_, _ = store.Search(context.Background(), "rony", 100) // noop
// Walk DB to see sections — easier: just re-read with a fresh load
_ = sections
// Verify queries work
for _, q := range []string{"rony-llm-agent", "Qwen", "agent harness", "tech stack", "deploy"} {
hits, err := store.Search(context.Background(), q, 3)
if err != nil {
t.Fatal(err)
}
t.Logf("query %q → %d hits", q, len(hits))
for _, h := range hits {
if !strings.Contains(strings.ToLower(h.Content), strings.ToLower(strings.SplitN(q, "-", 2)[0])) {
// FTS5 might do prefix matches; this is a soft assertion
}
}
}
}
func truncate(s string, n int) string {
s = strings.ReplaceAll(s, "\n", " ")
if len(s) <= n {
return s
}
return s[:n] + "..."
}

273
internal/server/handlers.go Normal file
View file

@ -0,0 +1,273 @@
package server
import (
"crypto/rand"
"encoding/hex"
"encoding/json"
"errors"
"fmt"
"log/slog"
"net/http"
"strings"
"time"
"github.com/VictorVargas/rony-chat-bot/internal/agent"
"github.com/VictorVargas/rony-chat-bot/internal/config"
"github.com/VictorVargas/rony-chat-bot/internal/portfolio"
"github.com/VictorVargas/rony-chat-bot/internal/streaming"
)
type Handlers struct {
cfg *config.Config
runner *agent.Runner
store *portfolio.Store
version string
}
func NewHandlers(cfg *config.Config, runner *agent.Runner, store *portfolio.Store, version string) *Handlers {
return &Handlers{cfg: cfg, runner: runner, store: store, version: version}
}
type ChatRequest struct {
Messages []ChatMessage `json:"messages"`
Stream *bool `json:"stream,omitempty"`
}
type ChatMessage struct {
Role string `json:"role"`
Content string `json:"content"`
}
type ChatResponse struct {
Content string `json:"content"`
Sources []string `json:"sources,omitempty"`
Usage streaming.Usage `json:"usage"`
}
func (h *Handlers) Chat(w http.ResponseWriter, r *http.Request) {
if r.Method != http.MethodPost {
http.Error(w, "method not allowed", http.StatusMethodNotAllowed)
return
}
var req ChatRequest
if err := json.NewDecoder(http.MaxBytesReader(w, r.Body, 1<<20)).Decode(&req); err != nil {
http.Error(w, "invalid JSON body: "+err.Error(), http.StatusBadRequest)
return
}
if err := req.validate(); err != nil {
http.Error(w, err.Error(), http.StatusBadRequest)
return
}
stream := true
if req.Stream != nil {
stream = *req.Stream
}
history := toAgentMessages(req.Messages)
if stream {
h.streamChat(w, r, history)
return
}
h.completeChat(w, r, history)
}
func toAgentMessages(in []ChatMessage) []agent.Message {
out := make([]agent.Message, len(in))
for i, m := range in {
out[i] = agent.Message{
Role: agent.Role(m.Role),
Content: m.Content,
}
}
return out
}
func (req *ChatRequest) validate() error {
if len(req.Messages) == 0 {
return errors.New("messages must not be empty")
}
for i, m := range req.Messages {
switch m.Role {
case "user", "assistant", "system":
default:
return fmt.Errorf("messages[%d].role %q is invalid", i, m.Role)
}
if strings.TrimSpace(m.Content) == "" {
return fmt.Errorf("messages[%d].content is empty", i)
}
}
return nil
}
// streamChat drives the LLM agent and forwards each chunk to the SSE
// stream using the protocol described in docs/architecture.md §3.2.
func (h *Handlers) streamChat(w http.ResponseWriter, r *http.Request, history []agent.Message) {
w.Header().Set("Content-Type", "text/event-stream")
w.Header().Set("Cache-Control", "no-cache")
w.Header().Set("Connection", "keep-alive")
w.Header().Set("X-Accel-Buffering", "no")
w.WriteHeader(http.StatusOK)
flusher, ok := w.(http.Flusher)
if !ok {
http.Error(w, "streaming unsupported", http.StatusInternalServerError)
return
}
ctx := r.Context()
convID := newConvID()
if err := streaming.WriteStart(w, convID); err != nil {
slog.Error("sse start failed", "err", err)
return
}
flusher.Flush()
// Report which RAG sources were used (search happens in BuildMessages).
_, ragContext, err := h.runner.BuildMessages(ctx, history)
if err != nil {
_ = streaming.WriteError(w, "rag: "+err.Error())
return
}
if ragContext != "" {
sources := extractSources(ragContext)
_ = streaming.WriteSources(w, sources)
}
// Stream from the model.
for chunk, err := range h.runner.Stream(ctx, history) {
if err != nil {
slog.Error("llm stream", "err", err)
_ = streaming.WriteError(w, "llm: "+err.Error())
return
}
if chunk.Delta != "" {
if err := streaming.WriteChunk(w, chunk.Delta); err != nil {
return
}
}
if chunk.FinishReason != "" && chunk.Usage.TotalTokens > 0 {
_ = streaming.WriteDone(w, streaming.Usage{
InputTokens: chunk.Usage.InputTokens,
OutputTokens: chunk.Usage.OutputTokens,
})
return
}
}
// Final usage chunk may come on the last iteration; if we never saw a
// finish_reason + usage in-band, surface what we recorded.
usage := h.runner.LastUsage()
_ = streaming.WriteDone(w, streaming.Usage{
InputTokens: usage.InputTokens,
OutputTokens: usage.OutputTokens,
})
}
func (h *Handlers) completeChat(w http.ResponseWriter, r *http.Request, history []agent.Message) {
w.Header().Set("Content-Type", "application/json")
var full strings.Builder
for chunk, err := range h.runner.Stream(r.Context(), history) {
if err != nil {
http.Error(w, "llm: "+err.Error(), http.StatusBadGateway)
return
}
full.WriteString(chunk.Delta)
}
usage := h.runner.LastUsage()
_, ragContext, _ := h.runner.BuildMessages(r.Context(), history)
resp := ChatResponse{
Content: full.String(),
Usage: streaming.Usage{
InputTokens: usage.InputTokens,
OutputTokens: usage.OutputTokens,
},
}
if ragContext != "" {
resp.Sources = extractSources(ragContext)
}
_ = json.NewEncoder(w).Encode(resp)
}
func extractSources(ragContext string) []string {
// The formatted hits look like "### [N] projectID — section\n..."
// so we scan the lines for that prefix and dedupe by projectID.
seen := map[string]bool{}
var out []string
for _, line := range strings.Split(ragContext, "\n") {
if !strings.HasPrefix(line, "### [") {
continue
}
// between "### [N] " and " — "
rest := strings.TrimPrefix(line, "### [")
rest = rest[strings.Index(rest, " ")+1:]
project := rest
if i := strings.Index(rest, " — "); i > 0 {
project = rest[:i]
}
if !seen[project] {
seen[project] = true
out = append(out, project)
}
}
return out
}
func truncate(s string, n int) string {
if len(s) <= n {
return s
}
return s[:n] + "…"
}
func newConvID() string {
var b [8]byte
_, _ = rand.Read(b[:])
return hex.EncodeToString(b[:])
}
func (h *Handlers) Info(w http.ResponseWriter, _ *http.Request) {
w.Header().Set("Content-Type", "application/json")
p := h.cfg.DefaultProvider()
_ = json.NewEncoder(w).Encode(map[string]any{
"name": h.cfg.Persona.Name,
"version": h.version,
"provider": p.Type,
"model": firstNonEmpty(p.Model, p.ModelPath),
"rag": h.cfg.RAG.Enabled,
"top_k": h.cfg.RAG.TopK,
"started": time.Now().UTC().Format(time.RFC3339),
})
}
func (h *Handlers) Reindex(w http.ResponseWriter, r *http.Request) {
if r.Method != http.MethodPost {
http.Error(w, "method not allowed", http.StatusMethodNotAllowed)
return
}
start := time.Now()
dbPath := h.cfg.RAG.DBPath
if dbPath == "" {
http.Error(w, "rag.db_path not configured", http.StatusBadRequest)
return
}
dur, files, chunks, err := portfolio.ReindexOnDisk(dbPath, h.cfg.RAG.DataPath, portfolio.DefaultChunkerConfig())
if err != nil {
http.Error(w, "reindex failed: "+err.Error(), http.StatusInternalServerError)
return
}
_ = dur
w.Header().Set("Content-Type", "application/json")
_ = json.NewEncoder(w).Encode(map[string]any{
"indexed_files": files,
"total_chunks": chunks,
"duration_ms": time.Since(start).Milliseconds(),
"db_path": dbPath,
})
}
func firstNonEmpty(vals ...string) string {
for _, v := range vals {
if v != "" {
return v
}
}
return ""
}

188
internal/server/health.go Normal file
View file

@ -0,0 +1,188 @@
package server
import (
"context"
"encoding/json"
"fmt"
"net/http"
"strings"
"sync"
"time"
)
// HealthResponse is the shape returned by /api/health.
//
// Status is "healthy" when every component is up, "degraded" when non-critical
// components are down, "unhealthy" when the bot cannot serve. The HTTP code
// is 200 for healthy, 503 for unhealthy. Degraded returns 200 (the bot can
// still answer, just without RAG or with no LLM).
type HealthResponse struct {
Status string `json:"status"`
Version string `json:"version"`
CheckedAt string `json:"checked_at"`
Components map[string]ComponentHealth `json:"components"`
}
// ComponentHealth reports the state of one dependency. Latency is a
// human-readable duration ("12ms"); Error is set only when Status="down".
type ComponentHealth struct {
Status string `json:"status"` // "up" | "down"
Latency string `json:"latency,omitempty"` // e.g. "12ms"
Error string `json:"error,omitempty"`
Details map[string]any `json:"details,omitempty"`
}
// healthCheckTimeout caps each component probe. The whole /api/health call
// finishes in roughly this duration even if a dependency is hung.
const healthCheckTimeout = 2 * time.Second
func (h *Handlers) Health(w http.ResponseWriter, r *http.Request) {
// `?deep=true` adds an FTS5 row count to the store probe; same latency
// budget. Cheap enough that we don't bother splitting into two endpoints.
deep := r.URL.Query().Get("deep") == "true"
// Pad the context so even a slow store doesn't kill the request before
// the LLM probe finishes.
ctx, cancel := context.WithTimeout(r.Context(), healthCheckTimeout+500*time.Millisecond)
defer cancel()
// Run both probes in parallel so a slow LLM doesn't delay the store
// check (and vice versa).
type probe struct {
name string
fn func(context.Context) ComponentHealth
}
results := make(map[string]ComponentHealth, 2)
var mu sync.Mutex
var wg sync.WaitGroup
for _, p := range []probe{
{"llm", h.checkLLM},
{"store", func(ctx context.Context) ComponentHealth { return h.checkStore(ctx, deep) }},
} {
wg.Add(1)
p := p
go func() {
defer wg.Done()
r := p.fn(ctx)
mu.Lock()
results[p.name] = r
mu.Unlock()
}()
}
wg.Wait()
resp := HealthResponse{
Status: "healthy",
Version: h.version,
CheckedAt: time.Now().UTC().Format(time.RFC3339),
Components: results,
}
// Decide overall status. LLM is critical; store is non-critical
// (RAG degrades to "I don't have that information" when it can't be read).
httpStatus := http.StatusOK
if results["llm"].Status != "up" {
resp.Status = "unhealthy"
httpStatus = http.StatusServiceUnavailable
} else if results["store"].Status != "up" {
resp.Status = "degraded"
}
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(httpStatus)
_ = json.NewEncoder(w).Encode(resp)
}
// checkLLM pings the provider's health endpoint with a short timeout.
func (h *Handlers) checkLLM(parent context.Context) ComponentHealth {
p := h.cfg.DefaultProvider()
healthURL := deriveHealthURL(p.Endpoint, p.Type)
if healthURL == "" {
return ComponentHealth{
Status: "down",
Error: "no health endpoint known for provider type " + p.Type,
Details: map[string]any{"provider": p.Type, "model": p.Model},
}
}
ctx, cancel := context.WithTimeout(parent, healthCheckTimeout)
defer cancel()
start := time.Now()
req, err := http.NewRequestWithContext(ctx, http.MethodGet, healthURL, nil)
if err != nil {
return ComponentHealth{Status: "down", Error: err.Error(), Latency: time.Since(start).String()}
}
resp, err := http.DefaultClient.Do(req)
latency := time.Since(start)
if err != nil {
return ComponentHealth{Status: "down", Error: err.Error(), Latency: latency.String()}
}
defer resp.Body.Close()
if resp.StatusCode >= 200 && resp.StatusCode < 300 {
return ComponentHealth{
Status: "up",
Latency: latency.String(),
Details: map[string]any{"provider": p.Type, "model": p.Model, "url": healthURL},
}
}
return ComponentHealth{
Status: "down",
Latency: latency.String(),
Error: fmt.Sprintf("HTTP %d", resp.StatusCode),
Details: map[string]any{"url": healthURL},
}
}
// deriveHealthURL maps a provider's chat-completions URL to its health URL.
// Returns "" when the provider has no usable health probe.
func deriveHealthURL(endpoint, providerType string) string {
if endpoint == "" {
return ""
}
// Strip trailing /v1, /chat/completions, etc., to get the base.
base := endpoint
for _, suffix := range []string{"/v1/chat/completions", "/chat/completions", "/v1"} {
if strings.HasSuffix(base, suffix) {
base = strings.TrimSuffix(base, suffix)
break
}
}
switch providerType {
case "llamacpp", "ollama":
// Both expose /health at the root.
return base + "/health"
case "openai":
// OpenAI doesn't have /health, but /models is auth-light and stable.
return base + "/models"
default:
// anthropic and unknown: no cheap probe without a real request.
return ""
}
}
// checkStore pings the SQLite store with a trivial query. In deep mode it
// also counts indexed chunks to confirm the FTS5 schema is alive.
func (h *Handlers) checkStore(parent context.Context, deep bool) ComponentHealth {
if h.store == nil {
return ComponentHealth{Status: "down", Error: "store not configured"}
}
ctx, cancel := context.WithTimeout(parent, healthCheckTimeout)
defer cancel()
start := time.Now()
var n int
err := h.store.DB().QueryRowContext(ctx, "SELECT 1").Scan(&n)
latency := time.Since(start)
if err != nil {
return ComponentHealth{Status: "down", Error: err.Error(), Latency: latency.String()}
}
details := map[string]any{}
if deep {
_ = h.store.DB().QueryRowContext(ctx,
"SELECT count(*) FROM portfolio_chunks").Scan(&n)
details["chunks"] = n
}
return ComponentHealth{Status: "up", Latency: latency.String(), Details: details}
}

View file

@ -0,0 +1,176 @@
package server
import (
"context"
"log/slog"
"net"
"net/http"
"strconv"
"strings"
"sync"
"time"
"github.com/VictorVargas/rony-chat-bot/internal/config"
)
type ctxKey string
const ctxKeyRequestID ctxKey = "requestID"
// RequestID assigns a short id per request and exposes it via context + header.
func RequestID(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
id := strconv.FormatInt(time.Now().UnixNano(), 36)
ctx := context.WithValue(r.Context(), ctxKeyRequestID, id)
w.Header().Set("X-Request-ID", id)
next.ServeHTTP(w, r.WithContext(ctx))
})
}
// Logging emits one structured log line per request after it completes.
func Logging(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
start := time.Now()
ww := &statusRecorder{ResponseWriter: w, status: http.StatusOK}
next.ServeHTTP(ww, r)
id, _ := r.Context().Value(ctxKeyRequestID).(string)
slog.Info("http",
"id", id,
"method", r.Method,
"path", r.URL.Path,
"status", ww.status,
"bytes", ww.bytes,
"duration_ms", time.Since(start).Milliseconds(),
"remote", clientIP(r),
)
})
}
type statusRecorder struct {
http.ResponseWriter
status int
bytes int
}
func (r *statusRecorder) WriteHeader(code int) {
r.status = code
r.ResponseWriter.WriteHeader(code)
}
func (r *statusRecorder) Write(b []byte) (int, error) {
n, err := r.ResponseWriter.Write(b)
r.bytes += n
return n, err
}
func (r *statusRecorder) Flush() {
if f, ok := r.ResponseWriter.(http.Flusher); ok {
f.Flush()
}
}
// CORS rejects requests whose Origin isn't on the allowlist. Requests with
// no Origin header (curl, server-to-server) are allowed through.
func CORS(allowed []string) func(http.Handler) http.Handler {
set := make(map[string]struct{}, len(allowed))
allowAll := false
for _, o := range allowed {
o = strings.TrimSpace(o)
if o == "*" {
allowAll = true
continue
}
set[o] = struct{}{}
}
return func(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
origin := r.Header.Get("Origin")
w.Header().Set("Access-Control-Allow-Methods", "GET, POST, OPTIONS")
w.Header().Set("Access-Control-Allow-Headers", "Content-Type, Authorization")
if origin != "" {
if !allowAll {
if _, ok := set[origin]; !ok {
http.Error(w, "origin not allowed", http.StatusForbidden)
return
}
}
w.Header().Set("Access-Control-Allow-Origin", origin)
w.Header().Set("Vary", "Origin")
w.Header().Set("Access-Control-Allow-Credentials", "true")
}
if r.Method == http.MethodOptions {
w.WriteHeader(http.StatusNoContent)
return
}
next.ServeHTTP(w, r)
})
}
}
// RateLimit is a per-IP token bucket. cfg.RateLimit.RequestsPerMinute sets
// the refill rate; cfg.RateLimit.Burst is the bucket size.
func RateLimit(cfg config.RateLimit) func(http.Handler) http.Handler {
type bucket struct {
tokens float64
lastFill time.Time
}
var mu sync.Mutex
buckets := make(map[string]*bucket)
rate := float64(cfg.RequestsPerMinute) / 60.0
burst := float64(cfg.Burst)
if rate <= 0 {
rate = 0.5
}
if burst <= 0 {
burst = 5
}
return func(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
ip := clientIP(r)
now := time.Now()
mu.Lock()
b, ok := buckets[ip]
if !ok {
b = &bucket{tokens: burst, lastFill: now}
buckets[ip] = b
}
elapsed := now.Sub(b.lastFill).Seconds()
b.tokens += elapsed * rate
if b.tokens > burst {
b.tokens = burst
}
b.lastFill = now
if b.tokens < 1 {
mu.Unlock()
w.Header().Set("Retry-After", "60")
http.Error(w, "rate limit exceeded", http.StatusTooManyRequests)
return
}
b.tokens--
mu.Unlock()
next.ServeHTTP(w, r)
})
}
}
func clientIP(r *http.Request) string {
if xff := r.Header.Get("X-Forwarded-For"); xff != "" {
if i := strings.IndexByte(xff, ','); i > 0 {
return strings.TrimSpace(xff[:i])
}
return strings.TrimSpace(xff)
}
host, _, err := net.SplitHostPort(r.RemoteAddr)
if err != nil {
return r.RemoteAddr
}
return host
}

62
internal/server/server.go Normal file
View file

@ -0,0 +1,62 @@
package server
import (
"context"
"errors"
"log/slog"
"net/http"
"time"
"github.com/go-chi/chi/v5"
"github.com/VictorVargas/rony-chat-bot/internal/config"
)
type Server struct {
httpSrv *http.Server
}
func New(cfg *config.Config, h *Handlers) *Server {
r := chi.NewRouter()
r.Use(RequestID)
r.Use(Logging)
r.Use(CORS(cfg.Server.CORSOrigins))
if cfg.Server.RateLimit.RequestsPerMinute > 0 {
r.Use(RateLimit(cfg.Server.RateLimit))
}
r.Get("/", func(w http.ResponseWriter, _ *http.Request) {
w.Header().Set("Content-Type", "text/plain; charset=utf-8")
_, _ = w.Write([]byte("Rony Chat Bot — see /api/health, /api/info, POST /api/chat\n"))
})
r.Route("/api", func(r chi.Router) {
r.Post("/chat", h.Chat)
r.Post("/reindex", h.Reindex)
r.Get("/health", h.Health)
r.Get("/info", h.Info)
})
srv := &http.Server{
Addr: cfg.Addr(),
Handler: r,
ReadTimeout: time.Duration(cfg.Server.ReadTimeoutMS) * time.Millisecond,
WriteTimeout: 0, // SSE streams must not be cut off by WriteTimeout
IdleTimeout: 120 * time.Second,
}
return &Server{httpSrv: srv}
}
func (s *Server) Start() error {
slog.Info("http server starting", "addr", s.httpSrv.Addr)
if err := s.httpSrv.ListenAndServe(); err != nil && !errors.Is(err, http.ErrServerClosed) {
return err
}
return nil
}
func (s *Server) Shutdown(ctx context.Context) error {
return s.httpSrv.Shutdown(ctx)
}

View file

@ -0,0 +1,420 @@
package server
import (
"bufio"
"encoding/json"
"io"
"net/http"
"net/http/httptest"
"path/filepath"
"strings"
"testing"
"github.com/VictorVargas/rony-llm-agent/pkg/llm"
"github.com/VictorVargas/rony-llm-agent/pkg/llm/mock"
llmpersona "github.com/VictorVargas/rony-llm-agent/pkg/persona"
"github.com/VictorVargas/rony-chat-bot/internal/agent"
"github.com/VictorVargas/rony-chat-bot/internal/config"
"github.com/VictorVargas/rony-chat-bot/internal/portfolio"
)
// newTestServer wires a Server backed by a mock LLM, listens on a random
// port, and returns the base URL plus a teardown. The persona is fixed and
// the RAG store is nil (these tests don't exercise retrieval).
//
// To make /api/health pass, the LLM endpoint is pointed at a tiny stub
// HTTP server that returns 200 on /health. Callers can override this with
// newTestServerWithLLM(t, llmStatus).
func newTestServer(t *testing.T) (string, *config.Config) {
t.Helper()
return newTestServerWithLLM(t, http.StatusOK)
}
func newTestServerWithLLM(t *testing.T, llmStatus int) (string, *config.Config) {
t.Helper()
cli := mock.NewWithStream(mockChunks("hello", "world"))
p := llmpersona.Persona{Name: "TestBot", Tone: "concise", Language: "English"}
// Stub HTTP server that mimics llama-server's /health. If llmStatus != 200
// the health probe will report down.
llmStub := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path == "/health" {
w.WriteHeader(llmStatus)
return
}
w.WriteHeader(http.StatusNotFound)
}))
t.Cleanup(llmStub.Close)
// Real (in-memory) FTS5 store so the store probe reports up. Without
// it the bot is "degraded" (200, but store down) — a confusing default.
dbPath := filepath.Join(t.TempDir(), "t.db")
store, err := portfolio.OpenStore(dbPath)
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { _ = store.Close() })
runner := agent.New(cli, p, "test system prompt", store, 5)
cfg := &config.Config{
Server: config.Server{
Host: "127.0.0.1",
Port: 0,
CORSOrigins: []string{"http://localhost:4321"},
RateLimit: config.RateLimit{RequestsPerMinute: 0, Burst: 0}, // disabled
},
Providers: []config.Provider{{
Name: "mock", Type: "llamacpp", Model: "test", Default: true,
Endpoint: llmStub.URL + "/v1",
}},
RAG: config.RAG{Enabled: false, TopK: 5, DataPath: ".", DBPath: dbPath},
Persona: config.Persona{Name: "TestBot", Language: "English"},
}
h := NewHandlers(cfg, runner, store, "test")
srv := New(cfg, h)
ts := httptest.NewUnstartedServer(srv.httpSrv.Handler)
ts.Start()
t.Cleanup(ts.Close)
return ts.URL, cfg
}
// mockChunks converts plain strings into a stream of single-word deltas
// followed by a final usage chunk, matching what a real provider would emit.
func mockChunks(words ...string) []llm.StreamChunk {
out := make([]llm.StreamChunk, 0, len(words)+1)
for _, w := range words {
out = append(out, llm.StreamChunk{Delta: w + " "})
}
out = append(out, llm.StreamChunk{
Delta: "",
FinishReason: "stop",
Usage: llm.TokenUsage{InputTokens: 5, OutputTokens: 3},
})
return out
}
func TestHealth(t *testing.T) {
url, _ := newTestServer(t)
resp, err := http.Get(url + "/api/health")
if err != nil {
t.Fatal(err)
}
defer resp.Body.Close()
if resp.StatusCode != 200 {
t.Errorf("status = %d, want 200", resp.StatusCode)
}
var body HealthResponse
if err := json.NewDecoder(resp.Body).Decode(&body); err != nil {
t.Fatal(err)
}
if body.Status != "healthy" {
t.Errorf("status = %q, want healthy", body.Status)
}
if body.Components["llm"].Status != "up" {
t.Errorf("llm component = %q, want up", body.Components["llm"].Status)
}
if body.Version == "" {
t.Error("version should be set")
}
}
func TestHealthLLMDown(t *testing.T) {
// Stub LLM that returns 500 on /health → bot should be "unhealthy" (503).
url, _ := newTestServerWithLLM(t, http.StatusInternalServerError)
resp, err := http.Get(url + "/api/health")
if err != nil {
t.Fatal(err)
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusServiceUnavailable {
t.Errorf("status = %d, want 503", resp.StatusCode)
}
var body HealthResponse
_ = json.NewDecoder(resp.Body).Decode(&body)
if body.Status != "unhealthy" {
t.Errorf("status = %q, want unhealthy", body.Status)
}
if body.Components["llm"].Status != "down" {
t.Errorf("llm component = %q, want down", body.Components["llm"].Status)
}
}
func TestHealthDeep(t *testing.T) {
url, _ := newTestServer(t)
resp, err := http.Get(url + "/api/health?deep=true")
if err != nil {
t.Fatal(err)
}
defer resp.Body.Close()
if resp.StatusCode != 200 {
t.Errorf("status = %d, want 200", resp.StatusCode)
}
var body HealthResponse
_ = json.NewDecoder(resp.Body).Decode(&body)
if body.Components["store"].Status != "up" {
t.Errorf("store component = %q, want up", body.Components["store"].Status)
}
// Deep mode adds a chunks count to the store details.
if _, ok := body.Components["store"].Details["chunks"]; !ok {
t.Errorf("deep mode should include chunks count in details, got: %v", body.Components["store"].Details)
}
}
func TestInfo(t *testing.T) {
url, _ := newTestServer(t)
resp, err := http.Get(url + "/api/info")
if err != nil {
t.Fatal(err)
}
defer resp.Body.Close()
if resp.StatusCode != 200 {
t.Errorf("status = %d, want 200", resp.StatusCode)
}
var body map[string]any
_ = json.NewDecoder(resp.Body).Decode(&body)
for _, k := range []string{"name", "version", "provider", "model", "rag", "top_k"} {
if _, ok := body[k]; !ok {
t.Errorf("missing field %q in /api/info: %v", k, body)
}
}
}
func TestChatSSE(t *testing.T) {
url, _ := newTestServer(t)
body := strings.NewReader(`{"messages":[{"role":"user","content":"hi"}],"stream":true}`)
resp, err := http.Post(url+"/api/chat", "application/json", body)
if err != nil {
t.Fatal(err)
}
defer resp.Body.Close()
if got := resp.Header.Get("Content-Type"); !strings.HasPrefix(got, "text/event-stream") {
t.Errorf("Content-Type = %q, want text/event-stream", got)
}
if resp.StatusCode != 200 {
t.Errorf("status = %d, want 200", resp.StatusCode)
}
events := parseSSE(t, resp.Body)
// No RAG store in this test → no "sources" event.
wantTypes := []string{"start", "chunk", "chunk", "done"}
if len(events) < len(wantTypes) {
t.Fatalf("got %d events, want >= %d (%v)", len(events), len(wantTypes), events)
}
for i, want := range wantTypes {
if events[i].event != want {
t.Errorf("event[%d] = %q, want %q", i, events[i].event, want)
}
}
if events[1].data == "" {
t.Errorf("first chunk data is empty")
}
}
func TestChatSSEWithRAG(t *testing.T) {
// Build a temp store with one project so the sources event fires.
dir := t.TempDir()
if err := writeFileR(dir+"/src/proj.md", "# Demo\n\n## Tech stack\n- Go\n- SQLite database\n"); err != nil {
t.Fatal(err)
}
store, err := openStoreForTest(dir + "/t.db", dir+"/src")
if err != nil {
t.Fatal(err)
}
defer store.Close()
cli := mock.NewWithStream(mockChunks("answer"))
p := llmpersona.Persona{Name: "TestBot", Tone: "concise", Language: "English"}
runner := agent.New(cli, p, "test system prompt", store, 5)
cfg := &config.Config{
Server: config.Server{Host: "127.0.0.1", Port: 0, CORSOrigins: []string{"*"}},
Providers: []config.Provider{{Name: "mock", Type: "llamacpp", Model: "test", Default: true}},
RAG: config.RAG{Enabled: true, TopK: 5},
Persona: config.Persona{Name: "TestBot", Language: "English"},
}
h := NewHandlers(cfg, runner, store, "test")
srv := New(cfg, h)
ts := httptest.NewUnstartedServer(srv.httpSrv.Handler)
ts.Start()
defer ts.Close()
body := strings.NewReader(`{"messages":[{"role":"user","content":"what database?"}],"stream":true}`)
resp, err := http.Post(ts.URL+"/api/chat", "application/json", body)
if err != nil {
t.Fatal(err)
}
defer resp.Body.Close()
events := parseSSE(t, resp.Body)
var sawSources bool
for _, e := range events {
if e.event == "sources" && strings.Contains(e.data, "proj") {
sawSources = true
}
}
if !sawSources {
t.Errorf("no sources event with project id, got events: %v", events)
}
}
func TestChatNoStream(t *testing.T) {
url, _ := newTestServer(t)
body := strings.NewReader(`{"messages":[{"role":"user","content":"hi"}],"stream":false}`)
resp, err := http.Post(url+"/api/chat", "application/json", body)
if err != nil {
t.Fatal(err)
}
defer resp.Body.Close()
if resp.StatusCode != 200 {
t.Errorf("status = %d, want 200", resp.StatusCode)
}
if got := resp.Header.Get("Content-Type"); !strings.HasPrefix(got, "application/json") {
t.Errorf("Content-Type = %q, want application/json", got)
}
var out struct {
Content string `json:"content"`
Sources []string `json:"sources"`
Usage struct {
InputTokens int `json:"input_tokens"`
OutputTokens int `json:"output_tokens"`
} `json:"usage"`
}
if err := json.NewDecoder(resp.Body).Decode(&out); err != nil {
t.Fatal(err)
}
if out.Content == "" {
t.Error("content is empty")
}
if out.Usage.InputTokens == 0 && out.Usage.OutputTokens == 0 {
t.Error("usage is all zero")
}
}
func TestChatInvalidBody(t *testing.T) {
url, _ := newTestServer(t)
cases := []struct {
name, body string
}{
{"empty messages", `{"messages":[]}`},
{"empty content", `{"messages":[{"role":"user","content":""}]}`},
{"invalid role", `{"messages":[{"role":"wizard","content":"x"}]}`},
{"bad json", `{not json`},
}
for _, c := range cases {
t.Run(c.name, func(t *testing.T) {
resp, err := http.Post(url+"/api/chat", "application/json", strings.NewReader(c.body))
if err != nil {
t.Fatal(err)
}
defer resp.Body.Close()
if resp.StatusCode != 400 {
t.Errorf("status = %d, want 400", resp.StatusCode)
}
})
}
}
func TestChatWrongMethod(t *testing.T) {
url, _ := newTestServer(t)
resp, err := http.Get(url + "/api/chat")
if err != nil {
t.Fatal(err)
}
defer resp.Body.Close()
if resp.StatusCode != 405 {
t.Errorf("status = %d, want 405", resp.StatusCode)
}
}
func TestCORSReject(t *testing.T) {
url, _ := newTestServer(t)
body := strings.NewReader(`{"messages":[{"role":"user","content":"hi"}]}`)
req, _ := http.NewRequest("POST", url+"/api/chat", body)
req.Header.Set("Origin", "https://evil.example")
req.Header.Set("Content-Type", "application/json")
resp, err := http.DefaultClient.Do(req)
if err != nil {
t.Fatal(err)
}
defer resp.Body.Close()
if resp.StatusCode != 403 {
t.Errorf("status = %d, want 403", resp.StatusCode)
}
if aco := resp.Header.Get("Access-Control-Allow-Origin"); aco != "" {
t.Errorf("ACAO = %q, want empty", aco)
}
}
func TestCORSAllow(t *testing.T) {
url, _ := newTestServer(t)
body := strings.NewReader(`{"messages":[{"role":"user","content":"hi"}]}`)
req, _ := http.NewRequest("POST", url+"/api/chat", body)
req.Header.Set("Origin", "http://localhost:4321")
req.Header.Set("Content-Type", "application/json")
resp, err := http.DefaultClient.Do(req)
if err != nil {
t.Fatal(err)
}
defer resp.Body.Close()
if resp.StatusCode != 200 {
t.Errorf("status = %d, want 200", resp.StatusCode)
}
if aco := resp.Header.Get("Access-Control-Allow-Origin"); aco != "http://localhost:4321" {
t.Errorf("ACAO = %q, want http://localhost:4321", aco)
}
}
// sseEvent is one parsed "event: x\ndata: y\n\n" record.
type sseEvent struct {
event string
data string
}
func parseSSE(t *testing.T, r io.Reader) []sseEvent {
t.Helper()
var out []sseEvent
scanner := bufio.NewScanner(r)
scanner.Buffer(make([]byte, 0, 64*1024), 4*1024*1024)
var cur sseEvent
flush := func() {
if cur.event != "" || cur.data != "" {
out = append(out, cur)
}
cur = sseEvent{}
}
for scanner.Scan() {
line := scanner.Text()
switch {
case line == "":
flush()
case strings.HasPrefix(line, "event: "):
cur.event = strings.TrimPrefix(line, "event: ")
case strings.HasPrefix(line, "data: "):
if cur.data != "" {
cur.data += "\n"
}
cur.data += strings.TrimPrefix(line, "data: ")
}
}
flush()
if err := scanner.Err(); err != nil {
t.Fatal(err)
}
return out
}
// Verify the unused imports are not actually unused (httptest, context, etc.).
var _ = httptest.NewRecorder
// writeFileR + openStoreForTest are tiny shims so the test file doesn't
// need to import os/filepath directly.
func writeFileR(path, content string) error {
return osWriteFile(path, []byte(content), 0o644)
}
func openStoreForTest(dbPath, srcDir string) (ragStore, error) {
return ragOpenStore(dbPath, srcDir)
}

View file

@ -0,0 +1,30 @@
package server
import (
"context"
"os"
"path/filepath"
"github.com/VictorVargas/rony-chat-bot/internal/portfolio"
)
type ragStore = *portfolio.Store
func osWriteFile(path string, data []byte, perm os.FileMode) error {
if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil {
return err
}
return os.WriteFile(path, data, perm)
}
func ragOpenStore(dbPath, srcDir string) (ragStore, error) {
store, err := portfolio.OpenStore(dbPath)
if err != nil {
return nil, err
}
if _, _, err := store.Reindex(context.Background(), srcDir, portfolio.DefaultChunkerConfig()); err != nil {
_ = store.Close()
return nil, err
}
return store, nil
}

63
internal/streaming/sse.go Normal file
View file

@ -0,0 +1,63 @@
package streaming
import (
"encoding/json"
"fmt"
"net/http"
)
type Event struct {
Type string `json:"type"`
Content string `json:"content,omitempty"`
}
func WriteEvent(w http.ResponseWriter, eventType string, payload any) error {
data, err := json.Marshal(payload)
if err != nil {
return fmt.Errorf("marshal event: %w", err)
}
if _, err := fmt.Fprintf(w, "event: %s\ndata: %s\n\n", eventType, data); err != nil {
return err
}
if f, ok := w.(http.Flusher); ok {
f.Flush()
}
return nil
}
func WriteChunk(w http.ResponseWriter, content string) error {
return WriteEvent(w, "chunk", Event{Type: "chunk", Content: content})
}
func WriteStart(w http.ResponseWriter, conversationID string) error {
return WriteEvent(w, "start", map[string]any{
"type": "start",
"conversation_id": conversationID,
})
}
func WriteSources(w http.ResponseWriter, sources []string) error {
return WriteEvent(w, "sources", map[string]any{
"type": "sources",
"documents": sources,
})
}
type Usage struct {
InputTokens int `json:"input_tokens"`
OutputTokens int `json:"output_tokens"`
}
func WriteDone(w http.ResponseWriter, usage Usage) error {
return WriteEvent(w, "done", map[string]any{
"type": "done",
"usage": usage,
})
}
func WriteError(w http.ResponseWriter, errMsg string) error {
return WriteEvent(w, "error", map[string]string{
"type": "error",
"error": errMsg,
})
}

164
web/README.md Normal file
View file

@ -0,0 +1,164 @@
# rony-chat-widget
A drop-in vanilla-JS chat widget that talks to the Rony Chat Bot backend over Server-Sent Events. No build step, no runtime dependencies, no global CSS pollution.
## Files
| File | Purpose |
|---|---|
| `chat-widget.js` | The widget. Self-contained ~12 KB. |
| `chat-widget.css` | Scoped styles, themable via CSS custom properties. |
| `example.html` | Standalone demo page (use with `python3 -m http.server`). |
## Quick start (any site)
```html
<link rel="stylesheet" href="/path/to/chat-widget.css">
<script src="/path/to/chat-widget.js"
data-api-url="https://your-chatbot.example.com"
data-title="Ask me anything"
data-greeting="Hi! Ask me about the projects."
data-position="bottom-right"
data-theme="auto"
defer></script>
```
The bubble appears bottom-right (or bottom-left), opens a 380×560 panel, and talks to `data-api-url/api/chat` over SSE.
## Configuration (all via `data-*` attributes on the `<script>` tag)
| Attribute | Default | Notes |
|---|---|---|
| `data-api-url` | *(required)* | Base URL of the chat-bot, e.g. `https://chat.example.com`. No trailing slash. |
| `data-title` | `"Chat"` | Header text. |
| `data-greeting` | `""` | First message shown when the panel opens (no greeting if empty). |
| `data-position` | `"bottom-right"` | `"bottom-right"` or `"bottom-left"`. |
| `data-theme` | `"auto"` | `"auto"` (follows `prefers-color-scheme`), `"light"`, or `"dark"`. |
## Language
The widget UI is bilingual (English / Spanish) with a toggle in the header.
- **Initial language**: `localStorage["rony-chat-lang"]` if set, else detected from `navigator.language` (anything starting with `es` → Spanish, else English).
- **Persisted** across page reloads via `localStorage`.
- **Conversation language is independent**: the bot auto-detects the language of each user message and replies in that language. The toggle only changes the *interface* (placeholder, status, errors, send button).
- **No build step**: strings live in a `STRINGS` object at the top of `chat-widget.js`. Add a new language by adding an entry.
To override the initial language (e.g., force English on a Spanish site):
```html
<script>
localStorage.setItem("rony-chat-lang", "en");
</script>
<script src="chat-widget.js" data-api-url="..." defer></script>
```
## Theming (override without forking)
All visual tokens are CSS custom properties on the root element. Set them in your site's stylesheet:
```css
.rony-chat-widget-root {
--rony-accent: #ff6b35; /* bubble + send button + links */
--rony-radius: 4px; /* tighter corners */
--rony-font: "Inter", sans-serif;
}
```
See the full list in `chat-widget.css` (search for `--rony-`).
## Astro integration
The simplest path is the drop-in. Add this to your `Layout.astro` (or any shared layout):
```astro
---
// src/layouts/ChatLayout.astro
import "../path/to/chat-widget.css";
const apiUrl = import.meta.env.PUBLIC_CHAT_API_URL || "http://localhost:7331";
---
<html>
<head>
<head><slot name="head" /></head>
</head>
<body>
<slot />
<script src="/path/to/chat-widget.js"
data-api-url={apiUrl}
data-title="Ask me anything"
data-position="bottom-right"
data-theme="auto"
defer is:inline></script>
</body>
</html>
```
Notes:
- `is:inline` keeps Astro from hashing/transforming the script tag, so the `data-*` attributes survive.
- `PUBLIC_CHAT_API_URL` is an Astro env var; set it in `.env` per environment.
- The bot's `cors_origins` in YAML must include your Astro dev origin (`http://localhost:4321`).
## React/Next.js
Mount the same script tag in your root layout:
```tsx
// app/layout.tsx
import Script from "next/script";
export default function RootLayout({ children }) {
return (
<html>
<head>
<link rel="stylesheet" href="/chat-widget.css" />
<Script src="/chat-widget.js"
data-api-url={process.env.NEXT_PUBLIC_CHAT_API_URL}
data-title="Ask me anything"
data-position="bottom-right"
data-theme="auto"
strategy="afterInteractive" />
</head>
<body>{children}</body>
</html>
);
}
```
## Backend requirements
The widget expects the bot to:
1. Expose `POST /api/chat` accepting `{ messages, stream }` (see `docs/architecture.md` §3.1).
2. Stream SSE events: `start`, `chunk`, `sources`, `done`, `error` (see `docs/architecture.md` §3.2).
3. Allow the page's origin via `cors_origins` in the bot's config.
## Running the example locally
```bash
# 1. Start the bot
./bin/chat-bot serve
# 2. Serve the widget (in another terminal)
cd web
python3 -m http.server 8000
# 3. Open http://localhost:8000/example.html in a browser
```
> Note: `localhost:8000` must be in the bot's `cors_origins` for the demo to work. The default config already includes it.
## Browser support
Modern browsers (Chrome/Edge 90+, Firefox 90+, Safari 15+). Uses:
- `fetch` + `ReadableStream` (for SSE)
- `AbortController`
- CSS custom properties + `prefers-color-scheme`
No polyfills, no transpilation.
## What's not in the widget (yet)
- **Conversation persistence** — each visit is a fresh conversation. The bot is stateless; add a `conversation_id` cookie + server-side history if you want continuity.
- **Markdown images / tables** — the renderer handles paragraphs, lists, code, links, bold/italic. Tables and images render as raw text. For richer output, swap `renderMarkdown` for `marked` or `markdown-it`.
- **Typing indicators beyond the streaming caret** — the caret at the end of the streaming response is the only indicator. Good enough for short answers.
- **Mobile sheet drag-to-dismiss** — the panel goes full-screen on phones, but can't be swiped away. Add a swipe handler if it matters.

269
web/chat-widget.css Normal file
View file

@ -0,0 +1,269 @@
/* rony-chat-widget scoped styles
All selectors are prefixed with .rony-chat-widget- to avoid clashing
with the host site's CSS. Override CSS custom properties (--rony-*)
to retheme without forking. */
.rony-chat-widget-root {
--rony-z: 2147483000;
--rony-bg: #ffffff;
--rony-bg-soft: #f7f7f8;
--rony-fg: #1a1a1a;
--rony-fg-soft: #5c5c66;
--rony-border: #e4e4e7;
--rony-accent: #2563eb;
--rony-accent-fg: #ffffff;
--rony-user-bubble: #2563eb;
--rony-user-fg: #ffffff;
--rony-assistant-bubble: #f1f1f3;
--rony-shadow: 0 8px 28px rgba(0, 0, 0, 0.12);
--rony-radius: 14px;
--rony-font: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, "Helvetica Neue", Arial, sans-serif;
position: fixed;
z-index: var(--rony-z);
font-family: var(--rony-font);
color: var(--rony-fg);
font-size: 14px;
line-height: 1.5;
}
.rony-chat-widget-root[data-position="bottom-left"] { left: 20px; bottom: 20px; }
.rony-chat-widget-root[data-position="bottom-right"] { right: 20px; bottom: 20px; }
/* Dark mode auto */
@media (prefers-color-scheme: dark) {
.rony-chat-widget-root[data-theme="auto"] {
--rony-bg: #1f1f23;
--rony-bg-soft: #2a2a30;
--rony-fg: #f1f1f3;
--rony-fg-soft: #a1a1aa;
--rony-border: #3a3a42;
--rony-assistant-bubble: #2a2a30;
--rony-shadow: 0 8px 28px rgba(0, 0, 0, 0.5);
}
}
.rony-chat-widget-root[data-theme="dark"] {
--rony-bg: #1f1f23;
--rony-bg-soft: #2a2a30;
--rony-fg: #f1f1f3;
--rony-fg-soft: #a1a1aa;
--rony-border: #3a3a42;
--rony-assistant-bubble: #2a2a30;
--rony-shadow: 0 8px 28px rgba(0, 0, 0, 0.5);
}
/* Bubble button */
.rony-chat-widget-bubble {
width: 56px;
height: 56px;
border-radius: 50%;
background: var(--rony-accent);
color: var(--rony-accent-fg);
border: none;
cursor: pointer;
box-shadow: var(--rony-shadow);
display: flex;
align-items: center;
justify-content: center;
transition: transform 120ms ease;
}
.rony-chat-widget-bubble:hover { transform: scale(1.05); }
.rony-chat-widget-bubble svg { width: 26px; height: 26px; }
/* Panel */
.rony-chat-widget-panel {
position: absolute;
bottom: 72px;
right: 0;
width: 380px;
max-width: calc(100vw - 40px);
height: 560px;
max-height: calc(100vh - 100px);
background: var(--rony-bg);
border: 1px solid var(--rony-border);
border-radius: var(--rony-radius);
box-shadow: var(--rony-shadow);
display: none;
flex-direction: column;
overflow: hidden;
}
.rony-chat-widget-root[data-open="true"] .rony-chat-widget-panel { display: flex; }
.rony-chat-widget-header {
padding: 12px 16px;
border-bottom: 1px solid var(--rony-border);
font-weight: 600;
display: flex;
align-items: center;
justify-content: space-between;
}
.rony-chat-widget-status {
font-size: 11px;
color: var(--rony-fg-soft);
font-weight: 400;
}
/* Language toggle (EN | ES) */
.rony-chat-widget-lang {
display: inline-flex;
border: 1px solid var(--rony-border);
border-radius: 6px;
overflow: hidden;
margin: 0 4px;
}
.rony-chat-widget-lang button {
background: transparent;
border: none;
color: var(--rony-fg-soft);
font-size: 10.5px;
font-weight: 600;
padding: 2px 6px;
cursor: pointer;
min-width: 26px;
font-family: inherit;
letter-spacing: 0.3px;
}
.rony-chat-widget-lang button:hover { color: var(--rony-fg); }
.rony-chat-widget-lang button.is-active {
background: var(--rony-accent);
color: var(--rony-accent-fg);
}
.rony-chat-widget-lang button + button {
border-left: 1px solid var(--rony-border);
}
.rony-chat-widget-close {
background: none;
border: none;
color: var(--rony-fg-soft);
cursor: pointer;
font-size: 20px;
padding: 0 4px;
line-height: 1;
}
.rony-chat-widget-messages {
flex: 1;
overflow-y: auto;
padding: 16px;
display: flex;
flex-direction: column;
gap: 10px;
}
.rony-chat-widget-msg {
max-width: 88%;
padding: 10px 12px;
border-radius: 12px;
word-wrap: break-word;
white-space: pre-wrap;
}
.rony-chat-widget-msg-user {
align-self: flex-end;
background: var(--rony-user-bubble);
color: var(--rony-user-fg);
white-space: pre-wrap;
}
.rony-chat-widget-msg-assistant {
align-self: flex-start;
background: var(--rony-assistant-bubble);
color: var(--rony-fg);
}
.rony-chat-widget-msg-error {
align-self: center;
background: #fee2e2;
color: #991b1b;
font-size: 12px;
}
/* Markdown inside assistant messages */
.rony-chat-widget-msg-assistant p { margin: 0 0 6px 0; }
.rony-chat-widget-msg-assistant p:last-child { margin-bottom: 0; }
.rony-chat-widget-msg-assistant code {
background: rgba(0, 0, 0, 0.06);
padding: 1px 5px;
border-radius: 4px;
font-family: ui-monospace, SFMono-Regular, Menlo, monospace;
font-size: 12.5px;
}
.rony-chat-widget-msg-assistant pre {
background: rgba(0, 0, 0, 0.06);
padding: 8px 10px;
border-radius: 6px;
overflow-x: auto;
margin: 6px 0;
}
.rony-chat-widget-msg-assistant pre code { background: none; padding: 0; }
.rony-chat-widget-msg-assistant ul, .rony-chat-widget-msg-assistant ol { margin: 4px 0 4px 18px; padding: 0; }
.rony-chat-widget-msg-assistant a { color: var(--rony-accent); }
/* Sources chip row */
.rony-chat-widget-sources {
display: flex;
flex-wrap: wrap;
gap: 4px;
margin-top: 6px;
}
.rony-chat-widget-source {
font-size: 10.5px;
background: var(--rony-bg-soft);
color: var(--rony-fg-soft);
padding: 2px 7px;
border-radius: 999px;
border: 1px solid var(--rony-border);
}
/* Streaming caret */
.rony-chat-widget-caret {
display: inline-block;
width: 6px;
height: 14px;
background: var(--rony-fg-soft);
margin-left: 2px;
vertical-align: text-bottom;
animation: rony-caret 1s steps(2) infinite;
}
@keyframes rony-caret { 50% { opacity: 0; } }
/* Input row */
.rony-chat-widget-form {
display: flex;
border-top: 1px solid var(--rony-border);
padding: 8px;
gap: 6px;
}
.rony-chat-widget-input {
flex: 1;
border: 1px solid var(--rony-border);
background: var(--rony-bg);
color: var(--rony-fg);
border-radius: 8px;
padding: 8px 10px;
font-family: inherit;
font-size: 14px;
resize: none;
outline: none;
max-height: 100px;
}
.rony-chat-widget-input:focus { border-color: var(--rony-accent); }
.rony-chat-widget-send {
background: var(--rony-accent);
color: var(--rony-accent-fg);
border: none;
border-radius: 8px;
padding: 0 14px;
font-weight: 600;
cursor: pointer;
}
.rony-chat-widget-send:disabled { opacity: 0.5; cursor: not-allowed; }
/* Mobile: full screen */
@media (max-width: 480px) {
.rony-chat-widget-panel {
position: fixed;
inset: 0;
width: 100vw;
height: 100vh;
max-height: 100vh;
border-radius: 0;
bottom: 0;
right: 0;
}
}

397
web/chat-widget.js Normal file
View file

@ -0,0 +1,397 @@
// rony-chat-widget — drop-in vanilla JS chat widget.
//
// Usage (HTML):
// <link rel="stylesheet" href="/path/to/chat-widget.css">
// <script src="/path/to/chat-widget.js"
// data-api-url="http://localhost:7331"
// data-title="Ask me anything"
// data-position="bottom-right"
// data-theme="auto"
// defer></script>
//
// All options are read from <script data-*="..."> attributes; everything is
// optional except data-api-url. The widget is self-contained: no build step,
// no runtime dependencies, no global CSS pollution.
(function () {
"use strict";
// ---- i18n -----------------------------------------------------------------
// The widget UI is bilingual. The conversation language (what the LLM
// answers in) is decided by the user query and is unaffected by this toggle.
var STRINGS = {
en: {
placeholder: "Type a message...",
send: "Send",
online: "online",
offline: "offline",
error: "error",
errorConnect: "Could not reach the server: ",
errorGeneric: "Unknown error",
ariaOpen: "Open chat",
ariaClose: "Close",
ariaLang: "Language",
ariaSend: "Send",
},
es: {
placeholder: "Escribe un mensaje...",
send: "Enviar",
online: "conectado",
offline: "desconectado",
error: "error",
errorConnect: "No se pudo conectar al servidor: ",
errorGeneric: "Error desconocido",
ariaOpen: "Abrir chat",
ariaClose: "Cerrar",
ariaLang: "Idioma",
ariaSend: "Enviar",
},
};
var LANG_KEY = "rony-chat-lang";
function pickInitialLang() {
var saved = null;
try { saved = localStorage.getItem(LANG_KEY); } catch (e) {}
if (saved === "en" || saved === "es") return saved;
var nav = (navigator.language || "en").toLowerCase();
return nav.indexOf("es") === 0 ? "es" : "en";
}
function saveLang(lang) {
try { localStorage.setItem(LANG_KEY, lang); } catch (e) {}
}
// ---- Config ----------------------------------------------------------------
function readConfig() {
var scripts = document.querySelectorAll("script[data-api-url], script[data-rony-chat]");
var tag = scripts[scripts.length - 1] || document.currentScript || {};
var ds = tag.dataset || {};
return {
apiUrl: (ds.apiUrl || "").replace(/\/+$/, ""),
title: ds.title || "Chat",
greeting: ds.greeting || "",
position: ds.position || "bottom-right",
theme: ds.theme || "auto",
};
}
// ---- Tiny markdown subset (no dependency) ----------------------------------
// Covers: **bold**, *italic*, `code`, ```fences```, lists, [links], paragraphs.
function escapeHTML(s) {
return s.replace(/[&<>"']/g, function (c) {
return { "&": "&amp;", "<": "&lt;", ">": "&gt;", '"': "&quot;", "'": "&#39;" }[c];
});
}
function renderMarkdown(src) {
var fences = [];
src = src.replace(/```([\s\S]*?)```/g, function (_, code) {
fences.push(code.replace(/^\n/, ""));
return "\u0000F" + (fences.length - 1) + "\u0000";
});
var inlines = [];
src = src.replace(/`([^`\n]+)`/g, function (_, code) {
inlines.push(code);
return "\u0000I" + (inlines.length - 1) + "\u0000";
});
src = escapeHTML(src);
src = src.replace(/\u0000I(\d+)\u0000/g, function (_, i) {
return "<code>" + escapeHTML(inlines[+i]) + "</code>";
});
src = src.replace(/\u0000F(\d+)\u0000/g, function (_, i) {
return "<pre><code>" + escapeHTML(fences[+i]) + "</code></pre>";
});
src = src.replace(/\*\*([^*]+)\*\*/g, "<strong>$1</strong>");
src = src.replace(/\*([^*]+)\*/g, "<em>$1</em>");
src = src.replace(/\[([^\]]+)\]\(([^)]+)\)/g, function (_, t, u) {
var safe = /^(https?:|mailto:|#|\/)/i.test(u) ? u : "#";
return '<a href="' + safe + '" target="_blank" rel="noopener noreferrer">' + t + "</a>";
});
src = src.replace(/(^|\n)((?:[-*] |\d+\. ).+(?:\n(?:[-*] |\d+\. ).+)*)/g, function (m, lead, block) {
var lines = block.split("\n");
var isOrdered = /^\d+\. /.test(lines[0]);
var tag = isOrdered ? "ol" : "ul";
var items = lines.map(function (l) {
return "<li>" + l.replace(/^[-*] |\d+\. /, "") + "</li>";
}).join("");
return lead + "<" + tag + ">" + items + "</" + tag + ">";
});
src = src
.split(/\n{2,}/)
.map(function (p) {
if (/^\s*<(pre|ul|ol|h\d|blockquote)/.test(p)) return p;
return "<p>" + p.replace(/\n/g, "<br>") + "</p>";
})
.join("\n");
return src;
}
// ---- SSE parsing -----------------------------------------------------------
function readSSE(response, onEvent, signal) {
var reader = response.body.getReader();
var decoder = new TextDecoder("utf-8");
var buffer = "";
var aborted = false;
if (signal) {
signal.addEventListener("abort", function () {
aborted = true;
try { reader.cancel(); } catch (e) {}
});
}
function pump() {
if (aborted) return;
return reader.read().then(function (r) {
if (r.done) return;
buffer += decoder.decode(r.value, { stream: true });
var idx;
while ((idx = buffer.indexOf("\n\n")) !== -1) {
var raw = buffer.slice(0, idx);
buffer = buffer.slice(idx + 2);
var ev = { event: "message", data: "" };
raw.split("\n").forEach(function (line) {
if (line.indexOf("event: ") === 0) ev.event = line.slice(7).trim();
else if (line.indexOf("data: ") === 0) ev.data += (ev.data ? "\n" : "") + line.slice(6);
});
if (ev.data) onEvent(ev.event, ev.data);
}
return pump();
});
}
return pump();
}
// ---- Widget construction --------------------------------------------------
function buildWidget(cfg) {
var root = document.createElement("div");
root.className = "rony-chat-widget-root";
root.setAttribute("data-position", cfg.position);
root.setAttribute("data-theme", cfg.theme);
root.setAttribute("data-open", "false");
root.innerHTML = [
'<div class="rony-chat-widget-panel" role="dialog" aria-label="' + escapeHTML(cfg.title) + '">',
' <div class="rony-chat-widget-header">',
' <span class="rony-chat-widget-title">' + escapeHTML(cfg.title) + '</span>',
' <div class="rony-chat-widget-lang" role="group" aria-label="Language">',
' <button type="button" data-lang-btn="en" aria-pressed="false">EN</button>',
' <button type="button" data-lang-btn="es" aria-pressed="false">ES</button>',
' </div>',
' <span class="rony-chat-widget-status" data-status>online</span>',
' <button class="rony-chat-widget-close" aria-label="Close" data-close>×</button>',
' </div>',
' <div class="rony-chat-widget-messages" data-messages></div>',
' <form class="rony-chat-widget-form" data-form>',
' <textarea class="rony-chat-widget-input" data-input rows="1" placeholder="Type a message..."></textarea>',
' <button class="rony-chat-widget-send" type="submit" data-send>Send</button>',
' </form>',
'</div>',
'<button class="rony-chat-widget-bubble" aria-label="Open chat" data-bubble>',
' <svg viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">',
' <path d="M21 15a2 2 0 0 1-2 2H7l-4 4V5a2 2 0 0 1 2-2h14a2 2 0 0 1 2 2z"/>',
' </svg>',
'</button>',
].join("\n");
return root;
}
function init() {
var cfg = readConfig();
if (!cfg.apiUrl) {
console.error("[rony-chat-widget] missing data-api-url on <script> tag");
return;
}
var widget = buildWidget(cfg);
document.body.appendChild(widget);
var $messages = widget.querySelector("[data-messages]");
var $input = widget.querySelector("[data-input]");
var $form = widget.querySelector("[data-form]");
var $send = widget.querySelector("[data-send]");
var $bubble = widget.querySelector("[data-bubble]");
var $close = widget.querySelector("[data-close]");
var $status = widget.querySelector("[data-status]");
var $langBtns = widget.querySelectorAll("[data-lang-btn]");
var $title = widget.querySelector(".rony-chat-widget-title");
var $panel = widget.querySelector(".rony-chat-widget-panel");
var $langGroup = widget.querySelector(".rony-chat-widget-lang");
var history = [];
var busy = false;
var abortCtrl = null;
var lang = pickInitialLang();
// ---- Language handling -----------------------------------------------
function applyLang(next) {
lang = next;
saveLang(next);
var t = STRINGS[next];
$input.placeholder = t.placeholder;
$send.textContent = t.send;
$send.setAttribute("aria-label", t.ariaSend);
$close.setAttribute("aria-label", t.ariaClose);
$bubble.setAttribute("aria-label", t.ariaOpen);
$langGroup.setAttribute("aria-label", t.ariaLang);
$panel.setAttribute("lang", next);
widget.setAttribute("data-lang", next);
for (var i = 0; i < $langBtns.length; i++) {
var active = $langBtns[i].getAttribute("data-lang-btn") === next;
$langBtns[i].setAttribute("aria-pressed", active ? "true" : "false");
$langBtns[i].classList.toggle("is-active", active);
}
// Refresh the visible status text with the new language
$status.textContent = t[lastStatusKey] || t.online;
}
applyLang(lang);
for (var j = 0; j < $langBtns.length; j++) {
$langBtns[j].addEventListener("click", function (e) {
var next = e.currentTarget.getAttribute("data-lang-btn");
if (next && next !== lang) applyLang(next);
});
}
// ---- Chat behavior --------------------------------------------------
var lastStatusKey = "online";
function setStatus(key) {
lastStatusKey = key;
$status.textContent = STRINGS[lang][key] || key;
}
function setOpen(open) {
widget.setAttribute("data-open", open ? "true" : "false");
if (open && !history.length && cfg.greeting) {
appendMessage("assistant", cfg.greeting);
history.push({ role: "assistant", content: cfg.greeting });
}
if (open) $input.focus();
}
function appendMessage(role, content) {
var div = document.createElement("div");
div.className = "rony-chat-widget-msg rony-chat-widget-msg-" + role;
if (role === "assistant") {
div.innerHTML = renderMarkdown(content);
} else {
div.textContent = content;
}
$messages.appendChild(div);
$messages.scrollTop = $messages.scrollHeight;
return div;
}
function appendError(msg) {
var div = document.createElement("div");
div.className = "rony-chat-widget-msg rony-chat-widget-msg-error";
div.textContent = msg;
$messages.appendChild(div);
$messages.scrollTop = $messages.scrollHeight;
}
function send(userText) {
if (busy || !userText.trim()) return;
busy = true;
$send.disabled = true;
$input.value = "";
autoSize();
history.push({ role: "user", content: userText });
appendMessage("user", userText);
var assistantDiv = appendMessage("assistant", "");
var caret = document.createElement("span");
caret.className = "rony-chat-widget-caret";
assistantDiv.appendChild(caret);
abortCtrl = new AbortController();
var collectedSources = [];
fetch(cfg.apiUrl + "/api/chat", {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({ messages: history, stream: true }),
signal: abortCtrl.signal,
}).then(function (resp) {
if (!resp.ok) {
throw new Error("HTTP " + resp.status);
}
return readSSE(resp, function (type, data) {
try {
var payload = JSON.parse(data);
} catch (e) {
return;
}
if (type === "chunk" && payload.content) {
assistantDiv.insertBefore(document.createTextNode(payload.content), caret);
$messages.scrollTop = $messages.scrollHeight;
} else if (type === "sources" && Array.isArray(payload.documents)) {
collectedSources = payload.documents;
} else if (type === "done") {
var finalText = assistantDiv.textContent.replace(/\s+$/, "");
history.push({ role: "assistant", content: finalText });
if (caret.parentNode) caret.parentNode.removeChild(caret);
assistantDiv.innerHTML = renderMarkdown(finalText);
if (collectedSources.length) {
var row = document.createElement("div");
row.className = "rony-chat-widget-sources";
collectedSources.forEach(function (s) {
var chip = document.createElement("span");
chip.className = "rony-chat-widget-source";
chip.textContent = s;
row.appendChild(chip);
});
assistantDiv.appendChild(row);
}
setStatus("online");
} else if (type === "error") {
if (caret.parentNode) caret.parentNode.removeChild(caret);
appendError(payload.error || STRINGS[lang].errorGeneric);
setStatus("error");
}
}, abortCtrl.signal);
}).catch(function (err) {
if (err.name === "AbortError") return;
if (caret.parentNode) caret.parentNode.removeChild(caret);
appendError(STRINGS[lang].errorConnect + err.message);
setStatus("offline");
}).then(function () {
busy = false;
$send.disabled = false;
abortCtrl = null;
});
}
function autoSize() {
$input.style.height = "auto";
$input.style.height = Math.min($input.scrollHeight, 100) + "px";
}
$bubble.addEventListener("click", function () { setOpen(true); });
$close.addEventListener("click", function () { setOpen(false); });
$input.addEventListener("input", autoSize);
$input.addEventListener("keydown", function (e) {
if (e.key === "Enter" && !e.shiftKey) {
e.preventDefault();
$form.requestSubmit();
}
});
$form.addEventListener("submit", function (e) {
e.preventDefault();
send($input.value);
});
}
if (document.readyState === "loading") {
document.addEventListener("DOMContentLoaded", init);
} else {
init();
}
})();

51
web/example.html Normal file
View file

@ -0,0 +1,51 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>Rony Chat Widget — demo</title>
<link rel="stylesheet" href="chat-widget.css">
<style>
body {
max-width: 720px;
margin: 40px auto;
padding: 0 20px;
font: 16px/1.6 -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif;
color: #1a1a1a;
}
h1 { margin-bottom: 4px; }
.muted { color: #5c5c66; }
pre { background: #f5f5f7; padding: 12px; border-radius: 8px; overflow-x: auto; }
</style>
</head>
<body>
<h1>Rony Chat Widget — local demo</h1>
<p class="muted">The bubble in the bottom-right is the widget. It talks to the local chat-bot server on :7331.</p>
<h2>Try asking</h2>
<ul>
<li>“What database does the project use?”</li>
<li>“¿De qué trata el portfolio?” (responde en inglés porque el contenido indexado lo está)</li>
<li>“Tell me about the tech stack”</li>
</ul>
<h2>Embed snippet (copy this into your site)</h2>
<pre>&lt;link rel="stylesheet" href="/chat-widget.css"&gt;
&lt;script src="/chat-widget.js"
data-api-url="http://localhost:7331"
data-title="Ask me anything"
data-greeting="Hi! Ask me about the projects."
data-position="bottom-right"
data-theme="auto"
defer&gt;&lt;/script&gt;</pre>
<!-- The widget itself -->
<script src="chat-widget.js"
data-api-url="http://localhost:7331"
data-title="Rony Chat"
data-greeting="Hi! I'm Rony's assistant. Ask me about the projects."
data-position="bottom-right"
data-theme="auto"
defer></script>
</body>
</html>