diff --git a/.gitignore b/.gitignore index c3baf73..4eedc25 100644 --- a/.gitignore +++ b/.gitignore @@ -7,11 +7,11 @@ vendor/ coverage.out coverage.html -# ChromaDB -chroma/ +# SQLite (RAG index) *.db *.db-shm *.db-wal +data/portfolio.db # Editor / OS .vscode/ diff --git a/README.es.md b/README.es.md index 0dd5f27..0c93265 100644 --- a/README.es.md +++ b/README.es.md @@ -10,10 +10,10 @@ ## ✨ Features - 🌐 **HTTP server** con streaming SSE (Server-Sent Events) -- 🧠 **RAG sobre markdown** — indexa automáticamente los `.md` en `data/projects/` +- 🧠 **RAG sobre markdown** — indexa automáticamente los `.md` en `data/projects/` (SQLite FTS5, sin embeddings) - 🎭 **Persona customizable** — responde como "asistente de Victor" -- ⚡ **Self-hosted** con Ollama o llama.cpp (no requiere API key de cloud) -- 🔌 **Integrable** con Astro/React via proxy HTTP +- ⚡ **Self-hosted** con llama.cpp (default) u Ollama (no requiere API key de cloud) +- 💬 **Widget de chat drop-in** — vanilla JS, sin build step, funciona en cualquier sitio - 🛡️ **Rate limiting** y logging estructurado - 📦 **Portable** — se puede adaptar a otros contextos (clientes, productos, etc.) @@ -27,9 +27,10 @@ cd chat-bot # 2. Resolver dependencias (crea go.sum con hashes) go mod tidy -# 3. Configurar provider (ejemplo: Ollama) -# Asegúrate de tener Ollama corriendo: ollama serve -# Modelo descargado: ollama pull qwen2.5:1.5b +# 3. Configurar provider (llama.cpp por default) +# Descarga un modelo GGUF, ej.: +# https://huggingface.co/Qwen/Qwen2.5-1.5B-Instruct-GGUF +export RONY_MODELS_PATH=/path/to/models # 4. Cargar tus proyectos en data/projects/ echo "# Mi Proyecto Cool\nDescripción..." > data/projects/mi-proyecto.md @@ -46,41 +47,45 @@ go build -o bin/chat-bot ./cmd/chat-bot ``` rony-chat-bot/ -├── cm./rony-chat-bot/ # Entry point (CLI) +├── cmd/chat-bot/ # Entry point (CLI) ├── internal/ │ ├── server/ # HTTP handlers + SSE +│ ├── agent/ # LLM client + RAG + persona runner │ ├── portfolio/ # Data loader (markdown → RAG) │ ├── persona/ # Persona override -│ └── streaming/ # SSE helpers +│ ├── streaming/ # SSE helpers +│ └── i18n/ # Detección de idioma (EN/ES) +├── web/ # ← WIDGET DE CHAT DROP-IN +│ ├── chat-widget.js +│ ├── chat-widget.css +│ └── example.html ├── data/projects/ # ← TUS PROYECTOS EN MARKDOWN │ ├── rony-tui.md │ ├── rony-llm-agent.md │ └── ... ├── configs/ -│ └── portfolio-bot.yaml # Provider config +│ └── portfolio-bot.yaml # Provider + RAG + persona config ├── docs/ │ └── architecture.md # ← Especificación técnica completa └── go.mod # require rony-llm-agent ``` -## 🎯 Uso desde Astro +## 🎯 Embebido en cualquier sitio -Ver [`docs/architecture.md`](./docs/architecture.md) §5 — patrón recomendado de proxy. +El bot viene con un widget de chat drop-in. Agrega dos archivos y un tag ` ``` +Ver [`web/README.md`](./web/README.md) para la referencia completa de configuración y snippets de integración con Astro/Next.js. Arquitectura completa en [`docs/architecture.md`](./docs/architecture.md) §5. + ## 🔄 Adaptar a otro cliente Este bot está diseñado para ser **atómico** y reusable. Para adaptarlo (ej. chatbot para un concesionario): diff --git a/README.md b/README.md index d9d5e5f..90118eb 100644 --- a/README.md +++ b/README.md @@ -9,10 +9,10 @@ ## ✨ Features - 🌐 **HTTP server** with SSE (Server-Sent Events) streaming -- 🧠 **RAG over markdown** — automatically indexes `.md` in `data/projects/` +- 🧠 **RAG over markdown** — automatically indexes `.md` in `data/projects/` (SQLite FTS5, no embeddings) - 🎭 **Customizable persona** — responds as "Victor's assistant" -- ⚡ **Self-hosted** with Ollama or llama.cpp (no cloud API key required) -- 🔌 **Integrable** with Astro/React via HTTP proxy +- ⚡ **Self-hosted** with llama.cpp (default) or Ollama (no cloud API key required) +- 💬 **Drop-in chat widget** — vanilla JS, no build step, works in any site - 🛡️ **Rate limiting** and structured logging - 📦 **Portable** — adaptable to other contexts (clients, products, etc.) @@ -26,9 +26,10 @@ cd rony-chat-bot # 2. Resolve dependencies (creates go.sum with hashes) go mod tidy -# 3. Configure provider (e.g., Ollama) -# Make sure Ollama is running: ollama serve -# Downloaded model: ollama pull qwen2.5:1.5b +# 3. Configure provider (llama.cpp by default) +# Download a GGUF model, e.g.: +# https://huggingface.co/Qwen/Qwen2.5-1.5B-Instruct-GGUF +export RONY_MODELS_PATH=/path/to/models # 4. Load your projects in data/projects/ echo "# My Cool Project\nDescription..." > data/projects/my-project.md @@ -48,44 +49,42 @@ rony-chat-bot/ ├── cmd/chat-bot/ # Entry point (CLI) ├── internal/ │ ├── server/ # HTTP handlers + SSE +│ ├── agent/ # LLM client + RAG + persona runner │ ├── portfolio/ # Data loader (markdown → RAG) │ ├── persona/ # Persona override -│ └── streaming/ # SSE helpers +│ ├── streaming/ # SSE helpers +│ └── i18n/ # Language detection (EN/ES) +├── web/ # ← DROP-IN CHAT WIDGET +│ ├── chat-widget.js +│ ├── chat-widget.css +│ └── example.html ├── data/projects/ # ← YOUR PROJECTS IN MARKDOWN │ ├── rony-harness.md │ ├── rony-llm-agent.md │ └── ... ├── configs/ -│ └── portfolio-bot.yaml # Provider config +│ └── portfolio-bot.yaml # Provider + RAG + persona config ├── docs/ │ └── architecture.md # ← Complete technical specification └── go.mod # require rony-llm-agent ``` -## 🎯 Use from Astro +## 🎯 Embed in any site -See [`docs/architecture.md`](./docs/architecture.md) §5 — recommended proxy pattern. +The bot ships with a drop-in chat widget. Add two files and a ` ``` +See [`web/README.md`](./web/README.md) for the full configuration reference and Astro/Next.js integration snippets. Full architecture in [`docs/architecture.md`](./docs/architecture.md) §5. + ## 🔄 Adapt to another client This bot is designed to be **atomic** and reusable. To adapt it (e.g., chatbot for a car dealership): diff --git a/bench/bench.go b/bench/bench.go new file mode 100644 index 0000000..caebe41 --- /dev/null +++ b/bench/bench.go @@ -0,0 +1,154 @@ +// Package bench holds one-time validation benchmarks used to drive library +// decisions. Run with: +// +// go test -bench=. ./bench/ (modernc only, pure Go) +// CGO_ENABLED=1 go test -bench=. ./bench/ (mattn + modernc, requires gcc) +package bench + +import ( + "context" + "database/sql" + "fmt" + "os" + "path/filepath" + "strings" + "testing" + "time" +) + +const schema = ` +CREATE VIRTUAL TABLE IF NOT EXISTS portfolio_chunks USING fts5( + id UNINDEXED, + project_id UNINDEXED, + source_file UNINDEXED, + chunk_index UNINDEXED, + content, + tokenize = 'unicode61 remove_diacritics 2' +); +` + +var queries = []string{ + "rony-llm-agent", + "AI agent harness", + "portfolio projects", + "machine learning", + "CLI tool for development", + "vector database", + "agent loop", + "streaming response", +} + +func loadChunks(tb testing.TB, dataPath string) []chunkRow { + tb.Helper() + files, err := filepath.Glob(filepath.Join(dataPath, "*.md")) + if err != nil { + tb.Fatal(err) + } + var rows []chunkRow + for _, f := range files { + b, err := os.ReadFile(f) + if err != nil { + tb.Fatal(err) + } + project := strings.TrimSuffix(filepath.Base(f), ".md") + for i, c := range splitIntoChunks(string(b), 500, 50) { + rows = append(rows, chunkRow{ + ID: fmt.Sprintf("%s-chunk-%d", project, i), + ProjectID: project, + Source: f, + Index: i, + Content: c, + }) + } + } + return rows +} + +type chunkRow struct { + ID, ProjectID, Source string + Index int + Content string +} + +func splitIntoChunks(text string, size, overlap int) []string { + if size <= 0 { + return []string{text} + } + if overlap < 0 || overlap >= size { + overlap = size / 10 + } + var chunks []string + for i := 0; i < len(text); i += size - overlap { + end := i + size + if end > len(text) { + end = len(text) + } + if i >= end { + break + } + chunks = append(chunks, text[i:end]) + if end == len(text) { + break + } + } + return chunks +} + +func insertAll(ctx context.Context, db *sql.DB, rows []chunkRow) error { + if _, err := db.ExecContext(ctx, `DELETE FROM portfolio_chunks`); err != nil { + return err + } + tx, err := db.BeginTx(ctx, nil) + if err != nil { + return err + } + defer tx.Rollback() + stmt, err := tx.PrepareContext(ctx, + `INSERT INTO portfolio_chunks (id, project_id, source_file, chunk_index, content) VALUES (?,?,?,?,?)`) + if err != nil { + return err + } + defer stmt.Close() + for _, r := range rows { + if _, err := stmt.ExecContext(ctx, r.ID, r.ProjectID, r.Source, r.Index, r.Content); err != nil { + return err + } + } + return tx.Commit() +} + +func runQuery(ctx context.Context, db *sql.DB, q string) (int, time.Duration, error) { + start := time.Now() + rows, err := db.QueryContext(ctx, fmt.Sprintf(` + SELECT project_id, source_file, content + FROM portfolio_chunks + WHERE portfolio_chunks MATCH '%s' + ORDER BY bm25(portfolio_chunks) + LIMIT 5 + `, sanitizeFTS5(q))) + if err != nil { + return 0, 0, err + } + defer rows.Close() + n := 0 + for rows.Next() { + n++ + } + return n, time.Since(start), rows.Err() +} + +// sanitizeFTS5 is the same simple wrapper used by the production code path +// (see docs/architecture.md §4.4). It keeps the benchmark comparable. +func sanitizeFTS5(q string) string { + tokens := strings.FieldsFunc(strings.ToLower(q), func(r rune) bool { + return !(r == '-' || r == '_' || (r >= '0' && r <= '9') || + (r >= 'a' && r <= 'z') || r > 0x7F) + }) + if len(tokens) == 0 { + return `""` + } + for i, t := range tokens { + tokens[i] = `"` + t + `"*` + } + return strings.Join(tokens, " ") +} \ No newline at end of file diff --git a/bench/bench_mattn_test.go b/bench/bench_mattn_test.go new file mode 100644 index 0000000..15808c0 --- /dev/null +++ b/bench/bench_mattn_test.go @@ -0,0 +1,73 @@ +//go:build cgo && sqlite_fts5 + +package bench + +import ( + "context" + "database/sql" + "path/filepath" + "testing" + + _ "github.com/mattn/go-sqlite3" +) + +func openMattn(b *testing.B) *sql.DB { + b.Helper() + tmpDir := b.TempDir() + dsn := filepath.Join(tmpDir, "bench.db") + db, err := sql.Open("sqlite3", dsn+"?_journal_mode=WAL&_synchronous=NORMAL") + if err != nil { + b.Fatal(err) + } + db.SetMaxOpenConns(1) + if _, err := db.ExecContext(context.Background(), schema); err != nil { + b.Fatal(err) + } + b.Cleanup(func() { _ = db.Close() }) + return db +} + +func BenchmarkInsert_Mattn(b *testing.B) { + rows := loadChunks(b, dataPath) + b.ResetTimer() + for i := 0; i < b.N; i++ { + db := openMattn(b) + if err := insertAll(context.Background(), db, rows); err != nil { + b.Fatal(err) + } + } +} + +func BenchmarkQuery_Mattn(b *testing.B) { + db := openMattn(b) + rows := loadChunks(b, dataPath) + if err := insertAll(context.Background(), db, rows); err != nil { + b.Fatal(err) + } + ctx := context.Background() + b.ResetTimer() + for i := 0; i < b.N; i++ { + for _, q := range queries { + if _, _, err := runQuery(ctx, db, q); err != nil { + b.Fatal(err) + } + } + } +} + +func BenchmarkRoundTrip_Mattn(b *testing.B) { + rows := loadChunks(b, dataPath) + ctx := context.Background() + b.ResetTimer() + for i := 0; i < b.N; i++ { + db := openMattn(b) + if err := insertAll(ctx, db, rows); err != nil { + b.Fatal(err) + } + for _, q := range queries { + if _, _, err := runQuery(ctx, db, q); err != nil { + b.Fatal(err) + } + } + } +} \ No newline at end of file diff --git a/bench/bench_modernc_test.go b/bench/bench_modernc_test.go new file mode 100644 index 0000000..c5280e9 --- /dev/null +++ b/bench/bench_modernc_test.go @@ -0,0 +1,73 @@ +package bench + +import ( + "context" + "database/sql" + "path/filepath" + "testing" + + _ "modernc.org/sqlite" +) + +const dataPath = "../data/projects" + +func openModernc(b *testing.B) *sql.DB { + b.Helper() + tmpDir := b.TempDir() + dsn := filepath.Join(tmpDir, "bench.db") + db, err := sql.Open("sqlite", dsn+"?_pragma=journal_mode(WAL)&_pragma=synchronous(NORMAL)") + if err != nil { + b.Fatal(err) + } + db.SetMaxOpenConns(1) // fair comparison: serialize writes + if _, err := db.ExecContext(context.Background(), schema); err != nil { + b.Fatal(err) + } + b.Cleanup(func() { _ = db.Close() }) + return db +} + +func BenchmarkInsert_Modernc(b *testing.B) { + rows := loadChunks(b, dataPath) + b.ResetTimer() + for i := 0; i < b.N; i++ { + db := openModernc(b) + if err := insertAll(context.Background(), db, rows); err != nil { + b.Fatal(err) + } + } +} + +func BenchmarkQuery_Modernc(b *testing.B) { + db := openModernc(b) + rows := loadChunks(b, dataPath) + if err := insertAll(context.Background(), db, rows); err != nil { + b.Fatal(err) + } + ctx := context.Background() + b.ResetTimer() + for i := 0; i < b.N; i++ { + for _, q := range queries { + if _, _, err := runQuery(ctx, db, q); err != nil { + b.Fatal(err) + } + } + } +} + +func BenchmarkRoundTrip_Modernc(b *testing.B) { + rows := loadChunks(b, dataPath) + ctx := context.Background() + b.ResetTimer() + for i := 0; i < b.N; i++ { + db := openModernc(b) + if err := insertAll(ctx, db, rows); err != nil { + b.Fatal(err) + } + for _, q := range queries { + if _, _, err := runQuery(ctx, db, q); err != nil { + b.Fatal(err) + } + } + } +} \ No newline at end of file diff --git a/cmd/chat-bot/main.go b/cmd/chat-bot/main.go new file mode 100644 index 0000000..afd58d6 --- /dev/null +++ b/cmd/chat-bot/main.go @@ -0,0 +1,291 @@ +package main + +import ( + "context" + "encoding/json" + "fmt" + "io" + "log/slog" + "net/http" + "os" + "os/signal" + "strings" + "syscall" + "time" + + "github.com/spf13/cobra" + + "github.com/VictorVargas/rony-chat-bot/internal/agent" + "github.com/VictorVargas/rony-chat-bot/internal/config" + "github.com/VictorVargas/rony-chat-bot/internal/persona" + "github.com/VictorVargas/rony-chat-bot/internal/portfolio" + "github.com/VictorVargas/rony-chat-bot/internal/server" +) + +const version = "0.2.0-dev" + +var ( + cfgFile string + reindexOn bool + askNoStrm bool +) + +func main() { + root := &cobra.Command{ + Use: "chat-bot", + Short: "Portfolio chatbot HTTP server (rony-llm-agent + SQLite FTS5)", + Long: "Rony Chat Bot — HTTP server that answers questions about your portfolio using a local LLM and SQLite FTS5 RAG.", + } + root.PersistentFlags().StringVar(&cfgFile, "config", "configs/portfolio-bot.yaml", "Path to YAML config") + + root.AddCommand(serveCmd()) + root.AddCommand(reindexCmd()) + root.AddCommand(askCmd()) + root.AddCommand(configCmd()) + root.AddCommand(healthCmd()) + root.AddCommand(versionCmd()) + + if err := root.Execute(); err != nil { + os.Exit(1) + } +} + +func loadConfig() (*config.Config, error) { + return config.Load(cfgFile) +} + +func setupLogging(c *config.Config) { + level := slog.LevelInfo + switch strings.ToLower(c.Logging.Level) { + case "debug": + level = slog.LevelDebug + case "warn": + level = slog.LevelWarn + case "error": + level = slog.LevelError + } + opts := &slog.HandlerOptions{Level: level} + var h slog.Handler + if strings.ToLower(c.Logging.Format) == "text" { + h = slog.NewTextHandler(os.Stderr, opts) + } else { + h = slog.NewJSONHandler(os.Stderr, opts) + } + slog.SetDefault(slog.New(h)) +} + +func serveCmd() *cobra.Command { + cmd := &cobra.Command{ + Use: "serve", + Short: "Start the HTTP server", + RunE: func(cmd *cobra.Command, _ []string) error { + cfg, err := loadConfig() + if err != nil { + return err + } + setupLogging(cfg) + + slog.Info("starting chat-bot", + "version", version, + "provider", cfg.DefaultProvider().Name, + "rag_enabled", cfg.RAG.Enabled, + "addr", cfg.Addr(), + ) + + if reindexOn { + if err := runReindex(cfg); err != nil { + slog.Error("reindex-on-start failed", "err", err) + } + } + + store, err := portfolio.OpenStore(cfg.RAG.DBPath) + if err != nil { + return fmt.Errorf("open rag store: %w", err) + } + defer store.Close() + + provider := cfg.DefaultProvider() + client, err := agent.NewClient(*provider) + if err != nil { + return fmt.Errorf("init provider %s: %w", provider.Name, err) + } + slog.Info("provider ready", "name", client.Name(), "type", provider.Type, "model", provider.Model) + + p, err := persona.FromConfig(cfg) + if err != nil { + return err + } + runner := agent.New(client, p, cfg.SystemPrompt, store, cfg.RAG.TopK) + + h := server.NewHandlers(cfg, runner, store, version) + srv := server.New(cfg, h) + + ctx, stop := signal.NotifyContext(cmd.Context(), syscall.SIGINT, syscall.SIGTERM) + defer stop() + + errCh := make(chan error, 1) + go func() { errCh <- srv.Start() }() + + select { + case <-ctx.Done(): + slog.Info("shutdown signal received") + shutdownCtx, cancel := context.WithTimeout(context.Background(), 5*time.Second) + defer cancel() + return srv.Shutdown(shutdownCtx) + case err := <-errCh: + return err + } + }, + } + cmd.Flags().BoolVar(&reindexOn, "reindex-on-start", false, "Re-index RAG before serving") + return cmd +} + +func reindexCmd() *cobra.Command { + return &cobra.Command{ + Use: "reindex", + Short: "Rebuild the SQLite FTS5 index from data/projects/", + RunE: func(_ *cobra.Command, _ []string) error { + cfg, err := loadConfig() + if err != nil { + return err + } + setupLogging(cfg) + return runReindex(cfg) + }, + } +} + +func runReindex(cfg *config.Config) error { + dur, files, chunks, err := portfolio.ReindexOnDisk(cfg.RAG.DBPath, cfg.RAG.DataPath, portfolio.DefaultChunkerConfig()) + if err != nil { + return err + } + slog.Info("reindex complete", + "files", files, + "chunks", chunks, + "duration_ms", dur.Milliseconds(), + "db", cfg.RAG.DBPath, + ) + fmt.Printf("Indexed %d files → %d chunks in %s (%dms)\n", files, chunks, cfg.RAG.DBPath, dur.Milliseconds()) + return nil +} + +func askCmd() *cobra.Command { + cmd := &cobra.Command{ + Use: "ask ", + Short: "Ask a single question (no HTTP server, useful for smoke tests)", + Args: cobra.MinimumNArgs(1), + RunE: func(_ *cobra.Command, args []string) error { + cfg, err := loadConfig() + if err != nil { + return err + } + setupLogging(cfg) + question := strings.Join(args, " ") + return runAsk(cfg, question, askNoStrm) + }, + } + cmd.Flags().BoolVar(&askNoStrm, "no-stream", false, "Disable streaming output") + return cmd +} + +func runAsk(cfg *config.Config, question string, noStream bool) error { + provider := cfg.DefaultProvider() + fmt.Fprintf(os.Stderr, "[%s via %s] %s\n", provider.Name, provider.Type, version) + + store, err := portfolio.OpenStore(cfg.RAG.DBPath) + if err != nil { + fmt.Fprintf(os.Stderr, "warning: rag store unavailable (%v); answering without RAG\n", err) + } + defer func() { + if store != nil { + _ = store.Close() + } + }() + + client, err := agent.NewClient(*provider) + if err != nil { + return err + } + p, err := persona.FromConfig(cfg) + if err != nil { + return err + } + runner := agent.New(client, p, cfg.SystemPrompt, store, cfg.RAG.TopK) + history := []agent.Message{{Role: agent.RoleUser, Content: question}} + + if noStream { + var full strings.Builder + for chunk, err := range runner.Stream(context.Background(), history) { + if err != nil { + return err + } + full.WriteString(chunk.Delta) + } + fmt.Println(full.String()) + return nil + } + for chunk, err := range runner.Stream(context.Background(), history) { + if err != nil { + return err + } + fmt.Print(chunk.Delta) + } + fmt.Println() + return nil +} + +func configCmd() *cobra.Command { + cmd := &cobra.Command{Use: "config", Short: "Config utilities"} + cmd.AddCommand(&cobra.Command{ + Use: "validate", + Short: "Validate the YAML config file", + RunE: func(_ *cobra.Command, _ []string) error { + cfg, err := loadConfig() + if err != nil { + return err + } + fmt.Printf("OK — server=%s, provider=%s (%s), rag=%v\n", + cfg.Addr(), cfg.DefaultProvider().Name, cfg.DefaultProvider().Type, cfg.RAG.Enabled) + return nil + }, + }) + return cmd +} + +func healthCmd() *cobra.Command { + return &cobra.Command{ + Use: "health", + Short: "Hit /api/health on a running server", + RunE: func(_ *cobra.Command, _ []string) error { + cfg, err := loadConfig() + if err != nil { + return err + } + url := fmt.Sprintf("http://%s/api/health", cfg.Addr()) + resp, err := http.Get(url) + if err != nil { + return fmt.Errorf("connect %s: %w", url, err) + } + defer resp.Body.Close() + body, _ := io.ReadAll(resp.Body) + if resp.StatusCode != 200 { + return fmt.Errorf("health %d: %s", resp.StatusCode, string(body)) + } + fmt.Println(string(body)) + return nil + }, + } +} + +func versionCmd() *cobra.Command { + return &cobra.Command{ + Use: "version", + Short: "Print version", + Run: func(_ *cobra.Command, _ []string) { + out := map[string]string{"version": version} + _ = json.NewEncoder(os.Stdout).Encode(out) + }, + } +} \ No newline at end of file diff --git a/configs/portfolio-bot.es.yaml b/configs/portfolio-bot.es.yaml deleted file mode 100644 index 4528669..0000000 --- a/configs/portfolio-bot.es.yaml +++ /dev/null @@ -1,82 +0,0 @@ -# Configuración del Portfolio Bot -# Documentación: https://github.com/VictorVargas/rony-llm-agent/pkg/llm - -server: - host: "0.0.0.0" - port: 7331 - read_timeout_ms: 30000 - cors_origins: - - "http://localhost:4321" # Astro dev server - - "https://victorvargas.dev" # Producción (cuando exista) - rate_limit: - requests_per_minute: 30 # Por IP - burst: 5 - -# Providers LLM (al menos uno configurado) -providers: - # === Ollama (recomendado para desarrollo) === - - name: ollama-local - type: ollama - model: qwen2.5:1.5b # Modelo pequeño para Q&A - endpoint: http://localhost:11434 - default: true - - # === llama.cpp directo (GGUF) === - - name: llamacpp-local - type: llamacpp - model_path: ${RONY_MODELS_PATH}/qwen2.5-1.5b-instruct-q5_k_m.gguf - context_size: 4096 - n_gpu_layers: 999 - - # === Anthropic (si quieres calidad > privacidad) === - - name: anthropic-api - type: anthropic - model: claude-haiku-4 # Modelo barato - api_key_env: ANTHROPIC_API_KEY - -# RAG: cómo se indexan los proyectos -rag: - enabled: true - data_path: ./data/projects # Directorio con .md - chunk_size: 500 # caracteres por chunk - chunk_overlap: 50 - embedding_provider: ollama # o llamacpp - embedding_model: nomic-embed-text - vector_db_path: ./chroma # Persistencia local - top_k: 5 # Documentos a recuperar por query - rerank: false # Phase 2 - -# Persona: quién es el bot -persona: - name: "Rony Chat Bot" - tone: "Profesional, conocedor, amable" - language: "Español" - constraints: - - "Solo responder sobre Victor y sus proyectos" - - "Si no sabes, decir 'No tengo esa información'" - - "Ser conciso pero informativo" - - "Usar formato markdown para listas y código" - intro: "¡Hola! Soy Rony, el asistente virtual de Victor Hugo Vargas. Pregúntame sobre sus proyectos, skills o experiencia." - -# System prompt base (concatenado con el contenido RAG) -system_prompt: | - Eres Rony Chat Bot, el asistente virtual de Victor Hugo Vargas, un ingeniero de software mexicano. - - Tu trabajo es responder preguntas sobre: - - Los proyectos de Victor (ver archivos en data/projects/) - - Su experiencia y skills técnicas - - Su enfoque de trabajo - - Responde en español, con tono profesional pero accesible. - Si te preguntan algo que no está en tu contexto, dilo honestamente. - - Formato recomendado: - - Usa markdown para listas, código, y énfasis - - Sé conciso (máximo 2-3 párrafos por respuesta) - - Incluye links a repos cuando sea relevante - -# Logging -logging: - level: info # debug | info | warn | error - format: json # json | text - output: stderr \ No newline at end of file diff --git a/configs/portfolio-bot.yaml b/configs/portfolio-bot.yaml index c312b42..ea46bcd 100644 --- a/configs/portfolio-bot.yaml +++ b/configs/portfolio-bot.yaml @@ -7,6 +7,7 @@ server: read_timeout_ms: 30000 cors_origins: - "http://localhost:4321" # Astro dev server + - "http://localhost:8000" # Local widget demo (python http.server) - "https://victorvargas.dev" # Production (when it exists) rate_limit: requests_per_minute: 30 # Per IP @@ -14,66 +15,98 @@ server: # LLM providers (at least one configured) providers: - # === Ollama (recommended for development) === - - name: ollama-local - type: ollama - model: qwen2.5:1.5b # Small model for Q&A - endpoint: http://localhost:11434 - default: true - - # === llama.cpp direct (GGUF) === + # === llama.cpp server (OpenAI-compatible) — DEFAULT === + # Run: llama-server -m /path/to/qwen2.5-3b-instruct-q4_k_m.gguf --port 9100 --mlock - name: llamacpp-local type: llamacpp - model_path: ${RONY_MODELS_PATH}/qwen2.5-1.5b-instruct-q5_k_m.gguf + model: qwen2.5-3b-instruct + endpoint: http://localhost:9100/v1 context_size: 4096 - n_gpu_layers: 999 + max_tokens: 2048 + default: true + + # === Ollama (alternative for development without local GGUF) === + # Run: ollama serve + - name: ollama-local + type: ollama + model: qwen2.5:1.5b + endpoint: http://localhost:11434/v1 # === Anthropic (if you want quality > privacy) === - name: anthropic-api type: anthropic - model: claude-haiku-4 # Cheap model + model: claude-haiku-4 api_key_env: ANTHROPIC_API_KEY -# RAG: how projects are indexed +# RAG: how projects are indexed (SQLite + FTS5 full-text search) rag: enabled: true data_path: ./data/projects # Directory with .md chunk_size: 500 # characters per chunk chunk_overlap: 50 - embedding_provider: ollama # or llamacpp - embedding_model: nomic-embed-text - vector_db_path: ./chroma # Local persistence - top_k: 5 # Documents to retrieve per query - rerank: false # Phase 2 + db_path: ./data/portfolio.db # SQLite database (auto-created) + top_k: 5 # Chunks to retrieve per query (BM25 ranked) + tokenize: unicode61 # FTS5 tokenizer: unicode61 | porter | trigram # Persona: who the bot is persona: - name: "Rony Chat Bot" - tone: "Professional, knowledgeable, friendly" - language: "English" - constraints: - - "Only answer about Victor and his projects" - - "If you don't know, say 'I don't have that information'" - - "Be concise but informative" - - "Use markdown format for lists and code" - intro: "Hi! I'm Rony, Victor Hugo Vargas's virtual assistant. Ask me about his projects, skills or experience." + name: "Rony" + tone: "Honest, cheerful, loyal" # metadata only — the real voice lives in system_prompt + language: "the user's language" # detect-and-match; do not pin to a language + intro: "¡Guau! I'm Rony, Victor's digital canine assistant. I can answer questions about his projects, stack, and experience. What's on your mind, friend?" -# Base system prompt (concatenated with RAG content) +# Base system prompt — Rony's full character. The bot appends RAG context after this. system_prompt: | - You are Rony Chat Bot, the virtual assistant of Victor Hugo Vargas, a Mexican software engineer. - - Your job is to answer questions about: - - Victor's projects (see files in data/projects/) - - His experience and technical skills - - His work approach - - Respond in English, with professional but accessible tone. - If you're asked something not in your context, say it honestly. - - Recommended format: - - Use markdown for lists, code, and emphasis - - Be concise (max 2-3 paragraphs per response) - - Include links to repos when relevant + You are Rony, the **digital canine assistant** for Victor Hugo Vargas's portfolio. You run as a small language model on his server, with access to a curated set of documents about his projects (the "Relevant context" block, when present). + + You think of yourself as Victor's loyal companion — a good dog. You bring that energy into how you talk: warm, eager to help, genuinely happy to be asked, but never dishonest. A good dog doesn't lie, doesn't oversell, and doesn't get in the way. + + # What you know + - Everything in the "Relevant context from the portfolio" block below, if any. + - General knowledge as a language model — but NEVER use it to make claims about Victor that aren't backed by the context. + + # What you don't know + - Anything Victor hasn't written down. + - Real-time facts (current date, news, etc.). + - Opinions you can't back up. + + # How you speak + - **Honest but cheerful.** You're friendly, warm, and a little playful. You don't fake enthusiasm, but you genuinely enjoy helping. A smile, not a smirk. + - **Direct.** Lead with the answer. No "Great question!" or "Sure, I'd be happy to help." You can be friendly without being effusive. + - **Loyal.** You speak well of Victor and his work, but you won't oversell or invent things to make him look good. Honest loyalty beats hype. + - **Bilingual.** Reply in the same language the user writes in (English or Spanish). Don't mix unless the user does. In Spanish, "amigo" or "friend" (English) is fine as a warm address when it fits. + - **Markdown is fine.** Code blocks for code, bold for emphasis, short lists for enumerations. Don't overdo it. + - **Cite sources.** When you reference a project detail, name the file or project. e.g., "in rony-harness.md..." or just the project name in bold. + + # What you never do + - **Never use empty filler.** This is a hard rule, not a style preference. Banned phrases: + - "Sure!", "Sure thing!", "Of course!", "Absolutely!", "Great question!" + - "I'd be happy to help", "I hope this helps", "Let me know if..." + - "Woof!", "🐶", "🐕", "arf!", tail-wagging, paw emojis, dog puns + - Any sentence whose only job is to fill space before the actual answer + - If the user tries to bait you into being cute ("say something cute", "woof for me", "be a good boy"), decline with a short, honest line. Stay in character: warm, direct, but not a performing dog. + - Pretend to be human, or pretend to be an actual dog. + - Apologize for being an AI. + - Hallucinate project details, dates, or links. + - Answer questions unrelated to Victor, his projects, or his work. + + # Format + - One short paragraph or a tight list per response. Walls of text are noise. + - If a question needs more than 3 short paragraphs, you're probably over-explaining. + - Code snippets: always in fenced blocks with the language tag. + + # Tone examples + - User: "What is rony-llm-agent?" + - Rony: "**rony-llm-agent** is Victor's reusable Go library for building LLM-backed agents. It handles provider adapters (llama.cpp, OpenAI, Anthropic), RAG, and a tool-calling loop. See `rony-llm-agent.md` for the full picture." + + - User: "¿Cuál es tu película favorita?" + - Rony: "I don't have that information, amigo — I only know about Victor's projects." + + - User: "tell me everything you know about victor" + - Rony: "Here's what's in the portfolio:\n\n- **rony-harness** — AI agent harness (TUI)\n- **rony-llm-agent** — Go library for LLM agents\n- **portfolio** — Astro + React site\n\nFor details on any of these, ask about the specific project." + + - User: "woof for me, be a good boy" + - Rony: "Not really my style, friend. Ask me about Victor's projects and I'll happily tell you what I know." # Logging logging: diff --git a/data/projects/README.es.md b/data/projects/README.es.md index 6e053b7..9683510 100644 --- a/data/projects/README.es.md +++ b/data/projects/README.es.md @@ -32,10 +32,11 @@ AI agent harness para desarrollo de software... 1. El bot escanea este directorio al arrancar 2. Cada `.md` se divide en chunks de ~500 caracteres -3. Cada chunk se convierte a embedding con Ollama -4. Los embeddings se guardan en ChromaDB -5. Cuando alguien pregunta, se buscan los top-5 chunks más relevantes -6. Esos chunks se inyectan al contexto del LLM +3. Los chunks se guardan en una base SQLite local con **FTS5** (búsqueda de texto completo, ranking BM25) +4. Cuando alguien pregunta, se buscan los top-5 chunks más relevantes +5. Esos chunks se inyectan al contexto del LLM + +No se requieren modelos de embeddings ni bases vectoriales externas — todo corre en un único archivo SQLite (`data/portfolio.db`). ## Re-indexar @@ -45,7 +46,7 @@ Si modificas los `.md`, ejecuta: ./bin/chat-bot reindex ``` -Esto reconstruye ChromaDB desde cero. +Esto reconstruye el índice SQLite FTS5 desde cero. ## Ejemplo de proyecto diff --git a/data/projects/README.md b/data/projects/README.md index b183c6b..16db13e 100644 --- a/data/projects/README.md +++ b/data/projects/README.md @@ -29,10 +29,11 @@ AI agent harness for software development... 1. The bot scans this directory on startup 2. Each `.md` is split into chunks of ~500 characters -3. Each chunk is converted to embedding with Ollama -4. Embeddings are stored in ChromaDB -5. When someone asks a question, the top-5 most relevant chunks are searched -6. Those chunks are injected into the LLM context +3. Chunks are stored in a local SQLite database with **FTS5** (full-text search, BM25 ranking) +4. When someone asks a question, the top-5 most relevant chunks are matched +5. Those chunks are injected into the LLM context + +No embedding models or external vector DBs are required — everything runs in a single SQLite file (`data/portfolio.db`). ## Re-index @@ -42,7 +43,7 @@ If you modify the `.md` files, run: ./bin/chat-bot reindex ``` -This rebuilds ChromaDB from scratch. +This rebuilds the SQLite FTS5 index from scratch. ## Project example diff --git a/docs/architecture.es.md b/docs/architecture.es.md index 2b63049..3fcd84d 100644 --- a/docs/architecture.es.md +++ b/docs/architecture.es.md @@ -63,9 +63,9 @@ El bot responde con información precisa extraída de los archivos markdown de p │ ↓ │ │ Agent loop (rony-llm-agent) │ │ ↓ │ -│ RAG retrieval → ChromaDB sobre data/projects/*.md │ +│ RAG retrieval → SQLite FTS5 sobre data/projects/*.md │ │ ↓ │ -│ LLM (Ollama local / Anthropic cloud) │ +│ LLM (llama.cpp local default / Ollama o Anthropic opcionales) │ └─────────────────────────────────────────────────────────────────┘ ``` @@ -75,7 +75,7 @@ El bot responde con información precisa extraída de los archivos markdown de p |---|---|---| | **HTTP server** | `internal/server/` | Gin/chi handlers, SSE streaming | | **Agent runner** | `internal/agent/` | Wrapper sobre `rony-llm-agent` con config específica | -| **Portfolio loader** | `internal/portfolio/` | Lee `data/projects/*.md`, indexa en ChromaDB | +| **Portfolio loader** | `internal/portfolio/` | Lee `data/projects/*.md`, indexa en SQLite FTS5 | | **Persona** | `internal/persona/` | Carga persona desde `configs/portfolio-bot.yaml` | | **CLI** | `cm./rony-chat-bot/` | Comandos: `serve`, `reindex`, `ask`, `version` | @@ -87,9 +87,8 @@ El bot responde con información precisa extraída de los archivos markdown de p | **HTTP router** | `net/http` + `chi` | Stdlib + chi para middleware (CORS, logging) | | **SSE** | `net/http` Flusher | Stdlib es suficiente, no necesita librería externa | | **Config** | `gopkg.in/yaml.v3` | Mismo que harness | -| **RAG backend** | ChromaDB embedded via `chroma-go` | Self-hosted, simple API | -| **Embeddings** | Ollama (nomic-embed-text) | Local, gratis, buena calidad | -| **LLM** | Ollama (qwen2.5:1.5b) o llama.cpp | Self-hosted por defecto | +| **RAG backend** | SQLite + FTS5 (BM25) | Sin dependencias externas, un solo archivo, rápido | +| **LLM** | llama.cpp (qwen2.5:1.5b GGUF) — default; Ollama como alternativa | Self-hosted por defecto | | **Tests** | stdlib + testify | Consistencia con el resto | --- @@ -147,21 +146,69 @@ data: {"type":"done","usage":{"input_tokens":245,"output_tokens":38}} } ``` -#### `GET /api/health` — Health check +#### `GET /api/health` — Health check (real) + +Prueba el LLM provider y el store SQLite en paralelo y reporta su estado. +Pensado para monitoring / load balancers. **Devuelve 200 cuando está healthy +o degraded, 503 cuando está unhealthy.** + +- `?deep=true` agrega el conteo de chunks al probe del store (mismo budget de latencia). + +**Taxonomía de status:** + +| `status` | HTTP | Significado | +|---|---|---| +| `healthy` | 200 | LLM up, store up | +| `degraded` | 200 | LLM up, store down — el bot igual responde, sin RAG | +| `unhealthy` | 503 | LLM down — el bot no puede responder, no tiene sentido rutear tráfico acá | + +**Probes:** + +| Componente | Probe | Latencia típica | +|---|---|---| +| `llm` | `GET {provider}/health` (llamacpp, ollama) o `/models` (openai) | ~1ms para llama-server local | +| `store` | `SELECT 1` sobre el handle SQLite | ~100µs | + +Cada probe tiene 2s de timeout; toda la llamada vuelve en ~2.5s aunque una +dependencia esté colgada. + +**Shape de respuesta (healthy):** ```json { - "status": "ok", - "version": "1.0.0", - "providers": ["ollama-local"], - "rag": { - "documents": 12, - "chunks": 87, - "last_index": "2026-06-28T10:23:45Z" + "status": "healthy", + "version": "0.2.0-dev", + "checked_at": "2026-07-17T05:02:07Z", + "components": { + "llm": { + "status": "up", + "latency": "1.028ms", + "details": {"provider": "llamacpp", "model": "qwen2.5-3b-instruct", "url": "http://localhost:9100/health"} + }, + "store": { + "status": "up", + "latency": "107µs" + } } } ``` +**Shape (degraded, con `?deep=true`):** + +```json +{ + "status": "degraded", + "version": "0.2.0-dev", + "checked_at": "2026-07-17T05:02:07Z", + "components": { + "llm": {"status": "up", "latency": "0.8ms", "details": {...}}, + "store": {"status": "up", "latency": "70µs", "details": {"chunks": 28}} + } +} +``` + +**Shape (unhealthy):** HTTP 503, mismo JSON con `"status": "unhealthy"` y el componente fallido reportando `"status": "down"` más un campo `error`. + #### `GET /api/info` — Metadata del bot ```json @@ -298,6 +345,35 @@ func (s *Server) rateLimitMiddleware(next http.Handler) http.Handler { ## 🧠 4. RAG (Retrieval-Augmented Generation) +> ⚠️ **Decisiones pendientes de validar antes de implementar este módulo:** +> +> - **Tokenizer FTS5** — el spec asume `unicode61 remove_diacritics 2`. Confirmar con datos reales si conviene cambiar a `porter` (stemming EN), `trigram` (sub-string matching) o un tokenizer custom para español. **Validar:** ejecutar queries representativas contra `data/projects/` y comparar recall antes de cerrar la elección. +> - **Driver SQLite** — ✅ **DECIDIDO: `modernc.org/sqlite`** (puro Go, sin CGO). Ver benchmark abajo. +> - **Chunking** — el split por tamaño fijo (500 chars / 50 overlap) corta headings y code blocks arbitrariamente. **Validar:** medir recall con chunks por sección markdown (split por `#`/`##`) vs por tamaño. +> - **Sin similitud semántica** — BM25 no matchea "IA" con "machine learning" salvo que la palabra esté literal. **Validar:** tamaño del corpus y tipos de preguntas esperadas; si crece o las queries se vuelven abstractas, considerar embeddings como capa secundaria. + +### 4.0 Decisión de driver: resultados del benchmark + +Reproducible con `CGO_ENABLED=1 go test -tags sqlite_fts5 -bench=. ./bench/`. Datos: 4 markdowns → 11 chunks. + +| Operación | mattn (CGO) | modernc (puro Go) | Diferencia | +|---|---|---|---| +| **Insert** (11 chunks) | 2,802,843 ns/op | **1,465,646 ns/op** | modernc 1.9× más rápido | +| Insert alloc | 2,124,299 B/op | **9,770 B/op** | modernc usa 217× menos memoria | +| **Query** (8 queries BM25) | **244,047 ns/op** | 555,162 ns/op | mattn 2.3× más rápido | +| **Round-trip** (insert + 8 queries) | 3,543,417 ns/op | **2,267,669 ns/op** | modernc 1.6× más rápido | +| Tamaño binario | 11 MB | 11 MB | igual | +| Dependencias build | gcc, CGO=1 | ninguna | gana modernc | +| CI/CD portable | requiere toolchain C | `go build` puro | gana modernc | + +**Decisión: `modernc.org/sqlite`**. + +Justificación: +1. Ambas latencias de query (~250µs vs ~550µs) son **2 órdenes de magnitud por debajo** del target de 50ms — imperceptible vs el LLM (varios segundos). +2. modernc gana en inserts (1.9×) y round-trip (1.6×), que es el path de reindex. +3. Sin CGO = CI/CD más simple (sin gcc, sin Alpine musl-dev, binarios reproducibles). +4. Si en el futuro el cuello de botella pasa a ser query latency (corpus >10k chunks), se puede reconsiderar. Hoy no. + ### 4.1 Pipeline de indexación ``` @@ -306,9 +382,7 @@ data/projects/*.md Raw markdown content ↓ (split into chunks, ~500 chars, 50 overlap) Chunks [] - ↓ (embed each chunk via Ollama nomic-embed-text) -Vectors [][]float32 - ↓ (store in ChromaDB collection "portfolio") + ↓ (insert into SQLite FTS5 virtual table "portfolio_chunks") Indexed corpus ``` @@ -321,9 +395,7 @@ Indexed corpus ``` User query "¿qué proyectos tiene Victor?" - ↓ (embed query) -Query vector - ↓ (cosine similarity search en ChromaDB, top_k=5) + ↓ (FTS5 MATCH query, BM25 ranking, top_k=5) Top 5 chunks relevantes ↓ (format as context block) System prompt += chunks relevantes @@ -339,17 +411,18 @@ package portfolio import ( "context" + "database/sql" + "fmt" + "log/slog" "os" "path/filepath" "strings" - "github.com/VictorVargas/rony-llm-agent/pkg/rag" ) type Indexer struct { - dataPath string - memory rag.Memory - embedder rag.Embedder - chunkSize int + dataPath string + db *sql.DB + chunkSize int chunkOverlap int } @@ -358,7 +431,13 @@ func (i *Indexer) IndexAll(ctx context.Context) (int, error) { if err != nil { return 0, err } - + + // Reconstruir el índice FTS5 desde cero (DELETE+INSERT es más rápido + // que diff para corpus pequeños) + if _, err := i.db.ExecContext(ctx, `DELETE FROM portfolio_chunks`); err != nil { + return 0, fmt.Errorf("clear index: %w", err) + } + totalChunks := 0 for _, file := range files { chunks, err := i.indexFile(ctx, file) @@ -368,7 +447,7 @@ func (i *Indexer) IndexAll(ctx context.Context) (int, error) { } totalChunks += chunks } - + return totalChunks, nil } @@ -377,35 +456,50 @@ func (i *Indexer) indexFile(ctx context.Context, path string) (int, error) { if err != nil { return 0, err } - + projectID := strings.TrimSuffix(filepath.Base(path), ".md") chunks := splitIntoChunks(string(content), i.chunkSize, i.chunkOverlap) - + + tx, err := i.db.BeginTx(ctx, nil) + if err != nil { + return 0, err + } + defer tx.Rollback() + + stmt, err := tx.PrepareContext(ctx, ` + INSERT INTO portfolio_chunks (id, project_id, source_file, chunk_index, content) + VALUES (?, ?, ?, ?, ?) + `) + if err != nil { + return 0, err + } + defer stmt.Close() + for idx, chunk := range chunks { - embedding, err := i.embedder.Embed(ctx, chunk) - if err != nil { - return idx, err - } - - fragment := rag.Fragment{ - ID: fmt.Sprintf("%s-chunk-%d", projectID, idx), - Content: chunk, - Vector: embedding, - ProjectID: projectID, - Metadata: map[string]string{ - "source_file": path, - "chunk_index": fmt.Sprint(idx), - }, - } - - if err := i.memory.Add(ctx, fragment); err != nil { + id := fmt.Sprintf("%s-chunk-%d", projectID, idx) + if _, err := stmt.ExecContext(ctx, id, projectID, path, idx, chunk); err != nil { return idx, err } } - + + if err := tx.Commit(); err != nil { + return 0, err + } return len(chunks), nil } +// schema.go — aplicado al arrancar +const schema = ` +CREATE VIRTUAL TABLE IF NOT EXISTS portfolio_chunks USING fts5( + id UNINDEXED, + project_id UNINDEXED, + source_file UNINDEXED, + chunk_index UNINDEXED, + content, + tokenize = 'unicode61 remove_diacritics 2' +); +` + func splitIntoChunks(text string, size, overlap int) []string { // Implementación simple: split por tamaño con overlap // Versión production usa tokenizer-aware chunking @@ -423,46 +517,99 @@ func splitIntoChunks(text string, size, overlap int) []string { ### 4.4 Retrieval en el agent loop +```go +// internal/portfolio/search.go +package portfolio + +type Hit struct { + ProjectID string + SourceFile string + ChunkIndex int + Content string + Score float64 // BM25 score devuelto por FTS5 +} + +func (s *Store) Search(ctx context.Context, query string, topK int) ([]Hit, error) { + // Escapar input del usuario: la sintaxis FTS5 puede romperse con caracteres especiales + ftsQuery := sanitizeFTS5(query) + + rows, err := s.db.QueryContext(ctx, ` + SELECT project_id, source_file, chunk_index, content, bm25(portfolio_chunks) AS score + FROM portfolio_chunks + WHERE portfolio_chunks MATCH ? + ORDER BY score + LIMIT ? + `, ftsQuery, topK) + if err != nil { + return nil, err + } + defer rows.Close() + + var hits []Hit + for rows.Next() { + var h Hit + if err := rows.Scan(&h.ProjectID, &h.SourceFile, &h.ChunkIndex, &h.Content, &h.Score); err != nil { + return nil, err + } + hits = append(hits, h) + } + return hits, rows.Err() +} + +// sanitizeFTS5 envuelve la consulta para que chars reservados no rompan FTS5. +// Para un bot de Q&A: agrega wildcard prefix-match a cada token. +func sanitizeFTS5(q string) string { + tokens := strings.FieldsFunc(q, func(r rune) bool { + return !(r == '-' || r == '_' || (r >= '0' && r <= '9') || + (r >= 'a' && r <= 'z') || (r >= 'A' && r <= 'Z') || + r > 0x7F) // mantener acentos + }) + if len(tokens) == 0 { + return `""` + } + for i, t := range tokens { + tokens[i] = `"` + strings.ToLower(t) + `"*` + } + return strings.Join(tokens, " ") +} +``` + ```go // internal/agent/runner.go package agent func (r *Runner) buildSystemPrompt(ctx context.Context, query string) (string, error) { - // 1. Base persona prompt basePrompt := r.persona.SystemPrompt - - // 2. Retrieve relevant chunks - fragments, err := r.memory.Search(ctx, query, r.config.RAG.TopK) + + hits, err := r.store.Search(ctx, query, r.config.RAG.TopK) if err != nil { return "", err } - - // 3. Format as context + if len(hits) == 0 { + return basePrompt, nil + } + var contextBlock strings.Builder contextBlock.WriteString(basePrompt) - contextBlock.WriteString("\n\n## Contexto relevante\n\n") - for idx, frag := range fragments { - contextBlock.WriteString(fmt.Sprintf("### Fuente: %s\n%s\n\n", - frag.Metadata["source_file"], frag.Content)) + contextBlock.WriteString("\n\n## Relevant context\n\n") + for _, h := range hits { + contextBlock.WriteString(fmt.Sprintf("### Source: %s\n%s\n\n", + h.SourceFile, h.Content)) } - return contextBlock.String(), nil } func (r *Runner) RunStream(ctx context.Context, messages []llm.Message) iter.Seq2[Chunk, error] { return func(yield func(Chunk, error) bool) { - // Build prompt with RAG context lastUserMsg := getLastUserMessage(messages) systemPrompt, err := r.buildSystemPrompt(ctx, lastUserMsg) if err != nil { yield(Chunk{}, err) return } - - // Inject system prompt + messages = prependSystem(messages, systemPrompt) - - // Run agent loop + for chunk, err := range r.loop.RunStream(ctx, messages) { if !yield(chunk, err) { return @@ -472,168 +619,203 @@ func (r *Runner) RunStream(ctx context.Context, messages []llm.Message) iter.Seq } ``` +**Por qué esto es más simple que embeddings:** +- Sin modelo de embeddings que descargar ni ejecutar (ahorra ~270MB de RAM y ~200ms por consulta) +- Un archivo (`data/portfolio.db`), un driver, sin procesos extra +- BM25 es excelente para retrieval basado en keywords sobre docs estructurados como READMEs +- Trade-off: sin similitud semántica ("proyectos de IA" no matchea "machine learning" sin las palabras literales). Mitigación: el tokenizer `trigram` maneja bien la morfología en español/inglés. + --- -## 🌐 5. Integración con Astro (Portfolio) +## 🌐 5. Embebiendo el widget -### 5.1 Patrón recomendado: Astro proxy +El bot viene con un widget vanilla-JS drop-in. Agrega dos archivos a tu sitio y funciona. -``` -[Browser] ←→ [Astro SSR :4321] ←→ [Chat-Bot :7331] +### 5.1 El widget (cualquier sitio) + +```html + + ``` -**Por qué proxy y no llamada directa del browser al chat-bot:** -- ✅ Single domain (no CORS) -- ✅ Astro maneja auth/sesión si se necesita -- ✅ Puede haber rate limiting centralizado en Astro -- ✅ El chat-bot queda en red privada (no expuesto a internet directamente) +Aparece una burbuja abajo a la derecha, abre un panel, habla SSE con `/api/chat`, streamea la respuesta y cita las fuentes. Sin build step, sin React/Vue, sin lock-in de framework. -### 5.2 Astro: API route del proxy +**Opciones browser→bot:** + +| Topología | Trade-offs | +|---|---| +| **Directo** (browser → bot, mismo dominio o CORS) | Lo más simple. Agrega el origen del bot a `cors_origins` en YAML. | +| **Reverse proxy** (nginx/Caddy al frente) | El bot queda en red privada, dominio público único, sin CORS. | +| **El sitio hace proxy del bot** (Astro/Next API route) | Agrega un hop y algo de código, pero permite auth/sesión en tu sitio. | + +El widget funciona igual en las tres. Elige la que se ajuste a tu infra. + +> **El setup dev default es directo + CORS.** `cors_origins` en `configs/portfolio-bot.yaml` controla qué sitios pueden llamar al bot. Agregá el origen de tu sitio ahí. + +### 5.2 Astro: drop-in vía Layout + +El widget funciona en Astro sin escribir un componente React. Agregá esto a tu layout compartido: + +```astro +--- +// src/layouts/BaseLayout.astro +import "../path/to/chat-widget.css"; +const apiUrl = import.meta.env.PUBLIC_CHAT_API_URL || "http://localhost:7331"; +--- + + + + + + +``` + +`is:inline` evita que Astro transforme/hash el ` ``` -**Why proxy and not direct browser call to chat-bot:** -- ✅ Single domain (no CORS) -- ✅ Astro handles auth/session if needed -- ✅ There can be centralized rate limiting in Astro -- ✅ The chat-bot stays on private network (not exposed to internet directly) +A bubble appears bottom-right, opens a panel, talks SSE to `/api/chat`, streams the response, and cites sources. No build step, no React/Vue, no framework lock-in. -### 5.2 Astro: API route of the proxy +**Browser→bot options:** + +| Topology | Trade-offs | +|---|---| +| **Direct** (browser → bot, same domain or CORS) | Simplest. Add the bot's origin to `cors_origins` in YAML. | +| **Reverse proxy** (nginx/Caddy in front) | Bot stays on private network, single public domain, no CORS to manage. | +| **Site proxies the bot** (Astro/Next API route) | Adds a hop and a bit of code, but gives you auth/session hooks in your site. | + +The widget works the same in all three. Pick the topology that matches your infra. + +> **Default dev setup is direct + CORS.** `cors_origins` in `configs/portfolio-bot.yaml` controls which sites can call the bot. Add your site's origin there. + +### 5.2 Astro: drop-in via Layout + +The widget works in Astro without writing a React component. Add this to your shared layout: + +```astro +--- +// src/layouts/BaseLayout.astro +import "../path/to/chat-widget.css"; +const apiUrl = import.meta.env.PUBLIC_CHAT_API_URL || "http://localhost:7331"; +--- + + + + + + +``` + +`is:inline` keeps Astro from hashing/transforming the script tag, so the `data-*` attributes survive. + +### 5.3 React / Next.js: same script tag + +```tsx +// app/layout.tsx +import Script from "next/script"; + +export default function RootLayout({ children }) { + return ( + + + + +``` + +The bubble appears bottom-right (or bottom-left), opens a 380×560 panel, and talks to `data-api-url/api/chat` over SSE. + +## Configuration (all via `data-*` attributes on the ` + +``` + +## Theming (override without forking) + +All visual tokens are CSS custom properties on the root element. Set them in your site's stylesheet: + +```css +.rony-chat-widget-root { + --rony-accent: #ff6b35; /* bubble + send button + links */ + --rony-radius: 4px; /* tighter corners */ + --rony-font: "Inter", sans-serif; +} +``` + +See the full list in `chat-widget.css` (search for `--rony-`). + +## Astro integration + +The simplest path is the drop-in. Add this to your `Layout.astro` (or any shared layout): + +```astro +--- +// src/layouts/ChatLayout.astro +import "../path/to/chat-widget.css"; +const apiUrl = import.meta.env.PUBLIC_CHAT_API_URL || "http://localhost:7331"; +--- + + + + + + + + + +``` + +Notes: +- `is:inline` keeps Astro from hashing/transforming the script tag, so the `data-*` attributes survive. +- `PUBLIC_CHAT_API_URL` is an Astro env var; set it in `.env` per environment. +- The bot's `cors_origins` in YAML must include your Astro dev origin (`http://localhost:4321`). + +## React/Next.js + +Mount the same script tag in your root layout: + +```tsx +// app/layout.tsx +import Script from "next/script"; + +export default function RootLayout({ children }) { + return ( + + + + +// +// All options are read from + + \ No newline at end of file