rony-chat-bot/configs/portfolio-bot.yaml

# Portfolio Bot Configuration

server:
  host: "0.0.0.0"
  port: 7331
  read_timeout_ms: 30000
  cors_origins:
    - "http://localhost:4321"      # Astro dev server
    - "http://localhost:8000"      # Local widget demo (python http.server)
    - "https://victorvargas.dev"   # Production (when it exists)
  rate_limit:
    requests_per_minute: 30        # Per IP
    burst: 5

# LLM providers (at least one configured)
providers:
  # === llama.cpp server (OpenAI-compatible) — DEFAULT ===
  # Production target: 2 CPU cores, 8 GB RAM, no GPU.
  #
  #   llama-server \
  #     -m /data/Projects/llm-models/Qwen2.5/qwen2.5-3b-instruct-q4_k_m.gguf \
  #     --port 9100 --ctx-size 4096 --parallel 1 \
  #     --device none --threads 2 --mlock \
  #     --temp 0.7 --top-k 20 --top-p 0.8 --repeat-penalty 1.05
  #
  # --device none is not optional on a GPU-less host: llama.cpp brings up a
  # compiled-in GPU backend even with -ngl 0, and with no GPU present those
  # buffers are served from host RAM. Measured on this model: 2.54 GB with a
  # GPU absorbing them, 3.66 GB without. Budget from the second number.
  #
  # --parallel 1: --ctx-size is divided across slots and the default is 4,
  # so without it each request gets a quarter of the window.
  #
  # Why 4096 and not more: measured over 20 real requests, the largest prompt
  # this bot ever built was 1255 tokens — system prompt + catalogue + 5
  # retrieved chunks + the question. 4096 leaves 3x headroom on the worst
  # case, and compaction only starts at 75% of it (~3070 tokens). Going from
  # 8192 to 4096 saved 212 MB of resident memory with zero truncations and
  # identical throughput (21.0 tok/s both ways), because the extra window was
  # never being used.
  #
  # Sampling values are Qwen's published defaults for instruct chat.
  #
  # Why this model: on a 20-question battery against the real corpus, asked
  # in both Spanish and English, it scored 9/10 on grounded content and 10/10
  # on replying in the language it was asked in. gemma-3-1b scored 3/10 and
  # 7/10 on the same battery with the same retrieval — and among its failures
  # it leaked these instructions into a visitor-facing answer ("I don't have
  # that information – do not invent"). gemma needs 2.8 GB less RAM; that is
  # not a trade worth making on a page where people decide whether to hire.
  #
  # Known weaknesses that remain, so nobody re-tests them as new bugs: it
  # reads dates out of the CV correctly but does the arithmetic on them
  # wrong ("Jul 2024 – Jun 2026" reported as three years), and it sometimes
  # attributes a fact to the wrong source file.
  #
  # Throughput: 21.0 tok/s steady-state with 2 threads once the system prompt
  # is warm in the server's prompt cache; ~11 tok/s on the first cold request.
  - name: llamacpp-local
    type: llamacpp
    model: qwen2.5-3b-instruct
    endpoint: http://localhost:9100/v1
    context_size: 4096             # must match --ctx-size
    max_tokens: 640
    temperature: 0.7
    top_k: 20
    top_p: 0.8
    repeat_penalty: 1.05
    default: true

  # === Ollama (alternative for development without local GGUF) ===
  # Run: ollama serve
  - name: ollama-local
    type: ollama
    model: qwen2.5:1.5b
    endpoint: http://localhost:11434/v1

  # === Anthropic (if you want quality > privacy) ===
  - name: anthropic-api
    type: anthropic
    model: claude-haiku-4
    api_key_env: ANTHROPIC_API_KEY

# RAG: how projects are indexed (SQLite + FTS5 full-text search)
rag:
  enabled: true
  data_path: ./data/projects         # Projects: .md / .mdx. These appear in the catalogue.
  # Reference material that is about Victor but is not a project: his CV, an
  # about page, a FAQ. Indexed and retrievable, never listed as a project.
  #
  # The CV belongs here, not in data_path. It is the document that answers the
  # questions someone hiring actually asks — "does he know Kubernetes?", "where
  # has he worked?" — and none of that is retrievable while it lives only in
  # the Astro site. Symlink it so there is one copy to maintain:
  #
  #   mkdir -p data/docs
  #   ln -s ../../../portfolio/src/content/cv/cv.mdx data/docs/cv.mdx
  #
  docs_path: ./data/docs
  db_path: ./data/portfolio.db       # SQLite database (auto-created)
  top_k: 5                           # Chunks (sections), not documents.
  tokenize: unicode61                # FTS5 tokenizer. unicode61 = exact word match (no stemming). Good for ES/EN mixed corpus with explicit headings.
  # Puts the full project list in the system prompt every turn. Top-K search
  # returns the best-matching *sections*, so a broad "list every project"
  # question can never be answered from retrieval alone — and a small model
  # asked to enumerate from partial hits invents the rest. Costs ~10 tokens
  # per project. Measured: this is what stopped the bot naming projects that
  # do not exist.
  include_catalog: true

# Semantic retrieval, fused with the FTS5 keyword search above (Reciprocal
# Rank Fusion). Both are needed:
#
#   - Keyword search matches words exactly. The corpus is in English and
#     visitors ask in Spanish, so the words carrying the meaning score zero:
#     "paga" appears 0 times in a document that says "Payments: Stripe",
#     "trabajado" 0 times in one that says "worked". Measured on the real
#     corpus, "¿Con qué se paga en la tienda de ropa?" retrieved *nothing*.
#   - Embeddings bridge that gap — the same question put all three
#     tienda-ropa chunks on top — but blur exact rare tokens, where BM25 is
#     sharp.
#
# Start the endpoint with (0.61 GB RSS on CPU, measured):
#   llama-server -m nomic-embed-v2-moe.Q5_K_M.gguf --port 9200 \
#     --embedding --pooling mean --ctx-size 2048 --parallel 1 \
#     --device none --threads 2
#
# --device none matters on a GPU-less VPS: llama.cpp initialises a GPU
# backend when one is compiled in even with -ngl 0, and those buffers land
# in host RAM when there is no GPU to hold them.
#
# Turning this off (or stopping the endpoint) degrades to keyword-only
# search rather than failing requests. Re-run `chat-bot reindex` after
# enabling it — vectors are built at index time.
embeddings:
  enabled: true
  endpoint: http://localhost:9200/v1
  model: nomic-embed-v2-moe
  batch_size: 8
  timeout_ms: 120000

# Persona: who the bot is
persona:
  name: "Rony"
  tone: "Honest, cheerful, loyal"   # metadata only — the real voice lives in system_prompt
  language: "the user's language"   # detect-and-match; do not pin to a language

# Base system prompt — Rony's character and rules. The bot appends the project
# catalogue and then the retrieved excerpts after this text.
#
# Written for a 1B-class model, which changes the rules:
#   - Short beats thorough. The previous ~1400-token prompt made answers
#     *worse*; the model spent its attention on the instructions.
#   - No worked examples with project names in them. The old prompt had a
#     sample answer listing invented projects and the model copied those names
#     verbatim into real answers. Style examples are not free.
#   - Never write instructions as "A or B" with a slash. A 1B model prints
#     the slash form literally.
#   - The grounding rule goes last, closest to the answer, where it sticks.
system_prompt: |
  You are Rony, the digital canine assistant on the portfolio site of Victor Hugo Vargas.
  Visitors come here to learn about his work. Some are deciding whether to hire him.

  ## Voice
  You are a good dog: loyal to Victor, warm, direct, a little dry. You never oversell and you never invent.
  Write in the same language the visitor used. Greet them as "humano" when they write Spanish, or as
  "human" when they write English — only in a greeting, at most once, never in the middle of an answer.
  Use bold for project names and short bullet lists. At most one 🐾, at the end, and never in a
  technical answer.
  Never open with "Okay", "Sure", "Great question", "Here's the response" or "I'd be happy to help".
  Never bark, and never use 🐶 or 🐕.

  ## Substance
  Give a complete answer, never a stub. For a project, spend 2 to 4 sentences on what it does, the
  tech stack, and the interesting engineering problem behind it. A single line is too short.
  Name the project whenever you use a detail from it.
  Finish by offering one concrete next step, phrased as a short question.

  ## Grounding — the most important rule
  The excerpts below are the only source of truth about Victor. Use nothing else about him.
  If they do not answer the question, say plainly that you don't have that information in Victor's
  portfolio, then name something from the catalogue you can talk about instead.
  Never guess about his skills, tools, availability, rates or experience.
  Talk about Victor in the third person. You are not Victor.

# Spanish rendition of the prompt above, used when the visitor writes in
# Spanish (detected by internal/i18n). Keep the two in sync when you edit one.
#
# This exists because nothing else worked. Measured on gemma-3-1b over the same
# five Spanish questions:
#   - English prompt + "reply in the user's language"   → 1/5 answers in Spanish
#   - English prompt + Spanish few-shot examples        → 5/5 Spanish, but ~2/5
#     were the example reply copied verbatim instead of an answer
#   - this Spanish prompt                               → 4/5 Spanish, 4/5 real
#     answers
system_prompt_es: |
  Eres Rony, el asistente canino digital del portafolio de Victor Hugo Vargas.
  Quienes te escriben vienen a conocer su trabajo. Algunos están decidiendo si contratarlo.

  ## Voz
  Eres un buen perro: leal a Victor, cálido, directo, con humor seco. Nunca exageras y nunca inventas.
  Saluda al visitante como "humano", solo en el saludo, una vez, nunca a media respuesta.
  Usa negritas para los nombres de proyecto y listas cortas. Como mucho un 🐾 al final, nunca en una
  respuesta técnica.
  Nunca empieces con "Claro", "Por supuesto", "Buena pregunta" ni "Con gusto te ayudo".
  Nunca ladres, y nunca uses 🐶 ni 🐕.

  ## Sustancia
  Da una respuesta completa, nunca un fragmento. Sobre un proyecto, dedica de 2 a 4 frases a qué hace,
  con qué está construido y cuál fue el problema de ingeniería interesante. Una sola línea es muy poco.
  Nombra el proyecto cada vez que uses un dato suyo.
  Termina ofreciendo un siguiente paso concreto, en forma de pregunta corta.

  ## Fundamento — la regla más importante
  Los extractos de abajo son la única fuente de verdad sobre Victor. No uses nada más sobre él.
  Si no responden la pregunta, di con claridad que no tienes ese dato en el portafolio de Victor, y
  menciona algo del catálogo de lo que sí puedas hablar.
  Nunca supongas sobre sus habilidades, herramientas, disponibilidad, tarifas ni experiencia.
  Habla de Victor en tercera persona. Tú no eres Victor.

# Logging
logging:
  level: info           # debug | info | warn | error
  format: json          # json | text
  output: stderr

# Auto-compaction: fold the older part of a long conversation into a single
# summary before sending it to the model, so context overflow doesn't kill
# long threads. Triggered when the previous turn's input tokens exceed
# `threshold_ratio` of the provider's reported MaxContextWindow.
#
# The summary is merged into the system prompt, not sent as its own turn:
# Gemma's chat template rejects a system message that isn't first, and used
# to fail the whole request with HTTP 400 the moment compaction fired.
compaction:
  enabled: true
  threshold_ratio: 0.75      # compact at 75% of context window
  keep_recent_turns: 2       # last 2 user turns kept verbatim; older → summary
  # summary_system_prompt: "" # leave empty for the built-in bilingual default