feat: bootstrap rony-chat-bot Go module
Initial implementation of the bot:
- cmd/chat-bot: CLI entrypoint (serve, reindex, ask, version)
- internal/agent: LLM provider client + agent runner with RAG injection
- internal/config: YAML config loader (providers, RAG, persona, server)
- internal/i18n: response-language detection (EN/ES)
- internal/persona: persona system prompt assembly from YAML
- internal/portfolio: heading-based chunker + SQLite FTS5 indexer
- internal/server: chi router with /api/chat (SSE), /api/health, /api/info,
/api/reindex, middleware (RequestID, Logging, CORS, RateLimit)
- internal/streaming: SSE protocol helpers (start, chunk, sources, done, error)
- web/: drop-in vanilla-JS chat widget (no build, no deps) + demo + README
- bench/: reproducible driver benchmark (modernc vs mattn SQLite)
- configs/portfolio-bot.yaml: llama.cpp default provider, SQLite RAG, canine persona
- docs/architecture.md / .es.md: aligned with SQLite FTS5 + llama.cpp decisions
- data/projects/README*.md: project data documentation
- README.md / .es.md: updated for current implementation
All tests pass (go test ./...). Bot is functional end-to-end with the
configured LLM provider.
2026-07-17 07:56:06 +00:00
|
|
|
// Package persona owns the bot's identity and the system prompt the LLM
|
|
|
|
|
// sees. The full prompt text lives in the YAML's `system_prompt` field
|
|
|
|
|
// (long, freeform, hand-tuned for the deployment). This package only
|
|
|
|
|
// adds the RAG context after it.
|
|
|
|
|
package persona
|
|
|
|
|
|
|
|
|
|
import (
|
|
|
|
|
"github.com/VictorVargas/rony-llm-agent/pkg/persona"
|
|
|
|
|
|
|
|
|
|
"github.com/VictorVargas/rony-chat-bot/internal/config"
|
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
// FromConfig returns a minimal persona used only for the UI greeting and
|
|
|
|
|
// the SSE event metadata. The system prompt itself comes from
|
|
|
|
|
// config.SystemPrompt, not from this struct — see BuildSystemPrompt.
|
|
|
|
|
func FromConfig(c *config.Config) (persona.Persona, error) {
|
|
|
|
|
lang := c.Persona.Language
|
|
|
|
|
if lang == "" {
|
|
|
|
|
lang = "the user's language"
|
|
|
|
|
}
|
|
|
|
|
return persona.Persona{
|
|
|
|
|
ID: "rony",
|
|
|
|
|
Name: c.Persona.Name,
|
|
|
|
|
Tone: c.Persona.Tone,
|
|
|
|
|
Language: lang,
|
|
|
|
|
}, nil
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
// BuildSystemPrompt returns the full prompt for one chat turn:
|
feat(rag): hybrid retrieval, reference documents, and vendor sampling
Answers were short, sometimes in the wrong language, and occasionally about
projects that do not exist. Measured on a 20-question battery against the real
corpus in both Spanish and English, this takes grounded content from 3/10 to
9/10 and language matching from 7/10 to 10/10.
Retrieval
- Fuse FTS5 keyword search with dense vectors via Reciprocal Rank Fusion.
Both halves are load-bearing: the corpus is English and visitors ask in
Spanish, so the meaningful words score zero. "paga" appears 0 times in a
document that says "Payments: Stripe" — the question "¿Con qué se paga en la
tienda de ropa?" retrieved nothing at all. Embeddings put all three of that
project's chunks on top. RRF ranks by agreement rather than comparing a BM25
score against a cosine, quantities with no shared scale.
- internal/embed: OpenAI-compatible embeddings client, unit-normalised so a
dot product is the cosine. Reorders by the response `index` field.
- Store a content hash beside each vector and skip rows where it no longer
matches the chunk. Chunk ids survive body edits, so without this an edited
document keeps serving embeddings that describe text that is gone —
reproduced live by changing a payment provider and watching the old one keep
coming back.
- Degrade to keyword-only when the embedder is down instead of failing.
Reference documents that are not projects
- Index `.mdx` alongside `.md`, and split sources into projects (announced in
the catalogue) and reference material (retrievable, never listed). A CV is
what someone deciding whether to hire actually reads, and it was unreachable
while it lived only in the Astro site — but filing it under projects made
the bot list "cv" as one of Victor's works.
- Skip each directory's README. `data/projects/README.md` was being indexed,
so the catalogue injected into every prompt announced "README" and
"README.es" as projects of Victor's.
- Exclude frontmatter from retrieval. It is dense metadata in a very short
chunk, which makes it a magnet for short queries: a CV's `location:` field
answered "¿Dónde ha trabajado Victor?" with a city instead of a work history.
- Split oversized sections at `###` before falling back to byte offsets. A CV's
Experience section is a list of jobs, and size-splitting cut one mid-word,
stranding the employer's name in the previous chunk.
Prompt and sampling
- Inject the full project catalogue every turn. Top-K search returns the best
matching sections, so "list every project" cannot be answered from retrieval
alone, and a small model asked to enumerate from partial hits invents the
rest. ~10 tokens per project; this is what stopped the invented names.
- Wire the sampling parameters the model authors publish (top_k, top_p, min_p,
repeat_penalty, presence_penalty) through config to llama.cpp. Leaving them
at llama.cpp's defaults produced 16-token stub answers.
- Localised system prompt selected by detected language. The English prompt
plus "reply in the user's language" answered 1/5 Spanish questions in
Spanish; few-shot examples fixed the language but got copied verbatim into
real answers.
- Fold compaction's system notes into the leading system message. Gemma's chat
template rejects a system message that is not first, and the whole request
failed with HTTP 400 the moment compaction fired.
Configuration and docs
- context_size 4096, down from 8192. The largest prompt this bot ever built
over 20 real requests was 1255 tokens, compaction starts at ~3070, and the
cut saved 212 MB resident with zero truncations and identical throughput.
- Correct the RAM figures throughout. They were measured with a GPU absorbing
llama.cpp's buffers; on a GPU-less VPS those come out of system RAM, which
is 1.1 GB more for qwen2.5-3b and 2.8 GB more for granite. Both READMEs
still started gemma-3-1b while the config defaulted to qwen, and neither
started the embedder at all.
Measured on the 2-core, 8 GB CPU-only target: 3.64 GB LLM + 0.91 GB embedder
+ 0.02 GB bot, 21.0 tok/s steady state.
Known and unfixed, so they are not re-filed as new bugs: the model reads dates
out of the CV correctly but does the arithmetic on them wrong, and "¿Dónde ha
trabajado Victor?" still answers with projects rather than employers, though
"¿En qué empresas ha trabajado?" works.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-30 21:45:37 +00:00
|
|
|
// 1. The hand-written system prompt from the YAML (who Rony is, how to speak)
|
|
|
|
|
// 2. The project catalogue (omitted when empty)
|
|
|
|
|
// 3. The RAG block (omitted when the index returns no hits)
|
feat: bootstrap rony-chat-bot Go module
Initial implementation of the bot:
- cmd/chat-bot: CLI entrypoint (serve, reindex, ask, version)
- internal/agent: LLM provider client + agent runner with RAG injection
- internal/config: YAML config loader (providers, RAG, persona, server)
- internal/i18n: response-language detection (EN/ES)
- internal/persona: persona system prompt assembly from YAML
- internal/portfolio: heading-based chunker + SQLite FTS5 indexer
- internal/server: chi router with /api/chat (SSE), /api/health, /api/info,
/api/reindex, middleware (RequestID, Logging, CORS, RateLimit)
- internal/streaming: SSE protocol helpers (start, chunk, sources, done, error)
- web/: drop-in vanilla-JS chat widget (no build, no deps) + demo + README
- bench/: reproducible driver benchmark (modernc vs mattn SQLite)
- configs/portfolio-bot.yaml: llama.cpp default provider, SQLite RAG, canine persona
- docs/architecture.md / .es.md: aligned with SQLite FTS5 + llama.cpp decisions
- data/projects/README*.md: project data documentation
- README.md / .es.md: updated for current implementation
All tests pass (go test ./...). Bot is functional end-to-end with the
configured LLM provider.
2026-07-17 07:56:06 +00:00
|
|
|
//
|
feat(rag): hybrid retrieval, reference documents, and vendor sampling
Answers were short, sometimes in the wrong language, and occasionally about
projects that do not exist. Measured on a 20-question battery against the real
corpus in both Spanish and English, this takes grounded content from 3/10 to
9/10 and language matching from 7/10 to 10/10.
Retrieval
- Fuse FTS5 keyword search with dense vectors via Reciprocal Rank Fusion.
Both halves are load-bearing: the corpus is English and visitors ask in
Spanish, so the meaningful words score zero. "paga" appears 0 times in a
document that says "Payments: Stripe" — the question "¿Con qué se paga en la
tienda de ropa?" retrieved nothing at all. Embeddings put all three of that
project's chunks on top. RRF ranks by agreement rather than comparing a BM25
score against a cosine, quantities with no shared scale.
- internal/embed: OpenAI-compatible embeddings client, unit-normalised so a
dot product is the cosine. Reorders by the response `index` field.
- Store a content hash beside each vector and skip rows where it no longer
matches the chunk. Chunk ids survive body edits, so without this an edited
document keeps serving embeddings that describe text that is gone —
reproduced live by changing a payment provider and watching the old one keep
coming back.
- Degrade to keyword-only when the embedder is down instead of failing.
Reference documents that are not projects
- Index `.mdx` alongside `.md`, and split sources into projects (announced in
the catalogue) and reference material (retrievable, never listed). A CV is
what someone deciding whether to hire actually reads, and it was unreachable
while it lived only in the Astro site — but filing it under projects made
the bot list "cv" as one of Victor's works.
- Skip each directory's README. `data/projects/README.md` was being indexed,
so the catalogue injected into every prompt announced "README" and
"README.es" as projects of Victor's.
- Exclude frontmatter from retrieval. It is dense metadata in a very short
chunk, which makes it a magnet for short queries: a CV's `location:` field
answered "¿Dónde ha trabajado Victor?" with a city instead of a work history.
- Split oversized sections at `###` before falling back to byte offsets. A CV's
Experience section is a list of jobs, and size-splitting cut one mid-word,
stranding the employer's name in the previous chunk.
Prompt and sampling
- Inject the full project catalogue every turn. Top-K search returns the best
matching sections, so "list every project" cannot be answered from retrieval
alone, and a small model asked to enumerate from partial hits invents the
rest. ~10 tokens per project; this is what stopped the invented names.
- Wire the sampling parameters the model authors publish (top_k, top_p, min_p,
repeat_penalty, presence_penalty) through config to llama.cpp. Leaving them
at llama.cpp's defaults produced 16-token stub answers.
- Localised system prompt selected by detected language. The English prompt
plus "reply in the user's language" answered 1/5 Spanish questions in
Spanish; few-shot examples fixed the language but got copied verbatim into
real answers.
- Fold compaction's system notes into the leading system message. Gemma's chat
template rejects a system message that is not first, and the whole request
failed with HTTP 400 the moment compaction fired.
Configuration and docs
- context_size 4096, down from 8192. The largest prompt this bot ever built
over 20 real requests was 1255 tokens, compaction starts at ~3070, and the
cut saved 212 MB resident with zero truncations and identical throughput.
- Correct the RAM figures throughout. They were measured with a GPU absorbing
llama.cpp's buffers; on a GPU-less VPS those come out of system RAM, which
is 1.1 GB more for qwen2.5-3b and 2.8 GB more for granite. Both READMEs
still started gemma-3-1b while the config defaulted to qwen, and neither
started the embedder at all.
Measured on the 2-core, 8 GB CPU-only target: 3.64 GB LLM + 0.91 GB embedder
+ 0.02 GB bot, 21.0 tok/s steady state.
Known and unfixed, so they are not re-filed as new bugs: the model reads dates
out of the CV correctly but does the arithmetic on them wrong, and "¿Dónde ha
trabajado Victor?" still answers with projects rather than employers, though
"¿En qué empresas ha trabajado?" works.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-30 21:45:37 +00:00
|
|
|
// The catalogue and RAG block are appended, not prepended, so the persona
|
|
|
|
|
// instructions always come first and the LLM never gets the chance to
|
|
|
|
|
// "forget" them.
|
|
|
|
|
//
|
|
|
|
|
// The catalogue goes before the excerpts on purpose: it is the closed set the
|
|
|
|
|
// model is allowed to name, and stating the boundary before showing the
|
|
|
|
|
// evidence measurably cuts invented project names on small models.
|
|
|
|
|
func BuildSystemPrompt(systemPrompt, catalog, ragContext string) string {
|
feat: bootstrap rony-chat-bot Go module
Initial implementation of the bot:
- cmd/chat-bot: CLI entrypoint (serve, reindex, ask, version)
- internal/agent: LLM provider client + agent runner with RAG injection
- internal/config: YAML config loader (providers, RAG, persona, server)
- internal/i18n: response-language detection (EN/ES)
- internal/persona: persona system prompt assembly from YAML
- internal/portfolio: heading-based chunker + SQLite FTS5 indexer
- internal/server: chi router with /api/chat (SSE), /api/health, /api/info,
/api/reindex, middleware (RequestID, Logging, CORS, RateLimit)
- internal/streaming: SSE protocol helpers (start, chunk, sources, done, error)
- web/: drop-in vanilla-JS chat widget (no build, no deps) + demo + README
- bench/: reproducible driver benchmark (modernc vs mattn SQLite)
- configs/portfolio-bot.yaml: llama.cpp default provider, SQLite RAG, canine persona
- docs/architecture.md / .es.md: aligned with SQLite FTS5 + llama.cpp decisions
- data/projects/README*.md: project data documentation
- README.md / .es.md: updated for current implementation
All tests pass (go test ./...). Bot is functional end-to-end with the
configured LLM provider.
2026-07-17 07:56:06 +00:00
|
|
|
out := systemPrompt
|
feat(rag): hybrid retrieval, reference documents, and vendor sampling
Answers were short, sometimes in the wrong language, and occasionally about
projects that do not exist. Measured on a 20-question battery against the real
corpus in both Spanish and English, this takes grounded content from 3/10 to
9/10 and language matching from 7/10 to 10/10.
Retrieval
- Fuse FTS5 keyword search with dense vectors via Reciprocal Rank Fusion.
Both halves are load-bearing: the corpus is English and visitors ask in
Spanish, so the meaningful words score zero. "paga" appears 0 times in a
document that says "Payments: Stripe" — the question "¿Con qué se paga en la
tienda de ropa?" retrieved nothing at all. Embeddings put all three of that
project's chunks on top. RRF ranks by agreement rather than comparing a BM25
score against a cosine, quantities with no shared scale.
- internal/embed: OpenAI-compatible embeddings client, unit-normalised so a
dot product is the cosine. Reorders by the response `index` field.
- Store a content hash beside each vector and skip rows where it no longer
matches the chunk. Chunk ids survive body edits, so without this an edited
document keeps serving embeddings that describe text that is gone —
reproduced live by changing a payment provider and watching the old one keep
coming back.
- Degrade to keyword-only when the embedder is down instead of failing.
Reference documents that are not projects
- Index `.mdx` alongside `.md`, and split sources into projects (announced in
the catalogue) and reference material (retrievable, never listed). A CV is
what someone deciding whether to hire actually reads, and it was unreachable
while it lived only in the Astro site — but filing it under projects made
the bot list "cv" as one of Victor's works.
- Skip each directory's README. `data/projects/README.md` was being indexed,
so the catalogue injected into every prompt announced "README" and
"README.es" as projects of Victor's.
- Exclude frontmatter from retrieval. It is dense metadata in a very short
chunk, which makes it a magnet for short queries: a CV's `location:` field
answered "¿Dónde ha trabajado Victor?" with a city instead of a work history.
- Split oversized sections at `###` before falling back to byte offsets. A CV's
Experience section is a list of jobs, and size-splitting cut one mid-word,
stranding the employer's name in the previous chunk.
Prompt and sampling
- Inject the full project catalogue every turn. Top-K search returns the best
matching sections, so "list every project" cannot be answered from retrieval
alone, and a small model asked to enumerate from partial hits invents the
rest. ~10 tokens per project; this is what stopped the invented names.
- Wire the sampling parameters the model authors publish (top_k, top_p, min_p,
repeat_penalty, presence_penalty) through config to llama.cpp. Leaving them
at llama.cpp's defaults produced 16-token stub answers.
- Localised system prompt selected by detected language. The English prompt
plus "reply in the user's language" answered 1/5 Spanish questions in
Spanish; few-shot examples fixed the language but got copied verbatim into
real answers.
- Fold compaction's system notes into the leading system message. Gemma's chat
template rejects a system message that is not first, and the whole request
failed with HTTP 400 the moment compaction fired.
Configuration and docs
- context_size 4096, down from 8192. The largest prompt this bot ever built
over 20 real requests was 1255 tokens, compaction starts at ~3070, and the
cut saved 212 MB resident with zero truncations and identical throughput.
- Correct the RAM figures throughout. They were measured with a GPU absorbing
llama.cpp's buffers; on a GPU-less VPS those come out of system RAM, which
is 1.1 GB more for qwen2.5-3b and 2.8 GB more for granite. Both READMEs
still started gemma-3-1b while the config defaulted to qwen, and neither
started the embedder at all.
Measured on the 2-core, 8 GB CPU-only target: 3.64 GB LLM + 0.91 GB embedder
+ 0.02 GB bot, 21.0 tok/s steady state.
Known and unfixed, so they are not re-filed as new bugs: the model reads dates
out of the CV correctly but does the arithmetic on them wrong, and "¿Dónde ha
trabajado Victor?" still answers with projects rather than employers, though
"¿En qué empresas ha trabajado?" works.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-30 21:45:37 +00:00
|
|
|
if catalog != "" {
|
|
|
|
|
out += "\n\n## Victor's complete project catalogue\n\n" + catalog +
|
|
|
|
|
"\n\nThat is every project that exists. A name outside this list is not a real " +
|
|
|
|
|
"project of Victor's and you must never use one."
|
|
|
|
|
}
|
feat: bootstrap rony-chat-bot Go module
Initial implementation of the bot:
- cmd/chat-bot: CLI entrypoint (serve, reindex, ask, version)
- internal/agent: LLM provider client + agent runner with RAG injection
- internal/config: YAML config loader (providers, RAG, persona, server)
- internal/i18n: response-language detection (EN/ES)
- internal/persona: persona system prompt assembly from YAML
- internal/portfolio: heading-based chunker + SQLite FTS5 indexer
- internal/server: chi router with /api/chat (SSE), /api/health, /api/info,
/api/reindex, middleware (RequestID, Logging, CORS, RateLimit)
- internal/streaming: SSE protocol helpers (start, chunk, sources, done, error)
- web/: drop-in vanilla-JS chat widget (no build, no deps) + demo + README
- bench/: reproducible driver benchmark (modernc vs mattn SQLite)
- configs/portfolio-bot.yaml: llama.cpp default provider, SQLite RAG, canine persona
- docs/architecture.md / .es.md: aligned with SQLite FTS5 + llama.cpp decisions
- data/projects/README*.md: project data documentation
- README.md / .es.md: updated for current implementation
All tests pass (go test ./...). Bot is functional end-to-end with the
configured LLM provider.
2026-07-17 07:56:06 +00:00
|
|
|
if ragContext != "" {
|
|
|
|
|
out += "\n\n## Relevant context from the portfolio\n\n" +
|
|
|
|
|
"Use these excerpts to answer. Cite the project filename when you reference a detail. " +
|
|
|
|
|
"If the excerpts don't contain the answer, say you don't have that information — do not invent.\n\n" +
|
|
|
|
|
ragContext
|
|
|
|
|
}
|
|
|
|
|
return out
|
|
|
|
|
}
|