129809067b
Answers were short, sometimes in the wrong language, and occasionally about projects that do not exist. Measured on a 20-question battery against the real corpus in both Spanish and English, this takes grounded content from 3/10 to 9/10 and language matching from 7/10 to 10/10. Retrieval - Fuse FTS5 keyword search with dense vectors via Reciprocal Rank Fusion. Both halves are load-bearing: the corpus is English and visitors ask in Spanish, so the meaningful words score zero. "paga" appears 0 times in a document that says "Payments: Stripe" — the question "¿Con qué se paga en la tienda de ropa?" retrieved nothing at all. Embeddings put all three of that project's chunks on top. RRF ranks by agreement rather than comparing a BM25 score against a cosine, quantities with no shared scale. - internal/embed: OpenAI-compatible embeddings client, unit-normalised so a dot product is the cosine. Reorders by the response `index` field. - Store a content hash beside each vector and skip rows where it no longer matches the chunk. Chunk ids survive body edits, so without this an edited document keeps serving embeddings that describe text that is gone — reproduced live by changing a payment provider and watching the old one keep coming back. - Degrade to keyword-only when the embedder is down instead of failing. Reference documents that are not projects - Index `.mdx` alongside `.md`, and split sources into projects (announced in the catalogue) and reference material (retrievable, never listed). A CV is what someone deciding whether to hire actually reads, and it was unreachable while it lived only in the Astro site — but filing it under projects made the bot list "cv" as one of Victor's works. - Skip each directory's README. `data/projects/README.md` was being indexed, so the catalogue injected into every prompt announced "README" and "README.es" as projects of Victor's. - Exclude frontmatter from retrieval. It is dense metadata in a very short chunk, which makes it a magnet for short queries: a CV's `location:` field answered "¿Dónde ha trabajado Victor?" with a city instead of a work history. - Split oversized sections at `###` before falling back to byte offsets. A CV's Experience section is a list of jobs, and size-splitting cut one mid-word, stranding the employer's name in the previous chunk. Prompt and sampling - Inject the full project catalogue every turn. Top-K search returns the best matching sections, so "list every project" cannot be answered from retrieval alone, and a small model asked to enumerate from partial hits invents the rest. ~10 tokens per project; this is what stopped the invented names. - Wire the sampling parameters the model authors publish (top_k, top_p, min_p, repeat_penalty, presence_penalty) through config to llama.cpp. Leaving them at llama.cpp's defaults produced 16-token stub answers. - Localised system prompt selected by detected language. The English prompt plus "reply in the user's language" answered 1/5 Spanish questions in Spanish; few-shot examples fixed the language but got copied verbatim into real answers. - Fold compaction's system notes into the leading system message. Gemma's chat template rejects a system message that is not first, and the whole request failed with HTTP 400 the moment compaction fired. Configuration and docs - context_size 4096, down from 8192. The largest prompt this bot ever built over 20 real requests was 1255 tokens, compaction starts at ~3070, and the cut saved 212 MB resident with zero truncations and identical throughput. - Correct the RAM figures throughout. They were measured with a GPU absorbing llama.cpp's buffers; on a GPU-less VPS those come out of system RAM, which is 1.1 GB more for qwen2.5-3b and 2.8 GB more for granite. Both READMEs still started gemma-3-1b while the config defaulted to qwen, and neither started the embedder at all. Measured on the 2-core, 8 GB CPU-only target: 3.64 GB LLM + 0.91 GB embedder + 0.02 GB bot, 21.0 tok/s steady state. Known and unfixed, so they are not re-filed as new bugs: the model reads dates out of the CV correctly but does the arithmetic on them wrong, and "¿Dónde ha trabajado Victor?" still answers with projects rather than employers, though "¿En qué empresas ha trabajado?" works. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
52 lines
1.9 KiB
Markdown
52 lines
1.9 KiB
Markdown
# Reference Documents
|
|
|
|
Material that is **about you but is not a project**: your CV, an about page, a
|
|
FAQ, a talk abstract. Everything here is indexed and retrievable, and never
|
|
announced in the project catalogue.
|
|
|
|
Accepts `.md` and `.mdx`.
|
|
|
|
## Why this directory exists
|
|
|
|
`data/projects/` is advertised. The bot injects the full list of what lives
|
|
there into every prompt, so visitors get told those are your projects.
|
|
|
|
Your CV is the document that answers what someone considering hiring you
|
|
actually asks — "does he know Kubernetes?", "where has he worked?", "how long
|
|
was he at that job?" — and none of it is retrievable while it lives only in
|
|
your site. But it is not a project, and putting it in `data/projects/` makes
|
|
the bot cheerfully list "cv" as one of your works. Hence the split.
|
|
|
|
## The contents are gitignored
|
|
|
|
Only this README is tracked. What goes here is personal (a CV) or a symlink to
|
|
a path that only exists on your machine, and neither travels well in a repo.
|
|
Set it up on each install:
|
|
|
|
```bash
|
|
ln -s ../../../portfolio/src/content/cv/cv.mdx data/docs/cv.mdx
|
|
./bin/chat-bot reindex
|
|
```
|
|
|
|
A symlink rather than a copy so there is one file to keep current — edit the
|
|
CV in your site, re-run `reindex`, and the bot is up to date.
|
|
|
|
## Configuration
|
|
|
|
```yaml
|
|
rag:
|
|
data_path: ./data/projects # projects → listed in the catalogue
|
|
docs_path: ./data/docs # this directory → retrievable, never listed
|
|
```
|
|
|
|
Leave `docs_path` empty to turn the whole thing off.
|
|
|
|
## Notes
|
|
|
|
- Frontmatter is excluded from retrieval. It is dense metadata in a very short
|
|
chunk, which makes it a magnet for short queries — a CV's `location:` field
|
|
was answering *"where has Victor worked?"* with a city instead of his work
|
|
history.
|
|
- Long sections split at `###` headings, so each job in a CV's Experience
|
|
section stays one chunk instead of being cut mid-sentence.
|
|
- This README is skipped by the indexer, as is the one in `data/projects/`.
|