docs: document auto-compaction behavior in architecture guide
Adds the §4.5 Auto-compaction section to architecture.md and architecture.es.md describing the trigger, fallback, persistence and the new SSE 'compaction' event so consumers know how to react. Includes the three placeholder projects used while exercising the feature end to end (bot-onboarding, dashboard-metricas, tienda-ropa) so /api/reindex picks them up without further setup.
This commit is contained in:
parent
550ba526c4
commit
ab510d8b31
5 changed files with 192 additions and 0 deletions
34
data/projects/bot-onboarding.md
Normal file
34
data/projects/bot-onboarding.md
Normal file
|
|
@ -0,0 +1,34 @@
|
||||||
|
---
|
||||||
|
title: "Telegram Onboarding Bot"
|
||||||
|
date: 2024-04
|
||||||
|
status: "archived"
|
||||||
|
tags: ["Node.js", "PostgreSQL", "Telegram", "bot", "onboarding"]
|
||||||
|
repo: "https://github.com/example/bot-onboarding"
|
||||||
|
demo: ""
|
||||||
|
---
|
||||||
|
|
||||||
|
# Telegram Onboarding Bot
|
||||||
|
|
||||||
|
Conversational bot that walks new users through the initial setup of a B2B product.
|
||||||
|
|
||||||
|
## Description
|
||||||
|
|
||||||
|
A Telegram assistant that reduces the time-to-first-value of a B2B product. Replaces an 8-step web form with a guided conversation. Each user has a long-running state machine that persists across days.
|
||||||
|
|
||||||
|
## Tech stack
|
||||||
|
|
||||||
|
- **Runtime:** Node.js
|
||||||
|
- **Database:** PostgreSQL (state persistence per user)
|
||||||
|
- **Bot API:** Telegram Bot API with inline keyboards
|
||||||
|
- **State machine:** Hand-rolled, no FSM library
|
||||||
|
|
||||||
|
## Main features
|
||||||
|
|
||||||
|
1. Conversational onboarding replacing the legacy 8-step form
|
||||||
|
2. Long-running per-user state machine with PostgreSQL-backed persistence
|
||||||
|
3. Telegram inline keyboards keep interaction inside the chat app
|
||||||
|
4. Watchdog that re-sends the last message if the user has not responded in 24h
|
||||||
|
|
||||||
|
## Learnings
|
||||||
|
|
||||||
|
Handling Telegram timeouts without losing user state was the main challenge. The watchdog pattern with an explicit "give up" option turned out to be more reliable than trying to be too clever about timeout semantics. The state machine approach beat a more declarative framework for this use case because every state had bespoke transitions.
|
||||||
34
data/projects/dashboard-metricas.md
Normal file
34
data/projects/dashboard-metricas.md
Normal file
|
|
@ -0,0 +1,34 @@
|
||||||
|
---
|
||||||
|
title: "Real-Time Metrics Dashboard"
|
||||||
|
date: 2024-11
|
||||||
|
status: "live"
|
||||||
|
tags: ["React", "WebSockets", "D3.js", "Go", "dashboard", "metrics"]
|
||||||
|
repo: ""
|
||||||
|
demo: "https://example.com"
|
||||||
|
---
|
||||||
|
|
||||||
|
# Real-Time Metrics Dashboard
|
||||||
|
|
||||||
|
Platform metrics dashboard streamed over WebSockets with server-side aggregations.
|
||||||
|
|
||||||
|
## Description
|
||||||
|
|
||||||
|
An internal dashboard for monitoring KPIs of a SaaS platform in real time: active users, hourly revenue, 5xx errors, p95 latency. The frontend is React with D3 charts; the backend is Go opening a WebSocket stream to all connected dashboards.
|
||||||
|
|
||||||
|
## Tech stack
|
||||||
|
|
||||||
|
- **Frontend:** React, D3.js (custom charts, not Recharts)
|
||||||
|
- **Backend:** Go with gorilla-style WebSockets
|
||||||
|
- **Transport:** Server-Sent Events style streaming with 5s aggregations
|
||||||
|
- **Charts:** D3 for transitions between time windows (1h, 24h, 7d)
|
||||||
|
|
||||||
|
## Main features
|
||||||
|
|
||||||
|
1. Real-time KPI tiles (active users, hourly revenue, error rate, p95 latency)
|
||||||
|
2. Streaming updates via WebSocket with 5s server-side aggregation
|
||||||
|
3. Backpressure: when the browser tab is backgrounded, server reduces frequency to 1/min
|
||||||
|
4. Custom D3 charts with smooth transitions between time windows
|
||||||
|
|
||||||
|
## Learnings
|
||||||
|
|
||||||
|
Avoiding memory leaks when switching time windows was tricky. Implemented a ring buffer per series and explicit disposal of Resize observers on unmount. The backpressure mechanism was the most impactful change in production — it stopped a flood of needless work when users had many tabs open.
|
||||||
34
data/projects/tienda-ropa.md
Normal file
34
data/projects/tienda-ropa.md
Normal file
|
|
@ -0,0 +1,34 @@
|
||||||
|
---
|
||||||
|
title: "Online Clothing Store"
|
||||||
|
date: 2025-03
|
||||||
|
status: "live"
|
||||||
|
tags: ["Next.js", "Stripe", "PostgreSQL", "e-commerce"]
|
||||||
|
repo: "https://github.com/example/tienda-ropa"
|
||||||
|
demo: "https://example.com"
|
||||||
|
---
|
||||||
|
|
||||||
|
# Online Clothing Store
|
||||||
|
|
||||||
|
End-to-end e-commerce platform for an independent clothing brand. Catalogue, cart, checkout and an admin dashboard in a single product.
|
||||||
|
|
||||||
|
## Description
|
||||||
|
|
||||||
|
A full-stack e-commerce site built for an independent fashion brand. Customers browse a catalogue, build a cart, and pay through Stripe Checkout; the merchant manages stock, orders, and fulfilment from a custom admin dashboard.
|
||||||
|
|
||||||
|
## Tech stack
|
||||||
|
|
||||||
|
- **Framework:** Next.js (App Router)
|
||||||
|
- **Payments:** Stripe (hosted Checkout + webhooks)
|
||||||
|
- **Database:** PostgreSQL with Prisma ORM
|
||||||
|
- **Rendering:** Server-side for catalogue SEO, ISR with on-demand revalidation when stock changes
|
||||||
|
|
||||||
|
## Main features
|
||||||
|
|
||||||
|
1. Catalogue with per-product SEO-friendly routes
|
||||||
|
2. Stripe Checkout integration without PCI scope
|
||||||
|
3. Admin dashboard for stock and order management
|
||||||
|
4. Webhook-driven order confirmation and stock decrement
|
||||||
|
|
||||||
|
## Learnings
|
||||||
|
|
||||||
|
Keeping inventory in sync between the admin dashboard and Stripe without race conditions was the hardest part. Solved with serializable transactions and an optimistic lock per SKU. On the next iteration, an event-driven approach with an outbox table would be more robust under load.
|
||||||
|
|
@ -627,6 +627,51 @@ func (r *Runner) RunStream(ctx context.Context, messages []llm.Message) iter.Seq
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
## 🗜️ 4.5 Auto-compactación
|
||||||
|
|
||||||
|
Las conversaciones largas eventualmente agotan el contexto — con la ventana de 4k de qwen2.5-3b, el system prompt de ~3k tokens + el bloque RAG sólo deja espacio para 2–3 turnos del usuario. La auto-compactación resuelve esto plegando la parte más antigua de la conversación en un único mensaje-resumen del sistema cuando los tokens de entrada del turno anterior cruzan un umbral configurable.
|
||||||
|
|
||||||
|
### Cuándo se dispara
|
||||||
|
|
||||||
|
`agent.Runner.Compact` corre una vez por request a `/api/chat`, antes de la búsqueda RAG. Compara los `Usage.InputTokens` más recientes del runner (reportados por el provider en el chunk streameado previo) contra `client.Capabilities().MaxContextWindow × threshold_ratio`.
|
||||||
|
|
||||||
|
| Config | Default | Qué controla |
|
||||||
|
|---|---|---|
|
||||||
|
| `compaction.enabled` | `false` | Switch maestro. |
|
||||||
|
| `compaction.threshold_ratio` | `0.75` | Dispara cuando tokens usados ≥ ventana × ratio. |
|
||||||
|
| `compaction.keep_recent_turns` | `4` | Cuántos turnos recientes del usuario se preservan literales tras la compactación. |
|
||||||
|
| `compaction.summary_system_prompt` | *(bilingüe built-in)* | Override de la instrucción enviada al LLM al resumir. |
|
||||||
|
|
||||||
|
Sale silenciosamente cuando la compactación está deshabilitada, el provider no reporta ventana (`Capabilities().MaxContextWindow == 0`), la historia es más corta que `keep_recent_turns`, o el usage aún es desconocido (primer turno).
|
||||||
|
|
||||||
|
### Cómo se hace el resumen
|
||||||
|
|
||||||
|
1. `splitByTurns(history, keep_recent_turns)` divide los mensajes en `(older, recent)` cortando en límites de rol `user`, así el par user/assistant de un turno preservado queda siempre junto.
|
||||||
|
2. `renderTranscript(older)` aplana los mensajes antiguos en una transcripción `User:` / `Assistant:` (saltando mensajes tool y placeholders vacíos de assistant).
|
||||||
|
3. El runner llama a `client.Generate(...)` con el prompt de resumen + la transcripción y un cap de 512 tokens para que la compactación en sí misma sea barata.
|
||||||
|
4. El texto devuelto se antepone como mensaje de sistema (`"Earlier conversation summary:\n…"`), seguido por la cola reciente.
|
||||||
|
5. `LastCompaction()` devuelve `CompactionStats` para que el handler SSE emita el evento `compaction` justo antes de los chunks streameados.
|
||||||
|
|
||||||
|
### Modo de falla
|
||||||
|
|
||||||
|
Si `Generate` falla o devuelve un resumen vacío, la compactación cae a `truncateToBudget`: descarta turnos antiguos del usuario uno por uno hasta que el slice restante entre en `threshold` tokens (heurística: `len(s) / 4 + 1`). El turno actual del usuario siempre se preserva. El fallback se loggea a nivel WARN y el request sigue — un fallo del resumidor nunca rompe la request del usuario.
|
||||||
|
|
||||||
|
### Protocolo de cable
|
||||||
|
|
||||||
|
Las respuestas streameadas ganan un evento opcional `compaction`:
|
||||||
|
|
||||||
|
```
|
||||||
|
data: {"type":"compaction","older_turns":6,"kept_turns":2,"summary_tokens":120,"window_tokens":4096,"used_tokens":3500}
|
||||||
|
```
|
||||||
|
|
||||||
|
Se emite después del `start` (cuando aplica) y antes de `sources` / `chunk`. El widget puede renderizar esto como un hint sutil "Contexto compactado" o ignorarlo — ambas son válidas.
|
||||||
|
|
||||||
|
### Persistencia
|
||||||
|
|
||||||
|
La compactación es **por-request**. La transcripción completa igual se guarda en `messages` en `data/portfolio.db` literal, así que `GET /api/conversations/{id}` siempre devuelve la historia original. Sólo se reduce lo que se le manda al LLM — la próxima sesión puede releer el thread completo desde la DB.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
## 🌐 5. Embebiendo el widget
|
## 🌐 5. Embebiendo el widget
|
||||||
|
|
||||||
El bot viene con un widget vanilla-JS drop-in. Agrega dos archivos a tu sitio y funciona.
|
El bot viene con un widget vanilla-JS drop-in. Agrega dos archivos a tu sitio y funciona.
|
||||||
|
|
|
||||||
|
|
@ -747,6 +747,51 @@ func (r *Runner) RunStream(ctx context.Context, messages []llm.Message) iter.Seq
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
## 🗜️ 4.5 Auto-compaction
|
||||||
|
|
||||||
|
Long conversations eventually run out of context — at qwen2.5-3b's 4k window, the ~3k-token system prompt + RAG block leaves only room for 2–3 user turns. Auto-compaction solves this by folding the older portion of the conversation into a single summary system message when the previous turn's input tokens cross a configurable threshold.
|
||||||
|
|
||||||
|
### When it fires
|
||||||
|
|
||||||
|
`agent.Runner.Compact` runs once per `/api/chat` request, before the RAG search. It compares the runner's most recent `Usage.InputTokens` (reported by the provider in the previous streamed chunk) against `client.Capabilities().MaxContextWindow × threshold_ratio`.
|
||||||
|
|
||||||
|
| Setting | Default | What it controls |
|
||||||
|
|---|---|---|
|
||||||
|
| `compaction.enabled` | `false` | Master switch. |
|
||||||
|
| `compaction.threshold_ratio` | `0.75` | Trigger when used tokens ≥ window × ratio. |
|
||||||
|
| `compaction.keep_recent_turns` | `4` | How many of the latest user turns are kept verbatim after compaction. |
|
||||||
|
| `compaction.summary_system_prompt` | *(built-in bilingual)* | Override the instruction sent to the LLM when summarizing. |
|
||||||
|
|
||||||
|
Short-circuits silently when compaction is disabled, the provider doesn't report a window (`Capabilities().MaxContextWindow == 0`), the history is shorter than `keep_recent_turns`, or usage is still unknown (first turn).
|
||||||
|
|
||||||
|
### How the summary is made
|
||||||
|
|
||||||
|
1. `splitByTurns(history, keep_recent_turns)` divides messages into `(older, recent)` on user-role boundaries so a kept turn's user/assistant pair always stays together.
|
||||||
|
2. `renderTranscript(older)` flattens older messages into a `User:` / `Assistant:` transcript (skipping tool messages and empty assistant placeholders).
|
||||||
|
3. The runner calls `client.Generate(...)` with the summary prompt + transcript and a 512-token cap so the compaction step itself stays cheap.
|
||||||
|
4. The returned text is prepended as a system message (`"Earlier conversation summary:\n…"`), followed by the recent tail.
|
||||||
|
5. `LastCompaction()` returns `CompactionStats` so the SSE handler can emit a `compaction` event right before the streamed chunks.
|
||||||
|
|
||||||
|
### Failure mode
|
||||||
|
|
||||||
|
If `Generate` errors or returns an empty summary, compaction falls back to `truncateToBudget`: drop oldest user-turns one at a time until the remaining slice fits `threshold` tokens (heuristic: `len(s) / 4 + 1`). The current user turn is always preserved. The fallback is logged at WARN and the request still proceeds — a flaky summarize call never fails the user's request.
|
||||||
|
|
||||||
|
### Wire protocol
|
||||||
|
|
||||||
|
Streaming responses gain an optional `compaction` event:
|
||||||
|
|
||||||
|
```
|
||||||
|
data: {"type":"compaction","older_turns":6,"kept_turns":2,"summary_tokens":120,"window_tokens":4096,"used_tokens":3500}
|
||||||
|
```
|
||||||
|
|
||||||
|
Emitted after `start` (when applicable) and before `sources` / `chunk`. The widget can render this as a subtle "Context compacted" hint or ignore it — both are valid.
|
||||||
|
|
||||||
|
### Persistence
|
||||||
|
|
||||||
|
Compaction is **per-request**. The full transcript is still saved to `messages` in `data/portfolio.db` verbatim, so `GET /api/conversations/{id}` always returns the original history. Only what we send to the LLM is reduced — the next session can re-read the full thread from the DB.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
## 🌐 5. Embedding the widget
|
## 🌐 5. Embedding the widget
|
||||||
|
|
||||||
The bot ships with a drop-in vanilla-JS widget. Add two files to your site and it works.
|
The bot ships with a drop-in vanilla-JS widget. Add two files to your site and it works.
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue