victorvargas/rony-llm-agent
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
rony-llm-agent/pkg/llm
History
Victor Vargas 0652023037 feat(rag): add SQLite+FTS5 backend, fix content/usage plumbing bugs 
Backend.Upsert never received the fragment's Content, so ChromaDB (and
any backend) stored the vector but silently dropped the actual text —
saved memories had nothing to retrieve later. Backend.Search now also
takes the raw query text, and a failed/missing embedding no longer
hard-fails Add/Search: it degrades to a nil vector so a lexical-capable
backend can still index/find the content (Chroma has no such fallback
and now says so explicitly instead of misbehaving).

Adds pkg/rag/backends/sqlitevec: a zero-dependency backend (pure-Go
SQLite, no external service) that does cosine similarity when a real
embedding vector is available and falls back to FTS5/BM25 full-text
search otherwise. Adds pkg/rag/embeddings.OpenAICompatible, covering
both a local llama.cpp server (`--embeddings` enabled) and real OpenAI
(or any OpenAI-shaped /embeddings endpoint) through the same client.

Also fixes token usage tracking for llama.cpp streaming: the client
never requested `stream_options.include_usage` nor parsed a usage-only
SSE event, and even when present, the agent loop's RunStream dropped
any chunk with no Delta/ReasoningDelta — silently discarding the only
chunk that carries usage.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-06 00:05:30 -07:00
..
mock test(llm,embeddings): add unit tests for mock client, types, and Ollama embedder 2026-07-03 14:22:41 -07:00
providers feat(rag): add SQLite+FTS5 backend, fix content/usage plumbing bugs 2026-07-06 00:05:30 -07:00
README.es.md docs(i18n): translate all docs to English (with .es.md as Spanish alternative) 2026-06-30 13:40:37 -07:00
README.md docs(i18n): translate all docs to English (with .es.md as Spanish alternative) 2026-06-30 13:40:37 -07:00
types.go feat: add history messages, reasoning content, and adaptive language support 2026-07-05 16:17:37 -07:00
types_test.go test(llm,embeddings): add unit tests for mock client, types, and Ollama embedder 2026-07-03 14:22:41 -07:00

README.md

pkg/llm

Multi-provider abstraction for language models.

Responsibility

Define a common interface (LLMClient) and adapters for the main providers.

Public API

type LLMClient interface {
    Generate(ctx context.Context, req CompletionRequest) (CompletionResponse, error)
    Stream(ctx context.Context, req CompletionRequest) iter.Seq2[StreamChunk, error]
    Name() string
    Capabilities() ProviderCapabilities
}

type CompletionRequest struct {
    Messages    []Message
    Tools       []tools.Tool
    ToolChoice  ToolChoice
    Model       string
    Temperature *float32
    MaxTokens   *int
}

type CompletionResponse struct {
    Content    string
    ToolCalls  []tools.Call
    Usage      TokenUsage
    StopReason string
}

type ProviderCapabilities struct {
    SupportsTools    bool
    SupportsVision   bool
    MaxContextWindow int
}

Included providers

Provider Package Tool support
OpenAI providers/openai ✅
Anthropic providers/anthropic ✅
Ollama providers/ollama ✅ (models that support it)
llama.cpp providers/llamacpp ✅ (with grammar)

Usage

import "github.com/VictorVargas/rony-llm-agent/pkg/llm/providers/anthropic"

client, err := anthropic.New(anthropic.Config{
    APIKey: os.Getenv("ANTHROPIC_API_KEY"),
    Model:  "claude-sonnet-4.5",
})

resp, err := client.Generate(ctx, llm.CompletionRequest{
    Messages: []llm.Message{
        {Role: llm.RoleUser, Content: "Hello"},
    },
})

Streaming

for chunk, err := range client.Stream(ctx, req) {
    if err != nil { return err }
    fmt.Print(chunk.Delta)
}

Mock for tests

import "github.com/VictorVargas/rony-llm-agent/pkg/llm/mock"

mockClient := mock.New(mock.Responses{
    {Match: "hello", Response: "Hi! How are you?"},
    {Match: "*",    Response: "default"},
})

See also