CPU-only llama.cpp image

This image provides the CPU-only llama-server OpenAI-compatible HTTP server for rony-chat-bot. It supports chat completions and embeddings, and is built for linux/amd64 and linux/arm64.

Image

src.sersofts.org/victorvargas/llama.cpp:bNNNN   # pinned to a llama.cpp release
src.sersofts.org/victorvargas/llama.cpp:latest  # rolling

Publishing a new version

Publishing happens automatically through Forgejo Actions on every tag push that matches a llama.cpp release tag:

git tag bNNNN            # e.g. b10400
git push origin bNNNN

The workflow compiles llama.cpp at that exact tag natively for linux/amd64 and linux/arm64, then publishes both the immutable version tag and latest.

To find the next bNNNN, check the llama.cpp releases page. Tags prefixed with v (e.g. v10400) are also accepted; the v is stripped before being passed to the build.

A repository secret carrying a Forgejo access token with write:package scope is required for the push step.

Manual/local build

For one-off builds outside CI, build.sh is provided. It expects an access token with write:package scope to be exported in the environment; see the script source for the variable name.

./build.sh bNNNN

Running the image

podman run --rm src.sersofts.org/victorvargas/llama.cpp:latest --help

Example server arguments: --host 0.0.0.0 --port 8080 --model /models/model.gguf --ctx-size 4096 --parallel 1 --threads 2 --device none. Embedding services can add --embedding --pooling mean.

Troubleshooting

Problem	Fix
CI doesn't trigger	Tag must match b[0-9]+ (or v[0-9]+). Make sure the tag was actually pushed with git push origin <tag>, not just created locally.
Multi-arch build fails	Ensure the runner has QEMU and Buildx available; retry without stale cache.
strip warning	Verify the build produced /src/build/bin/llama-server; harmless non-fatal warnings can be reviewed before removing strip.
Runtime loader/library error	The image is built fully static via -DBUILD_SHARED_LIBS=OFF, so only standard C/C++/OpenMP libraries are needed at runtime. If you switched to shared libs (default), llama-server loads libllama-server-impl.so at runtime; either keep the runtime stage self-contained (copy the whole build/bin/) or rebuild with BUILD_SHARED_LIBS=OFF.
Push denied / login fails	The CI secret must be a Forgejo access token with write:package scope on the package namespace. For local builds, use a token with the same scope.