Local RAG / vector search

Semantic search over this wiki using a local Ollama embedding model and a gitignored Chroma store. Not a content domain — tooling only.

Prerequisites

1. Ollama running (ollama serve if needed).
2. Models already pulled (tags used by config.py):

Role	Model tag
Embeddings	qwen3-embedding:8b-q8_0
RAG chat (ask.py)	huihui_ai/qwen3-coder-abliterated:30b-a3b-instruct-q3_K_M

1. Nix (for the Python env — no pip).

The vector store lives at meta/rag/.chroma/ and is not committed (see root .gitignore).

Enter the shell

From the repo root:

nix-shell meta/rag/shell.nix

If a private Nix binary cache is unreachable and evaluation stalls, force cache.nixos.org:

nix-shell meta/rag/shell.nix --option substituters 'https://cache.nixos.org'

Or one-shot:

nix-shell meta/rag/shell.nix --option substituters 'https://cache.nixos.org' --run 'python meta/rag/ingest.py --reset'

Commands

Rebuild the index (wipe + full ingest):

nix-shell meta/rag/shell.nix --run 'python meta/rag/ingest.py --reset'

Re-ingest without wiping (per-file delete + upsert):

nix-shell meta/rag/shell.nix --run 'python meta/rag/ingest.py'

Semantic search:

nix-shell meta/rag/shell.nix --run 'python meta/rag/search.py "impermanence neededForBoot" -k 8'

RAG ask (retrieve + local coder):

nix-shell meta/rag/shell.nix --run 'python meta/rag/ask.py "when should I use deploy-rs vs Colmena?"'

What gets indexed

All *.md under the repo root except docs/, site/, .git/, meta/rag/.chroma/, and other skip dirs listed in config.py. Chunks are heading-aware; queries use a Qwen3 instruction prefix for retrieval.

Rebuild after large content edits. The corpus is small (~hundreds of leaves); a full --reset is normal.

Layout

File	Role
shell.nix	python3 + chromadb + httpx
config.py	Models, paths, chunk sizes
chunk.py	Markdown chunker
ollama_client.py	Embed + chat HTTP
store.py	Chroma helpers
ingest.py	Build / refresh index
search.py	Ranked chunk search
ask.py	Thin RAG CLI