Problem and solution
01/Problem
02/Solution
03/How it works
The system, end to end.
- 01next →
Ingest
Add URLs, paste text, or upload files (PDF, .docx, .csv/.xlsx).
- 02next →
Chunk & Embed
RecursiveCharacterTextSplitter + snowflake-arctic-embed:110m via Ollama.
- 03next →
Store
Chroma collections per resource (kind_<resourceId>), registry mirrored to JSON.
- 04next →
Retrieve
Unified /api/chat queries top-K per collection, merges & ranks globally.
- 05loop
Generate
Local LLM (qwen2.5-coder:1.5b) answers with citations; streams to UI.
04/Architecture
Components and data flow.
How the moving parts fit together. Solid arrows are synchronous calls; dashed arrows are queued or batched.
- 01
Client (Next.js + React + Tailwind)
- 02
API routes (Next.js App Router style pages/api)
- 03
Unified chat endpoint (multi-resource retrieval)
- 04
Per-kind ingest endpoints (website, text, pdf, word, sheet)
- 05
Registry persistence (Map + JSON mirror, HMR-safe)
- 06
ChromaDB (vector store, cosine HNSW)
- 07
Ollama (embeddings + chat, fully local)
- 08
LangChain (text splitters, prompt templates)
- 09
localStorage (per-user library + chat history)
05/Technology
The stack.
- AI / ML
- Ollama (snowflake-arctic-embed:110m, qwen2.5-coder:1.5b)LangChain (text splitters, prompts)
- Frontend
- Next.js 14React 18Tailwind CSS
- Backend
- Next.js API routes (Node.js)LangChain
- Database
- ChromaDB (vector)SQLite (Chroma metadata)
- Infrastructure
- Docker-readyconcurrently (dev)npm scripts
- APIs
- Ollama REST API
06/Engineering decisions
What we picked, and why.
- 01
Unified chat endpoint for all resource kinds
Choice
Single /api/chat reads from registry, queries N collections in parallel, merges by distance
Alternatives
- Separate chat endpoints per kind
- Federated search service
Why
Keeps the frontend simple (one API call) and the backend DRY. The registry is the single source of truth mapping resourceId → collectionName + kind. Adding a new kind only requires an ingest route.
- 02
Deterministic resourceIds for idempotent ingestion
Choice
Hash of content (URLs, text, file bytes) → 12-char hex resourceId
Alternatives
- UUIDv4 per upload
- Auto-increment IDs
Why
Re-ingesting the same content returns the existing record (alreadyIngested: true). The UI can safely retry; the library deduplicates on resourceId. No orphan collections from duplicate uploads.
- 03
Registry persistence via JSON mirror (HMR survival)
Choice
In-memory Map mirrored to CHROMA_DB_PATH/.resource_registry.json with debounced atomic writes
Alternatives
- Plain module-scope Map
- External DB (Redis, Postgres)
Why
Next.js dev reloads wipe module scope. A plain Map loses all registrations. JSON mirror survives HMR with zero external deps. Debounced (50ms) + .tmp → rename avoids corruption.
- 04
ChromaDB + Ollama fully local (no cloud keys)
Choice
Chroma on localhost:8000, Ollama on localhost:11434
Alternatives
- Pinecone/Weaviate + OpenAI/Anthropic
- Embedded vector store (e.g., LanceDB)
Why
Zero API costs, full data privacy, works offline. Chroma's HTTP API is simple and the JS client is lightweight. Ollama serves both embeddings and chat from one binary.
- 05
Per-user isolation via localStorage (no auth backend)
Choice
Three keys: kr:user, kr:lib:<user>, kr:chats:<user>
Alternatives
- NextAuth + database
- Server-side sessions
Why
Keeps the stack minimal (no auth service, no user table). SSR-safe: server renders user="", client hydrates from localStorage. Multi-user on one machine via username switcher in header.
07/Evaluation
What we measured.
- 01
5
Resource kinds
- 02
200
Max URLs / ingest
- 03
100MB
Max PDF size
- 04
25MB
Max Word/Sheet size
- 05
1000chars
Chunk size
- 06
200chars
Chunk overlap
- 07
5
Top-K per collection
- 08
5
Global top-K
08/Challenges
What almost broke us.
- 01
Chroma Windows file locks on delete
What happened
deleteCollection leaves HNSW segment folders locked (EBUSY) on Windows.
What we did
DELETE route reads segment folder names from sqlite BEFORE deleteCollection, then runs async retry purge (fast 20×250ms + slow 30min). /api/resources/_purge-orphans for manual sweep.
- 02
HMR wiping in-memory registry
What happened
Next.js dev reload resets module scope; plain Map loses all resource registrations.
What we did
resourceStore.js mirrors Map to .resource_registry.json with debounced atomic writes; lazy reads on first access.
- 03
LangChain/Ollama role mismatch
What happened
LangChain uses human/ai roles; Ollama expects user/assistant — empty responses without normalization.
What we did
chatModel.js normalizes via ROLE_MAP reading from message._getType() / getType() / lc_kwargs.role.
- 04
Multi-file ingestion idempotency
What happened
Same file set uploaded twice should hit same Chroma collection.
What we did
Deterministic resourceId from sorted filenames + total bytes + combined SHA256 hash.
09/Results
Single binary (Ollama) + single vector DB (Chroma) + Next.js frontend delivers a complete local RAG stack. Five resource kinds ingest via unified pipeline. Per-user libraries persist across reloads. Chroma registry survives HMR. Windows file-lock cleanup automated with background purge + manual endpoint.
10/Future improvements
- ›Hybrid retrieval (BM25 + dense) via Chroma's built-in BM25 or Tantivy
- ›Cross-encoder reranker (local, via Ollama or ONNX)
- ›Streaming citations (progressive as chunks arrive)
- ›Multi-tenancy with isolated Chroma collections per user
- ›Eval harness (groundedness, citation precision, refusal rate)
- ›PWA support for offline query of already-ingested corpus
Have an interesting AI problem?
Let's build something.
If you're working on something that needs real AI engineering — not just a demo — let's talk.