In plain terms
ChromaDB is a vector database — a special kind of storage that lets the AI find relevant content in your documents based on meaning, not just keywords. When you upload a document, ChromaDB breaks it into chunks, converts them into numbers (embeddings), and stores them. When you ask a question, ChromaDB finds the most relevant chunks and passes them to the AI so it can answer accurately.
What it enables
- Ask questions across thousands of documents instantly
- AI answers are grounded in your actual documents — not guessed
- Works with PDFs, Word docs, text files, markdown
- Searches by meaning — "budget constraints" finds "cost limitations" too
- Completely local — documents never leave your server
Through Open WebUI (easiest)
Upload documents directly in the chat interface. Open WebUI sends them to ChromaDB automatically. You don't need to know ChromaDB exists — it works behind the scenes.
Go to Workspace → Knowledge in Open WebUI to create document collections you can query across multiple conversations.
Through Python (for developers)
Use the ChromaDB Python library to add documents, query them, and manage collections programmatically. This is how LangChain and custom applications connect to it.
API available at http://localhost:8000 for direct access from applications.
You upload a document
A PDF, Word doc, or text file. Could be a 200-page tender document or a one-page SOP.
ChromaDB splits and stores it
The document is split into overlapping chunks (e.g., 500 words each). Each chunk is converted into a numerical "embedding" that captures its meaning, and stored in ChromaDB.
You ask a question
Your question is also converted into an embedding. ChromaDB finds the chunks most similar in meaning to your question.
AI reads and answers
The top matching chunks are sent to the AI model along with your question. The AI reads them and gives you an accurate, sourced answer — not a guess.