78 lines
2.5 KiB
Markdown
78 lines
2.5 KiB
Markdown
# Cezen AI Suite — Installer
|
|
|
|
## Quick Start
|
|
|
|
```bash
|
|
git clone <cgit-url>
|
|
cd cgit
|
|
sudo bash install.sh
|
|
```
|
|
|
|
Server reboots automatically after NVIDIA drivers install. Phase 2 runs on its own after reboot.
|
|
|
|
## What Gets Installed (Entry Tier)
|
|
|
|
| Service | Port | Notes |
|
|
|---|---|---|
|
|
| Ollama | 11434 | LLM inference, 2 models pre-loaded |
|
|
| Open WebUI | 3001 | Chat interface |
|
|
| vLLM | 8000 | OpenAI-compatible API (start manually) |
|
|
| JupyterLab | 8888 | Token: `cezen2024` |
|
|
| ChromaDB | 8100 | Vector DB for RAG |
|
|
| MLflow | 5000 | Experiment tracking |
|
|
| MinIO | 9001 | Object storage (user: cezenadmin / Cezen@2024!) |
|
|
| Grafana | 3000 | GPU + system monitoring (admin / cezen2024) |
|
|
|
|
## Testing Without a GPU (Multipass)
|
|
|
|
```bash
|
|
# On your MacBook:
|
|
multipass launch 22.04 --name cezen-test --cpus 4 --mem 8G --disk 40G
|
|
multipass shell cezen-test
|
|
|
|
# Inside the VM:
|
|
git clone <cgit-url>
|
|
sudo bash install.sh
|
|
```
|
|
|
|
NVIDIA driver install will succeed but `nvidia-smi` won't show GPUs — that's expected. All other services will run fine.
|
|
|
|
## Pull More Models
|
|
|
|
```bash
|
|
bash models/pull-models.sh --tier=entry
|
|
```
|
|
|
|
## File Structure
|
|
|
|
```
|
|
cgit/
|
|
├── install.sh ← Entry point
|
|
├── ansible/
|
|
│ ├── phase1_nvidia.yml ← Phase 1: drivers (triggers reboot)
|
|
│ ├── entry.yml ← Phase 2: full stack
|
|
│ └── roles/
|
|
│ ├── base/ ← OS, Python, Miniconda, LangChain
|
|
│ ├── nvidia/ ← Drivers, CUDA 12.4, cuDNN 9
|
|
│ ├── docker/ ← Docker CE + NVIDIA Container Toolkit
|
|
│ ├── k3s/ ← Lightweight Kubernetes
|
|
│ ├── ollama/ ← Ollama + Open WebUI
|
|
│ ├── vllm/ ← vLLM inference server
|
|
│ ├── jupyterlab/ ← JupyterLab notebooks
|
|
│ ├── chromadb/ ← Vector database
|
|
│ ├── mlflow/ ← Experiment tracking
|
|
│ ├── minio/ ← Object storage
|
|
│ └── monitoring/ ← Grafana + Prometheus + DCGM
|
|
└── models/
|
|
└── pull-models.sh ← Pull additional models
|
|
```
|
|
|
|
## Change Default Passwords
|
|
|
|
Before shipping to a customer, update these:
|
|
|
|
- JupyterLab token: `/opt/cezen/.jupyter/jupyter_lab_config.py`
|
|
- MinIO: `/etc/default/minio`
|
|
- Grafana: environment vars in monitoring role, or via UI after first login
|
|
- MLflow: no auth by default (add reverse proxy if needed)
|