194 lines
8.9 KiB
HTML
194 lines
8.9 KiB
HTML
<!DOCTYPE html>
|
||
<html lang="en">
|
||
<head>
|
||
<meta charset="UTF-8">
|
||
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
||
<title>QLoRA / Unsloth — Nexus One AI Portal</title>
|
||
<link rel="stylesheet" href="style.css?v=4">
|
||
</head>
|
||
<body>
|
||
|
||
<header class="topnav">
|
||
<a href="index.html" class="brand">Nexus One <span>AI</span></a>
|
||
<nav>
|
||
<a href="index.html">Home</a>
|
||
<a href="quickstart.html">Quick Start</a>
|
||
<a href="prompts.html">Prompt Library</a>
|
||
<a href="usecases.html">Use Cases</a>
|
||
<span class="nav-sep"></span>
|
||
<div class="nav-dropdown">
|
||
<button class="nav-drop-btn">Help ▾</button>
|
||
<div class="nav-drop-menu">
|
||
<span class="nav-drop-cat">LEARN /</span>
|
||
<a href="quickstart.html">Quick Start</a>
|
||
<a href="models.html">Models</a>
|
||
<span class="nav-drop-cat">SUPPORT /</span>
|
||
<a href="troubleshooting.html">Troubleshoot</a>
|
||
<a href="faq.html">FAQ</a>
|
||
<span class="nav-drop-cat">MORE /</span>
|
||
<a href="glossary.html">Glossary</a>
|
||
<a href="whats-new.html">What's New</a>
|
||
</div>
|
||
</div>
|
||
<div class="nav-dropdown">
|
||
<button class="nav-drop-btn">Admin ▾</button>
|
||
<div class="nav-drop-menu nav-drop-menu-wide">
|
||
<span class="nav-drop-cat">DOCS /</span>
|
||
<a href="security.html">Security & Privacy</a>
|
||
<a href="admin.html">Admin Guide</a>
|
||
<span class="nav-drop-cat">MONITOR /</span>
|
||
<a href="dashboard.html">Dashboard</a>
|
||
<a href="analytics.html">Usage Analytics</a>
|
||
<a href="audit.html">Audit Log</a>
|
||
<a href="feedback.html">Feedback & Ratings</a>
|
||
<span class="nav-drop-cat">MANAGE /</span>
|
||
<a href="users.html">Users</a>
|
||
<a href="teams.html">Teams</a>
|
||
<a href="models-admin.html">Model Manager</a>
|
||
<a href="training.html">Training</a>
|
||
<a href="knowledge.html">Knowledge Base</a>
|
||
<span class="nav-drop-cat">TOOLS /</span>
|
||
<a href="apikeys.html">API Keys</a>
|
||
<a href="benchmark.html">Benchmarking</a>
|
||
<a href="model-compare.html">Model Compare</a>
|
||
<a href="api-playground.html">API Playground</a>
|
||
<a href="guardrails.html">Guardrails</a>
|
||
<a href="rag-quality.html">RAG Quality</a>
|
||
<a href="router.html">Model Router</a>
|
||
<a href="connectors.html">Connectors</a>
|
||
<span class="nav-drop-cat">SYSTEM /</span>
|
||
<a href="console.html">Console</a>
|
||
<a href="settings.html">Settings</a>
|
||
</div>
|
||
</div>
|
||
<div class="nav-dropdown">
|
||
<button class="nav-drop-btn">AI Tools ▾</button>
|
||
<div class="nav-drop-menu">
|
||
<span class="nav-drop-cat">INTELLIGENCE /</span>
|
||
<a href="documents.html">Document Intelligence</a>
|
||
<a href="chat-multi.html">Multimodal Chat</a>
|
||
<a href="prompt-studio.html">Prompt Studio</a>
|
||
<a href="meeting.html">Meeting Assistant</a>
|
||
<span class="nav-drop-cat">AUTOMATION /</span>
|
||
<a href="agents.html">Agent Builder</a>
|
||
<a href="schedules.html">Scheduled Jobs</a>
|
||
<a href="workflows.html">Workflow Automation</a>
|
||
<span class="nav-drop-cat">QUALITY /</span>
|
||
<a href="evals.html">AI Eval Suite</a>
|
||
<a href="chatrooms.html">Chat Rooms</a>
|
||
</div>
|
||
</div>
|
||
</nav>
|
||
<a href="notifications.html" style="position:relative">🔔</a>
|
||
<span class="badge" data-brand="tier">Basic Tier</span>
|
||
<div id="nav-org-logo" class="nav-org-logo"></div>
|
||
</header>
|
||
|
||
<div class="tool-hero">
|
||
<div class="tool-icon">🎯</div>
|
||
<div class="tool-meta">
|
||
<a href="index.html#tools" class="back-link">← All Tools</a>
|
||
<h1>QLoRA / Unsloth</h1>
|
||
<div class="tagline">Fine-tune a large AI model on your own data — efficiently, without needing massive GPU memory.</div>
|
||
<div class="hero-actions">
|
||
<a href="http://ai.local:8888" target="_blank" class="btn-primary">Open in Jupyter ↗</a>
|
||
<span class="access-pill">📓 Run fine-tuning jobs via Jupyter</span>
|
||
</div>
|
||
</div>
|
||
</div>
|
||
|
||
<div class="content">
|
||
|
||
<div class="section-title">What is fine-tuning?</div>
|
||
<div class="info-grid">
|
||
<div class="info-card">
|
||
<h4>In plain terms</h4>
|
||
<p>Fine-tuning is the process of taking a general-purpose AI model (like Llama 3.1) and continuing to train it on your own data so it learns to behave in a way specific to your organisation — using your terminology, following your formats, reflecting your policies. The result is a model that's significantly better at your specific tasks than the base model.</p>
|
||
</div>
|
||
<div class="info-card">
|
||
<h4>When fine-tuning makes sense</h4>
|
||
<ul>
|
||
<li>You want the AI to answer in your organisation's voice and style</li>
|
||
<li>The base model doesn't know your industry's terminology</li>
|
||
<li>You have hundreds of example question-answer pairs from your domain</li>
|
||
<li>RAG alone isn't giving accurate enough results</li>
|
||
<li>You want to teach the model a specific task format (extract fields from forms)</li>
|
||
</ul>
|
||
</div>
|
||
</div>
|
||
|
||
<div class="section-title">What QLoRA and Unsloth are</div>
|
||
<div class="info-grid">
|
||
<div class="info-card">
|
||
<h4>QLoRA</h4>
|
||
<p><strong>Quantised Low-Rank Adaptation</strong> — a technique that dramatically reduces the GPU memory needed for fine-tuning. Instead of updating all the weights in a model (which would need 4–8× the model's VRAM), QLoRA only trains a small set of adapter layers and keeps the rest of the model in 4-bit compressed format. This makes it possible to fine-tune a 7B or 8B model on the RTX Pro 6000.</p>
|
||
</div>
|
||
<div class="info-card">
|
||
<h4>Unsloth</h4>
|
||
<p>Unsloth is a Python library that makes QLoRA fine-tuning <strong>2–4× faster</strong> and uses <strong>40–70% less memory</strong> than standard QLoRA implementations. It achieves this through hand-optimised GPU kernels. For Cezen Entry tier with the RTX Pro 6000, Unsloth is the recommended way to fine-tune — it makes jobs that would otherwise take 10 hours complete in 3–4 hours.</p>
|
||
</div>
|
||
</div>
|
||
|
||
<div class="notice">
|
||
💡 <strong>Fine-tuning is an advanced task.</strong> You'll need training data in the right format and some Python knowledge. Start by exploring <a href="tool-openwebui.html">Open WebUI</a> and <a href="tool-chromadb.html">RAG with ChromaDB</a> — for most use cases, RAG gives 80% of the benefit of fine-tuning with 10% of the effort.
|
||
</div>
|
||
|
||
<div class="section-title">How to approach a fine-tuning project</div>
|
||
<div class="steps">
|
||
<div class="step">
|
||
<div class="step-num">1</div>
|
||
<div>
|
||
<h4>Prepare your training data</h4>
|
||
<p>Fine-tuning needs examples in a structured format — typically a JSONL file where each line is a conversation: a prompt and the ideal response. A minimum of 50–100 high-quality examples is needed; 500–2000 is better. Quality matters more than quantity.</p>
|
||
</div>
|
||
</div>
|
||
<div class="step">
|
||
<div class="step-num">2</div>
|
||
<div>
|
||
<h4>Open Jupyter and set up the training script</h4>
|
||
<p>Open <a href="tool-jupyter.html">Jupyter</a> at <code>http://ai.local:8888</code>. Load the Cezen fine-tuning notebook template (provided by your administrator) or start from scratch with the Unsloth documentation examples.</p>
|
||
</div>
|
||
</div>
|
||
<div class="step">
|
||
<div class="step-num">3</div>
|
||
<div>
|
||
<h4>Configure and run training</h4>
|
||
<p>Set your base model, data file path, and training parameters (epochs, learning rate). A typical fine-tuning run for Llama 3.1 8B on 500 examples takes 45–90 minutes on the Entry tier RTX Pro 6000.</p>
|
||
</div>
|
||
</div>
|
||
<div class="step">
|
||
<div class="step-num">4</div>
|
||
<div>
|
||
<h4>Export to Ollama format</h4>
|
||
<p>Once training is complete, export the fine-tuned model to GGUF format using Unsloth's export function. Then load it into Ollama with <code>ollama create my-model -f Modelfile</code> and use it in <a href="tool-openwebui.html">Open WebUI</a> like any other model.</p>
|
||
</div>
|
||
</div>
|
||
<div class="step">
|
||
<div class="step-num">5</div>
|
||
<div>
|
||
<h4>Test and iterate</h4>
|
||
<p>Compare your fine-tuned model against the base model on your specific tasks. If it's not where you need it, add more training examples focused on the weak areas and re-run.</p>
|
||
</div>
|
||
</div>
|
||
</div>
|
||
|
||
<div class="section-title">Works with</div>
|
||
<div class="works-with">
|
||
<a href="tool-jupyter.html">📓 Jupyter</a>
|
||
<a href="tool-ollama.html">🦙 Ollama</a>
|
||
<a href="tool-openwebui.html">💬 Open WebUI</a>
|
||
<a href="tool-dcgm.html">📡 DCGM</a>
|
||
</div>
|
||
|
||
</div>
|
||
|
||
<footer>
|
||
<p>Nexus One AI · Powered by Cezen · Basic Tier</p>
|
||
<p>Questions? <a href="mailto:support@cezentech.com">support@cezentech.com</a></p>
|
||
</footer>
|
||
|
||
<script src="auth.js"></script>
|
||
<script src="branding.js"></script>
|
||
</body>
|
||
</html>
|