# Offline Workstation Runtime Compatibility Baseline

**Status**: G1-14 direction approved under `CT-G1-2026-007`; immutable locks and executed evidence
remain open; no implementation authority.  
**Evidence date**: 2026-07-14.

## Repository Finding

The current runtime declarations are not one compatible immutable stack:

- `ansible/roles/nvidia/tasks/main.yml` installs `nvidia-driver-550-open`, CUDA Toolkit/CUDART 12.4
  and unversioned cuDNN 9 packages;
- `ansible/starter.yml` declares CUDA 12.6 and says RTX 5090 requires 12.6 or later;
- `autoinstall/offline/container-images.txt` and the vLLM role use `vllm/vllm-openai:v0.10.2`
  without an image digest;
- the vLLM 0.10.2 production image identifies its CUDA runtime as 12.8.1 in upstream diagnostics;
- Docker, NVIDIA Container Toolkit, Ollama, Python/Conda packages and the Ubuntu kernel are acquired
  without a complete exact-version/digest lock.

NVIDIA's published support matrix identifies CUDA 12.8 as the first CUDA Toolkit support for
Blackwell architectures. CUDA 12.8.1 requires Linux driver 570.124.06 or later for the toolkit-driver
pair. Therefore driver 550/CUDA 12.4 cannot be the accepted universal baseline while RTX 5090 and RTX
PRO Blackwell profiles are in scope.

## Proposed AMD64 Release Baseline

| Surface | Proposed decision | Immutable release evidence still required |
|---|---|---|
| OS installer | Ubuntu Server 22.04.5 AMD64 | Canonical ISO digest/signature and redistribution record |
| Kernel | Ubuntu 22.04 HWE kernel compatible with the selected R570 open module | Exact package/version/digest set, Secure Boot signing path, boot/rollback evidence on every physical profile |
| NVIDIA driver | R570 open-kernel-module branch; exact release MUST be at least 570.124.06 and identical across accepted NVIDIA profiles unless a signed exception matrix exists | Exact Ubuntu package versions/digests, supported GPU IDs, module signing and rollback package set |
| Host CUDA | CUDA Toolkit/CUDART 12.8.1 only where native build/training tools require it; container-only inference MUST NOT depend on an accidental host toolkit | Exact packages/digests and proof that native consumers require them |
| cuDNN | CUDA 12-compatible cuDNN release selected and locked with the native framework set | Exact version/packages/digests and framework compatibility |
| vLLM | `vllm/vllm-openai:v0.10.2` candidate with CUDA 12.8.1, stored and referenced by OCI digest rather than tag | Registry manifest/config/layer digests, architecture, SBOM/license, Phi-3 smoke inference on 24 GB and higher profiles |
| NVIDIA Container Toolkit | Stable `1.19.1-1` candidate and matching `base`, `libnvidia-container-tools`, and `libnvidia-container1` packages | Exact Ubuntu packages/digests; Docker and containerd configuration/reload tests |
| Container engine | One approved Docker/containerd package set; no `latest` repository resolution | Exact packages/digests/config and GPU smoke test after daemon reload/reboot |
| Ollama | One release binary/runtime bundle valid for AMD64 CPU and NVIDIA profiles | Exact version, archive/object digests, license/SBOM and CPU/8 GB/24 GB smoke evidence |
| Python/Conda | One explicit lock per environment, including Python, PyTorch, Transformers, PEFT, bitsandbytes and training dependencies | Exact interpreter/package/wheel/conda build IDs, hashes, indexes, licenses and import/inference tests |

The accepted artifact MUST record actual package versions and digests, not only the branch-level values
above. A repository snapshot may choose a later R570 maintenance release, but it cannot silently move
to R575/R580 or another CUDA/vLLM baseline without a new compatibility decision and regression run.

## Capability Behavior

- `core` and `cpu-ai` MUST install and remain healthy without NVIDIA packages being a readiness
  prerequisite. Ollama CPU smoke inference is required only for `cpu-ai`.
- NVIDIA profiles MUST verify GPU PCI identity, loaded driver/module, Secure Boot state, container
  runtime GPU enumeration, CUDA container smoke test and declared model-runtime smoke inference.
- `gpu-lite` uses Ollama only unless a separately tested vLLM configuration fits the exact 8 GB target.
- `gpu-starter` and `gpu-standard` may enable vLLM only after the selected model fits and passes the
  declared memory/health window. VRAM threshold alone never proves model compatibility.
- A driver/module load failure degrades to the truthful CPU profile when CPU prerequisites pass; it
  MUST NOT claim GPU readiness or automatically fetch a driver.
- AMD GPU/ROCm remains absent. An AMD64 host with an AMD GPU follows the CPU-only behavior.

## Required Compatibility Evidence

Before G1-14 is complete:

1. Resolve every package/binary/image/environment object to version, architecture, size, digest and
   license in the signed inventory.
2. Prove the selected HWE kernel and driver across no-GPU, 8 GB, 24 GB and 48 GB-or-higher physical
   targets, including Secure Boot enabled/disabled as supported.
3. Test install, reboot persistence, daemon reload, missing/corrupt driver, kernel upgrade, driver
   upgrade, rollback and rollback failure without network access.
4. Run CUDA container enumeration and model smoke tests from the exact OCI digests; record peak RAM,
   VRAM, disk and startup time.
5. Prove CPU-only installation does not fail because NVIDIA components are absent.
6. Reconcile the resolved runtime inventory against ISO contents, SBOM, provenance and rollback
   checkpoint.

## Approval Decision

Cezen Technologies approved this direction as G1-14 under `CT-G1-2026-007`, while leaving exact
package/image/object digests evidence-open. Gate 1 still fails until the immutable lock, physical compatibility results and
rollback evidence exist. Approval MUST also acknowledge that current driver 550/CUDA 12.4 automation
is contradictory legacy behavior and cannot be used unchanged for Blackwell-capable release media.

## Official Evidence

- NVIDIA CUDA 12.8 release notes and driver requirements:
  https://docs.nvidia.com/cuda/archive/12.8.0/cuda-toolkit-release-notes/index.html
- NVIDIA driver architecture support matrix:
  https://docs.nvidia.com/datacenter/tesla/pdf/NVIDIA_Datacenter_Drivers.pdf
- NVIDIA Container Toolkit install guide and current package pin example:
  https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html
- NVIDIA Container Toolkit supported platforms:
  https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/supported-platforms.html
- vLLM 0.10.2 GPU installation and production image documentation:
  https://docs.vllm.ai/en/v0.10.2/getting_started/installation/gpu.html
  https://docs.vllm.ai/en/v0.10.2/deployment/docker.html
- vLLM 0.10.2 PyPI release files and embedded wheel metadata used to derive the matching
  `torch==2.8.0`, `torchaudio==2.8.0` and `torchvision==0.23.0` candidate pins:
  https://pypi.org/project/vllm/0.10.2/
- PyTorch 2.8.0 CPU and CUDA 12.8 index selection matrix:
  https://pytorch.org/get-started/previous-versions/