release: freeze Nexus One AI 1.0.0-rc1

specs/001-enterprise-experience/evidence/gate2-build-host-handoff.md

@@ -0,0 +1,777 @@
+# Offline Workstation Gate 2 Build-Host Handoff
+
+**Date**: 2026-07-14  
+**Host**: `172.16.10.144` (`acc-srv`)  
+**Workspace**: `/mnt/nvme0/nexus-one-ai-offline-workstation/`  
+**Scope**: Gate 2 preparation; no new build or ISO claim.
+
+## Prior Run Disposition
+
+No matching build, bundle, ISO, squashfs, container-pull or model-pull process was running when
+inspected. The prior workspace was preserved rather than deleted:
+
+`/mnt/nvme0/nexus-one-ai-offline-workstation-legacy-20260714/`
+
+The legacy payload is approximately 30 GB and is not trusted as Gate 2 input. It contains mutable
+`latest`/`main` container exports, an unpinned Miniconda installer, mixed CUDA assumptions, partial
+root-owned APT state and a source copy without Git metadata. It remains quarantine/reference evidence
+only and MUST NOT be merged into a candidate payload by filename.
+
+## Clean Workspace
+
+Created with mode `0750` and separate `source`, `staging`, `artifacts`, `logs`, and `evidence`
+directories. The current local source/spec snapshot was synchronized while excluding `.git`, ISO
+files, caches, virtual environments, `node_modules`, and unrelated `how-it-flows.gif`.
+
+- Local committed base at synchronization: `bea082a2014a3edc31cb006108b952c8adbbc8bc`
+- Synchronized files: 219
+- Remote source checksum-index digest:
+  `a9a0150f656e142c32e3962058ed83ef38eaa0685ff6c233e41d2896dbb84389`
+- Checksum index: `/mnt/nvme0/nexus-one-ai-offline-workstation/evidence/source-sha256sums.txt`
+- Exclusion check: pass; no `.git`, `*.iso`, or `how-it-flows.gif` found in synchronized source.
+
+The snapshot includes local uncommitted specification work. Therefore the committed base alone is not
+its identity; the checksum index is the authoritative transferred-source identity for this handoff.
+
+## Host Baseline
+
+- Ubuntu 22.04 AMD64
+- Kernel `6.8.0-124-generic`
+- `/mnt/nvme0`: 596 GB total, 567 GB available at inspection
+- Available: Docker, xorriso, mksquashfs, jq, Python 3
+- Not present: podman, skopeo, crane, oras
+
+No build was started during this handoff. Gate 2 must first resolve immutable package/image/runtime/
+model inventory and correct C1-C4/C7. Starting the legacy bundle builder unchanged would violate the
+approved architecture.
+
+## Immutable-Lock Preflight Increment
+
+The synchronized source now includes `validate-payload-lock.py`, an intentionally unresolved
+`payload-lock.candidate.json`, standard-library tests, and a hard preflight at the start of
+`build-bundle.sh`. The legacy mutable acquisition body is unreachable even after a lock passes until
+lock-driven acquisition is implemented.
+
+Local evidence:
+
+- Three `unittest` cases pass: unresolved candidate rejection, structurally complete test-lock
+  acceptance, and mutable OCI/duplicate-target rejection.
+- `bash -n autoinstall/offline/build-bundle.sh`: pass.
+- Candidate validation: expected fail with unresolved/status/object-group reasons.
+
+Build-host evidence:
+
+- Preflight exit: `1` (expected fail closed).
+- Artifact/staging entry count before: `0`; after: `0`.
+- No acquisition, output deletion, payload assembly or ISO build occurred.
+- Logs: `/mnt/nvme0/nexus-one-ai-offline-workstation/logs/gate2-preflight.stdout` and
+  `gate2-preflight.stderr`.
+
+## Resolver Increment 001
+
+The connected resolver created a deterministic source archive and resolved six Linux/AMD64 OCI
+manifests without pulling image layers or creating an artifact.
+
+- Candidate lock: `/mnt/nvme0/nexus-one-ai-offline-workstation/evidence/payload-lock.resolution-001.json`
+- Candidate lock SHA-256:
+  `146719ad9de2c99f402dfb15e16c36a029328219e53cc3f2c8ca835d4d01e51d`
+- Source archive SHA-256:
+  `6c890fc6537f2ed51cc40a9930a1bad1d3db78c31b093259fb81f5638cf5aade`
+- Source archive reproducibility: pass; two independent archives are byte-identical.
+- OCI manifests resolved: 6; artifact directory remains empty.
+- Unresolved entries after this increment: 9.
+
+The result correctly remains a candidate. Open WebUI returned no direct config/layer byte total in the
+selected Docker manifest response, two container licenses remain review-required, mutable source tags
+must be replaced with approved release tags, and exported OCI archive bytes/digests require actual
+digest-pinned acquisition. Registry resolution also demonstrated drift: current resolved digests differ
+from the quarantined run for mutable/tag-based inputs. The quarantined digests are evidence of drift,
+not approved pins.
+
+APT closure, Python closure, Ubuntu media, Miniconda/replacement runtime, Ollama runtime and exact
+model files/notices remain unresolved. No payload or ISO build has started.
+
+## APT Resolver Increment 001
+
+The APT resolver used an empty dpkg status with authenticated package acquisition and
+`--no-install-recommends`. This prevents already-installed build-host packages from disappearing from
+the computed clean-system closure. It wrote downloads only under Gate 2 staging and installed nothing.
+
+- Requested packages: 26
+- Resolved `.deb` objects: 487
+- Measured bytes: 357,993,500
+- Candidate lock: `/mnt/nvme0/nexus-one-ai-offline-workstation/evidence/payload-lock.resolution-002.json`
+- Candidate lock SHA-256:
+  `351c842ab92995315b008865a3080a5b1469f002cdc235702fff28cdf7bcf518`
+- Artifact directory: empty
+- Candidate validator: expected fail; Python/runtime/model groups, Open WebUI measured bytes, reviewed
+  status and unresolved list remain incomplete.
+
+This closes only the measured Ubuntu/deadsnakes `.deb` byte closure for the currently requested list.
+The APT repository index/origin snapshot and its digests, package license/notices, and the separately
+approved R570/CUDA 12.8.1/cuDNN/NVIDIA Container Toolkit closure remain unresolved. Current repository
+URLs are not yet accepted as reproducible release identity.
+
+## Python Resolver Increment 001 — Boundary Correction and Candidate Closure
+
+The first Python 3.11 wheel-only run combined backend and platform requirements and correctly failed
+dependency solving:
+
+- Backend constraint: `bcrypt<4.0.0`.
+- Platform constraint: `chromadb==0.5.23`, whose declared dependency is `bcrypt>=4.0.1`.
+- Resolver result: `ResolutionImpossible`; exit `1`.
+- Candidate `payload-lock.resolution-003.json`: absent.
+- Downloaded wheel files in that run: `0`.
+- Artifact directory entries: `0`.
+- Failure log: `/mnt/nvme0/nexus-one-ai-offline-workstation/logs/python-resolution-001.stderr.log`.
+- Failure-log SHA-256:
+  `28fe44f5cab1d41d5ebf92322819b006c92eb13df79879b579e93260388f9b61`.
+
+Repository inspection then proved those inputs belong to separate runtime boundaries: the backend and
+Chroma Ansible roles create different virtual environments, and the legacy bundle layout already has
+separate `backend`, `chromadb` and `platform` wheelhouses. The conflict was therefore a resolver-boundary
+defect, not an application dependency contradiction. Neither dependency constraint was changed.
+
+The corrected resolver produced three independent closures:
+
+- Backend: 61 wheel objects.
+- ChromaDB: 88 wheel objects.
+- Platform: 214 wheel objects.
+- Total: 363 wheel objects; 3,238,751,953 measured bytes.
+- License fields requiring review: 49.
+- Candidate lock: `/mnt/nvme0/nexus-one-ai-offline-workstation/evidence/payload-lock.resolution-003.json`.
+- Candidate lock SHA-256:
+  `cde0f17371dde68af984511114a2f2fca449f501ecbbdd622f48c92c6508b261`.
+- Successful resolver log SHA-256:
+  `7d6b49f9f985b47190fd7b6c0fc6f00c72b80938e1905fed3b4a4524791c82cc`.
+- Artifact directory entries: `0`.
+- Candidate validator: expected fail because the lock is not reviewed, has 14 unresolved entries,
+  and still lacks complete runtime/model objects and measured Open WebUI bytes.
+
+The closure is evidence, not an approved payload. The unpinned platform requirements selected current
+Torch/CUDA dependencies including four explicitly CUDA-13-named objects, which conflicts with the
+approved CUDA 12.8.1 baseline. Gate 2 remains blocked on an explicitly pinned CUDA-12.8-compatible
+Python ML stack, index snapshot/origin and resolver-report approval, license/notice review, and clean
+Python 3.11 offline installation/import evidence for each independent environment.
+
+## Python Resolver Increment 002 — CUDA 12.8-Compatible Candidate
+
+Authoritative vLLM 0.10.2 wheel metadata pins `torch==2.8.0`, `torchaudio==2.8.0` and
+`torchvision==0.23.0`. The inspected AMD64 vLLM wheel SHA-256 is
+`e0cba6110483d9bf25c4402d8655cf78d366dd13e4155210980cc3480ed98b7b`, matching the
+published PyPI file digest. Those three versions were added to the platform input and resolved in a
+fresh staging directory; the previous CUDA 13 candidate remains preserved for comparison.
+
+- Backend: 61 wheel objects.
+- ChromaDB: 88 wheel objects.
+- Platform: 212 wheel objects.
+- Total: 361 wheel objects; 4,435,239,499 measured bytes.
+- Resolved Torch/Torchaudio/Torchvision: `2.8.0` / `2.8.0` / `0.23.0`.
+- Explicit CUDA 13 objects: 0.
+- Explicit CUDA 12 objects: 14.
+- License fields requiring review: 49.
+- Candidate lock: `/mnt/nvme0/nexus-one-ai-offline-workstation/evidence/payload-lock.resolution-004.json`.
+- Candidate lock SHA-256:
+  `3b1fef9705e8049b28d283383f16fa7fd66e7fc00434315b2cc86163ba8e93ac`.
+- Successful resolver log SHA-256:
+  `37711950c25a1b2409e1323ea35ce7f6cdab2f0a9e6d2aad9407baeda1325c35`.
+- Artifact directory entries: 0.
+- Candidate validator: expected fail because review/unresolved/runtime/model/Open WebUI evidence is
+  still incomplete.
+
+This closes the version-selection contradiction only. It does not yet prove execution against the
+approved R570/CUDA 12.8.1 physical profiles, clean disconnected Python 3.11 installation/import,
+license/notice acceptance, or NVIDIA-independent CPU operation. The universal Workstation installer
+also needs a profile-aware installation contract so CPU-only systems do not depend on NVIDIA packages
+merely because the signed universal media carries them.
+
+## Python Validation Increment 001 — Disconnected Install and GPU-Lite Smoke
+
+Validation used official `python:3.11.13-slim-bookworm` on Linux/AMD64, pinned after acquisition to
+`python@sha256:86adf8dbadc3d6e82ee5dd2c74bec2e1c2467cdad47886280501df722372d2e1`.
+Each installation ran in a disposable container with `--network none`, read-only wheel mounts and
+`pip --no-index --find-links`; no package index or other network fallback was possible.
+
+- Backend wheelhouse: pass. Imports succeeded and Passlib/bcrypt hash/verify succeeded.
+- ChromaDB wheelhouse: pass; `chromadb==0.5.23` imported successfully.
+- Platform wheelhouse CPU path: pass. Torch/Torchaudio/Torchvision reported `2.8.0+cu128` /
+  `2.8.0+cu128` / `0.23.0+cu128`; `torch.cuda.is_available()` was false with CUDA hidden and CPU tensor
+  execution returned the expected result.
+- Supplemental `gpu-lite` path: pass on RTX 3050 OEM with 8,222,670,848 bytes reported device memory.
+  Torch reported CUDA 12.8 and completed synchronized GPU tensor execution.
+- Host driver for that supplemental run: NVIDIA Open Kernel Module `595.58.03`, license `Dual MIT/GPL`.
+- Artifact directory entries: 0.
+
+Evidence log SHA-256 values:
+
+- Backend: `1d9be80fef4c565e13a9b12d00cc76a049441eb89d2136cb0d9e7703494a490a`.
+- ChromaDB: `6b4a907b6cee363a1260be8d70789cf7ab1bb38702917fb7b4e0ee60132d4982`.
+- Platform CPU: `35eb2e5861e98158e934771d1256567e3b68d23adb51a7fb6206a5af71853719`.
+- GPU-lite: `b51584484bf0317f8a79058f91b5613dd1b0f7a7d17e07856a43811652721fcb`.
+
+This proves disconnected wheel availability/importability and one supplemental 8 GB GPU smoke test.
+It does not prove a clean bare-metal install, the exact R570 reference driver, 24 GB/48 GB profiles,
+vLLM model serving, interruption/recovery, or selective CPU-profile installation without NVIDIA
+packages. Those claims remain blocked, as do index provenance approval and 49 license/notice reviews.
+
+## Python Profile Selection Increment 001 — Strict CPU Boundary
+
+The universal-media/profile-selective contract now defines exactly-one selection between `cpu-only`
+and `nvidia-cu128`. PyTorch's official 2.8.0 index matrix supplies distinct CPU and CUDA 12.8 builds.
+The CPU trio was acquired from the CPU index into staging, then combined with the reusable non-GPU
+platform wheels while excluding NVIDIA, CUDA, Triton and bitsandbytes objects.
+
+- CPU selected wheels: 196; measured bytes: 385,850,558.
+- Forbidden wheel count: 0.
+- CPU Torch SHA-256: `cb06175284673a581dd91fb1965662ae4ecaba6e5c357aa0ea7bb8b84b6b7eeb`.
+- CPU Torchaudio SHA-256: `e54bd7fc9472019308097d99102df9acee22aa2451ae808d27840bc874320292`.
+- CPU Torchvision SHA-256: `d83d8075db43b8ca89680bdeb2f100c832e2a3aa61ee42c038b1a146e5e511b6`.
+- Network-disabled full CPU-profile install/import: pass; Torch `2.8.0+cpu`,
+  `torch.version.cuda is None`, CPU tensor result correct.
+- Strict CPU validation log SHA-256:
+  `57e2ab86f6fae188502c019429b37ab03ed2f8bad8fdd1330ec39fd17f69173e`.
+- Machine-readable profile validator: pass, 196 wheels; log SHA-256:
+  `2f937161184294ea8c8e2413282db006498aca52517927f305fd3371e3defe14`.
+- Artifact directory entries: 0.
+
+The contract requires hardware-detection failure to default to CPU, records selection before mutation,
+reuses it on resume, and requires rollback before profile change. This proves the Python payload
+boundary, not the eventual installer's hardware detector, journal, rollback or signed membership.
+
+## Python Profile Planner Increment 001 — Non-Mutating State Machine
+
+The new planner consumes a normalized hardware probe, requested mode, operation ID, optional prior
+journal and a profile-membership verification receipt. It does not mutate the host. Auto-selection
+chooses NVIDIA only when presence, support and driver readiness are all explicitly true; incomplete or
+failed facts choose CPU. An active operation can only resume the same profile and operation ID. Profile
+change requires rollback, and a rolled-back operation ID cannot be reused.
+
+- Dependency-free planner tests: 6 pass; combined deployment tests: 11 pass.
+- GPU-lite structural start vector: selected `nvidia-cu128`; pass.
+- Resume vector: retained the journaled `nvidia-cu128` selection; pass.
+- Mid-operation CPU change vector: fail closed with `PROFILE_CHANGE_REQUIRES_ROLLBACK`; pass.
+- Planner-vector log SHA-256:
+  `a5d56b4ec355ac701ec3cd72f05298dc6cb97bd49b1c20fdc3fc548a994635f5`.
+- Artifact directory entries: 0.
+
+The membership receipt used here is explicitly marked structural-test-only. The planner trusts a future
+signature verifier's receipt; it does not verify cryptography. Production integration remains blocked
+until real signature verification, atomic durable journal writes, pre-mutation checkpoint creation,
+installer wiring and power-loss/rollback tests exist.
+
+## Profile Verification Receipt Increment 001
+
+A closed Draft 2020-12 receipt schema now binds verified profile membership to the exact manifest and
+envelope digests, key ID/fingerprint, trust snapshot, policy version, namespace, profiles, verification
+time and verifier build. The planner no longer accepts a bare verification boolean. It requires the
+receipt schema/result/namespace and rejects non-production receipts unless explicit test mode is set.
+
+- Combined deployment tests: 12 pass, including production rejection/explicit test acceptance.
+- Existing schema/signature/compatibility fixture suite: pass in a network-disabled Python 3.11
+  container using staged wheels.
+- Fixture-suite log SHA-256:
+  `6008edf74f86f181d84221ca49887751e047af87069ce9e451ecd336a7a40109`.
+- Production-mode structural test receipt: rejected with `TEST_RECEIPT_FORBIDDEN`; expected.
+- Explicit structural-test mode using the same receipt: accepted; expected.
+- Receipt-boundary log SHA-256:
+  `aa2369d4058fef580ef55f9cb8e7b8ced823b551503cc39a8fad9fca311b5191`.
+- Artifact directory entries: 0.
+
+This defines and tests the verifier/planner handoff but does not implement the production verifier,
+trust-policy evaluation or production key ceremony. Atomic journal/checkpoint persistence and
+installer integration remain open.
+
+## Atomic Profile Journal Increment 001
+
+The structural journal writer now uses exclusive locking, expected-generation compare-and-swap,
+canonical per-record digests, prior-record linkage, immutable operation/profile/manifest identity,
+mode `0600`, file fsync, atomic rename and directory fsync. Installation cannot enter `installing`
+before a `checkpoint_ready` record with a checkpoint identity exists.
+
+- Atomic journal unit tests: 5 pass; combined deployment tests: 17 pass.
+- Build-host generations 1 `profile_selected`, 2 `checkpoint_ready`, and 3 `installing`: pass.
+- Stale generation-1 writer after generation 2: `JOURNAL_GENERATION_STALE`; expected.
+- Persisted journal mode: `0600`.
+- Final journal SHA-256:
+  `35b3940893a27c4a09ac3d532fe992d23ec59396fa8f6883fc809f2df3803488`.
+- Transition-vector log SHA-256:
+  `48d2aabf56b882975ed539007fe3b0ef9f6cb4674c2cd68affdc7646214adbb1`.
+- Artifact directory entries: 0.
+
+Tests also cover digest tampering, identity change, missing checkpoint, invalid transition and terminal
+rollback. The writer does not create/verify a real filesystem/package checkpoint, and no forced
+power-loss test has yet proved filesystem-specific durability. Production signature verification,
+checkpoint implementation, crash injection and installer integration remain open.
+
+## Ed25519 Profile Verifier Increment 001
+
+The production-shaped verifier now canonicalizes the profile manifest, checks namespace and digest,
+enforces envelope algorithm/policy, resolves one active purpose/environment-scoped trust key, checks
+the public-key DER fingerprint, verifies Ed25519 with OpenSSL and emits the closed membership receipt.
+
+- Ubuntu/OpenSSL 3 combined deployment tests: 19 pass.
+- Positive Ed25519 path used an ephemeral test key generated only inside a disposable test directory.
+- Negative paths include manifest tampering and wrong key environment.
+- Verifier-test log SHA-256:
+  `b61d719273b7e755fd76aff8603f81e62b0449fde813eb01f7e2b0850633319c`.
+- Artifact directory entries: 0.
+
+The Mac OpenSSL build lacks Ed25519 and correctly skips the two crypto tests; authoritative crypto
+evidence is the Ubuntu/OpenSSL 3 run. No private test key entered the repository or logs. This is not a
+production ceremony or release signature: production public keys, trust snapshot, revocation/time
+policy, locked OpenSSL/verifier build and independent signing approval remain absent.
+
+## Python Environment Checkpoint Increment 001
+
+The checkpoint component now records whether the target existed and inventories directory, file and
+symlink state with modes, byte sizes, hashes and link targets. Creation and restore use same-filesystem
+temporary trees, file/directory fsync and atomic rename. Restore verifies the checkpoint before use and
+quarantines the failed/new tree rather than deleting it.
+
+- Checkpoint unit tests: 3 pass; combined local suite: 20 run, 19 pass and the unsupported Mac
+  Ed25519 test class skips.
+- Build-host create/verify/modify/restore vector: pass; original file SHA-256 restored exactly.
+- Checkpoint inventory SHA-256:
+  `488bfb0e7f67d75553b66ccbf981cb9139ef1b8610f1018d76e4832a5c748787`.
+- Checkpoint manifest file SHA-256:
+  `82f28057d73359d9b7d6fd75b23f9ff4718def883693104b88161001b16dfeab`.
+- Vector log SHA-256:
+  `cbbf47fd1d028635e2b2dce63c115a9066cff8df3e0fa0458d28f8691082df1d`.
+- Artifact directory entries: 0.
+
+This validates ordinary checkpoint/restore and tamper rejection in disposable staging. It does not yet
+prove crash consistency under forced termination at each fsync/rename boundary, real `/opt/cezen`
+ownership/ACL/xattr preservation, disk-full behavior or integration with journal/install mutation.
+
+## Python Checkpoint Crash-Recovery Increment 001
+
+Test-only process termination was injected at four fsync/rename boundaries on disposable build-host
+trees. Every injected process exited `97` as designed:
+
+- Before checkpoint publication: no checkpoint became visible; the original target remained authoritative.
+- After checkpoint publication: the visible checkpoint passed complete verification.
+- After failed-target quarantine but before activation: recovery completed verified restore and
+  preserved the failed tree.
+- After restore activation but before return: recovery recognized the exact restored inventory and
+  completed idempotently.
+
+Crash-vector log SHA-256:
+`17d0cb3f39b6c3f7114e2c81acc4e8884f96a496aa06a7788a9b380f801dc6f9`.
+Artifact directory entries remained 0.
+
+This is forced-process termination, not physical power removal. Filesystem/controller cache loss,
+disk-full/inode exhaustion, ACL/xattr preservation, journal/checkpoint orchestration and bare-metal
+recovery remain open.
+
+## Profile Pre-Mutation Orchestration Increment 001
+
+The isolated orchestrator now composes structural receipt acceptance, profile planning, exact wheel
+validation, atomic journal selection and verified environment checkpoint. Invalid payload membership
+creates neither journal nor checkpoint. Valid preparation stops at `checkpoint_ready` and contains no
+package mutation action.
+
+- Orchestrator unit tests: 2 pass; combined local suite: 23 run, 22 pass plus the expected Mac
+  Ed25519 capability skip.
+- Build-host strict CPU preparation: pass, generation 2, `checkpoint_ready`.
+- Immediate rerun: same checkpoint/generation, pass; `mutation_performed: false` both times.
+- Checkpoint inventory ID:
+  `12e8f917f8a388c0a1d9b45c5f494ce759667f331dc0bdfd03f2b51bda2c9d3a`.
+- Journal SHA-256:
+  `1fd29a06646ab917a01937bb498945e1b8f347b1c731d78e1d825f0f521d3e60`.
+- Checkpoint manifest SHA-256:
+  `99d10d9ac0e48da10b70ccfff08d64af4222d6637d4e2ad0a1b22e1b41134ed1`.
+- Evidence log SHA-256:
+  `ed3209620640105cc15373fc9759b1198af2a1d6713e4f0d76d000e1a439fcde`.
+- Artifact directory entries: 0.
+
+This closes component-composition only through the pre-mutation boundary. Actual offline installation,
+health validation, installed/rollback journal transitions, service integration and production receipt
+inputs remain blocked.
+
+## Disposable CPU Installation Increment 001
+
+The offline executor was run entirely inside a network-disabled Python 3.11 container against a new
+container-mounted staging target. It consumed the strict 196-wheel CPU selection, not the CUDA-bearing
+platform set. The final-code rerun included full new-tree and parent-directory fsync barriers.
+
+- Offline dependency-failure unit vector: original target unchanged; journal `rolled_back`; pass.
+- Positive prepare/checkpoint/install/pre-activation-health/atomic-activation/post-health flow: pass.
+- Final journal: generation 4, `installed`, mode `0600`.
+- Installed Torch: `2.8.0+cpu`; compiled CUDA version: none.
+- Journal SHA-256:
+  `61e77937fadaee985e568a6f78e4a5118ac5198f2f5b692a14cce6c1b1997ca5`.
+- Checkpoint manifest SHA-256:
+  `0ca232445458f2a48bd3319d4e56964b92e5d11fee8b9b92938b1517af157a73`.
+- Installation log SHA-256:
+  `941fa474229ddcb8a4e23bdc95cba27ef51390580c8f278c914332b1e47c8195`.
+- Artifact directory entries: 0.
+
+This proves the CPU Python environment mutation/health path only in disposable container staging. It
+does not prove service wiring, system ownership/ACLs, live `/opt/cezen`, reboot resume, NVIDIA install,
+post-activation forced failure, disk exhaustion or bare-metal installation.
+
+## Post-Activation CPU Rollback Increment 001
+
+The full strict-CPU environment was installed in disconnected disposable staging, fsynced and atomically
+activated. Test-only termination then exited `97` immediately after activation and before post-health/
+`installed`. Recovery consumed the generation-3 `installing` journal, recorded `rollback_required`,
+restored the verified pre-install checkpoint, preserved the activated environment in quarantine and
+recorded generation 5 `rolled_back`.
+
+- Original pre-install marker restored: pass.
+- Failed activated environment executable present in quarantine: pass.
+- Final journal SHA-256:
+  `6acdfd8c8a57586ad0edae5e5a4cae0bac0a6d1055864dd35113afb8199dabad`.
+- Checkpoint manifest SHA-256:
+  `d5c5ff6fd95743431832d115a0b4fe7de8e9ae0fa7ee36b3c349316cb9ebfc53`.
+- Crash/rollback log SHA-256:
+  `5791738b02f6c0a41218a94bd04e6a1cbd80dd42b04e28a14f55ab451c61f15c`.
+- Artifact directory entries: 0.
+
+This closes the disposable CPU post-activation process-crash case. It does not prove reboot-triggered
+recovery, physical power loss, service health rollback, disk exhaustion, NVIDIA rollback or live paths.
+
+## Disposable NVIDIA Installation Increment 001
+
+The full 212-wheel `nvidia-cu128` platform selection was installed inside a network-disabled Python
+3.11 container with access to the build-host RTX 3050. Preparation used exact profile validation and
+checkpointing; install used no index, pre/post activation health, full-tree fsync and atomic activation.
+
+- Final journal: generation 4, `installed`, mode `0600`.
+- Torch: `2.8.0+cu128`; compiled CUDA: 12.8.
+- GPU: NVIDIA GeForce RTX 3050 OEM; device tensor execution and synchronization: pass.
+- Journal SHA-256:
+  `92cfa8a9cab9de57c08019957c564394c51eab6257dda41be2e9d352b2a422b6`.
+- Checkpoint manifest SHA-256:
+  `66c9c43fd432b8aa87d9212e9cbf51619cd38db1c5c7768a0102afaf96431ee7`.
+- Installation log SHA-256:
+  `af29e27e5cf3e55d01874bc00a895b1ce2b6e96f50e347c5890b6b70f969dd36`.
+- Artifact directory entries: 0.
+
+This is supplemental 8 GB `gpu-lite` evidence on Open Driver 595.58.03. It does not replace the exact
+R570 reference, 24/48 GB profiles, vLLM model-serving, NVIDIA rollback/reboot or bare-metal proof.
+
+## Post-Limit Build-Host Refresh
+
+After the external-action quota reset, the complete current source/spec snapshot was synchronized to
+the clean build-host workspace. The full remote deployment suite ran under Ubuntu/OpenSSL 3:
+
+- Tests: 27 passed; 0 skipped; exit 0.
+- Suite log SHA-256:
+  `f9b4e43c8bd587a4c92bc64a4744d61b3b879e0408ea57056380136d35f8517c`.
+- Artifact directory entries: 0.
+
+The expected offline-resolution negative vector appears in the log and is followed by an overall pass.
+This refresh aligns the build-host copy with T156 and the latest local implementation; it does not
+change Gate 2 readiness or authorize assembly.
+
+## Post-Activation NVIDIA Rollback Increment 001
+
+The full `nvidia-cu128` environment was installed without network access, health-checked, fsynced and
+atomically activated on the RTX 3050. Test-only termination exited `97` after activation and before
+the final health/journal commit. Recovery restored the verified pre-install checkpoint, preserved the
+activated CUDA environment in quarantine and recorded generation 5 `rolled_back`.
+
+- Original pre-install marker restored: pass.
+- Quarantined environment remained executable and reported Torch `2.8.0+cu128`: pass.
+- Final journal SHA-256:
+  `306b3873e37b276d45af2446cdd25a9a56d4545e0a4495d327578028b84ac272`.
+- Checkpoint manifest SHA-256:
+  `ca88bf52a19c22b90b5a445985e60faca177ad21dc67efdd2d3572e4378d7af3`.
+- Crash/rollback log SHA-256:
+  `e0b01f43762fc518ea26557ec58b7b4c1bc6cff366890154a972df96d391711f`.
+- Artifact directory entries: 0.
+
+This closes disposable post-activation process-crash rollback for the 8 GB `gpu-lite` Python profile.
+It does not prove reboot/power-loss recovery, exact R570, 24/48 GB profiles, vLLM service/model health
+or bare-metal NVIDIA installation.
+
+## Boot-Dispatcher CPU Recovery Increment 001
+
+A fresh strict-CPU operation ran in a network-disabled container and was forcibly terminated with exit
+`97` after atomic activation, leaving the verified journal at generation 3 `installing`. A second,
+separately created network-disabled container invoked `recover-python-profile-on-boot.py`, simulating
+a fresh privileged boot-service process rather than an in-process exception handler.
+
+- Pre-recovery journal: generation 3, `installing`.
+- Post-recovery journal: generation 5, `rolled_back`.
+- Original marker restored: pass.
+- Activated environment preserved with executable Python in quarantine: pass.
+- Second dispatcher invocation on terminal state: safe no-op.
+- Journal SHA-256:
+  `f5c60f693fd9ecf572e35d58390c596a71916cc1dac4f44a698aa67c81d1c0d9`.
+- Checkpoint manifest SHA-256:
+  `e6c0ed458c8ed785a0d611fb4b6da0353b9632e008fbee2060c6b8e151b500c0`.
+- Combined crash/recovery log SHA-256:
+  `ab9d9a23b9c946a9ee7b86440584f06038988d09a701ef7d8bd10cc5ae2f7944`.
+- Artifact directory entries: 0.
+
+This proves recovery across distinct disposable process/container lifetimes. It does **not** prove an
+actual OS reboot, abrupt physical power loss, production boot ordering, live path/ownership/ACL/xattr
+behavior or service-level health recovery; those evidence gates remain open.
+
+After T159 documentation and tests were synchronized, the complete remote deployment suite passed
+29 tests with no skips or failures. The suite log SHA-256 is
+`0060f0cc33d37e2d35165908109e76e5cc8e9909bd2a40dbd989fe1baac25da5`; the artifact directory remained
+empty. The expected impossible-package negative-vector output is followed by the overall `OK` result.
+
+## Controlled Reboot Recovery Increment 001
+
+Before reboot testing, the build host was found to use DHCP on `enp8s0`. Its active values were
+preserved exactly in the persistent NetworkManager profile and changed to manual assignment:
+`172.16.10.144/24`, gateway `172.16.10.1`, DNS `8.8.8.8`, autoconnect enabled. After both controlled
+reboots, the profile remained `manual` and the default route reported `proto static`.
+
+The first reboot vector failed closed at `rollback_required` generation 4 with
+`CHECKPOINT_TARGET_MISMATCH`: its checkpoint identity used container path `/state/python-env`, while
+the initial systemd harness supplied the corresponding host path. No restore was attempted and no
+artifact was produced. This negative result confirms exact target-path binding and is not counted as
+a successful recovery.
+
+A fresh vector then used the same `/state` mount namespace for checkpoint creation and boot recovery.
+The test-only systemd one-shot was ordered after local filesystems and Docker and before
+`multi-user.target`; its recovery container used `--network none` and the digest-pinned Python image.
+After a distinct boot ID, systemd reported exit 0 and the dispatcher recovered generation 3
+`installing` to generation 5 `rolled_back`.
+
+- Static address after reboot: `172.16.10.144/24`; gateway `172.16.10.1`; DNS `8.8.8.8`.
+- Original marker restored: pass.
+- Activated environment preserved with executable Python in quarantine: pass.
+- Final journal SHA-256:
+  `f5c60f693fd9ecf572e35d58390c596a71916cc1dac4f44a698aa67c81d1c0d9`.
+- Checkpoint manifest SHA-256:
+  `e6c0ed458c8ed785a0d611fb4b6da0353b9632e008fbee2060c6b8e151b500c0`.
+- Static-network evidence SHA-256:
+  `25453d9f8129abcf840a973bf650c9da896050296149e9cfab52a6082182a0f9`.
+- Boot-service journal SHA-256:
+  `b03b15ec77223eeae99932db25c554ce68b0f71688bf33390174955d24f0e163`.
+- Artifact directory entries: 0.
+
+The temporary systemd unit was disabled, removed and daemon state reloaded after evidence capture; the
+static NetworkManager profile intentionally remains. This closes controlled OS-reboot recovery for a
+disposable strict-CPU target. It does not prove sudden physical power removal, production installer
+unit packaging, live `/opt/cezen` ownership/ACL/xattrs, disk exhaustion or service-level health.
+
+## Target Capacity Exhaustion Increment 001
+
+Two fresh strict-CPU installations ran with `--network none` against executable disposable tmpfs
+target filesystems. Journal and checkpoint state were mounted separately, modeling the required
+reserved recovery capacity. Both failures occurred during offline `pip` installation before atomic
+activation; the original marker remained intact and the executor durably recorded generation 5
+`rolled_back`. A separate read-only container verified each terminal journal.
+
+Byte-exhaustion vector:
+
+- Target capacity: 134,217,728 bytes; 16,467,981 inodes.
+- Failure: `ENOSPC` / `No space left on device`.
+- Journal SHA-256:
+  `12f4fc3d9e6f52c9961b508f3c47782cf14f43211196378e3009db61c9e622d0`.
+- Checkpoint manifest SHA-256:
+  `27832688460a1c58fbeac5df8ef7ee4ab158ccdcfc929a43b5c192dd07bb8964`.
+- Log SHA-256:
+  `7c0a4d4c56f58c1bcabb71ddc727163777516c18a7ea0ea9371fe9399dcd2b7c`.
+
+Inode-exhaustion vector:
+
+- Target capacity: 2,147,483,648 bytes; exactly 2,000 inodes.
+- Failure: `ENOSPC` while creating package metadata.
+- Journal SHA-256:
+  `0353ef7a66d0ac0d7c7e2703f7da68874c127731248875fdcfab976565ada2d3`.
+- Checkpoint manifest SHA-256:
+  `237ed9ca40a40633fbb69b4926719714ad8fd8d756e087eba1ac2ca3e2fe7242`.
+- Log SHA-256:
+  `fd2aefb954162454150f1dd2c5e5d90ae644b787279ec90414e25f02ca00a67f`.
+
+Artifact directory entries remained 0. This closes disposable target-byte and target-inode exhaustion
+before activation. It does not prove exhaustion after activation, exhaustion of the reserved journal/
+checkpoint capacity, production quotas/reservations, live paths/metadata or service-level rollback.
+
+## Post-Activation Service-Health Rollback Increment 001
+
+A fresh strict-CPU environment was installed from the 196-wheel selection with `--network none`,
+passed built-in runtime health, was fsynced and atomically activated. A direct executable service-health
+probe then verified that the activated target had an executable Python and the selected profile was
+`cpu-only`, wrote an observation receipt and deliberately exited `23` before the `installed` journal
+transition.
+
+The executor recorded `rollback_required`, restored the exact checkpoint, retained the activated
+environment with executable Python in quarantine and recorded generation 5 `rolled_back`. A separate
+read-only container re-verified the terminal journal, marker and quarantine.
+
+- Probe observed activated environment: pass; exit `23`.
+- Original marker restored: pass.
+- Final journal SHA-256:
+  `52e3f0313acc9291e79aff52843e956d247758e1a9a81f2f17d8eedcfed76100`.
+- Checkpoint manifest SHA-256:
+  `93d4bf2bef768c9431831dd63689604e2978155584bcb86261626d83a3761ddd`.
+- Probe observation SHA-256:
+  `fcef70518b5c9d61467cd211ade22eebd86876cb8f752a562a2aecb34594c9d5`.
+- Installation/rollback log SHA-256:
+  `a40728e335f97864de8a47e9f96bd3315907915bd7a8d173372f1af3c5489b3d`.
+- Artifact directory entries: 0.
+
+This closes an external post-activation service-probe failure for a disposable CPU environment. It
+does not prove production probe signing/packaging, systemd dependency ordering, the actual Nexus API/
+portal/model service stack, consecutive health-window behavior, customer traffic draining or live
+path/metadata rollback.
+
+After T162 synchronization, the complete build-host deployment suite passed 31 tests with no skips or
+failures. The suite log SHA-256 is
+`6b89ad62a58f1c160ef97a5ad72e946c6742d00503bbdd9e33f15eebc3e8f6ae`; artifact entries remained 0.
+
+## Isolated Systemd Dependency/Health-Window Increment 001
+
+Six uniquely named temporary systemd units modeled model runtime, vector store, API, proxy, a bounded
+health window and rollback. The four mock services used unused loopback ports and disposable staging;
+no existing Nexus unit, port, path or service was stopped or modified. Systemd inspection confirmed
+the API required and ordered after model/vector, the proxy required and ordered after API, and the
+health window required and ordered after proxy with `OnFailure` bound to rollback.
+
+The positive run completed three consecutive samples across four services: 12/12 semantic JSON health
+checks passed. A fresh failure run made only the vector endpoint return 503. The health window exited
+`23`, systemd triggered the rollback unit, and the dispatcher restored the verified checkpoint from
+generation 3 `installing` to generation 5 `rolled_back`.
+
+- Original marker restored: pass.
+- Candidate marker retained in quarantine: pass.
+- Journal SHA-256:
+  `ecc1d85eeae2d3be1d36ab2ca6d9583c3c3c9730f436f3e879c45ac7aad1d5ca`.
+- Checkpoint manifest SHA-256:
+  `ab6886f7541ab4b138b230949093caadce048ef88213fd7b38db975ffd9a5b0d`.
+- Positive 12-check window SHA-256:
+  `76b5349d2c06c4129316cce8e4e86b8977c469edb3d60f52b4893000cad8ced0`.
+- Forced-failure window SHA-256:
+  `eddfdead8f85e3299bde8e45f90b229cb22fb7ff2f9bacf3e2ca77b0fc571510`.
+- Systemd health/rollback log SHA-256:
+  `80844a1b67462f889ce6073d3e9bad90bd24a9f3a4626cffc492784d732d425f`.
+- Artifact directory entries: 0.
+
+All temporary units were stopped, removed and followed by `daemon-reload`. The host retained its static
+`172.16.10.144/24` profile. This proves isolated systemd dependency ordering, a three-sample health
+window and failure-triggered rollback; it does not validate the real Nexus services or production
+units. Repository inspection shows `cezen-api.service` only orders after Ollama, does not require it,
+does not order/require ChromaDB and does not encode proxy dependency or a health window.
+
+## Candidate Production Unit Design/Validation Increment 001
+
+Candidate-only unit definitions now live under `contracts/systemd/`; no file was copied to
+`/etc/systemd/system`, no daemon reload occurred and no runtime service was changed. The candidate API
+requires/orders after Ollama and ChromaDB. The nginx drop-in deliberately uses `Wants` plus `After` so
+static recovery/help remains available during API failure, while the transactional activation-health
+unit requires the complete graph and binds failure to the rollback unit. A root-owned active-operation
+environment contract supplies validated operation, timing, evidence and recovery paths.
+
+Ubuntu 22.04 systemd 249 validation had two layers:
+
+- Direct candidate verification failed only because this build host lacks the proposed `/opt/cezen`
+  executables and `nginx.service`; those are real environment prerequisites, not parser substitutions.
+- Structural verification used copies in `/tmp` with only executable paths replaced by `/bin/true` and
+  a temporary nginx base stub merged with the unchanged drop-in. It exited 0 with no candidate syntax
+  error or dependency cycle. The emitted Netplan/Snap/AnyDesk messages refer to unrelated host units.
+- Offline `systemd-analyze security` scoring was not accepted by systemd 249 and no units were installed
+  merely to obtain a score; hardening remains subject to production-host validation.
+
+Candidate SHA-256 values at validation:
+
+- `cezen-api.service`: `7824c1642310570c49e0e4a1a5a7f5b2f269eecdbff86bb7867e4339113646c0`.
+- `chromadb.service`: `e903c1df058ed9e5baf9584965c4d838fe8098eea6403fca18c8581c1eb788d0`.
+- `nexus-one-activation-health.service`:
+  `c3b4229a42cc74dd7323a3501e938490594daadb2b22c770c71ae74621216b5e`.
+- `nexus-one-activation-rollback.service`:
+  `2c967122edc5602a767df59787969cd0dcdaa4fc3cd1c9dbf057cea58a631019`.
+- `nginx-cezen-ordering.conf`:
+  `0a8ad3528adaa695c8259893143dda062b98cb82c342fded8643419ef2bfba3f`.
+- Final structural-validation log:
+  `d506c14455bf541f876bd925dc44410fc2085553f864fbca72d9fb0437b874d8`.
+- Artifact directory entries: 0.
+
+This closes design and structural/cycle validation only. Missing health-window implementation bytes,
+unit/script signing, real service executables, category-resolved graph generation, approved timing,
+actual service behavior, restart/timeout tests and production security scoring remain blockers.
+
+## Deterministic Activation Health Window Increment 001
+
+`activation-health-window.py` implements the candidate unit's loopback-only semantic window. It
+validates a closed component profile, rejects hostname or non-loopback URLs, bounds samples/interval/
+per-request timeout and response bytes, requires exact service identity/status JSON and publishes
+canonical mode-`0600` evidence with file and directory fsync plus atomic rename. Required degradation
+fails; optional degradation requires an explicit invocation flag. Stable exits are 0 for accepted,
+23 for semantic rejection/rollback trigger and 2 for invalid configuration or evidence I/O.
+
+Five focused local tests cover consecutive ready samples, explicit optional degradation, timeout,
+malformed response, non-loopback/hostname rejection, mode-`0600` evidence and exit-23 behavior. A real
+loopback HTTP integration then ran inside a digest-pinned Python container with `--network none`:
+
+- Ready: 3 samples, exit 0, outcome `ready`.
+- Optional degraded with explicit policy: 3 samples, exit 0, outcome `degraded`.
+- Timeout: rejected at sample 1, exit 23, code `TIMEOUT`.
+- Malformed JSON: rejected at sample 1, exit 23, code `MALFORMED_JSON`.
+- Forced service failure: rejected at sample 1, exit 23, code `HTTP_503`.
+
+Evidence SHA-256 values:
+
+- Tool: `8fefbbc3ce4fe6034ad1bb4d9710a2b43c03e6dee8775514ab4733daacdffc15`.
+- Updated candidate health unit:
+  `4c0d8cf9969f540376765cd94a5d9a82ee3e34b575d4d21ddd359a2e05d4c91a`.
+- Integration log:
+  `d0bd31faaaca659e6f16356bc20e136a6ea786df5e2a97bd688e96e420751e23`.
+- Integration summary:
+  `bba6ff23b07cda8655153d18a0adba2f96902a99f01fb2dd03fcc61752459d8d`.
+- Ready / degraded / timeout / malformed / 503 evidence respectively:
+  `9b73ff97f500f5ac269e29c7cfb22abf011d4bc09333b6742f43bcb29e7108bf`,
+  `f629349943e8742b154506ef86e9b0340cdf838fad0cc73419368c814b415bea`,
+  `5e72919d09a55e9374124caad7bd891d303022d4eb95e0938e34164a788a514d`,
+  `2b523f7bcad31db6b8298f26767b9f94d6d7c31e350b1eb81eef55e46d9fa3d1`,
+  `067de6c45079dd2d40e521e473af4df1b1abe93974c3c38cc6277f8e4bf7e76d`.
+- Artifact directory entries: 0.
+
+This closes deterministic tool behavior and disconnected loopback parsing only. Signed production
+health profiles, approved timing, real Nexus semantic endpoints, restart storms, total-window deadline,
+unit/script installation and production security scoring remain open.
+
+The updated candidate unit was rechecked with the correctly named `/tmp` nginx stub; systemd 249
+structural verification exited 0 with no candidate error/cycle. After final synchronization, the full
+build-host deployment suite passed 36 tests with no skips or failures. Suite log SHA-256:
+`4a92a77d516ae9f59cdb724ea798b125d65d3e58439065c5066045db693f85c4`.
+Artifact entries remained 0.
+
+## Total-Window Deadline and Restart-Storm Increment 001
+
+The health validator now has a monotonic total-window deadline distinct from its per-request timeout.
+Every request is capped by remaining window time; an expired deadline or an interval that makes the
+next mandatory sample impossible rejects with `WINDOW_TIMEOUT`. Component profiles also bind a literal
+systemd unit and maximum restart count. Before HTTP, a fixed `systemctl show` query validates
+`NRestarts`, `ActiveState` and `SubState`; excess restarts reject as `RESTART_STORM` before a stale
+endpoint can be accepted.
+
+Seven focused tests pass, including deterministic fake-clock deadline and restart-before-HTTP vectors.
+Two uniquely named temporary systemd services then provided real build-host evidence:
+
+- A healthy loopback service passed its systemd state check, but a 3-sample/1-second-interval window
+  with a 0.5-second total deadline rejected after sample 1 with exit 23 and `WINDOW_TIMEOUT`.
+- A failing service configured with `Restart=on-failure` reached `NRestarts=8`; a signed-profile limit
+  of 1 rejected before HTTP with exit 23 and `RESTART_STORM`, recording restart count 8.
+- Deadline evidence SHA-256:
+  `e1e26cbc66d5d575bf67f1a440b3e7b02179618cbd91c82f98a6edb175462231`.
+- Restart-storm evidence SHA-256:
+  `1f31898643775a55c7f672e375993f696877d6e7c3a8aa032b99a680e80a5063`.
+- Deadline and storm CLI logs SHA-256 (each):
+  `321d974cadea4ad833044190c40514041284a61e303ef3437ce82091d9ddb239`.
+- Restart-storm systemd log SHA-256:
+  `dae95e3fbb0ebbc7e80babef8c5a33a1f52138d1a4c9f664f4f4a9439c4ab408`.
+- Updated tool SHA-256:
+  `c152c7a9ab946a7566d6cf022962a616f54e3aff51ca83b06b872b335152f468`.
+- Updated candidate health unit SHA-256:
+  `3b81ee483a70e120a826bcb99d83b73a329c9b793186c0e78afc96005427bc35`.
+- Artifact directory entries: 0.
+
+Both temporary units were removed and daemon state reloaded; static networking remained unchanged.
+This closes isolated total-deadline and restart-counter behavior. Signed production limits, restart
+counter reset/baseline semantics across activation, real-unit storms and complete rollback chaining
+for these two codes remain open.
+
+The updated candidate unit passes `/tmp`-only systemd 249 structural/cycle validation. After final T166
+synchronization, the complete build-host deployment suite passed 38 tests with no skips or failures;
+suite log SHA-256:
+`a5378c125ce65d03ca1f96054feeefdadf2e01b3dfb23b2e2330f5342f10477c`.
+Artifact entries remained 0 and no `nexus-test-*` unit remained installed.