0 Fork 0

Files

T

Jino Jose 4f1419ca93 release: freeze Nexus One AI 1.0.0-rc1

2026-08-17 11:05:19 +05:30

43 KiB

Raw Permalink Blame History

Offline Workstation Gate 2 Build-Host Handoff

Date: 2026-07-14
Host: 172.16.10.144 (acc-srv)
Workspace: /mnt/nvme0/nexus-one-ai-offline-workstation/
Scope: Gate 2 preparation; no new build or ISO claim.

Prior Run Disposition

No matching build, bundle, ISO, squashfs, container-pull or model-pull process was running when inspected. The prior workspace was preserved rather than deleted:

/mnt/nvme0/nexus-one-ai-offline-workstation-legacy-20260714/

The legacy payload is approximately 30 GB and is not trusted as Gate 2 input. It contains mutable latest/main container exports, an unpinned Miniconda installer, mixed CUDA assumptions, partial root-owned APT state and a source copy without Git metadata. It remains quarantine/reference evidence only and MUST NOT be merged into a candidate payload by filename.

Clean Workspace

Created with mode 0750 and separate source, staging, artifacts, logs, and evidence directories. The current local source/spec snapshot was synchronized while excluding .git, ISO files, caches, virtual environments, node_modules, and unrelated how-it-flows.gif.

Local committed base at synchronization: bea082a2014a3edc31cb006108b952c8adbbc8bc
Synchronized files: 219
Remote source checksum-index digest: a9a0150f656e142c32e3962058ed83ef38eaa0685ff6c233e41d2896dbb84389
Checksum index: /mnt/nvme0/nexus-one-ai-offline-workstation/evidence/source-sha256sums.txt
Exclusion check: pass; no .git, *.iso, or how-it-flows.gif found in synchronized source.

The snapshot includes local uncommitted specification work. Therefore the committed base alone is not its identity; the checksum index is the authoritative transferred-source identity for this handoff.

Host Baseline

Ubuntu 22.04 AMD64
Kernel 6.8.0-124-generic
/mnt/nvme0: 596 GB total, 567 GB available at inspection
Available: Docker, xorriso, mksquashfs, jq, Python 3
Not present: podman, skopeo, crane, oras

No build was started during this handoff. Gate 2 must first resolve immutable package/image/runtime/ model inventory and correct C1-C4/C7. Starting the legacy bundle builder unchanged would violate the approved architecture.

Immutable-Lock Preflight Increment

The synchronized source now includes validate-payload-lock.py, an intentionally unresolved payload-lock.candidate.json, standard-library tests, and a hard preflight at the start of build-bundle.sh. The legacy mutable acquisition body is unreachable even after a lock passes until lock-driven acquisition is implemented.

Local evidence:

Three unittest cases pass: unresolved candidate rejection, structurally complete test-lock acceptance, and mutable OCI/duplicate-target rejection.
bash -n autoinstall/offline/build-bundle.sh: pass.
Candidate validation: expected fail with unresolved/status/object-group reasons.

Build-host evidence:

Preflight exit: 1 (expected fail closed).
Artifact/staging entry count before: 0; after: 0.
No acquisition, output deletion, payload assembly or ISO build occurred.
Logs: /mnt/nvme0/nexus-one-ai-offline-workstation/logs/gate2-preflight.stdout and gate2-preflight.stderr.

Resolver Increment 001

The connected resolver created a deterministic source archive and resolved six Linux/AMD64 OCI manifests without pulling image layers or creating an artifact.

Candidate lock: /mnt/nvme0/nexus-one-ai-offline-workstation/evidence/payload-lock.resolution-001.json
Candidate lock SHA-256: 146719ad9de2c99f402dfb15e16c36a029328219e53cc3f2c8ca835d4d01e51d
Source archive SHA-256: 6c890fc6537f2ed51cc40a9930a1bad1d3db78c31b093259fb81f5638cf5aade
Source archive reproducibility: pass; two independent archives are byte-identical.
OCI manifests resolved: 6; artifact directory remains empty.
Unresolved entries after this increment: 9.

The result correctly remains a candidate. Open WebUI returned no direct config/layer byte total in the selected Docker manifest response, two container licenses remain review-required, mutable source tags must be replaced with approved release tags, and exported OCI archive bytes/digests require actual digest-pinned acquisition. Registry resolution also demonstrated drift: current resolved digests differ from the quarantined run for mutable/tag-based inputs. The quarantined digests are evidence of drift, not approved pins.

APT closure, Python closure, Ubuntu media, Miniconda/replacement runtime, Ollama runtime and exact model files/notices remain unresolved. No payload or ISO build has started.

APT Resolver Increment 001

The APT resolver used an empty dpkg status with authenticated package acquisition and --no-install-recommends. This prevents already-installed build-host packages from disappearing from the computed clean-system closure. It wrote downloads only under Gate 2 staging and installed nothing.

Requested packages: 26
Resolved .deb objects: 487
Measured bytes: 357,993,500
Candidate lock: /mnt/nvme0/nexus-one-ai-offline-workstation/evidence/payload-lock.resolution-002.json
Candidate lock SHA-256: 351c842ab92995315b008865a3080a5b1469f002cdc235702fff28cdf7bcf518
Artifact directory: empty
Candidate validator: expected fail; Python/runtime/model groups, Open WebUI measured bytes, reviewed status and unresolved list remain incomplete.

This closes only the measured Ubuntu/deadsnakes .deb byte closure for the currently requested list. The APT repository index/origin snapshot and its digests, package license/notices, and the separately approved R570/CUDA 12.8.1/cuDNN/NVIDIA Container Toolkit closure remain unresolved. Current repository URLs are not yet accepted as reproducible release identity.

Python Resolver Increment 001 — Boundary Correction and Candidate Closure

The first Python 3.11 wheel-only run combined backend and platform requirements and correctly failed dependency solving:

Backend constraint: bcrypt<4.0.0.
Platform constraint: chromadb==0.5.23, whose declared dependency is bcrypt>=4.0.1.
Resolver result: ResolutionImpossible; exit 1.
Candidate payload-lock.resolution-003.json: absent.
Downloaded wheel files in that run: 0.
Artifact directory entries: 0.
Failure log: /mnt/nvme0/nexus-one-ai-offline-workstation/logs/python-resolution-001.stderr.log.
Failure-log SHA-256: 28fe44f5cab1d41d5ebf92322819b006c92eb13df79879b579e93260388f9b61.

Repository inspection then proved those inputs belong to separate runtime boundaries: the backend and Chroma Ansible roles create different virtual environments, and the legacy bundle layout already has separate backend, chromadb and platform wheelhouses. The conflict was therefore a resolver-boundary defect, not an application dependency contradiction. Neither dependency constraint was changed.

The corrected resolver produced three independent closures:

Backend: 61 wheel objects.
ChromaDB: 88 wheel objects.
Platform: 214 wheel objects.
Total: 363 wheel objects; 3,238,751,953 measured bytes.
License fields requiring review: 49.
Candidate lock: /mnt/nvme0/nexus-one-ai-offline-workstation/evidence/payload-lock.resolution-003.json.
Candidate lock SHA-256: cde0f17371dde68af984511114a2f2fca449f501ecbbdd622f48c92c6508b261.
Successful resolver log SHA-256: 7d6b49f9f985b47190fd7b6c0fc6f00c72b80938e1905fed3b4a4524791c82cc.
Artifact directory entries: 0.
Candidate validator: expected fail because the lock is not reviewed, has 14 unresolved entries, and still lacks complete runtime/model objects and measured Open WebUI bytes.

The closure is evidence, not an approved payload. The unpinned platform requirements selected current Torch/CUDA dependencies including four explicitly CUDA-13-named objects, which conflicts with the approved CUDA 12.8.1 baseline. Gate 2 remains blocked on an explicitly pinned CUDA-12.8-compatible Python ML stack, index snapshot/origin and resolver-report approval, license/notice review, and clean Python 3.11 offline installation/import evidence for each independent environment.

Python Resolver Increment 002 — CUDA 12.8-Compatible Candidate

Authoritative vLLM 0.10.2 wheel metadata pins torch==2.8.0, torchaudio==2.8.0 and torchvision==0.23.0. The inspected AMD64 vLLM wheel SHA-256 is e0cba6110483d9bf25c4402d8655cf78d366dd13e4155210980cc3480ed98b7b, matching the published PyPI file digest. Those three versions were added to the platform input and resolved in a fresh staging directory; the previous CUDA 13 candidate remains preserved for comparison.

Backend: 61 wheel objects.
ChromaDB: 88 wheel objects.
Platform: 212 wheel objects.
Total: 361 wheel objects; 4,435,239,499 measured bytes.
Resolved Torch/Torchaudio/Torchvision: 2.8.0 / 2.8.0 / 0.23.0.
Explicit CUDA 13 objects: 0.
Explicit CUDA 12 objects: 14.
License fields requiring review: 49.
Candidate lock: /mnt/nvme0/nexus-one-ai-offline-workstation/evidence/payload-lock.resolution-004.json.
Candidate lock SHA-256: 3b1fef9705e8049b28d283383f16fa7fd66e7fc00434315b2cc86163ba8e93ac.
Successful resolver log SHA-256: 37711950c25a1b2409e1323ea35ce7f6cdab2f0a9e6d2aad9407baeda1325c35.
Artifact directory entries: 0.
Candidate validator: expected fail because review/unresolved/runtime/model/Open WebUI evidence is still incomplete.

This closes the version-selection contradiction only. It does not yet prove execution against the approved R570/CUDA 12.8.1 physical profiles, clean disconnected Python 3.11 installation/import, license/notice acceptance, or NVIDIA-independent CPU operation. The universal Workstation installer also needs a profile-aware installation contract so CPU-only systems do not depend on NVIDIA packages merely because the signed universal media carries them.

Python Validation Increment 001 — Disconnected Install and GPU-Lite Smoke

Validation used official python:3.11.13-slim-bookworm on Linux/AMD64, pinned after acquisition to python@sha256:86adf8dbadc3d6e82ee5dd2c74bec2e1c2467cdad47886280501df722372d2e1. Each installation ran in a disposable container with --network none, read-only wheel mounts and pip --no-index --find-links; no package index or other network fallback was possible.

Backend wheelhouse: pass. Imports succeeded and Passlib/bcrypt hash/verify succeeded.
ChromaDB wheelhouse: pass; chromadb==0.5.23 imported successfully.
Platform wheelhouse CPU path: pass. Torch/Torchaudio/Torchvision reported 2.8.0+cu128 / 2.8.0+cu128 / 0.23.0+cu128; torch.cuda.is_available() was false with CUDA hidden and CPU tensor execution returned the expected result.
Supplemental gpu-lite path: pass on RTX 3050 OEM with 8,222,670,848 bytes reported device memory. Torch reported CUDA 12.8 and completed synchronized GPU tensor execution.
Host driver for that supplemental run: NVIDIA Open Kernel Module 595.58.03, license Dual MIT/GPL.
Artifact directory entries: 0.

Evidence log SHA-256 values:

Backend: 1d9be80fef4c565e13a9b12d00cc76a049441eb89d2136cb0d9e7703494a490a.
ChromaDB: 6b4a907b6cee363a1260be8d70789cf7ab1bb38702917fb7b4e0ee60132d4982.
Platform CPU: 35eb2e5861e98158e934771d1256567e3b68d23adb51a7fb6206a5af71853719.
GPU-lite: b51584484bf0317f8a79058f91b5613dd1b0f7a7d17e07856a43811652721fcb.

This proves disconnected wheel availability/importability and one supplemental 8 GB GPU smoke test. It does not prove a clean bare-metal install, the exact R570 reference driver, 24 GB/48 GB profiles, vLLM model serving, interruption/recovery, or selective CPU-profile installation without NVIDIA packages. Those claims remain blocked, as do index provenance approval and 49 license/notice reviews.

Python Profile Selection Increment 001 — Strict CPU Boundary

The universal-media/profile-selective contract now defines exactly-one selection between cpu-only and nvidia-cu128. PyTorch's official 2.8.0 index matrix supplies distinct CPU and CUDA 12.8 builds. The CPU trio was acquired from the CPU index into staging, then combined with the reusable non-GPU platform wheels while excluding NVIDIA, CUDA, Triton and bitsandbytes objects.

CPU selected wheels: 196; measured bytes: 385,850,558.
Forbidden wheel count: 0.
CPU Torch SHA-256: cb06175284673a581dd91fb1965662ae4ecaba6e5c357aa0ea7bb8b84b6b7eeb.
CPU Torchaudio SHA-256: e54bd7fc9472019308097d99102df9acee22aa2451ae808d27840bc874320292.
CPU Torchvision SHA-256: d83d8075db43b8ca89680bdeb2f100c832e2a3aa61ee42c038b1a146e5e511b6.
Network-disabled full CPU-profile install/import: pass; Torch 2.8.0+cpu, torch.version.cuda is None, CPU tensor result correct.
Strict CPU validation log SHA-256: 57e2ab86f6fae188502c019429b37ab03ed2f8bad8fdd1330ec39fd17f69173e.
Machine-readable profile validator: pass, 196 wheels; log SHA-256: 2f937161184294ea8c8e2413282db006498aca52517927f305fd3371e3defe14.
Artifact directory entries: 0.

The contract requires hardware-detection failure to default to CPU, records selection before mutation, reuses it on resume, and requires rollback before profile change. This proves the Python payload boundary, not the eventual installer's hardware detector, journal, rollback or signed membership.

Python Profile Planner Increment 001 — Non-Mutating State Machine

The new planner consumes a normalized hardware probe, requested mode, operation ID, optional prior journal and a profile-membership verification receipt. It does not mutate the host. Auto-selection chooses NVIDIA only when presence, support and driver readiness are all explicitly true; incomplete or failed facts choose CPU. An active operation can only resume the same profile and operation ID. Profile change requires rollback, and a rolled-back operation ID cannot be reused.

Dependency-free planner tests: 6 pass; combined deployment tests: 11 pass.
GPU-lite structural start vector: selected nvidia-cu128; pass.
Resume vector: retained the journaled nvidia-cu128 selection; pass.
Mid-operation CPU change vector: fail closed with PROFILE_CHANGE_REQUIRES_ROLLBACK; pass.
Planner-vector log SHA-256: a5d56b4ec355ac701ec3cd72f05298dc6cb97bd49b1c20fdc3fc548a994635f5.
Artifact directory entries: 0.

The membership receipt used here is explicitly marked structural-test-only. The planner trusts a future signature verifier's receipt; it does not verify cryptography. Production integration remains blocked until real signature verification, atomic durable journal writes, pre-mutation checkpoint creation, installer wiring and power-loss/rollback tests exist.

Profile Verification Receipt Increment 001

A closed Draft 2020-12 receipt schema now binds verified profile membership to the exact manifest and envelope digests, key ID/fingerprint, trust snapshot, policy version, namespace, profiles, verification time and verifier build. The planner no longer accepts a bare verification boolean. It requires the receipt schema/result/namespace and rejects non-production receipts unless explicit test mode is set.

Combined deployment tests: 12 pass, including production rejection/explicit test acceptance.
Existing schema/signature/compatibility fixture suite: pass in a network-disabled Python 3.11 container using staged wheels.
Fixture-suite log SHA-256: 6008edf74f86f181d84221ca49887751e047af87069ce9e451ecd336a7a40109.
Production-mode structural test receipt: rejected with TEST_RECEIPT_FORBIDDEN; expected.
Explicit structural-test mode using the same receipt: accepted; expected.
Receipt-boundary log SHA-256: aa2369d4058fef580ef55f9cb8e7b8ced823b551503cc39a8fad9fca311b5191.
Artifact directory entries: 0.

This defines and tests the verifier/planner handoff but does not implement the production verifier, trust-policy evaluation or production key ceremony. Atomic journal/checkpoint persistence and installer integration remain open.

Atomic Profile Journal Increment 001

The structural journal writer now uses exclusive locking, expected-generation compare-and-swap, canonical per-record digests, prior-record linkage, immutable operation/profile/manifest identity, mode 0600, file fsync, atomic rename and directory fsync. Installation cannot enter installing before a checkpoint_ready record with a checkpoint identity exists.

Atomic journal unit tests: 5 pass; combined deployment tests: 17 pass.
Build-host generations 1 profile_selected, 2 checkpoint_ready, and 3 installing: pass.
Stale generation-1 writer after generation 2: JOURNAL_GENERATION_STALE; expected.
Persisted journal mode: 0600.
Final journal SHA-256: 35b3940893a27c4a09ac3d532fe992d23ec59396fa8f6883fc809f2df3803488.
Transition-vector log SHA-256: 48d2aabf56b882975ed539007fe3b0ef9f6cb4674c2cd68affdc7646214adbb1.
Artifact directory entries: 0.

Tests also cover digest tampering, identity change, missing checkpoint, invalid transition and terminal rollback. The writer does not create/verify a real filesystem/package checkpoint, and no forced power-loss test has yet proved filesystem-specific durability. Production signature verification, checkpoint implementation, crash injection and installer integration remain open.

Ed25519 Profile Verifier Increment 001

The production-shaped verifier now canonicalizes the profile manifest, checks namespace and digest, enforces envelope algorithm/policy, resolves one active purpose/environment-scoped trust key, checks the public-key DER fingerprint, verifies Ed25519 with OpenSSL and emits the closed membership receipt.

Ubuntu/OpenSSL 3 combined deployment tests: 19 pass.
Positive Ed25519 path used an ephemeral test key generated only inside a disposable test directory.
Negative paths include manifest tampering and wrong key environment.
Verifier-test log SHA-256: b61d719273b7e755fd76aff8603f81e62b0449fde813eb01f7e2b0850633319c.
Artifact directory entries: 0.

The Mac OpenSSL build lacks Ed25519 and correctly skips the two crypto tests; authoritative crypto evidence is the Ubuntu/OpenSSL 3 run. No private test key entered the repository or logs. This is not a production ceremony or release signature: production public keys, trust snapshot, revocation/time policy, locked OpenSSL/verifier build and independent signing approval remain absent.

Python Environment Checkpoint Increment 001

The checkpoint component now records whether the target existed and inventories directory, file and symlink state with modes, byte sizes, hashes and link targets. Creation and restore use same-filesystem temporary trees, file/directory fsync and atomic rename. Restore verifies the checkpoint before use and quarantines the failed/new tree rather than deleting it.

Checkpoint unit tests: 3 pass; combined local suite: 20 run, 19 pass and the unsupported Mac Ed25519 test class skips.
Build-host create/verify/modify/restore vector: pass; original file SHA-256 restored exactly.
Checkpoint inventory SHA-256: 488bfb0e7f67d75553b66ccbf981cb9139ef1b8610f1018d76e4832a5c748787.
Checkpoint manifest file SHA-256: 82f28057d73359d9b7d6fd75b23f9ff4718def883693104b88161001b16dfeab.
Vector log SHA-256: cbbf47fd1d028635e2b2dce63c115a9066cff8df3e0fa0458d28f8691082df1d.
Artifact directory entries: 0.

This validates ordinary checkpoint/restore and tamper rejection in disposable staging. It does not yet prove crash consistency under forced termination at each fsync/rename boundary, real /opt/cezen ownership/ACL/xattr preservation, disk-full behavior or integration with journal/install mutation.

Python Checkpoint Crash-Recovery Increment 001

Test-only process termination was injected at four fsync/rename boundaries on disposable build-host trees. Every injected process exited 97 as designed:

Before checkpoint publication: no checkpoint became visible; the original target remained authoritative.
After checkpoint publication: the visible checkpoint passed complete verification.
After failed-target quarantine but before activation: recovery completed verified restore and preserved the failed tree.
After restore activation but before return: recovery recognized the exact restored inventory and completed idempotently.

Crash-vector log SHA-256: 17d0cb3f39b6c3f7114e2c81acc4e8884f96a496aa06a7788a9b380f801dc6f9. Artifact directory entries remained 0.

This is forced-process termination, not physical power removal. Filesystem/controller cache loss, disk-full/inode exhaustion, ACL/xattr preservation, journal/checkpoint orchestration and bare-metal recovery remain open.

Profile Pre-Mutation Orchestration Increment 001

The isolated orchestrator now composes structural receipt acceptance, profile planning, exact wheel validation, atomic journal selection and verified environment checkpoint. Invalid payload membership creates neither journal nor checkpoint. Valid preparation stops at checkpoint_ready and contains no package mutation action.

Orchestrator unit tests: 2 pass; combined local suite: 23 run, 22 pass plus the expected Mac Ed25519 capability skip.
Build-host strict CPU preparation: pass, generation 2, checkpoint_ready.
Immediate rerun: same checkpoint/generation, pass; mutation_performed: false both times.
Checkpoint inventory ID: 12e8f917f8a388c0a1d9b45c5f494ce759667f331dc0bdfd03f2b51bda2c9d3a.
Journal SHA-256: 1fd29a06646ab917a01937bb498945e1b8f347b1c731d78e1d825f0f521d3e60.
Checkpoint manifest SHA-256: 99d10d9ac0e48da10b70ccfff08d64af4222d6637d4e2ad0a1b22e1b41134ed1.
Evidence log SHA-256: ed3209620640105cc15373fc9759b1198af2a1d6713e4f0d76d000e1a439fcde.
Artifact directory entries: 0.

This closes component-composition only through the pre-mutation boundary. Actual offline installation, health validation, installed/rollback journal transitions, service integration and production receipt inputs remain blocked.

Disposable CPU Installation Increment 001

The offline executor was run entirely inside a network-disabled Python 3.11 container against a new container-mounted staging target. It consumed the strict 196-wheel CPU selection, not the CUDA-bearing platform set. The final-code rerun included full new-tree and parent-directory fsync barriers.

Offline dependency-failure unit vector: original target unchanged; journal rolled_back; pass.
Positive prepare/checkpoint/install/pre-activation-health/atomic-activation/post-health flow: pass.
Final journal: generation 4, installed, mode 0600.
Installed Torch: 2.8.0+cpu; compiled CUDA version: none.
Journal SHA-256: 61e77937fadaee985e568a6f78e4a5118ac5198f2f5b692a14cce6c1b1997ca5.
Checkpoint manifest SHA-256: 0ca232445458f2a48bd3319d4e56964b92e5d11fee8b9b92938b1517af157a73.
Installation log SHA-256: 941fa474229ddcb8a4e23bdc95cba27ef51390580c8f278c914332b1e47c8195.
Artifact directory entries: 0.

This proves the CPU Python environment mutation/health path only in disposable container staging. It does not prove service wiring, system ownership/ACLs, live /opt/cezen, reboot resume, NVIDIA install, post-activation forced failure, disk exhaustion or bare-metal installation.

Post-Activation CPU Rollback Increment 001

The full strict-CPU environment was installed in disconnected disposable staging, fsynced and atomically activated. Test-only termination then exited 97 immediately after activation and before post-health/ installed. Recovery consumed the generation-3 installing journal, recorded rollback_required, restored the verified pre-install checkpoint, preserved the activated environment in quarantine and recorded generation 5 rolled_back.

Original pre-install marker restored: pass.
Failed activated environment executable present in quarantine: pass.
Final journal SHA-256: 6acdfd8c8a57586ad0edae5e5a4cae0bac0a6d1055864dd35113afb8199dabad.
Checkpoint manifest SHA-256: d5c5ff6fd95743431832d115a0b4fe7de8e9ae0fa7ee36b3c349316cb9ebfc53.
Crash/rollback log SHA-256: 5791738b02f6c0a41218a94bd04e6a1cbd80dd42b04e28a14f55ab451c61f15c.
Artifact directory entries: 0.

This closes the disposable CPU post-activation process-crash case. It does not prove reboot-triggered recovery, physical power loss, service health rollback, disk exhaustion, NVIDIA rollback or live paths.

Disposable NVIDIA Installation Increment 001

The full 212-wheel nvidia-cu128 platform selection was installed inside a network-disabled Python 3.11 container with access to the build-host RTX 3050. Preparation used exact profile validation and checkpointing; install used no index, pre/post activation health, full-tree fsync and atomic activation.

Final journal: generation 4, installed, mode 0600.
Torch: 2.8.0+cu128; compiled CUDA: 12.8.
GPU: NVIDIA GeForce RTX 3050 OEM; device tensor execution and synchronization: pass.
Journal SHA-256: 92cfa8a9cab9de57c08019957c564394c51eab6257dda41be2e9d352b2a422b6.
Checkpoint manifest SHA-256: 66c9c43fd432b8aa87d9212e9cbf51619cd38db1c5c7768a0102afaf96431ee7.
Installation log SHA-256: af29e27e5cf3e55d01874bc00a895b1ce2b6e96f50e347c5890b6b70f969dd36.
Artifact directory entries: 0.

This is supplemental 8 GB gpu-lite evidence on Open Driver 595.58.03. It does not replace the exact R570 reference, 24/48 GB profiles, vLLM model-serving, NVIDIA rollback/reboot or bare-metal proof.

Post-Limit Build-Host Refresh

After the external-action quota reset, the complete current source/spec snapshot was synchronized to the clean build-host workspace. The full remote deployment suite ran under Ubuntu/OpenSSL 3:

Tests: 27 passed; 0 skipped; exit 0.
Suite log SHA-256: f9b4e43c8bd587a4c92bc64a4744d61b3b879e0408ea57056380136d35f8517c.
Artifact directory entries: 0.

The expected offline-resolution negative vector appears in the log and is followed by an overall pass. This refresh aligns the build-host copy with T156 and the latest local implementation; it does not change Gate 2 readiness or authorize assembly.

Post-Activation NVIDIA Rollback Increment 001

The full nvidia-cu128 environment was installed without network access, health-checked, fsynced and atomically activated on the RTX 3050. Test-only termination exited 97 after activation and before the final health/journal commit. Recovery restored the verified pre-install checkpoint, preserved the activated CUDA environment in quarantine and recorded generation 5 rolled_back.

Original pre-install marker restored: pass.
Quarantined environment remained executable and reported Torch 2.8.0+cu128: pass.
Final journal SHA-256: 306b3873e37b276d45af2446cdd25a9a56d4545e0a4495d327578028b84ac272.
Checkpoint manifest SHA-256: ca88bf52a19c22b90b5a445985e60faca177ad21dc67efdd2d3572e4378d7af3.
Crash/rollback log SHA-256: e0b01f43762fc518ea26557ec58b7b4c1bc6cff366890154a972df96d391711f.
Artifact directory entries: 0.

This closes disposable post-activation process-crash rollback for the 8 GB gpu-lite Python profile. It does not prove reboot/power-loss recovery, exact R570, 24/48 GB profiles, vLLM service/model health or bare-metal NVIDIA installation.

Boot-Dispatcher CPU Recovery Increment 001

A fresh strict-CPU operation ran in a network-disabled container and was forcibly terminated with exit 97 after atomic activation, leaving the verified journal at generation 3 installing. A second, separately created network-disabled container invoked recover-python-profile-on-boot.py, simulating a fresh privileged boot-service process rather than an in-process exception handler.

Pre-recovery journal: generation 3, installing.
Post-recovery journal: generation 5, rolled_back.
Original marker restored: pass.
Activated environment preserved with executable Python in quarantine: pass.
Second dispatcher invocation on terminal state: safe no-op.
Journal SHA-256: f5c60f693fd9ecf572e35d58390c596a71916cc1dac4f44a698aa67c81d1c0d9.
Checkpoint manifest SHA-256: e6c0ed458c8ed785a0d611fb4b6da0353b9632e008fbee2060c6b8e151b500c0.
Combined crash/recovery log SHA-256: ab9d9a23b9c946a9ee7b86440584f06038988d09a701ef7d8bd10cc5ae2f7944.
Artifact directory entries: 0.

This proves recovery across distinct disposable process/container lifetimes. It does not prove an actual OS reboot, abrupt physical power loss, production boot ordering, live path/ownership/ACL/xattr behavior or service-level health recovery; those evidence gates remain open.

After T159 documentation and tests were synchronized, the complete remote deployment suite passed 29 tests with no skips or failures. The suite log SHA-256 is 0060f0cc33d37e2d35165908109e76e5cc8e9909bd2a40dbd989fe1baac25da5; the artifact directory remained empty. The expected impossible-package negative-vector output is followed by the overall OK result.

Controlled Reboot Recovery Increment 001

Before reboot testing, the build host was found to use DHCP on enp8s0. Its active values were preserved exactly in the persistent NetworkManager profile and changed to manual assignment: 172.16.10.144/24, gateway 172.16.10.1, DNS 8.8.8.8, autoconnect enabled. After both controlled reboots, the profile remained manual and the default route reported proto static.

The first reboot vector failed closed at rollback_required generation 4 with CHECKPOINT_TARGET_MISMATCH: its checkpoint identity used container path /state/python-env, while the initial systemd harness supplied the corresponding host path. No restore was attempted and no artifact was produced. This negative result confirms exact target-path binding and is not counted as a successful recovery.

A fresh vector then used the same /state mount namespace for checkpoint creation and boot recovery. The test-only systemd one-shot was ordered after local filesystems and Docker and before multi-user.target; its recovery container used --network none and the digest-pinned Python image. After a distinct boot ID, systemd reported exit 0 and the dispatcher recovered generation 3 installing to generation 5 rolled_back.

Static address after reboot: 172.16.10.144/24; gateway 172.16.10.1; DNS 8.8.8.8.
Original marker restored: pass.
Activated environment preserved with executable Python in quarantine: pass.
Final journal SHA-256: f5c60f693fd9ecf572e35d58390c596a71916cc1dac4f44a698aa67c81d1c0d9.
Checkpoint manifest SHA-256: e6c0ed458c8ed785a0d611fb4b6da0353b9632e008fbee2060c6b8e151b500c0.
Static-network evidence SHA-256: 25453d9f8129abcf840a973bf650c9da896050296149e9cfab52a6082182a0f9.
Boot-service journal SHA-256: b03b15ec77223eeae99932db25c554ce68b0f71688bf33390174955d24f0e163.
Artifact directory entries: 0.

The temporary systemd unit was disabled, removed and daemon state reloaded after evidence capture; the static NetworkManager profile intentionally remains. This closes controlled OS-reboot recovery for a disposable strict-CPU target. It does not prove sudden physical power removal, production installer unit packaging, live /opt/cezen ownership/ACL/xattrs, disk exhaustion or service-level health.

Target Capacity Exhaustion Increment 001

Two fresh strict-CPU installations ran with --network none against executable disposable tmpfs target filesystems. Journal and checkpoint state were mounted separately, modeling the required reserved recovery capacity. Both failures occurred during offline pip installation before atomic activation; the original marker remained intact and the executor durably recorded generation 5 rolled_back. A separate read-only container verified each terminal journal.

Byte-exhaustion vector:

Target capacity: 134,217,728 bytes; 16,467,981 inodes.
Failure: ENOSPC / No space left on device.
Journal SHA-256: 12f4fc3d9e6f52c9961b508f3c47782cf14f43211196378e3009db61c9e622d0.
Checkpoint manifest SHA-256: 27832688460a1c58fbeac5df8ef7ee4ab158ccdcfc929a43b5c192dd07bb8964.
Log SHA-256: 7c0a4d4c56f58c1bcabb71ddc727163777516c18a7ea0ea9371fe9399dcd2b7c.

Inode-exhaustion vector:

Target capacity: 2,147,483,648 bytes; exactly 2,000 inodes.
Failure: ENOSPC while creating package metadata.
Journal SHA-256: 0353ef7a66d0ac0d7c7e2703f7da68874c127731248875fdcfab976565ada2d3.
Checkpoint manifest SHA-256: 237ed9ca40a40633fbb69b4926719714ad8fd8d756e087eba1ac2ca3e2fe7242.
Log SHA-256: fd2aefb954162454150f1dd2c5e5d90ae644b787279ec90414e25f02ca00a67f.

Artifact directory entries remained 0. This closes disposable target-byte and target-inode exhaustion before activation. It does not prove exhaustion after activation, exhaustion of the reserved journal/ checkpoint capacity, production quotas/reservations, live paths/metadata or service-level rollback.

Post-Activation Service-Health Rollback Increment 001

A fresh strict-CPU environment was installed from the 196-wheel selection with --network none, passed built-in runtime health, was fsynced and atomically activated. A direct executable service-health probe then verified that the activated target had an executable Python and the selected profile was cpu-only, wrote an observation receipt and deliberately exited 23 before the installed journal transition.

The executor recorded rollback_required, restored the exact checkpoint, retained the activated environment with executable Python in quarantine and recorded generation 5 rolled_back. A separate read-only container re-verified the terminal journal, marker and quarantine.

Probe observed activated environment: pass; exit 23.
Original marker restored: pass.
Final journal SHA-256: 52e3f0313acc9291e79aff52843e956d247758e1a9a81f2f17d8eedcfed76100.
Checkpoint manifest SHA-256: 93d4bf2bef768c9431831dd63689604e2978155584bcb86261626d83a3761ddd.
Probe observation SHA-256: fcef70518b5c9d61467cd211ade22eebd86876cb8f752a562a2aecb34594c9d5.
Installation/rollback log SHA-256: a40728e335f97864de8a47e9f96bd3315907915bd7a8d173372f1af3c5489b3d.
Artifact directory entries: 0.

This closes an external post-activation service-probe failure for a disposable CPU environment. It does not prove production probe signing/packaging, systemd dependency ordering, the actual Nexus API/ portal/model service stack, consecutive health-window behavior, customer traffic draining or live path/metadata rollback.

After T162 synchronization, the complete build-host deployment suite passed 31 tests with no skips or failures. The suite log SHA-256 is 6b89ad62a58f1c160ef97a5ad72e946c6742d00503bbdd9e33f15eebc3e8f6ae; artifact entries remained 0.

Isolated Systemd Dependency/Health-Window Increment 001

Six uniquely named temporary systemd units modeled model runtime, vector store, API, proxy, a bounded health window and rollback. The four mock services used unused loopback ports and disposable staging; no existing Nexus unit, port, path or service was stopped or modified. Systemd inspection confirmed the API required and ordered after model/vector, the proxy required and ordered after API, and the health window required and ordered after proxy with OnFailure bound to rollback.

The positive run completed three consecutive samples across four services: 12/12 semantic JSON health checks passed. A fresh failure run made only the vector endpoint return 503. The health window exited 23, systemd triggered the rollback unit, and the dispatcher restored the verified checkpoint from generation 3 installing to generation 5 rolled_back.

Original marker restored: pass.
Candidate marker retained in quarantine: pass.
Journal SHA-256: ecc1d85eeae2d3be1d36ab2ca6d9583c3c3c9730f436f3e879c45ac7aad1d5ca.
Checkpoint manifest SHA-256: ab6886f7541ab4b138b230949093caadce048ef88213fd7b38db975ffd9a5b0d.
Positive 12-check window SHA-256: 76b5349d2c06c4129316cce8e4e86b8977c469edb3d60f52b4893000cad8ced0.
Forced-failure window SHA-256: eddfdead8f85e3299bde8e45f90b229cb22fb7ff2f9bacf3e2ca77b0fc571510.
Systemd health/rollback log SHA-256: 80844a1b67462f889ce6073d3e9bad90bd24a9f3a4626cffc492784d732d425f.
Artifact directory entries: 0.

All temporary units were stopped, removed and followed by daemon-reload. The host retained its static 172.16.10.144/24 profile. This proves isolated systemd dependency ordering, a three-sample health window and failure-triggered rollback; it does not validate the real Nexus services or production units. Repository inspection shows cezen-api.service only orders after Ollama, does not require it, does not order/require ChromaDB and does not encode proxy dependency or a health window.

Candidate Production Unit Design/Validation Increment 001

Candidate-only unit definitions now live under contracts/systemd/; no file was copied to /etc/systemd/system, no daemon reload occurred and no runtime service was changed. The candidate API requires/orders after Ollama and ChromaDB. The nginx drop-in deliberately uses Wants plus After so static recovery/help remains available during API failure, while the transactional activation-health unit requires the complete graph and binds failure to the rollback unit. A root-owned active-operation environment contract supplies validated operation, timing, evidence and recovery paths.

Ubuntu 22.04 systemd 249 validation had two layers:

Direct candidate verification failed only because this build host lacks the proposed /opt/cezen executables and nginx.service; those are real environment prerequisites, not parser substitutions.
Structural verification used copies in /tmp with only executable paths replaced by /bin/true and a temporary nginx base stub merged with the unchanged drop-in. It exited 0 with no candidate syntax error or dependency cycle. The emitted Netplan/Snap/AnyDesk messages refer to unrelated host units.
Offline systemd-analyze security scoring was not accepted by systemd 249 and no units were installed merely to obtain a score; hardening remains subject to production-host validation.

Candidate SHA-256 values at validation:

cezen-api.service: 7824c1642310570c49e0e4a1a5a7f5b2f269eecdbff86bb7867e4339113646c0.
chromadb.service: e903c1df058ed9e5baf9584965c4d838fe8098eea6403fca18c8581c1eb788d0.
nexus-one-activation-health.service: c3b4229a42cc74dd7323a3501e938490594daadb2b22c770c71ae74621216b5e.
nexus-one-activation-rollback.service: 2c967122edc5602a767df59787969cd0dcdaa4fc3cd1c9dbf057cea58a631019.
nginx-cezen-ordering.conf: 0a8ad3528adaa695c8259893143dda062b98cb82c342fded8643419ef2bfba3f.
Final structural-validation log: d506c14455bf541f876bd925dc44410fc2085553f864fbca72d9fb0437b874d8.
Artifact directory entries: 0.

This closes design and structural/cycle validation only. Missing health-window implementation bytes, unit/script signing, real service executables, category-resolved graph generation, approved timing, actual service behavior, restart/timeout tests and production security scoring remain blockers.

Deterministic Activation Health Window Increment 001

activation-health-window.py implements the candidate unit's loopback-only semantic window. It validates a closed component profile, rejects hostname or non-loopback URLs, bounds samples/interval/ per-request timeout and response bytes, requires exact service identity/status JSON and publishes canonical mode-0600 evidence with file and directory fsync plus atomic rename. Required degradation fails; optional degradation requires an explicit invocation flag. Stable exits are 0 for accepted, 23 for semantic rejection/rollback trigger and 2 for invalid configuration or evidence I/O.

Five focused local tests cover consecutive ready samples, explicit optional degradation, timeout, malformed response, non-loopback/hostname rejection, mode-0600 evidence and exit-23 behavior. A real loopback HTTP integration then ran inside a digest-pinned Python container with --network none:

Ready: 3 samples, exit 0, outcome ready.
Optional degraded with explicit policy: 3 samples, exit 0, outcome degraded.
Timeout: rejected at sample 1, exit 23, code TIMEOUT.
Malformed JSON: rejected at sample 1, exit 23, code MALFORMED_JSON.
Forced service failure: rejected at sample 1, exit 23, code HTTP_503.

Evidence SHA-256 values:

Tool: 8fefbbc3ce4fe6034ad1bb4d9710a2b43c03e6dee8775514ab4733daacdffc15.
Updated candidate health unit: 4c0d8cf9969f540376765cd94a5d9a82ee3e34b575d4d21ddd359a2e05d4c91a.
Integration log: d0bd31faaaca659e6f16356bc20e136a6ea786df5e2a97bd688e96e420751e23.
Integration summary: bba6ff23b07cda8655153d18a0adba2f96902a99f01fb2dd03fcc61752459d8d.
Ready / degraded / timeout / malformed / 503 evidence respectively: 9b73ff97f500f5ac269e29c7cfb22abf011d4bc09333b6742f43bcb29e7108bf, f629349943e8742b154506ef86e9b0340cdf838fad0cc73419368c814b415bea, 5e72919d09a55e9374124caad7bd891d303022d4eb95e0938e34164a788a514d, 2b523f7bcad31db6b8298f26767b9f94d6d7c31e350b1eb81eef55e46d9fa3d1, 067de6c45079dd2d40e521e473af4df1b1abe93974c3c38cc6277f8e4bf7e76d.
Artifact directory entries: 0.

This closes deterministic tool behavior and disconnected loopback parsing only. Signed production health profiles, approved timing, real Nexus semantic endpoints, restart storms, total-window deadline, unit/script installation and production security scoring remain open.

The updated candidate unit was rechecked with the correctly named /tmp nginx stub; systemd 249 structural verification exited 0 with no candidate error/cycle. After final synchronization, the full build-host deployment suite passed 36 tests with no skips or failures. Suite log SHA-256: 4a92a77d516ae9f59cdb724ea798b125d65d3e58439065c5066045db693f85c4. Artifact entries remained 0.

Total-Window Deadline and Restart-Storm Increment 001

The health validator now has a monotonic total-window deadline distinct from its per-request timeout. Every request is capped by remaining window time; an expired deadline or an interval that makes the next mandatory sample impossible rejects with WINDOW_TIMEOUT. Component profiles also bind a literal systemd unit and maximum restart count. Before HTTP, a fixed systemctl show query validates NRestarts, ActiveState and SubState; excess restarts reject as RESTART_STORM before a stale endpoint can be accepted.

Seven focused tests pass, including deterministic fake-clock deadline and restart-before-HTTP vectors. Two uniquely named temporary systemd services then provided real build-host evidence:

A healthy loopback service passed its systemd state check, but a 3-sample/1-second-interval window with a 0.5-second total deadline rejected after sample 1 with exit 23 and WINDOW_TIMEOUT.
A failing service configured with Restart=on-failure reached NRestarts=8; a signed-profile limit of 1 rejected before HTTP with exit 23 and RESTART_STORM, recording restart count 8.
Deadline evidence SHA-256: e1e26cbc66d5d575bf67f1a440b3e7b02179618cbd91c82f98a6edb175462231.
Restart-storm evidence SHA-256: 1f31898643775a55c7f672e375993f696877d6e7c3a8aa032b99a680e80a5063.
Deadline and storm CLI logs SHA-256 (each): 321d974cadea4ad833044190c40514041284a61e303ef3437ce82091d9ddb239.
Restart-storm systemd log SHA-256: dae95e3fbb0ebbc7e80babef8c5a33a1f52138d1a4c9f664f4f4a9439c4ab408.
Updated tool SHA-256: c152c7a9ab946a7566d6cf022962a616f54e3aff51ca83b06b872b335152f468.
Updated candidate health unit SHA-256: 3b81ee483a70e120a826bcb99d83b73a329c9b793186c0e78afc96005427bc35.
Artifact directory entries: 0.

Both temporary units were removed and daemon state reloaded; static networking remained unchanged. This closes isolated total-deadline and restart-counter behavior. Signed production limits, restart counter reset/baseline semantics across activation, real-unit storms and complete rollback chaining for these two codes remain open.

The updated candidate unit passes /tmp-only systemd 249 structural/cycle validation. After final T166 synchronization, the complete build-host deployment suite passed 38 tests with no skips or failures; suite log SHA-256: a5378c125ce65d03ca1f96054feeefdadf2e01b3dfb23b2e2330f5342f10477c. Artifact entries remained 0 and no nexus-test-* unit remained installed.

43 KiB Raw Permalink Blame History