350 lines
32 KiB
Markdown
350 lines
32 KiB
Markdown
# Fresh Consistency Analysis: Installer Release Architecture
|
||
|
||
**Run date**: 2026-07-14
|
||
**Scope**: `spec.md`, `plan.md`, `research.md`, `data-model.md`, `quickstart.md`, `tasks.md`, constitution,
|
||
installer artifact contract, and observed repository installer/offline documentation.
|
||
**Result**: **CONSISTENT; GATE 1 PASS; GATE 2 IMPLEMENTATION/EVIDENCE IN PROGRESS; NO RELEASE ARTIFACT**
|
||
|
||
## Consistent Decisions
|
||
|
||
- The product taxonomy consistently keeps Workstation separate from Server S/M/L/Max and preserves
|
||
compatibility slugs `starter/basic/pro/max`.
|
||
- The agreed four bootable products, Workstation-first order, separate large-model packs, and shared
|
||
offline update format are now normative and do not require a separate feature spec.
|
||
- Zero-network acceptance, fail-closed integrity, durable operations, recovery evidence, and artifact
|
||
provenance align with the constitution's lifecycle, entitlement, and evidence principles.
|
||
- The existing broad three-phase customer-experience rollout and the packaging workstream can coexist:
|
||
packaging is cross-cutting and has independent artifact gates.
|
||
|
||
## Contradictions Requiring Correction During Implementation
|
||
|
||
| ID | Evidence | Contradiction | Required resolution |
|
||
|---|---|---|---|
|
||
| C1 | `autoinstall/offline/README.md` names `build-iso-offline.sh`; repository inventory lacks it | Documented build stage has no current build path | Implement only after approval, or change the referenced path when the design is selected |
|
||
| C2 | Offline directory describes “Server Payload” and script comments say “Server Starter”; README and variables describe Workstation | First implementation category is ambiguous in current runtime material | Rename/split runtime material during implementation and bind manifest category to `workstation` |
|
||
| C3 | README says ISO signature/integrity; current bundle builder writes SHA-256 only and asks for later signing | Current foundation does not enforce signature verification before mutation | Define manifest/signature tooling and trust policy before calling any artifact offline |
|
||
| C4 | Current builder fetches mutable install scripts/tags and records some digests after pull | Reproducibility and immutable input requirements are not yet met | Pin URLs by digest/version, images by digest, models by immutable revision, and record verified sources |
|
||
| C5 | `autoinstall/cezen-ai-ubuntu2204.iso` is 0 bytes | Repository artifact cannot be booted, verified, sized or called current | Replace only through an authorized reproducible build and record real artifact evidence |
|
||
| C6 | Server M/L/Max playbooks currently use the same role set; large models are now separate packs | Current source does not prove a materially distinct L/Max boot payload | Approve a support/validation/runtime distinction or reconsider duplication during implementation planning |
|
||
| C7 | NVIDIA role installs driver 550/CUDA 12.4; Starter declares CUDA 12.6; vLLM 0.10.2 uses CUDA 12.8.1; approved profiles include Blackwell | Current runtime declarations are mutually inconsistent and driver 550/CUDA 12.4 cannot cover the approved Blackwell-capable matrix | Approve `workstation-runtime-compatibility-baseline.md`, resolve exact R570/CUDA 12.8.1-compatible locks, then validate every physical profile and rollback path |
|
||
|
||
## Downstream Gaps and Readiness Blockers
|
||
|
||
The governing architecture choices are approved. These gaps are allocated to Gates 2-6 in
|
||
`gate-evidence-allocation.md` and remain real blockers at those downstream gates.
|
||
|
||
| ID | Severity | Blocker |
|
||
|---|---|---|
|
||
| B1 | Critical | `contracts/release-signing-trust.md` now defines separate release/license/test trust domains, purpose-constrained root/subordinate roles, separation of duties, dual-controlled offline custody, digest-bound signing receipts, rotation/expiry, monotonic offline trust distribution, revocation behavior, compromised-root recovery and acceptance vectors. Named owners, approved hardware/custody/backup, lifetimes/overlap, maximum revocation age, disconnected time policy, field root-recovery design, emergency authority and executed ceremony evidence remain outstanding. |
|
||
| B2 | Critical | Draft schemas, canonicalization, deterministic compatibility evaluation, all artifact-type examples and Ed25519 signatures, negative vectors, and passing automated Draft 2020-12/signature/compatibility evidence now exist. Technical fixture work is complete; security/platform approval remains outstanding. |
|
||
| B3 | Critical | Workstation is universal and hardware-adaptive within each CPU architecture; AMD64 scope/profile classes and later separate ARM64 identity are approved. `contracts/workstation-runtime-compatibility-baseline.md` now proposes an R570 (at least 570.124.06)/CUDA 12.8.1-compatible baseline because current driver 550/CUDA 12.4/CUDA 12.6/vLLM 12.8.1 declarations conflict. Exact physical machines, kernel/package/image/environment locks, model file inventory, redistribution approval and executed compatibility/rollback evidence remain outstanding. |
|
||
| B4 | Critical | `contracts/online-universal-acquisition.md` now defines endpoint classes, TLS/auth, signed index/manifest acquisition, cache/resume, failure recovery, and evidence gates. Production hostnames/owners, credentials, mirrors, proxy/customer-CA, retry/cache values, and approval remain outstanding. Current mutable default-branch clone remains contradictory legacy behavior. |
|
||
| B5 | High | `contracts/update-rollback-policy.md` now defines update classes, ownership, ordering, current/last-known-good retention, checkpoint scope, atomic activation target, ten-minute validation proposal, interruption and rollback outcomes. Actual switch/snapshot mechanisms, full backup inventory/SLA, supported prior versions, disk quota, driver/kernel recovery, and owner approval remain outstanding. Current data-only ZIP is explicitly insufficient as complete platform rollback. |
|
||
| B6 | High | `contracts/offline-license-policy.md` now defines offline import/activation, explicit staging, clock confidence/rollback, expiry continuity, binding/rehost, revocation/trust updates, interruption and evidence. Staging/expiry matrices, clock tolerance, device identity, rehost authority, revocation ages and owner approval remain. Current optional machine-ID enforcement and `basic` staging compatibility default are explicit implementation blockers. |
|
||
| B7 | High | `contracts/payload-inventory-sizing.md` now defines four-artifact boundaries, candidate inputs, role matrix, exact-digest deduplication, sizing worksheet, capacity arithmetic and evidence gates. Immutable resolved object inventories/build measurements are still absent; Server offline sizes remain unpublished and L/Max split justification remains open. |
|
||
| B8 | High | `contracts/offline-validation-lab.md` now defines isolation topology, packet-attempt detection, physical hardware matrix, clean/interruption/negative runs, evidence package, separation of duties and stop/retry rules. Named staff, exact machines/lab topology, approved tools, schedule, evidence storage and owner assignment remain outstanding. |
|
||
| B9 | Medium | `contracts/release-lifecycle-provenance.md` now proposes semantic versions, canonical filenames/identity, channels, SPDX 3.0.1 SBOM, in-toto Attestation Framework v1.2 Statement v1 with SLSA Provenance v1 (SLSA spec v1.2), reproducibility classes, retention, support/cadence and withdrawal. Toolchain, publication/archive locations, final periods/windows/cadence and named owner approval remain outstanding. |
|
||
| B10 | Medium | `contracts/model-pack-lifecycle.md` now defines bounded non-executable quarantine, malicious/partial import rejection, exact-digest object sharing, signed dependencies, atomic activation, uninstall/data preservation, reference-safe garbage collection, entitlement and redistribution-license changes, interruption tests and Gate 5 reconciliation. Store/transaction mechanisms, quotas/retention, customer-data rules, continuity/rehost policy, withdrawal/GC authority, named owners and executed evidence remain outstanding. |
|
||
| B11 | Critical | `contracts/installer-threat-model.md` now defines protected assets, adversaries, trust boundaries, eighteen threat/control cases, state invariants, abuse/interruption evidence and residual risks across source, build, signing, distribution, install, activation, recovery and evidence export. Owner acceptance, residual-risk assignment and linked executed control evidence remain outstanding; Secure Boot/firmware/TPM, build isolation, root/admin, disk encryption, evidence-signing and physical custody choices remain open. |
|
||
|
||
## Next-Step Remediation Completed
|
||
|
||
`contracts/architecture-decisions.md` records repository-derived defaults and separates them from
|
||
decisions requiring named Cezen approval. Draft JSON Schemas now cover the shared artifact manifest
|
||
and detached signature envelope. These drafts reduce B2/B3 ambiguity but do not close the gates until
|
||
owners approve them and immutable test vectors/payload revisions exist.
|
||
|
||
The next technical pass removed a circular self-digest from the manifest schema: the canonical
|
||
manifest digest belongs only in its detached signature envelope. `compatibility-evaluation.md` now
|
||
defines fail-closed evaluation order and stable reason codes, with non-production structural and
|
||
compatibility fixtures under `contracts/test-vectors/`. Cryptographic approval remains blocked until
|
||
a dedicated test key and real signatures are created under approved key-handling policy.
|
||
|
||
The B3 evidence pass verified immutable candidate pins for both Ollama starter models and the Hugging
|
||
Face Phi-3 snapshot, plus vendor hardware facts. It also surfaced a release-architecture contradiction:
|
||
the singular AMD64 Workstation ISO cannot also boot ARM64 DGX Spark. Following product clarification,
|
||
`contracts/workstation-hardware-model-baseline.md` now requires one universal hardware-adaptive ISO
|
||
per CPU architecture: the AMD64 artifact spans CPU-only and low/mid/high GPU classes, while a later
|
||
ARM64 artifact may cover DGX Spark under the same customer-facing Workstation category.
|
||
|
||
## Readiness Decision
|
||
|
||
The architecture is coherent and every Gate 1 requirement is approved. The closure audit in
|
||
`analysis/gate-evidence-allocation.md` records **Gate 1 PASS**. Subsequent explicit user authorization
|
||
permitted isolated Gate 2 resolver, verification, checkpoint and disposable-install evidence, but did
|
||
not authorize a release artifact, commit, push or deployment. Gate 2 remains blocked by operational,
|
||
immutable-inventory, mechanism, operator,
|
||
lab and evidence-storage items. Offline Workstation MUST remain the only first implementation candidate.
|
||
Offline Server S/M and L/Max remain planning-only until a real Workstation artifact passes the shared
|
||
zero-network, signature, update/model, interruption, rollback, and evidence gates. No ISO can be
|
||
called offline from the current repository material alone.
|
||
|
||
Gate 2 resolver evidence initially appeared to expose a bcrypt/Chroma contradiction, but fresh repo
|
||
inspection proved that the backend, Chroma and platform dependencies are installed into separate
|
||
virtual environments and wheelhouses. The corrected resolver preserved those boundaries and produced
|
||
363 measured wheel objects without changing either constraint. It also exposed the actual readiness
|
||
blocker: unpinned platform requirements currently select CUDA 13 Python components, conflicting with
|
||
the approved CUDA 12.8.1 runtime baseline. The candidate cannot be reviewed until that ML stack is
|
||
explicitly pinned and its three environments pass offline installation/import and license evidence.
|
||
|
||
Resolver increment 002 removes that version-selection contradiction using the exact dependency pins
|
||
embedded in vLLM 0.10.2: Torch 2.8.0, Torchaudio 2.8.0 and Torchvision 0.23.0. The fresh candidate has
|
||
no explicitly CUDA-13-named objects and 14 explicitly CUDA-12-named objects. This is not yet runtime
|
||
acceptance: physical R570/CUDA 12.8.1 testing, disconnected Python 3.11 install/import, license review,
|
||
and proof that the CPU-only profile does not require NVIDIA packages remain Gate 2 blockers.
|
||
|
||
Disconnected Python 3.11 validation subsequently passed for all three wheelhouses using a
|
||
digest-pinned validator container, `--network none` and `pip --no-index`. CPU execution passed with
|
||
CUDA hidden; a supplemental RTX 3050/Open Driver 595.58.03 test also executed CUDA 12.8 tensor work.
|
||
This narrows but does not close Gate 2: the 8 GB test is not the exact R570 reference or the 24/48 GB
|
||
matrix, and carrying CUDA wheels is not proof that CPU-profile installation excludes NVIDIA runtime
|
||
dependencies. License/provenance review and bare-metal/interruption/model-serving evidence remain open.
|
||
|
||
Profile-selection increment 001 closes the Python CPU payload-boundary ambiguity: the universal AMD64
|
||
medium carries profile alternatives, while installation selects exactly one. A strict 196-wheel CPU
|
||
selection contains no NVIDIA/CUDA/Triton/bitsandbytes objects and passed disconnected installation as
|
||
Torch `2.8.0+cpu` with no compiled CUDA runtime. The machine-readable selector and validator fail
|
||
closed on missing, altered or forbidden wheels. Installer integration, signed membership, detection,
|
||
journal/resume/rollback and tamper/interruption tests remain open and are allocated to T143.
|
||
|
||
Planner increment 001 now makes those state transitions deterministic without wiring them into the
|
||
installer. Six planner tests and build-host vectors prove safe CPU fallback, fully-ready NVIDIA
|
||
selection, same-profile resume and rollback-required profile changes. This is structural evidence only:
|
||
the test receipt is not cryptographically verified, and no atomic journal, checkpoint, installer
|
||
mutation, power-loss recovery or production signature-verifier integration exists yet. T143 remains
|
||
open; the planner prevents later wiring from inventing different state semantics.
|
||
|
||
Receipt increment 001 closes the unsafe bare-boolean handoff in the structural planner. A closed schema
|
||
now binds the manifest/envelope, key, trust snapshot, policy, namespace, profiles, time and verifier
|
||
build, and test receipts are forbidden by default. This is still not production signature evidence:
|
||
no production verifier, trust-policy evaluation, key ceremony or durable atomic journal/checkpoint
|
||
writer exists. Those blockers are explicitly allocated to T146 before T143 installer integration.
|
||
|
||
Atomic-journal increment 001 now implements the persistence mechanism and rejects stale writers,
|
||
altered records, identity changes, missing checkpoints and invalid transitions. Build-host vectors
|
||
reached `installing` only after a recorded checkpoint and retained mode `0600` plus digest linkage.
|
||
T146 is still open because a recorded checkpoint ID is not checkpoint creation/verification,
|
||
filesystem power loss has not been injected, and production Ed25519/trust-policy verification is not
|
||
implemented.
|
||
|
||
Ed25519 verifier increment 001 implements the cryptographic and structural handoff using OpenSSL and
|
||
passes positive/tamper/environment tests on Ubuntu with ephemeral test keys. It does not close T146:
|
||
there is no production key ceremony, approved trust snapshot, revocation/time-policy evaluation,
|
||
locked verifier/OpenSSL build, or independently approved production signature. The implementation is
|
||
ready for those inputs without treating test-key success as release evidence.
|
||
|
||
Checkpoint increment 001 now creates, verifies and restores hashed Python-environment snapshots and
|
||
handles originally absent targets without silent deletion. The build-host disposable vector restored
|
||
the original bytes exactly. Forced-process crash vectors now also prove that no partial checkpoint is
|
||
accepted, a published checkpoint verifies, interrupted restore completes from preserved quarantine,
|
||
and post-activation retry is idempotent. T146/T143 remain open because physical power loss,
|
||
disk-full tests, real target metadata coverage, orchestration and production trust inputs are absent.
|
||
|
||
Pre-mutation orchestration increment 001 connects the previously isolated components through durable
|
||
`checkpoint_ready` and proves idempotent re-entry with no package mutation. Remaining orchestration
|
||
starts at the mutation boundary: offline install, health validation, installed/rollback transitions,
|
||
service integration, real paths and production verification receipts remain unproved.
|
||
|
||
Disposable installation increment 001 crosses the Python mutation boundary for the strict CPU profile:
|
||
network-disabled install, pre/post activation health, fsynced atomic activation and final `installed`
|
||
state pass; offline dependency failure records rollback without changing the original target. This does
|
||
not close Gate 2 because NVIDIA execution, service integration, real paths/metadata, forced failure
|
||
after activation, reboot resume and bare-metal zero-network acceptance remain open.
|
||
|
||
Post-activation rollback increment 001 now proves the disposable CPU executor can recover from process
|
||
death after activation but before post-health/final journal commit. Exact checkpoint restoration,
|
||
failed-environment quarantine and terminal rollback all pass. Reboot/power-loss recovery, service-level
|
||
health, disk exhaustion, NVIDIA rollback and live installer integration remain blockers.
|
||
|
||
Disposable NVIDIA installation increment 001 proves the complete cu128 Python environment can install
|
||
and execute CUDA 12.8 tensors on the 8 GB RTX 3050 under Open Driver 595.58.03. It narrows the GPU gap
|
||
but does not satisfy the exact R570 reference, 24/48 GB matrix, vLLM/model-serving, NVIDIA rollback,
|
||
reboot recovery or bare-metal acceptance requirements.
|
||
|
||
`analysis/gate-1-decision-dossier.md` is now the consolidated approval surface. Its fresh count is
|
||
**30/30 mandatory architecture decisions approved; Gate 1 PASS; downstream evidence incomplete**. Cezen Technologies record `CT-G1-2026-001` approves G1-12
|
||
(AMD64 universal adaptive first-release direction with NVIDIA acceleration, CPU fallback and AMD GPU/
|
||
ROCm/Strix Halo acceleration excluded) and G1-17 (DGX Spark/ARM64 deferred to a separately evidenced
|
||
later build identity). G1-13's profile matrix is separately approved but lacks physical evidence.
|
||
G1-14 is approved under `CT-G1-2026-007` for Ubuntu 22.04.5 AMD64, R570 open driver at least
|
||
570.124.06, CUDA 12.8.1, vLLM 0.10.2 by immutable OCI digest and NVIDIA-independent CPU behavior;
|
||
exact locks and physical/rollback evidence remain open. G1-15 is approved under `CT-G1-2026-008` for
|
||
the pinned Phi-3 Mini Ollama manifest/layer, Nomic Embed Text v1.5 manifest and Microsoft Phi-3 Mini
|
||
commit with their recorded licenses and hardware boundary; packaged file/digest/notice and
|
||
redistribution evidence remain open. G1-16's Offline Workstation inclusion/exclusion boundary is
|
||
approved under `CT-G1-2026-009`; resolved signed object inventory, exact sizes/digests/licenses,
|
||
extracted-media reconciliation and measured capacity evidence remain open.
|
||
Existing schema/vector evidence and hardware/model
|
||
research are partial technical inputs, not approvals.
|
||
Cezen Technologies record `CT-G1-2026-002` additionally approves G1-08 through G1-11: the manifest/
|
||
signature schemas, deterministic default-deny compatibility, canonical naming/SBOM/provenance formats,
|
||
and immutable publication/withdrawal direction. Production tooling/sample-release evidence and exact
|
||
archive/access/retention values remain open.
|
||
Cezen Technologies record `CT-G1-2026-003` approves G1-01 through G1-07: separate trust domains,
|
||
offline-root/isolated-signer custody direction, rotation, monotonic revocation, compromised-root
|
||
recovery direction, TM-01–TM-18 controls and the requirement for a platform security baseline.
|
||
Concrete hardware/custody, timing values, recovery procedures, residual-risk assignments, exact
|
||
platform baseline and executed evidence remain open. Cezen Technologies record `CT-G1-2026-004`
|
||
approves G1-18 through G1-24: secure online endpoint/acquisition direction, scoped credentials and
|
||
resumable cache, staged/atomic update direction, last-known-good retention, explicit rollback classes,
|
||
restricted offline staging and binding/rehost/monotonic revocation. Production endpoints and numeric
|
||
operational values, exact mechanisms/runbooks and executed evidence remain open.
|
||
Cezen Technologies record `CT-G1-2026-005` approves G1-25 through G1-30: model quarantine/reference/
|
||
GC and customer-data preservation, the isolated lab and tamper-evident evidence methods, the complete
|
||
no-manual-repair acceptance inventory, and deferral of the Server L/Max split until Workstation evidence
|
||
proves a real distinction. Operational values, lab assignments and executed evidence remain open.
|
||
Cezen Technologies record `CT-G1-2026-006` approves the G1-13 representative profile matrix, but not
|
||
the existence or results of physical machines; exact CPU/GPU/RAM/storage, asset IDs, firmware and
|
||
availability remain evidence-open.
|
||
`gate-evidence-allocation.md` now separates Gate 1 approval evidence from Gates 2-6 execution evidence.
|
||
|
||
## Fresh Audit Pass 2
|
||
|
||
**Audit date**: 2026-07-14
|
||
**Result**: **DOCUMENT SET INTERNALLY CONSISTENT; SUPERSEDED BY PASS 3 GATE ALLOCATION**
|
||
|
||
- All 12 checked-in JSON schemas/fixtures parse as JSON; the recorded full vector-suite evidence
|
||
remains the technical result of record pending independent approval.
|
||
- Functional- and success-criterion identifiers have no duplicates; contract and analysis paths in
|
||
the plan exist; Markdown whitespace validation passes.
|
||
- At Pass 2, C1-C6 remained repository/runtime contradictions; later explicitly authorized Gate 2
|
||
increments changed the status of C2 and partially remediated C3/C4/C7 as recorded in Pass 4 below.
|
||
- Eight sizing rows contain 40 `TBD` markers in the measured payload worksheet. They are evidence
|
||
holes, not unspecified behavior: T106 cannot resolve them without authorized immutable dependency
|
||
resolution, artifact assembly and measurement.
|
||
- The signing decision package was aligned with the signing trust contract: the artifact signer is
|
||
isolated from builders and is not described as an online build signer.
|
||
- There are 18 unchecked installer task lines. They are not 18 sequential documentation passes:
|
||
most consolidate into the 30 decisions in the Gate 1 dossier, followed by evidence execution and
|
||
a Gate 1 rerun. T083 is intentionally later, after Workstation passes Gates 1-5.
|
||
|
||
Pass 2 correctly blocked while decisions were open. All decisions are now approved; Pass 3 in
|
||
`gate-evidence-allocation.md` supersedes that result and allocates remaining evidence to Gates 2-6.
|
||
|
||
## Fresh Audit Pass 4 — Gate 2 Implementation Evidence
|
||
|
||
**Audit date**: 2026-07-14
|
||
**Result**: **INTERNALLY CONSISTENT; GATE 2 NOT READY; OFFLINE WORKSTATION REMAINS THE ONLY BUILD CANDIDATE**
|
||
|
||
All specification JSON files parse, the current local deployment suite passes 36 tests with one
|
||
expected Mac OpenSSL/Ed25519 capability skip, shell/Python syntax and whitespace checks pass, and the
|
||
artifact directory remained empty in every recorded build-host increment.
|
||
|
||
### Contradiction Reconciliation
|
||
|
||
| ID | Pass 4 status | Evidence and remaining work |
|
||
|---|---|---|
|
||
| C1 | Open | `build-iso-offline.sh` is still referenced but absent. No lock-consuming media builder exists. |
|
||
| C2 | Resolved for the new path | New resolver, profile and evidence surfaces consistently identify Offline Workstation. Legacy Server material has not been promoted. |
|
||
| C3 | Partially remediated | Ed25519/profile receipt, fail-closed planner, journal/checkpoint and disposable mutation exist; production keys/trust policy and release-artifact signature enforcement do not. |
|
||
| C4 | Partially remediated | Source, OCI manifests, APT closure and Python wheels have candidate locks. Ubuntu media, runtime archives, OCI exported bytes, models, repository/index provenance and licenses remain unresolved. |
|
||
| C5 | Open | The checked-in ISO remains zero bytes; no bootable artifact exists or is claimed. |
|
||
| C6 | Deferred | Server L/Max material distinction remains intentionally deferred until Workstation passes its gates. |
|
||
| C7 | Partially remediated | Python uses the vLLM-aligned Torch 2.8.0/CUDA 12.8 split and CPU-only alternative. The production NVIDIA Ansible/package path still declares legacy driver/CUDA behavior and the exact R570 matrix is unproved. |
|
||
|
||
### Gate 2 Readiness Blockers
|
||
|
||
1. Candidate payload lock resolution 004 is not reviewed/signed and still lacks complete Ubuntu,
|
||
runtime, model and measured container objects.
|
||
2. APT/Python/OCI index provenance, licenses/notices and redistribution approval remain incomplete;
|
||
49 Python license fields require review.
|
||
3. Production release keys, trust snapshot, revocation/time rules, verifier/OpenSSL build locks and
|
||
independent signing evidence do not exist.
|
||
4. Exact R570 validation plus 24 GB and 48 GB-or-higher physical profiles remain absent; RTX 3050/
|
||
driver 595 evidence is supplemental `gpu-lite` coverage only.
|
||
5. No lock-driven acquisition/assembly path or bootable Offline Workstation ISO exists; the legacy
|
||
mutable builder remains intentionally unreachable.
|
||
6. Sudden-power-loss recovery, post-activation or recovery-reserve exhaustion, production service-stack
|
||
health, real ownership/ACL/xattr behavior and bare-metal zero-network acceptance remain unproved.
|
||
Controlled OS-reboot and pre-activation target byte/inode exhaustion are proven only for disposable
|
||
strict-CPU vectors; an external post-activation service-probe rollback is also proven only in
|
||
disposable staging.
|
||
7. Named lab/operators, restricted evidence store, retention and independent approval remain open.
|
||
8. The payload-sizing worksheet still has eight unmeasured rows (40 `TBD` markers); media capacity and
|
||
extracted peak/rollback space cannot be approved before real assembly evidence.
|
||
9. The checked-in real service graph is incomplete: `cezen-api.service` orders after Ollama without
|
||
requiring it, omits ChromaDB dependency/ordering, and no production proxy dependency or consecutive
|
||
semantic-health window is wired to rollback. T163 proves only an isolated mock graph; T164 provides
|
||
structurally validated candidate definitions but does not replace or validate the real units.
|
||
10. T165 supplies deterministic health-window implementation and disconnected loopback evidence, but
|
||
no signed production health profile, approved timing, real Nexus semantic endpoint matrix or
|
||
production unit installation/security evidence exists.
|
||
|
||
Gate 2 therefore remains **NOT READY**. The next implementation evidence may continue on Offline
|
||
Workstation, but Offline Server S/M and L/Max and all release/build claims remain blocked.
|
||
|
||
Operator-flow remediation T156 removes the misleading executable build sequence from the offline
|
||
README. It now identifies every current tool as candidate-resolution, structural/disposable evidence,
|
||
available fail-closed validation, or blocked/absent assembly. C1 remains openly documented rather than
|
||
being implied complete by instructions that referenced a nonexistent script.
|
||
|
||
NVIDIA rollback increment T158 closes the disposable 8 GB Python post-activation process-crash case:
|
||
checkpoint restoration, executable failed-environment quarantine and terminal rollback pass. The
|
||
remaining NVIDIA blockers are reboot/physical-power recovery, exact R570 and 24/48 GB hardware,
|
||
vLLM/service/model health, real paths/metadata and bare-metal zero-network installation.
|
||
|
||
Boot-dispatcher increment T159 closes the narrower fresh-process recovery wiring gap for the strict
|
||
CPU profile: a distinct offline container recovered a generation-3 `installing` journal to generation
|
||
5 `rolled_back`, restored the checkpoint, retained the activated environment and no-op'd on a second
|
||
terminal invocation. This is consistent with the interruption contract but does not change the Gate 2
|
||
decision: actual OS reboot and sudden-power-loss durability, production boot ordering, real filesystem
|
||
metadata, capacity exhaustion and service health remained unproved at T159; T160/T161 narrow those
|
||
gaps below.
|
||
|
||
The post-T159 synchronized build-host suite passes 29 tests with no skips and zero artifact entries.
|
||
This refresh does not alter any open contradiction or readiness blocker.
|
||
|
||
Controlled-reboot increment T160 closes the disposable strict-CPU OS-reboot recovery gap. It also
|
||
proved persistent static networking at `172.16.10.144/24`. An initial exact-path mismatch failed
|
||
closed before restore; a fresh mount-consistent vector then recovered during boot to generation 5
|
||
`rolled_back`. The temporary boot unit was removed. Gate 2 remains not ready because physical power
|
||
loss, production unit packaging/order, live paths and metadata, capacity exhaustion, service/model
|
||
health and bare-metal zero-network installation remained unproved at T160; T161 narrows target
|
||
pre-activation exhaustion below.
|
||
|
||
Capacity increment T161 closes pre-activation target byte and inode exhaustion for disposable
|
||
strict-CPU installs. The vectors independently constrained the executable target to 128 MiB and 2,000
|
||
inodes, produced real `ENOSPC`, preserved the original target and reached generation 5 `rolled_back`
|
||
using separately reserved journal/checkpoint capacity. Gate 2 remains not ready: production capacity
|
||
reservation/preflight, reserved-state exhaustion, post-activation exhaustion, live metadata and
|
||
service-level rollback are still open.
|
||
|
||
Service-health increment T162 closes the generic external-probe failure path after strict-CPU atomic
|
||
activation: the probe observed the new environment, exited `23`, and exact checkpoint rollback plus
|
||
quarantine and generation-5 terminal state passed. Gate 2 remains not ready because the probe is not
|
||
yet a locked production component and no real Nexus systemd/API/portal/model service dependency graph,
|
||
health window, live path metadata or traffic behavior has been validated.
|
||
|
||
The post-T162 synchronized build-host suite passes 31 tests with no skips and zero artifact entries.
|
||
|
||
Systemd increment T163 proves explicit dependency ordering, three consecutive semantic-health samples
|
||
and `OnFailure` rollback in a fully isolated four-service graph. A forced vector 503 restored the
|
||
checkpoint and quarantined the candidate at generation 5. All temporary units were removed. This does
|
||
not close production service readiness: the real unit graph and actual Nexus model/vector/API/proxy
|
||
health semantics remain unimplemented/unvalidated and are now an explicit Gate 2 blocker.
|
||
|
||
Candidate-unit increment T164 defines the corrected production dependency and rollback direction while
|
||
preserving nginx recovery/help access during API failure. `/tmp`-only systemd 249 structural validation
|
||
passes without candidate errors or cycles. Gate 2 remains not ready: the health-window executable,
|
||
signing/locking, category-specific graph generation, real executables/nginx, approved timing, actual
|
||
restart/timeout behavior and production security analysis are absent, and current Ansible units remain
|
||
unchanged.
|
||
|
||
Health-window increment T165 closes the isolated validator behavior for ready, explicitly allowed
|
||
optional degradation, timeout, malformed JSON and HTTP-503 rejection. Rejection exit 23 now matches the
|
||
candidate systemd `OnFailure` contract. Gate 2 remains not ready because profiles/tool/unit bytes are
|
||
unsigned candidates, the real service graph/endpoints and total health deadline are unproved, and no
|
||
production units have been installed or security-scored.
|
||
|
||
The post-T165 synchronized build-host suite passes 36 tests with no skips and zero artifact entries;
|
||
the updated candidate health unit also passes `/tmp`-only systemd 249 structural/cycle validation.
|
||
|
||
Deadline/storm increment T166 closes isolated monotonic total-window enforcement and pre-HTTP systemd
|
||
restart-ceiling behavior. Real temporary units produced exit-23 `WINDOW_TIMEOUT` and `RESTART_STORM`
|
||
evidence, and were removed afterward. Gate 2 remains not ready: production timing/restart limits are
|
||
unsigned/unapproved, restart baseline/reset semantics and real-unit storms are unproved, and these
|
||
failure codes have not yet been chained through the candidate rollback unit in one transaction.
|
||
|
||
The post-T166 synchronized build-host suite passes 38 tests with no skips and zero artifact entries;
|
||
the deadline-aware candidate unit also passes `/tmp`-only systemd 249 structural/cycle validation.
|