Validation Quickstart: Enterprise Appliance Experience Hardening

This guide defines the evidence path for implementation. Commands are representative and MUST be updated to the final test tooling before acceptance.

1. Prerequisites

Clean checkout of the intended source revision.
One representative Nexus One AI Workstation target.
One representative Nexus One AI Server target with a valid supported tier.
Test licenses for valid, missing/staging, expired, invalid-signature, and mismatch states.
At least 10 representative administrator participants across internal validation and pilot.
A minimum 20-case customer-correctable failure corpus covering all specified failure categories.
Ability to simulate network, service, storage, and reboot interruption safely.
Supported desktop browser plus keyboard-only and representative assistive-technology setup.
Controlled pilot approval and rollback plan.

Never use production customer secrets or content in validation fixtures.

2. Static and Local Validation

Run Python syntax and focused unit/contract tests for readiness, entitlement, operation transitions, audit redaction, backup compatibility, and error envelopes.
Run shell syntax checks for changed installer and ISO scripts.
Run Ansible syntax checks for every affected Workstation/Server deployment path.
Run portal checks for broken navigation, legacy naming, placeholder/sample data, inaccessible names, keyboard traps, contrast, and invalid markup.
Confirm canonical and deployed license helpers produce identical entitlement results for the test matrix.
Verify allowed and denied roles for every new operation, recovery, removal, diagnostic, and support-evidence interface.
Run readiness, navigation-feedback, and operation-acknowledgement timing trials against SC-014, SC-015, and SC-016 and record conditions, sample count, and percentile results.

Expected outcome: all checks pass; any intentionally retained sample content is explicitly labelled; no signed license payload, token, credential, or customer content appears in outputs.

3. Phase 1 Clean-Install Journeys

Run the following on both the representative Workstation and Server:

Start from the supported clean image or installation baseline.
Complete browser setup with valid network, customer, license, product/tier, and component choices.
Repeat the supported console setup path.
Verify progress survives browser refresh, session change, and setup-service restart.
Verify the resulting readiness view identifies product, entitlement, freshness, degraded states, and correct next actions.
Confirm locally available setup and recovery guidance works without public internet access.

Expected outcome: the administrator reaches a truthful ready or explicitly degraded state without undocumented intervention; Workstation never appears as a Server tier.

4. Phase 1 Failure and Recovery Matrix

For each applicable setup stage, simulate invalid input, network loss, service failure, storage shortage, license failure, interruption before reboot, interruption after reboot, and stale status.

Verify:

completed safe steps and non-secret inputs are retained;
the operation has one durable identity and truthful lifecycle state;
duplicate/conflicting submissions are prevented;
error text explains impact and safe retry/resume/recovery/escalation;
audit and diagnostic evidence share the correlation ID;
no raw exception or secret appears in the customer experience.

5. Accessibility Validation

On every changed setup and portal page:

Complete the primary and failure-recovery journey using keyboard only.
Verify focus order, visible focus, modal/confirmation behaviour, and return focus.
Verify status, progress, errors, and completion are announced appropriately.
Check contrast, non-colour cues, zoom/reflow, accessible names, headings, and landmarks.
Run the selected automated WCAG checks and manually review failures or unsupported criteria.

Expected outcome: no unresolved WCAG 2.2 Level A or AA failure at phase acceptance.

6. Upgrade and Backup Compatibility

Create a verified backup from the supported pre-feature appliance version.
Upgrade or reconfigure using the planned migration path.
Confirm existing authentication, users, audit history, licensing, settings, and supported customer data remain usable.
Exercise backup and restore with schema compatibility checks and a pre-restore safety snapshot.
Interrupt the operation at safe test points and verify documented recovery.
Exercise removal in a disposable environment; verify explicit retained/deleted data inventory, backup preservation, authorization, audit evidence, interruption outcome, and rollback limits.

Expected outcome: no silent data loss; incompatible restore is blocked before mutation; recovery evidence is sufficient for support.

7. Package and ISO Provenance

For each delivery candidate, record:

source revision and working-tree status;
portal, backend, installer, Ansible, entitlement helper, and local guidance versions;
ISO/package filename, build time, size, and SHA-256 checksum;
proof that the artifact contains the accepted source copies;
successful representative boot/install journey.

Expected outcome: a release is never called current based only on source state or filename.

8. Pilot and Acceptance

Complete internal evidence review on Workstation and Server.
Deploy only to approved controlled-pilot environments using the rollback plan.
Capture task completion, recovery results, accessibility findings, support effort, and customer feedback without collecting protected content.
Resolve and revalidate all critical customer-safety, task-completion, entitlement, and recovery findings.
Obtain recorded joint approval from product ownership and delivery/support.

Only then is the phase eligible for general release.

5.8 KiB Raw Blame History