105 lines
6.2 KiB
Markdown
105 lines
6.2 KiB
Markdown
# Server Phase 1 Validation Evidence
|
|
|
|
**Validation date:** 2026-07-12 to 2026-07-13
|
|
|
|
**Target:** Vast.ai full Ubuntu 22.04 VM, RTX 3090 24 GB, 23 vCPU, 52.8 GB usable RAM, 155.9 GB root filesystem
|
|
|
|
**Package revision at initial install:** `55a2a41` plus the uncommitted corrections listed below
|
|
|
|
**Install mode:** `install.sh --software-only --tier=starter`
|
|
|
|
## Executive result
|
|
|
|
The software package installs and the tested Phase 1 Server workflows operate on the representative
|
|
cloud VM after correcting defects found during validation. This is evidence for a controlled Server
|
|
pilot only. It is not ISO boot evidence, physical hardware evidence, Workstation evidence, a customer
|
|
pilot, WCAG manual approval, or general-release approval.
|
|
|
|
## Results
|
|
|
|
| Area | Result | Evidence |
|
|
|---|---|---|
|
|
| Starter installation | PASS after Docker repository fix | Ansible recap: 86 ok, 51 changed, 0 failed, 2 ignored |
|
|
| Source contract suite | PASS | 19 tests passed |
|
|
| Shell/Python/static validation | PASS | `bash -n`, `py_compile`, `git diff --check` |
|
|
| Tier playbook syntax | PASS | starter, entry, pro, max syntax checks |
|
|
| Portal/backend | PASS | HTTP 200 after install and reboot |
|
|
| Authenticated read endpoints | PASS | 51 HTTP 200 responses |
|
|
| Entitlement restrictions | PASS | model router and two fine-tuning routes correctly returned 403 |
|
|
| Authentication and RBAC | PASS | admin login; viewer permitted metrics and denied admin user list |
|
|
| User/session lifecycle | PASS | create, login, revoke by deletion, expired-session rejection, self-delete protection |
|
|
| API key lifecycle | PASS | create, verify, revoke, revoked-key rejection |
|
|
| Guardrails | PASS | temporary keyword rule blocked matching input and was removed |
|
|
| Prompt library | PASS | create, update and delete |
|
|
| Backup/restore | PASS | ZIP created; listed; restored; pre-restore safety snapshot created |
|
|
| Ollama inference | PASS | Qwen 2.5 1.5B GPU inference; model persisted across reboot |
|
|
| Open WebUI | PASS after network correction | healthy container and model visible from WebUI |
|
|
| RAG | PASS after corrections | collection, TXT upload, embedding, ready status, semantic retrieval, delete/cleanup |
|
|
| vLLM | PASS after corrections | Phi-3 readiness and OpenAI-compatible chat completion |
|
|
| Monitoring | PASS | Grafana health 200, Prometheus ready 200, DCGM and node exporter running |
|
|
| Web console | PASS after correction | service active and `/console/` HTTP 200 |
|
|
| Reboot persistence | PASS | all six services and five containers auto-started; models and backup persisted |
|
|
| Concurrent inference | PASS | vLLM used 21.3 GB and Ollama 1.8 GB on RTX 3090; both completed inference |
|
|
|
|
Post-reboot vLLM cold readiness occurred at approximately 144 seconds after boot.
|
|
|
|
## Defects discovered and corrected in source
|
|
|
|
1. Ubuntu ttyd does not support `--writable`, and the distribution ttyd service can occupy port 7681.
|
|
2. Open WebUI could not reach an Ollama process that had not reloaded the appliance service environment.
|
|
3. Backend targeted ChromaDB port 8000 while the service intentionally runs on 8100.
|
|
4. Unpinned latest ChromaDB removed the v1 API used by the backend.
|
|
5. Compatible ChromaDB dependencies conflicted with the shared AI environment; ChromaDB is now isolated.
|
|
6. Short RAG documents caused an infinite chunking loop.
|
|
7. Chroma query/upsert/delete data routes require collection UUIDs rather than collection names.
|
|
8. Starter Phi-3 was incorrectly configured as AWQ.
|
|
9. Blank optional vLLM arguments caused systemd command continuation into `ExecStop`.
|
|
10. `vllm:latest` required an incompatible CUDA/driver combination; the image is now pinned to v0.10.2.
|
|
11. vLLM was enabled but not explicitly started during installation.
|
|
|
|
## Expected warnings / environment gaps
|
|
|
|
- No final signed customer license was installed; field-staging mode was used.
|
|
- TLS was not configured; access used SSH tunnels.
|
|
- The VM began without a backup or models; both were created during testing.
|
|
- Direct public port 80 was blocked by the provider; this is not an appliance Nginx failure.
|
|
- Clean ISO boot, disk partitioning, BIOS/UEFI, USB media, physical NICs and offline ISO installation
|
|
cannot be validated on this rented VM.
|
|
- Workstation validation (T036), manual WCAG 2.2 AA evidence, interruption/restricted-network trials,
|
|
controlled customer pilot and release approvals remain pending.
|
|
|
|
## Release decision
|
|
|
|
**Not ready for customer shipment yet.** The corrected package is suitable for another clean Server
|
|
installation validation and controlled pilot preparation. A new ISO must be built from the corrected
|
|
revision and both Server and Workstation ISO paths must be tested before release consideration.
|
|
|
|
## Corrected-package clean-host regression — 2026-07-13
|
|
|
|
A second independent Vast.ai Ubuntu 22.04.5 VM was installed from the corrected local package:
|
|
|
|
- RTX 3090 24 GB, NVIDIA driver 580.159.03 after upgrade/reboot;
|
|
- 30 vCPU, 52.9 GB usable RAM, 126 GB root filesystem;
|
|
- clean starter install recap: 88 ok, 63 changed, 0 failed, 3 ignored;
|
|
- the ignored DCGM failure was caused by the required post-driver-upgrade reboot and passed afterward;
|
|
- Grafana provisioning was retried after its startup race and the Prometheus datasource was added;
|
|
- portal source permissions were normalized after clean-host validation exposed preserved macOS mode/UID values.
|
|
|
|
Regression results:
|
|
|
|
- 51 authenticated parameterless feature endpoints returned 200;
|
|
- 3 unlicensed features correctly returned 403;
|
|
- RBAC, user/session, API-key, guardrail, prompt, backup and restore cases passed;
|
|
- short-document RAG ingest, embedding, semantic retrieval and cleanup passed;
|
|
- vLLM Phi-3 OpenAI-compatible inference passed;
|
|
- Grafana, Prometheus, DCGM and node-exporter endpoints returned 200;
|
|
- audit report contained the expected test actions with zero recorded failures;
|
|
- after a final reboot, all six system services and six containers recovered;
|
|
- portal, backend, Ollama, ChromaDB, Grafana, Prometheus and DCGM returned 200;
|
|
- the embedding model and backup archive persisted;
|
|
- vLLM was ready by approximately 100 seconds after boot.
|
|
|
|
This confirms the corrected software package on two Server-class cloud VMs. It does not change the
|
|
remaining ISO boot, physical hardware, Workstation, manual accessibility, network-interruption,
|
|
customer-pilot or release-approval gaps above.
|