Integrate Debian host readiness checks
This commit is contained in:
parent
3293472cad
commit
86e548f2fc
@ -20,7 +20,7 @@ This summary explains what each milestone delivers and where it currently stands
|
||||
| 7 | Google connection recovery | OAuth reconnect, checkpoint recovery, grant revocation handling, and worker restart safety. | **Implemented; acceptance required.** Dedicated Google-account and worker-restart exercises have not yet been accepted. |
|
||||
| 8 | Operational readiness tooling | Release verification, diagnostics, encrypted backup, restore, preflight, and persistence tools. | **Implemented; acceptance required.** The tools must still be run against the exact release on the Debian host. |
|
||||
| 9 | Gitea and reproducible releases | A checksummed source package, Ansible-controlled installation, recorded image identities, retained artifacts, and approval tagging. | **In progress.** The runner-free deterministic packager is implemented and the Gitea Action is removed; the release-line identity must be confirmed, then the package, Ansible installation evidence, and approval record must be retained. |
|
||||
| 10 | Debian deployment and persistence | Secure Debian/Compose deployment, HTTPS, persistent database and key volumes, and reboot/recreation proof. | **In progress.** Acceptance tooling is ready, but Docker, correct HTTPS/network exposure, exact artifacts, privileged installation, and the supervised drills remain open. |
|
||||
| 10 | Debian deployment and persistence | Secure Debian/Compose deployment, HTTPS, persistent database and key volumes, and reboot/recreation proof. | **In progress.** The verified Ansible handoff now covers commit-bound installation, boot services, Nginx validation, listener restrictions and public HTTPS; privileged installation, firewall review, controlled reboot and supervised persistence evidence remain open. |
|
||||
| 11 | Backups, monitoring, and recovery | Scheduled encrypted backups, verified off-host transfer, Zabbix monitoring, restore, and rollback rehearsal. | **In progress.** Repository tooling is ready; installation and timed operational evidence are blocked until Milestones 9 and 10 pass. |
|
||||
| 12 | Google mailbox acceptance | End-to-end Gmail consent, import, recovery, reviewed sending, reconciliation, and revocation evidence. | **In progress.** The runbook and validator exist; the live synthetic-data exercise and independent review remain outstanding. |
|
||||
| 13 | Rezlynx/Guestline adapter | The real PMS provider adapter, mappings, idempotency, reconciliation, and ambiguous-write handling. | **Planned.** Provider contract and sandbox access are still required before implementation and acceptance. |
|
||||
@ -60,7 +60,7 @@ This summary explains what each milestone delivers and where it currently stands
|
||||
| 7 | Google connection recovery | B | Implemented / acceptance required | Connection epochs, checkpoint recovery, and revocation handling are promoted to local `main`; complete real Google acceptance and worker-restart exercises. |
|
||||
| 8 | Operational readiness tooling | A | Implemented / acceptance required | Backup, restore, release, and diagnostic tooling is promoted to local `main`; execute it on the actual Debian host and retain evidence. |
|
||||
| 9 | Gitea and reproducible releases | A | In progress | `deploy/package_source.py` now packages only an explicit committed ref, verifies matched application versions, produces deterministic gzip output and a SHA-256 source record, and refuses overwrite. Hand that package to a version-selected Ansible playbook following the CMS/CMSFront pattern. Ansible must verify and install it, build commit-tagged images, record their immutable IDs, and deploy without a Gitea runner. Retain the package/install evidence off-host and resolve the release-line/tag identity before approval; the existing `0.1.0` tag remains attached to the foundation release. |
|
||||
| 10 | Debian deployment and persistence | A | In progress | Compose uses separate named database and shared key volumes, private host configuration, loopback-only API access and bounded logs. The confirmation-gated persistence drill verifies restart and container-recreation behaviour. Run it on the provisioned Debian host, complete HTTPS and controlled-reboot acceptance, and retain the evidence. |
|
||||
| 10 | Debian deployment and persistence | A | In progress | Compose uses separate named database and shared key volumes, private host configuration, loopback-only API access and bounded logs. The Ansible handoff verifies the source on both controller and host, enables Docker/Nginx at boot, installs and validates the reviewed proxy, rejects exposed API/MongoDB listeners, and requires trusted public HTTPS before selecting the release. Run it on the provisioned Debian host, review the firewall, complete the confirmation-gated persistence drill and controlled reboot, and retain independent evidence. |
|
||||
| 11 | Backups, monitoring, and recovery | A | In progress | Encrypted backup and isolated restore tooling now includes opt-in systemd scheduling, checksum-verified rsync transfer, a restricted Zabbix status boundary, guarded local retention and a release-bound acceptance validator. Install and test it on Debian, configure the restricted store and alerts, name operational/review owners, and retain independently reviewed evidence from the timed restore and rollback drill. |
|
||||
| 12 | Google mailbox and reviewed-reply acceptance | B | In progress | The synthetic-data provider runbook, exact scenario set and restricted-record validator are implemented. Complete every scenario against the accepted Debian release and dedicated Google sandbox accounts, independently review the evidence, and retain the validated record. |
|
||||
| 13 | Rezlynx/Guestline adapter | C | Planned | Obtain the provider contract and sandbox, implement the adapter and mapping, and accept idempotency, stale-data, ambiguous-write, and reconciliation paths. |
|
||||
|
||||
@ -11,7 +11,7 @@ Required extra variables:
|
||||
- `guestops_source_record`: controller path to the matching `.source.json`;
|
||||
- `guestops_evidence_directory`: existing restricted controller directory for the fetched release record.
|
||||
|
||||
The target must already have Docker Engine with Compose, Python 3, and `/etc/guestops/guestops.env` owned by root with mode `0600`. The private environment file supplies MongoDB passwords, image references, provider-file paths and disabled-by-default feature flags. Provision it through Ansible Vault or the existing Futuresens secret process, never through this repository.
|
||||
The target must already have Docker Engine with Compose, Nginx, Python 3, `ss`, a trusted certificate under `/etc/letsencrypt/live/sandbox-guestops.futuresens.co.uk`, and `/etc/guestops/guestops.env` owned by root with mode `0600`. The private environment file supplies MongoDB passwords, image references, provider-file paths and disabled-by-default feature flags. Provision it through Ansible Vault or the existing Futuresens secret process, never through this repository.
|
||||
|
||||
Example invocation from the central Ansible checkout:
|
||||
|
||||
@ -26,6 +26,6 @@ ansible-playbook guestops.yml \
|
||||
-e guestops_evidence_directory=/secure/evidence/guestops/0.2.1
|
||||
```
|
||||
|
||||
The controller verifies the source package before transfer. The host independently checks the transferred archive checksum, extracts into a commit-specific directory, verifies the package again, selects the commit-tagged images and disabled send/FAQ defaults in the private environment, builds the API and worker images, records immutable image IDs, runs the offline preflight, validates Compose without printing expanded secrets, starts with `--no-build`, waits for readiness, and only then changes the `current` symlink. The source archive remains in the restricted controller store; the temporary host copy is removed after success.
|
||||
The controller verifies the source package before transfer. The host independently checks the transferred archive checksum, extracts into a commit-specific directory, verifies the package again, selects the commit-tagged images and disabled send/FAQ defaults in the private environment, builds the API and worker images, records immutable image IDs, runs the offline preflight, validates Compose without printing expanded secrets, starts with `--no-build`, and waits for loopback readiness. It then enables Docker and Nginx at boot, validates and installs the reviewed proxy site, rejects public API or MongoDB listeners, and checks the public redirect and certificate-backed HTTPS readiness before changing the `current` symlink. The source archive remains in the restricted controller store; the temporary host copy is removed after success.
|
||||
|
||||
This playbook does not provision DNS, TLS, Nginx, firewall rules, backup keys, monitoring, or the private environment. Those remain explicit Milestone 10 and 11 acceptance activities.
|
||||
|
||||
@ -7,6 +7,8 @@
|
||||
vars:
|
||||
guestops_root: /opt/guestops
|
||||
guestops_config_root: /etc/guestops
|
||||
guestops_public_hostname: sandbox-guestops.futuresens.co.uk
|
||||
guestops_public_origin: "https://{{ guestops_public_hostname }}"
|
||||
guestops_release_root: "{{ guestops_root }}/releases/{{ guestops_release_commit }}"
|
||||
guestops_staging_archive: "/var/tmp/{{ guestops_archive | basename }}"
|
||||
guestops_staging_record: "/var/tmp/{{ guestops_source_record | basename }}"
|
||||
@ -23,6 +25,8 @@
|
||||
- guestops_archive | length > 0
|
||||
- guestops_source_record | length > 0
|
||||
- guestops_evidence_directory | length > 0
|
||||
- guestops_public_hostname is match('^[A-Za-z0-9.-]+$')
|
||||
- guestops_public_hostname == 'sandbox-guestops.futuresens.co.uk'
|
||||
fail_msg: Release version, full commit, archive, SHA-256, source record and evidence directory are required.
|
||||
|
||||
- name: Verify source package on the Ansible controller
|
||||
@ -56,6 +60,28 @@
|
||||
fail_msg: /etc/guestops/guestops.env must already exist with mode 0600.
|
||||
|
||||
tasks:
|
||||
- name: Require supported Debian host
|
||||
ansible.builtin.assert:
|
||||
that:
|
||||
- ansible_facts.system == 'Linux'
|
||||
- ansible_facts.distribution == 'Debian'
|
||||
- ansible_facts.distribution_major_version | int >= 12
|
||||
- ansible_facts.processor_vcpus | default(0) | int >= 4
|
||||
- ansible_facts.memtotal_mb | default(0) | int >= 7300
|
||||
fail_msg: GuestOps requires Debian 12 or newer with at least 4 CPUs and 7.3 GiB RAM.
|
||||
|
||||
- name: Enable Docker at boot
|
||||
ansible.builtin.service:
|
||||
name: docker
|
||||
enabled: true
|
||||
state: started
|
||||
|
||||
- name: Enable Nginx at boot
|
||||
ansible.builtin.service:
|
||||
name: nginx
|
||||
enabled: true
|
||||
state: started
|
||||
|
||||
- name: Create release and evidence directories
|
||||
ansible.builtin.file:
|
||||
path: "{{ item.path }}"
|
||||
@ -264,6 +290,90 @@
|
||||
delay: 2
|
||||
until: guestops_readiness.status == 200
|
||||
|
||||
- name: Confirm TLS certificate files exist
|
||||
ansible.builtin.stat:
|
||||
path: "{{ item }}"
|
||||
follow: true
|
||||
loop:
|
||||
- "/etc/letsencrypt/live/{{ guestops_public_hostname }}/fullchain.pem"
|
||||
- "/etc/letsencrypt/live/{{ guestops_public_hostname }}/privkey.pem"
|
||||
register: guestops_certificates
|
||||
|
||||
- name: Require the pre-provisioned TLS certificate
|
||||
ansible.builtin.assert:
|
||||
that:
|
||||
- guestops_certificates.results | map(attribute='stat.exists') | min
|
||||
- guestops_certificates.results | map(attribute='stat.isreg') | min
|
||||
fail_msg: A valid pre-provisioned Let's Encrypt certificate is required before enabling Nginx.
|
||||
|
||||
- name: Install reviewed GuestOps Nginx site
|
||||
ansible.builtin.copy:
|
||||
src: "{{ playbook_dir }}/../nginx.conf"
|
||||
dest: /etc/nginx/sites-available/guestops.conf
|
||||
owner: root
|
||||
group: root
|
||||
mode: "0644"
|
||||
register: guestops_nginx_site
|
||||
|
||||
- name: Enable GuestOps Nginx site
|
||||
ansible.builtin.file:
|
||||
src: /etc/nginx/sites-available/guestops.conf
|
||||
dest: /etc/nginx/sites-enabled/guestops.conf
|
||||
state: link
|
||||
register: guestops_nginx_enabled
|
||||
|
||||
- name: Validate Nginx configuration
|
||||
ansible.builtin.command:
|
||||
argv: [nginx, -t]
|
||||
changed_when: false
|
||||
|
||||
- name: Reload Nginx after reviewed configuration change
|
||||
ansible.builtin.service:
|
||||
name: nginx
|
||||
enabled: true
|
||||
state: reloaded
|
||||
when: guestops_nginx_site.changed or guestops_nginx_enabled.changed
|
||||
|
||||
- name: Inspect host TCP listeners
|
||||
ansible.builtin.command:
|
||||
argv: [ss, -ltnH]
|
||||
register: guestops_tcp_listeners
|
||||
changed_when: false
|
||||
|
||||
- name: Reject public API or MongoDB listeners
|
||||
ansible.builtin.assert:
|
||||
that:
|
||||
- guestops_tcp_listeners.stdout_lines | select('search', ':8080(\\s|$)') | reject('search', '127\\.0\\.0\\.1:8080(\\s|$)') | list | length == 0
|
||||
- guestops_tcp_listeners.stdout_lines | select('search', ':27017(\\s|$)') | list | length == 0
|
||||
fail_msg: API port 8080 must be loopback-only and MongoDB must have no host listener.
|
||||
|
||||
- name: Verify public HTTP redirects to HTTPS
|
||||
ansible.builtin.uri:
|
||||
url: "http://{{ guestops_public_hostname }}/health/ready"
|
||||
method: GET
|
||||
follow_redirects: none
|
||||
status_code: [301, 302, 307, 308]
|
||||
return_content: false
|
||||
delegate_to: localhost
|
||||
become: false
|
||||
|
||||
- name: Verify public HTTPS readiness and certificate trust
|
||||
ansible.builtin.uri:
|
||||
url: "{{ guestops_public_origin }}/health/ready"
|
||||
method: GET
|
||||
status_code: 200
|
||||
return_content: true
|
||||
validate_certs: true
|
||||
register: guestops_public_readiness
|
||||
retries: 10
|
||||
delay: 3
|
||||
until:
|
||||
- guestops_public_readiness.status == 200
|
||||
- guestops_public_readiness.json is defined
|
||||
- guestops_public_readiness.json.status == 'ready'
|
||||
delegate_to: localhost
|
||||
become: false
|
||||
|
||||
- name: Select the current successful release
|
||||
ansible.builtin.file:
|
||||
src: "{{ guestops_release_root }}"
|
||||
|
||||
@ -32,6 +32,8 @@ Store the archive and source record under a versioned GuestOps files directory i
|
||||
|
||||
The application-owned handoff playbook and required variables are documented in [`deploy/ansible/README.md`](../deploy/ansible/README.md). Import or copy that playbook into the central private Ansible repository, bind its `guestops` inventory group, and keep inventory and secret values there.
|
||||
|
||||
The playbook requires Debian 12 or newer with the approved CPU/RAM baseline, enables Docker and Nginx at boot, installs the reviewed site only after loopback readiness, runs `nginx -t`, rejects a host MongoDB listener or non-loopback API listener, and verifies the public redirect and trusted HTTPS readiness before selecting the release. Firewall reachability, certificate renewal, controlled reboot, durable logs and the persistence exercise still require the supervised checks below.
|
||||
|
||||
```sh
|
||||
docker build --target api -t guestops-api:FULL_40_CHARACTER_SHA .
|
||||
docker build --target worker -t guestops-worker:FULL_40_CHARACTER_SHA .
|
||||
|
||||
@ -15,13 +15,15 @@ class AnsibleHandoffTests(unittest.TestCase):
|
||||
preflight = text.index("Run offline deployment preflight")
|
||||
compose_start = text.index("Start the verified release without rebuilding")
|
||||
readiness = text.index("Wait for loopback readiness")
|
||||
public_readiness = text.index("Verify public HTTPS readiness and certificate trust")
|
||||
current = text.index("Select the current successful release")
|
||||
self.assertLess(controller_verify, target_verify)
|
||||
self.assertLess(target_verify, api_build)
|
||||
self.assertLess(api_build, preflight)
|
||||
self.assertLess(preflight, compose_start)
|
||||
self.assertLess(compose_start, readiness)
|
||||
self.assertLess(readiness, current)
|
||||
self.assertLess(readiness, public_readiness)
|
||||
self.assertLess(public_readiness, current)
|
||||
|
||||
def test_playbook_uses_safe_release_controls(self):
|
||||
text = PLAYBOOK.read_text(encoding="utf-8")
|
||||
@ -37,6 +39,10 @@ class AnsibleHandoffTests(unittest.TestCase):
|
||||
"AUTO_REPLY_ENABLE_LIVE",
|
||||
"no_log: true",
|
||||
"http://127.0.0.1:8080/health/ready",
|
||||
"Reject public API or MongoDB listeners",
|
||||
"Validate Nginx configuration",
|
||||
"validate_certs: true",
|
||||
"enabled: true",
|
||||
):
|
||||
self.assertIn(required, text)
|
||||
self.assertNotIn("ansible.builtin.shell", text)
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user