grafan update
This commit is contained in:
@@ -1,7 +0,0 @@
|
|||||||
# Copy this file to .env for local use. Do not put passwords in this file.
|
|
||||||
# The directory must contain the three secret files documented in README.md.
|
|
||||||
SECRETS_DIR=./.local/secrets
|
|
||||||
|
|
||||||
# Sandbox-only loopback port for direct Grafana access.
|
|
||||||
GRAFANA_PORT=3000
|
|
||||||
|
|
||||||
+1
-6
@@ -1,7 +1,6 @@
|
|||||||
# Local environment and generated configuration
|
# Local environment and generated secrets
|
||||||
.env
|
.env
|
||||||
.env.*
|
.env.*
|
||||||
!.env.example
|
|
||||||
.local/
|
.local/
|
||||||
|
|
||||||
# Secrets and authentication material
|
# Secrets and authentication material
|
||||||
@@ -18,9 +17,6 @@ secrets/
|
|||||||
|
|
||||||
# Runtime and database state
|
# Runtime and database state
|
||||||
data/
|
data/
|
||||||
postgres-data/
|
|
||||||
grafana-data/
|
|
||||||
loki-data/
|
|
||||||
*.dump
|
*.dump
|
||||||
*.backup
|
*.backup
|
||||||
*.sql.gz
|
*.sql.gz
|
||||||
@@ -38,4 +34,3 @@ Thumbs.db
|
|||||||
.vscode/
|
.vscode/
|
||||||
__pycache__/
|
__pycache__/
|
||||||
.pytest_cache/
|
.pytest_cache/
|
||||||
|
|
||||||
|
|||||||
@@ -1,98 +1,123 @@
|
|||||||
# Kiosk observability proof of concept
|
# Kiosk observability proof of concept
|
||||||
|
|
||||||
This repository contains the first local milestone for the kiosk reporting
|
This repository contains the version-controlled dashboards, documentation, and
|
||||||
platform: Grafana OSS backed by a dedicated PostgreSQL metadata database. Loki,
|
test fixtures for the kiosk reporting platform. The first runnable milestone is
|
||||||
reporting aggregation, Nginx/TLS, tenant bootstrap, and production deployment
|
Grafana OSS backed by PostgreSQL on an Ubuntu 24.04 sandbox.
|
||||||
will be added in later milestones.
|
|
||||||
|
|
||||||
Grafana is available only through a loopback sandbox port. PostgreSQL has no
|
Grafana and PostgreSQL are installed natively through the central Salt and
|
||||||
published host port. Nginx will replace the direct Grafana port when HTTPS is
|
Pillar repositories, following the same operational model as CMS and CMS Front.
|
||||||
introduced.
|
Docker is not used. The Salt deployment templates are authoritative.
|
||||||
|
|
||||||
## Prerequisites
|
## Current milestone
|
||||||
|
|
||||||
- Docker Engine with the Docker Compose plugin
|
The native deployment installs and holds:
|
||||||
- PowerShell 7 or Windows PowerShell 5.1 for the local secret helper
|
|
||||||
|
|
||||||
The pinned images are `grafana/grafana:13.2.2` and
|
- Grafana OSS 13.2.2
|
||||||
`postgres:18.6-alpine`. Do not replace them with `latest`.
|
- PostgreSQL 18.6
|
||||||
|
|
||||||
## Prepare local secrets
|
Grafana stores its users, organizations, data-source configuration, and
|
||||||
|
dashboards in a dedicated PostgreSQL database. Both services listen only on the
|
||||||
|
sandbox loopback interface. Nginx, TLS, Loki, reporting aggregation, dashboards,
|
||||||
|
and production deployment are later milestones.
|
||||||
|
|
||||||
Copy the non-secret environment template and create three random secret files:
|
## Deployment sources
|
||||||
|
|
||||||
```powershell
|
The native automation is split between the existing repositories:
|
||||||
Copy-Item .env.example .env
|
|
||||||
./scripts/initialize-local-secrets.ps1
|
|
||||||
```
|
|
||||||
|
|
||||||
The helper creates these ignored files without printing their values:
|
|
||||||
|
|
||||||
```text
|
```text
|
||||||
.local/secrets/grafana_admin_password
|
grafana/ documentation, future dashboards, tests, and log fixtures
|
||||||
.local/secrets/grafana_database_password
|
salt/ package repositories, installation, configuration, and services
|
||||||
.local/secrets/postgres_admin_password
|
pillar/ non-secret versions, ports, paths, and database identifiers
|
||||||
```
|
```
|
||||||
|
|
||||||
It does not overwrite an existing secret. For a deployed Linux environment,
|
The reusable Salt state is deliberately not assigned to any minion. A sandbox
|
||||||
create equivalent restricted files below `/etc/kiosk-observability/secrets` and
|
minion must be named and authorized before it is added to either `top.sls`.
|
||||||
set `SECRETS_DIR` in the host's untracked `.env` file to that directory.
|
|
||||||
|
|
||||||
## Start and verify
|
## Prepare an Ubuntu sandbox
|
||||||
|
|
||||||
Start the local stack:
|
The target must be Ubuntu 24.04 AMD64 with Salt Minion access and outbound HTTPS
|
||||||
|
access to the official Grafana and PostgreSQL package repositories.
|
||||||
|
|
||||||
```powershell
|
After the sandbox minion is named and deployment is authorized, add that minion
|
||||||
docker compose -f compose.yaml -f compose.sandbox.yaml up -d
|
to the `grafana` state in `salt/top.sls` and to the non-secret `grafana` Pillar
|
||||||
docker compose -f compose.yaml -f compose.sandbox.yaml ps
|
in `pillar/top.sls`. Refresh its Pillar data before applying the commands below.
|
||||||
|
Do not reuse an existing production target for this assignment.
|
||||||
|
|
||||||
|
First install the packages, secret directories, and root-only secret helper:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sudo salt-call state.apply grafana.bootstrap
|
||||||
```
|
```
|
||||||
|
|
||||||
Open <http://127.0.0.1:3000> and sign in as `admin` with the value stored in
|
Generate the two initial secrets on the sandbox:
|
||||||
`.local/secrets/grafana_admin_password`. The health endpoint is
|
|
||||||
<http://127.0.0.1:3000/api/health>.
|
|
||||||
|
|
||||||
Confirm that PostgreSQL is not published to the host:
|
```bash
|
||||||
|
sudo /usr/local/sbin/kiosk-observability-init-secrets
|
||||||
```powershell
|
|
||||||
docker compose -f compose.yaml -f compose.sandbox.yaml port postgres 5432
|
|
||||||
```
|
```
|
||||||
|
|
||||||
The command should report that no public port exists. To confirm Grafana is
|
The helper creates, but never prints or overwrites:
|
||||||
using PostgreSQL, inspect the health response and container logs; the health
|
|
||||||
response should report `"database": "ok"`.
|
|
||||||
|
|
||||||
## Stop or reset
|
```text
|
||||||
|
/etc/kiosk-observability/secrets/grafana_admin_password
|
||||||
Stop containers while preserving their named volumes:
|
/etc/kiosk-observability/secrets/grafana_database_password
|
||||||
|
|
||||||
```powershell
|
|
||||||
docker compose -f compose.yaml -f compose.sandbox.yaml down
|
|
||||||
```
|
```
|
||||||
|
|
||||||
Starting the stack again should retain Grafana metadata. Removing volumes
|
Both files are owned by `root:grafana` and have mode `0640`. If the operator
|
||||||
permanently deletes the local databases and must only be done when an intentional
|
supplies the files through another approved secret-management process, use the
|
||||||
clean reset is required:
|
same paths, ownership, and permissions. The administrator password must satisfy
|
||||||
|
Grafana's strong-password policy; the database password must be a strong
|
||||||
|
single-line Base64 value.
|
||||||
|
|
||||||
```powershell
|
Preview the complete state before applying it:
|
||||||
docker compose -f compose.yaml -f compose.sandbox.yaml down --volumes
|
|
||||||
|
```bash
|
||||||
|
sudo salt-call state.apply grafana test=True
|
||||||
|
sudo salt-call state.apply grafana
|
||||||
```
|
```
|
||||||
|
|
||||||
|
The complete state refuses to start Grafana when a required secret is missing.
|
||||||
|
Applying it a second time should produce no changes.
|
||||||
|
|
||||||
|
## Verify the services
|
||||||
|
|
||||||
|
Run these checks on the sandbox:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sudo systemctl status postgresql grafana-server
|
||||||
|
sudo ss -lntp | grep -E ':(3000|5432)\b'
|
||||||
|
curl --fail --silent http://127.0.0.1:3000/api/health
|
||||||
|
apt-mark showhold | grep -E '^(grafana|postgresql-18)$'
|
||||||
|
```
|
||||||
|
|
||||||
|
Grafana must listen only on `127.0.0.1:3000`, PostgreSQL only on
|
||||||
|
`127.0.0.1:5432`, and the health response must include `"database":"ok"`.
|
||||||
|
|
||||||
|
## Connect through SSH
|
||||||
|
|
||||||
|
Until Nginx and TLS are implemented, do not expose port 3000 on the sandbox.
|
||||||
|
Create a tunnel from the operator workstation:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
ssh -L 3000:127.0.0.1:3000 <user>@<sandbox-host>
|
||||||
|
```
|
||||||
|
|
||||||
|
Open <http://127.0.0.1:3000> locally and sign in as `admin` with the initial
|
||||||
|
password from the sandbox secret file. The configured admin password is used
|
||||||
|
only when Grafana creates its first administrator. Later password rotations must
|
||||||
|
use Grafana's supported administrator workflow and update the external secret
|
||||||
|
record to match.
|
||||||
|
|
||||||
## Supply test logs
|
## Supply test logs
|
||||||
|
|
||||||
Put logs awaiting review in `testing/logs/incoming/`. The directory is present
|
Place logs awaiting review in `testing/logs/incoming/`. The directory is present
|
||||||
in Git, but its contents are ignored. Read `testing/logs/README.md` before adding
|
in Git, but its contents are ignored. Read `testing/logs/README.md` before adding
|
||||||
files. Only fully sanitized, explicitly approved samples may later be placed in
|
files. Only fully sanitized, explicitly approved samples may later be placed in
|
||||||
`tests/fixtures/`.
|
`tests/fixtures/`.
|
||||||
|
|
||||||
## Validate configuration
|
## Safety boundaries
|
||||||
|
|
||||||
After preparing `.env` and the local secrets, render the merged configuration:
|
|
||||||
|
|
||||||
```powershell
|
|
||||||
docker compose -f compose.yaml -f compose.sandbox.yaml config
|
|
||||||
```
|
|
||||||
|
|
||||||
The output must show secret file paths only. It must not contain the contents of
|
|
||||||
any password file. Do not commit `.env`, `.local/`, raw logs, certificates,
|
|
||||||
private keys, database dumps, or runtime data.
|
|
||||||
|
|
||||||
|
- Do not store passwords, tokens, certificates, private keys, database dumps,
|
||||||
|
or real guest logs in Git or Pillar.
|
||||||
|
- Do not assign the Grafana state to a production minion during this milestone.
|
||||||
|
- Do not expose Grafana or PostgreSQL directly to a network.
|
||||||
|
- Do not commit, push, or deploy changes without operator authorization.
|
||||||
|
|||||||
@@ -1,5 +0,0 @@
|
|||||||
services:
|
|
||||||
grafana:
|
|
||||||
ports:
|
|
||||||
- "127.0.0.1:${GRAFANA_PORT:-3000}:3000"
|
|
||||||
|
|
||||||
@@ -1,70 +0,0 @@
|
|||||||
name: kiosk-observability
|
|
||||||
|
|
||||||
services:
|
|
||||||
postgres:
|
|
||||||
image: postgres:18.6-alpine
|
|
||||||
restart: unless-stopped
|
|
||||||
environment:
|
|
||||||
POSTGRES_DB: postgres
|
|
||||||
POSTGRES_USER: postgres_admin
|
|
||||||
POSTGRES_PASSWORD_FILE: /run/secrets/postgres_admin_password
|
|
||||||
secrets:
|
|
||||||
- postgres_admin_password
|
|
||||||
- grafana_database_password
|
|
||||||
volumes:
|
|
||||||
- postgres_data:/var/lib/postgresql
|
|
||||||
- ./postgres/init/10-create-grafana-database.sh:/docker-entrypoint-initdb.d/10-create-grafana-database.sh:ro
|
|
||||||
networks:
|
|
||||||
- backend
|
|
||||||
healthcheck:
|
|
||||||
test: ["CMD-SHELL", "pg_isready -U postgres_admin -d postgres"]
|
|
||||||
interval: 10s
|
|
||||||
timeout: 5s
|
|
||||||
retries: 10
|
|
||||||
start_period: 10s
|
|
||||||
|
|
||||||
grafana:
|
|
||||||
image: grafana/grafana:13.2.2
|
|
||||||
restart: unless-stopped
|
|
||||||
depends_on:
|
|
||||||
postgres:
|
|
||||||
condition: service_healthy
|
|
||||||
environment:
|
|
||||||
GF_DATABASE_TYPE: postgres
|
|
||||||
GF_DATABASE_HOST: postgres:5432
|
|
||||||
GF_DATABASE_NAME: grafana
|
|
||||||
GF_DATABASE_USER: grafana
|
|
||||||
GF_DATABASE_PASSWORD__FILE: /run/secrets/grafana_database_password
|
|
||||||
GF_SECURITY_ADMIN_USER: admin
|
|
||||||
GF_SECURITY_ADMIN_PASSWORD__FILE: /run/secrets/grafana_admin_password
|
|
||||||
secrets:
|
|
||||||
- grafana_admin_password
|
|
||||||
- grafana_database_password
|
|
||||||
volumes:
|
|
||||||
- grafana_data:/var/lib/grafana
|
|
||||||
- ./grafana/grafana.ini:/etc/grafana/grafana.ini:ro
|
|
||||||
- ./grafana/provisioning:/etc/grafana/provisioning:ro
|
|
||||||
networks:
|
|
||||||
- backend
|
|
||||||
healthcheck:
|
|
||||||
test: ["CMD-SHELL", "wget --no-verbose --tries=1 --spider http://localhost:3000/api/health || exit 1"]
|
|
||||||
interval: 10s
|
|
||||||
timeout: 5s
|
|
||||||
retries: 12
|
|
||||||
start_period: 20s
|
|
||||||
|
|
||||||
networks:
|
|
||||||
backend:
|
|
||||||
internal: true
|
|
||||||
|
|
||||||
volumes:
|
|
||||||
grafana_data:
|
|
||||||
postgres_data:
|
|
||||||
|
|
||||||
secrets:
|
|
||||||
grafana_admin_password:
|
|
||||||
file: ${SECRETS_DIR:-./.local/secrets}/grafana_admin_password
|
|
||||||
grafana_database_password:
|
|
||||||
file: ${SECRETS_DIR:-./.local/secrets}/grafana_database_password
|
|
||||||
postgres_admin_password:
|
|
||||||
file: ${SECRETS_DIR:-./.local/secrets}/postgres_admin_password
|
|
||||||
@@ -1,38 +0,0 @@
|
|||||||
app_mode = production
|
|
||||||
|
|
||||||
[server]
|
|
||||||
protocol = http
|
|
||||||
http_addr = 0.0.0.0
|
|
||||||
http_port = 3000
|
|
||||||
domain = localhost
|
|
||||||
root_url = http://localhost:3000/
|
|
||||||
enforce_domain = false
|
|
||||||
|
|
||||||
[database]
|
|
||||||
ssl_mode = disable
|
|
||||||
|
|
||||||
[analytics]
|
|
||||||
reporting_enabled = false
|
|
||||||
check_for_updates = false
|
|
||||||
check_for_plugin_updates = false
|
|
||||||
|
|
||||||
[security]
|
|
||||||
disable_gravatar = true
|
|
||||||
cookie_secure = false
|
|
||||||
cookie_samesite = strict
|
|
||||||
strict_transport_security = false
|
|
||||||
|
|
||||||
[users]
|
|
||||||
allow_sign_up = false
|
|
||||||
allow_org_create = false
|
|
||||||
auto_assign_org = false
|
|
||||||
|
|
||||||
[auth.anonymous]
|
|
||||||
enabled = false
|
|
||||||
|
|
||||||
[log]
|
|
||||||
mode = console
|
|
||||||
level = info
|
|
||||||
|
|
||||||
[paths]
|
|
||||||
provisioning = /etc/grafana/provisioning
|
|
||||||
@@ -1,36 +0,0 @@
|
|||||||
#!/bin/sh
|
|
||||||
set -eu
|
|
||||||
|
|
||||||
password_file=/run/secrets/grafana_database_password
|
|
||||||
|
|
||||||
if [ ! -r "$password_file" ]; then
|
|
||||||
echo "Grafana database password file is missing or unreadable" >&2
|
|
||||||
exit 1
|
|
||||||
fi
|
|
||||||
|
|
||||||
grafana_password=$(cat "$password_file")
|
|
||||||
|
|
||||||
if [ -z "$grafana_password" ]; then
|
|
||||||
echo "Grafana database password must not be empty" >&2
|
|
||||||
exit 1
|
|
||||||
fi
|
|
||||||
|
|
||||||
psql \
|
|
||||||
--set=ON_ERROR_STOP=1 \
|
|
||||||
--set=grafana_password="$grafana_password" \
|
|
||||||
--username "$POSTGRES_USER" \
|
|
||||||
--dbname "$POSTGRES_DB" <<-'EOSQL'
|
|
||||||
SELECT format('CREATE ROLE grafana LOGIN NOSUPERUSER NOCREATEDB NOCREATEROLE NOINHERIT PASSWORD %L', :'grafana_password')
|
|
||||||
WHERE NOT EXISTS (SELECT FROM pg_catalog.pg_roles WHERE rolname = 'grafana') \gexec
|
|
||||||
|
|
||||||
ALTER ROLE grafana PASSWORD :'grafana_password';
|
|
||||||
|
|
||||||
SELECT 'CREATE DATABASE grafana OWNER grafana'
|
|
||||||
WHERE NOT EXISTS (SELECT FROM pg_catalog.pg_database WHERE datname = 'grafana') \gexec
|
|
||||||
|
|
||||||
REVOKE ALL ON DATABASE grafana FROM PUBLIC;
|
|
||||||
GRANT CONNECT, TEMPORARY ON DATABASE grafana TO grafana;
|
|
||||||
EOSQL
|
|
||||||
|
|
||||||
unset grafana_password
|
|
||||||
|
|
||||||
@@ -1,39 +0,0 @@
|
|||||||
[CmdletBinding()]
|
|
||||||
param(
|
|
||||||
[string]$Destination = (Join-Path $PSScriptRoot "..\.local\secrets")
|
|
||||||
)
|
|
||||||
|
|
||||||
$ErrorActionPreference = "Stop"
|
|
||||||
$secretNames = @(
|
|
||||||
"grafana_admin_password",
|
|
||||||
"grafana_database_password",
|
|
||||||
"postgres_admin_password"
|
|
||||||
)
|
|
||||||
|
|
||||||
$resolvedDestination = [System.IO.Path]::GetFullPath($Destination)
|
|
||||||
[System.IO.Directory]::CreateDirectory($resolvedDestination) | Out-Null
|
|
||||||
|
|
||||||
foreach ($secretName in $secretNames) {
|
|
||||||
$secretPath = Join-Path $resolvedDestination $secretName
|
|
||||||
|
|
||||||
if (Test-Path -LiteralPath $secretPath) {
|
|
||||||
Write-Host "Keeping existing secret: $secretName"
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
|
|
||||||
$bytes = New-Object byte[] 32
|
|
||||||
$generator = [System.Security.Cryptography.RandomNumberGenerator]::Create()
|
|
||||||
try {
|
|
||||||
$generator.GetBytes($bytes)
|
|
||||||
}
|
|
||||||
finally {
|
|
||||||
$generator.Dispose()
|
|
||||||
}
|
|
||||||
|
|
||||||
$value = [Convert]::ToBase64String($bytes)
|
|
||||||
[System.IO.File]::WriteAllText($secretPath, $value, [System.Text.UTF8Encoding]::new($false))
|
|
||||||
Write-Host "Created secret: $secretName"
|
|
||||||
}
|
|
||||||
|
|
||||||
Write-Host "Local secrets are ready in $resolvedDestination"
|
|
||||||
|
|
||||||
Reference in New Issue
Block a user