feat: add end-to-end observability (metrics, health, mobile RUM, dashboards, alerts)

- Backend: OTel metrics via OTLP -> Alloy -> Prometheus (OTEL_METRICS_EXPORTER=otlp) - New business metrics (src/metrics.js): searches, CIMA latency/errors, cache hits/misses, logins, rate-limits, pharmacy writes/links, push sent/failed, DB + Redis timings/errors, HTTP req count/duration, heartbeat - Backend health endpoints /healthz and /readyz - Mobile (Expo): Grafana Faro RUM via @grafana/faro-react-native - redis/postgres exporters in docker-compose + Prometheus scrape jobs - Grafana dashboards (backend, datastores, mobile RUM, overview) - Prometheus alert rules (farmafinder_*) -> existing Alertmanager (Telegram) - Design/spec saved to docs/superpowers/specs/
2026-07-13 15:57:52 +02:00
parent 2713be85a3
commit 839c64c12a
16 changed files with 907 additions and 27 deletions
@@ -0,0 +1,64 @@
+{
+  "annotations": { "list": [] },
+  "editable": true,
+  "fiscalYearStartMonth": 0,
+  "graphTooltip": 1,
+  "id": null,
+  "links": [],
+  "liveNow": false,
+  "panels": [
+    {
+      "id": 1, "title": "Backend req/s", "type": "stat",
+      "gridPos": { "h": 5, "w": 6, "x": 0, "y": 0 },
+      "datasource": { "type": "prometheus", "uid": "PBFA97CFB590B2093" },
+      "targets": [ { "expr": "sum(rate(http_requests_total{job=\"farmafinder-backend\"}[5m]))", "legendFormat": "req/s" } ]
+    },
+    {
+      "id": 2, "title": "Backend 5xx ratio", "type": "stat",
+      "gridPos": { "h": 5, "w": 6, "x": 6, "y": 0 },
+      "datasource": { "type": "prometheus", "uid": "PBFA97CFB590B2093" },
+      "targets": [ { "expr": "sum(rate(http_requests_total{status_class=\"5xx\",job=\"farmafinder-backend\"}[5m])) / sum(rate(http_requests_total{job=\"farmafinder-backend\"}[5m]))", "legendFormat": "5xx" } ],
+      "fieldConfig": { "defaults": { "unit": "percentunit", "thresholds": { "steps": [ { "color": "green", "value": 0 }, { "color": "yellow", "value": 0.01 }, { "color": "red", "value": 0.05 } ] } } }
+    },
+    {
+      "id": 3, "title": "CIMA errors/s", "type": "stat",
+      "gridPos": { "h": 5, "w": 6, "x": 12, "y": 0 },
+      "datasource": { "type": "prometheus", "uid": "PBFA97CFB590B2093" },
+      "targets": [ { "expr": "sum(rate(cima_requests_total{status=\"error\",job=\"farmafinder-backend\"}[5m]))", "legendFormat": "cima err/s" } ]
+    },
+    {
+      "id": 4, "title": "Mobile events/s", "type": "stat",
+      "gridPos": { "h": 5, "w": 6, "x": 18, "y": 0 },
+      "datasource": { "type": "loki", "uid": "bfowk7qjoccu8b" },
+      "targets": [ { "expr": "sum(count_over_time({app=\"farmafinder-mobile\"}[5m]))", "legendFormat": "mobile/s" } ]
+    },
+    {
+      "id": 5, "title": "Datastore health", "type": "stat",
+      "gridPos": { "h": 5, "w": 12, "x": 0, "y": 5 },
+      "datasource": { "type": "prometheus", "uid": "PBFA97CFB590B2093" },
+      "targets": [
+        { "expr": "redis_up{job=\"farmafinder-redis\"}", "legendFormat": "redis" },
+        { "expr": "pg_up{job=\"farmafinder-postgres\"}", "legendFormat": "postgres" }
+      ]
+    },
+    {
+      "id": 6, "title": "Navigation", "type": "text",
+      "gridPos": { "h": 5, "w": 12, "x": 12, "y": 5 },
+      "options": {
+        "mode": "markdown",
+        "content": "**Dashboards**\n- [Backend](/d/farmafinder-backend/farmafinder-backend)\n- [Datastores](/d/farmafinder-datastores/farmafinder-datastores)\n- [Mobile RUM](/d/farmafinder-mobile-rum/farmafinder-mobile-rum)\n- [Faro Web RUM](/d/faro-overview/faro-farmafinder-rum-overview)\n\n**Explore**\n- [Tempo (traces)](/explore?orgId=1&left=%5B%22now-6h%22,%22now%22,%22tempo%22,%7B%7D%5D)\n- [Loki (logs)](/explore?orgId=1&left=%5B%22now-6h%22,%22now%22,%22loki%22,%7B%7D%5D)\n- [Prometheus (metrics)](/explore?orgId=1&left=%5B%22now-6h%22,%22now%22,%22PBFA97CFB590B2093%22,%7B%7D%5D)"
+      }
+    }
+  ],
+  "schemaVersion": 39,
+  "style": "dark",
+  "tags": ["farmafinder", "overview"],
+  "templating": { "list": [] },
+  "time": { "from": "now-6h", "to": "now" },
+  "timepicker": {},
+  "timezone": "browser",
+  "title": "FarmaFinder — Overview",
+  "uid": "farmafinder-overview",
+  "version": 1,
+  "weekStart": ""
+}