Antoni Nuñez Romeu
155 lines
7.6 KiB
Markdown
155 lines
7.6 KiB
Markdown
# Plan: Grafana FARO + Alloy for FarmaFinder
|
|
|
|
> **Status**: Approved (user answers 2026-06-30).
|
|
> **Path note**: User said `/root/Monitoring/` but the stack lives at `/home/f80ans0/Projects/Monitoring/`.
|
|
|
|
## User decisions
|
|
1. **Network**: FarmaFinder runs on the same host as srv84-macos → use `host.docker.internal:4317/4318`.
|
|
2. **Auth**: Open on `back-tier` only, no auth.
|
|
3. **Existing otel-collector**: Remove it. Alloy is the only collector.
|
|
4. **Faro env**: Build-time via Vite `VITE_FARO_*` build args.
|
|
|
|
## Architecture
|
|
```
|
|
Browser (FarmaFinder PWA, Faro SDK)
|
|
│ OTLP HTTP
|
|
▼
|
|
┌─────────────────────────────────────────────┐
|
|
│ Grafana Alloy :4317 gRPC :4318 HTTP :8889│
|
|
│ otel.receiver → batch → exporters │
|
|
│ prometheus.exporter.self :12345 │
|
|
└──┬─────────────┬──────────────┬─────────────┘
|
|
│ traces │ logs │ metrics (:8889 scraped)
|
|
▼ ▼ ▼
|
|
Tempo:4317 Loki:3100/otlp Prometheus:9090
|
|
|
|
backend (Express) ── OTLP gRPC ─▶ Alloy:4317 (host.docker.internal)
|
|
pip-platform (FastAPI) ── OTLP gRPC ─▶ Alloy:4317 (host.docker.internal)
|
|
scraper (Puppeteer) ── OTLP gRPC ─▶ Alloy:4317 (host.docker.internal)
|
|
API helper (in-process, covered by backend OTel)
|
|
```
|
|
|
|
## Files Changed
|
|
|
|
### Monitoring (`/home/f80ans0/Projects/Monitoring/`)
|
|
- NEW `alloy-config.alloy` — River config (otel receiver, batch, exporters to tempo/loki/prometheus, self-scrape on 12345)
|
|
- MOD `docker-compose.yml` — add `alloy` service on `back-tier` (NO front-tier per Q2), bind 4317/4318/8889/12345 to `0.0.0.0` since same host; REMOVE the `otel-collector` service
|
|
- MOD `prometheus/prometheus.yml` — add `alloy` scrape job for `alloy:12345`
|
|
- MOD `grafana/provisioning/datasources/datasource.yml` — add Tempo + Loki; fix Prometheus URL from `http://localhost:9090` to `http://prometheus:9090`
|
|
- NEW `grafana/provisioning/dashboards/faro-web-vitals.json`
|
|
- NEW `grafana/provisioning/dashboards/faro-js-errors.json`
|
|
- NEW `grafana/provisioning/dashboards/faro-overview.json`
|
|
- NEW `README-FARO-ALLOY.md`
|
|
|
|
### Frontend (`/home/f80ans0/Projects/Webs/FarmaFinder/frontend/`)
|
|
- MOD `package.json` — add `@grafana/faro-react`, `faro-web-sdk`, `faro-web-tracing`, `faro-transport-otlp-http`
|
|
- NEW `src/utils/faro.js`
|
|
- MOD `src/main.jsx` — call `initFaro()` before render
|
|
- MOD `Dockerfile` — accept VITE_FARO_* build args
|
|
- MOD `vite.config.js` — no change needed (Vite reads VITE_* automatically)
|
|
- NEW `.env.example`
|
|
|
|
### Backend Node (`/home/f80ans0/Projects/Webs/FarmaFinder/backend/`)
|
|
- MOD `package.json` — add `@opentelemetry/api`, `sdk-node`, `auto-instrumentations-node`, `exporter-trace-otlp-grpc`, `exporter-logs-otlp-grpc`, `instrumentation-pino`, `pino`, `pino-pretty`
|
|
- NEW `src/tracing.js` — NodeSDK bootstrap (ESM)
|
|
- MOD `server.js` — `import './src/tracing.js';` as line 1; replace some `console.*` with pino
|
|
- MOD root `docker-compose.yml`:
|
|
- `backend.environment` add OTEL_* env vars
|
|
- `frontend.build.args` add `VITE_FARO_ENDPOINT`, `VITE_FARO_APP_NAME`, `VITE_FARO_ENV`
|
|
|
|
### pip-platform
|
|
- MOD `docker-compose.yml` — change `OTEL_EXPORTER_OTLP_ENDPOINT` from `http://otel-collector:4317` to `http://host.docker.internal:4317`; remove the in-stack `otel-collector`, `prometheus`, `grafana` services (or comment out) since they're replaced by the shared stack
|
|
- (For this phase, leave the pip-platform prometheus+grafana running to avoid disrupting the existing setup — change just the OTel endpoint. Document in README.)
|
|
|
|
### scraper
|
|
- MOD `package.json` — add OTel deps
|
|
- NEW `tracing.js`
|
|
- MOD `index.js` — wrap `run()` in a span
|
|
|
|
### API helper
|
|
- No changes (in-process; covered by backend's auto-instrumentation).
|
|
|
|
## Phases (8 total)
|
|
|
|
### Phase 1: Add Grafana Alloy to monitoring stack
|
|
1. Write `/home/f80ans0/Projects/Monitoring/alloy-config.alloy`
|
|
2. Modify `/home/f80ans0/Projects/Monitoring/docker-compose.yml`:
|
|
- Add `alloy` service (image `grafana/alloy:v1.6.1`)
|
|
- Remove the `otel-collector` service
|
|
3. Modify `/home/f80ans0/Projects/Monitoring/prometheus/prometheus.yml` — add `alloy` scrape job
|
|
4. Verify: `docker compose config -q`
|
|
|
|
### Phase 2: Frontend Faro SDK
|
|
1. Modify `frontend/package.json` (add deps)
|
|
2. Create `frontend/src/utils/faro.js`
|
|
3. Modify `frontend/src/main.jsx` (call initFaro)
|
|
4. Modify `frontend/Dockerfile` (build args)
|
|
5. Create `frontend/.env.example`
|
|
6. Verify: `cd frontend && npm ci && npm run build` succeeds
|
|
|
|
### Phase 3: Backend Node OTel + pino
|
|
1. Modify `backend/package.json` (add OTel + pino deps)
|
|
2. Create `backend/src/tracing.js`
|
|
3. Modify `backend/server.js` (import tracing first; add pino)
|
|
4. Modify root `docker-compose.yml` (backend env + frontend build args)
|
|
5. Verify: `cd backend && npm ci` succeeds; container starts; trace appears in Tempo
|
|
|
|
### Phase 4: pip-platform OTel re-route
|
|
1. Modify `pip-platform/docker-compose.yml` — change `OTEL_EXPORTER_OTLP_ENDPOINT`
|
|
2. Verify: pip-api trace appears in Tempo
|
|
|
|
### Phase 5: Scraper one-shot OTel
|
|
1. Modify `scraper/package.json` (add OTel deps)
|
|
2. Create `scraper/tracing.js`
|
|
3. Modify `scraper/index.js` (wrap run in span)
|
|
|
|
### Phase 6: Grafana provisioning
|
|
1. Modify `datasources/datasource.yml` (add Tempo + Loki, fix Prometheus URL)
|
|
2. Create 3 dashboard JSONs (web vitals, js errors, overview)
|
|
3. Verify: dashboards load in Grafana
|
|
|
|
### Phase 7: Documentation
|
|
1. Create `Monitoring/README-FARO-ALLOY.md`
|
|
|
|
### Phase 8: Verification
|
|
1. `docker compose config -q` for both stacks
|
|
2. `docker compose up -d alloy` from Monitoring dir
|
|
3. `curl -sf http://localhost:12345/-/ready` returns 200
|
|
4. `curl -sf -X POST http://localhost:4318/v1/traces` returns 200
|
|
5. Container logs show OTel init for backend + pip-platform
|
|
6. Grafana Explore → Tempo shows traces tagged `service.name=farmafinder-backend`
|
|
7. Grafana Explore → Loki shows `{app="farmafinder-frontend"}` logs
|
|
8. Grafana Explore → Prometheus shows `farovitals_*` metrics
|
|
|
|
## Risks
|
|
- **HIGH** — None remaining (Q1 resolved: same host).
|
|
- **MEDIUM** — Backend uses ESM; OTel NodeSDK needs ESM-compatible setup via `--import` flag in CMD, not just `import './tracing.js'`. Use the OTel SDK's `register` API in ESM, or use the `--import` Node flag with `instrumentation.mjs` shim.
|
|
- **MEDIUM** — Alloy config is River syntax, not YAML. Validate with `alloy fmt` and `alloy validate` if available locally.
|
|
- **LOW** — Dashboard JSON validity.
|
|
- **INFO (security)** — `alertmanager/config.yml` line 25 has a Telegram bot token in plain text. **Do not fix in this PR**; flag to user.
|
|
|
|
## Complexity
|
|
8-12 hours end-to-end.
|
|
|
|
## Verification commands
|
|
```bash
|
|
# Syntax
|
|
docker compose -f /home/f80ans0/Projects/Monitoring/docker-compose.yml config -q && echo "monitoring OK"
|
|
cd /home/f80ans0/Projects/Webs/FarmaFinder && docker compose config -q && echo "farmafinder OK"
|
|
|
|
# Start monitoring stack
|
|
cd /home/f80ans0/Projects/Monitoring && docker compose up -d alloy
|
|
curl -sf http://localhost:12345/-/ready && echo "alloy ready"
|
|
curl -sf -X POST http://localhost:4318/v1/traces -H 'Content-Type: application/x-protobuf' --data-binary @/dev/null -o /dev/null -w '%{http_code}\n'
|
|
|
|
# Start FarmaFinder
|
|
cd /home/f80ans0/Projects/Webs/FarmaFinder && docker compose up -d --build
|
|
docker logs farmafinder-frontend 2>&1 | grep -i faro
|
|
docker logs farmafinder-backend 2>&1 | grep -i opentelemetry
|
|
|
|
# Grafana Explore (manual):
|
|
# Tempo → service.name = farmafinder-backend
|
|
# Loki → {app="farmafinder-frontend"} | json
|
|
# Prometheus → farovitals_largest_contentful_paint
|
|
```
|