feat(webhooks): security hardening, tests, full documentation audit & polish (Phase 6)

2026-05-19 17:11:45 +01:00
parent 8609f03c5a
commit 1bef14d590
8 changed files with 888 additions and 22 deletions
@@ -6,6 +6,54 @@ This project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.htm

 ---

+## [1.4.0] - 2026-05-19
+
+### Added
+
+#### Webhook Integration (Phases 1–5.1)
+
+- **Webhook receiver endpoints** — `POST /api/webhook/sonarr` and `POST /api/webhook/radarr` accept push events from Sonarr and Radarr with shared-secret validation (`X-Sofarr-Webhook-Secret` header). Dashboard updates in < 1 second after any grab, import, or failure.
+- **Selective cache invalidation** — each incoming event is classified into *queue events* (`Grab`, `Download`, `DownloadFailed`, `ManualInteractionRequired`) or *history events* (`DownloadFolderImported`, `ImportFailed`, `EpisodeFileRenamed`, `MovieFileRenamed`). Only the affected cache key is refreshed via a lightweight re-poll of that instance, rather than a full poll cycle.
+- **SSE broadcast on webhook** — after refreshing the cache, `pollAllServices()` is called as a fire-and-forget, triggering an immediate SSE push to every connected browser.
+- **Notification management API proxy** — `GET /api/sonarr/api/v3/notification` and `POST /api/sonarr/api/v3/notification` (and Radarr equivalents) proxy the full *arr Notification API through sofarr with auth + CSRF enforcement.
+- **One-click webhook setup** — `POST /api/sonarr/webhook/enable` and `POST /api/radarr/webhook/enable` auto-configure the Sofarr webhook notification connection inside the respective *arr service. `POST /api/sonarr/webhook/test` and `/radarr/webhook/test` trigger a test event.
+- **Webhooks Configuration UI** — collapsible panel in the dashboard allowing users to enable, test, and view webhook status for each configured Sonarr/Radarr instance, with per-trigger type indicators showing which event types are active.
+- **`SOFARR_WEBHOOK_SECRET`** environment variable — required for webhook endpoints to accept requests. Generate with `openssl rand -hex 32`.
+- **`SOFARR_BASE_URL`** environment variable — public URL of sofarr, used by the one-click setup to tell Sonarr/Radarr where to POST events.
+
+#### Smart Polling Optimization (Phase 5)
+
+- **Webhook metrics tracking** — `cache.js` now maintains per-instance and global webhook metrics (`lastWebhookTimestamp`, `eventsReceived`, `pollsSkipped`) via `getWebhookMetrics()`, `updateWebhookMetrics()`, `incrementPollsSkipped()`, `getGlobalWebhookMetrics()`.
+- **Conditional poll skipping** — `poller.js` calls `shouldSkipInstancePolling()` before each Sonarr/Radarr fetch. If all instances of a type have received a webhook event within the fallback timeout window, their queue/history API calls are skipped entirely; existing cached data has its TTL extended instead.
+- **Webhook fallback** — if no webhook events have been received globally for `WEBHOOK_FALLBACK_TIMEOUT` minutes (default: 10), the poller forces a full poll regardless of per-instance state. Logged as `[Poller] Webhook fallback triggered`.
+- **Poll-skip logging** — logs `[Poller] Skipping sonarr/radarr polling for N instance(s) with active webhooks` when polling is skipped.
+- **`WEBHOOK_FALLBACK_TIMEOUT`** environment variable — minutes before fallback polling (default: `10`).
+- **`WEBHOOK_POLL_INTERVAL_MULTIPLIER`** environment variable — internal multiplier for TTL calculations when webhooks are active (default: `3`).
+- **Phase 5.1 metrics connection** — `webhook.js` calls `cache.updateWebhookMetrics(instance.url)` after every successfully validated event, activating the smart skip logic for that instance.
+
+#### Security Hardening (Phase 6)
+
+- **Dedicated webhook rate limiter** — 60 requests per minute per IP on `/api/webhook/*`, stricter than the global 300/15 min API limiter. Bypassed in tests via `SKIP_RATE_LIMIT=1`.
+- **Strict input validation** — `validatePayload()` rejects: non-object bodies, missing/non-string/overlong `eventType`, unrecognised event type values (allowlist of 18 known *arr event types), non-string `instanceName`. Returns `400` with a descriptive message.
+- **Replay protection** — `isReplay()` tracks recently-seen `(eventType, instanceName, date)` tuples in a `Map` with a 5-minute TTL. Duplicate events within the window return `200 { received: true, duplicate: true }` without triggering a cache refresh or SSE broadcast.
+- **35 new webhook integration tests** — cover secret validation (missing/wrong/unconfigured), payload validation (all invalid cases), replay protection, happy-path acceptance of all relevant event types, metrics increment, and secret-never-leaks assertions.
+
+#### Documentation (Phase 6)
+
+- **`README.md`** — updated architecture overview diagram, added Webhooks section with quick-setup guide, added PDCA/PALDRA/webhook architecture table, added Webhooks & Smart Polling env var section, added webhook API endpoints, updated test count.
+- **`CHANGELOG.md`** — this entry.
+- **`SECURITY.md`** — added webhook threat model rows, webhook-specific hardening checklist, and rate-limit table entry for webhook endpoints.
+- **`ARCHITECTURE.md`** (root) — new concise top-level architecture reference describing all pluggable layers and the full webhook + polling optimization flow.
+- **`.env.sample`** — added `WEBHOOK_FALLBACK_TIMEOUT` and `WEBHOOK_POLL_INTERVAL_MULTIPLIER` with explanatory comments; updated NOTES section.
+
+### Changed
+
+- `poller.js` — `pollAllServices()` now conditionally skips Sonarr/Radarr fetches when webhooks are active; extends TTL of existing cache entries instead of overwriting with empty data.
+- `cache.js` — exports four new webhook metrics helpers alongside the existing `MemoryCache` singleton.
+- `webhook.js` — imported `express-rate-limit`; added `validatePayload()`, `isReplay()`, `VALID_EVENT_TYPES` allowlist, `recentEvents` Map, and per-route `webhookLimiter` middleware.
+
+---
+
 ## [1.3.0] - 2026-05-17

 ### Added