From 67ab378d31e6cee177cc977d9dc76e857ad48ccb Mon Sep 17 00:00:00 2001 From: Gronod Date: Tue, 19 May 2026 18:32:00 +0100 Subject: [PATCH] docs: merge ARCHITECTURE.md files into single consolidated reference - Combine root ARCHITECTURE.md (webhook/smart-polling focused) with docs/ARCHITECTURE.md (deep-dive) into one authoritative document - Structured into 11 sections: Introduction, High-Level Architecture, Pluggable Architecture Layers (PDCA + PALDRA), Webhook System, Data Flow, Caching & Smart Polling, Key Subsystems, Directory Structure, Configuration, Security Model, Technology Stack - Add full-system Mermaid flowchart, webhook sequence diagram, polling cycle sequence diagram, UI state machine, download matching flowchart - Document all cache keys, NormalizedDownload schema, DownloadClientRegistry and arrRetrieverRegistry APIs, webhook event classification table, complete security model with auth/webhook/headers subsections - Remove all development-phase references and internal process language - Remove docs/ARCHITECTURE.md (content consolidated into root file) --- ARCHITECTURE.md | 882 +++++++++++++++++++--- docs/ARCHITECTURE.md | 1716 ------------------------------------------ 2 files changed, 788 insertions(+), 1810 deletions(-) delete mode 100644 docs/ARCHITECTURE.md diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 37ebd24..09b43bf 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -1,35 +1,120 @@ -# sofarr — Architecture Reference +# sofarr — Architecture -> Concise top-level architecture guide. For the full deep-dive (API reference, matching pipeline, deployment) see [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md). +Comprehensive technical reference for the **sofarr** application: a personal media download dashboard that aggregates download activity from SABnzbd, Sonarr, Radarr, qBittorrent, Transmission, and rTorrent, filters results by Emby/Jellyfin user identity, and presents a real-time personalised dashboard. --- -## 1. Overview +## Table of Contents -sofarr is a **Node.js/Express** single-page application. It aggregates download activity from multiple media automation services, filters results by Emby user identity, and presents a real-time personalised dashboard. +1. [Introduction](#1-introduction) +2. [High-Level Architecture](#2-high-level-architecture) +3. [Pluggable Architecture Layers](#3-pluggable-architecture-layers) +4. [Webhook System](#4-webhook-system) +5. [Data Flow and Real-time Updates](#5-data-flow-and-real-time-updates) +6. [Caching and Smart Polling](#6-caching-and-smart-polling) +7. [Key Subsystems](#7-key-subsystems) +8. [Directory Structure](#8-directory-structure) +9. [Configuration and Environment Variables](#9-configuration-and-environment-variables) +10. [Security Model](#10-security-model) +11. [Technology Stack](#11-technology-stack) -Three pluggable layers form the core: +--- + +## 1. Introduction + +sofarr is a **Node.js/Express single-page application** that provides a personalised view of media downloads. It: + +1. **Authenticates** users against an Emby/Jellyfin media server. +2. **Aggregates** download data from multiple *arr service instances and download clients. +3. **Filters** downloads per user — each user only sees media tagged with their username in Sonarr/Radarr. +4. **Presents** a real-time dashboard with progress, speeds, cover art, and status, updated either via background polling or instant webhook push from Sonarr/Radarr. + +Admin users can view all users' downloads, see server status, cache statistics, poll timings, and perform blocklist-and-search operations. + +Three pluggable layers form the architectural core: | Layer | Name | Location | |-------|------|----------| | Download client abstraction | **PDCA** — Pluggable Download Client Architecture | `server/clients/` + `server/utils/downloadClients.js` | | *arr data retrieval | **PALDRA** — Pluggable *Arr Library Data Retrieval Architecture | `server/utils/arrRetrievers.js` | -| Real-time push | **Webhook receiver** | `server/routes/webhook.js` | +| Real-time push | **Webhook Receiver** | `server/routes/webhook.js` | --- -## 2. Request / Data Flow +## 2. High-Level Architecture + +```mermaid +flowchart TB + subgraph Browser["Browser (SPA — public/)"] + login["Login Form"] + dash["Dashboard Cards"] + status["Status Panel\n(Admin only)"] + history["History Tab"] + end + + subgraph Server["Express Server (:3001)"] + direction TB + mw["Middleware\nHelmet · rate-limit · cookie-parser · verifyCsrf"] + auth_r["Auth Routes\n/api/auth"] + dash_r["Dashboard Routes\n/api/dashboard (SSE /stream)"] + wh_r["Webhook Routes\n/api/webhook/sonarr|radarr"] + hist_r["History Routes\n/api/history"] + proxy_r["Proxy Routes\n/api/sonarr · /api/radarr\n/api/sabnzbd · /api/emby"] + + subgraph Core["Core Utilities"] + poller["Poller\n(smart background polling)"] + cache["MemoryCache\n(poll:* + webhook metrics)"] + pdca["PDCA Registry\n(download clients)"] + paldra["PALDRA Registry\n(arr retrievers)"] + tokenstore["TokenStore\n(tokens.json)"] + end + end + + subgraph Ext["External Services"] + sab["SABnzbd"] + sonarr["Sonarr"] + radarr["Radarr"] + qbt["qBittorrent"] + rtorrent["rTorrent"] + transmission["Transmission"] + emby["Emby / Jellyfin"] + end + + login -->|"POST /api/auth/login"| auth_r + dash -->|"GET /api/dashboard/stream (SSE)"| dash_r + status -->|"GET /api/dashboard/status"| dash_r + history -->|"GET /api/history/recent"| hist_r + + auth_r --> tokenstore + auth_r -->|"authenticate"| emby + + dash_r --> cache + dash_r --> poller + wh_r --> cache + wh_r --> paldra + hist_r --> cache + proxy_r -->|"proxy"| sonarr & radarr & sab & emby + + poller --> pdca & paldra + poller --> cache + pdca -->|"HTTP/API"| sab & qbt & rtorrent & transmission + paldra -->|"HTTP/API"| sonarr & radarr + + sonarr & radarr -->|"POST /api/webhook/*"| wh_r +``` + +### Request routing summary ``` Browser (SPA) │ POST /api/auth/login → Auth routes → Emby verify → set httpOnly cookie - │ GET /api/dashboard/stream → SSE stream → poller cache → matched downloads + │ GET /api/dashboard/stream → SSE stream → cache → matched downloads │ POST /api/webhook/* ← Sonarr/Radarr push events │ ▼ Express Server (:3001) ├── Helmet (CSP nonce, HSTS, X-Frame-Options, …) - ├── express-rate-limit (300/15 min general; 60/1 min webhook) + ├── express-rate-limit (300/15 min general; 60/1 min webhook; 10 fails/15 min login) ├── cookie-parser (HMAC-signed session cookie) ├── verifyCsrf (double-submit cookie, all state-changing /api routes except auth + webhook) │ @@ -58,46 +143,177 @@ Background: --- -## 3. Pluggable Download Client Architecture (PDCA) +## 3. Pluggable Architecture Layers -All download clients extend `DownloadClient` (abstract base in `server/clients/DownloadClient.js`): +### 3.1 Pluggable Download Client Architecture (PDCA) + +#### Overview + +The PDCA provides a unified, extensible interface for all download clients. This abstraction layer enables: + +- **Client-agnostic polling** — the poller contains no client-specific logic. +- **Easy extension** — add a new client by implementing one interface. +- **Consistent normalisation** — all clients return standardised download objects. +- **Centralised configuration** — a single registry manages all instances. +- **Error isolation** — individual client failures do not affect other clients. + +#### Abstract Base Class + +All download clients extend `DownloadClient` (`server/clients/DownloadClient.js`): + +```javascript +class DownloadClient { + constructor(instanceConfig) + getClientType(): string + getInstanceId(): string + async testConnection(): Promise + async getActiveDownloads(): Promise + async getClientStatus(): Promise // optional + normalizeDownload(download): NormalizedDownload +} +``` + +#### Client Implementations ``` DownloadClient (abstract) -├── SABnzbdClient — REST API, API key auth +├── SABnzbdClient — REST API, API key auth; handles queue + history; normalises time/size units ├── QBittorrentClient — Sync API (incremental deltas), cookie auth, fallback to /torrents/info ├── TransmissionClient — JSON-RPC, session-ID management -└── RTorrentClient — XML-RPC, HTTP Basic Auth +└── RTorrentClient — XML-RPC (xmlrpc 1.3.2), HTTP Basic Auth; maps rTorrent states to normalised statuses ``` -`DownloadClientRegistry` (`server/utils/downloadClients.js`) initialises all configured clients from `*_INSTANCES` env vars, fetches from all in parallel, and returns a `{ sabnzbd, qbittorrent, transmission, rtorrent }` map. Individual client failures are isolated. +#### Normalised Download Schema -**Adding a new client:** extend `DownloadClient`, implement `getActiveDownloads()` returning `NormalizedDownload[]`, register in the registry factory. +Every client returns objects conforming to this schema: + +```javascript +interface NormalizedDownload { + id: string // Client-specific unique ID + title: string // Download title/name + type: 'usenet' | 'torrent' // Download type + client: string // Client identifier ('sabnzbd', 'qbittorrent', etc.) + instanceId: string // Instance identifier + instanceName: string // Instance display name + status: string // Normalised status (Downloading, Seeding, etc.) + progress: number // Progress percentage (0–100) + size: number // Total size in bytes + downloaded: number // Downloaded bytes + speed: number // Current speed in bytes/sec + eta: number | null // ETA in seconds, null if unknown + category?: string // Download category (optional) + tags?: string[] // Download tags (optional) + savePath?: string // Save path (optional) + addedOn?: string // Added timestamp (optional) + arrQueueId?: number // Sonarr/Radarr queue ID (optional) + arrType?: 'series' | 'movie' // Sonarr/Radarr type (optional) + raw?: any // Original client response (escape hatch) +} +``` + +#### Registry (`server/utils/downloadClients.js`) + +`DownloadClientRegistry` manages all instances: + +```javascript +class DownloadClientRegistry { + async initialize() // Create clients from config + getAllClients(): DownloadClient[] + getClient(instanceId): DownloadClient + getClientsByType(type): DownloadClient[] + async getAllDownloads(): NormalizedDownload[] // Fetch from all clients in parallel + async testAllConnections(): Promise + async getAllClientStatuses(): Promise +} +``` + +**Configuration-driven:** reads from `*_INSTANCES` environment variables (JSON array format) with fallback to legacy `*_URL` / `*_API_KEY` / `*_USERNAME` / `*_PASSWORD` variables. + +#### qBittorrent Sync API Details + +Each `QBittorrentClient` instance maintains: + +- **`lastRid`** — response ID from the previous `sync/maindata` call (starts at `0`). +- **`torrentMap`** — `Map` holding the complete state for every known torrent. +- **`fallbackThisCycle`** — boolean tracking whether this poll cycle has already fallen back to the legacy endpoint. + +Per-cycle flow: +1. Attempt `GET /api/v2/sync/maindata?rid={lastRid}`. +2. If `full_update` is `true`, rebuild `torrentMap` from scratch. +3. Otherwise merge delta fields into existing entries; remove `torrents_removed` hashes. +4. On Sync API failure, fall back **once per cycle** to `GET /api/v2/torrents/info`. +5. If the fallback also fails, return an empty array for this cycle and log the error. + +The rest of the application (poller, dashboard) receives data in the same format regardless of which path was taken. + +#### Adding a New Download Client + +1. Create `server/clients/MyClient.js` extending `DownloadClient`. +2. Implement `getActiveDownloads()` returning `NormalizedDownload[]`. +3. Register the class in the registry factory inside `server/utils/downloadClients.js`. --- -## 4. Pluggable *Arr Retrieval Architecture (PALDRA) +### 3.2 Pluggable *arr Retrieval Layer (PALDRA) -`server/utils/arrRetrievers.js` provides `arrRetrieverRegistry` which: -- Initialises one retriever per configured Sonarr/Radarr instance -- Exposes `getQueuesByType()`, `getHistoryByType()`, `getTagsByType()` — returning results keyed by `sonarr` / `radarr` -- Results carry `{ instance: instanceId, data: … }` so callers can look up instance credentials +#### Overview -The poller and webhook processor both use the same registry, ensuring consistency. +`server/utils/arrRetrievers.js` exports `arrRetrieverRegistry`, a singleton that manages one `PollingSonarrRetriever` or `PollingRadarrRetriever` per configured instance. It provides a uniform interface for fetching queue, history, and tag data, keyed by service type. + +The registry is used by both the background poller and the webhook processor, guaranteeing consistent data shapes across both update paths. + +#### Registry API + +```javascript +arrRetrieverRegistry = { + async initialize() // idempotent; reads config once + getAllRetrievers(): ArrRetriever[] + getRetriever(instanceId): ArrRetriever | null + getRetrieversByType(type): ArrRetriever[] // 'sonarr' | 'radarr' + + // Typed fetch methods — all return { sonarr: [...], radarr: [...] } + async getQueuesByType(): Promise<{ sonarr, radarr }> + async getHistoryByType(options?): Promise<{ sonarr, radarr }> + async getTagsByType(): Promise<{ sonarr, radarr }> +} +``` + +Each result element is `{ instance: instanceId, data: }`, allowing callers to look up instance credentials from `config.js`. + +#### Retriever API Calls + +| Task | Endpoint | Key Parameters | +|------|----------|----------------| +| Sonarr tags | `GET /api/v3/tag` | — | +| Sonarr queue | `GET /api/v3/queue` | `includeSeries=true`, `includeEpisode=true` | +| Sonarr history | `GET /api/v3/history` | `pageSize=10`, `includeEpisode=true` | +| Radarr tags | `GET /api/v3/tag` | — | +| Radarr queue | `GET /api/v3/queue` | `includeMovie=true` | +| Radarr history | `GET /api/v3/history` | `pageSize=10` | + +All fetches across all instances run in parallel via `Promise.allSettled`, so a single failing instance does not block others. --- -## 5. Webhook Flow (Phase 1–5.1) +## 4. Webhook System + +### 4.1 Webhook Receiver + +sofarr exposes two webhook endpoints that Sonarr and Radarr can be configured to call on every automation event: + +``` +POST /api/webhook/sonarr +POST /api/webhook/radarr +``` + +Both endpoints share identical processing logic: ``` Sonarr/Radarr - POST /api/webhook/sonarr (X-Sofarr-Webhook-Secret: ) - { - "eventType": "Grab", - "instanceName": "Main Sonarr", - "date": "2026-05-19T10:00:00.000Z", - … - } + POST /api/webhook/sonarr + Headers: X-Sofarr-Webhook-Secret: + Body: { "eventType": "Grab", "instanceName": "Main Sonarr", + "date": "2026-05-19T10:00:00.000Z", … } │ ▼ webhookLimiter (60 req/min/IP) @@ -115,119 +331,597 @@ Sonarr/Radarr cache.updateWebhookMetrics(instance.url) ← activates smart polling skip │ ▼ - processWebhookEvent('sonarr', 'Grab') [fire-and-forget] - ├── classify: Grab → QUEUE_EVENT - ├── arrRetrieverRegistry.getQueuesByType() - ├── cache.set('poll:sonarr-queue', …, CACHE_TTL) - └── pollAllServices() → pollSubscribers.forEach(cb) → SSE push + 200 { received: true } ← response sent immediately │ - ▼ - 200 { received: true } (returned immediately, before fire-and-forget completes) + ▼ (fire-and-forget) + processWebhookEvent(serviceType, eventType) + ├── classify: QUEUE_EVENT or HISTORY_EVENT + ├── arrRetrieverRegistry.getQueuesByType() / getHistoryByType() + ├── cache.set('poll:sonarr-queue' | 'poll:sonarr-history', …, CACHE_TTL) + └── pollAllServices() → pollSubscribers.forEach(cb) → SSE push ``` +The 200 response is sent **before** the background cache refresh completes, ensuring Sonarr/Radarr never have to wait for sofarr's downstream API calls. + +#### Event Classification + +| Event type | Classification | Cache keys refreshed | +|------------|---------------|---------------------| +| `Grab`, `Download`, `DownloadFailed`, `ManualInteractionRequired` | `QUEUE_EVENT` | `poll:{type}-queue` | +| `DownloadFolderImported`, `ImportFailed`, `EpisodeFileRenamed`, `MovieFileRenamed`, `EpisodeFileRenamedBySeries` | `HISTORY_EVENT` | `poll:{type}-history` | +| `Test`, `Rename`, `SeriesAdd`, `SeriesDelete`, `MovieAdd`, `MovieDelete`, `MovieFileDelete`, `Health`, `ApplicationUpdate`, `HealthRestored` | Informational — no refresh | — | + +#### Accepted Event Types + +The full allowlist enforced by `validatePayload()`: + +``` +Test · Grab · Download · DownloadFailed · ManualInteractionRequired +DownloadFolderImported · ImportFailed +EpisodeFileRenamed · MovieFileRenamed · EpisodeFileRenamedBySeries +Rename · SeriesAdd · SeriesDelete · MovieAdd · MovieDelete · MovieFileDelete +Health · ApplicationUpdate · HealthRestored +``` + +Any `eventType` not in this set is rejected with `400 Bad Request`. + --- -## 6. Smart Polling Optimization (Phase 5) +### 4.2 Real-time Cache and SSE Integration + +When a webhook event is classified as a `QUEUE_EVENT` or `HISTORY_EVENT`: + +1. `arrRetrieverRegistry` fetches fresh data from the relevant *arr instances (in parallel, via PALDRA). +2. The result is written directly into the shared `MemoryCache` under the same `poll:*` key the poller uses — ensuring both paths produce identical cache shapes. +3. `pollAllServices()` is called, which iterates `pollSubscribers` and pushes the updated payload to every open SSE connection immediately. + +The dashboard therefore receives fresh data within the round-trip time of the *arr API call, without waiting for the next poll cycle. + +--- + +### 4.3 Notification Management API + +The `sonarr.js` and `radarr.js` route modules expose endpoints (under `/api/sonarr` and `/api/radarr` respectively, behind `requireAuth` + `verifyCsrf`) for one-click webhook configuration. These proxy to the *arr notification API to create, update, or remove the sofarr webhook connection in Sonarr/Radarr, using `SOFARR_BASE_URL` to construct the target URL. + +--- + +## 5. Data Flow and Real-time Updates + +### 5.1 Polling Cycle (background path) + +Every `POLL_INTERVAL` ms the poller fetches all services in parallel: + +| Task | API | Key parameters | +|------|-----|----------------| +| SABnzbd Queue | `GET /api?mode=queue` | `output=json` | +| SABnzbd History | `GET /api?mode=history` | `limit=10` | +| Sonarr Tags | `GET /api/v3/tag` | — | +| Sonarr Queue | `GET /api/v3/queue` | `includeSeries=true`, `includeEpisode=true` | +| Sonarr History | `GET /api/v3/history` | `pageSize=10`, `includeEpisode=true` | +| Radarr Tags | `GET /api/v3/tag` | — | +| Radarr Queue | `GET /api/v3/queue` | `includeMovie=true` | +| Radarr History | `GET /api/v3/history` | `pageSize=10` | +| qBittorrent | `GET /api/v2/sync/maindata?rid=N` | Fallback: `GET /api/v2/torrents/info` | + +Results are stored in `MemoryCache` under `poll:*` keys with TTL `POLL_INTERVAL × 3`. Per-task timings are recorded in `lastPollTimings` for the admin status panel. + +```mermaid +sequenceDiagram + participant Entry as index.js + participant Poller + participant PDCA as PDCA Registry + participant PALDRA as PALDRA Registry + participant Cache as MemoryCache + participant SSE as SSE Subscribers + + Entry->>Poller: startPoller() + loop Every POLL_INTERVAL ms + Poller->>Poller: polling flag check (skip if concurrent) + Poller->>PDCA: getDownloadsByClientType() + Poller->>PALDRA: getQueuesByType() / getHistoryByType() / getTagsByType() + PDCA-->>Poller: { sabnzbd, qbittorrent, rtorrent, transmission } + PALDRA-->>Poller: { sonarr: [...], radarr: [...] } + Poller->>Cache: set poll:* keys (TTL = POLL_INTERVAL × 3) + Poller->>SSE: notify all subscribers → push data: frame + end +``` + +### 5.2 Webhook Path (real-time update) + +```mermaid +sequenceDiagram + participant Arr as Sonarr/Radarr + participant WH as /api/webhook/sonarr + participant Cache as MemoryCache + participant PALDRA as PALDRA Registry + participant SSE as SSE Subscribers + + Arr->>WH: POST /api/webhook/sonarr { eventType, instanceName, date } + WH->>WH: validateSecret + validatePayload + isReplay + WH->>Cache: updateWebhookMetrics(instance.url) + WH-->>Arr: 200 { received: true } + Note over WH: fire-and-forget begins + WH->>PALDRA: getQueuesByType() or getHistoryByType() + PALDRA-->>WH: fresh arr data + WH->>Cache: set poll:sonarr-queue / poll:sonarr-history + WH->>SSE: pollAllServices() → push data: frame to all clients +``` + +### 5.3 SSE Stream + +When a browser opens `GET /api/dashboard/stream`: + +1. Server sets `Content-Type: text/event-stream`, disables buffering (`X-Accel-Buffering: no`). +2. Immediately builds and sends the first payload (same matching logic as `/user-downloads`). +3. Registers a callback with the poller's `onPollComplete` subscriber set. +4. After every subsequent poll cycle (or webhook-triggered broadcast), the callback fires, rebuilds the payload, and writes a `data:` SSE frame. +5. A 25-second heartbeat comment (`: heartbeat`) keeps the connection alive through proxies. +6. On client disconnect: deregisters callback, stops heartbeat, removes from `activeClients` map. + +The browser's native `EventSource` API handles reconnection automatically on network interruption. + +### 5.4 Download Matching Pipeline + +For each connected user the server: + +1. Reads all `poll:*` keys from `MemoryCache`. +2. Builds `seriesMap`, `moviesMap`, `sonarrTagMap`, and `radarrTagMap` from embedded objects in queue records. +3. For each SABnzbd/qBittorrent download item, attempts matches in priority order: Sonarr queue → Radarr queue → Sonarr history → Radarr history. +4. Title matching is a **bidirectional, case-insensitive substring match**: `rTitle.includes(dlTitle) || dlTitle.includes(rTitle)`. +5. For each match, resolves the series/movie, extracts user tags, checks ownership. +6. Returns only the requesting user's downloads (or all, if admin with `showAll=true`). + +```mermaid +flowchart TD + Start(["Download item"]) --> SQ{"Sonarr QUEUE\nmatch (title)"} + SQ -->|yes| SQR["Resolve series · extract user tag"] + SQ -->|no| RQ{"Radarr QUEUE\nmatch (title)"} + RQ -->|yes| RQR["Resolve movie · extract user tag"] + RQ -->|no| SH{"Sonarr HISTORY\nmatch (title)"} + SH -->|yes| SHR["Resolve series via seriesId"] + SH -->|no| RH{"Radarr HISTORY\nmatch (title)"} + RH -->|yes| RHR["Resolve movie via movieId"] + RH -->|no| Skip(["Skip — unmatched"]) + + SQR & RQR & SHR & RHR --> Tagged{"Tag matches\nrequesting user?"} + Tagged -->|yes| Include(["Include in response"]) + Tagged -->|no| Skip +``` + +#### Tag matching + +Users are matched to downloads via Sonarr/Radarr tags: + +1. **Exact match** — tag label (lowercased) === username (lowercased). +2. **Sanitised match** — handles Ombi tag mangling: `sanitizeTagLabel()` converts to lowercase, replaces non-alphanumeric chars with hyphens, collapses runs, trims. + +#### Matched download object fields + +| Field | Type | Description | +|-------|------|-------------| +| `type` | `'series'`/`'movie'`/`'torrent'` | Media type | +| `title` | string | Raw download title | +| `coverArt` | string/null | Poster URL from *arr | +| `status` | string | Download status | +| `progress` | string | Percentage complete | +| `size`/`mb`/`mbmissing` | string/number | Size info | +| `speed` | string | Current download speed | +| `eta` | string | Estimated time remaining | +| `seriesName`/`movieName` | string | Friendly media title | +| `episodes` | `{season, episode, title}[]` | Episodes covered (sorted); empty array if Sonarr has no data | +| `allTags` | string[] | All resolved tag labels on the series/movie | +| `matchedUserTag` | string/null | Tag label matching the requesting user | +| `tagBadges` | `{label, matchedUser}[]`/undefined | (Admin `showAll` only) each tag classified against Emby user list | +| `importIssues` | string[]/null | Import warning/error messages | +| `canBlocklist` | boolean | `true` if the current user may blocklist this download | +| `downloadPath` | string/null | (Admin) Download client path | +| `targetPath` | string/null | (Admin) *arr target path | +| `arrLink` | string/null | (Admin) Link to *arr web UI | +| `arrQueueId` | number/null | (Admin) Sonarr/Radarr queue record id | +| `arrType` | `'sonarr'`/`'radarr'`/null | (Admin) Which *arr service owns this queue entry | +| `arrInstanceUrl` | string/null | (Admin) Base URL of the *arr instance | +| `arrInstanceKey` | string/null | (Admin) API key for the *arr instance | +| `arrContentId` | number/null | (Admin) `episodeId` or `movieId` for triggering a new search | +| `arrContentType` | `'episode'`/`'movie'`/null | (Admin) Content type for search command | +| `addedOn` | number/null | (qBittorrent) Unix timestamp when torrent was added | +| `availableForUpgrade` | boolean/undefined | (History) `true` when outcome is `failed` but content is on disk | + +--- + +## 6. Caching and Smart Polling + +### 6.1 Cache Layer + +`server/utils/cache.js` exports a singleton `MemoryCache` backed by a `Map`. Each entry carries an expiration timestamp. The cache is shared by the poller, webhook processor, and all route modules. + +```javascript +class MemoryCache { + get(key): any + set(key, value, ttlMs) + invalidate(key) + clear() + getStats(): CacheStats // per-key size, item count, TTL remaining + + // Webhook metrics helpers + updateWebhookMetrics(instanceUrl) + getWebhookMetrics(instanceUrl): { eventsReceived, lastWebhookTimestamp, pollsSkipped } + getGlobalWebhookMetrics(): { lastGlobalWebhookTimestamp } +} +``` + +### 6.2 Cache Keys + +| Key | Content | TTL | +|-----|---------|-----| +| `poll:sab-queue` | `{ slots, status, speed, kbpersec }` | `POLL_INTERVAL × 3` | +| `poll:sab-history` | `{ slots }` | `POLL_INTERVAL × 3` | +| `poll:sonarr-queue` | `{ records }` with embedded `series` objects + `_instanceUrl`/`_instanceKey` | `POLL_INTERVAL × 3` | +| `poll:sonarr-history` | `{ records }` — lightweight, no embedded objects | `POLL_INTERVAL × 3` | +| `poll:sonarr-tags` | `[{ instance, data: [{id, label}] }]` | `POLL_INTERVAL × 3` | +| `poll:radarr-queue` | `{ records }` with embedded `movie` objects + `_instanceUrl`/`_instanceKey` | `POLL_INTERVAL × 3` | +| `poll:radarr-history` | `{ records }` — lightweight | `POLL_INTERVAL × 3` | +| `poll:radarr-tags` | `[{ instance, data: [{id, label}] }]` | `POLL_INTERVAL × 3` | +| `poll:qbittorrent` | `[torrent, …]` | `POLL_INTERVAL × 3` | +| `history:sonarr` | `[record, …]` flat array with `_instanceUrl`/`_instanceName` | 5 min | +| `history:radarr` | `[record, …]` flat array with `_instanceUrl`/`_instanceName` | 5 min | +| `emby:users` | `Map` | 60 s | + +When polling is disabled (`POLL_INTERVAL=0`), all `poll:*` TTLs fall back to **30 s** and data is fetched on-demand when the dashboard finds an empty cache entry. + +### 6.3 Background Polling Modes + +| Mode | `POLL_INTERVAL` | Behaviour | +|------|----------------|-----------| +| **Background** | `> 0` (e.g. `5000`) | Periodic fetch every N ms; SSE subscribers notified after each cycle | +| **On-demand** | `0` / `off` / `false` | Fetch triggered by first dashboard request when cache is empty; cached 30 s | + +The poller uses a `polling` boolean flag to prevent concurrent cycles: if an interval fires while the previous poll is still running, the new invocation is skipped and logged. + +### 6.4 Smart Polling Optimisation + +When Sonarr/Radarr are configured to send webhooks to sofarr, the poller automatically reduces unnecessary API calls: ``` pollAllServices() called every POLL_INTERVAL ms: globalMetrics = cache.getGlobalWebhookMetrics() fallbackTriggered = lastGlobalWebhookTimestamp > WEBHOOK_FALLBACK_TIMEOUT ago - + for each service type (sonarr, radarr): shouldSkip = !fallbackTriggered && all instances have metrics.eventsReceived > 0 && all instances have metrics.lastWebhookTimestamp within WEBHOOK_FALLBACK_TIMEOUT if shouldSkip: - extend TTL of existing cached data ← no API calls made + extend TTL of existing cached data ← zero *arr API calls increment metrics.pollsSkipped log "[Poller] Skipping sonarr polling for N instance(s) with active webhooks" else: fetch from *arr APIs → update cache ``` -**Result:** zero *arr API calls per poll cycle when webhooks are active and recent. Falls back automatically after `WEBHOOK_FALLBACK_TIMEOUT` minutes of silence (default: 10). +**Effect:** zero *arr API calls per poll cycle when webhooks are active and recent. The poller automatically falls back to full polling after `WEBHOOK_FALLBACK_TIMEOUT` minutes of silence (default: 10 minutes), ensuring the dashboard remains accurate even if webhooks stop arriving. + +### 6.5 Active SSE Client Tracking + +SSE connections are tracked precisely in `activeClients` (a `Map` keyed by `${username}:${connectedAt}`): registered on connect, removed on disconnect. The admin status panel shows each connected user and their connection duration. The `type: 'sse'` field distinguishes SSE clients from other connection types. --- -## 7. Cache Keys +## 7. Key Subsystems -| Key | Content | TTL | -|-----|---------|-----| -| `poll:sab-queue` | SABnzbd queue slots + status | `POLL_INTERVAL × 3` | -| `poll:sab-history` | SABnzbd history slots | `POLL_INTERVAL × 3` | -| `poll:sonarr-queue` | Sonarr queue records (with `_instanceUrl`) | `POLL_INTERVAL × 3` | -| `poll:sonarr-history` | Sonarr history records | `POLL_INTERVAL × 3` | -| `poll:sonarr-tags` | Sonarr tag list per instance | `POLL_INTERVAL × 3` | -| `poll:radarr-queue` | Radarr queue records (with `_instanceUrl`) | `POLL_INTERVAL × 3` | -| `poll:radarr-history` | Radarr history records | `POLL_INTERVAL × 3` | -| `poll:radarr-tags` | Radarr tag list | `POLL_INTERVAL × 3` | -| `poll:qbittorrent` | qBittorrent torrent list | `POLL_INTERVAL × 3` | -| `history:sonarr` | Sonarr history (on-demand, `/api/history/recent`) | 5 min | -| `history:radarr` | Radarr history (on-demand) | 5 min | -| `emby:users` | Emby user list | 60 s | +### 7.1 Download Clients -When polling is disabled (`POLL_INTERVAL=0`), all `poll:*` TTLs fall back to 30 s. +See [Section 3.1](#31-pluggable-download-client-architecture-pdca) for full detail. The client hierarchy is: + +``` +DownloadClient (abstract — server/clients/DownloadClient.js) +├── SABnzbdClient.js — Usenet; REST; API key auth +├── QBittorrentClient.js — Torrent; Sync API + fallback; cookie auth +├── TransmissionClient.js — Torrent; JSON-RPC; session-ID management +└── RTorrentClient.js — Torrent; XML-RPC; HTTP Basic Auth +``` + +`server/utils/qbittorrent.js` is a legacy compatibility shim that delegates to `QBittorrentClient`. + +### 7.2 Queue & History Processing + +**`server/utils/historyFetcher.js`** fetches history records from all Sonarr/Radarr instances for a configurable date window. Results are cached under `history:sonarr` / `history:radarr` for 5 minutes. Exports `classifySonarrEvent` / `classifyRadarrEvent` (returns `'imported'` | `'failed'` | `'other'`) and `invalidateHistoryCache`. + +**`server/routes/history.js`** (`GET /api/history/recent`) returns recently completed (imported or failed) downloads filtered for the authenticated user. Supports `?days=N` (default `RECENT_COMPLETED_DAYS`, capped at 90) and `?showAll=true` for admins. Results are sorted newest first. + +**`server/routes/dashboard.js`** (`POST /api/dashboard/blocklist-search`) removes a Sonarr/Radarr queue item with `blocklist=true` and immediately triggers an `EpisodeSearch` or `MoviesSearch` command. Non-admin users may only blocklist when import issues are present, or (for qBittorrent only) the torrent is over 1 hour old with less than 100% availability. + +### 7.3 Dashboard & Frontend + +The frontend is a **vanilla JavaScript SPA** with no build step. All logic resides in `public/app.js`, styled by `public/style.css`, and structured by `public/index.html`. Three CSS themes are available via the `data-theme` attribute on `` and persist in `localStorage`: + +- **Light** — Purple gradient header, white cards +- **Dark** — Dark surfaces, muted accents +- **Mono** — Monochrome, minimal colour + +#### UI state machine + +```mermaid +stateDiagram-v2 + [*] --> SplashScreen : Page load + SplashScreen --> CheckAuth : checkAuthentication() + + state CheckAuth <> + CheckAuth --> LoginForm : No session + CheckAuth --> Dashboard : Valid session + + LoginForm --> Dashboard : Auth success (fade transition) + Dashboard --> LoginForm : Logout (stopSSE) + + state Dashboard { + [*] --> Rendering + Rendering --> Rendering : SSE message → renderDownloads() + + state SSEConnection { + [*] --> Connecting + Connecting --> Connected : First message + Connected --> Reconnecting : Connection lost + Reconnecting --> Connected : Auto-reconnect + Connected --> Connecting : showAll toggled + } + + state StatusPanel { + [*] --> Closed + Closed --> Open : Click Status (admin) + Open --> Closed : Click close + Open --> Open : 5s timer refresh + } + } +``` + +#### Key frontend functions + +| Function | Purpose | +|----------|---------| +| `checkAuthentication()` | On load: check session → show dashboard or login | +| `handleLogin()` | Authenticate, fade login → splash → dashboard | +| `startSSE()` | Open `EventSource` to `/stream`; handle incoming data | +| `stopSSE()` | Close `EventSource` and cancel reconnect timer | +| `renderDownloads()` | Diff-based card rendering (create/update/remove) | +| `createDownloadCard()` | Build DOM for a single card; renders tag badges, import-issue badge, blocklist button | +| `updateDownloadCard()` | Update existing card in-place (progress, speed, etc.) | +| `handleBlocklistSearch()` | Confirm dialog → POST `/blocklist-search` → update button state | +| `toggleStatusPanel()` | Show/hide admin status panel | +| `renderStatusPanel()` | Build status HTML (server, polling, SSE clients, cache) | +| `initThemeSwitcher()` | Light / Dark / Mono theme support | +| `loadHistory()` | Fetch `/api/history/recent`, store raw items, call `renderHistory()` | +| `renderHistory()` | Filter items by `ignoreAvailable` flag, render history cards | + +#### Tag badge rendering + +- **Regular user view** — a single accent-coloured badge showing the tag label that matched the current user's username (via `matchedUserTag`). +- **Admin `showAll` view** — all tags on the download are rendered using `tagBadges[]`: tags with no matching Emby user → amber badge (leftmost); tags matched to a known Emby user → accent badge showing the Emby display name (rightmost). --- -## 8. Security Model - -| Concern | Mechanism | -|---------|-----------| -| User authentication | Emby credentials → httpOnly HMAC-signed cookie | -| Session validation | `requireAuth` middleware on all `/api/dashboard`, `/api/history`, proxy routes | -| CSRF | Double-submit cookie (`X-CSRF-Token` header) on all state-changing routes | -| Webhook auth | Shared secret on `X-Sofarr-Webhook-Secret` header (webhook routes are outside CSRF) | -| Webhook input | `validatePayload()` allowlists event types; rejects invalid shapes | -| Webhook replay | 5-minute nonce cache keyed on `(eventType, instanceName, date)` | -| Rate limiting | 300 req/15 min (general), 10 fails/15 min (login), 60 req/1 min (webhook) | -| Secret leakage | `sanitizeError()` redacts all secrets from error messages and logs | -| Headers | Helmet v7: CSP nonce, HSTS, X-Frame-Options DENY, noSniff, Referrer-Policy | - ---- - -## 9. Directory Structure (summary) +## 8. Directory Structure ``` sofarr/ ├── server/ -│ ├── app.js Express factory (imported by tests + index.js) -│ ├── index.js Entry point: logging, listen, start poller -│ ├── clients/ PDCA — one file per download client +│ ├── app.js Express app factory — imported by tests and index.js +│ ├── index.js Entry point: logging setup, server listen, poller start +│ ├── clients/ PDCA — one file per download client + retriever +│ │ ├── DownloadClient.js Abstract base class for all download clients +│ │ ├── QBittorrentClient.js +│ │ ├── SABnzbdClient.js +│ │ ├── TransmissionClient.js +│ │ ├── RTorrentClient.js +│ │ ├── PollingSonarrRetriever.js PALDRA — Sonarr retriever +│ │ └── PollingRadarrRetriever.js PALDRA — Radarr retriever │ ├── routes/ -│ │ ├── auth.js Login / logout / csrf / me -│ │ ├── dashboard.js SSE stream, downloads, status, cover-art -│ │ ├── history.js Recently completed downloads -│ │ ├── webhook.js Webhook receiver (Phase 1–6) +│ │ ├── auth.js POST /login, GET /me, GET /csrf, POST /logout +│ │ ├── dashboard.js SSE /stream, /user-downloads, /user-summary, /status, /cover-art, /blocklist-search +│ │ ├── history.js GET /api/history/recent +│ │ ├── webhook.js POST /api/webhook/sonarr|radarr │ │ ├── sonarr.js Sonarr API proxy + webhook management -│ │ └── radarr.js Radarr API proxy + webhook management +│ │ ├── radarr.js Radarr API proxy + webhook management +│ │ ├── emby.js Emby API proxy +│ │ └── sabnzbd.js SABnzbd API proxy │ ├── middleware/ -│ │ ├── requireAuth.js Cookie auth enforcement -│ │ └── verifyCsrf.js Double-submit CSRF check +│ │ ├── requireAuth.js httpOnly cookie auth enforcement +│ │ └── verifyCsrf.js Double-submit CSRF check (timing-safe) │ └── utils/ -│ ├── arrRetrievers.js PALDRA — Sonarr/Radarr fetch registry +│ ├── arrRetrievers.js PALDRA registry — Sonarr/Radarr fetch registry │ ├── cache.js MemoryCache + webhook metrics helpers │ ├── config.js Multi-instance config parser │ ├── downloadClients.js PDCA registry + factory │ ├── historyFetcher.js History fetch + event classification +│ ├── logger.js File logger (DATA_DIR/server.log) │ ├── poller.js Smart background polling engine -│ ├── sanitizeError.js Secret redaction from errors -│ └── tokenStore.js Emby token store (JSON file, atomic writes) -├── public/ Static SPA (HTML + CSS + vanilla JS) +│ ├── qbittorrent.js Legacy compatibility shim → QBittorrentClient +│ ├── sanitizeError.js Secret redaction from errors/logs +│ └── tokenStore.js Emby token store (JSON file, atomic writes, 31-day TTL) +├── public/ Static SPA (served by Express) +│ ├── index.html HTML shell: splash, login, dashboard +│ ├── app.js All frontend logic +│ ├── style.css Themes, layout, responsive design +│ ├── favicon.ico / *.png Favicons +│ └── images/ Logo / splash screen assets ├── tests/ -│ ├── setup.js Isolated DATA_DIR, SKIP_RATE_LIMIT -│ ├── unit/ Pure unit tests +│ ├── README.md Testing approach and coverage targets +│ ├── setup.js Global setup: isolated DATA_DIR, rate-limit bypass +│ ├── unit/ Pure unit tests (no HTTP) │ └── integration/ Supertest + nock integration tests -├── docs/ARCHITECTURE.md Full deep-dive architecture documentation -├── ARCHITECTURE.md This file — concise reference -├── SECURITY.md Threat model + hardening guide +├── .gitea/workflows/ +│ ├── ci.yml Security audit + test/coverage on every push/PR +│ ├── build-image.yml Docker image build and push +│ ├── create-release.yml Release tagging workflow +│ ├── docs-check.yml Markdown lint + Mermaid validation +│ └── licence-check.yml Production dependency licence check +├── Dockerfile Multi-stage production image (node:22-alpine) +├── docker-compose.yaml Example compose deployment +├── vitest.config.js Test runner configuration with per-file coverage thresholds +├── package.json Dependencies and scripts +├── ARCHITECTURE.md This document +├── SECURITY.md Threat model and hardening guide ├── CHANGELOG.md Version history -└── .env.sample Annotated configuration template +└── .env.sample Annotated environment variable template ``` --- -*For complete API reference, data-flow diagrams, download matching pipeline, qBittorrent Sync API details, and deployment guidance see [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md).* +## 9. Configuration and Environment Variables + +### 9.1 Core Server + +| Variable | Required | Default | Description | +|----------|:--------:|---------|-------------| +| `PORT` | No | `3001` | Server listen port | +| `NODE_ENV` | No | — | Set to `production` for production logging and startup validation | +| `DATA_DIR` | No | `./data` | Directory for `tokens.json` and `server.log`. Must be writable. In Docker: `/app/data` (named volume). | +| `COOKIE_SECRET` | No* | — | Signs all session cookies with HMAC-SHA256. **Strongly recommended in production** (server exits on startup if unset in `NODE_ENV=production`). Generate with `openssl rand -hex 32`. | +| `TRUST_PROXY` | No | — | Express `trust proxy` setting. Set to `1` when behind a TLS-terminating reverse proxy (nginx, Caddy, Traefik) so `req.ip` and `req.secure` are correct. | +| `LOG_LEVEL` | No | `info` | `debug`, `info`, `warn`, `error`, `silent` | +| `RECENT_COMPLETED_DAYS` | No | `7` | Default lookback window for `/api/history/recent`. Overridable per-request via `?days=`. Capped at 90. | + +### 9.2 TLS / HTTPS + +| Variable | Required | Default | Description | +|----------|:--------:|---------|-------------| +| `TLS_ENABLED` | No | `true` | Set to `false` to run plain HTTP (e.g. when TLS is terminated by a reverse proxy). | +| `TLS_CERT` | No | `certs/snakeoil.crt` | Path to TLS certificate (PEM). Defaults to the bundled self-signed snakeoil certificate. | +| `TLS_KEY` | No | `certs/snakeoil.key` | Path to TLS private key (PEM). | + +### 9.3 Webhook + +| Variable | Required | Default | Description | +|----------|:--------:|---------|-------------| +| `SOFARR_WEBHOOK_SECRET` | Yes* | — | Shared secret validated on the `X-Sofarr-Webhook-Secret` header. Webhook endpoints reject all requests if this is not set. Generate with `openssl rand -hex 32`. | +| `SOFARR_BASE_URL` | Yes* | — | Public base URL of this sofarr instance (e.g. `https://sofarr.example.com`). Used by the one-click webhook configuration endpoints to tell Sonarr/Radarr where to send events. | +| `WEBHOOK_FALLBACK_TIMEOUT` | No | `10` | Minutes of silence after which the poller falls back to full polling even when webhooks were recently active. | + +### 9.4 Polling + +| Variable | Required | Default | Description | +|----------|:--------:|---------|-------------| +| `POLL_INTERVAL` | No | `5000` | Background poll interval in ms. Set to `0`, `off`, or `false` to disable and use on-demand mode. | + +### 9.5 Emby + +| Variable | Required | Default | Description | +|----------|:--------:|---------|-------------| +| `EMBY_URL` | Yes | — | Emby/Jellyfin base URL (e.g. `https://emby.example.com`) | +| `EMBY_API_KEY` | Yes | — | Emby API key — used by the poller to list users for tag badge classification | + +### 9.6 Service Instances + +All service instances support both a JSON array format (recommended) and a legacy single-instance format: + +| Variable | Required | Format | +|----------|:--------:|--------| +| `SONARR_INSTANCES` | Yes* | JSON array | +| `SONARR_URL` + `SONARR_API_KEY` | Yes* | Legacy single-instance | +| `RADARR_INSTANCES` | Yes* | JSON array | +| `RADARR_URL` + `RADARR_API_KEY` | Yes* | Legacy single-instance | +| `SABNZBD_INSTANCES` | Yes* | JSON array | +| `SABNZBD_URL` + `SABNZBD_API_KEY` | Yes* | Legacy single-instance | +| `QBITTORRENT_INSTANCES` | No | JSON array (uses `username`/`password` not `apiKey`) | +| `RTORRENT_INSTANCES` | No | JSON array (URL must include the full XML-RPC path, e.g. `/RPC2`) | + +\* Either `*_INSTANCES` or the legacy pair is required for each service. + +#### JSON array instance format + +```json +[ + { "name": "main", "url": "https://sonarr.example.com", "apiKey": "your-api-key" }, + { "name": "4k", "url": "https://sonarr4k.example.com", "apiKey": "your-4k-api-key" } +] +``` + +qBittorrent and rTorrent instances use `username` and `password` instead of `apiKey`. + +Each instance receives an `id` derived from `name` (or index if unnamed), used as the key in PDCA and PALDRA registries. + +--- + +## 10. Security Model + +### 10.1 Authentication and Sessions + +| Concern | Mechanism | +|---------|-----------| +| **User authentication** | Emby credentials via `POST /Users/authenticatebyname`. A deterministic `DeviceId` (SHA-256 of username, first 16 chars) ensures Emby reuses the same session on every login. | +| **Session cookie** | `httpOnly`, `sameSite: strict`, `secure` when `TRUST_PROXY` is set. Payload: `{ id, name, isAdmin }` only — the Emby `AccessToken` is **never** sent to the browser. Signed with HMAC when `COOKIE_SECRET` is set. | +| **Token store** | Emby `AccessToken`s stored server-side in `DATA_DIR/tokens.json` (atomic writes, 31-day TTL, hourly pruning). Used only for server-side Emby logout. | +| **Session validation** | `requireAuth` middleware on all `/api/dashboard`, `/api/history`, and proxy routes. Returns `401` if the cookie is absent, tampered, or schema-invalid. | +| **CSRF protection** | Double-submit cookie pattern. `verifyCsrf` middleware compares the `csrf_token` cookie against the `X-CSRF-Token` request header using `crypto.timingSafeEqual`. Applied to all state-changing requests (`POST`, `PUT`, `PATCH`, `DELETE`) under `/api/*` except auth and webhook routes. | +| **Remember-me** | `rememberMe: true` → persistent cookie, `Max-Age` 30 days. `rememberMe: false` → session cookie (expires on browser close). | + +### 10.2 Webhook Security + +| Concern | Mechanism | +|---------|-----------| +| **Secret validation** | Every webhook request must carry `X-Sofarr-Webhook-Secret` matching `SOFARR_WEBHOOK_SECRET`. Absent or wrong secret → `401`. Webhook endpoints function outside the CSRF middleware (they are not browser-initiated). | +| **Rate limiting** | Dedicated `webhookLimiter`: 60 req/min per IP (stricter than the general 300 req/15 min limiter). | +| **Payload validation** | `validatePayload()` enforces: JSON object body, `eventType` as a non-empty string ≤ 64 chars, `eventType` in the allowlist, `instanceName` as string if present. Rejects with `400` on any violation. | +| **Replay protection** | `isReplay()` caches a composite key `{eventType}:{instanceName}:{date}` for 5 minutes. Duplicate events within that window are acknowledged with `200 { received: true, duplicate: true }` and not processed. | + +### 10.3 Additional Security Measures + +| Concern | Mechanism | +|---------|-----------| +| **Rate limiting** | 300 req/15 min general (all API routes); 10 failed attempts/15 min login limiter; 60 req/1 min webhook limiter. | +| **Secret leakage** | `sanitizeError()` (`server/utils/sanitizeError.js`) redacts secrets from error messages and logs: URL query-param secrets (`apikey=`, `token=`), HTTP auth headers (`Authorization:`, `X-Emby-Authorization:`), Bearer tokens, and basic-auth credentials in URLs. | +| **HTTP headers** | Helmet v7: CSP with per-request nonce (`crypto.randomBytes(16)` for inline styles/scripts), HSTS, `X-Frame-Options: DENY`, `X-Content-Type-Options: nosniff`, `Referrer-Policy`, `Permissions-Policy`. | +| **Body size** | `express.json` body limit: 64 KB. | +| **Authorisation matrix** | Regular users see only their own downloads. Admins can view all users, see paths and *arr links, and blocklist any download. Non-admins can only blocklist when import issues exist or (for qBittorrent) the torrent is >1 h old with <100% availability. | +| **Container security** | Docker image runs as the non-root `node` user (UID 1000). `/app/data` is owned by `node`. | + +--- + +## 11. Technology Stack + +### Runtime and Framework + +| Layer | Technology | Notes | +|-------|-----------|-------| +| Runtime | Node.js 22 (Alpine) | LTS; ESM-ready; V8 coverage built-in | +| Framework | Express 4.x | HTTP server, routing, middleware | +| HTTP client | axios 1.x | External API communication | +| XML-RPC client | xmlrpc 1.3.2 | rTorrent communication | +| Frontend | Vanilla JS + CSS | SPA, no build step required | +| Containerisation | Docker multi-stage (node:22-alpine) | Non-root `node` user; minimal image | +| Logging | Custom logger + `console.*` redirection | File + stdout with configurable levels | + +### Security Middleware + +| Package | Version | Purpose | +|---------|---------|---------| +| `helmet` | 7.x | HTTP security headers (CSP nonce, HSTS, referrer policy, frame options) | +| `express-rate-limit` | 7.x | General, login, and webhook rate limiters | +| `cookie-parser` | 1.x | Signed cookie support (HMAC via `COOKIE_SECRET`) | + +### Auth and Session + +| Component | Technology | Details | +|-----------|-----------|---------| +| Identity provider | Emby / Jellyfin API | `POST /Users/authenticatebyname` | +| Session cookie | `httpOnly` + `sameSite: strict` | Signed when `COOKIE_SECRET` is set | +| CSRF protection | Double-submit cookie | `csrf_token` cookie + `X-CSRF-Token` header; `crypto.timingSafeEqual` | +| Token store | JSON file (`DATA_DIR/tokens.json`) | Atomic writes, 31-day TTL, hourly pruning | + +### Testing + +| Tool | Version | Purpose | +|------|---------|---------| +| `vitest` | 4.x | Test runner with V8 coverage; per-file coverage thresholds in `vitest.config.js` | +| `supertest` | 7.x | HTTP integration testing against the Express app factory | +| `nock` | 14.x | HTTP interception at Node layer (compatible with CJS `require('axios')`) | + +### CI/CD + +| Workflow file | Trigger | Purpose | +|---------------|---------|---------| +| `ci.yml` | Every push / PR | `npm audit --audit-level=high` + full test suite with V8 coverage | +| `build-image.yml` | Push to `release/**` or `develop` | Build and push Docker image to registry | +| `create-release.yml` | Tag push (`v*`) | Generate release notes and create a Gitea release | +| `docs-check.yml` | Push / PR touching `**.md` | Markdown lint + Mermaid diagram parse validation | +| `licence-check.yml` | Push / PR touching `package.json` | Verify production dependency licences are MIT-compatible | diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md deleted file mode 100644 index ffc6757..0000000 --- a/docs/ARCHITECTURE.md +++ /dev/null @@ -1,1716 +0,0 @@ -# sofarr — Architecture Documentation - -Comprehensive technical documentation covering the full architecture of the **sofarr** application: a personal media download dashboard that aggregates downloads from SABnzbd, Sonarr, Radarr, and qBittorrent, filtered by Emby user identity. - ---- - -## Table of Contents - -1. [System Overview](#1-system-overview) -2. [Technology Stack](#2-technology-stack) -3. [Directory Structure](#3-directory-structure) -4. [Component Architecture](#4-component-architecture) -5. [Data Flow](#5-data-flow) -6. [Authentication & Authorisation](#6-authentication--authorisation) -7. [Background Polling & Caching](#7-background-polling--caching) -8. [Download Matching Pipeline](#8-download-matching-pipeline) -9. [API Reference](#9-api-reference) -10. [Frontend Architecture](#10-frontend-architecture) -11. [Configuration](#11-configuration) -12. [Deployment](#12-deployment) -13. [Diagrams (Mermaid)](#13-diagrams) - ---- - -## 1. System Overview - -sofarr is a **single-page web application** with a Node.js/Express backend. It provides a personalised view of media downloads by: - -1. **Authenticating** users against an Emby/Jellyfin media server. -2. **Aggregating** download data from multiple *arr service instances and download clients. -3. **Filtering** downloads by user — each user only sees media tagged with their username in Sonarr/Radarr. -4. **Presenting** a real-time dashboard with progress, speeds, cover art, and status. - -Admin users can view all users' downloads, see server status, cache statistics, and poll timings. - -### High-Level Architecture - -```mermaid -flowchart TB - subgraph Browser["Browser (SPA)"] - login["Login Form"] - dash["Dashboard Cards"] - status["Status Panel\n(Admin only)"] - end - - subgraph Server["Express Server (:3001)"] - auth_r["Auth Routes\n/api/auth"] - dash_r["Dashboard Routes\n/api/dashboard"] - emby_r["Emby Routes\n/api/emby"] - static_f["Static Files\npublic/"] - - subgraph Utils["Utilities Layer"] - poller["Poller"] - cache["Cache"] - config["Config"] - qbt["qBittorrent"] - end - end - - subgraph Ext["External Services"] - sab["SABnzbd\n(Usenet)"] - sonarr["Sonarr\n(TV)"] - radarr["Radarr\n(Movies)"] - qbittorrent["qBittorrent\n(Torrent)"] - emby["Emby / Jellyfin\n(Auth + User DB)"] - end - - login -->|"POST /login"| auth_r - dash -->|"GET /stream SSE\nGET /user-downloads"| dash_r - status -->|"GET /status"| dash_r - - auth_r -->|"authenticate"| emby - emby_r -->|"proxy"| emby - dash_r --> Utils - poller -->|"HTTP/API calls"| sab & sonarr & radarr - qbt -->|"HTTP/API calls"| qbittorrent - static_f -.->|"serve"| Browser -``` - ---- - -## 2. Technology Stack - -### Runtime & Framework - -| Layer | Technology | Purpose | -|-------|-----------|------| -| **Runtime** | Node.js 22 (Alpine) | Server runtime | -| **Framework** | Express 4.x | HTTP server, routing, middleware | -| **HTTP Client** | axios 1.x | External API communication | -| **Frontend** | Vanilla JS + CSS | Single-page app, no build step | -| **Containerisation** | Docker multi-stage (Alpine) | Production deployment | -| **Logging** | Custom logger + `console.*` | File + stdout logging with levels | - -### Security Middleware - -| Package | Purpose | -|---------|--------| -| `helmet` 7.x | HTTP security headers (CSP with per-request nonce, HSTS, referrer policy) | -| `express-rate-limit` 7.x | 300 req/15 min general limiter; 10 failed attempts/15 min login limiter | -| `cookie-parser` 1.x | Signed cookie support (HMAC via `COOKIE_SECRET`) | - -### Auth & Session - -| Component | Technology | Details | -|-----------|-----------|--------| -| **Identity** | Emby API | `POST /Users/authenticatebyname` | -| **Session cookie** | `httpOnly` + `sameSite: strict` | Signed when `COOKIE_SECRET` set | -| **CSRF protection** | Double-submit cookie pattern | `csrf_token` cookie + `X-CSRF-Token` header | -| **Token store** | JSON file (`DATA_DIR/tokens.json`) | Atomic writes, 31-day TTL, hourly pruning | - -### Testing - -| Tool | Purpose | -|------|---------| -| `vitest` 4.x | Test runner (V8 coverage built-in) | -| `supertest` 7.x | HTTP integration testing | -| `nock` 14.x | HTTP interception at Node layer (works with CJS `require('axios')`) | - ---- - -## 3. Directory Structure - -``` -sofarr/ -├── server/ # Backend application -│ ├── index.js # Entry point: logging setup, server listen, poller start -│ ├── app.js # Express app factory (imported by index.js and tests) -│ ├── clients/ # Download client implementations (PDCA) -│ │ ├── DownloadClient.js # Abstract base class for all download clients -│ │ ├── QBittorrentClient.js # qBittorrent client implementation -│ │ ├── SABnzbdClient.js # SABnzbd client implementation -│ │ └── TransmissionClient.js # Transmission client implementation (proof-of-concept) -│ ├── routes/ -│ │ ├── auth.js # POST /login, GET /me, GET /csrf, POST /logout -│ │ ├── dashboard.js # GET /user-downloads, /user-summary, /status, /cover-art -│ │ ├── emby.js # Proxy routes to Emby API -│ │ ├── sabnzbd.js # Proxy routes to SABnzbd API -│ │ ├── sonarr.js # Proxy routes to Sonarr API -│ │ ├── radarr.js # Proxy routes to Radarr API -│ │ └── history.js # GET /api/history/recent — recently completed downloads -│ ├── middleware/ -│ │ ├── requireAuth.js # httpOnly cookie auth enforcement -│ │ └── verifyCsrf.js # CSRF double-submit cookie validation -│ └── utils/ -│ ├── cache.js # MemoryCache class (Map + TTL + stats) -│ ├── config.js # Multi-instance service configuration parser -│ ├── downloadClients.js # Registry and factory for download clients -│ ├── historyFetcher.js # Fetch + cache Sonarr/Radarr history; event classification -│ ├── logger.js # File logger (DATA_DIR/server.log) -│ ├── poller.js # Background polling engine + timing -│ ├── qbittorrent.js # Legacy compatibility layer (delegates to new system) -│ ├── sanitizeError.js # Redacts secrets from error messages before logging -│ └── tokenStore.js # JSON file-backed Emby token store (atomic, TTL) -├── public/ # Static frontend (served by Express) -│ ├── index.html # HTML shell: splash, login, dashboard -│ ├── app.js # All frontend logic (auth, rendering, status) -│ ├── style.css # Themes, layout, responsive design -│ ├── favicon.ico # Multi-size favicon (16/32/48px) -│ ├── favicon-32.png # 32px PNG favicon -│ ├── favicon-192.png # 192px PNG (apple-touch-icon / PWA) -│ └── images/ # Logo / splash screen assets -├── tests/ -│ ├── README.md # Testing approach, design decisions, coverage targets -│ ├── setup.js # Global setup: isolated DATA_DIR, rate-limit bypass -│ ├── unit/ # Pure unit tests (no HTTP) -│ └── integration/ # Supertest integration tests (nock for external HTTP) -├── docs/ -│ ├── ARCHITECTURE.md # This document -├── .gitea/workflows/ -│ ├── ci.yml # Security audit + test/coverage CI jobs -│ ├── build-image.yml # Docker image build and push -│ └── create-release.yml # Release tagging workflow -├── Dockerfile # Multi-stage production container image (node:22-alpine) -├── docker-compose.yaml # Example compose deployment -├── vitest.config.js # Test runner configuration with per-file coverage thresholds -├── package.json # Dependencies and scripts -├── .env.sample # Annotated environment variable template -└── README.md # User-facing documentation -``` - ---- - -## 4. Component Architecture - -### 4.1 Application Factory (`server/app.js`) and Entry Point (`server/index.js`) - -**`server/app.js`** is a `createApp(options?)` factory that builds and returns the configured Express `app` instance. It is imported by both `server/index.js` (production) and the test suite. Keeping app creation separate from `app.listen()` means tests get a clean instance without triggering log-file setup, `process.exit()` calls, or the background poller. - -`createApp` responsibilities: -- Configure `trust proxy` from `TRUST_PROXY` env var -- Apply `helmet` with a per-request CSP nonce (`crypto.randomBytes(16)`) for inline styles/scripts -- Add `Permissions-Policy` header -- Apply the general API rate limiter (300 req / 15 min per IP) -- Mount `cookie-parser` (signed when `COOKIE_SECRET` is set) -- Mount `express.json` (64 KB body limit) -- Expose `/health` and `/ready` endpoints (no auth, no rate limit) -- Mount `/api/auth` routes **before** CSRF middleware (login/logout are exempt) -- Mount `verifyCsrf` for all subsequent `/api` routes -- Mount remaining route modules under `/api/*` -- Register global error handler (500 with sanitized message) - -**`server/index.js`** entry point responsibilities: -- Load `.env` via `dotenv` -- Configure structured logging (`LOG_LEVEL`) with `console.*` redirection to both stdout and `DATA_DIR/server.log` -- Call `createApp()`, serve `public/` as static files, start `app.listen()` -- Start the background poller - -### 4.2 Route Modules - -| Module | Mount Point | Auth Required | CSRF Required | Purpose | -|--------|------------|:-------------:|:-------------:|--------| -| `auth.js` | `/api/auth` | No | No | Login, session check, CSRF token, logout | -| `dashboard.js` | `/api/dashboard` | Yes (`requireAuth`) | Yes (except `/stream` — GET) | SSE stream, aggregated download data, status, cover-art proxy | -| `emby.js` | `/api/emby` | Yes (`requireAuth`) | Yes | Proxy to Emby API | -| `sabnzbd.js` | `/api/sabnzbd` | Yes (`requireAuth`) | Yes | Proxy to SABnzbd API | -| `sonarr.js` | `/api/sonarr` | Yes (`requireAuth`) | Yes | Proxy to Sonarr API | -| `radarr.js` | `/api/radarr` | Yes (`requireAuth`) | Yes | Proxy to Radarr API | -| `history.js` | `/api/history` | Yes (`requireAuth`) | No (GET only) | Recently completed downloads from Sonarr/Radarr history | - -**`requireAuth`** (`server/middleware/requireAuth.js`) reads the `emby_user` cookie (signed if `COOKIE_SECRET` is set) and attaches the parsed `{ id, name, isAdmin }` user to `req.user`. Returns `401` if the cookie is absent, tampered, or schema-invalid. - -**`verifyCsrf`** (`server/middleware/verifyCsrf.js`) implements the double-submit cookie pattern for all state-changing requests (`POST`, `PUT`, `PATCH`, `DELETE`). Compares the `csrf_token` cookie against the `X-CSRF-Token` request header using `crypto.timingSafeEqual`. Safe methods (`GET`, `HEAD`, `OPTIONS`) are exempt. Auth routes are mounted **before** this middleware (login/logout are exempt by design — `sameSite: strict` provides equivalent protection). - -> **Note:** The proxy routes (`emby`, `sabnzbd`, `sonarr`, `radarr`) exist for direct API access. The dashboard itself reads from the poller cache — it does not proxy through these routes. - -### 4.3 Utility Modules - -**`config.js`** — Parses service instances from environment variables. Supports both JSON array format (`SONARR_INSTANCES=[{...}]`) and legacy single-instance format (`SONARR_URL` + `SONARR_API_KEY`). Each instance gets an `id` derived from `name` or index. - -**`cache.js`** — Singleton `MemoryCache` backed by a `Map`. Each entry has an expiration timestamp. Provides `get`, `set`, `invalidate`, `clear`, and `getStats` (returns per-key size, item count, TTL remaining). - -**`poller.js`** — Background polling engine. Fetches data from all configured service instances in parallel with per-task timing. Stores results in the cache with a configurable TTL. Can be disabled entirely (`POLL_INTERVAL=0`), in which case data is fetched on-demand. After each successful poll it notifies all registered SSE subscriber callbacks so connected clients receive data immediately. - -**`qbittorrent.js`** — Legacy compatibility layer that delegates to the new DownloadClient system. Maintains backward compatibility for existing code while the actual qBittorrent implementation has been moved to `server/clients/QBittorrentClient.js`. - -**`downloadClients.js`** — Registry and factory for download clients. Manages all configured download client instances (SABnzbd, qBittorrent, Transmission) and provides a unified interface for fetching downloads, testing connections, and getting client status. - -**`tokenStore.js`** — JSON file-backed store (`DATA_DIR/tokens.json`) for Emby `AccessToken`s. Tokens are stored server-side and **never sent to the client**. Writes are atomic (write to `.tmp` then rename). Entries expire after 31 days (slightly longer than the maximum 30-day cookie). Pruning runs on startup and hourly. - -**`sanitizeError.js`** — Redacts secrets from error message strings before they are logged or returned in API responses. Patterns: URL query-param secrets (`apikey=`, `token=`, etc.), HTTP auth headers (`Authorization:`, `X-Emby-Authorization:`, etc.), Bearer tokens, and basic-auth credentials in URLs. - -**`historyFetcher.js`** — Fetches history records from all Sonarr/Radarr instances for a configurable date window (`since`). Results are cached under `history:sonarr` / `history:radarr` for 5 minutes. Exports `classifySonarrEvent` / `classifyRadarrEvent` (returns `'imported'` | `'failed'` | `'other'`) and `invalidateHistoryCache`. - -**`logger.js`** — Simple file appender writing timestamped messages to `DATA_DIR/server.log`. - ---- - -## 4.4 Download Client Architecture (PDCA) - -### 4.4.1 Overview - -The **Pluggable Download Client Architecture (PDCA)** provides a unified, extensible interface for all download clients (SABnzbd, qBittorrent, Transmission, etc.). This abstraction layer enables: - -- **Client-agnostic polling**: The poller no longer needs client-specific logic -- **Easy addition of new clients**: Implement the `DownloadClient` interface -- **Consistent data normalization**: All clients return standardized download objects -- **Centralized configuration**: Single registry manages all client instances - -### 4.4.2 Abstract Base Class (`DownloadClient.js`) - -The `DownloadClient` abstract base class defines the contract that all download clients must implement: - -```javascript -class DownloadClient { - constructor(instanceConfig) - getClientType(): string - getInstanceId(): string - async testConnection(): Promise - async getActiveDownloads(): Promise - async getClientStatus(): Promise // Optional - normalizeDownload(download): NormalizedDownload -} -``` - -**Key Features:** -- Enforces implementation of required methods -- Provides common initialization logic -- Defines the normalized download schema - -### 4.4.3 Normalized Download Schema - -All clients must return objects matching this standardized schema: - -```javascript -interface NormalizedDownload { - id: string // Client-specific unique ID - title: string // Download title/name - type: 'usenet' | 'torrent' // Download type - client: string // Client identifier ('sabnzbd', 'qbittorrent', etc.) - instanceId: string // Instance identifier - instanceName: string // Instance display name - status: string // Normalized status (Downloading, Seeding, etc.) - progress: number // Progress percentage (0-100) - size: number // Total size in bytes - downloaded: number // Downloaded bytes - speed: number // Current speed in bytes/sec - eta: number | null // ETA in seconds, null if unknown - category?: string // Download category (optional) - tags?: string[] // Download tags (optional) - savePath?: string // Save path (optional) - addedOn?: string // Added timestamp (optional) - arrQueueId?: number // Sonarr/Radarr queue ID (optional) - arrType?: 'series' | 'movie' // Sonarr/Radarr type (optional) - raw?: any // Original client response (escape hatch) -} -``` - -### 4.4.4 Client Implementations - -#### QBittorrentClient -- Extends the existing qBittorrent implementation with Sync API support -- Maintains backward compatibility with legacy cache format -- Handles cookie authentication and automatic re-auth -- Preserves fallback logic for Sync API failures - -#### SABnzbdClient -- Extracts SABnzbd logic from the poller into a dedicated client -- Handles both queue and history data -- Normalizes time strings and size units -- Extracts Sonarr/Radarr information from filenames - -#### TransmissionClient -- Proof-of-concept implementation for Transmission daemon -- Uses JSON-RPC over HTTP -- Handles session ID management and conflict resolution -- Demonstrates how easy it is to add new client types - -#### RTorrentClient -- XML-RPC implementation for rTorrent daemon -- Uses the xmlrpc package (v1.3.2) for communication -- Supports HTTP Basic Auth when credentials are configured -- Maps rTorrent states (d.state, d.is_active, d.is_hash_checking) to normalized statuses -- Calculates ETA from download speed and remaining bytes - -### 4.4.5 Registry and Factory (`downloadClients.js`) - -The `DownloadClientRegistry` manages all client instances: - -```javascript -class DownloadClientRegistry { - async initialize() // Create clients from config - getAllClients(): DownloadClient[] // Get all registered clients - getClient(instanceId): DownloadClient // Get specific client - getClientsByType(type): DownloadClient[] // Get clients by type - async getAllDownloads(): NormalizedDownload[] // Fetch from all clients - async testAllConnections(): Promise - async getAllClientStatuses(): Promise -} -``` - -**Features:** -- **Configuration-driven**: Reads from `*_INSTANCES` environment variables -- **Parallel execution**: Fetches from all clients concurrently -- **Error isolation**: Individual client failures don't affect others -- **Singleton pattern**: Single registry instance shared across the application - -### 4.4.6 Integration with Poller - -The poller has been refactored to use the registry: - -```javascript -// Old approach (client-specific) -const sabQueues = await Promise.all(sabInstances.map(inst => - axios.get(`${inst.url}/api`, { params: { mode: 'queue' } }) -)); -const qbTorrents = await getTorrents(); - -// New approach (unified) -await initializeClients(); -const downloadsByType = await getDownloadsByClientType(); -``` - -**Benefits:** -- **30-40% reduction in poller code size** -- **Consistent error handling** across all clients -- **Unified timing and logging** -- **Zero breaking changes** to existing cache structure - -### 4.4.7 Backward Compatibility - -The PDCA implementation maintains **100% backward compatibility**: - -- **Cache keys**: `poll:sab-queue`, `poll:sab-history`, `poll:qbittorrent` unchanged -- **Data shapes**: Legacy formats preserved through transformation -- **API responses**: No changes to existing endpoints -- **Legacy functions**: `qbittorrent.js` delegates to new system - ---- - -## 5. Data Flow - -### 5.1 Polling Cycle - -Every `POLL_INTERVAL` ms (default 5000), the poller fetches from all services in parallel: - -| Task | API Call | Params | -|------|----------|--------| -| SABnzbd Queue | `GET /api?mode=queue` | `output=json` | -| SABnzbd History | `GET /api?mode=history` | `limit=10` | -| Sonarr Tags | `GET /api/v3/tag` | — | -| Sonarr Queue | `GET /api/v3/queue` | `includeSeries=true`, `includeEpisode=true` | -| Sonarr History | `GET /api/v3/history` | `pageSize=10`, `includeEpisode=true` | -| Radarr Queue | `GET /api/v3/queue` | `includeMovie=true` | -| Radarr History | `GET /api/v3/history` | `pageSize=10` | -| Radarr Tags | `GET /api/v3/tag` | — | -| qBittorrent | `GET /api/v2/sync/maindata?rid=N` | Fallback to `GET /api/v2/torrents/info` | - -Results are stored in the cache under `poll:*` keys with a TTL of `POLL_INTERVAL × 3`. - -#### qBittorrent Sync API Details - -Each `QBittorrentClient` instance maintains: -- **`lastRid`** — the response ID from the previous `sync/maindata` call (starts at `0`). -- **`torrentMap`** — a `Map` holding the complete state for every known torrent on this qBittorrent instance. -- **`fallbackThisCycle`** — boolean tracking whether this poll cycle has already fallen back to the legacy endpoint. - -**Flow per poll cycle:** - -1. `getAllTorrents()` resets `fallbackThisCycle = false` on every client. -2. `client.getTorrents()` attempts `GET /api/v2/sync/maindata?rid={lastRid}`. -3. qBittorrent returns: - - `rid` — new response ID to use next time. - - `full_update` — if `true`, `torrents` contains the complete current list (rebuild `torrentMap`). - - `torrents` — object keyed by hash; values are either full objects (first call / `full_update`) or delta objects (only changed fields). - - `torrents_removed` — array of hashes to delete from `torrentMap`. -4. The client merges delta fields into existing entries, removes deleted entries, and returns the current values of `torrentMap` as an array. -5. If the Sync API call fails (network error, 500, unexpected response shape), the client falls back **once per cycle** to `GET /api/v2/torrents/info`. -6. If the fallback also fails, the client returns an empty array for this poll and logs the error. - -**Backward compatibility:** The rest of the application (poller, dashboard) receives data in the exact same format as before; no routes or frontend code are aware of the sync mechanism. - -### 5.2 SSE Stream - -When a browser opens `GET /api/dashboard/stream` (after authentication): - -1. Server sets `Content-Type: text/event-stream`, disables buffering (`X-Accel-Buffering: no`) -2. Immediately builds and sends the first payload (same matching logic as below) -3. Registers a callback with the poller's `onPollComplete` subscriber set -4. After every subsequent poll cycle completes the callback fires, rebuilds the payload, and writes a `data:` SSE frame -5. A 25-second heartbeat comment (`: heartbeat`) keeps the connection alive through proxies -6. On client disconnect: deregisters callback, stops heartbeat, removes from active-client map - -The browser's native `EventSource` API handles reconnection automatically on network interruption. - -### 5.3 Download Matching - -For each connected user the server: - -1. Reads all `poll:*` keys from cache -2. Builds `seriesMap` and `moviesMap` from embedded objects in queue records -3. Builds `sonarrTagMap` and `radarrTagMap` from tag data -4. For each SABnzbd queue slot → tries to match against Sonarr/Radarr queue records by title -5. For each SABnzbd history slot → tries to match against Sonarr/Radarr history records -6. For each qBittorrent torrent → tries to match against Sonarr/Radarr queue, then history -7. For each match, resolves the series/movie, extracts the user tag, checks if it belongs to the requesting user -8. Returns only the user's downloads (or all, if admin with `showAll=true`) - ---- - -## 6. Authentication & Authorisation - -### Flow - -1. User submits credentials (+ optional `rememberMe`) via the login form -2. Backend applies the **login rate limiter** (max 10 failed attempts per IP per 15-minute window; successful requests don't count) -3. Backend calls Emby `POST /Users/authenticatebyname` using a deterministic `DeviceId` derived from the username — this causes Emby to reuse the existing session rather than creating a new one on every login -4. On success, fetches full user profile (`GET /Users/{id}`) to determine admin status -5. Stores the Emby `AccessToken` in **`tokenStore`** (server-side, never sent to the client) -6. Sets an `httpOnly` `emby_user` cookie containing only `{ id, name, isAdmin }`: - - **`rememberMe: true`** → persistent cookie, `Max-Age` 30 days - - **`rememberMe: false`** → session cookie (no `Max-Age`; expires when browser closes) - - `secure` flag enabled when `TRUST_PROXY` is set (i.e. a TLS-terminating reverse proxy is in front) - - Signed with HMAC when `COOKIE_SECRET` is set -7. Issues a `csrf_token` cookie (`httpOnly: false` so JS can read it) containing a 32-byte random hex token -8. Returns `{ success: true, user, csrfToken }` — the SPA stores `csrfToken` in memory and sends it as `X-CSRF-Token` on all subsequent state-changing requests -9. All subsequent dashboard requests read the `emby_user` cookie for identity via `requireAuth` - -### Authorisation Matrix - -| Feature | Regular User | Admin | -|---------|:----------:|:-----:| -| View own downloads | ✓ | ✓ | -| View all users' downloads | ✗ | ✓ (`showAll`) | -| See download/target paths | ✗ | ✓ | -| See Sonarr/Radarr links | ✗ | ✓ | -| View status panel | ✗ | ✓ | -| Blocklist & search | ✓ (when import issues OR torrent >1h old AND availability<100%) | ✓ (all downloads) | - -### Tag Matching - -Users are matched to downloads via tags in Sonarr/Radarr: - -1. **Exact match**: tag label (lowercased) === username (lowercased) -2. **Sanitised match**: handles Ombi's tag mangling — `sanitizeTagLabel()` converts to lowercase, replaces non-alphanumeric with hyphens, collapses, trims - ---- - -## 7. Background Polling & Caching - -### Polling Modes - -| Mode | `POLL_INTERVAL` | Behaviour | -|------|----------------|-----------| -| **Background** | `> 0` (e.g. `5000`) | Periodic fetch every N ms | -| **On-demand** | `0` / `off` / `false` | Fetch triggered by first user request when cache is empty | - -### Cache Keys - -| Key | Content | Source | -|-----|---------|--------| -| `poll:sab-queue` | `{ slots, status, speed, kbpersec }` | SABnzbd queue | -| `poll:sab-history` | `{ slots }` | SABnzbd history | -| `poll:sonarr-tags` | `[{ instance, data: [{id, label}] }]` | Sonarr tag API | -| `poll:sonarr-queue` | `{ records }` — includes embedded `series` objects | Sonarr queue (includeSeries) | -| `poll:sonarr-history` | `{ records }` — lightweight, no embedded objects | Sonarr history | -| `poll:radarr-queue` | `{ records }` — includes embedded `movie` objects | Radarr queue (includeMovie) | -| `poll:radarr-history` | `{ records }` — lightweight, no embedded objects | Radarr history | -| `poll:radarr-tags` | `[{id, label}]` | Radarr tag API | -| `poll:qbittorrent` | `[torrent, ...]` | qBittorrent all torrents | -| `emby:users` | `Map` | Full Emby user list (60s TTL) | -| `history:sonarr` | `[record, ...]` — flat array with `_instanceUrl` / `_instanceName` | Sonarr history (5 min TTL, fetched on-demand by `/api/history/recent`) | -| `history:radarr` | `[record, ...]` — flat array with `_instanceUrl` / `_instanceName` | Radarr history (5 min TTL, fetched on-demand by `/api/history/recent`) | - -### TTL Strategy - -- **Polling enabled**: `POLL_INTERVAL × 3` — ensures data survives between polls even if one poll is slow -- **Polling disabled**: `30000` ms — stale after 30s, next request triggers a fresh fetch - -### Active Client Tracking - -SSE connections are tracked precisely: registered on connect, removed on disconnect. The admin status panel shows each connected user and how long they have been connected. The `type: 'sse'` field distinguishes SSE clients from any legacy HTTP clients. - ---- - -## 8. Download Matching Pipeline - -The core logic in `dashboard.js` matches raw download client data to *arr service metadata to determine which user each download belongs to. - -### Matching Strategy - -For each download item (SABnzbd slot or qBittorrent torrent): - -```mermaid -flowchart TD - Start(["Download item"]) --> SQ{"Sonarr QUEUE\nmatch (title)"} - SQ -->|yes| SQR["Resolve series via seriesMap\nextract user tag, check match"] - SQ -->|no| RQ{"Radarr QUEUE\nmatch (title)"} - RQ -->|yes| RQR["Resolve movie via moviesMap\nextract user tag, check match"] - RQ -->|no| SH{"Sonarr HISTORY\nmatch (title)"} - SH -->|yes| SHR["Resolve series via seriesId\nextract user tag, check match"] - SH -->|no| RH{"Radarr HISTORY\nmatch (title)"} - RH -->|yes| RHR["Resolve movie via movieId\nextract user tag, check match"] - RH -->|no| Skip(["Skip - unmatched"]) - - SQR & RQR & SHR & RHR --> Tagged{"Tag matches\nrequesting user?"} - Tagged -->|yes| Include(["Include in response"]) - Tagged -->|no| Skip -``` - -### Title Matching - -Matches are **bidirectional substring matches** (case-insensitive): -```javascript -rTitle.includes(downloadTitle) || downloadTitle.includes(rTitle) -``` - -### Download Object Structure - -Each matched download produces an object with: - -| Field | Type | Description | -|-------|------|-------------| -| `type` | `'series'` / `'movie'` / `'torrent'` | Media type | -| `title` | string | Raw download title | -| `coverArt` | string / null | Poster URL from *arr | -| `status` | string | Download status | -| `progress` | string | Percentage complete | -| `size` / `mb` / `mbmissing` | string / number | Size info | -| `speed` | string | Current download speed | -| `eta` | string | Estimated time remaining | -| `seriesName` / `movieName` | string | Friendly media title | -| `episodes` | `{season, episode, title}[]` | (Series only) Episodes covered by this download, sorted by season/episode. Single-episode downloads have one entry; series packs have multiple. Empty array if Sonarr has no episode data. | -| `allTags` | string[] | All resolved tag labels on the series/movie | -| `matchedUserTag` | string / null | Tag label matching the requesting user, or `null` | -| `tagBadges` | `{label, matchedUser}[]` / undefined | (Admin `showAll` only) Each tag classified against full Emby user list | -| `importIssues` | string[] / null | Import warning/error messages | -| `availableForUpgrade` | boolean / undefined | (History) `true` when outcome is `failed` but the content is already on disk (failed upgrade attempt) | -| `canBlocklist` | boolean | `true` if the current user can blocklist this download (admin: always; non-admin: when import issues OR torrent >1h old AND availability<100%) | -| `downloadPath` | string / null | (Admin) Download client path | -| `targetPath` | string / null | (Admin) *arr target path | -| `arrLink` | string / null | (Admin) Link to *arr web UI | -| `arrQueueId` | number / null | (Admin, import-pending only) Sonarr/Radarr queue record id | -| `arrType` | `'sonarr'`/`'radarr'` / null | (Admin, import-pending only) Which *arr service owns this queue entry | -| `arrInstanceUrl` | string / null | (Admin, import-pending only) Base URL of the *arr instance | -| `arrInstanceKey` | string / null | (Admin, import-pending only) API key for the *arr instance | -| `arrContentId` | number / null | (Admin, import-pending only) `episodeId` (Sonarr) or `movieId` (Radarr) for triggering a new search | -| `arrContentType` | `'episode'`/`'movie'` / null | (Admin, import-pending only) Content type for the search command | -| `addedOn` | number / null | (qBittorrent only) Unix timestamp when the torrent was added, used for age-based blocklist eligibility | - ---- - -## 9. API Reference - -### `POST /api/auth/login` - -Authenticate a user via Emby. Rate-limited to **10 failed attempts per IP per 15 minutes**. - -**Request Body:** -```json -{ "username": "string", "password": "string", "rememberMe": false } -``` - -| Field | Required | Description | -|-------|:--------:|-----------| -| `username` | Yes | Max 128 chars, must be a non-empty string | -| `password` | Yes | Max 256 chars | -| `rememberMe` | No | `true` = 30-day persistent cookie; `false`/omitted = session cookie | - -**Response (200):** -```json -{ - "success": true, - "user": { "id": "string", "name": "string", "isAdmin": false }, - "csrfToken": "64-char hex string" -} -``` - -**Response (400):** Invalid input (empty/overlong username or password). - -**Response (401):** -```json -{ "success": false, "error": "Invalid username or password" } -``` - -**Response (429):** Too many failed attempts from this IP. - -**Side Effects:** -- Sets `emby_user` cookie: `httpOnly`, `sameSite: strict`, `secure` in production, signed if `COOKIE_SECRET` is set. Payload: `{ id, name, isAdmin }`. The Emby `AccessToken` is **never** included. -- Sets `csrf_token` cookie: `httpOnly: false` (JS-readable), same security flags. 64-char hex. -- Stores the Emby `AccessToken` server-side in `tokenStore` (used only for server-side Emby logout). - ---- - -### `GET /api/auth/me` - -Check current session (no auth required — returns unauthenticated state rather than 401). - -**Response (authenticated):** -```json -{ "authenticated": true, "user": { "id": "string", "name": "string", "isAdmin": false } } -``` - -**Response (not authenticated):** -```json -{ "authenticated": false } -``` - ---- - -### `GET /api/auth/csrf` - -Issue a fresh CSRF token. Used by the SPA after a page reload when the in-memory `csrfToken` variable is lost. - -**Response (200):** -```json -{ "csrfToken": "64-char hex string" } -``` - -**Side Effect:** Sets a new `csrf_token` cookie. - ---- - -### `POST /api/auth/logout` - -Clear session and revoke the Emby token server-side. Does **not** require a CSRF token (auth routes are mounted before `verifyCsrf`; `sameSite: strict` provides equivalent protection). - ---- - -### `GET /api/dashboard/stream` - -Server-Sent Events stream. Opens a persistent connection that pushes download data on every poll cycle. - -**Query Parameters:** -| Param | Type | Description | -|-------|------|-------------| -| `showAll` | `"true"` | (Admin) Include all users' downloads | - -**Response:** `Content-Type: text/event-stream` - -Each event is a `data:` frame containing JSON: -```json -{ - "user": "Alice", - "isAdmin": false, - "downloads": [ /* download objects — same shape as /user-downloads */ ] -} -``` - -The connection is kept alive with `: heartbeat` comments every 25 seconds. The browser's `EventSource` reconnects automatically on failure. - ---- - -### `GET /api/dashboard/user-downloads` - -Fetch downloads for the authenticated user (single HTTP request, no streaming). - -**Query Parameters:** -| Param | Type | Description | -|-------|------|-------------| -| `showAll` | `"true"` | (Admin) Show all users' downloads | - -**Response (200):** -```json -{ - "user": "string", - "isAdmin": true, - "downloads": [ /* download objects */ ] -} -``` - ---- - -### `GET /api/dashboard/status` - -Admin-only server status. - -**Response (200):** -```json -{ - "server": { - "uptimeSeconds": 3600, - "nodeVersion": "v18.19.0", - "memoryUsageMB": 45.2, - "heapUsedMB": 28.1, - "heapTotalMB": 35.0 - }, - "polling": { - "enabled": true, - "intervalMs": 5000, - "lastPoll": { - "totalMs": 1234, - "timestamp": "2026-05-16T00:00:00.000Z", - "tasks": [ - { "label": "SABnzbd Queue", "ms": 120 }, - { "label": "Sonarr Queue", "ms": 890 } - ] - } - }, - "cache": { - "entryCount": 9, - "totalSizeBytes": 51200, - "entries": [ - { "key": "poll:sab-queue", "sizeBytes": 1024, "itemCount": 3, "ttlRemainingMs": 12000, "expired": false } - ] - }, - "clients": [ - { "user": "Alice", "type": "sse", "connectedAt": 1715817600000, "lastSeen": 1715817600000 } - ] -} -``` - ---- - -### `GET /api/dashboard/user-summary` - -Admin-only per-user download counts (fetches live from APIs, not cached). - -**Response (200):** -```json -[ - { "username": "Alice", "seriesCount": 12, "movieCount": 5 } -] -``` - ---- - -### `POST /api/dashboard/blocklist-search` - -Removes a Sonarr/Radarr queue item with `blocklist=true` (preventing the same release being grabbed again), then immediately triggers an `EpisodeSearch` or `MoviesSearch` command. - -**Access:** Admin users can blocklist any download. Non-admin users can only blocklist downloads that meet specific eligibility criteria: import issues are present, OR (for qBittorrent torrents only) the download is more than 1 hour old AND has less than 100% availability. The frontend only shows the button when the user is eligible. - -Requires CSRF token (`X-CSRF-Token` header). - -**Request Body:** -```json -{ - "arrQueueId": 1234, - "arrType": "sonarr", - "arrInstanceUrl": "https://sonarr.example.com", - "arrInstanceKey": "your-api-key", - "arrContentId": 5678, - "arrContentType": "episode" -} -``` - -| Field | Required | Description | -|-------|:--------:|-------------| -| `arrQueueId` | Yes | Sonarr/Radarr queue record `id` | -| `arrType` | Yes | `"sonarr"` or `"radarr"` | -| `arrInstanceUrl` | Yes | Base URL of the *arr instance | -| `arrInstanceKey` | Yes | API key for the *arr instance | -| `arrContentId` | Yes | `episodeId` (Sonarr) or `movieId` (Radarr) | -| `arrContentType` | Yes | `"episode"` or `"movie"` | - -**Response (200):** `{ "ok": true }` - -**Response (400):** Missing or invalid fields. - -**Response (403):** Non-admin user attempting to blocklist without meeting eligibility criteria (no import issues and not an eligible torrent). - -**Response (502):** Upstream *arr call failed. - -**Side Effects:** -- Calls `DELETE /api/v3/queue/{id}?removeFromClient=true&blocklist=true` on the *arr instance -- Calls `POST /api/v3/command` with `EpisodeSearch`/`MoviesSearch` on the *arr instance -- Triggers a background `pollAllServices()` so the next SSE push reflects the removed item - ---- - -### `GET /api/history/recent` - -Returns recently completed (imported or failed) downloads from Sonarr/Radarr history for the authenticated user, filtered to the last `days` days. - -**Query Parameters:** -| Param | Type | Default | Description | -|-------|------|---------|-------------| -| `days` | integer | `RECENT_COMPLETED_DAYS` env (default `7`) | How many days back to search. Capped at 90. | -| `showAll` | `"true"` | — | (Admin) Return records for all tagged users, not just the current user | - -**Response (200):** -```json -{ - "user": "Alice", - "isAdmin": false, - "days": 7, - "history": [ - { - "type": "series", - "outcome": "imported", - "title": "Show.S01E01.720p", - "seriesName": "My Show", - "episodes": [ - { "season": 1, "episode": 1, "title": "Pilot" } - ], - "coverArt": "https://…/poster.jpg", - "completedAt": "2026-05-15T18:00:00.000Z", - "quality": "720p", - "instanceName": "Main Sonarr", - "arrLink": "https://sonarr.example.com/series/my-show", - "allTags": ["alice"], - "matchedUserTag": "alice", - "arrRecordId": 1234, - "failureMessage": null - } - ] -} -``` - -- `outcome` is `"imported"` or `"failed"`. Records with other event types (e.g. `grabbed`) are filtered out. -- `episodes` is a sorted array of `{ season, episode, title }` objects. Single-episode downloads have one entry; series packs have multiple. `title` is `null` if not returned by Sonarr. Empty array if Sonarr has no episode data. -- `failureMessage` is only included when the authenticated user is an admin and `outcome` is `"failed"`. -- `arrRecordId` is only included for admin users. -- Results are sorted newest first. -- History data is cached server-side for 5 minutes (`history:sonarr` / `history:radarr` cache keys). - ---- - -## 10. Frontend Architecture - -The frontend is a **vanilla JavaScript SPA** with no build step. All logic resides in `app.js`, styled by `style.css`, and structured by `index.html`. - -### UI States - -```mermaid -stateDiagram-v2 - [*] --> SplashScreen : Page load - SplashScreen --> LoginForm : No session - SplashScreen --> Dashboard : Valid session - LoginForm --> Dashboard : Auth success - Dashboard --> LoginForm : Logout - - state Dashboard { - [*] --> ActiveDownloads - ActiveDownloads --> ActiveDownloads : SSE update - - state StatusPanel { - [*] --> Closed - Closed --> Open : Click Status (admin) - Open --> Closed : Click close - Open --> Open : 5s refresh - } - } -``` - -### Key Frontend Functions - -| Function | Purpose | -|----------|---------| -| `checkAuthentication()` | On load: check session → show dashboard or login | -| `handleLogin()` | Authenticate, fade login → splash → dashboard | -| `goHome()` | Navigate to default view: switch to Active Downloads tab, close status panel, reset showAll | -| `startSSE()` | Open `EventSource` to `/stream`; handles incoming data + first-message loading hide | -| `stopSSE()` | Close `EventSource` and cancel reconnect timer | -| `renderDownloads()` | Diff-based card rendering (create/update/remove) | -| `createDownloadCard()` | Build DOM for a single download card; renders tag badges, import-issue badge, blocklist button | -| `updateDownloadCard()` | Update existing card in-place (progress, speed, etc.) | -| `handleBlocklistSearch()` | Confirm dialog → POST `/blocklist-search` → update button state | -| `toggleStatusPanel()` | Show/hide admin status panel | -| `renderStatusPanel()` | Build status HTML (server, polling, SSE clients, cache) | -| `initThemeSwitcher()` | Light / Dark / Mono theme support | -| `loadHistory()` | Fetch `/api/history/recent`, store raw items, call `renderHistory()` | -| `renderHistory()` | Filter items by `ignoreAvailable` flag, render history cards | -| `createHistoryCard()` | Build DOM for a single history card with outcome/upgrade badges | - -### Themes - -Three CSS themes via `data-theme` attribute on ``: -- **Light** — Purple gradient header, white cards -- **Dark** — Dark surfaces, muted accents -- **Mono** — Monochrome, minimal colour - -Theme selection persists in `localStorage`. - -### Tag Badge Rendering - -Download cards render tag badges in the card header: - -- **Normal user view**: a single accent-coloured badge showing the tag label that matched the current user's username (via `matchedUserTag`). -- **Admin `showAll` view**: all tags on the download are rendered using `tagBadges[]`: - - Tags with **no matching Emby user** → amber badge showing the raw tag label (leftmost) - - Tags **matched to a known Emby user** → accent badge showing the Emby display name (rightmost) - -### Live Push via SSE - -The dashboard receives updates via a persistent `EventSource` connection to `GET /api/dashboard/stream`. The server pushes a new `data:` event immediately after every poll cycle completes — there is no client-side timer. The browser's `EventSource` implementation handles reconnection automatically on network interruption. - -The status panel refreshes on a fixed 5-second timer and shows each SSE client with its connect duration. - ---- - -## 11. Configuration - -### Environment Variables - -#### Core - -| Variable | Required | Default | Description | -|----------|:--------:|---------|-------------| -| `PORT` | No | `3001` | Server listen port | -| `NODE_ENV` | No | — | Set to `production` for production logging and startup validation. Does **not** control cookie `secure` flag or CSP `upgrade-insecure-requests` (both gated on `TRUST_PROXY`). | -| `DATA_DIR` | No | `./data` | Directory for `tokens.json` and `server.log`. Must be writable by the server process. In Docker: `/app/data` (named volume). | -| `COOKIE_SECRET` | No | — | If set, signs all session cookies with HMAC-SHA256. Strongly recommended in production. | -| `TRUST_PROXY` | No | — | Express `trust proxy` setting. Set to `1` (or a hop count) when behind a reverse proxy (nginx, Traefik, etc.) so `req.ip` and `req.secure` are correct. | -| `TLS_ENABLED` | No | `true` | Set to `false` to disable HTTPS and run plain HTTP (e.g. when TLS is terminated by a reverse proxy). | -| `TLS_CERT` | No | `certs/snakeoil.crt` | Path to the TLS certificate file (PEM). Defaults to the bundled self-signed snakeoil certificate. | -| `TLS_KEY` | No | `certs/snakeoil.key` | Path to the TLS private key file (PEM). Defaults to the bundled snakeoil key. | - -#### Emby - -| Variable | Required | Default | Description | -|----------|:--------:|---------|-------------| -| `EMBY_URL` | Yes | — | Emby/Jellyfin base URL (e.g. `https://emby.example.com`) | -| `EMBY_API_KEY` | Yes | — | Emby API key (used by the poller to list users for tag badge classification) | - -#### Service Instances - -| Variable | Required | Default | Description | -|----------|:--------:|---------|-------------| -| `SONARR_INSTANCES` | Yes* | — | JSON array of Sonarr instances | -| `SONARR_URL` | Yes* | — | Legacy: single Sonarr URL | -| `SONARR_API_KEY` | Yes* | — | Legacy: single Sonarr API key | -| `RADARR_INSTANCES` | Yes* | — | JSON array of Radarr instances | -| `RADARR_URL` | Yes* | — | Legacy: single Radarr URL | -| `RADARR_API_KEY` | Yes* | — | Legacy: single Radarr API key | -| `SABNZBD_INSTANCES` | Yes* | — | JSON array of SABnzbd instances | -| `SABNZBD_URL` | Yes* | — | Legacy: single SABnzbd URL | -| `SABNZBD_API_KEY` | Yes* | — | Legacy: single SABnzbd API key | -| `QBITTORRENT_INSTANCES` | No | — | JSON array of qBittorrent instances | - -\* Either `*_INSTANCES` (JSON array) or legacy `*_URL` + `*_API_KEY` format is required. - -#### Tuning - -| Variable | Required | Default | Description | -|----------|:--------:|---------|-------------| -| `POLL_INTERVAL` | No | `5000` | Poll interval in ms. Set to `0`, `off`, or `false` to disable background polling (on-demand mode). | -| `RECENT_COMPLETED_DAYS` | No | `7` | Default lookback window (days) for `GET /api/history/recent`. Overridable per-request via `?days=`. Max 90. | -| `LOG_LEVEL` | No | `info` | `debug`, `info`, `warn`, `error`, `silent` | - -### Instance JSON Format - -```json -[ - { - "name": "main", - "url": "https://sonarr.example.com", - "apiKey": "your-api-key" - }, - { - "name": "4k", - "url": "https://sonarr4k.example.com", - "apiKey": "your-4k-api-key" - } -] -``` - -qBittorrent instances use `username` and `password` instead of `apiKey`. - ---- - -## 12. Deployment - -### Docker image - -The production image uses a two-stage build on `node:22-alpine`: - -1. **`deps` stage** — runs `npm ci --omit=dev` to install only production dependencies. -2. **`runtime` stage** — copies `node_modules`, `server/`, `public/`, and `package.json`. Runs as the built-in non-root `node` user (UID 1000). `/app/data` is owned by `node` for writable token store and logs. - -Key environment variables set in the image: -- `NODE_ENV=production` — enables production startup validation and logging -- `DATA_DIR=/app/data` — token store and log file location -- TLS is **enabled by default** using the bundled snakeoil self-signed certificate (`certs/snakeoil.crt`). Set `TLS_CERT`/`TLS_KEY` to your own certificate, or set `TLS_ENABLED=false` when terminating TLS at a reverse proxy. - -### Docker Compose - -```yaml -services: - sofarr: - image: docker.i3omb.com/sofarr:latest - container_name: sofarr - restart: unless-stopped - ports: - - "3001:3001" # HTTPS by default (snakeoil cert if no TLS_CERT set) - environment: - - NODE_ENV=production - - DATA_DIR=/app/data - - COOKIE_SECRET=change-me-to-a-long-random-string - # Option A: direct TLS (default). Supply your own cert/key: - # - TLS_CERT=/app/certs/server.crt - # - TLS_KEY=/app/certs/server.key - # Option B: behind a TLS-terminating reverse proxy: - # - TLS_ENABLED=false - # - TRUST_PROXY=1 - - EMBY_URL=https://emby.example.com - - EMBY_API_KEY=your-emby-api-key - - SONARR_INSTANCES=[{"name":"main","url":"...","apiKey":"..."}] - - RADARR_INSTANCES=[{"name":"main","url":"...","apiKey":"..."}] - - SABNZBD_INSTANCES=[{"name":"main","url":"...","apiKey":"..."}] - - QBITTORRENT_INSTANCES=[{"name":"main","url":"...","username":"...","password":"..."}] - - POLL_INTERVAL=5000 - - LOG_LEVEL=info - volumes: - - sofarr-data:/app/data - # Uncomment to supply your own certificate (Option A): - # - /path/to/server.crt:/app/certs/server.crt:ro - # - /path/to/server.key:/app/certs/server.key:ro - -volumes: - sofarr-data: -``` - -### Security hardening checklist - -- **Use HTTPS** — TLS is on by default (snakeoil cert). Supply `TLS_CERT`/`TLS_KEY` pointing to a CA-signed certificate for trusted HTTPS. Alternatively terminate TLS at a reverse proxy and set `TLS_ENABLED=false` + `TRUST_PROXY=1`. -- **Set `COOKIE_SECRET`** — enables HMAC-signed cookies, preventing client-side forgery. -- **Set `TRUST_PROXY=1`** only when a TLS-terminating reverse proxy sits in front — ensures `req.secure` is correct and the CSP `upgrade-insecure-requests` + `secure` cookie flag fire correctly. -- **Mount a named volume** for `DATA_DIR` — token store and log file survive container recreates. -- **Rate limiting** — the login endpoint is limited to 10 failed attempts per IP per 15 minutes; all API endpoints share a 300 req/15 min window. Set `TRUST_PROXY` correctly so the client IP is not the proxy's IP. - -### CI / CD - -The `.gitea/workflows/` directory contains five pipeline definitions: - -| File | Trigger | Purpose | -|------|---------|--------| -| `ci.yml` | Every push / PR (all branches) | Security audit (`npm audit --audit-level=high`) + tests with V8 coverage | -| `build-image.yml` | Push to `release/**` or `develop` | Build and push Docker image to `reg.i3omb.com`. `release/**` pushes versioned + `latest` tags; `develop` pushes a `:develop` tag. | -| `create-release.yml` | Tag push (`v*`) | Generate release notes from git log and create a Gitea release | -| `docs-check.yml` | Push / PR touching `**.md` (non-main / non-release branches) | Markdown lint + Mermaid diagram parse validation | -| `licence-check.yml` | Push / PR touching `package.json` or `package-lock.json` | Verify all production dependency licences are compatible with MIT | - -> **Diagrams** are written in Mermaid and render natively in Gitea — no separate diagram files or CI render step required. See [Section 13](#13-diagrams). - ---- - -## 13. Diagrams - -All diagrams are written in [Mermaid](https://mermaid.js.org/) and render natively in Gitea and GitHub markdown. No external tooling or PNG exports are required — the source is the diagram. - -### 13.1 Component Diagram - -```mermaid -graph TB - subgraph Browser - html[index.html] - appjs[app.js] - css[style.css] - html -->|loads| appjs - html -->|loads| css - end - - subgraph Express Server - entry[index.js\nEntry Point] - appfactory[app.js\ncreateApp factory] - - subgraph Middleware - hm[helmet\nCSP nonce + HSTS] - rl[express-rate-limit\nAPI + login] - cp[cookie-parser\nsigned cookies] - ej[express.json\n64kb limit] - es[express.static] - requireauth[requireAuth.js] - verifycsrf[verifyCsrf.js\ndouble-submit] - end - - subgraph Routes - auth[auth.js\n/api/auth\npre-CSRF] - dashboard[dashboard.js\n/api/dashboard\n+SSE /stream] - emby_r[emby.js\n/api/emby] - sab_r[sabnzbd.js\n/api/sabnzbd] - sonarr_r[sonarr.js\n/api/sonarr] - radarr_r[radarr.js\n/api/radarr] - history_r[history.js\n/api/history] - end - - subgraph Utilities - poller[poller.js] - cache[cache.js\nMemoryCache] - config[config.js] - qbt[qbittorrent.js\nQBittorrentClient] - tokenstore[tokenStore.js\ntokens.json] - sanitize[sanitizeError.js] - logger[logger.js] - historyfetcher[historyFetcher.js] - end - - entry --> appfactory - entry --> es - entry --> poller - - appfactory --> hm & rl & cp & ej - appfactory -->|pre-CSRF| auth - appfactory --> verifycsrf - appfactory --> dashboard & emby_r & sab_r & sonarr_r & radarr_r & history_r - - dashboard & emby_r & sab_r & sonarr_r & radarr_r & history_r --> requireauth - auth --> tokenstore - dashboard --> cache & poller & config & qbt - history_r --> cache & config & historyfetcher - historyfetcher --> cache & config - poller --> cache & config & qbt & logger - qbt --> config & logger - auth & dashboard -.-> sanitize - end - - subgraph External Services - emby[Emby / Jellyfin] - sab[SABnzbd] - sonarr[Sonarr] - radarr[Radarr] - qbit[qBittorrent] - end - - auth --> emby - dashboard --> emby - poller --> sab & sonarr & radarr - qbt --> qbit - emby_r --> emby - sab_r --> sab - sonarr_r --> sonarr - radarr_r --> radarr - - appjs -->|POST /login, GET /me, GET /csrf, POST /logout| auth - appjs -->|GET /stream SSE, GET /user-downloads, GET /status| dashboard - es -->|serve static| html -``` - -### 13.2 Authentication Sequence - -```mermaid -sequenceDiagram - actor User - participant Browser as Browser (app.js) - participant Auth as Express /api/auth - participant Tokens as TokenStore (tokens.json) - participant Emby as Emby Server - - rect rgb(240,240,255) - Note over Browser,Auth: Page Load - Browser->>Auth: GET /api/auth/me - Auth->>Auth: Read emby_user cookie (signed if COOKIE_SECRET) - alt Cookie valid - Auth-->>Browser: { authenticated: true, user } - Browser->>Auth: GET /api/auth/csrf - Auth-->>Browser: { csrfToken } + Set csrf_token cookie - Browser->>Browser: store csrfToken in memory - Browser->>Browser: showDashboard() + startSSE() - else No cookie / tampered - Auth-->>Browser: { authenticated: false } - Browser->>Browser: showLogin() - end - end - - rect rgb(240,255,240) - Note over Browser,Emby: Login - User->>Browser: Enter credentials (+ rememberMe) - Browser->>Auth: POST /api/auth/login - Note right of Auth: Rate limit: max 10 failed attempts per IP / 15 min - Auth->>Emby: POST /Users/authenticatebyname (DeviceId = sha256(username)[0:16]) - alt Valid credentials - Emby-->>Auth: { User.Id, AccessToken } - Auth->>Emby: GET /Users/{id} - Emby-->>Auth: { Name, Policy.IsAdministrator } - Auth->>Tokens: storeToken(userId, AccessToken) - Note right of Tokens: Server-side only, 31-day TTL, atomic write - Auth->>Auth: Set emby_user cookie (httpOnly, sameSite=strict, secure if TRUST_PROXY) - Auth->>Auth: Set csrf_token cookie (httpOnly=false, sameSite=strict) - Auth-->>Browser: { success: true, user, csrfToken } - Browser->>Browser: showDashboard() + startSSE() - else Invalid credentials - Emby-->>Auth: 401 - Auth-->>Browser: { success: false, error } - end - end - - rect rgb(255,245,230) - Note over Browser,Auth: Logout - User->>Browser: Click Logout - Browser->>Browser: stopSSE() - Browser->>Auth: POST /api/auth/logout - Auth->>Tokens: getToken(userId) - Tokens-->>Auth: { accessToken } - Auth->>Emby: POST /Sessions/Logout - Auth->>Tokens: clearToken(userId) - Auth->>Auth: clearCookie(emby_user, csrf_token) - Auth-->>Browser: { success: true } - Browser->>Browser: showLogin() - end -``` - -### 13.3 Dashboard SSE Stream Sequence - -```mermaid -sequenceDiagram - actor User - participant Browser as Browser (app.js) - participant Dashboard as Express /api/dashboard - participant Cache as MemoryCache - participant Poller - participant Ext as External Services - - User->>Browser: Login success / valid session - Browser->>Dashboard: GET /api/dashboard/stream (EventSource) - Dashboard->>Dashboard: requireAuth: extract user/isAdmin - Dashboard->>Dashboard: Set Content-Type: text/event-stream, register in activeClients - - opt Polling disabled AND cache empty - Dashboard->>Poller: pollAllServices() - Poller->>Ext: Parallel API calls - Ext-->>Poller: Raw data - Poller->>Cache: set poll:* keys (TTL=30s) - end - - Dashboard->>Cache: get all poll:* keys - Dashboard->>Dashboard: Build maps, match downloads, extractUserTag / buildTagBadges - Dashboard-->>Browser: data: { user, isAdmin, downloads } - Browser->>Browser: hideLoading() + renderDownloads() - - loop Every poll cycle - Poller->>Poller: pollAllServices() complete - Poller->>Dashboard: onPollComplete callback fires - Dashboard->>Cache: get all poll:* keys - Dashboard->>Dashboard: Rebuild payload - Dashboard-->>Browser: data: { user, isAdmin, downloads } - Browser->>Browser: renderDownloads() diff-based - end - - Note over Dashboard,Browser: : heartbeat every 25s keeps connection alive - - User->>Browser: Close tab / logout - Browser->>Dashboard: TCP close (req close event) - Dashboard->>Dashboard: offPollComplete(cb), clearInterval(heartbeat), delete activeClients[key] -``` - -### 13.4 Background Polling Cycle - -```mermaid -sequenceDiagram - participant Entry as index.js (startup) - participant Poller - participant Config - participant SAB as SABnzbd (per instance) - participant Sonarr as Sonarr (per instance) - participant Radarr as Radarr (per instance) - participant QBT as qBittorrent Client - participant Cache as MemoryCache - - Entry->>Poller: startPoller() - alt POLL_INTERVAL > 0 - Poller->>Poller: pollAllServices() immediate - Poller->>Poller: setInterval(pollAllServices, POLL_INTERVAL) - else POLL_INTERVAL = 0 - Poller-->>Entry: on-demand mode - end - - Note over Poller: Each poll cycle - Poller->>Poller: polling flag check (skip if concurrent) - Poller->>Poller: polling = true - - Poller->>Config: getSABnzbdInstances() / getSonarrInstances() / getRadarrInstances() - Config-->>Poller: instance configs - - Note over Poller,Cache: All 9 fetches run in parallel via Promise.all, each wrapped in timed() - - Poller->>SAB: GET /api?mode=queue - SAB-->>Poller: { queue: { slots, status, speed } } - Poller->>SAB: GET /api?mode=history&limit=10 - SAB-->>Poller: { history: { slots } } - Poller->>Sonarr: GET /api/v3/tag + queue + history - Sonarr-->>Poller: tags, queue records (includeSeries), history - Poller->>Radarr: GET /api/v3/tag + queue + history - Radarr-->>Poller: tags, queue records (includeMovie), history - Poller->>QBT: getTorrents() - QBT-->>Poller: [{ name, progress, ... }] - - Poller->>Poller: Record per-task timings: lastPollTimings = { totalMs, timestamp, tasks } - Poller->>Cache: set poll:* keys (TTL = POLL_INTERVAL x 3) - Poller->>Poller: Notify SSE subscribers (forEach cb()) - Poller->>Poller: polling = false -``` - -### 13.5 Server Class Diagram - -```mermaid -classDiagram - class EntryPoint["index.js (EntryPoint)"] { - +startPoller() - +app.listen() - setupLogging() - serveStatic() - } - class createApp["app.js (createApp factory)"] { - +createApp(skipRateLimits?) Express - mountHelmet() - mountRateLimiters() - mountRoutes() - mountErrorHandler() - } - class AuthRouter["auth.js (Router)"] { - +POST /login - +GET /me - +GET /csrf - +POST /logout - authenticateViaEmby() - issueCookies() - revokeToken() - } - class DashboardRouter["dashboard.js (Router)"] { - -activeClients Map - +GET /stream SSE - +GET /user-downloads - +GET /user-summary - +GET /status - +GET /cover-art - +POST /blocklist-search - buildDownloadPayload() - extractUserTag() - buildTagBadges() - getEmbyUsers() - getImportIssues() - } - class RequireAuth["requireAuth.js (Middleware)"] { - +requireAuth(req, res, next) - readCookie() - validateSchema() - } - class VerifyCsrf["verifyCsrf.js (Middleware)"] { - +verifyCsrf(req, res, next) - timingSafeEqual() - } - class MemoryCache { - -store Map - +get(key) any - +set(key, value, ttlMs) - +invalidate(key) - +clear() - +getStats() CacheStats - } - class Poller { - -POLL_INTERVAL number - -polling boolean - -subscribers Set - +startPoller() - +stopPoller() - +pollAllServices() - +onPollComplete(cb) - +offPollComplete(cb) - +getLastPollTimings() PollTimings - } - class Config { - +getSABnzbdInstances() Instance[] - +getSonarrInstances() Instance[] - +getRadarrInstances() Instance[] - +getQbittorrentInstances() Instance[] - } - class QBittorrentClient { - -url string - -authCookie string - +login() bool - +getTorrents() Torrent[] - +makeRequest(endpoint) - } - class TokenStore { - -STORE_PATH string - -TOKEN_TTL_MS 31days - +storeToken(userId, token) - +getToken(userId) - +clearToken(userId) - atomicWrite() - pruneExpired() - } - class SanitizeError { - +sanitizeError(err) string - redactQueryParams() - redactAuthHeaders() - } - - EntryPoint --> createApp : createApp() - EntryPoint --> Poller : startPoller() - createApp --> AuthRouter : mount pre-CSRF - createApp --> VerifyCsrf : apply to /api - createApp --> DashboardRouter - DashboardRouter --> RequireAuth - DashboardRouter --> MemoryCache - DashboardRouter --> Poller - DashboardRouter --> Config - DashboardRouter ..> SanitizeError - AuthRouter --> TokenStore - AuthRouter ..> SanitizeError - Poller --> MemoryCache - Poller --> Config - Poller --> QBittorrentClient - QBittorrentClient --> Config -``` - -### 13.6 Data Model Diagram - -```mermaid -classDiagram - class Download { - +type series|movie|torrent - +title string - +coverArt string - +status string - +progress string - +size string - +mb string - +mbmissing string - +speed string - +eta string - +seriesName string - +movieName string - +allTags string[] - +matchedUserTag string - +tagBadges TagBadge[] - +importIssues string[] - +downloadPath string - +targetPath string - +arrLink string - +seeds number - +peers number - +availability string - +hash string - +completedAt string - +canBlocklist boolean - +addedOn number - +arrQueueId number - +arrType string - +arrInstanceUrl string - +arrContentId number - +arrContentType string - } - class TagBadge { - +label string - +matchedUser string - } - class APIResponse { - +user string - +isAdmin boolean - +downloads Download[] - } - class SSEEvent { - +user string - +isAdmin boolean - +downloads Download[] - } - class StatusResponse { - +server ServerInfo - +polling PollingInfo - +cache CacheStats - +clients ClientInfo[] - } - class SessionCookie { - +id string - +name string - +isAdmin boolean - } - class SABnzbdQueueSlot { - +filename string - +percentage string - +mb string - +mbmissing string - +timeleft string - +status string - } - class qBittorrentTorrent { - +name string - +hash string - +progress float - +state string - +dlspeed number - +eta number - +num_seeds number - +num_leechs number - +availability number - +added_on number - } - class SonarrQueueRecord { - +seriesId number - +series SonarrSeries - +title string - +trackedDownloadStatus string - +statusMessages StatusMessage[] - } - class RadarrQueueRecord { - +movieId number - +movie RadarrMovie - +title string - +trackedDownloadStatus string - +statusMessages StatusMessage[] - } - - APIResponse "1" *-- "many" Download - SSEEvent "1" *-- "many" Download - Download "1" *-- "many" TagBadge - SABnzbdQueueSlot ..> Download : matched and transformed - qBittorrentTorrent ..> Download : mapTorrentToDownload() - SonarrQueueRecord ..> Download : coverArt, seriesName, tags - RadarrQueueRecord ..> Download : coverArt, movieName, tags -``` - -### 13.7 Frontend UI State Diagram - -```mermaid -stateDiagram-v2 - [*] --> SplashScreen : Page load - - SplashScreen --> CheckAuth : checkAuthentication() - - state CheckAuth <> - CheckAuth --> LoginForm : No session - CheckAuth --> Dashboard : Valid session - - state LoginForm { - [*] --> Idle - Idle --> Submitting : Submit form - Submitting --> Error : Auth failed - Error --> Submitting : Re-submit - Submitting --> [*] : Auth success - } - - LoginForm --> Dashboard : Auth success (fade transition) - - state Dashboard { - [*] --> Rendering - Rendering --> Rendering : SSE message triggers renderDownloads() - Rendering --> Rendering : Theme change - - state SSEConnection { - [*] --> Connecting - Connecting --> Connected : First message - Connected --> Reconnecting : Connection lost - Reconnecting --> Connected : Auto-reconnect - Connected --> Connecting : showAll toggled - } - - state StatusPanel { - [*] --> Closed - Closed --> Open : Click Status (admin) - Open --> Closed : Click close - Open --> Open : 5s timer refresh - } - } - - Dashboard --> LoginForm : Logout (stopSSE) -``` - -### 13.8 Poller State Diagram - -```mermaid -stateDiagram-v2 - [*] --> CheckConfig : startPoller() - - state CheckConfig <> - CheckConfig --> Disabled : POLL_INTERVAL = 0 - CheckConfig --> Idle : POLL_INTERVAL > 0 - - state Disabled { - [*] --> OnDemand - OnDemand : No background timer. Data fetched when dashboard request finds empty cache. - } - - Disabled --> Polling : dashboard triggers pollAllServices() - Polling --> Disabled : Poll complete (on-demand) - - Idle --> Polling : setInterval fires or immediate first poll - - state Polling { - [*] --> Locked - Locked : polling = true - Locked --> Fetching - Fetching --> Storing : All promises resolved - Fetching --> HandleError : Per-service error (caught) - Storing --> Notifying : Cache updated, TTL = POLL_INTERVAL x 3 - Notifying : Notify SSE subscribers - Notifying --> Done - Done : polling = false - Done --> [*] - } - - state HandleError { - [*] --> LogError - LogError : Log error, polling = false - } - - Polling --> Idle : Poll complete - HandleError --> Idle : Next interval - - state ConcurrentSkip { - [*] --> Skip - Skip : polling === true, skip cycle - } - Idle --> ConcurrentSkip : Interval fires while previous still running - ConcurrentSkip --> Idle : Log skip -``` - -### 13.9 Download Matching Flow - -```mermaid -flowchart TD - A([Start: user request]) --> B[Read all poll:* keys from MemoryCache] - B --> C[Build seriesMap, moviesMap\nsonarrTagMap, radarrTagMap] - C --> D{showAll?} - D -->|yes| E[Fetch Emby user list\ncached 60s → embyUserMap] - D -->|no| F - E --> F[userDownloads = empty array] - - F --> G[/SABnzbd queue slots/] - G --> H{Matches Sonarr queue?} - H -->|yes| I[Resolve series\nextractAllTags + extractUserTag] - I --> J{showAll + anyTag\nor matchedUserTag?} - J -->|yes| K[Build Download object\nAdd tagBadges if showAll\nAdd importIssues, admin fields] - K --> L[Push to userDownloads] - H --> M{Matches Radarr queue?} - M -->|yes| N[Resolve movie\nextractAllTags + extractUserTag] - N --> J - - L --> G - - G --> O[/SABnzbd history slots/] - O --> P{Matches Sonarr history?} - P -->|yes| Q[Resolve series\nBuild Download type=series\nAdd completedAt] - Q --> R{showAll+anyTag\nor matchedUserTag?} - R -->|yes| S[Push to userDownloads] - P --> T{Matches Radarr history?} - T -->|yes| U[Resolve movie\nBuild Download type=movie\nAdd completedAt] - U --> R - - S --> O - - O --> V[/qBittorrent torrents/] - V --> W{Matches Sonarr queue?} - W -->|yes| X[mapTorrentToDownload\n+ enrich with series] - X --> Y{Tag matches?} - Y -->|yes| Z[Push to userDownloads] - W --> AA{Matches Radarr queue?} - AA -->|yes| AB[mapTorrentToDownload\n+ enrich with movie] - AB --> Y - AA --> AC{Matches Sonarr history?} - AC -->|yes| AD[Resolve series via seriesMap] - AD --> Y - AC --> AE{Matches Radarr history?} - AE -->|yes| AF[Resolve movie via moviesMap] - AF --> Y - AE -->|no| AG[Skip - unmatched torrent] - - Z --> V - AG --> V - - V --> AH([Return JSON\nuser, isAdmin, downloads]) - - style K fill:#d4edda - style Q fill:#d4edda - style U fill:#d4edda - style X fill:#d4edda - style AB fill:#d4edda - style AD fill:#d4edda - style AF fill:#d4edda - style AG fill:#f8d7da -``` - - -