sofarr/ARCHITECTURE.md

# sofarr — Architecture Reference

> Concise top-level architecture guide. For the full deep-dive (API reference, matching pipeline, deployment) see [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md).

---

## 1. Overview

sofarr is a **Node.js/Express** single-page application. It aggregates download activity from multiple media automation services, filters results by Emby user identity, and presents a real-time personalised dashboard.

Three pluggable layers form the core:

| Layer | Name | Location |
|-------|------|----------|
| Download client abstraction | **PDCA** — Pluggable Download Client Architecture | `server/clients/` + `server/utils/downloadClients.js` |
| *arr data retrieval | **PALDRA** — Pluggable *Arr Library Data Retrieval Architecture | `server/utils/arrRetrievers.js` |
| Real-time push | **Webhook receiver** | `server/routes/webhook.js` |

---

## 2. Request / Data Flow

```
Browser (SPA)
    │  POST /api/auth/login          → Auth routes → Emby verify → set httpOnly cookie
    │  GET  /api/dashboard/stream    → SSE stream → poller cache → matched downloads
    │  POST /api/webhook/*           ← Sonarr/Radarr push events
    │
    ▼
Express Server (:3001)
    ├── Helmet (CSP nonce, HSTS, X-Frame-Options, …)
    ├── express-rate-limit (300/15 min general; 60/1 min webhook)
    ├── cookie-parser (HMAC-signed session cookie)
    ├── verifyCsrf (double-submit cookie, all state-changing /api routes except auth + webhook)
    │
    ├── /api/auth          → login, logout, me, csrf
    ├── /api/webhook       → [rate-limit] → [secret validation] → [payload validation]
    │                        → [replay check] → updateWebhookMetrics → processWebhookEvent
    ├── /api/dashboard     → requireAuth → read cache → match downloads → SSE/JSON
    ├── /api/history       → requireAuth → historyFetcher (5 min cache) → filter + dedup
    ├── /api/sonarr|radarr → requireAuth → verifyCsrf → proxy to *arr API
    └── /api/sabnzbd|emby  → requireAuth → verifyCsrf → proxy

Background:
    Poller (setInterval POLL_INTERVAL ms)
        └── shouldSkipInstancePolling? ──yes──► extend cache TTL, increment pollsSkipped
                │ no (or fallback triggered)
                ▼
            PDCA Registry.getDownloadsByClientType()
            PALDRA Registry.getQueuesByType() / getHistoryByType() / getTagsByType()
                │
                ▼
            cache.set('poll:*', data, TTL)
                │
                ▼
            notify pollSubscribers → SSE push to all connected browsers
```

---

## 3. Pluggable Download Client Architecture (PDCA)

All download clients extend `DownloadClient` (abstract base in `server/clients/DownloadClient.js`):

```
DownloadClient (abstract)
├── SABnzbdClient       — REST API, API key auth
├── QBittorrentClient   — Sync API (incremental deltas), cookie auth, fallback to /torrents/info
├── TransmissionClient  — JSON-RPC, session-ID management
└── RTorrentClient      — XML-RPC, HTTP Basic Auth
```

`DownloadClientRegistry` (`server/utils/downloadClients.js`) initialises all configured clients from `*_INSTANCES` env vars, fetches from all in parallel, and returns a `{ sabnzbd, qbittorrent, transmission, rtorrent }` map. Individual client failures are isolated.

**Adding a new client:** extend `DownloadClient`, implement `getActiveDownloads()` returning `NormalizedDownload[]`, register in the registry factory.

---

## 4. Pluggable *Arr Retrieval Architecture (PALDRA)

`server/utils/arrRetrievers.js` provides `arrRetrieverRegistry` which:
- Initialises one retriever per configured Sonarr/Radarr instance
- Exposes `getQueuesByType()`, `getHistoryByType()`, `getTagsByType()` — returning results keyed by `sonarr` / `radarr`
- Results carry `{ instance: instanceId, data: … }` so callers can look up instance credentials

The poller and webhook processor both use the same registry, ensuring consistency.

---

## 5. Webhook Flow (Phase 1–5.1)

```
Sonarr/Radarr
    POST /api/webhook/sonarr  (X-Sofarr-Webhook-Secret: <secret>)
    {
      "eventType": "Grab",
      "instanceName": "Main Sonarr",
      "date": "2026-05-19T10:00:00.000Z",
      …
    }
        │
        ▼
    webhookLimiter (60 req/min/IP)
        │
        ▼
    validateWebhookSecret()  ──fail──► 401 Unauthorized
        │ ok
        ▼
    validatePayload()        ──fail──► 400 Bad Request
        │ ok
        ▼
    isReplay()               ──yes───► 200 { received: true, duplicate: true }
        │ no
        ▼
    cache.updateWebhookMetrics(instance.url)   ← activates smart polling skip
        │
        ▼
    processWebhookEvent('sonarr', 'Grab')  [fire-and-forget]
        ├── classify: Grab → QUEUE_EVENT
        ├── arrRetrieverRegistry.getQueuesByType()
        ├── cache.set('poll:sonarr-queue', …, CACHE_TTL)
        └── pollAllServices()  → pollSubscribers.forEach(cb) → SSE push
        │
        ▼
    200 { received: true }  (returned immediately, before fire-and-forget completes)
```

---

## 6. Smart Polling Optimization (Phase 5)

```
pollAllServices() called every POLL_INTERVAL ms:

  globalMetrics = cache.getGlobalWebhookMetrics()
  fallbackTriggered = lastGlobalWebhookTimestamp > WEBHOOK_FALLBACK_TIMEOUT ago
  
  for each service type (sonarr, radarr):
    shouldSkip = !fallbackTriggered
                 && all instances have metrics.eventsReceived > 0
                 && all instances have metrics.lastWebhookTimestamp within WEBHOOK_FALLBACK_TIMEOUT

    if shouldSkip:
      extend TTL of existing cached data        ← no API calls made
      increment metrics.pollsSkipped
      log "[Poller] Skipping sonarr polling for N instance(s) with active webhooks"
    else:
      fetch from *arr APIs → update cache
```

**Result:** zero *arr API calls per poll cycle when webhooks are active and recent. Falls back automatically after `WEBHOOK_FALLBACK_TIMEOUT` minutes of silence (default: 10).

---

## 7. Cache Keys

| Key | Content | TTL |
|-----|---------|-----|
| `poll:sab-queue` | SABnzbd queue slots + status | `POLL_INTERVAL × 3` |
| `poll:sab-history` | SABnzbd history slots | `POLL_INTERVAL × 3` |
| `poll:sonarr-queue` | Sonarr queue records (with `_instanceUrl`) | `POLL_INTERVAL × 3` |
| `poll:sonarr-history` | Sonarr history records | `POLL_INTERVAL × 3` |
| `poll:sonarr-tags` | Sonarr tag list per instance | `POLL_INTERVAL × 3` |
| `poll:radarr-queue` | Radarr queue records (with `_instanceUrl`) | `POLL_INTERVAL × 3` |
| `poll:radarr-history` | Radarr history records | `POLL_INTERVAL × 3` |
| `poll:radarr-tags` | Radarr tag list | `POLL_INTERVAL × 3` |
| `poll:qbittorrent` | qBittorrent torrent list | `POLL_INTERVAL × 3` |
| `history:sonarr` | Sonarr history (on-demand, `/api/history/recent`) | 5 min |
| `history:radarr` | Radarr history (on-demand) | 5 min |
| `emby:users` | Emby user list | 60 s |

When polling is disabled (`POLL_INTERVAL=0`), all `poll:*` TTLs fall back to 30 s.

---

## 8. Security Model

| Concern | Mechanism |
|---------|-----------|
| User authentication | Emby credentials → httpOnly HMAC-signed cookie |
| Session validation | `requireAuth` middleware on all `/api/dashboard`, `/api/history`, proxy routes |
| CSRF | Double-submit cookie (`X-CSRF-Token` header) on all state-changing routes |
| Webhook auth | Shared secret on `X-Sofarr-Webhook-Secret` header (webhook routes are outside CSRF) |
| Webhook input | `validatePayload()` allowlists event types; rejects invalid shapes |
| Webhook replay | 5-minute nonce cache keyed on `(eventType, instanceName, date)` |
| Rate limiting | 300 req/15 min (general), 10 fails/15 min (login), 60 req/1 min (webhook) |
| Secret leakage | `sanitizeError()` redacts all secrets from error messages and logs |
| Headers | Helmet v7: CSP nonce, HSTS, X-Frame-Options DENY, noSniff, Referrer-Policy |

---

## 9. Directory Structure (summary)

```
sofarr/
├── server/
│   ├── app.js                  Express factory (imported by tests + index.js)
│   ├── index.js                Entry point: logging, listen, start poller
│   ├── clients/                PDCA — one file per download client
│   ├── routes/
│   │   ├── auth.js             Login / logout / csrf / me
│   │   ├── dashboard.js        SSE stream, downloads, status, cover-art
│   │   ├── history.js          Recently completed downloads
│   │   ├── webhook.js          Webhook receiver (Phase 1–6)
│   │   ├── sonarr.js           Sonarr API proxy + webhook management
│   │   └── radarr.js           Radarr API proxy + webhook management
│   ├── middleware/
│   │   ├── requireAuth.js      Cookie auth enforcement
│   │   └── verifyCsrf.js       Double-submit CSRF check
│   └── utils/
│       ├── arrRetrievers.js    PALDRA — Sonarr/Radarr fetch registry
│       ├── cache.js            MemoryCache + webhook metrics helpers
│       ├── config.js           Multi-instance config parser
│       ├── downloadClients.js  PDCA registry + factory
│       ├── historyFetcher.js   History fetch + event classification
│       ├── poller.js           Smart background polling engine
│       ├── sanitizeError.js    Secret redaction from errors
│       └── tokenStore.js       Emby token store (JSON file, atomic writes)
├── public/                     Static SPA (HTML + CSS + vanilla JS)
├── tests/
│   ├── setup.js                Isolated DATA_DIR, SKIP_RATE_LIMIT
│   ├── unit/                   Pure unit tests
│   └── integration/            Supertest + nock integration tests
├── docs/ARCHITECTURE.md        Full deep-dive architecture documentation
├── ARCHITECTURE.md             This file — concise reference
├── SECURITY.md                 Threat model + hardening guide
├── CHANGELOG.md                Version history
└── .env.sample                 Annotated configuration template
```

---

*For complete API reference, data-flow diagrams, download matching pipeline, qBittorrent Sync API details, and deployment guidance see [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md).*