diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index 0101a12..d588c67 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -221,7 +221,7 @@ sofarr/ **`cache.js`** — Singleton `MemoryCache` backed by a `Map`. Each entry has an expiration timestamp. Provides `get`, `set`, `invalidate`, `clear`, and `getStats` (returns per-key size, item count, TTL remaining). -**`poller.js`** — Background polling engine. Fetches data from all configured service instances in parallel with per-task timing. Stores results in the cache with a configurable TTL. Can be disabled entirely (`POLL_INTERVAL=0`), in which case data is fetched on-demand per dashboard request. +**`poller.js`** — Background polling engine. Fetches data from all configured service instances in parallel with per-task timing. Stores results in the cache with a configurable TTL. Can be disabled entirely (`POLL_INTERVAL=0`), in which case data is fetched on-demand. After each successful poll it notifies all registered SSE subscriber callbacks so connected clients receive data immediately. **`qbittorrent.js`** — `QBittorrentClient` class with cookie-based authentication, automatic re-auth on 403, and persistent client instances. Includes torrent-to-download mapping (`mapTorrentToDownload`) and formatting utilities (`formatBytes`, `formatSpeed`, `formatEta`). @@ -253,18 +253,31 @@ Every `POLL_INTERVAL` ms (default 5000), the poller fetches from all services in Results are stored in the cache under `poll:*` keys with a TTL of `POLL_INTERVAL × 3`. -### 5.2 Dashboard Request +### 5.2 SSE Stream -When a user requests `/api/dashboard/user-downloads`: +When a browser opens `GET /api/dashboard/stream` (after authentication): -1. Read all `poll:*` keys from cache -2. Build `seriesMap` and `moviesMap` from embedded objects in queue records -3. Build `sonarrTagMap` and `radarrTagMap` from tag data -4. For each SABnzbd queue slot → try to match against Sonarr/Radarr queue records by title -5. For each SABnzbd history slot → try to match against Sonarr/Radarr history records -6. For each qBittorrent torrent → try to match against Sonarr/Radarr queue, then history -7. For each match, resolve the series/movie, extract the user tag, check if it belongs to the requesting user -8. Return only the user's downloads (or all, if admin with `showAll=true`) +1. Server sets `Content-Type: text/event-stream`, disables buffering (`X-Accel-Buffering: no`) +2. Immediately builds and sends the first payload (same matching logic as below) +3. Registers a callback with the poller's `onPollComplete` subscriber set +4. After every subsequent poll cycle completes the callback fires, rebuilds the payload, and writes a `data:` SSE frame +5. A 25-second heartbeat comment (`: heartbeat`) keeps the connection alive through proxies +6. On client disconnect: deregisters callback, stops heartbeat, removes from active-client map + +The browser's native `EventSource` API handles reconnection automatically on network interruption. + +### 5.3 Download Matching + +For each connected user the server: + +1. Reads all `poll:*` keys from cache +2. Builds `seriesMap` and `moviesMap` from embedded objects in queue records +3. Builds `sonarrTagMap` and `radarrTagMap` from tag data +4. For each SABnzbd queue slot → tries to match against Sonarr/Radarr queue records by title +5. For each SABnzbd history slot → tries to match against Sonarr/Radarr history records +6. For each qBittorrent torrent → tries to match against Sonarr/Radarr queue, then history +7. For each match, resolves the series/movie, extracts the user tag, checks if it belongs to the requesting user +8. Returns only the user's downloads (or all, if admin with `showAll=true`) --- @@ -336,7 +349,7 @@ Users are matched to downloads via tags in Sonarr/Radarr: ### Active Client Tracking -Each dashboard request reports the client's refresh rate. The server tracks active clients in a `Map`, pruning entries older than 30 seconds. This data powers the admin status panel's "effective refresh mode" display. +SSE connections are tracked precisely: registered on connect, removed on disconnect. The admin status panel shows each connected user and how long they have been connected. The `type: 'sse'` field distinguishes SSE clients from any legacy HTTP clients. --- @@ -476,15 +489,38 @@ Clear session and revoke the Emby token server-side. Does **not** require a CSRF --- +### `GET /api/dashboard/stream` + +Server-Sent Events stream. Opens a persistent connection that pushes download data on every poll cycle. + +**Query Parameters:** +| Param | Type | Description | +|-------|------|-------------| +| `showAll` | `"true"` | (Admin) Include all users' downloads | + +**Response:** `Content-Type: text/event-stream` + +Each event is a `data:` frame containing JSON: +```json +{ + "user": "Alice", + "isAdmin": false, + "downloads": [ /* download objects — same shape as /user-downloads */ ] +} +``` + +The connection is kept alive with `: heartbeat` comments every 25 seconds. The browser's `EventSource` reconnects automatically on failure. + +--- + ### `GET /api/dashboard/user-downloads` -Fetch downloads for the authenticated user. +Fetch downloads for the authenticated user (single HTTP request, no streaming). **Query Parameters:** | Param | Type | Description | |-------|------|-------------| | `showAll` | `"true"` | (Admin) Show all users' downloads | -| `refreshRate` | number (ms) | Client's current refresh rate for tracking | **Response (200):** ```json @@ -531,7 +567,7 @@ Admin-only server status. ] }, "clients": [ - { "user": "Alice", "refreshRateMs": 5000, "lastSeen": 1715817600000 } + { "user": "Alice", "type": "sse", "connectedAt": 1715817600000, "lastSeen": 1715817600000 } ] } ``` @@ -577,13 +613,13 @@ The frontend is a **vanilla JavaScript SPA** with no build step. All logic resid |----------|---------| | `checkAuthentication()` | On load: check session → show dashboard or login | | `handleLogin()` | Authenticate, fade login → splash → dashboard | -| `fetchUserDownloads()` | GET `/user-downloads`, update state, re-render | +| `startSSE()` | Open `EventSource` to `/stream`; handles incoming data + first-message loading hide | +| `stopSSE()` | Close `EventSource` and cancel reconnect timer | | `renderDownloads()` | Diff-based card rendering (create/update/remove) | | `createDownloadCard()` | Build DOM for a single download card; renders tag badges | | `updateDownloadCard()` | Update existing card in-place (progress, speed, etc.) | | `toggleStatusPanel()` | Show/hide admin status panel | -| `renderStatusPanel()` | Build status HTML (server, polling, cache, clients) | -| `startAutoRefresh()` | Start periodic `fetchUserDownloads` | +| `renderStatusPanel()` | Build status HTML (server, polling, SSE clients, cache) | | `initThemeSwitcher()` | Light / Dark / Mono theme support | ### Themes @@ -604,9 +640,11 @@ Download cards render tag badges in the card header: - Tags with **no matching Emby user** → amber badge showing the raw tag label (leftmost) - Tags **matched to a known Emby user** → accent badge showing the Emby display name (rightmost) -### Auto-Refresh +### Live Push via SSE -The dashboard polls the server at the user-selected interval (1s, 5s, 10s, or Off). The status panel auto-refreshes in sync. Client refresh rate is sent to the server on each request for active-client tracking. +The dashboard receives updates via a persistent `EventSource` connection to `GET /api/dashboard/stream`. The server pushes a new `data:` event immediately after every poll cycle completes — there is no client-side timer. The browser's `EventSource` implementation handles reconnection automatically on network interruption. + +The status panel refreshes on a fixed 5-second timer and shows each SSE client with its connect duration. --- diff --git a/docs/diagrams/class-server.puml b/docs/diagrams/class-server.puml index fce162f..9d7eb87 100644 --- a/docs/diagrams/class-server.puml +++ b/docs/diagrams/class-server.puml @@ -42,9 +42,11 @@ package "server/routes" { - activeClients : Map - CLIENT_STALE_MS : 30000 -- + + GET /stream (SSE, text/event-stream) + GET /user-downloads + GET /user-summary + GET /status + + GET /cover-art -- - getCoverArt(item) : string|null - extractAllTags(tags, tagMap) : string[] @@ -224,7 +226,8 @@ package "server/utils" { class "ClientInfo" as ci <> { + user : string - + refreshRateMs : number + + type : 'sse' + + connectedAt : number (timestamp) + lastSeen : number (timestamp) } } diff --git a/docs/diagrams/component.puml b/docs/diagrams/component.puml index 59eead1..0d9cc9b 100644 --- a/docs/diagrams/component.puml +++ b/docs/diagrams/component.puml @@ -30,7 +30,7 @@ package "Express Server" as server { package "Routes" as routes { [auth.js\n/api/auth\n(pre-CSRF)] as auth - [dashboard.js\n/api/dashboard] as dashboard + [dashboard.js\n/api/dashboard\n(+SSE /stream)] as dashboard [emby.js\n/api/emby] as emby_route [sabnzbd.js\n/api/sabnzbd] as sab_route [sonarr.js\n/api/sonarr] as sonarr_route @@ -86,6 +86,9 @@ package "Express Server" as server { auth ..> sanitize dashboard ..> sanitize + + note "Browser uses EventSource\n(SSE) to /stream.\nServer pushes on each\npoll cycle — no client timer." as sseNote + sseNote .. dashboard } cloud "External Services" as external { diff --git a/public/app.js b/public/app.js index eacfc87..301dbac 100644 --- a/public/app.js +++ b/public/app.js @@ -1,12 +1,15 @@ let currentUser = null; let downloads = []; -let refreshInterval = null; -let currentRefreshRate = 5000; // default 5 seconds let isAdmin = false; let showAll = false; let csrfToken = null; // double-submit CSRF token, sent as X-CSRF-Token on mutating requests const SPLASH_MIN_MS = 1200; // minimum splash display time +// SSE stream state +let sseSource = null; +let sseReconnectTimer = null; +const SSE_RECONNECT_MS = 3000; // browser already retries, but we add explicit backoff too + // Apply saved theme immediately (before DOMContentLoaded to avoid flash) (function() { const saved = localStorage.getItem('sofarr-theme') || 'light'; @@ -20,7 +23,6 @@ document.addEventListener('DOMContentLoaded', () => { document.getElementById('login-form').addEventListener('submit', handleLogin); document.getElementById('logout-btn').addEventListener('click', handleLogout); - document.getElementById('refresh-rate').addEventListener('change', handleRefreshRateChange); document.getElementById('show-all-toggle').addEventListener('change', handleShowAllToggle); document.getElementById('status-btn').addEventListener('click', toggleStatusPanel); }); @@ -41,37 +43,51 @@ function setTheme(theme) { }); } -function startAutoRefresh() { - if (refreshInterval) clearInterval(refreshInterval); - if (currentRefreshRate > 0) { - refreshInterval = setInterval(() => fetchUserDownloads(false), currentRefreshRate); - } +// --- SSE connection management --- + +function startSSE() { + stopSSE(); + const params = showAll ? '?showAll=true' : ''; + const source = new EventSource('/api/dashboard/stream' + params); + sseSource = source; + + let firstMessage = true; + source.onmessage = (event) => { + try { + const data = JSON.parse(event.data); + currentUser = data.user; + isAdmin = !!data.isAdmin; + downloads = data.downloads; + document.getElementById('currentUser').textContent = currentUser || '-'; + renderDownloads(); + hideError(); + if (firstMessage) { firstMessage = false; hideLoading(); } + } catch (err) { + console.error('[SSE] Failed to parse message:', err); + } + }; + + source.onerror = () => { + // EventSource retries automatically; we just log and show a reconnecting indicator + console.warn('[SSE] Connection lost, browser will retry...'); + }; + + console.log('[SSE] Stream connected'); } -function handleRefreshRateChange(e) { - const rate = parseInt(e.target.value); - currentRefreshRate = rate; - startAutoRefresh(); - // Restart status panel refresh if it's open - const statusPanel = document.getElementById('status-panel'); - if (statusPanel && statusPanel.style.display !== 'none') { - if (statusRefreshHandle) { clearInterval(statusRefreshHandle); statusRefreshHandle = null; } - if (currentRefreshRate > 0) { - statusRefreshHandle = setInterval(refreshStatusPanel, currentRefreshRate); - } +function stopSSE() { + if (sseReconnectTimer) { clearTimeout(sseReconnectTimer); sseReconnectTimer = null; } + if (sseSource) { + sseSource.close(); + sseSource = null; + console.log('[SSE] Stream closed'); } } function handleShowAllToggle(e) { showAll = e.target.checked; - fetchUserDownloads(true); -} - -function stopAutoRefresh() { - if (refreshInterval) { - clearInterval(refreshInterval); - refreshInterval = null; - } + // Re-open stream with updated showAll param + startSSE(); } function fadeOutLogin() { @@ -132,8 +148,8 @@ async function checkAuthentication() { currentUser = data.user; isAdmin = !!data.user.isAdmin; showDashboard(); - await fetchUserDownloads(true); - startAutoRefresh(); + showLoading(); + startSSE(); await dismissSplash(splashStart); } else { await dismissSplash(splashStart); @@ -169,7 +185,7 @@ async function handleLogin(e) { isAdmin = !!data.user.isAdmin; // Store CSRF token returned by login for use in subsequent requests if (data.csrfToken) csrfToken = data.csrfToken; - // Fade out login, then show splash while loading data. + // Fade out login, then show splash while opening SSE stream. // requestAnimationFrame ensures the browser paints the splash at // opacity:1 before dismissSplash adds fade-out, so the CSS // transition fires and transitionend is guaranteed. @@ -177,9 +193,9 @@ async function handleLogin(e) { showSplash(); await new Promise(r => requestAnimationFrame(() => requestAnimationFrame(r))); showDashboard(); + showLoading(); const splashStart = Date.now(); - await fetchUserDownloads(true); - startAutoRefresh(); + startSSE(); await dismissSplash(splashStart); } else { showLoginError(data.error || 'Login failed'); @@ -192,7 +208,7 @@ async function handleLogin(e) { async function handleLogout() { try { - stopAutoRefresh(); + stopSSE(); await fetch('/api/auth/logout', { method: 'POST', headers: csrfToken ? { 'X-CSRF-Token': csrfToken } : {} @@ -230,40 +246,8 @@ function hideLoginError() { errorDiv.style.display = 'none'; } -async function fetchUserDownloads(isInitialLoad = false) { - if (isInitialLoad) { - showLoading(); - } - hideError(); - - try { - const params = new URLSearchParams(); - if (showAll) params.set('showAll', 'true'); - params.set('refreshRate', currentRefreshRate); - const url = '/api/dashboard/user-downloads?' + params.toString(); - const response = await fetch(url); - const data = await response.json(); - - currentUser = data.user; - isAdmin = !!data.isAdmin; - downloads = data.downloads; - - // Debug: log first download to see what fields are present - if (downloads.length > 0) { - console.log('[Dashboard] Download data:', JSON.stringify(downloads[0])); - } - - document.getElementById('currentUser').textContent = currentUser || '-'; - renderDownloads(); - } catch (err) { - showError('Failed to fetch downloads. Make sure all services are configured.'); - console.error(err); - } finally { - if (isInitialLoad) { - hideLoading(); - } - } -} +// fetchUserDownloads is kept for the showAll toggle re-connection case +// but the primary data path is now via SSE (startSSE / EventSource). function renderDownloads() { const downloadsList = document.getElementById('downloads-list'); @@ -628,6 +612,7 @@ function escapeHtml(str) { } let statusRefreshHandle = null; +const STATUS_REFRESH_MS = 5000; async function toggleStatusPanel() { const panel = document.getElementById('status-panel'); @@ -638,11 +623,8 @@ async function toggleStatusPanel() { } panel.style.display = 'block'; await refreshStatusPanel(); - // Auto-refresh in sync with dashboard refresh rate if (statusRefreshHandle) clearInterval(statusRefreshHandle); - if (currentRefreshRate > 0) { - statusRefreshHandle = setInterval(refreshStatusPanel, currentRefreshRate); - } + statusRefreshHandle = setInterval(refreshStatusPanel, STATUS_REFRESH_MS); } function closeStatusPanel() { @@ -693,11 +675,7 @@ function renderStatusPanel(data, panel) { const pollIntervalMs = data.polling.intervalMs; const clients = data.clients || []; - const activeRefreshers = clients.filter(c => c.refreshRateMs > 0); - const fastestClient = activeRefreshers.length > 0 - ? activeRefreshers.reduce((min, c) => c.refreshRateMs < min.refreshRateMs ? c : min) - : null; - const hasForegroundClient = fastestClient && data.polling.enabled && fastestClient.refreshRateMs < pollIntervalMs; + const sseClients = clients.filter(c => c.type === 'sse'); if (data.polling.enabled) { html += `
Background poll${pollIntervalMs / 1000}s
`; @@ -705,19 +683,15 @@ function renderStatusPanel(data, panel) { html += `
Background pollDisabled
`; } - if (hasForegroundClient) { - html += `
Effective modeForeground ${fastestClient.refreshRateMs / 1000}s
`; - } else if (activeRefreshers.length > 0) { - html += `
Effective mode${data.polling.enabled ? 'Background' : 'On-demand'}
`; - } else { - html += `
Effective modeIdle (no active clients)
`; - } + const mode = sseClients.length > 0 + ? `SSE push` + : (data.polling.enabled ? 'Background' : 'On-demand (idle)'); + html += `
Delivery mode${mode}
`; - html += `
Active clients${clients.length}
`; - for (const c of clients) { - const rate = c.refreshRateMs > 0 ? (c.refreshRateMs / 1000) + 's' : 'Off'; - const age = Math.round((Date.now() - c.lastSeen) / 1000); - html += `
${escapeHtml(c.user)}${rate} (${age}s ago)
`; + html += `
SSE clients${sseClients.length}
`; + for (const c of sseClients) { + const age = Math.round((Date.now() - c.connectedAt) / 1000); + html += `
${escapeHtml(c.user)}connected ${age}s ago
`; } html += ``; diff --git a/public/index.html b/public/index.html index 5c23feb..ae54cd0 100644 --- a/public/index.html +++ b/public/index.html @@ -53,15 +53,6 @@ -
- - -