80 Commits

Author	SHA1	Message	Date
gronod	0acd452ebd	Implement staged history loading with SSE push ... - Stage 1: Fetch 100 records immediately for fast display - Stage 2+: Background fetch up to 1000 records in batches of 100 - Date-based cursor pagination to avoid race conditions - Deduplication by record ID to prevent duplicates - SSE push to clients when history cache is updated - Shared background fetch state for concurrent user requests	2026-05-21 01:23:11 +01:00
gronod	b9b5d7d393	Increase history pageSize from 100 to 500 to fetch more records ... Fixes issue where series beyond position 100 in history were not appearing in recently completed section.	2026-05-21 01:10:36 +01:00
gronod	7424e70ea6	Add logging for total Sonarr/Radarr records fetched	2026-05-21 01:06:28 +01:00
gronod	830dea3d6b	Add logging for filtered event types and missing series/movie objects	2026-05-21 01:02:57 +01:00
gronod	4ff462b7f4	Add detailed logging for all series/movies with raw tag IDs to debug missing items	2026-05-21 01:01:14 +01:00
gronod	d9f1fc99a9	Add debugging logs for history filtering to diagnose missing series	2026-05-21 00:58:10 +01:00
gronod	a38fc4a8ce	refactor: extract status route and WebhookStatus service, slim dashboard.js ... - Extract /status route to server/routes/status.js - Create server/services/WebhookStatus.js with checkWebhookConfigured and aggregateMetrics - Slim dashboard.js to pure HTTP orchestration (559→283 lines, 49.4% reduction) - Remove /user-summary and /webhook-metrics routes from dashboard.js - Mount status router at /api/status in server/index.js and server/app.js - Update tests to use new /api/status/status endpoint - Fix test expectation for speed field (number vs string) All 571 tests passing.	2026-05-20 22:50:40 +01:00
gronod	2bf4cb2a0f	Refactor: Deduplicate download assembly logic into DownloadBuilder service ... - Created server/services/DownloadBuilder.js with buildUserDownloads function - Added private helpers: buildSeriesMapFromRecords, buildMoviesMapFromRecords, matchSabSlots, matchSabHistory, matchTorrents, getSlotStatusAndSpeed - Updated server/routes/dashboard.js to use buildUserDownloads in /user-downloads and SSE /stream - Removed ~500 lines of duplicated download-assembly logic - All unit tests passing (DownloadBuilder: 14, DownloadAssembler: 73, TagMatcher: 26)	2026-05-20 22:43:03 +01:00
gronod	9cffb96f29	Extract DownloadAssembler service from dashboard routes ... - Create server/services/DownloadAssembler.js with 7 pure functions: - getCoverArt, getImportIssues, getSonarrLink, getRadarrLink - canBlocklist, extractEpisode, gatherEpisodes - Update server/routes/dashboard.js to use DownloadAssembler - Add comprehensive unit tests (73 tests covering edge cases) - Fix null check in extractEpisode function - All tests passing: DownloadAssembler (73/73), TagMatcher (26/26)	2026-05-20 22:32:09 +01:00
gronod	4d61dd566f	Refactor: Extract tag functions to TagMatcher service ... - Extract six pure tag-related functions from dashboard.js into new server/services/TagMatcher.js - Functions: sanitizeTagLabel, tagMatchesUser, extractAllTags, extractUserTag, getEmbyUsers, buildTagBadges - Update dashboard.js to import TagMatcher and replace all inline function calls - Add comprehensive unit tests in tests/unit/services/TagMatcher.test.js (26 tests passing for 5 pure functions) - Note: getEmbyUsers tests excluded due to CommonJS mocking complexity	2026-05-20 22:21:01 +01:00
gronod	5ad525a760	fix: webhook replay cache atomicity and instanceName precision	2026-05-20 20:46:35 +01:00
gronod	7195a09562	Fix SABnzbd size and speed fields in SSE response	2026-05-19 23:34:24 +01:00
gronod	720de6688b	Add download client ordering and filtering to active downloads list	2026-05-19 23:29:38 +01:00
gronod	6c8c333c6a	debug: Add Sonarr queue titles to no-match output	2026-05-19 22:16:26 +01:00
gronod	5dfe0b1216	fix(matching): Match SAB to Sonarr by downloadId first ... Sonarr tracks the exact SAB download ID (nzo_id). Now tries to match by downloadId first, then falls back to title matching. Also adds debug to show if matches are via downloadId vs title, and logs downloadIds in history to verify the link exists.	2026-05-19 22:13:43 +01:00
gronod	77beef787f	debug(matching): Show queue vs history source and history titles ... When a match is found, logs whether it came from queue or history. When no match, shows history counts and sample titles to verify history is being checked properly.	2026-05-19 22:10:34 +01:00
gronod	235a866ec8	fix(matching): Check Sonarr/Radarr history for SAB matches ... SAB items often persist after Sonarr has processed them. Previously only checked the active queue, now also checks history records so completed downloads still appear.	2026-05-19 22:06:38 +01:00
gronod	f1d9de2a92	debug(sonarr): Log all available Sonarr queue fields ... Shows title, sourceTitle, series.title, episode.title for each Sonarr queue item to understand the data structure.	2026-05-19 22:04:11 +01:00
gronod	9d0e31ec9a	fix(matching): Normalize dots to spaces for SAB/Sonarr matching ... SAB filenames use dots (dora.the.explorer.s02e08) but Sonarr titles use spaces (Dora the Explorer - S02E08). Now tries matching with both formats to improve match rate. Also logs actual Sonarr titles when no match found for debugging.	2026-05-19 22:02:55 +01:00
gronod	42c3eebf18	debug(sse): Add detailed name matching logging ... Shows exactly which SAB items match/don't match to Sonarr/Radarr: - ✓ Sonarr match: SAB name → Sonarr name - ✓ Radarr match: SAB name → Radarr name - ✗ No match: SAB name (with Sonarr queue count) This will help diagnose why Sonarr Activity Queue shows matches but Sofarr doesn't.	2026-05-19 21:50:05 +01:00
gronod	f295e1c90d	debug(sse): Add SAB matching stats to trace filtering ... Shows how many SAB items were checked vs how many matched to Sonarr/Radarr. This will help diagnose why only ~10 of 60 SAB items are appearing.	2026-05-19 21:47:12 +01:00
gronod	f22dd0d1f6	fix(downloads): Fix SABnzbd/qBittorrent collision and webhook metrics ... 1. Fixed download client collision: - SABnzbd client with id 'i3omb' was being overwritten by qBittorrent - Now uses unique key ':' like the arr retrievers 2. Fixed webhook metrics showing 0: - instanceName from webhooks is generic ('Sonarr', 'Radarr') - Not the configured instance name ('i3omb') - Now updates metrics for ALL instances of that type	2026-05-19 21:40:53 +01:00
gronod	ccc3b6ffec	fix(status): Check actual webhook config, show enabled even with 0 events ... The status panel was showing webhooks as disabled (null) when no events had been received yet. Now it checks Sonarr/Radarr API to see if the Sofarr webhook notification is actually configured. - Added checkWebhookConfigured() to verify webhook exists in Sonarr/Radarr - Shows 'enabled: true' with 0 events when webhook is configured - Only shows null when webhook is not configured at all	2026-05-19 21:35:26 +01:00
gronod	4ec7d734b8	debug(sse): Add detailed logging for download matching ... Add debug logging to trace: - When downloads payload is built - Data sizes from cache (SAB, qBit, Sonarr, Radarr) - Number of downloads found and their titles This will help diagnose why Dora downloads aren't appearing.	2026-05-19 21:32:15 +01:00
gronod	2e85fae57a	fix(webhooks): Load collapsed by default, add webhook metrics to status panel ... - Fixed webhooks section to load collapsed (content hidden, toggle arrow reset) - Added webhook metrics card to status panel for admin users: - Shows Sonarr/Radarr enabled/disabled status - Shows events received and polls skipped counts - Updated /api/dashboard/status endpoint to include webhook metrics - Metrics are aggregated from all Sonarr/Radarr instances	2026-05-19 21:24:28 +01:00
gronod	0f3c02e52d	fix(webhooks): Use numeric method value (1=POST) in notification payload ... The webhook notification payload was using string 'POST' for the method field, but Sonarr/Radarr API expects numeric values: - 1 = POST - 2 = PUT Also added onManualInteractionRequired: false to match the schema. Fixes: Radarr/Sonarr rejecting webhook configuration with validation errors	2026-05-19 20:47:19 +01:00
gronod	9fd60bcfed	fix(webhooks): Use SONARR_INSTANCES/RADARR_INSTANCES config for notification routes ... The notification routes were using process.env.SONARR_URL directly, which is undefined when using the newer SONARR_INSTANCES JSON format. Changes: - Added getFirstSonarrInstance() and getFirstRadarrInstance() helpers - Updated /notifications, /notifications/test, and /notifications/sofarr-webhook routes to use instance config from getSonarrInstances()/getRadarrInstances() - Returns 503 error if no instances are configured Fixes: 'Invalid URL' errors when calling Sonarr/Radarr notification APIs	2026-05-19 20:42:59 +01:00
gronod	af58e1bf2a	debug(webhooks): Add console.error logging to Sonarr/Radarr notification routes ... Added detailed error logging to help diagnose 500 errors when calling Sonarr/Radarr notification APIs. Logs include: - Error message - Response status (if available) - Response data (if available) This will help identify if the issue is: - Missing SONARR_URL/RADARR_URL or API keys - Network connectivity issues - Sonarr/Radarr API version incompatibility	2026-05-19 20:39:37 +01:00
gronod	d06e24dbb6	feat(webhooks): display webhook statistics (events received, polls skipped, last event) in status panel	2026-05-19 19:18:29 +01:00
gronod	1bef14d590	feat(webhooks): security hardening, tests, full documentation audit & polish (Phase 6)	2026-05-19 17:11:45 +01:00
gronod	8609f03c5a	fix(webhooks): connect receiver to cache metrics for polling optimization (Phase 5.1)	2026-05-19 16:41:39 +01:00
gronod	e022db8ef5	feat(webhooks): add notification management API + one-click Sofarr webhook setup (Phase 3)	2026-05-19 15:31:50 +01:00
gronod	1d61ea8d83	feat(webhooks): integrate receiver with cache + SSE (Phase 2)	2026-05-19 15:24:43 +01:00
gronod	99ddb05dbe	feat(webhook): implement Phase 1 webhook receiver for Sonarr and Radarr ... - Added POST /api/webhook/sonarr and POST /api/webhook/radarr endpoints - Implemented webhook secret validation via SOFARR_WEBHOOK_SECRET environment variable - Added logging for all incoming webhook events using existing logToFile utility - Returns HTTP 200 immediately to prevent webhook retries - Mounted webhook routes before CSRF middleware (called by external services) - Non-breaking: no changes to polling, caching, SSE, or any existing behavior - Lays groundwork for Phase 2 (cache + SSE integration) without implementing it yet	2026-05-19 15:15:53 +01:00
gronod	8c4cc20551	Add MIT copyright headers to all source files	2026-05-19 09:07:42 +01:00
gronod	2747ca7754	feat: allow non-admin users to blocklist & search under specific conditions ... - Added addedOn timestamp to qBittorrent torrent mapping - Added canBlocklist helper function: true for admins, true for non-admins when (importIssues OR (torrent >1h old AND availability<100%)) - Added canBlocklist field to all download objects in /user-downloads and SSE /stream routes (8 blocks total) - Frontend button now shows when (isAdmin OR download.canBlocklist) && download.arrQueueId	2026-05-17 23:57:06 +01:00
gronod	0341540751	feat: show blocklist & search button on all admin downloads (not just import-pending) ... - Remove importIssues condition from arr action fields threading in /user-downloads route (all 4 blocks: SAB+Sonarr, SAB+Radarr, qBit+Sonarr, qBit+Radarr) - Remove importIssues condition from arr action fields threading in SSE /stream route (all 4 blocks) - Move blocklist button rendering outside importIssues condition in frontend — now shows for all admin downloads with arrQueueId	2026-05-17 23:43:37 +01:00
gronod	a6fcde58cf	fix: thread arr action fields through SSE handler; align import-issue tooltip with themed CSS pattern	2026-05-17 23:20:04 +01:00
gronod	d839fa98a0	feat: blocklist & search button for import-pending downloads with caution ... - Poller now stores _instanceKey alongside _instanceUrl on Sonarr/Radarr queue records - dashboard route threads arrQueueId/arrType/arrInstanceUrl/arrInstanceKey/arrContentId/arrContentType as admin-only fields on downloads with importIssues - POST /api/dashboard/blocklist-search: admin-only, removes queue item with blocklist=true then triggers EpisodeSearch/MoviesSearch - Button renders in download card header (admin + importIssues + arrQueueId only) - Confirm dialog, loading/success/error states on the button - Kicks a background poll on success so SSE reflects removed item promptly	2026-05-17 23:15:33 +01:00
gronod	6139095444	feat: deduplicate history — suppress failed records superseded by successful import, flag failed+hasFile as availableForUpgrade	2026-05-17 21:52:55 +01:00
gronod	37c8229061	fix: read episodeNumber from nested episode object in Sonarr records ... Sonarr queue and history records do not expose episodeNumber at the top level — it is only present inside the nested episode object (record.episode.episodeNumber). Same for seasonNumber. The original extractEpisode() read record.episodeNumber which was always undefined, so gatherEpisodes() always returned an empty array. Fix: prefer the nested episode object fields, falling back to the top-level fields for forward-compatibility.	2026-05-17 17:19:39 +01:00
gronod	d1496a76e2	feat: show episode info on download and history cards ... - Add includeEpisode:true to Sonarr queue and history API requests in both the poller and historyFetcher - Add extractEpisode() / gatherEpisodes() helpers in dashboard.js and history.js to build a sorted, deduplicated episodes array covering all records matching a download title (handles multi- episode packs and series packs) - Replace episodeInfo: sonarrMatch with episodes: gatherEpisodes() across all 8 assignment sites in dashboard.js - Add episodes field to /api/history/recent response items - Frontend: formatEpisodeInfo() renders S01E05 for single episodes or 'Multiple episodes' with hover tooltip listing all for packs - CSS: .episode-info and .multi-episode tooltip styles - ARCHITECTURE.md: update polling table and download/history schemas	2026-05-17 17:03:23 +01:00
gronod	dd7e3e2a90	fix(history): add tagBadges to history items in showAll mode	2026-05-17 13:05:23 +01:00
gronod	ddcfbda0c2	feat(history): add /api/history/recent endpoint with Sonarr/Radarr history fetching, tag filtering, and 5-min cache	2026-05-17 12:05:30 +01:00
gronod	f41d14b2a9	fix: gate cookie secure flag on TRUST_PROXY not NODE_ENV ... secure:true cookies are only sent by browsers over HTTPS connections. When NODE_ENV=production (always set in the Docker container) but no TLS proxy is in front, the browser receives the cookie on login but refuses to send it on subsequent HTTP requests — causing every authenticated endpoint (/stream, /status, etc.) to return 401. The correct signal is TRUST_PROXY: it is only set when a TLS-terminating reverse proxy is confirmed to be in front. Affects emby_user and csrf_token cookies across login, /csrf refresh, and logout.	2026-05-17 09:42:56 +01:00
gronod	abdd0da306	feat: replace client polling with Server-Sent Events (SSE) ... Server: - poller.js: add pollSubscribers Set with onPollComplete/offPollComplete; notify all SSE callbacks immediately after every successful poll - dashboard.js: add GET /api/dashboard/stream endpoint (text/event-stream) - requireAuth enforced via cookie (no CSRF needed — GET is a safe method) - X-Accel-Buffering: no for nginx proxy compatibility - 25s heartbeat comments to survive proxy idle timeouts - initial payload sent immediately on connect - cleanup on req.close: deregister callback, stop heartbeat, remove client - active client tracking updated: type='sse', connectedAt, no refreshRateMs Frontend: - app.js: replace setInterval/fetchUserDownloads with EventSource - startSSE() opens /api/dashboard/stream; stopSSE() closes it - first incoming message hides loading spinner - showAll toggle re-opens stream with ?showAll=true param - logout calls stopSSE() before POST /api/auth/logout - status panel: fixed 5s refresh, shows SSE clients + connect duration - statusRefreshHandle now always 5s, not tied to old refresh-rate selector - index.html: remove now-unused refresh-rate <select> element Docs: - ARCHITECTURE.md §4.3: update poller description - ARCHITECTURE.md §5: rename to SSE Stream (§5.2) + Download Matching (§5.3) - ARCHITECTURE.md §7: update active client tracking description - ARCHITECTURE.md §9: add /stream endpoint, update /status clients schema - ARCHITECTURE.md §10: update key functions table; replace Auto-Refresh section with Live Push via SSE - class-server.puml: add /stream to dashboard routes; update ClientInfo - component.puml: annotate dashboard with SSE note; update label	2026-05-17 08:35:22 +01:00
gronod	8c829f9651	docs: audit and update all documentation to reflect current codebase ... ARCHITECTURE.md: - Node version: 18+ → 22 (Alpine) - Tech stack: add helmet, express-rate-limit, cookie-parser, testing tools - Directory structure: add server/app.js, verifyCsrf.js, tokenStore.js, sanitizeError.js, tests/, docs/, .gitea/workflows/, vitest.config.js - §4.1: document app.js factory (createApp) vs index.js entry point; CSP nonce, rate limiters, CSRF middleware, trust proxy - §4.2: add CSRF Required column; document verifyCsrf; fix auth note - §4.3: add tokenStore.js and sanitizeError.js descriptions - §6 Auth flow: add rememberMe, rate limiter, stable DeviceId, server-side token store, CSRF token issuance, correct cookie TTL (session/30d not 24h) - §9 API: add csrfToken to login response, rememberMe field, 400/429 codes; add GET /api/auth/csrf endpoint; fix /me response; fix /logout CSRF note - §11 Config: add DATA_DIR, COOKIE_SECRET, TRUST_PROXY, NODE_ENV; split into Core / Emby / Service Instances / Tuning sections - §12 Deployment: update Dockerfile description to multi-stage node:22-alpine; add COOKIE_SECRET, TRUST_PROXY, named volume to compose example; add security hardening checklist; add CI/CD table diagrams/seq-auth.puml: - Add TokenStore participant - Add rememberMe, CSRF token issuance, stable DeviceId note - Add login rate limiter note - Add GET /csrf refresh flow - Add server-side token revocation on logout diagrams/class-server.puml: - Add app.js createApp() factory class - Add verifyCsrf middleware class - Add TokenStore and SanitizeError utility classes - Update auth.js routes (add GET /csrf) - Fix relationships: entry → appfn → routes diagrams/component.puml: - Add app.js factory component - Add helmet, express-rate-limit components - Add verifyCsrf middleware component - Add tokenStore.js and sanitizeError.js utility components - Fix wiring: entry → createApp() → mounts routes Dockerfile: - Fix stale comments referencing better-sqlite3 and SQLite server/routes/auth.js: - Fix stale comment: SQLite-backed → JSON file-backed	2026-05-17 08:05:08 +01:00
gronod	5fd55b4e1a	test: add comprehensive test suite (115 tests, Vitest + supertest + nock) ... Framework: - Vitest v4 as test runner (fast ESM/CJS support, V8 coverage built-in) - supertest for integration tests against createApp() factory - nock for HTTP interception (works with CJS require('axios'), unlike vi.mock) New files: - vitest.config.js — test config: node env, isolate, V8 coverage, per-file thresholds - tests/setup.js — isolated DATA_DIR per worker, SKIP_RATE_LIMIT, console suppression - tests/README.md — approach, structure, design decisions - server/app.js — testable Express factory (extracted from index.js side-effects) Unit tests (91 tests): - tests/unit/sanitizeError.test.js — secret redaction: apikey, token, bearer, basic-auth URLs - tests/unit/config.test.js — JSON array + legacy single-instance config parsing - tests/unit/requireAuth.test.js — valid/invalid/tampered cookies, schema validation - tests/unit/verifyCsrf.test.js — double-submit pattern, timing-safe compare, safe methods - tests/unit/qbittorrent.test.js — formatBytes, formatEta, mapTorrentToDownload state map - tests/unit/tokenStore.test.js — store/get/clear lifecycle, TTL expiry, atomic disk write Integration tests (24 tests): - tests/integration/health.test.js — /health and /ready endpoints - tests/integration/auth.test.js — full login/logout/me/csrf flows, input validation, cookie attributes, no token leakage, Emby mock via nock Production code changes (minimal, no behaviour change): - server/routes/auth.js: EMBY_URL captured at request-time (not module load) for testability - server/routes/auth.js: loginLimiter max → Number.MAX_SAFE_INTEGER when SKIP_RATE_LIMIT set - server/utils/sanitizeError.js: fix HEADER_PATTERN to redact full line (not just first token) CI: - .gitea/workflows/ci.yml: add parallel 'test' job (npm run test:coverage, artifact upload) - package.json: add test/test:watch/test:coverage/test:ui scripts - .gitignore: add coverage/	2026-05-17 07:45:33 +01:00
gronod	cc1e8af761	fix: proxy cover art through server to satisfy CSP img-src 'self' ... The new CSP blocks direct browser requests to external image origins (themoviedb.org, thetvdb.com, etc.) used for poster art. - dashboard.js: add GET /api/dashboard/cover-art?url=... proxy endpoint (auth-required, http/https only, image content-type validated, 5MB cap, 24h Cache-Control, streams response directly to client) - app.js: route coverArt src through /api/dashboard/cover-art proxy - server/utils/logger.js: fix hardcoded /app/server.log path (use DATA_DIR)	2026-05-17 07:24:15 +01:00
gronod	bdbbcabfbc	feat(security): production hardening for external deployment ... Container (Dockerfile): - Multi-stage build (deps + runtime) for minimal attack surface - Upgrade base image from node:18-alpine to node:22-alpine - Run as non-root 'node' user (UID 1000); source files owned by root - /app/data directory owned by node for SQLite + logs - Docker HEALTHCHECK: wget /health every 30s docker-compose.yaml: - Port bound to 127.0.0.1 only (expose via reverse proxy) - read_only: true filesystem; /tmp tmpfs for Node.js - no-new-privileges:true, cap_drop: ALL - Named volume sofarr-data for persistent data - TRUST_PROXY, COOKIE_SECRET, NODE_ENV added Helmet v7 + CSP nonce: - Upgrade helmet@4 → helmet@7, express-rate-limit@6 → @7 - CSP with per-request nonce injected into index.html script/link tags (replaces blanket unsafe-inline; nonce changes every request) - HSTS: max-age=1yr, includeSubDomains, preload - Referrer-Policy: strict-origin-when-cross-origin - Permissions-Policy: camera/mic/geolocation/payment/usb all off - index.html served dynamically with nonce injection; static assets served normally via express.static({index:false}) Trust proxy: - TRUST_PROXY env var configures app.set('trust proxy') so rate limiting and secure cookies work correctly behind Nginx/Caddy Session & auth: - Token store migrated from in-memory Map to SQLite via better-sqlite3 (server/utils/tokenStore.js): survives restarts, WAL mode, 31-day TTL - CSRF double-submit cookie pattern (server/middleware/verifyCsrf.js): POST/PUT/PATCH/DELETE on /api/* require X-CSRF-Token header matching the csrf_token cookie; timing-safe comparison - CSRF token issued on login + GET /api/auth/csrf; cleared on logout - Login input validation: username/password length + type checked before hitting Emby - skipSuccessfulRequests:true on login rate limiter (only count failures) - express.json({ limit: '64kb' }) to reject oversized payloads Rate limiting: - General API limiter: 300 req/15min per IP on all /api/* routes - Login limiter unchanged (10 failures/15min) but now only counts fails Logging: - Log file moved from /app/server.log to DATA_DIR/server.log (writable by non-root node user in container) - Size-based rotation: rotate at 10 MB, keep 3 files (server.log.1-3) - DATA_DIR defaults to ./data locally, /app/data in container Error handling: - Global Express error handler: catches unhandled errors, logs message, returns generic 500 (no stack traces to clients) Health/readiness: - GET /health: returns {status:'ok', uptime:N} — used by HEALTHCHECK - GET /ready: returns 503 if EMBY_URL not configured Error sanitization (sanitizeError.js): - Added patterns for password= params, bearer tokens, Basic auth in URLs Supply chain: - Remove unused cors dependency - add better-sqlite3@^9 - CI: upgrade to Node 22, raise audit level to --audit-level=high - .gitignore: add data/, *.db, *.db-wal, *.db-shm Docs: - SECURITY.md: threat model, hardening checklist, proxy examples, header table, rate limit table, Docker secrets guidance - .env.example + .env.sample: TRUST_PROXY, DATA_DIR documented	2026-05-17 06:47:25 +01:00