Commit Graph

22 Commits

Author SHA1 Message Date
ddcfbda0c2 feat(history): add /api/history/recent endpoint with Sonarr/Radarr history fetching, tag filtering, and 5-min cache 2026-05-17 12:05:30 +01:00
4af36fc926 fix: correct status panel cache stats and static asset caching
Some checks failed
Build and Push Docker Image / build (push) Successful in 23s
CI / Security audit (push) Successful in 42s
CI / Tests & coverage (push) Failing after 45s
cache.js: Map values serialise as '{}' under JSON.stringify, causing
emby:users to show 0 bytes and null item count in the status panel.
Convert Maps via Object.fromEntries before stringifying, and report
Map.size as itemCount.

index.js: JS and CSS served with Cache-Control: no-cache so browsers
always revalidate on load. ETag still prevents re-downloading unchanged
files — only a new deploy triggers an actual download.
2026-05-17 08:46:55 +01:00
abdd0da306 feat: replace client polling with Server-Sent Events (SSE)
Some checks failed
Build and Push Docker Image / build (push) Successful in 23s
CI / Security audit (push) Successful in 38s
CI / Tests & coverage (push) Failing after 38s
Server:
- poller.js: add pollSubscribers Set with onPollComplete/offPollComplete;
  notify all SSE callbacks immediately after every successful poll
- dashboard.js: add GET /api/dashboard/stream endpoint (text/event-stream)
  - requireAuth enforced via cookie (no CSRF needed — GET is a safe method)
  - X-Accel-Buffering: no for nginx proxy compatibility
  - 25s heartbeat comments to survive proxy idle timeouts
  - initial payload sent immediately on connect
  - cleanup on req.close: deregister callback, stop heartbeat, remove client
  - active client tracking updated: type='sse', connectedAt, no refreshRateMs

Frontend:
- app.js: replace setInterval/fetchUserDownloads with EventSource
  - startSSE() opens /api/dashboard/stream; stopSSE() closes it
  - first incoming message hides loading spinner
  - showAll toggle re-opens stream with ?showAll=true param
  - logout calls stopSSE() before POST /api/auth/logout
  - status panel: fixed 5s refresh, shows SSE clients + connect duration
  - statusRefreshHandle now always 5s, not tied to old refresh-rate selector
- index.html: remove now-unused refresh-rate <select> element

Docs:
- ARCHITECTURE.md §4.3: update poller description
- ARCHITECTURE.md §5: rename to SSE Stream (§5.2) + Download Matching (§5.3)
- ARCHITECTURE.md §7: update active client tracking description
- ARCHITECTURE.md §9: add /stream endpoint, update /status clients schema
- ARCHITECTURE.md §10: update key functions table; replace Auto-Refresh
  section with Live Push via SSE
- class-server.puml: add /stream to dashboard routes; update ClientInfo
- component.puml: annotate dashboard with SSE note; update label
2026-05-17 08:35:22 +01:00
5fd55b4e1a test: add comprehensive test suite (115 tests, Vitest + supertest + nock)
Some checks failed
Build and Push Docker Image / build (push) Successful in 49s
CI / Security audit (push) Successful in 1m23s
CI / Tests & coverage (push) Failing after 2m13s
Framework:
- Vitest v4 as test runner (fast ESM/CJS support, V8 coverage built-in)
- supertest for integration tests against createApp() factory
- nock for HTTP interception (works with CJS require('axios'), unlike vi.mock)

New files:
- vitest.config.js          — test config: node env, isolate, V8 coverage, per-file thresholds
- tests/setup.js             — isolated DATA_DIR per worker, SKIP_RATE_LIMIT, console suppression
- tests/README.md            — approach, structure, design decisions
- server/app.js              — testable Express factory (extracted from index.js side-effects)

Unit tests (91 tests):
- tests/unit/sanitizeError.test.js  — secret redaction: apikey, token, bearer, basic-auth URLs
- tests/unit/config.test.js         — JSON array + legacy single-instance config parsing
- tests/unit/requireAuth.test.js    — valid/invalid/tampered cookies, schema validation
- tests/unit/verifyCsrf.test.js     — double-submit pattern, timing-safe compare, safe methods
- tests/unit/qbittorrent.test.js    — formatBytes, formatEta, mapTorrentToDownload state map
- tests/unit/tokenStore.test.js     — store/get/clear lifecycle, TTL expiry, atomic disk write

Integration tests (24 tests):
- tests/integration/health.test.js  — /health and /ready endpoints
- tests/integration/auth.test.js    — full login/logout/me/csrf flows, input validation,
                                      cookie attributes, no token leakage, Emby mock via nock

Production code changes (minimal, no behaviour change):
- server/routes/auth.js: EMBY_URL captured at request-time (not module load) for testability
- server/routes/auth.js: loginLimiter max → Number.MAX_SAFE_INTEGER when SKIP_RATE_LIMIT set
- server/utils/sanitizeError.js: fix HEADER_PATTERN to redact full line (not just first token)

CI:
- .gitea/workflows/ci.yml: add parallel 'test' job (npm run test:coverage, artifact upload)
- package.json: add test/test:watch/test:coverage/test:ui scripts
- .gitignore: add coverage/
2026-05-17 07:45:33 +01:00
251c7376c9 fix: logger.js hardcoded server.log path breaks non-root container user
All checks were successful
Build and Push Docker Image / build (push) Successful in 24s
CI / Security audit (push) Successful in 26s
server/utils/logger.js was still writing to ../../server.log relative
to __dirname (/app/server.log) which is root-owned. The non-root node
user (UID 1000) cannot write there, causing an EACCES crash on startup.

Fix: use DATA_DIR env var (same as index.js) so all log writes go to
/app/data/server.log which is owned by the node user.
2026-05-17 07:21:43 +01:00
37c1b64982 fix(docker): replace better-sqlite3 with pure-JS JSON token store
All checks were successful
Build and Push Docker Image / build (push) Successful in 28s
CI / Security audit (push) Successful in 38s
better-sqlite3 is a native C++ addon that requires compilation on Alpine
(musl libc, no pre-built binaries exist) and fails on Debian slim too
because prebuild-install cannot detect the libc type correctly.

Replace with a pure-JS JSON file token store (server/utils/tokenStore.js):
- Atomic writes via temp file + rename (no corruption on crash)
- Same API: storeToken/getToken/clearToken
- TTL enforcement on read and hourly prune
- Zero native code, zero build tools required

Dockerfile:
- Revert to node:22-alpine (was node:22-slim)
- Remove build tools (python3/make/g++) — no longer needed
- Restore wget HEALTHCHECK (available in Alpine busybox)

docker-compose.yaml: restore wget healthcheck

package.json: remove better-sqlite3 dependency
2026-05-17 07:13:56 +01:00
bdbbcabfbc feat(security): production hardening for external deployment
All checks were successful
Build and Push Docker Image / build (push) Successful in 1m2s
CI / Security audit (push) Successful in 3m29s
Container (Dockerfile):
- Multi-stage build (deps + runtime) for minimal attack surface
- Upgrade base image from node:18-alpine to node:22-alpine
- Run as non-root 'node' user (UID 1000); source files owned by root
- /app/data directory owned by node for SQLite + logs
- Docker HEALTHCHECK: wget /health every 30s

docker-compose.yaml:
- Port bound to 127.0.0.1 only (expose via reverse proxy)
- read_only: true filesystem; /tmp tmpfs for Node.js
- no-new-privileges:true, cap_drop: ALL
- Named volume sofarr-data for persistent data
- TRUST_PROXY, COOKIE_SECRET, NODE_ENV added

Helmet v7 + CSP nonce:
- Upgrade helmet@4 → helmet@7, express-rate-limit@6 → @7
- CSP with per-request nonce injected into index.html script/link tags
  (replaces blanket unsafe-inline; nonce changes every request)
- HSTS: max-age=1yr, includeSubDomains, preload
- Referrer-Policy: strict-origin-when-cross-origin
- Permissions-Policy: camera/mic/geolocation/payment/usb all off
- index.html served dynamically with nonce injection; static assets
  served normally via express.static({index:false})

Trust proxy:
- TRUST_PROXY env var configures app.set('trust proxy') so rate
  limiting and secure cookies work correctly behind Nginx/Caddy

Session & auth:
- Token store migrated from in-memory Map to SQLite via better-sqlite3
  (server/utils/tokenStore.js): survives restarts, WAL mode, 31-day TTL
- CSRF double-submit cookie pattern (server/middleware/verifyCsrf.js):
  POST/PUT/PATCH/DELETE on /api/* require X-CSRF-Token header matching
  the csrf_token cookie; timing-safe comparison
- CSRF token issued on login + GET /api/auth/csrf; cleared on logout
- Login input validation: username/password length + type checked before
  hitting Emby
- skipSuccessfulRequests:true on login rate limiter (only count failures)
- express.json({ limit: '64kb' }) to reject oversized payloads

Rate limiting:
- General API limiter: 300 req/15min per IP on all /api/* routes
- Login limiter unchanged (10 failures/15min) but now only counts fails

Logging:
- Log file moved from /app/server.log to DATA_DIR/server.log (writable
  by non-root node user in container)
- Size-based rotation: rotate at 10 MB, keep 3 files (server.log.1-3)
- DATA_DIR defaults to ./data locally, /app/data in container

Error handling:
- Global Express error handler: catches unhandled errors, logs message,
  returns generic 500 (no stack traces to clients)

Health/readiness:
- GET /health: returns {status:'ok', uptime:N} — used by HEALTHCHECK
- GET /ready: returns 503 if EMBY_URL not configured

Error sanitization (sanitizeError.js):
- Added patterns for password= params, bearer tokens, Basic auth in URLs

Supply chain:
- Remove unused cors dependency
- add better-sqlite3@^9
- CI: upgrade to Node 22, raise audit level to --audit-level=high
- .gitignore: add data/, *.db, *.db-wal, *.db-shm

Docs:
- SECURITY.md: threat model, hardening checklist, proxy examples,
  header table, rate limit table, Docker secrets guidance
- .env.example + .env.sample: TRUST_PROXY, DATA_DIR documented
2026-05-17 06:47:25 +01:00
8fa20c6990 fix(security #10): sanitize error details to prevent API key leakage
Added server/utils/sanitizeError.js which redacts:
- ?apikey= query parameters (SABnzbd passes key in URL)
- ?token= query parameters
- X-Api-Key / X-MediaBrowser-Token / X-Emby-Authorization header
  values if they appear in the error message string

Applied to all catch blocks in emby.js, sabnzbd.js, sonarr.js,
radarr.js, and dashboard.js. Internal error.message still logged
server-side (unredacted) for debugging.
2026-05-16 16:22:11 +01:00
0c8d5d8a4a Revert "perf: split into fast poll + slow-cached library fetches"
This reverts commit 78a8737f29.
2026-05-16 00:32:16 +01:00
268238215e perf: split into fast poll + slow-cached library fetches
Fast poll (every cycle): SABnzbd, Sonarr/Radarr queue + history,
qBittorrent — all lightweight with no include* params.

Slow cache (5 min TTL): Sonarr series, Radarr movies, tags —
fetched only when cache expires. These rarely change.

This eliminates the 2s+ includeSeries/includeMovie joins from
every poll cycle. First poll is still slow (cold cache), but
subsequent polls should complete in <500ms.
2026-05-16 00:32:16 +01:00
31ff973eff perf: drop includeSeries/includeMovie from history fetches
Sonarr/Radarr history with include* params forces expensive DB
joins (~2s each). History records still have seriesId/movieId
for matching; the series/movie objects come from the queue-built
maps instead.

Trade-off: completed downloads only show if the series/movie
is also currently in the queue. Active downloads unaffected.
2026-05-16 00:32:16 +01:00
1327c8e466 perf: reduce history page sizes and drop includeEpisode
- Sonarr/Radarr history: pageSize 20 -> 10
- SABnzbd history: limit 20 -> 10
- Drop includeEpisode from Sonarr queue and history (never rendered)
- These reduce DB join overhead and response payload size
2026-05-16 00:32:16 +01:00
d50a6fe19c perf: eliminate full Sonarr Series + Radarr Movies library fetches
The poller was fetching the entire series and movie libraries on every
poll cycle (~9s each). Queue and history records already embed the full
series/movie object via includeSeries/includeMovie params.

Changes:
- Remove 'Sonarr Series' and 'Radarr Movies' timed fetches from poller
- Tag queue/history records with _instanceUrl in the poller instead
- Build seriesMap/moviesMap from embedded objects in dashboard
- Remove poll:sonarr-series and poll:radarr-movies cache keys
- Fix missing axios and config imports in dashboard
- Net result: ~18s saved per poll cycle, ~2 fewer API calls
2026-05-16 00:32:16 +01:00
57e1db18e2 feat: live-updating status panel with per-task poll timings
- Each service fetch is individually timed (SABnzbd, Sonarr, Radarr, qBit)
- Status panel shows timing bar chart with ms per task and total
- Shows 'Last Poll' age that updates live
- Shows client refresh rate (1s/5s/10s/Off)
- Status panel auto-refreshes in sync with dashboard refresh cycle
- Changing refresh rate restarts the status panel refresh too
- TTL counters update live on each refresh
2026-05-16 00:32:16 +01:00
c03e4620ea feat: add admin-only status page with cache stats
- New /api/dashboard/status endpoint (admin-only, 403 for non-admins)
- Returns server info (uptime, Node version, memory usage)
- Returns polling mode and interval
- Returns cache stats: entry count, total size, per-key breakdown
  with item count, size in KB, and TTL remaining
- Status button in admin controls header
- Collapsible status panel with grid layout
- Responsive: single column on mobile
2026-05-16 00:32:16 +01:00
85bac5994e feat: make background polling disablable with on-demand fallback
- Set POLL_INTERVAL=0, off, false, or disabled to disable background polling
- When disabled, data is fetched on-demand when a user opens the dashboard
- On-demand results cached for 30s so other users benefit from fresh data
- A user with a faster refresh rate keeps the cache warm for everyone
- When polling is enabled, behaviour is unchanged (default 5s)
2026-05-16 00:32:16 +01:00
f28d94d9a3 perf: background poller for near-instant dashboard responses
- New poller.js polls all services on a configurable interval
- POLL_INTERVAL env var (default 5000ms / 5 seconds)
- All data stored in cache with TTL of 3x poll interval
- Dashboard endpoint now reads from cache only (no network calls)
- API responses are near-instant regardless of service count
- First poll runs immediately on server start
2026-05-16 00:32:16 +01:00
b48332f075 perf: persist qBittorrent clients between requests
- Reuse client instances so auth cookies survive across requests
- Eliminates redundant login round-trips on every dashboard refresh
- Clients still re-authenticate automatically if session expires (403)
2026-05-16 00:32:16 +01:00
3edc98b8d6 perf: cache slow-changing data (series, movies, tags) with 60s TTL
- Add MemoryCache utility with get/set/invalidate/clear
- Cache Sonarr series, Sonarr tags, Radarr movies, Radarr tags
- 60-second TTL - first request fetches, subsequent requests served from cache
- Queue, history, and torrent data remain uncached (changes frequently)
- On cache hit, these 4 heavy API calls resolve instantly
2026-05-16 00:32:16 +01:00
d09b0ab40a fix: show full download path (content_path) for qBittorrent
- Prefer content_path over save_path for qBittorrent torrents
- content_path is the full path to the single file or top-level
  folder for multi-file torrents
- save_path is just the base download directory
2026-05-16 00:32:16 +01:00
b75cd18580 feat: show download/target paths for admin users
- Admin users see download path (SABnzbd storage / qBittorrent save_path)
- Admin users see target path (Sonarr series folder / Radarr movie folder)
- Paths displayed in monospace font at bottom of card details
- Non-admin users unaffected (paths not sent in API response)
2026-05-16 00:32:16 +01:00
f500f4db3b feat: fix download-to-user matching, add cover art to downloads
- Fix seriesMap key (use Sonarr internal id, not tvdbId)
- Fix Sonarr tag resolution (use tag map like Radarr)
- Use sourceTitle for history record matching
- Fall back to embedded movie/series objects when API timeouts
- Add includeMovie/includeSeries params to queue/history API calls
- Add coverArt field to all download responses (TMDB poster URLs)
- Add cover art display to frontend download cards
- Fix user-summary route to use instance config and tag maps
2026-05-15 14:54:21 +01:00