Compare commits

...

3 Commits

Author SHA1 Message Date
gronod 498eabc7bc fix(qbittorrent): add seeds/peers fields (num_seeds/num_leechs), guard empty response (closes #64)
Build and Push Docker Image / build (push) Successful in 50s
Licence Check / Licence compatibility and copyright header verification (push) Successful in 1m9s
Docs Check / Markdown lint (push) Successful in 1m19s
CI / Security audit (push) Successful in 2m8s
Docs Check / Mermaid diagram parse check (push) Successful in 2m33s
CI / Swagger Validation & Coverage (push) Successful in 2m40s
CI / Tests & coverage (push) Successful in 2m54s
2026-05-28 15:52:52 +01:00
gronod 6b73727d4e fix(queue): extract shared arr cache helper, annotate season packs, null-guard flatMap (closes #61) 2026-05-28 15:36:33 +01:00
gronod 593ad79670 fix(webhooks): redesign replay key with content identifiers, log instance fallback (closes #62) 2026-05-28 15:30:08 +01:00
7 changed files with 317 additions and 52 deletions
+8
View File
@@ -4,6 +4,14 @@ All notable changes to this project will be documented in this file.
Format follows [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
This project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [1.7.32] - YYYY-MM-DD
### Fixed
- **qBittorrentClient Peer Data & Response Safety (Issue #64)** — `QBittorrentClient.normalizeDownload()` now exposes two new fields on every torrent record: `seeds` (sourced from qBittorrent's `num_seeds`, the count of connected seed peers) and `peers` (sourced from `num_leechs`, the count of connected leecher peers). The connected counts were chosen deliberately over the swarm totals `num_complete`/`num_incomplete` so the values remain consistent with what other clients (Transmission via `peersConnected`/`peersSendingToUs`, rTorrent via `d.peers_connected`) report on the same normalised contract. `QBittorrentClient.getMainData()` now also defensively returns the existing in-memory torrent map (rather than dereferencing a null) when the qBittorrent server responds with an empty body to `/api/v2/sync/maindata`, eliminating a crash class observed against transiently-restarting qBittorrent instances. A regression test verifies the new fields are populated from `num_seeds`/`num_leechs` and not from the swarm-total fields. Resolves Gitea Issue [#64](https://git.i3omb.com/Gandalf/sofarr/issues/64).
- **Season Pack Queue Handling & Crash Prevention (Issue #61)** — Extracted a shared `buildArrQueueCache(queues, instances, mediaKey)` helper at `server/utils/arrQueueHelpers.js` covering both Sonarr and Radarr, replacing four previously-divergent inline `flatMap` blocks across the background poller (`server/utils/poller.js`) and the webhook event processor (`server/routes/webhook.js`) that built the `poll:sonarr-queue` and `poll:radarr-queue` cache entries. Sonarr queue records that share a `downloadId` (the canonical fingerprint for a season-pack release) are now annotated with `isSeasonPack: true` and `episodeCount: <n>` so downstream consumers — including the active-downloads matching service — can identify and de-duplicate season packs without re-deriving the grouping. The helper is wrapped in per-record and per-instance `try`/`catch` guards: malformed records (`null`, missing `data`, unknown instance ids) are skipped with a warning rather than throwing, eliminating a class of crashes that previously bubbled out of the `flatMap` and tore down the entire poll cycle or webhook refresh. Movies (Radarr) skip season-pack annotation by design. A new unit test suite at `tests/unit/utils/arrQueueHelpers.test.js` covers tagging, season-pack grouping, null-safety, and unknown-instance fallback. Resolves Gitea Issue [#61](https://git.i3omb.com/Gandalf/sofarr/issues/61).
- **Webhook Reliability (Issue #62)** — Hardened the webhook replay protection to prevent false-duplicate detection while preserving protection against genuine retries. The replay key for Sonarr and Radarr now incorporates a content identifier (`downloadId`, falling back to `series.id` or `movie.id`) alongside the existing `eventType:instanceName:eventDate` components, so that multiple distinct events sharing the same timestamp (for example, several `Grab` events fired in the same second for episodes in a season pack) no longer collide and get silently dropped. Events without a content identifier (such as `Test`) fall back gracefully to the previous key shape so existing behaviour is preserved. The Ombi handler — which already uses a distinct `requestId`-bearing key — is unchanged. Additionally, the Sonarr and Radarr handlers now log an explicit warning when the inbound `instanceName` fails to match any configured instance and processing falls back to the first instance, improving diagnosability of misconfigured webhook senders. Resolves Gitea Issue [#62](https://git.i3omb.com/Gandalf/sofarr/issues/62).
## [1.7.31] - 2026-05-28
### Fixed
+13
View File
@@ -104,6 +104,11 @@ class QBittorrentClient extends DownloadClient {
const response = await this.makeRequest(`/api/v2/sync/maindata?rid=${this.lastRid}`);
const data = response.data;
if (!data) {
logToFile(`[qBittorrent:${this.name}] Empty response from sync/maindata`);
return Array.from(this.torrentMap.values());
}
if (data.full_update) {
// Full refresh: rebuild the entire map
this.torrentMap.clear();
@@ -249,6 +254,14 @@ class QBittorrentClient extends DownloadClient {
downloaded: downloadedSize,
speed: torrent.dlspeed,
eta: torrent.eta < 0 || torrent.eta === 8640000 ? null : torrent.eta,
// Connected peer counts (Issue #64). qBittorrent exposes:
// num_seeds — connected seeds (peers we have a connection to)
// num_leechs — connected leechers (peers downloading from us)
// num_complete / num_incomplete — *swarm* totals reported by tracker
// We expose the connected counts to stay consistent with what other
// clients (e.g. Transmission via peersConnected/peersSendingToUs) report.
seeds: torrent.num_seeds ?? 0,
peers: torrent.num_leechs ?? 0,
category: torrent.category || undefined,
tags: torrent.tags ? torrent.tags.split(',').filter(tag => tag.trim()) : [],
savePath: torrent.content_path || torrent.save_path || undefined,
+30 -30
View File
@@ -5,6 +5,7 @@ const { logToFile } = require('../utils/logger');
const { getWebhookSecret, getSonarrInstances, getRadarrInstances, getOmbiInstances, getSofarrBaseUrl } = require('../utils/config');
const cache = require('../utils/cache');
const arrRetrieverRegistry = require('../utils/arrRetrievers');
const { buildArrQueueCache } = require('../utils/arrQueueHelpers');
const { pollAllServices, POLL_INTERVAL, POLLING_ENABLED } = require('../utils/poller');
const { extractRequestedUser } = require('../utils/ombiHelpers');
const requireAuth = require('../middleware/requireAuth');
@@ -106,9 +107,14 @@ function pruneReplayCache() {
// Prune the replay cache once per minute
setInterval(pruneReplayCache, 60 * 1000).unref();
function isReplay(eventType, instanceName, eventDate) {
function isReplay(eventType, instanceName, eventDate, contentId) {
if (!eventDate) return false;
const key = `${eventType}:${instanceName || ''}:${eventDate}`;
// Content-aware replay key: incorporates downloadId / series.id / movie.id when
// available so that distinct events sharing the same `date` (e.g. multiple
// Grab events for episodes in a season pack fired in the same second) do not
// falsely collide. Falls back to the prior shape when contentId is absent
// (e.g. Test events) so existing behaviour is preserved.
const key = `${eventType}:${instanceName || ''}:${contentId || ''}:${eventDate}`;
if (recentEvents.has(key)) return true;
recentEvents.set(key, Date.now());
return false;
@@ -202,17 +208,7 @@ async function processWebhookEvent(serviceType, eventType, payload = null) {
const queuesByType = await arrRetrieverRegistry.getQueuesByType();
const sonarrQueues = queuesByType.sonarr || [];
cache.set('poll:sonarr-queue', {
records: sonarrQueues.flatMap(q => {
const inst = sonarrInstances.find(i => i.id === q.instance);
const url = inst ? inst.url : null;
const key = inst ? inst.apiKey : null;
return (q.data.records || []).map(r => {
if (r.series) r.series._instanceUrl = url;
r._instanceUrl = url;
r._instanceKey = key;
return r;
});
})
records: buildArrQueueCache(sonarrQueues, sonarrInstances, 'series')
}, CACHE_TTL);
logToFile(`[Webhook] Refreshed poll:sonarr-queue (${sonarrQueues.length} instance(s))`);
}
@@ -232,17 +228,7 @@ async function processWebhookEvent(serviceType, eventType, payload = null) {
const queuesByType = await arrRetrieverRegistry.getQueuesByType();
const radarrQueues = queuesByType.radarr || [];
cache.set('poll:radarr-queue', {
records: radarrQueues.flatMap(q => {
const inst = radarrInstances.find(i => i.id === q.instance);
const url = inst ? inst.url : null;
const key = inst ? inst.apiKey : null;
return (q.data.records || []).map(r => {
if (r.movie) r.movie._instanceUrl = url;
r._instanceUrl = url;
r._instanceKey = key;
return r;
});
})
records: buildArrQueueCache(radarrQueues, radarrInstances, 'movie')
}, CACHE_TTL);
logToFile(`[Webhook] Refreshed poll:radarr-queue (${radarrQueues.length} instance(s))`);
}
@@ -480,11 +466,18 @@ router.post('/sonarr', webhookLimiter, (req, res) => {
const { eventType, instanceName, eventDate } = validation;
const sonarrInstances = getSonarrInstances();
const inst = sonarrInstances.find(i => i.name === instanceName || i.id === instanceName) || sonarrInstances[0];
const matchedInst = sonarrInstances.find(i => i.name === instanceName || i.id === instanceName);
const inst = matchedInst || sonarrInstances[0];
if (!matchedInst && instanceName) {
logToFile(`[Webhook] Sonarr instanceName "${instanceName}" did not match any configured instance; falling back to first instance (${inst ? inst.name : 'none'})`);
}
const resolvedInstanceName = inst ? inst.name : instanceName;
if (isReplay(eventType, resolvedInstanceName, eventDate)) {
logToFile(`[Webhook] Sonarr duplicate event ignored: ${eventType} @ ${eventDate}`);
// Content-aware replay key components (Issue #62)
const contentId = req.body.downloadId || req.body.series?.id || null;
if (isReplay(eventType, resolvedInstanceName, eventDate, contentId)) {
logToFile(`[Webhook] Sonarr duplicate event ignored: ${eventType} @ ${eventDate} (contentId=${contentId || 'none'})`);
return res.status(200).json({ received: true, duplicate: true });
}
@@ -634,11 +627,18 @@ router.post('/radarr', webhookLimiter, (req, res) => {
const { eventType, instanceName, eventDate } = validation;
const radarrInstances = getRadarrInstances();
const inst = radarrInstances.find(i => i.name === instanceName || i.id === instanceName) || radarrInstances[0];
const matchedInst = radarrInstances.find(i => i.name === instanceName || i.id === instanceName);
const inst = matchedInst || radarrInstances[0];
if (!matchedInst && instanceName) {
logToFile(`[Webhook] Radarr instanceName "${instanceName}" did not match any configured instance; falling back to first instance (${inst ? inst.name : 'none'})`);
}
const resolvedInstanceName = inst ? inst.name : instanceName;
if (isReplay(eventType, resolvedInstanceName, eventDate)) {
logToFile(`[Webhook] Radarr duplicate event ignored: ${eventType} @ ${eventDate}`);
// Content-aware replay key components (Issue #62)
const contentId = req.body.downloadId || req.body.movie?.id || null;
if (isReplay(eventType, resolvedInstanceName, eventDate, contentId)) {
logToFile(`[Webhook] Radarr duplicate event ignored: ${eventType} @ ${eventDate} (contentId=${contentId || 'none'})`);
return res.status(200).json({ received: true, duplicate: true });
}
+97
View File
@@ -0,0 +1,97 @@
// Copyright (c) 2026 Gordon Bolton. MIT License.
//
// Shared helpers for assembling the cached *arr queue payload.
//
// Both the background poller (`server/utils/poller.js`) and the webhook
// processor (`server/routes/webhook.js`) build the `poll:sonarr-queue` and
// `poll:radarr-queue` cache entries from an array of per-instance queue
// responses. Historically the same `flatMap` block was duplicated across all
// four call sites (Sonarr + Radarr × poller + webhook) and had begun to drift.
//
// This module centralises that logic, adds defensive null-guards, and — for
// Sonarr only — annotates season-pack records (queue entries sharing a
// `downloadId`) with `isSeasonPack` and `episodeCount`. See Issue #61.
//
const { logToFile } = require('./logger');
/**
* Build the flattened, instance-tagged `records` array for the
* `poll:sonarr-queue` / `poll:radarr-queue` cache entry.
*
* @param {Array<{ instance: string, data: { records?: Array<object> } }>} queues
* Per-instance queue responses as returned by
* `arrRetrieverRegistry.getQueuesByType()` (or the equivalent batched
* retrieval in the poller).
* @param {Array<{ id: string, url: string, apiKey: string, name?: string }>} instances
* Configured instances; used to resolve `_instanceUrl` / `_instanceKey`.
* @param {'series'|'movie'} mediaKey
* Sonarr records embed a `series` object; Radarr records embed a `movie`
* object. The embedded object is annotated with `_instanceUrl` so that
* downstream link builders work.
* @returns {Array<object>} The flattened, annotated records array.
*/
function buildArrQueueCache(queues, instances, mediaKey) {
if (!Array.isArray(queues) || queues.length === 0) return [];
if (mediaKey !== 'series' && mediaKey !== 'movie') {
logToFile(`[arrQueueHelpers] Invalid mediaKey "${mediaKey}"; expected 'series' or 'movie'`);
return [];
}
const safeInstances = Array.isArray(instances) ? instances : [];
const out = [];
for (const q of queues) {
try {
if (!q || !q.data) continue;
const inst = safeInstances.find(i => i.id === q.instance);
const url = inst ? inst.url : null;
const key = inst ? inst.apiKey : null;
const records = Array.isArray(q.data.records) ? q.data.records : [];
for (const r of records) {
try {
if (!r) continue;
if (r[mediaKey]) {
r[mediaKey]._instanceUrl = url;
}
r._instanceUrl = url;
r._instanceKey = key;
out.push(r);
} catch (perRecordErr) {
logToFile(`[arrQueueHelpers] Skipping malformed ${mediaKey} record: ${perRecordErr.message}`);
}
}
} catch (perInstanceErr) {
logToFile(`[arrQueueHelpers] Skipping malformed ${mediaKey} queue payload: ${perInstanceErr.message}`);
}
}
// Sonarr-only: season pack annotation. Group by downloadId; entries that
// share a downloadId are episodes belonging to the same release (a season
// pack). Movies (mediaKey === 'movie') are single-record by nature.
if (mediaKey === 'series') {
try {
const groups = new Map();
for (const r of out) {
const dlId = r && r.downloadId;
if (!dlId) continue;
if (!groups.has(dlId)) groups.set(dlId, []);
groups.get(dlId).push(r);
}
for (const group of groups.values()) {
if (group.length > 1) {
for (const r of group) {
r.isSeasonPack = true;
r.episodeCount = group.length;
}
}
}
} catch (annotateErr) {
logToFile(`[arrQueueHelpers] Season-pack annotation failed: ${annotateErr.message}`);
}
}
return out;
}
module.exports = {
buildArrQueueCache
};
+3 -22
View File
@@ -3,6 +3,7 @@ const axios = require('axios');
const cache = require('./cache');
const { initializeClients, getAllDownloads, getDownloadsByClientType } = require('./downloadClients');
const arrRetrieverRegistry = require('./arrRetrievers');
const { buildArrQueueCache } = require('./arrQueueHelpers');
const {
getSonarrInstances,
getRadarrInstances,
@@ -237,17 +238,7 @@ async function pollAllServices() {
cache.set('poll:sonarr-tags', sonarrTagsResults, cacheTTL);
// Tag queue/history records with _instanceUrl so embedded series/movie objects can build links
cache.set('poll:sonarr-queue', {
records: sonarrQueues.flatMap(q => {
const inst = sonarrInstances.find(i => i.id === q.instance);
const url = inst ? inst.url : null;
const key = inst ? inst.apiKey : null;
return (q.data.records || []).map(r => {
if (r.series) r.series._instanceUrl = url;
r._instanceUrl = url;
r._instanceKey = key;
return r;
});
})
records: buildArrQueueCache(sonarrQueues, sonarrInstances, 'series')
}, cacheTTL);
cache.set('poll:sonarr-history', {
records: sonarrHistories.flatMap(h => h.data.records || [])
@@ -265,17 +256,7 @@ async function pollAllServices() {
// Radarr
if (shouldPollRadarr) {
cache.set('poll:radarr-queue', {
records: radarrQueues.flatMap(q => {
const inst = radarrInstances.find(i => i.id === q.instance);
const url = inst ? inst.url : null;
const key = inst ? inst.apiKey : null;
return (q.data.records || []).map(r => {
if (r.movie) r.movie._instanceUrl = url;
r._instanceUrl = url;
r._instanceKey = key;
return r;
});
})
records: buildArrQueueCache(radarrQueues, radarrInstances, 'movie')
}, cacheTTL);
cache.set('poll:radarr-history', {
records: radarrHistories.flatMap(h => h.data.records || [])
@@ -130,6 +130,8 @@ describe('QBittorrentClient', () => {
downloaded: 750000000,
speed: 1048576,
eta: 3600,
seeds: 0,
peers: 0,
category: 'movies',
tags: ['movie', 'hd'],
savePath: '/downloads/test',
@@ -138,6 +140,28 @@ describe('QBittorrentClient', () => {
});
});
it('should expose connected seeds/peers from num_seeds and num_leechs (Issue #64)', () => {
const torrent = {
hash: 'def456',
name: 'Swarm Torrent',
state: 'downloading',
progress: 0.1,
size: 1000,
completed: 100,
dlspeed: 0,
eta: 0,
num_seeds: 7,
num_leechs: 3,
// Swarm totals — must NOT be picked up as connected counts
num_complete: 200,
num_incomplete: 50
};
const normalized = client.normalizeDownload(torrent);
expect(normalized.seeds).toBe(7);
expect(normalized.peers).toBe(3);
});
it('should handle unknown torrent states', () => {
const torrent = {
hash: 'abc123',
+142
View File
@@ -0,0 +1,142 @@
// Copyright (c) 2026 Gordon Bolton. MIT License.
// Tests for the shared `buildArrQueueCache` helper (Issue #61).
import { describe, it, expect } from 'vitest';
import { createRequire } from 'module';
const require = createRequire(import.meta.url);
const { buildArrQueueCache } = require('../../../server/utils/arrQueueHelpers');
const sonarrInstances = [
{ id: 's1', url: 'http://sonarr-1', apiKey: 'KEY_S1', name: 'Sonarr 1' }
];
const radarrInstances = [
{ id: 'r1', url: 'http://radarr-1', apiKey: 'KEY_R1', name: 'Radarr 1' }
];
describe('buildArrQueueCache', () => {
it('returns empty array for empty / missing input', () => {
expect(buildArrQueueCache([], sonarrInstances, 'series')).toEqual([]);
expect(buildArrQueueCache(null, sonarrInstances, 'series')).toEqual([]);
expect(buildArrQueueCache(undefined, sonarrInstances, 'series')).toEqual([]);
});
it('returns empty array for invalid mediaKey', () => {
const queues = [{ instance: 's1', data: { records: [{ id: 1 }] } }];
expect(buildArrQueueCache(queues, sonarrInstances, 'bogus')).toEqual([]);
});
it('tags Sonarr records with _instanceUrl/_instanceKey and decorates embedded series', () => {
const queues = [
{
instance: 's1',
data: {
records: [
{ id: 1, downloadId: 'dl-1', series: { id: 100, title: 'X' } }
]
}
}
];
const out = buildArrQueueCache(queues, sonarrInstances, 'series');
expect(out).toHaveLength(1);
expect(out[0]._instanceUrl).toBe('http://sonarr-1');
expect(out[0]._instanceKey).toBe('KEY_S1');
expect(out[0].series._instanceUrl).toBe('http://sonarr-1');
});
it('tags Radarr records and decorates embedded movie', () => {
const queues = [
{
instance: 'r1',
data: {
records: [
{ id: 11, downloadId: 'dl-r1', movie: { id: 555, title: 'M' } }
]
}
}
];
const out = buildArrQueueCache(queues, radarrInstances, 'movie');
expect(out).toHaveLength(1);
expect(out[0]._instanceUrl).toBe('http://radarr-1');
expect(out[0]._instanceKey).toBe('KEY_R1');
expect(out[0].movie._instanceUrl).toBe('http://radarr-1');
});
it('annotates Sonarr season pack records (multiple entries sharing downloadId)', () => {
const queues = [
{
instance: 's1',
data: {
records: [
{ id: 1, downloadId: 'pack-A', episodeId: 101 },
{ id: 2, downloadId: 'pack-A', episodeId: 102 },
{ id: 3, downloadId: 'pack-A', episodeId: 103 },
{ id: 4, downloadId: 'single-B', episodeId: 200 }
]
}
}
];
const out = buildArrQueueCache(queues, sonarrInstances, 'series');
expect(out).toHaveLength(4);
const packMembers = out.filter(r => r.downloadId === 'pack-A');
expect(packMembers).toHaveLength(3);
for (const r of packMembers) {
expect(r.isSeasonPack).toBe(true);
expect(r.episodeCount).toBe(3);
}
const single = out.find(r => r.downloadId === 'single-B');
expect(single.isSeasonPack).toBeUndefined();
expect(single.episodeCount).toBeUndefined();
});
it('does not annotate Radarr records as season packs even if downloadId repeats', () => {
const queues = [
{
instance: 'r1',
data: {
records: [
{ id: 1, downloadId: 'dup', movie: { id: 1 } },
{ id: 2, downloadId: 'dup', movie: { id: 2 } }
]
}
}
];
const out = buildArrQueueCache(queues, radarrInstances, 'movie');
expect(out).toHaveLength(2);
for (const r of out) {
expect(r.isSeasonPack).toBeUndefined();
expect(r.episodeCount).toBeUndefined();
}
});
it('skips malformed records and continues', () => {
const queues = [
{
instance: 's1',
data: {
records: [
null,
{ id: 1, downloadId: 'ok', series: { id: 1 } }
]
}
},
null,
{ instance: 's1' } // no data property
];
const out = buildArrQueueCache(queues, sonarrInstances, 'series');
expect(out).toHaveLength(1);
expect(out[0].id).toBe(1);
});
it('handles unknown instance id gracefully (null url/key)', () => {
const queues = [
{
instance: 'unknown-instance',
data: { records: [{ id: 1, downloadId: 'x' }] }
}
];
const out = buildArrQueueCache(queues, sonarrInstances, 'series');
expect(out).toHaveLength(1);
expect(out[0]._instanceUrl).toBeNull();
expect(out[0]._instanceKey).toBeNull();
});
});