sofarr/README.md

# sofarr

> *See your downloads "so far" while you relax on the sofa waiting for your *arr services to finish*

**sofarr** is a personal media download dashboard that aggregates and displays real-time download progress from all your media automation services. Named for the experience of checking what has downloaded "so far" while you wait comfortably on your "sofa" for Sonarr, Radarr, and your download clients to do their thing!

Version 1.4.x adds **real-time webhook integration**: Sonarr and Radarr can push events directly to sofarr, eliminating polling latency and automatically reducing background API calls when webhooks are active.

## What It Does

sofarr connects to your media stack and shows you a personalized view of:
- **Active Downloads** - See what's currently downloading from Usenet (SABnzbd) and BitTorrent (qBittorrent, Transmission, rTorrent)
- **Progress Tracking** - Real-time progress bars with speed, ETA, and completion estimates
- **Recently Completed** - History tab showing imported and failed downloads from Sonarr/Radarr with deduplication and upgrade-awareness
- **User Matching** - Downloads are matched to you based on tags in Sonarr/Radarr
- **Multi-Instance Support** - Connect to multiple instances of each service
- **Webhook Push Updates** - Sonarr/Radarr push events instantly to sofarr; dashboard updates in < 1s after a grab or import
- **Smart Polling** - Background polling automatically reduces (or skips) API calls for instances with active webhooks, with configurable fallback

## How It Works

### Architecture Overview

```
┌─────────────┐     ┌──────────────────────────────────────────────┐
│   Browser   │────▶│               sofarr Server                 │
│   (User)    │◀────│  Auth · Dashboard · History · Webhooks       │
└─────────────┘     │                                              │
   SSE push ◀───────│  Poller (smart: skips when webhooks active)  │
                    │  Cache · PDCA Download Registry · PALDRA     │
                    └───┬─────────────────────────┬────────────────┘
                        │ polls (background)       │ receives webhooks
                        ▼                          │
         ┌──────────────────────────┐    ┌─────────▼───────────────────┐
         │  Download Clients        │    │  *arr Services              │
         │  SABnzbd (Usenet)        │    │  Sonarr  (TV management)    │
         │  qBittorrent (Torrent)   │◀───│  Radarr  (Movie management) │
         │  Transmission (Torrent)  │    └─────────────────────────────┘
         │  rTorrent (Torrent)      │
         └──────────────────────────┘
                       │
              Emby / Jellyfin
              (User authentication)
```

**Three pluggable layers power sofarr:**

| Layer | Name | What it does |
|-------|------|--------------|
| Download clients | **PDCA** (Pluggable Download Client Architecture) | Normalised interface for SABnzbd, qBittorrent, Transmission, rTorrent; easy to add new clients |
| *arr data retrieval | **PALDRA** (Pluggable *Arr Library Data Retrieval Architecture) | Unified fetch layer for Sonarr/Radarr queue, history, and tags across multiple instances |
| Real-time push | **Webhook receiver** | Sonarr/Radarr POST events to sofarr; cache updated immediately; SSE pushes to browser in < 1s |

### Webhooks

When webhooks are configured, sofarr receives instant push notifications from Sonarr and Radarr whenever a download is grabbed, imported, failed, or renamed. The dashboard updates in under a second — no polling delay.

**Quick setup:**
1. Set `SOFARR_WEBHOOK_SECRET` and `SOFARR_BASE_URL` in your `.env`
2. Open the sofarr dashboard → **Webhooks Configuration** panel
3. Click **Enable** next to each Sonarr/Radarr instance
4. sofarr auto-configures the notification connection inside each *arr service

**Smart polling fallback:** Once webhooks are active, the background poller automatically skips queue/history API calls for those instances. If no webhook events arrive for `WEBHOOK_FALLBACK_TIMEOUT` minutes (default: 10), the poller resumes a full poll automatically.

**Webhook endpoints** (no user authentication required — protected by `X-Sofarr-Webhook-Secret`):
- `POST /api/webhook/sonarr` — receives Sonarr events
- `POST /api/webhook/radarr` — receives Radarr events

### The Matching Process

1. **User Authentication**: Login via Emby credentials
2. **Tag-Based Matching**:
   - Your media in Sonarr/Radarr is tagged with your username (e.g., "gordon")
   - sofarr checks Sonarr/Radarr activity to find items tagged with your name
   - Downloads (from SABnzbd, qBittorrent, Transmission, or rTorrent) are matched by title to that activity
   - Only your downloads appear on your dashboard

### Multi-Instance Support

sofarr supports multiple instances of each service via JSON array configuration:

```bash
# Single line JSON arrays in .env
QBITTORRENT_INSTANCES=[{"name":"server1","url":"..."},{"name":"server2","url":"..."}]
SONARR_INSTANCES=[{"name":"main","url":"...","apiKey":"..."}]
```

## Prerequisites

- **Docker** (recommended), or Node.js (v22+) for manual installation
- At least one download client: SABnzbd, qBittorrent, Transmission, or rTorrent
- Sonarr (optional, for TV tracking)
- Radarr (optional, for movie tracking)
- Emby (for user authentication)

## Docker Deployment (Recommended)

### Quick Start

```bash
docker run -d \
  --name sofarr \
  --restart unless-stopped \
  -p 3001:3001 \
  -v /path/to/your/.env:/app/.env \
  docker.i3omb.com/sofarr:latest
```

### Step-by-Step

1. **Create your environment file**:
```bash
mkdir -p /opt/sofarr
curl -o /opt/sofarr/.env https://git.i3omb.com/Gandalf/sofarr/raw/branch/main/.env.sample
# Edit /opt/sofarr/.env with your service details
nano /opt/sofarr/.env
```

2. **Run the container**:
```bash
docker run -d \
  --name sofarr \
  --restart unless-stopped \
  -p 3001:3001 \
  -v /opt/sofarr/.env:/app/.env \
  docker.i3omb.com/sofarr:latest
```

3. **Access the dashboard** at `http://your-server:3001`

### Using Environment Variables (Alternative to .env file)

All configuration can be passed directly as environment variables instead of mounting a `.env` file. This is the preferred approach for orchestrated deployments (Docker Compose, Kubernetes, Portainer, etc).

```bash
docker run -d \
  --name sofarr \
  --restart unless-stopped \
  -p 3001:3001 \
  -e EMBY_URL=http://emby.local:8096 \
  -e EMBY_API_KEY=your-emby-api-key \
  -e SONARR_INSTANCES='[{"name":"main","url":"http://sonarr:8989","apiKey":"your-key"}]' \
  -e RADARR_INSTANCES='[{"name":"main","url":"http://radarr:7878","apiKey":"your-key"}]' \
  -e SABNZBD_INSTANCES='[{"name":"main","url":"http://sabnzbd:8080","apiKey":"your-key"}]' \
  -e QBITTORRENT_INSTANCES='[{"name":"main","url":"http://qbit:8080","username":"admin","password":"pass"}]' \
  -e TRANSMISSION_INSTANCES='[{"name":"main","url":"http://transmission:9091","username":"admin","password":"pass"}]' \
  -e RTORRENT_INSTANCES='[{"name":"main","url":"http://rtorrent:8080/RPC2","username":"rtorrent","password":"rtorrent"}]' \
  -e LOG_LEVEL=info \
  -e POLL_INTERVAL=5000 \
  docker.i3omb.com/sofarr:latest
```

### Docker Compose

```yaml
version: "3"
services:
  sofarr:
    image: docker.i3omb.com/sofarr:latest
    container_name: sofarr
    restart: unless-stopped
    ports:
      - "3001:3001"
    environment:
      - EMBY_URL=http://emby:8096
      - EMBY_API_KEY=your-emby-api-key
      - SONARR_INSTANCES=[{"name":"main","url":"http://sonarr:8989","apiKey":"your-key"}]
      - RADARR_INSTANCES=[{"name":"main","url":"http://radarr:7878","apiKey":"your-key"}]
      - SABNZBD_INSTANCES=[{"name":"main","url":"http://sabnzbd:8080","apiKey":"your-key"}]
      - QBITTORRENT_INSTANCES=[{"name":"main","url":"http://qbit:8080","username":"admin","password":"pass"}]
      - TRANSMISSION_INSTANCES=[{"name":"main","url":"http://transmission:9091","username":"admin","password":"pass"}]
      - RTORRENT_INSTANCES=[{"name":"main","url":"http://rtorrent:8080/RPC2","username":"rtorrent","password":"rtorrent"}]
      - LOG_LEVEL=info
      - POLL_INTERVAL=5000
```

> **Tip:** You can also use a combination — mount a `.env` file for base config, and override specific values with `-e` flags. Environment variables always take precedence.

### Available Tags

| Tag | Description |
|-----|-------------|
| `latest` | Latest stable release |
| `1.0` | Latest patch for the 1.0.x release line |
| `1.0.0` | Specific version |

### Updating

```bash
docker pull docker.i3omb.com/sofarr:latest
docker rm -f sofarr
# Re-run the docker run command above
```

## Manual Installation

1. **Clone and install**:
```bash
git clone https://git.i3omb.com/Gandalf/sofarr.git
cd sofarr
npm install
```

2. **Configure environment**:
```bash
cp .env.sample .env
# Edit .env with your service details
```

3. **Start the server**:
```bash
npm start
# or for development with auto-restart:
npm run dev
```

4. **Access the dashboard**:
Open `http://localhost:3001` in your browser

## Configuration (.env)

### Basic Server Settings
```bash
PORT=3001                           # Server port
LOG_LEVEL=info                      # Logging: debug, info, warn, error, silent
POLL_INTERVAL=5000                  # Background polling interval in ms (default: 5000)
                                    # Set to 0 or "off" to disable (on-demand mode)
```

### Webhooks & Smart Polling
```bash
# Required for webhook endpoints to accept events
SOFARR_WEBHOOK_SECRET=your-secret   # Shared secret (generate: openssl rand -hex 32)
SOFARR_BASE_URL=https://sofarr.example.com  # Public URL used by one-click setup

# Optional tuning
WEBHOOK_FALLBACK_TIMEOUT=10         # Minutes without a webhook before forcing a full poll (default: 10)
WEBHOOK_POLL_INTERVAL_MULTIPLIER=3  # Internal multiplier used by the skip logic (default: 3)
```

### Download Clients (PDCA)

sofarr uses a **Pluggable Download Client Architecture (PDCA)** that provides a unified interface for all download clients. This enables consistent data normalization, easy addition of new client types, and centralized configuration management.

**Supported Download Clients:**

| Client | Protocol | Auth Method | Notes |
|--------|----------|-------------|-------|
| SABnzbd | REST API | API Key | Usenet downloads |
| qBittorrent | Sync API | Username/Password | BitTorrent with incremental updates |
| Transmission | JSON-RPC | Username/Password | BitTorrent with session management |
| rTorrent | XML-RPC | HTTP Basic Auth | BitTorrent, requires the full RPC endpoint in the url field (e.g. /RPC2 or /xmlrpc for whatbox.ca). No path is automatically appended. |

### Service Instances (JSON Array Format)

All services support multi-instance configuration via single-line JSON arrays:

```bash
# SABnzbd Instances
SABNZBD_INSTANCES=[{"name":"primary","url":"https://sabnzbd.example.com","apiKey":"your-api-key"}]

# qBittorrent Instances (uses username/password, not API key)
QBITTORRENT_INSTANCES=[{"name":"main","url":"https://qbittorrent.example.com","username":"admin","password":"secret"}]

# Transmission Instances (uses username/password)
TRANSMISSION_INSTANCES=[{"name":"main","url":"http://transmission:9091/transmission/rpc","username":"admin","password":"pass"}]

# rTorrent Instances (uses username/password, URL must include full RPC endpoint)
# Standard installs use /RPC2. Some providers like whatbox.ca use /xmlrpc.
# No path is automatically appended - always include the full RPC endpoint.
RTORRENT_INSTANCES=[{"name":"main","url":"http://rtorrent:8080/RPC2","username":"rtorrent","password":"rtorrent"}]

# For whatbox.ca (example):
# RTORRENT_INSTANCES=[{"name":"whatbox","url":"https://user.whatbox.ca/xmlrpc","username":"user","password":"pass"}]

# Sonarr Instances
SONARR_INSTANCES=[{"name":"hd","url":"https://sonarr.example.com","apiKey":"your-api-key"}]

# Radarr Instances
RADARR_INSTANCES=[{"name":"movies","url":"https://radarr.example.com","apiKey":"your-api-key"}]

# Emby (single instance for authentication)
EMBY_URL=https://emby.example.com
EMBY_API_KEY=your-emby-api-key
```

### Legacy Single-Instance Format (still supported)

If you only have one instance, you can use the legacy format:
```bash
SABNZBD_URL=https://sabnzbd.example.com
SABNZBD_API_KEY=your-api-key

QBITTORRENT_URL=https://qbittorrent.example.com
QBITTORRENT_USERNAME=admin
QBITTORRENT_PASSWORD=secret

TRANSMISSION_URL=http://transmission:9091/transmission/rpc
TRANSMISSION_USERNAME=admin
TRANSMISSION_PASSWORD=pass

RTORRENT_URL=http://rtorrent:8080/RPC2
RTORRENT_USERNAME=rtorrent
RTORRENT_PASSWORD=rtorrent
```

## Setting Up User Tags

To see your downloads, you need to tag your media in Sonarr/Radarr:

1. **In Sonarr** (TV Shows):
   - Go to Series → Edit Series
   - Add a tag with your username (lowercase)
   - Save

2. **In Radarr** (Movies):
   - Go to Movies → Edit Movie
   - Add a tag with your username (lowercase)
   - Save

3. **Result**: When sofarr sees a download matching a show/movie tagged with "gordon", it appears on gordon's dashboard

## Features in Detail

### Background Polling

sofarr polls all configured services in the background and caches the results. Dashboard requests read from cache, making them near-instant regardless of how many services you have.

| Setting | Behaviour |
|---------|----------|
| `POLL_INTERVAL=5000` (default) | Background poller fetches every 5s. Dashboard reads from cache instantly. |
| `POLL_INTERVAL=10000` | Poll every 10 seconds. |
| `POLL_INTERVAL=0` / `off` / `disabled` | No background polling. Data fetched on-demand when a user opens the dashboard, then cached for 30 seconds. |

**On-demand mode** is useful for low-resource setups. When one user's browser opens the SSE stream it triggers a fresh poll; the fetched data is cached and served to all other connected clients until the next poll.

### Real-Time Updates
- **Server-Sent Events (SSE)** — the server pushes updates to the browser immediately after each poll cycle. No client-side timer; no wasted requests.
- In-place DOM updates for smooth UI (no flickering)
- Browser reconnects automatically on network interruption

### Download Information Displayed
- **Progress bar** with visual completion percentage
- **Speed** - Current download speed
- **ETA** - Estimated time remaining
- **Size** - Total size and downloaded amount
- **Status** - Downloading, Paused, Queued, etc.
- **Instance** - Which server the download is from

### For qBittorrent Downloads
- **Seeds** - Number of seeders
- **Peers** - Number of peers
- **Availability** - Percentage available in swarm (shown in red when below 100%)

## API Endpoints

### Authentication
- `POST /api/auth/login` — Login with Emby credentials
- `POST /api/auth/logout` — Logout and revoke session
- `GET /api/auth/me` — Check current session
- `GET /api/csrf` — Fetch a CSRF token

### Dashboard
- `GET /api/dashboard/stream` — **SSE stream**: pushes `{ user, isAdmin, downloads }` on every poll cycle
- `GET /api/dashboard/user-downloads` — Single-request download fetch (no streaming)
- `GET /api/dashboard/user-summary` — Per-user download counts (admin)
- `GET /api/dashboard/status` — Server / polling / cache status (admin)
- `GET /api/dashboard/cover-art` — Proxied cover art image
- `POST /api/dashboard/blocklist-search` — Blocklist a release and trigger a new search (admin or non-admin with eligibility: import issues OR torrent >1h old AND availability<100%)

### History
- `GET /api/history/recent` — Recently completed downloads from Sonarr/Radarr history

### Webhook Receiver (no user auth — protected by `X-Sofarr-Webhook-Secret`)
- `POST /api/webhook/sonarr` — receive Sonarr webhook events
- `POST /api/webhook/radarr` — receive Radarr webhook events

### Webhook Management (requires auth + CSRF)
- `GET /api/sonarr/api/v3/notification` — list Sonarr notification connections
- `POST /api/sonarr/api/v3/notification` — create/update Sonarr notification connection
- `GET /api/radarr/api/v3/notification` — list Radarr notification connections
- `POST /api/radarr/api/v3/notification` — create/update Radarr notification connection
- `POST /api/sonarr/webhook/enable` — one-click enable Sofarr webhook in Sonarr
- `POST /api/radarr/webhook/enable` — one-click enable Sofarr webhook in Radarr
- `POST /api/sonarr/webhook/test` — trigger a Sonarr test event
- `POST /api/radarr/webhook/test` — trigger a Radarr test event

### Service APIs (proxy to your services)
- `GET /api/sabnzbd/*` — SABnzbd API proxy
- `GET /api/sonarr/*` — Sonarr API proxy
- `GET /api/radarr/*` — Radarr API proxy
- `GET /api/emby/*` — Emby API proxy

## Logging Levels

Set `LOG_LEVEL` in your `.env`:
- `debug` - Verbose logging for troubleshooting
- `info` - Standard operational logging (default)
- `warn` - Only warnings and errors
- `error` - Only errors
- `silent` - No logging

Logs are written to both console and `server.log` file.

## Troubleshooting

**"No downloads showing"**
- Verify your media is tagged with your username in Sonarr/Radarr
- Check that LOG_LEVEL=debug shows matching attempts
- Ensure download names match between client and *arr apps

**"Can't connect to service"**
- Check URLs are accessible from the sofarr server
- Verify API keys and credentials
- Check CORS settings on your services

**"qBittorrent not showing"**
- Ensure username/password are correct
- Check qBittorrent Web UI is enabled
- Verify the URL includes the full path (e.g., `https://qb.example.com`)

## Testing

```bash
npm test                # run all tests once
npm run test:watch      # watch mode
npm run test:coverage   # with V8 coverage report (outputs to coverage/)
npm run test:ui         # interactive Vitest UI
```

290 tests across 18 test files covering auth middleware, CSRF protection, secret sanitization, config parsing, token store, qBittorrent utilities, history deduplication/classification, download client architecture, webhook endpoint security, input validation, replay protection, and webhook metrics integration. See [`tests/README.md`](tests/README.md) for design decisions and coverage targets.

## Development

```bash
# Start with auto-restart on changes
npm run dev

# Production start
npm start
```

## License

MIT

---

*sofarr: See what has downloaded "so far" from the comfort of your "sofa"*