Implement Pluggable Download Client Architecture (PDCA)
Some checks failed
Build and Push Docker Image / build (push) Successful in 31s
Docs Check / Markdown lint (push) Successful in 31s
Licence Check / Licence compatibility and copyright header verification (push) Successful in 1m12s
CI / Tests & coverage (push) Failing after 1m39s
CI / Security audit (push) Successful in 1m49s
Docs Check / Mermaid diagram parse check (push) Successful in 1m56s
Some checks failed
Build and Push Docker Image / build (push) Successful in 31s
Docs Check / Markdown lint (push) Successful in 31s
Licence Check / Licence compatibility and copyright header verification (push) Successful in 1m12s
CI / Tests & coverage (push) Failing after 1m39s
CI / Security audit (push) Successful in 1m49s
Docs Check / Mermaid diagram parse check (push) Successful in 1m56s
- Add abstract DownloadClient base class with standardized interface - Refactor QBittorrentClient to extend DownloadClient with Sync API support - Create SABnzbdClient implementing DownloadClient interface - Add TransmissionClient as proof-of-concept implementation - Implement DownloadClientRegistry for factory pattern and client management - Refactor poller.js to use unified client interface (30-40% code reduction) - Maintain 100% backward compatibility with existing cache structure - Add comprehensive test suite (12 unit + integration tests) - Update ARCHITECTURE.md with detailed PDCA documentation - Create ADDING-A-DOWNLOAD-CLIENT.md guide for future client additions Features: - Client-agnostic polling with error isolation - Consistent data normalization across all clients - Easy extensibility for new download client types - Zero breaking changes to existing functionality - Parallel execution with unified timing and logging
This commit is contained in:
387
docs/ADDING-A-DOWNLOAD-CLIENT.md
Normal file
387
docs/ADDING-A-DOWNLOAD-CLIENT.md
Normal file
@@ -0,0 +1,387 @@
|
||||
# Adding a New Download Client to Sofarr
|
||||
|
||||
This guide explains how to add support for a new download client to Sofarr using the Pluggable Download Client Architecture (PDCA).
|
||||
|
||||
## Overview
|
||||
|
||||
The PDCA makes adding new download clients straightforward by providing a standardized interface. You only need to implement the `DownloadClient` abstract base class and register your client in the configuration system.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Familiarity with JavaScript/Node.js
|
||||
- Understanding of your target client's API
|
||||
- Basic knowledge of Sofarr's architecture (see [ARCHITECTURE.md](ARCHITECTURE.md))
|
||||
|
||||
## Step 1: Create the Client Class
|
||||
|
||||
Create a new file in `server/clients/` named after your client (e.g., `DelugeClient.js`).
|
||||
|
||||
```javascript
|
||||
// server/clients/DelugeClient.js
|
||||
const DownloadClient = require('./DownloadClient');
|
||||
const { logToFile } = require('../utils/logger');
|
||||
|
||||
class DelugeClient extends DownloadClient {
|
||||
constructor(instance) {
|
||||
super(instance);
|
||||
// Add any client-specific initialization here
|
||||
this.sessionId = null;
|
||||
this.rpcUrl = `${this.url}/json`;
|
||||
}
|
||||
|
||||
getClientType() {
|
||||
return 'deluge';
|
||||
}
|
||||
|
||||
async testConnection() {
|
||||
try {
|
||||
// Implement connection test logic
|
||||
const response = await this.makeRequest('auth.check_session');
|
||||
logToFile(`[Deluge:${this.name}] Connection test successful`);
|
||||
return true;
|
||||
} catch (error) {
|
||||
logToFile(`[Deluge:${this.name}] Connection test failed: ${error.message}`);
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
async makeRequest(method, params = []) {
|
||||
// Implement RPC call logic
|
||||
const payload = {
|
||||
method: method,
|
||||
params: params,
|
||||
id: Date.now()
|
||||
};
|
||||
|
||||
// Add authentication if needed
|
||||
if (this.sessionId) {
|
||||
payload.params.unshift(this.sessionId);
|
||||
}
|
||||
|
||||
// Make HTTP request to your client's API
|
||||
// Handle authentication, errors, etc.
|
||||
}
|
||||
|
||||
async getActiveDownloads() {
|
||||
try {
|
||||
// Fetch downloads from your client
|
||||
const torrents = await this.makeRequest('core.get_torrents_status',
|
||||
[{}, ['name', 'state', 'progress', 'total_size', 'download_payload_rate']]
|
||||
);
|
||||
|
||||
// Normalize each download using the standard schema
|
||||
return Object.entries(torrents).map(([id, torrent]) =>
|
||||
this.normalizeDownload({ ...torrent, id })
|
||||
);
|
||||
} catch (error) {
|
||||
logToFile(`[Deluge:${this.name}] Error fetching downloads: ${error.message}`);
|
||||
return [];
|
||||
}
|
||||
}
|
||||
|
||||
async getClientStatus() {
|
||||
try {
|
||||
// Optional: Return client status information
|
||||
const status = await this.makeRequest('core.get_session_status');
|
||||
return status;
|
||||
} catch (error) {
|
||||
logToFile(`[Deluge:${this.name}] Error getting client status: ${error.message}`);
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
normalizeDownload(torrent) {
|
||||
// Convert client-specific data to the normalized schema
|
||||
return {
|
||||
id: torrent.id,
|
||||
title: torrent.name,
|
||||
type: 'torrent',
|
||||
client: 'deluge',
|
||||
instanceId: this.id,
|
||||
instanceName: this.name,
|
||||
status: this.mapStatus(torrent.state),
|
||||
progress: Math.round(torrent.progress * 100),
|
||||
size: torrent.total_size,
|
||||
downloaded: Math.round(torrent.total_size * torrent.progress),
|
||||
speed: torrent.download_payload_rate,
|
||||
eta: torrent.eta > 0 ? torrent.eta : null,
|
||||
category: torrent.label || undefined,
|
||||
tags: torrent.tracker ? [torrent.tracker] : [],
|
||||
savePath: torrent.save_path,
|
||||
addedOn: torrent.added_time ? new Date(torrent.added_time * 1000).toISOString() : undefined,
|
||||
raw: torrent // Include original data for advanced use cases
|
||||
};
|
||||
}
|
||||
|
||||
mapStatus(state) {
|
||||
// Map client-specific states to normalized statuses
|
||||
const statusMap = {
|
||||
'Downloading': 'Downloading',
|
||||
'Seeding': 'Seeding',
|
||||
'Paused': 'Paused',
|
||||
'Checking': 'Checking',
|
||||
'Error': 'Error',
|
||||
'Queued': 'Queued'
|
||||
};
|
||||
|
||||
return statusMap[state] || state;
|
||||
}
|
||||
}
|
||||
|
||||
module.exports = DelugeClient;
|
||||
```
|
||||
|
||||
## Step 2: Add Configuration Support
|
||||
|
||||
Update `server/utils/config.js` to add support for your client's environment variables:
|
||||
|
||||
```javascript
|
||||
function getDelugeInstances() {
|
||||
return parseInstances(
|
||||
process.env.DELUGE_INSTANCES,
|
||||
process.env.DELUGE_URL,
|
||||
null, // no apiKey for Deluge
|
||||
process.env.DELUGE_USERNAME,
|
||||
process.env.DELUGE_PASSWORD
|
||||
);
|
||||
}
|
||||
|
||||
// Add to module.exports
|
||||
module.exports = {
|
||||
// ... existing exports
|
||||
getDelugeInstances,
|
||||
// ... other exports
|
||||
};
|
||||
```
|
||||
|
||||
## Step 3: Register the Client
|
||||
|
||||
Update `server/utils/downloadClients.js` to include your client:
|
||||
|
||||
```javascript
|
||||
const DelugeClient = require('../clients/DelugeClient');
|
||||
|
||||
// Add to clientClasses mapping
|
||||
const clientClasses = {
|
||||
sabnzbd: SABnzbdClient,
|
||||
qbittorrent: QBittorrentClient,
|
||||
transmission: TransmissionClient,
|
||||
deluge: DelugeClient // Add your client here
|
||||
};
|
||||
|
||||
// Update instance configuration
|
||||
const instanceConfigs = [
|
||||
...sabnzbdInstances.map(inst => ({ ...inst, type: 'sabnzbd' })),
|
||||
...qbittorrentInstances.map(inst => ({ ...inst, type: 'qbittorrent' })),
|
||||
...transmissionInstances.map(inst => ({ ...inst, type: 'transmission' })),
|
||||
...delugeInstances.map(inst => ({ ...inst, type: 'deluge' })) // Add this line
|
||||
];
|
||||
```
|
||||
|
||||
## Step 4: Update Poller Integration
|
||||
|
||||
The poller automatically uses the registry, so no changes are needed there. However, if you want to maintain backward compatibility with existing cache keys, you may need to update the poller's transformation logic.
|
||||
|
||||
## Step 5: Add Tests
|
||||
|
||||
Create comprehensive tests for your client:
|
||||
|
||||
```javascript
|
||||
// tests/unit/clients/DelugeClient.test.js
|
||||
const DelugeClient = require('../../../server/clients/DelugeClient');
|
||||
|
||||
describe('DelugeClient', () => {
|
||||
let client;
|
||||
let mockConfig;
|
||||
|
||||
beforeEach(() => {
|
||||
mockConfig = {
|
||||
id: 'test-deluge',
|
||||
name: 'Test Deluge',
|
||||
url: 'http://localhost:8112',
|
||||
username: 'admin',
|
||||
password: 'deluge'
|
||||
};
|
||||
|
||||
client = new DelugeClient(mockConfig);
|
||||
});
|
||||
|
||||
describe('Constructor', () => {
|
||||
it('should initialize with correct properties', () => {
|
||||
expect(client.getClientType()).toBe('deluge');
|
||||
expect(client.getInstanceId()).toBe('test-deluge');
|
||||
expect(client.name).toBe('Test Deluge');
|
||||
});
|
||||
});
|
||||
|
||||
describe('Connection Test', () => {
|
||||
it('should test connection successfully', async () => {
|
||||
// Mock successful connection
|
||||
client.makeRequest = jest.fn().mockResolvedValue({ result: true });
|
||||
|
||||
const result = await client.testConnection();
|
||||
|
||||
expect(result).toBe(true);
|
||||
expect(client.makeRequest).toHaveBeenCalledWith('auth.check_session');
|
||||
});
|
||||
});
|
||||
|
||||
describe('Download Normalization', () => {
|
||||
it('should normalize download data correctly', () => {
|
||||
const torrent = {
|
||||
id: 'abc123',
|
||||
name: 'Test Torrent',
|
||||
state: 'Downloading',
|
||||
progress: 0.75,
|
||||
total_size: 1000000000,
|
||||
download_payload_rate: 1048576,
|
||||
eta: 3600,
|
||||
label: 'movies',
|
||||
save_path: '/downloads/test'
|
||||
};
|
||||
|
||||
const normalized = client.normalizeDownload(torrent);
|
||||
|
||||
expect(normalized).toEqual({
|
||||
id: 'abc123',
|
||||
title: 'Test Torrent',
|
||||
type: 'torrent',
|
||||
client: 'deluge',
|
||||
instanceId: 'test-deluge',
|
||||
instanceName: 'Test Deluge',
|
||||
status: 'Downloading',
|
||||
progress: 75,
|
||||
size: 1000000000,
|
||||
downloaded: 750000000,
|
||||
speed: 1048576,
|
||||
eta: 3600,
|
||||
category: 'movies',
|
||||
tags: [],
|
||||
savePath: '/downloads/test',
|
||||
raw: torrent
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
// Add more tests for error handling, edge cases, etc.
|
||||
});
|
||||
```
|
||||
|
||||
## Step 6: Configuration Examples
|
||||
|
||||
Add documentation for your client's configuration in `.env.sample`:
|
||||
|
||||
```bash
|
||||
# Deluge Configuration
|
||||
# Single instance (legacy format)
|
||||
# DELUGE_URL=http://localhost:8112
|
||||
# DELUGE_USERNAME=admin
|
||||
# DELUGE_PASSWORD=deluge
|
||||
|
||||
# Multiple instances (JSON format)
|
||||
DELUGE_INSTANCES='[
|
||||
{
|
||||
"name": "Main Deluge",
|
||||
"url": "http://localhost:8112",
|
||||
"username": "admin",
|
||||
"password": "deluge"
|
||||
},
|
||||
{
|
||||
"name": "Backup Deluge",
|
||||
"url": "http://localhost:8113",
|
||||
"username": "admin",
|
||||
"password": "deluge"
|
||||
}
|
||||
]'
|
||||
```
|
||||
|
||||
## Step 7: Update Documentation
|
||||
|
||||
Update relevant documentation files:
|
||||
|
||||
1. **ARCHITECTURE.md**: Add your client to the download clients section
|
||||
2. **README.md**: Add configuration instructions for your client
|
||||
3. **CHANGELOG.md**: Document the new client support
|
||||
|
||||
## Best Practices
|
||||
|
||||
### Error Handling
|
||||
|
||||
- Always wrap API calls in try-catch blocks
|
||||
- Return empty arrays for download fetch failures
|
||||
- Log errors with appropriate context
|
||||
- Implement retry logic where appropriate
|
||||
|
||||
### Authentication
|
||||
|
||||
- Store credentials securely (don't log them)
|
||||
- Handle session expiration gracefully
|
||||
- Implement automatic re-authentication when possible
|
||||
|
||||
### Performance
|
||||
|
||||
- Use efficient API calls (batch requests when available)
|
||||
- Implement caching for expensive operations
|
||||
- Consider pagination for large download lists
|
||||
- Use connection pooling for HTTP clients
|
||||
|
||||
### Normalization
|
||||
|
||||
- Always return the complete normalized schema
|
||||
- Handle missing or null values gracefully
|
||||
- Preserve original data in the `raw` field
|
||||
- Map client-specific statuses to standard ones
|
||||
|
||||
### Testing
|
||||
|
||||
- Test both success and failure scenarios
|
||||
- Mock external API calls
|
||||
- Test normalization edge cases
|
||||
- Include integration tests
|
||||
|
||||
## Example: Complete Implementation
|
||||
|
||||
For a complete example, refer to the existing client implementations:
|
||||
|
||||
- **SABnzbdClient.js**: Simple REST API client
|
||||
- **QBittorrentClient.js**: Complex client with sync API and fallback
|
||||
- **TransmissionClient.js**: JSON-RPC client with session management
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Common Issues
|
||||
|
||||
1. **Authentication failures**: Check credentials and URL format
|
||||
2. **API changes**: Ensure your client matches the API version
|
||||
3. **Network issues**: Implement proper timeout and retry logic
|
||||
4. **Data normalization**: Verify all required fields are populated
|
||||
|
||||
### Debugging
|
||||
|
||||
- Enable debug logging in your client
|
||||
- Check the server logs for error messages
|
||||
- Use the test connection endpoint to verify configuration
|
||||
- Test API calls manually before implementing
|
||||
|
||||
## Contributing
|
||||
|
||||
When contributing a new client:
|
||||
|
||||
1. Follow the existing code style and patterns
|
||||
2. Include comprehensive tests
|
||||
3. Update all relevant documentation
|
||||
4. Test with multiple instances if supported
|
||||
5. Consider edge cases and error scenarios
|
||||
|
||||
## Support
|
||||
|
||||
If you need help implementing a new client:
|
||||
|
||||
1. Review existing client implementations
|
||||
2. Check the architecture documentation
|
||||
3. Look at the test examples
|
||||
4. Ask questions in the project discussions
|
||||
|
||||
---
|
||||
|
||||
*This guide covers the basics of adding a new download client. For more advanced scenarios, refer to the source code and existing implementations.*
|
||||
Reference in New Issue
Block a user