feat(spectro): add -u real-time JSON stream output to chartread #8

Merged
gronod merged 4 commits from feature/chartread-json-output into main 2026-08-21 21:51:45 +01:00
4 changed files with 460 additions and 3 deletions
+17 -3
View File
@@ -32,15 +32,29 @@ and of Argyll was in October 2000. Code development commenced in 1995. See
Changes Summary for an overview of changes since the last release. Changes Changes Summary for an overview of changes since the last release. Changes
between revisions is detailed in the log.txt file that accompanies the source code. between revisions is detailed in the log.txt file that accompanies the source code.
It is licensed under the Affero GNU Version 3 license. Modifications in this fork:
--------------------------
- Added `-u` command-line switch to `chartread` to emit real-time row-level
and patch-level colour measurement events in JSON format (`ROW_COLORS_JSON: ...`)
to `stdout` for external UI / subprocess integration.
Original Author: Graeme W. Gill
Modified source code repository: https://git.i3omb.com/gronod/argyllcms
License:
--------
ArgyllCMS is licensed under the GNU Affero General Public License (AGPL) Version 3.
In compliance with the AGPLv3, the complete corresponding source code for all
modifications is publicly available at https://git.i3omb.com/gronod/argyllcms.
For more detailed information, please consult the HTML documentation in For more detailed information, please consult the HTML documentation in
<doc/ArgyllDoc.html>, or <http://www.argyllcms.com/doc/ArgyllDoc.html>. <doc/ArgyllDoc.html>, or <http://www.argyllcms.com/doc/ArgyllDoc.html>.
For the most recent source code start at <http://www.argyllcms.com/index.html>. For the upstream source code start at <http://www.argyllcms.com/index.html>.
Contact me in regards to Argyll, icclib or cgatslib at: Contact Graeme in regards to original Argyll, icclib or cgatslib at:
Graeme at argyllcms dot com Graeme at argyllcms dot com
Enjoy! Enjoy!
+3
View File
@@ -49,6 +49,7 @@ Verbose
mode</tt><tt><br> mode</tt><tt><br>
</tt> <tt> &nbsp;</tt><tt><a href="#u">-u</a></tt><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Emit real-time JSON stream of row patch data to stdout</tt><tt><br>
</tt><tt> &nbsp;</tt><tt><a href="#c">-c listno</a></tt><tt> &nbsp; </tt><tt> &nbsp;</tt><tt><a href="#c">-c listno</a></tt><tt> &nbsp;
&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; Set communication port from the &nbsp;&nbsp; &nbsp;&nbsp;&nbsp; Set communication port from the
following list (default 1)</tt><tt><br> following list (default 1)</tt><tt><br>
@@ -656,6 +657,8 @@ Override
<a name="v"></a>The <b>-v</b> flag causes extra information to be <a name="v"></a>The <b>-v</b> flag causes extra information to be
printed out during chartread operation.<br> printed out during chartread operation.<br>
<br> <br>
<a name="u"></a>The <b>-u</b> flag enables real-time JSON streaming of row patch data to stdout. Each time a row is completed (either forward or reverse), a JSON object containing the expected and measured patch data, along with padding indicators, is emitted prefixed by <code>ROW_COLORS_JSON:</code>. This is designed for GUI applications wrapping chartread to display live swatches and calculate color differences during reading.<br>
<br>
<a name="c"></a>Normally instruments are connected via a serial <a name="c"></a>Normally instruments are connected via a serial
communication port, and the port used should be selected by communication port, and the port used should be selected by
supplying the correct parameter to the <b>-c</b> flag. If you supplying the correct parameter to the <b>-c</b> flag. If you
+331
View File
@@ -0,0 +1,331 @@
# ArgyllCMS `chartread -u` Subprocess Integration Specification & License Isolation Guide
This document specifies the communication protocol, JSON payload schema, process lifecycle, and licensing isolation architecture for integrating `chartread -u` into external user interfaces and control software.
---
## 1. Architectural & Licensing Boundary (AGPLv3 Isolation)
ArgyllCMS is licensed under the **GNU Affero General Public License (AGPL) Version 3**. To ensure a separate host project (proprietary, MIT, Apache, etc.) is **not tainted** by the AGPLv3 copyleft terms, the integration must maintain a strict, arm's-length inter-process communication (IPC) boundary:
### Core Isolation Rules
1. **No Library Linking**: The host application **must never** statically or dynamically link (`#include`, `.so`, `.dylib`, `.dll`, `.a`) against any ArgyllCMS C libraries (`libinst`, `libicc`, `libcgats`, `libyajl`, etc.).
2. **Subprocess Isolation via Standard OS Pipes**: `chartread` must run strictly as a standalone, decoupled child process.
3. **Standard IPC Only**: Communication is conducted exclusively over standard POSIX / Win32 file descriptors (`stdin`, `stdout`, `stderr`).
4. **Independent Binary Distribution**: The `chartread` binary should be treated as an external utility tool invoked by the OS shell or process manager.
```
+-------------------------------------------------------------+
| Host Application |
| (Electron, Web App, Qt, Python, Rust, etc.) |
| |
| +-----------------------------------------------------+ |
| | Line-by-Line Subprocess Stream Reader | |
| +-----------------------------------------------------+ |
+------------------------------|------------------------------+
|
Standard Pipes | (stdin / stdout / stderr)
|
+------------------------------v------------------------------+
| Isolated Child Process |
| `chartread -u ...` |
| (AGPLv3 Licensed Binary) |
+-------------------------------------------------------------+
```
---
## 2. Command Invocation
### Command Syntax
```bash
chartread [options] -u <target_basename>
```
* **`-u` Flag**: Enables real-time emission of JSON records to `stdout`.
* `<target_basename>`: The base name of the input `.ti2` target file and output `.ti3` measurement file (without extension).
### Key Command Options
| Flag | Description |
| :--- | :--- |
| `-u` | **Required for streaming**: Emits `ROW_COLORS_JSON: ...` payloads on `stdout`. |
| `-p` | Patch-by-patch spot mode (emits JSON after each individual patch). |
| `-n` | Disable spectral readings (omits `spectral` field from JSON). |
| `-c <port>` | Select communication port / instrument index. |
| `-d` | Display measurement mode. |
| `-t` | Transmission measurement mode. |
| `-e` | Emissive measurement mode. |
---
## 3. Stream Protocol & Framing Specification
When `-u` is provided, `chartread` outputs two types of lines to `stdout`:
1. **Human-Readable Status / Prompt Lines**: Unprefixed text intended for user prompts or diagnostics (e.g. `Hit [Space] to read strip A`).
2. **Structured JSON Events**: Single-line JSON objects strictly prefixed by the marker:
```
ROW_COLORS_JSON: <json_object>\n
```
### Stream Guarantees
* **Line-Delimited**: Every JSON event is serialized as a single, uninterrupted line terminated with `\n`.
* **Immediate Flush**: `chartread` explicitly invokes `fflush(stdout)` immediately after emitting each JSON line, guaranteeing low latency without OS-level output buffering.
* **Deterministic Ordering**: Patches within a row are always indexed in canonical left-to-right strip order (bi-directional scan reversals are normalized internally before emission).
---
## 4. JSON Payload Schema
### Top-Level Object Schema
```json
ROW_COLORS_JSON: {
"event": "row_complete",
"row_id": "A",
"row_index": 0,
"total_rows": 12,
"patch_count": 21,
"patches": [
/* Array of Patch Objects */
]
}
```
| Field | Type | Description |
| :--- | :--- | :--- |
| `event` | `string` | Event identifier. Currently always `"row_complete"`. |
| `row_id` | `string` | Human-readable label of the row/strip (e.g. `"A"`, `"B"`, `"1"`). |
| `row_index` | `integer` | 0-based index of the row within the target chart (`0 ... total_rows - 1`). |
| `total_rows` | `integer` | Total number of rows/passes defined in the target chart. |
| `patch_count`| `integer` | Number of patch elements contained in this row payload. |
| `patches` | `array` | List of individual patch data objects. |
---
### Patch Object Schema
```json
{
"id": "1",
"loc": "A1",
"is_pad": false,
"device": [0.0, 50.0, 100.0, 0.0],
"expected": {
"XYZ": [18.4210, 20.1234, 15.6789],
"Lab": [51.98, -8.45, 12.32]
},
"measured": {
"XYZ": [18.5120, 20.0451, 15.7100],
"Lab": [51.89, -8.31, 12.15],
"spectral": {
"bands": 36,
"start_nm": 380.0,
"end_nm": 730.0,
"norm": 100.0,
"values": [0.0120, 0.0135, 0.0180, 0.0245]
}
}
}
```
| Field | Type | Description |
| :--- | :--- | :--- |
| `id` | `string` | Patch ID string from `.ti2` file. If `"0"`, indicates a spacer/padding patch. |
| `loc` | `string` | Physical location coordinate string (e.g. `"A1"`, `"B12"`). |
| `is_pad` | `boolean` | `true` if patch is an alignment/lead-in spacer patch (`id == "0"`). UIs should typically render these distinctly or skip them in analysis. |
| `device` | `array[float]` | Device colorant drive values scaled to percentage `0.0 ... 100.0%` (e.g. `[C, M, Y, K]` or `[R, G, B]`). |
| `expected` | `object` *(optional)* | Expected reference values from `.ti2` (omitted if no reference data is present). |
| `expected.XYZ` | `array[float][3]` | Reference CIE XYZ values on reference scale `0.0 ... 100.0`. |
| `expected.Lab` | `array[float][3]` | Reference D50 $L^*a^*b^*$ computed via standard CIE transformation. |
| `measured` | `object` | Actual instrument readings. |
| `measured.XYZ` | `array[float][3]` | Measured CIE XYZ values on reference scale `0.0 ... 100.0`. |
| `measured.Lab` | `array[float][3]` | Measured D50 $L^*a^*b^*$ ($L^* \in [0, 100]$, $a^*, b^* \in [-128, 127]$). |
| `measured.spectral` | `object` *(optional)* | Spectral reflection/emission data (omitted if instrument is colorimeter-only or `-n` passed). |
| `measured.spectral.bands` | `integer` | Number of spectral sample bands. |
| `measured.spectral.start_nm` | `float` | Starting wavelength in nanometres (e.g. `380.0` or `400.0`). |
| `measured.spectral.end_nm` | `float` | Ending wavelength in nanometres (e.g. `700.0` or `730.0`). |
| `measured.spectral.norm` | `float` | Normalization scale factor (typically `100.0`). |
| `measured.spectral.values` | `array[float]` | Array of spectral reflectance / radiance values per band. |
---
## 5. Integration Implementation Examples
### Node.js / Electron / TypeScript Integration
```typescript
import { spawn, ChildProcessWithoutNullStreams } from 'child_process';
import * as readline from 'readline';
export interface PatchColorEvent {
event: string;
row_id: string;
row_index: number;
total_rows: number;
patch_count: number;
patches: Array<{
id: string;
loc: string;
is_pad: boolean;
device: number[];
expected?: {
XYZ: [number, number, number];
Lab: [number, number, number];
};
measured: {
XYZ: [number, number, number];
Lab: [number, number, number];
spectral?: {
bands: number;
start_nm: number;
end_nm: number;
norm: number;
values: number[];
};
};
}>;
}
export class ChartreadRunner {
private process: ChildProcessWithoutNullStreams | null = null;
private readonly JSON_PREFIX = 'ROW_COLORS_JSON: ';
public startSession(
targetBasename: string,
onRowData: (data: PatchColorEvent) => void,
onConsoleMessage: (msg: string) => void,
onError: (err: string) => void
): void {
// Spawn isolated process over standard pipes (No library linking)
this.process = spawn('chartread', ['-u', targetBasename]);
// Parse stdout line by line
const rlOut = readline.createInterface({ input: this.process.stdout });
rlOut.on('line', (line: string) => {
const trimmed = line.trim();
if (trimmed.startsWith(this.JSON_PREFIX)) {
try {
const jsonStr = trimmed.substring(this.JSON_PREFIX.length);
const payload: PatchColorEvent = JSON.parse(jsonStr);
onRowData(payload);
} catch (err) {
onError(`Failed to parse JSON row payload: ${err}`);
}
} else if (trimmed.length > 0) {
onConsoleMessage(trimmed);
}
});
// Capture stderr for warnings / errors
const rlErr = readline.createInterface({ input: this.process.stderr });
rlErr.on('line', (errLine: string) => {
onError(errLine);
});
this.process.on('close', (code) => {
onConsoleMessage(`chartread exited with code ${code}`);
this.process = null;
});
}
/**
* Send user keyboard triggers or confirmations to chartread (e.g. Spacebar or Enter)
*/
public sendInput(input: string): void {
if (this.process && this.process.stdin.writable) {
this.process.stdin.write(input);
}
}
/**
* Abort measurement session
*/
public abort(): void {
if (this.process) {
this.sendInput('q\n'); // Send standard quit character
setTimeout(() => {
if (this.process) {
this.process.kill('SIGTERM');
}
}, 500);
}
}
}
```
---
### Python Subprocess Integration
```python
import subprocess
import json
import threading
class ChartreadClient:
JSON_PREFIX = "ROW_COLORS_JSON: "
def __init__(self, target_basename: str, on_row_callback, on_status_callback):
self.target_basename = target_basename
self.on_row_callback = on_row_callback
self.on_status_callback = on_status_callback
self.process = None
def start(self):
# Arms-length subprocess invocation maintaining AGPL isolation
self.process = subprocess.Popen(
["chartread", "-u", self.target_basename],
stdin=subprocess.PIPE,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
text=True,
bufsize=1 # Line buffered
)
threading.Thread(target=self._read_stdout, daemon=True).start()
threading.Thread(target=self._read_stderr, daemon=True).start()
def _read_stdout(self):
for line in iter(self.process.stdout.readline, ''):
line_str = line.strip()
if line_str.startswith(self.JSON_PREFIX):
payload_str = line_str[len(self.JSON_PREFIX):]
try:
data = json.loads(payload_str)
self.on_row_callback(data)
except json.JSONDecodeError as ex:
print(f"JSON decode error: {ex}")
elif line_str:
self.on_status_callback(line_str)
def _read_stderr(self):
for line in iter(self.process.stderr.readline, ''):
if line.strip():
print(f"[chartread stderr] {line.strip()}")
def send_key(self, key: str):
if self.process and self.process.stdin:
self.process.stdin.write(f"{key}\n")
self.process.stdin.flush()
def terminate(self):
if self.process:
self.process.terminate()
```
---
## 6. Real-Time UI Visualisation Best Practices
1. **Rendering Device Colors**:
* For RGB targets: scale `device[0..2]` from $0..100$ to $0..255$ (`rgb(r%, g%, b%)`).
* For CMYK targets: use device simulation or convert `measured.Lab` / `expected.Lab` to sRGB for color swatch display.
2. **Delta E Calculation ($\Delta E_{00}$ or $\Delta E_{ab}$)**:
* When `expected.Lab` and `measured.Lab` are both present, calculate the color difference $\Delta E$.
* The basic Euclidean distance ($\Delta E_{ab}$) is simple to implement directly:
$$\Delta E_{ab} = \sqrt{(L_m^* - L_e^*)^2 + (a_m^* - a_e^*)^2 + (b_m^* - b_e^*)^2}$$
* **Recommendation**: For professional color work, use the modern $\Delta E_{00}$ (CIEDE2000) formula. Due to its complexity, it is highly recommended to use an established color math library (e.g., `colorjs.io` in JavaScript, or `colormath` in Python) rather than implementing it from scratch.
* Display green/amber/red indicator lights next to each patch in real-time based on your accepted $\Delta E$ tolerance.
3. **Handling Alignment / Spacer Patches (`is_pad == true`)**:
* Omit `is_pad == true` patches from quality score computations, or render them with dashed neutral borders to maintain layout grid accuracy without skewing statistics.
+109
View File
@@ -201,6 +201,76 @@ typedef struct {
xspect sp; /* Spectrum. sp.spec_n > 0 if valid, 100 scaled for ref. */ xspect sp; /* Spectrum. sp.spec_n > 0 if valid, 100 scaled for ref. */
} chcol; } chcol;
extern int json_ui_out;
int json_ui_out = 0;
static void compute_patch_metrics(chcol *scb, double *eLab, double *mLab) {
/* Expected Lab */
if (scb->eXYZ[0] != 0.0 || scb->eXYZ[1] != 0.0 || scb->eXYZ[2] != 0.0) {
double scaled_exyz[3];
scaled_exyz[0] = scb->eXYZ[0] / 100.0;
scaled_exyz[1] = scb->eXYZ[1] / 100.0;
scaled_exyz[2] = scb->eXYZ[2] / 100.0;
icmXYZ2Lab(&icmD50, eLab, scaled_exyz);
} else {
eLab[0] = eLab[1] = eLab[2] = 0.0;
}
/* Measured Lab */
{
double scaled_mxyz[3];
scaled_mxyz[0] = scb->XYZ[0] / 100.0;
scaled_mxyz[1] = scb->XYZ[1] / 100.0;
scaled_mxyz[2] = scb->XYZ[2] / 100.0;
icmXYZ2Lab(&icmD50, mLab, scaled_mxyz);
}
}
static void emit_row_json_colors(const char *row_id, int row_index, int total_rows, int patch_count, int nchan, chcol **scbs) {
int i, j;
if (!json_ui_out) return;
fprintf(stdout, "ROW_COLORS_JSON: {\"event\": \"row_complete\", \"row_id\": \"%s\", \"row_index\": %d, \"total_rows\": %d, \"patch_count\": %d, \"patches\": [", row_id ? row_id : "", row_index, total_rows, patch_count);
for (i = 0; i < patch_count; i++) {
chcol *scb = scbs[i];
int is_pad = 0;
double eLab[3], mLab[3];
if (scb->id && strcmp(scb->id, "0") == 0)
is_pad = 1;
compute_patch_metrics(scb, eLab, mLab);
fprintf(stdout, "%s{\"id\": \"%s\", \"loc\": \"%s\", \"is_pad\": %s, \"device\": [", i == 0 ? "" : ", ", scb->id ? scb->id : "", scb->loc ? scb->loc : "", is_pad ? "true" : "false");
for (j = 0; j < nchan; j++) {
fprintf(stdout, "%s%.4f", j == 0 ? "" : ", ", scb->dev[j] * 100.0);
}
fprintf(stdout, "]");
if (scb->eXYZ[0] != 0.0 || scb->eXYZ[1] != 0.0 || scb->eXYZ[2] != 0.0) {
fprintf(stdout, ", \"expected\": {\"XYZ\": [%.4f, %.4f, %.4f], \"Lab\": [%.4f, %.4f, %.4f]}",
scb->eXYZ[0], scb->eXYZ[1], scb->eXYZ[2],
eLab[0], eLab[1], eLab[2]);
}
fprintf(stdout, ", \"measured\": {\"XYZ\": [%.4f, %.4f, %.4f], \"Lab\": [%.4f, %.4f, %.4f]",
scb->XYZ[0], scb->XYZ[1], scb->XYZ[2],
mLab[0], mLab[1], mLab[2]);
if (scb->sp.spec_n > 0) {
fprintf(stdout, ", \"spectral\": {\"bands\": %d, \"start_nm\": %.1f, \"end_nm\": %.1f, \"norm\": %.1f, \"values\": [",
scb->sp.spec_n, scb->sp.spec_wl_short, scb->sp.spec_wl_long, scb->sp.norm);
for (j = 0; j < scb->sp.spec_n; j++) {
fprintf(stdout, "%s%.4f", j == 0 ? "" : ", ", scb->sp.spec[j]);
}
fprintf(stdout, "]}");
}
fprintf(stdout, "}}");
}
fprintf(stdout, "]}\n");
fflush(stdout);
}
/* Convert a base 62 character into a number */ /* Convert a base 62 character into a number */
/* (This is used for converting the PASSES_IN_STRIPS string */ /* (This is used for converting the PASSES_IN_STRIPS string */
/* (Could convert this to using an alphix("0-9A-Za-Z")) */ /* (Could convert this to using an alphix("0-9A-Za-Z")) */
@@ -1040,6 +1110,16 @@ a1log *log /* verb, debug & error log */
scols[i]->mcond = vals[i].mcond; scols[i]->mcond = vals[i].mcond;
scols[i]->rr = 1; /* Has been read */ scols[i]->rr = 1; /* Has been read */
} }
if (json_ui_out) {
int k;
for (k = 0; k < totpa; k++) {
char row_id[32];
sprintf(row_id, "%s", paix->aix(paix, k));
emit_row_json_colors(row_id, k, totpa, stipa, scols[0]->n, &scols[k * stipa]);
}
}
free(vals); free(vals);
@@ -1340,6 +1420,15 @@ a1log *log /* verb, debug & error log */
scols[sti]->rr = 1; /* Has been read */ scols[sti]->rr = 1; /* Has been read */
} }
if (json_ui_out) {
int k;
for (k = 0; k < paist; k++) {
char row_id[32];
sprintf(row_id, "%s", paix->aix(paix, pai + k));
emit_row_json_colors(row_id, pai + k, totpa, stipa, scols[0]->n, &scols[(pai + k) * stipa]);
}
}
if (cap2 & inst2_xy_holdrel) { if (cap2 & inst2_xy_holdrel) {
@@ -1948,6 +2037,13 @@ a1log *log /* verb, debug & error log */
} }
scb[i]->rr = 1; /* Has been read */ scb[i]->rr = 1; /* Has been read */
} }
if (json_ui_out) {
char row_id[32];
sprintf(row_id, "%s", paix->aix(paix, oroi));
emit_row_json_colors(row_id, oroi, totpa, stipa, scb[0]->n, scb);
}
incflag = 2; /* Skip to next unread */ incflag = 2; /* Skip to next unread */
@@ -2419,6 +2515,14 @@ a1log *log /* verb, debug & error log */
scols[pix]->sp = val.sp; scols[pix]->sp = val.sp;
} }
scols[pix]->rr = 1; /* Has been read */ scols[pix]->rr = 1; /* Has been read */
if (json_ui_out) {
char row_id[32];
int oroi = pix / stipa;
sprintf(row_id, "%s", paix->aix(paix, oroi));
emit_row_json_colors(row_id, oroi, totpa, 1, scols[pix]->n, &scols[pix]);
}
printf(" Patch read OK\n"); printf(" Patch read OK\n");
/* Advance to next patch. */ /* Advance to next patch. */
@@ -2481,6 +2585,7 @@ usage() {
fprintf(stderr," ** No ports found **\n"); fprintf(stderr," ** No ports found **\n");
} }
fprintf(stderr," -t Use transmission measurement mode\n"); fprintf(stderr," -t Use transmission measurement mode\n");
fprintf(stderr," -u Emit real-time JSON updates to stdout\n");
fprintf(stderr," -d Use display measurement mode (white Y relative results)\n"); fprintf(stderr," -d Use display measurement mode (white Y relative results)\n");
cap2 = inst_show_disptype_options(stderr, " -y ", icmps, 0, 0); cap2 = inst_show_disptype_options(stderr, " -y ", icmps, 0, 0);
fprintf(stderr," -e Emissive for transparency on a light box\n"); fprintf(stderr," -e Emissive for transparency on a light box\n");
@@ -2733,6 +2838,10 @@ int main(int argc, char *argv[]) {
trans = 0; trans = 0;
displ = 2; displ = 2;
/* Enable UI JSON output */
} else if (argv[fa][1] == 'u') {
json_ui_out = 1;
/* Request emissive measurement */ /* Request emissive measurement */
} else if (argv[fa][1] == 'e') { } else if (argv[fa][1] == 'e') {
emis = 1; emis = 1;