feat: implement telemetry persistence, data visualization, and improved bluetooth stream parsing with frame statistics (#9)

Reviewed-on: #9
Co-authored-by: Gandalf <gordon@i3omb.com>
Co-committed-by: Gandalf <gordon@i3omb.com>

README.md

@@ -1,119 +1,76 @@
-# ESP32 ALDL Dashboard
+# ESP32 ALDL Dashboard Android Application
 
-An Android application built using Jetpack Compose to display real-time engine telemetry from a **1986 Pontiac Fiero 2.8L V6 (1227170 ECM)**. The app connects to a Bluetooth serial device named **ESP32-ALDL** and decodes a raw binary stream of 27-byte frames containing ECM data packets under the `$24` mask specifications.
+A modern, high-performance Android application designed to interface with GM OBD1 systems—specifically the **1986 Pontiac Fiero 1227170 ECM**—using a custom ESP32 Bluetooth SPP bridge. The app decodes the low-speed 160-baud ALDL (Assembly Line Diagnostic Link) datastream in real-time, displaying live telemetry, logging diagnostic parameters, and generating tuning heatmaps.
 
 ---
 
-## Technical Specifications
+## 🌟 Key Features
 
-### Data Stream Frame Structure
-The ESP32 reads the raw 160-baud unidirectional ALDL stream from the ECM and encapsulates it in a 27-byte packet transmitted over Bluetooth SPP.
-
-| Byte Index | Length | Value / Field | Description |
-| :--- | :--- | :--- | :--- |
-| **0 - 1** | 2 bytes | `AA 55` | Frame Sync Header (added by ESP32) |
-| **2 - 26** | 25 bytes | Raw Data Payload | 1227170 ECM data stream (0-indexed indices 0 to 24) |
+*   **Real-Time Instrumentation Dashboard:** 
+    *   Vibrant, animated Canvas-based components including a radial RPM gauge and a Throttle Position Sensor (TPS) bar graph.
+    *   Instantaneous readouts for Engine Speed (RPM), Vehicle Speed (MPH), Coolant Temperature, Manifold Air Temp (MAT), Manifold Absolute Pressure (MAP), TPS voltage, O2 Sensor voltage, and Battery voltage.
+    *   **Status Flags:** Instant visibility into critical operating states like Closed Loop Mode, Rich/Lean Mixture, Torque Converter Clutch (TCC) Lockup, and A/C Clutch requests.
+    *   **Derived Metrics:** Real-time calculated estimates of Engine Load and Fuel Flow Hints.
+*   **BLM & INT Fuel Trim Heatmap Grid (New):**
+    *   A real-time diagnostic table indexing fuel trim metrics across **14 RPM bands** (600 to 4800 RPM) and **17 MAP bands** (20 to 100 kPa).
+    *   Color-coded cell visualisations using dynamic RGB interpolation: Green for lean fuel trims ($\le 120$), Blue for stoichiometric neutral ($128$), and Red for rich fuel trims ($\ge 150$).
+*   **Multi-Parameter Line Charting (New):**
+    *   A dynamic telemetry visualizer toggling between a **Single Chart Mode** (large-scale view of any metric) and a **Multi Chart Mode** displaying up to 4 selected metrics in a 2x2 grid.
+    *   Supports charting for **12 distinct parameters**: RPM, Coolant Temp, MAP, TPS, O2 Sensor, Battery Voltage, Spark Advance, Base Pulse Width (BPW), MAT, BLM, Integrator, and Idle Air Control (IAC) Position.
+*   **Trouble Code Diagnostic Engine:**
+    *   Decodes active ECM trouble codes into human-readable alerts (e.g., *Code 14 - Coolant Temperature Sensor High*) dynamically displayed at the bottom of the dashboard screen.
+*   **Dual-Logging Framework:**
+    *   **Room Database Sessions:** Saves all active telemetry packets to a local SQLite database using Jetpack Room.
+    *   **TunerPro RT CSV Export:** Automatically compiles sessions into `.csv` log files fully compatible with TunerPro RT, exported to `Downloads/ALDLLogs` using Android MediaStore.
+    *   **Raw Binary Stream Logging (New):** Optional raw capture recording direct 27-byte diagnostic datastream packets (incorporating `AA 55` headers and 25-byte payloads) to `.bin` files for advanced playback and diagnostic troubleshooting.
+*   **Logged Files Manager (New):**
+    *   An in-app browser inside the Settings panel that scans `Downloads/ALDLLogs` for CSV and BIN logs, enabling users to view details, open logs with default viewers, or share files via the Android Sharesheet.
+*   **Persistent Preferences & Guardrails:**
+    *   DataStore-backed settings for Temperature Unit toggle (°C/°F), Auto-Logging toggles, Coolant alert thresholds, Low Battery alert thresholds, and Raw Binary Stream recording toggles.
 
 ---
 
-## Parameter Offsets & Decoding Formulas
+## 🛠️ Architecture & Technical Highlights
 
-The telemetry parameters are parsed from the `$24` / `$24A` ECM mask definitions (`24-INT10.ads`). In TunerPro's `.ads` file, the byte numbers are 1-indexed (e.g. `btByteNumber = 4` maps to raw payload byte index `3`).
-
-### 1. Primary Sensors & Measurements
-
-| Parameter | Payload Index | Raw Size | Formula / Conversion | Units |
-| :--- | :--- | :--- | :--- | :--- |
-| **IAC Position** | Index 3 (Byte 4) | 8-bit | $Value = Raw$ | Steps |
-| **Coolant Temp** | Index 4 (Byte 5) | 8-bit | $C = (Raw \times 0.75) - 40$<br>$F = (Raw \times 1.35) - 40$ | °C / °F |
-| **Vehicle Speed** | Index 5 (Byte 6) | 8-bit | $Value = Raw$ (Operation 3) | MPH |
-| **MAP** | Index 6 (Byte 7) | 8-bit | $Volts = Raw \times 0.019608$<br>$kPa = (Raw \times 0.369) + 10.354$ | Volts / kPa |
-| **Engine Speed** | Index 7 (Byte 8) | 8-bit | $RPM = Raw \times 25$ | RPM |
-| **TPS** | Index 8 (Byte 9) | 8-bit | $Volts = Raw \times 0.019608$ | Volts |
-| **Integrator (INT)** | Index 9 (Byte 10) | 8-bit | $Value = Raw$ | — |
-| **O2 Sensor** | Index 10 (Byte 11) | 8-bit | $mV = Raw \times 4.44$ | mV |
-| **Battery Voltage** | Index 17 (Byte 18) | 8-bit | $Volts = Raw \times 0.1$ | Volts |
-| **BLM** | Index 18 (Byte 19) | 8-bit | $Value = Raw$ | — |
-| **Rich/Lean Crosses** | Index 19 (Byte 20) | 8-bit | $Value = Raw$ | Crosses |
-| **Spark Advance** | Index 20 (Byte 21) | 8-bit | $Degrees = Raw \times 0.351563$ | Degrees |
-| **EGR Duty Cycle** | Index 21 (Byte 22) | 8-bit | $Percent = Raw \times 0.392157$ | % |
-| **Manifold Air Temp (MAT)** | Index 22 (Byte 23) | 8-bit | **Linear Table Interpolation** (see below) | °C / °F |
-| **Base Pulse Width (BPW)** | Index 23-24 (Byte 24-25) | 16-bit | $Raw = (HighByte \ll 8) \vert LowByte$<br>$ms = Raw \times 0.015259$ | Milliseconds (ms) |
+*   **Jetpack Compose & MVVM:** Developed with a clean Model-View-ViewModel architecture. State emission is managed via reactive `StateFlow` and `SharedFlow` streams to guarantee lag-free rendering.
+*   **Circular Buffering & Packet Validation (New):**
+    *   Ingests raw Bluetooth stream buffers using an `ArrayDeque<Byte>` circular buffer.
+    *   Employs a **27-byte lookahead validation algorithm** to search for authentic `AA 55` frame headers, preventing sync misalignment or telemetry corruption from noisy serial lines.
+*   **TunerPro ADS Translation:** Uses an advanced [ALDLParser](file:///home/gordon/esp32-aldl-android/app/src/test/java/com/example/esp32aldldashboard/ALDLParserTest.kt) mapping raw 25-byte payloads into physical metrics using exact scale conversions, offsets, and non-linear lookup interpolations (derived from the `24-INT10.ads` definition file).
+*   **Foreground Service Operations (New):**
+    *   Runs a persistent `BluetoothForegroundService` to keep the Bluetooth socket open and stream telemetry continuously in the background, even when the phone screen is locked or the app is minimized.
+*   **Firebase Integration:** Incorporates Firebase Crashlytics to monitor application stability and track runtime exceptions.
 
 ---
 
-### 2. MAT Linear Interpolation Tables
-The Manifold Air Temperature (MAT) is read from Index 22. It is mapped to degrees C or F using interpolation curves specified in tables 52 and 53:
+## 📱 Requirements
 
-*   **Celsius (`MAT C` - Table 52):**
-    *   `0` -> `200.0`, `12` -> `150.0`, `13` -> `145.0`, `14` -> `140.0`, `16` -> `135.0`, `18` -> `130.0`, `21` -> `125.0`, `23` -> `120.0`, `26` -> `115.0`, `30` -> `110.0`, `34` -> `105.0`, `39` -> `100.0`, `44` -> `95.0`, `50` -> `90.0`, `56` -> `85.0`, `64` -> `80.0`, `72` -> `75.0`, `81` -> `70.0`, `92` -> `65.0`, `102` -> `60.0`, `114` -> `55.0`, `126` -> `50.0`, `139` -> `45.0`, `152` -> `40.0`, `165` -> `35.0`, `177` -> `30.0`, `189` -> `25.0`, `199` -> `20.0`, `209` -> `15.0`, `218` -> `10.0`, `225` -> `5.0`, `231` -> `0.0`, `237` -> `-5.0`, `241` -> `-10.0`, `245` -> `-15.0`, `247` -> `-20.0`, `250` -> `-25.0`, `251` -> `-30.0`, `255` -> `-40.0`
-*   **Fahrenheit (`MAT F` - Table 53):**
-    *   `0` -> `392.0`, `12` -> `302.0`, `13` -> `293.0`, `14` -> `284.0`, `16` -> `275.0`, `18` -> `266.0`, `21` -> `257.0`, `23` -> `248.0`, `26` -> `239.0`, `30` -> `230.0`, `34` -> `221.0`, `39` -> `212.0`, `44` -> `203.0`, `50` -> `194.0`, `56` -> `185.0`, `64` -> `176.0`, `72` -> `167.0`, `81` -> `158.0`, `92` -> `149.0`, `102` -> `140.0`, `114` -> `131.0`, `126` -> `122.0`, `139` -> `113.0`, `152` -> `104.0`, `165` -> `95.0`, `177` -> `86.0`, `189` -> `77.0`, `199` -> `68.0`, `209` -> `59.0`, `218` -> `50.0`, `225` -> `41.0`, `231` -> `32.0`, `237` -> `23.0`, `241` -> `14.0`, `245` -> `5.0`, `247` -> `-4.0`, `250` -> `-13.0`, `251` -> `-22.0`, `255` -> `-40.0`
+*   **Android Device:** Android 8.0 (API Level 26) or higher.
+*   **Bluetooth Permissions:** Requires Nearby Devices (Android 12+) and Legacy Bluetooth Admin access.
+*   **ESP32 Transceiver:** Bridge hardware programmed to output 160-baud serial data from the ALDL pin and stream it over Classic Bluetooth SPP (named `ESP32-ALDL`).
 
 ---
 
-### 3. Stored Fault Trouble Codes
+## 🚀 Setup & Usage
 
-Malfunction Indicator Codes are mapped as bit flags across three specific status bytes:
-
-#### Codes Byte 1 (Payload Index 11 / Byte 12)
-*   **Bit 7:** Code 12 - Crank Sensor / System Check
-*   **Bit 6:** Code 13 - O2 Sensor
-*   **Bit 5:** Code 14 - Coolant High Temp
-*   **Bit 4:** Code 15 - Coolant Low Temp
-*   **Bit 3:** Code 21 - TPS Voltage High
-*   **Bit 2:** Code 22 - TPS Voltage Low
-*   **Bit 1:** Code 23 - MAT Voltage Low
-*   **Bit 0:** Code 24 - Vehicle Speed Sensor (VSS)
-
-#### Codes Byte 2 (Payload Index 12 / Byte 13)
-*   **Bit 7:** Code 25 - MAT Voltage High
-*   **Bit 5:** Code 32 - EGR System
-*   **Bit 4:** Code 33 - MAP Sensor High
-*   **Bit 3:** Code 34 - MAP Sensor Low
-*   **Bit 2:** Code 35 - IAC Position
-*   **Bit 0:** Code 42 - Electronic Spark Timing (EST)
-
-#### Codes Byte 3 (Payload Index 13 / Byte 14)
-*   **Bit 7:** Code 43 - Electronic Spark Control (Knock Sensor)
-*   **Bit 6:** Code 44 - O2 Sensor Lean Exhaust
-*   **Bit 5:** Code 45 - O2 Sensor Rich Exhaust
-*   **Bit 4:** Code 51 - PROM Error
-*   **Bit 3:** Code 52 - Cal-Pack Error
-*   **Bit 2:** Code 53 - System Battery Over-Voltage
-*   **Bit 0:** Code 55 - ADU Error
+1.  Pair the `ESP32-ALDL` Bluetooth module in your Android system settings.
+2.  Launch the application and grant all requested permissions.
+3.  Tap **Connect BT** on the main dashboard to establish connection and start telemetry.
+4.  Navigate between screens (Dashboard, Charts, BLM Table, Settings) using the bottom navigation bar.
+5.  Configure logging preferences or browse logged CSV/BIN files in the **Settings** menu.
 
 ---
 
-### 4. Engine Status & Bit Flags
+## 🧪 Testing & Verification
 
-#### Misc Byte 1 (Payload Index 14 / Byte 15)
-*   **Bit 1:** BLM Enabled (1 = Yes)
-*   **Bit 3:** Quasi Pulse Mode (1 = Yes)
-*   **Bit 4:** Async Pulse Mode (1 = Yes)
-*   **Bit 6:** Rich/Lean Status (1 = Rich, 0 = Lean)
-*   **Bit 7:** Loop Status (1 = Closed, 0 = Open)
+The parsing logic, scaling calculations, and boundary conditions are validated by a unit test suite located in [ALDLParserTest.kt](file:///home/gordon/esp32-aldl-android/app/src/test/java/com/example/esp32aldldashboard/ALDLParserTest.kt).
 
-#### Misc Byte 2 (Payload Index 15 / Byte 16)
-*   **Bit 5:** A/C Status (0 = Enabled, 1 = Disabled/Idle)
-*   **Bit 7:** Park/Neutral Switch (1 = Park/Neutral, 0 = In Gear)
+To run the unit tests, execute the following Gradle command in the root project directory:
+```bash
+./gradlew test
+```
 
-#### Misc Byte 3 (Payload Index 16 / Byte 17)
-*   **Bit 0:** A/C Clutch Command (1 = Enabled)
-*   **Bit 2:** Torque Converter Clutch (TCC) (1 = Locked)
-*   **Bit 5:** Power Steering Cramp Switch (1 = Active)
-
----
-
-## App Features & Architecture
-
-1.  **Bluetooth Thread Management:**
-    *   Queries for paired devices and connects directly to `"ESP32-ALDL"` over standard SPP RFCOMM sockets.
-    *   Robust frame synchronization: buffers incoming streams and aligns on the double-byte header `0xAA 0x55`.
-2.  **State Management:**
-    *   State flows from the Bluetooth thread through Kotlin `StateFlow` to Compose UI.
-3.  **Modern Dash Dashboard UI:**
-    *   Dynamic animations for RPM and TPS sweeps.
-    *   Metrics displayed in responsive tiles with custom indicator colors (e.g. engine loop states, fuel trims, battery health).
-    *   Live trouble code alert panel.
-    *   Diagnostic pane displaying real-time raw hex buffers for physical signal verification.
+### Coverage Areas
+*   **Sample Frame Decoding:** Parses a real-world telemetry payload and verifies the output of IAC position, coolant/manifold temperatures, MAP, TPS, battery voltage, BLM, integrator, spark advance, base pulse width, closed-loop flags, and active trouble codes.
+*   **Boundary Guards:** Ensures outlier protection for engine speeds, battery voltages, TPS, and coolant temperatures, rejecting out-of-range sensor values as invalid data packets.
+*   **Lookahead Stability:** Validates parser behavior against truncated or incomplete frame fragments.