GTI Danville PA — Longitudinal Fleet Health GTI Danville PA — Salud Longitudinal

Facility 3003 · GTI - PA · Org 331 · Issue-start 2024-04-23 · Snapshot 2026-07-09 · Rebuilt with corrected radio↔sensor taxonomy Instalación 3003 · GTI-PA · Org 331 · Reconstruido con taxonomía correcta radio↔sensor

Snapshot SummaryResumen

Radios (mesh)Radios (malla)
734
gateway·sink·repeater·nodeg·s·r·n
Sensors (probes)Sensores (sondas)
689
TEROS-12·TEROS-One·ATMOS-14TEROS-12·One·ATMOS-14
CRIT radiosRadios CRIT
419
silent / batt / signal / LQI criticalsilen/bat/señal/LQI crit
WARN radiosRadios WARN
76
battery 3.0–3.4V or signal −75 to −85bat 3.0–3.4V o señal −75 a −85
OK radiosRadios OK
227
all metrics within OK bandtodas OK
Silent radiosRadios silenciosos
29
no data in current 2-day windowsin datos en 2d

Radio vs Sensor TaxonomyTaxonomía Radio vs Sensor

A cornerstone correction: 734 radios (mesh participants — battery/RSSI/link_quality/travel_time live here) are physically distinct from 689 sensors (probes — substrate/climate readings live here). The two roles have different telemetry, different failure modes, and different remediations. Prior counts that summed the two categories together were inflated. Corrección clave: 734 radios (miembros de la malla) son distintos físicamente de 689 sensores (sondas). Cada rol tiene telemetría, modos de fallo y remediación distintos.

Radios · 734 mesh membersRadios · 734

  • 647 × node
  • 75 × repeater
  • 8 × sink
  • 4 × gateway

Node (Wirepas battery Li-SOCl₂), TEROS-12 and the H340-series repeaters are battery-powered; gateways, sinks, legacy H11x repeaters and h421/h412/h810 climate hardware are PoE. Battery/RSSI/link_quality/travel_time telemetry is per-radio. Node, TEROS-12 y los repetidores H340 funcionan a batería; el resto PoE. Batería/RSSI/LQI/TT viven en la capa radio.

Sensors · 689 probesSensores · 689

  • 624 × ter12
  • 38 × atm14
  • 27 × ter15

TEROS-12 → soil_moist / pore_ec / soil_temp (plugs into Node via cable). TEROS-One → same metrics but self-contained radio (H343 base). ATMOS-14 → air_temp / rel_hum / vpd / solar (climate probe). TEROS-12 → substrato (por cable al Node). TEROS-One → substrato con radio integrada (H343). ATMOS-14 → clima.

Backbone Mesh Health · Current 2-day WindowSalud de Columna de Malla

Backbone radios = 87 (4 gateways + 8 sinks + 75 repeaters). Gateways & sinks emit NO device-chart series on public_api (architectural artifact, not offline), so health can only be evaluated on the 75 repeaters. The stacked bar below shows their current-2d classification. Columna: 87 radios. Gateways y sinks no emiten series en public_api. Salud clasificable solo en 75 repetidores.

64 CRIT 8 WARN 3 OK 8 SILENT

Per-Phase ScorecardTarjeta por Fase

P1 = F1–F7 (original build-out). P2 = F8–F14. P3 = F15–F21. P4 = F22–F40 (expansion, per Ashley 2026-05-21). Radios and sensors counted separately; health rollup counts only radios (mesh members). P1=F1–F7, P2=F8–F14, P3=F15–F21, P4=F22–F40 (expansión). Radios y sensores separados; salud solo cuenta radios.

PhaseFase RoomsSalas RadiosRadios SensorsSensores CRIT WARN OK SILENT Rooms (sample)
P1 4 57 47 43 7 7 4 Flower 1 · Flower 5 · Flower 6 · Flower 7
P2 7 105 118 86 11 8 6 Flower 10 · Flower 11 · Flower 12 · Flower 13 · Flower 14 · Flower 8 · Flower 9
P3 7 106 105 60 17 29 1 Flower 15 · Flower 16 · Flower 17 · Flower 18 · Flower 19 · Flower 20 · Flower 21
P4 19 393 395 181 32 180 13 Flower 22 · Flower 23 · Flower 24 · Flower 25 · Flower 26 · Flower 27 · Flower 28 · Flower 29 …
Other 2 73 24 49 9 3 5 Clone 1 · Hallways

Longitudinal Zone Data Delivery Density · 117 WeeksDensidad Longitudinal de Datos · 117 Semanas

Every active Flower room × every bimonthly 2-day sample (2024-04 → 2026-06). Cells show the total number of valid data points across ALL metrics in that 2-day window — the direct measurement of "how much data actually made it to AROYA." Red cells mean the room went dark; green means dense delivery; changes across the row are the signal of mesh degradation over time. This is the corrected pull using stitched /public_api/rooms/<id>/chart/ 2-day windows (43 rooms × 14 windows = 602 stitched calls, real historical data through the full window). Cada sala Flower × cada muestra bimestral. Rojo = sala silenciada; verde = alta densidad. Datos reales stitcheados en ventanas de 2 días.

Phase · Room 2024-042024-062024-082024-102024-122025-022025-042025-062025-082025-102025-122026-022026-042026-06
P1 · Flower 167206720672059006720672035203520352032733520352035203520
P1 · Flower 2
P1 · Flower 3
P1 · Flower 4
P1 · Flower 567206720672054403200156802208022080220802208022080220802208022080
P1 · Flower 6672067206720544054401696022080220802208022080220802208076807040
P1 · Flower 78000800080008000512051201955122080220802208022080220802208022080
P2 · Flower 86720672067203520384238402208022080220882208022080220802208819520
P2 · Flower 96720672067206720672067202176021760217602176021760217702176021760
P2 · Flower 106720672067206720217682176021760217602176021760217602080020808
P2 · Flower 118000800080002112021128211202208022080220802112021120220802208021281
P2 · Flower 126720672067202208022080220802208022088220802208822080220802112022080
P2 · Flower 136720672067202208020800220802208022088220802208022080220802028622080
P2 · Flower 146720672867201770922080202572208022080220802208022080220801103022080
P3 · Flower 1580008000219122208022080220882208022080220802207122080220802207122080
P3 · Flower 1667206720211202208022080220802208022080220802203222090208002208022080
P3 · Flower 17672067282208022080220806702056722080220802208022080220802208022080
P3 · Flower 1858026720220882208022080220802208022080220802208022080220802208019520
P3 · Flower 1980008000220802079120791208002208020800208002080022080220802207222071
P3 · Flower 2067306720220802208022080220802208021976220882208022080220802208022088
P3 · Flower 2167207712220802208022080220802208022080220802208022080220802208022080
P4 · Flower 226720672086482208020876208002208022080220802080022080208002208022080
P4 · Flower 239280928092802976029760284802848029760297602880028800288002976026371
P4 · Flower 249280928092802976029760297602976829760297602976029760284802976026416
P4 · Flower 259280928092802848025920297602976029760297602976029760297602848026420
P4 · Flower 2688929280297602976029760297602976029760297602976029760297602976026412
P4 · Flower 2792809280288002976028800288002884228800288082944027200297602976026404
P4 · Flower 2892889280222362976029760297602976829760297602976029760297602976026356
P4 · Flower 2992889290130282976029760297602976029760297602976829760297602976026412
P4 · Flower 309280928092802976829760290342976029760297602976028415284802976023856
P4 · Flower 319280928092802976029760297602688028401281852976029760284802976026412
P4 · Flower 326180928092802976029760297602976029760297602976025377297602688026880
P4 · Flower 339156928892802976029760297682976829760297602976025389297602688029768
P4 · Flower 346320928092802976029768296722976029760297602976025397297602976029760
P4 · Flower 3564099280297602976029760297602976829760288002976025401297602976029760
P4 · Flower 3688839280297602934828480284802976029760297602860425112294402944029440
P4 · Flower 3791649280215842976029760278482976029760297602976025377297602976029760
P4 · Flower 3891519280127202976029760297602976028864297602976025381297602976029760
P4 · Flower 399164928092802976029768284802977029760297602976025393297602977029760
P4 · Flower 403823928092802976026888264322976029768297602976025385295442731229760
≥500 pts (dense) 100–500 1–100 0 (silent)

Telemetry Map · Radio → Attached SensorsMapa de Telemetría · Radio → Sensores

Where each sensor's data flows. Every radio parents zero or more sensors within its zone. When a radio goes silent, all its downstream sensors go dark — this is why silent-radio count matters more than silent-sensor count. Health rollup per room shows the ratio of healthy vs impaired radios; each impaired radio takes N sensors offline (see next section for exact downstream count). Cada radio parenta cero o más sensores en su zona. Radio silencioso = todos los sensores hijos silenciosos.

Room Radios Sensors OK radios Silent Batt CRIT Signal CRIT
P1 · Flower 1 12 2 0 0 0 12
P1 · Flower 5 15 15 7 0 0 8
P1 · Flower 6 15 15 0 4 0 11
P1 · Flower 7 15 15 0 0 0 15
P2 · Flower 8 15 28 3 1 2 9
P2 · Flower 9 15 15 0 2 0 13
P2 · Flower 10 15 15 0 0 0 15
P2 · Flower 11 15 15 1 3 0 11
P2 · Flower 12 15 15 1 0 0 14
P2 · Flower 13 15 15 2 0 0 13
P2 · Flower 14 15 15 1 0 0 14
P3 · Flower 15 15 15 8 0 0 7
P3 · Flower 16 15 15 0 0 0 15
P3 · Flower 17 15 15 4 0 0 11
P3 · Flower 18 15 15 3 1 0 11
P3 · Flower 19 16 15 5 0 0 11
P3 · Flower 20 15 15 4 0 0 11
P3 · Flower 21 15 15 5 0 0 10
P4 · Flower 22 15 15 2 0 1 12
P4 · Flower 23 21 22 8 0 0 13
P4 · Flower 24 21 21 9 0 0 12
P4 · Flower 25 21 21 0 3 0 18
P4 · Flower 26 21 21 0 0 0 21
P4 · Flower 27 21 22 11 0 0 10
P4 · Flower 28 21 21 13 1 0 7
P4 · Flower 29 21 21 5 0 0 16
P4 · Flower 30 21 21 14 3 0 4
P4 · Flower 31 21 21 8 2 0 11
P4 · Flower 32 21 21 11 1 0 9
P4 · Flower 33 21 21 18 0 0 3
P4 · Flower 34 21 21 15 0 0 6
P4 · Flower 35 21 21 3 3 0 15
P4 · Flower 36 21 21 18 0 0 3
P4 · Flower 37 21 21 3 0 0 18
P4 · Flower 38 21 21 6 0 0 15
P4 · Flower 39 21 21 18 0 0 3
P4 · Flower 40 21 21 18 0 0 3

Full radio-level JSON: telemetry_map.json (804 radios each with model, room, zone, current health, attached sensors, per-sensor metric list). JSON completo: telemetry_map.json (804 radios con sensores hijos).

Top Silent Radios · Ranked by Downstream Sensor ImpactRadios Silenciosos · Impacto Aguas Abajo

The 29 silent radios, sorted by how many sensors go dark when they're offline. Fixing a silent Node with 2 attached sensors saves 2 lost data streams; fixing a silent repeater saves its entire downstream tree. Los 29 radios silenciosos ordenados por sensores hijos afectados.

Radio IDModelSerialRoomSensors offline
68583 repeater H1110005904 P2 · Flower 8 1
68566 repeater H1110005876 P2 · Flower 11 1
76944 repeater H1110005852 P4 · Flower 32 1
68808 node H3210010266 P1 · Flower 6 1
68823 node H3210010275 P1 · Flower 6 1
68902 node H3210010300 P1 · Flower 6 1
68884 node H3210010368 P1 · Flower 6 1
147831 node H3430002013 P2 · Flower 9 1
69021 node H3210010200 P2 · Flower 9 1
69025 node H3210010223 P2 · Flower 11 1
69023 node H3210010207 P2 · Flower 11 1
68927 node H3210010340 P3 · Flower 18 1
80315 node H3210011003 P4 · Flower 25 1
80663 node H3210011124 P4 · Flower 25 1
147841 node H3430002108 P4 · Flower 25 1
80255 node H3210011044 P4 · Flower 28 1
80353 node H3210011116 P4 · Flower 30 1
80341 node H3210011114 P4 · Flower 30 1
80683 node H3210011481 P4 · Flower 31 1
80697 node H3210011459 P4 · Flower 31 1

Silent Radios by RoomRadios Silenciosos por Sala

Clusters of silent radios in one room usually indicate a local coverage or shielding problem (bad hop path, obstruction, dead repeater). Clusters spanning multiple rooms indicate a shared gateway or firmware issue. Silenciosos agrupados en una sala = problema local; en múltiples salas = gateway o firmware compartido.

P1 — 4 silent radios

  • Flower 6: 4 radios — 4×node

P2 — 6 silent radios

  • Flower 11: 3 radios — 2×node, 1×repeater
  • Flower 9: 2 radios — 2×node
  • Flower 8: 1 radios — 1×repeater

P3 — 1 silent radios

  • Flower 18: 1 radios — 1×node

P4 — 13 silent radios

  • Flower 25: 3 radios — 3×node
  • Flower 30: 3 radios — 2×node, 1×repeater
  • Flower 35: 3 radios — 3×node
  • Flower 31: 2 radios — 2×node
  • Flower 28: 1 radios — 1×node
  • Flower 32: 1 radios — 1×repeater

Other — 5 silent radios

  • Hallways: 4 radios — 4×repeater
  • (no room): 1 radios — 1×node

Worst Signal (RSSI p05, Current 2-day Window)Peor Señal (RSSI p05)

Radios ranked by 5th-percentile signal — the "bad moments" that cause grower-visible dropouts. Signal < −85 dBm = CRIT (repeater/hop reposition required). P5 de señal — momentos críticos que causan caídas visibles.

IDModelSerialRoom Signal p05 (dBm)LQI p05Status
135963 repeater H4110006603 P4 · Flower 26 -91 24.0 CRIT
68703 node H3210010257 P2 · Flower 12 -89 33.0 CRIT
76961 repeater H1110005559 P4 · Flower 24 -89 33.0 CRIT
77015 repeater H1110006108 P4 · Flower 37 -89 34.0 CRIT
68561 repeater H1110005864 P2 · Flower 12 -88 29.0 CRIT
68886 node H3210010277 P3 · Flower 19 -88 52.0 CRIT
69141 node H3210010003 P3 · Flower 20 -88 53.0 CRIT
76958 repeater H1110005590 P4 · Flower 34 -88 36.0 CRIT
68568 repeater H1110005828 P2 · Flower 13 -87 35.0 CRIT
68562 repeater H1110005810 P2 · Flower 14 -87 36.0 CRIT
68588 repeater H1110005906 P3 · Flower 18 -87 42.0 CRIT
68744 node H3210010271 P3 · Flower 19 -87 64.0 CRIT

Worst Battery (Node fleet only)Peor Batería (Nodes)

Only Nodes are battery-powered (Li-SOCl₂). Below 3.4 V = WARN, below 3.0 V = CRIT — schedule replacement. Solo Nodes son a batería. <3.4 V = WARN, <3.0 V = CRIT.

IDModelSerialRoom Battery mean (V)Status
56751 node H3210004721 P4 · Flower 22 3.174 WARN
68668 node H3210010535 P2 · Flower 8 3.371 WARN
68666 node H3210010415 P2 · Flower 8 3.378 WARN
56722 node H3210004812 P1 · Flower 1 3.444 WARN
135976 node H3310009749 P4 · Flower 26 3.454 CRIT
56744 node H3210004777 P1 · Flower 1 3.459 CRIT
68627 node H3210010511 P1 · Flower 6 3.476 CRIT
56728 node H3210004800 P1 · Flower 1 3.540 CRIT
56723 node H3210004796 P1 · Flower 1 3.580 WARN
56705 node H3210004801 P1 · Flower 1 3.591 WARN
56746 node H3210004770 P1 · Flower 1 3.592 CRIT
80880 node H3210010886 P4 · Flower 33 3.603 OK

Issue Timeline · 2024-04 → 2026-07Cronología

2024-04-23
Teresa Plank reports Danville running with one less repeater than the prior week. Climate station H1110005600 offline. First documented issue. Teresa Plank reporta un repetidor menos. Estación climática H1110005600 fuera de línea.
2024-04-27
F20 offline 8 hours; F19 H1100004477 shows sporadic readings. Teresa confirms F20 = Phase 3. F20 fuera 8h; F19 lecturas esporádicas. F20 = Fase 3.
2024-05-01
John Akers (Addium): 6 additional repeaters shipped as remediation. Coverage-expansion install scheduled. Se envían 6 repetidores adicionales.
2024-12-25/27
Cian confirms F17 data-tracking failure is caused by loss of signal in that area. Teresa/Thomas gateway-reset attempted, no recovery. F17 fallo de rastreo por pérdida de señal.
2026-05-06 → 05-20
First formal audit window (published 2026-06-01). Primera auditoría formal.
2026-05-20
Ashley Hummel: P4 dropping out regularly, floor 2 currently dropped and not coming back online. Escalates P4 expansion mesh reliability. Ashley: P4 caídas regulares.
2026-05-21
Cian confirms gateway 119178 healthy. Ashley confirms P4 is a separate portal group. Gateway 119178 saludable. P4 grupo portal separado.
2026-07-08
Teresa: "blip with F12 + now today F20 is not showing". Case 00040677 escalated. Teresa: caída F12, F20 no muestra. Caso 00040677.
2026-07-09
Pete Steiner (Addium): H1110005900 (repeater #68560, Flower 20) — UI shows 44% signal, backend LQI ~13%. This rebuild published same day. Pete: H1110005900 — LQI real ~13% vs 44% UI.

Root-Cause HypothesisHipótesis de Causa Raíz

H1 — Signal/mesh degradation clustered in P2 & P4 · HIGH

Silent-radio counts by phase: 4 P1 · 6 P2 · 1 P3 · 13 P4. Clustering by room (not scattered) indicates a local coverage or shielding issue rather than fleet-wide firmware. Matches the 2024-05 6-repeater install (P1/P2/P3) predating P4 expansion. Silenciosos agrupados por sala → problema local de cobertura, no firmware.

H2 — Battery aging on legacy P1 Nodes · MEDIUM

First-wave Nodes (installed 2023–2024) now 2–3 years into their Li-SOCl₂ lifespan; batteries below 3.4 V begin appearing in Flower 1/6/8. Replacement wave should target lowest batt-p50 first. Nodos legacy en P1 muestran envejecimiento de baterías.

Data delivery preserved:Entrega de datos preservada: The longitudinal room-density trend shows NO systemic collapse — zone-level data delivery per 2-day window rose from ~180K points (2024-04, facility onboarding) to ~540K (mature) and holds steady through 2026-06. Localised silent-radio events are the failure mode, not fleet-wide. Densidad longitudinal muestra crecimiento sano; los eventos son locales, no sistémicos.

Recommendations · PrioritisedRecomendaciones

P1 — Immediate (this week)P1 — Inmediato

  1. Field-check the 29 silent radios (see ranked table above). Attend to backbone repeaters first — one silent repeater takes its full downstream tree offline. Verificar en sitio los 29 radios silenciosos.
  2. Reposition / add a repeater near Flower 20 to fix H1110005900 hop (LQI 13% vs UI 44%). Reubicar/añadir repetidor en Flower 20.
  3. Re-audit repeater density in P4 (F22–F40) — the 2024-05 6-repeater install predates the P4 expansion. Auditar densidad de repetidores en P4.

P2 — Preventive (4–8 weeks)P2 — Preventivo

  1. Battery-replacement wave for Nodes at batt_p50 < 3.4 V (see Worst Battery table). Reemplazo de baterías <3.4V.
  2. Firmware sweep — confirm all repeaters on latest Wirepas firmware. Firmware de repetidores al día.

P3 — MonitoringP3 — Monitoreo

  • Weekly re-run of this pipeline (both current-2d snapshot and 2-day room stitch for the trailing bimonthly window). Reejecutar semanalmente.
  • Alert when any room has ≥3 silent radios simultaneously. Alerta si sala ≥3 radios silenciosos.

Methodology · Radio vs Sensor SplitMetodología

Inventory: GET /public_api/devices/?facility=3003&pagination=0 — 1515 device records, classified 734 radios + 689 sensors after excluding retired room 11368. Radios = mesh participants (gateway·sink·repeater·node); sensors = probes (TEROS-12·TEROS-One·ATMOS-14). Only Node + TEROS-12 are battery-powered — everything else is PoE. Inventario: 1515 registros → 734 radios + 689 sensores.

Current snapshot: /public_api/devices/<id>/chart/ with start/end covering the trailing 2-day window. Called for all 734 radios, 3 workers with 0.12s inter-call sleep to stay under the ~2–3 worker 429 threshold documented in aroya-public-api-device-telemetry. Gateways & sinks return HTTP 200 with empty series (architectural artifact — same behaviour observed across every AROYA facility). Sensors are NOT called on this endpoint — their metrics live in the room chart. Instantánea actual: devices/<id>/chart/ para los 734 radios, 3 workers, sleep 0.12s.

Longitudinal (117 weeks): /public_api/rooms/<id>/chart/ for each active room × 14 bimonthly 2-day windows (2024-04-14 → 2026-06-14). This endpoint returns zone- and room-level metric series with a hard 2-day cap per call — matches the stitched pattern in aroya_14d_stitch_pull.py (memory aroya-stitch-pull-no-browser). 43 rooms × 14 windows = 602 stitched calls, ~ 41 min total runtime. Longitudinal (117 semanas): rooms/<id>/chart/ × 14 ventanas bimestrales. 602 llamadas stitcheadas.

Historical per-radio telemetry: The /devices/<id>/chart/ endpoint is empirically confirmed to return only the rolling recent ~2 days regardless of the requested start — historical windows return HTTP 200 with empty series. This is an architectural limit of the public API; per-radio 117-week battery/RSSI/LQI trending requires either the SPA /dataReadings/ endpoint (localStorage-scoped token, not accessible unattended) or a batch/history endpoint added to public_api. This report substitutes zone-level data delivery density from stitched room chart as the longitudinal proxy — since every zone is downstream of exactly one Node radio, zone-delivery density tracks mesh health at the Node granularity. Telemetría histórica por radio: El endpoint devices/<id>/chart/ solo devuelve ~2 días recientes; usamos densidad longitudinal por zona (proxy directo de salud de la radio parenta).

Thresholds: Signal p05 < −85 = CRIT, −75 to −85 = WARN; LQI p05 < 100 = CRIT; battery p50 < 3.0 V = CRIT, 3.0–3.4 = WARN. Umbrales: señal p05 < −85 CRIT; LQI p05 < 100 CRIT; bat < 3.0V CRIT.

Data caveats: Gateways & sinks show as "unclassifiable" (not silent) because they emit no device-chart series by design. TEROS-One (ter15) shares its base H343 serial with its parent Node — same physical device, listed twice in the API (one radio side, one sensor side). Total device-record count (1515) is the raw API count; dash-2 replacements were checked (none present in current inventory). Advertencias: Gateways/sinks emiten cero series por diseño; TEROS-One comparte serial con Node H343; total 1515 records raw sin duplicados dash-2.