Fire raw_device_initialized after registering endpoints

[zigpy-review-bot]

Follow-up to #179, from #179 (comment).

start_network() fires raw_device_initialized before register_endpoints(), so every listener sees a coordinator that still only has the XBee endpoint 0xE6. zigpy.appdb.PersistingListener.raw_device_initialized() clones the device synchronously inside the event, and _raw_device_initialized_internal() is the only code path that ever writes endpoints/clusters rows (PersistingListener.device_initialized() is a pass), so endpoints 1 and 2 never reach the database. ZHA's handler (zha/application/gateway.py:416) puts device.get_signature() into the event it emits, so that signature is missing them too.

Moving the event below register_endpoints() fixes both. The self.devices[...] insertion has to stay above register_endpoints(), since add_endpoint() resolves self._device out of self.devices — so only the listener_event() call moves.

Not a regression from #179: the event has been in this position all along. #179 is what makes it matter, because it gives endpoints 1 and 2 real profiles, device types and clusters for the first time. Impact is low in practice — start_network() rebuilds the coordinator device on every startup, so the incomplete rows self-heal and nothing user-visible reads them today.

This is stacked on #179 — it branches off that PR's head, so the diff shown here includes #179's commits until it merges. Only 1711d8f belongs to this PR. Happy to rebase onto dev once #179 is in.

Testing

• pytest: 108 passed, 100% coverage, DeprecationWarning as errors. The new test (test_start_network_raw_device_initialized_after_endpoints) asserts the listener sees all endpoints at event time, and fails on the unfixed ordering.

• pre-commit: all hooks pass.

• Probed against a real sqlite DB driving the actual start_network() sequence, at Support zigpy 2.0.0 #179's head:

  before:  endpoints_v15 [(230, 49413)]                          clusters: 2
  after:   endpoints_v15 [(1, 260), (2, 49246), (230, 49413)]    clusters: 13
  

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 100.00%. Comparing base (f5f9d6f) to head (1711d8f).

Additional details and impacted files

@@            Coverage Diff            @@
##               dev      #180   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files            7         7           
  Lines          718       733   +15     
=========================================
+ Hits           718       733   +15     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[TheJulianJES]

@zigpy-review-bot Is this even an issue it's not persisted to the DB (or shown in ZHA diagnostics)? Isn't it just needed to register, which is anyway done on every restart? Also do review rounds on this and push more commits if it makes sense to adjust anything.

[zigpy-review-bot]

Checked this properly — you're right, and it's worse than "not much of an issue": the premise I gave in #179 and in this PR's description doesn't hold, on either half.

The DB gets the complete coordinator with or without this PR. raw_device_initialized fires twice per startup and I only measured the first one. After start_network() returns, zigpy's initialize() calls _persist_coordinator_model_strings_in_db() (zigpy/application.py:288), which calls device_initialized(self._device) (:1808); the hand-built XBeeCoordinator has original_signature is None, so that runs _finalize_device() (:664) and fires raw_device_initialized again (:626) — with every endpoint, since register_endpoints() has long since run. appdb upserts, so the incomplete first write is immediately overwritten. My earlier probe stopped at start_network(), which is exactly why the rows looked lost.

Probe: both orderings against a real sqlite DB, driving start_network() plus the core persist step that always follows it (zigpy 2.1.0)

unfixed:  fires during start_network:  [{230: [6, 32774]}]
          after the core persist step: {230: [6, 32774], 1: [0, 6, 10, 25, 37, 1281], 2: [0]}
          DB endpoints_v15: [(1, 260), (2, 49246), (230, 49413)]   clusters rows: 13

PR #180:  fires during start_network:  [{230: [6, 32774], 1: [0, 6, 10, 25, 37, 1281], 2: [0]}]
          after the core persist step: (identical)
          DB endpoints_v15: [(1, 260), (2, 49246), (230, 49413)]   clusters rows: 13

Identical end state.

And ZHA never sees this event at all. It registers itself as an application listener at zha/application/gateway.py:267 — after await self.application_controller.startup(auto_form=True) on :258. Both coordinator fires happen inside startup(), so ZHA misses both. Its coordinator device comes from _find_coordinator_device() → get_device(nwk=0x0000), the live object that already has all the endpoints. So the get_signature() claim in the description is just wrong.

Diagnostics likewise — get_diagnostics_json() walks self.device.non_zdo_endpoints (zha/zigbee/device.py:1704), the live device; it never reads the DB or the event payload.

And "just needed to register, done on every restart" is exactly right. For XBee, add_endpoint() doesn't even talk to the radio — it just materialises the endpoints on the zigpy device object (zigpy_xbee/zigbee/application.py:239-248), so that state is rebuilt from scratch on every start_network() and the DB copy is pure redundancy for the coordinator.

Review rounds

Both reviewers came back clean on the diff itself — Copilot found nothing, and the Claude reviewer confirmed the reorder is correct, consistent with the other radio libs, and that the new test does provably fail on the unfixed ordering (re-verified). The change isn't wrong. It just doesn't fix anything.

What did come out of the rounds is the double-fire above. It's pre-existing on dev, and it's the real oddity here: zigpy-xbee is the only radio library that hand-fires raw_device_initialized for its coordinator. bellows, zigpy-znp and zigpy-deconz all call register_endpoints() and then let the coordinator go through the normal path (schedule_initialize() / DeconzDevice.new), where core's _finalize_device() fires the event once, after the endpoints exist.

Suggestion

I'd close this. Two alternatives if you'd rather not:

• Repurpose it — drop the manual self.listener_event("raw_device_initialized", xbee_dev) from start_network() entirely and rely on core's _finalize_device() fire, matching the other three radio libs. That removes a redundant duplicate DB write per startup and makes the ordering question moot. The end state is unchanged either way (probed above), so it looks safe — but it's a behaviour change on a released radio lib, so that's your call.
• Keep it as event-contract hygiene — in which case the description and the new in-code comment both need rewriting, since both currently justify the change with DB and ZHA effects that don't exist.

I deliberately pushed nothing: polishing a branch whose premise I'd just refuted seemed like the wrong move, and choosing between close and repurpose is yours. Re-mention me with a direction and I'll carry it out.