Show Notes
Why firmware and update debt matters in networked buildings
Modern commercial buildings depend on connected systems that quietly support daily operations: access control, digital directories, climate control platforms, gateways, controllers, and elevator-related systems. In this episode of Built, Wired & Secured, Alex Morgan and Michael Harrington explain why software and firmware lifecycle management is no longer a background IT task. It is an operational discipline that directly affects tenant experience, uptime, and business continuity.
The conversation opens with a realistic failure scenario: tenants arrive at a high-rise while a digital directory and access control platform both reboot after a background firmware push. At the same time, a delayed patch for a climate control controller fails, elevators enter degraded mode, deliveries are delayed, and the property team is left managing the fallout. The point is clear: update debt is not an abstract technical issue. It becomes visible when building systems fail in production.
How update debt builds over time
Michael describes update debt as something that usually starts small. A controller installed during a tenant fit-out stays on a firmware version that matched the integrator's original testing. Over time, vendors release security fixes and feature improvements, but updates are applied unevenly. Some devices get touched, others do not. That creates version drift, unsupported devices, and incompatibilities with newer management platforms.
He also introduces a useful phrase: inventory rot. Teams lose visibility into what is installed, what version it is running, when it was last updated, and who owns it. Without that basic record, surprises multiply.
- Devices remain on unsupported firmware longer than expected
- Ownership becomes unclear between facilities, IT, and vendors
- Critical building systems develop hidden single points of failure
- Emergency work becomes more expensive because teams are reacting instead of planning
What teams should track first
The episode argues for a lightweight but disciplined inventory. Michael recommends capturing four essentials for key assets: device type, installed firmware, last update date, and owner. He stresses that teams do not need endless spreadsheets tracking every patch forever. They need visibility into high-impact systems such as access control servers, elevator controllers, and core BMS gateways.
That simple ownership model makes quarterly reviews and maintenance scans more useful, because teams know where the highest operational risk sits.
Balancing uptime, security, and disruption
One of the strongest sections of the episode focuses on prioritization. When teams cannot update everything at once, they need a risk-based framework. Michael suggests evaluating each update with three questions:
- What is the impact if the update is missed?
- What is the likelihood of exploitation or failure?
- What is the cost or disruption required to apply it?
That approach keeps critical security fixes at the top of the list while allowing lower-risk feature updates to be grouped into planned maintenance windows. For property teams, this is a practical governance model that protects uptime without ignoring security exposure.
Staging, phased rollouts, and rollback planning
The discussion gets especially practical around testing. Michael recommends a minimal viable staging approach for teams without large lab environments. The first step is a small on-site sandbox with representative devices and the same network segmentation as production. The second is a phased rollout: start with a lower-risk subset, monitor for 48 to 72 hours, then expand.
For critical systems, he emphasizes vendor change control, test scripts, and validated rollback plans. A rollback strategy is not something to design during an outage. It should be ready before the rollout begins.
Coordination across vendors, IT, and facilities
Another major takeaway is that update debt is organizational as much as technical. Shared runbooks work best when roles are explicit and concise. Each update event should have a clear owner, decision gates for approval and sign-off, and communication templates ready for email, SMS, and on-site teams.
The episode also highlights vendor agreements as a control point. Property teams should require notification windows for breaking changes, support lifecycle clarity, release notes, known incompatibilities, and rollback images. Those contract details reduce friction when a critical update arrives under pressure.
Real-world examples and immediate checklist
Michael shares two short examples. In one successful rollout, a three-building access control environment was upgraded over six weeks using a sandbox, nightly maintenance windows, tracked metrics, and a prepared one-click rollback. In another case, an owned-by inventory and approval gate helped the team postpone a critical patch by 24 hours to avoid badge failures during a scheduled building event.
The closing checklist is direct and actionable:
- Maintain inventory hygiene
- Use risk-based prioritization
- Stage updates and roll them out in phases
- Set predictable maintenance windows and communicate them clearly
- Validate rollback plans and test scripts
- Build lifecycle requirements into vendor contracts
- Budget operating expense for recurring lifecycle work
The core message of the episode is simple: start with ownership. Know who is responsible for firmware in your buildings, and know when your next maintenance window is. Once those answers are clear, the rest of the lifecycle process becomes manageable.
Firmware and update debt is a building operations problem, not just an IT problem
Commercial buildings now depend on networked systems that shape daily operations in visible and invisible ways. Access control, digital directories, climate control platforms, gateways, controllers, and other connected systems all rely on software and firmware that must be maintained over time. In this episode of Built, Wired & Secured, Alex Morgan speaks with Michael Harrington about the hidden operational cost of that reality: update debt.
The discussion makes one point especially clear. When firmware and software lifecycle work is ignored, the risk does not stay tucked away in a dashboard or a maintenance log. It shows up at the worst possible moment, often in front of tenants, visitors, deliveries, and building staff.
The opening scenario captures that perfectly. A high-rise starts the day with an unexpected reboot on the digital directory and access control platform after a background firmware push. At the same time, a delayed security patch fails on a climate control controller, elevators move into degraded mode, tenants cannot badge in, and the property team is immediately in response mode. That is not an IT inconvenience. It is a direct business and operational disruption.
How update debt accumulates in the real world
Michael explains that update debt rarely begins with a dramatic mistake. More often, it starts with a reasonable decision that never gets revisited. A controller installed during a fit-out remains on the version originally validated by the integrator. New features and security fixes are released later, but updates happen unevenly across the environment. Some devices are touched, others are not. Over time, version drift sets in.
That drift creates a chain of practical problems. Devices become unsupported. Management platforms evolve beyond older firmware. Interdependencies remain undocumented. Teams inherit a portfolio where no one has a complete picture of what is current, what is aging, and what is vulnerable.
One of the most useful terms from the episode is inventory rot. It describes what happens when teams no longer maintain a living record of device versions, update history, and ownership. When that inventory degrades, decision-making degrades with it. Teams cannot prioritize what they cannot see, and they cannot coordinate what no one clearly owns.
Start with a lightweight inventory, not a perfect one
A practical strength of this episode is that it does not pretend every property team has unlimited time or resources. Michael does not call for massive documentation projects or endless site walks. Instead, he recommends starting with a lightweight inventory that tracks four things for key systems: device type, installed firmware, last update date, and owner.
That structure creates enough visibility to reduce surprise. It also supports simple scans during maintenance and ties naturally into quarterly reviews. The point is not to document every patch level forever. The point is to know where the highest-impact systems are and keep those visible. In the episode, examples include access control servers, elevator controllers, and core BMS gateways.
For decision-makers, this is important because it reframes lifecycle work as governance. A clean inventory is not clerical overhead. It is a control that reduces emergency work, tenant disruption, and avoidable downtime.
A better way to prioritize updates
One of the central questions in the episode is how teams should decide what to update first when they cannot do everything at once. Michael recommends a risk-based prioritization framework built around three questions:
- What is the impact if this update is missed?
- What is the likelihood of exploitation or failure?
- What is the cost or disruption required to apply it?
That framework is effective because it balances security and uptime instead of treating them as competing silos. A high-impact security fix on a control server may deserve immediate attention. A lower-risk feature update may be better grouped into a planned maintenance window. This approach gives property teams a decision model they can repeat across a portfolio.
That repeatability matters. Without a framework, updates tend to be driven by noise, urgency, or vendor pressure. With a framework, they can be aligned to business risk, operational timing, and tenant impact.
Maintenance windows only work when expectations are clear
The episode also addresses a common tension: maintenance windows are necessary, but tenants do not like surprises. Michael's answer is consistency and expectation setting. He recommends establishing a cadence, such as monthly late-night windows for non-critical updates and immediate windows for critical security patches.
That cadence does more than simplify scheduling. It builds credibility. When tenants understand when maintenance typically happens, who to contact, and what to expect if a short disruption occurs, the process becomes easier to manage. Communication becomes part of resilience.
Just as important, Michael stresses the value of having tested rollback plans. If stakeholders know an update has been staged and a rollback is ready, tolerance for controlled maintenance increases. That is a practical lesson for any building team managing tenant-facing systems: confidence comes from preparation, not from optimistic messaging.
What staging looks like when resources are limited
Property teams often know they should test updates first, but many do not have a full lab environment. The episode offers a realistic alternative. Michael describes a minimal viable staging model with two parts.
First, maintain a small on-site sandbox with representative devices and the same network segmentation as production. Second, use phased rollouts. Update a low-risk subset first, monitor for 48 to 72 hours, and then expand if results are stable.
This matters because it reduces blast radius. A staged process gives teams a controlled way to detect regressions, validate assumptions, and confirm that rollback works before a problem reaches the wider building population. For critical systems, the episode adds another layer: vendor change control and test scripts that help recreate failures and confirm successful rollback.
That is a strong operational model because it turns updates from one-time events into managed change processes.
Coordination is as important as technology
Another major insight from the conversation is that lifecycle failures are often coordination failures. IT, facilities, and vendors may all touch the same environment but operate with different assumptions. Michael recommends concise runbooks with explicit roles. Each update event should have a clear owner. Decision gates should define who approves rollout, who signs off after staging, and who triggers rollback if needed.
He also recommends keeping communication templates ready in advance, including email, SMS, and on-site radio scripts. During a live event, standard language reduces confusion and speeds response.
This is where mature building operations begin to look less reactive and more governed. Strong runbooks do not add bureaucracy for its own sake. They remove ambiguity when time matters.
Vendor agreements can reduce future friction
The episode does not stop at process. It also points to contracts as a leverage point. Michael advises building lifecycle expectations into vendor agreements, including notification windows for breaking changes, support lifecycle clarity, escalation paths, firmware release notes, known incompatibilities, and rollback images.
Those details can feel secondary when systems are first installed. Later, they become essential. When an urgent patch arrives, the quality of those vendor commitments directly affects how fast and safely a team can act. This is a valuable reminder for owners and operators: procurement decisions influence long-term operational resilience.
Two examples that show the value of preparation
Michael shares two short examples that bring the framework to life. In one successful rollout, a portfolio of three buildings with a common access control headend was updated through a six-week phased plan. The team built a sandbox, staged the vendor firmware, used nightly maintenance windows, tracked login latency and badge acceptance, and prepared a one-click rollback. That process caught a regression in a secondary module before it reached tenants.
In the second example, a vendor issued a critical patch that required a gateway reboot during business hours. Because the team had an owned-by inventory and an approval gate, they recognized a conflict with a scheduled building event and postponed the update by 24 hours. That short delay prevented mass badge failures during the event.
Both stories reinforce the same point: ownership and process prevent incidents from becoming public failures.
The operational checklist property teams can use now
The episode closes with a practical checklist that property and facilities teams can apply immediately:
- Keep inventory current with device type, firmware, owner, and last update date
- Prioritize updates by impact, likelihood, and disruption
- Use a sandbox and phased production rollouts
- Set predictable maintenance windows and communicate them clearly
- Validate rollback plans and test scripts before rollout
- Require vendor notifications, release notes, and support lifecycle clarity
- Budget recurring lifecycle work as operating expense rather than an afterthought
The final takeaway is especially useful because it is so simple. Start with ownership. Who is responsible for firmware in your building today? And when is your next maintenance window? If those two answers are unclear, update debt is already harder to manage than it should be.
If this episode speaks to the realities you face across building operations, vendor oversight, and long-term planning, listen to the full conversation for the complete framework and examples.