Show Notes
Why change control matters in modern buildings
This episode tackles a problem that hits commercial properties every day: routine changes that are treated like isolated technical tasks even though they affect shared building systems. A firmware update, control tweak, switch change, or patch may look minor on paper. In practice, it can ripple across HVAC, access control, tenant connectivity, and front desk operations within minutes.
The conversation opens with a familiar failure scenario. A vendor pushes a midweek firmware update, an HVAC controller reboots, access systems begin failing to authenticate on several floors, and the building team starts taking calls before anyone can identify who approved the change or how it should be rolled back. That gap is the real issue. The outage is not just about the device or the patch. It is about missing ownership, poor coordination, and no shared operating process.
Why routine changes go sideways
The root cause discussed throughout the episode is siloed decision making. Facilities may work with one vendor for building automation. IT may own switching and wireless. Tenants may bring in their own contractors. Each group schedules work against its own priorities, often without a common change view.
- Facilities, IT, and tenant vendors operate on separate calendars
- Documentation is often incomplete, outdated, or hidden
- Dependencies between systems are assumed instead of tested
- No single escalation path exists when something fails
One of the clearest points in the episode is that shared dependencies are often invisible until something breaks. A BAS controller may appear separate from access control, but both may rely on the same VLAN or network path. A patch can be applied and closed out as routine, even though the real risk lives in the overlap between systems.
Balancing speed with operational risk
The discussion does not argue for bureaucracy. In fact, one of the strongest themes is that teams do not need a heavy approval board to improve reliability. What they need are clear thresholds and a practical ownership model.
Rather than treating every change the same, the guests recommend tiered ownership. Low risk work can move quickly with a short checklist. Higher risk changes, especially anything that touches authentication, routing, or emergency systems, require joint signoff and a more controlled maintenance window.
- Low risk changes can follow a quick preflight and rollback check
- Higher risk changes should involve both operations and IT
- The riskiest work should include stage testing and a scheduled outage window
- The process should stay small enough that teams actually use it
That framing matters for building operations. The goal is not to slow down maintenance. The goal is to make sure speed does not create preventable downtime.
Low effort controls that prevent most outages
A big strength of this episode is how practical the guidance stays. Instead of prescribing a complex governance framework, the conversation focuses on a few controls that listeners can put in place immediately.
The first is a shared change log that is visible to facilities, IT, and tenant services. Not a private spreadsheet. Not notes buried in email. A visible, searchable record that everyone checks before non-trivial work begins.
- A one line summary of the change
- The scheduled date and time
- Who is performing the work
- A reference to the rollback plan
The second control is a two minute preflight checklist. Before a change starts, teams confirm that backups exist, a telemetry baseline has been captured, the rollback person is available, and the communication path is active. If any one of those pieces is missing, the change pauses.
The third control is a clear escalation path. When incidents start unfolding, teams lose time looking for the right person. A single phone line or Slack channel with clear on call ownership is presented as a simple but high value fix.
What to measure before and after a change
The episode also gives a practical answer to the telemetry question. Teams do not need exhaustive monitoring to reduce risk. They need a small baseline set that can confirm whether the system is behaving normally before and after the change.
- Controller CPU and uptime
- Authentication success rates
- A few key BAS sensor readings
Just as important, the team should agree in advance on the threshold that triggers rollback. That removes debate in the middle of an incident. If the metric crosses the line, rollback happens immediately.
Real world examples from the conversation
One example shows how coordination and reversibility limit damage. A tenant requested a control tweak affecting ventilation schedules. The change was applied during business hours, sensors began showing unexpected set points 90 minutes later, complaints started rolling in within 30 minutes, and the team rolled back in 20. The issue still caused disruption, but the rollback plan kept it from becoming a much larger operational event.
The second example shows the opposite. A network refresh went forward without facilities being notified. Access control servers lost reachability, doors defaulted to fail safe during a busy delivery window, tenants were delayed, and staff had to manually escort people. That one took hours to unwind. The contrast is sharp: notification plus reversibility materially reduces impact.
A simple three step starter plan
The episode closes with a practical pilot plan for teams that want to improve change control without introducing heavy process.
- Create a visible shared change log for any non-trivial update
- Adopt a two minute preflight checklist that includes rollback confirmation
- Define one escalation path with clear on call responsibilities
The recommendation is to run those steps as a four week pilot, track prevented incidents, and use the outcomes to build internal buy in. That is a useful reminder for property and operations teams: policy alone rarely changes behavior, but visible operational results do.
Bottom line
The central lesson from this episode is simple. Change control in commercial buildings does not have to start with a formal committee. It starts with visibility, reversibility, and one shared path for communication when something goes wrong. Treat each change like a small maintenance job, confirm who owns it, and know exactly how to back out if the system starts drifting. That discipline protects uptime, reduces blame, and keeps routine work from turning into a building wide problem.
Change control is now a building operations problem
In modern commercial buildings, routine technical changes rarely stay routine for long. A firmware update on a controller, a network patch, a vendor handoff, or a tenant requested tweak can start as a small maintenance task and turn into an operational disruption within minutes. When that happens, the visible problem is usually a reboot, an authentication failure, or a system going unreachable. The real problem is that the change happened without a shared decision path, a rollback plan, or a clear owner.
That is the central point of this episode of Built, Wired & Secured. The conversation focuses on OT and IT change control in commercial buildings, but the lesson is broader than any single building system. Reliability depends less on paperwork and more on whether teams have simple habits that force coordination before a change goes live.
Why so many building changes fail in familiar ways
The opening example captures the problem fast. A vendor pushes a firmware update in the middle of the afternoon. Ten minutes later, the HVAC controller reboots. Access systems on several floors begin failing to authenticate. The front desk starts fielding calls. For an hour, nobody can point to one coordinated decision that allowed the change.
That scenario matters because it is not unusual. Buildings now depend on tightly connected systems: HVAC, access control, wireless, switching, controller networks, tenant connectivity, and monitoring all live close enough together that one isolated change can create a much wider effect. Yet the teams around those systems are often split across facilities, IT, outside vendors, and tenant contractors. Each group may be competent on its own. The failure happens in the handoff between them.
The episode identifies silos as the usual root cause. Facilities may own one vendor relationship for building automation. IT may manage switching and wireless. Tenants may bring in their own contractors and schedule work based on their own business deadlines. Documentation is often weak, incomplete, or buried in places other teams never see. When those conditions combine, overlapping changes become almost inevitable.
What looks isolated on paper can be a shared dependency in practice. A controller that seems unrelated to access control may sit on the same VLAN. A change that appears local may affect authentication, routing, or telemetry far beyond the device being touched. If nobody checks those dependencies before the work starts, teams are depending on luck.
The wrong debate: speed versus process
A useful part of the conversation is its rejection of the usual false choice. Too many teams frame change control as a battle between speed and bureaucracy. Facilities teams want to move fast. IT teams want more review. The result is often frustration on both sides.
The episode argues for something more practical: thresholds instead of committees. Not every change should require the same level of review. The right question is not whether change control exists. It is whether the team can quickly identify what kind of change is being made, who needs to approve it, and what happens if it fails.
That is where tiered ownership comes in. Low risk work such as non disruptive telemetry tweaks can be handled by operations with a short checklist. Changes that touch authentication, routing, or emergency systems need joint signoff from IT and operations. Higher impact work should include stage testing and a scheduled outage window. The process stays light, but the controls tighten as the risk goes up.
This is the kind of operating model commercial buildings need. It protects uptime without forcing every maintenance action through an oversized approval path.
The small controls that do most of the work
One of the best takeaways from the episode is that most outages are not prevented by large frameworks. They are prevented by a handful of small controls that teams consistently follow.
The first is a shared change log that everyone can see. The episode is clear on this point: the log cannot live in a spreadsheet hidden on someone’s drive. It needs to be visible to facilities, IT, and tenant services. Before any non trivial work begins, everyone should be able to answer the same four questions.
- What is changing?
- When is it changing?
- Who is doing the work?
- What is the rollback reference?
That level of visibility alone prevents a surprising number of avoidable incidents. It catches scheduling collisions. It exposes missing owners. It gives front line teams enough awareness to prepare for possible impact.
The second control is a two minute preflight. This is not meant to become a heavy meeting. It is a fast readiness check. Teams confirm that backups exist, a telemetry baseline has been captured, the person who will execute rollback is available, and the communication channel is set. If any one of those conditions is missing, the change pauses.
The third control is a named escalation path. This may be the most underrated point in the discussion. When systems start drifting or complaints come in, people lose valuable time figuring out who to call. A single phone line or Slack channel with clear on call ownership cuts through that confusion immediately.
Why telemetry baselines matter more than debate
The conversation also keeps the monitoring guidance grounded. Teams do not need a perfect observability stack to improve change outcomes. They need a short baseline captured right before the work starts and a pre agreed threshold that triggers rollback.
The suggested examples are simple and useful: controller CPU, uptime, authentication success rates, and a few important BAS sensor readings. Those metrics do not solve every problem, but they tell the team whether the environment is still behaving within expected bounds.
The key is to decide beforehand what counts as failure. Once the threshold is crossed, rollback should happen immediately. That prevents the usual mid incident argument where one person wants to wait, another wants to reverse course, and nobody wants to own the decision while the building keeps degrading.
Two examples that show the difference
The episode shares a strong contrast between a change that remained recoverable and one that spiraled because coordination was missing.
In the first example, a tenant requested a control tweak that affected ventilation schedules. The vendor applied the change during business hours. About 90 minutes later, sensors started reporting unexpected set points. Complaints rose within 30 minutes, but the team rolled back in 20. That is not a perfect maintenance outcome, but it is a controlled one. The building experienced impact, the issue was recognized, and reversibility kept the event from expanding.
The second example is more costly. A network refresh was scheduled without facilities being notified. Access control servers lost reachability. Doors defaulted to fail safe during a busy delivery window. Tenants were delayed and staff had to manually escort people. The repair took hours. The technical issue was serious, but the bigger failure was procedural. Notification and dependency awareness were missing before the work started.
That is the operating lesson. Most teams cannot eliminate every incident. They can, however, reduce the blast radius by improving notification, ownership, and rollback readiness.
A three step starter plan for real teams
The closing recommendation is refreshingly practical. Instead of asking teams to adopt a full governance program at once, the episode suggests a three step pilot.
- Create a visible shared change log for any non trivial update
- Use a two minute preflight checklist that includes rollback confirmation
- Define a single escalation path with clear on call responsibilities
Then run that process for four weeks. Track prevented incidents. Track near misses. Track whether teams actually used the controls. That is a much smarter rollout model than dropping a policy document into the organization and hoping behavior changes.
In commercial real estate and building operations, buy in usually comes from results. If a simple log, a quick preflight, and one escalation path prevent even a few service disruptions, the case for broader governance becomes easy to make.
What this means for operators, owners, and service partners
The broader message of the episode is that building technology needs to be managed like an operating environment, not a loose collection of vendor tasks. Commercial properties depend on uptime, tenant trust, and predictable service. That means every change should be treated like a small maintenance event with clear communication, known dependencies, and a real way back.
For organizations managing mixed OT and IT environments, this is where a Technology Partner mindset matters. Someone has to connect the building systems, the network, the vendors, and the operational realities of the property. Without that coordination, routine maintenance keeps turning into blame, downtime, and emergency response.
If your team is seeing last minute vendor changes, unclear ownership, or repeated disruption during maintenance windows, this episode offers a good place to start. Keep the process visible. Keep it simple. And make sure rollback is part of the plan before the change begins.
Listen to the full episode for the complete discussion, then take the three step pilot into your next maintenance cycle and see what it prevents.