Show Notes
When a Fiber Cut Becomes a Building Operations Crisis
In this episode of Built, Wired & Secured, Alex Morgan sits down with Michael Harrington to walk through a practical first-hour playbook for one of the most disruptive failures a building can face: a fiber cut. The scenario they open with is immediate and familiar. A landscaping crew clips a street-level conduit. Fiber to the building goes dark. Lobby point-of-sale terminals fail. Turnstiles stop reporting. The HVAC management console loses connectivity. A clinic tenant reports a full outage on a vital device. Suddenly, front desk staff are fielding angry calls, tenants are anxious, and the building team is forced to make expensive decisions under pressure.
The core message is simple: the first hour shapes the entire outcome. Teams that already know what matters most, who owns each decision, and what temporary connectivity options are available can restore critical services faster. Teams that do not have those answers lose time to confusion, delayed approvals, and incomplete information.
The First 10 to 20 Minutes: Rapid Triage
Michael’s first recommendation is operationally straightforward: start with a rapid inventory. Do not jump straight to the carrier call and assume everything else will sort itself out. First, identify what systems are down, which systems are safety critical, which systems are revenue critical, and who owns each one.
He frames the first-stage triage around three questions:
- Can this tenant operate offline?
- Does this system have a local fallback?
- Who is the responsible owner to make the call?
That triage determines whether the team can buy time with local mitigations or whether it needs immediate temporary backhaul. It also prevents a common outage mistake: treating every impacted system as equally urgent. In reality, a building team has to separate essential life-safety or business-critical systems from services that can tolerate temporary degradation.
The discussion also reinforces that outages are not only technical events. They are decision-making events. If the team does not know who owns a system or who can authorize a response, the technical path forward becomes secondary to organizational delay.
Why Ownership Confusion Causes Expensive Delays
One of the strongest operational lessons in the episode is that tenants, IT, facilities, and carriers often assume someone else is making the call. Michael shares an example of a 90-minute delay where the building manager expected the tenant’s IT director to authorize a portable fiber pull, while the tenant expected the building owner to fund it. Nothing happened until the assumption gap became obvious.
That kind of delay is avoidable. The solution is not more discussion during the outage. The solution is a predefined escalation path before the outage.
Michael recommends assigning a single point of decision for each class of service and documenting it in the runbook. That means knowing, in advance:
- Who can sign off on temporary cellular contracts
- Who can authorize a portable fiber truck
- Who owns tenant communications
- Who is responsible for contacting the carrier
He also emphasizes that these decisions should be tested in drills, not left on paper. A runbook only matters if people can execute it under pressure.
Temporary Connectivity Options and Their Trade-Offs
The episode then moves into the most practical part of the conversation: choosing the right temporary connectivity option. Alex and Michael review four options mentioned in the episode setup: cellular failover, microwave links, portable fiber, and staged carrier repairs.
Michael positions cellular failover as the fastest and lowest-friction option. A building can flip to a cellular gateway or managed SD-WAN path and quickly restore enough connectivity for email, tenant notifications, and some building systems. But he is equally clear about its limits. Throughput and latency are constrained, and teams with clinical devices or point-of-sale systems that involve PCI requirements need to confirm compliance before relying on cellular for anything beyond a short bridge.
Microwave occupies a different middle ground. It can deliver higher capacity quickly when line of sight exists and can be cost effective because it avoids digging. At the same time, it depends on clear sight lines, may require licensed links in some jurisdictions, and needs suitable mounting points. Michael’s guidance is to use it when the building needs more sustained capacity but immediate fiber access is unavailable.
Portable fiber trucks offer the highest capacity and the closest match to native service, but they come with the highest cost and the most logistical complexity. They may require permits, manhole or vault access, and coordination with a carrier or third-party service provider. Michael’s recommendation is direct: this is the right move when a tenant has high-availability needs or when temporary degradation is unacceptable. If a building may need this option, rates and emergency service-level expectations should be negotiated before a real outage happens.
Why Carrier Coordination Often Breaks Down
Michael shares a downtown building example where a contractor struck a conduit and the carrier estimated a six-hour dispatch. Instead of waiting, the building team executed its runbook. They notified the carrier using a preassigned ticket code, activated a negotiated cellular failover, and called for a portable fiber truck with an agreed cost cap. Because roles were already defined, tenant managers focused on manual workarounds while ownership handled authorization. Critical services were restored in under three hours instead of waiting the full six.
That story highlights two coordination failures he sees repeatedly:
- Unclear escalation paths
- Lack of accurate outage scope
If the carrier is not given precise manhole or route details, the wrong crew may be dispatched. If the runbook does not include an accurate site contact and access plan, a truck can arrive and still sit idle. Michael argues that an effective runbook must include geo references, gate codes, and a single escalation contact who actually answers the phone.
How to Practice Without Disrupting Operations
The playbook is not just for emergencies. It needs repetition. Michael recommends quarterly tabletop exercises that simulate a cut and walk the team through the notification chain, temporary backhaul activation, and tenant communications. He suggests keeping those drills short, roughly 30 to 60 minutes, and using them to validate contact lists, decision authorities, and realistic vendor lead times.
Where possible, he also recommends scheduling a non-disruptive cellular failover test during a maintenance window. That step helps confirm that the technical process works, not just the communication chain.
The Three Takeaways and Immediate Actions
The episode closes with a concise operating framework. First, diagnose impact in the first hour using the three-question triage focused on safety, revenue, and ownership. Second, choose temporary connectivity based on service need and make sure it bridges to permanent repair. Third, preassign decision owners, carrier contacts, and authorized spend thresholds so time is not lost during escalation.
Michael adds one more practical instruction: create a one-page first-hours checklist and keep it where people can actually find it. He specifically mentions a lobby binder, a facilities tablet, and an emailed PDF to tenant representatives. That checklist should include carrier ticket codes, on-call names, the prioritized systems list, and authorized spend limits.
The three immediate actions for listeners are equally direct:
- Draft or update a one-page first-hours checklist and distribution list
- Confirm or negotiate emergency terms with your carrier and a portable fiber vendor
- Run a short tabletop exercise with facilities, IT, and tenant representatives
The broader lesson from this conversation is that resilience is not only about having backup connectivity. It is about reducing uncertainty. When teams know what to assess, which systems matter most, who can approve action, and how to reach the right partners fast, they shorten downtime and reduce chaos. That is the difference between waiting on an outage and actively managing one.
Fiber Cuts Are Operational Crises, Not Just Network Problems
When a building loses fiber connectivity, the failure rarely stays confined to the network closet. In the episode Fiber Cut Playbook: Rapid Restoration When Connectivity Goes Dark, Alex Morgan and Michael Harrington frame the issue the way building teams actually experience it: as a fast-moving operational disruption with technical, financial, and tenant-facing consequences.
The opening scenario makes that clear. A crew digging near the street clips a conduit. Fiber to the building goes dark. Lobby point-of-sale terminals fail. Turnstiles stop reporting. The HVAC management console loses visibility. A clinic tenant reports a full outage on a vital device. Front desk staff start fielding complaints while the building team races to answer a series of urgent questions. Do they call the carrier and wait? Activate cellular failover? Bring in a portable fiber truck at significant cost? Every minute matters, and the wrong delay compounds the damage.
The conversation avoids vendor-specific configuration details and instead focuses on what decision makers need most in the first hours of an outage: a repeatable framework for triage, escalation, temporary restoration, and communication.
The First Hour Sets the Tone for the Entire Incident
One of the clearest takeaways from the episode is that the first hour shapes the whole outcome. When teams improvise under pressure, they usually lose time to uncertainty. They do not know which systems matter most. They do not know whether tenants can operate offline. They do not know who can authorize emergency spend. And they often discover too late that the carrier, the building team, and the tenant all assumed someone else was making the call.
Michael recommends starting with a rapid inventory in the first 10 to 20 minutes. Instead of reacting system by system, teams should list what is down, determine which systems are safety critical, identify which services are revenue critical, and confirm who owns each one. He reduces that process to three operational questions:
- Can this tenant operate offline?
- Does this system have a local fallback?
- Who is the responsible owner to make the call?
This is not abstract incident management language. It is a practical way to separate the urgent from the inconvenient. A team dealing with an outage cannot treat lobby reporting, HVAC visibility, point-of-sale operations, emergency phones, access control, and clinical equipment as interchangeable. The purpose of triage is to establish what must come back first, what can tolerate a temporary workaround, and what requires immediate escalation.
Ownership Is Often the Real Failure Point
The episode makes an important distinction between technical recovery and decision recovery. In many outages, the hardest problem is not identifying that fiber is down. It is getting the right person to approve the right action fast enough.
Michael shares a story about a 90-minute delay caused by an ownership gap. The building manager assumed the tenant’s IT director would authorize a portable fiber pull. The tenant assumed the owner would fund it. While both sides waited, time was lost.
That lesson matters because it highlights a recurring weakness in many commercial environments: systems cross organizational boundaries. A building may own one part of the infrastructure, a tenant may own another, and the carrier sits outside both. During calm periods, those boundaries are manageable. During an outage, they create friction.
The answer is preassignment. Michael recommends documenting a single point of decision for each class of service in the runbook. Teams should know in advance who can authorize a temporary cellular contract, who can approve a portable fiber truck, who owns tenant communications, and who is responsible for escalation with the carrier. This should not be left to interpretation during a live incident.
Just as important, those authorities should be tested in drills. A runbook is only useful if people trust it and can execute it without debate.
Choosing the Right Temporary Connectivity Option
The middle section of the episode is a concise guide to temporary backhaul options and the trade-offs each brings.
Cellular failover is presented as the fastest and most accessible option. If a building already has a cellular gateway or a managed SD-WAN path available, it can restore enough connectivity for basic operations quickly. That may be enough for email, tenant notifications, and some building systems. But Michael is careful not to overstate its value. Throughput and latency are limited, and organizations supporting clinical devices or point-of-sale workloads with PCI considerations need to confirm compliance before relying on cellular longer than necessary. In other words, cellular is often an effective bridge, but not always a full substitute.
Microwave is framed as the higher-capacity middle option. When line of sight exists, it can provide substantial connectivity without waiting for excavation or physical fiber restoration. It may also be cost effective for incidents that last longer than a brief interruption. The limits are practical: clear sight lines, possible licensing requirements depending on jurisdiction, and suitable mounting points. It is not instant in every environment, but it can be the right answer when the building needs more sustained bandwidth and fiber is not immediately available.
Portable fiber trucks deliver the closest thing to native service, but they also carry the highest cost and the most complex logistics. Permits, access to manholes or vaults, and third-party coordination can all slow deployment if no one has planned ahead. Michael’s advice is straightforward: if the building serves high-availability tenants or cannot tolerate meaningful degradation, emergency portable fiber terms should be negotiated before the outage, not during it.
The strategic point behind all three options is consistent throughout the discussion: temporary connectivity should match service need and bridge cleanly to permanent repair.
Carrier Coordination Determines Whether Good Plans Actually Work
Even when a team chooses the right restoration path, execution can still fail if carrier coordination is weak. Michael describes a downtown building where a contractor hit a conduit and the carrier estimated a six-hour dispatch. The building team did not wait passively. They followed the runbook, notified the carrier with a preassigned ticket code, activated negotiated cellular failover, and called for a portable fiber truck with a defined cost cap. Because roles were already assigned, tenant managers handled manual workarounds while ownership handled financial authorization. Critical services came back in under three hours instead of waiting the full six.
That example highlights what carrier coordination should look like in practice. Teams need more than a support number. They need precise escalation tools and accurate site data.
Michael identifies two recurring issues. First, escalation paths are often unclear. Second, outage scope is often described poorly. If the carrier does not receive accurate manhole or route details, the wrong crew may be sent. If the site contact information or access plan is incomplete, a truck can arrive and still be unable to proceed.
For that reason, the runbook should include geo references, gate codes, a single carrier escalation contact, and information that helps dispatch the correct resources the first time.
Tabletops Matter Because Incidents Expose Assumptions
Michael mentions drills more than once, and for good reason. Outage plans tend to look complete until people try to use them. That is why he recommends quarterly tabletop exercises that simulate a fiber cut and walk the team through notifications, temporary backhaul activation, and tenant communications.
He recommends keeping those exercises short, roughly 30 to 60 minutes, and using them to validate several specific elements:
- Contact lists
- Decision authorities
- Vendor lead times
- Escalation accuracy
Where feasible, he also recommends a non-disruptive cellular failover test during a maintenance window. That step verifies that technical recovery procedures work in addition to the communication chain.
Three Actions That Shorten Downtime
The episode closes with three immediate actions listeners can take this week.
- Draft or update a one-page first-hours checklist and its circulation list
- Confirm or negotiate emergency terms with your carrier and a portable fiber vendor
- Run a short tabletop exercise with facilities, IT, and tenant representatives
Michael adds useful guidance on where that checklist should live: in a lobby binder, on a facilities tablet, and as an emailed PDF to tenant representatives. It should include carrier ticket codes, on-call names, the prioritized systems list, and authorized spend limits.
That recommendation gets to the heart of the episode. Resilience is not only about adding backup paths. It is about reducing uncertainty during the first hour. Teams recover faster when they know what to assess first, what systems are truly critical, who has decision authority, and which temporary restoration options are realistic for the site.
For building operators, facilities leaders, tenant stakeholders, and anyone responsible for keeping business systems available, this episode is a reminder that restoration speed is often won before the outage happens. A short, practiced playbook can turn a chaotic fiber cut into a controlled response.
If this topic is relevant to your environment, listen to the full episode and use it as a prompt to review your own first-hour response plan. The strongest recovery plans are usually not the most complex. They are the ones people can execute quickly when connectivity goes dark.