GDS Technology — Built, Wired and Secured podcast banner
Watch on YouTube →
Built to Operate: How a Lean On‑Site Ops Team Cuts Outage Recovery Time by ~30%
Episodes Built
Episode 5

Built to Operate: How a Lean On‑Site Ops Team Cuts Outage Recovery Time by ~30%

March 19, 2026
Key takeaways
  • A 30-day pilot with three defined roles helped reduce mean time to recovery by roughly 20 to 35 percent.
  • Vendor escalations fell about 30 to 50 percent when incident triage and communications were owned internally.
  • Short runbooks, a single ticket queue, and simple evidence capture created enough data to justify the pilot.
  • Sustainable on-call design depended on capped rotations, clear thresholds, and disciplined handoffs.
  • Operational success also required physical readiness, including spare parts access and a formalized parts workflow.

Show Notes

Episode Overview

This episode of Built, Wired & Secured opens with a familiar nightmare for commercial property teams: a 2:00 a.m. outage, tenants calling in, emergency lighting cycling, vendors hours away, and the property team stuck in a reactive loop. From there, the conversation turns practical. The focus is not theory, vendor promotion, or staffing politics. It is operational design: how a small, well-scoped on-site tech ops team can reduce chaos, speed up recovery, and give owners measurable evidence that a pilot is worth funding.

Alex Morgan frames the discussion around a tightly scoped pilot, and Michael Harrington, a senior property manager, explains what late-night incidents actually look like from the property side. The issue is not just one failed system. It is the cascade that follows. Reception loses the tools it depends on. Frontline staff need direction immediately. Tenants become anxious. Contractors get dragged into rushed fixes. And when the process is driven by call trees instead of clear ownership, organizations often pay more while still getting slower results.

The central idea of the episode is simple: a lean on-site ops function does not need to be large to be effective. If the roles are clearly defined, the on-call model is sustainable, and the team has short playbooks to follow, outage recovery can become far more predictable.

Why Reactive Vendor Loops Become Expensive

One of the strongest themes in the episode is that reactive vendor management creates friction at exactly the wrong moment. When an incident hits after hours, every delay compounds:

  • Tenants lose confidence quickly when communication is unclear.
  • Front desk and reception teams get flooded with questions they are not equipped to answer.
  • Property staff spend valuable time figuring out who owns the next step.
  • Vendors are often pulled in before the issue has been properly scoped.
  • Repeat call backs become more likely when temporary fixes are not tied to structured follow-through.

Michael describes the 2:00 a.m. call as a dozen cascading problems rather than a single technical event. That distinction matters. The business problem is not only downtime. It is confusion, wasted motion, repeat escalations, and a loss of tenant trust.

The Lean Pilot Model

The pilot discussed in the episode started intentionally small: one property, three people, and a 30-day window. Instead of trying to redesign operations across an entire portfolio, the team focused on a contained environment with clear success criteria.

The three roles were:

  • A site technician for quick diagnostics and temporary remediation.
  • An escalation lead to triage incidents and decide when vendors were actually needed.
  • A vendor coordinator to handle communications and manage SLA timing.

Success was defined upfront using a short list of measurable outcomes:

  • Reduce mean time to recovery.
  • Cut vendor escalations.
  • Improve tenant call backs by reducing repeat issues and confusion.

By week three, the team saw real movement. MTTR dropped by roughly 20 to 35 percent depending on incident type. Vendor escalations fell by around 30 to 50 percent. Just as important, tenants reported fewer repeat calls. The episode makes the point that those tenant-facing improvements are often the clearest signal that operations are becoming more stable.

What the Three Roles Actually Do

A useful part of the conversation is how clearly the roles are defined without turning them into broad job descriptions. Each role exists to remove a specific bottleneck.

  • Site technician: Handles first-response diagnostics, checks power paths, verifies basic network reachability, captures evidence, and performs safe temporary remediation. The role is not positioned as a deep systems expert. It is positioned as someone who knows how to isolate problems, patch quickly, and hand off cleanly.
  • Escalation lead: Serves as the incident decision-maker. This person decides whether the site technician can resolve the issue, whether a vendor should be engaged, and how the incident should be communicated downstream.
  • Vendor coordinator: Owns the vendor relationship and the SLA clock. This role keeps pressure on ETA, status, next steps, and tenant updates so the rest of the team is not distracted by logistics.

Together, those roles reduce the number of handoffs and prevent incidents from bouncing between people who lack context.

On-Call Design Without Burning Out the Team

The episode also addresses one of the hardest parts of any operational model: after-hours coverage. Too much rotation leads to fatigue. Too little coverage sends organizations right back to dependency on external responders.

The pilot used a lean design built around:

  • Short, predictable weeks.
  • Mostly weekday core coverage.
  • An on-call split for nights and weekends.
  • A threshold-based model where the site technician handles immediate issues up to a defined limit.
  • Escalation lead involvement only for more complex decisions.
  • Vendor coordinator engagement only when external support is actually required.

Michael notes that weekly on-call time was capped, night coverage rotated so no one carried nights more than one week in four, and handoff rituals were emphasized to avoid ambiguity. That is an operational detail with real business value. A tired team that inherits unclear information will lose the time savings the model is supposed to create.

The Minimal Tool Set That Made the Case

Another useful takeaway is how lightweight the pilot’s tooling remained. The team did not need an elaborate platform stack to prove value. Instead, they used:

  • A single ticket queue.
  • Templated incident types.
  • An evidence bucket with photos, screenshots, and timestamped notes.
  • Three runbooks for the most common incident classes: power interruptions, network handoffs, and access control issues.
  • A small dashboard comparing pilot performance against baseline.

The KPIs were tightly focused:

  • MTTR
  • Vendor escalation rate
  • Repeat tenant call backs within 24 hours
  • Time to first response

In several cases, those results justified pilot cost within a single lease cycle. That matters because the conversation keeps returning to owner-level concerns: uptime, operational predictability, and measurable return.

What a Good Runbook Looks Like at 2:00 A.M.

Runbooks in this episode are not long documents. They are one-page decision tools built for real conditions. Michael describes a structure that includes:

  • Safety check and securing the area.
  • Confirming scope: power, network, or both.
  • Capturing evidence and assigning a timestamped ticket.
  • Following a short diagnostic checklist.
  • Applying safe temporary remediation.
  • Escalating with packaged information if unresolved.

The point is clarity. One page. Prioritized steps. Bolded escalation thresholds. That kind of operational discipline reduces back-and-forth with vendors and helps first responders move faster under pressure.

Where One Pilot Fell Short

Not every pilot in the discussion was a win on the first try. One property failed to move MTTR meaningfully because the team assumed spare parts and staging access would be consistent. The technician could diagnose and stabilize, but permanent resolution stalled when parts were unavailable. That created repeat vendor calls and weakened early results.

The fix was operational, not abstract:

  • Create a small contingency stock.
  • Formalize a parts request workflow.
  • Expand the vendor coordinator’s remit to manage expedited orders.

After those changes, MTTR returned to the expected range. It is a strong reminder that outage response is shaped by physical and procurement realities, not just incident handling logic.

Three Immediate Actions from the Episode

  • Map the top five failure modes affecting tenants and estimate current MTTR.
  • Choose one property and scope a 30-day pilot with a site technician, escalation lead, and vendor coordinator.
  • Create one-page runbooks for top incident types and build a simple dashboard to track MTTR and vendor escalations.

Final Takeaway

The closing message is direct: tenants feel the impact immediately, and owners see the ROI in uptime. Start small, measure hard, and let the numbers determine whether the model should scale. This episode is a practical guide for property teams that want to move from reactive vendor dependence to faster, more controlled recovery when incidents hit.

Deeper dive

Built to Operate: Why Lean On-Site Ops Can Change Outage Recovery

At 2:00 a.m., building problems feel bigger than they look on paper.

An entire floor loses connectivity. Emergency lights cycle. Tenants start calling reception. The vendor is still two hours away. A contractor’s night team is tied up somewhere else. The property team is left trying to answer the same question from three directions at once: who can make a quick diagnosis and get this under control?

That opening scenario from this Built, Wired & Secured episode captures a reality many property teams know too well. Outages are not just technical events. They become communication failures, escalation failures, and decision-making failures when no one on-site has clear first-response ownership.

This episode stays tightly scoped on a practical solution: a lean on-site tech ops pilot built to reduce recovery time, cut unnecessary vendor escalations, and improve the tenant experience during incidents. It does not drift into broad staffing theory or legal advice. Instead, it offers an operating model that commercial property leaders can evaluate with measurable outcomes.

The Real Cost of Reactive Vendor Loops

Michael Harrington describes the 2:00 a.m. outage from an operator’s point of view, and his framing is useful. When a critical system goes down, the problem is not isolated to the technology itself. It spreads quickly.

  • Tenants panic when they do not know what is happening.
  • Reception teams are forced into triage mode.
  • Frontline staff need direction before they can give updates.
  • Contractors are often pulled into rushed, reactive fixes.
  • Repeated call backs increase when no one has time to diagnose properly.

That is why reactive vendor loops become a business problem, not just an operations problem. When every issue starts with call trees and fragmented communication, organizations spend more time coordinating than recovering. Even when a vendor eventually resolves the issue, the path there is inefficient, expensive, and frustrating for tenants.

In mixed-use and commercial environments, the consequences can extend beyond internet access. Michael points out that reception connectivity, card access, and even HVAC controls may be affected depending on the incident. That means outages can disrupt both business continuity and the day-to-day experience of the building.

What the Pilot Looked Like

Rather than trying to fix everything across a portfolio, the pilot described in the episode started with one property, three people, and 30 days. That decision matters. Small scope creates operational clarity. It also makes it easier to establish a baseline, compare results, and decide whether the model deserves broader investment.

The pilot centered on three compact roles:

  • Site technician
  • Escalation lead
  • Vendor coordinator

The success criteria were also intentionally narrow:

  • Reduce mean time to recovery
  • Cut vendor escalations
  • Improve tenant call backs

By week three, the numbers were already meaningful. MTTR dropped roughly 20 to 35 percent depending on the incident type. Vendor escalations fell about 30 to 50 percent. Tenants reported fewer repeat calls.

That last metric deserves attention. Faster technical recovery matters, but fewer repeat tenant calls often reflect something deeper: better communication, better ownership, and fewer unresolved loose ends.

The Three Roles That Remove Friction

One of the most practical parts of the episode is how cleanly the team roles are defined. This is not about adding headcount without structure. It is about assigning the right kind of ownership at the right point in the incident.

1. Site Technician

The site technician is the first responder. This person checks power paths, tests basic network reachability, captures evidence, and performs safe temporary remediation where possible. The role is not described as a deep specialist. Instead, it is someone who can isolate the problem quickly, stabilize what they can, and create a strong handoff when escalation is needed.

That distinction is important because many incidents lose time at the very beginning. If the first person on scene can gather evidence and narrow the scope quickly, the entire response improves.

2. Escalation Lead

The escalation lead is the incident decision-maker. This role determines whether the issue stays local, whether the site technician has enough to resolve it, and when a vendor needs to be engaged. It also controls downstream communications.

Without this role, organizations often default to over-escalation. Problems get handed to outside resources before anyone has a clear picture of what is actually wrong.

3. Vendor Coordinator

The vendor coordinator owns the vendor relationship and the SLA clock. This role manages ETA, status, handoff quality, and tenant update timing. That may sound administrative, but it is operationally critical. When vendors are involved, someone needs to stay focused on accountability and communication so the rest of the response team can keep working the issue itself.

Together, these three roles reduce friction by limiting the number of handoffs and keeping incidents from bouncing between people who lack context.

Designing On-Call Coverage That Stays Lean

After-hours design is where many good ideas fall apart. Too many rotations create burnout. Too little coverage pushes the team right back into dependence on external responders.

The model in this episode stays disciplined. Coverage is mostly built around weekday core support, with nights and weekends handled through a split on-call model. The site technician responds first within defined thresholds. The escalation lead is pulled in only when the issue requires higher-level decisions. The vendor coordinator becomes active when external support is required.

Several design choices stand out:

  • Short, predictable weeks
  • Capped weekly on-call time
  • Night rotations no more than one in four
  • Clear handoff rituals between staff

Those details matter because response quality drops when tired staff inherit unclear situations. The episode makes a strong case that sustainable design is part of faster recovery. You do not get sharp incident handling from an exhausted team.

Minimal Tools, Strong Evidence

Another strength of the case study is that it does not depend on a complex software stack. The team used a lightweight tool set to generate enough operational evidence to justify the pilot.

The setup included:

  • A single ticket queue
  • Templated incident types
  • An evidence bucket with photos, screenshots, and timestamped notes
  • Three runbooks for power interruptions, network handoffs, and access control issues
  • A small dashboard showing pilot performance versus baseline

The KPIs were equally simple:

  • MTTR
  • Vendor escalation rate
  • Repeat tenant call backs within 24 hours
  • Time to first response

In several cases, those metrics justified the pilot cost within a single lease cycle. That is a powerful business framing. It moves the conversation away from abstract operational improvement and toward measurable return tied directly to uptime and tenant experience.

Why Short Runbooks Matter

When Alex asks what a runbook looks like “in the dark hallway at 2:00 a.m.,” the answer gets to the heart of operational design. A good runbook at this scale is short, prioritized, and usable under pressure.

The structure described in the episode includes:

  • Safety check and secure area
  • Confirm scope: power only, network only, or both
  • Capture evidence and assign a timestamped ticket
  • Follow a three-step diagnostic checklist
  • Apply temporary remediation if safe
  • Escalate with packaged information if unresolved

That format removes ambiguity. It also improves vendor interactions because escalation happens with cleaner information, not vague descriptions and scattered notes.

What One Underperforming Pilot Revealed

Not every pilot in the discussion hit its targets immediately. One property saw little MTTR improvement because the team assumed spare parts and staging access would be readily available. As a result, the technician could diagnose and temporarily stabilize issues, but permanent fixes were delayed by parts availability. That created repeat vendor calls and weakened performance.

The correction was straightforward:

  • Build a small contingency stock
  • Formalize a parts request workflow
  • Expand the vendor coordinator’s remit to manage expedited orders

Once those changes were made, MTTR dropped back into the expected range.

This is an important lesson for owners and operators. Operational models succeed or fail in the real world, where storage, staging, procurement, and physical access all shape response outcomes. A lean team still needs the physical support structure to act effectively.

Three Immediate Actions for Property Teams

The episode closes with three clear next steps for listeners who want to test the model rather than debate it endlessly.

  • Map the top five tenant-impacting failure modes and estimate current MTTR.
  • Choose one property and run a 30-day pilot using the three compact roles.
  • Create one-page runbooks and a minimal dashboard to track MTTR and vendor escalations.

That advice is practical because it creates evidence fast. If the model works, the numbers will show it. If it does not, teams will know where the design broke down and what must be adjusted before scaling.

Why This Matters Beyond the Incident

The closing line from Michael Harrington is the clearest summary of the episode’s business case: tenants feel this immediately, and owners see the ROI in uptime.

That is the larger takeaway. A lean on-site ops function is not just about troubleshooting faster. It is about reducing uncertainty during incidents, improving the quality of communication, and replacing reactive vendor dependence with a more controlled response model.

For commercial real estate teams, that has strategic value. Faster recovery protects tenant confidence. Fewer unnecessary escalations reduce operational waste. Better evidence and cleaner handoffs create more informed decisions about where to invest next.

If your property team is stuck in late-night call trees and repeat outage cycles, this episode offers a grounded place to start: start small, measure hard, and let the numbers tell you whether the model should scale.

To hear the full case study and the operational details behind the pilot, listen to the episode on the Built, Wired & Secured show hub at https://builtwiredsecured.com/episodes/lean-on-site-ops-cuts-outage-recovery-time.