Print-Ready Version

Business Continuity & Disaster Recovery

Our Experience

We have responded to more than 5,800 disaster prevention and recovery engagements, including the 9/11 World Trade Center terrorist attack, the first World Trade Center bombing, the Oklahoma City bombing, Hurricane Andrew, the Hinsdale and Orangeburg central office fires and the Los Alamos area wildfires. With our experience and unique background, no one is better prepared to help you recover and safeguard the health of your network. Here are highlights of some of the challenges we faced, our response and the results.

Computer Center Fire

After a computer center fire damaged E911, billing and data collection systems, and network fast-packet services, we provided technical support to expedite service restoral, manage short-term risks and facilitate recovery efforts.

We facilitated interactions between more than 30 equipment suppliers, insurance companies and their consultants.

Financial impacts included the replacement of key systems ($5M), a reduction in cleaning costs of $200K, and settlement of this $10M loss with no record of an insurance claim. In addition, we investigated the cause of the fire to prevent recurrence in the facilities of other contracted service providers.

Boston Police Headquarters PSAP Fire

A fire in the rectifier shelf owned and maintained by our client at Boston Police Headquarters' new customer premise PSAP (Public Safety Answering Position) required rapid restoration, equipment disposition assessment, long-term reliability impact and root cause analysis -- all conducted under an extremely sensitive political situation.

Five hours after notification we were on-site, identifying damage, offering replacement recommendations, providing inexpensive cleaning procedures and restoring the facility and equipment to its pre-event condition.

Our joint failure analysis with the manufacturer and customer resulted in a "Class A" change, at the expense of the manufacturer, to reduce the risk of recurrence in other facilities.

Manhole Flooding

When new, fully-populated controlled environment manholes (CEMHs) in a metropolitan area were flooded in the equipment sections of the digital loop carrier (DLC) vaults, the source of water entry, and degree of damage to the equipment required identification and assessment.

We performed on-site testing, sample collection and analyses, inventoried damaged equipment, gathered evidence on the probable route of water entry and developed a full set of restoration needs and facility cleaning procedures for both CEMHs.

Our investigation allowed a completely substantiated cost recovery claim to be filed with the electric utility that installed the conduit. Our documentation of the damage and analytical data justified the replacement of the equipment and the ability to salvage the CEMH facility shell, and determined the primary water entry point. Our concern regarding the reaction of the DLC equipment shelves to the elevated relative humidity levels led to the filing of an engineering complaint with the equipment manufacturer.

Midwest Flooding

North Dakota and Minnesota floods resulted in more than 109 Federal disaster areas. Our client was challenged with an enormous outside plant recovery and restoration efforts.

Our experts conducted damage assessment and disaster recovery operations for several outside plant facilities and central office locations using unique cleaning, restoration, decontamination and work-flow procedures to expedite recovery, and documented lessons learned to minimize future flood damage.

Working with client personnel, we restored service, reduced trouble reports, assessed damage and determined cost-effective, long-term methods of restoration.

Tornado Destruction

After 1997's tornado activity restoration of telecommunications service at two damaged facilities presented unique challenges.

We immediately provided remote assistance via conference-bridge to prepare service recovery strategies. Once on site, we provided technical support including guidelines on select restoration efforts, assessing damage to equipment, application of environmental controls, and personnel safety.

Restoration of dial tone occurred in just over 48 hours. Our recommendations and documentation on safety, protection of plant, equipment salvageability (2440 circuit packs), hazardous waste disposal and environmental control was used for insurance purposes.

Improving Network Reliability and Disaster Recovery for a Cable Company Entering Telephony

A major cable company wanted to enter the telephony market to deliver voice and data services in addition to its existing cable TV and Internet access services.

We performed a Network Integrity Review, Quantitative Network Risk Assessment and a Physical Risk Assessment of key sites to determine whether the company could recover from service outages, minimize outage impacts on customers, and prioritize potential risks and issues.

As a result of our findings, recommendations and lists of priorities the company was able to minimize outages on its existing network, develop a network evolution plan to support new telephony services and expected service growth, and plan for swift, effective disaster recovery.

Performing Comprehensive Physical Risk Assessment for a Newly-Competitive South American Carrier

Facing deregulation and competition from new entrants, a South American telecommunications service provider recognized that efficient operations and superior network reliability would be critical to its survival and called us for help.

We went on-site, performed an end-to-end physical risk assessment, examined a representative sample of equipment locations and developed a comprehensive risk mitigation strategy for the client’s network.

Designed to enable rapid, effective disaster response from virtually any location and function across the company, our guide-lines and procedures resulted in higher operational efficiencies and service quality that enhanced the client’s competitive position.

Protecting a Key Central Office from the Threat of a Nearby Demolition Project

Concerned by the threat posed by an adjacent building’s demolition project, a regional telecommunications provider asked us to evaluate the risk and recommend mitigation for its central offices.

Our experts visited the demolition site and recommended an alternative demolition plan using conventional tools rather than explosives. We set up sensors to remotely monitor the client’s building and suggested alternative measures when vibration levels approached or exceeded acceptable levels.

A significant disaster was averted and the client was able to maintain network operations, minimize equipment shock and physical stress, avoid significant time, costs, and service disruption and increase the robustness of the installed equipment to withstand future shocks or catastrophic events.