Please note that this is a page from a previous version of Continuity Central and is no longer being updated.

To see the latest business continuity news, jobs and information click here.

Business continuity information

IT continuity: the benefits of a mobile NOC

Matthes Derdack discusses the benefits of mobilizing a network operations centre / center and outlines a financial justification for implementing such a strategy.

It is common practice for many large and medium-sized enterprises to operate a 24x7 network operations centre / center (NOC). A NOC is designed to continuously monitor IT, telecoms and core operations enabling it to immediately respond to any critical event that might threaten business continuity.

However the traditional NOC model with a heavy reliance on manual monitoring methods is fast being replaced with a more dynamic infrastructure that uses enterprise notification software to enhance service levels whilst lowering costs.

Deficiencies with the traditional NOC infrastructure

Permanent staffing at a NOC is standard practice for many organizations. However, there are two main drivers that threaten the long-term viability of this model:

1. There is an overall drive for cost reduction and operational efficiency in IT. A permanently staffed NOC requires a significant number of people to be dedicated to tasks of relatively low intensity. It is difficult to justify allocating the HR budget to staff that spend entire working shifts watching a computer screen waiting for the rare occasions of intense incident resolution in order to maintain business continuity.

For operations staff, this type of shift work can be exhausting with little personal fulfilment. In particular, the night time or out of business hours shift conflicts with daily family life but is often the majority of the time workers typically spend in a NOC. There can be ahigh turnover of staff which increases staffing and HR costs.

2. With the availability of a variety of mobile communication and collaboration technologies, it is an anachronism to follow old-style NOC practices which require IT analysts to be sat in front of large screens monitoring the status of various system consoles, waiting for an incident to happen.

Drivers for change and enterprise requirements

Forward thinking companies are therefore changing and adapting. And nowhere is this more visible than in smaller firms who cannot afford to operate a NOC. They have by necessity needed to innovate and have introduced smarter working practices. The approach most frequently used is based on a combination of monitoring and notification technology; mostly email or SMS text messaging.

The reason larger enterprises have not followed this approach, is due to the overall business impact of a critical system failure or downtime. The more critical an IT system is the more its uptime is required to avoid a serious threat to business operations and continuity.

Larger enterprises, especially those with 24x7 operations or those organizations being responsible for national infrastructure or public services, simply cannot afford a slow, inaccurate or failed response to a critical problem. The solutions that small organizations have in place are simply not powerful and reliable enough. The impact of a missed incident is proportionally larger, for example, and can go beyond the loss of factory output or email services, and could even threaten public safety.

Changing the established NOC practices

One of the core requisites that enterprises need to address before they can change established NOC practices is to examine the reliability of information delivery and alert messaging, especially outside of normal business hours.

How this is managed is the key requirement that will ultimately determine whether organizations will introduce effective new practices and go on to enjoy the cost savings from deploying operational staff ‘on demand’. Enterprises need a method of guaranteeing the delivery of any critical alert to responsible operators who are not physically located in the NOC.

Modern enterprise notification software can offer a solution to this challenge. The latest systems are capable of almost ensuring the delivery of alerts through a variety of mechanisms like real-time tracking, fully automated escalations and multi-modal communication. In combination with a powerful IT monitoring and management suite such as Microsoft System Center or IBM Tivoli, enterprise notification software can fully automate the overall alert procedure.

No manual interaction is needed to deliver a critical notification to a responsible person or even a group of people. Using rules based policies, an IT alert raised in a monitoring tool is automatically delivered using a combination of unified communication channels, for example instant message, text, voice, push and others. The delivery and active acknowledgement is tracked by the system and it can automatically escalate to other administrators, teams or managers in the event of non-delivery.

Once operators are aware of the problem, the enterprise notification software provides the means to remedy problems through remote actions, e.g. by restarting critical systems using mobile commands from a smartphone. This enables a highly reliable and automated ‘alert-acknowledge-act”’ procedure to be implemented and ensures the most rapid response.

But enterprises also have many reasonable concerns in relation to security. Solutions for distribution of critical information, mobile incident response and management should provide sufficient mechanisms to ensure the security of critical data and information and should prevent unauthorized access to IT and data centre systems.

By adopting enterprise notification software as a cornerstone of a mobile NOC strategy, organizations can reduce their reliance on manual staffing. Nowhere is this more evident than in how the out of business hours cover can be organized as staff can be deployed on-call or off-site.

Additionally there is greater reliability and speed in information delivery: which is needed to ensure a rapid response to critical incidents.

Technical blueprint

The following graphic shows the technical blueprint of a mobile Network Operations Center, using the example of the Enterprise Alert product .

Mobile NOC blueprint

[Click here to see a larger version of the above diagram]

There are three major building blocks of the overall concept:

1. Integration into various parts of the network operations centre:

  • IT monitoring for direct alerts on major and critical events within the shortest time;
  • Helpdesk systems in order to automate the assignment of resolution tasks and incidents and to inform multiple groups of people on resolution progress;
  • Orchestration tools to enable a mobile execution of IT automation tasks, e.g. the restart of a virtual machine.

2. Ability to notify multiple people in different roles:

  • IT service staff who can respond immediately;
  • Affected users in order to ensure customer satisfaction;
  • Management (who is often involved too late) in particular of measures for resolution fail.

3. The enterprise notification system which needs to fulfil a variety of requirements but in particular the following:

  • Real-time delivery tracking and audit trailing;
  • Multi-modal communication including voice, text, push, IM and email;
  • Smartphone apps for convenient alert management and mobile remote actions;
  • Support for fully automated notification workflows;
  • Secure communication and secure information distribution on a ‘need-to-know’ basis.

Examining the financial impact of a NOC strategy
Given the business benefits described above, we will now look at whether these are matched by the potential financial benefits on offer. One of the challenges is that the financial impact of a NOC mobilization strategy can be difficult to measure accurately. More to the point, can we build a return on investment (ROI) template that will stand up to detailed scrutiny?

In my experience there is a good way of measuring the impact of a NOC mobilization strategy. To establish a baseline for measurement, let us compare the relative costs for a traditional NOC that always has two operators or IT administrators onsite 24x7 and a NOC that has been mobilized and operates during the regular business hours of 09.00 to 17.00, five days per week, and employs on-call staff outside of these hours.

Assuming a move from a NOC that is permanently staffed 24x7 to one that operates on an 8x5 basis, one might think that the basic cost reduction would be a saving of two thirds of the original HR costs. However, staff salaries for out of business hours work is usually 50-100 percent higher than for regular office hours. Of course, employees on-call also earn an extra payment but are usually only paid a fixed rate plus the actual working hours in the event of being called upon to resolve a critical incident.

A NOC that is permanently staffed with two operators will usually require six people working shifts. If each operator earns 3,500 EUR per month (an average salary for an IT administrator) and with a 50 percent premium for working evenings and at night-time the total monthly salary payments would then be almost 29,000 EUR and nearly 350,000 EUR per year.

How does this compare to the costs under a mobile NOC strategy for weekends and out of business hours? This would change the extra payment into a standby-bonus plus actual working hours related to incidents that an on-duty administrator would need to manage. Let’s assume a standby bonus is around EUR 300 monthly per person. With an hourly cost of 31.25 EUR and an assumed incident-related workload of 32 hours (representing the actual work that is spend on resolving incidents within the standby time), the total amount would be 2,800 EUR monthly.

This represents just 35 percent of the original costs related to out of business hours NOC staffing. Over the course of a year, using the above assumptions an organization could save 62,160 EUR, or about 18 percent of the total NOC staffing costs.

Whilst the direct financial benefit is significant there is an often overlooked additional benefit. In our example, following a mobile NOC strategy will release four additional IT administrators to concentrate on other tasks and duties. The actual value of this is far higher than the salary that is paid to these people.

It is clear to see from the financial analysis presented above that the business case for enhancing the NOC infrastructure with enterprise notifications can be compelling. This is one of the reasons why companies that examine the business case closely are turning to a mobile NOC infrastructure to enable effective continuous monitoring of various business critical IT and telecommunications services.

There are additional ‘soft’ business benefits relating to operator satisfaction, loyalty and productivity that cannot be easily quantified but provide additional justification to pursue a mobile NOC strategy. And finally, the overall effort for business continuity that is the core idea of a NOC is maintained and supported.

The author
Matthes Derdack is chief executive officer of Derdack and responsible for the vision and the strategic direction of the company. He writes a regular blog at http://blog.derdack.com and is the author of a number of publications in IT magazines.

Contact Matthes at MDerdack@derdack.net

Derdack designs advanced enterprise notification and rapid response software. Derdack’s innovative products help leading enterprises and organizations in over 50 countries to automate alert notifications, to reliably distribute critical information and to rapidly respond to critical incidents and emergency situations before they can impact business continuity and customer service levels.

•Date: 30th Nov 2012 • Europe/World •Type: Article • Topic: ICT continuity

Business Continuity Newsletter Sign up for Continuity Briefing, our weekly roundup of business continuity news. For news as it happens, subscribe to Continuity Central on Twitter.

How to advertise How to advertise on Continuity Central.

To submit news stories to Continuity Central, e-mail the editor.

Want an RSS newsfeed for your website? Click here