What happens when the lights go out?

February 20th, 2015

A recent experience while out shopping at the weekend illustrated the value of Business Continuity planning.

It was Saturday lunchtime and just as I reached the check-out of an out-of-town retailer, the lights went out.  An ominous silence momentarily descended on the store.  Because it was a light and airy ‘shed’ we weren’t plunged into darkness and somebody had thought ahead and provisioned local battery back-ups for the Electronic Point-of-Sale terminals.  Unfortunately, it appeared that this was pretty well as far as the contingency planning went.

For the next few minutes the operators tried unsuccessfully to log back in on their Point-of-sale terminals while customers waited hopefully for the power to come back on and normal service to be resumed.  Then, the supervisor swung boldly into action.  The check-out operators were told to manually record the bar codes and costs of every item sold on duplicate receipt pads and to use manual credit card machines to enable them to continue accepting card payments.  At this point the supervisor disappeared leaving the operators to deal with the rapidly swelling queues of customers.

Unfortunately, although the checkout supervisor appeared prepared, the same couldn’t be said of the operators.  As is typical in many stores these days, items weren’t individually price marked and there wasn’t a back-up price-list available.  My trolley only contained a few items that I had mentally noted the prices of as I selected them.  I was fortunate that my operator was a trusting soul and was happy to accept my word for their prices.  How others responded to this opportunity, I can only speculate!

The next challenge was arriving at the total cost of my shopping.  The prospect of adding up six numbers caused consternation and much under the breath muttering by the operator about their inability to do maths.  Having found a calculator, the first two attempts produced incorrect results and an attempt at longhand addition was similarly unsuccessful.  Fortunately, the second attempt longhand iteration delivered the correct answer.

Now we could progress to payment.  This was also difficult as the operator did not been know how to complete a credit card voucher or use a manual imprint credit card swipe machine.  Fortunately, all those years of being a credit card customer before the advent of magnetic card swipes or ‘chip-and-pin’ proved invaluable.  Together we completed the voucher and took the card imprint.   I suspect that had I chosen to pay by cash, working out how much change was required would have been similarly problematic.

The overall process took about 10 minutes longer than it should – all the while the queues grew longer and more shoppers gave up, leaving their shopping behind and walking out of the store.

However, the impact of the disruption didn’t stop at the lost business and possibly inaccurate transactions.  A conversation with the store manager on the following Monday established that he had many hours of work waiting for him to enter the details of the transactions on to their stock control system and the card vouchers to process.

A Business Impact Assessment would have identified that the retail operation was heavily reliant on the operation of the IT system.  A Risk Assessment would indicate that because a power supply interruption would have a high impact and that the likelihood was also quite high (power cuts are a relatively common experience in this part of East Anglia).  A Risk Treatment Plan would show that elimination or mitigation of this risk could be cost effective.

Effective risk mitigation requires a systematic approach.  In this example, the system clearly includes people.  The human factors should be considered alongside other measures in reducing the reliance of retail operations on IT and how to improve the resilience of the IT system to issues such as a power cut.

The points that I’m keen to emphasise are that, before leaping to technical measures, consideration of the part that people play in the system is equally important. Secondly, for Business Continuity planning to be effective it requires the awareness and commitment of staff at all levels (particularly Directors, Managers and supervisors as well as front-of-house staff.

A truly systematic approach would include considering the attributes required of staff to cope in an incident, such as literacy and numeracy when recruiting as well as their subsequent training in using a calculator or a manual imprint card swipe machine!

Similarly, careful system design can eliminate many vulnerabilities before any equipment has been purchased or installed.  For an established system, consideration of technical measures should involve an audit of the IT system architecture, identifying items such as network switches, servers, wireless access points, internet gateways, etc., any of which might cause network connectivity to be lost if their power fails.

This example has focused on the impact of a power cut on a retail operation, but similar issues arise for every other organisation, whether commercial or not-for-profit, large or small.  For example, a hotel, restaurant or clinic may well be as reliant on other utilities such as water or gas.  A manufacturing business might be heavily reliant on the supply of raw materials or design information, a service organisation might depend on the availability of customer records or physical access to their premises. Finally, almost without exception, organisations are reliant upon the availability of their staff.  Food poisoning after a company event or a road traffic accident involving a several staff travelling in the same vehicle could all have a major impact.  A little upfront thought and planning costs relatively little but could make the difference between organisational survival or closure.