Who Do You Call at 3am?

Who are you going to call at 3:00 a.m. during an outage …and who are you hoping will answer the phone?

By: Steven Mauss

If you’re over 20 years old, you probably remember the campaign commercial about the red phone ringing in the oval office.  It just keeps ringing and ringing until, finally, the question is asked, “When the phone rings in the White House at 3:00 in the morning, who do YOU want to answer that call?”  The commercial was actually a remake of an earlier, similar “hit piece” used by another candidate.  The meaning is the same:  In an emergency, who do you hope will be available to help?

Someone with extraordinary knowledge and experience, of course, and someone you can trust to be there when they are most needed.

The story is told of a nuclear power utility starting a refueling outage, fully believing that everything is well-prepared to ensure a rapid turn-around.  The sooner the plant comes back online, the sooner the utility is providing safe, clean, reliable power to their customers.  For every day an electric power-generating nuclear reactor is not producing, it can cost the utility about $1.5 million in lost revenue.  As you can imagine, they are extraordinarily keen to make sure everything goes well.  As most of us know, though, having everything go 100% according to plan is, by far, the exception, not the rule.  Such was the case for this outage.

Their outage depended upon constant update and distribution of key project plans and metrics generated by a mission-critical software application.  Using this tool provides important automation, without which a lot of manual processing must be done every day.  This kind of problem can cost valuable time and resources if things go wrong.  As a result, multiple tests had been run prior to the outage to make sure it would operate successfully.  At about 5:00 a.m. on the first day of the outage, the tool stopped working.  No one knew why.  It was time to call for support.

But “Support” was located on the West Coast and the utility was in the Midwest.  It was still 5:00 a.m. PST.  Would the vendor even be awake yet?  Yes, they were.

Very few vendors offer 24/7 support during critical periods.  But this vendor has its senior management on call to answer the phone and assemble the needed support.  And that’s what happened.  A lot of trouble-shooting started taking place immediately until the problem was isolated and identified.  The vendor’s staff began working on a fix immediately and, in the interim, the vendor’s staff logged in to the client’s systems every morning (under strict protocols, of course) and manually produced all necessary data migration and metrics work so the plant could stay on schedule.  That is who you want answering the phone at 3:00 in the morning.

Who is this vendor?  It is the company I founded over 30 years ago, Knowledge Relay.  We’ve developed a company culture that doesn’t shy away from problems, no matter whose “fault” it is.  No one can claim there will never be errors or problems.  What matters is whether you can TRUST the people you work with to stick with, and solve, whatever problems arise.

So, who will YOU call next time you need help?