Who me ? The long arm of the law is cut unexpectedly by Microsoft Exchange’s antics this week as another reader explains why some of Her Majesty’s finest were once deprived of festive email. Welcome to Who, Me?
Today’s story, from a Regomized reader like “Sam”, takes us back to the mid-2000s and happier days before Brexit, Trump and the pandemic hit.
During these peaceful times, Sam was working as a civilian employee for a local British police force as a member of the office support team. He had been there for a few months when the holiday season came around.
Christmas parties were one thing for Sam and the crew, and each group had their own festivities during the day. Other groups would cover them as crackers were fired and silly hats were worn. Each party normally ran from lunchtime until late in the evening.
On the day in question, it was the server team’s turn to leave while Sam and the office team made sure everything was covered in their absence. Understandably, there was a bit of mistrust between the admins and the mortals, and so Sam and co naturally took to sticking festive wishes on the lasers under the admins’ mice as well as other japes.
It was all fun and games until 3 p.m. when two members of the server staff (who had left their cell numbers “just in case”) burst into the room.
Something had gone terribly wrong.
“A bit of background…” Sam explained, “We had 2 Exchange servers. Your placement depended on your number of employees; odd on one, even on the other.”
One of the Exchange servers had suddenly died. “As a result, half of the gendarmerie had lost their email!”
“As a public service, we had top-notch Microsoft support at a better price than many, so we made abundant use of it, well into the wee hours of the morning…”
Eventually it turned out that a disk on one of the servers had failed the day before. Sabers in the form of support contracts were tossed around and a replacement had arrived that morning. The server was fine – after all, it could handle a failed disk. The replacement was inserted and a rebuild started.
And then the office crew left for their festive frivolities. The server would manage, as expected.
What could go wrong?
In the end, a little.
“During the process of rebuilding the array, a second disk decided it was going to join the party and, while not entirely unsuccessful, caused enough disruption in the force, that Windows and Exchange violently soiled themselves and gave up, crying in the corner,” Sam said.
To be fair to the server team, they had warned that the Exchange setup was living on borrowed time, was well overloaded (compared to Microsoft’s recommendation) and, according to Sam, the phrase “It’s a disaster who’s waiting!” had been thrown at management on more than one occasion.
And then, in keeping with computer fashion, it happened.
“None of the many iterations of the increasingly technical guys at Microsoft helped one bit,” Sam said. “It wouldn’t repair and restore from a backup.
“It had to be rebuilt and restored by the datastore in order to get everything working again.”
The long and tedious job (which required restoring user data in batches) took just under two weeks, making Christmas and New Years miserable for everyone involved.
And the person at the end of the queue? The biggest police cheese: the Chief Constable. His calendar had been used as an audit trail of his activities, which meant most of a data store was devoted entirely to him, and retrieving it posed the most problems.
The curse of being a very important person.
While getting Bobbies on the beat might get favorable headlines, neglecting basic computer principles can lead to all sorts of disasters. Have you recently tested your disaster recovery plan? Or has your company not considered the consequences of when that innocuous beige box inevitably turns brown? Share your story with an email to Who, Me? ®