Seven months ago a huge hurricane with the code-name Sandy, hit the North East US. Major states, including New York & New Jersey, were heavily impacted for many weeks afterwards. People lost their lives, their homes and businesses and until today thousands are still struggling to get back on their feet. What I will say through my article is the view of a tech-guy on the problems we faced on infrastructure, predictions, security and disaster recovery and common mistakes that even big businesses with branded support couldn't avoid.
Prepare for the worst - Hope for the best.
When it comes to prediction and preparedness for a natural catastrophe or a simple data-failure accident, the least you can expect from your I.T. guy is to protect your business’s and clients’ data. Nowadays there are hundreds of different roads that a sophisticated I.T. manager can follow. At the end of the day what counts is how many hours the business was down, and what data was lost or incapable of restoration.
When it comes to data and information of any organization, it is really important that the business owners understand from the beginning the amount of money they have to spend to get back lost information or - in the worst case scenario - to be aware of the cost they'll have to face in case they lose it forever. Even the smallest business is obligated by law to keep transaction data in hard copy - we all know that. But what will happen to these hard copies if the hurricane comes through your place? They will be destroyed of course, and the only sanctuary that you'll be able to run to is your digital back-up. What would happen if you kept your back up in the same place? Well, just search how many companies went completely out of business after Sandy.
What to do - What to avoid
There are a few simple rules that you can follow, two of which are life savers:
1. Always have a complete backup of your server OS and files on the cloud or in a distance greater than 400 miles away from your business address.
I've talked with many people whose business flooded, and they are in the restoration process for many months now. Their major problem –even for those who kept backups -was that in 50% of the cases, their houses were flooded too, and so they lost absolutely everything.
2. Always have a secondary internet connection (the cheapest that you can get) available.
This way when phone lines goes down for 2 or 3 weeks or your internet provided suffers with problems you have a greater chance that the other provider will have less or no problems at all. This way you'll be able to have your internet connection and even a voice over ip phone to contact your clients (who will desperately need you during these types of situations).
Make inspections on a weekly basis to ensure your backups are up and running. The same goes for the data-backup and your internet connection. Be prepared to act when something happens, i.e write down what you should do for your router and the connection inside the building and don't panic.
- Not keeping a backup (Dah?)
- Not having UPS with a working battery on your servers
- Keeping critical data in your local system and not inside the server
- Keeping backup only on your raid array inside your server
- Keeping backup in an external HDD and leaving the HDD inside your building
- Keeping backup in an external HDD and having the HDD a few miles away from your business
- Keeping your backup in an external HDD without a very strong password
- Not storing a small generator, enough to run a few systems and lights
- Having cheap solutions for cloud backups. Go only with the big ones. You get exactly what you pay for.
Suggestions: Keep the really important files on the cloud. Don't sync unnecessary files, because in a case of emergency it's going to slow down restoration process. Write down on a small piece of paper - and keep it always with you - the model and brand of your server. It's 10 times faster and easier to restore the data on the same model, than to a new one. Make an emergency preparation checklist of what steps you should take when the catastrophe hits. Be prepared.
After the catastrophe we had many losers and winners. Companies that were prepared to defend themselves and their reputation, came out stronger, with a greater market impact. The ones that weren't prepared are still fighting for their future, still struggling to survive.
The point is, that it's up to the decision maker of a business - no matter how big or small - to acknowledge the need to be prepared in order to avoid the devastating impact critical data loss can have on the organization. It is also the I.T. team’s obligation to consult management about the risks and solutions they should follow. Weather is changing rapidly the last decade, and none of us are exempt from the catastrophe map. The question is no longer 'how' ,but 'when'...