Project Management in Real Life

It's a good idea to evaluate all your systems to determine on a scale of 0 - 5 what level of risk you are exposed too. Develop a spreadsheet to track all these systems. Items for example to list on the spreadsheet are hardware, operating system, database, applications, vendors and any other information that you feel is important to support the system. While you are doing the assessment it's a good idea to update your Data Center floor plan diagram identifying where each system resides. Once you complete the assessment place a sticker on the hardware indicating that it was evaluated already and come up with a code to cross reference it back to your floor plan diagram.

Doing a system failure assessment is a big project that takes commitment. Depending on the size of your organization it could take months to complete depending on how deep you dig for information. I suggest you do a thorough examination that will expose all vulnerabilities. While performing the evaluation it's a good time to review all your support contracts. You also get a heads up to budget for system upgrades or replacements. You really find out a lot of information to present to management from a system failure assessment.

Grading your findings for example could be a 5 for the highest level of risk if it's a black box that just runs and no one really has a clue how to support it because the person who installed it left 10 years ago and it just keeps on running, don't re-boot it ever. If you are running on an old hardware platform and it's off support and you are unable to patch the operating system that will be a level 5 risk. Just a few examples to give some ideas on how to rank your risks.

The System Failure Assessment will need to be a living document updated periodically to keep it effective and valuable. It will serves as a disaster avoidance / disaster recovery document. It's your decision how to act on level 3 - 5 risks. Don't wait act now.

(Note - this article was originally written by Drake Settsu and published on DrakeSettsu.BlogSpot.com in February 2015)

The Disaster 

A server containing multiple disks managed by a vendor for the State of Hawaii’s driver's licensing programs had multiple hard disks crash. The storage media is encrypted and secured with some data not readable. There was no security or data breach. It will take a couple of months to determine if the data can be recovered.

The Vendor

The spokesperson gives a briefing at a press conference similar to "the dog ate my homework".

1) There was a backup system in place that's supposed to protect the data when the hard drives crash, it was not properly configured.

2) They were not aware that certain documents or images were not getting backed up properly.

3) The backups are checked to make sure everything is working properly. Don't  know what the details are and what the plan was on checking the data.

Summary

When implementing a new system, I have few suggestions on how to make your system bulletproof. I used to be a Systems Administrator and I never lost any data on my watch.

1) The Project Manager in charge needs to have good Subject Matter Experts to recommend the technology, security, and procedures to be in place to ensure the system has the redundancy to withstand a disaster and prevent a security breach.

2) The Statement of Work needs to always include clear expectations.

3) Service Level Agreements need to be in place for the system.

4) Disaster Avoidance plan and Recovery Strategies to meet Service Level Agreements.

5) Test your systems periodically to ensure that the data is being replicated or backed up properly to a media that will go offsite.

6) Make sure you have a good hardware / software support contract in place.

7) Never ever trust the vendor. Make a checklist of your key deliverables to be reviewed and demonstrated when you begin the sign-off stage in the project. Any missing checks will result in a big missing check for that vendor to cash.