In Case File: Audit Server Disappeared, I noted that a friend of mine learned that IT had, on its own prerogative, wiped a server belonging to Internal Audit because “it never appeared to be used.”
Some of you already commented on some of the issues involved in this incident and the normal IT activities that should have prevented this incident (or at least alerted IT that something was wrong). Let’s review those comments and I’ll add some other details and comments.
First, what should have prevented or reduced the impact of this incident? Readers provided 4 of the answers.
- Audit Monkey: “The ability to ‘delete’ the server wasn’t restricted and any wiping should be authorised.” Access to the server was restricted to server admins, but the wipe was not authorized. In other words, no change control occurred. That’s a huge issue.
- 2Hats: “Perhaps training and communication too (even a junior should know, or have been told, to double-check before wiping production data, esp. wholesale).”
Since the wipe was enacted by a junior admin, that raises the issue as to whether this person should have admin access, or at least be supervised more closely. However, it was determined that change control was rather loosely controlled the previous year also–a key financial server was replaced with new hardware one weekend without any change control oversight or record (as IT noted, “the server couldn’t be upgraded to enough memory, so we swapped it out. No big deal.”)
Off the record, I learned that the VP of Audit was told about this financial server issue, but it wasn’t even written up. So if Audit does not care, why should IT? Until a change control problem affects Audit resources perhaps?
- coffeeking: “Lack of IT resources in the Audit team, if the last guy was able to do it then there should have been someone who took over the responsibility after he left.” If someone on the audit team had taken over responsibility for the server, it would have been restored much easier and faster. No one in the department was very familiar with the configuration and operational details of the server—not one of the auditors even knew the name of the server or where the documentation was stored.
- coffeeking: “Lack of asset identification. The guy probably had no idea who the asset owner was and probably didn’t bother finding out since there might not be such a procedure enforced.” This was also true. IT had identified the original auditor who requested the server and loaded the software as the asset owner. Since he was gone, the admin reasoned, the server was no longer needed. Perhaps the VP of Audit should have been the asset owner, with duties delegated to an auditor. If the VP of Audit leaves the company, IT is more likely to follow up with someone in Audit before making changes to an Audit department asset.
Assets are identified so that they can be classified for criticality, data classification, and data retention (among other things). The audit data processed by that server was highly confidential and retention was mandated by regulatory policies. It’s kind of hard to protect a server and the data if they are not classified. It’s kind of hard to classify systems without a classification policy. Both IT and (especially) Audit should know and do better.
- coffeeking: “Lack of policy implementation; data wiping policy. There must be some conditions to be met before anyone can wipe any data off.” This goes back to the change control issue, but data wipe policies usually require a final backup prior to wiping. This company doesn’t have a wiping policy. So no final backup was made.
Also, any admin worth her chocolate could have checked the access logs to determine when the server actually was last used. Or simply asked Audit about the server, especially since the department is right down the hall.
How should IT have detected the problem? The server was backed up, so wiping it would result in an error on the backup system (server not found). (Do you think that a junior admin who wiped a server without authorization would be diligent enough to alert the backup admin that the server backup needed to be removed from the backup system? He wasn’t.) The backup admin was “not sure why the error was not followed up.”
One Other Issue
Once the server was restored, my auditor friend noticed the server had not been patched since the auditor who installed the server left. The company doesn’t have a patch management policy or any monitoring.
Yes, Audit performed an investigation on the incident, but no report was produced. It was called an “unfortunate incident” and left at that. Are you surprised?
Multiple IT staff failed to ensure basic tasks were completed as they went about their jobs. In my opinion, Audit failed (again) to hold them responsible. So which came first, the bad egg or the audit chicken?