Mack-the-Auditor Gets Audited! Part 3

This is the third of 3 posts; this post describes how I audited the auditors and my perspective on the whole thing.

Read the first post (background) and the second post (audit results).

Mack Audits the Auditors (heh heh)

As I was busy with other projects, I didn’t have a chance to review the ACL log in the test environment until a few days after the firm left. I pulled up the log and found it mostly blank.

All that was in the log was when the firm opened ACL each day to look at the scripts.

Remember that you can remove everything from an ACL log except for one thing: ‘Portion of ACL log removed’.

Since that statement was NOT in the log, it meant that they didn’t remove anything.

Which meant that the firm didn’t run a single script.

Which meant the firm didn’t run their own test files.

Which meant the firm formed their opinion based SOLELY on what one person (me) gave them:

1.  Scripts
2. Source files
3. Results of the tests that I ran in the test environment, and those from the production environment, including the ACL logs of those runs.
4. Test environment

I could have created or changed any or all 4 of them to ensure the firm concluded what I wanted them to conclude: that the process/script worked as defined.

They had absolutely no proof that the scripts I gave them used the source files I gave them to produce the results I gave them. And I could have doctored the ACL log to show that it all worked. (For the record, I didn’t tweak anything, and everything I provided was legitimate.)

Try doing what this firm did in your next audit and see how far you get before you get called out. Do absolutely no independent verification or testing, just rely on what you’re given as gospel truth.

I questioned the consulting firm’s management about this, and they felt their tests were appropriate. They relied on the fact that another auditor in the CAE’s company helped me review and validate the results of the test environment. However, I reminded them that said auditor did NOT help me set up the test environment or see me actually run the scripts. All he did was review and validate the reports I gave him. They never asked that auditor to verify that the scripts, source files, results, and logs were legitimate.

When I mentioned this to the CAE, he said that he received what he needed: an opinion from an outside firm that the scripts worked as described. He also cited the previous work I have done for the company, and that my work is always spot on.

While I cannot grasp how the firm could place reliance on their work, I can almost understand the CAE’s conclusion; we have worked together several times and he trusts me explicitly. I appreciate his confidence, but in my opinion, I would trust, but verify–especially when others got fined when they were wrong.

I sure was glad this project for this company was over. Time to move on.

Takeaways

In summary, here’s my takeways from this experience (I looked at this from the auditor AND auditee viewpoints).

• Being audited is no fun. I knew that, but it made me think about it a bit more, and my need to show more compassion and understanding to those I audit. Again, I thought I did that pretty well already, but I saw that I could still improve.
• Being audited actually helped me. Although some of my auditees tell me this AND I was expecting to learn something from the firm’s audit, what I learned still surprised me. It reminded me that I don’t see and think of everything, and that another eye can be helpful. Time to get off my high horse.
• Being audited consumes a lot of your time. Again, I knew this, but it was a good reminder. My auditees have businesses to run.
• Give auditees advance notice of changes. Having the auditors change at the last minute caused me stress and a bit of rework, which could have been reduced with a simple phone call from the consulting firm.
• Provide a reason for changes, and determine whether an apology is warranted. I never received either from the firm. Reasons and apologies don’t alter the amount of work that changes sometimes require, but they mean that the offender has thought about how the changes impact you and acknowledges that.
• Do NOT request anything you really, really don’t need or intend to use. While I don’t know whether the consulting firm intended to use the test system I built and secured, I spend 2 weeks perfecting and testing it. I tend to think that the firm just didn’t put enough thought into how much work it would be to actually use the test system. They would have had to alter the input files and then do more work validating the results. Once they got on site, I think they realized that and skipped the test. (By the way, after the audit, the CAE had the test system dismantled.)
• When auditing, read (or at least skim) the information you are provided before asking questions. Some of the documentation I provided was only a page long and addressed most of the questions they asked. The longest document that I provided at their request was 7 pages. It was obvious the consulting firm staff was lazy.
• When you call out someone’s shortcomings, make sure your work supports it, as it may be challenged. This should be pretty routine for most auditors, but some auditors don’t dig deep enough or document their work in sufficient detail. That will make a challenge hard to defend. When I saw that the firm didn’t run any scripts, that solidified my opinion about their integrity or lack thereof.
• Remember that what you identify in an audit will impact some people, maybe many. When I issue an audit report, that causes work for many on the auditee team as they have to respond to the items in the report in writing. I had to do the same with the firm’s audit report. It reminded me to make sure that all the items I call out really NEED to be addressed; even if I call out items that are largely ignored (another term for ‘risk acceptance’), even ignoring that audit item requires thought, usually a meeting or two, and documentation, not only on the auditee’s part, but also my team.
• Don’t swallow too quickly. The firm’s auditors took what I gave them and never independently validated or checked it; they didn’t even ask the other auditor as to the validity of what I provided. Auditors should be skeptical until they can gain comfort based on evidence (which varies in the amount of confidence provided, of course).
• An auditor’s reputation is everything. If you have a good reputation, it opens doors and helps people trust you. They will tell you things that they won’t tell anyone else when they know you will handle the information appropriately. Guard your reputation above all else.

As always, I’m interested in your comments and questions.

 

 

Advertisement