Auditor Struggles, Part 1

Some auditors struggle with basic auditing. So when these auditors try to data analysis, well you can imagines how that goes.

I recently met with a team of auditors to give them input on what data profiling would be appropriate to perform. And what analytics might be insightful.

This is Part 1 of a 4-part Case File series that describes how real auditors tried to apply questionable methods to auditing and data profiling. Do not try these methods at home or work. Don’t even dream about them, awake or asleep. 

Some background:

1) These auditors came to me and asked for my help.

2) This was the first time this process has been audited since the department started its ‘do as much analytics and full population testing‘ emphasis.

3) I recently met with the same team to do the exact same thing on a previous, unrelated audit. So they have been though this drill before.

4) The audit team consisted of 3 senior auditors who had a combined 25+ years experience, combined.

5) The manager of this team and I understood that this audit was just getting started.

When the meeting was scheduled, I asked to be sent a high-level summary of anything they had so far, their ideas, etc , so I could prepare for the meeting. I received nothing.

All I knew was that Process X was being audited, a process that I had audited at least twice (the actual process doesn’t matter). In fact, I had done some impressive data analysis on this process before DA was cool (and had received some accolades from the Process X nowners, but that’s another story. My point is that I was very familiar with this audit.).

A couple days later, I arrived at the conference room and was handed 6 pages of notes about the work they had performed so far. I thought this audit had not started yet. They were way past the ‘gather ideas’ and ‘talk about it’ stage.

They had already downloaded their main data file, which a year of transactions. The file had about 20 fields in it.

They had profiled each field. For example, they noted that the “Status field had the following values: Complete (552), Pending Approval (100), Approved (343), Unassigned (33), Blank/No Value (219).”

They did the same for every field in the file.

When I looked up from their profiling document, they were smiling. “What do you think so far?” they asked.

“You’ve done a bit of work,” I replied, attempting to hide my emotions. I’m always trying to get auditors to do more, so it’s hard to criticize them when they do more, even when it’s more of the wrong thing.

So I started asking questions, hoping to lead them with a glove instead of a baseball bat.

• “Do you have definitions of all the fields?” No, they had requested a data dictionary, but had not received one yet (in all fairness, the system had been converted to new software since I had audited it).
• “Do you know which fields are required and which ones are not?” No, they were waiting on the data dictionary for that. Since most of the fields had at least 50 records with no values, I assumed most of them were not required.* They didn’t seem to have noticed that.
• Do you know which fields are key fields? Do you know which values in those fields drive internal controls or user behaviors? No, they said.
• Is there a field that indicates whether the transaction was successful, withdrawn, or denied? They weren’t sure. They wrote that question down so they remembered to ask their systems contact. 
• Turning to the most senior auditor who had been at this company longer than me, I asked, “Do you remember how critical that field was in the previous system? Yes, he said.
• Do you think it’s a critical field for the current process? Yes, one of the auditors said–that field used to drive the metrics in the old system. He don’t think that part of the process hasn’t changed; he said he’ll check as to why it wasn’t included in the transaction file”.

*This could also mean that the field edits were not applied by mistake or are not working. Always ask about which fields are required.

The glove wasn’t working, as they were still smiling. I reached for the bat.

• So why did you did you spend all this time profiling this data and writing up these notes before you knew the answers to those questions? 3 pages of notes? Silence.

Finally, one of the more junior auditors spoke up: “This didn’t take that much time.”

I saw one of the other auditors swallow slowly. He knew the path that I would be taking next.

See Part 2

 

 

Advertisement