In my previous Python post, I described the first steps of my python journey.
In this post, I want to respond to Grant’s comments that he left here re: auditors getting into programming, and specifically python.
[For my readers who don’t know, Grant is the founder, President and Chief Architect of Arbutus Software Inc, which specializes in audit analytics (he usually doesn’t mention that he was involved in writing the first versions of ACL too).
So his words have experience to back them up, and while I’m flattered he pops in here and there to comment on my little blog, I don’t hesitate to disagree with him occasionally .]
I include this post in my journey as I’ve faced similar comments from others at work.
I’ll quote Grants’s comments in blue text, and then respond. Make sure you read his full comments first so you don’t think I edited them to my advantage like the fake news media does…
The problem is that, historically, when auditors start programming they occasionally make errors, and these are very challenging to identify.
I understand where he’s going with this, and it’s true; I just don’t think it’s a reason that auditors should avoid programming or Python. Python is easy to learn but hard to be good at, no doubt. In a bit, I’ll defend why I think it’s imperative for auditors to learn programming.
It’s just that a little bit of knowledge can be a dangerous thing in the wrong hands.
I agree, and that’s what I’ve been telling all the professional programmers who I audit all the time when I find their mistakes. Sometimes you can see the mistakes clearly in their code, but usually just by playing with the results of their code (their apps) and asking good questions.
Yep, professional developers (the experts) make mistakes too. Lots and Lots of them (how many Microsoft & Apple patches have you applied since birth?). Many of them are mistakes of omission (I didn’t think of that) or coding mistakes (typos and insecure coding).
And yes, I agree that auditors will make the same mistakes and even more….
…identifying programming errors is a much more challenging exercise, and just as likely. This is the very reason that the audit analytics tools such as IDEA, ACL, or my tool Analyzer, were created. To minimize the amount of coding an auditor had to do.
While I agree that these tools make it easier to analyze data, you still have to code unless all you do is menu-driven reviews (click a menu option, select a few boxes, click OK). All these tools require coding if you want to run a script and automate the project end-to-end.
ACL has a scripting tool and you can copy code from the log, which makes scripting easier (I assume the other tools do too), but you still have to code loops and formulas, etc. I’ve seen auditors make horrible errors in scripting AND in menu-driven audits.
Grant’s point is true, but to sum up, to really use any of these tools, you have to learn to code and debug to an extent (again, Python is harder).
So if I have some auditors who have excelled in ACL scripting, I think one or two could probably learn Python, but they’d have to step up, which Grant seems to suggest most auditors could not do well enough. That is true* where I work, anyway, but not everywhere; I run into sharp auditors doing similar things like I am, and doing it well.
*This is a problem with audit profession. The world is changing, and few auditors or audit management are ready to adapt. More on that in my next post.
One final point on this particular comment: While programmers have team reviews and testing, audit departments required by the IIA to have “audit supervision”. In other words, any work an auditor does (especially scripting, coding, and complicated analyses) have to be reviewed by a competent audit manager.
And where I work, my python code will be reviewed by other auditors AND go through code reviews by others outside of audit and be checked for security errors, just like other Python code at the company. That may not occur at other companies, but it should. But even if it doesn’t, it’s still not a reason for auditors to avoid programming.
Now you are a very technical guy, and I’m sure that you will build in appropriate controls to ensure that your results are valid, but not everyone will be…what about the auditor who takes over responsibility for the program in the future?
Yes, I’ve built controls and as the lead guy, set standards, and train the other analytic auditors, and watch them constantly. Grant’s question is valid, but it is also valid with all the ACL scripts my team has created.
Our ACL environment is complicated and big. It’s documented, tested, and logged. People are trained and reviewed (they review all my work too!). Nothing is done in a corner.
Again, Python will be more complicated, so I’m not saying it will be the same as managing ACL. This technical guy is sweating it over the Python stuff.
But, when I leave, it will be management’s responsibility to ensure an appropriate person manages and maintains it (I’ve been training my team for this day, but they are still dreading it, so we’re not there yet.).
You can say that’s a cheap answer, but it’s the real answer. Even if I’ve done my job, management may still drop the ball. But is that a reason to not advance our audit team’s skills, and our ability to provide assurance and help assess the complicated, technical risks in our environment?
Also, not every programmer works in an environment where everything is documented and perfectly controlled, especially when people turn over.
Grant ends with this:
Programming audit procedures is often the most expedient way to solve a problem, but it is a double-edged sword. I encourage you to continue your journey, but please be aware of the consequences embodied in that choice.
So while it might sound like that Grant is against auditors taking this technical journey, he’s not (I’ll admit, I played this up a bit above). He just wants us to diligently count the cost and move forward carefully. I agree totally, but I still gets the feeling he’d rather we stick with a simpler tool.
Well, we can’t, and here’s why.
Here’s my biggest argument for auditors to learn programming — the best way to audit something is to live the process (for another take on this, see A Sneaky Way to Analyze IT Controls). Our company has lots of Python code in production, lots of machine learning models running, and all kinds of complicated stuff the company depends on daily.
How well can I review Python code* if I’ve never written any? Not made a ton of stupid mistakes? Misunderstood the results? Forgot to 1-hot-encode some data and then fed it to a model that only takes numerical data? Or worse, I thought 1-hot-encoded data only to find I assigned values 1, 2, 3 which indicate order and priority, which I didn’t intend?
*OK, reviewing Python code is not normally an auditor’s responsibility, and I certainly don’t do much of it. My point is that the more you understand what go wrong and how you can think you’re doing one thing while actually doing another, the better auditor you will be, and the more able you are to understand what’s going on under the hood. To truly do that, I maintain auditors have to start coding or get out. Yep, I said it.
Not only is the day of the checkbox auditor over, but the day of the non-technical auditor is over, but few realize it. Kind of like doing financial auditing without a CPA.
Grant is right, even the non-average, technical auditor is going to struggle with programming. I sure do. But that is something that has to change if auditors are going to remain relevant. Otherwise, get out your rubber stamps and a fresh inkpad.
Here’s the other thing, which I hinted at via the Sneaky Way link above. In my company, I’ll watch my Python code go through the company controls and I’ll learn all about that too. I might even see a few holes.
Gotta do it, and you should too. Otherwise, most auditors will be useless as audit is only getting more technical (regulators are demanding it also, which I’ve learned from experience).
By the way, I first mentioned the need for auditors to get more technical in 2017!
This post, No Analytics, No Audit Department, focuses on analytics, and while I don’t mention Python specifically, the arguments are the same. Same for Power BI, machine learning, and so on.
8 responses to “My Python Journey, Part 2”
Great back-and-forth by two people I respect so much! Totally agreed that (1) audit/GRC professionals are not as technically adept as they should (and need) to be for identifying inherent and residual risk…or even understanding the business/processes of their org and (2) this is a problem that will only be solved by a paradigm change in the audit/GRC space, one that I’m personally trying to do something about en masse this year with some specific industry-changing initiatives.
I believe there will, even in an ideal industry, be a need to help people at all levels of technical capability achieve their personal and professional goals (project/task-based or otherwise). I think it’s as unwise to think all auditors can/should be technical programmers as it is to think that none of them can/are. So, point to Grant that his Arbutus Analyzer tool is an amazing way for less technical people to gain insight into their data, without having to know Python or other programming software. I’ve scripted using ACL and Analyzer for 30+ years and have done some very technical things with the base language (ACLScript), so Analyzer is not “just” for non-techies. Those who want to go deeper into the tech world can do so (and can utilize Python code/output from Analyzer). Those who just want to get information quickly from their data without having to be a programmer can do so.
I agree with the vibe of the post that we should drive toward a MUCH better strategic technical capability within audit/GRC departments, which means having a diverse set of business knowledge/skills, technical capability/knowledge, and management of it all. Us techies don’t think like most accountants (thank the Lord!…LOL) and vice versa, not should we think through problems or their solutions the same way. My wife is NOT a techie at all, but she’s helped me (a programmer) solve problems with my code by asking questions my brain was too muddled to think about and come up with solutions that were simple whereas I was trying to code my way out of it.
Again, thank you both for the discussion on this! As always, I enjoy the thought-provoking blog posts on this site.
Always good to hear from you.
I know what you mean by your comment, “I think it’s as unwise to think all auditors can/should be technical programmers as it is to think that none of them can/are.”
However, I believe audit is getting too technical, or rather, doing a good audit these days requires highly technical skills, so while some may depend on the ChatGPTs of the future to do all the technical stuff, I still think auditors need to be technical.
If you’re a financial auditor and all that is being audited is the financial stuff, you are missing all the technical risk. That’s why SOX and MAR require the IT infrastructure and security be reviewed also (I’ll admit SOX and MAR don’t go deep enough, but that’s another story).
Even to do a decent SOX audit requires good technical understanding. At the same time, a lot of SOX audits are rubber stamps because some auditors don’t understand technology and just follow an external auditor’s checklist.
Again, the more technical a business is, the more technical the auditors need to be. As a profession, auditors are finally understanding how to audit the cloud.
If an auditor can’t do any programming at all, how will they know what to look for? You might as well send an IT auditor to audit financial statements.
Anyway, that’s my take, and I stand by it.
I fear that day is a long way away, still, as we are not really discussing it as a profession. We’re still hung up on doing cool analytics instead of figuring out how to audit them.
Spot on. I’d say the technology evolution throughout the world has raised the bar on minimum requirements to be an effective auditor, overall. Audit/GRC professionals (and really all others in the organization to some extent) should have a base level of inherent risk understanding, which means some degree of data/tech awareness and working knowledge. This doesn’t even get into the efficiency and effectiveness (ironically, good controls) around how auditors do what they need to do. Then, beyond a base level of “tech” for all auditors, there should be a different base requirement for IT auditors, who need to be the “data/tech risk experts” for the dept/org and who can/should help the rest of the dept/org identify data/tech risk, including internal process improvements via automation, reporting, etc. (other auditors should also have a responsibility to look for process improvements, too).
As a rising tide lifts all boats, the audit/GRC industry needs to understand and act upon a continually rising tide of tech evolution and risk, lifting its standards to effective minimum levels.
About 40 years ago, audit shops were asking which auditors should have computers. When I started my internal audit career in 1998 at a Fortune 500 company known for efficiency, our department was deciding which one or two people would get the “networked” computers so we could share files, which would use the color inkjet printer, and if we should upgrade one of our computers (mine) to a Pentium with 2 GB of storage capacity. The year prior, I was selling computers and a desktop with 2 GB of storage was the top of the line and cost about $3,000. Ten years later, I was with a Fortune 1 company and we were beginning to price out 1 TB hard drives. Today, you can easily get a 1 TB drive for $40. The “bad guys” have more fun toys to play with and more efficient technology with which to cause harm. So, audit departments absolutely need to realize and act on the situation. We can’t pretend like the risks are the same as they were 30-40 years ago and manage our audit teams accordingly. A minimum standard for tech capability by role should be discussed by dept leadership as an intentional strategic factor in their group’s ability to successfully help the org manage risk. Just my two cents. : )
LikeLiked by 1 person
Thanks for the history lesson. Great points
LOL, thanks. More of an old guy reminiscing in a lengthy blog reply. Hope it helped someone. : )
Pingback: My Python Journey, Part 3 | ITauditSecurity
Pingback: My Python Journey, Part 4 | ITauditSecurity
Pingback: Abandon ACL and Others? | ITauditSecurity